2024-05-09T00:24:10.836 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-09T00:24:10.837 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7698888 --owner scheduled_pdonnell@teuthology --name pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/suites/iogen}} --verbose -- /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7698888/orig.config.yaml 2024-05-09T00:24:10.901 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-09T00:24:11.019 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7698888 branch: wip-pdonnell-testing-20240508.183908-debug description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/suites/iogen}} email: pdonnell@redhat.com first_in_suite: false job_id: '7698888' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240508.183908-debug ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: c9da4049f638c0f84bdd9b4e33c42e86046ac554 subvols: create: 2 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported dry_run: true install: ceph: flavor: default sha1: c9da4049f638c0f84bdd9b4e33c42e86046ac554 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - wsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240508.183908-debug sha1: c9da4049f638c0f84bdd9b4e33c42e86046ac554 owner: scheduled_pdonnell@teuthology priority: 90 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 1722 sha1: c9da4049f638c0f84bdd9b4e33c42e86046ac554 sleep_before_teardown: 0 subset: 67/512 suite: fs suite_branch: wip-pdonnell-testing-20240508.183908-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_c9da4049f638c0f84bdd9b4e33c42e86046ac554/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: c9da4049f638c0f84bdd9b4e33c42e86046ac554 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi005.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBA4Hv9slO+MjNDhUYt2EwiomGMBiJHmDpPun2+FpDW89jU+j08x3tiRf6Z0AEh98LPxX9n24S3XvqJC/zlX33MM= smithi131.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOsjKm3PZ9Klq0XIIPYVde+hyKwTiazJE2JR/2Bmm8ryk58ZbvZ84cp3CXHlU8uctTCGyuGGVny5zR80Ba+8G+s= smithi157.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBemtDUBbgz4nYjiOeKNVoETJxKL8NwQeov5VR38uFbJ0YLisq23SxsTP6AvE4jQHTQ19FL6UmoAUn2dDPhi1rA= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/iogen.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-08_22:06:20 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-09T00:24:11.020 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_c9da4049f638c0f84bdd9b4e33c42e86046ac554/qa; will attempt to use it 2024-05-09T00:24:11.020 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_c9da4049f638c0f84bdd9b4e33c42e86046ac554/qa/tasks 2024-05-09T00:24:11.020 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-09T00:24:11.022 INFO:teuthology.task.internal:Checking packages... 2024-05-09T00:24:11.046 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'c9da4049f638c0f84bdd9b4e33c42e86046ac554' 2024-05-09T00:24:11.047 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-09T00:24:11.047 INFO:teuthology.packaging:ref: None 2024-05-09T00:24:11.047 INFO:teuthology.packaging:tag: None 2024-05-09T00:24:11.047 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240508.183908-debug 2024-05-09T00:24:11.047 INFO:teuthology.packaging:sha1: c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T00:24:11.047 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240508.183908-debug 2024-05-09T00:24:11.260 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3594.gc9da4049 2024-05-09T00:24:11.261 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-09T00:24:11.275 INFO:teuthology.task.internal:no buildpackages task found 2024-05-09T00:24:11.275 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-09T00:24:11.314 INFO:teuthology.task.internal:Saving configuration 2024-05-09T00:24:11.335 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-09T00:24:11.362 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-09T00:24:11.389 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi005.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7698888', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-09 00:18:33.124130', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBA4Hv9slO+MjNDhUYt2EwiomGMBiJHmDpPun2+FpDW89jU+j08x3tiRf6Z0AEh98LPxX9n24S3XvqJC/zlX33MM='} 2024-05-09T00:24:11.413 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi131.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7698888', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-09 00:18:33.121001', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOsjKm3PZ9Klq0XIIPYVde+hyKwTiazJE2JR/2Bmm8ryk58ZbvZ84cp3CXHlU8uctTCGyuGGVny5zR80Ba+8G+s='} 2024-05-09T00:24:11.437 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi157.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7698888', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-09 00:18:33.122676', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBemtDUBbgz4nYjiOeKNVoETJxKL8NwQeov5VR38uFbJ0YLisq23SxsTP6AvE4jQHTQ19FL6UmoAUn2dDPhi1rA='} 2024-05-09T00:24:11.437 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-09T00:24:11.442 INFO:teuthology.task.internal:roles: ubuntu@smithi005.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-05-09T00:24:11.442 INFO:teuthology.task.internal:roles: ubuntu@smithi131.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-05-09T00:24:11.442 INFO:teuthology.task.internal:roles: ubuntu@smithi157.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-05-09T00:24:11.442 INFO:teuthology.run_tasks:Running task console_log... 2024-05-09T00:24:11.577 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f2cb072c0d0>, signals=[15]) 2024-05-09T00:24:11.577 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-09T00:24:11.584 INFO:teuthology.task.internal:Opening connections... 2024-05-09T00:24:11.584 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi005.front.sepia.ceph.com 2024-05-09T00:24:11.587 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T00:24:11.669 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi131.front.sepia.ceph.com 2024-05-09T00:24:11.670 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T00:24:11.745 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi157.front.sepia.ceph.com 2024-05-09T00:24:11.745 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T00:24:11.822 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-09T00:24:11.832 DEBUG:teuthology.orchestra.run.smithi005:> uname -m 2024-05-09T00:24:11.851 INFO:teuthology.orchestra.run.smithi005.stdout:x86_64 2024-05-09T00:24:11.851 DEBUG:teuthology.orchestra.run.smithi005:> cat /etc/os-release 2024-05-09T00:24:11.909 INFO:teuthology.orchestra.run.smithi005.stdout:NAME="CentOS Stream" 2024-05-09T00:24:11.910 INFO:teuthology.orchestra.run.smithi005.stdout:VERSION="9" 2024-05-09T00:24:11.910 INFO:teuthology.orchestra.run.smithi005.stdout:ID="centos" 2024-05-09T00:24:11.910 INFO:teuthology.orchestra.run.smithi005.stdout:ID_LIKE="rhel fedora" 2024-05-09T00:24:11.910 INFO:teuthology.orchestra.run.smithi005.stdout:VERSION_ID="9" 2024-05-09T00:24:11.910 INFO:teuthology.orchestra.run.smithi005.stdout:PLATFORM_ID="platform:el9" 2024-05-09T00:24:11.910 INFO:teuthology.orchestra.run.smithi005.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-09T00:24:11.910 INFO:teuthology.orchestra.run.smithi005.stdout:ANSI_COLOR="0;31" 2024-05-09T00:24:11.910 INFO:teuthology.orchestra.run.smithi005.stdout:LOGO="fedora-logo-icon" 2024-05-09T00:24:11.910 INFO:teuthology.orchestra.run.smithi005.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-09T00:24:11.910 INFO:teuthology.orchestra.run.smithi005.stdout:HOME_URL="https://centos.org/" 2024-05-09T00:24:11.910 INFO:teuthology.orchestra.run.smithi005.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-09T00:24:11.911 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-09T00:24:11.911 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-09T00:24:11.911 INFO:teuthology.lock.ops:Updating smithi005.front.sepia.ceph.com on lock server 2024-05-09T00:24:11.938 DEBUG:teuthology.orchestra.run.smithi131:> uname -m 2024-05-09T00:24:11.956 INFO:teuthology.orchestra.run.smithi131.stdout:x86_64 2024-05-09T00:24:11.956 DEBUG:teuthology.orchestra.run.smithi131:> cat /etc/os-release 2024-05-09T00:24:12.013 INFO:teuthology.orchestra.run.smithi131.stdout:NAME="CentOS Stream" 2024-05-09T00:24:12.013 INFO:teuthology.orchestra.run.smithi131.stdout:VERSION="9" 2024-05-09T00:24:12.013 INFO:teuthology.orchestra.run.smithi131.stdout:ID="centos" 2024-05-09T00:24:12.013 INFO:teuthology.orchestra.run.smithi131.stdout:ID_LIKE="rhel fedora" 2024-05-09T00:24:12.013 INFO:teuthology.orchestra.run.smithi131.stdout:VERSION_ID="9" 2024-05-09T00:24:12.014 INFO:teuthology.orchestra.run.smithi131.stdout:PLATFORM_ID="platform:el9" 2024-05-09T00:24:12.014 INFO:teuthology.orchestra.run.smithi131.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-09T00:24:12.014 INFO:teuthology.orchestra.run.smithi131.stdout:ANSI_COLOR="0;31" 2024-05-09T00:24:12.014 INFO:teuthology.orchestra.run.smithi131.stdout:LOGO="fedora-logo-icon" 2024-05-09T00:24:12.014 INFO:teuthology.orchestra.run.smithi131.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-09T00:24:12.014 INFO:teuthology.orchestra.run.smithi131.stdout:HOME_URL="https://centos.org/" 2024-05-09T00:24:12.014 INFO:teuthology.orchestra.run.smithi131.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-09T00:24:12.014 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-09T00:24:12.014 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-09T00:24:12.014 INFO:teuthology.lock.ops:Updating smithi131.front.sepia.ceph.com on lock server 2024-05-09T00:24:12.038 DEBUG:teuthology.orchestra.run.smithi157:> uname -m 2024-05-09T00:24:12.056 INFO:teuthology.orchestra.run.smithi157.stdout:x86_64 2024-05-09T00:24:12.056 DEBUG:teuthology.orchestra.run.smithi157:> cat /etc/os-release 2024-05-09T00:24:12.113 INFO:teuthology.orchestra.run.smithi157.stdout:NAME="CentOS Stream" 2024-05-09T00:24:12.113 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION="9" 2024-05-09T00:24:12.113 INFO:teuthology.orchestra.run.smithi157.stdout:ID="centos" 2024-05-09T00:24:12.113 INFO:teuthology.orchestra.run.smithi157.stdout:ID_LIKE="rhel fedora" 2024-05-09T00:24:12.113 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION_ID="9" 2024-05-09T00:24:12.113 INFO:teuthology.orchestra.run.smithi157.stdout:PLATFORM_ID="platform:el9" 2024-05-09T00:24:12.113 INFO:teuthology.orchestra.run.smithi157.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-09T00:24:12.113 INFO:teuthology.orchestra.run.smithi157.stdout:ANSI_COLOR="0;31" 2024-05-09T00:24:12.113 INFO:teuthology.orchestra.run.smithi157.stdout:LOGO="fedora-logo-icon" 2024-05-09T00:24:12.114 INFO:teuthology.orchestra.run.smithi157.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-09T00:24:12.114 INFO:teuthology.orchestra.run.smithi157.stdout:HOME_URL="https://centos.org/" 2024-05-09T00:24:12.114 INFO:teuthology.orchestra.run.smithi157.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-09T00:24:12.114 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-09T00:24:12.114 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-09T00:24:12.114 INFO:teuthology.lock.ops:Updating smithi157.front.sepia.ceph.com on lock server 2024-05-09T00:24:12.138 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-09T00:24:12.151 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-09T00:24:12.171 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-09T00:24:12.171 DEBUG:teuthology.orchestra.run.smithi005:> test '!' -e /home/ubuntu/cephtest 2024-05-09T00:24:12.174 DEBUG:teuthology.orchestra.run.smithi131:> test '!' -e /home/ubuntu/cephtest 2024-05-09T00:24:12.176 DEBUG:teuthology.orchestra.run.smithi157:> test '!' -e /home/ubuntu/cephtest 2024-05-09T00:24:12.192 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-09T00:24:12.198 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-09T00:24:12.198 DEBUG:teuthology.orchestra.run.smithi005:> test -z $(ls -A /var/lib/ceph) 2024-05-09T00:24:12.233 DEBUG:teuthology.orchestra.run.smithi131:> test -z $(ls -A /var/lib/ceph) 2024-05-09T00:24:12.235 DEBUG:teuthology.orchestra.run.smithi157:> test -z $(ls -A /var/lib/ceph) 2024-05-09T00:24:12.263 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-09T00:24:12.352 INFO:teuthology.run_tasks:Running task kernel... 2024-05-09T00:24:12.366 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-05-09T00:24:12.366 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-05-09T00:24:12.366 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-05-09T00:24:12.367 DEBUG:teuthology.orchestra.run.smithi005:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-09T00:24:12.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T00:24:12.384 DEBUG:teuthology.orchestra.run.smithi005:> uname -r 2024-05-09T00:24:12.443 INFO:teuthology.orchestra.run.smithi005.stdout:5.14.0-437.el9.x86_64 2024-05-09T00:24:12.443 INFO:teuthology.task.kernel:Running kernel on smithi005: 5.14.0-437.el9.x86_64 2024-05-09T00:24:12.443 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum install -y kernel 2024-05-09T00:24:16.812 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - BaseOS 2.9 MB/s | 8.1 MB 00:02 2024-05-09T00:24:18.613 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - AppStream 29 MB/s | 19 MB 00:00 2024-05-09T00:24:22.764 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.2 MB 00:00 2024-05-09T00:24:24.481 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 16 kB 00:00 2024-05-09T00:24:38.231 INFO:teuthology.orchestra.run.smithi005.stdout:Extra Packages for Enterprise Linux 1.6 MB/s | 21 MB 00:13 2024-05-09T00:24:43.498 INFO:teuthology.orchestra.run.smithi005.stdout:lab-extras 46 kB/s | 1.7 kB 00:00 2024-05-09T00:24:45.562 INFO:teuthology.orchestra.run.smithi005.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-09T00:24:45.562 INFO:teuthology.orchestra.run.smithi005.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-09T00:24:45.665 INFO:teuthology.orchestra.run.smithi005.stdout:Dependencies resolved. 2024-05-09T00:24:45.671 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-05-09T00:24:45.671 INFO:teuthology.orchestra.run.smithi005.stdout: Package Architecture Version Repository Size 2024-05-09T00:24:45.671 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-05-09T00:24:45.671 INFO:teuthology.orchestra.run.smithi005.stdout:Installing: 2024-05-09T00:24:45.671 INFO:teuthology.orchestra.run.smithi005.stdout: kernel x86_64 5.14.0-444.el9 baseos 484 k 2024-05-09T00:24:45.671 INFO:teuthology.orchestra.run.smithi005.stdout:Installing dependencies: 2024-05-09T00:24:45.671 INFO:teuthology.orchestra.run.smithi005.stdout: kernel-core x86_64 5.14.0-444.el9 baseos 15 M 2024-05-09T00:24:45.672 INFO:teuthology.orchestra.run.smithi005.stdout: kernel-modules x86_64 5.14.0-444.el9 baseos 34 M 2024-05-09T00:24:45.672 INFO:teuthology.orchestra.run.smithi005.stdout: kernel-modules-core x86_64 5.14.0-444.el9 baseos 28 M 2024-05-09T00:24:45.672 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:24:45.672 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction Summary 2024-05-09T00:24:45.672 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-05-09T00:24:45.672 INFO:teuthology.orchestra.run.smithi005.stdout:Install 4 Packages 2024-05-09T00:24:45.672 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:24:45.673 INFO:teuthology.orchestra.run.smithi005.stdout:Total download size: 78 M 2024-05-09T00:24:45.673 INFO:teuthology.orchestra.run.smithi005.stdout:Installed size: 123 M 2024-05-09T00:24:45.673 INFO:teuthology.orchestra.run.smithi005.stdout:Downloading Packages: 2024-05-09T00:24:46.024 INFO:teuthology.orchestra.run.smithi005.stdout:(1/4): kernel-5.14.0-444.el9.x86_64.rpm 1.5 MB/s | 484 kB 00:00 2024-05-09T00:24:46.582 INFO:teuthology.orchestra.run.smithi005.stdout:(2/4): kernel-core-5.14.0-444.el9.x86_64.rpm 18 MB/s | 15 MB 00:00 2024-05-09T00:24:47.191 INFO:teuthology.orchestra.run.smithi005.stdout:(3/4): kernel-modules-5.14.0-444.el9.x86_64.rpm 23 MB/s | 34 MB 00:01 2024-05-09T00:24:47.458 INFO:teuthology.orchestra.run.smithi005.stdout:(4/4): kernel-modules-core-5.14.0-444.el9.x86_6 20 MB/s | 28 MB 00:01 2024-05-09T00:24:47.458 INFO:teuthology.orchestra.run.smithi005.stdout:-------------------------------------------------------------------------------- 2024-05-09T00:24:47.458 INFO:teuthology.orchestra.run.smithi005.stdout:Total 44 MB/s | 78 MB 00:01 2024-05-09T00:24:48.165 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction check 2024-05-09T00:24:48.572 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction check succeeded. 2024-05-09T00:24:48.572 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction test 2024-05-09T00:24:49.528 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction test succeeded. 2024-05-09T00:24:49.528 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction 2024-05-09T00:24:52.061 INFO:teuthology.orchestra.run.smithi005.stdout: Preparing : 1/1 2024-05-09T00:24:52.749 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : kernel-modules-core-5.14.0-444.el9.x86_64 1/4 2024-05-09T00:24:52.833 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T00:24:55.647 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T00:24:55.785 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T00:25:00.855 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T00:25:00.896 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : kernel-5.14.0-444.el9.x86_64 4/4 2024-05-09T00:25:05.989 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T00:25:37.074 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T00:25:37.113 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 4/4 2024-05-09T00:25:37.502 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: kernel-5.14.0-444.el9.x86_64 4/4 2024-05-09T00:25:37.502 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : kernel-5.14.0-444.el9.x86_64 1/4 2024-05-09T00:25:37.502 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T00:25:37.502 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T00:25:38.322 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T00:25:38.322 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:25:38.323 INFO:teuthology.orchestra.run.smithi005.stdout:Installed: 2024-05-09T00:25:38.323 INFO:teuthology.orchestra.run.smithi005.stdout: kernel-5.14.0-444.el9.x86_64 2024-05-09T00:25:38.323 INFO:teuthology.orchestra.run.smithi005.stdout: kernel-core-5.14.0-444.el9.x86_64 2024-05-09T00:25:38.323 INFO:teuthology.orchestra.run.smithi005.stdout: kernel-modules-5.14.0-444.el9.x86_64 2024-05-09T00:25:38.323 INFO:teuthology.orchestra.run.smithi005.stdout: kernel-modules-core-5.14.0-444.el9.x86_64 2024-05-09T00:25:38.323 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:25:38.323 INFO:teuthology.orchestra.run.smithi005.stdout:Complete! 2024-05-09T00:25:38.701 DEBUG:teuthology.orchestra.run.smithi005:> rpm -q kernel | sort -rV | head -n 1 2024-05-09T00:25:38.787 INFO:teuthology.orchestra.run.smithi005.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-09T00:25:38.789 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-09T00:25:38.789 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-444.el9.x86_64 2024-05-09T00:25:38.789 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-09T00:25:38.789 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-09T00:25:38.789 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-05-09T00:25:38.790 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi005.front.sepia.ceph.com, path=None, version=distro) 2024-05-09T00:25:38.790 DEBUG:teuthology.orchestra.run.smithi005:> rpm -q kernel | sort -rV | head -n 1 2024-05-09T00:25:38.845 INFO:teuthology.orchestra.run.smithi005.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-09T00:25:38.846 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-09T00:25:38.846 DEBUG:teuthology.orchestra.run.smithi005:> sudo rpm -qi grub2-tools 2024-05-09T00:25:38.878 INFO:teuthology.orchestra.run.smithi005.stdout:Name : grub2-tools 2024-05-09T00:25:38.878 INFO:teuthology.orchestra.run.smithi005.stdout:Epoch : 1 2024-05-09T00:25:38.879 INFO:teuthology.orchestra.run.smithi005.stdout:Version : 2.06 2024-05-09T00:25:38.879 INFO:teuthology.orchestra.run.smithi005.stdout:Release : 68.el9 2024-05-09T00:25:38.879 INFO:teuthology.orchestra.run.smithi005.stdout:Architecture: x86_64 2024-05-09T00:25:38.879 INFO:teuthology.orchestra.run.smithi005.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-09T00:25:38.879 INFO:teuthology.orchestra.run.smithi005.stdout:Group : Unspecified 2024-05-09T00:25:38.879 INFO:teuthology.orchestra.run.smithi005.stdout:Size : 8264254 2024-05-09T00:25:38.879 INFO:teuthology.orchestra.run.smithi005.stdout:License : GPLv3+ 2024-05-09T00:25:38.879 INFO:teuthology.orchestra.run.smithi005.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-09T00:25:38.879 INFO:teuthology.orchestra.run.smithi005.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-09T00:25:38.879 INFO:teuthology.orchestra.run.smithi005.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-09T00:25:38.879 INFO:teuthology.orchestra.run.smithi005.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-09T00:25:38.879 INFO:teuthology.orchestra.run.smithi005.stdout:Packager : builder@centos.org 2024-05-09T00:25:38.879 INFO:teuthology.orchestra.run.smithi005.stdout:Vendor : CentOS 2024-05-09T00:25:38.880 INFO:teuthology.orchestra.run.smithi005.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-09T00:25:38.880 INFO:teuthology.orchestra.run.smithi005.stdout:Summary : Support tools for GRUB. 2024-05-09T00:25:38.880 INFO:teuthology.orchestra.run.smithi005.stdout:Description : 2024-05-09T00:25:38.880 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:25:38.880 INFO:teuthology.orchestra.run.smithi005.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-09T00:25:38.880 INFO:teuthology.orchestra.run.smithi005.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-09T00:25:38.880 INFO:teuthology.orchestra.run.smithi005.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-09T00:25:38.880 INFO:teuthology.orchestra.run.smithi005.stdout:hardware devices. 2024-05-09T00:25:38.880 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:25:38.880 INFO:teuthology.orchestra.run.smithi005.stdout:This subpackage provides tools for support of all platforms. 2024-05-09T00:25:38.881 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-09T00:25:38.881 INFO:teuthology.task.kernel:Updating grub on smithi005 to boot 5.14.0-444.el9.x86_64 2024-05-09T00:25:38.881 DEBUG:teuthology.orchestra.run.smithi005:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-09T00:25:39.468 INFO:teuthology.orchestra.run.smithi005.stderr:Generating grub configuration file ... 2024-05-09T00:25:40.891 INFO:teuthology.orchestra.run.smithi005.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-09T00:25:40.922 INFO:teuthology.orchestra.run.smithi005.stderr:done 2024-05-09T00:25:40.934 DEBUG:teuthology.orchestra.run.smithi005:> mktemp 2024-05-09T00:25:40.950 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/tmp.ImDPjtpN9k 2024-05-09T00:25:40.950 DEBUG:teuthology.orchestra.run.smithi005:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.ImDPjtpN9k 2024-05-09T00:25:41.021 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 0666 /tmp/tmp.ImDPjtpN9k 2024-05-09T00:25:41.160 DEBUG:teuthology.orchestra.remote:smithi005:/tmp/tmp.ImDPjtpN9k is 6KB 2024-05-09T00:25:41.209 DEBUG:teuthology.orchestra.run.smithi005:> rm -fr /tmp/tmp.ImDPjtpN9k 2024-05-09T00:25:41.224 DEBUG:teuthology.orchestra.run.smithi005:> sudo /bin/ls /boot/loader/entries || true 2024-05-09T00:25:41.286 INFO:teuthology.orchestra.run.smithi005.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-09T00:25:41.286 INFO:teuthology.orchestra.run.smithi005.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-09T00:25:41.286 INFO:teuthology.orchestra.run.smithi005.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-09T00:25:41.286 INFO:teuthology.orchestra.run.smithi005.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64.conf 2024-05-09T00:25:41.287 DEBUG:teuthology.orchestra.run.smithi005:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64 2024-05-09T00:25:41.441 DEBUG:teuthology.orchestra.run.smithi005:> sudo shutdown -r now 2024-05-09T00:26:11.443 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-05-09T00:26:11.444 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-05-09T00:26:11.444 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T00:27:11.447 DEBUG:teuthology.orchestra.remote:timed out 2024-05-09T00:27:20.448 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-05-09T00:27:20.449 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T00:27:20.737 DEBUG:teuthology.orchestra.run.smithi005:> true 2024-05-09T00:27:21.155 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-05-09T00:27:21.155 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-444.el9.x86_64"... 2024-05-09T00:27:21.156 DEBUG:teuthology.orchestra.run.smithi005:> uname -r 2024-05-09T00:27:21.171 INFO:teuthology.orchestra.run.smithi005.stdout:5.14.0-444.el9.x86_64 2024-05-09T00:27:21.171 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-444.el9.x86_64 vs 5.14.0-444.el9.x86_64 2024-05-09T00:27:21.172 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-09T00:27:21.172 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-09T00:27:22.173 DEBUG:teuthology.parallel:result is None 2024-05-09T00:27:22.173 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-09T00:27:22.184 INFO:teuthology.task.internal:Creating test directory... 2024-05-09T00:27:22.184 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-09T00:27:22.186 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-09T00:27:22.189 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-09T00:27:22.207 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-09T00:27:22.218 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-09T00:27:22.262 INFO:teuthology.task.internal:Creating archive directory... 2024-05-09T00:27:22.262 DEBUG:teuthology.orchestra.run.smithi005:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-09T00:27:22.264 DEBUG:teuthology.orchestra.run.smithi131:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-09T00:27:22.267 DEBUG:teuthology.orchestra.run.smithi157:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-09T00:27:22.374 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-09T00:27:22.380 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-09T00:27:22.380 DEBUG:teuthology.orchestra.run.smithi131:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-09T00:27:22.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T00:27:22.397 DEBUG:teuthology.orchestra.run.smithi157:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-09T00:27:22.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T00:27:22.413 DEBUG:teuthology.orchestra.run.smithi005:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-09T00:27:22.415 DEBUG:teuthology.orchestra.run.smithi131:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-09T00:27:22.439 DEBUG:teuthology.orchestra.run.smithi157:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-09T00:27:22.478 INFO:teuthology.orchestra.run.smithi131.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T00:27:22.483 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T00:27:22.491 INFO:teuthology.orchestra.run.smithi131.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T00:27:22.501 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T00:27:22.503 INFO:teuthology.orchestra.run.smithi005.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T00:27:22.562 INFO:teuthology.orchestra.run.smithi005.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T00:27:22.564 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-09T00:27:22.576 INFO:teuthology.task.internal:Configuring sudo... 2024-05-09T00:27:22.576 DEBUG:teuthology.orchestra.run.smithi005:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-09T00:27:22.578 DEBUG:teuthology.orchestra.run.smithi131:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-09T00:27:22.580 DEBUG:teuthology.orchestra.run.smithi157:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-09T00:27:22.607 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-09T00:27:22.630 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-09T00:27:22.631 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-09T00:27:22.650 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-09T00:27:22.652 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-09T00:27:22.669 DEBUG:teuthology.orchestra.run.smithi005:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T00:27:22.752 DEBUG:teuthology.orchestra.run.smithi005:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T00:27:22.798 DEBUG:teuthology.orchestra.run.smithi005:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T00:27:22.884 DEBUG:teuthology.orchestra.run.smithi005:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T00:27:22.929 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-09T00:27:22.930 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-09T00:27:22.999 DEBUG:teuthology.orchestra.run.smithi131:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T00:27:23.043 DEBUG:teuthology.orchestra.run.smithi131:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T00:27:23.091 DEBUG:teuthology.orchestra.run.smithi131:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T00:27:23.173 DEBUG:teuthology.orchestra.run.smithi131:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T00:27:23.213 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-09T00:27:23.214 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-09T00:27:23.278 DEBUG:teuthology.orchestra.run.smithi157:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T00:27:23.325 DEBUG:teuthology.orchestra.run.smithi157:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T00:27:23.382 DEBUG:teuthology.orchestra.run.smithi157:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T00:27:23.461 DEBUG:teuthology.orchestra.run.smithi157:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T00:27:23.507 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-05-09T00:27:23.508 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-09T00:27:23.573 DEBUG:teuthology.orchestra.run.smithi005:> sudo service rsyslog restart 2024-05-09T00:27:23.576 DEBUG:teuthology.orchestra.run.smithi131:> sudo service rsyslog restart 2024-05-09T00:27:23.578 DEBUG:teuthology.orchestra.run.smithi157:> sudo service rsyslog restart 2024-05-09T00:27:23.611 INFO:teuthology.orchestra.run.smithi131.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T00:27:23.628 INFO:teuthology.orchestra.run.smithi005.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T00:27:23.655 INFO:teuthology.orchestra.run.smithi157.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T00:27:24.077 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-09T00:27:24.086 INFO:teuthology.task.internal:Starting timer... 2024-05-09T00:27:24.086 INFO:teuthology.run_tasks:Running task pcp... 2024-05-09T00:27:24.144 INFO:teuthology.run_tasks:Running task selinux... 2024-05-09T00:27:24.176 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-05-09T00:27:24.176 DEBUG:teuthology.orchestra.run.smithi005:> sudo service auditd rotate 2024-05-09T00:27:24.249 INFO:teuthology.orchestra.run.smithi005.stdout:Rotating logs: 2024-05-09T00:27:24.251 DEBUG:teuthology.orchestra.run.smithi131:> sudo service auditd rotate 2024-05-09T00:27:24.307 INFO:teuthology.orchestra.run.smithi131.stdout:Rotating logs: 2024-05-09T00:27:24.310 DEBUG:teuthology.orchestra.run.smithi157:> sudo service auditd rotate 2024-05-09T00:27:24.366 INFO:teuthology.orchestra.run.smithi157.stdout:Rotating logs: 2024-05-09T00:27:24.368 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-09T00:27:24.368 DEBUG:teuthology.orchestra.run.smithi005:> /usr/sbin/getenforce 2024-05-09T00:27:24.392 INFO:teuthology.orchestra.run.smithi005.stdout:Permissive 2024-05-09T00:27:24.393 DEBUG:teuthology.orchestra.run.smithi131:> /usr/sbin/getenforce 2024-05-09T00:27:24.412 INFO:teuthology.orchestra.run.smithi131.stdout:Permissive 2024-05-09T00:27:24.413 DEBUG:teuthology.orchestra.run.smithi157:> /usr/sbin/getenforce 2024-05-09T00:27:24.435 INFO:teuthology.orchestra.run.smithi157.stdout:Permissive 2024-05-09T00:27:24.435 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi005.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi131.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi157.front.sepia.ceph.com': 'permissive'} 2024-05-09T00:27:24.436 DEBUG:teuthology.orchestra.run.smithi005:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T00:27:24.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T00:27:24.463 DEBUG:teuthology.orchestra.run.smithi131:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T00:27:24.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T00:27:24.489 DEBUG:teuthology.orchestra.run.smithi157:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T00:27:24.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T00:27:24.515 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-09T00:27:24.515 DEBUG:teuthology.orchestra.run.smithi005:> sudo /usr/sbin/setenforce permissive 2024-05-09T00:27:24.553 DEBUG:teuthology.orchestra.run.smithi131:> sudo /usr/sbin/setenforce permissive 2024-05-09T00:27:24.577 DEBUG:teuthology.orchestra.run.smithi157:> sudo /usr/sbin/setenforce permissive 2024-05-09T00:27:24.602 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-09T00:27:24.611 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-05-09T00:27:24.617 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-05-09T00:27:24.746 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-09T00:27:24.783 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-09T00:27:24.796 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi005.front.sepia.ceph.com,smithi131.front.sepia.ceph.com,smithi157.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-09T00:31:40.985 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi005.front.sepia.ceph.com'), Remote(name='ubuntu@smithi131.front.sepia.ceph.com'), Remote(name='ubuntu@smithi157.front.sepia.ceph.com')] 2024-05-09T00:31:40.986 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-05-09T00:31:40.988 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T00:31:41.069 DEBUG:teuthology.orchestra.run.smithi005:> true 2024-05-09T00:31:41.151 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-05-09T00:31:41.151 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-05-09T00:31:41.152 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T00:31:41.225 DEBUG:teuthology.orchestra.run.smithi131:> true 2024-05-09T00:31:41.309 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-05-09T00:31:41.309 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-05-09T00:31:41.310 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T00:31:41.390 DEBUG:teuthology.orchestra.run.smithi157:> true 2024-05-09T00:31:41.465 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-05-09T00:31:41.466 INFO:teuthology.run_tasks:Running task clock... 2024-05-09T00:31:41.477 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-09T00:31:41.478 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-09T00:31:41.478 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T00:31:41.481 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-09T00:31:41.481 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T00:31:41.483 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-09T00:31:41.483 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T00:31:41.509 INFO:teuthology.orchestra.run.smithi131.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-09T00:31:41.514 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-09T00:31:41.523 INFO:teuthology.orchestra.run.smithi131.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-09T00:31:41.530 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-09T00:31:41.534 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-09T00:31:41.546 INFO:teuthology.orchestra.run.smithi131.stderr:sudo: ntpd: command not found 2024-05-09T00:31:41.547 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-09T00:31:41.556 INFO:teuthology.orchestra.run.smithi131.stdout:506 Cannot talk to daemon 2024-05-09T00:31:41.557 INFO:teuthology.orchestra.run.smithi005.stderr:sudo: ntpd: command not found 2024-05-09T00:31:41.569 INFO:teuthology.orchestra.run.smithi131.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-09T00:31:41.570 INFO:teuthology.orchestra.run.smithi005.stdout:506 Cannot talk to daemon 2024-05-09T00:31:41.570 INFO:teuthology.orchestra.run.smithi157.stderr:sudo: ntpd: command not found 2024-05-09T00:31:41.581 INFO:teuthology.orchestra.run.smithi157.stdout:506 Cannot talk to daemon 2024-05-09T00:31:41.582 INFO:teuthology.orchestra.run.smithi131.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-09T00:31:41.585 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-09T00:31:41.594 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-09T00:31:41.600 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-09T00:31:41.606 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-09T00:31:41.635 INFO:teuthology.orchestra.run.smithi131.stderr:bash: line 1: ntpq: command not found 2024-05-09T00:31:41.639 INFO:teuthology.orchestra.run.smithi131.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T00:31:41.639 INFO:teuthology.orchestra.run.smithi131.stdout:=============================================================================== 2024-05-09T00:31:41.639 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T00:31:41.640 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T00:31:41.640 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T00:31:41.640 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T00:31:41.654 INFO:teuthology.orchestra.run.smithi005.stderr:bash: line 1: ntpq: command not found 2024-05-09T00:31:41.658 INFO:teuthology.orchestra.run.smithi005.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T00:31:41.658 INFO:teuthology.orchestra.run.smithi005.stdout:=============================================================================== 2024-05-09T00:31:41.658 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T00:31:41.658 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T00:31:41.658 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T00:31:41.658 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T00:31:41.662 INFO:teuthology.orchestra.run.smithi157.stderr:bash: line 1: ntpq: command not found 2024-05-09T00:31:41.666 INFO:teuthology.orchestra.run.smithi157.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T00:31:41.666 INFO:teuthology.orchestra.run.smithi157.stdout:=============================================================================== 2024-05-09T00:31:41.666 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T00:31:41.666 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T00:31:41.667 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T00:31:41.667 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T00:31:41.667 INFO:teuthology.run_tasks:Running task install... 2024-05-09T00:31:41.675 DEBUG:teuthology.task.install:project ceph 2024-05-09T00:31:41.675 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'c9da4049f638c0f84bdd9b4e33c42e86046ac554'}} 2024-05-09T00:31:41.675 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'c9da4049f638c0f84bdd9b4e33c42e86046ac554'} 2024-05-09T00:31:41.675 INFO:teuthology.task.install:Using flavor: default 2024-05-09T00:31:41.683 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-09T00:31:41.683 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-09T00:31:41.684 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'c9da4049f638c0f84bdd9b4e33c42e86046ac554', 'tag': None, 'wait_for_package': False} 2024-05-09T00:31:41.684 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T00:31:41.686 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'c9da4049f638c0f84bdd9b4e33c42e86046ac554', 'tag': None, 'wait_for_package': False} 2024-05-09T00:31:41.686 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T00:31:41.687 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'c9da4049f638c0f84bdd9b4e33c42e86046ac554', 'tag': None, 'wait_for_package': False} 2024-05-09T00:31:41.687 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T00:31:41.829 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/ 2024-05-09T00:31:41.830 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3594.gc9da4049 2024-05-09T00:31:41.841 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/ 2024-05-09T00:31:41.842 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3594.gc9da4049 2024-05-09T00:31:41.852 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/ 2024-05-09T00:31:41.853 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3594.gc9da4049 2024-05-09T00:31:41.945 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-09T00:31:41.945 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-09T00:31:41.946 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-09T00:31:41.953 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-09T00:31:41.953 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-09T00:31:41.953 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-09T00:31:41.963 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-09T00:31:41.963 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-05-09T00:31:41.963 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-09T00:31:41.973 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-09T00:31:41.973 DEBUG:teuthology.orchestra.run.smithi005:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/c9da4049f638c0f84bdd9b4e33c42e86046ac554/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-09T00:31:41.977 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-09T00:31:41.978 DEBUG:teuthology.orchestra.run.smithi131:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/c9da4049f638c0f84bdd9b4e33c42e86046ac554/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-09T00:31:41.990 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-09T00:31:41.990 DEBUG:teuthology.orchestra.run.smithi157:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/c9da4049f638c0f84bdd9b4e33c42e86046ac554/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-09T00:31:42.042 DEBUG:teuthology.orchestra.run.smithi131:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-09T00:31:42.044 DEBUG:teuthology.orchestra.run.smithi005:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-09T00:31:42.055 DEBUG:teuthology.orchestra.run.smithi157:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-09T00:31:42.117 DEBUG:teuthology.orchestra.run.smithi131:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-09T00:31:42.122 DEBUG:teuthology.orchestra.run.smithi005:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-09T00:31:42.128 DEBUG:teuthology.orchestra.run.smithi157:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-09T00:31:42.181 INFO:teuthology.orchestra.run.smithi131.stdout:check_obsoletes = 1 2024-05-09T00:31:42.182 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum clean all 2024-05-09T00:31:42.188 INFO:teuthology.orchestra.run.smithi005.stdout:check_obsoletes = 1 2024-05-09T00:31:42.191 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum clean all 2024-05-09T00:31:42.192 INFO:teuthology.orchestra.run.smithi157.stdout:check_obsoletes = 1 2024-05-09T00:31:42.193 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum clean all 2024-05-09T00:31:42.407 INFO:teuthology.orchestra.run.smithi131.stdout:54 files removed 2024-05-09T00:31:42.421 INFO:teuthology.orchestra.run.smithi157.stdout:54 files removed 2024-05-09T00:31:42.435 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-09T00:31:42.441 INFO:teuthology.orchestra.run.smithi005.stdout:54 files removed 2024-05-09T00:31:42.449 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-09T00:31:42.468 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-09T00:31:43.064 INFO:teuthology.orchestra.run.smithi131.stdout:ceph packages for x86_64 228 kB/s | 84 kB 00:00 2024-05-09T00:31:43.078 INFO:teuthology.orchestra.run.smithi005.stdout:ceph packages for x86_64 260 kB/s | 84 kB 00:00 2024-05-09T00:31:43.094 INFO:teuthology.orchestra.run.smithi157.stdout:ceph packages for x86_64 220 kB/s | 84 kB 00:00 2024-05-09T00:31:43.453 INFO:teuthology.orchestra.run.smithi157.stdout:ceph noarch packages 37 kB/s | 12 kB 00:00 2024-05-09T00:31:43.482 INFO:teuthology.orchestra.run.smithi131.stdout:ceph noarch packages 31 kB/s | 12 kB 00:00 2024-05-09T00:31:43.537 INFO:teuthology.orchestra.run.smithi005.stdout:ceph noarch packages 29 kB/s | 12 kB 00:00 2024-05-09T00:31:43.803 INFO:teuthology.orchestra.run.smithi157.stdout:ceph source packages 6.0 kB/s | 1.9 kB 00:00 2024-05-09T00:31:43.824 INFO:teuthology.orchestra.run.smithi131.stdout:ceph source packages 6.2 kB/s | 1.9 kB 00:00 2024-05-09T00:31:43.979 INFO:teuthology.orchestra.run.smithi005.stdout:ceph source packages 4.8 kB/s | 1.9 kB 00:00 2024-05-09T00:31:45.554 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - BaseOS 4.7 MB/s | 8.1 MB 00:01 2024-05-09T00:31:47.312 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - BaseOS 2.5 MB/s | 8.1 MB 00:03 2024-05-09T00:31:47.330 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - AppStream 29 MB/s | 19 MB 00:00 2024-05-09T00:31:48.099 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - BaseOS 1.9 MB/s | 8.1 MB 00:04 2024-05-09T00:31:50.050 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - AppStream 23 MB/s | 19 MB 00:00 2024-05-09T00:31:53.031 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - CRB 2.9 MB/s | 6.2 MB 00:02 2024-05-09T00:31:54.151 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.2 MB 00:00 2024-05-09T00:31:55.473 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - Extras packages 13 kB/s | 16 kB 00:01 2024-05-09T00:31:56.535 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - Extras packages 13 kB/s | 16 kB 00:01 2024-05-09T00:31:58.645 INFO:teuthology.orchestra.run.smithi131.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-05-09T00:32:03.795 INFO:teuthology.orchestra.run.smithi131.stdout:lab-extras 17 kB/s | 1.7 kB 00:00 2024-05-09T00:32:04.070 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - AppStream 1.2 MB/s | 19 MB 00:15 2024-05-09T00:32:06.080 INFO:teuthology.orchestra.run.smithi131.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T00:32:06.081 INFO:teuthology.orchestra.run.smithi131.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T00:32:06.092 INFO:teuthology.orchestra.run.smithi131.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-09T00:32:06.092 INFO:teuthology.orchestra.run.smithi131.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-09T00:32:06.093 INFO:teuthology.orchestra.run.smithi131.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-09T00:32:06.094 INFO:teuthology.orchestra.run.smithi131.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-09T00:32:06.095 INFO:teuthology.orchestra.run.smithi131.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-09T00:32:06.098 INFO:teuthology.orchestra.run.smithi131.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-09T00:32:06.099 INFO:teuthology.orchestra.run.smithi131.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-09T00:32:06.159 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-05-09T00:32:06.188 INFO:teuthology.orchestra.run.smithi131.stdout:======================================================================================= 2024-05-09T00:32:06.189 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repository Size 2024-05-09T00:32:06.189 INFO:teuthology.orchestra.run.smithi131.stdout:======================================================================================= 2024-05-09T00:32:06.189 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-05-09T00:32:06.189 INFO:teuthology.orchestra.run.smithi131.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-09T00:32:06.189 INFO:teuthology.orchestra.run.smithi131.stdout: ceph x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 6.6 k 2024-05-09T00:32:06.189 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-base x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 5.4 M 2024-05-09T00:32:06.189 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-fuse x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 863 k 2024-05-09T00:32:06.189 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 147 k 2024-05-09T00:32:06.189 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 1.5 M 2024-05-09T00:32:06.189 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 142 k 2024-05-09T00:32:06.189 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 3.5 M 2024-05-09T00:32:06.189 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 7.4 M 2024-05-09T00:32:06.189 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-rook noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 49 k 2024-05-09T00:32:06.190 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-radosgw x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 10 M 2024-05-09T00:32:06.190 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-test x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 43 M 2024-05-09T00:32:06.190 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-volume noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 273 k 2024-05-09T00:32:06.190 INFO:teuthology.orchestra.run.smithi131.stdout: cephadm noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 764 k 2024-05-09T00:32:06.190 INFO:teuthology.orchestra.run.smithi131.stdout: cephfs-mirror x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 224 k 2024-05-09T00:32:06.190 INFO:teuthology.orchestra.run.smithi131.stdout: cephfs-shell x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 27 k 2024-05-09T00:32:06.190 INFO:teuthology.orchestra.run.smithi131.stdout: cephfs-top noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 18 k 2024-05-09T00:32:06.190 INFO:teuthology.orchestra.run.smithi131.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-09T00:32:06.190 INFO:teuthology.orchestra.run.smithi131.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-09T00:32:06.190 INFO:teuthology.orchestra.run.smithi131.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-09T00:32:06.190 INFO:teuthology.orchestra.run.smithi131.stdout: libcephfs-devel x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 32 k 2024-05-09T00:32:06.190 INFO:teuthology.orchestra.run.smithi131.stdout: libcephfs2 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 756 k 2024-05-09T00:32:06.190 INFO:teuthology.orchestra.run.smithi131.stdout: librados-devel x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 127 k 2024-05-09T00:32:06.190 INFO:teuthology.orchestra.run.smithi131.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-09T00:32:06.191 INFO:teuthology.orchestra.run.smithi131.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-09T00:32:06.191 INFO:teuthology.orchestra.run.smithi131.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-09T00:32:06.191 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-09T00:32:06.191 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-09T00:32:06.191 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-09T00:32:06.191 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cephfs x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 161 k 2024-05-09T00:32:06.191 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rados x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 322 k 2024-05-09T00:32:06.191 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rbd x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 300 k 2024-05-09T00:32:06.191 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rgw x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 99 k 2024-05-09T00:32:06.191 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-fuse x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 86 k 2024-05-09T00:32:06.191 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-mirror x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 3.0 M 2024-05-09T00:32:06.192 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-nbd x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 169 k 2024-05-09T00:32:06.192 INFO:teuthology.orchestra.run.smithi131.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-09T00:32:06.192 INFO:teuthology.orchestra.run.smithi131.stdout:Upgrading: 2024-05-09T00:32:06.192 INFO:teuthology.orchestra.run.smithi131.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-09T00:32:06.192 INFO:teuthology.orchestra.run.smithi131.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-09T00:32:06.192 INFO:teuthology.orchestra.run.smithi131.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-09T00:32:06.192 INFO:teuthology.orchestra.run.smithi131.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-09T00:32:06.192 INFO:teuthology.orchestra.run.smithi131.stdout: librados2 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 3.3 M 2024-05-09T00:32:06.192 INFO:teuthology.orchestra.run.smithi131.stdout: librbd1 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 3.1 M 2024-05-09T00:32:06.192 INFO:teuthology.orchestra.run.smithi131.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-09T00:32:06.192 INFO:teuthology.orchestra.run.smithi131.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-09T00:32:06.192 INFO:teuthology.orchestra.run.smithi131.stdout:Installing dependencies: 2024-05-09T00:32:06.192 INFO:teuthology.orchestra.run.smithi131.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-09T00:32:06.193 INFO:teuthology.orchestra.run.smithi131.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-09T00:32:06.193 INFO:teuthology.orchestra.run.smithi131.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-09T00:32:06.193 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-common x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 20 M 2024-05-09T00:32:06.193 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 34 k 2024-05-09T00:32:06.193 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mds x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 2.3 M 2024-05-09T00:32:06.193 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 274 k 2024-05-09T00:32:06.193 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mon x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 4.9 M 2024-05-09T00:32:06.193 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-osd x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 18 M 2024-05-09T00:32:06.193 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 16 k 2024-05-09T00:32:06.193 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-selinux x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 25 k 2024-05-09T00:32:06.193 INFO:teuthology.orchestra.run.smithi131.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-09T00:32:06.193 INFO:teuthology.orchestra.run.smithi131.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-09T00:32:06.193 INFO:teuthology.orchestra.run.smithi131.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-09T00:32:06.193 INFO:teuthology.orchestra.run.smithi131.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-09T00:32:06.193 INFO:teuthology.orchestra.run.smithi131.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-09T00:32:06.194 INFO:teuthology.orchestra.run.smithi131.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-09T00:32:06.194 INFO:teuthology.orchestra.run.smithi131.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-09T00:32:06.194 INFO:teuthology.orchestra.run.smithi131.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-09T00:32:06.194 INFO:teuthology.orchestra.run.smithi131.stdout: libcephsqlite x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 158 k 2024-05-09T00:32:06.194 INFO:teuthology.orchestra.run.smithi131.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-09T00:32:06.194 INFO:teuthology.orchestra.run.smithi131.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-09T00:32:06.194 INFO:teuthology.orchestra.run.smithi131.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-09T00:32:06.194 INFO:teuthology.orchestra.run.smithi131.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-09T00:32:06.194 INFO:teuthology.orchestra.run.smithi131.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-09T00:32:06.194 INFO:teuthology.orchestra.run.smithi131.stdout: libradosstriper1 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 497 k 2024-05-09T00:32:06.194 INFO:teuthology.orchestra.run.smithi131.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-09T00:32:06.194 INFO:teuthology.orchestra.run.smithi131.stdout: librgw2 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 5.2 M 2024-05-09T00:32:06.194 INFO:teuthology.orchestra.run.smithi131.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-09T00:32:06.195 INFO:teuthology.orchestra.run.smithi131.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-09T00:32:06.195 INFO:teuthology.orchestra.run.smithi131.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-09T00:32:06.195 INFO:teuthology.orchestra.run.smithi131.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-09T00:32:06.195 INFO:teuthology.orchestra.run.smithi131.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-09T00:32:06.195 INFO:teuthology.orchestra.run.smithi131.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-09T00:32:06.195 INFO:teuthology.orchestra.run.smithi131.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-09T00:32:06.195 INFO:teuthology.orchestra.run.smithi131.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-09T00:32:06.195 INFO:teuthology.orchestra.run.smithi131.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-09T00:32:06.195 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-09T00:32:06.195 INFO:teuthology.orchestra.run.smithi131.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-09T00:32:06.195 INFO:teuthology.orchestra.run.smithi131.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-09T00:32:06.195 INFO:teuthology.orchestra.run.smithi131.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-09T00:32:06.195 INFO:teuthology.orchestra.run.smithi131.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-09T00:32:06.195 INFO:teuthology.orchestra.run.smithi131.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-09T00:32:06.196 INFO:teuthology.orchestra.run.smithi131.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-09T00:32:06.196 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-09T00:32:06.196 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ceph-argparse x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 45 k 2024-05-09T00:32:06.196 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ceph-common x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 135 k 2024-05-09T00:32:06.196 INFO:teuthology.orchestra.run.smithi131.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-09T00:32:06.196 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-09T00:32:06.196 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-09T00:32:06.196 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-09T00:32:06.196 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-09T00:32:06.196 INFO:teuthology.orchestra.run.smithi131.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-09T00:32:06.196 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-09T00:32:06.196 INFO:teuthology.orchestra.run.smithi131.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-09T00:32:06.196 INFO:teuthology.orchestra.run.smithi131.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-09T00:32:06.196 INFO:teuthology.orchestra.run.smithi131.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-09T00:32:06.197 INFO:teuthology.orchestra.run.smithi131.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-09T00:32:06.197 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-09T00:32:06.197 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-09T00:32:06.197 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-09T00:32:06.197 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-09T00:32:06.197 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-09T00:32:06.197 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-09T00:32:06.197 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-09T00:32:06.197 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-09T00:32:06.197 INFO:teuthology.orchestra.run.smithi131.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-09T00:32:06.197 INFO:teuthology.orchestra.run.smithi131.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-09T00:32:06.197 INFO:teuthology.orchestra.run.smithi131.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-09T00:32:06.197 INFO:teuthology.orchestra.run.smithi131.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-09T00:32:06.198 INFO:teuthology.orchestra.run.smithi131.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-09T00:32:06.198 INFO:teuthology.orchestra.run.smithi131.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-09T00:32:06.198 INFO:teuthology.orchestra.run.smithi131.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-09T00:32:06.198 INFO:teuthology.orchestra.run.smithi131.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-09T00:32:06.198 INFO:teuthology.orchestra.run.smithi131.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-09T00:32:06.198 INFO:teuthology.orchestra.run.smithi131.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-09T00:32:06.198 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-09T00:32:06.198 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-09T00:32:06.198 INFO:teuthology.orchestra.run.smithi131.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-09T00:32:06.198 INFO:teuthology.orchestra.run.smithi131.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-09T00:32:06.198 INFO:teuthology.orchestra.run.smithi131.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-09T00:32:06.198 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-09T00:32:06.198 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-09T00:32:06.198 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-09T00:32:06.199 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-09T00:32:06.199 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-09T00:32:06.199 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-09T00:32:06.199 INFO:teuthology.orchestra.run.smithi131.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-09T00:32:06.199 INFO:teuthology.orchestra.run.smithi131.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-09T00:32:06.199 INFO:teuthology.orchestra.run.smithi131.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-09T00:32:06.199 INFO:teuthology.orchestra.run.smithi131.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-09T00:32:06.199 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-09T00:32:06.199 INFO:teuthology.orchestra.run.smithi131.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-09T00:32:06.199 INFO:teuthology.orchestra.run.smithi131.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-09T00:32:06.199 INFO:teuthology.orchestra.run.smithi131.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-09T00:32:06.199 INFO:teuthology.orchestra.run.smithi131.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-09T00:32:06.199 INFO:teuthology.orchestra.run.smithi131.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-09T00:32:06.199 INFO:teuthology.orchestra.run.smithi131.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-09T00:32:06.200 INFO:teuthology.orchestra.run.smithi131.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-09T00:32:06.200 INFO:teuthology.orchestra.run.smithi131.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-09T00:32:06.200 INFO:teuthology.orchestra.run.smithi131.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-09T00:32:06.200 INFO:teuthology.orchestra.run.smithi131.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-09T00:32:06.200 INFO:teuthology.orchestra.run.smithi131.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-09T00:32:06.200 INFO:teuthology.orchestra.run.smithi131.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-09T00:32:06.200 INFO:teuthology.orchestra.run.smithi131.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-09T00:32:06.200 INFO:teuthology.orchestra.run.smithi131.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-09T00:32:06.200 INFO:teuthology.orchestra.run.smithi131.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-09T00:32:06.200 INFO:teuthology.orchestra.run.smithi131.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-09T00:32:06.200 INFO:teuthology.orchestra.run.smithi131.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-09T00:32:06.200 INFO:teuthology.orchestra.run.smithi131.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-09T00:32:06.200 INFO:teuthology.orchestra.run.smithi131.stdout:Installing weak dependencies: 2024-05-09T00:32:06.200 INFO:teuthology.orchestra.run.smithi131.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-09T00:32:06.201 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-09T00:32:06.201 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-05-09T00:32:06.201 INFO:teuthology.orchestra.run.smithi131.stdout:======================================================================================= 2024-05-09T00:32:06.201 INFO:teuthology.orchestra.run.smithi131.stdout:Install 148 Packages 2024-05-09T00:32:06.201 INFO:teuthology.orchestra.run.smithi131.stdout:Upgrade 8 Packages 2024-05-09T00:32:06.201 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-09T00:32:06.201 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 219 M 2024-05-09T00:32:06.202 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-05-09T00:32:07.047 INFO:teuthology.orchestra.run.smithi131.stdout:(1/156): ceph-19.0.0-3594.gc9da4049.el9.x86_64. 29 kB/s | 6.6 kB 00:00 2024-05-09T00:32:07.314 INFO:teuthology.orchestra.run.smithi131.stdout:(2/156): ceph-fuse-19.0.0-3594.gc9da4049.el9.x8 3.2 MB/s | 863 kB 00:00 2024-05-09T00:32:07.406 INFO:teuthology.orchestra.run.smithi131.stdout:(3/156): ceph-immutable-object-cache-19.0.0-359 1.6 MB/s | 147 kB 00:00 2024-05-09T00:32:07.690 INFO:teuthology.orchestra.run.smithi131.stdout:(4/156): ceph-base-19.0.0-3594.gc9da4049.el9.x8 6.2 MB/s | 5.4 MB 00:00 2024-05-09T00:32:07.748 INFO:teuthology.orchestra.run.smithi131.stdout:(5/156): ceph-mds-19.0.0-3594.gc9da4049.el9.x86 6.6 MB/s | 2.3 MB 00:00 2024-05-09T00:32:07.899 INFO:teuthology.orchestra.run.smithi131.stdout:(6/156): ceph-mgr-19.0.0-3594.gc9da4049.el9.x86 7.2 MB/s | 1.5 MB 00:00 2024-05-09T00:32:08.460 INFO:teuthology.orchestra.run.smithi131.stdout:(7/156): ceph-mon-19.0.0-3594.gc9da4049.el9.x86 6.9 MB/s | 4.9 MB 00:00 2024-05-09T00:32:08.883 INFO:teuthology.orchestra.run.smithi131.stdout:(8/156): ceph-common-19.0.0-3594.gc9da4049.el9. 9.5 MB/s | 20 MB 00:02 2024-05-09T00:32:08.984 INFO:teuthology.orchestra.run.smithi131.stdout:(9/156): ceph-selinux-19.0.0-3594.gc9da4049.el9 251 kB/s | 25 kB 00:00 2024-05-09T00:32:09.608 INFO:teuthology.orchestra.run.smithi131.stdout:(10/156): ceph-radosgw-19.0.0-3594.gc9da4049.el 8.8 MB/s | 10 MB 00:01 2024-05-09T00:32:09.701 INFO:teuthology.orchestra.run.smithi131.stdout:(11/156): cephfs-mirror-19.0.0-3594.gc9da4049.e 2.4 MB/s | 224 kB 00:00 2024-05-09T00:32:09.784 INFO:teuthology.orchestra.run.smithi131.stdout:(12/156): cephfs-shell-19.0.0-3594.gc9da4049.el 324 kB/s | 27 kB 00:00 2024-05-09T00:32:09.838 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - CRB 3.0 MB/s | 6.2 MB 00:02 2024-05-09T00:32:09.867 INFO:teuthology.orchestra.run.smithi131.stdout:(13/156): libcephfs-devel-19.0.0-3594.gc9da4049 380 kB/s | 32 kB 00:00 2024-05-09T00:32:10.043 INFO:teuthology.orchestra.run.smithi131.stdout:(14/156): ceph-osd-19.0.0-3594.gc9da4049.el9.x8 8.3 MB/s | 18 MB 00:02 2024-05-09T00:32:10.076 INFO:teuthology.orchestra.run.smithi131.stdout:(15/156): libcephfs2-19.0.0-3594.gc9da4049.el9. 3.5 MB/s | 756 kB 00:00 2024-05-09T00:32:10.118 INFO:teuthology.orchestra.run.smithi131.stdout:(16/156): libcephsqlite-19.0.0-3594.gc9da4049.e 2.1 MB/s | 158 kB 00:00 2024-05-09T00:32:10.160 INFO:teuthology.orchestra.run.smithi131.stdout:(17/156): librados-devel-19.0.0-3594.gc9da4049. 1.5 MB/s | 127 kB 00:00 2024-05-09T00:32:10.202 INFO:teuthology.orchestra.run.smithi131.stdout:(18/156): libradosstriper1-19.0.0-3594.gc9da404 5.8 MB/s | 497 kB 00:00 2024-05-09T00:32:10.294 INFO:teuthology.orchestra.run.smithi131.stdout:(19/156): python3-ceph-argparse-19.0.0-3594.gc9 491 kB/s | 45 kB 00:00 2024-05-09T00:32:10.395 INFO:teuthology.orchestra.run.smithi131.stdout:(20/156): python3-ceph-common-19.0.0-3594.gc9da 1.3 MB/s | 135 kB 00:00 2024-05-09T00:32:10.495 INFO:teuthology.orchestra.run.smithi131.stdout:(21/156): python3-cephfs-19.0.0-3594.gc9da4049. 1.6 MB/s | 161 kB 00:00 2024-05-09T00:32:10.595 INFO:teuthology.orchestra.run.smithi131.stdout:(22/156): python3-rados-19.0.0-3594.gc9da4049.e 3.1 MB/s | 322 kB 00:00 2024-05-09T00:32:10.729 INFO:teuthology.orchestra.run.smithi131.stdout:(23/156): librgw2-19.0.0-3594.gc9da4049.el9.x86 9.1 MB/s | 5.2 MB 00:00 2024-05-09T00:32:10.754 INFO:teuthology.orchestra.run.smithi131.stdout:(24/156): python3-rbd-19.0.0-3594.gc9da4049.el9 1.8 MB/s | 300 kB 00:00 2024-05-09T00:32:10.813 INFO:teuthology.orchestra.run.smithi131.stdout:(25/156): python3-rgw-19.0.0-3594.gc9da4049.el9 1.2 MB/s | 99 kB 00:00 2024-05-09T00:32:10.838 INFO:teuthology.orchestra.run.smithi131.stdout:(26/156): rbd-fuse-19.0.0-3594.gc9da4049.el9.x8 1.0 MB/s | 86 kB 00:00 2024-05-09T00:32:10.997 INFO:teuthology.orchestra.run.smithi131.stdout:(27/156): rbd-nbd-19.0.0-3594.gc9da4049.el9.x86 1.0 MB/s | 169 kB 00:00 2024-05-09T00:32:11.097 INFO:teuthology.orchestra.run.smithi131.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3594.g 342 kB/s | 34 kB 00:00 2024-05-09T00:32:11.189 INFO:teuthology.orchestra.run.smithi131.stdout:(29/156): rbd-mirror-19.0.0-3594.gc9da4049.el9. 8.1 MB/s | 3.0 MB 00:00 2024-05-09T00:32:11.214 INFO:teuthology.orchestra.run.smithi131.stdout:(30/156): ceph-mgr-cephadm-19.0.0-3594.gc9da404 1.2 MB/s | 142 kB 00:00 2024-05-09T00:32:11.280 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - Extras packages 80 kB/s | 16 kB 00:00 2024-05-09T00:32:11.681 INFO:teuthology.orchestra.run.smithi131.stdout:(31/156): ceph-mgr-dashboard-19.0.0-3594.gc9da4 7.2 MB/s | 3.5 MB 00:00 2024-05-09T00:32:12.182 INFO:teuthology.orchestra.run.smithi131.stdout:(32/156): ceph-test-19.0.0-3594.gc9da4049.el9.x 14 MB/s | 43 MB 00:03 2024-05-09T00:32:12.207 INFO:teuthology.orchestra.run.smithi131.stdout:(33/156): ceph-mgr-modules-core-19.0.0-3594.gc9 522 kB/s | 274 kB 00:00 2024-05-09T00:32:12.241 INFO:teuthology.orchestra.run.smithi131.stdout:(34/156): ceph-mgr-rook-19.0.0-3594.gc9da4049.e 848 kB/s | 49 kB 00:00 2024-05-09T00:32:12.350 INFO:teuthology.orchestra.run.smithi131.stdout:(35/156): ceph-mgr-diskprediction-local-19.0.0- 6.5 MB/s | 7.4 MB 00:01 2024-05-09T00:32:12.375 INFO:teuthology.orchestra.run.smithi131.stdout:(36/156): ceph-prometheus-alerts-19.0.0-3594.gc 98 kB/s | 16 kB 00:00 2024-05-09T00:32:12.400 INFO:teuthology.orchestra.run.smithi131.stdout:(37/156): ceph-volume-19.0.0-3594.gc9da4049.el9 1.7 MB/s | 273 kB 00:00 2024-05-09T00:32:12.434 INFO:teuthology.orchestra.run.smithi131.stdout:(38/156): cephfs-top-19.0.0-3594.gc9da4049.el9. 311 kB/s | 18 kB 00:00 2024-05-09T00:32:12.476 INFO:teuthology.orchestra.run.smithi131.stdout:(39/156): cephadm-19.0.0-3594.gc9da4049.el9.noa 5.9 MB/s | 764 kB 00:00 2024-05-09T00:32:12.568 INFO:teuthology.orchestra.run.smithi131.stdout:(40/156): fuse-2.9.9-15.el9.x86_64.rpm 480 kB/s | 80 kB 00:00 2024-05-09T00:32:12.668 INFO:teuthology.orchestra.run.smithi131.stdout:(41/156): libquadmath-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 195 kB 00:00 2024-05-09T00:32:12.693 INFO:teuthology.orchestra.run.smithi131.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.0 MB/s | 253 kB 00:00 2024-05-09T00:32:12.752 INFO:teuthology.orchestra.run.smithi131.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-05-09T00:32:12.811 INFO:teuthology.orchestra.run.smithi131.stdout:(44/156): libgfortran-11.4.1-3.el9.x86_64.rpm 2.1 MB/s | 803 kB 00:00 2024-05-09T00:32:12.844 INFO:teuthology.orchestra.run.smithi131.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 1.4 MB/s | 135 kB 00:00 2024-05-09T00:32:12.869 INFO:teuthology.orchestra.run.smithi131.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-05-09T00:32:12.928 INFO:teuthology.orchestra.run.smithi131.stdout:(47/156): python3-urllib3-1.26.5-5.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-05-09T00:32:12.995 INFO:teuthology.orchestra.run.smithi131.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 5.4 MB/s | 689 kB 00:00 2024-05-09T00:32:13.054 INFO:teuthology.orchestra.run.smithi131.stdout:(49/156): bison-3.7.4-5.el9.x86_64.rpm 7.4 MB/s | 943 kB 00:00 2024-05-09T00:32:13.073 INFO:teuthology.orchestra.run.smithi005.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-05-09T00:32:13.087 INFO:teuthology.orchestra.run.smithi131.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 1.1 MB/s | 107 kB 00:00 2024-05-09T00:32:13.130 INFO:teuthology.orchestra.run.smithi131.stdout:(51/156): elfutils-libelf-devel-0.190-2.el9.x86 285 kB/s | 21 kB 00:00 2024-05-09T00:32:13.180 INFO:teuthology.orchestra.run.smithi131.stdout:(52/156): flexiblas-3.0.4-8.el9.x86_64.rpm 656 kB/s | 33 kB 00:00 2024-05-09T00:32:13.205 INFO:teuthology.orchestra.run.smithi131.stdout:(53/156): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-05-09T00:32:13.247 INFO:teuthology.orchestra.run.smithi131.stdout:(54/156): python3-cryptography-36.0.1-4.el9.x86 2.1 MB/s | 1.2 MB 00:00 2024-05-09T00:32:13.281 INFO:teuthology.orchestra.run.smithi131.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 242 kB/s | 18 kB 00:00 2024-05-09T00:32:13.306 INFO:teuthology.orchestra.run.smithi131.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 190 kB/s | 11 kB 00:00 2024-05-09T00:32:13.365 INFO:teuthology.orchestra.run.smithi131.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 776 kB/s | 45 kB 00:00 2024-05-09T00:32:13.390 INFO:teuthology.orchestra.run.smithi131.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 160 kB 00:00 2024-05-09T00:32:13.441 INFO:teuthology.orchestra.run.smithi131.stdout:(59/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 11 MB/s | 3.0 MB 00:00 2024-05-09T00:32:13.482 INFO:teuthology.orchestra.run.smithi131.stdout:(60/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 614 kB/s | 25 kB 00:00 2024-05-09T00:32:13.516 INFO:teuthology.orchestra.run.smithi131.stdout:(61/156): librdkafka-1.6.1-102.el9.x86_64.rpm 4.3 MB/s | 662 kB 00:00 2024-05-09T00:32:13.542 INFO:teuthology.orchestra.run.smithi131.stdout:(62/156): libtool-2.4.6-45.el9.x86_64.rpm 3.8 MB/s | 582 kB 00:00 2024-05-09T00:32:13.567 INFO:teuthology.orchestra.run.smithi131.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 562 kB/s | 47 kB 00:00 2024-05-09T00:32:13.617 INFO:teuthology.orchestra.run.smithi131.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.8 MB/s | 292 kB 00:00 2024-05-09T00:32:13.643 INFO:teuthology.orchestra.run.smithi131.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 468 kB/s | 35 kB 00:00 2024-05-09T00:32:13.668 INFO:teuthology.orchestra.run.smithi131.stdout:(66/156): lua-5.4.4-4.el9.x86_64.rpm 1.5 MB/s | 188 kB 00:00 2024-05-09T00:32:13.810 INFO:teuthology.orchestra.run.smithi131.stdout:(67/156): postgresql-13.14-1.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-05-09T00:32:13.893 INFO:teuthology.orchestra.run.smithi131.stdout:(68/156): openblas-openmp-0.3.21-2.el9.x86_64.r 17 MB/s | 4.7 MB 00:00 2024-05-09T00:32:13.952 INFO:teuthology.orchestra.run.smithi131.stdout:(69/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-05-09T00:32:14.019 INFO:teuthology.orchestra.run.smithi131.stdout:(70/156): postgresql-contrib-13.14-1.el9.x86_64 4.3 MB/s | 911 kB 00:00 2024-05-09T00:32:14.044 INFO:teuthology.orchestra.run.smithi131.stdout:(71/156): postgresql-private-libs-13.14-1.el9.x 885 kB/s | 133 kB 00:00 2024-05-09T00:32:14.103 INFO:teuthology.orchestra.run.smithi131.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-05-09T00:32:14.153 INFO:teuthology.orchestra.run.smithi131.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 7.5 MB/s | 1.0 MB 00:00 2024-05-09T00:32:14.263 INFO:teuthology.orchestra.run.smithi131.stdout:(74/156): postgresql-server-13.14-1.el9.x86_64. 19 MB/s | 5.8 MB 00:00 2024-05-09T00:32:14.312 INFO:teuthology.orchestra.run.smithi131.stdout:(75/156): python3-devel-3.9.18-3.el9.x86_64.rpm 1.5 MB/s | 244 kB 00:00 2024-05-09T00:32:14.363 INFO:teuthology.orchestra.run.smithi131.stdout:(76/156): python3-jinja2-2.11.3-5.el9.noarch.rp 2.4 MB/s | 248 kB 00:00 2024-05-09T00:32:14.388 INFO:teuthology.orchestra.run.smithi131.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 341 kB/s | 26 kB 00:00 2024-05-09T00:32:14.438 INFO:teuthology.orchestra.run.smithi131.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 254 kB/s | 19 kB 00:00 2024-05-09T00:32:14.522 INFO:teuthology.orchestra.run.smithi131.stdout:(79/156): python3-babel-2.9.1-2.el9.noarch.rpm 14 MB/s | 6.0 MB 00:00 2024-05-09T00:32:14.547 INFO:teuthology.orchestra.run.smithi131.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 320 kB/s | 35 kB 00:00 2024-05-09T00:32:14.573 INFO:teuthology.orchestra.run.smithi131.stdout:(81/156): python3-mako-1.1.4-6.el9.noarch.rpm 935 kB/s | 172 kB 00:00 2024-05-09T00:32:14.639 INFO:teuthology.orchestra.run.smithi131.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.7 MB/s | 250 kB 00:00 2024-05-09T00:32:14.665 INFO:teuthology.orchestra.run.smithi131.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.4 MB/s | 222 kB 00:00 2024-05-09T00:32:14.732 INFO:teuthology.orchestra.run.smithi131.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 840 kB/s | 77 kB 00:00 2024-05-09T00:32:14.733 INFO:teuthology.orchestra.run.smithi157.stdout:Extra Packages for Enterprise Linux 1.1 MB/s | 21 MB 00:19 2024-05-09T00:32:14.757 INFO:teuthology.orchestra.run.smithi131.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 459 kB/s | 42 kB 00:00 2024-05-09T00:32:14.824 INFO:teuthology.orchestra.run.smithi131.stdout:(86/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 17 MB/s | 5.1 MB 00:00 2024-05-09T00:32:14.875 INFO:teuthology.orchestra.run.smithi131.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 1.8 MB/s | 269 kB 00:00 2024-05-09T00:32:14.900 INFO:teuthology.orchestra.run.smithi131.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.1 MB/s | 159 kB 00:00 2024-05-09T00:32:14.933 INFO:teuthology.orchestra.run.smithi131.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.5 MB/s | 279 kB 00:00 2024-05-09T00:32:14.951 INFO:teuthology.orchestra.run.smithi131.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 673 kB/s | 51 kB 00:00 2024-05-09T00:32:14.976 INFO:teuthology.orchestra.run.smithi131.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 711 kB/s | 54 kB 00:00 2024-05-09T00:32:15.018 INFO:teuthology.orchestra.run.smithi131.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 622 kB/s | 42 kB 00:00 2024-05-09T00:32:15.043 INFO:teuthology.orchestra.run.smithi131.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 658 kB/s | 44 kB 00:00 2024-05-09T00:32:15.085 INFO:teuthology.orchestra.run.smithi131.stdout:(94/156): qatlib-service-24.02.0-1.el9.x86_64.r 861 kB/s | 36 kB 00:00 2024-05-09T00:32:15.118 INFO:teuthology.orchestra.run.smithi131.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 2.1 MB/s | 221 kB 00:00 2024-05-09T00:32:15.169 INFO:teuthology.orchestra.run.smithi131.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 565 kB/s | 47 kB 00:00 2024-05-09T00:32:15.219 INFO:teuthology.orchestra.run.smithi131.stdout:(97/156): socat-1.7.4.1-5.el9.x86_64.rpm 3.0 MB/s | 305 kB 00:00 2024-05-09T00:32:15.245 INFO:teuthology.orchestra.run.smithi131.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 768 kB/s | 58 kB 00:00 2024-05-09T00:32:15.312 INFO:teuthology.orchestra.run.smithi131.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.0 MB/s | 94 kB 00:00 2024-05-09T00:32:15.337 INFO:teuthology.orchestra.run.smithi131.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 691 kB/s | 64 kB 00:00 2024-05-09T00:32:15.395 INFO:teuthology.orchestra.run.smithi131.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 537 kB/s | 45 kB 00:00 2024-05-09T00:32:15.563 INFO:teuthology.orchestra.run.smithi131.stdout:(102/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 26 MB/s | 16 MB 00:00 2024-05-09T00:32:15.588 INFO:teuthology.orchestra.run.smithi131.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 89 kB/s | 22 kB 00:00 2024-05-09T00:32:15.793 INFO:teuthology.orchestra.run.smithi131.stdout:(104/156): gperftools-libs-2.9.1-2.el9.x86_64.r 1.5 MB/s | 309 kB 00:00 2024-05-09T00:32:15.838 INFO:teuthology.orchestra.run.smithi131.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 427 kB/s | 19 kB 00:00 2024-05-09T00:32:15.889 INFO:teuthology.orchestra.run.smithi131.stdout:(106/156): abseil-cpp-20211102.0-3.el9.x86_64.r 1.6 MB/s | 548 kB 00:00 2024-05-09T00:32:15.931 INFO:teuthology.orchestra.run.smithi131.stdout:(107/156): protobuf-compiler-3.14.0-13.el9.x86_ 1.6 MB/s | 863 kB 00:00 2024-05-09T00:32:15.956 INFO:teuthology.orchestra.run.smithi131.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 388 kB/s | 26 kB 00:00 2024-05-09T00:32:16.157 INFO:teuthology.orchestra.run.smithi131.stdout:(109/156): libarrow-9.0.0-11.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-05-09T00:32:16.182 INFO:teuthology.orchestra.run.smithi131.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 335 kB/s | 67 kB 00:00 2024-05-09T00:32:16.207 INFO:teuthology.orchestra.run.smithi131.stdout:(111/156): luarocks-3.9.2-1.el9.noarch.rpm 3.0 MB/s | 151 kB 00:00 2024-05-09T00:32:16.233 INFO:teuthology.orchestra.run.smithi131.stdout:(112/156): liboath-2.6.7-2.el9.x86_64.rpm 161 kB/s | 49 kB 00:00 2024-05-09T00:32:16.267 INFO:teuthology.orchestra.run.smithi131.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 9.7 MB/s | 839 kB 00:00 2024-05-09T00:32:16.292 INFO:teuthology.orchestra.run.smithi131.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 740 kB/s | 43 kB 00:00 2024-05-09T00:32:16.317 INFO:teuthology.orchestra.run.smithi131.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 642 kB/s | 32 kB 00:00 2024-05-09T00:32:16.342 INFO:teuthology.orchestra.run.smithi131.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 297 kB/s | 15 kB 00:00 2024-05-09T00:32:16.368 INFO:teuthology.orchestra.run.smithi131.stdout:(117/156): python3-asyncssh-2.13.2-5.el9.noarch 3.3 MB/s | 548 kB 00:00 2024-05-09T00:32:16.393 INFO:teuthology.orchestra.run.smithi131.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-05-09T00:32:16.418 INFO:teuthology.orchestra.run.smithi131.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-05-09T00:32:16.444 INFO:teuthology.orchestra.run.smithi131.stdout:(120/156): python3-colorama-0.4.4-12.el9.noarch 702 kB/s | 35 kB 00:00 2024-05-09T00:32:16.469 INFO:teuthology.orchestra.run.smithi131.stdout:(121/156): python3-cherrypy-18.6.1-2.el9.noarch 2.8 MB/s | 358 kB 00:00 2024-05-09T00:32:16.494 INFO:teuthology.orchestra.run.smithi131.stdout:(122/156): python3-google-auth-2.29.0-1.el9.noa 2.7 MB/s | 206 kB 00:00 2024-05-09T00:32:16.520 INFO:teuthology.orchestra.run.smithi131.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-05-09T00:32:16.545 INFO:teuthology.orchestra.run.smithi131.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 212 kB/s | 11 kB 00:00 2024-05-09T00:32:16.570 INFO:teuthology.orchestra.run.smithi131.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 351 kB/s | 18 kB 00:00 2024-05-09T00:32:16.595 INFO:teuthology.orchestra.run.smithi131.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 459 kB/s | 23 kB 00:00 2024-05-09T00:32:16.621 INFO:teuthology.orchestra.run.smithi131.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 385 kB/s | 19 kB 00:00 2024-05-09T00:32:16.646 INFO:teuthology.orchestra.run.smithi131.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 390 kB/s | 20 kB 00:00 2024-05-09T00:32:16.688 INFO:teuthology.orchestra.run.smithi131.stdout:(129/156): python3-kubernetes-26.1.0-2.el9.noar 15 MB/s | 1.0 MB 00:00 2024-05-09T00:32:16.714 INFO:teuthology.orchestra.run.smithi131.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 691 kB/s | 46 kB 00:00 2024-05-09T00:32:16.730 INFO:teuthology.orchestra.run.smithi131.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 1.8 MB/s | 79 kB 00:00 2024-05-09T00:32:16.756 INFO:teuthology.orchestra.run.smithi131.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 1.3 MB/s | 58 kB 00:00 2024-05-09T00:32:16.798 INFO:teuthology.orchestra.run.smithi131.stdout:(133/156): python3-grpcio-1.46.7-10.el9.x86_64. 5.8 MB/s | 2.0 MB 00:00 2024-05-09T00:32:16.815 INFO:teuthology.orchestra.run.smithi131.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 280 kB/s | 16 kB 00:00 2024-05-09T00:32:16.840 INFO:teuthology.orchestra.run.smithi131.stdout:(135/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-05-09T00:32:16.857 INFO:teuthology.orchestra.run.smithi131.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 672 kB/s | 28 kB 00:00 2024-05-09T00:32:16.882 INFO:teuthology.orchestra.run.smithi131.stdout:(137/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.0 MB/s | 90 kB 00:00 2024-05-09T00:32:16.899 INFO:teuthology.orchestra.run.smithi131.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 523 kB/s | 31 kB 00:00 2024-05-09T00:32:16.934 INFO:teuthology.orchestra.run.smithi131.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-05-09T00:32:16.958 INFO:teuthology.orchestra.run.smithi131.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 607 kB/s | 36 kB 00:00 2024-05-09T00:32:16.984 INFO:teuthology.orchestra.run.smithi131.stdout:(141/156): python3-rsa-4.9-2.el9.noarch.rpm 703 kB/s | 59 kB 00:00 2024-05-09T00:32:17.009 INFO:teuthology.orchestra.run.smithi131.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 675 kB/s | 51 kB 00:00 2024-05-09T00:32:17.034 INFO:teuthology.orchestra.run.smithi131.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-05-09T00:32:17.060 INFO:teuthology.orchestra.run.smithi131.stdout:(144/156): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-05-09T00:32:17.085 INFO:teuthology.orchestra.run.smithi131.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 397 kB/s | 20 kB 00:00 2024-05-09T00:32:17.110 INFO:teuthology.orchestra.run.smithi131.stdout:(146/156): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-05-09T00:32:17.136 INFO:teuthology.orchestra.run.smithi131.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-05-09T00:32:17.178 INFO:teuthology.orchestra.run.smithi131.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-05-09T00:32:17.570 INFO:teuthology.orchestra.run.smithi131.stdout:(149/156): librados2-19.0.0-3594.gc9da4049.el9. 7.3 MB/s | 3.3 MB 00:00 2024-05-09T00:32:17.620 INFO:teuthology.orchestra.run.smithi131.stdout:(150/156): librbd1-19.0.0-3594.gc9da4049.el9.x8 6.4 MB/s | 3.1 MB 00:00 2024-05-09T00:32:17.678 INFO:teuthology.orchestra.run.smithi131.stdout:(151/156): NetworkManager-team-1.47.5-1.el9.x86 715 kB/s | 42 kB 00:00 2024-05-09T00:32:17.720 INFO:teuthology.orchestra.run.smithi131.stdout:(152/156): NetworkManager-1.47.5-1.el9.x86_64.r 4.3 MB/s | 2.3 MB 00:00 2024-05-09T00:32:17.779 INFO:teuthology.orchestra.run.smithi131.stdout:(153/156): NetworkManager-tui-1.47.5-1.el9.x86_ 2.4 MB/s | 248 kB 00:00 2024-05-09T00:32:18.087 INFO:teuthology.orchestra.run.smithi131.stdout:(154/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 7.8 MB/s | 2.4 MB 00:00 2024-05-09T00:32:18.130 INFO:teuthology.orchestra.run.smithi131.stdout:(155/156): NetworkManager-libnm-1.47.5-1.el9.x8 3.3 MB/s | 1.8 MB 00:00 2024-05-09T00:32:18.196 INFO:teuthology.orchestra.run.smithi131.stdout:(156/156): openssl-3.2.1-1.el9.x86_64.rpm 2.8 MB/s | 1.3 MB 00:00 2024-05-09T00:32:18.202 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-05-09T00:32:18.202 INFO:teuthology.orchestra.run.smithi131.stdout:Total 18 MB/s | 219 MB 00:12 2024-05-09T00:32:18.390 INFO:teuthology.orchestra.run.smithi005.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-05-09T00:32:19.384 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-05-09T00:32:19.537 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-05-09T00:32:19.537 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-05-09T00:32:19.856 INFO:teuthology.orchestra.run.smithi157.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2024-05-09T00:32:20.793 INFO:teuthology.orchestra.run.smithi005.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T00:32:20.793 INFO:teuthology.orchestra.run.smithi005.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T00:32:20.805 INFO:teuthology.orchestra.run.smithi005.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-09T00:32:20.805 INFO:teuthology.orchestra.run.smithi005.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-09T00:32:20.806 INFO:teuthology.orchestra.run.smithi005.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-09T00:32:20.811 INFO:teuthology.orchestra.run.smithi005.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-09T00:32:20.811 INFO:teuthology.orchestra.run.smithi005.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-09T00:32:20.811 INFO:teuthology.orchestra.run.smithi005.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-09T00:32:20.812 INFO:teuthology.orchestra.run.smithi005.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-09T00:32:20.878 INFO:teuthology.orchestra.run.smithi005.stdout:Dependencies resolved. 2024-05-09T00:32:20.889 INFO:teuthology.orchestra.run.smithi005.stdout:======================================================================================= 2024-05-09T00:32:20.890 INFO:teuthology.orchestra.run.smithi005.stdout: Package Arch Version Repository Size 2024-05-09T00:32:20.890 INFO:teuthology.orchestra.run.smithi005.stdout:======================================================================================= 2024-05-09T00:32:20.890 INFO:teuthology.orchestra.run.smithi005.stdout:Installing: 2024-05-09T00:32:20.890 INFO:teuthology.orchestra.run.smithi005.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-09T00:32:20.890 INFO:teuthology.orchestra.run.smithi005.stdout: ceph x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 6.6 k 2024-05-09T00:32:20.890 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-base x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 5.4 M 2024-05-09T00:32:20.890 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-fuse x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 863 k 2024-05-09T00:32:20.890 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 147 k 2024-05-09T00:32:20.890 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 1.5 M 2024-05-09T00:32:20.890 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 142 k 2024-05-09T00:32:20.890 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 3.5 M 2024-05-09T00:32:20.890 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 7.4 M 2024-05-09T00:32:20.891 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-rook noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 49 k 2024-05-09T00:32:20.891 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-radosgw x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 10 M 2024-05-09T00:32:20.891 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-test x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 43 M 2024-05-09T00:32:20.891 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-volume noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 273 k 2024-05-09T00:32:20.891 INFO:teuthology.orchestra.run.smithi005.stdout: cephadm noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 764 k 2024-05-09T00:32:20.891 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-mirror x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 224 k 2024-05-09T00:32:20.891 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-shell x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 27 k 2024-05-09T00:32:20.891 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-top noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 18 k 2024-05-09T00:32:20.891 INFO:teuthology.orchestra.run.smithi005.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-09T00:32:20.891 INFO:teuthology.orchestra.run.smithi005.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-09T00:32:20.891 INFO:teuthology.orchestra.run.smithi005.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-09T00:32:20.892 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs-devel x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 32 k 2024-05-09T00:32:20.892 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs2 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 756 k 2024-05-09T00:32:20.892 INFO:teuthology.orchestra.run.smithi005.stdout: librados-devel x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 127 k 2024-05-09T00:32:20.892 INFO:teuthology.orchestra.run.smithi005.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-09T00:32:20.892 INFO:teuthology.orchestra.run.smithi005.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-09T00:32:20.892 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-09T00:32:20.892 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-09T00:32:20.892 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-09T00:32:20.892 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-09T00:32:20.893 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cephfs x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 161 k 2024-05-09T00:32:20.893 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rados x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 322 k 2024-05-09T00:32:20.893 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rbd x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 300 k 2024-05-09T00:32:20.893 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rgw x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 99 k 2024-05-09T00:32:20.893 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-fuse x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 86 k 2024-05-09T00:32:20.893 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-mirror x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 3.0 M 2024-05-09T00:32:20.893 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-nbd x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 169 k 2024-05-09T00:32:20.893 INFO:teuthology.orchestra.run.smithi005.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-09T00:32:20.893 INFO:teuthology.orchestra.run.smithi005.stdout:Upgrading: 2024-05-09T00:32:20.893 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-09T00:32:20.893 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-09T00:32:20.894 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-09T00:32:20.894 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-09T00:32:20.894 INFO:teuthology.orchestra.run.smithi005.stdout: librados2 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 3.3 M 2024-05-09T00:32:20.894 INFO:teuthology.orchestra.run.smithi005.stdout: librbd1 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 3.1 M 2024-05-09T00:32:20.894 INFO:teuthology.orchestra.run.smithi005.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-09T00:32:20.894 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-09T00:32:20.894 INFO:teuthology.orchestra.run.smithi005.stdout:Installing dependencies: 2024-05-09T00:32:20.894 INFO:teuthology.orchestra.run.smithi005.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-09T00:32:20.894 INFO:teuthology.orchestra.run.smithi005.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-09T00:32:20.894 INFO:teuthology.orchestra.run.smithi005.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-09T00:32:20.894 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-common x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 20 M 2024-05-09T00:32:20.895 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 34 k 2024-05-09T00:32:20.895 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mds x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 2.3 M 2024-05-09T00:32:20.895 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 274 k 2024-05-09T00:32:20.895 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mon x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 4.9 M 2024-05-09T00:32:20.895 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-osd x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 18 M 2024-05-09T00:32:20.895 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 16 k 2024-05-09T00:32:20.895 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-selinux x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 25 k 2024-05-09T00:32:20.895 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-09T00:32:20.895 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-09T00:32:20.895 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-09T00:32:20.895 INFO:teuthology.orchestra.run.smithi005.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-09T00:32:20.895 INFO:teuthology.orchestra.run.smithi005.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-09T00:32:20.896 INFO:teuthology.orchestra.run.smithi005.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-09T00:32:20.896 INFO:teuthology.orchestra.run.smithi005.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-09T00:32:20.896 INFO:teuthology.orchestra.run.smithi005.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-09T00:32:20.896 INFO:teuthology.orchestra.run.smithi005.stdout: libcephsqlite x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 158 k 2024-05-09T00:32:20.896 INFO:teuthology.orchestra.run.smithi005.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-09T00:32:20.896 INFO:teuthology.orchestra.run.smithi005.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-09T00:32:20.896 INFO:teuthology.orchestra.run.smithi005.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-09T00:32:20.896 INFO:teuthology.orchestra.run.smithi005.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-09T00:32:20.896 INFO:teuthology.orchestra.run.smithi005.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-09T00:32:20.896 INFO:teuthology.orchestra.run.smithi005.stdout: libradosstriper1 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 497 k 2024-05-09T00:32:20.896 INFO:teuthology.orchestra.run.smithi005.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-09T00:32:20.896 INFO:teuthology.orchestra.run.smithi005.stdout: librgw2 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 5.2 M 2024-05-09T00:32:20.897 INFO:teuthology.orchestra.run.smithi005.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-09T00:32:20.897 INFO:teuthology.orchestra.run.smithi005.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-09T00:32:20.897 INFO:teuthology.orchestra.run.smithi005.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-09T00:32:20.897 INFO:teuthology.orchestra.run.smithi005.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-09T00:32:20.897 INFO:teuthology.orchestra.run.smithi005.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-09T00:32:20.897 INFO:teuthology.orchestra.run.smithi005.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-09T00:32:20.897 INFO:teuthology.orchestra.run.smithi005.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-09T00:32:20.897 INFO:teuthology.orchestra.run.smithi005.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-09T00:32:20.897 INFO:teuthology.orchestra.run.smithi005.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-09T00:32:20.897 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-09T00:32:20.897 INFO:teuthology.orchestra.run.smithi005.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-09T00:32:20.897 INFO:teuthology.orchestra.run.smithi005.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-09T00:32:20.897 INFO:teuthology.orchestra.run.smithi005.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-09T00:32:20.898 INFO:teuthology.orchestra.run.smithi005.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-09T00:32:20.898 INFO:teuthology.orchestra.run.smithi005.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-09T00:32:20.898 INFO:teuthology.orchestra.run.smithi005.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-09T00:32:20.898 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-09T00:32:20.898 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-argparse x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 45 k 2024-05-09T00:32:20.898 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-common x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 135 k 2024-05-09T00:32:20.898 INFO:teuthology.orchestra.run.smithi005.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-09T00:32:20.898 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-09T00:32:20.898 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-09T00:32:20.898 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-09T00:32:20.898 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-09T00:32:20.898 INFO:teuthology.orchestra.run.smithi005.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-09T00:32:20.898 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-09T00:32:20.899 INFO:teuthology.orchestra.run.smithi005.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-09T00:32:20.899 INFO:teuthology.orchestra.run.smithi005.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-09T00:32:20.899 INFO:teuthology.orchestra.run.smithi005.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-09T00:32:20.899 INFO:teuthology.orchestra.run.smithi005.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-09T00:32:20.899 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-09T00:32:20.899 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-09T00:32:20.899 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-09T00:32:20.899 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-09T00:32:20.899 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-09T00:32:20.899 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-09T00:32:20.899 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-09T00:32:20.899 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-09T00:32:20.899 INFO:teuthology.orchestra.run.smithi005.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-09T00:32:20.900 INFO:teuthology.orchestra.run.smithi005.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-09T00:32:20.900 INFO:teuthology.orchestra.run.smithi005.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-09T00:32:20.900 INFO:teuthology.orchestra.run.smithi005.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-09T00:32:20.900 INFO:teuthology.orchestra.run.smithi005.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-09T00:32:20.900 INFO:teuthology.orchestra.run.smithi005.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-09T00:32:20.900 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-09T00:32:20.900 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-09T00:32:20.900 INFO:teuthology.orchestra.run.smithi005.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-09T00:32:20.900 INFO:teuthology.orchestra.run.smithi005.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-09T00:32:20.900 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-09T00:32:20.900 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-09T00:32:20.900 INFO:teuthology.orchestra.run.smithi005.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-09T00:32:20.900 INFO:teuthology.orchestra.run.smithi005.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-09T00:32:20.900 INFO:teuthology.orchestra.run.smithi005.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-09T00:32:20.901 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-09T00:32:20.901 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-09T00:32:20.901 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-09T00:32:20.901 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-09T00:32:20.901 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-09T00:32:20.901 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-09T00:32:20.901 INFO:teuthology.orchestra.run.smithi005.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-09T00:32:20.901 INFO:teuthology.orchestra.run.smithi005.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-09T00:32:20.901 INFO:teuthology.orchestra.run.smithi005.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-09T00:32:20.901 INFO:teuthology.orchestra.run.smithi005.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-09T00:32:20.901 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-09T00:32:20.901 INFO:teuthology.orchestra.run.smithi005.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-09T00:32:20.901 INFO:teuthology.orchestra.run.smithi005.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-09T00:32:20.901 INFO:teuthology.orchestra.run.smithi005.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-09T00:32:20.902 INFO:teuthology.orchestra.run.smithi005.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-09T00:32:20.902 INFO:teuthology.orchestra.run.smithi005.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-09T00:32:20.902 INFO:teuthology.orchestra.run.smithi005.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-09T00:32:20.902 INFO:teuthology.orchestra.run.smithi005.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-09T00:32:20.902 INFO:teuthology.orchestra.run.smithi005.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-09T00:32:20.902 INFO:teuthology.orchestra.run.smithi005.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-09T00:32:20.902 INFO:teuthology.orchestra.run.smithi005.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-09T00:32:20.902 INFO:teuthology.orchestra.run.smithi005.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-09T00:32:20.902 INFO:teuthology.orchestra.run.smithi005.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-09T00:32:20.902 INFO:teuthology.orchestra.run.smithi005.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-09T00:32:20.902 INFO:teuthology.orchestra.run.smithi005.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-09T00:32:20.902 INFO:teuthology.orchestra.run.smithi005.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-09T00:32:20.902 INFO:teuthology.orchestra.run.smithi005.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-09T00:32:20.903 INFO:teuthology.orchestra.run.smithi005.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-09T00:32:20.903 INFO:teuthology.orchestra.run.smithi005.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-09T00:32:20.903 INFO:teuthology.orchestra.run.smithi005.stdout:Installing weak dependencies: 2024-05-09T00:32:20.903 INFO:teuthology.orchestra.run.smithi005.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-09T00:32:20.903 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:32:20.903 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction Summary 2024-05-09T00:32:20.903 INFO:teuthology.orchestra.run.smithi005.stdout:======================================================================================= 2024-05-09T00:32:20.903 INFO:teuthology.orchestra.run.smithi005.stdout:Install 148 Packages 2024-05-09T00:32:20.903 INFO:teuthology.orchestra.run.smithi005.stdout:Upgrade 8 Packages 2024-05-09T00:32:20.903 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:32:20.904 INFO:teuthology.orchestra.run.smithi005.stdout:Total download size: 219 M 2024-05-09T00:32:20.904 INFO:teuthology.orchestra.run.smithi005.stdout:Downloading Packages: 2024-05-09T00:32:21.313 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-05-09T00:32:21.314 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-05-09T00:32:22.125 INFO:teuthology.orchestra.run.smithi005.stdout:(1/156): ceph-19.0.0-3594.gc9da4049.el9.x86_64. 25 kB/s | 6.6 kB 00:00 2024-05-09T00:32:22.141 INFO:teuthology.orchestra.run.smithi157.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T00:32:22.141 INFO:teuthology.orchestra.run.smithi157.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T00:32:22.152 INFO:teuthology.orchestra.run.smithi157.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-09T00:32:22.152 INFO:teuthology.orchestra.run.smithi157.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-09T00:32:22.153 INFO:teuthology.orchestra.run.smithi157.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-09T00:32:22.154 INFO:teuthology.orchestra.run.smithi157.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-09T00:32:22.156 INFO:teuthology.orchestra.run.smithi157.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-09T00:32:22.158 INFO:teuthology.orchestra.run.smithi157.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-09T00:32:22.159 INFO:teuthology.orchestra.run.smithi157.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-09T00:32:22.219 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-05-09T00:32:22.238 INFO:teuthology.orchestra.run.smithi157.stdout:======================================================================================= 2024-05-09T00:32:22.238 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repository Size 2024-05-09T00:32:22.238 INFO:teuthology.orchestra.run.smithi157.stdout:======================================================================================= 2024-05-09T00:32:22.238 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2024-05-09T00:32:22.238 INFO:teuthology.orchestra.run.smithi157.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-09T00:32:22.239 INFO:teuthology.orchestra.run.smithi157.stdout: ceph x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 6.6 k 2024-05-09T00:32:22.239 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-base x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 5.4 M 2024-05-09T00:32:22.239 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-fuse x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 863 k 2024-05-09T00:32:22.239 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 147 k 2024-05-09T00:32:22.239 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 1.5 M 2024-05-09T00:32:22.239 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 142 k 2024-05-09T00:32:22.239 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 3.5 M 2024-05-09T00:32:22.239 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 7.4 M 2024-05-09T00:32:22.239 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-rook noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 49 k 2024-05-09T00:32:22.239 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-radosgw x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 10 M 2024-05-09T00:32:22.239 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-test x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 43 M 2024-05-09T00:32:22.239 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-volume noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 273 k 2024-05-09T00:32:22.240 INFO:teuthology.orchestra.run.smithi157.stdout: cephadm noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 764 k 2024-05-09T00:32:22.240 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-mirror x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 224 k 2024-05-09T00:32:22.240 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-shell x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 27 k 2024-05-09T00:32:22.240 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-top noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 18 k 2024-05-09T00:32:22.240 INFO:teuthology.orchestra.run.smithi157.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-09T00:32:22.240 INFO:teuthology.orchestra.run.smithi157.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-09T00:32:22.240 INFO:teuthology.orchestra.run.smithi157.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-09T00:32:22.240 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs-devel x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 32 k 2024-05-09T00:32:22.240 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs2 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 756 k 2024-05-09T00:32:22.240 INFO:teuthology.orchestra.run.smithi157.stdout: librados-devel x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 127 k 2024-05-09T00:32:22.240 INFO:teuthology.orchestra.run.smithi157.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-09T00:32:22.240 INFO:teuthology.orchestra.run.smithi157.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-09T00:32:22.240 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-09T00:32:22.241 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-09T00:32:22.241 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-09T00:32:22.241 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-09T00:32:22.241 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cephfs x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 161 k 2024-05-09T00:32:22.241 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rados x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 322 k 2024-05-09T00:32:22.241 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rbd x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 300 k 2024-05-09T00:32:22.241 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rgw x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 99 k 2024-05-09T00:32:22.241 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-fuse x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 86 k 2024-05-09T00:32:22.241 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-mirror x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 3.0 M 2024-05-09T00:32:22.241 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-nbd x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 169 k 2024-05-09T00:32:22.241 INFO:teuthology.orchestra.run.smithi157.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-09T00:32:22.242 INFO:teuthology.orchestra.run.smithi157.stdout:Upgrading: 2024-05-09T00:32:22.242 INFO:teuthology.orchestra.run.smithi157.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-09T00:32:22.242 INFO:teuthology.orchestra.run.smithi157.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-09T00:32:22.242 INFO:teuthology.orchestra.run.smithi157.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-09T00:32:22.242 INFO:teuthology.orchestra.run.smithi157.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-09T00:32:22.242 INFO:teuthology.orchestra.run.smithi157.stdout: librados2 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 3.3 M 2024-05-09T00:32:22.242 INFO:teuthology.orchestra.run.smithi157.stdout: librbd1 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 3.1 M 2024-05-09T00:32:22.242 INFO:teuthology.orchestra.run.smithi157.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-09T00:32:22.242 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-09T00:32:22.242 INFO:teuthology.orchestra.run.smithi157.stdout:Installing dependencies: 2024-05-09T00:32:22.242 INFO:teuthology.orchestra.run.smithi157.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-09T00:32:22.242 INFO:teuthology.orchestra.run.smithi157.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-09T00:32:22.242 INFO:teuthology.orchestra.run.smithi157.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-09T00:32:22.243 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-common x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 20 M 2024-05-09T00:32:22.243 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 34 k 2024-05-09T00:32:22.243 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mds x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 2.3 M 2024-05-09T00:32:22.243 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 274 k 2024-05-09T00:32:22.243 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mon x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 4.9 M 2024-05-09T00:32:22.243 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-osd x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 18 M 2024-05-09T00:32:22.243 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 16 k 2024-05-09T00:32:22.243 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-selinux x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 25 k 2024-05-09T00:32:22.243 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-09T00:32:22.243 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-09T00:32:22.243 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-09T00:32:22.243 INFO:teuthology.orchestra.run.smithi157.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-09T00:32:22.244 INFO:teuthology.orchestra.run.smithi157.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-09T00:32:22.244 INFO:teuthology.orchestra.run.smithi157.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-09T00:32:22.244 INFO:teuthology.orchestra.run.smithi157.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-09T00:32:22.244 INFO:teuthology.orchestra.run.smithi157.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-09T00:32:22.244 INFO:teuthology.orchestra.run.smithi157.stdout: libcephsqlite x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 158 k 2024-05-09T00:32:22.244 INFO:teuthology.orchestra.run.smithi157.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-09T00:32:22.244 INFO:teuthology.orchestra.run.smithi157.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-09T00:32:22.244 INFO:teuthology.orchestra.run.smithi157.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-09T00:32:22.244 INFO:teuthology.orchestra.run.smithi157.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-09T00:32:22.244 INFO:teuthology.orchestra.run.smithi157.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-09T00:32:22.244 INFO:teuthology.orchestra.run.smithi157.stdout: libradosstriper1 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 497 k 2024-05-09T00:32:22.244 INFO:teuthology.orchestra.run.smithi157.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-09T00:32:22.244 INFO:teuthology.orchestra.run.smithi157.stdout: librgw2 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 5.2 M 2024-05-09T00:32:22.244 INFO:teuthology.orchestra.run.smithi157.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-09T00:32:22.245 INFO:teuthology.orchestra.run.smithi157.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-09T00:32:22.245 INFO:teuthology.orchestra.run.smithi157.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-09T00:32:22.245 INFO:teuthology.orchestra.run.smithi157.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-09T00:32:22.245 INFO:teuthology.orchestra.run.smithi157.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-09T00:32:22.245 INFO:teuthology.orchestra.run.smithi157.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-09T00:32:22.245 INFO:teuthology.orchestra.run.smithi157.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-09T00:32:22.245 INFO:teuthology.orchestra.run.smithi157.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-09T00:32:22.245 INFO:teuthology.orchestra.run.smithi157.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-09T00:32:22.245 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-09T00:32:22.245 INFO:teuthology.orchestra.run.smithi157.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-09T00:32:22.245 INFO:teuthology.orchestra.run.smithi157.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-09T00:32:22.245 INFO:teuthology.orchestra.run.smithi157.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-09T00:32:22.245 INFO:teuthology.orchestra.run.smithi157.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-09T00:32:22.245 INFO:teuthology.orchestra.run.smithi157.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-09T00:32:22.246 INFO:teuthology.orchestra.run.smithi157.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-09T00:32:22.246 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-09T00:32:22.246 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-argparse x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 45 k 2024-05-09T00:32:22.246 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-common x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 135 k 2024-05-09T00:32:22.246 INFO:teuthology.orchestra.run.smithi157.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-09T00:32:22.246 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-09T00:32:22.246 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-09T00:32:22.246 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-09T00:32:22.246 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-09T00:32:22.246 INFO:teuthology.orchestra.run.smithi157.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-09T00:32:22.246 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-09T00:32:22.246 INFO:teuthology.orchestra.run.smithi157.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-09T00:32:22.247 INFO:teuthology.orchestra.run.smithi157.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-09T00:32:22.247 INFO:teuthology.orchestra.run.smithi157.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-09T00:32:22.247 INFO:teuthology.orchestra.run.smithi157.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-09T00:32:22.247 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-09T00:32:22.247 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-09T00:32:22.247 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-09T00:32:22.247 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-09T00:32:22.247 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-09T00:32:22.247 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-09T00:32:22.247 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-09T00:32:22.247 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-09T00:32:22.247 INFO:teuthology.orchestra.run.smithi157.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-09T00:32:22.247 INFO:teuthology.orchestra.run.smithi157.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-09T00:32:22.248 INFO:teuthology.orchestra.run.smithi157.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-09T00:32:22.248 INFO:teuthology.orchestra.run.smithi157.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-09T00:32:22.248 INFO:teuthology.orchestra.run.smithi157.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-09T00:32:22.248 INFO:teuthology.orchestra.run.smithi157.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-09T00:32:22.248 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-09T00:32:22.248 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-09T00:32:22.248 INFO:teuthology.orchestra.run.smithi157.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-09T00:32:22.248 INFO:teuthology.orchestra.run.smithi157.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-09T00:32:22.248 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-09T00:32:22.248 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-09T00:32:22.248 INFO:teuthology.orchestra.run.smithi157.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-09T00:32:22.248 INFO:teuthology.orchestra.run.smithi157.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-09T00:32:22.248 INFO:teuthology.orchestra.run.smithi157.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-09T00:32:22.249 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-09T00:32:22.249 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-09T00:32:22.249 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-09T00:32:22.249 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-09T00:32:22.249 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-09T00:32:22.249 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-09T00:32:22.249 INFO:teuthology.orchestra.run.smithi157.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-09T00:32:22.249 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-09T00:32:22.249 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-09T00:32:22.249 INFO:teuthology.orchestra.run.smithi157.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-09T00:32:22.249 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-09T00:32:22.249 INFO:teuthology.orchestra.run.smithi157.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-09T00:32:22.249 INFO:teuthology.orchestra.run.smithi157.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-09T00:32:22.250 INFO:teuthology.orchestra.run.smithi157.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-09T00:32:22.250 INFO:teuthology.orchestra.run.smithi157.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-09T00:32:22.250 INFO:teuthology.orchestra.run.smithi157.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-09T00:32:22.250 INFO:teuthology.orchestra.run.smithi157.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-09T00:32:22.250 INFO:teuthology.orchestra.run.smithi157.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-09T00:32:22.250 INFO:teuthology.orchestra.run.smithi157.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-09T00:32:22.250 INFO:teuthology.orchestra.run.smithi157.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-09T00:32:22.250 INFO:teuthology.orchestra.run.smithi157.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-09T00:32:22.250 INFO:teuthology.orchestra.run.smithi157.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-09T00:32:22.250 INFO:teuthology.orchestra.run.smithi157.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-09T00:32:22.250 INFO:teuthology.orchestra.run.smithi157.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-09T00:32:22.250 INFO:teuthology.orchestra.run.smithi157.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-09T00:32:22.250 INFO:teuthology.orchestra.run.smithi157.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-09T00:32:22.251 INFO:teuthology.orchestra.run.smithi157.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-09T00:32:22.251 INFO:teuthology.orchestra.run.smithi157.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-09T00:32:22.251 INFO:teuthology.orchestra.run.smithi157.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-09T00:32:22.251 INFO:teuthology.orchestra.run.smithi157.stdout:Installing weak dependencies: 2024-05-09T00:32:22.251 INFO:teuthology.orchestra.run.smithi157.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-09T00:32:22.251 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-05-09T00:32:22.251 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-05-09T00:32:22.251 INFO:teuthology.orchestra.run.smithi157.stdout:======================================================================================= 2024-05-09T00:32:22.251 INFO:teuthology.orchestra.run.smithi157.stdout:Install 148 Packages 2024-05-09T00:32:22.251 INFO:teuthology.orchestra.run.smithi157.stdout:Upgrade 8 Packages 2024-05-09T00:32:22.251 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-05-09T00:32:22.252 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 219 M 2024-05-09T00:32:22.252 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-05-09T00:32:22.400 INFO:teuthology.orchestra.run.smithi005.stdout:(2/156): ceph-base-19.0.0-3594.gc9da4049.el9.x8 10 MB/s | 5.4 MB 00:00 2024-05-09T00:32:22.451 INFO:teuthology.orchestra.run.smithi005.stdout:(3/156): ceph-immutable-object-cache-19.0.0-359 2.9 MB/s | 147 kB 00:00 2024-05-09T00:32:22.593 INFO:teuthology.orchestra.run.smithi005.stdout:(4/156): ceph-mds-19.0.0-3594.gc9da4049.el9.x86 16 MB/s | 2.3 MB 00:00 2024-05-09T00:32:22.693 INFO:teuthology.orchestra.run.smithi005.stdout:(5/156): ceph-mgr-19.0.0-3594.gc9da4049.el9.x86 15 MB/s | 1.5 MB 00:00 2024-05-09T00:32:22.726 INFO:teuthology.orchestra.run.smithi005.stdout:(6/156): ceph-fuse-19.0.0-3594.gc9da4049.el9.x8 1.4 MB/s | 863 kB 00:00 2024-05-09T00:32:22.799 INFO:teuthology.orchestra.run.smithi157.stdout:(1/156): ceph-19.0.0-3594.gc9da4049.el9.x86_64. 21 kB/s | 6.6 kB 00:00 2024-05-09T00:32:23.010 INFO:teuthology.orchestra.run.smithi005.stdout:(7/156): ceph-mon-19.0.0-3594.gc9da4049.el9.x86 15 MB/s | 4.9 MB 00:00 2024-05-09T00:32:23.594 INFO:teuthology.orchestra.run.smithi005.stdout:(8/156): ceph-radosgw-19.0.0-3594.gc9da4049.el9 17 MB/s | 10 MB 00:00 2024-05-09T00:32:23.644 INFO:teuthology.orchestra.run.smithi005.stdout:(9/156): ceph-selinux-19.0.0-3594.gc9da4049.el9 507 kB/s | 25 kB 00:00 2024-05-09T00:32:23.783 INFO:teuthology.orchestra.run.smithi157.stdout:(2/156): ceph-fuse-19.0.0-3594.gc9da4049.el9.x8 878 kB/s | 863 kB 00:00 2024-05-09T00:32:24.008 INFO:teuthology.orchestra.run.smithi157.stdout:(3/156): ceph-immutable-object-cache-19.0.0-359 653 kB/s | 147 kB 00:00 2024-05-09T00:32:26.008 INFO:teuthology.orchestra.run.smithi157.stdout:(4/156): ceph-mds-19.0.0-3594.gc9da4049.el9.x86 1.1 MB/s | 2.3 MB 00:01 2024-05-09T00:32:26.136 INFO:teuthology.orchestra.run.smithi005.stdout:(10/156): ceph-test-19.0.0-3594.gc9da4049.el9.x 17 MB/s | 43 MB 00:02 2024-05-09T00:32:26.186 INFO:teuthology.orchestra.run.smithi005.stdout:(11/156): cephfs-mirror-19.0.0-3594.gc9da4049.e 4.4 MB/s | 224 kB 00:00 2024-05-09T00:32:26.236 INFO:teuthology.orchestra.run.smithi005.stdout:(12/156): cephfs-shell-19.0.0-3594.gc9da4049.el 541 kB/s | 27 kB 00:00 2024-05-09T00:32:26.286 INFO:teuthology.orchestra.run.smithi005.stdout:(13/156): libcephfs-devel-19.0.0-3594.gc9da4049 637 kB/s | 32 kB 00:00 2024-05-09T00:32:26.370 INFO:teuthology.orchestra.run.smithi005.stdout:(14/156): libcephfs2-19.0.0-3594.gc9da4049.el9. 8.9 MB/s | 756 kB 00:00 2024-05-09T00:32:26.420 INFO:teuthology.orchestra.run.smithi005.stdout:(15/156): libcephsqlite-19.0.0-3594.gc9da4049.e 3.1 MB/s | 158 kB 00:00 2024-05-09T00:32:26.479 INFO:teuthology.orchestra.run.smithi005.stdout:(16/156): librados-devel-19.0.0-3594.gc9da4049. 2.1 MB/s | 127 kB 00:00 2024-05-09T00:32:26.554 INFO:teuthology.orchestra.run.smithi005.stdout:(17/156): libradosstriper1-19.0.0-3594.gc9da404 6.5 MB/s | 497 kB 00:00 2024-05-09T00:32:26.608 INFO:teuthology.orchestra.run.smithi157.stdout:(5/156): ceph-mgr-19.0.0-3594.gc9da4049.el9.x86 2.5 MB/s | 1.5 MB 00:00 2024-05-09T00:32:26.888 INFO:teuthology.orchestra.run.smithi005.stdout:(18/156): librgw2-19.0.0-3594.gc9da4049.el9.x86 15 MB/s | 5.2 MB 00:00 2024-05-09T00:32:26.938 INFO:teuthology.orchestra.run.smithi005.stdout:(19/156): python3-ceph-argparse-19.0.0-3594.gc9 906 kB/s | 45 kB 00:00 2024-05-09T00:32:26.997 INFO:teuthology.orchestra.run.smithi005.stdout:(20/156): python3-ceph-common-19.0.0-3594.gc9da 2.3 MB/s | 135 kB 00:00 2024-05-09T00:32:27.064 INFO:teuthology.orchestra.run.smithi005.stdout:(21/156): python3-cephfs-19.0.0-3594.gc9da4049. 2.4 MB/s | 161 kB 00:00 2024-05-09T00:32:27.131 INFO:teuthology.orchestra.run.smithi005.stdout:(22/156): python3-rados-19.0.0-3594.gc9da4049.e 4.7 MB/s | 322 kB 00:00 2024-05-09T00:32:27.189 INFO:teuthology.orchestra.run.smithi005.stdout:(23/156): python3-rbd-19.0.0-3594.gc9da4049.el9 5.0 MB/s | 300 kB 00:00 2024-05-09T00:32:27.248 INFO:teuthology.orchestra.run.smithi005.stdout:(24/156): python3-rgw-19.0.0-3594.gc9da4049.el9 1.7 MB/s | 99 kB 00:00 2024-05-09T00:32:27.306 INFO:teuthology.orchestra.run.smithi005.stdout:(25/156): rbd-fuse-19.0.0-3594.gc9da4049.el9.x8 1.4 MB/s | 86 kB 00:00 2024-05-09T00:32:27.316 INFO:teuthology.orchestra.run.smithi157.stdout:(6/156): ceph-base-19.0.0-3594.gc9da4049.el9.x8 1.1 MB/s | 5.4 MB 00:04 2024-05-09T00:32:27.384 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-05-09T00:32:27.501 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-09T00:32:27.540 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-09T00:32:27.540 INFO:teuthology.orchestra.run.smithi005.stdout:(26/156): rbd-mirror-19.0.0-3594.gc9da4049.el9. 13 MB/s | 3.0 MB 00:00 2024-05-09T00:32:27.594 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-09T00:32:27.627 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-09T00:32:27.653 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-09T00:32:27.704 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-09T00:32:27.741 INFO:teuthology.orchestra.run.smithi005.stdout:(27/156): ceph-common-19.0.0-3594.gc9da4049.el9 3.3 MB/s | 20 MB 00:05 2024-05-09T00:32:27.766 INFO:teuthology.orchestra.run.smithi005.stdout:(28/156): rbd-nbd-19.0.0-3594.gc9da4049.el9.x86 751 kB/s | 169 kB 00:00 2024-05-09T00:32:27.800 INFO:teuthology.orchestra.run.smithi005.stdout:(29/156): ceph-grafana-dashboards-19.0.0-3594.g 588 kB/s | 34 kB 00:00 2024-05-09T00:32:27.825 INFO:teuthology.orchestra.run.smithi005.stdout:(30/156): ceph-mgr-cephadm-19.0.0-3594.gc9da404 2.4 MB/s | 142 kB 00:00 2024-05-09T00:32:28.033 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-09T00:32:28.146 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 8/164 2024-05-09T00:32:28.270 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 8/164 2024-05-09T00:32:28.305 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 9/164 2024-05-09T00:32:28.334 INFO:teuthology.orchestra.run.smithi005.stdout:(31/156): ceph-mgr-diskprediction-local-19.0.0- 15 MB/s | 7.4 MB 00:00 2024-05-09T00:32:28.389 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 9/164 2024-05-09T00:32:28.400 INFO:teuthology.orchestra.run.smithi005.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3594.gc9 4.0 MB/s | 274 kB 00:00 2024-05-09T00:32:28.418 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-rados-2:19.0.0-3594.gc9da4049.el9.x86_64 10/164 2024-05-09T00:32:28.451 INFO:teuthology.orchestra.run.smithi005.stdout:(33/156): ceph-mgr-rook-19.0.0-3594.gc9da4049.e 992 kB/s | 49 kB 00:00 2024-05-09T00:32:28.492 INFO:teuthology.orchestra.run.smithi005.stdout:(34/156): ceph-prometheus-alerts-19.0.0-3594.gc 396 kB/s | 16 kB 00:00 2024-05-09T00:32:28.551 INFO:teuthology.orchestra.run.smithi005.stdout:(35/156): ceph-volume-19.0.0-3594.gc9da4049.el9 4.6 MB/s | 273 kB 00:00 2024-05-09T00:32:28.570 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-09T00:32:28.583 INFO:teuthology.orchestra.run.smithi157.stdout:(7/156): ceph-mon-19.0.0-3594.gc9da4049.el9.x86 2.5 MB/s | 4.9 MB 00:01 2024-05-09T00:32:28.624 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-09T00:32:28.635 INFO:teuthology.orchestra.run.smithi005.stdout:(36/156): cephadm-19.0.0-3594.gc9da4049.el9.noa 9.0 MB/s | 764 kB 00:00 2024-05-09T00:32:28.652 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 13/164 2024-05-09T00:32:28.694 INFO:teuthology.orchestra.run.smithi005.stdout:(37/156): ceph-mgr-dashboard-19.0.0-3594.gc9da4 4.0 MB/s | 3.5 MB 00:00 2024-05-09T00:32:28.710 INFO:teuthology.orchestra.run.smithi005.stdout:(38/156): cephfs-top-19.0.0-3594.gc9da4049.el9. 243 kB/s | 18 kB 00:00 2024-05-09T00:32:28.757 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 13/164 2024-05-09T00:32:28.786 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86 14/164 2024-05-09T00:32:28.850 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86 14/164 2024-05-09T00:32:28.871 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T00:32:28.896 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T00:32:29.019 INFO:teuthology.orchestra.run.smithi005.stdout:(39/156): ceph-osd-19.0.0-3594.gc9da4049.el9.x8 2.8 MB/s | 18 MB 00:06 2024-05-09T00:32:29.088 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T00:32:29.128 INFO:teuthology.orchestra.run.smithi005.stdout:(40/156): fuse-2.9.9-15.el9.x86_64.rpm 185 kB/s | 80 kB 00:00 2024-05-09T00:32:29.292 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T00:32:29.450 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T00:32:29.515 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T00:32:29.528 INFO:teuthology.orchestra.run.smithi005.stdout:(41/156): libquadmath-11.4.1-3.el9.x86_64.rpm 384 kB/s | 195 kB 00:00 2024-05-09T00:32:29.553 INFO:teuthology.orchestra.run.smithi005.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 595 kB/s | 253 kB 00:00 2024-05-09T00:32:29.587 INFO:teuthology.orchestra.run.smithi005.stdout:(43/156): libgfortran-11.4.1-3.el9.x86_64.rpm 917 kB/s | 803 kB 00:00 2024-05-09T00:32:29.619 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T00:32:29.654 INFO:teuthology.orchestra.run.smithi005.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.0 MB/s | 106 kB 00:00 2024-05-09T00:32:29.687 INFO:teuthology.orchestra.run.smithi005.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 135 kB 00:00 2024-05-09T00:32:29.700 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T00:32:29.754 INFO:teuthology.orchestra.run.smithi005.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.2 MB/s | 125 kB 00:00 2024-05-09T00:32:29.796 INFO:teuthology.orchestra.run.smithi005.stdout:(47/156): python3-urllib3-1.26.5-5.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-05-09T00:32:29.895 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T00:32:29.921 INFO:teuthology.orchestra.run.smithi005.stdout:(48/156): python3-cryptography-36.0.1-4.el9.x86 3.1 MB/s | 1.2 MB 00:00 2024-05-09T00:32:30.007 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-09T00:32:30.030 INFO:teuthology.orchestra.run.smithi005.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 2.4 MB/s | 689 kB 00:00 2024-05-09T00:32:30.038 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-09T00:32:30.055 INFO:teuthology.orchestra.run.smithi005.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 800 kB/s | 107 kB 00:00 2024-05-09T00:32:30.072 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-09T00:32:30.097 INFO:teuthology.orchestra.run.smithi005.stdout:(51/156): bison-3.7.4-5.el9.x86_64.rpm 3.1 MB/s | 943 kB 00:00 2024-05-09T00:32:30.131 INFO:teuthology.orchestra.run.smithi005.stdout:(52/156): elfutils-libelf-devel-0.190-2.el9.x86 214 kB/s | 21 kB 00:00 2024-05-09T00:32:30.137 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-09T00:32:30.164 INFO:teuthology.orchestra.run.smithi005.stdout:(53/156): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-05-09T00:32:30.172 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-09T00:32:30.198 INFO:teuthology.orchestra.run.smithi005.stdout:(54/156): flexiblas-3.0.4-8.el9.x86_64.rpm 325 kB/s | 33 kB 00:00 2024-05-09T00:32:30.224 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-09T00:32:30.257 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-09T00:32:30.265 INFO:teuthology.orchestra.run.smithi005.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 181 kB/s | 18 kB 00:00 2024-05-09T00:32:30.294 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-09T00:32:30.299 INFO:teuthology.orchestra.run.smithi005.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 111 kB/s | 11 kB 00:00 2024-05-09T00:32:30.364 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-09T00:32:30.374 INFO:teuthology.orchestra.run.smithi005.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 160 kB 00:00 2024-05-09T00:32:30.408 INFO:teuthology.orchestra.run.smithi005.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 417 kB/s | 45 kB 00:00 2024-05-09T00:32:30.415 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-09T00:32:30.446 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-09T00:32:30.482 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-09T00:32:30.491 INFO:teuthology.orchestra.run.smithi005.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 5.5 MB/s | 662 kB 00:00 2024-05-09T00:32:30.525 INFO:teuthology.orchestra.run.smithi005.stdout:(60/156): libtool-2.4.6-45.el9.x86_64.rpm 4.9 MB/s | 582 kB 00:00 2024-05-09T00:32:30.558 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-09T00:32:30.575 INFO:teuthology.orchestra.run.smithi005.stdout:(61/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.7 MB/s | 3.0 MB 00:00 2024-05-09T00:32:30.592 INFO:teuthology.orchestra.run.smithi005.stdout:(62/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 253 kB/s | 25 kB 00:00 2024-05-09T00:32:30.593 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-ceph-common-2:19.0.0-3594.gc9da4049.el9. 32/164 2024-05-09T00:32:30.626 INFO:teuthology.orchestra.run.smithi005.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 473 kB/s | 47 kB 00:00 2024-05-09T00:32:30.637 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-ceph-argparse-2:19.0.0-3594.gc9da4049.el 33/164 2024-05-09T00:32:30.657 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cephfs-2:19.0.0-3594.gc9da4049.el9.x86_6 34/164 2024-05-09T00:32:30.684 INFO:teuthology.orchestra.run.smithi005.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-05-09T00:32:30.710 INFO:teuthology.orchestra.run.smithi005.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-05-09T00:32:30.727 INFO:teuthology.orchestra.run.smithi005.stdout:(66/156): openblas-0.3.21-2.el9.x86_64.rpm 350 kB/s | 35 kB 00:00 2024-05-09T00:32:30.727 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-09T00:32:30.771 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-09T00:32:30.846 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-09T00:32:30.982 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-09T00:32:31.019 INFO:teuthology.orchestra.run.smithi005.stdout:(67/156): postgresql-13.14-1.el9.x86_64.rpm 5.4 MB/s | 1.6 MB 00:00 2024-05-09T00:32:31.033 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-09T00:32:31.160 INFO:teuthology.orchestra.run.smithi005.stdout:(68/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 9.8 MB/s | 4.4 MB 00:00 2024-05-09T00:32:31.444 INFO:teuthology.orchestra.run.smithi005.stdout:(69/156): postgresql-private-libs-13.14-1.el9.x 470 kB/s | 133 kB 00:00 2024-05-09T00:32:31.565 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-09T00:32:31.646 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-09T00:32:31.675 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-09T00:32:31.717 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-09T00:32:31.755 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : librados-devel-2:19.0.0-3594.gc9da4049.el9.x86_6 44/164 2024-05-09T00:32:31.797 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-09T00:32:31.958 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-09T00:32:32.018 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-09T00:32:32.109 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-09T00:32:32.181 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-09T00:32:32.254 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-09T00:32:32.286 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-09T00:32:32.286 INFO:teuthology.orchestra.run.smithi005.stdout:(70/156): postgresql-contrib-13.14-1.el9.x86_64 719 kB/s | 911 kB 00:01 2024-05-09T00:32:32.322 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-09T00:32:32.369 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-09T00:32:32.405 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-09T00:32:32.447 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-09T00:32:32.490 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-09T00:32:32.626 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-09T00:32:32.671 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-09T00:32:32.701 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-09T00:32:32.742 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-09T00:32:32.837 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-09T00:32:32.876 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-09T00:32:32.916 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-09T00:32:33.101 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-09T00:32:33.117 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-09T00:32:33.194 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-09T00:32:33.327 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-09T00:32:33.358 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-09T00:32:33.386 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-09T00:32:33.429 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-09T00:32:33.468 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-09T00:32:33.499 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-09T00:32:33.529 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-09T00:32:33.573 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-09T00:32:33.606 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-09T00:32:33.648 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-09T00:32:33.653 INFO:teuthology.orchestra.run.smithi005.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 756 kB/s | 1.0 MB 00:01 2024-05-09T00:32:33.682 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-09T00:32:33.699 INFO:teuthology.orchestra.run.smithi157.stdout:(8/156): ceph-radosgw-19.0.0-3594.gc9da4049.el9 2.0 MB/s | 10 MB 00:05 2024-05-09T00:32:33.769 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-09T00:32:33.812 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-09T00:32:33.816 INFO:teuthology.orchestra.run.smithi157.stdout:(9/156): ceph-selinux-19.0.0-3594.gc9da4049.el9 216 kB/s | 25 kB 00:00 2024-05-09T00:32:33.844 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-09T00:32:33.851 INFO:teuthology.orchestra.run.smithi005.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 456 kB/s | 87 kB 00:00 2024-05-09T00:32:33.891 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-09T00:32:33.989 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-09T00:32:34.008 INFO:teuthology.orchestra.run.smithi157.stdout:(10/156): ceph-common-19.0.0-3594.gc9da4049.el9 1.7 MB/s | 20 MB 00:11 2024-05-09T00:32:34.037 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-09T00:32:34.109 INFO:teuthology.orchestra.run.smithi157.stdout:(11/156): cephfs-mirror-19.0.0-3594.gc9da4049.e 2.2 MB/s | 224 kB 00:00 2024-05-09T00:32:34.167 INFO:teuthology.orchestra.run.smithi157.stdout:(12/156): cephfs-shell-19.0.0-3594.gc9da4049.el 464 kB/s | 27 kB 00:00 2024-05-09T00:32:34.226 INFO:teuthology.orchestra.run.smithi157.stdout:(13/156): libcephfs-devel-19.0.0-3594.gc9da4049 544 kB/s | 32 kB 00:00 2024-05-09T00:32:34.476 INFO:teuthology.orchestra.run.smithi157.stdout:(14/156): libcephfs2-19.0.0-3594.gc9da4049.el9. 3.0 MB/s | 756 kB 00:00 2024-05-09T00:32:34.601 INFO:teuthology.orchestra.run.smithi157.stdout:(15/156): libcephsqlite-19.0.0-3594.gc9da4049.e 1.2 MB/s | 158 kB 00:00 2024-05-09T00:32:34.710 INFO:teuthology.orchestra.run.smithi157.stdout:(16/156): librados-devel-19.0.0-3594.gc9da4049. 1.1 MB/s | 127 kB 00:00 2024-05-09T00:32:34.774 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-09T00:32:34.820 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-09T00:32:34.853 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-09T00:32:34.935 INFO:teuthology.orchestra.run.smithi157.stdout:(17/156): libradosstriper1-19.0.0-3594.gc9da404 2.2 MB/s | 497 kB 00:00 2024-05-09T00:32:34.954 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-09T00:32:35.011 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-09T00:32:35.019 INFO:teuthology.orchestra.run.smithi005.stdout:(73/156): openblas-openmp-0.3.21-2.el9.x86_64.r 1.1 MB/s | 4.7 MB 00:04 2024-05-09T00:32:35.099 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-09T00:32:35.138 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-09T00:32:35.315 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-09T00:32:35.345 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-09T00:32:35.362 INFO:teuthology.orchestra.run.smithi005.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 715 kB/s | 244 kB 00:00 2024-05-09T00:32:35.422 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-09T00:32:35.703 INFO:teuthology.orchestra.run.smithi005.stdout:(75/156): python3-jinja2-2.11.3-5.el9.noarch.rp 725 kB/s | 248 kB 00:00 2024-05-09T00:32:35.804 INFO:teuthology.orchestra.run.smithi005.stdout:(76/156): python3-jsonpatch-1.21-16.el9.noarch. 256 kB/s | 26 kB 00:00 2024-05-09T00:32:35.904 INFO:teuthology.orchestra.run.smithi005.stdout:(77/156): python3-jsonpointer-2.0-4.el9.noarch. 192 kB/s | 19 kB 00:00 2024-05-09T00:32:36.072 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-09T00:32:36.162 INFO:teuthology.orchestra.run.smithi005.stdout:(78/156): python3-mako-1.1.4-6.el9.noarch.rpm 666 kB/s | 172 kB 00:00 2024-05-09T00:32:36.185 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-09T00:32:36.271 INFO:teuthology.orchestra.run.smithi005.stdout:(79/156): python3-markupsafe-1.1.1-12.el9.x86_6 321 kB/s | 35 kB 00:00 2024-05-09T00:32:36.360 INFO:teuthology.orchestra.run.smithi157.stdout:(18/156): librgw2-19.0.0-3594.gc9da4049.el9.x86 3.6 MB/s | 5.2 MB 00:01 2024-05-09T00:32:36.419 INFO:teuthology.orchestra.run.smithi157.stdout:(19/156): python3-ceph-argparse-19.0.0-3594.gc9 778 kB/s | 45 kB 00:00 2024-05-09T00:32:36.486 INFO:teuthology.orchestra.run.smithi157.stdout:(20/156): python3-ceph-common-19.0.0-3594.gc9da 2.0 MB/s | 135 kB 00:00 2024-05-09T00:32:36.553 INFO:teuthology.orchestra.run.smithi157.stdout:(21/156): python3-cephfs-19.0.0-3594.gc9da4049. 2.4 MB/s | 161 kB 00:00 2024-05-09T00:32:36.670 INFO:teuthology.orchestra.run.smithi157.stdout:(22/156): python3-rados-19.0.0-3594.gc9da4049.e 2.7 MB/s | 322 kB 00:00 2024-05-09T00:32:36.770 INFO:teuthology.orchestra.run.smithi157.stdout:(23/156): python3-rbd-19.0.0-3594.gc9da4049.el9 2.9 MB/s | 300 kB 00:00 2024-05-09T00:32:36.829 INFO:teuthology.orchestra.run.smithi157.stdout:(24/156): python3-rgw-19.0.0-3594.gc9da4049.el9 1.7 MB/s | 99 kB 00:00 2024-05-09T00:32:36.896 INFO:teuthology.orchestra.run.smithi157.stdout:(25/156): rbd-fuse-19.0.0-3594.gc9da4049.el9.x8 1.3 MB/s | 86 kB 00:00 2024-05-09T00:32:37.063 INFO:teuthology.orchestra.run.smithi157.stdout:(26/156): ceph-osd-19.0.0-3594.gc9da4049.el9.x8 1.8 MB/s | 18 MB 00:09 2024-05-09T00:32:37.163 INFO:teuthology.orchestra.run.smithi157.stdout:(27/156): rbd-nbd-19.0.0-3594.gc9da4049.el9.x86 1.7 MB/s | 169 kB 00:00 2024-05-09T00:32:37.213 INFO:teuthology.orchestra.run.smithi157.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3594.g 687 kB/s | 34 kB 00:00 2024-05-09T00:32:37.297 INFO:teuthology.orchestra.run.smithi157.stdout:(29/156): ceph-mgr-cephadm-19.0.0-3594.gc9da404 1.7 MB/s | 142 kB 00:00 2024-05-09T00:32:37.427 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-09T00:32:37.507 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-09T00:32:37.526 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-09T00:32:37.559 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-09T00:32:37.706 INFO:teuthology.orchestra.run.smithi157.stdout:(30/156): rbd-mirror-19.0.0-3594.gc9da4049.el9. 3.8 MB/s | 3.0 MB 00:00 2024-05-09T00:32:37.879 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-09T00:32:37.922 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 101/164 2024-05-09T00:32:38.012 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 101/164 2024-05-09T00:32:38.123 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-rbd-2:19.0.0-3594.gc9da4049.el9.x86_64 102/164 2024-05-09T00:32:38.180 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-09T00:32:38.206 INFO:teuthology.orchestra.run.smithi157.stdout:(31/156): ceph-mgr-dashboard-19.0.0-3594.gc9da4 3.9 MB/s | 3.5 MB 00:00 2024-05-09T00:32:38.314 INFO:teuthology.orchestra.run.smithi157.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3594.gc9 2.5 MB/s | 274 kB 00:00 2024-05-09T00:32:38.390 INFO:teuthology.orchestra.run.smithi157.stdout:(33/156): ceph-mgr-rook-19.0.0-3594.gc9da4049.e 658 kB/s | 49 kB 00:00 2024-05-09T00:32:38.457 INFO:teuthology.orchestra.run.smithi157.stdout:(34/156): ceph-prometheus-alerts-19.0.0-3594.gc 247 kB/s | 16 kB 00:00 2024-05-09T00:32:38.565 INFO:teuthology.orchestra.run.smithi157.stdout:(35/156): ceph-volume-19.0.0-3594.gc9da4049.el9 2.5 MB/s | 273 kB 00:00 2024-05-09T00:32:38.666 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-09T00:32:38.692 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 105/164 2024-05-09T00:32:38.754 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 105/164 2024-05-09T00:32:38.774 INFO:teuthology.orchestra.run.smithi157.stdout:(36/156): cephadm-19.0.0-3594.gc9da4049.el9.noa 3.6 MB/s | 764 kB 00:00 2024-05-09T00:32:38.776 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-rgw-2:19.0.0-3594.gc9da4049.el9.x86_64 106/164 2024-05-09T00:32:38.841 INFO:teuthology.orchestra.run.smithi157.stdout:(37/156): cephfs-top-19.0.0-3594.gc9da4049.el9. 274 kB/s | 18 kB 00:00 2024-05-09T00:32:39.025 INFO:teuthology.orchestra.run.smithi157.stdout:(38/156): fuse-2.9.9-15.el9.x86_64.rpm 438 kB/s | 80 kB 00:00 2024-05-09T00:32:39.201 INFO:teuthology.orchestra.run.smithi157.stdout:(39/156): libgfortran-11.4.1-3.el9.x86_64.rpm 4.5 MB/s | 803 kB 00:00 2024-05-09T00:32:39.238 INFO:teuthology.orchestra.run.smithi005.stdout:(80/156): postgresql-server-13.14-1.el9.x86_64. 759 kB/s | 5.8 MB 00:07 2024-05-09T00:32:39.258 INFO:teuthology.orchestra.run.smithi157.stdout:(40/156): libquadmath-11.4.1-3.el9.x86_64.rpm 3.3 MB/s | 195 kB 00:00 2024-05-09T00:32:39.317 INFO:teuthology.orchestra.run.smithi157.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.2 MB/s | 253 kB 00:00 2024-05-09T00:32:39.409 INFO:teuthology.orchestra.run.smithi157.stdout:(42/156): ceph-mgr-diskprediction-local-19.0.0- 4.3 MB/s | 7.4 MB 00:01 2024-05-09T00:32:39.460 INFO:teuthology.orchestra.run.smithi157.stdout:(43/156): python3-cryptography-36.0.1-4.el9.x86 8.6 MB/s | 1.2 MB 00:00 2024-05-09T00:32:39.519 INFO:teuthology.orchestra.run.smithi157.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-05-09T00:32:39.569 INFO:teuthology.orchestra.run.smithi157.stdout:(45/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-05-09T00:32:39.580 INFO:teuthology.orchestra.run.smithi005.stdout:(81/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 734 kB/s | 250 kB 00:00 2024-05-09T00:32:39.610 INFO:teuthology.orchestra.run.smithi157.stdout:(46/156): python3-ply-3.11-14.el9.noarch.rpm 531 kB/s | 106 kB 00:00 2024-05-09T00:32:39.636 INFO:teuthology.orchestra.run.smithi157.stdout:(47/156): python3-urllib3-1.26.5-5.el9.noarch.r 3.1 MB/s | 215 kB 00:00 2024-05-09T00:32:39.811 INFO:teuthology.orchestra.run.smithi157.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 3.4 MB/s | 689 kB 00:00 2024-05-09T00:32:39.853 INFO:teuthology.orchestra.run.smithi157.stdout:(49/156): bison-3.7.4-5.el9.x86_64.rpm 4.2 MB/s | 943 kB 00:00 2024-05-09T00:32:39.895 INFO:teuthology.orchestra.run.smithi157.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 1.3 MB/s | 107 kB 00:00 2024-05-09T00:32:39.920 INFO:teuthology.orchestra.run.smithi157.stdout:(51/156): elfutils-libelf-devel-0.190-2.el9.x86 321 kB/s | 21 kB 00:00 2024-05-09T00:32:39.921 INFO:teuthology.orchestra.run.smithi005.stdout:(82/156): python3-oauthlib-3.1.1-5.el9.noarch.r 650 kB/s | 222 kB 00:00 2024-05-09T00:32:39.962 INFO:teuthology.orchestra.run.smithi157.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 4.6 MB/s | 313 kB 00:00 2024-05-09T00:32:39.996 INFO:teuthology.orchestra.run.smithi157.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 433 kB/s | 33 kB 00:00 2024-05-09T00:32:40.046 INFO:teuthology.orchestra.run.smithi157.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 363 kB/s | 18 kB 00:00 2024-05-09T00:32:40.096 INFO:teuthology.orchestra.run.smithi157.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 223 kB/s | 11 kB 00:00 2024-05-09T00:32:40.105 INFO:teuthology.orchestra.run.smithi005.stdout:(83/156): python3-packaging-20.9-5.el9.noarch.r 420 kB/s | 77 kB 00:00 2024-05-09T00:32:40.155 INFO:teuthology.orchestra.run.smithi157.stdout:(56/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-05-09T00:32:40.206 INFO:teuthology.orchestra.run.smithi005.stdout:(84/156): python3-prettytable-0.7.2-27.el9.noar 423 kB/s | 42 kB 00:00 2024-05-09T00:32:40.206 INFO:teuthology.orchestra.run.smithi157.stdout:(57/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 12 MB/s | 3.0 MB 00:00 2024-05-09T00:32:40.222 INFO:teuthology.orchestra.run.smithi157.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 677 kB/s | 45 kB 00:00 2024-05-09T00:32:40.289 INFO:teuthology.orchestra.run.smithi157.stdout:(59/156): libtool-2.4.6-45.el9.x86_64.rpm 8.5 MB/s | 582 kB 00:00 2024-05-09T00:32:40.331 INFO:teuthology.orchestra.run.smithi157.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 5.2 MB/s | 662 kB 00:00 2024-05-09T00:32:40.348 INFO:teuthology.orchestra.run.smithi157.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 436 kB/s | 25 kB 00:00 2024-05-09T00:32:40.374 INFO:teuthology.orchestra.run.smithi157.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-05-09T00:32:40.416 INFO:teuthology.orchestra.run.smithi157.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.3 MB/s | 292 kB 00:00 2024-05-09T00:32:40.441 INFO:teuthology.orchestra.run.smithi157.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 2.8 MB/s | 188 kB 00:00 2024-05-09T00:32:40.458 INFO:teuthology.orchestra.run.smithi157.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 842 kB/s | 35 kB 00:00 2024-05-09T00:32:40.600 INFO:teuthology.orchestra.run.smithi157.stdout:(66/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 31 MB/s | 4.4 MB 00:00 2024-05-09T00:32:40.640 INFO:teuthology.orchestra.run.smithi005.stdout:(85/156): python3-protobuf-3.14.0-13.el9.noarch 621 kB/s | 269 kB 00:00 2024-05-09T00:32:40.675 INFO:teuthology.orchestra.run.smithi157.stdout:(67/156): openblas-openmp-0.3.21-2.el9.x86_64.r 20 MB/s | 4.7 MB 00:00 2024-05-09T00:32:40.725 INFO:teuthology.orchestra.run.smithi157.stdout:(68/156): postgresql-13.14-1.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-05-09T00:32:40.759 INFO:teuthology.orchestra.run.smithi157.stdout:(69/156): postgresql-contrib-13.14-1.el9.x86_64 11 MB/s | 911 kB 00:00 2024-05-09T00:32:40.784 INFO:teuthology.orchestra.run.smithi157.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-05-09T00:32:40.876 INFO:teuthology.orchestra.run.smithi157.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 11 MB/s | 1.0 MB 00:00 2024-05-09T00:32:40.906 INFO:teuthology.orchestra.run.smithi005.stdout:(86/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 596 kB/s | 159 kB 00:00 2024-05-09T00:32:40.962 INFO:teuthology.orchestra.run.smithi157.stdout:(72/156): postgresql-server-13.14-1.el9.x86_64. 29 MB/s | 5.8 MB 00:00 2024-05-09T00:32:40.985 INFO:teuthology.orchestra.run.smithi157.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 805 kB/s | 87 kB 00:00 2024-05-09T00:32:41.036 INFO:teuthology.orchestra.run.smithi157.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 4.8 MB/s | 244 kB 00:00 2024-05-09T00:32:41.049 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 107/164 2024-05-09T00:32:41.086 INFO:teuthology.orchestra.run.smithi157.stdout:(75/156): python3-jinja2-2.11.3-5.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-05-09T00:32:41.090 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 107/164 2024-05-09T00:32:41.162 INFO:teuthology.orchestra.run.smithi157.stdout:(76/156): python3-babel-2.9.1-2.el9.noarch.rpm 30 MB/s | 6.0 MB 00:00 2024-05-09T00:32:41.186 INFO:teuthology.orchestra.run.smithi157.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 256 kB/s | 26 kB 00:00 2024-05-09T00:32:41.220 INFO:teuthology.orchestra.run.smithi157.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 328 kB/s | 19 kB 00:00 2024-05-09T00:32:41.246 INFO:teuthology.orchestra.run.smithi157.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-05-09T00:32:41.289 INFO:teuthology.orchestra.run.smithi157.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 517 kB/s | 35 kB 00:00 2024-05-09T00:32:41.332 INFO:teuthology.orchestra.run.smithi005.stdout:(87/156): python3-pyasn1-modules-0.4.8-6.el9.no 656 kB/s | 279 kB 00:00 2024-05-09T00:32:41.338 INFO:teuthology.orchestra.run.smithi157.stdout:(81/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.9 MB/s | 250 kB 00:00 2024-05-09T00:32:41.438 INFO:teuthology.orchestra.run.smithi157.stdout:(82/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-05-09T00:32:41.440 INFO:teuthology.orchestra.run.smithi005.stdout:(88/156): python3-pytz-2021.1-5.el9.noarch.rpm 471 kB/s | 51 kB 00:00 2024-05-09T00:32:41.490 INFO:teuthology.orchestra.run.smithi157.stdout:(83/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-05-09T00:32:41.531 INFO:teuthology.orchestra.run.smithi157.stdout:(84/156): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-05-09T00:32:41.541 INFO:teuthology.orchestra.run.smithi005.stdout:(89/156): python3-requests-oauthlib-1.3.0-12.el 537 kB/s | 54 kB 00:00 2024-05-09T00:32:41.631 INFO:teuthology.orchestra.run.smithi157.stdout:(85/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 13 MB/s | 5.1 MB 00:00 2024-05-09T00:32:41.647 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 107/164 2024-05-09T00:32:41.656 INFO:teuthology.orchestra.run.smithi157.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 2.1 MB/s | 269 kB 00:00 2024-05-09T00:32:41.682 INFO:teuthology.orchestra.run.smithi157.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-05-09T00:32:41.684 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 108/164 2024-05-09T00:32:41.707 INFO:teuthology.orchestra.run.smithi157.stdout:(88/156): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-05-09T00:32:41.724 INFO:teuthology.orchestra.run.smithi157.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-05-09T00:32:41.749 INFO:teuthology.orchestra.run.smithi005.stdout:(90/156): python3-babel-2.9.1-2.el9.noarch.rpm 772 kB/s | 6.0 MB 00:07 2024-05-09T00:32:41.757 INFO:teuthology.orchestra.run.smithi157.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-05-09T00:32:41.770 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 108/164 2024-05-09T00:32:41.770 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-09T00:32:41.771 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-09T00:32:41.771 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-09T00:32:41.799 INFO:teuthology.orchestra.run.smithi157.stdout:(91/156): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-05-09T00:32:41.803 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 109/164 2024-05-09T00:32:41.858 INFO:teuthology.orchestra.run.smithi005.stdout:(91/156): python3-toml-0.10.2-6.el9.noarch.rpm 385 kB/s | 42 kB 00:00 2024-05-09T00:32:41.875 INFO:teuthology.orchestra.run.smithi157.stdout:(92/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 589 kB/s | 44 kB 00:00 2024-05-09T00:32:41.942 INFO:teuthology.orchestra.run.smithi157.stdout:(93/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.3 MB/s | 221 kB 00:00 2024-05-09T00:32:41.966 INFO:teuthology.orchestra.run.smithi005.stdout:(92/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 407 kB/s | 44 kB 00:00 2024-05-09T00:32:42.100 INFO:teuthology.orchestra.run.smithi157.stdout:(94/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 43 MB/s | 16 MB 00:00 2024-05-09T00:32:42.117 INFO:teuthology.orchestra.run.smithi157.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 205 kB/s | 36 kB 00:00 2024-05-09T00:32:42.151 INFO:teuthology.orchestra.run.smithi157.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-05-09T00:32:42.226 INFO:teuthology.orchestra.run.smithi157.stdout:(97/156): uuid-1.6.2-55.el9.x86_64.rpm 772 kB/s | 58 kB 00:00 2024-05-09T00:32:42.252 INFO:teuthology.orchestra.run.smithi157.stdout:(98/156): socat-1.7.4.1-5.el9.x86_64.rpm 2.2 MB/s | 305 kB 00:00 2024-05-09T00:32:42.293 INFO:teuthology.orchestra.run.smithi157.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.4 MB/s | 94 kB 00:00 2024-05-09T00:32:42.308 INFO:teuthology.orchestra.run.smithi005.stdout:(93/156): qatlib-24.02.0-1.el9.x86_64.rpm 645 kB/s | 221 kB 00:00 2024-05-09T00:32:42.344 INFO:teuthology.orchestra.run.smithi157.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 693 kB/s | 64 kB 00:00 2024-05-09T00:32:42.369 INFO:teuthology.orchestra.run.smithi157.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 596 kB/s | 45 kB 00:00 2024-05-09T00:32:42.400 INFO:teuthology.orchestra.run.smithi005.stdout:(94/156): qatlib-service-24.02.0-1.el9.x86_64.r 392 kB/s | 36 kB 00:00 2024-05-09T00:32:42.453 INFO:teuthology.orchestra.run.smithi157.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 206 kB/s | 22 kB 00:00 2024-05-09T00:32:42.501 INFO:teuthology.orchestra.run.smithi005.stdout:(95/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 473 kB/s | 47 kB 00:00 2024-05-09T00:32:42.720 INFO:teuthology.orchestra.run.smithi157.stdout:(103/156): protobuf-compiler-3.14.0-13.el9.x86_ 2.4 MB/s | 863 kB 00:00 2024-05-09T00:32:42.918 INFO:teuthology.orchestra.run.smithi005.stdout:(96/156): socat-1.7.4.1-5.el9.x86_64.rpm 732 kB/s | 305 kB 00:00 2024-05-09T00:32:43.001 INFO:teuthology.orchestra.run.smithi005.stdout:(97/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 776 kB/s | 5.1 MB 00:06 2024-05-09T00:32:43.018 INFO:teuthology.orchestra.run.smithi005.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 577 kB/s | 58 kB 00:00 2024-05-09T00:32:43.178 INFO:teuthology.orchestra.run.smithi157.stdout:(104/156): ceph-test-19.0.0-3594.gc9da4049.el9. 4.6 MB/s | 43 MB 00:09 2024-05-09T00:32:43.194 INFO:teuthology.orchestra.run.smithi005.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 488 kB/s | 94 kB 00:00 2024-05-09T00:32:43.210 INFO:teuthology.orchestra.run.smithi005.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 331 kB/s | 64 kB 00:00 2024-05-09T00:32:43.237 INFO:teuthology.orchestra.run.smithi157.stdout:(105/156): abseil-cpp-20211102.0-3.el9.x86_64.r 699 kB/s | 548 kB 00:00 2024-05-09T00:32:43.294 INFO:teuthology.orchestra.run.smithi005.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 447 kB/s | 45 kB 00:00 2024-05-09T00:32:43.311 INFO:teuthology.orchestra.run.smithi005.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 222 kB/s | 22 kB 00:00 2024-05-09T00:32:43.388 INFO:teuthology.orchestra.run.smithi157.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 105 kB/s | 19 kB 00:00 2024-05-09T00:32:43.430 INFO:teuthology.orchestra.run.smithi157.stdout:(107/156): gperftools-libs-2.9.1-2.el9.x86_64.r 435 kB/s | 309 kB 00:00 2024-05-09T00:32:43.455 INFO:teuthology.orchestra.run.smithi157.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 387 kB/s | 26 kB 00:00 2024-05-09T00:32:43.488 INFO:teuthology.orchestra.run.smithi157.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 830 kB/s | 49 kB 00:00 2024-05-09T00:32:43.547 INFO:teuthology.orchestra.run.smithi157.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 731 kB/s | 67 kB 00:00 2024-05-09T00:32:43.572 INFO:teuthology.orchestra.run.smithi157.stdout:(111/156): luarocks-3.9.2-1.el9.noarch.rpm 1.8 MB/s | 151 kB 00:00 2024-05-09T00:32:43.595 INFO:teuthology.orchestra.run.smithi005.stdout:(103/156): abseil-cpp-20211102.0-3.el9.x86_64.r 1.9 MB/s | 548 kB 00:00 2024-05-09T00:32:43.678 INFO:teuthology.orchestra.run.smithi005.stdout:(104/156): gperftools-libs-2.9.1-2.el9.x86_64.r 3.6 MB/s | 309 kB 00:00 2024-05-09T00:32:43.714 INFO:teuthology.orchestra.run.smithi157.stdout:(112/156): libarrow-9.0.0-11.el9.x86_64.rpm 9.8 MB/s | 4.4 MB 00:00 2024-05-09T00:32:43.746 INFO:teuthology.orchestra.run.smithi005.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 292 kB/s | 19 kB 00:00 2024-05-09T00:32:43.748 INFO:teuthology.orchestra.run.smithi157.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 3.1 MB/s | 548 kB 00:00 2024-05-09T00:32:43.773 INFO:teuthology.orchestra.run.smithi157.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 741 kB/s | 43 kB 00:00 2024-05-09T00:32:43.807 INFO:teuthology.orchestra.run.smithi157.stdout:(115/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.2 MB/s | 839 kB 00:00 2024-05-09T00:32:43.832 INFO:teuthology.orchestra.run.smithi157.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 255 kB/s | 15 kB 00:00 2024-05-09T00:32:43.857 INFO:teuthology.orchestra.run.smithi157.stdout:(117/156): python3-cachetools-4.2.4-1.el9.noarc 295 kB/s | 32 kB 00:00 2024-05-09T00:32:43.883 INFO:teuthology.orchestra.run.smithi157.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-05-09T00:32:43.933 INFO:teuthology.orchestra.run.smithi157.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-05-09T00:32:43.954 INFO:teuthology.orchestra.run.smithi005.stdout:(106/156): libarrow-9.0.0-11.el9.x86_64.rpm 21 MB/s | 4.4 MB 00:00 2024-05-09T00:32:43.958 INFO:teuthology.orchestra.run.smithi157.stdout:(120/156): python3-colorama-0.4.4-12.el9.noarch 468 kB/s | 35 kB 00:00 2024-05-09T00:32:43.984 INFO:teuthology.orchestra.run.smithi157.stdout:(121/156): python3-cherrypy-18.6.1-2.el9.noarch 2.3 MB/s | 358 kB 00:00 2024-05-09T00:32:44.009 INFO:teuthology.orchestra.run.smithi157.stdout:(122/156): python3-google-auth-2.29.0-1.el9.noa 2.7 MB/s | 206 kB 00:00 2024-05-09T00:32:44.021 INFO:teuthology.orchestra.run.smithi005.stdout:(107/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 391 kB/s | 26 kB 00:00 2024-05-09T00:32:44.034 INFO:teuthology.orchestra.run.smithi157.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-05-09T00:32:44.068 INFO:teuthology.orchestra.run.smithi157.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 182 kB/s | 11 kB 00:00 2024-05-09T00:32:44.088 INFO:teuthology.orchestra.run.smithi005.stdout:(108/156): liboath-2.6.7-2.el9.x86_64.rpm 727 kB/s | 49 kB 00:00 2024-05-09T00:32:44.093 INFO:teuthology.orchestra.run.smithi157.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 301 kB/s | 18 kB 00:00 2024-05-09T00:32:44.127 INFO:teuthology.orchestra.run.smithi157.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 394 kB/s | 23 kB 00:00 2024-05-09T00:32:44.152 INFO:teuthology.orchestra.run.smithi157.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 331 kB/s | 19 kB 00:00 2024-05-09T00:32:44.163 INFO:teuthology.orchestra.run.smithi005.stdout:(109/156): libunwind-1.6.2-1.el9.x86_64.rpm 899 kB/s | 67 kB 00:00 2024-05-09T00:32:44.194 INFO:teuthology.orchestra.run.smithi157.stdout:(128/156): python3-grpcio-1.46.7-10.el9.x86_64. 8.7 MB/s | 2.0 MB 00:00 2024-05-09T00:32:44.219 INFO:teuthology.orchestra.run.smithi157.stdout:(129/156): python3-jaraco-text-3.2.0-6.el9.noar 213 kB/s | 20 kB 00:00 2024-05-09T00:32:44.230 INFO:teuthology.orchestra.run.smithi005.stdout:(110/156): luarocks-3.9.2-1.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-05-09T00:32:44.253 INFO:teuthology.orchestra.run.smithi157.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 789 kB/s | 46 kB 00:00 2024-05-09T00:32:44.270 INFO:teuthology.orchestra.run.smithi157.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-05-09T00:32:44.312 INFO:teuthology.orchestra.run.smithi157.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 988 kB/s | 58 kB 00:00 2024-05-09T00:32:44.331 INFO:teuthology.orchestra.run.smithi005.stdout:(111/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.2 MB/s | 839 kB 00:00 2024-05-09T00:32:44.337 INFO:teuthology.orchestra.run.smithi157.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.9 MB/s | 272 kB 00:00 2024-05-09T00:32:44.363 INFO:teuthology.orchestra.run.smithi157.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 325 kB/s | 16 kB 00:00 2024-05-09T00:32:44.396 INFO:teuthology.orchestra.run.smithi157.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-05-09T00:32:44.406 INFO:teuthology.orchestra.run.smithi005.stdout:(112/156): python3-asyncssh-2.13.2-5.el9.noarch 7.2 MB/s | 548 kB 00:00 2024-05-09T00:32:44.413 INFO:teuthology.orchestra.run.smithi157.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 562 kB/s | 28 kB 00:00 2024-05-09T00:32:44.447 INFO:teuthology.orchestra.run.smithi157.stdout:(137/156): python3-kubernetes-26.1.0-2.el9.noar 3.5 MB/s | 1.0 MB 00:00 2024-05-09T00:32:44.456 INFO:teuthology.orchestra.run.smithi005.stdout:(113/156): protobuf-compiler-3.14.0-13.el9.x86_ 743 kB/s | 863 kB 00:01 2024-05-09T00:32:44.464 INFO:teuthology.orchestra.run.smithi157.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 457 kB/s | 31 kB 00:00 2024-05-09T00:32:44.481 INFO:teuthology.orchestra.run.smithi005.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 576 kB/s | 43 kB 00:00 2024-05-09T00:32:44.497 INFO:teuthology.orchestra.run.smithi157.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-05-09T00:32:44.514 INFO:teuthology.orchestra.run.smithi157.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 712 kB/s | 36 kB 00:00 2024-05-09T00:32:44.531 INFO:teuthology.orchestra.run.smithi157.stdout:(141/156): python3-rsa-4.9-2.el9.noarch.rpm 702 kB/s | 59 kB 00:00 2024-05-09T00:32:44.548 INFO:teuthology.orchestra.run.smithi157.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-05-09T00:32:44.549 INFO:teuthology.orchestra.run.smithi005.stdout:(115/156): python3-certifi-2021.10.8-2.el9.noar 225 kB/s | 15 kB 00:00 2024-05-09T00:32:44.574 INFO:teuthology.orchestra.run.smithi157.stdout:(143/156): python3-webob-1.8.7-6.el9.noarch.rpm 3.8 MB/s | 230 kB 00:00 2024-05-09T00:32:44.591 INFO:teuthology.orchestra.run.smithi005.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 240 kB/s | 32 kB 00:00 2024-05-09T00:32:44.599 INFO:teuthology.orchestra.run.smithi157.stdout:(144/156): python3-websocket-client-1.2.3-2.el9 1.3 MB/s | 90 kB 00:00 2024-05-09T00:32:44.616 INFO:teuthology.orchestra.run.smithi005.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-05-09T00:32:44.624 INFO:teuthology.orchestra.run.smithi157.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 5.5 MB/s | 427 kB 00:00 2024-05-09T00:32:44.650 INFO:teuthology.orchestra.run.smithi157.stdout:(146/156): python3-zc-lockfile-2.0-10.el9.noarc 265 kB/s | 20 kB 00:00 2024-05-09T00:32:44.683 INFO:teuthology.orchestra.run.smithi005.stdout:(118/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-05-09T00:32:44.683 INFO:teuthology.orchestra.run.smithi157.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 2.2 MB/s | 192 kB 00:00 2024-05-09T00:32:44.750 INFO:teuthology.orchestra.run.smithi005.stdout:(119/156): python3-colorama-0.4.4-12.el9.noarch 530 kB/s | 35 kB 00:00 2024-05-09T00:32:44.775 INFO:teuthology.orchestra.run.smithi005.stdout:(120/156): python3-cherrypy-18.6.1-2.el9.noarch 1.9 MB/s | 358 kB 00:00 2024-05-09T00:32:44.792 INFO:teuthology.orchestra.run.smithi157.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 9.5 MB/s | 1.6 MB 00:00 2024-05-09T00:32:44.808 INFO:teuthology.orchestra.run.smithi005.stdout:(121/156): python3-google-auth-2.29.0-1.el9.noa 3.4 MB/s | 206 kB 00:00 2024-05-09T00:32:45.059 INFO:teuthology.orchestra.run.smithi005.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 578 kB/s | 144 kB 00:00 2024-05-09T00:32:45.092 INFO:teuthology.orchestra.run.smithi157.stdout:(149/156): NetworkManager-1.47.5-1.el9.x86_64.r 7.7 MB/s | 2.3 MB 00:00 2024-05-09T00:32:45.117 INFO:teuthology.orchestra.run.smithi005.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 183 kB/s | 11 kB 00:00 2024-05-09T00:32:45.168 INFO:teuthology.orchestra.run.smithi005.stdout:(124/156): python3-grpcio-1.46.7-10.el9.x86_64. 5.2 MB/s | 2.0 MB 00:00 2024-05-09T00:32:45.193 INFO:teuthology.orchestra.run.smithi005.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 235 kB/s | 18 kB 00:00 2024-05-09T00:32:45.217 INFO:teuthology.orchestra.run.smithi157.stdout:(150/156): NetworkManager-libnm-1.47.5-1.el9.x8 15 MB/s | 1.8 MB 00:00 2024-05-09T00:32:45.218 INFO:teuthology.orchestra.run.smithi005.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-05-09T00:32:45.244 INFO:teuthology.orchestra.run.smithi005.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 386 kB/s | 19 kB 00:00 2024-05-09T00:32:45.268 INFO:teuthology.orchestra.run.smithi157.stdout:(151/156): librados2-19.0.0-3594.gc9da4049.el9. 5.4 MB/s | 3.3 MB 00:00 2024-05-09T00:32:45.269 INFO:teuthology.orchestra.run.smithi005.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-05-09T00:32:45.293 INFO:teuthology.orchestra.run.smithi157.stdout:(152/156): NetworkManager-team-1.47.5-1.el9.x86 553 kB/s | 42 kB 00:00 2024-05-09T00:32:45.319 INFO:teuthology.orchestra.run.smithi005.stdout:(129/156): python3-kubernetes-26.1.0-2.el9.noar 14 MB/s | 1.0 MB 00:00 2024-05-09T00:32:45.344 INFO:teuthology.orchestra.run.smithi005.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 616 kB/s | 46 kB 00:00 2024-05-09T00:32:45.368 INFO:teuthology.orchestra.run.smithi157.stdout:(153/156): openssl-3.2.1-1.el9.x86_64.rpm 18 MB/s | 1.3 MB 00:00 2024-05-09T00:32:45.378 INFO:teuthology.orchestra.run.smithi005.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-05-09T00:32:45.403 INFO:teuthology.orchestra.run.smithi005.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 987 kB/s | 58 kB 00:00 2024-05-09T00:32:45.470 INFO:teuthology.orchestra.run.smithi005.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-05-09T00:32:45.476 INFO:teuthology.orchestra.run.smithi157.stdout:(154/156): NetworkManager-tui-1.47.5-1.el9.x86_ 1.2 MB/s | 248 kB 00:00 2024-05-09T00:32:45.495 INFO:teuthology.orchestra.run.smithi005.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 178 kB/s | 16 kB 00:00 2024-05-09T00:32:45.518 INFO:teuthology.orchestra.run.smithi157.stdout:(155/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 16 MB/s | 2.4 MB 00:00 2024-05-09T00:32:45.537 INFO:teuthology.orchestra.run.smithi005.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-05-09T00:32:45.563 INFO:teuthology.orchestra.run.smithi005.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 421 kB/s | 28 kB 00:00 2024-05-09T00:32:45.613 INFO:teuthology.orchestra.run.smithi005.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 408 kB/s | 31 kB 00:00 2024-05-09T00:32:45.635 INFO:teuthology.orchestra.run.smithi157.stdout:(156/156): librbd1-19.0.0-3594.gc9da4049.el9.x8 3.2 MB/s | 3.1 MB 00:00 2024-05-09T00:32:45.638 INFO:teuthology.orchestra.run.smithi005.stdout:(138/156): python3-routes-2.5.1-5.el9.noarch.rp 2.5 MB/s | 188 kB 00:00 2024-05-09T00:32:45.640 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-05-09T00:32:45.640 INFO:teuthology.orchestra.run.smithi157.stdout:Total 9.4 MB/s | 219 MB 00:23 2024-05-09T00:32:45.672 INFO:teuthology.orchestra.run.smithi005.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-05-09T00:32:45.706 INFO:teuthology.orchestra.run.smithi005.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 532 kB/s | 36 kB 00:00 2024-05-09T00:32:45.731 INFO:teuthology.orchestra.run.smithi005.stdout:(141/156): python3-typing-extensions-4.4.0-2.el 868 kB/s | 51 kB 00:00 2024-05-09T00:32:45.790 INFO:teuthology.orchestra.run.smithi005.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-05-09T00:32:45.823 INFO:teuthology.orchestra.run.smithi005.stdout:(143/156): python3-webob-1.8.7-6.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-05-09T00:32:45.857 INFO:teuthology.orchestra.run.smithi005.stdout:(144/156): python3-werkzeug-2.0.3-3.el9.1.noarc 6.2 MB/s | 427 kB 00:00 2024-05-09T00:32:45.890 INFO:teuthology.orchestra.run.smithi005.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 299 kB/s | 20 kB 00:00 2024-05-09T00:32:45.924 INFO:teuthology.orchestra.run.smithi005.stdout:(146/156): re2-20211101-3.el9.x86_64.rpm 2.8 MB/s | 192 kB 00:00 2024-05-09T00:32:46.041 INFO:teuthology.orchestra.run.smithi005.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-05-09T00:32:46.316 INFO:teuthology.orchestra.run.smithi005.stdout:(148/156): librbd1-19.0.0-3594.gc9da4049.el9.x8 11 MB/s | 3.1 MB 00:00 2024-05-09T00:32:46.758 INFO:teuthology.orchestra.run.smithi005.stdout:(149/156): librados2-19.0.0-3594.gc9da4049.el9. 4.0 MB/s | 3.3 MB 00:00 2024-05-09T00:32:46.766 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-05-09T00:32:46.919 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-05-09T00:32:46.920 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-05-09T00:32:47.075 INFO:teuthology.orchestra.run.smithi005.stdout:(150/156): NetworkManager-1.47.5-1.el9.x86_64.r 3.0 MB/s | 2.3 MB 00:00 2024-05-09T00:32:47.183 INFO:teuthology.orchestra.run.smithi005.stdout:(151/156): NetworkManager-team-1.47.5-1.el9.x86 386 kB/s | 42 kB 00:00 2024-05-09T00:32:47.292 INFO:teuthology.orchestra.run.smithi005.stdout:(152/156): NetworkManager-tui-1.47.5-1.el9.x86_ 2.2 MB/s | 248 kB 00:00 2024-05-09T00:32:47.418 INFO:teuthology.orchestra.run.smithi005.stdout:(153/156): openssl-3.2.1-1.el9.x86_64.rpm 11 MB/s | 1.3 MB 00:00 2024-05-09T00:32:47.626 INFO:teuthology.orchestra.run.smithi005.stdout:(154/156): NetworkManager-libnm-1.47.5-1.el9.x8 2.1 MB/s | 1.8 MB 00:00 2024-05-09T00:32:47.868 INFO:teuthology.orchestra.run.smithi005.stdout:(155/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 5.3 MB/s | 2.4 MB 00:00 2024-05-09T00:32:48.718 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-05-09T00:32:48.719 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-05-09T00:32:53.701 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 109/164 2024-05-09T00:32:53.701 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /sys 2024-05-09T00:32:53.701 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /proc 2024-05-09T00:32:53.701 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /mnt 2024-05-09T00:32:53.701 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /var/tmp 2024-05-09T00:32:53.702 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /home 2024-05-09T00:32:53.702 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /root 2024-05-09T00:32:53.702 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /tmp 2024-05-09T00:32:53.702 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-09T00:32:53.874 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 110/164 2024-05-09T00:32:53.890 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 110/164 2024-05-09T00:32:53.890 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T00:32:53.890 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-09T00:32:53.890 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-09T00:32:54.795 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-05-09T00:32:54.890 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-09T00:32:54.929 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-09T00:32:54.991 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-09T00:32:55.024 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-09T00:32:55.050 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-09T00:32:55.093 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-09T00:32:55.430 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-09T00:32:55.444 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 111/164 2024-05-09T00:32:55.470 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 111/164 2024-05-09T00:32:55.470 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T00:32:55.470 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-09T00:32:55.470 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T00:32:55.470 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T00:32:55.471 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-09T00:32:55.559 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 8/164 2024-05-09T00:32:55.677 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 8/164 2024-05-09T00:32:55.695 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 112/164 2024-05-09T00:32:55.702 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 9/164 2024-05-09T00:32:55.717 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 112/164 2024-05-09T00:32:55.717 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T00:32:55.717 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-09T00:32:55.717 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T00:32:55.717 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T00:32:55.717 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-09T00:32:55.793 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 9/164 2024-05-09T00:32:55.823 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rados-2:19.0.0-3594.gc9da4049.el9.x86_64 10/164 2024-05-09T00:32:55.974 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-09T00:32:56.028 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-09T00:32:56.056 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 13/164 2024-05-09T00:32:56.159 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 13/164 2024-05-09T00:32:56.179 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 113/164 2024-05-09T00:32:56.182 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86 14/164 2024-05-09T00:32:56.201 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 113/164 2024-05-09T00:32:56.201 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T00:32:56.201 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-09T00:32:56.201 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T00:32:56.201 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T00:32:56.201 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-09T00:32:56.244 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86 14/164 2024-05-09T00:32:56.266 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T00:32:56.291 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T00:32:56.300 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-09T00:32:56.349 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-09T00:32:56.413 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-09T00:32:56.450 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T00:32:56.587 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T00:32:56.597 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-09T00:32:56.654 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-09T00:32:56.723 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-09T00:32:56.747 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T00:32:56.799 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T00:32:56.812 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-09T00:32:56.903 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T00:32:56.959 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-09T00:32:57.001 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T00:32:57.005 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-09T00:32:57.045 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-09T00:32:57.073 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-09T00:32:57.207 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T00:32:57.314 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-09T00:32:57.347 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-09T00:32:57.398 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-09T00:32:57.471 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-09T00:32:57.506 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-09T00:32:57.516 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 125/164 2024-05-09T00:32:57.545 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 125/164 2024-05-09T00:32:57.550 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-09T00:32:57.577 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3594.gc9da4049.e 126/164 2024-05-09T00:32:57.582 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-09T00:32:57.610 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-09T00:32:57.642 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3594.gc9da4049. 127/164 2024-05-09T00:32:57.677 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noa 128/164 2024-05-09T00:32:57.679 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-09T00:32:57.732 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-09T00:32:57.763 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-09T00:32:57.798 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-09T00:32:57.874 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-09T00:32:57.909 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-ceph-common-2:19.0.0-3594.gc9da4049.el9. 32/164 2024-05-09T00:32:57.952 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-ceph-argparse-2:19.0.0-3594.gc9da4049.el 33/164 2024-05-09T00:32:58.006 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cephfs-2:19.0.0-3594.gc9da4049.el9.x86_6 34/164 2024-05-09T00:32:58.077 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-09T00:32:58.129 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-09T00:32:58.204 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-09T00:32:58.337 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-09T00:32:58.399 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-09T00:32:58.724 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noa 128/164 2024-05-09T00:32:58.762 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.n 129/164 2024-05-09T00:32:58.921 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-09T00:32:59.011 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-09T00:32:59.040 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-09T00:32:59.083 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-09T00:32:59.121 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librados-devel-2:19.0.0-3594.gc9da4049.el9.x86_6 44/164 2024-05-09T00:32:59.162 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-09T00:32:59.317 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-09T00:32:59.382 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-09T00:32:59.473 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-09T00:32:59.538 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-09T00:32:59.611 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-09T00:32:59.642 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-09T00:32:59.678 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-09T00:32:59.725 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.n 129/164 2024-05-09T00:32:59.725 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-09T00:32:59.754 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9d 130/164 2024-05-09T00:32:59.770 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-09T00:32:59.813 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-09T00:32:59.855 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-09T00:32:59.866 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9d 130/164 2024-05-09T00:32:59.991 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-09T00:33:00.029 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3594.gc9da4049.el 131/164 2024-05-09T00:33:00.035 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-09T00:33:00.056 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 132/164 2024-05-09T00:33:00.065 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-09T00:33:00.079 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 132/164 2024-05-09T00:33:00.079 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T00:33:00.080 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-09T00:33:00.080 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T00:33:00.080 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T00:33:00.080 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-09T00:33:00.107 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-09T00:33:00.123 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 133/164 2024-05-09T00:33:00.134 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 133/164 2024-05-09T00:33:00.203 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-09T00:33:00.232 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-2:19.0.0-3594.gc9da4049.el9.x86_64 134/164 2024-05-09T00:33:00.240 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-09T00:33:00.280 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-09T00:33:00.288 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 135/164 2024-05-09T00:33:00.306 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-09T00:33:00.331 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-09T00:33:00.406 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-09T00:33:00.533 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-09T00:33:00.563 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-09T00:33:00.591 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-09T00:33:00.625 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-09T00:33:00.664 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-09T00:33:00.696 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-09T00:33:00.726 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-09T00:33:00.760 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-09T00:33:00.794 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-09T00:33:00.835 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-09T00:33:00.870 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-09T00:33:00.957 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-09T00:33:01.000 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-09T00:33:01.039 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-09T00:33:01.087 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-09T00:33:01.185 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-09T00:33:01.229 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-09T00:33:01.241 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-09T00:33:01.267 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 137/164 2024-05-09T00:33:01.291 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 137/164 2024-05-09T00:33:01.291 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T00:33:01.291 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-09T00:33:01.291 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T00:33:01.291 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T00:33:01.291 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-09T00:33:01.335 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3594.gc9da4 138/164 2024-05-09T00:33:01.357 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3594.gc9da4 138/164 2024-05-09T00:33:01.357 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T00:33:01.357 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-09T00:33:01.357 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-09T00:33:01.411 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 139/164 2024-05-09T00:33:01.433 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 139/164 2024-05-09T00:33:01.433 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T00:33:01.433 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-09T00:33:01.433 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T00:33:01.433 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T00:33:01.434 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-09T00:33:01.744 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 140/164 2024-05-09T00:33:01.766 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 140/164 2024-05-09T00:33:01.767 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T00:33:01.767 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-09T00:33:01.767 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T00:33:01.767 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T00:33:01.767 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-09T00:33:01.966 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-09T00:33:02.013 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-09T00:33:02.039 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-09T00:33:02.138 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-09T00:33:02.188 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-09T00:33:02.277 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-09T00:33:02.324 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-09T00:33:02.502 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-09T00:33:02.530 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-09T00:33:02.608 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-09T00:33:03.252 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-09T00:33:03.368 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-09T00:33:03.600 INFO:teuthology.orchestra.run.smithi005.stdout:(156/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 754 kB/s | 16 MB 00:22 2024-05-09T00:33:03.603 INFO:teuthology.orchestra.run.smithi005.stdout:-------------------------------------------------------------------------------- 2024-05-09T00:33:03.604 INFO:teuthology.orchestra.run.smithi005.stdout:Total 5.1 MB/s | 219 MB 00:42 2024-05-09T00:33:04.595 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-09T00:33:04.668 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-09T00:33:04.696 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-09T00:33:04.729 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-09T00:33:04.812 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction check 2024-05-09T00:33:04.979 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction check succeeded. 2024-05-09T00:33:04.979 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction test 2024-05-09T00:33:05.076 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-09T00:33:05.108 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 101/164 2024-05-09T00:33:05.199 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 101/164 2024-05-09T00:33:05.229 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rbd-2:19.0.0-3594.gc9da4049.el9.x86_64 102/164 2024-05-09T00:33:05.266 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-09T00:33:05.756 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-09T00:33:05.786 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 105/164 2024-05-09T00:33:05.857 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 105/164 2024-05-09T00:33:05.882 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rgw-2:19.0.0-3594.gc9da4049.el9.x86_64 106/164 2024-05-09T00:33:06.023 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-test-2:19.0.0-3594.gc9da4049.el9.x86_64 141/164 2024-05-09T00:33:06.064 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : rbd-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 142/164 2024-05-09T00:33:06.105 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : rbd-nbd-2:19.0.0-3594.gc9da4049.el9.x86_64 143/164 2024-05-09T00:33:06.137 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-09T00:33:06.173 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-09T00:33:06.358 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : cephfs-shell-2:19.0.0-3594.gc9da4049.el9.x86_64 146/164 2024-05-09T00:33:06.414 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-09T00:33:06.441 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libcephfs-devel-2:19.0.0-3594.gc9da4049.el9.x86_ 148/164 2024-05-09T00:33:06.470 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-09T00:33:06.516 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-09T00:33:07.167 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T00:33:07.199 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction test succeeded. 2024-05-09T00:33:07.199 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction 2024-05-09T00:33:07.205 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T00:33:07.225 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T00:33:08.056 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : cephfs-top-2:19.0.0-3594.gc9da4049.el9.noarch 152/164 2024-05-09T00:33:08.113 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 107/164 2024-05-09T00:33:08.143 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-09T00:33:08.153 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 107/164 2024-05-09T00:33:08.184 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-09T00:33:08.267 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-09T00:33:08.305 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-09T00:33:08.331 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-09T00:33:08.331 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T00:33:08.415 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T00:33:08.448 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-09T00:33:08.448 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T00:33:08.533 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T00:33:08.574 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-09T00:33:08.579 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T00:33:08.579 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T00:33:08.709 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 107/164 2024-05-09T00:33:08.756 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 108/164 2024-05-09T00:33:08.759 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T00:33:08.759 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T00:33:08.844 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T00:33:08.849 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 108/164 2024-05-09T00:33:08.849 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-09T00:33:08.849 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-09T00:33:08.850 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-05-09T00:33:08.874 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 109/164 2024-05-09T00:33:08.915 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T00:33:13.490 INFO:teuthology.orchestra.run.smithi005.stdout: Preparing : 1/1 2024-05-09T00:33:13.580 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-09T00:33:13.628 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-09T00:33:13.689 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-09T00:33:13.722 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-09T00:33:13.747 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-09T00:33:13.796 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-09T00:33:14.144 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-09T00:33:14.256 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 8/164 2024-05-09T00:33:14.396 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 8/164 2024-05-09T00:33:14.432 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 9/164 2024-05-09T00:33:14.525 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 9/164 2024-05-09T00:33:14.554 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rados-2:19.0.0-3594.gc9da4049.el9.x86_64 10/164 2024-05-09T00:33:14.608 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T00:33:14.608 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-2:19.0.0-3594.gc9da4049.el9.x86_64 1/164 2024-05-09T00:33:14.609 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 2/164 2024-05-09T00:33:14.609 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 3/164 2024-05-09T00:33:14.609 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 4/164 2024-05-09T00:33:14.609 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3594.gc9da4 5/164 2024-05-09T00:33:14.609 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 6/164 2024-05-09T00:33:14.609 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 7/164 2024-05-09T00:33:14.609 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 8/164 2024-05-09T00:33:14.609 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 9/164 2024-05-09T00:33:14.609 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 10/164 2024-05-09T00:33:14.609 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 11/164 2024-05-09T00:33:14.609 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-test-2:19.0.0-3594.gc9da4049.el9.x86_64 12/164 2024-05-09T00:33:14.609 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 13/164 2024-05-09T00:33:14.609 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : cephfs-shell-2:19.0.0-3594.gc9da4049.el9.x86_64 14/164 2024-05-09T00:33:14.609 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libcephfs-devel-2:19.0.0-3594.gc9da4049.el9.x86_ 15/164 2024-05-09T00:33:14.610 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 16/164 2024-05-09T00:33:14.610 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 17/164 2024-05-09T00:33:14.610 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librados-devel-2:19.0.0-3594.gc9da4049.el9.x86_6 18/164 2024-05-09T00:33:14.610 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86 19/164 2024-05-09T00:33:14.610 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 20/164 2024-05-09T00:33:14.610 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3594.gc9da4049.el 21/164 2024-05-09T00:33:14.610 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-ceph-common-2:19.0.0-3594.gc9da4049.el9. 22/164 2024-05-09T00:33:14.610 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cephfs-2:19.0.0-3594.gc9da4049.el9.x86_6 23/164 2024-05-09T00:33:14.610 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-rados-2:19.0.0-3594.gc9da4049.el9.x86_64 24/164 2024-05-09T00:33:14.610 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-rbd-2:19.0.0-3594.gc9da4049.el9.x86_64 25/164 2024-05-09T00:33:14.610 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-rgw-2:19.0.0-3594.gc9da4049.el9.x86_64 26/164 2024-05-09T00:33:14.610 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : rbd-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 27/164 2024-05-09T00:33:14.624 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 28/164 2024-05-09T00:33:14.624 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : rbd-nbd-2:19.0.0-3594.gc9da4049.el9.x86_64 29/164 2024-05-09T00:33:14.624 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3594.gc9da4049. 30/164 2024-05-09T00:33:14.624 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noa 31/164 2024-05-09T00:33:14.624 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.n 32/164 2024-05-09T00:33:14.625 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9d 33/164 2024-05-09T00:33:14.625 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3594.gc9da4049.el 34/164 2024-05-09T00:33:14.625 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 35/164 2024-05-09T00:33:14.625 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3594.gc9da4049.e 36/164 2024-05-09T00:33:14.625 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 37/164 2024-05-09T00:33:14.625 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 38/164 2024-05-09T00:33:14.625 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : cephfs-top-2:19.0.0-3594.gc9da4049.el9.noarch 39/164 2024-05-09T00:33:14.625 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-09T00:33:14.625 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-09T00:33:14.625 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-09T00:33:14.626 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-09T00:33:14.626 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-09T00:33:14.626 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-09T00:33:14.626 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-09T00:33:14.626 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-09T00:33:14.626 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-09T00:33:14.626 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-09T00:33:14.626 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-09T00:33:14.626 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-09T00:33:14.626 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-09T00:33:14.626 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-09T00:33:14.626 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-09T00:33:14.626 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-09T00:33:14.627 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-09T00:33:14.627 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-09T00:33:14.627 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-09T00:33:14.627 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-09T00:33:14.627 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-09T00:33:14.627 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-09T00:33:14.627 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-09T00:33:14.627 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-09T00:33:14.627 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-09T00:33:14.627 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-09T00:33:14.627 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-09T00:33:14.627 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-09T00:33:14.628 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-09T00:33:14.628 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-09T00:33:14.628 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-09T00:33:14.628 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-09T00:33:14.628 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-09T00:33:14.628 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-09T00:33:14.628 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-09T00:33:14.628 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-09T00:33:14.628 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-09T00:33:14.628 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-09T00:33:14.628 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-09T00:33:14.628 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-09T00:33:14.629 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-09T00:33:14.629 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-09T00:33:14.629 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-09T00:33:14.629 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-09T00:33:14.629 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-09T00:33:14.629 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-09T00:33:14.629 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-09T00:33:14.629 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-09T00:33:14.629 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-09T00:33:14.629 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-09T00:33:14.629 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-09T00:33:14.629 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-09T00:33:14.630 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-09T00:33:14.630 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-09T00:33:14.630 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-09T00:33:14.630 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-09T00:33:14.630 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-09T00:33:14.630 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-09T00:33:14.630 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-09T00:33:14.630 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-09T00:33:14.630 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-09T00:33:14.631 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-09T00:33:14.631 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-09T00:33:14.631 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-09T00:33:14.631 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-09T00:33:14.631 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-09T00:33:14.631 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-09T00:33:14.631 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-09T00:33:14.631 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-09T00:33:14.631 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-09T00:33:14.631 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-09T00:33:14.631 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-09T00:33:14.631 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-09T00:33:14.631 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-09T00:33:14.631 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-09T00:33:14.632 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-09T00:33:14.632 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-09T00:33:14.632 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-09T00:33:14.632 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-09T00:33:14.632 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-09T00:33:14.632 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-09T00:33:14.632 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-09T00:33:14.632 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-09T00:33:14.632 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-09T00:33:14.632 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-09T00:33:14.632 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-09T00:33:14.632 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-09T00:33:14.632 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-09T00:33:14.632 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-09T00:33:14.633 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-09T00:33:14.633 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-09T00:33:14.633 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-09T00:33:14.633 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-09T00:33:14.633 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-09T00:33:14.633 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-09T00:33:14.633 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-09T00:33:14.633 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-09T00:33:14.633 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-09T00:33:14.633 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-09T00:33:14.633 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-09T00:33:14.633 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-09T00:33:14.633 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-09T00:33:14.633 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-09T00:33:14.634 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-09T00:33:14.634 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-09T00:33:14.634 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-09T00:33:14.634 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-09T00:33:14.634 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-09T00:33:14.634 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-09T00:33:14.634 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 149/164 2024-05-09T00:33:14.635 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-09T00:33:14.635 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 151/164 2024-05-09T00:33:14.635 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-09T00:33:14.635 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-09T00:33:14.635 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-09T00:33:14.635 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-09T00:33:14.636 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-09T00:33:14.636 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-09T00:33:14.636 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-09T00:33:14.636 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-09T00:33:14.636 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-09T00:33:14.636 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-09T00:33:14.636 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-09T00:33:14.636 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-09T00:33:14.733 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-09T00:33:14.793 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-09T00:33:14.820 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 13/164 2024-05-09T00:33:14.930 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 13/164 2024-05-09T00:33:14.954 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86 14/164 2024-05-09T00:33:15.001 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86 14/164 2024-05-09T00:33:15.030 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T00:33:15.057 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T00:33:15.199 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T00:33:15.374 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T00:33:15.527 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T00:33:15.596 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T00:33:15.705 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T00:33:15.805 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T00:33:16.012 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T00:33:16.123 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-09T00:33:16.151 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-09T00:33:16.186 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-09T00:33:16.261 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-09T00:33:16.311 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-09T00:33:16.355 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-09T00:33:16.387 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-09T00:33:16.423 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-09T00:33:16.503 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-09T00:33:16.563 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-09T00:33:16.593 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-09T00:33:16.637 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-09T00:33:16.715 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-09T00:33:16.747 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-ceph-common-2:19.0.0-3594.gc9da4049.el9. 32/164 2024-05-09T00:33:16.791 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-ceph-argparse-2:19.0.0-3594.gc9da4049.el 33/164 2024-05-09T00:33:16.819 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cephfs-2:19.0.0-3594.gc9da4049.el9.x86_6 34/164 2024-05-09T00:33:16.884 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-09T00:33:16.934 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-09T00:33:17.011 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-09T00:33:17.156 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-09T00:33:17.213 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-09T00:33:17.251 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T00:33:17.251 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-09T00:33:17.251 INFO:teuthology.orchestra.run.smithi131.stdout:Upgraded: 2024-05-09T00:33:17.251 INFO:teuthology.orchestra.run.smithi131.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-09T00:33:17.252 INFO:teuthology.orchestra.run.smithi131.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-09T00:33:17.252 INFO:teuthology.orchestra.run.smithi131.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-09T00:33:17.252 INFO:teuthology.orchestra.run.smithi131.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-09T00:33:17.252 INFO:teuthology.orchestra.run.smithi131.stdout: librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:33:17.252 INFO:teuthology.orchestra.run.smithi131.stdout: librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:33:17.252 INFO:teuthology.orchestra.run.smithi131.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-09T00:33:17.252 INFO:teuthology.orchestra.run.smithi131.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-09T00:33:17.252 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-05-09T00:33:17.252 INFO:teuthology.orchestra.run.smithi131.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-09T00:33:17.253 INFO:teuthology.orchestra.run.smithi131.stdout: automake-1.16.2-8.el9.noarch 2024-05-09T00:33:17.253 INFO:teuthology.orchestra.run.smithi131.stdout: bison-3.7.4-5.el9.x86_64 2024-05-09T00:33:17.253 INFO:teuthology.orchestra.run.smithi131.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-09T00:33:17.253 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:33:17.253 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:33:17.253 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:33:17.253 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:33:17.253 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-grafana-dashboards-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T00:33:17.253 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-immutable-object-cache-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:33:17.253 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:33:17.253 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:33:17.253 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T00:33:17.253 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T00:33:17.254 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T00:33:17.254 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-modules-core-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T00:33:17.254 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T00:33:17.254 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:33:17.254 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:33:17.254 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-prometheus-alerts-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T00:33:17.254 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:33:17.254 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:33:17.254 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-test-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:33:17.254 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T00:33:17.254 INFO:teuthology.orchestra.run.smithi131.stdout: cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T00:33:17.254 INFO:teuthology.orchestra.run.smithi131.stdout: cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:33:17.255 INFO:teuthology.orchestra.run.smithi131.stdout: cephfs-shell-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:33:17.255 INFO:teuthology.orchestra.run.smithi131.stdout: cephfs-top-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T00:33:17.255 INFO:teuthology.orchestra.run.smithi131.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-09T00:33:17.255 INFO:teuthology.orchestra.run.smithi131.stdout: flex-2.6.4-9.el9.x86_64 2024-05-09T00:33:17.255 INFO:teuthology.orchestra.run.smithi131.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-09T00:33:17.255 INFO:teuthology.orchestra.run.smithi131.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-09T00:33:17.255 INFO:teuthology.orchestra.run.smithi131.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-09T00:33:17.255 INFO:teuthology.orchestra.run.smithi131.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-09T00:33:17.255 INFO:teuthology.orchestra.run.smithi131.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-09T00:33:17.255 INFO:teuthology.orchestra.run.smithi131.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-09T00:33:17.255 INFO:teuthology.orchestra.run.smithi131.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-09T00:33:17.255 INFO:teuthology.orchestra.run.smithi131.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-09T00:33:17.255 INFO:teuthology.orchestra.run.smithi131.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-09T00:33:17.256 INFO:teuthology.orchestra.run.smithi131.stdout: libcephfs-devel-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:33:17.256 INFO:teuthology.orchestra.run.smithi131.stdout: libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:33:17.256 INFO:teuthology.orchestra.run.smithi131.stdout: libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:33:17.256 INFO:teuthology.orchestra.run.smithi131.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-09T00:33:17.256 INFO:teuthology.orchestra.run.smithi131.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-09T00:33:17.256 INFO:teuthology.orchestra.run.smithi131.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-09T00:33:17.256 INFO:teuthology.orchestra.run.smithi131.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-09T00:33:17.256 INFO:teuthology.orchestra.run.smithi131.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-09T00:33:17.256 INFO:teuthology.orchestra.run.smithi131.stdout: librados-devel-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:33:17.256 INFO:teuthology.orchestra.run.smithi131.stdout: libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:33:17.256 INFO:teuthology.orchestra.run.smithi131.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-09T00:33:17.256 INFO:teuthology.orchestra.run.smithi131.stdout: librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:33:17.256 INFO:teuthology.orchestra.run.smithi131.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-09T00:33:17.257 INFO:teuthology.orchestra.run.smithi131.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-09T00:33:17.257 INFO:teuthology.orchestra.run.smithi131.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-09T00:33:17.257 INFO:teuthology.orchestra.run.smithi131.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-09T00:33:17.257 INFO:teuthology.orchestra.run.smithi131.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-09T00:33:17.257 INFO:teuthology.orchestra.run.smithi131.stdout: lua-5.4.4-4.el9.x86_64 2024-05-09T00:33:17.257 INFO:teuthology.orchestra.run.smithi131.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-09T00:33:17.257 INFO:teuthology.orchestra.run.smithi131.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-09T00:33:17.257 INFO:teuthology.orchestra.run.smithi131.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-09T00:33:17.257 INFO:teuthology.orchestra.run.smithi131.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-09T00:33:17.258 INFO:teuthology.orchestra.run.smithi131.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-09T00:33:17.258 INFO:teuthology.orchestra.run.smithi131.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-09T00:33:17.258 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-13.14-1.el9.x86_64 2024-05-09T00:33:17.258 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-09T00:33:17.258 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-09T00:33:17.258 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-09T00:33:17.258 INFO:teuthology.orchestra.run.smithi131.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-09T00:33:17.258 INFO:teuthology.orchestra.run.smithi131.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-09T00:33:17.258 INFO:teuthology.orchestra.run.smithi131.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-09T00:33:17.258 INFO:teuthology.orchestra.run.smithi131.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-09T00:33:17.259 INFO:teuthology.orchestra.run.smithi131.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-09T00:33:17.259 INFO:teuthology.orchestra.run.smithi131.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-09T00:33:17.259 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-09T00:33:17.259 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ceph-argparse-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:33:17.259 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:33:17.259 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cephfs-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:33:17.259 INFO:teuthology.orchestra.run.smithi131.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-09T00:33:17.259 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-09T00:33:17.259 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-09T00:33:17.259 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-09T00:33:17.259 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-09T00:33:17.259 INFO:teuthology.orchestra.run.smithi131.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-09T00:33:17.259 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-09T00:33:17.259 INFO:teuthology.orchestra.run.smithi131.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-09T00:33:17.260 INFO:teuthology.orchestra.run.smithi131.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-09T00:33:17.260 INFO:teuthology.orchestra.run.smithi131.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-09T00:33:17.260 INFO:teuthology.orchestra.run.smithi131.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-09T00:33:17.260 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-09T00:33:17.260 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-09T00:33:17.260 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-09T00:33:17.260 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-09T00:33:17.260 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-09T00:33:17.260 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-09T00:33:17.260 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-09T00:33:17.260 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-09T00:33:17.260 INFO:teuthology.orchestra.run.smithi131.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-09T00:33:17.260 INFO:teuthology.orchestra.run.smithi131.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-09T00:33:17.261 INFO:teuthology.orchestra.run.smithi131.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-09T00:33:17.261 INFO:teuthology.orchestra.run.smithi131.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-09T00:33:17.261 INFO:teuthology.orchestra.run.smithi131.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-09T00:33:17.261 INFO:teuthology.orchestra.run.smithi131.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-09T00:33:17.261 INFO:teuthology.orchestra.run.smithi131.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-09T00:33:17.261 INFO:teuthology.orchestra.run.smithi131.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-09T00:33:17.261 INFO:teuthology.orchestra.run.smithi131.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-09T00:33:17.261 INFO:teuthology.orchestra.run.smithi131.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-09T00:33:17.261 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-09T00:33:17.261 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ply-3.11-14.el9.noarch 2024-05-09T00:33:17.261 INFO:teuthology.orchestra.run.smithi131.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-09T00:33:17.261 INFO:teuthology.orchestra.run.smithi131.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-09T00:33:17.261 INFO:teuthology.orchestra.run.smithi131.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-09T00:33:17.262 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-09T00:33:17.262 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-09T00:33:17.262 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-09T00:33:17.262 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-09T00:33:17.262 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-09T00:33:17.262 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-09T00:33:17.262 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rados-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:33:17.262 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rbd-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:33:17.262 INFO:teuthology.orchestra.run.smithi131.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-09T00:33:17.262 INFO:teuthology.orchestra.run.smithi131.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-09T00:33:17.262 INFO:teuthology.orchestra.run.smithi131.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-09T00:33:17.262 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rgw-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:33:17.262 INFO:teuthology.orchestra.run.smithi131.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-09T00:33:17.262 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-09T00:33:17.263 INFO:teuthology.orchestra.run.smithi131.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-09T00:33:17.263 INFO:teuthology.orchestra.run.smithi131.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-09T00:33:17.263 INFO:teuthology.orchestra.run.smithi131.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-09T00:33:17.263 INFO:teuthology.orchestra.run.smithi131.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-09T00:33:17.263 INFO:teuthology.orchestra.run.smithi131.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-09T00:33:17.263 INFO:teuthology.orchestra.run.smithi131.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-09T00:33:17.263 INFO:teuthology.orchestra.run.smithi131.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-09T00:33:17.263 INFO:teuthology.orchestra.run.smithi131.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-09T00:33:17.263 INFO:teuthology.orchestra.run.smithi131.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-09T00:33:17.263 INFO:teuthology.orchestra.run.smithi131.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-09T00:33:17.263 INFO:teuthology.orchestra.run.smithi131.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-09T00:33:17.263 INFO:teuthology.orchestra.run.smithi131.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-09T00:33:17.263 INFO:teuthology.orchestra.run.smithi131.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-09T00:33:17.263 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:33:17.264 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:33:17.264 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-nbd-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:33:17.264 INFO:teuthology.orchestra.run.smithi131.stdout: re2-1:20211101-3.el9.x86_64 2024-05-09T00:33:17.264 INFO:teuthology.orchestra.run.smithi131.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-09T00:33:17.264 INFO:teuthology.orchestra.run.smithi131.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-09T00:33:17.264 INFO:teuthology.orchestra.run.smithi131.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-09T00:33:17.264 INFO:teuthology.orchestra.run.smithi131.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-09T00:33:17.264 INFO:teuthology.orchestra.run.smithi131.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-09T00:33:17.265 INFO:teuthology.orchestra.run.smithi131.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-09T00:33:17.265 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-09T00:33:17.265 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-05-09T00:33:17.691 DEBUG:teuthology.parallel:result is None 2024-05-09T00:33:17.772 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-09T00:33:17.868 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-09T00:33:17.903 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-09T00:33:17.946 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-09T00:33:17.984 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librados-devel-2:19.0.0-3594.gc9da4049.el9.x86_6 44/164 2024-05-09T00:33:18.026 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-09T00:33:18.181 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-09T00:33:18.229 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-09T00:33:18.323 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-09T00:33:18.386 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-09T00:33:18.461 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-09T00:33:18.496 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-09T00:33:18.533 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-09T00:33:18.581 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-09T00:33:18.616 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-09T00:33:18.660 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-09T00:33:18.701 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-09T00:33:18.845 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-09T00:33:18.890 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-09T00:33:18.920 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-09T00:33:18.962 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-09T00:33:19.060 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-09T00:33:19.103 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-09T00:33:19.144 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-09T00:33:19.177 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-09T00:33:19.202 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-09T00:33:19.302 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-09T00:33:19.434 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-09T00:33:19.467 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-09T00:33:19.504 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-09T00:33:19.531 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-09T00:33:19.569 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-09T00:33:19.609 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-09T00:33:19.639 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-09T00:33:19.673 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-09T00:33:19.722 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-09T00:33:19.766 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-09T00:33:19.800 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-09T00:33:19.890 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-09T00:33:19.943 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-09T00:33:19.978 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-09T00:33:20.017 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-09T00:33:20.129 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-09T00:33:20.189 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-09T00:33:20.386 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 109/164 2024-05-09T00:33:20.386 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /sys 2024-05-09T00:33:20.386 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /proc 2024-05-09T00:33:20.386 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /mnt 2024-05-09T00:33:20.386 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /var/tmp 2024-05-09T00:33:20.386 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /home 2024-05-09T00:33:20.386 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /root 2024-05-09T00:33:20.386 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /tmp 2024-05-09T00:33:20.386 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-05-09T00:33:20.565 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 110/164 2024-05-09T00:33:20.579 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 110/164 2024-05-09T00:33:20.579 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T00:33:20.580 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-09T00:33:20.580 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-05-09T00:33:20.952 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-09T00:33:21.010 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-09T00:33:21.052 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-09T00:33:21.155 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-09T00:33:21.219 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-09T00:33:21.318 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-09T00:33:21.354 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-09T00:33:21.543 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-09T00:33:21.568 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-09T00:33:21.649 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-09T00:33:22.151 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 111/164 2024-05-09T00:33:22.176 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 111/164 2024-05-09T00:33:22.176 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T00:33:22.176 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-09T00:33:22.176 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T00:33:22.176 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T00:33:22.177 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-05-09T00:33:22.339 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-09T00:33:22.401 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 112/164 2024-05-09T00:33:22.424 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 112/164 2024-05-09T00:33:22.424 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T00:33:22.424 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-09T00:33:22.424 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T00:33:22.424 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T00:33:22.424 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-05-09T00:33:22.454 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-09T00:33:22.894 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 113/164 2024-05-09T00:33:22.916 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 113/164 2024-05-09T00:33:22.916 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T00:33:22.916 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-09T00:33:22.916 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T00:33:22.916 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T00:33:22.916 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-05-09T00:33:23.015 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-09T00:33:23.073 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-09T00:33:23.129 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-09T00:33:23.316 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-09T00:33:23.361 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-09T00:33:23.440 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-09T00:33:23.537 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-09T00:33:23.678 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-09T00:33:23.719 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-09T00:33:23.752 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-09T00:33:23.771 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-09T00:33:23.779 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-09T00:33:23.855 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-09T00:33:23.883 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-09T00:33:23.924 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-09T00:33:24.198 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 125/164 2024-05-09T00:33:24.228 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 125/164 2024-05-09T00:33:24.260 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3594.gc9da4049.e 126/164 2024-05-09T00:33:24.261 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-09T00:33:24.304 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 101/164 2024-05-09T00:33:24.322 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3594.gc9da4049. 127/164 2024-05-09T00:33:24.351 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noa 128/164 2024-05-09T00:33:24.395 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 101/164 2024-05-09T00:33:24.425 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rbd-2:19.0.0-3594.gc9da4049.el9.x86_64 102/164 2024-05-09T00:33:24.473 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-09T00:33:24.984 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-09T00:33:25.007 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 105/164 2024-05-09T00:33:25.078 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 105/164 2024-05-09T00:33:25.100 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rgw-2:19.0.0-3594.gc9da4049.el9.x86_64 106/164 2024-05-09T00:33:25.408 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noa 128/164 2024-05-09T00:33:25.444 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.n 129/164 2024-05-09T00:33:26.416 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.n 129/164 2024-05-09T00:33:26.445 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9d 130/164 2024-05-09T00:33:26.557 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9d 130/164 2024-05-09T00:33:26.720 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3594.gc9da4049.el 131/164 2024-05-09T00:33:26.764 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 132/164 2024-05-09T00:33:26.786 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 132/164 2024-05-09T00:33:26.786 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T00:33:26.786 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-09T00:33:26.786 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T00:33:26.786 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T00:33:26.786 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-05-09T00:33:26.831 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 133/164 2024-05-09T00:33:26.842 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 133/164 2024-05-09T00:33:26.941 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-2:19.0.0-3594.gc9da4049.el9.x86_64 134/164 2024-05-09T00:33:26.997 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 135/164 2024-05-09T00:33:27.478 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 107/164 2024-05-09T00:33:27.513 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 107/164 2024-05-09T00:33:27.935 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-09T00:33:27.975 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 137/164 2024-05-09T00:33:27.997 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 137/164 2024-05-09T00:33:27.997 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T00:33:27.997 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-09T00:33:27.998 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T00:33:27.998 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T00:33:27.998 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-05-09T00:33:28.042 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3594.gc9da4 138/164 2024-05-09T00:33:28.064 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3594.gc9da4 138/164 2024-05-09T00:33:28.064 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T00:33:28.064 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-09T00:33:28.064 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-05-09T00:33:28.109 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 139/164 2024-05-09T00:33:28.115 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 107/164 2024-05-09T00:33:28.132 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 139/164 2024-05-09T00:33:28.132 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T00:33:28.132 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-09T00:33:28.132 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T00:33:28.132 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T00:33:28.132 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-05-09T00:33:28.149 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 108/164 2024-05-09T00:33:28.242 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 108/164 2024-05-09T00:33:28.242 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-09T00:33:28.242 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-09T00:33:28.242 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:33:28.275 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 109/164 2024-05-09T00:33:28.444 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 140/164 2024-05-09T00:33:28.466 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 140/164 2024-05-09T00:33:28.466 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T00:33:28.466 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-09T00:33:28.466 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T00:33:28.467 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T00:33:28.467 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-05-09T00:33:32.756 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-test-2:19.0.0-3594.gc9da4049.el9.x86_64 141/164 2024-05-09T00:33:32.797 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : rbd-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 142/164 2024-05-09T00:33:32.838 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : rbd-nbd-2:19.0.0-3594.gc9da4049.el9.x86_64 143/164 2024-05-09T00:33:32.870 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-09T00:33:32.897 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-09T00:33:33.090 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cephfs-shell-2:19.0.0-3594.gc9da4049.el9.x86_64 146/164 2024-05-09T00:33:33.127 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-09T00:33:33.157 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcephfs-devel-2:19.0.0-3594.gc9da4049.el9.x86_ 148/164 2024-05-09T00:33:33.186 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-09T00:33:33.226 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-09T00:33:33.842 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T00:33:33.889 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T00:33:33.908 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T00:33:34.747 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cephfs-top-2:19.0.0-3594.gc9da4049.el9.noarch 152/164 2024-05-09T00:33:34.836 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-09T00:33:34.877 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-09T00:33:34.960 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-09T00:33:34.989 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-09T00:33:35.015 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-09T00:33:35.015 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T00:33:35.099 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T00:33:35.132 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-09T00:33:35.132 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T00:33:35.200 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T00:33:35.225 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-09T00:33:35.229 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T00:33:35.229 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T00:33:35.410 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T00:33:35.410 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T00:33:35.486 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T00:33:35.549 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T00:33:40.487 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 109/164 2024-05-09T00:33:40.487 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /sys 2024-05-09T00:33:40.487 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /proc 2024-05-09T00:33:40.488 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /mnt 2024-05-09T00:33:40.488 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /var/tmp 2024-05-09T00:33:40.488 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /home 2024-05-09T00:33:40.488 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /root 2024-05-09T00:33:40.488 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /tmp 2024-05-09T00:33:40.488 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:33:40.666 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 110/164 2024-05-09T00:33:40.685 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 110/164 2024-05-09T00:33:40.686 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T00:33:40.686 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-09T00:33:40.686 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:33:41.253 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T00:33:41.253 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-2:19.0.0-3594.gc9da4049.el9.x86_64 1/164 2024-05-09T00:33:41.253 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 2/164 2024-05-09T00:33:41.254 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 3/164 2024-05-09T00:33:41.254 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 4/164 2024-05-09T00:33:41.254 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3594.gc9da4 5/164 2024-05-09T00:33:41.254 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 6/164 2024-05-09T00:33:41.254 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 7/164 2024-05-09T00:33:41.254 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 8/164 2024-05-09T00:33:41.254 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 9/164 2024-05-09T00:33:41.254 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 10/164 2024-05-09T00:33:41.254 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 11/164 2024-05-09T00:33:41.254 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-test-2:19.0.0-3594.gc9da4049.el9.x86_64 12/164 2024-05-09T00:33:41.254 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 13/164 2024-05-09T00:33:41.254 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cephfs-shell-2:19.0.0-3594.gc9da4049.el9.x86_64 14/164 2024-05-09T00:33:41.255 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcephfs-devel-2:19.0.0-3594.gc9da4049.el9.x86_ 15/164 2024-05-09T00:33:41.255 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 16/164 2024-05-09T00:33:41.255 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 17/164 2024-05-09T00:33:41.255 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librados-devel-2:19.0.0-3594.gc9da4049.el9.x86_6 18/164 2024-05-09T00:33:41.255 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86 19/164 2024-05-09T00:33:41.255 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 20/164 2024-05-09T00:33:41.255 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3594.gc9da4049.el 21/164 2024-05-09T00:33:41.255 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-ceph-common-2:19.0.0-3594.gc9da4049.el9. 22/164 2024-05-09T00:33:41.255 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cephfs-2:19.0.0-3594.gc9da4049.el9.x86_6 23/164 2024-05-09T00:33:41.255 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rados-2:19.0.0-3594.gc9da4049.el9.x86_64 24/164 2024-05-09T00:33:41.255 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rbd-2:19.0.0-3594.gc9da4049.el9.x86_64 25/164 2024-05-09T00:33:41.255 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rgw-2:19.0.0-3594.gc9da4049.el9.x86_64 26/164 2024-05-09T00:33:41.255 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : rbd-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 27/164 2024-05-09T00:33:41.256 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 28/164 2024-05-09T00:33:41.256 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : rbd-nbd-2:19.0.0-3594.gc9da4049.el9.x86_64 29/164 2024-05-09T00:33:41.256 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3594.gc9da4049. 30/164 2024-05-09T00:33:41.256 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noa 31/164 2024-05-09T00:33:41.256 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.n 32/164 2024-05-09T00:33:41.256 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9d 33/164 2024-05-09T00:33:41.256 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3594.gc9da4049.el 34/164 2024-05-09T00:33:41.256 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 35/164 2024-05-09T00:33:41.256 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3594.gc9da4049.e 36/164 2024-05-09T00:33:41.256 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 37/164 2024-05-09T00:33:41.256 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 38/164 2024-05-09T00:33:41.256 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cephfs-top-2:19.0.0-3594.gc9da4049.el9.noarch 39/164 2024-05-09T00:33:41.256 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-09T00:33:41.256 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-09T00:33:41.257 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-09T00:33:41.257 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-09T00:33:41.257 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-09T00:33:41.257 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-09T00:33:41.257 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-09T00:33:41.257 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-09T00:33:41.257 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-09T00:33:41.257 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-09T00:33:41.257 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-09T00:33:41.257 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-09T00:33:41.257 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-09T00:33:41.257 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-09T00:33:41.258 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-09T00:33:41.258 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-09T00:33:41.258 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-09T00:33:41.258 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-09T00:33:41.258 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-09T00:33:41.258 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-09T00:33:41.258 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-09T00:33:41.258 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-09T00:33:41.258 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-09T00:33:41.258 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-09T00:33:41.258 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-09T00:33:41.258 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-09T00:33:41.259 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-09T00:33:41.259 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-09T00:33:41.259 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-09T00:33:41.259 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-09T00:33:41.259 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-09T00:33:41.259 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-09T00:33:41.259 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-09T00:33:41.259 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-09T00:33:41.259 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-09T00:33:41.260 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-09T00:33:41.260 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-09T00:33:41.260 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-09T00:33:41.260 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-09T00:33:41.260 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-09T00:33:41.260 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-09T00:33:41.260 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-09T00:33:41.260 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-09T00:33:41.260 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-09T00:33:41.260 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-09T00:33:41.260 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-09T00:33:41.260 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-09T00:33:41.260 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-09T00:33:41.261 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-09T00:33:41.261 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-09T00:33:41.261 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-09T00:33:41.261 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-09T00:33:41.261 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-09T00:33:41.261 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-09T00:33:41.261 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-09T00:33:41.261 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-09T00:33:41.261 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-09T00:33:41.262 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-09T00:33:41.262 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-09T00:33:41.262 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-09T00:33:41.262 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-09T00:33:41.262 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-09T00:33:41.262 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-09T00:33:41.262 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-09T00:33:41.262 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-09T00:33:41.262 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-09T00:33:41.262 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-09T00:33:41.262 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-09T00:33:41.262 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-09T00:33:41.262 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-09T00:33:41.263 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-09T00:33:41.263 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-09T00:33:41.263 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-09T00:33:41.263 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-09T00:33:41.263 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-09T00:33:41.263 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-09T00:33:41.263 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-09T00:33:41.263 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-09T00:33:41.263 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-09T00:33:41.264 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-09T00:33:41.264 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-09T00:33:41.264 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-09T00:33:41.264 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-09T00:33:41.264 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-09T00:33:41.264 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-09T00:33:41.264 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-09T00:33:41.264 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-09T00:33:41.264 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-09T00:33:41.264 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-09T00:33:41.264 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-09T00:33:41.264 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-09T00:33:41.264 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-09T00:33:41.264 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-09T00:33:41.265 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-09T00:33:41.265 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-09T00:33:41.265 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-09T00:33:41.265 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-09T00:33:41.265 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-09T00:33:41.265 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-09T00:33:41.265 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-09T00:33:41.265 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-09T00:33:41.265 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-09T00:33:41.265 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-09T00:33:41.266 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-09T00:33:41.266 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-09T00:33:41.266 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-09T00:33:41.266 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-09T00:33:41.266 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-09T00:33:41.266 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-09T00:33:41.266 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 149/164 2024-05-09T00:33:41.266 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-09T00:33:41.266 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 151/164 2024-05-09T00:33:41.266 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-09T00:33:41.266 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-09T00:33:41.266 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-09T00:33:41.267 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-09T00:33:41.267 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-09T00:33:41.267 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-09T00:33:41.267 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-09T00:33:41.267 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-09T00:33:41.267 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-09T00:33:41.267 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-09T00:33:41.267 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-09T00:33:41.267 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-09T00:33:42.319 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 111/164 2024-05-09T00:33:42.348 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 111/164 2024-05-09T00:33:42.348 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T00:33:42.348 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-09T00:33:42.348 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T00:33:42.348 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T00:33:42.348 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:33:42.595 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 112/164 2024-05-09T00:33:42.619 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 112/164 2024-05-09T00:33:42.620 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T00:33:42.620 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-09T00:33:42.620 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T00:33:42.620 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T00:33:42.620 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:33:43.104 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 113/164 2024-05-09T00:33:43.128 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 113/164 2024-05-09T00:33:43.128 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T00:33:43.128 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-09T00:33:43.128 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T00:33:43.129 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T00:33:43.129 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:33:43.243 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-09T00:33:43.301 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-09T00:33:43.365 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-09T00:33:43.568 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-09T00:33:43.614 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-09T00:33:43.694 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-09T00:33:43.801 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-09T00:33:43.950 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-09T00:33:43.974 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T00:33:43.974 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-05-09T00:33:43.974 INFO:teuthology.orchestra.run.smithi157.stdout:Upgraded: 2024-05-09T00:33:43.974 INFO:teuthology.orchestra.run.smithi157.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-09T00:33:43.974 INFO:teuthology.orchestra.run.smithi157.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-09T00:33:43.974 INFO:teuthology.orchestra.run.smithi157.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-09T00:33:43.974 INFO:teuthology.orchestra.run.smithi157.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-09T00:33:43.974 INFO:teuthology.orchestra.run.smithi157.stdout: librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:33:43.975 INFO:teuthology.orchestra.run.smithi157.stdout: librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:33:43.975 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-09T00:33:43.975 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-09T00:33:43.975 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-05-09T00:33:43.975 INFO:teuthology.orchestra.run.smithi157.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-09T00:33:43.975 INFO:teuthology.orchestra.run.smithi157.stdout: automake-1.16.2-8.el9.noarch 2024-05-09T00:33:43.975 INFO:teuthology.orchestra.run.smithi157.stdout: bison-3.7.4-5.el9.x86_64 2024-05-09T00:33:43.975 INFO:teuthology.orchestra.run.smithi157.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-09T00:33:43.975 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:33:43.975 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:33:43.975 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:33:43.975 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:33:43.975 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-grafana-dashboards-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T00:33:43.976 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-immutable-object-cache-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:33:43.976 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:33:43.976 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:33:43.976 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T00:33:43.976 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T00:33:43.976 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T00:33:43.976 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-modules-core-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T00:33:43.976 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T00:33:43.976 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:33:43.976 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:33:43.976 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-prometheus-alerts-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T00:33:43.976 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:33:43.976 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:33:43.976 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-test-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:33:43.977 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T00:33:43.977 INFO:teuthology.orchestra.run.smithi157.stdout: cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T00:33:43.977 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:33:43.977 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-shell-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:33:43.977 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-top-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T00:33:43.977 INFO:teuthology.orchestra.run.smithi157.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-09T00:33:43.977 INFO:teuthology.orchestra.run.smithi157.stdout: flex-2.6.4-9.el9.x86_64 2024-05-09T00:33:43.977 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-09T00:33:43.977 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-09T00:33:43.977 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-09T00:33:43.977 INFO:teuthology.orchestra.run.smithi157.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-09T00:33:43.977 INFO:teuthology.orchestra.run.smithi157.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-09T00:33:43.977 INFO:teuthology.orchestra.run.smithi157.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-09T00:33:43.978 INFO:teuthology.orchestra.run.smithi157.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-09T00:33:43.978 INFO:teuthology.orchestra.run.smithi157.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-09T00:33:43.978 INFO:teuthology.orchestra.run.smithi157.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-09T00:33:43.978 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs-devel-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:33:43.978 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:33:43.978 INFO:teuthology.orchestra.run.smithi157.stdout: libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:33:43.978 INFO:teuthology.orchestra.run.smithi157.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-09T00:33:43.978 INFO:teuthology.orchestra.run.smithi157.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-09T00:33:43.978 INFO:teuthology.orchestra.run.smithi157.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-09T00:33:43.978 INFO:teuthology.orchestra.run.smithi157.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-09T00:33:43.978 INFO:teuthology.orchestra.run.smithi157.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-09T00:33:43.978 INFO:teuthology.orchestra.run.smithi157.stdout: librados-devel-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:33:43.979 INFO:teuthology.orchestra.run.smithi157.stdout: libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:33:43.979 INFO:teuthology.orchestra.run.smithi157.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-09T00:33:43.979 INFO:teuthology.orchestra.run.smithi157.stdout: librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:33:43.979 INFO:teuthology.orchestra.run.smithi157.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-09T00:33:43.979 INFO:teuthology.orchestra.run.smithi157.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-09T00:33:43.979 INFO:teuthology.orchestra.run.smithi157.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-09T00:33:43.979 INFO:teuthology.orchestra.run.smithi157.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-09T00:33:43.979 INFO:teuthology.orchestra.run.smithi157.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-09T00:33:43.979 INFO:teuthology.orchestra.run.smithi157.stdout: lua-5.4.4-4.el9.x86_64 2024-05-09T00:33:43.979 INFO:teuthology.orchestra.run.smithi157.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-09T00:33:43.979 INFO:teuthology.orchestra.run.smithi157.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-09T00:33:43.979 INFO:teuthology.orchestra.run.smithi157.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-09T00:33:43.979 INFO:teuthology.orchestra.run.smithi157.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-09T00:33:43.980 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-09T00:33:43.980 INFO:teuthology.orchestra.run.smithi157.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-09T00:33:43.980 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-13.14-1.el9.x86_64 2024-05-09T00:33:43.980 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-09T00:33:43.980 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-09T00:33:43.980 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-09T00:33:43.980 INFO:teuthology.orchestra.run.smithi157.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-09T00:33:43.980 INFO:teuthology.orchestra.run.smithi157.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-09T00:33:43.980 INFO:teuthology.orchestra.run.smithi157.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-09T00:33:43.980 INFO:teuthology.orchestra.run.smithi157.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-09T00:33:43.980 INFO:teuthology.orchestra.run.smithi157.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-09T00:33:43.980 INFO:teuthology.orchestra.run.smithi157.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-09T00:33:43.980 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-09T00:33:43.981 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-argparse-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:33:43.981 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:33:43.981 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cephfs-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:33:43.981 INFO:teuthology.orchestra.run.smithi157.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-09T00:33:43.981 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-09T00:33:43.981 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-09T00:33:43.981 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-09T00:33:43.981 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-09T00:33:43.981 INFO:teuthology.orchestra.run.smithi157.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-09T00:33:43.981 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-09T00:33:43.981 INFO:teuthology.orchestra.run.smithi157.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-09T00:33:43.981 INFO:teuthology.orchestra.run.smithi157.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-09T00:33:43.981 INFO:teuthology.orchestra.run.smithi157.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-09T00:33:43.981 INFO:teuthology.orchestra.run.smithi157.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-09T00:33:43.982 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-09T00:33:43.982 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-09T00:33:43.982 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-09T00:33:43.982 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-09T00:33:43.982 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-09T00:33:43.982 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-09T00:33:43.982 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-09T00:33:43.982 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-09T00:33:43.982 INFO:teuthology.orchestra.run.smithi157.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-09T00:33:43.982 INFO:teuthology.orchestra.run.smithi157.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-09T00:33:43.982 INFO:teuthology.orchestra.run.smithi157.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-09T00:33:43.982 INFO:teuthology.orchestra.run.smithi157.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-09T00:33:43.982 INFO:teuthology.orchestra.run.smithi157.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-09T00:33:43.983 INFO:teuthology.orchestra.run.smithi157.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-09T00:33:43.983 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-09T00:33:43.983 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-09T00:33:43.983 INFO:teuthology.orchestra.run.smithi157.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-09T00:33:43.983 INFO:teuthology.orchestra.run.smithi157.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-09T00:33:43.983 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-09T00:33:43.983 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ply-3.11-14.el9.noarch 2024-05-09T00:33:43.983 INFO:teuthology.orchestra.run.smithi157.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-09T00:33:43.983 INFO:teuthology.orchestra.run.smithi157.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-09T00:33:43.983 INFO:teuthology.orchestra.run.smithi157.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-09T00:33:43.983 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-09T00:33:43.983 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-09T00:33:43.983 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-09T00:33:43.984 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-09T00:33:43.984 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-09T00:33:43.984 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-09T00:33:43.984 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rados-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:33:43.984 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rbd-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:33:43.984 INFO:teuthology.orchestra.run.smithi157.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-09T00:33:43.984 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-09T00:33:43.984 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-09T00:33:43.984 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rgw-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:33:43.984 INFO:teuthology.orchestra.run.smithi157.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-09T00:33:43.984 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-09T00:33:43.984 INFO:teuthology.orchestra.run.smithi157.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-09T00:33:43.985 INFO:teuthology.orchestra.run.smithi157.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-09T00:33:43.985 INFO:teuthology.orchestra.run.smithi157.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-09T00:33:43.985 INFO:teuthology.orchestra.run.smithi157.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-09T00:33:43.985 INFO:teuthology.orchestra.run.smithi157.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-09T00:33:43.985 INFO:teuthology.orchestra.run.smithi157.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-09T00:33:43.985 INFO:teuthology.orchestra.run.smithi157.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-09T00:33:43.985 INFO:teuthology.orchestra.run.smithi157.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-09T00:33:43.985 INFO:teuthology.orchestra.run.smithi157.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-09T00:33:43.985 INFO:teuthology.orchestra.run.smithi157.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-09T00:33:43.985 INFO:teuthology.orchestra.run.smithi157.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-09T00:33:43.985 INFO:teuthology.orchestra.run.smithi157.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-09T00:33:43.985 INFO:teuthology.orchestra.run.smithi157.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-09T00:33:43.985 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:33:43.986 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:33:43.986 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-nbd-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:33:43.986 INFO:teuthology.orchestra.run.smithi157.stdout: re2-1:20211101-3.el9.x86_64 2024-05-09T00:33:43.986 INFO:teuthology.orchestra.run.smithi157.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-09T00:33:43.986 INFO:teuthology.orchestra.run.smithi157.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-09T00:33:43.986 INFO:teuthology.orchestra.run.smithi157.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-09T00:33:43.986 INFO:teuthology.orchestra.run.smithi157.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-09T00:33:43.986 INFO:teuthology.orchestra.run.smithi157.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-09T00:33:43.986 INFO:teuthology.orchestra.run.smithi157.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-09T00:33:43.987 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-05-09T00:33:43.987 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-05-09T00:33:44.005 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-09T00:33:44.038 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-09T00:33:44.065 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-09T00:33:44.380 DEBUG:teuthology.parallel:result is None 2024-05-09T00:33:44.664 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 125/164 2024-05-09T00:33:44.696 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 125/164 2024-05-09T00:33:44.729 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3594.gc9da4049.e 126/164 2024-05-09T00:33:44.794 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3594.gc9da4049. 127/164 2024-05-09T00:33:44.819 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noa 128/164 2024-05-09T00:33:45.913 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noa 128/164 2024-05-09T00:33:45.946 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.n 129/164 2024-05-09T00:33:46.960 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.n 129/164 2024-05-09T00:33:46.988 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9d 130/164 2024-05-09T00:33:47.106 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9d 130/164 2024-05-09T00:33:47.279 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3594.gc9da4049.el 131/164 2024-05-09T00:33:47.300 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 132/164 2024-05-09T00:33:47.323 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 132/164 2024-05-09T00:33:47.324 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T00:33:47.324 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-09T00:33:47.324 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T00:33:47.324 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T00:33:47.324 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:33:47.374 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 133/164 2024-05-09T00:33:47.387 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 133/164 2024-05-09T00:33:47.488 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-2:19.0.0-3594.gc9da4049.el9.x86_64 134/164 2024-05-09T00:33:47.555 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 135/164 2024-05-09T00:33:48.542 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-09T00:33:48.577 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 137/164 2024-05-09T00:33:48.601 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 137/164 2024-05-09T00:33:48.601 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T00:33:48.601 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-09T00:33:48.602 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T00:33:48.602 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T00:33:48.602 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:33:48.644 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3594.gc9da4 138/164 2024-05-09T00:33:48.668 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3594.gc9da4 138/164 2024-05-09T00:33:48.668 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T00:33:48.668 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-09T00:33:48.668 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:33:48.720 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 139/164 2024-05-09T00:33:48.744 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 139/164 2024-05-09T00:33:48.744 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T00:33:48.744 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-09T00:33:48.745 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T00:33:48.745 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T00:33:48.745 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:33:49.070 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 140/164 2024-05-09T00:33:49.095 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 140/164 2024-05-09T00:33:49.096 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T00:33:49.096 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-09T00:33:49.096 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T00:33:49.096 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T00:33:49.096 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:33:53.625 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-test-2:19.0.0-3594.gc9da4049.el9.x86_64 141/164 2024-05-09T00:33:53.666 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : rbd-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 142/164 2024-05-09T00:33:53.707 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : rbd-nbd-2:19.0.0-3594.gc9da4049.el9.x86_64 143/164 2024-05-09T00:33:53.738 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-09T00:33:53.765 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-09T00:33:53.974 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : cephfs-shell-2:19.0.0-3594.gc9da4049.el9.x86_64 146/164 2024-05-09T00:33:54.037 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-09T00:33:54.067 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libcephfs-devel-2:19.0.0-3594.gc9da4049.el9.x86_ 148/164 2024-05-09T00:33:54.097 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-09T00:33:54.143 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-09T00:33:54.755 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T00:33:54.799 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T00:33:54.820 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T00:33:55.712 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : cephfs-top-2:19.0.0-3594.gc9da4049.el9.noarch 152/164 2024-05-09T00:33:55.804 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-09T00:33:55.853 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-09T00:33:55.938 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-09T00:33:55.973 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-09T00:33:55.999 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-09T00:33:55.999 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T00:33:56.092 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T00:33:56.142 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-09T00:33:56.143 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T00:33:56.226 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T00:33:56.260 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-09T00:33:56.266 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T00:33:56.266 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T00:33:56.453 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T00:33:56.453 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T00:33:56.546 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T00:33:56.614 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T00:34:02.142 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T00:34:02.142 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-2:19.0.0-3594.gc9da4049.el9.x86_64 1/164 2024-05-09T00:34:02.142 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 2/164 2024-05-09T00:34:02.142 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 3/164 2024-05-09T00:34:02.142 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 4/164 2024-05-09T00:34:02.142 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3594.gc9da4 5/164 2024-05-09T00:34:02.142 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 6/164 2024-05-09T00:34:02.142 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 7/164 2024-05-09T00:34:02.143 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 8/164 2024-05-09T00:34:02.143 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 9/164 2024-05-09T00:34:02.143 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 10/164 2024-05-09T00:34:02.143 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 11/164 2024-05-09T00:34:02.143 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-test-2:19.0.0-3594.gc9da4049.el9.x86_64 12/164 2024-05-09T00:34:02.143 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 13/164 2024-05-09T00:34:02.143 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : cephfs-shell-2:19.0.0-3594.gc9da4049.el9.x86_64 14/164 2024-05-09T00:34:02.143 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libcephfs-devel-2:19.0.0-3594.gc9da4049.el9.x86_ 15/164 2024-05-09T00:34:02.143 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 16/164 2024-05-09T00:34:02.143 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 17/164 2024-05-09T00:34:02.143 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librados-devel-2:19.0.0-3594.gc9da4049.el9.x86_6 18/164 2024-05-09T00:34:02.143 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86 19/164 2024-05-09T00:34:02.143 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 20/164 2024-05-09T00:34:02.144 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3594.gc9da4049.el 21/164 2024-05-09T00:34:02.144 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-ceph-common-2:19.0.0-3594.gc9da4049.el9. 22/164 2024-05-09T00:34:02.144 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cephfs-2:19.0.0-3594.gc9da4049.el9.x86_6 23/164 2024-05-09T00:34:02.144 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rados-2:19.0.0-3594.gc9da4049.el9.x86_64 24/164 2024-05-09T00:34:02.144 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rbd-2:19.0.0-3594.gc9da4049.el9.x86_64 25/164 2024-05-09T00:34:02.144 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rgw-2:19.0.0-3594.gc9da4049.el9.x86_64 26/164 2024-05-09T00:34:02.144 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : rbd-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 27/164 2024-05-09T00:34:02.156 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 28/164 2024-05-09T00:34:02.156 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : rbd-nbd-2:19.0.0-3594.gc9da4049.el9.x86_64 29/164 2024-05-09T00:34:02.156 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3594.gc9da4049. 30/164 2024-05-09T00:34:02.156 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noa 31/164 2024-05-09T00:34:02.157 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.n 32/164 2024-05-09T00:34:02.157 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9d 33/164 2024-05-09T00:34:02.157 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3594.gc9da4049.el 34/164 2024-05-09T00:34:02.157 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 35/164 2024-05-09T00:34:02.157 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3594.gc9da4049.e 36/164 2024-05-09T00:34:02.157 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 37/164 2024-05-09T00:34:02.157 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 38/164 2024-05-09T00:34:02.157 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : cephfs-top-2:19.0.0-3594.gc9da4049.el9.noarch 39/164 2024-05-09T00:34:02.157 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-09T00:34:02.157 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-09T00:34:02.157 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-09T00:34:02.157 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-09T00:34:02.157 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-09T00:34:02.158 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-09T00:34:02.158 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-09T00:34:02.158 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-09T00:34:02.158 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-09T00:34:02.158 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-09T00:34:02.158 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-09T00:34:02.158 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-09T00:34:02.158 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-09T00:34:02.158 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-09T00:34:02.158 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-09T00:34:02.158 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-09T00:34:02.158 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-09T00:34:02.159 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-09T00:34:02.159 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-09T00:34:02.159 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-09T00:34:02.159 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-09T00:34:02.159 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-09T00:34:02.159 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-09T00:34:02.159 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-09T00:34:02.159 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-09T00:34:02.159 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-09T00:34:02.159 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-09T00:34:02.159 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-09T00:34:02.159 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-09T00:34:02.159 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-09T00:34:02.160 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-09T00:34:02.160 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-09T00:34:02.160 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-09T00:34:02.160 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-09T00:34:02.160 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-09T00:34:02.160 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-09T00:34:02.160 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-09T00:34:02.160 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-09T00:34:02.160 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-09T00:34:02.160 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-09T00:34:02.160 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-09T00:34:02.160 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-09T00:34:02.160 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-09T00:34:02.161 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-09T00:34:02.161 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-09T00:34:02.161 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-09T00:34:02.161 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-09T00:34:02.161 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-09T00:34:02.161 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-09T00:34:02.161 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-09T00:34:02.161 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-09T00:34:02.161 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-09T00:34:02.161 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-09T00:34:02.161 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-09T00:34:02.161 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-09T00:34:02.161 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-09T00:34:02.161 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-09T00:34:02.162 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-09T00:34:02.162 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-09T00:34:02.162 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-09T00:34:02.162 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-09T00:34:02.162 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-09T00:34:02.162 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-09T00:34:02.162 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-09T00:34:02.162 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-09T00:34:02.162 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-09T00:34:02.162 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-09T00:34:02.162 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-09T00:34:02.162 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-09T00:34:02.162 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-09T00:34:02.163 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-09T00:34:02.163 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-09T00:34:02.163 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-09T00:34:02.163 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-09T00:34:02.163 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-09T00:34:02.163 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-09T00:34:02.163 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-09T00:34:02.163 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-09T00:34:02.163 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-09T00:34:02.163 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-09T00:34:02.163 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-09T00:34:02.163 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-09T00:34:02.164 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-09T00:34:02.164 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-09T00:34:02.164 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-09T00:34:02.164 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-09T00:34:02.164 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-09T00:34:02.164 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-09T00:34:02.164 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-09T00:34:02.164 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-09T00:34:02.164 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-09T00:34:02.164 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-09T00:34:02.164 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-09T00:34:02.164 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-09T00:34:02.164 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-09T00:34:02.165 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-09T00:34:02.165 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-09T00:34:02.165 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-09T00:34:02.165 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-09T00:34:02.165 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-09T00:34:02.165 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-09T00:34:02.165 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-09T00:34:02.165 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-09T00:34:02.165 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-09T00:34:02.165 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-09T00:34:02.165 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-09T00:34:02.165 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-09T00:34:02.166 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-09T00:34:02.166 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-09T00:34:02.166 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 149/164 2024-05-09T00:34:02.166 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-09T00:34:02.166 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 151/164 2024-05-09T00:34:02.166 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-09T00:34:02.166 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-09T00:34:02.166 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-09T00:34:02.166 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-09T00:34:02.166 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-09T00:34:02.166 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-09T00:34:02.166 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-09T00:34:02.166 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-09T00:34:02.167 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-09T00:34:02.167 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-09T00:34:02.167 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-09T00:34:02.167 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-09T00:34:05.123 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T00:34:05.124 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:34:05.124 INFO:teuthology.orchestra.run.smithi005.stdout:Upgraded: 2024-05-09T00:34:05.124 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-09T00:34:05.124 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-09T00:34:05.124 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-09T00:34:05.124 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-09T00:34:05.125 INFO:teuthology.orchestra.run.smithi005.stdout: librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:34:05.125 INFO:teuthology.orchestra.run.smithi005.stdout: librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:34:05.125 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-09T00:34:05.125 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-09T00:34:05.125 INFO:teuthology.orchestra.run.smithi005.stdout:Installed: 2024-05-09T00:34:05.125 INFO:teuthology.orchestra.run.smithi005.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-09T00:34:05.126 INFO:teuthology.orchestra.run.smithi005.stdout: automake-1.16.2-8.el9.noarch 2024-05-09T00:34:05.126 INFO:teuthology.orchestra.run.smithi005.stdout: bison-3.7.4-5.el9.x86_64 2024-05-09T00:34:05.126 INFO:teuthology.orchestra.run.smithi005.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-09T00:34:05.126 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:34:05.126 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:34:05.127 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:34:05.127 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:34:05.127 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-grafana-dashboards-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T00:34:05.127 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-immutable-object-cache-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:34:05.127 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:34:05.128 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:34:05.128 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T00:34:05.128 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T00:34:05.128 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T00:34:05.128 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-modules-core-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T00:34:05.129 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T00:34:05.129 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:34:05.129 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:34:05.129 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-prometheus-alerts-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T00:34:05.129 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:34:05.130 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:34:05.130 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-test-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:34:05.130 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T00:34:05.130 INFO:teuthology.orchestra.run.smithi005.stdout: cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T00:34:05.131 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:34:05.131 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-shell-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:34:05.131 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-top-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T00:34:05.131 INFO:teuthology.orchestra.run.smithi005.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-09T00:34:05.131 INFO:teuthology.orchestra.run.smithi005.stdout: flex-2.6.4-9.el9.x86_64 2024-05-09T00:34:05.131 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-09T00:34:05.132 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-09T00:34:05.132 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-09T00:34:05.132 INFO:teuthology.orchestra.run.smithi005.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-09T00:34:05.132 INFO:teuthology.orchestra.run.smithi005.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-09T00:34:05.132 INFO:teuthology.orchestra.run.smithi005.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-09T00:34:05.132 INFO:teuthology.orchestra.run.smithi005.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-09T00:34:05.132 INFO:teuthology.orchestra.run.smithi005.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-09T00:34:05.132 INFO:teuthology.orchestra.run.smithi005.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-09T00:34:05.132 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs-devel-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:34:05.132 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:34:05.132 INFO:teuthology.orchestra.run.smithi005.stdout: libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:34:05.133 INFO:teuthology.orchestra.run.smithi005.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-09T00:34:05.133 INFO:teuthology.orchestra.run.smithi005.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-09T00:34:05.133 INFO:teuthology.orchestra.run.smithi005.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-09T00:34:05.133 INFO:teuthology.orchestra.run.smithi005.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-09T00:34:05.133 INFO:teuthology.orchestra.run.smithi005.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-09T00:34:05.133 INFO:teuthology.orchestra.run.smithi005.stdout: librados-devel-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:34:05.133 INFO:teuthology.orchestra.run.smithi005.stdout: libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:34:05.133 INFO:teuthology.orchestra.run.smithi005.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-09T00:34:05.133 INFO:teuthology.orchestra.run.smithi005.stdout: librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:34:05.133 INFO:teuthology.orchestra.run.smithi005.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-09T00:34:05.133 INFO:teuthology.orchestra.run.smithi005.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-09T00:34:05.133 INFO:teuthology.orchestra.run.smithi005.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-09T00:34:05.133 INFO:teuthology.orchestra.run.smithi005.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-09T00:34:05.134 INFO:teuthology.orchestra.run.smithi005.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-09T00:34:05.134 INFO:teuthology.orchestra.run.smithi005.stdout: lua-5.4.4-4.el9.x86_64 2024-05-09T00:34:05.134 INFO:teuthology.orchestra.run.smithi005.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-09T00:34:05.134 INFO:teuthology.orchestra.run.smithi005.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-09T00:34:05.134 INFO:teuthology.orchestra.run.smithi005.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-09T00:34:05.134 INFO:teuthology.orchestra.run.smithi005.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-09T00:34:05.134 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-09T00:34:05.134 INFO:teuthology.orchestra.run.smithi005.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-09T00:34:05.134 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-13.14-1.el9.x86_64 2024-05-09T00:34:05.134 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-09T00:34:05.134 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-09T00:34:05.134 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-09T00:34:05.135 INFO:teuthology.orchestra.run.smithi005.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-09T00:34:05.135 INFO:teuthology.orchestra.run.smithi005.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-09T00:34:05.135 INFO:teuthology.orchestra.run.smithi005.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-09T00:34:05.135 INFO:teuthology.orchestra.run.smithi005.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-09T00:34:05.135 INFO:teuthology.orchestra.run.smithi005.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-09T00:34:05.135 INFO:teuthology.orchestra.run.smithi005.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-09T00:34:05.135 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-09T00:34:05.135 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-argparse-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:34:05.135 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:34:05.135 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cephfs-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:34:05.135 INFO:teuthology.orchestra.run.smithi005.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-09T00:34:05.135 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-09T00:34:05.135 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-09T00:34:05.136 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-09T00:34:05.136 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-09T00:34:05.136 INFO:teuthology.orchestra.run.smithi005.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-09T00:34:05.136 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-09T00:34:05.136 INFO:teuthology.orchestra.run.smithi005.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-09T00:34:05.136 INFO:teuthology.orchestra.run.smithi005.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-09T00:34:05.136 INFO:teuthology.orchestra.run.smithi005.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-09T00:34:05.136 INFO:teuthology.orchestra.run.smithi005.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-09T00:34:05.136 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-09T00:34:05.136 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-09T00:34:05.136 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-09T00:34:05.136 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-09T00:34:05.136 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-09T00:34:05.137 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-09T00:34:05.137 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-09T00:34:05.137 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-09T00:34:05.137 INFO:teuthology.orchestra.run.smithi005.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-09T00:34:05.137 INFO:teuthology.orchestra.run.smithi005.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-09T00:34:05.137 INFO:teuthology.orchestra.run.smithi005.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-09T00:34:05.137 INFO:teuthology.orchestra.run.smithi005.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-09T00:34:05.137 INFO:teuthology.orchestra.run.smithi005.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-09T00:34:05.137 INFO:teuthology.orchestra.run.smithi005.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-09T00:34:05.137 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-09T00:34:05.137 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-09T00:34:05.137 INFO:teuthology.orchestra.run.smithi005.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-09T00:34:05.137 INFO:teuthology.orchestra.run.smithi005.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-09T00:34:05.138 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-09T00:34:05.138 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ply-3.11-14.el9.noarch 2024-05-09T00:34:05.138 INFO:teuthology.orchestra.run.smithi005.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-09T00:34:05.138 INFO:teuthology.orchestra.run.smithi005.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-09T00:34:05.138 INFO:teuthology.orchestra.run.smithi005.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-09T00:34:05.138 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-09T00:34:05.138 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-09T00:34:05.138 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-09T00:34:05.138 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-09T00:34:05.138 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-09T00:34:05.138 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-09T00:34:05.138 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rados-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:34:05.138 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rbd-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:34:05.139 INFO:teuthology.orchestra.run.smithi005.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-09T00:34:05.139 INFO:teuthology.orchestra.run.smithi005.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-09T00:34:05.139 INFO:teuthology.orchestra.run.smithi005.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-09T00:34:05.139 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rgw-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:34:05.139 INFO:teuthology.orchestra.run.smithi005.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-09T00:34:05.139 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-09T00:34:05.139 INFO:teuthology.orchestra.run.smithi005.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-09T00:34:05.139 INFO:teuthology.orchestra.run.smithi005.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-09T00:34:05.139 INFO:teuthology.orchestra.run.smithi005.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-09T00:34:05.139 INFO:teuthology.orchestra.run.smithi005.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-09T00:34:05.139 INFO:teuthology.orchestra.run.smithi005.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-09T00:34:05.139 INFO:teuthology.orchestra.run.smithi005.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-09T00:34:05.139 INFO:teuthology.orchestra.run.smithi005.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-09T00:34:05.140 INFO:teuthology.orchestra.run.smithi005.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-09T00:34:05.140 INFO:teuthology.orchestra.run.smithi005.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-09T00:34:05.140 INFO:teuthology.orchestra.run.smithi005.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-09T00:34:05.140 INFO:teuthology.orchestra.run.smithi005.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-09T00:34:05.140 INFO:teuthology.orchestra.run.smithi005.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-09T00:34:05.140 INFO:teuthology.orchestra.run.smithi005.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-09T00:34:05.140 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:34:05.140 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:34:05.140 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-nbd-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T00:34:05.140 INFO:teuthology.orchestra.run.smithi005.stdout: re2-1:20211101-3.el9.x86_64 2024-05-09T00:34:05.140 INFO:teuthology.orchestra.run.smithi005.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-09T00:34:05.141 INFO:teuthology.orchestra.run.smithi005.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-09T00:34:05.141 INFO:teuthology.orchestra.run.smithi005.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-09T00:34:05.141 INFO:teuthology.orchestra.run.smithi005.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-09T00:34:05.141 INFO:teuthology.orchestra.run.smithi005.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-09T00:34:05.141 INFO:teuthology.orchestra.run.smithi005.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-09T00:34:05.141 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:34:05.141 INFO:teuthology.orchestra.run.smithi005.stdout:Complete! 2024-05-09T00:34:05.543 DEBUG:teuthology.parallel:result is None 2024-05-09T00:34:05.544 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T00:34:05.692 DEBUG:teuthology.orchestra.run.smithi005:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-09T00:34:05.715 INFO:teuthology.orchestra.run.smithi005.stdout:19.0.0-3594.gc9da4049.el9 2024-05-09T00:34:05.716 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3594.gc9da4049.el9 2024-05-09T00:34:05.716 INFO:teuthology.task.install:The correct ceph version 19.0.0-3594.gc9da4049 is installed. 2024-05-09T00:34:05.718 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T00:34:05.930 DEBUG:teuthology.orchestra.run.smithi131:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-09T00:34:05.956 INFO:teuthology.orchestra.run.smithi131.stdout:19.0.0-3594.gc9da4049.el9 2024-05-09T00:34:05.956 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3594.gc9da4049.el9 2024-05-09T00:34:05.956 INFO:teuthology.task.install:The correct ceph version 19.0.0-3594.gc9da4049 is installed. 2024-05-09T00:34:05.958 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T00:34:06.177 DEBUG:teuthology.orchestra.run.smithi157:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-09T00:34:06.207 INFO:teuthology.orchestra.run.smithi157.stdout:19.0.0-3594.gc9da4049.el9 2024-05-09T00:34:06.207 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3594.gc9da4049.el9 2024-05-09T00:34:06.207 INFO:teuthology.task.install:The correct ceph version 19.0.0-3594.gc9da4049 is installed. 2024-05-09T00:34:06.209 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-09T00:34:06.209 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-09T00:34:06.209 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-09T00:34:06.240 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-09T00:34:06.240 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-09T00:34:06.272 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-05-09T00:34:06.272 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-09T00:34:06.303 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-09T00:34:06.303 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-09T00:34:06.303 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/usr/bin/daemon-helper 2024-05-09T00:34:06.332 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-09T00:34:06.401 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-09T00:34:06.401 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/usr/bin/daemon-helper 2024-05-09T00:34:06.430 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-09T00:34:06.497 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-05-09T00:34:06.497 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/daemon-helper 2024-05-09T00:34:06.521 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-09T00:34:06.585 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-09T00:34:06.585 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-09T00:34:06.585 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-09T00:34:06.618 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-09T00:34:06.685 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-09T00:34:06.685 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-09T00:34:06.713 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-09T00:34:06.778 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-05-09T00:34:06.778 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-09T00:34:06.802 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-09T00:34:06.866 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-09T00:34:06.867 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-09T00:34:06.867 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/usr/bin/stdin-killer 2024-05-09T00:34:06.896 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-09T00:34:06.964 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-09T00:34:06.964 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/usr/bin/stdin-killer 2024-05-09T00:34:06.987 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-09T00:34:07.053 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-05-09T00:34:07.053 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/stdin-killer 2024-05-09T00:34:07.077 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-09T00:34:07.140 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-09T00:34:07.251 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'c9da4049f638c0f84bdd9b4e33c42e86046ac554', 'subvols': {'create': 2}} 2024-05-09T00:34:07.251 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T00:34:07.251 INFO:tasks.cephadm:Cluster fsid is d85814c6-0d9b-11ef-bc97-c7b262605968 2024-05-09T00:34:07.252 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-09T00:34:07.252 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.5', 'mon.b': '172.21.15.131', 'mon.c': '172.21.15.157'} 2024-05-09T00:34:07.252 INFO:tasks.cephadm:First mon is mon.a on smithi005 2024-05-09T00:34:07.252 INFO:tasks.cephadm:First mgr is x 2024-05-09T00:34:07.252 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-09T00:34:07.252 DEBUG:teuthology.orchestra.run.smithi005:> sudo hostname $(hostname -s) 2024-05-09T00:34:07.283 DEBUG:teuthology.orchestra.run.smithi131:> sudo hostname $(hostname -s) 2024-05-09T00:34:07.308 DEBUG:teuthology.orchestra.run.smithi157:> sudo hostname $(hostname -s) 2024-05-09T00:34:07.331 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-05-09T00:34:07.332 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T00:34:07.471 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'c9da4049f638c0f84bdd9b4e33c42e86046ac554', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/79224/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3594-gc9da4049', 'node_name': '172.21.5.39+adami09', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3594.gc9da4049'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-08 19:52:58.989794', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240508.183908-debug', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-05-09T00:34:07.685 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-pdonnell-testing-20240508.183908-debug, sha1 c9da4049f638c0f84bdd9b4e33c42e86046ac554 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T00:34:07.687 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/x86_64/flavors/default/cephadm 2024-05-09T00:34:07.687 DEBUG:teuthology.orchestra.run.smithi005:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-09T00:34:08.278 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-r--r--. 1 ubuntu ubuntu 782809 May 9 00:34 /home/ubuntu/cephtest/cephadm 2024-05-09T00:34:08.278 DEBUG:teuthology.orchestra.run.smithi131:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-09T00:34:08.708 INFO:teuthology.orchestra.run.smithi131.stdout:-rw-r--r--. 1 ubuntu ubuntu 782809 May 9 00:34 /home/ubuntu/cephtest/cephadm 2024-05-09T00:34:08.709 DEBUG:teuthology.orchestra.run.smithi157:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-09T00:34:09.455 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r--. 1 ubuntu ubuntu 782809 May 9 00:34 /home/ubuntu/cephtest/cephadm 2024-05-09T00:34:09.455 DEBUG:teuthology.orchestra.run.smithi005:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-09T00:34:09.476 DEBUG:teuthology.orchestra.run.smithi131:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-09T00:34:09.495 DEBUG:teuthology.orchestra.run.smithi157:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-09T00:34:09.530 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 on all hosts... 2024-05-09T00:34:09.530 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 pull 2024-05-09T00:34:09.533 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 pull 2024-05-09T00:34:09.537 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 pull 2024-05-09T00:34:11.069 INFO:teuthology.orchestra.run.smithi005.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554... 2024-05-09T00:34:11.080 INFO:teuthology.orchestra.run.smithi157.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554... 2024-05-09T00:34:11.137 INFO:teuthology.orchestra.run.smithi131.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554... 2024-05-09T00:34:43.756 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-09T00:34:43.756 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_version": "ceph version 19.0.0-3594-gc9da4049 (c9da4049f638c0f84bdd9b4e33c42e86046ac554) squid (dev)", 2024-05-09T00:34:43.756 INFO:teuthology.orchestra.run.smithi005.stdout: "image_id": "db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7", 2024-05-09T00:34:43.756 INFO:teuthology.orchestra.run.smithi005.stdout: "repo_digests": [ 2024-05-09T00:34:43.756 INFO:teuthology.orchestra.run.smithi005.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584" 2024-05-09T00:34:43.756 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-09T00:34:43.756 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-09T00:34:47.059 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-09T00:34:47.059 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_version": "ceph version 19.0.0-3594-gc9da4049 (c9da4049f638c0f84bdd9b4e33c42e86046ac554) squid (dev)", 2024-05-09T00:34:47.059 INFO:teuthology.orchestra.run.smithi131.stdout: "image_id": "db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7", 2024-05-09T00:34:47.059 INFO:teuthology.orchestra.run.smithi131.stdout: "repo_digests": [ 2024-05-09T00:34:47.059 INFO:teuthology.orchestra.run.smithi131.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584" 2024-05-09T00:34:47.059 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2024-05-09T00:34:47.059 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-09T00:35:05.549 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-05-09T00:35:05.549 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_version": "ceph version 19.0.0-3594-gc9da4049 (c9da4049f638c0f84bdd9b4e33c42e86046ac554) squid (dev)", 2024-05-09T00:35:05.549 INFO:teuthology.orchestra.run.smithi157.stdout: "image_id": "db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7", 2024-05-09T00:35:05.549 INFO:teuthology.orchestra.run.smithi157.stdout: "repo_digests": [ 2024-05-09T00:35:05.549 INFO:teuthology.orchestra.run.smithi157.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584" 2024-05-09T00:35:05.550 INFO:teuthology.orchestra.run.smithi157.stdout: ] 2024-05-09T00:35:05.550 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-05-09T00:35:05.572 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /etc/ceph 2024-05-09T00:35:05.608 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /etc/ceph 2024-05-09T00:35:05.640 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /etc/ceph 2024-05-09T00:35:05.667 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 777 /etc/ceph 2024-05-09T00:35:05.699 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 777 /etc/ceph 2024-05-09T00:35:05.727 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 777 /etc/ceph 2024-05-09T00:35:05.751 INFO:tasks.cephadm:Writing seed config... 2024-05-09T00:35:05.753 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-05-09T00:35:05.753 INFO:tasks.cephadm: override: [client] debug client = 20 2024-05-09T00:35:05.753 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-05-09T00:35:05.753 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-05-09T00:35:05.753 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-05-09T00:35:05.753 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-05-09T00:35:05.753 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-05-09T00:35:05.753 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-05-09T00:35:05.754 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-05-09T00:35:05.754 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-05-09T00:35:05.754 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-05-09T00:35:05.754 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-05-09T00:35:05.754 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-05-09T00:35:05.754 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-05-09T00:35:05.754 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-05-09T00:35:05.754 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-05-09T00:35:05.754 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-05-09T00:35:05.754 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-05-09T00:35:05.754 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-05-09T00:35:05.755 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-05-09T00:35:05.755 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-05-09T00:35:05.755 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-05-09T00:35:05.755 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-09T00:35:05.755 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-09T00:35:05.755 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-09T00:35:05.755 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-09T00:35:05.755 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-09T00:35:05.755 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-05-09T00:35:05.755 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-05-09T00:35:05.755 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-05-09T00:35:05.755 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-05-09T00:35:05.755 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-05-09T00:35:05.756 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-05-09T00:35:05.756 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-05-09T00:35:05.756 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-09T00:35:05.756 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-09T00:35:05.756 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-05-09T00:35:05.756 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-05-09T00:35:05.756 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-05-09T00:35:05.756 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-05-09T00:35:05.756 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-05-09T00:35:05.756 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-05-09T00:35:05.756 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-05-09T00:35:05.756 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-05-09T00:35:05.756 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-05-09T00:35:05.757 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-09T00:35:05.757 DEBUG:teuthology.orchestra.run.smithi005:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-09T00:35:05.774 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = d85814c6-0d9b-11ef-bc97-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-05-09T00:35:05.775 DEBUG:teuthology.orchestra.run.smithi005:mon.a> sudo journalctl -f -n 0 -u ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@mon.a.service 2024-05-09T00:35:05.818 DEBUG:teuthology.orchestra.run.smithi005:mgr.x> sudo journalctl -f -n 0 -u ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@mgr.x.service 2024-05-09T00:35:05.861 INFO:tasks.cephadm:Bootstrapping... 2024-05-09T00:35:05.861 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 -v bootstrap --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.5 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-09T00:35:06.062 INFO:teuthology.orchestra.run.smithi005.stdout:-------------------------------------------------------------------------------- 2024-05-09T00:35:06.062 INFO:teuthology.orchestra.run.smithi005.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554', '-v', 'bootstrap', '--fsid', 'd85814c6-0d9b-11ef-bc97-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.5', '--skip-admin-label'] 2024-05-09T00:35:06.062 INFO:teuthology.orchestra.run.smithi005.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-09T00:35:06.063 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying podman|docker is present... 2024-05-09T00:35:06.092 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stdout 4.9.4-dev 2024-05-09T00:35:06.092 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying lvm2 is present... 2024-05-09T00:35:06.092 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying time synchronization is in place... 2024-05-09T00:35:06.101 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-09T00:35:06.101 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-09T00:35:06.108 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-09T00:35:06.109 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout inactive 2024-05-09T00:35:06.116 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout enabled 2024-05-09T00:35:06.124 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout active 2024-05-09T00:35:06.124 INFO:teuthology.orchestra.run.smithi005.stdout:Unit chronyd.service is enabled and running 2024-05-09T00:35:06.124 INFO:teuthology.orchestra.run.smithi005.stdout:Repeating the final host check... 2024-05-09T00:35:06.150 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stdout 4.9.4-dev 2024-05-09T00:35:06.151 INFO:teuthology.orchestra.run.smithi005.stdout:podman (/bin/podman) version 4.9.4 is present 2024-05-09T00:35:06.151 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl is present 2024-05-09T00:35:06.151 INFO:teuthology.orchestra.run.smithi005.stdout:lvcreate is present 2024-05-09T00:35:06.157 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-09T00:35:06.157 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-09T00:35:06.164 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-09T00:35:06.164 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout inactive 2024-05-09T00:35:06.171 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout enabled 2024-05-09T00:35:06.179 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout active 2024-05-09T00:35:06.179 INFO:teuthology.orchestra.run.smithi005.stdout:Unit chronyd.service is enabled and running 2024-05-09T00:35:06.179 INFO:teuthology.orchestra.run.smithi005.stdout:Host looks OK 2024-05-09T00:35:06.179 INFO:teuthology.orchestra.run.smithi005.stdout:Cluster fsid: d85814c6-0d9b-11ef-bc97-c7b262605968 2024-05-09T00:35:06.179 INFO:teuthology.orchestra.run.smithi005.stdout:Acquiring lock 140549099789760 on /run/cephadm/d85814c6-0d9b-11ef-bc97-c7b262605968.lock 2024-05-09T00:35:06.179 INFO:teuthology.orchestra.run.smithi005.stdout:Lock 140549099789760 acquired on /run/cephadm/d85814c6-0d9b-11ef-bc97-c7b262605968.lock 2024-05-09T00:35:06.179 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying IP 172.21.15.5 port 3300 ... 2024-05-09T00:35:06.180 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying IP 172.21.15.5 port 6789 ... 2024-05-09T00:35:06.180 INFO:teuthology.orchestra.run.smithi005.stdout:Base mon IP(s) is [172.21.15.5:3300, 172.21.15.5:6789], mon addrv is [v2:172.21.15.5:3300,v1:172.21.15.5:6789] 2024-05-09T00:35:06.183 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.5 metric 100 2024-05-09T00:35:06.183 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.5 metric 100 2024-05-09T00:35:06.187 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-09T00:35:06.187 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-05-09T00:35:06.187 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-05-09T00:35:06.189 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-09T00:35:06.189 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-09T00:35:06.189 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-09T00:35:06.189 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-05-09T00:35:06.189 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1442/64 scope link noprefixroute 2024-05-09T00:35:06.189 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-09T00:35:06.190 INFO:teuthology.orchestra.run.smithi005.stdout:Mon IP `172.21.15.5` is in CIDR network `172.21.0.0/20` 2024-05-09T00:35:06.190 INFO:teuthology.orchestra.run.smithi005.stdout:Mon IP `172.21.15.5` is in CIDR network `172.21.0.0/20` 2024-05-09T00:35:06.190 INFO:teuthology.orchestra.run.smithi005.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-09T00:35:06.191 INFO:teuthology.orchestra.run.smithi005.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-09T00:35:06.191 INFO:teuthology.orchestra.run.smithi005.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554... 2024-05-09T00:35:06.667 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stdout db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 2024-05-09T00:35:06.667 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554... 2024-05-09T00:35:06.667 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Getting image source signatures 2024-05-09T00:35:06.667 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Copying blob sha256:bf5b95bd68251da39c6cd2eb96c6ba4138bb37a62bdc39287908b25e92047d7e 2024-05-09T00:35:06.667 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Copying blob sha256:0411e0fd7a5d45f8ab31b23cec89f97c70494de39461f644c7cdc77de3b9cbca 2024-05-09T00:35:06.668 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Copying config sha256:db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 2024-05-09T00:35:06.668 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-09T00:35:07.590 INFO:teuthology.orchestra.run.smithi005.stdout:ceph: stdout ceph version 19.0.0-3594-gc9da4049 (c9da4049f638c0f84bdd9b4e33c42e86046ac554) squid (dev) 2024-05-09T00:35:07.590 INFO:teuthology.orchestra.run.smithi005.stdout:Ceph version: ceph version 19.0.0-3594-gc9da4049 (c9da4049f638c0f84bdd9b4e33c42e86046ac554) squid (dev) 2024-05-09T00:35:07.590 INFO:teuthology.orchestra.run.smithi005.stdout:Extracting ceph user uid/gid from container image... 2024-05-09T00:35:15.109 INFO:teuthology.orchestra.run.smithi005.stdout:stat: stdout 167 167 2024-05-09T00:35:15.109 INFO:teuthology.orchestra.run.smithi005.stdout:Creating initial keys... 2024-05-09T00:35:19.424 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph-authtool: stdout AQBFGjxmAvdFMBAAHJhA2YfjklHwxO4dpzAsIg== 2024-05-09T00:35:20.339 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph-authtool: stdout AQBHGjxm+VKZLhAAbULbyy4/5FAgVcaHeu9Ogg== 2024-05-09T00:35:21.760 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph-authtool: stdout AQBJGjxmKLW5CxAAIJncI3V+KR0hJs4xMn1+cQ== 2024-05-09T00:35:21.761 INFO:teuthology.orchestra.run.smithi005.stdout:Creating initial monmap... 2024-05-09T00:35:22.909 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-09T00:35:22.909 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-05-09T00:35:22.909 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to d85814c6-0d9b-11ef-bc97-c7b262605968 2024-05-09T00:35:22.909 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-09T00:35:22.909 INFO:teuthology.orchestra.run.smithi005.stdout:monmaptool for a [v2:172.21.15.5:3300,v1:172.21.15.5:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-09T00:35:22.909 INFO:teuthology.orchestra.run.smithi005.stdout:setting min_mon_release = quincy 2024-05-09T00:35:22.909 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/monmaptool: set fsid to d85814c6-0d9b-11ef-bc97-c7b262605968 2024-05-09T00:35:22.909 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-09T00:35:22.909 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:35:22.910 INFO:teuthology.orchestra.run.smithi005.stdout:Creating mon... 2024-05-09T00:35:23.890 INFO:teuthology.orchestra.run.smithi005.stdout:create mon.a on 2024-05-09T00:35:24.106 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-05-09T00:35:24.336 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-09T00:35:24.686 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-d85814c6-0d9b-11ef-bc97-c7b262605968.target → /etc/systemd/system/ceph-d85814c6-0d9b-11ef-bc97-c7b262605968.target. 2024-05-09T00:35:24.687 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-d85814c6-0d9b-11ef-bc97-c7b262605968.target → /etc/systemd/system/ceph-d85814c6-0d9b-11ef-bc97-c7b262605968.target. 2024-05-09T00:35:24.932 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@mon.a 2024-05-09T00:35:24.932 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Failed to reset failed state of unit ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@mon.a.service: Unit ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@mon.a.service not loaded. 2024-05-09T00:35:25.103 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-d85814c6-0d9b-11ef-bc97-c7b262605968.target.wants/ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@mon.a.service → /etc/systemd/system/ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@.service. 2024-05-09T00:35:25.440 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:25 smithi005 podman[29975]: 2024-05-09 00:35:25.253949362 +0000 UTC m=+0.018708419 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T00:35:25.665 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-09T00:35:25.666 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout disabled 2024-05-09T00:35:25.673 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-09T00:35:25.673 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout inactive 2024-05-09T00:35:25.673 INFO:teuthology.orchestra.run.smithi005.stdout:firewalld.service is not enabled 2024-05-09T00:35:25.673 INFO:teuthology.orchestra.run.smithi005.stdout:Not possible to enable service . firewalld.service is not available 2024-05-09T00:35:25.674 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for mon to start... 2024-05-09T00:35:25.674 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for mon... 2024-05-09T00:35:25.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:25 smithi005 podman[29975]: 2024-05-09 00:35:25.566643819 +0000 UTC m=+0.331402873 container init c42607cbc92705ce9647a5f04c1e1ead19a47f872c6e3a90d1c31f64dca2ebf1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T00:35:25.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:25 smithi005 podman[29975]: 2024-05-09 00:35:25.570590658 +0000 UTC m=+0.335349704 container start c42607cbc92705ce9647a5f04c1e1ead19a47f872c6e3a90d1c31f64dca2ebf1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mon-a, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T00:35:25.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:25 smithi005 ceph-mon[29995]: mkfs d85814c6-0d9b-11ef-bc97-c7b262605968 2024-05-09T00:35:25.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:25 smithi005 ceph-mon[29995]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-09T00:35:25.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:25 smithi005 bash[29975]: c42607cbc92705ce9647a5f04c1e1ead19a47f872c6e3a90d1c31f64dca2ebf1 2024-05-09T00:35:25.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:25 smithi005 systemd[1]: Started Ceph mon.a for d85814c6-0d9b-11ef-bc97-c7b262605968. 2024-05-09T00:35:26.825 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:26 smithi005 ceph-mon[29995]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-09T00:35:26.825 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:26 smithi005 ceph-mon[29995]: monmap epoch 1 2024-05-09T00:35:26.825 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:26 smithi005 ceph-mon[29995]: fsid d85814c6-0d9b-11ef-bc97-c7b262605968 2024-05-09T00:35:26.825 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:26 smithi005 ceph-mon[29995]: last_changed 2024-05-09T00:35:22.099371+0000 2024-05-09T00:35:26.825 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:26 smithi005 ceph-mon[29995]: created 2024-05-09T00:35:22.099371+0000 2024-05-09T00:35:26.825 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:26 smithi005 ceph-mon[29995]: min_mon_release 19 (squid) 2024-05-09T00:35:26.825 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:26 smithi005 ceph-mon[29995]: election_strategy: 1 2024-05-09T00:35:26.825 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:26 smithi005 ceph-mon[29995]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-05-09T00:35:26.826 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:26 smithi005 ceph-mon[29995]: fsmap 2024-05-09T00:35:26.826 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:26 smithi005 ceph-mon[29995]: osdmap e1: 0 total, 0 up, 0 in 2024-05-09T00:35:26.826 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:26 smithi005 ceph-mon[29995]: mgrmap e1: no daemons active 2024-05-09T00:35:26.826 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:26 smithi005 ceph-mon[29995]: from='client.? 172.21.15.5:0/2797926696' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-05-09T00:35:26.833 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout cluster: 2024-05-09T00:35:26.833 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout id: d85814c6-0d9b-11ef-bc97-c7b262605968 2024-05-09T00:35:26.833 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-09T00:35:26.833 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-05-09T00:35:26.833 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout services: 2024-05-09T00:35:26.833 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.602686s) 2024-05-09T00:35:26.833 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-09T00:35:26.833 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-09T00:35:26.833 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-05-09T00:35:26.833 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout data: 2024-05-09T00:35:26.833 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-09T00:35:26.834 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-09T00:35:26.834 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-09T00:35:26.834 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout pgs: 2024-05-09T00:35:26.834 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-05-09T00:35:26.834 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:26.140+0000 7f0b25d26700 1 Processor -- start 2024-05-09T00:35:26.834 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:26.140+0000 7f0b25d26700 1 -- start start 2024-05-09T00:35:26.834 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:26.140+0000 7f0b25d26700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b2010a560 0x7f0b2010c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:35:26.834 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:26.140+0000 7f0b25d26700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2007d6c0 con 0x7f0b2010a560 2024-05-09T00:35:26.834 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:26.141+0000 7f0b1f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b2010a560 0x7f0b2010c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:35:26.834 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:26.141+0000 7f0b1f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b2010a560 0x7f0b2010c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53222/0 (socket says 172.21.15.5:53222) 2024-05-09T00:35:26.834 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:26.141+0000 7f0b1f7fe700 1 -- 172.21.15.5:0/2839126640 learned_addr learned my addr 172.21.15.5:0/2839126640 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:35:26.834 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:26.141+0000 7f0b1f7fe700 1 -- 172.21.15.5:0/2839126640 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b2010cec0 con 0x7f0b2010a560 2024-05-09T00:35:26.835 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:26.141+0000 7f0b1f7fe700 1 --2- 172.21.15.5:0/2839126640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b2010a560 0x7f0b2010c980 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f0b100098a0 tx=0x7f0b10009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abd01031a270c03c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:35:26.835 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:26.141+0000 7f0b1e7fc700 1 -- 172.21.15.5:0/2839126640 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b10004160 con 0x7f0b2010a560 2024-05-09T00:35:26.835 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:26.141+0000 7f0b1e7fc700 1 -- 172.21.15.5:0/2839126640 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b10004300 con 0x7f0b2010a560 2024-05-09T00:35:26.835 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:26.141+0000 7f0b1e7fc700 1 -- 172.21.15.5:0/2839126640 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b100036a0 con 0x7f0b2010a560 2024-05-09T00:35:26.835 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:26.142+0000 7f0b25d26700 1 -- 172.21.15.5:0/2839126640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b2010a560 msgr2=0x7f0b2010c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:35:26.835 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:26.142+0000 7f0b25d26700 1 --2- 172.21.15.5:0/2839126640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b2010a560 0x7f0b2010c980 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f0b100098a0 tx=0x7f0b10009bb0 comp rx=0 tx=0).stop 2024-05-09T00:35:26.835 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:26.142+0000 7f0b25d26700 1 -- 172.21.15.5:0/2839126640 shutdown_connections 2024-05-09T00:35:26.835 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:26.142+0000 7f0b25d26700 1 --2- 172.21.15.5:0/2839126640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b2010a560 0x7f0b2010c980 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:35:26.835 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:26.142+0000 7f0b25d26700 1 -- 172.21.15.5:0/2839126640 >> 172.21.15.5:0/2839126640 conn(0x7f0b20104000 msgr2=0x7f0b20106450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:35:26.835 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:26.142+0000 7f0b25d26700 1 -- 172.21.15.5:0/2839126640 shutdown_connections 2024-05-09T00:35:26.835 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:26.142+0000 7f0b25d26700 1 -- 172.21.15.5:0/2839126640 wait complete. 2024-05-09T00:35:26.835 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:26.143+0000 7f0b25d26700 1 Processor -- start 2024-05-09T00:35:26.835 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:26.143+0000 7f0b25d26700 1 -- start start 2024-05-09T00:35:26.835 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:26.143+0000 7f0b25d26700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b2010a560 0x7f0b2019d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:35:26.836 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:26.143+0000 7f0b25d26700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2010d2d0 con 0x7f0b2010a560 2024-05-09T00:35:26.836 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:26.143+0000 7f0b1f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b2010a560 0x7f0b2019d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:35:26.836 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:26.143+0000 7f0b1f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b2010a560 0x7f0b2019d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53226/0 (socket says 172.21.15.5:53226) 2024-05-09T00:35:26.836 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:26.143+0000 7f0b1f7fe700 1 -- 172.21.15.5:0/2797926696 learned_addr learned my addr 172.21.15.5:0/2797926696 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:35:26.836 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:26.144+0000 7f0b1f7fe700 1 -- 172.21.15.5:0/2797926696 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b2019db40 con 0x7f0b2010a560 2024-05-09T00:35:26.836 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:26.144+0000 7f0b1f7fe700 1 --2- 172.21.15.5:0/2797926696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b2010a560 0x7f0b2019d600 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f0b10000c00 tx=0x7f0b10004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:35:26.836 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:26.144+0000 7f0b1cff9700 1 -- 172.21.15.5:0/2797926696 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b10003db0 con 0x7f0b2010a560 2024-05-09T00:35:26.836 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:26.144+0000 7f0b1cff9700 1 -- 172.21.15.5:0/2797926696 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b10011630 con 0x7f0b2010a560 2024-05-09T00:35:26.836 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:26.144+0000 7f0b25d26700 1 -- 172.21.15.5:0/2797926696 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b2019ddd0 con 0x7f0b2010a560 2024-05-09T00:35:26.836 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:26.144+0000 7f0b1cff9700 1 -- 172.21.15.5:0/2797926696 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b10011950 con 0x7f0b2010a560 2024-05-09T00:35:26.836 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:26.144+0000 7f0b25d26700 1 -- 172.21.15.5:0/2797926696 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b201a09e0 con 0x7f0b2010a560 2024-05-09T00:35:26.836 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:26.144+0000 7f0b1cff9700 1 -- 172.21.15.5:0/2797926696 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0b1001a460 con 0x7f0b2010a560 2024-05-09T00:35:26.836 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:26.144+0000 7f0b1cff9700 1 -- 172.21.15.5:0/2797926696 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0b1001b780 con 0x7f0b2010a560 2024-05-09T00:35:26.837 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:26.145+0000 7f0b25d26700 1 -- 172.21.15.5:0/2797926696 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b20064e20 con 0x7f0b2010a560 2024-05-09T00:35:26.837 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:26.148+0000 7f0b1cff9700 1 -- 172.21.15.5:0/2797926696 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f0b1001b980 con 0x7f0b2010a560 2024-05-09T00:35:26.837 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:26.223+0000 7f0b25d26700 1 -- 172.21.15.5:0/2797926696 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f0b2002e8c0 con 0x7f0b2010a560 2024-05-09T00:35:26.837 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:26.223+0000 7f0b1cff9700 1 -- 172.21.15.5:0/2797926696 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f0b1001bb60 con 0x7f0b2010a560 2024-05-09T00:35:26.837 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:26.224+0000 7f0b25d26700 1 -- 172.21.15.5:0/2797926696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b2010a560 msgr2=0x7f0b2019d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:35:26.837 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:26.224+0000 7f0b25d26700 1 --2- 172.21.15.5:0/2797926696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b2010a560 0x7f0b2019d600 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f0b10000c00 tx=0x7f0b10004750 comp rx=0 tx=0).stop 2024-05-09T00:35:26.837 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:26.224+0000 7f0b25d26700 1 -- 172.21.15.5:0/2797926696 shutdown_connections 2024-05-09T00:35:26.837 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:26.224+0000 7f0b25d26700 1 --2- 172.21.15.5:0/2797926696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b2010a560 0x7f0b2019d600 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:35:26.837 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:26.224+0000 7f0b25d26700 1 -- 172.21.15.5:0/2797926696 >> 172.21.15.5:0/2797926696 conn(0x7f0b20104000 msgr2=0x7f0b20105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:35:26.837 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:26.224+0000 7f0b25d26700 1 -- 172.21.15.5:0/2797926696 shutdown_connections 2024-05-09T00:35:26.837 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:26.224+0000 7f0b25d26700 1 -- 172.21.15.5:0/2797926696 wait complete. 2024-05-09T00:35:26.837 INFO:teuthology.orchestra.run.smithi005.stdout:mon is available 2024-05-09T00:35:26.837 INFO:teuthology.orchestra.run.smithi005.stdout:Assimilating anything we can from ceph.conf... 2024-05-09T00:35:27.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:27 smithi005 ceph-mon[29995]: from='client.? 172.21.15.5:0/1708260420' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-09T00:35:27.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:27 smithi005 ceph-mon[29995]: from='client.? 172.21.15.5:0/1708260420' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-09T00:35:27.915 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-05-09T00:35:27.915 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout [global] 2024-05-09T00:35:27.915 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout fsid = d85814c6-0d9b-11ef-bc97-c7b262605968 2024-05-09T00:35:27.915 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-09T00:35:27.915 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.5:3300,v1:172.21.15.5:6789] 2024-05-09T00:35:27.915 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-09T00:35:27.915 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-09T00:35:27.915 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-09T00:35:27.915 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-09T00:35:27.916 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-05-09T00:35:27.916 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-09T00:35:27.916 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-09T00:35:27.916 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-05-09T00:35:27.916 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout [osd] 2024-05-09T00:35:27.916 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-09T00:35:27.916 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-09T00:35:27.916 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-09T00:35:27.916 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-09T00:35:27.916 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-09T00:35:27.916 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:27.337+0000 7fe9f5e6c700 1 Processor -- start 2024-05-09T00:35:27.916 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:27.337+0000 7fe9f5e6c700 1 -- start start 2024-05-09T00:35:27.916 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:27.337+0000 7fe9f5e6c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9f0108970 0x7fe9f0108d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:35:27.917 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:27.337+0000 7fe9f5e6c700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f0109370 con 0x7fe9f0108970 2024-05-09T00:35:27.917 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:27.337+0000 7fe9ef7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9f0108970 0x7fe9f0108d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:35:27.917 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:27.338+0000 7fe9ef7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9f0108970 0x7fe9f0108d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53230/0 (socket says 172.21.15.5:53230) 2024-05-09T00:35:27.917 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:27.338+0000 7fe9ef7fe700 1 -- 172.21.15.5:0/4273265475 learned_addr learned my addr 172.21.15.5:0/4273265475 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:35:27.917 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:27.338+0000 7fe9ef7fe700 1 -- 172.21.15.5:0/4273265475 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9f0109b40 con 0x7fe9f0108970 2024-05-09T00:35:27.917 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:27.338+0000 7fe9ef7fe700 1 --2- 172.21.15.5:0/4273265475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9f0108970 0x7fe9f0108d80 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fe9e00098a0 tx=0x7fe9e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83808bc74ab18ef4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:35:27.917 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:27.338+0000 7fe9ee7fc700 1 -- 172.21.15.5:0/4273265475 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9e0004160 con 0x7fe9f0108970 2024-05-09T00:35:27.917 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:27.338+0000 7fe9ee7fc700 1 -- 172.21.15.5:0/4273265475 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9e0004300 con 0x7fe9f0108970 2024-05-09T00:35:27.917 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:27.338+0000 7fe9ee7fc700 1 -- 172.21.15.5:0/4273265475 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9e00036a0 con 0x7fe9f0108970 2024-05-09T00:35:27.917 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:27.339+0000 7fe9f5e6c700 1 -- 172.21.15.5:0/4273265475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9f0108970 msgr2=0x7fe9f0108d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:35:27.917 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:27.339+0000 7fe9f5e6c700 1 --2- 172.21.15.5:0/4273265475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9f0108970 0x7fe9f0108d80 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fe9e00098a0 tx=0x7fe9e0009bb0 comp rx=0 tx=0).stop 2024-05-09T00:35:27.917 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:27.339+0000 7fe9f5e6c700 1 -- 172.21.15.5:0/4273265475 shutdown_connections 2024-05-09T00:35:27.917 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:27.339+0000 7fe9f5e6c700 1 --2- 172.21.15.5:0/4273265475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9f0108970 0x7fe9f0108d80 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:35:27.918 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:27.339+0000 7fe9f5e6c700 1 -- 172.21.15.5:0/4273265475 >> 172.21.15.5:0/4273265475 conn(0x7fe9f0104050 msgr2=0x7fe9f01064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:35:27.918 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:27.339+0000 7fe9f5e6c700 1 -- 172.21.15.5:0/4273265475 shutdown_connections 2024-05-09T00:35:27.918 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:27.339+0000 7fe9f5e6c700 1 -- 172.21.15.5:0/4273265475 wait complete. 2024-05-09T00:35:27.918 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:27.340+0000 7fe9f5e6c700 1 Processor -- start 2024-05-09T00:35:27.918 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:27.340+0000 7fe9f5e6c700 1 -- start start 2024-05-09T00:35:27.918 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:27.340+0000 7fe9f5e6c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9f0108970 0x7fe9f0191400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:35:27.918 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:27.340+0000 7fe9f5e6c700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f010a070 con 0x7fe9f0108970 2024-05-09T00:35:27.918 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:27.341+0000 7fe9ef7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9f0108970 0x7fe9f0191400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:35:27.918 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:27.341+0000 7fe9ef7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9f0108970 0x7fe9f0191400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53242/0 (socket says 172.21.15.5:53242) 2024-05-09T00:35:27.918 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:27.341+0000 7fe9ef7fe700 1 -- 172.21.15.5:0/1708260420 learned_addr learned my addr 172.21.15.5:0/1708260420 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:35:27.918 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:27.341+0000 7fe9ef7fe700 1 -- 172.21.15.5:0/1708260420 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9f0191940 con 0x7fe9f0108970 2024-05-09T00:35:27.918 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:27.341+0000 7fe9ef7fe700 1 --2- 172.21.15.5:0/1708260420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9f0108970 0x7fe9f0191400 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0000c00 tx=0x7fe9e0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:35:27.919 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:27.341+0000 7fe9ecff9700 1 -- 172.21.15.5:0/1708260420 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9e0003db0 con 0x7fe9f0108970 2024-05-09T00:35:27.919 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:27.341+0000 7fe9ecff9700 1 -- 172.21.15.5:0/1708260420 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9e0011630 con 0x7fe9f0108970 2024-05-09T00:35:27.919 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:27.341+0000 7fe9f5e6c700 1 -- 172.21.15.5:0/1708260420 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9f018fb90 con 0x7fe9f0108970 2024-05-09T00:35:27.919 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:27.341+0000 7fe9ecff9700 1 -- 172.21.15.5:0/1708260420 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9e0011950 con 0x7fe9f0108970 2024-05-09T00:35:27.919 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:27.341+0000 7fe9f5e6c700 1 -- 172.21.15.5:0/1708260420 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9f0190070 con 0x7fe9f0108970 2024-05-09T00:35:27.919 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:27.341+0000 7fe9ecff9700 1 -- 172.21.15.5:0/1708260420 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe9e001a460 con 0x7fe9f0108970 2024-05-09T00:35:27.919 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:27.342+0000 7fe9ecff9700 1 -- 172.21.15.5:0/1708260420 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe9e001b780 con 0x7fe9f0108970 2024-05-09T00:35:27.919 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:27.342+0000 7fe9f5e6c700 1 -- 172.21.15.5:0/1708260420 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9f0064e20 con 0x7fe9f0108970 2024-05-09T00:35:27.919 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:27.345+0000 7fe9ecff9700 1 -- 172.21.15.5:0/1708260420 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fe9e0031020 con 0x7fe9f0108970 2024-05-09T00:35:27.919 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:27.416+0000 7fe9f5e6c700 1 -- 172.21.15.5:0/1708260420 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fe9f01967a0 con 0x7fe9f0108970 2024-05-09T00:35:27.919 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:27.420+0000 7fe9ecff9700 1 -- 172.21.15.5:0/1708260420 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+463 (secure 0 0 0) 0x7fe9e001bad0 con 0x7fe9f0108970 2024-05-09T00:35:27.919 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:27.421+0000 7fe9ecff9700 1 -- 172.21.15.5:0/1708260420 <== mon.0 v2:172.21.15.5:3300/0 8 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe9e001ad70 con 0x7fe9f0108970 2024-05-09T00:35:27.919 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:27.421+0000 7fe9f5e6c700 1 -- 172.21.15.5:0/1708260420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9f0108970 msgr2=0x7fe9f0191400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:35:27.920 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:27.421+0000 7fe9f5e6c700 1 --2- 172.21.15.5:0/1708260420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9f0108970 0x7fe9f0191400 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0000c00 tx=0x7fe9e0004750 comp rx=0 tx=0).stop 2024-05-09T00:35:27.920 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:27.421+0000 7fe9f5e6c700 1 -- 172.21.15.5:0/1708260420 shutdown_connections 2024-05-09T00:35:27.920 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:27.421+0000 7fe9f5e6c700 1 --2- 172.21.15.5:0/1708260420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9f0108970 0x7fe9f0191400 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:35:27.920 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:27.421+0000 7fe9f5e6c700 1 -- 172.21.15.5:0/1708260420 >> 172.21.15.5:0/1708260420 conn(0x7fe9f0104050 msgr2=0x7fe9f0193c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:35:27.920 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:27.421+0000 7fe9f5e6c700 1 -- 172.21.15.5:0/1708260420 shutdown_connections 2024-05-09T00:35:27.920 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:27.421+0000 7fe9f5e6c700 1 -- 172.21.15.5:0/1708260420 wait complete. 2024-05-09T00:35:27.920 INFO:teuthology.orchestra.run.smithi005.stdout:Generating new minimal ceph.conf... 2024-05-09T00:35:29.029 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:28 smithi005 ceph-mon[29995]: from='client.? 172.21.15.5:0/158277729' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf", "output-file": "/var/lib/ceph/mon/ceph-a/config"}]: dispatch 2024-05-09T00:35:29.037 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:28.329+0000 7fd508546700 1 Processor -- start 2024-05-09T00:35:29.037 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:28.330+0000 7fd508546700 1 -- start start 2024-05-09T00:35:29.037 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:28.330+0000 7fd508546700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd50007f660 0x7fd50007dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:35:29.037 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:28.330+0000 7fd508546700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd50007e020 con 0x7fd50007f660 2024-05-09T00:35:29.037 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:28.330+0000 7fd5062e2700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd50007f660 0x7fd50007dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:35:29.037 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:28.330+0000 7fd5062e2700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd50007f660 0x7fd50007dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53254/0 (socket says 172.21.15.5:53254) 2024-05-09T00:35:29.037 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:28.330+0000 7fd5062e2700 1 -- 172.21.15.5:0/1888853380 learned_addr learned my addr 172.21.15.5:0/1888853380 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:35:29.038 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:28.330+0000 7fd5062e2700 1 -- 172.21.15.5:0/1888853380 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd50007e1a0 con 0x7fd50007f660 2024-05-09T00:35:29.038 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:28.331+0000 7fd5062e2700 1 --2- 172.21.15.5:0/1888853380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd50007f660 0x7fd50007dae0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc0098a0 tx=0x7fd4fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78d67cf02ff0f337 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:35:29.038 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:28.331+0000 7fd5052e0700 1 -- 172.21.15.5:0/1888853380 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4fc004160 con 0x7fd50007f660 2024-05-09T00:35:29.038 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:28.331+0000 7fd5052e0700 1 -- 172.21.15.5:0/1888853380 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd4fc004300 con 0x7fd50007f660 2024-05-09T00:35:29.038 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:28.331+0000 7fd5052e0700 1 -- 172.21.15.5:0/1888853380 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4fc011af0 con 0x7fd50007f660 2024-05-09T00:35:29.038 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:28.331+0000 7fd508546700 1 -- 172.21.15.5:0/1888853380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd50007f660 msgr2=0x7fd50007dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:35:29.038 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:28.331+0000 7fd508546700 1 --2- 172.21.15.5:0/1888853380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd50007f660 0x7fd50007dae0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc0098a0 tx=0x7fd4fc009bb0 comp rx=0 tx=0).stop 2024-05-09T00:35:29.038 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:28.332+0000 7fd508546700 1 -- 172.21.15.5:0/1888853380 shutdown_connections 2024-05-09T00:35:29.038 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:28.332+0000 7fd508546700 1 --2- 172.21.15.5:0/1888853380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd50007f660 0x7fd50007dae0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:35:29.038 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:28.332+0000 7fd508546700 1 -- 172.21.15.5:0/1888853380 >> 172.21.15.5:0/1888853380 conn(0x7fd500105100 msgr2=0x7fd500107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:35:29.038 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:28.332+0000 7fd508546700 1 -- 172.21.15.5:0/1888853380 shutdown_connections 2024-05-09T00:35:29.038 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:28.332+0000 7fd508546700 1 -- 172.21.15.5:0/1888853380 wait complete. 2024-05-09T00:35:29.039 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:28.332+0000 7fd508546700 1 Processor -- start 2024-05-09T00:35:29.039 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:28.332+0000 7fd508546700 1 -- start start 2024-05-09T00:35:29.039 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:28.333+0000 7fd508546700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd50007f660 0x7fd5001a0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:35:29.039 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:28.333+0000 7fd508546700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd500110060 con 0x7fd50007f660 2024-05-09T00:35:29.039 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:28.333+0000 7fd5062e2700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd50007f660 0x7fd5001a0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:35:29.039 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:28.333+0000 7fd5062e2700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd50007f660 0x7fd5001a0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53266/0 (socket says 172.21.15.5:53266) 2024-05-09T00:35:29.039 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:28.333+0000 7fd5062e2700 1 -- 172.21.15.5:0/158277729 learned_addr learned my addr 172.21.15.5:0/158277729 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:35:29.039 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:28.333+0000 7fd5062e2700 1 -- 172.21.15.5:0/158277729 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5001a0650 con 0x7fd50007f660 2024-05-09T00:35:29.039 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:28.333+0000 7fd5062e2700 1 --2- 172.21.15.5:0/158277729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd50007f660 0x7fd5001a0110 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc009fb0 tx=0x7fd4fc011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:35:29.039 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:28.333+0000 7fd4f77fe700 1 -- 172.21.15.5:0/158277729 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4fc011bc0 con 0x7fd50007f660 2024-05-09T00:35:29.039 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:28.333+0000 7fd4f77fe700 1 -- 172.21.15.5:0/158277729 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd4fc024440 con 0x7fd50007f660 2024-05-09T00:35:29.039 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:28.333+0000 7fd508546700 1 -- 172.21.15.5:0/158277729 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5001a08e0 con 0x7fd50007f660 2024-05-09T00:35:29.039 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:28.333+0000 7fd4f77fe700 1 -- 172.21.15.5:0/158277729 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4fc01a5c0 con 0x7fd50007f660 2024-05-09T00:35:29.040 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:28.333+0000 7fd508546700 1 -- 172.21.15.5:0/158277729 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5001a34f0 con 0x7fd50007f660 2024-05-09T00:35:29.040 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:28.334+0000 7fd4f77fe700 1 -- 172.21.15.5:0/158277729 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd4fc01abc0 con 0x7fd50007f660 2024-05-09T00:35:29.040 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:28.334+0000 7fd4f77fe700 1 -- 172.21.15.5:0/158277729 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd4fc019700 con 0x7fd50007f660 2024-05-09T00:35:29.040 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:28.334+0000 7fd508546700 1 -- 172.21.15.5:0/158277729 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd500064e20 con 0x7fd50007f660 2024-05-09T00:35:29.040 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:28.337+0000 7fd4f77fe700 1 -- 172.21.15.5:0/158277729 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fd4fc02dd90 con 0x7fd50007f660 2024-05-09T00:35:29.040 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:28.408+0000 7fd508546700 1 -- 172.21.15.5:0/158277729 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config generate-minimal-conf", "output-file": "/var/lib/ceph/mon/ceph-a/config"} v 0) -- 0x7fd5001a37e0 con 0x7fd50007f660 2024-05-09T00:35:29.040 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:28.409+0000 7fd4f77fe700 1 -- 172.21.15.5:0/158277729 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf", "output-file": "/var/lib/ceph/mon/ceph-a/config"}]=0 v2) ==== 126+0+173 (secure 0 0 0) 0x7fd4fc0245f0 con 0x7fd50007f660 2024-05-09T00:35:29.040 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:28.409+0000 7fd508546700 1 -- 172.21.15.5:0/158277729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd50007f660 msgr2=0x7fd5001a0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:35:29.040 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:28.409+0000 7fd508546700 1 --2- 172.21.15.5:0/158277729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd50007f660 0x7fd5001a0110 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc009fb0 tx=0x7fd4fc011890 comp rx=0 tx=0).stop 2024-05-09T00:35:29.040 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:28.410+0000 7fd508546700 1 -- 172.21.15.5:0/158277729 shutdown_connections 2024-05-09T00:35:29.040 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:28.410+0000 7fd508546700 1 --2- 172.21.15.5:0/158277729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd50007f660 0x7fd5001a0110 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:35:29.040 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:28.410+0000 7fd508546700 1 -- 172.21.15.5:0/158277729 >> 172.21.15.5:0/158277729 conn(0x7fd500105100 msgr2=0x7fd500105b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:35:29.040 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:28.410+0000 7fd508546700 1 -- 172.21.15.5:0/158277729 shutdown_connections 2024-05-09T00:35:29.040 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:28.410+0000 7fd508546700 1 -- 172.21.15.5:0/158277729 wait complete. 2024-05-09T00:35:29.041 INFO:teuthology.orchestra.run.smithi005.stdout:Restarting the monitor... 2024-05-09T00:35:29.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:29 smithi005 systemd[1]: Stopping Ceph mon.a for d85814c6-0d9b-11ef-bc97-c7b262605968... 2024-05-09T00:35:29.650 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:29 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mon-a[29990]: 2024-05-09T00:35:29.297+0000 7f2f0675c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T00:35:29.650 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:29 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mon-a[29990]: 2024-05-09T00:35:29.297+0000 7f2f0675c700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-05-09T00:35:29.650 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:29 smithi005 podman[30239]: 2024-05-09 00:35:29.401532438 +0000 UTC m=+0.196196416 container died c42607cbc92705ce9647a5f04c1e1ead19a47f872c6e3a90d1c31f64dca2ebf1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mon-a, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507) 2024-05-09T00:35:30.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:29 smithi005 podman[30239]: 2024-05-09 00:35:29.748659484 +0000 UTC m=+0.543323463 container cleanup c42607cbc92705ce9647a5f04c1e1ead19a47f872c6e3a90d1c31f64dca2ebf1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mon-a, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T00:35:30.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:29 smithi005 bash[30239]: ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mon-a 2024-05-09T00:35:30.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:29 smithi005 podman[30254]: 2024-05-09 00:35:29.891365892 +0000 UTC m=+0.484730924 container remove c42607cbc92705ce9647a5f04c1e1ead19a47f872c6e3a90d1c31f64dca2ebf1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mon-a, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T00:35:30.442 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 systemd[1]: ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@mon.a.service: Deactivated successfully. 2024-05-09T00:35:30.442 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 systemd[1]: Stopped Ceph mon.a for d85814c6-0d9b-11ef-bc97-c7b262605968. 2024-05-09T00:35:30.442 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 systemd[1]: Starting Ceph mon.a for d85814c6-0d9b-11ef-bc97-c7b262605968... 2024-05-09T00:35:30.442 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 podman[30347]: 2024-05-09 00:35:30.236204122 +0000 UTC m=+0.018402174 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T00:35:30.443 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 podman[30347]: 2024-05-09 00:35:30.333967652 +0000 UTC m=+0.116165733 container create f5caa1eb3625918c35c8f3c9e52216eda888cedb2bb0bb7210833dc5e83eae49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T00:35:30.642 INFO:teuthology.orchestra.run.smithi005.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-09T00:35:30.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 podman[30347]: 2024-05-09 00:35:30.536303425 +0000 UTC m=+0.318501474 container init f5caa1eb3625918c35c8f3c9e52216eda888cedb2bb0bb7210833dc5e83eae49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-05-09T00:35:30.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 podman[30347]: 2024-05-09 00:35:30.539585525 +0000 UTC m=+0.321783564 container start f5caa1eb3625918c35c8f3c9e52216eda888cedb2bb0bb7210833dc5e83eae49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mon-a, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-05-09T00:35:30.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: set uid:gid to 167:167 (ceph:ceph) 2024-05-09T00:35:30.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: ceph version 19.0.0-3594-gc9da4049 (c9da4049f638c0f84bdd9b4e33c42e86046ac554) squid (dev), process ceph-mon, pid 2 2024-05-09T00:35:30.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: pidfile_write: ignore empty --pid-file 2024-05-09T00:35:30.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: load: jerasure load: lrc 2024-05-09T00:35:30.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: RocksDB version: 7.9.2 2024-05-09T00:35:30.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Git sha 0 2024-05-09T00:35:30.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Compile date 2024-05-08 19:33:14 2024-05-09T00:35:30.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: DB SUMMARY 2024-05-09T00:35:30.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: DB Session ID: 8F6OAEB38140II0O2KTS 2024-05-09T00:35:30.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: CURRENT file: CURRENT 2024-05-09T00:35:30.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: IDENTITY file: IDENTITY 2024-05-09T00:35:30.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-05-09T00:35:30.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-05-09T00:35:30.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104234 ; 2024-05-09T00:35:30.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.error_if_exists: 0 2024-05-09T00:35:30.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.create_if_missing: 0 2024-05-09T00:35:30.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.paranoid_checks: 1 2024-05-09T00:35:30.794 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-09T00:35:30.794 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-09T00:35:30.794 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-09T00:35:30.794 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.env: 0x55f2cb1f9500 2024-05-09T00:35:30.794 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.fs: PosixFileSystem 2024-05-09T00:35:30.794 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.info_log: 0x55f2cd58a020 2024-05-09T00:35:30.794 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.max_file_opening_threads: 16 2024-05-09T00:35:30.794 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.statistics: (nil) 2024-05-09T00:35:30.794 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.use_fsync: 0 2024-05-09T00:35:30.794 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.max_log_file_size: 0 2024-05-09T00:35:30.794 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-09T00:35:30.794 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-09T00:35:30.795 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.keep_log_file_num: 1000 2024-05-09T00:35:30.795 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.recycle_log_file_num: 0 2024-05-09T00:35:30.795 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.allow_fallocate: 1 2024-05-09T00:35:30.795 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.allow_mmap_reads: 0 2024-05-09T00:35:30.795 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.allow_mmap_writes: 0 2024-05-09T00:35:30.795 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.use_direct_reads: 0 2024-05-09T00:35:30.795 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-09T00:35:30.795 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.create_missing_column_families: 0 2024-05-09T00:35:30.795 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.db_log_dir: 2024-05-09T00:35:30.795 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.wal_dir: 2024-05-09T00:35:30.796 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-09T00:35:30.796 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-09T00:35:30.796 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-09T00:35:30.796 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-09T00:35:30.796 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-09T00:35:30.796 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-09T00:35:30.796 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.advise_random_on_open: 1 2024-05-09T00:35:30.796 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.db_write_buffer_size: 0 2024-05-09T00:35:30.797 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.write_buffer_manager: 0x55f2cd4548c0 2024-05-09T00:35:30.797 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-09T00:35:30.797 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-09T00:35:30.797 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-09T00:35:30.797 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.rate_limiter: (nil) 2024-05-09T00:35:30.797 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-09T00:35:30.797 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.wal_recovery_mode: 2 2024-05-09T00:35:30.797 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.enable_thread_tracking: 0 2024-05-09T00:35:30.797 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.enable_pipelined_write: 0 2024-05-09T00:35:30.797 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.unordered_write: 0 2024-05-09T00:35:30.797 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-09T00:35:30.797 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-09T00:35:30.797 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-09T00:35:30.798 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-09T00:35:30.798 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.row_cache: None 2024-05-09T00:35:30.798 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.wal_filter: None 2024-05-09T00:35:30.799 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-09T00:35:30.799 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.allow_ingest_behind: 0 2024-05-09T00:35:30.799 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.two_write_queues: 0 2024-05-09T00:35:30.799 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.manual_wal_flush: 0 2024-05-09T00:35:30.799 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.wal_compression: 0 2024-05-09T00:35:30.800 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.atomic_flush: 0 2024-05-09T00:35:30.800 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-09T00:35:30.800 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-09T00:35:30.800 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-09T00:35:30.800 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.log_readahead_size: 0 2024-05-09T00:35:30.800 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-09T00:35:30.800 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.best_efforts_recovery: 0 2024-05-09T00:35:30.800 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-09T00:35:30.800 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-09T00:35:30.800 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.allow_data_in_errors: 0 2024-05-09T00:35:30.800 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.db_host_id: __hostname__ 2024-05-09T00:35:30.800 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-09T00:35:30.801 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.max_background_jobs: 2 2024-05-09T00:35:30.801 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.max_background_compactions: -1 2024-05-09T00:35:30.801 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.max_subcompactions: 1 2024-05-09T00:35:30.801 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-09T00:35:30.801 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-09T00:35:30.801 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-09T00:35:30.801 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.max_total_wal_size: 0 2024-05-09T00:35:30.801 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-09T00:35:30.801 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-09T00:35:30.801 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-09T00:35:30.801 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-09T00:35:30.802 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.max_open_files: -1 2024-05-09T00:35:30.802 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.bytes_per_sync: 0 2024-05-09T00:35:30.802 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-09T00:35:30.802 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-09T00:35:30.802 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.compaction_readahead_size: 0 2024-05-09T00:35:30.802 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.max_background_flushes: -1 2024-05-09T00:35:30.802 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Compression algorithms supported: 2024-05-09T00:35:30.802 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-09T00:35:30.803 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: kZSTD supported: 0 2024-05-09T00:35:30.803 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: kXpressCompression supported: 0 2024-05-09T00:35:30.803 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: kLZ4HCCompression supported: 1 2024-05-09T00:35:30.803 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: kZlibCompression supported: 1 2024-05-09T00:35:30.803 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: kSnappyCompression supported: 1 2024-05-09T00:35:30.803 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: kLZ4Compression supported: 1 2024-05-09T00:35:30.803 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: kBZip2Compression supported: 0 2024-05-09T00:35:30.803 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-09T00:35:30.803 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-09T00:35:30.803 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-05-09T00:35:30.803 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-09T00:35:30.803 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-09T00:35:30.804 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.merge_operator: 2024-05-09T00:35:30.804 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.compaction_filter: None 2024-05-09T00:35:30.804 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.compaction_filter_factory: None 2024-05-09T00:35:30.804 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.sst_partitioner_factory: None 2024-05-09T00:35:30.804 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-09T00:35:30.804 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-09T00:35:30.804 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55f2cd58a1e0) 2024-05-09T00:35:30.804 INFO:journalctl@ceph.mon.a.smithi005.stdout: cache_index_and_filter_blocks: 1 2024-05-09T00:35:30.804 INFO:journalctl@ceph.mon.a.smithi005.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-09T00:35:30.804 INFO:journalctl@ceph.mon.a.smithi005.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-09T00:35:30.804 INFO:journalctl@ceph.mon.a.smithi005.stdout: pin_top_level_index_and_filter: 1 2024-05-09T00:35:30.805 INFO:journalctl@ceph.mon.a.smithi005.stdout: index_type: 0 2024-05-09T00:35:30.805 INFO:journalctl@ceph.mon.a.smithi005.stdout: data_block_index_type: 0 2024-05-09T00:35:30.805 INFO:journalctl@ceph.mon.a.smithi005.stdout: index_shortening: 1 2024-05-09T00:35:30.805 INFO:journalctl@ceph.mon.a.smithi005.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-09T00:35:30.805 INFO:journalctl@ceph.mon.a.smithi005.stdout: checksum: 4 2024-05-09T00:35:30.805 INFO:journalctl@ceph.mon.a.smithi005.stdout: no_block_cache: 0 2024-05-09T00:35:30.805 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_cache: 0x55f2cd4871f0 2024-05-09T00:35:30.805 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_cache_name: BinnedLRUCache 2024-05-09T00:35:30.805 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_cache_options: 2024-05-09T00:35:30.805 INFO:journalctl@ceph.mon.a.smithi005.stdout: capacity : 536870912 2024-05-09T00:35:30.805 INFO:journalctl@ceph.mon.a.smithi005.stdout: num_shard_bits : 4 2024-05-09T00:35:30.805 INFO:journalctl@ceph.mon.a.smithi005.stdout: strict_capacity_limit : 0 2024-05-09T00:35:30.806 INFO:journalctl@ceph.mon.a.smithi005.stdout: high_pri_pool_ratio: 0.000 2024-05-09T00:35:30.806 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_cache_compressed: (nil) 2024-05-09T00:35:30.806 INFO:journalctl@ceph.mon.a.smithi005.stdout: persistent_cache: (nil) 2024-05-09T00:35:30.806 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_size: 4096 2024-05-09T00:35:30.806 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_size_deviation: 10 2024-05-09T00:35:30.806 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_restart_interval: 16 2024-05-09T00:35:30.806 INFO:journalctl@ceph.mon.a.smithi005.stdout: index_block_restart_interval: 1 2024-05-09T00:35:30.806 INFO:journalctl@ceph.mon.a.smithi005.stdout: metadata_block_size: 4096 2024-05-09T00:35:30.806 INFO:journalctl@ceph.mon.a.smithi005.stdout: partition_filters: 0 2024-05-09T00:35:30.806 INFO:journalctl@ceph.mon.a.smithi005.stdout: use_delta_encoding: 1 2024-05-09T00:35:30.806 INFO:journalctl@ceph.mon.a.smithi005.stdout: filter_policy: bloomfilter 2024-05-09T00:35:30.806 INFO:journalctl@ceph.mon.a.smithi005.stdout: whole_key_filtering: 1 2024-05-09T00:35:30.807 INFO:journalctl@ceph.mon.a.smithi005.stdout: verify_compression: 0 2024-05-09T00:35:30.807 INFO:journalctl@ceph.mon.a.smithi005.stdout: read_amp_bytes_per_bit: 0 2024-05-09T00:35:30.807 INFO:journalctl@ceph.mon.a.smithi005.stdout: format_version: 5 2024-05-09T00:35:30.807 INFO:journalctl@ceph.mon.a.smithi005.stdout: enable_index_compression: 1 2024-05-09T00:35:30.807 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_align: 0 2024-05-09T00:35:30.807 INFO:journalctl@ceph.mon.a.smithi005.stdout: max_auto_readahead_size: 262144 2024-05-09T00:35:30.807 INFO:journalctl@ceph.mon.a.smithi005.stdout: prepopulate_block_cache: 0 2024-05-09T00:35:30.807 INFO:journalctl@ceph.mon.a.smithi005.stdout: initial_auto_readahead_size: 8192 2024-05-09T00:35:30.807 INFO:journalctl@ceph.mon.a.smithi005.stdout: num_file_reads_for_auto_readahead: 2 2024-05-09T00:35:30.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.write_buffer_size: 33554432 2024-05-09T00:35:30.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.max_write_buffer_number: 2 2024-05-09T00:35:30.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.compression: NoCompression 2024-05-09T00:35:30.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.bottommost_compression: Disabled 2024-05-09T00:35:30.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.prefix_extractor: nullptr 2024-05-09T00:35:30.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-09T00:35:30.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.num_levels: 7 2024-05-09T00:35:30.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-09T00:35:30.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-09T00:35:30.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-09T00:35:30.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-09T00:35:30.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-09T00:35:30.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-09T00:35:30.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-09T00:35:30.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-09T00:35:30.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-09T00:35:30.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-09T00:35:30.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-09T00:35:30.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-09T00:35:30.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-09T00:35:30.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.compression_opts.level: 32767 2024-05-09T00:35:30.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.compression_opts.strategy: 0 2024-05-09T00:35:30.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-09T00:35:30.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-09T00:35:30.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-09T00:35:30.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-09T00:35:30.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.compression_opts.enabled: false 2024-05-09T00:35:30.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-09T00:35:30.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-09T00:35:30.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-09T00:35:30.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-09T00:35:30.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.target_file_size_base: 67108864 2024-05-09T00:35:30.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-09T00:35:30.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-09T00:35:30.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-09T00:35:30.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-09T00:35:30.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-09T00:35:30.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-09T00:35:30.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-09T00:35:30.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-09T00:35:30.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-09T00:35:30.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-09T00:35:30.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-09T00:35:30.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-09T00:35:30.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-09T00:35:30.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-09T00:35:30.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.arena_block_size: 1048576 2024-05-09T00:35:30.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-09T00:35:30.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-09T00:35:30.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.disable_auto_compactions: 0 2024-05-09T00:35:30.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-09T00:35:30.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-09T00:35:30.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-09T00:35:30.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-09T00:35:30.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-09T00:35:30.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-09T00:35:30.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-09T00:35:30.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-09T00:35:30.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-09T00:35:30.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-09T00:35:30.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-09T00:35:30.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.inplace_update_support: 0 2024-05-09T00:35:30.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-09T00:35:30.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-09T00:35:30.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-09T00:35:30.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-09T00:35:30.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.bloom_locality: 0 2024-05-09T00:35:30.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.max_successive_merges: 0 2024-05-09T00:35:30.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-09T00:35:30.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.paranoid_file_checks: 0 2024-05-09T00:35:30.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.force_consistency_checks: 1 2024-05-09T00:35:30.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.report_bg_io_stats: 0 2024-05-09T00:35:30.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.ttl: 2592000 2024-05-09T00:35:30.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-09T00:35:30.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-09T00:35:30.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-09T00:35:30.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.enable_blob_files: false 2024-05-09T00:35:30.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.min_blob_size: 0 2024-05-09T00:35:30.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.blob_file_size: 268435456 2024-05-09T00:35:30.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-09T00:35:30.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-09T00:35:30.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-09T00:35:30.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-09T00:35:30.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-09T00:35:30.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.blob_file_starting_level: 0 2024-05-09T00:35:30.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-09T00:35:30.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-05-09T00:35:30.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-05-09T00:35:30.815 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: b11e6924-dcb2-4c0a-b60b-587845da70a7 2024-05-09T00:35:30.815 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715214930580309, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-05-09T00:35:30.815 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-05-09T00:35:30.815 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715214930581565, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99462, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 322, "table_properties": {"data_size": 97408, "index_size": 310, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13917, "raw_average_key_size": 49, "raw_value_size": 89802, "raw_average_value_size": 319, "num_data_blocks": 12, "num_entries": 281, "num_filter_entries": 281, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715214930, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "b11e6924-dcb2-4c0a-b60b-587845da70a7", "db_session_id": "8F6OAEB38140II0O2KTS", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-05-09T00:35:30.815 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715214930581653, "job": 1, "event": "recovery_finished"} 2024-05-09T00:35:30.815 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-05-09T00:35:30.815 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-09T00:35:30.815 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55f2cd46bc00 2024-05-09T00:35:30.815 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: DB pointer 0x55f2d225e000 2024-05-09T00:35:30.815 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-09T00:35:30.815 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-09T00:35:30.815 INFO:journalctl@ceph.mon.a.smithi005.stdout: ** DB Stats ** 2024-05-09T00:35:30.816 INFO:journalctl@ceph.mon.a.smithi005.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-09T00:35:30.816 INFO:journalctl@ceph.mon.a.smithi005.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-09T00:35:30.816 INFO:journalctl@ceph.mon.a.smithi005.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-09T00:35:30.816 INFO:journalctl@ceph.mon.a.smithi005.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-09T00:35:30.816 INFO:journalctl@ceph.mon.a.smithi005.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-09T00:35:30.816 INFO:journalctl@ceph.mon.a.smithi005.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-09T00:35:30.816 INFO:journalctl@ceph.mon.a.smithi005.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-09T00:35:30.816 INFO:journalctl@ceph.mon.a.smithi005.stdout: 2024-05-09T00:35:30.816 INFO:journalctl@ceph.mon.a.smithi005.stdout: ** Compaction Stats [default] ** 2024-05-09T00:35:30.816 INFO:journalctl@ceph.mon.a.smithi005.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-09T00:35:30.816 INFO:journalctl@ceph.mon.a.smithi005.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-09T00:35:30.816 INFO:journalctl@ceph.mon.a.smithi005.stdout: L0 2/0 98.99 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 143.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-09T00:35:30.817 INFO:journalctl@ceph.mon.a.smithi005.stdout: Sum 2/0 98.99 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 143.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-09T00:35:30.817 INFO:journalctl@ceph.mon.a.smithi005.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 143.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-09T00:35:30.817 INFO:journalctl@ceph.mon.a.smithi005.stdout: 2024-05-09T00:35:30.817 INFO:journalctl@ceph.mon.a.smithi005.stdout: ** Compaction Stats [default] ** 2024-05-09T00:35:30.817 INFO:journalctl@ceph.mon.a.smithi005.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-09T00:35:30.817 INFO:journalctl@ceph.mon.a.smithi005.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-09T00:35:30.817 INFO:journalctl@ceph.mon.a.smithi005.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 143.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-09T00:35:30.817 INFO:journalctl@ceph.mon.a.smithi005.stdout: 2024-05-09T00:35:30.817 INFO:journalctl@ceph.mon.a.smithi005.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-09T00:35:30.817 INFO:journalctl@ceph.mon.a.smithi005.stdout: 2024-05-09T00:35:30.817 INFO:journalctl@ceph.mon.a.smithi005.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-09T00:35:30.817 INFO:journalctl@ceph.mon.a.smithi005.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-09T00:35:30.818 INFO:journalctl@ceph.mon.a.smithi005.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-09T00:35:30.818 INFO:journalctl@ceph.mon.a.smithi005.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-09T00:35:30.818 INFO:journalctl@ceph.mon.a.smithi005.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-09T00:35:30.818 INFO:journalctl@ceph.mon.a.smithi005.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-09T00:35:30.818 INFO:journalctl@ceph.mon.a.smithi005.stdout: Cumulative compaction: 0.00 GB write, 27.35 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-09T00:35:30.818 INFO:journalctl@ceph.mon.a.smithi005.stdout: Interval compaction: 0.00 GB write, 27.35 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-09T00:35:30.818 INFO:journalctl@ceph.mon.a.smithi005.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-09T00:35:30.818 INFO:journalctl@ceph.mon.a.smithi005.stdout: Block cache BinnedLRUCache@0x55f2cd4871f0#2 capacity: 512.00 MB usage: 1.41 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-05-09T00:35:30.818 INFO:journalctl@ceph.mon.a.smithi005.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.91 KB,0.000172853%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-05-09T00:35:30.818 INFO:journalctl@ceph.mon.a.smithi005.stdout: 2024-05-09T00:35:30.818 INFO:journalctl@ceph.mon.a.smithi005.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-09T00:35:30.819 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: starting mon.a rank 0 at public addrs [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] at bind addrs [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid d85814c6-0d9b-11ef-bc97-c7b262605968 2024-05-09T00:35:30.819 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: mon.a@-1(???) e1 preinit fsid d85814c6-0d9b-11ef-bc97-c7b262605968 2024-05-09T00:35:30.819 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: mon.a@-1(???).mds e1 new map 2024-05-09T00:35:30.819 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: mon.a@-1(???).mds e1 print_map 2024-05-09T00:35:30.819 INFO:journalctl@ceph.mon.a.smithi005.stdout: e1 2024-05-09T00:35:30.819 INFO:journalctl@ceph.mon.a.smithi005.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-09T00:35:30.819 INFO:journalctl@ceph.mon.a.smithi005.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-09T00:35:30.819 INFO:journalctl@ceph.mon.a.smithi005.stdout: legacy client fscid: -1 2024-05-09T00:35:30.819 INFO:journalctl@ceph.mon.a.smithi005.stdout: 2024-05-09T00:35:30.819 INFO:journalctl@ceph.mon.a.smithi005.stdout: No filesystems configured 2024-05-09T00:35:30.819 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-09T00:35:30.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T00:35:30.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T00:35:30.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T00:35:30.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-09T00:35:30.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 bash[30347]: f5caa1eb3625918c35c8f3c9e52216eda888cedb2bb0bb7210833dc5e83eae49 2024-05-09T00:35:30.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 systemd[1]: Started Ceph mon.a for d85814c6-0d9b-11ef-bc97-c7b262605968. 2024-05-09T00:35:30.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-09T00:35:30.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: monmap epoch 1 2024-05-09T00:35:30.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: fsid d85814c6-0d9b-11ef-bc97-c7b262605968 2024-05-09T00:35:30.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: last_changed 2024-05-09T00:35:22.099371+0000 2024-05-09T00:35:30.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: created 2024-05-09T00:35:22.099371+0000 2024-05-09T00:35:30.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: min_mon_release 19 (squid) 2024-05-09T00:35:30.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: election_strategy: 1 2024-05-09T00:35:30.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-05-09T00:35:30.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: fsmap 2024-05-09T00:35:30.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: osdmap e1: 0 total, 0 up, 0 in 2024-05-09T00:35:30.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:30 smithi005 ceph-mon[30365]: mgrmap e1: no daemons active 2024-05-09T00:35:31.761 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:31.099+0000 7f778d593700 1 Processor -- start 2024-05-09T00:35:31.761 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:31.099+0000 7f778d593700 1 -- start start 2024-05-09T00:35:31.761 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:31.099+0000 7f778d593700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7788107d40 0x7f7788108150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:35:31.761 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:31.099+0000 7f778d593700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7788108720 con 0x7f7788107d40 2024-05-09T00:35:31.761 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:31.099+0000 7f7786ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7788107d40 0x7f7788108150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:35:31.761 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:31.099+0000 7f7786ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7788107d40 0x7f7788108150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53274/0 (socket says 172.21.15.5:53274) 2024-05-09T00:35:31.761 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:31.099+0000 7f7786ffd700 1 -- 172.21.15.5:0/2880843036 learned_addr learned my addr 172.21.15.5:0/2880843036 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:35:31.761 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:31.099+0000 7f7786ffd700 1 -- 172.21.15.5:0/2880843036 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f778810ad40 con 0x7f7788107d40 2024-05-09T00:35:31.761 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:31.099+0000 7f7786ffd700 1 --2- 172.21.15.5:0/2880843036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7788107d40 0x7f7788108150 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f77780098a0 tx=0x7f7778009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9393c212c508356a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:35:31.761 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:31.099+0000 7f7785ffb700 1 -- 172.21.15.5:0/2880843036 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7778004160 con 0x7f7788107d40 2024-05-09T00:35:31.762 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:31.099+0000 7f7785ffb700 1 -- 172.21.15.5:0/2880843036 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7778004300 con 0x7f7788107d40 2024-05-09T00:35:31.762 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:31.099+0000 7f7785ffb700 1 -- 172.21.15.5:0/2880843036 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7778011af0 con 0x7f7788107d40 2024-05-09T00:35:31.762 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:31.101+0000 7f778d593700 1 -- 172.21.15.5:0/2880843036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7788107d40 msgr2=0x7f7788108150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:35:31.762 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:31.101+0000 7f778d593700 1 --2- 172.21.15.5:0/2880843036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7788107d40 0x7f7788108150 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f77780098a0 tx=0x7f7778009bb0 comp rx=0 tx=0).stop 2024-05-09T00:35:31.762 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:31.101+0000 7f778d593700 1 -- 172.21.15.5:0/2880843036 shutdown_connections 2024-05-09T00:35:31.762 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:31.101+0000 7f778d593700 1 --2- 172.21.15.5:0/2880843036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7788107d40 0x7f7788108150 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:35:31.762 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:31.101+0000 7f778d593700 1 -- 172.21.15.5:0/2880843036 >> 172.21.15.5:0/2880843036 conn(0x7f7788103770 msgr2=0x7f7788105bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:35:31.762 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:31.101+0000 7f778d593700 1 -- 172.21.15.5:0/2880843036 shutdown_connections 2024-05-09T00:35:31.762 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:31.101+0000 7f778d593700 1 -- 172.21.15.5:0/2880843036 wait complete. 2024-05-09T00:35:31.763 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:31.101+0000 7f778d593700 1 Processor -- start 2024-05-09T00:35:31.763 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:31.101+0000 7f778d593700 1 -- start start 2024-05-09T00:35:31.763 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:31.101+0000 7f778d593700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7788107d40 0x7f778819e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:35:31.763 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:31.101+0000 7f778d593700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f778810df70 con 0x7f7788107d40 2024-05-09T00:35:31.763 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:31.101+0000 7f7786ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7788107d40 0x7f778819e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:35:31.763 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:31.101+0000 7f7786ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7788107d40 0x7f778819e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53280/0 (socket says 172.21.15.5:53280) 2024-05-09T00:35:31.763 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:31.101+0000 7f7786ffd700 1 -- 172.21.15.5:0/4077828924 learned_addr learned my addr 172.21.15.5:0/4077828924 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:35:31.763 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:31.101+0000 7f7786ffd700 1 -- 172.21.15.5:0/4077828924 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f778819e680 con 0x7f7788107d40 2024-05-09T00:35:31.763 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:31.102+0000 7f7786ffd700 1 --2- 172.21.15.5:0/4077828924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7788107d40 0x7f778819e140 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7778011670 tx=0x7f7778004280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:35:31.763 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:31.102+0000 7f776ffff700 1 -- 172.21.15.5:0/4077828924 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77780036a0 con 0x7f7788107d40 2024-05-09T00:35:31.763 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:31.102+0000 7f778d593700 1 -- 172.21.15.5:0/4077828924 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f778819e910 con 0x7f7788107d40 2024-05-09T00:35:31.763 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:31.102+0000 7f776ffff700 1 -- 172.21.15.5:0/4077828924 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f777801a430 con 0x7f7788107d40 2024-05-09T00:35:31.763 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:31.102+0000 7f776ffff700 1 -- 172.21.15.5:0/4077828924 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7778019550 con 0x7f7788107d40 2024-05-09T00:35:31.764 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:31.102+0000 7f778d593700 1 -- 172.21.15.5:0/4077828924 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77881a1520 con 0x7f7788107d40 2024-05-09T00:35:31.764 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:31.102+0000 7f776ffff700 1 -- 172.21.15.5:0/4077828924 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7778019ba0 con 0x7f7788107d40 2024-05-09T00:35:31.764 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:31.102+0000 7f776ffff700 1 -- 172.21.15.5:0/4077828924 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7778024730 con 0x7f7788107d40 2024-05-09T00:35:31.764 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:31.103+0000 7f778d593700 1 -- 172.21.15.5:0/4077828924 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f778810e0f0 con 0x7f7788107d40 2024-05-09T00:35:31.764 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:31.106+0000 7f776ffff700 1 -- 172.21.15.5:0/4077828924 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f7778043ec0 con 0x7f7788107d40 2024-05-09T00:35:31.764 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:31.176+0000 7f778d593700 1 -- 172.21.15.5:0/4077828924 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f77881a16f0 con 0x7f7788107d40 2024-05-09T00:35:31.764 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:31.179+0000 7f776ffff700 1 -- 172.21.15.5:0/4077828924 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f7778032020 con 0x7f7788107d40 2024-05-09T00:35:31.764 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:31.180+0000 7f778d593700 1 -- 172.21.15.5:0/4077828924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7788107d40 msgr2=0x7f778819e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:35:31.764 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:31.180+0000 7f778d593700 1 --2- 172.21.15.5:0/4077828924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7788107d40 0x7f778819e140 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7778011670 tx=0x7f7778004280 comp rx=0 tx=0).stop 2024-05-09T00:35:31.764 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:31.180+0000 7f778d593700 1 -- 172.21.15.5:0/4077828924 shutdown_connections 2024-05-09T00:35:31.764 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:31.180+0000 7f778d593700 1 --2- 172.21.15.5:0/4077828924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7788107d40 0x7f778819e140 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:35:31.764 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:31.180+0000 7f778d593700 1 -- 172.21.15.5:0/4077828924 >> 172.21.15.5:0/4077828924 conn(0x7f7788103770 msgr2=0x7f7788104260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:35:31.764 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:31.180+0000 7f778d593700 1 -- 172.21.15.5:0/4077828924 shutdown_connections 2024-05-09T00:35:31.764 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:31.180+0000 7f778d593700 1 -- 172.21.15.5:0/4077828924 wait complete. 2024-05-09T00:35:31.765 INFO:teuthology.orchestra.run.smithi005.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-09T00:35:31.785 INFO:teuthology.orchestra.run.smithi005.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-09T00:35:31.785 INFO:teuthology.orchestra.run.smithi005.stdout:Creating mgr... 2024-05-09T00:35:31.785 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying port 0.0.0.0:9283 ... 2024-05-09T00:35:31.786 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying port 0.0.0.0:8765 ... 2024-05-09T00:35:32.042 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@mgr.x 2024-05-09T00:35:32.042 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Failed to reset failed state of unit ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@mgr.x.service: Unit ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@mgr.x.service not loaded. 2024-05-09T00:35:32.194 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-d85814c6-0d9b-11ef-bc97-c7b262605968.target.wants/ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@.service. 2024-05-09T00:35:32.438 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:32 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/4077828924' entity='client.admin' 2024-05-09T00:35:32.438 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:32 smithi005 systemd[1]: Starting Ceph mgr.x for d85814c6-0d9b-11ef-bc97-c7b262605968... 2024-05-09T00:35:32.715 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:32 smithi005 podman[30584]: 2024-05-09 00:35:32.437009864 +0000 UTC m=+0.103297112 container create f58a8233012eb42c7810a06f2f9b2c763fd814149165533dee971ce9734c587e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-05-09T00:35:32.715 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:32 smithi005 podman[30584]: 2024-05-09 00:35:32.352243701 +0000 UTC m=+0.018530970 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T00:35:32.716 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:32 smithi005 podman[30584]: 2024-05-09 00:35:32.648167026 +0000 UTC m=+0.314454286 container init f58a8233012eb42c7810a06f2f9b2c763fd814149165533dee971ce9734c587e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0) 2024-05-09T00:35:32.716 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:32 smithi005 podman[30584]: 2024-05-09 00:35:32.65097264 +0000 UTC m=+0.317259889 container start f58a8233012eb42c7810a06f2f9b2c763fd814149165533dee971ce9734c587e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-05-09T00:35:32.716 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:32 smithi005 bash[30584]: f58a8233012eb42c7810a06f2f9b2c763fd814149165533dee971ce9734c587e 2024-05-09T00:35:32.731 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-09T00:35:32.731 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout disabled 2024-05-09T00:35:32.739 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-09T00:35:32.739 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout inactive 2024-05-09T00:35:32.739 INFO:teuthology.orchestra.run.smithi005.stdout:firewalld.service is not enabled 2024-05-09T00:35:32.739 INFO:teuthology.orchestra.run.smithi005.stdout:Not possible to enable service . firewalld.service is not available 2024-05-09T00:35:32.749 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-09T00:35:32.749 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout disabled 2024-05-09T00:35:32.758 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-09T00:35:32.758 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout inactive 2024-05-09T00:35:32.758 INFO:teuthology.orchestra.run.smithi005.stdout:firewalld.service is not enabled 2024-05-09T00:35:32.758 INFO:teuthology.orchestra.run.smithi005.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-05-09T00:35:32.758 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for mgr to start... 2024-05-09T00:35:32.758 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for mgr... 2024-05-09T00:35:32.979 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:32 smithi005 systemd[1]: Started Ceph mgr.x for d85814c6-0d9b-11ef-bc97-c7b262605968. 2024-05-09T00:35:32.979 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:32 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:35:32.830+0000 7f28a5716200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T00:35:33.229 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:33 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:35:33.030+0000 7f28a5716200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T00:35:33.229 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:33 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:35:33.138+0000 7f28a5716200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T00:35:33.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:33 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/832217075' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-09T00:35:33.541 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:33 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:35:33.475+0000 7f28a5716200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T00:35:33.905 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:33 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:35:33.567+0000 7f28a5716200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T00:35:33.908 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-05-09T00:35:33.924 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2024-05-09T00:35:33.924 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsid": "d85814c6-0d9b-11ef-bc97-c7b262605968", 2024-05-09T00:35:33.925 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "health": { 2024-05-09T00:35:33.925 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-09T00:35:33.925 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-09T00:35:33.925 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-09T00:35:33.925 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-05-09T00:35:33.925 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-09T00:35:33.925 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-09T00:35:33.925 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 0 2024-05-09T00:35:33.925 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-05-09T00:35:33.926 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-09T00:35:33.926 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "a" 2024-05-09T00:35:33.926 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-05-09T00:35:33.926 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-05-09T00:35:33.926 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-09T00:35:33.926 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T00:35:33.926 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-09T00:35:33.926 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-09T00:35:33.926 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-05-09T00:35:33.927 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-09T00:35:33.927 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T00:35:33.927 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-09T00:35:33.927 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-09T00:35:33.927 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-09T00:35:33.927 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-09T00:35:33.927 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-09T00:35:33.927 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-09T00:35:33.927 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-05-09T00:35:33.927 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-09T00:35:33.927 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-09T00:35:33.927 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-09T00:35:33.927 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-09T00:35:33.928 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-09T00:35:33.928 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-09T00:35:33.928 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-09T00:35:33.928 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-09T00:35:33.928 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-09T00:35:33.928 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-05-09T00:35:33.928 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-09T00:35:33.928 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T00:35:33.928 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-09T00:35:33.928 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-09T00:35:33.928 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-05-09T00:35:33.928 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-09T00:35:33.928 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-09T00:35:33.929 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-09T00:35:33.929 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-09T00:35:33.929 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-09T00:35:33.929 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-09T00:35:33.929 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "restful" 2024-05-09T00:35:33.929 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-05-09T00:35:33.929 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T00:35:33.929 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-05-09T00:35:33.929 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-09T00:35:33.929 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T00:35:33.929 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modified": "2024-05-09T00:35:25.624041+0000", 2024-05-09T00:35:33.929 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T00:35:33.929 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-05-09T00:35:33.929 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-09T00:35:33.930 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2024-05-09T00:35:33.930 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:33.223+0000 7f2472f55700 1 Processor -- start 2024-05-09T00:35:33.930 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:33.223+0000 7f2472f55700 1 -- start start 2024-05-09T00:35:33.930 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:33.224+0000 7f2472f55700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f246c1083e0 0x7f246c1087f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:35:33.930 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:33.224+0000 7f2472f55700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f246c07cdd0 con 0x7f246c1083e0 2024-05-09T00:35:33.930 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:33.224+0000 7f2470cf1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f246c1083e0 0x7f246c1087f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:35:33.930 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:33.224+0000 7f2470cf1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f246c1083e0 0x7f246c1087f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53304/0 (socket says 172.21.15.5:53304) 2024-05-09T00:35:33.930 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:33.224+0000 7f2470cf1700 1 -- 172.21.15.5:0/971185372 learned_addr learned my addr 172.21.15.5:0/971185372 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:35:33.930 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:33.224+0000 7f2470cf1700 1 -- 172.21.15.5:0/971185372 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f246c108d30 con 0x7f246c1083e0 2024-05-09T00:35:33.930 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:33.225+0000 7f2470cf1700 1 --2- 172.21.15.5:0/971185372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f246c1083e0 0x7f246c1087f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f24600098a0 tx=0x7f2460009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90df908bc799cc9e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:35:33.930 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:33.225+0000 7f246b7fe700 1 -- 172.21.15.5:0/971185372 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2460004160 con 0x7f246c1083e0 2024-05-09T00:35:33.930 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:33.225+0000 7f246b7fe700 1 -- 172.21.15.5:0/971185372 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2460004300 con 0x7f246c1083e0 2024-05-09T00:35:33.930 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:33.225+0000 7f246b7fe700 1 -- 172.21.15.5:0/971185372 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2460011af0 con 0x7f246c1083e0 2024-05-09T00:35:33.931 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:33.226+0000 7f2472f55700 1 -- 172.21.15.5:0/971185372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f246c1083e0 msgr2=0x7f246c1087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:35:33.931 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:33.226+0000 7f2472f55700 1 --2- 172.21.15.5:0/971185372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f246c1083e0 0x7f246c1087f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f24600098a0 tx=0x7f2460009bb0 comp rx=0 tx=0).stop 2024-05-09T00:35:33.931 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:33.226+0000 7f2472f55700 1 -- 172.21.15.5:0/971185372 shutdown_connections 2024-05-09T00:35:33.931 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:33.226+0000 7f2472f55700 1 --2- 172.21.15.5:0/971185372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f246c1083e0 0x7f246c1087f0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:35:33.931 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:33.226+0000 7f2472f55700 1 -- 172.21.15.5:0/971185372 >> 172.21.15.5:0/971185372 conn(0x7f246c104050 msgr2=0x7f246c1064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:35:33.931 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:33.226+0000 7f2472f55700 1 -- 172.21.15.5:0/971185372 shutdown_connections 2024-05-09T00:35:33.931 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:33.226+0000 7f2472f55700 1 -- 172.21.15.5:0/971185372 wait complete. 2024-05-09T00:35:33.931 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:33.226+0000 7f2472f55700 1 Processor -- start 2024-05-09T00:35:33.931 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:33.227+0000 7f2472f55700 1 -- start start 2024-05-09T00:35:33.931 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:33.227+0000 7f2472f55700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f246c1083e0 0x7f246c1a00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:35:33.931 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:33.227+0000 7f2472f55700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f246c109c30 con 0x7f246c1083e0 2024-05-09T00:35:33.931 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:33.227+0000 7f2470cf1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f246c1083e0 0x7f246c1a00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:35:33.931 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:33.227+0000 7f2470cf1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f246c1083e0 0x7f246c1a00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53312/0 (socket says 172.21.15.5:53312) 2024-05-09T00:35:33.931 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:33.227+0000 7f2470cf1700 1 -- 172.21.15.5:0/832217075 learned_addr learned my addr 172.21.15.5:0/832217075 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:35:33.932 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:33.227+0000 7f2470cf1700 1 -- 172.21.15.5:0/832217075 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f246c1a05e0 con 0x7f246c1083e0 2024-05-09T00:35:33.932 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:33.227+0000 7f2470cf1700 1 --2- 172.21.15.5:0/832217075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f246c1083e0 0x7f246c1a00a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f24600117b0 tx=0x7f2460011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:35:33.932 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:33.228+0000 7f2469ffb700 1 -- 172.21.15.5:0/832217075 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24600039c0 con 0x7f246c1083e0 2024-05-09T00:35:33.932 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:33.228+0000 7f2469ffb700 1 -- 172.21.15.5:0/832217075 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2460003b60 con 0x7f246c1083e0 2024-05-09T00:35:33.932 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:33.228+0000 7f2472f55700 1 -- 172.21.15.5:0/832217075 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f246c1a0870 con 0x7f246c1083e0 2024-05-09T00:35:33.932 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:33.228+0000 7f2469ffb700 1 -- 172.21.15.5:0/832217075 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24600196a0 con 0x7f246c1083e0 2024-05-09T00:35:33.932 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:33.228+0000 7f2472f55700 1 -- 172.21.15.5:0/832217075 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f246c1a3480 con 0x7f246c1083e0 2024-05-09T00:35:33.932 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:33.228+0000 7f2469ffb700 1 -- 172.21.15.5:0/832217075 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2460019cf0 con 0x7f246c1083e0 2024-05-09T00:35:33.932 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:33.228+0000 7f2469ffb700 1 -- 172.21.15.5:0/832217075 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2460024730 con 0x7f246c1083e0 2024-05-09T00:35:33.932 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:33.229+0000 7f2472f55700 1 -- 172.21.15.5:0/832217075 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f246c064e20 con 0x7f246c1083e0 2024-05-09T00:35:33.932 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:33.232+0000 7f2469ffb700 1 -- 172.21.15.5:0/832217075 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f2460043ec0 con 0x7f246c1083e0 2024-05-09T00:35:33.932 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:33.306+0000 7f2472f55700 1 -- 172.21.15.5:0/832217075 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f246c02e8f0 con 0x7f246c1083e0 2024-05-09T00:35:33.932 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:33.307+0000 7f2469ffb700 1 -- 172.21.15.5:0/832217075 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f2460032020 con 0x7f246c1083e0 2024-05-09T00:35:33.933 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:33.308+0000 7f2472f55700 1 -- 172.21.15.5:0/832217075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f246c1083e0 msgr2=0x7f246c1a00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:35:33.933 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:33.308+0000 7f2472f55700 1 --2- 172.21.15.5:0/832217075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f246c1083e0 0x7f246c1a00a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f24600117b0 tx=0x7f2460011b70 comp rx=0 tx=0).stop 2024-05-09T00:35:33.933 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:33.308+0000 7f2472f55700 1 -- 172.21.15.5:0/832217075 shutdown_connections 2024-05-09T00:35:33.933 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:33.308+0000 7f2472f55700 1 --2- 172.21.15.5:0/832217075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f246c1083e0 0x7f246c1a00a0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:35:33.933 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:33.308+0000 7f2472f55700 1 -- 172.21.15.5:0/832217075 >> 172.21.15.5:0/832217075 conn(0x7f246c104050 msgr2=0x7f246c194070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:35:33.933 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:33.308+0000 7f2472f55700 1 -- 172.21.15.5:0/832217075 shutdown_connections 2024-05-09T00:35:33.933 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:33.308+0000 7f2472f55700 1 -- 172.21.15.5:0/832217075 wait complete. 2024-05-09T00:35:33.933 INFO:teuthology.orchestra.run.smithi005.stdout:mgr not available, waiting (1/15)... 2024-05-09T00:35:35.192 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:34 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:35:34.844+0000 7f28a5716200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T00:35:35.193 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:35 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:35:35.024+0000 7f28a5716200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T00:35:35.512 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:35 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:35:35.202+0000 7f28a5716200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T00:35:35.512 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:35 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:35:35.510+0000 7f28a5716200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T00:35:35.791 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:35 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:35:35.608+0000 7f28a5716200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T00:35:36.150 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:35 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:35:35.821+0000 7f28a5716200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T00:35:36.150 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:36 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:35:36.032+0000 7f28a5716200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T00:35:36.787 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:36 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/425813658' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-09T00:35:36.787 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:36 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:35:36.689+0000 7f28a5716200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T00:35:37.026 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-05-09T00:35:37.026 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2024-05-09T00:35:37.027 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsid": "d85814c6-0d9b-11ef-bc97-c7b262605968", 2024-05-09T00:35:37.027 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "health": { 2024-05-09T00:35:37.027 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-09T00:35:37.027 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-09T00:35:37.027 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-09T00:35:37.027 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-05-09T00:35:37.027 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-09T00:35:37.027 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-09T00:35:37.027 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 0 2024-05-09T00:35:37.027 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-05-09T00:35:37.027 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-09T00:35:37.027 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "a" 2024-05-09T00:35:37.027 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-05-09T00:35:37.028 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-05-09T00:35:37.028 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-09T00:35:37.028 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T00:35:37.028 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-09T00:35:37.028 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-09T00:35:37.028 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-05-09T00:35:37.028 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-09T00:35:37.028 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T00:35:37.028 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-09T00:35:37.028 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-09T00:35:37.028 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-09T00:35:37.036 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-09T00:35:37.036 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-09T00:35:37.036 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-09T00:35:37.036 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-05-09T00:35:37.037 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-09T00:35:37.037 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-09T00:35:37.037 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-09T00:35:37.037 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-09T00:35:37.037 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-09T00:35:37.037 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-09T00:35:37.037 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-09T00:35:37.037 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-09T00:35:37.037 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-09T00:35:37.037 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-05-09T00:35:37.037 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-09T00:35:37.037 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T00:35:37.037 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-09T00:35:37.038 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-09T00:35:37.038 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-05-09T00:35:37.038 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-09T00:35:37.038 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-09T00:35:37.038 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-09T00:35:37.038 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-09T00:35:37.038 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-09T00:35:37.038 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-09T00:35:37.038 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "restful" 2024-05-09T00:35:37.038 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-05-09T00:35:37.038 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T00:35:37.038 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-05-09T00:35:37.038 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-09T00:35:37.039 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T00:35:37.039 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modified": "2024-05-09T00:35:25.624041+0000", 2024-05-09T00:35:37.039 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T00:35:37.039 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-05-09T00:35:37.039 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-09T00:35:37.039 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2024-05-09T00:35:37.039 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:36.381+0000 7fc79de9c700 1 Processor -- start 2024-05-09T00:35:37.039 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:36.381+0000 7fc79de9c700 1 -- start start 2024-05-09T00:35:37.039 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:36.382+0000 7fc79de9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc79810a610 0x7fc79810ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:35:37.039 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:36.382+0000 7fc79de9c700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc79807cdb0 con 0x7fc79810a610 2024-05-09T00:35:37.039 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:36.382+0000 7fc7977fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc79810a610 0x7fc79810ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:35:37.039 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:36.382+0000 7fc7977fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc79810a610 0x7fc79810ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47854/0 (socket says 172.21.15.5:47854) 2024-05-09T00:35:37.039 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:36.382+0000 7fc7977fe700 1 -- 172.21.15.5:0/900937277 learned_addr learned my addr 172.21.15.5:0/900937277 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:35:37.040 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:36.382+0000 7fc7977fe700 1 -- 172.21.15.5:0/900937277 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc79810cf70 con 0x7fc79810a610 2024-05-09T00:35:37.040 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:36.382+0000 7fc7977fe700 1 --2- 172.21.15.5:0/900937277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc79810a610 0x7fc79810ca30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc7880098a0 tx=0x7fc788009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=972f1b1e4e80d8f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:35:37.040 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:36.382+0000 7fc7967fc700 1 -- 172.21.15.5:0/900937277 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc788004160 con 0x7fc79810a610 2024-05-09T00:35:37.040 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:36.383+0000 7fc7967fc700 1 -- 172.21.15.5:0/900937277 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc788004300 con 0x7fc79810a610 2024-05-09T00:35:37.040 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:36.383+0000 7fc7967fc700 1 -- 172.21.15.5:0/900937277 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc788011af0 con 0x7fc79810a610 2024-05-09T00:35:37.040 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:36.383+0000 7fc79de9c700 1 -- 172.21.15.5:0/900937277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc79810a610 msgr2=0x7fc79810ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:35:37.040 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:36.383+0000 7fc79de9c700 1 --2- 172.21.15.5:0/900937277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc79810a610 0x7fc79810ca30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc7880098a0 tx=0x7fc788009bb0 comp rx=0 tx=0).stop 2024-05-09T00:35:37.040 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:36.383+0000 7fc79de9c700 1 -- 172.21.15.5:0/900937277 shutdown_connections 2024-05-09T00:35:37.040 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:36.383+0000 7fc79de9c700 1 --2- 172.21.15.5:0/900937277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc79810a610 0x7fc79810ca30 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:35:37.040 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:36.384+0000 7fc79de9c700 1 -- 172.21.15.5:0/900937277 >> 172.21.15.5:0/900937277 conn(0x7fc798104050 msgr2=0x7fc7981064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:35:37.040 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:36.384+0000 7fc79de9c700 1 -- 172.21.15.5:0/900937277 shutdown_connections 2024-05-09T00:35:37.040 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:36.384+0000 7fc79de9c700 1 -- 172.21.15.5:0/900937277 wait complete. 2024-05-09T00:35:37.040 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:36.384+0000 7fc79de9c700 1 Processor -- start 2024-05-09T00:35:37.041 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:36.384+0000 7fc79de9c700 1 -- start start 2024-05-09T00:35:37.041 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:36.385+0000 7fc79de9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc79810a610 0x7fc79819dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:35:37.041 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:36.385+0000 7fc79de9c700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc79810de70 con 0x7fc79810a610 2024-05-09T00:35:37.041 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:36.385+0000 7fc7977fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc79810a610 0x7fc79819dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:35:37.041 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:36.385+0000 7fc7977fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc79810a610 0x7fc79819dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47862/0 (socket says 172.21.15.5:47862) 2024-05-09T00:35:37.041 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:36.385+0000 7fc7977fe700 1 -- 172.21.15.5:0/425813658 learned_addr learned my addr 172.21.15.5:0/425813658 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:35:37.041 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:36.385+0000 7fc7977fe700 1 -- 172.21.15.5:0/425813658 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc79819e3e0 con 0x7fc79810a610 2024-05-09T00:35:37.041 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:36.385+0000 7fc7977fe700 1 --2- 172.21.15.5:0/425813658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc79810a610 0x7fc79819dea0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc788009fb0 tx=0x7fc788011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:35:37.041 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:36.386+0000 7fc794ff9700 1 -- 172.21.15.5:0/425813658 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc788011bc0 con 0x7fc79810a610 2024-05-09T00:35:37.041 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:36.386+0000 7fc79de9c700 1 -- 172.21.15.5:0/425813658 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc79819e670 con 0x7fc79810a610 2024-05-09T00:35:37.041 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:36.386+0000 7fc794ff9700 1 -- 172.21.15.5:0/425813658 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc788024440 con 0x7fc79810a610 2024-05-09T00:35:37.041 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:36.386+0000 7fc794ff9700 1 -- 172.21.15.5:0/425813658 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc78801a5c0 con 0x7fc79810a610 2024-05-09T00:35:37.041 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:36.386+0000 7fc79de9c700 1 -- 172.21.15.5:0/425813658 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7981a1280 con 0x7fc79810a610 2024-05-09T00:35:37.042 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:36.386+0000 7fc794ff9700 1 -- 172.21.15.5:0/425813658 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc78801ac10 con 0x7fc79810a610 2024-05-09T00:35:37.042 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:36.386+0000 7fc794ff9700 1 -- 172.21.15.5:0/425813658 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc788019700 con 0x7fc79810a610 2024-05-09T00:35:37.042 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:36.387+0000 7fc79de9c700 1 -- 172.21.15.5:0/425813658 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc798064e20 con 0x7fc79810a610 2024-05-09T00:35:37.042 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:36.389+0000 7fc794ff9700 1 -- 172.21.15.5:0/425813658 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fc78802ddf0 con 0x7fc79810a610 2024-05-09T00:35:37.042 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:36.463+0000 7fc79de9c700 1 -- 172.21.15.5:0/425813658 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fc7981a1450 con 0x7fc79810a610 2024-05-09T00:35:37.042 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:36.464+0000 7fc794ff9700 1 -- 172.21.15.5:0/425813658 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fc7880245f0 con 0x7fc79810a610 2024-05-09T00:35:37.042 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:36.465+0000 7fc79de9c700 1 -- 172.21.15.5:0/425813658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc79810a610 msgr2=0x7fc79819dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:35:37.042 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:36.465+0000 7fc79de9c700 1 --2- 172.21.15.5:0/425813658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc79810a610 0x7fc79819dea0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc788009fb0 tx=0x7fc788011890 comp rx=0 tx=0).stop 2024-05-09T00:35:37.042 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:36.466+0000 7fc79de9c700 1 -- 172.21.15.5:0/425813658 shutdown_connections 2024-05-09T00:35:37.042 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:36.466+0000 7fc79de9c700 1 --2- 172.21.15.5:0/425813658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc79810a610 0x7fc79819dea0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:35:37.042 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:36.466+0000 7fc79de9c700 1 -- 172.21.15.5:0/425813658 >> 172.21.15.5:0/425813658 conn(0x7fc798104050 msgr2=0x7fc798105a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:35:37.042 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:36.466+0000 7fc79de9c700 1 -- 172.21.15.5:0/425813658 shutdown_connections 2024-05-09T00:35:37.042 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:36.466+0000 7fc79de9c700 1 -- 172.21.15.5:0/425813658 wait complete. 2024-05-09T00:35:37.043 INFO:teuthology.orchestra.run.smithi005.stdout:mgr not available, waiting (2/15)... 2024-05-09T00:35:37.043 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:36 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:35:36.923+0000 7f28a5716200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-09T00:35:37.791 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:37 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:35:37.368+0000 7f28a5716200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T00:35:37.791 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:37 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:35:37.468+0000 7f28a5716200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T00:35:38.231 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:37 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:35:37.854+0000 7f28a5716200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T00:35:38.541 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:38 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:35:38.230+0000 7f28a5716200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T00:35:38.541 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:38 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:35:38.326+0000 7f28a5716200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T00:35:39.041 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:38 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:35:38.580+0000 7f28a5716200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T00:35:39.042 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:38 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:35:38.720+0000 7f28a5716200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T00:35:39.042 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:38 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:35:38.819+0000 7f28a5716200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T00:35:39.465 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:39 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:35:39.168+0000 7f28a5716200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T00:35:39.466 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:39 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:35:39.265+0000 7f28a5716200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T00:35:39.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:39 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/426411161' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-09T00:35:40.174 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-05-09T00:35:40.175 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2024-05-09T00:35:40.175 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsid": "d85814c6-0d9b-11ef-bc97-c7b262605968", 2024-05-09T00:35:40.175 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "health": { 2024-05-09T00:35:40.175 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-09T00:35:40.175 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-09T00:35:40.175 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-09T00:35:40.175 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-05-09T00:35:40.175 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-09T00:35:40.175 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-09T00:35:40.175 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 0 2024-05-09T00:35:40.175 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-05-09T00:35:40.175 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-09T00:35:40.175 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "a" 2024-05-09T00:35:40.176 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-05-09T00:35:40.176 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-05-09T00:35:40.176 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-09T00:35:40.176 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T00:35:40.176 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-09T00:35:40.176 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-09T00:35:40.176 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-05-09T00:35:40.176 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-09T00:35:40.176 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T00:35:40.176 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-09T00:35:40.176 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-09T00:35:40.176 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-09T00:35:40.184 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-09T00:35:40.184 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-09T00:35:40.184 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-09T00:35:40.184 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-05-09T00:35:40.184 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-09T00:35:40.185 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-09T00:35:40.185 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-09T00:35:40.185 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-09T00:35:40.185 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-09T00:35:40.185 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-09T00:35:40.185 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-09T00:35:40.185 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-09T00:35:40.185 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-09T00:35:40.185 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-05-09T00:35:40.185 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-09T00:35:40.186 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T00:35:40.186 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-09T00:35:40.186 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-09T00:35:40.186 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-05-09T00:35:40.186 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-09T00:35:40.186 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-09T00:35:40.186 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-09T00:35:40.186 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-09T00:35:40.186 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-09T00:35:40.186 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-09T00:35:40.186 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "restful" 2024-05-09T00:35:40.186 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-05-09T00:35:40.186 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T00:35:40.187 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-05-09T00:35:40.187 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-09T00:35:40.187 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T00:35:40.187 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modified": "2024-05-09T00:35:25.624041+0000", 2024-05-09T00:35:40.187 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T00:35:40.187 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-05-09T00:35:40.187 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-09T00:35:40.187 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2024-05-09T00:35:40.187 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:39.493+0000 7fcc05167700 1 Processor -- start 2024-05-09T00:35:40.187 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:39.493+0000 7fcc05167700 1 -- start start 2024-05-09T00:35:40.188 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:39.493+0000 7fcc05167700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc0010a570 0x7fcc0010c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:35:40.188 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:39.493+0000 7fcc05167700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcc0007d700 con 0x7fcc0010a570 2024-05-09T00:35:40.188 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:39.493+0000 7fcbfed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc0010a570 0x7fcc0010c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:35:40.188 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:39.493+0000 7fcbfed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc0010a570 0x7fcc0010c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47864/0 (socket says 172.21.15.5:47864) 2024-05-09T00:35:40.188 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:39.493+0000 7fcbfed9d700 1 -- 172.21.15.5:0/2370989894 learned_addr learned my addr 172.21.15.5:0/2370989894 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:35:40.188 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:39.494+0000 7fcbfed9d700 1 -- 172.21.15.5:0/2370989894 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc0010ced0 con 0x7fcc0010a570 2024-05-09T00:35:40.188 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:39.494+0000 7fcbfed9d700 1 --2- 172.21.15.5:0/2370989894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc0010a570 0x7fcc0010c990 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcbf00098a0 tx=0x7fcbf0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5da5b1b35bd59f0b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:35:40.188 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:39.494+0000 7fcbfdd9b700 1 -- 172.21.15.5:0/2370989894 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcbf0004160 con 0x7fcc0010a570 2024-05-09T00:35:40.188 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:39.494+0000 7fcbfdd9b700 1 -- 172.21.15.5:0/2370989894 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fcbf0004300 con 0x7fcc0010a570 2024-05-09T00:35:40.188 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:39.494+0000 7fcbfdd9b700 1 -- 172.21.15.5:0/2370989894 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcbf0011af0 con 0x7fcc0010a570 2024-05-09T00:35:40.188 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:39.495+0000 7fcc05167700 1 -- 172.21.15.5:0/2370989894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc0010a570 msgr2=0x7fcc0010c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:35:40.188 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:39.495+0000 7fcc05167700 1 --2- 172.21.15.5:0/2370989894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc0010a570 0x7fcc0010c990 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcbf00098a0 tx=0x7fcbf0009bb0 comp rx=0 tx=0).stop 2024-05-09T00:35:40.188 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:39.495+0000 7fcc05167700 1 -- 172.21.15.5:0/2370989894 shutdown_connections 2024-05-09T00:35:40.189 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:39.495+0000 7fcc05167700 1 --2- 172.21.15.5:0/2370989894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc0010a570 0x7fcc0010c990 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:35:40.189 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:39.495+0000 7fcc05167700 1 -- 172.21.15.5:0/2370989894 >> 172.21.15.5:0/2370989894 conn(0x7fcc00103ff0 msgr2=0x7fcc00106480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:35:40.189 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:39.495+0000 7fcc05167700 1 -- 172.21.15.5:0/2370989894 shutdown_connections 2024-05-09T00:35:40.189 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:39.495+0000 7fcc05167700 1 -- 172.21.15.5:0/2370989894 wait complete. 2024-05-09T00:35:40.189 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:39.496+0000 7fcc05167700 1 Processor -- start 2024-05-09T00:35:40.189 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:39.496+0000 7fcc05167700 1 -- start start 2024-05-09T00:35:40.189 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:39.496+0000 7fcc05167700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc0010a570 0x7fcc0019df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:35:40.189 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:39.496+0000 7fcc05167700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcc0010de90 con 0x7fcc0010a570 2024-05-09T00:35:40.189 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:39.496+0000 7fcbfed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc0010a570 0x7fcc0019df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:35:40.189 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:39.496+0000 7fcbfed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc0010a570 0x7fcc0019df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47880/0 (socket says 172.21.15.5:47880) 2024-05-09T00:35:40.189 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:39.496+0000 7fcbfed9d700 1 -- 172.21.15.5:0/426411161 learned_addr learned my addr 172.21.15.5:0/426411161 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:35:40.189 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:39.497+0000 7fcbfed9d700 1 -- 172.21.15.5:0/426411161 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc0019e450 con 0x7fcc0010a570 2024-05-09T00:35:40.189 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:39.497+0000 7fcbfed9d700 1 --2- 172.21.15.5:0/426411161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc0010a570 0x7fcc0019df10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0009fb0 tx=0x7fcbf0011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:35:40.189 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:39.497+0000 7fcbe7fff700 1 -- 172.21.15.5:0/426411161 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcbf0011bc0 con 0x7fcc0010a570 2024-05-09T00:35:40.190 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:39.497+0000 7fcbe7fff700 1 -- 172.21.15.5:0/426411161 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fcbf0024440 con 0x7fcc0010a570 2024-05-09T00:35:40.190 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:39.497+0000 7fcc05167700 1 -- 172.21.15.5:0/426411161 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc0019e6e0 con 0x7fcc0010a570 2024-05-09T00:35:40.190 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:39.497+0000 7fcbe7fff700 1 -- 172.21.15.5:0/426411161 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcbf001a5c0 con 0x7fcc0010a570 2024-05-09T00:35:40.190 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:39.497+0000 7fcc05167700 1 -- 172.21.15.5:0/426411161 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc001a12f0 con 0x7fcc0010a570 2024-05-09T00:35:40.190 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:39.498+0000 7fcbe7fff700 1 -- 172.21.15.5:0/426411161 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcbf001abc0 con 0x7fcc0010a570 2024-05-09T00:35:40.190 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:39.498+0000 7fcbe7fff700 1 -- 172.21.15.5:0/426411161 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fcbf0019700 con 0x7fcc0010a570 2024-05-09T00:35:40.190 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:39.498+0000 7fcc05167700 1 -- 172.21.15.5:0/426411161 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc00064e20 con 0x7fcc0010a570 2024-05-09T00:35:40.191 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:39.501+0000 7fcbe7fff700 1 -- 172.21.15.5:0/426411161 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fcbf002dd90 con 0x7fcc0010a570 2024-05-09T00:35:40.191 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:39.576+0000 7fcc05167700 1 -- 172.21.15.5:0/426411161 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fcc001a14c0 con 0x7fcc0010a570 2024-05-09T00:35:40.191 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:39.576+0000 7fcbe7fff700 1 -- 172.21.15.5:0/426411161 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fcbf00245f0 con 0x7fcc0010a570 2024-05-09T00:35:40.191 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:39.578+0000 7fcc05167700 1 -- 172.21.15.5:0/426411161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc0010a570 msgr2=0x7fcc0019df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:35:40.191 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:39.578+0000 7fcc05167700 1 --2- 172.21.15.5:0/426411161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc0010a570 0x7fcc0019df10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0009fb0 tx=0x7fcbf0011890 comp rx=0 tx=0).stop 2024-05-09T00:35:40.191 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:39.578+0000 7fcc05167700 1 -- 172.21.15.5:0/426411161 shutdown_connections 2024-05-09T00:35:40.191 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:39.578+0000 7fcc05167700 1 --2- 172.21.15.5:0/426411161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc0010a570 0x7fcc0019df10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:35:40.191 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:39.578+0000 7fcc05167700 1 -- 172.21.15.5:0/426411161 >> 172.21.15.5:0/426411161 conn(0x7fcc00103ff0 msgr2=0x7fcc00104a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:35:40.191 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:39.578+0000 7fcc05167700 1 -- 172.21.15.5:0/426411161 shutdown_connections 2024-05-09T00:35:40.191 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:39.578+0000 7fcc05167700 1 -- 172.21.15.5:0/426411161 wait complete. 2024-05-09T00:35:40.191 INFO:teuthology.orchestra.run.smithi005.stdout:mgr not available, waiting (3/15)... 2024-05-09T00:35:40.291 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:39 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:35:39.922+0000 7f28a5716200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T00:35:40.291 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:40 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:35:40.147+0000 7f28a5716200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T00:35:41.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:40 smithi005 ceph-mon[30365]: Activating manager daemon x 2024-05-09T00:35:41.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:40 smithi005 ceph-mon[30365]: mgrmap e2: x(active, starting, since 0.00356224s) 2024-05-09T00:35:41.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:40 smithi005 ceph-mon[30365]: from='mgr.14100 172.21.15.5:0/4177606167' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-09T00:35:41.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:40 smithi005 ceph-mon[30365]: from='mgr.14100 172.21.15.5:0/4177606167' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-09T00:35:41.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:40 smithi005 ceph-mon[30365]: from='mgr.14100 172.21.15.5:0/4177606167' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-09T00:35:41.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:40 smithi005 ceph-mon[30365]: from='mgr.14100 172.21.15.5:0/4177606167' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T00:35:41.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:40 smithi005 ceph-mon[30365]: from='mgr.14100 172.21.15.5:0/4177606167' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-09T00:35:41.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:40 smithi005 ceph-mon[30365]: Manager daemon x is now available 2024-05-09T00:35:41.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:40 smithi005 ceph-mon[30365]: from='mgr.14100 172.21.15.5:0/4177606167' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-09T00:35:41.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:40 smithi005 ceph-mon[30365]: from='mgr.14100 172.21.15.5:0/4177606167' entity='mgr.x' 2024-05-09T00:35:41.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:40 smithi005 ceph-mon[30365]: from='mgr.14100 172.21.15.5:0/4177606167' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-09T00:35:41.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:40 smithi005 ceph-mon[30365]: from='mgr.14100 172.21.15.5:0/4177606167' entity='mgr.x' 2024-05-09T00:35:41.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:40 smithi005 ceph-mon[30365]: from='mgr.14100 172.21.15.5:0/4177606167' entity='mgr.x' 2024-05-09T00:35:42.430 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:42 smithi005 ceph-mon[30365]: mgrmap e3: x(active, since 1.00921s) 2024-05-09T00:35:43.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:43 smithi005 ceph-mon[30365]: mgrmap e4: x(active, since 2s) 2024-05-09T00:35:43.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:43 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/1712357138' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-09T00:35:43.495 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-05-09T00:35:43.497 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2024-05-09T00:35:43.498 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsid": "d85814c6-0d9b-11ef-bc97-c7b262605968", 2024-05-09T00:35:43.498 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "health": { 2024-05-09T00:35:43.498 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-09T00:35:43.498 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-09T00:35:43.498 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-09T00:35:43.498 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-05-09T00:35:43.498 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-09T00:35:43.498 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-09T00:35:43.498 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 0 2024-05-09T00:35:43.498 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-05-09T00:35:43.498 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-09T00:35:43.498 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "a" 2024-05-09T00:35:43.499 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-05-09T00:35:43.499 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-05-09T00:35:43.499 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-09T00:35:43.499 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T00:35:43.499 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-09T00:35:43.499 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-09T00:35:43.499 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-05-09T00:35:43.499 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-09T00:35:43.499 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T00:35:43.499 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-09T00:35:43.499 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-09T00:35:43.510 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-09T00:35:43.510 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-09T00:35:43.510 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-09T00:35:43.510 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-09T00:35:43.510 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-05-09T00:35:43.510 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-09T00:35:43.510 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-09T00:35:43.511 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-09T00:35:43.511 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-09T00:35:43.511 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-09T00:35:43.511 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-09T00:35:43.511 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-09T00:35:43.511 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-09T00:35:43.511 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-09T00:35:43.511 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-05-09T00:35:43.511 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-09T00:35:43.511 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T00:35:43.511 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-09T00:35:43.511 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-09T00:35:43.511 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-05-09T00:35:43.512 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-09T00:35:43.512 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-09T00:35:43.512 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-09T00:35:43.512 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-09T00:35:43.512 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-09T00:35:43.512 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-09T00:35:43.512 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "restful" 2024-05-09T00:35:43.512 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-05-09T00:35:43.512 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T00:35:43.512 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-05-09T00:35:43.512 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-09T00:35:43.512 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T00:35:43.512 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modified": "2024-05-09T00:35:25.624041+0000", 2024-05-09T00:35:43.513 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T00:35:43.513 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-05-09T00:35:43.513 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-09T00:35:43.513 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2024-05-09T00:35:43.513 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:42.630+0000 7f984d5e3700 1 Processor -- start 2024-05-09T00:35:43.513 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:42.630+0000 7f984d5e3700 1 -- start start 2024-05-09T00:35:43.513 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:42.630+0000 7f984d5e3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9848107d60 0x7f9848108170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:35:43.513 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:42.630+0000 7f984d5e3700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9848108740 con 0x7f9848107d60 2024-05-09T00:35:43.513 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:42.630+0000 7f9846ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9848107d60 0x7f9848108170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:35:43.513 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:42.630+0000 7f9846ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9848107d60 0x7f9848108170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47946/0 (socket says 172.21.15.5:47946) 2024-05-09T00:35:43.513 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:42.630+0000 7f9846ffd700 1 -- 172.21.15.5:0/182364420 learned_addr learned my addr 172.21.15.5:0/182364420 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:35:43.513 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:42.630+0000 7f9846ffd700 1 -- 172.21.15.5:0/182364420 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f984810ad60 con 0x7f9848107d60 2024-05-09T00:35:43.514 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:42.631+0000 7f9846ffd700 1 --2- 172.21.15.5:0/182364420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9848107d60 0x7f9848108170 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f98380098a0 tx=0x7f9838009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=564848c96e2585d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:35:43.514 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:42.631+0000 7f9845ffb700 1 -- 172.21.15.5:0/182364420 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9838004160 con 0x7f9848107d60 2024-05-09T00:35:43.514 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:42.631+0000 7f9845ffb700 1 -- 172.21.15.5:0/182364420 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9838004300 con 0x7f9848107d60 2024-05-09T00:35:43.514 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:42.631+0000 7f9845ffb700 1 -- 172.21.15.5:0/182364420 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9838011af0 con 0x7f9848107d60 2024-05-09T00:35:43.514 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:42.632+0000 7f984d5e3700 1 -- 172.21.15.5:0/182364420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9848107d60 msgr2=0x7f9848108170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:35:43.514 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:42.632+0000 7f984d5e3700 1 --2- 172.21.15.5:0/182364420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9848107d60 0x7f9848108170 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f98380098a0 tx=0x7f9838009bb0 comp rx=0 tx=0).stop 2024-05-09T00:35:43.514 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:42.632+0000 7f984d5e3700 1 -- 172.21.15.5:0/182364420 shutdown_connections 2024-05-09T00:35:43.514 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:42.632+0000 7f984d5e3700 1 --2- 172.21.15.5:0/182364420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9848107d60 0x7f9848108170 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:35:43.514 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:42.632+0000 7f984d5e3700 1 -- 172.21.15.5:0/182364420 >> 172.21.15.5:0/182364420 conn(0x7f98481037d0 msgr2=0x7f9848105c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:35:43.514 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:42.632+0000 7f984d5e3700 1 -- 172.21.15.5:0/182364420 shutdown_connections 2024-05-09T00:35:43.514 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:42.632+0000 7f984d5e3700 1 -- 172.21.15.5:0/182364420 wait complete. 2024-05-09T00:35:43.514 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:42.632+0000 7f984d5e3700 1 Processor -- start 2024-05-09T00:35:43.515 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:42.633+0000 7f984d5e3700 1 -- start start 2024-05-09T00:35:43.515 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:42.633+0000 7f984d5e3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9848107d60 0x7f984810a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:35:43.515 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:42.633+0000 7f984d5e3700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f984810df90 con 0x7f9848107d60 2024-05-09T00:35:43.515 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:42.633+0000 7f9846ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9848107d60 0x7f984810a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:35:43.515 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:42.633+0000 7f9846ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9848107d60 0x7f984810a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47956/0 (socket says 172.21.15.5:47956) 2024-05-09T00:35:43.515 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:42.633+0000 7f9846ffd700 1 -- 172.21.15.5:0/1712357138 learned_addr learned my addr 172.21.15.5:0/1712357138 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:35:43.515 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:42.633+0000 7f9846ffd700 1 -- 172.21.15.5:0/1712357138 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f984810aa20 con 0x7f9848107d60 2024-05-09T00:35:43.515 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:42.633+0000 7f9846ffd700 1 --2- 172.21.15.5:0/1712357138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9848107d60 0x7f984810a4e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9838011670 tx=0x7f9838004280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:35:43.515 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:42.634+0000 7f982ffff700 1 -- 172.21.15.5:0/1712357138 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98380036a0 con 0x7f9848107d60 2024-05-09T00:35:43.515 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:42.634+0000 7f982ffff700 1 -- 172.21.15.5:0/1712357138 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f983801a430 con 0x7f9848107d60 2024-05-09T00:35:43.516 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:42.634+0000 7f984d5e3700 1 -- 172.21.15.5:0/1712357138 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9848108c70 con 0x7f9848107d60 2024-05-09T00:35:43.516 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:42.634+0000 7f982ffff700 1 -- 172.21.15.5:0/1712357138 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9838019550 con 0x7f9848107d60 2024-05-09T00:35:43.516 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:42.634+0000 7f984d5e3700 1 -- 172.21.15.5:0/1712357138 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9848109150 con 0x7f9848107d60 2024-05-09T00:35:43.516 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:42.635+0000 7f982ffff700 1 -- 172.21.15.5:0/1712357138 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7f98380197c0 con 0x7f9848107d60 2024-05-09T00:35:43.516 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:42.635+0000 7f984d5e3700 1 -- 172.21.15.5:0/1712357138 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9848064e20 con 0x7f9848107d60 2024-05-09T00:35:43.516 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:42.635+0000 7f982ffff700 1 --2- 172.21.15.5:0/1712357138 >> [v2:172.21.15.5:6800/752513054,v1:172.21.15.5:6801/752513054] conn(0x7f983003e420 0x7f9830040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:35:43.516 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:42.635+0000 7f982ffff700 1 -- 172.21.15.5:0/1712357138 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9838053700 con 0x7f9848107d60 2024-05-09T00:35:43.516 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:42.635+0000 7f98467fc700 1 --2- 172.21.15.5:0/1712357138 >> [v2:172.21.15.5:6800/752513054,v1:172.21.15.5:6801/752513054] conn(0x7f983003e420 0x7f9830040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:35:43.516 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:42.636+0000 7f98467fc700 1 --2- 172.21.15.5:0/1712357138 >> [v2:172.21.15.5:6800/752513054,v1:172.21.15.5:6801/752513054] conn(0x7f983003e420 0x7f9830040930 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f983c006fd0 tx=0x7f983c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:35:43.516 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:42.641+0000 7f982ffff700 1 -- 172.21.15.5:0/1712357138 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9838020240 con 0x7f9848107d60 2024-05-09T00:35:43.516 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:42.949+0000 7f984d5e3700 1 -- 172.21.15.5:0/1712357138 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f9848109720 con 0x7f9848107d60 2024-05-09T00:35:43.516 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:42.949+0000 7f982ffff700 1 -- 172.21.15.5:0/1712357138 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f9838053120 con 0x7f9848107d60 2024-05-09T00:35:43.516 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:42.951+0000 7f984d5e3700 1 -- 172.21.15.5:0/1712357138 >> [v2:172.21.15.5:6800/752513054,v1:172.21.15.5:6801/752513054] conn(0x7f983003e420 msgr2=0x7f9830040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:35:43.517 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:42.951+0000 7f984d5e3700 1 --2- 172.21.15.5:0/1712357138 >> [v2:172.21.15.5:6800/752513054,v1:172.21.15.5:6801/752513054] conn(0x7f983003e420 0x7f9830040930 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f983c006fd0 tx=0x7f983c006e40 comp rx=0 tx=0).stop 2024-05-09T00:35:43.517 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:42.951+0000 7f984d5e3700 1 -- 172.21.15.5:0/1712357138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9848107d60 msgr2=0x7f984810a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:35:43.517 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:42.951+0000 7f984d5e3700 1 --2- 172.21.15.5:0/1712357138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9848107d60 0x7f984810a4e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9838011670 tx=0x7f9838004280 comp rx=0 tx=0).stop 2024-05-09T00:35:43.517 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:42.952+0000 7f984d5e3700 1 -- 172.21.15.5:0/1712357138 shutdown_connections 2024-05-09T00:35:43.517 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:42.952+0000 7f984d5e3700 1 --2- 172.21.15.5:0/1712357138 >> [v2:172.21.15.5:6800/752513054,v1:172.21.15.5:6801/752513054] conn(0x7f983003e420 0x7f9830040930 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:35:43.517 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:42.952+0000 7f984d5e3700 1 --2- 172.21.15.5:0/1712357138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9848107d60 0x7f984810a4e0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:35:43.517 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:42.952+0000 7f984d5e3700 1 -- 172.21.15.5:0/1712357138 >> 172.21.15.5:0/1712357138 conn(0x7f98481037d0 msgr2=0x7f9848105330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:35:43.517 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:42.952+0000 7f984d5e3700 1 -- 172.21.15.5:0/1712357138 shutdown_connections 2024-05-09T00:35:43.517 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:42.952+0000 7f984d5e3700 1 -- 172.21.15.5:0/1712357138 wait complete. 2024-05-09T00:35:43.517 INFO:teuthology.orchestra.run.smithi005.stdout:mgr is available 2024-05-09T00:35:44.490 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:44 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/4282312017' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-09T00:35:44.775 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-05-09T00:35:44.775 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout [global] 2024-05-09T00:35:44.775 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout fsid = d85814c6-0d9b-11ef-bc97-c7b262605968 2024-05-09T00:35:44.775 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-09T00:35:44.776 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-09T00:35:44.776 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-09T00:35:44.776 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-09T00:35:44.776 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-09T00:35:44.776 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-05-09T00:35:44.776 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-09T00:35:44.776 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-09T00:35:44.776 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-05-09T00:35:44.776 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout [osd] 2024-05-09T00:35:44.776 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-09T00:35:44.776 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-09T00:35:44.776 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-09T00:35:44.776 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-09T00:35:44.776 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-09T00:35:44.777 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:43.951+0000 7ff681532700 1 Processor -- start 2024-05-09T00:35:44.777 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:43.951+0000 7ff681532700 1 -- start start 2024-05-09T00:35:44.777 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:43.951+0000 7ff681532700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff67c109aa0 0x7ff67c109eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:35:44.777 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:43.951+0000 7ff681532700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff67c10a4a0 con 0x7ff67c109aa0 2024-05-09T00:35:44.777 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:43.951+0000 7ff67affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff67c109aa0 0x7ff67c109eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:35:44.777 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:43.951+0000 7ff67affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff67c109aa0 0x7ff67c109eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47150/0 (socket says 172.21.15.5:47150) 2024-05-09T00:35:44.777 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:43.952+0000 7ff67affd700 1 -- 172.21.15.5:0/3568403274 learned_addr learned my addr 172.21.15.5:0/3568403274 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:35:44.777 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:43.952+0000 7ff67affd700 1 -- 172.21.15.5:0/3568403274 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff67c10ac20 con 0x7ff67c109aa0 2024-05-09T00:35:44.777 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:43.952+0000 7ff67affd700 1 --2- 172.21.15.5:0/3568403274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff67c109aa0 0x7ff67c109eb0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff66c0098a0 tx=0x7ff66c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28c3b84a6fe21226 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:35:44.777 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:43.952+0000 7ff679ffb700 1 -- 172.21.15.5:0/3568403274 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff66c004160 con 0x7ff67c109aa0 2024-05-09T00:35:44.777 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:43.952+0000 7ff679ffb700 1 -- 172.21.15.5:0/3568403274 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff66c004300 con 0x7ff67c109aa0 2024-05-09T00:35:44.777 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:43.952+0000 7ff679ffb700 1 -- 172.21.15.5:0/3568403274 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff66c011af0 con 0x7ff67c109aa0 2024-05-09T00:35:44.777 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:43.953+0000 7ff681532700 1 -- 172.21.15.5:0/3568403274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff67c109aa0 msgr2=0x7ff67c109eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:35:44.778 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:43.953+0000 7ff681532700 1 --2- 172.21.15.5:0/3568403274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff67c109aa0 0x7ff67c109eb0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff66c0098a0 tx=0x7ff66c009bb0 comp rx=0 tx=0).stop 2024-05-09T00:35:44.778 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:43.953+0000 7ff681532700 1 -- 172.21.15.5:0/3568403274 shutdown_connections 2024-05-09T00:35:44.778 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:43.953+0000 7ff681532700 1 --2- 172.21.15.5:0/3568403274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff67c109aa0 0x7ff67c109eb0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:35:44.778 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:43.953+0000 7ff681532700 1 -- 172.21.15.5:0/3568403274 >> 172.21.15.5:0/3568403274 conn(0x7ff67c105100 msgr2=0x7ff67c107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:35:44.778 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:43.953+0000 7ff681532700 1 -- 172.21.15.5:0/3568403274 shutdown_connections 2024-05-09T00:35:44.778 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:43.953+0000 7ff681532700 1 -- 172.21.15.5:0/3568403274 wait complete. 2024-05-09T00:35:44.778 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:43.954+0000 7ff681532700 1 Processor -- start 2024-05-09T00:35:44.778 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:43.954+0000 7ff681532700 1 -- start start 2024-05-09T00:35:44.778 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:43.954+0000 7ff681532700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff67c109aa0 0x7ff67c07f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:35:44.778 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:43.954+0000 7ff681532700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff67c10dfd0 con 0x7ff67c109aa0 2024-05-09T00:35:44.778 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:43.954+0000 7ff67affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff67c109aa0 0x7ff67c07f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:35:44.778 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:43.954+0000 7ff67affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff67c109aa0 0x7ff67c07f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47152/0 (socket says 172.21.15.5:47152) 2024-05-09T00:35:44.778 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:43.954+0000 7ff67affd700 1 -- 172.21.15.5:0/4282312017 learned_addr learned my addr 172.21.15.5:0/4282312017 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:35:44.778 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:43.955+0000 7ff67affd700 1 -- 172.21.15.5:0/4282312017 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff67c07f830 con 0x7ff67c109aa0 2024-05-09T00:35:44.779 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:43.955+0000 7ff67affd700 1 --2- 172.21.15.5:0/4282312017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff67c109aa0 0x7ff67c07f2f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff66c011670 tx=0x7ff66c004280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:35:44.779 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:43.955+0000 7ff663fff700 1 -- 172.21.15.5:0/4282312017 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff66c0036a0 con 0x7ff67c109aa0 2024-05-09T00:35:44.779 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:43.955+0000 7ff663fff700 1 -- 172.21.15.5:0/4282312017 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff66c003cd0 con 0x7ff67c109aa0 2024-05-09T00:35:44.779 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:43.955+0000 7ff681532700 1 -- 172.21.15.5:0/4282312017 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff67c07da80 con 0x7ff67c109aa0 2024-05-09T00:35:44.779 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:43.955+0000 7ff663fff700 1 -- 172.21.15.5:0/4282312017 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff66c0193c0 con 0x7ff67c109aa0 2024-05-09T00:35:44.779 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:43.955+0000 7ff681532700 1 -- 172.21.15.5:0/4282312017 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff67c07df60 con 0x7ff67c109aa0 2024-05-09T00:35:44.779 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:43.956+0000 7ff663fff700 1 -- 172.21.15.5:0/4282312017 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7ff66c019680 con 0x7ff67c109aa0 2024-05-09T00:35:44.779 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:43.956+0000 7ff663fff700 1 --2- 172.21.15.5:0/4282312017 >> [v2:172.21.15.5:6800/752513054,v1:172.21.15.5:6801/752513054] conn(0x7ff66403e420 0x7ff664040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:35:44.779 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:43.956+0000 7ff663fff700 1 -- 172.21.15.5:0/4282312017 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff66c0546a0 con 0x7ff67c109aa0 2024-05-09T00:35:44.779 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:43.956+0000 7ff681532700 1 -- 172.21.15.5:0/4282312017 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff67c064e20 con 0x7ff67c109aa0 2024-05-09T00:35:44.779 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:43.956+0000 7ff67a7fc700 1 --2- 172.21.15.5:0/4282312017 >> [v2:172.21.15.5:6800/752513054,v1:172.21.15.5:6801/752513054] conn(0x7ff66403e420 0x7ff664040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:35:44.779 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:43.957+0000 7ff67a7fc700 1 --2- 172.21.15.5:0/4282312017 >> [v2:172.21.15.5:6800/752513054,v1:172.21.15.5:6801/752513054] conn(0x7ff66403e420 0x7ff664040930 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff670006fd0 tx=0x7ff670006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:35:44.779 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:43.962+0000 7ff663fff700 1 -- 172.21.15.5:0/4282312017 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff66c017e90 con 0x7ff67c109aa0 2024-05-09T00:35:44.780 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:44.179+0000 7ff681532700 1 -- 172.21.15.5:0/4282312017 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7ff67c07e720 con 0x7ff67c109aa0 2024-05-09T00:35:44.780 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:44.180+0000 7ff663fff700 1 -- 172.21.15.5:0/4282312017 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+409 (secure 0 0 0) 0x7ff66c023390 con 0x7ff67c109aa0 2024-05-09T00:35:44.780 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:44.182+0000 7ff681532700 1 -- 172.21.15.5:0/4282312017 >> [v2:172.21.15.5:6800/752513054,v1:172.21.15.5:6801/752513054] conn(0x7ff66403e420 msgr2=0x7ff664040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:35:44.780 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:44.182+0000 7ff681532700 1 --2- 172.21.15.5:0/4282312017 >> [v2:172.21.15.5:6800/752513054,v1:172.21.15.5:6801/752513054] conn(0x7ff66403e420 0x7ff664040930 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff670006fd0 tx=0x7ff670006e40 comp rx=0 tx=0).stop 2024-05-09T00:35:44.780 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:44.182+0000 7ff681532700 1 -- 172.21.15.5:0/4282312017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff67c109aa0 msgr2=0x7ff67c07f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:35:44.780 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:44.182+0000 7ff681532700 1 --2- 172.21.15.5:0/4282312017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff67c109aa0 0x7ff67c07f2f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff66c011670 tx=0x7ff66c004280 comp rx=0 tx=0).stop 2024-05-09T00:35:44.780 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:44.182+0000 7ff681532700 1 -- 172.21.15.5:0/4282312017 shutdown_connections 2024-05-09T00:35:44.780 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:44.182+0000 7ff681532700 1 --2- 172.21.15.5:0/4282312017 >> [v2:172.21.15.5:6800/752513054,v1:172.21.15.5:6801/752513054] conn(0x7ff66403e420 0x7ff664040930 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:35:44.780 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:44.182+0000 7ff681532700 1 --2- 172.21.15.5:0/4282312017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff67c109aa0 0x7ff67c07f2f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:35:44.780 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:44.182+0000 7ff681532700 1 -- 172.21.15.5:0/4282312017 >> 172.21.15.5:0/4282312017 conn(0x7ff67c105100 msgr2=0x7ff67c105a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:35:44.780 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:44.183+0000 7ff681532700 1 -- 172.21.15.5:0/4282312017 shutdown_connections 2024-05-09T00:35:44.780 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:44.183+0000 7ff681532700 1 -- 172.21.15.5:0/4282312017 wait complete. 2024-05-09T00:35:44.780 INFO:teuthology.orchestra.run.smithi005.stdout:Enabling cephadm module... 2024-05-09T00:35:45.787 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:45 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/973728555' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-09T00:35:45.788 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:45 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: ignoring --setuser ceph since I am not root 2024-05-09T00:35:45.788 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:45 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: ignoring --setgroup ceph since I am not root 2024-05-09T00:35:45.788 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:45 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:35:45.786+0000 7fb2a3f4c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T00:35:46.041 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:45 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:35:45.984+0000 7fb2a3f4c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T00:35:46.206 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:45.234+0000 7f2883ec1700 1 Processor -- start 2024-05-09T00:35:46.206 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:45.234+0000 7f2883ec1700 1 -- start start 2024-05-09T00:35:46.206 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:45.234+0000 7f2883ec1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f287c07f640 0x7f287c07dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:35:46.206 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:45.234+0000 7f2883ec1700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f287c07e000 con 0x7f287c07f640 2024-05-09T00:35:46.206 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:45.235+0000 7f2881c5d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f287c07f640 0x7f287c07dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:35:46.206 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:45.235+0000 7f2881c5d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f287c07f640 0x7f287c07dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47158/0 (socket says 172.21.15.5:47158) 2024-05-09T00:35:46.206 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:45.235+0000 7f2881c5d700 1 -- 172.21.15.5:0/1271701830 learned_addr learned my addr 172.21.15.5:0/1271701830 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:35:46.207 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:45.235+0000 7f2881c5d700 1 -- 172.21.15.5:0/1271701830 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f287c07e180 con 0x7f287c07f640 2024-05-09T00:35:46.207 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:45.235+0000 7f2881c5d700 1 --2- 172.21.15.5:0/1271701830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f287c07f640 0x7f287c07dac0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f286c009e80 tx=0x7f286c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c6b16544363bb8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:35:46.207 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:45.235+0000 7f2880c5b700 1 -- 172.21.15.5:0/1271701830 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f286c017070 con 0x7f287c07f640 2024-05-09T00:35:46.207 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:45.235+0000 7f2880c5b700 1 -- 172.21.15.5:0/1271701830 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f286c00b8c0 con 0x7f287c07f640 2024-05-09T00:35:46.207 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:45.235+0000 7f2880c5b700 1 -- 172.21.15.5:0/1271701830 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f286c004490 con 0x7f287c07f640 2024-05-09T00:35:46.207 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:45.236+0000 7f2883ec1700 1 -- 172.21.15.5:0/1271701830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f287c07f640 msgr2=0x7f287c07dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:35:46.207 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:45.236+0000 7f2883ec1700 1 --2- 172.21.15.5:0/1271701830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f287c07f640 0x7f287c07dac0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f286c009e80 tx=0x7f286c00b330 comp rx=0 tx=0).stop 2024-05-09T00:35:46.207 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:45.236+0000 7f2883ec1700 1 -- 172.21.15.5:0/1271701830 shutdown_connections 2024-05-09T00:35:46.207 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:45.236+0000 7f2883ec1700 1 --2- 172.21.15.5:0/1271701830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f287c07f640 0x7f287c07dac0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:35:46.207 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:45.236+0000 7f2883ec1700 1 -- 172.21.15.5:0/1271701830 >> 172.21.15.5:0/1271701830 conn(0x7f287c1050d0 msgr2=0x7f287c107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:35:46.208 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:45.236+0000 7f2883ec1700 1 -- 172.21.15.5:0/1271701830 shutdown_connections 2024-05-09T00:35:46.208 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:45.236+0000 7f2883ec1700 1 -- 172.21.15.5:0/1271701830 wait complete. 2024-05-09T00:35:46.208 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:45.237+0000 7f2883ec1700 1 Processor -- start 2024-05-09T00:35:46.208 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:45.237+0000 7f2883ec1700 1 -- start start 2024-05-09T00:35:46.208 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:45.237+0000 7f2883ec1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f287c07f640 0x7f287c1a0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:35:46.208 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:45.237+0000 7f2883ec1700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f287c110050 con 0x7f287c07f640 2024-05-09T00:35:46.208 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:45.237+0000 7f2881c5d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f287c07f640 0x7f287c1a0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:35:46.208 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:45.237+0000 7f2881c5d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f287c07f640 0x7f287c1a0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47172/0 (socket says 172.21.15.5:47172) 2024-05-09T00:35:46.208 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:45.237+0000 7f2881c5d700 1 -- 172.21.15.5:0/973728555 learned_addr learned my addr 172.21.15.5:0/973728555 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:35:46.208 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:45.238+0000 7f2881c5d700 1 -- 172.21.15.5:0/973728555 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f287c1a05b0 con 0x7f287c07f640 2024-05-09T00:35:46.208 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:45.238+0000 7f2881c5d700 1 --2- 172.21.15.5:0/973728555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f287c07f640 0x7f287c1a0070 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f286c00b7b0 tx=0x7f286c00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:35:46.208 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:45.238+0000 7f287affd700 1 -- 172.21.15.5:0/973728555 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f286c020020 con 0x7f287c07f640 2024-05-09T00:35:46.208 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:45.238+0000 7f287affd700 1 -- 172.21.15.5:0/973728555 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f286c012430 con 0x7f287c07f640 2024-05-09T00:35:46.208 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:45.238+0000 7f287affd700 1 -- 172.21.15.5:0/973728555 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f286c017040 con 0x7f287c07f640 2024-05-09T00:35:46.209 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:45.238+0000 7f2883ec1700 1 -- 172.21.15.5:0/973728555 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f287c1a0840 con 0x7f287c07f640 2024-05-09T00:35:46.209 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:45.238+0000 7f2883ec1700 1 -- 172.21.15.5:0/973728555 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f287c1a3450 con 0x7f287c07f640 2024-05-09T00:35:46.209 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:45.239+0000 7f287affd700 1 -- 172.21.15.5:0/973728555 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7f286c023020 con 0x7f287c07f640 2024-05-09T00:35:46.209 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:45.239+0000 7f287affd700 1 --2- 172.21.15.5:0/973728555 >> [v2:172.21.15.5:6800/752513054,v1:172.21.15.5:6801/752513054] conn(0x7f286403e420 0x7f2864040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:35:46.209 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:45.239+0000 7f2883ec1700 1 -- 172.21.15.5:0/973728555 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f287c064e20 con 0x7f287c07f640 2024-05-09T00:35:46.209 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:45.239+0000 7f287affd700 1 -- 172.21.15.5:0/973728555 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f286c00f050 con 0x7f287c07f640 2024-05-09T00:35:46.209 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:45.239+0000 7f288145c700 1 --2- 172.21.15.5:0/973728555 >> [v2:172.21.15.5:6800/752513054,v1:172.21.15.5:6801/752513054] conn(0x7f286403e420 0x7f2864040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:35:46.209 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:45.240+0000 7f288145c700 1 --2- 172.21.15.5:0/973728555 >> [v2:172.21.15.5:6800/752513054,v1:172.21.15.5:6801/752513054] conn(0x7f286403e420 0x7f2864040930 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2870006fd0 tx=0x7f2870006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:35:46.209 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:45.245+0000 7f287affd700 1 -- 172.21.15.5:0/973728555 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f286c02a020 con 0x7f287c07f640 2024-05-09T00:35:46.209 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:45.521+0000 7f2883ec1700 1 -- 172.21.15.5:0/973728555 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f287c1a3620 con 0x7f287c07f640 2024-05-09T00:35:46.209 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:45.572+0000 7f287affd700 1 -- 172.21.15.5:0/973728555 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f287c1a3620 con 0x7f287c07f640 2024-05-09T00:35:46.209 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:45.574+0000 7f287affd700 1 -- 172.21.15.5:0/973728555 <== mon.0 v2:172.21.15.5:3300/0 8 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f286c0125e0 con 0x7f287c07f640 2024-05-09T00:35:46.209 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:45.575+0000 7f2883ec1700 1 -- 172.21.15.5:0/973728555 >> [v2:172.21.15.5:6800/752513054,v1:172.21.15.5:6801/752513054] conn(0x7f286403e420 msgr2=0x7f2864040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:35:46.210 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:45.575+0000 7f2883ec1700 1 --2- 172.21.15.5:0/973728555 >> [v2:172.21.15.5:6800/752513054,v1:172.21.15.5:6801/752513054] conn(0x7f286403e420 0x7f2864040930 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2870006fd0 tx=0x7f2870006e40 comp rx=0 tx=0).stop 2024-05-09T00:35:46.210 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:45.575+0000 7f2883ec1700 1 -- 172.21.15.5:0/973728555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f287c07f640 msgr2=0x7f287c1a0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:35:46.210 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:45.575+0000 7f2883ec1700 1 --2- 172.21.15.5:0/973728555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f287c07f640 0x7f287c1a0070 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f286c00b7b0 tx=0x7f286c00bd50 comp rx=0 tx=0).stop 2024-05-09T00:35:46.210 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:45.575+0000 7f2883ec1700 1 -- 172.21.15.5:0/973728555 shutdown_connections 2024-05-09T00:35:46.210 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:45.576+0000 7f2883ec1700 1 --2- 172.21.15.5:0/973728555 >> [v2:172.21.15.5:6800/752513054,v1:172.21.15.5:6801/752513054] conn(0x7f286403e420 0x7f2864040930 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:35:46.210 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:45.576+0000 7f2883ec1700 1 --2- 172.21.15.5:0/973728555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f287c07f640 0x7f287c1a0070 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:35:46.210 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:45.576+0000 7f2883ec1700 1 -- 172.21.15.5:0/973728555 >> 172.21.15.5:0/973728555 conn(0x7f287c1050d0 msgr2=0x7f287c105aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:35:46.210 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:45.576+0000 7f2883ec1700 1 -- 172.21.15.5:0/973728555 shutdown_connections 2024-05-09T00:35:46.210 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:45.576+0000 7f2883ec1700 1 -- 172.21.15.5:0/973728555 wait complete. 2024-05-09T00:35:46.415 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:46 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:35:46.097+0000 7fb2a3f4c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T00:35:46.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:46 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/973728555' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-09T00:35:46.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:46 smithi005 ceph-mon[30365]: mgrmap e5: x(active, since 5s) 2024-05-09T00:35:46.791 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:46 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:35:46.432+0000 7fb2a3f4c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T00:35:46.791 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:46 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:35:46.524+0000 7fb2a3f4c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T00:35:47.554 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2024-05-09T00:35:47.554 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-05-09T00:35:47.554 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-09T00:35:47.554 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-09T00:35:47.554 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-09T00:35:47.554 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2024-05-09T00:35:47.554 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:46.638+0000 7fbae3fa3700 1 Processor -- start 2024-05-09T00:35:47.555 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:46.638+0000 7fbae3fa3700 1 -- start start 2024-05-09T00:35:47.555 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:46.638+0000 7fbae3fa3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbadc10a590 0x7fbadc10c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:35:47.555 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:46.638+0000 7fbae3fa3700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbadc07ce30 con 0x7fbadc10a590 2024-05-09T00:35:47.555 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:46.638+0000 7fbae1d3f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbadc10a590 0x7fbadc10c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:35:47.555 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:46.638+0000 7fbae1d3f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbadc10a590 0x7fbadc10c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47196/0 (socket says 172.21.15.5:47196) 2024-05-09T00:35:47.555 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:46.638+0000 7fbae1d3f700 1 -- 172.21.15.5:0/1514639584 learned_addr learned my addr 172.21.15.5:0/1514639584 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:35:47.555 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:46.639+0000 7fbae1d3f700 1 -- 172.21.15.5:0/1514639584 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbadc10cef0 con 0x7fbadc10a590 2024-05-09T00:35:47.555 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:46.639+0000 7fbae1d3f700 1 --2- 172.21.15.5:0/1514639584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbadc10a590 0x7fbadc10c9b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbad80098a0 tx=0x7fbad8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70fd94d86c798c3a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:35:47.555 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:46.639+0000 7fbae0d3d700 1 -- 172.21.15.5:0/1514639584 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbad8004160 con 0x7fbadc10a590 2024-05-09T00:35:47.555 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:46.639+0000 7fbae0d3d700 1 -- 172.21.15.5:0/1514639584 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fbad8004300 con 0x7fbadc10a590 2024-05-09T00:35:47.555 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:46.639+0000 7fbae0d3d700 1 -- 172.21.15.5:0/1514639584 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbad8011af0 con 0x7fbadc10a590 2024-05-09T00:35:47.555 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:46.640+0000 7fbae3fa3700 1 -- 172.21.15.5:0/1514639584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbadc10a590 msgr2=0x7fbadc10c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:35:47.555 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:46.640+0000 7fbae3fa3700 1 --2- 172.21.15.5:0/1514639584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbadc10a590 0x7fbadc10c9b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbad80098a0 tx=0x7fbad8009bb0 comp rx=0 tx=0).stop 2024-05-09T00:35:47.556 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:46.640+0000 7fbae3fa3700 1 -- 172.21.15.5:0/1514639584 shutdown_connections 2024-05-09T00:35:47.556 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:46.640+0000 7fbae3fa3700 1 --2- 172.21.15.5:0/1514639584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbadc10a590 0x7fbadc10c9b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:35:47.556 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:46.640+0000 7fbae3fa3700 1 -- 172.21.15.5:0/1514639584 >> 172.21.15.5:0/1514639584 conn(0x7fbadc104050 msgr2=0x7fbadc1064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:35:47.556 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:46.640+0000 7fbae3fa3700 1 -- 172.21.15.5:0/1514639584 shutdown_connections 2024-05-09T00:35:47.556 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:46.640+0000 7fbae3fa3700 1 -- 172.21.15.5:0/1514639584 wait complete. 2024-05-09T00:35:47.556 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:46.641+0000 7fbae3fa3700 1 Processor -- start 2024-05-09T00:35:47.556 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:46.641+0000 7fbae3fa3700 1 -- start start 2024-05-09T00:35:47.556 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:46.641+0000 7fbae3fa3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbadc10a590 0x7fbadc109d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:35:47.556 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:46.641+0000 7fbae3fa3700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbadc10de50 con 0x7fbadc10a590 2024-05-09T00:35:47.556 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:46.641+0000 7fbae1d3f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbadc10a590 0x7fbadc109d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:35:47.556 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:46.641+0000 7fbae1d3f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbadc10a590 0x7fbadc109d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47204/0 (socket says 172.21.15.5:47204) 2024-05-09T00:35:47.556 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:46.641+0000 7fbae1d3f700 1 -- 172.21.15.5:0/3411135697 learned_addr learned my addr 172.21.15.5:0/3411135697 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:35:47.556 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:46.641+0000 7fbae1d3f700 1 -- 172.21.15.5:0/3411135697 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbadc10a250 con 0x7fbadc10a590 2024-05-09T00:35:47.556 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:46.641+0000 7fbae1d3f700 1 --2- 172.21.15.5:0/3411135697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbadc10a590 0x7fbadc109d10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbad80117b0 tx=0x7fbad8011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:35:47.557 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:46.642+0000 7fbad2ffd700 1 -- 172.21.15.5:0/3411135697 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbad80039c0 con 0x7fbadc10a590 2024-05-09T00:35:47.557 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:46.642+0000 7fbad2ffd700 1 -- 172.21.15.5:0/3411135697 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fbad8003b60 con 0x7fbadc10a590 2024-05-09T00:35:47.557 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:46.642+0000 7fbae3fa3700 1 -- 172.21.15.5:0/3411135697 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbadc1084a0 con 0x7fbadc10a590 2024-05-09T00:35:47.557 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:46.642+0000 7fbae3fa3700 1 -- 172.21.15.5:0/3411135697 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbadc108980 con 0x7fbadc10a590 2024-05-09T00:35:47.557 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:46.642+0000 7fbad2ffd700 1 -- 172.21.15.5:0/3411135697 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbad80196a0 con 0x7fbadc10a590 2024-05-09T00:35:47.557 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:46.643+0000 7fbad2ffd700 1 -- 172.21.15.5:0/3411135697 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7fbad8019910 con 0x7fbadc10a590 2024-05-09T00:35:47.557 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:46.643+0000 7fbae3fa3700 1 -- 172.21.15.5:0/3411135697 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbac0005360 con 0x7fbadc10a590 2024-05-09T00:35:47.557 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:46.643+0000 7fbad2ffd700 1 --2- 172.21.15.5:0/3411135697 >> [v2:172.21.15.5:6800/752513054,v1:172.21.15.5:6801/752513054] conn(0x7fbac803e4c0 0x7fbac80409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:35:47.557 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:46.643+0000 7fbad2ffd700 1 -- 172.21.15.5:0/3411135697 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fbad8053af0 con 0x7fbadc10a590 2024-05-09T00:35:47.557 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:46.643+0000 7fbae153e700 1 -- 172.21.15.5:0/3411135697 >> [v2:172.21.15.5:6800/752513054,v1:172.21.15.5:6801/752513054] conn(0x7fbac803e4c0 msgr2=0x7fbac80409d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/752513054 2024-05-09T00:35:47.557 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:46.643+0000 7fbae153e700 1 --2- 172.21.15.5:0/3411135697 >> [v2:172.21.15.5:6800/752513054,v1:172.21.15.5:6801/752513054] conn(0x7fbac803e4c0 0x7fbac80409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T00:35:47.557 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:46.648+0000 7fbad2ffd700 1 -- 172.21.15.5:0/3411135697 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbad80042a0 con 0x7fbadc10a590 2024-05-09T00:35:47.557 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:46.844+0000 7fbae153e700 1 -- 172.21.15.5:0/3411135697 >> [v2:172.21.15.5:6800/752513054,v1:172.21.15.5:6801/752513054] conn(0x7fbac803e4c0 msgr2=0x7fbac80409d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/752513054 2024-05-09T00:35:47.557 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:46.844+0000 7fbae153e700 1 --2- 172.21.15.5:0/3411135697 >> [v2:172.21.15.5:6800/752513054,v1:172.21.15.5:6801/752513054] conn(0x7fbac803e4c0 0x7fbac80409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T00:35:47.558 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:46.933+0000 7fbae3fa3700 1 -- 172.21.15.5:0/3411135697 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fbac0005e60 con 0x7fbadc10a590 2024-05-09T00:35:47.558 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:46.935+0000 7fbad2ffd700 1 -- 172.21.15.5:0/3411135697 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fbad8023390 con 0x7fbadc10a590 2024-05-09T00:35:47.558 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:46.938+0000 7fbae3fa3700 1 -- 172.21.15.5:0/3411135697 >> [v2:172.21.15.5:6800/752513054,v1:172.21.15.5:6801/752513054] conn(0x7fbac803e4c0 msgr2=0x7fbac80409d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T00:35:47.558 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:46.939+0000 7fbae3fa3700 1 --2- 172.21.15.5:0/3411135697 >> [v2:172.21.15.5:6800/752513054,v1:172.21.15.5:6801/752513054] conn(0x7fbac803e4c0 0x7fbac80409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:35:47.558 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:46.939+0000 7fbae3fa3700 1 -- 172.21.15.5:0/3411135697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbadc10a590 msgr2=0x7fbadc109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:35:47.558 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:46.939+0000 7fbae3fa3700 1 --2- 172.21.15.5:0/3411135697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbadc10a590 0x7fbadc109d10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbad80117b0 tx=0x7fbad8011b70 comp rx=0 tx=0).stop 2024-05-09T00:35:47.558 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:46.939+0000 7fbae3fa3700 1 -- 172.21.15.5:0/3411135697 shutdown_connections 2024-05-09T00:35:47.558 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:46.939+0000 7fbae3fa3700 1 --2- 172.21.15.5:0/3411135697 >> [v2:172.21.15.5:6800/752513054,v1:172.21.15.5:6801/752513054] conn(0x7fbac803e4c0 0x7fbac80409d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:35:47.558 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:46.939+0000 7fbae3fa3700 1 --2- 172.21.15.5:0/3411135697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbadc10a590 0x7fbadc109d10 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:35:47.558 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:46.939+0000 7fbae3fa3700 1 -- 172.21.15.5:0/3411135697 >> 172.21.15.5:0/3411135697 conn(0x7fbadc104050 msgr2=0x7fbadc105a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:35:47.558 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:46.939+0000 7fbae3fa3700 1 -- 172.21.15.5:0/3411135697 shutdown_connections 2024-05-09T00:35:47.558 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:46.939+0000 7fbae3fa3700 1 -- 172.21.15.5:0/3411135697 wait complete. 2024-05-09T00:35:47.558 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for the mgr to restart... 2024-05-09T00:35:47.558 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for mgr epoch 5... 2024-05-09T00:35:47.822 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:47 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/3411135697' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-09T00:35:48.190 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:47 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:35:47.820+0000 7fb2a3f4c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T00:35:48.190 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:48 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:35:48.008+0000 7fb2a3f4c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T00:35:48.190 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:48 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:35:48.188+0000 7fb2a3f4c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T00:35:48.791 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:48 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:35:48.497+0000 7fb2a3f4c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T00:35:48.791 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:48 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:35:48.596+0000 7fb2a3f4c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T00:35:49.294 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:48 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:35:48.809+0000 7fb2a3f4c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T00:35:49.294 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:49 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:35:49.016+0000 7fb2a3f4c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T00:35:49.791 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:49 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:35:49.655+0000 7fb2a3f4c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T00:35:50.291 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:49 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:35:49.889+0000 7fb2a3f4c200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-09T00:35:50.791 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:50 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:35:50.340+0000 7fb2a3f4c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T00:35:50.791 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:50 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:35:50.443+0000 7fb2a3f4c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T00:35:51.214 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:50 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:35:50.832+0000 7fb2a3f4c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T00:35:51.214 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:51 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:35:51.212+0000 7fb2a3f4c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T00:35:51.541 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:51 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:35:51.312+0000 7fb2a3f4c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T00:35:52.041 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:51 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:35:51.571+0000 7fb2a3f4c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T00:35:52.041 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:51 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:35:51.714+0000 7fb2a3f4c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T00:35:52.041 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:51 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:35:51.816+0000 7fb2a3f4c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T00:35:52.541 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:52 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:35:52.157+0000 7fb2a3f4c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T00:35:52.541 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:52 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:35:52.256+0000 7fb2a3f4c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T00:35:53.181 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:52 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:35:52.896+0000 7fb2a3f4c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T00:35:53.181 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:53 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:35:53.126+0000 7fb2a3f4c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T00:35:53.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:53 smithi005 ceph-mon[30365]: Active manager daemon x restarted 2024-05-09T00:35:53.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:53 smithi005 ceph-mon[30365]: Activating manager daemon x 2024-05-09T00:35:53.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:53 smithi005 ceph-mon[30365]: osdmap e2: 0 total, 0 up, 0 in 2024-05-09T00:35:53.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:53 smithi005 ceph-mon[30365]: mgrmap e6: x(active, starting, since 0.00394231s) 2024-05-09T00:35:53.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:53 smithi005 ceph-mon[30365]: from='mgr.14120 172.21.15.5:0/1775107462' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T00:35:53.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:53 smithi005 ceph-mon[30365]: from='mgr.14120 172.21.15.5:0/1775107462' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-09T00:35:53.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:53 smithi005 ceph-mon[30365]: from='mgr.14120 172.21.15.5:0/1775107462' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-09T00:35:53.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:53 smithi005 ceph-mon[30365]: from='mgr.14120 172.21.15.5:0/1775107462' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-09T00:35:53.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:53 smithi005 ceph-mon[30365]: from='mgr.14120 172.21.15.5:0/1775107462' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-09T00:35:53.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:53 smithi005 ceph-mon[30365]: Manager daemon x is now available 2024-05-09T00:35:53.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:53 smithi005 ceph-mon[30365]: from='mgr.14120 172.21.15.5:0/1775107462' entity='mgr.x' 2024-05-09T00:35:53.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:53 smithi005 ceph-mon[30365]: from='mgr.14120 172.21.15.5:0/1775107462' entity='mgr.x' 2024-05-09T00:35:53.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:53 smithi005 ceph-mon[30365]: from='mgr.14120 172.21.15.5:0/1775107462' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:35:53.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:53 smithi005 ceph-mon[30365]: from='mgr.14120 172.21.15.5:0/1775107462' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:35:54.460 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:54 smithi005 ceph-mon[30365]: Found migration_current of "None". Setting to last migration. 2024-05-09T00:35:54.460 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:54 smithi005 ceph-mon[30365]: from='mgr.14120 172.21.15.5:0/1775107462' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-09T00:35:54.460 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:54 smithi005 ceph-mon[30365]: from='mgr.14120 172.21.15.5:0/1775107462' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-09T00:35:54.460 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:54 smithi005 ceph-mon[30365]: from='mgr.14120 172.21.15.5:0/1775107462' entity='mgr.x' 2024-05-09T00:35:54.460 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:54 smithi005 ceph-mon[30365]: from='mgr.14120 172.21.15.5:0/1775107462' entity='mgr.x' 2024-05-09T00:35:54.461 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:54 smithi005 ceph-mon[30365]: mgrmap e7: x(active, since 1.00647s) 2024-05-09T00:35:54.713 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2024-05-09T00:35:54.713 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-05-09T00:35:54.713 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-09T00:35:54.713 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2024-05-09T00:35:54.713 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:48.037+0000 7fc8cc1ac700 1 Processor -- start 2024-05-09T00:35:54.713 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:48.037+0000 7fc8cc1ac700 1 -- start start 2024-05-09T00:35:54.713 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:48.038+0000 7fc8cc1ac700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8c410a550 0x7fc8c410c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:35:54.713 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:48.038+0000 7fc8cc1ac700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c407ce70 con 0x7fc8c410a550 2024-05-09T00:35:54.713 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:48.038+0000 7fc8c9f48700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8c410a550 0x7fc8c410c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:35:54.713 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:48.038+0000 7fc8c9f48700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8c410a550 0x7fc8c410c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47216/0 (socket says 172.21.15.5:47216) 2024-05-09T00:35:54.714 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:48.038+0000 7fc8c9f48700 1 -- 172.21.15.5:0/3785846101 learned_addr learned my addr 172.21.15.5:0/3785846101 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:35:54.714 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:48.038+0000 7fc8c9f48700 1 -- 172.21.15.5:0/3785846101 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8c410ceb0 con 0x7fc8c410a550 2024-05-09T00:35:54.714 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:48.038+0000 7fc8c9f48700 1 --2- 172.21.15.5:0/3785846101 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8c410a550 0x7fc8c410c970 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc8c00098a0 tx=0x7fc8c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f842f11684bc04cb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:35:54.714 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:48.038+0000 7fc8c8f46700 1 -- 172.21.15.5:0/3785846101 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc8c0004160 con 0x7fc8c410a550 2024-05-09T00:35:54.714 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:48.039+0000 7fc8c8f46700 1 -- 172.21.15.5:0/3785846101 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc8c0004300 con 0x7fc8c410a550 2024-05-09T00:35:54.714 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:48.039+0000 7fc8c8f46700 1 -- 172.21.15.5:0/3785846101 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc8c0011af0 con 0x7fc8c410a550 2024-05-09T00:35:54.714 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:48.039+0000 7fc8cc1ac700 1 -- 172.21.15.5:0/3785846101 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8c410a550 msgr2=0x7fc8c410c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:35:54.714 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:48.039+0000 7fc8cc1ac700 1 --2- 172.21.15.5:0/3785846101 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8c410a550 0x7fc8c410c970 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc8c00098a0 tx=0x7fc8c0009bb0 comp rx=0 tx=0).stop 2024-05-09T00:35:54.714 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:48.039+0000 7fc8cc1ac700 1 -- 172.21.15.5:0/3785846101 shutdown_connections 2024-05-09T00:35:54.714 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:48.039+0000 7fc8cc1ac700 1 --2- 172.21.15.5:0/3785846101 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8c410a550 0x7fc8c410c970 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:35:54.714 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:48.039+0000 7fc8cc1ac700 1 -- 172.21.15.5:0/3785846101 >> 172.21.15.5:0/3785846101 conn(0x7fc8c4104050 msgr2=0x7fc8c41064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:35:54.714 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:48.040+0000 7fc8cc1ac700 1 -- 172.21.15.5:0/3785846101 shutdown_connections 2024-05-09T00:35:54.714 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:48.040+0000 7fc8cc1ac700 1 -- 172.21.15.5:0/3785846101 wait complete. 2024-05-09T00:35:54.715 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:48.040+0000 7fc8cc1ac700 1 Processor -- start 2024-05-09T00:35:54.715 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:48.040+0000 7fc8cc1ac700 1 -- start start 2024-05-09T00:35:54.715 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:48.041+0000 7fc8cc1ac700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8c410a550 0x7fc8c419dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:35:54.715 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:48.041+0000 7fc8cc1ac700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c410de70 con 0x7fc8c410a550 2024-05-09T00:35:54.715 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:48.041+0000 7fc8c9f48700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8c410a550 0x7fc8c419dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:35:54.715 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:48.041+0000 7fc8c9f48700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8c410a550 0x7fc8c419dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47220/0 (socket says 172.21.15.5:47220) 2024-05-09T00:35:54.715 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:48.041+0000 7fc8c9f48700 1 -- 172.21.15.5:0/2089772992 learned_addr learned my addr 172.21.15.5:0/2089772992 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:35:54.715 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:48.041+0000 7fc8c9f48700 1 -- 172.21.15.5:0/2089772992 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8c419e3e0 con 0x7fc8c410a550 2024-05-09T00:35:54.715 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:48.041+0000 7fc8c9f48700 1 --2- 172.21.15.5:0/2089772992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8c410a550 0x7fc8c419dea0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc8c00117b0 tx=0x7fc8c0011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:35:54.715 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:48.041+0000 7fc8baffd700 1 -- 172.21.15.5:0/2089772992 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc8c00039c0 con 0x7fc8c410a550 2024-05-09T00:35:54.715 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:48.042+0000 7fc8cc1ac700 1 -- 172.21.15.5:0/2089772992 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8c419e670 con 0x7fc8c410a550 2024-05-09T00:35:54.715 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:48.042+0000 7fc8baffd700 1 -- 172.21.15.5:0/2089772992 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc8c0003b60 con 0x7fc8c410a550 2024-05-09T00:35:54.715 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:48.042+0000 7fc8baffd700 1 -- 172.21.15.5:0/2089772992 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc8c00196a0 con 0x7fc8c410a550 2024-05-09T00:35:54.716 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:48.042+0000 7fc8cc1ac700 1 -- 172.21.15.5:0/2089772992 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8c41a1280 con 0x7fc8c410a550 2024-05-09T00:35:54.716 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:48.043+0000 7fc8baffd700 1 -- 172.21.15.5:0/2089772992 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7fc8c0019960 con 0x7fc8c410a550 2024-05-09T00:35:54.716 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:48.043+0000 7fc8baffd700 1 --2- 172.21.15.5:0/2089772992 >> [v2:172.21.15.5:6800/752513054,v1:172.21.15.5:6801/752513054] conn(0x7fc8b003e470 0x7fc8b0040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:35:54.716 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:48.043+0000 7fc8baffd700 1 -- 172.21.15.5:0/2089772992 --> [v2:172.21.15.5:6800/752513054,v1:172.21.15.5:6801/752513054] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc8b0041090 con 0x7fc8b003e470 2024-05-09T00:35:54.716 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:48.043+0000 7fc8baffd700 1 -- 172.21.15.5:0/2089772992 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc8c0053b30 con 0x7fc8c410a550 2024-05-09T00:35:54.716 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:48.043+0000 7fc8c9747700 1 -- 172.21.15.5:0/2089772992 >> [v2:172.21.15.5:6800/752513054,v1:172.21.15.5:6801/752513054] conn(0x7fc8b003e470 msgr2=0x7fc8b0040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/752513054 2024-05-09T00:35:54.716 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:48.043+0000 7fc8c9747700 1 --2- 172.21.15.5:0/2089772992 >> [v2:172.21.15.5:6800/752513054,v1:172.21.15.5:6801/752513054] conn(0x7fc8b003e470 0x7fc8b0040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T00:35:54.716 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:48.244+0000 7fc8c9747700 1 -- 172.21.15.5:0/2089772992 >> [v2:172.21.15.5:6800/752513054,v1:172.21.15.5:6801/752513054] conn(0x7fc8b003e470 msgr2=0x7fc8b0040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/752513054 2024-05-09T00:35:54.716 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:48.244+0000 7fc8c9747700 1 --2- 172.21.15.5:0/2089772992 >> [v2:172.21.15.5:6800/752513054,v1:172.21.15.5:6801/752513054] conn(0x7fc8b003e470 0x7fc8b0040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T00:35:54.716 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:48.645+0000 7fc8c9747700 1 -- 172.21.15.5:0/2089772992 >> [v2:172.21.15.5:6800/752513054,v1:172.21.15.5:6801/752513054] conn(0x7fc8b003e470 msgr2=0x7fc8b0040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/752513054 2024-05-09T00:35:54.716 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:48.645+0000 7fc8c9747700 1 --2- 172.21.15.5:0/2089772992 >> [v2:172.21.15.5:6800/752513054,v1:172.21.15.5:6801/752513054] conn(0x7fc8b003e470 0x7fc8b0040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-09T00:35:54.716 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:49.446+0000 7fc8c9747700 1 -- 172.21.15.5:0/2089772992 >> [v2:172.21.15.5:6800/752513054,v1:172.21.15.5:6801/752513054] conn(0x7fc8b003e470 msgr2=0x7fc8b0040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/752513054 2024-05-09T00:35:54.716 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:49.446+0000 7fc8c9747700 1 --2- 172.21.15.5:0/2089772992 >> [v2:172.21.15.5:6800/752513054,v1:172.21.15.5:6801/752513054] conn(0x7fc8b003e470 0x7fc8b0040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-09T00:35:54.717 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:51.047+0000 7fc8c9747700 1 -- 172.21.15.5:0/2089772992 >> [v2:172.21.15.5:6800/752513054,v1:172.21.15.5:6801/752513054] conn(0x7fc8b003e470 msgr2=0x7fc8b0040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/752513054 2024-05-09T00:35:54.717 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:51.047+0000 7fc8c9747700 1 --2- 172.21.15.5:0/2089772992 >> [v2:172.21.15.5:6800/752513054,v1:172.21.15.5:6801/752513054] conn(0x7fc8b003e470 0x7fc8b0040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-09T00:35:54.717 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:53.130+0000 7fc8baffd700 1 -- 172.21.15.5:0/2089772992 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mgrmap(e 6) ==== 49989+0+0 (secure 0 0 0) 0x7fc8c000fd40 con 0x7fc8c410a550 2024-05-09T00:35:54.717 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:53.130+0000 7fc8baffd700 1 -- 172.21.15.5:0/2089772992 >> [v2:172.21.15.5:6800/752513054,v1:172.21.15.5:6801/752513054] conn(0x7fc8b003e470 msgr2=0x7fc8b0040980 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T00:35:54.717 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:53.130+0000 7fc8baffd700 1 --2- 172.21.15.5:0/2089772992 >> [v2:172.21.15.5:6800/752513054,v1:172.21.15.5:6801/752513054] conn(0x7fc8b003e470 0x7fc8b0040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:35:54.717 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:54.132+0000 7fc8baffd700 1 -- 172.21.15.5:0/2089772992 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mgrmap(e 7) ==== 50116+0+0 (secure 0 0 0) 0x7fc8c0055b40 con 0x7fc8c410a550 2024-05-09T00:35:54.717 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:54.132+0000 7fc8baffd700 1 --2- 172.21.15.5:0/2089772992 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7fc8b0041db0 0x7fc8b00441d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:35:54.717 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:54.132+0000 7fc8baffd700 1 -- 172.21.15.5:0/2089772992 --> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc8c002be50 con 0x7fc8b0041db0 2024-05-09T00:35:54.717 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:54.132+0000 7fc8c9747700 1 --2- 172.21.15.5:0/2089772992 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7fc8b0041db0 0x7fc8b00441d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:35:54.717 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:54.133+0000 7fc8c9747700 1 --2- 172.21.15.5:0/2089772992 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7fc8b0041db0 0x7fc8b00441d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc8b4003a10 tx=0x7fc8b4009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:35:54.717 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:54.134+0000 7fc8baffd700 1 -- 172.21.15.5:0/2089772992 <== mgr.14120 v2:172.21.15.5:6800/287842266 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fc8c002be50 con 0x7fc8b0041db0 2024-05-09T00:35:54.717 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:54.142+0000 7fc8cc1ac700 1 -- 172.21.15.5:0/2089772992 --> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fc8c4064e20 con 0x7fc8b0041db0 2024-05-09T00:35:54.717 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:54.142+0000 7fc8baffd700 1 -- 172.21.15.5:0/2089772992 <== mgr.14120 v2:172.21.15.5:6800/287842266 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fc8c4064e20 con 0x7fc8b0041db0 2024-05-09T00:35:54.717 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:54.143+0000 7fc8cc1ac700 1 -- 172.21.15.5:0/2089772992 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7fc8b0041db0 msgr2=0x7fc8b00441d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:35:54.717 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:54.143+0000 7fc8cc1ac700 1 --2- 172.21.15.5:0/2089772992 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7fc8b0041db0 0x7fc8b00441d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc8b4003a10 tx=0x7fc8b4009290 comp rx=0 tx=0).stop 2024-05-09T00:35:54.718 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:54.143+0000 7fc8cc1ac700 1 -- 172.21.15.5:0/2089772992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8c410a550 msgr2=0x7fc8c419dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:35:54.718 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:54.143+0000 7fc8cc1ac700 1 --2- 172.21.15.5:0/2089772992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8c410a550 0x7fc8c419dea0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc8c00117b0 tx=0x7fc8c0011b70 comp rx=0 tx=0).stop 2024-05-09T00:35:54.718 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:54.143+0000 7fc8cc1ac700 1 -- 172.21.15.5:0/2089772992 shutdown_connections 2024-05-09T00:35:54.718 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:54.143+0000 7fc8cc1ac700 1 --2- 172.21.15.5:0/2089772992 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7fc8b0041db0 0x7fc8b00441d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:35:54.718 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:54.143+0000 7fc8cc1ac700 1 --2- 172.21.15.5:0/2089772992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8c410a550 0x7fc8c419dea0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:35:54.718 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:54.143+0000 7fc8cc1ac700 1 --2- 172.21.15.5:0/2089772992 >> [v2:172.21.15.5:6800/752513054,v1:172.21.15.5:6801/752513054] conn(0x7fc8b003e470 0x7fc8b0040980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:35:54.718 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:54.143+0000 7fc8cc1ac700 1 -- 172.21.15.5:0/2089772992 >> 172.21.15.5:0/2089772992 conn(0x7fc8c4104050 msgr2=0x7fc8c4105a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:35:54.718 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:54.143+0000 7fc8cc1ac700 1 -- 172.21.15.5:0/2089772992 shutdown_connections 2024-05-09T00:35:54.718 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:54.143+0000 7fc8cc1ac700 1 -- 172.21.15.5:0/2089772992 wait complete. 2024-05-09T00:35:54.718 INFO:teuthology.orchestra.run.smithi005.stdout:mgr epoch 5 is available 2024-05-09T00:35:54.718 INFO:teuthology.orchestra.run.smithi005.stdout:Setting orchestrator backend to cephadm... 2024-05-09T00:35:55.433 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:55 smithi005 ceph-mon[30365]: [09/May/2024:00:35:54] ENGINE Bus STARTING 2024-05-09T00:35:55.433 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:55 smithi005 ceph-mon[30365]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-09T00:35:55.433 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:55 smithi005 ceph-mon[30365]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-09T00:35:55.433 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:55 smithi005 ceph-mon[30365]: [09/May/2024:00:35:54] ENGINE Serving on http://172.21.15.5:8765 2024-05-09T00:35:55.433 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:55 smithi005 ceph-mon[30365]: [09/May/2024:00:35:54] ENGINE Serving on https://172.21.15.5:7150 2024-05-09T00:35:55.433 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:55 smithi005 ceph-mon[30365]: [09/May/2024:00:35:54] ENGINE Bus STARTED 2024-05-09T00:35:55.433 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:55 smithi005 ceph-mon[30365]: from='mgr.14120 172.21.15.5:0/1775107462' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:35:56.135 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:55.183+0000 7fb76f5bf700 1 Processor -- start 2024-05-09T00:35:56.135 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:55.184+0000 7fb76f5bf700 1 -- start start 2024-05-09T00:35:56.135 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:55.184+0000 7fb76f5bf700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7681083a0 0x7fb7681087b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:35:56.135 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:55.184+0000 7fb76f5bf700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb76807ce10 con 0x7fb7681083a0 2024-05-09T00:35:56.135 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:55.184+0000 7fb76d35b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7681083a0 0x7fb7681087b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:35:56.135 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:55.184+0000 7fb76d35b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7681083a0 0x7fb7681087b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38686/0 (socket says 172.21.15.5:38686) 2024-05-09T00:35:56.136 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:55.184+0000 7fb76d35b700 1 -- 172.21.15.5:0/4083340920 learned_addr learned my addr 172.21.15.5:0/4083340920 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:35:56.136 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:55.184+0000 7fb76d35b700 1 -- 172.21.15.5:0/4083340920 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb768108cf0 con 0x7fb7681083a0 2024-05-09T00:35:56.136 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:55.185+0000 7fb76d35b700 1 --2- 172.21.15.5:0/4083340920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7681083a0 0x7fb7681087b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb7640098a0 tx=0x7fb764009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad40894a1aec52f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:35:56.136 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:55.187+0000 7fb75ffff700 1 -- 172.21.15.5:0/4083340920 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb764004160 con 0x7fb7681083a0 2024-05-09T00:35:56.136 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:55.187+0000 7fb75ffff700 1 -- 172.21.15.5:0/4083340920 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb764004300 con 0x7fb7681083a0 2024-05-09T00:35:56.136 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:55.187+0000 7fb75ffff700 1 -- 172.21.15.5:0/4083340920 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb764011af0 con 0x7fb7681083a0 2024-05-09T00:35:56.136 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:55.188+0000 7fb76f5bf700 1 -- 172.21.15.5:0/4083340920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7681083a0 msgr2=0x7fb7681087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:35:56.136 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:55.188+0000 7fb76f5bf700 1 --2- 172.21.15.5:0/4083340920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7681083a0 0x7fb7681087b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb7640098a0 tx=0x7fb764009bb0 comp rx=0 tx=0).stop 2024-05-09T00:35:56.136 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:55.188+0000 7fb76f5bf700 1 -- 172.21.15.5:0/4083340920 shutdown_connections 2024-05-09T00:35:56.136 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:55.188+0000 7fb76f5bf700 1 --2- 172.21.15.5:0/4083340920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7681083a0 0x7fb7681087b0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:35:56.136 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:55.188+0000 7fb76f5bf700 1 -- 172.21.15.5:0/4083340920 >> 172.21.15.5:0/4083340920 conn(0x7fb768104050 msgr2=0x7fb7681064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:35:56.136 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:55.188+0000 7fb76f5bf700 1 -- 172.21.15.5:0/4083340920 shutdown_connections 2024-05-09T00:35:56.136 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:55.188+0000 7fb76f5bf700 1 -- 172.21.15.5:0/4083340920 wait complete. 2024-05-09T00:35:56.136 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:55.189+0000 7fb76f5bf700 1 Processor -- start 2024-05-09T00:35:56.137 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:55.189+0000 7fb76f5bf700 1 -- start start 2024-05-09T00:35:56.137 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:55.189+0000 7fb76f5bf700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7681083a0 0x7fb7681a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:35:56.137 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:55.189+0000 7fb76f5bf700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb768109d70 con 0x7fb7681083a0 2024-05-09T00:35:56.137 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:55.189+0000 7fb76d35b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7681083a0 0x7fb7681a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:35:56.137 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:55.189+0000 7fb76d35b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7681083a0 0x7fb7681a01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38696/0 (socket says 172.21.15.5:38696) 2024-05-09T00:35:56.137 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:55.189+0000 7fb76d35b700 1 -- 172.21.15.5:0/3148945710 learned_addr learned my addr 172.21.15.5:0/3148945710 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:35:56.137 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:55.190+0000 7fb76d35b700 1 -- 172.21.15.5:0/3148945710 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7681a0730 con 0x7fb7681083a0 2024-05-09T00:35:56.137 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:55.190+0000 7fb76d35b700 1 --2- 172.21.15.5:0/3148945710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7681083a0 0x7fb7681a01f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb7640117b0 tx=0x7fb764011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:35:56.137 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:55.190+0000 7fb75e7fc700 1 -- 172.21.15.5:0/3148945710 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb7640039c0 con 0x7fb7681083a0 2024-05-09T00:35:56.137 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:55.190+0000 7fb75e7fc700 1 -- 172.21.15.5:0/3148945710 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb764003b60 con 0x7fb7681083a0 2024-05-09T00:35:56.137 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:55.190+0000 7fb76f5bf700 1 -- 172.21.15.5:0/3148945710 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7681a09c0 con 0x7fb7681083a0 2024-05-09T00:35:56.137 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:55.190+0000 7fb75e7fc700 1 -- 172.21.15.5:0/3148945710 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb7640196a0 con 0x7fb7681083a0 2024-05-09T00:35:56.137 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:55.190+0000 7fb76f5bf700 1 -- 172.21.15.5:0/3148945710 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7681a1400 con 0x7fb7681083a0 2024-05-09T00:35:56.138 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:55.191+0000 7fb76f5bf700 1 -- 172.21.15.5:0/3148945710 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb768064e20 con 0x7fb7681083a0 2024-05-09T00:35:56.138 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:55.191+0000 7fb75e7fc700 1 -- 172.21.15.5:0/3148945710 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fb764019960 con 0x7fb7681083a0 2024-05-09T00:35:56.138 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:55.191+0000 7fb75e7fc700 1 --2- 172.21.15.5:0/3148945710 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7fb75403e470 0x7fb754040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:35:56.138 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:55.191+0000 7fb75e7fc700 1 -- 172.21.15.5:0/3148945710 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb764053b40 con 0x7fb7681083a0 2024-05-09T00:35:56.138 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:55.192+0000 7fb76cb5a700 1 --2- 172.21.15.5:0/3148945710 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7fb75403e470 0x7fb754040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:35:56.138 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:55.192+0000 7fb76cb5a700 1 --2- 172.21.15.5:0/3148945710 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7fb75403e470 0x7fb754040980 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb758006fd0 tx=0x7fb758006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:35:56.138 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:55.197+0000 7fb75e7fc700 1 -- 172.21.15.5:0/3148945710 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb764017e90 con 0x7fb7681083a0 2024-05-09T00:35:56.138 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:55.431+0000 7fb76f5bf700 1 -- 172.21.15.5:0/3148945710 --> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fb7680008d0 con 0x7fb75403e470 2024-05-09T00:35:56.138 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:55.437+0000 7fb75e7fc700 1 -- 172.21.15.5:0/3148945710 <== mgr.14120 v2:172.21.15.5:6800/287842266 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb7680008d0 con 0x7fb75403e470 2024-05-09T00:35:56.138 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:55.439+0000 7fb76f5bf700 1 -- 172.21.15.5:0/3148945710 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7fb75403e470 msgr2=0x7fb754040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:35:56.138 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:55.439+0000 7fb76f5bf700 1 --2- 172.21.15.5:0/3148945710 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7fb75403e470 0x7fb754040980 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb758006fd0 tx=0x7fb758006e40 comp rx=0 tx=0).stop 2024-05-09T00:35:56.138 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:55.439+0000 7fb76f5bf700 1 -- 172.21.15.5:0/3148945710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7681083a0 msgr2=0x7fb7681a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:35:56.138 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:55.439+0000 7fb76f5bf700 1 --2- 172.21.15.5:0/3148945710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7681083a0 0x7fb7681a01f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb7640117b0 tx=0x7fb764011b70 comp rx=0 tx=0).stop 2024-05-09T00:35:56.139 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:55.439+0000 7fb76f5bf700 1 -- 172.21.15.5:0/3148945710 shutdown_connections 2024-05-09T00:35:56.139 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:55.439+0000 7fb76f5bf700 1 --2- 172.21.15.5:0/3148945710 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7fb75403e470 0x7fb754040980 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:35:56.139 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:55.439+0000 7fb76f5bf700 1 --2- 172.21.15.5:0/3148945710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7681083a0 0x7fb7681a01f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:35:56.139 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:55.439+0000 7fb76f5bf700 1 -- 172.21.15.5:0/3148945710 >> 172.21.15.5:0/3148945710 conn(0x7fb768104050 msgr2=0x7fb768196760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:35:56.139 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:55.439+0000 7fb76f5bf700 1 -- 172.21.15.5:0/3148945710 shutdown_connections 2024-05-09T00:35:56.139 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:55.439+0000 7fb76f5bf700 1 -- 172.21.15.5:0/3148945710 wait complete. 2024-05-09T00:35:56.479 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:56 smithi005 ceph-mon[30365]: mgrmap e8: x(active, since 2s) 2024-05-09T00:35:56.479 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:56 smithi005 ceph-mon[30365]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:35:56.479 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:56 smithi005 ceph-mon[30365]: from='mgr.14120 172.21.15.5:0/1775107462' entity='mgr.x' 2024-05-09T00:35:56.479 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:56 smithi005 ceph-mon[30365]: from='mgr.14120 172.21.15.5:0/1775107462' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:35:57.408 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-09T00:35:57.408 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:56.602+0000 7f5d02bcc700 1 Processor -- start 2024-05-09T00:35:57.408 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:56.602+0000 7f5d02bcc700 1 -- start start 2024-05-09T00:35:57.409 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:56.602+0000 7f5d02bcc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cfc108990 0x7f5cfc108da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:35:57.409 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:56.602+0000 7f5d02bcc700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5cfc109390 con 0x7f5cfc108990 2024-05-09T00:35:57.409 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:56.602+0000 7f5d00968700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cfc108990 0x7f5cfc108da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:35:57.409 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:56.602+0000 7f5d00968700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cfc108990 0x7f5cfc108da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38708/0 (socket says 172.21.15.5:38708) 2024-05-09T00:35:57.409 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:56.602+0000 7f5d00968700 1 -- 172.21.15.5:0/1069725550 learned_addr learned my addr 172.21.15.5:0/1069725550 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:35:57.409 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:56.602+0000 7f5d00968700 1 -- 172.21.15.5:0/1069725550 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cfc109b60 con 0x7f5cfc108990 2024-05-09T00:35:57.409 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:56.603+0000 7f5d00968700 1 --2- 172.21.15.5:0/1069725550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cfc108990 0x7f5cfc108da0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5cf00098a0 tx=0x7f5cf0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c7f29b22f197884 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:35:57.409 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:56.603+0000 7f5cfb7fe700 1 -- 172.21.15.5:0/1069725550 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5cf0004160 con 0x7f5cfc108990 2024-05-09T00:35:57.409 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:56.603+0000 7f5cfb7fe700 1 -- 172.21.15.5:0/1069725550 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5cf0004300 con 0x7f5cfc108990 2024-05-09T00:35:57.409 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:56.603+0000 7f5cfb7fe700 1 -- 172.21.15.5:0/1069725550 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5cf0011af0 con 0x7f5cfc108990 2024-05-09T00:35:57.409 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:56.604+0000 7f5d02bcc700 1 -- 172.21.15.5:0/1069725550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cfc108990 msgr2=0x7f5cfc108da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:35:57.409 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:56.604+0000 7f5d02bcc700 1 --2- 172.21.15.5:0/1069725550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cfc108990 0x7f5cfc108da0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5cf00098a0 tx=0x7f5cf0009bb0 comp rx=0 tx=0).stop 2024-05-09T00:35:57.410 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:56.604+0000 7f5d02bcc700 1 -- 172.21.15.5:0/1069725550 shutdown_connections 2024-05-09T00:35:57.410 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:56.604+0000 7f5d02bcc700 1 --2- 172.21.15.5:0/1069725550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cfc108990 0x7f5cfc108da0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:35:57.410 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:56.604+0000 7f5d02bcc700 1 -- 172.21.15.5:0/1069725550 >> 172.21.15.5:0/1069725550 conn(0x7f5cfc103ff0 msgr2=0x7f5cfc106480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:35:57.410 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:56.604+0000 7f5d02bcc700 1 -- 172.21.15.5:0/1069725550 shutdown_connections 2024-05-09T00:35:57.410 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:56.604+0000 7f5d02bcc700 1 -- 172.21.15.5:0/1069725550 wait complete. 2024-05-09T00:35:57.410 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:56.604+0000 7f5d02bcc700 1 Processor -- start 2024-05-09T00:35:57.410 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:56.604+0000 7f5d02bcc700 1 -- start start 2024-05-09T00:35:57.411 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:56.605+0000 7f5d02bcc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cfc108990 0x7f5cfc19ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:35:57.411 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:56.605+0000 7f5d02bcc700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5cfc10cf10 con 0x7f5cfc108990 2024-05-09T00:35:57.411 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:56.605+0000 7f5d00968700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cfc108990 0x7f5cfc19ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:35:57.411 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:56.605+0000 7f5d00968700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cfc108990 0x7f5cfc19ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38714/0 (socket says 172.21.15.5:38714) 2024-05-09T00:35:57.411 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:56.605+0000 7f5d00968700 1 -- 172.21.15.5:0/2292570287 learned_addr learned my addr 172.21.15.5:0/2292570287 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:35:57.411 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:56.605+0000 7f5d00968700 1 -- 172.21.15.5:0/2292570287 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cfc19d3c0 con 0x7f5cfc108990 2024-05-09T00:35:57.411 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:56.605+0000 7f5d00968700 1 --2- 172.21.15.5:0/2292570287 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cfc108990 0x7f5cfc19ce80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0011670 tx=0x7f5cf0004280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:35:57.411 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:56.605+0000 7f5cf9ffb700 1 -- 172.21.15.5:0/2292570287 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5cf00036a0 con 0x7f5cfc108990 2024-05-09T00:35:57.411 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:56.605+0000 7f5cf9ffb700 1 -- 172.21.15.5:0/2292570287 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5cf0003cd0 con 0x7f5cfc108990 2024-05-09T00:35:57.411 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:56.605+0000 7f5d02bcc700 1 -- 172.21.15.5:0/2292570287 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cfc19d650 con 0x7f5cfc108990 2024-05-09T00:35:57.411 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:56.605+0000 7f5cf9ffb700 1 -- 172.21.15.5:0/2292570287 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5cf00193c0 con 0x7f5cfc108990 2024-05-09T00:35:57.411 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:56.606+0000 7f5d02bcc700 1 -- 172.21.15.5:0/2292570287 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cfc1a0260 con 0x7f5cfc108990 2024-05-09T00:35:57.411 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:56.607+0000 7f5cf9ffb700 1 -- 172.21.15.5:0/2292570287 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f5cf0019630 con 0x7f5cfc108990 2024-05-09T00:35:57.412 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:56.607+0000 7f5d02bcc700 1 -- 172.21.15.5:0/2292570287 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cfc064e20 con 0x7f5cfc108990 2024-05-09T00:35:57.412 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:56.607+0000 7f5cf9ffb700 1 --2- 172.21.15.5:0/2292570287 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7f5cec03e470 0x7f5cec040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:35:57.412 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:56.607+0000 7f5cf9ffb700 1 -- 172.21.15.5:0/2292570287 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5cf00538f0 con 0x7f5cfc108990 2024-05-09T00:35:57.412 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:56.607+0000 7f5cfbfff700 1 --2- 172.21.15.5:0/2292570287 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7f5cec03e470 0x7f5cec040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:35:57.412 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:56.608+0000 7f5cfbfff700 1 --2- 172.21.15.5:0/2292570287 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7f5cec03e470 0x7f5cec040980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8006fd0 tx=0x7f5ce8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:35:57.412 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:56.612+0000 7f5cf9ffb700 1 -- 172.21.15.5:0/2292570287 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5cf0023390 con 0x7f5cfc108990 2024-05-09T00:35:57.412 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:56.829+0000 7f5d02bcc700 1 -- 172.21.15.5:0/2292570287 --> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f5cfc105110 con 0x7f5cec03e470 2024-05-09T00:35:57.412 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:56.829+0000 7f5cf9ffb700 1 -- 172.21.15.5:0/2292570287 <== mgr.14120 v2:172.21.15.5:6800/287842266 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f5cfc105110 con 0x7f5cec03e470 2024-05-09T00:35:57.412 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:56.832+0000 7f5d02bcc700 1 -- 172.21.15.5:0/2292570287 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7f5cec03e470 msgr2=0x7f5cec040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:35:57.412 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:56.832+0000 7f5d02bcc700 1 --2- 172.21.15.5:0/2292570287 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7f5cec03e470 0x7f5cec040980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8006fd0 tx=0x7f5ce8006e40 comp rx=0 tx=0).stop 2024-05-09T00:35:57.412 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:56.832+0000 7f5d02bcc700 1 -- 172.21.15.5:0/2292570287 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cfc108990 msgr2=0x7f5cfc19ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:35:57.412 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:56.832+0000 7f5d02bcc700 1 --2- 172.21.15.5:0/2292570287 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cfc108990 0x7f5cfc19ce80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0011670 tx=0x7f5cf0004280 comp rx=0 tx=0).stop 2024-05-09T00:35:57.412 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:56.832+0000 7f5d02bcc700 1 -- 172.21.15.5:0/2292570287 shutdown_connections 2024-05-09T00:35:57.412 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:56.832+0000 7f5d02bcc700 1 --2- 172.21.15.5:0/2292570287 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7f5cec03e470 0x7f5cec040980 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:35:57.413 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:56.832+0000 7f5d02bcc700 1 --2- 172.21.15.5:0/2292570287 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cfc108990 0x7f5cfc19ce80 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:35:57.413 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:56.832+0000 7f5d02bcc700 1 -- 172.21.15.5:0/2292570287 >> 172.21.15.5:0/2292570287 conn(0x7f5cfc103ff0 msgr2=0x7f5cfc1049a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:35:57.413 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:56.832+0000 7f5d02bcc700 1 -- 172.21.15.5:0/2292570287 shutdown_connections 2024-05-09T00:35:57.413 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:56.832+0000 7f5d02bcc700 1 -- 172.21.15.5:0/2292570287 wait complete. 2024-05-09T00:35:57.413 INFO:teuthology.orchestra.run.smithi005.stdout:Generating ssh key... 2024-05-09T00:35:58.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:57 smithi005 ceph-mon[30365]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:35:58.702 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:58 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: Generating public/private rsa key pair. 2024-05-09T00:35:58.702 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:58 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: Your identification has been saved in /tmp/tmpudjeunjw/key. 2024-05-09T00:35:58.702 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:58 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: Your public key has been saved in /tmp/tmpudjeunjw/key.pub. 2024-05-09T00:35:58.703 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:58 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: The key fingerprint is: 2024-05-09T00:35:58.703 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:58 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: SHA256:Cj8jo0c3z/Fce3/5dexc0tZnramalXZPBb1Ir54oXxA ceph-d85814c6-0d9b-11ef-bc97-c7b262605968 2024-05-09T00:35:58.703 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:58 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: The key's randomart image is: 2024-05-09T00:35:58.703 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:58 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: +---[RSA 3072]----+ 2024-05-09T00:35:58.703 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:58 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: | | 2024-05-09T00:35:58.703 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:58 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: | . | 2024-05-09T00:35:58.703 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:58 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: | E .. .| 2024-05-09T00:35:58.703 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:58 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: | o o..| 2024-05-09T00:35:58.703 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:58 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: | . S . . o.| 2024-05-09T00:35:58.704 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:58 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: | .oo.. o..o+| 2024-05-09T00:35:58.704 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:58 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: | .o.=+ + .++o #| 2024-05-09T00:35:58.704 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:58 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: | ..o oo.o+=.o%*| 2024-05-09T00:35:58.704 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:58 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: | .. =+.=o.O| 2024-05-09T00:35:58.704 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:35:58 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: +----[SHA256]-----+ 2024-05-09T00:35:58.982 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:57.879+0000 7fb43691a700 1 Processor -- start 2024-05-09T00:35:58.982 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:57.880+0000 7fb43691a700 1 -- start start 2024-05-09T00:35:58.982 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:57.880+0000 7fb43691a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb430108970 0x7fb430108d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:35:58.982 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:57.880+0000 7fb43691a700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb430109370 con 0x7fb430108970 2024-05-09T00:35:58.982 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:57.880+0000 7fb42ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb430108970 0x7fb430108d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:35:58.982 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:57.880+0000 7fb42ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb430108970 0x7fb430108d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38718/0 (socket says 172.21.15.5:38718) 2024-05-09T00:35:58.983 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:57.880+0000 7fb42ffff700 1 -- 172.21.15.5:0/2854117685 learned_addr learned my addr 172.21.15.5:0/2854117685 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:35:58.983 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:57.880+0000 7fb42ffff700 1 -- 172.21.15.5:0/2854117685 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb430109b40 con 0x7fb430108970 2024-05-09T00:35:58.983 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:57.880+0000 7fb42ffff700 1 --2- 172.21.15.5:0/2854117685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb430108970 0x7fb430108d80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb4200098a0 tx=0x7fb420009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cedfa3e63d87772c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:35:58.983 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:57.881+0000 7fb42f7fe700 1 -- 172.21.15.5:0/2854117685 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb420004160 con 0x7fb430108970 2024-05-09T00:35:58.983 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:57.881+0000 7fb42f7fe700 1 -- 172.21.15.5:0/2854117685 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb420004300 con 0x7fb430108970 2024-05-09T00:35:58.983 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:57.881+0000 7fb42f7fe700 1 -- 172.21.15.5:0/2854117685 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb420011af0 con 0x7fb430108970 2024-05-09T00:35:58.983 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:57.881+0000 7fb43691a700 1 -- 172.21.15.5:0/2854117685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb430108970 msgr2=0x7fb430108d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:35:58.983 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:57.881+0000 7fb43691a700 1 --2- 172.21.15.5:0/2854117685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb430108970 0x7fb430108d80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb4200098a0 tx=0x7fb420009bb0 comp rx=0 tx=0).stop 2024-05-09T00:35:58.984 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:57.881+0000 7fb43691a700 1 -- 172.21.15.5:0/2854117685 shutdown_connections 2024-05-09T00:35:58.984 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:57.881+0000 7fb43691a700 1 --2- 172.21.15.5:0/2854117685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb430108970 0x7fb430108d80 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:35:58.984 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:57.882+0000 7fb43691a700 1 -- 172.21.15.5:0/2854117685 >> 172.21.15.5:0/2854117685 conn(0x7fb430104050 msgr2=0x7fb4301064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:35:58.984 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:57.882+0000 7fb43691a700 1 -- 172.21.15.5:0/2854117685 shutdown_connections 2024-05-09T00:35:58.984 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:57.882+0000 7fb43691a700 1 -- 172.21.15.5:0/2854117685 wait complete. 2024-05-09T00:35:58.985 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:57.882+0000 7fb43691a700 1 Processor -- start 2024-05-09T00:35:58.985 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:57.882+0000 7fb43691a700 1 -- start start 2024-05-09T00:35:58.985 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:57.882+0000 7fb43691a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb430108970 0x7fb43007c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:35:58.985 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:57.882+0000 7fb43691a700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb43010cef0 con 0x7fb430108970 2024-05-09T00:35:58.985 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:57.883+0000 7fb42ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb430108970 0x7fb43007c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:35:58.985 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:57.883+0000 7fb42ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb430108970 0x7fb43007c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38728/0 (socket says 172.21.15.5:38728) 2024-05-09T00:35:58.985 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:57.883+0000 7fb42ffff700 1 -- 172.21.15.5:0/3008440194 learned_addr learned my addr 172.21.15.5:0/3008440194 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:35:58.985 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:57.883+0000 7fb42ffff700 1 -- 172.21.15.5:0/3008440194 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb43007ce80 con 0x7fb430108970 2024-05-09T00:35:58.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:57.883+0000 7fb42ffff700 1 --2- 172.21.15.5:0/3008440194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb430108970 0x7fb43007c940 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb420011670 tx=0x7fb420004280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:35:58.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:57.883+0000 7fb42dffb700 1 -- 172.21.15.5:0/3008440194 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4200036a0 con 0x7fb430108970 2024-05-09T00:35:58.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:57.883+0000 7fb42dffb700 1 -- 172.21.15.5:0/3008440194 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb420003cd0 con 0x7fb430108970 2024-05-09T00:35:58.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:57.883+0000 7fb43691a700 1 -- 172.21.15.5:0/3008440194 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4301a0cd0 con 0x7fb430108970 2024-05-09T00:35:58.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:57.883+0000 7fb42dffb700 1 -- 172.21.15.5:0/3008440194 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4200193c0 con 0x7fb430108970 2024-05-09T00:35:58.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:57.883+0000 7fb43691a700 1 -- 172.21.15.5:0/3008440194 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4301a1150 con 0x7fb430108970 2024-05-09T00:35:58.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:57.884+0000 7fb42dffb700 1 -- 172.21.15.5:0/3008440194 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fb420019630 con 0x7fb430108970 2024-05-09T00:35:58.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:57.885+0000 7fb42dffb700 1 --2- 172.21.15.5:0/3008440194 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7fb41c03e470 0x7fb41c040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:35:58.987 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:57.885+0000 7fb42dffb700 1 -- 172.21.15.5:0/3008440194 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb420054a70 con 0x7fb430108970 2024-05-09T00:35:58.987 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:57.885+0000 7fb43691a700 1 -- 172.21.15.5:0/3008440194 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb430064e20 con 0x7fb430108970 2024-05-09T00:35:58.987 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:57.885+0000 7fb427fff700 1 --2- 172.21.15.5:0/3008440194 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7fb41c03e470 0x7fb41c040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:35:58.987 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:57.885+0000 7fb427fff700 1 --2- 172.21.15.5:0/3008440194 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7fb41c03e470 0x7fb41c040980 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb418006fd0 tx=0x7fb418006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:35:58.987 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:57.890+0000 7fb42dffb700 1 -- 172.21.15.5:0/3008440194 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb420017e90 con 0x7fb430108970 2024-05-09T00:35:58.987 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:58.106+0000 7fb43691a700 1 -- 172.21.15.5:0/3008440194 --> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7fb4301a1490 con 0x7fb41c03e470 2024-05-09T00:35:58.987 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:58.444+0000 7fb42dffb700 1 -- 172.21.15.5:0/3008440194 <== mgr.14120 v2:172.21.15.5:6800/287842266 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb4301a1490 con 0x7fb41c03e470 2024-05-09T00:35:58.987 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:58.446+0000 7fb43691a700 1 -- 172.21.15.5:0/3008440194 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7fb41c03e470 msgr2=0x7fb41c040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:35:58.987 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:58.446+0000 7fb43691a700 1 --2- 172.21.15.5:0/3008440194 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7fb41c03e470 0x7fb41c040980 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb418006fd0 tx=0x7fb418006e40 comp rx=0 tx=0).stop 2024-05-09T00:35:58.987 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:58.446+0000 7fb43691a700 1 -- 172.21.15.5:0/3008440194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb430108970 msgr2=0x7fb43007c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:35:58.988 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:58.446+0000 7fb43691a700 1 --2- 172.21.15.5:0/3008440194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb430108970 0x7fb43007c940 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb420011670 tx=0x7fb420004280 comp rx=0 tx=0).stop 2024-05-09T00:35:58.988 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:58.446+0000 7fb43691a700 1 -- 172.21.15.5:0/3008440194 shutdown_connections 2024-05-09T00:35:58.988 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:58.446+0000 7fb43691a700 1 --2- 172.21.15.5:0/3008440194 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7fb41c03e470 0x7fb41c040980 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:35:58.988 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:58.446+0000 7fb43691a700 1 --2- 172.21.15.5:0/3008440194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb430108970 0x7fb43007c940 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:35:58.988 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:58.447+0000 7fb43691a700 1 -- 172.21.15.5:0/3008440194 >> 172.21.15.5:0/3008440194 conn(0x7fb430104050 msgr2=0x7fb430105a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:35:58.988 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:58.447+0000 7fb43691a700 1 -- 172.21.15.5:0/3008440194 shutdown_connections 2024-05-09T00:35:58.988 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:58.447+0000 7fb43691a700 1 -- 172.21.15.5:0/3008440194 wait complete. 2024-05-09T00:35:59.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:59 smithi005 ceph-mon[30365]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:35:59.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:59 smithi005 ceph-mon[30365]: Generating ssh key... 2024-05-09T00:35:59.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:59 smithi005 ceph-mon[30365]: from='mgr.14120 172.21.15.5:0/1775107462' entity='mgr.x' 2024-05-09T00:35:59.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:35:59 smithi005 ceph-mon[30365]: from='mgr.14120 172.21.15.5:0/1775107462' entity='mgr.x' 2024-05-09T00:36:00.239 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3Q373eMCgV6KQqan02BZzux9ole+ws4jzB8UmGChPUoJpFOElDEG0fJVOFVJpdDQ+5eZKZm3J8hRnfjAWNhT3h8Tdez/uiRi4EcM8op9B0KCL0Z4z5h066fKVQi4dhoxrEfW5T9/hmsC2axHQOb9iUoBEBfZIXY33RgdBFoK8Dy20sQuC2llQ5sS5tCkPipwGChS1NVijs1m4YJzJOzDpW6Q0DcPzbTXNrTWecdYOv5HwjSAE8Qlp6kTXezfeQXFv6+M9tOGCbve2tNxyFvaaDqLz7Vif1CeUccKnOC+hQIQwOnIUYqyoPmd+nvcR3wlTlqY+QNAvLK36hPsS6eG8Jr3ndGmXnT2fVQVRmW4KlhYKu6FSJIzEGN5s8iw/dz5piEvj1qGppDTOow1le/qs702U7IiGPzTdXEjg1wJcrRUhUpUVq0xYAT8CJW22gBwd60Y+c469RhudpxLSbzfO81ePAXa8JJaQkaXORIrG5SXdjt1n2jp8KRtceUgr6TE= ceph-d85814c6-0d9b-11ef-bc97-c7b262605968 2024-05-09T00:36:00.239 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:59.431+0000 7f91b7334700 1 Processor -- start 2024-05-09T00:36:00.239 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:59.431+0000 7f91b7334700 1 -- start start 2024-05-09T00:36:00.239 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:59.431+0000 7f91b7334700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91b007f660 0x7f91b007dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:36:00.239 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:59.431+0000 7f91b7334700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f91b007e020 con 0x7f91b007f660 2024-05-09T00:36:00.239 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:59.431+0000 7f91b50d0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91b007f660 0x7f91b007dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:36:00.239 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:59.431+0000 7f91b50d0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91b007f660 0x7f91b007dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38736/0 (socket says 172.21.15.5:38736) 2024-05-09T00:36:00.239 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:59.431+0000 7f91b50d0700 1 -- 172.21.15.5:0/2908976328 learned_addr learned my addr 172.21.15.5:0/2908976328 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:36:00.239 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:59.432+0000 7f91b50d0700 1 -- 172.21.15.5:0/2908976328 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91b007e1a0 con 0x7f91b007f660 2024-05-09T00:36:00.240 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:59.432+0000 7f91b50d0700 1 --2- 172.21.15.5:0/2908976328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91b007f660 0x7f91b007dae0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f91ac0098a0 tx=0x7f91ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b24d528c9b9a8632 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:36:00.240 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:59.432+0000 7f91a7fff700 1 -- 172.21.15.5:0/2908976328 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f91ac004160 con 0x7f91b007f660 2024-05-09T00:36:00.240 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:59.432+0000 7f91a7fff700 1 -- 172.21.15.5:0/2908976328 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f91ac004300 con 0x7f91b007f660 2024-05-09T00:36:00.240 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:59.432+0000 7f91a7fff700 1 -- 172.21.15.5:0/2908976328 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f91ac011af0 con 0x7f91b007f660 2024-05-09T00:36:00.240 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:59.433+0000 7f91b7334700 1 -- 172.21.15.5:0/2908976328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91b007f660 msgr2=0x7f91b007dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:36:00.240 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:59.433+0000 7f91b7334700 1 --2- 172.21.15.5:0/2908976328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91b007f660 0x7f91b007dae0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f91ac0098a0 tx=0x7f91ac009bb0 comp rx=0 tx=0).stop 2024-05-09T00:36:00.240 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:59.433+0000 7f91b7334700 1 -- 172.21.15.5:0/2908976328 shutdown_connections 2024-05-09T00:36:00.240 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:59.433+0000 7f91b7334700 1 --2- 172.21.15.5:0/2908976328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91b007f660 0x7f91b007dae0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:36:00.240 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:59.433+0000 7f91b7334700 1 -- 172.21.15.5:0/2908976328 >> 172.21.15.5:0/2908976328 conn(0x7f91b0105100 msgr2=0x7f91b0107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:36:00.240 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:59.433+0000 7f91b7334700 1 -- 172.21.15.5:0/2908976328 shutdown_connections 2024-05-09T00:36:00.241 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:59.433+0000 7f91b7334700 1 -- 172.21.15.5:0/2908976328 wait complete. 2024-05-09T00:36:00.241 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:59.434+0000 7f91b7334700 1 Processor -- start 2024-05-09T00:36:00.241 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:59.434+0000 7f91b7334700 1 -- start start 2024-05-09T00:36:00.241 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:59.434+0000 7f91b7334700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91b007f660 0x7f91b01a00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:36:00.241 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:59.434+0000 7f91b7334700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f91b0110060 con 0x7f91b007f660 2024-05-09T00:36:00.241 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:59.434+0000 7f91b50d0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91b007f660 0x7f91b01a00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:36:00.241 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:59.434+0000 7f91b50d0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91b007f660 0x7f91b01a00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38750/0 (socket says 172.21.15.5:38750) 2024-05-09T00:36:00.241 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:59.434+0000 7f91b50d0700 1 -- 172.21.15.5:0/1141005478 learned_addr learned my addr 172.21.15.5:0/1141005478 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:36:00.241 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:59.434+0000 7f91b50d0700 1 -- 172.21.15.5:0/1141005478 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91b01a05f0 con 0x7f91b007f660 2024-05-09T00:36:00.241 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:59.434+0000 7f91b50d0700 1 --2- 172.21.15.5:0/1141005478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91b007f660 0x7f91b01a00b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f91ac0117b0 tx=0x7f91ac011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:36:00.241 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:59.434+0000 7f91a67fc700 1 -- 172.21.15.5:0/1141005478 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f91ac0039c0 con 0x7f91b007f660 2024-05-09T00:36:00.241 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:59.435+0000 7f91b7334700 1 -- 172.21.15.5:0/1141005478 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91b01a0880 con 0x7f91b007f660 2024-05-09T00:36:00.241 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:59.435+0000 7f91a67fc700 1 -- 172.21.15.5:0/1141005478 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f91ac003b60 con 0x7f91b007f660 2024-05-09T00:36:00.241 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:59.435+0000 7f91a67fc700 1 -- 172.21.15.5:0/1141005478 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f91ac0196a0 con 0x7f91b007f660 2024-05-09T00:36:00.242 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:59.435+0000 7f91b7334700 1 -- 172.21.15.5:0/1141005478 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91b01a3490 con 0x7f91b007f660 2024-05-09T00:36:00.242 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:59.436+0000 7f91a67fc700 1 -- 172.21.15.5:0/1141005478 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f91ac01a430 con 0x7f91b007f660 2024-05-09T00:36:00.242 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:59.436+0000 7f91b7334700 1 -- 172.21.15.5:0/1141005478 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91b0064e20 con 0x7f91b007f660 2024-05-09T00:36:00.242 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:59.436+0000 7f91a67fc700 1 --2- 172.21.15.5:0/1141005478 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7f919c03e470 0x7f919c040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:36:00.242 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:59.436+0000 7f91a67fc700 1 -- 172.21.15.5:0/1141005478 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f91ac053a90 con 0x7f91b007f660 2024-05-09T00:36:00.242 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:59.436+0000 7f91b48cf700 1 --2- 172.21.15.5:0/1141005478 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7f919c03e470 0x7f919c040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:36:00.242 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:59.436+0000 7f91b48cf700 1 --2- 172.21.15.5:0/1141005478 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7f919c03e470 0x7f919c040980 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f91a0006fd0 tx=0x7f91a0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:36:00.242 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:59.441+0000 7f91a67fc700 1 -- 172.21.15.5:0/1141005478 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f91ac01a720 con 0x7f91b007f660 2024-05-09T00:36:00.242 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:59.660+0000 7f91b7334700 1 -- 172.21.15.5:0/1141005478 --> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f91b0106260 con 0x7f919c03e470 2024-05-09T00:36:00.242 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:59.660+0000 7f91a67fc700 1 -- 172.21.15.5:0/1141005478 <== mgr.14120 v2:172.21.15.5:6800/287842266 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f91b0106260 con 0x7f919c03e470 2024-05-09T00:36:00.242 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:59.663+0000 7f91b7334700 1 -- 172.21.15.5:0/1141005478 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7f919c03e470 msgr2=0x7f919c040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:36:00.242 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:59.663+0000 7f91b7334700 1 --2- 172.21.15.5:0/1141005478 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7f919c03e470 0x7f919c040980 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f91a0006fd0 tx=0x7f91a0006e40 comp rx=0 tx=0).stop 2024-05-09T00:36:00.242 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:59.663+0000 7f91b7334700 1 -- 172.21.15.5:0/1141005478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91b007f660 msgr2=0x7f91b01a00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:36:00.243 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:59.663+0000 7f91b7334700 1 --2- 172.21.15.5:0/1141005478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91b007f660 0x7f91b01a00b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f91ac0117b0 tx=0x7f91ac011b70 comp rx=0 tx=0).stop 2024-05-09T00:36:00.243 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:59.663+0000 7f91b7334700 1 -- 172.21.15.5:0/1141005478 shutdown_connections 2024-05-09T00:36:00.243 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:59.663+0000 7f91b7334700 1 --2- 172.21.15.5:0/1141005478 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7f919c03e470 0x7f919c040980 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:36:00.243 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:59.663+0000 7f91b7334700 1 --2- 172.21.15.5:0/1141005478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91b007f660 0x7f91b01a00b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:36:00.243 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:59.663+0000 7f91b7334700 1 -- 172.21.15.5:0/1141005478 >> 172.21.15.5:0/1141005478 conn(0x7f91b0105100 msgr2=0x7f91b0105ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:36:00.243 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:59.663+0000 7f91b7334700 1 -- 172.21.15.5:0/1141005478 shutdown_connections 2024-05-09T00:36:00.243 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:35:59.663+0000 7f91b7334700 1 -- 172.21.15.5:0/1141005478 wait complete. 2024-05-09T00:36:00.243 INFO:teuthology.orchestra.run.smithi005.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-09T00:36:00.243 INFO:teuthology.orchestra.run.smithi005.stdout:Adding key to root@localhost authorized_keys... 2024-05-09T00:36:00.243 INFO:teuthology.orchestra.run.smithi005.stdout:Adding host smithi005... 2024-05-09T00:36:00.925 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:00 smithi005 ceph-mon[30365]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:36:01.838 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:01 smithi005 ceph-mon[30365]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi005", "addr": "172.21.15.5", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:36:01.838 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:01 smithi005 ceph-mon[30365]: Deploying cephadm binary to smithi005 2024-05-09T00:36:05.802 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout Added host 'smithi005' with addr '172.21.15.5' 2024-05-09T00:36:05.802 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:00.691+0000 7f7394455700 1 Processor -- start 2024-05-09T00:36:05.802 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:00.691+0000 7f7394455700 1 -- start start 2024-05-09T00:36:05.802 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:00.691+0000 7f7394455700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f738c10a570 0x7f738c10c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:36:05.803 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:00.691+0000 7f7394455700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f738c07d700 con 0x7f738c10a570 2024-05-09T00:36:05.803 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:00.691+0000 7f73921f1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f738c10a570 0x7f738c10c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:36:05.803 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:00.691+0000 7f73921f1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f738c10a570 0x7f738c10c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38758/0 (socket says 172.21.15.5:38758) 2024-05-09T00:36:05.803 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:00.691+0000 7f73921f1700 1 -- 172.21.15.5:0/3660839733 learned_addr learned my addr 172.21.15.5:0/3660839733 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:36:05.803 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:00.692+0000 7f73921f1700 1 -- 172.21.15.5:0/3660839733 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f738c10ced0 con 0x7f738c10a570 2024-05-09T00:36:05.803 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:00.692+0000 7f73921f1700 1 --2- 172.21.15.5:0/3660839733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f738c10a570 0x7f738c10c990 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f73880098a0 tx=0x7f7388009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9031a3ac151cb32b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:36:05.808 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:00.692+0000 7f73911ef700 1 -- 172.21.15.5:0/3660839733 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7388004160 con 0x7f738c10a570 2024-05-09T00:36:05.808 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:00.692+0000 7f73911ef700 1 -- 172.21.15.5:0/3660839733 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7388004300 con 0x7f738c10a570 2024-05-09T00:36:05.809 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:00.692+0000 7f73911ef700 1 -- 172.21.15.5:0/3660839733 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7388011af0 con 0x7f738c10a570 2024-05-09T00:36:05.809 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:00.693+0000 7f7394455700 1 -- 172.21.15.5:0/3660839733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f738c10a570 msgr2=0x7f738c10c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:36:05.809 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:00.693+0000 7f7394455700 1 --2- 172.21.15.5:0/3660839733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f738c10a570 0x7f738c10c990 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f73880098a0 tx=0x7f7388009bb0 comp rx=0 tx=0).stop 2024-05-09T00:36:05.809 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:00.693+0000 7f7394455700 1 -- 172.21.15.5:0/3660839733 shutdown_connections 2024-05-09T00:36:05.809 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:00.693+0000 7f7394455700 1 --2- 172.21.15.5:0/3660839733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f738c10a570 0x7f738c10c990 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:36:05.809 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:00.693+0000 7f7394455700 1 -- 172.21.15.5:0/3660839733 >> 172.21.15.5:0/3660839733 conn(0x7f738c103ff0 msgr2=0x7f738c106480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:36:05.809 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:00.693+0000 7f7394455700 1 -- 172.21.15.5:0/3660839733 shutdown_connections 2024-05-09T00:36:05.809 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:00.693+0000 7f7394455700 1 -- 172.21.15.5:0/3660839733 wait complete. 2024-05-09T00:36:05.809 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:00.694+0000 7f7394455700 1 Processor -- start 2024-05-09T00:36:05.809 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:00.694+0000 7f7394455700 1 -- start start 2024-05-09T00:36:05.809 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:00.694+0000 7f7394455700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f738c10a570 0x7f738c109cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:36:05.809 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:00.694+0000 7f7394455700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f738c10de90 con 0x7f738c10a570 2024-05-09T00:36:05.809 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:00.694+0000 7f73921f1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f738c10a570 0x7f738c109cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:36:05.810 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:00.694+0000 7f73921f1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f738c10a570 0x7f738c109cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38768/0 (socket says 172.21.15.5:38768) 2024-05-09T00:36:05.810 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:00.694+0000 7f73921f1700 1 -- 172.21.15.5:0/4059990812 learned_addr learned my addr 172.21.15.5:0/4059990812 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:36:05.810 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:00.694+0000 7f73921f1700 1 -- 172.21.15.5:0/4059990812 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f738c10a230 con 0x7f738c10a570 2024-05-09T00:36:05.810 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:00.695+0000 7f73921f1700 1 --2- 172.21.15.5:0/4059990812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f738c10a570 0x7f738c109cf0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f73880117b0 tx=0x7f7388011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:36:05.810 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:00.695+0000 7f73837fe700 1 -- 172.21.15.5:0/4059990812 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73880039c0 con 0x7f738c10a570 2024-05-09T00:36:05.810 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:00.695+0000 7f73837fe700 1 -- 172.21.15.5:0/4059990812 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7388003b60 con 0x7f738c10a570 2024-05-09T00:36:05.810 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:00.695+0000 7f7394455700 1 -- 172.21.15.5:0/4059990812 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f738c108480 con 0x7f738c10a570 2024-05-09T00:36:05.810 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:00.695+0000 7f73837fe700 1 -- 172.21.15.5:0/4059990812 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73880196a0 con 0x7f738c10a570 2024-05-09T00:36:05.810 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:00.695+0000 7f7394455700 1 -- 172.21.15.5:0/4059990812 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f738c108960 con 0x7f738c10a570 2024-05-09T00:36:05.810 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:00.696+0000 7f7394455700 1 -- 172.21.15.5:0/4059990812 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f738c064e20 con 0x7f738c10a570 2024-05-09T00:36:05.810 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:00.696+0000 7f73837fe700 1 -- 172.21.15.5:0/4059990812 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f7388019910 con 0x7f738c10a570 2024-05-09T00:36:05.811 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:00.696+0000 7f73837fe700 1 --2- 172.21.15.5:0/4059990812 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7f737803e4c0 0x7f73780409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:36:05.811 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:00.696+0000 7f73837fe700 1 -- 172.21.15.5:0/4059990812 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f73880538d0 con 0x7f738c10a570 2024-05-09T00:36:05.811 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:00.696+0000 7f73919f0700 1 --2- 172.21.15.5:0/4059990812 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7f737803e4c0 0x7f73780409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:36:05.811 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:00.697+0000 7f73919f0700 1 --2- 172.21.15.5:0/4059990812 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7f737803e4c0 0x7f73780409d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f737c006fd0 tx=0x7f737c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:36:05.811 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:00.702+0000 7f73837fe700 1 -- 172.21.15.5:0/4059990812 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f738800b3e0 con 0x7f738c10a570 2024-05-09T00:36:05.811 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:00.923+0000 7f7394455700 1 -- 172.21.15.5:0/4059990812 --> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi005", "addr": "172.21.15.5", "target": ["mon-mgr", ""]}) -- 0x7f738c109110 con 0x7f737803e4c0 2024-05-09T00:36:05.811 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:05.199+0000 7f73837fe700 1 -- 172.21.15.5:0/4059990812 <== mgr.14120 v2:172.21.15.5:6800/287842266 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+47 (secure 0 0 0) 0x7f738c109110 con 0x7f737803e4c0 2024-05-09T00:36:05.811 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:05.202+0000 7f7394455700 1 -- 172.21.15.5:0/4059990812 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7f737803e4c0 msgr2=0x7f73780409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:36:05.811 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:05.202+0000 7f7394455700 1 --2- 172.21.15.5:0/4059990812 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7f737803e4c0 0x7f73780409d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f737c006fd0 tx=0x7f737c006e40 comp rx=0 tx=0).stop 2024-05-09T00:36:05.811 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:05.202+0000 7f7394455700 1 -- 172.21.15.5:0/4059990812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f738c10a570 msgr2=0x7f738c109cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:36:05.811 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:05.202+0000 7f7394455700 1 --2- 172.21.15.5:0/4059990812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f738c10a570 0x7f738c109cf0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f73880117b0 tx=0x7f7388011b70 comp rx=0 tx=0).stop 2024-05-09T00:36:05.811 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:05.202+0000 7f7394455700 1 -- 172.21.15.5:0/4059990812 shutdown_connections 2024-05-09T00:36:05.811 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:05.202+0000 7f7394455700 1 --2- 172.21.15.5:0/4059990812 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7f737803e4c0 0x7f73780409d0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:36:05.812 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:05.202+0000 7f7394455700 1 --2- 172.21.15.5:0/4059990812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f738c10a570 0x7f738c109cf0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:36:05.812 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:05.202+0000 7f7394455700 1 -- 172.21.15.5:0/4059990812 >> 172.21.15.5:0/4059990812 conn(0x7f738c103ff0 msgr2=0x7f738c104a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:36:05.812 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:05.202+0000 7f7394455700 1 -- 172.21.15.5:0/4059990812 shutdown_connections 2024-05-09T00:36:05.812 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:05.202+0000 7f7394455700 1 -- 172.21.15.5:0/4059990812 wait complete. 2024-05-09T00:36:05.812 INFO:teuthology.orchestra.run.smithi005.stdout:Deploying unmanaged mon service... 2024-05-09T00:36:06.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:06 smithi005 ceph-mon[30365]: from='mgr.14120 172.21.15.5:0/1775107462' entity='mgr.x' 2024-05-09T00:36:06.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:06 smithi005 ceph-mon[30365]: Added host smithi005 2024-05-09T00:36:06.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:06 smithi005 ceph-mon[30365]: from='mgr.14120 172.21.15.5:0/1775107462' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:36:07.694 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:07 smithi005 ceph-mon[30365]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:36:07.694 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:07 smithi005 ceph-mon[30365]: Saving service mon spec with placement count:5 2024-05-09T00:36:07.694 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:07 smithi005 ceph-mon[30365]: from='mgr.14120 172.21.15.5:0/1775107462' entity='mgr.x' 2024-05-09T00:36:07.696 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-09T00:36:07.702 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:06.301+0000 7f55761e1700 1 Processor -- start 2024-05-09T00:36:07.702 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:06.301+0000 7f55761e1700 1 -- start start 2024-05-09T00:36:07.702 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:06.302+0000 7f55761e1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f557010a550 0x7f557010c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:36:07.702 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:06.302+0000 7f55761e1700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f557007ce70 con 0x7f557010a550 2024-05-09T00:36:07.702 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:06.302+0000 7f556f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f557010a550 0x7f557010c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:36:07.702 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:06.302+0000 7f556f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f557010a550 0x7f557010c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58086/0 (socket says 172.21.15.5:58086) 2024-05-09T00:36:07.703 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:06.302+0000 7f556f7fe700 1 -- 172.21.15.5:0/176279566 learned_addr learned my addr 172.21.15.5:0/176279566 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:36:07.703 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:06.302+0000 7f556f7fe700 1 -- 172.21.15.5:0/176279566 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f557010ceb0 con 0x7f557010a550 2024-05-09T00:36:07.703 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:06.302+0000 7f556f7fe700 1 --2- 172.21.15.5:0/176279566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f557010a550 0x7f557010c970 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f55600098a0 tx=0x7f5560009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6deb638f3a333f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:36:07.703 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:06.302+0000 7f556effd700 1 -- 172.21.15.5:0/176279566 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5560004160 con 0x7f557010a550 2024-05-09T00:36:07.703 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:06.302+0000 7f556effd700 1 -- 172.21.15.5:0/176279566 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5560004300 con 0x7f557010a550 2024-05-09T00:36:07.703 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:06.302+0000 7f556effd700 1 -- 172.21.15.5:0/176279566 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5560011af0 con 0x7f557010a550 2024-05-09T00:36:07.703 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:06.303+0000 7f55761e1700 1 -- 172.21.15.5:0/176279566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f557010a550 msgr2=0x7f557010c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:36:07.703 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:06.303+0000 7f55761e1700 1 --2- 172.21.15.5:0/176279566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f557010a550 0x7f557010c970 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f55600098a0 tx=0x7f5560009bb0 comp rx=0 tx=0).stop 2024-05-09T00:36:07.703 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:06.303+0000 7f55761e1700 1 -- 172.21.15.5:0/176279566 shutdown_connections 2024-05-09T00:36:07.703 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:06.303+0000 7f55761e1700 1 --2- 172.21.15.5:0/176279566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f557010a550 0x7f557010c970 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:36:07.703 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:06.303+0000 7f55761e1700 1 -- 172.21.15.5:0/176279566 >> 172.21.15.5:0/176279566 conn(0x7f5570104050 msgr2=0x7f55701064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:36:07.703 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:06.303+0000 7f55761e1700 1 -- 172.21.15.5:0/176279566 shutdown_connections 2024-05-09T00:36:07.703 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:06.303+0000 7f55761e1700 1 -- 172.21.15.5:0/176279566 wait complete. 2024-05-09T00:36:07.703 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:06.304+0000 7f55761e1700 1 Processor -- start 2024-05-09T00:36:07.704 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:06.304+0000 7f55761e1700 1 -- start start 2024-05-09T00:36:07.704 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:06.304+0000 7f55761e1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f557010a550 0x7f557019dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:36:07.704 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:06.304+0000 7f55761e1700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f557010de70 con 0x7f557010a550 2024-05-09T00:36:07.704 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:06.304+0000 7f556f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f557010a550 0x7f557019dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:36:07.704 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:06.304+0000 7f556f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f557010a550 0x7f557019dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58090/0 (socket says 172.21.15.5:58090) 2024-05-09T00:36:07.704 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:06.304+0000 7f556f7fe700 1 -- 172.21.15.5:0/3667424052 learned_addr learned my addr 172.21.15.5:0/3667424052 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:36:07.704 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:06.305+0000 7f556f7fe700 1 -- 172.21.15.5:0/3667424052 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f557019e400 con 0x7f557010a550 2024-05-09T00:36:07.704 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:06.305+0000 7f556f7fe700 1 --2- 172.21.15.5:0/3667424052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f557010a550 0x7f557019dec0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f55600117b0 tx=0x7f5560011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:36:07.704 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:06.305+0000 7f556d7fa700 1 -- 172.21.15.5:0/3667424052 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55600039c0 con 0x7f557010a550 2024-05-09T00:36:07.704 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:06.305+0000 7f556d7fa700 1 -- 172.21.15.5:0/3667424052 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5560003b60 con 0x7f557010a550 2024-05-09T00:36:07.704 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:06.305+0000 7f55761e1700 1 -- 172.21.15.5:0/3667424052 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f557019e690 con 0x7f557010a550 2024-05-09T00:36:07.704 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:06.305+0000 7f556d7fa700 1 -- 172.21.15.5:0/3667424052 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55600196a0 con 0x7f557010a550 2024-05-09T00:36:07.704 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:06.305+0000 7f55761e1700 1 -- 172.21.15.5:0/3667424052 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55701a12a0 con 0x7f557010a550 2024-05-09T00:36:07.705 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:06.306+0000 7f556d7fa700 1 -- 172.21.15.5:0/3667424052 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f5560019910 con 0x7f557010a550 2024-05-09T00:36:07.705 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:06.306+0000 7f556d7fa700 1 --2- 172.21.15.5:0/3667424052 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7f555c03e4c0 0x7f555c0409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:36:07.705 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:06.306+0000 7f556d7fa700 1 -- 172.21.15.5:0/3667424052 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5560054a10 con 0x7f557010a550 2024-05-09T00:36:07.705 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:06.306+0000 7f55761e1700 1 -- 172.21.15.5:0/3667424052 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5570064e20 con 0x7f557010a550 2024-05-09T00:36:07.705 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:06.306+0000 7f5566ffd700 1 --2- 172.21.15.5:0/3667424052 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7f555c03e4c0 0x7f555c0409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:36:07.705 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:06.307+0000 7f5566ffd700 1 --2- 172.21.15.5:0/3667424052 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7f555c03e4c0 0x7f555c0409d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5558006fd0 tx=0x7f5558006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:36:07.705 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:06.312+0000 7f556d7fa700 1 -- 172.21.15.5:0/3667424052 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5560017e90 con 0x7f557010a550 2024-05-09T00:36:07.705 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:06.549+0000 7f55761e1700 1 -- 172.21.15.5:0/3667424052 --> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f55701061f0 con 0x7f555c03e4c0 2024-05-09T00:36:07.705 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:06.554+0000 7f556d7fa700 1 -- 172.21.15.5:0/3667424052 <== mgr.14120 v2:172.21.15.5:6800/287842266 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f55701061f0 con 0x7f555c03e4c0 2024-05-09T00:36:07.705 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:06.556+0000 7f55761e1700 1 -- 172.21.15.5:0/3667424052 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7f555c03e4c0 msgr2=0x7f555c0409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:36:07.705 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:06.556+0000 7f55761e1700 1 --2- 172.21.15.5:0/3667424052 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7f555c03e4c0 0x7f555c0409d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5558006fd0 tx=0x7f5558006e40 comp rx=0 tx=0).stop 2024-05-09T00:36:07.705 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:06.556+0000 7f55761e1700 1 -- 172.21.15.5:0/3667424052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f557010a550 msgr2=0x7f557019dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:36:07.705 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:06.556+0000 7f55761e1700 1 --2- 172.21.15.5:0/3667424052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f557010a550 0x7f557019dec0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f55600117b0 tx=0x7f5560011b70 comp rx=0 tx=0).stop 2024-05-09T00:36:07.705 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:06.557+0000 7f55761e1700 1 -- 172.21.15.5:0/3667424052 shutdown_connections 2024-05-09T00:36:07.705 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:06.557+0000 7f55761e1700 1 --2- 172.21.15.5:0/3667424052 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7f555c03e4c0 0x7f555c0409d0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:36:07.706 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:06.557+0000 7f55761e1700 1 --2- 172.21.15.5:0/3667424052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f557010a550 0x7f557019dec0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:36:07.706 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:06.557+0000 7f55761e1700 1 -- 172.21.15.5:0/3667424052 >> 172.21.15.5:0/3667424052 conn(0x7f5570104050 msgr2=0x7f5570105a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:36:07.706 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:06.557+0000 7f55761e1700 1 -- 172.21.15.5:0/3667424052 shutdown_connections 2024-05-09T00:36:07.706 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:06.557+0000 7f55761e1700 1 -- 172.21.15.5:0/3667424052 wait complete. 2024-05-09T00:36:07.706 INFO:teuthology.orchestra.run.smithi005.stdout:Deploying unmanaged mgr service... 2024-05-09T00:36:09.220 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-09T00:36:09.220 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:08.110+0000 7fcf4955e700 1 Processor -- start 2024-05-09T00:36:09.220 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:08.111+0000 7fcf4955e700 1 -- start start 2024-05-09T00:36:09.220 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:08.111+0000 7fcf4955e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf4410a580 0x7fcf4410c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:36:09.220 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:08.111+0000 7fcf4955e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4407d6a0 con 0x7fcf4410a580 2024-05-09T00:36:09.220 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:08.111+0000 7fcf42ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf4410a580 0x7fcf4410c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:36:09.220 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:08.111+0000 7fcf42ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf4410a580 0x7fcf4410c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58104/0 (socket says 172.21.15.5:58104) 2024-05-09T00:36:09.220 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:08.111+0000 7fcf42ffd700 1 -- 172.21.15.5:0/3520172456 learned_addr learned my addr 172.21.15.5:0/3520172456 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:36:09.220 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:08.111+0000 7fcf42ffd700 1 -- 172.21.15.5:0/3520172456 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf4410cee0 con 0x7fcf4410a580 2024-05-09T00:36:09.220 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:08.112+0000 7fcf42ffd700 1 --2- 172.21.15.5:0/3520172456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf4410a580 0x7fcf4410c9a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcf340098a0 tx=0x7fcf34009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d6f5943bcdf6291 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:36:09.221 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:08.112+0000 7fcf41ffb700 1 -- 172.21.15.5:0/3520172456 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf34004160 con 0x7fcf4410a580 2024-05-09T00:36:09.221 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:08.112+0000 7fcf41ffb700 1 -- 172.21.15.5:0/3520172456 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fcf34004300 con 0x7fcf4410a580 2024-05-09T00:36:09.221 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:08.112+0000 7fcf41ffb700 1 -- 172.21.15.5:0/3520172456 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf34011af0 con 0x7fcf4410a580 2024-05-09T00:36:09.221 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:08.112+0000 7fcf4955e700 1 -- 172.21.15.5:0/3520172456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf4410a580 msgr2=0x7fcf4410c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:36:09.221 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:08.112+0000 7fcf4955e700 1 --2- 172.21.15.5:0/3520172456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf4410a580 0x7fcf4410c9a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcf340098a0 tx=0x7fcf34009bb0 comp rx=0 tx=0).stop 2024-05-09T00:36:09.221 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:08.113+0000 7fcf4955e700 1 -- 172.21.15.5:0/3520172456 shutdown_connections 2024-05-09T00:36:09.221 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:08.113+0000 7fcf4955e700 1 --2- 172.21.15.5:0/3520172456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf4410a580 0x7fcf4410c9a0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:36:09.221 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:08.113+0000 7fcf4955e700 1 -- 172.21.15.5:0/3520172456 >> 172.21.15.5:0/3520172456 conn(0x7fcf44104000 msgr2=0x7fcf44106450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:36:09.221 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:08.113+0000 7fcf4955e700 1 -- 172.21.15.5:0/3520172456 shutdown_connections 2024-05-09T00:36:09.221 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:08.113+0000 7fcf4955e700 1 -- 172.21.15.5:0/3520172456 wait complete. 2024-05-09T00:36:09.221 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:08.113+0000 7fcf4955e700 1 Processor -- start 2024-05-09T00:36:09.221 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:08.114+0000 7fcf4955e700 1 -- start start 2024-05-09T00:36:09.221 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:08.114+0000 7fcf4955e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf4410a580 0x7fcf44109d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:36:09.221 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:08.114+0000 7fcf4955e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4410de40 con 0x7fcf4410a580 2024-05-09T00:36:09.221 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:08.114+0000 7fcf42ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf4410a580 0x7fcf44109d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:36:09.222 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:08.114+0000 7fcf42ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf4410a580 0x7fcf44109d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58108/0 (socket says 172.21.15.5:58108) 2024-05-09T00:36:09.222 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:08.114+0000 7fcf42ffd700 1 -- 172.21.15.5:0/3239960496 learned_addr learned my addr 172.21.15.5:0/3239960496 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:36:09.222 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:08.114+0000 7fcf42ffd700 1 -- 172.21.15.5:0/3239960496 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf4410a240 con 0x7fcf4410a580 2024-05-09T00:36:09.222 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:08.114+0000 7fcf42ffd700 1 --2- 172.21.15.5:0/3239960496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf4410a580 0x7fcf44109d00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcf340117b0 tx=0x7fcf34011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:36:09.222 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:08.115+0000 7fcf2bfff700 1 -- 172.21.15.5:0/3239960496 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf340039c0 con 0x7fcf4410a580 2024-05-09T00:36:09.222 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:08.115+0000 7fcf2bfff700 1 -- 172.21.15.5:0/3239960496 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fcf34003b60 con 0x7fcf4410a580 2024-05-09T00:36:09.222 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:08.115+0000 7fcf2bfff700 1 -- 172.21.15.5:0/3239960496 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf340196a0 con 0x7fcf4410a580 2024-05-09T00:36:09.222 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:08.115+0000 7fcf4955e700 1 -- 172.21.15.5:0/3239960496 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf44108490 con 0x7fcf4410a580 2024-05-09T00:36:09.222 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:08.115+0000 7fcf4955e700 1 -- 172.21.15.5:0/3239960496 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf44108970 con 0x7fcf4410a580 2024-05-09T00:36:09.222 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:08.116+0000 7fcf4955e700 1 -- 172.21.15.5:0/3239960496 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf44064e20 con 0x7fcf4410a580 2024-05-09T00:36:09.222 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:08.116+0000 7fcf2bfff700 1 -- 172.21.15.5:0/3239960496 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fcf34019840 con 0x7fcf4410a580 2024-05-09T00:36:09.222 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:08.116+0000 7fcf2bfff700 1 --2- 172.21.15.5:0/3239960496 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7fcf2c03e4c0 0x7fcf2c0409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:36:09.222 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:08.116+0000 7fcf2bfff700 1 -- 172.21.15.5:0/3239960496 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fcf34053b70 con 0x7fcf4410a580 2024-05-09T00:36:09.223 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:08.116+0000 7fcf427fc700 1 --2- 172.21.15.5:0/3239960496 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7fcf2c03e4c0 0x7fcf2c0409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:36:09.223 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:08.117+0000 7fcf427fc700 1 --2- 172.21.15.5:0/3239960496 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7fcf2c03e4c0 0x7fcf2c0409d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fcf38006fd0 tx=0x7fcf38006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:36:09.223 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:08.121+0000 7fcf2bfff700 1 -- 172.21.15.5:0/3239960496 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcf34017e90 con 0x7fcf4410a580 2024-05-09T00:36:09.223 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:08.358+0000 7fcf4955e700 1 -- 172.21.15.5:0/3239960496 --> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fcf44109110 con 0x7fcf2c03e4c0 2024-05-09T00:36:09.223 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:08.362+0000 7fcf2bfff700 1 -- 172.21.15.5:0/3239960496 <== mgr.14120 v2:172.21.15.5:6800/287842266 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fcf44109110 con 0x7fcf2c03e4c0 2024-05-09T00:36:09.223 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:08.365+0000 7fcf4955e700 1 -- 172.21.15.5:0/3239960496 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7fcf2c03e4c0 msgr2=0x7fcf2c0409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:36:09.223 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:08.365+0000 7fcf4955e700 1 --2- 172.21.15.5:0/3239960496 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7fcf2c03e4c0 0x7fcf2c0409d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fcf38006fd0 tx=0x7fcf38006e40 comp rx=0 tx=0).stop 2024-05-09T00:36:09.223 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:08.365+0000 7fcf4955e700 1 -- 172.21.15.5:0/3239960496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf4410a580 msgr2=0x7fcf44109d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:36:09.223 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:08.365+0000 7fcf4955e700 1 --2- 172.21.15.5:0/3239960496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf4410a580 0x7fcf44109d00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcf340117b0 tx=0x7fcf34011b70 comp rx=0 tx=0).stop 2024-05-09T00:36:09.223 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:08.365+0000 7fcf4955e700 1 -- 172.21.15.5:0/3239960496 shutdown_connections 2024-05-09T00:36:09.223 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:08.365+0000 7fcf4955e700 1 --2- 172.21.15.5:0/3239960496 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7fcf2c03e4c0 0x7fcf2c0409d0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:36:09.223 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:08.365+0000 7fcf4955e700 1 --2- 172.21.15.5:0/3239960496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf4410a580 0x7fcf44109d00 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:36:09.224 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:08.365+0000 7fcf4955e700 1 -- 172.21.15.5:0/3239960496 >> 172.21.15.5:0/3239960496 conn(0x7fcf44104000 msgr2=0x7fcf441059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:36:09.224 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:08.365+0000 7fcf4955e700 1 -- 172.21.15.5:0/3239960496 shutdown_connections 2024-05-09T00:36:09.224 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:08.365+0000 7fcf4955e700 1 -- 172.21.15.5:0/3239960496 wait complete. 2024-05-09T00:36:09.464 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:09 smithi005 ceph-mon[30365]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:36:09.464 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:09 smithi005 ceph-mon[30365]: Saving service mgr spec with placement count:2 2024-05-09T00:36:09.464 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:09 smithi005 ceph-mon[30365]: from='mgr.14120 172.21.15.5:0/1775107462' entity='mgr.x' 2024-05-09T00:36:09.464 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:09 smithi005 ceph-mon[30365]: from='mgr.14120 172.21.15.5:0/1775107462' entity='mgr.x' 2024-05-09T00:36:09.464 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:09 smithi005 ceph-mon[30365]: from='mgr.14120 172.21.15.5:0/1775107462' entity='mgr.x' 2024-05-09T00:36:11.123 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:09.676+0000 7f1d4d434700 1 Processor -- start 2024-05-09T00:36:11.124 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:09.677+0000 7f1d4d434700 1 -- start start 2024-05-09T00:36:11.124 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:09.677+0000 7f1d4d434700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d48108990 0x7f1d48108da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:36:11.124 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:09.677+0000 7f1d4d434700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1d48109390 con 0x7f1d48108990 2024-05-09T00:36:11.124 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:09.677+0000 7f1d46ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d48108990 0x7f1d48108da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:36:11.124 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:09.677+0000 7f1d46ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d48108990 0x7f1d48108da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58116/0 (socket says 172.21.15.5:58116) 2024-05-09T00:36:11.124 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:09.677+0000 7f1d46ffd700 1 -- 172.21.15.5:0/1416660923 learned_addr learned my addr 172.21.15.5:0/1416660923 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:36:11.124 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:09.677+0000 7f1d46ffd700 1 -- 172.21.15.5:0/1416660923 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d48109b60 con 0x7f1d48108990 2024-05-09T00:36:11.124 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:09.678+0000 7f1d46ffd700 1 --2- 172.21.15.5:0/1416660923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d48108990 0x7f1d48108da0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1d300098a0 tx=0x7f1d30009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffdec722f65051b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:36:11.124 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:09.678+0000 7f1d45ffb700 1 -- 172.21.15.5:0/1416660923 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1d30004160 con 0x7f1d48108990 2024-05-09T00:36:11.124 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:09.678+0000 7f1d45ffb700 1 -- 172.21.15.5:0/1416660923 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d30004300 con 0x7f1d48108990 2024-05-09T00:36:11.124 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:09.678+0000 7f1d45ffb700 1 -- 172.21.15.5:0/1416660923 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1d30011a20 con 0x7f1d48108990 2024-05-09T00:36:11.124 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:09.679+0000 7f1d4d434700 1 -- 172.21.15.5:0/1416660923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d48108990 msgr2=0x7f1d48108da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:36:11.124 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:09.679+0000 7f1d4d434700 1 --2- 172.21.15.5:0/1416660923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d48108990 0x7f1d48108da0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1d300098a0 tx=0x7f1d30009bb0 comp rx=0 tx=0).stop 2024-05-09T00:36:11.125 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:09.679+0000 7f1d4d434700 1 -- 172.21.15.5:0/1416660923 shutdown_connections 2024-05-09T00:36:11.125 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:09.679+0000 7f1d4d434700 1 --2- 172.21.15.5:0/1416660923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d48108990 0x7f1d48108da0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:36:11.125 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:09.679+0000 7f1d4d434700 1 -- 172.21.15.5:0/1416660923 >> 172.21.15.5:0/1416660923 conn(0x7f1d48103ff0 msgr2=0x7f1d48106480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:36:11.125 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:09.679+0000 7f1d4d434700 1 -- 172.21.15.5:0/1416660923 shutdown_connections 2024-05-09T00:36:11.125 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:09.679+0000 7f1d4d434700 1 -- 172.21.15.5:0/1416660923 wait complete. 2024-05-09T00:36:11.125 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:09.679+0000 7f1d4d434700 1 Processor -- start 2024-05-09T00:36:11.125 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:09.680+0000 7f1d4d434700 1 -- start start 2024-05-09T00:36:11.125 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:09.680+0000 7f1d4d434700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d48108990 0x7f1d4819ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:36:11.125 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:09.680+0000 7f1d4d434700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1d4810cf10 con 0x7f1d48108990 2024-05-09T00:36:11.125 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:09.680+0000 7f1d46ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d48108990 0x7f1d4819ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:36:11.125 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:09.680+0000 7f1d46ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d48108990 0x7f1d4819ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58128/0 (socket says 172.21.15.5:58128) 2024-05-09T00:36:11.125 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:09.680+0000 7f1d46ffd700 1 -- 172.21.15.5:0/1258540633 learned_addr learned my addr 172.21.15.5:0/1258540633 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:36:11.125 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:09.680+0000 7f1d46ffd700 1 -- 172.21.15.5:0/1258540633 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d4819d370 con 0x7f1d48108990 2024-05-09T00:36:11.125 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:09.681+0000 7f1d46ffd700 1 --2- 172.21.15.5:0/1258540633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d48108990 0x7f1d4819ce30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1d30011a20 tx=0x7f1d30011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:36:11.126 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:09.681+0000 7f1d3ffff700 1 -- 172.21.15.5:0/1258540633 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1d300039c0 con 0x7f1d48108990 2024-05-09T00:36:11.126 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:09.681+0000 7f1d4d434700 1 -- 172.21.15.5:0/1258540633 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d4819d600 con 0x7f1d48108990 2024-05-09T00:36:11.126 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:09.681+0000 7f1d3ffff700 1 -- 172.21.15.5:0/1258540633 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d30003b60 con 0x7f1d48108990 2024-05-09T00:36:11.126 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:09.681+0000 7f1d3ffff700 1 -- 172.21.15.5:0/1258540633 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1d30019700 con 0x7f1d48108990 2024-05-09T00:36:11.126 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:09.681+0000 7f1d4d434700 1 -- 172.21.15.5:0/1258540633 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d481a0210 con 0x7f1d48108990 2024-05-09T00:36:11.126 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:09.682+0000 7f1d3ffff700 1 -- 172.21.15.5:0/1258540633 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f1d30019a00 con 0x7f1d48108990 2024-05-09T00:36:11.126 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:09.682+0000 7f1d4d434700 1 -- 172.21.15.5:0/1258540633 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d48064e20 con 0x7f1d48108990 2024-05-09T00:36:11.126 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:09.682+0000 7f1d3ffff700 1 --2- 172.21.15.5:0/1258540633 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7f1d3403e420 0x7f1d34040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:36:11.126 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:09.682+0000 7f1d3ffff700 1 -- 172.21.15.5:0/1258540633 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1d30054b90 con 0x7f1d48108990 2024-05-09T00:36:11.126 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:09.682+0000 7f1d467fc700 1 --2- 172.21.15.5:0/1258540633 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7f1d3403e420 0x7f1d34040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:36:11.126 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:09.683+0000 7f1d467fc700 1 --2- 172.21.15.5:0/1258540633 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7f1d3403e420 0x7f1d34040930 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1d38006fd0 tx=0x7f1d38006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:36:11.126 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:09.688+0000 7f1d3ffff700 1 -- 172.21.15.5:0/1258540633 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1d30017e90 con 0x7f1d48108990 2024-05-09T00:36:11.127 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:09.915+0000 7f1d4d434700 1 -- 172.21.15.5:0/1258540633 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f1d481a0830 con 0x7f1d48108990 2024-05-09T00:36:11.127 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:09.918+0000 7f1d3ffff700 1 -- 172.21.15.5:0/1258540633 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f1d30019df0 con 0x7f1d48108990 2024-05-09T00:36:11.127 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:09.921+0000 7f1d4d434700 1 -- 172.21.15.5:0/1258540633 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7f1d3403e420 msgr2=0x7f1d34040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:36:11.127 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:09.921+0000 7f1d4d434700 1 --2- 172.21.15.5:0/1258540633 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7f1d3403e420 0x7f1d34040930 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1d38006fd0 tx=0x7f1d38006e40 comp rx=0 tx=0).stop 2024-05-09T00:36:11.127 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:09.921+0000 7f1d4d434700 1 -- 172.21.15.5:0/1258540633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d48108990 msgr2=0x7f1d4819ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:36:11.127 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:09.921+0000 7f1d4d434700 1 --2- 172.21.15.5:0/1258540633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d48108990 0x7f1d4819ce30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1d30011a20 tx=0x7f1d30011b00 comp rx=0 tx=0).stop 2024-05-09T00:36:11.127 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:09.921+0000 7f1d4d434700 1 -- 172.21.15.5:0/1258540633 shutdown_connections 2024-05-09T00:36:11.127 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:09.921+0000 7f1d4d434700 1 --2- 172.21.15.5:0/1258540633 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7f1d3403e420 0x7f1d34040930 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:36:11.127 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:09.921+0000 7f1d4d434700 1 --2- 172.21.15.5:0/1258540633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d48108990 0x7f1d4819ce30 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:36:11.127 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:09.921+0000 7f1d4d434700 1 -- 172.21.15.5:0/1258540633 >> 172.21.15.5:0/1258540633 conn(0x7f1d48103ff0 msgr2=0x7f1d48104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:36:11.127 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:09.921+0000 7f1d4d434700 1 -- 172.21.15.5:0/1258540633 shutdown_connections 2024-05-09T00:36:11.127 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:09.921+0000 7f1d4d434700 1 -- 172.21.15.5:0/1258540633 wait complete. 2024-05-09T00:36:11.247 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:10 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/1258540633' entity='client.admin' 2024-05-09T00:36:12.439 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:12 smithi005 ceph-mon[30365]: from='mgr.14120 172.21.15.5:0/1775107462' entity='mgr.x' 2024-05-09T00:36:12.439 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:12 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/2900865362' entity='client.admin' 2024-05-09T00:36:12.448 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:11.643+0000 7fc5f62d8700 1 Processor -- start 2024-05-09T00:36:12.448 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:11.643+0000 7fc5f62d8700 1 -- start start 2024-05-09T00:36:12.448 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:11.643+0000 7fc5f62d8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5f0108990 0x7fc5f0108da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:36:12.448 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:11.643+0000 7fc5f62d8700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f0109390 con 0x7fc5f0108990 2024-05-09T00:36:12.448 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:11.644+0000 7fc5effff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5f0108990 0x7fc5f0108da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:36:12.448 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:11.644+0000 7fc5effff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5f0108990 0x7fc5f0108da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58142/0 (socket says 172.21.15.5:58142) 2024-05-09T00:36:12.448 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:11.644+0000 7fc5effff700 1 -- 172.21.15.5:0/1852784336 learned_addr learned my addr 172.21.15.5:0/1852784336 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:36:12.448 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:11.644+0000 7fc5effff700 1 -- 172.21.15.5:0/1852784336 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5f0109b60 con 0x7fc5f0108990 2024-05-09T00:36:12.448 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:11.644+0000 7fc5effff700 1 --2- 172.21.15.5:0/1852784336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5f0108990 0x7fc5f0108da0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc5e00098a0 tx=0x7fc5e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e3bcbf2d6a7b480 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:36:12.449 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:11.645+0000 7fc5ef7fe700 1 -- 172.21.15.5:0/1852784336 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc5e0004160 con 0x7fc5f0108990 2024-05-09T00:36:12.449 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:11.645+0000 7fc5ef7fe700 1 -- 172.21.15.5:0/1852784336 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5e0004300 con 0x7fc5f0108990 2024-05-09T00:36:12.449 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:11.645+0000 7fc5ef7fe700 1 -- 172.21.15.5:0/1852784336 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc5e0011a20 con 0x7fc5f0108990 2024-05-09T00:36:12.449 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:11.645+0000 7fc5f62d8700 1 -- 172.21.15.5:0/1852784336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5f0108990 msgr2=0x7fc5f0108da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:36:12.449 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:11.645+0000 7fc5f62d8700 1 --2- 172.21.15.5:0/1852784336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5f0108990 0x7fc5f0108da0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc5e00098a0 tx=0x7fc5e0009bb0 comp rx=0 tx=0).stop 2024-05-09T00:36:12.449 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:11.645+0000 7fc5f62d8700 1 -- 172.21.15.5:0/1852784336 shutdown_connections 2024-05-09T00:36:12.449 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:11.645+0000 7fc5f62d8700 1 --2- 172.21.15.5:0/1852784336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5f0108990 0x7fc5f0108da0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:36:12.449 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:11.645+0000 7fc5f62d8700 1 -- 172.21.15.5:0/1852784336 >> 172.21.15.5:0/1852784336 conn(0x7fc5f0104030 msgr2=0x7fc5f0106480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:36:12.449 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:11.646+0000 7fc5f62d8700 1 -- 172.21.15.5:0/1852784336 shutdown_connections 2024-05-09T00:36:12.449 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:11.646+0000 7fc5f62d8700 1 -- 172.21.15.5:0/1852784336 wait complete. 2024-05-09T00:36:12.449 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:11.646+0000 7fc5f62d8700 1 Processor -- start 2024-05-09T00:36:12.449 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:11.646+0000 7fc5f62d8700 1 -- start start 2024-05-09T00:36:12.449 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:11.646+0000 7fc5f62d8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5f0108990 0x7fc5f019dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:36:12.450 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:11.647+0000 7fc5f62d8700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f010cf10 con 0x7fc5f0108990 2024-05-09T00:36:12.450 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:11.647+0000 7fc5effff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5f0108990 0x7fc5f019dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:36:12.450 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:11.647+0000 7fc5effff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5f0108990 0x7fc5f019dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58152/0 (socket says 172.21.15.5:58152) 2024-05-09T00:36:12.450 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:11.647+0000 7fc5effff700 1 -- 172.21.15.5:0/2900865362 learned_addr learned my addr 172.21.15.5:0/2900865362 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:36:12.450 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:11.647+0000 7fc5effff700 1 -- 172.21.15.5:0/2900865362 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5f019e400 con 0x7fc5f0108990 2024-05-09T00:36:12.450 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:11.647+0000 7fc5effff700 1 --2- 172.21.15.5:0/2900865362 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5f0108990 0x7fc5f019dec0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc5e0011a20 tx=0x7fc5e0011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:36:12.451 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:11.647+0000 7fc5edffb700 1 -- 172.21.15.5:0/2900865362 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc5e00039c0 con 0x7fc5f0108990 2024-05-09T00:36:12.451 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:11.647+0000 7fc5edffb700 1 -- 172.21.15.5:0/2900865362 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5e0003b60 con 0x7fc5f0108990 2024-05-09T00:36:12.451 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:11.647+0000 7fc5f62d8700 1 -- 172.21.15.5:0/2900865362 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5f019e690 con 0x7fc5f0108990 2024-05-09T00:36:12.451 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:11.647+0000 7fc5edffb700 1 -- 172.21.15.5:0/2900865362 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc5e0019700 con 0x7fc5f0108990 2024-05-09T00:36:12.451 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:11.647+0000 7fc5f62d8700 1 -- 172.21.15.5:0/2900865362 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5f01a12a0 con 0x7fc5f0108990 2024-05-09T00:36:12.451 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:11.648+0000 7fc5f62d8700 1 -- 172.21.15.5:0/2900865362 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5f0064e20 con 0x7fc5f0108990 2024-05-09T00:36:12.451 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:11.648+0000 7fc5edffb700 1 -- 172.21.15.5:0/2900865362 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fc5e00199b0 con 0x7fc5f0108990 2024-05-09T00:36:12.452 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:11.649+0000 7fc5edffb700 1 --2- 172.21.15.5:0/2900865362 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7fc5dc03e470 0x7fc5dc040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:36:12.452 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:11.649+0000 7fc5edffb700 1 -- 172.21.15.5:0/2900865362 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc5e0053b50 con 0x7fc5f0108990 2024-05-09T00:36:12.452 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:11.649+0000 7fc5e7fff700 1 --2- 172.21.15.5:0/2900865362 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7fc5dc03e470 0x7fc5dc040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:36:12.452 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:11.649+0000 7fc5e7fff700 1 --2- 172.21.15.5:0/2900865362 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7fc5dc03e470 0x7fc5dc040980 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8006fd0 tx=0x7fc5d8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:36:12.452 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:11.654+0000 7fc5edffb700 1 -- 172.21.15.5:0/2900865362 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc5e000b3e0 con 0x7fc5f0108990 2024-05-09T00:36:12.452 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:11.867+0000 7fc5f62d8700 1 -- 172.21.15.5:0/2900865362 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fc5f002e8d0 con 0x7fc5f0108990 2024-05-09T00:36:12.452 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:11.870+0000 7fc5edffb700 1 -- 172.21.15.5:0/2900865362 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fc5e0019df0 con 0x7fc5f0108990 2024-05-09T00:36:12.452 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:11.873+0000 7fc5f62d8700 1 -- 172.21.15.5:0/2900865362 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7fc5dc03e470 msgr2=0x7fc5dc040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:36:12.452 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:11.873+0000 7fc5f62d8700 1 --2- 172.21.15.5:0/2900865362 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7fc5dc03e470 0x7fc5dc040980 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8006fd0 tx=0x7fc5d8006e40 comp rx=0 tx=0).stop 2024-05-09T00:36:12.453 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:11.873+0000 7fc5f62d8700 1 -- 172.21.15.5:0/2900865362 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5f0108990 msgr2=0x7fc5f019dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:36:12.453 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:11.873+0000 7fc5f62d8700 1 --2- 172.21.15.5:0/2900865362 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5f0108990 0x7fc5f019dec0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc5e0011a20 tx=0x7fc5e0011b00 comp rx=0 tx=0).stop 2024-05-09T00:36:12.453 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:11.873+0000 7fc5f62d8700 1 -- 172.21.15.5:0/2900865362 shutdown_connections 2024-05-09T00:36:12.453 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:11.873+0000 7fc5f62d8700 1 --2- 172.21.15.5:0/2900865362 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7fc5dc03e470 0x7fc5dc040980 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:36:12.453 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:11.873+0000 7fc5f62d8700 1 --2- 172.21.15.5:0/2900865362 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5f0108990 0x7fc5f019dec0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:36:12.453 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:11.873+0000 7fc5f62d8700 1 -- 172.21.15.5:0/2900865362 >> 172.21.15.5:0/2900865362 conn(0x7fc5f0104030 msgr2=0x7fc5f0194450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:36:12.453 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:11.873+0000 7fc5f62d8700 1 -- 172.21.15.5:0/2900865362 shutdown_connections 2024-05-09T00:36:12.453 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:11.873+0000 7fc5f62d8700 1 -- 172.21.15.5:0/2900865362 wait complete. 2024-05-09T00:36:12.453 INFO:teuthology.orchestra.run.smithi005.stdout:Enabling the dashboard module... 2024-05-09T00:36:13.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:13 smithi005 ceph-mon[30365]: from='mgr.14120 172.21.15.5:0/1775107462' entity='mgr.x' 2024-05-09T00:36:13.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:13 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/3350431218' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-09T00:36:14.478 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:36:14 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: ignoring --setuser ceph since I am not root 2024-05-09T00:36:14.478 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:36:14 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: ignoring --setgroup ceph since I am not root 2024-05-09T00:36:14.478 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:36:14 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:36:14.360+0000 7f76b1c31200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T00:36:14.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:14 smithi005 ceph-mon[30365]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:36:14.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:14 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/3350431218' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-09T00:36:14.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:14 smithi005 ceph-mon[30365]: mgrmap e9: x(active, since 21s) 2024-05-09T00:36:14.792 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:36:14 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:36:14.558+0000 7f76b1c31200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T00:36:14.792 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:36:14 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:36:14.666+0000 7f76b1c31200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T00:36:14.802 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:12.925+0000 7f10df671700 1 Processor -- start 2024-05-09T00:36:14.802 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:12.925+0000 7f10df671700 1 -- start start 2024-05-09T00:36:14.802 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:12.925+0000 7f10df671700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10d8109f00 0x7f10d810a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:36:14.802 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:12.925+0000 7f10df671700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f10d810a940 con 0x7f10d8109f00 2024-05-09T00:36:14.802 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:12.926+0000 7f10dd40d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10d8109f00 0x7f10d810a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:36:14.802 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:12.926+0000 7f10dd40d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10d8109f00 0x7f10d810a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58160/0 (socket says 172.21.15.5:58160) 2024-05-09T00:36:14.802 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:12.926+0000 7f10dd40d700 1 -- 172.21.15.5:0/2325657993 learned_addr learned my addr 172.21.15.5:0/2325657993 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:36:14.802 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:12.926+0000 7f10dd40d700 1 -- 172.21.15.5:0/2325657993 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10d8108c10 con 0x7f10d8109f00 2024-05-09T00:36:14.802 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:12.926+0000 7f10dd40d700 1 --2- 172.21.15.5:0/2325657993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10d8109f00 0x7f10d810a330 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f10cc0098a0 tx=0x7f10cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da0cb9340a76952f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:36:14.803 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:12.927+0000 7f10cbfff700 1 -- 172.21.15.5:0/2325657993 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10cc004160 con 0x7f10d8109f00 2024-05-09T00:36:14.803 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:12.927+0000 7f10cbfff700 1 -- 172.21.15.5:0/2325657993 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10cc004300 con 0x7f10d8109f00 2024-05-09T00:36:14.803 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:12.927+0000 7f10cbfff700 1 -- 172.21.15.5:0/2325657993 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10cc011a20 con 0x7f10d8109f00 2024-05-09T00:36:14.803 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:12.928+0000 7f10df671700 1 -- 172.21.15.5:0/2325657993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10d8109f00 msgr2=0x7f10d810a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:36:14.803 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:12.928+0000 7f10df671700 1 --2- 172.21.15.5:0/2325657993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10d8109f00 0x7f10d810a330 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f10cc0098a0 tx=0x7f10cc009bb0 comp rx=0 tx=0).stop 2024-05-09T00:36:14.803 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:12.928+0000 7f10df671700 1 -- 172.21.15.5:0/2325657993 shutdown_connections 2024-05-09T00:36:14.803 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:12.928+0000 7f10df671700 1 --2- 172.21.15.5:0/2325657993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10d8109f00 0x7f10d810a330 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:36:14.803 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:12.928+0000 7f10df671700 1 -- 172.21.15.5:0/2325657993 >> 172.21.15.5:0/2325657993 conn(0x7f10d81037d0 msgr2=0x7f10d8105c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:36:14.803 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:12.928+0000 7f10df671700 1 -- 172.21.15.5:0/2325657993 shutdown_connections 2024-05-09T00:36:14.803 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:12.928+0000 7f10df671700 1 -- 172.21.15.5:0/2325657993 wait complete. 2024-05-09T00:36:14.803 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:12.929+0000 7f10df671700 1 Processor -- start 2024-05-09T00:36:14.803 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:12.929+0000 7f10df671700 1 -- start start 2024-05-09T00:36:14.803 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:12.929+0000 7f10df671700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10d8109f00 0x7f10d819cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:36:14.804 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:12.929+0000 7f10df671700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f10d810cfd0 con 0x7f10d8109f00 2024-05-09T00:36:14.804 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:12.929+0000 7f10dd40d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10d8109f00 0x7f10d819cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:36:14.804 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:12.929+0000 7f10dd40d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10d8109f00 0x7f10d819cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58162/0 (socket says 172.21.15.5:58162) 2024-05-09T00:36:14.804 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:12.929+0000 7f10dd40d700 1 -- 172.21.15.5:0/3350431218 learned_addr learned my addr 172.21.15.5:0/3350431218 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:36:14.804 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:12.929+0000 7f10dd40d700 1 -- 172.21.15.5:0/3350431218 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10d819d2d0 con 0x7f10d8109f00 2024-05-09T00:36:14.804 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:12.929+0000 7f10dd40d700 1 --2- 172.21.15.5:0/3350431218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10d8109f00 0x7f10d819cd90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f10cc004660 tx=0x7f10cc011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:36:14.804 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:12.930+0000 7f10ca7fc700 1 -- 172.21.15.5:0/3350431218 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10cc003a60 con 0x7f10d8109f00 2024-05-09T00:36:14.804 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:12.930+0000 7f10ca7fc700 1 -- 172.21.15.5:0/3350431218 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10cc01b8f0 con 0x7f10d8109f00 2024-05-09T00:36:14.804 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:12.930+0000 7f10df671700 1 -- 172.21.15.5:0/3350431218 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10d819d560 con 0x7f10d8109f00 2024-05-09T00:36:14.804 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:12.930+0000 7f10ca7fc700 1 -- 172.21.15.5:0/3350431218 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10cc023890 con 0x7f10d8109f00 2024-05-09T00:36:14.804 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:12.930+0000 7f10df671700 1 -- 172.21.15.5:0/3350431218 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10d8107b00 con 0x7f10d8109f00 2024-05-09T00:36:14.804 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:12.931+0000 7f10df671700 1 -- 172.21.15.5:0/3350431218 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10d8064e20 con 0x7f10d8109f00 2024-05-09T00:36:14.804 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:12.931+0000 7f10ca7fc700 1 -- 172.21.15.5:0/3350431218 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f10cc02c3f0 con 0x7f10d8109f00 2024-05-09T00:36:14.804 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:12.931+0000 7f10ca7fc700 1 --2- 172.21.15.5:0/3350431218 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7f10c4040630 0x7f10c4042b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:36:14.805 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:12.931+0000 7f10ca7fc700 1 -- 172.21.15.5:0/3350431218 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f10cc053f40 con 0x7f10d8109f00 2024-05-09T00:36:14.805 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:12.932+0000 7f10dcc0c700 1 --2- 172.21.15.5:0/3350431218 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7f10c4040630 0x7f10c4042b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:36:14.805 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:12.932+0000 7f10dcc0c700 1 --2- 172.21.15.5:0/3350431218 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7f10c4040630 0x7f10c4042b40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f10d4006fd0 tx=0x7f10d4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:36:14.805 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:12.937+0000 7f10ca7fc700 1 -- 172.21.15.5:0/3350431218 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f10cc01f030 con 0x7f10d8109f00 2024-05-09T00:36:14.805 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:13.221+0000 7f10df671700 1 -- 172.21.15.5:0/3350431218 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f10d8108110 con 0x7f10d8109f00 2024-05-09T00:36:14.805 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:14.137+0000 7f10ca7fc700 1 -- 172.21.15.5:0/3350431218 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f10cc00f2e0 con 0x7f10d8109f00 2024-05-09T00:36:14.805 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:14.140+0000 7f10ca7fc700 1 -- 172.21.15.5:0/3350431218 <== mon.0 v2:172.21.15.5:3300/0 8 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f10cc02c6e0 con 0x7f10d8109f00 2024-05-09T00:36:14.805 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:14.141+0000 7f10df671700 1 -- 172.21.15.5:0/3350431218 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7f10c4040630 msgr2=0x7f10c4042b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:36:14.805 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:14.141+0000 7f10df671700 1 --2- 172.21.15.5:0/3350431218 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7f10c4040630 0x7f10c4042b40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f10d4006fd0 tx=0x7f10d4006e40 comp rx=0 tx=0).stop 2024-05-09T00:36:14.805 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:14.141+0000 7f10df671700 1 -- 172.21.15.5:0/3350431218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10d8109f00 msgr2=0x7f10d819cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:36:14.805 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:14.141+0000 7f10df671700 1 --2- 172.21.15.5:0/3350431218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10d8109f00 0x7f10d819cd90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f10cc004660 tx=0x7f10cc011aa0 comp rx=0 tx=0).stop 2024-05-09T00:36:14.805 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:14.141+0000 7f10df671700 1 -- 172.21.15.5:0/3350431218 shutdown_connections 2024-05-09T00:36:14.805 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:14.141+0000 7f10df671700 1 --2- 172.21.15.5:0/3350431218 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7f10c4040630 0x7f10c4042b40 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:36:14.805 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:14.141+0000 7f10df671700 1 --2- 172.21.15.5:0/3350431218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10d8109f00 0x7f10d819cd90 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:36:14.806 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:14.141+0000 7f10df671700 1 -- 172.21.15.5:0/3350431218 >> 172.21.15.5:0/3350431218 conn(0x7f10d81037d0 msgr2=0x7f10d8105110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:36:14.806 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:14.141+0000 7f10df671700 1 -- 172.21.15.5:0/3350431218 shutdown_connections 2024-05-09T00:36:14.806 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:14.141+0000 7f10df671700 1 -- 172.21.15.5:0/3350431218 wait complete. 2024-05-09T00:36:15.045 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:36:14 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:36:14.996+0000 7f76b1c31200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T00:36:15.451 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:36:15 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:36:15.091+0000 7f76b1c31200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T00:36:15.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:15 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/593866812' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-09T00:36:16.094 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2024-05-09T00:36:16.101 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-05-09T00:36:16.101 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-09T00:36:16.101 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-09T00:36:16.102 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-09T00:36:16.102 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2024-05-09T00:36:16.102 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:15.242+0000 7fbd106e1700 1 Processor -- start 2024-05-09T00:36:16.102 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:15.242+0000 7fbd106e1700 1 -- start start 2024-05-09T00:36:16.102 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:15.242+0000 7fbd106e1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd0810a550 0x7fbd0810c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:36:16.102 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:15.242+0000 7fbd106e1700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0807ce70 con 0x7fbd0810a550 2024-05-09T00:36:16.102 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:15.243+0000 7fbd0e47d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd0810a550 0x7fbd0810c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:36:16.102 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:15.243+0000 7fbd0e47d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd0810a550 0x7fbd0810c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42806/0 (socket says 172.21.15.5:42806) 2024-05-09T00:36:16.102 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:15.243+0000 7fbd0e47d700 1 -- 172.21.15.5:0/1839180050 learned_addr learned my addr 172.21.15.5:0/1839180050 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:36:16.102 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:15.243+0000 7fbd0e47d700 1 -- 172.21.15.5:0/1839180050 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd0810ceb0 con 0x7fbd0810a550 2024-05-09T00:36:16.102 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:15.243+0000 7fbd0e47d700 1 --2- 172.21.15.5:0/1839180050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd0810a550 0x7fbd0810c970 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbd040098a0 tx=0x7fbd04009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b7a4e12075138e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:36:16.103 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:15.243+0000 7fbd0d47b700 1 -- 172.21.15.5:0/1839180050 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd04004160 con 0x7fbd0810a550 2024-05-09T00:36:16.103 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:15.243+0000 7fbd0d47b700 1 -- 172.21.15.5:0/1839180050 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd04004300 con 0x7fbd0810a550 2024-05-09T00:36:16.103 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:15.243+0000 7fbd0d47b700 1 -- 172.21.15.5:0/1839180050 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd04011a20 con 0x7fbd0810a550 2024-05-09T00:36:16.103 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:15.244+0000 7fbd106e1700 1 -- 172.21.15.5:0/1839180050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd0810a550 msgr2=0x7fbd0810c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:36:16.103 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:15.244+0000 7fbd106e1700 1 --2- 172.21.15.5:0/1839180050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd0810a550 0x7fbd0810c970 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbd040098a0 tx=0x7fbd04009bb0 comp rx=0 tx=0).stop 2024-05-09T00:36:16.103 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:15.244+0000 7fbd106e1700 1 -- 172.21.15.5:0/1839180050 shutdown_connections 2024-05-09T00:36:16.103 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:15.244+0000 7fbd106e1700 1 --2- 172.21.15.5:0/1839180050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd0810a550 0x7fbd0810c970 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:36:16.103 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:15.244+0000 7fbd106e1700 1 -- 172.21.15.5:0/1839180050 >> 172.21.15.5:0/1839180050 conn(0x7fbd08104050 msgr2=0x7fbd081064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:36:16.103 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:15.244+0000 7fbd106e1700 1 -- 172.21.15.5:0/1839180050 shutdown_connections 2024-05-09T00:36:16.103 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:15.244+0000 7fbd106e1700 1 -- 172.21.15.5:0/1839180050 wait complete. 2024-05-09T00:36:16.103 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:15.245+0000 7fbd106e1700 1 Processor -- start 2024-05-09T00:36:16.103 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:15.245+0000 7fbd106e1700 1 -- start start 2024-05-09T00:36:16.103 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:15.245+0000 7fbd106e1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd0810a550 0x7fbd0819de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:36:16.104 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:15.245+0000 7fbd106e1700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0810deb0 con 0x7fbd0810a550 2024-05-09T00:36:16.104 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:15.246+0000 7fbd0e47d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd0810a550 0x7fbd0819de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:36:16.104 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:15.246+0000 7fbd0e47d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd0810a550 0x7fbd0819de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42812/0 (socket says 172.21.15.5:42812) 2024-05-09T00:36:16.104 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:15.246+0000 7fbd0e47d700 1 -- 172.21.15.5:0/593866812 learned_addr learned my addr 172.21.15.5:0/593866812 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:36:16.104 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:15.246+0000 7fbd0e47d700 1 -- 172.21.15.5:0/593866812 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd0819e3b0 con 0x7fbd0810a550 2024-05-09T00:36:16.104 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:15.246+0000 7fbd0e47d700 1 --2- 172.21.15.5:0/593866812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd0810a550 0x7fbd0819de70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbd04009520 tx=0x7fbd04011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:36:16.104 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:15.246+0000 7fbcff7fe700 1 -- 172.21.15.5:0/593866812 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd040039e0 con 0x7fbd0810a550 2024-05-09T00:36:16.104 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:15.246+0000 7fbcff7fe700 1 -- 172.21.15.5:0/593866812 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd0401b8f0 con 0x7fbd0810a550 2024-05-09T00:36:16.104 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:15.246+0000 7fbd106e1700 1 -- 172.21.15.5:0/593866812 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd0819e640 con 0x7fbd0810a550 2024-05-09T00:36:16.104 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:15.246+0000 7fbcff7fe700 1 -- 172.21.15.5:0/593866812 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd040197d0 con 0x7fbd0810a550 2024-05-09T00:36:16.104 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:15.246+0000 7fbd106e1700 1 -- 172.21.15.5:0/593866812 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd081a1250 con 0x7fbd0810a550 2024-05-09T00:36:16.104 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:15.247+0000 7fbd106e1700 1 -- 172.21.15.5:0/593866812 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd08064e20 con 0x7fbd0810a550 2024-05-09T00:36:16.104 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:15.247+0000 7fbcff7fe700 1 -- 172.21.15.5:0/593866812 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7fbd04019a90 con 0x7fbd0810a550 2024-05-09T00:36:16.105 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:15.248+0000 7fbcff7fe700 1 --2- 172.21.15.5:0/593866812 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7fbcf403e4c0 0x7fbcf40409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:36:16.105 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:15.248+0000 7fbcff7fe700 1 -- 172.21.15.5:0/593866812 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbd04053fe0 con 0x7fbd0810a550 2024-05-09T00:36:16.105 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:15.248+0000 7fbd0dc7c700 1 -- 172.21.15.5:0/593866812 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7fbcf403e4c0 msgr2=0x7fbcf40409d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/287842266 2024-05-09T00:36:16.105 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:15.248+0000 7fbd0dc7c700 1 --2- 172.21.15.5:0/593866812 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7fbcf403e4c0 0x7fbcf40409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T00:36:16.105 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:15.253+0000 7fbcff7fe700 1 -- 172.21.15.5:0/593866812 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbd0401f030 con 0x7fbd0810a550 2024-05-09T00:36:16.105 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:15.449+0000 7fbd0dc7c700 1 -- 172.21.15.5:0/593866812 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7fbcf403e4c0 msgr2=0x7fbcf40409d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/287842266 2024-05-09T00:36:16.105 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:15.449+0000 7fbd0dc7c700 1 --2- 172.21.15.5:0/593866812 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7fbcf403e4c0 0x7fbcf40409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T00:36:16.105 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:15.572+0000 7fbd106e1700 1 -- 172.21.15.5:0/593866812 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fbd081a1420 con 0x7fbd0810a550 2024-05-09T00:36:16.105 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:15.572+0000 7fbcff7fe700 1 -- 172.21.15.5:0/593866812 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fbd0400e260 con 0x7fbd0810a550 2024-05-09T00:36:16.105 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:15.575+0000 7fbd106e1700 1 -- 172.21.15.5:0/593866812 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7fbcf403e4c0 msgr2=0x7fbcf40409d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T00:36:16.105 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:15.575+0000 7fbd106e1700 1 --2- 172.21.15.5:0/593866812 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7fbcf403e4c0 0x7fbcf40409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:36:16.105 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:15.575+0000 7fbd106e1700 1 -- 172.21.15.5:0/593866812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd0810a550 msgr2=0x7fbd0819de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:36:16.105 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:15.575+0000 7fbd106e1700 1 --2- 172.21.15.5:0/593866812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd0810a550 0x7fbd0819de70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbd04009520 tx=0x7fbd04011aa0 comp rx=0 tx=0).stop 2024-05-09T00:36:16.106 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:15.575+0000 7fbd106e1700 1 -- 172.21.15.5:0/593866812 shutdown_connections 2024-05-09T00:36:16.106 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:15.575+0000 7fbd106e1700 1 --2- 172.21.15.5:0/593866812 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7fbcf403e4c0 0x7fbcf40409d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:36:16.106 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:15.575+0000 7fbd106e1700 1 --2- 172.21.15.5:0/593866812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd0810a550 0x7fbd0819de70 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:36:16.106 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:15.575+0000 7fbd106e1700 1 -- 172.21.15.5:0/593866812 >> 172.21.15.5:0/593866812 conn(0x7fbd08104050 msgr2=0x7fbd081059f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:36:16.106 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:15.575+0000 7fbd106e1700 1 -- 172.21.15.5:0/593866812 shutdown_connections 2024-05-09T00:36:16.106 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:15.575+0000 7fbd106e1700 1 -- 172.21.15.5:0/593866812 wait complete. 2024-05-09T00:36:16.106 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for the mgr to restart... 2024-05-09T00:36:16.106 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for mgr epoch 9... 2024-05-09T00:36:16.541 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:36:16 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:36:16.388+0000 7f76b1c31200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T00:36:16.912 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:36:16 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:36:16.573+0000 7f76b1c31200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T00:36:16.912 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:36:16 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:36:16.758+0000 7f76b1c31200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T00:36:17.168 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:36:17 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:36:17.067+0000 7f76b1c31200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T00:36:17.541 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:36:17 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:36:17.166+0000 7f76b1c31200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T00:36:17.541 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:36:17 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:36:17.379+0000 7f76b1c31200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T00:36:18.041 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:36:17 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:36:17.585+0000 7f76b1c31200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T00:36:18.435 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:36:18 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:36:18.223+0000 7f76b1c31200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T00:36:18.791 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:36:18 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:36:18.484+0000 7f76b1c31200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-09T00:36:19.291 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:36:18 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:36:18.940+0000 7f76b1c31200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T00:36:19.291 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:36:19 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:36:19.044+0000 7f76b1c31200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T00:36:19.715 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:36:19 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:36:19.434+0000 7f76b1c31200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T00:36:20.041 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:36:19 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:36:19.814+0000 7f76b1c31200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T00:36:20.041 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:36:19 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:36:19.914+0000 7f76b1c31200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T00:36:20.541 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:36:20 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:36:20.176+0000 7f76b1c31200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T00:36:20.541 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:36:20 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:36:20.319+0000 7f76b1c31200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T00:36:20.541 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:36:20 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:36:20.423+0000 7f76b1c31200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T00:36:21.041 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:36:20 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:36:20.763+0000 7f76b1c31200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T00:36:21.041 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:36:20 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:36:20.862+0000 7f76b1c31200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T00:36:21.791 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:36:21 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:36:21.506+0000 7f76b1c31200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T00:36:21.791 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:36:21 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:36:21.740+0000 7f76b1c31200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T00:36:22.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:21 smithi005 ceph-mon[30365]: Active manager daemon x restarted 2024-05-09T00:36:22.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:21 smithi005 ceph-mon[30365]: Activating manager daemon x 2024-05-09T00:36:22.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:21 smithi005 ceph-mon[30365]: osdmap e3: 0 total, 0 up, 0 in 2024-05-09T00:36:22.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:21 smithi005 ceph-mon[30365]: mgrmap e10: x(active, starting, since 0.00408752s) 2024-05-09T00:36:22.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:21 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T00:36:22.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:21 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-09T00:36:22.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:21 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-09T00:36:22.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:21 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-09T00:36:22.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:21 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-09T00:36:22.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:21 smithi005 ceph-mon[30365]: Manager daemon x is now available 2024-05-09T00:36:22.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:21 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:36:23.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:22 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-09T00:36:23.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:22 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-09T00:36:23.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:22 smithi005 ceph-mon[30365]: mgrmap e11: x(active, since 1.00696s) 2024-05-09T00:36:23.352 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2024-05-09T00:36:23.353 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-05-09T00:36:23.353 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-09T00:36:23.353 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2024-05-09T00:36:23.353 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:16.703+0000 7ff10f178700 1 Processor -- start 2024-05-09T00:36:23.353 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:16.703+0000 7ff10f178700 1 -- start start 2024-05-09T00:36:23.353 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:16.703+0000 7ff10f178700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff108108970 0x7ff108108d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:36:23.353 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:16.703+0000 7ff10f178700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff108109370 con 0x7ff108108970 2024-05-09T00:36:23.353 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:16.704+0000 7ff10cf14700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff108108970 0x7ff108108d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:36:23.353 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:16.704+0000 7ff10cf14700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff108108970 0x7ff108108d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42822/0 (socket says 172.21.15.5:42822) 2024-05-09T00:36:23.353 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:16.704+0000 7ff10cf14700 1 -- 172.21.15.5:0/82875361 learned_addr learned my addr 172.21.15.5:0/82875361 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:36:23.353 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:16.704+0000 7ff10cf14700 1 -- 172.21.15.5:0/82875361 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff108109b40 con 0x7ff108108970 2024-05-09T00:36:23.353 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:16.704+0000 7ff10cf14700 1 --2- 172.21.15.5:0/82875361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff108108970 0x7ff108108d80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc0098a0 tx=0x7ff0fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=368654650004cf6c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:36:23.354 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:16.704+0000 7ff1077fe700 1 -- 172.21.15.5:0/82875361 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0fc004160 con 0x7ff108108970 2024-05-09T00:36:23.354 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:16.704+0000 7ff1077fe700 1 -- 172.21.15.5:0/82875361 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0fc004300 con 0x7ff108108970 2024-05-09T00:36:23.354 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:16.704+0000 7ff1077fe700 1 -- 172.21.15.5:0/82875361 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0fc011a20 con 0x7ff108108970 2024-05-09T00:36:23.354 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:16.705+0000 7ff10f178700 1 -- 172.21.15.5:0/82875361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff108108970 msgr2=0x7ff108108d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:36:23.354 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:16.705+0000 7ff10f178700 1 --2- 172.21.15.5:0/82875361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff108108970 0x7ff108108d80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc0098a0 tx=0x7ff0fc009bb0 comp rx=0 tx=0).stop 2024-05-09T00:36:23.354 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:16.705+0000 7ff10f178700 1 -- 172.21.15.5:0/82875361 shutdown_connections 2024-05-09T00:36:23.354 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:16.705+0000 7ff10f178700 1 --2- 172.21.15.5:0/82875361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff108108970 0x7ff108108d80 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:36:23.354 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:16.705+0000 7ff10f178700 1 -- 172.21.15.5:0/82875361 >> 172.21.15.5:0/82875361 conn(0x7ff108104050 msgr2=0x7ff1081064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:36:23.354 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:16.705+0000 7ff10f178700 1 -- 172.21.15.5:0/82875361 shutdown_connections 2024-05-09T00:36:23.354 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:16.705+0000 7ff10f178700 1 -- 172.21.15.5:0/82875361 wait complete. 2024-05-09T00:36:23.354 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:16.706+0000 7ff10f178700 1 Processor -- start 2024-05-09T00:36:23.354 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:16.706+0000 7ff10f178700 1 -- start start 2024-05-09T00:36:23.355 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:16.706+0000 7ff10f178700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff108108970 0x7ff10819def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:36:23.355 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:16.706+0000 7ff10f178700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff10810cef0 con 0x7ff108108970 2024-05-09T00:36:23.355 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:16.706+0000 7ff10cf14700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff108108970 0x7ff10819def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:36:23.355 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:16.706+0000 7ff10cf14700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff108108970 0x7ff10819def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42836/0 (socket says 172.21.15.5:42836) 2024-05-09T00:36:23.355 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:16.706+0000 7ff10cf14700 1 -- 172.21.15.5:0/1205162331 learned_addr learned my addr 172.21.15.5:0/1205162331 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:36:23.355 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:16.707+0000 7ff10cf14700 1 -- 172.21.15.5:0/1205162331 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff10819e430 con 0x7ff108108970 2024-05-09T00:36:23.355 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:16.707+0000 7ff10cf14700 1 --2- 172.21.15.5:0/1205162331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff108108970 0x7ff10819def0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc011a20 tx=0x7ff0fc011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:36:23.355 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:16.707+0000 7ff105ffb700 1 -- 172.21.15.5:0/1205162331 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0fc0039c0 con 0x7ff108108970 2024-05-09T00:36:23.355 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:16.707+0000 7ff105ffb700 1 -- 172.21.15.5:0/1205162331 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0fc003b60 con 0x7ff108108970 2024-05-09T00:36:23.355 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:16.707+0000 7ff10f178700 1 -- 172.21.15.5:0/1205162331 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff10819e6c0 con 0x7ff108108970 2024-05-09T00:36:23.355 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:16.707+0000 7ff105ffb700 1 -- 172.21.15.5:0/1205162331 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0fc019700 con 0x7ff108108970 2024-05-09T00:36:23.355 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:16.707+0000 7ff10f178700 1 -- 172.21.15.5:0/1205162331 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1081a12d0 con 0x7ff108108970 2024-05-09T00:36:23.355 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:16.708+0000 7ff105ffb700 1 -- 172.21.15.5:0/1205162331 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7ff0fc0199b0 con 0x7ff108108970 2024-05-09T00:36:23.356 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:16.708+0000 7ff105ffb700 1 --2- 172.21.15.5:0/1205162331 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7ff0f803e4d0 0x7ff0f80409e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:36:23.356 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:16.709+0000 7ff105ffb700 1 -- 172.21.15.5:0/1205162331 --> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff0f80410f0 con 0x7ff0f803e4d0 2024-05-09T00:36:23.356 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:16.709+0000 7ff105ffb700 1 -- 172.21.15.5:0/1205162331 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff0fc053e70 con 0x7ff108108970 2024-05-09T00:36:23.356 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:16.709+0000 7ff107fff700 1 -- 172.21.15.5:0/1205162331 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7ff0f803e4d0 msgr2=0x7ff0f80409e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/287842266 2024-05-09T00:36:23.356 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:16.709+0000 7ff107fff700 1 --2- 172.21.15.5:0/1205162331 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7ff0f803e4d0 0x7ff0f80409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T00:36:23.356 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:16.910+0000 7ff107fff700 1 -- 172.21.15.5:0/1205162331 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7ff0f803e4d0 msgr2=0x7ff0f80409e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/287842266 2024-05-09T00:36:23.356 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:16.910+0000 7ff107fff700 1 --2- 172.21.15.5:0/1205162331 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7ff0f803e4d0 0x7ff0f80409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T00:36:23.356 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:17.311+0000 7ff107fff700 1 -- 172.21.15.5:0/1205162331 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7ff0f803e4d0 msgr2=0x7ff0f80409e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/287842266 2024-05-09T00:36:23.356 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:17.311+0000 7ff107fff700 1 --2- 172.21.15.5:0/1205162331 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7ff0f803e4d0 0x7ff0f80409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-09T00:36:23.356 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:18.112+0000 7ff107fff700 1 -- 172.21.15.5:0/1205162331 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7ff0f803e4d0 msgr2=0x7ff0f80409e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/287842266 2024-05-09T00:36:23.356 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:18.112+0000 7ff107fff700 1 --2- 172.21.15.5:0/1205162331 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7ff0f803e4d0 0x7ff0f80409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-09T00:36:23.356 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:19.713+0000 7ff107fff700 1 -- 172.21.15.5:0/1205162331 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7ff0f803e4d0 msgr2=0x7ff0f80409e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/287842266 2024-05-09T00:36:23.356 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:19.713+0000 7ff107fff700 1 --2- 172.21.15.5:0/1205162331 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7ff0f803e4d0 0x7ff0f80409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-09T00:36:23.357 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:21.744+0000 7ff105ffb700 1 -- 172.21.15.5:0/1205162331 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mgrmap(e 10) ==== 50002+0+0 (secure 0 0 0) 0x7ff0fc02d440 con 0x7ff108108970 2024-05-09T00:36:23.357 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:21.744+0000 7ff105ffb700 1 -- 172.21.15.5:0/1205162331 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7ff0f803e4d0 msgr2=0x7ff0f80409e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T00:36:23.357 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:21.744+0000 7ff105ffb700 1 --2- 172.21.15.5:0/1205162331 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7ff0f803e4d0 0x7ff0f80409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:36:23.357 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:22.745+0000 7ff105ffb700 1 -- 172.21.15.5:0/1205162331 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mgrmap(e 11) ==== 50129+0+0 (secure 0 0 0) 0x7ff0fc055660 con 0x7ff108108970 2024-05-09T00:36:23.357 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:22.745+0000 7ff105ffb700 1 --2- 172.21.15.5:0/1205162331 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ff0fc053a00 0x7ff0f8043fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:36:23.357 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:22.746+0000 7ff105ffb700 1 -- 172.21.15.5:0/1205162331 --> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff0fc01ce50 con 0x7ff0fc053a00 2024-05-09T00:36:23.357 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:22.746+0000 7ff107fff700 1 --2- 172.21.15.5:0/1205162331 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ff0fc053a00 0x7ff0f8043fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:36:23.357 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:22.747+0000 7ff107fff700 1 --2- 172.21.15.5:0/1205162331 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ff0fc053a00 0x7ff0f8043fd0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7ff0f4003a10 tx=0x7ff0f40092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:36:23.357 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:22.748+0000 7ff105ffb700 1 -- 172.21.15.5:0/1205162331 <== mgr.14152 v2:172.21.15.5:6800/91856593 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7ff0fc01ce50 con 0x7ff0fc053a00 2024-05-09T00:36:23.357 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:22.756+0000 7ff10f178700 1 -- 172.21.15.5:0/1205162331 --> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7ff108064e20 con 0x7ff0fc053a00 2024-05-09T00:36:23.357 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:22.756+0000 7ff105ffb700 1 -- 172.21.15.5:0/1205162331 <== mgr.14152 v2:172.21.15.5:6800/91856593 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7ff108064e20 con 0x7ff0fc053a00 2024-05-09T00:36:23.357 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:22.757+0000 7ff10f178700 1 -- 172.21.15.5:0/1205162331 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ff0fc053a00 msgr2=0x7ff0f8043fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:36:23.357 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:22.757+0000 7ff10f178700 1 --2- 172.21.15.5:0/1205162331 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ff0fc053a00 0x7ff0f8043fd0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7ff0f4003a10 tx=0x7ff0f40092b0 comp rx=0 tx=0).stop 2024-05-09T00:36:23.358 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:22.757+0000 7ff10f178700 1 -- 172.21.15.5:0/1205162331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff108108970 msgr2=0x7ff10819def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:36:23.358 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:22.757+0000 7ff10f178700 1 --2- 172.21.15.5:0/1205162331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff108108970 0x7ff10819def0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc011a20 tx=0x7ff0fc011b00 comp rx=0 tx=0).stop 2024-05-09T00:36:23.358 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:22.757+0000 7ff10f178700 1 -- 172.21.15.5:0/1205162331 shutdown_connections 2024-05-09T00:36:23.358 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:22.757+0000 7ff10f178700 1 --2- 172.21.15.5:0/1205162331 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ff0fc053a00 0x7ff0f8043fd0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:36:23.358 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:22.757+0000 7ff10f178700 1 --2- 172.21.15.5:0/1205162331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff108108970 0x7ff10819def0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:36:23.358 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:22.757+0000 7ff10f178700 1 --2- 172.21.15.5:0/1205162331 >> [v2:172.21.15.5:6800/287842266,v1:172.21.15.5:6801/287842266] conn(0x7ff0f803e4d0 0x7ff0f80409e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:36:23.358 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:22.757+0000 7ff10f178700 1 -- 172.21.15.5:0/1205162331 >> 172.21.15.5:0/1205162331 conn(0x7ff108104050 msgr2=0x7ff108105990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:36:23.358 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:22.757+0000 7ff10f178700 1 -- 172.21.15.5:0/1205162331 shutdown_connections 2024-05-09T00:36:23.358 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:22.757+0000 7ff10f178700 1 -- 172.21.15.5:0/1205162331 wait complete. 2024-05-09T00:36:23.358 INFO:teuthology.orchestra.run.smithi005.stdout:mgr epoch 9 is available 2024-05-09T00:36:23.358 INFO:teuthology.orchestra.run.smithi005.stdout:Generating a dashboard self-signed certificate... 2024-05-09T00:36:23.996 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:23 smithi005 ceph-mon[30365]: [09/May/2024:00:36:22] ENGINE Bus STARTING 2024-05-09T00:36:23.996 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:23 smithi005 ceph-mon[30365]: [09/May/2024:00:36:22] ENGINE Serving on https://172.21.15.5:7150 2024-05-09T00:36:23.997 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:23 smithi005 ceph-mon[30365]: [09/May/2024:00:36:22] ENGINE Serving on http://172.21.15.5:8765 2024-05-09T00:36:23.997 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:23 smithi005 ceph-mon[30365]: [09/May/2024:00:36:22] ENGINE Bus STARTED 2024-05-09T00:36:23.997 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:23 smithi005 ceph-mon[30365]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-09T00:36:23.997 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:23 smithi005 ceph-mon[30365]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-09T00:36:25.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:24 smithi005 ceph-mon[30365]: mgrmap e12: x(active, since 2s) 2024-05-09T00:36:25.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:24 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:36:25.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:24 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:36:25.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:24 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:36:25.066 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-09T00:36:25.067 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:24.028+0000 7fbeba786700 1 Processor -- start 2024-05-09T00:36:25.067 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:24.028+0000 7fbeba786700 1 -- start start 2024-05-09T00:36:25.067 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:24.028+0000 7fbeba786700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbeb4108970 0x7fbeb4108d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:36:25.067 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:24.028+0000 7fbeba786700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb4109370 con 0x7fbeb4108970 2024-05-09T00:36:25.067 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:24.029+0000 7fbeb3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbeb4108970 0x7fbeb4108d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:36:25.067 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:24.029+0000 7fbeb3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbeb4108970 0x7fbeb4108d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54044/0 (socket says 172.21.15.5:54044) 2024-05-09T00:36:25.067 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:24.029+0000 7fbeb3fff700 1 -- 172.21.15.5:0/1133081420 learned_addr learned my addr 172.21.15.5:0/1133081420 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:36:25.067 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:24.029+0000 7fbeb3fff700 1 -- 172.21.15.5:0/1133081420 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeb4109b40 con 0x7fbeb4108970 2024-05-09T00:36:25.067 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:24.029+0000 7fbeb3fff700 1 --2- 172.21.15.5:0/1133081420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbeb4108970 0x7fbeb4108d80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbea40098a0 tx=0x7fbea4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=569ca0b9af870335 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:36:25.067 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:24.029+0000 7fbeb2ffd700 1 -- 172.21.15.5:0/1133081420 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbea4004160 con 0x7fbeb4108970 2024-05-09T00:36:25.067 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:24.029+0000 7fbeb2ffd700 1 -- 172.21.15.5:0/1133081420 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbea4004300 con 0x7fbeb4108970 2024-05-09T00:36:25.067 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:24.029+0000 7fbeb2ffd700 1 -- 172.21.15.5:0/1133081420 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbea4011a20 con 0x7fbeb4108970 2024-05-09T00:36:25.067 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:24.031+0000 7fbeba786700 1 -- 172.21.15.5:0/1133081420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbeb4108970 msgr2=0x7fbeb4108d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:36:25.068 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:24.031+0000 7fbeba786700 1 --2- 172.21.15.5:0/1133081420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbeb4108970 0x7fbeb4108d80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbea40098a0 tx=0x7fbea4009bb0 comp rx=0 tx=0).stop 2024-05-09T00:36:25.068 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:24.031+0000 7fbeba786700 1 -- 172.21.15.5:0/1133081420 shutdown_connections 2024-05-09T00:36:25.068 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:24.031+0000 7fbeba786700 1 --2- 172.21.15.5:0/1133081420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbeb4108970 0x7fbeb4108d80 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:36:25.068 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:24.031+0000 7fbeba786700 1 -- 172.21.15.5:0/1133081420 >> 172.21.15.5:0/1133081420 conn(0x7fbeb4104050 msgr2=0x7fbeb41064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:36:25.068 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:24.031+0000 7fbeba786700 1 -- 172.21.15.5:0/1133081420 shutdown_connections 2024-05-09T00:36:25.068 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:24.031+0000 7fbeba786700 1 -- 172.21.15.5:0/1133081420 wait complete. 2024-05-09T00:36:25.068 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:24.031+0000 7fbeba786700 1 Processor -- start 2024-05-09T00:36:25.068 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:24.032+0000 7fbeba786700 1 -- start start 2024-05-09T00:36:25.068 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:24.032+0000 7fbeba786700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbeb4108970 0x7fbeb4107230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:36:25.068 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:24.032+0000 7fbeba786700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb410cef0 con 0x7fbeb4108970 2024-05-09T00:36:25.068 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:24.032+0000 7fbeb3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbeb4108970 0x7fbeb4107230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:36:25.068 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:24.032+0000 7fbeb3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbeb4108970 0x7fbeb4107230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54060/0 (socket says 172.21.15.5:54060) 2024-05-09T00:36:25.069 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:24.032+0000 7fbeb3fff700 1 -- 172.21.15.5:0/3456238734 learned_addr learned my addr 172.21.15.5:0/3456238734 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:36:25.069 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:24.032+0000 7fbeb3fff700 1 -- 172.21.15.5:0/3456238734 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeb4107770 con 0x7fbeb4108970 2024-05-09T00:36:25.069 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:24.033+0000 7fbeb3fff700 1 --2- 172.21.15.5:0/3456238734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbeb4108970 0x7fbeb4107230 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbea4011a20 tx=0x7fbea4011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:36:25.069 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:24.033+0000 7fbeb17fa700 1 -- 172.21.15.5:0/3456238734 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbea40039c0 con 0x7fbeb4108970 2024-05-09T00:36:25.069 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:24.033+0000 7fbeb17fa700 1 -- 172.21.15.5:0/3456238734 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbea4003b60 con 0x7fbeb4108970 2024-05-09T00:36:25.069 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:24.033+0000 7fbeba786700 1 -- 172.21.15.5:0/3456238734 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbeb41059c0 con 0x7fbeb4108970 2024-05-09T00:36:25.069 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:24.033+0000 7fbeb17fa700 1 -- 172.21.15.5:0/3456238734 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbea4019700 con 0x7fbeb4108970 2024-05-09T00:36:25.069 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:24.033+0000 7fbeba786700 1 -- 172.21.15.5:0/3456238734 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbeb4105ea0 con 0x7fbeb4108970 2024-05-09T00:36:25.069 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:24.034+0000 7fbeb17fa700 1 -- 172.21.15.5:0/3456238734 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7fbea4019a00 con 0x7fbeb4108970 2024-05-09T00:36:25.069 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:24.034+0000 7fbeba786700 1 -- 172.21.15.5:0/3456238734 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbeb4064e20 con 0x7fbeb4108970 2024-05-09T00:36:25.069 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:24.034+0000 7fbeb17fa700 1 --2- 172.21.15.5:0/3456238734 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fbea003e470 0x7fbea0040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:36:25.069 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:24.034+0000 7fbeb17fa700 1 -- 172.21.15.5:0/3456238734 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fbea4054d30 con 0x7fbeb4108970 2024-05-09T00:36:25.069 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:24.035+0000 7fbeb37fe700 1 --2- 172.21.15.5:0/3456238734 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fbea003e470 0x7fbea0040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:36:25.070 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:24.035+0000 7fbeb37fe700 1 --2- 172.21.15.5:0/3456238734 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fbea003e470 0x7fbea0040980 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c006fd0 tx=0x7fbe9c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:36:25.070 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:24.040+0000 7fbeb17fa700 1 -- 172.21.15.5:0/3456238734 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbea4020e20 con 0x7fbeb4108970 2024-05-09T00:36:25.070 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:24.263+0000 7fbeba786700 1 -- 172.21.15.5:0/3456238734 --> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fbeb4197040 con 0x7fbea003e470 2024-05-09T00:36:25.070 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:24.484+0000 7fbeb17fa700 1 -- 172.21.15.5:0/3456238734 <== mgr.14152 v2:172.21.15.5:6800/91856593 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fbeb4197040 con 0x7fbea003e470 2024-05-09T00:36:25.070 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:24.486+0000 7fbeba786700 1 -- 172.21.15.5:0/3456238734 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fbea003e470 msgr2=0x7fbea0040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:36:25.070 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:24.486+0000 7fbeba786700 1 --2- 172.21.15.5:0/3456238734 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fbea003e470 0x7fbea0040980 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c006fd0 tx=0x7fbe9c006e40 comp rx=0 tx=0).stop 2024-05-09T00:36:25.070 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:24.486+0000 7fbeba786700 1 -- 172.21.15.5:0/3456238734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbeb4108970 msgr2=0x7fbeb4107230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:36:25.070 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:24.486+0000 7fbeba786700 1 --2- 172.21.15.5:0/3456238734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbeb4108970 0x7fbeb4107230 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbea4011a20 tx=0x7fbea4011b00 comp rx=0 tx=0).stop 2024-05-09T00:36:25.070 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:24.487+0000 7fbeba786700 1 -- 172.21.15.5:0/3456238734 shutdown_connections 2024-05-09T00:36:25.070 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:24.487+0000 7fbeba786700 1 --2- 172.21.15.5:0/3456238734 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fbea003e470 0x7fbea0040980 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:36:25.070 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:24.487+0000 7fbeba786700 1 --2- 172.21.15.5:0/3456238734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbeb4108970 0x7fbeb4107230 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:36:25.070 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:24.487+0000 7fbeba786700 1 -- 172.21.15.5:0/3456238734 >> 172.21.15.5:0/3456238734 conn(0x7fbeb4104050 msgr2=0x7fbeb4194440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:36:25.070 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:24.487+0000 7fbeba786700 1 -- 172.21.15.5:0/3456238734 shutdown_connections 2024-05-09T00:36:25.071 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:24.487+0000 7fbeba786700 1 -- 172.21.15.5:0/3456238734 wait complete. 2024-05-09T00:36:25.071 INFO:teuthology.orchestra.run.smithi005.stdout:Creating initial admin user... 2024-05-09T00:36:26.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:26 smithi005 ceph-mon[30365]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:36:26.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:26 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:36:26.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:26 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:36:27.414 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:27 smithi005 ceph-mon[30365]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:36:27.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$IhY0h/Tzijd62wc47AhITuO4kayeOAQoGkaH8zwAS7N/fI1LRPDEK", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1715214986, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-09T00:36:27.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:25.554+0000 7f0fe7881700 1 Processor -- start 2024-05-09T00:36:27.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:25.555+0000 7f0fe7881700 1 -- start start 2024-05-09T00:36:27.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:25.555+0000 7f0fe7881700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fe010a590 0x7f0fe010c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:36:27.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:25.555+0000 7f0fe7881700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe007ce30 con 0x7f0fe010a590 2024-05-09T00:36:27.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:25.555+0000 7f0fe561d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fe010a590 0x7f0fe010c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:36:27.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:25.555+0000 7f0fe561d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fe010a590 0x7f0fe010c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54068/0 (socket says 172.21.15.5:54068) 2024-05-09T00:36:27.423 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:25.555+0000 7f0fe561d700 1 -- 172.21.15.5:0/1803171975 learned_addr learned my addr 172.21.15.5:0/1803171975 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:36:27.423 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:25.556+0000 7f0fe561d700 1 -- 172.21.15.5:0/1803171975 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fe010cef0 con 0x7f0fe010a590 2024-05-09T00:36:27.423 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:25.556+0000 7f0fe561d700 1 --2- 172.21.15.5:0/1803171975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fe010a590 0x7f0fe010c9b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc0098a0 tx=0x7f0fdc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b307506c7e4789b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:36:27.423 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:25.556+0000 7f0fd7fff700 1 -- 172.21.15.5:0/1803171975 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0fdc004160 con 0x7f0fe010a590 2024-05-09T00:36:27.423 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:25.556+0000 7f0fd7fff700 1 -- 172.21.15.5:0/1803171975 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0fdc004300 con 0x7f0fe010a590 2024-05-09T00:36:27.423 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:25.556+0000 7f0fd7fff700 1 -- 172.21.15.5:0/1803171975 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0fdc011a20 con 0x7f0fe010a590 2024-05-09T00:36:27.423 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:25.557+0000 7f0fe7881700 1 -- 172.21.15.5:0/1803171975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fe010a590 msgr2=0x7f0fe010c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:36:27.423 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:25.557+0000 7f0fe7881700 1 --2- 172.21.15.5:0/1803171975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fe010a590 0x7f0fe010c9b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc0098a0 tx=0x7f0fdc009bb0 comp rx=0 tx=0).stop 2024-05-09T00:36:27.423 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:25.557+0000 7f0fe7881700 1 -- 172.21.15.5:0/1803171975 shutdown_connections 2024-05-09T00:36:27.423 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:25.557+0000 7f0fe7881700 1 --2- 172.21.15.5:0/1803171975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fe010a590 0x7f0fe010c9b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:36:27.423 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:25.557+0000 7f0fe7881700 1 -- 172.21.15.5:0/1803171975 >> 172.21.15.5:0/1803171975 conn(0x7f0fe0104050 msgr2=0x7f0fe01064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:36:27.423 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:25.557+0000 7f0fe7881700 1 -- 172.21.15.5:0/1803171975 shutdown_connections 2024-05-09T00:36:27.424 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:25.557+0000 7f0fe7881700 1 -- 172.21.15.5:0/1803171975 wait complete. 2024-05-09T00:36:27.424 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:25.558+0000 7f0fe7881700 1 Processor -- start 2024-05-09T00:36:27.424 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:25.558+0000 7f0fe7881700 1 -- start start 2024-05-09T00:36:27.424 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:25.558+0000 7f0fe7881700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fe010a590 0x7f0fe0109d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:36:27.424 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:25.558+0000 7f0fe7881700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe010de90 con 0x7f0fe010a590 2024-05-09T00:36:27.424 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:25.558+0000 7f0fe561d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fe010a590 0x7f0fe0109d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:36:27.424 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:25.558+0000 7f0fe561d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fe010a590 0x7f0fe0109d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54080/0 (socket says 172.21.15.5:54080) 2024-05-09T00:36:27.424 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:25.558+0000 7f0fe561d700 1 -- 172.21.15.5:0/2979255776 learned_addr learned my addr 172.21.15.5:0/2979255776 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:36:27.424 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:25.558+0000 7f0fe561d700 1 -- 172.21.15.5:0/2979255776 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fe010a250 con 0x7f0fe010a590 2024-05-09T00:36:27.424 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:25.559+0000 7f0fe561d700 1 --2- 172.21.15.5:0/2979255776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fe010a590 0x7f0fe0109d10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc004660 tx=0x7f0fdc011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:36:27.424 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:25.559+0000 7f0fd67fc700 1 -- 172.21.15.5:0/2979255776 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0fdc003a60 con 0x7f0fe010a590 2024-05-09T00:36:27.424 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:25.559+0000 7f0fd67fc700 1 -- 172.21.15.5:0/2979255776 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0fdc01b8f0 con 0x7f0fe010a590 2024-05-09T00:36:27.425 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:25.559+0000 7f0fe7881700 1 -- 172.21.15.5:0/2979255776 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fe01084a0 con 0x7f0fe010a590 2024-05-09T00:36:27.425 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:25.559+0000 7f0fd67fc700 1 -- 172.21.15.5:0/2979255776 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0fdc023890 con 0x7f0fe010a590 2024-05-09T00:36:27.425 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:25.559+0000 7f0fe7881700 1 -- 172.21.15.5:0/2979255776 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fe0108980 con 0x7f0fe010a590 2024-05-09T00:36:27.425 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:25.560+0000 7f0fd67fc700 1 -- 172.21.15.5:0/2979255776 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f0fdc02c3f0 con 0x7f0fe010a590 2024-05-09T00:36:27.425 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:25.560+0000 7f0fd67fc700 1 --2- 172.21.15.5:0/2979255776 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f0fcc03e4d0 0x7f0fcc0409e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:36:27.425 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:25.560+0000 7f0fe7881700 1 -- 172.21.15.5:0/2979255776 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fe0064e20 con 0x7f0fe010a590 2024-05-09T00:36:27.425 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:25.560+0000 7f0fd67fc700 1 -- 172.21.15.5:0/2979255776 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0fdc054020 con 0x7f0fe010a590 2024-05-09T00:36:27.425 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:25.560+0000 7f0fe4e1c700 1 --2- 172.21.15.5:0/2979255776 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f0fcc03e4d0 0x7f0fcc0409e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:36:27.425 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:25.561+0000 7f0fe4e1c700 1 --2- 172.21.15.5:0/2979255776 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f0fcc03e4d0 0x7f0fcc0409e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0fd0006fd0 tx=0x7f0fd0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:36:27.425 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:25.566+0000 7f0fd67fc700 1 -- 172.21.15.5:0/2979255776 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0fdc01f030 con 0x7f0fe010a590 2024-05-09T00:36:27.425 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:25.819+0000 7f0fe7881700 1 -- 172.21.15.5:0/2979255776 --> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f0fe0109000 con 0x7f0fcc03e4d0 2024-05-09T00:36:27.426 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:26.067+0000 7f0fd67fc700 1 -- 172.21.15.5:0/2979255776 <== mgr.14152 v2:172.21.15.5:6800/91856593 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f0fe0109000 con 0x7f0fcc03e4d0 2024-05-09T00:36:27.426 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:26.069+0000 7f0fe7881700 1 -- 172.21.15.5:0/2979255776 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f0fcc03e4d0 msgr2=0x7f0fcc0409e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:36:27.426 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:26.069+0000 7f0fe7881700 1 --2- 172.21.15.5:0/2979255776 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f0fcc03e4d0 0x7f0fcc0409e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0fd0006fd0 tx=0x7f0fd0006e40 comp rx=0 tx=0).stop 2024-05-09T00:36:27.426 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:26.069+0000 7f0fe7881700 1 -- 172.21.15.5:0/2979255776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fe010a590 msgr2=0x7f0fe0109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:36:27.426 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:26.069+0000 7f0fe7881700 1 --2- 172.21.15.5:0/2979255776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fe010a590 0x7f0fe0109d10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc004660 tx=0x7f0fdc011aa0 comp rx=0 tx=0).stop 2024-05-09T00:36:27.426 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:26.069+0000 7f0fe7881700 1 -- 172.21.15.5:0/2979255776 shutdown_connections 2024-05-09T00:36:27.426 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:26.069+0000 7f0fe7881700 1 --2- 172.21.15.5:0/2979255776 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f0fcc03e4d0 0x7f0fcc0409e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:36:27.426 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:26.069+0000 7f0fe7881700 1 --2- 172.21.15.5:0/2979255776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fe010a590 0x7f0fe0109d10 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:36:27.426 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:26.069+0000 7f0fe7881700 1 -- 172.21.15.5:0/2979255776 >> 172.21.15.5:0/2979255776 conn(0x7f0fe0104050 msgr2=0x7f0fe01059f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:36:27.426 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:26.069+0000 7f0fe7881700 1 -- 172.21.15.5:0/2979255776 shutdown_connections 2024-05-09T00:36:27.426 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:26.069+0000 7f0fe7881700 1 -- 172.21.15.5:0/2979255776 wait complete. 2024-05-09T00:36:27.427 INFO:teuthology.orchestra.run.smithi005.stdout:Fetching dashboard port number... 2024-05-09T00:36:28.319 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:28 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/1964480658' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-09T00:36:28.570 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 8443 2024-05-09T00:36:28.570 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:27.832+0000 7f9bdfaee700 1 Processor -- start 2024-05-09T00:36:28.570 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:27.833+0000 7f9bdfaee700 1 -- start start 2024-05-09T00:36:28.570 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:27.833+0000 7f9bdfaee700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9bd8108970 0x7f9bd8108d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:36:28.570 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:27.833+0000 7f9bdfaee700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd8109370 con 0x7f9bd8108970 2024-05-09T00:36:28.570 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:27.833+0000 7f9bdd88a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9bd8108970 0x7f9bd8108d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:36:28.570 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:27.833+0000 7f9bdd88a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9bd8108970 0x7f9bd8108d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54086/0 (socket says 172.21.15.5:54086) 2024-05-09T00:36:28.570 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:27.833+0000 7f9bdd88a700 1 -- 172.21.15.5:0/236569093 learned_addr learned my addr 172.21.15.5:0/236569093 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:36:28.570 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:27.833+0000 7f9bdd88a700 1 -- 172.21.15.5:0/236569093 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bd8109b40 con 0x7f9bd8108970 2024-05-09T00:36:28.571 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:27.833+0000 7f9bdd88a700 1 --2- 172.21.15.5:0/236569093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9bd8108970 0x7f9bd8108d80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9bd40098a0 tx=0x7f9bd4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb0e7f521d5b247a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:36:28.571 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:27.834+0000 7f9bdc888700 1 -- 172.21.15.5:0/236569093 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9bd4004160 con 0x7f9bd8108970 2024-05-09T00:36:28.571 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:27.834+0000 7f9bdc888700 1 -- 172.21.15.5:0/236569093 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bd4004300 con 0x7f9bd8108970 2024-05-09T00:36:28.571 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:27.834+0000 7f9bdc888700 1 -- 172.21.15.5:0/236569093 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9bd4011a20 con 0x7f9bd8108970 2024-05-09T00:36:28.571 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:27.834+0000 7f9bdfaee700 1 -- 172.21.15.5:0/236569093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9bd8108970 msgr2=0x7f9bd8108d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:36:28.571 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:27.834+0000 7f9bdfaee700 1 --2- 172.21.15.5:0/236569093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9bd8108970 0x7f9bd8108d80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9bd40098a0 tx=0x7f9bd4009bb0 comp rx=0 tx=0).stop 2024-05-09T00:36:28.571 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:27.834+0000 7f9bdfaee700 1 -- 172.21.15.5:0/236569093 shutdown_connections 2024-05-09T00:36:28.571 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:27.834+0000 7f9bdfaee700 1 --2- 172.21.15.5:0/236569093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9bd8108970 0x7f9bd8108d80 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:36:28.571 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:27.834+0000 7f9bdfaee700 1 -- 172.21.15.5:0/236569093 >> 172.21.15.5:0/236569093 conn(0x7f9bd8104050 msgr2=0x7f9bd81064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:36:28.571 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:27.835+0000 7f9bdfaee700 1 -- 172.21.15.5:0/236569093 shutdown_connections 2024-05-09T00:36:28.571 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:27.835+0000 7f9bdfaee700 1 -- 172.21.15.5:0/236569093 wait complete. 2024-05-09T00:36:28.571 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:27.835+0000 7f9bdfaee700 1 Processor -- start 2024-05-09T00:36:28.571 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:27.835+0000 7f9bdfaee700 1 -- start start 2024-05-09T00:36:28.572 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:27.835+0000 7f9bdfaee700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9bd8108970 0x7f9bd807c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:36:28.572 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:27.835+0000 7f9bdfaee700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd810cef0 con 0x7f9bd8108970 2024-05-09T00:36:28.572 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:27.836+0000 7f9bdd88a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9bd8108970 0x7f9bd807c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:36:28.572 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:27.836+0000 7f9bdd88a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9bd8108970 0x7f9bd807c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54098/0 (socket says 172.21.15.5:54098) 2024-05-09T00:36:28.572 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:27.836+0000 7f9bdd88a700 1 -- 172.21.15.5:0/1964480658 learned_addr learned my addr 172.21.15.5:0/1964480658 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:36:28.572 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:27.836+0000 7f9bdd88a700 1 -- 172.21.15.5:0/1964480658 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bd807ce80 con 0x7f9bd8108970 2024-05-09T00:36:28.572 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:27.836+0000 7f9bdd88a700 1 --2- 172.21.15.5:0/1964480658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9bd8108970 0x7f9bd807c940 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4011a20 tx=0x7f9bd4011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:36:28.572 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:27.836+0000 7f9bceffd700 1 -- 172.21.15.5:0/1964480658 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9bd40039c0 con 0x7f9bd8108970 2024-05-09T00:36:28.572 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:27.836+0000 7f9bceffd700 1 -- 172.21.15.5:0/1964480658 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bd4003b60 con 0x7f9bd8108970 2024-05-09T00:36:28.572 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:27.836+0000 7f9bdfaee700 1 -- 172.21.15.5:0/1964480658 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bd81a0ca0 con 0x7f9bd8108970 2024-05-09T00:36:28.572 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:27.836+0000 7f9bceffd700 1 -- 172.21.15.5:0/1964480658 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9bd4019700 con 0x7f9bd8108970 2024-05-09T00:36:28.572 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:27.836+0000 7f9bdfaee700 1 -- 172.21.15.5:0/1964480658 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bd81a1120 con 0x7f9bd8108970 2024-05-09T00:36:28.572 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:27.837+0000 7f9bceffd700 1 -- 172.21.15.5:0/1964480658 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f9bd401a430 con 0x7f9bd8108970 2024-05-09T00:36:28.573 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:27.837+0000 7f9bdfaee700 1 -- 172.21.15.5:0/1964480658 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bd8064e20 con 0x7f9bd8108970 2024-05-09T00:36:28.573 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:27.838+0000 7f9bceffd700 1 --2- 172.21.15.5:0/1964480658 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f9bc403e4d0 0x7f9bc40409e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:36:28.573 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:27.838+0000 7f9bceffd700 1 -- 172.21.15.5:0/1964480658 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9bd4053e20 con 0x7f9bd8108970 2024-05-09T00:36:28.573 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:27.838+0000 7f9bdd089700 1 --2- 172.21.15.5:0/1964480658 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f9bc403e4d0 0x7f9bc40409e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:36:28.573 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:27.838+0000 7f9bdd089700 1 --2- 172.21.15.5:0/1964480658 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f9bc403e4d0 0x7f9bc40409e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8006fd0 tx=0x7f9bc8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:36:28.573 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:27.843+0000 7f9bceffd700 1 -- 172.21.15.5:0/1964480658 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9bd4020e20 con 0x7f9bd8108970 2024-05-09T00:36:28.573 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:28.056+0000 7f9bdfaee700 1 -- 172.21.15.5:0/1964480658 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f9bd81a13a0 con 0x7f9bd8108970 2024-05-09T00:36:28.573 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:28.056+0000 7f9bceffd700 1 -- 172.21.15.5:0/1964480658 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f9bd402c3e0 con 0x7f9bd8108970 2024-05-09T00:36:28.573 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:28.058+0000 7f9bdfaee700 1 -- 172.21.15.5:0/1964480658 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f9bc403e4d0 msgr2=0x7f9bc40409e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:36:28.573 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:28.058+0000 7f9bdfaee700 1 --2- 172.21.15.5:0/1964480658 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f9bc403e4d0 0x7f9bc40409e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8006fd0 tx=0x7f9bc8006e40 comp rx=0 tx=0).stop 2024-05-09T00:36:28.573 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:28.058+0000 7f9bdfaee700 1 -- 172.21.15.5:0/1964480658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9bd8108970 msgr2=0x7f9bd807c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:36:28.573 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:28.058+0000 7f9bdfaee700 1 --2- 172.21.15.5:0/1964480658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9bd8108970 0x7f9bd807c940 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4011a20 tx=0x7f9bd4011b00 comp rx=0 tx=0).stop 2024-05-09T00:36:28.573 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:28.058+0000 7f9bdfaee700 1 -- 172.21.15.5:0/1964480658 shutdown_connections 2024-05-09T00:36:28.574 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:28.058+0000 7f9bdfaee700 1 --2- 172.21.15.5:0/1964480658 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f9bc403e4d0 0x7f9bc40409e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:36:28.574 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:28.058+0000 7f9bdfaee700 1 --2- 172.21.15.5:0/1964480658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9bd8108970 0x7f9bd807c940 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:36:28.574 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:28.058+0000 7f9bdfaee700 1 -- 172.21.15.5:0/1964480658 >> 172.21.15.5:0/1964480658 conn(0x7f9bd8104050 msgr2=0x7f9bd8105990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:36:28.574 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:28.059+0000 7f9bdfaee700 1 -- 172.21.15.5:0/1964480658 shutdown_connections 2024-05-09T00:36:28.574 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:28.059+0000 7f9bdfaee700 1 -- 172.21.15.5:0/1964480658 wait complete. 2024-05-09T00:36:28.577 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-09T00:36:28.578 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout disabled 2024-05-09T00:36:28.585 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-09T00:36:28.585 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout inactive 2024-05-09T00:36:28.585 INFO:teuthology.orchestra.run.smithi005.stdout:firewalld.service is not enabled 2024-05-09T00:36:28.586 INFO:teuthology.orchestra.run.smithi005.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-09T00:36:28.587 INFO:teuthology.orchestra.run.smithi005.stdout:Ceph Dashboard is now available at: 2024-05-09T00:36:28.587 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:36:28.587 INFO:teuthology.orchestra.run.smithi005.stdout: URL: https://smithi005.front.sepia.ceph.com:8443/ 2024-05-09T00:36:28.587 INFO:teuthology.orchestra.run.smithi005.stdout: User: admin 2024-05-09T00:36:28.587 INFO:teuthology.orchestra.run.smithi005.stdout: Password: z1pyb2wib3 2024-05-09T00:36:28.587 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:36:28.587 INFO:teuthology.orchestra.run.smithi005.stdout:Saving cluster configuration to /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/config directory 2024-05-09T00:36:28.588 INFO:teuthology.orchestra.run.smithi005.stdout:Enabling autotune for osd_memory_target 2024-05-09T00:36:29.227 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:29 smithi005 ceph-mon[30365]: mgrmap e13: x(active, since 6s) 2024-05-09T00:36:29.760 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:29.002+0000 7f1f056a7700 1 Processor -- start 2024-05-09T00:36:29.760 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:29.002+0000 7f1f056a7700 1 -- start start 2024-05-09T00:36:29.760 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:29.002+0000 7f1f056a7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f00108970 0x7f1f00108d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:36:29.760 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:29.002+0000 7f1f056a7700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1f00109370 con 0x7f1f00108970 2024-05-09T00:36:29.760 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:29.002+0000 7f1efeffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f00108970 0x7f1f00108d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:36:29.760 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:29.002+0000 7f1efeffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f00108970 0x7f1f00108d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54104/0 (socket says 172.21.15.5:54104) 2024-05-09T00:36:29.760 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:29.002+0000 7f1efeffd700 1 -- 172.21.15.5:0/2762304716 learned_addr learned my addr 172.21.15.5:0/2762304716 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:36:29.760 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:29.003+0000 7f1efeffd700 1 -- 172.21.15.5:0/2762304716 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f00109b40 con 0x7f1f00108970 2024-05-09T00:36:29.760 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:29.003+0000 7f1efeffd700 1 --2- 172.21.15.5:0/2762304716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f00108970 0x7f1f00108d80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0009e80 tx=0x7f1ef000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=508f64c72a064fc1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:36:29.760 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:29.003+0000 7f1efdffb700 1 -- 172.21.15.5:0/2762304716 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ef0017070 con 0x7f1f00108970 2024-05-09T00:36:29.760 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:29.003+0000 7f1efdffb700 1 -- 172.21.15.5:0/2762304716 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ef000b8c0 con 0x7f1f00108970 2024-05-09T00:36:29.761 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:29.003+0000 7f1efdffb700 1 -- 172.21.15.5:0/2762304716 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ef0004490 con 0x7f1f00108970 2024-05-09T00:36:29.761 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:29.004+0000 7f1f056a7700 1 -- 172.21.15.5:0/2762304716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f00108970 msgr2=0x7f1f00108d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:36:29.761 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:29.004+0000 7f1f056a7700 1 --2- 172.21.15.5:0/2762304716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f00108970 0x7f1f00108d80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0009e80 tx=0x7f1ef000b330 comp rx=0 tx=0).stop 2024-05-09T00:36:29.761 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:29.004+0000 7f1f056a7700 1 -- 172.21.15.5:0/2762304716 shutdown_connections 2024-05-09T00:36:29.761 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:29.004+0000 7f1f056a7700 1 --2- 172.21.15.5:0/2762304716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f00108970 0x7f1f00108d80 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:36:29.761 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:29.004+0000 7f1f056a7700 1 -- 172.21.15.5:0/2762304716 >> 172.21.15.5:0/2762304716 conn(0x7f1f00103fb0 msgr2=0x7f1f00106460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:36:29.761 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:29.004+0000 7f1f056a7700 1 -- 172.21.15.5:0/2762304716 shutdown_connections 2024-05-09T00:36:29.761 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:29.004+0000 7f1f056a7700 1 -- 172.21.15.5:0/2762304716 wait complete. 2024-05-09T00:36:29.761 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:29.004+0000 7f1f056a7700 1 Processor -- start 2024-05-09T00:36:29.761 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:29.005+0000 7f1f056a7700 1 -- start start 2024-05-09T00:36:29.761 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:29.005+0000 7f1f056a7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f00108970 0x7f1f0007d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:36:29.761 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:29.005+0000 7f1f056a7700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1f0010cef0 con 0x7f1f00108970 2024-05-09T00:36:29.761 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:29.005+0000 7f1efeffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f00108970 0x7f1f0007d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:36:29.761 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:29.005+0000 7f1efeffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f00108970 0x7f1f0007d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54118/0 (socket says 172.21.15.5:54118) 2024-05-09T00:36:29.762 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:29.005+0000 7f1efeffd700 1 -- 172.21.15.5:0/3055502259 learned_addr learned my addr 172.21.15.5:0/3055502259 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:36:29.762 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:29.005+0000 7f1efeffd700 1 -- 172.21.15.5:0/3055502259 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f0007d710 con 0x7f1f00108970 2024-05-09T00:36:29.762 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:29.005+0000 7f1efeffd700 1 --2- 172.21.15.5:0/3055502259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f00108970 0x7f1f0007d1d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0009fd0 tx=0x7f1ef000bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:36:29.762 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:29.006+0000 7f1ef7fff700 1 -- 172.21.15.5:0/3055502259 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ef0020020 con 0x7f1f00108970 2024-05-09T00:36:29.762 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:29.006+0000 7f1ef7fff700 1 -- 172.21.15.5:0/3055502259 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ef0012430 con 0x7f1f00108970 2024-05-09T00:36:29.762 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:29.006+0000 7f1f056a7700 1 -- 172.21.15.5:0/3055502259 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f001a0d10 con 0x7f1f00108970 2024-05-09T00:36:29.762 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:29.006+0000 7f1ef7fff700 1 -- 172.21.15.5:0/3055502259 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ef0017040 con 0x7f1f00108970 2024-05-09T00:36:29.762 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:29.006+0000 7f1f056a7700 1 -- 172.21.15.5:0/3055502259 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f001a11f0 con 0x7f1f00108970 2024-05-09T00:36:29.762 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:29.007+0000 7f1ef7fff700 1 -- 172.21.15.5:0/3055502259 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f1ef0023020 con 0x7f1f00108970 2024-05-09T00:36:29.762 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:29.007+0000 7f1f056a7700 1 -- 172.21.15.5:0/3055502259 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f00064e20 con 0x7f1f00108970 2024-05-09T00:36:29.762 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:29.007+0000 7f1ef7fff700 1 --2- 172.21.15.5:0/3055502259 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f1eec03e5b0 0x7f1eec040ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:36:29.762 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:29.007+0000 7f1ef7fff700 1 -- 172.21.15.5:0/3055502259 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f1ef000f050 con 0x7f1f00108970 2024-05-09T00:36:29.762 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:29.007+0000 7f1efe7fc700 1 --2- 172.21.15.5:0/3055502259 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f1eec03e5b0 0x7f1eec040ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:36:29.763 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:29.008+0000 7f1efe7fc700 1 --2- 172.21.15.5:0/3055502259 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f1eec03e5b0 0x7f1eec040ac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8006fd0 tx=0x7f1ee8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:36:29.763 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:29.012+0000 7f1ef7fff700 1 -- 172.21.15.5:0/3055502259 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1ef002a020 con 0x7f1f00108970 2024-05-09T00:36:29.763 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:29.226+0000 7f1f056a7700 1 -- 172.21.15.5:0/3055502259 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f1f001a1a70 con 0x7f1f00108970 2024-05-09T00:36:29.763 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:29.226+0000 7f1ef7fff700 1 -- 172.21.15.5:0/3055502259 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f1ef002dd60 con 0x7f1f00108970 2024-05-09T00:36:29.763 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:29.229+0000 7f1f056a7700 1 -- 172.21.15.5:0/3055502259 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f1eec03e5b0 msgr2=0x7f1eec040ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:36:29.763 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:29.229+0000 7f1f056a7700 1 --2- 172.21.15.5:0/3055502259 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f1eec03e5b0 0x7f1eec040ac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8006fd0 tx=0x7f1ee8006e40 comp rx=0 tx=0).stop 2024-05-09T00:36:29.763 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:29.229+0000 7f1f056a7700 1 -- 172.21.15.5:0/3055502259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f00108970 msgr2=0x7f1f0007d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:36:29.763 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:29.229+0000 7f1f056a7700 1 --2- 172.21.15.5:0/3055502259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f00108970 0x7f1f0007d1d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0009fd0 tx=0x7f1ef000bd50 comp rx=0 tx=0).stop 2024-05-09T00:36:29.763 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:29.229+0000 7f1f056a7700 1 -- 172.21.15.5:0/3055502259 shutdown_connections 2024-05-09T00:36:29.763 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:29.229+0000 7f1f056a7700 1 --2- 172.21.15.5:0/3055502259 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f1eec03e5b0 0x7f1eec040ac0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:36:29.764 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:29.229+0000 7f1f056a7700 1 --2- 172.21.15.5:0/3055502259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f00108970 0x7f1f0007d1d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:36:29.764 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:29.229+0000 7f1f056a7700 1 -- 172.21.15.5:0/3055502259 >> 172.21.15.5:0/3055502259 conn(0x7f1f00103fb0 msgr2=0x7f1f00104960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:36:29.764 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:29.229+0000 7f1f056a7700 1 -- 172.21.15.5:0/3055502259 shutdown_connections 2024-05-09T00:36:29.764 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:29.229+0000 7f1f056a7700 1 -- 172.21.15.5:0/3055502259 wait complete. 2024-05-09T00:36:31.082 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:30.156+0000 7f44db698700 1 Processor -- start 2024-05-09T00:36:31.083 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:30.156+0000 7f44db698700 1 -- start start 2024-05-09T00:36:31.083 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:30.157+0000 7f44db698700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44d4109a70 0x7f44d4109e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:36:31.083 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:30.157+0000 7f44db698700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f44d410a470 con 0x7f44d4109a70 2024-05-09T00:36:31.083 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:30.157+0000 7f44d9434700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44d4109a70 0x7f44d4109e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:36:31.083 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:30.157+0000 7f44d9434700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44d4109a70 0x7f44d4109e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54130/0 (socket says 172.21.15.5:54130) 2024-05-09T00:36:31.083 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:30.157+0000 7f44d9434700 1 -- 172.21.15.5:0/2504282532 learned_addr learned my addr 172.21.15.5:0/2504282532 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:36:31.083 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:30.157+0000 7f44d9434700 1 -- 172.21.15.5:0/2504282532 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44d410abf0 con 0x7f44d4109a70 2024-05-09T00:36:31.083 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:30.157+0000 7f44d9434700 1 --2- 172.21.15.5:0/2504282532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44d4109a70 0x7f44d4109e80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f44c40098a0 tx=0x7f44c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c063b882a441730e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:36:31.083 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:30.157+0000 7f44d3fff700 1 -- 172.21.15.5:0/2504282532 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f44c4004160 con 0x7f44d4109a70 2024-05-09T00:36:31.083 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:30.157+0000 7f44d3fff700 1 -- 172.21.15.5:0/2504282532 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44c4004300 con 0x7f44d4109a70 2024-05-09T00:36:31.083 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:30.157+0000 7f44d3fff700 1 -- 172.21.15.5:0/2504282532 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f44c4011a20 con 0x7f44d4109a70 2024-05-09T00:36:31.083 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:30.158+0000 7f44db698700 1 -- 172.21.15.5:0/2504282532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44d4109a70 msgr2=0x7f44d4109e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:36:31.083 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:30.158+0000 7f44db698700 1 --2- 172.21.15.5:0/2504282532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44d4109a70 0x7f44d4109e80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f44c40098a0 tx=0x7f44c4009bb0 comp rx=0 tx=0).stop 2024-05-09T00:36:31.084 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:30.158+0000 7f44db698700 1 -- 172.21.15.5:0/2504282532 shutdown_connections 2024-05-09T00:36:31.084 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:30.158+0000 7f44db698700 1 --2- 172.21.15.5:0/2504282532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44d4109a70 0x7f44d4109e80 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:36:31.084 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:30.158+0000 7f44db698700 1 -- 172.21.15.5:0/2504282532 >> 172.21.15.5:0/2504282532 conn(0x7f44d41050d0 msgr2=0x7f44d4107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:36:31.084 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:30.158+0000 7f44db698700 1 -- 172.21.15.5:0/2504282532 shutdown_connections 2024-05-09T00:36:31.084 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:30.158+0000 7f44db698700 1 -- 172.21.15.5:0/2504282532 wait complete. 2024-05-09T00:36:31.084 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:30.159+0000 7f44db698700 1 Processor -- start 2024-05-09T00:36:31.084 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:30.159+0000 7f44db698700 1 -- start start 2024-05-09T00:36:31.084 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:30.159+0000 7f44db698700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44d4109a70 0x7f44d419dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:36:31.084 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:30.159+0000 7f44db698700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f44d410dfa0 con 0x7f44d4109a70 2024-05-09T00:36:31.084 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:30.159+0000 7f44d9434700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44d4109a70 0x7f44d419dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:36:31.084 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:30.159+0000 7f44d9434700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44d4109a70 0x7f44d419dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54140/0 (socket says 172.21.15.5:54140) 2024-05-09T00:36:31.084 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:30.159+0000 7f44d9434700 1 -- 172.21.15.5:0/2035906772 learned_addr learned my addr 172.21.15.5:0/2035906772 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:36:31.084 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:30.160+0000 7f44d9434700 1 -- 172.21.15.5:0/2035906772 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44d419e420 con 0x7f44d4109a70 2024-05-09T00:36:31.085 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:30.160+0000 7f44d9434700 1 --2- 172.21.15.5:0/2035906772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44d4109a70 0x7f44d419dee0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f44c4011a20 tx=0x7f44c4011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:36:31.085 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:30.160+0000 7f44d27fc700 1 -- 172.21.15.5:0/2035906772 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f44c40039c0 con 0x7f44d4109a70 2024-05-09T00:36:31.085 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:30.160+0000 7f44d27fc700 1 -- 172.21.15.5:0/2035906772 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44c4003b60 con 0x7f44d4109a70 2024-05-09T00:36:31.085 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:30.160+0000 7f44db698700 1 -- 172.21.15.5:0/2035906772 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44d419e6b0 con 0x7f44d4109a70 2024-05-09T00:36:31.085 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:30.160+0000 7f44d27fc700 1 -- 172.21.15.5:0/2035906772 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f44c4019700 con 0x7f44d4109a70 2024-05-09T00:36:31.085 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:30.160+0000 7f44db698700 1 -- 172.21.15.5:0/2035906772 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44d41a12c0 con 0x7f44d4109a70 2024-05-09T00:36:31.085 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:30.161+0000 7f44db698700 1 -- 172.21.15.5:0/2035906772 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44d4064e20 con 0x7f44d4109a70 2024-05-09T00:36:31.085 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:30.161+0000 7f44d27fc700 1 -- 172.21.15.5:0/2035906772 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f44c40199b0 con 0x7f44d4109a70 2024-05-09T00:36:31.085 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:30.161+0000 7f44d27fc700 1 --2- 172.21.15.5:0/2035906772 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f44bc03e570 0x7f44bc040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:36:31.085 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:30.162+0000 7f44d27fc700 1 -- 172.21.15.5:0/2035906772 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f44c4053eb0 con 0x7f44d4109a70 2024-05-09T00:36:31.085 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:30.162+0000 7f44d8c33700 1 --2- 172.21.15.5:0/2035906772 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f44bc03e570 0x7f44bc040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:36:31.085 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:30.162+0000 7f44d8c33700 1 --2- 172.21.15.5:0/2035906772 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f44bc03e570 0x7f44bc040a80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f44c8006fd0 tx=0x7f44c8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:36:31.085 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:30.167+0000 7f44d27fc700 1 -- 172.21.15.5:0/2035906772 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f44c4017e90 con 0x7f44d4109a70 2024-05-09T00:36:31.086 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:30.510+0000 7f44db698700 1 -- 172.21.15.5:0/2035906772 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f44d41a15d0 con 0x7f44d4109a70 2024-05-09T00:36:31.086 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:30.513+0000 7f44d27fc700 1 -- 172.21.15.5:0/2035906772 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f44c4019df0 con 0x7f44d4109a70 2024-05-09T00:36:31.086 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:30.515+0000 7f44db698700 1 -- 172.21.15.5:0/2035906772 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f44bc03e570 msgr2=0x7f44bc040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:36:31.086 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:30.515+0000 7f44db698700 1 --2- 172.21.15.5:0/2035906772 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f44bc03e570 0x7f44bc040a80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f44c8006fd0 tx=0x7f44c8006e40 comp rx=0 tx=0).stop 2024-05-09T00:36:31.086 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:30.515+0000 7f44db698700 1 -- 172.21.15.5:0/2035906772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44d4109a70 msgr2=0x7f44d419dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:36:31.086 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:30.515+0000 7f44db698700 1 --2- 172.21.15.5:0/2035906772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44d4109a70 0x7f44d419dee0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f44c4011a20 tx=0x7f44c4011b00 comp rx=0 tx=0).stop 2024-05-09T00:36:31.086 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:30.515+0000 7f44db698700 1 -- 172.21.15.5:0/2035906772 shutdown_connections 2024-05-09T00:36:31.086 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:30.515+0000 7f44db698700 1 --2- 172.21.15.5:0/2035906772 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f44bc03e570 0x7f44bc040a80 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:36:31.086 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:30.515+0000 7f44db698700 1 --2- 172.21.15.5:0/2035906772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44d4109a70 0x7f44d419dee0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:36:31.086 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:30.515+0000 7f44db698700 1 -- 172.21.15.5:0/2035906772 >> 172.21.15.5:0/2035906772 conn(0x7f44d41050d0 msgr2=0x7f44d4105a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:36:31.086 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:30.515+0000 7f44db698700 1 -- 172.21.15.5:0/2035906772 shutdown_connections 2024-05-09T00:36:31.086 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-09T00:36:30.515+0000 7f44db698700 1 -- 172.21.15.5:0/2035906772 wait complete. 2024-05-09T00:36:31.087 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-09T00:36:31.087 INFO:teuthology.orchestra.run.smithi005.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-09T00:36:31.087 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:36:31.087 INFO:teuthology.orchestra.run.smithi005.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-09T00:36:31.087 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:36:31.087 INFO:teuthology.orchestra.run.smithi005.stdout:Or, if you are only running a single cluster on this host: 2024-05-09T00:36:31.087 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:36:31.087 INFO:teuthology.orchestra.run.smithi005.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-09T00:36:31.087 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:36:31.087 INFO:teuthology.orchestra.run.smithi005.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-09T00:36:31.087 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:36:31.087 INFO:teuthology.orchestra.run.smithi005.stdout: ceph telemetry on 2024-05-09T00:36:31.087 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:36:31.087 INFO:teuthology.orchestra.run.smithi005.stdout:For more information see: 2024-05-09T00:36:31.087 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:36:31.088 INFO:teuthology.orchestra.run.smithi005.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-05-09T00:36:31.088 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:36:31.088 INFO:teuthology.orchestra.run.smithi005.stdout:Bootstrap complete. 2024-05-09T00:36:31.123 INFO:tasks.cephadm:Fetching config... 2024-05-09T00:36:31.123 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-09T00:36:31.124 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-09T00:36:31.141 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-09T00:36:31.141 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-09T00:36:31.141 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-09T00:36:31.198 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-09T00:36:31.198 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-09T00:36:31.199 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/keyring of=/dev/stdout 2024-05-09T00:36:31.264 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-09T00:36:31.264 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-09T00:36:31.264 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-09T00:36:31.322 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-09T00:36:31.323 DEBUG:teuthology.orchestra.run.smithi005:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3Q373eMCgV6KQqan02BZzux9ole+ws4jzB8UmGChPUoJpFOElDEG0fJVOFVJpdDQ+5eZKZm3J8hRnfjAWNhT3h8Tdez/uiRi4EcM8op9B0KCL0Z4z5h066fKVQi4dhoxrEfW5T9/hmsC2axHQOb9iUoBEBfZIXY33RgdBFoK8Dy20sQuC2llQ5sS5tCkPipwGChS1NVijs1m4YJzJOzDpW6Q0DcPzbTXNrTWecdYOv5HwjSAE8Qlp6kTXezfeQXFv6+M9tOGCbve2tNxyFvaaDqLz7Vif1CeUccKnOC+hQIQwOnIUYqyoPmd+nvcR3wlTlqY+QNAvLK36hPsS6eG8Jr3ndGmXnT2fVQVRmW4KlhYKu6FSJIzEGN5s8iw/dz5piEvj1qGppDTOow1le/qs702U7IiGPzTdXEjg1wJcrRUhUpUVq0xYAT8CJW22gBwd60Y+c469RhudpxLSbzfO81ePAXa8JJaQkaXORIrG5SXdjt1n2jp8KRtceUgr6TE= ceph-d85814c6-0d9b-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-09T00:36:31.397 INFO:teuthology.orchestra.run.smithi005.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3Q373eMCgV6KQqan02BZzux9ole+ws4jzB8UmGChPUoJpFOElDEG0fJVOFVJpdDQ+5eZKZm3J8hRnfjAWNhT3h8Tdez/uiRi4EcM8op9B0KCL0Z4z5h066fKVQi4dhoxrEfW5T9/hmsC2axHQOb9iUoBEBfZIXY33RgdBFoK8Dy20sQuC2llQ5sS5tCkPipwGChS1NVijs1m4YJzJOzDpW6Q0DcPzbTXNrTWecdYOv5HwjSAE8Qlp6kTXezfeQXFv6+M9tOGCbve2tNxyFvaaDqLz7Vif1CeUccKnOC+hQIQwOnIUYqyoPmd+nvcR3wlTlqY+QNAvLK36hPsS6eG8Jr3ndGmXnT2fVQVRmW4KlhYKu6FSJIzEGN5s8iw/dz5piEvj1qGppDTOow1le/qs702U7IiGPzTdXEjg1wJcrRUhUpUVq0xYAT8CJW22gBwd60Y+c469RhudpxLSbzfO81ePAXa8JJaQkaXORIrG5SXdjt1n2jp8KRtceUgr6TE= ceph-d85814c6-0d9b-11ef-bc97-c7b262605968 2024-05-09T00:36:31.411 DEBUG:teuthology.orchestra.run.smithi131:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3Q373eMCgV6KQqan02BZzux9ole+ws4jzB8UmGChPUoJpFOElDEG0fJVOFVJpdDQ+5eZKZm3J8hRnfjAWNhT3h8Tdez/uiRi4EcM8op9B0KCL0Z4z5h066fKVQi4dhoxrEfW5T9/hmsC2axHQOb9iUoBEBfZIXY33RgdBFoK8Dy20sQuC2llQ5sS5tCkPipwGChS1NVijs1m4YJzJOzDpW6Q0DcPzbTXNrTWecdYOv5HwjSAE8Qlp6kTXezfeQXFv6+M9tOGCbve2tNxyFvaaDqLz7Vif1CeUccKnOC+hQIQwOnIUYqyoPmd+nvcR3wlTlqY+QNAvLK36hPsS6eG8Jr3ndGmXnT2fVQVRmW4KlhYKu6FSJIzEGN5s8iw/dz5piEvj1qGppDTOow1le/qs702U7IiGPzTdXEjg1wJcrRUhUpUVq0xYAT8CJW22gBwd60Y+c469RhudpxLSbzfO81ePAXa8JJaQkaXORIrG5SXdjt1n2jp8KRtceUgr6TE= ceph-d85814c6-0d9b-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-09T00:36:31.450 INFO:teuthology.orchestra.run.smithi131.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3Q373eMCgV6KQqan02BZzux9ole+ws4jzB8UmGChPUoJpFOElDEG0fJVOFVJpdDQ+5eZKZm3J8hRnfjAWNhT3h8Tdez/uiRi4EcM8op9B0KCL0Z4z5h066fKVQi4dhoxrEfW5T9/hmsC2axHQOb9iUoBEBfZIXY33RgdBFoK8Dy20sQuC2llQ5sS5tCkPipwGChS1NVijs1m4YJzJOzDpW6Q0DcPzbTXNrTWecdYOv5HwjSAE8Qlp6kTXezfeQXFv6+M9tOGCbve2tNxyFvaaDqLz7Vif1CeUccKnOC+hQIQwOnIUYqyoPmd+nvcR3wlTlqY+QNAvLK36hPsS6eG8Jr3ndGmXnT2fVQVRmW4KlhYKu6FSJIzEGN5s8iw/dz5piEvj1qGppDTOow1le/qs702U7IiGPzTdXEjg1wJcrRUhUpUVq0xYAT8CJW22gBwd60Y+c469RhudpxLSbzfO81ePAXa8JJaQkaXORIrG5SXdjt1n2jp8KRtceUgr6TE= ceph-d85814c6-0d9b-11ef-bc97-c7b262605968 2024-05-09T00:36:31.461 DEBUG:teuthology.orchestra.run.smithi157:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3Q373eMCgV6KQqan02BZzux9ole+ws4jzB8UmGChPUoJpFOElDEG0fJVOFVJpdDQ+5eZKZm3J8hRnfjAWNhT3h8Tdez/uiRi4EcM8op9B0KCL0Z4z5h066fKVQi4dhoxrEfW5T9/hmsC2axHQOb9iUoBEBfZIXY33RgdBFoK8Dy20sQuC2llQ5sS5tCkPipwGChS1NVijs1m4YJzJOzDpW6Q0DcPzbTXNrTWecdYOv5HwjSAE8Qlp6kTXezfeQXFv6+M9tOGCbve2tNxyFvaaDqLz7Vif1CeUccKnOC+hQIQwOnIUYqyoPmd+nvcR3wlTlqY+QNAvLK36hPsS6eG8Jr3ndGmXnT2fVQVRmW4KlhYKu6FSJIzEGN5s8iw/dz5piEvj1qGppDTOow1le/qs702U7IiGPzTdXEjg1wJcrRUhUpUVq0xYAT8CJW22gBwd60Y+c469RhudpxLSbzfO81ePAXa8JJaQkaXORIrG5SXdjt1n2jp8KRtceUgr6TE= ceph-d85814c6-0d9b-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-09T00:36:31.501 INFO:teuthology.orchestra.run.smithi157.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3Q373eMCgV6KQqan02BZzux9ole+ws4jzB8UmGChPUoJpFOElDEG0fJVOFVJpdDQ+5eZKZm3J8hRnfjAWNhT3h8Tdez/uiRi4EcM8op9B0KCL0Z4z5h066fKVQi4dhoxrEfW5T9/hmsC2axHQOb9iUoBEBfZIXY33RgdBFoK8Dy20sQuC2llQ5sS5tCkPipwGChS1NVijs1m4YJzJOzDpW6Q0DcPzbTXNrTWecdYOv5HwjSAE8Qlp6kTXezfeQXFv6+M9tOGCbve2tNxyFvaaDqLz7Vif1CeUccKnOC+hQIQwOnIUYqyoPmd+nvcR3wlTlqY+QNAvLK36hPsS6eG8Jr3ndGmXnT2fVQVRmW4KlhYKu6FSJIzEGN5s8iw/dz5piEvj1qGppDTOow1le/qs702U7IiGPzTdXEjg1wJcrRUhUpUVq0xYAT8CJW22gBwd60Y+c469RhudpxLSbzfO81ePAXa8JJaQkaXORIrG5SXdjt1n2jp8KRtceUgr6TE= ceph-d85814c6-0d9b-11ef-bc97-c7b262605968 2024-05-09T00:36:31.513 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-09T00:36:31.766 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:36:31.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:31 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/2035906772' entity='client.admin' 2024-05-09T00:36:33.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:33.967+0000 7fa5cbe16700 1 -- 172.21.15.5:0/3930304181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5c4102ae0 msgr2=0x7fa5c4102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:36:33.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:33.967+0000 7fa5cbe16700 1 --2- 172.21.15.5:0/3930304181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5c4102ae0 0x7fa5c4102ef0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa5b400b0a0 tx=0x7fa5b400b3b0 comp rx=0 tx=0).stop 2024-05-09T00:36:33.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:33.967+0000 7fa5cbe16700 1 -- 172.21.15.5:0/3930304181 shutdown_connections 2024-05-09T00:36:33.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:33.967+0000 7fa5cbe16700 1 --2- 172.21.15.5:0/3930304181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5c4102ae0 0x7fa5c4102ef0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:36:33.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:33.967+0000 7fa5cbe16700 1 -- 172.21.15.5:0/3930304181 >> 172.21.15.5:0/3930304181 conn(0x7fa5c40fe140 msgr2=0x7fa5c41005d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:36:33.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:33.967+0000 7fa5cbe16700 1 -- 172.21.15.5:0/3930304181 shutdown_connections 2024-05-09T00:36:33.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:33.967+0000 7fa5cbe16700 1 -- 172.21.15.5:0/3930304181 wait complete. 2024-05-09T00:36:33.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:33.968+0000 7fa5cbe16700 1 Processor -- start 2024-05-09T00:36:33.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:33.968+0000 7fa5cbe16700 1 -- start start 2024-05-09T00:36:33.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:33.968+0000 7fa5cbe16700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5c4102ae0 0x7fa5c406c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:36:33.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:33.968+0000 7fa5cbe16700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c4108aa0 con 0x7fa5c4102ae0 2024-05-09T00:36:33.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:33.969+0000 7fa5c9bb2700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5c4102ae0 0x7fa5c406c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:36:33.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:33.969+0000 7fa5c9bb2700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5c4102ae0 0x7fa5c406c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33378/0 (socket says 172.21.15.5:33378) 2024-05-09T00:36:33.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:33.969+0000 7fa5c9bb2700 1 -- 172.21.15.5:0/3596844764 learned_addr learned my addr 172.21.15.5:0/3596844764 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:36:33.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:33.969+0000 7fa5c9bb2700 1 -- 172.21.15.5:0/3596844764 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5c406c9c0 con 0x7fa5c4102ae0 2024-05-09T00:36:33.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:33.969+0000 7fa5c9bb2700 1 --2- 172.21.15.5:0/3596844764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5c4102ae0 0x7fa5c406c480 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4007670 tx=0x7fa5b4003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:36:33.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:33.969+0000 7fa5c2ffd700 1 -- 172.21.15.5:0/3596844764 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa5b4021020 con 0x7fa5c4102ae0 2024-05-09T00:36:33.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:33.969+0000 7fa5c2ffd700 1 -- 172.21.15.5:0/3596844764 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5b40044b0 con 0x7fa5c4102ae0 2024-05-09T00:36:33.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:33.969+0000 7fa5c2ffd700 1 -- 172.21.15.5:0/3596844764 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa5b4017040 con 0x7fa5c4102ae0 2024-05-09T00:36:33.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:33.969+0000 7fa5cbe16700 1 -- 172.21.15.5:0/3596844764 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5c419c420 con 0x7fa5c4102ae0 2024-05-09T00:36:33.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:33.969+0000 7fa5cbe16700 1 -- 172.21.15.5:0/3596844764 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5c419c880 con 0x7fa5c4102ae0 2024-05-09T00:36:33.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:33.970+0000 7fa5cbe16700 1 -- 172.21.15.5:0/3596844764 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5c4109220 con 0x7fa5c4102ae0 2024-05-09T00:36:33.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:33.970+0000 7fa5c2ffd700 1 -- 172.21.15.5:0/3596844764 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7fa5b4024020 con 0x7fa5c4102ae0 2024-05-09T00:36:33.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:33.970+0000 7fa5c2ffd700 1 --2- 172.21.15.5:0/3596844764 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fa5ac03e5b0 0x7fa5ac040ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:36:33.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:33.971+0000 7fa5c2ffd700 1 -- 172.21.15.5:0/3596844764 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa5b400f050 con 0x7fa5c4102ae0 2024-05-09T00:36:33.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:33.971+0000 7fa5c93b1700 1 --2- 172.21.15.5:0/3596844764 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fa5ac03e5b0 0x7fa5ac040ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:36:33.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:33.971+0000 7fa5c93b1700 1 --2- 172.21.15.5:0/3596844764 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fa5ac03e5b0 0x7fa5ac040ac0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8006fd0 tx=0x7fa5b8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:36:33.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:33.976+0000 7fa5c2ffd700 1 -- 172.21.15.5:0/3596844764 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa5b402b020 con 0x7fa5c4102ae0 2024-05-09T00:36:34.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:34.198+0000 7fa5cbe16700 1 -- 172.21.15.5:0/3596844764 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fa5c402e840 con 0x7fa5c4102ae0 2024-05-09T00:36:34.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:34.201+0000 7fa5c2ffd700 1 -- 172.21.15.5:0/3596844764 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fa5b401ce90 con 0x7fa5c4102ae0 2024-05-09T00:36:34.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:34.204+0000 7fa5cbe16700 1 -- 172.21.15.5:0/3596844764 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fa5ac03e5b0 msgr2=0x7fa5ac040ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:36:34.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:34.204+0000 7fa5cbe16700 1 --2- 172.21.15.5:0/3596844764 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fa5ac03e5b0 0x7fa5ac040ac0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8006fd0 tx=0x7fa5b8006e40 comp rx=0 tx=0).stop 2024-05-09T00:36:34.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:34.204+0000 7fa5cbe16700 1 -- 172.21.15.5:0/3596844764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5c4102ae0 msgr2=0x7fa5c406c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:36:34.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:34.204+0000 7fa5cbe16700 1 --2- 172.21.15.5:0/3596844764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5c4102ae0 0x7fa5c406c480 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4007670 tx=0x7fa5b4003ce0 comp rx=0 tx=0).stop 2024-05-09T00:36:34.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:34.204+0000 7fa5cbe16700 1 -- 172.21.15.5:0/3596844764 shutdown_connections 2024-05-09T00:36:34.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:34.204+0000 7fa5cbe16700 1 --2- 172.21.15.5:0/3596844764 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fa5ac03e5b0 0x7fa5ac040ac0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:36:34.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:34.204+0000 7fa5cbe16700 1 --2- 172.21.15.5:0/3596844764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5c4102ae0 0x7fa5c406c480 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:36:34.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:34.204+0000 7fa5cbe16700 1 -- 172.21.15.5:0/3596844764 >> 172.21.15.5:0/3596844764 conn(0x7fa5c40fe140 msgr2=0x7fa5c40fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:36:34.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:34.205+0000 7fa5cbe16700 1 -- 172.21.15.5:0/3596844764 shutdown_connections 2024-05-09T00:36:34.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:34.205+0000 7fa5cbe16700 1 -- 172.21.15.5:0/3596844764 wait complete. 2024-05-09T00:36:34.370 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:34 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:36:34.370 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:34 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:36:34.370 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:34 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi005", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:36:34.371 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:34 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:36:34.371 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:34 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:36:34.371 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:34 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:36:35.035 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-09T00:36:35.035 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-09T00:36:35.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:35 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/3596844764' entity='client.admin' 2024-05-09T00:36:35.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:35 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:36:35.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:35 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:36:35.673 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:36:37.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:37.184+0000 7f80b6205700 1 -- 172.21.15.5:0/2844229336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80b0102b10 msgr2=0x7f80b0102f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:36:37.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:37.184+0000 7f80b6205700 1 --2- 172.21.15.5:0/2844229336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80b0102b10 0x7f80b0102f20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f80a0009910 tx=0x7f80a0009c20 comp rx=0 tx=0).stop 2024-05-09T00:36:37.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:37.184+0000 7f80b6205700 1 -- 172.21.15.5:0/2844229336 shutdown_connections 2024-05-09T00:36:37.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:37.184+0000 7f80b6205700 1 --2- 172.21.15.5:0/2844229336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80b0102b10 0x7f80b0102f20 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:36:37.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:37.184+0000 7f80b6205700 1 -- 172.21.15.5:0/2844229336 >> 172.21.15.5:0/2844229336 conn(0x7f80b00fe130 msgr2=0x7f80b0100600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:36:37.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:37.184+0000 7f80b6205700 1 -- 172.21.15.5:0/2844229336 shutdown_connections 2024-05-09T00:36:37.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:37.184+0000 7f80b6205700 1 -- 172.21.15.5:0/2844229336 wait complete. 2024-05-09T00:36:37.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:37.185+0000 7f80b6205700 1 Processor -- start 2024-05-09T00:36:37.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:37.185+0000 7f80b6205700 1 -- start start 2024-05-09T00:36:37.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:37.186+0000 7f80b6205700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80b0102b10 0x7f80b0199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:36:37.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:37.186+0000 7f80b6205700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f80b0108ad0 con 0x7f80b0102b10 2024-05-09T00:36:37.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:37.186+0000 7f80af7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80b0102b10 0x7f80b0199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:36:37.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:37.186+0000 7f80af7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80b0102b10 0x7f80b0199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33400/0 (socket says 172.21.15.5:33400) 2024-05-09T00:36:37.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:37.186+0000 7f80af7fe700 1 -- 172.21.15.5:0/529077508 learned_addr learned my addr 172.21.15.5:0/529077508 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:36:37.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:37.186+0000 7f80af7fe700 1 -- 172.21.15.5:0/529077508 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80b0199c80 con 0x7f80b0102b10 2024-05-09T00:36:37.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:37.186+0000 7f80af7fe700 1 --2- 172.21.15.5:0/529077508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80b0102b10 0x7f80b0199740 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f80a00057d0 tx=0x7f80a0005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:36:37.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:37.186+0000 7f80ad7fa700 1 -- 172.21.15.5:0/529077508 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80a0027070 con 0x7f80b0102b10 2024-05-09T00:36:37.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:37.186+0000 7f80ad7fa700 1 -- 172.21.15.5:0/529077508 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80a000be10 con 0x7f80b0102b10 2024-05-09T00:36:37.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:37.186+0000 7f80b6205700 1 -- 172.21.15.5:0/529077508 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80b0199f10 con 0x7f80b0102b10 2024-05-09T00:36:37.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:37.186+0000 7f80ad7fa700 1 -- 172.21.15.5:0/529077508 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80a0022dd0 con 0x7f80b0102b10 2024-05-09T00:36:37.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:37.186+0000 7f80b6205700 1 -- 172.21.15.5:0/529077508 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80b019cb20 con 0x7f80b0102b10 2024-05-09T00:36:37.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:37.187+0000 7f80ad7fa700 1 -- 172.21.15.5:0/529077508 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f80a002a070 con 0x7f80b0102b10 2024-05-09T00:36:37.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:37.188+0000 7f80ad7fa700 1 --2- 172.21.15.5:0/529077508 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f809c03e4c0 0x7f809c0409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:36:37.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:37.188+0000 7f80b6205700 1 -- 172.21.15.5:0/529077508 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80b0109250 con 0x7f80b0102b10 2024-05-09T00:36:37.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:37.188+0000 7f80ad7fa700 1 -- 172.21.15.5:0/529077508 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f80a001e030 con 0x7f80b0102b10 2024-05-09T00:36:37.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:37.188+0000 7f80a6ffd700 1 --2- 172.21.15.5:0/529077508 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f809c03e4c0 0x7f809c0409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:36:37.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:37.188+0000 7f80a6ffd700 1 --2- 172.21.15.5:0/529077508 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f809c03e4c0 0x7f809c0409d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8098006fd0 tx=0x7f8098006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:36:37.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:37.193+0000 7f80ad7fa700 1 -- 172.21.15.5:0/529077508 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f80a0031070 con 0x7f80b0102b10 2024-05-09T00:36:37.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:37.410+0000 7f80b6205700 1 -- 172.21.15.5:0/529077508 --> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f80b006b500 con 0x7f809c03e4c0 2024-05-09T00:36:37.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:37.414+0000 7f80ad7fa700 1 -- 172.21.15.5:0/529077508 <== mgr.14152 v2:172.21.15.5:6800/91856593 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f80b006b500 con 0x7f809c03e4c0 2024-05-09T00:36:37.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:37.416+0000 7f80b6205700 1 -- 172.21.15.5:0/529077508 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f809c03e4c0 msgr2=0x7f809c0409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:36:37.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:37.416+0000 7f80b6205700 1 --2- 172.21.15.5:0/529077508 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f809c03e4c0 0x7f809c0409d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8098006fd0 tx=0x7f8098006e40 comp rx=0 tx=0).stop 2024-05-09T00:36:37.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:37.416+0000 7f80b6205700 1 -- 172.21.15.5:0/529077508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80b0102b10 msgr2=0x7f80b0199740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:36:37.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:37.416+0000 7f80b6205700 1 --2- 172.21.15.5:0/529077508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80b0102b10 0x7f80b0199740 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f80a00057d0 tx=0x7f80a0005800 comp rx=0 tx=0).stop 2024-05-09T00:36:37.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:37.416+0000 7f80b6205700 1 -- 172.21.15.5:0/529077508 shutdown_connections 2024-05-09T00:36:37.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:37.416+0000 7f80b6205700 1 --2- 172.21.15.5:0/529077508 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f809c03e4c0 0x7f809c0409d0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:36:37.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:37.417+0000 7f80b6205700 1 --2- 172.21.15.5:0/529077508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80b0102b10 0x7f80b0199740 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:36:37.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:37.417+0000 7f80b6205700 1 -- 172.21.15.5:0/529077508 >> 172.21.15.5:0/529077508 conn(0x7f80b00fe130 msgr2=0x7f80b006adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:36:37.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:37.417+0000 7f80b6205700 1 -- 172.21.15.5:0/529077508 shutdown_connections 2024-05-09T00:36:37.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:37.417+0000 7f80b6205700 1 -- 172.21.15.5:0/529077508 wait complete. 2024-05-09T00:36:38.167 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi131 2024-05-09T00:36:38.168 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-09T00:36:38.168 DEBUG:teuthology.orchestra.run.smithi131:> dd of=/etc/ceph/ceph.conf 2024-05-09T00:36:38.188 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-09T00:36:38.188 DEBUG:teuthology.orchestra.run.smithi131:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-09T00:36:38.246 INFO:tasks.cephadm:Adding host smithi131 to orchestrator... 2024-05-09T00:36:38.246 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph orch host add smithi131 2024-05-09T00:36:38.501 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:36:38.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:38 smithi005 ceph-mon[30365]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:36:38.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:38 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:36:38.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:38 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:36:38.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:38 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:36:38.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:38 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:36:38.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:38 smithi005 ceph-mon[30365]: Updating smithi005:/etc/ceph/ceph.conf 2024-05-09T00:36:38.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:38 smithi005 ceph-mon[30365]: Updating smithi005:/var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T00:36:38.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:38 smithi005 ceph-mon[30365]: Updating smithi005:/etc/ceph/ceph.client.admin.keyring 2024-05-09T00:36:39.692 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:39 smithi005 ceph-mon[30365]: Updating smithi005:/var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T00:36:39.692 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:39 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:36:39.692 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:39 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:36:39.692 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:39 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:36:40.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:40.030+0000 7f78351ab700 1 -- 172.21.15.5:0/360192045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7830102a60 msgr2=0x7f7830102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:36:40.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:40.030+0000 7f78351ab700 1 --2- 172.21.15.5:0/360192045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7830102a60 0x7f7830102e70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7824009910 tx=0x7f7824009c20 comp rx=0 tx=0).stop 2024-05-09T00:36:40.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:40.030+0000 7f78351ab700 1 -- 172.21.15.5:0/360192045 shutdown_connections 2024-05-09T00:36:40.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:40.030+0000 7f78351ab700 1 --2- 172.21.15.5:0/360192045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7830102a60 0x7f7830102e70 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:36:40.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:40.030+0000 7f78351ab700 1 -- 172.21.15.5:0/360192045 >> 172.21.15.5:0/360192045 conn(0x7f78300fe5a0 msgr2=0x7f78301009f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:36:40.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:40.030+0000 7f78351ab700 1 -- 172.21.15.5:0/360192045 shutdown_connections 2024-05-09T00:36:40.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:40.030+0000 7f78351ab700 1 -- 172.21.15.5:0/360192045 wait complete. 2024-05-09T00:36:40.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:40.031+0000 7f78351ab700 1 Processor -- start 2024-05-09T00:36:40.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:40.031+0000 7f78351ab700 1 -- start start 2024-05-09T00:36:40.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:40.031+0000 7f78351ab700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7830102a60 0x7f78301a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:36:40.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:40.031+0000 7f78351ab700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7830110320 con 0x7f7830102a60 2024-05-09T00:36:40.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:40.032+0000 7f782ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7830102a60 0x7f78301a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:36:40.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:40.032+0000 7f782ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7830102a60 0x7f78301a1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33426/0 (socket says 172.21.15.5:33426) 2024-05-09T00:36:40.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:40.032+0000 7f782ed9d700 1 -- 172.21.15.5:0/1509900621 learned_addr learned my addr 172.21.15.5:0/1509900621 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:36:40.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:40.032+0000 7f782ed9d700 1 -- 172.21.15.5:0/1509900621 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78301a1ca0 con 0x7f7830102a60 2024-05-09T00:36:40.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:40.032+0000 7f782ed9d700 1 --2- 172.21.15.5:0/1509900621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7830102a60 0x7f78301a1760 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f78240056d0 tx=0x7f78240057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:36:40.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:40.032+0000 7f7817fff700 1 -- 172.21.15.5:0/1509900621 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7824026070 con 0x7f7830102a60 2024-05-09T00:36:40.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:40.032+0000 7f7817fff700 1 -- 172.21.15.5:0/1509900621 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f782400bc00 con 0x7f7830102a60 2024-05-09T00:36:40.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:40.032+0000 7f78351ab700 1 -- 172.21.15.5:0/1509900621 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78301a1f30 con 0x7f7830102a60 2024-05-09T00:36:40.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:40.032+0000 7f7817fff700 1 -- 172.21.15.5:0/1509900621 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7824003920 con 0x7f7830102a60 2024-05-09T00:36:40.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:40.032+0000 7f78351ab700 1 -- 172.21.15.5:0/1509900621 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f783019b980 con 0x7f7830102a60 2024-05-09T00:36:40.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:40.033+0000 7f7817fff700 1 -- 172.21.15.5:0/1509900621 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f7824029070 con 0x7f7830102a60 2024-05-09T00:36:40.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:40.033+0000 7f7817fff700 1 --2- 172.21.15.5:0/1509900621 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f781803e510 0x7f7818040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:36:40.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:40.034+0000 7f7817fff700 1 -- 172.21.15.5:0/1509900621 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f782401e070 con 0x7f7830102a60 2024-05-09T00:36:40.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:40.034+0000 7f78351ab700 1 -- 172.21.15.5:0/1509900621 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f783006c460 con 0x7f7830102a60 2024-05-09T00:36:40.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:40.034+0000 7f782e59c700 1 --2- 172.21.15.5:0/1509900621 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f781803e510 0x7f7818040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:36:40.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:40.034+0000 7f782e59c700 1 --2- 172.21.15.5:0/1509900621 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f781803e510 0x7f7818040a20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7820006fd0 tx=0x7f7820006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:36:40.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:40.039+0000 7f7817fff700 1 -- 172.21.15.5:0/1509900621 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7824030080 con 0x7f7830102a60 2024-05-09T00:36:40.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:40.259+0000 7f78351ab700 1 -- 172.21.15.5:0/1509900621 --> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi131", "target": ["mon-mgr", ""]}) -- 0x7f783019ceb0 con 0x7f781803e510 2024-05-09T00:36:41.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:41 smithi005 ceph-mon[30365]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi131", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:36:41.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:41 smithi005 ceph-mon[30365]: Deploying cephadm binary to smithi131 2024-05-09T00:36:42.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:42 smithi005 ceph-mon[30365]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:36:44.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:44.509+0000 7f7817fff700 1 -- 172.21.15.5:0/1509900621 <== mgr.14152 v2:172.21.15.5:6800/91856593 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f783019ceb0 con 0x7f781803e510 2024-05-09T00:36:44.511 INFO:teuthology.orchestra.run.smithi005.stdout:Added host 'smithi131' with addr '172.21.15.131' 2024-05-09T00:36:44.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:44.512+0000 7f78351ab700 1 -- 172.21.15.5:0/1509900621 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f781803e510 msgr2=0x7f7818040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:36:44.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:44.513+0000 7f78351ab700 1 --2- 172.21.15.5:0/1509900621 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f781803e510 0x7f7818040a20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7820006fd0 tx=0x7f7820006e40 comp rx=0 tx=0).stop 2024-05-09T00:36:44.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:44.513+0000 7f78351ab700 1 -- 172.21.15.5:0/1509900621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7830102a60 msgr2=0x7f78301a1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:36:44.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:44.513+0000 7f78351ab700 1 --2- 172.21.15.5:0/1509900621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7830102a60 0x7f78301a1760 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f78240056d0 tx=0x7f78240057b0 comp rx=0 tx=0).stop 2024-05-09T00:36:44.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:44.513+0000 7f78351ab700 1 -- 172.21.15.5:0/1509900621 shutdown_connections 2024-05-09T00:36:44.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:44.513+0000 7f78351ab700 1 --2- 172.21.15.5:0/1509900621 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f781803e510 0x7f7818040a20 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:36:44.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:44.513+0000 7f78351ab700 1 --2- 172.21.15.5:0/1509900621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7830102a60 0x7f78301a1760 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:36:44.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:44.513+0000 7f78351ab700 1 -- 172.21.15.5:0/1509900621 >> 172.21.15.5:0/1509900621 conn(0x7f78300fe5a0 msgr2=0x7f7830100560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:36:44.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:44.513+0000 7f78351ab700 1 -- 172.21.15.5:0/1509900621 shutdown_connections 2024-05-09T00:36:44.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:44.513+0000 7f78351ab700 1 -- 172.21.15.5:0/1509900621 wait complete. 2024-05-09T00:36:45.114 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:44 smithi005 ceph-mon[30365]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:36:45.114 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:44 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:36:45.114 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:44 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:36:45.149 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph orch host ls --format=json 2024-05-09T00:36:45.401 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:36:46.053 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:46 smithi005 ceph-mon[30365]: Added host smithi131 2024-05-09T00:36:46.053 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:46 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:36:46.054 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:46 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:36:46.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:46.888+0000 7f6f5bfcc700 1 -- 172.21.15.5:0/3355574493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f54108ac0 msgr2=0x7f6f54108ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:36:46.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:46.888+0000 7f6f5bfcc700 1 --2- 172.21.15.5:0/3355574493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f54108ac0 0x7f6f54108ed0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6f50009910 tx=0x7f6f50009c20 comp rx=0 tx=0).stop 2024-05-09T00:36:46.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:46.888+0000 7f6f5bfcc700 1 -- 172.21.15.5:0/3355574493 shutdown_connections 2024-05-09T00:36:46.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:46.888+0000 7f6f5bfcc700 1 --2- 172.21.15.5:0/3355574493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f54108ac0 0x7f6f54108ed0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:36:46.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:46.888+0000 7f6f5bfcc700 1 -- 172.21.15.5:0/3355574493 >> 172.21.15.5:0/3355574493 conn(0x7f6f540fe5b0 msgr2=0x7f6f54100a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:36:46.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:46.888+0000 7f6f5bfcc700 1 -- 172.21.15.5:0/3355574493 shutdown_connections 2024-05-09T00:36:46.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:46.888+0000 7f6f5bfcc700 1 -- 172.21.15.5:0/3355574493 wait complete. 2024-05-09T00:36:46.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:46.889+0000 7f6f5bfcc700 1 Processor -- start 2024-05-09T00:36:46.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:46.889+0000 7f6f5bfcc700 1 -- start start 2024-05-09T00:36:46.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:46.889+0000 7f6f5bfcc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f54108ac0 0x7f6f540ff550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:36:46.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:46.889+0000 7f6f5bfcc700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6f5410d010 con 0x7f6f54108ac0 2024-05-09T00:36:46.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:46.889+0000 7f6f59d68700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f54108ac0 0x7f6f540ff550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:36:46.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:46.889+0000 7f6f59d68700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f54108ac0 0x7f6f540ff550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47184/0 (socket says 172.21.15.5:47184) 2024-05-09T00:36:46.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:46.889+0000 7f6f59d68700 1 -- 172.21.15.5:0/3957844832 learned_addr learned my addr 172.21.15.5:0/3957844832 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:36:46.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:46.889+0000 7f6f59d68700 1 -- 172.21.15.5:0/3957844832 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f541013a0 con 0x7f6f54108ac0 2024-05-09T00:36:46.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:46.890+0000 7f6f59d68700 1 --2- 172.21.15.5:0/3957844832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f54108ac0 0x7f6f540ff550 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6f500056d0 tx=0x7f6f500057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:36:46.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:46.890+0000 7f6f4affd700 1 -- 172.21.15.5:0/3957844832 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f50027070 con 0x7f6f54108ac0 2024-05-09T00:36:46.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:46.890+0000 7f6f4affd700 1 -- 172.21.15.5:0/3957844832 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f5000be10 con 0x7f6f54108ac0 2024-05-09T00:36:46.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:46.890+0000 7f6f5bfcc700 1 -- 172.21.15.5:0/3957844832 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f540ffa90 con 0x7f6f54108ac0 2024-05-09T00:36:46.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:46.890+0000 7f6f4affd700 1 -- 172.21.15.5:0/3957844832 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f50022d20 con 0x7f6f54108ac0 2024-05-09T00:36:46.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:46.890+0000 7f6f5bfcc700 1 -- 172.21.15.5:0/3957844832 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f540fff50 con 0x7f6f54108ac0 2024-05-09T00:36:46.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:46.891+0000 7f6f5bfcc700 1 -- 172.21.15.5:0/3957844832 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f541041d0 con 0x7f6f54108ac0 2024-05-09T00:36:46.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:46.891+0000 7f6f4affd700 1 -- 172.21.15.5:0/3957844832 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f6f5002a070 con 0x7f6f54108ac0 2024-05-09T00:36:46.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:46.891+0000 7f6f4affd700 1 --2- 172.21.15.5:0/3957844832 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f6f4003e510 0x7f6f40040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:36:46.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:46.891+0000 7f6f4affd700 1 -- 172.21.15.5:0/3957844832 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6f5001e030 con 0x7f6f54108ac0 2024-05-09T00:36:46.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:46.891+0000 7f6f59567700 1 --2- 172.21.15.5:0/3957844832 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f6f4003e510 0x7f6f40040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:36:46.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:46.892+0000 7f6f59567700 1 --2- 172.21.15.5:0/3957844832 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f6f4003e510 0x7f6f40040a20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6f44006fd0 tx=0x7f6f44006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:36:46.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:46.897+0000 7f6f4affd700 1 -- 172.21.15.5:0/3957844832 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6f50013070 con 0x7f6f54108ac0 2024-05-09T00:36:47.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:47.116+0000 7f6f5bfcc700 1 -- 172.21.15.5:0/3957844832 --> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6f54100a80 con 0x7f6f4003e510 2024-05-09T00:36:47.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:47.117+0000 7f6f4affd700 1 -- 172.21.15.5:0/3957844832 <== mgr.14152 v2:172.21.15.5:6800/91856593 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7f6f54100a80 con 0x7f6f4003e510 2024-05-09T00:36:47.118 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:36:47.118 INFO:teuthology.orchestra.run.smithi005.stdout:[{"addr": "172.21.15.5", "hostname": "smithi005", "labels": [], "status": ""}, {"addr": "172.21.15.131", "hostname": "smithi131", "labels": [], "status": ""}] 2024-05-09T00:36:47.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:47.119+0000 7f6f5bfcc700 1 -- 172.21.15.5:0/3957844832 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f6f4003e510 msgr2=0x7f6f40040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:36:47.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:47.119+0000 7f6f5bfcc700 1 --2- 172.21.15.5:0/3957844832 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f6f4003e510 0x7f6f40040a20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6f44006fd0 tx=0x7f6f44006e40 comp rx=0 tx=0).stop 2024-05-09T00:36:47.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:47.119+0000 7f6f5bfcc700 1 -- 172.21.15.5:0/3957844832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f54108ac0 msgr2=0x7f6f540ff550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:36:47.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:47.119+0000 7f6f5bfcc700 1 --2- 172.21.15.5:0/3957844832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f54108ac0 0x7f6f540ff550 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6f500056d0 tx=0x7f6f500057b0 comp rx=0 tx=0).stop 2024-05-09T00:36:47.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:47.119+0000 7f6f5bfcc700 1 -- 172.21.15.5:0/3957844832 shutdown_connections 2024-05-09T00:36:47.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:47.119+0000 7f6f5bfcc700 1 --2- 172.21.15.5:0/3957844832 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f6f4003e510 0x7f6f40040a20 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:36:47.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:47.119+0000 7f6f5bfcc700 1 --2- 172.21.15.5:0/3957844832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f54108ac0 0x7f6f540ff550 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:36:47.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:47.119+0000 7f6f5bfcc700 1 -- 172.21.15.5:0/3957844832 >> 172.21.15.5:0/3957844832 conn(0x7f6f540fe5b0 msgr2=0x7f6f541065a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:36:47.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:47.119+0000 7f6f5bfcc700 1 -- 172.21.15.5:0/3957844832 shutdown_connections 2024-05-09T00:36:47.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:47.119+0000 7f6f5bfcc700 1 -- 172.21.15.5:0/3957844832 wait complete. 2024-05-09T00:36:47.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:47 smithi005 ceph-mon[30365]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:36:47.745 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi157 2024-05-09T00:36:47.745 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-05-09T00:36:47.745 DEBUG:teuthology.orchestra.run.smithi157:> dd of=/etc/ceph/ceph.conf 2024-05-09T00:36:47.764 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-05-09T00:36:47.764 DEBUG:teuthology.orchestra.run.smithi157:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-09T00:36:47.822 INFO:tasks.cephadm:Adding host smithi157 to orchestrator... 2024-05-09T00:36:47.823 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph orch host add smithi157 2024-05-09T00:36:48.068 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:36:48.413 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:48 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:36:48.413 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:48 smithi005 ceph-mon[30365]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T00:36:49.353 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:49 smithi005 ceph-mon[30365]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:36:49.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:49.614+0000 7fb6edad8700 1 -- 172.21.15.5:0/4218509613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6e8109c30 msgr2=0x7fb6e810a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:36:49.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:49.614+0000 7fb6edad8700 1 --2- 172.21.15.5:0/4218509613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6e8109c30 0x7fb6e810a040 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb6d8009910 tx=0x7fb6d8009c20 comp rx=0 tx=0).stop 2024-05-09T00:36:49.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:49.614+0000 7fb6edad8700 1 -- 172.21.15.5:0/4218509613 shutdown_connections 2024-05-09T00:36:49.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:49.614+0000 7fb6edad8700 1 --2- 172.21.15.5:0/4218509613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6e8109c30 0x7fb6e810a040 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:36:49.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:49.614+0000 7fb6edad8700 1 -- 172.21.15.5:0/4218509613 >> 172.21.15.5:0/4218509613 conn(0x7fb6e80ff670 msgr2=0x7fb6e8101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:36:49.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:49.614+0000 7fb6edad8700 1 -- 172.21.15.5:0/4218509613 shutdown_connections 2024-05-09T00:36:49.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:49.614+0000 7fb6edad8700 1 -- 172.21.15.5:0/4218509613 wait complete. 2024-05-09T00:36:49.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:49.614+0000 7fb6edad8700 1 Processor -- start 2024-05-09T00:36:49.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:49.615+0000 7fb6edad8700 1 -- start start 2024-05-09T00:36:49.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:49.615+0000 7fb6edad8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6e8109c30 0x7fb6e8075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:36:49.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:49.615+0000 7fb6edad8700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e810e110 con 0x7fb6e8109c30 2024-05-09T00:36:49.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:49.615+0000 7fb6e77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6e8109c30 0x7fb6e8075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:36:49.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:49.615+0000 7fb6e77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6e8109c30 0x7fb6e8075ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47200/0 (socket says 172.21.15.5:47200) 2024-05-09T00:36:49.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:49.615+0000 7fb6e77fe700 1 -- 172.21.15.5:0/2628573201 learned_addr learned my addr 172.21.15.5:0/2628573201 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:36:49.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:49.615+0000 7fb6e77fe700 1 -- 172.21.15.5:0/2628573201 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6e8077910 con 0x7fb6e8109c30 2024-05-09T00:36:49.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:49.616+0000 7fb6e77fe700 1 --2- 172.21.15.5:0/2628573201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6e8109c30 0x7fb6e8075ac0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb6d80056d0 tx=0x7fb6d80057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:36:49.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:49.616+0000 7fb6e4ff9700 1 -- 172.21.15.5:0/2628573201 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb6d8027070 con 0x7fb6e8109c30 2024-05-09T00:36:49.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:49.616+0000 7fb6e4ff9700 1 -- 172.21.15.5:0/2628573201 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6d800be10 con 0x7fb6e8109c30 2024-05-09T00:36:49.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:49.616+0000 7fb6e4ff9700 1 -- 172.21.15.5:0/2628573201 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb6d8022d20 con 0x7fb6e8109c30 2024-05-09T00:36:49.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:49.616+0000 7fb6edad8700 1 -- 172.21.15.5:0/2628573201 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6e8076000 con 0x7fb6e8109c30 2024-05-09T00:36:49.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:49.616+0000 7fb6edad8700 1 -- 172.21.15.5:0/2628573201 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6e8076400 con 0x7fb6e8109c30 2024-05-09T00:36:49.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:49.617+0000 7fb6edad8700 1 -- 172.21.15.5:0/2628573201 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6e8105200 con 0x7fb6e8109c30 2024-05-09T00:36:49.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:49.617+0000 7fb6e4ff9700 1 -- 172.21.15.5:0/2628573201 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7fb6d802a070 con 0x7fb6e8109c30 2024-05-09T00:36:49.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:49.617+0000 7fb6e4ff9700 1 --2- 172.21.15.5:0/2628573201 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fb6d003e510 0x7fb6d0040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:36:49.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:49.617+0000 7fb6e4ff9700 1 -- 172.21.15.5:0/2628573201 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb6d801e030 con 0x7fb6e8109c30 2024-05-09T00:36:49.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:49.617+0000 7fb6e6ffd700 1 --2- 172.21.15.5:0/2628573201 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fb6d003e510 0x7fb6d0040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:36:49.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:49.618+0000 7fb6e6ffd700 1 --2- 172.21.15.5:0/2628573201 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fb6d003e510 0x7fb6d0040a20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc006fd0 tx=0x7fb6dc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:36:49.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:49.623+0000 7fb6e4ff9700 1 -- 172.21.15.5:0/2628573201 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb6d8013070 con 0x7fb6e8109c30 2024-05-09T00:36:49.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:49.844+0000 7fb6edad8700 1 -- 172.21.15.5:0/2628573201 --> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi157", "target": ["mon-mgr", ""]}) -- 0x7fb6e8076680 con 0x7fb6d003e510 2024-05-09T00:36:51.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:51 smithi005 ceph-mon[30365]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:36:51.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:51 smithi005 ceph-mon[30365]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi157", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:36:51.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:51 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:36:51.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:51 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:36:51.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:51 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:36:51.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:51 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:36:51.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:51 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi131", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:36:51.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:51 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:36:51.543 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:51 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:36:52.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:52 smithi005 ceph-mon[30365]: Updating smithi131:/etc/ceph/ceph.conf 2024-05-09T00:36:52.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:52 smithi005 ceph-mon[30365]: Deploying cephadm binary to smithi157 2024-05-09T00:36:52.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:52 smithi005 ceph-mon[30365]: Updating smithi131:/var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T00:36:52.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:52 smithi005 ceph-mon[30365]: Updating smithi131:/etc/ceph/ceph.client.admin.keyring 2024-05-09T00:36:53.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:53 smithi005 ceph-mon[30365]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:36:54.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:54.149+0000 7fb6e4ff9700 1 -- 172.21.15.5:0/2628573201 <== mgr.14152 v2:172.21.15.5:6800/91856593 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fb6e8076680 con 0x7fb6d003e510 2024-05-09T00:36:54.151 INFO:teuthology.orchestra.run.smithi005.stdout:Added host 'smithi157' with addr '172.21.15.157' 2024-05-09T00:36:54.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:54.152+0000 7fb6edad8700 1 -- 172.21.15.5:0/2628573201 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fb6d003e510 msgr2=0x7fb6d0040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:36:54.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:54.152+0000 7fb6edad8700 1 --2- 172.21.15.5:0/2628573201 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fb6d003e510 0x7fb6d0040a20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc006fd0 tx=0x7fb6dc006e40 comp rx=0 tx=0).stop 2024-05-09T00:36:54.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:54.152+0000 7fb6edad8700 1 -- 172.21.15.5:0/2628573201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6e8109c30 msgr2=0x7fb6e8075ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:36:54.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:54.152+0000 7fb6edad8700 1 --2- 172.21.15.5:0/2628573201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6e8109c30 0x7fb6e8075ac0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb6d80056d0 tx=0x7fb6d80057b0 comp rx=0 tx=0).stop 2024-05-09T00:36:54.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:54.152+0000 7fb6edad8700 1 -- 172.21.15.5:0/2628573201 shutdown_connections 2024-05-09T00:36:54.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:54.152+0000 7fb6edad8700 1 --2- 172.21.15.5:0/2628573201 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fb6d003e510 0x7fb6d0040a20 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:36:54.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:54.152+0000 7fb6edad8700 1 --2- 172.21.15.5:0/2628573201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6e8109c30 0x7fb6e8075ac0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:36:54.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:54.152+0000 7fb6edad8700 1 -- 172.21.15.5:0/2628573201 >> 172.21.15.5:0/2628573201 conn(0x7fb6e80ff670 msgr2=0x7fb6e8100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:36:54.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:54.152+0000 7fb6edad8700 1 -- 172.21.15.5:0/2628573201 shutdown_connections 2024-05-09T00:36:54.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:54.152+0000 7fb6edad8700 1 -- 172.21.15.5:0/2628573201 wait complete. 2024-05-09T00:36:54.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:54 smithi005 ceph-mon[30365]: Updating smithi131:/var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T00:36:54.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:54 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:36:54.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:54 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:36:54.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:54 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:36:54.841 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph orch host ls --format=json 2024-05-09T00:36:55.090 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:36:55.468 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:55 smithi005 ceph-mon[30365]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:36:55.468 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:55 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:36:55.468 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:55 smithi005 ceph-mon[30365]: Added host smithi157 2024-05-09T00:36:55.468 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:55 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:36:55.468 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:55 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:36:56.271 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:56 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:36:56.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:56.515+0000 7f950451b700 1 -- 172.21.15.5:0/4135862351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94fc103dd0 msgr2=0x7f94fc1041e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:36:56.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:56.515+0000 7f950451b700 1 --2- 172.21.15.5:0/4135862351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94fc103dd0 0x7f94fc1041e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f94f800b0a0 tx=0x7f94f800b3b0 comp rx=0 tx=0).stop 2024-05-09T00:36:56.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:56.516+0000 7f950451b700 1 -- 172.21.15.5:0/4135862351 shutdown_connections 2024-05-09T00:36:56.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:56.516+0000 7f950451b700 1 --2- 172.21.15.5:0/4135862351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94fc103dd0 0x7f94fc1041e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:36:56.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:56.516+0000 7f950451b700 1 -- 172.21.15.5:0/4135862351 >> 172.21.15.5:0/4135862351 conn(0x7f94fc0ff660 msgr2=0x7f94fc101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:36:56.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:56.516+0000 7f950451b700 1 -- 172.21.15.5:0/4135862351 shutdown_connections 2024-05-09T00:36:56.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:56.516+0000 7f950451b700 1 -- 172.21.15.5:0/4135862351 wait complete. 2024-05-09T00:36:56.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:56.516+0000 7f950451b700 1 Processor -- start 2024-05-09T00:36:56.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:56.516+0000 7f950451b700 1 -- start start 2024-05-09T00:36:56.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:56.517+0000 7f950451b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94fc103dd0 0x7f94fc075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:36:56.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:56.517+0000 7f950451b700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f94fc10e0c0 con 0x7f94fc103dd0 2024-05-09T00:36:56.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:56.517+0000 7f95022b7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94fc103dd0 0x7f94fc075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:36:56.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:56.517+0000 7f95022b7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94fc103dd0 0x7f94fc075ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51242/0 (socket says 172.21.15.5:51242) 2024-05-09T00:36:56.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:56.517+0000 7f95022b7700 1 -- 172.21.15.5:0/556716493 learned_addr learned my addr 172.21.15.5:0/556716493 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:36:56.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:56.517+0000 7f95022b7700 1 -- 172.21.15.5:0/556716493 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94fc077910 con 0x7f94fc103dd0 2024-05-09T00:36:56.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:56.517+0000 7f95022b7700 1 --2- 172.21.15.5:0/556716493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94fc103dd0 0x7f94fc075ac0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f94f8006130 tx=0x7f94f80097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:36:56.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:56.517+0000 7f94f37fe700 1 -- 172.21.15.5:0/556716493 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f94f8021020 con 0x7f94fc103dd0 2024-05-09T00:36:56.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:56.517+0000 7f94f37fe700 1 -- 172.21.15.5:0/556716493 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94f80079b0 con 0x7f94fc103dd0 2024-05-09T00:36:56.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:56.517+0000 7f950451b700 1 -- 172.21.15.5:0/556716493 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94fc076000 con 0x7f94fc103dd0 2024-05-09T00:36:56.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:56.517+0000 7f94f37fe700 1 -- 172.21.15.5:0/556716493 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f94f8017040 con 0x7f94fc103dd0 2024-05-09T00:36:56.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:56.518+0000 7f950451b700 1 -- 172.21.15.5:0/556716493 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94fc076460 con 0x7f94fc103dd0 2024-05-09T00:36:56.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:56.518+0000 7f950451b700 1 -- 172.21.15.5:0/556716493 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94fc105280 con 0x7f94fc103dd0 2024-05-09T00:36:56.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:56.518+0000 7f94f37fe700 1 -- 172.21.15.5:0/556716493 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f94f8024020 con 0x7f94fc103dd0 2024-05-09T00:36:56.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:56.519+0000 7f94f37fe700 1 --2- 172.21.15.5:0/556716493 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f94e803e570 0x7f94e8040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:36:56.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:56.519+0000 7f94f37fe700 1 -- 172.21.15.5:0/556716493 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f94f800f050 con 0x7f94fc103dd0 2024-05-09T00:36:56.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:56.519+0000 7f9501ab6700 1 --2- 172.21.15.5:0/556716493 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f94e803e570 0x7f94e8040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:36:56.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:56.519+0000 7f9501ab6700 1 --2- 172.21.15.5:0/556716493 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f94e803e570 0x7f94e8040a80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f94ec006fd0 tx=0x7f94ec006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:36:56.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:56.524+0000 7f94f37fe700 1 -- 172.21.15.5:0/556716493 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f94f802b020 con 0x7f94fc103dd0 2024-05-09T00:36:56.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:56.745+0000 7f950451b700 1 -- 172.21.15.5:0/556716493 --> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f94fc077090 con 0x7f94e803e570 2024-05-09T00:36:56.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:56.747+0000 7f94f37fe700 1 -- 172.21.15.5:0/556716493 <== mgr.14152 v2:172.21.15.5:6800/91856593 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7f94fc077090 con 0x7f94e803e570 2024-05-09T00:36:56.748 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:36:56.748 INFO:teuthology.orchestra.run.smithi005.stdout:[{"addr": "172.21.15.5", "hostname": "smithi005", "labels": [], "status": ""}, {"addr": "172.21.15.131", "hostname": "smithi131", "labels": [], "status": ""}, {"addr": "172.21.15.157", "hostname": "smithi157", "labels": [], "status": ""}] 2024-05-09T00:36:56.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:56.749+0000 7f950451b700 1 -- 172.21.15.5:0/556716493 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f94e803e570 msgr2=0x7f94e8040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:36:56.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:56.749+0000 7f950451b700 1 --2- 172.21.15.5:0/556716493 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f94e803e570 0x7f94e8040a80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f94ec006fd0 tx=0x7f94ec006e40 comp rx=0 tx=0).stop 2024-05-09T00:36:56.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:56.749+0000 7f950451b700 1 -- 172.21.15.5:0/556716493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94fc103dd0 msgr2=0x7f94fc075ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:36:56.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:56.749+0000 7f950451b700 1 --2- 172.21.15.5:0/556716493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94fc103dd0 0x7f94fc075ac0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f94f8006130 tx=0x7f94f80097d0 comp rx=0 tx=0).stop 2024-05-09T00:36:56.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:56.749+0000 7f950451b700 1 -- 172.21.15.5:0/556716493 shutdown_connections 2024-05-09T00:36:56.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:56.749+0000 7f950451b700 1 --2- 172.21.15.5:0/556716493 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f94e803e570 0x7f94e8040a80 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:36:56.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:56.749+0000 7f950451b700 1 --2- 172.21.15.5:0/556716493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94fc103dd0 0x7f94fc075ac0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:36:56.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:56.749+0000 7f950451b700 1 -- 172.21.15.5:0/556716493 >> 172.21.15.5:0/556716493 conn(0x7f94fc0ff660 msgr2=0x7f94fc109900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:36:56.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:56.749+0000 7f950451b700 1 -- 172.21.15.5:0/556716493 shutdown_connections 2024-05-09T00:36:56.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:56.749+0000 7f950451b700 1 -- 172.21.15.5:0/556716493 wait complete. 2024-05-09T00:36:57.336 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:57 smithi005 ceph-mon[30365]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:36:57.336 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:57 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:36:57.365 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-09T00:36:57.366 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph osd crush tunables default 2024-05-09T00:36:57.611 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:36:58.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:58 smithi005 ceph-mon[30365]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T00:36:59.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:59.101+0000 7f1f6d2b8700 1 -- 172.21.15.5:0/4266910993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f68103dd0 msgr2=0x7f1f681041e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:36:59.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:59.101+0000 7f1f6d2b8700 1 --2- 172.21.15.5:0/4266910993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f68103dd0 0x7f1f681041e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1f58009910 tx=0x7f1f58009c20 comp rx=0 tx=0).stop 2024-05-09T00:36:59.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:59.101+0000 7f1f6d2b8700 1 -- 172.21.15.5:0/4266910993 shutdown_connections 2024-05-09T00:36:59.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:59.101+0000 7f1f6d2b8700 1 --2- 172.21.15.5:0/4266910993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f68103dd0 0x7f1f681041e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:36:59.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:59.101+0000 7f1f6d2b8700 1 -- 172.21.15.5:0/4266910993 >> 172.21.15.5:0/4266910993 conn(0x7f1f680ff660 msgr2=0x7f1f68101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:36:59.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:59.101+0000 7f1f6d2b8700 1 -- 172.21.15.5:0/4266910993 shutdown_connections 2024-05-09T00:36:59.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:59.101+0000 7f1f6d2b8700 1 -- 172.21.15.5:0/4266910993 wait complete. 2024-05-09T00:36:59.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:59.102+0000 7f1f6d2b8700 1 Processor -- start 2024-05-09T00:36:59.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:59.102+0000 7f1f6d2b8700 1 -- start start 2024-05-09T00:36:59.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:59.102+0000 7f1f6d2b8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f68103dd0 0x7f1f6819f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:36:59.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:59.102+0000 7f1f6d2b8700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6810e0c0 con 0x7f1f68103dd0 2024-05-09T00:36:59.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:59.102+0000 7f1f66ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f68103dd0 0x7f1f6819f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:36:59.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:59.102+0000 7f1f66ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f68103dd0 0x7f1f6819f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51268/0 (socket says 172.21.15.5:51268) 2024-05-09T00:36:59.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:59.102+0000 7f1f66ffd700 1 -- 172.21.15.5:0/1333314358 learned_addr learned my addr 172.21.15.5:0/1333314358 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:36:59.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:59.103+0000 7f1f66ffd700 1 -- 172.21.15.5:0/1333314358 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f6819fb40 con 0x7f1f68103dd0 2024-05-09T00:36:59.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:59.103+0000 7f1f66ffd700 1 --2- 172.21.15.5:0/1333314358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f68103dd0 0x7f1f6819f600 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1f58006010 tx=0x7f1f58005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:36:59.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:59.103+0000 7f1f4ffff700 1 -- 172.21.15.5:0/1333314358 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f58026070 con 0x7f1f68103dd0 2024-05-09T00:36:59.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:59.103+0000 7f1f4ffff700 1 -- 172.21.15.5:0/1333314358 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f5800bba0 con 0x7f1f68103dd0 2024-05-09T00:36:59.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:59.103+0000 7f1f6d2b8700 1 -- 172.21.15.5:0/1333314358 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f6819fe30 con 0x7f1f68103dd0 2024-05-09T00:36:59.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:59.103+0000 7f1f4ffff700 1 -- 172.21.15.5:0/1333314358 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f58003910 con 0x7f1f68103dd0 2024-05-09T00:36:59.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:59.103+0000 7f1f6d2b8700 1 -- 172.21.15.5:0/1333314358 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f68199960 con 0x7f1f68103dd0 2024-05-09T00:36:59.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:59.104+0000 7f1f6d2b8700 1 -- 172.21.15.5:0/1333314358 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f68105280 con 0x7f1f68103dd0 2024-05-09T00:36:59.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:59.104+0000 7f1f4ffff700 1 -- 172.21.15.5:0/1333314358 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f1f58029070 con 0x7f1f68103dd0 2024-05-09T00:36:59.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:59.105+0000 7f1f4ffff700 1 --2- 172.21.15.5:0/1333314358 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f1f5003e570 0x7f1f50040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:36:59.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:59.105+0000 7f1f4ffff700 1 -- 172.21.15.5:0/1333314358 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f1f5801e070 con 0x7f1f68103dd0 2024-05-09T00:36:59.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:59.105+0000 7f1f667fc700 1 --2- 172.21.15.5:0/1333314358 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f1f5003e570 0x7f1f50040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:36:59.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:59.105+0000 7f1f667fc700 1 --2- 172.21.15.5:0/1333314358 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f1f5003e570 0x7f1f50040a80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c006fd0 tx=0x7f1f5c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:36:59.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:59.110+0000 7f1f4ffff700 1 -- 172.21.15.5:0/1333314358 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1f58013070 con 0x7f1f68103dd0 2024-05-09T00:36:59.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:36:59.334+0000 7f1f6d2b8700 1 -- 172.21.15.5:0/1333314358 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f1f681a0060 con 0x7f1f68103dd0 2024-05-09T00:36:59.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:36:59 smithi005 ceph-mon[30365]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:00.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:00.220+0000 7f1f4ffff700 1 -- 172.21.15.5:0/1333314358 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f1f5800e3e0 con 0x7f1f68103dd0 2024-05-09T00:37:00.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:00.222+0000 7f1f6d2b8700 1 -- 172.21.15.5:0/1333314358 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f1f5003e570 msgr2=0x7f1f50040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:37:00.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:00.222+0000 7f1f6d2b8700 1 --2- 172.21.15.5:0/1333314358 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f1f5003e570 0x7f1f50040a80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c006fd0 tx=0x7f1f5c006e40 comp rx=0 tx=0).stop 2024-05-09T00:37:00.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:00.223+0000 7f1f6d2b8700 1 -- 172.21.15.5:0/1333314358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f68103dd0 msgr2=0x7f1f6819f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:37:00.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:00.223+0000 7f1f6d2b8700 1 --2- 172.21.15.5:0/1333314358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f68103dd0 0x7f1f6819f600 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1f58006010 tx=0x7f1f58005780 comp rx=0 tx=0).stop 2024-05-09T00:37:00.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:00.223+0000 7f1f6d2b8700 1 -- 172.21.15.5:0/1333314358 shutdown_connections 2024-05-09T00:37:00.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:00.223+0000 7f1f6d2b8700 1 --2- 172.21.15.5:0/1333314358 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f1f5003e570 0x7f1f50040a80 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:37:00.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:00.223+0000 7f1f6d2b8700 1 --2- 172.21.15.5:0/1333314358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f68103dd0 0x7f1f6819f600 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:37:00.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:00.223+0000 7f1f6d2b8700 1 -- 172.21.15.5:0/1333314358 >> 172.21.15.5:0/1333314358 conn(0x7f1f680ff660 msgr2=0x7f1f68109900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:37:00.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:00.223+0000 7f1f6d2b8700 1 -- 172.21.15.5:0/1333314358 shutdown_connections 2024-05-09T00:37:00.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:00.223+0000 7f1f6d2b8700 1 -- 172.21.15.5:0/1333314358 wait complete. 2024-05-09T00:37:00.226 INFO:teuthology.orchestra.run.smithi005.stderr:adjusted tunables profile to default 2024-05-09T00:37:00.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:00 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/1333314358' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-09T00:37:00.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:00 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:00.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:00 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:00.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:00 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:00.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:00 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:00.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:00 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi157", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:37:00.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:00 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:37:00.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:00 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:37:00.833 INFO:tasks.cephadm:Adding mon.a on smithi005 2024-05-09T00:37:00.833 INFO:tasks.cephadm:Adding mon.b on smithi131 2024-05-09T00:37:00.834 INFO:tasks.cephadm:Adding mon.c on smithi157 2024-05-09T00:37:00.834 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph orch apply mon '3;smithi005:172.21.15.5=a;smithi131:172.21.15.131=b;smithi157:172.21.15.157=c' 2024-05-09T00:37:01.060 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T00:37:01.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:01 smithi005 ceph-mon[30365]: Updating smithi157:/etc/ceph/ceph.conf 2024-05-09T00:37:01.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:01 smithi005 ceph-mon[30365]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:01.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:01 smithi005 ceph-mon[30365]: Updating smithi157:/var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T00:37:01.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:01 smithi005 ceph-mon[30365]: Updating smithi157:/etc/ceph/ceph.client.admin.keyring 2024-05-09T00:37:01.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:01 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/1333314358' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-09T00:37:01.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:01 smithi005 ceph-mon[30365]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T00:37:01.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:01 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:01.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:01 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:01.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:01 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:02.506 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:02.503+0000 7efe47237700 1 -- 172.21.15.157:0/2814133779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe40108b00 msgr2=0x7efe40108f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:37:02.506 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:02.503+0000 7efe47237700 1 --2- 172.21.15.157:0/2814133779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe40108b00 0x7efe40108f10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7efe3800b0a0 tx=0x7efe3800b3b0 comp rx=0 tx=0).stop 2024-05-09T00:37:02.506 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:02.503+0000 7efe47237700 1 -- 172.21.15.157:0/2814133779 shutdown_connections 2024-05-09T00:37:02.507 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:02.503+0000 7efe47237700 1 --2- 172.21.15.157:0/2814133779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe40108b00 0x7efe40108f10 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:37:02.507 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:02.503+0000 7efe47237700 1 -- 172.21.15.157:0/2814133779 >> 172.21.15.157:0/2814133779 conn(0x7efe40077d30 msgr2=0x7efe40078170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:37:02.507 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:02.503+0000 7efe47237700 1 -- 172.21.15.157:0/2814133779 shutdown_connections 2024-05-09T00:37:02.507 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:02.503+0000 7efe47237700 1 -- 172.21.15.157:0/2814133779 wait complete. 2024-05-09T00:37:02.507 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:02.504+0000 7efe47237700 1 Processor -- start 2024-05-09T00:37:02.507 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:02.504+0000 7efe47237700 1 -- start start 2024-05-09T00:37:02.507 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:02.504+0000 7efe47237700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe40108b00 0x7efe4019fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:37:02.507 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:02.504+0000 7efe47237700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efe40114730 con 0x7efe40108b00 2024-05-09T00:37:02.507 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:02.505+0000 7efe44fd3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe40108b00 0x7efe4019fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:37:02.507 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:02.505+0000 7efe44fd3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe40108b00 0x7efe4019fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.157:59764/0 (socket says 172.21.15.157:59764) 2024-05-09T00:37:02.507 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:02.505+0000 7efe44fd3700 1 -- 172.21.15.157:0/331978192 learned_addr learned my addr 172.21.15.157:0/331978192 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-05-09T00:37:02.507 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:02.505+0000 7efe44fd3700 1 -- 172.21.15.157:0/331978192 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe401a0420 con 0x7efe40108b00 2024-05-09T00:37:02.508 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:02.505+0000 7efe44fd3700 1 --2- 172.21.15.157:0/331978192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe40108b00 0x7efe4019fee0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7efe3800b830 tx=0x7efe38009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:37:02.509 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:02.505+0000 7efe3dffb700 1 -- 172.21.15.157:0/331978192 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe38021020 con 0x7efe40108b00 2024-05-09T00:37:02.509 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:02.505+0000 7efe3dffb700 1 -- 172.21.15.157:0/331978192 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe38004000 con 0x7efe40108b00 2024-05-09T00:37:02.509 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:02.505+0000 7efe47237700 1 -- 172.21.15.157:0/331978192 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe401a06b0 con 0x7efe40108b00 2024-05-09T00:37:02.509 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:02.505+0000 7efe3dffb700 1 -- 172.21.15.157:0/331978192 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe38017040 con 0x7efe40108b00 2024-05-09T00:37:02.509 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:02.506+0000 7efe47237700 1 -- 172.21.15.157:0/331978192 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe401a4710 con 0x7efe40108b00 2024-05-09T00:37:02.509 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:02.507+0000 7efe47237700 1 -- 172.21.15.157:0/331978192 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe401031c0 con 0x7efe40108b00 2024-05-09T00:37:02.509 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:02.507+0000 7efe3dffb700 1 -- 172.21.15.157:0/331978192 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7efe38024020 con 0x7efe40108b00 2024-05-09T00:37:02.510 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:02.507+0000 7efe3dffb700 1 --2- 172.21.15.157:0/331978192 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7efe2c03e510 0x7efe2c040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:37:02.510 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:02.507+0000 7efe3dffb700 1 -- 172.21.15.157:0/331978192 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7efe3800f050 con 0x7efe40108b00 2024-05-09T00:37:02.510 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:02.507+0000 7efe3ffff700 1 --2- 172.21.15.157:0/331978192 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7efe2c03e510 0x7efe2c040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:37:02.510 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:02.508+0000 7efe3ffff700 1 --2- 172.21.15.157:0/331978192 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7efe2c03e510 0x7efe2c040a20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7efe28006fd0 tx=0x7efe28006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:37:02.514 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:02.513+0000 7efe3dffb700 1 -- 172.21.15.157:0/331978192 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efe3802b020 con 0x7efe40108b00 2024-05-09T00:37:02.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:02 smithi005 ceph-mon[30365]: Updating smithi157:/var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T00:37:02.732 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:02.731+0000 7efe47237700 1 -- 172.21.15.157:0/331978192 --> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi005:172.21.15.5=a;smithi131:172.21.15.131=b;smithi157:172.21.15.157=c", "target": ["mon-mgr", ""]}) -- 0x7efe401a1240 con 0x7efe2c03e510 2024-05-09T00:37:02.739 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:02.738+0000 7efe3dffb700 1 -- 172.21.15.157:0/331978192 <== mgr.14152 v2:172.21.15.5:6800/91856593 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7efe401a1240 con 0x7efe2c03e510 2024-05-09T00:37:02.739 INFO:teuthology.orchestra.run.smithi157.stdout:Scheduled mon update... 2024-05-09T00:37:02.741 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:02.740+0000 7efe47237700 1 -- 172.21.15.157:0/331978192 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7efe2c03e510 msgr2=0x7efe2c040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:37:02.742 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:02.740+0000 7efe47237700 1 --2- 172.21.15.157:0/331978192 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7efe2c03e510 0x7efe2c040a20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7efe28006fd0 tx=0x7efe28006e40 comp rx=0 tx=0).stop 2024-05-09T00:37:02.742 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:02.740+0000 7efe47237700 1 -- 172.21.15.157:0/331978192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe40108b00 msgr2=0x7efe4019fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:37:02.742 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:02.740+0000 7efe47237700 1 --2- 172.21.15.157:0/331978192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe40108b00 0x7efe4019fee0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7efe3800b830 tx=0x7efe38009790 comp rx=0 tx=0).stop 2024-05-09T00:37:02.742 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:02.740+0000 7efe47237700 1 -- 172.21.15.157:0/331978192 shutdown_connections 2024-05-09T00:37:02.742 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:02.740+0000 7efe47237700 1 --2- 172.21.15.157:0/331978192 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7efe2c03e510 0x7efe2c040a20 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:37:02.742 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:02.740+0000 7efe47237700 1 --2- 172.21.15.157:0/331978192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe40108b00 0x7efe4019fee0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:37:02.742 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:02.740+0000 7efe47237700 1 -- 172.21.15.157:0/331978192 >> 172.21.15.157:0/331978192 conn(0x7efe40077d30 msgr2=0x7efe40100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:37:02.742 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:02.740+0000 7efe47237700 1 -- 172.21.15.157:0/331978192 shutdown_connections 2024-05-09T00:37:02.742 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:02.740+0000 7efe47237700 1 -- 172.21.15.157:0/331978192 wait complete. 2024-05-09T00:37:03.247 DEBUG:teuthology.orchestra.run.smithi131:mon.b> sudo journalctl -f -n 0 -u ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@mon.b.service 2024-05-09T00:37:03.250 DEBUG:teuthology.orchestra.run.smithi157:mon.c> sudo journalctl -f -n 0 -u ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@mon.c.service 2024-05-09T00:37:03.252 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-09T00:37:03.253 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-09T00:37:03.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:03 smithi005 ceph-mon[30365]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:03.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:03 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:03.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:03 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:37:03.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:03 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:37:03.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:03 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:37:03.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:03 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:03.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:03 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T00:37:03.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:03 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:37:04.328 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T00:37:04.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:04 smithi005 ceph-mon[30365]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi005:172.21.15.5=a;smithi131:172.21.15.131=b;smithi157:172.21.15.157=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:37:04.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:04 smithi005 ceph-mon[30365]: Saving service mon spec with placement smithi005:172.21.15.5=a;smithi131:172.21.15.131=b;smithi157:172.21.15.157=c;count:3 2024-05-09T00:37:04.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:04 smithi005 ceph-mon[30365]: Deploying daemon mon.c on smithi157 2024-05-09T00:37:05.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:05 smithi005 ceph-mon[30365]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:06.971 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:06.967+0000 7f6790a24700 1 -- 172.21.15.157:0/501107180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f678c104d10 msgr2=0x7f678c105120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:37:06.971 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:06.967+0000 7f6790a24700 1 --2- 172.21.15.157:0/501107180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f678c104d10 0x7f678c105120 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f677c009910 tx=0x7f677c009c20 comp rx=0 tx=0).stop 2024-05-09T00:37:06.972 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:06.967+0000 7f6790a24700 1 -- 172.21.15.157:0/501107180 shutdown_connections 2024-05-09T00:37:06.972 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:06.967+0000 7f6790a24700 1 --2- 172.21.15.157:0/501107180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f678c104d10 0x7f678c105120 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:37:06.972 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:06.967+0000 7f6790a24700 1 -- 172.21.15.157:0/501107180 >> 172.21.15.157:0/501107180 conn(0x7f678c0fe5c0 msgr2=0x7f678c100a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:37:06.972 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:06.967+0000 7f6790a24700 1 -- 172.21.15.157:0/501107180 shutdown_connections 2024-05-09T00:37:06.972 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:06.967+0000 7f6790a24700 1 -- 172.21.15.157:0/501107180 wait complete. 2024-05-09T00:37:06.972 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:06.968+0000 7f6790a24700 1 Processor -- start 2024-05-09T00:37:06.972 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:06.968+0000 7f6790a24700 1 -- start start 2024-05-09T00:37:06.972 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:06.969+0000 7f6790a24700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f678c104d10 0x7f678c19f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:37:06.972 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:06.969+0000 7f6790a24700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f678c10e110 con 0x7f678c104d10 2024-05-09T00:37:06.972 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:06.969+0000 7f678a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f678c104d10 0x7f678c19f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:37:06.972 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:06.969+0000 7f678a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f678c104d10 0x7f678c19f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.157:59780/0 (socket says 172.21.15.157:59780) 2024-05-09T00:37:06.972 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:06.969+0000 7f678a59c700 1 -- 172.21.15.157:0/4016799748 learned_addr learned my addr 172.21.15.157:0/4016799748 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-05-09T00:37:06.974 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:06.969+0000 7f678a59c700 1 -- 172.21.15.157:0/4016799748 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f678c19fb20 con 0x7f678c104d10 2024-05-09T00:37:06.974 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:06.970+0000 7f678a59c700 1 --2- 172.21.15.157:0/4016799748 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f678c104d10 0x7f678c19f5e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f677c0056d0 tx=0x7f677c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:37:06.974 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:06.970+0000 7f677b7fe700 1 -- 172.21.15.157:0/4016799748 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f677c026070 con 0x7f678c104d10 2024-05-09T00:37:06.974 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:06.970+0000 7f6790a24700 1 -- 172.21.15.157:0/4016799748 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f678c19fdb0 con 0x7f678c104d10 2024-05-09T00:37:06.974 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:06.970+0000 7f677b7fe700 1 -- 172.21.15.157:0/4016799748 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f677c00bc00 con 0x7f678c104d10 2024-05-09T00:37:06.974 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:06.970+0000 7f677b7fe700 1 -- 172.21.15.157:0/4016799748 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f677c003920 con 0x7f678c104d10 2024-05-09T00:37:06.974 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:06.970+0000 7f6790a24700 1 -- 172.21.15.157:0/4016799748 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f678c199770 con 0x7f678c104d10 2024-05-09T00:37:06.975 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:06.971+0000 7f6790a24700 1 -- 172.21.15.157:0/4016799748 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f678c06c3a0 con 0x7f678c104d10 2024-05-09T00:37:06.975 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:06.971+0000 7f677b7fe700 1 -- 172.21.15.157:0/4016799748 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f677c029070 con 0x7f678c104d10 2024-05-09T00:37:06.975 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:06.971+0000 7f677b7fe700 1 --2- 172.21.15.157:0/4016799748 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f677403e510 0x7f6774040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:37:06.975 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:06.971+0000 7f677b7fe700 1 -- 172.21.15.157:0/4016799748 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f677c01e070 con 0x7f678c104d10 2024-05-09T00:37:06.975 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:06.971+0000 7f6789d9b700 1 --2- 172.21.15.157:0/4016799748 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f677403e510 0x7f6774040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:37:06.975 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:06.972+0000 7f6789d9b700 1 --2- 172.21.15.157:0/4016799748 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f677403e510 0x7f6774040a20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6780006fd0 tx=0x7f6780006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:37:06.979 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:06.977+0000 7f677b7fe700 1 -- 172.21.15.157:0/4016799748 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f677c013070 con 0x7f678c104d10 2024-05-09T00:37:07.296 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:07.293+0000 7f6790a24700 1 -- 172.21.15.157:0/4016799748 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f678c064e20 con 0x7f678c104d10 2024-05-09T00:37:07.297 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:07.294+0000 7f677b7fe700 1 -- 172.21.15.157:0/4016799748 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+741 (secure 0 0 0) 0x7f677c030090 con 0x7f678c104d10 2024-05-09T00:37:07.297 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-05-09T00:37:07.297 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":1,"fsid":"d85814c6-0d9b-11ef-bc97-c7b262605968","modified":"2024-05-09T00:35:22.099371Z","created":"2024-05-09T00:35:22.099371Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:3300","nonce":0},{"type":"v1","addr":"172.21.15.5:6789","nonce":0}]},"addr":"172.21.15.5:6789/0","public_addr":"172.21.15.5:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-09T00:37:07.299 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:07.297+0000 7f6790a24700 1 -- 172.21.15.157:0/4016799748 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f677403e510 msgr2=0x7f6774040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:37:07.299 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:07.297+0000 7f6790a24700 1 --2- 172.21.15.157:0/4016799748 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f677403e510 0x7f6774040a20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6780006fd0 tx=0x7f6780006e40 comp rx=0 tx=0).stop 2024-05-09T00:37:07.299 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:07.297+0000 7f6790a24700 1 -- 172.21.15.157:0/4016799748 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f678c104d10 msgr2=0x7f678c19f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:37:07.300 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:07.297+0000 7f6790a24700 1 --2- 172.21.15.157:0/4016799748 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f678c104d10 0x7f678c19f5e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f677c0056d0 tx=0x7f677c0057b0 comp rx=0 tx=0).stop 2024-05-09T00:37:07.300 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:07.297+0000 7f6790a24700 1 -- 172.21.15.157:0/4016799748 shutdown_connections 2024-05-09T00:37:07.300 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:07.297+0000 7f6790a24700 1 --2- 172.21.15.157:0/4016799748 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f677403e510 0x7f6774040a20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:37:07.300 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:07.297+0000 7f6790a24700 1 --2- 172.21.15.157:0/4016799748 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f678c104d10 0x7f678c19f5e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:37:07.300 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:07.297+0000 7f6790a24700 1 -- 172.21.15.157:0/4016799748 >> 172.21.15.157:0/4016799748 conn(0x7f678c0fe5c0 msgr2=0x7f678c105b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:37:07.300 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:07.297+0000 7f6790a24700 1 -- 172.21.15.157:0/4016799748 shutdown_connections 2024-05-09T00:37:07.300 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:07.297+0000 7f6790a24700 1 -- 172.21.15.157:0/4016799748 wait complete. 2024-05-09T00:37:07.300 INFO:teuthology.orchestra.run.smithi157.stderr:dumped monmap epoch 1 2024-05-09T00:37:07.430 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 podman[34261]: 2024-05-09 00:37:07.396741826 +0000 UTC m=+0.341049027 container start e534101f6e7ce052a3a73b84eb1564095aa21e0804c70da28b798dcecafb1683 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mon-c, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T00:37:07.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:07 smithi005 ceph-mon[30365]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:07.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: set uid:gid to 167:167 (ceph:ceph) 2024-05-09T00:37:07.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: ceph version 19.0.0-3594-gc9da4049 (c9da4049f638c0f84bdd9b4e33c42e86046ac554) squid (dev), process ceph-mon, pid 2 2024-05-09T00:37:07.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: pidfile_write: ignore empty --pid-file 2024-05-09T00:37:07.688 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: load: jerasure load: lrc 2024-05-09T00:37:07.688 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: RocksDB version: 7.9.2 2024-05-09T00:37:07.688 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Git sha 0 2024-05-09T00:37:07.688 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Compile date 2024-05-08 19:33:14 2024-05-09T00:37:07.688 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: DB SUMMARY 2024-05-09T00:37:07.688 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: DB Session ID: Z7UOOLCUMJZ17B3EZD8P 2024-05-09T00:37:07.688 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: CURRENT file: CURRENT 2024-05-09T00:37:07.688 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: IDENTITY file: IDENTITY 2024-05-09T00:37:07.688 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-05-09T00:37:07.688 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-05-09T00:37:07.688 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-05-09T00:37:07.688 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.error_if_exists: 0 2024-05-09T00:37:07.688 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.create_if_missing: 0 2024-05-09T00:37:07.689 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.paranoid_checks: 1 2024-05-09T00:37:07.689 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-09T00:37:07.689 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-09T00:37:07.689 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-09T00:37:07.689 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.env: 0x562d5892f500 2024-05-09T00:37:07.689 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.fs: PosixFileSystem 2024-05-09T00:37:07.689 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.info_log: 0x562d5b060020 2024-05-09T00:37:07.689 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.max_file_opening_threads: 16 2024-05-09T00:37:07.689 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.statistics: (nil) 2024-05-09T00:37:07.689 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.use_fsync: 0 2024-05-09T00:37:07.689 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.max_log_file_size: 0 2024-05-09T00:37:07.689 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-09T00:37:07.689 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-09T00:37:07.689 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.keep_log_file_num: 1000 2024-05-09T00:37:07.690 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.recycle_log_file_num: 0 2024-05-09T00:37:07.690 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.allow_fallocate: 1 2024-05-09T00:37:07.690 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.allow_mmap_reads: 0 2024-05-09T00:37:07.690 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.allow_mmap_writes: 0 2024-05-09T00:37:07.690 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.use_direct_reads: 0 2024-05-09T00:37:07.690 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-09T00:37:07.690 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.create_missing_column_families: 0 2024-05-09T00:37:07.690 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.db_log_dir: 2024-05-09T00:37:07.690 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.wal_dir: 2024-05-09T00:37:07.690 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-09T00:37:07.694 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-09T00:37:07.694 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-09T00:37:07.694 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-09T00:37:07.695 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-09T00:37:07.695 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-09T00:37:07.695 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.advise_random_on_open: 1 2024-05-09T00:37:07.695 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.db_write_buffer_size: 0 2024-05-09T00:37:07.695 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.write_buffer_manager: 0x562d5af2a8c0 2024-05-09T00:37:07.695 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-09T00:37:07.695 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-09T00:37:07.695 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-09T00:37:07.695 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.rate_limiter: (nil) 2024-05-09T00:37:07.695 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-09T00:37:07.695 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.wal_recovery_mode: 2 2024-05-09T00:37:07.695 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.enable_thread_tracking: 0 2024-05-09T00:37:07.696 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.enable_pipelined_write: 0 2024-05-09T00:37:07.696 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.unordered_write: 0 2024-05-09T00:37:07.696 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-09T00:37:07.696 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-09T00:37:07.696 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-09T00:37:07.696 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-09T00:37:07.696 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.row_cache: None 2024-05-09T00:37:07.696 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.wal_filter: None 2024-05-09T00:37:07.696 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-09T00:37:07.696 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.allow_ingest_behind: 0 2024-05-09T00:37:07.696 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.two_write_queues: 0 2024-05-09T00:37:07.696 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.manual_wal_flush: 0 2024-05-09T00:37:07.696 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.wal_compression: 0 2024-05-09T00:37:07.696 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.atomic_flush: 0 2024-05-09T00:37:07.697 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-09T00:37:07.697 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-09T00:37:07.697 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-09T00:37:07.697 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.log_readahead_size: 0 2024-05-09T00:37:07.697 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-09T00:37:07.697 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.best_efforts_recovery: 0 2024-05-09T00:37:07.697 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-09T00:37:07.697 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-09T00:37:07.697 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.allow_data_in_errors: 0 2024-05-09T00:37:07.697 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.db_host_id: __hostname__ 2024-05-09T00:37:07.697 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-09T00:37:07.697 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.max_background_jobs: 2 2024-05-09T00:37:07.697 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.max_background_compactions: -1 2024-05-09T00:37:07.697 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.max_subcompactions: 1 2024-05-09T00:37:07.698 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-09T00:37:07.698 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-09T00:37:07.698 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-09T00:37:07.698 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.max_total_wal_size: 0 2024-05-09T00:37:07.698 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-09T00:37:07.698 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-09T00:37:07.698 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-09T00:37:07.698 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-09T00:37:07.698 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.max_open_files: -1 2024-05-09T00:37:07.698 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.bytes_per_sync: 0 2024-05-09T00:37:07.698 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-09T00:37:07.698 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-09T00:37:07.699 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.compaction_readahead_size: 0 2024-05-09T00:37:07.699 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.max_background_flushes: -1 2024-05-09T00:37:07.699 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Compression algorithms supported: 2024-05-09T00:37:07.699 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-09T00:37:07.699 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: kZSTD supported: 0 2024-05-09T00:37:07.699 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: kXpressCompression supported: 0 2024-05-09T00:37:07.699 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: kLZ4HCCompression supported: 1 2024-05-09T00:37:07.699 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: kZlibCompression supported: 1 2024-05-09T00:37:07.699 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: kSnappyCompression supported: 1 2024-05-09T00:37:07.699 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: kLZ4Compression supported: 1 2024-05-09T00:37:07.699 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: kBZip2Compression supported: 0 2024-05-09T00:37:07.699 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-09T00:37:07.699 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-09T00:37:07.699 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-05-09T00:37:07.700 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-09T00:37:07.700 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-09T00:37:07.700 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.merge_operator: 2024-05-09T00:37:07.700 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.compaction_filter: None 2024-05-09T00:37:07.700 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.compaction_filter_factory: None 2024-05-09T00:37:07.700 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.sst_partitioner_factory: None 2024-05-09T00:37:07.700 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-09T00:37:07.700 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-09T00:37:07.700 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x562d5b0601e0) 2024-05-09T00:37:07.700 INFO:journalctl@ceph.mon.c.smithi157.stdout: cache_index_and_filter_blocks: 1 2024-05-09T00:37:07.700 INFO:journalctl@ceph.mon.c.smithi157.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-09T00:37:07.700 INFO:journalctl@ceph.mon.c.smithi157.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-09T00:37:07.700 INFO:journalctl@ceph.mon.c.smithi157.stdout: pin_top_level_index_and_filter: 1 2024-05-09T00:37:07.700 INFO:journalctl@ceph.mon.c.smithi157.stdout: index_type: 0 2024-05-09T00:37:07.701 INFO:journalctl@ceph.mon.c.smithi157.stdout: data_block_index_type: 0 2024-05-09T00:37:07.701 INFO:journalctl@ceph.mon.c.smithi157.stdout: index_shortening: 1 2024-05-09T00:37:07.701 INFO:journalctl@ceph.mon.c.smithi157.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-09T00:37:07.701 INFO:journalctl@ceph.mon.c.smithi157.stdout: checksum: 4 2024-05-09T00:37:07.701 INFO:journalctl@ceph.mon.c.smithi157.stdout: no_block_cache: 0 2024-05-09T00:37:07.701 INFO:journalctl@ceph.mon.c.smithi157.stdout: block_cache: 0x562d5af5d1f0 2024-05-09T00:37:07.701 INFO:journalctl@ceph.mon.c.smithi157.stdout: block_cache_name: BinnedLRUCache 2024-05-09T00:37:07.701 INFO:journalctl@ceph.mon.c.smithi157.stdout: block_cache_options: 2024-05-09T00:37:07.701 INFO:journalctl@ceph.mon.c.smithi157.stdout: capacity : 536870912 2024-05-09T00:37:07.701 INFO:journalctl@ceph.mon.c.smithi157.stdout: num_shard_bits : 4 2024-05-09T00:37:07.701 INFO:journalctl@ceph.mon.c.smithi157.stdout: strict_capacity_limit : 0 2024-05-09T00:37:07.701 INFO:journalctl@ceph.mon.c.smithi157.stdout: high_pri_pool_ratio: 0.000 2024-05-09T00:37:07.701 INFO:journalctl@ceph.mon.c.smithi157.stdout: block_cache_compressed: (nil) 2024-05-09T00:37:07.701 INFO:journalctl@ceph.mon.c.smithi157.stdout: persistent_cache: (nil) 2024-05-09T00:37:07.702 INFO:journalctl@ceph.mon.c.smithi157.stdout: block_size: 4096 2024-05-09T00:37:07.702 INFO:journalctl@ceph.mon.c.smithi157.stdout: block_size_deviation: 10 2024-05-09T00:37:07.702 INFO:journalctl@ceph.mon.c.smithi157.stdout: block_restart_interval: 16 2024-05-09T00:37:07.702 INFO:journalctl@ceph.mon.c.smithi157.stdout: index_block_restart_interval: 1 2024-05-09T00:37:07.702 INFO:journalctl@ceph.mon.c.smithi157.stdout: metadata_block_size: 4096 2024-05-09T00:37:07.702 INFO:journalctl@ceph.mon.c.smithi157.stdout: partition_filters: 0 2024-05-09T00:37:07.702 INFO:journalctl@ceph.mon.c.smithi157.stdout: use_delta_encoding: 1 2024-05-09T00:37:07.702 INFO:journalctl@ceph.mon.c.smithi157.stdout: filter_policy: bloomfilter 2024-05-09T00:37:07.702 INFO:journalctl@ceph.mon.c.smithi157.stdout: whole_key_filtering: 1 2024-05-09T00:37:07.702 INFO:journalctl@ceph.mon.c.smithi157.stdout: verify_compression: 0 2024-05-09T00:37:07.702 INFO:journalctl@ceph.mon.c.smithi157.stdout: read_amp_bytes_per_bit: 0 2024-05-09T00:37:07.702 INFO:journalctl@ceph.mon.c.smithi157.stdout: format_version: 5 2024-05-09T00:37:07.702 INFO:journalctl@ceph.mon.c.smithi157.stdout: enable_index_compression: 1 2024-05-09T00:37:07.702 INFO:journalctl@ceph.mon.c.smithi157.stdout: block_align: 0 2024-05-09T00:37:07.703 INFO:journalctl@ceph.mon.c.smithi157.stdout: max_auto_readahead_size: 262144 2024-05-09T00:37:07.703 INFO:journalctl@ceph.mon.c.smithi157.stdout: prepopulate_block_cache: 0 2024-05-09T00:37:07.703 INFO:journalctl@ceph.mon.c.smithi157.stdout: initial_auto_readahead_size: 8192 2024-05-09T00:37:07.703 INFO:journalctl@ceph.mon.c.smithi157.stdout: num_file_reads_for_auto_readahead: 2 2024-05-09T00:37:07.703 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.write_buffer_size: 33554432 2024-05-09T00:37:07.703 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.max_write_buffer_number: 2 2024-05-09T00:37:07.703 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.compression: NoCompression 2024-05-09T00:37:07.703 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.bottommost_compression: Disabled 2024-05-09T00:37:07.703 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.prefix_extractor: nullptr 2024-05-09T00:37:07.703 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-09T00:37:07.703 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.num_levels: 7 2024-05-09T00:37:07.703 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-09T00:37:07.703 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-09T00:37:07.704 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-09T00:37:07.704 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-09T00:37:07.704 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-09T00:37:07.704 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-09T00:37:07.704 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-09T00:37:07.704 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-09T00:37:07.704 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-09T00:37:07.704 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-09T00:37:07.704 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-09T00:37:07.704 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-09T00:37:07.704 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-09T00:37:07.704 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.compression_opts.level: 32767 2024-05-09T00:37:07.704 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.compression_opts.strategy: 0 2024-05-09T00:37:07.704 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-09T00:37:07.705 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-09T00:37:07.705 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-09T00:37:07.705 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-09T00:37:07.705 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.compression_opts.enabled: false 2024-05-09T00:37:07.705 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-09T00:37:07.705 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-09T00:37:07.705 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-09T00:37:07.705 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-09T00:37:07.705 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.target_file_size_base: 67108864 2024-05-09T00:37:07.705 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-09T00:37:07.705 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-09T00:37:07.705 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-09T00:37:07.705 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-09T00:37:07.705 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-09T00:37:07.705 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-09T00:37:07.706 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-09T00:37:07.706 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-09T00:37:07.706 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-09T00:37:07.706 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-09T00:37:07.706 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-09T00:37:07.706 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-09T00:37:07.706 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-09T00:37:07.706 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-09T00:37:07.706 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.arena_block_size: 1048576 2024-05-09T00:37:07.706 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-09T00:37:07.706 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-09T00:37:07.706 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.disable_auto_compactions: 0 2024-05-09T00:37:07.706 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-09T00:37:07.707 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-09T00:37:07.707 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-09T00:37:07.707 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-09T00:37:07.707 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-09T00:37:07.707 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-09T00:37:07.707 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-09T00:37:07.707 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-09T00:37:07.707 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-09T00:37:07.707 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-09T00:37:07.707 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-09T00:37:07.707 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.inplace_update_support: 0 2024-05-09T00:37:07.707 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-09T00:37:07.707 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-09T00:37:07.708 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-09T00:37:07.708 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-09T00:37:07.708 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.bloom_locality: 0 2024-05-09T00:37:07.708 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.max_successive_merges: 0 2024-05-09T00:37:07.708 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-09T00:37:07.708 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.paranoid_file_checks: 0 2024-05-09T00:37:07.708 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.force_consistency_checks: 1 2024-05-09T00:37:07.708 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.report_bg_io_stats: 0 2024-05-09T00:37:07.708 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.ttl: 2592000 2024-05-09T00:37:07.708 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-09T00:37:07.708 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-09T00:37:07.708 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-09T00:37:07.708 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.enable_blob_files: false 2024-05-09T00:37:07.709 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.min_blob_size: 0 2024-05-09T00:37:07.709 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.blob_file_size: 268435456 2024-05-09T00:37:07.709 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-09T00:37:07.709 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-09T00:37:07.709 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-09T00:37:07.709 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-09T00:37:07.709 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-09T00:37:07.709 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.blob_file_starting_level: 0 2024-05-09T00:37:07.709 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-09T00:37:07.709 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-09T00:37:07.709 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-09T00:37:07.709 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: e7369c12-f7dd-4fc4-9101-530b8b3f5b82 2024-05-09T00:37:07.709 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715215027436552, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-09T00:37:07.710 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-09T00:37:07.710 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715215027437135, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715215027, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "e7369c12-f7dd-4fc4-9101-530b8b3f5b82", "db_session_id": "Z7UOOLCUMJZ17B3EZD8P", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-09T00:37:07.710 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715215027437241, "job": 1, "event": "recovery_finished"} 2024-05-09T00:37:07.710 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-09T00:37:07.710 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-09T00:37:07.710 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x562d5af41c00 2024-05-09T00:37:07.710 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: DB pointer 0x562d5fd34000 2024-05-09T00:37:07.710 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-09T00:37:07.710 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-09T00:37:07.710 INFO:journalctl@ceph.mon.c.smithi157.stdout: ** DB Stats ** 2024-05-09T00:37:07.710 INFO:journalctl@ceph.mon.c.smithi157.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-09T00:37:07.710 INFO:journalctl@ceph.mon.c.smithi157.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-09T00:37:07.710 INFO:journalctl@ceph.mon.c.smithi157.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-09T00:37:07.710 INFO:journalctl@ceph.mon.c.smithi157.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-09T00:37:07.711 INFO:journalctl@ceph.mon.c.smithi157.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-09T00:37:07.711 INFO:journalctl@ceph.mon.c.smithi157.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-09T00:37:07.711 INFO:journalctl@ceph.mon.c.smithi157.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-09T00:37:07.711 INFO:journalctl@ceph.mon.c.smithi157.stdout: 2024-05-09T00:37:07.711 INFO:journalctl@ceph.mon.c.smithi157.stdout: ** Compaction Stats [default] ** 2024-05-09T00:37:07.711 INFO:journalctl@ceph.mon.c.smithi157.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-09T00:37:07.711 INFO:journalctl@ceph.mon.c.smithi157.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-09T00:37:07.711 INFO:journalctl@ceph.mon.c.smithi157.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-09T00:37:07.711 INFO:journalctl@ceph.mon.c.smithi157.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-09T00:37:07.711 INFO:journalctl@ceph.mon.c.smithi157.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-09T00:37:07.711 INFO:journalctl@ceph.mon.c.smithi157.stdout: 2024-05-09T00:37:07.711 INFO:journalctl@ceph.mon.c.smithi157.stdout: ** Compaction Stats [default] ** 2024-05-09T00:37:07.711 INFO:journalctl@ceph.mon.c.smithi157.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-09T00:37:07.711 INFO:journalctl@ceph.mon.c.smithi157.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-09T00:37:07.712 INFO:journalctl@ceph.mon.c.smithi157.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-09T00:37:07.712 INFO:journalctl@ceph.mon.c.smithi157.stdout: 2024-05-09T00:37:07.712 INFO:journalctl@ceph.mon.c.smithi157.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-09T00:37:07.712 INFO:journalctl@ceph.mon.c.smithi157.stdout: 2024-05-09T00:37:07.712 INFO:journalctl@ceph.mon.c.smithi157.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-09T00:37:07.712 INFO:journalctl@ceph.mon.c.smithi157.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-09T00:37:07.712 INFO:journalctl@ceph.mon.c.smithi157.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-09T00:37:07.712 INFO:journalctl@ceph.mon.c.smithi157.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-09T00:37:07.712 INFO:journalctl@ceph.mon.c.smithi157.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-09T00:37:07.712 INFO:journalctl@ceph.mon.c.smithi157.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-09T00:37:07.712 INFO:journalctl@ceph.mon.c.smithi157.stdout: Cumulative compaction: 0.00 GB write, 0.45 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-09T00:37:07.712 INFO:journalctl@ceph.mon.c.smithi157.stdout: Interval compaction: 0.00 GB write, 0.45 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-09T00:37:07.712 INFO:journalctl@ceph.mon.c.smithi157.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-09T00:37:07.712 INFO:journalctl@ceph.mon.c.smithi157.stdout: Block cache BinnedLRUCache@0x562d5af5d1f0#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-05-09T00:37:07.713 INFO:journalctl@ceph.mon.c.smithi157.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-09T00:37:07.713 INFO:journalctl@ceph.mon.c.smithi157.stdout: 2024-05-09T00:37:07.713 INFO:journalctl@ceph.mon.c.smithi157.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-09T00:37:07.713 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-05-09T00:37:07.713 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: using public_addr v2:172.21.15.157:0/0 -> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] 2024-05-09T00:37:07.713 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: starting mon.c rank -1 at public addrs [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] at bind addrs [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid d85814c6-0d9b-11ef-bc97-c7b262605968 2024-05-09T00:37:07.713 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: mon.c@-1(???) e0 preinit fsid d85814c6-0d9b-11ef-bc97-c7b262605968 2024-05-09T00:37:07.713 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: mon.c@-1(synchronizing).mds e1 new map 2024-05-09T00:37:07.713 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: mon.c@-1(synchronizing).mds e1 print_map 2024-05-09T00:37:07.713 INFO:journalctl@ceph.mon.c.smithi157.stdout: e1 2024-05-09T00:37:07.713 INFO:journalctl@ceph.mon.c.smithi157.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-09T00:37:07.713 INFO:journalctl@ceph.mon.c.smithi157.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-09T00:37:07.713 INFO:journalctl@ceph.mon.c.smithi157.stdout: legacy client fscid: -1 2024-05-09T00:37:07.714 INFO:journalctl@ceph.mon.c.smithi157.stdout: 2024-05-09T00:37:07.714 INFO:journalctl@ceph.mon.c.smithi157.stdout: No filesystems configured 2024-05-09T00:37:07.714 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-09T00:37:07.714 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-09T00:37:07.714 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-09T00:37:07.714 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-09T00:37:07.714 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-09T00:37:07.714 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-09T00:37:07.714 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-09T00:37:07.714 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T00:37:07.714 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T00:37:07.714 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T00:37:07.714 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:07.714 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T00:37:07.715 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:07.715 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:07.715 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi157", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:37:07.715 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:07.715 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:07.715 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:07.715 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:07.715 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi131", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:37:07.715 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:37:07.715 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:37:07.715 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: Updating smithi131:/etc/ceph/ceph.conf 2024-05-09T00:37:07.715 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: Deploying cephadm binary to smithi157 2024-05-09T00:37:07.715 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: Updating smithi131:/var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T00:37:07.715 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: Updating smithi131:/etc/ceph/ceph.client.admin.keyring 2024-05-09T00:37:07.716 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:07.716 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: Updating smithi131:/var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T00:37:07.716 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:07.716 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:07.716 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:07.716 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:07.716 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:07.716 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: Added host smithi157 2024-05-09T00:37:07.716 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:37:07.716 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:07.716 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:07.716 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:07.716 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:07.716 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T00:37:07.716 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:07.717 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/1333314358' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-09T00:37:07.717 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:07.717 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:07.717 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:07.717 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:07.717 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi157", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:37:07.717 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:37:07.717 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:37:07.717 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: Updating smithi157:/etc/ceph/ceph.conf 2024-05-09T00:37:07.717 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:07.717 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: Updating smithi157:/var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T00:37:07.717 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: Updating smithi157:/etc/ceph/ceph.client.admin.keyring 2024-05-09T00:37:07.717 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/1333314358' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-09T00:37:07.717 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T00:37:07.718 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:07.718 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:07.718 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:07.718 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: Updating smithi157:/var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T00:37:07.718 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:07.718 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:07.718 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:37:07.718 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:37:07.718 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:37:07.718 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:07.718 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T00:37:07.718 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:37:07.718 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi005:172.21.15.5=a;smithi131:172.21.15.131=b;smithi157:172.21.15.157=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:37:07.719 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: Saving service mon spec with placement smithi005:172.21.15.5=a;smithi131:172.21.15.131=b;smithi157:172.21.15.157=c;count:3 2024-05-09T00:37:07.719 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: Deploying daemon mon.c on smithi157 2024-05-09T00:37:07.719 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:07.719 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:07.719 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 ceph-mon[34294]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-09T00:37:08.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 bash[34261]: e534101f6e7ce052a3a73b84eb1564095aa21e0804c70da28b798dcecafb1683 2024-05-09T00:37:08.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:07 smithi157 systemd[1]: Started Ceph mon.c for d85814c6-0d9b-11ef-bc97-c7b262605968. 2024-05-09T00:37:09.261 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-09T00:37:09.262 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-09T00:37:09.505 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T00:37:12.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:12 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T00:37:12.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:12 smithi005 ceph-mon[30365]: mon.a calling monitor election 2024-05-09T00:37:12.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:12 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:37:12.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:12 smithi005 ceph-mon[30365]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:12.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:12 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:37:12.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:12 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:37:12.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:12 smithi005 ceph-mon[30365]: mon.c calling monitor election 2024-05-09T00:37:12.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:12 smithi005 ceph-mon[30365]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:12.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:12 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:37:12.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:12 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:37:12.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:12 smithi005 ceph-mon[30365]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:12.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:12 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:37:12.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:12 smithi005 ceph-mon[30365]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-09T00:37:12.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:12 smithi005 ceph-mon[30365]: monmap epoch 2 2024-05-09T00:37:12.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:12 smithi005 ceph-mon[30365]: fsid d85814c6-0d9b-11ef-bc97-c7b262605968 2024-05-09T00:37:12.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:12 smithi005 ceph-mon[30365]: last_changed 2024-05-09T00:37:07.467509+0000 2024-05-09T00:37:12.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:12 smithi005 ceph-mon[30365]: created 2024-05-09T00:35:22.099371+0000 2024-05-09T00:37:12.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:12 smithi005 ceph-mon[30365]: min_mon_release 19 (squid) 2024-05-09T00:37:12.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:12 smithi005 ceph-mon[30365]: election_strategy: 1 2024-05-09T00:37:12.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:12 smithi005 ceph-mon[30365]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-05-09T00:37:12.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:12 smithi005 ceph-mon[30365]: 1: [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] mon.c 2024-05-09T00:37:12.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:12 smithi005 ceph-mon[30365]: fsmap 2024-05-09T00:37:12.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:12 smithi005 ceph-mon[30365]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T00:37:12.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:12 smithi005 ceph-mon[30365]: mgrmap e13: x(active, since 50s) 2024-05-09T00:37:12.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:12 smithi005 ceph-mon[30365]: overall HEALTH_OK 2024-05-09T00:37:12.794 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:12 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:12.794 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:12 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:12.794 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:12 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:12.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:12 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T00:37:12.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:12 smithi157 ceph-mon[34294]: mon.a calling monitor election 2024-05-09T00:37:12.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:12 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:37:12.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:12 smithi157 ceph-mon[34294]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:12.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:12 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:37:12.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:12 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:37:12.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:12 smithi157 ceph-mon[34294]: mon.c calling monitor election 2024-05-09T00:37:12.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:12 smithi157 ceph-mon[34294]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:12.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:12 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:37:12.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:12 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:37:12.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:12 smithi157 ceph-mon[34294]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:12.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:12 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:37:12.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:12 smithi157 ceph-mon[34294]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-09T00:37:12.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:12 smithi157 ceph-mon[34294]: monmap epoch 2 2024-05-09T00:37:12.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:12 smithi157 ceph-mon[34294]: fsid d85814c6-0d9b-11ef-bc97-c7b262605968 2024-05-09T00:37:12.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:12 smithi157 ceph-mon[34294]: last_changed 2024-05-09T00:37:07.467509+0000 2024-05-09T00:37:12.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:12 smithi157 ceph-mon[34294]: created 2024-05-09T00:35:22.099371+0000 2024-05-09T00:37:12.939 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:12 smithi157 ceph-mon[34294]: min_mon_release 19 (squid) 2024-05-09T00:37:12.939 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:12 smithi157 ceph-mon[34294]: election_strategy: 1 2024-05-09T00:37:12.939 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:12 smithi157 ceph-mon[34294]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-05-09T00:37:12.939 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:12 smithi157 ceph-mon[34294]: 1: [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] mon.c 2024-05-09T00:37:12.939 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:12 smithi157 ceph-mon[34294]: fsmap 2024-05-09T00:37:12.939 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:12 smithi157 ceph-mon[34294]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T00:37:12.939 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:12 smithi157 ceph-mon[34294]: mgrmap e13: x(active, since 50s) 2024-05-09T00:37:12.939 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:12 smithi157 ceph-mon[34294]: overall HEALTH_OK 2024-05-09T00:37:12.939 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:12 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:12.939 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:12 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:12.939 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:12 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:13.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:13 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T00:37:13.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:13 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:37:13.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:13 smithi157 ceph-mon[34294]: Deploying daemon mon.b on smithi131 2024-05-09T00:37:13.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:13 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:37:13.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:13 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T00:37:13.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:13 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:37:13.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:13 smithi005 ceph-mon[30365]: Deploying daemon mon.b on smithi131 2024-05-09T00:37:13.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:13 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:37:13.910 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:13.907+0000 7fb799e7f700 1 -- 172.21.15.157:0/2802972970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb794103ed0 msgr2=0x7fb77c005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:37:13.911 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:13.907+0000 7fb799e7f700 1 --2- 172.21.15.157:0/2802972970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb794103ed0 0x7fb77c005610 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb784005fa0 tx=0x7fb78400ffc0 comp rx=0 tx=0).stop 2024-05-09T00:37:13.911 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:13.907+0000 7fb799e7f700 1 -- 172.21.15.157:0/2802972970 shutdown_connections 2024-05-09T00:37:13.911 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:13.907+0000 7fb799e7f700 1 --2- 172.21.15.157:0/2802972970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb794103ed0 0x7fb77c005610 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:37:13.911 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:13.907+0000 7fb799e7f700 1 -- 172.21.15.157:0/2802972970 >> 172.21.15.157:0/2802972970 conn(0x7fb7940ff7a0 msgr2=0x7fb794101bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:37:13.911 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:13.907+0000 7fb799e7f700 1 -- 172.21.15.157:0/2802972970 shutdown_connections 2024-05-09T00:37:13.911 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:13.907+0000 7fb799e7f700 1 -- 172.21.15.157:0/2802972970 wait complete. 2024-05-09T00:37:13.911 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:13.908+0000 7fb799e7f700 1 Processor -- start 2024-05-09T00:37:13.911 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:13.908+0000 7fb799e7f700 1 -- start start 2024-05-09T00:37:13.913 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:13.909+0000 7fb799e7f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb794103ed0 0x7fb79419f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:37:13.913 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:13.909+0000 7fb799e7f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb79419fc20 0x7fb794199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:37:13.913 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:13.909+0000 7fb799e7f700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb79410bc10 con 0x7fb794103ed0 2024-05-09T00:37:13.913 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:13.909+0000 7fb799e7f700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb79410ba90 con 0x7fb79419fc20 2024-05-09T00:37:13.913 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:13.909+0000 7fb792ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb79419fc20 0x7fb794199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:37:13.913 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:13.909+0000 7fb792ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb79419fc20 0x7fb794199900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40726/0 (socket says 172.21.15.157:40726) 2024-05-09T00:37:13.913 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:13.909+0000 7fb792ffd700 1 -- 172.21.15.157:0/1169876048 learned_addr learned my addr 172.21.15.157:0/1169876048 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-05-09T00:37:13.914 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:13.909+0000 7fb792ffd700 1 -- 172.21.15.157:0/1169876048 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb79419fc20 msgr2=0x7fb794199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-09T00:37:13.914 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:13.909+0000 7fb792ffd700 1 -- 172.21.15.157:0/1169876048 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb79419fc20 msgr2=0x7fb794199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-09T00:37:13.914 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:13.909+0000 7fb792ffd700 1 --2- 172.21.15.157:0/1169876048 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb79419fc20 0x7fb794199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T00:37:13.914 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:13.909+0000 7fb7937fe700 1 --2- 172.21.15.157:0/1169876048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb794103ed0 0x7fb79419f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:37:13.914 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:13.909+0000 7fb792ffd700 1 --2- 172.21.15.157:0/1169876048 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb79419fc20 0x7fb794199900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T00:37:13.914 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:13.909+0000 7fb7937fe700 1 -- 172.21.15.157:0/1169876048 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb79419fc20 msgr2=0x7fb794199900 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T00:37:13.914 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:13.909+0000 7fb7937fe700 1 --2- 172.21.15.157:0/1169876048 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb79419fc20 0x7fb794199900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:37:13.914 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:13.909+0000 7fb7937fe700 1 -- 172.21.15.157:0/1169876048 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7941a01f0 con 0x7fb794103ed0 2024-05-09T00:37:13.914 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:13.910+0000 7fb7937fe700 1 --2- 172.21.15.157:0/1169876048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb794103ed0 0x7fb79419f6e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb784005fa0 tx=0x7fb784004d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:37:13.915 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:13.910+0000 7fb790ff9700 1 -- 172.21.15.157:0/1169876048 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fb784005230 con 0x7fb794103ed0 2024-05-09T00:37:13.915 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:13.910+0000 7fb790ff9700 1 -- 172.21.15.157:0/1169876048 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb78401b430 con 0x7fb794103ed0 2024-05-09T00:37:13.915 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:13.910+0000 7fb799e7f700 1 -- 172.21.15.157:0/1169876048 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb794199ea0 con 0x7fb794103ed0 2024-05-09T00:37:13.915 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:13.910+0000 7fb790ff9700 1 -- 172.21.15.157:0/1169876048 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fb784011670 con 0x7fb794103ed0 2024-05-09T00:37:13.915 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:13.910+0000 7fb799e7f700 1 -- 172.21.15.157:0/1169876048 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb79419a430 con 0x7fb794103ed0 2024-05-09T00:37:13.916 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:13.911+0000 7fb799e7f700 1 -- 172.21.15.157:0/1169876048 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb794105380 con 0x7fb794103ed0 2024-05-09T00:37:13.916 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:13.911+0000 7fb790ff9700 1 -- 172.21.15.157:0/1169876048 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7fb784011920 con 0x7fb794103ed0 2024-05-09T00:37:13.916 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:13.911+0000 7fb790ff9700 1 --2- 172.21.15.157:0/1169876048 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fb78003e300 0x7fb780040810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:37:13.916 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:13.912+0000 7fb790ff9700 1 -- 172.21.15.157:0/1169876048 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb784054ed0 con 0x7fb794103ed0 2024-05-09T00:37:13.916 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:13.912+0000 7fb792ffd700 1 --2- 172.21.15.157:0/1169876048 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fb78003e300 0x7fb780040810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:37:13.916 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:13.912+0000 7fb792ffd700 1 --2- 172.21.15.157:0/1169876048 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fb78003e300 0x7fb780040810 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb794100070 tx=0x7fb788008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:37:13.920 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:13.917+0000 7fb790ff9700 1 -- 172.21.15.157:0/1169876048 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb784022070 con 0x7fb794103ed0 2024-05-09T00:37:14.209 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:14.206+0000 7fb799e7f700 1 -- 172.21.15.157:0/1169876048 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fb79419ae30 con 0x7fb794103ed0 2024-05-09T00:37:14.210 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:14.207+0000 7fb790ff9700 1 -- 172.21.15.157:0/1169876048 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1011 (secure 0 0 0) 0x7fb78402d360 con 0x7fb794103ed0 2024-05-09T00:37:14.210 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-05-09T00:37:14.210 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":2,"fsid":"d85814c6-0d9b-11ef-bc97-c7b262605968","modified":"2024-05-09T00:37:07.467509Z","created":"2024-05-09T00:35:22.099371Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:3300","nonce":0},{"type":"v1","addr":"172.21.15.5:6789","nonce":0}]},"addr":"172.21.15.5:6789/0","public_addr":"172.21.15.5:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:3300","nonce":0},{"type":"v1","addr":"172.21.15.157:6789","nonce":0}]},"addr":"172.21.15.157:6789/0","public_addr":"172.21.15.157:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-09T00:37:14.213 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:14.209+0000 7fb799e7f700 1 -- 172.21.15.157:0/1169876048 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fb78003e300 msgr2=0x7fb780040810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:37:14.213 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:14.209+0000 7fb799e7f700 1 --2- 172.21.15.157:0/1169876048 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fb78003e300 0x7fb780040810 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb794100070 tx=0x7fb788008040 comp rx=0 tx=0).stop 2024-05-09T00:37:14.213 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:14.209+0000 7fb799e7f700 1 -- 172.21.15.157:0/1169876048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb794103ed0 msgr2=0x7fb79419f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:37:14.213 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:14.209+0000 7fb799e7f700 1 --2- 172.21.15.157:0/1169876048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb794103ed0 0x7fb79419f6e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb784005fa0 tx=0x7fb784004d60 comp rx=0 tx=0).stop 2024-05-09T00:37:14.213 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:14.209+0000 7fb799e7f700 1 -- 172.21.15.157:0/1169876048 shutdown_connections 2024-05-09T00:37:14.213 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:14.209+0000 7fb799e7f700 1 --2- 172.21.15.157:0/1169876048 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fb78003e300 0x7fb780040810 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:37:14.214 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:14.209+0000 7fb799e7f700 1 --2- 172.21.15.157:0/1169876048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb794103ed0 0x7fb79419f6e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:37:14.214 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:14.209+0000 7fb799e7f700 1 --2- 172.21.15.157:0/1169876048 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb79419fc20 0x7fb794199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:37:14.214 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:14.209+0000 7fb799e7f700 1 -- 172.21.15.157:0/1169876048 >> 172.21.15.157:0/1169876048 conn(0x7fb7940ff7a0 msgr2=0x7fb794107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:37:14.214 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:14.209+0000 7fb799e7f700 1 -- 172.21.15.157:0/1169876048 shutdown_connections 2024-05-09T00:37:14.214 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:14.209+0000 7fb799e7f700 1 -- 172.21.15.157:0/1169876048 wait complete. 2024-05-09T00:37:14.214 INFO:teuthology.orchestra.run.smithi157.stderr:dumped monmap epoch 2 2024-05-09T00:37:14.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:14 smithi005 ceph-mon[30365]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:14.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:14 smithi005 ceph-mon[30365]: from='client.? 172.21.15.157:0/1169876048' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T00:37:14.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:14 smithi157 ceph-mon[34294]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:14.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:14 smithi157 ceph-mon[34294]: from='client.? 172.21.15.157:0/1169876048' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T00:37:15.753 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-09T00:37:15.754 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-09T00:37:16.013 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T00:37:16.277 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: mon.b@-1(???) e0 preinit fsid d85814c6-0d9b-11ef-bc97-c7b262605968 2024-05-09T00:37:16.277 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: mon.b@-1(synchronizing).mds e1 new map 2024-05-09T00:37:16.278 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: mon.b@-1(synchronizing).mds e1 print_map 2024-05-09T00:37:16.278 INFO:journalctl@ceph.mon.b.smithi131.stdout: e1 2024-05-09T00:37:16.278 INFO:journalctl@ceph.mon.b.smithi131.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-09T00:37:16.278 INFO:journalctl@ceph.mon.b.smithi131.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-09T00:37:16.278 INFO:journalctl@ceph.mon.b.smithi131.stdout: legacy client fscid: -1 2024-05-09T00:37:16.278 INFO:journalctl@ceph.mon.b.smithi131.stdout: 2024-05-09T00:37:16.278 INFO:journalctl@ceph.mon.b.smithi131.stdout: No filesystems configured 2024-05-09T00:37:16.278 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-09T00:37:16.278 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-09T00:37:16.278 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-09T00:37:16.607 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-09T00:37:16.607 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-09T00:37:16.607 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-09T00:37:16.607 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-09T00:37:16.607 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T00:37:16.607 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T00:37:16.607 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T00:37:16.607 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:16.607 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T00:37:16.608 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:16.608 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:16.608 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi157", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:37:16.608 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:16.608 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:16.608 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:16.608 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:16.608 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi131", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:37:16.608 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:37:16.608 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:37:16.608 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: Updating smithi131:/etc/ceph/ceph.conf 2024-05-09T00:37:16.608 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: Deploying cephadm binary to smithi157 2024-05-09T00:37:16.609 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: Updating smithi131:/var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T00:37:16.609 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: Updating smithi131:/etc/ceph/ceph.client.admin.keyring 2024-05-09T00:37:16.609 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:16.609 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: Updating smithi131:/var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T00:37:16.609 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:16.609 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:16.609 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:16.609 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:16.609 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:16.609 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: Added host smithi157 2024-05-09T00:37:16.609 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:37:16.609 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:16.610 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:16.610 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:16.610 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:16.610 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T00:37:16.610 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:16.610 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/1333314358' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-09T00:37:16.610 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:16.610 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:16.610 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:16.610 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:16.610 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi157", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:37:16.611 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:37:16.611 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:37:16.611 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: Updating smithi157:/etc/ceph/ceph.conf 2024-05-09T00:37:16.611 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:16.611 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: Updating smithi157:/var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T00:37:16.611 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: Updating smithi157:/etc/ceph/ceph.client.admin.keyring 2024-05-09T00:37:16.611 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/1333314358' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-09T00:37:16.611 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T00:37:16.611 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:16.611 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:16.611 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:16.612 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: Updating smithi157:/var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T00:37:16.612 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:16.612 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:16.612 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:37:16.612 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:37:16.612 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:37:16.612 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:16.612 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T00:37:16.612 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:37:16.612 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi005:172.21.15.5=a;smithi131:172.21.15.131=b;smithi157:172.21.15.157=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:37:16.613 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: Saving service mon spec with placement smithi005:172.21.15.5=a;smithi131:172.21.15.131=b;smithi157:172.21.15.157=c;count:3 2024-05-09T00:37:16.613 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: Deploying daemon mon.c on smithi157 2024-05-09T00:37:16.613 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:16.613 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:16.613 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T00:37:16.613 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: mon.a calling monitor election 2024-05-09T00:37:16.614 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:37:16.614 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:16.614 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:37:16.614 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:37:16.614 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: mon.c calling monitor election 2024-05-09T00:37:16.614 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:16.614 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:37:16.614 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:37:16.614 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:16.614 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:37:16.614 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-09T00:37:16.614 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: monmap epoch 2 2024-05-09T00:37:16.615 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: fsid d85814c6-0d9b-11ef-bc97-c7b262605968 2024-05-09T00:37:16.615 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: last_changed 2024-05-09T00:37:07.467509+0000 2024-05-09T00:37:16.615 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: created 2024-05-09T00:35:22.099371+0000 2024-05-09T00:37:16.615 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: min_mon_release 19 (squid) 2024-05-09T00:37:16.615 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: election_strategy: 1 2024-05-09T00:37:16.615 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-05-09T00:37:16.615 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: 1: [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] mon.c 2024-05-09T00:37:16.615 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: fsmap 2024-05-09T00:37:16.615 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T00:37:16.615 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: mgrmap e13: x(active, since 50s) 2024-05-09T00:37:16.615 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: overall HEALTH_OK 2024-05-09T00:37:16.615 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:16.616 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:16.616 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:16.616 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T00:37:16.616 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:37:16.616 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: Deploying daemon mon.b on smithi131 2024-05-09T00:37:16.616 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:37:16.616 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:16.616 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: from='client.? 172.21.15.157:0/1169876048' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T00:37:16.616 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 ceph-mon[34547]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-09T00:37:16.616 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 bash[34528]: 88c17aba445508300f76ffc89657bbc1fb77190e6fc153f51692d293f8dc6395 2024-05-09T00:37:16.616 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:16 smithi131 systemd[1]: Started Ceph mon.b for d85814c6-0d9b-11ef-bc97-c7b262605968. 2024-05-09T00:37:21.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:21 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T00:37:21.688 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:21 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:37:21.688 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:21 smithi157 ceph-mon[34294]: mon.c calling monitor election 2024-05-09T00:37:21.688 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:21 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:37:21.688 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:21 smithi157 ceph-mon[34294]: mon.a calling monitor election 2024-05-09T00:37:21.688 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:21 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:37:21.688 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:21 smithi157 ceph-mon[34294]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:21.688 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:21 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:37:21.688 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:21 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:37:21.688 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:21 smithi157 ceph-mon[34294]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:21.688 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:21 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:37:21.688 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:21 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:37:21.688 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:21 smithi157 ceph-mon[34294]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-09T00:37:21.688 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:21 smithi157 ceph-mon[34294]: monmap epoch 3 2024-05-09T00:37:21.689 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:21 smithi157 ceph-mon[34294]: fsid d85814c6-0d9b-11ef-bc97-c7b262605968 2024-05-09T00:37:21.689 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:21 smithi157 ceph-mon[34294]: last_changed 2024-05-09T00:37:16.283086+0000 2024-05-09T00:37:21.689 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:21 smithi157 ceph-mon[34294]: created 2024-05-09T00:35:22.099371+0000 2024-05-09T00:37:21.689 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:21 smithi157 ceph-mon[34294]: min_mon_release 19 (squid) 2024-05-09T00:37:21.689 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:21 smithi157 ceph-mon[34294]: election_strategy: 1 2024-05-09T00:37:21.689 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:21 smithi157 ceph-mon[34294]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-05-09T00:37:21.689 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:21 smithi157 ceph-mon[34294]: 1: [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] mon.c 2024-05-09T00:37:21.689 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:21 smithi157 ceph-mon[34294]: 2: [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] mon.b 2024-05-09T00:37:21.689 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:21 smithi157 ceph-mon[34294]: fsmap 2024-05-09T00:37:21.689 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:21 smithi157 ceph-mon[34294]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T00:37:21.689 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:21 smithi157 ceph-mon[34294]: mgrmap e13: x(active, since 59s) 2024-05-09T00:37:21.689 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:21 smithi157 ceph-mon[34294]: overall HEALTH_OK 2024-05-09T00:37:21.689 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:21 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:21.689 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:21 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:21.690 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:21 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:21.690 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:21 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:21.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:21 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T00:37:21.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:21 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:37:21.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:21 smithi005 ceph-mon[30365]: mon.c calling monitor election 2024-05-09T00:37:21.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:21 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:37:21.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:21 smithi005 ceph-mon[30365]: mon.a calling monitor election 2024-05-09T00:37:21.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:21 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:37:21.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:21 smithi005 ceph-mon[30365]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:21.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:21 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:37:21.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:21 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:37:21.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:21 smithi005 ceph-mon[30365]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:21.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:21 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:37:21.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:21 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:37:21.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:21 smithi005 ceph-mon[30365]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-09T00:37:21.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:21 smithi005 ceph-mon[30365]: monmap epoch 3 2024-05-09T00:37:21.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:21 smithi005 ceph-mon[30365]: fsid d85814c6-0d9b-11ef-bc97-c7b262605968 2024-05-09T00:37:21.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:21 smithi005 ceph-mon[30365]: last_changed 2024-05-09T00:37:16.283086+0000 2024-05-09T00:37:21.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:21 smithi005 ceph-mon[30365]: created 2024-05-09T00:35:22.099371+0000 2024-05-09T00:37:21.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:21 smithi005 ceph-mon[30365]: min_mon_release 19 (squid) 2024-05-09T00:37:21.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:21 smithi005 ceph-mon[30365]: election_strategy: 1 2024-05-09T00:37:21.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:21 smithi005 ceph-mon[30365]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-05-09T00:37:21.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:21 smithi005 ceph-mon[30365]: 1: [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] mon.c 2024-05-09T00:37:21.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:21 smithi005 ceph-mon[30365]: 2: [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] mon.b 2024-05-09T00:37:21.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:21 smithi005 ceph-mon[30365]: fsmap 2024-05-09T00:37:21.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:21 smithi005 ceph-mon[30365]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T00:37:21.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:21 smithi005 ceph-mon[30365]: mgrmap e13: x(active, since 59s) 2024-05-09T00:37:21.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:21 smithi005 ceph-mon[30365]: overall HEALTH_OK 2024-05-09T00:37:21.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:21 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:21.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:21 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:21.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:21 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:21.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:21 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:21.892 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:21.890+0000 7f667628d700 1 -- 172.21.15.157:0/1523281415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66580035c0 msgr2=0x7f6658005aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:37:21.892 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:21.890+0000 7f667628d700 1 --2- 172.21.15.157:0/1523281415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66580035c0 0x7f6658005aa0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f667006b390 tx=0x7f666000fe60 comp rx=0 tx=0).stop 2024-05-09T00:37:21.892 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:21.890+0000 7f667628d700 1 -- 172.21.15.157:0/1523281415 shutdown_connections 2024-05-09T00:37:21.893 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:21.890+0000 7f667628d700 1 --2- 172.21.15.157:0/1523281415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66580035c0 0x7f6658005aa0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:37:21.893 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:21.890+0000 7f667628d700 1 -- 172.21.15.157:0/1523281415 >> 172.21.15.157:0/1523281415 conn(0x7f66700fe670 msgr2=0x7f6670100ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:37:21.893 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:21.890+0000 7f667628d700 1 -- 172.21.15.157:0/1523281415 shutdown_connections 2024-05-09T00:37:21.893 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:21.890+0000 7f667628d700 1 -- 172.21.15.157:0/1523281415 wait complete. 2024-05-09T00:37:21.893 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:21.891+0000 7f667628d700 1 Processor -- start 2024-05-09T00:37:21.893 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:21.891+0000 7f667628d700 1 -- start start 2024-05-09T00:37:21.895 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:21.891+0000 7f667628d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66580035c0 0x7f66701a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:37:21.895 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:21.891+0000 7f667628d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66701a0850 0x7f66701a4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:37:21.895 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:21.891+0000 7f667628d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66701a5080 0x7f66701a5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:37:21.895 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:21.891+0000 7f667628d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f66701121d0 con 0x7f66701a5080 2024-05-09T00:37:21.896 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:21.891+0000 7f667628d700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6670112050 con 0x7f66701a0850 2024-05-09T00:37:21.896 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:21.891+0000 7f667628d700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6670112350 con 0x7f66580035c0 2024-05-09T00:37:21.896 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:21.892+0000 7f666ffff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66580035c0 0x7f66701a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:37:21.896 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:21.892+0000 7f666ffff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66580035c0 0x7f66701a0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:41538/0 (socket says 172.21.15.157:41538) 2024-05-09T00:37:21.896 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:21.892+0000 7f666ffff700 1 -- 172.21.15.157:0/1281520560 learned_addr learned my addr 172.21.15.157:0/1281520560 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-05-09T00:37:21.896 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:21.892+0000 7f667482a700 1 --2- 172.21.15.157:0/1281520560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66701a5080 0x7f66701a5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:37:21.896 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:21.892+0000 7f666ffff700 1 -- 172.21.15.157:0/1281520560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66580035c0 msgr2=0x7f66701a0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-09T00:37:21.896 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:21.892+0000 7f666ffff700 1 -- 172.21.15.157:0/1281520560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66580035c0 msgr2=0x7f66701a0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-09T00:37:21.896 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:21.892+0000 7f666ffff700 1 --2- 172.21.15.157:0/1281520560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66580035c0 0x7f66701a0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T00:37:21.896 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:21.892+0000 7f666ffff700 1 --2- 172.21.15.157:0/1281520560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66580035c0 0x7f66701a0310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T00:37:21.896 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:21.892+0000 7f666f7fe700 1 --2- 172.21.15.157:0/1281520560 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66701a0850 0x7f66701a4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:37:21.896 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:21.892+0000 7f666f7fe700 1 -- 172.21.15.157:0/1281520560 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66701a0850 msgr2=0x7f66701a4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-09T00:37:21.896 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:21.892+0000 7f666f7fe700 1 -- 172.21.15.157:0/1281520560 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66701a0850 msgr2=0x7f66701a4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-09T00:37:21.897 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:21.892+0000 7f666f7fe700 1 --2- 172.21.15.157:0/1281520560 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66701a0850 0x7f66701a4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T00:37:21.897 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:21.892+0000 7f666f7fe700 1 --2- 172.21.15.157:0/1281520560 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66701a0850 0x7f66701a4b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T00:37:21.897 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:21.892+0000 7f667482a700 1 -- 172.21.15.157:0/1281520560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66580035c0 msgr2=0x7f66701a0310 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T00:37:21.897 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:21.892+0000 7f667482a700 1 --2- 172.21.15.157:0/1281520560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66580035c0 0x7f66701a0310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:37:21.897 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:21.892+0000 7f667482a700 1 -- 172.21.15.157:0/1281520560 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66701a0850 msgr2=0x7f66701a4b40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T00:37:21.897 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:21.892+0000 7f667482a700 1 --2- 172.21.15.157:0/1281520560 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66701a0850 0x7f66701a4b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:37:21.897 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:21.892+0000 7f667482a700 1 -- 172.21.15.157:0/1281520560 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66701acdf0 con 0x7f66701a5080 2024-05-09T00:37:21.897 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:21.892+0000 7f667482a700 1 --2- 172.21.15.157:0/1281520560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66701a5080 0x7f66701a5490 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f666800bd50 tx=0x7f666800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:37:21.897 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:21.893+0000 7f666d7fa700 1 -- 172.21.15.157:0/1281520560 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6668019070 con 0x7f66701a5080 2024-05-09T00:37:21.897 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:21.893+0000 7f666d7fa700 1 -- 172.21.15.157:0/1281520560 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f666800cd10 con 0x7f66701a5080 2024-05-09T00:37:21.897 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:21.893+0000 7f667628d700 1 -- 172.21.15.157:0/1281520560 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66701acfc0 con 0x7f66701a5080 2024-05-09T00:37:21.897 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:21.893+0000 7f666d7fa700 1 -- 172.21.15.157:0/1281520560 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6668012700 con 0x7f66701a5080 2024-05-09T00:37:21.897 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:21.893+0000 7f667628d700 1 -- 172.21.15.157:0/1281520560 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66701ad410 con 0x7f66701a5080 2024-05-09T00:37:21.898 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:21.894+0000 7f666d7fa700 1 -- 172.21.15.157:0/1281520560 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f666801c070 con 0x7f66701a5080 2024-05-09T00:37:21.898 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:21.894+0000 7f667628d700 1 -- 172.21.15.157:0/1281520560 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6670064e20 con 0x7f66701a5080 2024-05-09T00:37:21.898 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:21.894+0000 7f666d7fa700 1 --2- 172.21.15.157:0/1281520560 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f665c03e9a0 0x7f665c040eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:37:21.898 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:21.894+0000 7f666d7fa700 1 -- 172.21.15.157:0/1281520560 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f6668052ff0 con 0x7f66701a5080 2024-05-09T00:37:21.898 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:21.895+0000 7f666ffff700 1 --2- 172.21.15.157:0/1281520560 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f665c03e9a0 0x7f665c040eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:37:21.898 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:21.896+0000 7f666ffff700 1 --2- 172.21.15.157:0/1281520560 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f665c03e9a0 0x7f665c040eb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f666000fa10 tx=0x7f666000f4e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:37:21.901 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:21.900+0000 7f666d7fa700 1 -- 172.21.15.157:0/1281520560 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f66680133a0 con 0x7f66701a5080 2024-05-09T00:37:22.232 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:22.230+0000 7f667628d700 1 -- 172.21.15.157:0/1281520560 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f66701ad890 con 0x7f66701a5080 2024-05-09T00:37:22.232 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:22.231+0000 7f666d7fa700 1 -- 172.21.15.157:0/1281520560 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1279 (secure 0 0 0) 0x7f6668014a10 con 0x7f66701a5080 2024-05-09T00:37:22.233 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-05-09T00:37:22.233 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":3,"fsid":"d85814c6-0d9b-11ef-bc97-c7b262605968","modified":"2024-05-09T00:37:16.283086Z","created":"2024-05-09T00:35:22.099371Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:3300","nonce":0},{"type":"v1","addr":"172.21.15.5:6789","nonce":0}]},"addr":"172.21.15.5:6789/0","public_addr":"172.21.15.5:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:3300","nonce":0},{"type":"v1","addr":"172.21.15.157:6789","nonce":0}]},"addr":"172.21.15.157:6789/0","public_addr":"172.21.15.157:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:3300","nonce":0},{"type":"v1","addr":"172.21.15.131:6789","nonce":0}]},"addr":"172.21.15.131:6789/0","public_addr":"172.21.15.131:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-09T00:37:22.235 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:22.233+0000 7f667628d700 1 -- 172.21.15.157:0/1281520560 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f665c03e9a0 msgr2=0x7f665c040eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:37:22.235 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:22.233+0000 7f667628d700 1 --2- 172.21.15.157:0/1281520560 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f665c03e9a0 0x7f665c040eb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f666000fa10 tx=0x7f666000f4e0 comp rx=0 tx=0).stop 2024-05-09T00:37:22.235 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:22.233+0000 7f667628d700 1 -- 172.21.15.157:0/1281520560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66701a5080 msgr2=0x7f66701a5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:37:22.235 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:22.233+0000 7f667628d700 1 --2- 172.21.15.157:0/1281520560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66701a5080 0x7f66701a5490 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f666800bd50 tx=0x7f666800be30 comp rx=0 tx=0).stop 2024-05-09T00:37:22.235 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:22.233+0000 7f667628d700 1 -- 172.21.15.157:0/1281520560 shutdown_connections 2024-05-09T00:37:22.235 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:22.233+0000 7f667628d700 1 --2- 172.21.15.157:0/1281520560 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f665c03e9a0 0x7f665c040eb0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:37:22.235 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:22.233+0000 7f667628d700 1 --2- 172.21.15.157:0/1281520560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66701a5080 0x7f66701a5490 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:37:22.235 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:22.233+0000 7f667628d700 1 --2- 172.21.15.157:0/1281520560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66580035c0 0x7f66701a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:37:22.236 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:22.233+0000 7f667628d700 1 --2- 172.21.15.157:0/1281520560 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66701a0850 0x7f66701a4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:37:22.236 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:22.233+0000 7f667628d700 1 -- 172.21.15.157:0/1281520560 >> 172.21.15.157:0/1281520560 conn(0x7f66700fe670 msgr2=0x7f66701016f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:37:22.236 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:22.233+0000 7f667628d700 1 -- 172.21.15.157:0/1281520560 shutdown_connections 2024-05-09T00:37:22.236 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:22.233+0000 7f667628d700 1 -- 172.21.15.157:0/1281520560 wait complete. 2024-05-09T00:37:22.236 INFO:teuthology.orchestra.run.smithi157.stderr:dumped monmap epoch 3 2024-05-09T00:37:22.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:22 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:37:22.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:22 smithi157 ceph-mon[34294]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:22.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:22 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:22.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:22 smithi157 ceph-mon[34294]: from='client.? 172.21.15.157:0/1281520560' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T00:37:22.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:22 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:37:22.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:22 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:37:22.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:22 smithi005 ceph-mon[30365]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:22.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:22 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:22.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:22 smithi005 ceph-mon[30365]: from='client.? 172.21.15.157:0/1281520560' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T00:37:22.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:22 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:37:23.041 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-09T00:37:23.042 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph config generate-minimal-conf 2024-05-09T00:37:23.302 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:37:23.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:23 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T00:37:23.607 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:23 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:37:23.607 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:23 smithi131 ceph-mon[34547]: mon.c calling monitor election 2024-05-09T00:37:23.607 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:23 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:37:23.607 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:23 smithi131 ceph-mon[34547]: mon.a calling monitor election 2024-05-09T00:37:23.607 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:23 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:37:23.607 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:23 smithi131 ceph-mon[34547]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:23.607 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:23 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:37:23.607 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:23 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:37:23.607 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:23 smithi131 ceph-mon[34547]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:23.607 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:23 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:37:23.607 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:23 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:37:23.607 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:23 smithi131 ceph-mon[34547]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-09T00:37:23.607 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:23 smithi131 ceph-mon[34547]: monmap epoch 3 2024-05-09T00:37:23.608 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:23 smithi131 ceph-mon[34547]: fsid d85814c6-0d9b-11ef-bc97-c7b262605968 2024-05-09T00:37:23.608 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:23 smithi131 ceph-mon[34547]: last_changed 2024-05-09T00:37:16.283086+0000 2024-05-09T00:37:23.608 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:23 smithi131 ceph-mon[34547]: created 2024-05-09T00:35:22.099371+0000 2024-05-09T00:37:23.608 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:23 smithi131 ceph-mon[34547]: min_mon_release 19 (squid) 2024-05-09T00:37:23.608 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:23 smithi131 ceph-mon[34547]: election_strategy: 1 2024-05-09T00:37:23.608 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:23 smithi131 ceph-mon[34547]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-05-09T00:37:23.608 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:23 smithi131 ceph-mon[34547]: 1: [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] mon.c 2024-05-09T00:37:23.608 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:23 smithi131 ceph-mon[34547]: 2: [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] mon.b 2024-05-09T00:37:23.608 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:23 smithi131 ceph-mon[34547]: fsmap 2024-05-09T00:37:23.608 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:23 smithi131 ceph-mon[34547]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T00:37:23.608 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:23 smithi131 ceph-mon[34547]: mgrmap e13: x(active, since 59s) 2024-05-09T00:37:23.608 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:23 smithi131 ceph-mon[34547]: overall HEALTH_OK 2024-05-09T00:37:23.608 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:23 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:23.608 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:23 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:23.609 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:23 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:23.609 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:23 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:23.609 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:23 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:37:23.609 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:23 smithi131 ceph-mon[34547]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:23.609 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:23 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:23.609 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:23 smithi131 ceph-mon[34547]: from='client.? 172.21.15.157:0/1281520560' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T00:37:23.609 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:23 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:37:24.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:23 smithi005 ceph-mon[30365]: mon.b calling monitor election 2024-05-09T00:37:24.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:23 smithi005 ceph-mon[30365]: mon.b calling monitor election 2024-05-09T00:37:24.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:23 smithi005 ceph-mon[30365]: mon.c calling monitor election 2024-05-09T00:37:24.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:23 smithi005 ceph-mon[30365]: mon.a calling monitor election 2024-05-09T00:37:24.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:23 smithi005 ceph-mon[30365]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-09T00:37:24.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:23 smithi005 ceph-mon[30365]: monmap epoch 3 2024-05-09T00:37:24.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:23 smithi005 ceph-mon[30365]: fsid d85814c6-0d9b-11ef-bc97-c7b262605968 2024-05-09T00:37:24.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:23 smithi005 ceph-mon[30365]: last_changed 2024-05-09T00:37:16.283086+0000 2024-05-09T00:37:24.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:23 smithi005 ceph-mon[30365]: created 2024-05-09T00:35:22.099371+0000 2024-05-09T00:37:24.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:23 smithi005 ceph-mon[30365]: min_mon_release 19 (squid) 2024-05-09T00:37:24.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:23 smithi005 ceph-mon[30365]: election_strategy: 1 2024-05-09T00:37:24.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:23 smithi005 ceph-mon[30365]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-05-09T00:37:24.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:23 smithi005 ceph-mon[30365]: 1: [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] mon.c 2024-05-09T00:37:24.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:23 smithi005 ceph-mon[30365]: 2: [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] mon.b 2024-05-09T00:37:24.043 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:23 smithi005 ceph-mon[30365]: fsmap 2024-05-09T00:37:24.043 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:23 smithi005 ceph-mon[30365]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T00:37:24.043 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:23 smithi005 ceph-mon[30365]: mgrmap e13: x(active, since 61s) 2024-05-09T00:37:24.043 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:23 smithi005 ceph-mon[30365]: overall HEALTH_OK 2024-05-09T00:37:24.043 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:23 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:24.043 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:23 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:24.043 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:23 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:37:24.043 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:23 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:37:24.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:23 smithi131 ceph-mon[34547]: mon.b calling monitor election 2024-05-09T00:37:24.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:23 smithi131 ceph-mon[34547]: mon.b calling monitor election 2024-05-09T00:37:24.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:23 smithi131 ceph-mon[34547]: mon.c calling monitor election 2024-05-09T00:37:24.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:23 smithi131 ceph-mon[34547]: mon.a calling monitor election 2024-05-09T00:37:24.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:23 smithi131 ceph-mon[34547]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-09T00:37:24.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:23 smithi131 ceph-mon[34547]: monmap epoch 3 2024-05-09T00:37:24.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:23 smithi131 ceph-mon[34547]: fsid d85814c6-0d9b-11ef-bc97-c7b262605968 2024-05-09T00:37:24.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:23 smithi131 ceph-mon[34547]: last_changed 2024-05-09T00:37:16.283086+0000 2024-05-09T00:37:24.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:23 smithi131 ceph-mon[34547]: created 2024-05-09T00:35:22.099371+0000 2024-05-09T00:37:24.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:23 smithi131 ceph-mon[34547]: min_mon_release 19 (squid) 2024-05-09T00:37:24.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:23 smithi131 ceph-mon[34547]: election_strategy: 1 2024-05-09T00:37:24.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:23 smithi131 ceph-mon[34547]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-05-09T00:37:24.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:23 smithi131 ceph-mon[34547]: 1: [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] mon.c 2024-05-09T00:37:24.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:23 smithi131 ceph-mon[34547]: 2: [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] mon.b 2024-05-09T00:37:24.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:23 smithi131 ceph-mon[34547]: fsmap 2024-05-09T00:37:24.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:23 smithi131 ceph-mon[34547]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T00:37:24.108 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:23 smithi131 ceph-mon[34547]: mgrmap e13: x(active, since 61s) 2024-05-09T00:37:24.108 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:23 smithi131 ceph-mon[34547]: overall HEALTH_OK 2024-05-09T00:37:24.108 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:23 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:24.108 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:23 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:24.108 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:23 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:37:24.108 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:23 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:37:24.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:23 smithi157 ceph-mon[34294]: mon.b calling monitor election 2024-05-09T00:37:24.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:23 smithi157 ceph-mon[34294]: mon.b calling monitor election 2024-05-09T00:37:24.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:23 smithi157 ceph-mon[34294]: mon.c calling monitor election 2024-05-09T00:37:24.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:23 smithi157 ceph-mon[34294]: mon.a calling monitor election 2024-05-09T00:37:24.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:23 smithi157 ceph-mon[34294]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-09T00:37:24.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:23 smithi157 ceph-mon[34294]: monmap epoch 3 2024-05-09T00:37:24.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:23 smithi157 ceph-mon[34294]: fsid d85814c6-0d9b-11ef-bc97-c7b262605968 2024-05-09T00:37:24.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:23 smithi157 ceph-mon[34294]: last_changed 2024-05-09T00:37:16.283086+0000 2024-05-09T00:37:24.188 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:23 smithi157 ceph-mon[34294]: created 2024-05-09T00:35:22.099371+0000 2024-05-09T00:37:24.188 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:23 smithi157 ceph-mon[34294]: min_mon_release 19 (squid) 2024-05-09T00:37:24.188 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:23 smithi157 ceph-mon[34294]: election_strategy: 1 2024-05-09T00:37:24.188 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:23 smithi157 ceph-mon[34294]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-05-09T00:37:24.188 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:23 smithi157 ceph-mon[34294]: 1: [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] mon.c 2024-05-09T00:37:24.188 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:23 smithi157 ceph-mon[34294]: 2: [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] mon.b 2024-05-09T00:37:24.188 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:23 smithi157 ceph-mon[34294]: fsmap 2024-05-09T00:37:24.188 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:23 smithi157 ceph-mon[34294]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T00:37:24.188 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:23 smithi157 ceph-mon[34294]: mgrmap e13: x(active, since 61s) 2024-05-09T00:37:24.188 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:23 smithi157 ceph-mon[34294]: overall HEALTH_OK 2024-05-09T00:37:24.188 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:23 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:24.188 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:23 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:24.188 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:23 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:37:24.189 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:23 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:37:24.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:24.908+0000 7f0224311700 1 -- 172.21.15.5:0/1939559962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f021c108bc0 msgr2=0x7f021c108fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:37:24.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:24.908+0000 7f0224311700 1 --2- 172.21.15.5:0/1939559962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f021c108bc0 0x7f021c108fd0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f020c00b0a0 tx=0x7f020c00b3b0 comp rx=0 tx=0).stop 2024-05-09T00:37:24.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:24.908+0000 7f0224311700 1 -- 172.21.15.5:0/1939559962 shutdown_connections 2024-05-09T00:37:24.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:24.908+0000 7f0224311700 1 --2- 172.21.15.5:0/1939559962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f021c108bc0 0x7f021c108fd0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:37:24.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:24.908+0000 7f0224311700 1 -- 172.21.15.5:0/1939559962 >> 172.21.15.5:0/1939559962 conn(0x7f021c078d60 msgr2=0x7f021c0791a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:37:24.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:24.908+0000 7f0224311700 1 -- 172.21.15.5:0/1939559962 shutdown_connections 2024-05-09T00:37:24.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:24.908+0000 7f0224311700 1 -- 172.21.15.5:0/1939559962 wait complete. 2024-05-09T00:37:24.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:24.909+0000 7f0224311700 1 Processor -- start 2024-05-09T00:37:24.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:24.909+0000 7f0224311700 1 -- start start 2024-05-09T00:37:24.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:24.909+0000 7f0224311700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f021c108bc0 0x7f021c199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:37:24.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:24.909+0000 7f0224311700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f021c19a250 0x7f021c19e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:37:24.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:24.909+0000 7f0224311700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f021c19ea80 0x7f021c19ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:37:24.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:24.909+0000 7f0224311700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f021c10e200 con 0x7f021c108bc0 2024-05-09T00:37:24.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:24.909+0000 7f0224311700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f021c10e080 con 0x7f021c19a250 2024-05-09T00:37:24.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:24.909+0000 7f0224311700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f021c10e380 con 0x7f021c19ea80 2024-05-09T00:37:24.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:24.909+0000 7f02220ad700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f021c108bc0 0x7f021c199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:37:24.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:24.909+0000 7f02220ad700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f021c108bc0 0x7f021c199d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58738/0 (socket says 172.21.15.5:58738) 2024-05-09T00:37:24.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:24.909+0000 7f02220ad700 1 -- 172.21.15.5:0/1437775900 learned_addr learned my addr 172.21.15.5:0/1437775900 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:37:24.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:24.909+0000 7f02220ad700 1 -- 172.21.15.5:0/1437775900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f021c19ea80 msgr2=0x7f021c19ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:37:24.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:24.909+0000 7f02220ad700 1 --2- 172.21.15.5:0/1437775900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f021c19ea80 0x7f021c19ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:37:24.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:24.909+0000 7f02220ad700 1 -- 172.21.15.5:0/1437775900 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f021c19a250 msgr2=0x7f021c19e540 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T00:37:24.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:24.909+0000 7f02220ad700 1 --2- 172.21.15.5:0/1437775900 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f021c19a250 0x7f021c19e540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:37:24.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:24.909+0000 7f02220ad700 1 -- 172.21.15.5:0/1437775900 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f021c1a67f0 con 0x7f021c108bc0 2024-05-09T00:37:24.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:24.910+0000 7f02220ad700 1 --2- 172.21.15.5:0/1437775900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f021c108bc0 0x7f021c199d10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f020c015040 tx=0x7f020c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:37:24.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:24.910+0000 7f021b7fe700 1 -- 172.21.15.5:0/1437775900 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f020c020070 con 0x7f021c108bc0 2024-05-09T00:37:24.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:24.910+0000 7f0224311700 1 -- 172.21.15.5:0/1437775900 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f021c1a69c0 con 0x7f021c108bc0 2024-05-09T00:37:24.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:24.910+0000 7f021b7fe700 1 -- 172.21.15.5:0/1437775900 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f020c003e40 con 0x7f021c108bc0 2024-05-09T00:37:24.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:24.910+0000 7f021b7fe700 1 -- 172.21.15.5:0/1437775900 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f020c017070 con 0x7f021c108bc0 2024-05-09T00:37:24.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:24.910+0000 7f0224311700 1 -- 172.21.15.5:0/1437775900 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f021c1a6f00 con 0x7f021c108bc0 2024-05-09T00:37:24.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:24.911+0000 7f021b7fe700 1 -- 172.21.15.5:0/1437775900 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f020c012bb0 con 0x7f021c108bc0 2024-05-09T00:37:24.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:24.911+0000 7f0224311700 1 -- 172.21.15.5:0/1437775900 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f021c064e20 con 0x7f021c108bc0 2024-05-09T00:37:24.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:24.911+0000 7f021b7fe700 1 --2- 172.21.15.5:0/1437775900 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f020403e9a0 0x7f0204040eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:37:24.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:24.911+0000 7f021b7fe700 1 -- 172.21.15.5:0/1437775900 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f020c00f020 con 0x7f021c108bc0 2024-05-09T00:37:24.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:24.912+0000 7f02218ac700 1 --2- 172.21.15.5:0/1437775900 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f020403e9a0 0x7f0204040eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:37:24.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:24.912+0000 7f02218ac700 1 --2- 172.21.15.5:0/1437775900 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f020403e9a0 0x7f0204040eb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f021c100120 tx=0x7f0210006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:37:24.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:24.917+0000 7f021b7fe700 1 -- 172.21.15.5:0/1437775900 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f020c012430 con 0x7f021c108bc0 2024-05-09T00:37:25.001 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:24 smithi005 ceph-mon[30365]: Updating smithi005:/etc/ceph/ceph.conf 2024-05-09T00:37:25.002 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:24 smithi005 ceph-mon[30365]: Updating smithi131:/etc/ceph/ceph.conf 2024-05-09T00:37:25.002 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:24 smithi005 ceph-mon[30365]: Updating smithi157:/etc/ceph/ceph.conf 2024-05-09T00:37:25.002 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:24 smithi005 ceph-mon[30365]: Updating smithi005:/var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T00:37:25.002 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:24 smithi005 ceph-mon[30365]: Updating smithi131:/var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T00:37:25.002 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:24 smithi005 ceph-mon[30365]: Updating smithi157:/var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T00:37:25.002 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:24 smithi005 ceph-mon[30365]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:25.002 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:24 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:25.002 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:24 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:25.002 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:24 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:25.002 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:24 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:25.003 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:24 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:25.003 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:24 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:25.003 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:24 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:25.003 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:24 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:25.003 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:24 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:25.003 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:24 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:25.003 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:24 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:25.003 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:24 smithi005 ceph-mon[30365]: Reconfiguring mon.a (unknown last config time)... 2024-05-09T00:37:25.003 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:24 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T00:37:25.003 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:24 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T00:37:25.003 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:24 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:37:25.003 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:24 smithi005 ceph-mon[30365]: Reconfiguring daemon mon.a on smithi005 2024-05-09T00:37:25.003 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:24 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:37:25.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:25.136+0000 7f0224311700 1 -- 172.21.15.5:0/1437775900 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f021c1a7230 con 0x7f021c108bc0 2024-05-09T00:37:25.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:25.136+0000 7f021b7fe700 1 -- 172.21.15.5:0/1437775900 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7f020c01cd20 con 0x7f021c108bc0 2024-05-09T00:37:25.138 INFO:teuthology.orchestra.run.smithi005.stdout:# minimal ceph.conf for d85814c6-0d9b-11ef-bc97-c7b262605968 2024-05-09T00:37:25.138 INFO:teuthology.orchestra.run.smithi005.stdout:[global] 2024-05-09T00:37:25.138 INFO:teuthology.orchestra.run.smithi005.stdout: fsid = d85814c6-0d9b-11ef-bc97-c7b262605968 2024-05-09T00:37:25.138 INFO:teuthology.orchestra.run.smithi005.stdout: mon_host = [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] 2024-05-09T00:37:25.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:25.139+0000 7f0224311700 1 -- 172.21.15.5:0/1437775900 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f020403e9a0 msgr2=0x7f0204040eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:37:25.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:25.139+0000 7f0224311700 1 --2- 172.21.15.5:0/1437775900 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f020403e9a0 0x7f0204040eb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f021c100120 tx=0x7f0210006d20 comp rx=0 tx=0).stop 2024-05-09T00:37:25.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:25.139+0000 7f0224311700 1 -- 172.21.15.5:0/1437775900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f021c108bc0 msgr2=0x7f021c199d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:37:25.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:25.139+0000 7f0224311700 1 --2- 172.21.15.5:0/1437775900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f021c108bc0 0x7f021c199d10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f020c015040 tx=0x7f020c009580 comp rx=0 tx=0).stop 2024-05-09T00:37:25.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:25.139+0000 7f0224311700 1 -- 172.21.15.5:0/1437775900 shutdown_connections 2024-05-09T00:37:25.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:25.139+0000 7f0224311700 1 --2- 172.21.15.5:0/1437775900 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f020403e9a0 0x7f0204040eb0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:37:25.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:25.139+0000 7f0224311700 1 --2- 172.21.15.5:0/1437775900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f021c19ea80 0x7f021c19ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:37:25.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:25.139+0000 7f0224311700 1 --2- 172.21.15.5:0/1437775900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f021c108bc0 0x7f021c199d10 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:37:25.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:25.139+0000 7f0224311700 1 --2- 172.21.15.5:0/1437775900 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f021c19a250 0x7f021c19e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:37:25.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:25.139+0000 7f0224311700 1 -- 172.21.15.5:0/1437775900 >> 172.21.15.5:0/1437775900 conn(0x7f021c078d60 msgr2=0x7f021c0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:37:25.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:25.139+0000 7f0224311700 1 -- 172.21.15.5:0/1437775900 shutdown_connections 2024-05-09T00:37:25.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:25.139+0000 7f0224311700 1 -- 172.21.15.5:0/1437775900 wait complete. 2024-05-09T00:37:25.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:24 smithi157 ceph-mon[34294]: Updating smithi005:/etc/ceph/ceph.conf 2024-05-09T00:37:25.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:24 smithi157 ceph-mon[34294]: Updating smithi131:/etc/ceph/ceph.conf 2024-05-09T00:37:25.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:24 smithi157 ceph-mon[34294]: Updating smithi157:/etc/ceph/ceph.conf 2024-05-09T00:37:25.188 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:24 smithi157 ceph-mon[34294]: Updating smithi005:/var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T00:37:25.188 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:24 smithi157 ceph-mon[34294]: Updating smithi131:/var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T00:37:25.188 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:24 smithi157 ceph-mon[34294]: Updating smithi157:/var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T00:37:25.188 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:24 smithi157 ceph-mon[34294]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:25.188 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:24 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:25.188 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:24 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:25.188 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:24 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:25.188 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:24 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:25.188 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:24 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:25.188 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:24 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:25.188 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:24 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:25.188 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:24 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:25.189 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:24 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:25.189 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:24 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:25.189 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:24 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:25.189 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:24 smithi157 ceph-mon[34294]: Reconfiguring mon.a (unknown last config time)... 2024-05-09T00:37:25.189 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:24 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T00:37:25.189 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:24 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T00:37:25.189 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:24 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:37:25.189 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:24 smithi157 ceph-mon[34294]: Reconfiguring daemon mon.a on smithi005 2024-05-09T00:37:25.189 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:24 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:37:25.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:24 smithi131 ceph-mon[34547]: Updating smithi005:/etc/ceph/ceph.conf 2024-05-09T00:37:25.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:24 smithi131 ceph-mon[34547]: Updating smithi131:/etc/ceph/ceph.conf 2024-05-09T00:37:25.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:24 smithi131 ceph-mon[34547]: Updating smithi157:/etc/ceph/ceph.conf 2024-05-09T00:37:25.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:24 smithi131 ceph-mon[34547]: Updating smithi005:/var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T00:37:25.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:24 smithi131 ceph-mon[34547]: Updating smithi131:/var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T00:37:25.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:24 smithi131 ceph-mon[34547]: Updating smithi157:/var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T00:37:25.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:24 smithi131 ceph-mon[34547]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:25.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:24 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:25.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:24 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:25.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:24 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:25.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:24 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:25.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:24 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:25.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:24 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:25.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:24 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:25.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:24 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:25.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:24 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:25.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:24 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:25.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:24 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:25.358 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:24 smithi131 ceph-mon[34547]: Reconfiguring mon.a (unknown last config time)... 2024-05-09T00:37:25.358 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:24 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T00:37:25.358 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:24 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T00:37:25.358 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:24 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:37:25.358 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:24 smithi131 ceph-mon[34547]: Reconfiguring daemon mon.a on smithi005 2024-05-09T00:37:25.358 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:24 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:37:25.541 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:37:25 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:37:25.284+0000 7f76776a8700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-05-09T00:37:26.087 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-09T00:37:26.087 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-09T00:37:26.088 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/ceph/ceph.conf 2024-05-09T00:37:26.115 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-09T00:37:26.116 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-09T00:37:26.179 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:25 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/1437775900' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:37:26.183 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-09T00:37:26.183 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/ceph/ceph.conf 2024-05-09T00:37:26.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:25 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/1437775900' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:37:26.209 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:25 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/1437775900' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:37:26.214 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-09T00:37:26.214 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-09T00:37:26.283 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-05-09T00:37:26.283 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/ceph/ceph.conf 2024-05-09T00:37:26.315 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-05-09T00:37:26.315 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-09T00:37:26.383 INFO:tasks.cephadm:Adding mgr.x on smithi005 2024-05-09T00:37:26.383 INFO:tasks.cephadm:Adding mgr.y on smithi131 2024-05-09T00:37:26.383 INFO:tasks.cephadm:Adding mgr.z on smithi157 2024-05-09T00:37:26.383 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph orch apply mgr '3;smithi005=x;smithi131=y;smithi157=z' 2024-05-09T00:37:26.658 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T00:37:27.119 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:26 smithi157 ceph-mon[34294]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:27.120 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:26 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:27.120 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:26 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:27.120 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:26 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T00:37:27.120 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:26 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T00:37:27.120 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:26 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:37:27.235 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:26 smithi131 ceph-mon[34547]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:27.235 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:26 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:27.235 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:26 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:27.235 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:26 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T00:37:27.235 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:26 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T00:37:27.235 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:26 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:37:27.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:26 smithi005 ceph-mon[30365]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:27.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:26 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:27.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:26 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:27.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:26 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T00:37:27.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:26 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T00:37:27.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:26 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:37:28.008 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:27 smithi157 ceph-mon[34294]: Reconfiguring mon.b (monmap changed)... 2024-05-09T00:37:28.008 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:27 smithi157 ceph-mon[34294]: Reconfiguring daemon mon.b on smithi131 2024-05-09T00:37:28.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:27 smithi131 ceph-mon[34547]: Reconfiguring mon.b (monmap changed)... 2024-05-09T00:37:28.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:27 smithi131 ceph-mon[34547]: Reconfiguring daemon mon.b on smithi131 2024-05-09T00:37:28.228 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:28.225+0000 7f725f53b700 1 -- 172.21.15.157:0/1790539421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7258102bf0 msgr2=0x7f7258103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:37:28.228 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:28.225+0000 7f725f53b700 1 --2- 172.21.15.157:0/1790539421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7258102bf0 0x7f7258103000 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7254009910 tx=0x7f7254009c20 comp rx=0 tx=0).stop 2024-05-09T00:37:28.228 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:28.225+0000 7f725f53b700 1 -- 172.21.15.157:0/1790539421 shutdown_connections 2024-05-09T00:37:28.228 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:28.225+0000 7f725f53b700 1 --2- 172.21.15.157:0/1790539421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7258102bf0 0x7f7258103000 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:37:28.228 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:28.225+0000 7f725f53b700 1 -- 172.21.15.157:0/1790539421 >> 172.21.15.157:0/1790539421 conn(0x7f72580fe250 msgr2=0x7f72581006e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:37:28.228 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:28.225+0000 7f725f53b700 1 -- 172.21.15.157:0/1790539421 shutdown_connections 2024-05-09T00:37:28.228 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:28.225+0000 7f725f53b700 1 -- 172.21.15.157:0/1790539421 wait complete. 2024-05-09T00:37:28.231 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:28.226+0000 7f725f53b700 1 Processor -- start 2024-05-09T00:37:28.231 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:28.226+0000 7f725f53b700 1 -- start start 2024-05-09T00:37:28.231 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:28.226+0000 7f725f53b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7258102bf0 0x7f72581999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:37:28.231 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:28.226+0000 7f725f53b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7258199f10 0x7f725819eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:37:28.231 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:28.226+0000 7f725f53b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f725819f3e0 0x7f72581a1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:37:28.231 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:28.226+0000 7f725f53b700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7258108bb0 con 0x7f725819f3e0 2024-05-09T00:37:28.231 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:28.226+0000 7f725f53b700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7258108a30 con 0x7f7258199f10 2024-05-09T00:37:28.231 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:28.226+0000 7f725f53b700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7258108d30 con 0x7f7258102bf0 2024-05-09T00:37:28.231 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:28.227+0000 7f725d2d7700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7258102bf0 0x7f72581999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:37:28.231 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:28.227+0000 7f725d2d7700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7258102bf0 0x7f72581999d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:41550/0 (socket says 172.21.15.157:41550) 2024-05-09T00:37:28.232 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:28.227+0000 7f725d2d7700 1 -- 172.21.15.157:0/2903414909 learned_addr learned my addr 172.21.15.157:0/2903414909 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-05-09T00:37:28.232 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:28.227+0000 7f725cad6700 1 --2- 172.21.15.157:0/2903414909 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7258199f10 0x7f725819eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:37:28.232 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:28.227+0000 7f725dad8700 1 --2- 172.21.15.157:0/2903414909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f725819f3e0 0x7f72581a1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:37:28.232 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:28.227+0000 7f725d2d7700 1 -- 172.21.15.157:0/2903414909 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7258199f10 msgr2=0x7f725819eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:37:28.232 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:28.227+0000 7f725d2d7700 1 --2- 172.21.15.157:0/2903414909 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7258199f10 0x7f725819eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:37:28.232 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:28.227+0000 7f725d2d7700 1 -- 172.21.15.157:0/2903414909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f725819f3e0 msgr2=0x7f72581a1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:37:28.232 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:28.227+0000 7f725d2d7700 1 --2- 172.21.15.157:0/2903414909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f725819f3e0 0x7f72581a1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:37:28.232 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:28.227+0000 7f725d2d7700 1 -- 172.21.15.157:0/2903414909 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72581a1d70 con 0x7f7258102bf0 2024-05-09T00:37:28.232 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:28.227+0000 7f725dad8700 1 --2- 172.21.15.157:0/2903414909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f725819f3e0 0x7f72581a1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:37:28.232 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:28.227+0000 7f725d2d7700 1 --2- 172.21.15.157:0/2903414909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7258102bf0 0x7f72581999d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f7254005850 tx=0x7f7254004ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:37:28.232 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:28.227+0000 7f724e7fc700 1 -- 172.21.15.157:0/2903414909 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7254026070 con 0x7f7258102bf0 2024-05-09T00:37:28.232 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:28.227+0000 7f724e7fc700 1 -- 172.21.15.157:0/2903414909 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f725400b750 con 0x7f7258102bf0 2024-05-09T00:37:28.232 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:28.227+0000 7f725f53b700 1 -- 172.21.15.157:0/2903414909 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72581a2000 con 0x7f7258102bf0 2024-05-09T00:37:28.233 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:28.227+0000 7f724e7fc700 1 -- 172.21.15.157:0/2903414909 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7254003970 con 0x7f7258102bf0 2024-05-09T00:37:28.233 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:28.228+0000 7f725f53b700 1 -- 172.21.15.157:0/2903414909 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72581a2560 con 0x7f7258102bf0 2024-05-09T00:37:28.233 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:28.228+0000 7f724e7fc700 1 -- 172.21.15.157:0/2903414909 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f7254029070 con 0x7f7258102bf0 2024-05-09T00:37:28.233 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:28.229+0000 7f724e7fc700 1 --2- 172.21.15.157:0/2903414909 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f724403e9a0 0x7f7244040eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:37:28.233 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:28.229+0000 7f724e7fc700 1 -- 172.21.15.157:0/2903414909 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f725405a060 con 0x7f7258102bf0 2024-05-09T00:37:28.233 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:28.229+0000 7f725cad6700 1 --2- 172.21.15.157:0/2903414909 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f724403e9a0 0x7f7244040eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:37:28.233 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:28.229+0000 7f725f53b700 1 -- 172.21.15.157:0/2903414909 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7258064e20 con 0x7f7258102bf0 2024-05-09T00:37:28.233 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:28.230+0000 7f725cad6700 1 --2- 172.21.15.157:0/2903414909 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f724403e9a0 0x7f7244040eb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f725819aa80 tx=0x7f7248009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:37:28.237 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:28.235+0000 7f724e7fc700 1 -- 172.21.15.157:0/2903414909 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7254013070 con 0x7f7258102bf0 2024-05-09T00:37:28.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:27 smithi005 ceph-mon[30365]: Reconfiguring mon.b (monmap changed)... 2024-05-09T00:37:28.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:27 smithi005 ceph-mon[30365]: Reconfiguring daemon mon.b on smithi131 2024-05-09T00:37:28.461 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:28.460+0000 7f725f53b700 1 -- 172.21.15.157:0/2903414909 --> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi005=x;smithi131=y;smithi157=z", "target": ["mon-mgr", ""]}) -- 0x7f7258063c00 con 0x7f724403e9a0 2024-05-09T00:37:28.468 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:28.467+0000 7f724e7fc700 1 -- 172.21.15.157:0/2903414909 <== mgr.14152 v2:172.21.15.5:6800/91856593 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f7258063c00 con 0x7f724403e9a0 2024-05-09T00:37:28.468 INFO:teuthology.orchestra.run.smithi157.stdout:Scheduled mgr update... 2024-05-09T00:37:28.471 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:28.469+0000 7f725f53b700 1 -- 172.21.15.157:0/2903414909 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f724403e9a0 msgr2=0x7f7244040eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:37:28.471 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:28.469+0000 7f725f53b700 1 --2- 172.21.15.157:0/2903414909 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f724403e9a0 0x7f7244040eb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f725819aa80 tx=0x7f7248009210 comp rx=0 tx=0).stop 2024-05-09T00:37:28.471 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:28.469+0000 7f725f53b700 1 -- 172.21.15.157:0/2903414909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7258102bf0 msgr2=0x7f72581999d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:37:28.471 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:28.469+0000 7f725f53b700 1 --2- 172.21.15.157:0/2903414909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7258102bf0 0x7f72581999d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f7254005850 tx=0x7f7254004ed0 comp rx=0 tx=0).stop 2024-05-09T00:37:28.471 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:28.469+0000 7f725f53b700 1 -- 172.21.15.157:0/2903414909 shutdown_connections 2024-05-09T00:37:28.471 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:28.469+0000 7f725f53b700 1 --2- 172.21.15.157:0/2903414909 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f724403e9a0 0x7f7244040eb0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:37:28.471 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:28.469+0000 7f725f53b700 1 --2- 172.21.15.157:0/2903414909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f725819f3e0 0x7f72581a1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:37:28.471 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:28.469+0000 7f725f53b700 1 --2- 172.21.15.157:0/2903414909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7258102bf0 0x7f72581999d0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:37:28.472 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:28.469+0000 7f725f53b700 1 --2- 172.21.15.157:0/2903414909 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7258199f10 0x7f725819eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:37:28.472 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:28.469+0000 7f725f53b700 1 -- 172.21.15.157:0/2903414909 >> 172.21.15.157:0/2903414909 conn(0x7f72580fe250 msgr2=0x7f725806c360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:37:28.472 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:28.470+0000 7f725f53b700 1 -- 172.21.15.157:0/2903414909 shutdown_connections 2024-05-09T00:37:28.472 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:37:28.470+0000 7f725f53b700 1 -- 172.21.15.157:0/2903414909 wait complete. 2024-05-09T00:37:29.139 DEBUG:teuthology.orchestra.run.smithi131:mgr.y> sudo journalctl -f -n 0 -u ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@mgr.y.service 2024-05-09T00:37:29.142 DEBUG:teuthology.orchestra.run.smithi157:mgr.z> sudo journalctl -f -n 0 -u ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@mgr.z.service 2024-05-09T00:37:29.145 INFO:tasks.cephadm:Deploying OSDs... 2024-05-09T00:37:29.145 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-09T00:37:29.145 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/scratch_devs of=/dev/stdout 2024-05-09T00:37:29.166 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-09T00:37:29.166 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_1 2024-05-09T00:37:29.225 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-09T00:37:29.225 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T00:37:29.226 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-05-09T00:37:29.226 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T00:37:29.226 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T00:37:29.226 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-05-09 00:36:34.798491545 +0000 2024-05-09T00:37:29.226 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-05-09 00:31:13.024061071 +0000 2024-05-09T00:37:29.226 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-05-09 00:31:13.024061071 +0000 2024-05-09T00:37:29.226 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2024-05-09 00:31:13.024061071 +0000 2024-05-09T00:37:29.226 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-09T00:37:29.294 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2024-05-09T00:37:29.294 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2024-05-09T00:37:29.295 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.000127154 s, 4.0 MB/s 2024-05-09T00:37:29.296 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-09T00:37:29.355 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_2 2024-05-09T00:37:29.414 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-09T00:37:29.414 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T00:37:29.414 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-05-09T00:37:29.415 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T00:37:29.415 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T00:37:29.415 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-05-09 00:36:34.833490830 +0000 2024-05-09T00:37:29.415 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-05-09 00:31:13.360054233 +0000 2024-05-09T00:37:29.415 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-05-09 00:31:13.360054233 +0000 2024-05-09T00:37:29.415 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2024-05-09 00:31:13.360054233 +0000 2024-05-09T00:37:29.415 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-09T00:37:29.416 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:29 smithi157 ceph-mon[34294]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:29.416 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:29 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:29.417 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:29 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:29.417 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:29 smithi157 ceph-mon[34294]: Reconfiguring mon.c (monmap changed)... 2024-05-09T00:37:29.417 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:29 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T00:37:29.417 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:29 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T00:37:29.417 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:29 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:37:29.417 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:29 smithi157 ceph-mon[34294]: Reconfiguring daemon mon.c on smithi157 2024-05-09T00:37:29.417 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:29 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:29.479 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:29 smithi005 ceph-mon[30365]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:29.480 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:29 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:29.480 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:29 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:29.480 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:29 smithi005 ceph-mon[30365]: Reconfiguring mon.c (monmap changed)... 2024-05-09T00:37:29.480 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:29 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T00:37:29.480 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:29 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T00:37:29.480 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:29 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:37:29.480 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:29 smithi005 ceph-mon[30365]: Reconfiguring daemon mon.c on smithi157 2024-05-09T00:37:29.480 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:29 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:29.482 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2024-05-09T00:37:29.482 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2024-05-09T00:37:29.483 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.000126284 s, 4.1 MB/s 2024-05-09T00:37:29.484 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-09T00:37:29.544 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_3 2024-05-09T00:37:29.602 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-09T00:37:29.602 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T00:37:29.602 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-05-09T00:37:29.603 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T00:37:29.603 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T00:37:29.603 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-05-09 00:36:34.867490136 +0000 2024-05-09T00:37:29.603 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-05-09 00:31:13.698047356 +0000 2024-05-09T00:37:29.603 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-05-09 00:31:13.698047356 +0000 2024-05-09T00:37:29.603 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2024-05-09 00:31:13.698047356 +0000 2024-05-09T00:37:29.603 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-09T00:37:29.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:29 smithi131 ceph-mon[34547]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:29.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:29 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:29.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:29 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:29.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:29 smithi131 ceph-mon[34547]: Reconfiguring mon.c (monmap changed)... 2024-05-09T00:37:29.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:29 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T00:37:29.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:29 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T00:37:29.607 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:29 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:37:29.607 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:29 smithi131 ceph-mon[34547]: Reconfiguring daemon mon.c on smithi157 2024-05-09T00:37:29.607 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:29 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:29.670 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2024-05-09T00:37:29.671 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2024-05-09T00:37:29.671 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.000126866 s, 4.0 MB/s 2024-05-09T00:37:29.673 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-09T00:37:29.733 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_4 2024-05-09T00:37:29.792 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-09T00:37:29.792 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T00:37:29.792 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-05-09T00:37:29.792 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T00:37:29.792 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T00:37:29.792 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-05-09 00:36:34.900489463 +0000 2024-05-09T00:37:29.792 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-05-09 00:31:14.044040315 +0000 2024-05-09T00:37:29.792 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-05-09 00:31:14.044040315 +0000 2024-05-09T00:37:29.792 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2024-05-09 00:31:14.044040315 +0000 2024-05-09T00:37:29.793 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-09T00:37:29.860 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2024-05-09T00:37:29.860 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2024-05-09T00:37:29.860 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.000128151 s, 4.0 MB/s 2024-05-09T00:37:29.861 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-09T00:37:29.921 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-09T00:37:29.921 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/scratch_devs of=/dev/stdout 2024-05-09T00:37:29.940 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-09T00:37:29.940 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_1 2024-05-09T00:37:29.999 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-09T00:37:29.999 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T00:37:29.999 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-05-09T00:37:30.000 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T00:37:30.000 INFO:teuthology.orchestra.run.smithi131.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T00:37:30.000 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2024-05-09 00:37:15.717762142 +0000 2024-05-09T00:37:30.000 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2024-05-09 00:31:17.384923501 +0000 2024-05-09T00:37:30.000 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2024-05-09 00:31:17.384923501 +0000 2024-05-09T00:37:30.000 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: 2024-05-09 00:31:17.384923501 +0000 2024-05-09T00:37:30.000 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-09T00:37:30.065 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2024-05-09T00:37:30.065 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2024-05-09T00:37:30.065 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.000356337 s, 1.4 MB/s 2024-05-09T00:37:30.066 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-09T00:37:30.123 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_2 2024-05-09T00:37:30.180 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-09T00:37:30.180 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T00:37:30.180 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-05-09T00:37:30.180 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T00:37:30.180 INFO:teuthology.orchestra.run.smithi131.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T00:37:30.180 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2024-05-09 00:37:15.717762142 +0000 2024-05-09T00:37:30.180 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2024-05-09 00:31:17.778914935 +0000 2024-05-09T00:37:30.180 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2024-05-09 00:31:17.778914935 +0000 2024-05-09T00:37:30.180 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: 2024-05-09 00:31:17.778914935 +0000 2024-05-09T00:37:30.181 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-09T00:37:30.244 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2024-05-09T00:37:30.245 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2024-05-09T00:37:30.245 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.000214552 s, 2.4 MB/s 2024-05-09T00:37:30.246 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-09T00:37:30.303 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_3 2024-05-09T00:37:30.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:30 smithi131 ceph-mon[34547]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi005=x;smithi131=y;smithi157=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:37:30.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:30 smithi131 ceph-mon[34547]: Saving service mgr spec with placement smithi005=x;smithi131=y;smithi157=z;count:3 2024-05-09T00:37:30.358 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-09T00:37:30.358 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T00:37:30.359 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-05-09T00:37:30.359 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T00:37:30.359 INFO:teuthology.orchestra.run.smithi131.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T00:37:30.359 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2024-05-09 00:37:15.717762142 +0000 2024-05-09T00:37:30.359 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2024-05-09 00:31:18.169906435 +0000 2024-05-09T00:37:30.359 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2024-05-09 00:31:18.169906435 +0000 2024-05-09T00:37:30.359 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: 2024-05-09 00:31:18.169906435 +0000 2024-05-09T00:37:30.359 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-09T00:37:30.422 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2024-05-09T00:37:30.422 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2024-05-09T00:37:30.422 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.000222322 s, 2.3 MB/s 2024-05-09T00:37:30.423 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-09T00:37:30.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:30 smithi157 ceph-mon[34294]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi005=x;smithi131=y;smithi157=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:37:30.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:30 smithi157 ceph-mon[34294]: Saving service mgr spec with placement smithi005=x;smithi131=y;smithi157=z;count:3 2024-05-09T00:37:30.480 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_4 2024-05-09T00:37:30.535 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-09T00:37:30.535 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T00:37:30.535 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-05-09T00:37:30.535 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T00:37:30.536 INFO:teuthology.orchestra.run.smithi131.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T00:37:30.536 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2024-05-09 00:37:15.718762121 +0000 2024-05-09T00:37:30.536 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2024-05-09 00:31:18.509899043 +0000 2024-05-09T00:37:30.536 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2024-05-09 00:31:18.509899043 +0000 2024-05-09T00:37:30.536 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: 2024-05-09 00:31:18.509899043 +0000 2024-05-09T00:37:30.536 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-09T00:37:30.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:30 smithi005 ceph-mon[30365]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi005=x;smithi131=y;smithi157=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:37:30.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:30 smithi005 ceph-mon[30365]: Saving service mgr spec with placement smithi005=x;smithi131=y;smithi157=z;count:3 2024-05-09T00:37:30.598 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2024-05-09T00:37:30.598 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2024-05-09T00:37:30.598 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.000202506 s, 2.5 MB/s 2024-05-09T00:37:30.599 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-09T00:37:30.655 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-05-09T00:37:30.655 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/scratch_devs of=/dev/stdout 2024-05-09T00:37:30.673 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-09T00:37:30.673 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_1 2024-05-09T00:37:30.729 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-09T00:37:30.730 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T00:37:30.730 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-05-09T00:37:30.730 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T00:37:30.730 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T00:37:30.730 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-05-09 00:37:06.688969666 +0000 2024-05-09T00:37:30.730 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-05-09 00:31:21.638518504 +0000 2024-05-09T00:37:30.730 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-05-09 00:31:21.638518504 +0000 2024-05-09T00:37:30.730 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2024-05-09 00:31:21.638518504 +0000 2024-05-09T00:37:30.730 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-09T00:37:30.795 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-05-09T00:37:30.795 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-05-09T00:37:30.795 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.000121318 s, 4.2 MB/s 2024-05-09T00:37:30.796 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-09T00:37:30.853 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_2 2024-05-09T00:37:30.908 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-09T00:37:30.909 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T00:37:30.909 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-05-09T00:37:30.909 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T00:37:30.909 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T00:37:30.909 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-05-09 00:37:06.688969666 +0000 2024-05-09T00:37:30.909 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-05-09 00:31:21.981512229 +0000 2024-05-09T00:37:30.909 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-05-09 00:31:21.981512229 +0000 2024-05-09T00:37:30.909 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2024-05-09 00:31:21.981512229 +0000 2024-05-09T00:37:30.909 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-09T00:37:30.971 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-05-09T00:37:30.972 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-05-09T00:37:30.972 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.000114395 s, 4.5 MB/s 2024-05-09T00:37:30.973 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-09T00:37:31.030 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_3 2024-05-09T00:37:31.085 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-09T00:37:31.086 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T00:37:31.086 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-05-09T00:37:31.086 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T00:37:31.086 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T00:37:31.086 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-05-09 00:37:06.689969648 +0000 2024-05-09T00:37:31.086 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-05-09 00:31:22.322505990 +0000 2024-05-09T00:37:31.086 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-05-09 00:31:22.322505990 +0000 2024-05-09T00:37:31.086 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2024-05-09 00:31:22.322505990 +0000 2024-05-09T00:37:31.086 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-09T00:37:31.150 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-05-09T00:37:31.150 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-05-09T00:37:31.150 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 8.9678e-05 s, 5.7 MB/s 2024-05-09T00:37:31.152 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-09T00:37:31.211 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_4 2024-05-09T00:37:31.269 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-09T00:37:31.269 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T00:37:31.269 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-05-09T00:37:31.269 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T00:37:31.269 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T00:37:31.269 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-05-09 00:37:06.689969648 +0000 2024-05-09T00:37:31.269 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-05-09 00:31:22.644500099 +0000 2024-05-09T00:37:31.269 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-05-09 00:31:22.644500099 +0000 2024-05-09T00:37:31.269 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2024-05-09 00:31:22.644500099 +0000 2024-05-09T00:37:31.270 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-09T00:37:31.333 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-05-09T00:37:31.333 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-05-09T00:37:31.333 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 9.4094e-05 s, 5.4 MB/s 2024-05-09T00:37:31.334 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-09T00:37:31.391 INFO:tasks.cephadm:Deploying osd.0 on smithi005 with /dev/vg_nvme/lv_4... 2024-05-09T00:37:31.391 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-09T00:37:31.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:31 smithi157 ceph-mon[34294]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:31.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:31 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:31.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:31 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:31.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:31 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:37:31.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:31 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:37:31.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:31 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:37:31.438 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:31 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:31.438 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:31 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T00:37:31.438 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:31 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-09T00:37:31.438 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:31 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T00:37:31.438 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:31 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:37:31.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:31 smithi005 ceph-mon[30365]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:31.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:31 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:31.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:31 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:31.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:31 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:37:31.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:31 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:37:31.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:31 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:37:31.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:31 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:31.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:31 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T00:37:31.543 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:31 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-09T00:37:31.543 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:31 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T00:37:31.543 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:31 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:37:31.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:31 smithi131 ceph-mon[34547]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:31.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:31 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:31.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:31 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:31.607 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:31 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:37:31.607 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:31 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:37:31.607 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:31 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:37:31.607 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:31 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:31.607 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:31 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T00:37:31.607 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:31 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-09T00:37:31.607 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:31 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T00:37:31.607 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:31 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:37:31.640 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:37:32.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:32 smithi005 ceph-mon[30365]: Deploying daemon mgr.z on smithi157 2024-05-09T00:37:32.509 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:32 smithi157 ceph-mon[34294]: Deploying daemon mgr.z on smithi157 2024-05-09T00:37:32.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:32 smithi131 ceph-mon[34547]: Deploying daemon mgr.z on smithi157 2024-05-09T00:37:32.937 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 09 00:37:32 smithi157 systemd[1]: Starting Ceph mgr.z for d85814c6-0d9b-11ef-bc97-c7b262605968... 2024-05-09T00:37:33.221 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 09 00:37:32 smithi157 podman[36145]: 2024-05-09 00:37:32.941938727 +0000 UTC m=+0.114369245 container create f48ec4b59ddabf4fae5feda5fd5dd9c51a43a3ade703d3cf1a476dc4f8aa867a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-z, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-05-09T00:37:33.221 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 09 00:37:32 smithi157 podman[36145]: 2024-05-09 00:37:32.844739992 +0000 UTC m=+0.017170538 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T00:37:33.221 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 09 00:37:33 smithi157 podman[36145]: 2024-05-09 00:37:33.136150389 +0000 UTC m=+0.308580908 container init f48ec4b59ddabf4fae5feda5fd5dd9c51a43a3ade703d3cf1a476dc4f8aa867a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-z, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T00:37:33.221 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 09 00:37:33 smithi157 podman[36145]: 2024-05-09 00:37:33.13936668 +0000 UTC m=+0.311797198 container start f48ec4b59ddabf4fae5feda5fd5dd9c51a43a3ade703d3cf1a476dc4f8aa867a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T00:37:33.221 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 09 00:37:33 smithi157 ceph-mgr[36165]: -- 172.21.15.157:0/545929401 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c81fdd45a0 con 0x55c824b5e480 2024-05-09T00:37:33.287 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:33 smithi005 ceph-mon[30365]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:33.503 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:33 smithi157 ceph-mon[34294]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:33.503 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 09 00:37:33 smithi157 bash[36145]: f48ec4b59ddabf4fae5feda5fd5dd9c51a43a3ade703d3cf1a476dc4f8aa867a 2024-05-09T00:37:33.504 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 09 00:37:33 smithi157 systemd[1]: Started Ceph mgr.z for d85814c6-0d9b-11ef-bc97-c7b262605968. 2024-05-09T00:37:33.504 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 09 00:37:33 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-z[36160]: 2024-05-09T00:37:33.315+0000 7f6d54fce200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T00:37:33.504 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 09 00:37:33 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-z[36160]: 2024-05-09T00:37:33.501+0000 7f6d54fce200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T00:37:33.507 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:33 smithi131 ceph-mon[34547]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:33.552 INFO:teuthology.orchestra.run.smithi005.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-09T00:37:33.552 INFO:teuthology.orchestra.run.smithi005.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-09T00:37:33.552 INFO:teuthology.orchestra.run.smithi005.stderr: stderr: 10+0 records in 2024-05-09T00:37:33.552 INFO:teuthology.orchestra.run.smithi005.stderr:10+0 records out 2024-05-09T00:37:33.552 INFO:teuthology.orchestra.run.smithi005.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00840609 s, 1.2 GB/s 2024-05-09T00:37:33.553 INFO:teuthology.orchestra.run.smithi005.stderr:--> Zapping successful for: 2024-05-09T00:37:33.913 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 09 00:37:33 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-z[36160]: 2024-05-09T00:37:33.604+0000 7f6d54fce200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T00:37:33.914 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 09 00:37:33 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-z[36160]: 2024-05-09T00:37:33.912+0000 7f6d54fce200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T00:37:34.187 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi005:vg_nvme/lv_4 2024-05-09T00:37:34.187 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 09 00:37:33 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-z[36160]: 2024-05-09T00:37:33.997+0000 7f6d54fce200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T00:37:34.439 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:37:34.521 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:34 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:34.521 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:34 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:34.521 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:34 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:34.521 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:34 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T00:37:34.521 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:34 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-09T00:37:34.521 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:34 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T00:37:34.521 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:34 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:37:34.521 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:34 smithi131 ceph-mon[34547]: Deploying daemon mgr.y on smithi131 2024-05-09T00:37:34.522 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:34 smithi131 ceph-mon[34547]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:34.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:34 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:34.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:34 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:34.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:34 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:34.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:34 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T00:37:34.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:34 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-09T00:37:34.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:34 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T00:37:34.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:34 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:37:34.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:34 smithi005 ceph-mon[30365]: Deploying daemon mgr.y on smithi131 2024-05-09T00:37:34.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:34 smithi005 ceph-mon[30365]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:34.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:34 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:34.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:34 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:34.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:34 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:34.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:34 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T00:37:34.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:34 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-09T00:37:34.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:34 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T00:37:34.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:34 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:37:34.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:34 smithi157 ceph-mon[34294]: Deploying daemon mgr.y on smithi131 2024-05-09T00:37:34.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:34 smithi157 ceph-mon[34294]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:35.550 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 09 00:37:35 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-z[36160]: 2024-05-09T00:37:35.206+0000 7f6d54fce200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T00:37:35.550 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 09 00:37:35 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-z[36160]: 2024-05-09T00:37:35.376+0000 7f6d54fce200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T00:37:35.550 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 09 00:37:35 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-z[36160]: 2024-05-09T00:37:35.548+0000 7f6d54fce200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T00:37:35.772 INFO:journalctl@ceph.mgr.y.smithi131.stdout:May 09 00:37:35 smithi131 ceph-mgr[35940]: -- 172.21.15.131:0/1293508435 <== mon.2 v2:172.21.15.131:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e8d6bd45a0 con 0x55e8db95e900 2024-05-09T00:37:36.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:36.047+0000 7f687eed3700 1 -- 172.21.15.5:0/885268104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6878104420 msgr2=0x7f6878104830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:37:36.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:36.047+0000 7f687eed3700 1 --2- 172.21.15.5:0/885268104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6878104420 0x7f6878104830 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6860009990 tx=0x7f6860009ca0 comp rx=0 tx=0).stop 2024-05-09T00:37:36.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:36.047+0000 7f687eed3700 1 -- 172.21.15.5:0/885268104 shutdown_connections 2024-05-09T00:37:36.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:36.047+0000 7f687eed3700 1 --2- 172.21.15.5:0/885268104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6878104dc0 0x7f687810b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:37:36.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:36.047+0000 7f687eed3700 1 --2- 172.21.15.5:0/885268104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f68781032f0 0x7f6878103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:37:36.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:36.047+0000 7f687eed3700 1 --2- 172.21.15.5:0/885268104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6878104420 0x7f6878104830 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:37:36.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:36.047+0000 7f687eed3700 1 -- 172.21.15.5:0/885268104 >> 172.21.15.5:0/885268104 conn(0x7f68780fe910 msgr2=0x7f6878100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:37:36.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:36.048+0000 7f687eed3700 1 -- 172.21.15.5:0/885268104 shutdown_connections 2024-05-09T00:37:36.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:36.048+0000 7f687eed3700 1 -- 172.21.15.5:0/885268104 wait complete. 2024-05-09T00:37:36.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:36.048+0000 7f687eed3700 1 Processor -- start 2024-05-09T00:37:36.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:36.048+0000 7f687eed3700 1 -- start start 2024-05-09T00:37:36.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:36.048+0000 7f687eed3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68781032f0 0x7f68781070b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:37:36.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:36.048+0000 7f687eed3700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6878104420 0x7f6878105760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:37:36.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:36.048+0000 7f687eed3700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6878104dc0 0x7f6878105ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:37:36.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:36.048+0000 7f687eed3700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f687810edb0 con 0x7f68781032f0 2024-05-09T00:37:36.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:36.048+0000 7f687eed3700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f687810ec30 con 0x7f6878104420 2024-05-09T00:37:36.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:36.048+0000 7f687eed3700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f687810ef30 con 0x7f6878104dc0 2024-05-09T00:37:36.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:36.049+0000 7f687cc6f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68781032f0 0x7f68781070b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:37:36.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:36.049+0000 7f687cc6f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68781032f0 0x7f68781070b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51264/0 (socket says 172.21.15.5:51264) 2024-05-09T00:37:36.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:36.049+0000 7f687cc6f700 1 -- 172.21.15.5:0/1471220038 learned_addr learned my addr 172.21.15.5:0/1471220038 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:37:36.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:36.049+0000 7f6877fff700 1 --2- 172.21.15.5:0/1471220038 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6878104420 0x7f6878105760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:37:36.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:36.049+0000 7f687d470700 1 --2- 172.21.15.5:0/1471220038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6878104dc0 0x7f6878105ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:37:36.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:36.049+0000 7f687cc6f700 1 -- 172.21.15.5:0/1471220038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6878104dc0 msgr2=0x7f6878105ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:37:36.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:36.049+0000 7f687cc6f700 1 --2- 172.21.15.5:0/1471220038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6878104dc0 0x7f6878105ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:37:36.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:36.049+0000 7f687cc6f700 1 -- 172.21.15.5:0/1471220038 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6878104420 msgr2=0x7f6878105760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:37:36.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:36.049+0000 7f687cc6f700 1 --2- 172.21.15.5:0/1471220038 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6878104420 0x7f6878105760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:37:36.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:36.049+0000 7f687cc6f700 1 -- 172.21.15.5:0/1471220038 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6878106400 con 0x7f68781032f0 2024-05-09T00:37:36.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:36.049+0000 7f6877fff700 1 --2- 172.21.15.5:0/1471220038 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6878104420 0x7f6878105760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:37:36.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:36.049+0000 7f687cc6f700 1 --2- 172.21.15.5:0/1471220038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68781032f0 0x7f68781070b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6868007ee0 tx=0x7f686800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:37:36.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:36.049+0000 7f6875ffb700 1 -- 172.21.15.5:0/1471220038 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686800db20 con 0x7f68781032f0 2024-05-09T00:37:36.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:36.049+0000 7f687eed3700 1 -- 172.21.15.5:0/1471220038 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68781a4ae0 con 0x7f68781032f0 2024-05-09T00:37:36.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:36.049+0000 7f6875ffb700 1 -- 172.21.15.5:0/1471220038 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f686800dcc0 con 0x7f68781032f0 2024-05-09T00:37:36.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:36.049+0000 7f6875ffb700 1 -- 172.21.15.5:0/1471220038 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6868004740 con 0x7f68781032f0 2024-05-09T00:37:36.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:36.050+0000 7f687eed3700 1 -- 172.21.15.5:0/1471220038 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68781a4fc0 con 0x7f68781032f0 2024-05-09T00:37:36.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:36.051+0000 7f6875ffb700 1 -- 172.21.15.5:0/1471220038 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f686801a020 con 0x7f68781032f0 2024-05-09T00:37:36.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:36.051+0000 7f687eed3700 1 -- 172.21.15.5:0/1471220038 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68781a5190 con 0x7f68781032f0 2024-05-09T00:37:36.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:36.051+0000 7f6875ffb700 1 --2- 172.21.15.5:0/1471220038 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f686403e750 0x7f6864040c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:37:36.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:36.051+0000 7f6875ffb700 1 -- 172.21.15.5:0/1471220038 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f68680521e0 con 0x7f68781032f0 2024-05-09T00:37:36.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:36.051+0000 7f6877fff700 1 --2- 172.21.15.5:0/1471220038 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f686403e750 0x7f6864040c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:37:36.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:36.052+0000 7f6877fff700 1 --2- 172.21.15.5:0/1471220038 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f686403e750 0x7f6864040c60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f68600038a0 tx=0x7f686001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:37:36.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:36.057+0000 7f6875ffb700 1 -- 172.21.15.5:0/1471220038 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6868010070 con 0x7f68781032f0 2024-05-09T00:37:36.062 INFO:journalctl@ceph.mgr.y.smithi131.stdout:May 09 00:37:35 smithi131 bash[35921]: 2b75e33e95712081ad67b8d6c76cb71de98da9f4888b782d69b0cca141487036 2024-05-09T00:37:36.062 INFO:journalctl@ceph.mgr.y.smithi131.stdout:May 09 00:37:35 smithi131 systemd[1]: Started Ceph mgr.y for d85814c6-0d9b-11ef-bc97-c7b262605968. 2024-05-09T00:37:36.062 INFO:journalctl@ceph.mgr.y.smithi131.stdout:May 09 00:37:35 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-y[35936]: 2024-05-09T00:37:35.872+0000 7f0b259bf200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T00:37:36.062 INFO:journalctl@ceph.mgr.y.smithi131.stdout:May 09 00:37:36 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-y[35936]: 2024-05-09T00:37:36.060+0000 7f0b259bf200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T00:37:36.142 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 09 00:37:35 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-z[36160]: 2024-05-09T00:37:35.836+0000 7f6d54fce200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T00:37:36.142 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 09 00:37:35 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-z[36160]: 2024-05-09T00:37:35.930+0000 7f6d54fce200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T00:37:36.143 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 09 00:37:36 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-z[36160]: 2024-05-09T00:37:36.141+0000 7f6d54fce200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T00:37:36.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:36.280+0000 7f687eed3700 1 -- 172.21.15.5:0/1471220038 --> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f6878100960 con 0x7f686403e750 2024-05-09T00:37:36.356 INFO:journalctl@ceph.mgr.y.smithi131.stdout:May 09 00:37:36 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-y[35936]: 2024-05-09T00:37:36.169+0000 7f0b259bf200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T00:37:36.437 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 09 00:37:36 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-z[36160]: 2024-05-09T00:37:36.342+0000 7f6d54fce200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T00:37:36.842 INFO:journalctl@ceph.mgr.y.smithi131.stdout:May 09 00:37:36 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-y[35936]: 2024-05-09T00:37:36.495+0000 7f0b259bf200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T00:37:36.842 INFO:journalctl@ceph.mgr.y.smithi131.stdout:May 09 00:37:36 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-y[35936]: 2024-05-09T00:37:36.599+0000 7f0b259bf200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T00:37:37.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:36 smithi131 ceph-mon[34547]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:37.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:36 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:37.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:36 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:37.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:36 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:37.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:36 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:37.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:36 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:37:37.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:36 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:37:37.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:36 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:37:37.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:36 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:37:37.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:36 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:37.124 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:36 smithi005 ceph-mon[30365]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:37.124 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:36 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:37.124 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:36 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:37.125 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:36 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:37.125 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:36 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:37.125 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:36 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:37:37.125 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:36 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:37:37.125 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:36 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:37:37.125 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:36 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:37:37.125 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:36 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:37.187 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 09 00:37:36 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-z[36160]: 2024-05-09T00:37:36.993+0000 7f6d54fce200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T00:37:37.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:36 smithi157 ceph-mon[34294]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:37.188 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:36 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:37.188 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:36 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:37.188 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:36 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:37.188 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:36 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:37.188 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:36 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:37:37.188 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:36 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:37:37.189 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:36 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:37:37.189 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:36 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:37:37.189 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:36 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:37.660 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 09 00:37:37 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-z[36160]: 2024-05-09T00:37:37.220+0000 7f6d54fce200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-09T00:37:37.937 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 09 00:37:37 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-z[36160]: 2024-05-09T00:37:37.658+0000 7f6d54fce200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T00:37:37.937 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 09 00:37:37 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-z[36160]: 2024-05-09T00:37:37.755+0000 7f6d54fce200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T00:37:37.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:37 smithi157 ceph-mon[34294]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:37:37.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:37 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:37.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:37 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:37.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:37 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:37.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:37 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:37.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:37 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:37:37.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:37 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:37:37.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:37 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:37.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:37 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T00:37:37.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:37 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T00:37:37.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:37 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:37:38.106 INFO:journalctl@ceph.mgr.y.smithi131.stdout:May 09 00:37:37 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-y[35936]: 2024-05-09T00:37:37.833+0000 7f0b259bf200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T00:37:38.106 INFO:journalctl@ceph.mgr.y.smithi131.stdout:May 09 00:37:38 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-y[35936]: 2024-05-09T00:37:38.003+0000 7f0b259bf200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T00:37:38.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:37 smithi131 ceph-mon[34547]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:37:38.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:37 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:38.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:37 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:38.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:37 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:38.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:37 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:38.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:37 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:37:38.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:37 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:37:38.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:37 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:38.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:37 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T00:37:38.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:37 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T00:37:38.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:37 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:37:38.189 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:37 smithi005 ceph-mon[30365]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:37:38.189 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:37 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:38.189 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:37 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:38.189 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:37 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:38.189 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:37 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:38.189 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:37 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:37:38.189 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:37 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:37:38.189 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:37 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:38.189 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:37 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T00:37:38.190 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:37 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T00:37:38.190 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:37 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:37:38.437 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 09 00:37:38 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-z[36160]: 2024-05-09T00:37:38.126+0000 7f6d54fce200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T00:37:38.459 INFO:journalctl@ceph.mgr.y.smithi131.stdout:May 09 00:37:38 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-y[35936]: 2024-05-09T00:37:38.174+0000 7f0b259bf200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T00:37:38.754 INFO:journalctl@ceph.mgr.y.smithi131.stdout:May 09 00:37:38 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-y[35936]: 2024-05-09T00:37:38.458+0000 7f0b259bf200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T00:37:38.754 INFO:journalctl@ceph.mgr.y.smithi131.stdout:May 09 00:37:38 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-y[35936]: 2024-05-09T00:37:38.549+0000 7f0b259bf200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T00:37:38.755 INFO:journalctl@ceph.mgr.y.smithi131.stdout:May 09 00:37:38 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-y[35936]: 2024-05-09T00:37:38.751+0000 7f0b259bf200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T00:37:38.816 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 09 00:37:38 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-z[36160]: 2024-05-09T00:37:38.481+0000 7f6d54fce200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T00:37:38.816 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 09 00:37:38 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-z[36160]: 2024-05-09T00:37:38.573+0000 7f6d54fce200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T00:37:39.106 INFO:journalctl@ceph.mgr.y.smithi131.stdout:May 09 00:37:38 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-y[35936]: 2024-05-09T00:37:38.943+0000 7f0b259bf200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T00:37:39.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:38 smithi131 ceph-mon[34547]: Reconfiguring mgr.x (unknown last config time)... 2024-05-09T00:37:39.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:38 smithi131 ceph-mon[34547]: Reconfiguring daemon mgr.x on smithi005 2024-05-09T00:37:39.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:38 smithi131 ceph-mon[34547]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:39.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:38 smithi157 ceph-mon[34294]: Reconfiguring mgr.x (unknown last config time)... 2024-05-09T00:37:39.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:38 smithi157 ceph-mon[34294]: Reconfiguring daemon mgr.x on smithi005 2024-05-09T00:37:39.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:38 smithi157 ceph-mon[34294]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:39.187 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 09 00:37:38 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-z[36160]: 2024-05-09T00:37:38.814+0000 7f6d54fce200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T00:37:39.187 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 09 00:37:38 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-z[36160]: 2024-05-09T00:37:38.951+0000 7f6d54fce200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T00:37:39.187 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 09 00:37:39 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-z[36160]: 2024-05-09T00:37:39.046+0000 7f6d54fce200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T00:37:39.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:38 smithi005 ceph-mon[30365]: Reconfiguring mgr.x (unknown last config time)... 2024-05-09T00:37:39.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:38 smithi005 ceph-mon[30365]: Reconfiguring daemon mgr.x on smithi005 2024-05-09T00:37:39.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:38 smithi005 ceph-mon[30365]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:39.687 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 09 00:37:39 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-z[36160]: 2024-05-09T00:37:39.375+0000 7f6d54fce200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T00:37:39.687 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 09 00:37:39 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-z[36160]: 2024-05-09T00:37:39.467+0000 7f6d54fce200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T00:37:39.846 INFO:journalctl@ceph.mgr.y.smithi131.stdout:May 09 00:37:39 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-y[35936]: 2024-05-09T00:37:39.540+0000 7f0b259bf200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T00:37:39.846 INFO:journalctl@ceph.mgr.y.smithi131.stdout:May 09 00:37:39 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-y[35936]: 2024-05-09T00:37:39.758+0000 7f0b259bf200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-09T00:37:40.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:39 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/4125928238' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8d484bb-2477-4377-ae68-5bce571333bc"}]: dispatch 2024-05-09T00:37:40.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:39 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/4125928238' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f8d484bb-2477-4377-ae68-5bce571333bc"}]': finished 2024-05-09T00:37:40.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:39 smithi131 ceph-mon[34547]: osdmap e5: 1 total, 0 up, 1 in 2024-05-09T00:37:40.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:39 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T00:37:40.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:39 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/13909309' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-0/activate.monmap"}]: dispatch 2024-05-09T00:37:40.187 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 09 00:37:40 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-z[36160]: 2024-05-09T00:37:40.069+0000 7f6d54fce200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T00:37:40.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:39 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/4125928238' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8d484bb-2477-4377-ae68-5bce571333bc"}]: dispatch 2024-05-09T00:37:40.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:39 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/4125928238' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f8d484bb-2477-4377-ae68-5bce571333bc"}]': finished 2024-05-09T00:37:40.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:39 smithi157 ceph-mon[34294]: osdmap e5: 1 total, 0 up, 1 in 2024-05-09T00:37:40.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:39 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T00:37:40.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:39 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/13909309' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-0/activate.monmap"}]: dispatch 2024-05-09T00:37:40.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:39 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/4125928238' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8d484bb-2477-4377-ae68-5bce571333bc"}]: dispatch 2024-05-09T00:37:40.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:39 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/4125928238' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f8d484bb-2477-4377-ae68-5bce571333bc"}]': finished 2024-05-09T00:37:40.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:39 smithi005 ceph-mon[30365]: osdmap e5: 1 total, 0 up, 1 in 2024-05-09T00:37:40.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:39 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T00:37:40.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:39 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/13909309' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-0/activate.monmap"}]: dispatch 2024-05-09T00:37:40.606 INFO:journalctl@ceph.mgr.y.smithi131.stdout:May 09 00:37:40 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-y[35936]: 2024-05-09T00:37:40.177+0000 7f0b259bf200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T00:37:40.606 INFO:journalctl@ceph.mgr.y.smithi131.stdout:May 09 00:37:40 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-y[35936]: 2024-05-09T00:37:40.271+0000 7f0b259bf200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T00:37:40.687 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 09 00:37:40 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-z[36160]: 2024-05-09T00:37:40.288+0000 7f6d54fce200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T00:37:40.992 INFO:journalctl@ceph.mgr.y.smithi131.stdout:May 09 00:37:40 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-y[35936]: 2024-05-09T00:37:40.642+0000 7f0b259bf200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T00:37:40.993 INFO:journalctl@ceph.mgr.y.smithi131.stdout:May 09 00:37:40 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-y[35936]: 2024-05-09T00:37:40.991+0000 7f0b259bf200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T00:37:41.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:41.100+0000 7f6875ffb700 1 -- 172.21.15.5:0/1471220038 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mgrmap(e 14) ==== 99936+0+0 (secure 0 0 0) 0x7f686801a310 con 0x7f68781032f0 2024-05-09T00:37:41.326 INFO:journalctl@ceph.mgr.y.smithi131.stdout:May 09 00:37:41 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-y[35936]: 2024-05-09T00:37:41.082+0000 7f0b259bf200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T00:37:41.326 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:41 smithi131 ceph-mon[34547]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:41.326 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:41 smithi131 ceph-mon[34547]: Standby manager daemon z started 2024-05-09T00:37:41.326 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:41 smithi131 ceph-mon[34547]: from='mgr.? 172.21.15.157:0/2539917333' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-09T00:37:41.326 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:41 smithi131 ceph-mon[34547]: from='mgr.? 172.21.15.157:0/2539917333' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T00:37:41.326 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:41 smithi131 ceph-mon[34547]: from='mgr.? 172.21.15.157:0/2539917333' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-09T00:37:41.326 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:41 smithi131 ceph-mon[34547]: from='mgr.? 172.21.15.157:0/2539917333' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T00:37:41.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:41 smithi157 ceph-mon[34294]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:41.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:41 smithi157 ceph-mon[34294]: Standby manager daemon z started 2024-05-09T00:37:41.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:41 smithi157 ceph-mon[34294]: from='mgr.? 172.21.15.157:0/2539917333' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-09T00:37:41.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:41 smithi157 ceph-mon[34294]: from='mgr.? 172.21.15.157:0/2539917333' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T00:37:41.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:41 smithi157 ceph-mon[34294]: from='mgr.? 172.21.15.157:0/2539917333' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-09T00:37:41.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:41 smithi157 ceph-mon[34294]: from='mgr.? 172.21.15.157:0/2539917333' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T00:37:41.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:41 smithi005 ceph-mon[30365]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:41.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:41 smithi005 ceph-mon[30365]: Standby manager daemon z started 2024-05-09T00:37:41.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:41 smithi005 ceph-mon[30365]: from='mgr.? 172.21.15.157:0/2539917333' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-09T00:37:41.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:41 smithi005 ceph-mon[30365]: from='mgr.? 172.21.15.157:0/2539917333' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T00:37:41.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:41 smithi005 ceph-mon[30365]: from='mgr.? 172.21.15.157:0/2539917333' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-09T00:37:41.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:41 smithi005 ceph-mon[30365]: from='mgr.? 172.21.15.157:0/2539917333' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T00:37:41.606 INFO:journalctl@ceph.mgr.y.smithi131.stdout:May 09 00:37:41 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-y[35936]: 2024-05-09T00:37:41.324+0000 7f0b259bf200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T00:37:41.606 INFO:journalctl@ceph.mgr.y.smithi131.stdout:May 09 00:37:41 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-y[35936]: 2024-05-09T00:37:41.457+0000 7f0b259bf200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T00:37:41.606 INFO:journalctl@ceph.mgr.y.smithi131.stdout:May 09 00:37:41 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-y[35936]: 2024-05-09T00:37:41.551+0000 7f0b259bf200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T00:37:42.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:42 smithi131 ceph-mon[34547]: mgrmap e14: x(active, since 79s), standbys: z 2024-05-09T00:37:42.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:42 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-09T00:37:42.356 INFO:journalctl@ceph.mgr.y.smithi131.stdout:May 09 00:37:41 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-y[35936]: 2024-05-09T00:37:41.877+0000 7f0b259bf200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T00:37:42.356 INFO:journalctl@ceph.mgr.y.smithi131.stdout:May 09 00:37:41 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-y[35936]: 2024-05-09T00:37:41.969+0000 7f0b259bf200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T00:37:42.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:42 smithi157 ceph-mon[34294]: mgrmap e14: x(active, since 79s), standbys: z 2024-05-09T00:37:42.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:42 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-09T00:37:42.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:42 smithi005 ceph-mon[30365]: mgrmap e14: x(active, since 79s), standbys: z 2024-05-09T00:37:42.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:42 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-09T00:37:42.856 INFO:journalctl@ceph.mgr.y.smithi131.stdout:May 09 00:37:42 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-y[35936]: 2024-05-09T00:37:42.565+0000 7f0b259bf200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T00:37:42.856 INFO:journalctl@ceph.mgr.y.smithi131.stdout:May 09 00:37:42 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-y[35936]: 2024-05-09T00:37:42.777+0000 7f0b259bf200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T00:37:43.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:43.119+0000 7f6875ffb700 1 -- 172.21.15.5:0/1471220038 <== mon.0 v2:172.21.15.5:3300/0 8 ==== mgrmap(e 15) ==== 149595+0+0 (secure 0 0 0) 0x7f6868017c70 con 0x7f68781032f0 2024-05-09T00:37:43.385 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:43 smithi005 ceph-mon[30365]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:43.386 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:43 smithi005 ceph-mon[30365]: Standby manager daemon y started 2024-05-09T00:37:43.386 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:43 smithi005 ceph-mon[30365]: from='mgr.? 172.21.15.131:0/2125408155' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-09T00:37:43.386 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:43 smithi005 ceph-mon[30365]: from='mgr.? 172.21.15.131:0/2125408155' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T00:37:43.386 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:43 smithi005 ceph-mon[30365]: from='mgr.? 172.21.15.131:0/2125408155' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-09T00:37:43.386 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:43 smithi005 ceph-mon[30365]: from='mgr.? 172.21.15.131:0/2125408155' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T00:37:43.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:43 smithi157 ceph-mon[34294]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:43.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:43 smithi157 ceph-mon[34294]: Standby manager daemon y started 2024-05-09T00:37:43.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:43 smithi157 ceph-mon[34294]: from='mgr.? 172.21.15.131:0/2125408155' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-09T00:37:43.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:43 smithi157 ceph-mon[34294]: from='mgr.? 172.21.15.131:0/2125408155' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T00:37:43.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:43 smithi157 ceph-mon[34294]: from='mgr.? 172.21.15.131:0/2125408155' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-09T00:37:43.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:43 smithi157 ceph-mon[34294]: from='mgr.? 172.21.15.131:0/2125408155' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T00:37:43.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:43 smithi131 ceph-mon[34547]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:43.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:43 smithi131 ceph-mon[34547]: Standby manager daemon y started 2024-05-09T00:37:43.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:43 smithi131 ceph-mon[34547]: from='mgr.? 172.21.15.131:0/2125408155' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-09T00:37:43.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:43 smithi131 ceph-mon[34547]: from='mgr.? 172.21.15.131:0/2125408155' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T00:37:43.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:43 smithi131 ceph-mon[34547]: from='mgr.? 172.21.15.131:0/2125408155' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-09T00:37:43.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:43 smithi131 ceph-mon[34547]: from='mgr.? 172.21.15.131:0/2125408155' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T00:37:44.400 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:44 smithi157 ceph-mon[34294]: mgrmap e15: x(active, since 81s), standbys: z, y 2024-05-09T00:37:44.401 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:44 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-09T00:37:44.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:44 smithi005 ceph-mon[30365]: mgrmap e15: x(active, since 81s), standbys: z, y 2024-05-09T00:37:44.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:44 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-09T00:37:44.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:44 smithi131 ceph-mon[34547]: mgrmap e15: x(active, since 81s), standbys: z, y 2024-05-09T00:37:44.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:44 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-09T00:37:45.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:45 smithi005 ceph-mon[30365]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:45.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:45 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:45.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:45 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:45.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:45 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:37:45.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:45 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:37:45.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:45 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:37:45.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:45 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:45.392 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:45 smithi131 ceph-mon[34547]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:45.392 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:45 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:45.392 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:45 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:45.392 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:45 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:37:45.392 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:45 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:37:45.392 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:45 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:37:45.393 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:45 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:45.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:45 smithi157 ceph-mon[34294]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:45.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:45 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:45.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:45 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:45.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:45 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:37:45.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:45 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:37:45.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:45 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:37:45.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:45 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:47.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:47 smithi157 ceph-mon[34294]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:47.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:47 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-09T00:37:47.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:47 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:37:47.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:47 smithi005 ceph-mon[30365]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:47.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:47 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-09T00:37:47.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:47 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:37:47.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:47 smithi131 ceph-mon[34547]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:47.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:47 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-09T00:37:47.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:47 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:37:48.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:48 smithi005 ceph-mon[30365]: Deploying daemon osd.0 on smithi005 2024-05-09T00:37:48.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:48 smithi157 ceph-mon[34294]: Deploying daemon osd.0 on smithi005 2024-05-09T00:37:48.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:48 smithi131 ceph-mon[34547]: Deploying daemon osd.0 on smithi005 2024-05-09T00:37:49.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:49 smithi157 ceph-mon[34294]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:49.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:49 smithi005 ceph-mon[30365]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:49.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:49 smithi131 ceph-mon[34547]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:51.402 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:51 smithi005 ceph-mon[30365]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:51.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:51 smithi157 ceph-mon[34294]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:51.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:51 smithi131 ceph-mon[34547]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:53.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:53 smithi005 ceph-mon[30365]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:53.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:53 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:53.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:53 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:53.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:53 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:37:53.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:53 smithi157 ceph-mon[34294]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:53.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:53 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:53.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:53 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:53.438 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:53 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:37:53.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:53 smithi131 ceph-mon[34547]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:53.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:53 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:53.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:53 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:53.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:53 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:37:55.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:55 smithi005 ceph-mon[30365]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:55.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:55 smithi157 ceph-mon[34294]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:55.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:55 smithi131 ceph-mon[34547]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:55.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:55.985+0000 7f6875ffb700 1 -- 172.21.15.5:0/1471220038 <== mgr.14152 v2:172.21.15.5:6800/91856593 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6878100960 con 0x7f686403e750 2024-05-09T00:37:55.989 INFO:teuthology.orchestra.run.smithi005.stdout:Created osd(s) 0 on host 'smithi005' 2024-05-09T00:37:55.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:55.988+0000 7f687eed3700 1 -- 172.21.15.5:0/1471220038 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f686403e750 msgr2=0x7f6864040c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:37:55.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:55.988+0000 7f687eed3700 1 --2- 172.21.15.5:0/1471220038 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f686403e750 0x7f6864040c60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f68600038a0 tx=0x7f686001a040 comp rx=0 tx=0).stop 2024-05-09T00:37:55.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:55.988+0000 7f687eed3700 1 -- 172.21.15.5:0/1471220038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68781032f0 msgr2=0x7f68781070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:37:55.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:55.988+0000 7f687eed3700 1 --2- 172.21.15.5:0/1471220038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68781032f0 0x7f68781070b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6868007ee0 tx=0x7f686800d3b0 comp rx=0 tx=0).stop 2024-05-09T00:37:55.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:55.988+0000 7f687eed3700 1 -- 172.21.15.5:0/1471220038 shutdown_connections 2024-05-09T00:37:55.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:55.988+0000 7f687eed3700 1 --2- 172.21.15.5:0/1471220038 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f686403e750 0x7f6864040c60 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:37:55.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:55.988+0000 7f687eed3700 1 --2- 172.21.15.5:0/1471220038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6878104dc0 0x7f6878105ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:37:55.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:55.988+0000 7f687eed3700 1 --2- 172.21.15.5:0/1471220038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68781032f0 0x7f68781070b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:37:55.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:55.988+0000 7f687eed3700 1 --2- 172.21.15.5:0/1471220038 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6878104420 0x7f6878105760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:37:55.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:55.988+0000 7f687eed3700 1 -- 172.21.15.5:0/1471220038 >> 172.21.15.5:0/1471220038 conn(0x7f68780fe910 msgr2=0x7f6878100250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:37:55.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:55.988+0000 7f687eed3700 1 -- 172.21.15.5:0/1471220038 shutdown_connections 2024-05-09T00:37:55.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:37:55.988+0000 7f687eed3700 1 -- 172.21.15.5:0/1471220038 wait complete. 2024-05-09T00:37:56.609 DEBUG:teuthology.orchestra.run.smithi005:osd.0> sudo journalctl -f -n 0 -u ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@osd.0.service 2024-05-09T00:37:56.611 INFO:tasks.cephadm:Deploying osd.1 on smithi131 with /dev/vg_nvme/lv_4... 2024-05-09T00:37:56.612 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-09T00:37:56.856 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T00:37:57.215 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:56 smithi005 ceph-mon[30365]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:57.215 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:56 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:57.215 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:56 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:57.215 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:56 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:57.215 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:56 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:57.215 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:56 smithi005 ceph-mon[30365]: from='osd.0 [v2:172.21.15.5:6802/2963385132,v1:172.21.15.5:6803/2963385132]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-09T00:37:57.311 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:56 smithi131 ceph-mon[34547]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:57.311 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:56 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:57.311 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:56 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:57.311 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:56 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:57.311 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:56 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:57.311 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:56 smithi131 ceph-mon[34547]: from='osd.0 [v2:172.21.15.5:6802/2963385132,v1:172.21.15.5:6803/2963385132]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-09T00:37:57.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:56 smithi157 ceph-mon[34294]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:57.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:56 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:57.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:56 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:57.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:56 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:57.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:56 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:37:57.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:56 smithi157 ceph-mon[34294]: from='osd.0 [v2:172.21.15.5:6802/2963385132,v1:172.21.15.5:6803/2963385132]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-09T00:37:58.321 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:58 smithi131 ceph-mon[34547]: from='osd.0 [v2:172.21.15.5:6802/2963385132,v1:172.21.15.5:6803/2963385132]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-09T00:37:58.321 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:58 smithi131 ceph-mon[34547]: osdmap e6: 1 total, 0 up, 1 in 2024-05-09T00:37:58.321 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:58 smithi131 ceph-mon[34547]: from='osd.0 [v2:172.21.15.5:6802/2963385132,v1:172.21.15.5:6803/2963385132]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-05-09T00:37:58.321 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:58 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T00:37:58.321 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:58 smithi131 ceph-mon[34547]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:58.497 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:58 smithi005 ceph-mon[30365]: from='osd.0 [v2:172.21.15.5:6802/2963385132,v1:172.21.15.5:6803/2963385132]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-09T00:37:58.497 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:58 smithi005 ceph-mon[30365]: osdmap e6: 1 total, 0 up, 1 in 2024-05-09T00:37:58.497 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:58 smithi005 ceph-mon[30365]: from='osd.0 [v2:172.21.15.5:6802/2963385132,v1:172.21.15.5:6803/2963385132]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-05-09T00:37:58.497 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:58 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T00:37:58.497 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:58 smithi005 ceph-mon[30365]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:58.498 INFO:journalctl@ceph.osd.0.smithi005.stdout:May 09 00:37:58 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-0[45505]: 2024-05-09T00:37:58.279+0000 7f67fb171700 -1 osd.0 0 waiting for initial osdmap 2024-05-09T00:37:58.498 INFO:journalctl@ceph.osd.0.smithi005.stdout:May 09 00:37:58 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-0[45505]: 2024-05-09T00:37:58.284+0000 7f67f6f26700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T00:37:58.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:58 smithi157 ceph-mon[34294]: from='osd.0 [v2:172.21.15.5:6802/2963385132,v1:172.21.15.5:6803/2963385132]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-09T00:37:58.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:58 smithi157 ceph-mon[34294]: osdmap e6: 1 total, 0 up, 1 in 2024-05-09T00:37:58.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:58 smithi157 ceph-mon[34294]: from='osd.0 [v2:172.21.15.5:6802/2963385132,v1:172.21.15.5:6803/2963385132]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-05-09T00:37:58.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:58 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T00:37:58.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:58 smithi157 ceph-mon[34294]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:37:58.870 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-09T00:37:58.870 INFO:teuthology.orchestra.run.smithi131.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-09T00:37:58.870 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 10+0 records in 2024-05-09T00:37:58.870 INFO:teuthology.orchestra.run.smithi131.stderr:10+0 records out 2024-05-09T00:37:58.870 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113629 s, 923 MB/s 2024-05-09T00:37:58.870 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping successful for: 2024-05-09T00:37:59.366 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:59 smithi005 ceph-mon[30365]: from='osd.0 [v2:172.21.15.5:6802/2963385132,v1:172.21.15.5:6803/2963385132]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-05-09T00:37:59.367 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:59 smithi005 ceph-mon[30365]: osdmap e7: 1 total, 0 up, 1 in 2024-05-09T00:37:59.367 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:59 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T00:37:59.367 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:37:59 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T00:37:59.454 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:59 smithi131 ceph-mon[34547]: from='osd.0 [v2:172.21.15.5:6802/2963385132,v1:172.21.15.5:6803/2963385132]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-05-09T00:37:59.454 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:59 smithi131 ceph-mon[34547]: osdmap e7: 1 total, 0 up, 1 in 2024-05-09T00:37:59.454 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:59 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T00:37:59.454 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:37:59 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T00:37:59.482 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi131:vg_nvme/lv_4 2024-05-09T00:37:59.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:59 smithi157 ceph-mon[34294]: from='osd.0 [v2:172.21.15.5:6802/2963385132,v1:172.21.15.5:6803/2963385132]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-05-09T00:37:59.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:59 smithi157 ceph-mon[34294]: osdmap e7: 1 total, 0 up, 1 in 2024-05-09T00:37:59.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:59 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T00:37:59.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:37:59 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T00:37:59.725 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T00:38:00.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:00 smithi131 ceph-mon[34547]: purged_snaps scrub starts 2024-05-09T00:38:00.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:00 smithi131 ceph-mon[34547]: purged_snaps scrub ok 2024-05-09T00:38:00.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:00 smithi131 ceph-mon[34547]: osd.0 [v2:172.21.15.5:6802/2963385132,v1:172.21.15.5:6803/2963385132] boot 2024-05-09T00:38:00.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:00 smithi131 ceph-mon[34547]: osdmap e8: 1 total, 1 up, 1 in 2024-05-09T00:38:00.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:00 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T00:38:00.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:00 smithi131 ceph-mon[34547]: Detected new or changed devices on smithi005 2024-05-09T00:38:00.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:00 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:00.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:00 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:00.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:00 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:38:00.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:00 smithi131 ceph-mon[34547]: Adjusting osd_memory_target on smithi005 to 17093M 2024-05-09T00:38:00.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:00 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:00.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:00 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:38:00.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:00 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:38:00.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:00 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:00.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:00 smithi131 ceph-mon[34547]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:38:00.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:00 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:38:00.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:00 smithi005 ceph-mon[30365]: purged_snaps scrub starts 2024-05-09T00:38:00.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:00 smithi005 ceph-mon[30365]: purged_snaps scrub ok 2024-05-09T00:38:00.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:00 smithi005 ceph-mon[30365]: osd.0 [v2:172.21.15.5:6802/2963385132,v1:172.21.15.5:6803/2963385132] boot 2024-05-09T00:38:00.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:00 smithi005 ceph-mon[30365]: osdmap e8: 1 total, 1 up, 1 in 2024-05-09T00:38:00.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:00 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T00:38:00.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:00 smithi005 ceph-mon[30365]: Detected new or changed devices on smithi005 2024-05-09T00:38:00.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:00 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:00.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:00 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:00.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:00 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:38:00.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:00 smithi005 ceph-mon[30365]: Adjusting osd_memory_target on smithi005 to 17093M 2024-05-09T00:38:00.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:00 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:00.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:00 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:38:00.543 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:00 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:38:00.543 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:00 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:00.543 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:00 smithi005 ceph-mon[30365]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:38:00.543 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:00 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:38:00.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:00 smithi157 ceph-mon[34294]: purged_snaps scrub starts 2024-05-09T00:38:00.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:00 smithi157 ceph-mon[34294]: purged_snaps scrub ok 2024-05-09T00:38:00.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:00 smithi157 ceph-mon[34294]: osd.0 [v2:172.21.15.5:6802/2963385132,v1:172.21.15.5:6803/2963385132] boot 2024-05-09T00:38:00.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:00 smithi157 ceph-mon[34294]: osdmap e8: 1 total, 1 up, 1 in 2024-05-09T00:38:00.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:00 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T00:38:00.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:00 smithi157 ceph-mon[34294]: Detected new or changed devices on smithi005 2024-05-09T00:38:00.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:00 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:00.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:00 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:00.688 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:00 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:38:00.688 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:00 smithi157 ceph-mon[34294]: Adjusting osd_memory_target on smithi005 to 17093M 2024-05-09T00:38:00.688 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:00 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:00.688 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:00 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:38:00.688 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:00 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:38:00.688 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:00 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:00.688 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:00 smithi157 ceph-mon[34294]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:38:00.688 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:00 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:38:01.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:38:01.184+0000 7fce1da62700 1 -- 172.21.15.131:0/892959358 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce18104370 msgr2=0x7fce18104780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:38:01.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:38:01.184+0000 7fce1da62700 1 --2- 172.21.15.131:0/892959358 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce18104370 0x7fce18104780 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fce0c009990 tx=0x7fce0c009ca0 comp rx=0 tx=0).stop 2024-05-09T00:38:01.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:38:01.184+0000 7fce1da62700 1 -- 172.21.15.131:0/892959358 shutdown_connections 2024-05-09T00:38:01.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:38:01.184+0000 7fce1da62700 1 --2- 172.21.15.131:0/892959358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce18104d10 0x7fce1810b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:38:01.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:38:01.184+0000 7fce1da62700 1 --2- 172.21.15.131:0/892959358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce18103240 0x7fce18103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:38:01.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:38:01.184+0000 7fce1da62700 1 --2- 172.21.15.131:0/892959358 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce18104370 0x7fce18104780 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:38:01.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:38:01.184+0000 7fce1da62700 1 -- 172.21.15.131:0/892959358 >> 172.21.15.131:0/892959358 conn(0x7fce180fe880 msgr2=0x7fce18100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:38:01.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:38:01.184+0000 7fce1da62700 1 -- 172.21.15.131:0/892959358 shutdown_connections 2024-05-09T00:38:01.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:38:01.184+0000 7fce1da62700 1 -- 172.21.15.131:0/892959358 wait complete. 2024-05-09T00:38:01.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:38:01.185+0000 7fce1da62700 1 Processor -- start 2024-05-09T00:38:01.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:38:01.185+0000 7fce1da62700 1 -- start start 2024-05-09T00:38:01.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:38:01.185+0000 7fce1da62700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce18103240 0x7fce1819a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:38:01.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:38:01.185+0000 7fce1da62700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce18104370 0x7fce1819ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:38:01.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:38:01.185+0000 7fce1da62700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce18104d10 0x7fce181a1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:38:01.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:38:01.185+0000 7fce1da62700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fce1810eca0 con 0x7fce18104d10 2024-05-09T00:38:01.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:38:01.185+0000 7fce1da62700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fce1810eb20 con 0x7fce18103240 2024-05-09T00:38:01.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:38:01.185+0000 7fce1da62700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fce1810ee20 con 0x7fce18104370 2024-05-09T00:38:01.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:38:01.185+0000 7fce177fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce18103240 0x7fce1819a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:38:01.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:38:01.185+0000 7fce177fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce18103240 0x7fce1819a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49398/0 (socket says 172.21.15.131:49398) 2024-05-09T00:38:01.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:38:01.185+0000 7fce177fe700 1 -- 172.21.15.131:0/1306125590 learned_addr learned my addr 172.21.15.131:0/1306125590 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-09T00:38:01.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:38:01.185+0000 7fce16ffd700 1 --2- 172.21.15.131:0/1306125590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce18104370 0x7fce1819ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:38:01.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:38:01.185+0000 7fce17fff700 1 --2- 172.21.15.131:0/1306125590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce18104d10 0x7fce181a1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:38:01.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:38:01.185+0000 7fce177fe700 1 -- 172.21.15.131:0/1306125590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce18104370 msgr2=0x7fce1819ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:38:01.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:38:01.185+0000 7fce177fe700 1 --2- 172.21.15.131:0/1306125590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce18104370 0x7fce1819ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:38:01.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:38:01.185+0000 7fce177fe700 1 -- 172.21.15.131:0/1306125590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce18104d10 msgr2=0x7fce181a1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:38:01.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:38:01.185+0000 7fce177fe700 1 --2- 172.21.15.131:0/1306125590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce18104d10 0x7fce181a1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:38:01.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:38:01.185+0000 7fce177fe700 1 -- 172.21.15.131:0/1306125590 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce181a23d0 con 0x7fce18103240 2024-05-09T00:38:01.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:38:01.185+0000 7fce16ffd700 1 --2- 172.21.15.131:0/1306125590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce18104370 0x7fce1819ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:38:01.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:38:01.186+0000 7fce177fe700 1 --2- 172.21.15.131:0/1306125590 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce18103240 0x7fce1819a900 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fce0800e940 tx=0x7fce0800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:38:01.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:38:01.186+0000 7fce14ff9700 1 -- 172.21.15.131:0/1306125590 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce0800cbd0 con 0x7fce18103240 2024-05-09T00:38:01.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:38:01.186+0000 7fce1da62700 1 -- 172.21.15.131:0/1306125590 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce181a26c0 con 0x7fce18103240 2024-05-09T00:38:01.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:38:01.186+0000 7fce14ff9700 1 -- 172.21.15.131:0/1306125590 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce080044d0 con 0x7fce18103240 2024-05-09T00:38:01.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:38:01.186+0000 7fce14ff9700 1 -- 172.21.15.131:0/1306125590 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce08003df0 con 0x7fce18103240 2024-05-09T00:38:01.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:38:01.186+0000 7fce1da62700 1 -- 172.21.15.131:0/1306125590 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce181a2c50 con 0x7fce18103240 2024-05-09T00:38:01.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:38:01.187+0000 7fce1da62700 1 -- 172.21.15.131:0/1306125590 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce180512c0 con 0x7fce18103240 2024-05-09T00:38:01.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:38:01.188+0000 7fce14ff9700 1 -- 172.21.15.131:0/1306125590 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 15) ==== 149595+0+0 (secure 0 0 0) 0x7fce08004000 con 0x7fce18103240 2024-05-09T00:38:01.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:38:01.189+0000 7fce14ff9700 1 --2- 172.21.15.131:0/1306125590 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fce000b2870 0x7fce000b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:38:01.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:38:01.189+0000 7fce14ff9700 1 -- 172.21.15.131:0/1306125590 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7fce080e0a90 con 0x7fce18103240 2024-05-09T00:38:01.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:38:01.189+0000 7fce16ffd700 1 --2- 172.21.15.131:0/1306125590 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fce000b2870 0x7fce000b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:38:01.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:38:01.190+0000 7fce16ffd700 1 --2- 172.21.15.131:0/1306125590 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fce000b2870 0x7fce000b4d80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fce0c00b6c0 tx=0x7fce0c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:38:01.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:38:01.193+0000 7fce14ff9700 1 -- 172.21.15.131:0/1306125590 <== mon.2 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fce080042f0 con 0x7fce18103240 2024-05-09T00:38:01.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:38:01.406+0000 7fce1da62700 1 -- 172.21.15.131:0/1306125590 --> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fce181a2f10 con 0x7fce000b2870 2024-05-09T00:38:01.588 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:01 smithi157 ceph-mon[34294]: osdmap e9: 1 total, 1 up, 1 in 2024-05-09T00:38:01.589 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:01 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:38:01.589 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:01 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:38:01.589 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:01 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:01.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:01 smithi131 ceph-mon[34547]: osdmap e9: 1 total, 1 up, 1 in 2024-05-09T00:38:01.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:01 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:38:01.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:01 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:38:01.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:01 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:01.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:01 smithi005 ceph-mon[30365]: osdmap e9: 1 total, 1 up, 1 in 2024-05-09T00:38:01.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:01 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:38:01.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:01 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:38:01.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:01 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:02.602 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:02 smithi157 ceph-mon[34294]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:38:02.602 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:02 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:38:02.602 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:02 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:38:02.602 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:02 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:38:02.602 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:02 smithi157 ceph-mon[34294]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:38:02.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:02 smithi131 ceph-mon[34547]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:38:02.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:02 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:38:02.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:02 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:38:02.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:02 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:38:02.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:02 smithi131 ceph-mon[34547]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:38:02.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:02 smithi005 ceph-mon[30365]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:38:02.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:02 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:38:02.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:02 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:38:02.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:02 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:38:02.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:02 smithi005 ceph-mon[30365]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:38:05.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:04 smithi131 ceph-mon[34547]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:38:05.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:04 smithi131 ceph-mon[34547]: from='client.? 172.21.15.131:0/1691294900' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1c123676-7144-4a67-93a6-fd49e36a5c10"}]: dispatch 2024-05-09T00:38:05.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:04 smithi131 ceph-mon[34547]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1c123676-7144-4a67-93a6-fd49e36a5c10"}]: dispatch 2024-05-09T00:38:05.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:04 smithi131 ceph-mon[34547]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1c123676-7144-4a67-93a6-fd49e36a5c10"}]': finished 2024-05-09T00:38:05.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:04 smithi131 ceph-mon[34547]: osdmap e10: 2 total, 1 up, 2 in 2024-05-09T00:38:05.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:04 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T00:38:05.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:04 smithi131 ceph-mon[34547]: from='client.? 172.21.15.131:0/2929648133' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-1/activate.monmap"}]: dispatch 2024-05-09T00:38:05.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:04 smithi157 ceph-mon[34294]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:38:05.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:04 smithi157 ceph-mon[34294]: from='client.? 172.21.15.131:0/1691294900' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1c123676-7144-4a67-93a6-fd49e36a5c10"}]: dispatch 2024-05-09T00:38:05.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:04 smithi157 ceph-mon[34294]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1c123676-7144-4a67-93a6-fd49e36a5c10"}]: dispatch 2024-05-09T00:38:05.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:04 smithi157 ceph-mon[34294]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1c123676-7144-4a67-93a6-fd49e36a5c10"}]': finished 2024-05-09T00:38:05.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:04 smithi157 ceph-mon[34294]: osdmap e10: 2 total, 1 up, 2 in 2024-05-09T00:38:05.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:04 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T00:38:05.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:04 smithi157 ceph-mon[34294]: from='client.? 172.21.15.131:0/2929648133' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-1/activate.monmap"}]: dispatch 2024-05-09T00:38:05.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:04 smithi005 ceph-mon[30365]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:38:05.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:04 smithi005 ceph-mon[30365]: from='client.? 172.21.15.131:0/1691294900' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1c123676-7144-4a67-93a6-fd49e36a5c10"}]: dispatch 2024-05-09T00:38:05.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:04 smithi005 ceph-mon[30365]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1c123676-7144-4a67-93a6-fd49e36a5c10"}]: dispatch 2024-05-09T00:38:05.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:04 smithi005 ceph-mon[30365]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1c123676-7144-4a67-93a6-fd49e36a5c10"}]': finished 2024-05-09T00:38:05.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:04 smithi005 ceph-mon[30365]: osdmap e10: 2 total, 1 up, 2 in 2024-05-09T00:38:05.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:04 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T00:38:05.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:04 smithi005 ceph-mon[30365]: from='client.? 172.21.15.131:0/2929648133' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-1/activate.monmap"}]: dispatch 2024-05-09T00:38:07.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:06 smithi131 ceph-mon[34547]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:38:07.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:06 smithi157 ceph-mon[34294]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:38:07.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:06 smithi005 ceph-mon[30365]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:38:09.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:08 smithi131 ceph-mon[34547]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:38:09.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:08 smithi157 ceph-mon[34294]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:38:09.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:08 smithi005 ceph-mon[30365]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:38:11.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:10 smithi131 ceph-mon[34547]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:38:11.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:10 smithi157 ceph-mon[34294]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:38:11.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:10 smithi005 ceph-mon[30365]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:38:12.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:11 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-09T00:38:12.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:11 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:38:12.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:11 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-09T00:38:12.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:11 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:38:12.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:11 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-09T00:38:12.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:11 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:38:13.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:12 smithi131 ceph-mon[34547]: Deploying daemon osd.1 on smithi131 2024-05-09T00:38:13.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:12 smithi131 ceph-mon[34547]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:38:13.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:12 smithi157 ceph-mon[34294]: Deploying daemon osd.1 on smithi131 2024-05-09T00:38:13.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:12 smithi157 ceph-mon[34294]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:38:13.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:12 smithi005 ceph-mon[30365]: Deploying daemon osd.1 on smithi131 2024-05-09T00:38:13.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:12 smithi005 ceph-mon[30365]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:38:15.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:14 smithi131 ceph-mon[34547]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:38:15.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:14 smithi157 ceph-mon[34294]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:38:15.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:14 smithi005 ceph-mon[30365]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:38:16.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:16 smithi131 ceph-mon[34547]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:38:16.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:16 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:16.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:16 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:16.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:16 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:38:17.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:16 smithi157 ceph-mon[34294]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:38:17.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:16 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:17.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:16 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:17.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:16 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:38:17.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:16 smithi005 ceph-mon[30365]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:38:17.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:16 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:17.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:16 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:17.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:16 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:38:19.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:18 smithi131 ceph-mon[34547]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:38:19.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:18 smithi157 ceph-mon[34294]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:38:19.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:18 smithi005 ceph-mon[30365]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:38:19.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:38:19.922+0000 7fce14ff9700 1 -- 172.21.15.131:0/1306125590 <== mgr.14152 v2:172.21.15.5:6800/91856593 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fce181a2f10 con 0x7fce000b2870 2024-05-09T00:38:19.924 INFO:teuthology.orchestra.run.smithi131.stdout:Created osd(s) 1 on host 'smithi131' 2024-05-09T00:38:19.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:38:19.924+0000 7fce1da62700 1 -- 172.21.15.131:0/1306125590 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fce000b2870 msgr2=0x7fce000b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:38:19.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:38:19.924+0000 7fce1da62700 1 --2- 172.21.15.131:0/1306125590 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fce000b2870 0x7fce000b4d80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fce0c00b6c0 tx=0x7fce0c005fb0 comp rx=0 tx=0).stop 2024-05-09T00:38:19.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:38:19.924+0000 7fce1da62700 1 -- 172.21.15.131:0/1306125590 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce18103240 msgr2=0x7fce1819a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:38:19.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:38:19.924+0000 7fce1da62700 1 --2- 172.21.15.131:0/1306125590 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce18103240 0x7fce1819a900 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fce0800e940 tx=0x7fce0800ed00 comp rx=0 tx=0).stop 2024-05-09T00:38:19.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:38:19.925+0000 7fce1da62700 1 -- 172.21.15.131:0/1306125590 shutdown_connections 2024-05-09T00:38:19.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:38:19.925+0000 7fce1da62700 1 --2- 172.21.15.131:0/1306125590 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fce000b2870 0x7fce000b4d80 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:38:19.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:38:19.925+0000 7fce1da62700 1 --2- 172.21.15.131:0/1306125590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce18104d10 0x7fce181a1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:38:19.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:38:19.925+0000 7fce1da62700 1 --2- 172.21.15.131:0/1306125590 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce18103240 0x7fce1819a900 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:38:19.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:38:19.925+0000 7fce1da62700 1 --2- 172.21.15.131:0/1306125590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce18104370 0x7fce1819ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:38:19.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:38:19.925+0000 7fce1da62700 1 -- 172.21.15.131:0/1306125590 >> 172.21.15.131:0/1306125590 conn(0x7fce180fe880 msgr2=0x7fce181001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:38:19.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:38:19.925+0000 7fce1da62700 1 -- 172.21.15.131:0/1306125590 shutdown_connections 2024-05-09T00:38:19.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:38:19.925+0000 7fce1da62700 1 -- 172.21.15.131:0/1306125590 wait complete. 2024-05-09T00:38:20.596 DEBUG:teuthology.orchestra.run.smithi131:osd.1> sudo journalctl -f -n 0 -u ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@osd.1.service 2024-05-09T00:38:20.598 INFO:tasks.cephadm:Deploying osd.2 on smithi157 with /dev/vg_nvme/lv_4... 2024-05-09T00:38:20.598 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-09T00:38:20.853 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T00:38:21.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:20 smithi157 ceph-mon[34294]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:38:21.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:20 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:21.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:20 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:21.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:20 smithi157 ceph-mon[34294]: from='osd.1 [v2:172.21.15.131:6800/3837683163,v1:172.21.15.131:6801/3837683163]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-09T00:38:21.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:20 smithi157 ceph-mon[34294]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-09T00:38:21.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:20 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:21.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:20 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:21.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:20 smithi005 ceph-mon[30365]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:38:21.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:20 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:21.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:20 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:21.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:20 smithi005 ceph-mon[30365]: from='osd.1 [v2:172.21.15.131:6800/3837683163,v1:172.21.15.131:6801/3837683163]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-09T00:38:21.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:20 smithi005 ceph-mon[30365]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-09T00:38:21.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:20 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:21.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:20 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:21.326 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:20 smithi131 ceph-mon[34547]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:38:21.327 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:20 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:21.327 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:20 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:21.327 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:20 smithi131 ceph-mon[34547]: from='osd.1 [v2:172.21.15.131:6800/3837683163,v1:172.21.15.131:6801/3837683163]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-09T00:38:21.327 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:20 smithi131 ceph-mon[34547]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-09T00:38:21.327 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:20 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:21.327 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:20 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:21.918 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:21 smithi157 ceph-mon[34294]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-09T00:38:21.918 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:21 smithi157 ceph-mon[34294]: osdmap e11: 2 total, 1 up, 2 in 2024-05-09T00:38:21.918 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:21 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T00:38:21.918 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:21 smithi157 ceph-mon[34294]: from='osd.1 [v2:172.21.15.131:6800/3837683163,v1:172.21.15.131:6801/3837683163]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-05-09T00:38:22.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:21 smithi157 ceph-mon[34294]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-05-09T00:38:22.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:21 smithi005 ceph-mon[30365]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-09T00:38:22.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:21 smithi005 ceph-mon[30365]: osdmap e11: 2 total, 1 up, 2 in 2024-05-09T00:38:22.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:21 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T00:38:22.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:21 smithi005 ceph-mon[30365]: from='osd.1 [v2:172.21.15.131:6800/3837683163,v1:172.21.15.131:6801/3837683163]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-05-09T00:38:22.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:21 smithi005 ceph-mon[30365]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-05-09T00:38:22.351 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:21 smithi131 ceph-mon[34547]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-09T00:38:22.351 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:21 smithi131 ceph-mon[34547]: osdmap e11: 2 total, 1 up, 2 in 2024-05-09T00:38:22.351 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:21 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T00:38:22.351 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:21 smithi131 ceph-mon[34547]: from='osd.1 [v2:172.21.15.131:6800/3837683163,v1:172.21.15.131:6801/3837683163]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-05-09T00:38:22.351 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:21 smithi131 ceph-mon[34547]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-05-09T00:38:22.606 INFO:journalctl@ceph.osd.1.smithi131.stdout:May 09 00:38:22 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-1[39147]: 2024-05-09T00:38:22.349+0000 7f2fee77e700 -1 osd.1 0 waiting for initial osdmap 2024-05-09T00:38:22.606 INFO:journalctl@ceph.osd.1.smithi131.stdout:May 09 00:38:22 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-1[39147]: 2024-05-09T00:38:22.352+0000 7f2fe652b700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T00:38:22.741 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-09T00:38:22.741 INFO:teuthology.orchestra.run.smithi157.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-09T00:38:22.741 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10+0 records in 2024-05-09T00:38:22.741 INFO:teuthology.orchestra.run.smithi157.stderr:10+0 records out 2024-05-09T00:38:22.742 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00858925 s, 1.2 GB/s 2024-05-09T00:38:22.742 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping successful for: 2024-05-09T00:38:22.953 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:22 smithi131 ceph-mon[34547]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:38:22.953 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:22 smithi131 ceph-mon[34547]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-05-09T00:38:22.953 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:22 smithi131 ceph-mon[34547]: osdmap e12: 2 total, 1 up, 2 in 2024-05-09T00:38:22.953 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:22 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T00:38:22.953 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:22 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T00:38:22.953 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:22 smithi131 ceph-mon[34547]: from='osd.1 ' entity='osd.1' 2024-05-09T00:38:23.082 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:22 smithi157 ceph-mon[34294]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:38:23.082 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:22 smithi157 ceph-mon[34294]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-05-09T00:38:23.082 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:22 smithi157 ceph-mon[34294]: osdmap e12: 2 total, 1 up, 2 in 2024-05-09T00:38:23.082 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:22 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T00:38:23.082 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:22 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T00:38:23.082 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:22 smithi157 ceph-mon[34294]: from='osd.1 ' entity='osd.1' 2024-05-09T00:38:23.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:22 smithi005 ceph-mon[30365]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:38:23.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:22 smithi005 ceph-mon[30365]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-05-09T00:38:23.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:22 smithi005 ceph-mon[30365]: osdmap e12: 2 total, 1 up, 2 in 2024-05-09T00:38:23.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:22 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T00:38:23.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:22 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T00:38:23.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:22 smithi005 ceph-mon[30365]: from='osd.1 ' entity='osd.1' 2024-05-09T00:38:23.336 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi157:vg_nvme/lv_4 2024-05-09T00:38:23.596 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T00:38:24.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:23 smithi157 ceph-mon[34294]: purged_snaps scrub starts 2024-05-09T00:38:24.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:23 smithi157 ceph-mon[34294]: purged_snaps scrub ok 2024-05-09T00:38:24.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:23 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T00:38:24.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:23 smithi157 ceph-mon[34294]: osd.1 [v2:172.21.15.131:6800/3837683163,v1:172.21.15.131:6801/3837683163] boot 2024-05-09T00:38:24.188 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:23 smithi157 ceph-mon[34294]: osdmap e13: 2 total, 2 up, 2 in 2024-05-09T00:38:24.188 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:23 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T00:38:24.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:23 smithi005 ceph-mon[30365]: purged_snaps scrub starts 2024-05-09T00:38:24.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:23 smithi005 ceph-mon[30365]: purged_snaps scrub ok 2024-05-09T00:38:24.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:23 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T00:38:24.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:23 smithi005 ceph-mon[30365]: osd.1 [v2:172.21.15.131:6800/3837683163,v1:172.21.15.131:6801/3837683163] boot 2024-05-09T00:38:24.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:23 smithi005 ceph-mon[30365]: osdmap e13: 2 total, 2 up, 2 in 2024-05-09T00:38:24.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:23 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T00:38:24.348 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:23 smithi131 ceph-mon[34547]: purged_snaps scrub starts 2024-05-09T00:38:24.349 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:23 smithi131 ceph-mon[34547]: purged_snaps scrub ok 2024-05-09T00:38:24.349 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:23 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T00:38:24.349 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:23 smithi131 ceph-mon[34547]: osd.1 [v2:172.21.15.131:6800/3837683163,v1:172.21.15.131:6801/3837683163] boot 2024-05-09T00:38:24.349 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:23 smithi131 ceph-mon[34547]: osdmap e13: 2 total, 2 up, 2 in 2024-05-09T00:38:24.349 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:23 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T00:38:25.117 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:24 smithi157 ceph-mon[34294]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:38:25.117 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:24 smithi157 ceph-mon[34294]: osdmap e14: 2 total, 2 up, 2 in 2024-05-09T00:38:25.118 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:24 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:25.118 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:24 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:25.118 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:24 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:38:25.118 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:24 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:25.118 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:24 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:38:25.118 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:24 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:38:25.118 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:24 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:25.118 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:24 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:38:25.118 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:24 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:38:25.118 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:24 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:38:25.118 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:24 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:25.159 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:38:25.155+0000 7fbd3711a700 1 -- 172.21.15.157:0/3685830196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd30103f60 msgr2=0x7fbd30104370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:38:25.159 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:38:25.155+0000 7fbd3711a700 1 --2- 172.21.15.157:0/3685830196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd30103f60 0x7fbd30104370 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c009990 tx=0x7fbd1c009ca0 comp rx=0 tx=0).stop 2024-05-09T00:38:25.159 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:38:25.155+0000 7fbd3711a700 1 -- 172.21.15.157:0/3685830196 shutdown_connections 2024-05-09T00:38:25.159 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:38:25.155+0000 7fbd3711a700 1 --2- 172.21.15.157:0/3685830196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd301049e0 0x7fbd3010c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:38:25.159 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:38:25.155+0000 7fbd3711a700 1 --2- 172.21.15.157:0/3685830196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd301059b0 0x7fbd30105dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:38:25.160 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:38:25.155+0000 7fbd3711a700 1 --2- 172.21.15.157:0/3685830196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd30103f60 0x7fbd30104370 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:38:25.160 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:38:25.155+0000 7fbd3711a700 1 -- 172.21.15.157:0/3685830196 >> 172.21.15.157:0/3685830196 conn(0x7fbd300ff9d0 msgr2=0x7fbd30101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:38:25.160 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:38:25.155+0000 7fbd3711a700 1 -- 172.21.15.157:0/3685830196 shutdown_connections 2024-05-09T00:38:25.160 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:38:25.155+0000 7fbd3711a700 1 -- 172.21.15.157:0/3685830196 wait complete. 2024-05-09T00:38:25.160 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:38:25.156+0000 7fbd3711a700 1 Processor -- start 2024-05-09T00:38:25.160 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:38:25.156+0000 7fbd3711a700 1 -- start start 2024-05-09T00:38:25.160 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:38:25.156+0000 7fbd3711a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd30103f60 0x7fbd3019aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:38:25.160 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:38:25.156+0000 7fbd3711a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd301049e0 0x7fbd3019afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:38:25.160 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:38:25.156+0000 7fbd3711a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd301059b0 0x7fbd301a1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:38:25.160 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:38:25.156+0000 7fbd3711a700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3010ff90 con 0x7fbd30103f60 2024-05-09T00:38:25.160 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:38:25.156+0000 7fbd3711a700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3010fe10 con 0x7fbd301049e0 2024-05-09T00:38:25.160 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:38:25.156+0000 7fbd3711a700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbd30110110 con 0x7fbd301059b0 2024-05-09T00:38:25.160 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:38:25.157+0000 7fbd356b7700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd301059b0 0x7fbd301a1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:38:25.161 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:38:25.157+0000 7fbd356b7700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd301059b0 0x7fbd301a1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50050/0 (socket says 172.21.15.157:50050) 2024-05-09T00:38:25.161 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:38:25.157+0000 7fbd356b7700 1 -- 172.21.15.157:0/2681582698 learned_addr learned my addr 172.21.15.157:0/2681582698 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-05-09T00:38:25.161 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:38:25.157+0000 7fbd2ffff700 1 --2- 172.21.15.157:0/2681582698 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd301049e0 0x7fbd3019afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:38:25.161 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:38:25.157+0000 7fbd34eb6700 1 --2- 172.21.15.157:0/2681582698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd30103f60 0x7fbd3019aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:38:25.161 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:38:25.157+0000 7fbd356b7700 1 -- 172.21.15.157:0/2681582698 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd301049e0 msgr2=0x7fbd3019afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:38:25.161 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:38:25.157+0000 7fbd356b7700 1 --2- 172.21.15.157:0/2681582698 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd301049e0 0x7fbd3019afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:38:25.161 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:38:25.157+0000 7fbd356b7700 1 -- 172.21.15.157:0/2681582698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd30103f60 msgr2=0x7fbd3019aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:38:25.161 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:38:25.157+0000 7fbd356b7700 1 --2- 172.21.15.157:0/2681582698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd30103f60 0x7fbd3019aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:38:25.161 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:38:25.157+0000 7fbd356b7700 1 -- 172.21.15.157:0/2681582698 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd301a2600 con 0x7fbd301059b0 2024-05-09T00:38:25.161 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:38:25.157+0000 7fbd356b7700 1 --2- 172.21.15.157:0/2681582698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd301059b0 0x7fbd301a1f80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbd2800ada0 tx=0x7fbd2800c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:38:25.161 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:38:25.157+0000 7fbd2dffb700 1 -- 172.21.15.157:0/2681582698 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd28017020 con 0x7fbd301059b0 2024-05-09T00:38:25.161 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:38:25.157+0000 7fbd2dffb700 1 -- 172.21.15.157:0/2681582698 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd2800cd80 con 0x7fbd301059b0 2024-05-09T00:38:25.161 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:38:25.157+0000 7fbd2dffb700 1 -- 172.21.15.157:0/2681582698 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd28005260 con 0x7fbd301059b0 2024-05-09T00:38:25.161 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:38:25.157+0000 7fbd3711a700 1 -- 172.21.15.157:0/2681582698 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd301a28f0 con 0x7fbd301059b0 2024-05-09T00:38:25.162 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:38:25.158+0000 7fbd3711a700 1 -- 172.21.15.157:0/2681582698 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd301a2e60 con 0x7fbd301059b0 2024-05-09T00:38:25.162 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:38:25.158+0000 7fbd3711a700 1 -- 172.21.15.157:0/2681582698 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd300512c0 con 0x7fbd301059b0 2024-05-09T00:38:25.163 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:38:25.160+0000 7fbd2dffb700 1 -- 172.21.15.157:0/2681582698 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 15) ==== 149595+0+0 (secure 0 0 0) 0x7fbd2800a090 con 0x7fbd301059b0 2024-05-09T00:38:25.163 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:38:25.160+0000 7fbd2dffb700 1 --2- 172.21.15.157:0/2681582698 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fbd200b2870 0x7fbd200b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:38:25.163 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:38:25.161+0000 7fbd2dffb700 1 -- 172.21.15.157:0/2681582698 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7fbd280e05f0 con 0x7fbd301059b0 2024-05-09T00:38:25.163 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:38:25.161+0000 7fbd34eb6700 1 --2- 172.21.15.157:0/2681582698 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fbd200b2870 0x7fbd200b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:38:25.163 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:38:25.161+0000 7fbd34eb6700 1 --2- 172.21.15.157:0/2681582698 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fbd200b2870 0x7fbd200b4d80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbd301053a0 tx=0x7fbd24009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:38:25.166 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:38:25.165+0000 7fbd2dffb700 1 -- 172.21.15.157:0/2681582698 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbd28021020 con 0x7fbd301059b0 2024-05-09T00:38:25.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:24 smithi005 ceph-mon[30365]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:38:25.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:24 smithi005 ceph-mon[30365]: osdmap e14: 2 total, 2 up, 2 in 2024-05-09T00:38:25.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:24 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:25.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:24 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:25.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:24 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:38:25.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:24 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:25.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:24 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:38:25.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:24 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:38:25.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:24 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:25.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:24 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:38:25.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:24 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:38:25.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:24 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:38:25.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:24 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:25.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:24 smithi131 ceph-mon[34547]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:38:25.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:24 smithi131 ceph-mon[34547]: osdmap e14: 2 total, 2 up, 2 in 2024-05-09T00:38:25.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:24 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:25.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:24 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:25.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:24 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:38:25.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:24 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:25.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:24 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:38:25.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:24 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:38:25.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:24 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:25.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:24 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:38:25.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:24 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:38:25.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:24 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:38:25.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:24 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:25.378 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:38:25.376+0000 7fbd3711a700 1 -- 172.21.15.157:0/2681582698 --> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fbd30063c00 con 0x7fbd200b2870 2024-05-09T00:38:26.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:25 smithi157 ceph-mon[34294]: Detected new or changed devices on smithi131 2024-05-09T00:38:26.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:25 smithi157 ceph-mon[34294]: Adjusting osd_memory_target on smithi131 to 17088M 2024-05-09T00:38:26.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:25 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:38:26.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:25 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:38:26.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:25 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:38:26.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:25 smithi005 ceph-mon[30365]: Detected new or changed devices on smithi131 2024-05-09T00:38:26.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:25 smithi005 ceph-mon[30365]: Adjusting osd_memory_target on smithi131 to 17088M 2024-05-09T00:38:26.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:25 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:38:26.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:25 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:38:26.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:25 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:38:26.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:25 smithi131 ceph-mon[34547]: Detected new or changed devices on smithi131 2024-05-09T00:38:26.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:25 smithi131 ceph-mon[34547]: Adjusting osd_memory_target on smithi131 to 17088M 2024-05-09T00:38:26.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:25 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:38:26.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:25 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:38:26.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:25 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:38:27.159 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:26 smithi157 ceph-mon[34294]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:38:27.160 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:26 smithi157 ceph-mon[34294]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:38:27.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:26 smithi005 ceph-mon[30365]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:38:27.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:26 smithi005 ceph-mon[30365]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:38:27.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:26 smithi131 ceph-mon[34547]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:38:27.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:26 smithi131 ceph-mon[34547]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:38:29.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:28 smithi157 ceph-mon[34294]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:38:29.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:28 smithi157 ceph-mon[34294]: from='client.? 172.21.15.157:0/1922961898' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95ca51f0-b48b-485c-a5f7-b15c01e754b5"}]: dispatch 2024-05-09T00:38:29.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:28 smithi157 ceph-mon[34294]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95ca51f0-b48b-485c-a5f7-b15c01e754b5"}]: dispatch 2024-05-09T00:38:29.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:28 smithi157 ceph-mon[34294]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "95ca51f0-b48b-485c-a5f7-b15c01e754b5"}]': finished 2024-05-09T00:38:29.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:28 smithi157 ceph-mon[34294]: osdmap e15: 3 total, 2 up, 3 in 2024-05-09T00:38:29.188 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:28 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T00:38:29.188 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:28 smithi157 ceph-mon[34294]: from='client.? 172.21.15.157:0/372941954' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-2/activate.monmap"}]: dispatch 2024-05-09T00:38:29.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:28 smithi005 ceph-mon[30365]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:38:29.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:28 smithi005 ceph-mon[30365]: from='client.? 172.21.15.157:0/1922961898' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95ca51f0-b48b-485c-a5f7-b15c01e754b5"}]: dispatch 2024-05-09T00:38:29.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:28 smithi005 ceph-mon[30365]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95ca51f0-b48b-485c-a5f7-b15c01e754b5"}]: dispatch 2024-05-09T00:38:29.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:28 smithi005 ceph-mon[30365]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "95ca51f0-b48b-485c-a5f7-b15c01e754b5"}]': finished 2024-05-09T00:38:29.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:28 smithi005 ceph-mon[30365]: osdmap e15: 3 total, 2 up, 3 in 2024-05-09T00:38:29.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:28 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T00:38:29.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:28 smithi005 ceph-mon[30365]: from='client.? 172.21.15.157:0/372941954' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-2/activate.monmap"}]: dispatch 2024-05-09T00:38:29.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:28 smithi131 ceph-mon[34547]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:38:29.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:28 smithi131 ceph-mon[34547]: from='client.? 172.21.15.157:0/1922961898' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95ca51f0-b48b-485c-a5f7-b15c01e754b5"}]: dispatch 2024-05-09T00:38:29.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:28 smithi131 ceph-mon[34547]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95ca51f0-b48b-485c-a5f7-b15c01e754b5"}]: dispatch 2024-05-09T00:38:29.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:28 smithi131 ceph-mon[34547]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "95ca51f0-b48b-485c-a5f7-b15c01e754b5"}]': finished 2024-05-09T00:38:29.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:28 smithi131 ceph-mon[34547]: osdmap e15: 3 total, 2 up, 3 in 2024-05-09T00:38:29.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:28 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T00:38:29.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:28 smithi131 ceph-mon[34547]: from='client.? 172.21.15.157:0/372941954' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-2/activate.monmap"}]: dispatch 2024-05-09T00:38:31.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:30 smithi157 ceph-mon[34294]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:38:31.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:30 smithi005 ceph-mon[30365]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:38:31.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:30 smithi131 ceph-mon[34547]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:38:33.227 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:32 smithi157 ceph-mon[34294]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:38:33.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:32 smithi005 ceph-mon[30365]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:38:33.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:32 smithi131 ceph-mon[34547]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:38:35.228 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:34 smithi157 ceph-mon[34294]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:38:35.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:34 smithi005 ceph-mon[30365]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:38:35.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:34 smithi131 ceph-mon[34547]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:38:36.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:35 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-09T00:38:36.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:35 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:38:36.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:35 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-09T00:38:36.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:35 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:38:36.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:35 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-09T00:38:36.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:35 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:38:37.161 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:36 smithi157 ceph-mon[34294]: Deploying daemon osd.2 on smithi157 2024-05-09T00:38:37.162 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:36 smithi157 ceph-mon[34294]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:38:37.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:36 smithi005 ceph-mon[30365]: Deploying daemon osd.2 on smithi157 2024-05-09T00:38:37.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:36 smithi005 ceph-mon[30365]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:38:37.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:36 smithi131 ceph-mon[34547]: Deploying daemon osd.2 on smithi157 2024-05-09T00:38:37.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:36 smithi131 ceph-mon[34547]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:38:39.095 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:38 smithi157 ceph-mon[34294]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:38:39.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:38 smithi005 ceph-mon[30365]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:38:39.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:38 smithi131 ceph-mon[34547]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:38:41.235 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:40 smithi157 ceph-mon[34294]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:38:41.235 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:40 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:41.235 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:40 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:38:41.236 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:40 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:41.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:40 smithi005 ceph-mon[30365]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:38:41.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:40 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:41.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:40 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:38:41.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:40 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:41.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:40 smithi131 ceph-mon[34547]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:38:41.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:40 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:41.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:40 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:38:41.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:40 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:43.225 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:42 smithi157 ceph-mon[34294]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:38:43.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:42 smithi005 ceph-mon[30365]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:38:43.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:42 smithi131 ceph-mon[34547]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:38:44.048 INFO:teuthology.orchestra.run.smithi157.stdout:Created osd(s) 2 on host 'smithi157' 2024-05-09T00:38:44.049 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:38:44.046+0000 7fbd2dffb700 1 -- 172.21.15.157:0/2681582698 <== mgr.14152 v2:172.21.15.5:6800/91856593 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fbd30063c00 con 0x7fbd200b2870 2024-05-09T00:38:44.051 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:38:44.049+0000 7fbd3711a700 1 -- 172.21.15.157:0/2681582698 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fbd200b2870 msgr2=0x7fbd200b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:38:44.051 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:38:44.049+0000 7fbd3711a700 1 --2- 172.21.15.157:0/2681582698 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fbd200b2870 0x7fbd200b4d80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbd301053a0 tx=0x7fbd24009190 comp rx=0 tx=0).stop 2024-05-09T00:38:44.051 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:38:44.049+0000 7fbd3711a700 1 -- 172.21.15.157:0/2681582698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd301059b0 msgr2=0x7fbd301a1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:38:44.051 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:38:44.049+0000 7fbd3711a700 1 --2- 172.21.15.157:0/2681582698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd301059b0 0x7fbd301a1f80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbd2800ada0 tx=0x7fbd2800c330 comp rx=0 tx=0).stop 2024-05-09T00:38:44.051 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:38:44.049+0000 7fbd3711a700 1 -- 172.21.15.157:0/2681582698 shutdown_connections 2024-05-09T00:38:44.051 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:38:44.049+0000 7fbd3711a700 1 --2- 172.21.15.157:0/2681582698 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fbd200b2870 0x7fbd200b4d80 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:38:44.051 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:38:44.049+0000 7fbd3711a700 1 --2- 172.21.15.157:0/2681582698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd301059b0 0x7fbd301a1f80 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:38:44.051 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:38:44.049+0000 7fbd3711a700 1 --2- 172.21.15.157:0/2681582698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd30103f60 0x7fbd3019aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:38:44.052 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:38:44.049+0000 7fbd3711a700 1 --2- 172.21.15.157:0/2681582698 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd301049e0 0x7fbd3019afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:38:44.052 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:38:44.049+0000 7fbd3711a700 1 -- 172.21.15.157:0/2681582698 >> 172.21.15.157:0/2681582698 conn(0x7fbd300ff9d0 msgr2=0x7fbd301086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:38:44.052 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:38:44.049+0000 7fbd3711a700 1 -- 172.21.15.157:0/2681582698 shutdown_connections 2024-05-09T00:38:44.052 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:38:44.049+0000 7fbd3711a700 1 -- 172.21.15.157:0/2681582698 wait complete. 2024-05-09T00:38:44.663 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:44 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:44.663 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:44 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:44.663 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:44 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:38:44.663 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:44 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:38:44.663 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:44 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:44.663 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:44 smithi157 ceph-mon[34294]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:38:44.663 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:44 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:44.663 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:44 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:44.663 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:44 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:38:44.689 DEBUG:teuthology.orchestra.run.smithi157:osd.2> sudo journalctl -f -n 0 -u ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@osd.2.service 2024-05-09T00:38:44.691 INFO:tasks.cephadm:Deploying osd.3 on smithi005 with /dev/vg_nvme/lv_3... 2024-05-09T00:38:44.692 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-09T00:38:44.724 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:44 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:44.724 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:44 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:44.724 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:44 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:38:44.724 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:44 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:38:44.724 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:44 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:44.724 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:44 smithi005 ceph-mon[30365]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:38:44.724 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:44 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:44.724 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:44 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:44.724 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:44 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:38:44.857 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:44 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:44.858 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:44 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:44.858 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:44 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:38:44.858 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:44 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:38:44.858 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:44 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:44.858 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:44 smithi131 ceph-mon[34547]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:38:44.858 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:44 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:44.858 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:44 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:44.858 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:44 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:38:44.937 INFO:journalctl@ceph.osd.2.smithi157.stdout:May 09 00:38:44 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-2[39453]: 2024-05-09T00:38:44.749+0000 7f736b95e7c0 -1 osd.2 0 log_to_monitors true 2024-05-09T00:38:44.951 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:38:45.762 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:45 smithi005 ceph-mon[30365]: from='osd.2 [v2:172.21.15.157:6800/484818139,v1:172.21.15.157:6801/484818139]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-09T00:38:45.762 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:45 smithi005 ceph-mon[30365]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-09T00:38:45.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:45 smithi131 ceph-mon[34547]: from='osd.2 [v2:172.21.15.157:6800/484818139,v1:172.21.15.157:6801/484818139]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-09T00:38:45.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:45 smithi131 ceph-mon[34547]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-09T00:38:45.870 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:45 smithi157 ceph-mon[34294]: from='osd.2 [v2:172.21.15.157:6800/484818139,v1:172.21.15.157:6801/484818139]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-09T00:38:45.870 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:45 smithi157 ceph-mon[34294]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-09T00:38:46.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:46 smithi005 ceph-mon[30365]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-09T00:38:46.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:46 smithi005 ceph-mon[30365]: osdmap e16: 3 total, 2 up, 3 in 2024-05-09T00:38:46.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:46 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T00:38:46.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:46 smithi005 ceph-mon[30365]: from='osd.2 [v2:172.21.15.157:6800/484818139,v1:172.21.15.157:6801/484818139]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-09T00:38:46.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:46 smithi005 ceph-mon[30365]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-09T00:38:46.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:46 smithi005 ceph-mon[30365]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:38:46.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:46 smithi131 ceph-mon[34547]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-09T00:38:46.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:46 smithi131 ceph-mon[34547]: osdmap e16: 3 total, 2 up, 3 in 2024-05-09T00:38:46.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:46 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T00:38:46.857 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:46 smithi131 ceph-mon[34547]: from='osd.2 [v2:172.21.15.157:6800/484818139,v1:172.21.15.157:6801/484818139]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-09T00:38:46.857 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:46 smithi131 ceph-mon[34547]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-09T00:38:46.857 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:46 smithi131 ceph-mon[34547]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:38:46.883 INFO:journalctl@ceph.osd.2.smithi157.stdout:May 09 00:38:46 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-2[39453]: 2024-05-09T00:38:46.501+0000 7f7360fc2700 -1 osd.2 0 waiting for initial osdmap 2024-05-09T00:38:46.884 INFO:journalctl@ceph.osd.2.smithi157.stdout:May 09 00:38:46 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-2[39453]: 2024-05-09T00:38:46.505+0000 7f735cd77700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T00:38:46.884 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:46 smithi157 ceph-mon[34294]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-09T00:38:46.884 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:46 smithi157 ceph-mon[34294]: osdmap e16: 3 total, 2 up, 3 in 2024-05-09T00:38:46.884 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:46 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T00:38:46.884 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:46 smithi157 ceph-mon[34294]: from='osd.2 [v2:172.21.15.157:6800/484818139,v1:172.21.15.157:6801/484818139]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-09T00:38:46.884 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:46 smithi157 ceph-mon[34294]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-09T00:38:46.884 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:46 smithi157 ceph-mon[34294]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:38:47.062 INFO:teuthology.orchestra.run.smithi005.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-09T00:38:47.063 INFO:teuthology.orchestra.run.smithi005.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-09T00:38:47.063 INFO:teuthology.orchestra.run.smithi005.stderr: stderr: 10+0 records in 2024-05-09T00:38:47.063 INFO:teuthology.orchestra.run.smithi005.stderr:10+0 records out 2024-05-09T00:38:47.063 INFO:teuthology.orchestra.run.smithi005.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00858126 s, 1.2 GB/s 2024-05-09T00:38:47.063 INFO:teuthology.orchestra.run.smithi005.stderr:--> Zapping successful for: 2024-05-09T00:38:47.712 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:47 smithi005 ceph-mon[30365]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-05-09T00:38:47.713 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:47 smithi005 ceph-mon[30365]: osdmap e17: 3 total, 2 up, 3 in 2024-05-09T00:38:47.713 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:47 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T00:38:47.713 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:47 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T00:38:47.742 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi005:vg_nvme/lv_3 2024-05-09T00:38:47.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:47 smithi131 ceph-mon[34547]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-05-09T00:38:47.857 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:47 smithi131 ceph-mon[34547]: osdmap e17: 3 total, 2 up, 3 in 2024-05-09T00:38:47.857 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:47 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T00:38:47.857 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:47 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T00:38:47.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:47 smithi157 ceph-mon[34294]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-05-09T00:38:47.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:47 smithi157 ceph-mon[34294]: osdmap e17: 3 total, 2 up, 3 in 2024-05-09T00:38:47.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:47 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T00:38:47.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:47 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T00:38:48.034 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:38:48.794 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:48 smithi005 ceph-mon[30365]: purged_snaps scrub starts 2024-05-09T00:38:48.794 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:48 smithi005 ceph-mon[30365]: purged_snaps scrub ok 2024-05-09T00:38:48.794 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:48 smithi005 ceph-mon[30365]: osd.2 [v2:172.21.15.157:6800/484818139,v1:172.21.15.157:6801/484818139] boot 2024-05-09T00:38:48.794 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:48 smithi005 ceph-mon[30365]: osdmap e18: 3 total, 3 up, 3 in 2024-05-09T00:38:48.794 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:48 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T00:38:48.794 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:48 smithi005 ceph-mon[30365]: Detected new or changed devices on smithi157 2024-05-09T00:38:48.794 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:48 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:48.794 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:48 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:48.794 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:48 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:38:48.794 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:48 smithi005 ceph-mon[30365]: Adjusting osd_memory_target on smithi157 to 17088M 2024-05-09T00:38:48.795 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:48 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:48.795 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:48 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:38:48.795 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:48 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:38:48.795 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:48 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:48.795 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:48 smithi005 ceph-mon[30365]: pgmap v81: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:38:48.795 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:48 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-09T00:38:48.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:48 smithi131 ceph-mon[34547]: purged_snaps scrub starts 2024-05-09T00:38:48.857 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:48 smithi131 ceph-mon[34547]: purged_snaps scrub ok 2024-05-09T00:38:48.857 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:48 smithi131 ceph-mon[34547]: osd.2 [v2:172.21.15.157:6800/484818139,v1:172.21.15.157:6801/484818139] boot 2024-05-09T00:38:48.857 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:48 smithi131 ceph-mon[34547]: osdmap e18: 3 total, 3 up, 3 in 2024-05-09T00:38:48.857 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:48 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T00:38:48.857 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:48 smithi131 ceph-mon[34547]: Detected new or changed devices on smithi157 2024-05-09T00:38:48.858 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:48 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:48.858 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:48 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:48.858 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:48 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:38:48.858 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:48 smithi131 ceph-mon[34547]: Adjusting osd_memory_target on smithi157 to 17088M 2024-05-09T00:38:48.858 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:48 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:48.858 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:48 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:38:48.859 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:48 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:38:48.859 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:48 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:48.859 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:48 smithi131 ceph-mon[34547]: pgmap v81: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:38:48.859 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:48 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-09T00:38:48.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:48 smithi157 ceph-mon[34294]: purged_snaps scrub starts 2024-05-09T00:38:48.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:48 smithi157 ceph-mon[34294]: purged_snaps scrub ok 2024-05-09T00:38:48.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:48 smithi157 ceph-mon[34294]: osd.2 [v2:172.21.15.157:6800/484818139,v1:172.21.15.157:6801/484818139] boot 2024-05-09T00:38:48.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:48 smithi157 ceph-mon[34294]: osdmap e18: 3 total, 3 up, 3 in 2024-05-09T00:38:48.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:48 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T00:38:48.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:48 smithi157 ceph-mon[34294]: Detected new or changed devices on smithi157 2024-05-09T00:38:48.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:48 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:48.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:48 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:48.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:48 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:38:48.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:48 smithi157 ceph-mon[34294]: Adjusting osd_memory_target on smithi157 to 17088M 2024-05-09T00:38:48.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:48 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:48.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:48 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:38:48.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:48 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:38:48.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:48 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:48.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:48 smithi157 ceph-mon[34294]: pgmap v81: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:38:48.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:48 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-09T00:38:49.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:38:49.648+0000 7f3edf946700 1 -- 172.21.15.5:0/2688938108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ed8103240 msgr2=0x7f3ed8103650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:38:49.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:38:49.648+0000 7f3edf946700 1 --2- 172.21.15.5:0/2688938108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ed8103240 0x7f3ed8103650 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3ed4009990 tx=0x7f3ed4009ca0 comp rx=0 tx=0).stop 2024-05-09T00:38:49.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:38:49.648+0000 7f3edf946700 1 -- 172.21.15.5:0/2688938108 shutdown_connections 2024-05-09T00:38:49.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:38:49.648+0000 7f3edf946700 1 --2- 172.21.15.5:0/2688938108 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ed8104d10 0x7f3ed810b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:38:49.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:38:49.648+0000 7f3edf946700 1 --2- 172.21.15.5:0/2688938108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ed8103240 0x7f3ed8103650 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:38:49.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:38:49.648+0000 7f3edf946700 1 --2- 172.21.15.5:0/2688938108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ed8104370 0x7f3ed8104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:38:49.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:38:49.648+0000 7f3edf946700 1 -- 172.21.15.5:0/2688938108 >> 172.21.15.5:0/2688938108 conn(0x7f3ed80fe880 msgr2=0x7f3ed8100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:38:49.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:38:49.648+0000 7f3edf946700 1 -- 172.21.15.5:0/2688938108 shutdown_connections 2024-05-09T00:38:49.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:38:49.648+0000 7f3edf946700 1 -- 172.21.15.5:0/2688938108 wait complete. 2024-05-09T00:38:49.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:38:49.649+0000 7f3edf946700 1 Processor -- start 2024-05-09T00:38:49.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:38:49.649+0000 7f3edf946700 1 -- start start 2024-05-09T00:38:49.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:38:49.649+0000 7f3edf946700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ed8103240 0x7f3ed806c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:38:49.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:38:49.649+0000 7f3edf946700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ed8104370 0x7f3ed819f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:38:49.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:38:49.649+0000 7f3edf946700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ed8104d10 0x7f3ed81a1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:38:49.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:38:49.649+0000 7f3edf946700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed810ec80 con 0x7f3ed8103240 2024-05-09T00:38:49.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:38:49.649+0000 7f3edf946700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed810eb00 con 0x7f3ed8104370 2024-05-09T00:38:49.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:38:49.649+0000 7f3edf946700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed810ee00 con 0x7f3ed8104d10 2024-05-09T00:38:49.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:38:49.649+0000 7f3edd6e2700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ed8103240 0x7f3ed806c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:38:49.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:38:49.649+0000 7f3edd6e2700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ed8103240 0x7f3ed806c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57432/0 (socket says 172.21.15.5:57432) 2024-05-09T00:38:49.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:38:49.649+0000 7f3edd6e2700 1 -- 172.21.15.5:0/4132211917 learned_addr learned my addr 172.21.15.5:0/4132211917 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:38:49.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:38:49.649+0000 7f3edcee1700 1 --2- 172.21.15.5:0/4132211917 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ed8104370 0x7f3ed819f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:38:49.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:38:49.649+0000 7f3eddee3700 1 --2- 172.21.15.5:0/4132211917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ed8104d10 0x7f3ed81a1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:38:49.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:38:49.650+0000 7f3edd6e2700 1 -- 172.21.15.5:0/4132211917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ed8104d10 msgr2=0x7f3ed81a1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:38:49.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:38:49.650+0000 7f3edd6e2700 1 --2- 172.21.15.5:0/4132211917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ed8104d10 0x7f3ed81a1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:38:49.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:38:49.650+0000 7f3edd6e2700 1 -- 172.21.15.5:0/4132211917 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ed8104370 msgr2=0x7f3ed819f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:38:49.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:38:49.650+0000 7f3edd6e2700 1 --2- 172.21.15.5:0/4132211917 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ed8104370 0x7f3ed819f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:38:49.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:38:49.650+0000 7f3edd6e2700 1 -- 172.21.15.5:0/4132211917 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ed81a2510 con 0x7f3ed8103240 2024-05-09T00:38:49.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:38:49.650+0000 7f3edd6e2700 1 --2- 172.21.15.5:0/4132211917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ed8103240 0x7f3ed806c070 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3ed400b6c0 tx=0x7f3ed4009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:38:49.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:38:49.654+0000 7f3ece7fc700 1 -- 172.21.15.5:0/4132211917 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed4026070 con 0x7f3ed8103240 2024-05-09T00:38:49.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:38:49.654+0000 7f3ece7fc700 1 -- 172.21.15.5:0/4132211917 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ed4021410 con 0x7f3ed8103240 2024-05-09T00:38:49.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:38:49.654+0000 7f3edf946700 1 -- 172.21.15.5:0/4132211917 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ed81a27a0 con 0x7f3ed8103240 2024-05-09T00:38:49.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:38:49.654+0000 7f3ece7fc700 1 -- 172.21.15.5:0/4132211917 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed4017650 con 0x7f3ed8103240 2024-05-09T00:38:49.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:38:49.654+0000 7f3edf946700 1 -- 172.21.15.5:0/4132211917 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ed81a2c50 con 0x7f3ed8103240 2024-05-09T00:38:49.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:38:49.655+0000 7f3edf946700 1 -- 172.21.15.5:0/4132211917 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ed80512c0 con 0x7f3ed8103240 2024-05-09T00:38:49.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:38:49.657+0000 7f3ece7fc700 1 -- 172.21.15.5:0/4132211917 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 15) ==== 149595+0+0 (secure 0 0 0) 0x7f3ed4029070 con 0x7f3ed8103240 2024-05-09T00:38:49.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:38:49.657+0000 7f3ece7fc700 1 --2- 172.21.15.5:0/4132211917 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f3ec40b2870 0x7f3ec40b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:38:49.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:38:49.658+0000 7f3edcee1700 1 --2- 172.21.15.5:0/4132211917 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f3ec40b2870 0x7f3ec40b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:38:49.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:38:49.658+0000 7f3ece7fc700 1 -- 172.21.15.5:0/4132211917 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 3237+0+0 (secure 0 0 0) 0x7f3ed40e1fa0 con 0x7f3ed8103240 2024-05-09T00:38:49.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:38:49.658+0000 7f3edcee1700 1 --2- 172.21.15.5:0/4132211917 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f3ec40b2870 0x7f3ec40b4d80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3ed806ba20 tx=0x7f3ec8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:38:49.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:38:49.662+0000 7f3ece7fc700 1 -- 172.21.15.5:0/4132211917 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3ed4013070 con 0x7f3ed8103240 2024-05-09T00:38:49.664 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:49 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-09T00:38:49.664 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:49 smithi005 ceph-mon[30365]: osdmap e19: 3 total, 3 up, 3 in 2024-05-09T00:38:49.664 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:49 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T00:38:49.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:38:49.906+0000 7f3edf946700 1 -- 172.21.15.5:0/4132211917 --> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f3ed8063c00 con 0x7f3ec40b2870 2024-05-09T00:38:49.923 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:49 smithi005 sudo[49738]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T00:38:49.923 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:49 smithi005 sudo[49738]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T00:38:49.924 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:49 smithi005 sudo[49738]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T00:38:49.924 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:49 smithi005 sudo[49738]: pam_unix(sudo:session): session closed for user root 2024-05-09T00:38:49.924 INFO:journalctl@ceph.osd.0.smithi005.stdout:May 09 00:38:49 smithi005 sudo[49732]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T00:38:49.924 INFO:journalctl@ceph.osd.0.smithi005.stdout:May 09 00:38:49 smithi005 sudo[49732]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T00:38:49.924 INFO:journalctl@ceph.osd.0.smithi005.stdout:May 09 00:38:49 smithi005 sudo[49732]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T00:38:49.924 INFO:journalctl@ceph.osd.0.smithi005.stdout:May 09 00:38:49 smithi005 sudo[49732]: pam_unix(sudo:session): session closed for user root 2024-05-09T00:38:49.924 INFO:journalctl@ceph.osd.0.smithi005.stdout:May 09 00:38:49 smithi005 sudo[49735]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T00:38:49.924 INFO:journalctl@ceph.osd.0.smithi005.stdout:May 09 00:38:49 smithi005 sudo[49735]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T00:38:49.925 INFO:journalctl@ceph.osd.0.smithi005.stdout:May 09 00:38:49 smithi005 sudo[49735]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T00:38:49.925 INFO:journalctl@ceph.osd.0.smithi005.stdout:May 09 00:38:49 smithi005 sudo[49735]: pam_unix(sudo:session): session closed for user root 2024-05-09T00:38:49.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:49 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-09T00:38:49.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:49 smithi157 ceph-mon[34294]: osdmap e19: 3 total, 3 up, 3 in 2024-05-09T00:38:49.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:49 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T00:38:49.938 INFO:journalctl@ceph.osd.2.smithi157.stdout:May 09 00:38:49 smithi157 sudo[43063]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T00:38:49.938 INFO:journalctl@ceph.osd.2.smithi157.stdout:May 09 00:38:49 smithi157 sudo[43063]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T00:38:49.938 INFO:journalctl@ceph.osd.2.smithi157.stdout:May 09 00:38:49 smithi157 sudo[43063]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T00:38:49.938 INFO:journalctl@ceph.osd.2.smithi157.stdout:May 09 00:38:49 smithi157 sudo[43063]: pam_unix(sudo:session): session closed for user root 2024-05-09T00:38:49.938 INFO:journalctl@ceph.osd.2.smithi157.stdout:May 09 00:38:49 smithi157 sudo[43066]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T00:38:49.938 INFO:journalctl@ceph.osd.2.smithi157.stdout:May 09 00:38:49 smithi157 sudo[43066]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T00:38:49.938 INFO:journalctl@ceph.osd.2.smithi157.stdout:May 09 00:38:49 smithi157 sudo[43066]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T00:38:49.938 INFO:journalctl@ceph.osd.2.smithi157.stdout:May 09 00:38:49 smithi157 sudo[43066]: pam_unix(sudo:session): session closed for user root 2024-05-09T00:38:49.976 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:49 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-09T00:38:49.976 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:49 smithi131 ceph-mon[34547]: osdmap e19: 3 total, 3 up, 3 in 2024-05-09T00:38:49.976 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:49 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T00:38:49.977 INFO:journalctl@ceph.osd.1.smithi131.stdout:May 09 00:38:49 smithi131 sudo[42987]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T00:38:49.977 INFO:journalctl@ceph.osd.1.smithi131.stdout:May 09 00:38:49 smithi131 sudo[42987]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T00:38:49.977 INFO:journalctl@ceph.osd.1.smithi131.stdout:May 09 00:38:49 smithi131 sudo[42987]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T00:38:49.977 INFO:journalctl@ceph.osd.1.smithi131.stdout:May 09 00:38:49 smithi131 sudo[42987]: pam_unix(sudo:session): session closed for user root 2024-05-09T00:38:49.977 INFO:journalctl@ceph.osd.1.smithi131.stdout:May 09 00:38:49 smithi131 sudo[42990]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T00:38:49.977 INFO:journalctl@ceph.osd.1.smithi131.stdout:May 09 00:38:49 smithi131 sudo[42990]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T00:38:49.977 INFO:journalctl@ceph.osd.1.smithi131.stdout:May 09 00:38:49 smithi131 sudo[42990]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T00:38:49.977 INFO:journalctl@ceph.osd.1.smithi131.stdout:May 09 00:38:49 smithi131 sudo[42990]: pam_unix(sudo:session): session closed for user root 2024-05-09T00:38:50.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:49 smithi005 sudo[49742]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T00:38:50.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:49 smithi005 sudo[49742]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T00:38:50.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:49 smithi005 sudo[49742]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T00:38:50.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:49 smithi005 sudo[49742]: pam_unix(sudo:session): session closed for user root 2024-05-09T00:38:50.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:49 smithi131 sudo[42993]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T00:38:50.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:49 smithi131 sudo[42993]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T00:38:50.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:49 smithi131 sudo[42993]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T00:38:50.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:49 smithi131 sudo[42993]: pam_unix(sudo:session): session closed for user root 2024-05-09T00:38:50.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:49 smithi131 sudo[42996]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T00:38:50.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:49 smithi131 sudo[42996]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T00:38:50.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:49 smithi131 sudo[42996]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T00:38:50.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:50 smithi131 sudo[42996]: pam_unix(sudo:session): session closed for user root 2024-05-09T00:38:50.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:49 smithi157 sudo[43069]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T00:38:50.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:49 smithi157 sudo[43069]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T00:38:50.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:49 smithi157 sudo[43069]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T00:38:50.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:49 smithi157 sudo[43069]: pam_unix(sudo:session): session closed for user root 2024-05-09T00:38:50.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:49 smithi157 sudo[43072]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T00:38:50.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:49 smithi157 sudo[43072]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T00:38:50.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:49 smithi157 sudo[43072]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T00:38:50.438 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:49 smithi157 sudo[43072]: pam_unix(sudo:session): session closed for user root 2024-05-09T00:38:50.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:38:50.773+0000 7f3ece7fc700 1 -- 172.21.15.5:0/4132211917 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f3ed4009d10 con 0x7f3ed8103240 2024-05-09T00:38:50.851 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:50 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-09T00:38:50.852 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:50 smithi005 ceph-mon[30365]: osdmap e20: 3 total, 3 up, 3 in 2024-05-09T00:38:50.852 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:50 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:50.852 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:50 smithi005 ceph-mon[30365]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:38:50.852 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:50 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:50.852 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:50 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:50.852 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:50 smithi005 ceph-mon[30365]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T00:38:50.852 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:50 smithi005 ceph-mon[30365]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:38:50.852 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:50 smithi005 ceph-mon[30365]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T00:38:50.852 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:50 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T00:38:50.852 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:50 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:38:50.852 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:50 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:38:50.852 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:50 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:38:50.852 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:50 smithi005 ceph-mon[30365]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T00:38:50.853 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:50 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T00:38:50.853 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:50 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:38:50.853 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:50 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:38:50.853 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:50 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:38:50.853 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:50 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:38:50.853 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:50 smithi005 ceph-mon[30365]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T00:38:50.853 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:50 smithi005 ceph-mon[30365]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T00:38:50.853 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:50 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T00:38:50.853 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:50 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:38:50.853 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:50 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:38:50.853 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:50 smithi005 ceph-mon[30365]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T00:38:50.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:50 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-09T00:38:50.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:50 smithi157 ceph-mon[34294]: osdmap e20: 3 total, 3 up, 3 in 2024-05-09T00:38:50.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:50 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:50.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:50 smithi157 ceph-mon[34294]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:38:50.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:50 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:50.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:50 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:50.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:50 smithi157 ceph-mon[34294]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T00:38:50.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:50 smithi157 ceph-mon[34294]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:38:50.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:50 smithi157 ceph-mon[34294]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T00:38:50.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:50 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T00:38:50.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:50 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:38:50.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:50 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:38:50.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:50 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:38:50.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:50 smithi157 ceph-mon[34294]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T00:38:50.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:50 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T00:38:50.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:50 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:38:50.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:50 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:38:50.939 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:50 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:38:50.939 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:50 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:38:50.939 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:50 smithi157 ceph-mon[34294]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T00:38:50.939 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:50 smithi157 ceph-mon[34294]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T00:38:50.939 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:50 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T00:38:50.939 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:50 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:38:50.939 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:50 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:38:50.939 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:50 smithi157 ceph-mon[34294]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T00:38:51.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:50 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-09T00:38:51.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:50 smithi131 ceph-mon[34547]: osdmap e20: 3 total, 3 up, 3 in 2024-05-09T00:38:51.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:50 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:51.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:50 smithi131 ceph-mon[34547]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:38:51.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:50 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:51.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:50 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:38:51.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:50 smithi131 ceph-mon[34547]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T00:38:51.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:50 smithi131 ceph-mon[34547]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:38:51.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:50 smithi131 ceph-mon[34547]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T00:38:51.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:50 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T00:38:51.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:50 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:38:51.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:50 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:38:51.108 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:50 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:38:51.108 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:50 smithi131 ceph-mon[34547]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T00:38:51.108 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:50 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T00:38:51.108 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:50 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:38:51.108 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:50 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:38:51.108 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:50 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:38:51.108 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:50 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:38:51.108 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:50 smithi131 ceph-mon[34547]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T00:38:51.108 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:50 smithi131 ceph-mon[34547]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T00:38:51.108 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:50 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T00:38:51.108 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:50 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:38:51.108 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:50 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:38:51.109 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:50 smithi131 ceph-mon[34547]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T00:38:51.873 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:51 smithi005 ceph-mon[30365]: osdmap e21: 3 total, 3 up, 3 in 2024-05-09T00:38:51.873 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:51 smithi005 ceph-mon[30365]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-09T00:38:51.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:51 smithi157 ceph-mon[34294]: osdmap e21: 3 total, 3 up, 3 in 2024-05-09T00:38:51.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:51 smithi157 ceph-mon[34294]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-09T00:38:52.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:51 smithi131 ceph-mon[34547]: osdmap e21: 3 total, 3 up, 3 in 2024-05-09T00:38:52.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:51 smithi131 ceph-mon[34547]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-09T00:38:52.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:52 smithi157 ceph-mon[34294]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:38:53.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:52 smithi005 ceph-mon[30365]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:38:53.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:52 smithi131 ceph-mon[34547]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:38:53.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:53 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/4003451301' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ed7577fb-3676-4a08-b40d-5474eb88974f"}]: dispatch 2024-05-09T00:38:53.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:53 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/4003451301' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ed7577fb-3676-4a08-b40d-5474eb88974f"}]': finished 2024-05-09T00:38:53.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:53 smithi157 ceph-mon[34294]: osdmap e22: 4 total, 3 up, 4 in 2024-05-09T00:38:53.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:53 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T00:38:53.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:53 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/4210687357' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-3/activate.monmap"}]: dispatch 2024-05-09T00:38:54.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:53 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/4003451301' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ed7577fb-3676-4a08-b40d-5474eb88974f"}]: dispatch 2024-05-09T00:38:54.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:53 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/4003451301' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ed7577fb-3676-4a08-b40d-5474eb88974f"}]': finished 2024-05-09T00:38:54.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:53 smithi005 ceph-mon[30365]: osdmap e22: 4 total, 3 up, 4 in 2024-05-09T00:38:54.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:53 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T00:38:54.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:53 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/4210687357' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-3/activate.monmap"}]: dispatch 2024-05-09T00:38:54.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:53 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/4003451301' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ed7577fb-3676-4a08-b40d-5474eb88974f"}]: dispatch 2024-05-09T00:38:54.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:53 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/4003451301' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ed7577fb-3676-4a08-b40d-5474eb88974f"}]': finished 2024-05-09T00:38:54.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:53 smithi131 ceph-mon[34547]: osdmap e22: 4 total, 3 up, 4 in 2024-05-09T00:38:54.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:53 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T00:38:54.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:53 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/4210687357' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-3/activate.monmap"}]: dispatch 2024-05-09T00:38:54.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:54 smithi157 ceph-mon[34294]: pgmap v88: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:38:55.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:54 smithi005 ceph-mon[30365]: pgmap v88: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:38:55.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:54 smithi131 ceph-mon[34547]: pgmap v88: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:38:57.080 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:56 smithi005 ceph-mon[30365]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:38:57.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:56 smithi131 ceph-mon[34547]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:38:57.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:56 smithi157 ceph-mon[34294]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:38:59.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:38:58 smithi131 ceph-mon[34547]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:38:59.113 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:38:58 smithi005 ceph-mon[30365]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:38:59.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:38:58 smithi157 ceph-mon[34294]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:39:01.095 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:00 smithi005 ceph-mon[30365]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:39:01.096 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:00 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-09T00:39:01.096 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:00 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:39:01.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:00 smithi131 ceph-mon[34547]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:39:01.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:00 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-09T00:39:01.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:00 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:39:01.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:00 smithi157 ceph-mon[34294]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:39:01.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:00 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-09T00:39:01.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:00 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:39:02.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:01 smithi005 ceph-mon[30365]: Deploying daemon osd.3 on smithi005 2024-05-09T00:39:02.088 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:01 smithi157 ceph-mon[34294]: Deploying daemon osd.3 on smithi005 2024-05-09T00:39:02.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:01 smithi131 ceph-mon[34547]: Deploying daemon osd.3 on smithi005 2024-05-09T00:39:03.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:02 smithi131 ceph-mon[34547]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:39:03.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:02 smithi157 ceph-mon[34294]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:39:03.224 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:02 smithi005 ceph-mon[30365]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:39:04.976 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:04 smithi005 ceph-mon[30365]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:39:05.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:04 smithi131 ceph-mon[34547]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:39:05.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:04 smithi157 ceph-mon[34294]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:39:07.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:07 smithi005 ceph-mon[30365]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:39:07.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:07 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:07.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:07 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:07.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:07 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:39:07.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:07 smithi131 ceph-mon[34547]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:39:07.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:07 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:07.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:07 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:07.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:07 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:39:07.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:07 smithi157 ceph-mon[34294]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:39:07.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:07 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:07.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:07 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:07.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:07 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:39:09.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:09 smithi005 ceph-mon[30365]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:39:09.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:09 smithi131 ceph-mon[34547]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:39:09.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:09 smithi157 ceph-mon[34294]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:39:09.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:39:09.485+0000 7f3ece7fc700 1 -- 172.21.15.5:0/4132211917 <== mgr.14152 v2:172.21.15.5:6800/91856593 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3ed8063c00 con 0x7f3ec40b2870 2024-05-09T00:39:09.487 INFO:teuthology.orchestra.run.smithi005.stdout:Created osd(s) 3 on host 'smithi005' 2024-05-09T00:39:09.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:39:09.487+0000 7f3edf946700 1 -- 172.21.15.5:0/4132211917 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f3ec40b2870 msgr2=0x7f3ec40b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:39:09.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:39:09.487+0000 7f3edf946700 1 --2- 172.21.15.5:0/4132211917 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f3ec40b2870 0x7f3ec40b4d80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3ed806ba20 tx=0x7f3ec8009260 comp rx=0 tx=0).stop 2024-05-09T00:39:09.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:39:09.487+0000 7f3edf946700 1 -- 172.21.15.5:0/4132211917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ed8103240 msgr2=0x7f3ed806c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:39:09.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:39:09.487+0000 7f3edf946700 1 --2- 172.21.15.5:0/4132211917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ed8103240 0x7f3ed806c070 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3ed400b6c0 tx=0x7f3ed4009580 comp rx=0 tx=0).stop 2024-05-09T00:39:09.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:39:09.487+0000 7f3edf946700 1 -- 172.21.15.5:0/4132211917 shutdown_connections 2024-05-09T00:39:09.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:39:09.487+0000 7f3edf946700 1 --2- 172.21.15.5:0/4132211917 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f3ec40b2870 0x7f3ec40b4d80 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:39:09.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:39:09.487+0000 7f3edf946700 1 --2- 172.21.15.5:0/4132211917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ed8104d10 0x7f3ed81a1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:39:09.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:39:09.488+0000 7f3edf946700 1 --2- 172.21.15.5:0/4132211917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ed8103240 0x7f3ed806c070 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:39:09.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:39:09.488+0000 7f3edf946700 1 --2- 172.21.15.5:0/4132211917 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ed8104370 0x7f3ed819f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:39:09.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:39:09.488+0000 7f3edf946700 1 -- 172.21.15.5:0/4132211917 >> 172.21.15.5:0/4132211917 conn(0x7f3ed80fe880 msgr2=0x7f3ed81075a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:39:09.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:39:09.488+0000 7f3edf946700 1 -- 172.21.15.5:0/4132211917 shutdown_connections 2024-05-09T00:39:09.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:39:09.488+0000 7f3edf946700 1 -- 172.21.15.5:0/4132211917 wait complete. 2024-05-09T00:39:10.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:10 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:10.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:10 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:10.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:10 smithi005 ceph-mon[30365]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:39:10.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:10 smithi005 ceph-mon[30365]: from='osd.3 [v2:172.21.15.5:6810/2357377277,v1:172.21.15.5:6811/2357377277]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-09T00:39:10.595 DEBUG:teuthology.orchestra.run.smithi005:osd.3> sudo journalctl -f -n 0 -u ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@osd.3.service 2024-05-09T00:39:10.598 INFO:tasks.cephadm:Deploying osd.4 on smithi131 with /dev/vg_nvme/lv_3... 2024-05-09T00:39:10.598 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-09T00:39:10.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:10 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:10.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:10 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:10.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:10 smithi157 ceph-mon[34294]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:39:10.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:10 smithi157 ceph-mon[34294]: from='osd.3 [v2:172.21.15.5:6810/2357377277,v1:172.21.15.5:6811/2357377277]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-09T00:39:10.843 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T00:39:10.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:10 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:10.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:10 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:10.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:10 smithi131 ceph-mon[34547]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:39:10.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:10 smithi131 ceph-mon[34547]: from='osd.3 [v2:172.21.15.5:6810/2357377277,v1:172.21.15.5:6811/2357377277]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-09T00:39:11.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:11 smithi005 ceph-mon[30365]: from='osd.3 [v2:172.21.15.5:6810/2357377277,v1:172.21.15.5:6811/2357377277]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-09T00:39:11.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:11 smithi005 ceph-mon[30365]: osdmap e23: 4 total, 3 up, 4 in 2024-05-09T00:39:11.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:11 smithi005 ceph-mon[30365]: from='osd.3 [v2:172.21.15.5:6810/2357377277,v1:172.21.15.5:6811/2357377277]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-05-09T00:39:11.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:11 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T00:39:11.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:11 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:11.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:11 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:11.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:11 smithi131 ceph-mon[34547]: from='osd.3 [v2:172.21.15.5:6810/2357377277,v1:172.21.15.5:6811/2357377277]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-09T00:39:11.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:11 smithi131 ceph-mon[34547]: osdmap e23: 4 total, 3 up, 4 in 2024-05-09T00:39:11.607 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:11 smithi131 ceph-mon[34547]: from='osd.3 [v2:172.21.15.5:6810/2357377277,v1:172.21.15.5:6811/2357377277]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-05-09T00:39:11.607 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:11 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T00:39:11.607 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:11 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:11.607 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:11 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:11.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:11 smithi157 ceph-mon[34294]: from='osd.3 [v2:172.21.15.5:6810/2357377277,v1:172.21.15.5:6811/2357377277]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-09T00:39:11.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:11 smithi157 ceph-mon[34294]: osdmap e23: 4 total, 3 up, 4 in 2024-05-09T00:39:11.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:11 smithi157 ceph-mon[34294]: from='osd.3 [v2:172.21.15.5:6810/2357377277,v1:172.21.15.5:6811/2357377277]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-05-09T00:39:11.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:11 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T00:39:11.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:11 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:11.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:11 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:12.292 INFO:journalctl@ceph.osd.3.smithi005.stdout:May 09 00:39:11 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-3[52012]: 2024-05-09T00:39:11.935+0000 7f4289634700 -1 osd.3 0 waiting for initial osdmap 2024-05-09T00:39:12.292 INFO:journalctl@ceph.osd.3.smithi005.stdout:May 09 00:39:11 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-3[52012]: 2024-05-09T00:39:11.941+0000 7f42813e1700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T00:39:12.589 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-09T00:39:12.589 INFO:teuthology.orchestra.run.smithi131.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-09T00:39:12.589 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 10+0 records in 2024-05-09T00:39:12.589 INFO:teuthology.orchestra.run.smithi131.stderr:10+0 records out 2024-05-09T00:39:12.589 INFO:teuthology.orchestra.run.smithi131.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0116743 s, 898 MB/s 2024-05-09T00:39:12.589 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping successful for: 2024-05-09T00:39:12.595 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:12 smithi005 ceph-mon[30365]: from='osd.3 [v2:172.21.15.5:6810/2357377277,v1:172.21.15.5:6811/2357377277]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-05-09T00:39:12.595 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:12 smithi005 ceph-mon[30365]: osdmap e24: 4 total, 3 up, 4 in 2024-05-09T00:39:12.595 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:12 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T00:39:12.596 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:12 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T00:39:12.596 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:12 smithi005 ceph-mon[30365]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:39:12.596 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:12 smithi005 ceph-mon[30365]: from='osd.3 [v2:172.21.15.5:6810/2357377277,v1:172.21.15.5:6811/2357377277]' entity='osd.3' 2024-05-09T00:39:12.614 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:12 smithi131 ceph-mon[34547]: from='osd.3 [v2:172.21.15.5:6810/2357377277,v1:172.21.15.5:6811/2357377277]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-05-09T00:39:12.614 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:12 smithi131 ceph-mon[34547]: osdmap e24: 4 total, 3 up, 4 in 2024-05-09T00:39:12.614 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:12 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T00:39:12.614 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:12 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T00:39:12.614 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:12 smithi131 ceph-mon[34547]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:39:12.614 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:12 smithi131 ceph-mon[34547]: from='osd.3 [v2:172.21.15.5:6810/2357377277,v1:172.21.15.5:6811/2357377277]' entity='osd.3' 2024-05-09T00:39:12.795 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:12 smithi157 ceph-mon[34294]: from='osd.3 [v2:172.21.15.5:6810/2357377277,v1:172.21.15.5:6811/2357377277]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-05-09T00:39:12.795 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:12 smithi157 ceph-mon[34294]: osdmap e24: 4 total, 3 up, 4 in 2024-05-09T00:39:12.795 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:12 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T00:39:12.796 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:12 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T00:39:12.796 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:12 smithi157 ceph-mon[34294]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:39:12.796 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:12 smithi157 ceph-mon[34294]: from='osd.3 [v2:172.21.15.5:6810/2357377277,v1:172.21.15.5:6811/2357377277]' entity='osd.3' 2024-05-09T00:39:13.278 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi131:vg_nvme/lv_3 2024-05-09T00:39:13.521 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T00:39:13.564 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:13 smithi131 ceph-mon[34547]: purged_snaps scrub starts 2024-05-09T00:39:13.564 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:13 smithi131 ceph-mon[34547]: purged_snaps scrub ok 2024-05-09T00:39:13.564 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:13 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T00:39:13.564 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:13 smithi131 ceph-mon[34547]: osd.3 [v2:172.21.15.5:6810/2357377277,v1:172.21.15.5:6811/2357377277] boot 2024-05-09T00:39:13.564 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:13 smithi131 ceph-mon[34547]: osdmap e25: 4 total, 4 up, 4 in 2024-05-09T00:39:13.564 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:13 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T00:39:13.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:13 smithi157 ceph-mon[34294]: purged_snaps scrub starts 2024-05-09T00:39:13.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:13 smithi157 ceph-mon[34294]: purged_snaps scrub ok 2024-05-09T00:39:13.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:13 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T00:39:13.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:13 smithi157 ceph-mon[34294]: osd.3 [v2:172.21.15.5:6810/2357377277,v1:172.21.15.5:6811/2357377277] boot 2024-05-09T00:39:13.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:13 smithi157 ceph-mon[34294]: osdmap e25: 4 total, 4 up, 4 in 2024-05-09T00:39:13.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:13 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T00:39:13.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:13 smithi005 ceph-mon[30365]: purged_snaps scrub starts 2024-05-09T00:39:13.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:13 smithi005 ceph-mon[30365]: purged_snaps scrub ok 2024-05-09T00:39:13.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:13 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T00:39:13.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:13 smithi005 ceph-mon[30365]: osd.3 [v2:172.21.15.5:6810/2357377277,v1:172.21.15.5:6811/2357377277] boot 2024-05-09T00:39:13.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:13 smithi005 ceph-mon[30365]: osdmap e25: 4 total, 4 up, 4 in 2024-05-09T00:39:13.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:13 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T00:39:14.573 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:14 smithi131 ceph-mon[34547]: osdmap e26: 4 total, 4 up, 4 in 2024-05-09T00:39:14.573 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:14 smithi131 ceph-mon[34547]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-09T00:39:14.574 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:14 smithi131 ceph-mon[34547]: Detected new or changed devices on smithi005 2024-05-09T00:39:14.574 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:14 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:14.574 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:14 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:14.574 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:14 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:39:14.574 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:14 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:39:14.574 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:14 smithi131 ceph-mon[34547]: Adjusting osd_memory_target on smithi005 to 8546M 2024-05-09T00:39:14.574 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:14 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:14.574 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:14 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:39:14.574 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:14 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:39:14.574 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:14 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:14.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:14 smithi005 ceph-mon[30365]: osdmap e26: 4 total, 4 up, 4 in 2024-05-09T00:39:14.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:14 smithi005 ceph-mon[30365]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-09T00:39:14.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:14 smithi005 ceph-mon[30365]: Detected new or changed devices on smithi005 2024-05-09T00:39:14.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:14 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:14.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:14 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:14.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:14 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:39:14.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:14 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:39:14.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:14 smithi005 ceph-mon[30365]: Adjusting osd_memory_target on smithi005 to 8546M 2024-05-09T00:39:14.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:14 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:14.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:14 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:39:14.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:14 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:39:14.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:14 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:14.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:14 smithi157 ceph-mon[34294]: osdmap e26: 4 total, 4 up, 4 in 2024-05-09T00:39:14.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:14 smithi157 ceph-mon[34294]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-09T00:39:14.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:14 smithi157 ceph-mon[34294]: Detected new or changed devices on smithi005 2024-05-09T00:39:14.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:14 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:14.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:14 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:14.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:14 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:39:14.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:14 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:39:14.939 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:14 smithi157 ceph-mon[34294]: Adjusting osd_memory_target on smithi005 to 8546M 2024-05-09T00:39:14.939 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:14 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:14.939 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:14 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:39:14.939 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:14 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:39:14.939 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:14 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:15.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:39:15.098+0000 7f5dcb789700 1 -- 172.21.15.131:0/3282566602 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5dc4104380 msgr2=0x7f5dc4104790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:39:15.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:39:15.098+0000 7f5dcb789700 1 --2- 172.21.15.131:0/3282566602 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5dc4104380 0x7f5dc4104790 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5dc0009990 tx=0x7f5dc0009ca0 comp rx=0 tx=0).stop 2024-05-09T00:39:15.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:39:15.098+0000 7f5dcb789700 1 -- 172.21.15.131:0/3282566602 shutdown_connections 2024-05-09T00:39:15.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:39:15.098+0000 7f5dcb789700 1 --2- 172.21.15.131:0/3282566602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5dc4104d20 0x7f5dc410b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:39:15.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:39:15.098+0000 7f5dcb789700 1 --2- 172.21.15.131:0/3282566602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5dc4103250 0x7f5dc4103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:39:15.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:39:15.098+0000 7f5dcb789700 1 --2- 172.21.15.131:0/3282566602 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5dc4104380 0x7f5dc4104790 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:39:15.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:39:15.098+0000 7f5dcb789700 1 -- 172.21.15.131:0/3282566602 >> 172.21.15.131:0/3282566602 conn(0x7f5dc40fe890 msgr2=0x7f5dc4100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:39:15.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:39:15.098+0000 7f5dcb789700 1 -- 172.21.15.131:0/3282566602 shutdown_connections 2024-05-09T00:39:15.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:39:15.098+0000 7f5dcb789700 1 -- 172.21.15.131:0/3282566602 wait complete. 2024-05-09T00:39:15.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:39:15.098+0000 7f5dcb789700 1 Processor -- start 2024-05-09T00:39:15.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:39:15.098+0000 7f5dcb789700 1 -- start start 2024-05-09T00:39:15.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:39:15.099+0000 7f5dcb789700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5dc4103250 0x7f5dc4076450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:39:15.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:39:15.099+0000 7f5dcb789700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5dc4104380 0x7f5dc41a1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:39:15.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:39:15.099+0000 7f5dcb789700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5dc4104d20 0x7f5dc41a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:39:15.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:39:15.099+0000 7f5dcb789700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc410ed10 con 0x7f5dc4103250 2024-05-09T00:39:15.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:39:15.099+0000 7f5dcb789700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc410eb90 con 0x7f5dc4104d20 2024-05-09T00:39:15.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:39:15.099+0000 7f5dcb789700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc410ee90 con 0x7f5dc4104380 2024-05-09T00:39:15.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:39:15.099+0000 7f5dc9d26700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5dc4104d20 0x7f5dc41a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:39:15.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:39:15.099+0000 7f5dc9d26700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5dc4104d20 0x7f5dc41a4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34798/0 (socket says 172.21.15.131:34798) 2024-05-09T00:39:15.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:39:15.099+0000 7f5dc9d26700 1 -- 172.21.15.131:0/1902719533 learned_addr learned my addr 172.21.15.131:0/1902719533 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-09T00:39:15.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:39:15.099+0000 7f5dc9525700 1 --2- 172.21.15.131:0/1902719533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5dc4103250 0x7f5dc4076450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:39:15.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:39:15.099+0000 7f5dc8d24700 1 --2- 172.21.15.131:0/1902719533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5dc4104380 0x7f5dc41a1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:39:15.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:39:15.099+0000 7f5dc9d26700 1 -- 172.21.15.131:0/1902719533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5dc4104380 msgr2=0x7f5dc41a1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:39:15.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:39:15.099+0000 7f5dc9d26700 1 --2- 172.21.15.131:0/1902719533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5dc4104380 0x7f5dc41a1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:39:15.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:39:15.099+0000 7f5dc9d26700 1 -- 172.21.15.131:0/1902719533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5dc4103250 msgr2=0x7f5dc4076450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:39:15.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:39:15.099+0000 7f5dc9d26700 1 --2- 172.21.15.131:0/1902719533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5dc4103250 0x7f5dc4076450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:39:15.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:39:15.099+0000 7f5dc9d26700 1 -- 172.21.15.131:0/1902719533 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dc41a4650 con 0x7f5dc4104d20 2024-05-09T00:39:15.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:39:15.099+0000 7f5dc9525700 1 --2- 172.21.15.131:0/1902719533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5dc4103250 0x7f5dc4076450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:39:15.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:39:15.099+0000 7f5dc9d26700 1 --2- 172.21.15.131:0/1902719533 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5dc4104d20 0x7f5dc41a4090 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc00ada0 tx=0x7f5dbc00c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:39:15.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:39:15.099+0000 7f5dba7fc700 1 -- 172.21.15.131:0/1902719533 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc017020 con 0x7f5dc4104d20 2024-05-09T00:39:15.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:39:15.099+0000 7f5dba7fc700 1 -- 172.21.15.131:0/1902719533 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dbc00ce30 con 0x7f5dc4104d20 2024-05-09T00:39:15.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:39:15.099+0000 7f5dcb789700 1 -- 172.21.15.131:0/1902719533 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dc41a4940 con 0x7f5dc4104d20 2024-05-09T00:39:15.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:39:15.099+0000 7f5dba7fc700 1 -- 172.21.15.131:0/1902719533 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc0054a0 con 0x7f5dc4104d20 2024-05-09T00:39:15.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:39:15.100+0000 7f5dcb789700 1 -- 172.21.15.131:0/1902719533 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dc41a4e30 con 0x7f5dc4104d20 2024-05-09T00:39:15.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:39:15.101+0000 7f5dcb789700 1 -- 172.21.15.131:0/1902719533 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dc40512c0 con 0x7f5dc4104d20 2024-05-09T00:39:15.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:39:15.102+0000 7f5dba7fc700 1 -- 172.21.15.131:0/1902719533 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f5dbc00c980 con 0x7f5dc4104d20 2024-05-09T00:39:15.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:39:15.103+0000 7f5dba7fc700 1 --2- 172.21.15.131:0/1902719533 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f5db00b2880 0x7f5db00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:39:15.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:39:15.103+0000 7f5dba7fc700 1 -- 172.21.15.131:0/1902719533 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3669+0+0 (secure 0 0 0) 0x7f5dbc0e0e80 con 0x7f5dc4104d20 2024-05-09T00:39:15.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:39:15.103+0000 7f5dc9525700 1 --2- 172.21.15.131:0/1902719533 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f5db00b2880 0x7f5db00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:39:15.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:39:15.104+0000 7f5dc9525700 1 --2- 172.21.15.131:0/1902719533 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f5db00b2880 0x7f5db00b4d90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4104280 tx=0x7f5db400a680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:39:15.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:39:15.107+0000 7f5dba7fc700 1 -- 172.21.15.131:0/1902719533 <== mon.2 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5dbc021020 con 0x7f5dc4104d20 2024-05-09T00:39:15.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:39:15.317+0000 7f5dcb789700 1 -- 172.21.15.131:0/1902719533 --> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f5dc4063c00 con 0x7f5db00b2880 2024-05-09T00:39:15.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:15 smithi131 ceph-mon[34547]: osdmap e27: 4 total, 4 up, 4 in 2024-05-09T00:39:15.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:15 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:39:15.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:15 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:39:15.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:15 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:39:15.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:15 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:15.607 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:15 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:39:15.607 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:15 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:39:15.607 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:15 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:39:15.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:15 smithi005 ceph-mon[30365]: osdmap e27: 4 total, 4 up, 4 in 2024-05-09T00:39:15.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:15 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:39:15.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:15 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:39:15.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:15 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:39:15.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:15 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:15.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:15 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:39:15.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:15 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:39:15.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:15 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:39:15.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:15 smithi157 ceph-mon[34294]: osdmap e27: 4 total, 4 up, 4 in 2024-05-09T00:39:15.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:15 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:39:15.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:15 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:39:15.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:15 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:39:15.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:15 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:15.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:15 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:39:15.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:15 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:39:15.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:15 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:39:16.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:16 smithi005 ceph-mon[30365]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:39:16.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:16 smithi005 ceph-mon[30365]: pgmap v104: 1 pgs: 1 peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-09T00:39:16.799 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:16 smithi131 ceph-mon[34547]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:39:16.800 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:16 smithi131 ceph-mon[34547]: pgmap v104: 1 pgs: 1 peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-09T00:39:16.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:16 smithi157 ceph-mon[34294]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:39:16.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:16 smithi157 ceph-mon[34294]: pgmap v104: 1 pgs: 1 peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-09T00:39:19.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:18 smithi131 ceph-mon[34547]: pgmap v105: 1 pgs: 1 peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-09T00:39:19.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:18 smithi131 ceph-mon[34547]: from='client.? 172.21.15.131:0/2538556044' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fb5dcf79-a6f1-4b1a-9e3a-087fb5132472"}]: dispatch 2024-05-09T00:39:19.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:18 smithi131 ceph-mon[34547]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fb5dcf79-a6f1-4b1a-9e3a-087fb5132472"}]: dispatch 2024-05-09T00:39:19.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:18 smithi131 ceph-mon[34547]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fb5dcf79-a6f1-4b1a-9e3a-087fb5132472"}]': finished 2024-05-09T00:39:19.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:18 smithi131 ceph-mon[34547]: osdmap e28: 5 total, 4 up, 5 in 2024-05-09T00:39:19.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:18 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T00:39:19.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:18 smithi131 ceph-mon[34547]: from='client.? 172.21.15.131:0/2193192402' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-4/activate.monmap"}]: dispatch 2024-05-09T00:39:19.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:18 smithi157 ceph-mon[34294]: pgmap v105: 1 pgs: 1 peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-09T00:39:19.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:18 smithi157 ceph-mon[34294]: from='client.? 172.21.15.131:0/2538556044' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fb5dcf79-a6f1-4b1a-9e3a-087fb5132472"}]: dispatch 2024-05-09T00:39:19.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:18 smithi157 ceph-mon[34294]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fb5dcf79-a6f1-4b1a-9e3a-087fb5132472"}]: dispatch 2024-05-09T00:39:19.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:18 smithi157 ceph-mon[34294]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fb5dcf79-a6f1-4b1a-9e3a-087fb5132472"}]': finished 2024-05-09T00:39:19.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:18 smithi157 ceph-mon[34294]: osdmap e28: 5 total, 4 up, 5 in 2024-05-09T00:39:19.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:18 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T00:39:19.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:18 smithi157 ceph-mon[34294]: from='client.? 172.21.15.131:0/2193192402' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-4/activate.monmap"}]: dispatch 2024-05-09T00:39:19.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:18 smithi005 ceph-mon[30365]: pgmap v105: 1 pgs: 1 peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-09T00:39:19.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:18 smithi005 ceph-mon[30365]: from='client.? 172.21.15.131:0/2538556044' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fb5dcf79-a6f1-4b1a-9e3a-087fb5132472"}]: dispatch 2024-05-09T00:39:19.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:18 smithi005 ceph-mon[30365]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fb5dcf79-a6f1-4b1a-9e3a-087fb5132472"}]: dispatch 2024-05-09T00:39:19.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:18 smithi005 ceph-mon[30365]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fb5dcf79-a6f1-4b1a-9e3a-087fb5132472"}]': finished 2024-05-09T00:39:19.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:18 smithi005 ceph-mon[30365]: osdmap e28: 5 total, 4 up, 5 in 2024-05-09T00:39:19.293 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:18 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T00:39:19.293 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:18 smithi005 ceph-mon[30365]: from='client.? 172.21.15.131:0/2193192402' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-4/activate.monmap"}]: dispatch 2024-05-09T00:39:21.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:20 smithi131 ceph-mon[34547]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-05-09T00:39:21.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:20 smithi157 ceph-mon[34294]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-05-09T00:39:21.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:20 smithi005 ceph-mon[30365]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-05-09T00:39:23.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:22 smithi131 ceph-mon[34547]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T00:39:23.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:22 smithi157 ceph-mon[34294]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T00:39:23.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:22 smithi005 ceph-mon[30365]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T00:39:25.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:24 smithi131 ceph-mon[34547]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-05-09T00:39:25.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:24 smithi157 ceph-mon[34294]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-05-09T00:39:25.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:24 smithi005 ceph-mon[30365]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-05-09T00:39:26.099 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:25 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-09T00:39:26.099 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:25 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:39:26.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:25 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-09T00:39:26.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:25 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:39:26.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:25 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-09T00:39:26.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:25 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:39:27.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:26 smithi131 ceph-mon[34547]: Deploying daemon osd.4 on smithi131 2024-05-09T00:39:27.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:26 smithi131 ceph-mon[34547]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T00:39:27.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:26 smithi157 ceph-mon[34294]: Deploying daemon osd.4 on smithi131 2024-05-09T00:39:27.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:26 smithi157 ceph-mon[34294]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T00:39:27.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:26 smithi005 ceph-mon[30365]: Deploying daemon osd.4 on smithi131 2024-05-09T00:39:27.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:26 smithi005 ceph-mon[30365]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T00:39:29.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:28 smithi131 ceph-mon[34547]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T00:39:29.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:28 smithi157 ceph-mon[34294]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T00:39:29.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:28 smithi005 ceph-mon[30365]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T00:39:31.108 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:30 smithi131 ceph-mon[34547]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-05-09T00:39:31.108 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:30 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:31.108 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:30 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:31.108 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:30 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:39:31.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:30 smithi157 ceph-mon[34294]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-05-09T00:39:31.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:30 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:31.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:30 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:31.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:30 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:39:31.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:30 smithi005 ceph-mon[30365]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-05-09T00:39:31.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:30 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:31.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:30 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:31.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:30 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:39:33.076 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:32 smithi131 ceph-mon[34547]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-09T00:39:33.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:32 smithi157 ceph-mon[34294]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-09T00:39:33.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:32 smithi005 ceph-mon[30365]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-09T00:39:34.089 INFO:teuthology.orchestra.run.smithi131.stdout:Created osd(s) 4 on host 'smithi131' 2024-05-09T00:39:34.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:39:34.086+0000 7f5dba7fc700 1 -- 172.21.15.131:0/1902719533 <== mgr.14152 v2:172.21.15.5:6800/91856593 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5dc4063c00 con 0x7f5db00b2880 2024-05-09T00:39:34.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:39:34.089+0000 7f5dcb789700 1 -- 172.21.15.131:0/1902719533 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f5db00b2880 msgr2=0x7f5db00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:39:34.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:39:34.089+0000 7f5dcb789700 1 --2- 172.21.15.131:0/1902719533 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f5db00b2880 0x7f5db00b4d90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4104280 tx=0x7f5db400a680 comp rx=0 tx=0).stop 2024-05-09T00:39:34.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:39:34.090+0000 7f5dcb789700 1 -- 172.21.15.131:0/1902719533 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5dc4104d20 msgr2=0x7f5dc41a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:39:34.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:39:34.090+0000 7f5dcb789700 1 --2- 172.21.15.131:0/1902719533 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5dc4104d20 0x7f5dc41a4090 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc00ada0 tx=0x7f5dbc00c330 comp rx=0 tx=0).stop 2024-05-09T00:39:34.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:39:34.090+0000 7f5dcb789700 1 -- 172.21.15.131:0/1902719533 shutdown_connections 2024-05-09T00:39:34.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:39:34.090+0000 7f5dcb789700 1 --2- 172.21.15.131:0/1902719533 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f5db00b2880 0x7f5db00b4d90 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:39:34.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:39:34.090+0000 7f5dcb789700 1 --2- 172.21.15.131:0/1902719533 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5dc4104d20 0x7f5dc41a4090 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:39:34.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:39:34.090+0000 7f5dcb789700 1 --2- 172.21.15.131:0/1902719533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5dc4103250 0x7f5dc4076450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:39:34.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:39:34.090+0000 7f5dcb789700 1 --2- 172.21.15.131:0/1902719533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5dc4104380 0x7f5dc41a1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:39:34.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:39:34.090+0000 7f5dcb789700 1 -- 172.21.15.131:0/1902719533 >> 172.21.15.131:0/1902719533 conn(0x7f5dc40fe890 msgr2=0x7f5dc41075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:39:34.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:39:34.090+0000 7f5dcb789700 1 -- 172.21.15.131:0/1902719533 shutdown_connections 2024-05-09T00:39:34.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:39:34.090+0000 7f5dcb789700 1 -- 172.21.15.131:0/1902719533 wait complete. 2024-05-09T00:39:34.675 DEBUG:teuthology.orchestra.run.smithi131:osd.4> sudo journalctl -f -n 0 -u ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@osd.4.service 2024-05-09T00:39:34.679 INFO:tasks.cephadm:Deploying osd.5 on smithi157 with /dev/vg_nvme/lv_3... 2024-05-09T00:39:34.679 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-09T00:39:34.928 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T00:39:35.061 INFO:journalctl@ceph.osd.4.smithi131.stdout:May 09 00:39:34 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-4[45803]: 2024-05-09T00:39:34.727+0000 7f71c1c8a7c0 -1 osd.4 0 log_to_monitors true 2024-05-09T00:39:35.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:35 smithi131 ceph-mon[34547]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-09T00:39:35.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:35 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:35.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:35 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:35.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:35 smithi131 ceph-mon[34547]: from='osd.4 [v2:172.21.15.131:6808/2915418378,v1:172.21.15.131:6809/2915418378]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T00:39:35.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:35 smithi131 ceph-mon[34547]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T00:39:35.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:35 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:35.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:35 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:35.385 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:35 smithi005 ceph-mon[30365]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-09T00:39:35.385 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:35 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:35.385 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:35 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:35.385 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:35 smithi005 ceph-mon[30365]: from='osd.4 [v2:172.21.15.131:6808/2915418378,v1:172.21.15.131:6809/2915418378]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T00:39:35.386 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:35 smithi005 ceph-mon[30365]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T00:39:35.386 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:35 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:35.386 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:35 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:35.410 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:35 smithi157 ceph-mon[34294]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-09T00:39:35.410 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:35 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:35.410 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:35 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:35.410 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:35 smithi157 ceph-mon[34294]: from='osd.4 [v2:172.21.15.131:6808/2915418378,v1:172.21.15.131:6809/2915418378]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T00:39:35.410 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:35 smithi157 ceph-mon[34294]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T00:39:35.411 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:35 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:35.411 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:35 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:36.323 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:36 smithi157 ceph-mon[34294]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-09T00:39:36.324 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:36 smithi157 ceph-mon[34294]: from='osd.4 [v2:172.21.15.131:6808/2915418378,v1:172.21.15.131:6809/2915418378]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-05-09T00:39:36.324 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:36 smithi157 ceph-mon[34294]: osdmap e29: 5 total, 4 up, 5 in 2024-05-09T00:39:36.324 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:36 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T00:39:36.324 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:36 smithi157 ceph-mon[34294]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-05-09T00:39:36.350 INFO:journalctl@ceph.osd.4.smithi131.stdout:May 09 00:39:36 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-4[45803]: 2024-05-09T00:39:36.077+0000 7f71b72ee700 -1 osd.4 0 waiting for initial osdmap 2024-05-09T00:39:36.350 INFO:journalctl@ceph.osd.4.smithi131.stdout:May 09 00:39:36 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-4[45803]: 2024-05-09T00:39:36.083+0000 7f71b30a3700 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T00:39:36.350 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:36 smithi131 ceph-mon[34547]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-09T00:39:36.350 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:36 smithi131 ceph-mon[34547]: from='osd.4 [v2:172.21.15.131:6808/2915418378,v1:172.21.15.131:6809/2915418378]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-05-09T00:39:36.350 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:36 smithi131 ceph-mon[34547]: osdmap e29: 5 total, 4 up, 5 in 2024-05-09T00:39:36.351 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:36 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T00:39:36.351 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:36 smithi131 ceph-mon[34547]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-05-09T00:39:36.361 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:36 smithi005 ceph-mon[30365]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-09T00:39:36.361 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:36 smithi005 ceph-mon[30365]: from='osd.4 [v2:172.21.15.131:6808/2915418378,v1:172.21.15.131:6809/2915418378]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-05-09T00:39:36.362 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:36 smithi005 ceph-mon[30365]: osdmap e29: 5 total, 4 up, 5 in 2024-05-09T00:39:36.362 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:36 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T00:39:36.362 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:36 smithi005 ceph-mon[30365]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-05-09T00:39:37.033 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-09T00:39:37.033 INFO:teuthology.orchestra.run.smithi157.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-09T00:39:37.033 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10+0 records in 2024-05-09T00:39:37.033 INFO:teuthology.orchestra.run.smithi157.stderr:10+0 records out 2024-05-09T00:39:37.033 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00848312 s, 1.2 GB/s 2024-05-09T00:39:37.033 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping successful for: 2024-05-09T00:39:37.331 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:37 smithi157 ceph-mon[34294]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-09T00:39:37.331 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:37 smithi157 ceph-mon[34294]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-05-09T00:39:37.331 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:37 smithi157 ceph-mon[34294]: osdmap e30: 5 total, 4 up, 5 in 2024-05-09T00:39:37.332 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:37 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T00:39:37.332 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:37 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T00:39:37.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:37 smithi131 ceph-mon[34547]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-09T00:39:37.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:37 smithi131 ceph-mon[34547]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-05-09T00:39:37.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:37 smithi131 ceph-mon[34547]: osdmap e30: 5 total, 4 up, 5 in 2024-05-09T00:39:37.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:37 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T00:39:37.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:37 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T00:39:37.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:37 smithi005 ceph-mon[30365]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-09T00:39:37.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:37 smithi005 ceph-mon[30365]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-05-09T00:39:37.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:37 smithi005 ceph-mon[30365]: osdmap e30: 5 total, 4 up, 5 in 2024-05-09T00:39:37.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:37 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T00:39:37.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:37 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T00:39:37.632 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi157:vg_nvme/lv_3 2024-05-09T00:39:37.888 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T00:39:38.319 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:38 smithi157 ceph-mon[34294]: purged_snaps scrub starts 2024-05-09T00:39:38.319 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:38 smithi157 ceph-mon[34294]: purged_snaps scrub ok 2024-05-09T00:39:38.319 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:38 smithi157 ceph-mon[34294]: osd.4 [v2:172.21.15.131:6808/2915418378,v1:172.21.15.131:6809/2915418378] boot 2024-05-09T00:39:38.320 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:38 smithi157 ceph-mon[34294]: osdmap e31: 5 total, 5 up, 5 in 2024-05-09T00:39:38.320 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:38 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T00:39:38.320 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:38 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:38.320 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:38 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:38.320 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:38 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:39:38.320 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:38 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:39:38.320 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:38 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:38.320 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:38 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:39:38.320 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:38 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:39:38.320 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:38 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:38.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:38 smithi131 ceph-mon[34547]: purged_snaps scrub starts 2024-05-09T00:39:38.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:38 smithi131 ceph-mon[34547]: purged_snaps scrub ok 2024-05-09T00:39:38.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:38 smithi131 ceph-mon[34547]: osd.4 [v2:172.21.15.131:6808/2915418378,v1:172.21.15.131:6809/2915418378] boot 2024-05-09T00:39:38.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:38 smithi131 ceph-mon[34547]: osdmap e31: 5 total, 5 up, 5 in 2024-05-09T00:39:38.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:38 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T00:39:38.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:38 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:38.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:38 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:38.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:38 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:39:38.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:38 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:39:38.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:38 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:38.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:38 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:39:38.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:38 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:39:38.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:38 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:38.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:38 smithi005 ceph-mon[30365]: purged_snaps scrub starts 2024-05-09T00:39:38.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:38 smithi005 ceph-mon[30365]: purged_snaps scrub ok 2024-05-09T00:39:38.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:38 smithi005 ceph-mon[30365]: osd.4 [v2:172.21.15.131:6808/2915418378,v1:172.21.15.131:6809/2915418378] boot 2024-05-09T00:39:38.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:38 smithi005 ceph-mon[30365]: osdmap e31: 5 total, 5 up, 5 in 2024-05-09T00:39:38.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:38 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T00:39:38.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:38 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:38.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:38 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:38.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:38 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:39:38.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:38 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:39:38.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:38 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:38.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:38 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:39:38.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:38 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:39:38.543 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:38 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:39.340 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:39 smithi157 ceph-mon[34294]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 145 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:39:39.340 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:39 smithi157 ceph-mon[34294]: Detected new or changed devices on smithi131 2024-05-09T00:39:39.340 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:39 smithi157 ceph-mon[34294]: Adjusting osd_memory_target on smithi131 to 8544M 2024-05-09T00:39:39.340 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:39 smithi157 ceph-mon[34294]: osdmap e32: 5 total, 5 up, 5 in 2024-05-09T00:39:39.340 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:39 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:39:39.340 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:39 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:39:39.340 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:39 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:39:39.340 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:39 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:39.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:39 smithi131 ceph-mon[34547]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 145 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:39:39.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:39 smithi131 ceph-mon[34547]: Detected new or changed devices on smithi131 2024-05-09T00:39:39.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:39 smithi131 ceph-mon[34547]: Adjusting osd_memory_target on smithi131 to 8544M 2024-05-09T00:39:39.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:39 smithi131 ceph-mon[34547]: osdmap e32: 5 total, 5 up, 5 in 2024-05-09T00:39:39.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:39 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:39:39.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:39 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:39:39.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:39 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:39:39.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:39 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:39.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:39 smithi005 ceph-mon[30365]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 145 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:39:39.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:39 smithi005 ceph-mon[30365]: Detected new or changed devices on smithi131 2024-05-09T00:39:39.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:39 smithi005 ceph-mon[30365]: Adjusting osd_memory_target on smithi131 to 8544M 2024-05-09T00:39:39.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:39 smithi005 ceph-mon[30365]: osdmap e32: 5 total, 5 up, 5 in 2024-05-09T00:39:39.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:39 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:39:39.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:39 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:39:39.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:39 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:39:39.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:39 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:39.554 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:39:39.552+0000 7f405cb90700 1 -- 172.21.15.157:0/1834365920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40581059b0 msgr2=0x7f4058105dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:39:39.554 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:39:39.552+0000 7f405cb90700 1 --2- 172.21.15.157:0/1834365920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40581059b0 0x7f4058105dc0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4048009990 tx=0x7f4048009ca0 comp rx=0 tx=0).stop 2024-05-09T00:39:39.554 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:39:39.552+0000 7f405cb90700 1 -- 172.21.15.157:0/1834365920 shutdown_connections 2024-05-09T00:39:39.554 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:39:39.552+0000 7f405cb90700 1 --2- 172.21.15.157:0/1834365920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40581049e0 0x7f405810c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:39:39.554 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:39:39.552+0000 7f405cb90700 1 --2- 172.21.15.157:0/1834365920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40581059b0 0x7f4058105dc0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:39:39.554 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:39:39.552+0000 7f405cb90700 1 --2- 172.21.15.157:0/1834365920 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4058103f60 0x7f4058104370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:39:39.554 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:39:39.552+0000 7f405cb90700 1 -- 172.21.15.157:0/1834365920 >> 172.21.15.157:0/1834365920 conn(0x7f40580ff9d0 msgr2=0x7f4058101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:39:39.554 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:39:39.552+0000 7f405cb90700 1 -- 172.21.15.157:0/1834365920 shutdown_connections 2024-05-09T00:39:39.554 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:39:39.552+0000 7f405cb90700 1 -- 172.21.15.157:0/1834365920 wait complete. 2024-05-09T00:39:39.557 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:39:39.553+0000 7f405cb90700 1 Processor -- start 2024-05-09T00:39:39.557 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:39:39.553+0000 7f405cb90700 1 -- start start 2024-05-09T00:39:39.557 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:39:39.553+0000 7f405cb90700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4058103f60 0x7f405819aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:39:39.557 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:39:39.553+0000 7f405cb90700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40581049e0 0x7f405819afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:39:39.557 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:39:39.553+0000 7f405cb90700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40581059b0 0x7f40581a1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:39:39.557 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:39:39.553+0000 7f405cb90700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f405810ff50 con 0x7f40581059b0 2024-05-09T00:39:39.557 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:39:39.553+0000 7f405cb90700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f405810fdd0 con 0x7f40581049e0 2024-05-09T00:39:39.557 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:39:39.553+0000 7f405cb90700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f40581100d0 con 0x7f4058103f60 2024-05-09T00:39:39.557 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:39:39.553+0000 7f405659c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4058103f60 0x7f405819aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:39:39.557 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:39:39.553+0000 7f405659c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4058103f60 0x7f405819aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40518/0 (socket says 172.21.15.157:40518) 2024-05-09T00:39:39.557 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:39:39.553+0000 7f405659c700 1 -- 172.21.15.157:0/964510777 learned_addr learned my addr 172.21.15.157:0/964510777 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-05-09T00:39:39.558 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:39:39.554+0000 7f4056d9d700 1 --2- 172.21.15.157:0/964510777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40581059b0 0x7f40581a1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:39:39.558 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:39:39.554+0000 7f4055d9b700 1 --2- 172.21.15.157:0/964510777 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40581049e0 0x7f405819afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:39:39.558 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:39:39.554+0000 7f405659c700 1 -- 172.21.15.157:0/964510777 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40581049e0 msgr2=0x7f405819afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:39:39.558 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:39:39.554+0000 7f405659c700 1 --2- 172.21.15.157:0/964510777 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40581049e0 0x7f405819afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:39:39.558 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:39:39.554+0000 7f405659c700 1 -- 172.21.15.157:0/964510777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40581059b0 msgr2=0x7f40581a1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:39:39.558 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:39:39.554+0000 7f405659c700 1 --2- 172.21.15.157:0/964510777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40581059b0 0x7f40581a1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:39:39.558 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:39:39.554+0000 7f405659c700 1 -- 172.21.15.157:0/964510777 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40581a25f0 con 0x7f4058103f60 2024-05-09T00:39:39.558 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:39:39.554+0000 7f4055d9b700 1 --2- 172.21.15.157:0/964510777 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40581049e0 0x7f405819afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:39:39.559 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:39:39.554+0000 7f405659c700 1 --2- 172.21.15.157:0/964510777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4058103f60 0x7f405819aa90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4048000c00 tx=0x7f40480047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:39:39.559 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:39:39.554+0000 7f40477fe700 1 -- 172.21.15.157:0/964510777 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4048027070 con 0x7f4058103f60 2024-05-09T00:39:39.559 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:39:39.554+0000 7f40477fe700 1 -- 172.21.15.157:0/964510777 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4048004000 con 0x7f4058103f60 2024-05-09T00:39:39.559 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:39:39.554+0000 7f40477fe700 1 -- 172.21.15.157:0/964510777 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4048022d80 con 0x7f4058103f60 2024-05-09T00:39:39.559 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:39:39.554+0000 7f405cb90700 1 -- 172.21.15.157:0/964510777 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40581a2880 con 0x7f4058103f60 2024-05-09T00:39:39.559 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:39:39.554+0000 7f405cb90700 1 -- 172.21.15.157:0/964510777 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40581a2d50 con 0x7f4058103f60 2024-05-09T00:39:39.559 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:39:39.555+0000 7f405cb90700 1 -- 172.21.15.157:0/964510777 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40580512c0 con 0x7f4058103f60 2024-05-09T00:39:39.560 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:39:39.557+0000 7f40477fe700 1 -- 172.21.15.157:0/964510777 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f404802a070 con 0x7f4058103f60 2024-05-09T00:39:39.560 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:39:39.558+0000 7f40477fe700 1 --2- 172.21.15.157:0/964510777 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f40400b2880 0x7f40400b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:39:39.560 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:39:39.558+0000 7f40477fe700 1 -- 172.21.15.157:0/964510777 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4101+0+0 (secure 0 0 0) 0x7f40480e2340 con 0x7f4058103f60 2024-05-09T00:39:39.560 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:39:39.558+0000 7f4055d9b700 1 --2- 172.21.15.157:0/964510777 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f40400b2880 0x7f40400b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:39:39.560 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:39:39.559+0000 7f4055d9b700 1 --2- 172.21.15.157:0/964510777 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f40400b2880 0x7f40400b4d90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f405819bb40 tx=0x7f404c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:39:39.563 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:39:39.562+0000 7f40477fe700 1 -- 172.21.15.157:0/964510777 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4048013070 con 0x7f4058103f60 2024-05-09T00:39:39.775 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:39:39.774+0000 7f405cb90700 1 -- 172.21.15.157:0/964510777 --> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f405802ea50 con 0x7f40400b2880 2024-05-09T00:39:40.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:40 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:39:40.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:40 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:39:40.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:40 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:39:40.389 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:40 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:39:40.389 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:40 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:39:40.389 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:40 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:39:40.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:40 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:39:40.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:40 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:39:40.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:40 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:39:41.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:41 smithi131 ceph-mon[34547]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:39:41.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:41 smithi131 ceph-mon[34547]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:39:41.412 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:41 smithi157 ceph-mon[34294]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:39:41.412 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:41 smithi157 ceph-mon[34294]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:39:41.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:41 smithi005 ceph-mon[30365]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:39:41.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:41 smithi005 ceph-mon[30365]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:39:43.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:43 smithi131 ceph-mon[34547]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:39:43.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:43 smithi131 ceph-mon[34547]: from='client.? 172.21.15.157:0/3479701349' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f01ed5d3-8066-4d25-a113-280c83e75ebb"}]: dispatch 2024-05-09T00:39:43.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:43 smithi131 ceph-mon[34547]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f01ed5d3-8066-4d25-a113-280c83e75ebb"}]: dispatch 2024-05-09T00:39:43.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:43 smithi131 ceph-mon[34547]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f01ed5d3-8066-4d25-a113-280c83e75ebb"}]': finished 2024-05-09T00:39:43.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:43 smithi131 ceph-mon[34547]: osdmap e33: 6 total, 5 up, 6 in 2024-05-09T00:39:43.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:43 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T00:39:43.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:43 smithi157 ceph-mon[34294]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:39:43.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:43 smithi157 ceph-mon[34294]: from='client.? 172.21.15.157:0/3479701349' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f01ed5d3-8066-4d25-a113-280c83e75ebb"}]: dispatch 2024-05-09T00:39:43.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:43 smithi157 ceph-mon[34294]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f01ed5d3-8066-4d25-a113-280c83e75ebb"}]: dispatch 2024-05-09T00:39:43.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:43 smithi157 ceph-mon[34294]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f01ed5d3-8066-4d25-a113-280c83e75ebb"}]': finished 2024-05-09T00:39:43.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:43 smithi157 ceph-mon[34294]: osdmap e33: 6 total, 5 up, 6 in 2024-05-09T00:39:43.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:43 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T00:39:43.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:43 smithi005 ceph-mon[30365]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:39:43.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:43 smithi005 ceph-mon[30365]: from='client.? 172.21.15.157:0/3479701349' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f01ed5d3-8066-4d25-a113-280c83e75ebb"}]: dispatch 2024-05-09T00:39:43.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:43 smithi005 ceph-mon[30365]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f01ed5d3-8066-4d25-a113-280c83e75ebb"}]: dispatch 2024-05-09T00:39:43.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:43 smithi005 ceph-mon[30365]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f01ed5d3-8066-4d25-a113-280c83e75ebb"}]': finished 2024-05-09T00:39:43.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:43 smithi005 ceph-mon[30365]: osdmap e33: 6 total, 5 up, 6 in 2024-05-09T00:39:43.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:43 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T00:39:44.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:44 smithi131 ceph-mon[34547]: from='client.? 172.21.15.157:0/4098931672' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-5/activate.monmap"}]: dispatch 2024-05-09T00:39:44.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:44 smithi157 ceph-mon[34294]: from='client.? 172.21.15.157:0/4098931672' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-5/activate.monmap"}]: dispatch 2024-05-09T00:39:44.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:44 smithi005 ceph-mon[30365]: from='client.? 172.21.15.157:0/4098931672' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-5/activate.monmap"}]: dispatch 2024-05-09T00:39:45.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:45 smithi131 ceph-mon[34547]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:39:45.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:45 smithi157 ceph-mon[34294]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:39:45.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:45 smithi005 ceph-mon[30365]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:39:47.418 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:47 smithi157 ceph-mon[34294]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:39:47.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:47 smithi005 ceph-mon[30365]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:39:47.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:47 smithi131 ceph-mon[34547]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:39:49.421 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:49 smithi157 ceph-mon[34294]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:39:49.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:49 smithi005 ceph-mon[30365]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:39:49.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:49 smithi131 ceph-mon[34547]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:39:50.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:50 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-09T00:39:50.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:50 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:39:50.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:50 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-09T00:39:50.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:50 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:39:50.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:50 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-09T00:39:50.607 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:50 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:39:51.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:51 smithi157 ceph-mon[34294]: Deploying daemon osd.5 on smithi157 2024-05-09T00:39:51.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:51 smithi157 ceph-mon[34294]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:39:51.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:51 smithi005 ceph-mon[30365]: Deploying daemon osd.5 on smithi157 2024-05-09T00:39:51.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:51 smithi005 ceph-mon[30365]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:39:51.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:51 smithi131 ceph-mon[34547]: Deploying daemon osd.5 on smithi157 2024-05-09T00:39:51.607 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:51 smithi131 ceph-mon[34547]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:39:53.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:53 smithi157 ceph-mon[34294]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:39:53.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:53 smithi005 ceph-mon[30365]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:39:53.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:53 smithi131 ceph-mon[34547]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:39:55.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:55 smithi157 ceph-mon[34294]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:39:55.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:55 smithi005 ceph-mon[30365]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:39:55.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:55 smithi131 ceph-mon[34547]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:39:56.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:56 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:56.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:56 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:56.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:56 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:39:56.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:56 smithi131 ceph-mon[34547]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:39:56.634 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:56 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:56.634 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:56 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:56.634 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:56 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:39:56.634 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:56 smithi157 ceph-mon[34294]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:39:56.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:56 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:56.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:56 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:56.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:56 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:39:56.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:56 smithi005 ceph-mon[30365]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:39:58.898 INFO:teuthology.orchestra.run.smithi157.stdout:Created osd(s) 5 on host 'smithi157' 2024-05-09T00:39:58.899 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:39:58.894+0000 7f40477fe700 1 -- 172.21.15.157:0/964510777 <== mgr.14152 v2:172.21.15.5:6800/91856593 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f404c007da0 con 0x7f40400b2880 2024-05-09T00:39:58.900 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:39:58.897+0000 7f405cb90700 1 -- 172.21.15.157:0/964510777 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f40400b2880 msgr2=0x7f40400b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:39:58.900 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:39:58.897+0000 7f405cb90700 1 --2- 172.21.15.157:0/964510777 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f40400b2880 0x7f40400b4d90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f405819bb40 tx=0x7f404c008040 comp rx=0 tx=0).stop 2024-05-09T00:39:58.900 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:39:58.897+0000 7f405cb90700 1 -- 172.21.15.157:0/964510777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4058103f60 msgr2=0x7f405819aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:39:58.900 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:39:58.897+0000 7f405cb90700 1 --2- 172.21.15.157:0/964510777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4058103f60 0x7f405819aa90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4048000c00 tx=0x7f40480047b0 comp rx=0 tx=0).stop 2024-05-09T00:39:58.900 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:39:58.897+0000 7f405cb90700 1 -- 172.21.15.157:0/964510777 shutdown_connections 2024-05-09T00:39:58.900 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:39:58.897+0000 7f405cb90700 1 --2- 172.21.15.157:0/964510777 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f40400b2880 0x7f40400b4d90 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:39:58.900 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:39:58.897+0000 7f405cb90700 1 --2- 172.21.15.157:0/964510777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40581059b0 0x7f40581a1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:39:58.901 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:39:58.897+0000 7f405cb90700 1 --2- 172.21.15.157:0/964510777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4058103f60 0x7f405819aa90 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:39:58.901 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:39:58.897+0000 7f405cb90700 1 --2- 172.21.15.157:0/964510777 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40581049e0 0x7f405819afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:39:58.901 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:39:58.897+0000 7f405cb90700 1 -- 172.21.15.157:0/964510777 >> 172.21.15.157:0/964510777 conn(0x7f40580ff9d0 msgr2=0x7f40581086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:39:58.901 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:39:58.897+0000 7f405cb90700 1 -- 172.21.15.157:0/964510777 shutdown_connections 2024-05-09T00:39:58.901 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:39:58.897+0000 7f405cb90700 1 -- 172.21.15.157:0/964510777 wait complete. 2024-05-09T00:39:59.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:59 smithi131 ceph-mon[34547]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:39:59.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:59 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:59.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:59 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:59.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:59 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:39:59.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:59 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:39:59.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:59 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:59.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:59 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:59.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:59 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:59.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:39:59 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:39:59.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:59 smithi157 ceph-mon[34294]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:39:59.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:59 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:59.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:59 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:59.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:59 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:39:59.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:59 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:39:59.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:59 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:59.438 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:59 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:59.438 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:59 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:59.438 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:39:59 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:39:59.509 DEBUG:teuthology.orchestra.run.smithi157:osd.5> sudo journalctl -f -n 0 -u ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@osd.5.service 2024-05-09T00:39:59.512 INFO:tasks.cephadm:Deploying osd.6 on smithi005 with /dev/vg_nvme/lv_2... 2024-05-09T00:39:59.512 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-09T00:39:59.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:59 smithi005 ceph-mon[30365]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:39:59.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:59 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:59.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:59 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:59.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:59 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:39:59.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:59 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:39:59.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:59 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:59.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:59 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:59.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:59 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:39:59.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:39:59 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:39:59.773 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:40:00.179 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:00 smithi005 ceph-mon[30365]: from='osd.5 [v2:172.21.15.157:6808/4188818663,v1:172.21.15.157:6809/4188818663]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T00:40:00.180 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:00 smithi005 ceph-mon[30365]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T00:40:00.180 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:00 smithi005 ceph-mon[30365]: overall HEALTH_OK 2024-05-09T00:40:00.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:00 smithi131 ceph-mon[34547]: from='osd.5 [v2:172.21.15.157:6808/4188818663,v1:172.21.15.157:6809/4188818663]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T00:40:00.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:00 smithi131 ceph-mon[34547]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T00:40:00.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:00 smithi131 ceph-mon[34547]: overall HEALTH_OK 2024-05-09T00:40:00.388 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:00 smithi157 ceph-mon[34294]: from='osd.5 [v2:172.21.15.157:6808/4188818663,v1:172.21.15.157:6809/4188818663]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T00:40:00.389 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:00 smithi157 ceph-mon[34294]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T00:40:00.389 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:00 smithi157 ceph-mon[34294]: overall HEALTH_OK 2024-05-09T00:40:01.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:01 smithi131 ceph-mon[34547]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:40:01.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:01 smithi131 ceph-mon[34547]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-09T00:40:01.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:01 smithi131 ceph-mon[34547]: osdmap e34: 6 total, 5 up, 6 in 2024-05-09T00:40:01.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:01 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T00:40:01.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:01 smithi131 ceph-mon[34547]: from='osd.5 [v2:172.21.15.157:6808/4188818663,v1:172.21.15.157:6809/4188818663]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-09T00:40:01.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:01 smithi131 ceph-mon[34547]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-09T00:40:01.423 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:01 smithi005 ceph-mon[30365]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:40:01.423 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:01 smithi005 ceph-mon[30365]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-09T00:40:01.423 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:01 smithi005 ceph-mon[30365]: osdmap e34: 6 total, 5 up, 6 in 2024-05-09T00:40:01.423 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:01 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T00:40:01.423 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:01 smithi005 ceph-mon[30365]: from='osd.5 [v2:172.21.15.157:6808/4188818663,v1:172.21.15.157:6809/4188818663]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-09T00:40:01.423 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:01 smithi005 ceph-mon[30365]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-09T00:40:01.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:01 smithi157 ceph-mon[34294]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:40:01.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:01 smithi157 ceph-mon[34294]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-09T00:40:01.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:01 smithi157 ceph-mon[34294]: osdmap e34: 6 total, 5 up, 6 in 2024-05-09T00:40:01.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:01 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T00:40:01.438 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:01 smithi157 ceph-mon[34294]: from='osd.5 [v2:172.21.15.157:6808/4188818663,v1:172.21.15.157:6809/4188818663]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-09T00:40:01.438 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:01 smithi157 ceph-mon[34294]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-09T00:40:01.438 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 09 00:40:01 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-5[45949]: 2024-05-09T00:40:01.097+0000 7f3e5f69c700 -1 osd.5 0 waiting for initial osdmap 2024-05-09T00:40:01.438 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 09 00:40:01 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-5[45949]: 2024-05-09T00:40:01.103+0000 7f3e5b451700 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T00:40:02.019 INFO:teuthology.orchestra.run.smithi005.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-09T00:40:02.020 INFO:teuthology.orchestra.run.smithi005.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-09T00:40:02.020 INFO:teuthology.orchestra.run.smithi005.stderr: stderr: 10+0 records in 2024-05-09T00:40:02.020 INFO:teuthology.orchestra.run.smithi005.stderr:10+0 records out 2024-05-09T00:40:02.020 INFO:teuthology.orchestra.run.smithi005.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00853127 s, 1.2 GB/s 2024-05-09T00:40:02.020 INFO:teuthology.orchestra.run.smithi005.stderr:--> Zapping successful for: 2024-05-09T00:40:02.337 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:02 smithi005 ceph-mon[30365]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-05-09T00:40:02.337 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:02 smithi005 ceph-mon[30365]: osdmap e35: 6 total, 5 up, 6 in 2024-05-09T00:40:02.337 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:02 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T00:40:02.337 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:02 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T00:40:02.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:02 smithi131 ceph-mon[34547]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-05-09T00:40:02.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:02 smithi131 ceph-mon[34547]: osdmap e35: 6 total, 5 up, 6 in 2024-05-09T00:40:02.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:02 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T00:40:02.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:02 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T00:40:02.388 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:02 smithi157 ceph-mon[34294]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-05-09T00:40:02.388 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:02 smithi157 ceph-mon[34294]: osdmap e35: 6 total, 5 up, 6 in 2024-05-09T00:40:02.388 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:02 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T00:40:02.389 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:02 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T00:40:02.703 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi005:vg_nvme/lv_2 2024-05-09T00:40:02.994 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:40:03.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:03 smithi005 ceph-mon[30365]: purged_snaps scrub starts 2024-05-09T00:40:03.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:03 smithi005 ceph-mon[30365]: purged_snaps scrub ok 2024-05-09T00:40:03.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:03 smithi005 ceph-mon[30365]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:40:03.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:03 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T00:40:03.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:03 smithi005 ceph-mon[30365]: osd.5 [v2:172.21.15.157:6808/4188818663,v1:172.21.15.157:6809/4188818663] boot 2024-05-09T00:40:03.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:03 smithi005 ceph-mon[30365]: osdmap e36: 6 total, 6 up, 6 in 2024-05-09T00:40:03.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:03 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T00:40:03.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:03 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:03.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:03 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:03.358 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:03 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:40:03.358 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:03 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:40:03.358 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:03 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:03.358 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:03 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:40:03.358 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:03 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:40:03.358 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:03 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:03.359 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:03 smithi131 ceph-mon[34547]: purged_snaps scrub starts 2024-05-09T00:40:03.359 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:03 smithi131 ceph-mon[34547]: purged_snaps scrub ok 2024-05-09T00:40:03.359 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:03 smithi131 ceph-mon[34547]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:40:03.359 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:03 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T00:40:03.359 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:03 smithi131 ceph-mon[34547]: osd.5 [v2:172.21.15.157:6808/4188818663,v1:172.21.15.157:6809/4188818663] boot 2024-05-09T00:40:03.360 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:03 smithi131 ceph-mon[34547]: osdmap e36: 6 total, 6 up, 6 in 2024-05-09T00:40:03.360 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:03 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T00:40:03.360 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:03 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:03.360 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:03 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:03.360 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:03 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:40:03.360 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:03 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:40:03.360 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:03 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:03.360 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:03 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:40:03.360 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:03 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:40:03.361 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:03 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:03.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:03 smithi157 ceph-mon[34294]: purged_snaps scrub starts 2024-05-09T00:40:03.438 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:03 smithi157 ceph-mon[34294]: purged_snaps scrub ok 2024-05-09T00:40:03.438 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:03 smithi157 ceph-mon[34294]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:40:03.438 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:03 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T00:40:03.438 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:03 smithi157 ceph-mon[34294]: osd.5 [v2:172.21.15.157:6808/4188818663,v1:172.21.15.157:6809/4188818663] boot 2024-05-09T00:40:03.438 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:03 smithi157 ceph-mon[34294]: osdmap e36: 6 total, 6 up, 6 in 2024-05-09T00:40:03.438 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:03 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T00:40:03.439 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:03 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:03.440 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:03 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:03.440 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:03 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:40:03.440 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:03 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:40:03.440 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:03 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:03.440 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:03 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:40:03.441 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:03 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:40:03.441 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:03 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:04.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:04 smithi157 ceph-mon[34294]: Detected new or changed devices on smithi157 2024-05-09T00:40:04.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:04 smithi157 ceph-mon[34294]: Adjusting osd_memory_target on smithi157 to 8544M 2024-05-09T00:40:04.188 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:04 smithi157 ceph-mon[34294]: osdmap e37: 6 total, 6 up, 6 in 2024-05-09T00:40:04.374 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:04 smithi005 ceph-mon[30365]: Detected new or changed devices on smithi157 2024-05-09T00:40:04.374 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:04 smithi005 ceph-mon[30365]: Adjusting osd_memory_target on smithi157 to 8544M 2024-05-09T00:40:04.374 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:04 smithi005 ceph-mon[30365]: osdmap e37: 6 total, 6 up, 6 in 2024-05-09T00:40:04.533 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:04 smithi131 ceph-mon[34547]: Detected new or changed devices on smithi157 2024-05-09T00:40:04.533 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:04 smithi131 ceph-mon[34547]: Adjusting osd_memory_target on smithi157 to 8544M 2024-05-09T00:40:04.533 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:04 smithi131 ceph-mon[34547]: osdmap e37: 6 total, 6 up, 6 in 2024-05-09T00:40:04.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:40:04.759+0000 7f7c315c7700 1 -- 172.21.15.5:0/1203165093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c2c103b10 msgr2=0x7f7c2c105f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:40:04.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:40:04.759+0000 7f7c315c7700 1 --2- 172.21.15.5:0/1203165093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c2c103b10 0x7f7c2c105f30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c009990 tx=0x7f7c1c009ca0 comp rx=0 tx=0).stop 2024-05-09T00:40:04.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:40:04.759+0000 7f7c315c7700 1 -- 172.21.15.5:0/1203165093 shutdown_connections 2024-05-09T00:40:04.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:40:04.759+0000 7f7c315c7700 1 --2- 172.21.15.5:0/1203165093 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c2c108f00 0x7f7c2c10b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:40:04.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:40:04.759+0000 7f7c315c7700 1 --2- 172.21.15.5:0/1203165093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c2c103b10 0x7f7c2c105f30 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:40:04.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:40:04.759+0000 7f7c315c7700 1 --2- 172.21.15.5:0/1203165093 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c2c106470 0x7f7c2c108890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:40:04.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:40:04.759+0000 7f7c315c7700 1 -- 172.21.15.5:0/1203165093 >> 172.21.15.5:0/1203165093 conn(0x7f7c2c0fdbd0 msgr2=0x7f7c2c100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:40:04.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:40:04.759+0000 7f7c315c7700 1 -- 172.21.15.5:0/1203165093 shutdown_connections 2024-05-09T00:40:04.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:40:04.759+0000 7f7c315c7700 1 -- 172.21.15.5:0/1203165093 wait complete. 2024-05-09T00:40:04.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:40:04.760+0000 7f7c315c7700 1 Processor -- start 2024-05-09T00:40:04.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:40:04.760+0000 7f7c315c7700 1 -- start start 2024-05-09T00:40:04.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:40:04.760+0000 7f7c315c7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c2c103b10 0x7f7c2c1032b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:40:04.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:40:04.760+0000 7f7c315c7700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c2c106470 0x7f7c2c19f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:40:04.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:40:04.760+0000 7f7c315c7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c2c108f00 0x7f7c2c1a1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:40:04.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:40:04.760+0000 7f7c315c7700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7c2c10ecd0 con 0x7f7c2c103b10 2024-05-09T00:40:04.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:40:04.760+0000 7f7c315c7700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7c2c10eb50 con 0x7f7c2c108f00 2024-05-09T00:40:04.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:40:04.760+0000 7f7c315c7700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7c2c10ee50 con 0x7f7c2c106470 2024-05-09T00:40:04.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:40:04.760+0000 7f7c2affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c2c103b10 0x7f7c2c1032b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:40:04.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:40:04.760+0000 7f7c2affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c2c103b10 0x7f7c2c1032b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41554/0 (socket says 172.21.15.5:41554) 2024-05-09T00:40:04.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:40:04.760+0000 7f7c2affd700 1 -- 172.21.15.5:0/3137140453 learned_addr learned my addr 172.21.15.5:0/3137140453 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:40:04.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:40:04.760+0000 7f7c227fc700 1 --2- 172.21.15.5:0/3137140453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c2c106470 0x7f7c2c19f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:40:04.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:40:04.760+0000 7f7c2b7fe700 1 --2- 172.21.15.5:0/3137140453 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c2c108f00 0x7f7c2c1a1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:40:04.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:40:04.761+0000 7f7c2affd700 1 -- 172.21.15.5:0/3137140453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c2c106470 msgr2=0x7f7c2c19f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:40:04.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:40:04.761+0000 7f7c2affd700 1 --2- 172.21.15.5:0/3137140453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c2c106470 0x7f7c2c19f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:40:04.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:40:04.761+0000 7f7c2affd700 1 -- 172.21.15.5:0/3137140453 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c2c108f00 msgr2=0x7f7c2c1a1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:40:04.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:40:04.761+0000 7f7c2affd700 1 --2- 172.21.15.5:0/3137140453 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c2c108f00 0x7f7c2c1a1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:40:04.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:40:04.761+0000 7f7c2affd700 1 -- 172.21.15.5:0/3137140453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c2c1a24b0 con 0x7f7c2c103b10 2024-05-09T00:40:04.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:40:04.761+0000 7f7c2b7fe700 1 --2- 172.21.15.5:0/3137140453 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c2c108f00 0x7f7c2c1a1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:40:04.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:40:04.761+0000 7f7c2affd700 1 --2- 172.21.15.5:0/3137140453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c2c103b10 0x7f7c2c1032b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c00b6c0 tx=0x7f7c1c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:40:04.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:40:04.761+0000 7f7c28ff9700 1 -- 172.21.15.5:0/3137140453 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c026070 con 0x7f7c2c103b10 2024-05-09T00:40:04.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:40:04.761+0000 7f7c315c7700 1 -- 172.21.15.5:0/3137140453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c2c1a2740 con 0x7f7c2c103b10 2024-05-09T00:40:04.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:40:04.761+0000 7f7c28ff9700 1 -- 172.21.15.5:0/3137140453 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c1c021410 con 0x7f7c2c103b10 2024-05-09T00:40:04.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:40:04.761+0000 7f7c28ff9700 1 -- 172.21.15.5:0/3137140453 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c017650 con 0x7f7c2c103b10 2024-05-09T00:40:04.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:40:04.761+0000 7f7c315c7700 1 -- 172.21.15.5:0/3137140453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c2c1a2ba0 con 0x7f7c2c103b10 2024-05-09T00:40:04.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:40:04.762+0000 7f7c315c7700 1 -- 172.21.15.5:0/3137140453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c2c0512c0 con 0x7f7c2c103b10 2024-05-09T00:40:04.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:40:04.765+0000 7f7c28ff9700 1 -- 172.21.15.5:0/3137140453 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f7c1c029070 con 0x7f7c2c103b10 2024-05-09T00:40:04.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:40:04.766+0000 7f7c28ff9700 1 --2- 172.21.15.5:0/3137140453 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f7c180b2880 0x7f7c180b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:40:04.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:40:04.766+0000 7f7c227fc700 1 --2- 172.21.15.5:0/3137140453 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f7c180b2880 0x7f7c180b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:40:04.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:40:04.766+0000 7f7c28ff9700 1 -- 172.21.15.5:0/3137140453 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4533+0+0 (secure 0 0 0) 0x7f7c1c0e2780 con 0x7f7c2c103b10 2024-05-09T00:40:04.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:40:04.766+0000 7f7c227fc700 1 --2- 172.21.15.5:0/3137140453 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f7c180b2880 0x7f7c180b4d90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c102c60 tx=0x7f7c14009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:40:04.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:40:04.770+0000 7f7c28ff9700 1 -- 172.21.15.5:0/3137140453 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7c1c013070 con 0x7f7c2c103b10 2024-05-09T00:40:04.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:40:04.993+0000 7f7c315c7700 1 -- 172.21.15.5:0/3137140453 --> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f7c2c105b70 con 0x7f7c180b2880 2024-05-09T00:40:05.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:05 smithi005 ceph-mon[30365]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-05-09T00:40:05.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:05 smithi005 ceph-mon[30365]: osdmap e38: 6 total, 6 up, 6 in 2024-05-09T00:40:05.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:05 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:40:05.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:05 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:40:05.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:05 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:40:05.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:05 smithi157 ceph-mon[34294]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-05-09T00:40:05.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:05 smithi157 ceph-mon[34294]: osdmap e38: 6 total, 6 up, 6 in 2024-05-09T00:40:05.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:05 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:40:05.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:05 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:40:05.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:05 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:40:05.462 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:05 smithi131 ceph-mon[34547]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-05-09T00:40:05.462 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:05 smithi131 ceph-mon[34547]: osdmap e38: 6 total, 6 up, 6 in 2024-05-09T00:40:05.462 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:05 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:40:05.462 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:05 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:40:05.462 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:05 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:40:06.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:06 smithi005 ceph-mon[30365]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:40:06.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:06 smithi157 ceph-mon[34294]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:40:06.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:06 smithi131 ceph-mon[34547]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:40:07.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:07 smithi157 ceph-mon[34294]: pgmap v140: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-09T00:40:07.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:07 smithi005 ceph-mon[30365]: pgmap v140: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-09T00:40:07.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:07 smithi131 ceph-mon[34547]: pgmap v140: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-09T00:40:08.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:08 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/4212911128' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "932379a8-c97b-4acc-8665-81f7d9cbce65"}]: dispatch 2024-05-09T00:40:08.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:08 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/4212911128' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "932379a8-c97b-4acc-8665-81f7d9cbce65"}]': finished 2024-05-09T00:40:08.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:08 smithi157 ceph-mon[34294]: osdmap e39: 7 total, 6 up, 7 in 2024-05-09T00:40:08.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:08 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T00:40:08.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:08 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/4212911128' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "932379a8-c97b-4acc-8665-81f7d9cbce65"}]: dispatch 2024-05-09T00:40:08.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:08 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/4212911128' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "932379a8-c97b-4acc-8665-81f7d9cbce65"}]': finished 2024-05-09T00:40:08.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:08 smithi005 ceph-mon[30365]: osdmap e39: 7 total, 6 up, 7 in 2024-05-09T00:40:08.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:08 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T00:40:08.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:08 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/4212911128' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "932379a8-c97b-4acc-8665-81f7d9cbce65"}]: dispatch 2024-05-09T00:40:08.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:08 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/4212911128' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "932379a8-c97b-4acc-8665-81f7d9cbce65"}]': finished 2024-05-09T00:40:08.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:08 smithi131 ceph-mon[34547]: osdmap e39: 7 total, 6 up, 7 in 2024-05-09T00:40:08.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:08 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T00:40:09.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:09 smithi157 ceph-mon[34294]: pgmap v142: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-09T00:40:09.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:09 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/2988984400' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-6/activate.monmap"}]: dispatch 2024-05-09T00:40:09.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:09 smithi005 ceph-mon[30365]: pgmap v142: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-09T00:40:09.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:09 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/2988984400' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-6/activate.monmap"}]: dispatch 2024-05-09T00:40:09.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:09 smithi131 ceph-mon[34547]: pgmap v142: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-09T00:40:09.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:09 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/2988984400' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-6/activate.monmap"}]: dispatch 2024-05-09T00:40:11.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:11 smithi157 ceph-mon[34294]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-05-09T00:40:11.476 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:11 smithi005 ceph-mon[30365]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-05-09T00:40:11.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:11 smithi131 ceph-mon[34547]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-05-09T00:40:13.247 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:13 smithi005 ceph-mon[30365]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T00:40:13.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:13 smithi157 ceph-mon[34294]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T00:40:13.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:13 smithi131 ceph-mon[34547]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T00:40:15.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:15 smithi157 ceph-mon[34294]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-09T00:40:15.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:15 smithi131 ceph-mon[34547]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-09T00:40:15.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:15 smithi005 ceph-mon[30365]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-09T00:40:16.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:16 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-09T00:40:16.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:16 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:40:16.439 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:16 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-09T00:40:16.439 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:16 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:40:16.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:16 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-09T00:40:16.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:16 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:40:17.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:17 smithi005 ceph-mon[30365]: Deploying daemon osd.6 on smithi005 2024-05-09T00:40:17.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:17 smithi005 ceph-mon[30365]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T00:40:17.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:17 smithi157 ceph-mon[34294]: Deploying daemon osd.6 on smithi005 2024-05-09T00:40:17.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:17 smithi157 ceph-mon[34294]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T00:40:17.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:17 smithi131 ceph-mon[34547]: Deploying daemon osd.6 on smithi005 2024-05-09T00:40:17.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:17 smithi131 ceph-mon[34547]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T00:40:19.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:19 smithi157 ceph-mon[34294]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T00:40:19.479 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:19 smithi005 ceph-mon[30365]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T00:40:19.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:19 smithi131 ceph-mon[34547]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T00:40:21.259 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:21 smithi005 ceph-mon[30365]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-09T00:40:21.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:21 smithi157 ceph-mon[34294]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-09T00:40:21.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:21 smithi131 ceph-mon[34547]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-09T00:40:22.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:22 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:22.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:22 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:22.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:22 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:40:22.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:22 smithi005 ceph-mon[30365]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-09T00:40:22.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:22 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:22.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:22 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:22.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:22 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:40:22.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:22 smithi131 ceph-mon[34547]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-09T00:40:22.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:22 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:22.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:22 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:22.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:22 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:40:22.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:22 smithi157 ceph-mon[34294]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-09T00:40:25.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:24 smithi131 ceph-mon[34547]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-09T00:40:25.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:24 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:25.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:24 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:25.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:24 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:40:25.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:24 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:40:25.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:24 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:25.159 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:24 smithi005 ceph-mon[30365]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-09T00:40:25.160 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:24 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:25.160 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:24 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:25.160 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:24 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:40:25.160 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:24 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:40:25.160 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:24 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:25.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:24 smithi157 ceph-mon[34294]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-09T00:40:25.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:24 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:25.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:24 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:25.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:24 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:40:25.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:24 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:40:25.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:24 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:25.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:40:25.511+0000 7f7c28ff9700 1 -- 172.21.15.5:0/3137140453 <== mgr.14152 v2:172.21.15.5:6800/91856593 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7c2c105b70 con 0x7f7c180b2880 2024-05-09T00:40:25.513 INFO:teuthology.orchestra.run.smithi005.stdout:Created osd(s) 6 on host 'smithi005' 2024-05-09T00:40:25.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:40:25.513+0000 7f7c315c7700 1 -- 172.21.15.5:0/3137140453 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f7c180b2880 msgr2=0x7f7c180b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:40:25.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:40:25.513+0000 7f7c315c7700 1 --2- 172.21.15.5:0/3137140453 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f7c180b2880 0x7f7c180b4d90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c102c60 tx=0x7f7c14009260 comp rx=0 tx=0).stop 2024-05-09T00:40:25.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:40:25.513+0000 7f7c315c7700 1 -- 172.21.15.5:0/3137140453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c2c103b10 msgr2=0x7f7c2c1032b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:40:25.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:40:25.513+0000 7f7c315c7700 1 --2- 172.21.15.5:0/3137140453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c2c103b10 0x7f7c2c1032b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c00b6c0 tx=0x7f7c1c009580 comp rx=0 tx=0).stop 2024-05-09T00:40:25.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:40:25.513+0000 7f7c315c7700 1 -- 172.21.15.5:0/3137140453 shutdown_connections 2024-05-09T00:40:25.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:40:25.513+0000 7f7c315c7700 1 --2- 172.21.15.5:0/3137140453 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f7c180b2880 0x7f7c180b4d90 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:40:25.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:40:25.513+0000 7f7c315c7700 1 --2- 172.21.15.5:0/3137140453 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c2c108f00 0x7f7c2c1a1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:40:25.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:40:25.513+0000 7f7c315c7700 1 --2- 172.21.15.5:0/3137140453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c2c103b10 0x7f7c2c1032b0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:40:25.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:40:25.513+0000 7f7c315c7700 1 --2- 172.21.15.5:0/3137140453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c2c106470 0x7f7c2c19f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:40:25.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:40:25.513+0000 7f7c315c7700 1 -- 172.21.15.5:0/3137140453 >> 172.21.15.5:0/3137140453 conn(0x7f7c2c0fdbd0 msgr2=0x7f7c2c10b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:40:25.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:40:25.514+0000 7f7c315c7700 1 -- 172.21.15.5:0/3137140453 shutdown_connections 2024-05-09T00:40:25.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:40:25.514+0000 7f7c315c7700 1 -- 172.21.15.5:0/3137140453 wait complete. 2024-05-09T00:40:26.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:25 smithi005 ceph-mon[30365]: from='osd.6 [v2:172.21.15.5:6818/2930397813,v1:172.21.15.5:6819/2930397813]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-09T00:40:26.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:25 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:26.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:25 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:40:26.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:25 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:26.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:25 smithi131 ceph-mon[34547]: from='osd.6 [v2:172.21.15.5:6818/2930397813,v1:172.21.15.5:6819/2930397813]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-09T00:40:26.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:25 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:26.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:25 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:40:26.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:25 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:26.130 DEBUG:teuthology.orchestra.run.smithi005:osd.6> sudo journalctl -f -n 0 -u ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@osd.6.service 2024-05-09T00:40:26.133 INFO:tasks.cephadm:Deploying osd.7 on smithi131 with /dev/vg_nvme/lv_2... 2024-05-09T00:40:26.133 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-09T00:40:26.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:25 smithi157 ceph-mon[34294]: from='osd.6 [v2:172.21.15.5:6818/2930397813,v1:172.21.15.5:6819/2930397813]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-09T00:40:26.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:25 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:26.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:25 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:40:26.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:25 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:26.379 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T00:40:27.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:26 smithi131 ceph-mon[34547]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-09T00:40:27.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:26 smithi131 ceph-mon[34547]: from='osd.6 [v2:172.21.15.5:6818/2930397813,v1:172.21.15.5:6819/2930397813]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-09T00:40:27.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:26 smithi131 ceph-mon[34547]: osdmap e40: 7 total, 6 up, 7 in 2024-05-09T00:40:27.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:26 smithi131 ceph-mon[34547]: from='osd.6 [v2:172.21.15.5:6818/2930397813,v1:172.21.15.5:6819/2930397813]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-05-09T00:40:27.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:26 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T00:40:27.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:26 smithi157 ceph-mon[34294]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-09T00:40:27.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:26 smithi157 ceph-mon[34294]: from='osd.6 [v2:172.21.15.5:6818/2930397813,v1:172.21.15.5:6819/2930397813]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-09T00:40:27.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:26 smithi157 ceph-mon[34294]: osdmap e40: 7 total, 6 up, 7 in 2024-05-09T00:40:27.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:26 smithi157 ceph-mon[34294]: from='osd.6 [v2:172.21.15.5:6818/2930397813,v1:172.21.15.5:6819/2930397813]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-05-09T00:40:27.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:26 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T00:40:27.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:26 smithi005 ceph-mon[30365]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-09T00:40:27.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:26 smithi005 ceph-mon[30365]: from='osd.6 [v2:172.21.15.5:6818/2930397813,v1:172.21.15.5:6819/2930397813]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-09T00:40:27.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:26 smithi005 ceph-mon[30365]: osdmap e40: 7 total, 6 up, 7 in 2024-05-09T00:40:27.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:26 smithi005 ceph-mon[30365]: from='osd.6 [v2:172.21.15.5:6818/2930397813,v1:172.21.15.5:6819/2930397813]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-05-09T00:40:27.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:26 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T00:40:27.293 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 00:40:26 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6[58994]: 2024-05-09T00:40:26.861+0000 7f5ac0134700 -1 osd.6 0 waiting for initial osdmap 2024-05-09T00:40:27.293 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 00:40:26 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6[58994]: 2024-05-09T00:40:26.872+0000 7f5abbee9700 -1 osd.6 41 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T00:40:27.871 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:27 smithi005 ceph-mon[30365]: from='osd.6 [v2:172.21.15.5:6818/2930397813,v1:172.21.15.5:6819/2930397813]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-05-09T00:40:27.871 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:27 smithi005 ceph-mon[30365]: osdmap e41: 7 total, 6 up, 7 in 2024-05-09T00:40:27.871 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:27 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T00:40:27.871 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:27 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T00:40:28.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:27 smithi157 ceph-mon[34294]: from='osd.6 [v2:172.21.15.5:6818/2930397813,v1:172.21.15.5:6819/2930397813]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-05-09T00:40:28.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:27 smithi157 ceph-mon[34294]: osdmap e41: 7 total, 6 up, 7 in 2024-05-09T00:40:28.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:27 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T00:40:28.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:27 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T00:40:28.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:27 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T00:40:28.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:27 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T00:40:28.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:27 smithi131 ceph-mon[34547]: from='osd.6 [v2:172.21.15.5:6818/2930397813,v1:172.21.15.5:6819/2930397813]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-05-09T00:40:28.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:27 smithi131 ceph-mon[34547]: osdmap e41: 7 total, 6 up, 7 in 2024-05-09T00:40:28.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:27 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T00:40:28.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:27 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T00:40:28.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:27 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T00:40:28.409 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-09T00:40:28.410 INFO:teuthology.orchestra.run.smithi131.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-09T00:40:28.410 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 10+0 records in 2024-05-09T00:40:28.410 INFO:teuthology.orchestra.run.smithi131.stderr:10+0 records out 2024-05-09T00:40:28.410 INFO:teuthology.orchestra.run.smithi131.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112963 s, 928 MB/s 2024-05-09T00:40:28.410 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping successful for: 2024-05-09T00:40:29.025 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:28 smithi131 ceph-mon[34547]: purged_snaps scrub starts 2024-05-09T00:40:29.025 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:28 smithi131 ceph-mon[34547]: purged_snaps scrub ok 2024-05-09T00:40:29.025 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:28 smithi131 ceph-mon[34547]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-09T00:40:29.025 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:28 smithi131 ceph-mon[34547]: osd.6 [v2:172.21.15.5:6818/2930397813,v1:172.21.15.5:6819/2930397813] boot 2024-05-09T00:40:29.025 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:28 smithi131 ceph-mon[34547]: osdmap e42: 7 total, 7 up, 7 in 2024-05-09T00:40:29.025 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:28 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T00:40:29.056 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi131:vg_nvme/lv_2 2024-05-09T00:40:29.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:28 smithi157 ceph-mon[34294]: purged_snaps scrub starts 2024-05-09T00:40:29.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:28 smithi157 ceph-mon[34294]: purged_snaps scrub ok 2024-05-09T00:40:29.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:28 smithi157 ceph-mon[34294]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-09T00:40:29.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:28 smithi157 ceph-mon[34294]: osd.6 [v2:172.21.15.5:6818/2930397813,v1:172.21.15.5:6819/2930397813] boot 2024-05-09T00:40:29.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:28 smithi157 ceph-mon[34294]: osdmap e42: 7 total, 7 up, 7 in 2024-05-09T00:40:29.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:28 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T00:40:29.227 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:28 smithi005 ceph-mon[30365]: purged_snaps scrub starts 2024-05-09T00:40:29.227 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:28 smithi005 ceph-mon[30365]: purged_snaps scrub ok 2024-05-09T00:40:29.227 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:28 smithi005 ceph-mon[30365]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-09T00:40:29.227 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:28 smithi005 ceph-mon[30365]: osd.6 [v2:172.21.15.5:6818/2930397813,v1:172.21.15.5:6819/2930397813] boot 2024-05-09T00:40:29.227 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:28 smithi005 ceph-mon[30365]: osdmap e42: 7 total, 7 up, 7 in 2024-05-09T00:40:29.227 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:28 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T00:40:29.299 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T00:40:29.989 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:29 smithi131 ceph-mon[34547]: osdmap e43: 7 total, 7 up, 7 in 2024-05-09T00:40:29.989 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:29 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:29.990 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:29 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:29.990 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:29 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:40:29.990 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:29 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:40:29.990 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:29 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:40:29.990 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:29 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:29.990 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:29 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:40:29.990 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:29 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:40:29.990 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:29 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:30.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:29 smithi157 ceph-mon[34294]: osdmap e43: 7 total, 7 up, 7 in 2024-05-09T00:40:30.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:29 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:30.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:29 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:30.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:29 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:40:30.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:29 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:40:30.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:29 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:40:30.188 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:29 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:30.188 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:29 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:40:30.188 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:29 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:40:30.188 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:29 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:30.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:29 smithi005 ceph-mon[30365]: osdmap e43: 7 total, 7 up, 7 in 2024-05-09T00:40:30.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:29 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:30.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:29 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:30.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:29 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:40:30.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:29 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:40:30.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:29 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:40:30.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:29 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:30.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:29 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:40:30.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:29 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:40:30.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:29 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:30.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:40:30.797+0000 7f57d67d7700 1 -- 172.21.15.131:0/3033513504 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f57d0104390 msgr2=0x7f57d01047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:40:30.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:40:30.797+0000 7f57d67d7700 1 --2- 172.21.15.131:0/3033513504 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f57d0104390 0x7f57d01047a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f57c4009990 tx=0x7f57c4009ca0 comp rx=0 tx=0).stop 2024-05-09T00:40:30.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:40:30.797+0000 7f57d67d7700 1 -- 172.21.15.131:0/3033513504 shutdown_connections 2024-05-09T00:40:30.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:40:30.797+0000 7f57d67d7700 1 --2- 172.21.15.131:0/3033513504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57d0104d30 0x7f57d010b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:40:30.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:40:30.797+0000 7f57d67d7700 1 --2- 172.21.15.131:0/3033513504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57d0103260 0x7f57d0103670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:40:30.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:40:30.797+0000 7f57d67d7700 1 --2- 172.21.15.131:0/3033513504 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f57d0104390 0x7f57d01047a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:40:30.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:40:30.797+0000 7f57d67d7700 1 -- 172.21.15.131:0/3033513504 >> 172.21.15.131:0/3033513504 conn(0x7f57d00fe8c0 msgr2=0x7f57d0100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:40:30.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:40:30.797+0000 7f57d67d7700 1 -- 172.21.15.131:0/3033513504 shutdown_connections 2024-05-09T00:40:30.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:40:30.797+0000 7f57d67d7700 1 -- 172.21.15.131:0/3033513504 wait complete. 2024-05-09T00:40:30.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:40:30.798+0000 7f57d67d7700 1 Processor -- start 2024-05-09T00:40:30.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:40:30.798+0000 7f57d67d7700 1 -- start start 2024-05-09T00:40:30.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:40:30.798+0000 7f57d67d7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57d0103260 0x7f57d01997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:40:30.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:40:30.798+0000 7f57d67d7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f57d0104390 0x7f57d0199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:40:30.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:40:30.798+0000 7f57d67d7700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57d0104d30 0x7f57d01a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:40:30.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:40:30.798+0000 7f57d67d7700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f57d010ecc0 con 0x7f57d0103260 2024-05-09T00:40:30.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:40:30.798+0000 7f57d67d7700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f57d010eb40 con 0x7f57d0104390 2024-05-09T00:40:30.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:40:30.798+0000 7f57d67d7700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f57d010ee40 con 0x7f57d0104d30 2024-05-09T00:40:30.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:40:30.798+0000 7f57cf7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f57d0104390 0x7f57d0199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:40:30.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:40:30.798+0000 7f57cf7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f57d0104390 0x7f57d0199d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44866/0 (socket says 172.21.15.131:44866) 2024-05-09T00:40:30.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:40:30.798+0000 7f57cf7fe700 1 -- 172.21.15.131:0/561020811 learned_addr learned my addr 172.21.15.131:0/561020811 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-09T00:40:30.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:40:30.798+0000 7f57d4d74700 1 --2- 172.21.15.131:0/561020811 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57d0104d30 0x7f57d01a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:40:30.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:40:30.798+0000 7f57cffff700 1 --2- 172.21.15.131:0/561020811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57d0103260 0x7f57d01997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:40:30.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:40:30.799+0000 7f57cf7fe700 1 -- 172.21.15.131:0/561020811 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57d0104d30 msgr2=0x7f57d01a0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:40:30.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:40:30.799+0000 7f57cf7fe700 1 --2- 172.21.15.131:0/561020811 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57d0104d30 0x7f57d01a0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:40:30.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:40:30.799+0000 7f57cf7fe700 1 -- 172.21.15.131:0/561020811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57d0103260 msgr2=0x7f57d01997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:40:30.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:40:30.799+0000 7f57cf7fe700 1 --2- 172.21.15.131:0/561020811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57d0103260 0x7f57d01997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:40:30.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:40:30.799+0000 7f57cf7fe700 1 -- 172.21.15.131:0/561020811 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57d01a1340 con 0x7f57d0104390 2024-05-09T00:40:30.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:40:30.799+0000 7f57cf7fe700 1 --2- 172.21.15.131:0/561020811 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f57d0104390 0x7f57d0199d20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f57c4006010 tx=0x7f57c40047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:40:30.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:40:30.799+0000 7f57cd7fa700 1 -- 172.21.15.131:0/561020811 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c4027070 con 0x7f57d0104390 2024-05-09T00:40:30.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:40:30.799+0000 7f57cd7fa700 1 -- 172.21.15.131:0/561020811 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57c4004000 con 0x7f57d0104390 2024-05-09T00:40:30.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:40:30.799+0000 7f57d67d7700 1 -- 172.21.15.131:0/561020811 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57d01a15d0 con 0x7f57d0104390 2024-05-09T00:40:30.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:40:30.799+0000 7f57cd7fa700 1 -- 172.21.15.131:0/561020811 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c4022d30 con 0x7f57d0104390 2024-05-09T00:40:30.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:40:30.799+0000 7f57d67d7700 1 -- 172.21.15.131:0/561020811 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57d01a1a20 con 0x7f57d0104390 2024-05-09T00:40:30.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:40:30.800+0000 7f57d67d7700 1 -- 172.21.15.131:0/561020811 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57d00512c0 con 0x7f57d0104390 2024-05-09T00:40:30.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:40:30.802+0000 7f57cd7fa700 1 -- 172.21.15.131:0/561020811 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f57c402a070 con 0x7f57d0104390 2024-05-09T00:40:30.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:40:30.802+0000 7f57cd7fa700 1 --2- 172.21.15.131:0/561020811 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f57b80b2880 0x7f57b80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:40:30.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:40:30.802+0000 7f57cd7fa700 1 -- 172.21.15.131:0/561020811 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4965+0+0 (secure 0 0 0) 0x7f57c40e25c0 con 0x7f57d0104390 2024-05-09T00:40:30.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:40:30.803+0000 7f57cffff700 1 --2- 172.21.15.131:0/561020811 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f57b80b2880 0x7f57b80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:40:30.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:40:30.803+0000 7f57cffff700 1 --2- 172.21.15.131:0/561020811 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f57b80b2880 0x7f57b80b4d90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f57d0104290 tx=0x7f57c000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:40:30.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:40:30.806+0000 7f57cd7fa700 1 -- 172.21.15.131:0/561020811 <== mon.2 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f57c4013070 con 0x7f57d0104390 2024-05-09T00:40:31.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:40:31.017+0000 7f57d67d7700 1 -- 172.21.15.131:0/561020811 --> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f57d0063c00 con 0x7f57b80b2880 2024-05-09T00:40:31.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:30 smithi157 ceph-mon[34294]: Detected new or changed devices on smithi005 2024-05-09T00:40:31.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:30 smithi157 ceph-mon[34294]: Adjusting osd_memory_target on smithi005 to 5697M 2024-05-09T00:40:31.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:30 smithi157 ceph-mon[34294]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:40:31.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:30 smithi005 ceph-mon[30365]: Detected new or changed devices on smithi005 2024-05-09T00:40:31.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:30 smithi005 ceph-mon[30365]: Adjusting osd_memory_target on smithi005 to 5697M 2024-05-09T00:40:31.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:30 smithi005 ceph-mon[30365]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:40:31.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:30 smithi131 ceph-mon[34547]: Detected new or changed devices on smithi005 2024-05-09T00:40:31.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:30 smithi131 ceph-mon[34547]: Adjusting osd_memory_target on smithi005 to 5697M 2024-05-09T00:40:31.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:30 smithi131 ceph-mon[34547]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:40:32.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:31 smithi157 ceph-mon[34294]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:40:32.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:31 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:40:32.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:31 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:40:32.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:31 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:40:32.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:31 smithi005 ceph-mon[30365]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:40:32.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:31 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:40:32.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:31 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:40:32.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:31 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:40:32.300 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:31 smithi131 ceph-mon[34547]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:40:32.300 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:31 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:40:32.300 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:31 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:40:32.300 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:31 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:40:33.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:32 smithi157 ceph-mon[34294]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:40:33.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:32 smithi005 ceph-mon[30365]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:40:33.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:32 smithi131 ceph-mon[34547]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:40:34.147 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:33 smithi131 ceph-mon[34547]: from='client.? 172.21.15.131:0/3931618488' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c0a0573-6465-4bf6-b70f-310d49857a6f"}]: dispatch 2024-05-09T00:40:34.147 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:33 smithi131 ceph-mon[34547]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c0a0573-6465-4bf6-b70f-310d49857a6f"}]: dispatch 2024-05-09T00:40:34.147 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:33 smithi131 ceph-mon[34547]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3c0a0573-6465-4bf6-b70f-310d49857a6f"}]': finished 2024-05-09T00:40:34.147 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:33 smithi131 ceph-mon[34547]: osdmap e44: 8 total, 7 up, 8 in 2024-05-09T00:40:34.147 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:33 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T00:40:34.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:33 smithi157 ceph-mon[34294]: from='client.? 172.21.15.131:0/3931618488' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c0a0573-6465-4bf6-b70f-310d49857a6f"}]: dispatch 2024-05-09T00:40:34.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:33 smithi157 ceph-mon[34294]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c0a0573-6465-4bf6-b70f-310d49857a6f"}]: dispatch 2024-05-09T00:40:34.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:33 smithi157 ceph-mon[34294]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3c0a0573-6465-4bf6-b70f-310d49857a6f"}]': finished 2024-05-09T00:40:34.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:33 smithi157 ceph-mon[34294]: osdmap e44: 8 total, 7 up, 8 in 2024-05-09T00:40:34.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:33 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T00:40:34.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:33 smithi005 ceph-mon[30365]: from='client.? 172.21.15.131:0/3931618488' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c0a0573-6465-4bf6-b70f-310d49857a6f"}]: dispatch 2024-05-09T00:40:34.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:33 smithi005 ceph-mon[30365]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c0a0573-6465-4bf6-b70f-310d49857a6f"}]: dispatch 2024-05-09T00:40:34.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:33 smithi005 ceph-mon[30365]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3c0a0573-6465-4bf6-b70f-310d49857a6f"}]': finished 2024-05-09T00:40:34.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:33 smithi005 ceph-mon[30365]: osdmap e44: 8 total, 7 up, 8 in 2024-05-09T00:40:34.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:33 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T00:40:35.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:34 smithi157 ceph-mon[34294]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:40:35.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:34 smithi157 ceph-mon[34294]: from='client.? 172.21.15.131:0/2547714539' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-7/activate.monmap"}]: dispatch 2024-05-09T00:40:35.192 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:34 smithi005 ceph-mon[30365]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:40:35.193 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:34 smithi005 ceph-mon[30365]: from='client.? 172.21.15.131:0/2547714539' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-7/activate.monmap"}]: dispatch 2024-05-09T00:40:35.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:34 smithi131 ceph-mon[34547]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:40:35.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:34 smithi131 ceph-mon[34547]: from='client.? 172.21.15.131:0/2547714539' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-7/activate.monmap"}]: dispatch 2024-05-09T00:40:37.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:36 smithi157 ceph-mon[34294]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:40:37.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:36 smithi005 ceph-mon[30365]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:40:37.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:36 smithi131 ceph-mon[34547]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:40:39.190 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:38 smithi157 ceph-mon[34294]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:40:39.229 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:38 smithi131 ceph-mon[34547]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:40:39.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:38 smithi005 ceph-mon[30365]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:40:41.182 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:40 smithi131 ceph-mon[34547]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:40:41.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:40 smithi157 ceph-mon[34294]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:40:41.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:40 smithi005 ceph-mon[30365]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:40:42.085 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:41 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-09T00:40:42.085 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:41 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:40:42.085 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:41 smithi131 ceph-mon[34547]: Deploying daemon osd.7 on smithi131 2024-05-09T00:40:42.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:41 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-09T00:40:42.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:41 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:40:42.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:41 smithi157 ceph-mon[34294]: Deploying daemon osd.7 on smithi131 2024-05-09T00:40:42.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:41 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-09T00:40:42.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:41 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:40:42.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:41 smithi005 ceph-mon[30365]: Deploying daemon osd.7 on smithi131 2024-05-09T00:40:43.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:42 smithi131 ceph-mon[34547]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:40:43.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:42 smithi157 ceph-mon[34294]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:40:43.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:42 smithi005 ceph-mon[30365]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:40:45.159 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:44 smithi131 ceph-mon[34547]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:40:45.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:44 smithi157 ceph-mon[34294]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:40:45.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:44 smithi005 ceph-mon[30365]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:40:47.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:46 smithi157 ceph-mon[34294]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:40:47.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:46 smithi005 ceph-mon[30365]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:40:47.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:46 smithi131 ceph-mon[34547]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:40:48.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:48 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:48.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:48 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:48.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:48 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:40:48.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:48 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:48.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:48 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:48.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:48 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:40:48.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:48 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:48.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:48 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:48.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:48 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:40:49.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:49 smithi131 ceph-mon[34547]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:40:49.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:49 smithi005 ceph-mon[30365]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:40:49.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:49 smithi157 ceph-mon[34294]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:40:50.939 INFO:teuthology.orchestra.run.smithi131.stdout:Created osd(s) 7 on host 'smithi131' 2024-05-09T00:40:50.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:40:50.937+0000 7f57cd7fa700 1 -- 172.21.15.131:0/561020811 <== mgr.14152 v2:172.21.15.5:6800/91856593 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f57d0063c00 con 0x7f57b80b2880 2024-05-09T00:40:50.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:40:50.939+0000 7f57d67d7700 1 -- 172.21.15.131:0/561020811 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f57b80b2880 msgr2=0x7f57b80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:40:50.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:40:50.939+0000 7f57d67d7700 1 --2- 172.21.15.131:0/561020811 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f57b80b2880 0x7f57b80b4d90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f57d0104290 tx=0x7f57c000b410 comp rx=0 tx=0).stop 2024-05-09T00:40:50.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:40:50.939+0000 7f57d67d7700 1 -- 172.21.15.131:0/561020811 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f57d0104390 msgr2=0x7f57d0199d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:40:50.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:40:50.939+0000 7f57d67d7700 1 --2- 172.21.15.131:0/561020811 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f57d0104390 0x7f57d0199d20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f57c4006010 tx=0x7f57c40047b0 comp rx=0 tx=0).stop 2024-05-09T00:40:50.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:40:50.940+0000 7f57d67d7700 1 -- 172.21.15.131:0/561020811 shutdown_connections 2024-05-09T00:40:50.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:40:50.940+0000 7f57d67d7700 1 --2- 172.21.15.131:0/561020811 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f57b80b2880 0x7f57b80b4d90 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:40:50.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:40:50.940+0000 7f57d67d7700 1 --2- 172.21.15.131:0/561020811 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57d0104d30 0x7f57d01a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:40:50.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:40:50.940+0000 7f57d67d7700 1 --2- 172.21.15.131:0/561020811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57d0103260 0x7f57d01997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:40:50.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:40:50.940+0000 7f57d67d7700 1 --2- 172.21.15.131:0/561020811 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f57d0104390 0x7f57d0199d20 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:40:50.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:40:50.940+0000 7f57d67d7700 1 -- 172.21.15.131:0/561020811 >> 172.21.15.131:0/561020811 conn(0x7f57d00fe8c0 msgr2=0x7f57d01075c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:40:50.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:40:50.940+0000 7f57d67d7700 1 -- 172.21.15.131:0/561020811 shutdown_connections 2024-05-09T00:40:50.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:40:50.940+0000 7f57d67d7700 1 -- 172.21.15.131:0/561020811 wait complete. 2024-05-09T00:40:51.221 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:51 smithi131 ceph-mon[34547]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:40:51.221 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:51 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:51.221 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:51 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:51.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:51 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:40:51.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:51 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:40:51.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:51 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:51.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:51 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:51.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:51 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:40:51.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:51 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:51.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:51 smithi131 ceph-mon[34547]: from='osd.7 [v2:172.21.15.131:6816/4125045219,v1:172.21.15.131:6817/4125045219]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-09T00:40:51.222 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:51 smithi131 ceph-mon[34547]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-09T00:40:51.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:51 smithi005 ceph-mon[30365]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:40:51.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:51 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:51.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:51 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:51.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:51 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:40:51.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:51 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:40:51.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:51 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:51.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:51 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:51.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:51 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:40:51.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:51 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:51.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:51 smithi005 ceph-mon[30365]: from='osd.7 [v2:172.21.15.131:6816/4125045219,v1:172.21.15.131:6817/4125045219]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-09T00:40:51.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:51 smithi005 ceph-mon[30365]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-09T00:40:51.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:51 smithi157 ceph-mon[34294]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:40:51.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:51 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:51.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:51 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:51.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:51 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:40:51.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:51 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:40:51.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:51 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:51.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:51 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:51.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:51 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:40:51.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:51 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:51.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:51 smithi157 ceph-mon[34294]: from='osd.7 [v2:172.21.15.131:6816/4125045219,v1:172.21.15.131:6817/4125045219]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-09T00:40:51.438 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:51 smithi157 ceph-mon[34294]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-09T00:40:51.559 DEBUG:teuthology.orchestra.run.smithi131:osd.7> sudo journalctl -f -n 0 -u ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@osd.7.service 2024-05-09T00:40:51.562 INFO:tasks.cephadm:Deploying osd.8 on smithi157 with /dev/vg_nvme/lv_2... 2024-05-09T00:40:51.563 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-09T00:40:51.808 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T00:40:53.110 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:52 smithi131 ceph-mon[34547]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:40:53.110 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:52 smithi131 ceph-mon[34547]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-09T00:40:53.110 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:52 smithi131 ceph-mon[34547]: osdmap e45: 8 total, 7 up, 8 in 2024-05-09T00:40:53.110 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:52 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T00:40:53.110 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:52 smithi131 ceph-mon[34547]: from='osd.7 [v2:172.21.15.131:6816/4125045219,v1:172.21.15.131:6817/4125045219]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-05-09T00:40:53.110 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:52 smithi131 ceph-mon[34547]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-05-09T00:40:53.111 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 00:40:52 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7[52359]: 2024-05-09T00:40:52.912+0000 7f4f9bd37700 -1 osd.7 0 waiting for initial osdmap 2024-05-09T00:40:53.111 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 00:40:52 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7[52359]: 2024-05-09T00:40:52.921+0000 7f4f97aec700 -1 osd.7 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T00:40:53.188 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:52 smithi157 ceph-mon[34294]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:40:53.188 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:52 smithi157 ceph-mon[34294]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-09T00:40:53.188 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:52 smithi157 ceph-mon[34294]: osdmap e45: 8 total, 7 up, 8 in 2024-05-09T00:40:53.188 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:52 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T00:40:53.188 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:52 smithi157 ceph-mon[34294]: from='osd.7 [v2:172.21.15.131:6816/4125045219,v1:172.21.15.131:6817/4125045219]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-05-09T00:40:53.188 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:52 smithi157 ceph-mon[34294]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-05-09T00:40:53.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:52 smithi005 ceph-mon[30365]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:40:53.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:52 smithi005 ceph-mon[30365]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-09T00:40:53.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:52 smithi005 ceph-mon[30365]: osdmap e45: 8 total, 7 up, 8 in 2024-05-09T00:40:53.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:52 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T00:40:53.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:52 smithi005 ceph-mon[30365]: from='osd.7 [v2:172.21.15.131:6816/4125045219,v1:172.21.15.131:6817/4125045219]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-05-09T00:40:53.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:52 smithi005 ceph-mon[30365]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-05-09T00:40:53.786 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-09T00:40:53.787 INFO:teuthology.orchestra.run.smithi157.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-09T00:40:53.787 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10+0 records in 2024-05-09T00:40:53.787 INFO:teuthology.orchestra.run.smithi157.stderr:10+0 records out 2024-05-09T00:40:53.787 INFO:teuthology.orchestra.run.smithi157.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00873202 s, 1.2 GB/s 2024-05-09T00:40:53.787 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 2024-05-09T00:40:53.787 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping successful for: 2024-05-09T00:40:53.998 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:53 smithi131 ceph-mon[34547]: purged_snaps scrub starts 2024-05-09T00:40:53.998 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:53 smithi131 ceph-mon[34547]: purged_snaps scrub ok 2024-05-09T00:40:53.998 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:53 smithi131 ceph-mon[34547]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-05-09T00:40:53.999 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:53 smithi131 ceph-mon[34547]: osdmap e46: 8 total, 7 up, 8 in 2024-05-09T00:40:53.999 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:53 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T00:40:53.999 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:53 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T00:40:54.149 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:53 smithi157 ceph-mon[34294]: purged_snaps scrub starts 2024-05-09T00:40:54.149 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:53 smithi157 ceph-mon[34294]: purged_snaps scrub ok 2024-05-09T00:40:54.149 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:53 smithi157 ceph-mon[34294]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-05-09T00:40:54.149 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:53 smithi157 ceph-mon[34294]: osdmap e46: 8 total, 7 up, 8 in 2024-05-09T00:40:54.149 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:53 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T00:40:54.149 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:53 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T00:40:54.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:53 smithi005 ceph-mon[30365]: purged_snaps scrub starts 2024-05-09T00:40:54.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:53 smithi005 ceph-mon[30365]: purged_snaps scrub ok 2024-05-09T00:40:54.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:53 smithi005 ceph-mon[30365]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-05-09T00:40:54.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:53 smithi005 ceph-mon[30365]: osdmap e46: 8 total, 7 up, 8 in 2024-05-09T00:40:54.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:53 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T00:40:54.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:53 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T00:40:54.479 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi157:vg_nvme/lv_2 2024-05-09T00:40:54.725 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T00:40:55.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:54 smithi157 ceph-mon[34294]: pgmap v172: 1 pgs: 1 unknown; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:40:55.053 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:54 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T00:40:55.054 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:54 smithi157 ceph-mon[34294]: osd.7 [v2:172.21.15.131:6816/4125045219,v1:172.21.15.131:6817/4125045219] boot 2024-05-09T00:40:55.054 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:54 smithi157 ceph-mon[34294]: osdmap e47: 8 total, 8 up, 8 in 2024-05-09T00:40:55.054 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:54 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T00:40:55.054 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:54 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:55.054 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:54 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:55.054 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:54 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:40:55.054 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:54 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:40:55.054 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:54 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:40:55.054 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:54 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:55.054 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:54 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:40:55.054 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:54 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:40:55.054 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:54 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:55.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:54 smithi005 ceph-mon[30365]: pgmap v172: 1 pgs: 1 unknown; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:40:55.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:54 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T00:40:55.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:54 smithi005 ceph-mon[30365]: osd.7 [v2:172.21.15.131:6816/4125045219,v1:172.21.15.131:6817/4125045219] boot 2024-05-09T00:40:55.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:54 smithi005 ceph-mon[30365]: osdmap e47: 8 total, 8 up, 8 in 2024-05-09T00:40:55.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:54 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T00:40:55.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:54 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:55.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:54 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:55.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:54 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:40:55.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:54 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:40:55.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:54 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:40:55.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:54 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:55.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:54 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:40:55.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:54 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:40:55.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:54 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:55.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:54 smithi131 ceph-mon[34547]: pgmap v172: 1 pgs: 1 unknown; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:40:55.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:54 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T00:40:55.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:54 smithi131 ceph-mon[34547]: osd.7 [v2:172.21.15.131:6816/4125045219,v1:172.21.15.131:6817/4125045219] boot 2024-05-09T00:40:55.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:54 smithi131 ceph-mon[34547]: osdmap e47: 8 total, 8 up, 8 in 2024-05-09T00:40:55.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:54 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T00:40:55.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:54 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:55.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:54 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:55.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:54 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:40:55.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:54 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:40:55.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:54 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:40:55.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:54 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:55.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:54 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:40:55.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:54 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:40:55.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:54 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:40:56.100 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:55 smithi157 ceph-mon[34294]: Detected new or changed devices on smithi131 2024-05-09T00:40:56.100 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:55 smithi157 ceph-mon[34294]: Adjusting osd_memory_target on smithi131 to 5696M 2024-05-09T00:40:56.100 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:55 smithi157 ceph-mon[34294]: osdmap e48: 8 total, 8 up, 8 in 2024-05-09T00:40:56.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:55 smithi005 ceph-mon[30365]: Detected new or changed devices on smithi131 2024-05-09T00:40:56.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:55 smithi005 ceph-mon[30365]: Adjusting osd_memory_target on smithi131 to 5696M 2024-05-09T00:40:56.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:55 smithi005 ceph-mon[30365]: osdmap e48: 8 total, 8 up, 8 in 2024-05-09T00:40:56.318 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:40:56.313+0000 7efc585fa700 1 -- 172.21.15.157:0/365644720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc50104340 msgr2=0x7efc50104750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:40:56.318 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:40:56.313+0000 7efc585fa700 1 --2- 172.21.15.157:0/365644720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc50104340 0x7efc50104750 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7efc4c00b0a0 tx=0x7efc4c00b3b0 comp rx=0 tx=0).stop 2024-05-09T00:40:56.318 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:40:56.313+0000 7efc585fa700 1 -- 172.21.15.157:0/365644720 shutdown_connections 2024-05-09T00:40:56.318 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:40:56.313+0000 7efc585fa700 1 --2- 172.21.15.157:0/365644720 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efc50105e10 0x7efc5010c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:40:56.318 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:40:56.313+0000 7efc585fa700 1 --2- 172.21.15.157:0/365644720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc50104340 0x7efc50104750 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:40:56.318 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:40:56.313+0000 7efc585fa700 1 --2- 172.21.15.157:0/365644720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc50105470 0x7efc50105880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:40:56.318 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:40:56.313+0000 7efc585fa700 1 -- 172.21.15.157:0/365644720 >> 172.21.15.157:0/365644720 conn(0x7efc500ff9c0 msgr2=0x7efc50101e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:40:56.319 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:40:56.313+0000 7efc585fa700 1 -- 172.21.15.157:0/365644720 shutdown_connections 2024-05-09T00:40:56.319 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:40:56.313+0000 7efc585fa700 1 -- 172.21.15.157:0/365644720 wait complete. 2024-05-09T00:40:56.319 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:40:56.314+0000 7efc585fa700 1 Processor -- start 2024-05-09T00:40:56.319 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:40:56.314+0000 7efc585fa700 1 -- start start 2024-05-09T00:40:56.319 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:40:56.314+0000 7efc585fa700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc50104340 0x7efc5019a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:40:56.319 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:40:56.314+0000 7efc585fa700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc50105470 0x7efc5019ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:40:56.319 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:40:56.314+0000 7efc585fa700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efc50105e10 0x7efc501a1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:40:56.319 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:40:56.314+0000 7efc585fa700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efc5010fdb0 con 0x7efc50105470 2024-05-09T00:40:56.319 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:40:56.314+0000 7efc585fa700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7efc5010fc30 con 0x7efc50105e10 2024-05-09T00:40:56.319 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:40:56.314+0000 7efc585fa700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7efc5010ff30 con 0x7efc50104340 2024-05-09T00:40:56.319 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:40:56.315+0000 7efc56396700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc50104340 0x7efc5019a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:40:56.319 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:40:56.315+0000 7efc56396700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc50104340 0x7efc5019a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:57662/0 (socket says 172.21.15.157:57662) 2024-05-09T00:40:56.319 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:40:56.315+0000 7efc56396700 1 -- 172.21.15.157:0/3336848150 learned_addr learned my addr 172.21.15.157:0/3336848150 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-05-09T00:40:56.320 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:40:56.315+0000 7efc56b97700 1 --2- 172.21.15.157:0/3336848150 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efc50105e10 0x7efc501a1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:40:56.320 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:40:56.315+0000 7efc55b95700 1 --2- 172.21.15.157:0/3336848150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc50105470 0x7efc5019ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:40:56.320 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:40:56.315+0000 7efc56396700 1 -- 172.21.15.157:0/3336848150 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efc50105e10 msgr2=0x7efc501a1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:40:56.320 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:40:56.315+0000 7efc56396700 1 --2- 172.21.15.157:0/3336848150 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efc50105e10 0x7efc501a1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:40:56.320 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:40:56.315+0000 7efc56396700 1 -- 172.21.15.157:0/3336848150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc50105470 msgr2=0x7efc5019ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:40:56.320 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:40:56.315+0000 7efc56396700 1 --2- 172.21.15.157:0/3336848150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc50105470 0x7efc5019ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:40:56.320 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:40:56.315+0000 7efc56396700 1 -- 172.21.15.157:0/3336848150 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc501a2480 con 0x7efc50104340 2024-05-09T00:40:56.320 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:40:56.315+0000 7efc55b95700 1 --2- 172.21.15.157:0/3336848150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc50105470 0x7efc5019ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:40:56.320 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:40:56.315+0000 7efc56396700 1 --2- 172.21.15.157:0/3336848150 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc50104340 0x7efc5019a920 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7efc4c00b780 tx=0x7efc4c004d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:40:56.320 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:40:56.315+0000 7efc477fe700 1 -- 172.21.15.157:0/3336848150 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc4c021070 con 0x7efc50104340 2024-05-09T00:40:56.320 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:40:56.315+0000 7efc585fa700 1 -- 172.21.15.157:0/3336848150 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc501a2710 con 0x7efc50104340 2024-05-09T00:40:56.320 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:40:56.315+0000 7efc477fe700 1 -- 172.21.15.157:0/3336848150 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc4c007c00 con 0x7efc50104340 2024-05-09T00:40:56.320 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:40:56.315+0000 7efc477fe700 1 -- 172.21.15.157:0/3336848150 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc4c0052c0 con 0x7efc50104340 2024-05-09T00:40:56.320 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:40:56.315+0000 7efc585fa700 1 -- 172.21.15.157:0/3336848150 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc501a2ca0 con 0x7efc50104340 2024-05-09T00:40:56.321 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:40:56.316+0000 7efc585fa700 1 -- 172.21.15.157:0/3336848150 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc500512c0 con 0x7efc50104340 2024-05-09T00:40:56.321 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:40:56.318+0000 7efc477fe700 1 -- 172.21.15.157:0/3336848150 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7efc4c024070 con 0x7efc50104340 2024-05-09T00:40:56.322 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:40:56.318+0000 7efc477fe700 1 --2- 172.21.15.157:0/3336848150 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7efc3c0b2880 0x7efc3c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:40:56.322 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:40:56.319+0000 7efc477fe700 1 -- 172.21.15.157:0/3336848150 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5397+0+0 (secure 0 0 0) 0x7efc4c00f020 con 0x7efc50104340 2024-05-09T00:40:56.322 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:40:56.319+0000 7efc55b95700 1 --2- 172.21.15.157:0/3336848150 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7efc3c0b2880 0x7efc3c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:40:56.322 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:40:56.319+0000 7efc55b95700 1 --2- 172.21.15.157:0/3336848150 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7efc3c0b2880 0x7efc3c0b4d90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7efc5019b9d0 tx=0x7efc4000b690 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:40:56.326 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:40:56.322+0000 7efc477fe700 1 -- 172.21.15.157:0/3336848150 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efc4c02b020 con 0x7efc50104340 2024-05-09T00:40:56.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:55 smithi131 ceph-mon[34547]: Detected new or changed devices on smithi131 2024-05-09T00:40:56.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:55 smithi131 ceph-mon[34547]: Adjusting osd_memory_target on smithi131 to 5696M 2024-05-09T00:40:56.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:55 smithi131 ceph-mon[34547]: osdmap e48: 8 total, 8 up, 8 in 2024-05-09T00:40:56.541 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:40:56.539+0000 7efc585fa700 1 -- 172.21.15.157:0/3336848150 --> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7efc50063c00 con 0x7efc3c0b2880 2024-05-09T00:40:57.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:56 smithi157 ceph-mon[34294]: pgmap v175: 1 pgs: 1 unknown; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-05-09T00:40:57.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:56 smithi157 ceph-mon[34294]: osdmap e49: 8 total, 8 up, 8 in 2024-05-09T00:40:57.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:56 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:40:57.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:56 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:40:57.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:56 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:40:57.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:56 smithi005 ceph-mon[30365]: pgmap v175: 1 pgs: 1 unknown; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-05-09T00:40:57.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:56 smithi005 ceph-mon[30365]: osdmap e49: 8 total, 8 up, 8 in 2024-05-09T00:40:57.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:56 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:40:57.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:56 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:40:57.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:56 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:40:57.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:56 smithi131 ceph-mon[34547]: pgmap v175: 1 pgs: 1 unknown; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-05-09T00:40:57.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:56 smithi131 ceph-mon[34547]: osdmap e49: 8 total, 8 up, 8 in 2024-05-09T00:40:57.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:56 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:40:57.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:56 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:40:57.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:56 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:40:58.198 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:57 smithi157 ceph-mon[34294]: from='client.24322 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:40:58.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:57 smithi005 ceph-mon[30365]: from='client.24322 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:40:58.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:57 smithi131 ceph-mon[34547]: from='client.24322 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:40:59.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:58 smithi005 ceph-mon[30365]: pgmap v177: 1 pgs: 1 unknown; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-05-09T00:40:59.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:58 smithi131 ceph-mon[34547]: pgmap v177: 1 pgs: 1 unknown; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-05-09T00:40:59.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:58 smithi157 ceph-mon[34294]: pgmap v177: 1 pgs: 1 unknown; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-05-09T00:41:00.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:59 smithi157 ceph-mon[34294]: from='client.? 172.21.15.157:0/670001467' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c49f0a00-f5ae-40ed-8f38-7b953ba1c17c"}]: dispatch 2024-05-09T00:41:00.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:59 smithi157 ceph-mon[34294]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c49f0a00-f5ae-40ed-8f38-7b953ba1c17c"}]: dispatch 2024-05-09T00:41:00.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:59 smithi157 ceph-mon[34294]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c49f0a00-f5ae-40ed-8f38-7b953ba1c17c"}]': finished 2024-05-09T00:41:00.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:59 smithi157 ceph-mon[34294]: osdmap e50: 9 total, 8 up, 9 in 2024-05-09T00:41:00.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:40:59 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T00:41:00.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:59 smithi005 ceph-mon[30365]: from='client.? 172.21.15.157:0/670001467' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c49f0a00-f5ae-40ed-8f38-7b953ba1c17c"}]: dispatch 2024-05-09T00:41:00.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:59 smithi005 ceph-mon[30365]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c49f0a00-f5ae-40ed-8f38-7b953ba1c17c"}]: dispatch 2024-05-09T00:41:00.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:59 smithi005 ceph-mon[30365]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c49f0a00-f5ae-40ed-8f38-7b953ba1c17c"}]': finished 2024-05-09T00:41:00.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:59 smithi005 ceph-mon[30365]: osdmap e50: 9 total, 8 up, 9 in 2024-05-09T00:41:00.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:40:59 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T00:41:00.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:59 smithi131 ceph-mon[34547]: from='client.? 172.21.15.157:0/670001467' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c49f0a00-f5ae-40ed-8f38-7b953ba1c17c"}]: dispatch 2024-05-09T00:41:00.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:59 smithi131 ceph-mon[34547]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c49f0a00-f5ae-40ed-8f38-7b953ba1c17c"}]: dispatch 2024-05-09T00:41:00.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:59 smithi131 ceph-mon[34547]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c49f0a00-f5ae-40ed-8f38-7b953ba1c17c"}]': finished 2024-05-09T00:41:00.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:59 smithi131 ceph-mon[34547]: osdmap e50: 9 total, 8 up, 9 in 2024-05-09T00:41:00.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:40:59 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T00:41:01.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:00 smithi005 ceph-mon[30365]: pgmap v179: 1 pgs: 1 unknown; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-05-09T00:41:01.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:00 smithi005 ceph-mon[30365]: from='client.? 172.21.15.157:0/4206594004' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-8/activate.monmap"}]: dispatch 2024-05-09T00:41:01.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:00 smithi131 ceph-mon[34547]: pgmap v179: 1 pgs: 1 unknown; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-05-09T00:41:01.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:00 smithi131 ceph-mon[34547]: from='client.? 172.21.15.157:0/4206594004' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-8/activate.monmap"}]: dispatch 2024-05-09T00:41:01.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:00 smithi157 ceph-mon[34294]: pgmap v179: 1 pgs: 1 unknown; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-05-09T00:41:01.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:00 smithi157 ceph-mon[34294]: from='client.? 172.21.15.157:0/4206594004' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-8/activate.monmap"}]: dispatch 2024-05-09T00:41:03.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:02 smithi157 ceph-mon[34294]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 168 KiB/s, 0 objects/s recovering 2024-05-09T00:41:03.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:02 smithi005 ceph-mon[30365]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 168 KiB/s, 0 objects/s recovering 2024-05-09T00:41:03.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:02 smithi131 ceph-mon[34547]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 168 KiB/s, 0 objects/s recovering 2024-05-09T00:41:05.200 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:04 smithi157 ceph-mon[34294]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-09T00:41:05.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:04 smithi005 ceph-mon[30365]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-09T00:41:05.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:04 smithi131 ceph-mon[34547]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-09T00:41:07.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:06 smithi157 ceph-mon[34294]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-05-09T00:41:07.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:06 smithi005 ceph-mon[30365]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-05-09T00:41:07.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:06 smithi131 ceph-mon[34547]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-05-09T00:41:08.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:07 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-09T00:41:08.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:07 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:41:08.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:07 smithi157 ceph-mon[34294]: Deploying daemon osd.8 on smithi157 2024-05-09T00:41:08.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:07 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-09T00:41:08.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:07 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:41:08.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:07 smithi005 ceph-mon[30365]: Deploying daemon osd.8 on smithi157 2024-05-09T00:41:08.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:07 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-09T00:41:08.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:07 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:41:08.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:07 smithi131 ceph-mon[34547]: Deploying daemon osd.8 on smithi157 2024-05-09T00:41:09.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:08 smithi157 ceph-mon[34294]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-09T00:41:09.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:08 smithi005 ceph-mon[30365]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-09T00:41:09.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:08 smithi131 ceph-mon[34547]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-09T00:41:11.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:10 smithi157 ceph-mon[34294]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-05-09T00:41:11.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:10 smithi005 ceph-mon[30365]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-05-09T00:41:11.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:10 smithi131 ceph-mon[34547]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-05-09T00:41:13.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:12 smithi005 ceph-mon[30365]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-09T00:41:13.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:12 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:41:13.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:12 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:41:13.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:12 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:41:13.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:12 smithi131 ceph-mon[34547]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-09T00:41:13.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:12 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:41:13.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:12 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:41:13.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:12 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:41:13.420 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:12 smithi157 ceph-mon[34294]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-09T00:41:13.420 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:12 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:41:13.420 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:12 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:41:13.420 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:12 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:41:15.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:14 smithi157 ceph-mon[34294]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-05-09T00:41:15.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:14 smithi005 ceph-mon[30365]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-05-09T00:41:15.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:14 smithi131 ceph-mon[34547]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-05-09T00:41:16.617 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:41:16.615+0000 7efc477fe700 1 -- 172.21.15.157:0/3336848150 <== mgr.14152 v2:172.21.15.5:6800/91856593 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7efc50063c00 con 0x7efc3c0b2880 2024-05-09T00:41:16.618 INFO:teuthology.orchestra.run.smithi157.stdout:Created osd(s) 8 on host 'smithi157' 2024-05-09T00:41:16.619 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:41:16.618+0000 7efc585fa700 1 -- 172.21.15.157:0/3336848150 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7efc3c0b2880 msgr2=0x7efc3c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:41:16.619 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:41:16.618+0000 7efc585fa700 1 --2- 172.21.15.157:0/3336848150 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7efc3c0b2880 0x7efc3c0b4d90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7efc5019b9d0 tx=0x7efc4000b690 comp rx=0 tx=0).stop 2024-05-09T00:41:16.619 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:41:16.618+0000 7efc585fa700 1 -- 172.21.15.157:0/3336848150 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc50104340 msgr2=0x7efc5019a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:41:16.620 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:41:16.618+0000 7efc585fa700 1 --2- 172.21.15.157:0/3336848150 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc50104340 0x7efc5019a920 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7efc4c00b780 tx=0x7efc4c004d70 comp rx=0 tx=0).stop 2024-05-09T00:41:16.620 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:41:16.618+0000 7efc585fa700 1 -- 172.21.15.157:0/3336848150 shutdown_connections 2024-05-09T00:41:16.620 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:41:16.618+0000 7efc585fa700 1 --2- 172.21.15.157:0/3336848150 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7efc3c0b2880 0x7efc3c0b4d90 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:41:16.620 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:41:16.618+0000 7efc585fa700 1 --2- 172.21.15.157:0/3336848150 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efc50105e10 0x7efc501a1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:41:16.620 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:41:16.618+0000 7efc585fa700 1 --2- 172.21.15.157:0/3336848150 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc50104340 0x7efc5019a920 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:41:16.620 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:41:16.618+0000 7efc585fa700 1 --2- 172.21.15.157:0/3336848150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc50105470 0x7efc5019ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:41:16.620 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:41:16.618+0000 7efc585fa700 1 -- 172.21.15.157:0/3336848150 >> 172.21.15.157:0/3336848150 conn(0x7efc500ff9c0 msgr2=0x7efc501086a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:41:16.620 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:41:16.618+0000 7efc585fa700 1 -- 172.21.15.157:0/3336848150 shutdown_connections 2024-05-09T00:41:16.620 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:41:16.618+0000 7efc585fa700 1 -- 172.21.15.157:0/3336848150 wait complete. 2024-05-09T00:41:16.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:16 smithi157 ceph-mon[34294]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-05-09T00:41:16.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:16 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:41:16.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:16 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:41:16.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:16 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:41:16.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:16 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:41:16.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:16 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:41:16.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:16 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:41:16.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:16 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:41:16.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:16 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:41:16.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:16 smithi157 ceph-mon[34294]: from='osd.8 [v2:172.21.15.157:6816/2001626768,v1:172.21.15.157:6817/2001626768]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-09T00:41:16.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:16 smithi157 ceph-mon[34294]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-09T00:41:17.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:16 smithi131 ceph-mon[34547]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-05-09T00:41:17.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:16 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:41:17.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:16 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:41:17.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:16 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:41:17.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:16 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:41:17.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:16 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:41:17.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:16 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:41:17.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:16 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:41:17.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:16 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:41:17.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:16 smithi131 ceph-mon[34547]: from='osd.8 [v2:172.21.15.157:6816/2001626768,v1:172.21.15.157:6817/2001626768]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-09T00:41:17.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:16 smithi131 ceph-mon[34547]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-09T00:41:17.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:16 smithi005 ceph-mon[30365]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-05-09T00:41:17.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:16 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:41:17.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:16 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:41:17.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:16 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:41:17.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:16 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:41:17.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:16 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:41:17.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:16 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:41:17.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:16 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:41:17.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:16 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:41:17.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:16 smithi005 ceph-mon[30365]: from='osd.8 [v2:172.21.15.157:6816/2001626768,v1:172.21.15.157:6817/2001626768]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-09T00:41:17.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:16 smithi005 ceph-mon[30365]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-09T00:41:17.330 DEBUG:teuthology.orchestra.run.smithi157:osd.8> sudo journalctl -f -n 0 -u ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@osd.8.service 2024-05-09T00:41:17.333 INFO:tasks.cephadm:Deploying osd.9 on smithi005 with /dev/vg_nvme/lv_1... 2024-05-09T00:41:17.333 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-09T00:41:17.594 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:41:18.857 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:18 smithi157 ceph-mon[34294]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-09T00:41:18.857 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:18 smithi157 ceph-mon[34294]: from='osd.8 [v2:172.21.15.157:6816/2001626768,v1:172.21.15.157:6817/2001626768]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-09T00:41:18.857 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:18 smithi157 ceph-mon[34294]: osdmap e51: 9 total, 8 up, 9 in 2024-05-09T00:41:18.857 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:18 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T00:41:18.857 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:18 smithi157 ceph-mon[34294]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-09T00:41:18.857 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:18 smithi157 ceph-mon[34294]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-05-09T00:41:18.858 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 00:41:18 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T00:41:18.618+0000 7f2aeecf9700 -1 osd.8 0 waiting for initial osdmap 2024-05-09T00:41:18.858 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 00:41:18 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T00:41:18.631+0000 7f2aeaaae700 -1 osd.8 52 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T00:41:18.934 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:18 smithi005 ceph-mon[30365]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-09T00:41:18.934 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:18 smithi005 ceph-mon[30365]: from='osd.8 [v2:172.21.15.157:6816/2001626768,v1:172.21.15.157:6817/2001626768]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-09T00:41:18.934 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:18 smithi005 ceph-mon[30365]: osdmap e51: 9 total, 8 up, 9 in 2024-05-09T00:41:18.935 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:18 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T00:41:18.935 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:18 smithi005 ceph-mon[30365]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-09T00:41:18.935 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:18 smithi005 ceph-mon[30365]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-05-09T00:41:19.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:18 smithi131 ceph-mon[34547]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-09T00:41:19.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:18 smithi131 ceph-mon[34547]: from='osd.8 [v2:172.21.15.157:6816/2001626768,v1:172.21.15.157:6817/2001626768]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-09T00:41:19.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:18 smithi131 ceph-mon[34547]: osdmap e51: 9 total, 8 up, 9 in 2024-05-09T00:41:19.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:18 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T00:41:19.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:18 smithi131 ceph-mon[34547]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-09T00:41:19.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:18 smithi131 ceph-mon[34547]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-05-09T00:41:19.872 INFO:teuthology.orchestra.run.smithi005.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-09T00:41:19.872 INFO:teuthology.orchestra.run.smithi005.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-09T00:41:19.872 INFO:teuthology.orchestra.run.smithi005.stderr: stderr: 10+0 records in 2024-05-09T00:41:19.872 INFO:teuthology.orchestra.run.smithi005.stderr:10+0 records out 2024-05-09T00:41:19.872 INFO:teuthology.orchestra.run.smithi005.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00844991 s, 1.2 GB/s 2024-05-09T00:41:19.873 INFO:teuthology.orchestra.run.smithi005.stderr:--> Zapping successful for: 2024-05-09T00:41:19.873 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:19 smithi005 ceph-mon[30365]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-05-09T00:41:19.873 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:19 smithi005 ceph-mon[30365]: osdmap e52: 9 total, 8 up, 9 in 2024-05-09T00:41:19.873 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:19 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T00:41:19.873 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:19 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T00:41:19.874 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:19 smithi005 ceph-mon[30365]: osd.8 [v2:172.21.15.157:6816/2001626768,v1:172.21.15.157:6817/2001626768] boot 2024-05-09T00:41:19.874 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:19 smithi005 ceph-mon[30365]: osdmap e53: 9 total, 9 up, 9 in 2024-05-09T00:41:19.874 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:19 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T00:41:19.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:19 smithi157 ceph-mon[34294]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-05-09T00:41:19.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:19 smithi157 ceph-mon[34294]: osdmap e52: 9 total, 8 up, 9 in 2024-05-09T00:41:19.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:19 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T00:41:19.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:19 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T00:41:19.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:19 smithi157 ceph-mon[34294]: osd.8 [v2:172.21.15.157:6816/2001626768,v1:172.21.15.157:6817/2001626768] boot 2024-05-09T00:41:19.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:19 smithi157 ceph-mon[34294]: osdmap e53: 9 total, 9 up, 9 in 2024-05-09T00:41:19.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:19 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T00:41:20.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:19 smithi131 ceph-mon[34547]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-05-09T00:41:20.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:19 smithi131 ceph-mon[34547]: osdmap e52: 9 total, 8 up, 9 in 2024-05-09T00:41:20.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:19 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T00:41:20.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:19 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T00:41:20.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:19 smithi131 ceph-mon[34547]: osd.8 [v2:172.21.15.157:6816/2001626768,v1:172.21.15.157:6817/2001626768] boot 2024-05-09T00:41:20.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:19 smithi131 ceph-mon[34547]: osdmap e53: 9 total, 9 up, 9 in 2024-05-09T00:41:20.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:19 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T00:41:20.569 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi005:vg_nvme/lv_1 2024-05-09T00:41:20.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:20 smithi005 ceph-mon[30365]: purged_snaps scrub starts 2024-05-09T00:41:20.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:20 smithi005 ceph-mon[30365]: purged_snaps scrub ok 2024-05-09T00:41:20.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:20 smithi005 ceph-mon[30365]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 804 GiB / 805 GiB avail 2024-05-09T00:41:20.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:20 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:41:20.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:20 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:41:20.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:20 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:41:20.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:20 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:41:20.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:20 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:41:20.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:20 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:41:20.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:20 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:41:20.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:20 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:41:20.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:20 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:41:20.830 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:41:20.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:20 smithi157 ceph-mon[34294]: purged_snaps scrub starts 2024-05-09T00:41:20.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:20 smithi157 ceph-mon[34294]: purged_snaps scrub ok 2024-05-09T00:41:20.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:20 smithi157 ceph-mon[34294]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 804 GiB / 805 GiB avail 2024-05-09T00:41:20.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:20 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:41:20.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:20 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:41:20.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:20 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:41:20.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:20 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:41:20.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:20 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:41:20.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:20 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:41:20.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:20 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:41:20.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:20 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:41:20.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:20 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:41:21.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:20 smithi131 ceph-mon[34547]: purged_snaps scrub starts 2024-05-09T00:41:21.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:20 smithi131 ceph-mon[34547]: purged_snaps scrub ok 2024-05-09T00:41:21.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:20 smithi131 ceph-mon[34547]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 804 GiB / 805 GiB avail 2024-05-09T00:41:21.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:20 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:41:21.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:20 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:41:21.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:20 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:41:21.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:20 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:41:21.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:20 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:41:21.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:20 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:41:21.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:20 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:41:21.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:20 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:41:21.107 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:20 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:41:21.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:21 smithi157 ceph-mon[34294]: Detected new or changed devices on smithi157 2024-05-09T00:41:21.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:21 smithi157 ceph-mon[34294]: Adjusting osd_memory_target on smithi157 to 5696M 2024-05-09T00:41:21.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:21 smithi157 ceph-mon[34294]: osdmap e54: 9 total, 9 up, 9 in 2024-05-09T00:41:22.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:21 smithi005 ceph-mon[30365]: Detected new or changed devices on smithi157 2024-05-09T00:41:22.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:21 smithi005 ceph-mon[30365]: Adjusting osd_memory_target on smithi157 to 5696M 2024-05-09T00:41:22.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:21 smithi005 ceph-mon[30365]: osdmap e54: 9 total, 9 up, 9 in 2024-05-09T00:41:22.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:21 smithi131 ceph-mon[34547]: Detected new or changed devices on smithi157 2024-05-09T00:41:22.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:21 smithi131 ceph-mon[34547]: Adjusting osd_memory_target on smithi157 to 5696M 2024-05-09T00:41:22.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:21 smithi131 ceph-mon[34547]: osdmap e54: 9 total, 9 up, 9 in 2024-05-09T00:41:22.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:41:22.606+0000 7efd116a4700 1 -- 172.21.15.5:0/2504392968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd0c1029e0 msgr2=0x7efd0c102df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:41:22.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:41:22.606+0000 7efd116a4700 1 --2- 172.21.15.5:0/2504392968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd0c1029e0 0x7efd0c102df0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7efcfc009990 tx=0x7efcfc009ca0 comp rx=0 tx=0).stop 2024-05-09T00:41:22.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:41:22.606+0000 7efd116a4700 1 -- 172.21.15.5:0/2504392968 shutdown_connections 2024-05-09T00:41:22.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:41:22.606+0000 7efd116a4700 1 --2- 172.21.15.5:0/2504392968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd0c1044b0 0x7efd0c10ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:41:22.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:41:22.606+0000 7efd116a4700 1 --2- 172.21.15.5:0/2504392968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd0c1029e0 0x7efd0c102df0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:41:22.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:41:22.606+0000 7efd116a4700 1 --2- 172.21.15.5:0/2504392968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd0c103b10 0x7efd0c103f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:41:22.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:41:22.606+0000 7efd116a4700 1 -- 172.21.15.5:0/2504392968 >> 172.21.15.5:0/2504392968 conn(0x7efd0c0fe020 msgr2=0x7efd0c1004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:41:22.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:41:22.606+0000 7efd116a4700 1 -- 172.21.15.5:0/2504392968 shutdown_connections 2024-05-09T00:41:22.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:41:22.606+0000 7efd116a4700 1 -- 172.21.15.5:0/2504392968 wait complete. 2024-05-09T00:41:22.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:41:22.607+0000 7efd116a4700 1 Processor -- start 2024-05-09T00:41:22.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:41:22.607+0000 7efd116a4700 1 -- start start 2024-05-09T00:41:22.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:41:22.607+0000 7efd116a4700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd0c1029e0 0x7efd0c106810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:41:22.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:41:22.607+0000 7efd116a4700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd0c103b10 0x7efd0c104ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:41:22.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:41:22.607+0000 7efd116a4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd0c1044b0 0x7efd0c105400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:41:22.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:41:22.607+0000 7efd116a4700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efd0c10eda0 con 0x7efd0c1044b0 2024-05-09T00:41:22.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:41:22.607+0000 7efd116a4700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7efd0c10ec20 con 0x7efd0c1029e0 2024-05-09T00:41:22.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:41:22.607+0000 7efd116a4700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7efd0c10ef20 con 0x7efd0c103b10 2024-05-09T00:41:22.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:41:22.608+0000 7efd0b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd0c1044b0 0x7efd0c105400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:41:22.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:41:22.608+0000 7efd0b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd0c1044b0 0x7efd0c105400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37172/0 (socket says 172.21.15.5:37172) 2024-05-09T00:41:22.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:41:22.608+0000 7efd0b7fe700 1 -- 172.21.15.5:0/856696485 learned_addr learned my addr 172.21.15.5:0/856696485 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:41:22.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:41:22.608+0000 7efd0a7fc700 1 --2- 172.21.15.5:0/856696485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd0c103b10 0x7efd0c104ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:41:22.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:41:22.608+0000 7efd0affd700 1 --2- 172.21.15.5:0/856696485 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd0c1029e0 0x7efd0c106810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:41:22.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:41:22.608+0000 7efd0b7fe700 1 -- 172.21.15.5:0/856696485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd0c103b10 msgr2=0x7efd0c104ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:41:22.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:41:22.608+0000 7efd0b7fe700 1 --2- 172.21.15.5:0/856696485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd0c103b10 0x7efd0c104ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:41:22.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:41:22.608+0000 7efd0b7fe700 1 -- 172.21.15.5:0/856696485 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd0c1029e0 msgr2=0x7efd0c106810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:41:22.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:41:22.608+0000 7efd0b7fe700 1 --2- 172.21.15.5:0/856696485 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd0c1029e0 0x7efd0c106810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:41:22.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:41:22.608+0000 7efd0b7fe700 1 -- 172.21.15.5:0/856696485 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd0c105b60 con 0x7efd0c1044b0 2024-05-09T00:41:22.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:41:22.608+0000 7efd0b7fe700 1 --2- 172.21.15.5:0/856696485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd0c1044b0 0x7efd0c105400 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7efd0400b970 tx=0x7efd0400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:41:22.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:41:22.608+0000 7efd03fff700 1 -- 172.21.15.5:0/856696485 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0400d860 con 0x7efd0c1044b0 2024-05-09T00:41:22.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:41:22.608+0000 7efd116a4700 1 -- 172.21.15.5:0/856696485 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd0c1a4b80 con 0x7efd0c1044b0 2024-05-09T00:41:22.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:41:22.608+0000 7efd03fff700 1 -- 172.21.15.5:0/856696485 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd0400e430 con 0x7efd0c1044b0 2024-05-09T00:41:22.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:41:22.608+0000 7efd03fff700 1 -- 172.21.15.5:0/856696485 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd04017660 con 0x7efd0c1044b0 2024-05-09T00:41:22.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:41:22.608+0000 7efd116a4700 1 -- 172.21.15.5:0/856696485 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd0c1a5060 con 0x7efd0c1044b0 2024-05-09T00:41:22.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:41:22.610+0000 7efd116a4700 1 -- 172.21.15.5:0/856696485 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd0c0512c0 con 0x7efd0c1044b0 2024-05-09T00:41:22.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:41:22.611+0000 7efd03fff700 1 -- 172.21.15.5:0/856696485 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7efd0400da00 con 0x7efd0c1044b0 2024-05-09T00:41:22.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:41:22.612+0000 7efd03fff700 1 --2- 172.21.15.5:0/856696485 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7efcf80b2880 0x7efcf80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:41:22.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:41:22.612+0000 7efd0affd700 1 --2- 172.21.15.5:0/856696485 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7efcf80b2880 0x7efcf80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:41:22.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:41:22.612+0000 7efd03fff700 1 -- 172.21.15.5:0/856696485 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5829+0+0 (secure 0 0 0) 0x7efd040e1480 con 0x7efd0c1044b0 2024-05-09T00:41:22.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:41:22.612+0000 7efd0affd700 1 --2- 172.21.15.5:0/856696485 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7efcf80b2880 0x7efcf80b4d90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7efcfc0038a0 tx=0x7efcfc01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:41:22.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:41:22.617+0000 7efd03fff700 1 -- 172.21.15.5:0/856696485 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efd04013030 con 0x7efd0c1044b0 2024-05-09T00:41:22.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:41:22.843+0000 7efd116a4700 1 -- 172.21.15.5:0/856696485 --> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7efd0c1a52f0 con 0x7efcf80b2880 2024-05-09T00:41:22.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:22 smithi157 ceph-mon[34294]: osdmap e55: 9 total, 9 up, 9 in 2024-05-09T00:41:22.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:22 smithi157 ceph-mon[34294]: pgmap v195: 1 pgs: 1 remapped+peering; 577 KiB data, 254 MiB used, 804 GiB / 805 GiB avail 2024-05-09T00:41:23.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:22 smithi005 ceph-mon[30365]: osdmap e55: 9 total, 9 up, 9 in 2024-05-09T00:41:23.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:22 smithi005 ceph-mon[30365]: pgmap v195: 1 pgs: 1 remapped+peering; 577 KiB data, 254 MiB used, 804 GiB / 805 GiB avail 2024-05-09T00:41:23.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:22 smithi131 ceph-mon[34547]: osdmap e55: 9 total, 9 up, 9 in 2024-05-09T00:41:23.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:22 smithi131 ceph-mon[34547]: pgmap v195: 1 pgs: 1 remapped+peering; 577 KiB data, 254 MiB used, 804 GiB / 805 GiB avail 2024-05-09T00:41:23.766 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:23 smithi005 ceph-mon[30365]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:41:23.766 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:23 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:41:23.766 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:23 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:41:23.766 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:23 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:41:23.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:23 smithi157 ceph-mon[34294]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:41:23.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:23 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:41:23.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:23 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:41:23.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:23 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:41:24.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:23 smithi131 ceph-mon[34547]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:41:24.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:23 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:41:24.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:23 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:41:24.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:23 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:41:24.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:24 smithi005 ceph-mon[30365]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 254 MiB used, 804 GiB / 805 GiB avail 2024-05-09T00:41:24.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:24 smithi157 ceph-mon[34294]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 254 MiB used, 804 GiB / 805 GiB avail 2024-05-09T00:41:25.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:24 smithi131 ceph-mon[34547]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 254 MiB used, 804 GiB / 805 GiB avail 2024-05-09T00:41:27.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:26 smithi157 ceph-mon[34294]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-05-09T00:41:27.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:26 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/884325189' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1927dfe7-c1af-4a15-bb71-d58d97c8734a"}]: dispatch 2024-05-09T00:41:27.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:26 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/884325189' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1927dfe7-c1af-4a15-bb71-d58d97c8734a"}]': finished 2024-05-09T00:41:27.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:26 smithi157 ceph-mon[34294]: osdmap e56: 10 total, 9 up, 10 in 2024-05-09T00:41:27.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:26 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T00:41:27.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:26 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/1574140169' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-9/activate.monmap"}]: dispatch 2024-05-09T00:41:27.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:26 smithi005 ceph-mon[30365]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-05-09T00:41:27.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:26 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/884325189' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1927dfe7-c1af-4a15-bb71-d58d97c8734a"}]: dispatch 2024-05-09T00:41:27.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:26 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/884325189' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1927dfe7-c1af-4a15-bb71-d58d97c8734a"}]': finished 2024-05-09T00:41:27.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:26 smithi005 ceph-mon[30365]: osdmap e56: 10 total, 9 up, 10 in 2024-05-09T00:41:27.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:26 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T00:41:27.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:26 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/1574140169' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-9/activate.monmap"}]: dispatch 2024-05-09T00:41:27.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:26 smithi131 ceph-mon[34547]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-05-09T00:41:27.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:26 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/884325189' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1927dfe7-c1af-4a15-bb71-d58d97c8734a"}]: dispatch 2024-05-09T00:41:27.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:26 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/884325189' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1927dfe7-c1af-4a15-bb71-d58d97c8734a"}]': finished 2024-05-09T00:41:27.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:26 smithi131 ceph-mon[34547]: osdmap e56: 10 total, 9 up, 10 in 2024-05-09T00:41:27.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:26 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T00:41:27.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:26 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/1574140169' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-9/activate.monmap"}]: dispatch 2024-05-09T00:41:29.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:28 smithi157 ceph-mon[34294]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-05-09T00:41:29.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:28 smithi005 ceph-mon[30365]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-05-09T00:41:29.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:28 smithi131 ceph-mon[34547]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-05-09T00:41:31.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:30 smithi005 ceph-mon[30365]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-05-09T00:41:31.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:30 smithi157 ceph-mon[34294]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-05-09T00:41:31.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:30 smithi131 ceph-mon[34547]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-05-09T00:41:33.158 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:32 smithi131 ceph-mon[34547]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T00:41:33.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:32 smithi157 ceph-mon[34294]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T00:41:33.266 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:32 smithi005 ceph-mon[30365]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T00:41:34.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:33 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-09T00:41:34.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:33 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:41:34.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:33 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-09T00:41:34.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:33 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:41:34.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:33 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-09T00:41:34.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:33 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:41:35.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:34 smithi005 ceph-mon[30365]: Deploying daemon osd.9 on smithi005 2024-05-09T00:41:35.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:34 smithi005 ceph-mon[30365]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T00:41:35.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:34 smithi157 ceph-mon[34294]: Deploying daemon osd.9 on smithi005 2024-05-09T00:41:35.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:34 smithi157 ceph-mon[34294]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T00:41:35.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:34 smithi131 ceph-mon[34547]: Deploying daemon osd.9 on smithi005 2024-05-09T00:41:35.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:34 smithi131 ceph-mon[34547]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T00:41:37.186 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:36 smithi005 ceph-mon[30365]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-05-09T00:41:37.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:36 smithi157 ceph-mon[34294]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-05-09T00:41:37.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:36 smithi131 ceph-mon[34547]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-05-09T00:41:39.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:38 smithi157 ceph-mon[34294]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-05-09T00:41:39.236 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:38 smithi005 ceph-mon[30365]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-05-09T00:41:39.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:38 smithi131 ceph-mon[34547]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-05-09T00:41:41.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:40 smithi157 ceph-mon[34294]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-05-09T00:41:41.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:40 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:41:41.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:40 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:41:41.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:40 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:41:41.280 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:40 smithi005 ceph-mon[30365]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-05-09T00:41:41.281 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:40 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:41:41.281 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:40 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:41:41.281 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:40 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:41:41.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:40 smithi131 ceph-mon[34547]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-05-09T00:41:41.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:40 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:41:41.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:40 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:41:41.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:40 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:41:43.178 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:42 smithi005 ceph-mon[30365]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-05-09T00:41:43.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:42 smithi157 ceph-mon[34294]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-05-09T00:41:43.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:42 smithi131 ceph-mon[34547]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-05-09T00:41:43.567 INFO:teuthology.orchestra.run.smithi005.stdout:Created osd(s) 9 on host 'smithi005' 2024-05-09T00:41:43.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:41:43.565+0000 7efd03fff700 1 -- 172.21.15.5:0/856696485 <== mgr.14152 v2:172.21.15.5:6800/91856593 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7efd0c1a52f0 con 0x7efcf80b2880 2024-05-09T00:41:43.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:41:43.568+0000 7efd116a4700 1 -- 172.21.15.5:0/856696485 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7efcf80b2880 msgr2=0x7efcf80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:41:43.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:41:43.568+0000 7efd116a4700 1 --2- 172.21.15.5:0/856696485 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7efcf80b2880 0x7efcf80b4d90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7efcfc0038a0 tx=0x7efcfc01a040 comp rx=0 tx=0).stop 2024-05-09T00:41:43.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:41:43.568+0000 7efd116a4700 1 -- 172.21.15.5:0/856696485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd0c1044b0 msgr2=0x7efd0c105400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:41:43.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:41:43.568+0000 7efd116a4700 1 --2- 172.21.15.5:0/856696485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd0c1044b0 0x7efd0c105400 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7efd0400b970 tx=0x7efd0400bc80 comp rx=0 tx=0).stop 2024-05-09T00:41:43.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:41:43.568+0000 7efd116a4700 1 -- 172.21.15.5:0/856696485 shutdown_connections 2024-05-09T00:41:43.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:41:43.568+0000 7efd116a4700 1 --2- 172.21.15.5:0/856696485 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7efcf80b2880 0x7efcf80b4d90 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:41:43.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:41:43.568+0000 7efd116a4700 1 --2- 172.21.15.5:0/856696485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd0c1044b0 0x7efd0c105400 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:41:43.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:41:43.568+0000 7efd116a4700 1 --2- 172.21.15.5:0/856696485 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd0c1029e0 0x7efd0c106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:41:43.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:41:43.568+0000 7efd116a4700 1 --2- 172.21.15.5:0/856696485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd0c103b10 0x7efd0c104ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:41:43.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:41:43.568+0000 7efd116a4700 1 -- 172.21.15.5:0/856696485 >> 172.21.15.5:0/856696485 conn(0x7efd0c0fe020 msgr2=0x7efd0c10b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:41:43.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:41:43.568+0000 7efd116a4700 1 -- 172.21.15.5:0/856696485 shutdown_connections 2024-05-09T00:41:43.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:41:43.568+0000 7efd116a4700 1 -- 172.21.15.5:0/856696485 wait complete. 2024-05-09T00:41:44.198 DEBUG:teuthology.orchestra.run.smithi005:osd.9> sudo journalctl -f -n 0 -u ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@osd.9.service 2024-05-09T00:41:44.201 INFO:tasks.cephadm:Deploying osd.10 on smithi131 with /dev/vg_nvme/lv_1... 2024-05-09T00:41:44.201 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-09T00:41:44.448 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T00:41:44.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:44 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:41:44.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:44 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:41:44.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:44 smithi157 ceph-mon[34294]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-05-09T00:41:44.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:44 smithi157 ceph-mon[34294]: from='osd.9 [v2:172.21.15.5:6826/4287220015,v1:172.21.15.5:6827/4287220015]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-09T00:41:44.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:44 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:41:44.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:44 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:41:44.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:44 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:41:44.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:44 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:41:44.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:44 smithi005 ceph-mon[30365]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-05-09T00:41:44.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:44 smithi005 ceph-mon[30365]: from='osd.9 [v2:172.21.15.5:6826/4287220015,v1:172.21.15.5:6827/4287220015]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-09T00:41:44.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:44 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:41:44.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:44 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:41:44.800 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:44 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:41:44.800 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:44 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:41:44.801 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:44 smithi131 ceph-mon[34547]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-05-09T00:41:44.801 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:44 smithi131 ceph-mon[34547]: from='osd.9 [v2:172.21.15.5:6826/4287220015,v1:172.21.15.5:6827/4287220015]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-09T00:41:44.801 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:44 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:41:44.801 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:44 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:41:45.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:45 smithi157 ceph-mon[34294]: from='osd.9 [v2:172.21.15.5:6826/4287220015,v1:172.21.15.5:6827/4287220015]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-09T00:41:45.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:45 smithi157 ceph-mon[34294]: osdmap e57: 10 total, 9 up, 10 in 2024-05-09T00:41:45.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:45 smithi157 ceph-mon[34294]: from='osd.9 [v2:172.21.15.5:6826/4287220015,v1:172.21.15.5:6827/4287220015]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-05-09T00:41:45.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:45 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T00:41:45.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:45 smithi005 ceph-mon[30365]: from='osd.9 [v2:172.21.15.5:6826/4287220015,v1:172.21.15.5:6827/4287220015]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-09T00:41:45.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:45 smithi005 ceph-mon[30365]: osdmap e57: 10 total, 9 up, 10 in 2024-05-09T00:41:45.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:45 smithi005 ceph-mon[30365]: from='osd.9 [v2:172.21.15.5:6826/4287220015,v1:172.21.15.5:6827/4287220015]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-05-09T00:41:45.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:45 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T00:41:45.792 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 00:41:45 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T00:41:45.530+0000 7f73528e5700 -1 osd.9 0 waiting for initial osdmap 2024-05-09T00:41:45.792 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 00:41:45 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T00:41:45.542+0000 7f734e69a700 -1 osd.9 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T00:41:45.852 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:45 smithi131 ceph-mon[34547]: from='osd.9 [v2:172.21.15.5:6826/4287220015,v1:172.21.15.5:6827/4287220015]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-09T00:41:45.852 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:45 smithi131 ceph-mon[34547]: osdmap e57: 10 total, 9 up, 10 in 2024-05-09T00:41:45.852 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:45 smithi131 ceph-mon[34547]: from='osd.9 [v2:172.21.15.5:6826/4287220015,v1:172.21.15.5:6827/4287220015]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-05-09T00:41:45.852 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:45 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T00:41:46.588 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-09T00:41:46.588 INFO:teuthology.orchestra.run.smithi131.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-09T00:41:46.588 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 10+0 records in 2024-05-09T00:41:46.588 INFO:teuthology.orchestra.run.smithi131.stderr:10+0 records out 2024-05-09T00:41:46.588 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011487 s, 913 MB/s 2024-05-09T00:41:46.588 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping successful for: 2024-05-09T00:41:46.631 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:46 smithi005 ceph-mon[30365]: from='osd.9 [v2:172.21.15.5:6826/4287220015,v1:172.21.15.5:6827/4287220015]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-05-09T00:41:46.631 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:46 smithi005 ceph-mon[30365]: osdmap e58: 10 total, 9 up, 10 in 2024-05-09T00:41:46.631 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:46 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T00:41:46.631 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:46 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T00:41:46.631 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:46 smithi005 ceph-mon[30365]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-05-09T00:41:46.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:46 smithi131 ceph-mon[34547]: from='osd.9 [v2:172.21.15.5:6826/4287220015,v1:172.21.15.5:6827/4287220015]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-05-09T00:41:46.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:46 smithi131 ceph-mon[34547]: osdmap e58: 10 total, 9 up, 10 in 2024-05-09T00:41:46.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:46 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T00:41:46.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:46 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T00:41:46.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:46 smithi131 ceph-mon[34547]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-05-09T00:41:46.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:46 smithi157 ceph-mon[34294]: from='osd.9 [v2:172.21.15.5:6826/4287220015,v1:172.21.15.5:6827/4287220015]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-05-09T00:41:46.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:46 smithi157 ceph-mon[34294]: osdmap e58: 10 total, 9 up, 10 in 2024-05-09T00:41:46.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:46 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T00:41:46.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:46 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T00:41:46.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:46 smithi157 ceph-mon[34294]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-05-09T00:41:47.266 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi131:vg_nvme/lv_1 2024-05-09T00:41:47.502 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T00:41:47.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:47 smithi005 ceph-mon[30365]: purged_snaps scrub starts 2024-05-09T00:41:47.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:47 smithi005 ceph-mon[30365]: purged_snaps scrub ok 2024-05-09T00:41:47.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:47 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T00:41:47.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:47 smithi005 ceph-mon[30365]: osd.9 [v2:172.21.15.5:6826/4287220015,v1:172.21.15.5:6827/4287220015] boot 2024-05-09T00:41:47.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:47 smithi005 ceph-mon[30365]: osdmap e59: 10 total, 10 up, 10 in 2024-05-09T00:41:47.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:47 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T00:41:47.835 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:47 smithi131 ceph-mon[34547]: purged_snaps scrub starts 2024-05-09T00:41:47.835 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:47 smithi131 ceph-mon[34547]: purged_snaps scrub ok 2024-05-09T00:41:47.835 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:47 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T00:41:47.835 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:47 smithi131 ceph-mon[34547]: osd.9 [v2:172.21.15.5:6826/4287220015,v1:172.21.15.5:6827/4287220015] boot 2024-05-09T00:41:47.835 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:47 smithi131 ceph-mon[34547]: osdmap e59: 10 total, 10 up, 10 in 2024-05-09T00:41:47.835 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:47 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T00:41:47.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:47 smithi157 ceph-mon[34294]: purged_snaps scrub starts 2024-05-09T00:41:47.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:47 smithi157 ceph-mon[34294]: purged_snaps scrub ok 2024-05-09T00:41:47.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:47 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T00:41:47.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:47 smithi157 ceph-mon[34294]: osd.9 [v2:172.21.15.5:6826/4287220015,v1:172.21.15.5:6827/4287220015] boot 2024-05-09T00:41:47.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:47 smithi157 ceph-mon[34294]: osdmap e59: 10 total, 10 up, 10 in 2024-05-09T00:41:47.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:47 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T00:41:48.799 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:48 smithi131 ceph-mon[34547]: osdmap e60: 10 total, 10 up, 10 in 2024-05-09T00:41:48.799 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:48 smithi131 ceph-mon[34547]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:41:48.799 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:48 smithi131 ceph-mon[34547]: Detected new or changed devices on smithi005 2024-05-09T00:41:48.799 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:48 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:41:48.800 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:48 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:41:48.800 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:48 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:41:48.800 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:48 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:41:48.800 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:48 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:41:48.800 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:48 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:41:48.800 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:48 smithi131 ceph-mon[34547]: Adjusting osd_memory_target on smithi005 to 4273M 2024-05-09T00:41:48.800 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:48 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:41:48.800 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:48 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:41:48.800 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:48 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:41:48.800 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:48 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:41:48.800 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:48 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:41:48.800 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:48 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:41:48.800 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:48 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:41:48.800 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:48 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:41:48.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:48 smithi157 ceph-mon[34294]: osdmap e60: 10 total, 10 up, 10 in 2024-05-09T00:41:48.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:48 smithi157 ceph-mon[34294]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:41:48.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:48 smithi157 ceph-mon[34294]: Detected new or changed devices on smithi005 2024-05-09T00:41:48.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:48 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:41:48.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:48 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:41:48.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:48 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:41:48.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:48 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:41:48.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:48 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:41:48.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:48 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:41:48.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:48 smithi157 ceph-mon[34294]: Adjusting osd_memory_target on smithi005 to 4273M 2024-05-09T00:41:48.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:48 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:41:48.939 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:48 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:41:48.939 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:48 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:41:48.939 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:48 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:41:48.939 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:48 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:41:48.939 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:48 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:41:48.939 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:48 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:41:48.939 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:48 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:41:49.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:48 smithi005 ceph-mon[30365]: osdmap e60: 10 total, 10 up, 10 in 2024-05-09T00:41:49.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:48 smithi005 ceph-mon[30365]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:41:49.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:48 smithi005 ceph-mon[30365]: Detected new or changed devices on smithi005 2024-05-09T00:41:49.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:48 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:41:49.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:48 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:41:49.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:48 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:41:49.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:48 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:41:49.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:48 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:41:49.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:48 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:41:49.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:48 smithi005 ceph-mon[30365]: Adjusting osd_memory_target on smithi005 to 4273M 2024-05-09T00:41:49.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:48 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:41:49.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:48 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:41:49.043 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:48 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:41:49.043 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:48 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:41:49.043 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:48 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:41:49.043 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:48 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:41:49.043 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:48 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:41:49.043 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:48 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:41:49.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:41:49.107+0000 7fe7cdcb6700 1 -- 172.21.15.131:0/3492072992 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7c8105900 msgr2=0x7fe7c8105d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:41:49.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:41:49.107+0000 7fe7cdcb6700 1 --2- 172.21.15.131:0/3492072992 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7c8105900 0x7fe7c8105d10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8009990 tx=0x7fe7b8009ca0 comp rx=0 tx=0).stop 2024-05-09T00:41:49.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:41:49.107+0000 7fe7cdcb6700 1 -- 172.21.15.131:0/3492072992 shutdown_connections 2024-05-09T00:41:49.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:41:49.107+0000 7fe7cdcb6700 1 --2- 172.21.15.131:0/3492072992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7c8102ca0 0x7fe7c810c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:41:49.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:41:49.107+0000 7fe7cdcb6700 1 --2- 172.21.15.131:0/3492072992 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7c8105900 0x7fe7c8105d10 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:41:49.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:41:49.107+0000 7fe7cdcb6700 1 --2- 172.21.15.131:0/3492072992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7c8102350 0x7fe7c8102760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:41:49.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:41:49.107+0000 7fe7cdcb6700 1 -- 172.21.15.131:0/3492072992 >> 172.21.15.131:0/3492072992 conn(0x7fe7c80fe020 msgr2=0x7fe7c81004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:41:49.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:41:49.107+0000 7fe7cdcb6700 1 -- 172.21.15.131:0/3492072992 shutdown_connections 2024-05-09T00:41:49.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:41:49.107+0000 7fe7cdcb6700 1 -- 172.21.15.131:0/3492072992 wait complete. 2024-05-09T00:41:49.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:41:49.108+0000 7fe7cdcb6700 1 Processor -- start 2024-05-09T00:41:49.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:41:49.108+0000 7fe7cdcb6700 1 -- start start 2024-05-09T00:41:49.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:41:49.109+0000 7fe7cdcb6700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7c8102350 0x7fe7c819a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:41:49.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:41:49.109+0000 7fe7cdcb6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7c8102ca0 0x7fe7c819ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:41:49.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:41:49.109+0000 7fe7cdcb6700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7c8105900 0x7fe7c81a1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:41:49.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:41:49.109+0000 7fe7cdcb6700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c810fda0 con 0x7fe7c8102ca0 2024-05-09T00:41:49.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:41:49.109+0000 7fe7cdcb6700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c810fc20 con 0x7fe7c8105900 2024-05-09T00:41:49.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:41:49.109+0000 7fe7cdcb6700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c810ff20 con 0x7fe7c8102350 2024-05-09T00:41:49.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:41:49.109+0000 7fe7c7fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7c8105900 0x7fe7c81a1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:41:49.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:41:49.109+0000 7fe7c7fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7c8105900 0x7fe7c81a1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52318/0 (socket says 172.21.15.131:52318) 2024-05-09T00:41:49.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:41:49.109+0000 7fe7c7fff700 1 -- 172.21.15.131:0/389703487 learned_addr learned my addr 172.21.15.131:0/389703487 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-09T00:41:49.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:41:49.109+0000 7fe7c77fe700 1 --2- 172.21.15.131:0/389703487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7c8102350 0x7fe7c819a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:41:49.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:41:49.109+0000 7fe7beffd700 1 --2- 172.21.15.131:0/389703487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7c8102ca0 0x7fe7c819ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:41:49.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:41:49.109+0000 7fe7c7fff700 1 -- 172.21.15.131:0/389703487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7c8102350 msgr2=0x7fe7c819a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:41:49.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:41:49.109+0000 7fe7c7fff700 1 --2- 172.21.15.131:0/389703487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7c8102350 0x7fe7c819a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:41:49.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:41:49.109+0000 7fe7c7fff700 1 -- 172.21.15.131:0/389703487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7c8102ca0 msgr2=0x7fe7c819ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:41:49.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:41:49.109+0000 7fe7c7fff700 1 --2- 172.21.15.131:0/389703487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7c8102ca0 0x7fe7c819ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:41:49.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:41:49.109+0000 7fe7c7fff700 1 -- 172.21.15.131:0/389703487 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7c81a2440 con 0x7fe7c8105900 2024-05-09T00:41:49.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:41:49.109+0000 7fe7c7fff700 1 --2- 172.21.15.131:0/389703487 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7c8105900 0x7fe7c81a1dc0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe7c0007ee0 tx=0x7fe7c000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:41:49.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:41:49.110+0000 7fe7c57fa700 1 -- 172.21.15.131:0/389703487 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c0004bb0 con 0x7fe7c8105900 2024-05-09T00:41:49.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:41:49.110+0000 7fe7cdcb6700 1 -- 172.21.15.131:0/389703487 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7c81a2730 con 0x7fe7c8105900 2024-05-09T00:41:49.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:41:49.110+0000 7fe7c57fa700 1 -- 172.21.15.131:0/389703487 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7c0004d50 con 0x7fe7c8105900 2024-05-09T00:41:49.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:41:49.110+0000 7fe7c57fa700 1 -- 172.21.15.131:0/389703487 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c0005920 con 0x7fe7c8105900 2024-05-09T00:41:49.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:41:49.110+0000 7fe7cdcb6700 1 -- 172.21.15.131:0/389703487 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7c81a2c70 con 0x7fe7c8105900 2024-05-09T00:41:49.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:41:49.111+0000 7fe7cdcb6700 1 -- 172.21.15.131:0/389703487 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7c80512c0 con 0x7fe7c8105900 2024-05-09T00:41:49.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:41:49.112+0000 7fe7c57fa700 1 -- 172.21.15.131:0/389703487 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fe7c001a020 con 0x7fe7c8105900 2024-05-09T00:41:49.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:41:49.113+0000 7fe7c57fa700 1 --2- 172.21.15.131:0/389703487 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe7b40b2880 0x7fe7b40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:41:49.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:41:49.113+0000 7fe7c57fa700 1 -- 172.21.15.131:0/389703487 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6261+0+0 (secure 0 0 0) 0x7fe7c00e1720 con 0x7fe7c8105900 2024-05-09T00:41:49.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:41:49.113+0000 7fe7c77fe700 1 --2- 172.21.15.131:0/389703487 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe7b40b2880 0x7fe7b40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:41:49.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:41:49.113+0000 7fe7c77fe700 1 --2- 172.21.15.131:0/389703487 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe7b40b2880 0x7fe7b40b4d90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe7b80038a0 tx=0x7fe7b801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:41:49.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:41:49.116+0000 7fe7c57fa700 1 -- 172.21.15.131:0/389703487 <== mon.2 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe7c0010070 con 0x7fe7c8105900 2024-05-09T00:41:49.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:41:49.327+0000 7fe7cdcb6700 1 -- 172.21.15.131:0/389703487 --> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fe7c8063c00 con 0x7fe7b40b2880 2024-05-09T00:41:49.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:49 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:41:49.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:49 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:41:49.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:49 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:41:49.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:49 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:41:49.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:49 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:41:49.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:49 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:41:50.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:49 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:41:50.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:49 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:41:50.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:49 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:41:50.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:50 smithi131 ceph-mon[34547]: from='client.24344 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:41:50.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:50 smithi131 ceph-mon[34547]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:41:50.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:50 smithi157 ceph-mon[34294]: from='client.24344 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:41:50.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:50 smithi157 ceph-mon[34294]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:41:51.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:50 smithi005 ceph-mon[30365]: from='client.24344 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:41:51.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:50 smithi005 ceph-mon[30365]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:41:53.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:52 smithi157 ceph-mon[34294]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:41:53.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:52 smithi157 ceph-mon[34294]: from='client.? 172.21.15.131:0/3777143852' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fbb4f3e3-eb7d-4ecb-a30b-55b8910375ac"}]: dispatch 2024-05-09T00:41:53.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:52 smithi157 ceph-mon[34294]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fbb4f3e3-eb7d-4ecb-a30b-55b8910375ac"}]: dispatch 2024-05-09T00:41:53.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:52 smithi157 ceph-mon[34294]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fbb4f3e3-eb7d-4ecb-a30b-55b8910375ac"}]': finished 2024-05-09T00:41:53.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:52 smithi157 ceph-mon[34294]: osdmap e61: 11 total, 10 up, 11 in 2024-05-09T00:41:53.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:52 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T00:41:53.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:52 smithi157 ceph-mon[34294]: from='client.? 172.21.15.131:0/533457438' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-10/activate.monmap"}]: dispatch 2024-05-09T00:41:53.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:52 smithi005 ceph-mon[30365]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:41:53.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:52 smithi005 ceph-mon[30365]: from='client.? 172.21.15.131:0/3777143852' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fbb4f3e3-eb7d-4ecb-a30b-55b8910375ac"}]: dispatch 2024-05-09T00:41:53.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:52 smithi005 ceph-mon[30365]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fbb4f3e3-eb7d-4ecb-a30b-55b8910375ac"}]: dispatch 2024-05-09T00:41:53.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:52 smithi005 ceph-mon[30365]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fbb4f3e3-eb7d-4ecb-a30b-55b8910375ac"}]': finished 2024-05-09T00:41:53.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:52 smithi005 ceph-mon[30365]: osdmap e61: 11 total, 10 up, 11 in 2024-05-09T00:41:53.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:52 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T00:41:53.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:52 smithi005 ceph-mon[30365]: from='client.? 172.21.15.131:0/533457438' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-10/activate.monmap"}]: dispatch 2024-05-09T00:41:53.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:52 smithi131 ceph-mon[34547]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:41:53.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:52 smithi131 ceph-mon[34547]: from='client.? 172.21.15.131:0/3777143852' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fbb4f3e3-eb7d-4ecb-a30b-55b8910375ac"}]: dispatch 2024-05-09T00:41:53.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:52 smithi131 ceph-mon[34547]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fbb4f3e3-eb7d-4ecb-a30b-55b8910375ac"}]: dispatch 2024-05-09T00:41:53.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:52 smithi131 ceph-mon[34547]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fbb4f3e3-eb7d-4ecb-a30b-55b8910375ac"}]': finished 2024-05-09T00:41:53.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:52 smithi131 ceph-mon[34547]: osdmap e61: 11 total, 10 up, 11 in 2024-05-09T00:41:53.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:52 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T00:41:53.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:52 smithi131 ceph-mon[34547]: from='client.? 172.21.15.131:0/533457438' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-10/activate.monmap"}]: dispatch 2024-05-09T00:41:55.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:54 smithi157 ceph-mon[34294]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:41:55.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:54 smithi005 ceph-mon[30365]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:41:55.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:54 smithi131 ceph-mon[34547]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:41:57.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:56 smithi131 ceph-mon[34547]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:41:57.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:56 smithi157 ceph-mon[34294]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:41:57.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:56 smithi005 ceph-mon[30365]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:41:59.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:58 smithi131 ceph-mon[34547]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:41:59.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:58 smithi157 ceph-mon[34294]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:41:59.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:58 smithi005 ceph-mon[30365]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:42:00.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:59 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-09T00:42:00.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:41:59 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:42:00.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:59 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-09T00:42:00.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:41:59 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:42:00.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:59 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-09T00:42:00.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:41:59 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:42:01.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:00 smithi131 ceph-mon[34547]: Deploying daemon osd.10 on smithi131 2024-05-09T00:42:01.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:00 smithi131 ceph-mon[34547]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:42:01.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:00 smithi157 ceph-mon[34294]: Deploying daemon osd.10 on smithi131 2024-05-09T00:42:01.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:00 smithi157 ceph-mon[34294]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:42:01.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:00 smithi005 ceph-mon[30365]: Deploying daemon osd.10 on smithi131 2024-05-09T00:42:01.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:00 smithi005 ceph-mon[30365]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:42:03.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:02 smithi131 ceph-mon[34547]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:42:03.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:02 smithi157 ceph-mon[34294]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:42:03.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:02 smithi005 ceph-mon[30365]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:42:05.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:04 smithi005 ceph-mon[30365]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:42:05.173 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:04 smithi131 ceph-mon[34547]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:42:05.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:04 smithi157 ceph-mon[34294]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:42:06.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:06 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:06.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:06 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:06.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:06 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:42:06.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:06 smithi131 ceph-mon[34547]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:42:06.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:06 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:06.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:06 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:06.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:06 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:42:06.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:06 smithi005 ceph-mon[30365]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:42:06.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:06 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:06.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:06 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:06.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:06 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:42:06.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:06 smithi157 ceph-mon[34294]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:42:09.095 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:08 smithi131 ceph-mon[34547]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:42:09.147 INFO:teuthology.orchestra.run.smithi131.stdout:Created osd(s) 10 on host 'smithi131' 2024-05-09T00:42:09.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:42:09.145+0000 7fe7c57fa700 1 -- 172.21.15.131:0/389703487 <== mgr.14152 v2:172.21.15.5:6800/91856593 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fe7c8063c00 con 0x7fe7b40b2880 2024-05-09T00:42:09.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:42:09.148+0000 7fe7cdcb6700 1 -- 172.21.15.131:0/389703487 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe7b40b2880 msgr2=0x7fe7b40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:42:09.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:42:09.148+0000 7fe7cdcb6700 1 --2- 172.21.15.131:0/389703487 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe7b40b2880 0x7fe7b40b4d90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe7b80038a0 tx=0x7fe7b801a040 comp rx=0 tx=0).stop 2024-05-09T00:42:09.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:42:09.148+0000 7fe7cdcb6700 1 -- 172.21.15.131:0/389703487 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7c8105900 msgr2=0x7fe7c81a1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:42:09.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:42:09.148+0000 7fe7cdcb6700 1 --2- 172.21.15.131:0/389703487 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7c8105900 0x7fe7c81a1dc0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe7c0007ee0 tx=0x7fe7c000d3b0 comp rx=0 tx=0).stop 2024-05-09T00:42:09.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:42:09.148+0000 7fe7cdcb6700 1 -- 172.21.15.131:0/389703487 shutdown_connections 2024-05-09T00:42:09.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:42:09.148+0000 7fe7cdcb6700 1 --2- 172.21.15.131:0/389703487 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe7b40b2880 0x7fe7b40b4d90 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:42:09.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:42:09.148+0000 7fe7cdcb6700 1 --2- 172.21.15.131:0/389703487 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7c8105900 0x7fe7c81a1dc0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:42:09.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:42:09.148+0000 7fe7cdcb6700 1 --2- 172.21.15.131:0/389703487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7c8102350 0x7fe7c819a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:42:09.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:42:09.148+0000 7fe7cdcb6700 1 --2- 172.21.15.131:0/389703487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7c8102ca0 0x7fe7c819ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:42:09.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:42:09.148+0000 7fe7cdcb6700 1 -- 172.21.15.131:0/389703487 >> 172.21.15.131:0/389703487 conn(0x7fe7c80fe020 msgr2=0x7fe7c81088a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:42:09.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:42:09.148+0000 7fe7cdcb6700 1 -- 172.21.15.131:0/389703487 shutdown_connections 2024-05-09T00:42:09.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-09T00:42:09.148+0000 7fe7cdcb6700 1 -- 172.21.15.131:0/389703487 wait complete. 2024-05-09T00:42:09.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:08 smithi157 ceph-mon[34294]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:42:09.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:08 smithi005 ceph-mon[30365]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:42:09.818 DEBUG:teuthology.orchestra.run.smithi131:osd.10> sudo journalctl -f -n 0 -u ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@osd.10.service 2024-05-09T00:42:09.821 INFO:tasks.cephadm:Deploying osd.11 on smithi157 with /dev/vg_nvme/lv_1... 2024-05-09T00:42:09.821 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-09T00:42:10.068 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T00:42:10.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:10 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:10.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:10 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:10.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:10 smithi157 ceph-mon[34294]: from='osd.10 [v2:172.21.15.131:6824/1571783985,v1:172.21.15.131:6825/1571783985]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-09T00:42:10.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:10 smithi157 ceph-mon[34294]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-09T00:42:10.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:10 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:10.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:10 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:10.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:10 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:10.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:10 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:10.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:10 smithi005 ceph-mon[30365]: from='osd.10 [v2:172.21.15.131:6824/1571783985,v1:172.21.15.131:6825/1571783985]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-09T00:42:10.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:10 smithi005 ceph-mon[30365]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-09T00:42:10.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:10 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:10.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:10 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:10.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:10 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:10.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:10 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:10.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:10 smithi131 ceph-mon[34547]: from='osd.10 [v2:172.21.15.131:6824/1571783985,v1:172.21.15.131:6825/1571783985]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-09T00:42:10.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:10 smithi131 ceph-mon[34547]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-09T00:42:10.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:10 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:10.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:10 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:11.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:11 smithi131 ceph-mon[34547]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:42:11.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:11 smithi131 ceph-mon[34547]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-09T00:42:11.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:11 smithi131 ceph-mon[34547]: osdmap e62: 11 total, 10 up, 11 in 2024-05-09T00:42:11.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:11 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T00:42:11.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:11 smithi131 ceph-mon[34547]: from='osd.10 [v2:172.21.15.131:6824/1571783985,v1:172.21.15.131:6825/1571783985]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-05-09T00:42:11.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:11 smithi131 ceph-mon[34547]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-05-09T00:42:11.357 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 00:42:11 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T00:42:11.152+0000 7f1029f2e700 -1 osd.10 0 waiting for initial osdmap 2024-05-09T00:42:11.357 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 00:42:11 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T00:42:11.163+0000 7f1025ce3700 -1 osd.10 63 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T00:42:11.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:11 smithi157 ceph-mon[34294]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:42:11.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:11 smithi157 ceph-mon[34294]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-09T00:42:11.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:11 smithi157 ceph-mon[34294]: osdmap e62: 11 total, 10 up, 11 in 2024-05-09T00:42:11.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:11 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T00:42:11.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:11 smithi157 ceph-mon[34294]: from='osd.10 [v2:172.21.15.131:6824/1571783985,v1:172.21.15.131:6825/1571783985]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-05-09T00:42:11.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:11 smithi157 ceph-mon[34294]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-05-09T00:42:11.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:11 smithi005 ceph-mon[30365]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:42:11.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:11 smithi005 ceph-mon[30365]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-09T00:42:11.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:11 smithi005 ceph-mon[30365]: osdmap e62: 11 total, 10 up, 11 in 2024-05-09T00:42:11.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:11 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T00:42:11.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:11 smithi005 ceph-mon[30365]: from='osd.10 [v2:172.21.15.131:6824/1571783985,v1:172.21.15.131:6825/1571783985]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-05-09T00:42:11.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:11 smithi005 ceph-mon[30365]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-05-09T00:42:12.187 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-09T00:42:12.187 INFO:teuthology.orchestra.run.smithi157.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-09T00:42:12.187 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10+0 records in 2024-05-09T00:42:12.188 INFO:teuthology.orchestra.run.smithi157.stderr:10+0 records out 2024-05-09T00:42:12.188 INFO:teuthology.orchestra.run.smithi157.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00887832 s, 1.2 GB/s 2024-05-09T00:42:12.188 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping successful for: 2024-05-09T00:42:12.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:12 smithi131 ceph-mon[34547]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-05-09T00:42:12.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:12 smithi131 ceph-mon[34547]: osdmap e63: 11 total, 10 up, 11 in 2024-05-09T00:42:12.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:12 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T00:42:12.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:12 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T00:42:12.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:12 smithi157 ceph-mon[34294]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-05-09T00:42:12.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:12 smithi157 ceph-mon[34294]: osdmap e63: 11 total, 10 up, 11 in 2024-05-09T00:42:12.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:12 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T00:42:12.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:12 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T00:42:12.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:12 smithi005 ceph-mon[30365]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-05-09T00:42:12.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:12 smithi005 ceph-mon[30365]: osdmap e63: 11 total, 10 up, 11 in 2024-05-09T00:42:12.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:12 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T00:42:12.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:12 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T00:42:12.851 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi157:vg_nvme/lv_1 2024-05-09T00:42:13.085 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T00:42:13.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:13 smithi131 ceph-mon[34547]: purged_snaps scrub starts 2024-05-09T00:42:13.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:13 smithi131 ceph-mon[34547]: purged_snaps scrub ok 2024-05-09T00:42:13.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:13 smithi131 ceph-mon[34547]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:42:13.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:13 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T00:42:13.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:13 smithi131 ceph-mon[34547]: osd.10 [v2:172.21.15.131:6824/1571783985,v1:172.21.15.131:6825/1571783985] boot 2024-05-09T00:42:13.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:13 smithi131 ceph-mon[34547]: osdmap e64: 11 total, 11 up, 11 in 2024-05-09T00:42:13.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:13 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T00:42:13.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:13 smithi157 ceph-mon[34294]: purged_snaps scrub starts 2024-05-09T00:42:13.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:13 smithi157 ceph-mon[34294]: purged_snaps scrub ok 2024-05-09T00:42:13.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:13 smithi157 ceph-mon[34294]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:42:13.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:13 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T00:42:13.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:13 smithi157 ceph-mon[34294]: osd.10 [v2:172.21.15.131:6824/1571783985,v1:172.21.15.131:6825/1571783985] boot 2024-05-09T00:42:13.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:13 smithi157 ceph-mon[34294]: osdmap e64: 11 total, 11 up, 11 in 2024-05-09T00:42:13.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:13 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T00:42:13.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:13 smithi005 ceph-mon[30365]: purged_snaps scrub starts 2024-05-09T00:42:13.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:13 smithi005 ceph-mon[30365]: purged_snaps scrub ok 2024-05-09T00:42:13.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:13 smithi005 ceph-mon[30365]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:42:13.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:13 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T00:42:13.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:13 smithi005 ceph-mon[30365]: osd.10 [v2:172.21.15.131:6824/1571783985,v1:172.21.15.131:6825/1571783985] boot 2024-05-09T00:42:13.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:13 smithi005 ceph-mon[30365]: osdmap e64: 11 total, 11 up, 11 in 2024-05-09T00:42:13.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:13 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T00:42:14.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:14 smithi157 ceph-mon[34294]: osdmap e65: 11 total, 11 up, 11 in 2024-05-09T00:42:14.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:14 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:14.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:14 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:14.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:14 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:42:14.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:14 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:42:14.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:14 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:42:14.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:14 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:42:14.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:14 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:14.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:14 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:42:14.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:14 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:42:14.188 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:14 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:14.188 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:14 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:42:14.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:14 smithi005 ceph-mon[30365]: osdmap e65: 11 total, 11 up, 11 in 2024-05-09T00:42:14.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:14 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:14.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:14 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:14.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:14 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:42:14.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:14 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:42:14.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:14 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:42:14.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:14 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:42:14.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:14 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:14.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:14 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:42:14.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:14 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:42:14.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:14 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:14.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:14 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:42:14.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:14 smithi131 ceph-mon[34547]: osdmap e65: 11 total, 11 up, 11 in 2024-05-09T00:42:14.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:14 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:14.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:14 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:14.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:14 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:42:14.607 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:14 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:42:14.607 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:14 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:42:14.607 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:14 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:42:14.607 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:14 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:14.607 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:14 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:42:14.607 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:14 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:42:14.607 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:14 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:14.607 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:14 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:42:14.866 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:14.863+0000 7f078b926700 1 -- 172.21.15.157:0/1054251980 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0784104370 msgr2=0x7f0784104780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:42:14.866 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:14.863+0000 7f078b926700 1 --2- 172.21.15.157:0/1054251980 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0784104370 0x7f0784104780 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0780009990 tx=0x7f0780009ca0 comp rx=0 tx=0).stop 2024-05-09T00:42:14.867 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:14.863+0000 7f078b926700 1 -- 172.21.15.157:0/1054251980 shutdown_connections 2024-05-09T00:42:14.867 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:14.863+0000 7f078b926700 1 --2- 172.21.15.157:0/1054251980 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0784105e40 0x7f078410c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:42:14.867 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:14.863+0000 7f078b926700 1 --2- 172.21.15.157:0/1054251980 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0784104370 0x7f0784104780 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:42:14.867 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:14.863+0000 7f078b926700 1 --2- 172.21.15.157:0/1054251980 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07841054a0 0x7f07841058b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:42:14.867 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:14.863+0000 7f078b926700 1 -- 172.21.15.157:0/1054251980 >> 172.21.15.157:0/1054251980 conn(0x7f07840ff9d0 msgr2=0x7f0784101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:42:14.867 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:14.863+0000 7f078b926700 1 -- 172.21.15.157:0/1054251980 shutdown_connections 2024-05-09T00:42:14.867 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:14.863+0000 7f078b926700 1 -- 172.21.15.157:0/1054251980 wait complete. 2024-05-09T00:42:14.867 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:14.864+0000 7f078b926700 1 Processor -- start 2024-05-09T00:42:14.867 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:14.864+0000 7f078b926700 1 -- start start 2024-05-09T00:42:14.867 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:14.864+0000 7f078b926700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0784104370 0x7f078419a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:42:14.867 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:14.864+0000 7f078b926700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f07841054a0 0x7f078419aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:42:14.867 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:14.864+0000 7f078b926700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0784105e40 0x7f07841a1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:42:14.867 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:14.864+0000 7f078b926700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f078410fe30 con 0x7f0784105e40 2024-05-09T00:42:14.868 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:14.864+0000 7f078b926700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f078410fcb0 con 0x7f07841054a0 2024-05-09T00:42:14.868 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:14.864+0000 7f078b926700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f078410ffb0 con 0x7f0784104370 2024-05-09T00:42:14.868 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:14.864+0000 7f07896c2700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0784104370 0x7f078419a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:42:14.868 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:14.864+0000 7f07896c2700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0784104370 0x7f078419a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:54840/0 (socket says 172.21.15.157:54840) 2024-05-09T00:42:14.868 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:14.864+0000 7f07896c2700 1 -- 172.21.15.157:0/953863312 learned_addr learned my addr 172.21.15.157:0/953863312 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-05-09T00:42:14.868 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:14.864+0000 7f0789ec3700 1 --2- 172.21.15.157:0/953863312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0784105e40 0x7f07841a1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:42:14.868 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:14.864+0000 7f0788ec1700 1 --2- 172.21.15.157:0/953863312 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f07841054a0 0x7f078419aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:42:14.868 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:14.864+0000 7f07896c2700 1 -- 172.21.15.157:0/953863312 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f07841054a0 msgr2=0x7f078419aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:42:14.868 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:14.864+0000 7f07896c2700 1 --2- 172.21.15.157:0/953863312 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f07841054a0 0x7f078419aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:42:14.868 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:14.864+0000 7f07896c2700 1 -- 172.21.15.157:0/953863312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0784105e40 msgr2=0x7f07841a1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:42:14.868 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:14.864+0000 7f07896c2700 1 --2- 172.21.15.157:0/953863312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0784105e40 0x7f07841a1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:42:14.868 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:14.864+0000 7f07896c2700 1 -- 172.21.15.157:0/953863312 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07841a2440 con 0x7f0784104370 2024-05-09T00:42:14.868 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:14.864+0000 7f0789ec3700 1 --2- 172.21.15.157:0/953863312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0784105e40 0x7f07841a1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:42:14.868 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:14.864+0000 7f07896c2700 1 --2- 172.21.15.157:0/953863312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0784104370 0x7f078419a970 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0780000c00 tx=0x7f07800047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:42:14.869 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:14.865+0000 7f077a7fc700 1 -- 172.21.15.157:0/953863312 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0780027070 con 0x7f0784104370 2024-05-09T00:42:14.869 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:14.865+0000 7f077a7fc700 1 -- 172.21.15.157:0/953863312 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0780022470 con 0x7f0784104370 2024-05-09T00:42:14.869 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:14.865+0000 7f078b926700 1 -- 172.21.15.157:0/953863312 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07841a26d0 con 0x7f0784104370 2024-05-09T00:42:14.869 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:14.865+0000 7f077a7fc700 1 -- 172.21.15.157:0/953863312 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f078000f6c0 con 0x7f0784104370 2024-05-09T00:42:14.869 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:14.865+0000 7f078b926700 1 -- 172.21.15.157:0/953863312 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07841a2c00 con 0x7f0784104370 2024-05-09T00:42:14.870 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:14.866+0000 7f078b926700 1 -- 172.21.15.157:0/953863312 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07840512c0 con 0x7f0784104370 2024-05-09T00:42:14.870 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:14.867+0000 7f077a7fc700 1 -- 172.21.15.157:0/953863312 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f078002a070 con 0x7f0784104370 2024-05-09T00:42:14.870 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:14.868+0000 7f077a7fc700 1 --2- 172.21.15.157:0/953863312 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f07700b2900 0x7f07700b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:42:14.870 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:14.868+0000 7f077a7fc700 1 -- 172.21.15.157:0/953863312 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6694+0+0 (secure 0 0 0) 0x7f07800e2ae0 con 0x7f0784104370 2024-05-09T00:42:14.870 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:14.868+0000 7f0788ec1700 1 --2- 172.21.15.157:0/953863312 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f07700b2900 0x7f07700b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:42:14.870 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:14.868+0000 7f0788ec1700 1 --2- 172.21.15.157:0/953863312 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f07700b2900 0x7f07700b4e10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0774005ea0 tx=0x7f0774005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:42:14.874 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:14.873+0000 7f077a7fc700 1 -- 172.21.15.157:0/953863312 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0780013070 con 0x7f0784104370 2024-05-09T00:42:15.087 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:15.086+0000 7f078b926700 1 -- 172.21.15.157:0/953863312 --> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f0784063c00 con 0x7f07700b2900 2024-05-09T00:42:15.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:15 smithi131 ceph-mon[34547]: Detected new or changed devices on smithi131 2024-05-09T00:42:15.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:15 smithi131 ceph-mon[34547]: Adjusting osd_memory_target on smithi131 to 4272M 2024-05-09T00:42:15.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:15 smithi131 ceph-mon[34547]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 983 GiB / 983 GiB avail 2024-05-09T00:42:15.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:15 smithi131 ceph-mon[34547]: osdmap e66: 11 total, 11 up, 11 in 2024-05-09T00:42:15.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:15 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:42:15.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:15 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:42:15.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:15 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:15.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:15 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:42:15.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:15 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:42:15.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:15 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:42:15.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:15 smithi157 ceph-mon[34294]: Detected new or changed devices on smithi131 2024-05-09T00:42:15.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:15 smithi157 ceph-mon[34294]: Adjusting osd_memory_target on smithi131 to 4272M 2024-05-09T00:42:15.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:15 smithi157 ceph-mon[34294]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 983 GiB / 983 GiB avail 2024-05-09T00:42:15.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:15 smithi157 ceph-mon[34294]: osdmap e66: 11 total, 11 up, 11 in 2024-05-09T00:42:15.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:15 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:42:15.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:15 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:42:15.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:15 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:15.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:15 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:42:15.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:15 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:42:15.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:15 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:42:15.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:15 smithi005 ceph-mon[30365]: Detected new or changed devices on smithi131 2024-05-09T00:42:15.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:15 smithi005 ceph-mon[30365]: Adjusting osd_memory_target on smithi131 to 4272M 2024-05-09T00:42:15.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:15 smithi005 ceph-mon[30365]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 983 GiB / 983 GiB avail 2024-05-09T00:42:15.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:15 smithi005 ceph-mon[30365]: osdmap e66: 11 total, 11 up, 11 in 2024-05-09T00:42:15.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:15 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:42:15.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:15 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:42:15.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:15 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:15.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:15 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:42:15.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:15 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:42:15.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:15 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:42:16.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:16 smithi157 ceph-mon[34294]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:42:16.457 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:16 smithi005 ceph-mon[30365]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:42:16.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:16 smithi131 ceph-mon[34547]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:42:17.322 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:17 smithi157 ceph-mon[34294]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-05-09T00:42:17.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:17 smithi005 ceph-mon[30365]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-05-09T00:42:17.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:17 smithi131 ceph-mon[34547]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-05-09T00:42:18.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:18 smithi005 ceph-mon[30365]: from='client.? 172.21.15.157:0/3325229804' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b858453f-eda1-4e25-89c8-94b54c830938"}]: dispatch 2024-05-09T00:42:18.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:18 smithi005 ceph-mon[30365]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b858453f-eda1-4e25-89c8-94b54c830938"}]: dispatch 2024-05-09T00:42:18.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:18 smithi005 ceph-mon[30365]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b858453f-eda1-4e25-89c8-94b54c830938"}]': finished 2024-05-09T00:42:18.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:18 smithi005 ceph-mon[30365]: osdmap e67: 12 total, 11 up, 12 in 2024-05-09T00:42:18.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:18 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T00:42:18.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:18 smithi131 ceph-mon[34547]: from='client.? 172.21.15.157:0/3325229804' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b858453f-eda1-4e25-89c8-94b54c830938"}]: dispatch 2024-05-09T00:42:18.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:18 smithi131 ceph-mon[34547]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b858453f-eda1-4e25-89c8-94b54c830938"}]: dispatch 2024-05-09T00:42:18.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:18 smithi131 ceph-mon[34547]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b858453f-eda1-4e25-89c8-94b54c830938"}]': finished 2024-05-09T00:42:18.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:18 smithi131 ceph-mon[34547]: osdmap e67: 12 total, 11 up, 12 in 2024-05-09T00:42:18.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:18 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T00:42:18.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:18 smithi157 ceph-mon[34294]: from='client.? 172.21.15.157:0/3325229804' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b858453f-eda1-4e25-89c8-94b54c830938"}]: dispatch 2024-05-09T00:42:18.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:18 smithi157 ceph-mon[34294]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b858453f-eda1-4e25-89c8-94b54c830938"}]: dispatch 2024-05-09T00:42:18.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:18 smithi157 ceph-mon[34294]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b858453f-eda1-4e25-89c8-94b54c830938"}]': finished 2024-05-09T00:42:18.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:18 smithi157 ceph-mon[34294]: osdmap e67: 12 total, 11 up, 12 in 2024-05-09T00:42:18.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:18 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T00:42:19.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:19 smithi005 ceph-mon[30365]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-09T00:42:19.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:19 smithi005 ceph-mon[30365]: from='client.? 172.21.15.157:0/3769997779' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-11/activate.monmap"}]: dispatch 2024-05-09T00:42:19.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:19 smithi131 ceph-mon[34547]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-09T00:42:19.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:19 smithi131 ceph-mon[34547]: from='client.? 172.21.15.157:0/3769997779' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-11/activate.monmap"}]: dispatch 2024-05-09T00:42:19.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:19 smithi157 ceph-mon[34294]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-09T00:42:19.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:19 smithi157 ceph-mon[34294]: from='client.? 172.21.15.157:0/3769997779' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-11/activate.monmap"}]: dispatch 2024-05-09T00:42:21.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:21 smithi005 ceph-mon[30365]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-05-09T00:42:21.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:21 smithi131 ceph-mon[34547]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-05-09T00:42:21.683 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:21 smithi157 ceph-mon[34294]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-05-09T00:42:23.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:23 smithi157 ceph-mon[34294]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T00:42:23.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:23 smithi005 ceph-mon[30365]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T00:42:23.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:23 smithi131 ceph-mon[34547]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T00:42:25.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:25 smithi005 ceph-mon[30365]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-09T00:42:25.554 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:25 smithi157 ceph-mon[34294]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-09T00:42:25.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:25 smithi131 ceph-mon[34547]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-09T00:42:26.473 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:26 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-09T00:42:26.473 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:26 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:42:26.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:26 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-09T00:42:26.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:26 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:42:26.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:26 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-09T00:42:26.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:26 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:42:27.521 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:27 smithi157 ceph-mon[34294]: Deploying daemon osd.11 on smithi157 2024-05-09T00:42:27.522 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:27 smithi157 ceph-mon[34294]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-05-09T00:42:27.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:27 smithi005 ceph-mon[30365]: Deploying daemon osd.11 on smithi157 2024-05-09T00:42:27.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:27 smithi005 ceph-mon[30365]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-05-09T00:42:27.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:27 smithi131 ceph-mon[34547]: Deploying daemon osd.11 on smithi157 2024-05-09T00:42:27.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:27 smithi131 ceph-mon[34547]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-05-09T00:42:29.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:29 smithi157 ceph-mon[34294]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-05-09T00:42:29.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:29 smithi005 ceph-mon[30365]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-05-09T00:42:29.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:29 smithi131 ceph-mon[34547]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-05-09T00:42:31.414 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:31 smithi157 ceph-mon[34294]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-05-09T00:42:31.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:31 smithi005 ceph-mon[30365]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-05-09T00:42:31.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:31 smithi131 ceph-mon[34547]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-05-09T00:42:33.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:32 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:33.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:32 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:33.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:32 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:42:33.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:32 smithi005 ceph-mon[30365]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-05-09T00:42:33.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:32 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:33.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:32 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:33.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:32 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:42:33.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:32 smithi131 ceph-mon[34547]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-05-09T00:42:33.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:32 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:33.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:32 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:33.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:32 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:42:33.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:32 smithi157 ceph-mon[34294]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-05-09T00:42:35.162 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:34 smithi157 ceph-mon[34294]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-05-09T00:42:35.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:34 smithi005 ceph-mon[30365]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-05-09T00:42:35.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:34 smithi131 ceph-mon[34547]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-05-09T00:42:35.561 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:35.559+0000 7f077a7fc700 1 -- 172.21.15.157:0/953863312 <== mgr.14152 v2:172.21.15.5:6800/91856593 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f0784063c00 con 0x7f07700b2900 2024-05-09T00:42:35.561 INFO:teuthology.orchestra.run.smithi157.stdout:Created osd(s) 11 on host 'smithi157' 2024-05-09T00:42:35.564 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:35.563+0000 7f078b926700 1 -- 172.21.15.157:0/953863312 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f07700b2900 msgr2=0x7f07700b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:42:35.564 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:35.563+0000 7f078b926700 1 --2- 172.21.15.157:0/953863312 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f07700b2900 0x7f07700b4e10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0774005ea0 tx=0x7f0774005e30 comp rx=0 tx=0).stop 2024-05-09T00:42:35.565 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:35.563+0000 7f078b926700 1 -- 172.21.15.157:0/953863312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0784104370 msgr2=0x7f078419a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:42:35.565 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:35.563+0000 7f078b926700 1 --2- 172.21.15.157:0/953863312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0784104370 0x7f078419a970 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0780000c00 tx=0x7f07800047b0 comp rx=0 tx=0).stop 2024-05-09T00:42:35.565 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:35.563+0000 7f078b926700 1 -- 172.21.15.157:0/953863312 shutdown_connections 2024-05-09T00:42:35.565 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:35.563+0000 7f078b926700 1 --2- 172.21.15.157:0/953863312 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f07700b2900 0x7f07700b4e10 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:42:35.565 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:35.563+0000 7f078b926700 1 --2- 172.21.15.157:0/953863312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0784105e40 0x7f07841a1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:42:35.565 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:35.563+0000 7f078b926700 1 --2- 172.21.15.157:0/953863312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0784104370 0x7f078419a970 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:42:35.565 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:35.563+0000 7f078b926700 1 --2- 172.21.15.157:0/953863312 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f07841054a0 0x7f078419aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:42:35.565 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:35.563+0000 7f078b926700 1 -- 172.21.15.157:0/953863312 >> 172.21.15.157:0/953863312 conn(0x7f07840ff9d0 msgr2=0x7f07841086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:42:35.565 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:35.563+0000 7f078b926700 1 -- 172.21.15.157:0/953863312 shutdown_connections 2024-05-09T00:42:35.565 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:35.563+0000 7f078b926700 1 -- 172.21.15.157:0/953863312 wait complete. 2024-05-09T00:42:36.235 DEBUG:teuthology.orchestra.run.smithi157:osd.11> sudo journalctl -f -n 0 -u ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@osd.11.service 2024-05-09T00:42:36.237 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-05-09T00:42:36.237 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph osd stat -f json 2024-05-09T00:42:36.507 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:42:36.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:36 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:36.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:36 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:36.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:36 smithi005 ceph-mon[30365]: from='osd.11 [v2:172.21.15.157:6824/3261151188,v1:172.21.15.157:6825/3261151188]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-09T00:42:36.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:36 smithi005 ceph-mon[30365]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-09T00:42:36.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:36 smithi005 ceph-mon[30365]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-05-09T00:42:36.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:36 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:36.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:36 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:36.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:36 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:36.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:36 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:36.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:36 smithi131 ceph-mon[34547]: from='osd.11 [v2:172.21.15.157:6824/3261151188,v1:172.21.15.157:6825/3261151188]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-09T00:42:36.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:36 smithi131 ceph-mon[34547]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-09T00:42:36.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:36 smithi131 ceph-mon[34547]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-05-09T00:42:36.857 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:36 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:36.857 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:36 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:36.904 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:36 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:36.904 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:36 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:36.904 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:36 smithi157 ceph-mon[34294]: from='osd.11 [v2:172.21.15.157:6824/3261151188,v1:172.21.15.157:6825/3261151188]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-09T00:42:36.904 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:36 smithi157 ceph-mon[34294]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-09T00:42:36.905 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:36 smithi157 ceph-mon[34294]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-05-09T00:42:36.905 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:36 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:36.905 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:36 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:37.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:37 smithi005 ceph-mon[30365]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-09T00:42:37.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:37 smithi005 ceph-mon[30365]: from='osd.11 [v2:172.21.15.157:6824/3261151188,v1:172.21.15.157:6825/3261151188]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-09T00:42:37.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:37 smithi005 ceph-mon[30365]: osdmap e68: 12 total, 11 up, 12 in 2024-05-09T00:42:37.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:37 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T00:42:37.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:37 smithi005 ceph-mon[30365]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-09T00:42:37.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:37 smithi131 ceph-mon[34547]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-09T00:42:37.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:37 smithi131 ceph-mon[34547]: from='osd.11 [v2:172.21.15.157:6824/3261151188,v1:172.21.15.157:6825/3261151188]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-09T00:42:37.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:37 smithi131 ceph-mon[34547]: osdmap e68: 12 total, 11 up, 12 in 2024-05-09T00:42:37.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:37 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T00:42:37.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:37 smithi131 ceph-mon[34547]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-09T00:42:37.929 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:37 smithi157 ceph-mon[34294]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-09T00:42:37.929 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:37 smithi157 ceph-mon[34294]: from='osd.11 [v2:172.21.15.157:6824/3261151188,v1:172.21.15.157:6825/3261151188]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-09T00:42:37.929 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:37 smithi157 ceph-mon[34294]: osdmap e68: 12 total, 11 up, 12 in 2024-05-09T00:42:37.929 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:37 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T00:42:37.930 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:37 smithi157 ceph-mon[34294]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-05-09T00:42:37.930 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 00:42:37 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T00:42:37.528+0000 7f3ba4669700 -1 osd.11 0 waiting for initial osdmap 2024-05-09T00:42:37.930 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 00:42:37 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T00:42:37.545+0000 7f3ba041e700 -1 osd.11 69 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T00:42:38.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:38.266+0000 7feaf0bd6700 1 -- 172.21.15.5:0/708750381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feaec105e40 msgr2=0x7feaec10c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:42:38.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:38.266+0000 7feaf0bd6700 1 --2- 172.21.15.5:0/708750381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feaec105e40 0x7feaec10c6f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7feae400b0a0 tx=0x7feae400b3b0 comp rx=0 tx=0).stop 2024-05-09T00:42:38.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:38.266+0000 7feaf0bd6700 1 -- 172.21.15.5:0/708750381 shutdown_connections 2024-05-09T00:42:38.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:38.266+0000 7feaf0bd6700 1 --2- 172.21.15.5:0/708750381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feaec105e40 0x7feaec10c6f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:42:38.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:38.266+0000 7feaf0bd6700 1 --2- 172.21.15.5:0/708750381 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feaec104370 0x7feaec104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:42:38.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:38.266+0000 7feaf0bd6700 1 --2- 172.21.15.5:0/708750381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feaec1054a0 0x7feaec1058b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:42:38.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:38.266+0000 7feaf0bd6700 1 -- 172.21.15.5:0/708750381 >> 172.21.15.5:0/708750381 conn(0x7feaec0ff9d0 msgr2=0x7feaec101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:42:38.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:38.266+0000 7feaf0bd6700 1 -- 172.21.15.5:0/708750381 shutdown_connections 2024-05-09T00:42:38.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:38.266+0000 7feaf0bd6700 1 -- 172.21.15.5:0/708750381 wait complete. 2024-05-09T00:42:38.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:38.267+0000 7feaf0bd6700 1 Processor -- start 2024-05-09T00:42:38.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:38.267+0000 7feaf0bd6700 1 -- start start 2024-05-09T00:42:38.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:38.267+0000 7feaf0bd6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feaec104370 0x7feaec078570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:42:38.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:38.267+0000 7feaf0bd6700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feaec1054a0 0x7feaec076c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:42:38.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:38.267+0000 7feaf0bd6700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feaec105e40 0x7feaec077160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:42:38.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:38.267+0000 7feaf0bd6700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7feaec10fde0 con 0x7feaec104370 2024-05-09T00:42:38.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:38.267+0000 7feaf0bd6700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7feaec10fc60 con 0x7feaec105e40 2024-05-09T00:42:38.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:38.267+0000 7feaf0bd6700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7feaec10ff60 con 0x7feaec1054a0 2024-05-09T00:42:38.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:38.267+0000 7feaea59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feaec104370 0x7feaec078570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:42:38.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:38.267+0000 7feaea59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feaec104370 0x7feaec078570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43434/0 (socket says 172.21.15.5:43434) 2024-05-09T00:42:38.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:38.267+0000 7feaea59c700 1 -- 172.21.15.5:0/1796061277 learned_addr learned my addr 172.21.15.5:0/1796061277 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:42:38.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:38.267+0000 7feae9d9b700 1 --2- 172.21.15.5:0/1796061277 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feaec1054a0 0x7feaec076c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:42:38.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:38.267+0000 7feaead9d700 1 --2- 172.21.15.5:0/1796061277 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feaec105e40 0x7feaec077160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:42:38.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:38.267+0000 7feaea59c700 1 -- 172.21.15.5:0/1796061277 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feaec1054a0 msgr2=0x7feaec076c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:42:38.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:38.267+0000 7feaea59c700 1 --2- 172.21.15.5:0/1796061277 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feaec1054a0 0x7feaec076c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:42:38.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:38.267+0000 7feaea59c700 1 -- 172.21.15.5:0/1796061277 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feaec105e40 msgr2=0x7feaec077160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:42:38.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:38.267+0000 7feaea59c700 1 --2- 172.21.15.5:0/1796061277 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feaec105e40 0x7feaec077160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:42:38.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:38.267+0000 7feaea59c700 1 -- 172.21.15.5:0/1796061277 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaec0778c0 con 0x7feaec104370 2024-05-09T00:42:38.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:38.267+0000 7feaead9d700 1 --2- 172.21.15.5:0/1796061277 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feaec105e40 0x7feaec077160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:42:38.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:38.268+0000 7feaea59c700 1 --2- 172.21.15.5:0/1796061277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feaec104370 0x7feaec078570 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7feadc00d950 tx=0x7feadc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:42:38.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:38.268+0000 7feadb7fe700 1 -- 172.21.15.5:0/1796061277 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feadc017070 con 0x7feaec104370 2024-05-09T00:42:38.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:38.268+0000 7feadb7fe700 1 -- 172.21.15.5:0/1796061277 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feadc009730 con 0x7feaec104370 2024-05-09T00:42:38.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:38.268+0000 7feaf0bd6700 1 -- 172.21.15.5:0/1796061277 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaec1a4960 con 0x7feaec104370 2024-05-09T00:42:38.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:38.268+0000 7feadb7fe700 1 -- 172.21.15.5:0/1796061277 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feadc003a50 con 0x7feaec104370 2024-05-09T00:42:38.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:38.268+0000 7feaf0bd6700 1 -- 172.21.15.5:0/1796061277 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaec1a4e40 con 0x7feaec104370 2024-05-09T00:42:38.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:38.269+0000 7feaf0bd6700 1 -- 172.21.15.5:0/1796061277 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feaec0512c0 con 0x7feaec104370 2024-05-09T00:42:38.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:38.272+0000 7feadb7fe700 1 -- 172.21.15.5:0/1796061277 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7feadc003bf0 con 0x7feaec104370 2024-05-09T00:42:38.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:38.272+0000 7feadb7fe700 1 --2- 172.21.15.5:0/1796061277 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fead40b2880 0x7fead40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:42:38.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:38.273+0000 7feae9d9b700 1 --2- 172.21.15.5:0/1796061277 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fead40b2880 0x7fead40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:42:38.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:38.273+0000 7feadb7fe700 1 -- 172.21.15.5:0/1796061277 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6847+0+0 (secure 0 0 0) 0x7feadc0e1f50 con 0x7feaec104370 2024-05-09T00:42:38.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:38.273+0000 7feae9d9b700 1 --2- 172.21.15.5:0/1796061277 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fead40b2880 0x7fead40b4d90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7feaec077f20 tx=0x7feae00073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:42:38.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:38.276+0000 7feadb7fe700 1 -- 172.21.15.5:0/1796061277 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7feadc020030 con 0x7feaec104370 2024-05-09T00:42:38.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:38.490+0000 7feaf0bd6700 1 -- 172.21.15.5:0/1796061277 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7feaec02ea30 con 0x7feaec104370 2024-05-09T00:42:38.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:38.491+0000 7feadb7fe700 1 -- 172.21.15.5:0/1796061277 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v69) ==== 74+0+133 (secure 0 0 0) 0x7feaec02ea30 con 0x7feaec104370 2024-05-09T00:42:38.493 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:42:38.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:38.493+0000 7feaf0bd6700 1 -- 172.21.15.5:0/1796061277 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fead40b2880 msgr2=0x7fead40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:42:38.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:38.493+0000 7feaf0bd6700 1 --2- 172.21.15.5:0/1796061277 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fead40b2880 0x7fead40b4d90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7feaec077f20 tx=0x7feae00073d0 comp rx=0 tx=0).stop 2024-05-09T00:42:38.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:38.493+0000 7feaf0bd6700 1 -- 172.21.15.5:0/1796061277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feaec104370 msgr2=0x7feaec078570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:42:38.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:38.493+0000 7feaf0bd6700 1 --2- 172.21.15.5:0/1796061277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feaec104370 0x7feaec078570 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7feadc00d950 tx=0x7feadc00dc60 comp rx=0 tx=0).stop 2024-05-09T00:42:38.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:38.493+0000 7feaf0bd6700 1 -- 172.21.15.5:0/1796061277 shutdown_connections 2024-05-09T00:42:38.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:38.493+0000 7feaf0bd6700 1 --2- 172.21.15.5:0/1796061277 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fead40b2880 0x7fead40b4d90 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:42:38.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:38.493+0000 7feaf0bd6700 1 --2- 172.21.15.5:0/1796061277 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feaec105e40 0x7feaec077160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:42:38.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:38.493+0000 7feaf0bd6700 1 --2- 172.21.15.5:0/1796061277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feaec104370 0x7feaec078570 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:42:38.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:38.493+0000 7feaf0bd6700 1 --2- 172.21.15.5:0/1796061277 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feaec1054a0 0x7feaec076c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:42:38.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:38.493+0000 7feaf0bd6700 1 -- 172.21.15.5:0/1796061277 >> 172.21.15.5:0/1796061277 conn(0x7feaec0ff9d0 msgr2=0x7feaec1086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:42:38.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:38.494+0000 7feaf0bd6700 1 -- 172.21.15.5:0/1796061277 shutdown_connections 2024-05-09T00:42:38.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:38.494+0000 7feaf0bd6700 1 -- 172.21.15.5:0/1796061277 wait complete. 2024-05-09T00:42:38.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:38 smithi005 ceph-mon[30365]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-05-09T00:42:38.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:38 smithi005 ceph-mon[30365]: osdmap e69: 12 total, 11 up, 12 in 2024-05-09T00:42:38.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:38 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T00:42:38.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:38 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T00:42:38.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:38 smithi005 ceph-mon[30365]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-05-09T00:42:38.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:38 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/1796061277' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-09T00:42:38.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:38 smithi131 ceph-mon[34547]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-05-09T00:42:38.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:38 smithi131 ceph-mon[34547]: osdmap e69: 12 total, 11 up, 12 in 2024-05-09T00:42:38.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:38 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T00:42:38.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:38 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T00:42:38.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:38 smithi131 ceph-mon[34547]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-05-09T00:42:38.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:38 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/1796061277' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-09T00:42:38.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:38 smithi157 ceph-mon[34294]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-05-09T00:42:38.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:38 smithi157 ceph-mon[34294]: osdmap e69: 12 total, 11 up, 12 in 2024-05-09T00:42:38.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:38 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T00:42:38.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:38 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T00:42:38.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:38 smithi157 ceph-mon[34294]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-05-09T00:42:38.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:38 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/1796061277' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-09T00:42:39.142 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":69,"num_osds":12,"num_up_osds":11,"osd_up_since":1715215332,"num_in_osds":12,"osd_in_since":1715215337,"num_remapped_pgs":0} 2024-05-09T00:42:39.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:39 smithi157 ceph-mon[34294]: purged_snaps scrub starts 2024-05-09T00:42:39.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:39 smithi157 ceph-mon[34294]: purged_snaps scrub ok 2024-05-09T00:42:39.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:39 smithi157 ceph-mon[34294]: osd.11 [v2:172.21.15.157:6824/3261151188,v1:172.21.15.157:6825/3261151188] boot 2024-05-09T00:42:39.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:39 smithi157 ceph-mon[34294]: osdmap e70: 12 total, 12 up, 12 in 2024-05-09T00:42:39.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:39 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T00:42:39.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:39 smithi005 ceph-mon[30365]: purged_snaps scrub starts 2024-05-09T00:42:39.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:39 smithi005 ceph-mon[30365]: purged_snaps scrub ok 2024-05-09T00:42:39.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:39 smithi005 ceph-mon[30365]: osd.11 [v2:172.21.15.157:6824/3261151188,v1:172.21.15.157:6825/3261151188] boot 2024-05-09T00:42:39.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:39 smithi005 ceph-mon[30365]: osdmap e70: 12 total, 12 up, 12 in 2024-05-09T00:42:39.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:39 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T00:42:39.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:39 smithi131 ceph-mon[34547]: purged_snaps scrub starts 2024-05-09T00:42:39.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:39 smithi131 ceph-mon[34547]: purged_snaps scrub ok 2024-05-09T00:42:39.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:39 smithi131 ceph-mon[34547]: osd.11 [v2:172.21.15.157:6824/3261151188,v1:172.21.15.157:6825/3261151188] boot 2024-05-09T00:42:39.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:39 smithi131 ceph-mon[34547]: osdmap e70: 12 total, 12 up, 12 in 2024-05-09T00:42:39.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:39 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T00:42:40.143 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph osd stat -f json 2024-05-09T00:42:40.396 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:42:40.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:40 smithi005 ceph-mon[30365]: osdmap e71: 12 total, 12 up, 12 in 2024-05-09T00:42:40.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:40 smithi005 ceph-mon[30365]: Detected new or changed devices on smithi157 2024-05-09T00:42:40.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:40 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:40.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:40 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:40.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:40 smithi005 ceph-mon[30365]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:42:40.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:40 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:42:40.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:40 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:42:40.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:40 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:42:40.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:40 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:42:40.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:40 smithi005 ceph-mon[30365]: Adjusting osd_memory_target on smithi157 to 4272M 2024-05-09T00:42:40.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:40 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:40.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:40 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:42:40.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:40 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:42:40.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:40 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:40.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:40 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:42:40.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:40 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:42:40.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:40 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:42:40.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:40 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:40.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:40 smithi131 ceph-mon[34547]: osdmap e71: 12 total, 12 up, 12 in 2024-05-09T00:42:40.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:40 smithi131 ceph-mon[34547]: Detected new or changed devices on smithi157 2024-05-09T00:42:40.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:40 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:40.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:40 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:40.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:40 smithi131 ceph-mon[34547]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:42:40.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:40 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:42:40.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:40 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:42:40.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:40 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:42:40.857 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:40 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:42:40.857 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:40 smithi131 ceph-mon[34547]: Adjusting osd_memory_target on smithi157 to 4272M 2024-05-09T00:42:40.857 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:40 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:40.857 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:40 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:42:40.857 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:40 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:42:40.857 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:40 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:40.857 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:40 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:42:40.857 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:40 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:42:40.857 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:40 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:42:40.857 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:40 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:40.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:40 smithi157 ceph-mon[34294]: osdmap e71: 12 total, 12 up, 12 in 2024-05-09T00:42:40.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:40 smithi157 ceph-mon[34294]: Detected new or changed devices on smithi157 2024-05-09T00:42:40.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:40 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:40.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:40 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:40.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:40 smithi157 ceph-mon[34294]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:42:40.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:40 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:42:40.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:40 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:42:40.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:40 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:42:40.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:40 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:42:40.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:40 smithi157 ceph-mon[34294]: Adjusting osd_memory_target on smithi157 to 4272M 2024-05-09T00:42:40.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:40 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:40.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:40 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:42:40.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:40 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:42:40.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:40 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:40.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:40 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:42:40.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:40 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:42:40.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:40 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:42:40.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:40 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:42.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:42.063+0000 7fda703f1700 1 -- 172.21.15.5:0/3510395404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda68105490 msgr2=0x7fda681058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:42:42.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:42.063+0000 7fda703f1700 1 --2- 172.21.15.5:0/3510395404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda68105490 0x7fda681058a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fda58009990 tx=0x7fda58009ca0 comp rx=0 tx=0).stop 2024-05-09T00:42:42.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:42.063+0000 7fda703f1700 1 -- 172.21.15.5:0/3510395404 shutdown_connections 2024-05-09T00:42:42.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:42.063+0000 7fda703f1700 1 --2- 172.21.15.5:0/3510395404 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda68105e30 0x7fda6810c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:42:42.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:42.063+0000 7fda703f1700 1 --2- 172.21.15.5:0/3510395404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda68104360 0x7fda68104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:42:42.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:42.063+0000 7fda703f1700 1 --2- 172.21.15.5:0/3510395404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda68105490 0x7fda681058a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:42:42.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:42.063+0000 7fda703f1700 1 -- 172.21.15.5:0/3510395404 >> 172.21.15.5:0/3510395404 conn(0x7fda680ff9c0 msgr2=0x7fda68101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:42:42.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:42.064+0000 7fda703f1700 1 -- 172.21.15.5:0/3510395404 shutdown_connections 2024-05-09T00:42:42.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:42.064+0000 7fda703f1700 1 -- 172.21.15.5:0/3510395404 wait complete. 2024-05-09T00:42:42.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:42.064+0000 7fda703f1700 1 Processor -- start 2024-05-09T00:42:42.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:42.064+0000 7fda703f1700 1 -- start start 2024-05-09T00:42:42.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:42.065+0000 7fda703f1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda68104360 0x7fda6819a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:42:42.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:42.065+0000 7fda703f1700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda68105490 0x7fda6819ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:42:42.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:42.065+0000 7fda703f1700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda68105e30 0x7fda681a1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:42:42.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:42.065+0000 7fda703f1700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fda6810fdc0 con 0x7fda68104360 2024-05-09T00:42:42.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:42.065+0000 7fda703f1700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fda6810fc40 con 0x7fda68105e30 2024-05-09T00:42:42.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:42.065+0000 7fda703f1700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fda6810ff40 con 0x7fda68105490 2024-05-09T00:42:42.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:42.065+0000 7fda6e18d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda68104360 0x7fda6819a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:42:42.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:42.065+0000 7fda6e18d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda68104360 0x7fda6819a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43462/0 (socket says 172.21.15.5:43462) 2024-05-09T00:42:42.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:42.065+0000 7fda6e18d700 1 -- 172.21.15.5:0/3779943144 learned_addr learned my addr 172.21.15.5:0/3779943144 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:42:42.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:42.065+0000 7fda6d98c700 1 --2- 172.21.15.5:0/3779943144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda68105490 0x7fda6819ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:42:42.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:42.065+0000 7fda6e98e700 1 --2- 172.21.15.5:0/3779943144 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda68105e30 0x7fda681a1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:42:42.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:42.065+0000 7fda6e18d700 1 -- 172.21.15.5:0/3779943144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda68105490 msgr2=0x7fda6819ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:42:42.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:42.065+0000 7fda6e18d700 1 --2- 172.21.15.5:0/3779943144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda68105490 0x7fda6819ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:42:42.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:42.065+0000 7fda6e18d700 1 -- 172.21.15.5:0/3779943144 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda68105e30 msgr2=0x7fda681a1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:42:42.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:42.065+0000 7fda6e18d700 1 --2- 172.21.15.5:0/3779943144 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda68105e30 0x7fda681a1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:42:42.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:42.065+0000 7fda6e18d700 1 -- 172.21.15.5:0/3779943144 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda681a2490 con 0x7fda68104360 2024-05-09T00:42:42.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:42.065+0000 7fda6e18d700 1 --2- 172.21.15.5:0/3779943144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda68104360 0x7fda6819a930 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fda6400ada0 tx=0x7fda6400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:42:42.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:42.066+0000 7fda5f7fe700 1 -- 172.21.15.5:0/3779943144 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda64017020 con 0x7fda68104360 2024-05-09T00:42:42.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:42.066+0000 7fda5f7fe700 1 -- 172.21.15.5:0/3779943144 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda6400a0d0 con 0x7fda68104360 2024-05-09T00:42:42.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:42.066+0000 7fda5f7fe700 1 -- 172.21.15.5:0/3779943144 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda64004280 con 0x7fda68104360 2024-05-09T00:42:42.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:42.066+0000 7fda703f1700 1 -- 172.21.15.5:0/3779943144 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda681a2780 con 0x7fda68104360 2024-05-09T00:42:42.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:42.066+0000 7fda703f1700 1 -- 172.21.15.5:0/3779943144 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda681a2c50 con 0x7fda68104360 2024-05-09T00:42:42.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:42.067+0000 7fda703f1700 1 -- 172.21.15.5:0/3779943144 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda680512c0 con 0x7fda68104360 2024-05-09T00:42:42.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:42.069+0000 7fda5f7fe700 1 -- 172.21.15.5:0/3779943144 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fda64004ad0 con 0x7fda68104360 2024-05-09T00:42:42.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:42.069+0000 7fda5f7fe700 1 --2- 172.21.15.5:0/3779943144 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fda540b2900 0x7fda540b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:42:42.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:42.069+0000 7fda6d98c700 1 --2- 172.21.15.5:0/3779943144 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fda540b2900 0x7fda540b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:42:42.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:42.069+0000 7fda5f7fe700 1 -- 172.21.15.5:0/3779943144 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fda640e1700 con 0x7fda68104360 2024-05-09T00:42:42.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:42.070+0000 7fda6d98c700 1 --2- 172.21.15.5:0/3779943144 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fda540b2900 0x7fda540b4e10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fda5800b5c0 tx=0x7fda5801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:42:42.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:42.073+0000 7fda5f7fe700 1 -- 172.21.15.5:0/3779943144 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fda64021020 con 0x7fda68104360 2024-05-09T00:42:42.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:42.290+0000 7fda703f1700 1 -- 172.21.15.5:0/3779943144 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fda6802e5f0 con 0x7fda68104360 2024-05-09T00:42:42.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:42.290+0000 7fda5f7fe700 1 -- 172.21.15.5:0/3779943144 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v71) ==== 74+0+133 (secure 0 0 0) 0x7fda640a9950 con 0x7fda68104360 2024-05-09T00:42:42.293 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:42:42.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:42.292+0000 7fda703f1700 1 -- 172.21.15.5:0/3779943144 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fda540b2900 msgr2=0x7fda540b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:42:42.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:42.292+0000 7fda703f1700 1 --2- 172.21.15.5:0/3779943144 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fda540b2900 0x7fda540b4e10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fda5800b5c0 tx=0x7fda5801a040 comp rx=0 tx=0).stop 2024-05-09T00:42:42.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:42.292+0000 7fda703f1700 1 -- 172.21.15.5:0/3779943144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda68104360 msgr2=0x7fda6819a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:42:42.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:42.292+0000 7fda703f1700 1 --2- 172.21.15.5:0/3779943144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda68104360 0x7fda6819a930 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fda6400ada0 tx=0x7fda6400c330 comp rx=0 tx=0).stop 2024-05-09T00:42:42.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:42.292+0000 7fda703f1700 1 -- 172.21.15.5:0/3779943144 shutdown_connections 2024-05-09T00:42:42.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:42.292+0000 7fda703f1700 1 --2- 172.21.15.5:0/3779943144 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fda540b2900 0x7fda540b4e10 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:42:42.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:42.292+0000 7fda703f1700 1 --2- 172.21.15.5:0/3779943144 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda68105e30 0x7fda681a1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:42:42.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:42.292+0000 7fda703f1700 1 --2- 172.21.15.5:0/3779943144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda68104360 0x7fda6819a930 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:42:42.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:42.292+0000 7fda703f1700 1 --2- 172.21.15.5:0/3779943144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda68105490 0x7fda6819ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:42:42.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:42.292+0000 7fda703f1700 1 -- 172.21.15.5:0/3779943144 >> 172.21.15.5:0/3779943144 conn(0x7fda680ff9c0 msgr2=0x7fda681086c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:42:42.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:42.293+0000 7fda703f1700 1 -- 172.21.15.5:0/3779943144 shutdown_connections 2024-05-09T00:42:42.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:42.293+0000 7fda703f1700 1 -- 172.21.15.5:0/3779943144 wait complete. 2024-05-09T00:42:43.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:42 smithi005 ceph-mon[30365]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:42:43.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:42 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/3779943144' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-09T00:42:43.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:42 smithi157 ceph-mon[34294]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:42:43.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:42 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/3779943144' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-09T00:42:43.198 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":71,"num_osds":12,"num_up_osds":12,"osd_up_since":1715215358,"num_in_osds":12,"osd_in_since":1715215337,"num_remapped_pgs":0} 2024-05-09T00:42:43.199 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T00:42:43.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:42 smithi131 ceph-mon[34547]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:42:43.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:42 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/3779943144' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-09T00:42:43.453 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:42:45.010 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:44 smithi005 ceph-mon[30365]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:42:45.191 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:44 smithi157 ceph-mon[34294]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:42:45.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:45.289+0000 7f0e1e95b700 1 -- 172.21.15.5:0/743095734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e18107df0 msgr2=0x7f0e18108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:42:45.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:45.289+0000 7f0e1e95b700 1 --2- 172.21.15.5:0/743095734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e18107df0 0x7f0e18108200 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0e000099c0 tx=0x7f0e00009cd0 comp rx=0 tx=0).stop 2024-05-09T00:42:45.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:45.289+0000 7f0e1e95b700 1 -- 172.21.15.5:0/743095734 shutdown_connections 2024-05-09T00:42:45.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:45.289+0000 7f0e1e95b700 1 --2- 172.21.15.5:0/743095734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e18108740 0x7f0e181100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:42:45.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:45.289+0000 7f0e1e95b700 1 --2- 172.21.15.5:0/743095734 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e181034c0 0x7f0e181038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:42:45.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:45.289+0000 7f0e1e95b700 1 --2- 172.21.15.5:0/743095734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e18107df0 0x7f0e18108200 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:42:45.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:45.289+0000 7f0e1e95b700 1 -- 172.21.15.5:0/743095734 >> 172.21.15.5:0/743095734 conn(0x7f0e180fedc0 msgr2=0x7f0e181011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:42:45.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:45.289+0000 7f0e1e95b700 1 -- 172.21.15.5:0/743095734 shutdown_connections 2024-05-09T00:42:45.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:45.289+0000 7f0e1e95b700 1 -- 172.21.15.5:0/743095734 wait complete. 2024-05-09T00:42:45.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:45.290+0000 7f0e1e95b700 1 Processor -- start 2024-05-09T00:42:45.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:45.290+0000 7f0e1e95b700 1 -- start start 2024-05-09T00:42:45.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:45.290+0000 7f0e1e95b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e181034c0 0x7f0e1819ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:42:45.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:45.290+0000 7f0e1e95b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e18107df0 0x7f0e1819b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:42:45.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:45.290+0000 7f0e1e95b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e18108740 0x7f0e1819f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:42:45.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:45.290+0000 7f0e1e95b700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0e181133c0 con 0x7f0e18107df0 2024-05-09T00:42:45.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:45.290+0000 7f0e1e95b700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0e18113240 con 0x7f0e18108740 2024-05-09T00:42:45.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:45.290+0000 7f0e1e95b700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0e18113540 con 0x7f0e181034c0 2024-05-09T00:42:45.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:45.290+0000 7f0e0ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e18107df0 0x7f0e1819b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:42:45.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:45.290+0000 7f0e0ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e18107df0 0x7f0e1819b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50790/0 (socket says 172.21.15.5:50790) 2024-05-09T00:42:45.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:45.290+0000 7f0e0ffff700 1 -- 172.21.15.5:0/1477373228 learned_addr learned my addr 172.21.15.5:0/1477373228 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:42:45.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:45.290+0000 7f0e17fff700 1 --2- 172.21.15.5:0/1477373228 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e181034c0 0x7f0e1819ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:42:45.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:45.291+0000 7f0e0ffff700 1 -- 172.21.15.5:0/1477373228 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e181034c0 msgr2=0x7f0e1819ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:42:45.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:45.291+0000 7f0e0ffff700 1 --2- 172.21.15.5:0/1477373228 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e181034c0 0x7f0e1819ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:42:45.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:45.291+0000 7f0e1cef8700 1 --2- 172.21.15.5:0/1477373228 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e18108740 0x7f0e1819f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:42:45.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:45.291+0000 7f0e0ffff700 1 -- 172.21.15.5:0/1477373228 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e18108740 msgr2=0x7f0e1819f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:42:45.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:45.291+0000 7f0e0ffff700 1 --2- 172.21.15.5:0/1477373228 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e18108740 0x7f0e1819f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:42:45.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:45.291+0000 7f0e0ffff700 1 -- 172.21.15.5:0/1477373228 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e1819fd30 con 0x7f0e18107df0 2024-05-09T00:42:45.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:45.291+0000 7f0e1cef8700 1 --2- 172.21.15.5:0/1477373228 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e18108740 0x7f0e1819f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:42:45.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:45.291+0000 7f0e0ffff700 1 --2- 172.21.15.5:0/1477373228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e18107df0 0x7f0e1819b400 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0e00006010 tx=0x7f0e000047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:42:45.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:45.291+0000 7f0e15ffb700 1 -- 172.21.15.5:0/1477373228 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e00026070 con 0x7f0e18107df0 2024-05-09T00:42:45.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:45.291+0000 7f0e15ffb700 1 -- 172.21.15.5:0/1477373228 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e00003f20 con 0x7f0e18107df0 2024-05-09T00:42:45.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:45.291+0000 7f0e1e95b700 1 -- 172.21.15.5:0/1477373228 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e1819ffc0 con 0x7f0e18107df0 2024-05-09T00:42:45.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:45.291+0000 7f0e15ffb700 1 -- 172.21.15.5:0/1477373228 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0000fcc0 con 0x7f0e18107df0 2024-05-09T00:42:45.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:45.291+0000 7f0e1e95b700 1 -- 172.21.15.5:0/1477373228 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e181a04a0 con 0x7f0e18107df0 2024-05-09T00:42:45.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:45.292+0000 7f0e1e95b700 1 -- 172.21.15.5:0/1477373228 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e180512c0 con 0x7f0e18107df0 2024-05-09T00:42:45.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:45.295+0000 7f0e15ffb700 1 -- 172.21.15.5:0/1477373228 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f0e00029070 con 0x7f0e18107df0 2024-05-09T00:42:45.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:45.295+0000 7f0e15ffb700 1 --2- 172.21.15.5:0/1477373228 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f0e040b2880 0x7f0e040b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:42:45.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:45.296+0000 7f0e17fff700 1 --2- 172.21.15.5:0/1477373228 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f0e040b2880 0x7f0e040b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:42:45.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:45.296+0000 7f0e15ffb700 1 -- 172.21.15.5:0/1477373228 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f0e000e2bf0 con 0x7f0e18107df0 2024-05-09T00:42:45.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:45.296+0000 7f0e17fff700 1 --2- 172.21.15.5:0/1477373228 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f0e040b2880 0x7f0e040b4d90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0e08007be0 tx=0x7f0e080073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:42:45.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:45.299+0000 7f0e15ffb700 1 -- 172.21.15.5:0/1477373228 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0e00013070 con 0x7f0e18107df0 2024-05-09T00:42:45.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:44 smithi131 ceph-mon[34547]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:42:45.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:45.516+0000 7f0e1e95b700 1 -- 172.21.15.5:0/1477373228 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0e1802e5f0 con 0x7f0e18107df0 2024-05-09T00:42:45.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:45.517+0000 7f0e15ffb700 1 -- 172.21.15.5:0/1477373228 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+18419 (secure 0 0 0) 0x7f0e000aae40 con 0x7f0e18107df0 2024-05-09T00:42:45.519 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:42:45.519 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":71,"fsid":"d85814c6-0d9b-11ef-bc97-c7b262605968","created":"2024-05-09T00:35:25.623377+0000","modified":"2024-05-09T00:42:39.531717+0000","last_up_change":"2024-05-09T00:42:38.525335+0000","last_in_change":"2024-05-09T00:42:17.990219+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T00:38:47.844734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f8d484bb-2477-4377-ae68-5bce571333bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2963385132},{"type":"v1","addr":"172.21.15.5:6803","nonce":2963385132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2963385132},{"type":"v1","addr":"172.21.15.5:6805","nonce":2963385132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2963385132},{"type":"v1","addr":"172.21.15.5:6809","nonce":2963385132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2963385132},{"type":"v1","addr":"172.21.15.5:6807","nonce":2963385132}]},"public_addr":"172.21.15.5:6803/2963385132","cluster_addr":"172.21.15.5:6805/2963385132","heartbeat_back_addr":"172.21.15.5:6809/2963385132","heartbeat_front_addr":"172.21.15.5:6807/2963385132","state":["exists","up"]},{"osd":1,"uuid":"1c123676-7144-4a67-93a6-fd49e36a5c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3837683163},{"type":"v1","addr":"172.21.15.131:6801","nonce":3837683163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3837683163},{"type":"v1","addr":"172.21.15.131:6803","nonce":3837683163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3837683163},{"type":"v1","addr":"172.21.15.131:6807","nonce":3837683163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3837683163},{"type":"v1","addr":"172.21.15.131:6805","nonce":3837683163}]},"public_addr":"172.21.15.131:6801/3837683163","cluster_addr":"172.21.15.131:6803/3837683163","heartbeat_back_addr":"172.21.15.131:6807/3837683163","heartbeat_front_addr":"172.21.15.131:6805/3837683163","state":["exists","up"]},{"osd":2,"uuid":"95ca51f0-b48b-485c-a5f7-b15c01e754b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":484818139},{"type":"v1","addr":"172.21.15.157:6801","nonce":484818139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":484818139},{"type":"v1","addr":"172.21.15.157:6803","nonce":484818139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":484818139},{"type":"v1","addr":"172.21.15.157:6807","nonce":484818139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":484818139},{"type":"v1","addr":"172.21.15.157:6805","nonce":484818139}]},"public_addr":"172.21.15.157:6801/484818139","cluster_addr":"172.21.15.157:6803/484818139","heartbeat_back_addr":"172.21.15.157:6807/484818139","heartbeat_front_addr":"172.21.15.157:6805/484818139","state":["exists","up"]},{"osd":3,"uuid":"ed7577fb-3676-4a08-b40d-5474eb88974f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":2357377277},{"type":"v1","addr":"172.21.15.5:6811","nonce":2357377277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":2357377277},{"type":"v1","addr":"172.21.15.5:6813","nonce":2357377277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":2357377277},{"type":"v1","addr":"172.21.15.5:6817","nonce":2357377277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":2357377277},{"type":"v1","addr":"172.21.15.5:6815","nonce":2357377277}]},"public_addr":"172.21.15.5:6811/2357377277","cluster_addr":"172.21.15.5:6813/2357377277","heartbeat_back_addr":"172.21.15.5:6817/2357377277","heartbeat_front_addr":"172.21.15.5:6815/2357377277","state":["exists","up"]},{"osd":4,"uuid":"fb5dcf79-a6f1-4b1a-9e3a-087fb5132472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":2915418378},{"type":"v1","addr":"172.21.15.131:6809","nonce":2915418378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":2915418378},{"type":"v1","addr":"172.21.15.131:6811","nonce":2915418378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":2915418378},{"type":"v1","addr":"172.21.15.131:6815","nonce":2915418378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":2915418378},{"type":"v1","addr":"172.21.15.131:6813","nonce":2915418378}]},"public_addr":"172.21.15.131:6809/2915418378","cluster_addr":"172.21.15.131:6811/2915418378","heartbeat_back_addr":"172.21.15.131:6815/2915418378","heartbeat_front_addr":"172.21.15.131:6813/2915418378","state":["exists","up"]},{"osd":5,"uuid":"f01ed5d3-8066-4d25-a113-280c83e75ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4188818663},{"type":"v1","addr":"172.21.15.157:6809","nonce":4188818663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":4188818663},{"type":"v1","addr":"172.21.15.157:6811","nonce":4188818663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":4188818663},{"type":"v1","addr":"172.21.15.157:6815","nonce":4188818663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":4188818663},{"type":"v1","addr":"172.21.15.157:6813","nonce":4188818663}]},"public_addr":"172.21.15.157:6809/4188818663","cluster_addr":"172.21.15.157:6811/4188818663","heartbeat_back_addr":"172.21.15.157:6815/4188818663","heartbeat_front_addr":"172.21.15.157:6813/4188818663","state":["exists","up"]},{"osd":6,"uuid":"932379a8-c97b-4acc-8665-81f7d9cbce65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2930397813},{"type":"v1","addr":"172.21.15.5:6819","nonce":2930397813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2930397813},{"type":"v1","addr":"172.21.15.5:6821","nonce":2930397813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2930397813},{"type":"v1","addr":"172.21.15.5:6825","nonce":2930397813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2930397813},{"type":"v1","addr":"172.21.15.5:6823","nonce":2930397813}]},"public_addr":"172.21.15.5:6819/2930397813","cluster_addr":"172.21.15.5:6821/2930397813","heartbeat_back_addr":"172.21.15.5:6825/2930397813","heartbeat_front_addr":"172.21.15.5:6823/2930397813","state":["exists","up"]},{"osd":7,"uuid":"3c0a0573-6465-4bf6-b70f-310d49857a6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":4125045219},{"type":"v1","addr":"172.21.15.131:6817","nonce":4125045219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":4125045219},{"type":"v1","addr":"172.21.15.131:6819","nonce":4125045219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":4125045219},{"type":"v1","addr":"172.21.15.131:6823","nonce":4125045219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":4125045219},{"type":"v1","addr":"172.21.15.131:6821","nonce":4125045219}]},"public_addr":"172.21.15.131:6817/4125045219","cluster_addr":"172.21.15.131:6819/4125045219","heartbeat_back_addr":"172.21.15.131:6823/4125045219","heartbeat_front_addr":"172.21.15.131:6821/4125045219","state":["exists","up"]},{"osd":8,"uuid":"c49f0a00-f5ae-40ed-8f38-7b953ba1c17c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2001626768},{"type":"v1","addr":"172.21.15.157:6817","nonce":2001626768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2001626768},{"type":"v1","addr":"172.21.15.157:6819","nonce":2001626768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2001626768},{"type":"v1","addr":"172.21.15.157:6823","nonce":2001626768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2001626768},{"type":"v1","addr":"172.21.15.157:6821","nonce":2001626768}]},"public_addr":"172.21.15.157:6817/2001626768","cluster_addr":"172.21.15.157:6819/2001626768","heartbeat_back_addr":"172.21.15.157:6823/2001626768","heartbeat_front_addr":"172.21.15.157:6821/2001626768","state":["exists","up"]},{"osd":9,"uuid":"1927dfe7-c1af-4a15-bb71-d58d97c8734a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":4287220015},{"type":"v1","addr":"172.21.15.5:6827","nonce":4287220015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":4287220015},{"type":"v1","addr":"172.21.15.5:6829","nonce":4287220015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":4287220015},{"type":"v1","addr":"172.21.15.5:6833","nonce":4287220015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":4287220015},{"type":"v1","addr":"172.21.15.5:6831","nonce":4287220015}]},"public_addr":"172.21.15.5:6827/4287220015","cluster_addr":"172.21.15.5:6829/4287220015","heartbeat_back_addr":"172.21.15.5:6833/4287220015","heartbeat_front_addr":"172.21.15.5:6831/4287220015","state":["exists","up"]},{"osd":10,"uuid":"fbb4f3e3-eb7d-4ecb-a30b-55b8910375ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1571783985},{"type":"v1","addr":"172.21.15.131:6825","nonce":1571783985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1571783985},{"type":"v1","addr":"172.21.15.131:6827","nonce":1571783985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1571783985},{"type":"v1","addr":"172.21.15.131:6831","nonce":1571783985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1571783985},{"type":"v1","addr":"172.21.15.131:6829","nonce":1571783985}]},"public_addr":"172.21.15.131:6825/1571783985","cluster_addr":"172.21.15.131:6827/1571783985","heartbeat_back_addr":"172.21.15.131:6831/1571783985","heartbeat_front_addr":"172.21.15.131:6829/1571783985","state":["exists","up"]},{"osd":11,"uuid":"b858453f-eda1-4e25-89c8-94b54c830938","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3261151188},{"type":"v1","addr":"172.21.15.157:6825","nonce":3261151188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":3261151188},{"type":"v1","addr":"172.21.15.157:6827","nonce":3261151188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":3261151188},{"type":"v1","addr":"172.21.15.157:6831","nonce":3261151188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":3261151188},{"type":"v1","addr":"172.21.15.157:6829","nonce":3261151188}]},"public_addr":"172.21.15.157:6825/3261151188","cluster_addr":"172.21.15.157:6827/3261151188","heartbeat_back_addr":"172.21.15.157:6831/3261151188","heartbeat_front_addr":"172.21.15.157:6829/3261151188","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:37:57.528483+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:38:21.477482+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:38:45.780266+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:39:10.938791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:39:35.734857+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:40:00.169040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:40:26.494635+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:40:51.881718+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:41:17.697112+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:41:44.826479+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:42:10.504768+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:42:36.605447+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/3405015786":"2024-05-10T00:36:21.742363+0000","172.21.15.5:0/2669115557":"2024-05-10T00:36:21.742363+0000","172.21.15.5:6801/287842266":"2024-05-10T00:36:21.742363+0000","172.21.15.5:6800/752513054":"2024-05-10T00:35:53.128955+0000","172.21.15.5:6800/287842266":"2024-05-10T00:36:21.742363+0000","172.21.15.5:6801/752513054":"2024-05-10T00:35:53.128955+0000","172.21.15.5:0/1954876662":"2024-05-10T00:35:53.128955+0000","172.21.15.5:0/2545175121":"2024-05-10T00:36:21.742363+0000","172.21.15.5:0/3754653663":"2024-05-10T00:35:53.128955+0000","172.21.15.5:0/651514359":"2024-05-10T00:35:53.128955+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T00:42:45.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:45.519+0000 7f0e1e95b700 1 -- 172.21.15.5:0/1477373228 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f0e040b2880 msgr2=0x7f0e040b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:42:45.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:45.519+0000 7f0e1e95b700 1 --2- 172.21.15.5:0/1477373228 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f0e040b2880 0x7f0e040b4d90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0e08007be0 tx=0x7f0e080073d0 comp rx=0 tx=0).stop 2024-05-09T00:42:45.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:45.519+0000 7f0e1e95b700 1 -- 172.21.15.5:0/1477373228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e18107df0 msgr2=0x7f0e1819b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:42:45.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:45.519+0000 7f0e1e95b700 1 --2- 172.21.15.5:0/1477373228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e18107df0 0x7f0e1819b400 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0e00006010 tx=0x7f0e000047b0 comp rx=0 tx=0).stop 2024-05-09T00:42:45.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:45.519+0000 7f0e1e95b700 1 -- 172.21.15.5:0/1477373228 shutdown_connections 2024-05-09T00:42:45.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:45.519+0000 7f0e1e95b700 1 --2- 172.21.15.5:0/1477373228 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f0e040b2880 0x7f0e040b4d90 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:42:45.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:45.519+0000 7f0e1e95b700 1 --2- 172.21.15.5:0/1477373228 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e18108740 0x7f0e1819f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:42:45.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:45.519+0000 7f0e1e95b700 1 --2- 172.21.15.5:0/1477373228 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e181034c0 0x7f0e1819ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:42:45.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:45.519+0000 7f0e1e95b700 1 --2- 172.21.15.5:0/1477373228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e18107df0 0x7f0e1819b400 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:42:45.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:45.519+0000 7f0e1e95b700 1 -- 172.21.15.5:0/1477373228 >> 172.21.15.5:0/1477373228 conn(0x7f0e180fedc0 msgr2=0x7f0e1806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:42:45.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:45.519+0000 7f0e1e95b700 1 -- 172.21.15.5:0/1477373228 shutdown_connections 2024-05-09T00:42:45.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:45.519+0000 7f0e1e95b700 1 -- 172.21.15.5:0/1477373228 wait complete. 2024-05-09T00:42:46.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:45 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/1477373228' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:42:46.242 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:45 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/1477373228' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:42:46.273 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T00:38:47.844734+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T00:42:46.274 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-09T00:42:46.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:45 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/1477373228' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:42:46.526 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:42:46.913 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:46 smithi005 ceph-mon[30365]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:42:47.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:46 smithi157 ceph-mon[34294]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:42:47.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:46 smithi131 ceph-mon[34547]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:42:48.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:48.375+0000 7f4113bfb700 1 -- 172.21.15.5:0/3551765378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f410c108740 msgr2=0x7f410c1100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:42:48.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:48.375+0000 7f4113bfb700 1 --2- 172.21.15.5:0/3551765378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f410c108740 0x7f410c1100a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f410400b0d0 tx=0x7f410400b3e0 comp rx=0 tx=0).stop 2024-05-09T00:42:48.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:48.375+0000 7f4113bfb700 1 -- 172.21.15.5:0/3551765378 shutdown_connections 2024-05-09T00:42:48.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:48.375+0000 7f4113bfb700 1 --2- 172.21.15.5:0/3551765378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f410c108740 0x7f410c1100a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:42:48.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:48.375+0000 7f4113bfb700 1 --2- 172.21.15.5:0/3551765378 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f410c1034c0 0x7f410c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:42:48.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:48.375+0000 7f4113bfb700 1 --2- 172.21.15.5:0/3551765378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f410c107df0 0x7f410c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:42:48.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:48.375+0000 7f4113bfb700 1 -- 172.21.15.5:0/3551765378 >> 172.21.15.5:0/3551765378 conn(0x7f410c0fedc0 msgr2=0x7f410c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:42:48.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:48.375+0000 7f4113bfb700 1 -- 172.21.15.5:0/3551765378 shutdown_connections 2024-05-09T00:42:48.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:48.375+0000 7f4113bfb700 1 -- 172.21.15.5:0/3551765378 wait complete. 2024-05-09T00:42:48.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:48.376+0000 7f4113bfb700 1 Processor -- start 2024-05-09T00:42:48.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:48.376+0000 7f4113bfb700 1 -- start start 2024-05-09T00:42:48.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:48.376+0000 7f4113bfb700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f410c1034c0 0x7f410c110a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:42:48.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:48.376+0000 7f4113bfb700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f410c107df0 0x7f410c110fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:42:48.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:48.376+0000 7f4113bfb700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f410c108740 0x7f410c111500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:42:48.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:48.376+0000 7f4113bfb700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f410c1144c0 con 0x7f410c108740 2024-05-09T00:42:48.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:48.376+0000 7f4113bfb700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f410c114340 con 0x7f410c1034c0 2024-05-09T00:42:48.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:48.376+0000 7f4113bfb700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f410c114640 con 0x7f410c107df0 2024-05-09T00:42:48.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:48.376+0000 7f4112198700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f410c108740 0x7f410c111500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:42:48.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:48.376+0000 7f4112198700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f410c108740 0x7f410c111500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50806/0 (socket says 172.21.15.5:50806) 2024-05-09T00:42:48.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:48.376+0000 7f4112198700 1 -- 172.21.15.5:0/1155427837 learned_addr learned my addr 172.21.15.5:0/1155427837 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:42:48.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:48.377+0000 7f4111997700 1 --2- 172.21.15.5:0/1155427837 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f410c1034c0 0x7f410c110a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:42:48.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:48.377+0000 7f4111196700 1 --2- 172.21.15.5:0/1155427837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f410c107df0 0x7f410c110fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:42:48.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:48.377+0000 7f4112198700 1 -- 172.21.15.5:0/1155427837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f410c107df0 msgr2=0x7f410c110fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:42:48.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:48.377+0000 7f4112198700 1 --2- 172.21.15.5:0/1155427837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f410c107df0 0x7f410c110fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:42:48.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:48.377+0000 7f4112198700 1 -- 172.21.15.5:0/1155427837 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f410c1034c0 msgr2=0x7f410c110a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:42:48.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:48.377+0000 7f4112198700 1 --2- 172.21.15.5:0/1155427837 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f410c1034c0 0x7f410c110a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:42:48.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:48.377+0000 7f4112198700 1 -- 172.21.15.5:0/1155427837 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f410c1a90a0 con 0x7f410c108740 2024-05-09T00:42:48.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:48.377+0000 7f4111997700 1 --2- 172.21.15.5:0/1155427837 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f410c1034c0 0x7f410c110a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:42:48.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:48.377+0000 7f4111196700 1 --2- 172.21.15.5:0/1155427837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f410c107df0 0x7f410c110fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:42:48.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:48.377+0000 7f4112198700 1 --2- 172.21.15.5:0/1155427837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f410c108740 0x7f410c111500 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f41040095c0 tx=0x7f4104007ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:42:48.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:48.377+0000 7f4102ffd700 1 -- 172.21.15.5:0/1155427837 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f410401f070 con 0x7f410c108740 2024-05-09T00:42:48.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:48.377+0000 7f4102ffd700 1 -- 172.21.15.5:0/1155427837 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4104003c00 con 0x7f410c108740 2024-05-09T00:42:48.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:48.377+0000 7f4113bfb700 1 -- 172.21.15.5:0/1155427837 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f410c1a9330 con 0x7f410c108740 2024-05-09T00:42:48.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:48.377+0000 7f4102ffd700 1 -- 172.21.15.5:0/1155427837 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f410401b6a0 con 0x7f410c108740 2024-05-09T00:42:48.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:48.377+0000 7f4113bfb700 1 -- 172.21.15.5:0/1155427837 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f410c1a9780 con 0x7f410c108740 2024-05-09T00:42:48.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:48.379+0000 7f4113bfb700 1 -- 172.21.15.5:0/1155427837 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f410c0512c0 con 0x7f410c108740 2024-05-09T00:42:48.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:48.380+0000 7f4102ffd700 1 -- 172.21.15.5:0/1155427837 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f4104022070 con 0x7f410c108740 2024-05-09T00:42:48.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:48.381+0000 7f4102ffd700 1 --2- 172.21.15.5:0/1155427837 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f40f80b2880 0x7f40f80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:42:48.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:48.381+0000 7f4111997700 1 --2- 172.21.15.5:0/1155427837 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f40f80b2880 0x7f40f80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:42:48.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:48.381+0000 7f4102ffd700 1 -- 172.21.15.5:0/1155427837 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f410400f020 con 0x7f410c108740 2024-05-09T00:42:48.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:48.381+0000 7f4111997700 1 --2- 172.21.15.5:0/1155427837 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f40f80b2880 0x7f40f80b4d90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4108005fd0 tx=0x7f4108005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:42:48.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:48.385+0000 7f4102ffd700 1 -- 172.21.15.5:0/1155427837 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4104029020 con 0x7f410c108740 2024-05-09T00:42:48.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:48.599+0000 7f4113bfb700 1 -- 172.21.15.5:0/1155427837 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f410c02e690 con 0x7f410c108740 2024-05-09T00:42:48.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:48.600+0000 7f4102ffd700 1 -- 172.21.15.5:0/1155427837 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) ==== 93+0+10 (secure 0 0 0) 0x7f41040ab780 con 0x7f410c108740 2024-05-09T00:42:48.602 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-09T00:42:48.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:48.602+0000 7f4113bfb700 1 -- 172.21.15.5:0/1155427837 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f40f80b2880 msgr2=0x7f40f80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:42:48.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:48.602+0000 7f4113bfb700 1 --2- 172.21.15.5:0/1155427837 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f40f80b2880 0x7f40f80b4d90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4108005fd0 tx=0x7f4108005e30 comp rx=0 tx=0).stop 2024-05-09T00:42:48.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:48.602+0000 7f4113bfb700 1 -- 172.21.15.5:0/1155427837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f410c108740 msgr2=0x7f410c111500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:42:48.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:48.602+0000 7f4113bfb700 1 --2- 172.21.15.5:0/1155427837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f410c108740 0x7f410c111500 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f41040095c0 tx=0x7f4104007ad0 comp rx=0 tx=0).stop 2024-05-09T00:42:48.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:48.602+0000 7f4113bfb700 1 -- 172.21.15.5:0/1155427837 shutdown_connections 2024-05-09T00:42:48.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:48.602+0000 7f4113bfb700 1 --2- 172.21.15.5:0/1155427837 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f40f80b2880 0x7f40f80b4d90 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:42:48.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:48.602+0000 7f4113bfb700 1 --2- 172.21.15.5:0/1155427837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f410c108740 0x7f410c111500 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:42:48.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:48.602+0000 7f4113bfb700 1 --2- 172.21.15.5:0/1155427837 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f410c1034c0 0x7f410c110a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:42:48.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:48.602+0000 7f4113bfb700 1 --2- 172.21.15.5:0/1155427837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f410c107df0 0x7f410c110fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:42:48.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:48.602+0000 7f4113bfb700 1 -- 172.21.15.5:0/1155427837 >> 172.21.15.5:0/1155427837 conn(0x7f410c0fedc0 msgr2=0x7f410c107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:42:48.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:48.602+0000 7f4113bfb700 1 -- 172.21.15.5:0/1155427837 shutdown_connections 2024-05-09T00:42:48.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:48.602+0000 7f4113bfb700 1 -- 172.21.15.5:0/1155427837 wait complete. 2024-05-09T00:42:49.185 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:48 smithi005 ceph-mon[30365]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:42:49.185 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:48 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/1155427837' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T00:42:49.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:48 smithi157 ceph-mon[34294]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:42:49.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:48 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/1155427837' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T00:42:49.217 INFO:tasks.cephadm:Adding mds.a on smithi005 2024-05-09T00:42:49.217 INFO:tasks.cephadm:Adding mds.d on smithi005 2024-05-09T00:42:49.217 INFO:tasks.cephadm:Adding mds.g on smithi005 2024-05-09T00:42:49.218 INFO:tasks.cephadm:Adding mds.j on smithi005 2024-05-09T00:42:49.218 INFO:tasks.cephadm:Adding mds.b on smithi131 2024-05-09T00:42:49.218 INFO:tasks.cephadm:Adding mds.e on smithi131 2024-05-09T00:42:49.218 INFO:tasks.cephadm:Adding mds.h on smithi131 2024-05-09T00:42:49.218 INFO:tasks.cephadm:Adding mds.k on smithi131 2024-05-09T00:42:49.218 INFO:tasks.cephadm:Adding mds.c on smithi157 2024-05-09T00:42:49.218 INFO:tasks.cephadm:Adding mds.f on smithi157 2024-05-09T00:42:49.218 INFO:tasks.cephadm:Adding mds.i on smithi157 2024-05-09T00:42:49.218 INFO:tasks.cephadm:Adding mds.l on smithi157 2024-05-09T00:42:49.219 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph orch apply mds all '12;smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi131=b;smithi131=e;smithi131=h;smithi131=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l' 2024-05-09T00:42:49.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:48 smithi131 ceph-mon[34547]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:42:49.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:48 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/1155427837' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T00:42:49.469 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T00:42:51.135 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:51.130+0000 7fac02387700 1 -- 172.21.15.157:0/4061367061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabfc103f60 msgr2=0x7fabfc104370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:42:51.136 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:51.130+0000 7fac02387700 1 --2- 172.21.15.157:0/4061367061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabfc103f60 0x7fabfc104370 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fabf0009990 tx=0x7fabf0009ca0 comp rx=0 tx=0).stop 2024-05-09T00:42:51.136 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:51.130+0000 7fac02387700 1 -- 172.21.15.157:0/4061367061 shutdown_connections 2024-05-09T00:42:51.136 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:51.130+0000 7fac02387700 1 --2- 172.21.15.157:0/4061367061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabfc1049e0 0x7fabfc10c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:42:51.136 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:51.130+0000 7fac02387700 1 --2- 172.21.15.157:0/4061367061 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fabfc1059b0 0x7fabfc105dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:42:51.136 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:51.130+0000 7fac02387700 1 --2- 172.21.15.157:0/4061367061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabfc103f60 0x7fabfc104370 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:42:51.136 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:51.130+0000 7fac02387700 1 -- 172.21.15.157:0/4061367061 >> 172.21.15.157:0/4061367061 conn(0x7fabfc0ff9d0 msgr2=0x7fabfc101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:42:51.136 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:51.130+0000 7fac02387700 1 -- 172.21.15.157:0/4061367061 shutdown_connections 2024-05-09T00:42:51.136 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:51.130+0000 7fac02387700 1 -- 172.21.15.157:0/4061367061 wait complete. 2024-05-09T00:42:51.136 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:51.131+0000 7fac02387700 1 Processor -- start 2024-05-09T00:42:51.136 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:51.131+0000 7fac02387700 1 -- start start 2024-05-09T00:42:51.136 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:51.131+0000 7fac02387700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabfc103f60 0x7fabfc19aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:42:51.137 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:51.131+0000 7fac02387700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabfc1049e0 0x7fabfc19af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:42:51.137 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:51.131+0000 7fac02387700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fabfc1059b0 0x7fabfc1a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:42:51.137 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:51.131+0000 7fac02387700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fabfc10ff70 con 0x7fabfc1049e0 2024-05-09T00:42:51.137 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:51.131+0000 7fac02387700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fabfc10fdf0 con 0x7fabfc1059b0 2024-05-09T00:42:51.137 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:51.131+0000 7fac02387700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fabfc1100f0 con 0x7fabfc103f60 2024-05-09T00:42:51.137 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:51.131+0000 7fabfbfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabfc103f60 0x7fabfc19aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:42:51.137 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:51.131+0000 7fabfbfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabfc103f60 0x7fabfc19aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35482/0 (socket says 172.21.15.157:35482) 2024-05-09T00:42:51.137 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:51.131+0000 7fabfbfff700 1 -- 172.21.15.157:0/1362272555 learned_addr learned my addr 172.21.15.157:0/1362272555 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-05-09T00:42:51.137 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:51.131+0000 7fabfb7fe700 1 --2- 172.21.15.157:0/1362272555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabfc1049e0 0x7fabfc19af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:42:51.137 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:51.132+0000 7fac00924700 1 --2- 172.21.15.157:0/1362272555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fabfc1059b0 0x7fabfc1a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:42:51.137 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:51.132+0000 7fabfbfff700 1 -- 172.21.15.157:0/1362272555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fabfc1059b0 msgr2=0x7fabfc1a1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:42:51.137 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:51.132+0000 7fabfbfff700 1 --2- 172.21.15.157:0/1362272555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fabfc1059b0 0x7fabfc1a1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:42:51.137 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:51.132+0000 7fabfbfff700 1 -- 172.21.15.157:0/1362272555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabfc1049e0 msgr2=0x7fabfc19af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:42:51.138 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:51.132+0000 7fabfbfff700 1 --2- 172.21.15.157:0/1362272555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabfc1049e0 0x7fabfc19af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:42:51.138 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:51.132+0000 7fabfbfff700 1 -- 172.21.15.157:0/1362272555 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabfc1a25b0 con 0x7fabfc103f60 2024-05-09T00:42:51.138 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:51.132+0000 7fabfbfff700 1 --2- 172.21.15.157:0/1362272555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabfc103f60 0x7fabfc19aa50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fabec00b4d0 tx=0x7fabec00b7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:42:51.138 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:51.132+0000 7fabf97fa700 1 -- 172.21.15.157:0/1362272555 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabec015020 con 0x7fabfc103f60 2024-05-09T00:42:51.138 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:51.132+0000 7fabf97fa700 1 -- 172.21.15.157:0/1362272555 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabec011ce0 con 0x7fabfc103f60 2024-05-09T00:42:51.138 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:51.132+0000 7fac02387700 1 -- 172.21.15.157:0/1362272555 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabfc1a28a0 con 0x7fabfc103f60 2024-05-09T00:42:51.138 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:51.132+0000 7fabf97fa700 1 -- 172.21.15.157:0/1362272555 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabec010cf0 con 0x7fabfc103f60 2024-05-09T00:42:51.138 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:51.132+0000 7fac02387700 1 -- 172.21.15.157:0/1362272555 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabfc1a2e60 con 0x7fabfc103f60 2024-05-09T00:42:51.138 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:51.133+0000 7fac02387700 1 -- 172.21.15.157:0/1362272555 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabfc0512c0 con 0x7fabfc103f60 2024-05-09T00:42:51.139 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:51.134+0000 7fabf97fa700 1 -- 172.21.15.157:0/1362272555 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fabec018020 con 0x7fabfc103f60 2024-05-09T00:42:51.139 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:51.135+0000 7fabf97fa700 1 --2- 172.21.15.157:0/1362272555 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fabe40b2880 0x7fabe40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:42:51.139 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:51.135+0000 7fabf97fa700 1 -- 172.21.15.157:0/1362272555 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fabec0e14c0 con 0x7fabfc103f60 2024-05-09T00:42:51.139 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:51.135+0000 7fabfb7fe700 1 --2- 172.21.15.157:0/1362272555 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fabe40b2880 0x7fabe40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:42:51.139 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:51.136+0000 7fabfb7fe700 1 --2- 172.21.15.157:0/1362272555 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fabe40b2880 0x7fabe40b4d90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fabf000b5c0 tx=0x7fabf001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:42:51.142 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:51.140+0000 7fabf97fa700 1 -- 172.21.15.157:0/1362272555 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fabec01f020 con 0x7fabfc103f60 2024-05-09T00:42:51.142 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:50 smithi157 ceph-mon[34294]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:42:51.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:50 smithi005 ceph-mon[30365]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:42:51.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:50 smithi131 ceph-mon[34547]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:42:51.363 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:51.360+0000 7fac02387700 1 -- 172.21.15.157:0/1362272555 --> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi131=b;smithi131=e;smithi131=h;smithi131=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l", "target": ["mon-mgr", ""]}) -- 0x7fabfc063c00 con 0x7fabe40b2880 2024-05-09T00:42:51.372 INFO:teuthology.orchestra.run.smithi157.stdout:Scheduled mds.all update... 2024-05-09T00:42:51.372 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:51.370+0000 7fabf97fa700 1 -- 172.21.15.157:0/1362272555 <== mgr.14152 v2:172.21.15.5:6800/91856593 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fabfc063c00 con 0x7fabe40b2880 2024-05-09T00:42:51.375 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:51.372+0000 7fac02387700 1 -- 172.21.15.157:0/1362272555 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fabe40b2880 msgr2=0x7fabe40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:42:51.375 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:51.372+0000 7fac02387700 1 --2- 172.21.15.157:0/1362272555 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fabe40b2880 0x7fabe40b4d90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fabf000b5c0 tx=0x7fabf001a040 comp rx=0 tx=0).stop 2024-05-09T00:42:51.375 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:51.372+0000 7fac02387700 1 -- 172.21.15.157:0/1362272555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabfc103f60 msgr2=0x7fabfc19aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:42:51.375 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:51.372+0000 7fac02387700 1 --2- 172.21.15.157:0/1362272555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabfc103f60 0x7fabfc19aa50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fabec00b4d0 tx=0x7fabec00b7e0 comp rx=0 tx=0).stop 2024-05-09T00:42:51.375 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:51.372+0000 7fac02387700 1 -- 172.21.15.157:0/1362272555 shutdown_connections 2024-05-09T00:42:51.375 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:51.372+0000 7fac02387700 1 --2- 172.21.15.157:0/1362272555 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fabe40b2880 0x7fabe40b4d90 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:42:51.375 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:51.372+0000 7fac02387700 1 --2- 172.21.15.157:0/1362272555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fabfc1059b0 0x7fabfc1a1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:42:51.375 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:51.372+0000 7fac02387700 1 --2- 172.21.15.157:0/1362272555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabfc103f60 0x7fabfc19aa50 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:42:51.375 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:51.372+0000 7fac02387700 1 --2- 172.21.15.157:0/1362272555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabfc1049e0 0x7fabfc19af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:42:51.375 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:51.372+0000 7fac02387700 1 -- 172.21.15.157:0/1362272555 >> 172.21.15.157:0/1362272555 conn(0x7fabfc0ff9d0 msgr2=0x7fabfc1086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:42:51.375 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:51.372+0000 7fac02387700 1 -- 172.21.15.157:0/1362272555 shutdown_connections 2024-05-09T00:42:51.375 INFO:teuthology.orchestra.run.smithi157.stderr:2024-05-09T00:42:51.372+0000 7fac02387700 1 -- 172.21.15.157:0/1362272555 wait complete. 2024-05-09T00:42:52.017 DEBUG:teuthology.orchestra.run.smithi005:mds.a> sudo journalctl -f -n 0 -u ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@mds.a.service 2024-05-09T00:42:52.020 DEBUG:teuthology.orchestra.run.smithi005:mds.d> sudo journalctl -f -n 0 -u ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@mds.d.service 2024-05-09T00:42:52.063 DEBUG:teuthology.orchestra.run.smithi005:mds.g> sudo journalctl -f -n 0 -u ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@mds.g.service 2024-05-09T00:42:52.106 DEBUG:teuthology.orchestra.run.smithi005:mds.j> sudo journalctl -f -n 0 -u ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@mds.j.service 2024-05-09T00:42:52.150 DEBUG:teuthology.orchestra.run.smithi131:mds.b> sudo journalctl -f -n 0 -u ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@mds.b.service 2024-05-09T00:42:52.153 DEBUG:teuthology.orchestra.run.smithi131:mds.e> sudo journalctl -f -n 0 -u ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@mds.e.service 2024-05-09T00:42:52.196 DEBUG:teuthology.orchestra.run.smithi131:mds.h> sudo journalctl -f -n 0 -u ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@mds.h.service 2024-05-09T00:42:52.240 DEBUG:teuthology.orchestra.run.smithi131:mds.k> sudo journalctl -f -n 0 -u ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@mds.k.service 2024-05-09T00:42:52.284 DEBUG:teuthology.orchestra.run.smithi157:mds.c> sudo journalctl -f -n 0 -u ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@mds.c.service 2024-05-09T00:42:52.287 DEBUG:teuthology.orchestra.run.smithi157:mds.f> sudo journalctl -f -n 0 -u ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@mds.f.service 2024-05-09T00:42:52.331 DEBUG:teuthology.orchestra.run.smithi157:mds.i> sudo journalctl -f -n 0 -u ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@mds.i.service 2024-05-09T00:42:52.374 DEBUG:teuthology.orchestra.run.smithi157:mds.l> sudo journalctl -f -n 0 -u ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@mds.l.service 2024-05-09T00:42:52.417 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-05-09T00:42:52.417 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T00:42:52.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:52 smithi131 ceph-mon[34547]: from='client.24445 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi131=b;smithi131=e;smithi131=h;smithi131=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:42:52.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:52 smithi131 ceph-mon[34547]: Saving service mds.all spec with placement smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi131=b;smithi131=e;smithi131=h;smithi131=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l;count:12 2024-05-09T00:42:52.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:52 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:52.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:52 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:42:52.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:52 smithi131 ceph-mon[34547]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:42:52.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:52 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:42:52.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:52 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:42:52.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:52 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:52.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:52 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:42:52.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:52 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:42:52.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:52 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:42:52.607 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:52 smithi131 ceph-mon[34547]: Deploying daemon mds.l on smithi157 2024-05-09T00:42:52.668 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:42:52.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:52 smithi157 ceph-mon[34294]: from='client.24445 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi131=b;smithi131=e;smithi131=h;smithi131=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:42:52.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:52 smithi157 ceph-mon[34294]: Saving service mds.all spec with placement smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi131=b;smithi131=e;smithi131=h;smithi131=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l;count:12 2024-05-09T00:42:52.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:52 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:52.688 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:52 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:42:52.688 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:52 smithi157 ceph-mon[34294]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:42:52.688 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:52 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:42:52.688 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:52 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:42:52.688 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:52 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:52.688 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:52 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:42:52.689 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:52 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:42:52.689 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:52 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:42:52.689 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:52 smithi157 ceph-mon[34294]: Deploying daemon mds.l on smithi157 2024-05-09T00:42:52.790 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:52 smithi005 ceph-mon[30365]: from='client.24445 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi131=b;smithi131=e;smithi131=h;smithi131=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:42:52.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:52 smithi005 ceph-mon[30365]: Saving service mds.all spec with placement smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi131=b;smithi131=e;smithi131=h;smithi131=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l;count:12 2024-05-09T00:42:52.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:52 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:52.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:52 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:42:52.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:52 smithi005 ceph-mon[30365]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:42:52.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:52 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:42:52.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:52 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:42:52.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:52 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:52.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:52 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:42:52.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:52 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:42:52.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:52 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:42:52.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:52 smithi005 ceph-mon[30365]: Deploying daemon mds.l on smithi157 2024-05-09T00:42:54.407 INFO:journalctl@ceph.mds.l.smithi157.stdout:May 09 00:42:54 smithi157 systemd[1]: Starting Ceph mds.l for d85814c6-0d9b-11ef-bc97-c7b262605968... 2024-05-09T00:42:54.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:54.465+0000 7f179068e700 1 -- 172.21.15.5:0/1992113379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17881034c0 msgr2=0x7f17881038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:42:54.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:54.465+0000 7f179068e700 1 --2- 172.21.15.5:0/1992113379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17881034c0 0x7f17881038d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f178400b0d0 tx=0x7f178400b3e0 comp rx=0 tx=0).stop 2024-05-09T00:42:54.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:54.465+0000 7f179068e700 1 -- 172.21.15.5:0/1992113379 shutdown_connections 2024-05-09T00:42:54.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:54.465+0000 7f179068e700 1 --2- 172.21.15.5:0/1992113379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1788108740 0x7f17881100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:42:54.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:54.465+0000 7f179068e700 1 --2- 172.21.15.5:0/1992113379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17881034c0 0x7f17881038d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:42:54.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:54.465+0000 7f179068e700 1 --2- 172.21.15.5:0/1992113379 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1788107df0 0x7f1788108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:42:54.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:54.465+0000 7f179068e700 1 -- 172.21.15.5:0/1992113379 >> 172.21.15.5:0/1992113379 conn(0x7f17880fedc0 msgr2=0x7f17881011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:42:54.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:54.465+0000 7f179068e700 1 -- 172.21.15.5:0/1992113379 shutdown_connections 2024-05-09T00:42:54.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:54.465+0000 7f179068e700 1 -- 172.21.15.5:0/1992113379 wait complete. 2024-05-09T00:42:54.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:54.466+0000 7f179068e700 1 Processor -- start 2024-05-09T00:42:54.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:54.466+0000 7f179068e700 1 -- start start 2024-05-09T00:42:54.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:54.466+0000 7f179068e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17881034c0 0x7f178819bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:42:54.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:54.466+0000 7f179068e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1788107df0 0x7f178819c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:42:54.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:54.466+0000 7f179068e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1788108740 0x7f17881a6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:42:54.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:54.466+0000 7f179068e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1788113370 con 0x7f1788108740 2024-05-09T00:42:54.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:54.466+0000 7f179068e700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f17881131f0 con 0x7f17881034c0 2024-05-09T00:42:54.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:54.466+0000 7f179068e700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f17881134f0 con 0x7f1788107df0 2024-05-09T00:42:54.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:54.467+0000 7f178ec2b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1788108740 0x7f17881a6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:42:54.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:54.467+0000 7f178ec2b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1788108740 0x7f17881a6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32956/0 (socket says 172.21.15.5:32956) 2024-05-09T00:42:54.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:54.467+0000 7f178dc29700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1788107df0 0x7f178819c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:42:54.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:54.467+0000 7f178ec2b700 1 -- 172.21.15.5:0/2313099080 learned_addr learned my addr 172.21.15.5:0/2313099080 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:42:54.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:54.467+0000 7f178e42a700 1 --2- 172.21.15.5:0/2313099080 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17881034c0 0x7f178819bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:42:54.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:54.467+0000 7f178ec2b700 1 -- 172.21.15.5:0/2313099080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1788107df0 msgr2=0x7f178819c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:42:54.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:54.467+0000 7f178ec2b700 1 --2- 172.21.15.5:0/2313099080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1788107df0 0x7f178819c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:42:54.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:54.467+0000 7f178ec2b700 1 -- 172.21.15.5:0/2313099080 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17881034c0 msgr2=0x7f178819bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:42:54.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:54.467+0000 7f178ec2b700 1 --2- 172.21.15.5:0/2313099080 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17881034c0 0x7f178819bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:42:54.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:54.467+0000 7f178ec2b700 1 -- 172.21.15.5:0/2313099080 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f178819c970 con 0x7f1788108740 2024-05-09T00:42:54.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:54.467+0000 7f178e42a700 1 --2- 172.21.15.5:0/2313099080 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17881034c0 0x7f178819bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:42:54.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:54.467+0000 7f178dc29700 1 --2- 172.21.15.5:0/2313099080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1788107df0 0x7f178819c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:42:54.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:54.467+0000 7f178ec2b700 1 --2- 172.21.15.5:0/2313099080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1788108740 0x7f17881a6c90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f178000e940 tx=0x7f178000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:42:54.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:54.467+0000 7f177f7fe700 1 -- 172.21.15.5:0/2313099080 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178000cbd0 con 0x7f1788108740 2024-05-09T00:42:54.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:54.467+0000 7f179068e700 1 -- 172.21.15.5:0/2313099080 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17881a7230 con 0x7f1788108740 2024-05-09T00:42:54.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:54.467+0000 7f177f7fe700 1 -- 172.21.15.5:0/2313099080 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f178000ee90 con 0x7f1788108740 2024-05-09T00:42:54.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:54.467+0000 7f177f7fe700 1 -- 172.21.15.5:0/2313099080 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1780005590 con 0x7f1788108740 2024-05-09T00:42:54.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:54.467+0000 7f179068e700 1 -- 172.21.15.5:0/2313099080 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17881a7720 con 0x7f1788108740 2024-05-09T00:42:54.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:54.468+0000 7f179068e700 1 -- 172.21.15.5:0/2313099080 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17880512c0 con 0x7f1788108740 2024-05-09T00:42:54.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:54.470+0000 7f177f7fe700 1 -- 172.21.15.5:0/2313099080 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f1780004750 con 0x7f1788108740 2024-05-09T00:42:54.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:54.471+0000 7f177f7fe700 1 --2- 172.21.15.5:0/2313099080 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f17740b2880 0x7f17740b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:42:54.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:54.471+0000 7f178e42a700 1 --2- 172.21.15.5:0/2313099080 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f17740b2880 0x7f17740b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:42:54.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:54.471+0000 7f177f7fe700 1 -- 172.21.15.5:0/2313099080 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f17800e21e0 con 0x7f1788108740 2024-05-09T00:42:54.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:54.472+0000 7f178e42a700 1 --2- 172.21.15.5:0/2313099080 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f17740b2880 0x7f17740b4d90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1784009fd0 tx=0x7f178400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:42:54.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:54.475+0000 7f177f7fe700 1 -- 172.21.15.5:0/2313099080 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1780054df0 con 0x7f1788108740 2024-05-09T00:42:54.669 INFO:journalctl@ceph.mds.l.smithi157.stdout:May 09 00:42:54 smithi157 podman[63920]: 2024-05-09 00:42:54.309083304 +0000 UTC m=+0.018255261 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T00:42:54.669 INFO:journalctl@ceph.mds.l.smithi157.stdout:May 09 00:42:54 smithi157 podman[63920]: 2024-05-09 00:42:54.423868273 +0000 UTC m=+0.133040214 container create de9011eb50a6b782f16ba9be1cc57d4e2845967f4cdb9c138874b76483a27097 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-l, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-05-09T00:42:54.669 INFO:journalctl@ceph.mds.l.smithi157.stdout:May 09 00:42:54 smithi157 podman[63920]: 2024-05-09 00:42:54.668309758 +0000 UTC m=+0.377481699 container init de9011eb50a6b782f16ba9be1cc57d4e2845967f4cdb9c138874b76483a27097 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-l, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD) 2024-05-09T00:42:54.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:54.740+0000 7f179068e700 1 -- 172.21.15.5:0/2313099080 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f178802e7b0 con 0x7f1788108740 2024-05-09T00:42:54.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:54.740+0000 7f177f7fe700 1 -- 172.21.15.5:0/2313099080 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f17800aa3a0 con 0x7f1788108740 2024-05-09T00:42:54.742 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:42:54.742 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2809525455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2809525455},{"type":"v1","addr":"172.21.15.157:6833","nonce":2809525455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-09T00:42:54.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:54.743+0000 7f179068e700 1 -- 172.21.15.5:0/2313099080 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f17740b2880 msgr2=0x7f17740b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:42:54.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:54.743+0000 7f179068e700 1 --2- 172.21.15.5:0/2313099080 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f17740b2880 0x7f17740b4d90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1784009fd0 tx=0x7f178400b040 comp rx=0 tx=0).stop 2024-05-09T00:42:54.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:54.743+0000 7f179068e700 1 -- 172.21.15.5:0/2313099080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1788108740 msgr2=0x7f17881a6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:42:54.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:54.743+0000 7f179068e700 1 --2- 172.21.15.5:0/2313099080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1788108740 0x7f17881a6c90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f178000e940 tx=0x7f178000ec50 comp rx=0 tx=0).stop 2024-05-09T00:42:54.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:54.743+0000 7f179068e700 1 -- 172.21.15.5:0/2313099080 shutdown_connections 2024-05-09T00:42:54.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:54.743+0000 7f179068e700 1 --2- 172.21.15.5:0/2313099080 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f17740b2880 0x7f17740b4d90 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:42:54.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:54.743+0000 7f179068e700 1 --2- 172.21.15.5:0/2313099080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1788108740 0x7f17881a6c90 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:42:54.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:54.743+0000 7f179068e700 1 --2- 172.21.15.5:0/2313099080 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17881034c0 0x7f178819bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:42:54.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:54.743+0000 7f179068e700 1 --2- 172.21.15.5:0/2313099080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1788107df0 0x7f178819c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:42:54.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:54.743+0000 7f179068e700 1 -- 172.21.15.5:0/2313099080 >> 172.21.15.5:0/2313099080 conn(0x7f17880fedc0 msgr2=0x7f17881096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:42:54.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:54.743+0000 7f179068e700 1 -- 172.21.15.5:0/2313099080 shutdown_connections 2024-05-09T00:42:54.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:42:54.743+0000 7f179068e700 1 -- 172.21.15.5:0/2313099080 wait complete. 2024-05-09T00:42:54.745 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 2 2024-05-09T00:42:54.936 INFO:journalctl@ceph.mds.l.smithi157.stdout:May 09 00:42:54 smithi157 podman[63920]: 2024-05-09 00:42:54.671206544 +0000 UTC m=+0.380378486 container start de9011eb50a6b782f16ba9be1cc57d4e2845967f4cdb9c138874b76483a27097 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-l, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-05-09T00:42:54.937 INFO:journalctl@ceph.mds.l.smithi157.stdout:May 09 00:42:54 smithi157 ceph-mds[63939]: -- 172.21.15.157:0/4292933312 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5565242a9a40 con 0x5565242f1a80 2024-05-09T00:42:54.937 INFO:journalctl@ceph.mds.l.smithi157.stdout:May 09 00:42:54 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-l[63935]: starting mds.l at 2024-05-09T00:42:54.937 INFO:journalctl@ceph.mds.l.smithi157.stdout:May 09 00:42:54 smithi157 bash[63920]: de9011eb50a6b782f16ba9be1cc57d4e2845967f4cdb9c138874b76483a27097 2024-05-09T00:42:54.937 INFO:journalctl@ceph.mds.l.smithi157.stdout:May 09 00:42:54 smithi157 systemd[1]: Started Ceph mds.l for d85814c6-0d9b-11ef-bc97-c7b262605968. 2024-05-09T00:42:54.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:54 smithi157 ceph-mon[34294]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:42:54.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:54 smithi157 ceph-mon[34294]: mds.? [v2:172.21.15.157:6832/2809525455,v1:172.21.15.157:6833/2809525455] up:boot 2024-05-09T00:42:54.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:54 smithi157 ceph-mon[34294]: fsmap 1 up:standby 2024-05-09T00:42:54.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:54 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T00:42:54.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:54 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/2313099080' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:42:54.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:54 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:54.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:54 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:54.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:54 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:54.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:54 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:42:54.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:54 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:42:54.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:54 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:42:55.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:54 smithi005 ceph-mon[30365]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:42:55.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:54 smithi005 ceph-mon[30365]: mds.? [v2:172.21.15.157:6832/2809525455,v1:172.21.15.157:6833/2809525455] up:boot 2024-05-09T00:42:55.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:54 smithi005 ceph-mon[30365]: fsmap 1 up:standby 2024-05-09T00:42:55.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:54 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T00:42:55.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:54 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/2313099080' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:42:55.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:54 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:55.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:54 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:55.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:54 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:55.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:54 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:42:55.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:54 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:42:55.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:54 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:42:55.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:54 smithi131 ceph-mon[34547]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:42:55.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:54 smithi131 ceph-mon[34547]: mds.? [v2:172.21.15.157:6832/2809525455,v1:172.21.15.157:6833/2809525455] up:boot 2024-05-09T00:42:55.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:54 smithi131 ceph-mon[34547]: fsmap 1 up:standby 2024-05-09T00:42:55.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:54 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T00:42:55.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:54 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/2313099080' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:42:55.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:54 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:55.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:54 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:55.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:54 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:55.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:54 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:42:55.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:54 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:42:55.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:54 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:42:56.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:55 smithi131 ceph-mon[34547]: Deploying daemon mds.k on smithi131 2024-05-09T00:42:56.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:55 smithi157 ceph-mon[34294]: Deploying daemon mds.k on smithi131 2024-05-09T00:42:56.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:55 smithi005 ceph-mon[30365]: Deploying daemon mds.k on smithi131 2024-05-09T00:42:57.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:56 smithi131 ceph-mon[34547]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:42:57.106 INFO:journalctl@ceph.mds.k.smithi131.stdout:May 09 00:42:56 smithi131 systemd[1]: Starting Ceph mds.k for d85814c6-0d9b-11ef-bc97-c7b262605968... 2024-05-09T00:42:57.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:56 smithi157 ceph-mon[34294]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:42:57.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:56 smithi005 ceph-mon[30365]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:42:57.477 INFO:journalctl@ceph.mds.k.smithi131.stdout:May 09 00:42:57 smithi131 podman[63652]: 2024-05-09 00:42:57.119401508 +0000 UTC m=+0.018359042 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T00:42:57.477 INFO:journalctl@ceph.mds.k.smithi131.stdout:May 09 00:42:57 smithi131 podman[63652]: 2024-05-09 00:42:57.237513805 +0000 UTC m=+0.136471362 container create 6b161fb19c155a80d32256cb62308c9ba27d8c96ca48ac764b7ad7809ac103e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-k, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-05-09T00:42:57.477 INFO:journalctl@ceph.mds.k.smithi131.stdout:May 09 00:42:57 smithi131 podman[63652]: 2024-05-09 00:42:57.440298886 +0000 UTC m=+0.339256401 container init 6b161fb19c155a80d32256cb62308c9ba27d8c96ca48ac764b7ad7809ac103e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-k, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T00:42:57.477 INFO:journalctl@ceph.mds.k.smithi131.stdout:May 09 00:42:57 smithi131 podman[63652]: 2024-05-09 00:42:57.442879745 +0000 UTC m=+0.341837263 container start 6b161fb19c155a80d32256cb62308c9ba27d8c96ca48ac764b7ad7809ac103e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-k, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5) 2024-05-09T00:42:57.477 INFO:journalctl@ceph.mds.k.smithi131.stdout:May 09 00:42:57 smithi131 ceph-mds[63671]: -- 172.21.15.131:0/1920202248 <== mon.2 v2:172.21.15.131:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56461c083a40 con 0x56461c0cb180 2024-05-09T00:42:57.551 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T00:42:57.829 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:42:57.856 INFO:journalctl@ceph.mds.k.smithi131.stdout:May 09 00:42:57 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-k[63667]: starting mds.k at 2024-05-09T00:42:57.856 INFO:journalctl@ceph.mds.k.smithi131.stdout:May 09 00:42:57 smithi131 bash[63652]: 6b161fb19c155a80d32256cb62308c9ba27d8c96ca48ac764b7ad7809ac103e3 2024-05-09T00:42:57.856 INFO:journalctl@ceph.mds.k.smithi131.stdout:May 09 00:42:57 smithi131 systemd[1]: Started Ceph mds.k for d85814c6-0d9b-11ef-bc97-c7b262605968. 2024-05-09T00:42:58.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:58 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:58.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:58 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:58.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:58 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:58.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:58 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:42:58.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:58 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:42:58.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:58 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:42:58.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:58 smithi131 ceph-mon[34547]: Deploying daemon mds.j on smithi005 2024-05-09T00:42:58.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:58 smithi131 ceph-mon[34547]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:42:58.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:58 smithi131 ceph-mon[34547]: mds.? [v2:172.21.15.131:6832/3272897742,v1:172.21.15.131:6833/3272897742] up:boot 2024-05-09T00:42:58.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:58 smithi131 ceph-mon[34547]: fsmap 2 up:standby 2024-05-09T00:42:58.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:42:58 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-09T00:42:58.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:58 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:58.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:58 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:58.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:58 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:58.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:58 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:42:58.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:58 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:42:58.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:58 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:42:58.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:58 smithi157 ceph-mon[34294]: Deploying daemon mds.j on smithi005 2024-05-09T00:42:58.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:58 smithi157 ceph-mon[34294]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:42:58.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:58 smithi157 ceph-mon[34294]: mds.? [v2:172.21.15.131:6832/3272897742,v1:172.21.15.131:6833/3272897742] up:boot 2024-05-09T00:42:58.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:58 smithi157 ceph-mon[34294]: fsmap 2 up:standby 2024-05-09T00:42:58.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:42:58 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-09T00:42:58.984 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:58 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:58.984 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:58 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:58.984 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:58 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:42:58.985 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:58 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:42:58.985 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:58 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:42:58.985 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:58 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:42:58.985 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:58 smithi005 ceph-mon[30365]: Deploying daemon mds.j on smithi005 2024-05-09T00:42:58.985 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:58 smithi005 ceph-mon[30365]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:42:58.985 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:58 smithi005 ceph-mon[30365]: mds.? [v2:172.21.15.131:6832/3272897742,v1:172.21.15.131:6833/3272897742] up:boot 2024-05-09T00:42:58.985 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:58 smithi005 ceph-mon[30365]: fsmap 2 up:standby 2024-05-09T00:42:58.985 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:42:58 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-09T00:43:00.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:00.808+0000 7fcfa139d700 1 -- 172.21.15.5:0/60897016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf9c107df0 msgr2=0x7fcf9c108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:00.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:00.808+0000 7fcfa139d700 1 --2- 172.21.15.5:0/60897016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf9c107df0 0x7fcf9c108200 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcf900099c0 tx=0x7fcf90009cd0 comp rx=0 tx=0).stop 2024-05-09T00:43:00.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:00.808+0000 7fcfa139d700 1 -- 172.21.15.5:0/60897016 shutdown_connections 2024-05-09T00:43:00.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:00.808+0000 7fcfa139d700 1 --2- 172.21.15.5:0/60897016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf9c108740 0x7fcf9c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:00.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:00.808+0000 7fcfa139d700 1 --2- 172.21.15.5:0/60897016 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf9c1034c0 0x7fcf9c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:00.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:00.808+0000 7fcfa139d700 1 --2- 172.21.15.5:0/60897016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf9c107df0 0x7fcf9c108200 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:00.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:00.808+0000 7fcfa139d700 1 -- 172.21.15.5:0/60897016 >> 172.21.15.5:0/60897016 conn(0x7fcf9c0fedc0 msgr2=0x7fcf9c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:43:00.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:00.808+0000 7fcfa139d700 1 -- 172.21.15.5:0/60897016 shutdown_connections 2024-05-09T00:43:00.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:00.808+0000 7fcfa139d700 1 -- 172.21.15.5:0/60897016 wait complete. 2024-05-09T00:43:00.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:00.809+0000 7fcfa139d700 1 Processor -- start 2024-05-09T00:43:00.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:00.810+0000 7fcfa139d700 1 -- start start 2024-05-09T00:43:00.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:00.810+0000 7fcfa139d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf9c1034c0 0x7fcf9c19f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:43:00.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:00.811+0000 7fcfa139d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf9c107df0 0x7fcf9c19fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:43:00.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:00.811+0000 7fcfa139d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf9c108740 0x7fcf9c199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:43:00.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:00.811+0000 7fcfa139d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcf9c113350 con 0x7fcf9c1034c0 2024-05-09T00:43:00.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:00.811+0000 7fcf9affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf9c1034c0 0x7fcf9c19f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:43:00.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:00.811+0000 7fcfa139d700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcf9c1131d0 con 0x7fcf9c107df0 2024-05-09T00:43:00.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:00.811+0000 7fcfa139d700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcf9c1134d0 con 0x7fcf9c108740 2024-05-09T00:43:00.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:00.811+0000 7fcf9affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf9c1034c0 0x7fcf9c19f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32976/0 (socket says 172.21.15.5:32976) 2024-05-09T00:43:00.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:00.811+0000 7fcf9affd700 1 -- 172.21.15.5:0/1898773808 learned_addr learned my addr 172.21.15.5:0/1898773808 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:43:00.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:00.811+0000 7fcf9a7fc700 1 --2- 172.21.15.5:0/1898773808 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf9c107df0 0x7fcf9c19fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:43:00.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:00.811+0000 7fcf9b7fe700 1 --2- 172.21.15.5:0/1898773808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf9c108740 0x7fcf9c199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:43:00.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:00.811+0000 7fcf9affd700 1 -- 172.21.15.5:0/1898773808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf9c108740 msgr2=0x7fcf9c199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:00.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:00.811+0000 7fcf9affd700 1 --2- 172.21.15.5:0/1898773808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf9c108740 0x7fcf9c199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:00.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:00.811+0000 7fcf9affd700 1 -- 172.21.15.5:0/1898773808 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf9c107df0 msgr2=0x7fcf9c19fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:00.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:00.811+0000 7fcf9affd700 1 --2- 172.21.15.5:0/1898773808 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf9c107df0 0x7fcf9c19fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:00.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:00.811+0000 7fcf9affd700 1 -- 172.21.15.5:0/1898773808 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf9c19a1d0 con 0x7fcf9c1034c0 2024-05-09T00:43:00.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:00.811+0000 7fcf9affd700 1 --2- 172.21.15.5:0/1898773808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf9c1034c0 0x7fcf9c19f8b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c010e40 tx=0x7fcf8c00b5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:43:00.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:00.812+0000 7fcf83fff700 1 -- 172.21.15.5:0/1898773808 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf8c00bdc0 con 0x7fcf9c1034c0 2024-05-09T00:43:00.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:00.812+0000 7fcfa139d700 1 -- 172.21.15.5:0/1898773808 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf9c19a4c0 con 0x7fcf9c1034c0 2024-05-09T00:43:00.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:00.812+0000 7fcf83fff700 1 -- 172.21.15.5:0/1898773808 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf8c0044d0 con 0x7fcf9c1034c0 2024-05-09T00:43:00.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:00.812+0000 7fcf83fff700 1 -- 172.21.15.5:0/1898773808 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf8c019670 con 0x7fcf9c1034c0 2024-05-09T00:43:00.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:00.812+0000 7fcfa139d700 1 -- 172.21.15.5:0/1898773808 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf9c06b610 con 0x7fcf9c1034c0 2024-05-09T00:43:00.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:00.813+0000 7fcfa139d700 1 -- 172.21.15.5:0/1898773808 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf9c0512c0 con 0x7fcf9c1034c0 2024-05-09T00:43:00.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:00.816+0000 7fcf83fff700 1 -- 172.21.15.5:0/1898773808 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fcf8c0037a0 con 0x7fcf9c1034c0 2024-05-09T00:43:00.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:00.816+0000 7fcf83fff700 1 --2- 172.21.15.5:0/1898773808 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fcf840b2900 0x7fcf840b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:43:00.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:00.817+0000 7fcf9a7fc700 1 --2- 172.21.15.5:0/1898773808 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fcf840b2900 0x7fcf840b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:43:00.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:00.817+0000 7fcf83fff700 1 -- 172.21.15.5:0/1898773808 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fcf8c0e5970 con 0x7fcf9c1034c0 2024-05-09T00:43:00.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:00.817+0000 7fcf9a7fc700 1 --2- 172.21.15.5:0/1898773808 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fcf840b2900 0x7fcf840b4e10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fcf90000c00 tx=0x7fcf9001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:43:00.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:00.821+0000 7fcf83fff700 1 -- 172.21.15.5:0/1898773808 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcf8c0ada90 con 0x7fcf9c1034c0 2024-05-09T00:43:01.108 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:00 smithi005 ceph-mon[30365]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:01.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:01.106+0000 7fcfa139d700 1 -- 172.21.15.5:0/1898773808 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcf9c19afe0 con 0x7fcf9c1034c0 2024-05-09T00:43:01.108 INFO:journalctl@ceph.mds.j.smithi005.stdout:May 09 00:43:01 smithi005 podman[70956]: 2024-05-09 00:43:01.049928894 +0000 UTC m=+0.146765933 container create 8569b4a1ca697e56da3ba3fbc190dfe2933f197a3eadc9725de9be5fc58a4568 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-j, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T00:43:01.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:01.107+0000 7fcf83fff700 1 -- 172.21.15.5:0/1898773808 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7fcf8c0ad390 con 0x7fcf9c1034c0 2024-05-09T00:43:01.109 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:43:01.109 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/3272897742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3272897742},{"type":"v1","addr":"172.21.15.131:6833","nonce":3272897742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2809525455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2809525455},{"type":"v1","addr":"172.21.15.157:6833","nonce":2809525455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-09T00:43:01.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:01.109+0000 7fcfa139d700 1 -- 172.21.15.5:0/1898773808 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fcf840b2900 msgr2=0x7fcf840b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:01.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:01.109+0000 7fcfa139d700 1 --2- 172.21.15.5:0/1898773808 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fcf840b2900 0x7fcf840b4e10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fcf90000c00 tx=0x7fcf9001a040 comp rx=0 tx=0).stop 2024-05-09T00:43:01.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:01.109+0000 7fcfa139d700 1 -- 172.21.15.5:0/1898773808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf9c1034c0 msgr2=0x7fcf9c19f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:01.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:01.109+0000 7fcfa139d700 1 --2- 172.21.15.5:0/1898773808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf9c1034c0 0x7fcf9c19f8b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c010e40 tx=0x7fcf8c00b5a0 comp rx=0 tx=0).stop 2024-05-09T00:43:01.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:01.110+0000 7fcfa139d700 1 -- 172.21.15.5:0/1898773808 shutdown_connections 2024-05-09T00:43:01.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:01.110+0000 7fcfa139d700 1 --2- 172.21.15.5:0/1898773808 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fcf840b2900 0x7fcf840b4e10 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:01.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:01.110+0000 7fcfa139d700 1 --2- 172.21.15.5:0/1898773808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf9c108740 0x7fcf9c199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:01.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:01.110+0000 7fcfa139d700 1 --2- 172.21.15.5:0/1898773808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf9c1034c0 0x7fcf9c19f8b0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:01.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:01.110+0000 7fcfa139d700 1 --2- 172.21.15.5:0/1898773808 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf9c107df0 0x7fcf9c19fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:01.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:01.110+0000 7fcfa139d700 1 -- 172.21.15.5:0/1898773808 >> 172.21.15.5:0/1898773808 conn(0x7fcf9c0fedc0 msgr2=0x7fcf9c1096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:43:01.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:01.110+0000 7fcfa139d700 1 -- 172.21.15.5:0/1898773808 shutdown_connections 2024-05-09T00:43:01.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:01.110+0000 7fcfa139d700 1 -- 172.21.15.5:0/1898773808 wait complete. 2024-05-09T00:43:01.112 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 3 2024-05-09T00:43:01.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:00 smithi157 ceph-mon[34294]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:01.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:00 smithi131 ceph-mon[34547]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:02.017 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:01 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/1898773808' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:43:02.018 INFO:journalctl@ceph.mds.j.smithi005.stdout:May 09 00:43:01 smithi005 podman[70956]: 2024-05-09 00:43:01.916712201 +0000 UTC m=+1.013549241 container init 8569b4a1ca697e56da3ba3fbc190dfe2933f197a3eadc9725de9be5fc58a4568 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-j, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=) 2024-05-09T00:43:02.018 INFO:journalctl@ceph.mds.j.smithi005.stdout:May 09 00:43:01 smithi005 podman[70956]: 2024-05-09 00:43:01.920138786 +0000 UTC m=+1.016975826 container start 8569b4a1ca697e56da3ba3fbc190dfe2933f197a3eadc9725de9be5fc58a4568 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True) 2024-05-09T00:43:02.018 INFO:journalctl@ceph.mds.j.smithi005.stdout:May 09 00:43:01 smithi005 ceph-mds[70990]: -- 172.21.15.5:0/2537210279 <== mon.0 v2:172.21.15.5:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ac49197a40 con 0x55ac491dfa80 2024-05-09T00:43:02.018 INFO:journalctl@ceph.mds.j.smithi005.stdout:May 09 00:43:01 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-j[70985]: starting mds.j at 2024-05-09T00:43:02.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:01 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/1898773808' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:43:02.291 INFO:journalctl@ceph.mds.j.smithi005.stdout:May 09 00:43:02 smithi005 bash[70956]: 8569b4a1ca697e56da3ba3fbc190dfe2933f197a3eadc9725de9be5fc58a4568 2024-05-09T00:43:02.291 INFO:journalctl@ceph.mds.j.smithi005.stdout:May 09 00:43:02 smithi005 systemd[1]: Started Ceph mds.j for d85814c6-0d9b-11ef-bc97-c7b262605968. 2024-05-09T00:43:02.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:01 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/1898773808' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:43:03.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:03 smithi005 ceph-mon[30365]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:03.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:03 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:03.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:03 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:03.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:03 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:03.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:03 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:43:03.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:03 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:43:03.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:03 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:43:03.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:03 smithi005 ceph-mon[30365]: Deploying daemon mds.d on smithi005 2024-05-09T00:43:03.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:03 smithi005 ceph-mon[30365]: mds.? [v2:172.21.15.5:6834/2685771887,v1:172.21.15.5:6835/2685771887] up:boot 2024-05-09T00:43:03.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:03 smithi005 ceph-mon[30365]: fsmap 3 up:standby 2024-05-09T00:43:03.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:03 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-09T00:43:03.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:03 smithi131 ceph-mon[34547]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:03.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:03 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:03.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:03 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:03.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:03 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:03.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:03 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:43:03.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:03 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:43:03.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:03 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:43:03.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:03 smithi131 ceph-mon[34547]: Deploying daemon mds.d on smithi005 2024-05-09T00:43:03.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:03 smithi131 ceph-mon[34547]: mds.? [v2:172.21.15.5:6834/2685771887,v1:172.21.15.5:6835/2685771887] up:boot 2024-05-09T00:43:03.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:03 smithi131 ceph-mon[34547]: fsmap 3 up:standby 2024-05-09T00:43:03.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:03 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-09T00:43:03.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:03 smithi157 ceph-mon[34294]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:03.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:03 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:03.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:03 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:03.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:03 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:03.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:03 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:43:03.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:03 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:43:03.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:03 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:43:03.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:03 smithi157 ceph-mon[34294]: Deploying daemon mds.d on smithi005 2024-05-09T00:43:03.438 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:03 smithi157 ceph-mon[34294]: mds.? [v2:172.21.15.5:6834/2685771887,v1:172.21.15.5:6835/2685771887] up:boot 2024-05-09T00:43:03.438 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:03 smithi157 ceph-mon[34294]: fsmap 3 up:standby 2024-05-09T00:43:03.438 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:03 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-09T00:43:03.780 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T00:43:04.067 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:43:05.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:05 smithi131 ceph-mon[34547]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:05.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:05 smithi157 ceph-mon[34294]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:05.476 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:05 smithi005 ceph-mon[30365]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:05.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:05.968+0000 7f9cc7bf8700 1 -- 172.21.15.5:0/1006793810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9cc0104330 msgr2=0x7f9cc0110010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:05.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:05.968+0000 7f9cc7bf8700 1 --2- 172.21.15.5:0/1006793810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9cc0104330 0x7f9cc0110010 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9cb800b0a0 tx=0x7f9cb800b3b0 comp rx=0 tx=0).stop 2024-05-09T00:43:05.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:05.968+0000 7f9cc7bf8700 1 -- 172.21.15.5:0/1006793810 shutdown_connections 2024-05-09T00:43:05.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:05.968+0000 7f9cc7bf8700 1 --2- 172.21.15.5:0/1006793810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9cc0104330 0x7f9cc0110010 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:05.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:05.968+0000 7f9cc7bf8700 1 --2- 172.21.15.5:0/1006793810 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cc01092d0 0x7f9cc01096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:05.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:05.968+0000 7f9cc7bf8700 1 --2- 172.21.15.5:0/1006793810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cc01039e0 0x7f9cc0103df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:05.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:05.968+0000 7f9cc7bf8700 1 -- 172.21.15.5:0/1006793810 >> 172.21.15.5:0/1006793810 conn(0x7f9cc00fe540 msgr2=0x7f9cc0100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:43:05.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:05.968+0000 7f9cc7bf8700 1 -- 172.21.15.5:0/1006793810 shutdown_connections 2024-05-09T00:43:05.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:05.968+0000 7f9cc7bf8700 1 -- 172.21.15.5:0/1006793810 wait complete. 2024-05-09T00:43:05.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:05.969+0000 7f9cc7bf8700 1 Processor -- start 2024-05-09T00:43:05.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:05.969+0000 7f9cc7bf8700 1 -- start start 2024-05-09T00:43:05.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:05.969+0000 7f9cc7bf8700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cc01039e0 0x7f9cc019bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:43:05.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:05.969+0000 7f9cc7bf8700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cc0104330 0x7f9cc019c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:43:05.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:05.969+0000 7f9cc7bf8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9cc01092d0 0x7f9cc019c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:43:05.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:05.969+0000 7f9cc7bf8700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc0113370 con 0x7f9cc01092d0 2024-05-09T00:43:05.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:05.969+0000 7f9cc7bf8700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc01131f0 con 0x7f9cc01039e0 2024-05-09T00:43:05.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:05.969+0000 7f9cc7bf8700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc01134f0 con 0x7f9cc0104330 2024-05-09T00:43:05.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:05.969+0000 7f9cc6195700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9cc01092d0 0x7f9cc019c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:43:05.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:05.969+0000 7f9cc6195700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9cc01092d0 0x7f9cc019c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34430/0 (socket says 172.21.15.5:34430) 2024-05-09T00:43:05.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:05.969+0000 7f9cc6195700 1 -- 172.21.15.5:0/3078223999 learned_addr learned my addr 172.21.15.5:0/3078223999 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:43:05.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:05.969+0000 7f9cc5193700 1 --2- 172.21.15.5:0/3078223999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cc0104330 0x7f9cc019c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:43:05.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:05.969+0000 7f9cc5994700 1 --2- 172.21.15.5:0/3078223999 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cc01039e0 0x7f9cc019bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:43:05.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:05.970+0000 7f9cc6195700 1 -- 172.21.15.5:0/3078223999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cc0104330 msgr2=0x7f9cc019c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:05.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:05.970+0000 7f9cc6195700 1 --2- 172.21.15.5:0/3078223999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cc0104330 0x7f9cc019c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:05.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:05.970+0000 7f9cc6195700 1 -- 172.21.15.5:0/3078223999 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cc01039e0 msgr2=0x7f9cc019bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:05.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:05.970+0000 7f9cc6195700 1 --2- 172.21.15.5:0/3078223999 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cc01039e0 0x7f9cc019bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:05.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:05.970+0000 7f9cc6195700 1 -- 172.21.15.5:0/3078223999 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cc01a6f20 con 0x7f9cc01092d0 2024-05-09T00:43:05.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:05.970+0000 7f9cc6195700 1 --2- 172.21.15.5:0/3078223999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9cc01092d0 0x7f9cc019c670 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9cb800b830 tx=0x7f9cb8004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:43:05.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:05.970+0000 7f9cb6ffd700 1 -- 172.21.15.5:0/3078223999 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb8021070 con 0x7f9cc01092d0 2024-05-09T00:43:05.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:05.970+0000 7f9cb6ffd700 1 -- 172.21.15.5:0/3078223999 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cb8007e30 con 0x7f9cc01092d0 2024-05-09T00:43:05.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:05.970+0000 7f9cc7bf8700 1 -- 172.21.15.5:0/3078223999 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cc01a71b0 con 0x7f9cc01092d0 2024-05-09T00:43:05.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:05.970+0000 7f9cb6ffd700 1 -- 172.21.15.5:0/3078223999 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb8004700 con 0x7f9cc01092d0 2024-05-09T00:43:05.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:05.970+0000 7f9cc7bf8700 1 -- 172.21.15.5:0/3078223999 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cc01a76b0 con 0x7f9cc01092d0 2024-05-09T00:43:05.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:05.971+0000 7f9cc7bf8700 1 -- 172.21.15.5:0/3078223999 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cc00512c0 con 0x7f9cc01092d0 2024-05-09T00:43:05.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:05.974+0000 7f9cb6ffd700 1 -- 172.21.15.5:0/3078223999 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f9cb8003d70 con 0x7f9cc01092d0 2024-05-09T00:43:05.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:05.974+0000 7f9cb6ffd700 1 --2- 172.21.15.5:0/3078223999 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f9cac0b2900 0x7f9cac0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:43:05.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:05.975+0000 7f9cc5994700 1 --2- 172.21.15.5:0/3078223999 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f9cac0b2900 0x7f9cac0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:43:05.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:05.975+0000 7f9cb6ffd700 1 -- 172.21.15.5:0/3078223999 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f9cb800f020 con 0x7f9cc01092d0 2024-05-09T00:43:05.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:05.975+0000 7f9cc5994700 1 --2- 172.21.15.5:0/3078223999 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f9cac0b2900 0x7f9cac0b4e10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc005950 tx=0x7f9cbc00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:43:05.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:05.978+0000 7f9cb6ffd700 1 -- 172.21.15.5:0/3078223999 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9cb802a020 con 0x7f9cc01092d0 2024-05-09T00:43:06.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:06.248+0000 7f9cc7bf8700 1 -- 172.21.15.5:0/3078223999 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9cc002e7b0 con 0x7f9cc01092d0 2024-05-09T00:43:06.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:06.249+0000 7f9cb6ffd700 1 -- 172.21.15.5:0/3078223999 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3046 (secure 0 0 0) 0x7f9cb80ab840 con 0x7f9cc01092d0 2024-05-09T00:43:06.250 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:43:06.250 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2685771887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2685771887},{"type":"v1","addr":"172.21.15.5:6835","nonce":2685771887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/3272897742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3272897742},{"type":"v1","addr":"172.21.15.131:6833","nonce":3272897742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2809525455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2809525455},{"type":"v1","addr":"172.21.15.157:6833","nonce":2809525455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-09T00:43:06.251 INFO:journalctl@ceph.mds.d.smithi005.stdout:May 09 00:43:06 smithi005 podman[71366]: 2024-05-09 00:43:06.12601763 +0000 UTC m=+1.173475285 container create c546b98bbf6a116bf988abcec95d62c860ff0440294a2e8e558ca53991a12f84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-d, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True) 2024-05-09T00:43:06.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:06.251+0000 7f9cc7bf8700 1 -- 172.21.15.5:0/3078223999 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f9cac0b2900 msgr2=0x7f9cac0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:06.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:06.251+0000 7f9cc7bf8700 1 --2- 172.21.15.5:0/3078223999 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f9cac0b2900 0x7f9cac0b4e10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc005950 tx=0x7f9cbc00a300 comp rx=0 tx=0).stop 2024-05-09T00:43:06.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:06.251+0000 7f9cc7bf8700 1 -- 172.21.15.5:0/3078223999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9cc01092d0 msgr2=0x7f9cc019c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:06.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:06.251+0000 7f9cc7bf8700 1 --2- 172.21.15.5:0/3078223999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9cc01092d0 0x7f9cc019c670 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9cb800b830 tx=0x7f9cb8004ce0 comp rx=0 tx=0).stop 2024-05-09T00:43:06.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:06.251+0000 7f9cc7bf8700 1 -- 172.21.15.5:0/3078223999 shutdown_connections 2024-05-09T00:43:06.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:06.251+0000 7f9cc7bf8700 1 --2- 172.21.15.5:0/3078223999 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f9cac0b2900 0x7f9cac0b4e10 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:06.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:06.251+0000 7f9cc7bf8700 1 --2- 172.21.15.5:0/3078223999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9cc01092d0 0x7f9cc019c670 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:06.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:06.251+0000 7f9cc7bf8700 1 --2- 172.21.15.5:0/3078223999 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cc01039e0 0x7f9cc019bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:06.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:06.251+0000 7f9cc7bf8700 1 --2- 172.21.15.5:0/3078223999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cc0104330 0x7f9cc019c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:06.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:06.251+0000 7f9cc7bf8700 1 -- 172.21.15.5:0/3078223999 >> 172.21.15.5:0/3078223999 conn(0x7f9cc00fe540 msgr2=0x7f9cc0100ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:43:06.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:06.252+0000 7f9cc7bf8700 1 -- 172.21.15.5:0/3078223999 shutdown_connections 2024-05-09T00:43:06.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:06.252+0000 7f9cc7bf8700 1 -- 172.21.15.5:0/3078223999 wait complete. 2024-05-09T00:43:06.254 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 4 2024-05-09T00:43:06.791 INFO:journalctl@ceph.mds.d.smithi005.stdout:May 09 00:43:06 smithi005 podman[71366]: 2024-05-09 00:43:06.624563678 +0000 UTC m=+1.672021332 container init c546b98bbf6a116bf988abcec95d62c860ff0440294a2e8e558ca53991a12f84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-d, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T00:43:06.791 INFO:journalctl@ceph.mds.d.smithi005.stdout:May 09 00:43:06 smithi005 podman[71366]: 2024-05-09 00:43:06.62800655 +0000 UTC m=+1.675464199 container start c546b98bbf6a116bf988abcec95d62c860ff0440294a2e8e558ca53991a12f84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-d, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T00:43:06.791 INFO:journalctl@ceph.mds.d.smithi005.stdout:May 09 00:43:06 smithi005 ceph-mds[71452]: -- 172.21.15.5:0/3872103179 <== mon.0 v2:172.21.15.5:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55749c531a40 con 0x55749c579600 2024-05-09T00:43:06.792 INFO:journalctl@ceph.mds.d.smithi005.stdout:May 09 00:43:06 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-d[71431]: starting mds.d at 2024-05-09T00:43:06.792 INFO:journalctl@ceph.mds.d.smithi005.stdout:May 09 00:43:06 smithi005 bash[71366]: c546b98bbf6a116bf988abcec95d62c860ff0440294a2e8e558ca53991a12f84 2024-05-09T00:43:06.792 INFO:journalctl@ceph.mds.d.smithi005.stdout:May 09 00:43:06 smithi005 systemd[1]: Started Ceph mds.d for d85814c6-0d9b-11ef-bc97-c7b262605968. 2024-05-09T00:43:07.203 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:07 smithi005 ceph-mon[30365]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:07.203 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:07 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/3078223999' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:43:07.204 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:07 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:07.204 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:07 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:07.204 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:07 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:07.204 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:07 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:43:07.204 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:07 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:43:07.204 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:07 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:43:07.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:07 smithi131 ceph-mon[34547]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:07.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:07 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/3078223999' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:43:07.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:07 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:07.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:07 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:07.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:07 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:07.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:07 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:43:07.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:07 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:43:07.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:07 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:43:07.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:07 smithi157 ceph-mon[34294]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:07.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:07 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/3078223999' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:43:07.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:07 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:07.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:07 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:07.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:07 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:07.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:07 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:43:07.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:07 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:43:07.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:07 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:43:08.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:08 smithi131 ceph-mon[34547]: Deploying daemon mds.e on smithi131 2024-05-09T00:43:08.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:08 smithi131 ceph-mon[34547]: mds.? [v2:172.21.15.5:6836/254017535,v1:172.21.15.5:6837/254017535] up:boot 2024-05-09T00:43:08.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:08 smithi131 ceph-mon[34547]: fsmap 4 up:standby 2024-05-09T00:43:08.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:08 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-09T00:43:08.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:08 smithi157 ceph-mon[34294]: Deploying daemon mds.e on smithi131 2024-05-09T00:43:08.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:08 smithi157 ceph-mon[34294]: mds.? [v2:172.21.15.5:6836/254017535,v1:172.21.15.5:6837/254017535] up:boot 2024-05-09T00:43:08.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:08 smithi157 ceph-mon[34294]: fsmap 4 up:standby 2024-05-09T00:43:08.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:08 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-09T00:43:08.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:08 smithi005 ceph-mon[30365]: Deploying daemon mds.e on smithi131 2024-05-09T00:43:08.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:08 smithi005 ceph-mon[30365]: mds.? [v2:172.21.15.5:6836/254017535,v1:172.21.15.5:6837/254017535] up:boot 2024-05-09T00:43:08.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:08 smithi005 ceph-mon[30365]: fsmap 4 up:standby 2024-05-09T00:43:08.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:08 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-09T00:43:09.090 INFO:journalctl@ceph.mds.e.smithi131.stdout:May 09 00:43:08 smithi131 systemd[1]: Starting Ceph mds.e for d85814c6-0d9b-11ef-bc97-c7b262605968... 2024-05-09T00:43:09.231 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T00:43:09.353 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:09 smithi131 ceph-mon[34547]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:09.354 INFO:journalctl@ceph.mds.e.smithi131.stdout:May 09 00:43:09 smithi131 podman[63973]: 2024-05-09 00:43:09.098207378 +0000 UTC m=+0.018940605 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T00:43:09.354 INFO:journalctl@ceph.mds.e.smithi131.stdout:May 09 00:43:09 smithi131 podman[63973]: 2024-05-09 00:43:09.209710549 +0000 UTC m=+0.130443764 container create 30bda55db0ed02e5af8b8fb07128aebf64e4430f76311cb1e675f1855fe47da1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-e, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-09T00:43:09.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:09 smithi157 ceph-mon[34294]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:09.499 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:43:09.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:09 smithi005 ceph-mon[30365]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:09.841 INFO:journalctl@ceph.mds.e.smithi131.stdout:May 09 00:43:09 smithi131 podman[63973]: 2024-05-09 00:43:09.465022492 +0000 UTC m=+0.385755705 container init 30bda55db0ed02e5af8b8fb07128aebf64e4430f76311cb1e675f1855fe47da1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-e, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T00:43:09.841 INFO:journalctl@ceph.mds.e.smithi131.stdout:May 09 00:43:09 smithi131 podman[63973]: 2024-05-09 00:43:09.467924933 +0000 UTC m=+0.388658146 container start 30bda55db0ed02e5af8b8fb07128aebf64e4430f76311cb1e675f1855fe47da1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-e, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-05-09T00:43:09.841 INFO:journalctl@ceph.mds.e.smithi131.stdout:May 09 00:43:09 smithi131 ceph-mds[63992]: -- 172.21.15.131:0/2920553464 <== mon.2 v2:172.21.15.131:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563efb995a40 con 0x563efb9dda80 2024-05-09T00:43:09.841 INFO:journalctl@ceph.mds.e.smithi131.stdout:May 09 00:43:09 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-e[63988]: starting mds.e at 2024-05-09T00:43:09.842 INFO:journalctl@ceph.mds.e.smithi131.stdout:May 09 00:43:09 smithi131 bash[63973]: 30bda55db0ed02e5af8b8fb07128aebf64e4430f76311cb1e675f1855fe47da1 2024-05-09T00:43:09.842 INFO:journalctl@ceph.mds.e.smithi131.stdout:May 09 00:43:09 smithi131 systemd[1]: Started Ceph mds.e for d85814c6-0d9b-11ef-bc97-c7b262605968. 2024-05-09T00:43:10.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:10 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:10.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:10 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:10.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:10 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:10.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:10 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:43:10.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:10 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:43:10.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:10 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:43:10.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:10 smithi131 ceph-mon[34547]: Deploying daemon mds.f on smithi157 2024-05-09T00:43:10.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:10 smithi131 ceph-mon[34547]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:10.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:10 smithi131 ceph-mon[34547]: mds.? [v2:172.21.15.131:6834/1283093898,v1:172.21.15.131:6835/1283093898] up:boot 2024-05-09T00:43:10.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:10 smithi131 ceph-mon[34547]: fsmap 5 up:standby 2024-05-09T00:43:10.857 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:10 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-09T00:43:10.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:10 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:10.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:10 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:10.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:10 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:10.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:10 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:43:10.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:10 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:43:10.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:10 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:43:10.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:10 smithi157 ceph-mon[34294]: Deploying daemon mds.f on smithi157 2024-05-09T00:43:10.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:10 smithi157 ceph-mon[34294]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:10.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:10 smithi157 ceph-mon[34294]: mds.? [v2:172.21.15.131:6834/1283093898,v1:172.21.15.131:6835/1283093898] up:boot 2024-05-09T00:43:10.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:10 smithi157 ceph-mon[34294]: fsmap 5 up:standby 2024-05-09T00:43:10.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:10 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-09T00:43:10.969 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:10 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:10.969 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:10 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:10.970 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:10 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:10.970 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:10 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:43:10.970 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:10 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:43:10.970 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:10 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:43:10.970 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:10 smithi005 ceph-mon[30365]: Deploying daemon mds.f on smithi157 2024-05-09T00:43:10.970 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:10 smithi005 ceph-mon[30365]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:10.970 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:10 smithi005 ceph-mon[30365]: mds.? [v2:172.21.15.131:6834/1283093898,v1:172.21.15.131:6835/1283093898] up:boot 2024-05-09T00:43:10.970 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:10 smithi005 ceph-mon[30365]: fsmap 5 up:standby 2024-05-09T00:43:10.970 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:10 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-09T00:43:11.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:11.215+0000 7f23e04ca700 1 -- 172.21.15.5:0/2037700487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23d8108740 msgr2=0x7f23d81100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:11.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:11.215+0000 7f23e04ca700 1 --2- 172.21.15.5:0/2037700487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23d8108740 0x7f23d81100a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f23d000b0d0 tx=0x7f23d000b3e0 comp rx=0 tx=0).stop 2024-05-09T00:43:11.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:11.215+0000 7f23e04ca700 1 -- 172.21.15.5:0/2037700487 shutdown_connections 2024-05-09T00:43:11.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:11.215+0000 7f23e04ca700 1 --2- 172.21.15.5:0/2037700487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23d8108740 0x7f23d81100a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:11.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:11.215+0000 7f23e04ca700 1 --2- 172.21.15.5:0/2037700487 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23d81034c0 0x7f23d81038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:11.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:11.215+0000 7f23e04ca700 1 --2- 172.21.15.5:0/2037700487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23d8107df0 0x7f23d8108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:11.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:11.215+0000 7f23e04ca700 1 -- 172.21.15.5:0/2037700487 >> 172.21.15.5:0/2037700487 conn(0x7f23d80fedc0 msgr2=0x7f23d81011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:43:11.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:11.215+0000 7f23e04ca700 1 -- 172.21.15.5:0/2037700487 shutdown_connections 2024-05-09T00:43:11.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:11.215+0000 7f23e04ca700 1 -- 172.21.15.5:0/2037700487 wait complete. 2024-05-09T00:43:11.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:11.216+0000 7f23e04ca700 1 Processor -- start 2024-05-09T00:43:11.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:11.216+0000 7f23e04ca700 1 -- start start 2024-05-09T00:43:11.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:11.216+0000 7f23e04ca700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23d81034c0 0x7f23d819bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:43:11.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:11.216+0000 7f23e04ca700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23d8107df0 0x7f23d819c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:43:11.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:11.216+0000 7f23e04ca700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23d8108740 0x7f23d819c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:43:11.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:11.216+0000 7f23e04ca700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f23d8113390 con 0x7f23d8108740 2024-05-09T00:43:11.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:11.216+0000 7f23e04ca700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f23d8113210 con 0x7f23d81034c0 2024-05-09T00:43:11.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:11.216+0000 7f23e04ca700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f23d8113510 con 0x7f23d8107df0 2024-05-09T00:43:11.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:11.216+0000 7f23dea67700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23d8108740 0x7f23d819c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:43:11.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:11.216+0000 7f23dea67700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23d8108740 0x7f23d819c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34476/0 (socket says 172.21.15.5:34476) 2024-05-09T00:43:11.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:11.216+0000 7f23dea67700 1 -- 172.21.15.5:0/1562023477 learned_addr learned my addr 172.21.15.5:0/1562023477 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:43:11.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:11.216+0000 7f23de266700 1 --2- 172.21.15.5:0/1562023477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23d81034c0 0x7f23d819bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:43:11.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:11.216+0000 7f23dda65700 1 --2- 172.21.15.5:0/1562023477 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23d8107df0 0x7f23d819c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:43:11.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:11.217+0000 7f23dea67700 1 -- 172.21.15.5:0/1562023477 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23d8107df0 msgr2=0x7f23d819c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:11.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:11.217+0000 7f23dea67700 1 --2- 172.21.15.5:0/1562023477 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23d8107df0 0x7f23d819c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:11.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:11.217+0000 7f23dea67700 1 -- 172.21.15.5:0/1562023477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23d81034c0 msgr2=0x7f23d819bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:11.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:11.217+0000 7f23dea67700 1 --2- 172.21.15.5:0/1562023477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23d81034c0 0x7f23d819bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:11.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:11.217+0000 7f23dea67700 1 -- 172.21.15.5:0/1562023477 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23d81a6f20 con 0x7f23d8108740 2024-05-09T00:43:11.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:11.217+0000 7f23dda65700 1 --2- 172.21.15.5:0/1562023477 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23d8107df0 0x7f23d819c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:43:11.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:11.217+0000 7f23dea67700 1 --2- 172.21.15.5:0/1562023477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23d8108740 0x7f23d819c670 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f23d0009850 tx=0x7f23d0007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:43:11.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:11.217+0000 7f23cf7fe700 1 -- 172.21.15.5:0/1562023477 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d001f070 con 0x7f23d8108740 2024-05-09T00:43:11.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:11.217+0000 7f23e04ca700 1 -- 172.21.15.5:0/1562023477 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23d81a71b0 con 0x7f23d8108740 2024-05-09T00:43:11.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:11.217+0000 7f23cf7fe700 1 -- 172.21.15.5:0/1562023477 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23d0004c50 con 0x7f23d8108740 2024-05-09T00:43:11.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:11.217+0000 7f23cf7fe700 1 -- 172.21.15.5:0/1562023477 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d001b7b0 con 0x7f23d8108740 2024-05-09T00:43:11.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:11.217+0000 7f23e04ca700 1 -- 172.21.15.5:0/1562023477 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23d81a7610 con 0x7f23d8108740 2024-05-09T00:43:11.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:11.218+0000 7f23e04ca700 1 -- 172.21.15.5:0/1562023477 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23d80512c0 con 0x7f23d8108740 2024-05-09T00:43:11.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:11.220+0000 7f23cf7fe700 1 -- 172.21.15.5:0/1562023477 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f23d0022070 con 0x7f23d8108740 2024-05-09T00:43:11.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:11.221+0000 7f23cf7fe700 1 --2- 172.21.15.5:0/1562023477 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f23c40b2880 0x7f23c40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:43:11.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:11.221+0000 7f23de266700 1 --2- 172.21.15.5:0/1562023477 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f23c40b2880 0x7f23c40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:43:11.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:11.221+0000 7f23cf7fe700 1 -- 172.21.15.5:0/1562023477 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f23d000f020 con 0x7f23d8108740 2024-05-09T00:43:11.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:11.221+0000 7f23de266700 1 --2- 172.21.15.5:0/1562023477 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f23c40b2880 0x7f23c40b4d90 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f23d4009c60 tx=0x7f23d4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:43:11.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:11.227+0000 7f23cf7fe700 1 -- 172.21.15.5:0/1562023477 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f23d0029020 con 0x7f23d8108740 2024-05-09T00:43:11.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:11.488+0000 7f23e04ca700 1 -- 172.21.15.5:0/1562023477 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f23d802e7b0 con 0x7f23d8108740 2024-05-09T00:43:11.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:11.489+0000 7f23cf7fe700 1 -- 172.21.15.5:0/1562023477 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4693 (secure 0 0 0) 0x7f23d00ab550 con 0x7f23d8108740 2024-05-09T00:43:11.490 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:43:11.490 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2685771887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2685771887},{"type":"v1","addr":"172.21.15.5:6835","nonce":2685771887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/254017535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":254017535},{"type":"v1","addr":"172.21.15.5:6837","nonce":254017535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/3272897742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3272897742},{"type":"v1","addr":"172.21.15.131:6833","nonce":3272897742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/1283093898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1283093898},{"type":"v1","addr":"172.21.15.131:6835","nonce":1283093898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2809525455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2809525455},{"type":"v1","addr":"172.21.15.157:6833","nonce":2809525455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-09T00:43:11.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:11.491+0000 7f23e04ca700 1 -- 172.21.15.5:0/1562023477 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f23c40b2880 msgr2=0x7f23c40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:11.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:11.491+0000 7f23e04ca700 1 --2- 172.21.15.5:0/1562023477 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f23c40b2880 0x7f23c40b4d90 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f23d4009c60 tx=0x7f23d4009260 comp rx=0 tx=0).stop 2024-05-09T00:43:11.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:11.491+0000 7f23e04ca700 1 -- 172.21.15.5:0/1562023477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23d8108740 msgr2=0x7f23d819c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:11.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:11.491+0000 7f23e04ca700 1 --2- 172.21.15.5:0/1562023477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23d8108740 0x7f23d819c670 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f23d0009850 tx=0x7f23d0007a20 comp rx=0 tx=0).stop 2024-05-09T00:43:11.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:11.491+0000 7f23e04ca700 1 -- 172.21.15.5:0/1562023477 shutdown_connections 2024-05-09T00:43:11.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:11.491+0000 7f23e04ca700 1 --2- 172.21.15.5:0/1562023477 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f23c40b2880 0x7f23c40b4d90 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:11.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:11.491+0000 7f23e04ca700 1 --2- 172.21.15.5:0/1562023477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23d8108740 0x7f23d819c670 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:11.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:11.491+0000 7f23e04ca700 1 --2- 172.21.15.5:0/1562023477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23d81034c0 0x7f23d819bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:11.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:11.491+0000 7f23e04ca700 1 --2- 172.21.15.5:0/1562023477 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23d8107df0 0x7f23d819c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:11.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:11.491+0000 7f23e04ca700 1 -- 172.21.15.5:0/1562023477 >> 172.21.15.5:0/1562023477 conn(0x7f23d80fedc0 msgr2=0x7f23d8107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:43:11.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:11.491+0000 7f23e04ca700 1 -- 172.21.15.5:0/1562023477 shutdown_connections 2024-05-09T00:43:11.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:11.491+0000 7f23e04ca700 1 -- 172.21.15.5:0/1562023477 wait complete. 2024-05-09T00:43:11.494 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 6 2024-05-09T00:43:11.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:11 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/1562023477' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:43:11.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:11 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/1562023477' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:43:11.937 INFO:journalctl@ceph.mds.f.smithi157.stdout:May 09 00:43:11 smithi157 systemd[1]: Starting Ceph mds.f for d85814c6-0d9b-11ef-bc97-c7b262605968... 2024-05-09T00:43:12.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:11 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/1562023477' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:43:12.390 INFO:journalctl@ceph.mds.f.smithi157.stdout:May 09 00:43:12 smithi157 podman[64239]: 2024-05-09 00:43:11.964151826 +0000 UTC m=+0.018101639 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T00:43:12.390 INFO:journalctl@ceph.mds.f.smithi157.stdout:May 09 00:43:12 smithi157 podman[64239]: 2024-05-09 00:43:12.120101215 +0000 UTC m=+0.174051046 container create e5281c48d3084a1e59d21fe4ad892c6cd3313fa9208fad7d0c615f1a6d3d12af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-f, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, ceph=True, CEPH_POINT_RELEASE=) 2024-05-09T00:43:12.390 INFO:journalctl@ceph.mds.f.smithi157.stdout:May 09 00:43:12 smithi157 podman[64239]: 2024-05-09 00:43:12.389196168 +0000 UTC m=+0.443145953 container init e5281c48d3084a1e59d21fe4ad892c6cd3313fa9208fad7d0c615f1a6d3d12af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-05-09T00:43:12.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:12 smithi157 ceph-mon[34294]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:12.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:12 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:12.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:12 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:12.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:12 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:12.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:12 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:43:12.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:12 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:43:12.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:12 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:43:12.687 INFO:journalctl@ceph.mds.f.smithi157.stdout:May 09 00:43:12 smithi157 podman[64239]: 2024-05-09 00:43:12.39183547 +0000 UTC m=+0.445785258 container start e5281c48d3084a1e59d21fe4ad892c6cd3313fa9208fad7d0c615f1a6d3d12af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-f, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-05-09T00:43:12.687 INFO:journalctl@ceph.mds.f.smithi157.stdout:May 09 00:43:12 smithi157 ceph-mds[64258]: -- 172.21.15.157:0/1325957173 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555aa8c2da40 con 0x555aa8c75180 2024-05-09T00:43:12.687 INFO:journalctl@ceph.mds.f.smithi157.stdout:May 09 00:43:12 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-f[64254]: starting mds.f at 2024-05-09T00:43:12.688 INFO:journalctl@ceph.mds.f.smithi157.stdout:May 09 00:43:12 smithi157 bash[64239]: e5281c48d3084a1e59d21fe4ad892c6cd3313fa9208fad7d0c615f1a6d3d12af 2024-05-09T00:43:12.688 INFO:journalctl@ceph.mds.f.smithi157.stdout:May 09 00:43:12 smithi157 systemd[1]: Started Ceph mds.f for d85814c6-0d9b-11ef-bc97-c7b262605968. 2024-05-09T00:43:13.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:12 smithi005 ceph-mon[30365]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:13.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:12 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:13.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:12 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:13.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:12 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:13.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:12 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:43:13.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:12 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:43:13.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:12 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:43:13.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:12 smithi131 ceph-mon[34547]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:13.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:12 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:13.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:12 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:13.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:12 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:13.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:12 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:43:13.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:12 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:43:13.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:12 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:43:13.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:13 smithi157 ceph-mon[34294]: Deploying daemon mds.c on smithi157 2024-05-09T00:43:13.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:13 smithi157 ceph-mon[34294]: mds.? [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] up:boot 2024-05-09T00:43:13.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:13 smithi157 ceph-mon[34294]: fsmap 6 up:standby 2024-05-09T00:43:13.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:13 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-09T00:43:14.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:13 smithi005 ceph-mon[30365]: Deploying daemon mds.c on smithi157 2024-05-09T00:43:14.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:13 smithi005 ceph-mon[30365]: mds.? [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] up:boot 2024-05-09T00:43:14.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:13 smithi005 ceph-mon[30365]: fsmap 6 up:standby 2024-05-09T00:43:14.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:13 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-09T00:43:14.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:13 smithi131 ceph-mon[34547]: Deploying daemon mds.c on smithi157 2024-05-09T00:43:14.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:13 smithi131 ceph-mon[34547]: mds.? [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] up:boot 2024-05-09T00:43:14.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:13 smithi131 ceph-mon[34547]: fsmap 6 up:standby 2024-05-09T00:43:14.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:13 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-09T00:43:14.168 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T00:43:14.433 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:43:14.867 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:14 smithi005 ceph-mon[30365]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:14.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:14 smithi157 ceph-mon[34294]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:14.937 INFO:journalctl@ceph.mds.c.smithi157.stdout:May 09 00:43:14 smithi157 systemd[1]: Starting Ceph mds.c for d85814c6-0d9b-11ef-bc97-c7b262605968... 2024-05-09T00:43:15.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:14 smithi131 ceph-mon[34547]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:15.229 INFO:journalctl@ceph.mds.c.smithi157.stdout:May 09 00:43:14 smithi157 podman[64549]: 2024-05-09 00:43:14.868227715 +0000 UTC m=+0.017871276 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T00:43:15.229 INFO:journalctl@ceph.mds.c.smithi157.stdout:May 09 00:43:14 smithi157 podman[64549]: 2024-05-09 00:43:14.975858229 +0000 UTC m=+0.125501776 container create 7b1acef3ad12f84662f6151558b6034649e614abb548d90ccbfc920a810a59c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-c, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-05-09T00:43:15.229 INFO:journalctl@ceph.mds.c.smithi157.stdout:May 09 00:43:15 smithi157 podman[64549]: 2024-05-09 00:43:15.228422466 +0000 UTC m=+0.378066013 container init 7b1acef3ad12f84662f6151558b6034649e614abb548d90ccbfc920a810a59c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240507, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T00:43:15.686 INFO:journalctl@ceph.mds.c.smithi157.stdout:May 09 00:43:15 smithi157 podman[64549]: 2024-05-09 00:43:15.23116351 +0000 UTC m=+0.380807059 container start 7b1acef3ad12f84662f6151558b6034649e614abb548d90ccbfc920a810a59c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-c, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-05-09T00:43:15.686 INFO:journalctl@ceph.mds.c.smithi157.stdout:May 09 00:43:15 smithi157 ceph-mds[64569]: -- 172.21.15.157:0/2686485791 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55aa8d3a5a40 con 0x55aa8d3eda80 2024-05-09T00:43:15.687 INFO:journalctl@ceph.mds.c.smithi157.stdout:May 09 00:43:15 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-c[64565]: starting mds.c at 2024-05-09T00:43:15.687 INFO:journalctl@ceph.mds.c.smithi157.stdout:May 09 00:43:15 smithi157 bash[64549]: 7b1acef3ad12f84662f6151558b6034649e614abb548d90ccbfc920a810a59c0 2024-05-09T00:43:15.687 INFO:journalctl@ceph.mds.c.smithi157.stdout:May 09 00:43:15 smithi157 systemd[1]: Started Ceph mds.c for d85814c6-0d9b-11ef-bc97-c7b262605968. 2024-05-09T00:43:16.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:16.111+0000 7f0523344700 1 -- 172.21.15.5:0/892767221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f051c107df0 msgr2=0x7f051c108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:16.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:16.111+0000 7f0523344700 1 --2- 172.21.15.5:0/892767221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f051c107df0 0x7f051c108200 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f050c0099c0 tx=0x7f050c009cd0 comp rx=0 tx=0).stop 2024-05-09T00:43:16.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:16.111+0000 7f0523344700 1 -- 172.21.15.5:0/892767221 shutdown_connections 2024-05-09T00:43:16.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:16.111+0000 7f0523344700 1 --2- 172.21.15.5:0/892767221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f051c108740 0x7f051c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:16.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:16.111+0000 7f0523344700 1 --2- 172.21.15.5:0/892767221 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f051c1034c0 0x7f051c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:16.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:16.111+0000 7f0523344700 1 --2- 172.21.15.5:0/892767221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f051c107df0 0x7f051c108200 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:16.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:16.111+0000 7f0523344700 1 -- 172.21.15.5:0/892767221 >> 172.21.15.5:0/892767221 conn(0x7f051c0fedf0 msgr2=0x7f051c101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:43:16.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:16.111+0000 7f0523344700 1 -- 172.21.15.5:0/892767221 shutdown_connections 2024-05-09T00:43:16.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:16.111+0000 7f0523344700 1 -- 172.21.15.5:0/892767221 wait complete. 2024-05-09T00:43:16.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:16.112+0000 7f0523344700 1 Processor -- start 2024-05-09T00:43:16.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:16.112+0000 7f0523344700 1 -- start start 2024-05-09T00:43:16.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:16.112+0000 7f0523344700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f051c1034c0 0x7f051c199da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:43:16.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:16.112+0000 7f0523344700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f051c107df0 0x7f051c19a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:43:16.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:16.112+0000 7f0523344700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f051c108740 0x7f051c19e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:43:16.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:16.112+0000 7f0523344700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f051c113350 con 0x7f051c1034c0 2024-05-09T00:43:16.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:16.112+0000 7f0523344700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f051c1131d0 con 0x7f051c108740 2024-05-09T00:43:16.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:16.112+0000 7f0523344700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f051c1134d0 con 0x7f051c107df0 2024-05-09T00:43:16.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:16.112+0000 7f05210e0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f051c1034c0 0x7f051c199da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:43:16.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:16.112+0000 7f05210e0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f051c1034c0 0x7f051c199da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52316/0 (socket says 172.21.15.5:52316) 2024-05-09T00:43:16.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:16.112+0000 7f05210e0700 1 -- 172.21.15.5:0/718955204 learned_addr learned my addr 172.21.15.5:0/718955204 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:43:16.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:16.113+0000 7f05218e1700 1 --2- 172.21.15.5:0/718955204 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f051c108740 0x7f051c19e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:43:16.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:16.113+0000 7f05208df700 1 --2- 172.21.15.5:0/718955204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f051c107df0 0x7f051c19a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:43:16.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:16.113+0000 7f05210e0700 1 -- 172.21.15.5:0/718955204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f051c107df0 msgr2=0x7f051c19a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:16.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:16.113+0000 7f05210e0700 1 --2- 172.21.15.5:0/718955204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f051c107df0 0x7f051c19a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:16.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:16.113+0000 7f05210e0700 1 -- 172.21.15.5:0/718955204 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f051c108740 msgr2=0x7f051c19e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:16.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:16.113+0000 7f05210e0700 1 --2- 172.21.15.5:0/718955204 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f051c108740 0x7f051c19e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:16.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:16.113+0000 7f05210e0700 1 -- 172.21.15.5:0/718955204 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f051c19ec70 con 0x7f051c1034c0 2024-05-09T00:43:16.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:16.113+0000 7f05210e0700 1 --2- 172.21.15.5:0/718955204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f051c1034c0 0x7f051c199da0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0518009970 tx=0x7f0518009c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:43:16.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:16.113+0000 7f05127fc700 1 -- 172.21.15.5:0/718955204 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f051801c070 con 0x7f051c1034c0 2024-05-09T00:43:16.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:16.113+0000 7f0523344700 1 -- 172.21.15.5:0/718955204 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f051c19ef60 con 0x7f051c1034c0 2024-05-09T00:43:16.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:16.113+0000 7f05127fc700 1 -- 172.21.15.5:0/718955204 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f051800d260 con 0x7f051c1034c0 2024-05-09T00:43:16.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:16.113+0000 7f05127fc700 1 -- 172.21.15.5:0/718955204 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0518017a60 con 0x7f051c1034c0 2024-05-09T00:43:16.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:16.113+0000 7f0523344700 1 -- 172.21.15.5:0/718955204 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f051c0759f0 con 0x7f051c1034c0 2024-05-09T00:43:16.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:16.114+0000 7f0523344700 1 -- 172.21.15.5:0/718955204 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f051c104970 con 0x7f051c1034c0 2024-05-09T00:43:16.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:16.116+0000 7f05127fc700 1 -- 172.21.15.5:0/718955204 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f051801f070 con 0x7f051c1034c0 2024-05-09T00:43:16.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:16.116+0000 7f05127fc700 1 --2- 172.21.15.5:0/718955204 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f05080b2880 0x7f05080b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:43:16.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:16.117+0000 7f05208df700 1 --2- 172.21.15.5:0/718955204 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f05080b2880 0x7f05080b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:43:16.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:16.117+0000 7f05127fc700 1 -- 172.21.15.5:0/718955204 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f05180e32c0 con 0x7f051c1034c0 2024-05-09T00:43:16.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:16.117+0000 7f05208df700 1 --2- 172.21.15.5:0/718955204 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f05080b2880 0x7f05080b4d90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f050c000c00 tx=0x7f050c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:43:16.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:16.120+0000 7f05127fc700 1 -- 172.21.15.5:0/718955204 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f05180ab480 con 0x7f051c1034c0 2024-05-09T00:43:16.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:16.381+0000 7f0523344700 1 -- 172.21.15.5:0/718955204 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f051c19b130 con 0x7f051c1034c0 2024-05-09T00:43:16.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:16.382+0000 7f05127fc700 1 -- 172.21.15.5:0/718955204 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6343 (secure 0 0 0) 0x7f051c19b130 con 0x7f051c1034c0 2024-05-09T00:43:16.383 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:43:16.383 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2685771887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2685771887},{"type":"v1","addr":"172.21.15.5:6835","nonce":2685771887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/254017535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":254017535},{"type":"v1","addr":"172.21.15.5:6837","nonce":254017535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/3272897742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3272897742},{"type":"v1","addr":"172.21.15.131:6833","nonce":3272897742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/1283093898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1283093898},{"type":"v1","addr":"172.21.15.131:6835","nonce":1283093898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2809525455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2809525455},{"type":"v1","addr":"172.21.15.157:6833","nonce":2809525455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/832898273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":832898273},{"type":"v1","addr":"172.21.15.157:6835","nonce":832898273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/744863009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":744863009},{"type":"v1","addr":"172.21.15.157:6837","nonce":744863009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-05-09T00:43:16.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:16.385+0000 7f0523344700 1 -- 172.21.15.5:0/718955204 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f05080b2880 msgr2=0x7f05080b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:16.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:16.385+0000 7f0523344700 1 --2- 172.21.15.5:0/718955204 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f05080b2880 0x7f05080b4d90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f050c000c00 tx=0x7f050c01a040 comp rx=0 tx=0).stop 2024-05-09T00:43:16.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:16.385+0000 7f0523344700 1 -- 172.21.15.5:0/718955204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f051c1034c0 msgr2=0x7f051c199da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:16.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:16.385+0000 7f0523344700 1 --2- 172.21.15.5:0/718955204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f051c1034c0 0x7f051c199da0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0518009970 tx=0x7f0518009c80 comp rx=0 tx=0).stop 2024-05-09T00:43:16.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:16.385+0000 7f0523344700 1 -- 172.21.15.5:0/718955204 shutdown_connections 2024-05-09T00:43:16.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:16.385+0000 7f0523344700 1 --2- 172.21.15.5:0/718955204 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f05080b2880 0x7f05080b4d90 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:16.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:16.385+0000 7f0523344700 1 --2- 172.21.15.5:0/718955204 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f051c108740 0x7f051c19e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:16.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:16.385+0000 7f0523344700 1 --2- 172.21.15.5:0/718955204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f051c1034c0 0x7f051c199da0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:16.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:16.385+0000 7f0523344700 1 --2- 172.21.15.5:0/718955204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f051c107df0 0x7f051c19a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:16.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:16.385+0000 7f0523344700 1 -- 172.21.15.5:0/718955204 >> 172.21.15.5:0/718955204 conn(0x7f051c0fedf0 msgr2=0x7f051c1096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:43:16.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:16.385+0000 7f0523344700 1 -- 172.21.15.5:0/718955204 shutdown_connections 2024-05-09T00:43:16.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:16.385+0000 7f0523344700 1 -- 172.21.15.5:0/718955204 wait complete. 2024-05-09T00:43:16.387 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 8 2024-05-09T00:43:16.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:16 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:16.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:16 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:16.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:16 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:16.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:16 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:43:16.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:16 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:43:16.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:16 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:43:16.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:16 smithi157 ceph-mon[34294]: Deploying daemon mds.h on smithi131 2024-05-09T00:43:16.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:16 smithi157 ceph-mon[34294]: mds.? [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] up:boot 2024-05-09T00:43:16.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:16 smithi157 ceph-mon[34294]: fsmap 7 up:standby 2024-05-09T00:43:16.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:16 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T00:43:16.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:16 smithi157 ceph-mon[34294]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:16.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:16 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/718955204' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:43:16.701 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:16 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:16.701 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:16 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:16.702 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:16 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:16.702 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:16 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:43:16.702 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:16 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:43:16.702 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:16 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:43:16.702 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:16 smithi005 ceph-mon[30365]: Deploying daemon mds.h on smithi131 2024-05-09T00:43:16.702 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:16 smithi005 ceph-mon[30365]: mds.? [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] up:boot 2024-05-09T00:43:16.702 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:16 smithi005 ceph-mon[30365]: fsmap 7 up:standby 2024-05-09T00:43:16.702 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:16 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T00:43:16.703 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:16 smithi005 ceph-mon[30365]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:16.703 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:16 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/718955204' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:43:16.831 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:16 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:16.831 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:16 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:16.831 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:16 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:16.831 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:16 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:43:16.831 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:16 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:43:16.831 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:16 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:43:16.831 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:16 smithi131 ceph-mon[34547]: Deploying daemon mds.h on smithi131 2024-05-09T00:43:16.831 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:16 smithi131 ceph-mon[34547]: mds.? [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] up:boot 2024-05-09T00:43:16.831 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:16 smithi131 ceph-mon[34547]: fsmap 7 up:standby 2024-05-09T00:43:16.831 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:16 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T00:43:16.832 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:16 smithi131 ceph-mon[34547]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:16.832 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:16 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/718955204' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:43:17.856 INFO:journalctl@ceph.mds.h.smithi131.stdout:May 09 00:43:17 smithi131 systemd[1]: Starting Ceph mds.h for d85814c6-0d9b-11ef-bc97-c7b262605968... 2024-05-09T00:43:18.159 INFO:journalctl@ceph.mds.h.smithi131.stdout:May 09 00:43:17 smithi131 podman[64284]: 2024-05-09 00:43:17.794756262 +0000 UTC m=+0.017903967 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T00:43:18.159 INFO:journalctl@ceph.mds.h.smithi131.stdout:May 09 00:43:17 smithi131 podman[64284]: 2024-05-09 00:43:17.893671214 +0000 UTC m=+0.116818902 container create ab258a8336b81c38d4b9066332ec596848cd2f1f72b11ed3249a9603aea10f25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-h, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-05-09T00:43:18.160 INFO:journalctl@ceph.mds.h.smithi131.stdout:May 09 00:43:18 smithi131 podman[64284]: 2024-05-09 00:43:18.123016138 +0000 UTC m=+0.346163831 container init ab258a8336b81c38d4b9066332ec596848cd2f1f72b11ed3249a9603aea10f25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-h, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux ) 2024-05-09T00:43:18.160 INFO:journalctl@ceph.mds.h.smithi131.stdout:May 09 00:43:18 smithi131 podman[64284]: 2024-05-09 00:43:18.125614214 +0000 UTC m=+0.348761904 container start ab258a8336b81c38d4b9066332ec596848cd2f1f72b11ed3249a9603aea10f25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-h, ceph=True, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-05-09T00:43:18.160 INFO:journalctl@ceph.mds.h.smithi131.stdout:May 09 00:43:18 smithi131 ceph-mds[64303]: -- 172.21.15.131:0/2280400555 <== mon.2 v2:172.21.15.131:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55af1e681a40 con 0x55af1e6c9600 2024-05-09T00:43:18.606 INFO:journalctl@ceph.mds.h.smithi131.stdout:May 09 00:43:18 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-h[64299]: starting mds.h at 2024-05-09T00:43:18.607 INFO:journalctl@ceph.mds.h.smithi131.stdout:May 09 00:43:18 smithi131 bash[64284]: ab258a8336b81c38d4b9066332ec596848cd2f1f72b11ed3249a9603aea10f25 2024-05-09T00:43:18.607 INFO:journalctl@ceph.mds.h.smithi131.stdout:May 09 00:43:18 smithi131 systemd[1]: Started Ceph mds.h for d85814c6-0d9b-11ef-bc97-c7b262605968. 2024-05-09T00:43:19.017 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T00:43:19.136 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:18 smithi157 ceph-mon[34294]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:19.136 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:18 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:19.136 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:18 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:19.136 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:18 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:19.136 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:18 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:43:19.136 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:18 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:43:19.137 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:18 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:43:19.283 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:43:19.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:18 smithi005 ceph-mon[30365]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:19.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:18 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:19.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:18 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:19.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:18 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:19.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:18 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:43:19.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:18 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:43:19.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:18 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:43:19.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:18 smithi131 ceph-mon[34547]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:19.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:18 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:19.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:18 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:19.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:18 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:19.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:18 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:43:19.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:18 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:43:19.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:18 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:43:20.028 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:19 smithi157 ceph-mon[34294]: Deploying daemon mds.i on smithi157 2024-05-09T00:43:20.029 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:19 smithi157 ceph-mon[34294]: mds.? [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] up:boot 2024-05-09T00:43:20.029 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:19 smithi157 ceph-mon[34294]: fsmap 8 up:standby 2024-05-09T00:43:20.029 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:19 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T00:43:20.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:19 smithi005 ceph-mon[30365]: Deploying daemon mds.i on smithi157 2024-05-09T00:43:20.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:19 smithi005 ceph-mon[30365]: mds.? [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] up:boot 2024-05-09T00:43:20.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:19 smithi005 ceph-mon[30365]: fsmap 8 up:standby 2024-05-09T00:43:20.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:19 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T00:43:20.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:19 smithi131 ceph-mon[34547]: Deploying daemon mds.i on smithi157 2024-05-09T00:43:20.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:19 smithi131 ceph-mon[34547]: mds.? [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] up:boot 2024-05-09T00:43:20.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:19 smithi131 ceph-mon[34547]: fsmap 8 up:standby 2024-05-09T00:43:20.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:19 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T00:43:20.686 INFO:journalctl@ceph.mds.i.smithi157.stdout:May 09 00:43:20 smithi157 systemd[1]: Starting Ceph mds.i for d85814c6-0d9b-11ef-bc97-c7b262605968... 2024-05-09T00:43:20.969 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:20 smithi157 ceph-mon[34294]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:20.970 INFO:journalctl@ceph.mds.i.smithi157.stdout:May 09 00:43:20 smithi157 podman[64853]: 2024-05-09 00:43:20.600178734 +0000 UTC m=+0.017957918 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T00:43:20.970 INFO:journalctl@ceph.mds.i.smithi157.stdout:May 09 00:43:20 smithi157 podman[64853]: 2024-05-09 00:43:20.714178726 +0000 UTC m=+0.131957900 container create a20c29b1f166f0fd8013c74f229f7f2ac920d0db7f033341c306d98d66aaed96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-i, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2) 2024-05-09T00:43:20.970 INFO:journalctl@ceph.mds.i.smithi157.stdout:May 09 00:43:20 smithi157 podman[64853]: 2024-05-09 00:43:20.933545546 +0000 UTC m=+0.351324717 container init a20c29b1f166f0fd8013c74f229f7f2ac920d0db7f033341c306d98d66aaed96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-i, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux ) 2024-05-09T00:43:20.970 INFO:journalctl@ceph.mds.i.smithi157.stdout:May 09 00:43:20 smithi157 podman[64853]: 2024-05-09 00:43:20.936358683 +0000 UTC m=+0.354137863 container start a20c29b1f166f0fd8013c74f229f7f2ac920d0db7f033341c306d98d66aaed96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-i, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-05-09T00:43:20.970 INFO:journalctl@ceph.mds.i.smithi157.stdout:May 09 00:43:20 smithi157 ceph-mds[64872]: -- 172.21.15.157:0/4272407906 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557176bdba40 con 0x557176c23180 2024-05-09T00:43:20.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:20.974+0000 7f54db3ea700 1 -- 172.21.15.5:0/873911811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54d41034e0 msgr2=0x7f54d41038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:20.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:20.974+0000 7f54db3ea700 1 --2- 172.21.15.5:0/873911811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54d41034e0 0x7f54d41038f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f54d00099c0 tx=0x7f54d0009cd0 comp rx=0 tx=0).stop 2024-05-09T00:43:20.976 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:20 smithi005 ceph-mon[30365]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:20.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:20.974+0000 7f54db3ea700 1 -- 172.21.15.5:0/873911811 shutdown_connections 2024-05-09T00:43:20.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:20.974+0000 7f54db3ea700 1 --2- 172.21.15.5:0/873911811 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54d4108760 0x7f54d41100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:20.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:20.974+0000 7f54db3ea700 1 --2- 172.21.15.5:0/873911811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54d41034e0 0x7f54d41038f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:20.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:20.974+0000 7f54db3ea700 1 --2- 172.21.15.5:0/873911811 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54d4107e10 0x7f54d4108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:20.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:20.974+0000 7f54db3ea700 1 -- 172.21.15.5:0/873911811 >> 172.21.15.5:0/873911811 conn(0x7f54d40fef30 msgr2=0x7f54d4101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:43:20.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:20.975+0000 7f54db3ea700 1 -- 172.21.15.5:0/873911811 shutdown_connections 2024-05-09T00:43:20.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:20.975+0000 7f54db3ea700 1 -- 172.21.15.5:0/873911811 wait complete. 2024-05-09T00:43:20.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:20.976+0000 7f54db3ea700 1 Processor -- start 2024-05-09T00:43:20.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:20.976+0000 7f54db3ea700 1 -- start start 2024-05-09T00:43:20.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:20.976+0000 7f54db3ea700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54d41034e0 0x7f54d419bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:43:20.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:20.976+0000 7f54db3ea700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54d4107e10 0x7f54d419c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:43:20.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:20.976+0000 7f54db3ea700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54d4108760 0x7f54d41a6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:43:20.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:20.976+0000 7f54db3ea700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f54d41133b0 con 0x7f54d4108760 2024-05-09T00:43:20.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:20.976+0000 7f54db3ea700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f54d4113230 con 0x7f54d4107e10 2024-05-09T00:43:20.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:20.976+0000 7f54db3ea700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f54d4113530 con 0x7f54d41034e0 2024-05-09T00:43:20.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:20.976+0000 7f54d9987700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54d4108760 0x7f54d41a6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:43:20.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:20.976+0000 7f54d9987700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54d4108760 0x7f54d41a6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52338/0 (socket says 172.21.15.5:52338) 2024-05-09T00:43:20.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:20.976+0000 7f54d9987700 1 -- 172.21.15.5:0/3694213229 learned_addr learned my addr 172.21.15.5:0/3694213229 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:43:20.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:20.976+0000 7f54d9186700 1 --2- 172.21.15.5:0/3694213229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54d41034e0 0x7f54d419bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:43:20.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:20.976+0000 7f54d8985700 1 --2- 172.21.15.5:0/3694213229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54d4107e10 0x7f54d419c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:43:20.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:20.977+0000 7f54d9987700 1 -- 172.21.15.5:0/3694213229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54d41034e0 msgr2=0x7f54d419bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:20.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:20.977+0000 7f54d9987700 1 --2- 172.21.15.5:0/3694213229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54d41034e0 0x7f54d419bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:20.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:20.977+0000 7f54d9987700 1 -- 172.21.15.5:0/3694213229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54d4107e10 msgr2=0x7f54d419c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:20.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:20.977+0000 7f54d9987700 1 --2- 172.21.15.5:0/3694213229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54d4107e10 0x7f54d419c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:20.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:20.977+0000 7f54d9987700 1 -- 172.21.15.5:0/3694213229 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54d419ca60 con 0x7f54d4108760 2024-05-09T00:43:20.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:20.977+0000 7f54d9987700 1 --2- 172.21.15.5:0/3694213229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54d4108760 0x7f54d41a6cf0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f54cc007ee0 tx=0x7f54cc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:43:20.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:20.977+0000 7f54ca7fc700 1 -- 172.21.15.5:0/3694213229 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54cc004bb0 con 0x7f54d4108760 2024-05-09T00:43:20.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:20.977+0000 7f54db3ea700 1 -- 172.21.15.5:0/3694213229 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54d4199ac0 con 0x7f54d4108760 2024-05-09T00:43:20.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:20.977+0000 7f54ca7fc700 1 -- 172.21.15.5:0/3694213229 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54cc004d50 con 0x7f54d4108760 2024-05-09T00:43:20.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:20.977+0000 7f54ca7fc700 1 -- 172.21.15.5:0/3694213229 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54cc005920 con 0x7f54d4108760 2024-05-09T00:43:20.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:20.977+0000 7f54db3ea700 1 -- 172.21.15.5:0/3694213229 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54d4199fb0 con 0x7f54d4108760 2024-05-09T00:43:20.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:20.978+0000 7f54db3ea700 1 -- 172.21.15.5:0/3694213229 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54d40512c0 con 0x7f54d4108760 2024-05-09T00:43:20.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:20.980+0000 7f54ca7fc700 1 -- 172.21.15.5:0/3694213229 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f54cc01a020 con 0x7f54d4108760 2024-05-09T00:43:20.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:20.981+0000 7f54ca7fc700 1 --2- 172.21.15.5:0/3694213229 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f54c00b2880 0x7f54c00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:43:20.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:20.981+0000 7f54d9186700 1 --2- 172.21.15.5:0/3694213229 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f54c00b2880 0x7f54c00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:43:20.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:20.981+0000 7f54ca7fc700 1 -- 172.21.15.5:0/3694213229 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f54cc0e1aa0 con 0x7f54d4108760 2024-05-09T00:43:20.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:20.981+0000 7f54d9186700 1 --2- 172.21.15.5:0/3694213229 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f54c00b2880 0x7f54c00b4d90 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f54d0000c00 tx=0x7f54d001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:43:20.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:20.984+0000 7f54ca7fc700 1 -- 172.21.15.5:0/3694213229 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f54cc010070 con 0x7f54d4108760 2024-05-09T00:43:21.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:21.251+0000 7f54db3ea700 1 -- 172.21.15.5:0/3694213229 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f54d402e5f0 con 0x7f54d4108760 2024-05-09T00:43:21.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:21.252+0000 7f54ca7fc700 1 -- 172.21.15.5:0/3694213229 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7171 (secure 0 0 0) 0x7f54cc0a9bc0 con 0x7f54d4108760 2024-05-09T00:43:21.253 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:43:21.253 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2685771887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2685771887},{"type":"v1","addr":"172.21.15.5:6835","nonce":2685771887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/254017535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":254017535},{"type":"v1","addr":"172.21.15.5:6837","nonce":254017535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/3272897742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3272897742},{"type":"v1","addr":"172.21.15.131:6833","nonce":3272897742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/1283093898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1283093898},{"type":"v1","addr":"172.21.15.131:6835","nonce":1283093898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2809525455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2809525455},{"type":"v1","addr":"172.21.15.157:6833","nonce":2809525455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24467,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/1633006186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":1633006186},{"type":"v1","addr":"172.21.15.131:6837","nonce":1633006186}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/832898273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":832898273},{"type":"v1","addr":"172.21.15.157:6835","nonce":832898273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/744863009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":744863009},{"type":"v1","addr":"172.21.15.157:6837","nonce":744863009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-05-09T00:43:21.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:21.254+0000 7f54db3ea700 1 -- 172.21.15.5:0/3694213229 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f54c00b2880 msgr2=0x7f54c00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:21.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:21.254+0000 7f54db3ea700 1 --2- 172.21.15.5:0/3694213229 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f54c00b2880 0x7f54c00b4d90 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f54d0000c00 tx=0x7f54d001a040 comp rx=0 tx=0).stop 2024-05-09T00:43:21.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:21.254+0000 7f54db3ea700 1 -- 172.21.15.5:0/3694213229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54d4108760 msgr2=0x7f54d41a6cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:21.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:21.254+0000 7f54db3ea700 1 --2- 172.21.15.5:0/3694213229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54d4108760 0x7f54d41a6cf0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f54cc007ee0 tx=0x7f54cc00d3b0 comp rx=0 tx=0).stop 2024-05-09T00:43:21.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:21.255+0000 7f54db3ea700 1 -- 172.21.15.5:0/3694213229 shutdown_connections 2024-05-09T00:43:21.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:21.255+0000 7f54db3ea700 1 --2- 172.21.15.5:0/3694213229 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f54c00b2880 0x7f54c00b4d90 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:21.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:21.255+0000 7f54db3ea700 1 --2- 172.21.15.5:0/3694213229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54d4108760 0x7f54d41a6cf0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:21.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:21.255+0000 7f54db3ea700 1 --2- 172.21.15.5:0/3694213229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54d41034e0 0x7f54d419bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:21.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:21.255+0000 7f54db3ea700 1 --2- 172.21.15.5:0/3694213229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54d4107e10 0x7f54d419c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:21.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:21.255+0000 7f54db3ea700 1 -- 172.21.15.5:0/3694213229 >> 172.21.15.5:0/3694213229 conn(0x7f54d40fef30 msgr2=0x7f54d41063c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:43:21.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:21.255+0000 7f54db3ea700 1 -- 172.21.15.5:0/3694213229 shutdown_connections 2024-05-09T00:43:21.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:21.255+0000 7f54db3ea700 1 -- 172.21.15.5:0/3694213229 wait complete. 2024-05-09T00:43:21.257 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 9 2024-05-09T00:43:21.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:20 smithi131 ceph-mon[34547]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:21.436 INFO:journalctl@ceph.mds.i.smithi157.stdout:May 09 00:43:20 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-i[64868]: starting mds.i at 2024-05-09T00:43:21.436 INFO:journalctl@ceph.mds.i.smithi157.stdout:May 09 00:43:21 smithi157 bash[64853]: a20c29b1f166f0fd8013c74f229f7f2ac920d0db7f033341c306d98d66aaed96 2024-05-09T00:43:21.437 INFO:journalctl@ceph.mds.i.smithi157.stdout:May 09 00:43:21 smithi157 systemd[1]: Started Ceph mds.i for d85814c6-0d9b-11ef-bc97-c7b262605968. 2024-05-09T00:43:22.133 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:22 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:22.133 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:22 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:22.133 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:22 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:22.133 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:22 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:43:22.133 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:22 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:43:22.133 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:22 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:43:22.133 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:22 smithi005 ceph-mon[30365]: Deploying daemon mds.g on smithi005 2024-05-09T00:43:22.133 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:22 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/3694213229' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:43:22.133 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:22 smithi005 ceph-mon[30365]: mds.? [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] up:boot 2024-05-09T00:43:22.133 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:22 smithi005 ceph-mon[30365]: fsmap 9 up:standby 2024-05-09T00:43:22.134 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:22 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T00:43:22.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:22 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:22.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:22 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:22.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:22 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:22.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:22 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:43:22.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:22 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:43:22.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:22 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:43:22.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:22 smithi131 ceph-mon[34547]: Deploying daemon mds.g on smithi005 2024-05-09T00:43:22.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:22 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/3694213229' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:43:22.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:22 smithi131 ceph-mon[34547]: mds.? [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] up:boot 2024-05-09T00:43:22.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:22 smithi131 ceph-mon[34547]: fsmap 9 up:standby 2024-05-09T00:43:22.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:22 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T00:43:22.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:22 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:22.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:22 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:22.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:22 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:22.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:22 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:43:22.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:22 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:43:22.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:22 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:43:22.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:22 smithi157 ceph-mon[34294]: Deploying daemon mds.g on smithi005 2024-05-09T00:43:22.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:22 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/3694213229' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:43:22.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:22 smithi157 ceph-mon[34294]: mds.? [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] up:boot 2024-05-09T00:43:22.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:22 smithi157 ceph-mon[34294]: fsmap 9 up:standby 2024-05-09T00:43:22.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:22 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T00:43:23.287 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:23 smithi005 ceph-mon[30365]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:23.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:23 smithi157 ceph-mon[34294]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:23.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:23 smithi131 ceph-mon[34547]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:23.892 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T00:43:23.917 INFO:journalctl@ceph.mds.g.smithi005.stdout:May 09 00:43:23 smithi005 systemd[1]: Starting Ceph mds.g for d85814c6-0d9b-11ef-bc97-c7b262605968... 2024-05-09T00:43:24.252 INFO:journalctl@ceph.mds.g.smithi005.stdout:May 09 00:43:23 smithi005 podman[72229]: 2024-05-09 00:43:23.881977116 +0000 UTC m=+0.019000483 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T00:43:24.252 INFO:journalctl@ceph.mds.g.smithi005.stdout:May 09 00:43:24 smithi005 podman[72229]: 2024-05-09 00:43:24.014485247 +0000 UTC m=+0.151508596 container create b9bda7917e09f666928b818ab5530b34a9dad48fcac19c782d8dce11a159f278 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-g, org.label-schema.build-date=20240507, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T00:43:24.252 INFO:journalctl@ceph.mds.g.smithi005.stdout:May 09 00:43:24 smithi005 podman[72229]: 2024-05-09 00:43:24.250992454 +0000 UTC m=+0.388015806 container init b9bda7917e09f666928b818ab5530b34a9dad48fcac19c782d8dce11a159f278 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-g, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240507) 2024-05-09T00:43:24.387 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:43:24.512 INFO:journalctl@ceph.mds.g.smithi005.stdout:May 09 00:43:24 smithi005 podman[72229]: 2024-05-09 00:43:24.254293935 +0000 UTC m=+0.391317284 container start b9bda7917e09f666928b818ab5530b34a9dad48fcac19c782d8dce11a159f278 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-g, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-05-09T00:43:24.512 INFO:journalctl@ceph.mds.g.smithi005.stdout:May 09 00:43:24 smithi005 ceph-mds[72292]: -- 172.21.15.5:0/2585145066 <== mon.0 v2:172.21.15.5:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5572f225da40 con 0x5572f22a5a80 2024-05-09T00:43:24.512 INFO:journalctl@ceph.mds.g.smithi005.stdout:May 09 00:43:24 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-g[72281]: starting mds.g at 2024-05-09T00:43:24.512 INFO:journalctl@ceph.mds.g.smithi005.stdout:May 09 00:43:24 smithi005 bash[72229]: b9bda7917e09f666928b818ab5530b34a9dad48fcac19c782d8dce11a159f278 2024-05-09T00:43:24.512 INFO:journalctl@ceph.mds.g.smithi005.stdout:May 09 00:43:24 smithi005 systemd[1]: Started Ceph mds.g for d85814c6-0d9b-11ef-bc97-c7b262605968. 2024-05-09T00:43:25.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:25 smithi157 ceph-mon[34294]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:25.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:25 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:25.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:25 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:25.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:25 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:25.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:25 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:43:25.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:25 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:43:25.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:25 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:43:25.477 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:25 smithi005 ceph-mon[30365]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:25.477 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:25 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:25.477 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:25 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:25.477 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:25 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:25.477 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:25 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:43:25.477 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:25 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:43:25.477 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:25 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:43:25.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:25 smithi131 ceph-mon[34547]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:25.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:25 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:25.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:25 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:25.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:25 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:25.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:25 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:43:25.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:25 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:43:25.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:25 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:43:25.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:25.980+0000 7fc08eae4700 1 -- 172.21.15.5:0/3216375596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0881092d0 msgr2=0x7fc0881096e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:25.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:25.980+0000 7fc08eae4700 1 --2- 172.21.15.5:0/3216375596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0881092d0 0x7fc0881096e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc07c009990 tx=0x7fc07c009ca0 comp rx=0 tx=0).stop 2024-05-09T00:43:25.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:25.980+0000 7fc08eae4700 1 -- 172.21.15.5:0/3216375596 shutdown_connections 2024-05-09T00:43:25.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:25.980+0000 7fc08eae4700 1 --2- 172.21.15.5:0/3216375596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc088104330 0x7fc088110010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:25.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:25.980+0000 7fc08eae4700 1 --2- 172.21.15.5:0/3216375596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0881092d0 0x7fc0881096e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:25.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:25.980+0000 7fc08eae4700 1 --2- 172.21.15.5:0/3216375596 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0881039e0 0x7fc088103df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:25.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:25.980+0000 7fc08eae4700 1 -- 172.21.15.5:0/3216375596 >> 172.21.15.5:0/3216375596 conn(0x7fc0880fe4e0 msgr2=0x7fc088100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:43:25.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:25.980+0000 7fc08eae4700 1 -- 172.21.15.5:0/3216375596 shutdown_connections 2024-05-09T00:43:25.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:25.980+0000 7fc08eae4700 1 -- 172.21.15.5:0/3216375596 wait complete. 2024-05-09T00:43:25.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:25.981+0000 7fc08eae4700 1 Processor -- start 2024-05-09T00:43:25.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:25.981+0000 7fc08eae4700 1 -- start start 2024-05-09T00:43:25.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:25.981+0000 7fc08eae4700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0881039e0 0x7fc088110aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:43:25.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:25.981+0000 7fc08eae4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc088104330 0x7fc088110fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:43:25.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:25.981+0000 7fc08eae4700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0881092d0 0x7fc088111520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:43:25.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:25.981+0000 7fc08eae4700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc088114560 con 0x7fc088104330 2024-05-09T00:43:25.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:25.981+0000 7fc08eae4700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc0881143e0 con 0x7fc0881092d0 2024-05-09T00:43:25.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:25.981+0000 7fc08eae4700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc0881146e0 con 0x7fc0881039e0 2024-05-09T00:43:25.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:25.981+0000 7fc087fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc088104330 0x7fc088110fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:43:25.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:25.981+0000 7fc087fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc088104330 0x7fc088110fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42074/0 (socket says 172.21.15.5:42074) 2024-05-09T00:43:25.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:25.981+0000 7fc087fff700 1 -- 172.21.15.5:0/2012938459 learned_addr learned my addr 172.21.15.5:0/2012938459 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:43:25.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:25.981+0000 7fc08c880700 1 --2- 172.21.15.5:0/2012938459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0881039e0 0x7fc088110aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:43:25.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:25.981+0000 7fc08d081700 1 --2- 172.21.15.5:0/2012938459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0881092d0 0x7fc088111520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:43:25.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:25.981+0000 7fc087fff700 1 -- 172.21.15.5:0/2012938459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0881039e0 msgr2=0x7fc088110aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:25.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:25.981+0000 7fc087fff700 1 --2- 172.21.15.5:0/2012938459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0881039e0 0x7fc088110aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:25.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:25.981+0000 7fc087fff700 1 -- 172.21.15.5:0/2012938459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0881092d0 msgr2=0x7fc088111520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:25.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:25.981+0000 7fc087fff700 1 --2- 172.21.15.5:0/2012938459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0881092d0 0x7fc088111520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:25.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:25.981+0000 7fc087fff700 1 -- 172.21.15.5:0/2012938459 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0881a92f0 con 0x7fc088104330 2024-05-09T00:43:25.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:25.981+0000 7fc08d081700 1 --2- 172.21.15.5:0/2012938459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0881092d0 0x7fc088111520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:43:25.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:25.982+0000 7fc087fff700 1 --2- 172.21.15.5:0/2012938459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc088104330 0x7fc088110fe0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc07400b500 tx=0x7fc07400b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:43:25.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:25.982+0000 7fc085ffb700 1 -- 172.21.15.5:0/2012938459 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc074015020 con 0x7fc088104330 2024-05-09T00:43:25.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:25.982+0000 7fc08eae4700 1 -- 172.21.15.5:0/2012938459 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0881a95e0 con 0x7fc088104330 2024-05-09T00:43:25.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:25.982+0000 7fc085ffb700 1 -- 172.21.15.5:0/2012938459 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc074011ce0 con 0x7fc088104330 2024-05-09T00:43:25.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:25.982+0000 7fc085ffb700 1 -- 172.21.15.5:0/2012938459 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc074010d40 con 0x7fc088104330 2024-05-09T00:43:25.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:25.982+0000 7fc08eae4700 1 -- 172.21.15.5:0/2012938459 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0881a9b70 con 0x7fc088104330 2024-05-09T00:43:25.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:25.983+0000 7fc08eae4700 1 -- 172.21.15.5:0/2012938459 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc08810b740 con 0x7fc088104330 2024-05-09T00:43:25.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:25.985+0000 7fc085ffb700 1 -- 172.21.15.5:0/2012938459 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fc074018020 con 0x7fc088104330 2024-05-09T00:43:25.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:25.985+0000 7fc085ffb700 1 --2- 172.21.15.5:0/2012938459 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fc0780b2880 0x7fc0780b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:43:25.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:25.986+0000 7fc08c880700 1 --2- 172.21.15.5:0/2012938459 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fc0780b2880 0x7fc0780b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:43:25.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:25.986+0000 7fc085ffb700 1 -- 172.21.15.5:0/2012938459 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fc0740e1a40 con 0x7fc088104330 2024-05-09T00:43:25.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:25.986+0000 7fc08c880700 1 --2- 172.21.15.5:0/2012938459 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fc0780b2880 0x7fc0780b4d90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc07c00b5c0 tx=0x7fc07c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:43:25.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:25.989+0000 7fc085ffb700 1 -- 172.21.15.5:0/2012938459 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc07401f020 con 0x7fc088104330 2024-05-09T00:43:26.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:26.253+0000 7fc08eae4700 1 -- 172.21.15.5:0/2012938459 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc0880512c0 con 0x7fc088104330 2024-05-09T00:43:26.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:26.254+0000 7fc085ffb700 1 -- 172.21.15.5:0/2012938459 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8824 (secure 0 0 0) 0x7fc0740a9c00 con 0x7fc088104330 2024-05-09T00:43:26.255 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:43:26.255 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2685771887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2685771887},{"type":"v1","addr":"172.21.15.5:6835","nonce":2685771887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/254017535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":254017535},{"type":"v1","addr":"172.21.15.5:6837","nonce":254017535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2109255228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2109255228},{"type":"v1","addr":"172.21.15.5:6839","nonce":2109255228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/3272897742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3272897742},{"type":"v1","addr":"172.21.15.131:6833","nonce":3272897742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/1283093898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1283093898},{"type":"v1","addr":"172.21.15.131:6835","nonce":1283093898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2809525455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2809525455},{"type":"v1","addr":"172.21.15.157:6833","nonce":2809525455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24467,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/1633006186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":1633006186},{"type":"v1","addr":"172.21.15.131:6837","nonce":1633006186}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/832898273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":832898273},{"type":"v1","addr":"172.21.15.157:6835","nonce":832898273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/744863009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":744863009},{"type":"v1","addr":"172.21.15.157:6837","nonce":744863009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24508,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6839/2779860399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":2779860399},{"type":"v1","addr":"172.21.15.157:6839","nonce":2779860399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-09T00:43:26.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:26.256+0000 7fc08eae4700 1 -- 172.21.15.5:0/2012938459 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fc0780b2880 msgr2=0x7fc0780b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:26.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:26.256+0000 7fc08eae4700 1 --2- 172.21.15.5:0/2012938459 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fc0780b2880 0x7fc0780b4d90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc07c00b5c0 tx=0x7fc07c005960 comp rx=0 tx=0).stop 2024-05-09T00:43:26.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:26.256+0000 7fc08eae4700 1 -- 172.21.15.5:0/2012938459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc088104330 msgr2=0x7fc088110fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:26.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:26.256+0000 7fc08eae4700 1 --2- 172.21.15.5:0/2012938459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc088104330 0x7fc088110fe0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc07400b500 tx=0x7fc07400b810 comp rx=0 tx=0).stop 2024-05-09T00:43:26.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:26.256+0000 7fc08eae4700 1 -- 172.21.15.5:0/2012938459 shutdown_connections 2024-05-09T00:43:26.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:26.256+0000 7fc08eae4700 1 --2- 172.21.15.5:0/2012938459 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fc0780b2880 0x7fc0780b4d90 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:26.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:26.256+0000 7fc08eae4700 1 --2- 172.21.15.5:0/2012938459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0881092d0 0x7fc088111520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:26.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:26.256+0000 7fc08eae4700 1 --2- 172.21.15.5:0/2012938459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0881039e0 0x7fc088110aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:26.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:26.256+0000 7fc08eae4700 1 --2- 172.21.15.5:0/2012938459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc088104330 0x7fc088110fe0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:26.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:26.256+0000 7fc08eae4700 1 -- 172.21.15.5:0/2012938459 >> 172.21.15.5:0/2012938459 conn(0x7fc0880fe4e0 msgr2=0x7fc088100aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:43:26.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:26.256+0000 7fc08eae4700 1 -- 172.21.15.5:0/2012938459 shutdown_connections 2024-05-09T00:43:26.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:26.256+0000 7fc08eae4700 1 -- 172.21.15.5:0/2012938459 wait complete. 2024-05-09T00:43:26.259 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 11 2024-05-09T00:43:26.358 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:26 smithi005 ceph-mon[30365]: Deploying daemon mds.a on smithi005 2024-05-09T00:43:26.358 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:26 smithi005 ceph-mon[30365]: mds.? [v2:172.21.15.5:6838/2109255228,v1:172.21.15.5:6839/2109255228] up:boot 2024-05-09T00:43:26.358 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:26 smithi005 ceph-mon[30365]: fsmap 10 up:standby 2024-05-09T00:43:26.358 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:26 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-09T00:43:26.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:26 smithi157 ceph-mon[34294]: Deploying daemon mds.a on smithi005 2024-05-09T00:43:26.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:26 smithi157 ceph-mon[34294]: mds.? [v2:172.21.15.5:6838/2109255228,v1:172.21.15.5:6839/2109255228] up:boot 2024-05-09T00:43:26.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:26 smithi157 ceph-mon[34294]: fsmap 10 up:standby 2024-05-09T00:43:26.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:26 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-09T00:43:26.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:26 smithi131 ceph-mon[34547]: Deploying daemon mds.a on smithi005 2024-05-09T00:43:26.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:26 smithi131 ceph-mon[34547]: mds.? [v2:172.21.15.5:6838/2109255228,v1:172.21.15.5:6839/2109255228] up:boot 2024-05-09T00:43:26.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:26 smithi131 ceph-mon[34547]: fsmap 10 up:standby 2024-05-09T00:43:26.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:26 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-09T00:43:27.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:27 smithi005 ceph-mon[30365]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:27.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:27 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/2012938459' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:43:27.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:27 smithi157 ceph-mon[34294]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:27.469 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:27 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/2012938459' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:43:27.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:27 smithi131 ceph-mon[34547]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:27.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:27 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/2012938459' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:43:27.862 INFO:journalctl@ceph.mds.a.smithi005.stdout:May 09 00:43:27 smithi005 systemd[1]: Starting Ceph mds.a for d85814c6-0d9b-11ef-bc97-c7b262605968... 2024-05-09T00:43:28.360 INFO:journalctl@ceph.mds.a.smithi005.stdout:May 09 00:43:28 smithi005 podman[72690]: 2024-05-09 00:43:27.987517024 +0000 UTC m=+0.018726645 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T00:43:28.361 INFO:journalctl@ceph.mds.a.smithi005.stdout:May 09 00:43:28 smithi005 podman[72690]: 2024-05-09 00:43:28.11503198 +0000 UTC m=+0.146241583 container create bc99ce756e918571edfb27ce835edd0c5f6db4366bca728d4a3be60faa2cac4a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-a, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T00:43:28.361 INFO:journalctl@ceph.mds.a.smithi005.stdout:May 09 00:43:28 smithi005 podman[72690]: 2024-05-09 00:43:28.359687099 +0000 UTC m=+0.390896695 container init bc99ce756e918571edfb27ce835edd0c5f6db4366bca728d4a3be60faa2cac4a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-a, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T00:43:28.791 INFO:journalctl@ceph.mds.a.smithi005.stdout:May 09 00:43:28 smithi005 podman[72690]: 2024-05-09 00:43:28.362968066 +0000 UTC m=+0.394177661 container start bc99ce756e918571edfb27ce835edd0c5f6db4366bca728d4a3be60faa2cac4a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T00:43:28.791 INFO:journalctl@ceph.mds.a.smithi005.stdout:May 09 00:43:28 smithi005 ceph-mds[72709]: -- 172.21.15.5:0/1487802935 <== mon.0 v2:172.21.15.5:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c495a0fa40 con 0x55c495a57a80 2024-05-09T00:43:28.791 INFO:journalctl@ceph.mds.a.smithi005.stdout:May 09 00:43:28 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-a[72705]: starting mds.a at 2024-05-09T00:43:28.792 INFO:journalctl@ceph.mds.a.smithi005.stdout:May 09 00:43:28 smithi005 bash[72690]: bc99ce756e918571edfb27ce835edd0c5f6db4366bca728d4a3be60faa2cac4a 2024-05-09T00:43:28.792 INFO:journalctl@ceph.mds.a.smithi005.stdout:May 09 00:43:28 smithi005 systemd[1]: Started Ceph mds.a for d85814c6-0d9b-11ef-bc97-c7b262605968. 2024-05-09T00:43:29.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:29 smithi157 ceph-mon[34294]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:29.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:29 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:29.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:29 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:29.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:29 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:29.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:29 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:43:29.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:29 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:43:29.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:29 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:43:29.439 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:29 smithi131 ceph-mon[34547]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:29.439 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:29 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:29.439 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:29 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:29.439 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:29 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:29.439 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:29 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:43:29.439 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:29 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:43:29.439 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:29 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:43:29.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:29 smithi005 ceph-mon[30365]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:29.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:29 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:29.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:29 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:29.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:29 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:29.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:29 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:43:29.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:29 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:43:29.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:29 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:43:29.862 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T00:43:30.132 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:43:30.436 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:30 smithi131 ceph-mon[34547]: Deploying daemon mds.b on smithi131 2024-05-09T00:43:30.436 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:30 smithi131 ceph-mon[34547]: mds.? [v2:172.21.15.5:6840/3809966329,v1:172.21.15.5:6841/3809966329] up:boot 2024-05-09T00:43:30.436 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:30 smithi131 ceph-mon[34547]: fsmap 11 up:standby 2024-05-09T00:43:30.436 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:30 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-09T00:43:30.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:30 smithi157 ceph-mon[34294]: Deploying daemon mds.b on smithi131 2024-05-09T00:43:30.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:30 smithi157 ceph-mon[34294]: mds.? [v2:172.21.15.5:6840/3809966329,v1:172.21.15.5:6841/3809966329] up:boot 2024-05-09T00:43:30.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:30 smithi157 ceph-mon[34294]: fsmap 11 up:standby 2024-05-09T00:43:30.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:30 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-09T00:43:30.464 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:30 smithi005 ceph-mon[30365]: Deploying daemon mds.b on smithi131 2024-05-09T00:43:30.464 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:30 smithi005 ceph-mon[30365]: mds.? [v2:172.21.15.5:6840/3809966329,v1:172.21.15.5:6841/3809966329] up:boot 2024-05-09T00:43:30.464 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:30 smithi005 ceph-mon[30365]: fsmap 11 up:standby 2024-05-09T00:43:30.464 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:30 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-09T00:43:30.838 INFO:journalctl@ceph.mds.b.smithi131.stdout:May 09 00:43:30 smithi131 systemd[1]: Starting Ceph mds.b for d85814c6-0d9b-11ef-bc97-c7b262605968... 2024-05-09T00:43:31.106 INFO:journalctl@ceph.mds.b.smithi131.stdout:May 09 00:43:30 smithi131 podman[64591]: 2024-05-09 00:43:30.740099636 +0000 UTC m=+0.018044671 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T00:43:31.110 INFO:journalctl@ceph.mds.b.smithi131.stdout:May 09 00:43:30 smithi131 podman[64591]: 2024-05-09 00:43:30.850788611 +0000 UTC m=+0.128733622 container create 2c466d9bfca33b33f227b54388e65cbe72a2701aed8ffe485bb70cd2622b6de7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-b, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-05-09T00:43:31.110 INFO:journalctl@ceph.mds.b.smithi131.stdout:May 09 00:43:31 smithi131 podman[64591]: 2024-05-09 00:43:31.078699817 +0000 UTC m=+0.356644831 container init 2c466d9bfca33b33f227b54388e65cbe72a2701aed8ffe485bb70cd2622b6de7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-b, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240507) 2024-05-09T00:43:31.111 INFO:journalctl@ceph.mds.b.smithi131.stdout:May 09 00:43:31 smithi131 podman[64591]: 2024-05-09 00:43:31.082158319 +0000 UTC m=+0.360103345 container start 2c466d9bfca33b33f227b54388e65cbe72a2701aed8ffe485bb70cd2622b6de7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-b, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True) 2024-05-09T00:43:31.434 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:31 smithi005 ceph-mon[30365]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:31.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:31 smithi157 ceph-mon[34294]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:31.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:31 smithi131 ceph-mon[34547]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:31.606 INFO:journalctl@ceph.mds.b.smithi131.stdout:May 09 00:43:31 smithi131 ceph-mds[64611]: -- 172.21.15.131:0/1386647199 <== mon.2 v2:172.21.15.131:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dce13dfa40 con 0x55dce1427a80 2024-05-09T00:43:31.606 INFO:journalctl@ceph.mds.b.smithi131.stdout:May 09 00:43:31 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-b[64607]: starting mds.b at 2024-05-09T00:43:31.606 INFO:journalctl@ceph.mds.b.smithi131.stdout:May 09 00:43:31 smithi131 bash[64591]: 2c466d9bfca33b33f227b54388e65cbe72a2701aed8ffe485bb70cd2622b6de7 2024-05-09T00:43:31.606 INFO:journalctl@ceph.mds.b.smithi131.stdout:May 09 00:43:31 smithi131 systemd[1]: Started Ceph mds.b for d85814c6-0d9b-11ef-bc97-c7b262605968. 2024-05-09T00:43:31.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:31.857+0000 7f51878b2700 1 -- 172.21.15.5:0/4141477526 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5180107df0 msgr2=0x7f5180108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:31.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:31.857+0000 7f51878b2700 1 --2- 172.21.15.5:0/4141477526 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5180107df0 0x7f5180108200 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f51740099c0 tx=0x7f5174009cd0 comp rx=0 tx=0).stop 2024-05-09T00:43:31.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:31.857+0000 7f51878b2700 1 -- 172.21.15.5:0/4141477526 shutdown_connections 2024-05-09T00:43:31.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:31.857+0000 7f51878b2700 1 --2- 172.21.15.5:0/4141477526 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5180108740 0x7f51801100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:31.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:31.857+0000 7f51878b2700 1 --2- 172.21.15.5:0/4141477526 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51801034c0 0x7f51801038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:31.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:31.857+0000 7f51878b2700 1 --2- 172.21.15.5:0/4141477526 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5180107df0 0x7f5180108200 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:31.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:31.857+0000 7f51878b2700 1 -- 172.21.15.5:0/4141477526 >> 172.21.15.5:0/4141477526 conn(0x7f51800fedc0 msgr2=0x7f51801011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:43:31.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:31.857+0000 7f51878b2700 1 -- 172.21.15.5:0/4141477526 shutdown_connections 2024-05-09T00:43:31.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:31.857+0000 7f51878b2700 1 -- 172.21.15.5:0/4141477526 wait complete. 2024-05-09T00:43:31.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:31.858+0000 7f51878b2700 1 Processor -- start 2024-05-09T00:43:31.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:31.858+0000 7f51878b2700 1 -- start start 2024-05-09T00:43:31.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:31.858+0000 7f51878b2700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51801034c0 0x7f51800ff2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:43:31.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:31.858+0000 7f51878b2700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5180107df0 0x7f51800ff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:43:31.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:31.858+0000 7f51878b2700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5180108740 0x7f51800ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:43:31.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:31.858+0000 7f51878b2700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5180113350 con 0x7f5180108740 2024-05-09T00:43:31.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:31.858+0000 7f51878b2700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f51801131d0 con 0x7f51801034c0 2024-05-09T00:43:31.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:31.858+0000 7f51878b2700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f51801134d0 con 0x7f5180107df0 2024-05-09T00:43:31.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:31.858+0000 7f5185e4f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5180108740 0x7f51800ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:43:31.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:31.858+0000 7f518564e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51801034c0 0x7f51800ff2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:43:31.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:31.858+0000 7f5184e4d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5180107df0 0x7f51800ff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:43:31.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:31.858+0000 7f5184e4d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5180107df0 0x7f51800ff7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:40506/0 (socket says 172.21.15.5:40506) 2024-05-09T00:43:31.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:31.858+0000 7f5184e4d700 1 -- 172.21.15.5:0/659350384 learned_addr learned my addr 172.21.15.5:0/659350384 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:43:31.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:31.858+0000 7f518564e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51801034c0 0x7f51800ff2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:52648/0 (socket says 172.21.15.5:52648) 2024-05-09T00:43:31.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:31.858+0000 7f5185e4f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5180108740 0x7f51800ffd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42112/0 (socket says 172.21.15.5:42112) 2024-05-09T00:43:31.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:31.858+0000 7f5184e4d700 1 -- 172.21.15.5:0/659350384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51801034c0 msgr2=0x7f51800ff2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:31.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:31.858+0000 7f5184e4d700 1 --2- 172.21.15.5:0/659350384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51801034c0 0x7f51800ff2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:31.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:31.858+0000 7f5184e4d700 1 -- 172.21.15.5:0/659350384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5180108740 msgr2=0x7f51800ffd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:31.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:31.858+0000 7f5184e4d700 1 --2- 172.21.15.5:0/659350384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5180108740 0x7f51800ffd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:31.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:31.858+0000 7f5184e4d700 1 -- 172.21.15.5:0/659350384 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51801a91b0 con 0x7f5180107df0 2024-05-09T00:43:31.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:31.859+0000 7f5184e4d700 1 --2- 172.21.15.5:0/659350384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5180107df0 0x7f51800ff7e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f51740045e0 tx=0x7f5174004610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:43:31.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:31.859+0000 7f517e7fc700 1 -- 172.21.15.5:0/659350384 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5174026070 con 0x7f5180107df0 2024-05-09T00:43:31.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:31.859+0000 7f517e7fc700 1 -- 172.21.15.5:0/659350384 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f517400f460 con 0x7f5180107df0 2024-05-09T00:43:31.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:31.859+0000 7f51878b2700 1 -- 172.21.15.5:0/659350384 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51801a9440 con 0x7f5180107df0 2024-05-09T00:43:31.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:31.859+0000 7f517e7fc700 1 -- 172.21.15.5:0/659350384 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5174009e60 con 0x7f5180107df0 2024-05-09T00:43:31.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:31.859+0000 7f51878b2700 1 -- 172.21.15.5:0/659350384 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51801a98f0 con 0x7f5180107df0 2024-05-09T00:43:31.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:31.860+0000 7f51878b2700 1 -- 172.21.15.5:0/659350384 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51800512c0 con 0x7f5180107df0 2024-05-09T00:43:31.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:31.862+0000 7f517e7fc700 1 -- 172.21.15.5:0/659350384 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f5174029070 con 0x7f5180107df0 2024-05-09T00:43:31.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:31.862+0000 7f517e7fc700 1 --2- 172.21.15.5:0/659350384 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f51680b2900 0x7f51680b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:43:31.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:31.862+0000 7f518564e700 1 --2- 172.21.15.5:0/659350384 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f51680b2900 0x7f51680b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:43:31.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:31.862+0000 7f517e7fc700 1 -- 172.21.15.5:0/659350384 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f51740e2810 con 0x7f5180107df0 2024-05-09T00:43:31.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:31.863+0000 7f518564e700 1 --2- 172.21.15.5:0/659350384 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f51680b2900 0x7f51680b4e10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5170009550 tx=0x7f51700091f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:43:31.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:31.866+0000 7f517e7fc700 1 -- 172.21.15.5:0/659350384 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5174013070 con 0x7f5180107df0 2024-05-09T00:43:32.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:32.139+0000 7f51878b2700 1 -- 172.21.15.5:0/659350384 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f518002e5f0 con 0x7f5180107df0 2024-05-09T00:43:32.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:32.141+0000 7f517e7fc700 1 -- 172.21.15.5:0/659350384 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10476 (secure 0 0 0) 0x7f51740aaa40 con 0x7f5180107df0 2024-05-09T00:43:32.143 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:43:32.143 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2685771887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2685771887},{"type":"v1","addr":"172.21.15.5:6835","nonce":2685771887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/254017535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":254017535},{"type":"v1","addr":"172.21.15.5:6837","nonce":254017535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2109255228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2109255228},{"type":"v1","addr":"172.21.15.5:6839","nonce":2109255228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3809966329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3809966329},{"type":"v1","addr":"172.21.15.5:6841","nonce":3809966329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/3272897742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3272897742},{"type":"v1","addr":"172.21.15.131:6833","nonce":3272897742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/1283093898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1283093898},{"type":"v1","addr":"172.21.15.131:6835","nonce":1283093898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2809525455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2809525455},{"type":"v1","addr":"172.21.15.157:6833","nonce":2809525455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24467,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/1633006186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":1633006186},{"type":"v1","addr":"172.21.15.131:6837","nonce":1633006186}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/832898273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":832898273},{"type":"v1","addr":"172.21.15.157:6835","nonce":832898273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24494,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6839/2748669631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2748669631},{"type":"v1","addr":"172.21.15.131:6839","nonce":2748669631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/744863009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":744863009},{"type":"v1","addr":"172.21.15.157:6837","nonce":744863009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24508,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6839/2779860399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":2779860399},{"type":"v1","addr":"172.21.15.157:6839","nonce":2779860399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-09T00:43:32.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:32.143+0000 7f51878b2700 1 -- 172.21.15.5:0/659350384 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f51680b2900 msgr2=0x7f51680b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:32.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:32.143+0000 7f51878b2700 1 --2- 172.21.15.5:0/659350384 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f51680b2900 0x7f51680b4e10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5170009550 tx=0x7f51700091f0 comp rx=0 tx=0).stop 2024-05-09T00:43:32.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:32.143+0000 7f51878b2700 1 -- 172.21.15.5:0/659350384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5180107df0 msgr2=0x7f51800ff7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:32.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:32.143+0000 7f51878b2700 1 --2- 172.21.15.5:0/659350384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5180107df0 0x7f51800ff7e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f51740045e0 tx=0x7f5174004610 comp rx=0 tx=0).stop 2024-05-09T00:43:32.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:32.143+0000 7f51878b2700 1 -- 172.21.15.5:0/659350384 shutdown_connections 2024-05-09T00:43:32.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:32.143+0000 7f51878b2700 1 --2- 172.21.15.5:0/659350384 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f51680b2900 0x7f51680b4e10 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:32.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:32.143+0000 7f51878b2700 1 --2- 172.21.15.5:0/659350384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5180108740 0x7f51800ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:32.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:32.143+0000 7f51878b2700 1 --2- 172.21.15.5:0/659350384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51801034c0 0x7f51800ff2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:32.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:32.143+0000 7f51878b2700 1 --2- 172.21.15.5:0/659350384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5180107df0 0x7f51800ff7e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:32.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:32.143+0000 7f51878b2700 1 -- 172.21.15.5:0/659350384 >> 172.21.15.5:0/659350384 conn(0x7f51800fedc0 msgr2=0x7f518006b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:43:32.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:32.143+0000 7f51878b2700 1 -- 172.21.15.5:0/659350384 shutdown_connections 2024-05-09T00:43:32.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:32.143+0000 7f51878b2700 1 -- 172.21.15.5:0/659350384 wait complete. 2024-05-09T00:43:32.146 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 13 2024-05-09T00:43:32.428 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:32 smithi005 ceph-mon[30365]: mds.? [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] up:boot 2024-05-09T00:43:32.429 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:32 smithi005 ceph-mon[30365]: fsmap 12 up:standby 2024-05-09T00:43:32.429 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:32 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T00:43:32.429 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:32 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:32.429 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:32 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:32.429 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:32 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:32.429 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:32 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:32.429 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:32 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:32.429 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:32 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:43:32.429 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:32 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:32.429 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:32 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/659350384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:43:32.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:32 smithi157 ceph-mon[34294]: mds.? [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] up:boot 2024-05-09T00:43:32.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:32 smithi157 ceph-mon[34294]: fsmap 12 up:standby 2024-05-09T00:43:32.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:32 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T00:43:32.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:32 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:32.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:32 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:32.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:32 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:32.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:32 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:32.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:32 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:32.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:32 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:43:32.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:32 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:32.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:32 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/659350384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:43:32.547 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:32 smithi131 ceph-mon[34547]: mds.? [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] up:boot 2024-05-09T00:43:32.547 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:32 smithi131 ceph-mon[34547]: fsmap 12 up:standby 2024-05-09T00:43:32.547 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:32 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T00:43:32.548 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:32 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:32.548 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:32 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:32.548 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:32 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:32.548 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:32 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:32.548 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:32 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:32.548 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:32 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:43:32.548 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:32 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:32.548 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:32 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/659350384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:43:33.417 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:33 smithi005 ceph-mon[30365]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:33.417 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:33 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:33.417 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:33 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:33.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:33 smithi157 ceph-mon[34294]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:33.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:33 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:33.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:33 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:33.444 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-05-09T00:43:33.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:33 smithi131 ceph-mon[34547]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:33.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:33 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:33.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:33 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:33.731 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:43:34.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:34 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:34.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:34 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:34.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:34 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:34.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:34 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:34.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:34 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:34.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:34 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:35.522 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:35 smithi005 ceph-mon[30365]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:35.522 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:35 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:35.522 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:35 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:35.587 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:35 smithi157 ceph-mon[34294]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:35.587 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:35 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:35.587 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:35 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:35.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:35 smithi131 ceph-mon[34547]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:35.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:35 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:35.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:35 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:35.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:35.637+0000 7f3c5e67a700 1 -- 172.21.15.5:0/3646122122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c5810e910 msgr2=0x7f3c58110df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:35.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:35.637+0000 7f3c5e67a700 1 --2- 172.21.15.5:0/3646122122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c5810e910 0x7f3c58110df0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3c5000b0a0 tx=0x7f3c5000b3b0 comp rx=0 tx=0).stop 2024-05-09T00:43:35.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:35.637+0000 7f3c5e67a700 1 -- 172.21.15.5:0/3646122122 shutdown_connections 2024-05-09T00:43:35.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:35.637+0000 7f3c5e67a700 1 --2- 172.21.15.5:0/3646122122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c5810e910 0x7f3c58110df0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:35.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:35.637+0000 7f3c5e67a700 1 --2- 172.21.15.5:0/3646122122 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c581054e0 0x7f3c581058f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:35.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:35.637+0000 7f3c5e67a700 1 --2- 172.21.15.5:0/3646122122 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c58105e30 0x7f3c5810e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:35.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:35.637+0000 7f3c5e67a700 1 -- 172.21.15.5:0/3646122122 >> 172.21.15.5:0/3646122122 conn(0x7f3c580fef30 msgr2=0x7f3c58101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:43:35.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:35.637+0000 7f3c5e67a700 1 -- 172.21.15.5:0/3646122122 shutdown_connections 2024-05-09T00:43:35.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:35.637+0000 7f3c5e67a700 1 -- 172.21.15.5:0/3646122122 wait complete. 2024-05-09T00:43:35.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:35.638+0000 7f3c5e67a700 1 Processor -- start 2024-05-09T00:43:35.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:35.638+0000 7f3c5e67a700 1 -- start start 2024-05-09T00:43:35.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:35.638+0000 7f3c5e67a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c581054e0 0x7f3c5819aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:43:35.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:35.639+0000 7f3c5e67a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c58105e30 0x7f3c5819b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:43:35.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:35.639+0000 7f3c5e67a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c5810e910 0x7f3c5819f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:43:35.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:35.639+0000 7f3c5e67a700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3c58114560 con 0x7f3c581054e0 2024-05-09T00:43:35.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:35.639+0000 7f3c5e67a700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3c581143e0 con 0x7f3c58105e30 2024-05-09T00:43:35.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:35.639+0000 7f3c5e67a700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3c581146e0 con 0x7f3c5810e910 2024-05-09T00:43:35.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:35.639+0000 7f3c57fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c581054e0 0x7f3c5819aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:43:35.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:35.639+0000 7f3c57fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c581054e0 0x7f3c5819aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54230/0 (socket says 172.21.15.5:54230) 2024-05-09T00:43:35.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:35.639+0000 7f3c57fff700 1 -- 172.21.15.5:0/3102232777 learned_addr learned my addr 172.21.15.5:0/3102232777 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:43:35.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:35.639+0000 7f3c577fe700 1 --2- 172.21.15.5:0/3102232777 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c58105e30 0x7f3c5819b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:43:35.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:35.639+0000 7f3c5cc17700 1 --2- 172.21.15.5:0/3102232777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c5810e910 0x7f3c5819f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:43:35.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:35.639+0000 7f3c57fff700 1 -- 172.21.15.5:0/3102232777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c5810e910 msgr2=0x7f3c5819f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:35.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:35.639+0000 7f3c57fff700 1 --2- 172.21.15.5:0/3102232777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c5810e910 0x7f3c5819f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:35.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:35.639+0000 7f3c57fff700 1 -- 172.21.15.5:0/3102232777 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c58105e30 msgr2=0x7f3c5819b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:35.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:35.639+0000 7f3c57fff700 1 --2- 172.21.15.5:0/3102232777 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c58105e30 0x7f3c5819b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:35.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:35.639+0000 7f3c57fff700 1 -- 172.21.15.5:0/3102232777 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c5819fda0 con 0x7f3c581054e0 2024-05-09T00:43:35.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:35.639+0000 7f3c57fff700 1 --2- 172.21.15.5:0/3102232777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c581054e0 0x7f3c5819aed0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3c4800b530 tx=0x7f3c4800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:43:35.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:35.640+0000 7f3c557fa700 1 -- 172.21.15.5:0/3102232777 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c48015020 con 0x7f3c581054e0 2024-05-09T00:43:35.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:35.640+0000 7f3c5e67a700 1 -- 172.21.15.5:0/3102232777 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c581a0090 con 0x7f3c581054e0 2024-05-09T00:43:35.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:35.640+0000 7f3c557fa700 1 -- 172.21.15.5:0/3102232777 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c48011ce0 con 0x7f3c581054e0 2024-05-09T00:43:35.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:35.640+0000 7f3c557fa700 1 -- 172.21.15.5:0/3102232777 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c48010dd0 con 0x7f3c581054e0 2024-05-09T00:43:35.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:35.640+0000 7f3c5e67a700 1 -- 172.21.15.5:0/3102232777 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c581a79b0 con 0x7f3c581054e0 2024-05-09T00:43:35.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:35.641+0000 7f3c5e67a700 1 -- 172.21.15.5:0/3102232777 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c580512c0 con 0x7f3c581054e0 2024-05-09T00:43:35.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:35.642+0000 7f3c557fa700 1 -- 172.21.15.5:0/3102232777 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f3c48018020 con 0x7f3c581054e0 2024-05-09T00:43:35.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:35.643+0000 7f3c557fa700 1 --2- 172.21.15.5:0/3102232777 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f3c400b2880 0x7f3c400b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:43:35.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:35.643+0000 7f3c577fe700 1 --2- 172.21.15.5:0/3102232777 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f3c400b2880 0x7f3c400b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:43:35.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:35.643+0000 7f3c557fa700 1 -- 172.21.15.5:0/3102232777 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f3c480e1480 con 0x7f3c581054e0 2024-05-09T00:43:35.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:35.644+0000 7f3c577fe700 1 --2- 172.21.15.5:0/3102232777 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f3c400b2880 0x7f3c400b4d90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3c58102370 tx=0x7f3c4c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:43:35.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:35.648+0000 7f3c557fa700 1 -- 172.21.15.5:0/3102232777 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3c4801f020 con 0x7f3c581054e0 2024-05-09T00:43:35.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:35.870+0000 7f3c5e67a700 1 -- 172.21.15.5:0/3102232777 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f3c5802e5f0 con 0x7f3c581054e0 2024-05-09T00:43:35.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:35.870+0000 7f3c557fa700 1 -- 172.21.15.5:0/3102232777 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v71) ==== 84+0+65 (secure 0 0 0) 0x7f3c480a9640 con 0x7f3c581054e0 2024-05-09T00:43:35.873 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:43:35.873 INFO:teuthology.orchestra.run.smithi005.stdout:[ 2024-05-09T00:43:35.873 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T00:43:35.873 INFO:teuthology.orchestra.run.smithi005.stdout: "poolnum": 1, 2024-05-09T00:43:35.873 INFO:teuthology.orchestra.run.smithi005.stdout: "poolname": ".mgr" 2024-05-09T00:43:35.873 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T00:43:35.873 INFO:teuthology.orchestra.run.smithi005.stdout:] 2024-05-09T00:43:35.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:35.873+0000 7f3c5e67a700 1 -- 172.21.15.5:0/3102232777 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f3c400b2880 msgr2=0x7f3c400b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:35.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:35.873+0000 7f3c5e67a700 1 --2- 172.21.15.5:0/3102232777 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f3c400b2880 0x7f3c400b4d90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3c58102370 tx=0x7f3c4c009190 comp rx=0 tx=0).stop 2024-05-09T00:43:35.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:35.873+0000 7f3c5e67a700 1 -- 172.21.15.5:0/3102232777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c581054e0 msgr2=0x7f3c5819aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:35.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:35.873+0000 7f3c5e67a700 1 --2- 172.21.15.5:0/3102232777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c581054e0 0x7f3c5819aed0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3c4800b530 tx=0x7f3c4800b840 comp rx=0 tx=0).stop 2024-05-09T00:43:35.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:35.873+0000 7f3c5e67a700 1 -- 172.21.15.5:0/3102232777 shutdown_connections 2024-05-09T00:43:35.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:35.873+0000 7f3c5e67a700 1 --2- 172.21.15.5:0/3102232777 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f3c400b2880 0x7f3c400b4d90 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:35.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:35.873+0000 7f3c5e67a700 1 --2- 172.21.15.5:0/3102232777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c5810e910 0x7f3c5819f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:35.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:35.873+0000 7f3c5e67a700 1 --2- 172.21.15.5:0/3102232777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c581054e0 0x7f3c5819aed0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:35.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:35.873+0000 7f3c5e67a700 1 --2- 172.21.15.5:0/3102232777 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c58105e30 0x7f3c5819b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:35.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:35.873+0000 7f3c5e67a700 1 -- 172.21.15.5:0/3102232777 >> 172.21.15.5:0/3102232777 conn(0x7f3c580fef30 msgr2=0x7f3c580ff5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:43:35.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:35.873+0000 7f3c5e67a700 1 -- 172.21.15.5:0/3102232777 shutdown_connections 2024-05-09T00:43:35.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:35.873+0000 7f3c5e67a700 1 -- 172.21.15.5:0/3102232777 wait complete. 2024-05-09T00:43:36.483 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:36 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:43:36.483 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:36 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:43:36.483 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:36 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:36.483 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:36 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/3102232777' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-09T00:43:36.483 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:36 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:43:36.546 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-09T00:43:36.546 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-09T00:43:36.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:36 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:43:36.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:36 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:43:36.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:36 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:36.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:36 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/3102232777' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-09T00:43:36.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:36 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:43:36.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:36 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:43:36.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:36 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:43:36.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:36 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:36.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:36 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/3102232777' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-09T00:43:36.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:36 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:43:37.495 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:43:37.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:37 smithi005 ceph-mon[30365]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:37.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:37 smithi131 ceph-mon[34547]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:37.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:37 smithi157 ceph-mon[34294]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:38.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:38 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:38.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:38 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:38.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:38 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:38.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:38 smithi131 ceph-mon[34547]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:38.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:38 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:38.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:38 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:38.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:38 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:38.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:38 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:38.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:38 smithi157 ceph-mon[34294]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:38.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:38 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:38.942 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:38 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:38.942 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:38 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:38.942 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:38 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:38.942 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:38 smithi005 ceph-mon[30365]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:38.942 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:38 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:39.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:39.163+0000 7ff50890f700 1 -- 172.21.15.5:0/1032799508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff504107df0 msgr2=0x7ff504108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:39.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:39.163+0000 7ff50890f700 1 --2- 172.21.15.5:0/1032799508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff504107df0 0x7ff504108200 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff4f80099c0 tx=0x7ff4f8009cd0 comp rx=0 tx=0).stop 2024-05-09T00:43:39.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:39.163+0000 7ff50890f700 1 -- 172.21.15.5:0/1032799508 shutdown_connections 2024-05-09T00:43:39.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:39.163+0000 7ff50890f700 1 --2- 172.21.15.5:0/1032799508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff504108740 0x7ff5041100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:39.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:39.163+0000 7ff50890f700 1 --2- 172.21.15.5:0/1032799508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5041034c0 0x7ff5041038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:39.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:39.163+0000 7ff50890f700 1 --2- 172.21.15.5:0/1032799508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff504107df0 0x7ff504108200 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:39.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:39.163+0000 7ff50890f700 1 -- 172.21.15.5:0/1032799508 >> 172.21.15.5:0/1032799508 conn(0x7ff5040fedc0 msgr2=0x7ff5041011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:43:39.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:39.163+0000 7ff50890f700 1 -- 172.21.15.5:0/1032799508 shutdown_connections 2024-05-09T00:43:39.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:39.163+0000 7ff50890f700 1 -- 172.21.15.5:0/1032799508 wait complete. 2024-05-09T00:43:39.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:39.164+0000 7ff50890f700 1 Processor -- start 2024-05-09T00:43:39.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:39.164+0000 7ff50890f700 1 -- start start 2024-05-09T00:43:39.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:39.164+0000 7ff50890f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5041034c0 0x7ff50419ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:43:39.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:39.164+0000 7ff50890f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff504107df0 0x7ff50419b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:43:39.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:39.164+0000 7ff50890f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff504108740 0x7ff50419f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:43:39.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:39.164+0000 7ff50890f700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff5041133c0 con 0x7ff504108740 2024-05-09T00:43:39.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:39.164+0000 7ff50890f700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff504113240 con 0x7ff5041034c0 2024-05-09T00:43:39.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:39.164+0000 7ff50890f700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff504113540 con 0x7ff504107df0 2024-05-09T00:43:39.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:39.164+0000 7ff502d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff504108740 0x7ff50419f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:43:39.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:39.164+0000 7ff502d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff504108740 0x7ff50419f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54252/0 (socket says 172.21.15.5:54252) 2024-05-09T00:43:39.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:39.164+0000 7ff502d9d700 1 -- 172.21.15.5:0/3636877924 learned_addr learned my addr 172.21.15.5:0/3636877924 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:43:39.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:39.165+0000 7ff50259c700 1 --2- 172.21.15.5:0/3636877924 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5041034c0 0x7ff50419ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:43:39.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:39.165+0000 7ff501d9b700 1 --2- 172.21.15.5:0/3636877924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff504107df0 0x7ff50419b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:43:39.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:39.165+0000 7ff502d9d700 1 -- 172.21.15.5:0/3636877924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff504107df0 msgr2=0x7ff50419b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:39.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:39.165+0000 7ff502d9d700 1 --2- 172.21.15.5:0/3636877924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff504107df0 0x7ff50419b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:39.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:39.165+0000 7ff502d9d700 1 -- 172.21.15.5:0/3636877924 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5041034c0 msgr2=0x7ff50419ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:39.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:39.165+0000 7ff502d9d700 1 --2- 172.21.15.5:0/3636877924 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5041034c0 0x7ff50419ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:39.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:39.165+0000 7ff502d9d700 1 -- 172.21.15.5:0/3636877924 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff50419fd10 con 0x7ff504108740 2024-05-09T00:43:39.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:39.165+0000 7ff502d9d700 1 --2- 172.21.15.5:0/3636877924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff504108740 0x7ff50419f670 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc007ee0 tx=0x7ff4fc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:43:39.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:39.165+0000 7ff4f37fe700 1 -- 172.21.15.5:0/3636877924 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc004bb0 con 0x7ff504108740 2024-05-09T00:43:39.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:39.165+0000 7ff50890f700 1 -- 172.21.15.5:0/3636877924 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5041a0000 con 0x7ff504108740 2024-05-09T00:43:39.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:39.165+0000 7ff4f37fe700 1 -- 172.21.15.5:0/3636877924 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4fc004d50 con 0x7ff504108740 2024-05-09T00:43:39.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:39.165+0000 7ff4f37fe700 1 -- 172.21.15.5:0/3636877924 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc005920 con 0x7ff504108740 2024-05-09T00:43:39.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:39.165+0000 7ff50890f700 1 -- 172.21.15.5:0/3636877924 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5041a7920 con 0x7ff504108740 2024-05-09T00:43:39.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:39.166+0000 7ff50890f700 1 -- 172.21.15.5:0/3636877924 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5040512c0 con 0x7ff504108740 2024-05-09T00:43:39.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:39.168+0000 7ff4f37fe700 1 -- 172.21.15.5:0/3636877924 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7ff4fc01a020 con 0x7ff504108740 2024-05-09T00:43:39.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:39.169+0000 7ff4f37fe700 1 --2- 172.21.15.5:0/3636877924 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ff4ec0b2880 0x7ff4ec0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:43:39.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:39.169+0000 7ff50259c700 1 --2- 172.21.15.5:0/3636877924 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ff4ec0b2880 0x7ff4ec0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:43:39.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:39.169+0000 7ff4f37fe700 1 -- 172.21.15.5:0/3636877924 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7ff4fc0e1b00 con 0x7ff504108740 2024-05-09T00:43:39.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:39.169+0000 7ff50259c700 1 --2- 172.21.15.5:0/3636877924 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ff4ec0b2880 0x7ff4ec0b4d90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4009a10 tx=0x7ff4f4009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:43:39.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:39.172+0000 7ff4f37fe700 1 -- 172.21.15.5:0/3636877924 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff4fc010070 con 0x7ff504108740 2024-05-09T00:43:39.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:39.445+0000 7ff50890f700 1 -- 172.21.15.5:0/3636877924 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7ff5041a7af0 con 0x7ff504108740 2024-05-09T00:43:39.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:39.836+0000 7ff4f37fe700 1 -- 172.21.15.5:0/3636877924 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v72) ==== 138+0+0 (secure 0 0 0) 0x7ff4fc0a9c20 con 0x7ff504108740 2024-05-09T00:43:39.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:39.838+0000 7ff50890f700 1 -- 172.21.15.5:0/3636877924 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ff4ec0b2880 msgr2=0x7ff4ec0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:39.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:39.838+0000 7ff50890f700 1 --2- 172.21.15.5:0/3636877924 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ff4ec0b2880 0x7ff4ec0b4d90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4009a10 tx=0x7ff4f4009190 comp rx=0 tx=0).stop 2024-05-09T00:43:39.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:39.838+0000 7ff50890f700 1 -- 172.21.15.5:0/3636877924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff504108740 msgr2=0x7ff50419f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:39.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:39.838+0000 7ff50890f700 1 --2- 172.21.15.5:0/3636877924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff504108740 0x7ff50419f670 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc007ee0 tx=0x7ff4fc00d3b0 comp rx=0 tx=0).stop 2024-05-09T00:43:39.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:39.838+0000 7ff50890f700 1 -- 172.21.15.5:0/3636877924 shutdown_connections 2024-05-09T00:43:39.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:39.838+0000 7ff50890f700 1 --2- 172.21.15.5:0/3636877924 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ff4ec0b2880 0x7ff4ec0b4d90 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:39.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:39.838+0000 7ff50890f700 1 --2- 172.21.15.5:0/3636877924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff504108740 0x7ff50419f670 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:39.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:39.838+0000 7ff50890f700 1 --2- 172.21.15.5:0/3636877924 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5041034c0 0x7ff50419ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:39.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:39.838+0000 7ff50890f700 1 --2- 172.21.15.5:0/3636877924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff504107df0 0x7ff50419b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:39.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:39.838+0000 7ff50890f700 1 -- 172.21.15.5:0/3636877924 >> 172.21.15.5:0/3636877924 conn(0x7ff5040fedc0 msgr2=0x7ff504107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:43:39.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:39.839+0000 7ff50890f700 1 -- 172.21.15.5:0/3636877924 shutdown_connections 2024-05-09T00:43:39.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:39.839+0000 7ff50890f700 1 -- 172.21.15.5:0/3636877924 wait complete. 2024-05-09T00:43:39.844 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' created 2024-05-09T00:43:40.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:39 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/3636877924' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-09T00:43:40.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:39 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:40.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:39 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:40.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:39 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:43:40.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:39 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:43:40.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:39 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:40.151 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:39 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/3636877924' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-09T00:43:40.151 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:39 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:40.151 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:39 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:40.151 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:39 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:43:40.151 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:39 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:43:40.152 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:39 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:40.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:39 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/3636877924' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-09T00:43:40.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:39 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:40.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:39 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:40.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:39 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:43:40.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:39 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:43:40.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:39 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:43:40.469 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-09T00:43:40.730 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:43:41.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:40 smithi131 ceph-mon[34547]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:41.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:40 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/3636877924' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-09T00:43:41.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:40 smithi131 ceph-mon[34547]: osdmap e72: 12 total, 12 up, 12 in 2024-05-09T00:43:41.113 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:40 smithi005 ceph-mon[30365]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:41.113 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:40 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/3636877924' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-09T00:43:41.113 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:40 smithi005 ceph-mon[30365]: osdmap e72: 12 total, 12 up, 12 in 2024-05-09T00:43:41.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:40 smithi157 ceph-mon[34294]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:41.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:40 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/3636877924' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-09T00:43:41.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:40 smithi157 ceph-mon[34294]: osdmap e72: 12 total, 12 up, 12 in 2024-05-09T00:43:42.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:41 smithi131 ceph-mon[34547]: osdmap e73: 12 total, 12 up, 12 in 2024-05-09T00:43:42.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:41 smithi157 ceph-mon[34294]: osdmap e73: 12 total, 12 up, 12 in 2024-05-09T00:43:42.204 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:41 smithi005 ceph-mon[30365]: osdmap e73: 12 total, 12 up, 12 in 2024-05-09T00:43:42.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:42.479+0000 7f00b9c58700 1 -- 172.21.15.5:0/2136591579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00b4108f30 msgr2=0x7f00b4109340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:42.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:42.479+0000 7f00b9c58700 1 --2- 172.21.15.5:0/2136591579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00b4108f30 0x7f00b4109340 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f009c0099c0 tx=0x7f009c009cd0 comp rx=0 tx=0).stop 2024-05-09T00:43:42.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:42.479+0000 7f00b9c58700 1 -- 172.21.15.5:0/2136591579 shutdown_connections 2024-05-09T00:43:42.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:42.479+0000 7f00b9c58700 1 --2- 172.21.15.5:0/2136591579 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00b4109880 0x7f00b41111e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:42.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:42.479+0000 7f00b9c58700 1 --2- 172.21.15.5:0/2136591579 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00b4104650 0x7f00b4104a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:42.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:42.479+0000 7f00b9c58700 1 --2- 172.21.15.5:0/2136591579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00b4108f30 0x7f00b4109340 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:42.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:42.479+0000 7f00b9c58700 1 -- 172.21.15.5:0/2136591579 >> 172.21.15.5:0/2136591579 conn(0x7f00b4079f00 msgr2=0x7f00b407a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:43:42.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:42.479+0000 7f00b9c58700 1 -- 172.21.15.5:0/2136591579 shutdown_connections 2024-05-09T00:43:42.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:42.479+0000 7f00b9c58700 1 -- 172.21.15.5:0/2136591579 wait complete. 2024-05-09T00:43:42.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:42.480+0000 7f00b9c58700 1 Processor -- start 2024-05-09T00:43:42.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:42.480+0000 7f00b9c58700 1 -- start start 2024-05-09T00:43:42.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:42.481+0000 7f00b9c58700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00b4104650 0x7f00b419ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:43:42.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:42.481+0000 7f00b9c58700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00b4108f30 0x7f00b419b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:43:42.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:42.481+0000 7f00b9c58700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00b4109880 0x7f00b419f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:43:42.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:42.481+0000 7f00b9c58700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f00b41144e0 con 0x7f00b4104650 2024-05-09T00:43:42.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:42.481+0000 7f00b9c58700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f00b4114360 con 0x7f00b4109880 2024-05-09T00:43:42.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:42.481+0000 7f00b9c58700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f00b4114660 con 0x7f00b4108f30 2024-05-09T00:43:42.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:42.481+0000 7f00b37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00b4104650 0x7f00b419ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:43:42.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:42.481+0000 7f00b37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00b4104650 0x7f00b419ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54274/0 (socket says 172.21.15.5:54274) 2024-05-09T00:43:42.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:42.481+0000 7f00b37fe700 1 -- 172.21.15.5:0/247840995 learned_addr learned my addr 172.21.15.5:0/247840995 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:43:42.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:42.481+0000 7f00b3fff700 1 --2- 172.21.15.5:0/247840995 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00b4109880 0x7f00b419f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:43:42.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:42.481+0000 7f00b2ffd700 1 --2- 172.21.15.5:0/247840995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00b4108f30 0x7f00b419b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:43:42.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:42.481+0000 7f00b37fe700 1 -- 172.21.15.5:0/247840995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00b4108f30 msgr2=0x7f00b419b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:42.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:42.481+0000 7f00b37fe700 1 --2- 172.21.15.5:0/247840995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00b4108f30 0x7f00b419b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:42.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:42.481+0000 7f00b37fe700 1 -- 172.21.15.5:0/247840995 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00b4109880 msgr2=0x7f00b419f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:42.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:42.481+0000 7f00b37fe700 1 --2- 172.21.15.5:0/247840995 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00b4109880 0x7f00b419f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:42.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:42.481+0000 7f00b37fe700 1 -- 172.21.15.5:0/247840995 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00b419fc90 con 0x7f00b4104650 2024-05-09T00:43:42.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:42.481+0000 7f00b3fff700 1 --2- 172.21.15.5:0/247840995 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00b4109880 0x7f00b419f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:43:42.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:42.482+0000 7f00b37fe700 1 --2- 172.21.15.5:0/247840995 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00b4104650 0x7f00b419ae70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f00a4007e20 tx=0x7f00a400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:43:42.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:42.482+0000 7f00b0ff9700 1 -- 172.21.15.5:0/247840995 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a4016020 con 0x7f00b4104650 2024-05-09T00:43:42.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:42.482+0000 7f00b0ff9700 1 -- 172.21.15.5:0/247840995 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00a400ac90 con 0x7f00b4104650 2024-05-09T00:43:42.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:42.482+0000 7f00b9c58700 1 -- 172.21.15.5:0/247840995 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00b419ff80 con 0x7f00b4104650 2024-05-09T00:43:42.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:42.482+0000 7f00b0ff9700 1 -- 172.21.15.5:0/247840995 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a40115e0 con 0x7f00b4104650 2024-05-09T00:43:42.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:42.482+0000 7f00b9c58700 1 -- 172.21.15.5:0/247840995 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00b41a0490 con 0x7f00b4104650 2024-05-09T00:43:42.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:42.483+0000 7f00b9c58700 1 -- 172.21.15.5:0/247840995 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00b4105ab0 con 0x7f00b4104650 2024-05-09T00:43:42.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:42.485+0000 7f00b0ff9700 1 -- 172.21.15.5:0/247840995 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f00a4019020 con 0x7f00b4104650 2024-05-09T00:43:42.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:42.485+0000 7f00b0ff9700 1 --2- 172.21.15.5:0/247840995 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f00a00b2880 0x7f00a00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:43:42.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:42.485+0000 7f00b2ffd700 1 --2- 172.21.15.5:0/247840995 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f00a00b2880 0x7f00a00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:43:42.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:42.486+0000 7f00b0ff9700 1 -- 172.21.15.5:0/247840995 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7497+0+0 (secure 0 0 0) 0x7f00a40e1fc0 con 0x7f00b4104650 2024-05-09T00:43:42.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:42.486+0000 7f00b2ffd700 1 --2- 172.21.15.5:0/247840995 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f00a00b2880 0x7f00a00b4d90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f009c009990 tx=0x7f009c009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:43:42.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:42.489+0000 7f00b0ff9700 1 -- 172.21.15.5:0/247840995 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f00a4020070 con 0x7f00b4104650 2024-05-09T00:43:42.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:42.703+0000 7f00b9c58700 1 -- 172.21.15.5:0/247840995 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f00b419c1d0 con 0x7f00b4104650 2024-05-09T00:43:42.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:42.850+0000 7f00b0ff9700 1 -- 172.21.15.5:0/247840995 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v75) ==== 170+0+0 (secure 0 0 0) 0x7f00a40aa010 con 0x7f00b4104650 2024-05-09T00:43:42.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:42.853+0000 7f00b9c58700 1 -- 172.21.15.5:0/247840995 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f00a00b2880 msgr2=0x7f00a00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:42.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:42.853+0000 7f00b9c58700 1 --2- 172.21.15.5:0/247840995 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f00a00b2880 0x7f00a00b4d90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f009c009990 tx=0x7f009c009f10 comp rx=0 tx=0).stop 2024-05-09T00:43:42.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:42.853+0000 7f00b9c58700 1 -- 172.21.15.5:0/247840995 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00b4104650 msgr2=0x7f00b419ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:42.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:42.853+0000 7f00b9c58700 1 --2- 172.21.15.5:0/247840995 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00b4104650 0x7f00b419ae70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f00a4007e20 tx=0x7f00a400a340 comp rx=0 tx=0).stop 2024-05-09T00:43:42.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:42.853+0000 7f00b9c58700 1 -- 172.21.15.5:0/247840995 shutdown_connections 2024-05-09T00:43:42.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:42.853+0000 7f00b9c58700 1 --2- 172.21.15.5:0/247840995 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f00a00b2880 0x7f00a00b4d90 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:42.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:42.853+0000 7f00b9c58700 1 --2- 172.21.15.5:0/247840995 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00b4109880 0x7f00b419f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:42.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:42.853+0000 7f00b9c58700 1 --2- 172.21.15.5:0/247840995 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00b4104650 0x7f00b419ae70 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:42.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:42.853+0000 7f00b9c58700 1 --2- 172.21.15.5:0/247840995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00b4108f30 0x7f00b419b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:42.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:42.853+0000 7f00b9c58700 1 -- 172.21.15.5:0/247840995 >> 172.21.15.5:0/247840995 conn(0x7f00b4079f00 msgr2=0x7f00b4102050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:43:42.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:42.855+0000 7f00b9c58700 1 -- 172.21.15.5:0/247840995 shutdown_connections 2024-05-09T00:43:42.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:42.855+0000 7f00b9c58700 1 -- 172.21.15.5:0/247840995 wait complete. 2024-05-09T00:43:42.861 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' created 2024-05-09T00:43:43.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:42 smithi157 ceph-mon[34294]: pgmap v283: 33 pgs: 3 creating+peering, 29 unknown, 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:43.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:42 smithi157 ceph-mon[34294]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-09T00:43:43.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:42 smithi157 ceph-mon[34294]: osdmap e74: 12 total, 12 up, 12 in 2024-05-09T00:43:43.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:42 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/247840995' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-09T00:43:43.236 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:42 smithi005 ceph-mon[30365]: pgmap v283: 33 pgs: 3 creating+peering, 29 unknown, 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:43.236 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:42 smithi005 ceph-mon[30365]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-09T00:43:43.236 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:42 smithi005 ceph-mon[30365]: osdmap e74: 12 total, 12 up, 12 in 2024-05-09T00:43:43.236 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:42 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/247840995' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-09T00:43:43.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:42 smithi131 ceph-mon[34547]: pgmap v283: 33 pgs: 3 creating+peering, 29 unknown, 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:43.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:42 smithi131 ceph-mon[34547]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-09T00:43:43.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:42 smithi131 ceph-mon[34547]: osdmap e74: 12 total, 12 up, 12 in 2024-05-09T00:43:43.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:42 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/247840995' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-09T00:43:43.606 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-09T00:43:43.878 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:43 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/247840995' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-09T00:43:43.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:43 smithi005 ceph-mon[30365]: osdmap e75: 12 total, 12 up, 12 in 2024-05-09T00:43:43.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:43 smithi005 ceph-mon[30365]: osdmap e76: 12 total, 12 up, 12 in 2024-05-09T00:43:43.888 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:43:44.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:43 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/247840995' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-09T00:43:44.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:43 smithi131 ceph-mon[34547]: osdmap e75: 12 total, 12 up, 12 in 2024-05-09T00:43:44.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:43 smithi131 ceph-mon[34547]: osdmap e76: 12 total, 12 up, 12 in 2024-05-09T00:43:44.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:43 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/247840995' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-09T00:43:44.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:43 smithi157 ceph-mon[34294]: osdmap e75: 12 total, 12 up, 12 in 2024-05-09T00:43:44.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:43 smithi157 ceph-mon[34294]: osdmap e76: 12 total, 12 up, 12 in 2024-05-09T00:43:45.037 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:44 smithi005 ceph-mon[30365]: pgmap v286: 97 pgs: 20 creating+peering, 57 unknown, 20 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:45.037 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:44 smithi005 ceph-mon[30365]: osdmap e77: 12 total, 12 up, 12 in 2024-05-09T00:43:45.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:44 smithi157 ceph-mon[34294]: pgmap v286: 97 pgs: 20 creating+peering, 57 unknown, 20 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:45.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:44 smithi157 ceph-mon[34294]: osdmap e77: 12 total, 12 up, 12 in 2024-05-09T00:43:45.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:44 smithi131 ceph-mon[34547]: pgmap v286: 97 pgs: 20 creating+peering, 57 unknown, 20 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:45.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:44 smithi131 ceph-mon[34547]: osdmap e77: 12 total, 12 up, 12 in 2024-05-09T00:43:45.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:45.580+0000 7f637b37e700 1 -- 172.21.15.5:0/2013890493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63741045d0 msgr2=0x7f63741049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:45.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:45.580+0000 7f637b37e700 1 --2- 172.21.15.5:0/2013890493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63741045d0 0x7f63741049e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f63700099c0 tx=0x7f6370009cd0 comp rx=0 tx=0).stop 2024-05-09T00:43:45.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:45.580+0000 7f637b37e700 1 -- 172.21.15.5:0/2013890493 shutdown_connections 2024-05-09T00:43:45.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:45.580+0000 7f637b37e700 1 --2- 172.21.15.5:0/2013890493 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6374109850 0x7f63741111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:45.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:45.580+0000 7f637b37e700 1 --2- 172.21.15.5:0/2013890493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63741045d0 0x7f63741049e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:45.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:45.580+0000 7f637b37e700 1 --2- 172.21.15.5:0/2013890493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6374108f00 0x7f6374109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:45.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:45.580+0000 7f637b37e700 1 -- 172.21.15.5:0/2013890493 >> 172.21.15.5:0/2013890493 conn(0x7f63740ffea0 msgr2=0x7f63741022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:43:45.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:45.580+0000 7f637b37e700 1 -- 172.21.15.5:0/2013890493 shutdown_connections 2024-05-09T00:43:45.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:45.580+0000 7f637b37e700 1 -- 172.21.15.5:0/2013890493 wait complete. 2024-05-09T00:43:45.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:45.581+0000 7f637b37e700 1 Processor -- start 2024-05-09T00:43:45.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:45.581+0000 7f637b37e700 1 -- start start 2024-05-09T00:43:45.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:45.581+0000 7f637b37e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63741045d0 0x7f6374078770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:43:45.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:45.581+0000 7f637b37e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6374108f00 0x7f6374076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:43:45.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:45.581+0000 7f637b37e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6374109850 0x7f63740771b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:43:45.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:45.581+0000 7f637b37e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6374114480 con 0x7f6374109850 2024-05-09T00:43:45.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:45.581+0000 7f637b37e700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6374114300 con 0x7f63741045d0 2024-05-09T00:43:45.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:45.581+0000 7f637b37e700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6374114600 con 0x7f6374108f00 2024-05-09T00:43:45.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:45.581+0000 7f637991b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6374109850 0x7f63740771b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:43:45.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:45.581+0000 7f637991b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6374109850 0x7f63740771b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45218/0 (socket says 172.21.15.5:45218) 2024-05-09T00:43:45.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:45.581+0000 7f6378919700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6374108f00 0x7f6374076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:43:45.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:45.581+0000 7f637991b700 1 -- 172.21.15.5:0/1110508387 learned_addr learned my addr 172.21.15.5:0/1110508387 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:43:45.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:45.581+0000 7f637911a700 1 --2- 172.21.15.5:0/1110508387 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63741045d0 0x7f6374078770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:43:45.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:45.581+0000 7f637991b700 1 -- 172.21.15.5:0/1110508387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6374108f00 msgr2=0x7f6374076bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:45.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:45.581+0000 7f637991b700 1 --2- 172.21.15.5:0/1110508387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6374108f00 0x7f6374076bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:45.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:45.581+0000 7f637991b700 1 -- 172.21.15.5:0/1110508387 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63741045d0 msgr2=0x7f6374078770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:45.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:45.581+0000 7f637991b700 1 --2- 172.21.15.5:0/1110508387 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63741045d0 0x7f6374078770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:45.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:45.581+0000 7f637991b700 1 -- 172.21.15.5:0/1110508387 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63740779e0 con 0x7f6374109850 2024-05-09T00:43:45.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:45.582+0000 7f637991b700 1 --2- 172.21.15.5:0/1110508387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6374109850 0x7f63740771b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f636c007ee0 tx=0x7f636c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:43:45.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:45.582+0000 7f636a7fc700 1 -- 172.21.15.5:0/1110508387 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636c004bb0 con 0x7f6374109850 2024-05-09T00:43:45.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:45.582+0000 7f637b37e700 1 -- 172.21.15.5:0/1110508387 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63741ad8a0 con 0x7f6374109850 2024-05-09T00:43:45.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:45.582+0000 7f636a7fc700 1 -- 172.21.15.5:0/1110508387 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f636c004d50 con 0x7f6374109850 2024-05-09T00:43:45.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:45.582+0000 7f636a7fc700 1 -- 172.21.15.5:0/1110508387 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636c005920 con 0x7f6374109850 2024-05-09T00:43:45.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:45.582+0000 7f637b37e700 1 -- 172.21.15.5:0/1110508387 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63741addb0 con 0x7f6374109850 2024-05-09T00:43:45.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:45.583+0000 7f637b37e700 1 -- 172.21.15.5:0/1110508387 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6374105a80 con 0x7f6374109850 2024-05-09T00:43:45.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:45.586+0000 7f636a7fc700 1 -- 172.21.15.5:0/1110508387 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f636c01a020 con 0x7f6374109850 2024-05-09T00:43:45.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:45.587+0000 7f636a7fc700 1 --2- 172.21.15.5:0/1110508387 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f63600b2900 0x7f63600b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:43:45.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:45.587+0000 7f637911a700 1 --2- 172.21.15.5:0/1110508387 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f63600b2900 0x7f63600b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:43:45.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:45.587+0000 7f636a7fc700 1 -- 172.21.15.5:0/1110508387 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7879+0+0 (secure 0 0 0) 0x7f636c0e1c50 con 0x7f6374109850 2024-05-09T00:43:45.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:45.588+0000 7f637911a700 1 --2- 172.21.15.5:0/1110508387 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f63600b2900 0x7f63600b4e10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6370009990 tx=0x7f63700036e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:43:45.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:45.590+0000 7f636a7fc700 1 -- 172.21.15.5:0/1110508387 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f636c010070 con 0x7f6374109850 2024-05-09T00:43:45.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:45.854+0000 7f637b37e700 1 -- 172.21.15.5:0/1110508387 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f637402e5f0 con 0x7f6374109850 2024-05-09T00:43:45.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:45.885+0000 7f636a7fc700 1 -- 172.21.15.5:0/1110508387 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f636c0a9b20 con 0x7f6374109850 2024-05-09T00:43:45.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:45.888+0000 7f637b37e700 1 -- 172.21.15.5:0/1110508387 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f63600b2900 msgr2=0x7f63600b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:45.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:45.888+0000 7f637b37e700 1 --2- 172.21.15.5:0/1110508387 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f63600b2900 0x7f63600b4e10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6370009990 tx=0x7f63700036e0 comp rx=0 tx=0).stop 2024-05-09T00:43:45.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:45.888+0000 7f637b37e700 1 -- 172.21.15.5:0/1110508387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6374109850 msgr2=0x7f63740771b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:45.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:45.888+0000 7f637b37e700 1 --2- 172.21.15.5:0/1110508387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6374109850 0x7f63740771b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f636c007ee0 tx=0x7f636c00d3b0 comp rx=0 tx=0).stop 2024-05-09T00:43:45.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:45.888+0000 7f637b37e700 1 -- 172.21.15.5:0/1110508387 shutdown_connections 2024-05-09T00:43:45.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:45.888+0000 7f637b37e700 1 --2- 172.21.15.5:0/1110508387 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f63600b2900 0x7f63600b4e10 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:45.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:45.888+0000 7f637b37e700 1 --2- 172.21.15.5:0/1110508387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6374109850 0x7f63740771b0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:45.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:45.888+0000 7f637b37e700 1 --2- 172.21.15.5:0/1110508387 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63741045d0 0x7f6374078770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:45.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:45.888+0000 7f637b37e700 1 --2- 172.21.15.5:0/1110508387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6374108f00 0x7f6374076bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:45.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:45.888+0000 7f637b37e700 1 -- 172.21.15.5:0/1110508387 >> 172.21.15.5:0/1110508387 conn(0x7f63740ffea0 msgr2=0x7f63741022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:43:45.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:45.888+0000 7f637b37e700 1 -- 172.21.15.5:0/1110508387 shutdown_connections 2024-05-09T00:43:45.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:45.888+0000 7f637b37e700 1 -- 172.21.15.5:0/1110508387 wait complete. 2024-05-09T00:43:45.890 INFO:teuthology.orchestra.run.smithi005.stderr:new fs with metadata pool 2 and data pool 3 2024-05-09T00:43:46.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:45 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/1110508387' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-09T00:43:46.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:45 smithi157 ceph-mon[34294]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-05-09T00:43:46.262 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:45 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/1110508387' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-09T00:43:46.263 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:45 smithi005 ceph-mon[30365]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-05-09T00:43:46.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:45 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/1110508387' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-09T00:43:46.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:45 smithi131 ceph-mon[34547]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-05-09T00:43:46.669 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T00:43:46.927 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:43:47.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:46 smithi005 ceph-mon[30365]: pgmap v289: 97 pgs: 1 creating+activating, 17 creating+peering, 8 unknown, 71 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:47.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:46 smithi005 ceph-mon[30365]: osdmap e78: 12 total, 12 up, 12 in 2024-05-09T00:43:47.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:46 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/1110508387' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-09T00:43:47.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:46 smithi005 ceph-mon[30365]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-05-09T00:43:47.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:46 smithi005 ceph-mon[30365]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-05-09T00:43:47.187 INFO:journalctl@ceph.mds.i.smithi157.stdout:May 09 00:43:46 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-i[64868]: 2024-05-09T00:43:46.903+0000 7f3299926700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-09T00:43:47.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:46 smithi157 ceph-mon[34294]: pgmap v289: 97 pgs: 1 creating+activating, 17 creating+peering, 8 unknown, 71 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:47.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:46 smithi157 ceph-mon[34294]: osdmap e78: 12 total, 12 up, 12 in 2024-05-09T00:43:47.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:46 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/1110508387' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-09T00:43:47.188 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:46 smithi157 ceph-mon[34294]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-05-09T00:43:47.188 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:46 smithi157 ceph-mon[34294]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-05-09T00:43:47.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:46 smithi131 ceph-mon[34547]: pgmap v289: 97 pgs: 1 creating+activating, 17 creating+peering, 8 unknown, 71 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:43:47.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:46 smithi131 ceph-mon[34547]: osdmap e78: 12 total, 12 up, 12 in 2024-05-09T00:43:47.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:46 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/1110508387' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-09T00:43:47.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:46 smithi131 ceph-mon[34547]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-05-09T00:43:47.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:46 smithi131 ceph-mon[34547]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-05-09T00:43:48.165 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:47 smithi005 ceph-mon[30365]: mds.? [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] up:active 2024-05-09T00:43:48.166 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:47 smithi005 ceph-mon[30365]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-09T00:43:48.166 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:47 smithi005 ceph-mon[30365]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-09T00:43:48.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:47 smithi157 ceph-mon[34294]: mds.? [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] up:active 2024-05-09T00:43:48.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:47 smithi157 ceph-mon[34294]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-09T00:43:48.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:47 smithi157 ceph-mon[34294]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-09T00:43:48.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:47 smithi131 ceph-mon[34547]: mds.? [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] up:active 2024-05-09T00:43:48.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:47 smithi131 ceph-mon[34547]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-09T00:43:48.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:47 smithi131 ceph-mon[34547]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-09T00:43:48.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:48.753+0000 7f9d4c555700 1 -- 172.21.15.5:0/1645664171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d441034c0 msgr2=0x7f9d441038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:48.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:48.753+0000 7f9d4c555700 1 --2- 172.21.15.5:0/1645664171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d441034c0 0x7f9d441038d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9d400099c0 tx=0x7f9d40009cd0 comp rx=0 tx=0).stop 2024-05-09T00:43:48.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:48.753+0000 7f9d4c555700 1 -- 172.21.15.5:0/1645664171 shutdown_connections 2024-05-09T00:43:48.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:48.753+0000 7f9d4c555700 1 --2- 172.21.15.5:0/1645664171 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d44108740 0x7f9d441100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:48.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:48.753+0000 7f9d4c555700 1 --2- 172.21.15.5:0/1645664171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d441034c0 0x7f9d441038d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:48.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:48.753+0000 7f9d4c555700 1 --2- 172.21.15.5:0/1645664171 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d44107df0 0x7f9d44108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:48.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:48.753+0000 7f9d4c555700 1 -- 172.21.15.5:0/1645664171 >> 172.21.15.5:0/1645664171 conn(0x7f9d440fedf0 msgr2=0x7f9d44101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:43:48.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:48.753+0000 7f9d4c555700 1 -- 172.21.15.5:0/1645664171 shutdown_connections 2024-05-09T00:43:48.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:48.753+0000 7f9d4c555700 1 -- 172.21.15.5:0/1645664171 wait complete. 2024-05-09T00:43:48.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:48.754+0000 7f9d4c555700 1 Processor -- start 2024-05-09T00:43:48.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:48.754+0000 7f9d4c555700 1 -- start start 2024-05-09T00:43:48.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:48.755+0000 7f9d4c555700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d441034c0 0x7f9d4419ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:43:48.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:48.755+0000 7f9d4c555700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d44107df0 0x7f9d4419b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:43:48.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:48.755+0000 7f9d4c555700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d44108740 0x7f9d4419f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:43:48.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:48.755+0000 7f9d4c555700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9d44113370 con 0x7f9d441034c0 2024-05-09T00:43:48.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:48.755+0000 7f9d4c555700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9d441131f0 con 0x7f9d44108740 2024-05-09T00:43:48.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:48.755+0000 7f9d4c555700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9d441134f0 con 0x7f9d44107df0 2024-05-09T00:43:48.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:48.755+0000 7f9d4a2f1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d441034c0 0x7f9d4419ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:43:48.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:48.755+0000 7f9d4a2f1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d441034c0 0x7f9d4419ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45226/0 (socket says 172.21.15.5:45226) 2024-05-09T00:43:48.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:48.755+0000 7f9d4a2f1700 1 -- 172.21.15.5:0/1872990801 learned_addr learned my addr 172.21.15.5:0/1872990801 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:43:48.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:48.755+0000 7f9d49af0700 1 --2- 172.21.15.5:0/1872990801 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d44107df0 0x7f9d4419b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:43:48.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:48.755+0000 7f9d4aaf2700 1 --2- 172.21.15.5:0/1872990801 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d44108740 0x7f9d4419f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:43:48.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:48.755+0000 7f9d4a2f1700 1 -- 172.21.15.5:0/1872990801 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d44107df0 msgr2=0x7f9d4419b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:48.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:48.755+0000 7f9d4a2f1700 1 --2- 172.21.15.5:0/1872990801 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d44107df0 0x7f9d4419b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:48.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:48.755+0000 7f9d4a2f1700 1 -- 172.21.15.5:0/1872990801 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d44108740 msgr2=0x7f9d4419f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:48.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:48.755+0000 7f9d4a2f1700 1 --2- 172.21.15.5:0/1872990801 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d44108740 0x7f9d4419f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:48.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:48.755+0000 7f9d4a2f1700 1 -- 172.21.15.5:0/1872990801 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d4419fd20 con 0x7f9d441034c0 2024-05-09T00:43:48.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:48.755+0000 7f9d4a2f1700 1 --2- 172.21.15.5:0/1872990801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d441034c0 0x7f9d4419ae50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9d4000b580 tx=0x7f9d400047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:43:48.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:48.755+0000 7f9d3b7fe700 1 -- 172.21.15.5:0/1872990801 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d40027070 con 0x7f9d441034c0 2024-05-09T00:43:48.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:48.755+0000 7f9d3b7fe700 1 -- 172.21.15.5:0/1872990801 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d40003f20 con 0x7f9d441034c0 2024-05-09T00:43:48.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:48.756+0000 7f9d4c555700 1 -- 172.21.15.5:0/1872990801 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d4419ffb0 con 0x7f9d441034c0 2024-05-09T00:43:48.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:48.756+0000 7f9d3b7fe700 1 -- 172.21.15.5:0/1872990801 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d40022d10 con 0x7f9d441034c0 2024-05-09T00:43:48.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:48.756+0000 7f9d4c555700 1 -- 172.21.15.5:0/1872990801 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d441a7930 con 0x7f9d441034c0 2024-05-09T00:43:48.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:48.757+0000 7f9d4c555700 1 -- 172.21.15.5:0/1872990801 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d440512c0 con 0x7f9d441034c0 2024-05-09T00:43:48.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:48.758+0000 7f9d3b7fe700 1 -- 172.21.15.5:0/1872990801 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f9d40022470 con 0x7f9d441034c0 2024-05-09T00:43:48.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:48.759+0000 7f9d3b7fe700 1 --2- 172.21.15.5:0/1872990801 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f9d300b2900 0x7f9d300b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:43:48.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:48.759+0000 7f9d49af0700 1 --2- 172.21.15.5:0/1872990801 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f9d300b2900 0x7f9d300b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:43:48.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:48.759+0000 7f9d3b7fe700 1 -- 172.21.15.5:0/1872990801 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f9d400e2c50 con 0x7f9d441034c0 2024-05-09T00:43:48.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:48.759+0000 7f9d49af0700 1 --2- 172.21.15.5:0/1872990801 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f9d300b2900 0x7f9d300b4e10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9d441027f0 tx=0x7f9d3400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:43:48.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:48.764+0000 7f9d3b7fe700 1 -- 172.21.15.5:0/1872990801 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9d400e8020 con 0x7f9d441034c0 2024-05-09T00:43:48.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:48.980+0000 7f9d4c555700 1 -- 172.21.15.5:0/1872990801 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9d441a7b00 con 0x7f9d441034c0 2024-05-09T00:43:48.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:48.982+0000 7f9d3b7fe700 1 -- 172.21.15.5:0/1872990801 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+22182 (secure 0 0 0) 0x7f9d40013070 con 0x7f9d441034c0 2024-05-09T00:43:48.983 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:43:48.983 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":78,"fsid":"d85814c6-0d9b-11ef-bc97-c7b262605968","created":"2024-05-09T00:35:25.623377+0000","modified":"2024-05-09T00:43:45.856106+0000","last_up_change":"2024-05-09T00:42:38.525335+0000","last_in_change":"2024-05-09T00:42:17.990219+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T00:38:47.844734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T00:43:39.452106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T00:43:42.708106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f8d484bb-2477-4377-ae68-5bce571333bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2963385132},{"type":"v1","addr":"172.21.15.5:6803","nonce":2963385132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2963385132},{"type":"v1","addr":"172.21.15.5:6805","nonce":2963385132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2963385132},{"type":"v1","addr":"172.21.15.5:6809","nonce":2963385132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2963385132},{"type":"v1","addr":"172.21.15.5:6807","nonce":2963385132}]},"public_addr":"172.21.15.5:6803/2963385132","cluster_addr":"172.21.15.5:6805/2963385132","heartbeat_back_addr":"172.21.15.5:6809/2963385132","heartbeat_front_addr":"172.21.15.5:6807/2963385132","state":["exists","up"]},{"osd":1,"uuid":"1c123676-7144-4a67-93a6-fd49e36a5c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3837683163},{"type":"v1","addr":"172.21.15.131:6801","nonce":3837683163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3837683163},{"type":"v1","addr":"172.21.15.131:6803","nonce":3837683163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3837683163},{"type":"v1","addr":"172.21.15.131:6807","nonce":3837683163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3837683163},{"type":"v1","addr":"172.21.15.131:6805","nonce":3837683163}]},"public_addr":"172.21.15.131:6801/3837683163","cluster_addr":"172.21.15.131:6803/3837683163","heartbeat_back_addr":"172.21.15.131:6807/3837683163","heartbeat_front_addr":"172.21.15.131:6805/3837683163","state":["exists","up"]},{"osd":2,"uuid":"95ca51f0-b48b-485c-a5f7-b15c01e754b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":484818139},{"type":"v1","addr":"172.21.15.157:6801","nonce":484818139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":484818139},{"type":"v1","addr":"172.21.15.157:6803","nonce":484818139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":484818139},{"type":"v1","addr":"172.21.15.157:6807","nonce":484818139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":484818139},{"type":"v1","addr":"172.21.15.157:6805","nonce":484818139}]},"public_addr":"172.21.15.157:6801/484818139","cluster_addr":"172.21.15.157:6803/484818139","heartbeat_back_addr":"172.21.15.157:6807/484818139","heartbeat_front_addr":"172.21.15.157:6805/484818139","state":["exists","up"]},{"osd":3,"uuid":"ed7577fb-3676-4a08-b40d-5474eb88974f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":2357377277},{"type":"v1","addr":"172.21.15.5:6811","nonce":2357377277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":2357377277},{"type":"v1","addr":"172.21.15.5:6813","nonce":2357377277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":2357377277},{"type":"v1","addr":"172.21.15.5:6817","nonce":2357377277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":2357377277},{"type":"v1","addr":"172.21.15.5:6815","nonce":2357377277}]},"public_addr":"172.21.15.5:6811/2357377277","cluster_addr":"172.21.15.5:6813/2357377277","heartbeat_back_addr":"172.21.15.5:6817/2357377277","heartbeat_front_addr":"172.21.15.5:6815/2357377277","state":["exists","up"]},{"osd":4,"uuid":"fb5dcf79-a6f1-4b1a-9e3a-087fb5132472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":2915418378},{"type":"v1","addr":"172.21.15.131:6809","nonce":2915418378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":2915418378},{"type":"v1","addr":"172.21.15.131:6811","nonce":2915418378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":2915418378},{"type":"v1","addr":"172.21.15.131:6815","nonce":2915418378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":2915418378},{"type":"v1","addr":"172.21.15.131:6813","nonce":2915418378}]},"public_addr":"172.21.15.131:6809/2915418378","cluster_addr":"172.21.15.131:6811/2915418378","heartbeat_back_addr":"172.21.15.131:6815/2915418378","heartbeat_front_addr":"172.21.15.131:6813/2915418378","state":["exists","up"]},{"osd":5,"uuid":"f01ed5d3-8066-4d25-a113-280c83e75ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4188818663},{"type":"v1","addr":"172.21.15.157:6809","nonce":4188818663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":4188818663},{"type":"v1","addr":"172.21.15.157:6811","nonce":4188818663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":4188818663},{"type":"v1","addr":"172.21.15.157:6815","nonce":4188818663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":4188818663},{"type":"v1","addr":"172.21.15.157:6813","nonce":4188818663}]},"public_addr":"172.21.15.157:6809/4188818663","cluster_addr":"172.21.15.157:6811/4188818663","heartbeat_back_addr":"172.21.15.157:6815/4188818663","heartbeat_front_addr":"172.21.15.157:6813/4188818663","state":["exists","up"]},{"osd":6,"uuid":"932379a8-c97b-4acc-8665-81f7d9cbce65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2930397813},{"type":"v1","addr":"172.21.15.5:6819","nonce":2930397813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2930397813},{"type":"v1","addr":"172.21.15.5:6821","nonce":2930397813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2930397813},{"type":"v1","addr":"172.21.15.5:6825","nonce":2930397813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2930397813},{"type":"v1","addr":"172.21.15.5:6823","nonce":2930397813}]},"public_addr":"172.21.15.5:6819/2930397813","cluster_addr":"172.21.15.5:6821/2930397813","heartbeat_back_addr":"172.21.15.5:6825/2930397813","heartbeat_front_addr":"172.21.15.5:6823/2930397813","state":["exists","up"]},{"osd":7,"uuid":"3c0a0573-6465-4bf6-b70f-310d49857a6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":4125045219},{"type":"v1","addr":"172.21.15.131:6817","nonce":4125045219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":4125045219},{"type":"v1","addr":"172.21.15.131:6819","nonce":4125045219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":4125045219},{"type":"v1","addr":"172.21.15.131:6823","nonce":4125045219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":4125045219},{"type":"v1","addr":"172.21.15.131:6821","nonce":4125045219}]},"public_addr":"172.21.15.131:6817/4125045219","cluster_addr":"172.21.15.131:6819/4125045219","heartbeat_back_addr":"172.21.15.131:6823/4125045219","heartbeat_front_addr":"172.21.15.131:6821/4125045219","state":["exists","up"]},{"osd":8,"uuid":"c49f0a00-f5ae-40ed-8f38-7b953ba1c17c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2001626768},{"type":"v1","addr":"172.21.15.157:6817","nonce":2001626768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2001626768},{"type":"v1","addr":"172.21.15.157:6819","nonce":2001626768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2001626768},{"type":"v1","addr":"172.21.15.157:6823","nonce":2001626768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2001626768},{"type":"v1","addr":"172.21.15.157:6821","nonce":2001626768}]},"public_addr":"172.21.15.157:6817/2001626768","cluster_addr":"172.21.15.157:6819/2001626768","heartbeat_back_addr":"172.21.15.157:6823/2001626768","heartbeat_front_addr":"172.21.15.157:6821/2001626768","state":["exists","up"]},{"osd":9,"uuid":"1927dfe7-c1af-4a15-bb71-d58d97c8734a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":4287220015},{"type":"v1","addr":"172.21.15.5:6827","nonce":4287220015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":4287220015},{"type":"v1","addr":"172.21.15.5:6829","nonce":4287220015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":4287220015},{"type":"v1","addr":"172.21.15.5:6833","nonce":4287220015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":4287220015},{"type":"v1","addr":"172.21.15.5:6831","nonce":4287220015}]},"public_addr":"172.21.15.5:6827/4287220015","cluster_addr":"172.21.15.5:6829/4287220015","heartbeat_back_addr":"172.21.15.5:6833/4287220015","heartbeat_front_addr":"172.21.15.5:6831/4287220015","state":["exists","up"]},{"osd":10,"uuid":"fbb4f3e3-eb7d-4ecb-a30b-55b8910375ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1571783985},{"type":"v1","addr":"172.21.15.131:6825","nonce":1571783985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1571783985},{"type":"v1","addr":"172.21.15.131:6827","nonce":1571783985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1571783985},{"type":"v1","addr":"172.21.15.131:6831","nonce":1571783985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1571783985},{"type":"v1","addr":"172.21.15.131:6829","nonce":1571783985}]},"public_addr":"172.21.15.131:6825/1571783985","cluster_addr":"172.21.15.131:6827/1571783985","heartbeat_back_addr":"172.21.15.131:6831/1571783985","heartbeat_front_addr":"172.21.15.131:6829/1571783985","state":["exists","up"]},{"osd":11,"uuid":"b858453f-eda1-4e25-89c8-94b54c830938","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3261151188},{"type":"v1","addr":"172.21.15.157:6825","nonce":3261151188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":3261151188},{"type":"v1","addr":"172.21.15.157:6827","nonce":3261151188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":3261151188},{"type":"v1","addr":"172.21.15.157:6831","nonce":3261151188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":3261151188},{"type":"v1","addr":"172.21.15.157:6829","nonce":3261151188}]},"public_addr":"172.21.15.157:6825/3261151188","cluster_addr":"172.21.15.157:6827/3261151188","heartbeat_back_addr":"172.21.15.157:6831/3261151188","heartbeat_front_addr":"172.21.15.157:6829/3261151188","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:37:57.528483+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:38:21.477482+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:38:45.780266+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:39:10.938791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:39:35.734857+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:40:00.169040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:40:26.494635+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:40:51.881718+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:41:17.697112+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:41:44.826479+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:42:10.504768+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:42:36.605447+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/3405015786":"2024-05-10T00:36:21.742363+0000","172.21.15.5:0/2669115557":"2024-05-10T00:36:21.742363+0000","172.21.15.5:6801/287842266":"2024-05-10T00:36:21.742363+0000","172.21.15.5:6800/752513054":"2024-05-10T00:35:53.128955+0000","172.21.15.5:6800/287842266":"2024-05-10T00:36:21.742363+0000","172.21.15.5:6801/752513054":"2024-05-10T00:35:53.128955+0000","172.21.15.5:0/1954876662":"2024-05-10T00:35:53.128955+0000","172.21.15.5:0/2545175121":"2024-05-10T00:36:21.742363+0000","172.21.15.5:0/3754653663":"2024-05-10T00:35:53.128955+0000","172.21.15.5:0/651514359":"2024-05-10T00:35:53.128955+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T00:43:48.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:48.984+0000 7f9d4c555700 1 -- 172.21.15.5:0/1872990801 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f9d300b2900 msgr2=0x7f9d300b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:48.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:48.984+0000 7f9d4c555700 1 --2- 172.21.15.5:0/1872990801 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f9d300b2900 0x7f9d300b4e10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9d441027f0 tx=0x7f9d3400b410 comp rx=0 tx=0).stop 2024-05-09T00:43:48.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:48.984+0000 7f9d4c555700 1 -- 172.21.15.5:0/1872990801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d441034c0 msgr2=0x7f9d4419ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:48.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:48.984+0000 7f9d4c555700 1 --2- 172.21.15.5:0/1872990801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d441034c0 0x7f9d4419ae50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9d4000b580 tx=0x7f9d400047b0 comp rx=0 tx=0).stop 2024-05-09T00:43:48.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:48.984+0000 7f9d4c555700 1 -- 172.21.15.5:0/1872990801 shutdown_connections 2024-05-09T00:43:48.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:48.984+0000 7f9d4c555700 1 --2- 172.21.15.5:0/1872990801 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f9d300b2900 0x7f9d300b4e10 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:48.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:48.984+0000 7f9d4c555700 1 --2- 172.21.15.5:0/1872990801 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d44108740 0x7f9d4419f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:48.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:48.984+0000 7f9d4c555700 1 --2- 172.21.15.5:0/1872990801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d441034c0 0x7f9d4419ae50 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:48.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:48.984+0000 7f9d4c555700 1 --2- 172.21.15.5:0/1872990801 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d44107df0 0x7f9d4419b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:48.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:48.984+0000 7f9d4c555700 1 -- 172.21.15.5:0/1872990801 >> 172.21.15.5:0/1872990801 conn(0x7f9d440fedf0 msgr2=0x7f9d441096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:43:48.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:48.984+0000 7f9d4c555700 1 -- 172.21.15.5:0/1872990801 shutdown_connections 2024-05-09T00:43:48.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:48.984+0000 7f9d4c555700 1 -- 172.21.15.5:0/1872990801 wait complete. 2024-05-09T00:43:48.997 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:48 smithi005 ceph-mon[30365]: pgmap v291: 97 pgs: 1 creating+activating, 17 creating+peering, 79 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 2 op/s 2024-05-09T00:43:48.997 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:48 smithi005 ceph-mon[30365]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-09T00:43:48.997 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:48 smithi005 ceph-mon[30365]: Cluster is now healthy 2024-05-09T00:43:49.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:48 smithi157 ceph-mon[34294]: pgmap v291: 97 pgs: 1 creating+activating, 17 creating+peering, 79 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 2 op/s 2024-05-09T00:43:49.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:48 smithi157 ceph-mon[34294]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-09T00:43:49.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:48 smithi157 ceph-mon[34294]: Cluster is now healthy 2024-05-09T00:43:49.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:48 smithi131 ceph-mon[34547]: pgmap v291: 97 pgs: 1 creating+activating, 17 creating+peering, 79 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 2 op/s 2024-05-09T00:43:49.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:48 smithi131 ceph-mon[34547]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-09T00:43:49.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:48 smithi131 ceph-mon[34547]: Cluster is now healthy 2024-05-09T00:43:49.767 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T00:43:50.032 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:43:50.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:49 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/1872990801' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:43:50.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:49 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/1872990801' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:43:50.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:49 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/1872990801' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:43:51.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:50 smithi157 ceph-mon[34294]: pgmap v292: 97 pgs: 97 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-05-09T00:43:51.278 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:50 smithi005 ceph-mon[30365]: pgmap v292: 97 pgs: 97 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-05-09T00:43:51.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:50 smithi131 ceph-mon[34547]: pgmap v292: 97 pgs: 97 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-05-09T00:43:51.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:51.843+0000 7f70197a8700 1 -- 172.21.15.5:0/2272534698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7014109850 msgr2=0x7f70141111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:51.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:51.843+0000 7f70197a8700 1 --2- 172.21.15.5:0/2272534698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7014109850 0x7f70141111b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f700c00b0d0 tx=0x7f700c00b3e0 comp rx=0 tx=0).stop 2024-05-09T00:43:51.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:51.843+0000 7f70197a8700 1 -- 172.21.15.5:0/2272534698 shutdown_connections 2024-05-09T00:43:51.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:51.843+0000 7f70197a8700 1 --2- 172.21.15.5:0/2272534698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7014109850 0x7f70141111b0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:51.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:51.843+0000 7f70197a8700 1 --2- 172.21.15.5:0/2272534698 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f70141045d0 0x7f70141049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:51.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:51.843+0000 7f70197a8700 1 --2- 172.21.15.5:0/2272534698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7014108f00 0x7f7014109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:51.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:51.843+0000 7f70197a8700 1 -- 172.21.15.5:0/2272534698 >> 172.21.15.5:0/2272534698 conn(0x7f70140ffea0 msgr2=0x7f70141022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:43:51.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:51.843+0000 7f70197a8700 1 -- 172.21.15.5:0/2272534698 shutdown_connections 2024-05-09T00:43:51.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:51.843+0000 7f70197a8700 1 -- 172.21.15.5:0/2272534698 wait complete. 2024-05-09T00:43:51.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:51.843+0000 7f70197a8700 1 Processor -- start 2024-05-09T00:43:51.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:51.844+0000 7f70197a8700 1 -- start start 2024-05-09T00:43:51.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:51.844+0000 7f70197a8700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f70141045d0 0x7f7014076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:43:51.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:51.844+0000 7f70197a8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7014108f00 0x7f7014077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:43:51.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:51.844+0000 7f70197a8700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7014109850 0x7f7014077650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:43:51.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:51.844+0000 7f70197a8700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7014114480 con 0x7f7014108f00 2024-05-09T00:43:51.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:51.844+0000 7f70197a8700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7014114300 con 0x7f70141045d0 2024-05-09T00:43:51.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:51.844+0000 7f70197a8700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7014114600 con 0x7f7014109850 2024-05-09T00:43:51.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:51.844+0000 7f70127fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7014108f00 0x7f7014077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:43:51.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:51.844+0000 7f70127fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7014108f00 0x7f7014077110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45252/0 (socket says 172.21.15.5:45252) 2024-05-09T00:43:51.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:51.844+0000 7f70127fc700 1 -- 172.21.15.5:0/1393901376 learned_addr learned my addr 172.21.15.5:0/1393901376 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:43:51.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:51.844+0000 7f70137fe700 1 --2- 172.21.15.5:0/1393901376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7014109850 0x7f7014077650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:43:51.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:51.844+0000 7f7012ffd700 1 --2- 172.21.15.5:0/1393901376 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f70141045d0 0x7f7014076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:43:51.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:51.845+0000 7f70127fc700 1 -- 172.21.15.5:0/1393901376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7014109850 msgr2=0x7f7014077650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:51.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:51.845+0000 7f70127fc700 1 --2- 172.21.15.5:0/1393901376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7014109850 0x7f7014077650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:51.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:51.845+0000 7f70127fc700 1 -- 172.21.15.5:0/1393901376 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f70141045d0 msgr2=0x7f7014076bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:51.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:51.845+0000 7f70127fc700 1 --2- 172.21.15.5:0/1393901376 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f70141045d0 0x7f7014076bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:51.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:51.845+0000 7f70127fc700 1 -- 172.21.15.5:0/1393901376 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70141a9020 con 0x7f7014108f00 2024-05-09T00:43:51.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:51.845+0000 7f70127fc700 1 --2- 172.21.15.5:0/1393901376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7014108f00 0x7f7014077110 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f700800c9e0 tx=0x7f700800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:43:51.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:51.845+0000 7f6ffbfff700 1 -- 172.21.15.5:0/1393901376 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7008007aa0 con 0x7f7014108f00 2024-05-09T00:43:51.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:51.845+0000 7f70197a8700 1 -- 172.21.15.5:0/1393901376 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70141a9310 con 0x7f7014108f00 2024-05-09T00:43:51.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:51.845+0000 7f6ffbfff700 1 -- 172.21.15.5:0/1393901376 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7008007c40 con 0x7f7014108f00 2024-05-09T00:43:51.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:51.845+0000 7f6ffbfff700 1 -- 172.21.15.5:0/1393901376 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f700800fe60 con 0x7f7014108f00 2024-05-09T00:43:51.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:51.845+0000 7f70197a8700 1 -- 172.21.15.5:0/1393901376 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70141a98a0 con 0x7f7014108f00 2024-05-09T00:43:51.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:51.846+0000 7f70197a8700 1 -- 172.21.15.5:0/1393901376 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70140512c0 con 0x7f7014108f00 2024-05-09T00:43:51.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:51.848+0000 7f6ffbfff700 1 -- 172.21.15.5:0/1393901376 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f700800f450 con 0x7f7014108f00 2024-05-09T00:43:51.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:51.848+0000 7f6ffbfff700 1 --2- 172.21.15.5:0/1393901376 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f6ffc0b2900 0x7f6ffc0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:43:51.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:51.849+0000 7f7012ffd700 1 --2- 172.21.15.5:0/1393901376 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f6ffc0b2900 0x7f6ffc0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:43:51.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:51.849+0000 7f6ffbfff700 1 -- 172.21.15.5:0/1393901376 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f70080e2160 con 0x7f7014108f00 2024-05-09T00:43:51.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:51.849+0000 7f7012ffd700 1 --2- 172.21.15.5:0/1393901376 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f6ffc0b2900 0x7f6ffc0b4e10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f700400ac50 tx=0x7f700400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:43:51.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:51.852+0000 7f6ffbfff700 1 -- 172.21.15.5:0/1393901376 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7008014030 con 0x7f7014108f00 2024-05-09T00:43:52.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:52.072+0000 7f70197a8700 1 -- 172.21.15.5:0/1393901376 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f701402e5f0 con 0x7f7014108f00 2024-05-09T00:43:52.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:52.073+0000 7f6ffbfff700 1 -- 172.21.15.5:0/1393901376 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+22182 (secure 0 0 0) 0x7f70080aa0e0 con 0x7f7014108f00 2024-05-09T00:43:52.074 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:43:52.075 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":78,"fsid":"d85814c6-0d9b-11ef-bc97-c7b262605968","created":"2024-05-09T00:35:25.623377+0000","modified":"2024-05-09T00:43:45.856106+0000","last_up_change":"2024-05-09T00:42:38.525335+0000","last_in_change":"2024-05-09T00:42:17.990219+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T00:38:47.844734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T00:43:39.452106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T00:43:42.708106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f8d484bb-2477-4377-ae68-5bce571333bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2963385132},{"type":"v1","addr":"172.21.15.5:6803","nonce":2963385132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2963385132},{"type":"v1","addr":"172.21.15.5:6805","nonce":2963385132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2963385132},{"type":"v1","addr":"172.21.15.5:6809","nonce":2963385132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2963385132},{"type":"v1","addr":"172.21.15.5:6807","nonce":2963385132}]},"public_addr":"172.21.15.5:6803/2963385132","cluster_addr":"172.21.15.5:6805/2963385132","heartbeat_back_addr":"172.21.15.5:6809/2963385132","heartbeat_front_addr":"172.21.15.5:6807/2963385132","state":["exists","up"]},{"osd":1,"uuid":"1c123676-7144-4a67-93a6-fd49e36a5c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3837683163},{"type":"v1","addr":"172.21.15.131:6801","nonce":3837683163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3837683163},{"type":"v1","addr":"172.21.15.131:6803","nonce":3837683163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3837683163},{"type":"v1","addr":"172.21.15.131:6807","nonce":3837683163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3837683163},{"type":"v1","addr":"172.21.15.131:6805","nonce":3837683163}]},"public_addr":"172.21.15.131:6801/3837683163","cluster_addr":"172.21.15.131:6803/3837683163","heartbeat_back_addr":"172.21.15.131:6807/3837683163","heartbeat_front_addr":"172.21.15.131:6805/3837683163","state":["exists","up"]},{"osd":2,"uuid":"95ca51f0-b48b-485c-a5f7-b15c01e754b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":484818139},{"type":"v1","addr":"172.21.15.157:6801","nonce":484818139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":484818139},{"type":"v1","addr":"172.21.15.157:6803","nonce":484818139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":484818139},{"type":"v1","addr":"172.21.15.157:6807","nonce":484818139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":484818139},{"type":"v1","addr":"172.21.15.157:6805","nonce":484818139}]},"public_addr":"172.21.15.157:6801/484818139","cluster_addr":"172.21.15.157:6803/484818139","heartbeat_back_addr":"172.21.15.157:6807/484818139","heartbeat_front_addr":"172.21.15.157:6805/484818139","state":["exists","up"]},{"osd":3,"uuid":"ed7577fb-3676-4a08-b40d-5474eb88974f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":2357377277},{"type":"v1","addr":"172.21.15.5:6811","nonce":2357377277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":2357377277},{"type":"v1","addr":"172.21.15.5:6813","nonce":2357377277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":2357377277},{"type":"v1","addr":"172.21.15.5:6817","nonce":2357377277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":2357377277},{"type":"v1","addr":"172.21.15.5:6815","nonce":2357377277}]},"public_addr":"172.21.15.5:6811/2357377277","cluster_addr":"172.21.15.5:6813/2357377277","heartbeat_back_addr":"172.21.15.5:6817/2357377277","heartbeat_front_addr":"172.21.15.5:6815/2357377277","state":["exists","up"]},{"osd":4,"uuid":"fb5dcf79-a6f1-4b1a-9e3a-087fb5132472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":2915418378},{"type":"v1","addr":"172.21.15.131:6809","nonce":2915418378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":2915418378},{"type":"v1","addr":"172.21.15.131:6811","nonce":2915418378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":2915418378},{"type":"v1","addr":"172.21.15.131:6815","nonce":2915418378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":2915418378},{"type":"v1","addr":"172.21.15.131:6813","nonce":2915418378}]},"public_addr":"172.21.15.131:6809/2915418378","cluster_addr":"172.21.15.131:6811/2915418378","heartbeat_back_addr":"172.21.15.131:6815/2915418378","heartbeat_front_addr":"172.21.15.131:6813/2915418378","state":["exists","up"]},{"osd":5,"uuid":"f01ed5d3-8066-4d25-a113-280c83e75ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4188818663},{"type":"v1","addr":"172.21.15.157:6809","nonce":4188818663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":4188818663},{"type":"v1","addr":"172.21.15.157:6811","nonce":4188818663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":4188818663},{"type":"v1","addr":"172.21.15.157:6815","nonce":4188818663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":4188818663},{"type":"v1","addr":"172.21.15.157:6813","nonce":4188818663}]},"public_addr":"172.21.15.157:6809/4188818663","cluster_addr":"172.21.15.157:6811/4188818663","heartbeat_back_addr":"172.21.15.157:6815/4188818663","heartbeat_front_addr":"172.21.15.157:6813/4188818663","state":["exists","up"]},{"osd":6,"uuid":"932379a8-c97b-4acc-8665-81f7d9cbce65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2930397813},{"type":"v1","addr":"172.21.15.5:6819","nonce":2930397813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2930397813},{"type":"v1","addr":"172.21.15.5:6821","nonce":2930397813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2930397813},{"type":"v1","addr":"172.21.15.5:6825","nonce":2930397813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2930397813},{"type":"v1","addr":"172.21.15.5:6823","nonce":2930397813}]},"public_addr":"172.21.15.5:6819/2930397813","cluster_addr":"172.21.15.5:6821/2930397813","heartbeat_back_addr":"172.21.15.5:6825/2930397813","heartbeat_front_addr":"172.21.15.5:6823/2930397813","state":["exists","up"]},{"osd":7,"uuid":"3c0a0573-6465-4bf6-b70f-310d49857a6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":4125045219},{"type":"v1","addr":"172.21.15.131:6817","nonce":4125045219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":4125045219},{"type":"v1","addr":"172.21.15.131:6819","nonce":4125045219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":4125045219},{"type":"v1","addr":"172.21.15.131:6823","nonce":4125045219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":4125045219},{"type":"v1","addr":"172.21.15.131:6821","nonce":4125045219}]},"public_addr":"172.21.15.131:6817/4125045219","cluster_addr":"172.21.15.131:6819/4125045219","heartbeat_back_addr":"172.21.15.131:6823/4125045219","heartbeat_front_addr":"172.21.15.131:6821/4125045219","state":["exists","up"]},{"osd":8,"uuid":"c49f0a00-f5ae-40ed-8f38-7b953ba1c17c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2001626768},{"type":"v1","addr":"172.21.15.157:6817","nonce":2001626768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2001626768},{"type":"v1","addr":"172.21.15.157:6819","nonce":2001626768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2001626768},{"type":"v1","addr":"172.21.15.157:6823","nonce":2001626768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2001626768},{"type":"v1","addr":"172.21.15.157:6821","nonce":2001626768}]},"public_addr":"172.21.15.157:6817/2001626768","cluster_addr":"172.21.15.157:6819/2001626768","heartbeat_back_addr":"172.21.15.157:6823/2001626768","heartbeat_front_addr":"172.21.15.157:6821/2001626768","state":["exists","up"]},{"osd":9,"uuid":"1927dfe7-c1af-4a15-bb71-d58d97c8734a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":4287220015},{"type":"v1","addr":"172.21.15.5:6827","nonce":4287220015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":4287220015},{"type":"v1","addr":"172.21.15.5:6829","nonce":4287220015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":4287220015},{"type":"v1","addr":"172.21.15.5:6833","nonce":4287220015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":4287220015},{"type":"v1","addr":"172.21.15.5:6831","nonce":4287220015}]},"public_addr":"172.21.15.5:6827/4287220015","cluster_addr":"172.21.15.5:6829/4287220015","heartbeat_back_addr":"172.21.15.5:6833/4287220015","heartbeat_front_addr":"172.21.15.5:6831/4287220015","state":["exists","up"]},{"osd":10,"uuid":"fbb4f3e3-eb7d-4ecb-a30b-55b8910375ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1571783985},{"type":"v1","addr":"172.21.15.131:6825","nonce":1571783985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1571783985},{"type":"v1","addr":"172.21.15.131:6827","nonce":1571783985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1571783985},{"type":"v1","addr":"172.21.15.131:6831","nonce":1571783985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1571783985},{"type":"v1","addr":"172.21.15.131:6829","nonce":1571783985}]},"public_addr":"172.21.15.131:6825/1571783985","cluster_addr":"172.21.15.131:6827/1571783985","heartbeat_back_addr":"172.21.15.131:6831/1571783985","heartbeat_front_addr":"172.21.15.131:6829/1571783985","state":["exists","up"]},{"osd":11,"uuid":"b858453f-eda1-4e25-89c8-94b54c830938","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3261151188},{"type":"v1","addr":"172.21.15.157:6825","nonce":3261151188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":3261151188},{"type":"v1","addr":"172.21.15.157:6827","nonce":3261151188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":3261151188},{"type":"v1","addr":"172.21.15.157:6831","nonce":3261151188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":3261151188},{"type":"v1","addr":"172.21.15.157:6829","nonce":3261151188}]},"public_addr":"172.21.15.157:6825/3261151188","cluster_addr":"172.21.15.157:6827/3261151188","heartbeat_back_addr":"172.21.15.157:6831/3261151188","heartbeat_front_addr":"172.21.15.157:6829/3261151188","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:37:57.528483+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:38:21.477482+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:38:45.780266+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:39:10.938791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:39:35.734857+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:40:00.169040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:40:26.494635+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:40:51.881718+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:41:17.697112+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:41:44.826479+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:42:10.504768+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:42:36.605447+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/3405015786":"2024-05-10T00:36:21.742363+0000","172.21.15.5:0/2669115557":"2024-05-10T00:36:21.742363+0000","172.21.15.5:6801/287842266":"2024-05-10T00:36:21.742363+0000","172.21.15.5:6800/752513054":"2024-05-10T00:35:53.128955+0000","172.21.15.5:6800/287842266":"2024-05-10T00:36:21.742363+0000","172.21.15.5:6801/752513054":"2024-05-10T00:35:53.128955+0000","172.21.15.5:0/1954876662":"2024-05-10T00:35:53.128955+0000","172.21.15.5:0/2545175121":"2024-05-10T00:36:21.742363+0000","172.21.15.5:0/3754653663":"2024-05-10T00:35:53.128955+0000","172.21.15.5:0/651514359":"2024-05-10T00:35:53.128955+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T00:43:52.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:52.075+0000 7f70197a8700 1 -- 172.21.15.5:0/1393901376 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f6ffc0b2900 msgr2=0x7f6ffc0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:52.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:52.075+0000 7f70197a8700 1 --2- 172.21.15.5:0/1393901376 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f6ffc0b2900 0x7f6ffc0b4e10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f700400ac50 tx=0x7f700400a380 comp rx=0 tx=0).stop 2024-05-09T00:43:52.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:52.075+0000 7f70197a8700 1 -- 172.21.15.5:0/1393901376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7014108f00 msgr2=0x7f7014077110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:52.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:52.075+0000 7f70197a8700 1 --2- 172.21.15.5:0/1393901376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7014108f00 0x7f7014077110 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f700800c9e0 tx=0x7f700800ccf0 comp rx=0 tx=0).stop 2024-05-09T00:43:52.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:52.075+0000 7f70197a8700 1 -- 172.21.15.5:0/1393901376 shutdown_connections 2024-05-09T00:43:52.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:52.075+0000 7f70197a8700 1 --2- 172.21.15.5:0/1393901376 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f6ffc0b2900 0x7f6ffc0b4e10 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:52.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:52.075+0000 7f70197a8700 1 --2- 172.21.15.5:0/1393901376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7014109850 0x7f7014077650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:52.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:52.075+0000 7f70197a8700 1 --2- 172.21.15.5:0/1393901376 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f70141045d0 0x7f7014076bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:52.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:52.075+0000 7f70197a8700 1 --2- 172.21.15.5:0/1393901376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7014108f00 0x7f7014077110 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:52.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:52.075+0000 7f70197a8700 1 -- 172.21.15.5:0/1393901376 >> 172.21.15.5:0/1393901376 conn(0x7f70140ffea0 msgr2=0x7f7014108670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:43:52.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:52.076+0000 7f70197a8700 1 -- 172.21.15.5:0/1393901376 shutdown_connections 2024-05-09T00:43:52.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:52.076+0000 7f70197a8700 1 -- 172.21.15.5:0/1393901376 wait complete. 2024-05-09T00:43:52.760 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-05-09T00:43:53.015 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:43:53.040 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:52 smithi005 ceph-mon[30365]: pgmap v293: 97 pgs: 97 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-05-09T00:43:53.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:52 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/1393901376' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:43:53.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:52 smithi157 ceph-mon[34294]: pgmap v293: 97 pgs: 97 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-05-09T00:43:53.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:52 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/1393901376' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:43:53.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:52 smithi131 ceph-mon[34547]: pgmap v293: 97 pgs: 97 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-05-09T00:43:53.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:52 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/1393901376' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:43:54.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:54.701+0000 7f336968e700 1 -- 172.21.15.5:0/3381336574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33641045d0 msgr2=0x7f33641049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:54.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:54.701+0000 7f336968e700 1 --2- 172.21.15.5:0/3381336574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33641045d0 0x7f33641049e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f33540099c0 tx=0x7f3354009cd0 comp rx=0 tx=0).stop 2024-05-09T00:43:54.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:54.701+0000 7f336968e700 1 -- 172.21.15.5:0/3381336574 shutdown_connections 2024-05-09T00:43:54.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:54.701+0000 7f336968e700 1 --2- 172.21.15.5:0/3381336574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3364109850 0x7f33641111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:54.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:54.701+0000 7f336968e700 1 --2- 172.21.15.5:0/3381336574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33641045d0 0x7f33641049e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:54.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:54.701+0000 7f336968e700 1 --2- 172.21.15.5:0/3381336574 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3364108f00 0x7f3364109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:54.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:54.701+0000 7f336968e700 1 -- 172.21.15.5:0/3381336574 >> 172.21.15.5:0/3381336574 conn(0x7f33640ffea0 msgr2=0x7f33641022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:43:54.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:54.701+0000 7f336968e700 1 -- 172.21.15.5:0/3381336574 shutdown_connections 2024-05-09T00:43:54.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:54.701+0000 7f336968e700 1 -- 172.21.15.5:0/3381336574 wait complete. 2024-05-09T00:43:54.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:54.702+0000 7f336968e700 1 Processor -- start 2024-05-09T00:43:54.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:54.702+0000 7f336968e700 1 -- start start 2024-05-09T00:43:54.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:54.702+0000 7f336968e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33641045d0 0x7f336419add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:43:54.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:54.702+0000 7f336968e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3364108f00 0x7f336419b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:43:54.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:54.702+0000 7f336968e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3364109850 0x7f336419f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:43:54.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:54.702+0000 7f336968e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3364114480 con 0x7f3364108f00 2024-05-09T00:43:54.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:54.702+0000 7f336968e700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3364114300 con 0x7f33641045d0 2024-05-09T00:43:54.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:54.702+0000 7f336968e700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3364114600 con 0x7f3364109850 2024-05-09T00:43:54.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:54.702+0000 7f33627fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3364108f00 0x7f336419b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:43:54.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:54.702+0000 7f33627fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3364108f00 0x7f336419b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60874/0 (socket says 172.21.15.5:60874) 2024-05-09T00:43:54.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:54.702+0000 7f33627fc700 1 -- 172.21.15.5:0/2390235164 learned_addr learned my addr 172.21.15.5:0/2390235164 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:43:54.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:54.702+0000 7f33637fe700 1 --2- 172.21.15.5:0/2390235164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3364109850 0x7f336419f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:43:54.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:54.702+0000 7f3362ffd700 1 --2- 172.21.15.5:0/2390235164 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33641045d0 0x7f336419add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:43:54.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:54.703+0000 7f33627fc700 1 -- 172.21.15.5:0/2390235164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3364109850 msgr2=0x7f336419f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:54.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:54.703+0000 7f33627fc700 1 --2- 172.21.15.5:0/2390235164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3364109850 0x7f336419f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:54.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:54.703+0000 7f33627fc700 1 -- 172.21.15.5:0/2390235164 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33641045d0 msgr2=0x7f336419add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:54.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:54.703+0000 7f33627fc700 1 --2- 172.21.15.5:0/2390235164 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33641045d0 0x7f336419add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:54.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:54.703+0000 7f33627fc700 1 -- 172.21.15.5:0/2390235164 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f336419fca0 con 0x7f3364108f00 2024-05-09T00:43:54.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:54.703+0000 7f3362ffd700 1 --2- 172.21.15.5:0/2390235164 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33641045d0 0x7f336419add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:43:54.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:54.703+0000 7f33627fc700 1 --2- 172.21.15.5:0/2390235164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3364108f00 0x7f336419b370 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f335800d980 tx=0x7f335800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:43:54.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:54.703+0000 7f334bfff700 1 -- 172.21.15.5:0/2390235164 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3358017070 con 0x7f3364108f00 2024-05-09T00:43:54.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:54.703+0000 7f334bfff700 1 -- 172.21.15.5:0/2390235164 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3358009de0 con 0x7f3364108f00 2024-05-09T00:43:54.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:54.703+0000 7f336968e700 1 -- 172.21.15.5:0/2390235164 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f336419ff90 con 0x7f3364108f00 2024-05-09T00:43:54.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:54.703+0000 7f334bfff700 1 -- 172.21.15.5:0/2390235164 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335800f3c0 con 0x7f3364108f00 2024-05-09T00:43:54.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:54.703+0000 7f336968e700 1 -- 172.21.15.5:0/2390235164 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33641a78b0 con 0x7f3364108f00 2024-05-09T00:43:54.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:54.704+0000 7f336968e700 1 -- 172.21.15.5:0/2390235164 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33640512c0 con 0x7f3364108f00 2024-05-09T00:43:54.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:54.706+0000 7f334bfff700 1 -- 172.21.15.5:0/2390235164 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f3358009930 con 0x7f3364108f00 2024-05-09T00:43:54.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:54.706+0000 7f334bfff700 1 --2- 172.21.15.5:0/2390235164 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f334c0b2880 0x7f334c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:43:54.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:54.707+0000 7f334bfff700 1 -- 172.21.15.5:0/2390235164 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f33580e1b90 con 0x7f3364108f00 2024-05-09T00:43:54.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:54.707+0000 7f3362ffd700 1 --2- 172.21.15.5:0/2390235164 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f334c0b2880 0x7f334c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:43:54.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:54.707+0000 7f3362ffd700 1 --2- 172.21.15.5:0/2390235164 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f334c0b2880 0x7f334c0b4d90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3354000c00 tx=0x7f335401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:43:54.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:54.710+0000 7f334bfff700 1 -- 172.21.15.5:0/2390235164 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f33580a9a00 con 0x7f3364108f00 2024-05-09T00:43:54.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:54.974+0000 7f336968e700 1 -- 172.21.15.5:0/2390235164 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f336402e5f0 con 0x7f3364108f00 2024-05-09T00:43:55.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:54 smithi157 ceph-mon[34294]: pgmap v294: 97 pgs: 97 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-05-09T00:43:55.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:54 smithi005 ceph-mon[30365]: pgmap v294: 97 pgs: 97 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-05-09T00:43:55.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:54 smithi131 ceph-mon[34547]: pgmap v294: 97 pgs: 97 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-05-09T00:43:55.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:55.910+0000 7f334bfff700 1 -- 172.21.15.5:0/2390235164 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f33580a9300 con 0x7f3364108f00 2024-05-09T00:43:55.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:55.913+0000 7f336968e700 1 -- 172.21.15.5:0/2390235164 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f334c0b2880 msgr2=0x7f334c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:55.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:55.913+0000 7f336968e700 1 --2- 172.21.15.5:0/2390235164 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f334c0b2880 0x7f334c0b4d90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3354000c00 tx=0x7f335401a040 comp rx=0 tx=0).stop 2024-05-09T00:43:55.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:55.913+0000 7f336968e700 1 -- 172.21.15.5:0/2390235164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3364108f00 msgr2=0x7f336419b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:55.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:55.913+0000 7f336968e700 1 --2- 172.21.15.5:0/2390235164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3364108f00 0x7f336419b370 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f335800d980 tx=0x7f335800dc90 comp rx=0 tx=0).stop 2024-05-09T00:43:55.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:55.913+0000 7f336968e700 1 -- 172.21.15.5:0/2390235164 shutdown_connections 2024-05-09T00:43:55.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:55.913+0000 7f336968e700 1 --2- 172.21.15.5:0/2390235164 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f334c0b2880 0x7f334c0b4d90 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:55.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:55.913+0000 7f336968e700 1 --2- 172.21.15.5:0/2390235164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3364109850 0x7f336419f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:55.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:55.913+0000 7f336968e700 1 --2- 172.21.15.5:0/2390235164 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33641045d0 0x7f336419add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:55.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:55.913+0000 7f336968e700 1 --2- 172.21.15.5:0/2390235164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3364108f00 0x7f336419b370 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:55.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:55.913+0000 7f336968e700 1 -- 172.21.15.5:0/2390235164 >> 172.21.15.5:0/2390235164 conn(0x7f33640ffea0 msgr2=0x7f336410a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:43:55.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:55.914+0000 7f336968e700 1 -- 172.21.15.5:0/2390235164 shutdown_connections 2024-05-09T00:43:55.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:55.914+0000 7f336968e700 1 -- 172.21.15.5:0/2390235164 wait complete. 2024-05-09T00:43:56.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:55 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/2390235164' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-09T00:43:56.248 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:55 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/2390235164' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-09T00:43:56.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:55 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/2390235164' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-09T00:43:56.641 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-05-09T00:43:56.642 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-05-09T00:43:56.904 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:43:57.184 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:56 smithi005 ceph-mon[30365]: pgmap v295: 97 pgs: 97 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-09T00:43:57.184 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:56 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/2390235164' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-09T00:43:57.184 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:56 smithi005 ceph-mon[30365]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-09T00:43:57.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:56 smithi157 ceph-mon[34294]: pgmap v295: 97 pgs: 97 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-09T00:43:57.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:56 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/2390235164' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-09T00:43:57.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:56 smithi157 ceph-mon[34294]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-09T00:43:57.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:56 smithi131 ceph-mon[34547]: pgmap v295: 97 pgs: 97 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-09T00:43:57.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:56 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/2390235164' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-09T00:43:57.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:56 smithi131 ceph-mon[34547]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-09T00:43:58.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:58.630+0000 7f003bf5c700 1 -- 172.21.15.5:0/519369953 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0034105dd0 msgr2=0x7f003410e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:58.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:58.630+0000 7f003bf5c700 1 --2- 172.21.15.5:0/519369953 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0034105dd0 0x7f003410e320 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0024009990 tx=0x7f0024009ca0 comp rx=0 tx=0).stop 2024-05-09T00:43:58.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:58.630+0000 7f003bf5c700 1 -- 172.21.15.5:0/519369953 shutdown_connections 2024-05-09T00:43:58.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:58.630+0000 7f003bf5c700 1 --2- 172.21.15.5:0/519369953 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f003410e8f0 0x7f0034110dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:58.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:58.630+0000 7f003bf5c700 1 --2- 172.21.15.5:0/519369953 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0034105480 0x7f0034105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:58.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:58.630+0000 7f003bf5c700 1 --2- 172.21.15.5:0/519369953 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0034105dd0 0x7f003410e320 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:58.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:58.630+0000 7f003bf5c700 1 -- 172.21.15.5:0/519369953 >> 172.21.15.5:0/519369953 conn(0x7f00340fef30 msgr2=0x7f0034101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:43:58.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:58.631+0000 7f003bf5c700 1 -- 172.21.15.5:0/519369953 shutdown_connections 2024-05-09T00:43:58.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:58.631+0000 7f003bf5c700 1 -- 172.21.15.5:0/519369953 wait complete. 2024-05-09T00:43:58.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:58.631+0000 7f003bf5c700 1 Processor -- start 2024-05-09T00:43:58.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:58.631+0000 7f003bf5c700 1 -- start start 2024-05-09T00:43:58.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:58.632+0000 7f003bf5c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0034105480 0x7f0034103170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:43:58.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:58.632+0000 7f003bf5c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0034105dd0 0x7f00341036b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:43:58.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:58.632+0000 7f003bf5c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f003410e8f0 0x7f0034103bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:43:58.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:58.632+0000 7f003bf5c700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f00341144d0 con 0x7f0034105dd0 2024-05-09T00:43:58.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:58.632+0000 7f003bf5c700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0034114350 con 0x7f0034105480 2024-05-09T00:43:58.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:58.632+0000 7f003bf5c700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0034114650 con 0x7f003410e8f0 2024-05-09T00:43:58.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:58.632+0000 7f00394f7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0034105dd0 0x7f00341036b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:43:58.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:58.632+0000 7f00394f7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0034105dd0 0x7f00341036b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60898/0 (socket says 172.21.15.5:60898) 2024-05-09T00:43:58.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:58.632+0000 7f00394f7700 1 -- 172.21.15.5:0/3268321799 learned_addr learned my addr 172.21.15.5:0/3268321799 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:43:58.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:58.632+0000 7f0039cf8700 1 --2- 172.21.15.5:0/3268321799 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0034105480 0x7f0034103170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:43:58.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:58.632+0000 7f003a4f9700 1 --2- 172.21.15.5:0/3268321799 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f003410e8f0 0x7f0034103bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:43:58.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:58.632+0000 7f00394f7700 1 -- 172.21.15.5:0/3268321799 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f003410e8f0 msgr2=0x7f0034103bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:58.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:58.632+0000 7f00394f7700 1 --2- 172.21.15.5:0/3268321799 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f003410e8f0 0x7f0034103bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:58.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:58.632+0000 7f00394f7700 1 -- 172.21.15.5:0/3268321799 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0034105480 msgr2=0x7f0034103170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:58.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:58.632+0000 7f00394f7700 1 --2- 172.21.15.5:0/3268321799 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0034105480 0x7f0034103170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:58.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:58.632+0000 7f00394f7700 1 -- 172.21.15.5:0/3268321799 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00341a91e0 con 0x7f0034105dd0 2024-05-09T00:43:58.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:58.632+0000 7f003a4f9700 1 --2- 172.21.15.5:0/3268321799 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f003410e8f0 0x7f0034103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:43:58.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:58.632+0000 7f00394f7700 1 --2- 172.21.15.5:0/3268321799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0034105dd0 0x7f00341036b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0024000c00 tx=0x7f00240047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:43:58.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:58.633+0000 7f002affd700 1 -- 172.21.15.5:0/3268321799 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0024027070 con 0x7f0034105dd0 2024-05-09T00:43:58.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:58.633+0000 7f002affd700 1 -- 172.21.15.5:0/3268321799 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0024003f20 con 0x7f0034105dd0 2024-05-09T00:43:58.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:58.633+0000 7f003bf5c700 1 -- 172.21.15.5:0/3268321799 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00341a9470 con 0x7f0034105dd0 2024-05-09T00:43:58.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:58.633+0000 7f002affd700 1 -- 172.21.15.5:0/3268321799 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0024022cf0 con 0x7f0034105dd0 2024-05-09T00:43:58.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:58.633+0000 7f003bf5c700 1 -- 172.21.15.5:0/3268321799 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00341a98c0 con 0x7f0034105dd0 2024-05-09T00:43:58.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:58.634+0000 7f003bf5c700 1 -- 172.21.15.5:0/3268321799 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00340512c0 con 0x7f0034105dd0 2024-05-09T00:43:58.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:58.635+0000 7f002affd700 1 -- 172.21.15.5:0/3268321799 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f002402a070 con 0x7f0034105dd0 2024-05-09T00:43:58.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:58.636+0000 7f002affd700 1 --2- 172.21.15.5:0/3268321799 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f00200b2880 0x7f00200b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:43:58.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:58.636+0000 7f0039cf8700 1 --2- 172.21.15.5:0/3268321799 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f00200b2880 0x7f00200b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:43:58.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:58.636+0000 7f002affd700 1 -- 172.21.15.5:0/3268321799 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f00240e3280 con 0x7f0034105dd0 2024-05-09T00:43:58.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:58.637+0000 7f0039cf8700 1 --2- 172.21.15.5:0/3268321799 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f00200b2880 0x7f00200b4d90 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0030009550 tx=0x7f00300091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:43:58.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:58.640+0000 7f002affd700 1 -- 172.21.15.5:0/3268321799 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0024013070 con 0x7f0034105dd0 2024-05-09T00:43:58.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:58.904+0000 7f003bf5c700 1 -- 172.21.15.5:0/3268321799 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f003402e7b0 con 0x7f0034105dd0 2024-05-09T00:43:58.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:58.914+0000 7f002affd700 1 -- 172.21.15.5:0/3268321799 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f00240ab180 con 0x7f0034105dd0 2024-05-09T00:43:58.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:58.917+0000 7f003bf5c700 1 -- 172.21.15.5:0/3268321799 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f00200b2880 msgr2=0x7f00200b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:58.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:58.917+0000 7f003bf5c700 1 --2- 172.21.15.5:0/3268321799 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f00200b2880 0x7f00200b4d90 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0030009550 tx=0x7f00300091b0 comp rx=0 tx=0).stop 2024-05-09T00:43:58.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:58.917+0000 7f003bf5c700 1 -- 172.21.15.5:0/3268321799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0034105dd0 msgr2=0x7f00341036b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:43:58.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:58.917+0000 7f003bf5c700 1 --2- 172.21.15.5:0/3268321799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0034105dd0 0x7f00341036b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0024000c00 tx=0x7f00240047b0 comp rx=0 tx=0).stop 2024-05-09T00:43:58.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:58.917+0000 7f003bf5c700 1 -- 172.21.15.5:0/3268321799 shutdown_connections 2024-05-09T00:43:58.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:58.917+0000 7f003bf5c700 1 --2- 172.21.15.5:0/3268321799 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f00200b2880 0x7f00200b4d90 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:58.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:58.917+0000 7f003bf5c700 1 --2- 172.21.15.5:0/3268321799 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f003410e8f0 0x7f0034103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:58.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:58.917+0000 7f003bf5c700 1 --2- 172.21.15.5:0/3268321799 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0034105480 0x7f0034103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:58.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:58.917+0000 7f003bf5c700 1 --2- 172.21.15.5:0/3268321799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0034105dd0 0x7f00341036b0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:43:58.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:58.917+0000 7f003bf5c700 1 -- 172.21.15.5:0/3268321799 >> 172.21.15.5:0/3268321799 conn(0x7f00340fef30 msgr2=0x7f00340ff680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:43:58.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:58.917+0000 7f003bf5c700 1 -- 172.21.15.5:0/3268321799 shutdown_connections 2024-05-09T00:43:58.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:43:58.917+0000 7f003bf5c700 1 -- 172.21.15.5:0/3268321799 wait complete. 2024-05-09T00:43:59.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:58 smithi157 ceph-mon[34294]: pgmap v296: 97 pgs: 97 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-05-09T00:43:59.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:58 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/3268321799' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-09T00:43:59.286 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:58 smithi005 ceph-mon[30365]: pgmap v296: 97 pgs: 97 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-05-09T00:43:59.286 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:58 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/3268321799' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-09T00:43:59.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:58 smithi131 ceph-mon[34547]: pgmap v296: 97 pgs: 97 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-05-09T00:43:59.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:58 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/3268321799' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-09T00:43:59.736 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-05-09T00:44:00.002 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:44:00.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:59 smithi005 ceph-mon[30365]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-09T00:44:00.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:59 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/3268321799' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-09T00:44:00.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:59 smithi005 ceph-mon[30365]: daemon mds.c assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-09T00:44:00.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:59 smithi005 ceph-mon[30365]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-09T00:44:00.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:59 smithi005 ceph-mon[30365]: fsmap cephfs:2 {0=i=up:active,1=c=up:creating} 10 up:standby 2024-05-09T00:44:00.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:43:59 smithi005 ceph-mon[30365]: daemon mds.c is now active in filesystem cephfs as rank 1 2024-05-09T00:44:00.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:59 smithi157 ceph-mon[34294]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-09T00:44:00.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:59 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/3268321799' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-09T00:44:00.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:59 smithi157 ceph-mon[34294]: daemon mds.c assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-09T00:44:00.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:59 smithi157 ceph-mon[34294]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-09T00:44:00.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:59 smithi157 ceph-mon[34294]: fsmap cephfs:2 {0=i=up:active,1=c=up:creating} 10 up:standby 2024-05-09T00:44:00.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:43:59 smithi157 ceph-mon[34294]: daemon mds.c is now active in filesystem cephfs as rank 1 2024-05-09T00:44:00.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:59 smithi131 ceph-mon[34547]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-09T00:44:00.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:59 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/3268321799' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-09T00:44:00.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:59 smithi131 ceph-mon[34547]: daemon mds.c assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-09T00:44:00.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:59 smithi131 ceph-mon[34547]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-09T00:44:00.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:59 smithi131 ceph-mon[34547]: fsmap cephfs:2 {0=i=up:active,1=c=up:creating} 10 up:standby 2024-05-09T00:44:00.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:43:59 smithi131 ceph-mon[34547]: daemon mds.c is now active in filesystem cephfs as rank 1 2024-05-09T00:44:01.151 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:00 smithi005 ceph-mon[30365]: pgmap v297: 97 pgs: 97 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s wr, 3 op/s 2024-05-09T00:44:01.151 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:00 smithi005 ceph-mon[30365]: mds.? [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] up:active 2024-05-09T00:44:01.151 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:00 smithi005 ceph-mon[30365]: daemon mds.b assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-09T00:44:01.151 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:00 smithi005 ceph-mon[30365]: fsmap cephfs:2 {0=i=up:active,1=c=up:active} 10 up:standby 2024-05-09T00:44:01.151 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:00 smithi005 ceph-mon[30365]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:creating} 9 up:standby 2024-05-09T00:44:01.151 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:00 smithi005 ceph-mon[30365]: daemon mds.b is now active in filesystem cephfs as rank 2 2024-05-09T00:44:01.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:00 smithi157 ceph-mon[34294]: pgmap v297: 97 pgs: 97 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s wr, 3 op/s 2024-05-09T00:44:01.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:00 smithi157 ceph-mon[34294]: mds.? [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] up:active 2024-05-09T00:44:01.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:00 smithi157 ceph-mon[34294]: daemon mds.b assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-09T00:44:01.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:00 smithi157 ceph-mon[34294]: fsmap cephfs:2 {0=i=up:active,1=c=up:active} 10 up:standby 2024-05-09T00:44:01.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:00 smithi157 ceph-mon[34294]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:creating} 9 up:standby 2024-05-09T00:44:01.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:00 smithi157 ceph-mon[34294]: daemon mds.b is now active in filesystem cephfs as rank 2 2024-05-09T00:44:01.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:00 smithi131 ceph-mon[34547]: pgmap v297: 97 pgs: 97 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s wr, 3 op/s 2024-05-09T00:44:01.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:00 smithi131 ceph-mon[34547]: mds.? [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] up:active 2024-05-09T00:44:01.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:00 smithi131 ceph-mon[34547]: daemon mds.b assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-09T00:44:01.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:00 smithi131 ceph-mon[34547]: fsmap cephfs:2 {0=i=up:active,1=c=up:active} 10 up:standby 2024-05-09T00:44:01.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:00 smithi131 ceph-mon[34547]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:creating} 9 up:standby 2024-05-09T00:44:01.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:00 smithi131 ceph-mon[34547]: daemon mds.b is now active in filesystem cephfs as rank 2 2024-05-09T00:44:01.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:01.726+0000 7f765d87c700 1 -- 172.21.15.5:0/2036682872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7658109860 msgr2=0x7f76581111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:01.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:01.726+0000 7f765d87c700 1 --2- 172.21.15.5:0/2036682872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7658109860 0x7f76581111c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f765000b0d0 tx=0x7f765000b3e0 comp rx=0 tx=0).stop 2024-05-09T00:44:01.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:01.726+0000 7f765d87c700 1 -- 172.21.15.5:0/2036682872 shutdown_connections 2024-05-09T00:44:01.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:01.726+0000 7f765d87c700 1 --2- 172.21.15.5:0/2036682872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7658109860 0x7f76581111c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:01.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:01.726+0000 7f765d87c700 1 --2- 172.21.15.5:0/2036682872 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f76581045e0 0x7f76581049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:01.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:01.726+0000 7f765d87c700 1 --2- 172.21.15.5:0/2036682872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7658108f10 0x7f7658109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:01.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:01.726+0000 7f765d87c700 1 -- 172.21.15.5:0/2036682872 >> 172.21.15.5:0/2036682872 conn(0x7f76580ffef0 msgr2=0x7f7658102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:44:01.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:01.726+0000 7f765d87c700 1 -- 172.21.15.5:0/2036682872 shutdown_connections 2024-05-09T00:44:01.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:01.726+0000 7f765d87c700 1 -- 172.21.15.5:0/2036682872 wait complete. 2024-05-09T00:44:01.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:01.727+0000 7f765d87c700 1 Processor -- start 2024-05-09T00:44:01.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:01.727+0000 7f765d87c700 1 -- start start 2024-05-09T00:44:01.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:01.727+0000 7f765d87c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76581045e0 0x7f765819ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:44:01.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:01.727+0000 7f765d87c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7658108f10 0x7f765819b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:44:01.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:01.727+0000 7f765d87c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7658109860 0x7f765819f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:44:01.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:01.727+0000 7f765d87c700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7658114490 con 0x7f76581045e0 2024-05-09T00:44:01.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:01.727+0000 7f765d87c700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7658114310 con 0x7f7658109860 2024-05-09T00:44:01.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:01.727+0000 7f765d87c700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7658114610 con 0x7f7658108f10 2024-05-09T00:44:01.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:01.727+0000 7f7656ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76581045e0 0x7f765819ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:44:01.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:01.727+0000 7f7656ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76581045e0 0x7f765819ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60906/0 (socket says 172.21.15.5:60906) 2024-05-09T00:44:01.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:01.727+0000 7f7656ffd700 1 -- 172.21.15.5:0/2422771954 learned_addr learned my addr 172.21.15.5:0/2422771954 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:44:01.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:01.727+0000 7f764ffff700 1 --2- 172.21.15.5:0/2422771954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7658108f10 0x7f765819b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:44:01.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:01.727+0000 7f76577fe700 1 --2- 172.21.15.5:0/2422771954 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7658109860 0x7f765819f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:44:01.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:01.728+0000 7f7656ffd700 1 -- 172.21.15.5:0/2422771954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7658108f10 msgr2=0x7f765819b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:01.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:01.728+0000 7f7656ffd700 1 --2- 172.21.15.5:0/2422771954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7658108f10 0x7f765819b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:01.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:01.728+0000 7f7656ffd700 1 -- 172.21.15.5:0/2422771954 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7658109860 msgr2=0x7f765819f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:01.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:01.728+0000 7f7656ffd700 1 --2- 172.21.15.5:0/2422771954 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7658109860 0x7f765819f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:01.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:01.728+0000 7f7656ffd700 1 -- 172.21.15.5:0/2422771954 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f765819fc70 con 0x7f76581045e0 2024-05-09T00:44:01.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:01.728+0000 7f7656ffd700 1 --2- 172.21.15.5:0/2422771954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76581045e0 0x7f765819ada0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f764800d950 tx=0x7f764800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:44:01.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:01.728+0000 7f765c87a700 1 -- 172.21.15.5:0/2422771954 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7648016070 con 0x7f76581045e0 2024-05-09T00:44:01.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:01.728+0000 7f765d87c700 1 -- 172.21.15.5:0/2422771954 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f765819ff60 con 0x7f76581045e0 2024-05-09T00:44:01.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:01.728+0000 7f765c87a700 1 -- 172.21.15.5:0/2422771954 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f764800fd40 con 0x7f76581045e0 2024-05-09T00:44:01.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:01.728+0000 7f765c87a700 1 -- 172.21.15.5:0/2422771954 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f764800e3b0 con 0x7f76581045e0 2024-05-09T00:44:01.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:01.728+0000 7f765d87c700 1 -- 172.21.15.5:0/2422771954 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76581a7880 con 0x7f76581045e0 2024-05-09T00:44:01.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:01.729+0000 7f765d87c700 1 -- 172.21.15.5:0/2422771954 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76580512c0 con 0x7f76581045e0 2024-05-09T00:44:01.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:01.731+0000 7f765c87a700 1 -- 172.21.15.5:0/2422771954 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f7648019070 con 0x7f76581045e0 2024-05-09T00:44:01.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:01.732+0000 7f765c87a700 1 --2- 172.21.15.5:0/2422771954 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f76440b2880 0x7f76440b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:44:01.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:01.732+0000 7f764ffff700 1 --2- 172.21.15.5:0/2422771954 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f76440b2880 0x7f76440b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:44:01.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:01.732+0000 7f765c87a700 1 -- 172.21.15.5:0/2422771954 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f76480e1f70 con 0x7f76581045e0 2024-05-09T00:44:01.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:01.732+0000 7f764ffff700 1 --2- 172.21.15.5:0/2422771954 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f76440b2880 0x7f76440b4d90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7658103950 tx=0x7f7640009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:44:01.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:01.735+0000 7f765c87a700 1 -- 172.21.15.5:0/2422771954 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f76480a9d40 con 0x7f76581045e0 2024-05-09T00:44:02.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:02.002+0000 7f765d87c700 1 -- 172.21.15.5:0/2422771954 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f765802e5f0 con 0x7f76581045e0 2024-05-09T00:44:02.004 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:01 smithi005 ceph-mon[30365]: mds.? [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] up:active 2024-05-09T00:44:02.004 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:01 smithi005 ceph-mon[30365]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-09T00:44:02.004 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:01 smithi005 ceph-mon[30365]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 9 up:standby 2024-05-09T00:44:02.004 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:01 smithi005 ceph-mon[30365]: fsmap cephfs:4 {0=i=up:active,1=c=up:active,2=b=up:active,3=f=up:creating} 8 up:standby 2024-05-09T00:44:02.004 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:01 smithi005 ceph-mon[30365]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-05-09T00:44:02.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:01 smithi157 ceph-mon[34294]: mds.? [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] up:active 2024-05-09T00:44:02.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:01 smithi157 ceph-mon[34294]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-09T00:44:02.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:01 smithi157 ceph-mon[34294]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 9 up:standby 2024-05-09T00:44:02.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:01 smithi157 ceph-mon[34294]: fsmap cephfs:4 {0=i=up:active,1=c=up:active,2=b=up:active,3=f=up:creating} 8 up:standby 2024-05-09T00:44:02.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:01 smithi157 ceph-mon[34294]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-05-09T00:44:02.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:01 smithi131 ceph-mon[34547]: mds.? [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] up:active 2024-05-09T00:44:02.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:01 smithi131 ceph-mon[34547]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-09T00:44:02.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:01 smithi131 ceph-mon[34547]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 9 up:standby 2024-05-09T00:44:02.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:01 smithi131 ceph-mon[34547]: fsmap cephfs:4 {0=i=up:active,1=c=up:active,2=b=up:active,3=f=up:creating} 8 up:standby 2024-05-09T00:44:02.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:01 smithi131 ceph-mon[34547]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-05-09T00:44:02.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:02.949+0000 7f765c87a700 1 -- 172.21.15.5:0/2422771954 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v26) ==== 121+0+0 (secure 0 0 0) 0x7f76480a9640 con 0x7f76581045e0 2024-05-09T00:44:02.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:02.952+0000 7f765d87c700 1 -- 172.21.15.5:0/2422771954 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f76440b2880 msgr2=0x7f76440b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:02.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:02.952+0000 7f765d87c700 1 --2- 172.21.15.5:0/2422771954 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f76440b2880 0x7f76440b4d90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7658103950 tx=0x7f7640009260 comp rx=0 tx=0).stop 2024-05-09T00:44:02.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:02.952+0000 7f765d87c700 1 -- 172.21.15.5:0/2422771954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76581045e0 msgr2=0x7f765819ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:02.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:02.952+0000 7f765d87c700 1 --2- 172.21.15.5:0/2422771954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76581045e0 0x7f765819ada0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f764800d950 tx=0x7f764800dc60 comp rx=0 tx=0).stop 2024-05-09T00:44:02.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:02.953+0000 7f765d87c700 1 -- 172.21.15.5:0/2422771954 shutdown_connections 2024-05-09T00:44:02.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:02.953+0000 7f765d87c700 1 --2- 172.21.15.5:0/2422771954 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f76440b2880 0x7f76440b4d90 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:02.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:02.953+0000 7f765d87c700 1 --2- 172.21.15.5:0/2422771954 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7658109860 0x7f765819f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:02.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:02.953+0000 7f765d87c700 1 --2- 172.21.15.5:0/2422771954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76581045e0 0x7f765819ada0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:02.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:02.953+0000 7f765d87c700 1 --2- 172.21.15.5:0/2422771954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7658108f10 0x7f765819b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:02.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:02.953+0000 7f765d87c700 1 -- 172.21.15.5:0/2422771954 >> 172.21.15.5:0/2422771954 conn(0x7f76580ffef0 msgr2=0x7f7658108680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:44:02.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:02.953+0000 7f765d87c700 1 -- 172.21.15.5:0/2422771954 shutdown_connections 2024-05-09T00:44:02.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:02.953+0000 7f765d87c700 1 -- 172.21.15.5:0/2422771954 wait complete. 2024-05-09T00:44:03.257 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:02 smithi131 ceph-mon[34547]: pgmap v298: 97 pgs: 97 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 1 op/s 2024-05-09T00:44:03.314 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:02 smithi131 ceph-mon[34547]: mds.? [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] up:active 2024-05-09T00:44:03.314 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:02 smithi131 ceph-mon[34547]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-09T00:44:03.314 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:02 smithi131 ceph-mon[34547]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-09T00:44:03.314 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:02 smithi131 ceph-mon[34547]: Cluster is now healthy 2024-05-09T00:44:03.314 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:02 smithi131 ceph-mon[34547]: fsmap cephfs:4 {0=i=up:active,1=c=up:active,2=b=up:active,3=f=up:active} 8 up:standby 2024-05-09T00:44:03.314 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:02 smithi131 ceph-mon[34547]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-05-09T00:44:03.315 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:02 smithi131 ceph-mon[34547]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-05-09T00:44:03.315 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:02 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/2422771954' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-09T00:44:03.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:02 smithi005 ceph-mon[30365]: pgmap v298: 97 pgs: 97 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 1 op/s 2024-05-09T00:44:03.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:02 smithi005 ceph-mon[30365]: mds.? [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] up:active 2024-05-09T00:44:03.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:02 smithi005 ceph-mon[30365]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-09T00:44:03.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:02 smithi005 ceph-mon[30365]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-09T00:44:03.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:02 smithi005 ceph-mon[30365]: Cluster is now healthy 2024-05-09T00:44:03.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:02 smithi005 ceph-mon[30365]: fsmap cephfs:4 {0=i=up:active,1=c=up:active,2=b=up:active,3=f=up:active} 8 up:standby 2024-05-09T00:44:03.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:02 smithi005 ceph-mon[30365]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-05-09T00:44:03.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:02 smithi005 ceph-mon[30365]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-05-09T00:44:03.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:02 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/2422771954' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-09T00:44:03.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:02 smithi157 ceph-mon[34294]: pgmap v298: 97 pgs: 97 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 1 op/s 2024-05-09T00:44:03.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:02 smithi157 ceph-mon[34294]: mds.? [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] up:active 2024-05-09T00:44:03.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:02 smithi157 ceph-mon[34294]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-09T00:44:03.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:02 smithi157 ceph-mon[34294]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-09T00:44:03.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:02 smithi157 ceph-mon[34294]: Cluster is now healthy 2024-05-09T00:44:03.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:02 smithi157 ceph-mon[34294]: fsmap cephfs:4 {0=i=up:active,1=c=up:active,2=b=up:active,3=f=up:active} 8 up:standby 2024-05-09T00:44:03.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:02 smithi157 ceph-mon[34294]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-05-09T00:44:03.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:02 smithi157 ceph-mon[34294]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-05-09T00:44:03.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:02 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/2422771954' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-09T00:44:03.779 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-05-09T00:44:04.040 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:44:04.040 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:03 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/2422771954' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-09T00:44:04.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:03 smithi005 ceph-mon[30365]: mds.? [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] up:active 2024-05-09T00:44:04.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:03 smithi005 ceph-mon[30365]: fsmap cephfs:5 7 up:standby 5 up:active 2024-05-09T00:44:04.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:03 smithi005 ceph-mon[30365]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T00:44:04.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:03 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/2422771954' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-09T00:44:04.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:03 smithi131 ceph-mon[34547]: mds.? [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] up:active 2024-05-09T00:44:04.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:03 smithi131 ceph-mon[34547]: fsmap cephfs:5 7 up:standby 5 up:active 2024-05-09T00:44:04.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:03 smithi131 ceph-mon[34547]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T00:44:04.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:03 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/2422771954' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-09T00:44:04.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:03 smithi157 ceph-mon[34294]: mds.? [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] up:active 2024-05-09T00:44:04.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:03 smithi157 ceph-mon[34294]: fsmap cephfs:5 7 up:standby 5 up:active 2024-05-09T00:44:04.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:03 smithi157 ceph-mon[34294]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T00:44:05.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:04 smithi005 ceph-mon[30365]: pgmap v299: 97 pgs: 97 active+clean; 583 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 2.4 KiB/s wr, 10 op/s 2024-05-09T00:44:05.337 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:04 smithi157 ceph-mon[34294]: pgmap v299: 97 pgs: 97 active+clean; 583 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 2.4 KiB/s wr, 10 op/s 2024-05-09T00:44:05.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:04 smithi131 ceph-mon[34547]: pgmap v299: 97 pgs: 97 active+clean; 583 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 2.4 KiB/s wr, 10 op/s 2024-05-09T00:44:05.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:05.758+0000 7f1e4447e700 1 -- 172.21.15.5:0/529790185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e3c105540 msgr2=0x7f1e3c105950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:05.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:05.758+0000 7f1e4447e700 1 --2- 172.21.15.5:0/529790185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e3c105540 0x7f1e3c105950 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c009990 tx=0x7f1e2c009ca0 comp rx=0 tx=0).stop 2024-05-09T00:44:05.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:05.758+0000 7f1e4447e700 1 -- 172.21.15.5:0/529790185 shutdown_connections 2024-05-09T00:44:05.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:05.758+0000 7f1e4447e700 1 --2- 172.21.15.5:0/529790185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e3c10e920 0x7f1e3c110de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:05.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:05.758+0000 7f1e4447e700 1 --2- 172.21.15.5:0/529790185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e3c105540 0x7f1e3c105950 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:05.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:05.758+0000 7f1e4447e700 1 --2- 172.21.15.5:0/529790185 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e3c105e90 0x7f1e3c10e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:05.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:05.758+0000 7f1e4447e700 1 -- 172.21.15.5:0/529790185 >> 172.21.15.5:0/529790185 conn(0x7f1e3c0fef30 msgr2=0x7f1e3c101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:44:05.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:05.758+0000 7f1e4447e700 1 -- 172.21.15.5:0/529790185 shutdown_connections 2024-05-09T00:44:05.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:05.758+0000 7f1e4447e700 1 -- 172.21.15.5:0/529790185 wait complete. 2024-05-09T00:44:05.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:05.759+0000 7f1e4447e700 1 Processor -- start 2024-05-09T00:44:05.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:05.759+0000 7f1e4447e700 1 -- start start 2024-05-09T00:44:05.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:05.759+0000 7f1e4447e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e3c105540 0x7f1e3c19ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:44:05.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:05.759+0000 7f1e4447e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e3c105e90 0x7f1e3c19b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:44:05.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:05.759+0000 7f1e4447e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e3c10e920 0x7f1e3c19f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:44:05.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:05.759+0000 7f1e4447e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1e3c1144d0 con 0x7f1e3c105e90 2024-05-09T00:44:05.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:05.759+0000 7f1e4447e700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1e3c114350 con 0x7f1e3c105540 2024-05-09T00:44:05.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:05.759+0000 7f1e4447e700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1e3c114650 con 0x7f1e3c10e920 2024-05-09T00:44:05.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:05.759+0000 7f1e41a19700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e3c105e90 0x7f1e3c19b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:44:05.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:05.759+0000 7f1e41a19700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e3c105e90 0x7f1e3c19b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46764/0 (socket says 172.21.15.5:46764) 2024-05-09T00:44:05.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:05.760+0000 7f1e41a19700 1 -- 172.21.15.5:0/3981900154 learned_addr learned my addr 172.21.15.5:0/3981900154 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:44:05.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:05.760+0000 7f1e42a1b700 1 --2- 172.21.15.5:0/3981900154 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e3c10e920 0x7f1e3c19f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:44:05.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:05.760+0000 7f1e4221a700 1 --2- 172.21.15.5:0/3981900154 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e3c105540 0x7f1e3c19ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:44:05.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:05.760+0000 7f1e41a19700 1 -- 172.21.15.5:0/3981900154 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e3c10e920 msgr2=0x7f1e3c19f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:05.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:05.760+0000 7f1e41a19700 1 --2- 172.21.15.5:0/3981900154 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e3c10e920 0x7f1e3c19f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:05.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:05.760+0000 7f1e41a19700 1 -- 172.21.15.5:0/3981900154 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e3c105540 msgr2=0x7f1e3c19ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:05.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:05.760+0000 7f1e41a19700 1 --2- 172.21.15.5:0/3981900154 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e3c105540 0x7f1e3c19ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:05.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:05.760+0000 7f1e41a19700 1 -- 172.21.15.5:0/3981900154 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e3c19fc40 con 0x7f1e3c105e90 2024-05-09T00:44:05.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:05.760+0000 7f1e4221a700 1 --2- 172.21.15.5:0/3981900154 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e3c105540 0x7f1e3c19ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:44:05.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:05.760+0000 7f1e41a19700 1 --2- 172.21.15.5:0/3981900154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e3c105e90 0x7f1e3c19b3c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1e3800c9b0 tx=0x7f1e3800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:44:05.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:05.760+0000 7f1e337fe700 1 -- 172.21.15.5:0/3981900154 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e38007b00 con 0x7f1e3c105e90 2024-05-09T00:44:05.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:05.760+0000 7f1e337fe700 1 -- 172.21.15.5:0/3981900154 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e38007ca0 con 0x7f1e3c105e90 2024-05-09T00:44:05.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:05.760+0000 7f1e4447e700 1 -- 172.21.15.5:0/3981900154 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e3c19ff30 con 0x7f1e3c105e90 2024-05-09T00:44:05.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:05.760+0000 7f1e337fe700 1 -- 172.21.15.5:0/3981900154 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e38018660 con 0x7f1e3c105e90 2024-05-09T00:44:05.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:05.760+0000 7f1e4447e700 1 -- 172.21.15.5:0/3981900154 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e3c1a7900 con 0x7f1e3c105e90 2024-05-09T00:44:05.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:05.761+0000 7f1e4447e700 1 -- 172.21.15.5:0/3981900154 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e3c0512c0 con 0x7f1e3c105e90 2024-05-09T00:44:05.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:05.764+0000 7f1e337fe700 1 -- 172.21.15.5:0/3981900154 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f1e3800f450 con 0x7f1e3c105e90 2024-05-09T00:44:05.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:05.765+0000 7f1e337fe700 1 --2- 172.21.15.5:0/3981900154 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f1e280b2900 0x7f1e280b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:44:05.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:05.765+0000 7f1e4221a700 1 --2- 172.21.15.5:0/3981900154 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f1e280b2900 0x7f1e280b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:44:05.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:05.765+0000 7f1e337fe700 1 -- 172.21.15.5:0/3981900154 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f1e380e2380 con 0x7f1e3c105e90 2024-05-09T00:44:05.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:05.765+0000 7f1e4221a700 1 --2- 172.21.15.5:0/3981900154 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f1e280b2900 0x7f1e280b4e10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c003800 tx=0x7f1e2c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:44:05.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:05.768+0000 7f1e337fe700 1 -- 172.21.15.5:0/3981900154 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1e38014030 con 0x7f1e3c105e90 2024-05-09T00:44:06.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:06.032+0000 7f1e4447e700 1 -- 172.21.15.5:0/3981900154 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f1e3c02e5f0 con 0x7f1e3c105e90 2024-05-09T00:44:06.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:06.965+0000 7f1e337fe700 1 -- 172.21.15.5:0/3981900154 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v28) ==== 115+0+0 (secure 0 0 0) 0x7f1e380aa260 con 0x7f1e3c105e90 2024-05-09T00:44:06.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:06.968+0000 7f1e4447e700 1 -- 172.21.15.5:0/3981900154 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f1e280b2900 msgr2=0x7f1e280b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:06.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:06.968+0000 7f1e4447e700 1 --2- 172.21.15.5:0/3981900154 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f1e280b2900 0x7f1e280b4e10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c003800 tx=0x7f1e2c01a040 comp rx=0 tx=0).stop 2024-05-09T00:44:06.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:06.968+0000 7f1e4447e700 1 -- 172.21.15.5:0/3981900154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e3c105e90 msgr2=0x7f1e3c19b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:06.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:06.968+0000 7f1e4447e700 1 --2- 172.21.15.5:0/3981900154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e3c105e90 0x7f1e3c19b3c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1e3800c9b0 tx=0x7f1e3800ccc0 comp rx=0 tx=0).stop 2024-05-09T00:44:06.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:06.968+0000 7f1e4447e700 1 -- 172.21.15.5:0/3981900154 shutdown_connections 2024-05-09T00:44:06.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:06.968+0000 7f1e4447e700 1 --2- 172.21.15.5:0/3981900154 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f1e280b2900 0x7f1e280b4e10 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:06.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:06.968+0000 7f1e4447e700 1 --2- 172.21.15.5:0/3981900154 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e3c10e920 0x7f1e3c19f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:06.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:06.968+0000 7f1e4447e700 1 --2- 172.21.15.5:0/3981900154 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e3c105540 0x7f1e3c19ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:06.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:06.968+0000 7f1e4447e700 1 --2- 172.21.15.5:0/3981900154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e3c105e90 0x7f1e3c19b3c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:06.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:06.968+0000 7f1e4447e700 1 -- 172.21.15.5:0/3981900154 >> 172.21.15.5:0/3981900154 conn(0x7f1e3c0fef30 msgr2=0x7f1e3c100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:44:06.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:06.968+0000 7f1e4447e700 1 -- 172.21.15.5:0/3981900154 shutdown_connections 2024-05-09T00:44:06.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:06.968+0000 7f1e4447e700 1 -- 172.21.15.5:0/3981900154 wait complete. 2024-05-09T00:44:07.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:06 smithi005 ceph-mon[30365]: pgmap v300: 97 pgs: 97 active+clean; 584 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4.0 KiB/s wr, 15 op/s 2024-05-09T00:44:07.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:06 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/3981900154' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-09T00:44:07.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:06 smithi131 ceph-mon[34547]: pgmap v300: 97 pgs: 97 active+clean; 584 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4.0 KiB/s wr, 15 op/s 2024-05-09T00:44:07.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:06 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/3981900154' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-09T00:44:07.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:06 smithi157 ceph-mon[34294]: pgmap v300: 97 pgs: 97 active+clean; 584 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4.0 KiB/s wr, 15 op/s 2024-05-09T00:44:07.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:06 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/3981900154' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-09T00:44:07.748 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-05-09T00:44:07.748 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-05-09T00:44:08.008 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:44:08.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:07 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/3981900154' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-09T00:44:08.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:07 smithi005 ceph-mon[30365]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T00:44:08.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:07 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/3981900154' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-09T00:44:08.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:07 smithi131 ceph-mon[34547]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T00:44:08.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:07 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/3981900154' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-09T00:44:08.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:07 smithi157 ceph-mon[34294]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T00:44:09.255 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:08 smithi005 ceph-mon[30365]: pgmap v301: 97 pgs: 97 active+clean; 584 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 4.0 KiB/s wr, 17 op/s 2024-05-09T00:44:09.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:08 smithi131 ceph-mon[34547]: pgmap v301: 97 pgs: 97 active+clean; 584 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 4.0 KiB/s wr, 17 op/s 2024-05-09T00:44:09.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:08 smithi157 ceph-mon[34294]: pgmap v301: 97 pgs: 97 active+clean; 584 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 4.0 KiB/s wr, 17 op/s 2024-05-09T00:44:09.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:09.845+0000 7f12912db700 1 -- 172.21.15.5:0/171484118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f128c1034c0 msgr2=0x7f128c1038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:09.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:09.845+0000 7f12912db700 1 --2- 172.21.15.5:0/171484118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f128c1034c0 0x7f128c1038d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f127c0099c0 tx=0x7f127c009cd0 comp rx=0 tx=0).stop 2024-05-09T00:44:09.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:09.845+0000 7f12912db700 1 -- 172.21.15.5:0/171484118 shutdown_connections 2024-05-09T00:44:09.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:09.845+0000 7f12912db700 1 --2- 172.21.15.5:0/171484118 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f128c108740 0x7f128c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:09.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:09.845+0000 7f12912db700 1 --2- 172.21.15.5:0/171484118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f128c1034c0 0x7f128c1038d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:09.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:09.845+0000 7f12912db700 1 --2- 172.21.15.5:0/171484118 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f128c107df0 0x7f128c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:09.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:09.845+0000 7f12912db700 1 -- 172.21.15.5:0/171484118 >> 172.21.15.5:0/171484118 conn(0x7f128c0fedc0 msgr2=0x7f128c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:44:09.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:09.845+0000 7f12912db700 1 -- 172.21.15.5:0/171484118 shutdown_connections 2024-05-09T00:44:09.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:09.845+0000 7f12912db700 1 -- 172.21.15.5:0/171484118 wait complete. 2024-05-09T00:44:09.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:09.846+0000 7f12912db700 1 Processor -- start 2024-05-09T00:44:09.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:09.846+0000 7f12912db700 1 -- start start 2024-05-09T00:44:09.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:09.847+0000 7f12912db700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f128c1034c0 0x7f128c19bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:44:09.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:09.847+0000 7f12912db700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f128c107df0 0x7f128c19c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:44:09.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:09.847+0000 7f12912db700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f128c108740 0x7f128c19c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:44:09.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:09.847+0000 7f12912db700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f128c113390 con 0x7f128c108740 2024-05-09T00:44:09.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:09.847+0000 7f12912db700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f128c113210 con 0x7f128c1034c0 2024-05-09T00:44:09.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:09.847+0000 7f12912db700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f128c113510 con 0x7f128c107df0 2024-05-09T00:44:09.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:09.847+0000 7f128b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f128c108740 0x7f128c19c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:44:09.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:09.847+0000 7f128b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f128c108740 0x7f128c19c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46786/0 (socket says 172.21.15.5:46786) 2024-05-09T00:44:09.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:09.847+0000 7f128b7fe700 1 -- 172.21.15.5:0/1404983848 learned_addr learned my addr 172.21.15.5:0/1404983848 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:44:09.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:09.847+0000 7f128a7fc700 1 --2- 172.21.15.5:0/1404983848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f128c107df0 0x7f128c19c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:44:09.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:09.847+0000 7f128affd700 1 --2- 172.21.15.5:0/1404983848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f128c1034c0 0x7f128c19bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:44:09.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:09.847+0000 7f128b7fe700 1 -- 172.21.15.5:0/1404983848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f128c107df0 msgr2=0x7f128c19c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:09.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:09.847+0000 7f128b7fe700 1 --2- 172.21.15.5:0/1404983848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f128c107df0 0x7f128c19c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:09.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:09.847+0000 7f128b7fe700 1 -- 172.21.15.5:0/1404983848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f128c1034c0 msgr2=0x7f128c19bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:09.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:09.847+0000 7f128b7fe700 1 --2- 172.21.15.5:0/1404983848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f128c1034c0 0x7f128c19bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:09.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:09.847+0000 7f128b7fe700 1 -- 172.21.15.5:0/1404983848 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f128c199a60 con 0x7f128c108740 2024-05-09T00:44:09.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:09.847+0000 7f128affd700 1 --2- 172.21.15.5:0/1404983848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f128c1034c0 0x7f128c19bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:44:09.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:09.847+0000 7f128b7fe700 1 --2- 172.21.15.5:0/1404983848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f128c108740 0x7f128c19c710 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1284007ee0 tx=0x7f128400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:44:09.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:09.848+0000 7f1273fff700 1 -- 172.21.15.5:0/1404983848 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f128400dbd0 con 0x7f128c108740 2024-05-09T00:44:09.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:09.848+0000 7f12912db700 1 -- 172.21.15.5:0/1404983848 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f128c199d50 con 0x7f128c108740 2024-05-09T00:44:09.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:09.848+0000 7f1273fff700 1 -- 172.21.15.5:0/1404983848 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f128400dd70 con 0x7f128c108740 2024-05-09T00:44:09.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:09.848+0000 7f1273fff700 1 -- 172.21.15.5:0/1404983848 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1284005990 con 0x7f128c108740 2024-05-09T00:44:09.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:09.848+0000 7f12912db700 1 -- 172.21.15.5:0/1404983848 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f128c19a2e0 con 0x7f128c108740 2024-05-09T00:44:09.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:09.849+0000 7f12912db700 1 -- 172.21.15.5:0/1404983848 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f128c0512c0 con 0x7f128c108740 2024-05-09T00:44:09.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:09.851+0000 7f1273fff700 1 -- 172.21.15.5:0/1404983848 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f128401a020 con 0x7f128c108740 2024-05-09T00:44:09.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:09.851+0000 7f1273fff700 1 --2- 172.21.15.5:0/1404983848 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f12740b2880 0x7f12740b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:44:09.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:09.852+0000 7f128affd700 1 --2- 172.21.15.5:0/1404983848 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f12740b2880 0x7f12740b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:44:09.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:09.852+0000 7f1273fff700 1 -- 172.21.15.5:0/1404983848 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f12840e1e30 con 0x7f128c108740 2024-05-09T00:44:09.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:09.852+0000 7f128affd700 1 --2- 172.21.15.5:0/1404983848 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f12740b2880 0x7f12740b4d90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f127c000c00 tx=0x7f127c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:44:09.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:09.855+0000 7f1273fff700 1 -- 172.21.15.5:0/1404983848 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1284010070 con 0x7f128c108740 2024-05-09T00:44:10.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:10.102+0000 7f12912db700 1 -- 172.21.15.5:0/1404983848 --> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f128c063c00 con 0x7f12740b2880 2024-05-09T00:44:10.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:10.134+0000 7f1273fff700 1 -- 172.21.15.5:0/1404983848 <== mgr.14152 v2:172.21.15.5:6800/91856593 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f128c063c00 con 0x7f12740b2880 2024-05-09T00:44:10.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:10.136+0000 7f12912db700 1 -- 172.21.15.5:0/1404983848 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f12740b2880 msgr2=0x7f12740b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:10.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:10.136+0000 7f12912db700 1 --2- 172.21.15.5:0/1404983848 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f12740b2880 0x7f12740b4d90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f127c000c00 tx=0x7f127c01a040 comp rx=0 tx=0).stop 2024-05-09T00:44:10.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:10.136+0000 7f12912db700 1 -- 172.21.15.5:0/1404983848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f128c108740 msgr2=0x7f128c19c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:10.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:10.136+0000 7f12912db700 1 --2- 172.21.15.5:0/1404983848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f128c108740 0x7f128c19c710 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1284007ee0 tx=0x7f128400d3b0 comp rx=0 tx=0).stop 2024-05-09T00:44:10.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:10.136+0000 7f12912db700 1 -- 172.21.15.5:0/1404983848 shutdown_connections 2024-05-09T00:44:10.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:10.136+0000 7f12912db700 1 --2- 172.21.15.5:0/1404983848 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f12740b2880 0x7f12740b4d90 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:10.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:10.136+0000 7f12912db700 1 --2- 172.21.15.5:0/1404983848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f128c108740 0x7f128c19c710 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:10.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:10.136+0000 7f12912db700 1 --2- 172.21.15.5:0/1404983848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f128c1034c0 0x7f128c19bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:10.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:10.136+0000 7f12912db700 1 --2- 172.21.15.5:0/1404983848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f128c107df0 0x7f128c19c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:10.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:10.136+0000 7f12912db700 1 -- 172.21.15.5:0/1404983848 >> 172.21.15.5:0/1404983848 conn(0x7f128c0fedc0 msgr2=0x7f128c1075d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:44:10.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:10.136+0000 7f12912db700 1 -- 172.21.15.5:0/1404983848 shutdown_connections 2024-05-09T00:44:10.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:10.136+0000 7f12912db700 1 -- 172.21.15.5:0/1404983848 wait complete. 2024-05-09T00:44:10.451 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:44:10 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:44:10.107+0000 7f7612d00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T00:44:10.451 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:44:10 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:44:10.107+0000 7f7612d00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T00:44:10.451 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:44:10 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:44:10.107+0000 7f7612d00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T00:44:10.451 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:44:10 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:44:10.107+0000 7f7612d00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T00:44:10.451 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 00:44:10 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T00:44:10.107+0000 7f7612d00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T00:44:10.898 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-05-09T00:44:11.159 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:44:11.273 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:10 smithi005 ceph-mon[30365]: pgmap v302: 97 pgs: 97 active+clean; 584 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 4.0 KiB/s wr, 20 op/s 2024-05-09T00:44:11.274 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:10 smithi005 ceph-mon[30365]: from='client.14685 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:44:11.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:10 smithi131 ceph-mon[34547]: pgmap v302: 97 pgs: 97 active+clean; 584 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 4.0 KiB/s wr, 20 op/s 2024-05-09T00:44:11.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:10 smithi131 ceph-mon[34547]: from='client.14685 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:44:11.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:10 smithi157 ceph-mon[34294]: pgmap v302: 97 pgs: 97 active+clean; 584 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 4.0 KiB/s wr, 20 op/s 2024-05-09T00:44:11.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:10 smithi157 ceph-mon[34294]: from='client.14685 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:44:12.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:12.910+0000 7f42d501b700 1 -- 172.21.15.5:0/3811092958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42d0109880 msgr2=0x7f42d01111e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:12.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:12.910+0000 7f42d501b700 1 --2- 172.21.15.5:0/3811092958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42d0109880 0x7f42d01111e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f42c80099c0 tx=0x7f42c8009cd0 comp rx=0 tx=0).stop 2024-05-09T00:44:12.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:12.911+0000 7f42d501b700 1 -- 172.21.15.5:0/3811092958 shutdown_connections 2024-05-09T00:44:12.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:12.911+0000 7f42d501b700 1 --2- 172.21.15.5:0/3811092958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42d0109880 0x7f42d01111e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:12.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:12.911+0000 7f42d501b700 1 --2- 172.21.15.5:0/3811092958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42d0104650 0x7f42d0104a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:12.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:12.911+0000 7f42d501b700 1 --2- 172.21.15.5:0/3811092958 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42d0108f30 0x7f42d0109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:12.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:12.911+0000 7f42d501b700 1 -- 172.21.15.5:0/3811092958 >> 172.21.15.5:0/3811092958 conn(0x7f42d0079f00 msgr2=0x7f42d007a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:44:12.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:12.911+0000 7f42d501b700 1 -- 172.21.15.5:0/3811092958 shutdown_connections 2024-05-09T00:44:12.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:12.911+0000 7f42d501b700 1 -- 172.21.15.5:0/3811092958 wait complete. 2024-05-09T00:44:12.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:12.912+0000 7f42d501b700 1 Processor -- start 2024-05-09T00:44:12.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:12.912+0000 7f42d501b700 1 -- start start 2024-05-09T00:44:12.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:12.912+0000 7f42d501b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42d0104650 0x7f42d0075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:44:12.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:12.912+0000 7f42d501b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42d0108f30 0x7f42d0076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:44:12.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:12.912+0000 7f42d501b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42d0109880 0x7f42d0076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:44:12.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:12.912+0000 7f42d501b700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f42d0114520 con 0x7f42d0109880 2024-05-09T00:44:12.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:12.912+0000 7f42d501b700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f42d01143a0 con 0x7f42d0108f30 2024-05-09T00:44:12.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:12.912+0000 7f42d501b700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f42d01146a0 con 0x7f42d0104650 2024-05-09T00:44:12.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:12.912+0000 7f42cf59e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42d0109880 0x7f42d0076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:44:12.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:12.912+0000 7f42cf59e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42d0109880 0x7f42d0076540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46832/0 (socket says 172.21.15.5:46832) 2024-05-09T00:44:12.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:12.912+0000 7f42cf59e700 1 -- 172.21.15.5:0/237796023 learned_addr learned my addr 172.21.15.5:0/237796023 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:44:12.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:12.912+0000 7f42ced9d700 1 --2- 172.21.15.5:0/237796023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42d0104650 0x7f42d0075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:44:12.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:12.912+0000 7f42ce59c700 1 --2- 172.21.15.5:0/237796023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42d0108f30 0x7f42d0076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:44:12.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:12.913+0000 7f42cf59e700 1 -- 172.21.15.5:0/237796023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42d0104650 msgr2=0x7f42d0075ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:12.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:12.913+0000 7f42cf59e700 1 --2- 172.21.15.5:0/237796023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42d0104650 0x7f42d0075ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:12.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:12.913+0000 7f42cf59e700 1 -- 172.21.15.5:0/237796023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42d0108f30 msgr2=0x7f42d0076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:12.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:12.913+0000 7f42cf59e700 1 --2- 172.21.15.5:0/237796023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42d0108f30 0x7f42d0076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:12.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:12.913+0000 7f42cf59e700 1 -- 172.21.15.5:0/237796023 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42d01a9140 con 0x7f42d0109880 2024-05-09T00:44:12.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:12.913+0000 7f42cf59e700 1 --2- 172.21.15.5:0/237796023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42d0109880 0x7f42d0076540 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f42c8009990 tx=0x7f42c80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:44:12.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:12.913+0000 7f42b7fff700 1 -- 172.21.15.5:0/237796023 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c8027070 con 0x7f42d0109880 2024-05-09T00:44:12.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:12.913+0000 7f42b7fff700 1 -- 172.21.15.5:0/237796023 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42c8004020 con 0x7f42d0109880 2024-05-09T00:44:12.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:12.913+0000 7f42d501b700 1 -- 172.21.15.5:0/237796023 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42d01a93d0 con 0x7f42d0109880 2024-05-09T00:44:12.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:12.913+0000 7f42b7fff700 1 -- 172.21.15.5:0/237796023 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c8022ce0 con 0x7f42d0109880 2024-05-09T00:44:12.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:12.913+0000 7f42d501b700 1 -- 172.21.15.5:0/237796023 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42d01a9820 con 0x7f42d0109880 2024-05-09T00:44:12.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:12.914+0000 7f42d501b700 1 -- 172.21.15.5:0/237796023 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42d00512c0 con 0x7f42d0109880 2024-05-09T00:44:12.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:12.916+0000 7f42b7fff700 1 -- 172.21.15.5:0/237796023 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f42c802a070 con 0x7f42d0109880 2024-05-09T00:44:12.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:12.917+0000 7f42b7fff700 1 --2- 172.21.15.5:0/237796023 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f42b80b2910 0x7f42b80b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:44:12.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:12.917+0000 7f42ced9d700 1 --2- 172.21.15.5:0/237796023 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f42b80b2910 0x7f42b80b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:44:12.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:12.917+0000 7f42b7fff700 1 -- 172.21.15.5:0/237796023 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f42c80e3310 con 0x7f42d0109880 2024-05-09T00:44:12.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:12.917+0000 7f42ced9d700 1 --2- 172.21.15.5:0/237796023 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f42b80b2910 0x7f42b80b4e20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f42c000f440 tx=0x7f42c00079e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:44:12.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:12.922+0000 7f42b7fff700 1 -- 172.21.15.5:0/237796023 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f42c8013070 con 0x7f42d0109880 2024-05-09T00:44:13.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:13.172+0000 7f42d501b700 1 -- 172.21.15.5:0/237796023 --> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f42d0063c00 con 0x7f42b80b2910 2024-05-09T00:44:13.173 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:12 smithi005 ceph-mon[30365]: pgmap v303: 97 pgs: 97 active+clean; 584 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 3.9 KiB/s wr, 19 op/s 2024-05-09T00:44:13.174 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:12 smithi005 ceph-mon[30365]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-05-09T00:44:13.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:13.205+0000 7f42b7fff700 1 -- 172.21.15.5:0/237796023 <== mgr.14152 v2:172.21.15.5:6800/91856593 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f42d0063c00 con 0x7f42b80b2910 2024-05-09T00:44:13.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:13.208+0000 7f42d501b700 1 -- 172.21.15.5:0/237796023 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f42b80b2910 msgr2=0x7f42b80b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:13.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:13.208+0000 7f42d501b700 1 --2- 172.21.15.5:0/237796023 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f42b80b2910 0x7f42b80b4e20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f42c000f440 tx=0x7f42c00079e0 comp rx=0 tx=0).stop 2024-05-09T00:44:13.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:13.208+0000 7f42d501b700 1 -- 172.21.15.5:0/237796023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42d0109880 msgr2=0x7f42d0076540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:13.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:13.208+0000 7f42d501b700 1 --2- 172.21.15.5:0/237796023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42d0109880 0x7f42d0076540 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f42c8009990 tx=0x7f42c80047b0 comp rx=0 tx=0).stop 2024-05-09T00:44:13.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:13.208+0000 7f42d501b700 1 -- 172.21.15.5:0/237796023 shutdown_connections 2024-05-09T00:44:13.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:13.208+0000 7f42d501b700 1 --2- 172.21.15.5:0/237796023 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f42b80b2910 0x7f42b80b4e20 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:13.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:13.208+0000 7f42d501b700 1 --2- 172.21.15.5:0/237796023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42d0109880 0x7f42d0076540 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:13.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:13.208+0000 7f42d501b700 1 --2- 172.21.15.5:0/237796023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42d0104650 0x7f42d0075ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:13.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:13.208+0000 7f42d501b700 1 --2- 172.21.15.5:0/237796023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42d0108f30 0x7f42d0076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:13.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:13.208+0000 7f42d501b700 1 -- 172.21.15.5:0/237796023 >> 172.21.15.5:0/237796023 conn(0x7f42d0079f00 msgr2=0x7f42d0101f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:44:13.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:13.208+0000 7f42d501b700 1 -- 172.21.15.5:0/237796023 shutdown_connections 2024-05-09T00:44:13.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:13.208+0000 7f42d501b700 1 -- 172.21.15.5:0/237796023 wait complete. 2024-05-09T00:44:13.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:12 smithi131 ceph-mon[34547]: pgmap v303: 97 pgs: 97 active+clean; 584 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 3.9 KiB/s wr, 19 op/s 2024-05-09T00:44:13.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:12 smithi131 ceph-mon[34547]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-05-09T00:44:13.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:12 smithi157 ceph-mon[34294]: pgmap v303: 97 pgs: 97 active+clean; 584 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 3.9 KiB/s wr, 19 op/s 2024-05-09T00:44:13.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:12 smithi157 ceph-mon[34294]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-05-09T00:44:13.973 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-05-09T00:44:14.228 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:44:14.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:13 smithi005 ceph-mon[30365]: from='client.14697 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:44:14.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:13 smithi131 ceph-mon[34547]: from='client.14697 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:44:14.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:13 smithi157 ceph-mon[34294]: from='client.14697 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:44:15.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:14 smithi005 ceph-mon[30365]: pgmap v304: 97 pgs: 97 active+clean; 592 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.0 KiB/s wr, 24 op/s 2024-05-09T00:44:15.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:14 smithi131 ceph-mon[34547]: pgmap v304: 97 pgs: 97 active+clean; 592 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.0 KiB/s wr, 24 op/s 2024-05-09T00:44:15.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:14 smithi157 ceph-mon[34294]: pgmap v304: 97 pgs: 97 active+clean; 592 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.0 KiB/s wr, 24 op/s 2024-05-09T00:44:15.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:15.990+0000 7f9fbe9df700 1 -- 172.21.15.5:0/585363511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fb8104650 msgr2=0x7f9fb8104a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:15.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:15.990+0000 7f9fbe9df700 1 --2- 172.21.15.5:0/585363511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fb8104650 0x7f9fb8104a60 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9fa80099c0 tx=0x7f9fa8009cd0 comp rx=0 tx=0).stop 2024-05-09T00:44:15.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:15.990+0000 7f9fbe9df700 1 -- 172.21.15.5:0/585363511 shutdown_connections 2024-05-09T00:44:15.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:15.990+0000 7f9fbe9df700 1 --2- 172.21.15.5:0/585363511 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9fb8109880 0x7f9fb81111e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:15.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:15.990+0000 7f9fbe9df700 1 --2- 172.21.15.5:0/585363511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fb8104650 0x7f9fb8104a60 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:15.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:15.990+0000 7f9fbe9df700 1 --2- 172.21.15.5:0/585363511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9fb8108f30 0x7f9fb8109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:15.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:15.990+0000 7f9fbe9df700 1 -- 172.21.15.5:0/585363511 >> 172.21.15.5:0/585363511 conn(0x7f9fb8079f00 msgr2=0x7f9fb807a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:44:15.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:15.990+0000 7f9fbe9df700 1 -- 172.21.15.5:0/585363511 shutdown_connections 2024-05-09T00:44:15.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:15.990+0000 7f9fbe9df700 1 -- 172.21.15.5:0/585363511 wait complete. 2024-05-09T00:44:15.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:15.991+0000 7f9fbe9df700 1 Processor -- start 2024-05-09T00:44:15.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:15.991+0000 7f9fbe9df700 1 -- start start 2024-05-09T00:44:15.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:15.991+0000 7f9fbe9df700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9fb8104650 0x7f9fb8075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:44:15.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:15.991+0000 7f9fbe9df700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9fb8108f30 0x7f9fb8076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:44:15.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:15.991+0000 7f9fbe9df700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fb8109880 0x7f9fb8076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:44:15.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:15.991+0000 7f9fbe9df700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb81144d0 con 0x7f9fb8109880 2024-05-09T00:44:15.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:15.991+0000 7f9fbe9df700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb8114350 con 0x7f9fb8108f30 2024-05-09T00:44:15.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:15.991+0000 7f9fbe9df700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb8114650 con 0x7f9fb8104650 2024-05-09T00:44:15.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:15.991+0000 7f9fbcf7c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fb8109880 0x7f9fb8076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:44:15.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:15.991+0000 7f9fbcf7c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fb8109880 0x7f9fb8076540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46378/0 (socket says 172.21.15.5:46378) 2024-05-09T00:44:15.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:15.991+0000 7f9fb77fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9fb8108f30 0x7f9fb8076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:44:15.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:15.991+0000 7f9fbcf7c700 1 -- 172.21.15.5:0/2235817441 learned_addr learned my addr 172.21.15.5:0/2235817441 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:44:15.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:15.991+0000 7f9fb7fff700 1 --2- 172.21.15.5:0/2235817441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9fb8104650 0x7f9fb8075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:44:15.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:15.992+0000 7f9fbcf7c700 1 -- 172.21.15.5:0/2235817441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9fb8104650 msgr2=0x7f9fb8075ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:15.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:15.992+0000 7f9fbcf7c700 1 --2- 172.21.15.5:0/2235817441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9fb8104650 0x7f9fb8075ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:15.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:15.992+0000 7f9fbcf7c700 1 -- 172.21.15.5:0/2235817441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9fb8108f30 msgr2=0x7f9fb8076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:15.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:15.992+0000 7f9fbcf7c700 1 --2- 172.21.15.5:0/2235817441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9fb8108f30 0x7f9fb8076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:15.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:15.992+0000 7f9fbcf7c700 1 -- 172.21.15.5:0/2235817441 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fb81a9280 con 0x7f9fb8109880 2024-05-09T00:44:15.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:15.992+0000 7f9fbcf7c700 1 --2- 172.21.15.5:0/2235817441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fb8109880 0x7f9fb8076540 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9fb000ada0 tx=0x7f9fb000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:44:15.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:15.994+0000 7f9fb57fa700 1 -- 172.21.15.5:0/2235817441 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb0017020 con 0x7f9fb8109880 2024-05-09T00:44:15.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:15.994+0000 7f9fb57fa700 1 -- 172.21.15.5:0/2235817441 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fb000cd80 con 0x7f9fb8109880 2024-05-09T00:44:15.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:15.994+0000 7f9fbe9df700 1 -- 172.21.15.5:0/2235817441 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fb81a9570 con 0x7f9fb8109880 2024-05-09T00:44:15.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:15.994+0000 7f9fb57fa700 1 -- 172.21.15.5:0/2235817441 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb0005260 con 0x7f9fb8109880 2024-05-09T00:44:15.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:15.994+0000 7f9fbe9df700 1 -- 172.21.15.5:0/2235817441 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fb81a9ab0 con 0x7f9fb8109880 2024-05-09T00:44:15.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:15.995+0000 7f9fbe9df700 1 -- 172.21.15.5:0/2235817441 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fb80512c0 con 0x7f9fb8109880 2024-05-09T00:44:15.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:15.998+0000 7f9fb57fa700 1 -- 172.21.15.5:0/2235817441 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f9fb000a090 con 0x7f9fb8109880 2024-05-09T00:44:15.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:15.999+0000 7f9fb57fa700 1 --2- 172.21.15.5:0/2235817441 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f9fa00b2990 0x7f9fa00b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:44:16.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:15.999+0000 7f9fb57fa700 1 -- 172.21.15.5:0/2235817441 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f9fb00e1df0 con 0x7f9fb8109880 2024-05-09T00:44:16.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:15.999+0000 7f9fb7fff700 1 --2- 172.21.15.5:0/2235817441 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f9fa00b2990 0x7f9fa00b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:44:16.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:15.999+0000 7f9fb7fff700 1 --2- 172.21.15.5:0/2235817441 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f9fa00b2990 0x7f9fa00b4ea0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9fa8009990 tx=0x7f9fa80036e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:44:16.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:16.002+0000 7f9fb57fa700 1 -- 172.21.15.5:0/2235817441 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9fb0021020 con 0x7f9fb8109880 2024-05-09T00:44:16.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:16.250+0000 7f9fbe9df700 1 -- 172.21.15.5:0/2235817441 --> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f9fb8063c00 con 0x7f9fa00b2990 2024-05-09T00:44:16.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:16.255+0000 7f9fb57fa700 1 -- 172.21.15.5:0/2235817441 <== mgr.14152 v2:172.21.15.5:6800/91856593 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f9fb8063c00 con 0x7f9fa00b2990 2024-05-09T00:44:16.256 INFO:teuthology.orchestra.run.smithi005.stdout:/volumes/qa/sv_0/82ce6d16-9945-4fb5-a47d-eb2887c37412 2024-05-09T00:44:16.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:16.257+0000 7f9fbe9df700 1 -- 172.21.15.5:0/2235817441 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f9fa00b2990 msgr2=0x7f9fa00b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:16.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:16.257+0000 7f9fbe9df700 1 --2- 172.21.15.5:0/2235817441 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f9fa00b2990 0x7f9fa00b4ea0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9fa8009990 tx=0x7f9fa80036e0 comp rx=0 tx=0).stop 2024-05-09T00:44:16.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:16.257+0000 7f9fbe9df700 1 -- 172.21.15.5:0/2235817441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fb8109880 msgr2=0x7f9fb8076540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:16.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:16.257+0000 7f9fbe9df700 1 --2- 172.21.15.5:0/2235817441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fb8109880 0x7f9fb8076540 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9fb000ada0 tx=0x7f9fb000c330 comp rx=0 tx=0).stop 2024-05-09T00:44:16.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:16.258+0000 7f9fbe9df700 1 -- 172.21.15.5:0/2235817441 shutdown_connections 2024-05-09T00:44:16.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:16.258+0000 7f9fbe9df700 1 --2- 172.21.15.5:0/2235817441 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f9fa00b2990 0x7f9fa00b4ea0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:16.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:16.258+0000 7f9fbe9df700 1 --2- 172.21.15.5:0/2235817441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fb8109880 0x7f9fb8076540 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:16.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:16.258+0000 7f9fbe9df700 1 --2- 172.21.15.5:0/2235817441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9fb8104650 0x7f9fb8075ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:16.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:16.258+0000 7f9fbe9df700 1 --2- 172.21.15.5:0/2235817441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9fb8108f30 0x7f9fb8076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:16.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:16.258+0000 7f9fbe9df700 1 -- 172.21.15.5:0/2235817441 >> 172.21.15.5:0/2235817441 conn(0x7f9fb8079f00 msgr2=0x7f9fb81020d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:44:16.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:16.258+0000 7f9fbe9df700 1 -- 172.21.15.5:0/2235817441 shutdown_connections 2024-05-09T00:44:16.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:16.258+0000 7f9fbe9df700 1 -- 172.21.15.5:0/2235817441 wait complete. 2024-05-09T00:44:16.897 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-05-09T00:44:17.153 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:44:17.276 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:16 smithi005 ceph-mon[30365]: pgmap v305: 97 pgs: 97 active+clean; 592 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.7 KiB/s wr, 15 op/s 2024-05-09T00:44:17.276 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:16 smithi005 ceph-mon[30365]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:44:17.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:16 smithi131 ceph-mon[34547]: pgmap v305: 97 pgs: 97 active+clean; 592 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.7 KiB/s wr, 15 op/s 2024-05-09T00:44:17.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:16 smithi131 ceph-mon[34547]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:44:17.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:16 smithi157 ceph-mon[34294]: pgmap v305: 97 pgs: 97 active+clean; 592 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.7 KiB/s wr, 15 op/s 2024-05-09T00:44:17.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:16 smithi157 ceph-mon[34294]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:44:18.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:18.828+0000 7fc1360a6700 1 -- 172.21.15.5:0/2962137953 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc130108740 msgr2=0x7fc1301100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:18.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:18.828+0000 7fc1360a6700 1 --2- 172.21.15.5:0/2962137953 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc130108740 0x7fc1301100a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc12800b0d0 tx=0x7fc12800b3e0 comp rx=0 tx=0).stop 2024-05-09T00:44:18.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:18.828+0000 7fc1360a6700 1 -- 172.21.15.5:0/2962137953 shutdown_connections 2024-05-09T00:44:18.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:18.828+0000 7fc1360a6700 1 --2- 172.21.15.5:0/2962137953 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc130108740 0x7fc1301100a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:18.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:18.828+0000 7fc1360a6700 1 --2- 172.21.15.5:0/2962137953 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1301034c0 0x7fc1301038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:18.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:18.828+0000 7fc1360a6700 1 --2- 172.21.15.5:0/2962137953 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc130107df0 0x7fc130108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:18.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:18.828+0000 7fc1360a6700 1 -- 172.21.15.5:0/2962137953 >> 172.21.15.5:0/2962137953 conn(0x7fc1300fedc0 msgr2=0x7fc1301011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:44:18.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:18.828+0000 7fc1360a6700 1 -- 172.21.15.5:0/2962137953 shutdown_connections 2024-05-09T00:44:18.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:18.828+0000 7fc1360a6700 1 -- 172.21.15.5:0/2962137953 wait complete. 2024-05-09T00:44:18.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:18.828+0000 7fc1360a6700 1 Processor -- start 2024-05-09T00:44:18.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:18.829+0000 7fc1360a6700 1 -- start start 2024-05-09T00:44:18.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:18.829+0000 7fc1360a6700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1301034c0 0x7fc13019bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:44:18.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:18.829+0000 7fc1360a6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc130107df0 0x7fc13019c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:44:18.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:18.829+0000 7fc1360a6700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc130108740 0x7fc13019c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:44:18.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:18.829+0000 7fc1360a6700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc130113370 con 0x7fc130107df0 2024-05-09T00:44:18.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:18.829+0000 7fc1360a6700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc1301131f0 con 0x7fc130108740 2024-05-09T00:44:18.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:18.829+0000 7fc1360a6700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc1301134f0 con 0x7fc1301034c0 2024-05-09T00:44:18.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:18.829+0000 7fc12effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc130107df0 0x7fc13019c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:44:18.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:18.829+0000 7fc12effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc130107df0 0x7fc13019c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46386/0 (socket says 172.21.15.5:46386) 2024-05-09T00:44:18.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:18.829+0000 7fc12effd700 1 -- 172.21.15.5:0/1828562037 learned_addr learned my addr 172.21.15.5:0/1828562037 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:44:18.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:18.829+0000 7fc12f7fe700 1 --2- 172.21.15.5:0/1828562037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1301034c0 0x7fc13019bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:44:18.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:18.830+0000 7fc12ffff700 1 --2- 172.21.15.5:0/1828562037 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc130108740 0x7fc13019c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:44:18.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:18.830+0000 7fc12effd700 1 -- 172.21.15.5:0/1828562037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1301034c0 msgr2=0x7fc13019bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:18.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:18.830+0000 7fc12effd700 1 --2- 172.21.15.5:0/1828562037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1301034c0 0x7fc13019bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:18.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:18.830+0000 7fc12effd700 1 -- 172.21.15.5:0/1828562037 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc130108740 msgr2=0x7fc13019c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:18.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:18.830+0000 7fc12effd700 1 --2- 172.21.15.5:0/1828562037 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc130108740 0x7fc13019c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:18.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:18.830+0000 7fc12effd700 1 -- 172.21.15.5:0/1828562037 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1301a7000 con 0x7fc130107df0 2024-05-09T00:44:18.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:18.830+0000 7fc12f7fe700 1 --2- 172.21.15.5:0/1828562037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1301034c0 0x7fc13019bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:44:18.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:18.830+0000 7fc12ffff700 1 --2- 172.21.15.5:0/1828562037 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc130108740 0x7fc13019c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:44:18.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:18.830+0000 7fc12effd700 1 --2- 172.21.15.5:0/1828562037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc130107df0 0x7fc13019c1f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc12400d950 tx=0x7fc12400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:44:18.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:18.830+0000 7fc12cff9700 1 -- 172.21.15.5:0/1828562037 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc124016070 con 0x7fc130107df0 2024-05-09T00:44:18.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:18.830+0000 7fc12cff9700 1 -- 172.21.15.5:0/1828562037 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc12400de30 con 0x7fc130107df0 2024-05-09T00:44:18.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:18.830+0000 7fc1360a6700 1 -- 172.21.15.5:0/1828562037 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1301a72f0 con 0x7fc130107df0 2024-05-09T00:44:18.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:18.831+0000 7fc12cff9700 1 -- 172.21.15.5:0/1828562037 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc124005350 con 0x7fc130107df0 2024-05-09T00:44:18.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:18.831+0000 7fc1360a6700 1 -- 172.21.15.5:0/1828562037 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1301a7880 con 0x7fc130107df0 2024-05-09T00:44:18.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:18.832+0000 7fc1360a6700 1 -- 172.21.15.5:0/1828562037 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1300512c0 con 0x7fc130107df0 2024-05-09T00:44:18.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:18.833+0000 7fc12cff9700 1 -- 172.21.15.5:0/1828562037 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fc1240054f0 con 0x7fc130107df0 2024-05-09T00:44:18.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:18.834+0000 7fc12cff9700 1 --2- 172.21.15.5:0/1828562037 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fc1180b2990 0x7fc1180b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:44:18.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:18.834+0000 7fc12f7fe700 1 --2- 172.21.15.5:0/1828562037 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fc1180b2990 0x7fc1180b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:44:18.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:18.834+0000 7fc12cff9700 1 -- 172.21.15.5:0/1828562037 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7fc1240e19b0 con 0x7fc130107df0 2024-05-09T00:44:18.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:18.834+0000 7fc12f7fe700 1 --2- 172.21.15.5:0/1828562037 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fc1180b2990 0x7fc1180b4ea0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc120005950 tx=0x7fc120005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:44:18.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:18.839+0000 7fc12cff9700 1 -- 172.21.15.5:0/1828562037 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc12401f070 con 0x7fc130107df0 2024-05-09T00:44:19.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:19.087+0000 7fc1360a6700 1 -- 172.21.15.5:0/1828562037 --> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fc13006bc00 con 0x7fc1180b2990 2024-05-09T00:44:19.089 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:18 smithi005 ceph-mon[30365]: pgmap v306: 97 pgs: 97 active+clean; 592 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-05-09T00:44:19.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:19.116+0000 7fc12cff9700 1 -- 172.21.15.5:0/1828562037 <== mgr.14152 v2:172.21.15.5:6800/91856593 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc13006bc00 con 0x7fc1180b2990 2024-05-09T00:44:19.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:19.119+0000 7fc1360a6700 1 -- 172.21.15.5:0/1828562037 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fc1180b2990 msgr2=0x7fc1180b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:19.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:19.119+0000 7fc1360a6700 1 --2- 172.21.15.5:0/1828562037 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fc1180b2990 0x7fc1180b4ea0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc120005950 tx=0x7fc120005f50 comp rx=0 tx=0).stop 2024-05-09T00:44:19.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:19.119+0000 7fc1360a6700 1 -- 172.21.15.5:0/1828562037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc130107df0 msgr2=0x7fc13019c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:19.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:19.119+0000 7fc1360a6700 1 --2- 172.21.15.5:0/1828562037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc130107df0 0x7fc13019c1f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc12400d950 tx=0x7fc12400dd10 comp rx=0 tx=0).stop 2024-05-09T00:44:19.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:19.119+0000 7fc1360a6700 1 -- 172.21.15.5:0/1828562037 shutdown_connections 2024-05-09T00:44:19.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:19.119+0000 7fc1360a6700 1 --2- 172.21.15.5:0/1828562037 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fc1180b2990 0x7fc1180b4ea0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:19.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:19.119+0000 7fc1360a6700 1 --2- 172.21.15.5:0/1828562037 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc130108740 0x7fc13019c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:19.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:19.119+0000 7fc1360a6700 1 --2- 172.21.15.5:0/1828562037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1301034c0 0x7fc13019bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:19.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:19.119+0000 7fc1360a6700 1 --2- 172.21.15.5:0/1828562037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc130107df0 0x7fc13019c1f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:19.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:19.119+0000 7fc1360a6700 1 -- 172.21.15.5:0/1828562037 >> 172.21.15.5:0/1828562037 conn(0x7fc1300fedc0 msgr2=0x7fc130107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:44:19.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:19.119+0000 7fc1360a6700 1 -- 172.21.15.5:0/1828562037 shutdown_connections 2024-05-09T00:44:19.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:19.119+0000 7fc1360a6700 1 -- 172.21.15.5:0/1828562037 wait complete. 2024-05-09T00:44:19.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:18 smithi131 ceph-mon[34547]: pgmap v306: 97 pgs: 97 active+clean; 592 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-05-09T00:44:19.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:18 smithi157 ceph-mon[34294]: pgmap v306: 97 pgs: 97 active+clean; 592 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-05-09T00:44:19.769 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-05-09T00:44:20.031 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:44:20.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:20 smithi005 ceph-mon[30365]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:44:20.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:20 smithi131 ceph-mon[34547]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:44:20.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:20 smithi157 ceph-mon[34294]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:44:21.194 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:21 smithi005 ceph-mon[30365]: pgmap v307: 97 pgs: 97 active+clean; 611 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 3.1 KiB/s wr, 12 op/s 2024-05-09T00:44:21.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:21 smithi131 ceph-mon[34547]: pgmap v307: 97 pgs: 97 active+clean; 611 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 3.1 KiB/s wr, 12 op/s 2024-05-09T00:44:21.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:21 smithi157 ceph-mon[34294]: pgmap v307: 97 pgs: 97 active+clean; 611 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 3.1 KiB/s wr, 12 op/s 2024-05-09T00:44:21.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:21.770+0000 7f3d2929f700 1 -- 172.21.15.5:0/1787413927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d24108740 msgr2=0x7f3d241100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:21.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:21.770+0000 7f3d2929f700 1 --2- 172.21.15.5:0/1787413927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d24108740 0x7f3d241100a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c00b0d0 tx=0x7f3d1c00b3e0 comp rx=0 tx=0).stop 2024-05-09T00:44:21.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:21.770+0000 7f3d2929f700 1 -- 172.21.15.5:0/1787413927 shutdown_connections 2024-05-09T00:44:21.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:21.770+0000 7f3d2929f700 1 --2- 172.21.15.5:0/1787413927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d24108740 0x7f3d241100a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:21.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:21.770+0000 7f3d2929f700 1 --2- 172.21.15.5:0/1787413927 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d241034c0 0x7f3d241038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:21.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:21.770+0000 7f3d2929f700 1 --2- 172.21.15.5:0/1787413927 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d24107df0 0x7f3d24108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:21.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:21.770+0000 7f3d2929f700 1 -- 172.21.15.5:0/1787413927 >> 172.21.15.5:0/1787413927 conn(0x7f3d240fedf0 msgr2=0x7f3d24101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:44:21.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:21.770+0000 7f3d2929f700 1 -- 172.21.15.5:0/1787413927 shutdown_connections 2024-05-09T00:44:21.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:21.770+0000 7f3d2929f700 1 -- 172.21.15.5:0/1787413927 wait complete. 2024-05-09T00:44:21.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:21.771+0000 7f3d2929f700 1 Processor -- start 2024-05-09T00:44:21.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:21.771+0000 7f3d2929f700 1 -- start start 2024-05-09T00:44:21.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:21.772+0000 7f3d2929f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d241034c0 0x7f3d2419bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:44:21.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:21.772+0000 7f3d2929f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d24107df0 0x7f3d2419c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:44:21.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:21.772+0000 7f3d2929f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d24108740 0x7f3d2419c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:44:21.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:21.772+0000 7f3d2929f700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3d24113370 con 0x7f3d24107df0 2024-05-09T00:44:21.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:21.772+0000 7f3d2929f700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3d241131f0 con 0x7f3d24108740 2024-05-09T00:44:21.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:21.772+0000 7f3d2929f700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3d241134f0 con 0x7f3d241034c0 2024-05-09T00:44:21.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:21.772+0000 7f3d227fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d24107df0 0x7f3d2419c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:44:21.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:21.772+0000 7f3d227fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d24107df0 0x7f3d2419c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46410/0 (socket says 172.21.15.5:46410) 2024-05-09T00:44:21.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:21.772+0000 7f3d227fc700 1 -- 172.21.15.5:0/1922139720 learned_addr learned my addr 172.21.15.5:0/1922139720 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:44:21.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:21.772+0000 7f3d22ffd700 1 --2- 172.21.15.5:0/1922139720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d241034c0 0x7f3d2419bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:44:21.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:21.772+0000 7f3d237fe700 1 --2- 172.21.15.5:0/1922139720 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d24108740 0x7f3d2419c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:44:21.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:21.772+0000 7f3d227fc700 1 -- 172.21.15.5:0/1922139720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d241034c0 msgr2=0x7f3d2419bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:21.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:21.772+0000 7f3d227fc700 1 --2- 172.21.15.5:0/1922139720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d241034c0 0x7f3d2419bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:21.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:21.772+0000 7f3d227fc700 1 -- 172.21.15.5:0/1922139720 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d24108740 msgr2=0x7f3d2419c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:21.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:21.772+0000 7f3d227fc700 1 --2- 172.21.15.5:0/1922139720 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d24108740 0x7f3d2419c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:21.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:21.772+0000 7f3d227fc700 1 -- 172.21.15.5:0/1922139720 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d241a6fe0 con 0x7f3d24107df0 2024-05-09T00:44:21.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:21.773+0000 7f3d227fc700 1 --2- 172.21.15.5:0/1922139720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d24107df0 0x7f3d2419c1b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3d1800d980 tx=0x7f3d1800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:44:21.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:21.773+0000 7f3d0bfff700 1 -- 172.21.15.5:0/1922139720 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d18017070 con 0x7f3d24107df0 2024-05-09T00:44:21.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:21.773+0000 7f3d0bfff700 1 -- 172.21.15.5:0/1922139720 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d1800fd40 con 0x7f3d24107df0 2024-05-09T00:44:21.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:21.773+0000 7f3d2929f700 1 -- 172.21.15.5:0/1922139720 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d241a72d0 con 0x7f3d24107df0 2024-05-09T00:44:21.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:21.773+0000 7f3d0bfff700 1 -- 172.21.15.5:0/1922139720 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d1800e3b0 con 0x7f3d24107df0 2024-05-09T00:44:21.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:21.773+0000 7f3d2929f700 1 -- 172.21.15.5:0/1922139720 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d241a7860 con 0x7f3d24107df0 2024-05-09T00:44:21.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:21.774+0000 7f3d2929f700 1 -- 172.21.15.5:0/1922139720 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d240512c0 con 0x7f3d24107df0 2024-05-09T00:44:21.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:21.776+0000 7f3d0bfff700 1 -- 172.21.15.5:0/1922139720 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f3d18010610 con 0x7f3d24107df0 2024-05-09T00:44:21.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:21.777+0000 7f3d0bfff700 1 --2- 172.21.15.5:0/1922139720 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f3d0c0b2990 0x7f3d0c0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:44:21.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:21.777+0000 7f3d22ffd700 1 --2- 172.21.15.5:0/1922139720 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f3d0c0b2990 0x7f3d0c0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:44:21.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:21.777+0000 7f3d0bfff700 1 -- 172.21.15.5:0/1922139720 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f3d180e1c80 con 0x7f3d24107df0 2024-05-09T00:44:21.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:21.778+0000 7f3d22ffd700 1 --2- 172.21.15.5:0/1922139720 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f3d0c0b2990 0x7f3d0c0b4ea0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3d14005950 tx=0x7f3d1400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:44:21.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:21.781+0000 7f3d0bfff700 1 -- 172.21.15.5:0/1922139720 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3d180a9970 con 0x7f3d24107df0 2024-05-09T00:44:22.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:22 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-05-09T00:44:22.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:22 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]: dispatch 2024-05-09T00:44:22.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:22 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6, 5, 9]}]: dispatch 2024-05-09T00:44:22.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:22 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8, 5, 6]}]: dispatch 2024-05-09T00:44:22.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:22 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [3, 8]}]: dispatch 2024-05-09T00:44:22.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:22 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 7]}]: dispatch 2024-05-09T00:44:22.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:22 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.35", "id": [1, 9]}]: dispatch 2024-05-09T00:44:22.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:22 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 7]}]: dispatch 2024-05-09T00:44:22.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:22.042+0000 7f3d2929f700 1 -- 172.21.15.5:0/1922139720 --> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f3d24063c00 con 0x7f3d0c0b2990 2024-05-09T00:44:22.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:22.047+0000 7f3d0bfff700 1 -- 172.21.15.5:0/1922139720 <== mgr.14152 v2:172.21.15.5:6800/91856593 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f3d24063c00 con 0x7f3d0c0b2990 2024-05-09T00:44:22.049 INFO:teuthology.orchestra.run.smithi005.stdout:/volumes/qa/sv_1/2cb5a101-4e52-403f-89d2-3ba7d2b5da5c 2024-05-09T00:44:22.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:22.050+0000 7f3d2929f700 1 -- 172.21.15.5:0/1922139720 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f3d0c0b2990 msgr2=0x7f3d0c0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:22.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:22.050+0000 7f3d2929f700 1 --2- 172.21.15.5:0/1922139720 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f3d0c0b2990 0x7f3d0c0b4ea0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3d14005950 tx=0x7f3d1400b410 comp rx=0 tx=0).stop 2024-05-09T00:44:22.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:22.050+0000 7f3d2929f700 1 -- 172.21.15.5:0/1922139720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d24107df0 msgr2=0x7f3d2419c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:22.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:22.050+0000 7f3d2929f700 1 --2- 172.21.15.5:0/1922139720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d24107df0 0x7f3d2419c1b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3d1800d980 tx=0x7f3d1800dc90 comp rx=0 tx=0).stop 2024-05-09T00:44:22.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:22.050+0000 7f3d2929f700 1 -- 172.21.15.5:0/1922139720 shutdown_connections 2024-05-09T00:44:22.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:22.050+0000 7f3d2929f700 1 --2- 172.21.15.5:0/1922139720 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f3d0c0b2990 0x7f3d0c0b4ea0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:22.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:22.050+0000 7f3d2929f700 1 --2- 172.21.15.5:0/1922139720 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d24108740 0x7f3d2419c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:22.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:22.050+0000 7f3d2929f700 1 --2- 172.21.15.5:0/1922139720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d241034c0 0x7f3d2419bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:22.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:22.050+0000 7f3d2929f700 1 --2- 172.21.15.5:0/1922139720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d24107df0 0x7f3d2419c1b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:22.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:22.050+0000 7f3d2929f700 1 -- 172.21.15.5:0/1922139720 >> 172.21.15.5:0/1922139720 conn(0x7f3d240fedf0 msgr2=0x7f3d24107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:44:22.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:22.050+0000 7f3d2929f700 1 -- 172.21.15.5:0/1922139720 shutdown_connections 2024-05-09T00:44:22.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:22.050+0000 7f3d2929f700 1 -- 172.21.15.5:0/1922139720 wait complete. 2024-05-09T00:44:22.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:22 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-05-09T00:44:22.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:22 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]: dispatch 2024-05-09T00:44:22.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:22 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6, 5, 9]}]: dispatch 2024-05-09T00:44:22.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:22 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8, 5, 6]}]: dispatch 2024-05-09T00:44:22.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:22 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [3, 8]}]: dispatch 2024-05-09T00:44:22.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:22 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 7]}]: dispatch 2024-05-09T00:44:22.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:22 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.35", "id": [1, 9]}]: dispatch 2024-05-09T00:44:22.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:22 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 7]}]: dispatch 2024-05-09T00:44:22.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:22 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-05-09T00:44:22.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:22 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]: dispatch 2024-05-09T00:44:22.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:22 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6, 5, 9]}]: dispatch 2024-05-09T00:44:22.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:22 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8, 5, 6]}]: dispatch 2024-05-09T00:44:22.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:22 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [3, 8]}]: dispatch 2024-05-09T00:44:22.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:22 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 7]}]: dispatch 2024-05-09T00:44:22.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:22 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.35", "id": [1, 9]}]: dispatch 2024-05-09T00:44:22.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:22 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 7]}]: dispatch 2024-05-09T00:44:22.702 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T00:44:22.960 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:44:23.276 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:23 smithi005 ceph-mon[30365]: pgmap v308: 97 pgs: 97 active+clean; 612 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.2 KiB/s wr, 10 op/s 2024-05-09T00:44:23.276 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:23 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-05-09T00:44:23.276 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:23 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]': finished 2024-05-09T00:44:23.277 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:23 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6, 5, 9]}]': finished 2024-05-09T00:44:23.277 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:23 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8, 5, 6]}]': finished 2024-05-09T00:44:23.277 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:23 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [3, 8]}]': finished 2024-05-09T00:44:23.277 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:23 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 7]}]': finished 2024-05-09T00:44:23.277 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:23 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.35", "id": [1, 9]}]': finished 2024-05-09T00:44:23.277 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:23 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 7]}]': finished 2024-05-09T00:44:23.277 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:23 smithi005 ceph-mon[30365]: osdmap e79: 12 total, 12 up, 12 in 2024-05-09T00:44:23.277 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:23 smithi005 ceph-mon[30365]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:44:23.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:23 smithi131 ceph-mon[34547]: pgmap v308: 97 pgs: 97 active+clean; 612 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.2 KiB/s wr, 10 op/s 2024-05-09T00:44:23.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:23 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-05-09T00:44:23.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:23 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]': finished 2024-05-09T00:44:23.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:23 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6, 5, 9]}]': finished 2024-05-09T00:44:23.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:23 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8, 5, 6]}]': finished 2024-05-09T00:44:23.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:23 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [3, 8]}]': finished 2024-05-09T00:44:23.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:23 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 7]}]': finished 2024-05-09T00:44:23.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:23 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.35", "id": [1, 9]}]': finished 2024-05-09T00:44:23.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:23 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 7]}]': finished 2024-05-09T00:44:23.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:23 smithi131 ceph-mon[34547]: osdmap e79: 12 total, 12 up, 12 in 2024-05-09T00:44:23.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:23 smithi131 ceph-mon[34547]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:44:23.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:23 smithi157 ceph-mon[34294]: pgmap v308: 97 pgs: 97 active+clean; 612 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.2 KiB/s wr, 10 op/s 2024-05-09T00:44:23.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:23 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-05-09T00:44:23.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:23 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]': finished 2024-05-09T00:44:23.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:23 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6, 5, 9]}]': finished 2024-05-09T00:44:23.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:23 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8, 5, 6]}]': finished 2024-05-09T00:44:23.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:23 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [3, 8]}]': finished 2024-05-09T00:44:23.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:23 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 7]}]': finished 2024-05-09T00:44:23.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:23 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.35", "id": [1, 9]}]': finished 2024-05-09T00:44:23.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:23 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 7]}]': finished 2024-05-09T00:44:23.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:23 smithi157 ceph-mon[34294]: osdmap e79: 12 total, 12 up, 12 in 2024-05-09T00:44:23.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:23 smithi157 ceph-mon[34294]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:44:24.112 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:24 smithi005 ceph-mon[30365]: osdmap e80: 12 total, 12 up, 12 in 2024-05-09T00:44:24.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:24 smithi131 ceph-mon[34547]: osdmap e80: 12 total, 12 up, 12 in 2024-05-09T00:44:24.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:24 smithi157 ceph-mon[34294]: osdmap e80: 12 total, 12 up, 12 in 2024-05-09T00:44:24.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:24.677+0000 7fc546212700 1 -- 172.21.15.5:0/522508016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc540108740 msgr2=0x7fc5401100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:24.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:24.677+0000 7fc546212700 1 --2- 172.21.15.5:0/522508016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc540108740 0x7fc5401100a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc53800b0d0 tx=0x7fc53800b3e0 comp rx=0 tx=0).stop 2024-05-09T00:44:24.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:24.678+0000 7fc546212700 1 -- 172.21.15.5:0/522508016 shutdown_connections 2024-05-09T00:44:24.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:24.678+0000 7fc546212700 1 --2- 172.21.15.5:0/522508016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc540108740 0x7fc5401100a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:24.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:24.678+0000 7fc546212700 1 --2- 172.21.15.5:0/522508016 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5401034c0 0x7fc5401038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:24.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:24.678+0000 7fc546212700 1 --2- 172.21.15.5:0/522508016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc540107df0 0x7fc540108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:24.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:24.678+0000 7fc546212700 1 -- 172.21.15.5:0/522508016 >> 172.21.15.5:0/522508016 conn(0x7fc5400fedf0 msgr2=0x7fc540101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:44:24.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:24.678+0000 7fc546212700 1 -- 172.21.15.5:0/522508016 shutdown_connections 2024-05-09T00:44:24.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:24.678+0000 7fc546212700 1 -- 172.21.15.5:0/522508016 wait complete. 2024-05-09T00:44:24.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:24.678+0000 7fc546212700 1 Processor -- start 2024-05-09T00:44:24.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:24.679+0000 7fc546212700 1 -- start start 2024-05-09T00:44:24.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:24.679+0000 7fc546212700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5401034c0 0x7fc54019aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:44:24.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:24.679+0000 7fc546212700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc540107df0 0x7fc54019b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:44:24.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:24.679+0000 7fc546212700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc540108740 0x7fc54019f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:44:24.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:24.679+0000 7fc546212700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc540113370 con 0x7fc540108740 2024-05-09T00:44:24.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:24.679+0000 7fc546212700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc5401131f0 con 0x7fc5401034c0 2024-05-09T00:44:24.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:24.679+0000 7fc546212700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc5401134f0 con 0x7fc540107df0 2024-05-09T00:44:24.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:24.679+0000 7fc53ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc540108740 0x7fc54019f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:44:24.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:24.679+0000 7fc53ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc540108740 0x7fc54019f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41094/0 (socket says 172.21.15.5:41094) 2024-05-09T00:44:24.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:24.679+0000 7fc53ffff700 1 -- 172.21.15.5:0/3038965063 learned_addr learned my addr 172.21.15.5:0/3038965063 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:44:24.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:24.679+0000 7fc53f7fe700 1 --2- 172.21.15.5:0/3038965063 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5401034c0 0x7fc54019aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:44:24.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:24.679+0000 7fc53effd700 1 --2- 172.21.15.5:0/3038965063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc540107df0 0x7fc54019b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:44:24.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:24.679+0000 7fc53ffff700 1 -- 172.21.15.5:0/3038965063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc540107df0 msgr2=0x7fc54019b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:24.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:24.679+0000 7fc53ffff700 1 --2- 172.21.15.5:0/3038965063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc540107df0 0x7fc54019b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:24.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:24.679+0000 7fc53ffff700 1 -- 172.21.15.5:0/3038965063 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5401034c0 msgr2=0x7fc54019aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:24.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:24.679+0000 7fc53ffff700 1 --2- 172.21.15.5:0/3038965063 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5401034c0 0x7fc54019aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:24.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:24.679+0000 7fc53ffff700 1 -- 172.21.15.5:0/3038965063 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc54019fda0 con 0x7fc540108740 2024-05-09T00:44:24.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:24.679+0000 7fc53f7fe700 1 --2- 172.21.15.5:0/3038965063 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5401034c0 0x7fc54019aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:44:24.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:24.680+0000 7fc53ffff700 1 --2- 172.21.15.5:0/3038965063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc540108740 0x7fc54019f700 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc538009850 tx=0x7fc538007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:44:24.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:24.680+0000 7fc53cff9700 1 -- 172.21.15.5:0/3038965063 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc53801f070 con 0x7fc540108740 2024-05-09T00:44:24.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:24.680+0000 7fc546212700 1 -- 172.21.15.5:0/3038965063 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5401a0030 con 0x7fc540108740 2024-05-09T00:44:24.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:24.680+0000 7fc53cff9700 1 -- 172.21.15.5:0/3038965063 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc538004c50 con 0x7fc540108740 2024-05-09T00:44:24.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:24.680+0000 7fc53cff9700 1 -- 172.21.15.5:0/3038965063 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc53801b690 con 0x7fc540108740 2024-05-09T00:44:24.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:24.680+0000 7fc546212700 1 -- 172.21.15.5:0/3038965063 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5401a0510 con 0x7fc540108740 2024-05-09T00:44:24.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:24.681+0000 7fc546212700 1 -- 172.21.15.5:0/3038965063 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5400512c0 con 0x7fc540108740 2024-05-09T00:44:24.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:24.683+0000 7fc53cff9700 1 -- 172.21.15.5:0/3038965063 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fc538022070 con 0x7fc540108740 2024-05-09T00:44:24.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:24.684+0000 7fc53cff9700 1 --2- 172.21.15.5:0/3038965063 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fc5280b2910 0x7fc5280b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:44:24.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:24.684+0000 7fc53f7fe700 1 --2- 172.21.15.5:0/3038965063 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fc5280b2910 0x7fc5280b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:44:24.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:24.684+0000 7fc53cff9700 1 -- 172.21.15.5:0/3038965063 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8227+0+0 (secure 0 0 0) 0x7fc53800f020 con 0x7fc540108740 2024-05-09T00:44:24.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:24.684+0000 7fc53f7fe700 1 --2- 172.21.15.5:0/3038965063 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fc5280b2910 0x7fc5280b4e20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc530011490 tx=0x7fc530009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:44:24.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:24.688+0000 7fc53cff9700 1 -- 172.21.15.5:0/3038965063 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc538029020 con 0x7fc540108740 2024-05-09T00:44:24.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:24.952+0000 7fc546212700 1 -- 172.21.15.5:0/3038965063 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc54002e5f0 con 0x7fc540108740 2024-05-09T00:44:24.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:24.952+0000 7fc53cff9700 1 -- 172.21.15.5:0/3038965063 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11926 (secure 0 0 0) 0x7fc5380ab0d0 con 0x7fc540108740 2024-05-09T00:44:24.955 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:44:24.955 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2685771887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2685771887},{"type":"v1","addr":"172.21.15.5:6835","nonce":2685771887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/254017535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":254017535},{"type":"v1","addr":"172.21.15.5:6837","nonce":254017535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T00:43:45.856066+0000","modified":"2024-05-09T00:44:06.033927+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24508,"mds_1":24499,"mds_2":24494,"mds_3":24493,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14598":{"gid":14598,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2109255228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2109255228},{"type":"v1","addr":"172.21.15.5:6839","nonce":2109255228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/3809966329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3809966329},{"type":"v1","addr":"172.21.15.5:6841","nonce":3809966329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6833/3272897742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3272897742},{"type":"v1","addr":"172.21.15.131:6833","nonce":3272897742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1283093898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1283093898},{"type":"v1","addr":"172.21.15.131:6835","nonce":1283093898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/2809525455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2809525455},{"type":"v1","addr":"172.21.15.157:6833","nonce":2809525455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.131:6837/1633006186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":1633006186},{"type":"v1","addr":"172.21.15.131:6837","nonce":1633006186}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.157:6835/832898273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":832898273},{"type":"v1","addr":"172.21.15.157:6835","nonce":832898273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.131:6839/2748669631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2748669631},{"type":"v1","addr":"172.21.15.131:6839","nonce":2748669631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.157:6837/744863009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":744863009},{"type":"v1","addr":"172.21.15.157:6837","nonce":744863009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.157:6839/2779860399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":2779860399},{"type":"v1","addr":"172.21.15.157:6839","nonce":2779860399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24467,24493,24499,24508,24494]},"id":1}]} 2024-05-09T00:44:24.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:24.955+0000 7fc546212700 1 -- 172.21.15.5:0/3038965063 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fc5280b2910 msgr2=0x7fc5280b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:24.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:24.955+0000 7fc546212700 1 --2- 172.21.15.5:0/3038965063 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fc5280b2910 0x7fc5280b4e20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc530011490 tx=0x7fc530009260 comp rx=0 tx=0).stop 2024-05-09T00:44:24.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:24.955+0000 7fc546212700 1 -- 172.21.15.5:0/3038965063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc540108740 msgr2=0x7fc54019f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:24.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:24.955+0000 7fc546212700 1 --2- 172.21.15.5:0/3038965063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc540108740 0x7fc54019f700 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc538009850 tx=0x7fc538007a20 comp rx=0 tx=0).stop 2024-05-09T00:44:24.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:24.955+0000 7fc546212700 1 -- 172.21.15.5:0/3038965063 shutdown_connections 2024-05-09T00:44:24.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:24.955+0000 7fc546212700 1 --2- 172.21.15.5:0/3038965063 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fc5280b2910 0x7fc5280b4e20 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:24.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:24.955+0000 7fc546212700 1 --2- 172.21.15.5:0/3038965063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc540108740 0x7fc54019f700 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:24.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:24.955+0000 7fc546212700 1 --2- 172.21.15.5:0/3038965063 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5401034c0 0x7fc54019aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:24.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:24.955+0000 7fc546212700 1 --2- 172.21.15.5:0/3038965063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc540107df0 0x7fc54019b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:24.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:24.955+0000 7fc546212700 1 -- 172.21.15.5:0/3038965063 >> 172.21.15.5:0/3038965063 conn(0x7fc5400fedf0 msgr2=0x7fc540075d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:44:24.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:24.955+0000 7fc546212700 1 -- 172.21.15.5:0/3038965063 shutdown_connections 2024-05-09T00:44:24.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:24.955+0000 7fc546212700 1 -- 172.21.15.5:0/3038965063 wait complete. 2024-05-09T00:44:24.958 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 28 2024-05-09T00:44:25.270 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:25 smithi005 ceph-mon[30365]: pgmap v311: 97 pgs: 1 activating+degraded, 2 peering, 94 active+clean; 636 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 6.6 KiB/s wr, 13 op/s; 6/306 objects degraded (1.961%) 2024-05-09T00:44:25.271 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:25 smithi005 ceph-mon[30365]: Health check failed: Degraded data redundancy: 6/306 objects degraded (1.961%), 1 pg degraded (PG_DEGRADED) 2024-05-09T00:44:25.271 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:25 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/3038965063' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:44:25.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:25 smithi131 ceph-mon[34547]: pgmap v311: 97 pgs: 1 activating+degraded, 2 peering, 94 active+clean; 636 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 6.6 KiB/s wr, 13 op/s; 6/306 objects degraded (1.961%) 2024-05-09T00:44:25.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:25 smithi131 ceph-mon[34547]: Health check failed: Degraded data redundancy: 6/306 objects degraded (1.961%), 1 pg degraded (PG_DEGRADED) 2024-05-09T00:44:25.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:25 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/3038965063' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:44:25.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:25 smithi157 ceph-mon[34294]: pgmap v311: 97 pgs: 1 activating+degraded, 2 peering, 94 active+clean; 636 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 6.6 KiB/s wr, 13 op/s; 6/306 objects degraded (1.961%) 2024-05-09T00:44:25.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:25 smithi157 ceph-mon[34294]: Health check failed: Degraded data redundancy: 6/306 objects degraded (1.961%), 1 pg degraded (PG_DEGRADED) 2024-05-09T00:44:25.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:25 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/3038965063' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:44:25.611 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T00:44:25.869 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:44:27.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:27 smithi005 ceph-mon[30365]: pgmap v312: 97 pgs: 1 activating+degraded, 2 peering, 94 active+clean; 636 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 6.4 KiB/s wr, 12 op/s; 6/306 objects degraded (1.961%) 2024-05-09T00:44:27.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:27 smithi131 ceph-mon[34547]: pgmap v312: 97 pgs: 1 activating+degraded, 2 peering, 94 active+clean; 636 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 6.4 KiB/s wr, 12 op/s; 6/306 objects degraded (1.961%) 2024-05-09T00:44:27.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:27 smithi157 ceph-mon[34294]: pgmap v312: 97 pgs: 1 activating+degraded, 2 peering, 94 active+clean; 636 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 6.4 KiB/s wr, 12 op/s; 6/306 objects degraded (1.961%) 2024-05-09T00:44:27.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:27.591+0000 7f357f488700 1 -- 172.21.15.5:0/3264850304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3578109850 msgr2=0x7f35781111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:27.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:27.591+0000 7f357f488700 1 --2- 172.21.15.5:0/3264850304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3578109850 0x7f35781111b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f357000b0d0 tx=0x7f357000b3e0 comp rx=0 tx=0).stop 2024-05-09T00:44:27.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:27.591+0000 7f357f488700 1 -- 172.21.15.5:0/3264850304 shutdown_connections 2024-05-09T00:44:27.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:27.591+0000 7f357f488700 1 --2- 172.21.15.5:0/3264850304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3578109850 0x7f35781111b0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:27.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:27.591+0000 7f357f488700 1 --2- 172.21.15.5:0/3264850304 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35781045d0 0x7f35781049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:27.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:27.591+0000 7f357f488700 1 --2- 172.21.15.5:0/3264850304 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3578108f00 0x7f3578109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:27.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:27.591+0000 7f357f488700 1 -- 172.21.15.5:0/3264850304 >> 172.21.15.5:0/3264850304 conn(0x7f35780ffea0 msgr2=0x7f35781022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:44:27.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:27.591+0000 7f357f488700 1 -- 172.21.15.5:0/3264850304 shutdown_connections 2024-05-09T00:44:27.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:27.591+0000 7f357f488700 1 -- 172.21.15.5:0/3264850304 wait complete. 2024-05-09T00:44:27.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:27.592+0000 7f357f488700 1 Processor -- start 2024-05-09T00:44:27.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:27.592+0000 7f357f488700 1 -- start start 2024-05-09T00:44:27.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:27.592+0000 7f357f488700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35781045d0 0x7f357819af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:44:27.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:27.592+0000 7f357f488700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3578108f00 0x7f357819b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:44:27.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:27.592+0000 7f357f488700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3578109850 0x7f357819f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:44:27.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:27.592+0000 7f357f488700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3578114570 con 0x7f35781045d0 2024-05-09T00:44:27.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:27.592+0000 7f357f488700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f35781143f0 con 0x7f3578108f00 2024-05-09T00:44:27.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:27.592+0000 7f357f488700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f35781146f0 con 0x7f3578109850 2024-05-09T00:44:27.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:27.592+0000 7f357d224700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35781045d0 0x7f357819af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:44:27.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:27.592+0000 7f357d224700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35781045d0 0x7f357819af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41120/0 (socket says 172.21.15.5:41120) 2024-05-09T00:44:27.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:27.592+0000 7f357d224700 1 -- 172.21.15.5:0/3303672417 learned_addr learned my addr 172.21.15.5:0/3303672417 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:44:27.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:27.593+0000 7f357da25700 1 --2- 172.21.15.5:0/3303672417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3578109850 0x7f357819f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:44:27.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:27.593+0000 7f357ca23700 1 --2- 172.21.15.5:0/3303672417 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3578108f00 0x7f357819b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:44:27.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:27.593+0000 7f357d224700 1 -- 172.21.15.5:0/3303672417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3578109850 msgr2=0x7f357819f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:27.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:27.593+0000 7f357d224700 1 --2- 172.21.15.5:0/3303672417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3578109850 0x7f357819f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:27.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:27.593+0000 7f357d224700 1 -- 172.21.15.5:0/3303672417 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3578108f00 msgr2=0x7f357819b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:27.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:27.593+0000 7f357d224700 1 --2- 172.21.15.5:0/3303672417 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3578108f00 0x7f357819b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:27.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:27.593+0000 7f357d224700 1 -- 172.21.15.5:0/3303672417 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f357819fe50 con 0x7f35781045d0 2024-05-09T00:44:27.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:27.593+0000 7f357d224700 1 --2- 172.21.15.5:0/3303672417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35781045d0 0x7f357819af80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f357400b530 tx=0x7f357400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:44:27.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:27.593+0000 7f356e7fc700 1 -- 172.21.15.5:0/3303672417 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3574015020 con 0x7f35781045d0 2024-05-09T00:44:27.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:27.593+0000 7f356e7fc700 1 -- 172.21.15.5:0/3303672417 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3574011ce0 con 0x7f35781045d0 2024-05-09T00:44:27.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:27.593+0000 7f357f488700 1 -- 172.21.15.5:0/3303672417 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35781a0140 con 0x7f35781045d0 2024-05-09T00:44:27.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:27.593+0000 7f356e7fc700 1 -- 172.21.15.5:0/3303672417 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3574010e40 con 0x7f35781045d0 2024-05-09T00:44:27.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:27.593+0000 7f357f488700 1 -- 172.21.15.5:0/3303672417 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35781a7a60 con 0x7f35781045d0 2024-05-09T00:44:27.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:27.594+0000 7f357f488700 1 -- 172.21.15.5:0/3303672417 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35780512c0 con 0x7f35781045d0 2024-05-09T00:44:27.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:27.596+0000 7f356e7fc700 1 -- 172.21.15.5:0/3303672417 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f3574018020 con 0x7f35781045d0 2024-05-09T00:44:27.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:27.597+0000 7f356e7fc700 1 --2- 172.21.15.5:0/3303672417 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f35640b2910 0x7f35640b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:44:27.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:27.597+0000 7f357ca23700 1 --2- 172.21.15.5:0/3303672417 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f35640b2910 0x7f35640b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:44:27.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:27.597+0000 7f356e7fc700 1 -- 172.21.15.5:0/3303672417 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8227+0+0 (secure 0 0 0) 0x7f35740e1fe0 con 0x7f35781045d0 2024-05-09T00:44:27.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:27.597+0000 7f357ca23700 1 --2- 172.21.15.5:0/3303672417 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f35640b2910 0x7f35640b4e20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3568006fd0 tx=0x7f3568008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:44:27.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:27.600+0000 7f356e7fc700 1 -- 172.21.15.5:0/3303672417 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f357401f020 con 0x7f35781045d0 2024-05-09T00:44:27.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:27.817+0000 7f357f488700 1 -- 172.21.15.5:0/3303672417 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f357819c2e0 con 0x7f35781045d0 2024-05-09T00:44:27.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:27.819+0000 7f356e7fc700 1 -- 172.21.15.5:0/3303672417 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22594 (secure 0 0 0) 0x7f35740a9d50 con 0x7f35781045d0 2024-05-09T00:44:27.821 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:44:27.821 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":80,"fsid":"d85814c6-0d9b-11ef-bc97-c7b262605968","created":"2024-05-09T00:35:25.623377+0000","modified":"2024-05-09T00:44:23.011279+0000","last_up_change":"2024-05-09T00:42:38.525335+0000","last_in_change":"2024-05-09T00:42:17.990219+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T00:38:47.844734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T00:43:39.452106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T00:43:42.708106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f8d484bb-2477-4377-ae68-5bce571333bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2963385132},{"type":"v1","addr":"172.21.15.5:6803","nonce":2963385132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2963385132},{"type":"v1","addr":"172.21.15.5:6805","nonce":2963385132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2963385132},{"type":"v1","addr":"172.21.15.5:6809","nonce":2963385132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2963385132},{"type":"v1","addr":"172.21.15.5:6807","nonce":2963385132}]},"public_addr":"172.21.15.5:6803/2963385132","cluster_addr":"172.21.15.5:6805/2963385132","heartbeat_back_addr":"172.21.15.5:6809/2963385132","heartbeat_front_addr":"172.21.15.5:6807/2963385132","state":["exists","up"]},{"osd":1,"uuid":"1c123676-7144-4a67-93a6-fd49e36a5c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3837683163},{"type":"v1","addr":"172.21.15.131:6801","nonce":3837683163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3837683163},{"type":"v1","addr":"172.21.15.131:6803","nonce":3837683163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3837683163},{"type":"v1","addr":"172.21.15.131:6807","nonce":3837683163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3837683163},{"type":"v1","addr":"172.21.15.131:6805","nonce":3837683163}]},"public_addr":"172.21.15.131:6801/3837683163","cluster_addr":"172.21.15.131:6803/3837683163","heartbeat_back_addr":"172.21.15.131:6807/3837683163","heartbeat_front_addr":"172.21.15.131:6805/3837683163","state":["exists","up"]},{"osd":2,"uuid":"95ca51f0-b48b-485c-a5f7-b15c01e754b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":484818139},{"type":"v1","addr":"172.21.15.157:6801","nonce":484818139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":484818139},{"type":"v1","addr":"172.21.15.157:6803","nonce":484818139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":484818139},{"type":"v1","addr":"172.21.15.157:6807","nonce":484818139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":484818139},{"type":"v1","addr":"172.21.15.157:6805","nonce":484818139}]},"public_addr":"172.21.15.157:6801/484818139","cluster_addr":"172.21.15.157:6803/484818139","heartbeat_back_addr":"172.21.15.157:6807/484818139","heartbeat_front_addr":"172.21.15.157:6805/484818139","state":["exists","up"]},{"osd":3,"uuid":"ed7577fb-3676-4a08-b40d-5474eb88974f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":2357377277},{"type":"v1","addr":"172.21.15.5:6811","nonce":2357377277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":2357377277},{"type":"v1","addr":"172.21.15.5:6813","nonce":2357377277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":2357377277},{"type":"v1","addr":"172.21.15.5:6817","nonce":2357377277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":2357377277},{"type":"v1","addr":"172.21.15.5:6815","nonce":2357377277}]},"public_addr":"172.21.15.5:6811/2357377277","cluster_addr":"172.21.15.5:6813/2357377277","heartbeat_back_addr":"172.21.15.5:6817/2357377277","heartbeat_front_addr":"172.21.15.5:6815/2357377277","state":["exists","up"]},{"osd":4,"uuid":"fb5dcf79-a6f1-4b1a-9e3a-087fb5132472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":2915418378},{"type":"v1","addr":"172.21.15.131:6809","nonce":2915418378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":2915418378},{"type":"v1","addr":"172.21.15.131:6811","nonce":2915418378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":2915418378},{"type":"v1","addr":"172.21.15.131:6815","nonce":2915418378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":2915418378},{"type":"v1","addr":"172.21.15.131:6813","nonce":2915418378}]},"public_addr":"172.21.15.131:6809/2915418378","cluster_addr":"172.21.15.131:6811/2915418378","heartbeat_back_addr":"172.21.15.131:6815/2915418378","heartbeat_front_addr":"172.21.15.131:6813/2915418378","state":["exists","up"]},{"osd":5,"uuid":"f01ed5d3-8066-4d25-a113-280c83e75ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4188818663},{"type":"v1","addr":"172.21.15.157:6809","nonce":4188818663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":4188818663},{"type":"v1","addr":"172.21.15.157:6811","nonce":4188818663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":4188818663},{"type":"v1","addr":"172.21.15.157:6815","nonce":4188818663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":4188818663},{"type":"v1","addr":"172.21.15.157:6813","nonce":4188818663}]},"public_addr":"172.21.15.157:6809/4188818663","cluster_addr":"172.21.15.157:6811/4188818663","heartbeat_back_addr":"172.21.15.157:6815/4188818663","heartbeat_front_addr":"172.21.15.157:6813/4188818663","state":["exists","up"]},{"osd":6,"uuid":"932379a8-c97b-4acc-8665-81f7d9cbce65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2930397813},{"type":"v1","addr":"172.21.15.5:6819","nonce":2930397813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2930397813},{"type":"v1","addr":"172.21.15.5:6821","nonce":2930397813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2930397813},{"type":"v1","addr":"172.21.15.5:6825","nonce":2930397813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2930397813},{"type":"v1","addr":"172.21.15.5:6823","nonce":2930397813}]},"public_addr":"172.21.15.5:6819/2930397813","cluster_addr":"172.21.15.5:6821/2930397813","heartbeat_back_addr":"172.21.15.5:6825/2930397813","heartbeat_front_addr":"172.21.15.5:6823/2930397813","state":["exists","up"]},{"osd":7,"uuid":"3c0a0573-6465-4bf6-b70f-310d49857a6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":4125045219},{"type":"v1","addr":"172.21.15.131:6817","nonce":4125045219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":4125045219},{"type":"v1","addr":"172.21.15.131:6819","nonce":4125045219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":4125045219},{"type":"v1","addr":"172.21.15.131:6823","nonce":4125045219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":4125045219},{"type":"v1","addr":"172.21.15.131:6821","nonce":4125045219}]},"public_addr":"172.21.15.131:6817/4125045219","cluster_addr":"172.21.15.131:6819/4125045219","heartbeat_back_addr":"172.21.15.131:6823/4125045219","heartbeat_front_addr":"172.21.15.131:6821/4125045219","state":["exists","up"]},{"osd":8,"uuid":"c49f0a00-f5ae-40ed-8f38-7b953ba1c17c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2001626768},{"type":"v1","addr":"172.21.15.157:6817","nonce":2001626768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2001626768},{"type":"v1","addr":"172.21.15.157:6819","nonce":2001626768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2001626768},{"type":"v1","addr":"172.21.15.157:6823","nonce":2001626768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2001626768},{"type":"v1","addr":"172.21.15.157:6821","nonce":2001626768}]},"public_addr":"172.21.15.157:6817/2001626768","cluster_addr":"172.21.15.157:6819/2001626768","heartbeat_back_addr":"172.21.15.157:6823/2001626768","heartbeat_front_addr":"172.21.15.157:6821/2001626768","state":["exists","up"]},{"osd":9,"uuid":"1927dfe7-c1af-4a15-bb71-d58d97c8734a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":4287220015},{"type":"v1","addr":"172.21.15.5:6827","nonce":4287220015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":4287220015},{"type":"v1","addr":"172.21.15.5:6829","nonce":4287220015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":4287220015},{"type":"v1","addr":"172.21.15.5:6833","nonce":4287220015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":4287220015},{"type":"v1","addr":"172.21.15.5:6831","nonce":4287220015}]},"public_addr":"172.21.15.5:6827/4287220015","cluster_addr":"172.21.15.5:6829/4287220015","heartbeat_back_addr":"172.21.15.5:6833/4287220015","heartbeat_front_addr":"172.21.15.5:6831/4287220015","state":["exists","up"]},{"osd":10,"uuid":"fbb4f3e3-eb7d-4ecb-a30b-55b8910375ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1571783985},{"type":"v1","addr":"172.21.15.131:6825","nonce":1571783985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1571783985},{"type":"v1","addr":"172.21.15.131:6827","nonce":1571783985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1571783985},{"type":"v1","addr":"172.21.15.131:6831","nonce":1571783985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1571783985},{"type":"v1","addr":"172.21.15.131:6829","nonce":1571783985}]},"public_addr":"172.21.15.131:6825/1571783985","cluster_addr":"172.21.15.131:6827/1571783985","heartbeat_back_addr":"172.21.15.131:6831/1571783985","heartbeat_front_addr":"172.21.15.131:6829/1571783985","state":["exists","up"]},{"osd":11,"uuid":"b858453f-eda1-4e25-89c8-94b54c830938","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3261151188},{"type":"v1","addr":"172.21.15.157:6825","nonce":3261151188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":3261151188},{"type":"v1","addr":"172.21.15.157:6827","nonce":3261151188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":3261151188},{"type":"v1","addr":"172.21.15.157:6831","nonce":3261151188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":3261151188},{"type":"v1","addr":"172.21.15.157:6829","nonce":3261151188}]},"public_addr":"172.21.15.157:6825/3261151188","cluster_addr":"172.21.15.157:6827/3261151188","heartbeat_back_addr":"172.21.15.157:6831/3261151188","heartbeat_front_addr":"172.21.15.157:6829/3261151188","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:37:57.528483+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:38:21.477482+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:38:45.780266+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:39:10.938791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:39:35.734857+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:40:00.169040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:40:26.494635+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:40:51.881718+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:41:17.697112+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:41:44.826479+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:42:10.504768+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:42:36.605447+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6},{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":6}]},{"pgid":"3.19","mappings":[{"from":3,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":7}]},{"pgid":"3.35","mappings":[{"from":1,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/3405015786":"2024-05-10T00:36:21.742363+0000","172.21.15.5:0/2669115557":"2024-05-10T00:36:21.742363+0000","172.21.15.5:6801/287842266":"2024-05-10T00:36:21.742363+0000","172.21.15.5:6800/752513054":"2024-05-10T00:35:53.128955+0000","172.21.15.5:6800/287842266":"2024-05-10T00:36:21.742363+0000","172.21.15.5:6801/752513054":"2024-05-10T00:35:53.128955+0000","172.21.15.5:0/1954876662":"2024-05-10T00:35:53.128955+0000","172.21.15.5:0/2545175121":"2024-05-10T00:36:21.742363+0000","172.21.15.5:0/3754653663":"2024-05-10T00:35:53.128955+0000","172.21.15.5:0/651514359":"2024-05-10T00:35:53.128955+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T00:44:27.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:27.821+0000 7f357f488700 1 -- 172.21.15.5:0/3303672417 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f35640b2910 msgr2=0x7f35640b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:27.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:27.821+0000 7f357f488700 1 --2- 172.21.15.5:0/3303672417 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f35640b2910 0x7f35640b4e20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3568006fd0 tx=0x7f3568008040 comp rx=0 tx=0).stop 2024-05-09T00:44:27.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:27.821+0000 7f357f488700 1 -- 172.21.15.5:0/3303672417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35781045d0 msgr2=0x7f357819af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:27.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:27.821+0000 7f357f488700 1 --2- 172.21.15.5:0/3303672417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35781045d0 0x7f357819af80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f357400b530 tx=0x7f357400b840 comp rx=0 tx=0).stop 2024-05-09T00:44:27.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:27.821+0000 7f357f488700 1 -- 172.21.15.5:0/3303672417 shutdown_connections 2024-05-09T00:44:27.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:27.821+0000 7f357f488700 1 --2- 172.21.15.5:0/3303672417 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f35640b2910 0x7f35640b4e20 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:27.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:27.821+0000 7f357f488700 1 --2- 172.21.15.5:0/3303672417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3578109850 0x7f357819f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:27.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:27.821+0000 7f357f488700 1 --2- 172.21.15.5:0/3303672417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35781045d0 0x7f357819af80 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:27.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:27.821+0000 7f357f488700 1 --2- 172.21.15.5:0/3303672417 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3578108f00 0x7f357819b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:27.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:27.821+0000 7f357f488700 1 -- 172.21.15.5:0/3303672417 >> 172.21.15.5:0/3303672417 conn(0x7f35780ffea0 msgr2=0x7f357810a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:44:27.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:27.821+0000 7f357f488700 1 -- 172.21.15.5:0/3303672417 shutdown_connections 2024-05-09T00:44:27.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:27.821+0000 7f357f488700 1 -- 172.21.15.5:0/3303672417 wait complete. 2024-05-09T00:44:28.196 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:28 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/3303672417' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:44:28.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:28 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/3303672417' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:44:28.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:28 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/3303672417' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:44:28.567 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-09T00:44:28.567 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-09T00:44:28.830 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:44:29.251 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:29 smithi005 ceph-mon[30365]: pgmap v313: 97 pgs: 1 activating+degraded, 96 active+clean; 636 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 3.6 KiB/s wr, 10 op/s; 6/306 objects degraded (1.961%) 2024-05-09T00:44:29.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:29 smithi131 ceph-mon[34547]: pgmap v313: 97 pgs: 1 activating+degraded, 96 active+clean; 636 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 3.6 KiB/s wr, 10 op/s; 6/306 objects degraded (1.961%) 2024-05-09T00:44:29.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:29 smithi157 ceph-mon[34294]: pgmap v313: 97 pgs: 1 activating+degraded, 96 active+clean; 636 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 3.6 KiB/s wr, 10 op/s; 6/306 objects degraded (1.961%) 2024-05-09T00:44:30.160 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:30 smithi005 ceph-mon[30365]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 6/306 objects degraded (1.961%), 1 pg degraded) 2024-05-09T00:44:30.160 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:30 smithi005 ceph-mon[30365]: Cluster is now healthy 2024-05-09T00:44:30.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:30 smithi131 ceph-mon[34547]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 6/306 objects degraded (1.961%), 1 pg degraded) 2024-05-09T00:44:30.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:30 smithi131 ceph-mon[34547]: Cluster is now healthy 2024-05-09T00:44:30.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:30.429+0000 7fbc47edf700 1 -- 172.21.15.5:0/2523652472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc40103360 msgr2=0x7fbc40112fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:30.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:30.429+0000 7fbc47edf700 1 --2- 172.21.15.5:0/2523652472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc40103360 0x7fbc40112fe0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbc3800b0a0 tx=0x7fbc3800b3b0 comp rx=0 tx=0).stop 2024-05-09T00:44:30.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:30.429+0000 7fbc47edf700 1 -- 172.21.15.5:0/2523652472 shutdown_connections 2024-05-09T00:44:30.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:30.429+0000 7fbc47edf700 1 --2- 172.21.15.5:0/2523652472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc40103360 0x7fbc40112fe0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:30.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:30.429+0000 7fbc47edf700 1 --2- 172.21.15.5:0/2523652472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc40105210 0x7fbc40102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:30.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:30.429+0000 7fbc47edf700 1 --2- 172.21.15.5:0/2523652472 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc40105620 0x7fbc40102cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:30.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:30.429+0000 7fbc47edf700 1 -- 172.21.15.5:0/2523652472 >> 172.21.15.5:0/2523652472 conn(0x7fbc400fe540 msgr2=0x7fbc40100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:44:30.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:30.429+0000 7fbc47edf700 1 -- 172.21.15.5:0/2523652472 shutdown_connections 2024-05-09T00:44:30.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:30.429+0000 7fbc47edf700 1 -- 172.21.15.5:0/2523652472 wait complete. 2024-05-09T00:44:30.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:30.430+0000 7fbc47edf700 1 Processor -- start 2024-05-09T00:44:30.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:30.430+0000 7fbc47edf700 1 -- start start 2024-05-09T00:44:30.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:30.430+0000 7fbc47edf700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc40103360 0x7fbc4019cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:44:30.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:30.430+0000 7fbc47edf700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc40105210 0x7fbc4019d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:44:30.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:30.430+0000 7fbc47edf700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc40105620 0x7fbc401a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:44:30.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:30.430+0000 7fbc47edf700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbc40116660 con 0x7fbc40105620 2024-05-09T00:44:30.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:30.430+0000 7fbc47edf700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fbc401164e0 con 0x7fbc40105210 2024-05-09T00:44:30.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:30.430+0000 7fbc47edf700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbc401167e0 con 0x7fbc40103360 2024-05-09T00:44:30.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:30.430+0000 7fbc4647c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc40105620 0x7fbc401a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:44:30.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:30.430+0000 7fbc4647c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc40105620 0x7fbc401a1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41130/0 (socket says 172.21.15.5:41130) 2024-05-09T00:44:30.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:30.430+0000 7fbc4647c700 1 -- 172.21.15.5:0/1512522503 learned_addr learned my addr 172.21.15.5:0/1512522503 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:44:30.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:30.430+0000 7fbc4547a700 1 --2- 172.21.15.5:0/1512522503 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc40105210 0x7fbc4019d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:44:30.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:30.430+0000 7fbc45c7b700 1 --2- 172.21.15.5:0/1512522503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc40103360 0x7fbc4019cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:44:30.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:30.430+0000 7fbc4647c700 1 -- 172.21.15.5:0/1512522503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc40103360 msgr2=0x7fbc4019cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:30.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:30.430+0000 7fbc4647c700 1 --2- 172.21.15.5:0/1512522503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc40103360 0x7fbc4019cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:30.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:30.430+0000 7fbc4647c700 1 -- 172.21.15.5:0/1512522503 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc40105210 msgr2=0x7fbc4019d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:30.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:30.430+0000 7fbc4647c700 1 --2- 172.21.15.5:0/1512522503 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc40105210 0x7fbc4019d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:30.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:30.430+0000 7fbc4647c700 1 -- 172.21.15.5:0/1512522503 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc401a1d30 con 0x7fbc40105620 2024-05-09T00:44:30.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:30.430+0000 7fbc45c7b700 1 --2- 172.21.15.5:0/1512522503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc40103360 0x7fbc4019cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:44:30.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:30.431+0000 7fbc4647c700 1 --2- 172.21.15.5:0/1512522503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc40105620 0x7fbc401a1740 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbc380062a0 tx=0x7fbc38004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:44:30.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:30.431+0000 7fbc36ffd700 1 -- 172.21.15.5:0/1512522503 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc38021070 con 0x7fbc40105620 2024-05-09T00:44:30.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:30.431+0000 7fbc36ffd700 1 -- 172.21.15.5:0/1512522503 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc38007e30 con 0x7fbc40105620 2024-05-09T00:44:30.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:30.431+0000 7fbc47edf700 1 -- 172.21.15.5:0/1512522503 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc401a1fc0 con 0x7fbc40105620 2024-05-09T00:44:30.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:30.431+0000 7fbc36ffd700 1 -- 172.21.15.5:0/1512522503 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc38004780 con 0x7fbc40105620 2024-05-09T00:44:30.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:30.431+0000 7fbc47edf700 1 -- 172.21.15.5:0/1512522503 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc401a2550 con 0x7fbc40105620 2024-05-09T00:44:30.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:30.432+0000 7fbc47edf700 1 -- 172.21.15.5:0/1512522503 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc400512c0 con 0x7fbc40105620 2024-05-09T00:44:30.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:30.434+0000 7fbc36ffd700 1 -- 172.21.15.5:0/1512522503 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fbc38007980 con 0x7fbc40105620 2024-05-09T00:44:30.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:30.435+0000 7fbc36ffd700 1 --2- 172.21.15.5:0/1512522503 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fbc2c0b2990 0x7fbc2c0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:44:30.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:30.435+0000 7fbc45c7b700 1 --2- 172.21.15.5:0/1512522503 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fbc2c0b2990 0x7fbc2c0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:44:30.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:30.435+0000 7fbc36ffd700 1 -- 172.21.15.5:0/1512522503 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8227+0+0 (secure 0 0 0) 0x7fbc3800f020 con 0x7fbc40105620 2024-05-09T00:44:30.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:30 smithi157 ceph-mon[34294]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 6/306 objects degraded (1.961%), 1 pg degraded) 2024-05-09T00:44:30.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:30 smithi157 ceph-mon[34294]: Cluster is now healthy 2024-05-09T00:44:30.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:30.436+0000 7fbc45c7b700 1 --2- 172.21.15.5:0/1512522503 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fbc2c0b2990 0x7fbc2c0b4ea0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbc4006c060 tx=0x7fbc3c00a680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:44:30.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:30.439+0000 7fbc36ffd700 1 -- 172.21.15.5:0/1512522503 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbc3802a020 con 0x7fbc40105620 2024-05-09T00:44:30.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:30.650+0000 7fbc47edf700 1 -- 172.21.15.5:0/1512522503 --> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbc4019e270 con 0x7fbc2c0b2990 2024-05-09T00:44:30.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:30.658+0000 7fbc36ffd700 1 -- 172.21.15.5:0/1512522503 <== mgr.14152 v2:172.21.15.5:6800/91856593 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258883 (secure 0 0 0) 0x7fbc4019e270 con 0x7fbc2c0b2990 2024-05-09T00:44:30.663 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:44:30.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:30.661+0000 7fbc47edf700 1 -- 172.21.15.5:0/1512522503 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fbc2c0b2990 msgr2=0x7fbc2c0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:30.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:30.661+0000 7fbc47edf700 1 --2- 172.21.15.5:0/1512522503 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fbc2c0b2990 0x7fbc2c0b4ea0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbc4006c060 tx=0x7fbc3c00a680 comp rx=0 tx=0).stop 2024-05-09T00:44:30.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:30.661+0000 7fbc47edf700 1 -- 172.21.15.5:0/1512522503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc40105620 msgr2=0x7fbc401a1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:30.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:30.661+0000 7fbc47edf700 1 --2- 172.21.15.5:0/1512522503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc40105620 0x7fbc401a1740 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbc380062a0 tx=0x7fbc38004c80 comp rx=0 tx=0).stop 2024-05-09T00:44:30.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:30.661+0000 7fbc47edf700 1 -- 172.21.15.5:0/1512522503 shutdown_connections 2024-05-09T00:44:30.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:30.661+0000 7fbc47edf700 1 --2- 172.21.15.5:0/1512522503 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fbc2c0b2990 0x7fbc2c0b4ea0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:30.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:30.661+0000 7fbc47edf700 1 --2- 172.21.15.5:0/1512522503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc40105620 0x7fbc401a1740 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:30.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:30.661+0000 7fbc47edf700 1 --2- 172.21.15.5:0/1512522503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc40103360 0x7fbc4019cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:30.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:30.661+0000 7fbc47edf700 1 --2- 172.21.15.5:0/1512522503 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc40105210 0x7fbc4019d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:30.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:30.661+0000 7fbc47edf700 1 -- 172.21.15.5:0/1512522503 >> 172.21.15.5:0/1512522503 conn(0x7fbc400fe540 msgr2=0x7fbc400ffa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:44:30.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:30.661+0000 7fbc47edf700 1 -- 172.21.15.5:0/1512522503 shutdown_connections 2024-05-09T00:44:30.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:30.661+0000 7fbc47edf700 1 -- 172.21.15.5:0/1512522503 wait complete. 2024-05-09T00:44:30.666 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-09T00:44:31.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:31 smithi131 ceph-mon[34547]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.5 KiB/s wr, 13 op/s; 6.4 KiB/s, 0 objects/s recovering 2024-05-09T00:44:31.424 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:31 smithi005 ceph-mon[30365]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.5 KiB/s wr, 13 op/s; 6.4 KiB/s, 0 objects/s recovering 2024-05-09T00:44:31.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:31 smithi157 ceph-mon[34294]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.5 KiB/s wr, 13 op/s; 6.4 KiB/s, 0 objects/s recovering 2024-05-09T00:44:31.457 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":314,"stamp":"2024-05-09T00:44:29.847883+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656275,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":363,"num_read_kb":272,"num_write":369,"num_write_kb":2286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":3004523,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":304,"ondisk_log_size":304,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":293,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":347500,"kb_used_data":10764,"kb_used_omap":18,"kb_used_meta":336557,"kb_avail":1124594324,"statfs":{"total":1151940427776,"available":1151584587776,"internally_reserved":0,"allocated":11022336,"data_stored":7063977,"data_compressed":116997,"data_compressed_allocated":147456,"data_compressed_original":1818624,"omap_allocated":19141,"internal_metadata":344634683},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29995,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":91,"num_read_kb":67,"num_write":20,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":52683,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002581"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.018605+0000","last_change":"2024-05-09T00:43:43.875357+0000","last_active":"2024-05-09T00:44:23.018605+0000","last_peered":"2024-05-09T00:44:23.018605+0000","last_clean":"2024-05-09T00:44:23.018605+0000","last_became_active":"2024-05-09T00:43:43.873765+0000","last_became_peered":"2024-05-09T00:43:43.873765+0000","last_unstale":"2024-05-09T00:44:23.018605+0000","last_undegraded":"2024-05-09T00:44:23.018605+0000","last_fullsized":"2024-05-09T00:44:23.018605+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:47.519110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.018171+0000","last_change":"2024-05-09T00:43:43.882464+0000","last_active":"2024-05-09T00:44:23.018171+0000","last_peered":"2024-05-09T00:44:23.018171+0000","last_clean":"2024-05-09T00:44:23.018171+0000","last_became_active":"2024-05-09T00:43:43.882102+0000","last_became_peered":"2024-05-09T00:43:43.882102+0000","last_unstale":"2024-05-09T00:44:23.018171+0000","last_undegraded":"2024-05-09T00:44:23.018171+0000","last_fullsized":"2024-05-09T00:44:23.018171+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:39:36.234474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.018181+0000","last_change":"2024-05-09T00:43:43.873098+0000","last_active":"2024-05-09T00:44:23.018181+0000","last_peered":"2024-05-09T00:44:23.018181+0000","last_clean":"2024-05-09T00:44:23.018181+0000","last_became_active":"2024-05-09T00:43:43.872764+0000","last_became_peered":"2024-05-09T00:43:43.872764+0000","last_unstale":"2024-05-09T00:44:23.018181+0000","last_undegraded":"2024-05-09T00:44:23.018181+0000","last_fullsized":"2024-05-09T00:44:23.018181+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:05:30.784111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.179963+0000","last_change":"2024-05-09T00:43:43.869807+0000","last_active":"2024-05-09T00:44:23.179963+0000","last_peered":"2024-05-09T00:44:23.179963+0000","last_clean":"2024-05-09T00:44:23.179963+0000","last_became_active":"2024-05-09T00:43:43.869470+0000","last_became_peered":"2024-05-09T00:43:43.869470+0000","last_unstale":"2024-05-09T00:44:23.179963+0000","last_undegraded":"2024-05-09T00:44:23.179963+0000","last_fullsized":"2024-05-09T00:44:23.179963+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:25:40.201844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.178978+0000","last_change":"2024-05-09T00:43:43.874110+0000","last_active":"2024-05-09T00:44:23.178978+0000","last_peered":"2024-05-09T00:44:23.178978+0000","last_clean":"2024-05-09T00:44:23.178978+0000","last_became_active":"2024-05-09T00:43:43.873759+0000","last_became_peered":"2024-05-09T00:43:43.873759+0000","last_unstale":"2024-05-09T00:44:23.178978+0000","last_undegraded":"2024-05-09T00:44:23.178978+0000","last_fullsized":"2024-05-09T00:44:23.178978+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:17:17.722319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.022285+0000","last_change":"2024-05-09T00:44:23.022285+0000","last_active":"2024-05-09T00:44:23.022285+0000","last_peered":"2024-05-09T00:44:23.022285+0000","last_clean":"2024-05-09T00:44:23.022285+0000","last_became_active":"2024-05-09T00:44:23.021887+0000","last_became_peered":"2024-05-09T00:44:23.021887+0000","last_unstale":"2024-05-09T00:44:23.022285+0000","last_undegraded":"2024-05-09T00:44:23.022285+0000","last_fullsized":"2024-05-09T00:44:23.022285+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:07:07.702178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,7],"acting":[10,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.020962+0000","last_change":"2024-05-09T00:43:43.883112+0000","last_active":"2024-05-09T00:44:23.020962+0000","last_peered":"2024-05-09T00:44:23.020962+0000","last_clean":"2024-05-09T00:44:23.020962+0000","last_became_active":"2024-05-09T00:43:43.882795+0000","last_became_peered":"2024-05-09T00:43:43.882795+0000","last_unstale":"2024-05-09T00:44:23.020962+0000","last_undegraded":"2024-05-09T00:44:23.020962+0000","last_fullsized":"2024-05-09T00:44:23.020962+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:33:58.724015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.014225+0000","last_change":"2024-05-09T00:43:43.880696+0000","last_active":"2024-05-09T00:44:23.014225+0000","last_peered":"2024-05-09T00:44:23.014225+0000","last_clean":"2024-05-09T00:44:23.014225+0000","last_became_active":"2024-05-09T00:43:43.880174+0000","last_became_peered":"2024-05-09T00:43:43.880174+0000","last_unstale":"2024-05-09T00:44:23.014225+0000","last_undegraded":"2024-05-09T00:44:23.014225+0000","last_fullsized":"2024-05-09T00:44:23.014225+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:40:05.682432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.019760+0000","last_change":"2024-05-09T00:43:43.883449+0000","last_active":"2024-05-09T00:44:23.019760+0000","last_peered":"2024-05-09T00:44:23.019760+0000","last_clean":"2024-05-09T00:44:23.019760+0000","last_became_active":"2024-05-09T00:43:43.883066+0000","last_became_peered":"2024-05-09T00:43:43.883066+0000","last_unstale":"2024-05-09T00:44:23.019760+0000","last_undegraded":"2024-05-09T00:44:23.019760+0000","last_fullsized":"2024-05-09T00:44:23.019760+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:47:49.116004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.016340+0000","last_change":"2024-05-09T00:43:43.877514+0000","last_active":"2024-05-09T00:44:23.016340+0000","last_peered":"2024-05-09T00:44:23.016340+0000","last_clean":"2024-05-09T00:44:23.016340+0000","last_became_active":"2024-05-09T00:43:43.877170+0000","last_became_peered":"2024-05-09T00:43:43.877170+0000","last_unstale":"2024-05-09T00:44:23.016340+0000","last_undegraded":"2024-05-09T00:44:23.016340+0000","last_fullsized":"2024-05-09T00:44:23.016340+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:47:26.738571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.180294+0000","last_change":"2024-05-09T00:44:23.180294+0000","last_active":"2024-05-09T00:44:23.180294+0000","last_peered":"2024-05-09T00:44:23.180294+0000","last_clean":"2024-05-09T00:44:23.180294+0000","last_became_active":"2024-05-09T00:44:23.179824+0000","last_became_peered":"2024-05-09T00:44:23.179824+0000","last_unstale":"2024-05-09T00:44:23.180294+0000","last_undegraded":"2024-05-09T00:44:23.180294+0000","last_fullsized":"2024-05-09T00:44:23.180294+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:43:20.919660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,9],"acting":[7,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.179922+0000","last_change":"2024-05-09T00:43:43.873836+0000","last_active":"2024-05-09T00:44:23.179922+0000","last_peered":"2024-05-09T00:44:23.179922+0000","last_clean":"2024-05-09T00:44:23.179922+0000","last_became_active":"2024-05-09T00:43:43.873512+0000","last_became_peered":"2024-05-09T00:43:43.873512+0000","last_unstale":"2024-05-09T00:44:23.179922+0000","last_undegraded":"2024-05-09T00:44:23.179922+0000","last_fullsized":"2024-05-09T00:44:23.179922+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:32:06.162638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.179009+0000","last_change":"2024-05-09T00:43:43.885614+0000","last_active":"2024-05-09T00:44:23.179009+0000","last_peered":"2024-05-09T00:44:23.179009+0000","last_clean":"2024-05-09T00:44:23.179009+0000","last_became_active":"2024-05-09T00:43:43.884682+0000","last_became_peered":"2024-05-09T00:43:43.884682+0000","last_unstale":"2024-05-09T00:44:23.179009+0000","last_undegraded":"2024-05-09T00:44:23.179009+0000","last_fullsized":"2024-05-09T00:44:23.179009+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:34:15.336173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.017792+0000","last_change":"2024-05-09T00:43:43.883860+0000","last_active":"2024-05-09T00:44:23.017792+0000","last_peered":"2024-05-09T00:44:23.017792+0000","last_clean":"2024-05-09T00:44:23.017792+0000","last_became_active":"2024-05-09T00:43:43.882870+0000","last_became_peered":"2024-05-09T00:43:43.882870+0000","last_unstale":"2024-05-09T00:44:23.017792+0000","last_undegraded":"2024-05-09T00:44:23.017792+0000","last_fullsized":"2024-05-09T00:44:23.017792+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:17:31.566819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.017866+0000","last_change":"2024-05-09T00:43:43.874114+0000","last_active":"2024-05-09T00:44:23.017866+0000","last_peered":"2024-05-09T00:44:23.017866+0000","last_clean":"2024-05-09T00:44:23.017866+0000","last_became_active":"2024-05-09T00:43:43.873762+0000","last_became_peered":"2024-05-09T00:43:43.873762+0000","last_unstale":"2024-05-09T00:44:23.017866+0000","last_undegraded":"2024-05-09T00:44:23.017866+0000","last_fullsized":"2024-05-09T00:44:23.017866+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:24:27.486100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.018352+0000","last_change":"2024-05-09T00:43:43.872982+0000","last_active":"2024-05-09T00:44:23.018352+0000","last_peered":"2024-05-09T00:44:23.018352+0000","last_clean":"2024-05-09T00:44:23.018352+0000","last_became_active":"2024-05-09T00:43:43.872464+0000","last_became_peered":"2024-05-09T00:43:43.872464+0000","last_unstale":"2024-05-09T00:44:23.018352+0000","last_undegraded":"2024-05-09T00:44:23.018352+0000","last_fullsized":"2024-05-09T00:44:23.018352+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:41.172001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.018730+0000","last_change":"2024-05-09T00:43:43.875662+0000","last_active":"2024-05-09T00:44:23.018730+0000","last_peered":"2024-05-09T00:44:23.018730+0000","last_clean":"2024-05-09T00:44:23.018730+0000","last_became_active":"2024-05-09T00:43:43.874701+0000","last_became_peered":"2024-05-09T00:43:43.874701+0000","last_unstale":"2024-05-09T00:44:23.018730+0000","last_undegraded":"2024-05-09T00:44:23.018730+0000","last_fullsized":"2024-05-09T00:44:23.018730+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:09:19.349190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.016202+0000","last_change":"2024-05-09T00:43:43.876100+0000","last_active":"2024-05-09T00:44:23.016202+0000","last_peered":"2024-05-09T00:44:23.016202+0000","last_clean":"2024-05-09T00:44:23.016202+0000","last_became_active":"2024-05-09T00:43:43.875768+0000","last_became_peered":"2024-05-09T00:43:43.875768+0000","last_unstale":"2024-05-09T00:44:23.016202+0000","last_undegraded":"2024-05-09T00:44:23.016202+0000","last_fullsized":"2024-05-09T00:44:23.016202+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:48:26.843064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.019634+0000","last_change":"2024-05-09T00:43:43.875900+0000","last_active":"2024-05-09T00:44:23.019634+0000","last_peered":"2024-05-09T00:44:23.019634+0000","last_clean":"2024-05-09T00:44:23.019634+0000","last_became_active":"2024-05-09T00:43:43.875497+0000","last_became_peered":"2024-05-09T00:43:43.875497+0000","last_unstale":"2024-05-09T00:44:23.019634+0000","last_undegraded":"2024-05-09T00:44:23.019634+0000","last_fullsized":"2024-05-09T00:44:23.019634+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:54:44.215936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'13","reported_seq":65,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:25.995387+0000","last_change":"2024-05-09T00:43:40.854337+0000","last_active":"2024-05-09T00:44:25.995387+0000","last_peered":"2024-05-09T00:44:25.995387+0000","last_clean":"2024-05-09T00:44:25.995387+0000","last_became_active":"2024-05-09T00:43:40.853986+0000","last_became_peered":"2024-05-09T00:43:40.853986+0000","last_unstale":"2024-05-09T00:44:25.995387+0000","last_undegraded":"2024-05-09T00:44:25.995387+0000","last_fullsized":"2024-05-09T00:44:25.995387+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:01:19.128533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.014907+0000","last_change":"2024-05-09T00:43:43.878081+0000","last_active":"2024-05-09T00:44:23.014907+0000","last_peered":"2024-05-09T00:44:23.014907+0000","last_clean":"2024-05-09T00:44:23.014907+0000","last_became_active":"2024-05-09T00:43:43.877585+0000","last_became_peered":"2024-05-09T00:43:43.877585+0000","last_unstale":"2024-05-09T00:44:23.014907+0000","last_undegraded":"2024-05-09T00:44:23.014907+0000","last_fullsized":"2024-05-09T00:44:23.014907+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:17:39.255841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.014977+0000","last_change":"2024-05-09T00:43:40.959386+0000","last_active":"2024-05-09T00:44:23.014977+0000","last_peered":"2024-05-09T00:44:23.014977+0000","last_clean":"2024-05-09T00:44:23.014977+0000","last_became_active":"2024-05-09T00:43:40.958878+0000","last_became_peered":"2024-05-09T00:43:40.958878+0000","last_unstale":"2024-05-09T00:44:23.014977+0000","last_undegraded":"2024-05-09T00:44:23.014977+0000","last_fullsized":"2024-05-09T00:44:23.014977+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:14:57.105078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.018181+0000","last_change":"2024-05-09T00:43:43.870563+0000","last_active":"2024-05-09T00:44:23.018181+0000","last_peered":"2024-05-09T00:44:23.018181+0000","last_clean":"2024-05-09T00:44:23.018181+0000","last_became_active":"2024-05-09T00:43:43.870237+0000","last_became_peered":"2024-05-09T00:43:43.870237+0000","last_unstale":"2024-05-09T00:44:23.018181+0000","last_undegraded":"2024-05-09T00:44:23.018181+0000","last_fullsized":"2024-05-09T00:44:23.018181+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:14:28.006343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"78'12","reported_seq":39,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.179571+0000","last_change":"2024-05-09T00:43:40.857670+0000","last_active":"2024-05-09T00:44:23.179571+0000","last_peered":"2024-05-09T00:44:23.179571+0000","last_clean":"2024-05-09T00:44:23.179571+0000","last_became_active":"2024-05-09T00:43:40.857247+0000","last_became_peered":"2024-05-09T00:43:40.857247+0000","last_unstale":"2024-05-09T00:44:23.179571+0000","last_undegraded":"2024-05-09T00:44:23.179571+0000","last_fullsized":"2024-05-09T00:44:23.179571+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:47:06.325593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.018774+0000","last_change":"2024-05-09T00:43:43.883179+0000","last_active":"2024-05-09T00:44:23.018774+0000","last_peered":"2024-05-09T00:44:23.018774+0000","last_clean":"2024-05-09T00:44:23.018774+0000","last_became_active":"2024-05-09T00:43:43.882820+0000","last_became_peered":"2024-05-09T00:43:43.882820+0000","last_unstale":"2024-05-09T00:44:23.018774+0000","last_undegraded":"2024-05-09T00:44:23.018774+0000","last_fullsized":"2024-05-09T00:44:23.018774+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:56:26.012495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"78'5","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.179162+0000","last_change":"2024-05-09T00:43:40.856024+0000","last_active":"2024-05-09T00:44:23.179162+0000","last_peered":"2024-05-09T00:44:23.179162+0000","last_clean":"2024-05-09T00:44:23.179162+0000","last_became_active":"2024-05-09T00:43:40.855634+0000","last_became_peered":"2024-05-09T00:43:40.855634+0000","last_unstale":"2024-05-09T00:44:23.179162+0000","last_undegraded":"2024-05-09T00:44:23.179162+0000","last_fullsized":"2024-05-09T00:44:23.179162+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:30:49.178547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.016130+0000","last_change":"2024-05-09T00:43:43.882950+0000","last_active":"2024-05-09T00:44:23.016130+0000","last_peered":"2024-05-09T00:44:23.016130+0000","last_clean":"2024-05-09T00:44:23.016130+0000","last_became_active":"2024-05-09T00:43:43.880679+0000","last_became_peered":"2024-05-09T00:43:43.880679+0000","last_unstale":"2024-05-09T00:44:23.016130+0000","last_undegraded":"2024-05-09T00:44:23.016130+0000","last_fullsized":"2024-05-09T00:44:23.016130+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:34:14.050987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"78'6","reported_seq":61,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:26.999183+0000","last_change":"2024-05-09T00:43:40.854953+0000","last_active":"2024-05-09T00:44:26.999183+0000","last_peered":"2024-05-09T00:44:26.999183+0000","last_clean":"2024-05-09T00:44:26.999183+0000","last_became_active":"2024-05-09T00:43:40.854562+0000","last_became_peered":"2024-05-09T00:43:40.854562+0000","last_unstale":"2024-05-09T00:44:26.999183+0000","last_undegraded":"2024-05-09T00:44:26.999183+0000","last_fullsized":"2024-05-09T00:44:26.999183+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:09:19.564806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.178886+0000","last_change":"2024-05-09T00:43:43.881881+0000","last_active":"2024-05-09T00:44:23.178886+0000","last_peered":"2024-05-09T00:44:23.178886+0000","last_clean":"2024-05-09T00:44:23.178886+0000","last_became_active":"2024-05-09T00:43:43.881461+0000","last_became_peered":"2024-05-09T00:43:43.881461+0000","last_unstale":"2024-05-09T00:44:23.178886+0000","last_undegraded":"2024-05-09T00:44:23.178886+0000","last_fullsized":"2024-05-09T00:44:23.178886+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:59:28.747084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"78'7","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.015495+0000","last_change":"2024-05-09T00:43:40.858521+0000","last_active":"2024-05-09T00:44:23.015495+0000","last_peered":"2024-05-09T00:44:23.015495+0000","last_clean":"2024-05-09T00:44:23.015495+0000","last_became_active":"2024-05-09T00:43:40.858102+0000","last_became_peered":"2024-05-09T00:43:40.858102+0000","last_unstale":"2024-05-09T00:44:23.015495+0000","last_undegraded":"2024-05-09T00:44:23.015495+0000","last_fullsized":"2024-05-09T00:44:23.015495+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:09:51.684630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.023472+0000","last_change":"2024-05-09T00:44:23.023472+0000","last_active":"2024-05-09T00:44:23.023472+0000","last_peered":"2024-05-09T00:44:23.023472+0000","last_clean":"2024-05-09T00:44:23.023472+0000","last_became_active":"2024-05-09T00:44:23.023017+0000","last_became_peered":"2024-05-09T00:44:23.023017+0000","last_unstale":"2024-05-09T00:44:23.023472+0000","last_undegraded":"2024-05-09T00:44:23.023472+0000","last_fullsized":"2024-05-09T00:44:23.023472+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:05:13.852606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.178682+0000","last_change":"2024-05-09T00:43:40.853329+0000","last_active":"2024-05-09T00:44:23.178682+0000","last_peered":"2024-05-09T00:44:23.178682+0000","last_clean":"2024-05-09T00:44:23.178682+0000","last_became_active":"2024-05-09T00:43:40.852979+0000","last_became_peered":"2024-05-09T00:43:40.852979+0000","last_unstale":"2024-05-09T00:44:23.178682+0000","last_undegraded":"2024-05-09T00:44:23.178682+0000","last_fullsized":"2024-05-09T00:44:23.178682+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:13:57.006031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.014305+0000","last_change":"2024-05-09T00:43:43.862876+0000","last_active":"2024-05-09T00:44:23.014305+0000","last_peered":"2024-05-09T00:44:23.014305+0000","last_clean":"2024-05-09T00:44:23.014305+0000","last_became_active":"2024-05-09T00:43:43.862521+0000","last_became_peered":"2024-05-09T00:43:43.862521+0000","last_unstale":"2024-05-09T00:44:23.014305+0000","last_undegraded":"2024-05-09T00:44:23.014305+0000","last_fullsized":"2024-05-09T00:44:23.014305+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:45:49.386844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'3","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.015554+0000","last_change":"2024-05-09T00:43:40.860395+0000","last_active":"2024-05-09T00:44:23.015554+0000","last_peered":"2024-05-09T00:44:23.015554+0000","last_clean":"2024-05-09T00:44:23.015554+0000","last_became_active":"2024-05-09T00:43:40.860040+0000","last_became_peered":"2024-05-09T00:43:40.860040+0000","last_unstale":"2024-05-09T00:44:23.015554+0000","last_undegraded":"2024-05-09T00:44:23.015554+0000","last_fullsized":"2024-05-09T00:44:23.015554+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:20:36.054685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.180905+0000","last_change":"2024-05-09T00:44:23.180905+0000","last_active":"2024-05-09T00:44:23.180905+0000","last_peered":"2024-05-09T00:44:23.180905+0000","last_clean":"2024-05-09T00:44:23.180905+0000","last_became_active":"2024-05-09T00:44:23.180296+0000","last_became_peered":"2024-05-09T00:44:23.180296+0000","last_unstale":"2024-05-09T00:44:23.180905+0000","last_undegraded":"2024-05-09T00:44:23.180905+0000","last_fullsized":"2024-05-09T00:44:23.180905+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:48:17.585276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,10],"acting":[9,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"78'4","reported_seq":82,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:27.999931+0000","last_change":"2024-05-09T00:43:40.860542+0000","last_active":"2024-05-09T00:44:27.999931+0000","last_peered":"2024-05-09T00:44:27.999931+0000","last_clean":"2024-05-09T00:44:27.999931+0000","last_became_active":"2024-05-09T00:43:40.860176+0000","last_became_peered":"2024-05-09T00:43:40.860176+0000","last_unstale":"2024-05-09T00:44:27.999931+0000","last_undegraded":"2024-05-09T00:44:27.999931+0000","last_fullsized":"2024-05-09T00:44:27.999931+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:44:50.375024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":52,"num_read_kb":52,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.019661+0000","last_change":"2024-05-09T00:43:43.882119+0000","last_active":"2024-05-09T00:44:23.019661+0000","last_peered":"2024-05-09T00:44:23.019661+0000","last_clean":"2024-05-09T00:44:23.019661+0000","last_became_active":"2024-05-09T00:43:43.881747+0000","last_became_peered":"2024-05-09T00:43:43.881747+0000","last_unstale":"2024-05-09T00:44:23.019661+0000","last_undegraded":"2024-05-09T00:44:23.019661+0000","last_fullsized":"2024-05-09T00:44:23.019661+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:54:19.420435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'4","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.021373+0000","last_change":"2024-05-09T00:43:40.859539+0000","last_active":"2024-05-09T00:44:23.021373+0000","last_peered":"2024-05-09T00:44:23.021373+0000","last_clean":"2024-05-09T00:44:23.021373+0000","last_became_active":"2024-05-09T00:43:40.859162+0000","last_became_peered":"2024-05-09T00:43:40.859162+0000","last_unstale":"2024-05-09T00:44:23.021373+0000","last_undegraded":"2024-05-09T00:44:23.021373+0000","last_fullsized":"2024-05-09T00:44:23.021373+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:37:15.190944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.180872+0000","last_change":"2024-05-09T00:44:23.180872+0000","last_active":"2024-05-09T00:44:23.180872+0000","last_peered":"2024-05-09T00:44:23.180872+0000","last_clean":"2024-05-09T00:44:23.180872+0000","last_became_active":"2024-05-09T00:44:23.180398+0000","last_became_peered":"2024-05-09T00:44:23.180398+0000","last_unstale":"2024-05-09T00:44:23.180872+0000","last_undegraded":"2024-05-09T00:44:23.180872+0000","last_fullsized":"2024-05-09T00:44:23.180872+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:19:27.909490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,6],"acting":[0,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'5","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.016141+0000","last_change":"2024-05-09T00:43:40.960126+0000","last_active":"2024-05-09T00:44:23.016141+0000","last_peered":"2024-05-09T00:44:23.016141+0000","last_clean":"2024-05-09T00:44:23.016141+0000","last_became_active":"2024-05-09T00:43:40.959441+0000","last_became_peered":"2024-05-09T00:43:40.959441+0000","last_unstale":"2024-05-09T00:44:23.016141+0000","last_undegraded":"2024-05-09T00:44:23.016141+0000","last_fullsized":"2024-05-09T00:44:23.016141+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:38:47.497029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.018911+0000","last_change":"2024-05-09T00:43:43.874223+0000","last_active":"2024-05-09T00:44:23.018911+0000","last_peered":"2024-05-09T00:44:23.018911+0000","last_clean":"2024-05-09T00:44:23.018911+0000","last_became_active":"2024-05-09T00:43:43.873875+0000","last_became_peered":"2024-05-09T00:43:43.873875+0000","last_unstale":"2024-05-09T00:44:23.018911+0000","last_undegraded":"2024-05-09T00:44:23.018911+0000","last_fullsized":"2024-05-09T00:44:23.018911+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:59:22.192886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'31","reported_seq":135,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:27.191685+0000","last_change":"2024-05-09T00:44:23.188953+0000","last_active":"2024-05-09T00:44:27.191685+0000","last_peered":"2024-05-09T00:44:27.191685+0000","last_clean":"2024-05-09T00:44:27.191685+0000","last_became_active":"2024-05-09T00:44:23.180858+0000","last_became_peered":"2024-05-09T00:44:23.180858+0000","last_unstale":"2024-05-09T00:44:27.191685+0000","last_undegraded":"2024-05-09T00:44:27.191685+0000","last_fullsized":"2024-05-09T00:44:27.191685+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:26:16.427665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58078,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":53,"num_read_kb":72,"num_write":33,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":52683,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.179509+0000","last_change":"2024-05-09T00:43:43.872019+0000","last_active":"2024-05-09T00:44:23.179509+0000","last_peered":"2024-05-09T00:44:23.179509+0000","last_clean":"2024-05-09T00:44:23.179509+0000","last_became_active":"2024-05-09T00:43:43.871622+0000","last_became_peered":"2024-05-09T00:43:43.871622+0000","last_unstale":"2024-05-09T00:44:23.179509+0000","last_undegraded":"2024-05-09T00:44:23.179509+0000","last_fullsized":"2024-05-09T00:44:23.179509+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:41:47.789443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'9","reported_seq":37,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.019299+0000","last_change":"2024-05-09T00:43:40.860111+0000","last_active":"2024-05-09T00:44:23.019299+0000","last_peered":"2024-05-09T00:44:23.019299+0000","last_clean":"2024-05-09T00:44:23.019299+0000","last_became_active":"2024-05-09T00:43:40.859732+0000","last_became_peered":"2024-05-09T00:43:40.859732+0000","last_unstale":"2024-05-09T00:44:23.019299+0000","last_undegraded":"2024-05-09T00:44:23.019299+0000","last_fullsized":"2024-05-09T00:44:23.019299+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:30:03.180679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.178898+0000","last_change":"2024-05-09T00:43:43.870604+0000","last_active":"2024-05-09T00:44:23.178898+0000","last_peered":"2024-05-09T00:44:23.178898+0000","last_clean":"2024-05-09T00:44:23.178898+0000","last_became_active":"2024-05-09T00:43:43.870286+0000","last_became_peered":"2024-05-09T00:43:43.870286+0000","last_unstale":"2024-05-09T00:44:23.178898+0000","last_undegraded":"2024-05-09T00:44:23.178898+0000","last_fullsized":"2024-05-09T00:44:23.178898+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:57:04.486631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'5","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.015759+0000","last_change":"2024-05-09T00:43:40.851966+0000","last_active":"2024-05-09T00:44:23.015759+0000","last_peered":"2024-05-09T00:44:23.015759+0000","last_clean":"2024-05-09T00:44:23.015759+0000","last_became_active":"2024-05-09T00:43:40.851537+0000","last_became_peered":"2024-05-09T00:43:40.851537+0000","last_unstale":"2024-05-09T00:44:23.015759+0000","last_undegraded":"2024-05-09T00:44:23.015759+0000","last_fullsized":"2024-05-09T00:44:23.015759+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:58:59.797183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.021364+0000","last_change":"2024-05-09T00:43:43.879509+0000","last_active":"2024-05-09T00:44:23.021364+0000","last_peered":"2024-05-09T00:44:23.021364+0000","last_clean":"2024-05-09T00:44:23.021364+0000","last_became_active":"2024-05-09T00:43:43.879151+0000","last_became_peered":"2024-05-09T00:43:43.879151+0000","last_unstale":"2024-05-09T00:44:23.021364+0000","last_undegraded":"2024-05-09T00:44:23.021364+0000","last_fullsized":"2024-05-09T00:44:23.021364+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:34:18.850545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"78'6","reported_seq":60,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:25.996358+0000","last_change":"2024-05-09T00:43:40.857629+0000","last_active":"2024-05-09T00:44:25.996358+0000","last_peered":"2024-05-09T00:44:25.996358+0000","last_clean":"2024-05-09T00:44:25.996358+0000","last_became_active":"2024-05-09T00:43:40.857260+0000","last_became_peered":"2024-05-09T00:43:40.857260+0000","last_unstale":"2024-05-09T00:44:25.996358+0000","last_undegraded":"2024-05-09T00:44:25.996358+0000","last_fullsized":"2024-05-09T00:44:25.996358+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:51:00.600870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":29,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.020628+0000","last_change":"2024-05-09T00:43:43.886100+0000","last_active":"2024-05-09T00:44:23.020628+0000","last_peered":"2024-05-09T00:44:23.020628+0000","last_clean":"2024-05-09T00:44:23.020628+0000","last_became_active":"2024-05-09T00:43:43.885775+0000","last_became_peered":"2024-05-09T00:43:43.885775+0000","last_unstale":"2024-05-09T00:44:23.020628+0000","last_undegraded":"2024-05-09T00:44:23.020628+0000","last_fullsized":"2024-05-09T00:44:23.020628+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:34:19.439163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'17","reported_seq":44,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.178810+0000","last_change":"2024-05-09T00:43:40.959873+0000","last_active":"2024-05-09T00:44:23.178810+0000","last_peered":"2024-05-09T00:44:23.178810+0000","last_clean":"2024-05-09T00:44:23.178810+0000","last_became_active":"2024-05-09T00:43:40.959437+0000","last_became_peered":"2024-05-09T00:43:40.959437+0000","last_unstale":"2024-05-09T00:44:23.178810+0000","last_undegraded":"2024-05-09T00:44:23.178810+0000","last_fullsized":"2024-05-09T00:44:23.178810+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:29:52.686937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.015736+0000","last_change":"2024-05-09T00:43:43.880076+0000","last_active":"2024-05-09T00:44:23.015736+0000","last_peered":"2024-05-09T00:44:23.015736+0000","last_clean":"2024-05-09T00:44:23.015736+0000","last_became_active":"2024-05-09T00:43:43.878733+0000","last_became_peered":"2024-05-09T00:43:43.878733+0000","last_unstale":"2024-05-09T00:44:23.015736+0000","last_undegraded":"2024-05-09T00:44:23.015736+0000","last_fullsized":"2024-05-09T00:44:23.015736+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:30:21.426855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.179296+0000","last_change":"2024-05-09T00:43:40.858033+0000","last_active":"2024-05-09T00:44:23.179296+0000","last_peered":"2024-05-09T00:44:23.179296+0000","last_clean":"2024-05-09T00:44:23.179296+0000","last_became_active":"2024-05-09T00:43:40.857075+0000","last_became_peered":"2024-05-09T00:43:40.857075+0000","last_unstale":"2024-05-09T00:44:23.179296+0000","last_undegraded":"2024-05-09T00:44:23.179296+0000","last_fullsized":"2024-05-09T00:44:23.179296+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:01:49.690226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.180515+0000","last_change":"2024-05-09T00:44:23.180515+0000","last_active":"2024-05-09T00:44:23.180515+0000","last_peered":"2024-05-09T00:44:23.180515+0000","last_clean":"2024-05-09T00:44:23.180515+0000","last_became_active":"2024-05-09T00:44:23.179834+0000","last_became_peered":"2024-05-09T00:44:23.179834+0000","last_unstale":"2024-05-09T00:44:23.180515+0000","last_undegraded":"2024-05-09T00:44:23.180515+0000","last_fullsized":"2024-05-09T00:44:23.180515+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:21:55.735438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,6],"acting":[11,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"78'1","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.015857+0000","last_change":"2024-05-09T00:43:40.960031+0000","last_active":"2024-05-09T00:44:23.015857+0000","last_peered":"2024-05-09T00:44:23.015857+0000","last_clean":"2024-05-09T00:44:23.015857+0000","last_became_active":"2024-05-09T00:43:40.959693+0000","last_became_peered":"2024-05-09T00:43:40.959693+0000","last_unstale":"2024-05-09T00:44:23.015857+0000","last_undegraded":"2024-05-09T00:44:23.015857+0000","last_fullsized":"2024-05-09T00:44:23.015857+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:55:51.903596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"78'1","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.015935+0000","last_change":"2024-05-09T00:43:43.886618+0000","last_active":"2024-05-09T00:44:23.015935+0000","last_peered":"2024-05-09T00:44:23.015935+0000","last_clean":"2024-05-09T00:44:23.015935+0000","last_became_active":"2024-05-09T00:43:43.885946+0000","last_became_peered":"2024-05-09T00:43:43.885946+0000","last_unstale":"2024-05-09T00:44:23.015935+0000","last_undegraded":"2024-05-09T00:44:23.015935+0000","last_fullsized":"2024-05-09T00:44:23.015935+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:16:22.169548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"78'5","reported_seq":54,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.021285+0000","last_change":"2024-05-09T00:43:40.856213+0000","last_active":"2024-05-09T00:44:23.021285+0000","last_peered":"2024-05-09T00:44:23.021285+0000","last_clean":"2024-05-09T00:44:23.021285+0000","last_became_active":"2024-05-09T00:43:40.855700+0000","last_became_peered":"2024-05-09T00:43:40.855700+0000","last_unstale":"2024-05-09T00:44:23.021285+0000","last_undegraded":"2024-05-09T00:44:23.021285+0000","last_fullsized":"2024-05-09T00:44:23.021285+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:48:50.238293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":23,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.015809+0000","last_change":"2024-05-09T00:43:43.885272+0000","last_active":"2024-05-09T00:44:23.015809+0000","last_peered":"2024-05-09T00:44:23.015809+0000","last_clean":"2024-05-09T00:44:23.015809+0000","last_became_active":"2024-05-09T00:43:43.884774+0000","last_became_peered":"2024-05-09T00:43:43.884774+0000","last_unstale":"2024-05-09T00:44:23.015809+0000","last_undegraded":"2024-05-09T00:44:23.015809+0000","last_fullsized":"2024-05-09T00:44:23.015809+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:14:02.193859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.015852+0000","last_change":"2024-05-09T00:43:43.883370+0000","last_active":"2024-05-09T00:44:23.015852+0000","last_peered":"2024-05-09T00:44:23.015852+0000","last_clean":"2024-05-09T00:44:23.015852+0000","last_became_active":"2024-05-09T00:43:43.881570+0000","last_became_peered":"2024-05-09T00:43:43.881570+0000","last_unstale":"2024-05-09T00:44:23.015852+0000","last_undegraded":"2024-05-09T00:44:23.015852+0000","last_fullsized":"2024-05-09T00:44:23.015852+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:48.936861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"78'5","reported_seq":58,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:26.997171+0000","last_change":"2024-05-09T00:43:40.959828+0000","last_active":"2024-05-09T00:44:26.997171+0000","last_peered":"2024-05-09T00:44:26.997171+0000","last_clean":"2024-05-09T00:44:26.997171+0000","last_became_active":"2024-05-09T00:43:40.958889+0000","last_became_peered":"2024-05-09T00:43:40.958889+0000","last_unstale":"2024-05-09T00:44:26.997171+0000","last_undegraded":"2024-05-09T00:44:26.997171+0000","last_fullsized":"2024-05-09T00:44:26.997171+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:24:50.113298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":27,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.014571+0000","last_change":"2024-05-09T00:43:43.876144+0000","last_active":"2024-05-09T00:44:23.014571+0000","last_peered":"2024-05-09T00:44:23.014571+0000","last_clean":"2024-05-09T00:44:23.014571+0000","last_became_active":"2024-05-09T00:43:43.875774+0000","last_became_peered":"2024-05-09T00:43:43.875774+0000","last_unstale":"2024-05-09T00:44:23.014571+0000","last_undegraded":"2024-05-09T00:44:23.014571+0000","last_fullsized":"2024-05-09T00:44:23.014571+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:05:23.145431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"78'6","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.179061+0000","last_change":"2024-05-09T00:43:40.857278+0000","last_active":"2024-05-09T00:44:23.179061+0000","last_peered":"2024-05-09T00:44:23.179061+0000","last_clean":"2024-05-09T00:44:23.179061+0000","last_became_active":"2024-05-09T00:43:40.856861+0000","last_became_peered":"2024-05-09T00:43:40.856861+0000","last_unstale":"2024-05-09T00:44:23.179061+0000","last_undegraded":"2024-05-09T00:44:23.179061+0000","last_fullsized":"2024-05-09T00:44:23.179061+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:36:16.603327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":98,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.020927+0000","last_change":"2024-05-09T00:42:14.528544+0000","last_active":"2024-05-09T00:44:23.020927+0000","last_peered":"2024-05-09T00:44:23.020927+0000","last_clean":"2024-05-09T00:44:23.020927+0000","last_became_active":"2024-05-09T00:42:14.523658+0000","last_became_peered":"2024-05-09T00:42:14.523658+0000","last_unstale":"2024-05-09T00:44:23.020927+0000","last_undegraded":"2024-05-09T00:44:23.020927+0000","last_fullsized":"2024-05-09T00:44:23.020927+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:38:48.637995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:38:48.637995+0000","last_clean_scrub_stamp":"2024-05-09T00:38:48.637995+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:42:28.249240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.178529+0000","last_change":"2024-05-09T00:43:43.873638+0000","last_active":"2024-05-09T00:44:23.178529+0000","last_peered":"2024-05-09T00:44:23.178529+0000","last_clean":"2024-05-09T00:44:23.178529+0000","last_became_active":"2024-05-09T00:43:43.871069+0000","last_became_peered":"2024-05-09T00:43:43.871069+0000","last_unstale":"2024-05-09T00:44:23.178529+0000","last_undegraded":"2024-05-09T00:44:23.178529+0000","last_fullsized":"2024-05-09T00:44:23.178529+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:28:34.996989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"78'4","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.021075+0000","last_change":"2024-05-09T00:43:40.856374+0000","last_active":"2024-05-09T00:44:23.021075+0000","last_peered":"2024-05-09T00:44:23.021075+0000","last_clean":"2024-05-09T00:44:23.021075+0000","last_became_active":"2024-05-09T00:43:40.855695+0000","last_became_peered":"2024-05-09T00:43:40.855695+0000","last_unstale":"2024-05-09T00:44:23.021075+0000","last_undegraded":"2024-05-09T00:44:23.021075+0000","last_fullsized":"2024-05-09T00:44:23.021075+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:33:43.734132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"78'4","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.179337+0000","last_change":"2024-05-09T00:43:40.959303+0000","last_active":"2024-05-09T00:44:23.179337+0000","last_peered":"2024-05-09T00:44:23.179337+0000","last_clean":"2024-05-09T00:44:23.179337+0000","last_became_active":"2024-05-09T00:43:40.958855+0000","last_became_peered":"2024-05-09T00:43:40.958855+0000","last_unstale":"2024-05-09T00:44:23.179337+0000","last_undegraded":"2024-05-09T00:44:23.179337+0000","last_fullsized":"2024-05-09T00:44:23.179337+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:53:22.566850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.178826+0000","last_change":"2024-05-09T00:43:43.872041+0000","last_active":"2024-05-09T00:44:23.178826+0000","last_peered":"2024-05-09T00:44:23.178826+0000","last_clean":"2024-05-09T00:44:23.178826+0000","last_became_active":"2024-05-09T00:43:43.870946+0000","last_became_peered":"2024-05-09T00:43:43.870946+0000","last_unstale":"2024-05-09T00:44:23.178826+0000","last_undegraded":"2024-05-09T00:44:23.178826+0000","last_fullsized":"2024-05-09T00:44:23.178826+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:16:28.190488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'14","reported_seq":40,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.016400+0000","last_change":"2024-05-09T00:43:40.853875+0000","last_active":"2024-05-09T00:44:23.016400+0000","last_peered":"2024-05-09T00:44:23.016400+0000","last_clean":"2024-05-09T00:44:23.016400+0000","last_became_active":"2024-05-09T00:43:40.853459+0000","last_became_peered":"2024-05-09T00:43:40.853459+0000","last_unstale":"2024-05-09T00:44:23.016400+0000","last_undegraded":"2024-05-09T00:44:23.016400+0000","last_fullsized":"2024-05-09T00:44:23.016400+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:45:29.992730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.014909+0000","last_change":"2024-05-09T00:43:43.870435+0000","last_active":"2024-05-09T00:44:23.014909+0000","last_peered":"2024-05-09T00:44:23.014909+0000","last_clean":"2024-05-09T00:44:23.014909+0000","last_became_active":"2024-05-09T00:43:43.870075+0000","last_became_peered":"2024-05-09T00:43:43.870075+0000","last_unstale":"2024-05-09T00:44:23.014909+0000","last_undegraded":"2024-05-09T00:44:23.014909+0000","last_fullsized":"2024-05-09T00:44:23.014909+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:53:12.958463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'7","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.179191+0000","last_change":"2024-05-09T00:43:41.119325+0000","last_active":"2024-05-09T00:44:23.179191+0000","last_peered":"2024-05-09T00:44:23.179191+0000","last_clean":"2024-05-09T00:44:23.179191+0000","last_became_active":"2024-05-09T00:43:41.119009+0000","last_became_peered":"2024-05-09T00:43:41.119009+0000","last_unstale":"2024-05-09T00:44:23.179191+0000","last_undegraded":"2024-05-09T00:44:23.179191+0000","last_fullsized":"2024-05-09T00:44:23.179191+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:10:05.775152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.018302+0000","last_change":"2024-05-09T00:43:43.875236+0000","last_active":"2024-05-09T00:44:23.018302+0000","last_peered":"2024-05-09T00:44:23.018302+0000","last_clean":"2024-05-09T00:44:23.018302+0000","last_became_active":"2024-05-09T00:43:43.874919+0000","last_became_peered":"2024-05-09T00:43:43.874919+0000","last_unstale":"2024-05-09T00:44:23.018302+0000","last_undegraded":"2024-05-09T00:44:23.018302+0000","last_fullsized":"2024-05-09T00:44:23.018302+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:45:33.232518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.018371+0000","last_change":"2024-05-09T00:43:40.959765+0000","last_active":"2024-05-09T00:44:23.018371+0000","last_peered":"2024-05-09T00:44:23.018371+0000","last_clean":"2024-05-09T00:44:23.018371+0000","last_became_active":"2024-05-09T00:43:40.959383+0000","last_became_peered":"2024-05-09T00:43:40.959383+0000","last_unstale":"2024-05-09T00:44:23.018371+0000","last_undegraded":"2024-05-09T00:44:23.018371+0000","last_fullsized":"2024-05-09T00:44:23.018371+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:13:28.766650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.178902+0000","last_change":"2024-05-09T00:43:43.881627+0000","last_active":"2024-05-09T00:44:23.178902+0000","last_peered":"2024-05-09T00:44:23.178902+0000","last_clean":"2024-05-09T00:44:23.178902+0000","last_became_active":"2024-05-09T00:43:43.881207+0000","last_became_peered":"2024-05-09T00:43:43.881207+0000","last_unstale":"2024-05-09T00:44:23.178902+0000","last_undegraded":"2024-05-09T00:44:23.178902+0000","last_fullsized":"2024-05-09T00:44:23.178902+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:44:08.053638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'7","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.014430+0000","last_change":"2024-05-09T00:43:40.859566+0000","last_active":"2024-05-09T00:44:23.014430+0000","last_peered":"2024-05-09T00:44:23.014430+0000","last_clean":"2024-05-09T00:44:23.014430+0000","last_became_active":"2024-05-09T00:43:40.859182+0000","last_became_peered":"2024-05-09T00:43:40.859182+0000","last_unstale":"2024-05-09T00:44:23.014430+0000","last_undegraded":"2024-05-09T00:44:23.014430+0000","last_fullsized":"2024-05-09T00:44:23.014430+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:58:12.693271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.017931+0000","last_change":"2024-05-09T00:43:43.869193+0000","last_active":"2024-05-09T00:44:23.017931+0000","last_peered":"2024-05-09T00:44:23.017931+0000","last_clean":"2024-05-09T00:44:23.017931+0000","last_became_active":"2024-05-09T00:43:43.868823+0000","last_became_peered":"2024-05-09T00:43:43.868823+0000","last_unstale":"2024-05-09T00:44:23.017931+0000","last_undegraded":"2024-05-09T00:44:23.017931+0000","last_fullsized":"2024-05-09T00:44:23.017931+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:15:53.929873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.019050+0000","last_change":"2024-05-09T00:43:40.855038+0000","last_active":"2024-05-09T00:44:23.019050+0000","last_peered":"2024-05-09T00:44:23.019050+0000","last_clean":"2024-05-09T00:44:23.019050+0000","last_became_active":"2024-05-09T00:43:40.854334+0000","last_became_peered":"2024-05-09T00:43:40.854334+0000","last_unstale":"2024-05-09T00:44:23.019050+0000","last_undegraded":"2024-05-09T00:44:23.019050+0000","last_fullsized":"2024-05-09T00:44:23.019050+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:47:13.897697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.019151+0000","last_change":"2024-05-09T00:44:23.019151+0000","last_active":"2024-05-09T00:44:23.019151+0000","last_peered":"2024-05-09T00:44:23.019151+0000","last_clean":"2024-05-09T00:44:23.019151+0000","last_became_active":"2024-05-09T00:44:23.018752+0000","last_became_peered":"2024-05-09T00:44:23.018752+0000","last_unstale":"2024-05-09T00:44:23.019151+0000","last_undegraded":"2024-05-09T00:44:23.019151+0000","last_fullsized":"2024-05-09T00:44:23.019151+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:28:41.875747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.178988+0000","last_change":"2024-05-09T00:43:40.859461+0000","last_active":"2024-05-09T00:44:23.178988+0000","last_peered":"2024-05-09T00:44:23.178988+0000","last_clean":"2024-05-09T00:44:23.178988+0000","last_became_active":"2024-05-09T00:43:40.857814+0000","last_became_peered":"2024-05-09T00:43:40.857814+0000","last_unstale":"2024-05-09T00:44:23.178988+0000","last_undegraded":"2024-05-09T00:44:23.178988+0000","last_fullsized":"2024-05-09T00:44:23.178988+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:56:06.279762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.021237+0000","last_change":"2024-05-09T00:43:43.878120+0000","last_active":"2024-05-09T00:44:23.021237+0000","last_peered":"2024-05-09T00:44:23.021237+0000","last_clean":"2024-05-09T00:44:23.021237+0000","last_became_active":"2024-05-09T00:43:43.877782+0000","last_became_peered":"2024-05-09T00:43:43.877782+0000","last_unstale":"2024-05-09T00:44:23.021237+0000","last_undegraded":"2024-05-09T00:44:23.021237+0000","last_fullsized":"2024-05-09T00:44:23.021237+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:19:48.558992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'6","reported_seq":60,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:25.997625+0000","last_change":"2024-05-09T00:43:41.119347+0000","last_active":"2024-05-09T00:44:25.997625+0000","last_peered":"2024-05-09T00:44:25.997625+0000","last_clean":"2024-05-09T00:44:25.997625+0000","last_became_active":"2024-05-09T00:43:41.118847+0000","last_became_peered":"2024-05-09T00:43:41.118847+0000","last_unstale":"2024-05-09T00:44:25.997625+0000","last_undegraded":"2024-05-09T00:44:25.997625+0000","last_fullsized":"2024-05-09T00:44:25.997625+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:07:03.977756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":28,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.021588+0000","last_change":"2024-05-09T00:43:43.875203+0000","last_active":"2024-05-09T00:44:23.021588+0000","last_peered":"2024-05-09T00:44:23.021588+0000","last_clean":"2024-05-09T00:44:23.021588+0000","last_became_active":"2024-05-09T00:43:43.874882+0000","last_became_peered":"2024-05-09T00:43:43.874882+0000","last_unstale":"2024-05-09T00:44:23.021588+0000","last_undegraded":"2024-05-09T00:44:23.021588+0000","last_fullsized":"2024-05-09T00:44:23.021588+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:54:27.845086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'3","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.015990+0000","last_change":"2024-05-09T00:43:40.857955+0000","last_active":"2024-05-09T00:44:23.015990+0000","last_peered":"2024-05-09T00:44:23.015990+0000","last_clean":"2024-05-09T00:44:23.015990+0000","last_became_active":"2024-05-09T00:43:40.857596+0000","last_became_peered":"2024-05-09T00:43:40.857596+0000","last_unstale":"2024-05-09T00:44:23.015990+0000","last_undegraded":"2024-05-09T00:44:23.015990+0000","last_fullsized":"2024-05-09T00:44:23.015990+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:13:56.676929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"78'1","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.021334+0000","last_change":"2024-05-09T00:43:43.882487+0000","last_active":"2024-05-09T00:44:23.021334+0000","last_peered":"2024-05-09T00:44:23.021334+0000","last_clean":"2024-05-09T00:44:23.021334+0000","last_became_active":"2024-05-09T00:43:43.882176+0000","last_became_peered":"2024-05-09T00:43:43.882176+0000","last_unstale":"2024-05-09T00:44:23.021334+0000","last_undegraded":"2024-05-09T00:44:23.021334+0000","last_fullsized":"2024-05-09T00:44:23.021334+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:48:42.354433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'5","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.178994+0000","last_change":"2024-05-09T00:43:40.857215+0000","last_active":"2024-05-09T00:44:23.178994+0000","last_peered":"2024-05-09T00:44:23.178994+0000","last_clean":"2024-05-09T00:44:23.178994+0000","last_became_active":"2024-05-09T00:43:40.856858+0000","last_became_peered":"2024-05-09T00:43:40.856858+0000","last_unstale":"2024-05-09T00:44:23.178994+0000","last_undegraded":"2024-05-09T00:44:23.178994+0000","last_fullsized":"2024-05-09T00:44:23.178994+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:40:47.943482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.021561+0000","last_change":"2024-05-09T00:43:43.881192+0000","last_active":"2024-05-09T00:44:23.021561+0000","last_peered":"2024-05-09T00:44:23.021561+0000","last_clean":"2024-05-09T00:44:23.021561+0000","last_became_active":"2024-05-09T00:43:43.880781+0000","last_became_peered":"2024-05-09T00:43:43.880781+0000","last_unstale":"2024-05-09T00:44:23.021561+0000","last_undegraded":"2024-05-09T00:44:23.021561+0000","last_fullsized":"2024-05-09T00:44:23.021561+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:18:31.577354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.018034+0000","last_change":"2024-05-09T00:43:43.876869+0000","last_active":"2024-05-09T00:44:23.018034+0000","last_peered":"2024-05-09T00:44:23.018034+0000","last_clean":"2024-05-09T00:44:23.018034+0000","last_became_active":"2024-05-09T00:43:43.876439+0000","last_became_peered":"2024-05-09T00:43:43.876439+0000","last_unstale":"2024-05-09T00:44:23.018034+0000","last_undegraded":"2024-05-09T00:44:23.018034+0000","last_fullsized":"2024-05-09T00:44:23.018034+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:39:06.776267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.019040+0000","last_change":"2024-05-09T00:43:43.884677+0000","last_active":"2024-05-09T00:44:23.019040+0000","last_peered":"2024-05-09T00:44:23.019040+0000","last_clean":"2024-05-09T00:44:23.019040+0000","last_became_active":"2024-05-09T00:43:43.884238+0000","last_became_peered":"2024-05-09T00:43:43.884238+0000","last_unstale":"2024-05-09T00:44:23.019040+0000","last_undegraded":"2024-05-09T00:44:23.019040+0000","last_fullsized":"2024-05-09T00:44:23.019040+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:18:13.787584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.019033+0000","last_change":"2024-05-09T00:43:43.876638+0000","last_active":"2024-05-09T00:44:23.019033+0000","last_peered":"2024-05-09T00:44:23.019033+0000","last_clean":"2024-05-09T00:44:23.019033+0000","last_became_active":"2024-05-09T00:43:43.876315+0000","last_became_peered":"2024-05-09T00:43:43.876315+0000","last_unstale":"2024-05-09T00:44:23.019033+0000","last_undegraded":"2024-05-09T00:44:23.019033+0000","last_fullsized":"2024-05-09T00:44:23.019033+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:22:41.618613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.017968+0000","last_change":"2024-05-09T00:43:43.869798+0000","last_active":"2024-05-09T00:44:23.017968+0000","last_peered":"2024-05-09T00:44:23.017968+0000","last_clean":"2024-05-09T00:44:23.017968+0000","last_became_active":"2024-05-09T00:43:43.869444+0000","last_became_peered":"2024-05-09T00:43:43.869444+0000","last_unstale":"2024-05-09T00:44:23.017968+0000","last_undegraded":"2024-05-09T00:44:23.017968+0000","last_fullsized":"2024-05-09T00:44:23.017968+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:15:53.267441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.178838+0000","last_change":"2024-05-09T00:43:43.873215+0000","last_active":"2024-05-09T00:44:23.178838+0000","last_peered":"2024-05-09T00:44:23.178838+0000","last_clean":"2024-05-09T00:44:23.178838+0000","last_became_active":"2024-05-09T00:43:43.872763+0000","last_became_peered":"2024-05-09T00:43:43.872763+0000","last_unstale":"2024-05-09T00:44:23.178838+0000","last_undegraded":"2024-05-09T00:44:23.178838+0000","last_fullsized":"2024-05-09T00:44:23.178838+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:08.997387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.018254+0000","last_change":"2024-05-09T00:43:43.876514+0000","last_active":"2024-05-09T00:44:23.018254+0000","last_peered":"2024-05-09T00:44:23.018254+0000","last_clean":"2024-05-09T00:44:23.018254+0000","last_became_active":"2024-05-09T00:43:43.876147+0000","last_became_peered":"2024-05-09T00:43:43.876147+0000","last_unstale":"2024-05-09T00:44:23.018254+0000","last_undegraded":"2024-05-09T00:44:23.018254+0000","last_fullsized":"2024-05-09T00:44:23.018254+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:58:34.745819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.019620+0000","last_change":"2024-05-09T00:43:43.871869+0000","last_active":"2024-05-09T00:44:23.019620+0000","last_peered":"2024-05-09T00:44:23.019620+0000","last_clean":"2024-05-09T00:44:23.019620+0000","last_became_active":"2024-05-09T00:43:43.871447+0000","last_became_peered":"2024-05-09T00:43:43.871447+0000","last_unstale":"2024-05-09T00:44:23.019620+0000","last_undegraded":"2024-05-09T00:44:23.019620+0000","last_fullsized":"2024-05-09T00:44:23.019620+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:11:42.128931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.014446+0000","last_change":"2024-05-09T00:43:43.871291+0000","last_active":"2024-05-09T00:44:23.014446+0000","last_peered":"2024-05-09T00:44:23.014446+0000","last_clean":"2024-05-09T00:44:23.014446+0000","last_became_active":"2024-05-09T00:43:43.870899+0000","last_became_peered":"2024-05-09T00:43:43.870899+0000","last_unstale":"2024-05-09T00:44:23.014446+0000","last_undegraded":"2024-05-09T00:44:23.014446+0000","last_fullsized":"2024-05-09T00:44:23.014446+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:28:53.013543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.015578+0000","last_change":"2024-05-09T00:43:43.884791+0000","last_active":"2024-05-09T00:44:23.015578+0000","last_peered":"2024-05-09T00:44:23.015578+0000","last_clean":"2024-05-09T00:44:23.015578+0000","last_became_active":"2024-05-09T00:43:43.877333+0000","last_became_peered":"2024-05-09T00:43:43.877333+0000","last_unstale":"2024-05-09T00:44:23.015578+0000","last_undegraded":"2024-05-09T00:44:23.015578+0000","last_fullsized":"2024-05-09T00:44:23.015578+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:37:07.535388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.014278+0000","last_change":"2024-05-09T00:43:43.863773+0000","last_active":"2024-05-09T00:44:23.014278+0000","last_peered":"2024-05-09T00:44:23.014278+0000","last_clean":"2024-05-09T00:44:23.014278+0000","last_became_active":"2024-05-09T00:43:43.863436+0000","last_became_peered":"2024-05-09T00:43:43.863436+0000","last_unstale":"2024-05-09T00:44:23.014278+0000","last_undegraded":"2024-05-09T00:44:23.014278+0000","last_fullsized":"2024-05-09T00:44:23.014278+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:59:57.100234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.018880+0000","last_change":"2024-05-09T00:43:43.873461+0000","last_active":"2024-05-09T00:44:23.018880+0000","last_peered":"2024-05-09T00:44:23.018880+0000","last_clean":"2024-05-09T00:44:23.018880+0000","last_became_active":"2024-05-09T00:43:43.873139+0000","last_became_peered":"2024-05-09T00:43:43.873139+0000","last_unstale":"2024-05-09T00:44:23.018880+0000","last_undegraded":"2024-05-09T00:44:23.018880+0000","last_fullsized":"2024-05-09T00:44:23.018880+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:24:57.865569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.178871+0000","last_change":"2024-05-09T00:43:43.874401+0000","last_active":"2024-05-09T00:44:23.178871+0000","last_peered":"2024-05-09T00:44:23.178871+0000","last_clean":"2024-05-09T00:44:23.178871+0000","last_became_active":"2024-05-09T00:43:43.873765+0000","last_became_peered":"2024-05-09T00:43:43.873765+0000","last_unstale":"2024-05-09T00:44:23.178871+0000","last_undegraded":"2024-05-09T00:44:23.178871+0000","last_fullsized":"2024-05-09T00:44:23.178871+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:23:59.509415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.018033+0000","last_change":"2024-05-09T00:43:43.875219+0000","last_active":"2024-05-09T00:44:23.018033+0000","last_peered":"2024-05-09T00:44:23.018033+0000","last_clean":"2024-05-09T00:44:23.018033+0000","last_became_active":"2024-05-09T00:43:43.874824+0000","last_became_peered":"2024-05-09T00:43:43.874824+0000","last_unstale":"2024-05-09T00:44:23.018033+0000","last_undegraded":"2024-05-09T00:44:23.018033+0000","last_fullsized":"2024-05-09T00:44:23.018033+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:39:41.929521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":262144,"data_stored":2951840,"data_compressed":179218,"data_compressed_allocated":225280,"data_compressed_original":2932736,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65673,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":287,"num_read_kb":208,"num_write":244,"num_write_kb":134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":52683,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":528384,"data_stored":197019,"data_compressed":8493,"data_compressed_allocated":12288,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},"log_size":210,"ondisk_log_size":210,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710744,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":872448,"data_stored":426179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906974,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":1075286,"data_compressed":36168,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070499,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":425718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266728,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":892928,"data_stored":426096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462957,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":946176,"data_stored":1019005,"data_compressed":36168,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626483,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":880640,"data_stored":425435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":942080,"data_stored":484426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738448,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":427165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411398,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":427350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182466,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36220,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708932,"statfs":{"total":95995035648,"available":95957946368,"internally_reserved":0,"allocated":897024,"data_stored":425260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986236,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":913408,"data_stored":483765,"data_compressed":8493,"data_compressed_allocated":12288,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822712,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":1018292,"data_compressed":36168,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34546,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36168,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36168,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36168,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36168,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":59830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":59169,"data_compressed":8493,"data_compressed_allocated":12288,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":60205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T00:44:31.462 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-09T00:44:31.718 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:44:32.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:32 smithi131 ceph-mon[34547]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T00:44:32.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:32 smithi157 ceph-mon[34294]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T00:44:32.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:32 smithi005 ceph-mon[30365]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T00:44:33.224 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:33 smithi005 ceph-mon[30365]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.7 KiB/s wr, 11 op/s; 5.2 KiB/s, 0 objects/s recovering 2024-05-09T00:44:33.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:33 smithi131 ceph-mon[34547]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.7 KiB/s wr, 11 op/s; 5.2 KiB/s, 0 objects/s recovering 2024-05-09T00:44:33.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:33 smithi157 ceph-mon[34294]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.7 KiB/s wr, 11 op/s; 5.2 KiB/s, 0 objects/s recovering 2024-05-09T00:44:33.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:33.504+0000 7f4a5496d700 1 -- 172.21.15.5:0/1871345641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a50109880 msgr2=0x7f4a501111e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:33.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:33.504+0000 7f4a5496d700 1 --2- 172.21.15.5:0/1871345641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a50109880 0x7f4a501111e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4a4800b0d0 tx=0x7f4a4800b3e0 comp rx=0 tx=0).stop 2024-05-09T00:44:33.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:33.504+0000 7f4a5496d700 1 -- 172.21.15.5:0/1871345641 shutdown_connections 2024-05-09T00:44:33.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:33.504+0000 7f4a5496d700 1 --2- 172.21.15.5:0/1871345641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a50109880 0x7f4a501111e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:33.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:33.504+0000 7f4a5496d700 1 --2- 172.21.15.5:0/1871345641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a50104650 0x7f4a50104a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:33.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:33.504+0000 7f4a5496d700 1 --2- 172.21.15.5:0/1871345641 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a50108f30 0x7f4a50109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:33.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:33.504+0000 7f4a5496d700 1 -- 172.21.15.5:0/1871345641 >> 172.21.15.5:0/1871345641 conn(0x7f4a50079f00 msgr2=0x7f4a5007a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:44:33.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:33.504+0000 7f4a5496d700 1 -- 172.21.15.5:0/1871345641 shutdown_connections 2024-05-09T00:44:33.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:33.504+0000 7f4a5496d700 1 -- 172.21.15.5:0/1871345641 wait complete. 2024-05-09T00:44:33.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:33.505+0000 7f4a5496d700 1 Processor -- start 2024-05-09T00:44:33.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:33.505+0000 7f4a5496d700 1 -- start start 2024-05-09T00:44:33.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:33.505+0000 7f4a5496d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a50104650 0x7f4a5019ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:44:33.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:33.505+0000 7f4a5496d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a50108f30 0x7f4a5019b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:44:33.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:33.505+0000 7f4a5496d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a50109880 0x7f4a5019f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:44:33.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:33.505+0000 7f4a5496d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4a501144b0 con 0x7f4a50109880 2024-05-09T00:44:33.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:33.505+0000 7f4a5496d700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4a50114330 con 0x7f4a50104650 2024-05-09T00:44:33.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:33.505+0000 7f4a5496d700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4a50114630 con 0x7f4a50108f30 2024-05-09T00:44:33.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:33.505+0000 7f4a4ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a50109880 0x7f4a5019f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:44:33.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:33.505+0000 7f4a4ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a50109880 0x7f4a5019f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41136/0 (socket says 172.21.15.5:41136) 2024-05-09T00:44:33.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:33.505+0000 7f4a4ed9d700 1 -- 172.21.15.5:0/1491250105 learned_addr learned my addr 172.21.15.5:0/1491250105 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:44:33.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:33.505+0000 7f4a4e59c700 1 --2- 172.21.15.5:0/1491250105 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a50104650 0x7f4a5019ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:44:33.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:33.506+0000 7f4a4dd9b700 1 --2- 172.21.15.5:0/1491250105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a50108f30 0x7f4a5019b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:44:33.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:33.506+0000 7f4a4ed9d700 1 -- 172.21.15.5:0/1491250105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a50108f30 msgr2=0x7f4a5019b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:33.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:33.506+0000 7f4a4ed9d700 1 --2- 172.21.15.5:0/1491250105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a50108f30 0x7f4a5019b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:33.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:33.506+0000 7f4a4ed9d700 1 -- 172.21.15.5:0/1491250105 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a50104650 msgr2=0x7f4a5019ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:33.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:33.506+0000 7f4a4ed9d700 1 --2- 172.21.15.5:0/1491250105 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a50104650 0x7f4a5019ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:33.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:33.506+0000 7f4a4ed9d700 1 -- 172.21.15.5:0/1491250105 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a5019fc40 con 0x7f4a50109880 2024-05-09T00:44:33.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:33.506+0000 7f4a4ed9d700 1 --2- 172.21.15.5:0/1491250105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a50109880 0x7f4a5019f650 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4a480095c0 tx=0x7f4a48007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:44:33.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:33.506+0000 7f4a3f7fe700 1 -- 172.21.15.5:0/1491250105 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4801f070 con 0x7f4a50109880 2024-05-09T00:44:33.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:33.506+0000 7f4a3f7fe700 1 -- 172.21.15.5:0/1491250105 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a48007e20 con 0x7f4a50109880 2024-05-09T00:44:33.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:33.506+0000 7f4a5496d700 1 -- 172.21.15.5:0/1491250105 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a5019fed0 con 0x7f4a50109880 2024-05-09T00:44:33.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:33.506+0000 7f4a3f7fe700 1 -- 172.21.15.5:0/1491250105 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4801b4a0 con 0x7f4a50109880 2024-05-09T00:44:33.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:33.506+0000 7f4a5496d700 1 -- 172.21.15.5:0/1491250105 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a501a03b0 con 0x7f4a50109880 2024-05-09T00:44:33.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:33.507+0000 7f4a5496d700 1 -- 172.21.15.5:0/1491250105 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a500512c0 con 0x7f4a50109880 2024-05-09T00:44:33.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:33.509+0000 7f4a3f7fe700 1 -- 172.21.15.5:0/1491250105 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f4a48022070 con 0x7f4a50109880 2024-05-09T00:44:33.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:33.509+0000 7f4a3f7fe700 1 --2- 172.21.15.5:0/1491250105 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f4a380b2910 0x7f4a380b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:44:33.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:33.510+0000 7f4a4e59c700 1 --2- 172.21.15.5:0/1491250105 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f4a380b2910 0x7f4a380b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:44:33.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:33.510+0000 7f4a3f7fe700 1 -- 172.21.15.5:0/1491250105 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8227+0+0 (secure 0 0 0) 0x7f4a4800f020 con 0x7f4a50109880 2024-05-09T00:44:33.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:33.510+0000 7f4a4e59c700 1 --2- 172.21.15.5:0/1491250105 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f4a380b2910 0x7f4a380b4e20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4a4000ac50 tx=0x7f4a4000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:44:33.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:33.514+0000 7f4a3f7fe700 1 -- 172.21.15.5:0/1491250105 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4a48029020 con 0x7f4a50109880 2024-05-09T00:44:33.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:33.723+0000 7f4a5496d700 1 -- 172.21.15.5:0/1491250105 --> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4a501035d0 con 0x7f4a380b2910 2024-05-09T00:44:33.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:33.731+0000 7f4a3f7fe700 1 -- 172.21.15.5:0/1491250105 <== mgr.14152 v2:172.21.15.5:6800/91856593 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258883 (secure 0 0 0) 0x7f4a501035d0 con 0x7f4a380b2910 2024-05-09T00:44:33.735 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:44:33.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:33.734+0000 7f4a5496d700 1 -- 172.21.15.5:0/1491250105 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f4a380b2910 msgr2=0x7f4a380b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:33.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:33.734+0000 7f4a5496d700 1 --2- 172.21.15.5:0/1491250105 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f4a380b2910 0x7f4a380b4e20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4a4000ac50 tx=0x7f4a4000a380 comp rx=0 tx=0).stop 2024-05-09T00:44:33.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:33.734+0000 7f4a5496d700 1 -- 172.21.15.5:0/1491250105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a50109880 msgr2=0x7f4a5019f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:33.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:33.734+0000 7f4a5496d700 1 --2- 172.21.15.5:0/1491250105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a50109880 0x7f4a5019f650 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4a480095c0 tx=0x7f4a48007a20 comp rx=0 tx=0).stop 2024-05-09T00:44:33.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:33.734+0000 7f4a5496d700 1 -- 172.21.15.5:0/1491250105 shutdown_connections 2024-05-09T00:44:33.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:33.734+0000 7f4a5496d700 1 --2- 172.21.15.5:0/1491250105 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f4a380b2910 0x7f4a380b4e20 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:33.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:33.734+0000 7f4a5496d700 1 --2- 172.21.15.5:0/1491250105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a50109880 0x7f4a5019f650 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:33.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:33.734+0000 7f4a5496d700 1 --2- 172.21.15.5:0/1491250105 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a50104650 0x7f4a5019ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:33.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:33.734+0000 7f4a5496d700 1 --2- 172.21.15.5:0/1491250105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a50108f30 0x7f4a5019b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:33.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:33.734+0000 7f4a5496d700 1 -- 172.21.15.5:0/1491250105 >> 172.21.15.5:0/1491250105 conn(0x7f4a50079f00 msgr2=0x7f4a50102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:44:33.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:33.734+0000 7f4a5496d700 1 -- 172.21.15.5:0/1491250105 shutdown_connections 2024-05-09T00:44:33.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:33.734+0000 7f4a5496d700 1 -- 172.21.15.5:0/1491250105 wait complete. 2024-05-09T00:44:33.738 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-09T00:44:34.443 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":315,"stamp":"2024-05-09T00:44:31.848590+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656275,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":368,"num_read_kb":272,"num_write":369,"num_write_kb":2286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":3004523,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":304,"ondisk_log_size":304,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":293,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":347500,"kb_used_data":10764,"kb_used_omap":18,"kb_used_meta":336557,"kb_avail":1124594324,"statfs":{"total":1151940427776,"available":1151584587776,"internally_reserved":0,"allocated":11022336,"data_stored":7063977,"data_compressed":116997,"data_compressed_allocated":147456,"data_compressed_original":1818624,"omap_allocated":19141,"internal_metadata":344634683},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29995,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":96,"num_read_kb":67,"num_write":20,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":52683,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.836297"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.018605+0000","last_change":"2024-05-09T00:43:43.875357+0000","last_active":"2024-05-09T00:44:23.018605+0000","last_peered":"2024-05-09T00:44:23.018605+0000","last_clean":"2024-05-09T00:44:23.018605+0000","last_became_active":"2024-05-09T00:43:43.873765+0000","last_became_peered":"2024-05-09T00:43:43.873765+0000","last_unstale":"2024-05-09T00:44:23.018605+0000","last_undegraded":"2024-05-09T00:44:23.018605+0000","last_fullsized":"2024-05-09T00:44:23.018605+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:47.519110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.018171+0000","last_change":"2024-05-09T00:43:43.882464+0000","last_active":"2024-05-09T00:44:23.018171+0000","last_peered":"2024-05-09T00:44:23.018171+0000","last_clean":"2024-05-09T00:44:23.018171+0000","last_became_active":"2024-05-09T00:43:43.882102+0000","last_became_peered":"2024-05-09T00:43:43.882102+0000","last_unstale":"2024-05-09T00:44:23.018171+0000","last_undegraded":"2024-05-09T00:44:23.018171+0000","last_fullsized":"2024-05-09T00:44:23.018171+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:39:36.234474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.018181+0000","last_change":"2024-05-09T00:43:43.873098+0000","last_active":"2024-05-09T00:44:23.018181+0000","last_peered":"2024-05-09T00:44:23.018181+0000","last_clean":"2024-05-09T00:44:23.018181+0000","last_became_active":"2024-05-09T00:43:43.872764+0000","last_became_peered":"2024-05-09T00:43:43.872764+0000","last_unstale":"2024-05-09T00:44:23.018181+0000","last_undegraded":"2024-05-09T00:44:23.018181+0000","last_fullsized":"2024-05-09T00:44:23.018181+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:05:30.784111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.179963+0000","last_change":"2024-05-09T00:43:43.869807+0000","last_active":"2024-05-09T00:44:23.179963+0000","last_peered":"2024-05-09T00:44:23.179963+0000","last_clean":"2024-05-09T00:44:23.179963+0000","last_became_active":"2024-05-09T00:43:43.869470+0000","last_became_peered":"2024-05-09T00:43:43.869470+0000","last_unstale":"2024-05-09T00:44:23.179963+0000","last_undegraded":"2024-05-09T00:44:23.179963+0000","last_fullsized":"2024-05-09T00:44:23.179963+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:25:40.201844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.178978+0000","last_change":"2024-05-09T00:43:43.874110+0000","last_active":"2024-05-09T00:44:23.178978+0000","last_peered":"2024-05-09T00:44:23.178978+0000","last_clean":"2024-05-09T00:44:23.178978+0000","last_became_active":"2024-05-09T00:43:43.873759+0000","last_became_peered":"2024-05-09T00:43:43.873759+0000","last_unstale":"2024-05-09T00:44:23.178978+0000","last_undegraded":"2024-05-09T00:44:23.178978+0000","last_fullsized":"2024-05-09T00:44:23.178978+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:17:17.722319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.022285+0000","last_change":"2024-05-09T00:44:23.022285+0000","last_active":"2024-05-09T00:44:23.022285+0000","last_peered":"2024-05-09T00:44:23.022285+0000","last_clean":"2024-05-09T00:44:23.022285+0000","last_became_active":"2024-05-09T00:44:23.021887+0000","last_became_peered":"2024-05-09T00:44:23.021887+0000","last_unstale":"2024-05-09T00:44:23.022285+0000","last_undegraded":"2024-05-09T00:44:23.022285+0000","last_fullsized":"2024-05-09T00:44:23.022285+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:07:07.702178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,7],"acting":[10,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.020962+0000","last_change":"2024-05-09T00:43:43.883112+0000","last_active":"2024-05-09T00:44:23.020962+0000","last_peered":"2024-05-09T00:44:23.020962+0000","last_clean":"2024-05-09T00:44:23.020962+0000","last_became_active":"2024-05-09T00:43:43.882795+0000","last_became_peered":"2024-05-09T00:43:43.882795+0000","last_unstale":"2024-05-09T00:44:23.020962+0000","last_undegraded":"2024-05-09T00:44:23.020962+0000","last_fullsized":"2024-05-09T00:44:23.020962+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:33:58.724015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.014225+0000","last_change":"2024-05-09T00:43:43.880696+0000","last_active":"2024-05-09T00:44:23.014225+0000","last_peered":"2024-05-09T00:44:23.014225+0000","last_clean":"2024-05-09T00:44:23.014225+0000","last_became_active":"2024-05-09T00:43:43.880174+0000","last_became_peered":"2024-05-09T00:43:43.880174+0000","last_unstale":"2024-05-09T00:44:23.014225+0000","last_undegraded":"2024-05-09T00:44:23.014225+0000","last_fullsized":"2024-05-09T00:44:23.014225+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:40:05.682432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.019760+0000","last_change":"2024-05-09T00:43:43.883449+0000","last_active":"2024-05-09T00:44:23.019760+0000","last_peered":"2024-05-09T00:44:23.019760+0000","last_clean":"2024-05-09T00:44:23.019760+0000","last_became_active":"2024-05-09T00:43:43.883066+0000","last_became_peered":"2024-05-09T00:43:43.883066+0000","last_unstale":"2024-05-09T00:44:23.019760+0000","last_undegraded":"2024-05-09T00:44:23.019760+0000","last_fullsized":"2024-05-09T00:44:23.019760+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:47:49.116004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.016340+0000","last_change":"2024-05-09T00:43:43.877514+0000","last_active":"2024-05-09T00:44:23.016340+0000","last_peered":"2024-05-09T00:44:23.016340+0000","last_clean":"2024-05-09T00:44:23.016340+0000","last_became_active":"2024-05-09T00:43:43.877170+0000","last_became_peered":"2024-05-09T00:43:43.877170+0000","last_unstale":"2024-05-09T00:44:23.016340+0000","last_undegraded":"2024-05-09T00:44:23.016340+0000","last_fullsized":"2024-05-09T00:44:23.016340+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:47:26.738571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.180294+0000","last_change":"2024-05-09T00:44:23.180294+0000","last_active":"2024-05-09T00:44:23.180294+0000","last_peered":"2024-05-09T00:44:23.180294+0000","last_clean":"2024-05-09T00:44:23.180294+0000","last_became_active":"2024-05-09T00:44:23.179824+0000","last_became_peered":"2024-05-09T00:44:23.179824+0000","last_unstale":"2024-05-09T00:44:23.180294+0000","last_undegraded":"2024-05-09T00:44:23.180294+0000","last_fullsized":"2024-05-09T00:44:23.180294+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:43:20.919660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,9],"acting":[7,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.179922+0000","last_change":"2024-05-09T00:43:43.873836+0000","last_active":"2024-05-09T00:44:23.179922+0000","last_peered":"2024-05-09T00:44:23.179922+0000","last_clean":"2024-05-09T00:44:23.179922+0000","last_became_active":"2024-05-09T00:43:43.873512+0000","last_became_peered":"2024-05-09T00:43:43.873512+0000","last_unstale":"2024-05-09T00:44:23.179922+0000","last_undegraded":"2024-05-09T00:44:23.179922+0000","last_fullsized":"2024-05-09T00:44:23.179922+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:32:06.162638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.179009+0000","last_change":"2024-05-09T00:43:43.885614+0000","last_active":"2024-05-09T00:44:23.179009+0000","last_peered":"2024-05-09T00:44:23.179009+0000","last_clean":"2024-05-09T00:44:23.179009+0000","last_became_active":"2024-05-09T00:43:43.884682+0000","last_became_peered":"2024-05-09T00:43:43.884682+0000","last_unstale":"2024-05-09T00:44:23.179009+0000","last_undegraded":"2024-05-09T00:44:23.179009+0000","last_fullsized":"2024-05-09T00:44:23.179009+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:34:15.336173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.017792+0000","last_change":"2024-05-09T00:43:43.883860+0000","last_active":"2024-05-09T00:44:23.017792+0000","last_peered":"2024-05-09T00:44:23.017792+0000","last_clean":"2024-05-09T00:44:23.017792+0000","last_became_active":"2024-05-09T00:43:43.882870+0000","last_became_peered":"2024-05-09T00:43:43.882870+0000","last_unstale":"2024-05-09T00:44:23.017792+0000","last_undegraded":"2024-05-09T00:44:23.017792+0000","last_fullsized":"2024-05-09T00:44:23.017792+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:17:31.566819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.017866+0000","last_change":"2024-05-09T00:43:43.874114+0000","last_active":"2024-05-09T00:44:23.017866+0000","last_peered":"2024-05-09T00:44:23.017866+0000","last_clean":"2024-05-09T00:44:23.017866+0000","last_became_active":"2024-05-09T00:43:43.873762+0000","last_became_peered":"2024-05-09T00:43:43.873762+0000","last_unstale":"2024-05-09T00:44:23.017866+0000","last_undegraded":"2024-05-09T00:44:23.017866+0000","last_fullsized":"2024-05-09T00:44:23.017866+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:24:27.486100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.018352+0000","last_change":"2024-05-09T00:43:43.872982+0000","last_active":"2024-05-09T00:44:23.018352+0000","last_peered":"2024-05-09T00:44:23.018352+0000","last_clean":"2024-05-09T00:44:23.018352+0000","last_became_active":"2024-05-09T00:43:43.872464+0000","last_became_peered":"2024-05-09T00:43:43.872464+0000","last_unstale":"2024-05-09T00:44:23.018352+0000","last_undegraded":"2024-05-09T00:44:23.018352+0000","last_fullsized":"2024-05-09T00:44:23.018352+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:41.172001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.018730+0000","last_change":"2024-05-09T00:43:43.875662+0000","last_active":"2024-05-09T00:44:23.018730+0000","last_peered":"2024-05-09T00:44:23.018730+0000","last_clean":"2024-05-09T00:44:23.018730+0000","last_became_active":"2024-05-09T00:43:43.874701+0000","last_became_peered":"2024-05-09T00:43:43.874701+0000","last_unstale":"2024-05-09T00:44:23.018730+0000","last_undegraded":"2024-05-09T00:44:23.018730+0000","last_fullsized":"2024-05-09T00:44:23.018730+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:09:19.349190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.016202+0000","last_change":"2024-05-09T00:43:43.876100+0000","last_active":"2024-05-09T00:44:23.016202+0000","last_peered":"2024-05-09T00:44:23.016202+0000","last_clean":"2024-05-09T00:44:23.016202+0000","last_became_active":"2024-05-09T00:43:43.875768+0000","last_became_peered":"2024-05-09T00:43:43.875768+0000","last_unstale":"2024-05-09T00:44:23.016202+0000","last_undegraded":"2024-05-09T00:44:23.016202+0000","last_fullsized":"2024-05-09T00:44:23.016202+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:48:26.843064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.019634+0000","last_change":"2024-05-09T00:43:43.875900+0000","last_active":"2024-05-09T00:44:23.019634+0000","last_peered":"2024-05-09T00:44:23.019634+0000","last_clean":"2024-05-09T00:44:23.019634+0000","last_became_active":"2024-05-09T00:43:43.875497+0000","last_became_peered":"2024-05-09T00:43:43.875497+0000","last_unstale":"2024-05-09T00:44:23.019634+0000","last_undegraded":"2024-05-09T00:44:23.019634+0000","last_fullsized":"2024-05-09T00:44:23.019634+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:54:44.215936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'13","reported_seq":65,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:25.995387+0000","last_change":"2024-05-09T00:43:40.854337+0000","last_active":"2024-05-09T00:44:25.995387+0000","last_peered":"2024-05-09T00:44:25.995387+0000","last_clean":"2024-05-09T00:44:25.995387+0000","last_became_active":"2024-05-09T00:43:40.853986+0000","last_became_peered":"2024-05-09T00:43:40.853986+0000","last_unstale":"2024-05-09T00:44:25.995387+0000","last_undegraded":"2024-05-09T00:44:25.995387+0000","last_fullsized":"2024-05-09T00:44:25.995387+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:01:19.128533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.014907+0000","last_change":"2024-05-09T00:43:43.878081+0000","last_active":"2024-05-09T00:44:23.014907+0000","last_peered":"2024-05-09T00:44:23.014907+0000","last_clean":"2024-05-09T00:44:23.014907+0000","last_became_active":"2024-05-09T00:43:43.877585+0000","last_became_peered":"2024-05-09T00:43:43.877585+0000","last_unstale":"2024-05-09T00:44:23.014907+0000","last_undegraded":"2024-05-09T00:44:23.014907+0000","last_fullsized":"2024-05-09T00:44:23.014907+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:17:39.255841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.014977+0000","last_change":"2024-05-09T00:43:40.959386+0000","last_active":"2024-05-09T00:44:23.014977+0000","last_peered":"2024-05-09T00:44:23.014977+0000","last_clean":"2024-05-09T00:44:23.014977+0000","last_became_active":"2024-05-09T00:43:40.958878+0000","last_became_peered":"2024-05-09T00:43:40.958878+0000","last_unstale":"2024-05-09T00:44:23.014977+0000","last_undegraded":"2024-05-09T00:44:23.014977+0000","last_fullsized":"2024-05-09T00:44:23.014977+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:14:57.105078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.018181+0000","last_change":"2024-05-09T00:43:43.870563+0000","last_active":"2024-05-09T00:44:23.018181+0000","last_peered":"2024-05-09T00:44:23.018181+0000","last_clean":"2024-05-09T00:44:23.018181+0000","last_became_active":"2024-05-09T00:43:43.870237+0000","last_became_peered":"2024-05-09T00:43:43.870237+0000","last_unstale":"2024-05-09T00:44:23.018181+0000","last_undegraded":"2024-05-09T00:44:23.018181+0000","last_fullsized":"2024-05-09T00:44:23.018181+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:14:28.006343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"78'12","reported_seq":39,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.179571+0000","last_change":"2024-05-09T00:43:40.857670+0000","last_active":"2024-05-09T00:44:23.179571+0000","last_peered":"2024-05-09T00:44:23.179571+0000","last_clean":"2024-05-09T00:44:23.179571+0000","last_became_active":"2024-05-09T00:43:40.857247+0000","last_became_peered":"2024-05-09T00:43:40.857247+0000","last_unstale":"2024-05-09T00:44:23.179571+0000","last_undegraded":"2024-05-09T00:44:23.179571+0000","last_fullsized":"2024-05-09T00:44:23.179571+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:47:06.325593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.018774+0000","last_change":"2024-05-09T00:43:43.883179+0000","last_active":"2024-05-09T00:44:23.018774+0000","last_peered":"2024-05-09T00:44:23.018774+0000","last_clean":"2024-05-09T00:44:23.018774+0000","last_became_active":"2024-05-09T00:43:43.882820+0000","last_became_peered":"2024-05-09T00:43:43.882820+0000","last_unstale":"2024-05-09T00:44:23.018774+0000","last_undegraded":"2024-05-09T00:44:23.018774+0000","last_fullsized":"2024-05-09T00:44:23.018774+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:56:26.012495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"78'5","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.179162+0000","last_change":"2024-05-09T00:43:40.856024+0000","last_active":"2024-05-09T00:44:23.179162+0000","last_peered":"2024-05-09T00:44:23.179162+0000","last_clean":"2024-05-09T00:44:23.179162+0000","last_became_active":"2024-05-09T00:43:40.855634+0000","last_became_peered":"2024-05-09T00:43:40.855634+0000","last_unstale":"2024-05-09T00:44:23.179162+0000","last_undegraded":"2024-05-09T00:44:23.179162+0000","last_fullsized":"2024-05-09T00:44:23.179162+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:30:49.178547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.016130+0000","last_change":"2024-05-09T00:43:43.882950+0000","last_active":"2024-05-09T00:44:23.016130+0000","last_peered":"2024-05-09T00:44:23.016130+0000","last_clean":"2024-05-09T00:44:23.016130+0000","last_became_active":"2024-05-09T00:43:43.880679+0000","last_became_peered":"2024-05-09T00:43:43.880679+0000","last_unstale":"2024-05-09T00:44:23.016130+0000","last_undegraded":"2024-05-09T00:44:23.016130+0000","last_fullsized":"2024-05-09T00:44:23.016130+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:34:14.050987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"78'6","reported_seq":61,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:26.999183+0000","last_change":"2024-05-09T00:43:40.854953+0000","last_active":"2024-05-09T00:44:26.999183+0000","last_peered":"2024-05-09T00:44:26.999183+0000","last_clean":"2024-05-09T00:44:26.999183+0000","last_became_active":"2024-05-09T00:43:40.854562+0000","last_became_peered":"2024-05-09T00:43:40.854562+0000","last_unstale":"2024-05-09T00:44:26.999183+0000","last_undegraded":"2024-05-09T00:44:26.999183+0000","last_fullsized":"2024-05-09T00:44:26.999183+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:09:19.564806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.178886+0000","last_change":"2024-05-09T00:43:43.881881+0000","last_active":"2024-05-09T00:44:23.178886+0000","last_peered":"2024-05-09T00:44:23.178886+0000","last_clean":"2024-05-09T00:44:23.178886+0000","last_became_active":"2024-05-09T00:43:43.881461+0000","last_became_peered":"2024-05-09T00:43:43.881461+0000","last_unstale":"2024-05-09T00:44:23.178886+0000","last_undegraded":"2024-05-09T00:44:23.178886+0000","last_fullsized":"2024-05-09T00:44:23.178886+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:59:28.747084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"78'7","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.015495+0000","last_change":"2024-05-09T00:43:40.858521+0000","last_active":"2024-05-09T00:44:23.015495+0000","last_peered":"2024-05-09T00:44:23.015495+0000","last_clean":"2024-05-09T00:44:23.015495+0000","last_became_active":"2024-05-09T00:43:40.858102+0000","last_became_peered":"2024-05-09T00:43:40.858102+0000","last_unstale":"2024-05-09T00:44:23.015495+0000","last_undegraded":"2024-05-09T00:44:23.015495+0000","last_fullsized":"2024-05-09T00:44:23.015495+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:09:51.684630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.023472+0000","last_change":"2024-05-09T00:44:23.023472+0000","last_active":"2024-05-09T00:44:23.023472+0000","last_peered":"2024-05-09T00:44:23.023472+0000","last_clean":"2024-05-09T00:44:23.023472+0000","last_became_active":"2024-05-09T00:44:23.023017+0000","last_became_peered":"2024-05-09T00:44:23.023017+0000","last_unstale":"2024-05-09T00:44:23.023472+0000","last_undegraded":"2024-05-09T00:44:23.023472+0000","last_fullsized":"2024-05-09T00:44:23.023472+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:05:13.852606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.178682+0000","last_change":"2024-05-09T00:43:40.853329+0000","last_active":"2024-05-09T00:44:23.178682+0000","last_peered":"2024-05-09T00:44:23.178682+0000","last_clean":"2024-05-09T00:44:23.178682+0000","last_became_active":"2024-05-09T00:43:40.852979+0000","last_became_peered":"2024-05-09T00:43:40.852979+0000","last_unstale":"2024-05-09T00:44:23.178682+0000","last_undegraded":"2024-05-09T00:44:23.178682+0000","last_fullsized":"2024-05-09T00:44:23.178682+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:13:57.006031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.014305+0000","last_change":"2024-05-09T00:43:43.862876+0000","last_active":"2024-05-09T00:44:23.014305+0000","last_peered":"2024-05-09T00:44:23.014305+0000","last_clean":"2024-05-09T00:44:23.014305+0000","last_became_active":"2024-05-09T00:43:43.862521+0000","last_became_peered":"2024-05-09T00:43:43.862521+0000","last_unstale":"2024-05-09T00:44:23.014305+0000","last_undegraded":"2024-05-09T00:44:23.014305+0000","last_fullsized":"2024-05-09T00:44:23.014305+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:45:49.386844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'3","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.015554+0000","last_change":"2024-05-09T00:43:40.860395+0000","last_active":"2024-05-09T00:44:23.015554+0000","last_peered":"2024-05-09T00:44:23.015554+0000","last_clean":"2024-05-09T00:44:23.015554+0000","last_became_active":"2024-05-09T00:43:40.860040+0000","last_became_peered":"2024-05-09T00:43:40.860040+0000","last_unstale":"2024-05-09T00:44:23.015554+0000","last_undegraded":"2024-05-09T00:44:23.015554+0000","last_fullsized":"2024-05-09T00:44:23.015554+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:20:36.054685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.180905+0000","last_change":"2024-05-09T00:44:23.180905+0000","last_active":"2024-05-09T00:44:23.180905+0000","last_peered":"2024-05-09T00:44:23.180905+0000","last_clean":"2024-05-09T00:44:23.180905+0000","last_became_active":"2024-05-09T00:44:23.180296+0000","last_became_peered":"2024-05-09T00:44:23.180296+0000","last_unstale":"2024-05-09T00:44:23.180905+0000","last_undegraded":"2024-05-09T00:44:23.180905+0000","last_fullsized":"2024-05-09T00:44:23.180905+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:48:17.585276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,10],"acting":[9,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"78'4","reported_seq":82,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:27.999931+0000","last_change":"2024-05-09T00:43:40.860542+0000","last_active":"2024-05-09T00:44:27.999931+0000","last_peered":"2024-05-09T00:44:27.999931+0000","last_clean":"2024-05-09T00:44:27.999931+0000","last_became_active":"2024-05-09T00:43:40.860176+0000","last_became_peered":"2024-05-09T00:43:40.860176+0000","last_unstale":"2024-05-09T00:44:27.999931+0000","last_undegraded":"2024-05-09T00:44:27.999931+0000","last_fullsized":"2024-05-09T00:44:27.999931+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:44:50.375024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":52,"num_read_kb":52,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.019661+0000","last_change":"2024-05-09T00:43:43.882119+0000","last_active":"2024-05-09T00:44:23.019661+0000","last_peered":"2024-05-09T00:44:23.019661+0000","last_clean":"2024-05-09T00:44:23.019661+0000","last_became_active":"2024-05-09T00:43:43.881747+0000","last_became_peered":"2024-05-09T00:43:43.881747+0000","last_unstale":"2024-05-09T00:44:23.019661+0000","last_undegraded":"2024-05-09T00:44:23.019661+0000","last_fullsized":"2024-05-09T00:44:23.019661+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:54:19.420435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'4","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.021373+0000","last_change":"2024-05-09T00:43:40.859539+0000","last_active":"2024-05-09T00:44:23.021373+0000","last_peered":"2024-05-09T00:44:23.021373+0000","last_clean":"2024-05-09T00:44:23.021373+0000","last_became_active":"2024-05-09T00:43:40.859162+0000","last_became_peered":"2024-05-09T00:43:40.859162+0000","last_unstale":"2024-05-09T00:44:23.021373+0000","last_undegraded":"2024-05-09T00:44:23.021373+0000","last_fullsized":"2024-05-09T00:44:23.021373+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:37:15.190944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.180872+0000","last_change":"2024-05-09T00:44:23.180872+0000","last_active":"2024-05-09T00:44:23.180872+0000","last_peered":"2024-05-09T00:44:23.180872+0000","last_clean":"2024-05-09T00:44:23.180872+0000","last_became_active":"2024-05-09T00:44:23.180398+0000","last_became_peered":"2024-05-09T00:44:23.180398+0000","last_unstale":"2024-05-09T00:44:23.180872+0000","last_undegraded":"2024-05-09T00:44:23.180872+0000","last_fullsized":"2024-05-09T00:44:23.180872+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:19:27.909490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,6],"acting":[0,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'5","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.016141+0000","last_change":"2024-05-09T00:43:40.960126+0000","last_active":"2024-05-09T00:44:23.016141+0000","last_peered":"2024-05-09T00:44:23.016141+0000","last_clean":"2024-05-09T00:44:23.016141+0000","last_became_active":"2024-05-09T00:43:40.959441+0000","last_became_peered":"2024-05-09T00:43:40.959441+0000","last_unstale":"2024-05-09T00:44:23.016141+0000","last_undegraded":"2024-05-09T00:44:23.016141+0000","last_fullsized":"2024-05-09T00:44:23.016141+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:38:47.497029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.018911+0000","last_change":"2024-05-09T00:43:43.874223+0000","last_active":"2024-05-09T00:44:23.018911+0000","last_peered":"2024-05-09T00:44:23.018911+0000","last_clean":"2024-05-09T00:44:23.018911+0000","last_became_active":"2024-05-09T00:43:43.873875+0000","last_became_peered":"2024-05-09T00:43:43.873875+0000","last_unstale":"2024-05-09T00:44:23.018911+0000","last_undegraded":"2024-05-09T00:44:23.018911+0000","last_fullsized":"2024-05-09T00:44:23.018911+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:59:22.192886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'31","reported_seq":135,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:27.191685+0000","last_change":"2024-05-09T00:44:23.188953+0000","last_active":"2024-05-09T00:44:27.191685+0000","last_peered":"2024-05-09T00:44:27.191685+0000","last_clean":"2024-05-09T00:44:27.191685+0000","last_became_active":"2024-05-09T00:44:23.180858+0000","last_became_peered":"2024-05-09T00:44:23.180858+0000","last_unstale":"2024-05-09T00:44:27.191685+0000","last_undegraded":"2024-05-09T00:44:27.191685+0000","last_fullsized":"2024-05-09T00:44:27.191685+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:26:16.427665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58078,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":53,"num_read_kb":72,"num_write":33,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":52683,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.179509+0000","last_change":"2024-05-09T00:43:43.872019+0000","last_active":"2024-05-09T00:44:23.179509+0000","last_peered":"2024-05-09T00:44:23.179509+0000","last_clean":"2024-05-09T00:44:23.179509+0000","last_became_active":"2024-05-09T00:43:43.871622+0000","last_became_peered":"2024-05-09T00:43:43.871622+0000","last_unstale":"2024-05-09T00:44:23.179509+0000","last_undegraded":"2024-05-09T00:44:23.179509+0000","last_fullsized":"2024-05-09T00:44:23.179509+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:41:47.789443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'9","reported_seq":37,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.019299+0000","last_change":"2024-05-09T00:43:40.860111+0000","last_active":"2024-05-09T00:44:23.019299+0000","last_peered":"2024-05-09T00:44:23.019299+0000","last_clean":"2024-05-09T00:44:23.019299+0000","last_became_active":"2024-05-09T00:43:40.859732+0000","last_became_peered":"2024-05-09T00:43:40.859732+0000","last_unstale":"2024-05-09T00:44:23.019299+0000","last_undegraded":"2024-05-09T00:44:23.019299+0000","last_fullsized":"2024-05-09T00:44:23.019299+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:30:03.180679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.178898+0000","last_change":"2024-05-09T00:43:43.870604+0000","last_active":"2024-05-09T00:44:23.178898+0000","last_peered":"2024-05-09T00:44:23.178898+0000","last_clean":"2024-05-09T00:44:23.178898+0000","last_became_active":"2024-05-09T00:43:43.870286+0000","last_became_peered":"2024-05-09T00:43:43.870286+0000","last_unstale":"2024-05-09T00:44:23.178898+0000","last_undegraded":"2024-05-09T00:44:23.178898+0000","last_fullsized":"2024-05-09T00:44:23.178898+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:57:04.486631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'5","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.015759+0000","last_change":"2024-05-09T00:43:40.851966+0000","last_active":"2024-05-09T00:44:23.015759+0000","last_peered":"2024-05-09T00:44:23.015759+0000","last_clean":"2024-05-09T00:44:23.015759+0000","last_became_active":"2024-05-09T00:43:40.851537+0000","last_became_peered":"2024-05-09T00:43:40.851537+0000","last_unstale":"2024-05-09T00:44:23.015759+0000","last_undegraded":"2024-05-09T00:44:23.015759+0000","last_fullsized":"2024-05-09T00:44:23.015759+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:58:59.797183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.021364+0000","last_change":"2024-05-09T00:43:43.879509+0000","last_active":"2024-05-09T00:44:23.021364+0000","last_peered":"2024-05-09T00:44:23.021364+0000","last_clean":"2024-05-09T00:44:23.021364+0000","last_became_active":"2024-05-09T00:43:43.879151+0000","last_became_peered":"2024-05-09T00:43:43.879151+0000","last_unstale":"2024-05-09T00:44:23.021364+0000","last_undegraded":"2024-05-09T00:44:23.021364+0000","last_fullsized":"2024-05-09T00:44:23.021364+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:34:18.850545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"78'6","reported_seq":60,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:25.996358+0000","last_change":"2024-05-09T00:43:40.857629+0000","last_active":"2024-05-09T00:44:25.996358+0000","last_peered":"2024-05-09T00:44:25.996358+0000","last_clean":"2024-05-09T00:44:25.996358+0000","last_became_active":"2024-05-09T00:43:40.857260+0000","last_became_peered":"2024-05-09T00:43:40.857260+0000","last_unstale":"2024-05-09T00:44:25.996358+0000","last_undegraded":"2024-05-09T00:44:25.996358+0000","last_fullsized":"2024-05-09T00:44:25.996358+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:51:00.600870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":29,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.020628+0000","last_change":"2024-05-09T00:43:43.886100+0000","last_active":"2024-05-09T00:44:23.020628+0000","last_peered":"2024-05-09T00:44:23.020628+0000","last_clean":"2024-05-09T00:44:23.020628+0000","last_became_active":"2024-05-09T00:43:43.885775+0000","last_became_peered":"2024-05-09T00:43:43.885775+0000","last_unstale":"2024-05-09T00:44:23.020628+0000","last_undegraded":"2024-05-09T00:44:23.020628+0000","last_fullsized":"2024-05-09T00:44:23.020628+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:34:19.439163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'17","reported_seq":44,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.178810+0000","last_change":"2024-05-09T00:43:40.959873+0000","last_active":"2024-05-09T00:44:23.178810+0000","last_peered":"2024-05-09T00:44:23.178810+0000","last_clean":"2024-05-09T00:44:23.178810+0000","last_became_active":"2024-05-09T00:43:40.959437+0000","last_became_peered":"2024-05-09T00:43:40.959437+0000","last_unstale":"2024-05-09T00:44:23.178810+0000","last_undegraded":"2024-05-09T00:44:23.178810+0000","last_fullsized":"2024-05-09T00:44:23.178810+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:29:52.686937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.015736+0000","last_change":"2024-05-09T00:43:43.880076+0000","last_active":"2024-05-09T00:44:23.015736+0000","last_peered":"2024-05-09T00:44:23.015736+0000","last_clean":"2024-05-09T00:44:23.015736+0000","last_became_active":"2024-05-09T00:43:43.878733+0000","last_became_peered":"2024-05-09T00:43:43.878733+0000","last_unstale":"2024-05-09T00:44:23.015736+0000","last_undegraded":"2024-05-09T00:44:23.015736+0000","last_fullsized":"2024-05-09T00:44:23.015736+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:30:21.426855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.179296+0000","last_change":"2024-05-09T00:43:40.858033+0000","last_active":"2024-05-09T00:44:23.179296+0000","last_peered":"2024-05-09T00:44:23.179296+0000","last_clean":"2024-05-09T00:44:23.179296+0000","last_became_active":"2024-05-09T00:43:40.857075+0000","last_became_peered":"2024-05-09T00:43:40.857075+0000","last_unstale":"2024-05-09T00:44:23.179296+0000","last_undegraded":"2024-05-09T00:44:23.179296+0000","last_fullsized":"2024-05-09T00:44:23.179296+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:01:49.690226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.180515+0000","last_change":"2024-05-09T00:44:23.180515+0000","last_active":"2024-05-09T00:44:23.180515+0000","last_peered":"2024-05-09T00:44:23.180515+0000","last_clean":"2024-05-09T00:44:23.180515+0000","last_became_active":"2024-05-09T00:44:23.179834+0000","last_became_peered":"2024-05-09T00:44:23.179834+0000","last_unstale":"2024-05-09T00:44:23.180515+0000","last_undegraded":"2024-05-09T00:44:23.180515+0000","last_fullsized":"2024-05-09T00:44:23.180515+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:21:55.735438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,6],"acting":[11,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"78'1","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.015857+0000","last_change":"2024-05-09T00:43:40.960031+0000","last_active":"2024-05-09T00:44:23.015857+0000","last_peered":"2024-05-09T00:44:23.015857+0000","last_clean":"2024-05-09T00:44:23.015857+0000","last_became_active":"2024-05-09T00:43:40.959693+0000","last_became_peered":"2024-05-09T00:43:40.959693+0000","last_unstale":"2024-05-09T00:44:23.015857+0000","last_undegraded":"2024-05-09T00:44:23.015857+0000","last_fullsized":"2024-05-09T00:44:23.015857+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:55:51.903596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"78'1","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.015935+0000","last_change":"2024-05-09T00:43:43.886618+0000","last_active":"2024-05-09T00:44:23.015935+0000","last_peered":"2024-05-09T00:44:23.015935+0000","last_clean":"2024-05-09T00:44:23.015935+0000","last_became_active":"2024-05-09T00:43:43.885946+0000","last_became_peered":"2024-05-09T00:43:43.885946+0000","last_unstale":"2024-05-09T00:44:23.015935+0000","last_undegraded":"2024-05-09T00:44:23.015935+0000","last_fullsized":"2024-05-09T00:44:23.015935+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:16:22.169548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"78'5","reported_seq":59,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:27.998893+0000","last_change":"2024-05-09T00:43:40.856213+0000","last_active":"2024-05-09T00:44:27.998893+0000","last_peered":"2024-05-09T00:44:27.998893+0000","last_clean":"2024-05-09T00:44:27.998893+0000","last_became_active":"2024-05-09T00:43:40.855700+0000","last_became_peered":"2024-05-09T00:43:40.855700+0000","last_unstale":"2024-05-09T00:44:27.998893+0000","last_undegraded":"2024-05-09T00:44:27.998893+0000","last_fullsized":"2024-05-09T00:44:27.998893+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:48:50.238293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":28,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.015809+0000","last_change":"2024-05-09T00:43:43.885272+0000","last_active":"2024-05-09T00:44:23.015809+0000","last_peered":"2024-05-09T00:44:23.015809+0000","last_clean":"2024-05-09T00:44:23.015809+0000","last_became_active":"2024-05-09T00:43:43.884774+0000","last_became_peered":"2024-05-09T00:43:43.884774+0000","last_unstale":"2024-05-09T00:44:23.015809+0000","last_undegraded":"2024-05-09T00:44:23.015809+0000","last_fullsized":"2024-05-09T00:44:23.015809+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:14:02.193859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.015852+0000","last_change":"2024-05-09T00:43:43.883370+0000","last_active":"2024-05-09T00:44:23.015852+0000","last_peered":"2024-05-09T00:44:23.015852+0000","last_clean":"2024-05-09T00:44:23.015852+0000","last_became_active":"2024-05-09T00:43:43.881570+0000","last_became_peered":"2024-05-09T00:43:43.881570+0000","last_unstale":"2024-05-09T00:44:23.015852+0000","last_undegraded":"2024-05-09T00:44:23.015852+0000","last_fullsized":"2024-05-09T00:44:23.015852+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:48.936861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"78'5","reported_seq":58,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:26.997171+0000","last_change":"2024-05-09T00:43:40.959828+0000","last_active":"2024-05-09T00:44:26.997171+0000","last_peered":"2024-05-09T00:44:26.997171+0000","last_clean":"2024-05-09T00:44:26.997171+0000","last_became_active":"2024-05-09T00:43:40.958889+0000","last_became_peered":"2024-05-09T00:43:40.958889+0000","last_unstale":"2024-05-09T00:44:26.997171+0000","last_undegraded":"2024-05-09T00:44:26.997171+0000","last_fullsized":"2024-05-09T00:44:26.997171+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:24:50.113298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":27,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.014571+0000","last_change":"2024-05-09T00:43:43.876144+0000","last_active":"2024-05-09T00:44:23.014571+0000","last_peered":"2024-05-09T00:44:23.014571+0000","last_clean":"2024-05-09T00:44:23.014571+0000","last_became_active":"2024-05-09T00:43:43.875774+0000","last_became_peered":"2024-05-09T00:43:43.875774+0000","last_unstale":"2024-05-09T00:44:23.014571+0000","last_undegraded":"2024-05-09T00:44:23.014571+0000","last_fullsized":"2024-05-09T00:44:23.014571+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:05:23.145431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"78'6","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.179061+0000","last_change":"2024-05-09T00:43:40.857278+0000","last_active":"2024-05-09T00:44:23.179061+0000","last_peered":"2024-05-09T00:44:23.179061+0000","last_clean":"2024-05-09T00:44:23.179061+0000","last_became_active":"2024-05-09T00:43:40.856861+0000","last_became_peered":"2024-05-09T00:43:40.856861+0000","last_unstale":"2024-05-09T00:44:23.179061+0000","last_undegraded":"2024-05-09T00:44:23.179061+0000","last_fullsized":"2024-05-09T00:44:23.179061+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:36:16.603327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":98,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.020927+0000","last_change":"2024-05-09T00:42:14.528544+0000","last_active":"2024-05-09T00:44:23.020927+0000","last_peered":"2024-05-09T00:44:23.020927+0000","last_clean":"2024-05-09T00:44:23.020927+0000","last_became_active":"2024-05-09T00:42:14.523658+0000","last_became_peered":"2024-05-09T00:42:14.523658+0000","last_unstale":"2024-05-09T00:44:23.020927+0000","last_undegraded":"2024-05-09T00:44:23.020927+0000","last_fullsized":"2024-05-09T00:44:23.020927+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:38:48.637995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:38:48.637995+0000","last_clean_scrub_stamp":"2024-05-09T00:38:48.637995+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:42:28.249240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.178529+0000","last_change":"2024-05-09T00:43:43.873638+0000","last_active":"2024-05-09T00:44:23.178529+0000","last_peered":"2024-05-09T00:44:23.178529+0000","last_clean":"2024-05-09T00:44:23.178529+0000","last_became_active":"2024-05-09T00:43:43.871069+0000","last_became_peered":"2024-05-09T00:43:43.871069+0000","last_unstale":"2024-05-09T00:44:23.178529+0000","last_undegraded":"2024-05-09T00:44:23.178529+0000","last_fullsized":"2024-05-09T00:44:23.178529+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:28:34.996989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"78'4","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.021075+0000","last_change":"2024-05-09T00:43:40.856374+0000","last_active":"2024-05-09T00:44:23.021075+0000","last_peered":"2024-05-09T00:44:23.021075+0000","last_clean":"2024-05-09T00:44:23.021075+0000","last_became_active":"2024-05-09T00:43:40.855695+0000","last_became_peered":"2024-05-09T00:43:40.855695+0000","last_unstale":"2024-05-09T00:44:23.021075+0000","last_undegraded":"2024-05-09T00:44:23.021075+0000","last_fullsized":"2024-05-09T00:44:23.021075+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:33:43.734132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"78'4","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.179337+0000","last_change":"2024-05-09T00:43:40.959303+0000","last_active":"2024-05-09T00:44:23.179337+0000","last_peered":"2024-05-09T00:44:23.179337+0000","last_clean":"2024-05-09T00:44:23.179337+0000","last_became_active":"2024-05-09T00:43:40.958855+0000","last_became_peered":"2024-05-09T00:43:40.958855+0000","last_unstale":"2024-05-09T00:44:23.179337+0000","last_undegraded":"2024-05-09T00:44:23.179337+0000","last_fullsized":"2024-05-09T00:44:23.179337+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:53:22.566850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.178826+0000","last_change":"2024-05-09T00:43:43.872041+0000","last_active":"2024-05-09T00:44:23.178826+0000","last_peered":"2024-05-09T00:44:23.178826+0000","last_clean":"2024-05-09T00:44:23.178826+0000","last_became_active":"2024-05-09T00:43:43.870946+0000","last_became_peered":"2024-05-09T00:43:43.870946+0000","last_unstale":"2024-05-09T00:44:23.178826+0000","last_undegraded":"2024-05-09T00:44:23.178826+0000","last_fullsized":"2024-05-09T00:44:23.178826+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:16:28.190488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'14","reported_seq":40,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.016400+0000","last_change":"2024-05-09T00:43:40.853875+0000","last_active":"2024-05-09T00:44:23.016400+0000","last_peered":"2024-05-09T00:44:23.016400+0000","last_clean":"2024-05-09T00:44:23.016400+0000","last_became_active":"2024-05-09T00:43:40.853459+0000","last_became_peered":"2024-05-09T00:43:40.853459+0000","last_unstale":"2024-05-09T00:44:23.016400+0000","last_undegraded":"2024-05-09T00:44:23.016400+0000","last_fullsized":"2024-05-09T00:44:23.016400+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:45:29.992730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.014909+0000","last_change":"2024-05-09T00:43:43.870435+0000","last_active":"2024-05-09T00:44:23.014909+0000","last_peered":"2024-05-09T00:44:23.014909+0000","last_clean":"2024-05-09T00:44:23.014909+0000","last_became_active":"2024-05-09T00:43:43.870075+0000","last_became_peered":"2024-05-09T00:43:43.870075+0000","last_unstale":"2024-05-09T00:44:23.014909+0000","last_undegraded":"2024-05-09T00:44:23.014909+0000","last_fullsized":"2024-05-09T00:44:23.014909+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:53:12.958463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'7","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.179191+0000","last_change":"2024-05-09T00:43:41.119325+0000","last_active":"2024-05-09T00:44:23.179191+0000","last_peered":"2024-05-09T00:44:23.179191+0000","last_clean":"2024-05-09T00:44:23.179191+0000","last_became_active":"2024-05-09T00:43:41.119009+0000","last_became_peered":"2024-05-09T00:43:41.119009+0000","last_unstale":"2024-05-09T00:44:23.179191+0000","last_undegraded":"2024-05-09T00:44:23.179191+0000","last_fullsized":"2024-05-09T00:44:23.179191+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:10:05.775152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.018302+0000","last_change":"2024-05-09T00:43:43.875236+0000","last_active":"2024-05-09T00:44:23.018302+0000","last_peered":"2024-05-09T00:44:23.018302+0000","last_clean":"2024-05-09T00:44:23.018302+0000","last_became_active":"2024-05-09T00:43:43.874919+0000","last_became_peered":"2024-05-09T00:43:43.874919+0000","last_unstale":"2024-05-09T00:44:23.018302+0000","last_undegraded":"2024-05-09T00:44:23.018302+0000","last_fullsized":"2024-05-09T00:44:23.018302+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:45:33.232518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.018371+0000","last_change":"2024-05-09T00:43:40.959765+0000","last_active":"2024-05-09T00:44:23.018371+0000","last_peered":"2024-05-09T00:44:23.018371+0000","last_clean":"2024-05-09T00:44:23.018371+0000","last_became_active":"2024-05-09T00:43:40.959383+0000","last_became_peered":"2024-05-09T00:43:40.959383+0000","last_unstale":"2024-05-09T00:44:23.018371+0000","last_undegraded":"2024-05-09T00:44:23.018371+0000","last_fullsized":"2024-05-09T00:44:23.018371+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:13:28.766650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.178902+0000","last_change":"2024-05-09T00:43:43.881627+0000","last_active":"2024-05-09T00:44:23.178902+0000","last_peered":"2024-05-09T00:44:23.178902+0000","last_clean":"2024-05-09T00:44:23.178902+0000","last_became_active":"2024-05-09T00:43:43.881207+0000","last_became_peered":"2024-05-09T00:43:43.881207+0000","last_unstale":"2024-05-09T00:44:23.178902+0000","last_undegraded":"2024-05-09T00:44:23.178902+0000","last_fullsized":"2024-05-09T00:44:23.178902+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:44:08.053638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'7","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.014430+0000","last_change":"2024-05-09T00:43:40.859566+0000","last_active":"2024-05-09T00:44:23.014430+0000","last_peered":"2024-05-09T00:44:23.014430+0000","last_clean":"2024-05-09T00:44:23.014430+0000","last_became_active":"2024-05-09T00:43:40.859182+0000","last_became_peered":"2024-05-09T00:43:40.859182+0000","last_unstale":"2024-05-09T00:44:23.014430+0000","last_undegraded":"2024-05-09T00:44:23.014430+0000","last_fullsized":"2024-05-09T00:44:23.014430+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:58:12.693271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.017931+0000","last_change":"2024-05-09T00:43:43.869193+0000","last_active":"2024-05-09T00:44:23.017931+0000","last_peered":"2024-05-09T00:44:23.017931+0000","last_clean":"2024-05-09T00:44:23.017931+0000","last_became_active":"2024-05-09T00:43:43.868823+0000","last_became_peered":"2024-05-09T00:43:43.868823+0000","last_unstale":"2024-05-09T00:44:23.017931+0000","last_undegraded":"2024-05-09T00:44:23.017931+0000","last_fullsized":"2024-05-09T00:44:23.017931+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:15:53.929873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.019050+0000","last_change":"2024-05-09T00:43:40.855038+0000","last_active":"2024-05-09T00:44:23.019050+0000","last_peered":"2024-05-09T00:44:23.019050+0000","last_clean":"2024-05-09T00:44:23.019050+0000","last_became_active":"2024-05-09T00:43:40.854334+0000","last_became_peered":"2024-05-09T00:43:40.854334+0000","last_unstale":"2024-05-09T00:44:23.019050+0000","last_undegraded":"2024-05-09T00:44:23.019050+0000","last_fullsized":"2024-05-09T00:44:23.019050+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:47:13.897697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.019151+0000","last_change":"2024-05-09T00:44:23.019151+0000","last_active":"2024-05-09T00:44:23.019151+0000","last_peered":"2024-05-09T00:44:23.019151+0000","last_clean":"2024-05-09T00:44:23.019151+0000","last_became_active":"2024-05-09T00:44:23.018752+0000","last_became_peered":"2024-05-09T00:44:23.018752+0000","last_unstale":"2024-05-09T00:44:23.019151+0000","last_undegraded":"2024-05-09T00:44:23.019151+0000","last_fullsized":"2024-05-09T00:44:23.019151+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:28:41.875747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.178988+0000","last_change":"2024-05-09T00:43:40.859461+0000","last_active":"2024-05-09T00:44:23.178988+0000","last_peered":"2024-05-09T00:44:23.178988+0000","last_clean":"2024-05-09T00:44:23.178988+0000","last_became_active":"2024-05-09T00:43:40.857814+0000","last_became_peered":"2024-05-09T00:43:40.857814+0000","last_unstale":"2024-05-09T00:44:23.178988+0000","last_undegraded":"2024-05-09T00:44:23.178988+0000","last_fullsized":"2024-05-09T00:44:23.178988+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:56:06.279762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.021237+0000","last_change":"2024-05-09T00:43:43.878120+0000","last_active":"2024-05-09T00:44:23.021237+0000","last_peered":"2024-05-09T00:44:23.021237+0000","last_clean":"2024-05-09T00:44:23.021237+0000","last_became_active":"2024-05-09T00:43:43.877782+0000","last_became_peered":"2024-05-09T00:43:43.877782+0000","last_unstale":"2024-05-09T00:44:23.021237+0000","last_undegraded":"2024-05-09T00:44:23.021237+0000","last_fullsized":"2024-05-09T00:44:23.021237+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:19:48.558992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'6","reported_seq":60,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:25.997625+0000","last_change":"2024-05-09T00:43:41.119347+0000","last_active":"2024-05-09T00:44:25.997625+0000","last_peered":"2024-05-09T00:44:25.997625+0000","last_clean":"2024-05-09T00:44:25.997625+0000","last_became_active":"2024-05-09T00:43:41.118847+0000","last_became_peered":"2024-05-09T00:43:41.118847+0000","last_unstale":"2024-05-09T00:44:25.997625+0000","last_undegraded":"2024-05-09T00:44:25.997625+0000","last_fullsized":"2024-05-09T00:44:25.997625+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:07:03.977756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":28,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.021588+0000","last_change":"2024-05-09T00:43:43.875203+0000","last_active":"2024-05-09T00:44:23.021588+0000","last_peered":"2024-05-09T00:44:23.021588+0000","last_clean":"2024-05-09T00:44:23.021588+0000","last_became_active":"2024-05-09T00:43:43.874882+0000","last_became_peered":"2024-05-09T00:43:43.874882+0000","last_unstale":"2024-05-09T00:44:23.021588+0000","last_undegraded":"2024-05-09T00:44:23.021588+0000","last_fullsized":"2024-05-09T00:44:23.021588+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:54:27.845086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'3","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.015990+0000","last_change":"2024-05-09T00:43:40.857955+0000","last_active":"2024-05-09T00:44:23.015990+0000","last_peered":"2024-05-09T00:44:23.015990+0000","last_clean":"2024-05-09T00:44:23.015990+0000","last_became_active":"2024-05-09T00:43:40.857596+0000","last_became_peered":"2024-05-09T00:43:40.857596+0000","last_unstale":"2024-05-09T00:44:23.015990+0000","last_undegraded":"2024-05-09T00:44:23.015990+0000","last_fullsized":"2024-05-09T00:44:23.015990+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:13:56.676929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"78'1","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.021334+0000","last_change":"2024-05-09T00:43:43.882487+0000","last_active":"2024-05-09T00:44:23.021334+0000","last_peered":"2024-05-09T00:44:23.021334+0000","last_clean":"2024-05-09T00:44:23.021334+0000","last_became_active":"2024-05-09T00:43:43.882176+0000","last_became_peered":"2024-05-09T00:43:43.882176+0000","last_unstale":"2024-05-09T00:44:23.021334+0000","last_undegraded":"2024-05-09T00:44:23.021334+0000","last_fullsized":"2024-05-09T00:44:23.021334+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:48:42.354433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'5","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.178994+0000","last_change":"2024-05-09T00:43:40.857215+0000","last_active":"2024-05-09T00:44:23.178994+0000","last_peered":"2024-05-09T00:44:23.178994+0000","last_clean":"2024-05-09T00:44:23.178994+0000","last_became_active":"2024-05-09T00:43:40.856858+0000","last_became_peered":"2024-05-09T00:43:40.856858+0000","last_unstale":"2024-05-09T00:44:23.178994+0000","last_undegraded":"2024-05-09T00:44:23.178994+0000","last_fullsized":"2024-05-09T00:44:23.178994+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:40:47.943482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.021561+0000","last_change":"2024-05-09T00:43:43.881192+0000","last_active":"2024-05-09T00:44:23.021561+0000","last_peered":"2024-05-09T00:44:23.021561+0000","last_clean":"2024-05-09T00:44:23.021561+0000","last_became_active":"2024-05-09T00:43:43.880781+0000","last_became_peered":"2024-05-09T00:43:43.880781+0000","last_unstale":"2024-05-09T00:44:23.021561+0000","last_undegraded":"2024-05-09T00:44:23.021561+0000","last_fullsized":"2024-05-09T00:44:23.021561+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:18:31.577354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.018034+0000","last_change":"2024-05-09T00:43:43.876869+0000","last_active":"2024-05-09T00:44:23.018034+0000","last_peered":"2024-05-09T00:44:23.018034+0000","last_clean":"2024-05-09T00:44:23.018034+0000","last_became_active":"2024-05-09T00:43:43.876439+0000","last_became_peered":"2024-05-09T00:43:43.876439+0000","last_unstale":"2024-05-09T00:44:23.018034+0000","last_undegraded":"2024-05-09T00:44:23.018034+0000","last_fullsized":"2024-05-09T00:44:23.018034+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:39:06.776267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.019040+0000","last_change":"2024-05-09T00:43:43.884677+0000","last_active":"2024-05-09T00:44:23.019040+0000","last_peered":"2024-05-09T00:44:23.019040+0000","last_clean":"2024-05-09T00:44:23.019040+0000","last_became_active":"2024-05-09T00:43:43.884238+0000","last_became_peered":"2024-05-09T00:43:43.884238+0000","last_unstale":"2024-05-09T00:44:23.019040+0000","last_undegraded":"2024-05-09T00:44:23.019040+0000","last_fullsized":"2024-05-09T00:44:23.019040+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:18:13.787584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.019033+0000","last_change":"2024-05-09T00:43:43.876638+0000","last_active":"2024-05-09T00:44:23.019033+0000","last_peered":"2024-05-09T00:44:23.019033+0000","last_clean":"2024-05-09T00:44:23.019033+0000","last_became_active":"2024-05-09T00:43:43.876315+0000","last_became_peered":"2024-05-09T00:43:43.876315+0000","last_unstale":"2024-05-09T00:44:23.019033+0000","last_undegraded":"2024-05-09T00:44:23.019033+0000","last_fullsized":"2024-05-09T00:44:23.019033+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:22:41.618613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.017968+0000","last_change":"2024-05-09T00:43:43.869798+0000","last_active":"2024-05-09T00:44:23.017968+0000","last_peered":"2024-05-09T00:44:23.017968+0000","last_clean":"2024-05-09T00:44:23.017968+0000","last_became_active":"2024-05-09T00:43:43.869444+0000","last_became_peered":"2024-05-09T00:43:43.869444+0000","last_unstale":"2024-05-09T00:44:23.017968+0000","last_undegraded":"2024-05-09T00:44:23.017968+0000","last_fullsized":"2024-05-09T00:44:23.017968+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:15:53.267441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.178838+0000","last_change":"2024-05-09T00:43:43.873215+0000","last_active":"2024-05-09T00:44:23.178838+0000","last_peered":"2024-05-09T00:44:23.178838+0000","last_clean":"2024-05-09T00:44:23.178838+0000","last_became_active":"2024-05-09T00:43:43.872763+0000","last_became_peered":"2024-05-09T00:43:43.872763+0000","last_unstale":"2024-05-09T00:44:23.178838+0000","last_undegraded":"2024-05-09T00:44:23.178838+0000","last_fullsized":"2024-05-09T00:44:23.178838+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:08.997387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.018254+0000","last_change":"2024-05-09T00:43:43.876514+0000","last_active":"2024-05-09T00:44:23.018254+0000","last_peered":"2024-05-09T00:44:23.018254+0000","last_clean":"2024-05-09T00:44:23.018254+0000","last_became_active":"2024-05-09T00:43:43.876147+0000","last_became_peered":"2024-05-09T00:43:43.876147+0000","last_unstale":"2024-05-09T00:44:23.018254+0000","last_undegraded":"2024-05-09T00:44:23.018254+0000","last_fullsized":"2024-05-09T00:44:23.018254+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:58:34.745819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.019620+0000","last_change":"2024-05-09T00:43:43.871869+0000","last_active":"2024-05-09T00:44:23.019620+0000","last_peered":"2024-05-09T00:44:23.019620+0000","last_clean":"2024-05-09T00:44:23.019620+0000","last_became_active":"2024-05-09T00:43:43.871447+0000","last_became_peered":"2024-05-09T00:43:43.871447+0000","last_unstale":"2024-05-09T00:44:23.019620+0000","last_undegraded":"2024-05-09T00:44:23.019620+0000","last_fullsized":"2024-05-09T00:44:23.019620+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:11:42.128931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.014446+0000","last_change":"2024-05-09T00:43:43.871291+0000","last_active":"2024-05-09T00:44:23.014446+0000","last_peered":"2024-05-09T00:44:23.014446+0000","last_clean":"2024-05-09T00:44:23.014446+0000","last_became_active":"2024-05-09T00:43:43.870899+0000","last_became_peered":"2024-05-09T00:43:43.870899+0000","last_unstale":"2024-05-09T00:44:23.014446+0000","last_undegraded":"2024-05-09T00:44:23.014446+0000","last_fullsized":"2024-05-09T00:44:23.014446+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:28:53.013543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.015578+0000","last_change":"2024-05-09T00:43:43.884791+0000","last_active":"2024-05-09T00:44:23.015578+0000","last_peered":"2024-05-09T00:44:23.015578+0000","last_clean":"2024-05-09T00:44:23.015578+0000","last_became_active":"2024-05-09T00:43:43.877333+0000","last_became_peered":"2024-05-09T00:43:43.877333+0000","last_unstale":"2024-05-09T00:44:23.015578+0000","last_undegraded":"2024-05-09T00:44:23.015578+0000","last_fullsized":"2024-05-09T00:44:23.015578+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:37:07.535388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.014278+0000","last_change":"2024-05-09T00:43:43.863773+0000","last_active":"2024-05-09T00:44:23.014278+0000","last_peered":"2024-05-09T00:44:23.014278+0000","last_clean":"2024-05-09T00:44:23.014278+0000","last_became_active":"2024-05-09T00:43:43.863436+0000","last_became_peered":"2024-05-09T00:43:43.863436+0000","last_unstale":"2024-05-09T00:44:23.014278+0000","last_undegraded":"2024-05-09T00:44:23.014278+0000","last_fullsized":"2024-05-09T00:44:23.014278+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:59:57.100234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.018880+0000","last_change":"2024-05-09T00:43:43.873461+0000","last_active":"2024-05-09T00:44:23.018880+0000","last_peered":"2024-05-09T00:44:23.018880+0000","last_clean":"2024-05-09T00:44:23.018880+0000","last_became_active":"2024-05-09T00:43:43.873139+0000","last_became_peered":"2024-05-09T00:43:43.873139+0000","last_unstale":"2024-05-09T00:44:23.018880+0000","last_undegraded":"2024-05-09T00:44:23.018880+0000","last_fullsized":"2024-05-09T00:44:23.018880+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:24:57.865569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.178871+0000","last_change":"2024-05-09T00:43:43.874401+0000","last_active":"2024-05-09T00:44:23.178871+0000","last_peered":"2024-05-09T00:44:23.178871+0000","last_clean":"2024-05-09T00:44:23.178871+0000","last_became_active":"2024-05-09T00:43:43.873765+0000","last_became_peered":"2024-05-09T00:43:43.873765+0000","last_unstale":"2024-05-09T00:44:23.178871+0000","last_undegraded":"2024-05-09T00:44:23.178871+0000","last_fullsized":"2024-05-09T00:44:23.178871+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:23:59.509415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:44:23.018033+0000","last_change":"2024-05-09T00:43:43.875219+0000","last_active":"2024-05-09T00:44:23.018033+0000","last_peered":"2024-05-09T00:44:23.018033+0000","last_clean":"2024-05-09T00:44:23.018033+0000","last_became_active":"2024-05-09T00:43:43.874824+0000","last_became_peered":"2024-05-09T00:43:43.874824+0000","last_unstale":"2024-05-09T00:44:23.018033+0000","last_undegraded":"2024-05-09T00:44:23.018033+0000","last_fullsized":"2024-05-09T00:44:23.018033+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:39:41.929521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":262144,"data_stored":2951840,"data_compressed":179218,"data_compressed_allocated":225280,"data_compressed_original":2932736,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65673,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":292,"num_read_kb":208,"num_write":244,"num_write_kb":134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":52683,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":528384,"data_stored":197019,"data_compressed":8493,"data_compressed_allocated":12288,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},"log_size":210,"ondisk_log_size":210,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710744,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":872448,"data_stored":426179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906974,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":1075286,"data_compressed":36168,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070499,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":425718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266728,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":892928,"data_stored":426096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462958,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":946176,"data_stored":1019005,"data_compressed":36168,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626483,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":880640,"data_stored":425435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":942080,"data_stored":484426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738449,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":427165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411398,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":427350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182466,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36220,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708932,"statfs":{"total":95995035648,"available":95957946368,"internally_reserved":0,"allocated":897024,"data_stored":425260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986236,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":913408,"data_stored":483765,"data_compressed":8493,"data_compressed_allocated":12288,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822712,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":1018292,"data_compressed":36168,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34546,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36168,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36168,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36168,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36168,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":59830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":59169,"data_compressed":8493,"data_compressed_allocated":12288,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":60205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T00:44:34.447 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-09T00:44:34.448 INFO:tasks.cephadm:Setting up client nodes... 2024-05-09T00:44:34.448 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-09T00:44:34.705 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:44:35.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:35 smithi131 ceph-mon[34547]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T00:44:35.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:35 smithi131 ceph-mon[34547]: pgmap v316: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 3.6 KiB/s wr, 12 op/s; 4.7 KiB/s, 0 objects/s recovering 2024-05-09T00:44:35.422 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:35 smithi005 ceph-mon[30365]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T00:44:35.422 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:35 smithi005 ceph-mon[30365]: pgmap v316: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 3.6 KiB/s wr, 12 op/s; 4.7 KiB/s, 0 objects/s recovering 2024-05-09T00:44:35.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:35 smithi157 ceph-mon[34294]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T00:44:35.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:35 smithi157 ceph-mon[34294]: pgmap v316: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 3.6 KiB/s wr, 12 op/s; 4.7 KiB/s, 0 objects/s recovering 2024-05-09T00:44:36.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:36.534+0000 7f2cd35d8700 1 -- 172.21.15.5:0/1192113044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ccc105e40 msgr2=0x7f2ccc10c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:36.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:36.534+0000 7f2cd35d8700 1 --2- 172.21.15.5:0/1192113044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ccc105e40 0x7f2ccc10c6f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2cc400b0a0 tx=0x7f2cc400b3b0 comp rx=0 tx=0).stop 2024-05-09T00:44:36.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:36.534+0000 7f2cd35d8700 1 -- 172.21.15.5:0/1192113044 shutdown_connections 2024-05-09T00:44:36.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:36.534+0000 7f2cd35d8700 1 --2- 172.21.15.5:0/1192113044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ccc105e40 0x7f2ccc10c6f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:36.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:36.534+0000 7f2cd35d8700 1 --2- 172.21.15.5:0/1192113044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ccc104370 0x7f2ccc104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:36.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:36.534+0000 7f2cd35d8700 1 --2- 172.21.15.5:0/1192113044 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ccc1054a0 0x7f2ccc1058b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:36.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:36.534+0000 7f2cd35d8700 1 -- 172.21.15.5:0/1192113044 >> 172.21.15.5:0/1192113044 conn(0x7f2ccc0ff9d0 msgr2=0x7f2ccc101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:44:36.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:36.535+0000 7f2cd35d8700 1 -- 172.21.15.5:0/1192113044 shutdown_connections 2024-05-09T00:44:36.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:36.535+0000 7f2cd35d8700 1 -- 172.21.15.5:0/1192113044 wait complete. 2024-05-09T00:44:36.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:36.535+0000 7f2cd35d8700 1 Processor -- start 2024-05-09T00:44:36.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:36.535+0000 7f2cd35d8700 1 -- start start 2024-05-09T00:44:36.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:36.536+0000 7f2cd35d8700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ccc104370 0x7f2ccc078570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:44:36.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:36.536+0000 7f2cd35d8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ccc1054a0 0x7f2ccc076c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:44:36.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:36.536+0000 7f2cd35d8700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ccc105e40 0x7f2ccc077160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:44:36.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:36.536+0000 7f2cd35d8700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2ccc10fe50 con 0x7f2ccc1054a0 2024-05-09T00:44:36.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:36.536+0000 7f2cd35d8700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2ccc10fcd0 con 0x7f2ccc105e40 2024-05-09T00:44:36.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:36.536+0000 7f2cd35d8700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2ccc10ffd0 con 0x7f2ccc104370 2024-05-09T00:44:36.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:36.536+0000 7f2cd0b73700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ccc1054a0 0x7f2ccc076c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:44:36.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:36.536+0000 7f2cd0b73700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ccc1054a0 0x7f2ccc076c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44590/0 (socket says 172.21.15.5:44590) 2024-05-09T00:44:36.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:36.536+0000 7f2cd0b73700 1 -- 172.21.15.5:0/795518116 learned_addr learned my addr 172.21.15.5:0/795518116 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:44:36.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:36.536+0000 7f2cd1b75700 1 --2- 172.21.15.5:0/795518116 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ccc105e40 0x7f2ccc077160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:44:36.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:36.536+0000 7f2cd1374700 1 --2- 172.21.15.5:0/795518116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ccc104370 0x7f2ccc078570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:44:36.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:36.536+0000 7f2cd0b73700 1 -- 172.21.15.5:0/795518116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ccc104370 msgr2=0x7f2ccc078570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:36.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:36.536+0000 7f2cd0b73700 1 --2- 172.21.15.5:0/795518116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ccc104370 0x7f2ccc078570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:36.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:36.536+0000 7f2cd0b73700 1 -- 172.21.15.5:0/795518116 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ccc105e40 msgr2=0x7f2ccc077160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:36.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:36.536+0000 7f2cd0b73700 1 --2- 172.21.15.5:0/795518116 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ccc105e40 0x7f2ccc077160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:36.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:36.536+0000 7f2cd0b73700 1 -- 172.21.15.5:0/795518116 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ccc077830 con 0x7f2ccc1054a0 2024-05-09T00:44:36.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:36.536+0000 7f2cd1374700 1 --2- 172.21.15.5:0/795518116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ccc104370 0x7f2ccc078570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:44:36.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:36.536+0000 7f2cd0b73700 1 --2- 172.21.15.5:0/795518116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ccc1054a0 0x7f2ccc076c20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc00c9b0 tx=0x7f2cbc00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:44:36.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:36.536+0000 7f2cc27fc700 1 -- 172.21.15.5:0/795518116 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cbc007b00 con 0x7f2ccc1054a0 2024-05-09T00:44:36.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:36.536+0000 7f2cc27fc700 1 -- 172.21.15.5:0/795518116 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2cbc007ca0 con 0x7f2ccc1054a0 2024-05-09T00:44:36.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:36.536+0000 7f2cc27fc700 1 -- 172.21.15.5:0/795518116 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cbc018660 con 0x7f2ccc1054a0 2024-05-09T00:44:36.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:36.536+0000 7f2cd35d8700 1 -- 172.21.15.5:0/795518116 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ccc1a49d0 con 0x7f2ccc1054a0 2024-05-09T00:44:36.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:36.537+0000 7f2cd35d8700 1 -- 172.21.15.5:0/795518116 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ccc1a4f00 con 0x7f2ccc1054a0 2024-05-09T00:44:36.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:36.537+0000 7f2cd35d8700 1 -- 172.21.15.5:0/795518116 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ccc0512c0 con 0x7f2ccc1054a0 2024-05-09T00:44:36.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:36.540+0000 7f2cc27fc700 1 -- 172.21.15.5:0/795518116 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f2cbc00f450 con 0x7f2ccc1054a0 2024-05-09T00:44:36.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:36.541+0000 7f2cc27fc700 1 --2- 172.21.15.5:0/795518116 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f2cb80b2990 0x7f2cb80b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:44:36.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:36.541+0000 7f2cd1374700 1 --2- 172.21.15.5:0/795518116 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f2cb80b2990 0x7f2cb80b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:44:36.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:36.541+0000 7f2cc27fc700 1 -- 172.21.15.5:0/795518116 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8227+0+0 (secure 0 0 0) 0x7f2cbc0e2610 con 0x7f2ccc1054a0 2024-05-09T00:44:36.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:36.541+0000 7f2cd1374700 1 --2- 172.21.15.5:0/795518116 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f2cb80b2990 0x7f2cb80b4ea0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc1053a0 tx=0x7f2cc8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:44:36.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:36.544+0000 7f2cc27fc700 1 -- 172.21.15.5:0/795518116 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2cbc014030 con 0x7f2ccc1054a0 2024-05-09T00:44:36.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:36.852+0000 7f2cd35d8700 1 -- 172.21.15.5:0/795518116 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f2ccc02e5f0 con 0x7f2ccc1054a0 2024-05-09T00:44:36.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:36.855+0000 7f2cc27fc700 1 -- 172.21.15.5:0/795518116 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f2cbc0aa280 con 0x7f2ccc1054a0 2024-05-09T00:44:36.856 INFO:teuthology.orchestra.run.smithi005.stdout:[client.0] 2024-05-09T00:44:36.856 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQB0HDxmJpDfMhAAaUfgK5qpwoilYJCY58tJSg== 2024-05-09T00:44:36.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:36.858+0000 7f2cd35d8700 1 -- 172.21.15.5:0/795518116 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f2cb80b2990 msgr2=0x7f2cb80b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:36.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:36.858+0000 7f2cd35d8700 1 --2- 172.21.15.5:0/795518116 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f2cb80b2990 0x7f2cb80b4ea0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc1053a0 tx=0x7f2cc8008040 comp rx=0 tx=0).stop 2024-05-09T00:44:36.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:36.858+0000 7f2cd35d8700 1 -- 172.21.15.5:0/795518116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ccc1054a0 msgr2=0x7f2ccc076c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:36.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:36.858+0000 7f2cd35d8700 1 --2- 172.21.15.5:0/795518116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ccc1054a0 0x7f2ccc076c20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc00c9b0 tx=0x7f2cbc00ccc0 comp rx=0 tx=0).stop 2024-05-09T00:44:36.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:36.858+0000 7f2cd35d8700 1 -- 172.21.15.5:0/795518116 shutdown_connections 2024-05-09T00:44:36.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:36.858+0000 7f2cd35d8700 1 --2- 172.21.15.5:0/795518116 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f2cb80b2990 0x7f2cb80b4ea0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:36.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:36.858+0000 7f2cd35d8700 1 --2- 172.21.15.5:0/795518116 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ccc105e40 0x7f2ccc077160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:36.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:36.858+0000 7f2cd35d8700 1 --2- 172.21.15.5:0/795518116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ccc104370 0x7f2ccc078570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:36.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:36.858+0000 7f2cd35d8700 1 --2- 172.21.15.5:0/795518116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ccc1054a0 0x7f2ccc076c20 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:36.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:36.858+0000 7f2cd35d8700 1 -- 172.21.15.5:0/795518116 >> 172.21.15.5:0/795518116 conn(0x7f2ccc0ff9d0 msgr2=0x7f2ccc1086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:44:36.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:36.858+0000 7f2cd35d8700 1 -- 172.21.15.5:0/795518116 shutdown_connections 2024-05-09T00:44:36.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:36.858+0000 7f2cd35d8700 1 -- 172.21.15.5:0/795518116 wait complete. 2024-05-09T00:44:37.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:37 smithi131 ceph-mon[34547]: pgmap v317: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.1 KiB/s wr, 10 op/s; 4.3 KiB/s, 0 objects/s recovering 2024-05-09T00:44:37.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:37 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/795518116' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-09T00:44:37.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:37 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/795518116' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-09T00:44:37.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:37 smithi157 ceph-mon[34294]: pgmap v317: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.1 KiB/s wr, 10 op/s; 4.3 KiB/s, 0 objects/s recovering 2024-05-09T00:44:37.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:37 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/795518116' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-09T00:44:37.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:37 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/795518116' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-09T00:44:37.537 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:37 smithi005 ceph-mon[30365]: pgmap v317: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.1 KiB/s wr, 10 op/s; 4.3 KiB/s, 0 objects/s recovering 2024-05-09T00:44:37.538 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:37 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/795518116' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-09T00:44:37.538 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:37 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/795518116' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-09T00:44:37.571 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-09T00:44:37.571 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-09T00:44:37.571 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-09T00:44:37.613 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-09T00:44:37.613 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-09T00:44:37.613 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph mgr dump --format=json 2024-05-09T00:44:37.907 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:44:39.365 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:39 smithi005 ceph-mon[30365]: pgmap v318: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.1 KiB/s wr, 10 op/s; 4.3 KiB/s, 0 objects/s recovering 2024-05-09T00:44:39.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:39 smithi157 ceph-mon[34294]: pgmap v318: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.1 KiB/s wr, 10 op/s; 4.3 KiB/s, 0 objects/s recovering 2024-05-09T00:44:39.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:39 smithi131 ceph-mon[34547]: pgmap v318: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.1 KiB/s wr, 10 op/s; 4.3 KiB/s, 0 objects/s recovering 2024-05-09T00:44:39.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:39.617+0000 7febfc129700 1 -- 172.21.15.5:0/1773187872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febf41034c0 msgr2=0x7febf41038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:39.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:39.617+0000 7febfc129700 1 --2- 172.21.15.5:0/1773187872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febf41034c0 0x7febf41038d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7febf00099c0 tx=0x7febf0009cd0 comp rx=0 tx=0).stop 2024-05-09T00:44:39.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:39.617+0000 7febfc129700 1 -- 172.21.15.5:0/1773187872 shutdown_connections 2024-05-09T00:44:39.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:39.617+0000 7febfc129700 1 --2- 172.21.15.5:0/1773187872 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7febf4108740 0x7febf41100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:39.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:39.617+0000 7febfc129700 1 --2- 172.21.15.5:0/1773187872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febf41034c0 0x7febf41038d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:39.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:39.617+0000 7febfc129700 1 --2- 172.21.15.5:0/1773187872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febf4107df0 0x7febf4108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:39.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:39.617+0000 7febfc129700 1 -- 172.21.15.5:0/1773187872 >> 172.21.15.5:0/1773187872 conn(0x7febf40fedf0 msgr2=0x7febf4101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:44:39.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:39.617+0000 7febfc129700 1 -- 172.21.15.5:0/1773187872 shutdown_connections 2024-05-09T00:44:39.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:39.617+0000 7febfc129700 1 -- 172.21.15.5:0/1773187872 wait complete. 2024-05-09T00:44:39.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:39.618+0000 7febfc129700 1 Processor -- start 2024-05-09T00:44:39.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:39.618+0000 7febfc129700 1 -- start start 2024-05-09T00:44:39.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:39.618+0000 7febfc129700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febf41034c0 0x7febf4199d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:44:39.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:39.618+0000 7febfc129700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7febf4107df0 0x7febf419a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:44:39.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:39.618+0000 7febfc129700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febf4108740 0x7febf419e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:44:39.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:39.619+0000 7febfc129700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7febf41133c0 con 0x7febf4108740 2024-05-09T00:44:39.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:39.619+0000 7febfc129700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7febf4113240 con 0x7febf4107df0 2024-05-09T00:44:39.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:39.619+0000 7febfc129700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7febf4113540 con 0x7febf41034c0 2024-05-09T00:44:39.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:39.619+0000 7febfa6c6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febf4108740 0x7febf419e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:44:39.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:39.619+0000 7febfa6c6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febf4108740 0x7febf419e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44612/0 (socket says 172.21.15.5:44612) 2024-05-09T00:44:39.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:39.619+0000 7febfa6c6700 1 -- 172.21.15.5:0/1162468913 learned_addr learned my addr 172.21.15.5:0/1162468913 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:44:39.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:39.619+0000 7febf9ec5700 1 --2- 172.21.15.5:0/1162468913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febf41034c0 0x7febf4199d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:44:39.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:39.619+0000 7febf96c4700 1 --2- 172.21.15.5:0/1162468913 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7febf4107df0 0x7febf419a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:44:39.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:39.619+0000 7febfa6c6700 1 -- 172.21.15.5:0/1162468913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febf41034c0 msgr2=0x7febf4199d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:39.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:39.619+0000 7febfa6c6700 1 --2- 172.21.15.5:0/1162468913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febf41034c0 0x7febf4199d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:39.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:39.619+0000 7febfa6c6700 1 -- 172.21.15.5:0/1162468913 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7febf4107df0 msgr2=0x7febf419a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:39.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:39.619+0000 7febfa6c6700 1 --2- 172.21.15.5:0/1162468913 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7febf4107df0 0x7febf419a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:39.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:39.619+0000 7febfa6c6700 1 -- 172.21.15.5:0/1162468913 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febf419ebb0 con 0x7febf4108740 2024-05-09T00:44:39.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:39.619+0000 7febf96c4700 1 --2- 172.21.15.5:0/1162468913 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7febf4107df0 0x7febf419a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:44:39.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:39.619+0000 7febfa6c6700 1 --2- 172.21.15.5:0/1162468913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febf4108740 0x7febf419e5c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7febec00efb0 tx=0x7febec00c550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:44:39.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:39.619+0000 7febe6ffd700 1 -- 172.21.15.5:0/1162468913 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febec011670 con 0x7febf4108740 2024-05-09T00:44:39.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:39.619+0000 7febe6ffd700 1 -- 172.21.15.5:0/1162468913 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febec011cc0 con 0x7febf4108740 2024-05-09T00:44:39.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:39.619+0000 7febe6ffd700 1 -- 172.21.15.5:0/1162468913 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febec01acd0 con 0x7febf4108740 2024-05-09T00:44:39.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:39.619+0000 7febfc129700 1 -- 172.21.15.5:0/1162468913 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febf419eea0 con 0x7febf4108740 2024-05-09T00:44:39.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:39.620+0000 7febfc129700 1 -- 172.21.15.5:0/1162468913 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febf419f390 con 0x7febf4108740 2024-05-09T00:44:39.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:39.621+0000 7febfc129700 1 -- 172.21.15.5:0/1162468913 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febf40512c0 con 0x7febf4108740 2024-05-09T00:44:39.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:39.622+0000 7febe6ffd700 1 -- 172.21.15.5:0/1162468913 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7febec011810 con 0x7febf4108740 2024-05-09T00:44:39.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:39.623+0000 7febe6ffd700 1 --2- 172.21.15.5:0/1162468913 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7febe00b2990 0x7febe00b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:44:39.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:39.623+0000 7febe6ffd700 1 -- 172.21.15.5:0/1162468913 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8227+0+0 (secure 0 0 0) 0x7febec0e4b90 con 0x7febf4108740 2024-05-09T00:44:39.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:39.623+0000 7febf9ec5700 1 --2- 172.21.15.5:0/1162468913 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7febe00b2990 0x7febe00b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:44:39.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:39.624+0000 7febf9ec5700 1 --2- 172.21.15.5:0/1162468913 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7febe00b2990 0x7febe00b4ea0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7febf000b580 tx=0x7febf00036e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:44:39.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:39.627+0000 7febe6ffd700 1 -- 172.21.15.5:0/1162468913 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7febec016030 con 0x7febf4108740 2024-05-09T00:44:39.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:39.909+0000 7febfc129700 1 -- 172.21.15.5:0/1162468913 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7febf419b0f0 con 0x7febf4108740 2024-05-09T00:44:39.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:39.915+0000 7febe6ffd700 1 -- 172.21.15.5:0/1162468913 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+288631 (secure 0 0 0) 0x7febf419b0f0 con 0x7febf4108740 2024-05-09T00:44:39.920 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:44:39.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:39.920+0000 7febfc129700 1 -- 172.21.15.5:0/1162468913 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7febe00b2990 msgr2=0x7febe00b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:39.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:39.920+0000 7febfc129700 1 --2- 172.21.15.5:0/1162468913 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7febe00b2990 0x7febe00b4ea0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7febf000b580 tx=0x7febf00036e0 comp rx=0 tx=0).stop 2024-05-09T00:44:39.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:39.920+0000 7febfc129700 1 -- 172.21.15.5:0/1162468913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febf4108740 msgr2=0x7febf419e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:39.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:39.920+0000 7febfc129700 1 --2- 172.21.15.5:0/1162468913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febf4108740 0x7febf419e5c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7febec00efb0 tx=0x7febec00c550 comp rx=0 tx=0).stop 2024-05-09T00:44:39.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:39.920+0000 7febfc129700 1 -- 172.21.15.5:0/1162468913 shutdown_connections 2024-05-09T00:44:39.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:39.920+0000 7febfc129700 1 --2- 172.21.15.5:0/1162468913 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7febe00b2990 0x7febe00b4ea0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:39.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:39.920+0000 7febfc129700 1 --2- 172.21.15.5:0/1162468913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febf4108740 0x7febf419e5c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:39.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:39.920+0000 7febfc129700 1 --2- 172.21.15.5:0/1162468913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febf41034c0 0x7febf4199d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:39.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:39.920+0000 7febfc129700 1 --2- 172.21.15.5:0/1162468913 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7febf4107df0 0x7febf419a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:39.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:39.920+0000 7febfc129700 1 -- 172.21.15.5:0/1162468913 >> 172.21.15.5:0/1162468913 conn(0x7febf40fedf0 msgr2=0x7febf4101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:44:39.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:39.920+0000 7febfc129700 1 -- 172.21.15.5:0/1162468913 shutdown_connections 2024-05-09T00:44:39.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:39.920+0000 7febfc129700 1 -- 172.21.15.5:0/1162468913 wait complete. 2024-05-09T00:44:40.266 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:40 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:44:40.267 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:40 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/1162468913' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-09T00:44:40.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:40 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:44:40.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:40 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/1162468913' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-09T00:44:40.529 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:40 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:44:40.529 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:40 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/1162468913' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-09T00:44:40.608 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":91856593},{"type":"v1","addr":"172.21.15.5:6801","nonce":91856593}]},"active_addr":"172.21.15.5:6801/91856593","active_change":"2024-05-09T00:36:21.742491+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24106,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.5:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":1108120909}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":911031686}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":2452940029}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":3323633009}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":2886826262}]}]} 2024-05-09T00:44:40.615 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-09T00:44:40.615 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-09T00:44:40.616 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T00:44:41.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:41 smithi005 ceph-mon[30365]: pgmap v319: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.1 KiB/s wr, 11 op/s; 4.3 KiB/s, 0 objects/s recovering 2024-05-09T00:44:41.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:41 smithi157 ceph-mon[34294]: pgmap v319: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.1 KiB/s wr, 11 op/s; 4.3 KiB/s, 0 objects/s recovering 2024-05-09T00:44:41.504 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:44:41.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:41 smithi131 ceph-mon[34547]: pgmap v319: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.1 KiB/s wr, 11 op/s; 4.3 KiB/s, 0 objects/s recovering 2024-05-09T00:44:42.640 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:42 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:44:42.640 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:42 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:44:42.640 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:42 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:44:42.640 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:42 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:44:42.640 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:42 smithi005 ceph-mon[30365]: pgmap v320: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-09T00:44:42.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:42 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:44:42.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:42 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:44:42.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:42 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:44:42.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:42 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:44:42.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:42 smithi157 ceph-mon[34294]: pgmap v320: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-09T00:44:42.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:42 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:44:42.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:42 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:44:42.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:42 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:44:42.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:42 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:44:42.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:42 smithi131 ceph-mon[34547]: pgmap v320: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-09T00:44:43.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:43.233+0000 7fa201881700 1 -- 172.21.15.5:0/2457784733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1fc10e8d0 msgr2=0x7fa1fc110db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:43.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:43.233+0000 7fa201881700 1 --2- 172.21.15.5:0/2457784733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1fc10e8d0 0x7fa1fc110db0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa1f400b0a0 tx=0x7fa1f400b3b0 comp rx=0 tx=0).stop 2024-05-09T00:44:43.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:43.233+0000 7fa201881700 1 -- 172.21.15.5:0/2457784733 shutdown_connections 2024-05-09T00:44:43.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:43.233+0000 7fa201881700 1 --2- 172.21.15.5:0/2457784733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1fc10e8d0 0x7fa1fc110db0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:43.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:43.233+0000 7fa201881700 1 --2- 172.21.15.5:0/2457784733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1fc105460 0x7fa1fc105870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:43.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:43.233+0000 7fa201881700 1 --2- 172.21.15.5:0/2457784733 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1fc105db0 0x7fa1fc10e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:43.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:43.233+0000 7fa201881700 1 -- 172.21.15.5:0/2457784733 >> 172.21.15.5:0/2457784733 conn(0x7fa1fc0fedc0 msgr2=0x7fa1fc1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:44:43.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:43.233+0000 7fa201881700 1 -- 172.21.15.5:0/2457784733 shutdown_connections 2024-05-09T00:44:43.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:43.233+0000 7fa201881700 1 -- 172.21.15.5:0/2457784733 wait complete. 2024-05-09T00:44:43.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:43.234+0000 7fa201881700 1 Processor -- start 2024-05-09T00:44:43.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:43.235+0000 7fa201881700 1 -- start start 2024-05-09T00:44:43.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:43.235+0000 7fa201881700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1fc105460 0x7fa1fc19ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:44:43.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:43.235+0000 7fa201881700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1fc105db0 0x7fa1fc19b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:44:43.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:43.235+0000 7fa201881700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1fc10e8d0 0x7fa1fc19f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:44:43.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:43.235+0000 7fa201881700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa1fc114430 con 0x7fa1fc105460 2024-05-09T00:44:43.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:43.235+0000 7fa201881700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa1fc1142b0 con 0x7fa1fc105db0 2024-05-09T00:44:43.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:43.235+0000 7fa201881700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa1fc1145b0 con 0x7fa1fc10e8d0 2024-05-09T00:44:43.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:43.235+0000 7fa1faffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1fc105460 0x7fa1fc19ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:44:43.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:43.235+0000 7fa1faffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1fc105460 0x7fa1fc19ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44632/0 (socket says 172.21.15.5:44632) 2024-05-09T00:44:43.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:43.235+0000 7fa1faffd700 1 -- 172.21.15.5:0/3764195609 learned_addr learned my addr 172.21.15.5:0/3764195609 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:44:43.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:43.235+0000 7fa1fb7fe700 1 --2- 172.21.15.5:0/3764195609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1fc10e8d0 0x7fa1fc19f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:44:43.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:43.235+0000 7fa1fa7fc700 1 --2- 172.21.15.5:0/3764195609 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1fc105db0 0x7fa1fc19b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:44:43.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:43.236+0000 7fa1faffd700 1 -- 172.21.15.5:0/3764195609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1fc10e8d0 msgr2=0x7fa1fc19f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:43.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:43.236+0000 7fa1faffd700 1 --2- 172.21.15.5:0/3764195609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1fc10e8d0 0x7fa1fc19f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:43.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:43.236+0000 7fa1faffd700 1 -- 172.21.15.5:0/3764195609 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1fc105db0 msgr2=0x7fa1fc19b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:43.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:43.236+0000 7fa1faffd700 1 --2- 172.21.15.5:0/3764195609 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1fc105db0 0x7fa1fc19b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:43.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:43.236+0000 7fa1faffd700 1 -- 172.21.15.5:0/3764195609 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1fc19fc50 con 0x7fa1fc105460 2024-05-09T00:44:43.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:43.236+0000 7fa1faffd700 1 --2- 172.21.15.5:0/3764195609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1fc105460 0x7fa1fc19ad80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec00e9a0 tx=0x7fa1ec00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:44:43.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:43.236+0000 7fa20087f700 1 -- 172.21.15.5:0/3764195609 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec00ca10 con 0x7fa1fc105460 2024-05-09T00:44:43.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:43.237+0000 7fa201881700 1 -- 172.21.15.5:0/3764195609 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1fc19ff40 con 0x7fa1fc105460 2024-05-09T00:44:43.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:43.237+0000 7fa20087f700 1 -- 172.21.15.5:0/3764195609 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1ec00cbb0 con 0x7fa1fc105460 2024-05-09T00:44:43.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:43.237+0000 7fa20087f700 1 -- 172.21.15.5:0/3764195609 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec018680 con 0x7fa1fc105460 2024-05-09T00:44:43.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:43.237+0000 7fa201881700 1 -- 172.21.15.5:0/3764195609 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1fc103080 con 0x7fa1fc105460 2024-05-09T00:44:43.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:43.238+0000 7fa201881700 1 -- 172.21.15.5:0/3764195609 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1fc0512c0 con 0x7fa1fc105460 2024-05-09T00:44:43.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:43.241+0000 7fa20087f700 1 -- 172.21.15.5:0/3764195609 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fa1ec010430 con 0x7fa1fc105460 2024-05-09T00:44:43.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:43.241+0000 7fa20087f700 1 --2- 172.21.15.5:0/3764195609 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fa1e40b2990 0x7fa1e40b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:44:43.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:43.241+0000 7fa1fa7fc700 1 --2- 172.21.15.5:0/3764195609 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fa1e40b2990 0x7fa1e40b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:44:43.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:43.241+0000 7fa20087f700 1 -- 172.21.15.5:0/3764195609 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8227+0+0 (secure 0 0 0) 0x7fa1ec0e2840 con 0x7fa1fc105460 2024-05-09T00:44:43.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:43.242+0000 7fa1fa7fc700 1 --2- 172.21.15.5:0/3764195609 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fa1e40b2990 0x7fa1e40b4ea0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc1022b0 tx=0x7fa1f00073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:44:43.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:43.244+0000 7fa20087f700 1 -- 172.21.15.5:0/3764195609 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa1ec0aa400 con 0x7fa1fc105460 2024-05-09T00:44:43.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:43.485+0000 7fa201881700 1 -- 172.21.15.5:0/3764195609 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa1fc19c0e0 con 0x7fa1fc105460 2024-05-09T00:44:43.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:43.487+0000 7fa20087f700 1 -- 172.21.15.5:0/3764195609 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22594 (secure 0 0 0) 0x7fa1ec0a9d00 con 0x7fa1fc105460 2024-05-09T00:44:43.489 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:44:43.489 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":80,"fsid":"d85814c6-0d9b-11ef-bc97-c7b262605968","created":"2024-05-09T00:35:25.623377+0000","modified":"2024-05-09T00:44:23.011279+0000","last_up_change":"2024-05-09T00:42:38.525335+0000","last_in_change":"2024-05-09T00:42:17.990219+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T00:38:47.844734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T00:43:39.452106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T00:43:42.708106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f8d484bb-2477-4377-ae68-5bce571333bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2963385132},{"type":"v1","addr":"172.21.15.5:6803","nonce":2963385132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2963385132},{"type":"v1","addr":"172.21.15.5:6805","nonce":2963385132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2963385132},{"type":"v1","addr":"172.21.15.5:6809","nonce":2963385132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2963385132},{"type":"v1","addr":"172.21.15.5:6807","nonce":2963385132}]},"public_addr":"172.21.15.5:6803/2963385132","cluster_addr":"172.21.15.5:6805/2963385132","heartbeat_back_addr":"172.21.15.5:6809/2963385132","heartbeat_front_addr":"172.21.15.5:6807/2963385132","state":["exists","up"]},{"osd":1,"uuid":"1c123676-7144-4a67-93a6-fd49e36a5c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3837683163},{"type":"v1","addr":"172.21.15.131:6801","nonce":3837683163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3837683163},{"type":"v1","addr":"172.21.15.131:6803","nonce":3837683163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3837683163},{"type":"v1","addr":"172.21.15.131:6807","nonce":3837683163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3837683163},{"type":"v1","addr":"172.21.15.131:6805","nonce":3837683163}]},"public_addr":"172.21.15.131:6801/3837683163","cluster_addr":"172.21.15.131:6803/3837683163","heartbeat_back_addr":"172.21.15.131:6807/3837683163","heartbeat_front_addr":"172.21.15.131:6805/3837683163","state":["exists","up"]},{"osd":2,"uuid":"95ca51f0-b48b-485c-a5f7-b15c01e754b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":484818139},{"type":"v1","addr":"172.21.15.157:6801","nonce":484818139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":484818139},{"type":"v1","addr":"172.21.15.157:6803","nonce":484818139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":484818139},{"type":"v1","addr":"172.21.15.157:6807","nonce":484818139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":484818139},{"type":"v1","addr":"172.21.15.157:6805","nonce":484818139}]},"public_addr":"172.21.15.157:6801/484818139","cluster_addr":"172.21.15.157:6803/484818139","heartbeat_back_addr":"172.21.15.157:6807/484818139","heartbeat_front_addr":"172.21.15.157:6805/484818139","state":["exists","up"]},{"osd":3,"uuid":"ed7577fb-3676-4a08-b40d-5474eb88974f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":2357377277},{"type":"v1","addr":"172.21.15.5:6811","nonce":2357377277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":2357377277},{"type":"v1","addr":"172.21.15.5:6813","nonce":2357377277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":2357377277},{"type":"v1","addr":"172.21.15.5:6817","nonce":2357377277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":2357377277},{"type":"v1","addr":"172.21.15.5:6815","nonce":2357377277}]},"public_addr":"172.21.15.5:6811/2357377277","cluster_addr":"172.21.15.5:6813/2357377277","heartbeat_back_addr":"172.21.15.5:6817/2357377277","heartbeat_front_addr":"172.21.15.5:6815/2357377277","state":["exists","up"]},{"osd":4,"uuid":"fb5dcf79-a6f1-4b1a-9e3a-087fb5132472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":2915418378},{"type":"v1","addr":"172.21.15.131:6809","nonce":2915418378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":2915418378},{"type":"v1","addr":"172.21.15.131:6811","nonce":2915418378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":2915418378},{"type":"v1","addr":"172.21.15.131:6815","nonce":2915418378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":2915418378},{"type":"v1","addr":"172.21.15.131:6813","nonce":2915418378}]},"public_addr":"172.21.15.131:6809/2915418378","cluster_addr":"172.21.15.131:6811/2915418378","heartbeat_back_addr":"172.21.15.131:6815/2915418378","heartbeat_front_addr":"172.21.15.131:6813/2915418378","state":["exists","up"]},{"osd":5,"uuid":"f01ed5d3-8066-4d25-a113-280c83e75ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4188818663},{"type":"v1","addr":"172.21.15.157:6809","nonce":4188818663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":4188818663},{"type":"v1","addr":"172.21.15.157:6811","nonce":4188818663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":4188818663},{"type":"v1","addr":"172.21.15.157:6815","nonce":4188818663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":4188818663},{"type":"v1","addr":"172.21.15.157:6813","nonce":4188818663}]},"public_addr":"172.21.15.157:6809/4188818663","cluster_addr":"172.21.15.157:6811/4188818663","heartbeat_back_addr":"172.21.15.157:6815/4188818663","heartbeat_front_addr":"172.21.15.157:6813/4188818663","state":["exists","up"]},{"osd":6,"uuid":"932379a8-c97b-4acc-8665-81f7d9cbce65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2930397813},{"type":"v1","addr":"172.21.15.5:6819","nonce":2930397813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2930397813},{"type":"v1","addr":"172.21.15.5:6821","nonce":2930397813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2930397813},{"type":"v1","addr":"172.21.15.5:6825","nonce":2930397813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2930397813},{"type":"v1","addr":"172.21.15.5:6823","nonce":2930397813}]},"public_addr":"172.21.15.5:6819/2930397813","cluster_addr":"172.21.15.5:6821/2930397813","heartbeat_back_addr":"172.21.15.5:6825/2930397813","heartbeat_front_addr":"172.21.15.5:6823/2930397813","state":["exists","up"]},{"osd":7,"uuid":"3c0a0573-6465-4bf6-b70f-310d49857a6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":4125045219},{"type":"v1","addr":"172.21.15.131:6817","nonce":4125045219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":4125045219},{"type":"v1","addr":"172.21.15.131:6819","nonce":4125045219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":4125045219},{"type":"v1","addr":"172.21.15.131:6823","nonce":4125045219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":4125045219},{"type":"v1","addr":"172.21.15.131:6821","nonce":4125045219}]},"public_addr":"172.21.15.131:6817/4125045219","cluster_addr":"172.21.15.131:6819/4125045219","heartbeat_back_addr":"172.21.15.131:6823/4125045219","heartbeat_front_addr":"172.21.15.131:6821/4125045219","state":["exists","up"]},{"osd":8,"uuid":"c49f0a00-f5ae-40ed-8f38-7b953ba1c17c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2001626768},{"type":"v1","addr":"172.21.15.157:6817","nonce":2001626768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2001626768},{"type":"v1","addr":"172.21.15.157:6819","nonce":2001626768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2001626768},{"type":"v1","addr":"172.21.15.157:6823","nonce":2001626768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2001626768},{"type":"v1","addr":"172.21.15.157:6821","nonce":2001626768}]},"public_addr":"172.21.15.157:6817/2001626768","cluster_addr":"172.21.15.157:6819/2001626768","heartbeat_back_addr":"172.21.15.157:6823/2001626768","heartbeat_front_addr":"172.21.15.157:6821/2001626768","state":["exists","up"]},{"osd":9,"uuid":"1927dfe7-c1af-4a15-bb71-d58d97c8734a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":4287220015},{"type":"v1","addr":"172.21.15.5:6827","nonce":4287220015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":4287220015},{"type":"v1","addr":"172.21.15.5:6829","nonce":4287220015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":4287220015},{"type":"v1","addr":"172.21.15.5:6833","nonce":4287220015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":4287220015},{"type":"v1","addr":"172.21.15.5:6831","nonce":4287220015}]},"public_addr":"172.21.15.5:6827/4287220015","cluster_addr":"172.21.15.5:6829/4287220015","heartbeat_back_addr":"172.21.15.5:6833/4287220015","heartbeat_front_addr":"172.21.15.5:6831/4287220015","state":["exists","up"]},{"osd":10,"uuid":"fbb4f3e3-eb7d-4ecb-a30b-55b8910375ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1571783985},{"type":"v1","addr":"172.21.15.131:6825","nonce":1571783985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1571783985},{"type":"v1","addr":"172.21.15.131:6827","nonce":1571783985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1571783985},{"type":"v1","addr":"172.21.15.131:6831","nonce":1571783985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1571783985},{"type":"v1","addr":"172.21.15.131:6829","nonce":1571783985}]},"public_addr":"172.21.15.131:6825/1571783985","cluster_addr":"172.21.15.131:6827/1571783985","heartbeat_back_addr":"172.21.15.131:6831/1571783985","heartbeat_front_addr":"172.21.15.131:6829/1571783985","state":["exists","up"]},{"osd":11,"uuid":"b858453f-eda1-4e25-89c8-94b54c830938","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3261151188},{"type":"v1","addr":"172.21.15.157:6825","nonce":3261151188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":3261151188},{"type":"v1","addr":"172.21.15.157:6827","nonce":3261151188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":3261151188},{"type":"v1","addr":"172.21.15.157:6831","nonce":3261151188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":3261151188},{"type":"v1","addr":"172.21.15.157:6829","nonce":3261151188}]},"public_addr":"172.21.15.157:6825/3261151188","cluster_addr":"172.21.15.157:6827/3261151188","heartbeat_back_addr":"172.21.15.157:6831/3261151188","heartbeat_front_addr":"172.21.15.157:6829/3261151188","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:37:57.528483+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:38:21.477482+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:38:45.780266+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:39:10.938791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:39:35.734857+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:40:00.169040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:40:26.494635+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:40:51.881718+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:41:17.697112+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:41:44.826479+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:42:10.504768+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:42:36.605447+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6},{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":6}]},{"pgid":"3.19","mappings":[{"from":3,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":7}]},{"pgid":"3.35","mappings":[{"from":1,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/3405015786":"2024-05-10T00:36:21.742363+0000","172.21.15.5:0/2669115557":"2024-05-10T00:36:21.742363+0000","172.21.15.5:6801/287842266":"2024-05-10T00:36:21.742363+0000","172.21.15.5:6800/752513054":"2024-05-10T00:35:53.128955+0000","172.21.15.5:6800/287842266":"2024-05-10T00:36:21.742363+0000","172.21.15.5:6801/752513054":"2024-05-10T00:35:53.128955+0000","172.21.15.5:0/1954876662":"2024-05-10T00:35:53.128955+0000","172.21.15.5:0/2545175121":"2024-05-10T00:36:21.742363+0000","172.21.15.5:0/3754653663":"2024-05-10T00:35:53.128955+0000","172.21.15.5:0/651514359":"2024-05-10T00:35:53.128955+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T00:44:43.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:43.490+0000 7fa201881700 1 -- 172.21.15.5:0/3764195609 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fa1e40b2990 msgr2=0x7fa1e40b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:43.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:43.490+0000 7fa201881700 1 --2- 172.21.15.5:0/3764195609 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fa1e40b2990 0x7fa1e40b4ea0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc1022b0 tx=0x7fa1f00073d0 comp rx=0 tx=0).stop 2024-05-09T00:44:43.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:43.490+0000 7fa201881700 1 -- 172.21.15.5:0/3764195609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1fc105460 msgr2=0x7fa1fc19ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:43.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:43.490+0000 7fa201881700 1 --2- 172.21.15.5:0/3764195609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1fc105460 0x7fa1fc19ad80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec00e9a0 tx=0x7fa1ec00ecb0 comp rx=0 tx=0).stop 2024-05-09T00:44:43.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:43.490+0000 7fa201881700 1 -- 172.21.15.5:0/3764195609 shutdown_connections 2024-05-09T00:44:43.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:43.490+0000 7fa201881700 1 --2- 172.21.15.5:0/3764195609 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fa1e40b2990 0x7fa1e40b4ea0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:43.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:43.490+0000 7fa201881700 1 --2- 172.21.15.5:0/3764195609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1fc10e8d0 0x7fa1fc19f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:43.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:43.490+0000 7fa201881700 1 --2- 172.21.15.5:0/3764195609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1fc105460 0x7fa1fc19ad80 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:43.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:43.490+0000 7fa201881700 1 --2- 172.21.15.5:0/3764195609 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1fc105db0 0x7fa1fc19b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:43.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:43.490+0000 7fa201881700 1 -- 172.21.15.5:0/3764195609 >> 172.21.15.5:0/3764195609 conn(0x7fa1fc0fedc0 msgr2=0x7fa1fc0ff450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:44:43.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:43.490+0000 7fa201881700 1 -- 172.21.15.5:0/3764195609 shutdown_connections 2024-05-09T00:44:43.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:43.490+0000 7fa201881700 1 -- 172.21.15.5:0/3764195609 wait complete. 2024-05-09T00:44:44.205 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-09T00:44:44.205 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T00:44:44.466 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:44:44.711 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:44 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:44:44.711 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:44 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:44:44.711 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:44 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/3764195609' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:44:44.711 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:44 smithi005 ceph-mon[30365]: pgmap v321: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-09T00:44:44.712 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:44 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:44:44.712 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:44 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:44:44.712 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:44 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:44:44.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:44 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:44:44.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:44 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:44:44.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:44 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/3764195609' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:44:44.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:44 smithi131 ceph-mon[34547]: pgmap v321: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-09T00:44:44.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:44 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:44:44.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:44 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:44:44.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:44 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:44:44.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:44 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:44:44.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:44 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:44:44.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:44 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/3764195609' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:44:44.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:44 smithi157 ceph-mon[34294]: pgmap v321: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-09T00:44:44.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:44 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:44:44.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:44 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:44:44.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:44 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:44:46.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:46.206+0000 7f630fe43700 1 -- 172.21.15.5:0/3759382854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6308108740 msgr2=0x7f63081100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:46.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:46.206+0000 7f630fe43700 1 --2- 172.21.15.5:0/3759382854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6308108740 0x7f63081100a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f63000099c0 tx=0x7f6300009cd0 comp rx=0 tx=0).stop 2024-05-09T00:44:46.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:46.206+0000 7f630fe43700 1 -- 172.21.15.5:0/3759382854 shutdown_connections 2024-05-09T00:44:46.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:46.206+0000 7f630fe43700 1 --2- 172.21.15.5:0/3759382854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6308108740 0x7f63081100a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:46.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:46.206+0000 7f630fe43700 1 --2- 172.21.15.5:0/3759382854 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63081034c0 0x7f63081038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:46.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:46.206+0000 7f630fe43700 1 --2- 172.21.15.5:0/3759382854 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6308107df0 0x7f6308108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:46.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:46.206+0000 7f630fe43700 1 -- 172.21.15.5:0/3759382854 >> 172.21.15.5:0/3759382854 conn(0x7f63080fedf0 msgr2=0x7f6308101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:44:46.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:46.206+0000 7f630fe43700 1 -- 172.21.15.5:0/3759382854 shutdown_connections 2024-05-09T00:44:46.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:46.206+0000 7f630fe43700 1 -- 172.21.15.5:0/3759382854 wait complete. 2024-05-09T00:44:46.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:46.206+0000 7f630fe43700 1 Processor -- start 2024-05-09T00:44:46.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:46.207+0000 7f630fe43700 1 -- start start 2024-05-09T00:44:46.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:46.207+0000 7f630fe43700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63081034c0 0x7f630819ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:44:46.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:46.207+0000 7f630fe43700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6308107df0 0x7f630819b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:44:46.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:46.207+0000 7f630fe43700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6308108740 0x7f630819f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:44:46.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:46.207+0000 7f630fe43700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6308113370 con 0x7f6308107df0 2024-05-09T00:44:46.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:46.207+0000 7f630fe43700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f63081131f0 con 0x7f6308108740 2024-05-09T00:44:46.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:46.207+0000 7f630fe43700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f63081134f0 con 0x7f63081034c0 2024-05-09T00:44:46.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:46.207+0000 7f630d3de700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6308107df0 0x7f630819b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:44:46.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:46.207+0000 7f630d3de700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6308107df0 0x7f630819b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39864/0 (socket says 172.21.15.5:39864) 2024-05-09T00:44:46.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:46.207+0000 7f630d3de700 1 -- 172.21.15.5:0/2972842352 learned_addr learned my addr 172.21.15.5:0/2972842352 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:44:46.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:46.207+0000 7f630dbdf700 1 --2- 172.21.15.5:0/2972842352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63081034c0 0x7f630819ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:44:46.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:46.207+0000 7f630e3e0700 1 --2- 172.21.15.5:0/2972842352 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6308108740 0x7f630819f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:44:46.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:46.207+0000 7f630d3de700 1 -- 172.21.15.5:0/2972842352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63081034c0 msgr2=0x7f630819ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:46.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:46.207+0000 7f630d3de700 1 --2- 172.21.15.5:0/2972842352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63081034c0 0x7f630819ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:46.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:46.207+0000 7f630d3de700 1 -- 172.21.15.5:0/2972842352 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6308108740 msgr2=0x7f630819f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:46.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:46.207+0000 7f630d3de700 1 --2- 172.21.15.5:0/2972842352 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6308108740 0x7f630819f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:46.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:46.207+0000 7f630d3de700 1 -- 172.21.15.5:0/2972842352 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f630819fc60 con 0x7f6308107df0 2024-05-09T00:44:46.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:46.208+0000 7f630d3de700 1 --2- 172.21.15.5:0/2972842352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6308107df0 0x7f630819b330 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f62f800d980 tx=0x7f62f800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:44:46.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:46.208+0000 7f62feffd700 1 -- 172.21.15.5:0/2972842352 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f8017070 con 0x7f6308107df0 2024-05-09T00:44:46.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:46.208+0000 7f630fe43700 1 -- 172.21.15.5:0/2972842352 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f630819ff50 con 0x7f6308107df0 2024-05-09T00:44:46.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:46.208+0000 7f62feffd700 1 -- 172.21.15.5:0/2972842352 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62f8009de0 con 0x7f6308107df0 2024-05-09T00:44:46.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:46.208+0000 7f62feffd700 1 -- 172.21.15.5:0/2972842352 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f800f3c0 con 0x7f6308107df0 2024-05-09T00:44:46.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:46.208+0000 7f630fe43700 1 -- 172.21.15.5:0/2972842352 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63081a7870 con 0x7f6308107df0 2024-05-09T00:44:46.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:46.209+0000 7f630fe43700 1 -- 172.21.15.5:0/2972842352 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63080512c0 con 0x7f6308107df0 2024-05-09T00:44:46.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:46.211+0000 7f62feffd700 1 -- 172.21.15.5:0/2972842352 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f62f8009930 con 0x7f6308107df0 2024-05-09T00:44:46.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:46.211+0000 7f62feffd700 1 --2- 172.21.15.5:0/2972842352 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f62f40b2910 0x7f62f40b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:44:46.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:46.211+0000 7f630dbdf700 1 --2- 172.21.15.5:0/2972842352 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f62f40b2910 0x7f62f40b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:44:46.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:46.211+0000 7f62feffd700 1 -- 172.21.15.5:0/2972842352 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8227+0+0 (secure 0 0 0) 0x7f62f80e1de0 con 0x7f6308107df0 2024-05-09T00:44:46.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:46.212+0000 7f630dbdf700 1 --2- 172.21.15.5:0/2972842352 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f62f40b2910 0x7f62f40b4e20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6304009550 tx=0x7f63040091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:44:46.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:46.215+0000 7f62feffd700 1 -- 172.21.15.5:0/2972842352 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f62f80a99a0 con 0x7f6308107df0 2024-05-09T00:44:46.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:46.431+0000 7f630fe43700 1 -- 172.21.15.5:0/2972842352 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f630802e5f0 con 0x7f6308107df0 2024-05-09T00:44:46.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:46.433+0000 7f62feffd700 1 -- 172.21.15.5:0/2972842352 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22594 (secure 0 0 0) 0x7f62f80a92a0 con 0x7f6308107df0 2024-05-09T00:44:46.434 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:44:46.435 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":80,"fsid":"d85814c6-0d9b-11ef-bc97-c7b262605968","created":"2024-05-09T00:35:25.623377+0000","modified":"2024-05-09T00:44:23.011279+0000","last_up_change":"2024-05-09T00:42:38.525335+0000","last_in_change":"2024-05-09T00:42:17.990219+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T00:38:47.844734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T00:43:39.452106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T00:43:42.708106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f8d484bb-2477-4377-ae68-5bce571333bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2963385132},{"type":"v1","addr":"172.21.15.5:6803","nonce":2963385132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2963385132},{"type":"v1","addr":"172.21.15.5:6805","nonce":2963385132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2963385132},{"type":"v1","addr":"172.21.15.5:6809","nonce":2963385132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2963385132},{"type":"v1","addr":"172.21.15.5:6807","nonce":2963385132}]},"public_addr":"172.21.15.5:6803/2963385132","cluster_addr":"172.21.15.5:6805/2963385132","heartbeat_back_addr":"172.21.15.5:6809/2963385132","heartbeat_front_addr":"172.21.15.5:6807/2963385132","state":["exists","up"]},{"osd":1,"uuid":"1c123676-7144-4a67-93a6-fd49e36a5c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3837683163},{"type":"v1","addr":"172.21.15.131:6801","nonce":3837683163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3837683163},{"type":"v1","addr":"172.21.15.131:6803","nonce":3837683163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3837683163},{"type":"v1","addr":"172.21.15.131:6807","nonce":3837683163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3837683163},{"type":"v1","addr":"172.21.15.131:6805","nonce":3837683163}]},"public_addr":"172.21.15.131:6801/3837683163","cluster_addr":"172.21.15.131:6803/3837683163","heartbeat_back_addr":"172.21.15.131:6807/3837683163","heartbeat_front_addr":"172.21.15.131:6805/3837683163","state":["exists","up"]},{"osd":2,"uuid":"95ca51f0-b48b-485c-a5f7-b15c01e754b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":484818139},{"type":"v1","addr":"172.21.15.157:6801","nonce":484818139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":484818139},{"type":"v1","addr":"172.21.15.157:6803","nonce":484818139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":484818139},{"type":"v1","addr":"172.21.15.157:6807","nonce":484818139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":484818139},{"type":"v1","addr":"172.21.15.157:6805","nonce":484818139}]},"public_addr":"172.21.15.157:6801/484818139","cluster_addr":"172.21.15.157:6803/484818139","heartbeat_back_addr":"172.21.15.157:6807/484818139","heartbeat_front_addr":"172.21.15.157:6805/484818139","state":["exists","up"]},{"osd":3,"uuid":"ed7577fb-3676-4a08-b40d-5474eb88974f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":2357377277},{"type":"v1","addr":"172.21.15.5:6811","nonce":2357377277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":2357377277},{"type":"v1","addr":"172.21.15.5:6813","nonce":2357377277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":2357377277},{"type":"v1","addr":"172.21.15.5:6817","nonce":2357377277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":2357377277},{"type":"v1","addr":"172.21.15.5:6815","nonce":2357377277}]},"public_addr":"172.21.15.5:6811/2357377277","cluster_addr":"172.21.15.5:6813/2357377277","heartbeat_back_addr":"172.21.15.5:6817/2357377277","heartbeat_front_addr":"172.21.15.5:6815/2357377277","state":["exists","up"]},{"osd":4,"uuid":"fb5dcf79-a6f1-4b1a-9e3a-087fb5132472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":2915418378},{"type":"v1","addr":"172.21.15.131:6809","nonce":2915418378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":2915418378},{"type":"v1","addr":"172.21.15.131:6811","nonce":2915418378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":2915418378},{"type":"v1","addr":"172.21.15.131:6815","nonce":2915418378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":2915418378},{"type":"v1","addr":"172.21.15.131:6813","nonce":2915418378}]},"public_addr":"172.21.15.131:6809/2915418378","cluster_addr":"172.21.15.131:6811/2915418378","heartbeat_back_addr":"172.21.15.131:6815/2915418378","heartbeat_front_addr":"172.21.15.131:6813/2915418378","state":["exists","up"]},{"osd":5,"uuid":"f01ed5d3-8066-4d25-a113-280c83e75ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4188818663},{"type":"v1","addr":"172.21.15.157:6809","nonce":4188818663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":4188818663},{"type":"v1","addr":"172.21.15.157:6811","nonce":4188818663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":4188818663},{"type":"v1","addr":"172.21.15.157:6815","nonce":4188818663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":4188818663},{"type":"v1","addr":"172.21.15.157:6813","nonce":4188818663}]},"public_addr":"172.21.15.157:6809/4188818663","cluster_addr":"172.21.15.157:6811/4188818663","heartbeat_back_addr":"172.21.15.157:6815/4188818663","heartbeat_front_addr":"172.21.15.157:6813/4188818663","state":["exists","up"]},{"osd":6,"uuid":"932379a8-c97b-4acc-8665-81f7d9cbce65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2930397813},{"type":"v1","addr":"172.21.15.5:6819","nonce":2930397813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2930397813},{"type":"v1","addr":"172.21.15.5:6821","nonce":2930397813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2930397813},{"type":"v1","addr":"172.21.15.5:6825","nonce":2930397813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2930397813},{"type":"v1","addr":"172.21.15.5:6823","nonce":2930397813}]},"public_addr":"172.21.15.5:6819/2930397813","cluster_addr":"172.21.15.5:6821/2930397813","heartbeat_back_addr":"172.21.15.5:6825/2930397813","heartbeat_front_addr":"172.21.15.5:6823/2930397813","state":["exists","up"]},{"osd":7,"uuid":"3c0a0573-6465-4bf6-b70f-310d49857a6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":4125045219},{"type":"v1","addr":"172.21.15.131:6817","nonce":4125045219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":4125045219},{"type":"v1","addr":"172.21.15.131:6819","nonce":4125045219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":4125045219},{"type":"v1","addr":"172.21.15.131:6823","nonce":4125045219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":4125045219},{"type":"v1","addr":"172.21.15.131:6821","nonce":4125045219}]},"public_addr":"172.21.15.131:6817/4125045219","cluster_addr":"172.21.15.131:6819/4125045219","heartbeat_back_addr":"172.21.15.131:6823/4125045219","heartbeat_front_addr":"172.21.15.131:6821/4125045219","state":["exists","up"]},{"osd":8,"uuid":"c49f0a00-f5ae-40ed-8f38-7b953ba1c17c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2001626768},{"type":"v1","addr":"172.21.15.157:6817","nonce":2001626768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2001626768},{"type":"v1","addr":"172.21.15.157:6819","nonce":2001626768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2001626768},{"type":"v1","addr":"172.21.15.157:6823","nonce":2001626768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2001626768},{"type":"v1","addr":"172.21.15.157:6821","nonce":2001626768}]},"public_addr":"172.21.15.157:6817/2001626768","cluster_addr":"172.21.15.157:6819/2001626768","heartbeat_back_addr":"172.21.15.157:6823/2001626768","heartbeat_front_addr":"172.21.15.157:6821/2001626768","state":["exists","up"]},{"osd":9,"uuid":"1927dfe7-c1af-4a15-bb71-d58d97c8734a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":4287220015},{"type":"v1","addr":"172.21.15.5:6827","nonce":4287220015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":4287220015},{"type":"v1","addr":"172.21.15.5:6829","nonce":4287220015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":4287220015},{"type":"v1","addr":"172.21.15.5:6833","nonce":4287220015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":4287220015},{"type":"v1","addr":"172.21.15.5:6831","nonce":4287220015}]},"public_addr":"172.21.15.5:6827/4287220015","cluster_addr":"172.21.15.5:6829/4287220015","heartbeat_back_addr":"172.21.15.5:6833/4287220015","heartbeat_front_addr":"172.21.15.5:6831/4287220015","state":["exists","up"]},{"osd":10,"uuid":"fbb4f3e3-eb7d-4ecb-a30b-55b8910375ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1571783985},{"type":"v1","addr":"172.21.15.131:6825","nonce":1571783985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1571783985},{"type":"v1","addr":"172.21.15.131:6827","nonce":1571783985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1571783985},{"type":"v1","addr":"172.21.15.131:6831","nonce":1571783985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1571783985},{"type":"v1","addr":"172.21.15.131:6829","nonce":1571783985}]},"public_addr":"172.21.15.131:6825/1571783985","cluster_addr":"172.21.15.131:6827/1571783985","heartbeat_back_addr":"172.21.15.131:6831/1571783985","heartbeat_front_addr":"172.21.15.131:6829/1571783985","state":["exists","up"]},{"osd":11,"uuid":"b858453f-eda1-4e25-89c8-94b54c830938","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3261151188},{"type":"v1","addr":"172.21.15.157:6825","nonce":3261151188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":3261151188},{"type":"v1","addr":"172.21.15.157:6827","nonce":3261151188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":3261151188},{"type":"v1","addr":"172.21.15.157:6831","nonce":3261151188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":3261151188},{"type":"v1","addr":"172.21.15.157:6829","nonce":3261151188}]},"public_addr":"172.21.15.157:6825/3261151188","cluster_addr":"172.21.15.157:6827/3261151188","heartbeat_back_addr":"172.21.15.157:6831/3261151188","heartbeat_front_addr":"172.21.15.157:6829/3261151188","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:37:57.528483+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:38:21.477482+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:38:45.780266+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:39:10.938791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:39:35.734857+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:40:00.169040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:40:26.494635+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:40:51.881718+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:41:17.697112+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:41:44.826479+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:42:10.504768+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:42:36.605447+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6},{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":6}]},{"pgid":"3.19","mappings":[{"from":3,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":7}]},{"pgid":"3.35","mappings":[{"from":1,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/3405015786":"2024-05-10T00:36:21.742363+0000","172.21.15.5:0/2669115557":"2024-05-10T00:36:21.742363+0000","172.21.15.5:6801/287842266":"2024-05-10T00:36:21.742363+0000","172.21.15.5:6800/752513054":"2024-05-10T00:35:53.128955+0000","172.21.15.5:6800/287842266":"2024-05-10T00:36:21.742363+0000","172.21.15.5:6801/752513054":"2024-05-10T00:35:53.128955+0000","172.21.15.5:0/1954876662":"2024-05-10T00:35:53.128955+0000","172.21.15.5:0/2545175121":"2024-05-10T00:36:21.742363+0000","172.21.15.5:0/3754653663":"2024-05-10T00:35:53.128955+0000","172.21.15.5:0/651514359":"2024-05-10T00:35:53.128955+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T00:44:46.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:46.435+0000 7f630fe43700 1 -- 172.21.15.5:0/2972842352 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f62f40b2910 msgr2=0x7f62f40b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:46.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:46.435+0000 7f630fe43700 1 --2- 172.21.15.5:0/2972842352 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f62f40b2910 0x7f62f40b4e20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6304009550 tx=0x7f63040091b0 comp rx=0 tx=0).stop 2024-05-09T00:44:46.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:46.435+0000 7f630fe43700 1 -- 172.21.15.5:0/2972842352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6308107df0 msgr2=0x7f630819b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:44:46.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:46.435+0000 7f630fe43700 1 --2- 172.21.15.5:0/2972842352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6308107df0 0x7f630819b330 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f62f800d980 tx=0x7f62f800dc90 comp rx=0 tx=0).stop 2024-05-09T00:44:46.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:46.435+0000 7f630fe43700 1 -- 172.21.15.5:0/2972842352 shutdown_connections 2024-05-09T00:44:46.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:46.435+0000 7f630fe43700 1 --2- 172.21.15.5:0/2972842352 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f62f40b2910 0x7f62f40b4e20 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:46.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:46.435+0000 7f630fe43700 1 --2- 172.21.15.5:0/2972842352 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6308108740 0x7f630819f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:46.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:46.435+0000 7f630fe43700 1 --2- 172.21.15.5:0/2972842352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63081034c0 0x7f630819ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:46.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:46.436+0000 7f630fe43700 1 --2- 172.21.15.5:0/2972842352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6308107df0 0x7f630819b330 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:44:46.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:46.436+0000 7f630fe43700 1 -- 172.21.15.5:0/2972842352 >> 172.21.15.5:0/2972842352 conn(0x7f63080fedf0 msgr2=0x7f6308075cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:44:46.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:46.436+0000 7f630fe43700 1 -- 172.21.15.5:0/2972842352 shutdown_connections 2024-05-09T00:44:46.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:44:46.436+0000 7f630fe43700 1 -- 172.21.15.5:0/2972842352 wait complete. 2024-05-09T00:44:47.095 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:46 smithi005 ceph-mon[30365]: pgmap v322: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T00:44:47.096 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:46 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/2972842352' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:44:47.128 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-09T00:44:47.129 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-09T00:44:47.129 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-09T00:44:47.130 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-09T00:44:47.130 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-09T00:44:47.131 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-09T00:44:47.131 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-05-09T00:44:47.131 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-05-09T00:44:47.132 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-05-09T00:44:47.132 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-05-09T00:44:47.133 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-05-09T00:44:47.133 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-05-09T00:44:47.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:46 smithi157 ceph-mon[34294]: pgmap v322: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T00:44:47.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:46 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/2972842352' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:44:47.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:46 smithi131 ceph-mon[34547]: pgmap v322: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T00:44:47.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:46 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/2972842352' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:44:47.989 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:44:48.005 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:44:48.005 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:44:48.006 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:44:48.006 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:44:48.009 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:44:48.014 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:44:48.014 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:44:48.037 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:44:48.040 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:44:48.047 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:44:48.077 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:44:49.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:48 smithi005 ceph-mon[30365]: pgmap v323: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T00:44:49.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:48 smithi157 ceph-mon[34294]: pgmap v323: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T00:44:49.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:48 smithi131 ceph-mon[34547]: pgmap v323: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T00:44:51.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:50 smithi157 ceph-mon[34294]: pgmap v324: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-09T00:44:51.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:50 smithi005 ceph-mon[30365]: pgmap v324: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-09T00:44:51.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:50 smithi131 ceph-mon[34547]: pgmap v324: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-09T00:44:53.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:52 smithi157 ceph-mon[34294]: pgmap v325: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T00:44:53.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:52 smithi005 ceph-mon[30365]: pgmap v325: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T00:44:53.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:52 smithi131 ceph-mon[34547]: pgmap v325: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T00:44:55.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:54 smithi157 ceph-mon[34294]: pgmap v326: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-09T00:44:55.264 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:54 smithi005 ceph-mon[30365]: pgmap v326: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-09T00:44:55.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:54 smithi131 ceph-mon[34547]: pgmap v326: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-09T00:44:57.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:56 smithi005 ceph-mon[30365]: pgmap v327: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T00:44:57.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:56 smithi157 ceph-mon[34294]: pgmap v327: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T00:44:57.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:56 smithi131 ceph-mon[34547]: pgmap v327: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T00:44:58.984 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:44:58 smithi005 ceph-mon[30365]: pgmap v328: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T00:44:59.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:44:58 smithi157 ceph-mon[34294]: pgmap v328: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T00:44:59.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:44:58 smithi131 ceph-mon[34547]: pgmap v328: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T00:45:00.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:00.853+0000 7fa367ced700 1 -- 172.21.15.5:0/2201437876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa360108760 msgr2=0x7fa3601100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:00.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:00.853+0000 7fa367ced700 1 --2- 172.21.15.5:0/2201437876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa360108760 0x7fa3601100c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa35800b0d0 tx=0x7fa35800b3e0 comp rx=0 tx=0).stop 2024-05-09T00:45:00.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:00.853+0000 7fa367ced700 1 -- 172.21.15.5:0/2201437876 shutdown_connections 2024-05-09T00:45:00.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:00.853+0000 7fa367ced700 1 --2- 172.21.15.5:0/2201437876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa360108760 0x7fa3601100c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:00.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:00.853+0000 7fa367ced700 1 --2- 172.21.15.5:0/2201437876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3601034e0 0x7fa3601038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:00.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:00.853+0000 7fa367ced700 1 --2- 172.21.15.5:0/2201437876 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa360107e10 0x7fa360108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:00.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:00.853+0000 7fa367ced700 1 -- 172.21.15.5:0/2201437876 >> 172.21.15.5:0/2201437876 conn(0x7fa3600fef30 msgr2=0x7fa360101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:45:00.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:00.853+0000 7fa367ced700 1 -- 172.21.15.5:0/2201437876 shutdown_connections 2024-05-09T00:45:00.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:00.853+0000 7fa367ced700 1 -- 172.21.15.5:0/2201437876 wait complete. 2024-05-09T00:45:00.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:00.854+0000 7fa367ced700 1 Processor -- start 2024-05-09T00:45:00.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:00.854+0000 7fa367ced700 1 -- start start 2024-05-09T00:45:00.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:00.854+0000 7fa367ced700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3601034e0 0x7fa360199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:00.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:00.854+0000 7fa367ced700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa360107e10 0x7fa36019a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:00.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:00.854+0000 7fa367ced700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa360108760 0x7fa36019e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:00.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:00.854+0000 7fa367ced700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa3601133e0 con 0x7fa360107e10 2024-05-09T00:45:00.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:00.854+0000 7fa367ced700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa360113260 con 0x7fa3601034e0 2024-05-09T00:45:00.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:00.854+0000 7fa367ced700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa360113560 con 0x7fa360108760 2024-05-09T00:45:00.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:00.854+0000 7fa365288700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa360107e10 0x7fa36019a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:00.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:00.854+0000 7fa365a89700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3601034e0 0x7fa360199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:00.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:00.854+0000 7fa365a89700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3601034e0 0x7fa360199d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:46182/0 (socket says 172.21.15.5:46182) 2024-05-09T00:45:00.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:00.854+0000 7fa365a89700 1 -- 172.21.15.5:0/19016172 learned_addr learned my addr 172.21.15.5:0/19016172 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:45:00.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:00.854+0000 7fa36628a700 1 --2- 172.21.15.5:0/19016172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa360108760 0x7fa36019e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:00.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:00.855+0000 7fa365a89700 1 -- 172.21.15.5:0/19016172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa360108760 msgr2=0x7fa36019e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:00.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:00.855+0000 7fa365a89700 1 --2- 172.21.15.5:0/19016172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa360108760 0x7fa36019e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:00.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:00.855+0000 7fa365a89700 1 -- 172.21.15.5:0/19016172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa360107e10 msgr2=0x7fa36019a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:00.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:00.855+0000 7fa365a89700 1 --2- 172.21.15.5:0/19016172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa360107e10 0x7fa36019a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:00.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:00.855+0000 7fa365a89700 1 -- 172.21.15.5:0/19016172 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa36019eb70 con 0x7fa3601034e0 2024-05-09T00:45:00.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:00.855+0000 7fa365288700 1 --2- 172.21.15.5:0/19016172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa360107e10 0x7fa36019a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:45:00.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:00.855+0000 7fa365a89700 1 --2- 172.21.15.5:0/19016172 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3601034e0 0x7fa360199d50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa35c00c9b0 tx=0x7fa35c00ccc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:00.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:00.855+0000 7fa356ffd700 1 -- 172.21.15.5:0/19016172 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa35c007b00 con 0x7fa3601034e0 2024-05-09T00:45:00.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:00.855+0000 7fa367ced700 1 -- 172.21.15.5:0/19016172 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa36019ee60 con 0x7fa3601034e0 2024-05-09T00:45:00.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:00.855+0000 7fa356ffd700 1 -- 172.21.15.5:0/19016172 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa35c00ce50 con 0x7fa3601034e0 2024-05-09T00:45:00.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:00.855+0000 7fa356ffd700 1 -- 172.21.15.5:0/19016172 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa35c018660 con 0x7fa3601034e0 2024-05-09T00:45:00.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:00.855+0000 7fa367ced700 1 -- 172.21.15.5:0/19016172 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa36019f3a0 con 0x7fa3601034e0 2024-05-09T00:45:00.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:00.856+0000 7fa367ced700 1 -- 172.21.15.5:0/19016172 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa36006c480 con 0x7fa3601034e0 2024-05-09T00:45:00.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:00.859+0000 7fa356ffd700 1 -- 172.21.15.5:0/19016172 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fa35c00f450 con 0x7fa3601034e0 2024-05-09T00:45:00.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:00.859+0000 7fa356ffd700 1 --2- 172.21.15.5:0/19016172 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fa34c0b2910 0x7fa34c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:00.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:00.859+0000 7fa365288700 1 --2- 172.21.15.5:0/19016172 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fa34c0b2910 0x7fa34c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:00.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:00.860+0000 7fa356ffd700 1 -- 172.21.15.5:0/19016172 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8227+0+0 (secure 0 0 0) 0x7fa35c0e2790 con 0x7fa3601034e0 2024-05-09T00:45:00.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:00.860+0000 7fa356ffd700 1 --2- 172.21.15.5:0/19016172 >> [v2:172.21.15.5:6826/4287220015,v1:172.21.15.5:6827/4287220015] conn(0x7fa34c0bc4a0 0x7fa34c0be990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:00.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:00.860+0000 7fa356ffd700 1 -- 172.21.15.5:0/19016172 --> [v2:172.21.15.5:6826/4287220015,v1:172.21.15.5:6827/4287220015] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa34c0bf060 con 0x7fa34c0bc4a0 2024-05-09T00:45:00.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:00.860+0000 7fa356ffd700 1 -- 172.21.15.5:0/19016172 <== mon.2 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7fa35c0e2be0 con 0x7fa3601034e0 2024-05-09T00:45:00.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:00.860+0000 7fa36628a700 1 --2- 172.21.15.5:0/19016172 >> [v2:172.21.15.5:6826/4287220015,v1:172.21.15.5:6827/4287220015] conn(0x7fa34c0bc4a0 0x7fa34c0be990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:00.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:00.860+0000 7fa365288700 1 --2- 172.21.15.5:0/19016172 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fa34c0b2910 0x7fa34c0b4e20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa360102960 tx=0x7fa35000e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:00.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:00.860+0000 7fa36628a700 1 --2- 172.21.15.5:0/19016172 >> [v2:172.21.15.5:6826/4287220015,v1:172.21.15.5:6827/4287220015] conn(0x7fa34c0bc4a0 0x7fa34c0be990 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:00.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:00.862+0000 7fa356ffd700 1 -- 172.21.15.5:0/19016172 <== osd.9 v2:172.21.15.5:6826/4287220015 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa34c0bf060 con 0x7fa34c0bc4a0 2024-05-09T00:45:00.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:00.879+0000 7fa367ced700 1 -- 172.21.15.5:0/19016172 --> [v2:172.21.15.5:6826/4287220015,v1:172.21.15.5:6827/4287220015] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa36006c900 con 0x7fa34c0bc4a0 2024-05-09T00:45:00.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:00.879+0000 7fa356ffd700 1 -- 172.21.15.5:0/19016172 <== osd.9 v2:172.21.15.5:6826/4287220015 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa36006c900 con 0x7fa34c0bc4a0 2024-05-09T00:45:00.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:00.879+0000 7fa367ced700 1 -- 172.21.15.5:0/19016172 >> [v2:172.21.15.5:6826/4287220015,v1:172.21.15.5:6827/4287220015] conn(0x7fa34c0bc4a0 msgr2=0x7fa34c0be990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:00.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:00.879+0000 7fa367ced700 1 --2- 172.21.15.5:0/19016172 >> [v2:172.21.15.5:6826/4287220015,v1:172.21.15.5:6827/4287220015] conn(0x7fa34c0bc4a0 0x7fa34c0be990 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:00.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:00.879+0000 7fa367ced700 1 -- 172.21.15.5:0/19016172 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fa34c0b2910 msgr2=0x7fa34c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:00.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:00.879+0000 7fa367ced700 1 --2- 172.21.15.5:0/19016172 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fa34c0b2910 0x7fa34c0b4e20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa360102960 tx=0x7fa35000e040 comp rx=0 tx=0).stop 2024-05-09T00:45:00.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:00.879+0000 7fa367ced700 1 -- 172.21.15.5:0/19016172 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3601034e0 msgr2=0x7fa360199d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:00.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:00.879+0000 7fa367ced700 1 --2- 172.21.15.5:0/19016172 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3601034e0 0x7fa360199d50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa35c00c9b0 tx=0x7fa35c00ccc0 comp rx=0 tx=0).stop 2024-05-09T00:45:00.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:00.879+0000 7fa36628a700 1 -- 172.21.15.5:0/19016172 reap_dead start 2024-05-09T00:45:00.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:00.879+0000 7fa367ced700 1 -- 172.21.15.5:0/19016172 shutdown_connections 2024-05-09T00:45:00.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:00.879+0000 7fa367ced700 1 -- 172.21.15.5:0/19016172 >> 172.21.15.5:0/19016172 conn(0x7fa3600fef30 msgr2=0x7fa360107580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:45:00.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:00.880+0000 7fa367ced700 1 -- 172.21.15.5:0/19016172 shutdown_connections 2024-05-09T00:45:00.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:00.880+0000 7fa367ced700 1 -- 172.21.15.5:0/19016172 wait complete. 2024-05-09T00:45:01.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:45:00 smithi157 ceph-mon[34294]: pgmap v329: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-09T00:45:01.197 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:45:00 smithi005 ceph-mon[30365]: pgmap v329: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-09T00:45:01.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:45:00 smithi131 ceph-mon[34547]: pgmap v329: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-09T00:45:01.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:01.625+0000 7fac5a171700 1 -- 172.21.15.5:0/1455142069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac541034c0 msgr2=0x7fac541038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:01.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:01.625+0000 7fac5a171700 1 --2- 172.21.15.5:0/1455142069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac541034c0 0x7fac541038d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fac440099c0 tx=0x7fac44009cd0 comp rx=0 tx=0).stop 2024-05-09T00:45:01.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:01.625+0000 7fac5a171700 1 -- 172.21.15.5:0/1455142069 shutdown_connections 2024-05-09T00:45:01.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:01.625+0000 7fac5a171700 1 --2- 172.21.15.5:0/1455142069 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac54108740 0x7fac541100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:01.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:01.625+0000 7fac5a171700 1 --2- 172.21.15.5:0/1455142069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac541034c0 0x7fac541038d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:01.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:01.625+0000 7fac5a171700 1 --2- 172.21.15.5:0/1455142069 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac54107df0 0x7fac54108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:01.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:01.625+0000 7fac5a171700 1 -- 172.21.15.5:0/1455142069 >> 172.21.15.5:0/1455142069 conn(0x7fac540fedf0 msgr2=0x7fac54101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:45:01.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:01.625+0000 7fac5a171700 1 -- 172.21.15.5:0/1455142069 shutdown_connections 2024-05-09T00:45:01.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:01.625+0000 7fac5a171700 1 -- 172.21.15.5:0/1455142069 wait complete. 2024-05-09T00:45:01.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:01.626+0000 7fac5a171700 1 Processor -- start 2024-05-09T00:45:01.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:01.626+0000 7fac5a171700 1 -- start start 2024-05-09T00:45:01.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:01.627+0000 7fac5a171700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac541034c0 0x7fac54110a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:01.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:01.627+0000 7fac5a171700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac54107df0 0x7fac54110fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:01.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:01.627+0000 7fac5a171700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac54108740 0x7fac54111500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:01.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:01.627+0000 7fac5a171700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fac54114560 con 0x7fac541034c0 2024-05-09T00:45:01.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:01.627+0000 7fac5a171700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fac541143e0 con 0x7fac54107df0 2024-05-09T00:45:01.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:01.627+0000 7fac5a171700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fac541146e0 con 0x7fac54108740 2024-05-09T00:45:01.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:01.627+0000 7fac537fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac541034c0 0x7fac54110a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:01.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:01.627+0000 7fac537fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac541034c0 0x7fac54110a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39148/0 (socket says 172.21.15.5:39148) 2024-05-09T00:45:01.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:01.627+0000 7fac537fe700 1 -- 172.21.15.5:0/2033140248 learned_addr learned my addr 172.21.15.5:0/2033140248 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:45:01.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:01.627+0000 7fac52ffd700 1 --2- 172.21.15.5:0/2033140248 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac54107df0 0x7fac54110fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:01.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:01.627+0000 7fac53fff700 1 --2- 172.21.15.5:0/2033140248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac54108740 0x7fac54111500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:01.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:01.627+0000 7fac537fe700 1 -- 172.21.15.5:0/2033140248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac54108740 msgr2=0x7fac54111500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:01.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:01.627+0000 7fac537fe700 1 --2- 172.21.15.5:0/2033140248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac54108740 0x7fac54111500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:01.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:01.627+0000 7fac537fe700 1 -- 172.21.15.5:0/2033140248 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac54107df0 msgr2=0x7fac54110fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:01.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:01.627+0000 7fac537fe700 1 --2- 172.21.15.5:0/2033140248 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac54107df0 0x7fac54110fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:01.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:01.627+0000 7fac537fe700 1 -- 172.21.15.5:0/2033140248 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac541a92d0 con 0x7fac541034c0 2024-05-09T00:45:01.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:01.627+0000 7fac537fe700 1 --2- 172.21.15.5:0/2033140248 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac541034c0 0x7fac54110a80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fac4400b580 tx=0x7fac440047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:01.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:01.628+0000 7fac50ff9700 1 -- 172.21.15.5:0/2033140248 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac44027070 con 0x7fac541034c0 2024-05-09T00:45:01.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:01.628+0000 7fac50ff9700 1 -- 172.21.15.5:0/2033140248 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac44004020 con 0x7fac541034c0 2024-05-09T00:45:01.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:01.628+0000 7fac50ff9700 1 -- 172.21.15.5:0/2033140248 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac44022e10 con 0x7fac541034c0 2024-05-09T00:45:01.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:01.628+0000 7fac5a171700 1 -- 172.21.15.5:0/2033140248 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac541a9560 con 0x7fac541034c0 2024-05-09T00:45:01.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:01.628+0000 7fac5a171700 1 -- 172.21.15.5:0/2033140248 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac541a99b0 con 0x7fac541034c0 2024-05-09T00:45:01.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:01.629+0000 7fac5a171700 1 -- 172.21.15.5:0/2033140248 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fac54063ed0 con 0x7fac541034c0 2024-05-09T00:45:01.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:01.631+0000 7fac50ff9700 1 -- 172.21.15.5:0/2033140248 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fac4402a070 con 0x7fac541034c0 2024-05-09T00:45:01.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:01.631+0000 7fac50ff9700 1 --2- 172.21.15.5:0/2033140248 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fac3c0b2910 0x7fac3c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:01.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:01.631+0000 7fac52ffd700 1 --2- 172.21.15.5:0/2033140248 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fac3c0b2910 0x7fac3c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:01.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:01.631+0000 7fac50ff9700 1 -- 172.21.15.5:0/2033140248 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8227+0+0 (secure 0 0 0) 0x7fac440e3780 con 0x7fac541034c0 2024-05-09T00:45:01.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:01.632+0000 7fac50ff9700 1 --2- 172.21.15.5:0/2033140248 >> [v2:172.21.15.157:6800/484818139,v1:172.21.15.157:6801/484818139] conn(0x7fac3c0bc4a0 0x7fac3c0be990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:01.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:01.632+0000 7fac50ff9700 1 -- 172.21.15.5:0/2033140248 --> [v2:172.21.15.157:6800/484818139,v1:172.21.15.157:6801/484818139] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fac3c0bf060 con 0x7fac3c0bc4a0 2024-05-09T00:45:01.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:01.632+0000 7fac50ff9700 1 -- 172.21.15.5:0/2033140248 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7fac440e3bd0 con 0x7fac541034c0 2024-05-09T00:45:01.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:01.632+0000 7fac52ffd700 1 --2- 172.21.15.5:0/2033140248 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fac3c0b2910 0x7fac3c0b4e20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fac541067e0 tx=0x7fac48009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:01.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:01.632+0000 7fac53fff700 1 --2- 172.21.15.5:0/2033140248 >> [v2:172.21.15.157:6800/484818139,v1:172.21.15.157:6801/484818139] conn(0x7fac3c0bc4a0 0x7fac3c0be990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:01.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:01.632+0000 7fac53fff700 1 --2- 172.21.15.5:0/2033140248 >> [v2:172.21.15.157:6800/484818139,v1:172.21.15.157:6801/484818139] conn(0x7fac3c0bc4a0 0x7fac3c0be990 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:01.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:01.633+0000 7fac50ff9700 1 -- 172.21.15.5:0/2033140248 <== osd.2 v2:172.21.15.157:6800/484818139 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fac3c0bf060 con 0x7fac3c0bc4a0 2024-05-09T00:45:01.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:01.650+0000 7fac5a171700 1 -- 172.21.15.5:0/2033140248 --> [v2:172.21.15.157:6800/484818139,v1:172.21.15.157:6801/484818139] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fac540642d0 con 0x7fac3c0bc4a0 2024-05-09T00:45:01.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:01.650+0000 7fac50ff9700 1 -- 172.21.15.5:0/2033140248 <== osd.2 v2:172.21.15.157:6800/484818139 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fac540642d0 con 0x7fac3c0bc4a0 2024-05-09T00:45:01.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:01.651+0000 7fac5a171700 1 -- 172.21.15.5:0/2033140248 >> [v2:172.21.15.157:6800/484818139,v1:172.21.15.157:6801/484818139] conn(0x7fac3c0bc4a0 msgr2=0x7fac3c0be990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:01.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:01.651+0000 7fac5a171700 1 --2- 172.21.15.5:0/2033140248 >> [v2:172.21.15.157:6800/484818139,v1:172.21.15.157:6801/484818139] conn(0x7fac3c0bc4a0 0x7fac3c0be990 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:01.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:01.651+0000 7fac5a171700 1 -- 172.21.15.5:0/2033140248 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fac3c0b2910 msgr2=0x7fac3c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:01.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:01.651+0000 7fac5a171700 1 --2- 172.21.15.5:0/2033140248 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fac3c0b2910 0x7fac3c0b4e20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fac541067e0 tx=0x7fac48009190 comp rx=0 tx=0).stop 2024-05-09T00:45:01.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:01.651+0000 7fac5a171700 1 -- 172.21.15.5:0/2033140248 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac541034c0 msgr2=0x7fac54110a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:01.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:01.651+0000 7fac5a171700 1 --2- 172.21.15.5:0/2033140248 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac541034c0 0x7fac54110a80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fac4400b580 tx=0x7fac440047b0 comp rx=0 tx=0).stop 2024-05-09T00:45:01.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:01.651+0000 7fac53fff700 1 -- 172.21.15.5:0/2033140248 reap_dead start 2024-05-09T00:45:01.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:01.651+0000 7fac5a171700 1 -- 172.21.15.5:0/2033140248 shutdown_connections 2024-05-09T00:45:01.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:01.651+0000 7fac5a171700 1 -- 172.21.15.5:0/2033140248 >> 172.21.15.5:0/2033140248 conn(0x7fac540fedf0 msgr2=0x7fac54107120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:45:01.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:01.651+0000 7fac5a171700 1 -- 172.21.15.5:0/2033140248 shutdown_connections 2024-05-09T00:45:01.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:01.651+0000 7fac5a171700 1 -- 172.21.15.5:0/2033140248 wait complete. 2024-05-09T00:45:02.681 INFO:teuthology.orchestra.run.smithi005.stdout:77309411405 2024-05-09T00:45:02.682 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-09T00:45:03.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.168+0000 7fef77094700 1 -- 172.21.15.5:0/4015387704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef70107df0 msgr2=0x7fef70108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:03.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.168+0000 7fef77094700 1 --2- 172.21.15.5:0/4015387704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef70107df0 0x7fef70108200 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fef6000b0d0 tx=0x7fef6000b3e0 comp rx=0 tx=0).stop 2024-05-09T00:45:03.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.168+0000 7fef77094700 1 -- 172.21.15.5:0/4015387704 shutdown_connections 2024-05-09T00:45:03.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.168+0000 7fef77094700 1 --2- 172.21.15.5:0/4015387704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef70108740 0x7fef701100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:03.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.168+0000 7fef77094700 1 --2- 172.21.15.5:0/4015387704 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef701034c0 0x7fef701038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:03.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.168+0000 7fef77094700 1 --2- 172.21.15.5:0/4015387704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef70107df0 0x7fef70108200 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:03.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.168+0000 7fef77094700 1 -- 172.21.15.5:0/4015387704 >> 172.21.15.5:0/4015387704 conn(0x7fef700fedf0 msgr2=0x7fef70101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:45:03.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.169+0000 7fef77094700 1 -- 172.21.15.5:0/4015387704 shutdown_connections 2024-05-09T00:45:03.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.169+0000 7fef77094700 1 -- 172.21.15.5:0/4015387704 wait complete. 2024-05-09T00:45:03.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.169+0000 7fef77094700 1 Processor -- start 2024-05-09T00:45:03.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.169+0000 7fef77094700 1 -- start start 2024-05-09T00:45:03.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.170+0000 7fef77094700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef701034c0 0x7fef7019bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:03.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.170+0000 7fef77094700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef70107df0 0x7fef7019c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:03.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.170+0000 7fef77094700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef70108740 0x7fef7019c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:03.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.170+0000 7fef77094700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fef70113350 con 0x7fef70107df0 2024-05-09T00:45:03.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.170+0000 7fef77094700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fef701131d0 con 0x7fef701034c0 2024-05-09T00:45:03.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.170+0000 7fef77094700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fef701134d0 con 0x7fef70108740 2024-05-09T00:45:03.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.170+0000 7fef6ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef70107df0 0x7fef7019c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:03.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.170+0000 7fef6ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef70107df0 0x7fef7019c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39174/0 (socket says 172.21.15.5:39174) 2024-05-09T00:45:03.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.170+0000 7fef6ffff700 1 -- 172.21.15.5:0/3857897306 learned_addr learned my addr 172.21.15.5:0/3857897306 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:45:03.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.170+0000 7fef74e30700 1 --2- 172.21.15.5:0/3857897306 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef701034c0 0x7fef7019bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:03.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.170+0000 7fef75631700 1 --2- 172.21.15.5:0/3857897306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef70108740 0x7fef7019c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:03.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.170+0000 7fef6ffff700 1 -- 172.21.15.5:0/3857897306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef70108740 msgr2=0x7fef7019c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:03.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.170+0000 7fef6ffff700 1 --2- 172.21.15.5:0/3857897306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef70108740 0x7fef7019c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:03.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.170+0000 7fef6ffff700 1 -- 172.21.15.5:0/3857897306 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef701034c0 msgr2=0x7fef7019bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:03.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.170+0000 7fef6ffff700 1 --2- 172.21.15.5:0/3857897306 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef701034c0 0x7fef7019bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:03.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.170+0000 7fef6ffff700 1 -- 172.21.15.5:0/3857897306 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef701a7000 con 0x7fef70107df0 2024-05-09T00:45:03.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.170+0000 7fef6ffff700 1 --2- 172.21.15.5:0/3857897306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef70107df0 0x7fef7019c1f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fef60009850 tx=0x7fef60007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:03.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.171+0000 7fef6dffb700 1 -- 172.21.15.5:0/3857897306 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef60004c50 con 0x7fef70107df0 2024-05-09T00:45:03.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.171+0000 7fef77094700 1 -- 172.21.15.5:0/3857897306 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef70199aa0 con 0x7fef70107df0 2024-05-09T00:45:03.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.171+0000 7fef6dffb700 1 -- 172.21.15.5:0/3857897306 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef60004df0 con 0x7fef70107df0 2024-05-09T00:45:03.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.171+0000 7fef6dffb700 1 -- 172.21.15.5:0/3857897306 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef60005710 con 0x7fef70107df0 2024-05-09T00:45:03.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.171+0000 7fef77094700 1 -- 172.21.15.5:0/3857897306 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef70199f80 con 0x7fef70107df0 2024-05-09T00:45:03.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.172+0000 7fef77094700 1 -- 172.21.15.5:0/3857897306 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fef70063e90 con 0x7fef70107df0 2024-05-09T00:45:03.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.173+0000 7fef6dffb700 1 -- 172.21.15.5:0/3857897306 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fef600058b0 con 0x7fef70107df0 2024-05-09T00:45:03.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.174+0000 7fef6dffb700 1 --2- 172.21.15.5:0/3857897306 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fef5c0b2910 0x7fef5c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:03.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.174+0000 7fef74e30700 1 --2- 172.21.15.5:0/3857897306 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fef5c0b2910 0x7fef5c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:03.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.174+0000 7fef6dffb700 1 -- 172.21.15.5:0/3857897306 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8227+0+0 (secure 0 0 0) 0x7fef6000f030 con 0x7fef70107df0 2024-05-09T00:45:03.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.174+0000 7fef6dffb700 1 --2- 172.21.15.5:0/3857897306 >> [v2:172.21.15.131:6816/4125045219,v1:172.21.15.131:6817/4125045219] conn(0x7fef5c0bc4a0 0x7fef5c0be990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:03.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.174+0000 7fef6dffb700 1 -- 172.21.15.5:0/3857897306 --> [v2:172.21.15.131:6816/4125045219,v1:172.21.15.131:6817/4125045219] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fef600075c0 con 0x7fef5c0bc4a0 2024-05-09T00:45:03.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.174+0000 7fef6dffb700 1 -- 172.21.15.5:0/3857897306 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7fef600e78c0 con 0x7fef70107df0 2024-05-09T00:45:03.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.175+0000 7fef74e30700 1 --2- 172.21.15.5:0/3857897306 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fef5c0b2910 0x7fef5c0b4e20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fef58009c60 tx=0x7fef58009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:03.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.175+0000 7fef75631700 1 --2- 172.21.15.5:0/3857897306 >> [v2:172.21.15.131:6816/4125045219,v1:172.21.15.131:6817/4125045219] conn(0x7fef5c0bc4a0 0x7fef5c0be990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:03.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.175+0000 7fef75631700 1 --2- 172.21.15.5:0/3857897306 >> [v2:172.21.15.131:6816/4125045219,v1:172.21.15.131:6817/4125045219] conn(0x7fef5c0bc4a0 0x7fef5c0be990 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:03.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.176+0000 7fef6dffb700 1 -- 172.21.15.5:0/3857897306 <== osd.7 v2:172.21.15.131:6816/4125045219 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fef600075c0 con 0x7fef5c0bc4a0 2024-05-09T00:45:03.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.193+0000 7fef77094700 1 -- 172.21.15.5:0/3857897306 --> [v2:172.21.15.131:6816/4125045219,v1:172.21.15.131:6817/4125045219] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fef700642c0 con 0x7fef5c0bc4a0 2024-05-09T00:45:03.194 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:45:02 smithi005 ceph-mon[30365]: pgmap v330: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-05-09T00:45:03.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.193+0000 7fef6dffb700 1 -- 172.21.15.5:0/3857897306 <== osd.7 v2:172.21.15.131:6816/4125045219 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fef700642c0 con 0x7fef5c0bc4a0 2024-05-09T00:45:03.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.194+0000 7fef77094700 1 -- 172.21.15.5:0/3857897306 >> [v2:172.21.15.131:6816/4125045219,v1:172.21.15.131:6817/4125045219] conn(0x7fef5c0bc4a0 msgr2=0x7fef5c0be990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:03.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.194+0000 7fef77094700 1 --2- 172.21.15.5:0/3857897306 >> [v2:172.21.15.131:6816/4125045219,v1:172.21.15.131:6817/4125045219] conn(0x7fef5c0bc4a0 0x7fef5c0be990 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:03.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.194+0000 7fef77094700 1 -- 172.21.15.5:0/3857897306 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fef5c0b2910 msgr2=0x7fef5c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:03.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.194+0000 7fef77094700 1 --2- 172.21.15.5:0/3857897306 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fef5c0b2910 0x7fef5c0b4e20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fef58009c60 tx=0x7fef58009260 comp rx=0 tx=0).stop 2024-05-09T00:45:03.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.194+0000 7fef77094700 1 -- 172.21.15.5:0/3857897306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef70107df0 msgr2=0x7fef7019c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:03.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.194+0000 7fef77094700 1 --2- 172.21.15.5:0/3857897306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef70107df0 0x7fef7019c1f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fef60009850 tx=0x7fef60007a20 comp rx=0 tx=0).stop 2024-05-09T00:45:03.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.194+0000 7fef75631700 1 -- 172.21.15.5:0/3857897306 reap_dead start 2024-05-09T00:45:03.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.194+0000 7fef77094700 1 -- 172.21.15.5:0/3857897306 shutdown_connections 2024-05-09T00:45:03.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.194+0000 7fef77094700 1 -- 172.21.15.5:0/3857897306 >> 172.21.15.5:0/3857897306 conn(0x7fef700fedf0 msgr2=0x7fef701075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:45:03.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.194+0000 7fef77094700 1 -- 172.21.15.5:0/3857897306 shutdown_connections 2024-05-09T00:45:03.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.194+0000 7fef77094700 1 -- 172.21.15.5:0/3857897306 wait complete. 2024-05-09T00:45:03.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:45:02 smithi131 ceph-mon[34547]: pgmap v330: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-05-09T00:45:03.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:45:02 smithi157 ceph-mon[34294]: pgmap v330: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-05-09T00:45:04.016 INFO:teuthology.orchestra.run.smithi005.stdout:201863462965 2024-05-09T00:45:04.017 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-05-09T00:45:04.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:04.623+0000 7f372765a700 1 -- 172.21.15.5:0/1106304981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37201045f0 msgr2=0x7f3720104a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:04.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:04.623+0000 7f372765a700 1 --2- 172.21.15.5:0/1106304981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37201045f0 0x7f3720104a00 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f371c00b0d0 tx=0x7f371c00b3e0 comp rx=0 tx=0).stop 2024-05-09T00:45:04.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:04.623+0000 7f372765a700 1 -- 172.21.15.5:0/1106304981 shutdown_connections 2024-05-09T00:45:04.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:04.623+0000 7f372765a700 1 --2- 172.21.15.5:0/1106304981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3720109870 0x7f37201111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:04.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:04.623+0000 7f372765a700 1 --2- 172.21.15.5:0/1106304981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37201045f0 0x7f3720104a00 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:04.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:04.623+0000 7f372765a700 1 --2- 172.21.15.5:0/1106304981 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3720108f20 0x7f3720109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:04.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:04.623+0000 7f372765a700 1 -- 172.21.15.5:0/1106304981 >> 172.21.15.5:0/1106304981 conn(0x7f37200fffe0 msgr2=0x7f3720102400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:45:04.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:04.623+0000 7f372765a700 1 -- 172.21.15.5:0/1106304981 shutdown_connections 2024-05-09T00:45:04.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:04.623+0000 7f372765a700 1 -- 172.21.15.5:0/1106304981 wait complete. 2024-05-09T00:45:04.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:04.624+0000 7f372765a700 1 Processor -- start 2024-05-09T00:45:04.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:04.624+0000 7f372765a700 1 -- start start 2024-05-09T00:45:04.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:04.624+0000 7f372765a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37201045f0 0x7f3720075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:04.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:04.624+0000 7f372765a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3720108f20 0x7f3720076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:04.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:04.624+0000 7f372765a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3720109870 0x7f3720076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:04.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:04.624+0000 7f372765a700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3720114520 con 0x7f37201045f0 2024-05-09T00:45:04.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:04.624+0000 7f372765a700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f37201143a0 con 0x7f3720108f20 2024-05-09T00:45:04.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:04.624+0000 7f372765a700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f37201146a0 con 0x7f3720109870 2024-05-09T00:45:04.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:04.624+0000 7f37253f6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37201045f0 0x7f3720075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:04.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:04.624+0000 7f37253f6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37201045f0 0x7f3720075ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37304/0 (socket says 172.21.15.5:37304) 2024-05-09T00:45:04.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:04.624+0000 7f37253f6700 1 -- 172.21.15.5:0/173609408 learned_addr learned my addr 172.21.15.5:0/173609408 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:45:04.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:04.624+0000 7f3725bf7700 1 --2- 172.21.15.5:0/173609408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3720109870 0x7f3720076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:04.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:04.624+0000 7f3724bf5700 1 --2- 172.21.15.5:0/173609408 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3720108f20 0x7f3720076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:04.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:04.625+0000 7f37253f6700 1 -- 172.21.15.5:0/173609408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3720109870 msgr2=0x7f3720076540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:04.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:04.625+0000 7f37253f6700 1 --2- 172.21.15.5:0/173609408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3720109870 0x7f3720076540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:04.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:04.625+0000 7f37253f6700 1 -- 172.21.15.5:0/173609408 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3720108f20 msgr2=0x7f3720076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:04.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:04.625+0000 7f37253f6700 1 --2- 172.21.15.5:0/173609408 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3720108f20 0x7f3720076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:04.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:04.625+0000 7f37253f6700 1 -- 172.21.15.5:0/173609408 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37201a9230 con 0x7f37201045f0 2024-05-09T00:45:04.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:04.625+0000 7f37253f6700 1 --2- 172.21.15.5:0/173609408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37201045f0 0x7f3720075ac0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f371c00d010 tx=0x7f371c007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:04.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:04.625+0000 7f37167fc700 1 -- 172.21.15.5:0/173609408 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c01f070 con 0x7f37201045f0 2024-05-09T00:45:04.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:04.625+0000 7f37167fc700 1 -- 172.21.15.5:0/173609408 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f371c007e50 con 0x7f37201045f0 2024-05-09T00:45:04.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:04.625+0000 7f372765a700 1 -- 172.21.15.5:0/173609408 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37201a94c0 con 0x7f37201045f0 2024-05-09T00:45:04.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:04.625+0000 7f37167fc700 1 -- 172.21.15.5:0/173609408 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c01b4a0 con 0x7f37201045f0 2024-05-09T00:45:04.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:04.625+0000 7f372765a700 1 -- 172.21.15.5:0/173609408 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37201a9970 con 0x7f37201045f0 2024-05-09T00:45:04.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:04.626+0000 7f372765a700 1 -- 172.21.15.5:0/173609408 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3720063ed0 con 0x7f37201045f0 2024-05-09T00:45:04.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:04.628+0000 7f37167fc700 1 -- 172.21.15.5:0/173609408 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f371c022070 con 0x7f37201045f0 2024-05-09T00:45:04.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:04.629+0000 7f37167fc700 1 --2- 172.21.15.5:0/173609408 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f370c0b2990 0x7f370c0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:04.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:04.629+0000 7f3724bf5700 1 --2- 172.21.15.5:0/173609408 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f370c0b2990 0x7f370c0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:04.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:04.629+0000 7f37167fc700 1 -- 172.21.15.5:0/173609408 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8227+0+0 (secure 0 0 0) 0x7f371c00f020 con 0x7f37201045f0 2024-05-09T00:45:04.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:04.629+0000 7f37167fc700 1 --2- 172.21.15.5:0/173609408 >> [v2:172.21.15.131:6808/2915418378,v1:172.21.15.131:6809/2915418378] conn(0x7f370c0bc520 0x7f370c0bea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:04.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:04.630+0000 7f37167fc700 1 -- 172.21.15.5:0/173609408 --> [v2:172.21.15.131:6808/2915418378,v1:172.21.15.131:6809/2915418378] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f371c02de50 con 0x7f370c0bc520 2024-05-09T00:45:04.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:04.630+0000 7f37167fc700 1 -- 172.21.15.5:0/173609408 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f371c0e3ac0 con 0x7f37201045f0 2024-05-09T00:45:04.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:04.630+0000 7f3724bf5700 1 --2- 172.21.15.5:0/173609408 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f370c0b2990 0x7f370c0b4ea0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3720103ad0 tx=0x7f371000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:04.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:04.630+0000 7f3725bf7700 1 --2- 172.21.15.5:0/173609408 >> [v2:172.21.15.131:6808/2915418378,v1:172.21.15.131:6809/2915418378] conn(0x7f370c0bc520 0x7f370c0bea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:04.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:04.630+0000 7f3725bf7700 1 --2- 172.21.15.5:0/173609408 >> [v2:172.21.15.131:6808/2915418378,v1:172.21.15.131:6809/2915418378] conn(0x7f370c0bc520 0x7f370c0bea10 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:04.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:04.632+0000 7f37167fc700 1 -- 172.21.15.5:0/173609408 <== osd.4 v2:172.21.15.131:6808/2915418378 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f371c02de50 con 0x7f370c0bc520 2024-05-09T00:45:04.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:04.650+0000 7f372765a700 1 -- 172.21.15.5:0/173609408 --> [v2:172.21.15.131:6808/2915418378,v1:172.21.15.131:6809/2915418378] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f37200642d0 con 0x7f370c0bc520 2024-05-09T00:45:04.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:04.651+0000 7f37167fc700 1 -- 172.21.15.5:0/173609408 <== osd.4 v2:172.21.15.131:6808/2915418378 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f37200642d0 con 0x7f370c0bc520 2024-05-09T00:45:04.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:04.651+0000 7f372765a700 1 -- 172.21.15.5:0/173609408 >> [v2:172.21.15.131:6808/2915418378,v1:172.21.15.131:6809/2915418378] conn(0x7f370c0bc520 msgr2=0x7f370c0bea10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:04.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:04.651+0000 7f372765a700 1 --2- 172.21.15.5:0/173609408 >> [v2:172.21.15.131:6808/2915418378,v1:172.21.15.131:6809/2915418378] conn(0x7f370c0bc520 0x7f370c0bea10 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:04.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:04.651+0000 7f372765a700 1 -- 172.21.15.5:0/173609408 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f370c0b2990 msgr2=0x7f370c0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:04.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:04.651+0000 7f372765a700 1 --2- 172.21.15.5:0/173609408 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f370c0b2990 0x7f370c0b4ea0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3720103ad0 tx=0x7f371000b410 comp rx=0 tx=0).stop 2024-05-09T00:45:04.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:04.651+0000 7f372765a700 1 -- 172.21.15.5:0/173609408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37201045f0 msgr2=0x7f3720075ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:04.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:04.651+0000 7f372765a700 1 --2- 172.21.15.5:0/173609408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37201045f0 0x7f3720075ac0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f371c00d010 tx=0x7f371c007a20 comp rx=0 tx=0).stop 2024-05-09T00:45:04.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:04.652+0000 7f3725bf7700 1 -- 172.21.15.5:0/173609408 reap_dead start 2024-05-09T00:45:04.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:04.652+0000 7f372765a700 1 -- 172.21.15.5:0/173609408 shutdown_connections 2024-05-09T00:45:04.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:04.652+0000 7f372765a700 1 -- 172.21.15.5:0/173609408 >> 172.21.15.5:0/173609408 conn(0x7f37200fffe0 msgr2=0x7f3720108690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:45:04.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:04.652+0000 7f372765a700 1 -- 172.21.15.5:0/173609408 shutdown_connections 2024-05-09T00:45:04.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:04.652+0000 7f372765a700 1 -- 172.21.15.5:0/173609408 wait complete. 2024-05-09T00:45:05.238 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:45:04 smithi157 ceph-mon[34294]: pgmap v331: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-09T00:45:05.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.421+0000 7f5ef5eef700 1 -- 172.21.15.5:0/712066705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ef01089f0 msgr2=0x7f5ef0108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:05.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.421+0000 7f5ef5eef700 1 --2- 172.21.15.5:0/712066705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ef01089f0 0x7f5ef0108e00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5ee000aaa0 tx=0x7f5ee000adb0 comp rx=0 tx=0).stop 2024-05-09T00:45:05.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.421+0000 7f5ef5eef700 1 -- 172.21.15.5:0/712066705 shutdown_connections 2024-05-09T00:45:05.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.421+0000 7f5ef5eef700 1 --2- 172.21.15.5:0/712066705 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ef0103310 0x7f5ef010f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:05.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.421+0000 7f5ef5eef700 1 --2- 172.21.15.5:0/712066705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ef01089f0 0x7f5ef0108e00 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:05.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.421+0000 7f5ef5eef700 1 --2- 172.21.15.5:0/712066705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ef01029c0 0x7f5ef0102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:05.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.421+0000 7f5ef5eef700 1 -- 172.21.15.5:0/712066705 >> 172.21.15.5:0/712066705 conn(0x7f5ef00fe4e0 msgr2=0x7f5ef0100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:45:05.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.421+0000 7f5ef5eef700 1 -- 172.21.15.5:0/712066705 shutdown_connections 2024-05-09T00:45:05.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.421+0000 7f5ef5eef700 1 -- 172.21.15.5:0/712066705 wait complete. 2024-05-09T00:45:05.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.422+0000 7f5ef5eef700 1 Processor -- start 2024-05-09T00:45:05.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.422+0000 7f5ef5eef700 1 -- start start 2024-05-09T00:45:05.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.422+0000 7f5ef5eef700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ef01029c0 0x7f5ef019af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:05.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.422+0000 7f5ef5eef700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ef0103310 0x7f5ef019b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:05.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.422+0000 7f5ef5eef700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ef01089f0 0x7f5ef019f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:05.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.422+0000 7f5ef5eef700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef01122f0 con 0x7f5ef01029c0 2024-05-09T00:45:05.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.422+0000 7f5ef5eef700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef0112170 con 0x7f5ef0103310 2024-05-09T00:45:05.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.422+0000 7f5ef5eef700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef0112470 con 0x7f5ef01089f0 2024-05-09T00:45:05.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.422+0000 7f5eef7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ef01029c0 0x7f5ef019af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:05.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.423+0000 7f5eef7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ef01029c0 0x7f5ef019af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39186/0 (socket says 172.21.15.5:39186) 2024-05-09T00:45:05.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.423+0000 7f5eef7fe700 1 -- 172.21.15.5:0/3172705587 learned_addr learned my addr 172.21.15.5:0/3172705587 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:45:05.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.423+0000 7f5ee6ffd700 1 --2- 172.21.15.5:0/3172705587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ef0103310 0x7f5ef019b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:05.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.423+0000 7f5eef7fe700 1 -- 172.21.15.5:0/3172705587 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ef01089f0 msgr2=0x7f5ef019f7b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T00:45:05.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.423+0000 7f5eef7fe700 1 --2- 172.21.15.5:0/3172705587 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ef01089f0 0x7f5ef019f7b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:05.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.423+0000 7f5eef7fe700 1 -- 172.21.15.5:0/3172705587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ef0103310 msgr2=0x7f5ef019b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:05.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.423+0000 7f5eef7fe700 1 --2- 172.21.15.5:0/3172705587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ef0103310 0x7f5ef019b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:05.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.423+0000 7f5eef7fe700 1 -- 172.21.15.5:0/3172705587 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ef019fe50 con 0x7f5ef01029c0 2024-05-09T00:45:05.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.423+0000 7f5eef7fe700 1 --2- 172.21.15.5:0/3172705587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ef01029c0 0x7f5ef019af80 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0003930 tx=0x7f5ee0003960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:05.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.423+0000 7f5eed7fa700 1 -- 172.21.15.5:0/3172705587 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee0023030 con 0x7f5ef01029c0 2024-05-09T00:45:05.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.423+0000 7f5eed7fa700 1 -- 172.21.15.5:0/3172705587 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ee0004830 con 0x7f5ef01029c0 2024-05-09T00:45:05.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.423+0000 7f5ef5eef700 1 -- 172.21.15.5:0/3172705587 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ef01a00e0 con 0x7f5ef01029c0 2024-05-09T00:45:05.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.423+0000 7f5eed7fa700 1 -- 172.21.15.5:0/3172705587 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee0005380 con 0x7f5ef01029c0 2024-05-09T00:45:05.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.423+0000 7f5ef5eef700 1 -- 172.21.15.5:0/3172705587 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ef01a05c0 con 0x7f5ef01029c0 2024-05-09T00:45:05.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.424+0000 7f5ef5eef700 1 -- 172.21.15.5:0/3172705587 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5ef0063ed0 con 0x7f5ef01029c0 2024-05-09T00:45:05.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.426+0000 7f5eed7fa700 1 -- 172.21.15.5:0/3172705587 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f5ee0026070 con 0x7f5ef01029c0 2024-05-09T00:45:05.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.427+0000 7f5eed7fa700 1 --2- 172.21.15.5:0/3172705587 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f5edc0b2910 0x7f5edc0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:05.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.427+0000 7f5ee6ffd700 1 --2- 172.21.15.5:0/3172705587 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f5edc0b2910 0x7f5edc0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:05.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.427+0000 7f5eed7fa700 1 -- 172.21.15.5:0/3172705587 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8227+0+0 (secure 0 0 0) 0x7f5ee00e3d50 con 0x7f5ef01029c0 2024-05-09T00:45:05.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.427+0000 7f5eed7fa700 1 --2- 172.21.15.5:0/3172705587 >> [v2:172.21.15.157:6808/4188818663,v1:172.21.15.157:6809/4188818663] conn(0x7f5edc0bc4a0 0x7f5edc0be990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:05.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.427+0000 7f5eed7fa700 1 -- 172.21.15.5:0/3172705587 --> [v2:172.21.15.157:6808/4188818663,v1:172.21.15.157:6809/4188818663] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5ee0023ec0 con 0x7f5edc0bc4a0 2024-05-09T00:45:05.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.427+0000 7f5eed7fa700 1 -- 172.21.15.5:0/3172705587 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f5ee00e4100 con 0x7f5ef01029c0 2024-05-09T00:45:05.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.427+0000 7f5ee6ffd700 1 --2- 172.21.15.5:0/3172705587 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f5edc0b2910 0x7f5edc0b4e20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0100430 tx=0x7f5ed800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:05.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.427+0000 7f5eeffff700 1 --2- 172.21.15.5:0/3172705587 >> [v2:172.21.15.157:6808/4188818663,v1:172.21.15.157:6809/4188818663] conn(0x7f5edc0bc4a0 0x7f5edc0be990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:05.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.428+0000 7f5eeffff700 1 --2- 172.21.15.5:0/3172705587 >> [v2:172.21.15.157:6808/4188818663,v1:172.21.15.157:6809/4188818663] conn(0x7f5edc0bc4a0 0x7f5edc0be990 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:05.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.429+0000 7f5eed7fa700 1 -- 172.21.15.5:0/3172705587 <== osd.5 v2:172.21.15.157:6808/4188818663 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5ee0023ec0 con 0x7f5edc0bc4a0 2024-05-09T00:45:05.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.445+0000 7f5ef5eef700 1 -- 172.21.15.5:0/3172705587 --> [v2:172.21.15.157:6808/4188818663,v1:172.21.15.157:6809/4188818663] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5ef00642d0 con 0x7f5edc0bc4a0 2024-05-09T00:45:05.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.445+0000 7f5eed7fa700 1 -- 172.21.15.5:0/3172705587 <== osd.5 v2:172.21.15.157:6808/4188818663 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5ef00642d0 con 0x7f5edc0bc4a0 2024-05-09T00:45:05.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.446+0000 7f5ef5eef700 1 -- 172.21.15.5:0/3172705587 >> [v2:172.21.15.157:6808/4188818663,v1:172.21.15.157:6809/4188818663] conn(0x7f5edc0bc4a0 msgr2=0x7f5edc0be990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:05.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.446+0000 7f5ef5eef700 1 --2- 172.21.15.5:0/3172705587 >> [v2:172.21.15.157:6808/4188818663,v1:172.21.15.157:6809/4188818663] conn(0x7f5edc0bc4a0 0x7f5edc0be990 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:05.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.446+0000 7f5ef5eef700 1 -- 172.21.15.5:0/3172705587 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f5edc0b2910 msgr2=0x7f5edc0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:05.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.446+0000 7f5ef5eef700 1 --2- 172.21.15.5:0/3172705587 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f5edc0b2910 0x7f5edc0b4e20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0100430 tx=0x7f5ed800b410 comp rx=0 tx=0).stop 2024-05-09T00:45:05.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.446+0000 7f5ef5eef700 1 -- 172.21.15.5:0/3172705587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ef01029c0 msgr2=0x7f5ef019af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:05.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.446+0000 7f5ef5eef700 1 --2- 172.21.15.5:0/3172705587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ef01029c0 0x7f5ef019af80 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0003930 tx=0x7f5ee0003960 comp rx=0 tx=0).stop 2024-05-09T00:45:05.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.446+0000 7f5eeffff700 1 -- 172.21.15.5:0/3172705587 reap_dead start 2024-05-09T00:45:05.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.446+0000 7f5ef5eef700 1 -- 172.21.15.5:0/3172705587 shutdown_connections 2024-05-09T00:45:05.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.446+0000 7f5ef5eef700 1 -- 172.21.15.5:0/3172705587 >> 172.21.15.5:0/3172705587 conn(0x7f5ef00fe4e0 msgr2=0x7f5ef0100e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:45:05.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.446+0000 7f5ef5eef700 1 -- 172.21.15.5:0/3172705587 shutdown_connections 2024-05-09T00:45:05.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:03.446+0000 7f5ef5eef700 1 -- 172.21.15.5:0/3172705587 wait complete. 2024-05-09T00:45:05.251 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:45:04 smithi005 ceph-mon[30365]: pgmap v331: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-09T00:45:05.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:45:04 smithi131 ceph-mon[34547]: pgmap v331: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-09T00:45:06.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:06.728+0000 7f78ba1df700 1 -- 172.21.15.5:0/327960006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78b4109850 msgr2=0x7f78b41111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:06.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:06.728+0000 7f78ba1df700 1 --2- 172.21.15.5:0/327960006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78b4109850 0x7f78b41111b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f78ac00b0d0 tx=0x7f78ac00b3e0 comp rx=0 tx=0).stop 2024-05-09T00:45:06.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:06.728+0000 7f78ba1df700 1 -- 172.21.15.5:0/327960006 shutdown_connections 2024-05-09T00:45:06.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:06.728+0000 7f78ba1df700 1 --2- 172.21.15.5:0/327960006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78b4109850 0x7f78b41111b0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:06.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:06.728+0000 7f78ba1df700 1 --2- 172.21.15.5:0/327960006 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f78b4102ae0 0x7f78b4102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:06.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:06.728+0000 7f78ba1df700 1 --2- 172.21.15.5:0/327960006 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78b4108f00 0x7f78b4109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:06.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:06.728+0000 7f78ba1df700 1 -- 172.21.15.5:0/327960006 >> 172.21.15.5:0/327960006 conn(0x7f78b40fe540 msgr2=0x7f78b4100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:45:06.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:06.728+0000 7f78ba1df700 1 -- 172.21.15.5:0/327960006 shutdown_connections 2024-05-09T00:45:06.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:06.728+0000 7f78ba1df700 1 -- 172.21.15.5:0/327960006 wait complete. 2024-05-09T00:45:06.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:06.729+0000 7f78ba1df700 1 Processor -- start 2024-05-09T00:45:06.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:06.730+0000 7f78ba1df700 1 -- start start 2024-05-09T00:45:06.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:06.730+0000 7f78ba1df700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78b4102ae0 0x7f78b419ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:06.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:06.730+0000 7f78ba1df700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f78b4108f00 0x7f78b419b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:06.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:06.730+0000 7f78ba1df700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78b4109850 0x7f78b419f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:06.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:06.730+0000 7f78ba1df700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f78b4114480 con 0x7f78b4102ae0 2024-05-09T00:45:06.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:06.730+0000 7f78ba1df700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f78b4114300 con 0x7f78b4108f00 2024-05-09T00:45:06.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:06.730+0000 7f78ba1df700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f78b4114600 con 0x7f78b4109850 2024-05-09T00:45:06.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:06.730+0000 7f78b37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78b4102ae0 0x7f78b419ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:06.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:06.730+0000 7f78b37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78b4102ae0 0x7f78b419ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37322/0 (socket says 172.21.15.5:37322) 2024-05-09T00:45:06.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:06.730+0000 7f78b37fe700 1 -- 172.21.15.5:0/2529404694 learned_addr learned my addr 172.21.15.5:0/2529404694 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:45:06.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:06.730+0000 7f78b3fff700 1 --2- 172.21.15.5:0/2529404694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78b4109850 0x7f78b419f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:06.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:06.731+0000 7f78b2ffd700 1 --2- 172.21.15.5:0/2529404694 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f78b4108f00 0x7f78b419b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:06.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:06.731+0000 7f78b37fe700 1 -- 172.21.15.5:0/2529404694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78b4109850 msgr2=0x7f78b419f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:06.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:06.731+0000 7f78b37fe700 1 --2- 172.21.15.5:0/2529404694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78b4109850 0x7f78b419f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:06.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:06.731+0000 7f78b37fe700 1 -- 172.21.15.5:0/2529404694 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f78b4108f00 msgr2=0x7f78b419b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:06.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:06.731+0000 7f78b37fe700 1 --2- 172.21.15.5:0/2529404694 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f78b4108f00 0x7f78b419b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:06.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:06.731+0000 7f78b37fe700 1 -- 172.21.15.5:0/2529404694 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78b419fbb0 con 0x7f78b4102ae0 2024-05-09T00:45:06.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:06.731+0000 7f78b3fff700 1 --2- 172.21.15.5:0/2529404694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78b4109850 0x7f78b419f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:45:06.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:06.731+0000 7f78b37fe700 1 --2- 172.21.15.5:0/2529404694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78b4102ae0 0x7f78b419ad90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f78a400c9b0 tx=0x7f78a400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:06.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:06.731+0000 7f78b0ff9700 1 -- 172.21.15.5:0/2529404694 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a4007b00 con 0x7f78b4102ae0 2024-05-09T00:45:06.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:06.731+0000 7f78b0ff9700 1 -- 172.21.15.5:0/2529404694 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78a4007ca0 con 0x7f78b4102ae0 2024-05-09T00:45:06.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:06.731+0000 7f78ba1df700 1 -- 172.21.15.5:0/2529404694 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78b419fea0 con 0x7f78b4102ae0 2024-05-09T00:45:06.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:06.731+0000 7f78b0ff9700 1 -- 172.21.15.5:0/2529404694 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a4018730 con 0x7f78b4102ae0 2024-05-09T00:45:06.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:06.731+0000 7f78ba1df700 1 -- 172.21.15.5:0/2529404694 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78b41a7870 con 0x7f78b4102ae0 2024-05-09T00:45:06.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:06.732+0000 7f78ba1df700 1 -- 172.21.15.5:0/2529404694 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f78b4063f50 con 0x7f78b4102ae0 2024-05-09T00:45:06.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:06.734+0000 7f78b0ff9700 1 -- 172.21.15.5:0/2529404694 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f78a400f450 con 0x7f78b4102ae0 2024-05-09T00:45:06.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:06.735+0000 7f78b0ff9700 1 --2- 172.21.15.5:0/2529404694 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f789c0b2990 0x7f789c0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:06.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:06.735+0000 7f78b2ffd700 1 --2- 172.21.15.5:0/2529404694 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f789c0b2990 0x7f789c0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:06.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:06.735+0000 7f78b0ff9700 1 -- 172.21.15.5:0/2529404694 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8227+0+0 (secure 0 0 0) 0x7f78a40e28b0 con 0x7f78b4102ae0 2024-05-09T00:45:06.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:06.736+0000 7f78b0ff9700 1 --2- 172.21.15.5:0/2529404694 >> [v2:172.21.15.157:6816/2001626768,v1:172.21.15.157:6817/2001626768] conn(0x7f789c0bc520 0x7f789c0bea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:06.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:06.736+0000 7f78b0ff9700 1 -- 172.21.15.5:0/2529404694 --> [v2:172.21.15.157:6816/2001626768,v1:172.21.15.157:6817/2001626768] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f78a401fe50 con 0x7f789c0bc520 2024-05-09T00:45:06.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:06.736+0000 7f78b0ff9700 1 -- 172.21.15.5:0/2529404694 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f78a40e2c60 con 0x7f78b4102ae0 2024-05-09T00:45:06.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:06.736+0000 7f78b2ffd700 1 --2- 172.21.15.5:0/2529404694 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f789c0b2990 0x7f789c0b4ea0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f78b4108780 tx=0x7f78a8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:06.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:06.736+0000 7f78b3fff700 1 --2- 172.21.15.5:0/2529404694 >> [v2:172.21.15.157:6816/2001626768,v1:172.21.15.157:6817/2001626768] conn(0x7f789c0bc520 0x7f789c0bea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:06.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:06.736+0000 7f78b3fff700 1 --2- 172.21.15.5:0/2529404694 >> [v2:172.21.15.157:6816/2001626768,v1:172.21.15.157:6817/2001626768] conn(0x7f789c0bc520 0x7f789c0bea10 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:06.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:06.737+0000 7f78b0ff9700 1 -- 172.21.15.5:0/2529404694 <== osd.8 v2:172.21.15.157:6816/2001626768 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f78a401fe50 con 0x7f789c0bc520 2024-05-09T00:45:06.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:06.773+0000 7f78ba1df700 1 -- 172.21.15.5:0/2529404694 --> [v2:172.21.15.157:6816/2001626768,v1:172.21.15.157:6817/2001626768] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f78b4064350 con 0x7f789c0bc520 2024-05-09T00:45:06.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:06.773+0000 7f78b0ff9700 1 -- 172.21.15.5:0/2529404694 <== osd.8 v2:172.21.15.157:6816/2001626768 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f78b4064350 con 0x7f789c0bc520 2024-05-09T00:45:06.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:06.774+0000 7f78ba1df700 1 -- 172.21.15.5:0/2529404694 >> [v2:172.21.15.157:6816/2001626768,v1:172.21.15.157:6817/2001626768] conn(0x7f789c0bc520 msgr2=0x7f789c0bea10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:06.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:06.774+0000 7f78ba1df700 1 --2- 172.21.15.5:0/2529404694 >> [v2:172.21.15.157:6816/2001626768,v1:172.21.15.157:6817/2001626768] conn(0x7f789c0bc520 0x7f789c0bea10 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:06.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:06.774+0000 7f78ba1df700 1 -- 172.21.15.5:0/2529404694 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f789c0b2990 msgr2=0x7f789c0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:06.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:06.774+0000 7f78ba1df700 1 --2- 172.21.15.5:0/2529404694 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f789c0b2990 0x7f789c0b4ea0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f78b4108780 tx=0x7f78a8009260 comp rx=0 tx=0).stop 2024-05-09T00:45:06.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:06.774+0000 7f78ba1df700 1 -- 172.21.15.5:0/2529404694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78b4102ae0 msgr2=0x7f78b419ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:06.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:06.774+0000 7f78ba1df700 1 --2- 172.21.15.5:0/2529404694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78b4102ae0 0x7f78b419ad90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f78a400c9b0 tx=0x7f78a400cd70 comp rx=0 tx=0).stop 2024-05-09T00:45:06.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:06.774+0000 7f78b3fff700 1 -- 172.21.15.5:0/2529404694 reap_dead start 2024-05-09T00:45:06.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:06.774+0000 7f78ba1df700 1 -- 172.21.15.5:0/2529404694 shutdown_connections 2024-05-09T00:45:06.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:06.774+0000 7f78ba1df700 1 -- 172.21.15.5:0/2529404694 >> 172.21.15.5:0/2529404694 conn(0x7f78b40fe540 msgr2=0x7f78b4106ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:45:06.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:06.774+0000 7f78ba1df700 1 -- 172.21.15.5:0/2529404694 shutdown_connections 2024-05-09T00:45:06.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:06.774+0000 7f78ba1df700 1 -- 172.21.15.5:0/2529404694 wait complete. 2024-05-09T00:45:07.040 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:45:06 smithi005 ceph-mon[30365]: pgmap v332: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T00:45:07.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:45:06 smithi131 ceph-mon[34547]: pgmap v332: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T00:45:07.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:45:06 smithi157 ceph-mon[34294]: pgmap v332: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T00:45:07.579 INFO:teuthology.orchestra.run.smithi005.stdout:253403070506 2024-05-09T00:45:07.580 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-05-09T00:45:07.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.771+0000 7f868eb76700 1 -- 172.21.15.5:0/575126914 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86881054c0 msgr2=0x7f86881058d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:07.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.771+0000 7f868eb76700 1 --2- 172.21.15.5:0/575126914 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86881054c0 0x7f86881058d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8684009990 tx=0x7f8684009ca0 comp rx=0 tx=0).stop 2024-05-09T00:45:07.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.772+0000 7f868eb76700 1 -- 172.21.15.5:0/575126914 shutdown_connections 2024-05-09T00:45:07.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.772+0000 7f868eb76700 1 --2- 172.21.15.5:0/575126914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f868810e8f0 0x7f8688110dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:07.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.772+0000 7f868eb76700 1 --2- 172.21.15.5:0/575126914 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86881054c0 0x7f86881058d0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:07.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.772+0000 7f868eb76700 1 --2- 172.21.15.5:0/575126914 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8688105e10 0x7f868810e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:07.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.772+0000 7f868eb76700 1 -- 172.21.15.5:0/575126914 >> 172.21.15.5:0/575126914 conn(0x7f86880fedc0 msgr2=0x7f86881011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:45:07.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.772+0000 7f868eb76700 1 -- 172.21.15.5:0/575126914 shutdown_connections 2024-05-09T00:45:07.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.772+0000 7f868eb76700 1 -- 172.21.15.5:0/575126914 wait complete. 2024-05-09T00:45:07.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.772+0000 7f868eb76700 1 Processor -- start 2024-05-09T00:45:07.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.772+0000 7f868eb76700 1 -- start start 2024-05-09T00:45:07.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.773+0000 7f868eb76700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86881054c0 0x7f86881031b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:07.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.773+0000 7f868eb76700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8688105e10 0x7f86881036f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:07.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.773+0000 7f868eb76700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f868810e8f0 0x7f8688103c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:07.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.773+0000 7f868eb76700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8688114440 con 0x7f86881054c0 2024-05-09T00:45:07.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.773+0000 7f868eb76700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f86881142c0 con 0x7f8688105e10 2024-05-09T00:45:07.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.773+0000 7f868eb76700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f86881145c0 con 0x7f868810e8f0 2024-05-09T00:45:07.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.773+0000 7f868c912700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86881054c0 0x7f86881031b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:07.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.773+0000 7f868c912700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86881054c0 0x7f86881031b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37362/0 (socket says 172.21.15.5:37362) 2024-05-09T00:45:07.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.773+0000 7f868c912700 1 -- 172.21.15.5:0/1384685697 learned_addr learned my addr 172.21.15.5:0/1384685697 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:45:07.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.773+0000 7f868d113700 1 --2- 172.21.15.5:0/1384685697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f868810e8f0 0x7f8688103c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:07.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.773+0000 7f867ffff700 1 --2- 172.21.15.5:0/1384685697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8688105e10 0x7f86881036f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:07.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.773+0000 7f868c912700 1 -- 172.21.15.5:0/1384685697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f868810e8f0 msgr2=0x7f8688103c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:07.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.773+0000 7f868c912700 1 --2- 172.21.15.5:0/1384685697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f868810e8f0 0x7f8688103c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:07.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.773+0000 7f868c912700 1 -- 172.21.15.5:0/1384685697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8688105e10 msgr2=0x7f86881036f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:07.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.773+0000 7f868c912700 1 --2- 172.21.15.5:0/1384685697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8688105e10 0x7f86881036f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:07.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.773+0000 7f868c912700 1 -- 172.21.15.5:0/1384685697 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86881a9110 con 0x7f86881054c0 2024-05-09T00:45:07.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.773+0000 7f867ffff700 1 --2- 172.21.15.5:0/1384685697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8688105e10 0x7f86881036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:45:07.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.773+0000 7f868c912700 1 --2- 172.21.15.5:0/1384685697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86881054c0 0x7f86881031b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8684000c00 tx=0x7f86840047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:07.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.773+0000 7f867dffb700 1 -- 172.21.15.5:0/1384685697 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8684027070 con 0x7f86881054c0 2024-05-09T00:45:07.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.774+0000 7f867dffb700 1 -- 172.21.15.5:0/1384685697 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8684003f20 con 0x7f86881054c0 2024-05-09T00:45:07.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.774+0000 7f867dffb700 1 -- 172.21.15.5:0/1384685697 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8684022cf0 con 0x7f86881054c0 2024-05-09T00:45:07.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.774+0000 7f868eb76700 1 -- 172.21.15.5:0/1384685697 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86881a93a0 con 0x7f86881054c0 2024-05-09T00:45:07.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.774+0000 7f868eb76700 1 -- 172.21.15.5:0/1384685697 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86881a9880 con 0x7f86881054c0 2024-05-09T00:45:07.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.775+0000 7f868eb76700 1 -- 172.21.15.5:0/1384685697 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8688063ed0 con 0x7f86881054c0 2024-05-09T00:45:07.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.777+0000 7f867dffb700 1 -- 172.21.15.5:0/1384685697 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f868402a070 con 0x7f86881054c0 2024-05-09T00:45:07.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.777+0000 7f867dffb700 1 --2- 172.21.15.5:0/1384685697 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f86700b2910 0x7f86700b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:07.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.778+0000 7f867ffff700 1 --2- 172.21.15.5:0/1384685697 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f86700b2910 0x7f86700b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:07.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.778+0000 7f867dffb700 1 -- 172.21.15.5:0/1384685697 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8227+0+0 (secure 0 0 0) 0x7f86840e3820 con 0x7f86881054c0 2024-05-09T00:45:07.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.778+0000 7f867dffb700 1 --2- 172.21.15.5:0/1384685697 >> [v2:172.21.15.5:6802/2963385132,v1:172.21.15.5:6803/2963385132] conn(0x7f86700bc4a0 0x7f86700be990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:07.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.778+0000 7f867dffb700 1 -- 172.21.15.5:0/1384685697 --> [v2:172.21.15.5:6802/2963385132,v1:172.21.15.5:6803/2963385132] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8684001d30 con 0x7f86700bc4a0 2024-05-09T00:45:07.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.778+0000 7f867dffb700 1 -- 172.21.15.5:0/1384685697 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f86840e3c70 con 0x7f86881054c0 2024-05-09T00:45:07.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.778+0000 7f868d113700 1 --2- 172.21.15.5:0/1384685697 >> [v2:172.21.15.5:6802/2963385132,v1:172.21.15.5:6803/2963385132] conn(0x7f86700bc4a0 0x7f86700be990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:07.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.778+0000 7f867ffff700 1 --2- 172.21.15.5:0/1384685697 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f86700b2910 0x7f86700b4e20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8678005950 tx=0x7f86780058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:07.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.778+0000 7f868d113700 1 --2- 172.21.15.5:0/1384685697 >> [v2:172.21.15.5:6802/2963385132,v1:172.21.15.5:6803/2963385132] conn(0x7f86700bc4a0 0x7f86700be990 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:07.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.779+0000 7f867dffb700 1 -- 172.21.15.5:0/1384685697 <== osd.0 v2:172.21.15.5:6802/2963385132 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8684001d30 con 0x7f86700bc4a0 2024-05-09T00:45:07.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.795+0000 7f868eb76700 1 -- 172.21.15.5:0/1384685697 --> [v2:172.21.15.5:6802/2963385132,v1:172.21.15.5:6803/2963385132] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f86880642d0 con 0x7f86700bc4a0 2024-05-09T00:45:07.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.796+0000 7f867dffb700 1 -- 172.21.15.5:0/1384685697 <== osd.0 v2:172.21.15.5:6802/2963385132 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f86880642d0 con 0x7f86700bc4a0 2024-05-09T00:45:07.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.796+0000 7f868eb76700 1 -- 172.21.15.5:0/1384685697 >> [v2:172.21.15.5:6802/2963385132,v1:172.21.15.5:6803/2963385132] conn(0x7f86700bc4a0 msgr2=0x7f86700be990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:07.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.796+0000 7f868eb76700 1 --2- 172.21.15.5:0/1384685697 >> [v2:172.21.15.5:6802/2963385132,v1:172.21.15.5:6803/2963385132] conn(0x7f86700bc4a0 0x7f86700be990 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:07.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.796+0000 7f868eb76700 1 -- 172.21.15.5:0/1384685697 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f86700b2910 msgr2=0x7f86700b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:07.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.796+0000 7f868eb76700 1 --2- 172.21.15.5:0/1384685697 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f86700b2910 0x7f86700b4e20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8678005950 tx=0x7f86780058e0 comp rx=0 tx=0).stop 2024-05-09T00:45:07.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.796+0000 7f868eb76700 1 -- 172.21.15.5:0/1384685697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86881054c0 msgr2=0x7f86881031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:07.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.796+0000 7f868eb76700 1 --2- 172.21.15.5:0/1384685697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86881054c0 0x7f86881031b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8684000c00 tx=0x7f86840047b0 comp rx=0 tx=0).stop 2024-05-09T00:45:07.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.796+0000 7f868d113700 1 -- 172.21.15.5:0/1384685697 reap_dead start 2024-05-09T00:45:07.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.796+0000 7f868eb76700 1 -- 172.21.15.5:0/1384685697 shutdown_connections 2024-05-09T00:45:07.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.796+0000 7f868eb76700 1 -- 172.21.15.5:0/1384685697 >> 172.21.15.5:0/1384685697 conn(0x7f86880fedc0 msgr2=0x7f86880ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:45:07.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.797+0000 7f868eb76700 1 -- 172.21.15.5:0/1384685697 shutdown_connections 2024-05-09T00:45:07.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.797+0000 7f868eb76700 1 -- 172.21.15.5:0/1384685697 wait complete. 2024-05-09T00:45:08.219 INFO:teuthology.orchestra.run.smithi005.stdout:154618822720 2024-05-09T00:45:08.220 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-09T00:45:08.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:08.410+0000 7f7b4a6ee700 1 -- 172.21.15.5:0/2398831384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b441034c0 msgr2=0x7f7b441038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:08.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:08.410+0000 7f7b4a6ee700 1 --2- 172.21.15.5:0/2398831384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b441034c0 0x7f7b441038d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7b340099c0 tx=0x7f7b34009cd0 comp rx=0 tx=0).stop 2024-05-09T00:45:08.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:08.410+0000 7f7b4a6ee700 1 -- 172.21.15.5:0/2398831384 shutdown_connections 2024-05-09T00:45:08.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:08.410+0000 7f7b4a6ee700 1 --2- 172.21.15.5:0/2398831384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b44108740 0x7f7b441100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:08.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:08.410+0000 7f7b4a6ee700 1 --2- 172.21.15.5:0/2398831384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b441034c0 0x7f7b441038d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:08.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:08.410+0000 7f7b4a6ee700 1 --2- 172.21.15.5:0/2398831384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b44107df0 0x7f7b44108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:08.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:08.410+0000 7f7b4a6ee700 1 -- 172.21.15.5:0/2398831384 >> 172.21.15.5:0/2398831384 conn(0x7f7b440fedc0 msgr2=0x7f7b441011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:45:08.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:08.410+0000 7f7b4a6ee700 1 -- 172.21.15.5:0/2398831384 shutdown_connections 2024-05-09T00:45:08.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:08.410+0000 7f7b4a6ee700 1 -- 172.21.15.5:0/2398831384 wait complete. 2024-05-09T00:45:08.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:08.411+0000 7f7b4a6ee700 1 Processor -- start 2024-05-09T00:45:08.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:08.411+0000 7f7b4a6ee700 1 -- start start 2024-05-09T00:45:08.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:08.411+0000 7f7b4a6ee700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b441034c0 0x7f7b4419bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:08.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:08.411+0000 7f7b4a6ee700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b44107df0 0x7f7b4419c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:08.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:08.411+0000 7f7b4a6ee700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b44108740 0x7f7b4419c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:08.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:08.411+0000 7f7b4a6ee700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7b44113370 con 0x7f7b44107df0 2024-05-09T00:45:08.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:08.411+0000 7f7b4a6ee700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7b441131f0 con 0x7f7b441034c0 2024-05-09T00:45:08.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:08.411+0000 7f7b4a6ee700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7b441134f0 con 0x7f7b44108740 2024-05-09T00:45:08.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:08.411+0000 7f7b3bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b44107df0 0x7f7b4419c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:08.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:08.411+0000 7f7b3bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b44107df0 0x7f7b4419c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37372/0 (socket says 172.21.15.5:37372) 2024-05-09T00:45:08.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:08.411+0000 7f7b3bfff700 1 -- 172.21.15.5:0/2668944768 learned_addr learned my addr 172.21.15.5:0/2668944768 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:45:08.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:08.411+0000 7f7b43fff700 1 --2- 172.21.15.5:0/2668944768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b441034c0 0x7f7b4419bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:08.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:08.411+0000 7f7b48c8b700 1 --2- 172.21.15.5:0/2668944768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b44108740 0x7f7b4419c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:08.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:08.411+0000 7f7b3bfff700 1 -- 172.21.15.5:0/2668944768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b44108740 msgr2=0x7f7b4419c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:08.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:08.411+0000 7f7b3bfff700 1 --2- 172.21.15.5:0/2668944768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b44108740 0x7f7b4419c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:08.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:08.411+0000 7f7b3bfff700 1 -- 172.21.15.5:0/2668944768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b441034c0 msgr2=0x7f7b4419bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:08.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:08.411+0000 7f7b3bfff700 1 --2- 172.21.15.5:0/2668944768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b441034c0 0x7f7b4419bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:08.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:08.411+0000 7f7b3bfff700 1 -- 172.21.15.5:0/2668944768 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b441a7020 con 0x7f7b44107df0 2024-05-09T00:45:08.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:08.412+0000 7f7b3bfff700 1 --2- 172.21.15.5:0/2668944768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b44107df0 0x7f7b4419c210 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c00d980 tx=0x7f7b2c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:08.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:08.412+0000 7f7b41ffb700 1 -- 172.21.15.5:0/2668944768 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b2c017070 con 0x7f7b44107df0 2024-05-09T00:45:08.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:08.412+0000 7f7b41ffb700 1 -- 172.21.15.5:0/2668944768 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b2c00fd40 con 0x7f7b44107df0 2024-05-09T00:45:08.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:08.412+0000 7f7b4a6ee700 1 -- 172.21.15.5:0/2668944768 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b441a7310 con 0x7f7b44107df0 2024-05-09T00:45:08.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:08.412+0000 7f7b41ffb700 1 -- 172.21.15.5:0/2668944768 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b2c00e3b0 con 0x7f7b44107df0 2024-05-09T00:45:08.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:08.412+0000 7f7b4a6ee700 1 -- 172.21.15.5:0/2668944768 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b441a7850 con 0x7f7b44107df0 2024-05-09T00:45:08.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:08.413+0000 7f7b4a6ee700 1 -- 172.21.15.5:0/2668944768 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7b44063ed0 con 0x7f7b44107df0 2024-05-09T00:45:08.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:08.415+0000 7f7b41ffb700 1 -- 172.21.15.5:0/2668944768 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f7b2c010610 con 0x7f7b44107df0 2024-05-09T00:45:08.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:08.416+0000 7f7b41ffb700 1 --2- 172.21.15.5:0/2668944768 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f7b300b2910 0x7f7b300b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:08.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:08.416+0000 7f7b43fff700 1 --2- 172.21.15.5:0/2668944768 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f7b300b2910 0x7f7b300b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:08.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:08.416+0000 7f7b41ffb700 1 -- 172.21.15.5:0/2668944768 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8227+0+0 (secure 0 0 0) 0x7f7b2c0e2440 con 0x7f7b44107df0 2024-05-09T00:45:08.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:08.416+0000 7f7b41ffb700 1 --2- 172.21.15.5:0/2668944768 >> [v2:172.21.15.5:6818/2930397813,v1:172.21.15.5:6819/2930397813] conn(0x7f7b300bc4a0 0x7f7b300be990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:08.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:08.417+0000 7f7b41ffb700 1 -- 172.21.15.5:0/2668944768 --> [v2:172.21.15.5:6818/2930397813,v1:172.21.15.5:6819/2930397813] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7b2c016ec0 con 0x7f7b300bc4a0 2024-05-09T00:45:08.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:08.417+0000 7f7b41ffb700 1 -- 172.21.15.5:0/2668944768 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f7b2c0e27f0 con 0x7f7b44107df0 2024-05-09T00:45:08.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:08.417+0000 7f7b48c8b700 1 --2- 172.21.15.5:0/2668944768 >> [v2:172.21.15.5:6818/2930397813,v1:172.21.15.5:6819/2930397813] conn(0x7f7b300bc4a0 0x7f7b300be990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:08.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:08.417+0000 7f7b43fff700 1 --2- 172.21.15.5:0/2668944768 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f7b300b2910 0x7f7b300b4e20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7b34009990 tx=0x7f7b340036e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:08.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:08.417+0000 7f7b48c8b700 1 --2- 172.21.15.5:0/2668944768 >> [v2:172.21.15.5:6818/2930397813,v1:172.21.15.5:6819/2930397813] conn(0x7f7b300bc4a0 0x7f7b300be990 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:08.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:08.418+0000 7f7b41ffb700 1 -- 172.21.15.5:0/2668944768 <== osd.6 v2:172.21.15.5:6818/2930397813 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7b2c016ec0 con 0x7f7b300bc4a0 2024-05-09T00:45:08.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:08.441+0000 7f7b4a6ee700 1 -- 172.21.15.5:0/2668944768 --> [v2:172.21.15.5:6818/2930397813,v1:172.21.15.5:6819/2930397813] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7b440642d0 con 0x7f7b300bc4a0 2024-05-09T00:45:08.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:08.441+0000 7f7b41ffb700 1 -- 172.21.15.5:0/2668944768 <== osd.6 v2:172.21.15.5:6818/2930397813 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7b440642d0 con 0x7f7b300bc4a0 2024-05-09T00:45:08.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:08.442+0000 7f7b4a6ee700 1 -- 172.21.15.5:0/2668944768 >> [v2:172.21.15.5:6818/2930397813,v1:172.21.15.5:6819/2930397813] conn(0x7f7b300bc4a0 msgr2=0x7f7b300be990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:08.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:08.442+0000 7f7b4a6ee700 1 --2- 172.21.15.5:0/2668944768 >> [v2:172.21.15.5:6818/2930397813,v1:172.21.15.5:6819/2930397813] conn(0x7f7b300bc4a0 0x7f7b300be990 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:08.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:08.442+0000 7f7b4a6ee700 1 -- 172.21.15.5:0/2668944768 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f7b300b2910 msgr2=0x7f7b300b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:08.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:08.442+0000 7f7b4a6ee700 1 --2- 172.21.15.5:0/2668944768 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f7b300b2910 0x7f7b300b4e20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7b34009990 tx=0x7f7b340036e0 comp rx=0 tx=0).stop 2024-05-09T00:45:08.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:08.442+0000 7f7b4a6ee700 1 -- 172.21.15.5:0/2668944768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b44107df0 msgr2=0x7f7b4419c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:08.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:08.442+0000 7f7b4a6ee700 1 --2- 172.21.15.5:0/2668944768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b44107df0 0x7f7b4419c210 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c00d980 tx=0x7f7b2c00dc90 comp rx=0 tx=0).stop 2024-05-09T00:45:08.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:08.442+0000 7f7b48c8b700 1 -- 172.21.15.5:0/2668944768 reap_dead start 2024-05-09T00:45:08.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:08.442+0000 7f7b4a6ee700 1 -- 172.21.15.5:0/2668944768 shutdown_connections 2024-05-09T00:45:08.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:08.442+0000 7f7b4a6ee700 1 -- 172.21.15.5:0/2668944768 >> 172.21.15.5:0/2668944768 conn(0x7f7b440fedc0 msgr2=0x7f7b441096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:45:08.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:08.442+0000 7f7b4a6ee700 1 -- 172.21.15.5:0/2668944768 shutdown_connections 2024-05-09T00:45:08.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:08.442+0000 7f7b4a6ee700 1 -- 172.21.15.5:0/2668944768 wait complete. 2024-05-09T00:45:09.036 INFO:teuthology.orchestra.run.smithi005.stdout:227633266736 2024-05-09T00:45:09.037 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-05-09T00:45:09.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:45:08 smithi005 ceph-mon[30365]: pgmap v333: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T00:45:09.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:45:08 smithi131 ceph-mon[34547]: pgmap v333: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T00:45:09.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:45:08 smithi157 ceph-mon[34294]: pgmap v333: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T00:45:09.893 INFO:teuthology.orchestra.run.smithi005.stdout:133143986244 2024-05-09T00:45:09.893 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-09T00:45:09.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.166+0000 7f21812b1700 1 -- 172.21.15.5:0/2277162003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f217c108200 msgr2=0x7f217c108610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:09.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.166+0000 7f21812b1700 1 --2- 172.21.15.5:0/2277162003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f217c108200 0x7f217c108610 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f216c009990 tx=0x7f216c009ca0 comp rx=0 tx=0).stop 2024-05-09T00:45:09.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.166+0000 7f21812b1700 1 -- 172.21.15.5:0/2277162003 shutdown_connections 2024-05-09T00:45:09.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.166+0000 7f21812b1700 1 --2- 172.21.15.5:0/2277162003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f217c1042b0 0x7f217c110070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:09.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.166+0000 7f21812b1700 1 --2- 172.21.15.5:0/2277162003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f217c108200 0x7f217c108610 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:09.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.166+0000 7f21812b1700 1 --2- 172.21.15.5:0/2277162003 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f217c103960 0x7f217c103d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:09.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.166+0000 7f21812b1700 1 -- 172.21.15.5:0/2277162003 >> 172.21.15.5:0/2277162003 conn(0x7f217c0fe520 msgr2=0x7f217c100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:45:09.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.166+0000 7f21812b1700 1 -- 172.21.15.5:0/2277162003 shutdown_connections 2024-05-09T00:45:09.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.166+0000 7f21812b1700 1 -- 172.21.15.5:0/2277162003 wait complete. 2024-05-09T00:45:09.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.166+0000 7f21812b1700 1 Processor -- start 2024-05-09T00:45:09.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.167+0000 7f21812b1700 1 -- start start 2024-05-09T00:45:09.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.167+0000 7f21812b1700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f217c103960 0x7f217c19f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:09.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.167+0000 7f21812b1700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f217c1042b0 0x7f217c19fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:09.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.167+0000 7f21812b1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f217c108200 0x7f217c199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:09.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.167+0000 7f21812b1700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f217c113370 con 0x7f217c108200 2024-05-09T00:45:09.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.167+0000 7f21812b1700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f217c1131f0 con 0x7f217c1042b0 2024-05-09T00:45:09.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.167+0000 7f21812b1700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f217c1134f0 con 0x7f217c103960 2024-05-09T00:45:09.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.167+0000 7f217b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f217c108200 0x7f217c199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:09.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.167+0000 7f217b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f217c108200 0x7f217c199af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37348/0 (socket says 172.21.15.5:37348) 2024-05-09T00:45:09.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.167+0000 7f217b7fe700 1 -- 172.21.15.5:0/3126503353 learned_addr learned my addr 172.21.15.5:0/3126503353 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:45:09.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.167+0000 7f217affd700 1 --2- 172.21.15.5:0/3126503353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f217c103960 0x7f217c19f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:09.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.167+0000 7f217a7fc700 1 --2- 172.21.15.5:0/3126503353 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f217c1042b0 0x7f217c19fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:09.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.167+0000 7f217b7fe700 1 -- 172.21.15.5:0/3126503353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f217c103960 msgr2=0x7f217c19f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:09.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.167+0000 7f217b7fe700 1 --2- 172.21.15.5:0/3126503353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f217c103960 0x7f217c19f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:09.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.167+0000 7f217b7fe700 1 -- 172.21.15.5:0/3126503353 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f217c1042b0 msgr2=0x7f217c19fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:09.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.167+0000 7f217b7fe700 1 --2- 172.21.15.5:0/3126503353 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f217c1042b0 0x7f217c19fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:09.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.168+0000 7f217b7fe700 1 -- 172.21.15.5:0/3126503353 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f217c19a1d0 con 0x7f217c108200 2024-05-09T00:45:09.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.168+0000 7f217affd700 1 --2- 172.21.15.5:0/3126503353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f217c103960 0x7f217c19f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:45:09.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.168+0000 7f217b7fe700 1 --2- 172.21.15.5:0/3126503353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f217c108200 0x7f217c199af0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f217400ef30 tx=0x7f217400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:09.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.168+0000 7f2163fff700 1 -- 172.21.15.5:0/3126503353 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21740092e0 con 0x7f217c108200 2024-05-09T00:45:09.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.168+0000 7f21812b1700 1 -- 172.21.15.5:0/3126503353 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f217c19a460 con 0x7f217c108200 2024-05-09T00:45:09.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.168+0000 7f2163fff700 1 -- 172.21.15.5:0/3126503353 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2174004bb0 con 0x7f217c108200 2024-05-09T00:45:09.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.168+0000 7f2163fff700 1 -- 172.21.15.5:0/3126503353 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2174004020 con 0x7f217c108200 2024-05-09T00:45:09.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.168+0000 7f21812b1700 1 -- 172.21.15.5:0/3126503353 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f217c1a6760 con 0x7f217c108200 2024-05-09T00:45:09.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.169+0000 7f21812b1700 1 -- 172.21.15.5:0/3126503353 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f217c063f50 con 0x7f217c108200 2024-05-09T00:45:09.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.171+0000 7f2163fff700 1 -- 172.21.15.5:0/3126503353 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f217401b030 con 0x7f217c108200 2024-05-09T00:45:09.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.171+0000 7f2163fff700 1 --2- 172.21.15.5:0/3126503353 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f21640b2910 0x7f21640b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:09.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.172+0000 7f217affd700 1 --2- 172.21.15.5:0/3126503353 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f21640b2910 0x7f21640b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:09.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.172+0000 7f2163fff700 1 -- 172.21.15.5:0/3126503353 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8227+0+0 (secure 0 0 0) 0x7f21740e2bc0 con 0x7f217c108200 2024-05-09T00:45:09.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.172+0000 7f2163fff700 1 --2- 172.21.15.5:0/3126503353 >> [v2:172.21.15.131:6800/3837683163,v1:172.21.15.131:6801/3837683163] conn(0x7f21640bc4a0 0x7f21640be990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:10.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.172+0000 7f2163fff700 1 -- 172.21.15.5:0/3126503353 --> [v2:172.21.15.131:6800/3837683163,v1:172.21.15.131:6801/3837683163] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f21640bf060 con 0x7f21640bc4a0 2024-05-09T00:45:10.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.172+0000 7f2163fff700 1 -- 172.21.15.5:0/3126503353 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f21740e3010 con 0x7f217c108200 2024-05-09T00:45:10.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.172+0000 7f217affd700 1 --2- 172.21.15.5:0/3126503353 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f21640b2910 0x7f21640b4e20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f216c0059d0 tx=0x7f216c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:10.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.172+0000 7f217a7fc700 1 --2- 172.21.15.5:0/3126503353 >> [v2:172.21.15.131:6800/3837683163,v1:172.21.15.131:6801/3837683163] conn(0x7f21640bc4a0 0x7f21640be990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:10.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.172+0000 7f217a7fc700 1 --2- 172.21.15.5:0/3126503353 >> [v2:172.21.15.131:6800/3837683163,v1:172.21.15.131:6801/3837683163] conn(0x7f21640bc4a0 0x7f21640be990 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:10.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.173+0000 7f2163fff700 1 -- 172.21.15.5:0/3126503353 <== osd.1 v2:172.21.15.131:6800/3837683163 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f21640bf060 con 0x7f21640bc4a0 2024-05-09T00:45:10.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.190+0000 7f21812b1700 1 -- 172.21.15.5:0/3126503353 --> [v2:172.21.15.131:6800/3837683163,v1:172.21.15.131:6801/3837683163] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f217c0643d0 con 0x7f21640bc4a0 2024-05-09T00:45:10.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.190+0000 7f2163fff700 1 -- 172.21.15.5:0/3126503353 <== osd.1 v2:172.21.15.131:6800/3837683163 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f217c0643d0 con 0x7f21640bc4a0 2024-05-09T00:45:10.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.190+0000 7f21812b1700 1 -- 172.21.15.5:0/3126503353 >> [v2:172.21.15.131:6800/3837683163,v1:172.21.15.131:6801/3837683163] conn(0x7f21640bc4a0 msgr2=0x7f21640be990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:10.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.191+0000 7f21812b1700 1 --2- 172.21.15.5:0/3126503353 >> [v2:172.21.15.131:6800/3837683163,v1:172.21.15.131:6801/3837683163] conn(0x7f21640bc4a0 0x7f21640be990 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:10.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.191+0000 7f21812b1700 1 -- 172.21.15.5:0/3126503353 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f21640b2910 msgr2=0x7f21640b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:10.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.191+0000 7f21812b1700 1 --2- 172.21.15.5:0/3126503353 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f21640b2910 0x7f21640b4e20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f216c0059d0 tx=0x7f216c005960 comp rx=0 tx=0).stop 2024-05-09T00:45:10.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.191+0000 7f21812b1700 1 -- 172.21.15.5:0/3126503353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f217c108200 msgr2=0x7f217c199af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:10.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.191+0000 7f21812b1700 1 --2- 172.21.15.5:0/3126503353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f217c108200 0x7f217c199af0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f217400ef30 tx=0x7f217400c3a0 comp rx=0 tx=0).stop 2024-05-09T00:45:10.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.191+0000 7f217b7fe700 1 -- 172.21.15.5:0/3126503353 reap_dead start 2024-05-09T00:45:10.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.191+0000 7f21812b1700 1 -- 172.21.15.5:0/3126503353 shutdown_connections 2024-05-09T00:45:10.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.191+0000 7f21812b1700 1 -- 172.21.15.5:0/3126503353 >> 172.21.15.5:0/3126503353 conn(0x7f217c0fe520 msgr2=0x7f217c1076e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:45:10.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.191+0000 7f21812b1700 1 -- 172.21.15.5:0/3126503353 shutdown_connections 2024-05-09T00:45:10.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:07.191+0000 7f21812b1700 1 -- 172.21.15.5:0/3126503353 wait complete. 2024-05-09T00:45:10.634 INFO:teuthology.orchestra.run.smithi005.stdout:55834574932 2024-05-09T00:45:10.635 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-09T00:45:11.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:45:10 smithi005 ceph-mon[30365]: pgmap v334: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-09T00:45:11.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:45:10 smithi131 ceph-mon[34547]: pgmap v334: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-09T00:45:11.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:45:10 smithi157 ceph-mon[34294]: pgmap v334: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-09T00:45:11.568 INFO:teuthology.orchestra.run.smithi005.stdout:180388626491 2024-05-09T00:45:11.568 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-05-09T00:45:11.877 INFO:teuthology.orchestra.run.smithi005.stdout:34359738457 2024-05-09T00:45:11.878 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-09T00:45:12.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.206+0000 7f1f4c646700 1 -- 172.21.15.5:0/4246605351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f44108760 msgr2=0x7f1f441100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:12.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.206+0000 7f1f4c646700 1 --2- 172.21.15.5:0/4246605351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f44108760 0x7f1f441100c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c00b0d0 tx=0x7f1f3c00b3e0 comp rx=0 tx=0).stop 2024-05-09T00:45:12.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.206+0000 7f1f4c646700 1 -- 172.21.15.5:0/4246605351 shutdown_connections 2024-05-09T00:45:12.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.206+0000 7f1f4c646700 1 --2- 172.21.15.5:0/4246605351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f44108760 0x7f1f441100c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:12.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.206+0000 7f1f4c646700 1 --2- 172.21.15.5:0/4246605351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f441034e0 0x7f1f441038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:12.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.206+0000 7f1f4c646700 1 --2- 172.21.15.5:0/4246605351 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f44107e10 0x7f1f44108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:12.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.206+0000 7f1f4c646700 1 -- 172.21.15.5:0/4246605351 >> 172.21.15.5:0/4246605351 conn(0x7f1f440fef30 msgr2=0x7f1f44101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:45:12.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.206+0000 7f1f4c646700 1 -- 172.21.15.5:0/4246605351 shutdown_connections 2024-05-09T00:45:12.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.206+0000 7f1f4c646700 1 -- 172.21.15.5:0/4246605351 wait complete. 2024-05-09T00:45:12.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.207+0000 7f1f4c646700 1 Processor -- start 2024-05-09T00:45:12.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.207+0000 7f1f4c646700 1 -- start start 2024-05-09T00:45:12.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.207+0000 7f1f4c646700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f441034e0 0x7f1f4410f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:12.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.208+0000 7f1f4c646700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f44107e10 0x7f1f4410fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:12.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.208+0000 7f1f4c646700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f44108760 0x7f1f4410cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:12.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.208+0000 7f1f4c646700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1f44111cf0 con 0x7f1f44108760 2024-05-09T00:45:12.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.208+0000 7f1f4c646700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1f44111b70 con 0x7f1f441034e0 2024-05-09T00:45:12.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.208+0000 7f1f4c646700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1f44111e70 con 0x7f1f44107e10 2024-05-09T00:45:12.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.208+0000 7f1f4abe3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f44108760 0x7f1f4410cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:12.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.208+0000 7f1f4abe3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f44108760 0x7f1f4410cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37386/0 (socket says 172.21.15.5:37386) 2024-05-09T00:45:12.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.208+0000 7f1f4abe3700 1 -- 172.21.15.5:0/1360867894 learned_addr learned my addr 172.21.15.5:0/1360867894 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:45:12.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.208+0000 7f1f49be1700 1 --2- 172.21.15.5:0/1360867894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f44107e10 0x7f1f4410fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:12.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.208+0000 7f1f4a3e2700 1 --2- 172.21.15.5:0/1360867894 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f441034e0 0x7f1f4410f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:12.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.208+0000 7f1f49be1700 1 -- 172.21.15.5:0/1360867894 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f441034e0 msgr2=0x7f1f4410f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:12.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.208+0000 7f1f49be1700 1 --2- 172.21.15.5:0/1360867894 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f441034e0 0x7f1f4410f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:12.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.208+0000 7f1f49be1700 1 -- 172.21.15.5:0/1360867894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f44108760 msgr2=0x7f1f4410cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:12.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.208+0000 7f1f49be1700 1 --2- 172.21.15.5:0/1360867894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f44108760 0x7f1f4410cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:12.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.208+0000 7f1f49be1700 1 -- 172.21.15.5:0/1360867894 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f4410d270 con 0x7f1f44107e10 2024-05-09T00:45:12.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.208+0000 7f1f4abe3700 1 --2- 172.21.15.5:0/1360867894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f44108760 0x7f1f4410cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:45:12.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.209+0000 7f1f49be1700 1 --2- 172.21.15.5:0/1360867894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f44107e10 0x7f1f4410fd80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1f3400e9c0 tx=0x7f1f3400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:12.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.209+0000 7f1f3b7fe700 1 -- 172.21.15.5:0/1360867894 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3400c9c0 con 0x7f1f44107e10 2024-05-09T00:45:12.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.209+0000 7f1f4c646700 1 -- 172.21.15.5:0/1360867894 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f4410d560 con 0x7f1f44107e10 2024-05-09T00:45:12.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.209+0000 7f1f3b7fe700 1 -- 172.21.15.5:0/1360867894 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f3400cb60 con 0x7f1f44107e10 2024-05-09T00:45:12.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.209+0000 7f1f3b7fe700 1 -- 172.21.15.5:0/1360867894 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f34018680 con 0x7f1f44107e10 2024-05-09T00:45:12.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.209+0000 7f1f4c646700 1 -- 172.21.15.5:0/1360867894 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f441ade90 con 0x7f1f44107e10 2024-05-09T00:45:12.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.210+0000 7f1f4c646700 1 -- 172.21.15.5:0/1360867894 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1f44105c80 con 0x7f1f44107e10 2024-05-09T00:45:12.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.212+0000 7f1f3b7fe700 1 -- 172.21.15.5:0/1360867894 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f1f34010430 con 0x7f1f44107e10 2024-05-09T00:45:12.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.213+0000 7f1f3b7fe700 1 --2- 172.21.15.5:0/1360867894 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f1f300b2990 0x7f1f300b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:12.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.213+0000 7f1f4a3e2700 1 --2- 172.21.15.5:0/1360867894 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f1f300b2990 0x7f1f300b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:12.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.213+0000 7f1f3b7fe700 1 -- 172.21.15.5:0/1360867894 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8227+0+0 (secure 0 0 0) 0x7f1f340e2a80 con 0x7f1f44107e10 2024-05-09T00:45:12.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.213+0000 7f1f3b7fe700 1 --2- 172.21.15.5:0/1360867894 >> [v2:172.21.15.157:6824/3261151188,v1:172.21.15.157:6825/3261151188] conn(0x7f1f300bc520 0x7f1f300bea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:12.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.213+0000 7f1f3b7fe700 1 -- 172.21.15.5:0/1360867894 --> [v2:172.21.15.157:6824/3261151188,v1:172.21.15.157:6825/3261151188] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1f300bf0e0 con 0x7f1f300bc520 2024-05-09T00:45:12.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.213+0000 7f1f3b7fe700 1 -- 172.21.15.5:0/1360867894 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f1f340e2e30 con 0x7f1f44107e10 2024-05-09T00:45:12.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.214+0000 7f1f4abe3700 1 --2- 172.21.15.5:0/1360867894 >> [v2:172.21.15.157:6824/3261151188,v1:172.21.15.157:6825/3261151188] conn(0x7f1f300bc520 0x7f1f300bea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:12.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.214+0000 7f1f4a3e2700 1 --2- 172.21.15.5:0/1360867894 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f1f300b2990 0x7f1f300b4ea0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1f40009c60 tx=0x7f1f40009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:12.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.214+0000 7f1f4abe3700 1 --2- 172.21.15.5:0/1360867894 >> [v2:172.21.15.157:6824/3261151188,v1:172.21.15.157:6825/3261151188] conn(0x7f1f300bc520 0x7f1f300bea10 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:12.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.215+0000 7f1f3b7fe700 1 -- 172.21.15.5:0/1360867894 <== osd.11 v2:172.21.15.157:6824/3261151188 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1f300bf0e0 con 0x7f1f300bc520 2024-05-09T00:45:12.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.232+0000 7f1f4c646700 1 -- 172.21.15.5:0/1360867894 --> [v2:172.21.15.157:6824/3261151188,v1:172.21.15.157:6825/3261151188] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1f4410e3e0 con 0x7f1f300bc520 2024-05-09T00:45:12.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.232+0000 7f1f3b7fe700 1 -- 172.21.15.5:0/1360867894 <== osd.11 v2:172.21.15.157:6824/3261151188 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1f3c009360 con 0x7f1f300bc520 2024-05-09T00:45:12.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.233+0000 7f1f4c646700 1 -- 172.21.15.5:0/1360867894 >> [v2:172.21.15.157:6824/3261151188,v1:172.21.15.157:6825/3261151188] conn(0x7f1f300bc520 msgr2=0x7f1f300bea10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:12.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.233+0000 7f1f4c646700 1 --2- 172.21.15.5:0/1360867894 >> [v2:172.21.15.157:6824/3261151188,v1:172.21.15.157:6825/3261151188] conn(0x7f1f300bc520 0x7f1f300bea10 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:12.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.233+0000 7f1f4c646700 1 -- 172.21.15.5:0/1360867894 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f1f300b2990 msgr2=0x7f1f300b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:12.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.233+0000 7f1f4c646700 1 --2- 172.21.15.5:0/1360867894 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f1f300b2990 0x7f1f300b4ea0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1f40009c60 tx=0x7f1f40009260 comp rx=0 tx=0).stop 2024-05-09T00:45:12.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.233+0000 7f1f4c646700 1 -- 172.21.15.5:0/1360867894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f44107e10 msgr2=0x7f1f4410fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:12.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.233+0000 7f1f4c646700 1 --2- 172.21.15.5:0/1360867894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f44107e10 0x7f1f4410fd80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1f3400e9c0 tx=0x7f1f3400ecd0 comp rx=0 tx=0).stop 2024-05-09T00:45:12.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.233+0000 7f1f4abe3700 1 -- 172.21.15.5:0/1360867894 reap_dead start 2024-05-09T00:45:12.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.233+0000 7f1f4c646700 1 -- 172.21.15.5:0/1360867894 shutdown_connections 2024-05-09T00:45:12.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.233+0000 7f1f4c646700 1 -- 172.21.15.5:0/1360867894 >> 172.21.15.5:0/1360867894 conn(0x7f1f440fef30 msgr2=0x7f1f441052a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:45:12.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.233+0000 7f1f4c646700 1 -- 172.21.15.5:0/1360867894 shutdown_connections 2024-05-09T00:45:12.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.233+0000 7f1f4c646700 1 -- 172.21.15.5:0/1360867894 wait complete. 2024-05-09T00:45:12.664 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:45:12.664 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:45:12.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.885+0000 7f6897960700 1 -- 172.21.15.5:0/1800755474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68901045d0 msgr2=0x7f68901049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:12.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.885+0000 7f6897960700 1 --2- 172.21.15.5:0/1800755474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68901045d0 0x7f68901049e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f688c0099c0 tx=0x7f688c009cd0 comp rx=0 tx=0).stop 2024-05-09T00:45:12.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.885+0000 7f6897960700 1 -- 172.21.15.5:0/1800755474 shutdown_connections 2024-05-09T00:45:12.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.885+0000 7f6897960700 1 --2- 172.21.15.5:0/1800755474 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6890109850 0x7f68901111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:12.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.885+0000 7f6897960700 1 --2- 172.21.15.5:0/1800755474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68901045d0 0x7f68901049e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:12.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.885+0000 7f6897960700 1 --2- 172.21.15.5:0/1800755474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6890108f00 0x7f6890109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:12.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.885+0000 7f6897960700 1 -- 172.21.15.5:0/1800755474 >> 172.21.15.5:0/1800755474 conn(0x7f68900ffea0 msgr2=0x7f68901022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:45:12.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.885+0000 7f6897960700 1 -- 172.21.15.5:0/1800755474 shutdown_connections 2024-05-09T00:45:12.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.885+0000 7f6897960700 1 -- 172.21.15.5:0/1800755474 wait complete. 2024-05-09T00:45:12.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.886+0000 7f6897960700 1 Processor -- start 2024-05-09T00:45:12.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.886+0000 7f6897960700 1 -- start start 2024-05-09T00:45:12.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.886+0000 7f6897960700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68901045d0 0x7f689019ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:12.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.886+0000 7f6897960700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6890108f00 0x7f689019b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:12.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.887+0000 7f6897960700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6890109850 0x7f689019f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:12.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.887+0000 7f6897960700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6890114480 con 0x7f68901045d0 2024-05-09T00:45:12.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.887+0000 7f6897960700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6890114300 con 0x7f6890109850 2024-05-09T00:45:12.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.887+0000 7f6897960700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6890114600 con 0x7f6890108f00 2024-05-09T00:45:12.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.887+0000 7f68956fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68901045d0 0x7f689019ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:12.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.887+0000 7f68956fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68901045d0 0x7f689019ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37394/0 (socket says 172.21.15.5:37394) 2024-05-09T00:45:12.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.887+0000 7f68956fc700 1 -- 172.21.15.5:0/795931437 learned_addr learned my addr 172.21.15.5:0/795931437 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:45:12.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.887+0000 7f6894efb700 1 --2- 172.21.15.5:0/795931437 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6890108f00 0x7f689019b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:12.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.887+0000 7f6895efd700 1 --2- 172.21.15.5:0/795931437 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6890109850 0x7f689019f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:12.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.887+0000 7f6894efb700 1 -- 172.21.15.5:0/795931437 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6890109850 msgr2=0x7f689019f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:12.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.887+0000 7f6894efb700 1 --2- 172.21.15.5:0/795931437 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6890109850 0x7f689019f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:12.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.887+0000 7f6894efb700 1 -- 172.21.15.5:0/795931437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68901045d0 msgr2=0x7f689019ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:12.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.887+0000 7f6894efb700 1 --2- 172.21.15.5:0/795931437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68901045d0 0x7f689019ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:12.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.887+0000 7f6894efb700 1 -- 172.21.15.5:0/795931437 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f689019fce0 con 0x7f6890108f00 2024-05-09T00:45:12.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.887+0000 7f68956fc700 1 --2- 172.21.15.5:0/795931437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68901045d0 0x7f689019ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:45:12.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.887+0000 7f6895efd700 1 --2- 172.21.15.5:0/795931437 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6890109850 0x7f689019f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:45:12.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.887+0000 7f6894efb700 1 --2- 172.21.15.5:0/795931437 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6890108f00 0x7f689019b3b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f688000e9c0 tx=0x7f688000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:12.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.888+0000 7f68867fc700 1 -- 172.21.15.5:0/795931437 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688000cbd0 con 0x7f6890108f00 2024-05-09T00:45:12.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.888+0000 7f6897960700 1 -- 172.21.15.5:0/795931437 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f689019ffd0 con 0x7f6890108f00 2024-05-09T00:45:12.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.888+0000 7f68867fc700 1 -- 172.21.15.5:0/795931437 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f688000cd70 con 0x7f6890108f00 2024-05-09T00:45:12.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.888+0000 7f68867fc700 1 -- 172.21.15.5:0/795931437 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6880018680 con 0x7f6890108f00 2024-05-09T00:45:12.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.888+0000 7f6897960700 1 -- 172.21.15.5:0/795931437 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68901a78f0 con 0x7f6890108f00 2024-05-09T00:45:12.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.888+0000 7f6897960700 1 -- 172.21.15.5:0/795931437 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6890063f50 con 0x7f6890108f00 2024-05-09T00:45:12.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.891+0000 7f68867fc700 1 -- 172.21.15.5:0/795931437 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f6880010430 con 0x7f6890108f00 2024-05-09T00:45:12.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.891+0000 7f68867fc700 1 --2- 172.21.15.5:0/795931437 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f687c0b2910 0x7f687c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:12.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.891+0000 7f68956fc700 1 --2- 172.21.15.5:0/795931437 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f687c0b2910 0x7f687c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:12.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.892+0000 7f68867fc700 1 -- 172.21.15.5:0/795931437 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8227+0+0 (secure 0 0 0) 0x7f68800e3070 con 0x7f6890108f00 2024-05-09T00:45:12.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.892+0000 7f68867fc700 1 --2- 172.21.15.5:0/795931437 >> [v2:172.21.15.5:6810/2357377277,v1:172.21.15.5:6811/2357377277] conn(0x7f687c0bc4a0 0x7f687c0be990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:12.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.892+0000 7f68867fc700 1 -- 172.21.15.5:0/795931437 --> [v2:172.21.15.5:6810/2357377277,v1:172.21.15.5:6811/2357377277] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f687c0bf060 con 0x7f687c0bc4a0 2024-05-09T00:45:12.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.892+0000 7f68867fc700 1 -- 172.21.15.5:0/795931437 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f68800e34c0 con 0x7f6890108f00 2024-05-09T00:45:12.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.892+0000 7f6895efd700 1 --2- 172.21.15.5:0/795931437 >> [v2:172.21.15.5:6810/2357377277,v1:172.21.15.5:6811/2357377277] conn(0x7f687c0bc4a0 0x7f687c0be990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:12.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.892+0000 7f68956fc700 1 --2- 172.21.15.5:0/795931437 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f687c0b2910 0x7f687c0b4e20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f688c009990 tx=0x7f688c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:12.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.892+0000 7f6895efd700 1 --2- 172.21.15.5:0/795931437 >> [v2:172.21.15.5:6810/2357377277,v1:172.21.15.5:6811/2357377277] conn(0x7f687c0bc4a0 0x7f687c0be990 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:12.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.893+0000 7f68867fc700 1 -- 172.21.15.5:0/795931437 <== osd.3 v2:172.21.15.5:6810/2357377277 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f687c0bf060 con 0x7f687c0bc4a0 2024-05-09T00:45:12.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.909+0000 7f6897960700 1 -- 172.21.15.5:0/795931437 --> [v2:172.21.15.5:6810/2357377277,v1:172.21.15.5:6811/2357377277] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6890064350 con 0x7f687c0bc4a0 2024-05-09T00:45:12.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.910+0000 7f68867fc700 1 -- 172.21.15.5:0/795931437 <== osd.3 v2:172.21.15.5:6810/2357377277 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6890064350 con 0x7f687c0bc4a0 2024-05-09T00:45:12.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.910+0000 7f6897960700 1 -- 172.21.15.5:0/795931437 >> [v2:172.21.15.5:6810/2357377277,v1:172.21.15.5:6811/2357377277] conn(0x7f687c0bc4a0 msgr2=0x7f687c0be990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:12.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.910+0000 7f6897960700 1 --2- 172.21.15.5:0/795931437 >> [v2:172.21.15.5:6810/2357377277,v1:172.21.15.5:6811/2357377277] conn(0x7f687c0bc4a0 0x7f687c0be990 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:12.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.910+0000 7f6897960700 1 -- 172.21.15.5:0/795931437 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f687c0b2910 msgr2=0x7f687c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:12.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.910+0000 7f6897960700 1 --2- 172.21.15.5:0/795931437 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f687c0b2910 0x7f687c0b4e20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f688c009990 tx=0x7f688c01a040 comp rx=0 tx=0).stop 2024-05-09T00:45:12.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.910+0000 7f6897960700 1 -- 172.21.15.5:0/795931437 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6890108f00 msgr2=0x7f689019b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:12.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.910+0000 7f6897960700 1 --2- 172.21.15.5:0/795931437 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6890108f00 0x7f689019b3b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f688000e9c0 tx=0x7f688000ecd0 comp rx=0 tx=0).stop 2024-05-09T00:45:12.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.910+0000 7f6895efd700 1 -- 172.21.15.5:0/795931437 reap_dead start 2024-05-09T00:45:12.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.910+0000 7f6897960700 1 -- 172.21.15.5:0/795931437 shutdown_connections 2024-05-09T00:45:12.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.910+0000 7f6897960700 1 -- 172.21.15.5:0/795931437 >> 172.21.15.5:0/795931437 conn(0x7f68900ffea0 msgr2=0x7f6890102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:45:12.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.910+0000 7f6897960700 1 -- 172.21.15.5:0/795931437 shutdown_connections 2024-05-09T00:45:12.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:12.910+0000 7f6897960700 1 -- 172.21.15.5:0/795931437 wait complete. 2024-05-09T00:45:13.040 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:45:12 smithi005 ceph-mon[30365]: pgmap v335: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T00:45:13.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:45:12 smithi131 ceph-mon[34547]: pgmap v335: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T00:45:13.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:45:12 smithi157 ceph-mon[34294]: pgmap v335: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T00:45:13.868 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:45:13.880 INFO:teuthology.orchestra.run.smithi005.stdout:107374182475 2024-05-09T00:45:13.880 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-09T00:45:15.139 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:45:14 smithi005 ceph-mon[30365]: pgmap v336: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-05-09T00:45:15.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:45:14 smithi131 ceph-mon[34547]: pgmap v336: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-05-09T00:45:15.365 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:45:15.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:45:14 smithi157 ceph-mon[34294]: pgmap v336: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-05-09T00:45:16.875 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:45:16 smithi005 ceph-mon[30365]: pgmap v337: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-09T00:45:16.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:16.905+0000 7f71d7343700 1 -- 172.21.15.5:0/1841874498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71d01034e0 msgr2=0x7f71d01038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:16.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:16.905+0000 7f71d7343700 1 --2- 172.21.15.5:0/1841874498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71d01034e0 0x7f71d01038f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f71cc00b0d0 tx=0x7f71cc00b3e0 comp rx=0 tx=0).stop 2024-05-09T00:45:16.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:16.905+0000 7f71d7343700 1 -- 172.21.15.5:0/1841874498 shutdown_connections 2024-05-09T00:45:16.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:16.905+0000 7f71d7343700 1 --2- 172.21.15.5:0/1841874498 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71d0108760 0x7f71d01100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:16.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:16.905+0000 7f71d7343700 1 --2- 172.21.15.5:0/1841874498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71d01034e0 0x7f71d01038f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:16.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:16.905+0000 7f71d7343700 1 --2- 172.21.15.5:0/1841874498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71d0107e10 0x7f71d0108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:16.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:16.905+0000 7f71d7343700 1 -- 172.21.15.5:0/1841874498 >> 172.21.15.5:0/1841874498 conn(0x7f71d00fef50 msgr2=0x7f71d0101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:45:16.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:16.905+0000 7f71d7343700 1 -- 172.21.15.5:0/1841874498 shutdown_connections 2024-05-09T00:45:16.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:16.906+0000 7f71d7343700 1 -- 172.21.15.5:0/1841874498 wait complete. 2024-05-09T00:45:16.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:16.906+0000 7f71d7343700 1 Processor -- start 2024-05-09T00:45:16.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:16.906+0000 7f71d7343700 1 -- start start 2024-05-09T00:45:16.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:16.907+0000 7f71d7343700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71d01034e0 0x7f71d0199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:16.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:16.907+0000 7f71d7343700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71d0107e10 0x7f71d019a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:16.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:16.907+0000 7f71d7343700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71d0108760 0x7f71d019a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:16.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:16.907+0000 7f71d7343700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f71d0113410 con 0x7f71d01034e0 2024-05-09T00:45:16.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:16.907+0000 7f71d7343700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f71d0113290 con 0x7f71d0107e10 2024-05-09T00:45:16.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:16.907+0000 7f71d7343700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f71d0113590 con 0x7f71d0108760 2024-05-09T00:45:16.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:16.907+0000 7f71d50df700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71d01034e0 0x7f71d0199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:16.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:16.907+0000 7f71d50df700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71d01034e0 0x7f71d0199ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46142/0 (socket says 172.21.15.5:46142) 2024-05-09T00:45:16.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:16.907+0000 7f71d50df700 1 -- 172.21.15.5:0/4231558874 learned_addr learned my addr 172.21.15.5:0/4231558874 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:45:16.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:16.907+0000 7f71d58e0700 1 --2- 172.21.15.5:0/4231558874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71d0108760 0x7f71d019a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:16.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:16.907+0000 7f71d48de700 1 --2- 172.21.15.5:0/4231558874 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71d0107e10 0x7f71d019a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:16.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:16.907+0000 7f71d50df700 1 -- 172.21.15.5:0/4231558874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71d0108760 msgr2=0x7f71d019a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:16.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:16.907+0000 7f71d50df700 1 --2- 172.21.15.5:0/4231558874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71d0108760 0x7f71d019a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:16.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:16.907+0000 7f71d50df700 1 -- 172.21.15.5:0/4231558874 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71d0107e10 msgr2=0x7f71d019a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:16.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:16.907+0000 7f71d50df700 1 --2- 172.21.15.5:0/4231558874 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71d0107e10 0x7f71d019a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:16.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:16.907+0000 7f71d50df700 1 -- 172.21.15.5:0/4231558874 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71d01a91e0 con 0x7f71d01034e0 2024-05-09T00:45:16.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:16.907+0000 7f71d50df700 1 --2- 172.21.15.5:0/4231558874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71d01034e0 0x7f71d0199ae0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f71cc00d010 tx=0x7f71cc007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:16.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:16.908+0000 7f71c67fc700 1 -- 172.21.15.5:0/4231558874 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71cc01f070 con 0x7f71d01034e0 2024-05-09T00:45:16.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:16.908+0000 7f71d7343700 1 -- 172.21.15.5:0/4231558874 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71d01a9470 con 0x7f71d01034e0 2024-05-09T00:45:16.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:16.908+0000 7f71c67fc700 1 -- 172.21.15.5:0/4231558874 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71cc007e50 con 0x7f71d01034e0 2024-05-09T00:45:16.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:16.908+0000 7f71c67fc700 1 -- 172.21.15.5:0/4231558874 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71cc01b4a0 con 0x7f71d01034e0 2024-05-09T00:45:16.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:16.908+0000 7f71d7343700 1 -- 172.21.15.5:0/4231558874 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71d01a9920 con 0x7f71d01034e0 2024-05-09T00:45:16.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:16.909+0000 7f71d7343700 1 -- 172.21.15.5:0/4231558874 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f71d0063f50 con 0x7f71d01034e0 2024-05-09T00:45:16.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:16.911+0000 7f71c67fc700 1 -- 172.21.15.5:0/4231558874 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f71cc022070 con 0x7f71d01034e0 2024-05-09T00:45:16.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:16.911+0000 7f71c67fc700 1 --2- 172.21.15.5:0/4231558874 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f71bc0b2990 0x7f71bc0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:16.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:16.912+0000 7f71d48de700 1 --2- 172.21.15.5:0/4231558874 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f71bc0b2990 0x7f71bc0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:16.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:16.912+0000 7f71c67fc700 1 -- 172.21.15.5:0/4231558874 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8227+0+0 (secure 0 0 0) 0x7f71cc00f020 con 0x7f71d01034e0 2024-05-09T00:45:16.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:16.912+0000 7f71c67fc700 1 --2- 172.21.15.5:0/4231558874 >> [v2:172.21.15.131:6824/1571783985,v1:172.21.15.131:6825/1571783985] conn(0x7f71bc0bc520 0x7f71bc0bea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:16.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:16.912+0000 7f71c67fc700 1 -- 172.21.15.5:0/4231558874 --> [v2:172.21.15.131:6824/1571783985,v1:172.21.15.131:6825/1571783985] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f71cc02de50 con 0x7f71bc0bc520 2024-05-09T00:45:16.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:16.912+0000 7f71c67fc700 1 -- 172.21.15.5:0/4231558874 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f71cc0e3ac0 con 0x7f71d01034e0 2024-05-09T00:45:16.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:16.912+0000 7f71d48de700 1 --2- 172.21.15.5:0/4231558874 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f71bc0b2990 0x7f71bc0b4ea0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f71d01066c0 tx=0x7f71c000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:16.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:16.912+0000 7f71d58e0700 1 --2- 172.21.15.5:0/4231558874 >> [v2:172.21.15.131:6824/1571783985,v1:172.21.15.131:6825/1571783985] conn(0x7f71bc0bc520 0x7f71bc0bea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:16.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:16.913+0000 7f71d58e0700 1 --2- 172.21.15.5:0/4231558874 >> [v2:172.21.15.131:6824/1571783985,v1:172.21.15.131:6825/1571783985] conn(0x7f71bc0bc520 0x7f71bc0bea10 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:16.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:16.913+0000 7f71c67fc700 1 -- 172.21.15.5:0/4231558874 <== osd.10 v2:172.21.15.131:6824/1571783985 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f71cc02de50 con 0x7f71bc0bc520 2024-05-09T00:45:16.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:16.930+0000 7f71d7343700 1 -- 172.21.15.5:0/4231558874 --> [v2:172.21.15.131:6824/1571783985,v1:172.21.15.131:6825/1571783985] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f71d0064350 con 0x7f71bc0bc520 2024-05-09T00:45:16.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:16.931+0000 7f71c67fc700 1 -- 172.21.15.5:0/4231558874 <== osd.10 v2:172.21.15.131:6824/1571783985 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f71d0064350 con 0x7f71bc0bc520 2024-05-09T00:45:16.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:16.931+0000 7f71d7343700 1 -- 172.21.15.5:0/4231558874 >> [v2:172.21.15.131:6824/1571783985,v1:172.21.15.131:6825/1571783985] conn(0x7f71bc0bc520 msgr2=0x7f71bc0bea10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:16.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:16.931+0000 7f71d7343700 1 --2- 172.21.15.5:0/4231558874 >> [v2:172.21.15.131:6824/1571783985,v1:172.21.15.131:6825/1571783985] conn(0x7f71bc0bc520 0x7f71bc0bea10 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:16.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:16.932+0000 7f71d7343700 1 -- 172.21.15.5:0/4231558874 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f71bc0b2990 msgr2=0x7f71bc0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:16.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:16.932+0000 7f71d7343700 1 --2- 172.21.15.5:0/4231558874 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f71bc0b2990 0x7f71bc0b4ea0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f71d01066c0 tx=0x7f71c000b410 comp rx=0 tx=0).stop 2024-05-09T00:45:16.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:16.932+0000 7f71d7343700 1 -- 172.21.15.5:0/4231558874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71d01034e0 msgr2=0x7f71d0199ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:16.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:16.932+0000 7f71d7343700 1 --2- 172.21.15.5:0/4231558874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71d01034e0 0x7f71d0199ae0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f71cc00d010 tx=0x7f71cc007a20 comp rx=0 tx=0).stop 2024-05-09T00:45:16.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:16.932+0000 7f71d58e0700 1 -- 172.21.15.5:0/4231558874 reap_dead start 2024-05-09T00:45:16.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:16.932+0000 7f71d7343700 1 -- 172.21.15.5:0/4231558874 shutdown_connections 2024-05-09T00:45:16.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:16.932+0000 7f71d7343700 1 -- 172.21.15.5:0/4231558874 >> 172.21.15.5:0/4231558874 conn(0x7f71d00fef50 msgr2=0x7f71d0107120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:45:16.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:16.932+0000 7f71d7343700 1 -- 172.21.15.5:0/4231558874 shutdown_connections 2024-05-09T00:45:16.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:16.932+0000 7f71d7343700 1 -- 172.21.15.5:0/4231558874 wait complete. 2024-05-09T00:45:17.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:45:16 smithi131 ceph-mon[34547]: pgmap v337: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-09T00:45:17.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:45:16 smithi157 ceph-mon[34294]: pgmap v337: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-09T00:45:18.078 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:45:18.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:18.297+0000 7fc9b8c06700 1 -- 172.21.15.5:0/3889036395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9b41034c0 msgr2=0x7fc9b41038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:18.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:18.297+0000 7fc9b8c06700 1 --2- 172.21.15.5:0/3889036395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9b41034c0 0x7fc9b41038d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc9a40099c0 tx=0x7fc9a4009cd0 comp rx=0 tx=0).stop 2024-05-09T00:45:18.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:18.297+0000 7fc9b8c06700 1 -- 172.21.15.5:0/3889036395 shutdown_connections 2024-05-09T00:45:18.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:18.297+0000 7fc9b8c06700 1 --2- 172.21.15.5:0/3889036395 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc9b4108740 0x7fc9b41100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:18.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:18.297+0000 7fc9b8c06700 1 --2- 172.21.15.5:0/3889036395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9b41034c0 0x7fc9b41038d0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:18.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:18.297+0000 7fc9b8c06700 1 --2- 172.21.15.5:0/3889036395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9b4107df0 0x7fc9b4108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:18.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:18.297+0000 7fc9b8c06700 1 -- 172.21.15.5:0/3889036395 >> 172.21.15.5:0/3889036395 conn(0x7fc9b40fedc0 msgr2=0x7fc9b41011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:45:18.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:18.297+0000 7fc9b8c06700 1 -- 172.21.15.5:0/3889036395 shutdown_connections 2024-05-09T00:45:18.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:18.297+0000 7fc9b8c06700 1 -- 172.21.15.5:0/3889036395 wait complete. 2024-05-09T00:45:18.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:18.299+0000 7fc9b8c06700 1 Processor -- start 2024-05-09T00:45:18.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:18.299+0000 7fc9b8c06700 1 -- start start 2024-05-09T00:45:18.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:18.299+0000 7fc9b8c06700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9b41034c0 0x7fc9b4199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:18.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:18.299+0000 7fc9b8c06700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9b4107df0 0x7fc9b419a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:18.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:18.299+0000 7fc9b8c06700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc9b4108740 0x7fc9b419e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:18.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:18.299+0000 7fc9b8c06700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b41133c0 con 0x7fc9b4107df0 2024-05-09T00:45:18.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:18.299+0000 7fc9b8c06700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b4113240 con 0x7fc9b4108740 2024-05-09T00:45:18.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:18.299+0000 7fc9b8c06700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b4113540 con 0x7fc9b41034c0 2024-05-09T00:45:18.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:18.299+0000 7fc9b1d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9b4107df0 0x7fc9b419a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:18.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:18.299+0000 7fc9b1d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9b4107df0 0x7fc9b419a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46172/0 (socket says 172.21.15.5:46172) 2024-05-09T00:45:18.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:18.299+0000 7fc9b1d9b700 1 -- 172.21.15.5:0/2587549530 learned_addr learned my addr 172.21.15.5:0/2587549530 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:45:18.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:18.299+0000 7fc9b2d9d700 1 --2- 172.21.15.5:0/2587549530 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc9b4108740 0x7fc9b419e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:18.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:18.299+0000 7fc9b259c700 1 --2- 172.21.15.5:0/2587549530 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9b41034c0 0x7fc9b4199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:18.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:18.300+0000 7fc9b1d9b700 1 -- 172.21.15.5:0/2587549530 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9b41034c0 msgr2=0x7fc9b4199db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:18.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:18.300+0000 7fc9b1d9b700 1 --2- 172.21.15.5:0/2587549530 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9b41034c0 0x7fc9b4199db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:18.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:18.300+0000 7fc9b1d9b700 1 -- 172.21.15.5:0/2587549530 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc9b4108740 msgr2=0x7fc9b419e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:18.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:18.300+0000 7fc9b1d9b700 1 --2- 172.21.15.5:0/2587549530 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc9b4108740 0x7fc9b419e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:18.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:18.300+0000 7fc9b1d9b700 1 -- 172.21.15.5:0/2587549530 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9b419ec80 con 0x7fc9b4107df0 2024-05-09T00:45:18.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:18.300+0000 7fc9b2d9d700 1 --2- 172.21.15.5:0/2587549530 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc9b4108740 0x7fc9b419e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:45:18.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:18.300+0000 7fc9b1d9b700 1 --2- 172.21.15.5:0/2587549530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9b4107df0 0x7fc9b419a350 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc9a800c8f0 tx=0x7fc9a800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:18.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:18.300+0000 7fc9a37fe700 1 -- 172.21.15.5:0/2587549530 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a8007b00 con 0x7fc9b4107df0 2024-05-09T00:45:18.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:18.300+0000 7fc9b8c06700 1 -- 172.21.15.5:0/2587549530 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9b419ef70 con 0x7fc9b4107df0 2024-05-09T00:45:18.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:18.300+0000 7fc9a37fe700 1 -- 172.21.15.5:0/2587549530 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9a8007ca0 con 0x7fc9b4107df0 2024-05-09T00:45:18.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:18.300+0000 7fc9a37fe700 1 -- 172.21.15.5:0/2587549530 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a8018660 con 0x7fc9b4107df0 2024-05-09T00:45:18.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:18.300+0000 7fc9b8c06700 1 -- 172.21.15.5:0/2587549530 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9b406b610 con 0x7fc9b4107df0 2024-05-09T00:45:18.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:18.301+0000 7fc9b8c06700 1 -- 172.21.15.5:0/2587549530 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9b4104970 con 0x7fc9b4107df0 2024-05-09T00:45:18.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:18.303+0000 7fc9a37fe700 1 -- 172.21.15.5:0/2587549530 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fc9a800f450 con 0x7fc9b4107df0 2024-05-09T00:45:18.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:18.304+0000 7fc9a37fe700 1 --2- 172.21.15.5:0/2587549530 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fc99c0b2910 0x7fc99c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:18.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:18.304+0000 7fc9b259c700 1 --2- 172.21.15.5:0/2587549530 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fc99c0b2910 0x7fc99c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:18.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:18.304+0000 7fc9a37fe700 1 -- 172.21.15.5:0/2587549530 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8227+0+0 (secure 0 0 0) 0x7fc9a80e2480 con 0x7fc9b4107df0 2024-05-09T00:45:18.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:18.304+0000 7fc9b259c700 1 --2- 172.21.15.5:0/2587549530 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fc99c0b2910 0x7fc99c0b4e20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc9a4003800 tx=0x7fc9a401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:18.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:18.308+0000 7fc9a37fe700 1 -- 172.21.15.5:0/2587549530 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc9a8014030 con 0x7fc9b4107df0 2024-05-09T00:45:18.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:18.527+0000 7fc9b8c06700 1 -- 172.21.15.5:0/2587549530 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fc9b40512c0 con 0x7fc9b4107df0 2024-05-09T00:45:18.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:18.527+0000 7fc9a37fe700 1 -- 172.21.15.5:0/2587549530 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc9a80aa0d0 con 0x7fc9b4107df0 2024-05-09T00:45:18.528 INFO:teuthology.orchestra.run.smithi005.stdout:77309411409 2024-05-09T00:45:18.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:18.530+0000 7fc9b8c06700 1 -- 172.21.15.5:0/2587549530 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fc99c0b2910 msgr2=0x7fc99c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:18.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:18.530+0000 7fc9b8c06700 1 --2- 172.21.15.5:0/2587549530 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fc99c0b2910 0x7fc99c0b4e20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc9a4003800 tx=0x7fc9a401a040 comp rx=0 tx=0).stop 2024-05-09T00:45:18.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:18.530+0000 7fc9b8c06700 1 -- 172.21.15.5:0/2587549530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9b4107df0 msgr2=0x7fc9b419a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:18.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:18.530+0000 7fc9b8c06700 1 --2- 172.21.15.5:0/2587549530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9b4107df0 0x7fc9b419a350 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc9a800c8f0 tx=0x7fc9a800ccb0 comp rx=0 tx=0).stop 2024-05-09T00:45:18.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:18.530+0000 7fc9b8c06700 1 -- 172.21.15.5:0/2587549530 shutdown_connections 2024-05-09T00:45:18.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:18.530+0000 7fc9b8c06700 1 --2- 172.21.15.5:0/2587549530 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fc99c0b2910 0x7fc99c0b4e20 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:18.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:18.530+0000 7fc9b8c06700 1 --2- 172.21.15.5:0/2587549530 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc9b4108740 0x7fc9b419e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:18.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:18.530+0000 7fc9b8c06700 1 --2- 172.21.15.5:0/2587549530 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9b41034c0 0x7fc9b4199db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:18.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:18.530+0000 7fc9b8c06700 1 --2- 172.21.15.5:0/2587549530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9b4107df0 0x7fc9b419a350 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:18.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:18.530+0000 7fc9b8c06700 1 -- 172.21.15.5:0/2587549530 >> 172.21.15.5:0/2587549530 conn(0x7fc9b40fedc0 msgr2=0x7fc9b41096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:45:18.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:18.530+0000 7fc9b8c06700 1 -- 172.21.15.5:0/2587549530 shutdown_connections 2024-05-09T00:45:18.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:18.530+0000 7fc9b8c06700 1 -- 172.21.15.5:0/2587549530 wait complete. 2024-05-09T00:45:19.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:45:18 smithi157 ceph-mon[34294]: pgmap v338: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T00:45:19.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:45:18 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/2587549530' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-09T00:45:19.279 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:45:18 smithi005 ceph-mon[30365]: pgmap v338: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T00:45:19.279 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:45:18 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/2587549530' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-09T00:45:19.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:45:18 smithi131 ceph-mon[34547]: pgmap v338: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T00:45:19.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:45:18 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/2587549530' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-09T00:45:20.649 INFO:teuthology.orchestra.run.smithi005.stdout:274877906984 2024-05-09T00:45:20.650 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-05-09T00:45:20.656 INFO:teuthology.orchestra.run.smithi005.stdout:300647710753 2024-05-09T00:45:20.656 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-05-09T00:45:21.058 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:45:21.059 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:45:21.059 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:45:21.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:45:20 smithi157 ceph-mon[34294]: pgmap v339: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-09T00:45:21.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:45:20 smithi005 ceph-mon[30365]: pgmap v339: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-09T00:45:21.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:45:20 smithi131 ceph-mon[34547]: pgmap v339: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-09T00:45:22.076 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:45:22.878 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411405 got 77309411409 for osd.2 2024-05-09T00:45:22.878 DEBUG:teuthology.parallel:result is None 2024-05-09T00:45:23.164 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:45:22 smithi005 ceph-mon[30365]: pgmap v340: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T00:45:23.164 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:45:22 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-05-09T00:45:23.164 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:45:22 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-05-09T00:45:23.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:45:22 smithi157 ceph-mon[34294]: pgmap v340: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T00:45:23.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:45:22 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-05-09T00:45:23.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:45:22 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-05-09T00:45:23.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:45:22 smithi131 ceph-mon[34547]: pgmap v340: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T00:45:23.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:45:22 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-05-09T00:45:23.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:45:22 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-05-09T00:45:23.700 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:45:23.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.713+0000 7f7cf07be700 1 -- 172.21.15.5:0/3546394848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ce81034e0 msgr2=0x7f7ce81038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:23.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.713+0000 7f7cf07be700 1 --2- 172.21.15.5:0/3546394848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ce81034e0 0x7f7ce81038f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7ce40099c0 tx=0x7f7ce4009cd0 comp rx=0 tx=0).stop 2024-05-09T00:45:23.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.713+0000 7f7cf07be700 1 -- 172.21.15.5:0/3546394848 shutdown_connections 2024-05-09T00:45:23.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.713+0000 7f7cf07be700 1 --2- 172.21.15.5:0/3546394848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ce8108760 0x7f7ce81100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:23.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.713+0000 7f7cf07be700 1 --2- 172.21.15.5:0/3546394848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ce81034e0 0x7f7ce81038f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:23.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.713+0000 7f7cf07be700 1 --2- 172.21.15.5:0/3546394848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ce8107e10 0x7f7ce8108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:23.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.713+0000 7f7cf07be700 1 -- 172.21.15.5:0/3546394848 >> 172.21.15.5:0/3546394848 conn(0x7f7ce80fef30 msgr2=0x7f7ce8101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:45:23.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.713+0000 7f7cf07be700 1 -- 172.21.15.5:0/3546394848 shutdown_connections 2024-05-09T00:45:23.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.713+0000 7f7cf07be700 1 -- 172.21.15.5:0/3546394848 wait complete. 2024-05-09T00:45:23.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.714+0000 7f7cf07be700 1 Processor -- start 2024-05-09T00:45:23.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.714+0000 7f7cf07be700 1 -- start start 2024-05-09T00:45:23.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.714+0000 7f7cf07be700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ce81034e0 0x7f7ce819bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:23.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.714+0000 7f7cf07be700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ce8107e10 0x7f7ce819c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:23.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.714+0000 7f7cf07be700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ce8108760 0x7f7ce819c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:23.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.714+0000 7f7cf07be700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce8113390 con 0x7f7ce8107e10 2024-05-09T00:45:23.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.714+0000 7f7cf07be700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce8113210 con 0x7f7ce8108760 2024-05-09T00:45:23.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.714+0000 7f7cf07be700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce8113510 con 0x7f7ce81034e0 2024-05-09T00:45:23.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.715+0000 7f7cedd59700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ce8107e10 0x7f7ce819c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:23.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.715+0000 7f7cedd59700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ce8107e10 0x7f7ce819c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39530/0 (socket says 172.21.15.5:39530) 2024-05-09T00:45:23.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.715+0000 7f7cedd59700 1 -- 172.21.15.5:0/1045888793 learned_addr learned my addr 172.21.15.5:0/1045888793 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:45:23.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.715+0000 7f7cee55a700 1 --2- 172.21.15.5:0/1045888793 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ce81034e0 0x7f7ce819bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:23.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.715+0000 7f7ceed5b700 1 --2- 172.21.15.5:0/1045888793 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ce8108760 0x7f7ce819c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:23.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.715+0000 7f7cedd59700 1 -- 172.21.15.5:0/1045888793 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ce81034e0 msgr2=0x7f7ce819bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:23.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.715+0000 7f7cedd59700 1 --2- 172.21.15.5:0/1045888793 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ce81034e0 0x7f7ce819bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:23.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.715+0000 7f7cedd59700 1 -- 172.21.15.5:0/1045888793 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ce8108760 msgr2=0x7f7ce819c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:23.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.715+0000 7f7cedd59700 1 --2- 172.21.15.5:0/1045888793 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ce8108760 0x7f7ce819c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:23.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.715+0000 7f7cedd59700 1 -- 172.21.15.5:0/1045888793 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ce81a7020 con 0x7f7ce8107e10 2024-05-09T00:45:23.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.715+0000 7f7ceed5b700 1 --2- 172.21.15.5:0/1045888793 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ce8108760 0x7f7ce819c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:45:23.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.715+0000 7f7cedd59700 1 --2- 172.21.15.5:0/1045888793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ce8107e10 0x7f7ce819c210 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7cd800d980 tx=0x7f7cd800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:23.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.715+0000 7f7cdf7fe700 1 -- 172.21.15.5:0/1045888793 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd8017070 con 0x7f7ce8107e10 2024-05-09T00:45:23.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.715+0000 7f7cf07be700 1 -- 172.21.15.5:0/1045888793 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ce8199b20 con 0x7f7ce8107e10 2024-05-09T00:45:23.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.715+0000 7f7cdf7fe700 1 -- 172.21.15.5:0/1045888793 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cd8009de0 con 0x7f7ce8107e10 2024-05-09T00:45:23.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.716+0000 7f7cdf7fe700 1 -- 172.21.15.5:0/1045888793 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd800f3c0 con 0x7f7ce8107e10 2024-05-09T00:45:23.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.716+0000 7f7cf07be700 1 -- 172.21.15.5:0/1045888793 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ce819a0b0 con 0x7f7ce8107e10 2024-05-09T00:45:23.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.717+0000 7f7cf07be700 1 -- 172.21.15.5:0/1045888793 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ce80512c0 con 0x7f7ce8107e10 2024-05-09T00:45:23.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.719+0000 7f7cdf7fe700 1 -- 172.21.15.5:0/1045888793 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f7cd8009930 con 0x7f7ce8107e10 2024-05-09T00:45:23.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.720+0000 7f7cdf7fe700 1 --2- 172.21.15.5:0/1045888793 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f7cd40b2910 0x7f7cd40b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:23.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.720+0000 7f7cee55a700 1 --2- 172.21.15.5:0/1045888793 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f7cd40b2910 0x7f7cd40b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:23.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.720+0000 7f7cdf7fe700 1 -- 172.21.15.5:0/1045888793 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8285+0+0 (secure 0 0 0) 0x7f7cd80e22b0 con 0x7f7ce8107e10 2024-05-09T00:45:23.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.721+0000 7f7cee55a700 1 --2- 172.21.15.5:0/1045888793 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f7cd40b2910 0x7f7cd40b4e20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7ce4009990 tx=0x7f7ce40036e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:23.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.724+0000 7f7cdf7fe700 1 -- 172.21.15.5:0/1045888793 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7cd80a9de0 con 0x7f7ce8107e10 2024-05-09T00:45:23.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.904+0000 7ff34d250700 1 -- 172.21.15.5:0/183295849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff34810e8e0 msgr2=0x7ff348110df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:23.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.904+0000 7ff34d250700 1 --2- 172.21.15.5:0/183295849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff34810e8e0 0x7ff348110df0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff34000b0a0 tx=0x7ff34000b3b0 comp rx=0 tx=0).stop 2024-05-09T00:45:23.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.904+0000 7ff34d250700 1 -- 172.21.15.5:0/183295849 shutdown_connections 2024-05-09T00:45:23.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.904+0000 7ff34d250700 1 --2- 172.21.15.5:0/183295849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff34810e8e0 0x7ff348110df0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:23.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.904+0000 7ff34d250700 1 --2- 172.21.15.5:0/183295849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff348105500 0x7ff348105910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:23.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.904+0000 7ff34d250700 1 --2- 172.21.15.5:0/183295849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff348105e50 0x7ff34810e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:23.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.904+0000 7ff34d250700 1 -- 172.21.15.5:0/183295849 >> 172.21.15.5:0/183295849 conn(0x7ff3480fef50 msgr2=0x7ff348101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:45:23.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.904+0000 7ff34d250700 1 -- 172.21.15.5:0/183295849 shutdown_connections 2024-05-09T00:45:23.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.904+0000 7ff34d250700 1 -- 172.21.15.5:0/183295849 wait complete. 2024-05-09T00:45:23.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.905+0000 7ff34d250700 1 Processor -- start 2024-05-09T00:45:23.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.905+0000 7ff34d250700 1 -- start start 2024-05-09T00:45:23.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.905+0000 7ff34d250700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff348105500 0x7ff34819ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:23.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.905+0000 7ff34d250700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff348105e50 0x7ff34819b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:23.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.905+0000 7ff34d250700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff34810e8e0 0x7ff34819f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:23.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.905+0000 7ff34d250700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff348114490 con 0x7ff348105e50 2024-05-09T00:45:23.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.906+0000 7ff34d250700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff348114310 con 0x7ff348105500 2024-05-09T00:45:23.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.906+0000 7ff34d250700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff348114610 con 0x7ff34810e8e0 2024-05-09T00:45:23.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.906+0000 7ff34659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff348105e50 0x7ff34819b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:23.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.906+0000 7ff34659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff348105e50 0x7ff34819b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39546/0 (socket says 172.21.15.5:39546) 2024-05-09T00:45:23.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.906+0000 7ff34659c700 1 -- 172.21.15.5:0/1660382285 learned_addr learned my addr 172.21.15.5:0/1660382285 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:45:23.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.906+0000 7ff34759e700 1 --2- 172.21.15.5:0/1660382285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff34810e8e0 0x7ff34819f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:23.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.906+0000 7ff346d9d700 1 --2- 172.21.15.5:0/1660382285 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff348105500 0x7ff34819ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:23.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.906+0000 7ff34659c700 1 -- 172.21.15.5:0/1660382285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff34810e8e0 msgr2=0x7ff34819f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:23.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.906+0000 7ff34659c700 1 --2- 172.21.15.5:0/1660382285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff34810e8e0 0x7ff34819f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:23.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.906+0000 7ff34659c700 1 -- 172.21.15.5:0/1660382285 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff348105500 msgr2=0x7ff34819ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:23.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.906+0000 7ff34659c700 1 --2- 172.21.15.5:0/1660382285 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff348105500 0x7ff34819ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:23.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.906+0000 7ff34659c700 1 -- 172.21.15.5:0/1660382285 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff34819fd10 con 0x7ff348105e50 2024-05-09T00:45:23.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.906+0000 7ff34659c700 1 --2- 172.21.15.5:0/1660382285 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff348105e50 0x7ff34819b3e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff33c00d980 tx=0x7ff33c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:23.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.907+0000 7ff32ffff700 1 -- 172.21.15.5:0/1660382285 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff33c017070 con 0x7ff348105e50 2024-05-09T00:45:23.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.907+0000 7ff34d250700 1 -- 172.21.15.5:0/1660382285 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3481a0000 con 0x7ff348105e50 2024-05-09T00:45:23.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.907+0000 7ff32ffff700 1 -- 172.21.15.5:0/1660382285 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff33c00fd40 con 0x7ff348105e50 2024-05-09T00:45:23.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.907+0000 7ff32ffff700 1 -- 172.21.15.5:0/1660382285 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff33c00e3b0 con 0x7ff348105e50 2024-05-09T00:45:23.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.907+0000 7ff34d250700 1 -- 172.21.15.5:0/1660382285 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3481a7920 con 0x7ff348105e50 2024-05-09T00:45:23.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.908+0000 7ff34d250700 1 -- 172.21.15.5:0/1660382285 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3480512c0 con 0x7ff348105e50 2024-05-09T00:45:23.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.910+0000 7ff32ffff700 1 -- 172.21.15.5:0/1660382285 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7ff33c00f890 con 0x7ff348105e50 2024-05-09T00:45:23.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.911+0000 7ff32ffff700 1 --2- 172.21.15.5:0/1660382285 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ff3300b2990 0x7ff3300b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:23.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.911+0000 7ff346d9d700 1 --2- 172.21.15.5:0/1660382285 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ff3300b2990 0x7ff3300b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:23.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.911+0000 7ff32ffff700 1 -- 172.21.15.5:0/1660382285 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8285+0+0 (secure 0 0 0) 0x7ff33c0e20d0 con 0x7ff348105e50 2024-05-09T00:45:23.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.912+0000 7ff346d9d700 1 --2- 172.21.15.5:0/1660382285 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ff3300b2990 0x7ff3300b4ea0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff338005950 tx=0x7ff33800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:23.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.915+0000 7ff32ffff700 1 -- 172.21.15.5:0/1660382285 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff33c0a9c00 con 0x7ff348105e50 2024-05-09T00:45:23.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.950+0000 7f7cf07be700 1 -- 172.21.15.5:0/1045888793 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f7ce806c630 con 0x7f7ce8107e10 2024-05-09T00:45:23.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.950+0000 7f7cdf7fe700 1 -- 172.21.15.5:0/1045888793 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f7cd80a96e0 con 0x7f7ce8107e10 2024-05-09T00:45:23.951 INFO:teuthology.orchestra.run.smithi005.stdout:133143986248 2024-05-09T00:45:23.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.953+0000 7f7cf07be700 1 -- 172.21.15.5:0/1045888793 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f7cd40b2910 msgr2=0x7f7cd40b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:23.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.953+0000 7f7cf07be700 1 --2- 172.21.15.5:0/1045888793 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f7cd40b2910 0x7f7cd40b4e20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7ce4009990 tx=0x7f7ce40036e0 comp rx=0 tx=0).stop 2024-05-09T00:45:23.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.953+0000 7f7cf07be700 1 -- 172.21.15.5:0/1045888793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ce8107e10 msgr2=0x7f7ce819c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:23.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.953+0000 7f7cf07be700 1 --2- 172.21.15.5:0/1045888793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ce8107e10 0x7f7ce819c210 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7cd800d980 tx=0x7f7cd800dc90 comp rx=0 tx=0).stop 2024-05-09T00:45:23.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.953+0000 7f7cf07be700 1 -- 172.21.15.5:0/1045888793 shutdown_connections 2024-05-09T00:45:23.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.953+0000 7f7cf07be700 1 --2- 172.21.15.5:0/1045888793 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f7cd40b2910 0x7f7cd40b4e20 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:23.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.953+0000 7f7cf07be700 1 --2- 172.21.15.5:0/1045888793 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ce8108760 0x7f7ce819c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:23.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.953+0000 7f7cf07be700 1 --2- 172.21.15.5:0/1045888793 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ce81034e0 0x7f7ce819bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:23.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.953+0000 7f7cf07be700 1 --2- 172.21.15.5:0/1045888793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ce8107e10 0x7f7ce819c210 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:23.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.953+0000 7f7cf07be700 1 -- 172.21.15.5:0/1045888793 >> 172.21.15.5:0/1045888793 conn(0x7f7ce80fef30 msgr2=0x7f7ce81096c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:45:23.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.954+0000 7f7cf07be700 1 -- 172.21.15.5:0/1045888793 shutdown_connections 2024-05-09T00:45:23.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:23.954+0000 7f7cf07be700 1 -- 172.21.15.5:0/1045888793 wait complete. 2024-05-09T00:45:24.040 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:45:23 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-05-09T00:45:24.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:45:23 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-05-09T00:45:24.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:45:23 smithi005 ceph-mon[30365]: osdmap e81: 12 total, 12 up, 12 in 2024-05-09T00:45:24.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:24.153+0000 7ff34d250700 1 -- 172.21.15.5:0/1660382285 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7ff34819c1a0 con 0x7ff348105e50 2024-05-09T00:45:24.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:24.153+0000 7ff32ffff700 1 -- 172.21.15.5:0/1660382285 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff33c0a9500 con 0x7ff348105e50 2024-05-09T00:45:24.154 INFO:teuthology.orchestra.run.smithi005.stdout:201863462969 2024-05-09T00:45:24.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:24.155+0000 7ff34d250700 1 -- 172.21.15.5:0/1660382285 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ff3300b2990 msgr2=0x7ff3300b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:24.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:24.155+0000 7ff34d250700 1 --2- 172.21.15.5:0/1660382285 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ff3300b2990 0x7ff3300b4ea0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff338005950 tx=0x7ff33800b410 comp rx=0 tx=0).stop 2024-05-09T00:45:24.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:24.155+0000 7ff34d250700 1 -- 172.21.15.5:0/1660382285 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff348105e50 msgr2=0x7ff34819b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:24.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:24.155+0000 7ff34d250700 1 --2- 172.21.15.5:0/1660382285 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff348105e50 0x7ff34819b3e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff33c00d980 tx=0x7ff33c00dc90 comp rx=0 tx=0).stop 2024-05-09T00:45:24.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:24.155+0000 7ff34d250700 1 -- 172.21.15.5:0/1660382285 shutdown_connections 2024-05-09T00:45:24.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:24.155+0000 7ff34d250700 1 --2- 172.21.15.5:0/1660382285 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ff3300b2990 0x7ff3300b4ea0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:24.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:24.155+0000 7ff34d250700 1 --2- 172.21.15.5:0/1660382285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff34810e8e0 0x7ff34819f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:24.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:24.155+0000 7ff34d250700 1 --2- 172.21.15.5:0/1660382285 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff348105500 0x7ff34819ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:24.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:24.155+0000 7ff34d250700 1 --2- 172.21.15.5:0/1660382285 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff348105e50 0x7ff34819b3e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:24.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:24.155+0000 7ff34d250700 1 -- 172.21.15.5:0/1660382285 >> 172.21.15.5:0/1660382285 conn(0x7ff3480fef50 msgr2=0x7ff3480ff540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:45:24.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:24.155+0000 7ff34d250700 1 -- 172.21.15.5:0/1660382285 shutdown_connections 2024-05-09T00:45:24.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:24.155+0000 7ff34d250700 1 -- 172.21.15.5:0/1660382285 wait complete. 2024-05-09T00:45:24.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:45:23 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-05-09T00:45:24.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:45:23 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-05-09T00:45:24.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:45:23 smithi157 ceph-mon[34294]: osdmap e81: 12 total, 12 up, 12 in 2024-05-09T00:45:24.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:45:23 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-05-09T00:45:24.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:45:23 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-05-09T00:45:24.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:45:23 smithi131 ceph-mon[34547]: osdmap e81: 12 total, 12 up, 12 in 2024-05-09T00:45:25.040 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:45:24 smithi005 ceph-mon[30365]: pgmap v342: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T00:45:25.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:45:24 smithi005 ceph-mon[30365]: osdmap e82: 12 total, 12 up, 12 in 2024-05-09T00:45:25.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:45:24 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/1045888793' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-09T00:45:25.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:45:24 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/1660382285' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-09T00:45:25.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:45:24 smithi157 ceph-mon[34294]: pgmap v342: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T00:45:25.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:45:24 smithi157 ceph-mon[34294]: osdmap e82: 12 total, 12 up, 12 in 2024-05-09T00:45:25.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:45:24 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/1045888793' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-09T00:45:25.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:45:24 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/1660382285' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-09T00:45:25.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:45:24 smithi131 ceph-mon[34547]: pgmap v342: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T00:45:25.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:45:24 smithi131 ceph-mon[34547]: osdmap e82: 12 total, 12 up, 12 in 2024-05-09T00:45:25.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:45:24 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/1045888793' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-09T00:45:25.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:45:24 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/1660382285' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-09T00:45:25.856 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986244 got 133143986248 for osd.4 2024-05-09T00:45:25.857 DEBUG:teuthology.parallel:result is None 2024-05-09T00:45:26.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:26.515+0000 7f61e3150700 1 -- 172.21.15.5:0/833449656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61dc108f50 msgr2=0x7f61dc109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:26.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:26.515+0000 7f61e3150700 1 --2- 172.21.15.5:0/833449656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61dc108f50 0x7f61dc109360 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f61d0009990 tx=0x7f61d0009ca0 comp rx=0 tx=0).stop 2024-05-09T00:45:26.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:26.515+0000 7f61e3150700 1 -- 172.21.15.5:0/833449656 shutdown_connections 2024-05-09T00:45:26.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:26.515+0000 7f61e3150700 1 --2- 172.21.15.5:0/833449656 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61dc1098a0 0x7f61dc1111e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:26.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:26.515+0000 7f61e3150700 1 --2- 172.21.15.5:0/833449656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61dc102b30 0x7f61dc102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:26.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:26.515+0000 7f61e3150700 1 --2- 172.21.15.5:0/833449656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61dc108f50 0x7f61dc109360 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:26.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:26.515+0000 7f61e3150700 1 -- 172.21.15.5:0/833449656 >> 172.21.15.5:0/833449656 conn(0x7f61dc0fe530 msgr2=0x7f61dc100980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:45:26.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:26.515+0000 7f61e3150700 1 -- 172.21.15.5:0/833449656 shutdown_connections 2024-05-09T00:45:26.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:26.515+0000 7f61e3150700 1 -- 172.21.15.5:0/833449656 wait complete. 2024-05-09T00:45:26.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:26.516+0000 7f61e3150700 1 Processor -- start 2024-05-09T00:45:26.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:26.516+0000 7f61e3150700 1 -- start start 2024-05-09T00:45:26.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:26.516+0000 7f61e3150700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61dc102b30 0x7f61dc19cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:26.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:26.516+0000 7f61e3150700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61dc108f50 0x7f61dc19d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:26.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:26.516+0000 7f61e3150700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61dc1098a0 0x7f61dc1a1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:26.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:26.516+0000 7f61e3150700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc114450 con 0x7f61dc102b30 2024-05-09T00:45:26.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:26.516+0000 7f61e3150700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc1142d0 con 0x7f61dc1098a0 2024-05-09T00:45:26.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:26.516+0000 7f61e3150700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc1145d0 con 0x7f61dc108f50 2024-05-09T00:45:26.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:26.516+0000 7f61e0eec700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61dc102b30 0x7f61dc19cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:26.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:26.516+0000 7f61e0eec700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61dc102b30 0x7f61dc19cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39566/0 (socket says 172.21.15.5:39566) 2024-05-09T00:45:26.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:26.516+0000 7f61e0eec700 1 -- 172.21.15.5:0/738753633 learned_addr learned my addr 172.21.15.5:0/738753633 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:45:26.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:26.516+0000 7f61e16ed700 1 --2- 172.21.15.5:0/738753633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61dc1098a0 0x7f61dc1a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:26.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:26.516+0000 7f61dbfff700 1 --2- 172.21.15.5:0/738753633 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61dc108f50 0x7f61dc19d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:26.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:26.516+0000 7f61e0eec700 1 -- 172.21.15.5:0/738753633 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61dc108f50 msgr2=0x7f61dc19d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:26.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:26.516+0000 7f61e0eec700 1 --2- 172.21.15.5:0/738753633 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61dc108f50 0x7f61dc19d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:26.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:26.516+0000 7f61e0eec700 1 -- 172.21.15.5:0/738753633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61dc1098a0 msgr2=0x7f61dc1a1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:26.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:26.516+0000 7f61e0eec700 1 --2- 172.21.15.5:0/738753633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61dc1098a0 0x7f61dc1a1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:26.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:26.516+0000 7f61e0eec700 1 -- 172.21.15.5:0/738753633 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61dc1a1eb0 con 0x7f61dc102b30 2024-05-09T00:45:26.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:26.517+0000 7f61dbfff700 1 --2- 172.21.15.5:0/738753633 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61dc108f50 0x7f61dc19d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:45:26.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:26.517+0000 7f61e0eec700 1 --2- 172.21.15.5:0/738753633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61dc102b30 0x7f61dc19cfe0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f61cc00d950 tx=0x7f61cc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:26.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:26.517+0000 7f61d9ffb700 1 -- 172.21.15.5:0/738753633 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61cc016070 con 0x7f61dc102b30 2024-05-09T00:45:26.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:26.517+0000 7f61d9ffb700 1 -- 172.21.15.5:0/738753633 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61cc009df0 con 0x7f61dc102b30 2024-05-09T00:45:26.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:26.517+0000 7f61e3150700 1 -- 172.21.15.5:0/738753633 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61dc1a21a0 con 0x7f61dc102b30 2024-05-09T00:45:26.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:26.517+0000 7f61d9ffb700 1 -- 172.21.15.5:0/738753633 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61cc00f3c0 con 0x7f61dc102b30 2024-05-09T00:45:26.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:26.517+0000 7f61e3150700 1 -- 172.21.15.5:0/738753633 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61dc1a9ac0 con 0x7f61dc102b30 2024-05-09T00:45:26.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:26.518+0000 7f61e3150700 1 -- 172.21.15.5:0/738753633 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61dc0512c0 con 0x7f61dc102b30 2024-05-09T00:45:26.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:26.520+0000 7f61d9ffb700 1 -- 172.21.15.5:0/738753633 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f61cc019070 con 0x7f61dc102b30 2024-05-09T00:45:26.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:26.521+0000 7f61d9ffb700 1 --2- 172.21.15.5:0/738753633 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f61c40b2990 0x7f61c40b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:26.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:26.521+0000 7f61dbfff700 1 --2- 172.21.15.5:0/738753633 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f61c40b2990 0x7f61c40b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:26.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:26.521+0000 7f61d9ffb700 1 -- 172.21.15.5:0/738753633 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f61cc0e1f40 con 0x7f61dc102b30 2024-05-09T00:45:26.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:26.521+0000 7f61dbfff700 1 --2- 172.21.15.5:0/738753633 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f61c40b2990 0x7f61c40b4ea0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f61dc108da0 tx=0x7f61d001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:26.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:26.525+0000 7f61d9ffb700 1 -- 172.21.15.5:0/738753633 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f61cc0a9ad0 con 0x7f61dc102b30 2024-05-09T00:45:26.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:26.749+0000 7f61e3150700 1 -- 172.21.15.5:0/738753633 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f61dc1a9c90 con 0x7f61dc102b30 2024-05-09T00:45:26.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:26.750+0000 7f61d9ffb700 1 -- 172.21.15.5:0/738753633 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f61cc0a93d0 con 0x7f61dc102b30 2024-05-09T00:45:26.751 INFO:teuthology.orchestra.run.smithi005.stdout:253403070511 2024-05-09T00:45:26.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:26.752+0000 7f61e3150700 1 -- 172.21.15.5:0/738753633 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f61c40b2990 msgr2=0x7f61c40b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:26.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:26.752+0000 7f61e3150700 1 --2- 172.21.15.5:0/738753633 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f61c40b2990 0x7f61c40b4ea0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f61dc108da0 tx=0x7f61d001a040 comp rx=0 tx=0).stop 2024-05-09T00:45:26.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:26.752+0000 7f61e3150700 1 -- 172.21.15.5:0/738753633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61dc102b30 msgr2=0x7f61dc19cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:26.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:26.752+0000 7f61e3150700 1 --2- 172.21.15.5:0/738753633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61dc102b30 0x7f61dc19cfe0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f61cc00d950 tx=0x7f61cc00dc60 comp rx=0 tx=0).stop 2024-05-09T00:45:26.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:26.752+0000 7f61e3150700 1 -- 172.21.15.5:0/738753633 shutdown_connections 2024-05-09T00:45:26.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:26.752+0000 7f61e3150700 1 --2- 172.21.15.5:0/738753633 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f61c40b2990 0x7f61c40b4ea0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:26.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:26.752+0000 7f61e3150700 1 --2- 172.21.15.5:0/738753633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61dc1098a0 0x7f61dc1a1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:26.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:26.752+0000 7f61e3150700 1 --2- 172.21.15.5:0/738753633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61dc102b30 0x7f61dc19cfe0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:26.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:26.752+0000 7f61e3150700 1 --2- 172.21.15.5:0/738753633 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61dc108f50 0x7f61dc19d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:26.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:26.752+0000 7f61e3150700 1 -- 172.21.15.5:0/738753633 >> 172.21.15.5:0/738753633 conn(0x7f61dc0fe530 msgr2=0x7f61dc110480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:45:26.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:26.752+0000 7f61e3150700 1 -- 172.21.15.5:0/738753633 shutdown_connections 2024-05-09T00:45:26.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:26.752+0000 7f61e3150700 1 -- 172.21.15.5:0/738753633 wait complete. 2024-05-09T00:45:27.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:45:26 smithi005 ceph-mon[30365]: pgmap v344: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s; 72 B/s, 0 objects/s recovering 2024-05-09T00:45:27.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:45:26 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/738753633' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-09T00:45:27.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:45:26 smithi131 ceph-mon[34547]: pgmap v344: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s; 72 B/s, 0 objects/s recovering 2024-05-09T00:45:27.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:45:26 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/738753633' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-09T00:45:27.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:45:26 smithi157 ceph-mon[34294]: pgmap v344: 97 pgs: 97 active+clean; 645 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s; 72 B/s, 0 objects/s recovering 2024-05-09T00:45:27.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:45:26 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/738753633' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-09T00:45:27.795 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462965 got 201863462969 for osd.7 2024-05-09T00:45:27.796 DEBUG:teuthology.parallel:result is None 2024-05-09T00:45:28.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:28.023+0000 7f31e86b0700 1 -- 172.21.15.5:0/448533835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31e0110c50 msgr2=0x7f31e0113070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:28.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:28.023+0000 7f31e86b0700 1 --2- 172.21.15.5:0/448533835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31e0110c50 0x7f31e0113070 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f31d800b0a0 tx=0x7f31d800b3b0 comp rx=0 tx=0).stop 2024-05-09T00:45:28.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:28.023+0000 7f31e86b0700 1 -- 172.21.15.5:0/448533835 shutdown_connections 2024-05-09T00:45:28.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:28.023+0000 7f31e86b0700 1 --2- 172.21.15.5:0/448533835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31e0110c50 0x7f31e0113070 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:28.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:28.023+0000 7f31e86b0700 1 --2- 172.21.15.5:0/448533835 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31e01030a0 0x7f31e01034b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:28.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:28.023+0000 7f31e86b0700 1 --2- 172.21.15.5:0/448533835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31e0103a80 0x7f31e0110710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:28.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:28.023+0000 7f31e86b0700 1 -- 172.21.15.5:0/448533835 >> 172.21.15.5:0/448533835 conn(0x7f31e00fef30 msgr2=0x7f31e0101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:45:28.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:28.023+0000 7f31e86b0700 1 -- 172.21.15.5:0/448533835 shutdown_connections 2024-05-09T00:45:28.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:28.023+0000 7f31e86b0700 1 -- 172.21.15.5:0/448533835 wait complete. 2024-05-09T00:45:28.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:28.024+0000 7f31e86b0700 1 Processor -- start 2024-05-09T00:45:28.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:28.024+0000 7f31e86b0700 1 -- start start 2024-05-09T00:45:28.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:28.024+0000 7f31e86b0700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31e01030a0 0x7f31e019cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:28.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:28.024+0000 7f31e86b0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31e0103a80 0x7f31e019d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:28.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:28.024+0000 7f31e86b0700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31e0110c50 0x7f31e01a1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:28.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:28.024+0000 7f31e86b0700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f31e01166b0 con 0x7f31e0103a80 2024-05-09T00:45:28.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:28.024+0000 7f31e86b0700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f31e0116530 con 0x7f31e0110c50 2024-05-09T00:45:28.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:28.024+0000 7f31e86b0700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f31e0116830 con 0x7f31e01030a0 2024-05-09T00:45:28.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:28.024+0000 7f31e5c4b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31e0103a80 0x7f31e019d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:28.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:28.024+0000 7f31e5c4b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31e0103a80 0x7f31e019d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39576/0 (socket says 172.21.15.5:39576) 2024-05-09T00:45:28.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:28.024+0000 7f31e5c4b700 1 -- 172.21.15.5:0/332738743 learned_addr learned my addr 172.21.15.5:0/332738743 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:45:28.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:28.024+0000 7f31e644c700 1 --2- 172.21.15.5:0/332738743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31e01030a0 0x7f31e019cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:28.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:28.025+0000 7f31e6c4d700 1 --2- 172.21.15.5:0/332738743 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31e0110c50 0x7f31e01a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:28.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:28.025+0000 7f31e5c4b700 1 -- 172.21.15.5:0/332738743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31e01030a0 msgr2=0x7f31e019cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:28.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:28.025+0000 7f31e5c4b700 1 --2- 172.21.15.5:0/332738743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31e01030a0 0x7f31e019cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:28.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:28.025+0000 7f31e5c4b700 1 -- 172.21.15.5:0/332738743 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31e0110c50 msgr2=0x7f31e01a1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:28.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:28.025+0000 7f31e5c4b700 1 --2- 172.21.15.5:0/332738743 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31e0110c50 0x7f31e01a1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:28.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:28.025+0000 7f31e5c4b700 1 -- 172.21.15.5:0/332738743 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31e01a1eb0 con 0x7f31e0103a80 2024-05-09T00:45:28.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:28.025+0000 7f31e6c4d700 1 --2- 172.21.15.5:0/332738743 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31e0110c50 0x7f31e01a1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:45:28.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:28.025+0000 7f31e5c4b700 1 --2- 172.21.15.5:0/332738743 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31e0103a80 0x7f31e019d580 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f31d000d950 tx=0x7f31d000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:28.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:28.025+0000 7f31d77fe700 1 -- 172.21.15.5:0/332738743 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d0017070 con 0x7f31e0103a80 2024-05-09T00:45:28.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:28.025+0000 7f31e86b0700 1 -- 172.21.15.5:0/332738743 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31e01a21a0 con 0x7f31e0103a80 2024-05-09T00:45:28.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:28.025+0000 7f31d77fe700 1 -- 172.21.15.5:0/332738743 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31d0009730 con 0x7f31e0103a80 2024-05-09T00:45:28.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:28.025+0000 7f31d77fe700 1 -- 172.21.15.5:0/332738743 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d0003a00 con 0x7f31e0103a80 2024-05-09T00:45:28.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:28.025+0000 7f31e86b0700 1 -- 172.21.15.5:0/332738743 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31e01a9ac0 con 0x7f31e0103a80 2024-05-09T00:45:28.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:28.026+0000 7f31e86b0700 1 -- 172.21.15.5:0/332738743 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31e00512c0 con 0x7f31e0103a80 2024-05-09T00:45:28.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:28.028+0000 7f31d77fe700 1 -- 172.21.15.5:0/332738743 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f31d0003ba0 con 0x7f31e0103a80 2024-05-09T00:45:28.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:28.029+0000 7f31d77fe700 1 --2- 172.21.15.5:0/332738743 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f31cc0b2910 0x7f31cc0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:28.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:28.029+0000 7f31e644c700 1 --2- 172.21.15.5:0/332738743 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f31cc0b2910 0x7f31cc0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:28.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:28.029+0000 7f31d77fe700 1 -- 172.21.15.5:0/332738743 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f31d00e26d0 con 0x7f31e0103a80 2024-05-09T00:45:28.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:28.029+0000 7f31e644c700 1 --2- 172.21.15.5:0/332738743 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f31cc0b2910 0x7f31cc0b4e20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f31dc005950 tx=0x7f31dc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:28.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:28.033+0000 7f31d77fe700 1 -- 172.21.15.5:0/332738743 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f31d0020030 con 0x7f31e0103a80 2024-05-09T00:45:28.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:28.260+0000 7f31e86b0700 1 -- 172.21.15.5:0/332738743 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f31e002e5f0 con 0x7f31e0103a80 2024-05-09T00:45:28.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:28.260+0000 7f31d77fe700 1 -- 172.21.15.5:0/332738743 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f31d00aa200 con 0x7f31e0103a80 2024-05-09T00:45:28.261 INFO:teuthology.orchestra.run.smithi005.stdout:154618822724 2024-05-09T00:45:28.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:28.262+0000 7f31e86b0700 1 -- 172.21.15.5:0/332738743 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f31cc0b2910 msgr2=0x7f31cc0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:28.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:28.262+0000 7f31e86b0700 1 --2- 172.21.15.5:0/332738743 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f31cc0b2910 0x7f31cc0b4e20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f31dc005950 tx=0x7f31dc00b410 comp rx=0 tx=0).stop 2024-05-09T00:45:28.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:28.262+0000 7f31e86b0700 1 -- 172.21.15.5:0/332738743 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31e0103a80 msgr2=0x7f31e019d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:28.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:28.262+0000 7f31e86b0700 1 --2- 172.21.15.5:0/332738743 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31e0103a80 0x7f31e019d580 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f31d000d950 tx=0x7f31d000dc60 comp rx=0 tx=0).stop 2024-05-09T00:45:28.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:28.262+0000 7f31e86b0700 1 -- 172.21.15.5:0/332738743 shutdown_connections 2024-05-09T00:45:28.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:28.262+0000 7f31e86b0700 1 --2- 172.21.15.5:0/332738743 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f31cc0b2910 0x7f31cc0b4e20 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:28.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:28.262+0000 7f31e86b0700 1 --2- 172.21.15.5:0/332738743 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31e0110c50 0x7f31e01a1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:28.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:28.262+0000 7f31e86b0700 1 --2- 172.21.15.5:0/332738743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31e01030a0 0x7f31e019cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:28.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:28.262+0000 7f31e86b0700 1 --2- 172.21.15.5:0/332738743 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31e0103a80 0x7f31e019d580 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:28.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:28.262+0000 7f31e86b0700 1 -- 172.21.15.5:0/332738743 >> 172.21.15.5:0/332738743 conn(0x7f31e00fef30 msgr2=0x7f31e0112630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:45:28.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:28.262+0000 7f31e86b0700 1 -- 172.21.15.5:0/332738743 shutdown_connections 2024-05-09T00:45:28.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:28.262+0000 7f31e86b0700 1 -- 172.21.15.5:0/332738743 wait complete. 2024-05-09T00:45:29.166 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:45:28 smithi005 ceph-mon[30365]: pgmap v345: 97 pgs: 97 active+clean; 645 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 8 op/s; 72 B/s, 0 objects/s recovering 2024-05-09T00:45:29.166 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:45:28 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/332738743' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-09T00:45:29.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:45:28 smithi131 ceph-mon[34547]: pgmap v345: 97 pgs: 97 active+clean; 645 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 8 op/s; 72 B/s, 0 objects/s recovering 2024-05-09T00:45:29.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:45:28 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/332738743' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-09T00:45:29.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:45:28 smithi157 ceph-mon[34294]: pgmap v345: 97 pgs: 97 active+clean; 645 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 8 op/s; 72 B/s, 0 objects/s recovering 2024-05-09T00:45:29.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:45:28 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/332738743' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-09T00:45:30.024 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:45:31.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:45:30 smithi005 ceph-mon[30365]: pgmap v346: 97 pgs: 97 active+clean; 645 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s; 72 B/s, 0 objects/s recovering 2024-05-09T00:45:31.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:45:30 smithi131 ceph-mon[34547]: pgmap v346: 97 pgs: 97 active+clean; 645 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s; 72 B/s, 0 objects/s recovering 2024-05-09T00:45:31.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:45:30 smithi157 ceph-mon[34294]: pgmap v346: 97 pgs: 97 active+clean; 645 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s; 72 B/s, 0 objects/s recovering 2024-05-09T00:45:32.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.039+0000 7f5dabc78700 1 -- 172.21.15.5:0/1225071594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5da41039d0 msgr2=0x7f5da41106f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:32.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.039+0000 7f5dabc78700 1 --2- 172.21.15.5:0/1225071594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5da41039d0 0x7f5da41106f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5d94009990 tx=0x7f5d94009ca0 comp rx=0 tx=0).stop 2024-05-09T00:45:32.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.039+0000 7f5dabc78700 1 -- 172.21.15.5:0/1225071594 shutdown_connections 2024-05-09T00:45:32.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.039+0000 7f5dabc78700 1 --2- 172.21.15.5:0/1225071594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5da4110c30 0x7f5da4113050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:32.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.039+0000 7f5dabc78700 1 --2- 172.21.15.5:0/1225071594 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5da4103080 0x7f5da4103490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:32.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.039+0000 7f5dabc78700 1 --2- 172.21.15.5:0/1225071594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5da41039d0 0x7f5da41106f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:32.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.039+0000 7f5dabc78700 1 -- 172.21.15.5:0/1225071594 >> 172.21.15.5:0/1225071594 conn(0x7f5da40fedf0 msgr2=0x7f5da4101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:45:32.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.039+0000 7f5dabc78700 1 -- 172.21.15.5:0/1225071594 shutdown_connections 2024-05-09T00:45:32.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.039+0000 7f5dabc78700 1 -- 172.21.15.5:0/1225071594 wait complete. 2024-05-09T00:45:32.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.040+0000 7f5dabc78700 1 Processor -- start 2024-05-09T00:45:32.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.040+0000 7f5dabc78700 1 -- start start 2024-05-09T00:45:32.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.041+0000 7f5dabc78700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5da4103080 0x7f5da419d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:32.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.041+0000 7f5dabc78700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5da41039d0 0x7f5da419d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:32.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.041+0000 7f5dabc78700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5da4110c30 0x7f5da41a1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:32.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.041+0000 7f5dabc78700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5da4116660 con 0x7f5da41039d0 2024-05-09T00:45:32.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.041+0000 7f5dabc78700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5da41164e0 con 0x7f5da4103080 2024-05-09T00:45:32.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.041+0000 7f5dabc78700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5da41167e0 con 0x7f5da4110c30 2024-05-09T00:45:32.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.041+0000 7f5da9213700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5da41039d0 0x7f5da419d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:32.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.041+0000 7f5da9213700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5da41039d0 0x7f5da419d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39584/0 (socket says 172.21.15.5:39584) 2024-05-09T00:45:32.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.041+0000 7f5da9213700 1 -- 172.21.15.5:0/348478833 learned_addr learned my addr 172.21.15.5:0/348478833 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:45:32.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.041+0000 7f5da9a14700 1 --2- 172.21.15.5:0/348478833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5da4103080 0x7f5da419d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:32.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.041+0000 7f5daa215700 1 --2- 172.21.15.5:0/348478833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5da4110c30 0x7f5da41a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:32.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.041+0000 7f5da9213700 1 -- 172.21.15.5:0/348478833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5da4110c30 msgr2=0x7f5da41a1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:32.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.041+0000 7f5da9213700 1 --2- 172.21.15.5:0/348478833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5da4110c30 0x7f5da41a1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:32.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.041+0000 7f5da9213700 1 -- 172.21.15.5:0/348478833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5da4103080 msgr2=0x7f5da419d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:32.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.041+0000 7f5da9213700 1 --2- 172.21.15.5:0/348478833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5da4103080 0x7f5da419d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:32.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.041+0000 7f5da9213700 1 -- 172.21.15.5:0/348478833 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5da41a1e30 con 0x7f5da41039d0 2024-05-09T00:45:32.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.041+0000 7f5da9213700 1 --2- 172.21.15.5:0/348478833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5da41039d0 0x7f5da419d5b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5d94006010 tx=0x7f5d940047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:32.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.041+0000 7f5d9affd700 1 -- 172.21.15.5:0/348478833 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d94027070 con 0x7f5da41039d0 2024-05-09T00:45:32.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.041+0000 7f5d9affd700 1 -- 172.21.15.5:0/348478833 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d94003f20 con 0x7f5da41039d0 2024-05-09T00:45:32.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.041+0000 7f5dabc78700 1 -- 172.21.15.5:0/348478833 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5da41a20c0 con 0x7f5da41039d0 2024-05-09T00:45:32.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.042+0000 7f5d9affd700 1 -- 172.21.15.5:0/348478833 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d94022a20 con 0x7f5da41039d0 2024-05-09T00:45:32.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.042+0000 7f5dabc78700 1 -- 172.21.15.5:0/348478833 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5da41a25f0 con 0x7f5da41039d0 2024-05-09T00:45:32.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.043+0000 7f5dabc78700 1 -- 172.21.15.5:0/348478833 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5da40512c0 con 0x7f5da41039d0 2024-05-09T00:45:32.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.045+0000 7f5d9affd700 1 -- 172.21.15.5:0/348478833 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f5d9402a070 con 0x7f5da41039d0 2024-05-09T00:45:32.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.046+0000 7f5d9affd700 1 --2- 172.21.15.5:0/348478833 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f5d900b2990 0x7f5d900b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:32.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.046+0000 7f5da9a14700 1 --2- 172.21.15.5:0/348478833 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f5d900b2990 0x7f5d900b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:32.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.046+0000 7f5d9affd700 1 -- 172.21.15.5:0/348478833 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f5d940e3bd0 con 0x7f5da41039d0 2024-05-09T00:45:32.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.047+0000 7f5da9a14700 1 --2- 172.21.15.5:0/348478833 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f5d900b2990 0x7f5d900b4ea0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5da000ac50 tx=0x7f5da000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:32.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.050+0000 7f5d9affd700 1 -- 172.21.15.5:0/348478833 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5d94013070 con 0x7f5da41039d0 2024-05-09T00:45:32.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.265+0000 7fb3f4b2b700 1 -- 172.21.15.5:0/3689470186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3f01034c0 msgr2=0x7fb3f01038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:32.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.265+0000 7fb3f4b2b700 1 --2- 172.21.15.5:0/3689470186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3f01034c0 0x7fb3f01038d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb3d80099c0 tx=0x7fb3d8009cd0 comp rx=0 tx=0).stop 2024-05-09T00:45:32.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.265+0000 7fb3f4b2b700 1 -- 172.21.15.5:0/3689470186 shutdown_connections 2024-05-09T00:45:32.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.265+0000 7fb3f4b2b700 1 --2- 172.21.15.5:0/3689470186 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3f0108740 0x7fb3f01100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:32.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.265+0000 7fb3f4b2b700 1 --2- 172.21.15.5:0/3689470186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3f01034c0 0x7fb3f01038d0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:32.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.265+0000 7fb3f4b2b700 1 --2- 172.21.15.5:0/3689470186 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3f0107df0 0x7fb3f0108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:32.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.265+0000 7fb3f4b2b700 1 -- 172.21.15.5:0/3689470186 >> 172.21.15.5:0/3689470186 conn(0x7fb3f00fedc0 msgr2=0x7fb3f01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:45:32.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.266+0000 7fb3f4b2b700 1 -- 172.21.15.5:0/3689470186 shutdown_connections 2024-05-09T00:45:32.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.266+0000 7fb3f4b2b700 1 -- 172.21.15.5:0/3689470186 wait complete. 2024-05-09T00:45:32.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.266+0000 7fb3f4b2b700 1 Processor -- start 2024-05-09T00:45:32.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.266+0000 7fb3f4b2b700 1 -- start start 2024-05-09T00:45:32.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.267+0000 7fb3f4b2b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3f01034c0 0x7fb3f019bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:32.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.267+0000 7fb3f4b2b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3f0107df0 0x7fb3f019c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:32.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.267+0000 7fb3f4b2b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3f0108740 0x7fb3f019c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:32.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.267+0000 7fb3f4b2b700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f01133c0 con 0x7fb3f01034c0 2024-05-09T00:45:32.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.267+0000 7fb3f4b2b700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f0113240 con 0x7fb3f0107df0 2024-05-09T00:45:32.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.267+0000 7fb3f4b2b700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f0113540 con 0x7fb3f0108740 2024-05-09T00:45:32.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.267+0000 7fb3ee59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3f01034c0 0x7fb3f019bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:32.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.267+0000 7fb3ee59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3f01034c0 0x7fb3f019bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39606/0 (socket says 172.21.15.5:39606) 2024-05-09T00:45:32.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.267+0000 7fb3ee59c700 1 -- 172.21.15.5:0/1286509965 learned_addr learned my addr 172.21.15.5:0/1286509965 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:45:32.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.267+0000 7fb3edd9b700 1 --2- 172.21.15.5:0/1286509965 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3f0107df0 0x7fb3f019c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:32.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.267+0000 7fb3eed9d700 1 --2- 172.21.15.5:0/1286509965 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3f0108740 0x7fb3f019c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:32.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.267+0000 7f5dabc78700 1 -- 172.21.15.5:0/348478833 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f5da402e5f0 con 0x7f5da41039d0 2024-05-09T00:45:32.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.267+0000 7fb3ee59c700 1 -- 172.21.15.5:0/1286509965 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3f0108740 msgr2=0x7fb3f019c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:32.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.267+0000 7fb3ee59c700 1 --2- 172.21.15.5:0/1286509965 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3f0108740 0x7fb3f019c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:32.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.267+0000 7fb3ee59c700 1 -- 172.21.15.5:0/1286509965 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3f0107df0 msgr2=0x7fb3f019c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:32.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.267+0000 7fb3ee59c700 1 --2- 172.21.15.5:0/1286509965 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3f0107df0 0x7fb3f019c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:32.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.267+0000 7fb3ee59c700 1 -- 172.21.15.5:0/1286509965 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3f01a7030 con 0x7fb3f01034c0 2024-05-09T00:45:32.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.268+0000 7fb3ee59c700 1 --2- 172.21.15.5:0/1286509965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3f01034c0 0x7fb3f019bce0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb3d8009990 tx=0x7fb3d80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:32.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.268+0000 7f5d9affd700 1 -- 172.21.15.5:0/348478833 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f5d940ab790 con 0x7f5da41039d0 2024-05-09T00:45:32.275 INFO:teuthology.orchestra.run.smithi005.stdout:107374182479 2024-05-09T00:45:32.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.268+0000 7fb3e77fe700 1 -- 172.21.15.5:0/1286509965 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d8026070 con 0x7fb3f01034c0 2024-05-09T00:45:32.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.268+0000 7fb3f4b2b700 1 -- 172.21.15.5:0/1286509965 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3f01a72c0 con 0x7fb3f01034c0 2024-05-09T00:45:32.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.268+0000 7fb3e77fe700 1 -- 172.21.15.5:0/1286509965 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3d8003f20 con 0x7fb3f01034c0 2024-05-09T00:45:32.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.268+0000 7fb3e77fe700 1 -- 172.21.15.5:0/1286509965 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d800fc00 con 0x7fb3f01034c0 2024-05-09T00:45:32.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.268+0000 7fb3f4b2b700 1 -- 172.21.15.5:0/1286509965 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3f01a7770 con 0x7fb3f01034c0 2024-05-09T00:45:32.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.269+0000 7fb3f4b2b700 1 -- 172.21.15.5:0/1286509965 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3f00512c0 con 0x7fb3f01034c0 2024-05-09T00:45:32.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.270+0000 7f5dabc78700 1 -- 172.21.15.5:0/348478833 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f5d900b2990 msgr2=0x7f5d900b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:32.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.270+0000 7f5dabc78700 1 --2- 172.21.15.5:0/348478833 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f5d900b2990 0x7f5d900b4ea0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5da000ac50 tx=0x7f5da000a380 comp rx=0 tx=0).stop 2024-05-09T00:45:32.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.270+0000 7f5dabc78700 1 -- 172.21.15.5:0/348478833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5da41039d0 msgr2=0x7f5da419d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:32.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.270+0000 7f5dabc78700 1 --2- 172.21.15.5:0/348478833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5da41039d0 0x7f5da419d5b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5d94006010 tx=0x7f5d940047b0 comp rx=0 tx=0).stop 2024-05-09T00:45:32.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.270+0000 7f5dabc78700 1 -- 172.21.15.5:0/348478833 shutdown_connections 2024-05-09T00:45:32.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.270+0000 7f5dabc78700 1 --2- 172.21.15.5:0/348478833 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f5d900b2990 0x7f5d900b4ea0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:32.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.270+0000 7f5dabc78700 1 --2- 172.21.15.5:0/348478833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5da4110c30 0x7f5da41a1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:32.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.270+0000 7f5dabc78700 1 --2- 172.21.15.5:0/348478833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5da4103080 0x7f5da419d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:32.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.270+0000 7f5dabc78700 1 --2- 172.21.15.5:0/348478833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5da41039d0 0x7f5da419d5b0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:32.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.270+0000 7f5dabc78700 1 -- 172.21.15.5:0/348478833 >> 172.21.15.5:0/348478833 conn(0x7f5da40fedf0 msgr2=0x7f5da4112800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:45:32.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.270+0000 7f5dabc78700 1 -- 172.21.15.5:0/348478833 shutdown_connections 2024-05-09T00:45:32.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.270+0000 7f5dabc78700 1 -- 172.21.15.5:0/348478833 wait complete. 2024-05-09T00:45:32.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.272+0000 7fb3e77fe700 1 -- 172.21.15.5:0/1286509965 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fb3d8029070 con 0x7fb3f01034c0 2024-05-09T00:45:32.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.273+0000 7fb3e77fe700 1 --2- 172.21.15.5:0/1286509965 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fb3dc0b2910 0x7fb3dc0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:32.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.273+0000 7fb3edd9b700 1 --2- 172.21.15.5:0/1286509965 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fb3dc0b2910 0x7fb3dc0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:32.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.273+0000 7fb3e77fe700 1 -- 172.21.15.5:0/1286509965 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fb3d80e34b0 con 0x7fb3f01034c0 2024-05-09T00:45:32.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.273+0000 7fb3edd9b700 1 --2- 172.21.15.5:0/1286509965 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fb3dc0b2910 0x7fb3dc0b4e20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0106930 tx=0x7fb3e00073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:32.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.276+0000 7fb3e77fe700 1 -- 172.21.15.5:0/1286509965 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb3d8013070 con 0x7fb3f01034c0 2024-05-09T00:45:32.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.500+0000 7fb3f4b2b700 1 -- 172.21.15.5:0/1286509965 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fb3f002e5f0 con 0x7fb3f01034c0 2024-05-09T00:45:32.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.501+0000 7fb3e77fe700 1 -- 172.21.15.5:0/1286509965 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb3d80ab0d0 con 0x7fb3f01034c0 2024-05-09T00:45:32.502 INFO:teuthology.orchestra.run.smithi005.stdout:34359738462 2024-05-09T00:45:32.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.503+0000 7fb3f4b2b700 1 -- 172.21.15.5:0/1286509965 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fb3dc0b2910 msgr2=0x7fb3dc0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:32.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.503+0000 7fb3f4b2b700 1 --2- 172.21.15.5:0/1286509965 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fb3dc0b2910 0x7fb3dc0b4e20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0106930 tx=0x7fb3e00073d0 comp rx=0 tx=0).stop 2024-05-09T00:45:32.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.503+0000 7fb3f4b2b700 1 -- 172.21.15.5:0/1286509965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3f01034c0 msgr2=0x7fb3f019bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:32.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.503+0000 7fb3f4b2b700 1 --2- 172.21.15.5:0/1286509965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3f01034c0 0x7fb3f019bce0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb3d8009990 tx=0x7fb3d80047b0 comp rx=0 tx=0).stop 2024-05-09T00:45:32.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.503+0000 7fb3f4b2b700 1 -- 172.21.15.5:0/1286509965 shutdown_connections 2024-05-09T00:45:32.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.503+0000 7fb3f4b2b700 1 --2- 172.21.15.5:0/1286509965 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fb3dc0b2910 0x7fb3dc0b4e20 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:32.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.503+0000 7fb3f4b2b700 1 --2- 172.21.15.5:0/1286509965 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3f0108740 0x7fb3f019c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:32.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.503+0000 7fb3f4b2b700 1 --2- 172.21.15.5:0/1286509965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3f01034c0 0x7fb3f019bce0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:32.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.503+0000 7fb3f4b2b700 1 --2- 172.21.15.5:0/1286509965 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3f0107df0 0x7fb3f019c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:32.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.503+0000 7fb3f4b2b700 1 -- 172.21.15.5:0/1286509965 >> 172.21.15.5:0/1286509965 conn(0x7fb3f00fedc0 msgr2=0x7fb3f01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:45:32.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.504+0000 7fb3f4b2b700 1 -- 172.21.15.5:0/1286509965 shutdown_connections 2024-05-09T00:45:32.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:32.504+0000 7fb3f4b2b700 1 -- 172.21.15.5:0/1286509965 wait complete. 2024-05-09T00:45:33.234 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:45:32 smithi005 ceph-mon[30365]: pgmap v347: 97 pgs: 97 active+clean; 645 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 64 B/s, 0 objects/s recovering 2024-05-09T00:45:33.234 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:45:32 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/348478833' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-09T00:45:33.234 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:45:32 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/1286509965' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-09T00:45:33.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:45:32 smithi131 ceph-mon[34547]: pgmap v347: 97 pgs: 97 active+clean; 645 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 64 B/s, 0 objects/s recovering 2024-05-09T00:45:33.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:45:32 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/348478833' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-09T00:45:33.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:45:32 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/1286509965' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-09T00:45:33.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:45:32 smithi157 ceph-mon[34294]: pgmap v347: 97 pgs: 97 active+clean; 645 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 64 B/s, 0 objects/s recovering 2024-05-09T00:45:33.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:45:32 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/348478833' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-09T00:45:33.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:45:32 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/1286509965' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-09T00:45:33.898 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822720 got 154618822724 for osd.5 2024-05-09T00:45:33.898 DEBUG:teuthology.parallel:result is None 2024-05-09T00:45:34.357 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738457 got 34359738462 for osd.0 2024-05-09T00:45:34.358 DEBUG:teuthology.parallel:result is None 2024-05-09T00:45:34.784 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182475 got 107374182479 for osd.3 2024-05-09T00:45:34.784 DEBUG:teuthology.parallel:result is None 2024-05-09T00:45:35.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:45:34 smithi005 ceph-mon[30365]: pgmap v348: 97 pgs: 97 active+clean; 645 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 58 B/s, 0 objects/s recovering 2024-05-09T00:45:35.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:45:34 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/2424953901' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-09T00:45:35.238 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:45:34 smithi157 ceph-mon[34294]: pgmap v348: 97 pgs: 97 active+clean; 645 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 58 B/s, 0 objects/s recovering 2024-05-09T00:45:35.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:45:34 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/2424953901' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-09T00:45:35.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:45:34 smithi131 ceph-mon[34547]: pgmap v348: 97 pgs: 97 active+clean; 645 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 58 B/s, 0 objects/s recovering 2024-05-09T00:45:35.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:45:34 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/2424953901' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-09T00:45:35.451 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070506 got 253403070511 for osd.9 2024-05-09T00:45:35.472 DEBUG:teuthology.parallel:result is None 2024-05-09T00:45:36.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:36.218+0000 7f9ff599f700 1 -- 172.21.15.5:0/3700905217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ff0109150 msgr2=0x7f9ff0109560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:36.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:36.218+0000 7f9ff599f700 1 --2- 172.21.15.5:0/3700905217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ff0109150 0x7f9ff0109560 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9fec00b0a0 tx=0x7f9fec00b3b0 comp rx=0 tx=0).stop 2024-05-09T00:45:36.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:36.219+0000 7f9ff599f700 1 -- 172.21.15.5:0/3700905217 shutdown_connections 2024-05-09T00:45:36.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:36.219+0000 7f9ff599f700 1 --2- 172.21.15.5:0/3700905217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ff0102f30 0x7f9ff0110fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:36.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:36.219+0000 7f9ff599f700 1 --2- 172.21.15.5:0/3700905217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ff0109150 0x7f9ff0109560 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:36.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:36.219+0000 7f9ff599f700 1 --2- 172.21.15.5:0/3700905217 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ff01025e0 0x7f9ff01029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:36.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:36.219+0000 7f9ff599f700 1 -- 172.21.15.5:0/3700905217 >> 172.21.15.5:0/3700905217 conn(0x7f9ff0078370 msgr2=0x7f9ff00787b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:45:36.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:36.219+0000 7f9ff599f700 1 -- 172.21.15.5:0/3700905217 shutdown_connections 2024-05-09T00:45:36.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:36.219+0000 7f9ff599f700 1 -- 172.21.15.5:0/3700905217 wait complete. 2024-05-09T00:45:36.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:36.219+0000 7f9ff599f700 1 Processor -- start 2024-05-09T00:45:36.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:36.220+0000 7f9ff599f700 1 -- start start 2024-05-09T00:45:36.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:36.220+0000 7f9ff599f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ff01025e0 0x7f9ff01037c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:36.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:36.220+0000 7f9ff599f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ff0102f30 0x7f9ff0103d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:36.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:36.220+0000 7f9ff599f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ff0109150 0x7f9ff0104240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:36.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:36.220+0000 7f9ff599f700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff01143e0 con 0x7f9ff01025e0 2024-05-09T00:45:36.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:36.220+0000 7f9ff599f700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff0114260 con 0x7f9ff0102f30 2024-05-09T00:45:36.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:36.220+0000 7f9ff599f700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff0114560 con 0x7f9ff0109150 2024-05-09T00:45:36.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:36.220+0000 7f9ff499d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ff01025e0 0x7f9ff01037c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:36.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:36.220+0000 7f9ff499d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ff01025e0 0x7f9ff01037c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37196/0 (socket says 172.21.15.5:37196) 2024-05-09T00:45:36.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:36.220+0000 7f9ff499d700 1 -- 172.21.15.5:0/1749939774 learned_addr learned my addr 172.21.15.5:0/1749939774 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:45:36.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:36.220+0000 7f9fe7fff700 1 --2- 172.21.15.5:0/1749939774 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ff0102f30 0x7f9ff0103d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:36.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:36.220+0000 7f9ff519e700 1 --2- 172.21.15.5:0/1749939774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ff0109150 0x7f9ff0104240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:36.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:36.220+0000 7f9ff499d700 1 -- 172.21.15.5:0/1749939774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ff0109150 msgr2=0x7f9ff0104240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:36.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:36.220+0000 7f9ff499d700 1 --2- 172.21.15.5:0/1749939774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ff0109150 0x7f9ff0104240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:36.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:36.220+0000 7f9ff499d700 1 -- 172.21.15.5:0/1749939774 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ff0102f30 msgr2=0x7f9ff0103d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:36.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:36.220+0000 7f9ff499d700 1 --2- 172.21.15.5:0/1749939774 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ff0102f30 0x7f9ff0103d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:36.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:36.220+0000 7f9ff499d700 1 -- 172.21.15.5:0/1749939774 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ff01a9020 con 0x7f9ff01025e0 2024-05-09T00:45:36.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:36.221+0000 7f9ff499d700 1 --2- 172.21.15.5:0/1749939774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ff01025e0 0x7f9ff01037c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9fec000f80 tx=0x7f9fec004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:36.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:36.221+0000 7f9fe5ffb700 1 -- 172.21.15.5:0/1749939774 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fec021070 con 0x7f9ff01025e0 2024-05-09T00:45:36.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:36.221+0000 7f9ff599f700 1 -- 172.21.15.5:0/1749939774 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ff01a92b0 con 0x7f9ff01025e0 2024-05-09T00:45:36.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:36.221+0000 7f9fe5ffb700 1 -- 172.21.15.5:0/1749939774 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fec007b80 con 0x7f9ff01025e0 2024-05-09T00:45:36.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:36.221+0000 7f9fe5ffb700 1 -- 172.21.15.5:0/1749939774 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fec0052a0 con 0x7f9ff01025e0 2024-05-09T00:45:36.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:36.221+0000 7f9ff599f700 1 -- 172.21.15.5:0/1749939774 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ff01a97b0 con 0x7f9ff01025e0 2024-05-09T00:45:36.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:36.222+0000 7f9ff599f700 1 -- 172.21.15.5:0/1749939774 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ff010c6f0 con 0x7f9ff01025e0 2024-05-09T00:45:36.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:36.224+0000 7f9fe5ffb700 1 -- 172.21.15.5:0/1749939774 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f9fec024070 con 0x7f9ff01025e0 2024-05-09T00:45:36.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:36.224+0000 7f9fe5ffb700 1 --2- 172.21.15.5:0/1749939774 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f9fd80b2910 0x7f9fd80b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:36.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:36.224+0000 7f9fe7fff700 1 --2- 172.21.15.5:0/1749939774 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f9fd80b2910 0x7f9fd80b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:36.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:36.224+0000 7f9fe5ffb700 1 -- 172.21.15.5:0/1749939774 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f9fec00f020 con 0x7f9ff01025e0 2024-05-09T00:45:36.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:36.225+0000 7f9fe7fff700 1 --2- 172.21.15.5:0/1749939774 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f9fd80b2910 0x7f9fd80b4e20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9ff00ff4f0 tx=0x7f9fe000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:36.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:36.228+0000 7f9fe5ffb700 1 -- 172.21.15.5:0/1749939774 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9fec02b020 con 0x7f9ff01025e0 2024-05-09T00:45:36.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:33.780+0000 7fca5c955700 1 -- 172.21.15.5:0/4262777423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca58103550 msgr2=0x7fca58103960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:36.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:33.780+0000 7fca5c955700 1 --2- 172.21.15.5:0/4262777423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca58103550 0x7fca58103960 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fca4800b0d0 tx=0x7fca4800b3e0 comp rx=0 tx=0).stop 2024-05-09T00:45:36.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:33.780+0000 7fca5c955700 1 -- 172.21.15.5:0/4262777423 shutdown_connections 2024-05-09T00:45:36.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:33.780+0000 7fca5c955700 1 --2- 172.21.15.5:0/4262777423 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca58108780 0x7fca581100e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:36.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:33.780+0000 7fca5c955700 1 --2- 172.21.15.5:0/4262777423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca58103550 0x7fca58103960 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:36.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:33.780+0000 7fca5c955700 1 --2- 172.21.15.5:0/4262777423 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca58107e30 0x7fca58108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:36.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:33.780+0000 7fca5c955700 1 -- 172.21.15.5:0/4262777423 >> 172.21.15.5:0/4262777423 conn(0x7fca58078e00 msgr2=0x7fca58079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:45:36.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:33.780+0000 7fca5c955700 1 -- 172.21.15.5:0/4262777423 shutdown_connections 2024-05-09T00:45:36.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:33.780+0000 7fca5c955700 1 -- 172.21.15.5:0/4262777423 wait complete. 2024-05-09T00:45:36.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:33.781+0000 7fca5c955700 1 Processor -- start 2024-05-09T00:45:36.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:33.781+0000 7fca5c955700 1 -- start start 2024-05-09T00:45:36.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:33.782+0000 7fca5c955700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca58103550 0x7fca5819bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:36.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:33.782+0000 7fca5c955700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca58107e30 0x7fca5819c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:36.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:33.782+0000 7fca5c955700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca58108780 0x7fca5819c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:36.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:33.782+0000 7fca5c955700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fca581133b0 con 0x7fca58107e30 2024-05-09T00:45:36.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:33.782+0000 7fca5c955700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fca58113230 con 0x7fca58108780 2024-05-09T00:45:36.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:33.782+0000 7fca5c955700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fca58113530 con 0x7fca58103550 2024-05-09T00:45:36.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:33.782+0000 7fca55d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca58107e30 0x7fca5819c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:36.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:33.782+0000 7fca55d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca58107e30 0x7fca5819c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37170/0 (socket says 172.21.15.5:37170) 2024-05-09T00:45:36.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:33.782+0000 7fca55d9b700 1 -- 172.21.15.5:0/2424953901 learned_addr learned my addr 172.21.15.5:0/2424953901 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:45:36.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:33.782+0000 7fca5659c700 1 --2- 172.21.15.5:0/2424953901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca58103550 0x7fca5819bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:36.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:33.782+0000 7fca56d9d700 1 --2- 172.21.15.5:0/2424953901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca58108780 0x7fca5819c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:36.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:33.782+0000 7fca55d9b700 1 -- 172.21.15.5:0/2424953901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca58103550 msgr2=0x7fca5819bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:36.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:33.782+0000 7fca55d9b700 1 --2- 172.21.15.5:0/2424953901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca58103550 0x7fca5819bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:36.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:33.782+0000 7fca55d9b700 1 -- 172.21.15.5:0/2424953901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca58108780 msgr2=0x7fca5819c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:36.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:33.782+0000 7fca55d9b700 1 --2- 172.21.15.5:0/2424953901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca58108780 0x7fca5819c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:36.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:33.782+0000 7fca55d9b700 1 -- 172.21.15.5:0/2424953901 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca581a70a0 con 0x7fca58107e30 2024-05-09T00:45:36.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:33.782+0000 7fca55d9b700 1 --2- 172.21.15.5:0/2424953901 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca58107e30 0x7fca5819c290 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fca4c00d980 tx=0x7fca4c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:36.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:33.782+0000 7fca477fe700 1 -- 172.21.15.5:0/2424953901 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca4c017070 con 0x7fca58107e30 2024-05-09T00:45:36.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:33.783+0000 7fca477fe700 1 -- 172.21.15.5:0/2424953901 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca4c009de0 con 0x7fca58107e30 2024-05-09T00:45:36.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:33.783+0000 7fca5c955700 1 -- 172.21.15.5:0/2424953901 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca581a7390 con 0x7fca58107e30 2024-05-09T00:45:36.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:33.783+0000 7fca477fe700 1 -- 172.21.15.5:0/2424953901 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca4c00f3c0 con 0x7fca58107e30 2024-05-09T00:45:36.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:33.783+0000 7fca5c955700 1 -- 172.21.15.5:0/2424953901 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca581a7920 con 0x7fca58107e30 2024-05-09T00:45:36.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:33.784+0000 7fca5c955700 1 -- 172.21.15.5:0/2424953901 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca580512c0 con 0x7fca58107e30 2024-05-09T00:45:36.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:33.785+0000 7fca477fe700 1 -- 172.21.15.5:0/2424953901 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fca4c009930 con 0x7fca58107e30 2024-05-09T00:45:36.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:33.786+0000 7fca477fe700 1 --2- 172.21.15.5:0/2424953901 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fca400b2990 0x7fca400b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:36.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:33.786+0000 7fca5659c700 1 --2- 172.21.15.5:0/2424953901 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fca400b2990 0x7fca400b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:36.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:33.786+0000 7fca477fe700 1 -- 172.21.15.5:0/2424953901 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fca4c0e1e30 con 0x7fca58107e30 2024-05-09T00:45:36.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:33.787+0000 7fca5659c700 1 --2- 172.21.15.5:0/2424953901 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fca400b2990 0x7fca400b4ea0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fca48009900 tx=0x7fca4800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:36.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:33.790+0000 7fca477fe700 1 -- 172.21.15.5:0/2424953901 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fca4c0a9a40 con 0x7fca58107e30 2024-05-09T00:45:36.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:34.013+0000 7fca5c955700 1 -- 172.21.15.5:0/2424953901 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fca58075fe0 con 0x7fca58107e30 2024-05-09T00:45:36.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:34.014+0000 7fca477fe700 1 -- 172.21.15.5:0/2424953901 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fca4c0a9340 con 0x7fca58107e30 2024-05-09T00:45:36.314 INFO:teuthology.orchestra.run.smithi005.stdout:227633266741 2024-05-09T00:45:36.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:34.017+0000 7fca5c955700 1 -- 172.21.15.5:0/2424953901 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fca400b2990 msgr2=0x7fca400b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:36.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:34.017+0000 7fca5c955700 1 --2- 172.21.15.5:0/2424953901 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fca400b2990 0x7fca400b4ea0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fca48009900 tx=0x7fca4800b040 comp rx=0 tx=0).stop 2024-05-09T00:45:36.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:34.017+0000 7fca5c955700 1 -- 172.21.15.5:0/2424953901 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca58107e30 msgr2=0x7fca5819c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:36.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:34.017+0000 7fca5c955700 1 --2- 172.21.15.5:0/2424953901 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca58107e30 0x7fca5819c290 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fca4c00d980 tx=0x7fca4c00dc90 comp rx=0 tx=0).stop 2024-05-09T00:45:36.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:34.017+0000 7fca5c955700 1 -- 172.21.15.5:0/2424953901 shutdown_connections 2024-05-09T00:45:36.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:34.017+0000 7fca5c955700 1 --2- 172.21.15.5:0/2424953901 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fca400b2990 0x7fca400b4ea0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:36.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:34.017+0000 7fca5c955700 1 --2- 172.21.15.5:0/2424953901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca58108780 0x7fca5819c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:36.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:34.017+0000 7fca5c955700 1 --2- 172.21.15.5:0/2424953901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca58103550 0x7fca5819bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:36.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:34.017+0000 7fca5c955700 1 --2- 172.21.15.5:0/2424953901 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca58107e30 0x7fca5819c290 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:36.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:34.017+0000 7fca5c955700 1 -- 172.21.15.5:0/2424953901 >> 172.21.15.5:0/2424953901 conn(0x7fca58078e00 msgr2=0x7fca58101010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:45:36.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:34.017+0000 7fca5c955700 1 -- 172.21.15.5:0/2424953901 shutdown_connections 2024-05-09T00:45:36.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:34.017+0000 7fca5c955700 1 -- 172.21.15.5:0/2424953901 wait complete. 2024-05-09T00:45:36.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:36.443+0000 7f9ff599f700 1 -- 172.21.15.5:0/1749939774 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f9ff0104cf0 con 0x7f9ff01025e0 2024-05-09T00:45:36.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:36.443+0000 7f9fe5ffb700 1 -- 172.21.15.5:0/1749939774 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9fec0aba10 con 0x7f9ff01025e0 2024-05-09T00:45:36.445 INFO:teuthology.orchestra.run.smithi005.stdout:55834574938 2024-05-09T00:45:36.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:36.446+0000 7f9ff599f700 1 -- 172.21.15.5:0/1749939774 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f9fd80b2910 msgr2=0x7f9fd80b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:36.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:36.446+0000 7f9ff599f700 1 --2- 172.21.15.5:0/1749939774 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f9fd80b2910 0x7f9fd80b4e20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9ff00ff4f0 tx=0x7f9fe000b410 comp rx=0 tx=0).stop 2024-05-09T00:45:36.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:36.446+0000 7f9ff599f700 1 -- 172.21.15.5:0/1749939774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ff01025e0 msgr2=0x7f9ff01037c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:36.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:36.446+0000 7f9ff599f700 1 --2- 172.21.15.5:0/1749939774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ff01025e0 0x7f9ff01037c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9fec000f80 tx=0x7f9fec004ce0 comp rx=0 tx=0).stop 2024-05-09T00:45:36.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:36.446+0000 7f9ff599f700 1 -- 172.21.15.5:0/1749939774 shutdown_connections 2024-05-09T00:45:36.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:36.446+0000 7f9ff599f700 1 --2- 172.21.15.5:0/1749939774 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f9fd80b2910 0x7f9fd80b4e20 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:36.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:36.446+0000 7f9ff599f700 1 --2- 172.21.15.5:0/1749939774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ff0109150 0x7f9ff0104240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:36.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:36.446+0000 7f9ff599f700 1 --2- 172.21.15.5:0/1749939774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ff01025e0 0x7f9ff01037c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:36.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:36.446+0000 7f9ff599f700 1 --2- 172.21.15.5:0/1749939774 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ff0102f30 0x7f9ff0103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:36.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:36.446+0000 7f9ff599f700 1 -- 172.21.15.5:0/1749939774 >> 172.21.15.5:0/1749939774 conn(0x7f9ff0078370 msgr2=0x7f9ff00ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:45:36.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:36.446+0000 7f9ff599f700 1 -- 172.21.15.5:0/1749939774 shutdown_connections 2024-05-09T00:45:36.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:36.446+0000 7f9ff599f700 1 -- 172.21.15.5:0/1749939774 wait complete. 2024-05-09T00:45:37.218 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:45:36 smithi005 ceph-mon[30365]: pgmap v349: 97 pgs: 97 active+clean; 645 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s; 48 B/s, 0 objects/s recovering 2024-05-09T00:45:37.219 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:45:36 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/1749939774' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-09T00:45:37.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:45:36 smithi131 ceph-mon[34547]: pgmap v349: 97 pgs: 97 active+clean; 645 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s; 48 B/s, 0 objects/s recovering 2024-05-09T00:45:37.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:45:36 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/1749939774' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-09T00:45:37.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:45:36 smithi157 ceph-mon[34294]: pgmap v349: 97 pgs: 97 active+clean; 645 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s; 48 B/s, 0 objects/s recovering 2024-05-09T00:45:37.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:45:36 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/1749939774' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-09T00:45:37.542 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574932 got 55834574938 for osd.1 2024-05-09T00:45:37.542 DEBUG:teuthology.parallel:result is None 2024-05-09T00:45:37.732 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:45:37.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:37.732+0000 7f745dce4700 1 -- 172.21.15.5:0/570804951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7458110bf0 msgr2=0x7f7458113010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:37.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:37.732+0000 7f745dce4700 1 --2- 172.21.15.5:0/570804951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7458110bf0 0x7f7458113010 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f745000b0a0 tx=0x7f745000b3b0 comp rx=0 tx=0).stop 2024-05-09T00:45:37.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:37.732+0000 7f745dce4700 1 -- 172.21.15.5:0/570804951 shutdown_connections 2024-05-09T00:45:37.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:37.732+0000 7f745dce4700 1 --2- 172.21.15.5:0/570804951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7458110bf0 0x7f7458113010 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:37.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:37.732+0000 7f745dce4700 1 --2- 172.21.15.5:0/570804951 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7458075b00 0x7f7458075f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:37.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:37.732+0000 7f745dce4700 1 --2- 172.21.15.5:0/570804951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7458076450 0x7f74581106b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:37.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:37.732+0000 7f745dce4700 1 -- 172.21.15.5:0/570804951 >> 172.21.15.5:0/570804951 conn(0x7f74580ffe70 msgr2=0x7f74581022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:45:37.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:37.732+0000 7f745dce4700 1 -- 172.21.15.5:0/570804951 shutdown_connections 2024-05-09T00:45:37.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:37.732+0000 7f745dce4700 1 -- 172.21.15.5:0/570804951 wait complete. 2024-05-09T00:45:37.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:37.733+0000 7f745dce4700 1 Processor -- start 2024-05-09T00:45:37.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:37.733+0000 7f745dce4700 1 -- start start 2024-05-09T00:45:37.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:37.733+0000 7f745dce4700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7458075b00 0x7f745819d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:37.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:37.734+0000 7f745dce4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7458076450 0x7f745819d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:37.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:37.734+0000 7f745dce4700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7458110bf0 0x7f74581a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:37.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:37.734+0000 7f745dce4700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f74581166b0 con 0x7f7458076450 2024-05-09T00:45:37.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:37.734+0000 7f745dce4700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7458116530 con 0x7f7458110bf0 2024-05-09T00:45:37.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:37.734+0000 7f745dce4700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7458116830 con 0x7f7458075b00 2024-05-09T00:45:37.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:37.734+0000 7f7456ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7458076450 0x7f745819d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:37.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:37.734+0000 7f7456ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7458076450 0x7f745819d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37214/0 (socket says 172.21.15.5:37214) 2024-05-09T00:45:37.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:37.734+0000 7f7456ffd700 1 -- 172.21.15.5:0/1393405929 learned_addr learned my addr 172.21.15.5:0/1393405929 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:45:37.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:37.734+0000 7f74577fe700 1 --2- 172.21.15.5:0/1393405929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7458075b00 0x7f745819d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:37.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:37.734+0000 7f7457fff700 1 --2- 172.21.15.5:0/1393405929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7458110bf0 0x7f74581a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:37.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:37.734+0000 7f7456ffd700 1 -- 172.21.15.5:0/1393405929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7458075b00 msgr2=0x7f745819d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:37.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:37.734+0000 7f7456ffd700 1 --2- 172.21.15.5:0/1393405929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7458075b00 0x7f745819d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:37.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:37.734+0000 7f7456ffd700 1 -- 172.21.15.5:0/1393405929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7458110bf0 msgr2=0x7f74581a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:37.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:37.734+0000 7f7456ffd700 1 --2- 172.21.15.5:0/1393405929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7458110bf0 0x7f74581a1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:37.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:37.734+0000 7f7456ffd700 1 -- 172.21.15.5:0/1393405929 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74581a1ed0 con 0x7f7458076450 2024-05-09T00:45:37.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:37.735+0000 7f7456ffd700 1 --2- 172.21.15.5:0/1393405929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7458076450 0x7f745819d5a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f744c00e970 tx=0x7f744c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:37.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:37.735+0000 7f7454ff9700 1 -- 172.21.15.5:0/1393405929 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744c004a30 con 0x7f7458076450 2024-05-09T00:45:37.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:37.735+0000 7f7454ff9700 1 -- 172.21.15.5:0/1393405929 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f744c00cc90 con 0x7f7458076450 2024-05-09T00:45:37.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:37.735+0000 7f745dce4700 1 -- 172.21.15.5:0/1393405929 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74581a21c0 con 0x7f7458076450 2024-05-09T00:45:37.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:37.736+0000 7f7454ff9700 1 -- 172.21.15.5:0/1393405929 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744c005610 con 0x7f7458076450 2024-05-09T00:45:37.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:37.736+0000 7f745dce4700 1 -- 172.21.15.5:0/1393405929 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74581a9ae0 con 0x7f7458076450 2024-05-09T00:45:37.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:37.737+0000 7f745dce4700 1 -- 172.21.15.5:0/1393405929 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74580512c0 con 0x7f7458076450 2024-05-09T00:45:37.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:37.739+0000 7f7454ff9700 1 -- 172.21.15.5:0/1393405929 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f744c00c7e0 con 0x7f7458076450 2024-05-09T00:45:37.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:37.740+0000 7f7454ff9700 1 --2- 172.21.15.5:0/1393405929 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f74400b2990 0x7f74400b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:37.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:37.740+0000 7f74577fe700 1 --2- 172.21.15.5:0/1393405929 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f74400b2990 0x7f74400b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:37.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:37.740+0000 7f7454ff9700 1 -- 172.21.15.5:0/1393405929 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f744c0e2600 con 0x7f7458076450 2024-05-09T00:45:37.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:37.741+0000 7f74577fe700 1 --2- 172.21.15.5:0/1393405929 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f74400b2990 0x7f74400b4ea0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7448007be0 tx=0x7f74480073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:37.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:37.745+0000 7f7454ff9700 1 -- 172.21.15.5:0/1393405929 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f744c0aa190 con 0x7f7458076450 2024-05-09T00:45:37.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:37.968+0000 7f745dce4700 1 -- 172.21.15.5:0/1393405929 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f745819e360 con 0x7f7458076450 2024-05-09T00:45:37.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:37.968+0000 7f7454ff9700 1 -- 172.21.15.5:0/1393405929 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f744c0a5020 con 0x7f7458076450 2024-05-09T00:45:37.970 INFO:teuthology.orchestra.run.smithi005.stdout:180388626497 2024-05-09T00:45:37.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:37.971+0000 7f745dce4700 1 -- 172.21.15.5:0/1393405929 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f74400b2990 msgr2=0x7f74400b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:37.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:37.971+0000 7f745dce4700 1 --2- 172.21.15.5:0/1393405929 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f74400b2990 0x7f74400b4ea0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7448007be0 tx=0x7f74480073d0 comp rx=0 tx=0).stop 2024-05-09T00:45:37.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:37.971+0000 7f745dce4700 1 -- 172.21.15.5:0/1393405929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7458076450 msgr2=0x7f745819d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:37.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:37.971+0000 7f745dce4700 1 --2- 172.21.15.5:0/1393405929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7458076450 0x7f745819d5a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f744c00e970 tx=0x7f744c00ec80 comp rx=0 tx=0).stop 2024-05-09T00:45:37.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:37.971+0000 7f745dce4700 1 -- 172.21.15.5:0/1393405929 shutdown_connections 2024-05-09T00:45:37.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:37.971+0000 7f745dce4700 1 --2- 172.21.15.5:0/1393405929 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f74400b2990 0x7f74400b4ea0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:37.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:37.971+0000 7f745dce4700 1 --2- 172.21.15.5:0/1393405929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7458110bf0 0x7f74581a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:37.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:37.971+0000 7f745dce4700 1 --2- 172.21.15.5:0/1393405929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7458075b00 0x7f745819d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:37.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:37.971+0000 7f745dce4700 1 --2- 172.21.15.5:0/1393405929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7458076450 0x7f745819d5a0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:37.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:37.971+0000 7f745dce4700 1 -- 172.21.15.5:0/1393405929 >> 172.21.15.5:0/1393405929 conn(0x7f74580ffe70 msgr2=0x7f7458100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:45:37.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:37.971+0000 7f745dce4700 1 -- 172.21.15.5:0/1393405929 shutdown_connections 2024-05-09T00:45:37.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:37.971+0000 7f745dce4700 1 -- 172.21.15.5:0/1393405929 wait complete. 2024-05-09T00:45:38.214 INFO:tasks.cephadm.ceph_manager.ceph:need seq 227633266736 got 227633266741 for osd.8 2024-05-09T00:45:38.215 DEBUG:teuthology.parallel:result is None 2024-05-09T00:45:39.040 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:45:38 smithi005 ceph-mon[30365]: pgmap v350: 97 pgs: 97 active+clean; 645 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T00:45:39.040 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:45:38 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/1393405929' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-09T00:45:39.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:45:38 smithi131 ceph-mon[34547]: pgmap v350: 97 pgs: 97 active+clean; 645 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T00:45:39.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:45:38 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/1393405929' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-09T00:45:39.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:45:38 smithi157 ceph-mon[34294]: pgmap v350: 97 pgs: 97 active+clean; 645 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T00:45:39.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:45:38 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/1393405929' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-09T00:45:39.944 INFO:tasks.cephadm.ceph_manager.ceph:need seq 180388626491 got 180388626497 for osd.6 2024-05-09T00:45:39.945 DEBUG:teuthology.parallel:result is None 2024-05-09T00:45:41.040 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:45:40 smithi005 ceph-mon[30365]: pgmap v351: 97 pgs: 97 active+clean; 645 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-09T00:45:41.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.176+0000 7fc015215700 1 -- 172.21.15.5:0/1323287479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0101034c0 msgr2=0x7fc0101038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:41.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.176+0000 7fc015215700 1 --2- 172.21.15.5:0/1323287479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0101034c0 0x7fc0101038d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc0000099c0 tx=0x7fc000009cd0 comp rx=0 tx=0).stop 2024-05-09T00:45:41.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.176+0000 7fc015215700 1 -- 172.21.15.5:0/1323287479 shutdown_connections 2024-05-09T00:45:41.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.176+0000 7fc015215700 1 --2- 172.21.15.5:0/1323287479 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc010108740 0x7fc0101100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:41.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.176+0000 7fc015215700 1 --2- 172.21.15.5:0/1323287479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0101034c0 0x7fc0101038d0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:41.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.176+0000 7fc015215700 1 --2- 172.21.15.5:0/1323287479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc010107df0 0x7fc010108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:41.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.176+0000 7fc015215700 1 -- 172.21.15.5:0/1323287479 >> 172.21.15.5:0/1323287479 conn(0x7fc0100fedf0 msgr2=0x7fc010101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:45:41.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.176+0000 7fc015215700 1 -- 172.21.15.5:0/1323287479 shutdown_connections 2024-05-09T00:45:41.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.176+0000 7fc015215700 1 -- 172.21.15.5:0/1323287479 wait complete. 2024-05-09T00:45:41.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.177+0000 7fc015215700 1 Processor -- start 2024-05-09T00:45:41.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.177+0000 7fc015215700 1 -- start start 2024-05-09T00:45:41.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.177+0000 7fc015215700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0101034c0 0x7fc01019d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:41.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.177+0000 7fc015215700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc010107df0 0x7fc01019d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:41.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.178+0000 7fc015215700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc010108740 0x7fc0101a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:41.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.178+0000 7fc015215700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc0101144e0 con 0x7fc010107df0 2024-05-09T00:45:41.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.178+0000 7fc015215700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc010114360 con 0x7fc010108740 2024-05-09T00:45:41.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.178+0000 7fc015215700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc010114660 con 0x7fc0101034c0 2024-05-09T00:45:41.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.178+0000 7fc00e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc010107df0 0x7fc01019d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:41.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.178+0000 7fc00e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc010107df0 0x7fc01019d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37234/0 (socket says 172.21.15.5:37234) 2024-05-09T00:45:41.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.178+0000 7fc00e59c700 1 -- 172.21.15.5:0/3515883878 learned_addr learned my addr 172.21.15.5:0/3515883878 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:45:41.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.178+0000 7fc00ed9d700 1 --2- 172.21.15.5:0/3515883878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0101034c0 0x7fc01019d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:41.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.178+0000 7fc00f59e700 1 --2- 172.21.15.5:0/3515883878 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc010108740 0x7fc0101a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:41.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.178+0000 7fc00e59c700 1 -- 172.21.15.5:0/3515883878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0101034c0 msgr2=0x7fc01019d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:41.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.178+0000 7fc00e59c700 1 --2- 172.21.15.5:0/3515883878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0101034c0 0x7fc01019d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:41.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.178+0000 7fc00e59c700 1 -- 172.21.15.5:0/3515883878 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc010108740 msgr2=0x7fc0101a18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:41.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.178+0000 7fc00e59c700 1 --2- 172.21.15.5:0/3515883878 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc010108740 0x7fc0101a18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:41.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.178+0000 7fc00e59c700 1 -- 172.21.15.5:0/3515883878 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0101a1f80 con 0x7fc010107df0 2024-05-09T00:45:41.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.178+0000 7fc00f59e700 1 --2- 172.21.15.5:0/3515883878 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc010108740 0x7fc0101a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:45:41.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.178+0000 7fc00e59c700 1 --2- 172.21.15.5:0/3515883878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc010107df0 0x7fc01019d650 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc00400b500 tx=0x7fc00400b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:41.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.178+0000 7fbff7fff700 1 -- 172.21.15.5:0/3515883878 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc004015020 con 0x7fc010107df0 2024-05-09T00:45:41.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.178+0000 7fc015215700 1 -- 172.21.15.5:0/3515883878 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0101a2270 con 0x7fc010107df0 2024-05-09T00:45:41.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.178+0000 7fbff7fff700 1 -- 172.21.15.5:0/3515883878 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc004011ce0 con 0x7fc010107df0 2024-05-09T00:45:41.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.178+0000 7fbff7fff700 1 -- 172.21.15.5:0/3515883878 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc004010d40 con 0x7fc010107df0 2024-05-09T00:45:41.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.179+0000 7fc015215700 1 -- 172.21.15.5:0/3515883878 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0101a9b90 con 0x7fc010107df0 2024-05-09T00:45:41.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.180+0000 7fc015215700 1 -- 172.21.15.5:0/3515883878 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0100512c0 con 0x7fc010107df0 2024-05-09T00:45:41.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.181+0000 7fbff7fff700 1 -- 172.21.15.5:0/3515883878 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fc004018020 con 0x7fc010107df0 2024-05-09T00:45:41.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.182+0000 7fbff7fff700 1 --2- 172.21.15.5:0/3515883878 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fbff80b2910 0x7fbff80b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:41.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.182+0000 7fc00ed9d700 1 --2- 172.21.15.5:0/3515883878 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fbff80b2910 0x7fbff80b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:41.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.182+0000 7fbff7fff700 1 -- 172.21.15.5:0/3515883878 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fc0040e1a40 con 0x7fc010107df0 2024-05-09T00:45:41.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.182+0000 7fc00ed9d700 1 --2- 172.21.15.5:0/3515883878 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fbff80b2910 0x7fbff80b4e20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc000009990 tx=0x7fc000000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:41.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.187+0000 7fbff7fff700 1 -- 172.21.15.5:0/3515883878 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc00401f020 con 0x7fc010107df0 2024-05-09T00:45:41.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:45:40 smithi131 ceph-mon[34547]: pgmap v351: 97 pgs: 97 active+clean; 645 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-09T00:45:41.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.404+0000 7fc015215700 1 -- 172.21.15.5:0/3515883878 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fc0101a9dc0 con 0x7fc010107df0 2024-05-09T00:45:41.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.405+0000 7fbff7fff700 1 -- 172.21.15.5:0/3515883878 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fc0040a96e0 con 0x7fc010107df0 2024-05-09T00:45:41.406 INFO:teuthology.orchestra.run.smithi005.stdout:274877906989 2024-05-09T00:45:41.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.407+0000 7fc015215700 1 -- 172.21.15.5:0/3515883878 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fbff80b2910 msgr2=0x7fbff80b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:41.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.407+0000 7fc015215700 1 --2- 172.21.15.5:0/3515883878 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fbff80b2910 0x7fbff80b4e20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc000009990 tx=0x7fc000000bc0 comp rx=0 tx=0).stop 2024-05-09T00:45:41.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.407+0000 7fc015215700 1 -- 172.21.15.5:0/3515883878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc010107df0 msgr2=0x7fc01019d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:41.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.407+0000 7fc015215700 1 --2- 172.21.15.5:0/3515883878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc010107df0 0x7fc01019d650 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc00400b500 tx=0x7fc00400b810 comp rx=0 tx=0).stop 2024-05-09T00:45:41.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.407+0000 7fc015215700 1 -- 172.21.15.5:0/3515883878 shutdown_connections 2024-05-09T00:45:41.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.407+0000 7fc015215700 1 --2- 172.21.15.5:0/3515883878 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fbff80b2910 0x7fbff80b4e20 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:41.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.407+0000 7fc015215700 1 --2- 172.21.15.5:0/3515883878 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc010108740 0x7fc0101a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:41.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.407+0000 7fc015215700 1 --2- 172.21.15.5:0/3515883878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0101034c0 0x7fc01019d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:41.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.407+0000 7fc015215700 1 --2- 172.21.15.5:0/3515883878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc010107df0 0x7fc01019d650 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:41.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.408+0000 7fc015215700 1 -- 172.21.15.5:0/3515883878 >> 172.21.15.5:0/3515883878 conn(0x7fc0100fedf0 msgr2=0x7fc0100ffc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:45:41.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.408+0000 7fc015215700 1 -- 172.21.15.5:0/3515883878 shutdown_connections 2024-05-09T00:45:41.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.408+0000 7fc015215700 1 -- 172.21.15.5:0/3515883878 wait complete. 2024-05-09T00:45:41.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.420+0000 7f96de6fc700 1 -- 172.21.15.5:0/4091998449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96d8108750 msgr2=0x7f96d81100b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:41.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.420+0000 7f96de6fc700 1 --2- 172.21.15.5:0/4091998449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96d8108750 0x7f96d81100b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f96d000b0d0 tx=0x7f96d000b3e0 comp rx=0 tx=0).stop 2024-05-09T00:45:41.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.420+0000 7f96de6fc700 1 -- 172.21.15.5:0/4091998449 shutdown_connections 2024-05-09T00:45:41.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.420+0000 7f96de6fc700 1 --2- 172.21.15.5:0/4091998449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96d8108750 0x7f96d81100b0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:41.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.420+0000 7f96de6fc700 1 --2- 172.21.15.5:0/4091998449 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96d8103bc0 0x7f96d8103fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:41.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.420+0000 7f96de6fc700 1 --2- 172.21.15.5:0/4091998449 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96d8107e00 0x7f96d8108210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:41.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.420+0000 7f96de6fc700 1 -- 172.21.15.5:0/4091998449 >> 172.21.15.5:0/4091998449 conn(0x7f96d80fe6f0 msgr2=0x7f96d8100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:45:41.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.420+0000 7f96de6fc700 1 -- 172.21.15.5:0/4091998449 shutdown_connections 2024-05-09T00:45:41.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.420+0000 7f96de6fc700 1 -- 172.21.15.5:0/4091998449 wait complete. 2024-05-09T00:45:41.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.421+0000 7f96de6fc700 1 Processor -- start 2024-05-09T00:45:41.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.421+0000 7f96de6fc700 1 -- start start 2024-05-09T00:45:41.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.421+0000 7f96de6fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96d8103bc0 0x7f96d819bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:41.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.421+0000 7f96de6fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96d8107e00 0x7f96d819c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:41.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.421+0000 7f96de6fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96d8108750 0x7f96d81a6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:41.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.421+0000 7f96de6fc700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f96d81133d0 con 0x7f96d8107e00 2024-05-09T00:45:41.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.421+0000 7f96de6fc700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f96d8113250 con 0x7f96d8108750 2024-05-09T00:45:41.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.421+0000 7f96de6fc700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f96d8113550 con 0x7f96d8103bc0 2024-05-09T00:45:41.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.421+0000 7f96d77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96d8107e00 0x7f96d819c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:41.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.421+0000 7f96d77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96d8107e00 0x7f96d819c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37246/0 (socket says 172.21.15.5:37246) 2024-05-09T00:45:41.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.421+0000 7f96d77fe700 1 -- 172.21.15.5:0/3126306607 learned_addr learned my addr 172.21.15.5:0/3126306607 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:45:41.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.422+0000 7f96d7fff700 1 --2- 172.21.15.5:0/3126306607 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96d8103bc0 0x7f96d819bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:41.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.422+0000 7f96dcc99700 1 --2- 172.21.15.5:0/3126306607 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96d8108750 0x7f96d81a6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:41.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.422+0000 7f96d77fe700 1 -- 172.21.15.5:0/3126306607 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96d8103bc0 msgr2=0x7f96d819bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:41.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.422+0000 7f96d77fe700 1 --2- 172.21.15.5:0/3126306607 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96d8103bc0 0x7f96d819bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:41.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.422+0000 7f96d77fe700 1 -- 172.21.15.5:0/3126306607 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96d8108750 msgr2=0x7f96d81a6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:41.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.422+0000 7f96d77fe700 1 --2- 172.21.15.5:0/3126306607 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96d8108750 0x7f96d81a6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:41.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.422+0000 7f96d77fe700 1 -- 172.21.15.5:0/3126306607 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96d819c930 con 0x7f96d8107e00 2024-05-09T00:45:41.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.422+0000 7f96dcc99700 1 --2- 172.21.15.5:0/3126306607 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96d8108750 0x7f96d81a6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:45:41.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.422+0000 7f96d77fe700 1 --2- 172.21.15.5:0/3126306607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96d8107e00 0x7f96d819c110 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f96c000c9e0 tx=0x7f96c000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:41.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.422+0000 7f96d57fa700 1 -- 172.21.15.5:0/3126306607 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c0007aa0 con 0x7f96d8107e00 2024-05-09T00:45:41.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.422+0000 7f96d57fa700 1 -- 172.21.15.5:0/3126306607 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96c0007c40 con 0x7f96d8107e00 2024-05-09T00:45:41.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.422+0000 7f96de6fc700 1 -- 172.21.15.5:0/3126306607 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96d81a71f0 con 0x7f96d8107e00 2024-05-09T00:45:41.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.422+0000 7f96d57fa700 1 -- 172.21.15.5:0/3126306607 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c0018660 con 0x7f96d8107e00 2024-05-09T00:45:41.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.422+0000 7f96de6fc700 1 -- 172.21.15.5:0/3126306607 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96d81a7730 con 0x7f96d8107e00 2024-05-09T00:45:41.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.424+0000 7f96de6fc700 1 -- 172.21.15.5:0/3126306607 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96d80512c0 con 0x7f96d8107e00 2024-05-09T00:45:41.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.425+0000 7f96d57fa700 1 -- 172.21.15.5:0/3126306607 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f96c0018800 con 0x7f96d8107e00 2024-05-09T00:45:41.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.426+0000 7f96d57fa700 1 --2- 172.21.15.5:0/3126306607 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f96c40b2910 0x7f96c40b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:41.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.426+0000 7f96d7fff700 1 --2- 172.21.15.5:0/3126306607 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f96c40b2910 0x7f96c40b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:41.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.426+0000 7f96d57fa700 1 -- 172.21.15.5:0/3126306607 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f96c00e25a0 con 0x7f96d8107e00 2024-05-09T00:45:41.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.427+0000 7f96d7fff700 1 --2- 172.21.15.5:0/3126306607 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f96c40b2910 0x7f96c40b4e20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f96c8007be0 tx=0x7f96c80073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:41.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.430+0000 7f96d57fa700 1 -- 172.21.15.5:0/3126306607 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f96c0014030 con 0x7f96d8107e00 2024-05-09T00:45:41.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:45:40 smithi157 ceph-mon[34294]: pgmap v351: 97 pgs: 97 active+clean; 645 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-09T00:45:41.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.649+0000 7f96de6fc700 1 -- 172.21.15.5:0/3126306607 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f96d802e5f0 con 0x7f96d8107e00 2024-05-09T00:45:41.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.650+0000 7f96d57fa700 1 -- 172.21.15.5:0/3126306607 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f96c00aa1c0 con 0x7f96d8107e00 2024-05-09T00:45:41.651 INFO:teuthology.orchestra.run.smithi005.stdout:300647710759 2024-05-09T00:45:41.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.652+0000 7f96de6fc700 1 -- 172.21.15.5:0/3126306607 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f96c40b2910 msgr2=0x7f96c40b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:41.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.652+0000 7f96de6fc700 1 --2- 172.21.15.5:0/3126306607 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f96c40b2910 0x7f96c40b4e20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f96c8007be0 tx=0x7f96c80073d0 comp rx=0 tx=0).stop 2024-05-09T00:45:41.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.652+0000 7f96de6fc700 1 -- 172.21.15.5:0/3126306607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96d8107e00 msgr2=0x7f96d819c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:41.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.652+0000 7f96de6fc700 1 --2- 172.21.15.5:0/3126306607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96d8107e00 0x7f96d819c110 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f96c000c9e0 tx=0x7f96c000ccf0 comp rx=0 tx=0).stop 2024-05-09T00:45:41.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.652+0000 7f96de6fc700 1 -- 172.21.15.5:0/3126306607 shutdown_connections 2024-05-09T00:45:41.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.652+0000 7f96de6fc700 1 --2- 172.21.15.5:0/3126306607 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f96c40b2910 0x7f96c40b4e20 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:41.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.652+0000 7f96de6fc700 1 --2- 172.21.15.5:0/3126306607 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96d8108750 0x7f96d81a6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:41.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.652+0000 7f96de6fc700 1 --2- 172.21.15.5:0/3126306607 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96d8103bc0 0x7f96d819bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:41.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.652+0000 7f96de6fc700 1 --2- 172.21.15.5:0/3126306607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96d8107e00 0x7f96d819c110 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:41.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.652+0000 7f96de6fc700 1 -- 172.21.15.5:0/3126306607 >> 172.21.15.5:0/3126306607 conn(0x7f96d80fe6f0 msgr2=0x7f96d81074b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:45:41.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.652+0000 7f96de6fc700 1 -- 172.21.15.5:0/3126306607 shutdown_connections 2024-05-09T00:45:41.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:41.653+0000 7f96de6fc700 1 -- 172.21.15.5:0/3126306607 wait complete. 2024-05-09T00:45:42.113 INFO:tasks.cephadm.ceph_manager.ceph:need seq 274877906984 got 274877906989 for osd.10 2024-05-09T00:45:42.113 DEBUG:teuthology.parallel:result is None 2024-05-09T00:45:42.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:45:41 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/3515883878' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-09T00:45:42.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:45:41 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/3126306607' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-09T00:45:42.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:45:41 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/3515883878' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-09T00:45:42.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:45:41 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/3126306607' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-09T00:45:42.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:45:41 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/3515883878' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-09T00:45:42.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:45:41 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/3126306607' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-09T00:45:42.728 INFO:tasks.cephadm.ceph_manager.ceph:need seq 300647710753 got 300647710759 for osd.11 2024-05-09T00:45:42.728 DEBUG:teuthology.parallel:result is None 2024-05-09T00:45:42.728 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-09T00:45:42.729 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-09T00:45:42.987 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:45:43.252 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:45:42 smithi005 ceph-mon[30365]: pgmap v352: 97 pgs: 97 active+clean; 645 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T00:45:43.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:45:42 smithi131 ceph-mon[34547]: pgmap v352: 97 pgs: 97 active+clean; 645 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T00:45:43.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:45:42 smithi157 ceph-mon[34294]: pgmap v352: 97 pgs: 97 active+clean; 645 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T00:45:44.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:44.736+0000 7fd2056e7700 1 -- 172.21.15.5:0/3227262291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd200107df0 msgr2=0x7fd200108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:44.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:44.736+0000 7fd2056e7700 1 --2- 172.21.15.5:0/3227262291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd200107df0 0x7fd200108200 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd1f40099c0 tx=0x7fd1f4009cd0 comp rx=0 tx=0).stop 2024-05-09T00:45:44.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:44.736+0000 7fd2056e7700 1 -- 172.21.15.5:0/3227262291 shutdown_connections 2024-05-09T00:45:44.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:44.736+0000 7fd2056e7700 1 --2- 172.21.15.5:0/3227262291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd200108740 0x7fd2001100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:44.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:44.736+0000 7fd2056e7700 1 --2- 172.21.15.5:0/3227262291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2001034c0 0x7fd2001038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:44.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:44.736+0000 7fd2056e7700 1 --2- 172.21.15.5:0/3227262291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd200107df0 0x7fd200108200 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:44.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:44.736+0000 7fd2056e7700 1 -- 172.21.15.5:0/3227262291 >> 172.21.15.5:0/3227262291 conn(0x7fd2000fedc0 msgr2=0x7fd2001011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:45:44.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:44.736+0000 7fd2056e7700 1 -- 172.21.15.5:0/3227262291 shutdown_connections 2024-05-09T00:45:44.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:44.736+0000 7fd2056e7700 1 -- 172.21.15.5:0/3227262291 wait complete. 2024-05-09T00:45:44.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:44.737+0000 7fd2056e7700 1 Processor -- start 2024-05-09T00:45:44.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:44.737+0000 7fd2056e7700 1 -- start start 2024-05-09T00:45:44.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:44.737+0000 7fd2056e7700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2001034c0 0x7fd20019d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:44.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:44.737+0000 7fd2056e7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd200107df0 0x7fd20019d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:44.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:44.737+0000 7fd2056e7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd200108740 0x7fd2001a1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:44.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:44.738+0000 7fd2056e7700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd200113350 con 0x7fd200107df0 2024-05-09T00:45:44.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:44.738+0000 7fd2056e7700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd2001131d0 con 0x7fd200108740 2024-05-09T00:45:44.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:44.738+0000 7fd2056e7700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd2001134d0 con 0x7fd2001034c0 2024-05-09T00:45:44.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:44.738+0000 7fd1fe7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd200107df0 0x7fd20019d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:44.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:44.738+0000 7fd1fe7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd200107df0 0x7fd20019d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39672/0 (socket says 172.21.15.5:39672) 2024-05-09T00:45:44.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:44.738+0000 7fd1fe7fc700 1 -- 172.21.15.5:0/2572490001 learned_addr learned my addr 172.21.15.5:0/2572490001 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:45:44.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:44.738+0000 7fd1feffd700 1 --2- 172.21.15.5:0/2572490001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2001034c0 0x7fd20019d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:44.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:44.738+0000 7fd1ff7fe700 1 --2- 172.21.15.5:0/2572490001 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd200108740 0x7fd2001a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:44.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:44.738+0000 7fd1fe7fc700 1 -- 172.21.15.5:0/2572490001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2001034c0 msgr2=0x7fd20019d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:44.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:44.738+0000 7fd1fe7fc700 1 --2- 172.21.15.5:0/2572490001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2001034c0 0x7fd20019d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:44.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:44.738+0000 7fd1fe7fc700 1 -- 172.21.15.5:0/2572490001 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd200108740 msgr2=0x7fd2001a1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:44.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:44.738+0000 7fd1fe7fc700 1 --2- 172.21.15.5:0/2572490001 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd200108740 0x7fd2001a1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:44.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:44.738+0000 7fd1fe7fc700 1 -- 172.21.15.5:0/2572490001 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2001a1ee0 con 0x7fd200107df0 2024-05-09T00:45:44.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:44.738+0000 7fd1feffd700 1 --2- 172.21.15.5:0/2572490001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2001034c0 0x7fd20019d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:45:44.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:44.738+0000 7fd1fe7fc700 1 --2- 172.21.15.5:0/2572490001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd200107df0 0x7fd20019d5b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4005f50 tx=0x7fd1f4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:44.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:44.738+0000 7fd1e7fff700 1 -- 172.21.15.5:0/2572490001 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f4026070 con 0x7fd200107df0 2024-05-09T00:45:44.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:44.738+0000 7fd1e7fff700 1 -- 172.21.15.5:0/2572490001 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1f4003c70 con 0x7fd200107df0 2024-05-09T00:45:44.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:44.738+0000 7fd2056e7700 1 -- 172.21.15.5:0/2572490001 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2001a2170 con 0x7fd200107df0 2024-05-09T00:45:44.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:44.738+0000 7fd1e7fff700 1 -- 172.21.15.5:0/2572490001 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f400f9c0 con 0x7fd200107df0 2024-05-09T00:45:44.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:44.739+0000 7fd2056e7700 1 -- 172.21.15.5:0/2572490001 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2001a2600 con 0x7fd200107df0 2024-05-09T00:45:44.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:44.739+0000 7fd2056e7700 1 -- 172.21.15.5:0/2572490001 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2000512c0 con 0x7fd200107df0 2024-05-09T00:45:44.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:44.741+0000 7fd1e7fff700 1 -- 172.21.15.5:0/2572490001 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fd1f4029070 con 0x7fd200107df0 2024-05-09T00:45:44.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:44.742+0000 7fd1e7fff700 1 --2- 172.21.15.5:0/2572490001 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fd1e80b2910 0x7fd1e80b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:44.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:44.742+0000 7fd1feffd700 1 --2- 172.21.15.5:0/2572490001 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fd1e80b2910 0x7fd1e80b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:44.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:44.742+0000 7fd1e7fff700 1 -- 172.21.15.5:0/2572490001 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fd1f40e2f80 con 0x7fd200107df0 2024-05-09T00:45:44.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:44.742+0000 7fd1feffd700 1 --2- 172.21.15.5:0/2572490001 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fd1e80b2910 0x7fd1e80b4e20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd1f0011490 tx=0x7fd1f0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:44.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:44.745+0000 7fd1e7fff700 1 -- 172.21.15.5:0/2572490001 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd1f4030080 con 0x7fd200107df0 2024-05-09T00:45:44.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:44.957+0000 7fd2056e7700 1 -- 172.21.15.5:0/2572490001 --> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd20019e370 con 0x7fd1e80b2910 2024-05-09T00:45:44.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:44.965+0000 7fd1e7fff700 1 -- 172.21.15.5:0/2572490001 <== mgr.14152 v2:172.21.15.5:6800/91856593 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258981 (secure 0 0 0) 0x7fd20019e370 con 0x7fd1e80b2910 2024-05-09T00:45:44.970 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:45:44.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:44.969+0000 7fd2056e7700 1 -- 172.21.15.5:0/2572490001 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fd1e80b2910 msgr2=0x7fd1e80b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:44.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:44.969+0000 7fd2056e7700 1 --2- 172.21.15.5:0/2572490001 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fd1e80b2910 0x7fd1e80b4e20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd1f0011490 tx=0x7fd1f0009260 comp rx=0 tx=0).stop 2024-05-09T00:45:44.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:44.969+0000 7fd2056e7700 1 -- 172.21.15.5:0/2572490001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd200107df0 msgr2=0x7fd20019d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:44.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:44.969+0000 7fd2056e7700 1 --2- 172.21.15.5:0/2572490001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd200107df0 0x7fd20019d5b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4005f50 tx=0x7fd1f4004270 comp rx=0 tx=0).stop 2024-05-09T00:45:44.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:44.969+0000 7fd2056e7700 1 -- 172.21.15.5:0/2572490001 shutdown_connections 2024-05-09T00:45:44.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:44.969+0000 7fd2056e7700 1 --2- 172.21.15.5:0/2572490001 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fd1e80b2910 0x7fd1e80b4e20 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:44.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:44.969+0000 7fd2056e7700 1 --2- 172.21.15.5:0/2572490001 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd200108740 0x7fd2001a1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:44.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:44.969+0000 7fd2056e7700 1 --2- 172.21.15.5:0/2572490001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2001034c0 0x7fd20019d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:44.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:44.969+0000 7fd2056e7700 1 --2- 172.21.15.5:0/2572490001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd200107df0 0x7fd20019d5b0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:44.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:44.969+0000 7fd2056e7700 1 -- 172.21.15.5:0/2572490001 >> 172.21.15.5:0/2572490001 conn(0x7fd2000fedc0 msgr2=0x7fd200109740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:45:44.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:44.969+0000 7fd2056e7700 1 -- 172.21.15.5:0/2572490001 shutdown_connections 2024-05-09T00:45:44.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:44.969+0000 7fd2056e7700 1 -- 172.21.15.5:0/2572490001 wait complete. 2024-05-09T00:45:44.973 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-09T00:45:45.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:45:44 smithi005 ceph-mon[30365]: pgmap v353: 97 pgs: 97 active+clean; 645 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-09T00:45:45.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:45:44 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:45:45.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:45:44 smithi131 ceph-mon[34547]: pgmap v353: 97 pgs: 97 active+clean; 645 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-09T00:45:45.358 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:45:44 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:45:45.426 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:45:44 smithi157 ceph-mon[34294]: pgmap v353: 97 pgs: 97 active+clean; 645 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-09T00:45:45.427 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:45:44 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:45:45.635 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":353,"stamp":"2024-05-09T00:45:43.888848+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660532,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1108,"num_read_kb":646,"num_write":372,"num_write_kb":2291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":3005105,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":306,"ondisk_log_size":306,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":347816,"kb_used_data":11064,"kb_used_omap":18,"kb_used_meta":336557,"kb_avail":1124594008,"statfs":{"total":1151940427776,"available":1151584264192,"internally_reserved":0,"allocated":11329536,"data_stored":7281276,"data_compressed":186585,"data_compressed_allocated":245760,"data_compressed_original":2015232,"omap_allocated":19141,"internal_metadata":344634683},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":140,"num_read_kb":70,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003369"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.934799+0000","last_change":"2024-05-09T00:43:43.875357+0000","last_active":"2024-05-09T00:45:23.934799+0000","last_peered":"2024-05-09T00:45:23.934799+0000","last_clean":"2024-05-09T00:45:23.934799+0000","last_became_active":"2024-05-09T00:43:43.873765+0000","last_became_peered":"2024-05-09T00:43:43.873765+0000","last_unstale":"2024-05-09T00:45:23.934799+0000","last_undegraded":"2024-05-09T00:45:23.934799+0000","last_fullsized":"2024-05-09T00:45:23.934799+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:47.519110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.940366+0000","last_change":"2024-05-09T00:43:43.882464+0000","last_active":"2024-05-09T00:45:23.940366+0000","last_peered":"2024-05-09T00:45:23.940366+0000","last_clean":"2024-05-09T00:45:23.940366+0000","last_became_active":"2024-05-09T00:43:43.882102+0000","last_became_peered":"2024-05-09T00:43:43.882102+0000","last_unstale":"2024-05-09T00:45:23.940366+0000","last_undegraded":"2024-05-09T00:45:23.940366+0000","last_fullsized":"2024-05-09T00:45:23.940366+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:39:36.234474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.940163+0000","last_change":"2024-05-09T00:43:43.873098+0000","last_active":"2024-05-09T00:45:23.940163+0000","last_peered":"2024-05-09T00:45:23.940163+0000","last_clean":"2024-05-09T00:45:23.940163+0000","last_became_active":"2024-05-09T00:43:43.872764+0000","last_became_peered":"2024-05-09T00:43:43.872764+0000","last_unstale":"2024-05-09T00:45:23.940163+0000","last_undegraded":"2024-05-09T00:45:23.940163+0000","last_fullsized":"2024-05-09T00:45:23.940163+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:05:30.784111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.939479+0000","last_change":"2024-05-09T00:43:43.869807+0000","last_active":"2024-05-09T00:45:23.939479+0000","last_peered":"2024-05-09T00:45:23.939479+0000","last_clean":"2024-05-09T00:45:23.939479+0000","last_became_active":"2024-05-09T00:43:43.869470+0000","last_became_peered":"2024-05-09T00:43:43.869470+0000","last_unstale":"2024-05-09T00:45:23.939479+0000","last_undegraded":"2024-05-09T00:45:23.939479+0000","last_fullsized":"2024-05-09T00:45:23.939479+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:25:40.201844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.937265+0000","last_change":"2024-05-09T00:43:43.874110+0000","last_active":"2024-05-09T00:45:23.937265+0000","last_peered":"2024-05-09T00:45:23.937265+0000","last_clean":"2024-05-09T00:45:23.937265+0000","last_became_active":"2024-05-09T00:43:43.873759+0000","last_became_peered":"2024-05-09T00:43:43.873759+0000","last_unstale":"2024-05-09T00:45:23.937265+0000","last_undegraded":"2024-05-09T00:45:23.937265+0000","last_fullsized":"2024-05-09T00:45:23.937265+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:17:17.722319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.080583+0000","last_change":"2024-05-09T00:44:23.022285+0000","last_active":"2024-05-09T00:45:24.080583+0000","last_peered":"2024-05-09T00:45:24.080583+0000","last_clean":"2024-05-09T00:45:24.080583+0000","last_became_active":"2024-05-09T00:44:23.021887+0000","last_became_peered":"2024-05-09T00:44:23.021887+0000","last_unstale":"2024-05-09T00:45:24.080583+0000","last_undegraded":"2024-05-09T00:45:24.080583+0000","last_fullsized":"2024-05-09T00:45:24.080583+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:07:07.702178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,7],"acting":[10,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.080401+0000","last_change":"2024-05-09T00:43:43.883112+0000","last_active":"2024-05-09T00:45:24.080401+0000","last_peered":"2024-05-09T00:45:24.080401+0000","last_clean":"2024-05-09T00:45:24.080401+0000","last_became_active":"2024-05-09T00:43:43.882795+0000","last_became_peered":"2024-05-09T00:43:43.882795+0000","last_unstale":"2024-05-09T00:45:24.080401+0000","last_undegraded":"2024-05-09T00:45:24.080401+0000","last_fullsized":"2024-05-09T00:45:24.080401+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:33:58.724015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.115355+0000","last_change":"2024-05-09T00:43:43.880696+0000","last_active":"2024-05-09T00:45:24.115355+0000","last_peered":"2024-05-09T00:45:24.115355+0000","last_clean":"2024-05-09T00:45:24.115355+0000","last_became_active":"2024-05-09T00:43:43.880174+0000","last_became_peered":"2024-05-09T00:43:43.880174+0000","last_unstale":"2024-05-09T00:45:24.115355+0000","last_undegraded":"2024-05-09T00:45:24.115355+0000","last_fullsized":"2024-05-09T00:45:24.115355+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:40:05.682432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.939132+0000","last_change":"2024-05-09T00:43:43.883449+0000","last_active":"2024-05-09T00:45:23.939132+0000","last_peered":"2024-05-09T00:45:23.939132+0000","last_clean":"2024-05-09T00:45:23.939132+0000","last_became_active":"2024-05-09T00:43:43.883066+0000","last_became_peered":"2024-05-09T00:43:43.883066+0000","last_unstale":"2024-05-09T00:45:23.939132+0000","last_undegraded":"2024-05-09T00:45:23.939132+0000","last_fullsized":"2024-05-09T00:45:23.939132+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:47:49.116004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.081169+0000","last_change":"2024-05-09T00:43:43.877514+0000","last_active":"2024-05-09T00:45:24.081169+0000","last_peered":"2024-05-09T00:45:24.081169+0000","last_clean":"2024-05-09T00:45:24.081169+0000","last_became_active":"2024-05-09T00:43:43.877170+0000","last_became_peered":"2024-05-09T00:43:43.877170+0000","last_unstale":"2024-05-09T00:45:24.081169+0000","last_undegraded":"2024-05-09T00:45:24.081169+0000","last_fullsized":"2024-05-09T00:45:24.081169+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:47:26.738571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.081150+0000","last_change":"2024-05-09T00:44:23.180294+0000","last_active":"2024-05-09T00:45:24.081150+0000","last_peered":"2024-05-09T00:45:24.081150+0000","last_clean":"2024-05-09T00:45:24.081150+0000","last_became_active":"2024-05-09T00:44:23.179824+0000","last_became_peered":"2024-05-09T00:44:23.179824+0000","last_unstale":"2024-05-09T00:45:24.081150+0000","last_undegraded":"2024-05-09T00:45:24.081150+0000","last_fullsized":"2024-05-09T00:45:24.081150+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:43:20.919660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,9],"acting":[7,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.939457+0000","last_change":"2024-05-09T00:43:43.873836+0000","last_active":"2024-05-09T00:45:23.939457+0000","last_peered":"2024-05-09T00:45:23.939457+0000","last_clean":"2024-05-09T00:45:23.939457+0000","last_became_active":"2024-05-09T00:43:43.873512+0000","last_became_peered":"2024-05-09T00:43:43.873512+0000","last_unstale":"2024-05-09T00:45:23.939457+0000","last_undegraded":"2024-05-09T00:45:23.939457+0000","last_fullsized":"2024-05-09T00:45:23.939457+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:32:06.162638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.934307+0000","last_change":"2024-05-09T00:43:43.885614+0000","last_active":"2024-05-09T00:45:23.934307+0000","last_peered":"2024-05-09T00:45:23.934307+0000","last_clean":"2024-05-09T00:45:23.934307+0000","last_became_active":"2024-05-09T00:43:43.884682+0000","last_became_peered":"2024-05-09T00:43:43.884682+0000","last_unstale":"2024-05-09T00:45:23.934307+0000","last_undegraded":"2024-05-09T00:45:23.934307+0000","last_fullsized":"2024-05-09T00:45:23.934307+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:34:15.336173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.939998+0000","last_change":"2024-05-09T00:43:43.883860+0000","last_active":"2024-05-09T00:45:23.939998+0000","last_peered":"2024-05-09T00:45:23.939998+0000","last_clean":"2024-05-09T00:45:23.939998+0000","last_became_active":"2024-05-09T00:43:43.882870+0000","last_became_peered":"2024-05-09T00:43:43.882870+0000","last_unstale":"2024-05-09T00:45:23.939998+0000","last_undegraded":"2024-05-09T00:45:23.939998+0000","last_fullsized":"2024-05-09T00:45:23.939998+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:17:31.566819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.939920+0000","last_change":"2024-05-09T00:43:43.874114+0000","last_active":"2024-05-09T00:45:23.939920+0000","last_peered":"2024-05-09T00:45:23.939920+0000","last_clean":"2024-05-09T00:45:23.939920+0000","last_became_active":"2024-05-09T00:43:43.873762+0000","last_became_peered":"2024-05-09T00:43:43.873762+0000","last_unstale":"2024-05-09T00:45:23.939920+0000","last_undegraded":"2024-05-09T00:45:23.939920+0000","last_fullsized":"2024-05-09T00:45:23.939920+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:24:27.486100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.938359+0000","last_change":"2024-05-09T00:43:43.872982+0000","last_active":"2024-05-09T00:45:23.938359+0000","last_peered":"2024-05-09T00:45:23.938359+0000","last_clean":"2024-05-09T00:45:23.938359+0000","last_became_active":"2024-05-09T00:43:43.872464+0000","last_became_peered":"2024-05-09T00:43:43.872464+0000","last_unstale":"2024-05-09T00:45:23.938359+0000","last_undegraded":"2024-05-09T00:45:23.938359+0000","last_fullsized":"2024-05-09T00:45:23.938359+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:41.172001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.934939+0000","last_change":"2024-05-09T00:43:43.875662+0000","last_active":"2024-05-09T00:45:23.934939+0000","last_peered":"2024-05-09T00:45:23.934939+0000","last_clean":"2024-05-09T00:45:23.934939+0000","last_became_active":"2024-05-09T00:43:43.874701+0000","last_became_peered":"2024-05-09T00:43:43.874701+0000","last_unstale":"2024-05-09T00:45:23.934939+0000","last_undegraded":"2024-05-09T00:45:23.934939+0000","last_fullsized":"2024-05-09T00:45:23.934939+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:09:19.349190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.938226+0000","last_change":"2024-05-09T00:43:43.876100+0000","last_active":"2024-05-09T00:45:23.938226+0000","last_peered":"2024-05-09T00:45:23.938226+0000","last_clean":"2024-05-09T00:45:23.938226+0000","last_became_active":"2024-05-09T00:43:43.875768+0000","last_became_peered":"2024-05-09T00:43:43.875768+0000","last_unstale":"2024-05-09T00:45:23.938226+0000","last_undegraded":"2024-05-09T00:45:23.938226+0000","last_fullsized":"2024-05-09T00:45:23.938226+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:48:26.843064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.116311+0000","last_change":"2024-05-09T00:43:43.875900+0000","last_active":"2024-05-09T00:45:24.116311+0000","last_peered":"2024-05-09T00:45:24.116311+0000","last_clean":"2024-05-09T00:45:24.116311+0000","last_became_active":"2024-05-09T00:43:43.875497+0000","last_became_peered":"2024-05-09T00:43:43.875497+0000","last_unstale":"2024-05-09T00:45:24.116311+0000","last_undegraded":"2024-05-09T00:45:24.116311+0000","last_fullsized":"2024-05-09T00:45:24.116311+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:54:44.215936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'13","reported_seq":144,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:41.103629+0000","last_change":"2024-05-09T00:43:40.854337+0000","last_active":"2024-05-09T00:45:41.103629+0000","last_peered":"2024-05-09T00:45:41.103629+0000","last_clean":"2024-05-09T00:45:41.103629+0000","last_became_active":"2024-05-09T00:43:40.853986+0000","last_became_peered":"2024-05-09T00:43:40.853986+0000","last_unstale":"2024-05-09T00:45:41.103629+0000","last_undegraded":"2024-05-09T00:45:41.103629+0000","last_fullsized":"2024-05-09T00:45:41.103629+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:01:19.128533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.115812+0000","last_change":"2024-05-09T00:43:43.878081+0000","last_active":"2024-05-09T00:45:24.115812+0000","last_peered":"2024-05-09T00:45:24.115812+0000","last_clean":"2024-05-09T00:45:24.115812+0000","last_became_active":"2024-05-09T00:43:43.877585+0000","last_became_peered":"2024-05-09T00:43:43.877585+0000","last_unstale":"2024-05-09T00:45:24.115812+0000","last_undegraded":"2024-05-09T00:45:24.115812+0000","last_fullsized":"2024-05-09T00:45:24.115812+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:17:39.255841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.115936+0000","last_change":"2024-05-09T00:43:40.959386+0000","last_active":"2024-05-09T00:45:24.115936+0000","last_peered":"2024-05-09T00:45:24.115936+0000","last_clean":"2024-05-09T00:45:24.115936+0000","last_became_active":"2024-05-09T00:43:40.958878+0000","last_became_peered":"2024-05-09T00:43:40.958878+0000","last_unstale":"2024-05-09T00:45:24.115936+0000","last_undegraded":"2024-05-09T00:45:24.115936+0000","last_fullsized":"2024-05-09T00:45:24.115936+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:14:57.105078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.940324+0000","last_change":"2024-05-09T00:43:43.870563+0000","last_active":"2024-05-09T00:45:23.940324+0000","last_peered":"2024-05-09T00:45:23.940324+0000","last_clean":"2024-05-09T00:45:23.940324+0000","last_became_active":"2024-05-09T00:43:43.870237+0000","last_became_peered":"2024-05-09T00:43:43.870237+0000","last_unstale":"2024-05-09T00:45:23.940324+0000","last_undegraded":"2024-05-09T00:45:23.940324+0000","last_fullsized":"2024-05-09T00:45:23.940324+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:14:28.006343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"78'12","reported_seq":43,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.937305+0000","last_change":"2024-05-09T00:43:40.857670+0000","last_active":"2024-05-09T00:45:23.937305+0000","last_peered":"2024-05-09T00:45:23.937305+0000","last_clean":"2024-05-09T00:45:23.937305+0000","last_became_active":"2024-05-09T00:43:40.857247+0000","last_became_peered":"2024-05-09T00:43:40.857247+0000","last_unstale":"2024-05-09T00:45:23.937305+0000","last_undegraded":"2024-05-09T00:45:23.937305+0000","last_fullsized":"2024-05-09T00:45:23.937305+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:47:06.325593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.938611+0000","last_change":"2024-05-09T00:43:43.883179+0000","last_active":"2024-05-09T00:45:23.938611+0000","last_peered":"2024-05-09T00:45:23.938611+0000","last_clean":"2024-05-09T00:45:23.938611+0000","last_became_active":"2024-05-09T00:43:43.882820+0000","last_became_peered":"2024-05-09T00:43:43.882820+0000","last_unstale":"2024-05-09T00:45:23.938611+0000","last_undegraded":"2024-05-09T00:45:23.938611+0000","last_fullsized":"2024-05-09T00:45:23.938611+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:56:26.012495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"78'5","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.946133+0000","last_change":"2024-05-09T00:45:23.945720+0000","last_active":"2024-05-09T00:45:23.946133+0000","last_peered":"2024-05-09T00:45:23.946133+0000","last_clean":"2024-05-09T00:45:23.946133+0000","last_became_active":"2024-05-09T00:45:23.940421+0000","last_became_peered":"2024-05-09T00:45:23.940421+0000","last_unstale":"2024-05-09T00:45:23.946133+0000","last_undegraded":"2024-05-09T00:45:23.946133+0000","last_fullsized":"2024-05-09T00:45:23.946133+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:23:24.905715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.938187+0000","last_change":"2024-05-09T00:43:43.882950+0000","last_active":"2024-05-09T00:45:23.938187+0000","last_peered":"2024-05-09T00:45:23.938187+0000","last_clean":"2024-05-09T00:45:23.938187+0000","last_became_active":"2024-05-09T00:43:43.880679+0000","last_became_peered":"2024-05-09T00:43:43.880679+0000","last_unstale":"2024-05-09T00:45:23.938187+0000","last_undegraded":"2024-05-09T00:45:23.938187+0000","last_fullsized":"2024-05-09T00:45:23.938187+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:34:14.050987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"78'6","reported_seq":140,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:42.107398+0000","last_change":"2024-05-09T00:43:40.854953+0000","last_active":"2024-05-09T00:45:42.107398+0000","last_peered":"2024-05-09T00:45:42.107398+0000","last_clean":"2024-05-09T00:45:42.107398+0000","last_became_active":"2024-05-09T00:43:40.854562+0000","last_became_peered":"2024-05-09T00:43:40.854562+0000","last_unstale":"2024-05-09T00:45:42.107398+0000","last_undegraded":"2024-05-09T00:45:42.107398+0000","last_fullsized":"2024-05-09T00:45:42.107398+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:09:19.564806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.937264+0000","last_change":"2024-05-09T00:43:43.881881+0000","last_active":"2024-05-09T00:45:23.937264+0000","last_peered":"2024-05-09T00:45:23.937264+0000","last_clean":"2024-05-09T00:45:23.937264+0000","last_became_active":"2024-05-09T00:43:43.881461+0000","last_became_peered":"2024-05-09T00:43:43.881461+0000","last_unstale":"2024-05-09T00:45:23.937264+0000","last_undegraded":"2024-05-09T00:45:23.937264+0000","last_fullsized":"2024-05-09T00:45:23.937264+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:59:28.747084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"78'7","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.080529+0000","last_change":"2024-05-09T00:43:40.858521+0000","last_active":"2024-05-09T00:45:24.080529+0000","last_peered":"2024-05-09T00:45:24.080529+0000","last_clean":"2024-05-09T00:45:24.080529+0000","last_became_active":"2024-05-09T00:43:40.858102+0000","last_became_peered":"2024-05-09T00:43:40.858102+0000","last_unstale":"2024-05-09T00:45:24.080529+0000","last_undegraded":"2024-05-09T00:45:24.080529+0000","last_fullsized":"2024-05-09T00:45:24.080529+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:09:51.684630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.937931+0000","last_change":"2024-05-09T00:44:23.023472+0000","last_active":"2024-05-09T00:45:23.937931+0000","last_peered":"2024-05-09T00:45:23.937931+0000","last_clean":"2024-05-09T00:45:23.937931+0000","last_became_active":"2024-05-09T00:44:23.023017+0000","last_became_peered":"2024-05-09T00:44:23.023017+0000","last_unstale":"2024-05-09T00:45:23.937931+0000","last_undegraded":"2024-05-09T00:45:23.937931+0000","last_fullsized":"2024-05-09T00:45:23.937931+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:05:13.852606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"78'3","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.933923+0000","last_change":"2024-05-09T00:43:40.853329+0000","last_active":"2024-05-09T00:45:23.933923+0000","last_peered":"2024-05-09T00:45:23.933923+0000","last_clean":"2024-05-09T00:45:23.933923+0000","last_became_active":"2024-05-09T00:43:40.852979+0000","last_became_peered":"2024-05-09T00:43:40.852979+0000","last_unstale":"2024-05-09T00:45:23.933923+0000","last_undegraded":"2024-05-09T00:45:23.933923+0000","last_fullsized":"2024-05-09T00:45:23.933923+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:13:57.006031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.115441+0000","last_change":"2024-05-09T00:43:43.862876+0000","last_active":"2024-05-09T00:45:24.115441+0000","last_peered":"2024-05-09T00:45:24.115441+0000","last_clean":"2024-05-09T00:45:24.115441+0000","last_became_active":"2024-05-09T00:43:43.862521+0000","last_became_peered":"2024-05-09T00:43:43.862521+0000","last_unstale":"2024-05-09T00:45:24.115441+0000","last_undegraded":"2024-05-09T00:45:24.115441+0000","last_fullsized":"2024-05-09T00:45:24.115441+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:45:49.386844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'3","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.937477+0000","last_change":"2024-05-09T00:43:40.860395+0000","last_active":"2024-05-09T00:45:23.937477+0000","last_peered":"2024-05-09T00:45:23.937477+0000","last_clean":"2024-05-09T00:45:23.937477+0000","last_became_active":"2024-05-09T00:43:40.860040+0000","last_became_peered":"2024-05-09T00:43:40.860040+0000","last_unstale":"2024-05-09T00:45:23.937477+0000","last_undegraded":"2024-05-09T00:45:23.937477+0000","last_fullsized":"2024-05-09T00:45:23.937477+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:20:36.054685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.937729+0000","last_change":"2024-05-09T00:44:23.180905+0000","last_active":"2024-05-09T00:45:23.937729+0000","last_peered":"2024-05-09T00:45:23.937729+0000","last_clean":"2024-05-09T00:45:23.937729+0000","last_became_active":"2024-05-09T00:44:23.180296+0000","last_became_peered":"2024-05-09T00:44:23.180296+0000","last_unstale":"2024-05-09T00:45:23.937729+0000","last_undegraded":"2024-05-09T00:45:23.937729+0000","last_fullsized":"2024-05-09T00:45:23.937729+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:48:17.585276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,10],"acting":[9,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"78'4","reported_seq":236,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:43.108092+0000","last_change":"2024-05-09T00:43:40.860542+0000","last_active":"2024-05-09T00:45:43.108092+0000","last_peered":"2024-05-09T00:45:43.108092+0000","last_clean":"2024-05-09T00:45:43.108092+0000","last_became_active":"2024-05-09T00:43:40.860176+0000","last_became_peered":"2024-05-09T00:43:40.860176+0000","last_unstale":"2024-05-09T00:45:43.108092+0000","last_undegraded":"2024-05-09T00:45:43.108092+0000","last_fullsized":"2024-05-09T00:45:43.108092+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:44:50.375024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":202,"num_read_kb":202,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.939053+0000","last_change":"2024-05-09T00:43:43.882119+0000","last_active":"2024-05-09T00:45:23.939053+0000","last_peered":"2024-05-09T00:45:23.939053+0000","last_clean":"2024-05-09T00:45:23.939053+0000","last_became_active":"2024-05-09T00:43:43.881747+0000","last_became_peered":"2024-05-09T00:43:43.881747+0000","last_unstale":"2024-05-09T00:45:23.939053+0000","last_undegraded":"2024-05-09T00:45:23.939053+0000","last_fullsized":"2024-05-09T00:45:23.939053+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:54:19.420435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'4","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.081233+0000","last_change":"2024-05-09T00:43:40.859539+0000","last_active":"2024-05-09T00:45:24.081233+0000","last_peered":"2024-05-09T00:45:24.081233+0000","last_clean":"2024-05-09T00:45:24.081233+0000","last_became_active":"2024-05-09T00:43:40.859162+0000","last_became_peered":"2024-05-09T00:43:40.859162+0000","last_unstale":"2024-05-09T00:45:24.081233+0000","last_undegraded":"2024-05-09T00:45:24.081233+0000","last_fullsized":"2024-05-09T00:45:24.081233+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:37:15.190944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.940407+0000","last_change":"2024-05-09T00:44:23.180872+0000","last_active":"2024-05-09T00:45:23.940407+0000","last_peered":"2024-05-09T00:45:23.940407+0000","last_clean":"2024-05-09T00:45:23.940407+0000","last_became_active":"2024-05-09T00:44:23.180398+0000","last_became_peered":"2024-05-09T00:44:23.180398+0000","last_unstale":"2024-05-09T00:45:23.940407+0000","last_undegraded":"2024-05-09T00:45:23.940407+0000","last_fullsized":"2024-05-09T00:45:23.940407+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:19:27.909490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,6],"acting":[0,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'5","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.938299+0000","last_change":"2024-05-09T00:43:40.960126+0000","last_active":"2024-05-09T00:45:23.938299+0000","last_peered":"2024-05-09T00:45:23.938299+0000","last_clean":"2024-05-09T00:45:23.938299+0000","last_became_active":"2024-05-09T00:43:40.959441+0000","last_became_peered":"2024-05-09T00:43:40.959441+0000","last_unstale":"2024-05-09T00:45:23.938299+0000","last_undegraded":"2024-05-09T00:45:23.938299+0000","last_fullsized":"2024-05-09T00:45:23.938299+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:38:47.497029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.938766+0000","last_change":"2024-05-09T00:43:43.874223+0000","last_active":"2024-05-09T00:45:23.938766+0000","last_peered":"2024-05-09T00:45:23.938766+0000","last_clean":"2024-05-09T00:45:23.938766+0000","last_became_active":"2024-05-09T00:43:43.873875+0000","last_became_peered":"2024-05-09T00:43:43.873875+0000","last_unstale":"2024-05-09T00:45:23.938766+0000","last_undegraded":"2024-05-09T00:45:23.938766+0000","last_fullsized":"2024-05-09T00:45:23.938766+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:59:22.192886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'32","reported_seq":291,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:42.315058+0000","last_change":"2024-05-09T00:44:23.188953+0000","last_active":"2024-05-09T00:45:42.315058+0000","last_peered":"2024-05-09T00:45:42.315058+0000","last_clean":"2024-05-09T00:45:42.315058+0000","last_became_active":"2024-05-09T00:44:23.180858+0000","last_became_peered":"2024-05-09T00:44:23.180858+0000","last_unstale":"2024-05-09T00:45:42.315058+0000","last_undegraded":"2024-05-09T00:45:42.315058+0000","last_fullsized":"2024-05-09T00:45:42.315058+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:26:16.427665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62335,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":204,"num_read_kb":152,"num_write":34,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":52683,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.936978+0000","last_change":"2024-05-09T00:43:43.872019+0000","last_active":"2024-05-09T00:45:23.936978+0000","last_peered":"2024-05-09T00:45:23.936978+0000","last_clean":"2024-05-09T00:45:23.936978+0000","last_became_active":"2024-05-09T00:43:43.871622+0000","last_became_peered":"2024-05-09T00:43:43.871622+0000","last_unstale":"2024-05-09T00:45:23.936978+0000","last_undegraded":"2024-05-09T00:45:23.936978+0000","last_fullsized":"2024-05-09T00:45:23.936978+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:41:47.789443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'9","reported_seq":41,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.116151+0000","last_change":"2024-05-09T00:43:40.860111+0000","last_active":"2024-05-09T00:45:24.116151+0000","last_peered":"2024-05-09T00:45:24.116151+0000","last_clean":"2024-05-09T00:45:24.116151+0000","last_became_active":"2024-05-09T00:43:40.859732+0000","last_became_peered":"2024-05-09T00:43:40.859732+0000","last_unstale":"2024-05-09T00:45:24.116151+0000","last_undegraded":"2024-05-09T00:45:24.116151+0000","last_fullsized":"2024-05-09T00:45:24.116151+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:30:03.180679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.934086+0000","last_change":"2024-05-09T00:43:43.870604+0000","last_active":"2024-05-09T00:45:23.934086+0000","last_peered":"2024-05-09T00:45:23.934086+0000","last_clean":"2024-05-09T00:45:23.934086+0000","last_became_active":"2024-05-09T00:43:43.870286+0000","last_became_peered":"2024-05-09T00:43:43.870286+0000","last_unstale":"2024-05-09T00:45:23.934086+0000","last_undegraded":"2024-05-09T00:45:23.934086+0000","last_fullsized":"2024-05-09T00:45:23.934086+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:57:04.486631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'5","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.937799+0000","last_change":"2024-05-09T00:43:40.851966+0000","last_active":"2024-05-09T00:45:23.937799+0000","last_peered":"2024-05-09T00:45:23.937799+0000","last_clean":"2024-05-09T00:45:23.937799+0000","last_became_active":"2024-05-09T00:43:40.851537+0000","last_became_peered":"2024-05-09T00:43:40.851537+0000","last_unstale":"2024-05-09T00:45:23.937799+0000","last_undegraded":"2024-05-09T00:45:23.937799+0000","last_fullsized":"2024-05-09T00:45:23.937799+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:58:59.797183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.081006+0000","last_change":"2024-05-09T00:43:43.879509+0000","last_active":"2024-05-09T00:45:24.081006+0000","last_peered":"2024-05-09T00:45:24.081006+0000","last_clean":"2024-05-09T00:45:24.081006+0000","last_became_active":"2024-05-09T00:43:43.879151+0000","last_became_peered":"2024-05-09T00:43:43.879151+0000","last_unstale":"2024-05-09T00:45:24.081006+0000","last_undegraded":"2024-05-09T00:45:24.081006+0000","last_fullsized":"2024-05-09T00:45:24.081006+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:34:18.850545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"78'6","reported_seq":139,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:41.104445+0000","last_change":"2024-05-09T00:43:40.857629+0000","last_active":"2024-05-09T00:45:41.104445+0000","last_peered":"2024-05-09T00:45:41.104445+0000","last_clean":"2024-05-09T00:45:41.104445+0000","last_became_active":"2024-05-09T00:43:40.857260+0000","last_became_peered":"2024-05-09T00:43:40.857260+0000","last_unstale":"2024-05-09T00:45:41.104445+0000","last_undegraded":"2024-05-09T00:45:41.104445+0000","last_fullsized":"2024-05-09T00:45:41.104445+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:51:00.600870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":104,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.080625+0000","last_change":"2024-05-09T00:43:43.886100+0000","last_active":"2024-05-09T00:45:24.080625+0000","last_peered":"2024-05-09T00:45:24.080625+0000","last_clean":"2024-05-09T00:45:24.080625+0000","last_became_active":"2024-05-09T00:43:43.885775+0000","last_became_peered":"2024-05-09T00:43:43.885775+0000","last_unstale":"2024-05-09T00:45:24.080625+0000","last_undegraded":"2024-05-09T00:45:24.080625+0000","last_fullsized":"2024-05-09T00:45:24.080625+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:34:19.439163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'17","reported_seq":48,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.936761+0000","last_change":"2024-05-09T00:43:40.959873+0000","last_active":"2024-05-09T00:45:23.936761+0000","last_peered":"2024-05-09T00:45:23.936761+0000","last_clean":"2024-05-09T00:45:23.936761+0000","last_became_active":"2024-05-09T00:43:40.959437+0000","last_became_peered":"2024-05-09T00:43:40.959437+0000","last_unstale":"2024-05-09T00:45:23.936761+0000","last_undegraded":"2024-05-09T00:45:23.936761+0000","last_fullsized":"2024-05-09T00:45:23.936761+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:29:52.686937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.937632+0000","last_change":"2024-05-09T00:43:43.880076+0000","last_active":"2024-05-09T00:45:23.937632+0000","last_peered":"2024-05-09T00:45:23.937632+0000","last_clean":"2024-05-09T00:45:23.937632+0000","last_became_active":"2024-05-09T00:43:43.878733+0000","last_became_peered":"2024-05-09T00:43:43.878733+0000","last_unstale":"2024-05-09T00:45:23.937632+0000","last_undegraded":"2024-05-09T00:45:23.937632+0000","last_fullsized":"2024-05-09T00:45:23.937632+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:30:21.426855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.937133+0000","last_change":"2024-05-09T00:43:40.858033+0000","last_active":"2024-05-09T00:45:23.937133+0000","last_peered":"2024-05-09T00:45:23.937133+0000","last_clean":"2024-05-09T00:45:23.937133+0000","last_became_active":"2024-05-09T00:43:40.857075+0000","last_became_peered":"2024-05-09T00:43:40.857075+0000","last_unstale":"2024-05-09T00:45:23.937133+0000","last_undegraded":"2024-05-09T00:45:23.937133+0000","last_fullsized":"2024-05-09T00:45:23.937133+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:01:49.690226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.934621+0000","last_change":"2024-05-09T00:44:23.180515+0000","last_active":"2024-05-09T00:45:23.934621+0000","last_peered":"2024-05-09T00:45:23.934621+0000","last_clean":"2024-05-09T00:45:23.934621+0000","last_became_active":"2024-05-09T00:44:23.179834+0000","last_became_peered":"2024-05-09T00:44:23.179834+0000","last_unstale":"2024-05-09T00:45:23.934621+0000","last_undegraded":"2024-05-09T00:45:23.934621+0000","last_fullsized":"2024-05-09T00:45:23.934621+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:21:55.735438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,6],"acting":[11,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"78'1","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.941489+0000","last_change":"2024-05-09T00:45:23.941489+0000","last_active":"2024-05-09T00:45:23.941489+0000","last_peered":"2024-05-09T00:45:23.941489+0000","last_clean":"2024-05-09T00:45:23.941489+0000","last_became_active":"2024-05-09T00:45:23.941072+0000","last_became_peered":"2024-05-09T00:45:23.941072+0000","last_unstale":"2024-05-09T00:45:23.941489+0000","last_undegraded":"2024-05-09T00:45:23.941489+0000","last_fullsized":"2024-05-09T00:45:23.941489+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:20:05.755293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"78'1","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.081020+0000","last_change":"2024-05-09T00:43:43.886618+0000","last_active":"2024-05-09T00:45:24.081020+0000","last_peered":"2024-05-09T00:45:24.081020+0000","last_clean":"2024-05-09T00:45:24.081020+0000","last_became_active":"2024-05-09T00:43:43.885946+0000","last_became_peered":"2024-05-09T00:43:43.885946+0000","last_unstale":"2024-05-09T00:45:24.081020+0000","last_undegraded":"2024-05-09T00:45:24.081020+0000","last_fullsized":"2024-05-09T00:45:24.081020+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:16:22.169548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"78'5","reported_seq":133,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:38.099828+0000","last_change":"2024-05-09T00:43:40.856213+0000","last_active":"2024-05-09T00:45:38.099828+0000","last_peered":"2024-05-09T00:45:38.099828+0000","last_clean":"2024-05-09T00:45:38.099828+0000","last_became_active":"2024-05-09T00:43:40.855700+0000","last_became_peered":"2024-05-09T00:43:40.855700+0000","last_unstale":"2024-05-09T00:45:38.099828+0000","last_undegraded":"2024-05-09T00:45:38.099828+0000","last_fullsized":"2024-05-09T00:45:38.099828+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:48:50.238293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":98,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.080522+0000","last_change":"2024-05-09T00:43:43.885272+0000","last_active":"2024-05-09T00:45:24.080522+0000","last_peered":"2024-05-09T00:45:24.080522+0000","last_clean":"2024-05-09T00:45:24.080522+0000","last_became_active":"2024-05-09T00:43:43.884774+0000","last_became_peered":"2024-05-09T00:43:43.884774+0000","last_unstale":"2024-05-09T00:45:24.080522+0000","last_undegraded":"2024-05-09T00:45:24.080522+0000","last_fullsized":"2024-05-09T00:45:24.080522+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:14:02.193859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.937922+0000","last_change":"2024-05-09T00:43:43.883370+0000","last_active":"2024-05-09T00:45:23.937922+0000","last_peered":"2024-05-09T00:45:23.937922+0000","last_clean":"2024-05-09T00:45:23.937922+0000","last_became_active":"2024-05-09T00:43:43.881570+0000","last_became_peered":"2024-05-09T00:43:43.881570+0000","last_unstale":"2024-05-09T00:45:23.937922+0000","last_undegraded":"2024-05-09T00:45:23.937922+0000","last_fullsized":"2024-05-09T00:45:23.937922+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:48.936861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"78'5","reported_seq":132,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:37.097905+0000","last_change":"2024-05-09T00:43:40.959828+0000","last_active":"2024-05-09T00:45:37.097905+0000","last_peered":"2024-05-09T00:45:37.097905+0000","last_clean":"2024-05-09T00:45:37.097905+0000","last_became_active":"2024-05-09T00:43:40.958889+0000","last_became_peered":"2024-05-09T00:43:40.958889+0000","last_unstale":"2024-05-09T00:45:37.097905+0000","last_undegraded":"2024-05-09T00:45:37.097905+0000","last_fullsized":"2024-05-09T00:45:37.097905+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:24:50.113298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":97,"num_read_kb":97,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.115638+0000","last_change":"2024-05-09T00:43:43.876144+0000","last_active":"2024-05-09T00:45:24.115638+0000","last_peered":"2024-05-09T00:45:24.115638+0000","last_clean":"2024-05-09T00:45:24.115638+0000","last_became_active":"2024-05-09T00:43:43.875774+0000","last_became_peered":"2024-05-09T00:43:43.875774+0000","last_unstale":"2024-05-09T00:45:24.115638+0000","last_undegraded":"2024-05-09T00:45:24.115638+0000","last_fullsized":"2024-05-09T00:45:24.115638+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:05:23.145431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"78'6","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.939530+0000","last_change":"2024-05-09T00:43:40.857278+0000","last_active":"2024-05-09T00:45:23.939530+0000","last_peered":"2024-05-09T00:45:23.939530+0000","last_clean":"2024-05-09T00:45:23.939530+0000","last_became_active":"2024-05-09T00:43:40.856861+0000","last_became_peered":"2024-05-09T00:43:40.856861+0000","last_unstale":"2024-05-09T00:45:23.939530+0000","last_undegraded":"2024-05-09T00:45:23.939530+0000","last_fullsized":"2024-05-09T00:45:23.939530+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:36:16.603327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":103,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.080385+0000","last_change":"2024-05-09T00:42:14.528544+0000","last_active":"2024-05-09T00:45:24.080385+0000","last_peered":"2024-05-09T00:45:24.080385+0000","last_clean":"2024-05-09T00:45:24.080385+0000","last_became_active":"2024-05-09T00:42:14.523658+0000","last_became_peered":"2024-05-09T00:42:14.523658+0000","last_unstale":"2024-05-09T00:45:24.080385+0000","last_undegraded":"2024-05-09T00:45:24.080385+0000","last_fullsized":"2024-05-09T00:45:24.080385+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:38:48.637995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:38:48.637995+0000","last_clean_scrub_stamp":"2024-05-09T00:38:48.637995+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:42:28.249240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.933721+0000","last_change":"2024-05-09T00:43:43.873638+0000","last_active":"2024-05-09T00:45:23.933721+0000","last_peered":"2024-05-09T00:45:23.933721+0000","last_clean":"2024-05-09T00:45:23.933721+0000","last_became_active":"2024-05-09T00:43:43.871069+0000","last_became_peered":"2024-05-09T00:43:43.871069+0000","last_unstale":"2024-05-09T00:45:23.933721+0000","last_undegraded":"2024-05-09T00:45:23.933721+0000","last_fullsized":"2024-05-09T00:45:23.933721+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:28:34.996989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"78'4","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.080673+0000","last_change":"2024-05-09T00:43:40.856374+0000","last_active":"2024-05-09T00:45:24.080673+0000","last_peered":"2024-05-09T00:45:24.080673+0000","last_clean":"2024-05-09T00:45:24.080673+0000","last_became_active":"2024-05-09T00:43:40.855695+0000","last_became_peered":"2024-05-09T00:43:40.855695+0000","last_unstale":"2024-05-09T00:45:24.080673+0000","last_undegraded":"2024-05-09T00:45:24.080673+0000","last_fullsized":"2024-05-09T00:45:24.080673+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:33:43.734132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"78'4","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.937089+0000","last_change":"2024-05-09T00:43:40.959303+0000","last_active":"2024-05-09T00:45:23.937089+0000","last_peered":"2024-05-09T00:45:23.937089+0000","last_clean":"2024-05-09T00:45:23.937089+0000","last_became_active":"2024-05-09T00:43:40.958855+0000","last_became_peered":"2024-05-09T00:43:40.958855+0000","last_unstale":"2024-05-09T00:45:23.937089+0000","last_undegraded":"2024-05-09T00:45:23.937089+0000","last_fullsized":"2024-05-09T00:45:23.937089+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:53:22.566850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.934495+0000","last_change":"2024-05-09T00:43:43.872041+0000","last_active":"2024-05-09T00:45:23.934495+0000","last_peered":"2024-05-09T00:45:23.934495+0000","last_clean":"2024-05-09T00:45:23.934495+0000","last_became_active":"2024-05-09T00:43:43.870946+0000","last_became_peered":"2024-05-09T00:43:43.870946+0000","last_unstale":"2024-05-09T00:45:23.934495+0000","last_undegraded":"2024-05-09T00:45:23.934495+0000","last_fullsized":"2024-05-09T00:45:23.934495+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:16:28.190488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'15","reported_seq":45,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.938203+0000","last_change":"2024-05-09T00:43:40.853875+0000","last_active":"2024-05-09T00:45:23.938203+0000","last_peered":"2024-05-09T00:45:23.938203+0000","last_clean":"2024-05-09T00:45:23.938203+0000","last_became_active":"2024-05-09T00:43:40.853459+0000","last_became_peered":"2024-05-09T00:43:40.853459+0000","last_unstale":"2024-05-09T00:45:23.938203+0000","last_undegraded":"2024-05-09T00:45:23.938203+0000","last_fullsized":"2024-05-09T00:45:23.938203+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:45:29.992730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.115875+0000","last_change":"2024-05-09T00:43:43.870435+0000","last_active":"2024-05-09T00:45:24.115875+0000","last_peered":"2024-05-09T00:45:24.115875+0000","last_clean":"2024-05-09T00:45:24.115875+0000","last_became_active":"2024-05-09T00:43:43.870075+0000","last_became_peered":"2024-05-09T00:43:43.870075+0000","last_unstale":"2024-05-09T00:45:24.115875+0000","last_undegraded":"2024-05-09T00:45:24.115875+0000","last_fullsized":"2024-05-09T00:45:24.115875+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:53:12.958463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'7","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.939765+0000","last_change":"2024-05-09T00:43:41.119325+0000","last_active":"2024-05-09T00:45:23.939765+0000","last_peered":"2024-05-09T00:45:23.939765+0000","last_clean":"2024-05-09T00:45:23.939765+0000","last_became_active":"2024-05-09T00:43:41.119009+0000","last_became_peered":"2024-05-09T00:43:41.119009+0000","last_unstale":"2024-05-09T00:45:23.939765+0000","last_undegraded":"2024-05-09T00:45:23.939765+0000","last_fullsized":"2024-05-09T00:45:23.939765+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:10:05.775152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.940519+0000","last_change":"2024-05-09T00:43:43.875236+0000","last_active":"2024-05-09T00:45:23.940519+0000","last_peered":"2024-05-09T00:45:23.940519+0000","last_clean":"2024-05-09T00:45:23.940519+0000","last_became_active":"2024-05-09T00:43:43.874919+0000","last_became_peered":"2024-05-09T00:43:43.874919+0000","last_unstale":"2024-05-09T00:45:23.940519+0000","last_undegraded":"2024-05-09T00:45:23.940519+0000","last_fullsized":"2024-05-09T00:45:23.940519+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:45:33.232518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.938358+0000","last_change":"2024-05-09T00:43:40.959765+0000","last_active":"2024-05-09T00:45:23.938358+0000","last_peered":"2024-05-09T00:45:23.938358+0000","last_clean":"2024-05-09T00:45:23.938358+0000","last_became_active":"2024-05-09T00:43:40.959383+0000","last_became_peered":"2024-05-09T00:43:40.959383+0000","last_unstale":"2024-05-09T00:45:23.938358+0000","last_undegraded":"2024-05-09T00:45:23.938358+0000","last_fullsized":"2024-05-09T00:45:23.938358+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:13:28.766650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.937040+0000","last_change":"2024-05-09T00:43:43.881627+0000","last_active":"2024-05-09T00:45:23.937040+0000","last_peered":"2024-05-09T00:45:23.937040+0000","last_clean":"2024-05-09T00:45:23.937040+0000","last_became_active":"2024-05-09T00:43:43.881207+0000","last_became_peered":"2024-05-09T00:43:43.881207+0000","last_unstale":"2024-05-09T00:45:23.937040+0000","last_undegraded":"2024-05-09T00:45:23.937040+0000","last_fullsized":"2024-05-09T00:45:23.937040+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:44:08.053638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'7","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.115434+0000","last_change":"2024-05-09T00:43:40.859566+0000","last_active":"2024-05-09T00:45:24.115434+0000","last_peered":"2024-05-09T00:45:24.115434+0000","last_clean":"2024-05-09T00:45:24.115434+0000","last_became_active":"2024-05-09T00:43:40.859182+0000","last_became_peered":"2024-05-09T00:43:40.859182+0000","last_unstale":"2024-05-09T00:45:24.115434+0000","last_undegraded":"2024-05-09T00:45:24.115434+0000","last_fullsized":"2024-05-09T00:45:24.115434+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:58:12.693271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.934380+0000","last_change":"2024-05-09T00:43:43.869193+0000","last_active":"2024-05-09T00:45:23.934380+0000","last_peered":"2024-05-09T00:45:23.934380+0000","last_clean":"2024-05-09T00:45:23.934380+0000","last_became_active":"2024-05-09T00:43:43.868823+0000","last_became_peered":"2024-05-09T00:43:43.868823+0000","last_unstale":"2024-05-09T00:45:23.934380+0000","last_undegraded":"2024-05-09T00:45:23.934380+0000","last_fullsized":"2024-05-09T00:45:23.934380+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:15:53.929873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'3","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.116089+0000","last_change":"2024-05-09T00:43:40.855038+0000","last_active":"2024-05-09T00:45:24.116089+0000","last_peered":"2024-05-09T00:45:24.116089+0000","last_clean":"2024-05-09T00:45:24.116089+0000","last_became_active":"2024-05-09T00:43:40.854334+0000","last_became_peered":"2024-05-09T00:43:40.854334+0000","last_unstale":"2024-05-09T00:45:24.116089+0000","last_undegraded":"2024-05-09T00:45:24.116089+0000","last_fullsized":"2024-05-09T00:45:24.116089+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:47:13.897697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.937849+0000","last_change":"2024-05-09T00:44:23.019151+0000","last_active":"2024-05-09T00:45:23.937849+0000","last_peered":"2024-05-09T00:45:23.937849+0000","last_clean":"2024-05-09T00:45:23.937849+0000","last_became_active":"2024-05-09T00:44:23.018752+0000","last_became_peered":"2024-05-09T00:44:23.018752+0000","last_unstale":"2024-05-09T00:45:23.937849+0000","last_undegraded":"2024-05-09T00:45:23.937849+0000","last_fullsized":"2024-05-09T00:45:23.937849+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:28:41.875747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.937399+0000","last_change":"2024-05-09T00:43:40.859461+0000","last_active":"2024-05-09T00:45:23.937399+0000","last_peered":"2024-05-09T00:45:23.937399+0000","last_clean":"2024-05-09T00:45:23.937399+0000","last_became_active":"2024-05-09T00:43:40.857814+0000","last_became_peered":"2024-05-09T00:43:40.857814+0000","last_unstale":"2024-05-09T00:45:23.937399+0000","last_undegraded":"2024-05-09T00:45:23.937399+0000","last_fullsized":"2024-05-09T00:45:23.937399+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:56:06.279762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.080858+0000","last_change":"2024-05-09T00:43:43.878120+0000","last_active":"2024-05-09T00:45:24.080858+0000","last_peered":"2024-05-09T00:45:24.080858+0000","last_clean":"2024-05-09T00:45:24.080858+0000","last_became_active":"2024-05-09T00:43:43.877782+0000","last_became_peered":"2024-05-09T00:43:43.877782+0000","last_unstale":"2024-05-09T00:45:24.080858+0000","last_undegraded":"2024-05-09T00:45:24.080858+0000","last_fullsized":"2024-05-09T00:45:24.080858+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:19:48.558992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'6","reported_seq":139,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:41.155936+0000","last_change":"2024-05-09T00:43:41.119347+0000","last_active":"2024-05-09T00:45:41.155936+0000","last_peered":"2024-05-09T00:45:41.155936+0000","last_clean":"2024-05-09T00:45:41.155936+0000","last_became_active":"2024-05-09T00:43:41.118847+0000","last_became_peered":"2024-05-09T00:43:41.118847+0000","last_unstale":"2024-05-09T00:45:41.155936+0000","last_undegraded":"2024-05-09T00:45:41.155936+0000","last_fullsized":"2024-05-09T00:45:41.155936+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:07:03.977756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":103,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.081339+0000","last_change":"2024-05-09T00:43:43.875203+0000","last_active":"2024-05-09T00:45:24.081339+0000","last_peered":"2024-05-09T00:45:24.081339+0000","last_clean":"2024-05-09T00:45:24.081339+0000","last_became_active":"2024-05-09T00:43:43.874882+0000","last_became_peered":"2024-05-09T00:43:43.874882+0000","last_unstale":"2024-05-09T00:45:24.081339+0000","last_undegraded":"2024-05-09T00:45:24.081339+0000","last_fullsized":"2024-05-09T00:45:24.081339+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:54:27.845086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'3","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.938122+0000","last_change":"2024-05-09T00:43:40.857955+0000","last_active":"2024-05-09T00:45:23.938122+0000","last_peered":"2024-05-09T00:45:23.938122+0000","last_clean":"2024-05-09T00:45:23.938122+0000","last_became_active":"2024-05-09T00:43:40.857596+0000","last_became_peered":"2024-05-09T00:43:40.857596+0000","last_unstale":"2024-05-09T00:45:23.938122+0000","last_undegraded":"2024-05-09T00:45:23.938122+0000","last_fullsized":"2024-05-09T00:45:23.938122+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:13:56.676929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"78'1","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.080858+0000","last_change":"2024-05-09T00:43:43.882487+0000","last_active":"2024-05-09T00:45:24.080858+0000","last_peered":"2024-05-09T00:45:24.080858+0000","last_clean":"2024-05-09T00:45:24.080858+0000","last_became_active":"2024-05-09T00:43:43.882176+0000","last_became_peered":"2024-05-09T00:43:43.882176+0000","last_unstale":"2024-05-09T00:45:24.080858+0000","last_undegraded":"2024-05-09T00:45:24.080858+0000","last_fullsized":"2024-05-09T00:45:24.080858+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:48:42.354433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'5","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.934689+0000","last_change":"2024-05-09T00:43:40.857215+0000","last_active":"2024-05-09T00:45:23.934689+0000","last_peered":"2024-05-09T00:45:23.934689+0000","last_clean":"2024-05-09T00:45:23.934689+0000","last_became_active":"2024-05-09T00:43:40.856858+0000","last_became_peered":"2024-05-09T00:43:40.856858+0000","last_unstale":"2024-05-09T00:45:23.934689+0000","last_undegraded":"2024-05-09T00:45:23.934689+0000","last_fullsized":"2024-05-09T00:45:23.934689+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:40:47.943482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.081373+0000","last_change":"2024-05-09T00:43:43.881192+0000","last_active":"2024-05-09T00:45:24.081373+0000","last_peered":"2024-05-09T00:45:24.081373+0000","last_clean":"2024-05-09T00:45:24.081373+0000","last_became_active":"2024-05-09T00:43:43.880781+0000","last_became_peered":"2024-05-09T00:43:43.880781+0000","last_unstale":"2024-05-09T00:45:24.081373+0000","last_undegraded":"2024-05-09T00:45:24.081373+0000","last_fullsized":"2024-05-09T00:45:24.081373+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:18:31.577354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.934461+0000","last_change":"2024-05-09T00:43:43.876869+0000","last_active":"2024-05-09T00:45:23.934461+0000","last_peered":"2024-05-09T00:45:23.934461+0000","last_clean":"2024-05-09T00:45:23.934461+0000","last_became_active":"2024-05-09T00:43:43.876439+0000","last_became_peered":"2024-05-09T00:43:43.876439+0000","last_unstale":"2024-05-09T00:45:23.934461+0000","last_undegraded":"2024-05-09T00:45:23.934461+0000","last_fullsized":"2024-05-09T00:45:23.934461+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:39:06.776267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.116064+0000","last_change":"2024-05-09T00:43:43.884677+0000","last_active":"2024-05-09T00:45:24.116064+0000","last_peered":"2024-05-09T00:45:24.116064+0000","last_clean":"2024-05-09T00:45:24.116064+0000","last_became_active":"2024-05-09T00:43:43.884238+0000","last_became_peered":"2024-05-09T00:43:43.884238+0000","last_unstale":"2024-05-09T00:45:24.116064+0000","last_undegraded":"2024-05-09T00:45:24.116064+0000","last_fullsized":"2024-05-09T00:45:24.116064+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:18:13.787584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.116035+0000","last_change":"2024-05-09T00:43:43.876638+0000","last_active":"2024-05-09T00:45:24.116035+0000","last_peered":"2024-05-09T00:45:24.116035+0000","last_clean":"2024-05-09T00:45:24.116035+0000","last_became_active":"2024-05-09T00:43:43.876315+0000","last_became_peered":"2024-05-09T00:43:43.876315+0000","last_unstale":"2024-05-09T00:45:24.116035+0000","last_undegraded":"2024-05-09T00:45:24.116035+0000","last_fullsized":"2024-05-09T00:45:24.116035+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:22:41.618613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.940191+0000","last_change":"2024-05-09T00:43:43.869798+0000","last_active":"2024-05-09T00:45:23.940191+0000","last_peered":"2024-05-09T00:45:23.940191+0000","last_clean":"2024-05-09T00:45:23.940191+0000","last_became_active":"2024-05-09T00:43:43.869444+0000","last_became_peered":"2024-05-09T00:43:43.869444+0000","last_unstale":"2024-05-09T00:45:23.940191+0000","last_undegraded":"2024-05-09T00:45:23.940191+0000","last_fullsized":"2024-05-09T00:45:23.940191+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:15:53.267441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.934600+0000","last_change":"2024-05-09T00:43:43.873215+0000","last_active":"2024-05-09T00:45:23.934600+0000","last_peered":"2024-05-09T00:45:23.934600+0000","last_clean":"2024-05-09T00:45:23.934600+0000","last_became_active":"2024-05-09T00:43:43.872763+0000","last_became_peered":"2024-05-09T00:43:43.872763+0000","last_unstale":"2024-05-09T00:45:23.934600+0000","last_undegraded":"2024-05-09T00:45:23.934600+0000","last_fullsized":"2024-05-09T00:45:23.934600+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:08.997387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.934702+0000","last_change":"2024-05-09T00:43:43.876514+0000","last_active":"2024-05-09T00:45:23.934702+0000","last_peered":"2024-05-09T00:45:23.934702+0000","last_clean":"2024-05-09T00:45:23.934702+0000","last_became_active":"2024-05-09T00:43:43.876147+0000","last_became_peered":"2024-05-09T00:43:43.876147+0000","last_unstale":"2024-05-09T00:45:23.934702+0000","last_undegraded":"2024-05-09T00:45:23.934702+0000","last_fullsized":"2024-05-09T00:45:23.934702+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:58:34.745819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.116418+0000","last_change":"2024-05-09T00:43:43.871869+0000","last_active":"2024-05-09T00:45:24.116418+0000","last_peered":"2024-05-09T00:45:24.116418+0000","last_clean":"2024-05-09T00:45:24.116418+0000","last_became_active":"2024-05-09T00:43:43.871447+0000","last_became_peered":"2024-05-09T00:43:43.871447+0000","last_unstale":"2024-05-09T00:45:24.116418+0000","last_undegraded":"2024-05-09T00:45:24.116418+0000","last_fullsized":"2024-05-09T00:45:24.116418+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:11:42.128931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.115849+0000","last_change":"2024-05-09T00:43:43.871291+0000","last_active":"2024-05-09T00:45:24.115849+0000","last_peered":"2024-05-09T00:45:24.115849+0000","last_clean":"2024-05-09T00:45:24.115849+0000","last_became_active":"2024-05-09T00:43:43.870899+0000","last_became_peered":"2024-05-09T00:43:43.870899+0000","last_unstale":"2024-05-09T00:45:24.115849+0000","last_undegraded":"2024-05-09T00:45:24.115849+0000","last_fullsized":"2024-05-09T00:45:24.115849+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:28:53.013543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.937586+0000","last_change":"2024-05-09T00:43:43.884791+0000","last_active":"2024-05-09T00:45:23.937586+0000","last_peered":"2024-05-09T00:45:23.937586+0000","last_clean":"2024-05-09T00:45:23.937586+0000","last_became_active":"2024-05-09T00:43:43.877333+0000","last_became_peered":"2024-05-09T00:43:43.877333+0000","last_unstale":"2024-05-09T00:45:23.937586+0000","last_undegraded":"2024-05-09T00:45:23.937586+0000","last_fullsized":"2024-05-09T00:45:23.937586+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:37:07.535388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.115386+0000","last_change":"2024-05-09T00:43:43.863773+0000","last_active":"2024-05-09T00:45:24.115386+0000","last_peered":"2024-05-09T00:45:24.115386+0000","last_clean":"2024-05-09T00:45:24.115386+0000","last_became_active":"2024-05-09T00:43:43.863436+0000","last_became_peered":"2024-05-09T00:43:43.863436+0000","last_unstale":"2024-05-09T00:45:24.115386+0000","last_undegraded":"2024-05-09T00:45:24.115386+0000","last_fullsized":"2024-05-09T00:45:24.115386+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:59:57.100234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.938606+0000","last_change":"2024-05-09T00:43:43.873461+0000","last_active":"2024-05-09T00:45:23.938606+0000","last_peered":"2024-05-09T00:45:23.938606+0000","last_clean":"2024-05-09T00:45:23.938606+0000","last_became_active":"2024-05-09T00:43:43.873139+0000","last_became_peered":"2024-05-09T00:43:43.873139+0000","last_unstale":"2024-05-09T00:45:23.938606+0000","last_undegraded":"2024-05-09T00:45:23.938606+0000","last_fullsized":"2024-05-09T00:45:23.938606+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:24:57.865569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.934304+0000","last_change":"2024-05-09T00:43:43.874401+0000","last_active":"2024-05-09T00:45:23.934304+0000","last_peered":"2024-05-09T00:45:23.934304+0000","last_clean":"2024-05-09T00:45:23.934304+0000","last_became_active":"2024-05-09T00:43:43.873765+0000","last_became_peered":"2024-05-09T00:43:43.873765+0000","last_unstale":"2024-05-09T00:45:23.934304+0000","last_undegraded":"2024-05-09T00:45:23.934304+0000","last_fullsized":"2024-05-09T00:45:23.934304+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:23:59.509415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.934710+0000","last_change":"2024-05-09T00:43:43.875219+0000","last_active":"2024-05-09T00:45:23.934710+0000","last_peered":"2024-05-09T00:45:23.934710+0000","last_clean":"2024-05-09T00:45:23.934710+0000","last_became_active":"2024-05-09T00:43:43.874824+0000","last_became_peered":"2024-05-09T00:43:43.874824+0000","last_unstale":"2024-05-09T00:45:23.934710+0000","last_undegraded":"2024-05-09T00:45:23.934710+0000","last_fullsized":"2024-05-09T00:45:23.934710+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:39:41.929521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":262144,"data_stored":2951840,"data_compressed":179218,"data_compressed_allocated":225280,"data_compressed_original":2932736,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69930,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1032,"num_read_kb":582,"num_write":247,"num_write_kb":139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":53265,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":540672,"data_stored":209790,"data_compressed":8493,"data_compressed_allocated":12288,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},"log_size":212,"ondisk_log_size":212,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710759,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":443805,"data_compressed":5799,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906990,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":1096587,"data_compressed":41967,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070515,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":921600,"data_stored":442762,"data_compressed":5799,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266743,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":913408,"data_stored":442558,"data_compressed":5799,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462973,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":1036049,"data_compressed":41967,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626498,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":905216,"data_stored":442479,"data_compressed":5799,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":970752,"data_stored":505727,"data_compressed":5799,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738464,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":444209,"data_compressed":5799,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411414,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":444394,"data_compressed":5799,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182482,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36244,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708908,"statfs":{"total":95995035648,"available":95957921792,"internally_reserved":0,"allocated":921600,"data_stored":442304,"data_compressed":5799,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986252,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":942080,"data_stored":505066,"data_compressed":14292,"data_compressed_allocated":20480,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822728,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":1035336,"data_compressed":41967,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34546,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36168,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36168,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36168,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36168,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":64087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":63426,"data_compressed":8493,"data_compressed_allocated":12288,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":64462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T00:45:45.639 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-09T00:45:46.245 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:45:45 smithi005 ceph-mon[30365]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T00:45:46.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:45:45 smithi131 ceph-mon[34547]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T00:45:46.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:45:45 smithi157 ceph-mon[34294]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T00:45:46.553 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:45:47.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:45:47 smithi005 ceph-mon[30365]: pgmap v354: 97 pgs: 97 active+clean; 645 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T00:45:47.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:45:47 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:45:47.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:45:47 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:45:47.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:45:47 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:45:47.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:45:47 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:45:47.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:45:47 smithi131 ceph-mon[34547]: pgmap v354: 97 pgs: 97 active+clean; 645 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T00:45:47.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:45:47 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:45:47.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:45:47 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:45:47.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:45:47 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:45:47.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:45:47 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:45:47.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:45:47 smithi157 ceph-mon[34294]: pgmap v354: 97 pgs: 97 active+clean; 645 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T00:45:47.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:45:47 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:45:47.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:45:47 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:45:47.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:45:47 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:45:47.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:45:47 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:45:48.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:48.334+0000 7f130c9b0700 1 -- 172.21.15.5:0/670478424 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1308108f00 msgr2=0x7f1308109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:48.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:48.334+0000 7f130c9b0700 1 --2- 172.21.15.5:0/670478424 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1308108f00 0x7f1308109310 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f12fc0099c0 tx=0x7f12fc009cd0 comp rx=0 tx=0).stop 2024-05-09T00:45:48.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:48.335+0000 7f130c9b0700 1 -- 172.21.15.5:0/670478424 shutdown_connections 2024-05-09T00:45:48.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:48.335+0000 7f130c9b0700 1 --2- 172.21.15.5:0/670478424 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1308109850 0x7f13081111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:48.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:48.335+0000 7f130c9b0700 1 --2- 172.21.15.5:0/670478424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1308102ae0 0x7f1308102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:48.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:48.335+0000 7f130c9b0700 1 --2- 172.21.15.5:0/670478424 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1308108f00 0x7f1308109310 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:48.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:48.335+0000 7f130c9b0700 1 -- 172.21.15.5:0/670478424 >> 172.21.15.5:0/670478424 conn(0x7f13080fe520 msgr2=0x7f1308100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:45:48.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:48.335+0000 7f130c9b0700 1 -- 172.21.15.5:0/670478424 shutdown_connections 2024-05-09T00:45:48.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:48.335+0000 7f130c9b0700 1 -- 172.21.15.5:0/670478424 wait complete. 2024-05-09T00:45:48.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:48.336+0000 7f130c9b0700 1 Processor -- start 2024-05-09T00:45:48.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:48.336+0000 7f130c9b0700 1 -- start start 2024-05-09T00:45:48.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:48.336+0000 7f130c9b0700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1308102ae0 0x7f130819ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:48.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:48.336+0000 7f130c9b0700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1308108f00 0x7f130819b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:48.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:48.336+0000 7f130c9b0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1308109850 0x7f130819f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:48.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:48.336+0000 7f130c9b0700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1308114460 con 0x7f1308109850 2024-05-09T00:45:48.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:48.336+0000 7f130c9b0700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f13081142e0 con 0x7f1308102ae0 2024-05-09T00:45:48.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:48.336+0000 7f130c9b0700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f13081145e0 con 0x7f1308108f00 2024-05-09T00:45:48.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:48.336+0000 7f1306d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1308109850 0x7f130819f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:48.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:48.336+0000 7f1306d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1308109850 0x7f130819f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39694/0 (socket says 172.21.15.5:39694) 2024-05-09T00:45:48.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:48.336+0000 7f1306d9d700 1 -- 172.21.15.5:0/3507722519 learned_addr learned my addr 172.21.15.5:0/3507722519 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:45:48.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:48.337+0000 7f130659c700 1 --2- 172.21.15.5:0/3507722519 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1308102ae0 0x7f130819ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:48.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:48.337+0000 7f1305d9b700 1 --2- 172.21.15.5:0/3507722519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1308108f00 0x7f130819b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:48.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:48.337+0000 7f1306d9d700 1 -- 172.21.15.5:0/3507722519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1308108f00 msgr2=0x7f130819b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:48.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:48.337+0000 7f1306d9d700 1 --2- 172.21.15.5:0/3507722519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1308108f00 0x7f130819b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:48.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:48.337+0000 7f1306d9d700 1 -- 172.21.15.5:0/3507722519 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1308102ae0 msgr2=0x7f130819ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:48.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:48.337+0000 7f1306d9d700 1 --2- 172.21.15.5:0/3507722519 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1308102ae0 0x7f130819ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:48.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:48.337+0000 7f1306d9d700 1 -- 172.21.15.5:0/3507722519 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f130819fc30 con 0x7f1308109850 2024-05-09T00:45:48.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:48.337+0000 7f1306d9d700 1 --2- 172.21.15.5:0/3507722519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1308109850 0x7f130819f640 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f130000c900 tx=0x7f130000cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:48.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:48.337+0000 7f12f77fe700 1 -- 172.21.15.5:0/3507722519 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13000078d0 con 0x7f1308109850 2024-05-09T00:45:48.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:48.337+0000 7f130c9b0700 1 -- 172.21.15.5:0/3507722519 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f130819ff20 con 0x7f1308109850 2024-05-09T00:45:48.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:48.337+0000 7f12f77fe700 1 -- 172.21.15.5:0/3507722519 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f130000f450 con 0x7f1308109850 2024-05-09T00:45:48.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:48.337+0000 7f12f77fe700 1 -- 172.21.15.5:0/3507722519 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1300018620 con 0x7f1308109850 2024-05-09T00:45:48.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:48.337+0000 7f130c9b0700 1 -- 172.21.15.5:0/3507722519 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13081a0430 con 0x7f1308109850 2024-05-09T00:45:48.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:48.338+0000 7f130c9b0700 1 -- 172.21.15.5:0/3507722519 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13080512c0 con 0x7f1308109850 2024-05-09T00:45:48.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:48.340+0000 7f12f77fe700 1 -- 172.21.15.5:0/3507722519 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f1300007a70 con 0x7f1308109850 2024-05-09T00:45:48.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:48.341+0000 7f12f77fe700 1 --2- 172.21.15.5:0/3507722519 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f12f00b2910 0x7f12f00b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:48.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:48.341+0000 7f130659c700 1 --2- 172.21.15.5:0/3507722519 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f12f00b2910 0x7f12f00b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:48.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:48.341+0000 7f12f77fe700 1 -- 172.21.15.5:0/3507722519 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f13000e2230 con 0x7f1308109850 2024-05-09T00:45:48.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:48.341+0000 7f130659c700 1 --2- 172.21.15.5:0/3507722519 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f12f00b2910 0x7f12f00b4e20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f12f80060b0 tx=0x7f12f800c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:48.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:48.345+0000 7f12f77fe700 1 -- 172.21.15.5:0/3507722519 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1300014030 con 0x7f1308109850 2024-05-09T00:45:48.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:48.571+0000 7f130c9b0700 1 -- 172.21.15.5:0/3507722519 --> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1308063c00 con 0x7f12f00b2910 2024-05-09T00:45:48.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:48.578+0000 7f12f77fe700 1 -- 172.21.15.5:0/3507722519 <== mgr.14152 v2:172.21.15.5:6800/91856593 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258984 (secure 0 0 0) 0x7f1308063c00 con 0x7f12f00b2910 2024-05-09T00:45:48.584 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:45:48.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:48.582+0000 7f130c9b0700 1 -- 172.21.15.5:0/3507722519 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f12f00b2910 msgr2=0x7f12f00b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:48.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:48.582+0000 7f130c9b0700 1 --2- 172.21.15.5:0/3507722519 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f12f00b2910 0x7f12f00b4e20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f12f80060b0 tx=0x7f12f800c040 comp rx=0 tx=0).stop 2024-05-09T00:45:48.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:48.582+0000 7f130c9b0700 1 -- 172.21.15.5:0/3507722519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1308109850 msgr2=0x7f130819f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:48.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:48.582+0000 7f130c9b0700 1 --2- 172.21.15.5:0/3507722519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1308109850 0x7f130819f640 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f130000c900 tx=0x7f130000cc10 comp rx=0 tx=0).stop 2024-05-09T00:45:48.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:48.582+0000 7f130c9b0700 1 -- 172.21.15.5:0/3507722519 shutdown_connections 2024-05-09T00:45:48.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:48.582+0000 7f130c9b0700 1 --2- 172.21.15.5:0/3507722519 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f12f00b2910 0x7f12f00b4e20 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:48.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:48.582+0000 7f130c9b0700 1 --2- 172.21.15.5:0/3507722519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1308109850 0x7f130819f640 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:48.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:48.582+0000 7f130c9b0700 1 --2- 172.21.15.5:0/3507722519 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1308102ae0 0x7f130819ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:48.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:48.582+0000 7f130c9b0700 1 --2- 172.21.15.5:0/3507722519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1308108f00 0x7f130819b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:48.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:48.582+0000 7f130c9b0700 1 -- 172.21.15.5:0/3507722519 >> 172.21.15.5:0/3507722519 conn(0x7f13080fe520 msgr2=0x7f13081086e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:45:48.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:48.582+0000 7f130c9b0700 1 -- 172.21.15.5:0/3507722519 shutdown_connections 2024-05-09T00:45:48.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:48.582+0000 7f130c9b0700 1 -- 172.21.15.5:0/3507722519 wait complete. 2024-05-09T00:45:48.588 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-09T00:45:49.305 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":355,"stamp":"2024-05-09T00:45:47.890013+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660532,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1133,"num_read_kb":656,"num_write":372,"num_write_kb":2291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":3005105,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":306,"ondisk_log_size":306,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":347816,"kb_used_data":11064,"kb_used_omap":18,"kb_used_meta":336557,"kb_avail":1124594008,"statfs":{"total":1151940427776,"available":1151584264192,"internally_reserved":0,"allocated":11329536,"data_stored":7281276,"data_compressed":186585,"data_compressed_allocated":245760,"data_compressed_original":2015232,"omap_allocated":19141,"internal_metadata":344634683},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":115,"num_read_kb":55,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003547"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.934799+0000","last_change":"2024-05-09T00:43:43.875357+0000","last_active":"2024-05-09T00:45:23.934799+0000","last_peered":"2024-05-09T00:45:23.934799+0000","last_clean":"2024-05-09T00:45:23.934799+0000","last_became_active":"2024-05-09T00:43:43.873765+0000","last_became_peered":"2024-05-09T00:43:43.873765+0000","last_unstale":"2024-05-09T00:45:23.934799+0000","last_undegraded":"2024-05-09T00:45:23.934799+0000","last_fullsized":"2024-05-09T00:45:23.934799+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:47.519110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.940366+0000","last_change":"2024-05-09T00:43:43.882464+0000","last_active":"2024-05-09T00:45:23.940366+0000","last_peered":"2024-05-09T00:45:23.940366+0000","last_clean":"2024-05-09T00:45:23.940366+0000","last_became_active":"2024-05-09T00:43:43.882102+0000","last_became_peered":"2024-05-09T00:43:43.882102+0000","last_unstale":"2024-05-09T00:45:23.940366+0000","last_undegraded":"2024-05-09T00:45:23.940366+0000","last_fullsized":"2024-05-09T00:45:23.940366+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:39:36.234474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.940163+0000","last_change":"2024-05-09T00:43:43.873098+0000","last_active":"2024-05-09T00:45:23.940163+0000","last_peered":"2024-05-09T00:45:23.940163+0000","last_clean":"2024-05-09T00:45:23.940163+0000","last_became_active":"2024-05-09T00:43:43.872764+0000","last_became_peered":"2024-05-09T00:43:43.872764+0000","last_unstale":"2024-05-09T00:45:23.940163+0000","last_undegraded":"2024-05-09T00:45:23.940163+0000","last_fullsized":"2024-05-09T00:45:23.940163+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:05:30.784111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.939537+0000","last_change":"2024-05-09T00:43:43.869807+0000","last_active":"2024-05-09T00:45:23.939537+0000","last_peered":"2024-05-09T00:45:23.939537+0000","last_clean":"2024-05-09T00:45:23.939537+0000","last_became_active":"2024-05-09T00:43:43.869470+0000","last_became_peered":"2024-05-09T00:43:43.869470+0000","last_unstale":"2024-05-09T00:45:23.939537+0000","last_undegraded":"2024-05-09T00:45:23.939537+0000","last_fullsized":"2024-05-09T00:45:23.939537+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:25:40.201844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.937265+0000","last_change":"2024-05-09T00:43:43.874110+0000","last_active":"2024-05-09T00:45:23.937265+0000","last_peered":"2024-05-09T00:45:23.937265+0000","last_clean":"2024-05-09T00:45:23.937265+0000","last_became_active":"2024-05-09T00:43:43.873759+0000","last_became_peered":"2024-05-09T00:43:43.873759+0000","last_unstale":"2024-05-09T00:45:23.937265+0000","last_undegraded":"2024-05-09T00:45:23.937265+0000","last_fullsized":"2024-05-09T00:45:23.937265+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:17:17.722319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.080583+0000","last_change":"2024-05-09T00:44:23.022285+0000","last_active":"2024-05-09T00:45:24.080583+0000","last_peered":"2024-05-09T00:45:24.080583+0000","last_clean":"2024-05-09T00:45:24.080583+0000","last_became_active":"2024-05-09T00:44:23.021887+0000","last_became_peered":"2024-05-09T00:44:23.021887+0000","last_unstale":"2024-05-09T00:45:24.080583+0000","last_undegraded":"2024-05-09T00:45:24.080583+0000","last_fullsized":"2024-05-09T00:45:24.080583+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:07:07.702178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,7],"acting":[10,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.080401+0000","last_change":"2024-05-09T00:43:43.883112+0000","last_active":"2024-05-09T00:45:24.080401+0000","last_peered":"2024-05-09T00:45:24.080401+0000","last_clean":"2024-05-09T00:45:24.080401+0000","last_became_active":"2024-05-09T00:43:43.882795+0000","last_became_peered":"2024-05-09T00:43:43.882795+0000","last_unstale":"2024-05-09T00:45:24.080401+0000","last_undegraded":"2024-05-09T00:45:24.080401+0000","last_fullsized":"2024-05-09T00:45:24.080401+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:33:58.724015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.115355+0000","last_change":"2024-05-09T00:43:43.880696+0000","last_active":"2024-05-09T00:45:24.115355+0000","last_peered":"2024-05-09T00:45:24.115355+0000","last_clean":"2024-05-09T00:45:24.115355+0000","last_became_active":"2024-05-09T00:43:43.880174+0000","last_became_peered":"2024-05-09T00:43:43.880174+0000","last_unstale":"2024-05-09T00:45:24.115355+0000","last_undegraded":"2024-05-09T00:45:24.115355+0000","last_fullsized":"2024-05-09T00:45:24.115355+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:40:05.682432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.939132+0000","last_change":"2024-05-09T00:43:43.883449+0000","last_active":"2024-05-09T00:45:23.939132+0000","last_peered":"2024-05-09T00:45:23.939132+0000","last_clean":"2024-05-09T00:45:23.939132+0000","last_became_active":"2024-05-09T00:43:43.883066+0000","last_became_peered":"2024-05-09T00:43:43.883066+0000","last_unstale":"2024-05-09T00:45:23.939132+0000","last_undegraded":"2024-05-09T00:45:23.939132+0000","last_fullsized":"2024-05-09T00:45:23.939132+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:47:49.116004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.081169+0000","last_change":"2024-05-09T00:43:43.877514+0000","last_active":"2024-05-09T00:45:24.081169+0000","last_peered":"2024-05-09T00:45:24.081169+0000","last_clean":"2024-05-09T00:45:24.081169+0000","last_became_active":"2024-05-09T00:43:43.877170+0000","last_became_peered":"2024-05-09T00:43:43.877170+0000","last_unstale":"2024-05-09T00:45:24.081169+0000","last_undegraded":"2024-05-09T00:45:24.081169+0000","last_fullsized":"2024-05-09T00:45:24.081169+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:47:26.738571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.081150+0000","last_change":"2024-05-09T00:44:23.180294+0000","last_active":"2024-05-09T00:45:24.081150+0000","last_peered":"2024-05-09T00:45:24.081150+0000","last_clean":"2024-05-09T00:45:24.081150+0000","last_became_active":"2024-05-09T00:44:23.179824+0000","last_became_peered":"2024-05-09T00:44:23.179824+0000","last_unstale":"2024-05-09T00:45:24.081150+0000","last_undegraded":"2024-05-09T00:45:24.081150+0000","last_fullsized":"2024-05-09T00:45:24.081150+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:43:20.919660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,9],"acting":[7,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.939457+0000","last_change":"2024-05-09T00:43:43.873836+0000","last_active":"2024-05-09T00:45:23.939457+0000","last_peered":"2024-05-09T00:45:23.939457+0000","last_clean":"2024-05-09T00:45:23.939457+0000","last_became_active":"2024-05-09T00:43:43.873512+0000","last_became_peered":"2024-05-09T00:43:43.873512+0000","last_unstale":"2024-05-09T00:45:23.939457+0000","last_undegraded":"2024-05-09T00:45:23.939457+0000","last_fullsized":"2024-05-09T00:45:23.939457+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:32:06.162638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.934307+0000","last_change":"2024-05-09T00:43:43.885614+0000","last_active":"2024-05-09T00:45:23.934307+0000","last_peered":"2024-05-09T00:45:23.934307+0000","last_clean":"2024-05-09T00:45:23.934307+0000","last_became_active":"2024-05-09T00:43:43.884682+0000","last_became_peered":"2024-05-09T00:43:43.884682+0000","last_unstale":"2024-05-09T00:45:23.934307+0000","last_undegraded":"2024-05-09T00:45:23.934307+0000","last_fullsized":"2024-05-09T00:45:23.934307+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:34:15.336173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.939998+0000","last_change":"2024-05-09T00:43:43.883860+0000","last_active":"2024-05-09T00:45:23.939998+0000","last_peered":"2024-05-09T00:45:23.939998+0000","last_clean":"2024-05-09T00:45:23.939998+0000","last_became_active":"2024-05-09T00:43:43.882870+0000","last_became_peered":"2024-05-09T00:43:43.882870+0000","last_unstale":"2024-05-09T00:45:23.939998+0000","last_undegraded":"2024-05-09T00:45:23.939998+0000","last_fullsized":"2024-05-09T00:45:23.939998+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:17:31.566819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.939920+0000","last_change":"2024-05-09T00:43:43.874114+0000","last_active":"2024-05-09T00:45:23.939920+0000","last_peered":"2024-05-09T00:45:23.939920+0000","last_clean":"2024-05-09T00:45:23.939920+0000","last_became_active":"2024-05-09T00:43:43.873762+0000","last_became_peered":"2024-05-09T00:43:43.873762+0000","last_unstale":"2024-05-09T00:45:23.939920+0000","last_undegraded":"2024-05-09T00:45:23.939920+0000","last_fullsized":"2024-05-09T00:45:23.939920+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:24:27.486100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.938359+0000","last_change":"2024-05-09T00:43:43.872982+0000","last_active":"2024-05-09T00:45:23.938359+0000","last_peered":"2024-05-09T00:45:23.938359+0000","last_clean":"2024-05-09T00:45:23.938359+0000","last_became_active":"2024-05-09T00:43:43.872464+0000","last_became_peered":"2024-05-09T00:43:43.872464+0000","last_unstale":"2024-05-09T00:45:23.938359+0000","last_undegraded":"2024-05-09T00:45:23.938359+0000","last_fullsized":"2024-05-09T00:45:23.938359+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:41.172001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.934939+0000","last_change":"2024-05-09T00:43:43.875662+0000","last_active":"2024-05-09T00:45:23.934939+0000","last_peered":"2024-05-09T00:45:23.934939+0000","last_clean":"2024-05-09T00:45:23.934939+0000","last_became_active":"2024-05-09T00:43:43.874701+0000","last_became_peered":"2024-05-09T00:43:43.874701+0000","last_unstale":"2024-05-09T00:45:23.934939+0000","last_undegraded":"2024-05-09T00:45:23.934939+0000","last_fullsized":"2024-05-09T00:45:23.934939+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:09:19.349190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.938226+0000","last_change":"2024-05-09T00:43:43.876100+0000","last_active":"2024-05-09T00:45:23.938226+0000","last_peered":"2024-05-09T00:45:23.938226+0000","last_clean":"2024-05-09T00:45:23.938226+0000","last_became_active":"2024-05-09T00:43:43.875768+0000","last_became_peered":"2024-05-09T00:43:43.875768+0000","last_unstale":"2024-05-09T00:45:23.938226+0000","last_undegraded":"2024-05-09T00:45:23.938226+0000","last_fullsized":"2024-05-09T00:45:23.938226+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:48:26.843064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.116311+0000","last_change":"2024-05-09T00:43:43.875900+0000","last_active":"2024-05-09T00:45:24.116311+0000","last_peered":"2024-05-09T00:45:24.116311+0000","last_clean":"2024-05-09T00:45:24.116311+0000","last_became_active":"2024-05-09T00:43:43.875497+0000","last_became_peered":"2024-05-09T00:43:43.875497+0000","last_unstale":"2024-05-09T00:45:24.116311+0000","last_undegraded":"2024-05-09T00:45:24.116311+0000","last_fullsized":"2024-05-09T00:45:24.116311+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:54:44.215936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'13","reported_seq":150,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:47.112106+0000","last_change":"2024-05-09T00:43:40.854337+0000","last_active":"2024-05-09T00:45:47.112106+0000","last_peered":"2024-05-09T00:45:47.112106+0000","last_clean":"2024-05-09T00:45:47.112106+0000","last_became_active":"2024-05-09T00:43:40.853986+0000","last_became_peered":"2024-05-09T00:43:40.853986+0000","last_unstale":"2024-05-09T00:45:47.112106+0000","last_undegraded":"2024-05-09T00:45:47.112106+0000","last_fullsized":"2024-05-09T00:45:47.112106+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:01:19.128533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":106,"num_read_kb":106,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.115812+0000","last_change":"2024-05-09T00:43:43.878081+0000","last_active":"2024-05-09T00:45:24.115812+0000","last_peered":"2024-05-09T00:45:24.115812+0000","last_clean":"2024-05-09T00:45:24.115812+0000","last_became_active":"2024-05-09T00:43:43.877585+0000","last_became_peered":"2024-05-09T00:43:43.877585+0000","last_unstale":"2024-05-09T00:45:24.115812+0000","last_undegraded":"2024-05-09T00:45:24.115812+0000","last_fullsized":"2024-05-09T00:45:24.115812+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:17:39.255841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.115936+0000","last_change":"2024-05-09T00:43:40.959386+0000","last_active":"2024-05-09T00:45:24.115936+0000","last_peered":"2024-05-09T00:45:24.115936+0000","last_clean":"2024-05-09T00:45:24.115936+0000","last_became_active":"2024-05-09T00:43:40.958878+0000","last_became_peered":"2024-05-09T00:43:40.958878+0000","last_unstale":"2024-05-09T00:45:24.115936+0000","last_undegraded":"2024-05-09T00:45:24.115936+0000","last_fullsized":"2024-05-09T00:45:24.115936+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:14:57.105078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.940324+0000","last_change":"2024-05-09T00:43:43.870563+0000","last_active":"2024-05-09T00:45:23.940324+0000","last_peered":"2024-05-09T00:45:23.940324+0000","last_clean":"2024-05-09T00:45:23.940324+0000","last_became_active":"2024-05-09T00:43:43.870237+0000","last_became_peered":"2024-05-09T00:43:43.870237+0000","last_unstale":"2024-05-09T00:45:23.940324+0000","last_undegraded":"2024-05-09T00:45:23.940324+0000","last_fullsized":"2024-05-09T00:45:23.940324+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:14:28.006343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"78'12","reported_seq":43,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.937305+0000","last_change":"2024-05-09T00:43:40.857670+0000","last_active":"2024-05-09T00:45:23.937305+0000","last_peered":"2024-05-09T00:45:23.937305+0000","last_clean":"2024-05-09T00:45:23.937305+0000","last_became_active":"2024-05-09T00:43:40.857247+0000","last_became_peered":"2024-05-09T00:43:40.857247+0000","last_unstale":"2024-05-09T00:45:23.937305+0000","last_undegraded":"2024-05-09T00:45:23.937305+0000","last_fullsized":"2024-05-09T00:45:23.937305+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:47:06.325593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.938611+0000","last_change":"2024-05-09T00:43:43.883179+0000","last_active":"2024-05-09T00:45:23.938611+0000","last_peered":"2024-05-09T00:45:23.938611+0000","last_clean":"2024-05-09T00:45:23.938611+0000","last_became_active":"2024-05-09T00:43:43.882820+0000","last_became_peered":"2024-05-09T00:43:43.882820+0000","last_unstale":"2024-05-09T00:45:23.938611+0000","last_undegraded":"2024-05-09T00:45:23.938611+0000","last_fullsized":"2024-05-09T00:45:23.938611+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:56:26.012495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"78'5","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.946133+0000","last_change":"2024-05-09T00:45:23.945720+0000","last_active":"2024-05-09T00:45:23.946133+0000","last_peered":"2024-05-09T00:45:23.946133+0000","last_clean":"2024-05-09T00:45:23.946133+0000","last_became_active":"2024-05-09T00:45:23.940421+0000","last_became_peered":"2024-05-09T00:45:23.940421+0000","last_unstale":"2024-05-09T00:45:23.946133+0000","last_undegraded":"2024-05-09T00:45:23.946133+0000","last_fullsized":"2024-05-09T00:45:23.946133+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:23:24.905715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.938187+0000","last_change":"2024-05-09T00:43:43.882950+0000","last_active":"2024-05-09T00:45:23.938187+0000","last_peered":"2024-05-09T00:45:23.938187+0000","last_clean":"2024-05-09T00:45:23.938187+0000","last_became_active":"2024-05-09T00:43:43.880679+0000","last_became_peered":"2024-05-09T00:43:43.880679+0000","last_unstale":"2024-05-09T00:45:23.938187+0000","last_undegraded":"2024-05-09T00:45:23.938187+0000","last_fullsized":"2024-05-09T00:45:23.938187+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:34:14.050987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"78'6","reported_seq":140,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:42.107398+0000","last_change":"2024-05-09T00:43:40.854953+0000","last_active":"2024-05-09T00:45:42.107398+0000","last_peered":"2024-05-09T00:45:42.107398+0000","last_clean":"2024-05-09T00:45:42.107398+0000","last_became_active":"2024-05-09T00:43:40.854562+0000","last_became_peered":"2024-05-09T00:43:40.854562+0000","last_unstale":"2024-05-09T00:45:42.107398+0000","last_undegraded":"2024-05-09T00:45:42.107398+0000","last_fullsized":"2024-05-09T00:45:42.107398+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:09:19.564806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.937264+0000","last_change":"2024-05-09T00:43:43.881881+0000","last_active":"2024-05-09T00:45:23.937264+0000","last_peered":"2024-05-09T00:45:23.937264+0000","last_clean":"2024-05-09T00:45:23.937264+0000","last_became_active":"2024-05-09T00:43:43.881461+0000","last_became_peered":"2024-05-09T00:43:43.881461+0000","last_unstale":"2024-05-09T00:45:23.937264+0000","last_undegraded":"2024-05-09T00:45:23.937264+0000","last_fullsized":"2024-05-09T00:45:23.937264+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:59:28.747084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"78'7","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.080529+0000","last_change":"2024-05-09T00:43:40.858521+0000","last_active":"2024-05-09T00:45:24.080529+0000","last_peered":"2024-05-09T00:45:24.080529+0000","last_clean":"2024-05-09T00:45:24.080529+0000","last_became_active":"2024-05-09T00:43:40.858102+0000","last_became_peered":"2024-05-09T00:43:40.858102+0000","last_unstale":"2024-05-09T00:45:24.080529+0000","last_undegraded":"2024-05-09T00:45:24.080529+0000","last_fullsized":"2024-05-09T00:45:24.080529+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:09:51.684630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.937931+0000","last_change":"2024-05-09T00:44:23.023472+0000","last_active":"2024-05-09T00:45:23.937931+0000","last_peered":"2024-05-09T00:45:23.937931+0000","last_clean":"2024-05-09T00:45:23.937931+0000","last_became_active":"2024-05-09T00:44:23.023017+0000","last_became_peered":"2024-05-09T00:44:23.023017+0000","last_unstale":"2024-05-09T00:45:23.937931+0000","last_undegraded":"2024-05-09T00:45:23.937931+0000","last_fullsized":"2024-05-09T00:45:23.937931+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:05:13.852606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"78'3","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.933923+0000","last_change":"2024-05-09T00:43:40.853329+0000","last_active":"2024-05-09T00:45:23.933923+0000","last_peered":"2024-05-09T00:45:23.933923+0000","last_clean":"2024-05-09T00:45:23.933923+0000","last_became_active":"2024-05-09T00:43:40.852979+0000","last_became_peered":"2024-05-09T00:43:40.852979+0000","last_unstale":"2024-05-09T00:45:23.933923+0000","last_undegraded":"2024-05-09T00:45:23.933923+0000","last_fullsized":"2024-05-09T00:45:23.933923+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:13:57.006031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.115441+0000","last_change":"2024-05-09T00:43:43.862876+0000","last_active":"2024-05-09T00:45:24.115441+0000","last_peered":"2024-05-09T00:45:24.115441+0000","last_clean":"2024-05-09T00:45:24.115441+0000","last_became_active":"2024-05-09T00:43:43.862521+0000","last_became_peered":"2024-05-09T00:43:43.862521+0000","last_unstale":"2024-05-09T00:45:24.115441+0000","last_undegraded":"2024-05-09T00:45:24.115441+0000","last_fullsized":"2024-05-09T00:45:24.115441+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:45:49.386844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'3","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.937477+0000","last_change":"2024-05-09T00:43:40.860395+0000","last_active":"2024-05-09T00:45:23.937477+0000","last_peered":"2024-05-09T00:45:23.937477+0000","last_clean":"2024-05-09T00:45:23.937477+0000","last_became_active":"2024-05-09T00:43:40.860040+0000","last_became_peered":"2024-05-09T00:43:40.860040+0000","last_unstale":"2024-05-09T00:45:23.937477+0000","last_undegraded":"2024-05-09T00:45:23.937477+0000","last_fullsized":"2024-05-09T00:45:23.937477+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:20:36.054685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.937729+0000","last_change":"2024-05-09T00:44:23.180905+0000","last_active":"2024-05-09T00:45:23.937729+0000","last_peered":"2024-05-09T00:45:23.937729+0000","last_clean":"2024-05-09T00:45:23.937729+0000","last_became_active":"2024-05-09T00:44:23.180296+0000","last_became_peered":"2024-05-09T00:44:23.180296+0000","last_unstale":"2024-05-09T00:45:23.937729+0000","last_undegraded":"2024-05-09T00:45:23.937729+0000","last_fullsized":"2024-05-09T00:45:23.937729+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:48:17.585276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,10],"acting":[9,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"78'4","reported_seq":236,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:43.108092+0000","last_change":"2024-05-09T00:43:40.860542+0000","last_active":"2024-05-09T00:45:43.108092+0000","last_peered":"2024-05-09T00:45:43.108092+0000","last_clean":"2024-05-09T00:45:43.108092+0000","last_became_active":"2024-05-09T00:43:40.860176+0000","last_became_peered":"2024-05-09T00:43:40.860176+0000","last_unstale":"2024-05-09T00:45:43.108092+0000","last_undegraded":"2024-05-09T00:45:43.108092+0000","last_fullsized":"2024-05-09T00:45:43.108092+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:44:50.375024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":202,"num_read_kb":202,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.939053+0000","last_change":"2024-05-09T00:43:43.882119+0000","last_active":"2024-05-09T00:45:23.939053+0000","last_peered":"2024-05-09T00:45:23.939053+0000","last_clean":"2024-05-09T00:45:23.939053+0000","last_became_active":"2024-05-09T00:43:43.881747+0000","last_became_peered":"2024-05-09T00:43:43.881747+0000","last_unstale":"2024-05-09T00:45:23.939053+0000","last_undegraded":"2024-05-09T00:45:23.939053+0000","last_fullsized":"2024-05-09T00:45:23.939053+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:54:19.420435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'4","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.081233+0000","last_change":"2024-05-09T00:43:40.859539+0000","last_active":"2024-05-09T00:45:24.081233+0000","last_peered":"2024-05-09T00:45:24.081233+0000","last_clean":"2024-05-09T00:45:24.081233+0000","last_became_active":"2024-05-09T00:43:40.859162+0000","last_became_peered":"2024-05-09T00:43:40.859162+0000","last_unstale":"2024-05-09T00:45:24.081233+0000","last_undegraded":"2024-05-09T00:45:24.081233+0000","last_fullsized":"2024-05-09T00:45:24.081233+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:37:15.190944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.940407+0000","last_change":"2024-05-09T00:44:23.180872+0000","last_active":"2024-05-09T00:45:23.940407+0000","last_peered":"2024-05-09T00:45:23.940407+0000","last_clean":"2024-05-09T00:45:23.940407+0000","last_became_active":"2024-05-09T00:44:23.180398+0000","last_became_peered":"2024-05-09T00:44:23.180398+0000","last_unstale":"2024-05-09T00:45:23.940407+0000","last_undegraded":"2024-05-09T00:45:23.940407+0000","last_fullsized":"2024-05-09T00:45:23.940407+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:19:27.909490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,6],"acting":[0,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'5","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.938299+0000","last_change":"2024-05-09T00:43:40.960126+0000","last_active":"2024-05-09T00:45:23.938299+0000","last_peered":"2024-05-09T00:45:23.938299+0000","last_clean":"2024-05-09T00:45:23.938299+0000","last_became_active":"2024-05-09T00:43:40.959441+0000","last_became_peered":"2024-05-09T00:43:40.959441+0000","last_unstale":"2024-05-09T00:45:23.938299+0000","last_undegraded":"2024-05-09T00:45:23.938299+0000","last_fullsized":"2024-05-09T00:45:23.938299+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:38:47.497029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.938766+0000","last_change":"2024-05-09T00:43:43.874223+0000","last_active":"2024-05-09T00:45:23.938766+0000","last_peered":"2024-05-09T00:45:23.938766+0000","last_clean":"2024-05-09T00:45:23.938766+0000","last_became_active":"2024-05-09T00:43:43.873875+0000","last_became_peered":"2024-05-09T00:43:43.873875+0000","last_unstale":"2024-05-09T00:45:23.938766+0000","last_undegraded":"2024-05-09T00:45:23.938766+0000","last_fullsized":"2024-05-09T00:45:23.938766+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:59:22.192886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'32","reported_seq":291,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:42.315058+0000","last_change":"2024-05-09T00:44:23.188953+0000","last_active":"2024-05-09T00:45:42.315058+0000","last_peered":"2024-05-09T00:45:42.315058+0000","last_clean":"2024-05-09T00:45:42.315058+0000","last_became_active":"2024-05-09T00:44:23.180858+0000","last_became_peered":"2024-05-09T00:44:23.180858+0000","last_unstale":"2024-05-09T00:45:42.315058+0000","last_undegraded":"2024-05-09T00:45:42.315058+0000","last_fullsized":"2024-05-09T00:45:42.315058+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:26:16.427665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62335,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":204,"num_read_kb":152,"num_write":34,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":52683,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.936978+0000","last_change":"2024-05-09T00:43:43.872019+0000","last_active":"2024-05-09T00:45:23.936978+0000","last_peered":"2024-05-09T00:45:23.936978+0000","last_clean":"2024-05-09T00:45:23.936978+0000","last_became_active":"2024-05-09T00:43:43.871622+0000","last_became_peered":"2024-05-09T00:43:43.871622+0000","last_unstale":"2024-05-09T00:45:23.936978+0000","last_undegraded":"2024-05-09T00:45:23.936978+0000","last_fullsized":"2024-05-09T00:45:23.936978+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:41:47.789443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'9","reported_seq":41,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.116151+0000","last_change":"2024-05-09T00:43:40.860111+0000","last_active":"2024-05-09T00:45:24.116151+0000","last_peered":"2024-05-09T00:45:24.116151+0000","last_clean":"2024-05-09T00:45:24.116151+0000","last_became_active":"2024-05-09T00:43:40.859732+0000","last_became_peered":"2024-05-09T00:43:40.859732+0000","last_unstale":"2024-05-09T00:45:24.116151+0000","last_undegraded":"2024-05-09T00:45:24.116151+0000","last_fullsized":"2024-05-09T00:45:24.116151+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:30:03.180679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.934086+0000","last_change":"2024-05-09T00:43:43.870604+0000","last_active":"2024-05-09T00:45:23.934086+0000","last_peered":"2024-05-09T00:45:23.934086+0000","last_clean":"2024-05-09T00:45:23.934086+0000","last_became_active":"2024-05-09T00:43:43.870286+0000","last_became_peered":"2024-05-09T00:43:43.870286+0000","last_unstale":"2024-05-09T00:45:23.934086+0000","last_undegraded":"2024-05-09T00:45:23.934086+0000","last_fullsized":"2024-05-09T00:45:23.934086+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:57:04.486631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'5","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.937799+0000","last_change":"2024-05-09T00:43:40.851966+0000","last_active":"2024-05-09T00:45:23.937799+0000","last_peered":"2024-05-09T00:45:23.937799+0000","last_clean":"2024-05-09T00:45:23.937799+0000","last_became_active":"2024-05-09T00:43:40.851537+0000","last_became_peered":"2024-05-09T00:43:40.851537+0000","last_unstale":"2024-05-09T00:45:23.937799+0000","last_undegraded":"2024-05-09T00:45:23.937799+0000","last_fullsized":"2024-05-09T00:45:23.937799+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:58:59.797183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.081006+0000","last_change":"2024-05-09T00:43:43.879509+0000","last_active":"2024-05-09T00:45:24.081006+0000","last_peered":"2024-05-09T00:45:24.081006+0000","last_clean":"2024-05-09T00:45:24.081006+0000","last_became_active":"2024-05-09T00:43:43.879151+0000","last_became_peered":"2024-05-09T00:43:43.879151+0000","last_unstale":"2024-05-09T00:45:24.081006+0000","last_undegraded":"2024-05-09T00:45:24.081006+0000","last_fullsized":"2024-05-09T00:45:24.081006+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:34:18.850545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"78'6","reported_seq":144,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:46.111260+0000","last_change":"2024-05-09T00:43:40.857629+0000","last_active":"2024-05-09T00:45:46.111260+0000","last_peered":"2024-05-09T00:45:46.111260+0000","last_clean":"2024-05-09T00:45:46.111260+0000","last_became_active":"2024-05-09T00:43:40.857260+0000","last_became_peered":"2024-05-09T00:43:40.857260+0000","last_unstale":"2024-05-09T00:45:46.111260+0000","last_undegraded":"2024-05-09T00:45:46.111260+0000","last_fullsized":"2024-05-09T00:45:46.111260+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:51:00.600870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":109,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.080625+0000","last_change":"2024-05-09T00:43:43.886100+0000","last_active":"2024-05-09T00:45:24.080625+0000","last_peered":"2024-05-09T00:45:24.080625+0000","last_clean":"2024-05-09T00:45:24.080625+0000","last_became_active":"2024-05-09T00:43:43.885775+0000","last_became_peered":"2024-05-09T00:43:43.885775+0000","last_unstale":"2024-05-09T00:45:24.080625+0000","last_undegraded":"2024-05-09T00:45:24.080625+0000","last_fullsized":"2024-05-09T00:45:24.080625+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:34:19.439163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'17","reported_seq":48,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.936761+0000","last_change":"2024-05-09T00:43:40.959873+0000","last_active":"2024-05-09T00:45:23.936761+0000","last_peered":"2024-05-09T00:45:23.936761+0000","last_clean":"2024-05-09T00:45:23.936761+0000","last_became_active":"2024-05-09T00:43:40.959437+0000","last_became_peered":"2024-05-09T00:43:40.959437+0000","last_unstale":"2024-05-09T00:45:23.936761+0000","last_undegraded":"2024-05-09T00:45:23.936761+0000","last_fullsized":"2024-05-09T00:45:23.936761+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:29:52.686937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.937632+0000","last_change":"2024-05-09T00:43:43.880076+0000","last_active":"2024-05-09T00:45:23.937632+0000","last_peered":"2024-05-09T00:45:23.937632+0000","last_clean":"2024-05-09T00:45:23.937632+0000","last_became_active":"2024-05-09T00:43:43.878733+0000","last_became_peered":"2024-05-09T00:43:43.878733+0000","last_unstale":"2024-05-09T00:45:23.937632+0000","last_undegraded":"2024-05-09T00:45:23.937632+0000","last_fullsized":"2024-05-09T00:45:23.937632+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:30:21.426855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.937133+0000","last_change":"2024-05-09T00:43:40.858033+0000","last_active":"2024-05-09T00:45:23.937133+0000","last_peered":"2024-05-09T00:45:23.937133+0000","last_clean":"2024-05-09T00:45:23.937133+0000","last_became_active":"2024-05-09T00:43:40.857075+0000","last_became_peered":"2024-05-09T00:43:40.857075+0000","last_unstale":"2024-05-09T00:45:23.937133+0000","last_undegraded":"2024-05-09T00:45:23.937133+0000","last_fullsized":"2024-05-09T00:45:23.937133+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:01:49.690226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.934621+0000","last_change":"2024-05-09T00:44:23.180515+0000","last_active":"2024-05-09T00:45:23.934621+0000","last_peered":"2024-05-09T00:45:23.934621+0000","last_clean":"2024-05-09T00:45:23.934621+0000","last_became_active":"2024-05-09T00:44:23.179834+0000","last_became_peered":"2024-05-09T00:44:23.179834+0000","last_unstale":"2024-05-09T00:45:23.934621+0000","last_undegraded":"2024-05-09T00:45:23.934621+0000","last_fullsized":"2024-05-09T00:45:23.934621+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:21:55.735438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,6],"acting":[11,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"78'1","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.941920+0000","last_change":"2024-05-09T00:45:23.941489+0000","last_active":"2024-05-09T00:45:23.941920+0000","last_peered":"2024-05-09T00:45:23.941920+0000","last_clean":"2024-05-09T00:45:23.941920+0000","last_became_active":"2024-05-09T00:45:23.941072+0000","last_became_peered":"2024-05-09T00:45:23.941072+0000","last_unstale":"2024-05-09T00:45:23.941920+0000","last_undegraded":"2024-05-09T00:45:23.941920+0000","last_fullsized":"2024-05-09T00:45:23.941920+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:20:05.755293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"78'1","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.081020+0000","last_change":"2024-05-09T00:43:43.886618+0000","last_active":"2024-05-09T00:45:24.081020+0000","last_peered":"2024-05-09T00:45:24.081020+0000","last_clean":"2024-05-09T00:45:24.081020+0000","last_became_active":"2024-05-09T00:43:43.885946+0000","last_became_peered":"2024-05-09T00:43:43.885946+0000","last_unstale":"2024-05-09T00:45:24.081020+0000","last_undegraded":"2024-05-09T00:45:24.081020+0000","last_fullsized":"2024-05-09T00:45:24.081020+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:16:22.169548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"78'5","reported_seq":139,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:44.108807+0000","last_change":"2024-05-09T00:43:40.856213+0000","last_active":"2024-05-09T00:45:44.108807+0000","last_peered":"2024-05-09T00:45:44.108807+0000","last_clean":"2024-05-09T00:45:44.108807+0000","last_became_active":"2024-05-09T00:43:40.855700+0000","last_became_peered":"2024-05-09T00:43:40.855700+0000","last_unstale":"2024-05-09T00:45:44.108807+0000","last_undegraded":"2024-05-09T00:45:44.108807+0000","last_fullsized":"2024-05-09T00:45:44.108807+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:48:50.238293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":103,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.080522+0000","last_change":"2024-05-09T00:43:43.885272+0000","last_active":"2024-05-09T00:45:24.080522+0000","last_peered":"2024-05-09T00:45:24.080522+0000","last_clean":"2024-05-09T00:45:24.080522+0000","last_became_active":"2024-05-09T00:43:43.884774+0000","last_became_peered":"2024-05-09T00:43:43.884774+0000","last_unstale":"2024-05-09T00:45:24.080522+0000","last_undegraded":"2024-05-09T00:45:24.080522+0000","last_fullsized":"2024-05-09T00:45:24.080522+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:14:02.193859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.937922+0000","last_change":"2024-05-09T00:43:43.883370+0000","last_active":"2024-05-09T00:45:23.937922+0000","last_peered":"2024-05-09T00:45:23.937922+0000","last_clean":"2024-05-09T00:45:23.937922+0000","last_became_active":"2024-05-09T00:43:43.881570+0000","last_became_peered":"2024-05-09T00:43:43.881570+0000","last_unstale":"2024-05-09T00:45:23.937922+0000","last_undegraded":"2024-05-09T00:45:23.937922+0000","last_fullsized":"2024-05-09T00:45:23.937922+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:48.936861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"78'5","reported_seq":137,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:42.105278+0000","last_change":"2024-05-09T00:43:40.959828+0000","last_active":"2024-05-09T00:45:42.105278+0000","last_peered":"2024-05-09T00:45:42.105278+0000","last_clean":"2024-05-09T00:45:42.105278+0000","last_became_active":"2024-05-09T00:43:40.958889+0000","last_became_peered":"2024-05-09T00:43:40.958889+0000","last_unstale":"2024-05-09T00:45:42.105278+0000","last_undegraded":"2024-05-09T00:45:42.105278+0000","last_fullsized":"2024-05-09T00:45:42.105278+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:24:50.113298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":102,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.115638+0000","last_change":"2024-05-09T00:43:43.876144+0000","last_active":"2024-05-09T00:45:24.115638+0000","last_peered":"2024-05-09T00:45:24.115638+0000","last_clean":"2024-05-09T00:45:24.115638+0000","last_became_active":"2024-05-09T00:43:43.875774+0000","last_became_peered":"2024-05-09T00:43:43.875774+0000","last_unstale":"2024-05-09T00:45:24.115638+0000","last_undegraded":"2024-05-09T00:45:24.115638+0000","last_fullsized":"2024-05-09T00:45:24.115638+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:05:23.145431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"78'6","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.939598+0000","last_change":"2024-05-09T00:43:40.857278+0000","last_active":"2024-05-09T00:45:23.939598+0000","last_peered":"2024-05-09T00:45:23.939598+0000","last_clean":"2024-05-09T00:45:23.939598+0000","last_became_active":"2024-05-09T00:43:40.856861+0000","last_became_peered":"2024-05-09T00:43:40.856861+0000","last_unstale":"2024-05-09T00:45:23.939598+0000","last_undegraded":"2024-05-09T00:45:23.939598+0000","last_fullsized":"2024-05-09T00:45:23.939598+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:36:16.603327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":103,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.080385+0000","last_change":"2024-05-09T00:42:14.528544+0000","last_active":"2024-05-09T00:45:24.080385+0000","last_peered":"2024-05-09T00:45:24.080385+0000","last_clean":"2024-05-09T00:45:24.080385+0000","last_became_active":"2024-05-09T00:42:14.523658+0000","last_became_peered":"2024-05-09T00:42:14.523658+0000","last_unstale":"2024-05-09T00:45:24.080385+0000","last_undegraded":"2024-05-09T00:45:24.080385+0000","last_fullsized":"2024-05-09T00:45:24.080385+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:38:48.637995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:38:48.637995+0000","last_clean_scrub_stamp":"2024-05-09T00:38:48.637995+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:42:28.249240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.933721+0000","last_change":"2024-05-09T00:43:43.873638+0000","last_active":"2024-05-09T00:45:23.933721+0000","last_peered":"2024-05-09T00:45:23.933721+0000","last_clean":"2024-05-09T00:45:23.933721+0000","last_became_active":"2024-05-09T00:43:43.871069+0000","last_became_peered":"2024-05-09T00:43:43.871069+0000","last_unstale":"2024-05-09T00:45:23.933721+0000","last_undegraded":"2024-05-09T00:45:23.933721+0000","last_fullsized":"2024-05-09T00:45:23.933721+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:28:34.996989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"78'4","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.080673+0000","last_change":"2024-05-09T00:43:40.856374+0000","last_active":"2024-05-09T00:45:24.080673+0000","last_peered":"2024-05-09T00:45:24.080673+0000","last_clean":"2024-05-09T00:45:24.080673+0000","last_became_active":"2024-05-09T00:43:40.855695+0000","last_became_peered":"2024-05-09T00:43:40.855695+0000","last_unstale":"2024-05-09T00:45:24.080673+0000","last_undegraded":"2024-05-09T00:45:24.080673+0000","last_fullsized":"2024-05-09T00:45:24.080673+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:33:43.734132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"78'4","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.937089+0000","last_change":"2024-05-09T00:43:40.959303+0000","last_active":"2024-05-09T00:45:23.937089+0000","last_peered":"2024-05-09T00:45:23.937089+0000","last_clean":"2024-05-09T00:45:23.937089+0000","last_became_active":"2024-05-09T00:43:40.958855+0000","last_became_peered":"2024-05-09T00:43:40.958855+0000","last_unstale":"2024-05-09T00:45:23.937089+0000","last_undegraded":"2024-05-09T00:45:23.937089+0000","last_fullsized":"2024-05-09T00:45:23.937089+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:53:22.566850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.934495+0000","last_change":"2024-05-09T00:43:43.872041+0000","last_active":"2024-05-09T00:45:23.934495+0000","last_peered":"2024-05-09T00:45:23.934495+0000","last_clean":"2024-05-09T00:45:23.934495+0000","last_became_active":"2024-05-09T00:43:43.870946+0000","last_became_peered":"2024-05-09T00:43:43.870946+0000","last_unstale":"2024-05-09T00:45:23.934495+0000","last_undegraded":"2024-05-09T00:45:23.934495+0000","last_fullsized":"2024-05-09T00:45:23.934495+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:16:28.190488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'15","reported_seq":45,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.938203+0000","last_change":"2024-05-09T00:43:40.853875+0000","last_active":"2024-05-09T00:45:23.938203+0000","last_peered":"2024-05-09T00:45:23.938203+0000","last_clean":"2024-05-09T00:45:23.938203+0000","last_became_active":"2024-05-09T00:43:40.853459+0000","last_became_peered":"2024-05-09T00:43:40.853459+0000","last_unstale":"2024-05-09T00:45:23.938203+0000","last_undegraded":"2024-05-09T00:45:23.938203+0000","last_fullsized":"2024-05-09T00:45:23.938203+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:45:29.992730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.115875+0000","last_change":"2024-05-09T00:43:43.870435+0000","last_active":"2024-05-09T00:45:24.115875+0000","last_peered":"2024-05-09T00:45:24.115875+0000","last_clean":"2024-05-09T00:45:24.115875+0000","last_became_active":"2024-05-09T00:43:43.870075+0000","last_became_peered":"2024-05-09T00:43:43.870075+0000","last_unstale":"2024-05-09T00:45:24.115875+0000","last_undegraded":"2024-05-09T00:45:24.115875+0000","last_fullsized":"2024-05-09T00:45:24.115875+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:53:12.958463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'7","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.939765+0000","last_change":"2024-05-09T00:43:41.119325+0000","last_active":"2024-05-09T00:45:23.939765+0000","last_peered":"2024-05-09T00:45:23.939765+0000","last_clean":"2024-05-09T00:45:23.939765+0000","last_became_active":"2024-05-09T00:43:41.119009+0000","last_became_peered":"2024-05-09T00:43:41.119009+0000","last_unstale":"2024-05-09T00:45:23.939765+0000","last_undegraded":"2024-05-09T00:45:23.939765+0000","last_fullsized":"2024-05-09T00:45:23.939765+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:10:05.775152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.940519+0000","last_change":"2024-05-09T00:43:43.875236+0000","last_active":"2024-05-09T00:45:23.940519+0000","last_peered":"2024-05-09T00:45:23.940519+0000","last_clean":"2024-05-09T00:45:23.940519+0000","last_became_active":"2024-05-09T00:43:43.874919+0000","last_became_peered":"2024-05-09T00:43:43.874919+0000","last_unstale":"2024-05-09T00:45:23.940519+0000","last_undegraded":"2024-05-09T00:45:23.940519+0000","last_fullsized":"2024-05-09T00:45:23.940519+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:45:33.232518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.938358+0000","last_change":"2024-05-09T00:43:40.959765+0000","last_active":"2024-05-09T00:45:23.938358+0000","last_peered":"2024-05-09T00:45:23.938358+0000","last_clean":"2024-05-09T00:45:23.938358+0000","last_became_active":"2024-05-09T00:43:40.959383+0000","last_became_peered":"2024-05-09T00:43:40.959383+0000","last_unstale":"2024-05-09T00:45:23.938358+0000","last_undegraded":"2024-05-09T00:45:23.938358+0000","last_fullsized":"2024-05-09T00:45:23.938358+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:13:28.766650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.937040+0000","last_change":"2024-05-09T00:43:43.881627+0000","last_active":"2024-05-09T00:45:23.937040+0000","last_peered":"2024-05-09T00:45:23.937040+0000","last_clean":"2024-05-09T00:45:23.937040+0000","last_became_active":"2024-05-09T00:43:43.881207+0000","last_became_peered":"2024-05-09T00:43:43.881207+0000","last_unstale":"2024-05-09T00:45:23.937040+0000","last_undegraded":"2024-05-09T00:45:23.937040+0000","last_fullsized":"2024-05-09T00:45:23.937040+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:44:08.053638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'7","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.115434+0000","last_change":"2024-05-09T00:43:40.859566+0000","last_active":"2024-05-09T00:45:24.115434+0000","last_peered":"2024-05-09T00:45:24.115434+0000","last_clean":"2024-05-09T00:45:24.115434+0000","last_became_active":"2024-05-09T00:43:40.859182+0000","last_became_peered":"2024-05-09T00:43:40.859182+0000","last_unstale":"2024-05-09T00:45:24.115434+0000","last_undegraded":"2024-05-09T00:45:24.115434+0000","last_fullsized":"2024-05-09T00:45:24.115434+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:58:12.693271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.934380+0000","last_change":"2024-05-09T00:43:43.869193+0000","last_active":"2024-05-09T00:45:23.934380+0000","last_peered":"2024-05-09T00:45:23.934380+0000","last_clean":"2024-05-09T00:45:23.934380+0000","last_became_active":"2024-05-09T00:43:43.868823+0000","last_became_peered":"2024-05-09T00:43:43.868823+0000","last_unstale":"2024-05-09T00:45:23.934380+0000","last_undegraded":"2024-05-09T00:45:23.934380+0000","last_fullsized":"2024-05-09T00:45:23.934380+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:15:53.929873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'3","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.116089+0000","last_change":"2024-05-09T00:43:40.855038+0000","last_active":"2024-05-09T00:45:24.116089+0000","last_peered":"2024-05-09T00:45:24.116089+0000","last_clean":"2024-05-09T00:45:24.116089+0000","last_became_active":"2024-05-09T00:43:40.854334+0000","last_became_peered":"2024-05-09T00:43:40.854334+0000","last_unstale":"2024-05-09T00:45:24.116089+0000","last_undegraded":"2024-05-09T00:45:24.116089+0000","last_fullsized":"2024-05-09T00:45:24.116089+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:47:13.897697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.937849+0000","last_change":"2024-05-09T00:44:23.019151+0000","last_active":"2024-05-09T00:45:23.937849+0000","last_peered":"2024-05-09T00:45:23.937849+0000","last_clean":"2024-05-09T00:45:23.937849+0000","last_became_active":"2024-05-09T00:44:23.018752+0000","last_became_peered":"2024-05-09T00:44:23.018752+0000","last_unstale":"2024-05-09T00:45:23.937849+0000","last_undegraded":"2024-05-09T00:45:23.937849+0000","last_fullsized":"2024-05-09T00:45:23.937849+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:28:41.875747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.937399+0000","last_change":"2024-05-09T00:43:40.859461+0000","last_active":"2024-05-09T00:45:23.937399+0000","last_peered":"2024-05-09T00:45:23.937399+0000","last_clean":"2024-05-09T00:45:23.937399+0000","last_became_active":"2024-05-09T00:43:40.857814+0000","last_became_peered":"2024-05-09T00:43:40.857814+0000","last_unstale":"2024-05-09T00:45:23.937399+0000","last_undegraded":"2024-05-09T00:45:23.937399+0000","last_fullsized":"2024-05-09T00:45:23.937399+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:56:06.279762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.080858+0000","last_change":"2024-05-09T00:43:43.878120+0000","last_active":"2024-05-09T00:45:24.080858+0000","last_peered":"2024-05-09T00:45:24.080858+0000","last_clean":"2024-05-09T00:45:24.080858+0000","last_became_active":"2024-05-09T00:43:43.877782+0000","last_became_peered":"2024-05-09T00:43:43.877782+0000","last_unstale":"2024-05-09T00:45:24.080858+0000","last_undegraded":"2024-05-09T00:45:24.080858+0000","last_fullsized":"2024-05-09T00:45:24.080858+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:19:48.558992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'6","reported_seq":144,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:46.164013+0000","last_change":"2024-05-09T00:43:41.119347+0000","last_active":"2024-05-09T00:45:46.164013+0000","last_peered":"2024-05-09T00:45:46.164013+0000","last_clean":"2024-05-09T00:45:46.164013+0000","last_became_active":"2024-05-09T00:43:41.118847+0000","last_became_peered":"2024-05-09T00:43:41.118847+0000","last_unstale":"2024-05-09T00:45:46.164013+0000","last_undegraded":"2024-05-09T00:45:46.164013+0000","last_fullsized":"2024-05-09T00:45:46.164013+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:07:03.977756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.081339+0000","last_change":"2024-05-09T00:43:43.875203+0000","last_active":"2024-05-09T00:45:24.081339+0000","last_peered":"2024-05-09T00:45:24.081339+0000","last_clean":"2024-05-09T00:45:24.081339+0000","last_became_active":"2024-05-09T00:43:43.874882+0000","last_became_peered":"2024-05-09T00:43:43.874882+0000","last_unstale":"2024-05-09T00:45:24.081339+0000","last_undegraded":"2024-05-09T00:45:24.081339+0000","last_fullsized":"2024-05-09T00:45:24.081339+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:54:27.845086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'3","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.938122+0000","last_change":"2024-05-09T00:43:40.857955+0000","last_active":"2024-05-09T00:45:23.938122+0000","last_peered":"2024-05-09T00:45:23.938122+0000","last_clean":"2024-05-09T00:45:23.938122+0000","last_became_active":"2024-05-09T00:43:40.857596+0000","last_became_peered":"2024-05-09T00:43:40.857596+0000","last_unstale":"2024-05-09T00:45:23.938122+0000","last_undegraded":"2024-05-09T00:45:23.938122+0000","last_fullsized":"2024-05-09T00:45:23.938122+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:13:56.676929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"78'1","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.080858+0000","last_change":"2024-05-09T00:43:43.882487+0000","last_active":"2024-05-09T00:45:24.080858+0000","last_peered":"2024-05-09T00:45:24.080858+0000","last_clean":"2024-05-09T00:45:24.080858+0000","last_became_active":"2024-05-09T00:43:43.882176+0000","last_became_peered":"2024-05-09T00:43:43.882176+0000","last_unstale":"2024-05-09T00:45:24.080858+0000","last_undegraded":"2024-05-09T00:45:24.080858+0000","last_fullsized":"2024-05-09T00:45:24.080858+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:48:42.354433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'5","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.934689+0000","last_change":"2024-05-09T00:43:40.857215+0000","last_active":"2024-05-09T00:45:23.934689+0000","last_peered":"2024-05-09T00:45:23.934689+0000","last_clean":"2024-05-09T00:45:23.934689+0000","last_became_active":"2024-05-09T00:43:40.856858+0000","last_became_peered":"2024-05-09T00:43:40.856858+0000","last_unstale":"2024-05-09T00:45:23.934689+0000","last_undegraded":"2024-05-09T00:45:23.934689+0000","last_fullsized":"2024-05-09T00:45:23.934689+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:40:47.943482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.081373+0000","last_change":"2024-05-09T00:43:43.881192+0000","last_active":"2024-05-09T00:45:24.081373+0000","last_peered":"2024-05-09T00:45:24.081373+0000","last_clean":"2024-05-09T00:45:24.081373+0000","last_became_active":"2024-05-09T00:43:43.880781+0000","last_became_peered":"2024-05-09T00:43:43.880781+0000","last_unstale":"2024-05-09T00:45:24.081373+0000","last_undegraded":"2024-05-09T00:45:24.081373+0000","last_fullsized":"2024-05-09T00:45:24.081373+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:18:31.577354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.934461+0000","last_change":"2024-05-09T00:43:43.876869+0000","last_active":"2024-05-09T00:45:23.934461+0000","last_peered":"2024-05-09T00:45:23.934461+0000","last_clean":"2024-05-09T00:45:23.934461+0000","last_became_active":"2024-05-09T00:43:43.876439+0000","last_became_peered":"2024-05-09T00:43:43.876439+0000","last_unstale":"2024-05-09T00:45:23.934461+0000","last_undegraded":"2024-05-09T00:45:23.934461+0000","last_fullsized":"2024-05-09T00:45:23.934461+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:39:06.776267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.116064+0000","last_change":"2024-05-09T00:43:43.884677+0000","last_active":"2024-05-09T00:45:24.116064+0000","last_peered":"2024-05-09T00:45:24.116064+0000","last_clean":"2024-05-09T00:45:24.116064+0000","last_became_active":"2024-05-09T00:43:43.884238+0000","last_became_peered":"2024-05-09T00:43:43.884238+0000","last_unstale":"2024-05-09T00:45:24.116064+0000","last_undegraded":"2024-05-09T00:45:24.116064+0000","last_fullsized":"2024-05-09T00:45:24.116064+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:18:13.787584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.116035+0000","last_change":"2024-05-09T00:43:43.876638+0000","last_active":"2024-05-09T00:45:24.116035+0000","last_peered":"2024-05-09T00:45:24.116035+0000","last_clean":"2024-05-09T00:45:24.116035+0000","last_became_active":"2024-05-09T00:43:43.876315+0000","last_became_peered":"2024-05-09T00:43:43.876315+0000","last_unstale":"2024-05-09T00:45:24.116035+0000","last_undegraded":"2024-05-09T00:45:24.116035+0000","last_fullsized":"2024-05-09T00:45:24.116035+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:22:41.618613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.940191+0000","last_change":"2024-05-09T00:43:43.869798+0000","last_active":"2024-05-09T00:45:23.940191+0000","last_peered":"2024-05-09T00:45:23.940191+0000","last_clean":"2024-05-09T00:45:23.940191+0000","last_became_active":"2024-05-09T00:43:43.869444+0000","last_became_peered":"2024-05-09T00:43:43.869444+0000","last_unstale":"2024-05-09T00:45:23.940191+0000","last_undegraded":"2024-05-09T00:45:23.940191+0000","last_fullsized":"2024-05-09T00:45:23.940191+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:15:53.267441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.934600+0000","last_change":"2024-05-09T00:43:43.873215+0000","last_active":"2024-05-09T00:45:23.934600+0000","last_peered":"2024-05-09T00:45:23.934600+0000","last_clean":"2024-05-09T00:45:23.934600+0000","last_became_active":"2024-05-09T00:43:43.872763+0000","last_became_peered":"2024-05-09T00:43:43.872763+0000","last_unstale":"2024-05-09T00:45:23.934600+0000","last_undegraded":"2024-05-09T00:45:23.934600+0000","last_fullsized":"2024-05-09T00:45:23.934600+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:08.997387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.934702+0000","last_change":"2024-05-09T00:43:43.876514+0000","last_active":"2024-05-09T00:45:23.934702+0000","last_peered":"2024-05-09T00:45:23.934702+0000","last_clean":"2024-05-09T00:45:23.934702+0000","last_became_active":"2024-05-09T00:43:43.876147+0000","last_became_peered":"2024-05-09T00:43:43.876147+0000","last_unstale":"2024-05-09T00:45:23.934702+0000","last_undegraded":"2024-05-09T00:45:23.934702+0000","last_fullsized":"2024-05-09T00:45:23.934702+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:58:34.745819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.116418+0000","last_change":"2024-05-09T00:43:43.871869+0000","last_active":"2024-05-09T00:45:24.116418+0000","last_peered":"2024-05-09T00:45:24.116418+0000","last_clean":"2024-05-09T00:45:24.116418+0000","last_became_active":"2024-05-09T00:43:43.871447+0000","last_became_peered":"2024-05-09T00:43:43.871447+0000","last_unstale":"2024-05-09T00:45:24.116418+0000","last_undegraded":"2024-05-09T00:45:24.116418+0000","last_fullsized":"2024-05-09T00:45:24.116418+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:11:42.128931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.115849+0000","last_change":"2024-05-09T00:43:43.871291+0000","last_active":"2024-05-09T00:45:24.115849+0000","last_peered":"2024-05-09T00:45:24.115849+0000","last_clean":"2024-05-09T00:45:24.115849+0000","last_became_active":"2024-05-09T00:43:43.870899+0000","last_became_peered":"2024-05-09T00:43:43.870899+0000","last_unstale":"2024-05-09T00:45:24.115849+0000","last_undegraded":"2024-05-09T00:45:24.115849+0000","last_fullsized":"2024-05-09T00:45:24.115849+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:28:53.013543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.937586+0000","last_change":"2024-05-09T00:43:43.884791+0000","last_active":"2024-05-09T00:45:23.937586+0000","last_peered":"2024-05-09T00:45:23.937586+0000","last_clean":"2024-05-09T00:45:23.937586+0000","last_became_active":"2024-05-09T00:43:43.877333+0000","last_became_peered":"2024-05-09T00:43:43.877333+0000","last_unstale":"2024-05-09T00:45:23.937586+0000","last_undegraded":"2024-05-09T00:45:23.937586+0000","last_fullsized":"2024-05-09T00:45:23.937586+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:37:07.535388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.115386+0000","last_change":"2024-05-09T00:43:43.863773+0000","last_active":"2024-05-09T00:45:24.115386+0000","last_peered":"2024-05-09T00:45:24.115386+0000","last_clean":"2024-05-09T00:45:24.115386+0000","last_became_active":"2024-05-09T00:43:43.863436+0000","last_became_peered":"2024-05-09T00:43:43.863436+0000","last_unstale":"2024-05-09T00:45:24.115386+0000","last_undegraded":"2024-05-09T00:45:24.115386+0000","last_fullsized":"2024-05-09T00:45:24.115386+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:59:57.100234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.938606+0000","last_change":"2024-05-09T00:43:43.873461+0000","last_active":"2024-05-09T00:45:23.938606+0000","last_peered":"2024-05-09T00:45:23.938606+0000","last_clean":"2024-05-09T00:45:23.938606+0000","last_became_active":"2024-05-09T00:43:43.873139+0000","last_became_peered":"2024-05-09T00:43:43.873139+0000","last_unstale":"2024-05-09T00:45:23.938606+0000","last_undegraded":"2024-05-09T00:45:23.938606+0000","last_fullsized":"2024-05-09T00:45:23.938606+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:24:57.865569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.934304+0000","last_change":"2024-05-09T00:43:43.874401+0000","last_active":"2024-05-09T00:45:23.934304+0000","last_peered":"2024-05-09T00:45:23.934304+0000","last_clean":"2024-05-09T00:45:23.934304+0000","last_became_active":"2024-05-09T00:43:43.873765+0000","last_became_peered":"2024-05-09T00:43:43.873765+0000","last_unstale":"2024-05-09T00:45:23.934304+0000","last_undegraded":"2024-05-09T00:45:23.934304+0000","last_fullsized":"2024-05-09T00:45:23.934304+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:23:59.509415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.934710+0000","last_change":"2024-05-09T00:43:43.875219+0000","last_active":"2024-05-09T00:45:23.934710+0000","last_peered":"2024-05-09T00:45:23.934710+0000","last_clean":"2024-05-09T00:45:23.934710+0000","last_became_active":"2024-05-09T00:43:43.874824+0000","last_became_peered":"2024-05-09T00:43:43.874824+0000","last_unstale":"2024-05-09T00:45:23.934710+0000","last_undegraded":"2024-05-09T00:45:23.934710+0000","last_fullsized":"2024-05-09T00:45:23.934710+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:39:41.929521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":262144,"data_stored":2951840,"data_compressed":179218,"data_compressed_allocated":225280,"data_compressed_original":2932736,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69930,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1057,"num_read_kb":592,"num_write":247,"num_write_kb":139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":53265,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":540672,"data_stored":209790,"data_compressed":8493,"data_compressed_allocated":12288,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},"log_size":212,"ondisk_log_size":212,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710760,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":443805,"data_compressed":5799,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906990,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":1096587,"data_compressed":41967,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070516,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":921600,"data_stored":442762,"data_compressed":5799,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266744,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":913408,"data_stored":442558,"data_compressed":5799,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462974,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":1036049,"data_compressed":41967,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626499,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":905216,"data_stored":442479,"data_compressed":5799,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":970752,"data_stored":505727,"data_compressed":5799,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738465,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":444209,"data_compressed":5799,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411415,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":444394,"data_compressed":5799,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182482,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36244,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708908,"statfs":{"total":95995035648,"available":95957921792,"internally_reserved":0,"allocated":921600,"data_stored":442304,"data_compressed":5799,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986253,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":942080,"data_stored":505066,"data_compressed":14292,"data_compressed_allocated":20480,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822728,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":1035336,"data_compressed":41967,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34546,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36168,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36168,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36168,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36168,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":64087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":63426,"data_compressed":8493,"data_compressed_allocated":12288,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":64462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T00:45:49.310 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-09T00:45:49.310 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-09T00:45:49.310 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-09T00:45:49.310 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph health --format=json 2024-05-09T00:45:49.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:45:49 smithi005 ceph-mon[30365]: pgmap v355: 97 pgs: 97 active+clean; 645 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T00:45:49.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:45:49 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:45:49.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:45:49 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:45:49.582 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:45:49.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:45:49 smithi131 ceph-mon[34547]: pgmap v355: 97 pgs: 97 active+clean; 645 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T00:45:49.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:45:49 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:45:49.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:45:49 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:45:49.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:45:49 smithi157 ceph-mon[34294]: pgmap v355: 97 pgs: 97 active+clean; 645 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T00:45:49.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:45:49 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:45:49.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:45:49 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:45:50.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:45:50 smithi005 ceph-mon[30365]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T00:45:50.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:45:50 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:45:50.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:45:50 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:45:50.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:45:50 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:45:50.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:45:50 smithi131 ceph-mon[34547]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T00:45:50.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:45:50 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:45:50.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:45:50 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:45:50.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:45:50 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:45:50.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:45:50 smithi157 ceph-mon[34294]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T00:45:50.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:45:50 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:45:50.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:45:50 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:45:50.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:45:50 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:45:51.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:51.336+0000 7f14fc876700 1 -- 172.21.15.5:0/682916200 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14f810a3e0 msgr2=0x7f14f810a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:51.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:51.336+0000 7f14fc876700 1 --2- 172.21.15.5:0/682916200 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14f810a3e0 0x7f14f810a7f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f14e800b0a0 tx=0x7f14e800b3b0 comp rx=0 tx=0).stop 2024-05-09T00:45:51.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:51.337+0000 7f14fc876700 1 -- 172.21.15.5:0/682916200 shutdown_connections 2024-05-09T00:45:51.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:51.337+0000 7f14fc876700 1 --2- 172.21.15.5:0/682916200 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14f8103130 0x7f14f8111120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:51.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:51.337+0000 7f14fc876700 1 --2- 172.21.15.5:0/682916200 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14f810a3e0 0x7f14f810a7f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:51.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:51.337+0000 7f14fc876700 1 --2- 172.21.15.5:0/682916200 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14f81027e0 0x7f14f8102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:51.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:51.337+0000 7f14fc876700 1 -- 172.21.15.5:0/682916200 >> 172.21.15.5:0/682916200 conn(0x7f14f80fe6f0 msgr2=0x7f14f8100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:45:51.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:51.337+0000 7f14fc876700 1 -- 172.21.15.5:0/682916200 shutdown_connections 2024-05-09T00:45:51.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:51.337+0000 7f14fc876700 1 -- 172.21.15.5:0/682916200 wait complete. 2024-05-09T00:45:51.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:51.338+0000 7f14fc876700 1 Processor -- start 2024-05-09T00:45:51.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:51.338+0000 7f14fc876700 1 -- start start 2024-05-09T00:45:51.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:51.338+0000 7f14fc876700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14f81027e0 0x7f14f819ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:51.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:51.338+0000 7f14fc876700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14f8103130 0x7f14f819b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:51.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:51.338+0000 7f14fc876700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14f810a3e0 0x7f14f819f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:51.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:51.338+0000 7f14fc876700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f14f8114480 con 0x7f14f81027e0 2024-05-09T00:45:51.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:51.338+0000 7f14fc876700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f14f8114300 con 0x7f14f810a3e0 2024-05-09T00:45:51.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:51.338+0000 7f14fc876700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f14f8114600 con 0x7f14f8103130 2024-05-09T00:45:51.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:51.338+0000 7f14f659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14f81027e0 0x7f14f819ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:51.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:51.338+0000 7f14f659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14f81027e0 0x7f14f819ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39712/0 (socket says 172.21.15.5:39712) 2024-05-09T00:45:51.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:51.338+0000 7f14f659c700 1 -- 172.21.15.5:0/713055527 learned_addr learned my addr 172.21.15.5:0/713055527 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:45:51.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:51.338+0000 7f14f5d9b700 1 --2- 172.21.15.5:0/713055527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14f8103130 0x7f14f819b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:51.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:51.338+0000 7f14f659c700 1 -- 172.21.15.5:0/713055527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14f8103130 msgr2=0x7f14f819b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:51.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:51.338+0000 7f14f659c700 1 --2- 172.21.15.5:0/713055527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14f8103130 0x7f14f819b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:51.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:51.338+0000 7f14f659c700 1 -- 172.21.15.5:0/713055527 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14f810a3e0 msgr2=0x7f14f819f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:51.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:51.338+0000 7f14f659c700 1 --2- 172.21.15.5:0/713055527 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14f810a3e0 0x7f14f819f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:51.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:51.339+0000 7f14f659c700 1 -- 172.21.15.5:0/713055527 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14f819fce0 con 0x7f14f81027e0 2024-05-09T00:45:51.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:51.339+0000 7f14f5d9b700 1 --2- 172.21.15.5:0/713055527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14f8103130 0x7f14f819b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:45:51.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:51.339+0000 7f14f659c700 1 --2- 172.21.15.5:0/713055527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14f81027e0 0x7f14f819ae10 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f14e800b070 tx=0x7f14e8004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:51.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:51.339+0000 7f14ef7fe700 1 -- 172.21.15.5:0/713055527 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e8021070 con 0x7f14f81027e0 2024-05-09T00:45:51.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:51.339+0000 7f14ef7fe700 1 -- 172.21.15.5:0/713055527 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14e8007c80 con 0x7f14f81027e0 2024-05-09T00:45:51.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:51.339+0000 7f14fc876700 1 -- 172.21.15.5:0/713055527 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14f819ff70 con 0x7f14f81027e0 2024-05-09T00:45:51.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:51.339+0000 7f14ef7fe700 1 -- 172.21.15.5:0/713055527 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e801d450 con 0x7f14f81027e0 2024-05-09T00:45:51.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:51.339+0000 7f14fc876700 1 -- 172.21.15.5:0/713055527 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14f81a78f0 con 0x7f14f81027e0 2024-05-09T00:45:51.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:51.340+0000 7f14fc876700 1 -- 172.21.15.5:0/713055527 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14f810c850 con 0x7f14f81027e0 2024-05-09T00:45:51.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:51.342+0000 7f14ef7fe700 1 -- 172.21.15.5:0/713055527 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f14e8004000 con 0x7f14f81027e0 2024-05-09T00:45:51.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:51.342+0000 7f14ef7fe700 1 --2- 172.21.15.5:0/713055527 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f14e40b2910 0x7f14e40b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:51.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:51.342+0000 7f14f5d9b700 1 --2- 172.21.15.5:0/713055527 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f14e40b2910 0x7f14e40b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:51.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:51.343+0000 7f14ef7fe700 1 -- 172.21.15.5:0/713055527 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f14e800f020 con 0x7f14f81027e0 2024-05-09T00:45:51.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:51.343+0000 7f14f5d9b700 1 --2- 172.21.15.5:0/713055527 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f14e40b2910 0x7f14e40b4e20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f14e0005fd0 tx=0x7f14e0005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:51.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:51.346+0000 7f14ef7fe700 1 -- 172.21.15.5:0/713055527 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f14e802a020 con 0x7f14f81027e0 2024-05-09T00:45:51.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:45:51 smithi005 ceph-mon[30365]: pgmap v356: 97 pgs: 97 active+clean; 645 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-09T00:45:51.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:45:51 smithi131 ceph-mon[34547]: pgmap v356: 97 pgs: 97 active+clean; 645 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-09T00:45:51.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:51.611+0000 7f14fc876700 1 -- 172.21.15.5:0/713055527 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f14f8105b00 con 0x7f14f81027e0 2024-05-09T00:45:51.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:51.611+0000 7f14ef7fe700 1 -- 172.21.15.5:0/713055527 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f14e80ab9e0 con 0x7f14f81027e0 2024-05-09T00:45:51.612 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:45:51.612 INFO:teuthology.orchestra.run.smithi005.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-09T00:45:51.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:51.613+0000 7f14fc876700 1 -- 172.21.15.5:0/713055527 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f14e40b2910 msgr2=0x7f14e40b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:51.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:51.613+0000 7f14fc876700 1 --2- 172.21.15.5:0/713055527 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f14e40b2910 0x7f14e40b4e20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f14e0005fd0 tx=0x7f14e0005dc0 comp rx=0 tx=0).stop 2024-05-09T00:45:51.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:51.613+0000 7f14fc876700 1 -- 172.21.15.5:0/713055527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14f81027e0 msgr2=0x7f14f819ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:51.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:51.613+0000 7f14fc876700 1 --2- 172.21.15.5:0/713055527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14f81027e0 0x7f14f819ae10 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f14e800b070 tx=0x7f14e8004db0 comp rx=0 tx=0).stop 2024-05-09T00:45:51.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:51.614+0000 7f14fc876700 1 -- 172.21.15.5:0/713055527 shutdown_connections 2024-05-09T00:45:51.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:51.614+0000 7f14fc876700 1 --2- 172.21.15.5:0/713055527 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f14e40b2910 0x7f14e40b4e20 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:51.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:51.614+0000 7f14fc876700 1 --2- 172.21.15.5:0/713055527 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14f810a3e0 0x7f14f819f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:51.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:51.614+0000 7f14fc876700 1 --2- 172.21.15.5:0/713055527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14f81027e0 0x7f14f819ae10 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:51.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:51.614+0000 7f14fc876700 1 --2- 172.21.15.5:0/713055527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14f8103130 0x7f14f819b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:51.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:51.614+0000 7f14fc876700 1 -- 172.21.15.5:0/713055527 >> 172.21.15.5:0/713055527 conn(0x7f14f80fe6f0 msgr2=0x7f14f8100b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:45:51.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:51.614+0000 7f14fc876700 1 -- 172.21.15.5:0/713055527 shutdown_connections 2024-05-09T00:45:51.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:51.614+0000 7f14fc876700 1 -- 172.21.15.5:0/713055527 wait complete. 2024-05-09T00:45:51.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:45:51 smithi157 ceph-mon[34294]: pgmap v356: 97 pgs: 97 active+clean; 645 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-09T00:45:52.302 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-09T00:45:52.302 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T00:45:52.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:45:52 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/713055527' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-09T00:45:52.556 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:45:52.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:45:52 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/713055527' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-09T00:45:52.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:45:52 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/713055527' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-09T00:45:53.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:45:53 smithi005 ceph-mon[30365]: pgmap v357: 97 pgs: 97 active+clean; 645 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T00:45:53.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:45:53 smithi005 ceph-mon[30365]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-05-09T00:45:53.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:45:53 smithi131 ceph-mon[34547]: pgmap v357: 97 pgs: 97 active+clean; 645 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T00:45:53.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:45:53 smithi131 ceph-mon[34547]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-05-09T00:45:53.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:45:53 smithi157 ceph-mon[34294]: pgmap v357: 97 pgs: 97 active+clean; 645 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T00:45:53.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:45:53 smithi157 ceph-mon[34294]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-05-09T00:45:54.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:54.250+0000 7fc7e4f63700 1 -- 172.21.15.5:0/1458522682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7e010e970 msgr2=0x7fc7e0110e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:54.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:54.250+0000 7fc7e4f63700 1 --2- 172.21.15.5:0/1458522682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7e010e970 0x7fc7e0110e10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc7d800b0a0 tx=0x7fc7d800b3b0 comp rx=0 tx=0).stop 2024-05-09T00:45:54.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:54.250+0000 7fc7e4f63700 1 -- 172.21.15.5:0/1458522682 shutdown_connections 2024-05-09T00:45:54.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:54.250+0000 7fc7e4f63700 1 --2- 172.21.15.5:0/1458522682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7e010e970 0x7fc7e0110e10 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:54.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:54.250+0000 7fc7e4f63700 1 --2- 172.21.15.5:0/1458522682 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7e0105590 0x7fc7e01059a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:54.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:54.250+0000 7fc7e4f63700 1 --2- 172.21.15.5:0/1458522682 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7e0105ee0 0x7fc7e010e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:54.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:54.250+0000 7fc7e4f63700 1 -- 172.21.15.5:0/1458522682 >> 172.21.15.5:0/1458522682 conn(0x7fc7e0078e00 msgr2=0x7fc7e0079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:45:54.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:54.250+0000 7fc7e4f63700 1 -- 172.21.15.5:0/1458522682 shutdown_connections 2024-05-09T00:45:54.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:54.250+0000 7fc7e4f63700 1 -- 172.21.15.5:0/1458522682 wait complete. 2024-05-09T00:45:54.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:54.251+0000 7fc7e4f63700 1 Processor -- start 2024-05-09T00:45:54.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:54.251+0000 7fc7e4f63700 1 -- start start 2024-05-09T00:45:54.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:54.252+0000 7fc7e4f63700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7e0105590 0x7fc7e019ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:54.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:54.252+0000 7fc7e4f63700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7e0105ee0 0x7fc7e019b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:54.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:54.252+0000 7fc7e4f63700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7e010e970 0x7fc7e019f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:54.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:54.252+0000 7fc7e4f63700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e0114490 con 0x7fc7e0105590 2024-05-09T00:45:54.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:54.252+0000 7fc7e4f63700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e0114310 con 0x7fc7e010e970 2024-05-09T00:45:54.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:54.252+0000 7fc7e4f63700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e0114610 con 0x7fc7e0105ee0 2024-05-09T00:45:54.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:54.252+0000 7fc7de59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7e0105590 0x7fc7e019ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:54.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:54.252+0000 7fc7de59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7e0105590 0x7fc7e019ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59170/0 (socket says 172.21.15.5:59170) 2024-05-09T00:45:54.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:54.252+0000 7fc7de59c700 1 -- 172.21.15.5:0/1085168087 learned_addr learned my addr 172.21.15.5:0/1085168087 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:45:54.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:54.252+0000 7fc7ded9d700 1 --2- 172.21.15.5:0/1085168087 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7e010e970 0x7fc7e019f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:54.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:54.252+0000 7fc7ddd9b700 1 --2- 172.21.15.5:0/1085168087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7e0105ee0 0x7fc7e019b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:54.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:54.252+0000 7fc7de59c700 1 -- 172.21.15.5:0/1085168087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7e0105ee0 msgr2=0x7fc7e019b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:54.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:54.252+0000 7fc7de59c700 1 --2- 172.21.15.5:0/1085168087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7e0105ee0 0x7fc7e019b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:54.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:54.252+0000 7fc7de59c700 1 -- 172.21.15.5:0/1085168087 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7e010e970 msgr2=0x7fc7e019f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:54.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:54.252+0000 7fc7de59c700 1 --2- 172.21.15.5:0/1085168087 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7e010e970 0x7fc7e019f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:54.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:54.252+0000 7fc7de59c700 1 -- 172.21.15.5:0/1085168087 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7e019fc50 con 0x7fc7e0105590 2024-05-09T00:45:54.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:54.252+0000 7fc7ddd9b700 1 --2- 172.21.15.5:0/1085168087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7e0105ee0 0x7fc7e019b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:45:54.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:54.252+0000 7fc7de59c700 1 --2- 172.21.15.5:0/1085168087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7e0105590 0x7fc7e019ad80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc7d000d950 tx=0x7fc7d000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:54.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:54.253+0000 7fc7cf7fe700 1 -- 172.21.15.5:0/1085168087 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d0016070 con 0x7fc7e0105590 2024-05-09T00:45:54.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:54.253+0000 7fc7e4f63700 1 -- 172.21.15.5:0/1085168087 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7e019ff40 con 0x7fc7e0105590 2024-05-09T00:45:54.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:54.253+0000 7fc7cf7fe700 1 -- 172.21.15.5:0/1085168087 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7d0009df0 con 0x7fc7e0105590 2024-05-09T00:45:54.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:54.253+0000 7fc7cf7fe700 1 -- 172.21.15.5:0/1085168087 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d000f3c0 con 0x7fc7e0105590 2024-05-09T00:45:54.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:54.253+0000 7fc7e4f63700 1 -- 172.21.15.5:0/1085168087 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7e01a7860 con 0x7fc7e0105590 2024-05-09T00:45:54.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:54.254+0000 7fc7e4f63700 1 -- 172.21.15.5:0/1085168087 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7e00512c0 con 0x7fc7e0105590 2024-05-09T00:45:54.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:54.255+0000 7fc7cf7fe700 1 -- 172.21.15.5:0/1085168087 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fc7d0019070 con 0x7fc7e0105590 2024-05-09T00:45:54.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:54.256+0000 7fc7cf7fe700 1 --2- 172.21.15.5:0/1085168087 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fc7c80b2880 0x7fc7c80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:54.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:54.256+0000 7fc7cf7fe700 1 -- 172.21.15.5:0/1085168087 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fc7d00e1e30 con 0x7fc7e0105590 2024-05-09T00:45:54.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:54.256+0000 7fc7ddd9b700 1 --2- 172.21.15.5:0/1085168087 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fc7c80b2880 0x7fc7c80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:54.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:54.257+0000 7fc7ddd9b700 1 --2- 172.21.15.5:0/1085168087 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fc7c80b2880 0x7fc7c80b4d90 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc7d4005e10 tx=0x7fc7d4005d60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:54.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:54.260+0000 7fc7cf7fe700 1 -- 172.21.15.5:0/1085168087 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc7d00a9b20 con 0x7fc7e0105590 2024-05-09T00:45:54.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:54.524+0000 7fc7e4f63700 1 -- 172.21.15.5:0/1085168087 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc7e01a7a30 con 0x7fc7e0105590 2024-05-09T00:45:54.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:54.525+0000 7fc7cf7fe700 1 -- 172.21.15.5:0/1085168087 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11926 (secure 0 0 0) 0x7fc7d00a9420 con 0x7fc7e0105590 2024-05-09T00:45:54.526 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:45:54.526 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2685771887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2685771887},{"type":"v1","addr":"172.21.15.5:6835","nonce":2685771887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/254017535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":254017535},{"type":"v1","addr":"172.21.15.5:6837","nonce":254017535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T00:43:45.856066+0000","modified":"2024-05-09T00:44:06.033927+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24508,"mds_1":24499,"mds_2":24494,"mds_3":24493,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14598":{"gid":14598,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2109255228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2109255228},{"type":"v1","addr":"172.21.15.5:6839","nonce":2109255228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/3809966329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3809966329},{"type":"v1","addr":"172.21.15.5:6841","nonce":3809966329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6833/3272897742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3272897742},{"type":"v1","addr":"172.21.15.131:6833","nonce":3272897742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1283093898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1283093898},{"type":"v1","addr":"172.21.15.131:6835","nonce":1283093898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/2809525455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2809525455},{"type":"v1","addr":"172.21.15.157:6833","nonce":2809525455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.131:6837/1633006186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":1633006186},{"type":"v1","addr":"172.21.15.131:6837","nonce":1633006186}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.157:6835/832898273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":832898273},{"type":"v1","addr":"172.21.15.157:6835","nonce":832898273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.131:6839/2748669631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2748669631},{"type":"v1","addr":"172.21.15.131:6839","nonce":2748669631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.157:6837/744863009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":744863009},{"type":"v1","addr":"172.21.15.157:6837","nonce":744863009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.157:6839/2779860399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":2779860399},{"type":"v1","addr":"172.21.15.157:6839","nonce":2779860399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24467,24493,24499,24508,24494]},"id":1}]} 2024-05-09T00:45:54.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:54.527+0000 7fc7e4f63700 1 -- 172.21.15.5:0/1085168087 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fc7c80b2880 msgr2=0x7fc7c80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:54.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:54.527+0000 7fc7e4f63700 1 --2- 172.21.15.5:0/1085168087 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fc7c80b2880 0x7fc7c80b4d90 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc7d4005e10 tx=0x7fc7d4005d60 comp rx=0 tx=0).stop 2024-05-09T00:45:54.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:54.527+0000 7fc7e4f63700 1 -- 172.21.15.5:0/1085168087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7e0105590 msgr2=0x7fc7e019ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:54.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:54.527+0000 7fc7e4f63700 1 --2- 172.21.15.5:0/1085168087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7e0105590 0x7fc7e019ad80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc7d000d950 tx=0x7fc7d000dc60 comp rx=0 tx=0).stop 2024-05-09T00:45:54.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:54.527+0000 7fc7e4f63700 1 -- 172.21.15.5:0/1085168087 shutdown_connections 2024-05-09T00:45:54.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:54.527+0000 7fc7e4f63700 1 --2- 172.21.15.5:0/1085168087 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fc7c80b2880 0x7fc7c80b4d90 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:54.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:54.527+0000 7fc7e4f63700 1 --2- 172.21.15.5:0/1085168087 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7e010e970 0x7fc7e019f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:54.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:54.527+0000 7fc7e4f63700 1 --2- 172.21.15.5:0/1085168087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7e0105590 0x7fc7e019ad80 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:54.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:54.527+0000 7fc7e4f63700 1 --2- 172.21.15.5:0/1085168087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7e0105ee0 0x7fc7e019b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:54.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:54.527+0000 7fc7e4f63700 1 -- 172.21.15.5:0/1085168087 >> 172.21.15.5:0/1085168087 conn(0x7fc7e0078e00 msgr2=0x7fc7e00fffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:45:54.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:54.527+0000 7fc7e4f63700 1 -- 172.21.15.5:0/1085168087 shutdown_connections 2024-05-09T00:45:54.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:54.527+0000 7fc7e4f63700 1 -- 172.21.15.5:0/1085168087 wait complete. 2024-05-09T00:45:54.529 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 28 2024-05-09T00:45:55.217 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T00:45:55.471 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:45:55.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:45:55 smithi005 ceph-mon[30365]: pgmap v358: 97 pgs: 97 active+clean; 646 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 682 B/s wr, 12 op/s 2024-05-09T00:45:55.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:45:55 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/1085168087' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:45:55.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:45:55 smithi131 ceph-mon[34547]: pgmap v358: 97 pgs: 97 active+clean; 646 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 682 B/s wr, 12 op/s 2024-05-09T00:45:55.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:45:55 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/1085168087' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:45:55.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:45:55 smithi157 ceph-mon[34294]: pgmap v358: 97 pgs: 97 active+clean; 646 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 682 B/s wr, 12 op/s 2024-05-09T00:45:55.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:45:55 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/1085168087' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:45:57.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:57.208+0000 7f74e6ee7700 1 -- 172.21.15.5:0/3580186426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74e0109850 msgr2=0x7f74e01111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:57.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:57.208+0000 7f74e6ee7700 1 --2- 172.21.15.5:0/3580186426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74e0109850 0x7f74e01111b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f74d800b0d0 tx=0x7f74d800b3e0 comp rx=0 tx=0).stop 2024-05-09T00:45:57.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:57.208+0000 7f74e6ee7700 1 -- 172.21.15.5:0/3580186426 shutdown_connections 2024-05-09T00:45:57.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:57.208+0000 7f74e6ee7700 1 --2- 172.21.15.5:0/3580186426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74e0109850 0x7f74e01111b0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:57.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:57.208+0000 7f74e6ee7700 1 --2- 172.21.15.5:0/3580186426 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74e01045d0 0x7f74e01049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:57.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:57.208+0000 7f74e6ee7700 1 --2- 172.21.15.5:0/3580186426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74e0108f00 0x7f74e0109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:57.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:57.208+0000 7f74e6ee7700 1 -- 172.21.15.5:0/3580186426 >> 172.21.15.5:0/3580186426 conn(0x7f74e00ffea0 msgr2=0x7f74e01022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:45:57.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:57.208+0000 7f74e6ee7700 1 -- 172.21.15.5:0/3580186426 shutdown_connections 2024-05-09T00:45:57.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:57.208+0000 7f74e6ee7700 1 -- 172.21.15.5:0/3580186426 wait complete. 2024-05-09T00:45:57.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:57.209+0000 7f74e6ee7700 1 Processor -- start 2024-05-09T00:45:57.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:57.209+0000 7f74e6ee7700 1 -- start start 2024-05-09T00:45:57.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:57.209+0000 7f74e6ee7700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74e01045d0 0x7f74e019ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:57.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:57.209+0000 7f74e6ee7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74e0108f00 0x7f74e019b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:57.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:57.209+0000 7f74e6ee7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74e0109850 0x7f74e019f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:57.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:57.209+0000 7f74e6ee7700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f74e0114480 con 0x7f74e0109850 2024-05-09T00:45:57.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:57.209+0000 7f74e6ee7700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f74e0114300 con 0x7f74e0108f00 2024-05-09T00:45:57.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:57.209+0000 7f74e6ee7700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f74e0114600 con 0x7f74e01045d0 2024-05-09T00:45:57.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:57.209+0000 7f74e5484700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74e0109850 0x7f74e019f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:57.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:57.209+0000 7f74e5484700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74e0109850 0x7f74e019f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59198/0 (socket says 172.21.15.5:59198) 2024-05-09T00:45:57.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:57.209+0000 7f74e5484700 1 -- 172.21.15.5:0/1292644832 learned_addr learned my addr 172.21.15.5:0/1292644832 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:45:57.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:57.209+0000 7f74e4c83700 1 --2- 172.21.15.5:0/1292644832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74e01045d0 0x7f74e019ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:57.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:57.210+0000 7f74dffff700 1 --2- 172.21.15.5:0/1292644832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74e0108f00 0x7f74e019b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:57.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:57.210+0000 7f74e5484700 1 -- 172.21.15.5:0/1292644832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74e01045d0 msgr2=0x7f74e019ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:57.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:57.210+0000 7f74e5484700 1 --2- 172.21.15.5:0/1292644832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74e01045d0 0x7f74e019ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:57.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:57.210+0000 7f74e5484700 1 -- 172.21.15.5:0/1292644832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74e0108f00 msgr2=0x7f74e019b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:57.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:57.210+0000 7f74e5484700 1 --2- 172.21.15.5:0/1292644832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74e0108f00 0x7f74e019b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:57.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:57.210+0000 7f74e5484700 1 -- 172.21.15.5:0/1292644832 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74e019fc20 con 0x7f74e0109850 2024-05-09T00:45:57.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:57.210+0000 7f74dffff700 1 --2- 172.21.15.5:0/1292644832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74e0108f00 0x7f74e019b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:45:57.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:57.210+0000 7f74e5484700 1 --2- 172.21.15.5:0/1292644832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74e0109850 0x7f74e019f580 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f74d80095c0 tx=0x7f74d8007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:57.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:57.210+0000 7f74ddffb700 1 -- 172.21.15.5:0/1292644832 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d801f070 con 0x7f74e0109850 2024-05-09T00:45:57.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:57.210+0000 7f74ddffb700 1 -- 172.21.15.5:0/1292644832 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74d8003c00 con 0x7f74e0109850 2024-05-09T00:45:57.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:57.210+0000 7f74e6ee7700 1 -- 172.21.15.5:0/1292644832 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74e019feb0 con 0x7f74e0109850 2024-05-09T00:45:57.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:57.210+0000 7f74ddffb700 1 -- 172.21.15.5:0/1292644832 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d801b6a0 con 0x7f74e0109850 2024-05-09T00:45:57.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:57.210+0000 7f74e6ee7700 1 -- 172.21.15.5:0/1292644832 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74e01a7830 con 0x7f74e0109850 2024-05-09T00:45:57.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:57.211+0000 7f74e6ee7700 1 -- 172.21.15.5:0/1292644832 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74e00512c0 con 0x7f74e0109850 2024-05-09T00:45:57.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:57.213+0000 7f74ddffb700 1 -- 172.21.15.5:0/1292644832 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f74d8022070 con 0x7f74e0109850 2024-05-09T00:45:57.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:57.214+0000 7f74ddffb700 1 --2- 172.21.15.5:0/1292644832 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f74d00b2900 0x7f74d00b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:45:57.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:57.214+0000 7f74e4c83700 1 --2- 172.21.15.5:0/1292644832 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f74d00b2900 0x7f74d00b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:45:57.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:57.214+0000 7f74ddffb700 1 -- 172.21.15.5:0/1292644832 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f74d800f020 con 0x7f74e0109850 2024-05-09T00:45:57.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:57.214+0000 7f74e4c83700 1 --2- 172.21.15.5:0/1292644832 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f74d00b2900 0x7f74d00b4e10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f74d4005950 tx=0x7f74d400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:45:57.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:57.217+0000 7f74ddffb700 1 -- 172.21.15.5:0/1292644832 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f74d8029020 con 0x7f74e0109850 2024-05-09T00:45:57.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:57.478+0000 7f74e6ee7700 1 -- 172.21.15.5:0/1292644832 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f74e019c0b0 con 0x7f74e0109850 2024-05-09T00:45:57.481 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:45:57 smithi005 ceph-mon[30365]: pgmap v359: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 10 op/s 2024-05-09T00:45:57.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:57.479+0000 7f74ddffb700 1 -- 172.21.15.5:0/1292644832 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11926 (secure 0 0 0) 0x7f74d80ab5d0 con 0x7f74e0109850 2024-05-09T00:45:57.482 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:45:57.482 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2685771887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2685771887},{"type":"v1","addr":"172.21.15.5:6835","nonce":2685771887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/254017535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":254017535},{"type":"v1","addr":"172.21.15.5:6837","nonce":254017535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T00:43:45.856066+0000","modified":"2024-05-09T00:44:06.033927+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24508,"mds_1":24499,"mds_2":24494,"mds_3":24493,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14598":{"gid":14598,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2109255228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2109255228},{"type":"v1","addr":"172.21.15.5:6839","nonce":2109255228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/3809966329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3809966329},{"type":"v1","addr":"172.21.15.5:6841","nonce":3809966329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6833/3272897742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3272897742},{"type":"v1","addr":"172.21.15.131:6833","nonce":3272897742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1283093898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1283093898},{"type":"v1","addr":"172.21.15.131:6835","nonce":1283093898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/2809525455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2809525455},{"type":"v1","addr":"172.21.15.157:6833","nonce":2809525455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.131:6837/1633006186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":1633006186},{"type":"v1","addr":"172.21.15.131:6837","nonce":1633006186}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.157:6835/832898273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":832898273},{"type":"v1","addr":"172.21.15.157:6835","nonce":832898273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.131:6839/2748669631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2748669631},{"type":"v1","addr":"172.21.15.131:6839","nonce":2748669631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.157:6837/744863009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":744863009},{"type":"v1","addr":"172.21.15.157:6837","nonce":744863009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.157:6839/2779860399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":2779860399},{"type":"v1","addr":"172.21.15.157:6839","nonce":2779860399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24467,24493,24499,24508,24494]},"id":1}]} 2024-05-09T00:45:57.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:57.481+0000 7f74e6ee7700 1 -- 172.21.15.5:0/1292644832 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f74d00b2900 msgr2=0x7f74d00b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:57.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:57.481+0000 7f74e6ee7700 1 --2- 172.21.15.5:0/1292644832 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f74d00b2900 0x7f74d00b4e10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f74d4005950 tx=0x7f74d400a300 comp rx=0 tx=0).stop 2024-05-09T00:45:57.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:57.481+0000 7f74e6ee7700 1 -- 172.21.15.5:0/1292644832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74e0109850 msgr2=0x7f74e019f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:45:57.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:57.481+0000 7f74e6ee7700 1 --2- 172.21.15.5:0/1292644832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74e0109850 0x7f74e019f580 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f74d80095c0 tx=0x7f74d8007a20 comp rx=0 tx=0).stop 2024-05-09T00:45:57.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:57.482+0000 7f74e6ee7700 1 -- 172.21.15.5:0/1292644832 shutdown_connections 2024-05-09T00:45:57.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:57.482+0000 7f74e6ee7700 1 --2- 172.21.15.5:0/1292644832 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f74d00b2900 0x7f74d00b4e10 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:57.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:57.482+0000 7f74e6ee7700 1 --2- 172.21.15.5:0/1292644832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74e0109850 0x7f74e019f580 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:57.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:57.482+0000 7f74e6ee7700 1 --2- 172.21.15.5:0/1292644832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74e01045d0 0x7f74e019ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:57.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:57.482+0000 7f74e6ee7700 1 --2- 172.21.15.5:0/1292644832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74e0108f00 0x7f74e019b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:45:57.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:57.482+0000 7f74e6ee7700 1 -- 172.21.15.5:0/1292644832 >> 172.21.15.5:0/1292644832 conn(0x7f74e00ffea0 msgr2=0x7f74e0108670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:45:57.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:57.482+0000 7f74e6ee7700 1 -- 172.21.15.5:0/1292644832 shutdown_connections 2024-05-09T00:45:57.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:45:57.482+0000 7f74e6ee7700 1 -- 172.21.15.5:0/1292644832 wait complete. 2024-05-09T00:45:57.486 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 28 2024-05-09T00:45:57.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:45:57 smithi131 ceph-mon[34547]: pgmap v359: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 10 op/s 2024-05-09T00:45:57.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:45:57 smithi157 ceph-mon[34294]: pgmap v359: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 10 op/s 2024-05-09T00:45:58.138 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T00:45:58.397 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:45:58.520 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:45:58 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/1292644832' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:45:58.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:45:58 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/1292644832' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:45:58.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:45:58 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/1292644832' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:45:59.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:45:59 smithi005 ceph-mon[30365]: pgmap v360: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 10 op/s 2024-05-09T00:45:59.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:45:59 smithi131 ceph-mon[34547]: pgmap v360: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 10 op/s 2024-05-09T00:45:59.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:45:59 smithi157 ceph-mon[34294]: pgmap v360: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 10 op/s 2024-05-09T00:46:00.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:00.126+0000 7fee5d1b5700 1 -- 172.21.15.5:0/3880395705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee58108f00 msgr2=0x7fee58109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:00.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:00.126+0000 7fee5d1b5700 1 --2- 172.21.15.5:0/3880395705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee58108f00 0x7fee58109310 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fee4c0099c0 tx=0x7fee4c009cd0 comp rx=0 tx=0).stop 2024-05-09T00:46:00.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:00.127+0000 7fee5d1b5700 1 -- 172.21.15.5:0/3880395705 shutdown_connections 2024-05-09T00:46:00.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:00.127+0000 7fee5d1b5700 1 --2- 172.21.15.5:0/3880395705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee58109850 0x7fee581111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:00.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:00.127+0000 7fee5d1b5700 1 --2- 172.21.15.5:0/3880395705 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee58102ae0 0x7fee58102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:00.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:00.127+0000 7fee5d1b5700 1 --2- 172.21.15.5:0/3880395705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee58108f00 0x7fee58109310 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:00.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:00.127+0000 7fee5d1b5700 1 -- 172.21.15.5:0/3880395705 >> 172.21.15.5:0/3880395705 conn(0x7fee580fe540 msgr2=0x7fee58100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:00.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:00.127+0000 7fee5d1b5700 1 -- 172.21.15.5:0/3880395705 shutdown_connections 2024-05-09T00:46:00.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:00.127+0000 7fee5d1b5700 1 -- 172.21.15.5:0/3880395705 wait complete. 2024-05-09T00:46:00.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:00.128+0000 7fee5d1b5700 1 Processor -- start 2024-05-09T00:46:00.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:00.128+0000 7fee5d1b5700 1 -- start start 2024-05-09T00:46:00.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:00.128+0000 7fee5d1b5700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee58102ae0 0x7fee5819afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:00.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:00.128+0000 7fee5d1b5700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee58108f00 0x7fee5819b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:00.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:00.128+0000 7fee5d1b5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee58109850 0x7fee5819f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:00.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:00.128+0000 7fee5d1b5700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fee581144d0 con 0x7fee58109850 2024-05-09T00:46:00.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:00.128+0000 7fee5d1b5700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fee58114350 con 0x7fee58102ae0 2024-05-09T00:46:00.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:00.128+0000 7fee5d1b5700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fee58114650 con 0x7fee58108f00 2024-05-09T00:46:00.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:00.128+0000 7fee5759e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee58109850 0x7fee5819f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:00.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:00.128+0000 7fee5759e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee58109850 0x7fee5819f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59216/0 (socket says 172.21.15.5:59216) 2024-05-09T00:46:00.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:00.128+0000 7fee5759e700 1 -- 172.21.15.5:0/1784694113 learned_addr learned my addr 172.21.15.5:0/1784694113 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:46:00.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:00.128+0000 7fee56d9d700 1 --2- 172.21.15.5:0/1784694113 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee58102ae0 0x7fee5819afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:00.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:00.128+0000 7fee5659c700 1 --2- 172.21.15.5:0/1784694113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee58108f00 0x7fee5819b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:00.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:00.129+0000 7fee5759e700 1 -- 172.21.15.5:0/1784694113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee58108f00 msgr2=0x7fee5819b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:00.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:00.129+0000 7fee5759e700 1 --2- 172.21.15.5:0/1784694113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee58108f00 0x7fee5819b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:00.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:00.129+0000 7fee5759e700 1 -- 172.21.15.5:0/1784694113 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee58102ae0 msgr2=0x7fee5819afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:00.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:00.129+0000 7fee5759e700 1 --2- 172.21.15.5:0/1784694113 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee58102ae0 0x7fee5819afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:00.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:00.129+0000 7fee5759e700 1 -- 172.21.15.5:0/1784694113 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee5819fe20 con 0x7fee58109850 2024-05-09T00:46:00.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:00.129+0000 7fee5759e700 1 --2- 172.21.15.5:0/1784694113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee58109850 0x7fee5819f780 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fee5000ada0 tx=0x7fee5000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:00.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:00.129+0000 7fee3ffff700 1 -- 172.21.15.5:0/1784694113 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee50017020 con 0x7fee58109850 2024-05-09T00:46:00.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:00.129+0000 7fee5d1b5700 1 -- 172.21.15.5:0/1784694113 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee581a0110 con 0x7fee58109850 2024-05-09T00:46:00.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:00.129+0000 7fee3ffff700 1 -- 172.21.15.5:0/1784694113 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee5000cd80 con 0x7fee58109850 2024-05-09T00:46:00.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:00.129+0000 7fee3ffff700 1 -- 172.21.15.5:0/1784694113 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee50005260 con 0x7fee58109850 2024-05-09T00:46:00.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:00.129+0000 7fee5d1b5700 1 -- 172.21.15.5:0/1784694113 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee58103960 con 0x7fee58109850 2024-05-09T00:46:00.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:00.130+0000 7fee5d1b5700 1 -- 172.21.15.5:0/1784694113 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee580512c0 con 0x7fee58109850 2024-05-09T00:46:00.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:00.133+0000 7fee3ffff700 1 -- 172.21.15.5:0/1784694113 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fee5000a090 con 0x7fee58109850 2024-05-09T00:46:00.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:00.134+0000 7fee3ffff700 1 --2- 172.21.15.5:0/1784694113 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fee400b2880 0x7fee400b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:00.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:00.134+0000 7fee56d9d700 1 --2- 172.21.15.5:0/1784694113 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fee400b2880 0x7fee400b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:00.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:00.134+0000 7fee3ffff700 1 -- 172.21.15.5:0/1784694113 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fee500e1f70 con 0x7fee58109850 2024-05-09T00:46:00.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:00.134+0000 7fee56d9d700 1 --2- 172.21.15.5:0/1784694113 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fee400b2880 0x7fee400b4d90 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fee48009a10 tx=0x7fee48009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:00.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:00.137+0000 7fee3ffff700 1 -- 172.21.15.5:0/1784694113 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fee50021020 con 0x7fee58109850 2024-05-09T00:46:00.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:00.355+0000 7fee5d1b5700 1 -- 172.21.15.5:0/1784694113 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fee5819c2b0 con 0x7fee58109850 2024-05-09T00:46:00.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:00.356+0000 7fee3ffff700 1 -- 172.21.15.5:0/1784694113 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22745 (secure 0 0 0) 0x7fee500a9c90 con 0x7fee58109850 2024-05-09T00:46:00.358 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:46:00.358 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":82,"fsid":"d85814c6-0d9b-11ef-bc97-c7b262605968","created":"2024-05-09T00:35:25.623377+0000","modified":"2024-05-09T00:45:23.926493+0000","last_up_change":"2024-05-09T00:42:38.525335+0000","last_in_change":"2024-05-09T00:42:17.990219+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T00:38:47.844734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T00:43:39.452106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T00:43:42.708106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f8d484bb-2477-4377-ae68-5bce571333bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2963385132},{"type":"v1","addr":"172.21.15.5:6803","nonce":2963385132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2963385132},{"type":"v1","addr":"172.21.15.5:6805","nonce":2963385132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2963385132},{"type":"v1","addr":"172.21.15.5:6809","nonce":2963385132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2963385132},{"type":"v1","addr":"172.21.15.5:6807","nonce":2963385132}]},"public_addr":"172.21.15.5:6803/2963385132","cluster_addr":"172.21.15.5:6805/2963385132","heartbeat_back_addr":"172.21.15.5:6809/2963385132","heartbeat_front_addr":"172.21.15.5:6807/2963385132","state":["exists","up"]},{"osd":1,"uuid":"1c123676-7144-4a67-93a6-fd49e36a5c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3837683163},{"type":"v1","addr":"172.21.15.131:6801","nonce":3837683163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3837683163},{"type":"v1","addr":"172.21.15.131:6803","nonce":3837683163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3837683163},{"type":"v1","addr":"172.21.15.131:6807","nonce":3837683163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3837683163},{"type":"v1","addr":"172.21.15.131:6805","nonce":3837683163}]},"public_addr":"172.21.15.131:6801/3837683163","cluster_addr":"172.21.15.131:6803/3837683163","heartbeat_back_addr":"172.21.15.131:6807/3837683163","heartbeat_front_addr":"172.21.15.131:6805/3837683163","state":["exists","up"]},{"osd":2,"uuid":"95ca51f0-b48b-485c-a5f7-b15c01e754b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":484818139},{"type":"v1","addr":"172.21.15.157:6801","nonce":484818139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":484818139},{"type":"v1","addr":"172.21.15.157:6803","nonce":484818139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":484818139},{"type":"v1","addr":"172.21.15.157:6807","nonce":484818139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":484818139},{"type":"v1","addr":"172.21.15.157:6805","nonce":484818139}]},"public_addr":"172.21.15.157:6801/484818139","cluster_addr":"172.21.15.157:6803/484818139","heartbeat_back_addr":"172.21.15.157:6807/484818139","heartbeat_front_addr":"172.21.15.157:6805/484818139","state":["exists","up"]},{"osd":3,"uuid":"ed7577fb-3676-4a08-b40d-5474eb88974f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":2357377277},{"type":"v1","addr":"172.21.15.5:6811","nonce":2357377277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":2357377277},{"type":"v1","addr":"172.21.15.5:6813","nonce":2357377277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":2357377277},{"type":"v1","addr":"172.21.15.5:6817","nonce":2357377277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":2357377277},{"type":"v1","addr":"172.21.15.5:6815","nonce":2357377277}]},"public_addr":"172.21.15.5:6811/2357377277","cluster_addr":"172.21.15.5:6813/2357377277","heartbeat_back_addr":"172.21.15.5:6817/2357377277","heartbeat_front_addr":"172.21.15.5:6815/2357377277","state":["exists","up"]},{"osd":4,"uuid":"fb5dcf79-a6f1-4b1a-9e3a-087fb5132472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":2915418378},{"type":"v1","addr":"172.21.15.131:6809","nonce":2915418378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":2915418378},{"type":"v1","addr":"172.21.15.131:6811","nonce":2915418378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":2915418378},{"type":"v1","addr":"172.21.15.131:6815","nonce":2915418378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":2915418378},{"type":"v1","addr":"172.21.15.131:6813","nonce":2915418378}]},"public_addr":"172.21.15.131:6809/2915418378","cluster_addr":"172.21.15.131:6811/2915418378","heartbeat_back_addr":"172.21.15.131:6815/2915418378","heartbeat_front_addr":"172.21.15.131:6813/2915418378","state":["exists","up"]},{"osd":5,"uuid":"f01ed5d3-8066-4d25-a113-280c83e75ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4188818663},{"type":"v1","addr":"172.21.15.157:6809","nonce":4188818663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":4188818663},{"type":"v1","addr":"172.21.15.157:6811","nonce":4188818663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":4188818663},{"type":"v1","addr":"172.21.15.157:6815","nonce":4188818663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":4188818663},{"type":"v1","addr":"172.21.15.157:6813","nonce":4188818663}]},"public_addr":"172.21.15.157:6809/4188818663","cluster_addr":"172.21.15.157:6811/4188818663","heartbeat_back_addr":"172.21.15.157:6815/4188818663","heartbeat_front_addr":"172.21.15.157:6813/4188818663","state":["exists","up"]},{"osd":6,"uuid":"932379a8-c97b-4acc-8665-81f7d9cbce65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2930397813},{"type":"v1","addr":"172.21.15.5:6819","nonce":2930397813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2930397813},{"type":"v1","addr":"172.21.15.5:6821","nonce":2930397813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2930397813},{"type":"v1","addr":"172.21.15.5:6825","nonce":2930397813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2930397813},{"type":"v1","addr":"172.21.15.5:6823","nonce":2930397813}]},"public_addr":"172.21.15.5:6819/2930397813","cluster_addr":"172.21.15.5:6821/2930397813","heartbeat_back_addr":"172.21.15.5:6825/2930397813","heartbeat_front_addr":"172.21.15.5:6823/2930397813","state":["exists","up"]},{"osd":7,"uuid":"3c0a0573-6465-4bf6-b70f-310d49857a6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":4125045219},{"type":"v1","addr":"172.21.15.131:6817","nonce":4125045219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":4125045219},{"type":"v1","addr":"172.21.15.131:6819","nonce":4125045219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":4125045219},{"type":"v1","addr":"172.21.15.131:6823","nonce":4125045219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":4125045219},{"type":"v1","addr":"172.21.15.131:6821","nonce":4125045219}]},"public_addr":"172.21.15.131:6817/4125045219","cluster_addr":"172.21.15.131:6819/4125045219","heartbeat_back_addr":"172.21.15.131:6823/4125045219","heartbeat_front_addr":"172.21.15.131:6821/4125045219","state":["exists","up"]},{"osd":8,"uuid":"c49f0a00-f5ae-40ed-8f38-7b953ba1c17c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2001626768},{"type":"v1","addr":"172.21.15.157:6817","nonce":2001626768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2001626768},{"type":"v1","addr":"172.21.15.157:6819","nonce":2001626768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2001626768},{"type":"v1","addr":"172.21.15.157:6823","nonce":2001626768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2001626768},{"type":"v1","addr":"172.21.15.157:6821","nonce":2001626768}]},"public_addr":"172.21.15.157:6817/2001626768","cluster_addr":"172.21.15.157:6819/2001626768","heartbeat_back_addr":"172.21.15.157:6823/2001626768","heartbeat_front_addr":"172.21.15.157:6821/2001626768","state":["exists","up"]},{"osd":9,"uuid":"1927dfe7-c1af-4a15-bb71-d58d97c8734a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":4287220015},{"type":"v1","addr":"172.21.15.5:6827","nonce":4287220015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":4287220015},{"type":"v1","addr":"172.21.15.5:6829","nonce":4287220015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":4287220015},{"type":"v1","addr":"172.21.15.5:6833","nonce":4287220015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":4287220015},{"type":"v1","addr":"172.21.15.5:6831","nonce":4287220015}]},"public_addr":"172.21.15.5:6827/4287220015","cluster_addr":"172.21.15.5:6829/4287220015","heartbeat_back_addr":"172.21.15.5:6833/4287220015","heartbeat_front_addr":"172.21.15.5:6831/4287220015","state":["exists","up"]},{"osd":10,"uuid":"fbb4f3e3-eb7d-4ecb-a30b-55b8910375ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1571783985},{"type":"v1","addr":"172.21.15.131:6825","nonce":1571783985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1571783985},{"type":"v1","addr":"172.21.15.131:6827","nonce":1571783985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1571783985},{"type":"v1","addr":"172.21.15.131:6831","nonce":1571783985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1571783985},{"type":"v1","addr":"172.21.15.131:6829","nonce":1571783985}]},"public_addr":"172.21.15.131:6825/1571783985","cluster_addr":"172.21.15.131:6827/1571783985","heartbeat_back_addr":"172.21.15.131:6831/1571783985","heartbeat_front_addr":"172.21.15.131:6829/1571783985","state":["exists","up"]},{"osd":11,"uuid":"b858453f-eda1-4e25-89c8-94b54c830938","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3261151188},{"type":"v1","addr":"172.21.15.157:6825","nonce":3261151188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":3261151188},{"type":"v1","addr":"172.21.15.157:6827","nonce":3261151188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":3261151188},{"type":"v1","addr":"172.21.15.157:6831","nonce":3261151188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":3261151188},{"type":"v1","addr":"172.21.15.157:6829","nonce":3261151188}]},"public_addr":"172.21.15.157:6825/3261151188","cluster_addr":"172.21.15.157:6827/3261151188","heartbeat_back_addr":"172.21.15.157:6831/3261151188","heartbeat_front_addr":"172.21.15.157:6829/3261151188","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:37:57.528483+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:38:21.477482+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:38:45.780266+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:39:10.938791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:39:35.734857+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:40:00.169040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:40:26.494635+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:40:51.881718+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:41:17.697112+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:41:44.826479+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:42:10.504768+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:42:36.605447+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6},{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":6}]},{"pgid":"3.19","mappings":[{"from":3,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":7}]},{"pgid":"3.35","mappings":[{"from":1,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/3405015786":"2024-05-10T00:36:21.742363+0000","172.21.15.5:0/2669115557":"2024-05-10T00:36:21.742363+0000","172.21.15.5:6801/287842266":"2024-05-10T00:36:21.742363+0000","172.21.15.5:6800/752513054":"2024-05-10T00:35:53.128955+0000","172.21.15.5:6800/287842266":"2024-05-10T00:36:21.742363+0000","172.21.15.5:6801/752513054":"2024-05-10T00:35:53.128955+0000","172.21.15.5:0/1954876662":"2024-05-10T00:35:53.128955+0000","172.21.15.5:0/2545175121":"2024-05-10T00:36:21.742363+0000","172.21.15.5:0/3754653663":"2024-05-10T00:35:53.128955+0000","172.21.15.5:0/651514359":"2024-05-10T00:35:53.128955+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T00:46:00.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:00.359+0000 7fee5d1b5700 1 -- 172.21.15.5:0/1784694113 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fee400b2880 msgr2=0x7fee400b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:00.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:00.359+0000 7fee5d1b5700 1 --2- 172.21.15.5:0/1784694113 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fee400b2880 0x7fee400b4d90 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fee48009a10 tx=0x7fee48009190 comp rx=0 tx=0).stop 2024-05-09T00:46:00.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:00.359+0000 7fee5d1b5700 1 -- 172.21.15.5:0/1784694113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee58109850 msgr2=0x7fee5819f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:00.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:00.359+0000 7fee5d1b5700 1 --2- 172.21.15.5:0/1784694113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee58109850 0x7fee5819f780 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fee5000ada0 tx=0x7fee5000c330 comp rx=0 tx=0).stop 2024-05-09T00:46:00.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:00.359+0000 7fee5d1b5700 1 -- 172.21.15.5:0/1784694113 shutdown_connections 2024-05-09T00:46:00.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:00.359+0000 7fee5d1b5700 1 --2- 172.21.15.5:0/1784694113 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fee400b2880 0x7fee400b4d90 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:00.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:00.359+0000 7fee5d1b5700 1 --2- 172.21.15.5:0/1784694113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee58109850 0x7fee5819f780 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:00.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:00.359+0000 7fee5d1b5700 1 --2- 172.21.15.5:0/1784694113 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee58102ae0 0x7fee5819afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:00.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:00.359+0000 7fee5d1b5700 1 --2- 172.21.15.5:0/1784694113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee58108f00 0x7fee5819b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:00.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:00.359+0000 7fee5d1b5700 1 -- 172.21.15.5:0/1784694113 >> 172.21.15.5:0/1784694113 conn(0x7fee580fe540 msgr2=0x7fee58107250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:00.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:00.359+0000 7fee5d1b5700 1 -- 172.21.15.5:0/1784694113 shutdown_connections 2024-05-09T00:46:00.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:00.359+0000 7fee5d1b5700 1 -- 172.21.15.5:0/1784694113 wait complete. 2024-05-09T00:46:01.037 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T00:46:01.289 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:46:01.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:46:01 smithi005 ceph-mon[30365]: pgmap v361: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 767 B/s wr, 11 op/s 2024-05-09T00:46:01.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:46:01 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/1784694113' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:46:01.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:46:01 smithi131 ceph-mon[34547]: pgmap v361: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 767 B/s wr, 11 op/s 2024-05-09T00:46:01.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:46:01 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/1784694113' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:46:01.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:46:01 smithi157 ceph-mon[34294]: pgmap v361: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 767 B/s wr, 11 op/s 2024-05-09T00:46:01.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:46:01 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/1784694113' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:46:03.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:03.012+0000 7f0467390700 1 -- 172.21.15.5:0/1634606116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0460108f00 msgr2=0x7f0460109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:03.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:03.012+0000 7f0467390700 1 --2- 172.21.15.5:0/1634606116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0460108f00 0x7f0460109310 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f04540099c0 tx=0x7f0454009cd0 comp rx=0 tx=0).stop 2024-05-09T00:46:03.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:03.012+0000 7f0467390700 1 -- 172.21.15.5:0/1634606116 shutdown_connections 2024-05-09T00:46:03.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:03.012+0000 7f0467390700 1 --2- 172.21.15.5:0/1634606116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0460109850 0x7f04601111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:03.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:03.012+0000 7f0467390700 1 --2- 172.21.15.5:0/1634606116 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04601045d0 0x7f04601049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:03.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:03.012+0000 7f0467390700 1 --2- 172.21.15.5:0/1634606116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0460108f00 0x7f0460109310 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:03.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:03.012+0000 7f0467390700 1 -- 172.21.15.5:0/1634606116 >> 172.21.15.5:0/1634606116 conn(0x7f04600ffea0 msgr2=0x7f04601022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:03.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:03.012+0000 7f0467390700 1 -- 172.21.15.5:0/1634606116 shutdown_connections 2024-05-09T00:46:03.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:03.012+0000 7f0467390700 1 -- 172.21.15.5:0/1634606116 wait complete. 2024-05-09T00:46:03.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:03.013+0000 7f0467390700 1 Processor -- start 2024-05-09T00:46:03.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:03.013+0000 7f0467390700 1 -- start start 2024-05-09T00:46:03.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:03.014+0000 7f0467390700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04601045d0 0x7f046019aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:03.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:03.014+0000 7f0467390700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0460108f00 0x7f046019b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:03.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:03.014+0000 7f0467390700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0460109850 0x7f046019f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:03.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:03.014+0000 7f0467390700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f04601144b0 con 0x7f0460108f00 2024-05-09T00:46:03.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:03.014+0000 7f0467390700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0460114330 con 0x7f0460109850 2024-05-09T00:46:03.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:03.014+0000 7f0467390700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0460114630 con 0x7f04601045d0 2024-05-09T00:46:03.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:03.014+0000 7f046492b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0460108f00 0x7f046019b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:03.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:03.014+0000 7f046492b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0460108f00 0x7f046019b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59238/0 (socket says 172.21.15.5:59238) 2024-05-09T00:46:03.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:03.014+0000 7f046492b700 1 -- 172.21.15.5:0/3072922863 learned_addr learned my addr 172.21.15.5:0/3072922863 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:46:03.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:03.014+0000 7f046512c700 1 --2- 172.21.15.5:0/3072922863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04601045d0 0x7f046019aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:03.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:03.014+0000 7f046592d700 1 --2- 172.21.15.5:0/3072922863 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0460109850 0x7f046019f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:03.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:03.014+0000 7f046492b700 1 -- 172.21.15.5:0/3072922863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04601045d0 msgr2=0x7f046019aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:03.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:03.014+0000 7f046492b700 1 --2- 172.21.15.5:0/3072922863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04601045d0 0x7f046019aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:03.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:03.014+0000 7f046492b700 1 -- 172.21.15.5:0/3072922863 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0460109850 msgr2=0x7f046019f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:03.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:03.014+0000 7f046492b700 1 --2- 172.21.15.5:0/3072922863 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0460109850 0x7f046019f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:03.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:03.014+0000 7f046492b700 1 -- 172.21.15.5:0/3072922863 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f046019fd90 con 0x7f0460108f00 2024-05-09T00:46:03.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:03.014+0000 7f046492b700 1 --2- 172.21.15.5:0/3072922863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0460108f00 0x7f046019b460 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0454009f50 tx=0x7f04540057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:03.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:03.014+0000 7f04527fc700 1 -- 172.21.15.5:0/3072922863 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0454026070 con 0x7f0460108f00 2024-05-09T00:46:03.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:03.015+0000 7f0467390700 1 -- 172.21.15.5:0/3072922863 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04601a0020 con 0x7f0460108f00 2024-05-09T00:46:03.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:03.015+0000 7f04527fc700 1 -- 172.21.15.5:0/3072922863 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0454003f20 con 0x7f0460108f00 2024-05-09T00:46:03.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:03.015+0000 7f04527fc700 1 -- 172.21.15.5:0/3072922863 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045400fc20 con 0x7f0460108f00 2024-05-09T00:46:03.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:03.015+0000 7f0467390700 1 -- 172.21.15.5:0/3072922863 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04601a0500 con 0x7f0460108f00 2024-05-09T00:46:03.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:03.016+0000 7f0467390700 1 -- 172.21.15.5:0/3072922863 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04600512c0 con 0x7f0460108f00 2024-05-09T00:46:03.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:03.017+0000 7f04527fc700 1 -- 172.21.15.5:0/3072922863 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f0454029070 con 0x7f0460108f00 2024-05-09T00:46:03.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:03.018+0000 7f04527fc700 1 --2- 172.21.15.5:0/3072922863 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f044c0b2880 0x7f044c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:03.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:03.018+0000 7f046512c700 1 --2- 172.21.15.5:0/3072922863 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f044c0b2880 0x7f044c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:03.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:03.018+0000 7f04527fc700 1 -- 172.21.15.5:0/3072922863 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f04540e31e0 con 0x7f0460108f00 2024-05-09T00:46:03.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:03.019+0000 7f046512c700 1 --2- 172.21.15.5:0/3072922863 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f044c0b2880 0x7f044c0b4d90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f045c005950 tx=0x7f045c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:03.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:03.022+0000 7f04527fc700 1 -- 172.21.15.5:0/3072922863 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0454013070 con 0x7f0460108f00 2024-05-09T00:46:03.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:03.287+0000 7f0467390700 1 -- 172.21.15.5:0/3072922863 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f046019c220 con 0x7f0460108f00 2024-05-09T00:46:03.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:03.288+0000 7f04527fc700 1 -- 172.21.15.5:0/3072922863 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11926 (secure 0 0 0) 0x7f04540aaf00 con 0x7f0460108f00 2024-05-09T00:46:03.290 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:46:03.290 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2685771887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2685771887},{"type":"v1","addr":"172.21.15.5:6835","nonce":2685771887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/254017535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":254017535},{"type":"v1","addr":"172.21.15.5:6837","nonce":254017535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T00:43:45.856066+0000","modified":"2024-05-09T00:44:06.033927+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24508,"mds_1":24499,"mds_2":24494,"mds_3":24493,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14598":{"gid":14598,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2109255228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2109255228},{"type":"v1","addr":"172.21.15.5:6839","nonce":2109255228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/3809966329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3809966329},{"type":"v1","addr":"172.21.15.5:6841","nonce":3809966329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6833/3272897742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3272897742},{"type":"v1","addr":"172.21.15.131:6833","nonce":3272897742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1283093898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1283093898},{"type":"v1","addr":"172.21.15.131:6835","nonce":1283093898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/2809525455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2809525455},{"type":"v1","addr":"172.21.15.157:6833","nonce":2809525455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.131:6837/1633006186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":1633006186},{"type":"v1","addr":"172.21.15.131:6837","nonce":1633006186}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.157:6835/832898273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":832898273},{"type":"v1","addr":"172.21.15.157:6835","nonce":832898273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.131:6839/2748669631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2748669631},{"type":"v1","addr":"172.21.15.131:6839","nonce":2748669631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.157:6837/744863009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":744863009},{"type":"v1","addr":"172.21.15.157:6837","nonce":744863009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.157:6839/2779860399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":2779860399},{"type":"v1","addr":"172.21.15.157:6839","nonce":2779860399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24467,24493,24499,24508,24494]},"id":1}]} 2024-05-09T00:46:03.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:03.290+0000 7f0467390700 1 -- 172.21.15.5:0/3072922863 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f044c0b2880 msgr2=0x7f044c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:03.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:03.290+0000 7f0467390700 1 --2- 172.21.15.5:0/3072922863 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f044c0b2880 0x7f044c0b4d90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f045c005950 tx=0x7f045c00b410 comp rx=0 tx=0).stop 2024-05-09T00:46:03.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:03.290+0000 7f0467390700 1 -- 172.21.15.5:0/3072922863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0460108f00 msgr2=0x7f046019b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:03.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:03.290+0000 7f0467390700 1 --2- 172.21.15.5:0/3072922863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0460108f00 0x7f046019b460 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0454009f50 tx=0x7f04540057c0 comp rx=0 tx=0).stop 2024-05-09T00:46:03.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:03.290+0000 7f0467390700 1 -- 172.21.15.5:0/3072922863 shutdown_connections 2024-05-09T00:46:03.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:03.290+0000 7f0467390700 1 --2- 172.21.15.5:0/3072922863 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f044c0b2880 0x7f044c0b4d90 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:03.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:03.290+0000 7f0467390700 1 --2- 172.21.15.5:0/3072922863 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0460109850 0x7f046019f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:03.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:03.290+0000 7f0467390700 1 --2- 172.21.15.5:0/3072922863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04601045d0 0x7f046019aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:03.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:03.290+0000 7f0467390700 1 --2- 172.21.15.5:0/3072922863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0460108f00 0x7f046019b460 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:03.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:03.290+0000 7f0467390700 1 -- 172.21.15.5:0/3072922863 >> 172.21.15.5:0/3072922863 conn(0x7f04600ffea0 msgr2=0x7f046010a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:03.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:03.291+0000 7f0467390700 1 -- 172.21.15.5:0/3072922863 shutdown_connections 2024-05-09T00:46:03.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:03.291+0000 7f0467390700 1 -- 172.21.15.5:0/3072922863 wait complete. 2024-05-09T00:46:03.293 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 28 2024-05-09T00:46:03.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:46:03 smithi005 ceph-mon[30365]: pgmap v362: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 767 B/s wr, 9 op/s 2024-05-09T00:46:03.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:46:03 smithi131 ceph-mon[34547]: pgmap v362: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 767 B/s wr, 9 op/s 2024-05-09T00:46:03.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:46:03 smithi157 ceph-mon[34294]: pgmap v362: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 767 B/s wr, 9 op/s 2024-05-09T00:46:04.006 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-09T00:43:45.856066+0000', 'modified': '2024-05-09T00:44:06.033927+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24508, 'mds_1': 24499, 'mds_2': 24494, 'mds_3': 24493, 'mds_4': 24467}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14598': {'gid': 14598, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.5:6839/2109255228', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6838', 'nonce': 2109255228}, {'type': 'v1', 'addr': '172.21.15.5:6839', 'nonce': 2109255228}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14610': {'gid': 14610, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.5:6841/3809966329', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6840', 'nonce': 3809966329}, {'type': 'v1', 'addr': '172.21.15.5:6841', 'nonce': 3809966329}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24419': {'gid': 24419, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.131:6833/3272897742', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6832', 'nonce': 3272897742}, {'type': 'v1', 'addr': '172.21.15.131:6833', 'nonce': 3272897742}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24443': {'gid': 24443, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.131:6835/1283093898', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6834', 'nonce': 1283093898}, {'type': 'v1', 'addr': '172.21.15.131:6835', 'nonce': 1283093898}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24457': {'gid': 24457, 'name': 'l', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.157:6833/2809525455', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6832', 'nonce': 2809525455}, {'type': 'v1', 'addr': '172.21.15.157:6833', 'nonce': 2809525455}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24467': {'gid': 24467, 'name': 'h', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.131:6837/1633006186', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6836', 'nonce': 1633006186}, {'type': 'v1', 'addr': '172.21.15.131:6837', 'nonce': 1633006186}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24493': {'gid': 24493, 'name': 'f', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.157:6835/832898273', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6834', 'nonce': 832898273}, {'type': 'v1', 'addr': '172.21.15.157:6835', 'nonce': 832898273}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24494': {'gid': 24494, 'name': 'b', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.131:6839/2748669631', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6838', 'nonce': 2748669631}, {'type': 'v1', 'addr': '172.21.15.131:6839', 'nonce': 2748669631}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24499': {'gid': 24499, 'name': 'c', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.157:6837/744863009', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6836', 'nonce': 744863009}, {'type': 'v1', 'addr': '172.21.15.157:6837', 'nonce': 744863009}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24508': {'gid': 24508, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.157:6839/2779860399', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6838', 'nonce': 2779860399}, {'type': 'v1', 'addr': '172.21.15.157:6839', 'nonce': 2779860399}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24508, 'qdb_cluster': [24467, 24493, 24499, 24508, 24494]} 2024-05-09T00:46:04.006 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-05-09T00:46:04.006 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.h status 2024-05-09T00:46:04.267 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:46:04.531 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:46:04 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/3072922863' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:46:04.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:46:04 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/3072922863' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:46:04.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:46:04 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/3072922863' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:46:05.401 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:46:05 smithi005 ceph-mon[30365]: pgmap v363: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 767 B/s wr, 12 op/s 2024-05-09T00:46:05.587 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:46:05 smithi157 ceph-mon[34294]: pgmap v363: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 767 B/s wr, 12 op/s 2024-05-09T00:46:05.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:46:05 smithi131 ceph-mon[34547]: pgmap v363: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 767 B/s wr, 12 op/s 2024-05-09T00:46:05.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.949+0000 7f507d365700 1 -- 172.21.15.5:0/4250420016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50781034c0 msgr2=0x7f50781038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:05.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.949+0000 7f507d365700 1 --2- 172.21.15.5:0/4250420016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50781034c0 0x7f50781038d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f50680099c0 tx=0x7f5068009cd0 comp rx=0 tx=0).stop 2024-05-09T00:46:05.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.949+0000 7f507d365700 1 -- 172.21.15.5:0/4250420016 shutdown_connections 2024-05-09T00:46:05.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.949+0000 7f507d365700 1 --2- 172.21.15.5:0/4250420016 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5078108740 0x7f50781100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:05.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.949+0000 7f507d365700 1 --2- 172.21.15.5:0/4250420016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50781034c0 0x7f50781038d0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:05.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.949+0000 7f507d365700 1 --2- 172.21.15.5:0/4250420016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5078107df0 0x7f5078108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:05.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.949+0000 7f507d365700 1 -- 172.21.15.5:0/4250420016 >> 172.21.15.5:0/4250420016 conn(0x7f50780fedf0 msgr2=0x7f5078101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:05.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.949+0000 7f507d365700 1 -- 172.21.15.5:0/4250420016 shutdown_connections 2024-05-09T00:46:05.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.949+0000 7f507d365700 1 -- 172.21.15.5:0/4250420016 wait complete. 2024-05-09T00:46:05.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.949+0000 7f507d365700 1 Processor -- start 2024-05-09T00:46:05.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.950+0000 7f507d365700 1 -- start start 2024-05-09T00:46:05.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.950+0000 7f507d365700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50781034c0 0x7f507819ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:05.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.950+0000 7f507d365700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5078107df0 0x7f507819b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:05.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.950+0000 7f507d365700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5078108740 0x7f507819f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:05.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.950+0000 7f507d365700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5078113390 con 0x7f5078108740 2024-05-09T00:46:05.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.950+0000 7f507d365700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5078113210 con 0x7f50781034c0 2024-05-09T00:46:05.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.950+0000 7f507d365700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5078113510 con 0x7f5078107df0 2024-05-09T00:46:05.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.950+0000 7f50777fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5078108740 0x7f507819f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:05.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.950+0000 7f50777fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5078108740 0x7f507819f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58758/0 (socket says 172.21.15.5:58758) 2024-05-09T00:46:05.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.950+0000 7f50777fe700 1 -- 172.21.15.5:0/3503570179 learned_addr learned my addr 172.21.15.5:0/3503570179 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:46:05.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.950+0000 7f5076ffd700 1 --2- 172.21.15.5:0/3503570179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50781034c0 0x7f507819ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:05.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.950+0000 7f506e7fc700 1 --2- 172.21.15.5:0/3503570179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5078107df0 0x7f507819b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:05.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.950+0000 7f50777fe700 1 -- 172.21.15.5:0/3503570179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5078107df0 msgr2=0x7f507819b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:05.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.950+0000 7f50777fe700 1 --2- 172.21.15.5:0/3503570179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5078107df0 0x7f507819b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:05.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.950+0000 7f50777fe700 1 -- 172.21.15.5:0/3503570179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50781034c0 msgr2=0x7f507819ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:05.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.950+0000 7f50777fe700 1 --2- 172.21.15.5:0/3503570179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50781034c0 0x7f507819ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:05.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.951+0000 7f50777fe700 1 -- 172.21.15.5:0/3503570179 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f507819fc70 con 0x7f5078108740 2024-05-09T00:46:05.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.951+0000 7f50777fe700 1 --2- 172.21.15.5:0/3503570179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5078108740 0x7f507819f680 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5070007ee0 tx=0x7f507000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:05.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.951+0000 7f5074ff9700 1 -- 172.21.15.5:0/3503570179 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507000dbd0 con 0x7f5078108740 2024-05-09T00:46:05.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.951+0000 7f5074ff9700 1 -- 172.21.15.5:0/3503570179 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f507000dd70 con 0x7f5078108740 2024-05-09T00:46:05.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.951+0000 7f507d365700 1 -- 172.21.15.5:0/3503570179 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f507819ff60 con 0x7f5078108740 2024-05-09T00:46:05.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.951+0000 7f5074ff9700 1 -- 172.21.15.5:0/3503570179 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5070005990 con 0x7f5078108740 2024-05-09T00:46:05.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.951+0000 7f507d365700 1 -- 172.21.15.5:0/3503570179 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50781a0450 con 0x7f5078108740 2024-05-09T00:46:05.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.954+0000 7f5074ff9700 1 -- 172.21.15.5:0/3503570179 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f507001a020 con 0x7f5078108740 2024-05-09T00:46:05.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.955+0000 7f5074ff9700 1 --2- 172.21.15.5:0/3503570179 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f50640b2880 0x7f50640b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:05.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.955+0000 7f5076ffd700 1 --2- 172.21.15.5:0/3503570179 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f50640b2880 0x7f50640b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:05.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.955+0000 7f5074ff9700 1 -- 172.21.15.5:0/3503570179 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f50700e19e0 con 0x7f5078108740 2024-05-09T00:46:05.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.955+0000 7f508bd7bb80 1 Processor -- start 2024-05-09T00:46:05.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.955+0000 7f508bd7bb80 1 -- start start 2024-05-09T00:46:05.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.955+0000 7f5076ffd700 1 --2- 172.21.15.5:0/3503570179 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f50640b2880 0x7f50640b4d90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5068009990 tx=0x7f50680036e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:05.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.955+0000 7f508bd7bb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f08ade3cb0 0x55f08aeeaa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:05.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.955+0000 7f508bd7bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f08ad69450 0x55f08aeeafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:05.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.955+0000 7f508bd7bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f08aeef190 0x55f08aeef5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:05.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.955+0000 7f508bd7bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f08ae09050 con 0x55f08ad69450 2024-05-09T00:46:05.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.956+0000 7f508bd7bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55f08adae950 con 0x55f08aeef190 2024-05-09T00:46:05.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.956+0000 7f508bd7bb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55f08ae05010 con 0x55f08ade3cb0 2024-05-09T00:46:05.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.956+0000 7f506e7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f08ad69450 0x55f08aeeafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:05.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.956+0000 7f506e7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f08ad69450 0x55f08aeeafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58774/0 (socket says 172.21.15.5:58774) 2024-05-09T00:46:05.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.956+0000 7f506e7fc700 1 -- 172.21.15.5:0/1417337971 learned_addr learned my addr 172.21.15.5:0/1417337971 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:46:05.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.956+0000 7f5076ffd700 1 --2- 172.21.15.5:0/1417337971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f08ade3cb0 0x55f08aeeaa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:05.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.956+0000 7f50777fe700 1 --2- 172.21.15.5:0/1417337971 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f08aeef190 0x55f08aeef5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:05.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.956+0000 7f506e7fc700 1 -- 172.21.15.5:0/1417337971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f08ade3cb0 msgr2=0x55f08aeeaa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:05.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.956+0000 7f506e7fc700 1 --2- 172.21.15.5:0/1417337971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f08ade3cb0 0x55f08aeeaa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:05.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.956+0000 7f506e7fc700 1 -- 172.21.15.5:0/1417337971 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f08aeef190 msgr2=0x55f08aeef5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:05.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.956+0000 7f506e7fc700 1 --2- 172.21.15.5:0/1417337971 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f08aeef190 0x55f08aeef5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:05.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.956+0000 7f506e7fc700 1 -- 172.21.15.5:0/1417337971 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f08ad9f4b0 con 0x55f08ad69450 2024-05-09T00:46:05.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.956+0000 7f50777fe700 1 --2- 172.21.15.5:0/1417337971 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f08aeef190 0x55f08aeef5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:46:05.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.956+0000 7f506e7fc700 1 --2- 172.21.15.5:0/1417337971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f08ad69450 0x55f08aeeafe0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5060008c00 tx=0x7f5060008f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93c2e2a670531abd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:05.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.956+0000 7f506d7fa700 1 -- 172.21.15.5:0/1417337971 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5060014070 con 0x55f08ad69450 2024-05-09T00:46:05.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.956+0000 7f506d7fa700 1 -- 172.21.15.5:0/1417337971 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5060004ce0 con 0x55f08ad69450 2024-05-09T00:46:05.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.956+0000 7f506d7fa700 1 -- 172.21.15.5:0/1417337971 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5060005660 con 0x55f08ad69450 2024-05-09T00:46:05.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.957+0000 7f508bd7bb80 1 -- 172.21.15.5:0/1417337971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f08ad69450 msgr2=0x55f08aeeafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:05.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.957+0000 7f508bd7bb80 1 --2- 172.21.15.5:0/1417337971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f08ad69450 0x55f08aeeafe0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5060008c00 tx=0x7f5060008f10 comp rx=0 tx=0).stop 2024-05-09T00:46:05.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.957+0000 7f508bd7bb80 1 -- 172.21.15.5:0/1417337971 shutdown_connections 2024-05-09T00:46:05.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.957+0000 7f508bd7bb80 1 --2- 172.21.15.5:0/1417337971 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f08aeef190 0x55f08aeef5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:05.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.957+0000 7f508bd7bb80 1 --2- 172.21.15.5:0/1417337971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f08ade3cb0 0x55f08aeeaa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:05.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.957+0000 7f508bd7bb80 1 --2- 172.21.15.5:0/1417337971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f08ad69450 0x55f08aeeafe0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:05.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.957+0000 7f508bd7bb80 1 -- 172.21.15.5:0/1417337971 >> 172.21.15.5:0/1417337971 conn(0x55f08ad6cf20 msgr2=0x55f08aee11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:05.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.958+0000 7f508bd7bb80 1 -- 172.21.15.5:0/1417337971 shutdown_connections 2024-05-09T00:46:05.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.958+0000 7f508bd7bb80 1 -- 172.21.15.5:0/1417337971 wait complete. 2024-05-09T00:46:05.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.958+0000 7f508bd7bb80 1 -- start start 2024-05-09T00:46:05.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.958+0000 7f508bd7bb80 1 Processor -- start 2024-05-09T00:46:05.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.958+0000 7f508bd7bb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f08ad69450 0x55f08aef4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:05.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.958+0000 7f508bd7bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f08ade3cb0 0x55f08aef4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:05.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.958+0000 7f508bd7bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f08aeef190 0x55f08aef4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:05.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.958+0000 7f508bd7bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f08aeefeb0 con 0x55f08ade3cb0 2024-05-09T00:46:05.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.958+0000 7f508bd7bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55f08ada3ad0 con 0x55f08aeef190 2024-05-09T00:46:05.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.958+0000 7f508bd7bb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55f08aef7420 con 0x55f08ad69450 2024-05-09T00:46:05.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.959+0000 7f508bd7bb80 10 client.0 _ll_register_callbacks cb 0x55f08ac7a670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:46:05.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.959+0000 7f506e7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f08ade3cb0 0x55f08aef4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:05.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.959+0000 7f506e7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f08ade3cb0 0x55f08aef4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58776/0 (socket says 172.21.15.5:58776) 2024-05-09T00:46:05.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.959+0000 7f506e7fc700 1 -- 172.21.15.5:0/1986532160 learned_addr learned my addr 172.21.15.5:0/1986532160 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:46:05.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.959+0000 7f50777fe700 1 --2- 172.21.15.5:0/1986532160 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f08aeef190 0x55f08aef4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:05.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.959+0000 7f5076ffd700 1 --2- 172.21.15.5:0/1986532160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f08ad69450 0x55f08aef4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:05.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.959+0000 7f506e7fc700 1 -- 172.21.15.5:0/1986532160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f08ad69450 msgr2=0x55f08aef4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:05.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.959+0000 7f506e7fc700 1 --2- 172.21.15.5:0/1986532160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f08ad69450 0x55f08aef4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:05.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.959+0000 7f506e7fc700 1 -- 172.21.15.5:0/1986532160 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f08aeef190 msgr2=0x55f08aef4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:05.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.959+0000 7f506e7fc700 1 --2- 172.21.15.5:0/1986532160 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f08aeef190 0x55f08aef4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:05.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.959+0000 7f506e7fc700 1 -- 172.21.15.5:0/1986532160 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f08ac84440 con 0x55f08ade3cb0 2024-05-09T00:46:05.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.959+0000 7f506e7fc700 1 --2- 172.21.15.5:0/1986532160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f08ade3cb0 0x55f08aef4870 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5060005200 tx=0x7f5060005120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed9c640fb66ca0d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:05.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.959+0000 7f505e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:46:05.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.959+0000 7f505e7fc700 1 -- 172.21.15.5:0/1986532160 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5060027060 con 0x55f08ade3cb0 2024-05-09T00:46:05.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.959+0000 7f505e7fc700 1 -- 172.21.15.5:0/1986532160 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f5060004750 con 0x55f08ade3cb0 2024-05-09T00:46:05.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.959+0000 7f508bd7bb80 1 -- 172.21.15.5:0/1986532160 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5060004ce0 con 0x55f08ade3cb0 2024-05-09T00:46:05.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.959+0000 7f505e7fc700 1 -- 172.21.15.5:0/1986532160 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5060014070 con 0x55f08ade3cb0 2024-05-09T00:46:05.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.960+0000 7f508bd7bb80 10 client.14961 fetch_fsmap learned FSMap version 28 2024-05-09T00:46:05.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.960+0000 7f508bd7bb80 1 -- 172.21.15.5:0/1986532160 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f08af01cc0 con 0x55f08ade3cb0 2024-05-09T00:46:05.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.960+0000 7f505e7fc700 1 -- 172.21.15.5:0/1986532160 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5060004ce0 con 0x55f08ade3cb0 2024-05-09T00:46:05.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.960+0000 7f505e7fc700 1 -- 172.21.15.5:0/1986532160 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f506002d410 con 0x55f08ade3cb0 2024-05-09T00:46:05.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.960+0000 7f508bd7bb80 10 client.14961 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T00:46:05.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.960+0000 7f508bd7bb80 10 client.14961 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-09T00:46:05.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.960+0000 7f508bd7bb80 20 client.14961 populate_metadata read hostname 'smithi005' 2024-05-09T00:46:05.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.960+0000 7f508bd7bb80 1 --2- 172.21.15.5:0/1986532160 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x55f08af03e20 0x55f08af05290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:05.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.960+0000 7f508bd7bb80 4 client.14961 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:46:05.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.960+0000 7f508bd7bb80 1 -- 172.21.15.5:0/1986532160 --> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f08ae16a30 con 0x55f08af03e20 2024-05-09T00:46:05.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.960+0000 7f5076ffd700 1 --2- 172.21.15.5:0/1986532160 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x55f08af03e20 0x55f08af05290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:05.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.961+0000 7f5076ffd700 1 --2- 172.21.15.5:0/1986532160 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x55f08af03e20 0x55f08af05290 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ed95c3f7d0965387 server_cookie=b54847ca63a4272d in_seq=0 out_seq=0 2024-05-09T00:46:05.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.961+0000 7f505e7fc700 10 client.14961 ms_handle_connect on v2:172.21.15.131:6836/1633006186 2024-05-09T00:46:05.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.962+0000 7f505e7fc700 1 -- 172.21.15.5:0/1986532160 <== mds.4 v2:172.21.15.131:6836/1633006186 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f08ae16a30 con 0x55f08af03e20 2024-05-09T00:46:05.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.962+0000 7f505e7fc700 10 client.14961 handle_command_reply: tid=0 2024-05-09T00:46:05.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.962+0000 7f508bd7bb80 1 client.14961 shutdown 2024-05-09T00:46:05.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.962+0000 7f508bd7bb80 1 -- 172.21.15.5:0/1986532160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f08ade3cb0 msgr2=0x55f08aef4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:05.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.962+0000 7f508bd7bb80 1 --2- 172.21.15.5:0/1986532160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f08ade3cb0 0x55f08aef4870 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5060005200 tx=0x7f5060005120 comp rx=0 tx=0).stop 2024-05-09T00:46:05.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.962+0000 7f508bd7bb80 1 -- 172.21.15.5:0/1986532160 shutdown_connections 2024-05-09T00:46:05.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.962+0000 7f508bd7bb80 1 --2- 172.21.15.5:0/1986532160 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x55f08af03e20 0x55f08af05290 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:05.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.962+0000 7f508bd7bb80 1 --2- 172.21.15.5:0/1986532160 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f08aeef190 0x55f08aef4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:05.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.962+0000 7f508bd7bb80 1 --2- 172.21.15.5:0/1986532160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f08ad69450 0x55f08aef4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:05.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.962+0000 7f508bd7bb80 1 --2- 172.21.15.5:0/1986532160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f08ade3cb0 0x55f08aef4870 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:05.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.962+0000 7f505e7fc700 0 client.14961 ms_handle_reset on v2:172.21.15.131:6836/1633006186 2024-05-09T00:46:05.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.962+0000 7f508bd7bb80 1 -- 172.21.15.5:0/1986532160 >> 172.21.15.5:0/1986532160 conn(0x55f08ad6cf20 msgr2=0x55f08aedafa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:05.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.962+0000 7f508bd7bb80 1 -- 172.21.15.5:0/1986532160 shutdown_connections 2024-05-09T00:46:05.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.962+0000 7f508bd7bb80 1 -- 172.21.15.5:0/1986532160 wait complete. 2024-05-09T00:46:05.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.962+0000 7f508bd7bb80 20 client.14961 trim_cache size 0 max 16384 2024-05-09T00:46:05.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.983+0000 7f508bd7bb80 1 Processor -- start 2024-05-09T00:46:05.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.983+0000 7f508bd7bb80 1 -- start start 2024-05-09T00:46:05.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.984+0000 7f508bd7bb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f08ad69450 0x55f08aee1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:05.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.984+0000 7f508bd7bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f08ade3cb0 0x55f08af00360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:05.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.984+0000 7f508bd7bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f08aeef190 0x55f08aedb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:05.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.984+0000 7f508bd7bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f08af035d0 con 0x55f08ade3cb0 2024-05-09T00:46:05.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.984+0000 7f508bd7bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55f08af03ca0 con 0x55f08aeef190 2024-05-09T00:46:05.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.984+0000 7f508bd7bb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55f08af04370 con 0x55f08ad69450 2024-05-09T00:46:05.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.984+0000 7f506e7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f08ade3cb0 0x55f08af00360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:05.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.984+0000 7f506e7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f08ade3cb0 0x55f08af00360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58786/0 (socket says 172.21.15.5:58786) 2024-05-09T00:46:05.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.984+0000 7f506e7fc700 1 -- 172.21.15.5:0/1016866728 learned_addr learned my addr 172.21.15.5:0/1016866728 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:46:05.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.984+0000 7f5076ffd700 1 --2- 172.21.15.5:0/1016866728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f08ad69450 0x55f08aee1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:05.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.984+0000 7f50777fe700 1 --2- 172.21.15.5:0/1016866728 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f08aeef190 0x55f08aedb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:05.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.984+0000 7f506e7fc700 1 -- 172.21.15.5:0/1016866728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f08ad69450 msgr2=0x55f08aee1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:05.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.984+0000 7f506e7fc700 1 --2- 172.21.15.5:0/1016866728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f08ad69450 0x55f08aee1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:05.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.984+0000 7f506e7fc700 1 -- 172.21.15.5:0/1016866728 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f08aeef190 msgr2=0x55f08aedb6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:05.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.984+0000 7f506e7fc700 1 --2- 172.21.15.5:0/1016866728 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f08aeef190 0x55f08aedb6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:05.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.984+0000 7f506e7fc700 1 -- 172.21.15.5:0/1016866728 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f08aef28a0 con 0x55f08ade3cb0 2024-05-09T00:46:05.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.984+0000 7f50777fe700 1 --2- 172.21.15.5:0/1016866728 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f08aeef190 0x55f08aedb6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:46:05.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.984+0000 7f506e7fc700 1 --2- 172.21.15.5:0/1016866728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f08ade3cb0 0x55f08af00360 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5060008b20 tx=0x7f5060008f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68264bb733d23af9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:05.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.985+0000 7f506cff9700 1 -- 172.21.15.5:0/1016866728 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5060027040 con 0x55f08ade3cb0 2024-05-09T00:46:05.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.985+0000 7f506cff9700 1 -- 172.21.15.5:0/1016866728 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f506002d410 con 0x55f08ade3cb0 2024-05-09T00:46:05.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.985+0000 7f506cff9700 1 -- 172.21.15.5:0/1016866728 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5060010430 con 0x55f08ade3cb0 2024-05-09T00:46:05.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.985+0000 7f508bd7bb80 1 -- 172.21.15.5:0/1016866728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f08ade3cb0 msgr2=0x55f08af00360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:05.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.985+0000 7f508bd7bb80 1 --2- 172.21.15.5:0/1016866728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f08ade3cb0 0x55f08af00360 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5060008b20 tx=0x7f5060008f10 comp rx=0 tx=0).stop 2024-05-09T00:46:05.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.985+0000 7f508bd7bb80 1 -- 172.21.15.5:0/1016866728 shutdown_connections 2024-05-09T00:46:05.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.985+0000 7f508bd7bb80 1 --2- 172.21.15.5:0/1016866728 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f08aeef190 0x55f08aedb6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:05.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.985+0000 7f508bd7bb80 1 --2- 172.21.15.5:0/1016866728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f08ad69450 0x55f08aee1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:05.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.985+0000 7f508bd7bb80 1 --2- 172.21.15.5:0/1016866728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f08ade3cb0 0x55f08af00360 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:05.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.985+0000 7f508bd7bb80 1 -- 172.21.15.5:0/1016866728 >> 172.21.15.5:0/1016866728 conn(0x55f08ad6cf20 msgr2=0x55f08aedafa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:05.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.986+0000 7f508bd7bb80 1 -- 172.21.15.5:0/1016866728 shutdown_connections 2024-05-09T00:46:05.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.986+0000 7f508bd7bb80 1 -- 172.21.15.5:0/1016866728 wait complete. 2024-05-09T00:46:05.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.986+0000 7f508bd7bb80 1 -- start start 2024-05-09T00:46:05.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.986+0000 7f508bd7bb80 1 Processor -- start 2024-05-09T00:46:05.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.986+0000 7f508bd7bb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f08ad69450 0x55f08af0e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:05.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.986+0000 7f508bd7bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f08ade3cb0 0x55f08af0ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:05.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.986+0000 7f508bd7bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f08aeef190 0x55f08af096c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:05.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.986+0000 7f508bd7bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f08af05110 con 0x55f08ade3cb0 2024-05-09T00:46:05.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.986+0000 7f508bd7bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55f08af04a40 con 0x55f08aeef190 2024-05-09T00:46:05.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.986+0000 7f508bd7bb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55f08af057e0 con 0x55f08ad69450 2024-05-09T00:46:05.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.986+0000 7f508bd7bb80 10 client.0 _ll_register_callbacks cb 0x55f08ac7a670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:46:05.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.986+0000 7f506e7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f08ade3cb0 0x55f08af0ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:05.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.986+0000 7f506e7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f08ade3cb0 0x55f08af0ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58790/0 (socket says 172.21.15.5:58790) 2024-05-09T00:46:05.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.986+0000 7f506e7fc700 1 -- 172.21.15.5:0/354848761 learned_addr learned my addr 172.21.15.5:0/354848761 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:46:05.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.987+0000 7f5076ffd700 1 --2- 172.21.15.5:0/354848761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f08ad69450 0x55f08af0e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:05.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.987+0000 7f506e7fc700 1 -- 172.21.15.5:0/354848761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f08ad69450 msgr2=0x55f08af0e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:05.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.987+0000 7f506e7fc700 1 --2- 172.21.15.5:0/354848761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f08ad69450 0x55f08af0e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:05.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.987+0000 7f50777fe700 1 --2- 172.21.15.5:0/354848761 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f08aeef190 0x55f08af096c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:05.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.987+0000 7f506e7fc700 1 -- 172.21.15.5:0/354848761 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f08aeef190 msgr2=0x55f08af096c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:05.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.987+0000 7f506e7fc700 1 --2- 172.21.15.5:0/354848761 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f08aeef190 0x55f08af096c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:05.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.987+0000 7f506e7fc700 1 -- 172.21.15.5:0/354848761 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f08af09c00 con 0x55f08ade3cb0 2024-05-09T00:46:05.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.987+0000 7f506e7fc700 1 --2- 172.21.15.5:0/354848761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f08ade3cb0 0x55f08af0ec50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f506002d950 tx=0x7f506002dd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b62bbc3509f5cb2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:05.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.987+0000 7f505e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:46:05.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.987+0000 7f505e7fc700 1 -- 172.21.15.5:0/354848761 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5060003890 con 0x55f08ade3cb0 2024-05-09T00:46:05.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.987+0000 7f505e7fc700 1 -- 172.21.15.5:0/354848761 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f5060003a30 con 0x55f08ade3cb0 2024-05-09T00:46:05.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.987+0000 7f508bd7bb80 1 -- 172.21.15.5:0/354848761 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f506002d410 con 0x55f08ade3cb0 2024-05-09T00:46:05.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.987+0000 7f505e7fc700 1 -- 172.21.15.5:0/354848761 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5060003c30 con 0x55f08ade3cb0 2024-05-09T00:46:05.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.987+0000 7f508bd7bb80 10 client.14967 fetch_fsmap learned FSMap version 28 2024-05-09T00:46:05.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.987+0000 7f508bd7bb80 1 -- 172.21.15.5:0/354848761 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f08af6f880 con 0x55f08ade3cb0 2024-05-09T00:46:05.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.988+0000 7f505e7fc700 1 -- 172.21.15.5:0/354848761 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f506002d410 con 0x55f08ade3cb0 2024-05-09T00:46:05.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.988+0000 7f505e7fc700 1 -- 172.21.15.5:0/354848761 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f506003d030 con 0x55f08ade3cb0 2024-05-09T00:46:05.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.988+0000 7f508bd7bb80 10 client.14967 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T00:46:05.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.988+0000 7f508bd7bb80 10 client.14967 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-09T00:46:05.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.988+0000 7f508bd7bb80 20 client.14967 populate_metadata read hostname 'smithi005' 2024-05-09T00:46:05.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.988+0000 7f508bd7bb80 1 --2- 172.21.15.5:0/354848761 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x55f08af719a0 0x55f08af72e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:05.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.988+0000 7f508bd7bb80 4 client.14967 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T00:46:05.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.988+0000 7f508bd7bb80 1 -- 172.21.15.5:0/354848761 --> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] -- command(tid 0: {"prefix": "status"}) -- 0x55f08ae10ad0 con 0x55f08af719a0 2024-05-09T00:46:05.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.988+0000 7f5076ffd700 1 --2- 172.21.15.5:0/354848761 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x55f08af719a0 0x55f08af72e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:05.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.989+0000 7f505e7fc700 10 client.14967 ms_handle_connect on v2:172.21.15.131:6836/1633006186 2024-05-09T00:46:05.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.989+0000 7f5076ffd700 1 --2- 172.21.15.5:0/354848761 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x55f08af719a0 0x55f08af72e10 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6bdd790402f3be0b server_cookie=481b7406e4506f96 in_seq=0 out_seq=0 2024-05-09T00:46:05.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.989+0000 7f505e7fc700 1 -- 172.21.15.5:0/354848761 <== mds.4 v2:172.21.15.131:6836/1633006186 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55f08ae10ad0 con 0x55f08af719a0 2024-05-09T00:46:05.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.989+0000 7f505e7fc700 10 client.14967 handle_command_reply: tid=0 2024-05-09T00:46:05.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.989+0000 7f508bd7bb80 1 client.14967 shutdown 2024-05-09T00:46:05.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.989+0000 7f508bd7bb80 1 -- 172.21.15.5:0/354848761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f08ade3cb0 msgr2=0x55f08af0ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:05.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.989+0000 7f508bd7bb80 1 --2- 172.21.15.5:0/354848761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f08ade3cb0 0x55f08af0ec50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f506002d950 tx=0x7f506002dd00 comp rx=0 tx=0).stop 2024-05-09T00:46:05.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.989+0000 7f508bd7bb80 1 -- 172.21.15.5:0/354848761 shutdown_connections 2024-05-09T00:46:05.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.989+0000 7f508bd7bb80 1 --2- 172.21.15.5:0/354848761 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x55f08af719a0 0x55f08af72e10 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:05.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.989+0000 7f508bd7bb80 1 --2- 172.21.15.5:0/354848761 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f08aeef190 0x55f08af096c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:05.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.989+0000 7f508bd7bb80 1 --2- 172.21.15.5:0/354848761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f08ad69450 0x55f08af0e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:05.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.989+0000 7f505e7fc700 0 client.14967 ms_handle_reset on v2:172.21.15.131:6836/1633006186 2024-05-09T00:46:05.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.989+0000 7f508bd7bb80 1 --2- 172.21.15.5:0/354848761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f08ade3cb0 0x55f08af0ec50 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:05.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.989+0000 7f508bd7bb80 1 -- 172.21.15.5:0/354848761 >> 172.21.15.5:0/354848761 conn(0x55f08ad6cf20 msgr2=0x55f08aeee820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:05.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.989+0000 7f508bd7bb80 1 -- 172.21.15.5:0/354848761 shutdown_connections 2024-05-09T00:46:05.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.989+0000 7f508bd7bb80 1 -- 172.21.15.5:0/354848761 wait complete. 2024-05-09T00:46:05.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.989+0000 7f508bd7bb80 20 client.14967 trim_cache size 0 max 16384 2024-05-09T00:46:05.997 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-09T00:46:05.997 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "d85814c6-0d9b-11ef-bc97-c7b262605968", 2024-05-09T00:46:05.997 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 4, 2024-05-09T00:46:05.997 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24467, 2024-05-09T00:46:05.997 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-05-09T00:46:05.997 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-05-09T00:46:05.997 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-05-09T00:46:05.997 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 124.043200171, 2024-05-09T00:46:05.997 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 28, 2024-05-09T00:46:05.997 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 82, 2024-05-09T00:46:05.997 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 78, 2024-05-09T00:46:05.997 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 167.829038161 2024-05-09T00:46:05.997 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-09T00:46:05.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.990+0000 7f507d365700 1 -- 172.21.15.5:0/3503570179 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f50640b2880 msgr2=0x7f50640b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:05.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.990+0000 7f507d365700 1 --2- 172.21.15.5:0/3503570179 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f50640b2880 0x7f50640b4d90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5068009990 tx=0x7f50680036e0 comp rx=0 tx=0).stop 2024-05-09T00:46:05.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.990+0000 7f507d365700 1 -- 172.21.15.5:0/3503570179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5078108740 msgr2=0x7f507819f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:05.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.990+0000 7f507d365700 1 --2- 172.21.15.5:0/3503570179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5078108740 0x7f507819f680 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5070007ee0 tx=0x7f507000d3b0 comp rx=0 tx=0).stop 2024-05-09T00:46:05.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.990+0000 7f507d365700 1 -- 172.21.15.5:0/3503570179 shutdown_connections 2024-05-09T00:46:05.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.990+0000 7f507d365700 1 --2- 172.21.15.5:0/3503570179 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f50640b2880 0x7f50640b4d90 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:05.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.990+0000 7f507d365700 1 --2- 172.21.15.5:0/3503570179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5078108740 0x7f507819f680 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:05.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.990+0000 7f507d365700 1 --2- 172.21.15.5:0/3503570179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50781034c0 0x7f507819ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:05.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.990+0000 7f507d365700 1 --2- 172.21.15.5:0/3503570179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5078107df0 0x7f507819b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:05.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.990+0000 7f507d365700 1 -- 172.21.15.5:0/3503570179 >> 172.21.15.5:0/3503570179 conn(0x7f50780fedf0 msgr2=0x7f50781063a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:05.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.990+0000 7f507d365700 1 -- 172.21.15.5:0/3503570179 shutdown_connections 2024-05-09T00:46:05.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:05.990+0000 7f507d365700 1 -- 172.21.15.5:0/3503570179 wait complete. 2024-05-09T00:46:06.706 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.f status 2024-05-09T00:46:06.963 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:46:07.365 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:46:07 smithi005 ceph-mon[30365]: pgmap v364: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-09T00:46:07.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:46:07 smithi131 ceph-mon[34547]: pgmap v364: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-09T00:46:07.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:46:07 smithi157 ceph-mon[34294]: pgmap v364: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-09T00:46:08.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.699+0000 7f53cf02f700 1 -- 172.21.15.5:0/3391525039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53c8108740 msgr2=0x7f53c81100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:08.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.699+0000 7f53cf02f700 1 --2- 172.21.15.5:0/3391525039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53c8108740 0x7f53c81100a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f53c000b0d0 tx=0x7f53c000b3e0 comp rx=0 tx=0).stop 2024-05-09T00:46:08.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.699+0000 7f53cf02f700 1 -- 172.21.15.5:0/3391525039 shutdown_connections 2024-05-09T00:46:08.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.699+0000 7f53cf02f700 1 --2- 172.21.15.5:0/3391525039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53c8108740 0x7f53c81100a0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:08.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.699+0000 7f53cf02f700 1 --2- 172.21.15.5:0/3391525039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53c81034c0 0x7f53c81038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:08.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.699+0000 7f53cf02f700 1 --2- 172.21.15.5:0/3391525039 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53c8107df0 0x7f53c8108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:08.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.699+0000 7f53cf02f700 1 -- 172.21.15.5:0/3391525039 >> 172.21.15.5:0/3391525039 conn(0x7f53c80fedf0 msgr2=0x7f53c8101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:08.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.699+0000 7f53cf02f700 1 -- 172.21.15.5:0/3391525039 shutdown_connections 2024-05-09T00:46:08.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.699+0000 7f53cf02f700 1 -- 172.21.15.5:0/3391525039 wait complete. 2024-05-09T00:46:08.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.700+0000 7f53cf02f700 1 Processor -- start 2024-05-09T00:46:08.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.700+0000 7f53cf02f700 1 -- start start 2024-05-09T00:46:08.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.700+0000 7f53cf02f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53c81034c0 0x7f53c819d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:08.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.700+0000 7f53cf02f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53c8107df0 0x7f53c819d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:08.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.700+0000 7f53cf02f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53c8108740 0x7f53c81a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:08.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.700+0000 7f53cf02f700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f53c81133e0 con 0x7f53c8107df0 2024-05-09T00:46:08.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.700+0000 7f53cf02f700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f53c8113260 con 0x7f53c81034c0 2024-05-09T00:46:08.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.700+0000 7f53cf02f700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f53c8113560 con 0x7f53c8108740 2024-05-09T00:46:08.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.700+0000 7f53c7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53c8107df0 0x7f53c819d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:08.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.700+0000 7f53c7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53c8107df0 0x7f53c819d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58806/0 (socket says 172.21.15.5:58806) 2024-05-09T00:46:08.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.700+0000 7f53c7fff700 1 -- 172.21.15.5:0/2541521453 learned_addr learned my addr 172.21.15.5:0/2541521453 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:46:08.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.701+0000 7f53cd5cc700 1 --2- 172.21.15.5:0/2541521453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53c8108740 0x7f53c81a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:08.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.701+0000 7f53ccdcb700 1 --2- 172.21.15.5:0/2541521453 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53c81034c0 0x7f53c819d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:08.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.701+0000 7f53c7fff700 1 -- 172.21.15.5:0/2541521453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53c8108740 msgr2=0x7f53c81a1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:08.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.701+0000 7f53c7fff700 1 --2- 172.21.15.5:0/2541521453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53c8108740 0x7f53c81a1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:08.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.701+0000 7f53c7fff700 1 -- 172.21.15.5:0/2541521453 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53c81034c0 msgr2=0x7f53c819d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:08.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.701+0000 7f53c7fff700 1 --2- 172.21.15.5:0/2541521453 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53c81034c0 0x7f53c819d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:08.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.701+0000 7f53c7fff700 1 -- 172.21.15.5:0/2541521453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53c81a1ef0 con 0x7f53c8107df0 2024-05-09T00:46:08.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.701+0000 7f53cd5cc700 1 --2- 172.21.15.5:0/2541521453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53c8108740 0x7f53c81a1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:46:08.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.701+0000 7f53ccdcb700 1 --2- 172.21.15.5:0/2541521453 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53c81034c0 0x7f53c819d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:46:08.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.701+0000 7f53c7fff700 1 --2- 172.21.15.5:0/2541521453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53c8107df0 0x7f53c819d5c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f53b400b500 tx=0x7f53b400b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:08.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.701+0000 7f53c5ffb700 1 -- 172.21.15.5:0/2541521453 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b4015020 con 0x7f53c8107df0 2024-05-09T00:46:08.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.701+0000 7f53cf02f700 1 -- 172.21.15.5:0/2541521453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53c81a21e0 con 0x7f53c8107df0 2024-05-09T00:46:08.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.701+0000 7f53c5ffb700 1 -- 172.21.15.5:0/2541521453 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53b4011ce0 con 0x7f53c8107df0 2024-05-09T00:46:08.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.701+0000 7f53c5ffb700 1 -- 172.21.15.5:0/2541521453 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b4010d40 con 0x7f53c8107df0 2024-05-09T00:46:08.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.701+0000 7f53cf02f700 1 -- 172.21.15.5:0/2541521453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53c81a9b00 con 0x7f53c8107df0 2024-05-09T00:46:08.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.704+0000 7f53c5ffb700 1 -- 172.21.15.5:0/2541521453 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f53b4018020 con 0x7f53c8107df0 2024-05-09T00:46:08.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.705+0000 7f53c5ffb700 1 --2- 172.21.15.5:0/2541521453 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f53b80b2880 0x7f53b80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:08.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.705+0000 7f53ccdcb700 1 --2- 172.21.15.5:0/2541521453 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f53b80b2880 0x7f53b80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:08.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.705+0000 7f53c5ffb700 1 -- 172.21.15.5:0/2541521453 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f53b40e1a90 con 0x7f53c8107df0 2024-05-09T00:46:08.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.705+0000 7f53ccdcb700 1 --2- 172.21.15.5:0/2541521453 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f53b80b2880 0x7f53b80b4d90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f53bc005950 tx=0x7f53bc005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:08.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.706+0000 7f53dda45b80 1 Processor -- start 2024-05-09T00:46:08.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.706+0000 7f53dda45b80 1 -- start start 2024-05-09T00:46:08.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.706+0000 7f53dda45b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568e5206110 0x5568e529f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:08.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.706+0000 7f53dda45b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568e511c450 0x5568e529f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:08.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.706+0000 7f53dda45b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5568e5299650 0x5568e5299a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:08.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.706+0000 7f53dda45b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5568e51bbfa0 con 0x5568e511c450 2024-05-09T00:46:08.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.706+0000 7f53dda45b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5568e515a7a0 con 0x5568e5299650 2024-05-09T00:46:08.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.706+0000 7f53dda45b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5568e51b7f60 con 0x5568e5206110 2024-05-09T00:46:08.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.706+0000 7f53c7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568e511c450 0x5568e529f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:08.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.706+0000 7f53c7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568e511c450 0x5568e529f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58820/0 (socket says 172.21.15.5:58820) 2024-05-09T00:46:08.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.706+0000 7f53c7fff700 1 -- 172.21.15.5:0/2913839246 learned_addr learned my addr 172.21.15.5:0/2913839246 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:46:08.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.706+0000 7f53ccdcb700 1 --2- 172.21.15.5:0/2913839246 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568e5206110 0x5568e529f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:08.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.707+0000 7f53cd5cc700 1 --2- 172.21.15.5:0/2913839246 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5568e5299650 0x5568e5299a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:08.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.707+0000 7f53c7fff700 1 -- 172.21.15.5:0/2913839246 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568e5206110 msgr2=0x5568e529f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:08.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.707+0000 7f53c7fff700 1 --2- 172.21.15.5:0/2913839246 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568e5206110 0x5568e529f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:08.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.707+0000 7f53c7fff700 1 -- 172.21.15.5:0/2913839246 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5568e5299650 msgr2=0x5568e5299a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:08.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.707+0000 7f53c7fff700 1 --2- 172.21.15.5:0/2913839246 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5568e5299650 0x5568e5299a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:08.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.707+0000 7f53c7fff700 1 -- 172.21.15.5:0/2913839246 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568e5152320 con 0x5568e511c450 2024-05-09T00:46:08.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.707+0000 7f53c7fff700 1 --2- 172.21.15.5:0/2913839246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568e511c450 0x5568e529f9e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f53b40a8d80 tx=0x7f53b4004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efd8bb52bc852581 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:08.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.707+0000 7f53b2ffd700 1 -- 172.21.15.5:0/2913839246 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b40a9780 con 0x5568e511c450 2024-05-09T00:46:08.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.707+0000 7f53b2ffd700 1 -- 172.21.15.5:0/2913839246 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53b40a9080 con 0x5568e511c450 2024-05-09T00:46:08.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.707+0000 7f53b2ffd700 1 -- 172.21.15.5:0/2913839246 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b40b1a50 con 0x5568e511c450 2024-05-09T00:46:08.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.708+0000 7f53dda45b80 1 -- 172.21.15.5:0/2913839246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568e511c450 msgr2=0x5568e529f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:08.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.708+0000 7f53dda45b80 1 --2- 172.21.15.5:0/2913839246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568e511c450 0x5568e529f9e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f53b40a8d80 tx=0x7f53b4004860 comp rx=0 tx=0).stop 2024-05-09T00:46:08.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.708+0000 7f53dda45b80 1 -- 172.21.15.5:0/2913839246 shutdown_connections 2024-05-09T00:46:08.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.708+0000 7f53dda45b80 1 --2- 172.21.15.5:0/2913839246 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5568e5299650 0x5568e5299a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:08.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.708+0000 7f53dda45b80 1 --2- 172.21.15.5:0/2913839246 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568e5206110 0x5568e529f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:08.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.708+0000 7f53dda45b80 1 --2- 172.21.15.5:0/2913839246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568e511c450 0x5568e529f9e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:08.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.708+0000 7f53dda45b80 1 -- 172.21.15.5:0/2913839246 >> 172.21.15.5:0/2913839246 conn(0x5568e517c650 msgr2=0x5568e52943b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:08.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.708+0000 7f53dda45b80 1 -- 172.21.15.5:0/2913839246 shutdown_connections 2024-05-09T00:46:08.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.708+0000 7f53dda45b80 1 -- 172.21.15.5:0/2913839246 wait complete. 2024-05-09T00:46:08.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.708+0000 7f53dda45b80 1 -- start start 2024-05-09T00:46:08.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.709+0000 7f53dda45b80 1 Processor -- start 2024-05-09T00:46:08.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.709+0000 7f53dda45b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568e511c450 0x5568e52a3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:08.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.709+0000 7f53dda45b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568e5206110 0x5568e52a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:08.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.709+0000 7f53dda45b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5568e5299650 0x5568e52a3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:08.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.709+0000 7f53dda45b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5568e5037b50 con 0x5568e511c450 2024-05-09T00:46:08.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.709+0000 7f53dda45b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5568e5176100 con 0x5568e5299650 2024-05-09T00:46:08.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.709+0000 7f53dda45b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5568e529a490 con 0x5568e5206110 2024-05-09T00:46:08.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.709+0000 7f53dda45b80 10 client.0 _ll_register_callbacks cb 0x5568e502d670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:46:08.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.709+0000 7f53ccdcb700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568e511c450 0x5568e52a3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:08.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.709+0000 7f53ccdcb700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568e511c450 0x5568e52a3270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58826/0 (socket says 172.21.15.5:58826) 2024-05-09T00:46:08.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.709+0000 7f53ccdcb700 1 -- 172.21.15.5:0/2167080358 learned_addr learned my addr 172.21.15.5:0/2167080358 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:46:08.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.709+0000 7f53cd5cc700 1 --2- 172.21.15.5:0/2167080358 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5568e5299650 0x5568e52a3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:08.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.709+0000 7f53c7fff700 1 --2- 172.21.15.5:0/2167080358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568e5206110 0x5568e52a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:08.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.710+0000 7f53ccdcb700 1 -- 172.21.15.5:0/2167080358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568e5206110 msgr2=0x5568e52a37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:08.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.710+0000 7f53ccdcb700 1 --2- 172.21.15.5:0/2167080358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568e5206110 0x5568e52a37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:08.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.710+0000 7f53ccdcb700 1 -- 172.21.15.5:0/2167080358 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5568e5299650 msgr2=0x5568e52a3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:08.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.710+0000 7f53ccdcb700 1 --2- 172.21.15.5:0/2167080358 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5568e5299650 0x5568e52a3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:08.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.710+0000 7f53ccdcb700 1 -- 172.21.15.5:0/2167080358 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568e5037440 con 0x5568e511c450 2024-05-09T00:46:08.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.710+0000 7f53ccdcb700 1 --2- 172.21.15.5:0/2167080358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568e511c450 0x5568e52a3270 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f53bc00aa10 tx=0x7f53bc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d9acfaa4705ad66 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:08.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.710+0000 7f53a7fff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:46:08.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.710+0000 7f53a7fff700 1 -- 172.21.15.5:0/2167080358 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53bc00fa50 con 0x5568e511c450 2024-05-09T00:46:08.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.710+0000 7f53dda45b80 1 -- 172.21.15.5:0/2167080358 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53b40a9080 con 0x5568e511c450 2024-05-09T00:46:08.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.710+0000 7f53a7fff700 1 -- 172.21.15.5:0/2167080358 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f53bc00fbf0 con 0x5568e511c450 2024-05-09T00:46:08.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.710+0000 7f53a7fff700 1 -- 172.21.15.5:0/2167080358 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f53bc00fdf0 con 0x5568e511c450 2024-05-09T00:46:08.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.710+0000 7f53dda45b80 10 client.14979 fetch_fsmap learned FSMap version 28 2024-05-09T00:46:08.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.710+0000 7f53dda45b80 1 -- 172.21.15.5:0/2167080358 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5568e52b0be0 con 0x5568e511c450 2024-05-09T00:46:08.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.710+0000 7f53a7fff700 1 -- 172.21.15.5:0/2167080358 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53b40a9080 con 0x5568e511c450 2024-05-09T00:46:08.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.711+0000 7f53a7fff700 1 -- 172.21.15.5:0/2167080358 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f53bc02f070 con 0x5568e511c450 2024-05-09T00:46:08.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.711+0000 7f53dda45b80 10 client.14979 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T00:46:08.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.711+0000 7f53dda45b80 10 client.14979 resolve_mds: resolved name 'f' to daemon mds.f 2024-05-09T00:46:08.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.711+0000 7f53dda45b80 20 client.14979 populate_metadata read hostname 'smithi005' 2024-05-09T00:46:08.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.711+0000 7f53dda45b80 1 --2- 172.21.15.5:0/2167080358 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x5568e52b3220 0x5568e52b4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:08.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.711+0000 7f53dda45b80 4 client.14979 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:46:08.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.711+0000 7f53dda45b80 1 -- 172.21.15.5:0/2167080358 --> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568e51c9980 con 0x5568e52b3220 2024-05-09T00:46:08.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.711+0000 7f53c7fff700 1 --2- 172.21.15.5:0/2167080358 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x5568e52b3220 0x5568e52b4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:08.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.712+0000 7f53a7fff700 10 client.14979 ms_handle_connect on v2:172.21.15.157:6834/832898273 2024-05-09T00:46:08.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.712+0000 7f53c7fff700 1 --2- 172.21.15.5:0/2167080358 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x5568e52b3220 0x5568e52b4690 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8db8a5d4743e09d7 server_cookie=9faf6a336c0f7cba in_seq=0 out_seq=0 2024-05-09T00:46:08.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.712+0000 7f53a7fff700 1 -- 172.21.15.5:0/2167080358 <== mds.3 v2:172.21.15.157:6834/832898273 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5568e51c9980 con 0x5568e52b3220 2024-05-09T00:46:08.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.712+0000 7f53a7fff700 10 client.14979 handle_command_reply: tid=0 2024-05-09T00:46:08.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.712+0000 7f53dda45b80 1 client.14979 shutdown 2024-05-09T00:46:08.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.713+0000 7f53dda45b80 1 -- 172.21.15.5:0/2167080358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568e511c450 msgr2=0x5568e52a3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:08.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.713+0000 7f53dda45b80 1 --2- 172.21.15.5:0/2167080358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568e511c450 0x5568e52a3270 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f53bc00aa10 tx=0x7f53bc004750 comp rx=0 tx=0).stop 2024-05-09T00:46:08.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.713+0000 7f53dda45b80 1 -- 172.21.15.5:0/2167080358 shutdown_connections 2024-05-09T00:46:08.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.713+0000 7f53dda45b80 1 --2- 172.21.15.5:0/2167080358 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x5568e52b3220 0x5568e52b4690 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:08.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.713+0000 7f53a7fff700 0 client.14979 ms_handle_reset on v2:172.21.15.157:6834/832898273 2024-05-09T00:46:08.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.713+0000 7f53dda45b80 1 --2- 172.21.15.5:0/2167080358 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5568e5299650 0x5568e52a3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:08.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.713+0000 7f53dda45b80 1 --2- 172.21.15.5:0/2167080358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568e511c450 0x5568e52a3270 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:08.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.713+0000 7f53dda45b80 1 --2- 172.21.15.5:0/2167080358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568e5206110 0x5568e52a37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:08.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.713+0000 7f53dda45b80 1 -- 172.21.15.5:0/2167080358 >> 172.21.15.5:0/2167080358 conn(0x5568e517c650 msgr2=0x5568e5294170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:08.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.713+0000 7f53dda45b80 1 -- 172.21.15.5:0/2167080358 shutdown_connections 2024-05-09T00:46:08.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.713+0000 7f53dda45b80 1 -- 172.21.15.5:0/2167080358 wait complete. 2024-05-09T00:46:08.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.713+0000 7f53dda45b80 20 client.14979 trim_cache size 0 max 16384 2024-05-09T00:46:08.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.734+0000 7f53dda45b80 1 Processor -- start 2024-05-09T00:46:08.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.734+0000 7f53dda45b80 1 -- start start 2024-05-09T00:46:08.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.734+0000 7f53dda45b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568e511c450 0x5568e52a1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:08.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.734+0000 7f53dda45b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5568e5206110 0x5568e52a20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:08.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.734+0000 7f53dda45b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568e5299650 0x5568e5298640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:08.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.734+0000 7f53dda45b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5568e52b29d0 con 0x5568e511c450 2024-05-09T00:46:08.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.734+0000 7f53dda45b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5568e52b30a0 con 0x5568e5206110 2024-05-09T00:46:08.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.734+0000 7f53dda45b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5568e52b3770 con 0x5568e5299650 2024-05-09T00:46:08.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.734+0000 7f53ccdcb700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568e511c450 0x5568e52a1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:08.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.735+0000 7f53ccdcb700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568e511c450 0x5568e52a1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58832/0 (socket says 172.21.15.5:58832) 2024-05-09T00:46:08.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.735+0000 7f53ccdcb700 1 -- 172.21.15.5:0/1851779415 learned_addr learned my addr 172.21.15.5:0/1851779415 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:46:08.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.735+0000 7f53cd5cc700 1 --2- 172.21.15.5:0/1851779415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568e5299650 0x5568e5298640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:08.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.735+0000 7f53c7fff700 1 --2- 172.21.15.5:0/1851779415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5568e5206110 0x5568e52a20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:08.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.735+0000 7f53ccdcb700 1 -- 172.21.15.5:0/1851779415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568e5299650 msgr2=0x5568e5298640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:08.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.735+0000 7f53ccdcb700 1 --2- 172.21.15.5:0/1851779415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568e5299650 0x5568e5298640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:08.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.735+0000 7f53ccdcb700 1 -- 172.21.15.5:0/1851779415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5568e5206110 msgr2=0x5568e52a20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:08.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.735+0000 7f53ccdcb700 1 --2- 172.21.15.5:0/1851779415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5568e5206110 0x5568e52a20a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:08.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.735+0000 7f53ccdcb700 1 -- 172.21.15.5:0/1851779415 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568e529f000 con 0x5568e511c450 2024-05-09T00:46:08.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.735+0000 7f53ccdcb700 1 --2- 172.21.15.5:0/1851779415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568e511c450 0x5568e52a1b60 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f53bc000c00 tx=0x7f53bc01e5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e400f73066cb0ec server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:08.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.735+0000 7f53b27fc700 1 -- 172.21.15.5:0/1851779415 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53bc005000 con 0x5568e511c450 2024-05-09T00:46:08.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.735+0000 7f53b27fc700 1 -- 172.21.15.5:0/1851779415 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53bc02c070 con 0x5568e511c450 2024-05-09T00:46:08.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.735+0000 7f53b27fc700 1 -- 172.21.15.5:0/1851779415 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53bc004bb0 con 0x5568e511c450 2024-05-09T00:46:08.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.736+0000 7f53dda45b80 1 -- 172.21.15.5:0/1851779415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568e511c450 msgr2=0x5568e52a1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:08.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.736+0000 7f53dda45b80 1 --2- 172.21.15.5:0/1851779415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568e511c450 0x5568e52a1b60 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f53bc000c00 tx=0x7f53bc01e5c0 comp rx=0 tx=0).stop 2024-05-09T00:46:08.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.736+0000 7f53dda45b80 1 -- 172.21.15.5:0/1851779415 shutdown_connections 2024-05-09T00:46:08.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.736+0000 7f53dda45b80 1 --2- 172.21.15.5:0/1851779415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568e5299650 0x5568e5298640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:08.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.736+0000 7f53dda45b80 1 --2- 172.21.15.5:0/1851779415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568e511c450 0x5568e52a1b60 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:08.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.736+0000 7f53dda45b80 1 --2- 172.21.15.5:0/1851779415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5568e5206110 0x5568e52a20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:08.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.736+0000 7f53dda45b80 1 -- 172.21.15.5:0/1851779415 >> 172.21.15.5:0/1851779415 conn(0x5568e517c650 msgr2=0x5568e529e310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:08.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.736+0000 7f53dda45b80 1 -- 172.21.15.5:0/1851779415 shutdown_connections 2024-05-09T00:46:08.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.736+0000 7f53dda45b80 1 -- 172.21.15.5:0/1851779415 wait complete. 2024-05-09T00:46:08.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.736+0000 7f53dda45b80 1 -- start start 2024-05-09T00:46:08.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.737+0000 7f53dda45b80 1 Processor -- start 2024-05-09T00:46:08.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.737+0000 7f53dda45b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568e511c450 0x5568e52a4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:08.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.737+0000 7f53dda45b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568e5206110 0x5568e52a5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:08.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.737+0000 7f53dda45b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5568e5299650 0x5568e52a04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:08.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.737+0000 7f53dda45b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5568e52b4510 con 0x5568e5206110 2024-05-09T00:46:08.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.737+0000 7f53dda45b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5568e52b3e40 con 0x5568e5299650 2024-05-09T00:46:08.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.737+0000 7f53dda45b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5568e52b4be0 con 0x5568e511c450 2024-05-09T00:46:08.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.737+0000 7f53dda45b80 10 client.0 _ll_register_callbacks cb 0x5568e502d670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:46:08.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.737+0000 7f53c7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568e5206110 0x5568e52a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:08.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.737+0000 7f53c7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568e5206110 0x5568e52a5100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58840/0 (socket says 172.21.15.5:58840) 2024-05-09T00:46:08.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.737+0000 7f53c7fff700 1 -- 172.21.15.5:0/1613676738 learned_addr learned my addr 172.21.15.5:0/1613676738 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:46:08.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.737+0000 7f53cd5cc700 1 --2- 172.21.15.5:0/1613676738 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5568e5299650 0x5568e52a04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:08.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.737+0000 7f53ccdcb700 1 --2- 172.21.15.5:0/1613676738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568e511c450 0x5568e52a4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:08.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.738+0000 7f53c7fff700 1 -- 172.21.15.5:0/1613676738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568e511c450 msgr2=0x5568e52a4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:08.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.738+0000 7f53c7fff700 1 --2- 172.21.15.5:0/1613676738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568e511c450 0x5568e52a4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:08.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.738+0000 7f53c7fff700 1 -- 172.21.15.5:0/1613676738 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5568e5299650 msgr2=0x5568e52a04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:08.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.738+0000 7f53c7fff700 1 --2- 172.21.15.5:0/1613676738 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5568e5299650 0x5568e52a04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:08.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.738+0000 7f53c7fff700 1 -- 172.21.15.5:0/1613676738 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568e52a58f0 con 0x5568e5206110 2024-05-09T00:46:08.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.738+0000 7f53c7fff700 1 --2- 172.21.15.5:0/1613676738 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568e5206110 0x5568e52a5100 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f53b40a5100 tx=0x7f53b4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6576cb239e4b9315 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:08.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.738+0000 7f53a7fff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:46:08.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.738+0000 7f53a7fff700 1 -- 172.21.15.5:0/1613676738 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b40c7030 con 0x5568e5206110 2024-05-09T00:46:08.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.738+0000 7f53a7fff700 1 -- 172.21.15.5:0/1613676738 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f53b40a5670 con 0x5568e5206110 2024-05-09T00:46:08.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.738+0000 7f53dda45b80 1 -- 172.21.15.5:0/1613676738 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53bc02c070 con 0x5568e5206110 2024-05-09T00:46:08.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.738+0000 7f53a7fff700 1 -- 172.21.15.5:0/1613676738 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f53b40c2070 con 0x5568e5206110 2024-05-09T00:46:08.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.738+0000 7f53dda45b80 10 client.14985 fetch_fsmap learned FSMap version 28 2024-05-09T00:46:08.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.738+0000 7f53dda45b80 1 -- 172.21.15.5:0/1613676738 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5568e531ec00 con 0x5568e5206110 2024-05-09T00:46:08.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.738+0000 7f53a7fff700 1 -- 172.21.15.5:0/1613676738 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53bc02c070 con 0x5568e5206110 2024-05-09T00:46:08.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.738+0000 7f53a7fff700 1 -- 172.21.15.5:0/1613676738 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f53b40b1a50 con 0x5568e5206110 2024-05-09T00:46:08.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.738+0000 7f53dda45b80 10 client.14985 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T00:46:08.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.739+0000 7f53dda45b80 10 client.14985 resolve_mds: resolved name 'f' to daemon mds.f 2024-05-09T00:46:08.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.739+0000 7f53dda45b80 20 client.14985 populate_metadata read hostname 'smithi005' 2024-05-09T00:46:08.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.739+0000 7f53dda45b80 1 --2- 172.21.15.5:0/1613676738 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x5568e52b3220 0x5568e5321f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:08.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.739+0000 7f53dda45b80 4 client.14985 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T00:46:08.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.739+0000 7f53dda45b80 1 -- 172.21.15.5:0/1613676738 --> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] -- command(tid 0: {"prefix": "status"}) -- 0x5568e51c3a20 con 0x5568e52b3220 2024-05-09T00:46:08.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.739+0000 7f53ccdcb700 1 --2- 172.21.15.5:0/1613676738 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x5568e52b3220 0x5568e5321f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:08.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.739+0000 7f53a7fff700 10 client.14985 ms_handle_connect on v2:172.21.15.157:6834/832898273 2024-05-09T00:46:08.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.739+0000 7f53ccdcb700 1 --2- 172.21.15.5:0/1613676738 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x5568e52b3220 0x5568e5321f80 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b4890f4cd27599a2 server_cookie=bd07b0af93a5f1f7 in_seq=0 out_seq=0 2024-05-09T00:46:08.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.739+0000 7f53a7fff700 1 -- 172.21.15.5:0/1613676738 <== mds.3 v2:172.21.15.157:6834/832898273 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x5568e51c3a20 con 0x5568e52b3220 2024-05-09T00:46:08.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.739+0000 7f53a7fff700 10 client.14985 handle_command_reply: tid=0 2024-05-09T00:46:08.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.739+0000 7f53dda45b80 1 client.14985 shutdown 2024-05-09T00:46:08.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.740+0000 7f53dda45b80 1 -- 172.21.15.5:0/1613676738 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568e5206110 msgr2=0x5568e52a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:08.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.740+0000 7f53dda45b80 1 --2- 172.21.15.5:0/1613676738 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568e5206110 0x5568e52a5100 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f53b40a5100 tx=0x7f53b4004750 comp rx=0 tx=0).stop 2024-05-09T00:46:08.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.740+0000 7f53dda45b80 1 -- 172.21.15.5:0/1613676738 shutdown_connections 2024-05-09T00:46:08.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.740+0000 7f53dda45b80 1 --2- 172.21.15.5:0/1613676738 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x5568e52b3220 0x5568e5321f80 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:08.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.740+0000 7f53dda45b80 1 --2- 172.21.15.5:0/1613676738 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5568e5299650 0x5568e52a04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:08.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.740+0000 7f53dda45b80 1 --2- 172.21.15.5:0/1613676738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5568e511c450 0x5568e52a4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:08.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.740+0000 7f53a7fff700 0 client.14985 ms_handle_reset on v2:172.21.15.157:6834/832898273 2024-05-09T00:46:08.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.740+0000 7f53dda45b80 1 --2- 172.21.15.5:0/1613676738 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568e5206110 0x5568e52a5100 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:08.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.740+0000 7f53dda45b80 1 -- 172.21.15.5:0/1613676738 >> 172.21.15.5:0/1613676738 conn(0x5568e517c650 msgr2=0x5568e52ae250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:08.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.740+0000 7f53dda45b80 1 -- 172.21.15.5:0/1613676738 shutdown_connections 2024-05-09T00:46:08.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.740+0000 7f53dda45b80 1 -- 172.21.15.5:0/1613676738 wait complete. 2024-05-09T00:46:08.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.740+0000 7f53dda45b80 20 client.14985 trim_cache size 0 max 16384 2024-05-09T00:46:08.745 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-09T00:46:08.746 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "d85814c6-0d9b-11ef-bc97-c7b262605968", 2024-05-09T00:46:08.746 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 3, 2024-05-09T00:46:08.746 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24493, 2024-05-09T00:46:08.746 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-05-09T00:46:08.746 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-05-09T00:46:08.746 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-05-09T00:46:08.746 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 127.80189288699999, 2024-05-09T00:46:08.746 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 28, 2024-05-09T00:46:08.746 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 82, 2024-05-09T00:46:08.746 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 78, 2024-05-09T00:46:08.746 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 176.31235973 2024-05-09T00:46:08.746 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-09T00:46:08.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.741+0000 7f53cf02f700 1 -- 172.21.15.5:0/2541521453 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f53b80b2880 msgr2=0x7f53b80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:08.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.741+0000 7f53cf02f700 1 --2- 172.21.15.5:0/2541521453 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f53b80b2880 0x7f53b80b4d90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f53bc005950 tx=0x7f53bc005f50 comp rx=0 tx=0).stop 2024-05-09T00:46:08.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.741+0000 7f53cf02f700 1 -- 172.21.15.5:0/2541521453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53c8107df0 msgr2=0x7f53c819d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:08.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.741+0000 7f53cf02f700 1 --2- 172.21.15.5:0/2541521453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53c8107df0 0x7f53c819d5c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f53b400b500 tx=0x7f53b400b8c0 comp rx=0 tx=0).stop 2024-05-09T00:46:08.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.741+0000 7f53cf02f700 1 -- 172.21.15.5:0/2541521453 shutdown_connections 2024-05-09T00:46:08.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.741+0000 7f53cf02f700 1 --2- 172.21.15.5:0/2541521453 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f53b80b2880 0x7f53b80b4d90 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:08.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.741+0000 7f53cf02f700 1 --2- 172.21.15.5:0/2541521453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53c8108740 0x7f53c81a1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:08.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.741+0000 7f53cf02f700 1 --2- 172.21.15.5:0/2541521453 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53c81034c0 0x7f53c819d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:08.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.741+0000 7f53cf02f700 1 --2- 172.21.15.5:0/2541521453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53c8107df0 0x7f53c819d5c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:08.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.741+0000 7f53cf02f700 1 -- 172.21.15.5:0/2541521453 >> 172.21.15.5:0/2541521453 conn(0x7f53c80fedf0 msgr2=0x7f53c81096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:08.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.741+0000 7f53cf02f700 1 -- 172.21.15.5:0/2541521453 shutdown_connections 2024-05-09T00:46:08.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:08.741+0000 7f53cf02f700 1 -- 172.21.15.5:0/2541521453 wait complete. 2024-05-09T00:46:09.438 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:46:09 smithi005 ceph-mon[30365]: pgmap v365: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-09T00:46:09.468 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.b status 2024-05-09T00:46:09.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:46:09 smithi131 ceph-mon[34547]: pgmap v365: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-09T00:46:09.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:46:09 smithi157 ceph-mon[34294]: pgmap v365: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-09T00:46:09.724 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:46:11.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.522+0000 7f4c154bd700 1 -- 172.21.15.5:0/671624117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c10108740 msgr2=0x7f4c101100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:11.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.522+0000 7f4c154bd700 1 --2- 172.21.15.5:0/671624117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c10108740 0x7f4c101100a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4c0800b0d0 tx=0x7f4c0800b3e0 comp rx=0 tx=0).stop 2024-05-09T00:46:11.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.522+0000 7f4c154bd700 1 -- 172.21.15.5:0/671624117 shutdown_connections 2024-05-09T00:46:11.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.522+0000 7f4c154bd700 1 --2- 172.21.15.5:0/671624117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c10108740 0x7f4c101100a0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:11.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.522+0000 7f4c154bd700 1 --2- 172.21.15.5:0/671624117 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c101034c0 0x7f4c101038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:11.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.522+0000 7f4c154bd700 1 --2- 172.21.15.5:0/671624117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c10107df0 0x7f4c10108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:11.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.522+0000 7f4c154bd700 1 -- 172.21.15.5:0/671624117 >> 172.21.15.5:0/671624117 conn(0x7f4c100fedf0 msgr2=0x7f4c10101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:11.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.522+0000 7f4c154bd700 1 -- 172.21.15.5:0/671624117 shutdown_connections 2024-05-09T00:46:11.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.522+0000 7f4c154bd700 1 -- 172.21.15.5:0/671624117 wait complete. 2024-05-09T00:46:11.525 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:46:11 smithi005 ceph-mon[30365]: pgmap v366: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-09T00:46:11.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.523+0000 7f4c154bd700 1 Processor -- start 2024-05-09T00:46:11.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.524+0000 7f4c154bd700 1 -- start start 2024-05-09T00:46:11.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.524+0000 7f4c154bd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c101034c0 0x7f4c1019bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:11.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.524+0000 7f4c154bd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c10107df0 0x7f4c1019c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:11.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.524+0000 7f4c154bd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c10108740 0x7f4c1019c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:11.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.524+0000 7f4c154bd700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4c101133c0 con 0x7f4c10108740 2024-05-09T00:46:11.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.524+0000 7f4c154bd700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4c10113240 con 0x7f4c101034c0 2024-05-09T00:46:11.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.524+0000 7f4c154bd700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4c10113540 con 0x7f4c10107df0 2024-05-09T00:46:11.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.524+0000 7f4c0f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c10108740 0x7f4c1019c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:11.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.524+0000 7f4c0f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c10108740 0x7f4c1019c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58856/0 (socket says 172.21.15.5:58856) 2024-05-09T00:46:11.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.524+0000 7f4c0f7fe700 1 -- 172.21.15.5:0/255526687 learned_addr learned my addr 172.21.15.5:0/255526687 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:46:11.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.524+0000 7f4c0e7fc700 1 --2- 172.21.15.5:0/255526687 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c10107df0 0x7f4c1019c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:11.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.524+0000 7f4c0effd700 1 --2- 172.21.15.5:0/255526687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c101034c0 0x7f4c1019bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:11.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.525+0000 7f4c0f7fe700 1 -- 172.21.15.5:0/255526687 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c10107df0 msgr2=0x7f4c1019c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:11.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.525+0000 7f4c0f7fe700 1 --2- 172.21.15.5:0/255526687 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c10107df0 0x7f4c1019c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:11.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.525+0000 7f4c0f7fe700 1 -- 172.21.15.5:0/255526687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c101034c0 msgr2=0x7f4c1019bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:11.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.525+0000 7f4c0f7fe700 1 --2- 172.21.15.5:0/255526687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c101034c0 0x7f4c1019bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:11.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.525+0000 7f4c0f7fe700 1 -- 172.21.15.5:0/255526687 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c101a7010 con 0x7f4c10108740 2024-05-09T00:46:11.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.525+0000 7f4c0effd700 1 --2- 172.21.15.5:0/255526687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c101034c0 0x7f4c1019bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:46:11.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.525+0000 7f4c0f7fe700 1 --2- 172.21.15.5:0/255526687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c10108740 0x7f4c1019c760 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4c0800d010 tx=0x7f4c080079a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:11.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.525+0000 7f4c07fff700 1 -- 172.21.15.5:0/255526687 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c0801f070 con 0x7f4c10108740 2024-05-09T00:46:11.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.525+0000 7f4c07fff700 1 -- 172.21.15.5:0/255526687 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c08003c00 con 0x7f4c10108740 2024-05-09T00:46:11.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.525+0000 7f4c154bd700 1 -- 172.21.15.5:0/255526687 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c101a72a0 con 0x7f4c10108740 2024-05-09T00:46:11.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.525+0000 7f4c07fff700 1 -- 172.21.15.5:0/255526687 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c0801b6a0 con 0x7f4c10108740 2024-05-09T00:46:11.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.525+0000 7f4c154bd700 1 -- 172.21.15.5:0/255526687 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c101a76a0 con 0x7f4c10108740 2024-05-09T00:46:11.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.528+0000 7f4c07fff700 1 -- 172.21.15.5:0/255526687 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f4c08022070 con 0x7f4c10108740 2024-05-09T00:46:11.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.528+0000 7f4c07fff700 1 --2- 172.21.15.5:0/255526687 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f4bfc0b2880 0x7f4bfc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:11.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.529+0000 7f4c0effd700 1 --2- 172.21.15.5:0/255526687 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f4bfc0b2880 0x7f4bfc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:11.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.529+0000 7f4c07fff700 1 -- 172.21.15.5:0/255526687 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f4c080e2ff0 con 0x7f4c10108740 2024-05-09T00:46:11.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.529+0000 7f4c23ed3b80 1 Processor -- start 2024-05-09T00:46:11.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.529+0000 7f4c23ed3b80 1 -- start start 2024-05-09T00:46:11.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.529+0000 7f4c0effd700 1 --2- 172.21.15.5:0/255526687 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f4bfc0b2880 0x7f4bfc0b4d90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8005950 tx=0x7f4bf80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:11.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.529+0000 7f4c23ed3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d7696110 0x5587d772f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:11.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.529+0000 7f4c23ed3b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5587d75ac450 0x5587d772f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:11.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.529+0000 7f4c23ed3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587d7729650 0x5587d7729a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:11.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.529+0000 7f4c23ed3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5587d764bfa0 con 0x5587d7729650 2024-05-09T00:46:11.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.529+0000 7f4c23ed3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5587d75ea7a0 con 0x5587d7696110 2024-05-09T00:46:11.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.529+0000 7f4c23ed3b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5587d7647f60 con 0x5587d75ac450 2024-05-09T00:46:11.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.529+0000 7f4c0f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587d7729650 0x5587d7729a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:11.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.529+0000 7f4c0f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587d7729650 0x5587d7729a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58870/0 (socket says 172.21.15.5:58870) 2024-05-09T00:46:11.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.530+0000 7f4c0f7fe700 1 -- 172.21.15.5:0/57565819 learned_addr learned my addr 172.21.15.5:0/57565819 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:46:11.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.530+0000 7f4c0e7fc700 1 --2- 172.21.15.5:0/57565819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5587d75ac450 0x5587d772f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:11.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.530+0000 7f4c0effd700 1 --2- 172.21.15.5:0/57565819 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d7696110 0x5587d772f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:11.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.530+0000 7f4c0f7fe700 1 -- 172.21.15.5:0/57565819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5587d75ac450 msgr2=0x5587d772f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:11.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.530+0000 7f4c0f7fe700 1 --2- 172.21.15.5:0/57565819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5587d75ac450 0x5587d772f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:11.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.530+0000 7f4c0f7fe700 1 -- 172.21.15.5:0/57565819 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d7696110 msgr2=0x5587d772f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:11.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.530+0000 7f4c0f7fe700 1 --2- 172.21.15.5:0/57565819 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d7696110 0x5587d772f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:11.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.530+0000 7f4c0f7fe700 1 -- 172.21.15.5:0/57565819 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587d75e2320 con 0x5587d7729650 2024-05-09T00:46:11.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.530+0000 7f4c0e7fc700 1 --2- 172.21.15.5:0/57565819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5587d75ac450 0x5587d772f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:46:11.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.530+0000 7f4c0f7fe700 1 --2- 172.21.15.5:0/57565819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587d7729650 0x5587d7729a80 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4c080a78e0 tx=0x7f4c080099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5412ccf5ec527df2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:11.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.530+0000 7f4c057fa700 1 -- 172.21.15.5:0/57565819 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c08029070 con 0x5587d7729650 2024-05-09T00:46:11.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.530+0000 7f4c057fa700 1 -- 172.21.15.5:0/57565819 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c080aad70 con 0x5587d7729650 2024-05-09T00:46:11.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.530+0000 7f4c057fa700 1 -- 172.21.15.5:0/57565819 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c08090be0 con 0x5587d7729650 2024-05-09T00:46:11.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.531+0000 7f4c23ed3b80 1 -- 172.21.15.5:0/57565819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587d7729650 msgr2=0x5587d7729a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:11.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.531+0000 7f4c23ed3b80 1 --2- 172.21.15.5:0/57565819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587d7729650 0x5587d7729a80 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4c080a78e0 tx=0x7f4c080099e0 comp rx=0 tx=0).stop 2024-05-09T00:46:11.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.531+0000 7f4c23ed3b80 1 -- 172.21.15.5:0/57565819 shutdown_connections 2024-05-09T00:46:11.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.531+0000 7f4c23ed3b80 1 --2- 172.21.15.5:0/57565819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587d7729650 0x5587d7729a80 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:11.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.531+0000 7f4c23ed3b80 1 --2- 172.21.15.5:0/57565819 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d7696110 0x5587d772f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:11.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.531+0000 7f4c23ed3b80 1 --2- 172.21.15.5:0/57565819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5587d75ac450 0x5587d772f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:11.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.531+0000 7f4c23ed3b80 1 -- 172.21.15.5:0/57565819 >> 172.21.15.5:0/57565819 conn(0x5587d760c650 msgr2=0x5587d77243b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:11.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.531+0000 7f4c23ed3b80 1 -- 172.21.15.5:0/57565819 shutdown_connections 2024-05-09T00:46:11.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.531+0000 7f4c23ed3b80 1 -- 172.21.15.5:0/57565819 wait complete. 2024-05-09T00:46:11.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.532+0000 7f4c23ed3b80 1 -- start start 2024-05-09T00:46:11.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.532+0000 7f4c23ed3b80 1 Processor -- start 2024-05-09T00:46:11.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.532+0000 7f4c23ed3b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5587d75ac450 0x5587d7738cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:11.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.532+0000 7f4c23ed3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587d7696110 0x5587d7739210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:11.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.532+0000 7f4c23ed3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d7729650 0x5587d77330d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:11.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.532+0000 7f4c23ed3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5587d74c7b50 con 0x5587d7696110 2024-05-09T00:46:11.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.532+0000 7f4c23ed3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5587d7606100 con 0x5587d7729650 2024-05-09T00:46:11.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.532+0000 7f4c23ed3b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5587d772a310 con 0x5587d75ac450 2024-05-09T00:46:11.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.532+0000 7f4c23ed3b80 10 client.0 _ll_register_callbacks cb 0x5587d74bd670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:46:11.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.532+0000 7f4c0e7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587d7696110 0x5587d7739210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:11.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.532+0000 7f4c0e7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587d7696110 0x5587d7739210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58886/0 (socket says 172.21.15.5:58886) 2024-05-09T00:46:11.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.532+0000 7f4c0e7fc700 1 -- 172.21.15.5:0/933917787 learned_addr learned my addr 172.21.15.5:0/933917787 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:46:11.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.532+0000 7f4c0effd700 1 --2- 172.21.15.5:0/933917787 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5587d75ac450 0x5587d7738cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:11.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.533+0000 7f4c0e7fc700 1 -- 172.21.15.5:0/933917787 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5587d75ac450 msgr2=0x5587d7738cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:11.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.533+0000 7f4c0e7fc700 1 --2- 172.21.15.5:0/933917787 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5587d75ac450 0x5587d7738cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:11.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.533+0000 7f4c0f7fe700 1 --2- 172.21.15.5:0/933917787 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d7729650 0x5587d77330d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:11.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.533+0000 7f4c0e7fc700 1 -- 172.21.15.5:0/933917787 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d7729650 msgr2=0x5587d77330d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:11.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.533+0000 7f4c0e7fc700 1 --2- 172.21.15.5:0/933917787 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d7729650 0x5587d77330d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:11.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.533+0000 7f4c0e7fc700 1 -- 172.21.15.5:0/933917787 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587d74c7440 con 0x5587d7696110 2024-05-09T00:46:11.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.533+0000 7f4c0e7fc700 1 --2- 172.21.15.5:0/933917787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587d7696110 0x5587d7739210 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4c00007ec0 tx=0x7f4c00012730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e98bf30c1fe1aac server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:11.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.533+0000 7f4bf67fc700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:46:11.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.533+0000 7f4bf67fc700 1 -- 172.21.15.5:0/933917787 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c00004520 con 0x5587d7696110 2024-05-09T00:46:11.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.533+0000 7f4bf67fc700 1 -- 172.21.15.5:0/933917787 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f4c00003680 con 0x5587d7696110 2024-05-09T00:46:11.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.533+0000 7f4c23ed3b80 1 -- 172.21.15.5:0/933917787 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c080aad70 con 0x5587d7696110 2024-05-09T00:46:11.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.533+0000 7f4bf67fc700 1 -- 172.21.15.5:0/933917787 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4c00021030 con 0x5587d7696110 2024-05-09T00:46:11.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.533+0000 7f4c23ed3b80 10 client.14997 fetch_fsmap learned FSMap version 28 2024-05-09T00:46:11.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.533+0000 7f4c23ed3b80 1 -- 172.21.15.5:0/933917787 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5587d7740950 con 0x5587d7696110 2024-05-09T00:46:11.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.533+0000 7f4bf67fc700 1 -- 172.21.15.5:0/933917787 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c080aad70 con 0x5587d7696110 2024-05-09T00:46:11.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.534+0000 7f4bf67fc700 1 -- 172.21.15.5:0/933917787 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f4c00019020 con 0x5587d7696110 2024-05-09T00:46:11.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.534+0000 7f4c23ed3b80 10 client.14997 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T00:46:11.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.534+0000 7f4c23ed3b80 10 client.14997 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-09T00:46:11.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.534+0000 7f4c23ed3b80 20 client.14997 populate_metadata read hostname 'smithi005' 2024-05-09T00:46:11.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.534+0000 7f4c23ed3b80 1 --2- 172.21.15.5:0/933917787 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x5587d7742fc0 0x5587d7744430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:11.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.534+0000 7f4c23ed3b80 4 client.14997 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:46:11.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.534+0000 7f4c23ed3b80 1 -- 172.21.15.5:0/933917787 --> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587d7659980 con 0x5587d7742fc0 2024-05-09T00:46:11.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.534+0000 7f4c0effd700 1 --2- 172.21.15.5:0/933917787 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x5587d7742fc0 0x5587d7744430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:11.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.535+0000 7f4c0effd700 1 --2- 172.21.15.5:0/933917787 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x5587d7742fc0 0x5587d7744430 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2941e2e8a4293e53 server_cookie=9bd40d1e0e04900e in_seq=0 out_seq=0 2024-05-09T00:46:11.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.535+0000 7f4bf67fc700 10 client.14997 ms_handle_connect on v2:172.21.15.131:6838/2748669631 2024-05-09T00:46:11.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.535+0000 7f4bf67fc700 1 -- 172.21.15.5:0/933917787 <== mds.2 v2:172.21.15.131:6838/2748669631 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5587d7659980 con 0x5587d7742fc0 2024-05-09T00:46:11.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.535+0000 7f4bf67fc700 10 client.14997 handle_command_reply: tid=0 2024-05-09T00:46:11.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.536+0000 7f4c23ed3b80 1 client.14997 shutdown 2024-05-09T00:46:11.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.536+0000 7f4c23ed3b80 1 -- 172.21.15.5:0/933917787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587d7696110 msgr2=0x5587d7739210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:11.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.536+0000 7f4c23ed3b80 1 --2- 172.21.15.5:0/933917787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587d7696110 0x5587d7739210 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4c00007ec0 tx=0x7f4c00012730 comp rx=0 tx=0).stop 2024-05-09T00:46:11.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.536+0000 7f4c23ed3b80 1 -- 172.21.15.5:0/933917787 shutdown_connections 2024-05-09T00:46:11.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.536+0000 7f4c23ed3b80 1 --2- 172.21.15.5:0/933917787 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x5587d7742fc0 0x5587d7744430 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:11.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.536+0000 7f4bf67fc700 0 client.14997 ms_handle_reset on v2:172.21.15.131:6838/2748669631 2024-05-09T00:46:11.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.536+0000 7f4c23ed3b80 1 --2- 172.21.15.5:0/933917787 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d7729650 0x5587d77330d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:11.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.536+0000 7f4c23ed3b80 1 --2- 172.21.15.5:0/933917787 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5587d75ac450 0x5587d7738cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:11.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.536+0000 7f4c23ed3b80 1 --2- 172.21.15.5:0/933917787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587d7696110 0x5587d7739210 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:11.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.536+0000 7f4c23ed3b80 1 -- 172.21.15.5:0/933917787 >> 172.21.15.5:0/933917787 conn(0x5587d760c650 msgr2=0x5587d7724170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:11.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.536+0000 7f4c23ed3b80 1 -- 172.21.15.5:0/933917787 shutdown_connections 2024-05-09T00:46:11.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.536+0000 7f4c23ed3b80 1 -- 172.21.15.5:0/933917787 wait complete. 2024-05-09T00:46:11.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.536+0000 7f4c23ed3b80 20 client.14997 trim_cache size 0 max 16384 2024-05-09T00:46:11.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.557+0000 7f4c23ed3b80 1 Processor -- start 2024-05-09T00:46:11.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.557+0000 7f4c23ed3b80 1 -- start start 2024-05-09T00:46:11.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.558+0000 7f4c23ed3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d75ac450 0x5587d772e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:11.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.558+0000 7f4c23ed3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587d7696110 0x5587d774d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:11.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.558+0000 7f4c23ed3b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5587d7729650 0x5587d774ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:11.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.558+0000 7f4c23ed3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5587d7742770 con 0x5587d7696110 2024-05-09T00:46:11.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.558+0000 7f4c23ed3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5587d7742e40 con 0x5587d75ac450 2024-05-09T00:46:11.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.558+0000 7f4c23ed3b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5587d7743510 con 0x5587d7729650 2024-05-09T00:46:11.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.558+0000 7f4c0e7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587d7696110 0x5587d774d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:11.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.558+0000 7f4c0e7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587d7696110 0x5587d774d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58892/0 (socket says 172.21.15.5:58892) 2024-05-09T00:46:11.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.558+0000 7f4c0e7fc700 1 -- 172.21.15.5:0/1704574150 learned_addr learned my addr 172.21.15.5:0/1704574150 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:46:11.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.558+0000 7f4c0effd700 1 --2- 172.21.15.5:0/1704574150 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d75ac450 0x5587d772e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:11.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.558+0000 7f4c0f7fe700 1 --2- 172.21.15.5:0/1704574150 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5587d7729650 0x5587d774ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:11.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.558+0000 7f4c0e7fc700 1 -- 172.21.15.5:0/1704574150 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5587d7729650 msgr2=0x5587d774ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:11.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.558+0000 7f4c0e7fc700 1 --2- 172.21.15.5:0/1704574150 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5587d7729650 0x5587d774ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:11.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.558+0000 7f4c0e7fc700 1 -- 172.21.15.5:0/1704574150 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d75ac450 msgr2=0x5587d772e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:11.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.558+0000 7f4c0e7fc700 1 --2- 172.21.15.5:0/1704574150 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d75ac450 0x5587d772e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:11.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.558+0000 7f4c0e7fc700 1 -- 172.21.15.5:0/1704574150 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587d772f000 con 0x5587d7696110 2024-05-09T00:46:11.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.558+0000 7f4c0f7fe700 1 --2- 172.21.15.5:0/1704574150 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5587d7729650 0x5587d774ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:46:11.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.558+0000 7f4c0e7fc700 1 --2- 172.21.15.5:0/1704574150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587d7696110 0x5587d774d890 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4c00005a80 tx=0x7f4c00012b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab7881953b277125 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:11.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.558+0000 7f4c04ff9700 1 -- 172.21.15.5:0/1704574150 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c00005390 con 0x5587d7696110 2024-05-09T00:46:11.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.559+0000 7f4c04ff9700 1 -- 172.21.15.5:0/1704574150 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c0001b8f0 con 0x5587d7696110 2024-05-09T00:46:11.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.559+0000 7f4c04ff9700 1 -- 172.21.15.5:0/1704574150 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c00003e90 con 0x5587d7696110 2024-05-09T00:46:11.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.559+0000 7f4c23ed3b80 1 -- 172.21.15.5:0/1704574150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587d7696110 msgr2=0x5587d774d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:11.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.559+0000 7f4c23ed3b80 1 --2- 172.21.15.5:0/1704574150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587d7696110 0x5587d774d890 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4c00005a80 tx=0x7f4c00012b30 comp rx=0 tx=0).stop 2024-05-09T00:46:11.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.559+0000 7f4c23ed3b80 1 -- 172.21.15.5:0/1704574150 shutdown_connections 2024-05-09T00:46:11.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.559+0000 7f4c23ed3b80 1 --2- 172.21.15.5:0/1704574150 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5587d7729650 0x5587d774ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:11.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.559+0000 7f4c23ed3b80 1 --2- 172.21.15.5:0/1704574150 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d75ac450 0x5587d772e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:11.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.559+0000 7f4c23ed3b80 1 --2- 172.21.15.5:0/1704574150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587d7696110 0x5587d774d890 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:11.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.559+0000 7f4c23ed3b80 1 -- 172.21.15.5:0/1704574150 >> 172.21.15.5:0/1704574150 conn(0x5587d760c650 msgr2=0x5587d772e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:11.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.559+0000 7f4c23ed3b80 1 -- 172.21.15.5:0/1704574150 shutdown_connections 2024-05-09T00:46:11.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.560+0000 7f4c23ed3b80 1 -- 172.21.15.5:0/1704574150 wait complete. 2024-05-09T00:46:11.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.560+0000 7f4c23ed3b80 1 -- start start 2024-05-09T00:46:11.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.560+0000 7f4c23ed3b80 1 Processor -- start 2024-05-09T00:46:11.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.560+0000 7f4c23ed3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587d75ac450 0x5587d773bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:11.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.560+0000 7f4c23ed3b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5587d7696110 0x5587d773c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:11.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.560+0000 7f4c23ed3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d7729650 0x5587d77ad430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:11.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.560+0000 7f4c23ed3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5587d77442b0 con 0x5587d75ac450 2024-05-09T00:46:11.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.560+0000 7f4c23ed3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5587d7743be0 con 0x5587d7729650 2024-05-09T00:46:11.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.560+0000 7f4c23ed3b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5587d7744980 con 0x5587d7696110 2024-05-09T00:46:11.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.560+0000 7f4c23ed3b80 10 client.0 _ll_register_callbacks cb 0x5587d74bd670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:46:11.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.560+0000 7f4c0effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587d75ac450 0x5587d773bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:11.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.560+0000 7f4c0effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587d75ac450 0x5587d773bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58908/0 (socket says 172.21.15.5:58908) 2024-05-09T00:46:11.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.560+0000 7f4c0effd700 1 -- 172.21.15.5:0/1179524578 learned_addr learned my addr 172.21.15.5:0/1179524578 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:46:11.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.561+0000 7f4c0f7fe700 1 --2- 172.21.15.5:0/1179524578 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d7729650 0x5587d77ad430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:11.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.561+0000 7f4c0e7fc700 1 --2- 172.21.15.5:0/1179524578 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5587d7696110 0x5587d773c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:11.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.561+0000 7f4c0effd700 1 -- 172.21.15.5:0/1179524578 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5587d7696110 msgr2=0x5587d773c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:11.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.561+0000 7f4c0effd700 1 --2- 172.21.15.5:0/1179524578 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5587d7696110 0x5587d773c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:11.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.561+0000 7f4c0effd700 1 -- 172.21.15.5:0/1179524578 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d7729650 msgr2=0x5587d77ad430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:11.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.561+0000 7f4c0effd700 1 --2- 172.21.15.5:0/1179524578 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d7729650 0x5587d77ad430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:11.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.561+0000 7f4c0effd700 1 -- 172.21.15.5:0/1179524578 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587d77ad970 con 0x5587d75ac450 2024-05-09T00:46:11.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.561+0000 7f4c0effd700 1 --2- 172.21.15.5:0/1179524578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587d75ac450 0x5587d773bc30 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4bf800f9d0 tx=0x7f4bf8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b35bca69d13c3e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:11.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.561+0000 7f4bf67fc700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:46:11.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.561+0000 7f4bf67fc700 1 -- 172.21.15.5:0/1179524578 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf8003880 con 0x5587d75ac450 2024-05-09T00:46:11.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.561+0000 7f4bf67fc700 1 -- 172.21.15.5:0/1179524578 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f4bf8003a20 con 0x5587d75ac450 2024-05-09T00:46:11.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.561+0000 7f4c23ed3b80 1 -- 172.21.15.5:0/1179524578 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c0001b8f0 con 0x5587d75ac450 2024-05-09T00:46:11.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.561+0000 7f4bf67fc700 1 -- 172.21.15.5:0/1179524578 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4bf8003c20 con 0x5587d75ac450 2024-05-09T00:46:11.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.561+0000 7f4c23ed3b80 10 client.15003 fetch_fsmap learned FSMap version 28 2024-05-09T00:46:11.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.562+0000 7f4c23ed3b80 1 -- 172.21.15.5:0/1179524578 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5587d77aeb20 con 0x5587d75ac450 2024-05-09T00:46:11.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.562+0000 7f4bf67fc700 1 -- 172.21.15.5:0/1179524578 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c0001b8f0 con 0x5587d75ac450 2024-05-09T00:46:11.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.562+0000 7f4bf67fc700 1 -- 172.21.15.5:0/1179524578 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f4bf803a070 con 0x5587d75ac450 2024-05-09T00:46:11.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.562+0000 7f4c23ed3b80 10 client.15003 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T00:46:11.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.562+0000 7f4c23ed3b80 10 client.15003 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-09T00:46:11.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.562+0000 7f4c23ed3b80 20 client.15003 populate_metadata read hostname 'smithi005' 2024-05-09T00:46:11.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.562+0000 7f4c23ed3b80 1 --2- 172.21.15.5:0/1179524578 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x5587d7742fc0 0x5587d77b2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:11.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.562+0000 7f4c23ed3b80 4 client.15003 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T00:46:11.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.562+0000 7f4c23ed3b80 1 -- 172.21.15.5:0/1179524578 --> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] -- command(tid 0: {"prefix": "status"}) -- 0x5587d7653a20 con 0x5587d7742fc0 2024-05-09T00:46:11.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.562+0000 7f4c0e7fc700 1 --2- 172.21.15.5:0/1179524578 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x5587d7742fc0 0x5587d77b2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:11.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.563+0000 7f4bf67fc700 10 client.15003 ms_handle_connect on v2:172.21.15.131:6838/2748669631 2024-05-09T00:46:11.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.563+0000 7f4c0e7fc700 1 --2- 172.21.15.5:0/1179524578 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x5587d7742fc0 0x5587d77b2440 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e663c0d524867b07 server_cookie=b842c8d1f0144ce6 in_seq=0 out_seq=0 2024-05-09T00:46:11.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.563+0000 7f4bf67fc700 1 -- 172.21.15.5:0/1179524578 <== mds.2 v2:172.21.15.131:6838/2748669631 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5587d7653a20 con 0x5587d7742fc0 2024-05-09T00:46:11.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.563+0000 7f4bf67fc700 10 client.15003 handle_command_reply: tid=0 2024-05-09T00:46:11.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.563+0000 7f4c23ed3b80 1 client.15003 shutdown 2024-05-09T00:46:11.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.563+0000 7f4c23ed3b80 1 -- 172.21.15.5:0/1179524578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587d75ac450 msgr2=0x5587d773bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:11.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.564+0000 7f4c23ed3b80 1 --2- 172.21.15.5:0/1179524578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587d75ac450 0x5587d773bc30 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4bf800f9d0 tx=0x7f4bf8012000 comp rx=0 tx=0).stop 2024-05-09T00:46:11.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.564+0000 7f4c23ed3b80 1 -- 172.21.15.5:0/1179524578 shutdown_connections 2024-05-09T00:46:11.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.564+0000 7f4c23ed3b80 1 --2- 172.21.15.5:0/1179524578 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x5587d7742fc0 0x5587d77b2440 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:11.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.564+0000 7f4c23ed3b80 1 --2- 172.21.15.5:0/1179524578 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d7729650 0x5587d77ad430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:11.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.564+0000 7f4c23ed3b80 1 --2- 172.21.15.5:0/1179524578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587d75ac450 0x5587d773bc30 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:11.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.564+0000 7f4c23ed3b80 1 --2- 172.21.15.5:0/1179524578 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5587d7696110 0x5587d773c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:11.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.564+0000 7f4c23ed3b80 1 -- 172.21.15.5:0/1179524578 >> 172.21.15.5:0/1179524578 conn(0x5587d760c650 msgr2=0x5587d7732980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:11.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.564+0000 7f4bf67fc700 0 client.15003 ms_handle_reset on v2:172.21.15.131:6838/2748669631 2024-05-09T00:46:11.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.564+0000 7f4c23ed3b80 1 -- 172.21.15.5:0/1179524578 shutdown_connections 2024-05-09T00:46:11.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.564+0000 7f4c23ed3b80 1 -- 172.21.15.5:0/1179524578 wait complete. 2024-05-09T00:46:11.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.564+0000 7f4c23ed3b80 20 client.15003 trim_cache size 0 max 16384 2024-05-09T00:46:11.571 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-09T00:46:11.571 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "d85814c6-0d9b-11ef-bc97-c7b262605968", 2024-05-09T00:46:11.571 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 2, 2024-05-09T00:46:11.571 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24494, 2024-05-09T00:46:11.571 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-05-09T00:46:11.571 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-05-09T00:46:11.571 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-05-09T00:46:11.571 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 131.63090817599999, 2024-05-09T00:46:11.571 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 28, 2024-05-09T00:46:11.572 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 82, 2024-05-09T00:46:11.572 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 78, 2024-05-09T00:46:11.572 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 160.44611235100001 2024-05-09T00:46:11.572 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-09T00:46:11.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.564+0000 7f4c154bd700 1 -- 172.21.15.5:0/255526687 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f4bfc0b2880 msgr2=0x7f4bfc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:11.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.564+0000 7f4c154bd700 1 --2- 172.21.15.5:0/255526687 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f4bfc0b2880 0x7f4bfc0b4d90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8005950 tx=0x7f4bf80058e0 comp rx=0 tx=0).stop 2024-05-09T00:46:11.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.564+0000 7f4c154bd700 1 -- 172.21.15.5:0/255526687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c10108740 msgr2=0x7f4c1019c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:11.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.564+0000 7f4c154bd700 1 --2- 172.21.15.5:0/255526687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c10108740 0x7f4c1019c760 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4c0800d010 tx=0x7f4c080079a0 comp rx=0 tx=0).stop 2024-05-09T00:46:11.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.564+0000 7f4c154bd700 1 -- 172.21.15.5:0/255526687 shutdown_connections 2024-05-09T00:46:11.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.564+0000 7f4c154bd700 1 --2- 172.21.15.5:0/255526687 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f4bfc0b2880 0x7f4bfc0b4d90 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:11.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.564+0000 7f4c154bd700 1 --2- 172.21.15.5:0/255526687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c10108740 0x7f4c1019c760 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:11.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.564+0000 7f4c154bd700 1 --2- 172.21.15.5:0/255526687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c101034c0 0x7f4c1019bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:11.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.564+0000 7f4c154bd700 1 --2- 172.21.15.5:0/255526687 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c10107df0 0x7f4c1019c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:11.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.565+0000 7f4c154bd700 1 -- 172.21.15.5:0/255526687 >> 172.21.15.5:0/255526687 conn(0x7f4c100fedf0 msgr2=0x7f4c10107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:11.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.565+0000 7f4c154bd700 1 -- 172.21.15.5:0/255526687 shutdown_connections 2024-05-09T00:46:11.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:11.565+0000 7f4c154bd700 1 -- 172.21.15.5:0/255526687 wait complete. 2024-05-09T00:46:11.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:46:11 smithi131 ceph-mon[34547]: pgmap v366: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-09T00:46:11.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:46:11 smithi157 ceph-mon[34294]: pgmap v366: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-09T00:46:12.266 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.c status 2024-05-09T00:46:12.527 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:46:13.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:46:13 smithi131 ceph-mon[34547]: pgmap v367: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T00:46:13.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:46:13 smithi005 ceph-mon[30365]: pgmap v367: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T00:46:13.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:46:13 smithi157 ceph-mon[34294]: pgmap v367: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T00:46:14.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.254+0000 7f494b3d1700 1 -- 172.21.15.5:0/596096739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4944108740 msgr2=0x7f49441100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:14.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.254+0000 7f494b3d1700 1 --2- 172.21.15.5:0/596096739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4944108740 0x7f49441100a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f493c0099c0 tx=0x7f493c009cd0 comp rx=0 tx=0).stop 2024-05-09T00:46:14.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.254+0000 7f494b3d1700 1 -- 172.21.15.5:0/596096739 shutdown_connections 2024-05-09T00:46:14.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.254+0000 7f494b3d1700 1 --2- 172.21.15.5:0/596096739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4944108740 0x7f49441100a0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:14.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.255+0000 7f494b3d1700 1 --2- 172.21.15.5:0/596096739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49441034c0 0x7f49441038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:14.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.255+0000 7f494b3d1700 1 --2- 172.21.15.5:0/596096739 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4944107df0 0x7f4944108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:14.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.255+0000 7f494b3d1700 1 -- 172.21.15.5:0/596096739 >> 172.21.15.5:0/596096739 conn(0x7f49440fedc0 msgr2=0x7f49441011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:14.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.255+0000 7f494b3d1700 1 -- 172.21.15.5:0/596096739 shutdown_connections 2024-05-09T00:46:14.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.255+0000 7f494b3d1700 1 -- 172.21.15.5:0/596096739 wait complete. 2024-05-09T00:46:14.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.255+0000 7f494b3d1700 1 Processor -- start 2024-05-09T00:46:14.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.255+0000 7f494b3d1700 1 -- start start 2024-05-09T00:46:14.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.256+0000 7f494b3d1700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49441034c0 0x7f49440ff2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:14.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.256+0000 7f494b3d1700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4944107df0 0x7f49440ff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:14.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.256+0000 7f494b3d1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4944108740 0x7f49440ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:14.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.256+0000 7f494b3d1700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4944113370 con 0x7f4944108740 2024-05-09T00:46:14.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.256+0000 7f494b3d1700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f49441131f0 con 0x7f4944107df0 2024-05-09T00:46:14.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.256+0000 7f494b3d1700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f49441134f0 con 0x7f49441034c0 2024-05-09T00:46:14.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.256+0000 7f494996e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4944108740 0x7f49440ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:14.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.256+0000 7f494996e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4944108740 0x7f49440ffd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53272/0 (socket says 172.21.15.5:53272) 2024-05-09T00:46:14.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.256+0000 7f494996e700 1 -- 172.21.15.5:0/3025142068 learned_addr learned my addr 172.21.15.5:0/3025142068 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:46:14.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.256+0000 7f494896c700 1 --2- 172.21.15.5:0/3025142068 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4944107df0 0x7f49440ff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:14.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.256+0000 7f494916d700 1 --2- 172.21.15.5:0/3025142068 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49441034c0 0x7f49440ff2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:14.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.256+0000 7f494996e700 1 -- 172.21.15.5:0/3025142068 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49441034c0 msgr2=0x7f49440ff2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:14.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.256+0000 7f494996e700 1 --2- 172.21.15.5:0/3025142068 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49441034c0 0x7f49440ff2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:14.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.256+0000 7f494996e700 1 -- 172.21.15.5:0/3025142068 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4944107df0 msgr2=0x7f49440ff7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:14.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.256+0000 7f494996e700 1 --2- 172.21.15.5:0/3025142068 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4944107df0 0x7f49440ff7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:14.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.256+0000 7f494996e700 1 -- 172.21.15.5:0/3025142068 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49441a9210 con 0x7f4944108740 2024-05-09T00:46:14.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.256+0000 7f494916d700 1 --2- 172.21.15.5:0/3025142068 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49441034c0 0x7f49440ff2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:46:14.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.257+0000 7f494996e700 1 --2- 172.21.15.5:0/3025142068 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4944108740 0x7f49440ffd20 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f493c005910 tx=0x7f493c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:14.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.257+0000 7f493a7fc700 1 -- 172.21.15.5:0/3025142068 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f493c027070 con 0x7f4944108740 2024-05-09T00:46:14.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.257+0000 7f493a7fc700 1 -- 172.21.15.5:0/3025142068 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f493c003f20 con 0x7f4944108740 2024-05-09T00:46:14.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.257+0000 7f494b3d1700 1 -- 172.21.15.5:0/3025142068 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49441a94a0 con 0x7f4944108740 2024-05-09T00:46:14.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.257+0000 7f493a7fc700 1 -- 172.21.15.5:0/3025142068 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f493c022c50 con 0x7f4944108740 2024-05-09T00:46:14.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.257+0000 7f494b3d1700 1 -- 172.21.15.5:0/3025142068 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49441a9950 con 0x7f4944108740 2024-05-09T00:46:14.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.261+0000 7f493a7fc700 1 -- 172.21.15.5:0/3025142068 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f493c02a070 con 0x7f4944108740 2024-05-09T00:46:14.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.262+0000 7f493a7fc700 1 --2- 172.21.15.5:0/3025142068 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f49300b2900 0x7f49300b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:14.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.262+0000 7f4959de7b80 1 Processor -- start 2024-05-09T00:46:14.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.262+0000 7f494916d700 1 --2- 172.21.15.5:0/3025142068 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f49300b2900 0x7f49300b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:14.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.262+0000 7f4959de7b80 1 -- start start 2024-05-09T00:46:14.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.262+0000 7f493a7fc700 1 -- 172.21.15.5:0/3025142068 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f493c0e2e00 con 0x7f4944108740 2024-05-09T00:46:14.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.262+0000 7f4959de7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7c4a2110 0x560c7c53b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:14.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.262+0000 7f4959de7b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c7c3b8450 0x560c7c53b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:14.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.262+0000 7f4959de7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c7c535650 0x560c7c535a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:14.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.262+0000 7f4959de7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560c7c457fa0 con 0x560c7c4a2110 2024-05-09T00:46:14.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.262+0000 7f4959de7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x560c7c3f67a0 con 0x560c7c535650 2024-05-09T00:46:14.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.262+0000 7f4959de7b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x560c7c453f60 con 0x560c7c3b8450 2024-05-09T00:46:14.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.262+0000 7f494916d700 1 --2- 172.21.15.5:0/3025142068 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f49300b2900 0x7f49300b4e10 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4940009dd0 tx=0x7f4940009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:14.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.262+0000 7f494916d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7c4a2110 0x560c7c53b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:14.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.262+0000 7f494916d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7c4a2110 0x560c7c53b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53274/0 (socket says 172.21.15.5:53274) 2024-05-09T00:46:14.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.262+0000 7f494916d700 1 -- 172.21.15.5:0/1238371617 learned_addr learned my addr 172.21.15.5:0/1238371617 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:46:14.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.263+0000 7f494896c700 1 --2- 172.21.15.5:0/1238371617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c7c3b8450 0x560c7c53b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:14.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.263+0000 7f494996e700 1 --2- 172.21.15.5:0/1238371617 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c7c535650 0x560c7c535a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:14.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.263+0000 7f494916d700 1 -- 172.21.15.5:0/1238371617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c7c3b8450 msgr2=0x560c7c53b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:14.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.263+0000 7f494916d700 1 --2- 172.21.15.5:0/1238371617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c7c3b8450 0x560c7c53b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:14.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.263+0000 7f494916d700 1 -- 172.21.15.5:0/1238371617 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c7c535650 msgr2=0x560c7c535a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:14.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.263+0000 7f494916d700 1 --2- 172.21.15.5:0/1238371617 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c7c535650 0x560c7c535a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:14.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.263+0000 7f494916d700 1 -- 172.21.15.5:0/1238371617 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c7c3ee320 con 0x560c7c4a2110 2024-05-09T00:46:14.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.263+0000 7f494996e700 1 --2- 172.21.15.5:0/1238371617 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c7c535650 0x560c7c535a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:46:14.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.263+0000 7f494916d700 1 --2- 172.21.15.5:0/1238371617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7c4a2110 0x560c7c53b4a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4940016550 tx=0x7f4940012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef095dd6e3bdec7a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:14.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.263+0000 7f492f7fe700 1 -- 172.21.15.5:0/1238371617 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4940026070 con 0x560c7c4a2110 2024-05-09T00:46:14.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.263+0000 7f492f7fe700 1 -- 172.21.15.5:0/1238371617 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4940021af0 con 0x560c7c4a2110 2024-05-09T00:46:14.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.263+0000 7f492f7fe700 1 -- 172.21.15.5:0/1238371617 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4940014a90 con 0x560c7c4a2110 2024-05-09T00:46:14.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.264+0000 7f4959de7b80 1 -- 172.21.15.5:0/1238371617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7c4a2110 msgr2=0x560c7c53b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:14.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.264+0000 7f4959de7b80 1 --2- 172.21.15.5:0/1238371617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7c4a2110 0x560c7c53b4a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4940016550 tx=0x7f4940012000 comp rx=0 tx=0).stop 2024-05-09T00:46:14.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.264+0000 7f4959de7b80 1 -- 172.21.15.5:0/1238371617 shutdown_connections 2024-05-09T00:46:14.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.264+0000 7f4959de7b80 1 --2- 172.21.15.5:0/1238371617 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c7c535650 0x560c7c535a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:14.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.264+0000 7f4959de7b80 1 --2- 172.21.15.5:0/1238371617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7c4a2110 0x560c7c53b4a0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:14.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.264+0000 7f4959de7b80 1 --2- 172.21.15.5:0/1238371617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c7c3b8450 0x560c7c53b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:14.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.264+0000 7f4959de7b80 1 -- 172.21.15.5:0/1238371617 >> 172.21.15.5:0/1238371617 conn(0x560c7c418650 msgr2=0x560c7c5303b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:14.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.264+0000 7f4959de7b80 1 -- 172.21.15.5:0/1238371617 shutdown_connections 2024-05-09T00:46:14.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.264+0000 7f4959de7b80 1 -- 172.21.15.5:0/1238371617 wait complete. 2024-05-09T00:46:14.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.265+0000 7f4959de7b80 1 -- start start 2024-05-09T00:46:14.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.265+0000 7f4959de7b80 1 Processor -- start 2024-05-09T00:46:14.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.265+0000 7f4959de7b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c7c3b8450 0x560c7c53f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:14.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.265+0000 7f4959de7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c7c4a2110 0x560c7c53f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:14.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.265+0000 7f4959de7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7c535650 0x560c7c53fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:14.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.265+0000 7f4959de7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560c7c2d3b50 con 0x560c7c535650 2024-05-09T00:46:14.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.265+0000 7f4959de7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x560c7c412100 con 0x560c7c4a2110 2024-05-09T00:46:14.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.265+0000 7f4959de7b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x560c7c542180 con 0x560c7c3b8450 2024-05-09T00:46:14.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.265+0000 7f4959de7b80 10 client.0 _ll_register_callbacks cb 0x560c7c2c9670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:46:14.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.266+0000 7f494996e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7c535650 0x560c7c53fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:14.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.266+0000 7f494996e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7c535650 0x560c7c53fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53286/0 (socket says 172.21.15.5:53286) 2024-05-09T00:46:14.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.266+0000 7f494996e700 1 -- 172.21.15.5:0/1760914201 learned_addr learned my addr 172.21.15.5:0/1760914201 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:46:14.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.266+0000 7f494916d700 1 --2- 172.21.15.5:0/1760914201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c7c3b8450 0x560c7c53f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:14.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.266+0000 7f494896c700 1 --2- 172.21.15.5:0/1760914201 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c7c4a2110 0x560c7c53f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:14.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.266+0000 7f494996e700 1 -- 172.21.15.5:0/1760914201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c7c3b8450 msgr2=0x560c7c53f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:14.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.266+0000 7f494996e700 1 --2- 172.21.15.5:0/1760914201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c7c3b8450 0x560c7c53f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:14.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.266+0000 7f494996e700 1 -- 172.21.15.5:0/1760914201 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c7c4a2110 msgr2=0x560c7c53f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:14.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.266+0000 7f494996e700 1 --2- 172.21.15.5:0/1760914201 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c7c4a2110 0x560c7c53f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:14.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.266+0000 7f494996e700 1 -- 172.21.15.5:0/1760914201 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c7c2d3440 con 0x560c7c535650 2024-05-09T00:46:14.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.266+0000 7f494996e700 1 --2- 172.21.15.5:0/1760914201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7c535650 0x560c7c53fc30 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f493c0a76f0 tx=0x7f493c0aa380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f610e067c3c25b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:14.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.266+0000 7f492cff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:46:14.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.266+0000 7f492cff9700 1 -- 172.21.15.5:0/1760914201 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f493c0a9f90 con 0x560c7c535650 2024-05-09T00:46:14.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.266+0000 7f492cff9700 1 -- 172.21.15.5:0/1760914201 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f493c0aa6e0 con 0x560c7c535650 2024-05-09T00:46:14.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.266+0000 7f4959de7b80 1 -- 172.21.15.5:0/1760914201 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4940021af0 con 0x560c7c535650 2024-05-09T00:46:14.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.266+0000 7f492cff9700 1 -- 172.21.15.5:0/1760914201 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f493c0aab80 con 0x560c7c535650 2024-05-09T00:46:14.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.266+0000 7f4959de7b80 10 client.15015 fetch_fsmap learned FSMap version 28 2024-05-09T00:46:14.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.266+0000 7f4959de7b80 1 -- 172.21.15.5:0/1760914201 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560c7c54cb40 con 0x560c7c535650 2024-05-09T00:46:14.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.267+0000 7f492cff9700 1 -- 172.21.15.5:0/1760914201 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4940021af0 con 0x560c7c535650 2024-05-09T00:46:14.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.267+0000 7f492cff9700 1 -- 172.21.15.5:0/1760914201 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f493c0b2dc0 con 0x560c7c535650 2024-05-09T00:46:14.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.267+0000 7f4959de7b80 10 client.15015 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T00:46:14.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.267+0000 7f4959de7b80 10 client.15015 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-09T00:46:14.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.267+0000 7f4959de7b80 20 client.15015 populate_metadata read hostname 'smithi005' 2024-05-09T00:46:14.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.267+0000 7f4959de7b80 1 --2- 172.21.15.5:0/1760914201 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x560c7c54f170 0x560c7c5505e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:14.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.267+0000 7f4959de7b80 4 client.15015 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:46:14.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.267+0000 7f4959de7b80 1 -- 172.21.15.5:0/1760914201 --> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c7c465980 con 0x560c7c54f170 2024-05-09T00:46:14.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.267+0000 7f494916d700 1 --2- 172.21.15.5:0/1760914201 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x560c7c54f170 0x560c7c5505e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:14.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.268+0000 7f494916d700 1 --2- 172.21.15.5:0/1760914201 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x560c7c54f170 0x560c7c5505e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bec0f2c8e005d9f5 server_cookie=b9ed037107489d91 in_seq=0 out_seq=0 2024-05-09T00:46:14.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.268+0000 7f492cff9700 10 client.15015 ms_handle_connect on v2:172.21.15.157:6836/744863009 2024-05-09T00:46:14.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.269+0000 7f492cff9700 1 -- 172.21.15.5:0/1760914201 <== mds.1 v2:172.21.15.157:6836/744863009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560c7c465980 con 0x560c7c54f170 2024-05-09T00:46:14.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.269+0000 7f492cff9700 10 client.15015 handle_command_reply: tid=0 2024-05-09T00:46:14.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.269+0000 7f4959de7b80 1 client.15015 shutdown 2024-05-09T00:46:14.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.269+0000 7f4959de7b80 1 -- 172.21.15.5:0/1760914201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7c535650 msgr2=0x560c7c53fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:14.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.269+0000 7f4959de7b80 1 --2- 172.21.15.5:0/1760914201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7c535650 0x560c7c53fc30 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f493c0a76f0 tx=0x7f493c0aa380 comp rx=0 tx=0).stop 2024-05-09T00:46:14.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.269+0000 7f4959de7b80 1 -- 172.21.15.5:0/1760914201 shutdown_connections 2024-05-09T00:46:14.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.269+0000 7f4959de7b80 1 --2- 172.21.15.5:0/1760914201 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x560c7c54f170 0x560c7c5505e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:14.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.269+0000 7f4959de7b80 1 --2- 172.21.15.5:0/1760914201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7c535650 0x560c7c53fc30 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:14.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.269+0000 7f4959de7b80 1 --2- 172.21.15.5:0/1760914201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c7c3b8450 0x560c7c53f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:14.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.269+0000 7f4959de7b80 1 --2- 172.21.15.5:0/1760914201 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c7c4a2110 0x560c7c53f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:14.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.269+0000 7f4959de7b80 1 -- 172.21.15.5:0/1760914201 >> 172.21.15.5:0/1760914201 conn(0x560c7c418650 msgr2=0x560c7c530170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:14.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.269+0000 7f492cff9700 0 client.15015 ms_handle_reset on v2:172.21.15.157:6836/744863009 2024-05-09T00:46:14.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.269+0000 7f4959de7b80 1 -- 172.21.15.5:0/1760914201 shutdown_connections 2024-05-09T00:46:14.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.269+0000 7f4959de7b80 1 -- 172.21.15.5:0/1760914201 wait complete. 2024-05-09T00:46:14.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.269+0000 7f4959de7b80 20 client.15015 trim_cache size 0 max 16384 2024-05-09T00:46:14.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.291+0000 7f4959de7b80 1 Processor -- start 2024-05-09T00:46:14.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.291+0000 7f4959de7b80 1 -- start start 2024-05-09T00:46:14.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.291+0000 7f4959de7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7c3b8450 0x560c7c53ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:14.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.291+0000 7f4959de7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c7c4a2110 0x560c7c54af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:14.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.291+0000 7f4959de7b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c7c535650 0x560c7c559a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:14.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.291+0000 7f4959de7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560c7c54e920 con 0x560c7c3b8450 2024-05-09T00:46:14.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.291+0000 7f4959de7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x560c7c54eff0 con 0x560c7c4a2110 2024-05-09T00:46:14.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.291+0000 7f4959de7b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x560c7c54f6c0 con 0x560c7c535650 2024-05-09T00:46:14.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.291+0000 7f494916d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7c3b8450 0x560c7c53ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:14.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.291+0000 7f494916d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7c3b8450 0x560c7c53ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53296/0 (socket says 172.21.15.5:53296) 2024-05-09T00:46:14.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.291+0000 7f494916d700 1 -- 172.21.15.5:0/1392345259 learned_addr learned my addr 172.21.15.5:0/1392345259 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:46:14.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.291+0000 7f494996e700 1 --2- 172.21.15.5:0/1392345259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c7c535650 0x560c7c559a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:14.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.291+0000 7f494916d700 1 -- 172.21.15.5:0/1392345259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c7c535650 msgr2=0x560c7c559a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:14.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.291+0000 7f494916d700 1 --2- 172.21.15.5:0/1392345259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c7c535650 0x560c7c559a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:14.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.291+0000 7f494896c700 1 --2- 172.21.15.5:0/1392345259 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c7c4a2110 0x560c7c54af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:14.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.291+0000 7f494916d700 1 -- 172.21.15.5:0/1392345259 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c7c4a2110 msgr2=0x560c7c54af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:14.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.291+0000 7f494916d700 1 --2- 172.21.15.5:0/1392345259 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c7c4a2110 0x560c7c54af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:14.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.291+0000 7f494916d700 1 -- 172.21.15.5:0/1392345259 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c7c531cb0 con 0x560c7c3b8450 2024-05-09T00:46:14.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.292+0000 7f494916d700 1 --2- 172.21.15.5:0/1392345259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7c3b8450 0x560c7c53ce20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4940024040 tx=0x7f49400145f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c12ae562c9f1424 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:14.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.292+0000 7f492effd700 1 -- 172.21.15.5:0/1392345259 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4940026040 con 0x560c7c3b8450 2024-05-09T00:46:14.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.292+0000 7f492effd700 1 -- 172.21.15.5:0/1392345259 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4940004c90 con 0x560c7c3b8450 2024-05-09T00:46:14.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.292+0000 7f492effd700 1 -- 172.21.15.5:0/1392345259 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49400149d0 con 0x560c7c3b8450 2024-05-09T00:46:14.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.293+0000 7f4959de7b80 1 -- 172.21.15.5:0/1392345259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7c3b8450 msgr2=0x560c7c53ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:14.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.293+0000 7f4959de7b80 1 --2- 172.21.15.5:0/1392345259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7c3b8450 0x560c7c53ce20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4940024040 tx=0x7f49400145f0 comp rx=0 tx=0).stop 2024-05-09T00:46:14.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.293+0000 7f4959de7b80 1 -- 172.21.15.5:0/1392345259 shutdown_connections 2024-05-09T00:46:14.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.293+0000 7f4959de7b80 1 --2- 172.21.15.5:0/1392345259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c7c535650 0x560c7c559a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:14.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.293+0000 7f4959de7b80 1 --2- 172.21.15.5:0/1392345259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7c3b8450 0x560c7c53ce20 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:14.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.293+0000 7f4959de7b80 1 --2- 172.21.15.5:0/1392345259 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c7c4a2110 0x560c7c54af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:14.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.293+0000 7f4959de7b80 1 -- 172.21.15.5:0/1392345259 >> 172.21.15.5:0/1392345259 conn(0x560c7c418650 msgr2=0x560c7c53daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:14.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.293+0000 7f4959de7b80 1 -- 172.21.15.5:0/1392345259 shutdown_connections 2024-05-09T00:46:14.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.293+0000 7f4959de7b80 1 -- 172.21.15.5:0/1392345259 wait complete. 2024-05-09T00:46:14.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.293+0000 7f4959de7b80 1 -- start start 2024-05-09T00:46:14.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.293+0000 7f4959de7b80 1 Processor -- start 2024-05-09T00:46:14.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.294+0000 7f4959de7b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c7c3b8450 0x560c7c54a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:14.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.294+0000 7f4959de7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c7c4a2110 0x560c7c54abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:14.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.294+0000 7f4959de7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7c535650 0x560c7c54b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:14.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.294+0000 7f4959de7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560c7c550460 con 0x560c7c535650 2024-05-09T00:46:14.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.294+0000 7f4959de7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x560c7c54fd90 con 0x560c7c4a2110 2024-05-09T00:46:14.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.294+0000 7f4959de7b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x560c7c550b30 con 0x560c7c3b8450 2024-05-09T00:46:14.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.294+0000 7f4959de7b80 10 client.0 _ll_register_callbacks cb 0x560c7c2c9670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:46:14.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.294+0000 7f494996e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7c535650 0x560c7c54b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:14.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.294+0000 7f494996e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7c535650 0x560c7c54b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53308/0 (socket says 172.21.15.5:53308) 2024-05-09T00:46:14.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.294+0000 7f494996e700 1 -- 172.21.15.5:0/976992280 learned_addr learned my addr 172.21.15.5:0/976992280 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:46:14.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.294+0000 7f494896c700 1 --2- 172.21.15.5:0/976992280 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c7c4a2110 0x560c7c54abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:14.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.294+0000 7f494916d700 1 --2- 172.21.15.5:0/976992280 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c7c3b8450 0x560c7c54a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:14.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.294+0000 7f494996e700 1 -- 172.21.15.5:0/976992280 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c7c3b8450 msgr2=0x560c7c54a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:14.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.294+0000 7f494996e700 1 --2- 172.21.15.5:0/976992280 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c7c3b8450 0x560c7c54a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:14.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.294+0000 7f494996e700 1 -- 172.21.15.5:0/976992280 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c7c4a2110 msgr2=0x560c7c54abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:14.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.294+0000 7f494996e700 1 --2- 172.21.15.5:0/976992280 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c7c4a2110 0x560c7c54abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:14.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.294+0000 7f494996e700 1 -- 172.21.15.5:0/976992280 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c7c557d30 con 0x560c7c535650 2024-05-09T00:46:14.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.294+0000 7f494896c700 1 --2- 172.21.15.5:0/976992280 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c7c4a2110 0x560c7c54abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:46:14.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.295+0000 7f494996e700 1 --2- 172.21.15.5:0/976992280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7c535650 0x560c7c54b100 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f493c0aa180 tx=0x7f493c0aa910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dd2f8c9944db5d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:14.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.295+0000 7f492cff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:46:14.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.295+0000 7f492cff9700 1 -- 172.21.15.5:0/976992280 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f493c0ae830 con 0x560c7c535650 2024-05-09T00:46:14.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.295+0000 7f492cff9700 1 -- 172.21.15.5:0/976992280 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f493c003c80 con 0x560c7c535650 2024-05-09T00:46:14.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.295+0000 7f4959de7b80 1 -- 172.21.15.5:0/976992280 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4940004c90 con 0x560c7c535650 2024-05-09T00:46:14.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.295+0000 7f492cff9700 1 -- 172.21.15.5:0/976992280 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f493c0b3250 con 0x560c7c535650 2024-05-09T00:46:14.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.295+0000 7f4959de7b80 10 client.15021 fetch_fsmap learned FSMap version 28 2024-05-09T00:46:14.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.295+0000 7f4959de7b80 1 -- 172.21.15.5:0/976992280 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560c7c5ba7c0 con 0x560c7c535650 2024-05-09T00:46:14.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.295+0000 7f492cff9700 1 -- 172.21.15.5:0/976992280 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4940004c90 con 0x560c7c535650 2024-05-09T00:46:14.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.295+0000 7f492cff9700 1 -- 172.21.15.5:0/976992280 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f493c0c4ce0 con 0x560c7c535650 2024-05-09T00:46:14.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.295+0000 7f4959de7b80 10 client.15021 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T00:46:14.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.295+0000 7f4959de7b80 10 client.15021 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-09T00:46:14.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.295+0000 7f4959de7b80 20 client.15021 populate_metadata read hostname 'smithi005' 2024-05-09T00:46:14.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.295+0000 7f4959de7b80 1 --2- 172.21.15.5:0/976992280 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x560c7c5bca00 0x560c7c5bde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:14.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.295+0000 7f4959de7b80 4 client.15021 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T00:46:14.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.295+0000 7f4959de7b80 1 -- 172.21.15.5:0/976992280 --> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] -- command(tid 0: {"prefix": "status"}) -- 0x560c7c45fa20 con 0x560c7c5bca00 2024-05-09T00:46:14.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.296+0000 7f494916d700 1 --2- 172.21.15.5:0/976992280 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x560c7c5bca00 0x560c7c5bde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:14.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.296+0000 7f494916d700 1 --2- 172.21.15.5:0/976992280 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x560c7c5bca00 0x560c7c5bde70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5f1ffa095c328b6b server_cookie=3d5a18b0d6ae7865 in_seq=0 out_seq=0 2024-05-09T00:46:14.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.296+0000 7f492cff9700 10 client.15021 ms_handle_connect on v2:172.21.15.157:6836/744863009 2024-05-09T00:46:14.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.296+0000 7f492cff9700 1 -- 172.21.15.5:0/976992280 <== mds.1 v2:172.21.15.157:6836/744863009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x560c7c45fa20 con 0x560c7c5bca00 2024-05-09T00:46:14.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.296+0000 7f492cff9700 10 client.15021 handle_command_reply: tid=0 2024-05-09T00:46:14.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.296+0000 7f4959de7b80 1 client.15021 shutdown 2024-05-09T00:46:14.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.296+0000 7f4959de7b80 1 -- 172.21.15.5:0/976992280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7c535650 msgr2=0x560c7c54b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:14.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.296+0000 7f4959de7b80 1 --2- 172.21.15.5:0/976992280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7c535650 0x560c7c54b100 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f493c0aa180 tx=0x7f493c0aa910 comp rx=0 tx=0).stop 2024-05-09T00:46:14.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.297+0000 7f4959de7b80 1 -- 172.21.15.5:0/976992280 shutdown_connections 2024-05-09T00:46:14.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.297+0000 7f4959de7b80 1 --2- 172.21.15.5:0/976992280 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x560c7c5bca00 0x560c7c5bde70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:14.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.297+0000 7f492cff9700 0 client.15021 ms_handle_reset on v2:172.21.15.157:6836/744863009 2024-05-09T00:46:14.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.297+0000 7f4959de7b80 1 --2- 172.21.15.5:0/976992280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7c535650 0x560c7c54b100 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:14.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.297+0000 7f4959de7b80 1 --2- 172.21.15.5:0/976992280 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c7c3b8450 0x560c7c54a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:14.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.297+0000 7f4959de7b80 1 --2- 172.21.15.5:0/976992280 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c7c4a2110 0x560c7c54abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:14.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.297+0000 7f4959de7b80 1 -- 172.21.15.5:0/976992280 >> 172.21.15.5:0/976992280 conn(0x560c7c418650 msgr2=0x560c7c538db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:14.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.297+0000 7f4959de7b80 1 -- 172.21.15.5:0/976992280 shutdown_connections 2024-05-09T00:46:14.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.297+0000 7f4959de7b80 1 -- 172.21.15.5:0/976992280 wait complete. 2024-05-09T00:46:14.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.297+0000 7f4959de7b80 20 client.15021 trim_cache size 0 max 16384 2024-05-09T00:46:14.303 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-09T00:46:14.304 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "d85814c6-0d9b-11ef-bc97-c7b262605968", 2024-05-09T00:46:14.304 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 1, 2024-05-09T00:46:14.304 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24499, 2024-05-09T00:46:14.304 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-05-09T00:46:14.304 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-05-09T00:46:14.304 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-05-09T00:46:14.304 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 135.37560205700001, 2024-05-09T00:46:14.304 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 28, 2024-05-09T00:46:14.304 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 82, 2024-05-09T00:46:14.304 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 78, 2024-05-09T00:46:14.304 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 179.03045418900001 2024-05-09T00:46:14.304 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-09T00:46:14.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.297+0000 7f494b3d1700 1 -- 172.21.15.5:0/3025142068 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f49300b2900 msgr2=0x7f49300b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:14.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.297+0000 7f494b3d1700 1 --2- 172.21.15.5:0/3025142068 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f49300b2900 0x7f49300b4e10 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4940009dd0 tx=0x7f4940009040 comp rx=0 tx=0).stop 2024-05-09T00:46:14.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.297+0000 7f494b3d1700 1 -- 172.21.15.5:0/3025142068 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4944108740 msgr2=0x7f49440ffd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:14.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.297+0000 7f494b3d1700 1 --2- 172.21.15.5:0/3025142068 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4944108740 0x7f49440ffd20 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f493c005910 tx=0x7f493c009580 comp rx=0 tx=0).stop 2024-05-09T00:46:14.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.298+0000 7f494b3d1700 1 -- 172.21.15.5:0/3025142068 shutdown_connections 2024-05-09T00:46:14.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.298+0000 7f494b3d1700 1 --2- 172.21.15.5:0/3025142068 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f49300b2900 0x7f49300b4e10 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:14.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.298+0000 7f494b3d1700 1 --2- 172.21.15.5:0/3025142068 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4944108740 0x7f49440ffd20 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:14.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.298+0000 7f494b3d1700 1 --2- 172.21.15.5:0/3025142068 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49441034c0 0x7f49440ff2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:14.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.298+0000 7f494b3d1700 1 --2- 172.21.15.5:0/3025142068 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4944107df0 0x7f49440ff7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:14.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.298+0000 7f494b3d1700 1 -- 172.21.15.5:0/3025142068 >> 172.21.15.5:0/3025142068 conn(0x7f49440fedc0 msgr2=0x7f494406b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:14.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.298+0000 7f494b3d1700 1 -- 172.21.15.5:0/3025142068 shutdown_connections 2024-05-09T00:46:14.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:14.298+0000 7f494b3d1700 1 -- 172.21.15.5:0/3025142068 wait complete. 2024-05-09T00:46:14.940 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.i status 2024-05-09T00:46:15.199 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:46:15.587 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:46:15 smithi005 ceph-mon[30365]: pgmap v368: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-09T00:46:15.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:46:15 smithi131 ceph-mon[34547]: pgmap v368: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-09T00:46:15.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:46:15 smithi157 ceph-mon[34294]: pgmap v368: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-09T00:46:16.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.880+0000 7f274acbc700 1 -- 172.21.15.5:0/2133314441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27441034c0 msgr2=0x7f27441038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:16.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.880+0000 7f274acbc700 1 --2- 172.21.15.5:0/2133314441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27441034c0 0x7f27441038d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f27400099c0 tx=0x7f2740009cd0 comp rx=0 tx=0).stop 2024-05-09T00:46:16.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.880+0000 7f274acbc700 1 -- 172.21.15.5:0/2133314441 shutdown_connections 2024-05-09T00:46:16.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.880+0000 7f274acbc700 1 --2- 172.21.15.5:0/2133314441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2744108740 0x7f27441100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:16.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.880+0000 7f274acbc700 1 --2- 172.21.15.5:0/2133314441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27441034c0 0x7f27441038d0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:16.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.880+0000 7f274acbc700 1 --2- 172.21.15.5:0/2133314441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2744107df0 0x7f2744108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:16.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.880+0000 7f274acbc700 1 -- 172.21.15.5:0/2133314441 >> 172.21.15.5:0/2133314441 conn(0x7f27440fedc0 msgr2=0x7f27441011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:16.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.880+0000 7f274acbc700 1 -- 172.21.15.5:0/2133314441 shutdown_connections 2024-05-09T00:46:16.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.880+0000 7f274acbc700 1 -- 172.21.15.5:0/2133314441 wait complete. 2024-05-09T00:46:16.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.881+0000 7f274acbc700 1 Processor -- start 2024-05-09T00:46:16.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.881+0000 7f274acbc700 1 -- start start 2024-05-09T00:46:16.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.881+0000 7f274acbc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27441034c0 0x7f274419ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:16.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.881+0000 7f274acbc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2744107df0 0x7f274419b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:16.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.881+0000 7f274acbc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2744108740 0x7f274419f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:16.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.881+0000 7f274acbc700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2744113370 con 0x7f2744108740 2024-05-09T00:46:16.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.881+0000 7f274acbc700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f27441131f0 con 0x7f27441034c0 2024-05-09T00:46:16.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.881+0000 7f274acbc700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f27441134f0 con 0x7f2744107df0 2024-05-09T00:46:16.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.881+0000 7f273bfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2744107df0 0x7f274419b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:16.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.881+0000 7f2749259700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2744108740 0x7f274419f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:16.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.881+0000 7f2748a58700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27441034c0 0x7f274419ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:16.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.881+0000 7f273bfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2744107df0 0x7f274419b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:42832/0 (socket says 172.21.15.5:42832) 2024-05-09T00:46:16.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.881+0000 7f273bfff700 1 -- 172.21.15.5:0/528033911 learned_addr learned my addr 172.21.15.5:0/528033911 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:46:16.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.882+0000 7f2749259700 1 -- 172.21.15.5:0/528033911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2744107df0 msgr2=0x7f274419b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:16.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.882+0000 7f2749259700 1 --2- 172.21.15.5:0/528033911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2744107df0 0x7f274419b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:16.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.882+0000 7f2749259700 1 -- 172.21.15.5:0/528033911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27441034c0 msgr2=0x7f274419ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:16.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.882+0000 7f2749259700 1 --2- 172.21.15.5:0/528033911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27441034c0 0x7f274419ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:16.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.882+0000 7f2749259700 1 -- 172.21.15.5:0/528033911 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f274419fce0 con 0x7f2744108740 2024-05-09T00:46:16.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.882+0000 7f273bfff700 1 --2- 172.21.15.5:0/528033911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2744107df0 0x7f274419b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:46:16.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.882+0000 7f2749259700 1 --2- 172.21.15.5:0/528033911 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2744108740 0x7f274419f640 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f273c00efc0 tx=0x7f273c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:16.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.882+0000 7f2739ffb700 1 -- 172.21.15.5:0/528033911 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273c0092e0 con 0x7f2744108740 2024-05-09T00:46:16.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.882+0000 7f274acbc700 1 -- 172.21.15.5:0/528033911 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f274419ff70 con 0x7f2744108740 2024-05-09T00:46:16.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.882+0000 7f2739ffb700 1 -- 172.21.15.5:0/528033911 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f273c004bb0 con 0x7f2744108740 2024-05-09T00:46:16.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.882+0000 7f2739ffb700 1 -- 172.21.15.5:0/528033911 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273c004020 con 0x7f2744108740 2024-05-09T00:46:16.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.882+0000 7f274acbc700 1 -- 172.21.15.5:0/528033911 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27440ff1d0 con 0x7f2744108740 2024-05-09T00:46:16.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.885+0000 7f2739ffb700 1 -- 172.21.15.5:0/528033911 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f273c01b030 con 0x7f2744108740 2024-05-09T00:46:16.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.886+0000 7f2739ffb700 1 --2- 172.21.15.5:0/528033911 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f272c0b2900 0x7f272c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:16.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.886+0000 7f2748a58700 1 --2- 172.21.15.5:0/528033911 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f272c0b2900 0x7f272c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:16.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.886+0000 7f2739ffb700 1 -- 172.21.15.5:0/528033911 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f273c0e1cb0 con 0x7f2744108740 2024-05-09T00:46:16.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.886+0000 7f27596d2b80 1 Processor -- start 2024-05-09T00:46:16.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.886+0000 7f2748a58700 1 --2- 172.21.15.5:0/528033911 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f272c0b2900 0x7f272c0b4e10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2740000c00 tx=0x7f274001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:16.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.886+0000 7f27596d2b80 1 -- start start 2024-05-09T00:46:16.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.887+0000 7f27596d2b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d508240cb0 0x55d5083491f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:16.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.887+0000 7f27596d2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d5081c6450 0x55d508349730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:16.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.887+0000 7f27596d2b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5083433b0 0x55d5083437c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:16.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.887+0000 7f27596d2b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d508266050 con 0x55d5083433b0 2024-05-09T00:46:16.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.887+0000 7f27596d2b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55d50820b950 con 0x55d5081c6450 2024-05-09T00:46:16.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.887+0000 7f27596d2b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d508262010 con 0x55d508240cb0 2024-05-09T00:46:16.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.887+0000 7f2749259700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5083433b0 0x55d5083437c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:16.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.887+0000 7f2749259700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5083433b0 0x55d5083437c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53332/0 (socket says 172.21.15.5:53332) 2024-05-09T00:46:16.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.887+0000 7f2749259700 1 -- 172.21.15.5:0/3193958734 learned_addr learned my addr 172.21.15.5:0/3193958734 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:46:16.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.887+0000 7f273bfff700 1 --2- 172.21.15.5:0/3193958734 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d5081c6450 0x55d508349730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:16.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.887+0000 7f2748a58700 1 --2- 172.21.15.5:0/3193958734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d508240cb0 0x55d5083491f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:16.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.887+0000 7f2749259700 1 -- 172.21.15.5:0/3193958734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d508240cb0 msgr2=0x55d5083491f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:16.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.887+0000 7f2749259700 1 --2- 172.21.15.5:0/3193958734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d508240cb0 0x55d5083491f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:16.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.887+0000 7f2749259700 1 -- 172.21.15.5:0/3193958734 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d5081c6450 msgr2=0x55d508349730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:16.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.887+0000 7f2749259700 1 --2- 172.21.15.5:0/3193958734 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d5081c6450 0x55d508349730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:16.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.887+0000 7f2749259700 1 -- 172.21.15.5:0/3193958734 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5081fc4b0 con 0x55d5083433b0 2024-05-09T00:46:16.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.887+0000 7f2748a58700 1 --2- 172.21.15.5:0/3193958734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d508240cb0 0x55d5083491f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:46:16.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.887+0000 7f2749259700 1 --2- 172.21.15.5:0/3193958734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5083433b0 0x55d5083437c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f273c00c6f0 tx=0x7f273c0a91d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f00440f60e7c7171 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:16.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.887+0000 7f272affd700 1 -- 172.21.15.5:0/3193958734 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273c0a92d0 con 0x55d5083433b0 2024-05-09T00:46:16.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.888+0000 7f272affd700 1 -- 172.21.15.5:0/3193958734 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f273c0ad680 con 0x55d5083433b0 2024-05-09T00:46:16.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.888+0000 7f272affd700 1 -- 172.21.15.5:0/3193958734 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273c0a8530 con 0x55d5083433b0 2024-05-09T00:46:16.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.888+0000 7f27596d2b80 1 -- 172.21.15.5:0/3193958734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5083433b0 msgr2=0x55d5083437c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:16.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.888+0000 7f27596d2b80 1 --2- 172.21.15.5:0/3193958734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5083433b0 0x55d5083437c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f273c00c6f0 tx=0x7f273c0a91d0 comp rx=0 tx=0).stop 2024-05-09T00:46:16.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.888+0000 7f27596d2b80 1 -- 172.21.15.5:0/3193958734 shutdown_connections 2024-05-09T00:46:16.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.888+0000 7f27596d2b80 1 --2- 172.21.15.5:0/3193958734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5083433b0 0x55d5083437c0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:16.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.888+0000 7f27596d2b80 1 --2- 172.21.15.5:0/3193958734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d508240cb0 0x55d5083491f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:16.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.888+0000 7f27596d2b80 1 --2- 172.21.15.5:0/3193958734 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d5081c6450 0x55d508349730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:16.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.888+0000 7f27596d2b80 1 -- 172.21.15.5:0/3193958734 >> 172.21.15.5:0/3193958734 conn(0x55d5081c9f20 msgr2=0x55d50833e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:16.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.889+0000 7f27596d2b80 1 -- 172.21.15.5:0/3193958734 shutdown_connections 2024-05-09T00:46:16.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.889+0000 7f27596d2b80 1 -- 172.21.15.5:0/3193958734 wait complete. 2024-05-09T00:46:16.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.889+0000 7f27596d2b80 1 -- start start 2024-05-09T00:46:16.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.889+0000 7f27596d2b80 1 Processor -- start 2024-05-09T00:46:16.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.890+0000 7f27596d2b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d5081c6450 0x55d50834cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:16.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.890+0000 7f27596d2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d508240cb0 0x55d50834d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:16.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.890+0000 7f27596d2b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5083433b0 0x55d50834d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:16.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.890+0000 7f27596d2b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d5083440b0 con 0x55d5083433b0 2024-05-09T00:46:16.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.890+0000 7f27596d2b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55d508200ad0 con 0x55d508240cb0 2024-05-09T00:46:16.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.890+0000 7f27596d2b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d508359180 con 0x55d5081c6450 2024-05-09T00:46:16.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.890+0000 7f27596d2b80 10 client.0 _ll_register_callbacks cb 0x55d5080d7670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:46:16.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.890+0000 7f2749259700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5083433b0 0x55d50834d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:16.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.890+0000 7f273bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d508240cb0 0x55d50834d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:16.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.890+0000 7f2749259700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5083433b0 0x55d50834d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53340/0 (socket says 172.21.15.5:53340) 2024-05-09T00:46:16.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.890+0000 7f2749259700 1 -- 172.21.15.5:0/1845786988 learned_addr learned my addr 172.21.15.5:0/1845786988 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:46:16.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.890+0000 7f2748a58700 1 --2- 172.21.15.5:0/1845786988 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d5081c6450 0x55d50834cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:16.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.890+0000 7f2749259700 1 -- 172.21.15.5:0/1845786988 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d5081c6450 msgr2=0x55d50834cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:16.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.890+0000 7f2749259700 1 --2- 172.21.15.5:0/1845786988 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d5081c6450 0x55d50834cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:16.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.890+0000 7f2749259700 1 -- 172.21.15.5:0/1845786988 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d508240cb0 msgr2=0x55d50834d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:16.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.890+0000 7f2749259700 1 --2- 172.21.15.5:0/1845786988 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d508240cb0 0x55d50834d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:16.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.890+0000 7f2749259700 1 -- 172.21.15.5:0/1845786988 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5080e1440 con 0x55d5083433b0 2024-05-09T00:46:16.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.890+0000 7f2748a58700 1 --2- 172.21.15.5:0/1845786988 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d5081c6450 0x55d50834cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:46:16.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.890+0000 7f2749259700 1 --2- 172.21.15.5:0/1845786988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5083433b0 0x55d50834d9c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f273c0a9930 tx=0x7f273c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df1bb81dad3e0804 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:16.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.890+0000 7f2723fff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:46:16.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.890+0000 7f2723fff700 1 -- 172.21.15.5:0/1845786988 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273c0c2070 con 0x55d5083433b0 2024-05-09T00:46:16.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.890+0000 7f2723fff700 1 -- 172.21.15.5:0/1845786988 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f273c0ad380 con 0x55d5083433b0 2024-05-09T00:46:16.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.890+0000 7f27596d2b80 1 -- 172.21.15.5:0/1845786988 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f273c0ad680 con 0x55d5083433b0 2024-05-09T00:46:16.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.891+0000 7f2723fff700 1 -- 172.21.15.5:0/1845786988 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f273c0a92d0 con 0x55d5083433b0 2024-05-09T00:46:16.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.891+0000 7f27596d2b80 10 client.15033 fetch_fsmap learned FSMap version 28 2024-05-09T00:46:16.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.891+0000 7f27596d2b80 1 -- 172.21.15.5:0/1845786988 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d50835aa20 con 0x55d5083433b0 2024-05-09T00:46:16.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.891+0000 7f2723fff700 1 -- 172.21.15.5:0/1845786988 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f273c0ad680 con 0x55d5083433b0 2024-05-09T00:46:16.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.891+0000 7f2723fff700 1 -- 172.21.15.5:0/1845786988 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f273c0a7fa0 con 0x55d5083433b0 2024-05-09T00:46:16.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.891+0000 7f27596d2b80 10 client.15033 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T00:46:16.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.891+0000 7f27596d2b80 10 client.15033 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-09T00:46:16.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.891+0000 7f27596d2b80 20 client.15033 populate_metadata read hostname 'smithi005' 2024-05-09T00:46:16.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.891+0000 7f27596d2b80 1 --2- 172.21.15.5:0/1845786988 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55d50835cb20 0x55d50835df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:16.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.891+0000 7f27596d2b80 4 client.15033 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:46:16.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.891+0000 7f27596d2b80 1 -- 172.21.15.5:0/1845786988 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d508273a30 con 0x55d50835cb20 2024-05-09T00:46:16.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.891+0000 7f2748a58700 1 --2- 172.21.15.5:0/1845786988 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55d50835cb20 0x55d50835df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:16.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.892+0000 7f2723fff700 10 client.15033 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:46:16.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.892+0000 7f2748a58700 1 --2- 172.21.15.5:0/1845786988 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55d50835cb20 0x55d50835df90 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50dfeb41761ddefd server_cookie=39643427dda849bb in_seq=0 out_seq=0 2024-05-09T00:46:16.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.893+0000 7f2723fff700 1 -- 172.21.15.5:0/1845786988 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d508273a30 con 0x55d50835cb20 2024-05-09T00:46:16.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.893+0000 7f2723fff700 10 client.15033 handle_command_reply: tid=0 2024-05-09T00:46:16.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.893+0000 7f27596d2b80 1 client.15033 shutdown 2024-05-09T00:46:16.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.893+0000 7f27596d2b80 1 -- 172.21.15.5:0/1845786988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5083433b0 msgr2=0x55d50834d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:16.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.893+0000 7f27596d2b80 1 --2- 172.21.15.5:0/1845786988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5083433b0 0x55d50834d9c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f273c0a9930 tx=0x7f273c00c810 comp rx=0 tx=0).stop 2024-05-09T00:46:16.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.893+0000 7f27596d2b80 1 -- 172.21.15.5:0/1845786988 shutdown_connections 2024-05-09T00:46:16.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.893+0000 7f27596d2b80 1 --2- 172.21.15.5:0/1845786988 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55d50835cb20 0x55d50835df90 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:16.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.893+0000 7f27596d2b80 1 --2- 172.21.15.5:0/1845786988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5083433b0 0x55d50834d9c0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:16.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.893+0000 7f27596d2b80 1 --2- 172.21.15.5:0/1845786988 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d5081c6450 0x55d50834cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:16.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.893+0000 7f27596d2b80 1 --2- 172.21.15.5:0/1845786988 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d508240cb0 0x55d50834d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:16.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.893+0000 7f2723fff700 0 client.15033 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:46:16.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.893+0000 7f27596d2b80 1 -- 172.21.15.5:0/1845786988 >> 172.21.15.5:0/1845786988 conn(0x55d5081c9f20 msgr2=0x55d508337fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:16.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.893+0000 7f27596d2b80 1 -- 172.21.15.5:0/1845786988 shutdown_connections 2024-05-09T00:46:16.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.893+0000 7f27596d2b80 1 -- 172.21.15.5:0/1845786988 wait complete. 2024-05-09T00:46:16.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.893+0000 7f27596d2b80 20 client.15033 trim_cache size 0 max 16384 2024-05-09T00:46:16.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.914+0000 7f27596d2b80 1 Processor -- start 2024-05-09T00:46:16.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.914+0000 7f27596d2b80 1 -- start start 2024-05-09T00:46:16.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.915+0000 7f27596d2b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d5081c6450 0x55d5083590f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:16.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.915+0000 7f27596d2b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d508240cb0 0x55d50834bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:16.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.915+0000 7f27596d2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d5083433b0 0x55d508337fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:16.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.915+0000 7f27596d2b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d50835c2d0 con 0x55d508240cb0 2024-05-09T00:46:16.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.915+0000 7f27596d2b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55d50835c9a0 con 0x55d5083433b0 2024-05-09T00:46:16.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.915+0000 7f27596d2b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d50835d070 con 0x55d5081c6450 2024-05-09T00:46:16.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.915+0000 7f273bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d508240cb0 0x55d50834bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:16.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.915+0000 7f273bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d508240cb0 0x55d50834bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53344/0 (socket says 172.21.15.5:53344) 2024-05-09T00:46:16.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.915+0000 7f273bfff700 1 -- 172.21.15.5:0/461895188 learned_addr learned my addr 172.21.15.5:0/461895188 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:46:16.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.915+0000 7f2748a58700 1 --2- 172.21.15.5:0/461895188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d5081c6450 0x55d5083590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:16.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.915+0000 7f2749259700 1 --2- 172.21.15.5:0/461895188 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d5083433b0 0x55d508337fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:16.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.915+0000 7f273bfff700 1 -- 172.21.15.5:0/461895188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d5081c6450 msgr2=0x55d5083590f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:16.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.915+0000 7f273bfff700 1 --2- 172.21.15.5:0/461895188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d5081c6450 0x55d5083590f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:16.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.915+0000 7f273bfff700 1 -- 172.21.15.5:0/461895188 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d5083433b0 msgr2=0x55d508337fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:16.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.915+0000 7f273bfff700 1 --2- 172.21.15.5:0/461895188 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d5083433b0 0x55d508337fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:16.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.915+0000 7f273bfff700 1 -- 172.21.15.5:0/461895188 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d50833a980 con 0x55d508240cb0 2024-05-09T00:46:16.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.915+0000 7f2749259700 1 --2- 172.21.15.5:0/461895188 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d5083433b0 0x55d508337fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:46:16.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.915+0000 7f273bfff700 1 --2- 172.21.15.5:0/461895188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d508240cb0 0x55d50834bf40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2734004410 tx=0x7f2734003960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2703c979b73214f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:16.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.915+0000 7f272a7fc700 1 -- 172.21.15.5:0/461895188 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2734004500 con 0x55d508240cb0 2024-05-09T00:46:16.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.916+0000 7f272a7fc700 1 -- 172.21.15.5:0/461895188 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27340118f0 con 0x55d508240cb0 2024-05-09T00:46:16.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.916+0000 7f272a7fc700 1 -- 172.21.15.5:0/461895188 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273401b890 con 0x55d508240cb0 2024-05-09T00:46:16.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.916+0000 7f27596d2b80 1 -- 172.21.15.5:0/461895188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d508240cb0 msgr2=0x55d50834bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:16.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.916+0000 7f27596d2b80 1 --2- 172.21.15.5:0/461895188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d508240cb0 0x55d50834bf40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2734004410 tx=0x7f2734003960 comp rx=0 tx=0).stop 2024-05-09T00:46:16.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.916+0000 7f27596d2b80 1 -- 172.21.15.5:0/461895188 shutdown_connections 2024-05-09T00:46:16.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.916+0000 7f27596d2b80 1 --2- 172.21.15.5:0/461895188 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d5083433b0 0x55d508337fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:16.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.916+0000 7f27596d2b80 1 --2- 172.21.15.5:0/461895188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d5081c6450 0x55d5083590f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:16.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.916+0000 7f27596d2b80 1 --2- 172.21.15.5:0/461895188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d508240cb0 0x55d50834bf40 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:16.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.916+0000 7f27596d2b80 1 -- 172.21.15.5:0/461895188 >> 172.21.15.5:0/461895188 conn(0x55d5081c9f20 msgr2=0x55d50834b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:16.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.916+0000 7f27596d2b80 1 -- 172.21.15.5:0/461895188 shutdown_connections 2024-05-09T00:46:16.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.916+0000 7f27596d2b80 1 -- 172.21.15.5:0/461895188 wait complete. 2024-05-09T00:46:16.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.917+0000 7f27596d2b80 1 -- start start 2024-05-09T00:46:16.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.917+0000 7f27596d2b80 1 Processor -- start 2024-05-09T00:46:16.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.917+0000 7f27596d2b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5081c6450 0x55d508367410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:16.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.917+0000 7f27596d2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d508240cb0 0x55d508367950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:16.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.917+0000 7f27596d2b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d5083433b0 0x55d5083623c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:16.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.918+0000 7f27596d2b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d50835de10 con 0x55d5081c6450 2024-05-09T00:46:16.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.918+0000 7f27596d2b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55d50835d740 con 0x55d508240cb0 2024-05-09T00:46:16.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.918+0000 7f27596d2b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d50835e4e0 con 0x55d5083433b0 2024-05-09T00:46:16.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.918+0000 7f2748a58700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5081c6450 0x55d508367410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:16.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.918+0000 7f27596d2b80 10 client.0 _ll_register_callbacks cb 0x55d5080d7670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:46:16.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.918+0000 7f2748a58700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5081c6450 0x55d508367410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53356/0 (socket says 172.21.15.5:53356) 2024-05-09T00:46:16.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.918+0000 7f2748a58700 1 -- 172.21.15.5:0/1799300972 learned_addr learned my addr 172.21.15.5:0/1799300972 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:46:16.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.918+0000 7f273bfff700 1 --2- 172.21.15.5:0/1799300972 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d508240cb0 0x55d508367950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:16.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.918+0000 7f2749259700 1 --2- 172.21.15.5:0/1799300972 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d5083433b0 0x55d5083623c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:16.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.918+0000 7f2748a58700 1 -- 172.21.15.5:0/1799300972 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d5083433b0 msgr2=0x55d5083623c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:16.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.918+0000 7f2748a58700 1 --2- 172.21.15.5:0/1799300972 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d5083433b0 0x55d5083623c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:16.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.918+0000 7f2748a58700 1 -- 172.21.15.5:0/1799300972 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d508240cb0 msgr2=0x55d508367950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:16.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.918+0000 7f2748a58700 1 --2- 172.21.15.5:0/1799300972 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d508240cb0 0x55d508367950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:16.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.918+0000 7f2748a58700 1 -- 172.21.15.5:0/1799300972 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d508362950 con 0x55d5081c6450 2024-05-09T00:46:16.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.918+0000 7f2749259700 1 --2- 172.21.15.5:0/1799300972 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d5083433b0 0x55d5083623c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:46:16.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.918+0000 7f2748a58700 1 --2- 172.21.15.5:0/1799300972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5081c6450 0x55d508367410 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f27400099c0 tx=0x7f2740005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=149197ecf2d48842 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:16.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.918+0000 7f2723fff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:46:16.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.918+0000 7f2723fff700 1 -- 172.21.15.5:0/1799300972 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2740037060 con 0x55d5081c6450 2024-05-09T00:46:16.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.918+0000 7f2723fff700 1 -- 172.21.15.5:0/1799300972 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f27400328f0 con 0x55d5081c6450 2024-05-09T00:46:16.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.918+0000 7f27596d2b80 1 -- 172.21.15.5:0/1799300972 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27340118f0 con 0x55d5081c6450 2024-05-09T00:46:16.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.919+0000 7f2723fff700 1 -- 172.21.15.5:0/1799300972 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f274000b790 con 0x55d5081c6450 2024-05-09T00:46:16.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.919+0000 7f27596d2b80 10 client.15039 fetch_fsmap learned FSMap version 28 2024-05-09T00:46:16.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.919+0000 7f27596d2b80 1 -- 172.21.15.5:0/1799300972 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d5083c87b0 con 0x55d5081c6450 2024-05-09T00:46:16.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.919+0000 7f2723fff700 1 -- 172.21.15.5:0/1799300972 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27340118f0 con 0x55d5081c6450 2024-05-09T00:46:16.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.919+0000 7f2723fff700 1 -- 172.21.15.5:0/1799300972 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2740021070 con 0x55d5081c6450 2024-05-09T00:46:16.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.919+0000 7f27596d2b80 10 client.15039 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T00:46:16.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.919+0000 7f27596d2b80 10 client.15039 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-09T00:46:16.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.919+0000 7f27596d2b80 20 client.15039 populate_metadata read hostname 'smithi005' 2024-05-09T00:46:16.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.919+0000 7f27596d2b80 1 --2- 172.21.15.5:0/1799300972 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55d5083cac80 0x55d5083cc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:16.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.919+0000 7f27596d2b80 4 client.15039 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T00:46:16.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.919+0000 7f27596d2b80 1 -- 172.21.15.5:0/1799300972 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "status"}) -- 0x55d50826dad0 con 0x55d5083cac80 2024-05-09T00:46:16.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.919+0000 7f273bfff700 1 --2- 172.21.15.5:0/1799300972 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55d5083cac80 0x55d5083cc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:16.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.920+0000 7f273bfff700 1 --2- 172.21.15.5:0/1799300972 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55d5083cac80 0x55d5083cc0f0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53e8113901e68224 server_cookie=cda8449c0dc30c89 in_seq=0 out_seq=0 2024-05-09T00:46:16.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.920+0000 7f2723fff700 10 client.15039 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:46:16.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.920+0000 7f2723fff700 1 -- 172.21.15.5:0/1799300972 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55d50826dad0 con 0x55d5083cac80 2024-05-09T00:46:16.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.920+0000 7f2723fff700 10 client.15039 handle_command_reply: tid=0 2024-05-09T00:46:16.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.920+0000 7f27596d2b80 1 client.15039 shutdown 2024-05-09T00:46:16.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.920+0000 7f27596d2b80 1 -- 172.21.15.5:0/1799300972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5081c6450 msgr2=0x55d508367410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:16.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.920+0000 7f27596d2b80 1 --2- 172.21.15.5:0/1799300972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5081c6450 0x55d508367410 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f27400099c0 tx=0x7f2740005040 comp rx=0 tx=0).stop 2024-05-09T00:46:16.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.920+0000 7f27596d2b80 1 -- 172.21.15.5:0/1799300972 shutdown_connections 2024-05-09T00:46:16.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.920+0000 7f27596d2b80 1 --2- 172.21.15.5:0/1799300972 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55d5083cac80 0x55d5083cc0f0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:16.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.920+0000 7f27596d2b80 1 --2- 172.21.15.5:0/1799300972 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d5083433b0 0x55d5083623c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:16.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.920+0000 7f27596d2b80 1 --2- 172.21.15.5:0/1799300972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5081c6450 0x55d508367410 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:16.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.920+0000 7f27596d2b80 1 --2- 172.21.15.5:0/1799300972 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d508240cb0 0x55d508367950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:16.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.920+0000 7f27596d2b80 1 -- 172.21.15.5:0/1799300972 >> 172.21.15.5:0/1799300972 conn(0x55d5081c9f20 msgr2=0x55d508348830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:16.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.920+0000 7f2723fff700 0 client.15039 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:46:16.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.921+0000 7f27596d2b80 1 -- 172.21.15.5:0/1799300972 shutdown_connections 2024-05-09T00:46:16.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.921+0000 7f27596d2b80 1 -- 172.21.15.5:0/1799300972 wait complete. 2024-05-09T00:46:16.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.921+0000 7f27596d2b80 20 client.15039 trim_cache size 0 max 16384 2024-05-09T00:46:16.927 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-09T00:46:16.927 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "d85814c6-0d9b-11ef-bc97-c7b262605968", 2024-05-09T00:46:16.927 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-05-09T00:46:16.927 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24508, 2024-05-09T00:46:16.927 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-05-09T00:46:16.928 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-05-09T00:46:16.928 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-05-09T00:46:16.928 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 151.03337277599999, 2024-05-09T00:46:16.928 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 28, 2024-05-09T00:46:16.928 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 82, 2024-05-09T00:46:16.928 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 78, 2024-05-09T00:46:16.928 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 175.948473347 2024-05-09T00:46:16.928 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-09T00:46:16.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.921+0000 7f274acbc700 1 -- 172.21.15.5:0/528033911 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f272c0b2900 msgr2=0x7f272c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:16.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.921+0000 7f274acbc700 1 --2- 172.21.15.5:0/528033911 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f272c0b2900 0x7f272c0b4e10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2740000c00 tx=0x7f274001a040 comp rx=0 tx=0).stop 2024-05-09T00:46:16.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.921+0000 7f274acbc700 1 -- 172.21.15.5:0/528033911 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2744108740 msgr2=0x7f274419f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:16.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.921+0000 7f274acbc700 1 --2- 172.21.15.5:0/528033911 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2744108740 0x7f274419f640 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f273c00efc0 tx=0x7f273c00c3a0 comp rx=0 tx=0).stop 2024-05-09T00:46:16.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.921+0000 7f274acbc700 1 -- 172.21.15.5:0/528033911 shutdown_connections 2024-05-09T00:46:16.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.921+0000 7f274acbc700 1 --2- 172.21.15.5:0/528033911 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f272c0b2900 0x7f272c0b4e10 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:16.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.921+0000 7f274acbc700 1 --2- 172.21.15.5:0/528033911 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2744108740 0x7f274419f640 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:16.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.921+0000 7f274acbc700 1 --2- 172.21.15.5:0/528033911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27441034c0 0x7f274419ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:16.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.921+0000 7f274acbc700 1 --2- 172.21.15.5:0/528033911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2744107df0 0x7f274419b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:16.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.921+0000 7f274acbc700 1 -- 172.21.15.5:0/528033911 >> 172.21.15.5:0/528033911 conn(0x7f27440fedc0 msgr2=0x7f274406b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:16.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.921+0000 7f274acbc700 1 -- 172.21.15.5:0/528033911 shutdown_connections 2024-05-09T00:46:16.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:16.922+0000 7f274acbc700 1 -- 172.21.15.5:0/528033911 wait complete. 2024-05-09T00:46:17.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:46:17 smithi005 ceph-mon[30365]: pgmap v369: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T00:46:17.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:46:17 smithi131 ceph-mon[34547]: pgmap v369: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T00:46:17.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:46:17 smithi157 ceph-mon[34294]: pgmap v369: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T00:46:17.750 INFO:tasks.cephadm:Setup complete, yielding 2024-05-09T00:46:17.751 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-09T00:46:17.761 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi005.front.sepia.ceph.com 2024-05-09T00:46:17.761 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- bash -c 'ceph orch status' 2024-05-09T00:46:18.016 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:46:19.416 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:46:19 smithi005 ceph-mon[30365]: pgmap v370: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T00:46:19.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:46:19 smithi131 ceph-mon[34547]: pgmap v370: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T00:46:19.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:46:19 smithi157 ceph-mon[34294]: pgmap v370: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T00:46:19.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:19.788+0000 7f2a8d155700 1 -- 172.21.15.5:0/3290288367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a88103f60 msgr2=0x7f2a88104370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:19.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:19.788+0000 7f2a8d155700 1 --2- 172.21.15.5:0/3290288367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a88103f60 0x7f2a88104370 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2a70009990 tx=0x7f2a70009ca0 comp rx=0 tx=0).stop 2024-05-09T00:46:19.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:19.788+0000 7f2a8d155700 1 -- 172.21.15.5:0/3290288367 shutdown_connections 2024-05-09T00:46:19.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:19.788+0000 7f2a8d155700 1 --2- 172.21.15.5:0/3290288367 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a881049e0 0x7f2a8810c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:19.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:19.788+0000 7f2a8d155700 1 --2- 172.21.15.5:0/3290288367 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a881059b0 0x7f2a88105dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:19.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:19.788+0000 7f2a8d155700 1 --2- 172.21.15.5:0/3290288367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a88103f60 0x7f2a88104370 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:19.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:19.788+0000 7f2a8d155700 1 -- 172.21.15.5:0/3290288367 >> 172.21.15.5:0/3290288367 conn(0x7f2a880ff9d0 msgr2=0x7f2a88101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:19.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:19.788+0000 7f2a8d155700 1 -- 172.21.15.5:0/3290288367 shutdown_connections 2024-05-09T00:46:19.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:19.788+0000 7f2a8d155700 1 -- 172.21.15.5:0/3290288367 wait complete. 2024-05-09T00:46:19.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:19.789+0000 7f2a8d155700 1 Processor -- start 2024-05-09T00:46:19.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:19.789+0000 7f2a8d155700 1 -- start start 2024-05-09T00:46:19.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:19.790+0000 7f2a8d155700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a88103f60 0x7f2a88078570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:19.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:19.790+0000 7f2a8d155700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a881049e0 0x7f2a88076c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:19.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:19.790+0000 7f2a8d155700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a881059b0 0x7f2a88077160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:19.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:19.790+0000 7f2a8d155700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8810fef0 con 0x7f2a881049e0 2024-05-09T00:46:19.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:19.790+0000 7f2a8d155700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8810fd70 con 0x7f2a881059b0 2024-05-09T00:46:19.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:19.790+0000 7f2a8d155700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2a88110070 con 0x7f2a88103f60 2024-05-09T00:46:19.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:19.790+0000 7f2a8659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a881049e0 0x7f2a88076c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:19.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:19.790+0000 7f2a8659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a881049e0 0x7f2a88076c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53378/0 (socket says 172.21.15.5:53378) 2024-05-09T00:46:19.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:19.790+0000 7f2a8659c700 1 -- 172.21.15.5:0/1276337301 learned_addr learned my addr 172.21.15.5:0/1276337301 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:46:19.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:19.790+0000 7f2a8759e700 1 --2- 172.21.15.5:0/1276337301 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a881059b0 0x7f2a88077160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:19.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:19.790+0000 7f2a86d9d700 1 --2- 172.21.15.5:0/1276337301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a88103f60 0x7f2a88078570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:19.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:19.790+0000 7f2a8659c700 1 -- 172.21.15.5:0/1276337301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a88103f60 msgr2=0x7f2a88078570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:19.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:19.790+0000 7f2a8659c700 1 --2- 172.21.15.5:0/1276337301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a88103f60 0x7f2a88078570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:19.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:19.790+0000 7f2a8659c700 1 -- 172.21.15.5:0/1276337301 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a881059b0 msgr2=0x7f2a88077160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:19.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:19.790+0000 7f2a8659c700 1 --2- 172.21.15.5:0/1276337301 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a881059b0 0x7f2a88077160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:19.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:19.790+0000 7f2a8659c700 1 -- 172.21.15.5:0/1276337301 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a880778c0 con 0x7f2a881049e0 2024-05-09T00:46:19.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:19.790+0000 7f2a86d9d700 1 --2- 172.21.15.5:0/1276337301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a88103f60 0x7f2a88078570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:46:19.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:19.790+0000 7f2a8659c700 1 --2- 172.21.15.5:0/1276337301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a881049e0 0x7f2a88076c20 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2a70006010 tx=0x7f2a700047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:19.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:19.791+0000 7f2a7ffff700 1 -- 172.21.15.5:0/1276337301 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a70027070 con 0x7f2a881049e0 2024-05-09T00:46:19.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:19.791+0000 7f2a8d155700 1 -- 172.21.15.5:0/1276337301 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a881a4ad0 con 0x7f2a881049e0 2024-05-09T00:46:19.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:19.791+0000 7f2a7ffff700 1 -- 172.21.15.5:0/1276337301 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a700040c0 con 0x7f2a881049e0 2024-05-09T00:46:19.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:19.791+0000 7f2a7ffff700 1 -- 172.21.15.5:0/1276337301 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a70022d90 con 0x7f2a881049e0 2024-05-09T00:46:19.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:19.791+0000 7f2a8d155700 1 -- 172.21.15.5:0/1276337301 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a881a4e70 con 0x7f2a881049e0 2024-05-09T00:46:19.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:19.792+0000 7f2a8d155700 1 -- 172.21.15.5:0/1276337301 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a881106b0 con 0x7f2a881049e0 2024-05-09T00:46:19.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:19.793+0000 7f2a7ffff700 1 -- 172.21.15.5:0/1276337301 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f2a7002a070 con 0x7f2a881049e0 2024-05-09T00:46:19.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:19.794+0000 7f2a7ffff700 1 --2- 172.21.15.5:0/1276337301 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f2a740b2880 0x7f2a740b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:19.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:19.794+0000 7f2a86d9d700 1 --2- 172.21.15.5:0/1276337301 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f2a740b2880 0x7f2a740b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:19.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:19.794+0000 7f2a7ffff700 1 -- 172.21.15.5:0/1276337301 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f2a700e3450 con 0x7f2a881049e0 2024-05-09T00:46:19.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:19.795+0000 7f2a86d9d700 1 --2- 172.21.15.5:0/1276337301 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f2a740b2880 0x7f2a740b4d90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2a881053a0 tx=0x7f2a7800b5f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:19.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:19.798+0000 7f2a7ffff700 1 -- 172.21.15.5:0/1276337301 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2a70013070 con 0x7f2a881049e0 2024-05-09T00:46:20.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:20.031+0000 7f2a8d155700 1 -- 172.21.15.5:0/1276337301 --> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f2a8802e920 con 0x7f2a740b2880 2024-05-09T00:46:20.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:20.033+0000 7f2a7ffff700 1 -- 172.21.15.5:0/1276337301 <== mgr.14152 v2:172.21.15.5:6800/91856593 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f2a8802e920 con 0x7f2a740b2880 2024-05-09T00:46:20.033 INFO:teuthology.orchestra.run.smithi005.stdout:Backend: cephadm 2024-05-09T00:46:20.033 INFO:teuthology.orchestra.run.smithi005.stdout:Available: Yes 2024-05-09T00:46:20.034 INFO:teuthology.orchestra.run.smithi005.stdout:Paused: No 2024-05-09T00:46:20.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:20.035+0000 7f2a8d155700 1 -- 172.21.15.5:0/1276337301 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f2a740b2880 msgr2=0x7f2a740b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:20.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:20.035+0000 7f2a8d155700 1 --2- 172.21.15.5:0/1276337301 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f2a740b2880 0x7f2a740b4d90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2a881053a0 tx=0x7f2a7800b5f0 comp rx=0 tx=0).stop 2024-05-09T00:46:20.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:20.035+0000 7f2a8d155700 1 -- 172.21.15.5:0/1276337301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a881049e0 msgr2=0x7f2a88076c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:20.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:20.035+0000 7f2a8d155700 1 --2- 172.21.15.5:0/1276337301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a881049e0 0x7f2a88076c20 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2a70006010 tx=0x7f2a700047b0 comp rx=0 tx=0).stop 2024-05-09T00:46:20.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:20.035+0000 7f2a8d155700 1 -- 172.21.15.5:0/1276337301 shutdown_connections 2024-05-09T00:46:20.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:20.035+0000 7f2a8d155700 1 --2- 172.21.15.5:0/1276337301 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f2a740b2880 0x7f2a740b4d90 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:20.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:20.035+0000 7f2a8d155700 1 --2- 172.21.15.5:0/1276337301 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a881059b0 0x7f2a88077160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:20.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:20.035+0000 7f2a8d155700 1 --2- 172.21.15.5:0/1276337301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a88103f60 0x7f2a88078570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:20.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:20.035+0000 7f2a8d155700 1 --2- 172.21.15.5:0/1276337301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a881049e0 0x7f2a88076c20 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:20.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:20.035+0000 7f2a8d155700 1 -- 172.21.15.5:0/1276337301 >> 172.21.15.5:0/1276337301 conn(0x7f2a880ff9d0 msgr2=0x7f2a881086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:20.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:20.035+0000 7f2a8d155700 1 -- 172.21.15.5:0/1276337301 shutdown_connections 2024-05-09T00:46:20.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:20.036+0000 7f2a8d155700 1 -- 172.21.15.5:0/1276337301 wait complete. 2024-05-09T00:46:20.837 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-09T00:46:21.094 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:46:21.487 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:46:21 smithi005 ceph-mon[30365]: pgmap v371: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-09T00:46:21.487 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:46:21 smithi005 ceph-mon[30365]: from='client.15045 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:46:21.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:46:21 smithi131 ceph-mon[34547]: pgmap v371: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-09T00:46:21.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:46:21 smithi131 ceph-mon[34547]: from='client.15045 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:46:21.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:46:21 smithi157 ceph-mon[34294]: pgmap v371: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-09T00:46:21.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:46:21 smithi157 ceph-mon[34294]: from='client.15045 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:46:22.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:22.864+0000 7ffb38fa0700 1 -- 172.21.15.5:0/3299332430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb3410a0f0 msgr2=0x7ffb3410c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:22.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:22.864+0000 7ffb38fa0700 1 --2- 172.21.15.5:0/3299332430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb3410a0f0 0x7ffb3410c510 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c00b0a0 tx=0x7ffb2c00b3b0 comp rx=0 tx=0).stop 2024-05-09T00:46:22.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:22.864+0000 7ffb38fa0700 1 -- 172.21.15.5:0/3299332430 shutdown_connections 2024-05-09T00:46:22.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:22.864+0000 7ffb38fa0700 1 --2- 172.21.15.5:0/3299332430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb3410a0f0 0x7ffb3410c510 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:22.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:22.864+0000 7ffb38fa0700 1 --2- 172.21.15.5:0/3299332430 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb34104e30 0x7ffb34107250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:22.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:22.864+0000 7ffb38fa0700 1 --2- 172.21.15.5:0/3299332430 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb34107790 0x7ffb34109bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:22.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:22.864+0000 7ffb38fa0700 1 -- 172.21.15.5:0/3299332430 >> 172.21.15.5:0/3299332430 conn(0x7ffb340fe870 msgr2=0x7ffb34100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:22.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:22.864+0000 7ffb38fa0700 1 -- 172.21.15.5:0/3299332430 shutdown_connections 2024-05-09T00:46:22.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:22.864+0000 7ffb38fa0700 1 -- 172.21.15.5:0/3299332430 wait complete. 2024-05-09T00:46:22.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:22.865+0000 7ffb38fa0700 1 Processor -- start 2024-05-09T00:46:22.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:22.865+0000 7ffb38fa0700 1 -- start start 2024-05-09T00:46:22.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:22.865+0000 7ffb38fa0700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb34104e30 0x7ffb3419a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:22.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:22.865+0000 7ffb38fa0700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb34107790 0x7ffb3419ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:22.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:22.865+0000 7ffb38fa0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb3410a0f0 0x7ffb341a1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:22.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:22.865+0000 7ffb38fa0700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffb3410fd90 con 0x7ffb3410a0f0 2024-05-09T00:46:22.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:22.865+0000 7ffb38fa0700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ffb3410fc10 con 0x7ffb34104e30 2024-05-09T00:46:22.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:22.865+0000 7ffb38fa0700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ffb3410ff10 con 0x7ffb34107790 2024-05-09T00:46:22.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:22.866+0000 7ffb32d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb3410a0f0 0x7ffb341a1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:22.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:22.866+0000 7ffb3259c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb34104e30 0x7ffb3419a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:22.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:22.866+0000 7ffb32d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb3410a0f0 0x7ffb341a1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53396/0 (socket says 172.21.15.5:53396) 2024-05-09T00:46:22.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:22.866+0000 7ffb32d9d700 1 -- 172.21.15.5:0/1618836174 learned_addr learned my addr 172.21.15.5:0/1618836174 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:46:22.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:22.866+0000 7ffb31d9b700 1 --2- 172.21.15.5:0/1618836174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb34107790 0x7ffb3419ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:22.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:22.866+0000 7ffb32d9d700 1 -- 172.21.15.5:0/1618836174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb34107790 msgr2=0x7ffb3419ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:22.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:22.866+0000 7ffb32d9d700 1 --2- 172.21.15.5:0/1618836174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb34107790 0x7ffb3419ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:22.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:22.866+0000 7ffb32d9d700 1 -- 172.21.15.5:0/1618836174 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb34104e30 msgr2=0x7ffb3419a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:22.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:22.866+0000 7ffb32d9d700 1 --2- 172.21.15.5:0/1618836174 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb34104e30 0x7ffb3419a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:22.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:22.866+0000 7ffb32d9d700 1 -- 172.21.15.5:0/1618836174 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb341a23b0 con 0x7ffb3410a0f0 2024-05-09T00:46:22.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:22.866+0000 7ffb32d9d700 1 --2- 172.21.15.5:0/1618836174 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb3410a0f0 0x7ffb341a1d30 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c0062a0 tx=0x7ffb2c004e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:22.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:22.866+0000 7ffb237fe700 1 -- 172.21.15.5:0/1618836174 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c021070 con 0x7ffb3410a0f0 2024-05-09T00:46:22.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:22.866+0000 7ffb237fe700 1 -- 172.21.15.5:0/1618836174 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb2c007b80 con 0x7ffb3410a0f0 2024-05-09T00:46:22.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:22.866+0000 7ffb38fa0700 1 -- 172.21.15.5:0/1618836174 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb341a2640 con 0x7ffb3410a0f0 2024-05-09T00:46:22.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:22.866+0000 7ffb237fe700 1 -- 172.21.15.5:0/1618836174 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c004670 con 0x7ffb3410a0f0 2024-05-09T00:46:22.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:22.866+0000 7ffb38fa0700 1 -- 172.21.15.5:0/1618836174 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb341a2bd0 con 0x7ffb3410a0f0 2024-05-09T00:46:22.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:22.868+0000 7ffb38fa0700 1 -- 172.21.15.5:0/1618836174 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb340512c0 con 0x7ffb3410a0f0 2024-05-09T00:46:22.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:22.869+0000 7ffb237fe700 1 -- 172.21.15.5:0/1618836174 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7ffb2c003a00 con 0x7ffb3410a0f0 2024-05-09T00:46:22.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:22.871+0000 7ffb237fe700 1 --2- 172.21.15.5:0/1618836174 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ffb1c0b2900 0x7ffb1c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:22.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:22.871+0000 7ffb3259c700 1 --2- 172.21.15.5:0/1618836174 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ffb1c0b2900 0x7ffb1c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:22.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:22.871+0000 7ffb237fe700 1 -- 172.21.15.5:0/1618836174 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7ffb2c00f020 con 0x7ffb3410a0f0 2024-05-09T00:46:22.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:22.871+0000 7ffb3259c700 1 --2- 172.21.15.5:0/1618836174 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ffb1c0b2900 0x7ffb1c0b4e10 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ffb24005950 tx=0x7ffb2400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:22.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:22.875+0000 7ffb237fe700 1 -- 172.21.15.5:0/1618836174 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ffb2c02a020 con 0x7ffb3410a0f0 2024-05-09T00:46:23.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:23.100+0000 7ffb38fa0700 1 -- 172.21.15.5:0/1618836174 --> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7ffb34063c00 con 0x7ffb1c0b2900 2024-05-09T00:46:23.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:23.114+0000 7ffb237fe700 1 -- 172.21.15.5:0/1618836174 <== mgr.14152 v2:172.21.15.5:6800/91856593 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7ffb34063c00 con 0x7ffb1c0b2900 2024-05-09T00:46:23.115 INFO:teuthology.orchestra.run.smithi005.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-09T00:46:23.115 INFO:teuthology.orchestra.run.smithi005.stdout:mds.a smithi005 running (2m) 34s ago 2m 89.1M - 19.0.0-3594-gc9da4049 db794d706f49 bc99ce756e91 2024-05-09T00:46:23.115 INFO:teuthology.orchestra.run.smithi005.stdout:mds.b smithi131 running (2m) 36s ago 2m 89.7M - 19.0.0-3594-gc9da4049 db794d706f49 2c466d9bfca3 2024-05-09T00:46:23.115 INFO:teuthology.orchestra.run.smithi005.stdout:mds.c smithi157 running (3m) 36s ago 3m 87.7M - 19.0.0-3594-gc9da4049 db794d706f49 7b1acef3ad12 2024-05-09T00:46:23.115 INFO:teuthology.orchestra.run.smithi005.stdout:mds.d smithi005 running (3m) 34s ago 3m 82.1M - 19.0.0-3594-gc9da4049 db794d706f49 c546b98bbf6a 2024-05-09T00:46:23.116 INFO:teuthology.orchestra.run.smithi005.stdout:mds.e smithi131 running (3m) 36s ago 3m 91.1M - 19.0.0-3594-gc9da4049 db794d706f49 30bda55db0ed 2024-05-09T00:46:23.116 INFO:teuthology.orchestra.run.smithi005.stdout:mds.f smithi157 running (3m) 36s ago 3m 89.3M - 19.0.0-3594-gc9da4049 db794d706f49 e5281c48d308 2024-05-09T00:46:23.116 INFO:teuthology.orchestra.run.smithi005.stdout:mds.g smithi005 running (2m) 34s ago 2m 86.4M - 19.0.0-3594-gc9da4049 db794d706f49 b9bda7917e09 2024-05-09T00:46:23.116 INFO:teuthology.orchestra.run.smithi005.stdout:mds.h smithi131 running (3m) 36s ago 3m 88.2M - 19.0.0-3594-gc9da4049 db794d706f49 ab258a8336b8 2024-05-09T00:46:23.116 INFO:teuthology.orchestra.run.smithi005.stdout:mds.i smithi157 running (3m) 36s ago 3m 96.7M - 19.0.0-3594-gc9da4049 db794d706f49 a20c29b1f166 2024-05-09T00:46:23.116 INFO:teuthology.orchestra.run.smithi005.stdout:mds.j smithi005 running (3m) 34s ago 3m 82.2M - 19.0.0-3594-gc9da4049 db794d706f49 8569b4a1ca69 2024-05-09T00:46:23.116 INFO:teuthology.orchestra.run.smithi005.stdout:mds.k smithi131 running (3m) 36s ago 3m 88.3M - 19.0.0-3594-gc9da4049 db794d706f49 6b161fb19c15 2024-05-09T00:46:23.116 INFO:teuthology.orchestra.run.smithi005.stdout:mds.l smithi157 running (3m) 36s ago 3m 89.4M - 19.0.0-3594-gc9da4049 db794d706f49 de9011eb50a6 2024-05-09T00:46:23.116 INFO:teuthology.orchestra.run.smithi005.stdout:mgr.x smithi005 *:9283,8765 running (10m) 34s ago 10m 601M - 19.0.0-3594-gc9da4049 db794d706f49 f58a8233012e 2024-05-09T00:46:23.116 INFO:teuthology.orchestra.run.smithi005.stdout:mgr.y smithi131 *:8443,8765 running (8m) 36s ago 8m 537M - 19.0.0-3594-gc9da4049 db794d706f49 2b75e33e9571 2024-05-09T00:46:23.116 INFO:teuthology.orchestra.run.smithi005.stdout:mgr.z smithi157 *:8443,8765 running (8m) 36s ago 8m 537M - 19.0.0-3594-gc9da4049 db794d706f49 f48ec4b59dda 2024-05-09T00:46:23.116 INFO:teuthology.orchestra.run.smithi005.stdout:mon.a smithi005 running (10m) 34s ago 10m 119M 2048M 19.0.0-3594-gc9da4049 db794d706f49 f5caa1eb3625 2024-05-09T00:46:23.116 INFO:teuthology.orchestra.run.smithi005.stdout:mon.b smithi131 running (9m) 36s ago 9m 111M 2048M 19.0.0-3594-gc9da4049 db794d706f49 88c17aba4455 2024-05-09T00:46:23.117 INFO:teuthology.orchestra.run.smithi005.stdout:mon.c smithi157 running (9m) 36s ago 9m 107M 2048M 19.0.0-3594-gc9da4049 db794d706f49 e534101f6e7c 2024-05-09T00:46:23.117 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0 smithi005 running (8m) 34s ago 8m 115M 4273M 19.0.0-3594-gc9da4049 db794d706f49 1eccb3bc3b56 2024-05-09T00:46:23.117 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1 smithi131 running (8m) 36s ago 8m 123M 4272M 19.0.0-3594-gc9da4049 db794d706f49 b700fc5c1809 2024-05-09T00:46:23.117 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2 smithi157 running (7m) 36s ago 7m 113M 4272M 19.0.0-3594-gc9da4049 db794d706f49 7198533d124e 2024-05-09T00:46:23.117 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3 smithi005 running (7m) 34s ago 7m 121M 4273M 19.0.0-3594-gc9da4049 db794d706f49 7f23345f1379 2024-05-09T00:46:23.117 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4 smithi131 running (6m) 36s ago 6m 111M 4272M 19.0.0-3594-gc9da4049 db794d706f49 4d4f2d76c4f3 2024-05-09T00:46:23.117 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5 smithi157 running (6m) 36s ago 6m 113M 4272M 19.0.0-3594-gc9da4049 db794d706f49 c9277eab325a 2024-05-09T00:46:23.117 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6 smithi005 running (6m) 34s ago 6m 110M 4273M 19.0.0-3594-gc9da4049 db794d706f49 5347863aa7fc 2024-05-09T00:46:23.117 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7 smithi131 running (5m) 36s ago 5m 111M 4272M 19.0.0-3594-gc9da4049 db794d706f49 3c7858244e71 2024-05-09T00:46:23.117 INFO:teuthology.orchestra.run.smithi005.stdout:osd.8 smithi157 running (5m) 36s ago 5m 111M 4272M 19.0.0-3594-gc9da4049 db794d706f49 049b0aa67a46 2024-05-09T00:46:23.117 INFO:teuthology.orchestra.run.smithi005.stdout:osd.9 smithi005 running (4m) 34s ago 4m 109M 4273M 19.0.0-3594-gc9da4049 db794d706f49 ec5d178f6a12 2024-05-09T00:46:23.117 INFO:teuthology.orchestra.run.smithi005.stdout:osd.10 smithi131 running (4m) 36s ago 4m 111M 4272M 19.0.0-3594-gc9da4049 db794d706f49 aded6b04f52b 2024-05-09T00:46:23.117 INFO:teuthology.orchestra.run.smithi005.stdout:osd.11 smithi157 running (3m) 36s ago 3m 111M 4272M 19.0.0-3594-gc9da4049 db794d706f49 cd0bccf8b439 2024-05-09T00:46:23.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:23.116+0000 7ffb38fa0700 1 -- 172.21.15.5:0/1618836174 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ffb1c0b2900 msgr2=0x7ffb1c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:23.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:23.116+0000 7ffb38fa0700 1 --2- 172.21.15.5:0/1618836174 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ffb1c0b2900 0x7ffb1c0b4e10 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ffb24005950 tx=0x7ffb2400b410 comp rx=0 tx=0).stop 2024-05-09T00:46:23.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:23.116+0000 7ffb38fa0700 1 -- 172.21.15.5:0/1618836174 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb3410a0f0 msgr2=0x7ffb341a1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:23.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:23.116+0000 7ffb38fa0700 1 --2- 172.21.15.5:0/1618836174 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb3410a0f0 0x7ffb341a1d30 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c0062a0 tx=0x7ffb2c004e40 comp rx=0 tx=0).stop 2024-05-09T00:46:23.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:23.117+0000 7ffb38fa0700 1 -- 172.21.15.5:0/1618836174 shutdown_connections 2024-05-09T00:46:23.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:23.117+0000 7ffb38fa0700 1 --2- 172.21.15.5:0/1618836174 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ffb1c0b2900 0x7ffb1c0b4e10 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:23.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:23.117+0000 7ffb38fa0700 1 --2- 172.21.15.5:0/1618836174 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb3410a0f0 0x7ffb341a1d30 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:23.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:23.117+0000 7ffb38fa0700 1 --2- 172.21.15.5:0/1618836174 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb34104e30 0x7ffb3419a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:23.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:23.117+0000 7ffb38fa0700 1 --2- 172.21.15.5:0/1618836174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb34107790 0x7ffb3419ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:23.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:23.117+0000 7ffb38fa0700 1 -- 172.21.15.5:0/1618836174 >> 172.21.15.5:0/1618836174 conn(0x7ffb340fe870 msgr2=0x7ffb34105920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:23.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:23.117+0000 7ffb38fa0700 1 -- 172.21.15.5:0/1618836174 shutdown_connections 2024-05-09T00:46:23.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:23.117+0000 7ffb38fa0700 1 -- 172.21.15.5:0/1618836174 wait complete. 2024-05-09T00:46:23.435 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:46:23 smithi005 ceph-mon[30365]: pgmap v372: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T00:46:23.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:46:23 smithi131 ceph-mon[34547]: pgmap v372: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T00:46:23.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:46:23 smithi157 ceph-mon[34294]: pgmap v372: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T00:46:23.904 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- bash -c 'ceph orch ls' 2024-05-09T00:46:24.160 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:46:24.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:46:24 smithi005 ceph-mon[30365]: from='client.15051 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:46:24.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:46:24 smithi131 ceph-mon[34547]: from='client.15051 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:46:24.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:46:24 smithi157 ceph-mon[34294]: from='client.15051 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:46:25.515 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:46:25 smithi005 ceph-mon[30365]: pgmap v373: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-09T00:46:25.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:46:25 smithi131 ceph-mon[34547]: pgmap v373: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-09T00:46:25.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:46:25 smithi157 ceph-mon[34294]: pgmap v373: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-09T00:46:25.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:25.786+0000 7f1d680c1700 1 -- 172.21.15.5:0/3805385437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d60103240 msgr2=0x7f1d60103650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:25.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:25.786+0000 7f1d680c1700 1 --2- 172.21.15.5:0/3805385437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d60103240 0x7f1d60103650 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c00b0a0 tx=0x7f1d5c00b3b0 comp rx=0 tx=0).stop 2024-05-09T00:46:25.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:25.786+0000 7f1d680c1700 1 -- 172.21.15.5:0/3805385437 shutdown_connections 2024-05-09T00:46:25.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:25.786+0000 7f1d680c1700 1 --2- 172.21.15.5:0/3805385437 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d60104d10 0x7f1d6010b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:25.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:25.786+0000 7f1d680c1700 1 --2- 172.21.15.5:0/3805385437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d60103240 0x7f1d60103650 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:25.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:25.786+0000 7f1d680c1700 1 --2- 172.21.15.5:0/3805385437 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d60104370 0x7f1d60104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:25.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:25.786+0000 7f1d680c1700 1 -- 172.21.15.5:0/3805385437 >> 172.21.15.5:0/3805385437 conn(0x7f1d600fe880 msgr2=0x7f1d60100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:25.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:25.786+0000 7f1d680c1700 1 -- 172.21.15.5:0/3805385437 shutdown_connections 2024-05-09T00:46:25.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:25.786+0000 7f1d680c1700 1 -- 172.21.15.5:0/3805385437 wait complete. 2024-05-09T00:46:25.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:25.787+0000 7f1d680c1700 1 Processor -- start 2024-05-09T00:46:25.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:25.787+0000 7f1d680c1700 1 -- start start 2024-05-09T00:46:25.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:25.787+0000 7f1d680c1700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d60103240 0x7f1d6006c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:25.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:25.787+0000 7f1d680c1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d60104370 0x7f1d6019f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:25.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:25.787+0000 7f1d680c1700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d60104d10 0x7f1d601a1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:25.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:25.787+0000 7f1d680c1700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6010ec80 con 0x7f1d60104370 2024-05-09T00:46:25.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:25.787+0000 7f1d680c1700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6010eb00 con 0x7f1d60103240 2024-05-09T00:46:25.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:25.787+0000 7f1d680c1700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6010ee00 con 0x7f1d60104d10 2024-05-09T00:46:25.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:25.788+0000 7f1d6565c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d60104370 0x7f1d6019f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:25.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:25.788+0000 7f1d6565c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d60104370 0x7f1d6019f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39226/0 (socket says 172.21.15.5:39226) 2024-05-09T00:46:25.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:25.788+0000 7f1d6565c700 1 -- 172.21.15.5:0/1909142544 learned_addr learned my addr 172.21.15.5:0/1909142544 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:46:25.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:25.788+0000 7f1d6665e700 1 --2- 172.21.15.5:0/1909142544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d60104d10 0x7f1d601a1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:25.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:25.788+0000 7f1d65e5d700 1 --2- 172.21.15.5:0/1909142544 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d60103240 0x7f1d6006c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:25.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:25.788+0000 7f1d6565c700 1 -- 172.21.15.5:0/1909142544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d60104d10 msgr2=0x7f1d601a1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:25.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:25.788+0000 7f1d6565c700 1 --2- 172.21.15.5:0/1909142544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d60104d10 0x7f1d601a1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:25.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:25.788+0000 7f1d6565c700 1 -- 172.21.15.5:0/1909142544 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d60103240 msgr2=0x7f1d6006c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:25.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:25.788+0000 7f1d6565c700 1 --2- 172.21.15.5:0/1909142544 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d60103240 0x7f1d6006c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:25.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:25.788+0000 7f1d6565c700 1 -- 172.21.15.5:0/1909142544 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d601a2480 con 0x7f1d60104370 2024-05-09T00:46:25.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:25.788+0000 7f1d65e5d700 1 --2- 172.21.15.5:0/1909142544 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d60103240 0x7f1d6006c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:46:25.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:25.788+0000 7f1d6565c700 1 --2- 172.21.15.5:0/1909142544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d60104370 0x7f1d6019f6e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f1d5000d950 tx=0x7f1d5000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:25.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:25.788+0000 7f1d56ffd700 1 -- 172.21.15.5:0/1909142544 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d50016070 con 0x7f1d60104370 2024-05-09T00:46:25.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:25.788+0000 7f1d680c1700 1 -- 172.21.15.5:0/1909142544 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d601a2770 con 0x7f1d60104370 2024-05-09T00:46:25.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:25.788+0000 7f1d56ffd700 1 -- 172.21.15.5:0/1909142544 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d50009df0 con 0x7f1d60104370 2024-05-09T00:46:25.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:25.788+0000 7f1d56ffd700 1 -- 172.21.15.5:0/1909142544 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5000f3c0 con 0x7f1d60104370 2024-05-09T00:46:25.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:25.788+0000 7f1d680c1700 1 -- 172.21.15.5:0/1909142544 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d601a2cb0 con 0x7f1d60104370 2024-05-09T00:46:25.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:25.789+0000 7f1d680c1700 1 -- 172.21.15.5:0/1909142544 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d600512c0 con 0x7f1d60104370 2024-05-09T00:46:25.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:25.791+0000 7f1d56ffd700 1 -- 172.21.15.5:0/1909142544 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f1d50019070 con 0x7f1d60104370 2024-05-09T00:46:25.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:25.792+0000 7f1d56ffd700 1 --2- 172.21.15.5:0/1909142544 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f1d4c0b2880 0x7f1d4c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:25.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:25.792+0000 7f1d65e5d700 1 --2- 172.21.15.5:0/1909142544 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f1d4c0b2880 0x7f1d4c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:25.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:25.792+0000 7f1d56ffd700 1 -- 172.21.15.5:0/1909142544 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f1d500e1eb0 con 0x7f1d60104370 2024-05-09T00:46:25.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:25.792+0000 7f1d65e5d700 1 --2- 172.21.15.5:0/1909142544 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f1d4c0b2880 0x7f1d4c0b4d90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c009fd0 tx=0x7f1d5c009c10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:25.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:25.795+0000 7f1d56ffd700 1 -- 172.21.15.5:0/1909142544 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1d500a9b40 con 0x7f1d60104370 2024-05-09T00:46:26.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:26.017+0000 7f1d680c1700 1 -- 172.21.15.5:0/1909142544 --> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f1d60063c00 con 0x7f1d4c0b2880 2024-05-09T00:46:26.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:26.024+0000 7f1d56ffd700 1 -- 172.21.15.5:0/1909142544 <== mgr.14152 v2:172.21.15.5:6800/91856593 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f1d60063c00 con 0x7f1d4c0b2880 2024-05-09T00:46:26.024 INFO:teuthology.orchestra.run.smithi005.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-09T00:46:26.024 INFO:teuthology.orchestra.run.smithi005.stdout:mds.all 12/12 39s ago 3m smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi131=b;smithi131=e;smithi131=h;smithi131=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l;count:12 2024-05-09T00:46:26.025 INFO:teuthology.orchestra.run.smithi005.stdout:mgr 3/3 39s ago 8m smithi005=x;smithi131=y;smithi157=z;count:3 2024-05-09T00:46:26.025 INFO:teuthology.orchestra.run.smithi005.stdout:mon 3/3 39s ago 9m smithi005:172.21.15.5=a;smithi131:172.21.15.131=b;smithi157:172.21.15.157=c;count:3 2024-05-09T00:46:26.025 INFO:teuthology.orchestra.run.smithi005.stdout:osd 12 39s ago - 2024-05-09T00:46:26.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:26.026+0000 7f1d680c1700 1 -- 172.21.15.5:0/1909142544 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f1d4c0b2880 msgr2=0x7f1d4c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:26.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:26.026+0000 7f1d680c1700 1 --2- 172.21.15.5:0/1909142544 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f1d4c0b2880 0x7f1d4c0b4d90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c009fd0 tx=0x7f1d5c009c10 comp rx=0 tx=0).stop 2024-05-09T00:46:26.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:26.026+0000 7f1d680c1700 1 -- 172.21.15.5:0/1909142544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d60104370 msgr2=0x7f1d6019f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:26.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:26.026+0000 7f1d680c1700 1 --2- 172.21.15.5:0/1909142544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d60104370 0x7f1d6019f6e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f1d5000d950 tx=0x7f1d5000dc60 comp rx=0 tx=0).stop 2024-05-09T00:46:26.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:26.026+0000 7f1d680c1700 1 -- 172.21.15.5:0/1909142544 shutdown_connections 2024-05-09T00:46:26.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:26.026+0000 7f1d680c1700 1 --2- 172.21.15.5:0/1909142544 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f1d4c0b2880 0x7f1d4c0b4d90 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:26.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:26.026+0000 7f1d680c1700 1 --2- 172.21.15.5:0/1909142544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d60104d10 0x7f1d601a1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:26.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:26.026+0000 7f1d680c1700 1 --2- 172.21.15.5:0/1909142544 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d60103240 0x7f1d6006c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:26.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:26.026+0000 7f1d680c1700 1 --2- 172.21.15.5:0/1909142544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d60104370 0x7f1d6019f6e0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:26.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:26.026+0000 7f1d680c1700 1 -- 172.21.15.5:0/1909142544 >> 172.21.15.5:0/1909142544 conn(0x7f1d600fe880 msgr2=0x7f1d601075a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:26.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:26.026+0000 7f1d680c1700 1 -- 172.21.15.5:0/1909142544 shutdown_connections 2024-05-09T00:46:26.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:26.026+0000 7f1d680c1700 1 -- 172.21.15.5:0/1909142544 wait complete. 2024-05-09T00:46:26.739 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-09T00:46:26.992 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:46:27.365 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:46:27 smithi005 ceph-mon[30365]: pgmap v374: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T00:46:27.365 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:46:27 smithi005 ceph-mon[30365]: from='client.15057 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:46:27.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:46:27 smithi131 ceph-mon[34547]: pgmap v374: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T00:46:27.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:46:27 smithi131 ceph-mon[34547]: from='client.15057 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:46:27.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:46:27 smithi157 ceph-mon[34294]: pgmap v374: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T00:46:27.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:46:27 smithi157 ceph-mon[34294]: from='client.15057 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:46:28.638 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:46:28 smithi005 ceph-mon[30365]: pgmap v375: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T00:46:28.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:28.636+0000 7f752ad9b700 1 -- 172.21.15.5:0/57073038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75241054a0 msgr2=0x7f75241058b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:28.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:28.636+0000 7f752ad9b700 1 --2- 172.21.15.5:0/57073038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75241054a0 0x7f75241058b0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7510009990 tx=0x7f7510009ca0 comp rx=0 tx=0).stop 2024-05-09T00:46:28.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:28.637+0000 7f752ad9b700 1 -- 172.21.15.5:0/57073038 shutdown_connections 2024-05-09T00:46:28.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:28.637+0000 7f752ad9b700 1 --2- 172.21.15.5:0/57073038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7524105e40 0x7f752410c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:28.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:28.637+0000 7f752ad9b700 1 --2- 172.21.15.5:0/57073038 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7524104370 0x7f7524104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:28.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:28.637+0000 7f752ad9b700 1 --2- 172.21.15.5:0/57073038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75241054a0 0x7f75241058b0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:28.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:28.637+0000 7f752ad9b700 1 -- 172.21.15.5:0/57073038 >> 172.21.15.5:0/57073038 conn(0x7f75240ff9d0 msgr2=0x7f7524101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:28.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:28.637+0000 7f752ad9b700 1 -- 172.21.15.5:0/57073038 shutdown_connections 2024-05-09T00:46:28.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:28.637+0000 7f752ad9b700 1 -- 172.21.15.5:0/57073038 wait complete. 2024-05-09T00:46:28.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:28.637+0000 7f752ad9b700 1 Processor -- start 2024-05-09T00:46:28.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:28.637+0000 7f752ad9b700 1 -- start start 2024-05-09T00:46:28.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:28.638+0000 7f752ad9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7524104370 0x7f7524078570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:28.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:28.638+0000 7f752ad9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75241054a0 0x7f7524076c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:28.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:28.638+0000 7f752ad9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7524105e40 0x7f7524077160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:28.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:28.638+0000 7f752ad9b700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f752410fdf0 con 0x7f75241054a0 2024-05-09T00:46:28.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:28.638+0000 7f752ad9b700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f752410fc70 con 0x7f7524104370 2024-05-09T00:46:28.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:28.638+0000 7f752ad9b700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f752410ff70 con 0x7f7524105e40 2024-05-09T00:46:28.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:28.638+0000 7f7523fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75241054a0 0x7f7524076c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:28.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:28.638+0000 7f7528b37700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7524104370 0x7f7524078570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:28.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:28.638+0000 7f7523fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75241054a0 0x7f7524076c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39244/0 (socket says 172.21.15.5:39244) 2024-05-09T00:46:28.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:28.638+0000 7f7523fff700 1 -- 172.21.15.5:0/1622270859 learned_addr learned my addr 172.21.15.5:0/1622270859 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:46:28.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:28.638+0000 7f7529338700 1 --2- 172.21.15.5:0/1622270859 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7524105e40 0x7f7524077160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:28.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:28.638+0000 7f7523fff700 1 -- 172.21.15.5:0/1622270859 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7524105e40 msgr2=0x7f7524077160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:28.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:28.638+0000 7f7523fff700 1 --2- 172.21.15.5:0/1622270859 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7524105e40 0x7f7524077160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:28.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:28.638+0000 7f7523fff700 1 -- 172.21.15.5:0/1622270859 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7524104370 msgr2=0x7f7524078570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:28.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:28.638+0000 7f7523fff700 1 --2- 172.21.15.5:0/1622270859 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7524104370 0x7f7524078570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:28.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:28.638+0000 7f7523fff700 1 -- 172.21.15.5:0/1622270859 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75240778c0 con 0x7f75241054a0 2024-05-09T00:46:28.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:28.638+0000 7f7528b37700 1 --2- 172.21.15.5:0/1622270859 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7524104370 0x7f7524078570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:46:28.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:28.639+0000 7f7523fff700 1 --2- 172.21.15.5:0/1622270859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75241054a0 0x7f7524076c20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f75100092e0 tx=0x7f7510005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:28.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:28.639+0000 7f7521ffb700 1 -- 172.21.15.5:0/1622270859 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7510004830 con 0x7f75241054a0 2024-05-09T00:46:28.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:28.639+0000 7f752ad9b700 1 -- 172.21.15.5:0/1622270859 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75241a49e0 con 0x7f75241054a0 2024-05-09T00:46:28.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:28.639+0000 7f7521ffb700 1 -- 172.21.15.5:0/1622270859 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7510003c00 con 0x7f75241054a0 2024-05-09T00:46:28.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:28.639+0000 7f7521ffb700 1 -- 172.21.15.5:0/1622270859 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7510005280 con 0x7f75241054a0 2024-05-09T00:46:28.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:28.639+0000 7f752ad9b700 1 -- 172.21.15.5:0/1622270859 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75241a4d80 con 0x7f75241054a0 2024-05-09T00:46:28.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:28.640+0000 7f752ad9b700 1 -- 172.21.15.5:0/1622270859 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75240512c0 con 0x7f75241054a0 2024-05-09T00:46:28.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:28.643+0000 7f7521ffb700 1 -- 172.21.15.5:0/1622270859 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f7510029060 con 0x7f75241054a0 2024-05-09T00:46:28.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:28.643+0000 7f7521ffb700 1 --2- 172.21.15.5:0/1622270859 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f75140b2880 0x7f75140b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:28.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:28.644+0000 7f7528b37700 1 --2- 172.21.15.5:0/1622270859 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f75140b2880 0x7f75140b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:28.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:28.644+0000 7f7521ffb700 1 -- 172.21.15.5:0/1622270859 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f75100e3250 con 0x7f75241054a0 2024-05-09T00:46:28.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:28.644+0000 7f7528b37700 1 --2- 172.21.15.5:0/1622270859 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f75140b2880 0x7f75140b4d90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f75241053a0 tx=0x7f751800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:28.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:28.647+0000 7f7521ffb700 1 -- 172.21.15.5:0/1622270859 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7510013070 con 0x7f75241054a0 2024-05-09T00:46:28.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:46:28 smithi157 ceph-mon[34294]: pgmap v375: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T00:46:28.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:46:28 smithi131 ceph-mon[34547]: pgmap v375: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T00:46:28.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:28.866+0000 7f752ad9b700 1 -- 172.21.15.5:0/1622270859 --> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f7524063cc0 con 0x7f75140b2880 2024-05-09T00:46:28.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:28.868+0000 7f7521ffb700 1 -- 172.21.15.5:0/1622270859 <== mgr.14152 v2:172.21.15.5:6800/91856593 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f7524063cc0 con 0x7f75140b2880 2024-05-09T00:46:28.869 INFO:teuthology.orchestra.run.smithi005.stdout:HOST ADDR LABELS STATUS 2024-05-09T00:46:28.869 INFO:teuthology.orchestra.run.smithi005.stdout:smithi005 172.21.15.5 2024-05-09T00:46:28.869 INFO:teuthology.orchestra.run.smithi005.stdout:smithi131 172.21.15.131 2024-05-09T00:46:28.869 INFO:teuthology.orchestra.run.smithi005.stdout:smithi157 172.21.15.157 2024-05-09T00:46:28.869 INFO:teuthology.orchestra.run.smithi005.stdout:3 hosts in cluster 2024-05-09T00:46:28.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:28.870+0000 7f752ad9b700 1 -- 172.21.15.5:0/1622270859 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f75140b2880 msgr2=0x7f75140b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:28.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:28.870+0000 7f752ad9b700 1 --2- 172.21.15.5:0/1622270859 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f75140b2880 0x7f75140b4d90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f75241053a0 tx=0x7f751800b410 comp rx=0 tx=0).stop 2024-05-09T00:46:28.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:28.870+0000 7f752ad9b700 1 -- 172.21.15.5:0/1622270859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75241054a0 msgr2=0x7f7524076c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:28.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:28.870+0000 7f752ad9b700 1 --2- 172.21.15.5:0/1622270859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75241054a0 0x7f7524076c20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f75100092e0 tx=0x7f7510005ef0 comp rx=0 tx=0).stop 2024-05-09T00:46:28.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:28.871+0000 7f752ad9b700 1 -- 172.21.15.5:0/1622270859 shutdown_connections 2024-05-09T00:46:28.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:28.871+0000 7f752ad9b700 1 --2- 172.21.15.5:0/1622270859 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f75140b2880 0x7f75140b4d90 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:28.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:28.871+0000 7f752ad9b700 1 --2- 172.21.15.5:0/1622270859 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7524105e40 0x7f7524077160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:28.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:28.871+0000 7f752ad9b700 1 --2- 172.21.15.5:0/1622270859 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7524104370 0x7f7524078570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:28.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:28.871+0000 7f752ad9b700 1 --2- 172.21.15.5:0/1622270859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75241054a0 0x7f7524076c20 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:28.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:28.871+0000 7f752ad9b700 1 -- 172.21.15.5:0/1622270859 >> 172.21.15.5:0/1622270859 conn(0x7f75240ff9d0 msgr2=0x7f75241086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:28.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:28.871+0000 7f752ad9b700 1 -- 172.21.15.5:0/1622270859 shutdown_connections 2024-05-09T00:46:28.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:28.871+0000 7f752ad9b700 1 -- 172.21.15.5:0/1622270859 wait complete. 2024-05-09T00:46:29.643 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:46:29 smithi005 ceph-mon[30365]: from='client.15063 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:46:29.645 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-09T00:46:29.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:46:29 smithi157 ceph-mon[34294]: from='client.15063 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:46:29.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:46:29 smithi131 ceph-mon[34547]: from='client.15063 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:46:29.896 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:46:30.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:46:30 smithi005 ceph-mon[30365]: pgmap v376: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-09T00:46:30.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:46:30 smithi157 ceph-mon[34294]: pgmap v376: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-09T00:46:30.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:46:30 smithi131 ceph-mon[34547]: pgmap v376: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-09T00:46:31.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:31.547+0000 7fe959030700 1 -- 172.21.15.5:0/2469444818 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe954103460 msgr2=0x7fe954103870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:31.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:31.547+0000 7fe959030700 1 --2- 172.21.15.5:0/2469444818 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe954103460 0x7fe954103870 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe93c009990 tx=0x7fe93c009ca0 comp rx=0 tx=0).stop 2024-05-09T00:46:31.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:31.548+0000 7fe959030700 1 -- 172.21.15.5:0/2469444818 shutdown_connections 2024-05-09T00:46:31.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:31.548+0000 7fe959030700 1 --2- 172.21.15.5:0/2469444818 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe954103db0 0x7fe95410b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:31.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:31.548+0000 7fe959030700 1 --2- 172.21.15.5:0/2469444818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe954104750 0x7fe954104b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:31.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:31.548+0000 7fe959030700 1 --2- 172.21.15.5:0/2469444818 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe954103460 0x7fe954103870 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:31.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:31.548+0000 7fe959030700 1 -- 172.21.15.5:0/2469444818 >> 172.21.15.5:0/2469444818 conn(0x7fe9540fe000 msgr2=0x7fe9541004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:31.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:31.548+0000 7fe959030700 1 -- 172.21.15.5:0/2469444818 shutdown_connections 2024-05-09T00:46:31.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:31.548+0000 7fe959030700 1 -- 172.21.15.5:0/2469444818 wait complete. 2024-05-09T00:46:31.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:31.549+0000 7fe959030700 1 Processor -- start 2024-05-09T00:46:31.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:31.549+0000 7fe959030700 1 -- start start 2024-05-09T00:46:31.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:31.549+0000 7fe959030700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe954103460 0x7fe954102e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:31.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:31.549+0000 7fe959030700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe954103db0 0x7fe95419f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:31.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:31.549+0000 7fe959030700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe954104750 0x7fe9541a1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:31.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:31.549+0000 7fe952d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe954103460 0x7fe954102e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:31.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:31.549+0000 7fe952d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe954103460 0x7fe954102e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39264/0 (socket says 172.21.15.5:39264) 2024-05-09T00:46:31.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:31.549+0000 7fe952d9d700 1 -- 172.21.15.5:0/1651996210 learned_addr learned my addr 172.21.15.5:0/1651996210 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:46:31.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:31.549+0000 7fe959030700 1 -- 172.21.15.5:0/1651996210 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe95410ecf0 con 0x7fe954103460 2024-05-09T00:46:31.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:31.549+0000 7fe959030700 1 -- 172.21.15.5:0/1651996210 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe95410eb70 con 0x7fe954103db0 2024-05-09T00:46:31.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:31.549+0000 7fe959030700 1 -- 172.21.15.5:0/1651996210 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe95410ee70 con 0x7fe954104750 2024-05-09T00:46:31.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:31.549+0000 7fe94bfff700 1 --2- 172.21.15.5:0/1651996210 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe954103db0 0x7fe95419f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:31.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:31.550+0000 7fe952d9d700 1 -- 172.21.15.5:0/1651996210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe954104750 msgr2=0x7fe9541a1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:31.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:31.550+0000 7fe95359e700 1 --2- 172.21.15.5:0/1651996210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe954104750 0x7fe9541a1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:31.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:31.550+0000 7fe952d9d700 1 --2- 172.21.15.5:0/1651996210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe954104750 0x7fe9541a1c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:31.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:31.550+0000 7fe952d9d700 1 -- 172.21.15.5:0/1651996210 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe954103db0 msgr2=0x7fe95419f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:31.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:31.550+0000 7fe952d9d700 1 --2- 172.21.15.5:0/1651996210 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe954103db0 0x7fe95419f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:31.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:31.550+0000 7fe952d9d700 1 -- 172.21.15.5:0/1651996210 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9541a2430 con 0x7fe954103460 2024-05-09T00:46:31.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:31.550+0000 7fe94bfff700 1 --2- 172.21.15.5:0/1651996210 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe954103db0 0x7fe95419f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:46:31.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:31.550+0000 7fe952d9d700 1 --2- 172.21.15.5:0/1651996210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe954103460 0x7fe954102e10 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe94400d980 tx=0x7fe94400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:31.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:31.550+0000 7fe950d99700 1 -- 172.21.15.5:0/1651996210 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe944017070 con 0x7fe954103460 2024-05-09T00:46:31.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:31.550+0000 7fe950d99700 1 -- 172.21.15.5:0/1651996210 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe944010460 con 0x7fe954103460 2024-05-09T00:46:31.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:31.550+0000 7fe959030700 1 -- 172.21.15.5:0/1651996210 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9541a2720 con 0x7fe954103460 2024-05-09T00:46:31.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:31.550+0000 7fe950d99700 1 -- 172.21.15.5:0/1651996210 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe94400de60 con 0x7fe954103460 2024-05-09T00:46:31.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:31.550+0000 7fe959030700 1 -- 172.21.15.5:0/1651996210 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9541a2c60 con 0x7fe954103460 2024-05-09T00:46:31.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:31.551+0000 7fe959030700 1 -- 172.21.15.5:0/1651996210 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9540512c0 con 0x7fe954103460 2024-05-09T00:46:31.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:31.554+0000 7fe950d99700 1 -- 172.21.15.5:0/1651996210 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fe944010610 con 0x7fe954103460 2024-05-09T00:46:31.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:31.554+0000 7fe950d99700 1 --2- 172.21.15.5:0/1651996210 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe9400b2880 0x7fe9400b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:31.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:31.555+0000 7fe94bfff700 1 --2- 172.21.15.5:0/1651996210 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe9400b2880 0x7fe9400b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:31.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:31.555+0000 7fe950d99700 1 -- 172.21.15.5:0/1651996210 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fe9440e1e10 con 0x7fe954103460 2024-05-09T00:46:31.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:31.555+0000 7fe94bfff700 1 --2- 172.21.15.5:0/1651996210 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe9400b2880 0x7fe9400b4d90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe93c0038a0 tx=0x7fe93c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:31.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:31.558+0000 7fe950d99700 1 -- 172.21.15.5:0/1651996210 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe9440a9b00 con 0x7fe954103460 2024-05-09T00:46:31.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:31.781+0000 7fe959030700 1 -- 172.21.15.5:0/1651996210 --> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fe954063c00 con 0x7fe9400b2880 2024-05-09T00:46:31.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:31.783+0000 7fe950d99700 1 -- 172.21.15.5:0/1651996210 <== mgr.14152 v2:172.21.15.5:6800/91856593 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fe954063c00 con 0x7fe9400b2880 2024-05-09T00:46:31.783 INFO:teuthology.orchestra.run.smithi005.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-09T00:46:31.784 INFO:teuthology.orchestra.run.smithi005.stdout:smithi005 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB751300EW480DGN 447G No 4m ago Has a FileSystem, LVM detected 2024-05-09T00:46:31.784 INFO:teuthology.orchestra.run.smithi005.stdout:smithi131 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300N4400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-05-09T00:46:31.784 INFO:teuthology.orchestra.run.smithi005.stdout:smithi157 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7476002V480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-05-09T00:46:31.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:31.785+0000 7fe959030700 1 -- 172.21.15.5:0/1651996210 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe9400b2880 msgr2=0x7fe9400b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:31.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:31.785+0000 7fe959030700 1 --2- 172.21.15.5:0/1651996210 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe9400b2880 0x7fe9400b4d90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe93c0038a0 tx=0x7fe93c01a040 comp rx=0 tx=0).stop 2024-05-09T00:46:31.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:31.785+0000 7fe959030700 1 -- 172.21.15.5:0/1651996210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe954103460 msgr2=0x7fe954102e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:31.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:31.785+0000 7fe959030700 1 --2- 172.21.15.5:0/1651996210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe954103460 0x7fe954102e10 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe94400d980 tx=0x7fe94400dd40 comp rx=0 tx=0).stop 2024-05-09T00:46:31.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:31.785+0000 7fe959030700 1 -- 172.21.15.5:0/1651996210 shutdown_connections 2024-05-09T00:46:31.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:31.785+0000 7fe959030700 1 --2- 172.21.15.5:0/1651996210 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe9400b2880 0x7fe9400b4d90 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:31.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:31.785+0000 7fe959030700 1 --2- 172.21.15.5:0/1651996210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe954104750 0x7fe9541a1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:31.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:31.785+0000 7fe959030700 1 --2- 172.21.15.5:0/1651996210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe954103460 0x7fe954102e10 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:31.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:31.785+0000 7fe959030700 1 --2- 172.21.15.5:0/1651996210 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe954103db0 0x7fe95419f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:31.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:31.785+0000 7fe959030700 1 -- 172.21.15.5:0/1651996210 >> 172.21.15.5:0/1651996210 conn(0x7fe9540fe000 msgr2=0x7fe9541076a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:31.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:31.785+0000 7fe959030700 1 -- 172.21.15.5:0/1651996210 shutdown_connections 2024-05-09T00:46:31.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:31.785+0000 7fe959030700 1 -- 172.21.15.5:0/1651996210 wait complete. 2024-05-09T00:46:32.547 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-09T00:46:32.557 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi005.front.sepia.ceph.com 2024-05-09T00:46:32.557 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-09T00:46:32.811 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:46:33.185 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:46:32 smithi005 ceph-mon[30365]: from='client.15069 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:46:33.185 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:46:32 smithi005 ceph-mon[30365]: pgmap v377: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T00:46:33.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:46:32 smithi131 ceph-mon[34547]: from='client.15069 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:46:33.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:46:32 smithi131 ceph-mon[34547]: pgmap v377: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T00:46:33.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:46:32 smithi157 ceph-mon[34294]: from='client.15069 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:46:33.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:46:32 smithi157 ceph-mon[34294]: pgmap v377: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T00:46:34.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:34.532+0000 7f2a6211d700 1 -- 172.21.15.5:0/1207062648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a5c103250 msgr2=0x7f2a5c103660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:34.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:34.532+0000 7f2a6211d700 1 --2- 172.21.15.5:0/1207062648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a5c103250 0x7f2a5c103660 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c009990 tx=0x7f2a4c009ca0 comp rx=0 tx=0).stop 2024-05-09T00:46:34.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:34.533+0000 7f2a6211d700 1 -- 172.21.15.5:0/1207062648 shutdown_connections 2024-05-09T00:46:34.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:34.533+0000 7f2a6211d700 1 --2- 172.21.15.5:0/1207062648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a5c104d20 0x7f2a5c10b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:34.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:34.533+0000 7f2a6211d700 1 --2- 172.21.15.5:0/1207062648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a5c103250 0x7f2a5c103660 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:34.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:34.533+0000 7f2a6211d700 1 --2- 172.21.15.5:0/1207062648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a5c104380 0x7f2a5c104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:34.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:34.533+0000 7f2a6211d700 1 -- 172.21.15.5:0/1207062648 >> 172.21.15.5:0/1207062648 conn(0x7f2a5c0fe890 msgr2=0x7f2a5c100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:34.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:34.533+0000 7f2a6211d700 1 -- 172.21.15.5:0/1207062648 shutdown_connections 2024-05-09T00:46:34.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:34.533+0000 7f2a6211d700 1 -- 172.21.15.5:0/1207062648 wait complete. 2024-05-09T00:46:34.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:34.533+0000 7f2a6211d700 1 Processor -- start 2024-05-09T00:46:34.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:34.533+0000 7f2a6211d700 1 -- start start 2024-05-09T00:46:34.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:34.534+0000 7f2a6211d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a5c103250 0x7f2a5c107010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:34.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:34.534+0000 7f2a6211d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a5c104380 0x7f2a5c1056c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:34.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:34.534+0000 7f2a6211d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a5c104d20 0x7f2a5c105c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:34.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:34.534+0000 7f2a6211d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c10ecc0 con 0x7f2a5c104380 2024-05-09T00:46:34.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:34.534+0000 7f2a6211d700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c10eb40 con 0x7f2a5c104d20 2024-05-09T00:46:34.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:34.534+0000 7f2a6211d700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c10ee40 con 0x7f2a5c103250 2024-05-09T00:46:34.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:34.534+0000 7f2a5affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a5c104380 0x7f2a5c1056c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:34.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:34.534+0000 7f2a5affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a5c104380 0x7f2a5c1056c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58822/0 (socket says 172.21.15.5:58822) 2024-05-09T00:46:34.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:34.534+0000 7f2a5affd700 1 -- 172.21.15.5:0/628875929 learned_addr learned my addr 172.21.15.5:0/628875929 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:46:34.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:34.534+0000 7f2a5b7fe700 1 --2- 172.21.15.5:0/628875929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a5c103250 0x7f2a5c107010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:34.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:34.534+0000 7f2a5bfff700 1 --2- 172.21.15.5:0/628875929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a5c104d20 0x7f2a5c105c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:34.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:34.534+0000 7f2a5affd700 1 -- 172.21.15.5:0/628875929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a5c103250 msgr2=0x7f2a5c107010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:34.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:34.534+0000 7f2a5affd700 1 --2- 172.21.15.5:0/628875929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a5c103250 0x7f2a5c107010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:34.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:34.534+0000 7f2a5affd700 1 -- 172.21.15.5:0/628875929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a5c104d20 msgr2=0x7f2a5c105c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:34.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:34.534+0000 7f2a5affd700 1 --2- 172.21.15.5:0/628875929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a5c104d20 0x7f2a5c105c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:34.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:34.534+0000 7f2a5affd700 1 -- 172.21.15.5:0/628875929 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a5c106360 con 0x7f2a5c104380 2024-05-09T00:46:34.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:34.534+0000 7f2a5bfff700 1 --2- 172.21.15.5:0/628875929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a5c104d20 0x7f2a5c105c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:46:34.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:34.534+0000 7f2a5b7fe700 1 --2- 172.21.15.5:0/628875929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a5c103250 0x7f2a5c107010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:46:34.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:34.534+0000 7f2a5affd700 1 --2- 172.21.15.5:0/628875929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a5c104380 0x7f2a5c1056c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2a5000d980 tx=0x7f2a5000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:34.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:34.535+0000 7f2a58ff9700 1 -- 172.21.15.5:0/628875929 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a50017070 con 0x7f2a5c104380 2024-05-09T00:46:34.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:34.535+0000 7f2a6211d700 1 -- 172.21.15.5:0/628875929 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a5c1a4a50 con 0x7f2a5c104380 2024-05-09T00:46:34.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:34.535+0000 7f2a58ff9700 1 -- 172.21.15.5:0/628875929 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a50004830 con 0x7f2a5c104380 2024-05-09T00:46:34.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:34.535+0000 7f2a58ff9700 1 -- 172.21.15.5:0/628875929 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a50005440 con 0x7f2a5c104380 2024-05-09T00:46:34.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:34.535+0000 7f2a6211d700 1 -- 172.21.15.5:0/628875929 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a5c1a4ee0 con 0x7f2a5c104380 2024-05-09T00:46:34.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:34.536+0000 7f2a6211d700 1 -- 172.21.15.5:0/628875929 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a5c0512c0 con 0x7f2a5c104380 2024-05-09T00:46:34.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:34.537+0000 7f2a58ff9700 1 -- 172.21.15.5:0/628875929 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f2a500055e0 con 0x7f2a5c104380 2024-05-09T00:46:34.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:34.538+0000 7f2a58ff9700 1 --2- 172.21.15.5:0/628875929 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f2a440b2880 0x7f2a440b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:34.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:34.538+0000 7f2a5b7fe700 1 --2- 172.21.15.5:0/628875929 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f2a440b2880 0x7f2a440b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:34.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:34.538+0000 7f2a58ff9700 1 -- 172.21.15.5:0/628875929 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f2a500e2660 con 0x7f2a5c104380 2024-05-09T00:46:34.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:34.539+0000 7f2a5b7fe700 1 --2- 172.21.15.5:0/628875929 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f2a440b2880 0x7f2a440b4d90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c0038a0 tx=0x7f2a4c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:34.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:34.542+0000 7f2a58ff9700 1 -- 172.21.15.5:0/628875929 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2a50020020 con 0x7f2a5c104380 2024-05-09T00:46:34.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:34.807+0000 7f2a6211d700 1 -- 172.21.15.5:0/628875929 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f2a5c1a51d0 con 0x7f2a5c104380 2024-05-09T00:46:34.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:34.807+0000 7f2a58ff9700 1 -- 172.21.15.5:0/628875929 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) ==== 76+0+3087 (secure 0 0 0) 0x7f2a500aa2f0 con 0x7f2a5c104380 2024-05-09T00:46:34.808 INFO:teuthology.orchestra.run.smithi005.stdout:e28 2024-05-09T00:46:34.808 INFO:teuthology.orchestra.run.smithi005.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-09T00:46:34.808 INFO:teuthology.orchestra.run.smithi005.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-09T00:46:34.808 INFO:teuthology.orchestra.run.smithi005.stdout:legacy client fscid: 1 2024-05-09T00:46:34.808 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:46:34.808 INFO:teuthology.orchestra.run.smithi005.stdout:Filesystem 'cephfs' (1) 2024-05-09T00:46:34.809 INFO:teuthology.orchestra.run.smithi005.stdout:fs_name cephfs 2024-05-09T00:46:34.809 INFO:teuthology.orchestra.run.smithi005.stdout:epoch 28 2024-05-09T00:46:34.809 INFO:teuthology.orchestra.run.smithi005.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-09T00:46:34.809 INFO:teuthology.orchestra.run.smithi005.stdout:created 2024-05-09T00:43:45.856066+0000 2024-05-09T00:46:34.809 INFO:teuthology.orchestra.run.smithi005.stdout:modified 2024-05-09T00:44:06.033927+0000 2024-05-09T00:46:34.809 INFO:teuthology.orchestra.run.smithi005.stdout:tableserver 0 2024-05-09T00:46:34.809 INFO:teuthology.orchestra.run.smithi005.stdout:root 0 2024-05-09T00:46:34.809 INFO:teuthology.orchestra.run.smithi005.stdout:session_timeout 300 2024-05-09T00:46:34.809 INFO:teuthology.orchestra.run.smithi005.stdout:session_autoclose 300 2024-05-09T00:46:34.809 INFO:teuthology.orchestra.run.smithi005.stdout:max_file_size 1099511627776 2024-05-09T00:46:34.809 INFO:teuthology.orchestra.run.smithi005.stdout:max_xattr_size 65536 2024-05-09T00:46:34.809 INFO:teuthology.orchestra.run.smithi005.stdout:required_client_features {} 2024-05-09T00:46:34.809 INFO:teuthology.orchestra.run.smithi005.stdout:last_failure 0 2024-05-09T00:46:34.809 INFO:teuthology.orchestra.run.smithi005.stdout:last_failure_osd_epoch 0 2024-05-09T00:46:34.810 INFO:teuthology.orchestra.run.smithi005.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-09T00:46:34.810 INFO:teuthology.orchestra.run.smithi005.stdout:max_mds 5 2024-05-09T00:46:34.810 INFO:teuthology.orchestra.run.smithi005.stdout:in 0,1,2,3,4 2024-05-09T00:46:34.810 INFO:teuthology.orchestra.run.smithi005.stdout:up {0=24508,1=24499,2=24494,3=24493,4=24467} 2024-05-09T00:46:34.810 INFO:teuthology.orchestra.run.smithi005.stdout:failed 2024-05-09T00:46:34.810 INFO:teuthology.orchestra.run.smithi005.stdout:damaged 2024-05-09T00:46:34.810 INFO:teuthology.orchestra.run.smithi005.stdout:stopped 2024-05-09T00:46:34.810 INFO:teuthology.orchestra.run.smithi005.stdout:data_pools [3] 2024-05-09T00:46:34.810 INFO:teuthology.orchestra.run.smithi005.stdout:metadata_pool 2 2024-05-09T00:46:34.810 INFO:teuthology.orchestra.run.smithi005.stdout:inline_data disabled 2024-05-09T00:46:34.810 INFO:teuthology.orchestra.run.smithi005.stdout:balancer 2024-05-09T00:46:34.810 INFO:teuthology.orchestra.run.smithi005.stdout:bal_rank_mask -1 2024-05-09T00:46:34.810 INFO:teuthology.orchestra.run.smithi005.stdout:standby_count_wanted 0 2024-05-09T00:46:34.811 INFO:teuthology.orchestra.run.smithi005.stdout:qdb_cluster leader: 24508 members: 24467,24493,24499,24508,24494 2024-05-09T00:46:34.811 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.i{0:24508} state up:active seq 8 addr [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T00:46:34.811 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.l{0:24457} state up:standby-replay seq 1 addr [v2:172.21.15.157:6832/2809525455,v1:172.21.15.157:6833/2809525455] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T00:46:34.811 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.c{1:24499} state up:active seq 12 addr [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T00:46:34.811 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.e{1:24443} state up:standby-replay seq 1 addr [v2:172.21.15.131:6834/1283093898,v1:172.21.15.131:6835/1283093898] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T00:46:34.811 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.b{2:24494} state up:active seq 9 addr [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T00:46:34.811 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.k{2:24419} state up:standby-replay seq 1 addr [v2:172.21.15.131:6832/3272897742,v1:172.21.15.131:6833/3272897742] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T00:46:34.811 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.f{3:24493} state up:active seq 14 addr [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T00:46:34.811 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.a{3:14610} state up:standby-replay seq 1 addr [v2:172.21.15.5:6840/3809966329,v1:172.21.15.5:6841/3809966329] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T00:46:34.811 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.h{4:24467} state up:active seq 12 addr [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T00:46:34.811 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.g{4:14598} state up:standby-replay seq 1 addr [v2:172.21.15.5:6838/2109255228,v1:172.21.15.5:6839/2109255228] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T00:46:34.812 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:46:34.812 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:46:34.812 INFO:teuthology.orchestra.run.smithi005.stdout:Standby daemons: 2024-05-09T00:46:34.812 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:46:34.812 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.j{-1:14538} state up:standby seq 1 addr [v2:172.21.15.5:6834/2685771887,v1:172.21.15.5:6835/2685771887] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T00:46:34.812 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.d{-1:14550} state up:standby seq 1 addr [v2:172.21.15.5:6836/254017535,v1:172.21.15.5:6837/254017535] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T00:46:34.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:34.809+0000 7f2a6211d700 1 -- 172.21.15.5:0/628875929 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f2a440b2880 msgr2=0x7f2a440b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:34.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:34.809+0000 7f2a6211d700 1 --2- 172.21.15.5:0/628875929 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f2a440b2880 0x7f2a440b4d90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c0038a0 tx=0x7f2a4c01a040 comp rx=0 tx=0).stop 2024-05-09T00:46:34.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:34.809+0000 7f2a6211d700 1 -- 172.21.15.5:0/628875929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a5c104380 msgr2=0x7f2a5c1056c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:34.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:34.809+0000 7f2a6211d700 1 --2- 172.21.15.5:0/628875929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a5c104380 0x7f2a5c1056c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2a5000d980 tx=0x7f2a5000dd40 comp rx=0 tx=0).stop 2024-05-09T00:46:34.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:34.809+0000 7f2a6211d700 1 -- 172.21.15.5:0/628875929 shutdown_connections 2024-05-09T00:46:34.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:34.809+0000 7f2a6211d700 1 --2- 172.21.15.5:0/628875929 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f2a440b2880 0x7f2a440b4d90 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:34.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:34.809+0000 7f2a6211d700 1 --2- 172.21.15.5:0/628875929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a5c104d20 0x7f2a5c105c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:34.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:34.809+0000 7f2a6211d700 1 --2- 172.21.15.5:0/628875929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a5c103250 0x7f2a5c107010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:34.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:34.809+0000 7f2a6211d700 1 --2- 172.21.15.5:0/628875929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a5c104380 0x7f2a5c1056c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:34.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:34.809+0000 7f2a6211d700 1 -- 172.21.15.5:0/628875929 >> 172.21.15.5:0/628875929 conn(0x7f2a5c0fe890 msgr2=0x7f2a5c100220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:34.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:34.810+0000 7f2a6211d700 1 -- 172.21.15.5:0/628875929 shutdown_connections 2024-05-09T00:46:34.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:34.810+0000 7f2a6211d700 1 -- 172.21.15.5:0/628875929 wait complete. 2024-05-09T00:46:34.814 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 28 2024-05-09T00:46:35.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:46:34 smithi005 ceph-mon[30365]: pgmap v378: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-09T00:46:35.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:46:34 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/628875929' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T00:46:35.238 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:46:34 smithi157 ceph-mon[34294]: pgmap v378: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-09T00:46:35.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:46:34 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/628875929' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T00:46:35.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:46:34 smithi131 ceph-mon[34547]: pgmap v378: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-09T00:46:35.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:46:34 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/628875929' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T00:46:35.615 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- bash -c 'ceph osd dump' 2024-05-09T00:46:35.873 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:46:37.282 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:46:36 smithi005 ceph-mon[30365]: pgmap v379: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-09T00:46:37.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:46:36 smithi131 ceph-mon[34547]: pgmap v379: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-09T00:46:37.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:46:36 smithi157 ceph-mon[34294]: pgmap v379: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-09T00:46:37.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:37.702+0000 7fd7f5e05700 1 -- 172.21.15.5:0/1938211194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7f0104d20 msgr2=0x7fd7f010b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:37.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:37.702+0000 7fd7f5e05700 1 --2- 172.21.15.5:0/1938211194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7f0104d20 0x7fd7f010b5d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd7e800b0a0 tx=0x7fd7e800b3b0 comp rx=0 tx=0).stop 2024-05-09T00:46:37.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:37.702+0000 7fd7f5e05700 1 -- 172.21.15.5:0/1938211194 shutdown_connections 2024-05-09T00:46:37.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:37.702+0000 7fd7f5e05700 1 --2- 172.21.15.5:0/1938211194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7f0104d20 0x7fd7f010b5d0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:37.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:37.702+0000 7fd7f5e05700 1 --2- 172.21.15.5:0/1938211194 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7f0103250 0x7fd7f0103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:37.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:37.702+0000 7fd7f5e05700 1 --2- 172.21.15.5:0/1938211194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7f0104380 0x7fd7f0104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:37.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:37.702+0000 7fd7f5e05700 1 -- 172.21.15.5:0/1938211194 >> 172.21.15.5:0/1938211194 conn(0x7fd7f00fe890 msgr2=0x7fd7f0100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:37.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:37.702+0000 7fd7f5e05700 1 -- 172.21.15.5:0/1938211194 shutdown_connections 2024-05-09T00:46:37.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:37.702+0000 7fd7f5e05700 1 -- 172.21.15.5:0/1938211194 wait complete. 2024-05-09T00:46:37.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:37.703+0000 7fd7f5e05700 1 Processor -- start 2024-05-09T00:46:37.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:37.703+0000 7fd7f5e05700 1 -- start start 2024-05-09T00:46:37.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:37.703+0000 7fd7f5e05700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7f0103250 0x7fd7f010d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:37.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:37.703+0000 7fd7f5e05700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7f0104380 0x7fd7f010c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:37.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:37.703+0000 7fd7f5e05700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7f0104d20 0x7fd7f010c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:37.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:37.703+0000 7fd7f5e05700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f010fe50 con 0x7fd7f0104d20 2024-05-09T00:46:37.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:37.703+0000 7fd7f5e05700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f010fcd0 con 0x7fd7f0104380 2024-05-09T00:46:37.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:37.703+0000 7fd7f5e05700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f010ffd0 con 0x7fd7f0103250 2024-05-09T00:46:37.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:37.703+0000 7fd7effff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7f0104d20 0x7fd7f010c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:37.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:37.703+0000 7fd7effff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7f0104d20 0x7fd7f010c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58838/0 (socket says 172.21.15.5:58838) 2024-05-09T00:46:37.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:37.703+0000 7fd7effff700 1 -- 172.21.15.5:0/361500596 learned_addr learned my addr 172.21.15.5:0/361500596 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:46:37.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:37.703+0000 7fd7ef7fe700 1 --2- 172.21.15.5:0/361500596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7f0103250 0x7fd7f010d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:37.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:37.704+0000 7fd7eeffd700 1 --2- 172.21.15.5:0/361500596 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7f0104380 0x7fd7f010c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:37.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:37.704+0000 7fd7effff700 1 -- 172.21.15.5:0/361500596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7f0103250 msgr2=0x7fd7f010d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:37.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:37.704+0000 7fd7effff700 1 --2- 172.21.15.5:0/361500596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7f0103250 0x7fd7f010d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:37.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:37.704+0000 7fd7effff700 1 -- 172.21.15.5:0/361500596 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7f0104380 msgr2=0x7fd7f010c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:37.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:37.704+0000 7fd7effff700 1 --2- 172.21.15.5:0/361500596 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7f0104380 0x7fd7f010c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:37.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:37.704+0000 7fd7effff700 1 -- 172.21.15.5:0/361500596 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7f010cce0 con 0x7fd7f0104d20 2024-05-09T00:46:37.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:37.704+0000 7fd7effff700 1 --2- 172.21.15.5:0/361500596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7f0104d20 0x7fd7f010c580 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd7e800b070 tx=0x7fd7e8004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:37.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:37.704+0000 7fd7ecff9700 1 -- 172.21.15.5:0/361500596 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e8021070 con 0x7fd7f0104d20 2024-05-09T00:46:37.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:37.704+0000 7fd7ecff9700 1 -- 172.21.15.5:0/361500596 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7e8007b80 con 0x7fd7f0104d20 2024-05-09T00:46:37.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:37.704+0000 7fd7f5e05700 1 -- 172.21.15.5:0/361500596 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7f01a4970 con 0x7fd7f0104d20 2024-05-09T00:46:37.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:37.704+0000 7fd7ecff9700 1 -- 172.21.15.5:0/361500596 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e80052a0 con 0x7fd7f0104d20 2024-05-09T00:46:37.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:37.704+0000 7fd7f5e05700 1 -- 172.21.15.5:0/361500596 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7f01a4e20 con 0x7fd7f0104d20 2024-05-09T00:46:37.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:37.705+0000 7fd7f5e05700 1 -- 172.21.15.5:0/361500596 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7f00512c0 con 0x7fd7f0104d20 2024-05-09T00:46:37.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:37.707+0000 7fd7ecff9700 1 -- 172.21.15.5:0/361500596 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fd7e8024070 con 0x7fd7f0104d20 2024-05-09T00:46:37.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:37.708+0000 7fd7ecff9700 1 --2- 172.21.15.5:0/361500596 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fd7d80b2900 0x7fd7d80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:37.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:37.708+0000 7fd7ef7fe700 1 --2- 172.21.15.5:0/361500596 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fd7d80b2900 0x7fd7d80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:37.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:37.708+0000 7fd7ecff9700 1 -- 172.21.15.5:0/361500596 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fd7e800f020 con 0x7fd7f0104d20 2024-05-09T00:46:37.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:37.709+0000 7fd7ef7fe700 1 --2- 172.21.15.5:0/361500596 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fd7d80b2900 0x7fd7d80b4e10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0104280 tx=0x7fd7e0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:37.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:37.712+0000 7fd7ecff9700 1 -- 172.21.15.5:0/361500596 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd7e802b020 con 0x7fd7f0104d20 2024-05-09T00:46:37.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:37.930+0000 7fd7f5e05700 1 -- 172.21.15.5:0/361500596 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fd7f002e5f0 con 0x7fd7f0104d20 2024-05-09T00:46:37.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:37.931+0000 7fd7ecff9700 1 -- 172.21.15.5:0/361500596 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v82) ==== 56+0+5322 (secure 0 0 0) 0x7fd7e80aba90 con 0x7fd7f0104d20 2024-05-09T00:46:37.934 INFO:teuthology.orchestra.run.smithi005.stdout:epoch 82 2024-05-09T00:46:37.934 INFO:teuthology.orchestra.run.smithi005.stdout:fsid d85814c6-0d9b-11ef-bc97-c7b262605968 2024-05-09T00:46:37.934 INFO:teuthology.orchestra.run.smithi005.stdout:created 2024-05-09T00:35:25.623377+0000 2024-05-09T00:46:37.935 INFO:teuthology.orchestra.run.smithi005.stdout:modified 2024-05-09T00:45:23.926493+0000 2024-05-09T00:46:37.935 INFO:teuthology.orchestra.run.smithi005.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-05-09T00:46:37.935 INFO:teuthology.orchestra.run.smithi005.stdout:crush_version 26 2024-05-09T00:46:37.935 INFO:teuthology.orchestra.run.smithi005.stdout:full_ratio 0.95 2024-05-09T00:46:37.935 INFO:teuthology.orchestra.run.smithi005.stdout:backfillfull_ratio 0.9 2024-05-09T00:46:37.935 INFO:teuthology.orchestra.run.smithi005.stdout:nearfull_ratio 0.85 2024-05-09T00:46:37.935 INFO:teuthology.orchestra.run.smithi005.stdout:require_min_compat_client luminous 2024-05-09T00:46:37.935 INFO:teuthology.orchestra.run.smithi005.stdout:min_compat_client luminous 2024-05-09T00:46:37.935 INFO:teuthology.orchestra.run.smithi005.stdout:require_osd_release squid 2024-05-09T00:46:37.935 INFO:teuthology.orchestra.run.smithi005.stdout:stretch_mode_enabled false 2024-05-09T00:46:37.936 INFO:teuthology.orchestra.run.smithi005.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-05-09T00:46:37.936 INFO:teuthology.orchestra.run.smithi005.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 78 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-05-09T00:46:37.936 INFO:teuthology.orchestra.run.smithi005.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 78 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-05-09T00:46:37.936 INFO:teuthology.orchestra.run.smithi005.stdout:max_osd 12 2024-05-09T00:46:37.936 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0 up in weight 1 up_from 8 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.5:6802/2963385132,v1:172.21.15.5:6803/2963385132] [v2:172.21.15.5:6804/2963385132,v1:172.21.15.5:6805/2963385132] exists,up f8d484bb-2477-4377-ae68-5bce571333bc 2024-05-09T00:46:37.936 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1 up in weight 1 up_from 13 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.131:6800/3837683163,v1:172.21.15.131:6801/3837683163] [v2:172.21.15.131:6802/3837683163,v1:172.21.15.131:6803/3837683163] exists,up 1c123676-7144-4a67-93a6-fd49e36a5c10 2024-05-09T00:46:37.936 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2 up in weight 1 up_from 18 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.157:6800/484818139,v1:172.21.15.157:6801/484818139] [v2:172.21.15.157:6802/484818139,v1:172.21.15.157:6803/484818139] exists,up 95ca51f0-b48b-485c-a5f7-b15c01e754b5 2024-05-09T00:46:37.936 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3 up in weight 1 up_from 25 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.5:6810/2357377277,v1:172.21.15.5:6811/2357377277] [v2:172.21.15.5:6812/2357377277,v1:172.21.15.5:6813/2357377277] exists,up ed7577fb-3676-4a08-b40d-5474eb88974f 2024-05-09T00:46:37.936 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4 up in weight 1 up_from 31 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.131:6808/2915418378,v1:172.21.15.131:6809/2915418378] [v2:172.21.15.131:6810/2915418378,v1:172.21.15.131:6811/2915418378] exists,up fb5dcf79-a6f1-4b1a-9e3a-087fb5132472 2024-05-09T00:46:37.936 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5 up in weight 1 up_from 36 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.157:6808/4188818663,v1:172.21.15.157:6809/4188818663] [v2:172.21.15.157:6810/4188818663,v1:172.21.15.157:6811/4188818663] exists,up f01ed5d3-8066-4d25-a113-280c83e75ebb 2024-05-09T00:46:37.936 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6 up in weight 1 up_from 42 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.5:6818/2930397813,v1:172.21.15.5:6819/2930397813] [v2:172.21.15.5:6820/2930397813,v1:172.21.15.5:6821/2930397813] exists,up 932379a8-c97b-4acc-8665-81f7d9cbce65 2024-05-09T00:46:37.936 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7 up in weight 1 up_from 47 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.131:6816/4125045219,v1:172.21.15.131:6817/4125045219] [v2:172.21.15.131:6818/4125045219,v1:172.21.15.131:6819/4125045219] exists,up 3c0a0573-6465-4bf6-b70f-310d49857a6f 2024-05-09T00:46:37.936 INFO:teuthology.orchestra.run.smithi005.stdout:osd.8 up in weight 1 up_from 53 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.157:6816/2001626768,v1:172.21.15.157:6817/2001626768] [v2:172.21.15.157:6818/2001626768,v1:172.21.15.157:6819/2001626768] exists,up c49f0a00-f5ae-40ed-8f38-7b953ba1c17c 2024-05-09T00:46:37.937 INFO:teuthology.orchestra.run.smithi005.stdout:osd.9 up in weight 1 up_from 59 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.5:6826/4287220015,v1:172.21.15.5:6827/4287220015] [v2:172.21.15.5:6828/4287220015,v1:172.21.15.5:6829/4287220015] exists,up 1927dfe7-c1af-4a15-bb71-d58d97c8734a 2024-05-09T00:46:37.937 INFO:teuthology.orchestra.run.smithi005.stdout:osd.10 up in weight 1 up_from 64 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.131:6824/1571783985,v1:172.21.15.131:6825/1571783985] [v2:172.21.15.131:6826/1571783985,v1:172.21.15.131:6827/1571783985] exists,up fbb4f3e3-eb7d-4ecb-a30b-55b8910375ac 2024-05-09T00:46:37.937 INFO:teuthology.orchestra.run.smithi005.stdout:osd.11 up in weight 1 up_from 70 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.157:6824/3261151188,v1:172.21.15.157:6825/3261151188] [v2:172.21.15.157:6826/3261151188,v1:172.21.15.157:6827/3261151188] exists,up b858453f-eda1-4e25-89c8-94b54c830938 2024-05-09T00:46:37.937 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 2.4 [2,11] 2024-05-09T00:46:37.937 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 2.14 [2,4] 2024-05-09T00:46:37.937 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 2.1c [8,11] 2024-05-09T00:46:37.937 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 3.b [3,9] 2024-05-09T00:46:37.937 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 3.e [4,6,5,9] 2024-05-09T00:46:37.937 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 3.17 [4,8,5,6] 2024-05-09T00:46:37.937 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 3.19 [3,8] 2024-05-09T00:46:37.937 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 3.1b [4,7] 2024-05-09T00:46:37.937 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 3.35 [1,9] 2024-05-09T00:46:37.937 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 3.3a [4,7] 2024-05-09T00:46:37.937 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:0/3405015786 expires 2024-05-10T00:36:21.742363+0000 2024-05-09T00:46:37.938 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:0/2669115557 expires 2024-05-10T00:36:21.742363+0000 2024-05-09T00:46:37.938 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:6801/287842266 expires 2024-05-10T00:36:21.742363+0000 2024-05-09T00:46:37.938 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:6800/752513054 expires 2024-05-10T00:35:53.128955+0000 2024-05-09T00:46:37.938 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:6800/287842266 expires 2024-05-10T00:36:21.742363+0000 2024-05-09T00:46:37.938 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:6801/752513054 expires 2024-05-10T00:35:53.128955+0000 2024-05-09T00:46:37.938 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:0/1954876662 expires 2024-05-10T00:35:53.128955+0000 2024-05-09T00:46:37.938 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:0/2545175121 expires 2024-05-10T00:36:21.742363+0000 2024-05-09T00:46:37.938 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:0/3754653663 expires 2024-05-10T00:35:53.128955+0000 2024-05-09T00:46:37.938 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:0/651514359 expires 2024-05-10T00:35:53.128955+0000 2024-05-09T00:46:37.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:37.934+0000 7fd7f5e05700 1 -- 172.21.15.5:0/361500596 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fd7d80b2900 msgr2=0x7fd7d80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:37.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:37.934+0000 7fd7f5e05700 1 --2- 172.21.15.5:0/361500596 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fd7d80b2900 0x7fd7d80b4e10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0104280 tx=0x7fd7e0009260 comp rx=0 tx=0).stop 2024-05-09T00:46:37.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:37.934+0000 7fd7f5e05700 1 -- 172.21.15.5:0/361500596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7f0104d20 msgr2=0x7fd7f010c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:37.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:37.934+0000 7fd7f5e05700 1 --2- 172.21.15.5:0/361500596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7f0104d20 0x7fd7f010c580 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd7e800b070 tx=0x7fd7e8004ce0 comp rx=0 tx=0).stop 2024-05-09T00:46:37.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:37.934+0000 7fd7f5e05700 1 -- 172.21.15.5:0/361500596 shutdown_connections 2024-05-09T00:46:37.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:37.934+0000 7fd7f5e05700 1 --2- 172.21.15.5:0/361500596 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fd7d80b2900 0x7fd7d80b4e10 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:37.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:37.934+0000 7fd7f5e05700 1 --2- 172.21.15.5:0/361500596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7f0104d20 0x7fd7f010c580 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:37.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:37.934+0000 7fd7f5e05700 1 --2- 172.21.15.5:0/361500596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7f0103250 0x7fd7f010d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:37.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:37.934+0000 7fd7f5e05700 1 --2- 172.21.15.5:0/361500596 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7f0104380 0x7fd7f010c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:37.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:37.934+0000 7fd7f5e05700 1 -- 172.21.15.5:0/361500596 >> 172.21.15.5:0/361500596 conn(0x7fd7f00fe890 msgr2=0x7fd7f01075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:37.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:37.934+0000 7fd7f5e05700 1 -- 172.21.15.5:0/361500596 shutdown_connections 2024-05-09T00:46:37.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:37.935+0000 7fd7f5e05700 1 -- 172.21.15.5:0/361500596 wait complete. 2024-05-09T00:46:38.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:46:37 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/361500596' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-09T00:46:38.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:46:37 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/361500596' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-09T00:46:38.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:46:37 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/361500596' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-09T00:46:38.738 INFO:teuthology.run_tasks:Running task fs.ready... 2024-05-09T00:46:38.750 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T00:46:39.008 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:46:39.258 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:46:38 smithi005 ceph-mon[30365]: pgmap v380: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-09T00:46:39.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:46:38 smithi131 ceph-mon[34547]: pgmap v380: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-09T00:46:39.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:46:38 smithi157 ceph-mon[34294]: pgmap v380: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-09T00:46:40.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:40.746+0000 7f991631d700 1 -- 172.21.15.5:0/1374529784 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9910107df0 msgr2=0x7f9910108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:40.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:40.746+0000 7f991631d700 1 --2- 172.21.15.5:0/1374529784 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9910107df0 0x7f9910108200 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f98f80099c0 tx=0x7f98f8009cd0 comp rx=0 tx=0).stop 2024-05-09T00:46:40.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:40.746+0000 7f991631d700 1 -- 172.21.15.5:0/1374529784 shutdown_connections 2024-05-09T00:46:40.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:40.746+0000 7f991631d700 1 --2- 172.21.15.5:0/1374529784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9910108740 0x7f99101100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:40.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:40.746+0000 7f991631d700 1 --2- 172.21.15.5:0/1374529784 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99101034c0 0x7f99101038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:40.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:40.746+0000 7f991631d700 1 --2- 172.21.15.5:0/1374529784 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9910107df0 0x7f9910108200 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:40.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:40.746+0000 7f991631d700 1 -- 172.21.15.5:0/1374529784 >> 172.21.15.5:0/1374529784 conn(0x7f99100fedf0 msgr2=0x7f9910101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:40.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:40.747+0000 7f991631d700 1 -- 172.21.15.5:0/1374529784 shutdown_connections 2024-05-09T00:46:40.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:40.747+0000 7f991631d700 1 -- 172.21.15.5:0/1374529784 wait complete. 2024-05-09T00:46:40.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:40.747+0000 7f991631d700 1 Processor -- start 2024-05-09T00:46:40.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:40.748+0000 7f991631d700 1 -- start start 2024-05-09T00:46:40.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:40.748+0000 7f991631d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99101034c0 0x7f991019bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:40.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:40.748+0000 7f991631d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9910107df0 0x7f991019c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:40.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:40.748+0000 7f991631d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9910108740 0x7f991019c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:40.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:40.748+0000 7f991631d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9910113350 con 0x7f9910107df0 2024-05-09T00:46:40.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:40.748+0000 7f991631d700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f99101131d0 con 0x7f99101034c0 2024-05-09T00:46:40.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:40.748+0000 7f991631d700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f99101134d0 con 0x7f9910108740 2024-05-09T00:46:40.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:40.748+0000 7f9907fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9910107df0 0x7f991019c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:40.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:40.748+0000 7f9907fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9910107df0 0x7f991019c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58850/0 (socket says 172.21.15.5:58850) 2024-05-09T00:46:40.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:40.748+0000 7f9907fff700 1 -- 172.21.15.5:0/2799640846 learned_addr learned my addr 172.21.15.5:0/2799640846 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:46:40.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:40.748+0000 7f990ffff700 1 --2- 172.21.15.5:0/2799640846 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99101034c0 0x7f991019bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:40.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:40.748+0000 7f99148ba700 1 --2- 172.21.15.5:0/2799640846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9910108740 0x7f991019c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:40.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:40.749+0000 7f9907fff700 1 -- 172.21.15.5:0/2799640846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9910108740 msgr2=0x7f991019c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:40.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:40.749+0000 7f9907fff700 1 --2- 172.21.15.5:0/2799640846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9910108740 0x7f991019c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:40.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:40.749+0000 7f9907fff700 1 -- 172.21.15.5:0/2799640846 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99101034c0 msgr2=0x7f991019bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:40.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:40.749+0000 7f9907fff700 1 --2- 172.21.15.5:0/2799640846 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99101034c0 0x7f991019bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:40.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:40.749+0000 7f9907fff700 1 -- 172.21.15.5:0/2799640846 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99101a6fd0 con 0x7f9910107df0 2024-05-09T00:46:40.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:40.749+0000 7f9907fff700 1 --2- 172.21.15.5:0/2799640846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9910107df0 0x7f991019c230 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f98f8009f50 tx=0x7f98f80037c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:40.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:40.749+0000 7f990dffb700 1 -- 172.21.15.5:0/2799640846 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f8027070 con 0x7f9910107df0 2024-05-09T00:46:40.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:40.749+0000 7f991631d700 1 -- 172.21.15.5:0/2799640846 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99101a7260 con 0x7f9910107df0 2024-05-09T00:46:40.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:40.749+0000 7f991631d700 1 -- 172.21.15.5:0/2799640846 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99101a7710 con 0x7f9910107df0 2024-05-09T00:46:40.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:40.749+0000 7f990dffb700 1 -- 172.21.15.5:0/2799640846 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98f8003fa0 con 0x7f9910107df0 2024-05-09T00:46:40.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:40.749+0000 7f990dffb700 1 -- 172.21.15.5:0/2799640846 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f8022d20 con 0x7f9910107df0 2024-05-09T00:46:40.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:40.750+0000 7f991631d700 1 -- 172.21.15.5:0/2799640846 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98f0005330 con 0x7f9910107df0 2024-05-09T00:46:40.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:40.752+0000 7f990dffb700 1 -- 172.21.15.5:0/2799640846 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f98f802a070 con 0x7f9910107df0 2024-05-09T00:46:40.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:40.753+0000 7f990dffb700 1 --2- 172.21.15.5:0/2799640846 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f98fc0b2900 0x7f98fc0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:40.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:40.753+0000 7f990ffff700 1 --2- 172.21.15.5:0/2799640846 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f98fc0b2900 0x7f98fc0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:40.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:40.753+0000 7f990dffb700 1 -- 172.21.15.5:0/2799640846 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f98f80e3610 con 0x7f9910107df0 2024-05-09T00:46:40.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:40.753+0000 7f990ffff700 1 --2- 172.21.15.5:0/2799640846 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f98fc0b2900 0x7f98fc0b4e10 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f990000ac50 tx=0x7f990000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:40.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:40.757+0000 7f990dffb700 1 -- 172.21.15.5:0/2799640846 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f98f8031020 con 0x7f9910107df0 2024-05-09T00:46:41.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:41.020+0000 7f991631d700 1 -- 172.21.15.5:0/2799640846 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f98f0005620 con 0x7f9910107df0 2024-05-09T00:46:41.021 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:46:40 smithi005 ceph-mon[30365]: pgmap v381: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-09T00:46:41.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:41.021+0000 7f990dffb700 1 -- 172.21.15.5:0/2799640846 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11926 (secure 0 0 0) 0x7f98f8013070 con 0x7f9910107df0 2024-05-09T00:46:41.022 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:46:41.022 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2685771887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2685771887},{"type":"v1","addr":"172.21.15.5:6835","nonce":2685771887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/254017535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":254017535},{"type":"v1","addr":"172.21.15.5:6837","nonce":254017535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T00:43:45.856066+0000","modified":"2024-05-09T00:44:06.033927+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24508,"mds_1":24499,"mds_2":24494,"mds_3":24493,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14598":{"gid":14598,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2109255228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2109255228},{"type":"v1","addr":"172.21.15.5:6839","nonce":2109255228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/3809966329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3809966329},{"type":"v1","addr":"172.21.15.5:6841","nonce":3809966329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6833/3272897742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3272897742},{"type":"v1","addr":"172.21.15.131:6833","nonce":3272897742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1283093898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1283093898},{"type":"v1","addr":"172.21.15.131:6835","nonce":1283093898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/2809525455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2809525455},{"type":"v1","addr":"172.21.15.157:6833","nonce":2809525455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.131:6837/1633006186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":1633006186},{"type":"v1","addr":"172.21.15.131:6837","nonce":1633006186}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.157:6835/832898273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":832898273},{"type":"v1","addr":"172.21.15.157:6835","nonce":832898273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.131:6839/2748669631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2748669631},{"type":"v1","addr":"172.21.15.131:6839","nonce":2748669631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.157:6837/744863009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":744863009},{"type":"v1","addr":"172.21.15.157:6837","nonce":744863009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.157:6839/2779860399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":2779860399},{"type":"v1","addr":"172.21.15.157:6839","nonce":2779860399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24467,24493,24499,24508,24494]},"id":1}]} 2024-05-09T00:46:41.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:41.023+0000 7f991631d700 1 -- 172.21.15.5:0/2799640846 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f98fc0b2900 msgr2=0x7f98fc0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:41.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:41.023+0000 7f991631d700 1 --2- 172.21.15.5:0/2799640846 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f98fc0b2900 0x7f98fc0b4e10 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f990000ac50 tx=0x7f990000a380 comp rx=0 tx=0).stop 2024-05-09T00:46:41.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:41.023+0000 7f991631d700 1 -- 172.21.15.5:0/2799640846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9910107df0 msgr2=0x7f991019c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:41.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:41.023+0000 7f991631d700 1 --2- 172.21.15.5:0/2799640846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9910107df0 0x7f991019c230 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f98f8009f50 tx=0x7f98f80037c0 comp rx=0 tx=0).stop 2024-05-09T00:46:41.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:41.023+0000 7f991631d700 1 -- 172.21.15.5:0/2799640846 shutdown_connections 2024-05-09T00:46:41.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:41.023+0000 7f991631d700 1 --2- 172.21.15.5:0/2799640846 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f98fc0b2900 0x7f98fc0b4e10 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:41.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:41.023+0000 7f991631d700 1 --2- 172.21.15.5:0/2799640846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9910108740 0x7f991019c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:41.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:41.023+0000 7f991631d700 1 --2- 172.21.15.5:0/2799640846 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99101034c0 0x7f991019bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:41.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:41.023+0000 7f991631d700 1 --2- 172.21.15.5:0/2799640846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9910107df0 0x7f991019c230 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:41.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:41.023+0000 7f991631d700 1 -- 172.21.15.5:0/2799640846 >> 172.21.15.5:0/2799640846 conn(0x7f99100fedf0 msgr2=0x7f99101096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:41.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:41.024+0000 7f991631d700 1 -- 172.21.15.5:0/2799640846 shutdown_connections 2024-05-09T00:46:41.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:41.024+0000 7f991631d700 1 -- 172.21.15.5:0/2799640846 wait complete. 2024-05-09T00:46:41.026 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 28 2024-05-09T00:46:41.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:46:40 smithi131 ceph-mon[34547]: pgmap v381: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-09T00:46:41.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:46:40 smithi157 ceph-mon[34294]: pgmap v381: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-09T00:46:41.951 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T00:46:42.210 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:46:42.294 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:46:41 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/2799640846' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:46:42.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:46:41 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/2799640846' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:46:42.438 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:46:41 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/2799640846' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:46:43.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:46:42 smithi005 ceph-mon[30365]: pgmap v382: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-09T00:46:43.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:46:42 smithi131 ceph-mon[34547]: pgmap v382: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-09T00:46:43.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:46:42 smithi157 ceph-mon[34294]: pgmap v382: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-09T00:46:43.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:43.985+0000 7f64e3058700 1 -- 172.21.15.5:0/2188030934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64dc108f10 msgr2=0x7f64dc109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:43.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:43.985+0000 7f64e3058700 1 --2- 172.21.15.5:0/2188030934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64dc108f10 0x7f64dc109320 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f64c80099c0 tx=0x7f64c8009cd0 comp rx=0 tx=0).stop 2024-05-09T00:46:43.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:43.985+0000 7f64e3058700 1 -- 172.21.15.5:0/2188030934 shutdown_connections 2024-05-09T00:46:43.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:43.985+0000 7f64e3058700 1 --2- 172.21.15.5:0/2188030934 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64dc109860 0x7f64dc1111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:43.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:43.985+0000 7f64e3058700 1 --2- 172.21.15.5:0/2188030934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64dc1045e0 0x7f64dc1049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:43.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:43.985+0000 7f64e3058700 1 --2- 172.21.15.5:0/2188030934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64dc108f10 0x7f64dc109320 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:43.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:43.985+0000 7f64e3058700 1 -- 172.21.15.5:0/2188030934 >> 172.21.15.5:0/2188030934 conn(0x7f64dc0ffed0 msgr2=0x7f64dc102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:43.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:43.985+0000 7f64e3058700 1 -- 172.21.15.5:0/2188030934 shutdown_connections 2024-05-09T00:46:43.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:43.985+0000 7f64e3058700 1 -- 172.21.15.5:0/2188030934 wait complete. 2024-05-09T00:46:43.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:43.986+0000 7f64e3058700 1 Processor -- start 2024-05-09T00:46:43.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:43.986+0000 7f64e3058700 1 -- start start 2024-05-09T00:46:43.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:43.986+0000 7f64e3058700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64dc1045e0 0x7f64dc076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:43.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:43.986+0000 7f64e3058700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64dc108f10 0x7f64dc077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:43.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:43.986+0000 7f64e3058700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64dc109860 0x7f64dc077650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:43.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:43.986+0000 7f64e3058700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f64dc114470 con 0x7f64dc109860 2024-05-09T00:46:43.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:43.986+0000 7f64e3058700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f64dc1142f0 con 0x7f64dc108f10 2024-05-09T00:46:43.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:43.986+0000 7f64e3058700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f64dc1145f0 con 0x7f64dc1045e0 2024-05-09T00:46:43.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:43.986+0000 7f64e15f5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64dc109860 0x7f64dc077650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:43.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:43.986+0000 7f64e15f5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64dc109860 0x7f64dc077650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32938/0 (socket says 172.21.15.5:32938) 2024-05-09T00:46:43.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:43.986+0000 7f64e0df4700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64dc1045e0 0x7f64dc076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:43.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:43.986+0000 7f64e15f5700 1 -- 172.21.15.5:0/3480656623 learned_addr learned my addr 172.21.15.5:0/3480656623 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:46:43.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:43.986+0000 7f64dbfff700 1 --2- 172.21.15.5:0/3480656623 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64dc108f10 0x7f64dc077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:43.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:43.986+0000 7f64e15f5700 1 -- 172.21.15.5:0/3480656623 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64dc1045e0 msgr2=0x7f64dc076bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:43.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:43.986+0000 7f64e15f5700 1 --2- 172.21.15.5:0/3480656623 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64dc1045e0 0x7f64dc076bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:43.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:43.986+0000 7f64e15f5700 1 -- 172.21.15.5:0/3480656623 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64dc108f10 msgr2=0x7f64dc077110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:43.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:43.986+0000 7f64e15f5700 1 --2- 172.21.15.5:0/3480656623 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64dc108f10 0x7f64dc077110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:43.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:43.986+0000 7f64e15f5700 1 -- 172.21.15.5:0/3480656623 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64dc1a9200 con 0x7f64dc109860 2024-05-09T00:46:43.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:43.987+0000 7f64e15f5700 1 --2- 172.21.15.5:0/3480656623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64dc109860 0x7f64dc077650 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f64d4007ee0 tx=0x7f64d400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:43.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:43.987+0000 7f64d9ffb700 1 -- 172.21.15.5:0/3480656623 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d400dd20 con 0x7f64dc109860 2024-05-09T00:46:43.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:43.987+0000 7f64e3058700 1 -- 172.21.15.5:0/3480656623 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64dc1a94f0 con 0x7f64dc109860 2024-05-09T00:46:43.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:43.987+0000 7f64d9ffb700 1 -- 172.21.15.5:0/3480656623 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64d4004020 con 0x7f64dc109860 2024-05-09T00:46:43.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:43.987+0000 7f64d9ffb700 1 -- 172.21.15.5:0/3480656623 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d4015660 con 0x7f64dc109860 2024-05-09T00:46:43.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:43.987+0000 7f64e3058700 1 -- 172.21.15.5:0/3480656623 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64dc1a9a00 con 0x7f64dc109860 2024-05-09T00:46:43.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:43.988+0000 7f64e3058700 1 -- 172.21.15.5:0/3480656623 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64dc0512c0 con 0x7f64dc109860 2024-05-09T00:46:43.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:43.989+0000 7f64d9ffb700 1 -- 172.21.15.5:0/3480656623 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f64d40041d0 con 0x7f64dc109860 2024-05-09T00:46:43.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:43.990+0000 7f64d9ffb700 1 --2- 172.21.15.5:0/3480656623 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f64cc0b2880 0x7f64cc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:43.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:43.990+0000 7f64e0df4700 1 --2- 172.21.15.5:0/3480656623 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f64cc0b2880 0x7f64cc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:43.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:43.990+0000 7f64d9ffb700 1 -- 172.21.15.5:0/3480656623 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f64d40e1b90 con 0x7f64dc109860 2024-05-09T00:46:43.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:43.991+0000 7f64e0df4700 1 --2- 172.21.15.5:0/3480656623 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f64cc0b2880 0x7f64cc0b4d90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f64d0009c60 tx=0x7f64d0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:43.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:43.995+0000 7f64d9ffb700 1 -- 172.21.15.5:0/3480656623 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f64d4010070 con 0x7f64dc109860 2024-05-09T00:46:44.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:44.260+0000 7f64e3058700 1 -- 172.21.15.5:0/3480656623 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f64dc02e5f0 con 0x7f64dc109860 2024-05-09T00:46:44.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:44.261+0000 7f64d9ffb700 1 -- 172.21.15.5:0/3480656623 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11926 (secure 0 0 0) 0x7f64d40a9880 con 0x7f64dc109860 2024-05-09T00:46:44.262 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:46:44.262 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2685771887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2685771887},{"type":"v1","addr":"172.21.15.5:6835","nonce":2685771887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/254017535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":254017535},{"type":"v1","addr":"172.21.15.5:6837","nonce":254017535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T00:43:45.856066+0000","modified":"2024-05-09T00:44:06.033927+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24508,"mds_1":24499,"mds_2":24494,"mds_3":24493,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14598":{"gid":14598,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2109255228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2109255228},{"type":"v1","addr":"172.21.15.5:6839","nonce":2109255228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/3809966329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3809966329},{"type":"v1","addr":"172.21.15.5:6841","nonce":3809966329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6833/3272897742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3272897742},{"type":"v1","addr":"172.21.15.131:6833","nonce":3272897742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1283093898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1283093898},{"type":"v1","addr":"172.21.15.131:6835","nonce":1283093898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/2809525455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2809525455},{"type":"v1","addr":"172.21.15.157:6833","nonce":2809525455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.131:6837/1633006186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":1633006186},{"type":"v1","addr":"172.21.15.131:6837","nonce":1633006186}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.157:6835/832898273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":832898273},{"type":"v1","addr":"172.21.15.157:6835","nonce":832898273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.131:6839/2748669631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2748669631},{"type":"v1","addr":"172.21.15.131:6839","nonce":2748669631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.157:6837/744863009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":744863009},{"type":"v1","addr":"172.21.15.157:6837","nonce":744863009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.157:6839/2779860399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":2779860399},{"type":"v1","addr":"172.21.15.157:6839","nonce":2779860399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24467,24493,24499,24508,24494]},"id":1}]} 2024-05-09T00:46:44.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:44.264+0000 7f64e3058700 1 -- 172.21.15.5:0/3480656623 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f64cc0b2880 msgr2=0x7f64cc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:44.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:44.264+0000 7f64e3058700 1 --2- 172.21.15.5:0/3480656623 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f64cc0b2880 0x7f64cc0b4d90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f64d0009c60 tx=0x7f64d0009260 comp rx=0 tx=0).stop 2024-05-09T00:46:44.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:44.264+0000 7f64e3058700 1 -- 172.21.15.5:0/3480656623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64dc109860 msgr2=0x7f64dc077650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:44.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:44.264+0000 7f64e3058700 1 --2- 172.21.15.5:0/3480656623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64dc109860 0x7f64dc077650 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f64d4007ee0 tx=0x7f64d400d3b0 comp rx=0 tx=0).stop 2024-05-09T00:46:44.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:44.264+0000 7f64e3058700 1 -- 172.21.15.5:0/3480656623 shutdown_connections 2024-05-09T00:46:44.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:44.264+0000 7f64e3058700 1 --2- 172.21.15.5:0/3480656623 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f64cc0b2880 0x7f64cc0b4d90 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:44.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:44.264+0000 7f64e3058700 1 --2- 172.21.15.5:0/3480656623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64dc109860 0x7f64dc077650 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:44.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:44.264+0000 7f64e3058700 1 --2- 172.21.15.5:0/3480656623 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64dc1045e0 0x7f64dc076bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:44.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:44.264+0000 7f64e3058700 1 --2- 172.21.15.5:0/3480656623 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64dc108f10 0x7f64dc077110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:44.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:44.264+0000 7f64e3058700 1 -- 172.21.15.5:0/3480656623 >> 172.21.15.5:0/3480656623 conn(0x7f64dc0ffed0 msgr2=0x7f64dc10a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:44.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:44.264+0000 7f64e3058700 1 -- 172.21.15.5:0/3480656623 shutdown_connections 2024-05-09T00:46:44.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:44.264+0000 7f64e3058700 1 -- 172.21.15.5:0/3480656623 wait complete. 2024-05-09T00:46:44.266 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 28 2024-05-09T00:46:44.929 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T00:46:45.184 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:46:45.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:46:44 smithi005 ceph-mon[30365]: pgmap v383: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-09T00:46:45.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:46:44 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/3480656623' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:46:45.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:46:44 smithi131 ceph-mon[34547]: pgmap v383: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-09T00:46:45.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:46:44 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/3480656623' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:46:45.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:46:44 smithi157 ceph-mon[34294]: pgmap v383: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-09T00:46:45.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:46:44 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/3480656623' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:46:47.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:47.002+0000 7fd9e5309700 1 -- 172.21.15.5:0/2840282234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9e0107e10 msgr2=0x7fd9e0108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:47.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:47.002+0000 7fd9e5309700 1 --2- 172.21.15.5:0/2840282234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9e0107e10 0x7fd9e0108220 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd9d40099c0 tx=0x7fd9d4009cd0 comp rx=0 tx=0).stop 2024-05-09T00:46:47.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:47.002+0000 7fd9e5309700 1 -- 172.21.15.5:0/2840282234 shutdown_connections 2024-05-09T00:46:47.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:47.002+0000 7fd9e5309700 1 --2- 172.21.15.5:0/2840282234 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9e0108760 0x7fd9e01100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:47.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:47.002+0000 7fd9e5309700 1 --2- 172.21.15.5:0/2840282234 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9e01034e0 0x7fd9e01038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:47.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:47.002+0000 7fd9e5309700 1 --2- 172.21.15.5:0/2840282234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9e0107e10 0x7fd9e0108220 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:47.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:47.002+0000 7fd9e5309700 1 -- 172.21.15.5:0/2840282234 >> 172.21.15.5:0/2840282234 conn(0x7fd9e00fef30 msgr2=0x7fd9e0101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:47.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:47.002+0000 7fd9e5309700 1 -- 172.21.15.5:0/2840282234 shutdown_connections 2024-05-09T00:46:47.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:47.002+0000 7fd9e5309700 1 -- 172.21.15.5:0/2840282234 wait complete. 2024-05-09T00:46:47.005 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:46:46 smithi005 ceph-mon[30365]: pgmap v384: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T00:46:47.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:47.003+0000 7fd9e5309700 1 Processor -- start 2024-05-09T00:46:47.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:47.003+0000 7fd9e5309700 1 -- start start 2024-05-09T00:46:47.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:47.003+0000 7fd9e5309700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9e01034e0 0x7fd9e019d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:47.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:47.003+0000 7fd9e5309700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9e0107e10 0x7fd9e019d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:47.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:47.003+0000 7fd9e5309700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9e0108760 0x7fd9e01a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:47.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:47.003+0000 7fd9e5309700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e01144e0 con 0x7fd9e0108760 2024-05-09T00:46:47.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:47.003+0000 7fd9e5309700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e0114360 con 0x7fd9e0107e10 2024-05-09T00:46:47.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:47.003+0000 7fd9e5309700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e0114660 con 0x7fd9e01034e0 2024-05-09T00:46:47.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:47.003+0000 7fd9df7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9e0108760 0x7fd9e01a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:47.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:47.003+0000 7fd9df7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9e0108760 0x7fd9e01a1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32962/0 (socket says 172.21.15.5:32962) 2024-05-09T00:46:47.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:47.003+0000 7fd9df7fe700 1 -- 172.21.15.5:0/2813291549 learned_addr learned my addr 172.21.15.5:0/2813291549 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:46:47.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:47.003+0000 7fd9de7fc700 1 --2- 172.21.15.5:0/2813291549 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9e0107e10 0x7fd9e019d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:47.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:47.003+0000 7fd9deffd700 1 --2- 172.21.15.5:0/2813291549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9e01034e0 0x7fd9e019d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:47.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:47.004+0000 7fd9df7fe700 1 -- 172.21.15.5:0/2813291549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9e01034e0 msgr2=0x7fd9e019d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:47.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:47.004+0000 7fd9df7fe700 1 --2- 172.21.15.5:0/2813291549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9e01034e0 0x7fd9e019d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:47.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:47.004+0000 7fd9df7fe700 1 -- 172.21.15.5:0/2813291549 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9e0107e10 msgr2=0x7fd9e019d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:47.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:47.004+0000 7fd9df7fe700 1 --2- 172.21.15.5:0/2813291549 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9e0107e10 0x7fd9e019d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:47.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:47.004+0000 7fd9df7fe700 1 -- 172.21.15.5:0/2813291549 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9e01a1fa0 con 0x7fd9e0108760 2024-05-09T00:46:47.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:47.004+0000 7fd9df7fe700 1 --2- 172.21.15.5:0/2813291549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9e0108760 0x7fd9e01a1900 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd9d800b970 tx=0x7fd9d800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:47.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:47.004+0000 7fd9c7fff700 1 -- 172.21.15.5:0/2813291549 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d8019070 con 0x7fd9e0108760 2024-05-09T00:46:47.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:47.004+0000 7fd9e5309700 1 -- 172.21.15.5:0/2813291549 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9e01a2290 con 0x7fd9e0108760 2024-05-09T00:46:47.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:47.004+0000 7fd9c7fff700 1 -- 172.21.15.5:0/2813291549 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9d800cde0 con 0x7fd9e0108760 2024-05-09T00:46:47.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:47.004+0000 7fd9c7fff700 1 -- 172.21.15.5:0/2813291549 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d8007370 con 0x7fd9e0108760 2024-05-09T00:46:47.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:47.004+0000 7fd9e5309700 1 -- 172.21.15.5:0/2813291549 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9e01a9bb0 con 0x7fd9e0108760 2024-05-09T00:46:47.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:47.005+0000 7fd9e5309700 1 -- 172.21.15.5:0/2813291549 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9e00512c0 con 0x7fd9e0108760 2024-05-09T00:46:47.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:47.008+0000 7fd9c7fff700 1 -- 172.21.15.5:0/2813291549 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fd9d801c070 con 0x7fd9e0108760 2024-05-09T00:46:47.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:47.009+0000 7fd9c7fff700 1 --2- 172.21.15.5:0/2813291549 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fd9c80b2880 0x7fd9c80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:47.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:47.009+0000 7fd9deffd700 1 --2- 172.21.15.5:0/2813291549 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fd9c80b2880 0x7fd9c80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:47.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:47.009+0000 7fd9c7fff700 1 -- 172.21.15.5:0/2813291549 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fd9d80e1f00 con 0x7fd9e0108760 2024-05-09T00:46:47.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:47.009+0000 7fd9deffd700 1 --2- 172.21.15.5:0/2813291549 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fd9c80b2880 0x7fd9c80b4d90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd9d0005fd0 tx=0x7fd9d000a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:47.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:47.012+0000 7fd9c7fff700 1 -- 172.21.15.5:0/2813291549 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd9d8023070 con 0x7fd9e0108760 2024-05-09T00:46:47.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:47.228+0000 7fd9e5309700 1 -- 172.21.15.5:0/2813291549 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd9e01a9d80 con 0x7fd9e0108760 2024-05-09T00:46:47.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:47.230+0000 7fd9c7fff700 1 -- 172.21.15.5:0/2813291549 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22745 (secure 0 0 0) 0x7fd9d80a9b90 con 0x7fd9e0108760 2024-05-09T00:46:47.232 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:46:47.232 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":82,"fsid":"d85814c6-0d9b-11ef-bc97-c7b262605968","created":"2024-05-09T00:35:25.623377+0000","modified":"2024-05-09T00:45:23.926493+0000","last_up_change":"2024-05-09T00:42:38.525335+0000","last_in_change":"2024-05-09T00:42:17.990219+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T00:38:47.844734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T00:43:39.452106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T00:43:42.708106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f8d484bb-2477-4377-ae68-5bce571333bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2963385132},{"type":"v1","addr":"172.21.15.5:6803","nonce":2963385132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2963385132},{"type":"v1","addr":"172.21.15.5:6805","nonce":2963385132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2963385132},{"type":"v1","addr":"172.21.15.5:6809","nonce":2963385132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2963385132},{"type":"v1","addr":"172.21.15.5:6807","nonce":2963385132}]},"public_addr":"172.21.15.5:6803/2963385132","cluster_addr":"172.21.15.5:6805/2963385132","heartbeat_back_addr":"172.21.15.5:6809/2963385132","heartbeat_front_addr":"172.21.15.5:6807/2963385132","state":["exists","up"]},{"osd":1,"uuid":"1c123676-7144-4a67-93a6-fd49e36a5c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3837683163},{"type":"v1","addr":"172.21.15.131:6801","nonce":3837683163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3837683163},{"type":"v1","addr":"172.21.15.131:6803","nonce":3837683163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3837683163},{"type":"v1","addr":"172.21.15.131:6807","nonce":3837683163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3837683163},{"type":"v1","addr":"172.21.15.131:6805","nonce":3837683163}]},"public_addr":"172.21.15.131:6801/3837683163","cluster_addr":"172.21.15.131:6803/3837683163","heartbeat_back_addr":"172.21.15.131:6807/3837683163","heartbeat_front_addr":"172.21.15.131:6805/3837683163","state":["exists","up"]},{"osd":2,"uuid":"95ca51f0-b48b-485c-a5f7-b15c01e754b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":484818139},{"type":"v1","addr":"172.21.15.157:6801","nonce":484818139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":484818139},{"type":"v1","addr":"172.21.15.157:6803","nonce":484818139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":484818139},{"type":"v1","addr":"172.21.15.157:6807","nonce":484818139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":484818139},{"type":"v1","addr":"172.21.15.157:6805","nonce":484818139}]},"public_addr":"172.21.15.157:6801/484818139","cluster_addr":"172.21.15.157:6803/484818139","heartbeat_back_addr":"172.21.15.157:6807/484818139","heartbeat_front_addr":"172.21.15.157:6805/484818139","state":["exists","up"]},{"osd":3,"uuid":"ed7577fb-3676-4a08-b40d-5474eb88974f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":2357377277},{"type":"v1","addr":"172.21.15.5:6811","nonce":2357377277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":2357377277},{"type":"v1","addr":"172.21.15.5:6813","nonce":2357377277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":2357377277},{"type":"v1","addr":"172.21.15.5:6817","nonce":2357377277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":2357377277},{"type":"v1","addr":"172.21.15.5:6815","nonce":2357377277}]},"public_addr":"172.21.15.5:6811/2357377277","cluster_addr":"172.21.15.5:6813/2357377277","heartbeat_back_addr":"172.21.15.5:6817/2357377277","heartbeat_front_addr":"172.21.15.5:6815/2357377277","state":["exists","up"]},{"osd":4,"uuid":"fb5dcf79-a6f1-4b1a-9e3a-087fb5132472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":2915418378},{"type":"v1","addr":"172.21.15.131:6809","nonce":2915418378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":2915418378},{"type":"v1","addr":"172.21.15.131:6811","nonce":2915418378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":2915418378},{"type":"v1","addr":"172.21.15.131:6815","nonce":2915418378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":2915418378},{"type":"v1","addr":"172.21.15.131:6813","nonce":2915418378}]},"public_addr":"172.21.15.131:6809/2915418378","cluster_addr":"172.21.15.131:6811/2915418378","heartbeat_back_addr":"172.21.15.131:6815/2915418378","heartbeat_front_addr":"172.21.15.131:6813/2915418378","state":["exists","up"]},{"osd":5,"uuid":"f01ed5d3-8066-4d25-a113-280c83e75ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4188818663},{"type":"v1","addr":"172.21.15.157:6809","nonce":4188818663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":4188818663},{"type":"v1","addr":"172.21.15.157:6811","nonce":4188818663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":4188818663},{"type":"v1","addr":"172.21.15.157:6815","nonce":4188818663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":4188818663},{"type":"v1","addr":"172.21.15.157:6813","nonce":4188818663}]},"public_addr":"172.21.15.157:6809/4188818663","cluster_addr":"172.21.15.157:6811/4188818663","heartbeat_back_addr":"172.21.15.157:6815/4188818663","heartbeat_front_addr":"172.21.15.157:6813/4188818663","state":["exists","up"]},{"osd":6,"uuid":"932379a8-c97b-4acc-8665-81f7d9cbce65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2930397813},{"type":"v1","addr":"172.21.15.5:6819","nonce":2930397813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2930397813},{"type":"v1","addr":"172.21.15.5:6821","nonce":2930397813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2930397813},{"type":"v1","addr":"172.21.15.5:6825","nonce":2930397813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2930397813},{"type":"v1","addr":"172.21.15.5:6823","nonce":2930397813}]},"public_addr":"172.21.15.5:6819/2930397813","cluster_addr":"172.21.15.5:6821/2930397813","heartbeat_back_addr":"172.21.15.5:6825/2930397813","heartbeat_front_addr":"172.21.15.5:6823/2930397813","state":["exists","up"]},{"osd":7,"uuid":"3c0a0573-6465-4bf6-b70f-310d49857a6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":4125045219},{"type":"v1","addr":"172.21.15.131:6817","nonce":4125045219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":4125045219},{"type":"v1","addr":"172.21.15.131:6819","nonce":4125045219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":4125045219},{"type":"v1","addr":"172.21.15.131:6823","nonce":4125045219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":4125045219},{"type":"v1","addr":"172.21.15.131:6821","nonce":4125045219}]},"public_addr":"172.21.15.131:6817/4125045219","cluster_addr":"172.21.15.131:6819/4125045219","heartbeat_back_addr":"172.21.15.131:6823/4125045219","heartbeat_front_addr":"172.21.15.131:6821/4125045219","state":["exists","up"]},{"osd":8,"uuid":"c49f0a00-f5ae-40ed-8f38-7b953ba1c17c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2001626768},{"type":"v1","addr":"172.21.15.157:6817","nonce":2001626768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2001626768},{"type":"v1","addr":"172.21.15.157:6819","nonce":2001626768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2001626768},{"type":"v1","addr":"172.21.15.157:6823","nonce":2001626768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2001626768},{"type":"v1","addr":"172.21.15.157:6821","nonce":2001626768}]},"public_addr":"172.21.15.157:6817/2001626768","cluster_addr":"172.21.15.157:6819/2001626768","heartbeat_back_addr":"172.21.15.157:6823/2001626768","heartbeat_front_addr":"172.21.15.157:6821/2001626768","state":["exists","up"]},{"osd":9,"uuid":"1927dfe7-c1af-4a15-bb71-d58d97c8734a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":4287220015},{"type":"v1","addr":"172.21.15.5:6827","nonce":4287220015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":4287220015},{"type":"v1","addr":"172.21.15.5:6829","nonce":4287220015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":4287220015},{"type":"v1","addr":"172.21.15.5:6833","nonce":4287220015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":4287220015},{"type":"v1","addr":"172.21.15.5:6831","nonce":4287220015}]},"public_addr":"172.21.15.5:6827/4287220015","cluster_addr":"172.21.15.5:6829/4287220015","heartbeat_back_addr":"172.21.15.5:6833/4287220015","heartbeat_front_addr":"172.21.15.5:6831/4287220015","state":["exists","up"]},{"osd":10,"uuid":"fbb4f3e3-eb7d-4ecb-a30b-55b8910375ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1571783985},{"type":"v1","addr":"172.21.15.131:6825","nonce":1571783985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1571783985},{"type":"v1","addr":"172.21.15.131:6827","nonce":1571783985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1571783985},{"type":"v1","addr":"172.21.15.131:6831","nonce":1571783985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1571783985},{"type":"v1","addr":"172.21.15.131:6829","nonce":1571783985}]},"public_addr":"172.21.15.131:6825/1571783985","cluster_addr":"172.21.15.131:6827/1571783985","heartbeat_back_addr":"172.21.15.131:6831/1571783985","heartbeat_front_addr":"172.21.15.131:6829/1571783985","state":["exists","up"]},{"osd":11,"uuid":"b858453f-eda1-4e25-89c8-94b54c830938","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3261151188},{"type":"v1","addr":"172.21.15.157:6825","nonce":3261151188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":3261151188},{"type":"v1","addr":"172.21.15.157:6827","nonce":3261151188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":3261151188},{"type":"v1","addr":"172.21.15.157:6831","nonce":3261151188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":3261151188},{"type":"v1","addr":"172.21.15.157:6829","nonce":3261151188}]},"public_addr":"172.21.15.157:6825/3261151188","cluster_addr":"172.21.15.157:6827/3261151188","heartbeat_back_addr":"172.21.15.157:6831/3261151188","heartbeat_front_addr":"172.21.15.157:6829/3261151188","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:37:57.528483+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:38:21.477482+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:38:45.780266+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:39:10.938791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:39:35.734857+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:40:00.169040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:40:26.494635+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:40:51.881718+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:41:17.697112+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:41:44.826479+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:42:10.504768+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:42:36.605447+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6},{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":6}]},{"pgid":"3.19","mappings":[{"from":3,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":7}]},{"pgid":"3.35","mappings":[{"from":1,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/3405015786":"2024-05-10T00:36:21.742363+0000","172.21.15.5:0/2669115557":"2024-05-10T00:36:21.742363+0000","172.21.15.5:6801/287842266":"2024-05-10T00:36:21.742363+0000","172.21.15.5:6800/752513054":"2024-05-10T00:35:53.128955+0000","172.21.15.5:6800/287842266":"2024-05-10T00:36:21.742363+0000","172.21.15.5:6801/752513054":"2024-05-10T00:35:53.128955+0000","172.21.15.5:0/1954876662":"2024-05-10T00:35:53.128955+0000","172.21.15.5:0/2545175121":"2024-05-10T00:36:21.742363+0000","172.21.15.5:0/3754653663":"2024-05-10T00:35:53.128955+0000","172.21.15.5:0/651514359":"2024-05-10T00:35:53.128955+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T00:46:47.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:47.232+0000 7fd9e5309700 1 -- 172.21.15.5:0/2813291549 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fd9c80b2880 msgr2=0x7fd9c80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:47.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:47.232+0000 7fd9e5309700 1 --2- 172.21.15.5:0/2813291549 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fd9c80b2880 0x7fd9c80b4d90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd9d0005fd0 tx=0x7fd9d000a560 comp rx=0 tx=0).stop 2024-05-09T00:46:47.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:47.232+0000 7fd9e5309700 1 -- 172.21.15.5:0/2813291549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9e0108760 msgr2=0x7fd9e01a1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:47.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:47.232+0000 7fd9e5309700 1 --2- 172.21.15.5:0/2813291549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9e0108760 0x7fd9e01a1900 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd9d800b970 tx=0x7fd9d800bc80 comp rx=0 tx=0).stop 2024-05-09T00:46:47.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:47.232+0000 7fd9e5309700 1 -- 172.21.15.5:0/2813291549 shutdown_connections 2024-05-09T00:46:47.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:47.232+0000 7fd9e5309700 1 --2- 172.21.15.5:0/2813291549 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fd9c80b2880 0x7fd9c80b4d90 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:47.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:47.232+0000 7fd9e5309700 1 --2- 172.21.15.5:0/2813291549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9e0108760 0x7fd9e01a1900 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:47.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:47.232+0000 7fd9e5309700 1 --2- 172.21.15.5:0/2813291549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9e01034e0 0x7fd9e019d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:47.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:47.232+0000 7fd9e5309700 1 --2- 172.21.15.5:0/2813291549 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9e0107e10 0x7fd9e019d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:47.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:47.232+0000 7fd9e5309700 1 -- 172.21.15.5:0/2813291549 >> 172.21.15.5:0/2813291549 conn(0x7fd9e00fef30 msgr2=0x7fd9e01096c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:47.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:47.232+0000 7fd9e5309700 1 -- 172.21.15.5:0/2813291549 shutdown_connections 2024-05-09T00:46:47.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:47.232+0000 7fd9e5309700 1 -- 172.21.15.5:0/2813291549 wait complete. 2024-05-09T00:46:47.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:46:46 smithi131 ceph-mon[34547]: pgmap v384: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T00:46:47.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:46:46 smithi157 ceph-mon[34294]: pgmap v384: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T00:46:48.024 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-09T00:43:45.856066+0000', 'modified': '2024-05-09T00:44:06.033927+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24508, 'mds_1': 24499, 'mds_2': 24494, 'mds_3': 24493, 'mds_4': 24467}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14598': {'gid': 14598, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.5:6839/2109255228', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6838', 'nonce': 2109255228}, {'type': 'v1', 'addr': '172.21.15.5:6839', 'nonce': 2109255228}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14610': {'gid': 14610, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.5:6841/3809966329', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6840', 'nonce': 3809966329}, {'type': 'v1', 'addr': '172.21.15.5:6841', 'nonce': 3809966329}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24419': {'gid': 24419, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.131:6833/3272897742', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6832', 'nonce': 3272897742}, {'type': 'v1', 'addr': '172.21.15.131:6833', 'nonce': 3272897742}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24443': {'gid': 24443, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.131:6835/1283093898', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6834', 'nonce': 1283093898}, {'type': 'v1', 'addr': '172.21.15.131:6835', 'nonce': 1283093898}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24457': {'gid': 24457, 'name': 'l', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.157:6833/2809525455', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6832', 'nonce': 2809525455}, {'type': 'v1', 'addr': '172.21.15.157:6833', 'nonce': 2809525455}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24467': {'gid': 24467, 'name': 'h', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.131:6837/1633006186', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6836', 'nonce': 1633006186}, {'type': 'v1', 'addr': '172.21.15.131:6837', 'nonce': 1633006186}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24493': {'gid': 24493, 'name': 'f', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.157:6835/832898273', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6834', 'nonce': 832898273}, {'type': 'v1', 'addr': '172.21.15.157:6835', 'nonce': 832898273}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24494': {'gid': 24494, 'name': 'b', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.131:6839/2748669631', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6838', 'nonce': 2748669631}, {'type': 'v1', 'addr': '172.21.15.131:6839', 'nonce': 2748669631}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24499': {'gid': 24499, 'name': 'c', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.157:6837/744863009', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6836', 'nonce': 744863009}, {'type': 'v1', 'addr': '172.21.15.157:6837', 'nonce': 744863009}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24508': {'gid': 24508, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.157:6839/2779860399', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6838', 'nonce': 2779860399}, {'type': 'v1', 'addr': '172.21.15.157:6839', 'nonce': 2779860399}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24508, 'qdb_cluster': [24467, 24493, 24499, 24508, 24494]} 2024-05-09T00:46:48.024 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-05-09T00:46:48.024 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.h status 2024-05-09T00:46:48.278 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:46:48.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:46:47 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/2813291549' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:46:48.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:46:47 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/2813291549' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:46:48.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:46:47 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/2813291549' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:46:49.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:46:48 smithi005 ceph-mon[30365]: pgmap v385: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T00:46:49.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:46:48 smithi131 ceph-mon[34547]: pgmap v385: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T00:46:49.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:46:48 smithi157 ceph-mon[34294]: pgmap v385: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T00:46:50.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.017+0000 7f4fa0c49700 1 -- 172.21.15.5:0/2810678088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f9c107df0 msgr2=0x7f4f9c108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:50.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.017+0000 7f4fa0c49700 1 --2- 172.21.15.5:0/2810678088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f9c107df0 0x7f4f9c108200 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c0099c0 tx=0x7f4f8c009cd0 comp rx=0 tx=0).stop 2024-05-09T00:46:50.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.018+0000 7f4fa0c49700 1 -- 172.21.15.5:0/2810678088 shutdown_connections 2024-05-09T00:46:50.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.018+0000 7f4fa0c49700 1 --2- 172.21.15.5:0/2810678088 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f9c108740 0x7f4f9c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:50.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.018+0000 7f4fa0c49700 1 --2- 172.21.15.5:0/2810678088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f9c1034c0 0x7f4f9c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:50.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.018+0000 7f4fa0c49700 1 --2- 172.21.15.5:0/2810678088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f9c107df0 0x7f4f9c108200 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:50.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.018+0000 7f4fa0c49700 1 -- 172.21.15.5:0/2810678088 >> 172.21.15.5:0/2810678088 conn(0x7f4f9c0fedc0 msgr2=0x7f4f9c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:50.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.018+0000 7f4fa0c49700 1 -- 172.21.15.5:0/2810678088 shutdown_connections 2024-05-09T00:46:50.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.018+0000 7f4fa0c49700 1 -- 172.21.15.5:0/2810678088 wait complete. 2024-05-09T00:46:50.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.019+0000 7f4fa0c49700 1 Processor -- start 2024-05-09T00:46:50.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.019+0000 7f4fa0c49700 1 -- start start 2024-05-09T00:46:50.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.019+0000 7f4fa0c49700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f9c1034c0 0x7f4f9c199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:50.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.019+0000 7f4fa0c49700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f9c107df0 0x7f4f9c19a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:50.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.019+0000 7f4fa0c49700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f9c108740 0x7f4f9c19e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:50.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.019+0000 7f4fa0c49700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9c1133a0 con 0x7f4f9c108740 2024-05-09T00:46:50.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.019+0000 7f4fa0c49700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9c113220 con 0x7f4f9c107df0 2024-05-09T00:46:50.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.019+0000 7f4fa0c49700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9c113520 con 0x7f4f9c1034c0 2024-05-09T00:46:50.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.019+0000 7f4f99d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f9c107df0 0x7f4f9c19a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:50.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.019+0000 7f4f9ad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f9c108740 0x7f4f9c19e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:50.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.019+0000 7f4f9a59c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f9c1034c0 0x7f4f9c199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:50.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.019+0000 7f4f99d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f9c107df0 0x7f4f9c19a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:36130/0 (socket says 172.21.15.5:36130) 2024-05-09T00:46:50.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.019+0000 7f4f99d9b700 1 -- 172.21.15.5:0/2105656475 learned_addr learned my addr 172.21.15.5:0/2105656475 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:46:50.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.020+0000 7f4f99d9b700 1 -- 172.21.15.5:0/2105656475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f9c1034c0 msgr2=0x7f4f9c199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:50.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.020+0000 7f4f99d9b700 1 --2- 172.21.15.5:0/2105656475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f9c1034c0 0x7f4f9c199d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:50.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.020+0000 7f4f99d9b700 1 -- 172.21.15.5:0/2105656475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f9c108740 msgr2=0x7f4f9c19e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:50.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.020+0000 7f4f99d9b700 1 --2- 172.21.15.5:0/2105656475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f9c108740 0x7f4f9c19e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:50.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.020+0000 7f4f99d9b700 1 -- 172.21.15.5:0/2105656475 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f9c19ec20 con 0x7f4f9c107df0 2024-05-09T00:46:50.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.020+0000 7f4f99d9b700 1 --2- 172.21.15.5:0/2105656475 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f9c107df0 0x7f4f9c19a2f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c00b580 tx=0x7f4f8c003680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:50.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.020+0000 7f4f937fe700 1 -- 172.21.15.5:0/2105656475 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c0039c0 con 0x7f4f9c107df0 2024-05-09T00:46:50.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.020+0000 7f4fa0c49700 1 -- 172.21.15.5:0/2105656475 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f9c19eeb0 con 0x7f4f9c107df0 2024-05-09T00:46:50.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.020+0000 7f4f937fe700 1 -- 172.21.15.5:0/2105656475 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f8c0054e0 con 0x7f4f9c107df0 2024-05-09T00:46:50.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.020+0000 7f4f937fe700 1 -- 172.21.15.5:0/2105656475 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c004990 con 0x7f4f9c107df0 2024-05-09T00:46:50.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.020+0000 7f4fa0c49700 1 -- 172.21.15.5:0/2105656475 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f9c06b610 con 0x7f4f9c107df0 2024-05-09T00:46:50.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.023+0000 7f4f937fe700 1 -- 172.21.15.5:0/2105656475 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f4f8c004b30 con 0x7f4f9c107df0 2024-05-09T00:46:50.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.024+0000 7f4f937fe700 1 --2- 172.21.15.5:0/2105656475 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f4f880b2880 0x7f4f880b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:50.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.024+0000 7f4faf65fb80 1 Processor -- start 2024-05-09T00:46:50.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.024+0000 7f4f9a59c700 1 --2- 172.21.15.5:0/2105656475 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f4f880b2880 0x7f4f880b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:50.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.024+0000 7f4f937fe700 1 -- 172.21.15.5:0/2105656475 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f4f8c0e5aa0 con 0x7f4f9c107df0 2024-05-09T00:46:50.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.024+0000 7f4faf65fb80 1 -- start start 2024-05-09T00:46:50.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.025+0000 7f4faf65fb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a4a9f5110 0x564a4aa8cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:50.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.025+0000 7f4faf65fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a4a90b450 0x564a4aa8d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:50.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.025+0000 7f4faf65fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a4aa913b0 0x564a4aa917c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:50.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.025+0000 7f4faf65fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564a4a9aafa0 con 0x564a4a90b450 2024-05-09T00:46:50.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.025+0000 7f4faf65fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x564a4a9497a0 con 0x564a4aa913b0 2024-05-09T00:46:50.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.025+0000 7f4faf65fb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x564a4a9a6f60 con 0x564a4a9f5110 2024-05-09T00:46:50.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.025+0000 7f4f99d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a4a90b450 0x564a4aa8d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:50.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.025+0000 7f4f99d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a4a90b450 0x564a4aa8d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32998/0 (socket says 172.21.15.5:32998) 2024-05-09T00:46:50.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.025+0000 7f4f99d9b700 1 -- 172.21.15.5:0/1273055086 learned_addr learned my addr 172.21.15.5:0/1273055086 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:46:50.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.025+0000 7f4f9a59c700 1 --2- 172.21.15.5:0/1273055086 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a4a9f5110 0x564a4aa8cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:50.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.025+0000 7f4f9a59c700 1 --2- 172.21.15.5:0/2105656475 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f4f880b2880 0x7f4f880b4d90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4f84009c60 tx=0x7f4f84009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:50.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.025+0000 7f4f9ad9d700 1 --2- 172.21.15.5:0/1273055086 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a4aa913b0 0x564a4aa917c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:50.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.025+0000 7f4f9a59c700 1 -- 172.21.15.5:0/1273055086 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a4aa913b0 msgr2=0x564a4aa917c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:50.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.025+0000 7f4f9a59c700 1 --2- 172.21.15.5:0/1273055086 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a4aa913b0 0x564a4aa917c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:50.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.025+0000 7f4f9a59c700 1 -- 172.21.15.5:0/1273055086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a4a90b450 msgr2=0x564a4aa8d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:50.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.025+0000 7f4f9a59c700 1 --2- 172.21.15.5:0/1273055086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a4a90b450 0x564a4aa8d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:50.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.025+0000 7f4f9a59c700 1 -- 172.21.15.5:0/1273055086 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a4a941320 con 0x564a4a9f5110 2024-05-09T00:46:50.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.025+0000 7f4f99d9b700 1 --2- 172.21.15.5:0/1273055086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a4a90b450 0x564a4aa8d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:46:50.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.025+0000 7f4f9a59c700 1 --2- 172.21.15.5:0/1273055086 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a4a9f5110 0x564a4aa8cd00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4f8400e3b0 tx=0x7f4f84014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4aa3d770ea9c6cd3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:50.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.026+0000 7f4f90ff9700 1 -- 172.21.15.5:0/1273055086 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8400ed50 con 0x564a4a9f5110 2024-05-09T00:46:50.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.026+0000 7f4f90ff9700 1 -- 172.21.15.5:0/1273055086 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f840044e0 con 0x564a4a9f5110 2024-05-09T00:46:50.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.026+0000 7f4f90ff9700 1 -- 172.21.15.5:0/1273055086 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f840266d0 con 0x564a4a9f5110 2024-05-09T00:46:50.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.027+0000 7f4faf65fb80 1 -- 172.21.15.5:0/1273055086 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a4a9f5110 msgr2=0x564a4aa8cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:50.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.027+0000 7f4faf65fb80 1 --2- 172.21.15.5:0/1273055086 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a4a9f5110 0x564a4aa8cd00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4f8400e3b0 tx=0x7f4f84014000 comp rx=0 tx=0).stop 2024-05-09T00:46:50.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.027+0000 7f4faf65fb80 1 -- 172.21.15.5:0/1273055086 shutdown_connections 2024-05-09T00:46:50.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.027+0000 7f4faf65fb80 1 --2- 172.21.15.5:0/1273055086 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a4aa913b0 0x564a4aa917c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:50.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.027+0000 7f4faf65fb80 1 --2- 172.21.15.5:0/1273055086 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a4a9f5110 0x564a4aa8cd00 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:50.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.027+0000 7f4faf65fb80 1 --2- 172.21.15.5:0/1273055086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a4a90b450 0x564a4aa8d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:50.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.027+0000 7f4faf65fb80 1 -- 172.21.15.5:0/1273055086 >> 172.21.15.5:0/1273055086 conn(0x564a4a96b650 msgr2=0x564a4aa833b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:50.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.027+0000 7f4faf65fb80 1 -- 172.21.15.5:0/1273055086 shutdown_connections 2024-05-09T00:46:50.032 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:46:49 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:46:50.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.027+0000 7f4faf65fb80 1 -- 172.21.15.5:0/1273055086 wait complete. 2024-05-09T00:46:50.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.027+0000 7f4faf65fb80 1 -- start start 2024-05-09T00:46:50.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.027+0000 7f4faf65fb80 1 Processor -- start 2024-05-09T00:46:50.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.028+0000 7f4faf65fb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a4a90b450 0x564a4aa96530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:50.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.028+0000 7f4faf65fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a4a9f5110 0x564a4aa96a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:50.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.028+0000 7f4faf65fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a4aa913b0 0x564a4aa96fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:50.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.028+0000 7f4faf65fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564a4a826b50 con 0x564a4aa913b0 2024-05-09T00:46:50.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.028+0000 7f4faf65fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x564a4a965100 con 0x564a4a9f5110 2024-05-09T00:46:50.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.028+0000 7f4faf65fb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x564a4aa92160 con 0x564a4a90b450 2024-05-09T00:46:50.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.028+0000 7f4faf65fb80 10 client.0 _ll_register_callbacks cb 0x564a4a81c670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:46:50.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.028+0000 7f4f9ad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a4aa913b0 0x564a4aa96fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:50.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.028+0000 7f4f9a59c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a4a90b450 0x564a4aa96530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:50.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.028+0000 7f4f9ad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a4aa913b0 0x564a4aa96fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33002/0 (socket says 172.21.15.5:33002) 2024-05-09T00:46:50.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.028+0000 7f4f9ad9d700 1 -- 172.21.15.5:0/1100286533 learned_addr learned my addr 172.21.15.5:0/1100286533 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:46:50.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.028+0000 7f4f99d9b700 1 --2- 172.21.15.5:0/1100286533 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a4a9f5110 0x564a4aa96a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:50.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.028+0000 7f4f9a59c700 1 -- 172.21.15.5:0/1100286533 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a4a9f5110 msgr2=0x564a4aa96a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:50.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.028+0000 7f4f9a59c700 1 --2- 172.21.15.5:0/1100286533 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a4a9f5110 0x564a4aa96a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:50.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.028+0000 7f4f9a59c700 1 -- 172.21.15.5:0/1100286533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a4aa913b0 msgr2=0x564a4aa96fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:50.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.028+0000 7f4f9a59c700 1 --2- 172.21.15.5:0/1100286533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a4aa913b0 0x564a4aa96fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:50.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.028+0000 7f4f9a59c700 1 -- 172.21.15.5:0/1100286533 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a4a826440 con 0x564a4a90b450 2024-05-09T00:46:50.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.029+0000 7f4f9a59c700 1 --2- 172.21.15.5:0/1100286533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a4a90b450 0x564a4aa96530 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4f84003ca0 tx=0x7f4f84026650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fad72cde9e2fbb1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:50.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.029+0000 7f4f81ffb700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-09T00:46:50.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.029+0000 7f4f81ffb700 1 -- 172.21.15.5:0/1100286533 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f84026990 con 0x564a4a90b450 2024-05-09T00:46:50.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.029+0000 7f4faf65fb80 1 -- 172.21.15.5:0/1100286533 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f840044e0 con 0x564a4a90b450 2024-05-09T00:46:50.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.029+0000 7f4f81ffb700 1 -- 172.21.15.5:0/1100286533 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f4f84026b30 con 0x564a4a90b450 2024-05-09T00:46:50.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.029+0000 7f4f81ffb700 1 -- 172.21.15.5:0/1100286533 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4f84026d30 con 0x564a4a90b450 2024-05-09T00:46:50.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.029+0000 7f4faf65fb80 10 client.24925 fetch_fsmap learned FSMap version 28 2024-05-09T00:46:50.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.029+0000 7f4faf65fb80 1 -- 172.21.15.5:0/1100286533 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564a4aaa3e50 con 0x564a4a90b450 2024-05-09T00:46:50.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.029+0000 7f4f81ffb700 1 -- 172.21.15.5:0/1100286533 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f840044e0 con 0x564a4a90b450 2024-05-09T00:46:50.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.030+0000 7f4f81ffb700 1 -- 172.21.15.5:0/1100286533 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f4f84039730 con 0x564a4a90b450 2024-05-09T00:46:50.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.030+0000 7f4faf65fb80 10 client.24925 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T00:46:50.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.030+0000 7f4faf65fb80 10 client.24925 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-09T00:46:50.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.030+0000 7f4faf65fb80 20 client.24925 populate_metadata read hostname 'smithi005' 2024-05-09T00:46:50.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.030+0000 7f4faf65fb80 1 --2- 172.21.15.5:0/1100286533 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x564a4aaa6000 0x564a4aaa7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:50.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.030+0000 7f4faf65fb80 4 client.24925 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:46:50.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.030+0000 7f4faf65fb80 1 -- 172.21.15.5:0/1100286533 --> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a4a9b8980 con 0x564a4aaa6000 2024-05-09T00:46:50.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.030+0000 7f4f99d9b700 1 --2- 172.21.15.5:0/1100286533 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x564a4aaa6000 0x564a4aaa7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:50.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.030+0000 7f4f81ffb700 10 client.24925 ms_handle_connect on v2:172.21.15.131:6836/1633006186 2024-05-09T00:46:50.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.030+0000 7f4f99d9b700 1 --2- 172.21.15.5:0/1100286533 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x564a4aaa6000 0x564a4aaa7470 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7e5ce5f5908924f9 server_cookie=da2677403e27ea67 in_seq=0 out_seq=0 2024-05-09T00:46:50.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.031+0000 7f4f81ffb700 1 -- 172.21.15.5:0/1100286533 <== mds.4 v2:172.21.15.131:6836/1633006186 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564a4a9b8980 con 0x564a4aaa6000 2024-05-09T00:46:50.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.031+0000 7f4f81ffb700 10 client.24925 handle_command_reply: tid=0 2024-05-09T00:46:50.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.031+0000 7f4faf65fb80 1 client.24925 shutdown 2024-05-09T00:46:50.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.031+0000 7f4faf65fb80 1 -- 172.21.15.5:0/1100286533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a4a90b450 msgr2=0x564a4aa96530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:50.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.031+0000 7f4faf65fb80 1 --2- 172.21.15.5:0/1100286533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a4a90b450 0x564a4aa96530 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4f84003ca0 tx=0x7f4f84026650 comp rx=0 tx=0).stop 2024-05-09T00:46:50.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.031+0000 7f4faf65fb80 1 -- 172.21.15.5:0/1100286533 shutdown_connections 2024-05-09T00:46:50.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.031+0000 7f4faf65fb80 1 --2- 172.21.15.5:0/1100286533 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x564a4aaa6000 0x564a4aaa7470 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:50.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.031+0000 7f4f81ffb700 0 client.24925 ms_handle_reset on v2:172.21.15.131:6836/1633006186 2024-05-09T00:46:50.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.031+0000 7f4faf65fb80 1 --2- 172.21.15.5:0/1100286533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a4aa913b0 0x564a4aa96fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:50.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.031+0000 7f4faf65fb80 1 --2- 172.21.15.5:0/1100286533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a4a90b450 0x564a4aa96530 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:50.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.031+0000 7f4faf65fb80 1 --2- 172.21.15.5:0/1100286533 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a4a9f5110 0x564a4aa96a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:50.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.032+0000 7f4faf65fb80 1 -- 172.21.15.5:0/1100286533 >> 172.21.15.5:0/1100286533 conn(0x564a4a96b650 msgr2=0x564a4aa83170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:50.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.032+0000 7f4faf65fb80 1 -- 172.21.15.5:0/1100286533 shutdown_connections 2024-05-09T00:46:50.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.032+0000 7f4faf65fb80 1 -- 172.21.15.5:0/1100286533 wait complete. 2024-05-09T00:46:50.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.032+0000 7f4faf65fb80 20 client.24925 trim_cache size 0 max 16384 2024-05-09T00:46:50.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.053+0000 7f4faf65fb80 1 Processor -- start 2024-05-09T00:46:50.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.053+0000 7f4faf65fb80 1 -- start start 2024-05-09T00:46:50.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.054+0000 7f4faf65fb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a4a90b450 0x564a4aaa2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:50.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.054+0000 7f4faf65fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a4a9f5110 0x564a4aa96df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:50.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.054+0000 7f4faf65fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a4aa913b0 0x564a4aa909c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:50.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.054+0000 7f4faf65fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564a4aaa57b0 con 0x564a4a9f5110 2024-05-09T00:46:50.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.054+0000 7f4faf65fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x564a4aaa5e80 con 0x564a4aa913b0 2024-05-09T00:46:50.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.054+0000 7f4faf65fb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x564a4aaa6550 con 0x564a4a90b450 2024-05-09T00:46:50.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.054+0000 7f4f99d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a4a9f5110 0x564a4aa96df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:50.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.054+0000 7f4f99d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a4a9f5110 0x564a4aa96df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33006/0 (socket says 172.21.15.5:33006) 2024-05-09T00:46:50.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.054+0000 7f4f99d9b700 1 -- 172.21.15.5:0/1638767143 learned_addr learned my addr 172.21.15.5:0/1638767143 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:46:50.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.054+0000 7f4f9ad9d700 1 --2- 172.21.15.5:0/1638767143 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a4aa913b0 0x564a4aa909c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:50.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.054+0000 7f4f9a59c700 1 --2- 172.21.15.5:0/1638767143 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a4a90b450 0x564a4aaa2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:50.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.054+0000 7f4f99d9b700 1 -- 172.21.15.5:0/1638767143 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a4a90b450 msgr2=0x564a4aaa2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:50.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.054+0000 7f4f99d9b700 1 --2- 172.21.15.5:0/1638767143 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a4a90b450 0x564a4aaa2740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:50.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.054+0000 7f4f99d9b700 1 -- 172.21.15.5:0/1638767143 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a4aa913b0 msgr2=0x564a4aa909c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:50.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.054+0000 7f4f99d9b700 1 --2- 172.21.15.5:0/1638767143 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a4aa913b0 0x564a4aa909c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:50.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.054+0000 7f4f99d9b700 1 -- 172.21.15.5:0/1638767143 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a4aa911e0 con 0x564a4a9f5110 2024-05-09T00:46:50.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.055+0000 7f4f9a59c700 1 --2- 172.21.15.5:0/1638767143 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a4a90b450 0x564a4aaa2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:46:50.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.055+0000 7f4f99d9b700 1 --2- 172.21.15.5:0/1638767143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a4a9f5110 0x564a4aa96df0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c0aa330 tx=0x7f4f8c003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81e75dc4f1def917 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:50.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.055+0000 7f4f83fff700 1 -- 172.21.15.5:0/1638767143 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c0c7070 con 0x564a4a9f5110 2024-05-09T00:46:50.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.055+0000 7f4f83fff700 1 -- 172.21.15.5:0/1638767143 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f8c0b1470 con 0x564a4a9f5110 2024-05-09T00:46:50.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.055+0000 7f4f83fff700 1 -- 172.21.15.5:0/1638767143 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c0ad320 con 0x564a4a9f5110 2024-05-09T00:46:50.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.056+0000 7f4faf65fb80 1 -- 172.21.15.5:0/1638767143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a4a9f5110 msgr2=0x564a4aa96df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:50.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.056+0000 7f4faf65fb80 1 --2- 172.21.15.5:0/1638767143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a4a9f5110 0x564a4aa96df0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c0aa330 tx=0x7f4f8c003f90 comp rx=0 tx=0).stop 2024-05-09T00:46:50.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.056+0000 7f4faf65fb80 1 -- 172.21.15.5:0/1638767143 shutdown_connections 2024-05-09T00:46:50.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.056+0000 7f4faf65fb80 1 --2- 172.21.15.5:0/1638767143 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a4aa913b0 0x564a4aa909c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:50.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.056+0000 7f4faf65fb80 1 --2- 172.21.15.5:0/1638767143 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a4a90b450 0x564a4aaa2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:50.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.056+0000 7f4faf65fb80 1 --2- 172.21.15.5:0/1638767143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a4a9f5110 0x564a4aa96df0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:50.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.056+0000 7f4faf65fb80 1 -- 172.21.15.5:0/1638767143 >> 172.21.15.5:0/1638767143 conn(0x564a4a96b650 msgr2=0x564a4aa902b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:50.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.056+0000 7f4faf65fb80 1 -- 172.21.15.5:0/1638767143 shutdown_connections 2024-05-09T00:46:50.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.056+0000 7f4faf65fb80 1 -- 172.21.15.5:0/1638767143 wait complete. 2024-05-09T00:46:50.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.056+0000 7f4faf65fb80 1 -- start start 2024-05-09T00:46:50.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.056+0000 7f4faf65fb80 1 Processor -- start 2024-05-09T00:46:50.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.057+0000 7f4faf65fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a4a90b450 0x564a4aa92ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:50.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.057+0000 7f4faf65fb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a4a9f5110 0x564a4aa93410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:50.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.057+0000 7f4faf65fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a4aa913b0 0x564a4aaaf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:50.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.057+0000 7f4faf65fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564a4aaa72f0 con 0x564a4aa913b0 2024-05-09T00:46:50.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.057+0000 7f4faf65fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x564a4aaa6c20 con 0x564a4a90b450 2024-05-09T00:46:50.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.057+0000 7f4faf65fb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x564a4aaa79c0 con 0x564a4a9f5110 2024-05-09T00:46:50.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.057+0000 7f4f9ad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a4aa913b0 0x564a4aaaf8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:50.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.057+0000 7f4f9ad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a4aa913b0 0x564a4aaaf8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33014/0 (socket says 172.21.15.5:33014) 2024-05-09T00:46:50.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.057+0000 7f4faf65fb80 10 client.0 _ll_register_callbacks cb 0x564a4a81c670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:46:50.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.057+0000 7f4f9ad9d700 1 -- 172.21.15.5:0/3340447664 learned_addr learned my addr 172.21.15.5:0/3340447664 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:46:50.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.057+0000 7f4f99d9b700 1 --2- 172.21.15.5:0/3340447664 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a4a9f5110 0x564a4aa93410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:50.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.057+0000 7f4f9a59c700 1 --2- 172.21.15.5:0/3340447664 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a4a90b450 0x564a4aa92ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:50.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.057+0000 7f4f9ad9d700 1 -- 172.21.15.5:0/3340447664 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a4a9f5110 msgr2=0x564a4aa93410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:50.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.057+0000 7f4f9ad9d700 1 --2- 172.21.15.5:0/3340447664 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a4a9f5110 0x564a4aa93410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:50.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.057+0000 7f4f9ad9d700 1 -- 172.21.15.5:0/3340447664 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a4a90b450 msgr2=0x564a4aa92ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:50.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.057+0000 7f4f9ad9d700 1 --2- 172.21.15.5:0/3340447664 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a4a90b450 0x564a4aa92ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:50.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.057+0000 7f4f9ad9d700 1 -- 172.21.15.5:0/3340447664 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a4aaafe20 con 0x564a4aa913b0 2024-05-09T00:46:50.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.058+0000 7f4f9ad9d700 1 --2- 172.21.15.5:0/3340447664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a4aa913b0 0x564a4aaaf8e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f4f94004660 tx=0x7f4f94004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cd0780241acb9e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:50.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.058+0000 7f4f81ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:46:50.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.058+0000 7f4f81ffb700 1 -- 172.21.15.5:0/3340447664 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f94004bb0 con 0x564a4aa913b0 2024-05-09T00:46:50.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.058+0000 7f4f81ffb700 1 -- 172.21.15.5:0/3340447664 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f4f94004d50 con 0x564a4aa913b0 2024-05-09T00:46:50.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.058+0000 7f4faf65fb80 1 -- 172.21.15.5:0/3340447664 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f8c0b1470 con 0x564a4aa913b0 2024-05-09T00:46:50.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.058+0000 7f4f81ffb700 1 -- 172.21.15.5:0/3340447664 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4f940077d0 con 0x564a4aa913b0 2024-05-09T00:46:50.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.058+0000 7f4faf65fb80 10 client.15117 fetch_fsmap learned FSMap version 28 2024-05-09T00:46:50.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.058+0000 7f4faf65fb80 1 -- 172.21.15.5:0/3340447664 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564a4aab10e0 con 0x564a4aa913b0 2024-05-09T00:46:50.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.058+0000 7f4f81ffb700 1 -- 172.21.15.5:0/3340447664 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f8c0b1470 con 0x564a4aa913b0 2024-05-09T00:46:50.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.058+0000 7f4f81ffb700 1 -- 172.21.15.5:0/3340447664 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f4f9402a070 con 0x564a4aa913b0 2024-05-09T00:46:50.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.058+0000 7f4faf65fb80 10 client.15117 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T00:46:50.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.058+0000 7f4faf65fb80 10 client.15117 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-09T00:46:50.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.058+0000 7f4faf65fb80 20 client.15117 populate_metadata read hostname 'smithi005' 2024-05-09T00:46:50.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.058+0000 7f4faf65fb80 1 --2- 172.21.15.5:0/3340447664 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x564a4aaa6000 0x564a4ab14890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:50.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.059+0000 7f4faf65fb80 4 client.15117 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T00:46:50.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.059+0000 7f4faf65fb80 1 -- 172.21.15.5:0/3340447664 --> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] -- command(tid 0: {"prefix": "status"}) -- 0x564a4a9b2a20 con 0x564a4aaa6000 2024-05-09T00:46:50.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.059+0000 7f4f9a59c700 1 --2- 172.21.15.5:0/3340447664 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x564a4aaa6000 0x564a4ab14890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:50.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.059+0000 7f4f9a59c700 1 --2- 172.21.15.5:0/3340447664 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x564a4aaa6000 0x564a4ab14890 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bb8bc41cbbc454f3 server_cookie=4551eaa074bf4199 in_seq=0 out_seq=0 2024-05-09T00:46:50.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.059+0000 7f4f81ffb700 10 client.15117 ms_handle_connect on v2:172.21.15.131:6836/1633006186 2024-05-09T00:46:50.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.059+0000 7f4f81ffb700 1 -- 172.21.15.5:0/3340447664 <== mds.4 v2:172.21.15.131:6836/1633006186 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x564a4a9b2a20 con 0x564a4aaa6000 2024-05-09T00:46:50.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.059+0000 7f4f81ffb700 10 client.15117 handle_command_reply: tid=0 2024-05-09T00:46:50.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.059+0000 7f4faf65fb80 1 client.15117 shutdown 2024-05-09T00:46:50.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.059+0000 7f4faf65fb80 1 -- 172.21.15.5:0/3340447664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a4aa913b0 msgr2=0x564a4aaaf8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:50.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.059+0000 7f4faf65fb80 1 --2- 172.21.15.5:0/3340447664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a4aa913b0 0x564a4aaaf8e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f4f94004660 tx=0x7f4f94004690 comp rx=0 tx=0).stop 2024-05-09T00:46:50.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.060+0000 7f4faf65fb80 1 -- 172.21.15.5:0/3340447664 shutdown_connections 2024-05-09T00:46:50.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.060+0000 7f4faf65fb80 1 --2- 172.21.15.5:0/3340447664 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x564a4aaa6000 0x564a4ab14890 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:50.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.060+0000 7f4faf65fb80 1 --2- 172.21.15.5:0/3340447664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a4aa913b0 0x564a4aaaf8e0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:50.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.060+0000 7f4f81ffb700 0 client.15117 ms_handle_reset on v2:172.21.15.131:6836/1633006186 2024-05-09T00:46:50.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.060+0000 7f4faf65fb80 1 --2- 172.21.15.5:0/3340447664 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a4a90b450 0x564a4aa92ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:50.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.060+0000 7f4faf65fb80 1 --2- 172.21.15.5:0/3340447664 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a4a9f5110 0x564a4aa93410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:50.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.060+0000 7f4faf65fb80 1 -- 172.21.15.5:0/3340447664 >> 172.21.15.5:0/3340447664 conn(0x564a4a96b650 msgr2=0x564a4aa90a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:50.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.060+0000 7f4faf65fb80 1 -- 172.21.15.5:0/3340447664 shutdown_connections 2024-05-09T00:46:50.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.060+0000 7f4faf65fb80 1 -- 172.21.15.5:0/3340447664 wait complete. 2024-05-09T00:46:50.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.060+0000 7f4faf65fb80 20 client.15117 trim_cache size 0 max 16384 2024-05-09T00:46:50.068 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-09T00:46:50.068 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "d85814c6-0d9b-11ef-bc97-c7b262605968", 2024-05-09T00:46:50.068 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 4, 2024-05-09T00:46:50.068 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24467, 2024-05-09T00:46:50.068 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-05-09T00:46:50.068 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-05-09T00:46:50.068 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-05-09T00:46:50.068 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 168.11371895400001, 2024-05-09T00:46:50.068 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 28, 2024-05-09T00:46:50.068 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 82, 2024-05-09T00:46:50.068 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 78, 2024-05-09T00:46:50.069 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 211.89955620699999 2024-05-09T00:46:50.069 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-09T00:46:50.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.060+0000 7f4fa0c49700 1 -- 172.21.15.5:0/2105656475 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f4f880b2880 msgr2=0x7f4f880b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:50.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.060+0000 7f4fa0c49700 1 --2- 172.21.15.5:0/2105656475 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f4f880b2880 0x7f4f880b4d90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4f84009c60 tx=0x7f4f84009260 comp rx=0 tx=0).stop 2024-05-09T00:46:50.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.060+0000 7f4fa0c49700 1 -- 172.21.15.5:0/2105656475 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f9c107df0 msgr2=0x7f4f9c19a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:50.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.060+0000 7f4fa0c49700 1 --2- 172.21.15.5:0/2105656475 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f9c107df0 0x7f4f9c19a2f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c00b580 tx=0x7f4f8c003680 comp rx=0 tx=0).stop 2024-05-09T00:46:50.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.060+0000 7f4fa0c49700 1 -- 172.21.15.5:0/2105656475 shutdown_connections 2024-05-09T00:46:50.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.060+0000 7f4fa0c49700 1 --2- 172.21.15.5:0/2105656475 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f4f880b2880 0x7f4f880b4d90 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:50.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.060+0000 7f4fa0c49700 1 --2- 172.21.15.5:0/2105656475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f9c108740 0x7f4f9c19e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:50.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.060+0000 7f4fa0c49700 1 --2- 172.21.15.5:0/2105656475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f9c1034c0 0x7f4f9c199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:50.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.060+0000 7f4fa0c49700 1 --2- 172.21.15.5:0/2105656475 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f9c107df0 0x7f4f9c19a2f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:50.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.060+0000 7f4fa0c49700 1 -- 172.21.15.5:0/2105656475 >> 172.21.15.5:0/2105656475 conn(0x7f4f9c0fedc0 msgr2=0x7f4f9c1096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:50.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.061+0000 7f4fa0c49700 1 -- 172.21.15.5:0/2105656475 shutdown_connections 2024-05-09T00:46:50.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:50.061+0000 7f4fa0c49700 1 -- 172.21.15.5:0/2105656475 wait complete. 2024-05-09T00:46:50.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:46:50 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:46:50.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:46:49 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:46:50.738 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.f status 2024-05-09T00:46:51.027 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:46:51.409 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:46:51 smithi005 ceph-mon[30365]: pgmap v386: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-09T00:46:51.409 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:46:51 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:46:51.409 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:46:51 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:46:51.409 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:46:51 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:46:51.409 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:46:51 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:46:51.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:46:51 smithi131 ceph-mon[34547]: pgmap v386: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-09T00:46:51.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:46:51 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:46:51.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:46:51 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:46:51.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:46:51 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:46:51.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:46:51 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:46:51.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:46:51 smithi157 ceph-mon[34294]: pgmap v386: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-09T00:46:51.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:46:51 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:46:51.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:46:51 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:46:51.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:46:51 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:46:51.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:46:51 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:46:52.522 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:46:52 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:46:52.522 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:46:52 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:46:52.522 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:46:52 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:46:52.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:46:52 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:46:52.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:46:52 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:46:52.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:46:52 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:46:52.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:46:52 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:46:52.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:46:52 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:46:52.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:46:52 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:46:52.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.766+0000 7fe7d58b6700 1 -- 172.21.15.5:0/3527725439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7d0108740 msgr2=0x7fe7d01100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:52.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.766+0000 7fe7d58b6700 1 --2- 172.21.15.5:0/3527725439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7d0108740 0x7fe7d01100a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fe7c800b0d0 tx=0x7fe7c800b3e0 comp rx=0 tx=0).stop 2024-05-09T00:46:52.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.766+0000 7fe7d58b6700 1 -- 172.21.15.5:0/3527725439 shutdown_connections 2024-05-09T00:46:52.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.766+0000 7fe7d58b6700 1 --2- 172.21.15.5:0/3527725439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7d0108740 0x7fe7d01100a0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:52.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.766+0000 7fe7d58b6700 1 --2- 172.21.15.5:0/3527725439 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7d01034c0 0x7fe7d01038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:52.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.766+0000 7fe7d58b6700 1 --2- 172.21.15.5:0/3527725439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7d0107df0 0x7fe7d0108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:52.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.766+0000 7fe7d58b6700 1 -- 172.21.15.5:0/3527725439 >> 172.21.15.5:0/3527725439 conn(0x7fe7d00fedf0 msgr2=0x7fe7d0101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:52.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.766+0000 7fe7d58b6700 1 -- 172.21.15.5:0/3527725439 shutdown_connections 2024-05-09T00:46:52.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.766+0000 7fe7d58b6700 1 -- 172.21.15.5:0/3527725439 wait complete. 2024-05-09T00:46:52.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.767+0000 7fe7d58b6700 1 Processor -- start 2024-05-09T00:46:52.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.767+0000 7fe7d58b6700 1 -- start start 2024-05-09T00:46:52.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.767+0000 7fe7d58b6700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7d01034c0 0x7fe7d019bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:52.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.767+0000 7fe7d58b6700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7d0107df0 0x7fe7d019c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:52.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.767+0000 7fe7d58b6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7d0108740 0x7fe7d019c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:52.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.767+0000 7fe7d58b6700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d0113370 con 0x7fe7d0108740 2024-05-09T00:46:52.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.767+0000 7fe7d58b6700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d01131f0 con 0x7fe7d01034c0 2024-05-09T00:46:52.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.767+0000 7fe7d58b6700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d01134f0 con 0x7fe7d0107df0 2024-05-09T00:46:52.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.767+0000 7fe7cf7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7d0108740 0x7fe7d019c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:52.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.767+0000 7fe7cf7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7d0108740 0x7fe7d019c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33022/0 (socket says 172.21.15.5:33022) 2024-05-09T00:46:52.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.767+0000 7fe7cf7fe700 1 -- 172.21.15.5:0/4142042303 learned_addr learned my addr 172.21.15.5:0/4142042303 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:46:52.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.767+0000 7fe7ce7fc700 1 --2- 172.21.15.5:0/4142042303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7d0107df0 0x7fe7d019c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:52.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.767+0000 7fe7ceffd700 1 --2- 172.21.15.5:0/4142042303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7d01034c0 0x7fe7d019bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:52.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.768+0000 7fe7cf7fe700 1 -- 172.21.15.5:0/4142042303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7d0107df0 msgr2=0x7fe7d019c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:52.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.768+0000 7fe7cf7fe700 1 --2- 172.21.15.5:0/4142042303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7d0107df0 0x7fe7d019c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:52.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.768+0000 7fe7cf7fe700 1 -- 172.21.15.5:0/4142042303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7d01034c0 msgr2=0x7fe7d019bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:52.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.768+0000 7fe7cf7fe700 1 --2- 172.21.15.5:0/4142042303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7d01034c0 0x7fe7d019bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:52.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.768+0000 7fe7cf7fe700 1 -- 172.21.15.5:0/4142042303 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7d01a6ed0 con 0x7fe7d0108740 2024-05-09T00:46:52.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.768+0000 7fe7ce7fc700 1 --2- 172.21.15.5:0/4142042303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7d0107df0 0x7fe7d019c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:46:52.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.768+0000 7fe7ceffd700 1 --2- 172.21.15.5:0/4142042303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7d01034c0 0x7fe7d019bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:46:52.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.768+0000 7fe7cf7fe700 1 --2- 172.21.15.5:0/4142042303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7d0108740 0x7fe7d019c6b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8009fd0 tx=0x7fe7c8007ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:52.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.768+0000 7fe7d48b4700 1 -- 172.21.15.5:0/4142042303 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c801f070 con 0x7fe7d0108740 2024-05-09T00:46:52.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.768+0000 7fe7d48b4700 1 -- 172.21.15.5:0/4142042303 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7c8003c00 con 0x7fe7d0108740 2024-05-09T00:46:52.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.768+0000 7fe7d58b6700 1 -- 172.21.15.5:0/4142042303 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7d01a7160 con 0x7fe7d0108740 2024-05-09T00:46:52.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.768+0000 7fe7d48b4700 1 -- 172.21.15.5:0/4142042303 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c801b8e0 con 0x7fe7d0108740 2024-05-09T00:46:52.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.768+0000 7fe7d58b6700 1 -- 172.21.15.5:0/4142042303 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7d01a75b0 con 0x7fe7d0108740 2024-05-09T00:46:52.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.771+0000 7fe7d48b4700 1 -- 172.21.15.5:0/4142042303 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fe7c8022070 con 0x7fe7d0108740 2024-05-09T00:46:52.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.772+0000 7fe7d48b4700 1 --2- 172.21.15.5:0/4142042303 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe7b80b2880 0x7fe7b80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:52.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.772+0000 7fe7ceffd700 1 --2- 172.21.15.5:0/4142042303 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe7b80b2880 0x7fe7b80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:52.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.772+0000 7fe7e42ccb80 1 Processor -- start 2024-05-09T00:46:52.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.772+0000 7fe7d48b4700 1 -- 172.21.15.5:0/4142042303 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fe7c800f020 con 0x7fe7d0108740 2024-05-09T00:46:52.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.772+0000 7fe7e42ccb80 1 -- start start 2024-05-09T00:46:52.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.773+0000 7fe7ceffd700 1 --2- 172.21.15.5:0/4142042303 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe7b80b2880 0x7fe7b80b4d90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe7c0005950 tx=0x7fe7c0005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:52.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.773+0000 7fe7e42ccb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dbe3f1bcb0 0x55dbe40241f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:52.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.773+0000 7fe7e42ccb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbe3ea1450 0x55dbe4024730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:52.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.773+0000 7fe7e42ccb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dbe401e3b0 0x55dbe401e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:52.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.773+0000 7fe7e42ccb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dbe3f41050 con 0x55dbe3ea1450 2024-05-09T00:46:52.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.773+0000 7fe7e42ccb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55dbe3ee6950 con 0x55dbe3f1bcb0 2024-05-09T00:46:52.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.773+0000 7fe7e42ccb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55dbe3f3d010 con 0x55dbe401e3b0 2024-05-09T00:46:52.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.773+0000 7fe7ce7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbe3ea1450 0x55dbe4024730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:52.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.773+0000 7fe7ce7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbe3ea1450 0x55dbe4024730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33032/0 (socket says 172.21.15.5:33032) 2024-05-09T00:46:52.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.773+0000 7fe7ce7fc700 1 -- 172.21.15.5:0/2258248459 learned_addr learned my addr 172.21.15.5:0/2258248459 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:46:52.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.773+0000 7fe7cf7fe700 1 --2- 172.21.15.5:0/2258248459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dbe401e3b0 0x55dbe401e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:52.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.773+0000 7fe7ceffd700 1 --2- 172.21.15.5:0/2258248459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dbe3f1bcb0 0x55dbe40241f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:52.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.773+0000 7fe7ce7fc700 1 -- 172.21.15.5:0/2258248459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dbe401e3b0 msgr2=0x55dbe401e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:52.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.773+0000 7fe7ce7fc700 1 --2- 172.21.15.5:0/2258248459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dbe401e3b0 0x55dbe401e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:52.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.773+0000 7fe7ce7fc700 1 -- 172.21.15.5:0/2258248459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dbe3f1bcb0 msgr2=0x55dbe40241f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:52.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.773+0000 7fe7ce7fc700 1 --2- 172.21.15.5:0/2258248459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dbe3f1bcb0 0x55dbe40241f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:52.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.773+0000 7fe7ce7fc700 1 -- 172.21.15.5:0/2258248459 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbe3ed74b0 con 0x55dbe3ea1450 2024-05-09T00:46:52.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.773+0000 7fe7ce7fc700 1 --2- 172.21.15.5:0/2258248459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbe3ea1450 0x55dbe4024730 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe7c40046e0 tx=0x7fe7c4008f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a7ba6822df3a883 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:52.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.774+0000 7fe7b5ffb700 1 -- 172.21.15.5:0/2258248459 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c4014070 con 0x55dbe3ea1450 2024-05-09T00:46:52.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.774+0000 7fe7b5ffb700 1 -- 172.21.15.5:0/2258248459 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7c4004ce0 con 0x55dbe3ea1450 2024-05-09T00:46:52.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.774+0000 7fe7b5ffb700 1 -- 172.21.15.5:0/2258248459 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c4010430 con 0x55dbe3ea1450 2024-05-09T00:46:52.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.775+0000 7fe7e42ccb80 1 -- 172.21.15.5:0/2258248459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbe3ea1450 msgr2=0x55dbe4024730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:52.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.775+0000 7fe7e42ccb80 1 --2- 172.21.15.5:0/2258248459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbe3ea1450 0x55dbe4024730 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe7c40046e0 tx=0x7fe7c4008f30 comp rx=0 tx=0).stop 2024-05-09T00:46:52.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.775+0000 7fe7e42ccb80 1 -- 172.21.15.5:0/2258248459 shutdown_connections 2024-05-09T00:46:52.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.775+0000 7fe7e42ccb80 1 --2- 172.21.15.5:0/2258248459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dbe401e3b0 0x55dbe401e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:52.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.775+0000 7fe7e42ccb80 1 --2- 172.21.15.5:0/2258248459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dbe3f1bcb0 0x55dbe40241f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:52.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.775+0000 7fe7e42ccb80 1 --2- 172.21.15.5:0/2258248459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbe3ea1450 0x55dbe4024730 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:52.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.775+0000 7fe7e42ccb80 1 -- 172.21.15.5:0/2258248459 >> 172.21.15.5:0/2258248459 conn(0x55dbe3ea4f20 msgr2=0x55dbe40191d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:52.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.775+0000 7fe7e42ccb80 1 -- 172.21.15.5:0/2258248459 shutdown_connections 2024-05-09T00:46:52.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.775+0000 7fe7e42ccb80 1 -- 172.21.15.5:0/2258248459 wait complete. 2024-05-09T00:46:52.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.776+0000 7fe7e42ccb80 1 -- start start 2024-05-09T00:46:52.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.776+0000 7fe7e42ccb80 1 Processor -- start 2024-05-09T00:46:52.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.776+0000 7fe7e42ccb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dbe3ea1450 0x55dbe4028000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:52.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.776+0000 7fe7e42ccb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dbe3f1bcb0 0x55dbe4028540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:52.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.776+0000 7fe7e42ccb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbe401e3b0 0x55dbe4028a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:52.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.776+0000 7fe7e42ccb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dbe401f110 con 0x55dbe401e3b0 2024-05-09T00:46:52.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.776+0000 7fe7e42ccb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55dbe3edbad0 con 0x55dbe3f1bcb0 2024-05-09T00:46:52.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.776+0000 7fe7e42ccb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55dbe402afd0 con 0x55dbe3ea1450 2024-05-09T00:46:52.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.776+0000 7fe7e42ccb80 10 client.0 _ll_register_callbacks cb 0x55dbe3db2670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:46:52.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.776+0000 7fe7cf7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbe401e3b0 0x55dbe4028a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:52.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.776+0000 7fe7cf7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbe401e3b0 0x55dbe4028a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33034/0 (socket says 172.21.15.5:33034) 2024-05-09T00:46:52.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.776+0000 7fe7cf7fe700 1 -- 172.21.15.5:0/364846808 learned_addr learned my addr 172.21.15.5:0/364846808 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:46:52.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.777+0000 7fe7ceffd700 1 --2- 172.21.15.5:0/364846808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dbe3ea1450 0x55dbe4028000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:52.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.777+0000 7fe7ce7fc700 1 --2- 172.21.15.5:0/364846808 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dbe3f1bcb0 0x55dbe4028540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:52.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.777+0000 7fe7cf7fe700 1 -- 172.21.15.5:0/364846808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dbe3ea1450 msgr2=0x55dbe4028000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:52.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.777+0000 7fe7cf7fe700 1 --2- 172.21.15.5:0/364846808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dbe3ea1450 0x55dbe4028000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:52.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.777+0000 7fe7cf7fe700 1 -- 172.21.15.5:0/364846808 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dbe3f1bcb0 msgr2=0x55dbe4028540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:52.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.777+0000 7fe7cf7fe700 1 --2- 172.21.15.5:0/364846808 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dbe3f1bcb0 0x55dbe4028540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:52.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.777+0000 7fe7cf7fe700 1 -- 172.21.15.5:0/364846808 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbe3dbc440 con 0x55dbe401e3b0 2024-05-09T00:46:52.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.777+0000 7fe7ceffd700 1 --2- 172.21.15.5:0/364846808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dbe3ea1450 0x55dbe4028000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:46:52.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.777+0000 7fe7cf7fe700 1 --2- 172.21.15.5:0/364846808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbe401e3b0 0x55dbe4028a80 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe7c80e10f0 tx=0x7fe7c801af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b8381ac9f60156b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:52.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.777+0000 7fe7aeffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:46:52.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.777+0000 7fe7aeffd700 1 -- 172.21.15.5:0/364846808 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c80b3a70 con 0x55dbe401e3b0 2024-05-09T00:46:52.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.777+0000 7fe7aeffd700 1 -- 172.21.15.5:0/364846808 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fe7c80b37f0 con 0x55dbe401e3b0 2024-05-09T00:46:52.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.777+0000 7fe7e42ccb80 1 -- 172.21.15.5:0/364846808 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7c4004ce0 con 0x55dbe401e3b0 2024-05-09T00:46:52.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.777+0000 7fe7aeffd700 1 -- 172.21.15.5:0/364846808 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe7c80c5200 con 0x55dbe401e3b0 2024-05-09T00:46:52.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.777+0000 7fe7e42ccb80 10 client.15129 fetch_fsmap learned FSMap version 28 2024-05-09T00:46:52.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.777+0000 7fe7e42ccb80 1 -- 172.21.15.5:0/364846808 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dbe4035a70 con 0x55dbe401e3b0 2024-05-09T00:46:52.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.778+0000 7fe7aeffd700 1 -- 172.21.15.5:0/364846808 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7c4004ce0 con 0x55dbe401e3b0 2024-05-09T00:46:52.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.778+0000 7fe7aeffd700 1 -- 172.21.15.5:0/364846808 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fe7c8011070 con 0x55dbe401e3b0 2024-05-09T00:46:52.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.778+0000 7fe7e42ccb80 10 client.15129 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T00:46:52.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.778+0000 7fe7e42ccb80 10 client.15129 resolve_mds: resolved name 'f' to daemon mds.f 2024-05-09T00:46:52.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.778+0000 7fe7e42ccb80 20 client.15129 populate_metadata read hostname 'smithi005' 2024-05-09T00:46:52.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.778+0000 7fe7e42ccb80 1 --2- 172.21.15.5:0/364846808 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x55dbe4038160 0x55dbe40395d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:52.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.778+0000 7fe7e42ccb80 4 client.15129 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:46:52.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.778+0000 7fe7e42ccb80 1 -- 172.21.15.5:0/364846808 --> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dbe3f4ea30 con 0x55dbe4038160 2024-05-09T00:46:52.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.778+0000 7fe7ceffd700 1 --2- 172.21.15.5:0/364846808 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x55dbe4038160 0x55dbe40395d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:52.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.779+0000 7fe7aeffd700 10 client.15129 ms_handle_connect on v2:172.21.15.157:6834/832898273 2024-05-09T00:46:52.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.779+0000 7fe7ceffd700 1 --2- 172.21.15.5:0/364846808 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x55dbe4038160 0x55dbe40395d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cb52b17416b90894 server_cookie=5bebd5d44099b6d in_seq=0 out_seq=0 2024-05-09T00:46:52.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.780+0000 7fe7aeffd700 1 -- 172.21.15.5:0/364846808 <== mds.3 v2:172.21.15.157:6834/832898273 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dbe3f4ea30 con 0x55dbe4038160 2024-05-09T00:46:52.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.780+0000 7fe7aeffd700 10 client.15129 handle_command_reply: tid=0 2024-05-09T00:46:52.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.780+0000 7fe7e42ccb80 1 client.15129 shutdown 2024-05-09T00:46:52.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.780+0000 7fe7e42ccb80 1 -- 172.21.15.5:0/364846808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbe401e3b0 msgr2=0x55dbe4028a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:52.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.780+0000 7fe7e42ccb80 1 --2- 172.21.15.5:0/364846808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbe401e3b0 0x55dbe4028a80 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe7c80e10f0 tx=0x7fe7c801af90 comp rx=0 tx=0).stop 2024-05-09T00:46:52.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.780+0000 7fe7e42ccb80 1 -- 172.21.15.5:0/364846808 shutdown_connections 2024-05-09T00:46:52.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.780+0000 7fe7e42ccb80 1 --2- 172.21.15.5:0/364846808 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x55dbe4038160 0x55dbe40395d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:52.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.780+0000 7fe7e42ccb80 1 --2- 172.21.15.5:0/364846808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbe401e3b0 0x55dbe4028a80 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:52.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.780+0000 7fe7e42ccb80 1 --2- 172.21.15.5:0/364846808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dbe3ea1450 0x55dbe4028000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:52.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.780+0000 7fe7e42ccb80 1 --2- 172.21.15.5:0/364846808 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dbe3f1bcb0 0x55dbe4028540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:52.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.780+0000 7fe7aeffd700 0 client.15129 ms_handle_reset on v2:172.21.15.157:6834/832898273 2024-05-09T00:46:52.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.780+0000 7fe7e42ccb80 1 -- 172.21.15.5:0/364846808 >> 172.21.15.5:0/364846808 conn(0x55dbe3ea4f20 msgr2=0x55dbe4012fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:52.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.780+0000 7fe7e42ccb80 1 -- 172.21.15.5:0/364846808 shutdown_connections 2024-05-09T00:46:52.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.780+0000 7fe7e42ccb80 1 -- 172.21.15.5:0/364846808 wait complete. 2024-05-09T00:46:52.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.781+0000 7fe7e42ccb80 20 client.15129 trim_cache size 0 max 16384 2024-05-09T00:46:52.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.802+0000 7fe7e42ccb80 1 Processor -- start 2024-05-09T00:46:52.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.802+0000 7fe7e42ccb80 1 -- start start 2024-05-09T00:46:52.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.802+0000 7fe7e42ccb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbe3ea1450 0x55dbe4023800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:52.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.803+0000 7fe7e42ccb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dbe3f1bcb0 0x55dbe40288c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:52.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.803+0000 7fe7e42ccb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dbe401e3b0 0x55dbe4012fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:52.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.803+0000 7fe7e42ccb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dbe4037910 con 0x55dbe3ea1450 2024-05-09T00:46:52.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.803+0000 7fe7e42ccb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55dbe4037fe0 con 0x55dbe3f1bcb0 2024-05-09T00:46:52.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.803+0000 7fe7e42ccb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55dbe40386b0 con 0x55dbe401e3b0 2024-05-09T00:46:52.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.803+0000 7fe7ceffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbe3ea1450 0x55dbe4023800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:52.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.803+0000 7fe7ceffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbe3ea1450 0x55dbe4023800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33046/0 (socket says 172.21.15.5:33046) 2024-05-09T00:46:52.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.803+0000 7fe7ceffd700 1 -- 172.21.15.5:0/1238216733 learned_addr learned my addr 172.21.15.5:0/1238216733 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:46:52.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.803+0000 7fe7cf7fe700 1 --2- 172.21.15.5:0/1238216733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dbe401e3b0 0x55dbe4012fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:52.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.803+0000 7fe7ce7fc700 1 --2- 172.21.15.5:0/1238216733 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dbe3f1bcb0 0x55dbe40288c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:52.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.803+0000 7fe7ceffd700 1 -- 172.21.15.5:0/1238216733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dbe401e3b0 msgr2=0x55dbe4012fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:52.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.803+0000 7fe7ceffd700 1 --2- 172.21.15.5:0/1238216733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dbe401e3b0 0x55dbe4012fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:52.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.803+0000 7fe7ceffd700 1 -- 172.21.15.5:0/1238216733 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dbe3f1bcb0 msgr2=0x55dbe40288c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:52.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.803+0000 7fe7ceffd700 1 --2- 172.21.15.5:0/1238216733 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dbe3f1bcb0 0x55dbe40288c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:52.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.803+0000 7fe7ceffd700 1 -- 172.21.15.5:0/1238216733 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbe4015980 con 0x55dbe3ea1450 2024-05-09T00:46:52.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.803+0000 7fe7ce7fc700 1 --2- 172.21.15.5:0/1238216733 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dbe3f1bcb0 0x55dbe40288c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:46:52.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.803+0000 7fe7ceffd700 1 --2- 172.21.15.5:0/1238216733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbe3ea1450 0x55dbe4023800 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x55dbe4027880 tx=0x7fe7c0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9516aabcb1ee761b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:52.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.803+0000 7fe7b57fa700 1 -- 172.21.15.5:0/1238216733 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c0004830 con 0x55dbe3ea1450 2024-05-09T00:46:52.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.803+0000 7fe7b57fa700 1 -- 172.21.15.5:0/1238216733 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7c0005020 con 0x55dbe3ea1450 2024-05-09T00:46:52.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.804+0000 7fe7b57fa700 1 -- 172.21.15.5:0/1238216733 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c0028600 con 0x55dbe3ea1450 2024-05-09T00:46:52.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.804+0000 7fe7e42ccb80 1 -- 172.21.15.5:0/1238216733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbe3ea1450 msgr2=0x55dbe4023800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:52.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.804+0000 7fe7e42ccb80 1 --2- 172.21.15.5:0/1238216733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbe3ea1450 0x55dbe4023800 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x55dbe4027880 tx=0x7fe7c0014000 comp rx=0 tx=0).stop 2024-05-09T00:46:52.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.804+0000 7fe7e42ccb80 1 -- 172.21.15.5:0/1238216733 shutdown_connections 2024-05-09T00:46:52.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.804+0000 7fe7e42ccb80 1 --2- 172.21.15.5:0/1238216733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dbe401e3b0 0x55dbe4012fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:52.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.804+0000 7fe7e42ccb80 1 --2- 172.21.15.5:0/1238216733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbe3ea1450 0x55dbe4023800 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:52.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.804+0000 7fe7e42ccb80 1 --2- 172.21.15.5:0/1238216733 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dbe3f1bcb0 0x55dbe40288c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:52.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.804+0000 7fe7e42ccb80 1 -- 172.21.15.5:0/1238216733 >> 172.21.15.5:0/1238216733 conn(0x55dbe3ea4f20 msgr2=0x55dbe40230f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:52.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.805+0000 7fe7e42ccb80 1 -- 172.21.15.5:0/1238216733 shutdown_connections 2024-05-09T00:46:52.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.805+0000 7fe7e42ccb80 1 -- 172.21.15.5:0/1238216733 wait complete. 2024-05-09T00:46:52.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.805+0000 7fe7e42ccb80 1 -- start start 2024-05-09T00:46:52.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.805+0000 7fe7e42ccb80 1 Processor -- start 2024-05-09T00:46:52.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.805+0000 7fe7e42ccb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dbe3ea1450 0x55dbe4042a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:52.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.805+0000 7fe7e42ccb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dbe3f1bcb0 0x55dbe4042f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:52.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.806+0000 7fe7e42ccb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbe401e3b0 0x55dbe403da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:52.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.806+0000 7fe7e42ccb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dbe4039450 con 0x55dbe401e3b0 2024-05-09T00:46:52.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.806+0000 7fe7e42ccb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55dbe4038d80 con 0x55dbe3ea1450 2024-05-09T00:46:52.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.806+0000 7fe7e42ccb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55dbe4039b20 con 0x55dbe3f1bcb0 2024-05-09T00:46:52.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.806+0000 7fe7e42ccb80 10 client.0 _ll_register_callbacks cb 0x55dbe3db2670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:46:52.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.806+0000 7fe7cf7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbe401e3b0 0x55dbe403da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:52.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.806+0000 7fe7cf7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbe401e3b0 0x55dbe403da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33050/0 (socket says 172.21.15.5:33050) 2024-05-09T00:46:52.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.806+0000 7fe7cf7fe700 1 -- 172.21.15.5:0/1996500970 learned_addr learned my addr 172.21.15.5:0/1996500970 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:46:52.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.806+0000 7fe7ce7fc700 1 --2- 172.21.15.5:0/1996500970 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dbe3f1bcb0 0x55dbe4042f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:52.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.806+0000 7fe7ceffd700 1 --2- 172.21.15.5:0/1996500970 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dbe3ea1450 0x55dbe4042a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:52.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.806+0000 7fe7cf7fe700 1 -- 172.21.15.5:0/1996500970 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dbe3f1bcb0 msgr2=0x55dbe4042f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:52.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.806+0000 7fe7cf7fe700 1 --2- 172.21.15.5:0/1996500970 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dbe3f1bcb0 0x55dbe4042f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:52.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.806+0000 7fe7cf7fe700 1 -- 172.21.15.5:0/1996500970 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dbe3ea1450 msgr2=0x55dbe4042a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:52.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.806+0000 7fe7cf7fe700 1 --2- 172.21.15.5:0/1996500970 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dbe3ea1450 0x55dbe4042a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:52.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.806+0000 7fe7cf7fe700 1 -- 172.21.15.5:0/1996500970 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbe403df40 con 0x55dbe401e3b0 2024-05-09T00:46:52.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.806+0000 7fe7cf7fe700 1 --2- 172.21.15.5:0/1996500970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbe401e3b0 0x55dbe403da00 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe7c80a7400 tx=0x7fe7c8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a72d804b1bd02f5a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:52.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.806+0000 7fe7aeffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:46:52.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.806+0000 7fe7aeffd700 1 -- 172.21.15.5:0/1996500970 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c80ca060 con 0x55dbe401e3b0 2024-05-09T00:46:52.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.806+0000 7fe7aeffd700 1 -- 172.21.15.5:0/1996500970 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fe7c80e33e0 con 0x55dbe401e3b0 2024-05-09T00:46:52.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.806+0000 7fe7e42ccb80 1 -- 172.21.15.5:0/1996500970 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7c0005020 con 0x55dbe401e3b0 2024-05-09T00:46:52.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.807+0000 7fe7aeffd700 1 -- 172.21.15.5:0/1996500970 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe7c80aa970 con 0x55dbe401e3b0 2024-05-09T00:46:52.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.807+0000 7fe7e42ccb80 10 client.15135 fetch_fsmap learned FSMap version 28 2024-05-09T00:46:52.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.807+0000 7fe7e42ccb80 1 -- 172.21.15.5:0/1996500970 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dbe403df40 con 0x55dbe401e3b0 2024-05-09T00:46:52.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.807+0000 7fe7aeffd700 1 -- 172.21.15.5:0/1996500970 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7c0005020 con 0x55dbe401e3b0 2024-05-09T00:46:52.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.807+0000 7fe7aeffd700 1 -- 172.21.15.5:0/1996500970 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fe7c8011040 con 0x55dbe401e3b0 2024-05-09T00:46:52.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.807+0000 7fe7e42ccb80 10 client.15135 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T00:46:52.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.807+0000 7fe7e42ccb80 10 client.15135 resolve_mds: resolved name 'f' to daemon mds.f 2024-05-09T00:46:52.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.807+0000 7fe7e42ccb80 20 client.15135 populate_metadata read hostname 'smithi005' 2024-05-09T00:46:52.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.807+0000 7fe7e42ccb80 1 --2- 172.21.15.5:0/1996500970 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x55dbe40a5cc0 0x55dbe40a7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:52.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.807+0000 7fe7e42ccb80 4 client.15135 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T00:46:52.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.807+0000 7fe7e42ccb80 1 -- 172.21.15.5:0/1996500970 --> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] -- command(tid 0: {"prefix": "status"}) -- 0x55dbe3f48ad0 con 0x55dbe40a5cc0 2024-05-09T00:46:52.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.807+0000 7fe7ceffd700 1 --2- 172.21.15.5:0/1996500970 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x55dbe40a5cc0 0x55dbe40a7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:52.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.808+0000 7fe7aeffd700 10 client.15135 ms_handle_connect on v2:172.21.15.157:6834/832898273 2024-05-09T00:46:52.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.808+0000 7fe7ceffd700 1 --2- 172.21.15.5:0/1996500970 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x55dbe40a5cc0 0x55dbe40a7130 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4b3282f6959f527c server_cookie=381722bb0d7677e3 in_seq=0 out_seq=0 2024-05-09T00:46:52.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.808+0000 7fe7aeffd700 1 -- 172.21.15.5:0/1996500970 <== mds.3 v2:172.21.15.157:6834/832898273 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55dbe3f48ad0 con 0x55dbe40a5cc0 2024-05-09T00:46:52.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.808+0000 7fe7aeffd700 10 client.15135 handle_command_reply: tid=0 2024-05-09T00:46:52.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.808+0000 7fe7e42ccb80 1 client.15135 shutdown 2024-05-09T00:46:52.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.808+0000 7fe7e42ccb80 1 -- 172.21.15.5:0/1996500970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbe401e3b0 msgr2=0x55dbe403da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:52.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.808+0000 7fe7e42ccb80 1 --2- 172.21.15.5:0/1996500970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbe401e3b0 0x55dbe403da00 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe7c80a7400 tx=0x7fe7c8005f90 comp rx=0 tx=0).stop 2024-05-09T00:46:52.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.808+0000 7fe7e42ccb80 1 -- 172.21.15.5:0/1996500970 shutdown_connections 2024-05-09T00:46:52.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.808+0000 7fe7e42ccb80 1 --2- 172.21.15.5:0/1996500970 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x55dbe40a5cc0 0x55dbe40a7130 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:52.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.808+0000 7fe7e42ccb80 1 --2- 172.21.15.5:0/1996500970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbe401e3b0 0x55dbe403da00 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:52.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.808+0000 7fe7e42ccb80 1 --2- 172.21.15.5:0/1996500970 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dbe3ea1450 0x55dbe4042a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:52.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.808+0000 7fe7e42ccb80 1 --2- 172.21.15.5:0/1996500970 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dbe3f1bcb0 0x55dbe4042f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:52.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.808+0000 7fe7e42ccb80 1 -- 172.21.15.5:0/1996500970 >> 172.21.15.5:0/1996500970 conn(0x55dbe3ea4f20 msgr2=0x55dbe4026530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:52.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.808+0000 7fe7aeffd700 0 client.15135 ms_handle_reset on v2:172.21.15.157:6834/832898273 2024-05-09T00:46:52.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.809+0000 7fe7e42ccb80 1 -- 172.21.15.5:0/1996500970 shutdown_connections 2024-05-09T00:46:52.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.809+0000 7fe7e42ccb80 1 -- 172.21.15.5:0/1996500970 wait complete. 2024-05-09T00:46:52.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.809+0000 7fe7e42ccb80 20 client.15135 trim_cache size 0 max 16384 2024-05-09T00:46:52.815 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-09T00:46:52.816 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "d85814c6-0d9b-11ef-bc97-c7b262605968", 2024-05-09T00:46:52.816 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 3, 2024-05-09T00:46:52.816 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24493, 2024-05-09T00:46:52.816 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-05-09T00:46:52.816 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-05-09T00:46:52.816 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-05-09T00:46:52.816 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 171.87160289400001, 2024-05-09T00:46:52.816 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 28, 2024-05-09T00:46:52.816 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 82, 2024-05-09T00:46:52.816 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 78, 2024-05-09T00:46:52.816 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 220.38206979399999 2024-05-09T00:46:52.816 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-09T00:46:52.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.809+0000 7fe7d58b6700 1 -- 172.21.15.5:0/4142042303 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe7b80b2880 msgr2=0x7fe7b80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:52.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.809+0000 7fe7d58b6700 1 --2- 172.21.15.5:0/4142042303 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe7b80b2880 0x7fe7b80b4d90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe7c0005950 tx=0x7fe7c0005f50 comp rx=0 tx=0).stop 2024-05-09T00:46:52.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.809+0000 7fe7d58b6700 1 -- 172.21.15.5:0/4142042303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7d0108740 msgr2=0x7fe7d019c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:52.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.809+0000 7fe7d58b6700 1 --2- 172.21.15.5:0/4142042303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7d0108740 0x7fe7d019c6b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8009fd0 tx=0x7fe7c8007ad0 comp rx=0 tx=0).stop 2024-05-09T00:46:52.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.809+0000 7fe7d58b6700 1 -- 172.21.15.5:0/4142042303 shutdown_connections 2024-05-09T00:46:52.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.809+0000 7fe7d58b6700 1 --2- 172.21.15.5:0/4142042303 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe7b80b2880 0x7fe7b80b4d90 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:52.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.810+0000 7fe7d58b6700 1 --2- 172.21.15.5:0/4142042303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7d0108740 0x7fe7d019c6b0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:52.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.810+0000 7fe7d58b6700 1 --2- 172.21.15.5:0/4142042303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7d01034c0 0x7fe7d019bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:52.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.810+0000 7fe7d58b6700 1 --2- 172.21.15.5:0/4142042303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7d0107df0 0x7fe7d019c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:52.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.810+0000 7fe7d58b6700 1 -- 172.21.15.5:0/4142042303 >> 172.21.15.5:0/4142042303 conn(0x7fe7d00fedf0 msgr2=0x7fe7d01096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:52.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.810+0000 7fe7d58b6700 1 -- 172.21.15.5:0/4142042303 shutdown_connections 2024-05-09T00:46:52.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:52.810+0000 7fe7d58b6700 1 -- 172.21.15.5:0/4142042303 wait complete. 2024-05-09T00:46:53.485 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:46:53 smithi005 ceph-mon[30365]: pgmap v387: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T00:46:53.515 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.b status 2024-05-09T00:46:53.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:46:53 smithi131 ceph-mon[34547]: pgmap v387: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T00:46:53.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:46:53 smithi157 ceph-mon[34294]: pgmap v387: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T00:46:53.770 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:46:55.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.467+0000 7fcbb81b5700 1 -- 172.21.15.5:0/2668210746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbb0103310 msgr2=0x7fcbb010f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:55.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.467+0000 7fcbb81b5700 1 --2- 172.21.15.5:0/2668210746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbb0103310 0x7fcbb010f890 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fcba800c210 tx=0x7fcba800c520 comp rx=0 tx=0).stop 2024-05-09T00:46:55.469 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:46:55 smithi005 ceph-mon[30365]: pgmap v388: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-09T00:46:55.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.467+0000 7fcbb81b5700 1 -- 172.21.15.5:0/2668210746 shutdown_connections 2024-05-09T00:46:55.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.467+0000 7fcbb81b5700 1 --2- 172.21.15.5:0/2668210746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbb0103310 0x7fcbb010f890 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:55.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.467+0000 7fcbb81b5700 1 --2- 172.21.15.5:0/2668210746 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbb01089f0 0x7fcbb0108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:55.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.467+0000 7fcbb81b5700 1 --2- 172.21.15.5:0/2668210746 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbb01029c0 0x7fcbb0102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:55.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.467+0000 7fcbb81b5700 1 -- 172.21.15.5:0/2668210746 >> 172.21.15.5:0/2668210746 conn(0x7fcbb00fe520 msgr2=0x7fcbb0100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:55.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.467+0000 7fcbb81b5700 1 -- 172.21.15.5:0/2668210746 shutdown_connections 2024-05-09T00:46:55.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.467+0000 7fcbb81b5700 1 -- 172.21.15.5:0/2668210746 wait complete. 2024-05-09T00:46:55.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.468+0000 7fcbb81b5700 1 Processor -- start 2024-05-09T00:46:55.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.468+0000 7fcbb81b5700 1 -- start start 2024-05-09T00:46:55.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.469+0000 7fcbb81b5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbb01029c0 0x7fcbb019ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:55.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.469+0000 7fcbb81b5700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbb0103310 0x7fcbb019b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:55.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.469+0000 7fcbb81b5700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbb01089f0 0x7fcbb019f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:55.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.469+0000 7fcbb81b5700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb01122e0 con 0x7fcbb01029c0 2024-05-09T00:46:55.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.469+0000 7fcbb81b5700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb0112160 con 0x7fcbb01089f0 2024-05-09T00:46:55.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.469+0000 7fcbb81b5700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb0112460 con 0x7fcbb0103310 2024-05-09T00:46:55.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.469+0000 7fcbb5f51700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbb01029c0 0x7fcbb019ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:55.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.469+0000 7fcbb5f51700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbb01029c0 0x7fcbb019ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54992/0 (socket says 172.21.15.5:54992) 2024-05-09T00:46:55.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.469+0000 7fcbb5f51700 1 -- 172.21.15.5:0/2418802551 learned_addr learned my addr 172.21.15.5:0/2418802551 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:46:55.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.469+0000 7fcbb6752700 1 --2- 172.21.15.5:0/2418802551 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbb01089f0 0x7fcbb019f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:55.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.469+0000 7fcbb5750700 1 --2- 172.21.15.5:0/2418802551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbb0103310 0x7fcbb019b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:55.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.469+0000 7fcbb5f51700 1 -- 172.21.15.5:0/2418802551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbb0103310 msgr2=0x7fcbb019b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:55.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.469+0000 7fcbb5f51700 1 --2- 172.21.15.5:0/2418802551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbb0103310 0x7fcbb019b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:55.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.469+0000 7fcbb5f51700 1 -- 172.21.15.5:0/2418802551 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbb01089f0 msgr2=0x7fcbb019f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:55.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.469+0000 7fcbb5f51700 1 --2- 172.21.15.5:0/2418802551 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbb01089f0 0x7fcbb019f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:55.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.469+0000 7fcbb5f51700 1 -- 172.21.15.5:0/2418802551 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbb019fd10 con 0x7fcbb01029c0 2024-05-09T00:46:55.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.469+0000 7fcbb5750700 1 --2- 172.21.15.5:0/2418802551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbb0103310 0x7fcbb019b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:46:55.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.469+0000 7fcbb5f51700 1 --2- 172.21.15.5:0/2418802551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbb01029c0 0x7fcbb019ae40 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fcbac00d980 tx=0x7fcbac00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:55.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.470+0000 7fcba2ffd700 1 -- 172.21.15.5:0/2418802551 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbac017070 con 0x7fcbb01029c0 2024-05-09T00:46:55.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.470+0000 7fcba2ffd700 1 -- 172.21.15.5:0/2418802551 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbac009de0 con 0x7fcbb01029c0 2024-05-09T00:46:55.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.470+0000 7fcbb81b5700 1 -- 172.21.15.5:0/2418802551 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbb01a0000 con 0x7fcbb01029c0 2024-05-09T00:46:55.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.470+0000 7fcba2ffd700 1 -- 172.21.15.5:0/2418802551 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbac00f3c0 con 0x7fcbb01029c0 2024-05-09T00:46:55.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.470+0000 7fcbb81b5700 1 -- 172.21.15.5:0/2418802551 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbb01a7920 con 0x7fcbb01029c0 2024-05-09T00:46:55.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.472+0000 7fcba2ffd700 1 -- 172.21.15.5:0/2418802551 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fcbac009930 con 0x7fcbb01029c0 2024-05-09T00:46:55.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.473+0000 7fcba2ffd700 1 --2- 172.21.15.5:0/2418802551 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fcb9c0b2880 0x7fcb9c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:55.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.473+0000 7fcbb5750700 1 --2- 172.21.15.5:0/2418802551 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fcb9c0b2880 0x7fcb9c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:55.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.473+0000 7fcba2ffd700 1 -- 172.21.15.5:0/2418802551 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fcbac0e1cd0 con 0x7fcbb01029c0 2024-05-09T00:46:55.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.474+0000 7fcbc6bcbb80 1 Processor -- start 2024-05-09T00:46:55.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.474+0000 7fcbb5750700 1 --2- 172.21.15.5:0/2418802551 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fcb9c0b2880 0x7fcb9c0b4d90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fcba4005fd0 tx=0x7fcba4005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:55.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.474+0000 7fcbc6bcbb80 1 -- start start 2024-05-09T00:46:55.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.474+0000 7fcbc6bcbb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db4e797110 0x55db4e82ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:55.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.474+0000 7fcbc6bcbb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db4e6ad450 0x55db4e82f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:55.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.474+0000 7fcbc6bcbb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db4e8333b0 0x55db4e8337c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:55.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.474+0000 7fcbc6bcbb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55db4e74cfa0 con 0x55db4e8333b0 2024-05-09T00:46:55.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.474+0000 7fcbc6bcbb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55db4e6eb7a0 con 0x55db4e6ad450 2024-05-09T00:46:55.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.474+0000 7fcbc6bcbb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55db4e748f60 con 0x55db4e797110 2024-05-09T00:46:55.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.474+0000 7fcbb6752700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db4e8333b0 0x55db4e8337c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:55.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.474+0000 7fcbb6752700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db4e8333b0 0x55db4e8337c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54994/0 (socket says 172.21.15.5:54994) 2024-05-09T00:46:55.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.474+0000 7fcbb6752700 1 -- 172.21.15.5:0/3610555041 learned_addr learned my addr 172.21.15.5:0/3610555041 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:46:55.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.475+0000 7fcbb5750700 1 --2- 172.21.15.5:0/3610555041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db4e6ad450 0x55db4e82f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:55.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.475+0000 7fcbb5f51700 1 --2- 172.21.15.5:0/3610555041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db4e797110 0x55db4e82ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:55.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.475+0000 7fcbb6752700 1 -- 172.21.15.5:0/3610555041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db4e797110 msgr2=0x55db4e82ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:55.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.475+0000 7fcbb6752700 1 --2- 172.21.15.5:0/3610555041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db4e797110 0x55db4e82ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:55.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.475+0000 7fcbb6752700 1 -- 172.21.15.5:0/3610555041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db4e6ad450 msgr2=0x55db4e82f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:55.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.475+0000 7fcbb6752700 1 --2- 172.21.15.5:0/3610555041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db4e6ad450 0x55db4e82f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:55.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.475+0000 7fcbb6752700 1 -- 172.21.15.5:0/3610555041 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db4e6e3320 con 0x55db4e8333b0 2024-05-09T00:46:55.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.475+0000 7fcbb5f51700 1 --2- 172.21.15.5:0/3610555041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db4e797110 0x55db4e82ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:46:55.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.475+0000 7fcbb6752700 1 --2- 172.21.15.5:0/3610555041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db4e8333b0 0x55db4e8337c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fcba800cef0 tx=0x7fcba800ae90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=739421a6b60bee13 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:55.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.475+0000 7fcb9bfff700 1 -- 172.21.15.5:0/3610555041 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba8008420 con 0x55db4e8333b0 2024-05-09T00:46:55.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.475+0000 7fcb9bfff700 1 -- 172.21.15.5:0/3610555041 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcba800a0d0 con 0x55db4e8333b0 2024-05-09T00:46:55.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.475+0000 7fcb9bfff700 1 -- 172.21.15.5:0/3610555041 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba8008ad0 con 0x55db4e8333b0 2024-05-09T00:46:55.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.476+0000 7fcbc6bcbb80 1 -- 172.21.15.5:0/3610555041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db4e8333b0 msgr2=0x55db4e8337c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:55.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.476+0000 7fcbc6bcbb80 1 --2- 172.21.15.5:0/3610555041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db4e8333b0 0x55db4e8337c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fcba800cef0 tx=0x7fcba800ae90 comp rx=0 tx=0).stop 2024-05-09T00:46:55.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.476+0000 7fcbc6bcbb80 1 -- 172.21.15.5:0/3610555041 shutdown_connections 2024-05-09T00:46:55.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.476+0000 7fcbc6bcbb80 1 --2- 172.21.15.5:0/3610555041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db4e8333b0 0x55db4e8337c0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:55.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.476+0000 7fcbc6bcbb80 1 --2- 172.21.15.5:0/3610555041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db4e797110 0x55db4e82ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:55.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.476+0000 7fcbc6bcbb80 1 --2- 172.21.15.5:0/3610555041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db4e6ad450 0x55db4e82f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:55.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.476+0000 7fcbc6bcbb80 1 -- 172.21.15.5:0/3610555041 >> 172.21.15.5:0/3610555041 conn(0x55db4e70d650 msgr2=0x55db4e8253b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:55.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.476+0000 7fcbc6bcbb80 1 -- 172.21.15.5:0/3610555041 shutdown_connections 2024-05-09T00:46:55.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.476+0000 7fcbc6bcbb80 1 -- 172.21.15.5:0/3610555041 wait complete. 2024-05-09T00:46:55.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.477+0000 7fcbc6bcbb80 1 -- start start 2024-05-09T00:46:55.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.477+0000 7fcbc6bcbb80 1 Processor -- start 2024-05-09T00:46:55.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.477+0000 7fcbc6bcbb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db4e6ad450 0x55db4e8385b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:55.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.477+0000 7fcbc6bcbb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db4e797110 0x55db4e838af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:55.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.477+0000 7fcbc6bcbb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db4e8333b0 0x55db4e839030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:55.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.477+0000 7fcbc6bcbb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55db4e5c8b50 con 0x55db4e8333b0 2024-05-09T00:46:55.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.477+0000 7fcbc6bcbb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55db4e707100 con 0x55db4e6ad450 2024-05-09T00:46:55.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.477+0000 7fcbc6bcbb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55db4e834100 con 0x55db4e797110 2024-05-09T00:46:55.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.477+0000 7fcbc6bcbb80 10 client.0 _ll_register_callbacks cb 0x55db4e5be670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:46:55.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.477+0000 7fcbb6752700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db4e8333b0 0x55db4e839030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:55.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.477+0000 7fcbb6752700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db4e8333b0 0x55db4e839030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55000/0 (socket says 172.21.15.5:55000) 2024-05-09T00:46:55.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.477+0000 7fcbb6752700 1 -- 172.21.15.5:0/2563828286 learned_addr learned my addr 172.21.15.5:0/2563828286 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:46:55.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.478+0000 7fcbb5f51700 1 --2- 172.21.15.5:0/2563828286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db4e6ad450 0x55db4e8385b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:55.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.478+0000 7fcbb6752700 1 -- 172.21.15.5:0/2563828286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db4e797110 msgr2=0x55db4e838af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:55.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.478+0000 7fcbb5750700 1 --2- 172.21.15.5:0/2563828286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db4e797110 0x55db4e838af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:55.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.478+0000 7fcbb6752700 1 --2- 172.21.15.5:0/2563828286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db4e797110 0x55db4e838af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:55.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.478+0000 7fcbb6752700 1 -- 172.21.15.5:0/2563828286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db4e6ad450 msgr2=0x55db4e8385b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:55.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.478+0000 7fcbb6752700 1 --2- 172.21.15.5:0/2563828286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db4e6ad450 0x55db4e8385b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:55.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.478+0000 7fcbb6752700 1 -- 172.21.15.5:0/2563828286 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db4e5c8440 con 0x55db4e8333b0 2024-05-09T00:46:55.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.478+0000 7fcbb5f51700 1 --2- 172.21.15.5:0/2563828286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db4e6ad450 0x55db4e8385b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:46:55.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.478+0000 7fcbb6752700 1 --2- 172.21.15.5:0/2563828286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db4e8333b0 0x55db4e839030 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fcba800e010 tx=0x7fcba800a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dac3b46b61ed79d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:55.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.478+0000 7fcb997fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:46:55.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.478+0000 7fcb997fa700 1 -- 172.21.15.5:0/2563828286 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba80088d0 con 0x55db4e8333b0 2024-05-09T00:46:55.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.478+0000 7fcb997fa700 1 -- 172.21.15.5:0/2563828286 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fcba8008a70 con 0x55db4e8333b0 2024-05-09T00:46:55.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.478+0000 7fcbc6bcbb80 1 -- 172.21.15.5:0/2563828286 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcba800a0d0 con 0x55db4e8333b0 2024-05-09T00:46:55.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.478+0000 7fcb997fa700 1 -- 172.21.15.5:0/2563828286 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcba8036030 con 0x55db4e8333b0 2024-05-09T00:46:55.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.478+0000 7fcbc6bcbb80 10 client.15147 fetch_fsmap learned FSMap version 28 2024-05-09T00:46:55.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.478+0000 7fcbc6bcbb80 1 -- 172.21.15.5:0/2563828286 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55db4e845e00 con 0x55db4e8333b0 2024-05-09T00:46:55.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.479+0000 7fcb997fa700 1 -- 172.21.15.5:0/2563828286 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcba800a0d0 con 0x55db4e8333b0 2024-05-09T00:46:55.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.479+0000 7fcb997fa700 1 -- 172.21.15.5:0/2563828286 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fcba8012070 con 0x55db4e8333b0 2024-05-09T00:46:55.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.479+0000 7fcbc6bcbb80 10 client.15147 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T00:46:55.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.479+0000 7fcbc6bcbb80 10 client.15147 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-09T00:46:55.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.479+0000 7fcbc6bcbb80 20 client.15147 populate_metadata read hostname 'smithi005' 2024-05-09T00:46:55.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.479+0000 7fcbc6bcbb80 1 --2- 172.21.15.5:0/2563828286 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x55db4e848020 0x55db4e849490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:55.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.479+0000 7fcbc6bcbb80 4 client.15147 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:46:55.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.479+0000 7fcbc6bcbb80 1 -- 172.21.15.5:0/2563828286 --> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db4e75a980 con 0x55db4e848020 2024-05-09T00:46:55.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.479+0000 7fcbb5f51700 1 --2- 172.21.15.5:0/2563828286 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x55db4e848020 0x55db4e849490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:55.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.480+0000 7fcbb5f51700 1 --2- 172.21.15.5:0/2563828286 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x55db4e848020 0x55db4e849490 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=10df0d38843e0eca server_cookie=fa9c031246af2e0c in_seq=0 out_seq=0 2024-05-09T00:46:55.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.480+0000 7fcb997fa700 10 client.15147 ms_handle_connect on v2:172.21.15.131:6838/2748669631 2024-05-09T00:46:55.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.480+0000 7fcb997fa700 1 -- 172.21.15.5:0/2563828286 <== mds.2 v2:172.21.15.131:6838/2748669631 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55db4e75a980 con 0x55db4e848020 2024-05-09T00:46:55.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.480+0000 7fcb997fa700 10 client.15147 handle_command_reply: tid=0 2024-05-09T00:46:55.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.481+0000 7fcbc6bcbb80 1 client.15147 shutdown 2024-05-09T00:46:55.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.481+0000 7fcbc6bcbb80 1 -- 172.21.15.5:0/2563828286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db4e8333b0 msgr2=0x55db4e839030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:55.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.481+0000 7fcbc6bcbb80 1 --2- 172.21.15.5:0/2563828286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db4e8333b0 0x55db4e839030 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fcba800e010 tx=0x7fcba800a6a0 comp rx=0 tx=0).stop 2024-05-09T00:46:55.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.481+0000 7fcbc6bcbb80 1 -- 172.21.15.5:0/2563828286 shutdown_connections 2024-05-09T00:46:55.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.481+0000 7fcbc6bcbb80 1 --2- 172.21.15.5:0/2563828286 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x55db4e848020 0x55db4e849490 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:55.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.481+0000 7fcbc6bcbb80 1 --2- 172.21.15.5:0/2563828286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db4e8333b0 0x55db4e839030 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:55.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.481+0000 7fcbc6bcbb80 1 --2- 172.21.15.5:0/2563828286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db4e6ad450 0x55db4e8385b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:55.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.481+0000 7fcbc6bcbb80 1 --2- 172.21.15.5:0/2563828286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db4e797110 0x55db4e838af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:55.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.481+0000 7fcb997fa700 0 client.15147 ms_handle_reset on v2:172.21.15.131:6838/2748669631 2024-05-09T00:46:55.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.481+0000 7fcbc6bcbb80 1 -- 172.21.15.5:0/2563828286 >> 172.21.15.5:0/2563828286 conn(0x55db4e70d650 msgr2=0x55db4e825170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:55.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.481+0000 7fcbc6bcbb80 1 -- 172.21.15.5:0/2563828286 shutdown_connections 2024-05-09T00:46:55.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.481+0000 7fcbc6bcbb80 1 -- 172.21.15.5:0/2563828286 wait complete. 2024-05-09T00:46:55.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.481+0000 7fcbc6bcbb80 20 client.15147 trim_cache size 0 max 16384 2024-05-09T00:46:55.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.502+0000 7fcbc6bcbb80 1 Processor -- start 2024-05-09T00:46:55.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.502+0000 7fcbc6bcbb80 1 -- start start 2024-05-09T00:46:55.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.502+0000 7fcbc6bcbb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db4e6ad450 0x55db4e825e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:55.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.502+0000 7fcbc6bcbb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db4e797110 0x55db4e824890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:55.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.502+0000 7fcbc6bcbb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db4e8333b0 0x55db4e8325a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:55.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.502+0000 7fcbc6bcbb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55db4e8477d0 con 0x55db4e8333b0 2024-05-09T00:46:55.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.502+0000 7fcbc6bcbb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55db4e847ea0 con 0x55db4e6ad450 2024-05-09T00:46:55.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.502+0000 7fcbc6bcbb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55db4e848570 con 0x55db4e797110 2024-05-09T00:46:55.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.502+0000 7fcbb6752700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db4e8333b0 0x55db4e8325a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:55.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.502+0000 7fcbb6752700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db4e8333b0 0x55db4e8325a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55010/0 (socket says 172.21.15.5:55010) 2024-05-09T00:46:55.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.502+0000 7fcbb6752700 1 -- 172.21.15.5:0/1683866102 learned_addr learned my addr 172.21.15.5:0/1683866102 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:46:55.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.502+0000 7fcbb5750700 1 --2- 172.21.15.5:0/1683866102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db4e797110 0x55db4e824890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:55.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.503+0000 7fcbb5f51700 1 --2- 172.21.15.5:0/1683866102 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db4e6ad450 0x55db4e825e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:55.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.503+0000 7fcbb6752700 1 -- 172.21.15.5:0/1683866102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db4e797110 msgr2=0x55db4e824890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:55.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.503+0000 7fcbb6752700 1 --2- 172.21.15.5:0/1683866102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db4e797110 0x55db4e824890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:55.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.503+0000 7fcbb6752700 1 -- 172.21.15.5:0/1683866102 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db4e6ad450 msgr2=0x55db4e825e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:55.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.503+0000 7fcbb6752700 1 --2- 172.21.15.5:0/1683866102 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db4e6ad450 0x55db4e825e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:55.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.503+0000 7fcbb6752700 1 -- 172.21.15.5:0/1683866102 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db4e836b40 con 0x55db4e8333b0 2024-05-09T00:46:55.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.503+0000 7fcbb5750700 1 --2- 172.21.15.5:0/1683866102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db4e797110 0x55db4e824890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:46:55.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.503+0000 7fcbb6752700 1 --2- 172.21.15.5:0/1683866102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db4e8333b0 0x55db4e8325a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fcba800ab30 tx=0x7fcba800c1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df12a4b105ade51e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:55.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.503+0000 7fcb9b7fe700 1 -- 172.21.15.5:0/1683866102 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba803f030 con 0x55db4e8333b0 2024-05-09T00:46:55.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.503+0000 7fcb9b7fe700 1 -- 172.21.15.5:0/1683866102 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcba8003ab0 con 0x55db4e8333b0 2024-05-09T00:46:55.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.503+0000 7fcb9b7fe700 1 -- 172.21.15.5:0/1683866102 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba8005040 con 0x55db4e8333b0 2024-05-09T00:46:55.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.504+0000 7fcbc6bcbb80 1 -- 172.21.15.5:0/1683866102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db4e8333b0 msgr2=0x55db4e8325a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:55.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.504+0000 7fcbc6bcbb80 1 --2- 172.21.15.5:0/1683866102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db4e8333b0 0x55db4e8325a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fcba800ab30 tx=0x7fcba800c1d0 comp rx=0 tx=0).stop 2024-05-09T00:46:55.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.504+0000 7fcbc6bcbb80 1 -- 172.21.15.5:0/1683866102 shutdown_connections 2024-05-09T00:46:55.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.504+0000 7fcbc6bcbb80 1 --2- 172.21.15.5:0/1683866102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db4e8333b0 0x55db4e8325a0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:55.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.504+0000 7fcbc6bcbb80 1 --2- 172.21.15.5:0/1683866102 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db4e6ad450 0x55db4e825e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:55.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.504+0000 7fcbc6bcbb80 1 --2- 172.21.15.5:0/1683866102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db4e797110 0x55db4e824890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:55.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.504+0000 7fcbc6bcbb80 1 -- 172.21.15.5:0/1683866102 >> 172.21.15.5:0/1683866102 conn(0x55db4e70d650 msgr2=0x55db4e835fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:55.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.504+0000 7fcbc6bcbb80 1 -- 172.21.15.5:0/1683866102 shutdown_connections 2024-05-09T00:46:55.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.504+0000 7fcbc6bcbb80 1 -- 172.21.15.5:0/1683866102 wait complete. 2024-05-09T00:46:55.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.504+0000 7fcbc6bcbb80 1 -- start start 2024-05-09T00:46:55.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.504+0000 7fcbc6bcbb80 1 Processor -- start 2024-05-09T00:46:55.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.505+0000 7fcbc6bcbb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db4e6ad450 0x55db4e84ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:55.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.505+0000 7fcbc6bcbb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db4e797110 0x55db4e84f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:55.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.505+0000 7fcbc6bcbb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db4e8333b0 0x55db4e8b2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:55.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.505+0000 7fcbc6bcbb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55db4e849310 con 0x55db4e8333b0 2024-05-09T00:46:55.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.505+0000 7fcbc6bcbb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55db4e848c40 con 0x55db4e797110 2024-05-09T00:46:55.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.505+0000 7fcbc6bcbb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55db4e8499e0 con 0x55db4e6ad450 2024-05-09T00:46:55.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.505+0000 7fcbc6bcbb80 10 client.0 _ll_register_callbacks cb 0x55db4e5be670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:46:55.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.505+0000 7fcbb6752700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db4e8333b0 0x55db4e8b2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:55.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.505+0000 7fcbb6752700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db4e8333b0 0x55db4e8b2490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55014/0 (socket says 172.21.15.5:55014) 2024-05-09T00:46:55.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.505+0000 7fcbb6752700 1 -- 172.21.15.5:0/117524451 learned_addr learned my addr 172.21.15.5:0/117524451 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:46:55.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.505+0000 7fcbb5f51700 1 --2- 172.21.15.5:0/117524451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db4e6ad450 0x55db4e84ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:55.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.505+0000 7fcbb5750700 1 --2- 172.21.15.5:0/117524451 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db4e797110 0x55db4e84f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:55.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.505+0000 7fcbb6752700 1 -- 172.21.15.5:0/117524451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db4e6ad450 msgr2=0x55db4e84ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:55.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.505+0000 7fcbb6752700 1 --2- 172.21.15.5:0/117524451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db4e6ad450 0x55db4e84ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:55.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.505+0000 7fcbb6752700 1 -- 172.21.15.5:0/117524451 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db4e797110 msgr2=0x55db4e84f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:55.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.505+0000 7fcbb6752700 1 --2- 172.21.15.5:0/117524451 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db4e797110 0x55db4e84f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:55.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.505+0000 7fcbb6752700 1 -- 172.21.15.5:0/117524451 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db4e8b2a20 con 0x55db4e8333b0 2024-05-09T00:46:55.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.505+0000 7fcbb6752700 1 --2- 172.21.15.5:0/117524451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db4e8333b0 0x55db4e8b2490 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fcba800a390 tx=0x7fcba8008900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69f31dc1088211b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:55.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.505+0000 7fcb997fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:46:55.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.506+0000 7fcb997fa700 1 -- 172.21.15.5:0/117524451 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba8048060 con 0x55db4e8333b0 2024-05-09T00:46:55.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.506+0000 7fcb997fa700 1 -- 172.21.15.5:0/117524451 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fcba8008d40 con 0x55db4e8333b0 2024-05-09T00:46:55.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.506+0000 7fcbc6bcbb80 1 -- 172.21.15.5:0/117524451 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55db4e754a20 con 0x55db4e8333b0 2024-05-09T00:46:55.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.506+0000 7fcb997fa700 1 -- 172.21.15.5:0/117524451 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcba803f070 con 0x55db4e8333b0 2024-05-09T00:46:55.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.506+0000 7fcbc6bcbb80 10 client.15153 fetch_fsmap learned FSMap version 28 2024-05-09T00:46:55.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.506+0000 7fcbc6bcbb80 1 -- 172.21.15.5:0/117524451 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55db4e8b3c80 con 0x55db4e8333b0 2024-05-09T00:46:55.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.506+0000 7fcb997fa700 1 -- 172.21.15.5:0/117524451 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55db4e754a20 con 0x55db4e8333b0 2024-05-09T00:46:55.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.506+0000 7fcb997fa700 1 -- 172.21.15.5:0/117524451 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fcba8012040 con 0x55db4e8333b0 2024-05-09T00:46:55.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.506+0000 7fcbc6bcbb80 10 client.15153 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T00:46:55.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.506+0000 7fcbc6bcbb80 10 client.15153 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-09T00:46:55.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.506+0000 7fcbc6bcbb80 20 client.15153 populate_metadata read hostname 'smithi005' 2024-05-09T00:46:55.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.506+0000 7fcbc6bcbb80 1 --2- 172.21.15.5:0/117524451 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x55db4e8b5c70 0x55db4e8b70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:55.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.506+0000 7fcbc6bcbb80 4 client.15153 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T00:46:55.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.506+0000 7fcbc6bcbb80 1 -- 172.21.15.5:0/117524451 --> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] -- command(tid 0: {"prefix": "status"}) -- 0x55db4e5c8920 con 0x55db4e8b5c70 2024-05-09T00:46:55.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.507+0000 7fcbb5f51700 1 --2- 172.21.15.5:0/117524451 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x55db4e8b5c70 0x55db4e8b70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:55.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.507+0000 7fcbb5f51700 1 --2- 172.21.15.5:0/117524451 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x55db4e8b5c70 0x55db4e8b70e0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5a372ccc4fe0fbc0 server_cookie=68bf2d801b902ce in_seq=0 out_seq=0 2024-05-09T00:46:55.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.507+0000 7fcb997fa700 10 client.15153 ms_handle_connect on v2:172.21.15.131:6838/2748669631 2024-05-09T00:46:55.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.507+0000 7fcb997fa700 1 -- 172.21.15.5:0/117524451 <== mds.2 v2:172.21.15.131:6838/2748669631 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55db4e5c8920 con 0x55db4e8b5c70 2024-05-09T00:46:55.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.507+0000 7fcb997fa700 10 client.15153 handle_command_reply: tid=0 2024-05-09T00:46:55.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.507+0000 7fcbc6bcbb80 1 client.15153 shutdown 2024-05-09T00:46:55.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.507+0000 7fcbc6bcbb80 1 -- 172.21.15.5:0/117524451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db4e8333b0 msgr2=0x55db4e8b2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:55.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.507+0000 7fcbc6bcbb80 1 --2- 172.21.15.5:0/117524451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db4e8333b0 0x55db4e8b2490 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fcba800a390 tx=0x7fcba8008900 comp rx=0 tx=0).stop 2024-05-09T00:46:55.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.508+0000 7fcbc6bcbb80 1 -- 172.21.15.5:0/117524451 shutdown_connections 2024-05-09T00:46:55.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.508+0000 7fcbc6bcbb80 1 --2- 172.21.15.5:0/117524451 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x55db4e8b5c70 0x55db4e8b70e0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:55.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.508+0000 7fcbc6bcbb80 1 --2- 172.21.15.5:0/117524451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db4e8333b0 0x55db4e8b2490 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:55.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.508+0000 7fcbc6bcbb80 1 --2- 172.21.15.5:0/117524451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db4e6ad450 0x55db4e84ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:55.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.508+0000 7fcbc6bcbb80 1 --2- 172.21.15.5:0/117524451 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db4e797110 0x55db4e84f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:55.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.508+0000 7fcb997fa700 0 client.15153 ms_handle_reset on v2:172.21.15.131:6838/2748669631 2024-05-09T00:46:55.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.508+0000 7fcbc6bcbb80 1 -- 172.21.15.5:0/117524451 >> 172.21.15.5:0/117524451 conn(0x55db4e70d650 msgr2=0x55db4e835fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:55.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.508+0000 7fcbc6bcbb80 1 -- 172.21.15.5:0/117524451 shutdown_connections 2024-05-09T00:46:55.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.508+0000 7fcbc6bcbb80 1 -- 172.21.15.5:0/117524451 wait complete. 2024-05-09T00:46:55.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.508+0000 7fcbc6bcbb80 20 client.15153 trim_cache size 0 max 16384 2024-05-09T00:46:55.513 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-09T00:46:55.513 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "d85814c6-0d9b-11ef-bc97-c7b262605968", 2024-05-09T00:46:55.514 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 2, 2024-05-09T00:46:55.514 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24494, 2024-05-09T00:46:55.514 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-05-09T00:46:55.514 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-05-09T00:46:55.514 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-05-09T00:46:55.514 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 175.575105436, 2024-05-09T00:46:55.514 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 28, 2024-05-09T00:46:55.514 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 82, 2024-05-09T00:46:55.514 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 78, 2024-05-09T00:46:55.514 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 204.39030904099999 2024-05-09T00:46:55.514 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-09T00:46:55.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.508+0000 7fcbb81b5700 1 -- 172.21.15.5:0/2418802551 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fcb9c0b2880 msgr2=0x7fcb9c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:55.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.508+0000 7fcbb81b5700 1 --2- 172.21.15.5:0/2418802551 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fcb9c0b2880 0x7fcb9c0b4d90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fcba4005fd0 tx=0x7fcba4005ee0 comp rx=0 tx=0).stop 2024-05-09T00:46:55.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.508+0000 7fcbb81b5700 1 -- 172.21.15.5:0/2418802551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbb01029c0 msgr2=0x7fcbb019ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:55.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.508+0000 7fcbb81b5700 1 --2- 172.21.15.5:0/2418802551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbb01029c0 0x7fcbb019ae40 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fcbac00d980 tx=0x7fcbac00dc90 comp rx=0 tx=0).stop 2024-05-09T00:46:55.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.508+0000 7fcbb81b5700 1 -- 172.21.15.5:0/2418802551 shutdown_connections 2024-05-09T00:46:55.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.508+0000 7fcbb81b5700 1 --2- 172.21.15.5:0/2418802551 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fcb9c0b2880 0x7fcb9c0b4d90 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:55.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.508+0000 7fcbb81b5700 1 --2- 172.21.15.5:0/2418802551 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbb01089f0 0x7fcbb019f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:55.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.508+0000 7fcbb81b5700 1 --2- 172.21.15.5:0/2418802551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbb01029c0 0x7fcbb019ae40 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:55.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.508+0000 7fcbb81b5700 1 --2- 172.21.15.5:0/2418802551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbb0103310 0x7fcbb019b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:55.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.508+0000 7fcbb81b5700 1 -- 172.21.15.5:0/2418802551 >> 172.21.15.5:0/2418802551 conn(0x7fcbb00fe520 msgr2=0x7fcbb0107c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:55.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.508+0000 7fcbb81b5700 1 -- 172.21.15.5:0/2418802551 shutdown_connections 2024-05-09T00:46:55.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:55.509+0000 7fcbb81b5700 1 -- 172.21.15.5:0/2418802551 wait complete. 2024-05-09T00:46:55.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:46:55 smithi131 ceph-mon[34547]: pgmap v388: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-09T00:46:55.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:46:55 smithi157 ceph-mon[34294]: pgmap v388: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-09T00:46:56.235 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.c status 2024-05-09T00:46:56.485 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:46:57.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:46:57 smithi005 ceph-mon[30365]: pgmap v389: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T00:46:57.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:46:57 smithi131 ceph-mon[34547]: pgmap v389: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T00:46:57.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:46:57 smithi157 ceph-mon[34294]: pgmap v389: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T00:46:58.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.206+0000 7fe19b7cd700 1 -- 172.21.15.5:0/2696711028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1941034c0 msgr2=0x7fe1941038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:58.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.206+0000 7fe19b7cd700 1 --2- 172.21.15.5:0/2696711028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1941034c0 0x7fe1941038d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe1900099c0 tx=0x7fe190009cd0 comp rx=0 tx=0).stop 2024-05-09T00:46:58.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.206+0000 7fe19b7cd700 1 -- 172.21.15.5:0/2696711028 shutdown_connections 2024-05-09T00:46:58.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.206+0000 7fe19b7cd700 1 --2- 172.21.15.5:0/2696711028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe194108740 0x7fe1941100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:58.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.206+0000 7fe19b7cd700 1 --2- 172.21.15.5:0/2696711028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1941034c0 0x7fe1941038d0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:58.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.206+0000 7fe19b7cd700 1 --2- 172.21.15.5:0/2696711028 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe194107df0 0x7fe194108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:58.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.206+0000 7fe19b7cd700 1 -- 172.21.15.5:0/2696711028 >> 172.21.15.5:0/2696711028 conn(0x7fe1940fedf0 msgr2=0x7fe194101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:58.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.207+0000 7fe19b7cd700 1 -- 172.21.15.5:0/2696711028 shutdown_connections 2024-05-09T00:46:58.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.207+0000 7fe19b7cd700 1 -- 172.21.15.5:0/2696711028 wait complete. 2024-05-09T00:46:58.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.208+0000 7fe19b7cd700 1 Processor -- start 2024-05-09T00:46:58.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.208+0000 7fe19b7cd700 1 -- start start 2024-05-09T00:46:58.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.208+0000 7fe19b7cd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1941034c0 0x7fe194199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:58.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.208+0000 7fe19b7cd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe194107df0 0x7fe19419a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:58.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.208+0000 7fe19b7cd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe194108740 0x7fe19419e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:58.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.208+0000 7fe19b7cd700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe194113370 con 0x7fe194108740 2024-05-09T00:46:58.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.208+0000 7fe19b7cd700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe1941131f0 con 0x7fe1941034c0 2024-05-09T00:46:58.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.208+0000 7fe19b7cd700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe1941134f0 con 0x7fe194107df0 2024-05-09T00:46:58.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.208+0000 7fe199d6a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe194108740 0x7fe19419e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:58.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.208+0000 7fe199d6a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe194108740 0x7fe19419e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55038/0 (socket says 172.21.15.5:55038) 2024-05-09T00:46:58.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.208+0000 7fe199d6a700 1 -- 172.21.15.5:0/1062966524 learned_addr learned my addr 172.21.15.5:0/1062966524 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:46:58.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.208+0000 7fe198d68700 1 --2- 172.21.15.5:0/1062966524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe194107df0 0x7fe19419a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:58.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.208+0000 7fe199569700 1 --2- 172.21.15.5:0/1062966524 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1941034c0 0x7fe194199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:58.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.209+0000 7fe199d6a700 1 -- 172.21.15.5:0/1062966524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe194107df0 msgr2=0x7fe19419a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:58.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.209+0000 7fe199d6a700 1 --2- 172.21.15.5:0/1062966524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe194107df0 0x7fe19419a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:58.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.209+0000 7fe199d6a700 1 -- 172.21.15.5:0/1062966524 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1941034c0 msgr2=0x7fe194199d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:58.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.209+0000 7fe199d6a700 1 --2- 172.21.15.5:0/1062966524 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1941034c0 0x7fe194199d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:58.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.209+0000 7fe199d6a700 1 -- 172.21.15.5:0/1062966524 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe19419eb80 con 0x7fe194108740 2024-05-09T00:46:58.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.209+0000 7fe198d68700 1 --2- 172.21.15.5:0/1062966524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe194107df0 0x7fe19419a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:46:58.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.209+0000 7fe199569700 1 --2- 172.21.15.5:0/1062966524 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1941034c0 0x7fe194199d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:46:58.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.209+0000 7fe199d6a700 1 --2- 172.21.15.5:0/1062966524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe194108740 0x7fe19419e590 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe18c007d70 tx=0x7fe18c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:58.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.209+0000 7fe18a7fc700 1 -- 172.21.15.5:0/1062966524 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18c016020 con 0x7fe194108740 2024-05-09T00:46:58.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.209+0000 7fe18a7fc700 1 -- 172.21.15.5:0/1062966524 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe18c00ad40 con 0x7fe194108740 2024-05-09T00:46:58.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.209+0000 7fe18a7fc700 1 -- 172.21.15.5:0/1062966524 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18c0115e0 con 0x7fe194108740 2024-05-09T00:46:58.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.209+0000 7fe19b7cd700 1 -- 172.21.15.5:0/1062966524 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe19419ee70 con 0x7fe194108740 2024-05-09T00:46:58.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.209+0000 7fe19b7cd700 1 -- 172.21.15.5:0/1062966524 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe19419f320 con 0x7fe194108740 2024-05-09T00:46:58.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.212+0000 7fe18a7fc700 1 -- 172.21.15.5:0/1062966524 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fe18c019020 con 0x7fe194108740 2024-05-09T00:46:58.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.213+0000 7fe1aa1e3b80 1 Processor -- start 2024-05-09T00:46:58.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.213+0000 7fe1aa1e3b80 1 -- start start 2024-05-09T00:46:58.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.213+0000 7fe18a7fc700 1 --2- 172.21.15.5:0/1062966524 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe1800b2900 0x7fe1800b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:58.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.214+0000 7fe1aa1e3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d75ed33110 0x55d75edcc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:58.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.214+0000 7fe1aa1e3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d75ec49450 0x55d75edcc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:58.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.214+0000 7fe1aa1e3b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d75edc6650 0x55d75edc6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:58.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.214+0000 7fe1aa1e3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d75ece8fa0 con 0x55d75ec49450 2024-05-09T00:46:58.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.214+0000 7fe1aa1e3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55d75ec877a0 con 0x55d75ed33110 2024-05-09T00:46:58.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.214+0000 7fe1aa1e3b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d75ece4f60 con 0x55d75edc6650 2024-05-09T00:46:58.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.214+0000 7fe18a7fc700 1 -- 172.21.15.5:0/1062966524 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fe18c0e2170 con 0x7fe194108740 2024-05-09T00:46:58.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.214+0000 7fe198d68700 1 --2- 172.21.15.5:0/1062966524 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe1800b2900 0x7fe1800b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:58.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.214+0000 7fe199d6a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d75edc6650 0x55d75edc6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:58.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.214+0000 7fe198d68700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d75ec49450 0x55d75edcc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:58.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.214+0000 7fe199d6a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d75edc6650 0x55d75edc6a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:53082/0 (socket says 172.21.15.5:53082) 2024-05-09T00:46:58.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.214+0000 7fe199d6a700 1 -- 172.21.15.5:0/498954536 learned_addr learned my addr 172.21.15.5:0/498954536 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:46:58.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.214+0000 7fe199d6a700 1 -- 172.21.15.5:0/498954536 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d75ed33110 msgr2=0x55d75edcc4a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:46:58.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.214+0000 7fe199d6a700 1 --2- 172.21.15.5:0/498954536 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d75ed33110 0x55d75edcc4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:58.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.214+0000 7fe199d6a700 1 -- 172.21.15.5:0/498954536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d75ec49450 msgr2=0x55d75edcc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:58.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.214+0000 7fe199d6a700 1 --2- 172.21.15.5:0/498954536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d75ec49450 0x55d75edcc9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:58.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.214+0000 7fe199d6a700 1 -- 172.21.15.5:0/498954536 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d75ec7f320 con 0x55d75edc6650 2024-05-09T00:46:58.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.214+0000 7fe198d68700 1 --2- 172.21.15.5:0/498954536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d75ec49450 0x55d75edcc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:46:58.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.214+0000 7fe198d68700 1 --2- 172.21.15.5:0/1062966524 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe1800b2900 0x7fe1800b4e10 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe194107d70 tx=0x7fe18400bc10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:58.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.214+0000 7fe199d6a700 1 --2- 172.21.15.5:0/498954536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d75edc6650 0x55d75edc6a80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe18c0a96f0 tx=0x7fe18c0a8ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cf9e8fda2796ded server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:58.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.216+0000 7fe17f7fe700 1 -- 172.21.15.5:0/498954536 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18c0a9e60 con 0x55d75edc6650 2024-05-09T00:46:58.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.216+0000 7fe17f7fe700 1 -- 172.21.15.5:0/498954536 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe18c0a9760 con 0x55d75edc6650 2024-05-09T00:46:58.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.216+0000 7fe17f7fe700 1 -- 172.21.15.5:0/498954536 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18c0910d0 con 0x55d75edc6650 2024-05-09T00:46:58.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.217+0000 7fe1aa1e3b80 1 -- 172.21.15.5:0/498954536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d75edc6650 msgr2=0x55d75edc6a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:58.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.217+0000 7fe1aa1e3b80 1 --2- 172.21.15.5:0/498954536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d75edc6650 0x55d75edc6a80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe18c0a96f0 tx=0x7fe18c0a8ed0 comp rx=0 tx=0).stop 2024-05-09T00:46:58.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.217+0000 7fe1aa1e3b80 1 -- 172.21.15.5:0/498954536 shutdown_connections 2024-05-09T00:46:58.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.217+0000 7fe1aa1e3b80 1 --2- 172.21.15.5:0/498954536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d75edc6650 0x55d75edc6a80 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:58.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.217+0000 7fe1aa1e3b80 1 --2- 172.21.15.5:0/498954536 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d75ed33110 0x55d75edcc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:58.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.217+0000 7fe1aa1e3b80 1 --2- 172.21.15.5:0/498954536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d75ec49450 0x55d75edcc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:58.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.217+0000 7fe1aa1e3b80 1 -- 172.21.15.5:0/498954536 >> 172.21.15.5:0/498954536 conn(0x55d75eca9650 msgr2=0x55d75edc13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:58.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.217+0000 7fe1aa1e3b80 1 -- 172.21.15.5:0/498954536 shutdown_connections 2024-05-09T00:46:58.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.217+0000 7fe1aa1e3b80 1 -- 172.21.15.5:0/498954536 wait complete. 2024-05-09T00:46:58.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.218+0000 7fe1aa1e3b80 1 -- start start 2024-05-09T00:46:58.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.218+0000 7fe1aa1e3b80 1 Processor -- start 2024-05-09T00:46:58.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.218+0000 7fe1aa1e3b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d75ec49450 0x55d75edd01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:58.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.218+0000 7fe1aa1e3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d75ed33110 0x55d75edd0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:58.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.218+0000 7fe1aa1e3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d75edc6650 0x55d75edd0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:58.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.218+0000 7fe1aa1e3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d75eb64b50 con 0x55d75ed33110 2024-05-09T00:46:58.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.218+0000 7fe1aa1e3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55d75eca3100 con 0x55d75edc6650 2024-05-09T00:46:58.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.218+0000 7fe1aa1e3b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d75edc73d0 con 0x55d75ec49450 2024-05-09T00:46:58.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.218+0000 7fe1aa1e3b80 10 client.0 _ll_register_callbacks cb 0x55d75eb5a670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:46:58.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.218+0000 7fe198d68700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d75ed33110 0x55d75edd0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:58.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.219+0000 7fe198d68700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d75ed33110 0x55d75edd0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55056/0 (socket says 172.21.15.5:55056) 2024-05-09T00:46:58.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.219+0000 7fe198d68700 1 -- 172.21.15.5:0/385146862 learned_addr learned my addr 172.21.15.5:0/385146862 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:46:58.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.219+0000 7fe199d6a700 1 --2- 172.21.15.5:0/385146862 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d75edc6650 0x55d75edd0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:58.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.219+0000 7fe199569700 1 --2- 172.21.15.5:0/385146862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d75ec49450 0x55d75edd01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:58.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.219+0000 7fe198d68700 1 -- 172.21.15.5:0/385146862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d75ec49450 msgr2=0x55d75edd01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:58.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.219+0000 7fe198d68700 1 --2- 172.21.15.5:0/385146862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d75ec49450 0x55d75edd01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:58.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.219+0000 7fe198d68700 1 -- 172.21.15.5:0/385146862 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d75edc6650 msgr2=0x55d75edd0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:58.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.219+0000 7fe198d68700 1 --2- 172.21.15.5:0/385146862 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d75edc6650 0x55d75edd0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:58.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.219+0000 7fe198d68700 1 -- 172.21.15.5:0/385146862 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d75eb64440 con 0x55d75ed33110 2024-05-09T00:46:58.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.219+0000 7fe198d68700 1 --2- 172.21.15.5:0/385146862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d75ed33110 0x55d75edd0720 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe184011c40 tx=0x7fe1840058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dd5d9ca0e47082 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:58.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.219+0000 7fe17cff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:46:58.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.219+0000 7fe17cff9700 1 -- 172.21.15.5:0/385146862 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe184012cf0 con 0x55d75ed33110 2024-05-09T00:46:58.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.219+0000 7fe17cff9700 1 -- 172.21.15.5:0/385146862 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fe184020420 con 0x55d75ed33110 2024-05-09T00:46:58.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.219+0000 7fe1aa1e3b80 1 -- 172.21.15.5:0/385146862 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe18c0a9760 con 0x55d75ed33110 2024-05-09T00:46:58.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.219+0000 7fe17cff9700 1 -- 172.21.15.5:0/385146862 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe18400b5e0 con 0x55d75ed33110 2024-05-09T00:46:58.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.219+0000 7fe1aa1e3b80 10 client.15165 fetch_fsmap learned FSMap version 28 2024-05-09T00:46:58.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.219+0000 7fe1aa1e3b80 1 -- 172.21.15.5:0/385146862 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d75eddda40 con 0x55d75ed33110 2024-05-09T00:46:58.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.220+0000 7fe17cff9700 1 -- 172.21.15.5:0/385146862 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe18c0a9760 con 0x55d75ed33110 2024-05-09T00:46:58.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.220+0000 7fe17cff9700 1 -- 172.21.15.5:0/385146862 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fe184033070 con 0x55d75ed33110 2024-05-09T00:46:58.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.220+0000 7fe1aa1e3b80 10 client.15165 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T00:46:58.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.220+0000 7fe1aa1e3b80 10 client.15165 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-09T00:46:58.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.220+0000 7fe1aa1e3b80 20 client.15165 populate_metadata read hostname 'smithi005' 2024-05-09T00:46:58.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.220+0000 7fe1aa1e3b80 1 --2- 172.21.15.5:0/385146862 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x55d75ede0040 0x55d75ede14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:58.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.220+0000 7fe1aa1e3b80 4 client.15165 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:46:58.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.220+0000 7fe1aa1e3b80 1 -- 172.21.15.5:0/385146862 --> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d75ecf6980 con 0x55d75ede0040 2024-05-09T00:46:58.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.220+0000 7fe199569700 1 --2- 172.21.15.5:0/385146862 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x55d75ede0040 0x55d75ede14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:58.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.221+0000 7fe199569700 1 --2- 172.21.15.5:0/385146862 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x55d75ede0040 0x55d75ede14b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4dc1bed61eaa0237 server_cookie=f22979edb965d355 in_seq=0 out_seq=0 2024-05-09T00:46:58.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.221+0000 7fe17cff9700 10 client.15165 ms_handle_connect on v2:172.21.15.157:6836/744863009 2024-05-09T00:46:58.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.222+0000 7fe17cff9700 1 -- 172.21.15.5:0/385146862 <== mds.1 v2:172.21.15.157:6836/744863009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d75ecf6980 con 0x55d75ede0040 2024-05-09T00:46:58.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.222+0000 7fe17cff9700 10 client.15165 handle_command_reply: tid=0 2024-05-09T00:46:58.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.222+0000 7fe1aa1e3b80 1 client.15165 shutdown 2024-05-09T00:46:58.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.222+0000 7fe1aa1e3b80 1 -- 172.21.15.5:0/385146862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d75ed33110 msgr2=0x55d75edd0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:58.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.222+0000 7fe1aa1e3b80 1 --2- 172.21.15.5:0/385146862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d75ed33110 0x55d75edd0720 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe184011c40 tx=0x7fe1840058e0 comp rx=0 tx=0).stop 2024-05-09T00:46:58.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.222+0000 7fe1aa1e3b80 1 -- 172.21.15.5:0/385146862 shutdown_connections 2024-05-09T00:46:58.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.222+0000 7fe1aa1e3b80 1 --2- 172.21.15.5:0/385146862 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x55d75ede0040 0x55d75ede14b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:58.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.222+0000 7fe17cff9700 0 client.15165 ms_handle_reset on v2:172.21.15.157:6836/744863009 2024-05-09T00:46:58.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.222+0000 7fe1aa1e3b80 1 --2- 172.21.15.5:0/385146862 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d75edc6650 0x55d75edd0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:58.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.222+0000 7fe1aa1e3b80 1 --2- 172.21.15.5:0/385146862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d75ec49450 0x55d75edd01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:58.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.222+0000 7fe1aa1e3b80 1 --2- 172.21.15.5:0/385146862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d75ed33110 0x55d75edd0720 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:58.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.222+0000 7fe1aa1e3b80 1 -- 172.21.15.5:0/385146862 >> 172.21.15.5:0/385146862 conn(0x55d75eca9650 msgr2=0x55d75edc1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:58.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.222+0000 7fe1aa1e3b80 1 -- 172.21.15.5:0/385146862 shutdown_connections 2024-05-09T00:46:58.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.222+0000 7fe1aa1e3b80 1 -- 172.21.15.5:0/385146862 wait complete. 2024-05-09T00:46:58.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.223+0000 7fe1aa1e3b80 20 client.15165 trim_cache size 0 max 16384 2024-05-09T00:46:58.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.244+0000 7fe1aa1e3b80 1 Processor -- start 2024-05-09T00:46:58.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.244+0000 7fe1aa1e3b80 1 -- start start 2024-05-09T00:46:58.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.244+0000 7fe1aa1e3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d75ec49450 0x55d75edc9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:58.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.244+0000 7fe1aa1e3b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d75ed33110 0x55d75edca2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:58.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.244+0000 7fe1aa1e3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d75edc6650 0x55d75edca830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:58.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.244+0000 7fe1aa1e3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d75eddf7f0 con 0x55d75ec49450 2024-05-09T00:46:58.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.244+0000 7fe1aa1e3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55d75eddfec0 con 0x55d75edc6650 2024-05-09T00:46:58.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.244+0000 7fe1aa1e3b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d75ede0590 con 0x55d75ed33110 2024-05-09T00:46:58.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.244+0000 7fe199569700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d75ec49450 0x55d75edc9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:58.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.244+0000 7fe199569700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d75ec49450 0x55d75edc9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55072/0 (socket says 172.21.15.5:55072) 2024-05-09T00:46:58.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.244+0000 7fe199569700 1 -- 172.21.15.5:0/4291339517 learned_addr learned my addr 172.21.15.5:0/4291339517 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:46:58.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.244+0000 7fe198d68700 1 --2- 172.21.15.5:0/4291339517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d75ed33110 0x55d75edca2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:58.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.244+0000 7fe199d6a700 1 --2- 172.21.15.5:0/4291339517 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d75edc6650 0x55d75edca830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:58.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.244+0000 7fe199569700 1 -- 172.21.15.5:0/4291339517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d75ed33110 msgr2=0x55d75edca2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:58.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.244+0000 7fe199569700 1 --2- 172.21.15.5:0/4291339517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d75ed33110 0x55d75edca2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:58.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.244+0000 7fe199569700 1 -- 172.21.15.5:0/4291339517 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d75edc6650 msgr2=0x55d75edca830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:58.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.244+0000 7fe199569700 1 --2- 172.21.15.5:0/4291339517 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d75edc6650 0x55d75edca830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:58.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.245+0000 7fe199569700 1 -- 172.21.15.5:0/4291339517 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d75edc3130 con 0x55d75ec49450 2024-05-09T00:46:58.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.245+0000 7fe198d68700 1 --2- 172.21.15.5:0/4291339517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d75ed33110 0x55d75edca2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:46:58.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.245+0000 7fe199d6a700 1 --2- 172.21.15.5:0/4291339517 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d75edc6650 0x55d75edca830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:46:58.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.245+0000 7fe199569700 1 --2- 172.21.15.5:0/4291339517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d75ec49450 0x55d75edc9db0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe19000b580 tx=0x7fe19000bad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25f7947aaa8c4527 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:58.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.245+0000 7fe17effd700 1 -- 172.21.15.5:0/4291339517 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe190003860 con 0x55d75ec49450 2024-05-09T00:46:58.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.245+0000 7fe17effd700 1 -- 172.21.15.5:0/4291339517 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe190005070 con 0x55d75ec49450 2024-05-09T00:46:58.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.245+0000 7fe17effd700 1 -- 172.21.15.5:0/4291339517 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1900283d0 con 0x55d75ec49450 2024-05-09T00:46:58.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.246+0000 7fe1aa1e3b80 1 -- 172.21.15.5:0/4291339517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d75ec49450 msgr2=0x55d75edc9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:58.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.246+0000 7fe1aa1e3b80 1 --2- 172.21.15.5:0/4291339517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d75ec49450 0x55d75edc9db0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe19000b580 tx=0x7fe19000bad0 comp rx=0 tx=0).stop 2024-05-09T00:46:58.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.246+0000 7fe1aa1e3b80 1 -- 172.21.15.5:0/4291339517 shutdown_connections 2024-05-09T00:46:58.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.246+0000 7fe1aa1e3b80 1 --2- 172.21.15.5:0/4291339517 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d75edc6650 0x55d75edca830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:58.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.246+0000 7fe1aa1e3b80 1 --2- 172.21.15.5:0/4291339517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d75ec49450 0x55d75edc9db0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:58.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.246+0000 7fe1aa1e3b80 1 --2- 172.21.15.5:0/4291339517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d75ed33110 0x55d75edca2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:58.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.246+0000 7fe1aa1e3b80 1 -- 172.21.15.5:0/4291339517 >> 172.21.15.5:0/4291339517 conn(0x55d75eca9650 msgr2=0x55d75edcb2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:58.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.246+0000 7fe1aa1e3b80 1 -- 172.21.15.5:0/4291339517 shutdown_connections 2024-05-09T00:46:58.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.246+0000 7fe1aa1e3b80 1 -- 172.21.15.5:0/4291339517 wait complete. 2024-05-09T00:46:58.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.246+0000 7fe1aa1e3b80 1 -- start start 2024-05-09T00:46:58.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.246+0000 7fe1aa1e3b80 1 Processor -- start 2024-05-09T00:46:58.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.247+0000 7fe1aa1e3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d75ec49450 0x55d75ede70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:58.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.247+0000 7fe1aa1e3b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d75ed33110 0x55d75ede7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:58.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.247+0000 7fe1aa1e3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d75edc6650 0x55d75ee4a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:58.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.247+0000 7fe1aa1e3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d75ede1330 con 0x55d75edc6650 2024-05-09T00:46:58.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.247+0000 7fe1aa1e3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55d75ede0c60 con 0x55d75ec49450 2024-05-09T00:46:58.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.247+0000 7fe1aa1e3b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d75ede1a00 con 0x55d75ed33110 2024-05-09T00:46:58.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.247+0000 7fe1aa1e3b80 10 client.0 _ll_register_callbacks cb 0x55d75eb5a670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:46:58.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.247+0000 7fe199d6a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d75edc6650 0x55d75ee4a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:58.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.247+0000 7fe199d6a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d75edc6650 0x55d75ee4a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55084/0 (socket says 172.21.15.5:55084) 2024-05-09T00:46:58.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.247+0000 7fe199d6a700 1 -- 172.21.15.5:0/2794688228 learned_addr learned my addr 172.21.15.5:0/2794688228 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:46:58.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.247+0000 7fe199569700 1 --2- 172.21.15.5:0/2794688228 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d75ec49450 0x55d75ede70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:58.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.247+0000 7fe198d68700 1 --2- 172.21.15.5:0/2794688228 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d75ed33110 0x55d75ede7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:58.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.247+0000 7fe199d6a700 1 -- 172.21.15.5:0/2794688228 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d75ed33110 msgr2=0x55d75ede7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:58.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.247+0000 7fe199d6a700 1 --2- 172.21.15.5:0/2794688228 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d75ed33110 0x55d75ede7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:58.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.247+0000 7fe199d6a700 1 -- 172.21.15.5:0/2794688228 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d75ec49450 msgr2=0x55d75ede70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:46:58.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.247+0000 7fe199d6a700 1 --2- 172.21.15.5:0/2794688228 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d75ec49450 0x55d75ede70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:58.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.247+0000 7fe199d6a700 1 -- 172.21.15.5:0/2794688228 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d75ee4aa20 con 0x55d75edc6650 2024-05-09T00:46:58.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.248+0000 7fe199d6a700 1 --2- 172.21.15.5:0/2794688228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d75edc6650 0x55d75ee4a4b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe18c0a69d0 tx=0x7fe18c0b2360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43e17ecc831298d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:46:58.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.248+0000 7fe17cff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:46:58.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.248+0000 7fe17cff9700 1 -- 172.21.15.5:0/2794688228 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18c0adb10 con 0x55d75edc6650 2024-05-09T00:46:58.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.248+0000 7fe17cff9700 1 -- 172.21.15.5:0/2794688228 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fe18c0910d0 con 0x55d75edc6650 2024-05-09T00:46:58.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.248+0000 7fe1aa1e3b80 1 -- 172.21.15.5:0/2794688228 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe190005070 con 0x55d75edc6650 2024-05-09T00:46:58.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.248+0000 7fe17cff9700 1 -- 172.21.15.5:0/2794688228 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe18c0a9e60 con 0x55d75edc6650 2024-05-09T00:46:58.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.248+0000 7fe1aa1e3b80 10 client.15171 fetch_fsmap learned FSMap version 28 2024-05-09T00:46:58.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.248+0000 7fe1aa1e3b80 1 -- 172.21.15.5:0/2794688228 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d75ee4bc60 con 0x55d75edc6650 2024-05-09T00:46:58.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.248+0000 7fe17cff9700 1 -- 172.21.15.5:0/2794688228 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe190005070 con 0x55d75edc6650 2024-05-09T00:46:58.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.248+0000 7fe17cff9700 1 -- 172.21.15.5:0/2794688228 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fe18c08c070 con 0x55d75edc6650 2024-05-09T00:46:58.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.248+0000 7fe1aa1e3b80 10 client.15171 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T00:46:58.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.248+0000 7fe1aa1e3b80 10 client.15171 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-09T00:46:58.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.248+0000 7fe1aa1e3b80 20 client.15171 populate_metadata read hostname 'smithi005' 2024-05-09T00:46:58.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.248+0000 7fe1aa1e3b80 1 --2- 172.21.15.5:0/2794688228 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x55d75ede0040 0x55d75ee4f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:46:58.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.248+0000 7fe1aa1e3b80 4 client.15171 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T00:46:58.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.248+0000 7fe1aa1e3b80 1 -- 172.21.15.5:0/2794688228 --> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] -- command(tid 0: {"prefix": "status"}) -- 0x55d75ecf0a20 con 0x55d75ede0040 2024-05-09T00:46:58.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.249+0000 7fe199569700 1 --2- 172.21.15.5:0/2794688228 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x55d75ede0040 0x55d75ee4f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:46:58.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.249+0000 7fe17cff9700 10 client.15171 ms_handle_connect on v2:172.21.15.157:6836/744863009 2024-05-09T00:46:58.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.249+0000 7fe199569700 1 --2- 172.21.15.5:0/2794688228 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x55d75ede0040 0x55d75ee4f2a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=43125c74c89fa292 server_cookie=d7231327a2f42d15 in_seq=0 out_seq=0 2024-05-09T00:46:58.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.249+0000 7fe17cff9700 1 -- 172.21.15.5:0/2794688228 <== mds.1 v2:172.21.15.157:6836/744863009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55d75ecf0a20 con 0x55d75ede0040 2024-05-09T00:46:58.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.249+0000 7fe17cff9700 10 client.15171 handle_command_reply: tid=0 2024-05-09T00:46:58.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.249+0000 7fe1aa1e3b80 1 client.15171 shutdown 2024-05-09T00:46:58.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.249+0000 7fe1aa1e3b80 1 -- 172.21.15.5:0/2794688228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d75edc6650 msgr2=0x55d75ee4a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:58.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.249+0000 7fe1aa1e3b80 1 --2- 172.21.15.5:0/2794688228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d75edc6650 0x55d75ee4a4b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe18c0a69d0 tx=0x7fe18c0b2360 comp rx=0 tx=0).stop 2024-05-09T00:46:58.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.249+0000 7fe1aa1e3b80 1 -- 172.21.15.5:0/2794688228 shutdown_connections 2024-05-09T00:46:58.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.249+0000 7fe1aa1e3b80 1 --2- 172.21.15.5:0/2794688228 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x55d75ede0040 0x55d75ee4f2a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:58.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.250+0000 7fe1aa1e3b80 1 --2- 172.21.15.5:0/2794688228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d75edc6650 0x55d75ee4a4b0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:58.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.250+0000 7fe1aa1e3b80 1 --2- 172.21.15.5:0/2794688228 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d75ec49450 0x55d75ede70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:58.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.250+0000 7fe1aa1e3b80 1 --2- 172.21.15.5:0/2794688228 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d75ed33110 0x55d75ede7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:58.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.250+0000 7fe1aa1e3b80 1 -- 172.21.15.5:0/2794688228 >> 172.21.15.5:0/2794688228 conn(0x55d75eca9650 msgr2=0x55d75edc9db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:58.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.250+0000 7fe17cff9700 0 client.15171 ms_handle_reset on v2:172.21.15.157:6836/744863009 2024-05-09T00:46:58.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.250+0000 7fe1aa1e3b80 1 -- 172.21.15.5:0/2794688228 shutdown_connections 2024-05-09T00:46:58.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.250+0000 7fe1aa1e3b80 1 -- 172.21.15.5:0/2794688228 wait complete. 2024-05-09T00:46:58.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.250+0000 7fe1aa1e3b80 20 client.15171 trim_cache size 0 max 16384 2024-05-09T00:46:58.256 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-09T00:46:58.256 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "d85814c6-0d9b-11ef-bc97-c7b262605968", 2024-05-09T00:46:58.256 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 1, 2024-05-09T00:46:58.256 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24499, 2024-05-09T00:46:58.256 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-05-09T00:46:58.256 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-05-09T00:46:58.256 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-05-09T00:46:58.256 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 179.32856937099999, 2024-05-09T00:46:58.256 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 28, 2024-05-09T00:46:58.257 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 82, 2024-05-09T00:46:58.257 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 78, 2024-05-09T00:46:58.257 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 222.98342242300001 2024-05-09T00:46:58.257 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-09T00:46:58.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.250+0000 7fe19b7cd700 1 -- 172.21.15.5:0/1062966524 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe1800b2900 msgr2=0x7fe1800b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:58.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.250+0000 7fe19b7cd700 1 --2- 172.21.15.5:0/1062966524 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe1800b2900 0x7fe1800b4e10 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe194107d70 tx=0x7fe18400bc10 comp rx=0 tx=0).stop 2024-05-09T00:46:58.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.250+0000 7fe19b7cd700 1 -- 172.21.15.5:0/1062966524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe194108740 msgr2=0x7fe19419e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:46:58.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.250+0000 7fe19b7cd700 1 --2- 172.21.15.5:0/1062966524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe194108740 0x7fe19419e590 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe18c007d70 tx=0x7fe18c00a340 comp rx=0 tx=0).stop 2024-05-09T00:46:58.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.250+0000 7fe19b7cd700 1 -- 172.21.15.5:0/1062966524 shutdown_connections 2024-05-09T00:46:58.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.250+0000 7fe19b7cd700 1 --2- 172.21.15.5:0/1062966524 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe1800b2900 0x7fe1800b4e10 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:58.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.250+0000 7fe19b7cd700 1 --2- 172.21.15.5:0/1062966524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe194108740 0x7fe19419e590 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:58.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.250+0000 7fe19b7cd700 1 --2- 172.21.15.5:0/1062966524 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1941034c0 0x7fe194199d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:58.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.250+0000 7fe19b7cd700 1 --2- 172.21.15.5:0/1062966524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe194107df0 0x7fe19419a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:46:58.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.250+0000 7fe19b7cd700 1 -- 172.21.15.5:0/1062966524 >> 172.21.15.5:0/1062966524 conn(0x7fe1940fedf0 msgr2=0x7fe194101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:46:58.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.251+0000 7fe19b7cd700 1 -- 172.21.15.5:0/1062966524 shutdown_connections 2024-05-09T00:46:58.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:46:58.251+0000 7fe19b7cd700 1 -- 172.21.15.5:0/1062966524 wait complete. 2024-05-09T00:46:59.018 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.i status 2024-05-09T00:46:59.270 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:46:59.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:46:59 smithi005 ceph-mon[30365]: pgmap v390: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T00:46:59.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:46:59 smithi131 ceph-mon[34547]: pgmap v390: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T00:46:59.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:46:59 smithi157 ceph-mon[34294]: pgmap v390: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T00:47:00.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.958+0000 7ff890d72700 1 -- 172.21.15.5:0/2183260720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff88c109850 msgr2=0x7ff88c1111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:00.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.958+0000 7ff890d72700 1 --2- 172.21.15.5:0/2183260720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff88c109850 0x7ff88c1111b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff88400b0d0 tx=0x7ff88400b3e0 comp rx=0 tx=0).stop 2024-05-09T00:47:00.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.958+0000 7ff890d72700 1 -- 172.21.15.5:0/2183260720 shutdown_connections 2024-05-09T00:47:00.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.958+0000 7ff890d72700 1 --2- 172.21.15.5:0/2183260720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff88c109850 0x7ff88c1111b0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:00.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.958+0000 7ff890d72700 1 --2- 172.21.15.5:0/2183260720 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff88c1045d0 0x7ff88c1049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:00.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.958+0000 7ff890d72700 1 --2- 172.21.15.5:0/2183260720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff88c108f00 0x7ff88c109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:00.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.958+0000 7ff890d72700 1 -- 172.21.15.5:0/2183260720 >> 172.21.15.5:0/2183260720 conn(0x7ff88c0ffea0 msgr2=0x7ff88c1022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:00.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.958+0000 7ff890d72700 1 -- 172.21.15.5:0/2183260720 shutdown_connections 2024-05-09T00:47:00.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.958+0000 7ff890d72700 1 -- 172.21.15.5:0/2183260720 wait complete. 2024-05-09T00:47:00.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.959+0000 7ff890d72700 1 Processor -- start 2024-05-09T00:47:00.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.959+0000 7ff890d72700 1 -- start start 2024-05-09T00:47:00.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.959+0000 7ff890d72700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff88c1045d0 0x7ff88c19ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:00.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.959+0000 7ff890d72700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff88c108f00 0x7ff88c19b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:00.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.959+0000 7ff890d72700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff88c109850 0x7ff88c19f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:00.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.959+0000 7ff890d72700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff88c114480 con 0x7ff88c109850 2024-05-09T00:47:00.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.959+0000 7ff890d72700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff88c114300 con 0x7ff88c1045d0 2024-05-09T00:47:00.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.959+0000 7ff890d72700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff88c114600 con 0x7ff88c108f00 2024-05-09T00:47:00.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.960+0000 7ff88ad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff88c109850 0x7ff88c19f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:00.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.960+0000 7ff88ad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff88c109850 0x7ff88c19f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55108/0 (socket says 172.21.15.5:55108) 2024-05-09T00:47:00.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.960+0000 7ff889d9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff88c108f00 0x7ff88c19b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:00.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.960+0000 7ff88a59c700 1 --2- 172.21.15.5:0/3086536139 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff88c1045d0 0x7ff88c19ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:00.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.960+0000 7ff88ad9d700 1 -- 172.21.15.5:0/3086536139 learned_addr learned my addr 172.21.15.5:0/3086536139 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:47:00.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.960+0000 7ff889d9b700 1 -- 172.21.15.5:0/3086536139 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff88c1045d0 msgr2=0x7ff88c19ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:00.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.960+0000 7ff889d9b700 1 --2- 172.21.15.5:0/3086536139 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff88c1045d0 0x7ff88c19ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:00.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.960+0000 7ff889d9b700 1 -- 172.21.15.5:0/3086536139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff88c109850 msgr2=0x7ff88c19f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:00.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.960+0000 7ff889d9b700 1 --2- 172.21.15.5:0/3086536139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff88c109850 0x7ff88c19f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:00.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.960+0000 7ff889d9b700 1 -- 172.21.15.5:0/3086536139 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff88c19fc60 con 0x7ff88c108f00 2024-05-09T00:47:00.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.960+0000 7ff88ad9d700 1 --2- 172.21.15.5:0/3086536139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff88c109850 0x7ff88c19f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:47:00.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.960+0000 7ff88a59c700 1 --2- 172.21.15.5:0/3086536139 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff88c1045d0 0x7ff88c19ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:47:00.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.960+0000 7ff889d9b700 1 --2- 172.21.15.5:0/3086536139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff88c108f00 0x7ff88c19b330 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff88c06c170 tx=0x7ff88000dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:00.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.960+0000 7ff87b7fe700 1 -- 172.21.15.5:0/3086536139 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff880016070 con 0x7ff88c108f00 2024-05-09T00:47:00.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.960+0000 7ff890d72700 1 -- 172.21.15.5:0/3086536139 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff88c19ff50 con 0x7ff88c108f00 2024-05-09T00:47:00.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.961+0000 7ff87b7fe700 1 -- 172.21.15.5:0/3086536139 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff88000fd40 con 0x7ff88c108f00 2024-05-09T00:47:00.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.961+0000 7ff87b7fe700 1 -- 172.21.15.5:0/3086536139 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff88000e3b0 con 0x7ff88c108f00 2024-05-09T00:47:00.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.961+0000 7ff890d72700 1 -- 172.21.15.5:0/3086536139 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff88c076b00 con 0x7ff88c108f00 2024-05-09T00:47:00.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.964+0000 7ff87b7fe700 1 -- 172.21.15.5:0/3086536139 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7ff880019070 con 0x7ff88c108f00 2024-05-09T00:47:00.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.964+0000 7ff89f788b80 1 Processor -- start 2024-05-09T00:47:00.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.964+0000 7ff89f788b80 1 -- start start 2024-05-09T00:47:00.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.965+0000 7ff87b7fe700 1 --2- 172.21.15.5:0/3086536139 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ff8740b2880 0x7ff8740b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:00.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.965+0000 7ff89f788b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb1f0b3110 0x55bb1f14c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:00.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.965+0000 7ff89f788b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb1efc9450 0x55bb1f14c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:00.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.965+0000 7ff89f788b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb1f146650 0x55bb1f146a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:00.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.965+0000 7ff89f788b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bb1f068fa0 con 0x55bb1f0b3110 2024-05-09T00:47:00.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.965+0000 7ff89f788b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55bb1f0077a0 con 0x55bb1f146650 2024-05-09T00:47:00.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.965+0000 7ff89f788b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55bb1f064f60 con 0x55bb1efc9450 2024-05-09T00:47:00.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.965+0000 7ff889d9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb1efc9450 0x55bb1f14c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:00.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.965+0000 7ff889d9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb1efc9450 0x55bb1f14c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:53148/0 (socket says 172.21.15.5:53148) 2024-05-09T00:47:00.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.965+0000 7ff889d9b700 1 -- 172.21.15.5:0/4164073663 learned_addr learned my addr 172.21.15.5:0/4164073663 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:47:00.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.965+0000 7ff88a59c700 1 --2- 172.21.15.5:0/4164073663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb1f0b3110 0x55bb1f14c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:00.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.966+0000 7ff87b7fe700 1 -- 172.21.15.5:0/3086536139 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7ff8800e1b90 con 0x7ff88c108f00 2024-05-09T00:47:00.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.966+0000 7ff889d9b700 1 -- 172.21.15.5:0/4164073663 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb1f146650 msgr2=0x55bb1f146a80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:47:00.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.966+0000 7ff88ad9d700 1 --2- 172.21.15.5:0/4164073663 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb1f146650 0x55bb1f146a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:00.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.966+0000 7ff889d9b700 1 --2- 172.21.15.5:0/4164073663 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb1f146650 0x55bb1f146a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:00.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.966+0000 7ff88ad9d700 1 --2- 172.21.15.5:0/3086536139 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ff8740b2880 0x7ff8740b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:00.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.966+0000 7ff889d9b700 1 -- 172.21.15.5:0/4164073663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb1f0b3110 msgr2=0x55bb1f14c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:47:00.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.966+0000 7ff889d9b700 1 --2- 172.21.15.5:0/4164073663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb1f0b3110 0x55bb1f14c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:00.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.966+0000 7ff889d9b700 1 -- 172.21.15.5:0/4164073663 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb1efff320 con 0x55bb1efc9450 2024-05-09T00:47:00.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.966+0000 7ff88a59c700 1 --2- 172.21.15.5:0/4164073663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb1f0b3110 0x55bb1f14c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:47:00.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.966+0000 7ff889d9b700 1 --2- 172.21.15.5:0/4164073663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb1efc9450 0x55bb1f14c9e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff8800a5200 tx=0x7ff88000bf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e1c58593569c419 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:00.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.966+0000 7ff878ff9700 1 -- 172.21.15.5:0/4164073663 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff880020070 con 0x55bb1efc9450 2024-05-09T00:47:00.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.966+0000 7ff88ad9d700 1 --2- 172.21.15.5:0/3086536139 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ff8740b2880 0x7ff8740b4d90 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff884009900 tx=0x7ff88400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:00.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.966+0000 7ff878ff9700 1 -- 172.21.15.5:0/4164073663 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8800a8c90 con 0x55bb1efc9450 2024-05-09T00:47:00.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.966+0000 7ff878ff9700 1 -- 172.21.15.5:0/4164073663 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8800b1fe0 con 0x55bb1efc9450 2024-05-09T00:47:00.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.967+0000 7ff89f788b80 1 -- 172.21.15.5:0/4164073663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb1efc9450 msgr2=0x55bb1f14c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:00.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.967+0000 7ff89f788b80 1 --2- 172.21.15.5:0/4164073663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb1efc9450 0x55bb1f14c9e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff8800a5200 tx=0x7ff88000bf90 comp rx=0 tx=0).stop 2024-05-09T00:47:00.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.967+0000 7ff89f788b80 1 -- 172.21.15.5:0/4164073663 shutdown_connections 2024-05-09T00:47:00.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.967+0000 7ff89f788b80 1 --2- 172.21.15.5:0/4164073663 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb1f146650 0x55bb1f146a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:00.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.967+0000 7ff89f788b80 1 --2- 172.21.15.5:0/4164073663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb1f0b3110 0x55bb1f14c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:00.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.967+0000 7ff89f788b80 1 --2- 172.21.15.5:0/4164073663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb1efc9450 0x55bb1f14c9e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:00.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.967+0000 7ff89f788b80 1 -- 172.21.15.5:0/4164073663 >> 172.21.15.5:0/4164073663 conn(0x55bb1f029650 msgr2=0x55bb1f1413b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:00.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.967+0000 7ff89f788b80 1 -- 172.21.15.5:0/4164073663 shutdown_connections 2024-05-09T00:47:00.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.967+0000 7ff89f788b80 1 -- 172.21.15.5:0/4164073663 wait complete. 2024-05-09T00:47:00.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.968+0000 7ff89f788b80 1 -- start start 2024-05-09T00:47:00.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.968+0000 7ff89f788b80 1 Processor -- start 2024-05-09T00:47:00.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.968+0000 7ff89f788b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb1efc9450 0x55bb1f150150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:00.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.968+0000 7ff89f788b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb1f0b3110 0x55bb1f150690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:00.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.968+0000 7ff89f788b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb1f146650 0x55bb1f150bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:00.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.968+0000 7ff89f788b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bb1eee4b50 con 0x55bb1efc9450 2024-05-09T00:47:00.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.968+0000 7ff89f788b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55bb1f023100 con 0x55bb1f146650 2024-05-09T00:47:00.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.968+0000 7ff89f788b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55bb1f147490 con 0x55bb1f0b3110 2024-05-09T00:47:00.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.968+0000 7ff89f788b80 10 client.0 _ll_register_callbacks cb 0x55bb1eeda670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:47:00.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.968+0000 7ff88a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb1efc9450 0x55bb1f150150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:00.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.968+0000 7ff88a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb1efc9450 0x55bb1f150150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55126/0 (socket says 172.21.15.5:55126) 2024-05-09T00:47:00.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.968+0000 7ff88a59c700 1 -- 172.21.15.5:0/910694739 learned_addr learned my addr 172.21.15.5:0/910694739 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:47:00.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.969+0000 7ff889d9b700 1 --2- 172.21.15.5:0/910694739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb1f0b3110 0x55bb1f150690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:00.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.969+0000 7ff88ad9d700 1 --2- 172.21.15.5:0/910694739 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb1f146650 0x55bb1f150bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:00.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.969+0000 7ff88a59c700 1 -- 172.21.15.5:0/910694739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb1f0b3110 msgr2=0x55bb1f150690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:47:00.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.969+0000 7ff88a59c700 1 --2- 172.21.15.5:0/910694739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb1f0b3110 0x55bb1f150690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:00.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.969+0000 7ff88a59c700 1 -- 172.21.15.5:0/910694739 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb1f146650 msgr2=0x55bb1f150bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:47:00.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.969+0000 7ff88a59c700 1 --2- 172.21.15.5:0/910694739 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb1f146650 0x55bb1f150bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:00.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.969+0000 7ff88a59c700 1 -- 172.21.15.5:0/910694739 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb1eee4440 con 0x55bb1efc9450 2024-05-09T00:47:00.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.969+0000 7ff88a59c700 1 --2- 172.21.15.5:0/910694739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb1efc9450 0x55bb1f150150 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff87c009f40 tx=0x7ff87c007580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a8b686402b8bc61 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:00.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.969+0000 7ff871ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:47:00.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.970+0000 7ff871ffb700 1 -- 172.21.15.5:0/910694739 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87c004030 con 0x55bb1efc9450 2024-05-09T00:47:00.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.970+0000 7ff871ffb700 1 -- 172.21.15.5:0/910694739 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7ff87c0041d0 con 0x55bb1efc9450 2024-05-09T00:47:00.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.970+0000 7ff89f788b80 1 -- 172.21.15.5:0/910694739 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8800a8c90 con 0x55bb1efc9450 2024-05-09T00:47:00.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.970+0000 7ff871ffb700 1 -- 172.21.15.5:0/910694739 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff87c02a050 con 0x55bb1efc9450 2024-05-09T00:47:00.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.970+0000 7ff89f788b80 10 client.15183 fetch_fsmap learned FSMap version 28 2024-05-09T00:47:00.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.970+0000 7ff89f788b80 1 -- 172.21.15.5:0/910694739 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bb1f15dac0 con 0x55bb1efc9450 2024-05-09T00:47:00.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.970+0000 7ff871ffb700 1 -- 172.21.15.5:0/910694739 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff87c0043d0 con 0x55bb1efc9450 2024-05-09T00:47:00.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.970+0000 7ff871ffb700 1 -- 172.21.15.5:0/910694739 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff87c01b640 con 0x55bb1efc9450 2024-05-09T00:47:00.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.970+0000 7ff89f788b80 10 client.15183 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T00:47:00.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.970+0000 7ff89f788b80 10 client.15183 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-09T00:47:00.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.970+0000 7ff89f788b80 20 client.15183 populate_metadata read hostname 'smithi005' 2024-05-09T00:47:00.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.971+0000 7ff89f788b80 1 --2- 172.21.15.5:0/910694739 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55bb1f160130 0x55bb1f1615a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:00.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.971+0000 7ff89f788b80 4 client.15183 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:47:00.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.971+0000 7ff89f788b80 1 -- 172.21.15.5:0/910694739 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff8800a8c90 con 0x55bb1f160130 2024-05-09T00:47:00.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.971+0000 7ff889d9b700 1 --2- 172.21.15.5:0/910694739 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55bb1f160130 0x55bb1f1615a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:00.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.971+0000 7ff871ffb700 10 client.15183 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:47:00.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.971+0000 7ff889d9b700 1 --2- 172.21.15.5:0/910694739 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55bb1f160130 0x55bb1f1615a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53c1e7b06f9da708 server_cookie=2c3aa83acd0b4113 in_seq=0 out_seq=0 2024-05-09T00:47:00.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.972+0000 7ff871ffb700 1 -- 172.21.15.5:0/910694739 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7ff8800a8c90 con 0x55bb1f160130 2024-05-09T00:47:00.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.972+0000 7ff871ffb700 10 client.15183 handle_command_reply: tid=0 2024-05-09T00:47:00.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.972+0000 7ff89f788b80 1 client.15183 shutdown 2024-05-09T00:47:00.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.972+0000 7ff89f788b80 1 -- 172.21.15.5:0/910694739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb1efc9450 msgr2=0x55bb1f150150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:00.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.972+0000 7ff89f788b80 1 --2- 172.21.15.5:0/910694739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb1efc9450 0x55bb1f150150 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff87c009f40 tx=0x7ff87c007580 comp rx=0 tx=0).stop 2024-05-09T00:47:00.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.972+0000 7ff89f788b80 1 -- 172.21.15.5:0/910694739 shutdown_connections 2024-05-09T00:47:00.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.972+0000 7ff89f788b80 1 --2- 172.21.15.5:0/910694739 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55bb1f160130 0x55bb1f1615a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:00.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.972+0000 7ff89f788b80 1 --2- 172.21.15.5:0/910694739 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb1f146650 0x55bb1f150bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:00.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.972+0000 7ff89f788b80 1 --2- 172.21.15.5:0/910694739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb1efc9450 0x55bb1f150150 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:00.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.972+0000 7ff89f788b80 1 --2- 172.21.15.5:0/910694739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb1f0b3110 0x55bb1f150690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:00.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.972+0000 7ff89f788b80 1 -- 172.21.15.5:0/910694739 >> 172.21.15.5:0/910694739 conn(0x55bb1f029650 msgr2=0x55bb1f141170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:00.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.972+0000 7ff871ffb700 0 client.15183 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:47:00.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.972+0000 7ff89f788b80 1 -- 172.21.15.5:0/910694739 shutdown_connections 2024-05-09T00:47:00.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.972+0000 7ff89f788b80 1 -- 172.21.15.5:0/910694739 wait complete. 2024-05-09T00:47:00.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.973+0000 7ff89f788b80 20 client.15183 trim_cache size 0 max 16384 2024-05-09T00:47:00.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.994+0000 7ff89f788b80 1 Processor -- start 2024-05-09T00:47:00.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.994+0000 7ff89f788b80 1 -- start start 2024-05-09T00:47:00.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.994+0000 7ff89f788b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb1efc9450 0x55bb1f14a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:00.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.994+0000 7ff89f788b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb1f0b3110 0x55bb1f15c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:00.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.994+0000 7ff89f788b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb1f146650 0x55bb1f151ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:00.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.994+0000 7ff89f788b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bb1f15f8e0 con 0x55bb1f0b3110 2024-05-09T00:47:00.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.994+0000 7ff89f788b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55bb1f15ffb0 con 0x55bb1f146650 2024-05-09T00:47:00.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.994+0000 7ff89f788b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55bb1f160680 con 0x55bb1efc9450 2024-05-09T00:47:00.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.994+0000 7ff889d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb1f0b3110 0x55bb1f15c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:00.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.994+0000 7ff889d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb1f0b3110 0x55bb1f15c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55128/0 (socket says 172.21.15.5:55128) 2024-05-09T00:47:00.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.994+0000 7ff889d9b700 1 -- 172.21.15.5:0/1774212302 learned_addr learned my addr 172.21.15.5:0/1774212302 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:47:00.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.994+0000 7ff88a59c700 1 --2- 172.21.15.5:0/1774212302 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb1efc9450 0x55bb1f14a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:00.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.995+0000 7ff889d9b700 1 -- 172.21.15.5:0/1774212302 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb1efc9450 msgr2=0x55bb1f14a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:47:00.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.995+0000 7ff889d9b700 1 --2- 172.21.15.5:0/1774212302 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb1efc9450 0x55bb1f14a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:00.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.995+0000 7ff889d9b700 1 -- 172.21.15.5:0/1774212302 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb1f146650 msgr2=0x55bb1f151ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:47:00.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.995+0000 7ff889d9b700 1 --2- 172.21.15.5:0/1774212302 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb1f146650 0x55bb1f151ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:00.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.995+0000 7ff889d9b700 1 -- 172.21.15.5:0/1774212302 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb1f143100 con 0x55bb1f0b3110 2024-05-09T00:47:00.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.995+0000 7ff889d9b700 1 --2- 172.21.15.5:0/1774212302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb1f0b3110 0x55bb1f15c260 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff8800df9d0 tx=0x7ff88000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5550eb6091d7c981 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:00.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.995+0000 7ff873fff700 1 -- 172.21.15.5:0/1774212302 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8800c3070 con 0x55bb1f0b3110 2024-05-09T00:47:00.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.995+0000 7ff873fff700 1 -- 172.21.15.5:0/1774212302 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8800a9880 con 0x55bb1f0b3110 2024-05-09T00:47:00.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.995+0000 7ff873fff700 1 -- 172.21.15.5:0/1774212302 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff880020030 con 0x55bb1f0b3110 2024-05-09T00:47:01.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.996+0000 7ff89f788b80 1 -- 172.21.15.5:0/1774212302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb1f0b3110 msgr2=0x55bb1f15c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:01.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.996+0000 7ff89f788b80 1 --2- 172.21.15.5:0/1774212302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb1f0b3110 0x55bb1f15c260 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff8800df9d0 tx=0x7ff88000b4a0 comp rx=0 tx=0).stop 2024-05-09T00:47:01.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.996+0000 7ff89f788b80 1 -- 172.21.15.5:0/1774212302 shutdown_connections 2024-05-09T00:47:01.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.996+0000 7ff89f788b80 1 --2- 172.21.15.5:0/1774212302 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb1f146650 0x55bb1f151ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:01.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.996+0000 7ff89f788b80 1 --2- 172.21.15.5:0/1774212302 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb1efc9450 0x55bb1f14a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:01.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.996+0000 7ff89f788b80 1 --2- 172.21.15.5:0/1774212302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb1f0b3110 0x55bb1f15c260 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:01.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.996+0000 7ff89f788b80 1 -- 172.21.15.5:0/1774212302 >> 172.21.15.5:0/1774212302 conn(0x55bb1f029650 msgr2=0x55bb1f1517b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:01.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.996+0000 7ff89f788b80 1 -- 172.21.15.5:0/1774212302 shutdown_connections 2024-05-09T00:47:01.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.996+0000 7ff89f788b80 1 -- 172.21.15.5:0/1774212302 wait complete. 2024-05-09T00:47:01.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.996+0000 7ff89f788b80 1 -- start start 2024-05-09T00:47:01.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.996+0000 7ff89f788b80 1 Processor -- start 2024-05-09T00:47:01.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.997+0000 7ff89f788b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb1efc9450 0x55bb1f15b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:01.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.997+0000 7ff89f788b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb1f0b3110 0x55bb1f15bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:01.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.997+0000 7ff89f788b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb1f146650 0x55bb1f15c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:01.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.997+0000 7ff89f788b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bb1f161420 con 0x55bb1f0b3110 2024-05-09T00:47:01.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.997+0000 7ff89f788b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55bb1f160d50 con 0x55bb1f146650 2024-05-09T00:47:01.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.997+0000 7ff89f788b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55bb1f161af0 con 0x55bb1efc9450 2024-05-09T00:47:01.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.997+0000 7ff89f788b80 10 client.0 _ll_register_callbacks cb 0x55bb1eeda670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:47:01.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.997+0000 7ff889d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb1f0b3110 0x55bb1f15bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:01.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.997+0000 7ff889d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb1f0b3110 0x55bb1f15bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55132/0 (socket says 172.21.15.5:55132) 2024-05-09T00:47:01.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.997+0000 7ff889d9b700 1 -- 172.21.15.5:0/3926213497 learned_addr learned my addr 172.21.15.5:0/3926213497 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:47:01.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.997+0000 7ff88a59c700 1 --2- 172.21.15.5:0/3926213497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb1efc9450 0x55bb1f15b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:01.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.997+0000 7ff88ad9d700 1 --2- 172.21.15.5:0/3926213497 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb1f146650 0x55bb1f15c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:01.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.997+0000 7ff889d9b700 1 -- 172.21.15.5:0/3926213497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb1efc9450 msgr2=0x55bb1f15b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:47:01.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.997+0000 7ff889d9b700 1 --2- 172.21.15.5:0/3926213497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb1efc9450 0x55bb1f15b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:01.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.997+0000 7ff889d9b700 1 -- 172.21.15.5:0/3926213497 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb1f146650 msgr2=0x55bb1f15c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:47:01.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.997+0000 7ff889d9b700 1 --2- 172.21.15.5:0/3926213497 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb1f146650 0x55bb1f15c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:01.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.997+0000 7ff889d9b700 1 -- 172.21.15.5:0/3926213497 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb1f167130 con 0x55bb1f0b3110 2024-05-09T00:47:01.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.997+0000 7ff88ad9d700 1 --2- 172.21.15.5:0/3926213497 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb1f146650 0x55bb1f15c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:47:01.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.997+0000 7ff889d9b700 1 --2- 172.21.15.5:0/3926213497 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb1f0b3110 0x55bb1f15bb90 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff8800a8e80 tx=0x7ff8800a9180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ca70b0c62f24ecf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:01.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.997+0000 7ff871ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:47:01.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.997+0000 7ff871ffb700 1 -- 172.21.15.5:0/3926213497 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff880020030 con 0x55bb1f0b3110 2024-05-09T00:47:01.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.997+0000 7ff871ffb700 1 -- 172.21.15.5:0/3926213497 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7ff8800a95e0 con 0x55bb1f0b3110 2024-05-09T00:47:01.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.998+0000 7ff89f788b80 1 -- 172.21.15.5:0/3926213497 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8800a9880 con 0x55bb1f0b3110 2024-05-09T00:47:01.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.998+0000 7ff871ffb700 1 -- 172.21.15.5:0/3926213497 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff8800c3050 con 0x55bb1f0b3110 2024-05-09T00:47:01.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.998+0000 7ff89f788b80 10 client.15189 fetch_fsmap learned FSMap version 28 2024-05-09T00:47:01.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.998+0000 7ff89f788b80 1 -- 172.21.15.5:0/3926213497 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bb1f1cb9b0 con 0x55bb1f0b3110 2024-05-09T00:47:01.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.998+0000 7ff871ffb700 1 -- 172.21.15.5:0/3926213497 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8800a9880 con 0x55bb1f0b3110 2024-05-09T00:47:01.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.998+0000 7ff871ffb700 1 -- 172.21.15.5:0/3926213497 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff8800a8500 con 0x55bb1f0b3110 2024-05-09T00:47:01.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.998+0000 7ff89f788b80 10 client.15189 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T00:47:01.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.998+0000 7ff89f788b80 10 client.15189 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-09T00:47:01.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.998+0000 7ff89f788b80 20 client.15189 populate_metadata read hostname 'smithi005' 2024-05-09T00:47:01.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.998+0000 7ff89f788b80 1 --2- 172.21.15.5:0/3926213497 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55bb1f1cd530 0x55bb1f1ce9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:01.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.998+0000 7ff89f788b80 4 client.15189 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T00:47:01.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.998+0000 7ff89f788b80 1 -- 172.21.15.5:0/3926213497 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "status"}) -- 0x55bb1f076980 con 0x55bb1f1cd530 2024-05-09T00:47:01.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.999+0000 7ff88a59c700 1 --2- 172.21.15.5:0/3926213497 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55bb1f1cd530 0x55bb1f1ce9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:01.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.999+0000 7ff871ffb700 10 client.15189 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:47:01.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.999+0000 7ff88a59c700 1 --2- 172.21.15.5:0/3926213497 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55bb1f1cd530 0x55bb1f1ce9a0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa48256b4404233 server_cookie=5ab48a2f9ae7e584 in_seq=0 out_seq=0 2024-05-09T00:47:01.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.999+0000 7ff871ffb700 1 -- 172.21.15.5:0/3926213497 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55bb1f076980 con 0x55bb1f1cd530 2024-05-09T00:47:01.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.999+0000 7ff871ffb700 10 client.15189 handle_command_reply: tid=0 2024-05-09T00:47:01.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.999+0000 7ff89f788b80 1 client.15189 shutdown 2024-05-09T00:47:01.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.999+0000 7ff89f788b80 1 -- 172.21.15.5:0/3926213497 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb1f0b3110 msgr2=0x55bb1f15bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:01.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.999+0000 7ff89f788b80 1 --2- 172.21.15.5:0/3926213497 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb1f0b3110 0x55bb1f15bb90 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff8800a8e80 tx=0x7ff8800a9180 comp rx=0 tx=0).stop 2024-05-09T00:47:01.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.999+0000 7ff89f788b80 1 -- 172.21.15.5:0/3926213497 shutdown_connections 2024-05-09T00:47:01.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.999+0000 7ff89f788b80 1 --2- 172.21.15.5:0/3926213497 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55bb1f1cd530 0x55bb1f1ce9a0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:01.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.999+0000 7ff89f788b80 1 --2- 172.21.15.5:0/3926213497 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb1f146650 0x55bb1f15c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:01.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.999+0000 7ff89f788b80 1 --2- 172.21.15.5:0/3926213497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb1efc9450 0x55bb1f15b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:01.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.999+0000 7ff871ffb700 0 client.15189 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:47:01.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.999+0000 7ff89f788b80 1 --2- 172.21.15.5:0/3926213497 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb1f0b3110 0x55bb1f15bb90 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:01.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.999+0000 7ff89f788b80 1 -- 172.21.15.5:0/3926213497 >> 172.21.15.5:0/3926213497 conn(0x55bb1f029650 msgr2=0x55bb1f14b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:01.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.999+0000 7ff89f788b80 1 -- 172.21.15.5:0/3926213497 shutdown_connections 2024-05-09T00:47:01.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:00.999+0000 7ff89f788b80 1 -- 172.21.15.5:0/3926213497 wait complete. 2024-05-09T00:47:01.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:01.000+0000 7ff89f788b80 20 client.15189 trim_cache size 0 max 16384 2024-05-09T00:47:01.006 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-09T00:47:01.006 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "d85814c6-0d9b-11ef-bc97-c7b262605968", 2024-05-09T00:47:01.006 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-05-09T00:47:01.006 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24508, 2024-05-09T00:47:01.006 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-05-09T00:47:01.006 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-05-09T00:47:01.006 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-05-09T00:47:01.006 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 195.11241673800001, 2024-05-09T00:47:01.006 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 28, 2024-05-09T00:47:01.007 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 82, 2024-05-09T00:47:01.007 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 78, 2024-05-09T00:47:01.007 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 220.027516303 2024-05-09T00:47:01.007 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-09T00:47:01.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:01.000+0000 7ff890d72700 1 -- 172.21.15.5:0/3086536139 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ff8740b2880 msgr2=0x7ff8740b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:01.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:01.000+0000 7ff890d72700 1 --2- 172.21.15.5:0/3086536139 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ff8740b2880 0x7ff8740b4d90 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff884009900 tx=0x7ff88400b040 comp rx=0 tx=0).stop 2024-05-09T00:47:01.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:01.000+0000 7ff890d72700 1 -- 172.21.15.5:0/3086536139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff88c108f00 msgr2=0x7ff88c19b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:01.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:01.000+0000 7ff890d72700 1 --2- 172.21.15.5:0/3086536139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff88c108f00 0x7ff88c19b330 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff88c06c170 tx=0x7ff88000dc30 comp rx=0 tx=0).stop 2024-05-09T00:47:01.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:01.000+0000 7ff890d72700 1 -- 172.21.15.5:0/3086536139 shutdown_connections 2024-05-09T00:47:01.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:01.000+0000 7ff890d72700 1 --2- 172.21.15.5:0/3086536139 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ff8740b2880 0x7ff8740b4d90 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:01.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:01.000+0000 7ff890d72700 1 --2- 172.21.15.5:0/3086536139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff88c109850 0x7ff88c19f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:01.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:01.000+0000 7ff890d72700 1 --2- 172.21.15.5:0/3086536139 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff88c1045d0 0x7ff88c19ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:01.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:01.000+0000 7ff890d72700 1 --2- 172.21.15.5:0/3086536139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff88c108f00 0x7ff88c19b330 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:01.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:01.000+0000 7ff890d72700 1 -- 172.21.15.5:0/3086536139 >> 172.21.15.5:0/3086536139 conn(0x7ff88c0ffea0 msgr2=0x7ff88c108670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:01.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:01.000+0000 7ff890d72700 1 -- 172.21.15.5:0/3086536139 shutdown_connections 2024-05-09T00:47:01.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:01.000+0000 7ff890d72700 1 -- 172.21.15.5:0/3086536139 wait complete. 2024-05-09T00:47:01.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:47:01 smithi005 ceph-mon[30365]: pgmap v391: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-09T00:47:01.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:47:01 smithi131 ceph-mon[34547]: pgmap v391: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-09T00:47:01.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:47:01 smithi157 ceph-mon[34294]: pgmap v391: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-09T00:47:01.745 INFO:teuthology.run_tasks:Running task sequential... 2024-05-09T00:47:01.757 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-09T00:47:01.757 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-09T00:47:01.758 INFO:teuthology.task.print:Enabling mgr modules 2024-05-09T00:47:01.758 INFO:teuthology.run_tasks:Running task kclient... 2024-05-09T00:47:01.800 INFO:tasks.kclient:Mounting kernel clients... 2024-05-09T00:47:01.801 INFO:tasks.kclient:config is {'client.0': {}} 2024-05-09T00:47:01.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T00:47:01.801 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-05-09T00:47:01.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T00:47:01.819 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link delete ceph-brx 2024-05-09T00:47:01.888 INFO:teuthology.orchestra.run.smithi005.stderr:Cannot find device "ceph-brx" 2024-05-09T00:47:01.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T00:47:01.889 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-05-09T00:47:01.889 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-09T00:47:01.889 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-09T00:47:01.890 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph fs ls 2024-05-09T00:47:02.182 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:47:03.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:47:03 smithi005 ceph-mon[30365]: pgmap v392: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T00:47:03.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:47:03 smithi131 ceph-mon[34547]: pgmap v392: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T00:47:03.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:47:03 smithi157 ceph-mon[34294]: pgmap v392: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T00:47:03.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:03.883+0000 7fa3cc91f700 1 -- 172.21.15.5:0/2559638291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3c81045e0 msgr2=0x7fa3c81049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:03.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:03.883+0000 7fa3cc91f700 1 --2- 172.21.15.5:0/2559638291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3c81045e0 0x7fa3c81049f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa3b80099c0 tx=0x7fa3b8009cd0 comp rx=0 tx=0).stop 2024-05-09T00:47:03.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:03.883+0000 7fa3cc91f700 1 -- 172.21.15.5:0/2559638291 shutdown_connections 2024-05-09T00:47:03.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:03.883+0000 7fa3cc91f700 1 --2- 172.21.15.5:0/2559638291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3c8109860 0x7fa3c81111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:03.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:03.883+0000 7fa3cc91f700 1 --2- 172.21.15.5:0/2559638291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3c81045e0 0x7fa3c81049f0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:03.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:03.883+0000 7fa3cc91f700 1 --2- 172.21.15.5:0/2559638291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3c8108f10 0x7fa3c8109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:03.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:03.883+0000 7fa3cc91f700 1 -- 172.21.15.5:0/2559638291 >> 172.21.15.5:0/2559638291 conn(0x7fa3c80ffe70 msgr2=0x7fa3c81022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:03.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:03.884+0000 7fa3cc91f700 1 -- 172.21.15.5:0/2559638291 shutdown_connections 2024-05-09T00:47:03.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:03.884+0000 7fa3cc91f700 1 -- 172.21.15.5:0/2559638291 wait complete. 2024-05-09T00:47:03.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:03.884+0000 7fa3cc91f700 1 Processor -- start 2024-05-09T00:47:03.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:03.885+0000 7fa3cc91f700 1 -- start start 2024-05-09T00:47:03.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:03.885+0000 7fa3cc91f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3c81045e0 0x7fa3c8075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:03.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:03.885+0000 7fa3cc91f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3c8108f10 0x7fa3c8076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:03.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:03.885+0000 7fa3cc91f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3c8109860 0x7fa3c8076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:03.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:03.885+0000 7fa3cc91f700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c8114490 con 0x7fa3c8109860 2024-05-09T00:47:03.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:03.885+0000 7fa3cc91f700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c8114310 con 0x7fa3c8108f10 2024-05-09T00:47:03.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:03.885+0000 7fa3cc91f700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c8114610 con 0x7fa3c81045e0 2024-05-09T00:47:03.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:03.885+0000 7fa3c6d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3c8109860 0x7fa3c8076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:03.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:03.885+0000 7fa3c659c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3c81045e0 0x7fa3c8075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:03.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:03.885+0000 7fa3c659c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3c81045e0 0x7fa3c8075ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:44198/0 (socket says 172.21.15.5:44198) 2024-05-09T00:47:03.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:03.885+0000 7fa3c6d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3c8109860 0x7fa3c8076540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34334/0 (socket says 172.21.15.5:34334) 2024-05-09T00:47:03.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:03.885+0000 7fa3c659c700 1 -- 172.21.15.5:0/134407949 learned_addr learned my addr 172.21.15.5:0/134407949 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:47:03.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:03.885+0000 7fa3c5d9b700 1 --2- 172.21.15.5:0/134407949 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3c8108f10 0x7fa3c8076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:03.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:03.886+0000 7fa3c6d9d700 1 -- 172.21.15.5:0/134407949 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3c81045e0 msgr2=0x7fa3c8075ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:03.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:03.886+0000 7fa3c6d9d700 1 --2- 172.21.15.5:0/134407949 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3c81045e0 0x7fa3c8075ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:03.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:03.886+0000 7fa3c6d9d700 1 -- 172.21.15.5:0/134407949 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3c8108f10 msgr2=0x7fa3c8076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:03.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:03.886+0000 7fa3c6d9d700 1 --2- 172.21.15.5:0/134407949 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3c8108f10 0x7fa3c8076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:03.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:03.886+0000 7fa3c6d9d700 1 -- 172.21.15.5:0/134407949 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3c81a91a0 con 0x7fa3c8109860 2024-05-09T00:47:03.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:03.886+0000 7fa3c5d9b700 1 --2- 172.21.15.5:0/134407949 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3c8108f10 0x7fa3c8076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:47:03.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:03.886+0000 7fa3c6d9d700 1 --2- 172.21.15.5:0/134407949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3c8109860 0x7fa3c8076540 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa3c000efc0 tx=0x7fa3c000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:03.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:03.886+0000 7fa3b77fe700 1 -- 172.21.15.5:0/134407949 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c00092e0 con 0x7fa3c8109860 2024-05-09T00:47:03.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:03.886+0000 7fa3b77fe700 1 -- 172.21.15.5:0/134407949 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3c0004bb0 con 0x7fa3c8109860 2024-05-09T00:47:03.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:03.886+0000 7fa3b77fe700 1 -- 172.21.15.5:0/134407949 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c0004080 con 0x7fa3c8109860 2024-05-09T00:47:03.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:03.886+0000 7fa3cc91f700 1 -- 172.21.15.5:0/134407949 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3c81a9430 con 0x7fa3c8109860 2024-05-09T00:47:03.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:03.886+0000 7fa3cc91f700 1 -- 172.21.15.5:0/134407949 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3c81a9920 con 0x7fa3c8109860 2024-05-09T00:47:03.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:03.887+0000 7fa3cc91f700 1 -- 172.21.15.5:0/134407949 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3c80512c0 con 0x7fa3c8109860 2024-05-09T00:47:03.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:03.890+0000 7fa3b77fe700 1 -- 172.21.15.5:0/134407949 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fa3c001b030 con 0x7fa3c8109860 2024-05-09T00:47:03.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:03.891+0000 7fa3b77fe700 1 --2- 172.21.15.5:0/134407949 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fa3b00b2900 0x7fa3b00b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:03.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:03.891+0000 7fa3c659c700 1 --2- 172.21.15.5:0/134407949 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fa3b00b2900 0x7fa3b00b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:03.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:03.891+0000 7fa3b77fe700 1 -- 172.21.15.5:0/134407949 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fa3c00e26b0 con 0x7fa3c8109860 2024-05-09T00:47:03.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:03.891+0000 7fa3c659c700 1 --2- 172.21.15.5:0/134407949 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fa3b00b2900 0x7fa3b00b4e10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8003760 tx=0x7fa3b801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:03.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:03.894+0000 7fa3b77fe700 1 -- 172.21.15.5:0/134407949 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa3c0022020 con 0x7fa3c8109860 2024-05-09T00:47:04.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:04.159+0000 7fa3cc91f700 1 -- 172.21.15.5:0/134407949 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa3c802e5f0 con 0x7fa3c8109860 2024-05-09T00:47:04.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:04.160+0000 7fa3b77fe700 1 -- 172.21.15.5:0/134407949 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v28) ==== 53+0+73 (secure 0 0 0) 0x7fa3c0010040 con 0x7fa3c8109860 2024-05-09T00:47:04.161 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-09T00:47:04.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:04.162+0000 7fa3cc91f700 1 -- 172.21.15.5:0/134407949 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fa3b00b2900 msgr2=0x7fa3b00b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:04.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:04.162+0000 7fa3cc91f700 1 --2- 172.21.15.5:0/134407949 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fa3b00b2900 0x7fa3b00b4e10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8003760 tx=0x7fa3b801a040 comp rx=0 tx=0).stop 2024-05-09T00:47:04.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:04.162+0000 7fa3cc91f700 1 -- 172.21.15.5:0/134407949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3c8109860 msgr2=0x7fa3c8076540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:04.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:04.162+0000 7fa3cc91f700 1 --2- 172.21.15.5:0/134407949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3c8109860 0x7fa3c8076540 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa3c000efc0 tx=0x7fa3c000c3a0 comp rx=0 tx=0).stop 2024-05-09T00:47:04.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:04.162+0000 7fa3cc91f700 1 -- 172.21.15.5:0/134407949 shutdown_connections 2024-05-09T00:47:04.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:04.162+0000 7fa3cc91f700 1 --2- 172.21.15.5:0/134407949 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fa3b00b2900 0x7fa3b00b4e10 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:04.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:04.162+0000 7fa3cc91f700 1 --2- 172.21.15.5:0/134407949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3c8109860 0x7fa3c8076540 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:04.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:04.162+0000 7fa3cc91f700 1 --2- 172.21.15.5:0/134407949 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3c81045e0 0x7fa3c8075ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:04.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:04.162+0000 7fa3cc91f700 1 --2- 172.21.15.5:0/134407949 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3c8108f10 0x7fa3c8076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:04.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:04.162+0000 7fa3cc91f700 1 -- 172.21.15.5:0/134407949 >> 172.21.15.5:0/134407949 conn(0x7fa3c80ffe70 msgr2=0x7fa3c8107190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:04.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:04.162+0000 7fa3cc91f700 1 -- 172.21.15.5:0/134407949 shutdown_connections 2024-05-09T00:47:04.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:04.162+0000 7fa3cc91f700 1 -- 172.21.15.5:0/134407949 wait complete. 2024-05-09T00:47:04.526 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:47:04 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/134407949' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-09T00:47:04.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:47:04 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/134407949' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-09T00:47:04.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:47:04 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/134407949' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-09T00:47:04.859 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-09T00:47:04.859 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-09T00:47:04.860 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi005.front.sepia.ceph.com 2024-05-09T00:47:04.860 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-09T00:47:04.860 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-09T00:47:04.860 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-09T00:47:04.860 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-09T00:47:04.860 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-09T00:47:04.860 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-09T00:47:04.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T00:47:04.860 DEBUG:teuthology.orchestra.run.smithi005:> ip addr 2024-05-09T00:47:04.878 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T00:47:04.878 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T00:47:04.878 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T00:47:04.879 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-09T00:47:04.879 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2024-05-09T00:47:04.879 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-09T00:47:04.879 INFO:teuthology.orchestra.run.smithi005.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T00:47:04.879 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:de brd ff:ff:ff:ff:ff:ff 2024-05-09T00:47:04.879 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f0 2024-05-09T00:47:04.879 INFO:teuthology.orchestra.run.smithi005.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T00:47:04.879 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:df brd ff:ff:ff:ff:ff:ff 2024-05-09T00:47:04.879 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f1 2024-05-09T00:47:04.879 INFO:teuthology.orchestra.run.smithi005.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-09T00:47:04.879 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:42 brd ff:ff:ff:ff:ff:ff 2024-05-09T00:47:04.879 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f0 2024-05-09T00:47:04.879 INFO:teuthology.orchestra.run.smithi005.stdout: inet 172.21.15.5/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-09T00:47:04.879 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft 42011sec preferred_lft 42011sec 2024-05-09T00:47:04.880 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ec4:7aff:febd:1442/64 scope link noprefixroute 2024-05-09T00:47:04.880 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-09T00:47:04.880 INFO:teuthology.orchestra.run.smithi005.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T00:47:04.880 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:43 brd ff:ff:ff:ff:ff:ff 2024-05-09T00:47:04.880 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f1 2024-05-09T00:47:04.880 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-09T00:47:04.880 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T00:47:04.880 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-05-09T00:47:04.880 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link add name ceph-brx type bridge 2024-05-09T00:47:04.880 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip addr flush dev ceph-brx 2024-05-09T00:47:04.881 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set ceph-brx up 2024-05-09T00:47:04.881 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-09T00:47:04.881 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-05-09T00:47:04.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T00:47:05.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T00:47:05.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T00:47:05.079 DEBUG:teuthology.orchestra.run.smithi005:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-09T00:47:05.105 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-05-09T00:47:05.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T00:47:05.107 DEBUG:teuthology.orchestra.run.smithi005:> route 2024-05-09T00:47:05.179 INFO:teuthology.orchestra.run.smithi005.stdout:Kernel IP routing table 2024-05-09T00:47:05.179 INFO:teuthology.orchestra.run.smithi005.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-09T00:47:05.179 INFO:teuthology.orchestra.run.smithi005.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-09T00:47:05.180 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-09T00:47:05.180 INFO:teuthology.orchestra.run.smithi005.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-09T00:47:05.180 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T00:47:05.180 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-05-09T00:47:05.180 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-09T00:47:05.180 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-09T00:47:05.180 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-09T00:47:05.180 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-05-09T00:47:05.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T00:47:05.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:47:05 smithi005 ceph-mon[30365]: pgmap v393: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-09T00:47:05.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T00:47:05.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T00:47:05.480 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-05-09T00:47:05.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T00:47:05.497 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list-id 2024-05-09T00:47:05.552 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T00:47:05.552 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-05-09T00:47:05.552 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T00:47:05.552 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-09T00:47:05.553 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-05-09T00:47:05.587 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:47:05 smithi157 ceph-mon[34294]: pgmap v393: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-09T00:47:05.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:47:05 smithi131 ceph-mon[34547]: pgmap v393: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-09T00:47:05.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T00:47:05.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T00:47:05.673 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-09T00:47:05.673 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T00:47:05.674 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-05-09T00:47:05.674 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-09T00:47:05.674 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-09T00:47:05.674 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-09T00:47:05.674 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-09T00:47:05.674 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-09T00:47:05.674 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-05-09T00:47:05.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T00:47:05.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T00:47:05.808 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T00:47:05.808 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-05-09T00:47:05.808 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set brx.0 up 2024-05-09T00:47:05.808 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set dev brx.0 master ceph-brx 2024-05-09T00:47:05.808 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-05-09T00:47:05.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T00:47:05.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T00:47:05.890 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T00:47:05.890 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-09T00:47:05.906 INFO:teuthology.orchestra.run.smithi005.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-09T00:47:05.906 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T00:47:05.906 DEBUG:teuthology.orchestra.run.smithi005:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-09T00:47:05.962 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-05-09T00:47:05.962 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-05-09T00:47:05.962 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/2cb5a101-4e52-403f-89d2-3ba7d2b5da5c 2024-05-09T00:47:05.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T00:47:05.962 DEBUG:teuthology.orchestra.run.smithi005:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/2cb5a101-4e52-403f-89d2-3ba7d2b5da5c /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-05-09T00:47:06.087 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-05-09T00:47:06.087 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync". 2024-05-09T00:47:06.207 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-05-09T00:47:06.207 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync". 2024-05-09T00:47:06.207 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.131:3300,172.21.15.157:3300" 2024-05-09T00:47:06.207 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: trying mount with new device syntax: 0@d85814c6-0d9b-11ef-bc97-c7b262605968.cephfs=/volumes/qa/sv_1/2cb5a101-4e52-403f-89d2-3ba7d2b5da5c 2024-05-09T00:47:06.207 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync,name=0,key=0,mon_addr=172.21.15.5:3300/172.21.15.131:3300/172.21.15.157:3300" will pass to kernel 2024-05-09T00:47:06.209 INFO:teuthology.orchestra.run.smithi005.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-09T00:47:06.209 INFO:teuthology.orchestra.run.smithi005.stdout: You just mounted a file system that supports labels which does not 2024-05-09T00:47:06.209 INFO:teuthology.orchestra.run.smithi005.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-09T00:47:06.209 INFO:teuthology.orchestra.run.smithi005.stdout: applications will generate AVC messages and not be allowed access to 2024-05-09T00:47:06.209 INFO:teuthology.orchestra.run.smithi005.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-09T00:47:06.211 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-09T00:47:06.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T00:47:06.211 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-09T00:47:06.243 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-09T00:47:06.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:06 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-09T00:47:06.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T00:47:06.345 INFO:teuthology.orchestra.run.smithi005.stdout:client15198 2024-05-09T00:47:06.346 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-09T00:47:06.346 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T00:47:06.364 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-09T00:47:06.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:06 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-09T00:47:06.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T00:47:06.465 INFO:teuthology.orchestra.run.smithi005.stdout:client15198 2024-05-09T00:47:06.466 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-09T00:47:06.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:06 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-09T00:47:06.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T00:47:06.527 INFO:teuthology.orchestra.run.smithi005.stdout:client15198 2024-05-09T00:47:06.528 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968.client15198/status') 2024-05-09T00:47:06.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T00:47:06.590 INFO:teuthology.orchestra.run.smithi005.stdout:instance: client.15198 (3)192.168.0.1:0/2438244353 2024-05-09T00:47:06.590 INFO:teuthology.orchestra.run.smithi005.stdout:blocklisted: false 2024-05-09T00:47:06.591 INFO:teuthology.orchestra.run.smithi005.stderr:0+1 records in 2024-05-09T00:47:06.591 INFO:teuthology.orchestra.run.smithi005.stderr:0+1 records out 2024-05-09T00:47:06.591 INFO:teuthology.orchestra.run.smithi005.stderr:70 bytes copied, 2.6557e-05 s, 2.6 MB/s 2024-05-09T00:47:06.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T00:47:06.598 INFO:teuthology.run_tasks:Running task exec... 2024-05-09T00:47:06.615 INFO:teuthology.task.exec:Executing custom commands... 2024-05-09T00:47:06.615 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi005.front.sepia.ceph.com 2024-05-09T00:47:06.616 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-05-09T00:47:06.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:06.733+0000 7f5988dba640 1 -- 172.21.15.5:0/3337742892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59841023c0 msgr2=0x7f598410be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:06.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:06.733+0000 7f5988dba640 1 --2- 172.21.15.5:0/3337742892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59841023c0 0x7f598410be20 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f597c009ca0 tx=0x7f597c02a8c0 comp rx=0 tx=0).stop 2024-05-09T00:47:06.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:06.733+0000 7f5988dba640 1 -- 172.21.15.5:0/3337742892 shutdown_connections 2024-05-09T00:47:06.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:06.733+0000 7f5988dba640 1 --2- 172.21.15.5:0/3337742892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59841023c0 0x7f598410be20 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:06.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:06.733+0000 7f5988dba640 1 --2- 172.21.15.5:0/3337742892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59841018c0 0x7f5984101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:06.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:06.733+0000 7f5988dba640 1 --2- 172.21.15.5:0/3337742892 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5984105210 0x7f5984105620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:06.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:06.733+0000 7f5988dba640 1 -- 172.21.15.5:0/3337742892 >> 172.21.15.5:0/3337742892 conn(0x7f59840fd4c0 msgr2=0x7f59840ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:06.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:06.733+0000 7f5988dba640 1 -- 172.21.15.5:0/3337742892 shutdown_connections 2024-05-09T00:47:06.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:06.733+0000 7f5988dba640 1 -- 172.21.15.5:0/3337742892 wait complete. 2024-05-09T00:47:06.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:06.734+0000 7f5988dba640 1 Processor -- start 2024-05-09T00:47:06.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:06.734+0000 7f5988dba640 1 -- start start 2024-05-09T00:47:06.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:06.735+0000 7f5988dba640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59841018c0 0x7f598419a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:06.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:06.735+0000 7f5988dba640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59841023c0 0x7f598419a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:06.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:06.735+0000 7f5988dba640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5984105210 0x7f59841a1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:06.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:06.735+0000 7f5988dba640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f598410f780 con 0x7f59841023c0 2024-05-09T00:47:06.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:06.735+0000 7f5988dba640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f598410f600 con 0x7f59841018c0 2024-05-09T00:47:06.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:06.735+0000 7f5988dba640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f598410f900 con 0x7f5984105210 2024-05-09T00:47:06.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:06.735+0000 7f597bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59841023c0 0x7f598419a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:06.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:06.735+0000 7f597bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59841023c0 0x7f598419a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34360/0 (socket says 172.21.15.5:34360) 2024-05-09T00:47:06.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:06.735+0000 7f597bfff640 1 -- 172.21.15.5:0/934739329 learned_addr learned my addr 172.21.15.5:0/934739329 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:47:06.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:06.735+0000 7f598259c640 1 --2- 172.21.15.5:0/934739329 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59841018c0 0x7f598419a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:06.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:06.735+0000 7f5982d9d640 1 --2- 172.21.15.5:0/934739329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5984105210 0x7f59841a1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:06.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:06.735+0000 7f597bfff640 1 -- 172.21.15.5:0/934739329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5984105210 msgr2=0x7f59841a1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:06.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:06.735+0000 7f597bfff640 1 --2- 172.21.15.5:0/934739329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5984105210 0x7f59841a1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:06.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:06.735+0000 7f597bfff640 1 -- 172.21.15.5:0/934739329 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59841018c0 msgr2=0x7f598419a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:06.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:06.735+0000 7f597bfff640 1 --2- 172.21.15.5:0/934739329 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59841018c0 0x7f598419a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:06.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:06.735+0000 7f597bfff640 1 -- 172.21.15.5:0/934739329 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59841a1e00 con 0x7f59841023c0 2024-05-09T00:47:06.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:06.735+0000 7f5982d9d640 1 --2- 172.21.15.5:0/934739329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5984105210 0x7f59841a1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:47:06.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:06.735+0000 7f597bfff640 1 --2- 172.21.15.5:0/934739329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59841023c0 0x7f598419a800 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f596c00d950 tx=0x7f596c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:06.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:06.736+0000 7f597b7fe640 1 -- 172.21.15.5:0/934739329 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f596c017070 con 0x7f59841023c0 2024-05-09T00:47:06.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:06.736+0000 7f597b7fe640 1 -- 172.21.15.5:0/934739329 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f596c009c30 con 0x7f59841023c0 2024-05-09T00:47:06.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:06.736+0000 7f5988dba640 1 -- 172.21.15.5:0/934739329 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59841a20f0 con 0x7f59841023c0 2024-05-09T00:47:06.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:06.736+0000 7f597b7fe640 1 -- 172.21.15.5:0/934739329 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f596c00f620 con 0x7f59841023c0 2024-05-09T00:47:06.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:06.736+0000 7f5988dba640 1 -- 172.21.15.5:0/934739329 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59841a2640 con 0x7f59841023c0 2024-05-09T00:47:06.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:06.737+0000 7f5988dba640 1 -- 172.21.15.5:0/934739329 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f598410abb0 con 0x7f59841023c0 2024-05-09T00:47:06.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:06.738+0000 7f597b7fe640 1 -- 172.21.15.5:0/934739329 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f596c010460 con 0x7f59841023c0 2024-05-09T00:47:06.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:06.739+0000 7f597b7fe640 1 --2- 172.21.15.5:0/934739329 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f59700b27e0 0x7f59700b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:06.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:06.740+0000 7f598259c640 1 --2- 172.21.15.5:0/934739329 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f59700b27e0 0x7f59700b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:06.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:06.740+0000 7f597b7fe640 1 -- 172.21.15.5:0/934739329 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f596c0e1e90 con 0x7f59841023c0 2024-05-09T00:47:06.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:06.740+0000 7f598259c640 1 --2- 172.21.15.5:0/934739329 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f59700b27e0 0x7f59700b4cf0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5974005fd0 tx=0x7f597400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:06.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:06.744+0000 7f597b7fe640 1 -- 172.21.15.5:0/934739329 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f596c0a9b80 con 0x7f59841023c0 2024-05-09T00:47:06.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:06.979+0000 7f5988dba640 1 -- 172.21.15.5:0/934739329 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f598402dc30 con 0x7f59841023c0 2024-05-09T00:47:07.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.229+0000 7f597b7fe640 1 -- 172.21.15.5:0/934739329 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v29) ==== 117+0+0 (secure 0 0 0) 0x7f596c009de0 con 0x7f59841023c0 2024-05-09T00:47:07.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.231+0000 7f5988dba640 1 -- 172.21.15.5:0/934739329 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f59700b27e0 msgr2=0x7f59700b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:07.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.231+0000 7f5988dba640 1 --2- 172.21.15.5:0/934739329 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f59700b27e0 0x7f59700b4cf0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5974005fd0 tx=0x7f597400b410 comp rx=0 tx=0).stop 2024-05-09T00:47:07.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.231+0000 7f5988dba640 1 -- 172.21.15.5:0/934739329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59841023c0 msgr2=0x7f598419a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:07.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.231+0000 7f5988dba640 1 --2- 172.21.15.5:0/934739329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59841023c0 0x7f598419a800 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f596c00d950 tx=0x7f596c00de10 comp rx=0 tx=0).stop 2024-05-09T00:47:07.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.231+0000 7f5988dba640 1 -- 172.21.15.5:0/934739329 shutdown_connections 2024-05-09T00:47:07.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.231+0000 7f5988dba640 1 --2- 172.21.15.5:0/934739329 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f59700b27e0 0x7f59700b4cf0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:07.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.231+0000 7f5988dba640 1 --2- 172.21.15.5:0/934739329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5984105210 0x7f59841a1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:07.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.231+0000 7f5988dba640 1 --2- 172.21.15.5:0/934739329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59841023c0 0x7f598419a800 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:07.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.231+0000 7f5988dba640 1 --2- 172.21.15.5:0/934739329 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59841018c0 0x7f598419a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:07.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.231+0000 7f5988dba640 1 -- 172.21.15.5:0/934739329 >> 172.21.15.5:0/934739329 conn(0x7f59840fd4c0 msgr2=0x7f5984107d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:07.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.232+0000 7f5988dba640 1 -- 172.21.15.5:0/934739329 shutdown_connections 2024-05-09T00:47:07.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.232+0000 7f5988dba640 1 -- 172.21.15.5:0/934739329 wait complete. 2024-05-09T00:47:07.245 INFO:teuthology.run_tasks:Running task check-counter... 2024-05-09T00:47:07.256 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-05-09T00:47:07.326 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T00:47:07.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.442+0000 7f9adcb5f640 1 -- 172.21.15.5:0/3207556990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ad8102b30 msgr2=0x7f9ad8103830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:07.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.442+0000 7f9adcb5f640 1 --2- 172.21.15.5:0/3207556990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ad8102b30 0x7f9ad8103830 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9ac8009900 tx=0x7f9ac802a410 comp rx=0 tx=0).stop 2024-05-09T00:47:07.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.442+0000 7f9adcb5f640 1 -- 172.21.15.5:0/3207556990 shutdown_connections 2024-05-09T00:47:07.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.442+0000 7f9adcb5f640 1 --2- 172.21.15.5:0/3207556990 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ad8108a70 0x7f9ad810ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:07.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.442+0000 7f9adcb5f640 1 --2- 172.21.15.5:0/3207556990 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ad8103d70 0x7f9ad8104180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:07.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.442+0000 7f9adcb5f640 1 --2- 172.21.15.5:0/3207556990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ad8102b30 0x7f9ad8103830 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:07.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.442+0000 7f9adcb5f640 1 -- 172.21.15.5:0/3207556990 >> 172.21.15.5:0/3207556990 conn(0x7f9ad80fdd20 msgr2=0x7f9ad81001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:07.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.442+0000 7f9adcb5f640 1 -- 172.21.15.5:0/3207556990 shutdown_connections 2024-05-09T00:47:07.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.442+0000 7f9adcb5f640 1 -- 172.21.15.5:0/3207556990 wait complete. 2024-05-09T00:47:07.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.443+0000 7f9adcb5f640 1 Processor -- start 2024-05-09T00:47:07.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.443+0000 7f9adcb5f640 1 -- start start 2024-05-09T00:47:07.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.443+0000 7f9adcb5f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ad8102b30 0x7f9ad819a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:07.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.443+0000 7f9adcb5f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ad8103d70 0x7f9ad819a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:07.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.443+0000 7f9adcb5f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ad8108a70 0x7f9ad81a1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:07.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.443+0000 7f9adcb5f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad810f760 con 0x7f9ad8108a70 2024-05-09T00:47:07.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.444+0000 7f9adcb5f640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad810f5e0 con 0x7f9ad8102b30 2024-05-09T00:47:07.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.444+0000 7f9adcb5f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad810f8e0 con 0x7f9ad8103d70 2024-05-09T00:47:07.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.444+0000 7f9ad6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ad8108a70 0x7f9ad81a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:07.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.444+0000 7f9ad6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ad8108a70 0x7f9ad81a1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34380/0 (socket says 172.21.15.5:34380) 2024-05-09T00:47:07.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.444+0000 7f9ad6d9d640 1 -- 172.21.15.5:0/2687210717 learned_addr learned my addr 172.21.15.5:0/2687210717 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:47:07.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.444+0000 7f9ad659c640 1 --2- 172.21.15.5:0/2687210717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ad8102b30 0x7f9ad819a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:07.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.444+0000 7f9ad5d9b640 1 --2- 172.21.15.5:0/2687210717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ad8103d70 0x7f9ad819a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:07.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.444+0000 7f9ad6d9d640 1 -- 172.21.15.5:0/2687210717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ad8103d70 msgr2=0x7f9ad819a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:07.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.444+0000 7f9ad6d9d640 1 --2- 172.21.15.5:0/2687210717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ad8103d70 0x7f9ad819a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:07.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.444+0000 7f9ad6d9d640 1 -- 172.21.15.5:0/2687210717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ad8102b30 msgr2=0x7f9ad819a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:07.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.444+0000 7f9ad6d9d640 1 --2- 172.21.15.5:0/2687210717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ad8102b30 0x7f9ad819a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:07.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.444+0000 7f9ad6d9d640 1 -- 172.21.15.5:0/2687210717 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ad81a1e80 con 0x7f9ad8108a70 2024-05-09T00:47:07.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.444+0000 7f9ad6d9d640 1 --2- 172.21.15.5:0/2687210717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ad8108a70 0x7f9ad81a1810 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9ad000cad0 tx=0x7f9ad000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:07.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.444+0000 7f9ac77fe640 1 -- 172.21.15.5:0/2687210717 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad000ae20 con 0x7f9ad8108a70 2024-05-09T00:47:07.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.444+0000 7f9ac77fe640 1 -- 172.21.15.5:0/2687210717 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ad0005020 con 0x7f9ad8108a70 2024-05-09T00:47:07.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.444+0000 7f9adcb5f640 1 -- 172.21.15.5:0/2687210717 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ad81a2170 con 0x7f9ad8108a70 2024-05-09T00:47:07.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.444+0000 7f9ac77fe640 1 -- 172.21.15.5:0/2687210717 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad000f6b0 con 0x7f9ad8108a70 2024-05-09T00:47:07.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.444+0000 7f9adcb5f640 1 -- 172.21.15.5:0/2687210717 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ad81a26c0 con 0x7f9ad8108a70 2024-05-09T00:47:07.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.445+0000 7f9adcb5f640 1 -- 172.21.15.5:0/2687210717 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ad8107880 con 0x7f9ad8108a70 2024-05-09T00:47:07.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.447+0000 7f9ac77fe640 1 -- 172.21.15.5:0/2687210717 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f9ad00051d0 con 0x7f9ad8108a70 2024-05-09T00:47:07.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.448+0000 7f9ac77fe640 1 --2- 172.21.15.5:0/2687210717 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f9ac00b27f0 0x7f9ac00b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:07.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.448+0000 7f9ad659c640 1 --2- 172.21.15.5:0/2687210717 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f9ac00b27f0 0x7f9ac00b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:07.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.448+0000 7f9ac77fe640 1 -- 172.21.15.5:0/2687210717 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f9ad00e25e0 con 0x7f9ad8108a70 2024-05-09T00:47:07.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.448+0000 7f9ad659c640 1 --2- 172.21.15.5:0/2687210717 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f9ac00b27f0 0x7f9ac00b4d00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9ac8005ab0 tx=0x7f9ac8031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:07.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.451+0000 7f9ac77fe640 1 -- 172.21.15.5:0/2687210717 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9ad0014030 con 0x7f9ad8108a70 2024-05-09T00:47:07.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:47:07 smithi005 ceph-mon[30365]: pgmap v394: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T00:47:07.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:47:07 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/934739329' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-09T00:47:07.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:47:07 smithi131 ceph-mon[34547]: pgmap v394: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T00:47:07.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:47:07 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/934739329' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-09T00:47:07.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.645+0000 7f9adcb5f640 1 -- 172.21.15.5:0/2687210717 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9ad802de00 con 0x7f9ad8108a70 2024-05-09T00:47:07.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.647+0000 7f9ac77fe640 1 -- 172.21.15.5:0/2687210717 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22745 (secure 0 0 0) 0x7f9ad00cae20 con 0x7f9ad8108a70 2024-05-09T00:47:07.648 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:47:07.648 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":82,"fsid":"d85814c6-0d9b-11ef-bc97-c7b262605968","created":"2024-05-09T00:35:25.623377+0000","modified":"2024-05-09T00:45:23.926493+0000","last_up_change":"2024-05-09T00:42:38.525335+0000","last_in_change":"2024-05-09T00:42:17.990219+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T00:38:47.844734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T00:43:39.452106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T00:43:42.708106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f8d484bb-2477-4377-ae68-5bce571333bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2963385132},{"type":"v1","addr":"172.21.15.5:6803","nonce":2963385132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2963385132},{"type":"v1","addr":"172.21.15.5:6805","nonce":2963385132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2963385132},{"type":"v1","addr":"172.21.15.5:6809","nonce":2963385132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2963385132},{"type":"v1","addr":"172.21.15.5:6807","nonce":2963385132}]},"public_addr":"172.21.15.5:6803/2963385132","cluster_addr":"172.21.15.5:6805/2963385132","heartbeat_back_addr":"172.21.15.5:6809/2963385132","heartbeat_front_addr":"172.21.15.5:6807/2963385132","state":["exists","up"]},{"osd":1,"uuid":"1c123676-7144-4a67-93a6-fd49e36a5c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3837683163},{"type":"v1","addr":"172.21.15.131:6801","nonce":3837683163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3837683163},{"type":"v1","addr":"172.21.15.131:6803","nonce":3837683163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3837683163},{"type":"v1","addr":"172.21.15.131:6807","nonce":3837683163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3837683163},{"type":"v1","addr":"172.21.15.131:6805","nonce":3837683163}]},"public_addr":"172.21.15.131:6801/3837683163","cluster_addr":"172.21.15.131:6803/3837683163","heartbeat_back_addr":"172.21.15.131:6807/3837683163","heartbeat_front_addr":"172.21.15.131:6805/3837683163","state":["exists","up"]},{"osd":2,"uuid":"95ca51f0-b48b-485c-a5f7-b15c01e754b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":484818139},{"type":"v1","addr":"172.21.15.157:6801","nonce":484818139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":484818139},{"type":"v1","addr":"172.21.15.157:6803","nonce":484818139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":484818139},{"type":"v1","addr":"172.21.15.157:6807","nonce":484818139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":484818139},{"type":"v1","addr":"172.21.15.157:6805","nonce":484818139}]},"public_addr":"172.21.15.157:6801/484818139","cluster_addr":"172.21.15.157:6803/484818139","heartbeat_back_addr":"172.21.15.157:6807/484818139","heartbeat_front_addr":"172.21.15.157:6805/484818139","state":["exists","up"]},{"osd":3,"uuid":"ed7577fb-3676-4a08-b40d-5474eb88974f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":2357377277},{"type":"v1","addr":"172.21.15.5:6811","nonce":2357377277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":2357377277},{"type":"v1","addr":"172.21.15.5:6813","nonce":2357377277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":2357377277},{"type":"v1","addr":"172.21.15.5:6817","nonce":2357377277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":2357377277},{"type":"v1","addr":"172.21.15.5:6815","nonce":2357377277}]},"public_addr":"172.21.15.5:6811/2357377277","cluster_addr":"172.21.15.5:6813/2357377277","heartbeat_back_addr":"172.21.15.5:6817/2357377277","heartbeat_front_addr":"172.21.15.5:6815/2357377277","state":["exists","up"]},{"osd":4,"uuid":"fb5dcf79-a6f1-4b1a-9e3a-087fb5132472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":2915418378},{"type":"v1","addr":"172.21.15.131:6809","nonce":2915418378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":2915418378},{"type":"v1","addr":"172.21.15.131:6811","nonce":2915418378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":2915418378},{"type":"v1","addr":"172.21.15.131:6815","nonce":2915418378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":2915418378},{"type":"v1","addr":"172.21.15.131:6813","nonce":2915418378}]},"public_addr":"172.21.15.131:6809/2915418378","cluster_addr":"172.21.15.131:6811/2915418378","heartbeat_back_addr":"172.21.15.131:6815/2915418378","heartbeat_front_addr":"172.21.15.131:6813/2915418378","state":["exists","up"]},{"osd":5,"uuid":"f01ed5d3-8066-4d25-a113-280c83e75ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4188818663},{"type":"v1","addr":"172.21.15.157:6809","nonce":4188818663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":4188818663},{"type":"v1","addr":"172.21.15.157:6811","nonce":4188818663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":4188818663},{"type":"v1","addr":"172.21.15.157:6815","nonce":4188818663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":4188818663},{"type":"v1","addr":"172.21.15.157:6813","nonce":4188818663}]},"public_addr":"172.21.15.157:6809/4188818663","cluster_addr":"172.21.15.157:6811/4188818663","heartbeat_back_addr":"172.21.15.157:6815/4188818663","heartbeat_front_addr":"172.21.15.157:6813/4188818663","state":["exists","up"]},{"osd":6,"uuid":"932379a8-c97b-4acc-8665-81f7d9cbce65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2930397813},{"type":"v1","addr":"172.21.15.5:6819","nonce":2930397813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2930397813},{"type":"v1","addr":"172.21.15.5:6821","nonce":2930397813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2930397813},{"type":"v1","addr":"172.21.15.5:6825","nonce":2930397813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2930397813},{"type":"v1","addr":"172.21.15.5:6823","nonce":2930397813}]},"public_addr":"172.21.15.5:6819/2930397813","cluster_addr":"172.21.15.5:6821/2930397813","heartbeat_back_addr":"172.21.15.5:6825/2930397813","heartbeat_front_addr":"172.21.15.5:6823/2930397813","state":["exists","up"]},{"osd":7,"uuid":"3c0a0573-6465-4bf6-b70f-310d49857a6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":4125045219},{"type":"v1","addr":"172.21.15.131:6817","nonce":4125045219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":4125045219},{"type":"v1","addr":"172.21.15.131:6819","nonce":4125045219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":4125045219},{"type":"v1","addr":"172.21.15.131:6823","nonce":4125045219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":4125045219},{"type":"v1","addr":"172.21.15.131:6821","nonce":4125045219}]},"public_addr":"172.21.15.131:6817/4125045219","cluster_addr":"172.21.15.131:6819/4125045219","heartbeat_back_addr":"172.21.15.131:6823/4125045219","heartbeat_front_addr":"172.21.15.131:6821/4125045219","state":["exists","up"]},{"osd":8,"uuid":"c49f0a00-f5ae-40ed-8f38-7b953ba1c17c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2001626768},{"type":"v1","addr":"172.21.15.157:6817","nonce":2001626768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2001626768},{"type":"v1","addr":"172.21.15.157:6819","nonce":2001626768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2001626768},{"type":"v1","addr":"172.21.15.157:6823","nonce":2001626768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2001626768},{"type":"v1","addr":"172.21.15.157:6821","nonce":2001626768}]},"public_addr":"172.21.15.157:6817/2001626768","cluster_addr":"172.21.15.157:6819/2001626768","heartbeat_back_addr":"172.21.15.157:6823/2001626768","heartbeat_front_addr":"172.21.15.157:6821/2001626768","state":["exists","up"]},{"osd":9,"uuid":"1927dfe7-c1af-4a15-bb71-d58d97c8734a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":4287220015},{"type":"v1","addr":"172.21.15.5:6827","nonce":4287220015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":4287220015},{"type":"v1","addr":"172.21.15.5:6829","nonce":4287220015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":4287220015},{"type":"v1","addr":"172.21.15.5:6833","nonce":4287220015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":4287220015},{"type":"v1","addr":"172.21.15.5:6831","nonce":4287220015}]},"public_addr":"172.21.15.5:6827/4287220015","cluster_addr":"172.21.15.5:6829/4287220015","heartbeat_back_addr":"172.21.15.5:6833/4287220015","heartbeat_front_addr":"172.21.15.5:6831/4287220015","state":["exists","up"]},{"osd":10,"uuid":"fbb4f3e3-eb7d-4ecb-a30b-55b8910375ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1571783985},{"type":"v1","addr":"172.21.15.131:6825","nonce":1571783985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1571783985},{"type":"v1","addr":"172.21.15.131:6827","nonce":1571783985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1571783985},{"type":"v1","addr":"172.21.15.131:6831","nonce":1571783985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1571783985},{"type":"v1","addr":"172.21.15.131:6829","nonce":1571783985}]},"public_addr":"172.21.15.131:6825/1571783985","cluster_addr":"172.21.15.131:6827/1571783985","heartbeat_back_addr":"172.21.15.131:6831/1571783985","heartbeat_front_addr":"172.21.15.131:6829/1571783985","state":["exists","up"]},{"osd":11,"uuid":"b858453f-eda1-4e25-89c8-94b54c830938","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3261151188},{"type":"v1","addr":"172.21.15.157:6825","nonce":3261151188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":3261151188},{"type":"v1","addr":"172.21.15.157:6827","nonce":3261151188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":3261151188},{"type":"v1","addr":"172.21.15.157:6831","nonce":3261151188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":3261151188},{"type":"v1","addr":"172.21.15.157:6829","nonce":3261151188}]},"public_addr":"172.21.15.157:6825/3261151188","cluster_addr":"172.21.15.157:6827/3261151188","heartbeat_back_addr":"172.21.15.157:6831/3261151188","heartbeat_front_addr":"172.21.15.157:6829/3261151188","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:37:57.528483+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:38:21.477482+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:38:45.780266+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:39:10.938791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:39:35.734857+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:40:00.169040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:40:26.494635+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:40:51.881718+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:41:17.697112+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:41:44.826479+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:42:10.504768+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:42:36.605447+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6},{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":6}]},{"pgid":"3.19","mappings":[{"from":3,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":7}]},{"pgid":"3.35","mappings":[{"from":1,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/3405015786":"2024-05-10T00:36:21.742363+0000","172.21.15.5:0/2669115557":"2024-05-10T00:36:21.742363+0000","172.21.15.5:6801/287842266":"2024-05-10T00:36:21.742363+0000","172.21.15.5:6800/752513054":"2024-05-10T00:35:53.128955+0000","172.21.15.5:6800/287842266":"2024-05-10T00:36:21.742363+0000","172.21.15.5:6801/752513054":"2024-05-10T00:35:53.128955+0000","172.21.15.5:0/1954876662":"2024-05-10T00:35:53.128955+0000","172.21.15.5:0/2545175121":"2024-05-10T00:36:21.742363+0000","172.21.15.5:0/3754653663":"2024-05-10T00:35:53.128955+0000","172.21.15.5:0/651514359":"2024-05-10T00:35:53.128955+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T00:47:07.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.649+0000 7f9adcb5f640 1 -- 172.21.15.5:0/2687210717 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f9ac00b27f0 msgr2=0x7f9ac00b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:07.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.649+0000 7f9adcb5f640 1 --2- 172.21.15.5:0/2687210717 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f9ac00b27f0 0x7f9ac00b4d00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9ac8005ab0 tx=0x7f9ac8031400 comp rx=0 tx=0).stop 2024-05-09T00:47:07.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.649+0000 7f9adcb5f640 1 -- 172.21.15.5:0/2687210717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ad8108a70 msgr2=0x7f9ad81a1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:07.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.649+0000 7f9adcb5f640 1 --2- 172.21.15.5:0/2687210717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ad8108a70 0x7f9ad81a1810 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9ad000cad0 tx=0x7f9ad000cf90 comp rx=0 tx=0).stop 2024-05-09T00:47:07.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.649+0000 7f9adcb5f640 1 -- 172.21.15.5:0/2687210717 shutdown_connections 2024-05-09T00:47:07.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.649+0000 7f9adcb5f640 1 --2- 172.21.15.5:0/2687210717 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f9ac00b27f0 0x7f9ac00b4d00 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:07.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.649+0000 7f9adcb5f640 1 --2- 172.21.15.5:0/2687210717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ad8108a70 0x7f9ad81a1810 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:07.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.649+0000 7f9adcb5f640 1 --2- 172.21.15.5:0/2687210717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ad8103d70 0x7f9ad819a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:07.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.649+0000 7f9adcb5f640 1 --2- 172.21.15.5:0/2687210717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ad8102b30 0x7f9ad819a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:07.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.649+0000 7f9adcb5f640 1 -- 172.21.15.5:0/2687210717 >> 172.21.15.5:0/2687210717 conn(0x7f9ad80fdd20 msgr2=0x7f9ad8104a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:07.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.649+0000 7f9adcb5f640 1 -- 172.21.15.5:0/2687210717 shutdown_connections 2024-05-09T00:47:07.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.649+0000 7f9adcb5f640 1 -- 172.21.15.5:0/2687210717 wait complete. 2024-05-09T00:47:07.661 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T00:38:47.844734+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T00:43:39.452106+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T00:43:42.708106+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T00:47:07.662 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T00:47:07.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:47:07 smithi157 ceph-mon[34294]: pgmap v394: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T00:47:07.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:47:07 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/934739329' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-09T00:47:07.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.780+0000 7fcb3a661640 1 -- 172.21.15.5:0/279569924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb34102b10 msgr2=0x7fcb34103810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:07.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.780+0000 7fcb3a661640 1 --2- 172.21.15.5:0/279569924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb34102b10 0x7fcb34103810 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fcb24009900 tx=0x7fcb2402a410 comp rx=0 tx=0).stop 2024-05-09T00:47:07.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.780+0000 7fcb3a661640 1 -- 172.21.15.5:0/279569924 shutdown_connections 2024-05-09T00:47:07.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.780+0000 7fcb3a661640 1 --2- 172.21.15.5:0/279569924 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb34108a50 0x7fcb3410ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:07.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.780+0000 7fcb3a661640 1 --2- 172.21.15.5:0/279569924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb34103d50 0x7fcb34104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:07.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.780+0000 7fcb3a661640 1 --2- 172.21.15.5:0/279569924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb34102b10 0x7fcb34103810 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:07.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.780+0000 7fcb3a661640 1 -- 172.21.15.5:0/279569924 >> 172.21.15.5:0/279569924 conn(0x7fcb340fdd00 msgr2=0x7fcb341001b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:07.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.780+0000 7fcb3a661640 1 -- 172.21.15.5:0/279569924 shutdown_connections 2024-05-09T00:47:07.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.780+0000 7fcb3a661640 1 -- 172.21.15.5:0/279569924 wait complete. 2024-05-09T00:47:07.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.781+0000 7fcb3a661640 1 Processor -- start 2024-05-09T00:47:07.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.781+0000 7fcb3a661640 1 -- start start 2024-05-09T00:47:07.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.781+0000 7fcb3a661640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb34102b10 0x7fcb3406aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:07.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.781+0000 7fcb3a661640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb34103d50 0x7fcb3419f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:07.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.781+0000 7fcb3a661640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb34108a50 0x7fcb341a1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:07.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.781+0000 7fcb3a661640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3410e5f0 con 0x7fcb34102b10 2024-05-09T00:47:07.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.781+0000 7fcb3a661640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3410e470 con 0x7fcb34108a50 2024-05-09T00:47:07.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.781+0000 7fcb3a661640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3410e770 con 0x7fcb34103d50 2024-05-09T00:47:07.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.781+0000 7fcb33fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb34102b10 0x7fcb3406aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:07.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.781+0000 7fcb33fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb34102b10 0x7fcb3406aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34394/0 (socket says 172.21.15.5:34394) 2024-05-09T00:47:07.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.781+0000 7fcb33fff640 1 -- 172.21.15.5:0/3466615826 learned_addr learned my addr 172.21.15.5:0/3466615826 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:47:07.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.781+0000 7fcb38bfe640 1 --2- 172.21.15.5:0/3466615826 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb34108a50 0x7fcb341a1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:07.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.781+0000 7fcb33fff640 1 -- 172.21.15.5:0/3466615826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb34103d50 msgr2=0x7fcb3419f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:07.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.781+0000 7fcb33fff640 1 --2- 172.21.15.5:0/3466615826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb34103d50 0x7fcb3419f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:07.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.782+0000 7fcb33fff640 1 -- 172.21.15.5:0/3466615826 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb34108a50 msgr2=0x7fcb341a1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:07.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.782+0000 7fcb33fff640 1 --2- 172.21.15.5:0/3466615826 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb34108a50 0x7fcb341a1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:07.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.782+0000 7fcb33fff640 1 -- 172.21.15.5:0/3466615826 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb341a2170 con 0x7fcb34102b10 2024-05-09T00:47:07.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.782+0000 7fcb38bfe640 1 --2- 172.21.15.5:0/3466615826 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb34108a50 0x7fcb341a1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:47:07.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.782+0000 7fcb33fff640 1 --2- 172.21.15.5:0/3466615826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb34102b10 0x7fcb3406aa30 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fcb24006010 tx=0x7fcb24004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:07.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.782+0000 7fcb31ffb640 1 -- 172.21.15.5:0/3466615826 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb2403f030 con 0x7fcb34102b10 2024-05-09T00:47:07.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.782+0000 7fcb3a661640 1 -- 172.21.15.5:0/3466615826 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb341a2400 con 0x7fcb34102b10 2024-05-09T00:47:07.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.782+0000 7fcb31ffb640 1 -- 172.21.15.5:0/3466615826 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb24003aa0 con 0x7fcb34102b10 2024-05-09T00:47:07.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.782+0000 7fcb31ffb640 1 -- 172.21.15.5:0/3466615826 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb24035070 con 0x7fcb34102b10 2024-05-09T00:47:07.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.782+0000 7fcb3a661640 1 -- 172.21.15.5:0/3466615826 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb341a2940 con 0x7fcb34102b10 2024-05-09T00:47:07.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.783+0000 7fcb3a661640 1 -- 172.21.15.5:0/3466615826 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb34107860 con 0x7fcb34102b10 2024-05-09T00:47:07.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.785+0000 7fcb31ffb640 1 -- 172.21.15.5:0/3466615826 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fcb24042030 con 0x7fcb34102b10 2024-05-09T00:47:07.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.785+0000 7fcb31ffb640 1 --2- 172.21.15.5:0/3466615826 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fcb200b27f0 0x7fcb200b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:07.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.785+0000 7fcb31ffb640 1 -- 172.21.15.5:0/3466615826 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fcb24102e20 con 0x7fcb34102b10 2024-05-09T00:47:07.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.785+0000 7fcb2bfff640 1 --2- 172.21.15.5:0/3466615826 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fcb200b27f0 0x7fcb200b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:07.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.786+0000 7fcb2bfff640 1 --2- 172.21.15.5:0/3466615826 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fcb200b27f0 0x7fcb200b4d00 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c005fd0 tx=0x7fcb1c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:07.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.789+0000 7fcb31ffb640 1 -- 172.21.15.5:0/3466615826 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcb240cab10 con 0x7fcb34102b10 2024-05-09T00:47:07.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.980+0000 7fcb3a661640 1 -- 172.21.15.5:0/3466615826 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcb3402dd10 con 0x7fcb34102b10 2024-05-09T00:47:07.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.981+0000 7fcb31ffb640 1 -- 172.21.15.5:0/3466615826 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7fcb240ca410 con 0x7fcb34102b10 2024-05-09T00:47:07.982 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-09T00:47:07.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.983+0000 7fcb3a661640 1 -- 172.21.15.5:0/3466615826 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fcb200b27f0 msgr2=0x7fcb200b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:07.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.983+0000 7fcb3a661640 1 --2- 172.21.15.5:0/3466615826 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fcb200b27f0 0x7fcb200b4d00 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c005fd0 tx=0x7fcb1c005850 comp rx=0 tx=0).stop 2024-05-09T00:47:07.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.983+0000 7fcb3a661640 1 -- 172.21.15.5:0/3466615826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb34102b10 msgr2=0x7fcb3406aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:07.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.983+0000 7fcb3a661640 1 --2- 172.21.15.5:0/3466615826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb34102b10 0x7fcb3406aa30 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fcb24006010 tx=0x7fcb24004270 comp rx=0 tx=0).stop 2024-05-09T00:47:07.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.983+0000 7fcb3a661640 1 -- 172.21.15.5:0/3466615826 shutdown_connections 2024-05-09T00:47:07.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.983+0000 7fcb3a661640 1 --2- 172.21.15.5:0/3466615826 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fcb200b27f0 0x7fcb200b4d00 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:07.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.983+0000 7fcb3a661640 1 --2- 172.21.15.5:0/3466615826 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb34108a50 0x7fcb341a1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:07.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.983+0000 7fcb3a661640 1 --2- 172.21.15.5:0/3466615826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb34103d50 0x7fcb3419f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:07.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.983+0000 7fcb3a661640 1 --2- 172.21.15.5:0/3466615826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb34102b10 0x7fcb3406aa30 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:07.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.983+0000 7fcb3a661640 1 -- 172.21.15.5:0/3466615826 >> 172.21.15.5:0/3466615826 conn(0x7fcb340fdd00 msgr2=0x7fcb34104a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:07.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.984+0000 7fcb3a661640 1 -- 172.21.15.5:0/3466615826 shutdown_connections 2024-05-09T00:47:07.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:07.984+0000 7fcb3a661640 1 -- 172.21.15.5:0/3466615826 wait complete. 2024-05-09T00:47:07.996 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T00:47:08.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.114+0000 7fd7796d8640 1 -- 172.21.15.5:0/2377329658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd774102380 msgr2=0x7fd77410bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:08.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.114+0000 7fd7796d8640 1 --2- 172.21.15.5:0/2377329658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd774102380 0x7fd77410bde0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd76c009900 tx=0x7fd76c02a410 comp rx=0 tx=0).stop 2024-05-09T00:47:08.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.114+0000 7fd7796d8640 1 -- 172.21.15.5:0/2377329658 shutdown_connections 2024-05-09T00:47:08.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.114+0000 7fd7796d8640 1 --2- 172.21.15.5:0/2377329658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd774102380 0x7fd77410bde0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:08.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.114+0000 7fd7796d8640 1 --2- 172.21.15.5:0/2377329658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd774101880 0x7fd774101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:08.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.114+0000 7fd7796d8640 1 --2- 172.21.15.5:0/2377329658 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7741051d0 0x7fd7741055e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:08.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.114+0000 7fd7796d8640 1 -- 172.21.15.5:0/2377329658 >> 172.21.15.5:0/2377329658 conn(0x7fd7740fd480 msgr2=0x7fd7740ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:08.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.114+0000 7fd7796d8640 1 -- 172.21.15.5:0/2377329658 shutdown_connections 2024-05-09T00:47:08.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.114+0000 7fd7796d8640 1 -- 172.21.15.5:0/2377329658 wait complete. 2024-05-09T00:47:08.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.114+0000 7fd7796d8640 1 Processor -- start 2024-05-09T00:47:08.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.114+0000 7fd7796d8640 1 -- start start 2024-05-09T00:47:08.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.115+0000 7fd7796d8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd774101880 0x7fd774104380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:08.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.115+0000 7fd7796d8640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd774102380 0x7fd774102a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:08.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.115+0000 7fd7796d8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7741051d0 0x7fd774102f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:08.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.115+0000 7fd7796d8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd77410f740 con 0x7fd7741051d0 2024-05-09T00:47:08.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.115+0000 7fd7796d8640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd77410f5c0 con 0x7fd774102380 2024-05-09T00:47:08.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.115+0000 7fd7796d8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd77410f8c0 con 0x7fd774101880 2024-05-09T00:47:08.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.115+0000 7fd7737fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7741051d0 0x7fd774102f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:08.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.115+0000 7fd7737fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7741051d0 0x7fd774102f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34414/0 (socket says 172.21.15.5:34414) 2024-05-09T00:47:08.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.115+0000 7fd7737fe640 1 -- 172.21.15.5:0/4096670581 learned_addr learned my addr 172.21.15.5:0/4096670581 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:47:08.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.115+0000 7fd7727fc640 1 --2- 172.21.15.5:0/4096670581 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd774102380 0x7fd774102a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:08.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.115+0000 7fd772ffd640 1 --2- 172.21.15.5:0/4096670581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd774101880 0x7fd774104380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:08.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.115+0000 7fd7737fe640 1 -- 172.21.15.5:0/4096670581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd774101880 msgr2=0x7fd774104380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:08.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.115+0000 7fd7737fe640 1 --2- 172.21.15.5:0/4096670581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd774101880 0x7fd774104380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:08.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.115+0000 7fd7737fe640 1 -- 172.21.15.5:0/4096670581 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd774102380 msgr2=0x7fd774102a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:08.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.115+0000 7fd7737fe640 1 --2- 172.21.15.5:0/4096670581 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd774102380 0x7fd774102a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:08.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.115+0000 7fd7737fe640 1 -- 172.21.15.5:0/4096670581 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd774103690 con 0x7fd7741051d0 2024-05-09T00:47:08.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.115+0000 7fd7737fe640 1 --2- 172.21.15.5:0/4096670581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7741051d0 0x7fd774102f60 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fd76c009230 tx=0x7fd76c031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:08.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.115+0000 7fd75bfff640 1 -- 172.21.15.5:0/4096670581 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76c03f030 con 0x7fd7741051d0 2024-05-09T00:47:08.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.116+0000 7fd75bfff640 1 -- 172.21.15.5:0/4096670581 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd76c003aa0 con 0x7fd7741051d0 2024-05-09T00:47:08.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.116+0000 7fd75bfff640 1 -- 172.21.15.5:0/4096670581 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76c035070 con 0x7fd7741051d0 2024-05-09T00:47:08.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.116+0000 7fd7796d8640 1 -- 172.21.15.5:0/4096670581 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7741a4370 con 0x7fd7741051d0 2024-05-09T00:47:08.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.116+0000 7fd7796d8640 1 -- 172.21.15.5:0/4096670581 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7741a4720 con 0x7fd7741051d0 2024-05-09T00:47:08.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.116+0000 7fd7796d8640 1 -- 172.21.15.5:0/4096670581 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd77406a290 con 0x7fd7741051d0 2024-05-09T00:47:08.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.118+0000 7fd75bfff640 1 -- 172.21.15.5:0/4096670581 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fd76c042030 con 0x7fd7741051d0 2024-05-09T00:47:08.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.119+0000 7fd75bfff640 1 --2- 172.21.15.5:0/4096670581 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fd75c0b27f0 0x7fd75c0b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:08.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.119+0000 7fd75bfff640 1 -- 172.21.15.5:0/4096670581 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fd76c102d40 con 0x7fd7741051d0 2024-05-09T00:47:08.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.119+0000 7fd772ffd640 1 --2- 172.21.15.5:0/4096670581 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fd75c0b27f0 0x7fd75c0b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:08.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.119+0000 7fd772ffd640 1 --2- 172.21.15.5:0/4096670581 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fd75c0b27f0 0x7fd75c0b4d00 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd77406b420 tx=0x7fd764009870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:08.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.122+0000 7fd75bfff640 1 -- 172.21.15.5:0/4096670581 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd76c0caa30 con 0x7fd7741051d0 2024-05-09T00:47:08.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:47:08 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/934739329' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-09T00:47:08.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:47:08 smithi005 ceph-mon[30365]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T00:47:08.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:47:08 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/2687210717' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:47:08.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:47:08 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/3466615826' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T00:47:08.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.317+0000 7fd7796d8640 1 -- 172.21.15.5:0/4096670581 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd77402dc30 con 0x7fd7741051d0 2024-05-09T00:47:08.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.317+0000 7fd75bfff640 1 -- 172.21.15.5:0/4096670581 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) ==== 104+0+11 (secure 0 0 0) 0x7fd76c0ca330 con 0x7fd7741051d0 2024-05-09T00:47:08.318 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-09T00:47:08.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.320+0000 7fd7796d8640 1 -- 172.21.15.5:0/4096670581 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fd75c0b27f0 msgr2=0x7fd75c0b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:08.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.320+0000 7fd7796d8640 1 --2- 172.21.15.5:0/4096670581 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fd75c0b27f0 0x7fd75c0b4d00 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd77406b420 tx=0x7fd764009870 comp rx=0 tx=0).stop 2024-05-09T00:47:08.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.320+0000 7fd7796d8640 1 -- 172.21.15.5:0/4096670581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7741051d0 msgr2=0x7fd774102f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:08.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.320+0000 7fd7796d8640 1 --2- 172.21.15.5:0/4096670581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7741051d0 0x7fd774102f60 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fd76c009230 tx=0x7fd76c031a50 comp rx=0 tx=0).stop 2024-05-09T00:47:08.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.320+0000 7fd7796d8640 1 -- 172.21.15.5:0/4096670581 shutdown_connections 2024-05-09T00:47:08.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.320+0000 7fd7796d8640 1 --2- 172.21.15.5:0/4096670581 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fd75c0b27f0 0x7fd75c0b4d00 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:08.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.320+0000 7fd7796d8640 1 --2- 172.21.15.5:0/4096670581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7741051d0 0x7fd774102f60 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:08.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.320+0000 7fd7796d8640 1 --2- 172.21.15.5:0/4096670581 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd774102380 0x7fd774102a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:08.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.320+0000 7fd7796d8640 1 --2- 172.21.15.5:0/4096670581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd774101880 0x7fd774104380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:08.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.320+0000 7fd7796d8640 1 -- 172.21.15.5:0/4096670581 >> 172.21.15.5:0/4096670581 conn(0x7fd7740fd480 msgr2=0x7fd774107d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:08.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.320+0000 7fd7796d8640 1 -- 172.21.15.5:0/4096670581 shutdown_connections 2024-05-09T00:47:08.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.320+0000 7fd7796d8640 1 -- 172.21.15.5:0/4096670581 wait complete. 2024-05-09T00:47:08.332 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T00:47:08.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.450+0000 7fa0c416b640 1 -- 172.21.15.5:0/2216606190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0bc103d60 msgr2=0x7fa0bc104170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:08.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.450+0000 7fa0c416b640 1 --2- 172.21.15.5:0/2216606190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0bc103d60 0x7fa0bc104170 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac009900 tx=0x7fa0ac02a410 comp rx=0 tx=0).stop 2024-05-09T00:47:08.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.450+0000 7fa0c416b640 1 -- 172.21.15.5:0/2216606190 shutdown_connections 2024-05-09T00:47:08.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.450+0000 7fa0c416b640 1 --2- 172.21.15.5:0/2216606190 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0bc108a60 0x7fa0bc10ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:08.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.450+0000 7fa0c416b640 1 --2- 172.21.15.5:0/2216606190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0bc103d60 0x7fa0bc104170 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:08.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.450+0000 7fa0c416b640 1 --2- 172.21.15.5:0/2216606190 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0bc102b20 0x7fa0bc103820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:08.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.450+0000 7fa0c416b640 1 -- 172.21.15.5:0/2216606190 >> 172.21.15.5:0/2216606190 conn(0x7fa0bc0fdcf0 msgr2=0x7fa0bc1001c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:08.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.450+0000 7fa0c416b640 1 -- 172.21.15.5:0/2216606190 shutdown_connections 2024-05-09T00:47:08.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.450+0000 7fa0c416b640 1 -- 172.21.15.5:0/2216606190 wait complete. 2024-05-09T00:47:08.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.451+0000 7fa0c416b640 1 Processor -- start 2024-05-09T00:47:08.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.451+0000 7fa0c416b640 1 -- start start 2024-05-09T00:47:08.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.451+0000 7fa0c416b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0bc102b20 0x7fa0bc1992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:08.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.451+0000 7fa0c416b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0bc103d60 0x7fa0bc1997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:08.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.451+0000 7fa0c416b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0bc108a60 0x7fa0bc1a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:08.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.451+0000 7fa0c416b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa0bc10e640 con 0x7fa0bc103d60 2024-05-09T00:47:08.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.451+0000 7fa0c416b640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa0bc10e4c0 con 0x7fa0bc102b20 2024-05-09T00:47:08.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.451+0000 7fa0c416b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa0bc10e7c0 con 0x7fa0bc108a60 2024-05-09T00:47:08.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.451+0000 7fa0c1706640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0bc103d60 0x7fa0bc1997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:08.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.451+0000 7fa0c1706640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0bc103d60 0x7fa0bc1997e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34424/0 (socket says 172.21.15.5:34424) 2024-05-09T00:47:08.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.451+0000 7fa0c1706640 1 -- 172.21.15.5:0/3042070078 learned_addr learned my addr 172.21.15.5:0/3042070078 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:47:08.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.451+0000 7fa0c2708640 1 --2- 172.21.15.5:0/3042070078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0bc108a60 0x7fa0bc1a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:08.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.451+0000 7fa0c1706640 1 -- 172.21.15.5:0/3042070078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0bc108a60 msgr2=0x7fa0bc1a0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:08.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.451+0000 7fa0c1f07640 1 --2- 172.21.15.5:0/3042070078 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0bc102b20 0x7fa0bc1992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:08.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.451+0000 7fa0c1706640 1 --2- 172.21.15.5:0/3042070078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0bc108a60 0x7fa0bc1a0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:08.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.451+0000 7fa0c1706640 1 -- 172.21.15.5:0/3042070078 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0bc102b20 msgr2=0x7fa0bc1992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:08.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.451+0000 7fa0c1706640 1 --2- 172.21.15.5:0/3042070078 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0bc102b20 0x7fa0bc1992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:08.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.452+0000 7fa0c1706640 1 -- 172.21.15.5:0/3042070078 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0bc1a0de0 con 0x7fa0bc103d60 2024-05-09T00:47:08.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.452+0000 7fa0c1706640 1 --2- 172.21.15.5:0/3042070078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0bc103d60 0x7fa0bc1997e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac006010 tx=0x7fa0ac031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:08.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.452+0000 7fa0b2ffd640 1 -- 172.21.15.5:0/3042070078 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac03f030 con 0x7fa0bc103d60 2024-05-09T00:47:08.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.452+0000 7fa0c416b640 1 -- 172.21.15.5:0/3042070078 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0bc1a1070 con 0x7fa0bc103d60 2024-05-09T00:47:08.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.452+0000 7fa0b2ffd640 1 -- 172.21.15.5:0/3042070078 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0ac003aa0 con 0x7fa0bc103d60 2024-05-09T00:47:08.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.452+0000 7fa0b2ffd640 1 -- 172.21.15.5:0/3042070078 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac035070 con 0x7fa0bc103d60 2024-05-09T00:47:08.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.452+0000 7fa0c416b640 1 -- 172.21.15.5:0/3042070078 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0bc1a1560 con 0x7fa0bc103d60 2024-05-09T00:47:08.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.453+0000 7fa0c416b640 1 -- 172.21.15.5:0/3042070078 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0bc107870 con 0x7fa0bc103d60 2024-05-09T00:47:08.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.455+0000 7fa0b2ffd640 1 -- 172.21.15.5:0/3042070078 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fa0ac042030 con 0x7fa0bc103d60 2024-05-09T00:47:08.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.456+0000 7fa0b2ffd640 1 --2- 172.21.15.5:0/3042070078 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fa0a80b27f0 0x7fa0a80b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:08.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.456+0000 7fa0c1f07640 1 --2- 172.21.15.5:0/3042070078 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fa0a80b27f0 0x7fa0a80b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:08.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.456+0000 7fa0b2ffd640 1 -- 172.21.15.5:0/3042070078 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fa0ac102fc0 con 0x7fa0bc103d60 2024-05-09T00:47:08.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.457+0000 7fa0c1f07640 1 --2- 172.21.15.5:0/3042070078 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fa0a80b27f0 0x7fa0a80b4d00 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc103720 tx=0x7fa0b8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:08.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.460+0000 7fa0b2ffd640 1 -- 172.21.15.5:0/3042070078 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa0ac0cac50 con 0x7fa0bc103d60 2024-05-09T00:47:08.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:47:08 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/934739329' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-09T00:47:08.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:47:08 smithi131 ceph-mon[34547]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T00:47:08.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:47:08 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/2687210717' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:47:08.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:47:08 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/3466615826' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T00:47:08.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.653+0000 7fa0c416b640 1 -- 172.21.15.5:0/3042070078 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa0bc0638d0 con 0x7fa0bc103d60 2024-05-09T00:47:08.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.654+0000 7fa0b2ffd640 1 -- 172.21.15.5:0/3042070078 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) ==== 100+0+11 (secure 0 0 0) 0x7fa0ac0ca550 con 0x7fa0bc103d60 2024-05-09T00:47:08.654 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-05-09T00:47:08.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.656+0000 7fa0c416b640 1 -- 172.21.15.5:0/3042070078 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fa0a80b27f0 msgr2=0x7fa0a80b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:08.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.656+0000 7fa0c416b640 1 --2- 172.21.15.5:0/3042070078 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fa0a80b27f0 0x7fa0a80b4d00 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc103720 tx=0x7fa0b8005f90 comp rx=0 tx=0).stop 2024-05-09T00:47:08.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.656+0000 7fa0c416b640 1 -- 172.21.15.5:0/3042070078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0bc103d60 msgr2=0x7fa0bc1997e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:08.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.656+0000 7fa0c416b640 1 --2- 172.21.15.5:0/3042070078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0bc103d60 0x7fa0bc1997e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac006010 tx=0x7fa0ac031a50 comp rx=0 tx=0).stop 2024-05-09T00:47:08.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.656+0000 7fa0c416b640 1 -- 172.21.15.5:0/3042070078 shutdown_connections 2024-05-09T00:47:08.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.656+0000 7fa0c416b640 1 --2- 172.21.15.5:0/3042070078 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fa0a80b27f0 0x7fa0a80b4d00 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:08.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.656+0000 7fa0c416b640 1 --2- 172.21.15.5:0/3042070078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0bc108a60 0x7fa0bc1a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:08.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.656+0000 7fa0c416b640 1 --2- 172.21.15.5:0/3042070078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0bc103d60 0x7fa0bc1997e0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:08.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.656+0000 7fa0c416b640 1 --2- 172.21.15.5:0/3042070078 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0bc102b20 0x7fa0bc1992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:08.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.656+0000 7fa0c416b640 1 -- 172.21.15.5:0/3042070078 >> 172.21.15.5:0/3042070078 conn(0x7fa0bc0fdcf0 msgr2=0x7fa0bc104a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:08.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.656+0000 7fa0c416b640 1 -- 172.21.15.5:0/3042070078 shutdown_connections 2024-05-09T00:47:08.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:08.656+0000 7fa0c416b640 1 -- 172.21.15.5:0/3042070078 wait complete. 2024-05-09T00:47:08.667 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-05-09T00:47:08.668 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T00:47:08.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:47:08 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/934739329' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-09T00:47:08.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:47:08 smithi157 ceph-mon[34294]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T00:47:08.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:47:08 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/2687210717' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:47:08.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:47:08 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/3466615826' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T00:47:08.921 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:47:09.326 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:47:09 smithi005 ceph-mon[30365]: pgmap v395: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-05-09T00:47:09.327 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:47:09 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/4096670581' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T00:47:09.327 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:47:09 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/3042070078' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T00:47:09.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:47:09 smithi131 ceph-mon[34547]: pgmap v395: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-05-09T00:47:09.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:47:09 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/4096670581' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T00:47:09.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:47:09 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/3042070078' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T00:47:09.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:47:09 smithi157 ceph-mon[34294]: pgmap v395: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-05-09T00:47:09.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:47:09 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/4096670581' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T00:47:09.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:47:09 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/3042070078' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T00:47:10.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:10.668+0000 7fda2cda2700 1 -- 172.21.15.5:0/2287229631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda28108740 msgr2=0x7fda281100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:10.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:10.668+0000 7fda2cda2700 1 --2- 172.21.15.5:0/2287229631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda28108740 0x7fda281100a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fda2000b0d0 tx=0x7fda2000b3e0 comp rx=0 tx=0).stop 2024-05-09T00:47:10.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:10.668+0000 7fda2cda2700 1 -- 172.21.15.5:0/2287229631 shutdown_connections 2024-05-09T00:47:10.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:10.668+0000 7fda2cda2700 1 --2- 172.21.15.5:0/2287229631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda28108740 0x7fda281100a0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:10.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:10.668+0000 7fda2cda2700 1 --2- 172.21.15.5:0/2287229631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda281034c0 0x7fda281038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:10.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:10.668+0000 7fda2cda2700 1 --2- 172.21.15.5:0/2287229631 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda28107df0 0x7fda28108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:10.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:10.668+0000 7fda2cda2700 1 -- 172.21.15.5:0/2287229631 >> 172.21.15.5:0/2287229631 conn(0x7fda280fedc0 msgr2=0x7fda281011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:10.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:10.668+0000 7fda2cda2700 1 -- 172.21.15.5:0/2287229631 shutdown_connections 2024-05-09T00:47:10.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:10.668+0000 7fda2cda2700 1 -- 172.21.15.5:0/2287229631 wait complete. 2024-05-09T00:47:10.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:10.669+0000 7fda2cda2700 1 Processor -- start 2024-05-09T00:47:10.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:10.669+0000 7fda2cda2700 1 -- start start 2024-05-09T00:47:10.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:10.669+0000 7fda2cda2700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda281034c0 0x7fda2819f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:10.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:10.669+0000 7fda2cda2700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda28107df0 0x7fda2819fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:10.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:10.669+0000 7fda2cda2700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda28108740 0x7fda28199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:10.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:10.669+0000 7fda2cda2700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fda281133c0 con 0x7fda28108740 2024-05-09T00:47:10.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:10.669+0000 7fda2cda2700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fda28113240 con 0x7fda28107df0 2024-05-09T00:47:10.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:10.669+0000 7fda2cda2700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fda28113540 con 0x7fda281034c0 2024-05-09T00:47:10.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:10.669+0000 7fda26d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda28108740 0x7fda28199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:10.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:10.670+0000 7fda26d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda28108740 0x7fda28199b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34448/0 (socket says 172.21.15.5:34448) 2024-05-09T00:47:10.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:10.670+0000 7fda26d9d700 1 -- 172.21.15.5:0/4196448079 learned_addr learned my addr 172.21.15.5:0/4196448079 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:47:10.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:10.670+0000 7fda2659c700 1 --2- 172.21.15.5:0/4196448079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda281034c0 0x7fda2819f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:10.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:10.670+0000 7fda25d9b700 1 --2- 172.21.15.5:0/4196448079 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda28107df0 0x7fda2819fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:10.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:10.670+0000 7fda26d9d700 1 -- 172.21.15.5:0/4196448079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda281034c0 msgr2=0x7fda2819f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:10.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:10.670+0000 7fda26d9d700 1 --2- 172.21.15.5:0/4196448079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda281034c0 0x7fda2819f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:10.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:10.670+0000 7fda26d9d700 1 -- 172.21.15.5:0/4196448079 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda28107df0 msgr2=0x7fda2819fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:10.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:10.670+0000 7fda26d9d700 1 --2- 172.21.15.5:0/4196448079 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda28107df0 0x7fda2819fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:10.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:10.670+0000 7fda26d9d700 1 -- 172.21.15.5:0/4196448079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda2819a170 con 0x7fda28108740 2024-05-09T00:47:10.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:10.670+0000 7fda26d9d700 1 --2- 172.21.15.5:0/4196448079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda28108740 0x7fda28199b00 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fda2000d010 tx=0x7fda200079a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:10.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:10.670+0000 7fda177fe700 1 -- 172.21.15.5:0/4196448079 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2001f070 con 0x7fda28108740 2024-05-09T00:47:10.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:10.670+0000 7fda2cda2700 1 -- 172.21.15.5:0/4196448079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda2819a400 con 0x7fda28108740 2024-05-09T00:47:10.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:10.670+0000 7fda177fe700 1 -- 172.21.15.5:0/4196448079 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda20003c00 con 0x7fda28108740 2024-05-09T00:47:10.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:10.670+0000 7fda177fe700 1 -- 172.21.15.5:0/4196448079 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2001b6a0 con 0x7fda28108740 2024-05-09T00:47:10.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:10.670+0000 7fda2cda2700 1 -- 172.21.15.5:0/4196448079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda2819a8e0 con 0x7fda28108740 2024-05-09T00:47:10.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:10.671+0000 7fda2cda2700 1 -- 172.21.15.5:0/4196448079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda280512c0 con 0x7fda28108740 2024-05-09T00:47:10.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:10.673+0000 7fda177fe700 1 -- 172.21.15.5:0/4196448079 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fda20022070 con 0x7fda28108740 2024-05-09T00:47:10.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:10.674+0000 7fda177fe700 1 --2- 172.21.15.5:0/4196448079 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fda100b2880 0x7fda100b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:10.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:10.674+0000 7fda2659c700 1 --2- 172.21.15.5:0/4196448079 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fda100b2880 0x7fda100b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:10.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:10.674+0000 7fda177fe700 1 -- 172.21.15.5:0/4196448079 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fda2000f020 con 0x7fda28108740 2024-05-09T00:47:10.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:10.674+0000 7fda2659c700 1 --2- 172.21.15.5:0/4196448079 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fda100b2880 0x7fda100b4d90 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fda2819ac40 tx=0x7fda1800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:10.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:10.677+0000 7fda177fe700 1 -- 172.21.15.5:0/4196448079 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fda20029020 con 0x7fda28108740 2024-05-09T00:47:10.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:10.936+0000 7fda2cda2700 1 -- 172.21.15.5:0/4196448079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fda2810b1d0 con 0x7fda28108740 2024-05-09T00:47:10.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:10.936+0000 7fda177fe700 1 -- 172.21.15.5:0/4196448079 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11926 (secure 0 0 0) 0x7fda200ab570 con 0x7fda28108740 2024-05-09T00:47:10.937 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:47:10.938 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2685771887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2685771887},{"type":"v1","addr":"172.21.15.5:6835","nonce":2685771887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/254017535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":254017535},{"type":"v1","addr":"172.21.15.5:6837","nonce":254017535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T00:43:45.856066+0000","modified":"2024-05-09T00:47:06.980541+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24508,"mds_1":24499,"mds_2":24494,"mds_3":24493,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14598":{"gid":14598,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2109255228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2109255228},{"type":"v1","addr":"172.21.15.5:6839","nonce":2109255228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/3809966329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3809966329},{"type":"v1","addr":"172.21.15.5:6841","nonce":3809966329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6833/3272897742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3272897742},{"type":"v1","addr":"172.21.15.131:6833","nonce":3272897742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1283093898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1283093898},{"type":"v1","addr":"172.21.15.131:6835","nonce":1283093898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/2809525455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2809525455},{"type":"v1","addr":"172.21.15.157:6833","nonce":2809525455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.131:6837/1633006186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":1633006186},{"type":"v1","addr":"172.21.15.131:6837","nonce":1633006186}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.157:6835/832898273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":832898273},{"type":"v1","addr":"172.21.15.157:6835","nonce":832898273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.131:6839/2748669631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2748669631},{"type":"v1","addr":"172.21.15.131:6839","nonce":2748669631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.157:6837/744863009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":744863009},{"type":"v1","addr":"172.21.15.157:6837","nonce":744863009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.157:6839/2779860399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":2779860399},{"type":"v1","addr":"172.21.15.157:6839","nonce":2779860399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24494,24508,24499,24467,24493]},"id":1}]} 2024-05-09T00:47:10.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:10.939+0000 7fda2cda2700 1 -- 172.21.15.5:0/4196448079 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fda100b2880 msgr2=0x7fda100b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:10.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:10.939+0000 7fda2cda2700 1 --2- 172.21.15.5:0/4196448079 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fda100b2880 0x7fda100b4d90 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fda2819ac40 tx=0x7fda1800a380 comp rx=0 tx=0).stop 2024-05-09T00:47:10.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:10.939+0000 7fda2cda2700 1 -- 172.21.15.5:0/4196448079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda28108740 msgr2=0x7fda28199b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:10.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:10.939+0000 7fda2cda2700 1 --2- 172.21.15.5:0/4196448079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda28108740 0x7fda28199b00 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fda2000d010 tx=0x7fda200079a0 comp rx=0 tx=0).stop 2024-05-09T00:47:10.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:10.939+0000 7fda2cda2700 1 -- 172.21.15.5:0/4196448079 shutdown_connections 2024-05-09T00:47:10.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:10.939+0000 7fda2cda2700 1 --2- 172.21.15.5:0/4196448079 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fda100b2880 0x7fda100b4d90 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:10.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:10.939+0000 7fda2cda2700 1 --2- 172.21.15.5:0/4196448079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda28108740 0x7fda28199b00 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:10.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:10.939+0000 7fda2cda2700 1 --2- 172.21.15.5:0/4196448079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda281034c0 0x7fda2819f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:10.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:10.939+0000 7fda2cda2700 1 --2- 172.21.15.5:0/4196448079 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda28107df0 0x7fda2819fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:10.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:10.939+0000 7fda2cda2700 1 -- 172.21.15.5:0/4196448079 >> 172.21.15.5:0/4196448079 conn(0x7fda280fedc0 msgr2=0x7fda28107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:10.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:10.939+0000 7fda2cda2700 1 -- 172.21.15.5:0/4196448079 shutdown_connections 2024-05-09T00:47:10.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:10.939+0000 7fda2cda2700 1 -- 172.21.15.5:0/4196448079 wait complete. 2024-05-09T00:47:10.941 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 29 2024-05-09T00:47:11.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:47:11 smithi005 ceph-mon[30365]: pgmap v396: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 511 B/s wr, 11 op/s 2024-05-09T00:47:11.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:47:11 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/4196448079' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:47:11.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:47:11 smithi131 ceph-mon[34547]: pgmap v396: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 511 B/s wr, 11 op/s 2024-05-09T00:47:11.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:47:11 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/4196448079' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:47:11.628 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-05-09T00:47:11.628 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-05-09T00:47:11.628 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T00:47:11.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:47:11 smithi157 ceph-mon[34294]: pgmap v396: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 511 B/s wr, 11 op/s 2024-05-09T00:47:11.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:47:11 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/4196448079' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:47:11.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:11.748+0000 7f638e340640 1 -- 172.21.15.5:0/866950832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6388109b40 msgr2=0x7f638810bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:11.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:11.748+0000 7f638e340640 1 --2- 172.21.15.5:0/866950832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6388109b40 0x7f638810bf60 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6380009ca0 tx=0x7f638002a8c0 comp rx=0 tx=0).stop 2024-05-09T00:47:11.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:11.748+0000 7f638e340640 1 -- 172.21.15.5:0/866950832 shutdown_connections 2024-05-09T00:47:11.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:11.748+0000 7f638e340640 1 --2- 172.21.15.5:0/866950832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6388109b40 0x7f638810bf60 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:11.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:11.748+0000 7f638e340640 1 --2- 172.21.15.5:0/866950832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63881071e0 0x7f6388109600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:11.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:11.748+0000 7f638e340640 1 --2- 172.21.15.5:0/866950832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6388102030 0x7f6388106ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:11.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:11.748+0000 7f638e340640 1 -- 172.21.15.5:0/866950832 >> 172.21.15.5:0/866950832 conn(0x7f63880fd470 msgr2=0x7f63880ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:11.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:11.748+0000 7f638e340640 1 -- 172.21.15.5:0/866950832 shutdown_connections 2024-05-09T00:47:11.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:11.748+0000 7f638e340640 1 -- 172.21.15.5:0/866950832 wait complete. 2024-05-09T00:47:11.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:11.749+0000 7f638e340640 1 Processor -- start 2024-05-09T00:47:11.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:11.749+0000 7f638e340640 1 -- start start 2024-05-09T00:47:11.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:11.749+0000 7f638e340640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6388102030 0x7f638819a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:11.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:11.749+0000 7f638e340640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63881071e0 0x7f638819a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:11.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:11.749+0000 7f638e340640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6388109b40 0x7f63881a1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:11.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:11.749+0000 7f638e340640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f638810f7c0 con 0x7f63881071e0 2024-05-09T00:47:11.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:11.749+0000 7f638e340640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f638810f640 con 0x7f6388102030 2024-05-09T00:47:11.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:11.749+0000 7f638e340640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f638810f940 con 0x7f6388109b40 2024-05-09T00:47:11.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:11.749+0000 7f63877fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63881071e0 0x7f638819a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:11.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:11.749+0000 7f63877fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63881071e0 0x7f638819a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34460/0 (socket says 172.21.15.5:34460) 2024-05-09T00:47:11.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:11.749+0000 7f63877fe640 1 -- 172.21.15.5:0/3164953147 learned_addr learned my addr 172.21.15.5:0/3164953147 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:47:11.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:11.750+0000 7f6387fff640 1 --2- 172.21.15.5:0/3164953147 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6388102030 0x7f638819a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:11.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:11.750+0000 7f638c8dd640 1 --2- 172.21.15.5:0/3164953147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6388109b40 0x7f63881a1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:11.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:11.750+0000 7f63877fe640 1 -- 172.21.15.5:0/3164953147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6388109b40 msgr2=0x7f63881a1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:11.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:11.750+0000 7f63877fe640 1 --2- 172.21.15.5:0/3164953147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6388109b40 0x7f63881a1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:11.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:11.750+0000 7f63877fe640 1 -- 172.21.15.5:0/3164953147 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6388102030 msgr2=0x7f638819a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:11.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:11.750+0000 7f63877fe640 1 --2- 172.21.15.5:0/3164953147 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6388102030 0x7f638819a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:11.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:11.750+0000 7f63877fe640 1 -- 172.21.15.5:0/3164953147 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63881a1e70 con 0x7f63881071e0 2024-05-09T00:47:11.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:11.750+0000 7f638c8dd640 1 --2- 172.21.15.5:0/3164953147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6388109b40 0x7f63881a1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:47:11.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:11.750+0000 7f6387fff640 1 --2- 172.21.15.5:0/3164953147 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6388102030 0x7f638819a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:47:11.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:11.750+0000 7f63877fe640 1 --2- 172.21.15.5:0/3164953147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63881071e0 0x7f638819a870 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f637c00d950 tx=0x7f637c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:11.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:11.750+0000 7f63857fa640 1 -- 172.21.15.5:0/3164953147 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f637c017030 con 0x7f63881071e0 2024-05-09T00:47:11.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:11.750+0000 7f638e340640 1 -- 172.21.15.5:0/3164953147 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63881a2160 con 0x7f63881071e0 2024-05-09T00:47:11.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:11.750+0000 7f63857fa640 1 -- 172.21.15.5:0/3164953147 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f637c004020 con 0x7f63881071e0 2024-05-09T00:47:11.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:11.750+0000 7f63857fa640 1 -- 172.21.15.5:0/3164953147 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f637c0047a0 con 0x7f63881071e0 2024-05-09T00:47:11.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:11.750+0000 7f638e340640 1 -- 172.21.15.5:0/3164953147 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63881a26b0 con 0x7f63881071e0 2024-05-09T00:47:11.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:11.751+0000 7f638e340640 1 -- 172.21.15.5:0/3164953147 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63880fef90 con 0x7f63881071e0 2024-05-09T00:47:11.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:11.753+0000 7f63857fa640 1 -- 172.21.15.5:0/3164953147 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f637c009a30 con 0x7f63881071e0 2024-05-09T00:47:11.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:11.754+0000 7f63857fa640 1 --2- 172.21.15.5:0/3164953147 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f63700b27f0 0x7f63700b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:11.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:11.754+0000 7f6387fff640 1 --2- 172.21.15.5:0/3164953147 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f63700b27f0 0x7f63700b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:11.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:11.754+0000 7f63857fa640 1 -- 172.21.15.5:0/3164953147 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f637c0e23a0 con 0x7f63881071e0 2024-05-09T00:47:11.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:11.755+0000 7f6387fff640 1 --2- 172.21.15.5:0/3164953147 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f63700b27f0 0x7f63700b4d00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6388104c20 tx=0x7f6378005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:11.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:11.758+0000 7f63857fa640 1 -- 172.21.15.5:0/3164953147 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f637c0aa090 con 0x7f63881071e0 2024-05-09T00:47:11.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:11.951+0000 7f638e340640 1 -- 172.21.15.5:0/3164953147 --> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6388063110 con 0x7f63700b27f0 2024-05-09T00:47:11.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:11.958+0000 7f63857fa640 1 -- 172.21.15.5:0/3164953147 <== mgr.14152 v2:172.21.15.5:6800/91856593 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259094 (secure 0 0 0) 0x7f6388063110 con 0x7f63700b27f0 2024-05-09T00:47:11.962 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:47:11.964 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-09T00:47:11.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:11.961+0000 7f638e340640 1 -- 172.21.15.5:0/3164953147 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f63700b27f0 msgr2=0x7f63700b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:11.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:11.961+0000 7f638e340640 1 --2- 172.21.15.5:0/3164953147 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f63700b27f0 0x7f63700b4d00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6388104c20 tx=0x7f6378005f90 comp rx=0 tx=0).stop 2024-05-09T00:47:11.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:11.961+0000 7f638e340640 1 -- 172.21.15.5:0/3164953147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63881071e0 msgr2=0x7f638819a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:11.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:11.961+0000 7f638e340640 1 --2- 172.21.15.5:0/3164953147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63881071e0 0x7f638819a870 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f637c00d950 tx=0x7f637c00de10 comp rx=0 tx=0).stop 2024-05-09T00:47:11.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:11.962+0000 7f638e340640 1 -- 172.21.15.5:0/3164953147 shutdown_connections 2024-05-09T00:47:11.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:11.962+0000 7f638e340640 1 --2- 172.21.15.5:0/3164953147 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f63700b27f0 0x7f63700b4d00 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:11.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:11.962+0000 7f638e340640 1 --2- 172.21.15.5:0/3164953147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6388109b40 0x7f63881a1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:11.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:11.962+0000 7f638e340640 1 --2- 172.21.15.5:0/3164953147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63881071e0 0x7f638819a870 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:11.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:11.962+0000 7f638e340640 1 --2- 172.21.15.5:0/3164953147 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6388102030 0x7f638819a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:11.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:11.962+0000 7f638e340640 1 -- 172.21.15.5:0/3164953147 >> 172.21.15.5:0/3164953147 conn(0x7f63880fd470 msgr2=0x7f6388105330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:11.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:11.962+0000 7f638e340640 1 -- 172.21.15.5:0/3164953147 shutdown_connections 2024-05-09T00:47:11.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:11.962+0000 7f638e340640 1 -- 172.21.15.5:0/3164953147 wait complete. 2024-05-09T00:47:11.974 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":397,"stamp":"2024-05-09T00:47:11.913507+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661820,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1970,"num_read_kb":1080,"num_write":391,"num_write_kb":2306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":3005105,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":822,"ondisk_log_size":822,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":351896,"kb_used_data":11064,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124589928,"statfs":{"total":1151940427776,"available":1151580086272,"internally_reserved":0,"allocated":11329536,"data_stored":7285140,"data_compressed":186585,"data_compressed_allocated":245760,"data_compressed_original":2015232,"omap_allocated":19141,"internal_metadata":348894523},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":616,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":107,"num_read_kb":52,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003299"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.877185+0000","last_change":"2024-05-09T00:43:43.875357+0000","last_active":"2024-05-09T00:45:51.877185+0000","last_peered":"2024-05-09T00:45:51.877185+0000","last_clean":"2024-05-09T00:45:51.877185+0000","last_became_active":"2024-05-09T00:43:43.873765+0000","last_became_peered":"2024-05-09T00:43:43.873765+0000","last_unstale":"2024-05-09T00:45:51.877185+0000","last_undegraded":"2024-05-09T00:45:51.877185+0000","last_fullsized":"2024-05-09T00:45:51.877185+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:47.519110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.875117+0000","last_change":"2024-05-09T00:43:43.882464+0000","last_active":"2024-05-09T00:45:51.875117+0000","last_peered":"2024-05-09T00:45:51.875117+0000","last_clean":"2024-05-09T00:45:51.875117+0000","last_became_active":"2024-05-09T00:43:43.882102+0000","last_became_peered":"2024-05-09T00:43:43.882102+0000","last_unstale":"2024-05-09T00:45:51.875117+0000","last_undegraded":"2024-05-09T00:45:51.875117+0000","last_fullsized":"2024-05-09T00:45:51.875117+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:39:36.234474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.872356+0000","last_change":"2024-05-09T00:43:43.873098+0000","last_active":"2024-05-09T00:45:51.872356+0000","last_peered":"2024-05-09T00:45:51.872356+0000","last_clean":"2024-05-09T00:45:51.872356+0000","last_became_active":"2024-05-09T00:43:43.872764+0000","last_became_peered":"2024-05-09T00:43:43.872764+0000","last_unstale":"2024-05-09T00:45:51.872356+0000","last_undegraded":"2024-05-09T00:45:51.872356+0000","last_fullsized":"2024-05-09T00:45:51.872356+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:05:30.784111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.869925+0000","last_change":"2024-05-09T00:43:43.869807+0000","last_active":"2024-05-09T00:45:51.869925+0000","last_peered":"2024-05-09T00:45:51.869925+0000","last_clean":"2024-05-09T00:45:51.869925+0000","last_became_active":"2024-05-09T00:43:43.869470+0000","last_became_peered":"2024-05-09T00:43:43.869470+0000","last_unstale":"2024-05-09T00:45:51.869925+0000","last_undegraded":"2024-05-09T00:45:51.869925+0000","last_fullsized":"2024-05-09T00:45:51.869925+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:25:40.201844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.852785+0000","last_change":"2024-05-09T00:43:43.874110+0000","last_active":"2024-05-09T00:45:51.852785+0000","last_peered":"2024-05-09T00:45:51.852785+0000","last_clean":"2024-05-09T00:45:51.852785+0000","last_became_active":"2024-05-09T00:43:43.873759+0000","last_became_peered":"2024-05-09T00:43:43.873759+0000","last_unstale":"2024-05-09T00:45:51.852785+0000","last_undegraded":"2024-05-09T00:45:51.852785+0000","last_fullsized":"2024-05-09T00:45:51.852785+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:17:17.722319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.862325+0000","last_change":"2024-05-09T00:44:23.022285+0000","last_active":"2024-05-09T00:45:51.862325+0000","last_peered":"2024-05-09T00:45:51.862325+0000","last_clean":"2024-05-09T00:45:51.862325+0000","last_became_active":"2024-05-09T00:44:23.021887+0000","last_became_peered":"2024-05-09T00:44:23.021887+0000","last_unstale":"2024-05-09T00:45:51.862325+0000","last_undegraded":"2024-05-09T00:45:51.862325+0000","last_fullsized":"2024-05-09T00:45:51.862325+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:07:07.702178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,7],"acting":[10,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.873552+0000","last_change":"2024-05-09T00:43:43.883112+0000","last_active":"2024-05-09T00:45:51.873552+0000","last_peered":"2024-05-09T00:45:51.873552+0000","last_clean":"2024-05-09T00:45:51.873552+0000","last_became_active":"2024-05-09T00:43:43.882795+0000","last_became_peered":"2024-05-09T00:43:43.882795+0000","last_unstale":"2024-05-09T00:45:51.873552+0000","last_undegraded":"2024-05-09T00:45:51.873552+0000","last_fullsized":"2024-05-09T00:45:51.873552+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:33:58.724015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.862472+0000","last_change":"2024-05-09T00:43:43.880696+0000","last_active":"2024-05-09T00:45:51.862472+0000","last_peered":"2024-05-09T00:45:51.862472+0000","last_clean":"2024-05-09T00:45:51.862472+0000","last_became_active":"2024-05-09T00:43:43.880174+0000","last_became_peered":"2024-05-09T00:43:43.880174+0000","last_unstale":"2024-05-09T00:45:51.862472+0000","last_undegraded":"2024-05-09T00:45:51.862472+0000","last_fullsized":"2024-05-09T00:45:51.862472+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:40:05.682432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.879518+0000","last_change":"2024-05-09T00:43:43.883449+0000","last_active":"2024-05-09T00:45:51.879518+0000","last_peered":"2024-05-09T00:45:51.879518+0000","last_clean":"2024-05-09T00:45:51.879518+0000","last_became_active":"2024-05-09T00:43:43.883066+0000","last_became_peered":"2024-05-09T00:43:43.883066+0000","last_unstale":"2024-05-09T00:45:51.879518+0000","last_undegraded":"2024-05-09T00:45:51.879518+0000","last_fullsized":"2024-05-09T00:45:51.879518+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:47:49.116004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":41,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.868635+0000","last_change":"2024-05-09T00:43:43.877514+0000","last_active":"2024-05-09T00:45:51.868635+0000","last_peered":"2024-05-09T00:45:51.868635+0000","last_clean":"2024-05-09T00:45:51.868635+0000","last_became_active":"2024-05-09T00:43:43.877170+0000","last_became_peered":"2024-05-09T00:43:43.877170+0000","last_unstale":"2024-05-09T00:45:51.868635+0000","last_undegraded":"2024-05-09T00:45:51.868635+0000","last_fullsized":"2024-05-09T00:45:51.868635+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:47:26.738571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.878663+0000","last_change":"2024-05-09T00:44:23.180294+0000","last_active":"2024-05-09T00:45:51.878663+0000","last_peered":"2024-05-09T00:45:51.878663+0000","last_clean":"2024-05-09T00:45:51.878663+0000","last_became_active":"2024-05-09T00:44:23.179824+0000","last_became_peered":"2024-05-09T00:44:23.179824+0000","last_unstale":"2024-05-09T00:45:51.878663+0000","last_undegraded":"2024-05-09T00:45:51.878663+0000","last_fullsized":"2024-05-09T00:45:51.878663+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:43:20.919660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,9],"acting":[7,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.858638+0000","last_change":"2024-05-09T00:43:43.873836+0000","last_active":"2024-05-09T00:45:51.858638+0000","last_peered":"2024-05-09T00:45:51.858638+0000","last_clean":"2024-05-09T00:45:51.858638+0000","last_became_active":"2024-05-09T00:43:43.873512+0000","last_became_peered":"2024-05-09T00:43:43.873512+0000","last_unstale":"2024-05-09T00:45:51.858638+0000","last_undegraded":"2024-05-09T00:45:51.858638+0000","last_fullsized":"2024-05-09T00:45:51.858638+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:32:06.162638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.864447+0000","last_change":"2024-05-09T00:43:43.885614+0000","last_active":"2024-05-09T00:45:51.864447+0000","last_peered":"2024-05-09T00:45:51.864447+0000","last_clean":"2024-05-09T00:45:51.864447+0000","last_became_active":"2024-05-09T00:43:43.884682+0000","last_became_peered":"2024-05-09T00:43:43.884682+0000","last_unstale":"2024-05-09T00:45:51.864447+0000","last_undegraded":"2024-05-09T00:45:51.864447+0000","last_fullsized":"2024-05-09T00:45:51.864447+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:34:15.336173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.872171+0000","last_change":"2024-05-09T00:43:43.883860+0000","last_active":"2024-05-09T00:45:51.872171+0000","last_peered":"2024-05-09T00:45:51.872171+0000","last_clean":"2024-05-09T00:45:51.872171+0000","last_became_active":"2024-05-09T00:43:43.882870+0000","last_became_peered":"2024-05-09T00:43:43.882870+0000","last_unstale":"2024-05-09T00:45:51.872171+0000","last_undegraded":"2024-05-09T00:45:51.872171+0000","last_fullsized":"2024-05-09T00:45:51.872171+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:17:31.566819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.869693+0000","last_change":"2024-05-09T00:43:43.874114+0000","last_active":"2024-05-09T00:45:51.869693+0000","last_peered":"2024-05-09T00:45:51.869693+0000","last_clean":"2024-05-09T00:45:51.869693+0000","last_became_active":"2024-05-09T00:43:43.873762+0000","last_became_peered":"2024-05-09T00:43:43.873762+0000","last_unstale":"2024-05-09T00:45:51.869693+0000","last_undegraded":"2024-05-09T00:45:51.869693+0000","last_fullsized":"2024-05-09T00:45:51.869693+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:24:27.486100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.879829+0000","last_change":"2024-05-09T00:43:43.872982+0000","last_active":"2024-05-09T00:45:51.879829+0000","last_peered":"2024-05-09T00:45:51.879829+0000","last_clean":"2024-05-09T00:45:51.879829+0000","last_became_active":"2024-05-09T00:43:43.872464+0000","last_became_peered":"2024-05-09T00:43:43.872464+0000","last_unstale":"2024-05-09T00:45:51.879829+0000","last_undegraded":"2024-05-09T00:45:51.879829+0000","last_fullsized":"2024-05-09T00:45:51.879829+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:41.172001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.877289+0000","last_change":"2024-05-09T00:43:43.875662+0000","last_active":"2024-05-09T00:45:51.877289+0000","last_peered":"2024-05-09T00:45:51.877289+0000","last_clean":"2024-05-09T00:45:51.877289+0000","last_became_active":"2024-05-09T00:43:43.874701+0000","last_became_peered":"2024-05-09T00:43:43.874701+0000","last_unstale":"2024-05-09T00:45:51.877289+0000","last_undegraded":"2024-05-09T00:45:51.877289+0000","last_fullsized":"2024-05-09T00:45:51.877289+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:09:19.349190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.864219+0000","last_change":"2024-05-09T00:43:43.876100+0000","last_active":"2024-05-09T00:45:51.864219+0000","last_peered":"2024-05-09T00:45:51.864219+0000","last_clean":"2024-05-09T00:45:51.864219+0000","last_became_active":"2024-05-09T00:43:43.875768+0000","last_became_peered":"2024-05-09T00:43:43.875768+0000","last_unstale":"2024-05-09T00:45:51.864219+0000","last_undegraded":"2024-05-09T00:45:51.864219+0000","last_fullsized":"2024-05-09T00:45:51.864219+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:48:26.843064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.865952+0000","last_change":"2024-05-09T00:43:43.875900+0000","last_active":"2024-05-09T00:45:51.865952+0000","last_peered":"2024-05-09T00:45:51.865952+0000","last_clean":"2024-05-09T00:45:51.865952+0000","last_became_active":"2024-05-09T00:43:43.875497+0000","last_became_peered":"2024-05-09T00:43:43.875497+0000","last_unstale":"2024-05-09T00:45:51.865952+0000","last_undegraded":"2024-05-09T00:45:51.865952+0000","last_fullsized":"2024-05-09T00:45:51.865952+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:54:44.215936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'13","reported_seq":230,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:47:06.232722+0000","last_change":"2024-05-09T00:43:40.854337+0000","last_active":"2024-05-09T00:47:06.232722+0000","last_peered":"2024-05-09T00:47:06.232722+0000","last_clean":"2024-05-09T00:47:06.232722+0000","last_became_active":"2024-05-09T00:43:40.853986+0000","last_became_peered":"2024-05-09T00:43:40.853986+0000","last_unstale":"2024-05-09T00:47:06.232722+0000","last_undegraded":"2024-05-09T00:47:06.232722+0000","last_fullsized":"2024-05-09T00:47:06.232722+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:01:19.128533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":186,"num_read_kb":186,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.865652+0000","last_change":"2024-05-09T00:43:43.878081+0000","last_active":"2024-05-09T00:45:51.865652+0000","last_peered":"2024-05-09T00:45:51.865652+0000","last_clean":"2024-05-09T00:45:51.865652+0000","last_became_active":"2024-05-09T00:43:43.877585+0000","last_became_peered":"2024-05-09T00:43:43.877585+0000","last_unstale":"2024-05-09T00:45:51.865652+0000","last_undegraded":"2024-05-09T00:45:51.865652+0000","last_fullsized":"2024-05-09T00:45:51.865652+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:17:39.255841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.115936+0000","last_change":"2024-05-09T00:43:40.959386+0000","last_active":"2024-05-09T00:45:24.115936+0000","last_peered":"2024-05-09T00:45:24.115936+0000","last_clean":"2024-05-09T00:45:24.115936+0000","last_became_active":"2024-05-09T00:43:40.958878+0000","last_became_peered":"2024-05-09T00:43:40.958878+0000","last_unstale":"2024-05-09T00:45:24.115936+0000","last_undegraded":"2024-05-09T00:45:24.115936+0000","last_fullsized":"2024-05-09T00:45:24.115936+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:14:57.105078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.873650+0000","last_change":"2024-05-09T00:43:43.870563+0000","last_active":"2024-05-09T00:45:51.873650+0000","last_peered":"2024-05-09T00:45:51.873650+0000","last_clean":"2024-05-09T00:45:51.873650+0000","last_became_active":"2024-05-09T00:43:43.870237+0000","last_became_peered":"2024-05-09T00:43:43.870237+0000","last_unstale":"2024-05-09T00:45:51.873650+0000","last_undegraded":"2024-05-09T00:45:51.873650+0000","last_fullsized":"2024-05-09T00:45:51.873650+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:14:28.006343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"78'12","reported_seq":44,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.937368+0000","last_change":"2024-05-09T00:43:40.857670+0000","last_active":"2024-05-09T00:45:23.937368+0000","last_peered":"2024-05-09T00:45:23.937368+0000","last_clean":"2024-05-09T00:45:23.937368+0000","last_became_active":"2024-05-09T00:43:40.857247+0000","last_became_peered":"2024-05-09T00:43:40.857247+0000","last_unstale":"2024-05-09T00:45:23.937368+0000","last_undegraded":"2024-05-09T00:45:23.937368+0000","last_fullsized":"2024-05-09T00:45:23.937368+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:47:06.325593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.873557+0000","last_change":"2024-05-09T00:43:43.883179+0000","last_active":"2024-05-09T00:45:51.873557+0000","last_peered":"2024-05-09T00:45:51.873557+0000","last_clean":"2024-05-09T00:45:51.873557+0000","last_became_active":"2024-05-09T00:43:43.882820+0000","last_became_peered":"2024-05-09T00:43:43.882820+0000","last_unstale":"2024-05-09T00:45:51.873557+0000","last_undegraded":"2024-05-09T00:45:51.873557+0000","last_fullsized":"2024-05-09T00:45:51.873557+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:56:26.012495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"78'5","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.946133+0000","last_change":"2024-05-09T00:45:23.945720+0000","last_active":"2024-05-09T00:45:23.946133+0000","last_peered":"2024-05-09T00:45:23.946133+0000","last_clean":"2024-05-09T00:45:23.946133+0000","last_became_active":"2024-05-09T00:45:23.940421+0000","last_became_peered":"2024-05-09T00:45:23.940421+0000","last_unstale":"2024-05-09T00:45:23.946133+0000","last_undegraded":"2024-05-09T00:45:23.946133+0000","last_fullsized":"2024-05-09T00:45:23.946133+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:23:24.905715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.876297+0000","last_change":"2024-05-09T00:43:43.882950+0000","last_active":"2024-05-09T00:45:51.876297+0000","last_peered":"2024-05-09T00:45:51.876297+0000","last_clean":"2024-05-09T00:45:51.876297+0000","last_became_active":"2024-05-09T00:43:43.880679+0000","last_became_peered":"2024-05-09T00:43:43.880679+0000","last_unstale":"2024-05-09T00:45:51.876297+0000","last_undegraded":"2024-05-09T00:45:51.876297+0000","last_fullsized":"2024-05-09T00:45:51.876297+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:34:14.050987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"82'7","reported_seq":227,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:47:06.234636+0000","last_change":"2024-05-09T00:43:40.854953+0000","last_active":"2024-05-09T00:47:06.234636+0000","last_peered":"2024-05-09T00:47:06.234636+0000","last_clean":"2024-05-09T00:47:06.234636+0000","last_became_active":"2024-05-09T00:43:40.854562+0000","last_became_peered":"2024-05-09T00:43:40.854562+0000","last_unstale":"2024-05-09T00:47:06.234636+0000","last_undegraded":"2024-05-09T00:47:06.234636+0000","last_fullsized":"2024-05-09T00:47:06.234636+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:09:19.564806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":189,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.857389+0000","last_change":"2024-05-09T00:43:43.881881+0000","last_active":"2024-05-09T00:45:51.857389+0000","last_peered":"2024-05-09T00:45:51.857389+0000","last_clean":"2024-05-09T00:45:51.857389+0000","last_became_active":"2024-05-09T00:43:43.881461+0000","last_became_peered":"2024-05-09T00:43:43.881461+0000","last_unstale":"2024-05-09T00:45:51.857389+0000","last_undegraded":"2024-05-09T00:45:51.857389+0000","last_fullsized":"2024-05-09T00:45:51.857389+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:59:28.747084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"78'7","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.080620+0000","last_change":"2024-05-09T00:43:40.858521+0000","last_active":"2024-05-09T00:45:24.080620+0000","last_peered":"2024-05-09T00:45:24.080620+0000","last_clean":"2024-05-09T00:45:24.080620+0000","last_became_active":"2024-05-09T00:43:40.858102+0000","last_became_peered":"2024-05-09T00:43:40.858102+0000","last_unstale":"2024-05-09T00:45:24.080620+0000","last_undegraded":"2024-05-09T00:45:24.080620+0000","last_fullsized":"2024-05-09T00:45:24.080620+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:09:51.684630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.873396+0000","last_change":"2024-05-09T00:44:23.023472+0000","last_active":"2024-05-09T00:45:51.873396+0000","last_peered":"2024-05-09T00:45:51.873396+0000","last_clean":"2024-05-09T00:45:51.873396+0000","last_became_active":"2024-05-09T00:44:23.023017+0000","last_became_peered":"2024-05-09T00:44:23.023017+0000","last_unstale":"2024-05-09T00:45:51.873396+0000","last_undegraded":"2024-05-09T00:45:51.873396+0000","last_fullsized":"2024-05-09T00:45:51.873396+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:05:13.852606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"78'3","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.934006+0000","last_change":"2024-05-09T00:43:40.853329+0000","last_active":"2024-05-09T00:45:23.934006+0000","last_peered":"2024-05-09T00:45:23.934006+0000","last_clean":"2024-05-09T00:45:23.934006+0000","last_became_active":"2024-05-09T00:43:40.852979+0000","last_became_peered":"2024-05-09T00:43:40.852979+0000","last_unstale":"2024-05-09T00:45:23.934006+0000","last_undegraded":"2024-05-09T00:45:23.934006+0000","last_fullsized":"2024-05-09T00:45:23.934006+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:13:57.006031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.875355+0000","last_change":"2024-05-09T00:43:43.862876+0000","last_active":"2024-05-09T00:45:51.875355+0000","last_peered":"2024-05-09T00:45:51.875355+0000","last_clean":"2024-05-09T00:45:51.875355+0000","last_became_active":"2024-05-09T00:43:43.862521+0000","last_became_peered":"2024-05-09T00:43:43.862521+0000","last_unstale":"2024-05-09T00:45:51.875355+0000","last_undegraded":"2024-05-09T00:45:51.875355+0000","last_fullsized":"2024-05-09T00:45:51.875355+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:45:49.386844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'3","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.937546+0000","last_change":"2024-05-09T00:43:40.860395+0000","last_active":"2024-05-09T00:45:23.937546+0000","last_peered":"2024-05-09T00:45:23.937546+0000","last_clean":"2024-05-09T00:45:23.937546+0000","last_became_active":"2024-05-09T00:43:40.860040+0000","last_became_peered":"2024-05-09T00:43:40.860040+0000","last_unstale":"2024-05-09T00:45:23.937546+0000","last_undegraded":"2024-05-09T00:45:23.937546+0000","last_fullsized":"2024-05-09T00:45:23.937546+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:20:36.054685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.862285+0000","last_change":"2024-05-09T00:44:23.180905+0000","last_active":"2024-05-09T00:45:51.862285+0000","last_peered":"2024-05-09T00:45:51.862285+0000","last_clean":"2024-05-09T00:45:51.862285+0000","last_became_active":"2024-05-09T00:44:23.180296+0000","last_became_peered":"2024-05-09T00:44:23.180296+0000","last_unstale":"2024-05-09T00:45:51.862285+0000","last_undegraded":"2024-05-09T00:45:51.862285+0000","last_fullsized":"2024-05-09T00:45:51.862285+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:48:17.585276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,10],"acting":[9,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'6","reported_seq":407,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:47:07.235641+0000","last_change":"2024-05-09T00:43:40.860542+0000","last_active":"2024-05-09T00:47:07.235641+0000","last_peered":"2024-05-09T00:47:07.235641+0000","last_clean":"2024-05-09T00:47:07.235641+0000","last_became_active":"2024-05-09T00:43:40.860176+0000","last_became_peered":"2024-05-09T00:43:40.860176+0000","last_unstale":"2024-05-09T00:47:07.235641+0000","last_undegraded":"2024-05-09T00:47:07.235641+0000","last_fullsized":"2024-05-09T00:47:07.235641+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:44:50.375024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":370,"num_read_kb":370,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.872797+0000","last_change":"2024-05-09T00:43:43.882119+0000","last_active":"2024-05-09T00:45:51.872797+0000","last_peered":"2024-05-09T00:45:51.872797+0000","last_clean":"2024-05-09T00:45:51.872797+0000","last_became_active":"2024-05-09T00:43:43.881747+0000","last_became_peered":"2024-05-09T00:43:43.881747+0000","last_unstale":"2024-05-09T00:45:51.872797+0000","last_undegraded":"2024-05-09T00:45:51.872797+0000","last_fullsized":"2024-05-09T00:45:51.872797+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:54:19.420435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'4","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.081294+0000","last_change":"2024-05-09T00:43:40.859539+0000","last_active":"2024-05-09T00:45:24.081294+0000","last_peered":"2024-05-09T00:45:24.081294+0000","last_clean":"2024-05-09T00:45:24.081294+0000","last_became_active":"2024-05-09T00:43:40.859162+0000","last_became_peered":"2024-05-09T00:43:40.859162+0000","last_unstale":"2024-05-09T00:45:24.081294+0000","last_undegraded":"2024-05-09T00:45:24.081294+0000","last_fullsized":"2024-05-09T00:45:24.081294+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:37:15.190944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.872246+0000","last_change":"2024-05-09T00:44:23.180872+0000","last_active":"2024-05-09T00:45:51.872246+0000","last_peered":"2024-05-09T00:45:51.872246+0000","last_clean":"2024-05-09T00:45:51.872246+0000","last_became_active":"2024-05-09T00:44:23.180398+0000","last_became_peered":"2024-05-09T00:44:23.180398+0000","last_unstale":"2024-05-09T00:45:51.872246+0000","last_undegraded":"2024-05-09T00:45:51.872246+0000","last_fullsized":"2024-05-09T00:45:51.872246+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:19:27.909490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,6],"acting":[0,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'5","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.938347+0000","last_change":"2024-05-09T00:43:40.960126+0000","last_active":"2024-05-09T00:45:23.938347+0000","last_peered":"2024-05-09T00:45:23.938347+0000","last_clean":"2024-05-09T00:45:23.938347+0000","last_became_active":"2024-05-09T00:43:40.959441+0000","last_became_peered":"2024-05-09T00:43:40.959441+0000","last_unstale":"2024-05-09T00:45:23.938347+0000","last_undegraded":"2024-05-09T00:45:23.938347+0000","last_fullsized":"2024-05-09T00:45:23.938347+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:38:47.497029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.881221+0000","last_change":"2024-05-09T00:43:43.874223+0000","last_active":"2024-05-09T00:45:51.881221+0000","last_peered":"2024-05-09T00:45:51.881221+0000","last_clean":"2024-05-09T00:45:51.881221+0000","last_became_active":"2024-05-09T00:43:43.873875+0000","last_became_peered":"2024-05-09T00:43:43.873875+0000","last_unstale":"2024-05-09T00:45:51.881221+0000","last_undegraded":"2024-05-09T00:45:51.881221+0000","last_fullsized":"2024-05-09T00:45:51.881221+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:59:22.192886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'37","reported_seq":469,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:47:07.457164+0000","last_change":"2024-05-09T00:44:23.188953+0000","last_active":"2024-05-09T00:47:07.457164+0000","last_peered":"2024-05-09T00:47:07.457164+0000","last_clean":"2024-05-09T00:47:07.457164+0000","last_became_active":"2024-05-09T00:44:23.180858+0000","last_became_peered":"2024-05-09T00:44:23.180858+0000","last_unstale":"2024-05-09T00:47:07.457164+0000","last_undegraded":"2024-05-09T00:47:07.457164+0000","last_fullsized":"2024-05-09T00:47:07.457164+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:26:16.427665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62892,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":376,"num_read_kb":239,"num_write":39,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":52683,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.871155+0000","last_change":"2024-05-09T00:43:43.872019+0000","last_active":"2024-05-09T00:45:51.871155+0000","last_peered":"2024-05-09T00:45:51.871155+0000","last_clean":"2024-05-09T00:45:51.871155+0000","last_became_active":"2024-05-09T00:43:43.871622+0000","last_became_peered":"2024-05-09T00:43:43.871622+0000","last_unstale":"2024-05-09T00:45:51.871155+0000","last_undegraded":"2024-05-09T00:45:51.871155+0000","last_fullsized":"2024-05-09T00:45:51.871155+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:41:47.789443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'9","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.116231+0000","last_change":"2024-05-09T00:43:40.860111+0000","last_active":"2024-05-09T00:45:24.116231+0000","last_peered":"2024-05-09T00:45:24.116231+0000","last_clean":"2024-05-09T00:45:24.116231+0000","last_became_active":"2024-05-09T00:43:40.859732+0000","last_became_peered":"2024-05-09T00:43:40.859732+0000","last_unstale":"2024-05-09T00:45:24.116231+0000","last_undegraded":"2024-05-09T00:45:24.116231+0000","last_fullsized":"2024-05-09T00:45:24.116231+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:30:03.180679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.867875+0000","last_change":"2024-05-09T00:43:43.870604+0000","last_active":"2024-05-09T00:45:51.867875+0000","last_peered":"2024-05-09T00:45:51.867875+0000","last_clean":"2024-05-09T00:45:51.867875+0000","last_became_active":"2024-05-09T00:43:43.870286+0000","last_became_peered":"2024-05-09T00:43:43.870286+0000","last_unstale":"2024-05-09T00:45:51.867875+0000","last_undegraded":"2024-05-09T00:45:51.867875+0000","last_fullsized":"2024-05-09T00:45:51.867875+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:57:04.486631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'5","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.937866+0000","last_change":"2024-05-09T00:43:40.851966+0000","last_active":"2024-05-09T00:45:23.937866+0000","last_peered":"2024-05-09T00:45:23.937866+0000","last_clean":"2024-05-09T00:45:23.937866+0000","last_became_active":"2024-05-09T00:43:40.851537+0000","last_became_peered":"2024-05-09T00:43:40.851537+0000","last_unstale":"2024-05-09T00:45:23.937866+0000","last_undegraded":"2024-05-09T00:45:23.937866+0000","last_fullsized":"2024-05-09T00:45:23.937866+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:58:59.797183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.879660+0000","last_change":"2024-05-09T00:43:43.879509+0000","last_active":"2024-05-09T00:45:51.879660+0000","last_peered":"2024-05-09T00:45:51.879660+0000","last_clean":"2024-05-09T00:45:51.879660+0000","last_became_active":"2024-05-09T00:43:43.879151+0000","last_became_peered":"2024-05-09T00:43:43.879151+0000","last_unstale":"2024-05-09T00:45:51.879660+0000","last_undegraded":"2024-05-09T00:45:51.879660+0000","last_fullsized":"2024-05-09T00:45:51.879660+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:34:18.850545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'8","reported_seq":228,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:47:06.233264+0000","last_change":"2024-05-09T00:43:40.857629+0000","last_active":"2024-05-09T00:47:06.233264+0000","last_peered":"2024-05-09T00:47:06.233264+0000","last_clean":"2024-05-09T00:47:06.233264+0000","last_became_active":"2024-05-09T00:43:40.857260+0000","last_became_peered":"2024-05-09T00:43:40.857260+0000","last_unstale":"2024-05-09T00:47:06.233264+0000","last_undegraded":"2024-05-09T00:47:06.233264+0000","last_fullsized":"2024-05-09T00:47:06.233264+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:51:00.600870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1428,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":190,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.875060+0000","last_change":"2024-05-09T00:43:43.886100+0000","last_active":"2024-05-09T00:45:51.875060+0000","last_peered":"2024-05-09T00:45:51.875060+0000","last_clean":"2024-05-09T00:45:51.875060+0000","last_became_active":"2024-05-09T00:43:43.885775+0000","last_became_peered":"2024-05-09T00:43:43.885775+0000","last_unstale":"2024-05-09T00:45:51.875060+0000","last_undegraded":"2024-05-09T00:45:51.875060+0000","last_fullsized":"2024-05-09T00:45:51.875060+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:34:19.439163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'17","reported_seq":49,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.936845+0000","last_change":"2024-05-09T00:43:40.959873+0000","last_active":"2024-05-09T00:45:23.936845+0000","last_peered":"2024-05-09T00:45:23.936845+0000","last_clean":"2024-05-09T00:45:23.936845+0000","last_became_active":"2024-05-09T00:43:40.959437+0000","last_became_peered":"2024-05-09T00:43:40.959437+0000","last_unstale":"2024-05-09T00:45:23.936845+0000","last_undegraded":"2024-05-09T00:45:23.936845+0000","last_fullsized":"2024-05-09T00:45:23.936845+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:29:52.686937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.875543+0000","last_change":"2024-05-09T00:43:43.880076+0000","last_active":"2024-05-09T00:45:51.875543+0000","last_peered":"2024-05-09T00:45:51.875543+0000","last_clean":"2024-05-09T00:45:51.875543+0000","last_became_active":"2024-05-09T00:43:43.878733+0000","last_became_peered":"2024-05-09T00:43:43.878733+0000","last_unstale":"2024-05-09T00:45:51.875543+0000","last_undegraded":"2024-05-09T00:45:51.875543+0000","last_fullsized":"2024-05-09T00:45:51.875543+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:30:21.426855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'3","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:59.646186+0000","last_change":"2024-05-09T00:43:40.858033+0000","last_active":"2024-05-09T00:45:59.646186+0000","last_peered":"2024-05-09T00:45:59.646186+0000","last_clean":"2024-05-09T00:45:59.646186+0000","last_became_active":"2024-05-09T00:43:40.857075+0000","last_became_peered":"2024-05-09T00:43:40.857075+0000","last_unstale":"2024-05-09T00:45:59.646186+0000","last_undegraded":"2024-05-09T00:45:59.646186+0000","last_fullsized":"2024-05-09T00:45:59.646186+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:01:49.690226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.871817+0000","last_change":"2024-05-09T00:44:23.180515+0000","last_active":"2024-05-09T00:45:51.871817+0000","last_peered":"2024-05-09T00:45:51.871817+0000","last_clean":"2024-05-09T00:45:51.871817+0000","last_became_active":"2024-05-09T00:44:23.179834+0000","last_became_peered":"2024-05-09T00:44:23.179834+0000","last_unstale":"2024-05-09T00:45:51.871817+0000","last_undegraded":"2024-05-09T00:45:51.871817+0000","last_fullsized":"2024-05-09T00:45:51.871817+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:21:55.735438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,6],"acting":[11,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"78'1","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.941920+0000","last_change":"2024-05-09T00:45:23.941489+0000","last_active":"2024-05-09T00:45:23.941920+0000","last_peered":"2024-05-09T00:45:23.941920+0000","last_clean":"2024-05-09T00:45:23.941920+0000","last_became_active":"2024-05-09T00:45:23.941072+0000","last_became_peered":"2024-05-09T00:45:23.941072+0000","last_unstale":"2024-05-09T00:45:23.941920+0000","last_undegraded":"2024-05-09T00:45:23.941920+0000","last_fullsized":"2024-05-09T00:45:23.941920+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:20:05.755293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.849757+0000","last_change":"2024-05-09T00:43:43.886618+0000","last_active":"2024-05-09T00:45:51.849757+0000","last_peered":"2024-05-09T00:45:51.849757+0000","last_clean":"2024-05-09T00:45:51.849757+0000","last_became_active":"2024-05-09T00:43:43.885946+0000","last_became_peered":"2024-05-09T00:43:43.885946+0000","last_unstale":"2024-05-09T00:45:51.849757+0000","last_undegraded":"2024-05-09T00:45:51.849757+0000","last_fullsized":"2024-05-09T00:45:51.849757+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:16:22.169548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"78'5","reported_seq":224,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:47:08.235950+0000","last_change":"2024-05-09T00:43:40.856213+0000","last_active":"2024-05-09T00:47:08.235950+0000","last_peered":"2024-05-09T00:47:08.235950+0000","last_clean":"2024-05-09T00:47:08.235950+0000","last_became_active":"2024-05-09T00:43:40.855700+0000","last_became_peered":"2024-05-09T00:43:40.855700+0000","last_unstale":"2024-05-09T00:47:08.235950+0000","last_undegraded":"2024-05-09T00:47:08.235950+0000","last_fullsized":"2024-05-09T00:47:08.235950+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:48:50.238293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":188,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.857202+0000","last_change":"2024-05-09T00:43:43.885272+0000","last_active":"2024-05-09T00:45:51.857202+0000","last_peered":"2024-05-09T00:45:51.857202+0000","last_clean":"2024-05-09T00:45:51.857202+0000","last_became_active":"2024-05-09T00:43:43.884774+0000","last_became_peered":"2024-05-09T00:43:43.884774+0000","last_unstale":"2024-05-09T00:45:51.857202+0000","last_undegraded":"2024-05-09T00:45:51.857202+0000","last_fullsized":"2024-05-09T00:45:51.857202+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:14:02.193859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.863000+0000","last_change":"2024-05-09T00:43:43.883370+0000","last_active":"2024-05-09T00:45:51.863000+0000","last_peered":"2024-05-09T00:45:51.863000+0000","last_clean":"2024-05-09T00:45:51.863000+0000","last_became_active":"2024-05-09T00:43:43.881570+0000","last_became_peered":"2024-05-09T00:43:43.881570+0000","last_unstale":"2024-05-09T00:45:51.863000+0000","last_undegraded":"2024-05-09T00:45:51.863000+0000","last_fullsized":"2024-05-09T00:45:51.863000+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:48.936861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'7","reported_seq":225,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:47:07.234443+0000","last_change":"2024-05-09T00:43:40.959828+0000","last_active":"2024-05-09T00:47:07.234443+0000","last_peered":"2024-05-09T00:47:07.234443+0000","last_clean":"2024-05-09T00:47:07.234443+0000","last_became_active":"2024-05-09T00:43:40.958889+0000","last_became_peered":"2024-05-09T00:43:40.958889+0000","last_unstale":"2024-05-09T00:47:07.234443+0000","last_undegraded":"2024-05-09T00:47:07.234443+0000","last_fullsized":"2024-05-09T00:47:07.234443+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:24:50.113298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":187,"num_read_kb":187,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.861559+0000","last_change":"2024-05-09T00:43:43.876144+0000","last_active":"2024-05-09T00:45:51.861559+0000","last_peered":"2024-05-09T00:45:51.861559+0000","last_clean":"2024-05-09T00:45:51.861559+0000","last_became_active":"2024-05-09T00:43:43.875774+0000","last_became_peered":"2024-05-09T00:43:43.875774+0000","last_unstale":"2024-05-09T00:45:51.861559+0000","last_undegraded":"2024-05-09T00:45:51.861559+0000","last_fullsized":"2024-05-09T00:45:51.861559+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:05:23.145431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"78'6","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.939598+0000","last_change":"2024-05-09T00:43:40.857278+0000","last_active":"2024-05-09T00:45:23.939598+0000","last_peered":"2024-05-09T00:45:23.939598+0000","last_clean":"2024-05-09T00:45:23.939598+0000","last_became_active":"2024-05-09T00:43:40.856861+0000","last_became_peered":"2024-05-09T00:43:40.856861+0000","last_unstale":"2024-05-09T00:45:23.939598+0000","last_undegraded":"2024-05-09T00:45:23.939598+0000","last_fullsized":"2024-05-09T00:45:23.939598+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:36:16.603327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":104,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.080451+0000","last_change":"2024-05-09T00:42:14.528544+0000","last_active":"2024-05-09T00:45:24.080451+0000","last_peered":"2024-05-09T00:45:24.080451+0000","last_clean":"2024-05-09T00:45:24.080451+0000","last_became_active":"2024-05-09T00:42:14.523658+0000","last_became_peered":"2024-05-09T00:42:14.523658+0000","last_unstale":"2024-05-09T00:45:24.080451+0000","last_undegraded":"2024-05-09T00:45:24.080451+0000","last_fullsized":"2024-05-09T00:45:24.080451+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:38:48.637995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:38:48.637995+0000","last_clean_scrub_stamp":"2024-05-09T00:38:48.637995+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:42:28.249240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.852620+0000","last_change":"2024-05-09T00:43:43.873638+0000","last_active":"2024-05-09T00:45:51.852620+0000","last_peered":"2024-05-09T00:45:51.852620+0000","last_clean":"2024-05-09T00:45:51.852620+0000","last_became_active":"2024-05-09T00:43:43.871069+0000","last_became_peered":"2024-05-09T00:43:43.871069+0000","last_unstale":"2024-05-09T00:45:51.852620+0000","last_undegraded":"2024-05-09T00:45:51.852620+0000","last_fullsized":"2024-05-09T00:45:51.852620+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:28:34.996989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"78'4","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.080673+0000","last_change":"2024-05-09T00:43:40.856374+0000","last_active":"2024-05-09T00:45:24.080673+0000","last_peered":"2024-05-09T00:45:24.080673+0000","last_clean":"2024-05-09T00:45:24.080673+0000","last_became_active":"2024-05-09T00:43:40.855695+0000","last_became_peered":"2024-05-09T00:43:40.855695+0000","last_unstale":"2024-05-09T00:45:24.080673+0000","last_undegraded":"2024-05-09T00:45:24.080673+0000","last_fullsized":"2024-05-09T00:45:24.080673+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:33:43.734132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"78'4","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.937141+0000","last_change":"2024-05-09T00:43:40.959303+0000","last_active":"2024-05-09T00:45:23.937141+0000","last_peered":"2024-05-09T00:45:23.937141+0000","last_clean":"2024-05-09T00:45:23.937141+0000","last_became_active":"2024-05-09T00:43:40.958855+0000","last_became_peered":"2024-05-09T00:43:40.958855+0000","last_unstale":"2024-05-09T00:45:23.937141+0000","last_undegraded":"2024-05-09T00:45:23.937141+0000","last_fullsized":"2024-05-09T00:45:23.937141+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:53:22.566850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.861738+0000","last_change":"2024-05-09T00:43:43.872041+0000","last_active":"2024-05-09T00:45:51.861738+0000","last_peered":"2024-05-09T00:45:51.861738+0000","last_clean":"2024-05-09T00:45:51.861738+0000","last_became_active":"2024-05-09T00:43:43.870946+0000","last_became_peered":"2024-05-09T00:43:43.870946+0000","last_unstale":"2024-05-09T00:45:51.861738+0000","last_undegraded":"2024-05-09T00:45:51.861738+0000","last_fullsized":"2024-05-09T00:45:51.861738+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:16:28.190488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'17","reported_seq":48,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:52.555000+0000","last_change":"2024-05-09T00:43:40.853875+0000","last_active":"2024-05-09T00:45:52.555000+0000","last_peered":"2024-05-09T00:45:52.555000+0000","last_clean":"2024-05-09T00:45:52.555000+0000","last_became_active":"2024-05-09T00:43:40.853459+0000","last_became_peered":"2024-05-09T00:43:40.853459+0000","last_unstale":"2024-05-09T00:45:52.555000+0000","last_undegraded":"2024-05-09T00:45:52.555000+0000","last_fullsized":"2024-05-09T00:45:52.555000+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:45:29.992730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.876758+0000","last_change":"2024-05-09T00:43:43.870435+0000","last_active":"2024-05-09T00:45:51.876758+0000","last_peered":"2024-05-09T00:45:51.876758+0000","last_clean":"2024-05-09T00:45:51.876758+0000","last_became_active":"2024-05-09T00:43:43.870075+0000","last_became_peered":"2024-05-09T00:43:43.870075+0000","last_unstale":"2024-05-09T00:45:51.876758+0000","last_undegraded":"2024-05-09T00:45:51.876758+0000","last_fullsized":"2024-05-09T00:45:51.876758+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:53:12.958463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'7","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.939824+0000","last_change":"2024-05-09T00:43:41.119325+0000","last_active":"2024-05-09T00:45:23.939824+0000","last_peered":"2024-05-09T00:45:23.939824+0000","last_clean":"2024-05-09T00:45:23.939824+0000","last_became_active":"2024-05-09T00:43:41.119009+0000","last_became_peered":"2024-05-09T00:43:41.119009+0000","last_unstale":"2024-05-09T00:45:23.939824+0000","last_undegraded":"2024-05-09T00:45:23.939824+0000","last_fullsized":"2024-05-09T00:45:23.939824+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:10:05.775152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.873671+0000","last_change":"2024-05-09T00:43:43.875236+0000","last_active":"2024-05-09T00:45:51.873671+0000","last_peered":"2024-05-09T00:45:51.873671+0000","last_clean":"2024-05-09T00:45:51.873671+0000","last_became_active":"2024-05-09T00:43:43.874919+0000","last_became_peered":"2024-05-09T00:43:43.874919+0000","last_unstale":"2024-05-09T00:45:51.873671+0000","last_undegraded":"2024-05-09T00:45:51.873671+0000","last_fullsized":"2024-05-09T00:45:51.873671+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:45:33.232518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.938556+0000","last_change":"2024-05-09T00:43:40.959765+0000","last_active":"2024-05-09T00:45:23.938556+0000","last_peered":"2024-05-09T00:45:23.938556+0000","last_clean":"2024-05-09T00:45:23.938556+0000","last_became_active":"2024-05-09T00:43:40.959383+0000","last_became_peered":"2024-05-09T00:43:40.959383+0000","last_unstale":"2024-05-09T00:45:23.938556+0000","last_undegraded":"2024-05-09T00:45:23.938556+0000","last_fullsized":"2024-05-09T00:45:23.938556+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:13:28.766650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.870027+0000","last_change":"2024-05-09T00:43:43.881627+0000","last_active":"2024-05-09T00:45:51.870027+0000","last_peered":"2024-05-09T00:45:51.870027+0000","last_clean":"2024-05-09T00:45:51.870027+0000","last_became_active":"2024-05-09T00:43:43.881207+0000","last_became_peered":"2024-05-09T00:43:43.881207+0000","last_unstale":"2024-05-09T00:45:51.870027+0000","last_undegraded":"2024-05-09T00:45:51.870027+0000","last_fullsized":"2024-05-09T00:45:51.870027+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:44:08.053638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'7","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.115513+0000","last_change":"2024-05-09T00:43:40.859566+0000","last_active":"2024-05-09T00:45:24.115513+0000","last_peered":"2024-05-09T00:45:24.115513+0000","last_clean":"2024-05-09T00:45:24.115513+0000","last_became_active":"2024-05-09T00:43:40.859182+0000","last_became_peered":"2024-05-09T00:43:40.859182+0000","last_unstale":"2024-05-09T00:45:24.115513+0000","last_undegraded":"2024-05-09T00:45:24.115513+0000","last_fullsized":"2024-05-09T00:45:24.115513+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:58:12.693271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.874341+0000","last_change":"2024-05-09T00:43:43.869193+0000","last_active":"2024-05-09T00:45:51.874341+0000","last_peered":"2024-05-09T00:45:51.874341+0000","last_clean":"2024-05-09T00:45:51.874341+0000","last_became_active":"2024-05-09T00:43:43.868823+0000","last_became_peered":"2024-05-09T00:43:43.868823+0000","last_unstale":"2024-05-09T00:45:51.874341+0000","last_undegraded":"2024-05-09T00:45:51.874341+0000","last_fullsized":"2024-05-09T00:45:51.874341+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:15:53.929873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'3","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.116148+0000","last_change":"2024-05-09T00:43:40.855038+0000","last_active":"2024-05-09T00:45:24.116148+0000","last_peered":"2024-05-09T00:45:24.116148+0000","last_clean":"2024-05-09T00:45:24.116148+0000","last_became_active":"2024-05-09T00:43:40.854334+0000","last_became_peered":"2024-05-09T00:43:40.854334+0000","last_unstale":"2024-05-09T00:45:24.116148+0000","last_undegraded":"2024-05-09T00:45:24.116148+0000","last_fullsized":"2024-05-09T00:45:24.116148+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:47:13.897697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.863165+0000","last_change":"2024-05-09T00:44:23.019151+0000","last_active":"2024-05-09T00:45:51.863165+0000","last_peered":"2024-05-09T00:45:51.863165+0000","last_clean":"2024-05-09T00:45:51.863165+0000","last_became_active":"2024-05-09T00:44:23.018752+0000","last_became_peered":"2024-05-09T00:44:23.018752+0000","last_unstale":"2024-05-09T00:45:51.863165+0000","last_undegraded":"2024-05-09T00:45:51.863165+0000","last_fullsized":"2024-05-09T00:45:51.863165+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:28:41.875747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.937463+0000","last_change":"2024-05-09T00:43:40.859461+0000","last_active":"2024-05-09T00:45:23.937463+0000","last_peered":"2024-05-09T00:45:23.937463+0000","last_clean":"2024-05-09T00:45:23.937463+0000","last_became_active":"2024-05-09T00:43:40.857814+0000","last_became_peered":"2024-05-09T00:43:40.857814+0000","last_unstale":"2024-05-09T00:45:23.937463+0000","last_undegraded":"2024-05-09T00:45:23.937463+0000","last_fullsized":"2024-05-09T00:45:23.937463+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:56:06.279762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.877886+0000","last_change":"2024-05-09T00:43:43.878120+0000","last_active":"2024-05-09T00:45:51.877886+0000","last_peered":"2024-05-09T00:45:51.877886+0000","last_clean":"2024-05-09T00:45:51.877886+0000","last_became_active":"2024-05-09T00:43:43.877782+0000","last_became_peered":"2024-05-09T00:43:43.877782+0000","last_unstale":"2024-05-09T00:45:51.877886+0000","last_undegraded":"2024-05-09T00:45:51.877886+0000","last_fullsized":"2024-05-09T00:45:51.877886+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:19:48.558992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'7","reported_seq":227,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:47:06.304442+0000","last_change":"2024-05-09T00:43:41.119347+0000","last_active":"2024-05-09T00:47:06.304442+0000","last_peered":"2024-05-09T00:47:06.304442+0000","last_clean":"2024-05-09T00:47:06.304442+0000","last_became_active":"2024-05-09T00:43:41.118847+0000","last_became_peered":"2024-05-09T00:43:41.118847+0000","last_unstale":"2024-05-09T00:47:06.304442+0000","last_undegraded":"2024-05-09T00:47:06.304442+0000","last_fullsized":"2024-05-09T00:47:06.304442+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:07:03.977756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":189,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.878473+0000","last_change":"2024-05-09T00:43:43.875203+0000","last_active":"2024-05-09T00:45:51.878473+0000","last_peered":"2024-05-09T00:45:51.878473+0000","last_clean":"2024-05-09T00:45:51.878473+0000","last_became_active":"2024-05-09T00:43:43.874882+0000","last_became_peered":"2024-05-09T00:43:43.874882+0000","last_unstale":"2024-05-09T00:45:51.878473+0000","last_undegraded":"2024-05-09T00:45:51.878473+0000","last_fullsized":"2024-05-09T00:45:51.878473+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:54:27.845086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'3","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.938177+0000","last_change":"2024-05-09T00:43:40.857955+0000","last_active":"2024-05-09T00:45:23.938177+0000","last_peered":"2024-05-09T00:45:23.938177+0000","last_clean":"2024-05-09T00:45:23.938177+0000","last_became_active":"2024-05-09T00:43:40.857596+0000","last_became_peered":"2024-05-09T00:43:40.857596+0000","last_unstale":"2024-05-09T00:45:23.938177+0000","last_undegraded":"2024-05-09T00:45:23.938177+0000","last_fullsized":"2024-05-09T00:45:23.938177+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:13:56.676929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.872219+0000","last_change":"2024-05-09T00:43:43.882487+0000","last_active":"2024-05-09T00:45:51.872219+0000","last_peered":"2024-05-09T00:45:51.872219+0000","last_clean":"2024-05-09T00:45:51.872219+0000","last_became_active":"2024-05-09T00:43:43.882176+0000","last_became_peered":"2024-05-09T00:43:43.882176+0000","last_unstale":"2024-05-09T00:45:51.872219+0000","last_undegraded":"2024-05-09T00:45:51.872219+0000","last_fullsized":"2024-05-09T00:45:51.872219+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:48:42.354433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'5","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.934749+0000","last_change":"2024-05-09T00:43:40.857215+0000","last_active":"2024-05-09T00:45:23.934749+0000","last_peered":"2024-05-09T00:45:23.934749+0000","last_clean":"2024-05-09T00:45:23.934749+0000","last_became_active":"2024-05-09T00:43:40.856858+0000","last_became_peered":"2024-05-09T00:43:40.856858+0000","last_unstale":"2024-05-09T00:45:23.934749+0000","last_undegraded":"2024-05-09T00:45:23.934749+0000","last_fullsized":"2024-05-09T00:45:23.934749+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:40:47.943482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.878949+0000","last_change":"2024-05-09T00:43:43.881192+0000","last_active":"2024-05-09T00:45:51.878949+0000","last_peered":"2024-05-09T00:45:51.878949+0000","last_clean":"2024-05-09T00:45:51.878949+0000","last_became_active":"2024-05-09T00:43:43.880781+0000","last_became_peered":"2024-05-09T00:43:43.880781+0000","last_unstale":"2024-05-09T00:45:51.878949+0000","last_undegraded":"2024-05-09T00:45:51.878949+0000","last_fullsized":"2024-05-09T00:45:51.878949+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:18:31.577354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.870420+0000","last_change":"2024-05-09T00:43:43.876869+0000","last_active":"2024-05-09T00:45:51.870420+0000","last_peered":"2024-05-09T00:45:51.870420+0000","last_clean":"2024-05-09T00:45:51.870420+0000","last_became_active":"2024-05-09T00:43:43.876439+0000","last_became_peered":"2024-05-09T00:43:43.876439+0000","last_unstale":"2024-05-09T00:45:51.870420+0000","last_undegraded":"2024-05-09T00:45:51.870420+0000","last_fullsized":"2024-05-09T00:45:51.870420+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:39:06.776267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.871096+0000","last_change":"2024-05-09T00:43:43.884677+0000","last_active":"2024-05-09T00:45:51.871096+0000","last_peered":"2024-05-09T00:45:51.871096+0000","last_clean":"2024-05-09T00:45:51.871096+0000","last_became_active":"2024-05-09T00:43:43.884238+0000","last_became_peered":"2024-05-09T00:43:43.884238+0000","last_unstale":"2024-05-09T00:45:51.871096+0000","last_undegraded":"2024-05-09T00:45:51.871096+0000","last_fullsized":"2024-05-09T00:45:51.871096+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:18:13.787584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.867942+0000","last_change":"2024-05-09T00:43:43.876638+0000","last_active":"2024-05-09T00:45:51.867942+0000","last_peered":"2024-05-09T00:45:51.867942+0000","last_clean":"2024-05-09T00:45:51.867942+0000","last_became_active":"2024-05-09T00:43:43.876315+0000","last_became_peered":"2024-05-09T00:43:43.876315+0000","last_unstale":"2024-05-09T00:45:51.867942+0000","last_undegraded":"2024-05-09T00:45:51.867942+0000","last_fullsized":"2024-05-09T00:45:51.867942+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:22:41.618613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.867553+0000","last_change":"2024-05-09T00:43:43.869798+0000","last_active":"2024-05-09T00:45:51.867553+0000","last_peered":"2024-05-09T00:45:51.867553+0000","last_clean":"2024-05-09T00:45:51.867553+0000","last_became_active":"2024-05-09T00:43:43.869444+0000","last_became_peered":"2024-05-09T00:43:43.869444+0000","last_unstale":"2024-05-09T00:45:51.867553+0000","last_undegraded":"2024-05-09T00:45:51.867553+0000","last_fullsized":"2024-05-09T00:45:51.867553+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:15:53.267441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.858958+0000","last_change":"2024-05-09T00:43:43.873215+0000","last_active":"2024-05-09T00:45:51.858958+0000","last_peered":"2024-05-09T00:45:51.858958+0000","last_clean":"2024-05-09T00:45:51.858958+0000","last_became_active":"2024-05-09T00:43:43.872763+0000","last_became_peered":"2024-05-09T00:43:43.872763+0000","last_unstale":"2024-05-09T00:45:51.858958+0000","last_undegraded":"2024-05-09T00:45:51.858958+0000","last_fullsized":"2024-05-09T00:45:51.858958+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:08.997387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.870845+0000","last_change":"2024-05-09T00:43:43.876514+0000","last_active":"2024-05-09T00:45:51.870845+0000","last_peered":"2024-05-09T00:45:51.870845+0000","last_clean":"2024-05-09T00:45:51.870845+0000","last_became_active":"2024-05-09T00:43:43.876147+0000","last_became_peered":"2024-05-09T00:43:43.876147+0000","last_unstale":"2024-05-09T00:45:51.870845+0000","last_undegraded":"2024-05-09T00:45:51.870845+0000","last_fullsized":"2024-05-09T00:45:51.870845+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:58:34.745819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.864498+0000","last_change":"2024-05-09T00:43:43.871869+0000","last_active":"2024-05-09T00:45:51.864498+0000","last_peered":"2024-05-09T00:45:51.864498+0000","last_clean":"2024-05-09T00:45:51.864498+0000","last_became_active":"2024-05-09T00:43:43.871447+0000","last_became_peered":"2024-05-09T00:43:43.871447+0000","last_unstale":"2024-05-09T00:45:51.864498+0000","last_undegraded":"2024-05-09T00:45:51.864498+0000","last_fullsized":"2024-05-09T00:45:51.864498+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:11:42.128931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.865595+0000","last_change":"2024-05-09T00:43:43.871291+0000","last_active":"2024-05-09T00:45:51.865595+0000","last_peered":"2024-05-09T00:45:51.865595+0000","last_clean":"2024-05-09T00:45:51.865595+0000","last_became_active":"2024-05-09T00:43:43.870899+0000","last_became_peered":"2024-05-09T00:43:43.870899+0000","last_unstale":"2024-05-09T00:45:51.865595+0000","last_undegraded":"2024-05-09T00:45:51.865595+0000","last_fullsized":"2024-05-09T00:45:51.865595+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:28:53.013543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.864359+0000","last_change":"2024-05-09T00:43:43.884791+0000","last_active":"2024-05-09T00:45:51.864359+0000","last_peered":"2024-05-09T00:45:51.864359+0000","last_clean":"2024-05-09T00:45:51.864359+0000","last_became_active":"2024-05-09T00:43:43.877333+0000","last_became_peered":"2024-05-09T00:43:43.877333+0000","last_unstale":"2024-05-09T00:45:51.864359+0000","last_undegraded":"2024-05-09T00:45:51.864359+0000","last_fullsized":"2024-05-09T00:45:51.864359+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:37:07.535388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.874993+0000","last_change":"2024-05-09T00:43:43.863773+0000","last_active":"2024-05-09T00:45:51.874993+0000","last_peered":"2024-05-09T00:45:51.874993+0000","last_clean":"2024-05-09T00:45:51.874993+0000","last_became_active":"2024-05-09T00:43:43.863436+0000","last_became_peered":"2024-05-09T00:43:43.863436+0000","last_unstale":"2024-05-09T00:45:51.874993+0000","last_undegraded":"2024-05-09T00:45:51.874993+0000","last_fullsized":"2024-05-09T00:45:51.874993+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:59:57.100234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.880217+0000","last_change":"2024-05-09T00:43:43.873461+0000","last_active":"2024-05-09T00:45:51.880217+0000","last_peered":"2024-05-09T00:45:51.880217+0000","last_clean":"2024-05-09T00:45:51.880217+0000","last_became_active":"2024-05-09T00:43:43.873139+0000","last_became_peered":"2024-05-09T00:43:43.873139+0000","last_unstale":"2024-05-09T00:45:51.880217+0000","last_undegraded":"2024-05-09T00:45:51.880217+0000","last_fullsized":"2024-05-09T00:45:51.880217+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:24:57.865569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.867837+0000","last_change":"2024-05-09T00:43:43.874401+0000","last_active":"2024-05-09T00:45:51.867837+0000","last_peered":"2024-05-09T00:45:51.867837+0000","last_clean":"2024-05-09T00:45:51.867837+0000","last_became_active":"2024-05-09T00:43:43.873765+0000","last_became_peered":"2024-05-09T00:43:43.873765+0000","last_unstale":"2024-05-09T00:45:51.867837+0000","last_undegraded":"2024-05-09T00:45:51.867837+0000","last_fullsized":"2024-05-09T00:45:51.867837+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:23:59.509415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.877398+0000","last_change":"2024-05-09T00:43:43.875219+0000","last_active":"2024-05-09T00:45:51.877398+0000","last_peered":"2024-05-09T00:45:51.877398+0000","last_clean":"2024-05-09T00:45:51.877398+0000","last_became_active":"2024-05-09T00:43:43.874824+0000","last_became_peered":"2024-05-09T00:43:43.874824+0000","last_unstale":"2024-05-09T00:45:51.877398+0000","last_undegraded":"2024-05-09T00:45:51.877398+0000","last_fullsized":"2024-05-09T00:45:51.877398+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:39:41.929521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":262144,"data_stored":2951840,"data_compressed":179218,"data_compressed_allocated":225280,"data_compressed_original":2932736,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71218,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1894,"num_read_kb":1016,"num_write":266,"num_write_kb":154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":53265,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":540672,"data_stored":213654,"data_compressed":8493,"data_compressed_allocated":12288,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},"log_size":228,"ondisk_log_size":228,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710777,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":901120,"data_stored":443946,"data_compressed":5799,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877907007,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":1097593,"data_compressed":41967,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070532,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":442762,"data_compressed":5799,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266761,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":913408,"data_stored":442558,"data_compressed":5799,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462991,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":1036639,"data_compressed":41967,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626516,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":905216,"data_stored":442479,"data_compressed":5799,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574957,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":970752,"data_stored":506284,"data_compressed":5799,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738482,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":925696,"data_stored":444350,"data_compressed":5799,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411431,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":917504,"data_stored":444676,"data_compressed":5799,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182499,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36236,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708916,"statfs":{"total":95995035648,"available":95957929984,"internally_reserved":0,"allocated":921600,"data_stored":442304,"data_compressed":5799,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986269,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":506072,"data_compressed":14292,"data_compressed_allocated":20480,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822745,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":1035477,"data_compressed":41967,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34546,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36168,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36168,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36168,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36168,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":64644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":64432,"data_compressed":8493,"data_compressed_allocated":12288,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":65468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T00:47:11.979 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T00:47:12.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.128+0000 7fd50b984640 1 -- 172.21.15.5:0/2037000463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd504103d50 msgr2=0x7fd504104160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:12.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.128+0000 7fd50b984640 1 --2- 172.21.15.5:0/2037000463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd504103d50 0x7fd504104160 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd4f8009ca0 tx=0x7fd4f802a8c0 comp rx=0 tx=0).stop 2024-05-09T00:47:12.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.128+0000 7fd50b984640 1 -- 172.21.15.5:0/2037000463 shutdown_connections 2024-05-09T00:47:12.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.128+0000 7fd50b984640 1 --2- 172.21.15.5:0/2037000463 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd504108a50 0x7fd50410ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:12.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.128+0000 7fd50b984640 1 --2- 172.21.15.5:0/2037000463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd504103d50 0x7fd504104160 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:12.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.128+0000 7fd50b984640 1 --2- 172.21.15.5:0/2037000463 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd504102b10 0x7fd504103810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:12.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.128+0000 7fd50b984640 1 -- 172.21.15.5:0/2037000463 >> 172.21.15.5:0/2037000463 conn(0x7fd5040fdd20 msgr2=0x7fd5041001b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:12.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.129+0000 7fd50b984640 1 -- 172.21.15.5:0/2037000463 shutdown_connections 2024-05-09T00:47:12.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.129+0000 7fd50b984640 1 -- 172.21.15.5:0/2037000463 wait complete. 2024-05-09T00:47:12.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.129+0000 7fd50b984640 1 Processor -- start 2024-05-09T00:47:12.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.129+0000 7fd50b984640 1 -- start start 2024-05-09T00:47:12.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.129+0000 7fd50b984640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd504102b10 0x7fd5041992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:12.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.130+0000 7fd50b984640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd504103d50 0x7fd504199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:12.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.130+0000 7fd50b984640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd504108a50 0x7fd5041a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:12.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.130+0000 7fd50b984640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd50410e630 con 0x7fd504108a50 2024-05-09T00:47:12.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.130+0000 7fd50b984640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd50410e4b0 con 0x7fd504102b10 2024-05-09T00:47:12.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.130+0000 7fd50b984640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd50410e7b0 con 0x7fd504103d50 2024-05-09T00:47:12.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.130+0000 7fd509f21640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd504108a50 0x7fd5041a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:12.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.130+0000 7fd509f21640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd504108a50 0x7fd5041a0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34480/0 (socket says 172.21.15.5:34480) 2024-05-09T00:47:12.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.130+0000 7fd509720640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd504102b10 0x7fd5041992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:12.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.130+0000 7fd509f21640 1 -- 172.21.15.5:0/460173863 learned_addr learned my addr 172.21.15.5:0/460173863 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:47:12.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.130+0000 7fd508f1f640 1 --2- 172.21.15.5:0/460173863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd504103d50 0x7fd504199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:12.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.130+0000 7fd509f21640 1 -- 172.21.15.5:0/460173863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd504103d50 msgr2=0x7fd504199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:12.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.130+0000 7fd509f21640 1 --2- 172.21.15.5:0/460173863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd504103d50 0x7fd504199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:12.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.130+0000 7fd509f21640 1 -- 172.21.15.5:0/460173863 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd504102b10 msgr2=0x7fd5041992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:12.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.130+0000 7fd509f21640 1 --2- 172.21.15.5:0/460173863 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd504102b10 0x7fd5041992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:12.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.130+0000 7fd509f21640 1 -- 172.21.15.5:0/460173863 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5041a0de0 con 0x7fd504108a50 2024-05-09T00:47:12.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.130+0000 7fd509f21640 1 --2- 172.21.15.5:0/460173863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd504108a50 0x7fd5041a0770 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc00e970 tx=0x7fd4fc00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:12.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.130+0000 7fd4f67fc640 1 -- 172.21.15.5:0/460173863 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4fc00ccc0 con 0x7fd504108a50 2024-05-09T00:47:12.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.130+0000 7fd4f67fc640 1 -- 172.21.15.5:0/460173863 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4fc00ce60 con 0x7fd504108a50 2024-05-09T00:47:12.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.130+0000 7fd50b984640 1 -- 172.21.15.5:0/460173863 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5041a10d0 con 0x7fd504108a50 2024-05-09T00:47:12.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.130+0000 7fd4f67fc640 1 -- 172.21.15.5:0/460173863 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4fc010430 con 0x7fd504108a50 2024-05-09T00:47:12.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.131+0000 7fd50b984640 1 -- 172.21.15.5:0/460173863 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5041a15d0 con 0x7fd504108a50 2024-05-09T00:47:12.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.131+0000 7fd50b984640 1 -- 172.21.15.5:0/460173863 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd504107860 con 0x7fd504108a50 2024-05-09T00:47:12.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.133+0000 7fd4f67fc640 1 -- 172.21.15.5:0/460173863 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fd4fc004fb0 con 0x7fd504108a50 2024-05-09T00:47:12.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.134+0000 7fd4f67fc640 1 --2- 172.21.15.5:0/460173863 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fd4f00b27f0 0x7fd4f00b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:12.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.134+0000 7fd509720640 1 --2- 172.21.15.5:0/460173863 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fd4f00b27f0 0x7fd4f00b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:12.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.134+0000 7fd4f67fc640 1 -- 172.21.15.5:0/460173863 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fd4fc0e2860 con 0x7fd504108a50 2024-05-09T00:47:12.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.134+0000 7fd509720640 1 --2- 172.21.15.5:0/460173863 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fd4f00b27f0 0x7fd4f00b4d00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd504103710 tx=0x7fd500005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:12.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.137+0000 7fd4f67fc640 1 -- 172.21.15.5:0/460173863 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd4fc054df0 con 0x7fd504108a50 2024-05-09T00:47:12.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.329+0000 7fd50b984640 1 -- 172.21.15.5:0/460173863 --> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd504063110 con 0x7fd4f00b27f0 2024-05-09T00:47:12.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.336+0000 7fd4f67fc640 1 -- 172.21.15.5:0/460173863 <== mgr.14152 v2:172.21.15.5:6800/91856593 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259094 (secure 0 0 0) 0x7fd504063110 con 0x7fd4f00b27f0 2024-05-09T00:47:12.340 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:47:12.341 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-09T00:47:12.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.339+0000 7fd50b984640 1 -- 172.21.15.5:0/460173863 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fd4f00b27f0 msgr2=0x7fd4f00b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:12.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.339+0000 7fd50b984640 1 --2- 172.21.15.5:0/460173863 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fd4f00b27f0 0x7fd4f00b4d00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd504103710 tx=0x7fd500005f90 comp rx=0 tx=0).stop 2024-05-09T00:47:12.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.339+0000 7fd50b984640 1 -- 172.21.15.5:0/460173863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd504108a50 msgr2=0x7fd5041a0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:12.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.339+0000 7fd50b984640 1 --2- 172.21.15.5:0/460173863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd504108a50 0x7fd5041a0770 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc00e970 tx=0x7fd4fc00ed70 comp rx=0 tx=0).stop 2024-05-09T00:47:12.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.339+0000 7fd50b984640 1 -- 172.21.15.5:0/460173863 shutdown_connections 2024-05-09T00:47:12.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.339+0000 7fd50b984640 1 --2- 172.21.15.5:0/460173863 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fd4f00b27f0 0x7fd4f00b4d00 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:12.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.339+0000 7fd50b984640 1 --2- 172.21.15.5:0/460173863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd504108a50 0x7fd5041a0770 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:12.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.339+0000 7fd50b984640 1 --2- 172.21.15.5:0/460173863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd504103d50 0x7fd504199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:12.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.339+0000 7fd50b984640 1 --2- 172.21.15.5:0/460173863 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd504102b10 0x7fd5041992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:12.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.340+0000 7fd50b984640 1 -- 172.21.15.5:0/460173863 >> 172.21.15.5:0/460173863 conn(0x7fd5040fdd20 msgr2=0x7fd504104a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:12.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.340+0000 7fd50b984640 1 -- 172.21.15.5:0/460173863 shutdown_connections 2024-05-09T00:47:12.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.340+0000 7fd50b984640 1 -- 172.21.15.5:0/460173863 wait complete. 2024-05-09T00:47:12.352 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":397,"stamp":"2024-05-09T00:47:11.913507+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661820,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1970,"num_read_kb":1080,"num_write":391,"num_write_kb":2306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":3005105,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":822,"ondisk_log_size":822,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":351896,"kb_used_data":11064,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124589928,"statfs":{"total":1151940427776,"available":1151580086272,"internally_reserved":0,"allocated":11329536,"data_stored":7285140,"data_compressed":186585,"data_compressed_allocated":245760,"data_compressed_original":2015232,"omap_allocated":19141,"internal_metadata":348894523},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":616,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":107,"num_read_kb":52,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003299"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.877185+0000","last_change":"2024-05-09T00:43:43.875357+0000","last_active":"2024-05-09T00:45:51.877185+0000","last_peered":"2024-05-09T00:45:51.877185+0000","last_clean":"2024-05-09T00:45:51.877185+0000","last_became_active":"2024-05-09T00:43:43.873765+0000","last_became_peered":"2024-05-09T00:43:43.873765+0000","last_unstale":"2024-05-09T00:45:51.877185+0000","last_undegraded":"2024-05-09T00:45:51.877185+0000","last_fullsized":"2024-05-09T00:45:51.877185+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:47.519110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.875117+0000","last_change":"2024-05-09T00:43:43.882464+0000","last_active":"2024-05-09T00:45:51.875117+0000","last_peered":"2024-05-09T00:45:51.875117+0000","last_clean":"2024-05-09T00:45:51.875117+0000","last_became_active":"2024-05-09T00:43:43.882102+0000","last_became_peered":"2024-05-09T00:43:43.882102+0000","last_unstale":"2024-05-09T00:45:51.875117+0000","last_undegraded":"2024-05-09T00:45:51.875117+0000","last_fullsized":"2024-05-09T00:45:51.875117+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:39:36.234474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.872356+0000","last_change":"2024-05-09T00:43:43.873098+0000","last_active":"2024-05-09T00:45:51.872356+0000","last_peered":"2024-05-09T00:45:51.872356+0000","last_clean":"2024-05-09T00:45:51.872356+0000","last_became_active":"2024-05-09T00:43:43.872764+0000","last_became_peered":"2024-05-09T00:43:43.872764+0000","last_unstale":"2024-05-09T00:45:51.872356+0000","last_undegraded":"2024-05-09T00:45:51.872356+0000","last_fullsized":"2024-05-09T00:45:51.872356+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:05:30.784111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.869925+0000","last_change":"2024-05-09T00:43:43.869807+0000","last_active":"2024-05-09T00:45:51.869925+0000","last_peered":"2024-05-09T00:45:51.869925+0000","last_clean":"2024-05-09T00:45:51.869925+0000","last_became_active":"2024-05-09T00:43:43.869470+0000","last_became_peered":"2024-05-09T00:43:43.869470+0000","last_unstale":"2024-05-09T00:45:51.869925+0000","last_undegraded":"2024-05-09T00:45:51.869925+0000","last_fullsized":"2024-05-09T00:45:51.869925+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:25:40.201844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.852785+0000","last_change":"2024-05-09T00:43:43.874110+0000","last_active":"2024-05-09T00:45:51.852785+0000","last_peered":"2024-05-09T00:45:51.852785+0000","last_clean":"2024-05-09T00:45:51.852785+0000","last_became_active":"2024-05-09T00:43:43.873759+0000","last_became_peered":"2024-05-09T00:43:43.873759+0000","last_unstale":"2024-05-09T00:45:51.852785+0000","last_undegraded":"2024-05-09T00:45:51.852785+0000","last_fullsized":"2024-05-09T00:45:51.852785+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:17:17.722319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.862325+0000","last_change":"2024-05-09T00:44:23.022285+0000","last_active":"2024-05-09T00:45:51.862325+0000","last_peered":"2024-05-09T00:45:51.862325+0000","last_clean":"2024-05-09T00:45:51.862325+0000","last_became_active":"2024-05-09T00:44:23.021887+0000","last_became_peered":"2024-05-09T00:44:23.021887+0000","last_unstale":"2024-05-09T00:45:51.862325+0000","last_undegraded":"2024-05-09T00:45:51.862325+0000","last_fullsized":"2024-05-09T00:45:51.862325+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:07:07.702178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,7],"acting":[10,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.873552+0000","last_change":"2024-05-09T00:43:43.883112+0000","last_active":"2024-05-09T00:45:51.873552+0000","last_peered":"2024-05-09T00:45:51.873552+0000","last_clean":"2024-05-09T00:45:51.873552+0000","last_became_active":"2024-05-09T00:43:43.882795+0000","last_became_peered":"2024-05-09T00:43:43.882795+0000","last_unstale":"2024-05-09T00:45:51.873552+0000","last_undegraded":"2024-05-09T00:45:51.873552+0000","last_fullsized":"2024-05-09T00:45:51.873552+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:33:58.724015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.862472+0000","last_change":"2024-05-09T00:43:43.880696+0000","last_active":"2024-05-09T00:45:51.862472+0000","last_peered":"2024-05-09T00:45:51.862472+0000","last_clean":"2024-05-09T00:45:51.862472+0000","last_became_active":"2024-05-09T00:43:43.880174+0000","last_became_peered":"2024-05-09T00:43:43.880174+0000","last_unstale":"2024-05-09T00:45:51.862472+0000","last_undegraded":"2024-05-09T00:45:51.862472+0000","last_fullsized":"2024-05-09T00:45:51.862472+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:40:05.682432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.879518+0000","last_change":"2024-05-09T00:43:43.883449+0000","last_active":"2024-05-09T00:45:51.879518+0000","last_peered":"2024-05-09T00:45:51.879518+0000","last_clean":"2024-05-09T00:45:51.879518+0000","last_became_active":"2024-05-09T00:43:43.883066+0000","last_became_peered":"2024-05-09T00:43:43.883066+0000","last_unstale":"2024-05-09T00:45:51.879518+0000","last_undegraded":"2024-05-09T00:45:51.879518+0000","last_fullsized":"2024-05-09T00:45:51.879518+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:47:49.116004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":41,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.868635+0000","last_change":"2024-05-09T00:43:43.877514+0000","last_active":"2024-05-09T00:45:51.868635+0000","last_peered":"2024-05-09T00:45:51.868635+0000","last_clean":"2024-05-09T00:45:51.868635+0000","last_became_active":"2024-05-09T00:43:43.877170+0000","last_became_peered":"2024-05-09T00:43:43.877170+0000","last_unstale":"2024-05-09T00:45:51.868635+0000","last_undegraded":"2024-05-09T00:45:51.868635+0000","last_fullsized":"2024-05-09T00:45:51.868635+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:47:26.738571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.878663+0000","last_change":"2024-05-09T00:44:23.180294+0000","last_active":"2024-05-09T00:45:51.878663+0000","last_peered":"2024-05-09T00:45:51.878663+0000","last_clean":"2024-05-09T00:45:51.878663+0000","last_became_active":"2024-05-09T00:44:23.179824+0000","last_became_peered":"2024-05-09T00:44:23.179824+0000","last_unstale":"2024-05-09T00:45:51.878663+0000","last_undegraded":"2024-05-09T00:45:51.878663+0000","last_fullsized":"2024-05-09T00:45:51.878663+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:43:20.919660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,9],"acting":[7,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.858638+0000","last_change":"2024-05-09T00:43:43.873836+0000","last_active":"2024-05-09T00:45:51.858638+0000","last_peered":"2024-05-09T00:45:51.858638+0000","last_clean":"2024-05-09T00:45:51.858638+0000","last_became_active":"2024-05-09T00:43:43.873512+0000","last_became_peered":"2024-05-09T00:43:43.873512+0000","last_unstale":"2024-05-09T00:45:51.858638+0000","last_undegraded":"2024-05-09T00:45:51.858638+0000","last_fullsized":"2024-05-09T00:45:51.858638+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:32:06.162638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.864447+0000","last_change":"2024-05-09T00:43:43.885614+0000","last_active":"2024-05-09T00:45:51.864447+0000","last_peered":"2024-05-09T00:45:51.864447+0000","last_clean":"2024-05-09T00:45:51.864447+0000","last_became_active":"2024-05-09T00:43:43.884682+0000","last_became_peered":"2024-05-09T00:43:43.884682+0000","last_unstale":"2024-05-09T00:45:51.864447+0000","last_undegraded":"2024-05-09T00:45:51.864447+0000","last_fullsized":"2024-05-09T00:45:51.864447+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:34:15.336173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.872171+0000","last_change":"2024-05-09T00:43:43.883860+0000","last_active":"2024-05-09T00:45:51.872171+0000","last_peered":"2024-05-09T00:45:51.872171+0000","last_clean":"2024-05-09T00:45:51.872171+0000","last_became_active":"2024-05-09T00:43:43.882870+0000","last_became_peered":"2024-05-09T00:43:43.882870+0000","last_unstale":"2024-05-09T00:45:51.872171+0000","last_undegraded":"2024-05-09T00:45:51.872171+0000","last_fullsized":"2024-05-09T00:45:51.872171+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:17:31.566819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.869693+0000","last_change":"2024-05-09T00:43:43.874114+0000","last_active":"2024-05-09T00:45:51.869693+0000","last_peered":"2024-05-09T00:45:51.869693+0000","last_clean":"2024-05-09T00:45:51.869693+0000","last_became_active":"2024-05-09T00:43:43.873762+0000","last_became_peered":"2024-05-09T00:43:43.873762+0000","last_unstale":"2024-05-09T00:45:51.869693+0000","last_undegraded":"2024-05-09T00:45:51.869693+0000","last_fullsized":"2024-05-09T00:45:51.869693+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:24:27.486100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.879829+0000","last_change":"2024-05-09T00:43:43.872982+0000","last_active":"2024-05-09T00:45:51.879829+0000","last_peered":"2024-05-09T00:45:51.879829+0000","last_clean":"2024-05-09T00:45:51.879829+0000","last_became_active":"2024-05-09T00:43:43.872464+0000","last_became_peered":"2024-05-09T00:43:43.872464+0000","last_unstale":"2024-05-09T00:45:51.879829+0000","last_undegraded":"2024-05-09T00:45:51.879829+0000","last_fullsized":"2024-05-09T00:45:51.879829+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:41.172001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.877289+0000","last_change":"2024-05-09T00:43:43.875662+0000","last_active":"2024-05-09T00:45:51.877289+0000","last_peered":"2024-05-09T00:45:51.877289+0000","last_clean":"2024-05-09T00:45:51.877289+0000","last_became_active":"2024-05-09T00:43:43.874701+0000","last_became_peered":"2024-05-09T00:43:43.874701+0000","last_unstale":"2024-05-09T00:45:51.877289+0000","last_undegraded":"2024-05-09T00:45:51.877289+0000","last_fullsized":"2024-05-09T00:45:51.877289+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:09:19.349190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.864219+0000","last_change":"2024-05-09T00:43:43.876100+0000","last_active":"2024-05-09T00:45:51.864219+0000","last_peered":"2024-05-09T00:45:51.864219+0000","last_clean":"2024-05-09T00:45:51.864219+0000","last_became_active":"2024-05-09T00:43:43.875768+0000","last_became_peered":"2024-05-09T00:43:43.875768+0000","last_unstale":"2024-05-09T00:45:51.864219+0000","last_undegraded":"2024-05-09T00:45:51.864219+0000","last_fullsized":"2024-05-09T00:45:51.864219+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:48:26.843064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.865952+0000","last_change":"2024-05-09T00:43:43.875900+0000","last_active":"2024-05-09T00:45:51.865952+0000","last_peered":"2024-05-09T00:45:51.865952+0000","last_clean":"2024-05-09T00:45:51.865952+0000","last_became_active":"2024-05-09T00:43:43.875497+0000","last_became_peered":"2024-05-09T00:43:43.875497+0000","last_unstale":"2024-05-09T00:45:51.865952+0000","last_undegraded":"2024-05-09T00:45:51.865952+0000","last_fullsized":"2024-05-09T00:45:51.865952+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:54:44.215936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'13","reported_seq":230,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:47:06.232722+0000","last_change":"2024-05-09T00:43:40.854337+0000","last_active":"2024-05-09T00:47:06.232722+0000","last_peered":"2024-05-09T00:47:06.232722+0000","last_clean":"2024-05-09T00:47:06.232722+0000","last_became_active":"2024-05-09T00:43:40.853986+0000","last_became_peered":"2024-05-09T00:43:40.853986+0000","last_unstale":"2024-05-09T00:47:06.232722+0000","last_undegraded":"2024-05-09T00:47:06.232722+0000","last_fullsized":"2024-05-09T00:47:06.232722+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:01:19.128533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":186,"num_read_kb":186,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.865652+0000","last_change":"2024-05-09T00:43:43.878081+0000","last_active":"2024-05-09T00:45:51.865652+0000","last_peered":"2024-05-09T00:45:51.865652+0000","last_clean":"2024-05-09T00:45:51.865652+0000","last_became_active":"2024-05-09T00:43:43.877585+0000","last_became_peered":"2024-05-09T00:43:43.877585+0000","last_unstale":"2024-05-09T00:45:51.865652+0000","last_undegraded":"2024-05-09T00:45:51.865652+0000","last_fullsized":"2024-05-09T00:45:51.865652+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:17:39.255841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.115936+0000","last_change":"2024-05-09T00:43:40.959386+0000","last_active":"2024-05-09T00:45:24.115936+0000","last_peered":"2024-05-09T00:45:24.115936+0000","last_clean":"2024-05-09T00:45:24.115936+0000","last_became_active":"2024-05-09T00:43:40.958878+0000","last_became_peered":"2024-05-09T00:43:40.958878+0000","last_unstale":"2024-05-09T00:45:24.115936+0000","last_undegraded":"2024-05-09T00:45:24.115936+0000","last_fullsized":"2024-05-09T00:45:24.115936+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:14:57.105078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.873650+0000","last_change":"2024-05-09T00:43:43.870563+0000","last_active":"2024-05-09T00:45:51.873650+0000","last_peered":"2024-05-09T00:45:51.873650+0000","last_clean":"2024-05-09T00:45:51.873650+0000","last_became_active":"2024-05-09T00:43:43.870237+0000","last_became_peered":"2024-05-09T00:43:43.870237+0000","last_unstale":"2024-05-09T00:45:51.873650+0000","last_undegraded":"2024-05-09T00:45:51.873650+0000","last_fullsized":"2024-05-09T00:45:51.873650+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:14:28.006343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"78'12","reported_seq":44,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.937368+0000","last_change":"2024-05-09T00:43:40.857670+0000","last_active":"2024-05-09T00:45:23.937368+0000","last_peered":"2024-05-09T00:45:23.937368+0000","last_clean":"2024-05-09T00:45:23.937368+0000","last_became_active":"2024-05-09T00:43:40.857247+0000","last_became_peered":"2024-05-09T00:43:40.857247+0000","last_unstale":"2024-05-09T00:45:23.937368+0000","last_undegraded":"2024-05-09T00:45:23.937368+0000","last_fullsized":"2024-05-09T00:45:23.937368+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:47:06.325593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.873557+0000","last_change":"2024-05-09T00:43:43.883179+0000","last_active":"2024-05-09T00:45:51.873557+0000","last_peered":"2024-05-09T00:45:51.873557+0000","last_clean":"2024-05-09T00:45:51.873557+0000","last_became_active":"2024-05-09T00:43:43.882820+0000","last_became_peered":"2024-05-09T00:43:43.882820+0000","last_unstale":"2024-05-09T00:45:51.873557+0000","last_undegraded":"2024-05-09T00:45:51.873557+0000","last_fullsized":"2024-05-09T00:45:51.873557+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:56:26.012495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"78'5","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.946133+0000","last_change":"2024-05-09T00:45:23.945720+0000","last_active":"2024-05-09T00:45:23.946133+0000","last_peered":"2024-05-09T00:45:23.946133+0000","last_clean":"2024-05-09T00:45:23.946133+0000","last_became_active":"2024-05-09T00:45:23.940421+0000","last_became_peered":"2024-05-09T00:45:23.940421+0000","last_unstale":"2024-05-09T00:45:23.946133+0000","last_undegraded":"2024-05-09T00:45:23.946133+0000","last_fullsized":"2024-05-09T00:45:23.946133+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:23:24.905715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.876297+0000","last_change":"2024-05-09T00:43:43.882950+0000","last_active":"2024-05-09T00:45:51.876297+0000","last_peered":"2024-05-09T00:45:51.876297+0000","last_clean":"2024-05-09T00:45:51.876297+0000","last_became_active":"2024-05-09T00:43:43.880679+0000","last_became_peered":"2024-05-09T00:43:43.880679+0000","last_unstale":"2024-05-09T00:45:51.876297+0000","last_undegraded":"2024-05-09T00:45:51.876297+0000","last_fullsized":"2024-05-09T00:45:51.876297+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:34:14.050987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"82'7","reported_seq":227,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:47:06.234636+0000","last_change":"2024-05-09T00:43:40.854953+0000","last_active":"2024-05-09T00:47:06.234636+0000","last_peered":"2024-05-09T00:47:06.234636+0000","last_clean":"2024-05-09T00:47:06.234636+0000","last_became_active":"2024-05-09T00:43:40.854562+0000","last_became_peered":"2024-05-09T00:43:40.854562+0000","last_unstale":"2024-05-09T00:47:06.234636+0000","last_undegraded":"2024-05-09T00:47:06.234636+0000","last_fullsized":"2024-05-09T00:47:06.234636+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:09:19.564806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":189,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.857389+0000","last_change":"2024-05-09T00:43:43.881881+0000","last_active":"2024-05-09T00:45:51.857389+0000","last_peered":"2024-05-09T00:45:51.857389+0000","last_clean":"2024-05-09T00:45:51.857389+0000","last_became_active":"2024-05-09T00:43:43.881461+0000","last_became_peered":"2024-05-09T00:43:43.881461+0000","last_unstale":"2024-05-09T00:45:51.857389+0000","last_undegraded":"2024-05-09T00:45:51.857389+0000","last_fullsized":"2024-05-09T00:45:51.857389+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:59:28.747084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"78'7","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.080620+0000","last_change":"2024-05-09T00:43:40.858521+0000","last_active":"2024-05-09T00:45:24.080620+0000","last_peered":"2024-05-09T00:45:24.080620+0000","last_clean":"2024-05-09T00:45:24.080620+0000","last_became_active":"2024-05-09T00:43:40.858102+0000","last_became_peered":"2024-05-09T00:43:40.858102+0000","last_unstale":"2024-05-09T00:45:24.080620+0000","last_undegraded":"2024-05-09T00:45:24.080620+0000","last_fullsized":"2024-05-09T00:45:24.080620+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:09:51.684630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.873396+0000","last_change":"2024-05-09T00:44:23.023472+0000","last_active":"2024-05-09T00:45:51.873396+0000","last_peered":"2024-05-09T00:45:51.873396+0000","last_clean":"2024-05-09T00:45:51.873396+0000","last_became_active":"2024-05-09T00:44:23.023017+0000","last_became_peered":"2024-05-09T00:44:23.023017+0000","last_unstale":"2024-05-09T00:45:51.873396+0000","last_undegraded":"2024-05-09T00:45:51.873396+0000","last_fullsized":"2024-05-09T00:45:51.873396+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:05:13.852606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"78'3","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.934006+0000","last_change":"2024-05-09T00:43:40.853329+0000","last_active":"2024-05-09T00:45:23.934006+0000","last_peered":"2024-05-09T00:45:23.934006+0000","last_clean":"2024-05-09T00:45:23.934006+0000","last_became_active":"2024-05-09T00:43:40.852979+0000","last_became_peered":"2024-05-09T00:43:40.852979+0000","last_unstale":"2024-05-09T00:45:23.934006+0000","last_undegraded":"2024-05-09T00:45:23.934006+0000","last_fullsized":"2024-05-09T00:45:23.934006+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:13:57.006031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.875355+0000","last_change":"2024-05-09T00:43:43.862876+0000","last_active":"2024-05-09T00:45:51.875355+0000","last_peered":"2024-05-09T00:45:51.875355+0000","last_clean":"2024-05-09T00:45:51.875355+0000","last_became_active":"2024-05-09T00:43:43.862521+0000","last_became_peered":"2024-05-09T00:43:43.862521+0000","last_unstale":"2024-05-09T00:45:51.875355+0000","last_undegraded":"2024-05-09T00:45:51.875355+0000","last_fullsized":"2024-05-09T00:45:51.875355+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:45:49.386844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'3","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.937546+0000","last_change":"2024-05-09T00:43:40.860395+0000","last_active":"2024-05-09T00:45:23.937546+0000","last_peered":"2024-05-09T00:45:23.937546+0000","last_clean":"2024-05-09T00:45:23.937546+0000","last_became_active":"2024-05-09T00:43:40.860040+0000","last_became_peered":"2024-05-09T00:43:40.860040+0000","last_unstale":"2024-05-09T00:45:23.937546+0000","last_undegraded":"2024-05-09T00:45:23.937546+0000","last_fullsized":"2024-05-09T00:45:23.937546+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:20:36.054685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.862285+0000","last_change":"2024-05-09T00:44:23.180905+0000","last_active":"2024-05-09T00:45:51.862285+0000","last_peered":"2024-05-09T00:45:51.862285+0000","last_clean":"2024-05-09T00:45:51.862285+0000","last_became_active":"2024-05-09T00:44:23.180296+0000","last_became_peered":"2024-05-09T00:44:23.180296+0000","last_unstale":"2024-05-09T00:45:51.862285+0000","last_undegraded":"2024-05-09T00:45:51.862285+0000","last_fullsized":"2024-05-09T00:45:51.862285+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:48:17.585276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,10],"acting":[9,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'6","reported_seq":407,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:47:07.235641+0000","last_change":"2024-05-09T00:43:40.860542+0000","last_active":"2024-05-09T00:47:07.235641+0000","last_peered":"2024-05-09T00:47:07.235641+0000","last_clean":"2024-05-09T00:47:07.235641+0000","last_became_active":"2024-05-09T00:43:40.860176+0000","last_became_peered":"2024-05-09T00:43:40.860176+0000","last_unstale":"2024-05-09T00:47:07.235641+0000","last_undegraded":"2024-05-09T00:47:07.235641+0000","last_fullsized":"2024-05-09T00:47:07.235641+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:44:50.375024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":370,"num_read_kb":370,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.872797+0000","last_change":"2024-05-09T00:43:43.882119+0000","last_active":"2024-05-09T00:45:51.872797+0000","last_peered":"2024-05-09T00:45:51.872797+0000","last_clean":"2024-05-09T00:45:51.872797+0000","last_became_active":"2024-05-09T00:43:43.881747+0000","last_became_peered":"2024-05-09T00:43:43.881747+0000","last_unstale":"2024-05-09T00:45:51.872797+0000","last_undegraded":"2024-05-09T00:45:51.872797+0000","last_fullsized":"2024-05-09T00:45:51.872797+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:54:19.420435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'4","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.081294+0000","last_change":"2024-05-09T00:43:40.859539+0000","last_active":"2024-05-09T00:45:24.081294+0000","last_peered":"2024-05-09T00:45:24.081294+0000","last_clean":"2024-05-09T00:45:24.081294+0000","last_became_active":"2024-05-09T00:43:40.859162+0000","last_became_peered":"2024-05-09T00:43:40.859162+0000","last_unstale":"2024-05-09T00:45:24.081294+0000","last_undegraded":"2024-05-09T00:45:24.081294+0000","last_fullsized":"2024-05-09T00:45:24.081294+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:37:15.190944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.872246+0000","last_change":"2024-05-09T00:44:23.180872+0000","last_active":"2024-05-09T00:45:51.872246+0000","last_peered":"2024-05-09T00:45:51.872246+0000","last_clean":"2024-05-09T00:45:51.872246+0000","last_became_active":"2024-05-09T00:44:23.180398+0000","last_became_peered":"2024-05-09T00:44:23.180398+0000","last_unstale":"2024-05-09T00:45:51.872246+0000","last_undegraded":"2024-05-09T00:45:51.872246+0000","last_fullsized":"2024-05-09T00:45:51.872246+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:19:27.909490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,6],"acting":[0,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'5","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.938347+0000","last_change":"2024-05-09T00:43:40.960126+0000","last_active":"2024-05-09T00:45:23.938347+0000","last_peered":"2024-05-09T00:45:23.938347+0000","last_clean":"2024-05-09T00:45:23.938347+0000","last_became_active":"2024-05-09T00:43:40.959441+0000","last_became_peered":"2024-05-09T00:43:40.959441+0000","last_unstale":"2024-05-09T00:45:23.938347+0000","last_undegraded":"2024-05-09T00:45:23.938347+0000","last_fullsized":"2024-05-09T00:45:23.938347+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:38:47.497029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.881221+0000","last_change":"2024-05-09T00:43:43.874223+0000","last_active":"2024-05-09T00:45:51.881221+0000","last_peered":"2024-05-09T00:45:51.881221+0000","last_clean":"2024-05-09T00:45:51.881221+0000","last_became_active":"2024-05-09T00:43:43.873875+0000","last_became_peered":"2024-05-09T00:43:43.873875+0000","last_unstale":"2024-05-09T00:45:51.881221+0000","last_undegraded":"2024-05-09T00:45:51.881221+0000","last_fullsized":"2024-05-09T00:45:51.881221+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:59:22.192886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'37","reported_seq":469,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:47:07.457164+0000","last_change":"2024-05-09T00:44:23.188953+0000","last_active":"2024-05-09T00:47:07.457164+0000","last_peered":"2024-05-09T00:47:07.457164+0000","last_clean":"2024-05-09T00:47:07.457164+0000","last_became_active":"2024-05-09T00:44:23.180858+0000","last_became_peered":"2024-05-09T00:44:23.180858+0000","last_unstale":"2024-05-09T00:47:07.457164+0000","last_undegraded":"2024-05-09T00:47:07.457164+0000","last_fullsized":"2024-05-09T00:47:07.457164+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:26:16.427665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62892,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":376,"num_read_kb":239,"num_write":39,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":52683,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.871155+0000","last_change":"2024-05-09T00:43:43.872019+0000","last_active":"2024-05-09T00:45:51.871155+0000","last_peered":"2024-05-09T00:45:51.871155+0000","last_clean":"2024-05-09T00:45:51.871155+0000","last_became_active":"2024-05-09T00:43:43.871622+0000","last_became_peered":"2024-05-09T00:43:43.871622+0000","last_unstale":"2024-05-09T00:45:51.871155+0000","last_undegraded":"2024-05-09T00:45:51.871155+0000","last_fullsized":"2024-05-09T00:45:51.871155+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:41:47.789443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'9","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.116231+0000","last_change":"2024-05-09T00:43:40.860111+0000","last_active":"2024-05-09T00:45:24.116231+0000","last_peered":"2024-05-09T00:45:24.116231+0000","last_clean":"2024-05-09T00:45:24.116231+0000","last_became_active":"2024-05-09T00:43:40.859732+0000","last_became_peered":"2024-05-09T00:43:40.859732+0000","last_unstale":"2024-05-09T00:45:24.116231+0000","last_undegraded":"2024-05-09T00:45:24.116231+0000","last_fullsized":"2024-05-09T00:45:24.116231+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:30:03.180679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.867875+0000","last_change":"2024-05-09T00:43:43.870604+0000","last_active":"2024-05-09T00:45:51.867875+0000","last_peered":"2024-05-09T00:45:51.867875+0000","last_clean":"2024-05-09T00:45:51.867875+0000","last_became_active":"2024-05-09T00:43:43.870286+0000","last_became_peered":"2024-05-09T00:43:43.870286+0000","last_unstale":"2024-05-09T00:45:51.867875+0000","last_undegraded":"2024-05-09T00:45:51.867875+0000","last_fullsized":"2024-05-09T00:45:51.867875+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:57:04.486631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'5","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.937866+0000","last_change":"2024-05-09T00:43:40.851966+0000","last_active":"2024-05-09T00:45:23.937866+0000","last_peered":"2024-05-09T00:45:23.937866+0000","last_clean":"2024-05-09T00:45:23.937866+0000","last_became_active":"2024-05-09T00:43:40.851537+0000","last_became_peered":"2024-05-09T00:43:40.851537+0000","last_unstale":"2024-05-09T00:45:23.937866+0000","last_undegraded":"2024-05-09T00:45:23.937866+0000","last_fullsized":"2024-05-09T00:45:23.937866+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:58:59.797183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.879660+0000","last_change":"2024-05-09T00:43:43.879509+0000","last_active":"2024-05-09T00:45:51.879660+0000","last_peered":"2024-05-09T00:45:51.879660+0000","last_clean":"2024-05-09T00:45:51.879660+0000","last_became_active":"2024-05-09T00:43:43.879151+0000","last_became_peered":"2024-05-09T00:43:43.879151+0000","last_unstale":"2024-05-09T00:45:51.879660+0000","last_undegraded":"2024-05-09T00:45:51.879660+0000","last_fullsized":"2024-05-09T00:45:51.879660+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:34:18.850545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'8","reported_seq":228,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:47:06.233264+0000","last_change":"2024-05-09T00:43:40.857629+0000","last_active":"2024-05-09T00:47:06.233264+0000","last_peered":"2024-05-09T00:47:06.233264+0000","last_clean":"2024-05-09T00:47:06.233264+0000","last_became_active":"2024-05-09T00:43:40.857260+0000","last_became_peered":"2024-05-09T00:43:40.857260+0000","last_unstale":"2024-05-09T00:47:06.233264+0000","last_undegraded":"2024-05-09T00:47:06.233264+0000","last_fullsized":"2024-05-09T00:47:06.233264+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:51:00.600870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1428,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":190,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.875060+0000","last_change":"2024-05-09T00:43:43.886100+0000","last_active":"2024-05-09T00:45:51.875060+0000","last_peered":"2024-05-09T00:45:51.875060+0000","last_clean":"2024-05-09T00:45:51.875060+0000","last_became_active":"2024-05-09T00:43:43.885775+0000","last_became_peered":"2024-05-09T00:43:43.885775+0000","last_unstale":"2024-05-09T00:45:51.875060+0000","last_undegraded":"2024-05-09T00:45:51.875060+0000","last_fullsized":"2024-05-09T00:45:51.875060+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:34:19.439163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'17","reported_seq":49,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.936845+0000","last_change":"2024-05-09T00:43:40.959873+0000","last_active":"2024-05-09T00:45:23.936845+0000","last_peered":"2024-05-09T00:45:23.936845+0000","last_clean":"2024-05-09T00:45:23.936845+0000","last_became_active":"2024-05-09T00:43:40.959437+0000","last_became_peered":"2024-05-09T00:43:40.959437+0000","last_unstale":"2024-05-09T00:45:23.936845+0000","last_undegraded":"2024-05-09T00:45:23.936845+0000","last_fullsized":"2024-05-09T00:45:23.936845+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:29:52.686937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.875543+0000","last_change":"2024-05-09T00:43:43.880076+0000","last_active":"2024-05-09T00:45:51.875543+0000","last_peered":"2024-05-09T00:45:51.875543+0000","last_clean":"2024-05-09T00:45:51.875543+0000","last_became_active":"2024-05-09T00:43:43.878733+0000","last_became_peered":"2024-05-09T00:43:43.878733+0000","last_unstale":"2024-05-09T00:45:51.875543+0000","last_undegraded":"2024-05-09T00:45:51.875543+0000","last_fullsized":"2024-05-09T00:45:51.875543+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:30:21.426855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'3","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:59.646186+0000","last_change":"2024-05-09T00:43:40.858033+0000","last_active":"2024-05-09T00:45:59.646186+0000","last_peered":"2024-05-09T00:45:59.646186+0000","last_clean":"2024-05-09T00:45:59.646186+0000","last_became_active":"2024-05-09T00:43:40.857075+0000","last_became_peered":"2024-05-09T00:43:40.857075+0000","last_unstale":"2024-05-09T00:45:59.646186+0000","last_undegraded":"2024-05-09T00:45:59.646186+0000","last_fullsized":"2024-05-09T00:45:59.646186+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:01:49.690226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.871817+0000","last_change":"2024-05-09T00:44:23.180515+0000","last_active":"2024-05-09T00:45:51.871817+0000","last_peered":"2024-05-09T00:45:51.871817+0000","last_clean":"2024-05-09T00:45:51.871817+0000","last_became_active":"2024-05-09T00:44:23.179834+0000","last_became_peered":"2024-05-09T00:44:23.179834+0000","last_unstale":"2024-05-09T00:45:51.871817+0000","last_undegraded":"2024-05-09T00:45:51.871817+0000","last_fullsized":"2024-05-09T00:45:51.871817+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:21:55.735438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,6],"acting":[11,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"78'1","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.941920+0000","last_change":"2024-05-09T00:45:23.941489+0000","last_active":"2024-05-09T00:45:23.941920+0000","last_peered":"2024-05-09T00:45:23.941920+0000","last_clean":"2024-05-09T00:45:23.941920+0000","last_became_active":"2024-05-09T00:45:23.941072+0000","last_became_peered":"2024-05-09T00:45:23.941072+0000","last_unstale":"2024-05-09T00:45:23.941920+0000","last_undegraded":"2024-05-09T00:45:23.941920+0000","last_fullsized":"2024-05-09T00:45:23.941920+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:20:05.755293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.849757+0000","last_change":"2024-05-09T00:43:43.886618+0000","last_active":"2024-05-09T00:45:51.849757+0000","last_peered":"2024-05-09T00:45:51.849757+0000","last_clean":"2024-05-09T00:45:51.849757+0000","last_became_active":"2024-05-09T00:43:43.885946+0000","last_became_peered":"2024-05-09T00:43:43.885946+0000","last_unstale":"2024-05-09T00:45:51.849757+0000","last_undegraded":"2024-05-09T00:45:51.849757+0000","last_fullsized":"2024-05-09T00:45:51.849757+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:16:22.169548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"78'5","reported_seq":224,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:47:08.235950+0000","last_change":"2024-05-09T00:43:40.856213+0000","last_active":"2024-05-09T00:47:08.235950+0000","last_peered":"2024-05-09T00:47:08.235950+0000","last_clean":"2024-05-09T00:47:08.235950+0000","last_became_active":"2024-05-09T00:43:40.855700+0000","last_became_peered":"2024-05-09T00:43:40.855700+0000","last_unstale":"2024-05-09T00:47:08.235950+0000","last_undegraded":"2024-05-09T00:47:08.235950+0000","last_fullsized":"2024-05-09T00:47:08.235950+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:48:50.238293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":188,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.857202+0000","last_change":"2024-05-09T00:43:43.885272+0000","last_active":"2024-05-09T00:45:51.857202+0000","last_peered":"2024-05-09T00:45:51.857202+0000","last_clean":"2024-05-09T00:45:51.857202+0000","last_became_active":"2024-05-09T00:43:43.884774+0000","last_became_peered":"2024-05-09T00:43:43.884774+0000","last_unstale":"2024-05-09T00:45:51.857202+0000","last_undegraded":"2024-05-09T00:45:51.857202+0000","last_fullsized":"2024-05-09T00:45:51.857202+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:14:02.193859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.863000+0000","last_change":"2024-05-09T00:43:43.883370+0000","last_active":"2024-05-09T00:45:51.863000+0000","last_peered":"2024-05-09T00:45:51.863000+0000","last_clean":"2024-05-09T00:45:51.863000+0000","last_became_active":"2024-05-09T00:43:43.881570+0000","last_became_peered":"2024-05-09T00:43:43.881570+0000","last_unstale":"2024-05-09T00:45:51.863000+0000","last_undegraded":"2024-05-09T00:45:51.863000+0000","last_fullsized":"2024-05-09T00:45:51.863000+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:48.936861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'7","reported_seq":225,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:47:07.234443+0000","last_change":"2024-05-09T00:43:40.959828+0000","last_active":"2024-05-09T00:47:07.234443+0000","last_peered":"2024-05-09T00:47:07.234443+0000","last_clean":"2024-05-09T00:47:07.234443+0000","last_became_active":"2024-05-09T00:43:40.958889+0000","last_became_peered":"2024-05-09T00:43:40.958889+0000","last_unstale":"2024-05-09T00:47:07.234443+0000","last_undegraded":"2024-05-09T00:47:07.234443+0000","last_fullsized":"2024-05-09T00:47:07.234443+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:24:50.113298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":187,"num_read_kb":187,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.861559+0000","last_change":"2024-05-09T00:43:43.876144+0000","last_active":"2024-05-09T00:45:51.861559+0000","last_peered":"2024-05-09T00:45:51.861559+0000","last_clean":"2024-05-09T00:45:51.861559+0000","last_became_active":"2024-05-09T00:43:43.875774+0000","last_became_peered":"2024-05-09T00:43:43.875774+0000","last_unstale":"2024-05-09T00:45:51.861559+0000","last_undegraded":"2024-05-09T00:45:51.861559+0000","last_fullsized":"2024-05-09T00:45:51.861559+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:05:23.145431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"78'6","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.939598+0000","last_change":"2024-05-09T00:43:40.857278+0000","last_active":"2024-05-09T00:45:23.939598+0000","last_peered":"2024-05-09T00:45:23.939598+0000","last_clean":"2024-05-09T00:45:23.939598+0000","last_became_active":"2024-05-09T00:43:40.856861+0000","last_became_peered":"2024-05-09T00:43:40.856861+0000","last_unstale":"2024-05-09T00:45:23.939598+0000","last_undegraded":"2024-05-09T00:45:23.939598+0000","last_fullsized":"2024-05-09T00:45:23.939598+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:36:16.603327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":104,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.080451+0000","last_change":"2024-05-09T00:42:14.528544+0000","last_active":"2024-05-09T00:45:24.080451+0000","last_peered":"2024-05-09T00:45:24.080451+0000","last_clean":"2024-05-09T00:45:24.080451+0000","last_became_active":"2024-05-09T00:42:14.523658+0000","last_became_peered":"2024-05-09T00:42:14.523658+0000","last_unstale":"2024-05-09T00:45:24.080451+0000","last_undegraded":"2024-05-09T00:45:24.080451+0000","last_fullsized":"2024-05-09T00:45:24.080451+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:38:48.637995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:38:48.637995+0000","last_clean_scrub_stamp":"2024-05-09T00:38:48.637995+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:42:28.249240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.852620+0000","last_change":"2024-05-09T00:43:43.873638+0000","last_active":"2024-05-09T00:45:51.852620+0000","last_peered":"2024-05-09T00:45:51.852620+0000","last_clean":"2024-05-09T00:45:51.852620+0000","last_became_active":"2024-05-09T00:43:43.871069+0000","last_became_peered":"2024-05-09T00:43:43.871069+0000","last_unstale":"2024-05-09T00:45:51.852620+0000","last_undegraded":"2024-05-09T00:45:51.852620+0000","last_fullsized":"2024-05-09T00:45:51.852620+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:28:34.996989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"78'4","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.080673+0000","last_change":"2024-05-09T00:43:40.856374+0000","last_active":"2024-05-09T00:45:24.080673+0000","last_peered":"2024-05-09T00:45:24.080673+0000","last_clean":"2024-05-09T00:45:24.080673+0000","last_became_active":"2024-05-09T00:43:40.855695+0000","last_became_peered":"2024-05-09T00:43:40.855695+0000","last_unstale":"2024-05-09T00:45:24.080673+0000","last_undegraded":"2024-05-09T00:45:24.080673+0000","last_fullsized":"2024-05-09T00:45:24.080673+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:33:43.734132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"78'4","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.937141+0000","last_change":"2024-05-09T00:43:40.959303+0000","last_active":"2024-05-09T00:45:23.937141+0000","last_peered":"2024-05-09T00:45:23.937141+0000","last_clean":"2024-05-09T00:45:23.937141+0000","last_became_active":"2024-05-09T00:43:40.958855+0000","last_became_peered":"2024-05-09T00:43:40.958855+0000","last_unstale":"2024-05-09T00:45:23.937141+0000","last_undegraded":"2024-05-09T00:45:23.937141+0000","last_fullsized":"2024-05-09T00:45:23.937141+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:53:22.566850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.861738+0000","last_change":"2024-05-09T00:43:43.872041+0000","last_active":"2024-05-09T00:45:51.861738+0000","last_peered":"2024-05-09T00:45:51.861738+0000","last_clean":"2024-05-09T00:45:51.861738+0000","last_became_active":"2024-05-09T00:43:43.870946+0000","last_became_peered":"2024-05-09T00:43:43.870946+0000","last_unstale":"2024-05-09T00:45:51.861738+0000","last_undegraded":"2024-05-09T00:45:51.861738+0000","last_fullsized":"2024-05-09T00:45:51.861738+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:16:28.190488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'17","reported_seq":48,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:52.555000+0000","last_change":"2024-05-09T00:43:40.853875+0000","last_active":"2024-05-09T00:45:52.555000+0000","last_peered":"2024-05-09T00:45:52.555000+0000","last_clean":"2024-05-09T00:45:52.555000+0000","last_became_active":"2024-05-09T00:43:40.853459+0000","last_became_peered":"2024-05-09T00:43:40.853459+0000","last_unstale":"2024-05-09T00:45:52.555000+0000","last_undegraded":"2024-05-09T00:45:52.555000+0000","last_fullsized":"2024-05-09T00:45:52.555000+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:45:29.992730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.876758+0000","last_change":"2024-05-09T00:43:43.870435+0000","last_active":"2024-05-09T00:45:51.876758+0000","last_peered":"2024-05-09T00:45:51.876758+0000","last_clean":"2024-05-09T00:45:51.876758+0000","last_became_active":"2024-05-09T00:43:43.870075+0000","last_became_peered":"2024-05-09T00:43:43.870075+0000","last_unstale":"2024-05-09T00:45:51.876758+0000","last_undegraded":"2024-05-09T00:45:51.876758+0000","last_fullsized":"2024-05-09T00:45:51.876758+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:53:12.958463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'7","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.939824+0000","last_change":"2024-05-09T00:43:41.119325+0000","last_active":"2024-05-09T00:45:23.939824+0000","last_peered":"2024-05-09T00:45:23.939824+0000","last_clean":"2024-05-09T00:45:23.939824+0000","last_became_active":"2024-05-09T00:43:41.119009+0000","last_became_peered":"2024-05-09T00:43:41.119009+0000","last_unstale":"2024-05-09T00:45:23.939824+0000","last_undegraded":"2024-05-09T00:45:23.939824+0000","last_fullsized":"2024-05-09T00:45:23.939824+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:10:05.775152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.873671+0000","last_change":"2024-05-09T00:43:43.875236+0000","last_active":"2024-05-09T00:45:51.873671+0000","last_peered":"2024-05-09T00:45:51.873671+0000","last_clean":"2024-05-09T00:45:51.873671+0000","last_became_active":"2024-05-09T00:43:43.874919+0000","last_became_peered":"2024-05-09T00:43:43.874919+0000","last_unstale":"2024-05-09T00:45:51.873671+0000","last_undegraded":"2024-05-09T00:45:51.873671+0000","last_fullsized":"2024-05-09T00:45:51.873671+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:45:33.232518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.938556+0000","last_change":"2024-05-09T00:43:40.959765+0000","last_active":"2024-05-09T00:45:23.938556+0000","last_peered":"2024-05-09T00:45:23.938556+0000","last_clean":"2024-05-09T00:45:23.938556+0000","last_became_active":"2024-05-09T00:43:40.959383+0000","last_became_peered":"2024-05-09T00:43:40.959383+0000","last_unstale":"2024-05-09T00:45:23.938556+0000","last_undegraded":"2024-05-09T00:45:23.938556+0000","last_fullsized":"2024-05-09T00:45:23.938556+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:13:28.766650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.870027+0000","last_change":"2024-05-09T00:43:43.881627+0000","last_active":"2024-05-09T00:45:51.870027+0000","last_peered":"2024-05-09T00:45:51.870027+0000","last_clean":"2024-05-09T00:45:51.870027+0000","last_became_active":"2024-05-09T00:43:43.881207+0000","last_became_peered":"2024-05-09T00:43:43.881207+0000","last_unstale":"2024-05-09T00:45:51.870027+0000","last_undegraded":"2024-05-09T00:45:51.870027+0000","last_fullsized":"2024-05-09T00:45:51.870027+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:44:08.053638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'7","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.115513+0000","last_change":"2024-05-09T00:43:40.859566+0000","last_active":"2024-05-09T00:45:24.115513+0000","last_peered":"2024-05-09T00:45:24.115513+0000","last_clean":"2024-05-09T00:45:24.115513+0000","last_became_active":"2024-05-09T00:43:40.859182+0000","last_became_peered":"2024-05-09T00:43:40.859182+0000","last_unstale":"2024-05-09T00:45:24.115513+0000","last_undegraded":"2024-05-09T00:45:24.115513+0000","last_fullsized":"2024-05-09T00:45:24.115513+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:58:12.693271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.874341+0000","last_change":"2024-05-09T00:43:43.869193+0000","last_active":"2024-05-09T00:45:51.874341+0000","last_peered":"2024-05-09T00:45:51.874341+0000","last_clean":"2024-05-09T00:45:51.874341+0000","last_became_active":"2024-05-09T00:43:43.868823+0000","last_became_peered":"2024-05-09T00:43:43.868823+0000","last_unstale":"2024-05-09T00:45:51.874341+0000","last_undegraded":"2024-05-09T00:45:51.874341+0000","last_fullsized":"2024-05-09T00:45:51.874341+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:15:53.929873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'3","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.116148+0000","last_change":"2024-05-09T00:43:40.855038+0000","last_active":"2024-05-09T00:45:24.116148+0000","last_peered":"2024-05-09T00:45:24.116148+0000","last_clean":"2024-05-09T00:45:24.116148+0000","last_became_active":"2024-05-09T00:43:40.854334+0000","last_became_peered":"2024-05-09T00:43:40.854334+0000","last_unstale":"2024-05-09T00:45:24.116148+0000","last_undegraded":"2024-05-09T00:45:24.116148+0000","last_fullsized":"2024-05-09T00:45:24.116148+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:47:13.897697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.863165+0000","last_change":"2024-05-09T00:44:23.019151+0000","last_active":"2024-05-09T00:45:51.863165+0000","last_peered":"2024-05-09T00:45:51.863165+0000","last_clean":"2024-05-09T00:45:51.863165+0000","last_became_active":"2024-05-09T00:44:23.018752+0000","last_became_peered":"2024-05-09T00:44:23.018752+0000","last_unstale":"2024-05-09T00:45:51.863165+0000","last_undegraded":"2024-05-09T00:45:51.863165+0000","last_fullsized":"2024-05-09T00:45:51.863165+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:28:41.875747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.937463+0000","last_change":"2024-05-09T00:43:40.859461+0000","last_active":"2024-05-09T00:45:23.937463+0000","last_peered":"2024-05-09T00:45:23.937463+0000","last_clean":"2024-05-09T00:45:23.937463+0000","last_became_active":"2024-05-09T00:43:40.857814+0000","last_became_peered":"2024-05-09T00:43:40.857814+0000","last_unstale":"2024-05-09T00:45:23.937463+0000","last_undegraded":"2024-05-09T00:45:23.937463+0000","last_fullsized":"2024-05-09T00:45:23.937463+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:56:06.279762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.877886+0000","last_change":"2024-05-09T00:43:43.878120+0000","last_active":"2024-05-09T00:45:51.877886+0000","last_peered":"2024-05-09T00:45:51.877886+0000","last_clean":"2024-05-09T00:45:51.877886+0000","last_became_active":"2024-05-09T00:43:43.877782+0000","last_became_peered":"2024-05-09T00:43:43.877782+0000","last_unstale":"2024-05-09T00:45:51.877886+0000","last_undegraded":"2024-05-09T00:45:51.877886+0000","last_fullsized":"2024-05-09T00:45:51.877886+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:19:48.558992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'7","reported_seq":227,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:47:06.304442+0000","last_change":"2024-05-09T00:43:41.119347+0000","last_active":"2024-05-09T00:47:06.304442+0000","last_peered":"2024-05-09T00:47:06.304442+0000","last_clean":"2024-05-09T00:47:06.304442+0000","last_became_active":"2024-05-09T00:43:41.118847+0000","last_became_peered":"2024-05-09T00:43:41.118847+0000","last_unstale":"2024-05-09T00:47:06.304442+0000","last_undegraded":"2024-05-09T00:47:06.304442+0000","last_fullsized":"2024-05-09T00:47:06.304442+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:07:03.977756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":189,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.878473+0000","last_change":"2024-05-09T00:43:43.875203+0000","last_active":"2024-05-09T00:45:51.878473+0000","last_peered":"2024-05-09T00:45:51.878473+0000","last_clean":"2024-05-09T00:45:51.878473+0000","last_became_active":"2024-05-09T00:43:43.874882+0000","last_became_peered":"2024-05-09T00:43:43.874882+0000","last_unstale":"2024-05-09T00:45:51.878473+0000","last_undegraded":"2024-05-09T00:45:51.878473+0000","last_fullsized":"2024-05-09T00:45:51.878473+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:54:27.845086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'3","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.938177+0000","last_change":"2024-05-09T00:43:40.857955+0000","last_active":"2024-05-09T00:45:23.938177+0000","last_peered":"2024-05-09T00:45:23.938177+0000","last_clean":"2024-05-09T00:45:23.938177+0000","last_became_active":"2024-05-09T00:43:40.857596+0000","last_became_peered":"2024-05-09T00:43:40.857596+0000","last_unstale":"2024-05-09T00:45:23.938177+0000","last_undegraded":"2024-05-09T00:45:23.938177+0000","last_fullsized":"2024-05-09T00:45:23.938177+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:13:56.676929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.872219+0000","last_change":"2024-05-09T00:43:43.882487+0000","last_active":"2024-05-09T00:45:51.872219+0000","last_peered":"2024-05-09T00:45:51.872219+0000","last_clean":"2024-05-09T00:45:51.872219+0000","last_became_active":"2024-05-09T00:43:43.882176+0000","last_became_peered":"2024-05-09T00:43:43.882176+0000","last_unstale":"2024-05-09T00:45:51.872219+0000","last_undegraded":"2024-05-09T00:45:51.872219+0000","last_fullsized":"2024-05-09T00:45:51.872219+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:48:42.354433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'5","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.934749+0000","last_change":"2024-05-09T00:43:40.857215+0000","last_active":"2024-05-09T00:45:23.934749+0000","last_peered":"2024-05-09T00:45:23.934749+0000","last_clean":"2024-05-09T00:45:23.934749+0000","last_became_active":"2024-05-09T00:43:40.856858+0000","last_became_peered":"2024-05-09T00:43:40.856858+0000","last_unstale":"2024-05-09T00:45:23.934749+0000","last_undegraded":"2024-05-09T00:45:23.934749+0000","last_fullsized":"2024-05-09T00:45:23.934749+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:40:47.943482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.878949+0000","last_change":"2024-05-09T00:43:43.881192+0000","last_active":"2024-05-09T00:45:51.878949+0000","last_peered":"2024-05-09T00:45:51.878949+0000","last_clean":"2024-05-09T00:45:51.878949+0000","last_became_active":"2024-05-09T00:43:43.880781+0000","last_became_peered":"2024-05-09T00:43:43.880781+0000","last_unstale":"2024-05-09T00:45:51.878949+0000","last_undegraded":"2024-05-09T00:45:51.878949+0000","last_fullsized":"2024-05-09T00:45:51.878949+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:18:31.577354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.870420+0000","last_change":"2024-05-09T00:43:43.876869+0000","last_active":"2024-05-09T00:45:51.870420+0000","last_peered":"2024-05-09T00:45:51.870420+0000","last_clean":"2024-05-09T00:45:51.870420+0000","last_became_active":"2024-05-09T00:43:43.876439+0000","last_became_peered":"2024-05-09T00:43:43.876439+0000","last_unstale":"2024-05-09T00:45:51.870420+0000","last_undegraded":"2024-05-09T00:45:51.870420+0000","last_fullsized":"2024-05-09T00:45:51.870420+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:39:06.776267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.871096+0000","last_change":"2024-05-09T00:43:43.884677+0000","last_active":"2024-05-09T00:45:51.871096+0000","last_peered":"2024-05-09T00:45:51.871096+0000","last_clean":"2024-05-09T00:45:51.871096+0000","last_became_active":"2024-05-09T00:43:43.884238+0000","last_became_peered":"2024-05-09T00:43:43.884238+0000","last_unstale":"2024-05-09T00:45:51.871096+0000","last_undegraded":"2024-05-09T00:45:51.871096+0000","last_fullsized":"2024-05-09T00:45:51.871096+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:18:13.787584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.867942+0000","last_change":"2024-05-09T00:43:43.876638+0000","last_active":"2024-05-09T00:45:51.867942+0000","last_peered":"2024-05-09T00:45:51.867942+0000","last_clean":"2024-05-09T00:45:51.867942+0000","last_became_active":"2024-05-09T00:43:43.876315+0000","last_became_peered":"2024-05-09T00:43:43.876315+0000","last_unstale":"2024-05-09T00:45:51.867942+0000","last_undegraded":"2024-05-09T00:45:51.867942+0000","last_fullsized":"2024-05-09T00:45:51.867942+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:22:41.618613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.867553+0000","last_change":"2024-05-09T00:43:43.869798+0000","last_active":"2024-05-09T00:45:51.867553+0000","last_peered":"2024-05-09T00:45:51.867553+0000","last_clean":"2024-05-09T00:45:51.867553+0000","last_became_active":"2024-05-09T00:43:43.869444+0000","last_became_peered":"2024-05-09T00:43:43.869444+0000","last_unstale":"2024-05-09T00:45:51.867553+0000","last_undegraded":"2024-05-09T00:45:51.867553+0000","last_fullsized":"2024-05-09T00:45:51.867553+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:15:53.267441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.858958+0000","last_change":"2024-05-09T00:43:43.873215+0000","last_active":"2024-05-09T00:45:51.858958+0000","last_peered":"2024-05-09T00:45:51.858958+0000","last_clean":"2024-05-09T00:45:51.858958+0000","last_became_active":"2024-05-09T00:43:43.872763+0000","last_became_peered":"2024-05-09T00:43:43.872763+0000","last_unstale":"2024-05-09T00:45:51.858958+0000","last_undegraded":"2024-05-09T00:45:51.858958+0000","last_fullsized":"2024-05-09T00:45:51.858958+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:08.997387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.870845+0000","last_change":"2024-05-09T00:43:43.876514+0000","last_active":"2024-05-09T00:45:51.870845+0000","last_peered":"2024-05-09T00:45:51.870845+0000","last_clean":"2024-05-09T00:45:51.870845+0000","last_became_active":"2024-05-09T00:43:43.876147+0000","last_became_peered":"2024-05-09T00:43:43.876147+0000","last_unstale":"2024-05-09T00:45:51.870845+0000","last_undegraded":"2024-05-09T00:45:51.870845+0000","last_fullsized":"2024-05-09T00:45:51.870845+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:58:34.745819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.864498+0000","last_change":"2024-05-09T00:43:43.871869+0000","last_active":"2024-05-09T00:45:51.864498+0000","last_peered":"2024-05-09T00:45:51.864498+0000","last_clean":"2024-05-09T00:45:51.864498+0000","last_became_active":"2024-05-09T00:43:43.871447+0000","last_became_peered":"2024-05-09T00:43:43.871447+0000","last_unstale":"2024-05-09T00:45:51.864498+0000","last_undegraded":"2024-05-09T00:45:51.864498+0000","last_fullsized":"2024-05-09T00:45:51.864498+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:11:42.128931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.865595+0000","last_change":"2024-05-09T00:43:43.871291+0000","last_active":"2024-05-09T00:45:51.865595+0000","last_peered":"2024-05-09T00:45:51.865595+0000","last_clean":"2024-05-09T00:45:51.865595+0000","last_became_active":"2024-05-09T00:43:43.870899+0000","last_became_peered":"2024-05-09T00:43:43.870899+0000","last_unstale":"2024-05-09T00:45:51.865595+0000","last_undegraded":"2024-05-09T00:45:51.865595+0000","last_fullsized":"2024-05-09T00:45:51.865595+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:28:53.013543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.864359+0000","last_change":"2024-05-09T00:43:43.884791+0000","last_active":"2024-05-09T00:45:51.864359+0000","last_peered":"2024-05-09T00:45:51.864359+0000","last_clean":"2024-05-09T00:45:51.864359+0000","last_became_active":"2024-05-09T00:43:43.877333+0000","last_became_peered":"2024-05-09T00:43:43.877333+0000","last_unstale":"2024-05-09T00:45:51.864359+0000","last_undegraded":"2024-05-09T00:45:51.864359+0000","last_fullsized":"2024-05-09T00:45:51.864359+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:37:07.535388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.874993+0000","last_change":"2024-05-09T00:43:43.863773+0000","last_active":"2024-05-09T00:45:51.874993+0000","last_peered":"2024-05-09T00:45:51.874993+0000","last_clean":"2024-05-09T00:45:51.874993+0000","last_became_active":"2024-05-09T00:43:43.863436+0000","last_became_peered":"2024-05-09T00:43:43.863436+0000","last_unstale":"2024-05-09T00:45:51.874993+0000","last_undegraded":"2024-05-09T00:45:51.874993+0000","last_fullsized":"2024-05-09T00:45:51.874993+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:59:57.100234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.880217+0000","last_change":"2024-05-09T00:43:43.873461+0000","last_active":"2024-05-09T00:45:51.880217+0000","last_peered":"2024-05-09T00:45:51.880217+0000","last_clean":"2024-05-09T00:45:51.880217+0000","last_became_active":"2024-05-09T00:43:43.873139+0000","last_became_peered":"2024-05-09T00:43:43.873139+0000","last_unstale":"2024-05-09T00:45:51.880217+0000","last_undegraded":"2024-05-09T00:45:51.880217+0000","last_fullsized":"2024-05-09T00:45:51.880217+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:24:57.865569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.867837+0000","last_change":"2024-05-09T00:43:43.874401+0000","last_active":"2024-05-09T00:45:51.867837+0000","last_peered":"2024-05-09T00:45:51.867837+0000","last_clean":"2024-05-09T00:45:51.867837+0000","last_became_active":"2024-05-09T00:43:43.873765+0000","last_became_peered":"2024-05-09T00:43:43.873765+0000","last_unstale":"2024-05-09T00:45:51.867837+0000","last_undegraded":"2024-05-09T00:45:51.867837+0000","last_fullsized":"2024-05-09T00:45:51.867837+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:23:59.509415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.877398+0000","last_change":"2024-05-09T00:43:43.875219+0000","last_active":"2024-05-09T00:45:51.877398+0000","last_peered":"2024-05-09T00:45:51.877398+0000","last_clean":"2024-05-09T00:45:51.877398+0000","last_became_active":"2024-05-09T00:43:43.874824+0000","last_became_peered":"2024-05-09T00:43:43.874824+0000","last_unstale":"2024-05-09T00:45:51.877398+0000","last_undegraded":"2024-05-09T00:45:51.877398+0000","last_fullsized":"2024-05-09T00:45:51.877398+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:39:41.929521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":262144,"data_stored":2951840,"data_compressed":179218,"data_compressed_allocated":225280,"data_compressed_original":2932736,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71218,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1894,"num_read_kb":1016,"num_write":266,"num_write_kb":154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":53265,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":540672,"data_stored":213654,"data_compressed":8493,"data_compressed_allocated":12288,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},"log_size":228,"ondisk_log_size":228,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710777,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":901120,"data_stored":443946,"data_compressed":5799,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877907007,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":1097593,"data_compressed":41967,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070532,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":442762,"data_compressed":5799,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266761,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":913408,"data_stored":442558,"data_compressed":5799,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462991,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":1036639,"data_compressed":41967,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626516,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":905216,"data_stored":442479,"data_compressed":5799,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574957,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":970752,"data_stored":506284,"data_compressed":5799,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738482,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":925696,"data_stored":444350,"data_compressed":5799,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411431,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":917504,"data_stored":444676,"data_compressed":5799,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182499,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36236,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708916,"statfs":{"total":95995035648,"available":95957929984,"internally_reserved":0,"allocated":921600,"data_stored":442304,"data_compressed":5799,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986269,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":506072,"data_compressed":14292,"data_compressed_allocated":20480,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822745,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":1035477,"data_compressed":41967,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34546,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36168,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36168,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36168,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36168,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":64644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":64432,"data_compressed":8493,"data_compressed_allocated":12288,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":65468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T00:47:12.356 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-05-09T00:47:12.356 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T00:47:12.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.507+0000 7f8fde014640 1 -- 172.21.15.5:0/2112393609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fd81071f0 msgr2=0x7f8fd8109610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:12.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.507+0000 7f8fde014640 1 --2- 172.21.15.5:0/2112393609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fd81071f0 0x7f8fd8109610 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc009900 tx=0x7f8fcc02a410 comp rx=0 tx=0).stop 2024-05-09T00:47:12.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.507+0000 7f8fde014640 1 -- 172.21.15.5:0/2112393609 shutdown_connections 2024-05-09T00:47:12.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.507+0000 7f8fde014640 1 --2- 172.21.15.5:0/2112393609 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8fd8109b50 0x7f8fd810bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:12.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.507+0000 7f8fde014640 1 --2- 172.21.15.5:0/2112393609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fd81071f0 0x7f8fd8109610 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:12.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.507+0000 7f8fde014640 1 --2- 172.21.15.5:0/2112393609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fd8102040 0x7f8fd8106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:12.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.507+0000 7f8fde014640 1 -- 172.21.15.5:0/2112393609 >> 172.21.15.5:0/2112393609 conn(0x7f8fd80fd4a0 msgr2=0x7f8fd80ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:12.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.507+0000 7f8fde014640 1 -- 172.21.15.5:0/2112393609 shutdown_connections 2024-05-09T00:47:12.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.507+0000 7f8fde014640 1 -- 172.21.15.5:0/2112393609 wait complete. 2024-05-09T00:47:12.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.508+0000 7f8fde014640 1 Processor -- start 2024-05-09T00:47:12.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.508+0000 7f8fde014640 1 -- start start 2024-05-09T00:47:12.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.508+0000 7f8fde014640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fd8102040 0x7f8fd819a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:12.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.508+0000 7f8fde014640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fd81071f0 0x7f8fd819a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:12.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.508+0000 7f8fde014640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8fd8109b50 0x7f8fd81a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:12.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.508+0000 7f8fde014640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd810f7d0 con 0x7f8fd81071f0 2024-05-09T00:47:12.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.508+0000 7f8fde014640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd810f650 con 0x7f8fd8109b50 2024-05-09T00:47:12.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.508+0000 7f8fde014640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd810f950 con 0x7f8fd8102040 2024-05-09T00:47:12.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.508+0000 7f8fd6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fd81071f0 0x7f8fd819a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:12.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.508+0000 7f8fd6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fd81071f0 0x7f8fd819a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34510/0 (socket says 172.21.15.5:34510) 2024-05-09T00:47:12.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.508+0000 7f8fd6ffd640 1 -- 172.21.15.5:0/471405949 learned_addr learned my addr 172.21.15.5:0/471405949 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:47:12.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.508+0000 7f8fd77fe640 1 --2- 172.21.15.5:0/471405949 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fd8102040 0x7f8fd819a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:12.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.508+0000 7f8fd7fff640 1 --2- 172.21.15.5:0/471405949 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8fd8109b50 0x7f8fd81a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:12.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.509+0000 7f8fd6ffd640 1 -- 172.21.15.5:0/471405949 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fd8102040 msgr2=0x7f8fd819a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:12.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.509+0000 7f8fd6ffd640 1 --2- 172.21.15.5:0/471405949 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fd8102040 0x7f8fd819a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:12.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.509+0000 7f8fd6ffd640 1 -- 172.21.15.5:0/471405949 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8fd8109b50 msgr2=0x7f8fd81a1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:12.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.509+0000 7f8fd6ffd640 1 --2- 172.21.15.5:0/471405949 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8fd8109b50 0x7f8fd81a1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:12.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.509+0000 7f8fd6ffd640 1 -- 172.21.15.5:0/471405949 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fd81a1ec0 con 0x7f8fd81071f0 2024-05-09T00:47:12.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.509+0000 7f8fd6ffd640 1 --2- 172.21.15.5:0/471405949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fd81071f0 0x7f8fd819a8c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc006010 tx=0x7f8fcc004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:12.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.509+0000 7f8fd4ff9640 1 -- 172.21.15.5:0/471405949 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc03f030 con 0x7f8fd81071f0 2024-05-09T00:47:12.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.509+0000 7f8fde014640 1 -- 172.21.15.5:0/471405949 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fd81a2150 con 0x7f8fd81071f0 2024-05-09T00:47:12.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.509+0000 7f8fd4ff9640 1 -- 172.21.15.5:0/471405949 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fcc003cb0 con 0x7f8fd81071f0 2024-05-09T00:47:12.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.509+0000 7f8fd4ff9640 1 -- 172.21.15.5:0/471405949 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc035070 con 0x7f8fd81071f0 2024-05-09T00:47:12.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.509+0000 7f8fde014640 1 -- 172.21.15.5:0/471405949 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fd81a2690 con 0x7f8fd81071f0 2024-05-09T00:47:12.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.510+0000 7f8fde014640 1 -- 172.21.15.5:0/471405949 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fd80fefc0 con 0x7f8fd81071f0 2024-05-09T00:47:12.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.512+0000 7f8fd4ff9640 1 -- 172.21.15.5:0/471405949 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f8fcc042030 con 0x7f8fd81071f0 2024-05-09T00:47:12.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.513+0000 7f8fd4ff9640 1 --2- 172.21.15.5:0/471405949 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f8fc00b27f0 0x7f8fc00b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:12.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.513+0000 7f8fd77fe640 1 --2- 172.21.15.5:0/471405949 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f8fc00b27f0 0x7f8fc00b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:12.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.513+0000 7f8fd4ff9640 1 -- 172.21.15.5:0/471405949 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f8fcc102f50 con 0x7f8fd81071f0 2024-05-09T00:47:12.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.513+0000 7f8fd77fe640 1 --2- 172.21.15.5:0/471405949 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f8fc00b27f0 0x7f8fc00b4d00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8005d00 tx=0x7f8fc800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:12.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.516+0000 7f8fd4ff9640 1 -- 172.21.15.5:0/471405949 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8fcc0cac40 con 0x7f8fd81071f0 2024-05-09T00:47:12.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.707+0000 7f8fde014640 1 -- 172.21.15.5:0/471405949 --> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8fd8063110 con 0x7f8fc00b27f0 2024-05-09T00:47:12.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.714+0000 7f8fd4ff9640 1 -- 172.21.15.5:0/471405949 <== mgr.14152 v2:172.21.15.5:6800/91856593 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259094 (secure 0 0 0) 0x7f8fd8063110 con 0x7f8fc00b27f0 2024-05-09T00:47:12.718 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:47:12.719 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-09T00:47:12.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.718+0000 7f8fde014640 1 -- 172.21.15.5:0/471405949 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f8fc00b27f0 msgr2=0x7f8fc00b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:12.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.718+0000 7f8fde014640 1 --2- 172.21.15.5:0/471405949 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f8fc00b27f0 0x7f8fc00b4d00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8005d00 tx=0x7f8fc800b410 comp rx=0 tx=0).stop 2024-05-09T00:47:12.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.718+0000 7f8fde014640 1 -- 172.21.15.5:0/471405949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fd81071f0 msgr2=0x7f8fd819a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:12.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.718+0000 7f8fde014640 1 --2- 172.21.15.5:0/471405949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fd81071f0 0x7f8fd819a8c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc006010 tx=0x7f8fcc004340 comp rx=0 tx=0).stop 2024-05-09T00:47:12.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.718+0000 7f8fde014640 1 -- 172.21.15.5:0/471405949 shutdown_connections 2024-05-09T00:47:12.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.718+0000 7f8fde014640 1 --2- 172.21.15.5:0/471405949 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f8fc00b27f0 0x7f8fc00b4d00 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:12.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.718+0000 7f8fde014640 1 --2- 172.21.15.5:0/471405949 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8fd8109b50 0x7f8fd81a1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:12.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.718+0000 7f8fde014640 1 --2- 172.21.15.5:0/471405949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fd81071f0 0x7f8fd819a8c0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:12.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.718+0000 7f8fde014640 1 --2- 172.21.15.5:0/471405949 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fd8102040 0x7f8fd819a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:12.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.718+0000 7f8fde014640 1 -- 172.21.15.5:0/471405949 >> 172.21.15.5:0/471405949 conn(0x7f8fd80fd4a0 msgr2=0x7f8fd8105790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:12.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.718+0000 7f8fde014640 1 -- 172.21.15.5:0/471405949 shutdown_connections 2024-05-09T00:47:12.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:12.718+0000 7f8fde014640 1 -- 172.21.15.5:0/471405949 wait complete. 2024-05-09T00:47:12.730 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":397,"stamp":"2024-05-09T00:47:11.913507+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661820,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1970,"num_read_kb":1080,"num_write":391,"num_write_kb":2306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":3005105,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":822,"ondisk_log_size":822,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":351896,"kb_used_data":11064,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124589928,"statfs":{"total":1151940427776,"available":1151580086272,"internally_reserved":0,"allocated":11329536,"data_stored":7285140,"data_compressed":186585,"data_compressed_allocated":245760,"data_compressed_original":2015232,"omap_allocated":19141,"internal_metadata":348894523},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":616,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":107,"num_read_kb":52,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003299"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.877185+0000","last_change":"2024-05-09T00:43:43.875357+0000","last_active":"2024-05-09T00:45:51.877185+0000","last_peered":"2024-05-09T00:45:51.877185+0000","last_clean":"2024-05-09T00:45:51.877185+0000","last_became_active":"2024-05-09T00:43:43.873765+0000","last_became_peered":"2024-05-09T00:43:43.873765+0000","last_unstale":"2024-05-09T00:45:51.877185+0000","last_undegraded":"2024-05-09T00:45:51.877185+0000","last_fullsized":"2024-05-09T00:45:51.877185+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:47.519110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.875117+0000","last_change":"2024-05-09T00:43:43.882464+0000","last_active":"2024-05-09T00:45:51.875117+0000","last_peered":"2024-05-09T00:45:51.875117+0000","last_clean":"2024-05-09T00:45:51.875117+0000","last_became_active":"2024-05-09T00:43:43.882102+0000","last_became_peered":"2024-05-09T00:43:43.882102+0000","last_unstale":"2024-05-09T00:45:51.875117+0000","last_undegraded":"2024-05-09T00:45:51.875117+0000","last_fullsized":"2024-05-09T00:45:51.875117+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:39:36.234474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.872356+0000","last_change":"2024-05-09T00:43:43.873098+0000","last_active":"2024-05-09T00:45:51.872356+0000","last_peered":"2024-05-09T00:45:51.872356+0000","last_clean":"2024-05-09T00:45:51.872356+0000","last_became_active":"2024-05-09T00:43:43.872764+0000","last_became_peered":"2024-05-09T00:43:43.872764+0000","last_unstale":"2024-05-09T00:45:51.872356+0000","last_undegraded":"2024-05-09T00:45:51.872356+0000","last_fullsized":"2024-05-09T00:45:51.872356+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:05:30.784111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.869925+0000","last_change":"2024-05-09T00:43:43.869807+0000","last_active":"2024-05-09T00:45:51.869925+0000","last_peered":"2024-05-09T00:45:51.869925+0000","last_clean":"2024-05-09T00:45:51.869925+0000","last_became_active":"2024-05-09T00:43:43.869470+0000","last_became_peered":"2024-05-09T00:43:43.869470+0000","last_unstale":"2024-05-09T00:45:51.869925+0000","last_undegraded":"2024-05-09T00:45:51.869925+0000","last_fullsized":"2024-05-09T00:45:51.869925+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:25:40.201844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.852785+0000","last_change":"2024-05-09T00:43:43.874110+0000","last_active":"2024-05-09T00:45:51.852785+0000","last_peered":"2024-05-09T00:45:51.852785+0000","last_clean":"2024-05-09T00:45:51.852785+0000","last_became_active":"2024-05-09T00:43:43.873759+0000","last_became_peered":"2024-05-09T00:43:43.873759+0000","last_unstale":"2024-05-09T00:45:51.852785+0000","last_undegraded":"2024-05-09T00:45:51.852785+0000","last_fullsized":"2024-05-09T00:45:51.852785+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:17:17.722319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.862325+0000","last_change":"2024-05-09T00:44:23.022285+0000","last_active":"2024-05-09T00:45:51.862325+0000","last_peered":"2024-05-09T00:45:51.862325+0000","last_clean":"2024-05-09T00:45:51.862325+0000","last_became_active":"2024-05-09T00:44:23.021887+0000","last_became_peered":"2024-05-09T00:44:23.021887+0000","last_unstale":"2024-05-09T00:45:51.862325+0000","last_undegraded":"2024-05-09T00:45:51.862325+0000","last_fullsized":"2024-05-09T00:45:51.862325+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:07:07.702178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,7],"acting":[10,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.873552+0000","last_change":"2024-05-09T00:43:43.883112+0000","last_active":"2024-05-09T00:45:51.873552+0000","last_peered":"2024-05-09T00:45:51.873552+0000","last_clean":"2024-05-09T00:45:51.873552+0000","last_became_active":"2024-05-09T00:43:43.882795+0000","last_became_peered":"2024-05-09T00:43:43.882795+0000","last_unstale":"2024-05-09T00:45:51.873552+0000","last_undegraded":"2024-05-09T00:45:51.873552+0000","last_fullsized":"2024-05-09T00:45:51.873552+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:33:58.724015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.862472+0000","last_change":"2024-05-09T00:43:43.880696+0000","last_active":"2024-05-09T00:45:51.862472+0000","last_peered":"2024-05-09T00:45:51.862472+0000","last_clean":"2024-05-09T00:45:51.862472+0000","last_became_active":"2024-05-09T00:43:43.880174+0000","last_became_peered":"2024-05-09T00:43:43.880174+0000","last_unstale":"2024-05-09T00:45:51.862472+0000","last_undegraded":"2024-05-09T00:45:51.862472+0000","last_fullsized":"2024-05-09T00:45:51.862472+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:40:05.682432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.879518+0000","last_change":"2024-05-09T00:43:43.883449+0000","last_active":"2024-05-09T00:45:51.879518+0000","last_peered":"2024-05-09T00:45:51.879518+0000","last_clean":"2024-05-09T00:45:51.879518+0000","last_became_active":"2024-05-09T00:43:43.883066+0000","last_became_peered":"2024-05-09T00:43:43.883066+0000","last_unstale":"2024-05-09T00:45:51.879518+0000","last_undegraded":"2024-05-09T00:45:51.879518+0000","last_fullsized":"2024-05-09T00:45:51.879518+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:47:49.116004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":41,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.868635+0000","last_change":"2024-05-09T00:43:43.877514+0000","last_active":"2024-05-09T00:45:51.868635+0000","last_peered":"2024-05-09T00:45:51.868635+0000","last_clean":"2024-05-09T00:45:51.868635+0000","last_became_active":"2024-05-09T00:43:43.877170+0000","last_became_peered":"2024-05-09T00:43:43.877170+0000","last_unstale":"2024-05-09T00:45:51.868635+0000","last_undegraded":"2024-05-09T00:45:51.868635+0000","last_fullsized":"2024-05-09T00:45:51.868635+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:47:26.738571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.878663+0000","last_change":"2024-05-09T00:44:23.180294+0000","last_active":"2024-05-09T00:45:51.878663+0000","last_peered":"2024-05-09T00:45:51.878663+0000","last_clean":"2024-05-09T00:45:51.878663+0000","last_became_active":"2024-05-09T00:44:23.179824+0000","last_became_peered":"2024-05-09T00:44:23.179824+0000","last_unstale":"2024-05-09T00:45:51.878663+0000","last_undegraded":"2024-05-09T00:45:51.878663+0000","last_fullsized":"2024-05-09T00:45:51.878663+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:43:20.919660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,9],"acting":[7,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.858638+0000","last_change":"2024-05-09T00:43:43.873836+0000","last_active":"2024-05-09T00:45:51.858638+0000","last_peered":"2024-05-09T00:45:51.858638+0000","last_clean":"2024-05-09T00:45:51.858638+0000","last_became_active":"2024-05-09T00:43:43.873512+0000","last_became_peered":"2024-05-09T00:43:43.873512+0000","last_unstale":"2024-05-09T00:45:51.858638+0000","last_undegraded":"2024-05-09T00:45:51.858638+0000","last_fullsized":"2024-05-09T00:45:51.858638+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:32:06.162638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.864447+0000","last_change":"2024-05-09T00:43:43.885614+0000","last_active":"2024-05-09T00:45:51.864447+0000","last_peered":"2024-05-09T00:45:51.864447+0000","last_clean":"2024-05-09T00:45:51.864447+0000","last_became_active":"2024-05-09T00:43:43.884682+0000","last_became_peered":"2024-05-09T00:43:43.884682+0000","last_unstale":"2024-05-09T00:45:51.864447+0000","last_undegraded":"2024-05-09T00:45:51.864447+0000","last_fullsized":"2024-05-09T00:45:51.864447+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:34:15.336173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.872171+0000","last_change":"2024-05-09T00:43:43.883860+0000","last_active":"2024-05-09T00:45:51.872171+0000","last_peered":"2024-05-09T00:45:51.872171+0000","last_clean":"2024-05-09T00:45:51.872171+0000","last_became_active":"2024-05-09T00:43:43.882870+0000","last_became_peered":"2024-05-09T00:43:43.882870+0000","last_unstale":"2024-05-09T00:45:51.872171+0000","last_undegraded":"2024-05-09T00:45:51.872171+0000","last_fullsized":"2024-05-09T00:45:51.872171+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:17:31.566819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.869693+0000","last_change":"2024-05-09T00:43:43.874114+0000","last_active":"2024-05-09T00:45:51.869693+0000","last_peered":"2024-05-09T00:45:51.869693+0000","last_clean":"2024-05-09T00:45:51.869693+0000","last_became_active":"2024-05-09T00:43:43.873762+0000","last_became_peered":"2024-05-09T00:43:43.873762+0000","last_unstale":"2024-05-09T00:45:51.869693+0000","last_undegraded":"2024-05-09T00:45:51.869693+0000","last_fullsized":"2024-05-09T00:45:51.869693+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:24:27.486100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.879829+0000","last_change":"2024-05-09T00:43:43.872982+0000","last_active":"2024-05-09T00:45:51.879829+0000","last_peered":"2024-05-09T00:45:51.879829+0000","last_clean":"2024-05-09T00:45:51.879829+0000","last_became_active":"2024-05-09T00:43:43.872464+0000","last_became_peered":"2024-05-09T00:43:43.872464+0000","last_unstale":"2024-05-09T00:45:51.879829+0000","last_undegraded":"2024-05-09T00:45:51.879829+0000","last_fullsized":"2024-05-09T00:45:51.879829+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:41.172001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.877289+0000","last_change":"2024-05-09T00:43:43.875662+0000","last_active":"2024-05-09T00:45:51.877289+0000","last_peered":"2024-05-09T00:45:51.877289+0000","last_clean":"2024-05-09T00:45:51.877289+0000","last_became_active":"2024-05-09T00:43:43.874701+0000","last_became_peered":"2024-05-09T00:43:43.874701+0000","last_unstale":"2024-05-09T00:45:51.877289+0000","last_undegraded":"2024-05-09T00:45:51.877289+0000","last_fullsized":"2024-05-09T00:45:51.877289+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:09:19.349190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.864219+0000","last_change":"2024-05-09T00:43:43.876100+0000","last_active":"2024-05-09T00:45:51.864219+0000","last_peered":"2024-05-09T00:45:51.864219+0000","last_clean":"2024-05-09T00:45:51.864219+0000","last_became_active":"2024-05-09T00:43:43.875768+0000","last_became_peered":"2024-05-09T00:43:43.875768+0000","last_unstale":"2024-05-09T00:45:51.864219+0000","last_undegraded":"2024-05-09T00:45:51.864219+0000","last_fullsized":"2024-05-09T00:45:51.864219+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:48:26.843064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.865952+0000","last_change":"2024-05-09T00:43:43.875900+0000","last_active":"2024-05-09T00:45:51.865952+0000","last_peered":"2024-05-09T00:45:51.865952+0000","last_clean":"2024-05-09T00:45:51.865952+0000","last_became_active":"2024-05-09T00:43:43.875497+0000","last_became_peered":"2024-05-09T00:43:43.875497+0000","last_unstale":"2024-05-09T00:45:51.865952+0000","last_undegraded":"2024-05-09T00:45:51.865952+0000","last_fullsized":"2024-05-09T00:45:51.865952+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:54:44.215936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'13","reported_seq":230,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:47:06.232722+0000","last_change":"2024-05-09T00:43:40.854337+0000","last_active":"2024-05-09T00:47:06.232722+0000","last_peered":"2024-05-09T00:47:06.232722+0000","last_clean":"2024-05-09T00:47:06.232722+0000","last_became_active":"2024-05-09T00:43:40.853986+0000","last_became_peered":"2024-05-09T00:43:40.853986+0000","last_unstale":"2024-05-09T00:47:06.232722+0000","last_undegraded":"2024-05-09T00:47:06.232722+0000","last_fullsized":"2024-05-09T00:47:06.232722+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:01:19.128533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":186,"num_read_kb":186,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.865652+0000","last_change":"2024-05-09T00:43:43.878081+0000","last_active":"2024-05-09T00:45:51.865652+0000","last_peered":"2024-05-09T00:45:51.865652+0000","last_clean":"2024-05-09T00:45:51.865652+0000","last_became_active":"2024-05-09T00:43:43.877585+0000","last_became_peered":"2024-05-09T00:43:43.877585+0000","last_unstale":"2024-05-09T00:45:51.865652+0000","last_undegraded":"2024-05-09T00:45:51.865652+0000","last_fullsized":"2024-05-09T00:45:51.865652+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:17:39.255841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.115936+0000","last_change":"2024-05-09T00:43:40.959386+0000","last_active":"2024-05-09T00:45:24.115936+0000","last_peered":"2024-05-09T00:45:24.115936+0000","last_clean":"2024-05-09T00:45:24.115936+0000","last_became_active":"2024-05-09T00:43:40.958878+0000","last_became_peered":"2024-05-09T00:43:40.958878+0000","last_unstale":"2024-05-09T00:45:24.115936+0000","last_undegraded":"2024-05-09T00:45:24.115936+0000","last_fullsized":"2024-05-09T00:45:24.115936+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:14:57.105078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.873650+0000","last_change":"2024-05-09T00:43:43.870563+0000","last_active":"2024-05-09T00:45:51.873650+0000","last_peered":"2024-05-09T00:45:51.873650+0000","last_clean":"2024-05-09T00:45:51.873650+0000","last_became_active":"2024-05-09T00:43:43.870237+0000","last_became_peered":"2024-05-09T00:43:43.870237+0000","last_unstale":"2024-05-09T00:45:51.873650+0000","last_undegraded":"2024-05-09T00:45:51.873650+0000","last_fullsized":"2024-05-09T00:45:51.873650+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:14:28.006343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"78'12","reported_seq":44,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.937368+0000","last_change":"2024-05-09T00:43:40.857670+0000","last_active":"2024-05-09T00:45:23.937368+0000","last_peered":"2024-05-09T00:45:23.937368+0000","last_clean":"2024-05-09T00:45:23.937368+0000","last_became_active":"2024-05-09T00:43:40.857247+0000","last_became_peered":"2024-05-09T00:43:40.857247+0000","last_unstale":"2024-05-09T00:45:23.937368+0000","last_undegraded":"2024-05-09T00:45:23.937368+0000","last_fullsized":"2024-05-09T00:45:23.937368+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:47:06.325593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.873557+0000","last_change":"2024-05-09T00:43:43.883179+0000","last_active":"2024-05-09T00:45:51.873557+0000","last_peered":"2024-05-09T00:45:51.873557+0000","last_clean":"2024-05-09T00:45:51.873557+0000","last_became_active":"2024-05-09T00:43:43.882820+0000","last_became_peered":"2024-05-09T00:43:43.882820+0000","last_unstale":"2024-05-09T00:45:51.873557+0000","last_undegraded":"2024-05-09T00:45:51.873557+0000","last_fullsized":"2024-05-09T00:45:51.873557+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:56:26.012495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"78'5","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.946133+0000","last_change":"2024-05-09T00:45:23.945720+0000","last_active":"2024-05-09T00:45:23.946133+0000","last_peered":"2024-05-09T00:45:23.946133+0000","last_clean":"2024-05-09T00:45:23.946133+0000","last_became_active":"2024-05-09T00:45:23.940421+0000","last_became_peered":"2024-05-09T00:45:23.940421+0000","last_unstale":"2024-05-09T00:45:23.946133+0000","last_undegraded":"2024-05-09T00:45:23.946133+0000","last_fullsized":"2024-05-09T00:45:23.946133+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:23:24.905715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.876297+0000","last_change":"2024-05-09T00:43:43.882950+0000","last_active":"2024-05-09T00:45:51.876297+0000","last_peered":"2024-05-09T00:45:51.876297+0000","last_clean":"2024-05-09T00:45:51.876297+0000","last_became_active":"2024-05-09T00:43:43.880679+0000","last_became_peered":"2024-05-09T00:43:43.880679+0000","last_unstale":"2024-05-09T00:45:51.876297+0000","last_undegraded":"2024-05-09T00:45:51.876297+0000","last_fullsized":"2024-05-09T00:45:51.876297+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:34:14.050987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"82'7","reported_seq":227,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:47:06.234636+0000","last_change":"2024-05-09T00:43:40.854953+0000","last_active":"2024-05-09T00:47:06.234636+0000","last_peered":"2024-05-09T00:47:06.234636+0000","last_clean":"2024-05-09T00:47:06.234636+0000","last_became_active":"2024-05-09T00:43:40.854562+0000","last_became_peered":"2024-05-09T00:43:40.854562+0000","last_unstale":"2024-05-09T00:47:06.234636+0000","last_undegraded":"2024-05-09T00:47:06.234636+0000","last_fullsized":"2024-05-09T00:47:06.234636+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:09:19.564806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":189,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.857389+0000","last_change":"2024-05-09T00:43:43.881881+0000","last_active":"2024-05-09T00:45:51.857389+0000","last_peered":"2024-05-09T00:45:51.857389+0000","last_clean":"2024-05-09T00:45:51.857389+0000","last_became_active":"2024-05-09T00:43:43.881461+0000","last_became_peered":"2024-05-09T00:43:43.881461+0000","last_unstale":"2024-05-09T00:45:51.857389+0000","last_undegraded":"2024-05-09T00:45:51.857389+0000","last_fullsized":"2024-05-09T00:45:51.857389+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:59:28.747084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"78'7","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.080620+0000","last_change":"2024-05-09T00:43:40.858521+0000","last_active":"2024-05-09T00:45:24.080620+0000","last_peered":"2024-05-09T00:45:24.080620+0000","last_clean":"2024-05-09T00:45:24.080620+0000","last_became_active":"2024-05-09T00:43:40.858102+0000","last_became_peered":"2024-05-09T00:43:40.858102+0000","last_unstale":"2024-05-09T00:45:24.080620+0000","last_undegraded":"2024-05-09T00:45:24.080620+0000","last_fullsized":"2024-05-09T00:45:24.080620+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:09:51.684630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.873396+0000","last_change":"2024-05-09T00:44:23.023472+0000","last_active":"2024-05-09T00:45:51.873396+0000","last_peered":"2024-05-09T00:45:51.873396+0000","last_clean":"2024-05-09T00:45:51.873396+0000","last_became_active":"2024-05-09T00:44:23.023017+0000","last_became_peered":"2024-05-09T00:44:23.023017+0000","last_unstale":"2024-05-09T00:45:51.873396+0000","last_undegraded":"2024-05-09T00:45:51.873396+0000","last_fullsized":"2024-05-09T00:45:51.873396+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:05:13.852606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"78'3","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.934006+0000","last_change":"2024-05-09T00:43:40.853329+0000","last_active":"2024-05-09T00:45:23.934006+0000","last_peered":"2024-05-09T00:45:23.934006+0000","last_clean":"2024-05-09T00:45:23.934006+0000","last_became_active":"2024-05-09T00:43:40.852979+0000","last_became_peered":"2024-05-09T00:43:40.852979+0000","last_unstale":"2024-05-09T00:45:23.934006+0000","last_undegraded":"2024-05-09T00:45:23.934006+0000","last_fullsized":"2024-05-09T00:45:23.934006+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:13:57.006031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.875355+0000","last_change":"2024-05-09T00:43:43.862876+0000","last_active":"2024-05-09T00:45:51.875355+0000","last_peered":"2024-05-09T00:45:51.875355+0000","last_clean":"2024-05-09T00:45:51.875355+0000","last_became_active":"2024-05-09T00:43:43.862521+0000","last_became_peered":"2024-05-09T00:43:43.862521+0000","last_unstale":"2024-05-09T00:45:51.875355+0000","last_undegraded":"2024-05-09T00:45:51.875355+0000","last_fullsized":"2024-05-09T00:45:51.875355+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:45:49.386844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'3","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.937546+0000","last_change":"2024-05-09T00:43:40.860395+0000","last_active":"2024-05-09T00:45:23.937546+0000","last_peered":"2024-05-09T00:45:23.937546+0000","last_clean":"2024-05-09T00:45:23.937546+0000","last_became_active":"2024-05-09T00:43:40.860040+0000","last_became_peered":"2024-05-09T00:43:40.860040+0000","last_unstale":"2024-05-09T00:45:23.937546+0000","last_undegraded":"2024-05-09T00:45:23.937546+0000","last_fullsized":"2024-05-09T00:45:23.937546+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:20:36.054685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.862285+0000","last_change":"2024-05-09T00:44:23.180905+0000","last_active":"2024-05-09T00:45:51.862285+0000","last_peered":"2024-05-09T00:45:51.862285+0000","last_clean":"2024-05-09T00:45:51.862285+0000","last_became_active":"2024-05-09T00:44:23.180296+0000","last_became_peered":"2024-05-09T00:44:23.180296+0000","last_unstale":"2024-05-09T00:45:51.862285+0000","last_undegraded":"2024-05-09T00:45:51.862285+0000","last_fullsized":"2024-05-09T00:45:51.862285+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:48:17.585276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,10],"acting":[9,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'6","reported_seq":407,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:47:07.235641+0000","last_change":"2024-05-09T00:43:40.860542+0000","last_active":"2024-05-09T00:47:07.235641+0000","last_peered":"2024-05-09T00:47:07.235641+0000","last_clean":"2024-05-09T00:47:07.235641+0000","last_became_active":"2024-05-09T00:43:40.860176+0000","last_became_peered":"2024-05-09T00:43:40.860176+0000","last_unstale":"2024-05-09T00:47:07.235641+0000","last_undegraded":"2024-05-09T00:47:07.235641+0000","last_fullsized":"2024-05-09T00:47:07.235641+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:44:50.375024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":370,"num_read_kb":370,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.872797+0000","last_change":"2024-05-09T00:43:43.882119+0000","last_active":"2024-05-09T00:45:51.872797+0000","last_peered":"2024-05-09T00:45:51.872797+0000","last_clean":"2024-05-09T00:45:51.872797+0000","last_became_active":"2024-05-09T00:43:43.881747+0000","last_became_peered":"2024-05-09T00:43:43.881747+0000","last_unstale":"2024-05-09T00:45:51.872797+0000","last_undegraded":"2024-05-09T00:45:51.872797+0000","last_fullsized":"2024-05-09T00:45:51.872797+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:54:19.420435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'4","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.081294+0000","last_change":"2024-05-09T00:43:40.859539+0000","last_active":"2024-05-09T00:45:24.081294+0000","last_peered":"2024-05-09T00:45:24.081294+0000","last_clean":"2024-05-09T00:45:24.081294+0000","last_became_active":"2024-05-09T00:43:40.859162+0000","last_became_peered":"2024-05-09T00:43:40.859162+0000","last_unstale":"2024-05-09T00:45:24.081294+0000","last_undegraded":"2024-05-09T00:45:24.081294+0000","last_fullsized":"2024-05-09T00:45:24.081294+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:37:15.190944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.872246+0000","last_change":"2024-05-09T00:44:23.180872+0000","last_active":"2024-05-09T00:45:51.872246+0000","last_peered":"2024-05-09T00:45:51.872246+0000","last_clean":"2024-05-09T00:45:51.872246+0000","last_became_active":"2024-05-09T00:44:23.180398+0000","last_became_peered":"2024-05-09T00:44:23.180398+0000","last_unstale":"2024-05-09T00:45:51.872246+0000","last_undegraded":"2024-05-09T00:45:51.872246+0000","last_fullsized":"2024-05-09T00:45:51.872246+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:19:27.909490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,6],"acting":[0,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'5","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.938347+0000","last_change":"2024-05-09T00:43:40.960126+0000","last_active":"2024-05-09T00:45:23.938347+0000","last_peered":"2024-05-09T00:45:23.938347+0000","last_clean":"2024-05-09T00:45:23.938347+0000","last_became_active":"2024-05-09T00:43:40.959441+0000","last_became_peered":"2024-05-09T00:43:40.959441+0000","last_unstale":"2024-05-09T00:45:23.938347+0000","last_undegraded":"2024-05-09T00:45:23.938347+0000","last_fullsized":"2024-05-09T00:45:23.938347+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:38:47.497029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.881221+0000","last_change":"2024-05-09T00:43:43.874223+0000","last_active":"2024-05-09T00:45:51.881221+0000","last_peered":"2024-05-09T00:45:51.881221+0000","last_clean":"2024-05-09T00:45:51.881221+0000","last_became_active":"2024-05-09T00:43:43.873875+0000","last_became_peered":"2024-05-09T00:43:43.873875+0000","last_unstale":"2024-05-09T00:45:51.881221+0000","last_undegraded":"2024-05-09T00:45:51.881221+0000","last_fullsized":"2024-05-09T00:45:51.881221+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:59:22.192886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'37","reported_seq":469,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:47:07.457164+0000","last_change":"2024-05-09T00:44:23.188953+0000","last_active":"2024-05-09T00:47:07.457164+0000","last_peered":"2024-05-09T00:47:07.457164+0000","last_clean":"2024-05-09T00:47:07.457164+0000","last_became_active":"2024-05-09T00:44:23.180858+0000","last_became_peered":"2024-05-09T00:44:23.180858+0000","last_unstale":"2024-05-09T00:47:07.457164+0000","last_undegraded":"2024-05-09T00:47:07.457164+0000","last_fullsized":"2024-05-09T00:47:07.457164+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:26:16.427665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62892,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":376,"num_read_kb":239,"num_write":39,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":52683,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.871155+0000","last_change":"2024-05-09T00:43:43.872019+0000","last_active":"2024-05-09T00:45:51.871155+0000","last_peered":"2024-05-09T00:45:51.871155+0000","last_clean":"2024-05-09T00:45:51.871155+0000","last_became_active":"2024-05-09T00:43:43.871622+0000","last_became_peered":"2024-05-09T00:43:43.871622+0000","last_unstale":"2024-05-09T00:45:51.871155+0000","last_undegraded":"2024-05-09T00:45:51.871155+0000","last_fullsized":"2024-05-09T00:45:51.871155+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:41:47.789443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'9","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.116231+0000","last_change":"2024-05-09T00:43:40.860111+0000","last_active":"2024-05-09T00:45:24.116231+0000","last_peered":"2024-05-09T00:45:24.116231+0000","last_clean":"2024-05-09T00:45:24.116231+0000","last_became_active":"2024-05-09T00:43:40.859732+0000","last_became_peered":"2024-05-09T00:43:40.859732+0000","last_unstale":"2024-05-09T00:45:24.116231+0000","last_undegraded":"2024-05-09T00:45:24.116231+0000","last_fullsized":"2024-05-09T00:45:24.116231+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:30:03.180679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.867875+0000","last_change":"2024-05-09T00:43:43.870604+0000","last_active":"2024-05-09T00:45:51.867875+0000","last_peered":"2024-05-09T00:45:51.867875+0000","last_clean":"2024-05-09T00:45:51.867875+0000","last_became_active":"2024-05-09T00:43:43.870286+0000","last_became_peered":"2024-05-09T00:43:43.870286+0000","last_unstale":"2024-05-09T00:45:51.867875+0000","last_undegraded":"2024-05-09T00:45:51.867875+0000","last_fullsized":"2024-05-09T00:45:51.867875+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:57:04.486631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'5","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.937866+0000","last_change":"2024-05-09T00:43:40.851966+0000","last_active":"2024-05-09T00:45:23.937866+0000","last_peered":"2024-05-09T00:45:23.937866+0000","last_clean":"2024-05-09T00:45:23.937866+0000","last_became_active":"2024-05-09T00:43:40.851537+0000","last_became_peered":"2024-05-09T00:43:40.851537+0000","last_unstale":"2024-05-09T00:45:23.937866+0000","last_undegraded":"2024-05-09T00:45:23.937866+0000","last_fullsized":"2024-05-09T00:45:23.937866+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:58:59.797183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.879660+0000","last_change":"2024-05-09T00:43:43.879509+0000","last_active":"2024-05-09T00:45:51.879660+0000","last_peered":"2024-05-09T00:45:51.879660+0000","last_clean":"2024-05-09T00:45:51.879660+0000","last_became_active":"2024-05-09T00:43:43.879151+0000","last_became_peered":"2024-05-09T00:43:43.879151+0000","last_unstale":"2024-05-09T00:45:51.879660+0000","last_undegraded":"2024-05-09T00:45:51.879660+0000","last_fullsized":"2024-05-09T00:45:51.879660+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:34:18.850545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'8","reported_seq":228,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:47:06.233264+0000","last_change":"2024-05-09T00:43:40.857629+0000","last_active":"2024-05-09T00:47:06.233264+0000","last_peered":"2024-05-09T00:47:06.233264+0000","last_clean":"2024-05-09T00:47:06.233264+0000","last_became_active":"2024-05-09T00:43:40.857260+0000","last_became_peered":"2024-05-09T00:43:40.857260+0000","last_unstale":"2024-05-09T00:47:06.233264+0000","last_undegraded":"2024-05-09T00:47:06.233264+0000","last_fullsized":"2024-05-09T00:47:06.233264+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:51:00.600870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1428,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":190,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.875060+0000","last_change":"2024-05-09T00:43:43.886100+0000","last_active":"2024-05-09T00:45:51.875060+0000","last_peered":"2024-05-09T00:45:51.875060+0000","last_clean":"2024-05-09T00:45:51.875060+0000","last_became_active":"2024-05-09T00:43:43.885775+0000","last_became_peered":"2024-05-09T00:43:43.885775+0000","last_unstale":"2024-05-09T00:45:51.875060+0000","last_undegraded":"2024-05-09T00:45:51.875060+0000","last_fullsized":"2024-05-09T00:45:51.875060+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:34:19.439163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'17","reported_seq":49,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.936845+0000","last_change":"2024-05-09T00:43:40.959873+0000","last_active":"2024-05-09T00:45:23.936845+0000","last_peered":"2024-05-09T00:45:23.936845+0000","last_clean":"2024-05-09T00:45:23.936845+0000","last_became_active":"2024-05-09T00:43:40.959437+0000","last_became_peered":"2024-05-09T00:43:40.959437+0000","last_unstale":"2024-05-09T00:45:23.936845+0000","last_undegraded":"2024-05-09T00:45:23.936845+0000","last_fullsized":"2024-05-09T00:45:23.936845+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:29:52.686937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.875543+0000","last_change":"2024-05-09T00:43:43.880076+0000","last_active":"2024-05-09T00:45:51.875543+0000","last_peered":"2024-05-09T00:45:51.875543+0000","last_clean":"2024-05-09T00:45:51.875543+0000","last_became_active":"2024-05-09T00:43:43.878733+0000","last_became_peered":"2024-05-09T00:43:43.878733+0000","last_unstale":"2024-05-09T00:45:51.875543+0000","last_undegraded":"2024-05-09T00:45:51.875543+0000","last_fullsized":"2024-05-09T00:45:51.875543+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:30:21.426855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'3","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:59.646186+0000","last_change":"2024-05-09T00:43:40.858033+0000","last_active":"2024-05-09T00:45:59.646186+0000","last_peered":"2024-05-09T00:45:59.646186+0000","last_clean":"2024-05-09T00:45:59.646186+0000","last_became_active":"2024-05-09T00:43:40.857075+0000","last_became_peered":"2024-05-09T00:43:40.857075+0000","last_unstale":"2024-05-09T00:45:59.646186+0000","last_undegraded":"2024-05-09T00:45:59.646186+0000","last_fullsized":"2024-05-09T00:45:59.646186+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:01:49.690226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.871817+0000","last_change":"2024-05-09T00:44:23.180515+0000","last_active":"2024-05-09T00:45:51.871817+0000","last_peered":"2024-05-09T00:45:51.871817+0000","last_clean":"2024-05-09T00:45:51.871817+0000","last_became_active":"2024-05-09T00:44:23.179834+0000","last_became_peered":"2024-05-09T00:44:23.179834+0000","last_unstale":"2024-05-09T00:45:51.871817+0000","last_undegraded":"2024-05-09T00:45:51.871817+0000","last_fullsized":"2024-05-09T00:45:51.871817+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:21:55.735438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,6],"acting":[11,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"78'1","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.941920+0000","last_change":"2024-05-09T00:45:23.941489+0000","last_active":"2024-05-09T00:45:23.941920+0000","last_peered":"2024-05-09T00:45:23.941920+0000","last_clean":"2024-05-09T00:45:23.941920+0000","last_became_active":"2024-05-09T00:45:23.941072+0000","last_became_peered":"2024-05-09T00:45:23.941072+0000","last_unstale":"2024-05-09T00:45:23.941920+0000","last_undegraded":"2024-05-09T00:45:23.941920+0000","last_fullsized":"2024-05-09T00:45:23.941920+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:20:05.755293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.849757+0000","last_change":"2024-05-09T00:43:43.886618+0000","last_active":"2024-05-09T00:45:51.849757+0000","last_peered":"2024-05-09T00:45:51.849757+0000","last_clean":"2024-05-09T00:45:51.849757+0000","last_became_active":"2024-05-09T00:43:43.885946+0000","last_became_peered":"2024-05-09T00:43:43.885946+0000","last_unstale":"2024-05-09T00:45:51.849757+0000","last_undegraded":"2024-05-09T00:45:51.849757+0000","last_fullsized":"2024-05-09T00:45:51.849757+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:16:22.169548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"78'5","reported_seq":224,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:47:08.235950+0000","last_change":"2024-05-09T00:43:40.856213+0000","last_active":"2024-05-09T00:47:08.235950+0000","last_peered":"2024-05-09T00:47:08.235950+0000","last_clean":"2024-05-09T00:47:08.235950+0000","last_became_active":"2024-05-09T00:43:40.855700+0000","last_became_peered":"2024-05-09T00:43:40.855700+0000","last_unstale":"2024-05-09T00:47:08.235950+0000","last_undegraded":"2024-05-09T00:47:08.235950+0000","last_fullsized":"2024-05-09T00:47:08.235950+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:48:50.238293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":188,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.857202+0000","last_change":"2024-05-09T00:43:43.885272+0000","last_active":"2024-05-09T00:45:51.857202+0000","last_peered":"2024-05-09T00:45:51.857202+0000","last_clean":"2024-05-09T00:45:51.857202+0000","last_became_active":"2024-05-09T00:43:43.884774+0000","last_became_peered":"2024-05-09T00:43:43.884774+0000","last_unstale":"2024-05-09T00:45:51.857202+0000","last_undegraded":"2024-05-09T00:45:51.857202+0000","last_fullsized":"2024-05-09T00:45:51.857202+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:14:02.193859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.863000+0000","last_change":"2024-05-09T00:43:43.883370+0000","last_active":"2024-05-09T00:45:51.863000+0000","last_peered":"2024-05-09T00:45:51.863000+0000","last_clean":"2024-05-09T00:45:51.863000+0000","last_became_active":"2024-05-09T00:43:43.881570+0000","last_became_peered":"2024-05-09T00:43:43.881570+0000","last_unstale":"2024-05-09T00:45:51.863000+0000","last_undegraded":"2024-05-09T00:45:51.863000+0000","last_fullsized":"2024-05-09T00:45:51.863000+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:48.936861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'7","reported_seq":225,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:47:07.234443+0000","last_change":"2024-05-09T00:43:40.959828+0000","last_active":"2024-05-09T00:47:07.234443+0000","last_peered":"2024-05-09T00:47:07.234443+0000","last_clean":"2024-05-09T00:47:07.234443+0000","last_became_active":"2024-05-09T00:43:40.958889+0000","last_became_peered":"2024-05-09T00:43:40.958889+0000","last_unstale":"2024-05-09T00:47:07.234443+0000","last_undegraded":"2024-05-09T00:47:07.234443+0000","last_fullsized":"2024-05-09T00:47:07.234443+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:24:50.113298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":187,"num_read_kb":187,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.861559+0000","last_change":"2024-05-09T00:43:43.876144+0000","last_active":"2024-05-09T00:45:51.861559+0000","last_peered":"2024-05-09T00:45:51.861559+0000","last_clean":"2024-05-09T00:45:51.861559+0000","last_became_active":"2024-05-09T00:43:43.875774+0000","last_became_peered":"2024-05-09T00:43:43.875774+0000","last_unstale":"2024-05-09T00:45:51.861559+0000","last_undegraded":"2024-05-09T00:45:51.861559+0000","last_fullsized":"2024-05-09T00:45:51.861559+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:05:23.145431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"78'6","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.939598+0000","last_change":"2024-05-09T00:43:40.857278+0000","last_active":"2024-05-09T00:45:23.939598+0000","last_peered":"2024-05-09T00:45:23.939598+0000","last_clean":"2024-05-09T00:45:23.939598+0000","last_became_active":"2024-05-09T00:43:40.856861+0000","last_became_peered":"2024-05-09T00:43:40.856861+0000","last_unstale":"2024-05-09T00:45:23.939598+0000","last_undegraded":"2024-05-09T00:45:23.939598+0000","last_fullsized":"2024-05-09T00:45:23.939598+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:36:16.603327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":104,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.080451+0000","last_change":"2024-05-09T00:42:14.528544+0000","last_active":"2024-05-09T00:45:24.080451+0000","last_peered":"2024-05-09T00:45:24.080451+0000","last_clean":"2024-05-09T00:45:24.080451+0000","last_became_active":"2024-05-09T00:42:14.523658+0000","last_became_peered":"2024-05-09T00:42:14.523658+0000","last_unstale":"2024-05-09T00:45:24.080451+0000","last_undegraded":"2024-05-09T00:45:24.080451+0000","last_fullsized":"2024-05-09T00:45:24.080451+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:38:48.637995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:38:48.637995+0000","last_clean_scrub_stamp":"2024-05-09T00:38:48.637995+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:42:28.249240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.852620+0000","last_change":"2024-05-09T00:43:43.873638+0000","last_active":"2024-05-09T00:45:51.852620+0000","last_peered":"2024-05-09T00:45:51.852620+0000","last_clean":"2024-05-09T00:45:51.852620+0000","last_became_active":"2024-05-09T00:43:43.871069+0000","last_became_peered":"2024-05-09T00:43:43.871069+0000","last_unstale":"2024-05-09T00:45:51.852620+0000","last_undegraded":"2024-05-09T00:45:51.852620+0000","last_fullsized":"2024-05-09T00:45:51.852620+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:28:34.996989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"78'4","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.080673+0000","last_change":"2024-05-09T00:43:40.856374+0000","last_active":"2024-05-09T00:45:24.080673+0000","last_peered":"2024-05-09T00:45:24.080673+0000","last_clean":"2024-05-09T00:45:24.080673+0000","last_became_active":"2024-05-09T00:43:40.855695+0000","last_became_peered":"2024-05-09T00:43:40.855695+0000","last_unstale":"2024-05-09T00:45:24.080673+0000","last_undegraded":"2024-05-09T00:45:24.080673+0000","last_fullsized":"2024-05-09T00:45:24.080673+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:33:43.734132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"78'4","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.937141+0000","last_change":"2024-05-09T00:43:40.959303+0000","last_active":"2024-05-09T00:45:23.937141+0000","last_peered":"2024-05-09T00:45:23.937141+0000","last_clean":"2024-05-09T00:45:23.937141+0000","last_became_active":"2024-05-09T00:43:40.958855+0000","last_became_peered":"2024-05-09T00:43:40.958855+0000","last_unstale":"2024-05-09T00:45:23.937141+0000","last_undegraded":"2024-05-09T00:45:23.937141+0000","last_fullsized":"2024-05-09T00:45:23.937141+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:53:22.566850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.861738+0000","last_change":"2024-05-09T00:43:43.872041+0000","last_active":"2024-05-09T00:45:51.861738+0000","last_peered":"2024-05-09T00:45:51.861738+0000","last_clean":"2024-05-09T00:45:51.861738+0000","last_became_active":"2024-05-09T00:43:43.870946+0000","last_became_peered":"2024-05-09T00:43:43.870946+0000","last_unstale":"2024-05-09T00:45:51.861738+0000","last_undegraded":"2024-05-09T00:45:51.861738+0000","last_fullsized":"2024-05-09T00:45:51.861738+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:16:28.190488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'17","reported_seq":48,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:52.555000+0000","last_change":"2024-05-09T00:43:40.853875+0000","last_active":"2024-05-09T00:45:52.555000+0000","last_peered":"2024-05-09T00:45:52.555000+0000","last_clean":"2024-05-09T00:45:52.555000+0000","last_became_active":"2024-05-09T00:43:40.853459+0000","last_became_peered":"2024-05-09T00:43:40.853459+0000","last_unstale":"2024-05-09T00:45:52.555000+0000","last_undegraded":"2024-05-09T00:45:52.555000+0000","last_fullsized":"2024-05-09T00:45:52.555000+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:45:29.992730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.876758+0000","last_change":"2024-05-09T00:43:43.870435+0000","last_active":"2024-05-09T00:45:51.876758+0000","last_peered":"2024-05-09T00:45:51.876758+0000","last_clean":"2024-05-09T00:45:51.876758+0000","last_became_active":"2024-05-09T00:43:43.870075+0000","last_became_peered":"2024-05-09T00:43:43.870075+0000","last_unstale":"2024-05-09T00:45:51.876758+0000","last_undegraded":"2024-05-09T00:45:51.876758+0000","last_fullsized":"2024-05-09T00:45:51.876758+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:53:12.958463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'7","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.939824+0000","last_change":"2024-05-09T00:43:41.119325+0000","last_active":"2024-05-09T00:45:23.939824+0000","last_peered":"2024-05-09T00:45:23.939824+0000","last_clean":"2024-05-09T00:45:23.939824+0000","last_became_active":"2024-05-09T00:43:41.119009+0000","last_became_peered":"2024-05-09T00:43:41.119009+0000","last_unstale":"2024-05-09T00:45:23.939824+0000","last_undegraded":"2024-05-09T00:45:23.939824+0000","last_fullsized":"2024-05-09T00:45:23.939824+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:10:05.775152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.873671+0000","last_change":"2024-05-09T00:43:43.875236+0000","last_active":"2024-05-09T00:45:51.873671+0000","last_peered":"2024-05-09T00:45:51.873671+0000","last_clean":"2024-05-09T00:45:51.873671+0000","last_became_active":"2024-05-09T00:43:43.874919+0000","last_became_peered":"2024-05-09T00:43:43.874919+0000","last_unstale":"2024-05-09T00:45:51.873671+0000","last_undegraded":"2024-05-09T00:45:51.873671+0000","last_fullsized":"2024-05-09T00:45:51.873671+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:45:33.232518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.938556+0000","last_change":"2024-05-09T00:43:40.959765+0000","last_active":"2024-05-09T00:45:23.938556+0000","last_peered":"2024-05-09T00:45:23.938556+0000","last_clean":"2024-05-09T00:45:23.938556+0000","last_became_active":"2024-05-09T00:43:40.959383+0000","last_became_peered":"2024-05-09T00:43:40.959383+0000","last_unstale":"2024-05-09T00:45:23.938556+0000","last_undegraded":"2024-05-09T00:45:23.938556+0000","last_fullsized":"2024-05-09T00:45:23.938556+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:13:28.766650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.870027+0000","last_change":"2024-05-09T00:43:43.881627+0000","last_active":"2024-05-09T00:45:51.870027+0000","last_peered":"2024-05-09T00:45:51.870027+0000","last_clean":"2024-05-09T00:45:51.870027+0000","last_became_active":"2024-05-09T00:43:43.881207+0000","last_became_peered":"2024-05-09T00:43:43.881207+0000","last_unstale":"2024-05-09T00:45:51.870027+0000","last_undegraded":"2024-05-09T00:45:51.870027+0000","last_fullsized":"2024-05-09T00:45:51.870027+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:44:08.053638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'7","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.115513+0000","last_change":"2024-05-09T00:43:40.859566+0000","last_active":"2024-05-09T00:45:24.115513+0000","last_peered":"2024-05-09T00:45:24.115513+0000","last_clean":"2024-05-09T00:45:24.115513+0000","last_became_active":"2024-05-09T00:43:40.859182+0000","last_became_peered":"2024-05-09T00:43:40.859182+0000","last_unstale":"2024-05-09T00:45:24.115513+0000","last_undegraded":"2024-05-09T00:45:24.115513+0000","last_fullsized":"2024-05-09T00:45:24.115513+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:58:12.693271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.874341+0000","last_change":"2024-05-09T00:43:43.869193+0000","last_active":"2024-05-09T00:45:51.874341+0000","last_peered":"2024-05-09T00:45:51.874341+0000","last_clean":"2024-05-09T00:45:51.874341+0000","last_became_active":"2024-05-09T00:43:43.868823+0000","last_became_peered":"2024-05-09T00:43:43.868823+0000","last_unstale":"2024-05-09T00:45:51.874341+0000","last_undegraded":"2024-05-09T00:45:51.874341+0000","last_fullsized":"2024-05-09T00:45:51.874341+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:15:53.929873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'3","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:24.116148+0000","last_change":"2024-05-09T00:43:40.855038+0000","last_active":"2024-05-09T00:45:24.116148+0000","last_peered":"2024-05-09T00:45:24.116148+0000","last_clean":"2024-05-09T00:45:24.116148+0000","last_became_active":"2024-05-09T00:43:40.854334+0000","last_became_peered":"2024-05-09T00:43:40.854334+0000","last_unstale":"2024-05-09T00:45:24.116148+0000","last_undegraded":"2024-05-09T00:45:24.116148+0000","last_fullsized":"2024-05-09T00:45:24.116148+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:47:13.897697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.863165+0000","last_change":"2024-05-09T00:44:23.019151+0000","last_active":"2024-05-09T00:45:51.863165+0000","last_peered":"2024-05-09T00:45:51.863165+0000","last_clean":"2024-05-09T00:45:51.863165+0000","last_became_active":"2024-05-09T00:44:23.018752+0000","last_became_peered":"2024-05-09T00:44:23.018752+0000","last_unstale":"2024-05-09T00:45:51.863165+0000","last_undegraded":"2024-05-09T00:45:51.863165+0000","last_fullsized":"2024-05-09T00:45:51.863165+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:28:41.875747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.937463+0000","last_change":"2024-05-09T00:43:40.859461+0000","last_active":"2024-05-09T00:45:23.937463+0000","last_peered":"2024-05-09T00:45:23.937463+0000","last_clean":"2024-05-09T00:45:23.937463+0000","last_became_active":"2024-05-09T00:43:40.857814+0000","last_became_peered":"2024-05-09T00:43:40.857814+0000","last_unstale":"2024-05-09T00:45:23.937463+0000","last_undegraded":"2024-05-09T00:45:23.937463+0000","last_fullsized":"2024-05-09T00:45:23.937463+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:56:06.279762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.877886+0000","last_change":"2024-05-09T00:43:43.878120+0000","last_active":"2024-05-09T00:45:51.877886+0000","last_peered":"2024-05-09T00:45:51.877886+0000","last_clean":"2024-05-09T00:45:51.877886+0000","last_became_active":"2024-05-09T00:43:43.877782+0000","last_became_peered":"2024-05-09T00:43:43.877782+0000","last_unstale":"2024-05-09T00:45:51.877886+0000","last_undegraded":"2024-05-09T00:45:51.877886+0000","last_fullsized":"2024-05-09T00:45:51.877886+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:19:48.558992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'7","reported_seq":227,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:47:06.304442+0000","last_change":"2024-05-09T00:43:41.119347+0000","last_active":"2024-05-09T00:47:06.304442+0000","last_peered":"2024-05-09T00:47:06.304442+0000","last_clean":"2024-05-09T00:47:06.304442+0000","last_became_active":"2024-05-09T00:43:41.118847+0000","last_became_peered":"2024-05-09T00:43:41.118847+0000","last_unstale":"2024-05-09T00:47:06.304442+0000","last_undegraded":"2024-05-09T00:47:06.304442+0000","last_fullsized":"2024-05-09T00:47:06.304442+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:07:03.977756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":189,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.878473+0000","last_change":"2024-05-09T00:43:43.875203+0000","last_active":"2024-05-09T00:45:51.878473+0000","last_peered":"2024-05-09T00:45:51.878473+0000","last_clean":"2024-05-09T00:45:51.878473+0000","last_became_active":"2024-05-09T00:43:43.874882+0000","last_became_peered":"2024-05-09T00:43:43.874882+0000","last_unstale":"2024-05-09T00:45:51.878473+0000","last_undegraded":"2024-05-09T00:45:51.878473+0000","last_fullsized":"2024-05-09T00:45:51.878473+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:54:27.845086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'3","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.938177+0000","last_change":"2024-05-09T00:43:40.857955+0000","last_active":"2024-05-09T00:45:23.938177+0000","last_peered":"2024-05-09T00:45:23.938177+0000","last_clean":"2024-05-09T00:45:23.938177+0000","last_became_active":"2024-05-09T00:43:40.857596+0000","last_became_peered":"2024-05-09T00:43:40.857596+0000","last_unstale":"2024-05-09T00:45:23.938177+0000","last_undegraded":"2024-05-09T00:45:23.938177+0000","last_fullsized":"2024-05-09T00:45:23.938177+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:13:56.676929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.872219+0000","last_change":"2024-05-09T00:43:43.882487+0000","last_active":"2024-05-09T00:45:51.872219+0000","last_peered":"2024-05-09T00:45:51.872219+0000","last_clean":"2024-05-09T00:45:51.872219+0000","last_became_active":"2024-05-09T00:43:43.882176+0000","last_became_peered":"2024-05-09T00:43:43.882176+0000","last_unstale":"2024-05-09T00:45:51.872219+0000","last_undegraded":"2024-05-09T00:45:51.872219+0000","last_fullsized":"2024-05-09T00:45:51.872219+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:48:42.354433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'5","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:23.934749+0000","last_change":"2024-05-09T00:43:40.857215+0000","last_active":"2024-05-09T00:45:23.934749+0000","last_peered":"2024-05-09T00:45:23.934749+0000","last_clean":"2024-05-09T00:45:23.934749+0000","last_became_active":"2024-05-09T00:43:40.856858+0000","last_became_peered":"2024-05-09T00:43:40.856858+0000","last_unstale":"2024-05-09T00:45:23.934749+0000","last_undegraded":"2024-05-09T00:45:23.934749+0000","last_fullsized":"2024-05-09T00:45:23.934749+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:39.831690+0000","last_clean_scrub_stamp":"2024-05-09T00:43:39.831690+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:40:47.943482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.878949+0000","last_change":"2024-05-09T00:43:43.881192+0000","last_active":"2024-05-09T00:45:51.878949+0000","last_peered":"2024-05-09T00:45:51.878949+0000","last_clean":"2024-05-09T00:45:51.878949+0000","last_became_active":"2024-05-09T00:43:43.880781+0000","last_became_peered":"2024-05-09T00:43:43.880781+0000","last_unstale":"2024-05-09T00:45:51.878949+0000","last_undegraded":"2024-05-09T00:45:51.878949+0000","last_fullsized":"2024-05-09T00:45:51.878949+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:18:31.577354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.870420+0000","last_change":"2024-05-09T00:43:43.876869+0000","last_active":"2024-05-09T00:45:51.870420+0000","last_peered":"2024-05-09T00:45:51.870420+0000","last_clean":"2024-05-09T00:45:51.870420+0000","last_became_active":"2024-05-09T00:43:43.876439+0000","last_became_peered":"2024-05-09T00:43:43.876439+0000","last_unstale":"2024-05-09T00:45:51.870420+0000","last_undegraded":"2024-05-09T00:45:51.870420+0000","last_fullsized":"2024-05-09T00:45:51.870420+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:39:06.776267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.871096+0000","last_change":"2024-05-09T00:43:43.884677+0000","last_active":"2024-05-09T00:45:51.871096+0000","last_peered":"2024-05-09T00:45:51.871096+0000","last_clean":"2024-05-09T00:45:51.871096+0000","last_became_active":"2024-05-09T00:43:43.884238+0000","last_became_peered":"2024-05-09T00:43:43.884238+0000","last_unstale":"2024-05-09T00:45:51.871096+0000","last_undegraded":"2024-05-09T00:45:51.871096+0000","last_fullsized":"2024-05-09T00:45:51.871096+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:18:13.787584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.867942+0000","last_change":"2024-05-09T00:43:43.876638+0000","last_active":"2024-05-09T00:45:51.867942+0000","last_peered":"2024-05-09T00:45:51.867942+0000","last_clean":"2024-05-09T00:45:51.867942+0000","last_became_active":"2024-05-09T00:43:43.876315+0000","last_became_peered":"2024-05-09T00:43:43.876315+0000","last_unstale":"2024-05-09T00:45:51.867942+0000","last_undegraded":"2024-05-09T00:45:51.867942+0000","last_fullsized":"2024-05-09T00:45:51.867942+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:22:41.618613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.867553+0000","last_change":"2024-05-09T00:43:43.869798+0000","last_active":"2024-05-09T00:45:51.867553+0000","last_peered":"2024-05-09T00:45:51.867553+0000","last_clean":"2024-05-09T00:45:51.867553+0000","last_became_active":"2024-05-09T00:43:43.869444+0000","last_became_peered":"2024-05-09T00:43:43.869444+0000","last_unstale":"2024-05-09T00:45:51.867553+0000","last_undegraded":"2024-05-09T00:45:51.867553+0000","last_fullsized":"2024-05-09T00:45:51.867553+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:15:53.267441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.858958+0000","last_change":"2024-05-09T00:43:43.873215+0000","last_active":"2024-05-09T00:45:51.858958+0000","last_peered":"2024-05-09T00:45:51.858958+0000","last_clean":"2024-05-09T00:45:51.858958+0000","last_became_active":"2024-05-09T00:43:43.872763+0000","last_became_peered":"2024-05-09T00:43:43.872763+0000","last_unstale":"2024-05-09T00:45:51.858958+0000","last_undegraded":"2024-05-09T00:45:51.858958+0000","last_fullsized":"2024-05-09T00:45:51.858958+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:08.997387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.870845+0000","last_change":"2024-05-09T00:43:43.876514+0000","last_active":"2024-05-09T00:45:51.870845+0000","last_peered":"2024-05-09T00:45:51.870845+0000","last_clean":"2024-05-09T00:45:51.870845+0000","last_became_active":"2024-05-09T00:43:43.876147+0000","last_became_peered":"2024-05-09T00:43:43.876147+0000","last_unstale":"2024-05-09T00:45:51.870845+0000","last_undegraded":"2024-05-09T00:45:51.870845+0000","last_fullsized":"2024-05-09T00:45:51.870845+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:58:34.745819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.864498+0000","last_change":"2024-05-09T00:43:43.871869+0000","last_active":"2024-05-09T00:45:51.864498+0000","last_peered":"2024-05-09T00:45:51.864498+0000","last_clean":"2024-05-09T00:45:51.864498+0000","last_became_active":"2024-05-09T00:43:43.871447+0000","last_became_peered":"2024-05-09T00:43:43.871447+0000","last_unstale":"2024-05-09T00:45:51.864498+0000","last_undegraded":"2024-05-09T00:45:51.864498+0000","last_fullsized":"2024-05-09T00:45:51.864498+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:11:42.128931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.865595+0000","last_change":"2024-05-09T00:43:43.871291+0000","last_active":"2024-05-09T00:45:51.865595+0000","last_peered":"2024-05-09T00:45:51.865595+0000","last_clean":"2024-05-09T00:45:51.865595+0000","last_became_active":"2024-05-09T00:43:43.870899+0000","last_became_peered":"2024-05-09T00:43:43.870899+0000","last_unstale":"2024-05-09T00:45:51.865595+0000","last_undegraded":"2024-05-09T00:45:51.865595+0000","last_fullsized":"2024-05-09T00:45:51.865595+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:28:53.013543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.864359+0000","last_change":"2024-05-09T00:43:43.884791+0000","last_active":"2024-05-09T00:45:51.864359+0000","last_peered":"2024-05-09T00:45:51.864359+0000","last_clean":"2024-05-09T00:45:51.864359+0000","last_became_active":"2024-05-09T00:43:43.877333+0000","last_became_peered":"2024-05-09T00:43:43.877333+0000","last_unstale":"2024-05-09T00:45:51.864359+0000","last_undegraded":"2024-05-09T00:45:51.864359+0000","last_fullsized":"2024-05-09T00:45:51.864359+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:37:07.535388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.874993+0000","last_change":"2024-05-09T00:43:43.863773+0000","last_active":"2024-05-09T00:45:51.874993+0000","last_peered":"2024-05-09T00:45:51.874993+0000","last_clean":"2024-05-09T00:45:51.874993+0000","last_became_active":"2024-05-09T00:43:43.863436+0000","last_became_peered":"2024-05-09T00:43:43.863436+0000","last_unstale":"2024-05-09T00:45:51.874993+0000","last_undegraded":"2024-05-09T00:45:51.874993+0000","last_fullsized":"2024-05-09T00:45:51.874993+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:59:57.100234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.880217+0000","last_change":"2024-05-09T00:43:43.873461+0000","last_active":"2024-05-09T00:45:51.880217+0000","last_peered":"2024-05-09T00:45:51.880217+0000","last_clean":"2024-05-09T00:45:51.880217+0000","last_became_active":"2024-05-09T00:43:43.873139+0000","last_became_peered":"2024-05-09T00:43:43.873139+0000","last_unstale":"2024-05-09T00:45:51.880217+0000","last_undegraded":"2024-05-09T00:45:51.880217+0000","last_fullsized":"2024-05-09T00:45:51.880217+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:24:57.865569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.867837+0000","last_change":"2024-05-09T00:43:43.874401+0000","last_active":"2024-05-09T00:45:51.867837+0000","last_peered":"2024-05-09T00:45:51.867837+0000","last_clean":"2024-05-09T00:45:51.867837+0000","last_became_active":"2024-05-09T00:43:43.873765+0000","last_became_peered":"2024-05-09T00:43:43.873765+0000","last_unstale":"2024-05-09T00:45:51.867837+0000","last_undegraded":"2024-05-09T00:45:51.867837+0000","last_fullsized":"2024-05-09T00:45:51.867837+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:23:59.509415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T00:45:51.877398+0000","last_change":"2024-05-09T00:43:43.875219+0000","last_active":"2024-05-09T00:45:51.877398+0000","last_peered":"2024-05-09T00:45:51.877398+0000","last_clean":"2024-05-09T00:45:51.877398+0000","last_became_active":"2024-05-09T00:43:43.874824+0000","last_became_peered":"2024-05-09T00:43:43.874824+0000","last_unstale":"2024-05-09T00:45:51.877398+0000","last_undegraded":"2024-05-09T00:45:51.877398+0000","last_fullsized":"2024-05-09T00:45:51.877398+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:43:42.849434+0000","last_clean_scrub_stamp":"2024-05-09T00:43:42.849434+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:39:41.929521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":262144,"data_stored":2951840,"data_compressed":179218,"data_compressed_allocated":225280,"data_compressed_original":2932736,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71218,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1894,"num_read_kb":1016,"num_write":266,"num_write_kb":154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":53265,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":540672,"data_stored":213654,"data_compressed":8493,"data_compressed_allocated":12288,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},"log_size":228,"ondisk_log_size":228,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710777,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":901120,"data_stored":443946,"data_compressed":5799,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877907007,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":1097593,"data_compressed":41967,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070532,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":442762,"data_compressed":5799,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266761,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":913408,"data_stored":442558,"data_compressed":5799,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462991,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":1036639,"data_compressed":41967,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626516,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":905216,"data_stored":442479,"data_compressed":5799,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574957,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":970752,"data_stored":506284,"data_compressed":5799,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738482,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":925696,"data_stored":444350,"data_compressed":5799,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411431,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":917504,"data_stored":444676,"data_compressed":5799,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182499,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36236,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708916,"statfs":{"total":95995035648,"available":95957929984,"internally_reserved":0,"allocated":921600,"data_stored":442304,"data_compressed":5799,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986269,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":506072,"data_compressed":14292,"data_compressed_allocated":20480,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822745,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":1035477,"data_compressed":41967,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34546,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36168,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36168,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36168,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36168,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":64644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":64432,"data_compressed":8493,"data_compressed_allocated":12288,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":65468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T00:47:12.735 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T00:47:13.023 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:47:13.403 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:47:13 smithi005 ceph-mon[30365]: pgmap v397: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 511 B/s wr, 9 op/s 2024-05-09T00:47:13.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:47:13 smithi005 ceph-mon[30365]: from='client.15240 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T00:47:13.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:47:13 smithi005 ceph-mon[30365]: from='client.15246 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T00:47:13.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:47:13 smithi131 ceph-mon[34547]: pgmap v397: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 511 B/s wr, 9 op/s 2024-05-09T00:47:13.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:47:13 smithi131 ceph-mon[34547]: from='client.15240 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T00:47:13.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:47:13 smithi131 ceph-mon[34547]: from='client.15246 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T00:47:13.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:47:13 smithi157 ceph-mon[34294]: pgmap v397: 97 pgs: 97 active+clean; 646 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 511 B/s wr, 9 op/s 2024-05-09T00:47:13.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:47:13 smithi157 ceph-mon[34294]: from='client.15240 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T00:47:13.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:47:13 smithi157 ceph-mon[34294]: from='client.15246 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T00:47:14.505 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:47:14 smithi005 ceph-mon[30365]: from='client.15252 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T00:47:14.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:47:14 smithi131 ceph-mon[34547]: from='client.15252 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T00:47:14.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:47:14 smithi157 ceph-mon[34294]: from='client.15252 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T00:47:14.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:14.763+0000 7fea83a5b700 1 -- 172.21.15.5:0/928793596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea7c108740 msgr2=0x7fea7c1100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:14.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:14.763+0000 7fea83a5b700 1 --2- 172.21.15.5:0/928793596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea7c108740 0x7fea7c1100a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fea7400b0d0 tx=0x7fea7400b3e0 comp rx=0 tx=0).stop 2024-05-09T00:47:14.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:14.763+0000 7fea83a5b700 1 -- 172.21.15.5:0/928793596 shutdown_connections 2024-05-09T00:47:14.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:14.763+0000 7fea83a5b700 1 --2- 172.21.15.5:0/928793596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea7c108740 0x7fea7c1100a0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:14.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:14.763+0000 7fea83a5b700 1 --2- 172.21.15.5:0/928793596 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea7c1034c0 0x7fea7c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:14.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:14.763+0000 7fea83a5b700 1 --2- 172.21.15.5:0/928793596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea7c107df0 0x7fea7c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:14.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:14.763+0000 7fea83a5b700 1 -- 172.21.15.5:0/928793596 >> 172.21.15.5:0/928793596 conn(0x7fea7c0fedf0 msgr2=0x7fea7c101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:14.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:14.764+0000 7fea83a5b700 1 -- 172.21.15.5:0/928793596 shutdown_connections 2024-05-09T00:47:14.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:14.764+0000 7fea83a5b700 1 -- 172.21.15.5:0/928793596 wait complete. 2024-05-09T00:47:14.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:14.764+0000 7fea83a5b700 1 Processor -- start 2024-05-09T00:47:14.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:14.764+0000 7fea83a5b700 1 -- start start 2024-05-09T00:47:14.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:14.765+0000 7fea83a5b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea7c1034c0 0x7fea7c19adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:14.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:14.765+0000 7fea83a5b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea7c107df0 0x7fea7c19b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:14.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:14.765+0000 7fea83a5b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea7c108740 0x7fea7c19f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:14.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:14.765+0000 7fea83a5b700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fea7c113370 con 0x7fea7c107df0 2024-05-09T00:47:14.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:14.765+0000 7fea83a5b700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fea7c1131f0 con 0x7fea7c1034c0 2024-05-09T00:47:14.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:14.765+0000 7fea83a5b700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fea7c1134f0 con 0x7fea7c108740 2024-05-09T00:47:14.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:14.765+0000 7fea80ff6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea7c107df0 0x7fea7c19b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:14.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:14.765+0000 7fea80ff6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea7c107df0 0x7fea7c19b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39030/0 (socket says 172.21.15.5:39030) 2024-05-09T00:47:14.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:14.765+0000 7fea80ff6700 1 -- 172.21.15.5:0/3918366512 learned_addr learned my addr 172.21.15.5:0/3918366512 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:47:14.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:14.765+0000 7fea817f7700 1 --2- 172.21.15.5:0/3918366512 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea7c1034c0 0x7fea7c19adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:14.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:14.765+0000 7fea81ff8700 1 --2- 172.21.15.5:0/3918366512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea7c108740 0x7fea7c19f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:14.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:14.765+0000 7fea80ff6700 1 -- 172.21.15.5:0/3918366512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea7c108740 msgr2=0x7fea7c19f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:14.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:14.765+0000 7fea80ff6700 1 --2- 172.21.15.5:0/3918366512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea7c108740 0x7fea7c19f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:14.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:14.765+0000 7fea80ff6700 1 -- 172.21.15.5:0/3918366512 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea7c1034c0 msgr2=0x7fea7c19adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:14.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:14.765+0000 7fea80ff6700 1 --2- 172.21.15.5:0/3918366512 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea7c1034c0 0x7fea7c19adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:14.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:14.765+0000 7fea80ff6700 1 -- 172.21.15.5:0/3918366512 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea7c19fbd0 con 0x7fea7c107df0 2024-05-09T00:47:14.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:14.765+0000 7fea80ff6700 1 --2- 172.21.15.5:0/3918366512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea7c107df0 0x7fea7c19b350 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fea6c00c9e0 tx=0x7fea6c00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:14.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:14.766+0000 7fea727fc700 1 -- 172.21.15.5:0/3918366512 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6c007aa0 con 0x7fea7c107df0 2024-05-09T00:47:14.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:14.766+0000 7fea727fc700 1 -- 172.21.15.5:0/3918366512 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea6c007c40 con 0x7fea7c107df0 2024-05-09T00:47:14.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:14.766+0000 7fea727fc700 1 -- 172.21.15.5:0/3918366512 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6c00fe60 con 0x7fea7c107df0 2024-05-09T00:47:14.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:14.766+0000 7fea83a5b700 1 -- 172.21.15.5:0/3918366512 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea7c19fec0 con 0x7fea7c107df0 2024-05-09T00:47:14.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:14.766+0000 7fea83a5b700 1 -- 172.21.15.5:0/3918366512 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea7c1a03b0 con 0x7fea7c107df0 2024-05-09T00:47:14.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:14.767+0000 7fea83a5b700 1 -- 172.21.15.5:0/3918366512 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea7c0512c0 con 0x7fea7c107df0 2024-05-09T00:47:14.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:14.768+0000 7fea727fc700 1 -- 172.21.15.5:0/3918366512 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fea6c00f450 con 0x7fea7c107df0 2024-05-09T00:47:14.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:14.769+0000 7fea727fc700 1 --2- 172.21.15.5:0/3918366512 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fea680b2880 0x7fea680b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:14.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:14.769+0000 7fea817f7700 1 --2- 172.21.15.5:0/3918366512 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fea680b2880 0x7fea680b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:14.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:14.769+0000 7fea727fc700 1 -- 172.21.15.5:0/3918366512 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fea6c0e2020 con 0x7fea7c107df0 2024-05-09T00:47:14.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:14.770+0000 7fea817f7700 1 --2- 172.21.15.5:0/3918366512 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fea680b2880 0x7fea680b4d90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fea78007be0 tx=0x7fea780073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:14.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:14.774+0000 7fea727fc700 1 -- 172.21.15.5:0/3918366512 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fea6c014030 con 0x7fea7c107df0 2024-05-09T00:47:15.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:15.035+0000 7fea83a5b700 1 -- 172.21.15.5:0/3918366512 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fea7c19c110 con 0x7fea7c107df0 2024-05-09T00:47:15.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:15.036+0000 7fea727fc700 1 -- 172.21.15.5:0/3918366512 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11926 (secure 0 0 0) 0x7fea6c0a9db0 con 0x7fea7c107df0 2024-05-09T00:47:15.037 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:47:15.037 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2685771887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2685771887},{"type":"v1","addr":"172.21.15.5:6835","nonce":2685771887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/254017535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":254017535},{"type":"v1","addr":"172.21.15.5:6837","nonce":254017535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T00:43:45.856066+0000","modified":"2024-05-09T00:47:06.980541+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24508,"mds_1":24499,"mds_2":24494,"mds_3":24493,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14598":{"gid":14598,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2109255228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2109255228},{"type":"v1","addr":"172.21.15.5:6839","nonce":2109255228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/3809966329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3809966329},{"type":"v1","addr":"172.21.15.5:6841","nonce":3809966329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6833/3272897742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3272897742},{"type":"v1","addr":"172.21.15.131:6833","nonce":3272897742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1283093898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1283093898},{"type":"v1","addr":"172.21.15.131:6835","nonce":1283093898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/2809525455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2809525455},{"type":"v1","addr":"172.21.15.157:6833","nonce":2809525455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.131:6837/1633006186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":1633006186},{"type":"v1","addr":"172.21.15.131:6837","nonce":1633006186}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.157:6835/832898273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":832898273},{"type":"v1","addr":"172.21.15.157:6835","nonce":832898273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.131:6839/2748669631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2748669631},{"type":"v1","addr":"172.21.15.131:6839","nonce":2748669631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.157:6837/744863009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":744863009},{"type":"v1","addr":"172.21.15.157:6837","nonce":744863009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.157:6839/2779860399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":2779860399},{"type":"v1","addr":"172.21.15.157:6839","nonce":2779860399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24494,24508,24499,24467,24493]},"id":1}]} 2024-05-09T00:47:15.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:15.038+0000 7fea83a5b700 1 -- 172.21.15.5:0/3918366512 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fea680b2880 msgr2=0x7fea680b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:15.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:15.038+0000 7fea83a5b700 1 --2- 172.21.15.5:0/3918366512 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fea680b2880 0x7fea680b4d90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fea78007be0 tx=0x7fea780073d0 comp rx=0 tx=0).stop 2024-05-09T00:47:15.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:15.038+0000 7fea83a5b700 1 -- 172.21.15.5:0/3918366512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea7c107df0 msgr2=0x7fea7c19b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:15.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:15.038+0000 7fea83a5b700 1 --2- 172.21.15.5:0/3918366512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea7c107df0 0x7fea7c19b350 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fea6c00c9e0 tx=0x7fea6c00ccf0 comp rx=0 tx=0).stop 2024-05-09T00:47:15.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:15.039+0000 7fea83a5b700 1 -- 172.21.15.5:0/3918366512 shutdown_connections 2024-05-09T00:47:15.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:15.039+0000 7fea83a5b700 1 --2- 172.21.15.5:0/3918366512 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fea680b2880 0x7fea680b4d90 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:15.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:15.039+0000 7fea83a5b700 1 --2- 172.21.15.5:0/3918366512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea7c108740 0x7fea7c19f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:15.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:15.039+0000 7fea83a5b700 1 --2- 172.21.15.5:0/3918366512 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea7c1034c0 0x7fea7c19adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:15.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:15.039+0000 7fea83a5b700 1 --2- 172.21.15.5:0/3918366512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea7c107df0 0x7fea7c19b350 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:15.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:15.039+0000 7fea83a5b700 1 -- 172.21.15.5:0/3918366512 >> 172.21.15.5:0/3918366512 conn(0x7fea7c0fedf0 msgr2=0x7fea7c075cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:15.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:15.039+0000 7fea83a5b700 1 -- 172.21.15.5:0/3918366512 shutdown_connections 2024-05-09T00:47:15.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:15.039+0000 7fea83a5b700 1 -- 172.21.15.5:0/3918366512 wait complete. 2024-05-09T00:47:15.040 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 29 2024-05-09T00:47:15.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:47:15 smithi005 ceph-mon[30365]: pgmap v398: 97 pgs: 97 active+clean; 651 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 938 B/s wr, 13 op/s 2024-05-09T00:47:15.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:47:15 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/3918366512' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:47:15.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:47:15 smithi131 ceph-mon[34547]: pgmap v398: 97 pgs: 97 active+clean; 651 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 938 B/s wr, 13 op/s 2024-05-09T00:47:15.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:47:15 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/3918366512' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:47:15.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:47:15 smithi157 ceph-mon[34294]: pgmap v398: 97 pgs: 97 active+clean; 651 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 938 B/s wr, 13 op/s 2024-05-09T00:47:15.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:47:15 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/3918366512' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:47:15.708 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T00:47:15.964 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:47:17.366 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:47:17 smithi005 ceph-mon[30365]: pgmap v399: 97 pgs: 97 active+clean; 651 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 9 op/s 2024-05-09T00:47:17.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:47:17 smithi131 ceph-mon[34547]: pgmap v399: 97 pgs: 97 active+clean; 651 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 9 op/s 2024-05-09T00:47:17.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:17.655+0000 7f104b5f4700 1 -- 172.21.15.5:0/34870420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10441045f0 msgr2=0x7f1044104a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:17.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:17.655+0000 7f104b5f4700 1 --2- 172.21.15.5:0/34870420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10441045f0 0x7f1044104a00 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f10400099c0 tx=0x7f1040009cd0 comp rx=0 tx=0).stop 2024-05-09T00:47:17.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:17.655+0000 7f104b5f4700 1 -- 172.21.15.5:0/34870420 shutdown_connections 2024-05-09T00:47:17.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:17.655+0000 7f104b5f4700 1 --2- 172.21.15.5:0/34870420 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1044109870 0x7f10441111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:17.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:17.655+0000 7f104b5f4700 1 --2- 172.21.15.5:0/34870420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10441045f0 0x7f1044104a00 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:17.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:17.655+0000 7f104b5f4700 1 --2- 172.21.15.5:0/34870420 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1044108f20 0x7f1044109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:17.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:17.655+0000 7f104b5f4700 1 -- 172.21.15.5:0/34870420 >> 172.21.15.5:0/34870420 conn(0x7f1044100040 msgr2=0x7f1044102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:17.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:17.655+0000 7f104b5f4700 1 -- 172.21.15.5:0/34870420 shutdown_connections 2024-05-09T00:47:17.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:17.655+0000 7f104b5f4700 1 -- 172.21.15.5:0/34870420 wait complete. 2024-05-09T00:47:17.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:17.656+0000 7f104b5f4700 1 Processor -- start 2024-05-09T00:47:17.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:17.656+0000 7f104b5f4700 1 -- start start 2024-05-09T00:47:17.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:17.656+0000 7f104b5f4700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10441045f0 0x7f104419ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:17.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:17.656+0000 7f104b5f4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1044108f20 0x7f104419b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:17.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:17.656+0000 7f104b5f4700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1044109870 0x7f104419f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:17.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:17.656+0000 7f104b5f4700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f10441144a0 con 0x7f1044108f20 2024-05-09T00:47:17.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:17.656+0000 7f104b5f4700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1044114320 con 0x7f1044109870 2024-05-09T00:47:17.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:17.656+0000 7f104b5f4700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1044114620 con 0x7f10441045f0 2024-05-09T00:47:17.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:17.656+0000 7f1048b8f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1044108f20 0x7f104419b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:17.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:17.656+0000 7f1048b8f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1044108f20 0x7f104419b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39050/0 (socket says 172.21.15.5:39050) 2024-05-09T00:47:17.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:17.656+0000 7f1048b8f700 1 -- 172.21.15.5:0/297916620 learned_addr learned my addr 172.21.15.5:0/297916620 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:47:17.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:17.656+0000 7f1049390700 1 --2- 172.21.15.5:0/297916620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10441045f0 0x7f104419ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:17.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:17.656+0000 7f1049b91700 1 --2- 172.21.15.5:0/297916620 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1044109870 0x7f104419f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:17.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:17.656+0000 7f1048b8f700 1 -- 172.21.15.5:0/297916620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10441045f0 msgr2=0x7f104419ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:17.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:17.656+0000 7f1048b8f700 1 --2- 172.21.15.5:0/297916620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10441045f0 0x7f104419ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:17.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:17.656+0000 7f1048b8f700 1 -- 172.21.15.5:0/297916620 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1044109870 msgr2=0x7f104419f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:17.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:17.656+0000 7f1048b8f700 1 --2- 172.21.15.5:0/297916620 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1044109870 0x7f104419f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:17.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:17.656+0000 7f1048b8f700 1 -- 172.21.15.5:0/297916620 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f104419fd40 con 0x7f1044108f20 2024-05-09T00:47:17.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:17.657+0000 7f1048b8f700 1 --2- 172.21.15.5:0/297916620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1044108f20 0x7f104419b410 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f103400d980 tx=0x7f103400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:17.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:17.657+0000 7f103a7fc700 1 -- 172.21.15.5:0/297916620 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1034017070 con 0x7f1044108f20 2024-05-09T00:47:17.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:17.657+0000 7f103a7fc700 1 -- 172.21.15.5:0/297916620 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f103400fd40 con 0x7f1044108f20 2024-05-09T00:47:17.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:17.657+0000 7f104b5f4700 1 -- 172.21.15.5:0/297916620 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10441a0030 con 0x7f1044108f20 2024-05-09T00:47:17.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:17.657+0000 7f103a7fc700 1 -- 172.21.15.5:0/297916620 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f103400e3b0 con 0x7f1044108f20 2024-05-09T00:47:17.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:17.657+0000 7f104b5f4700 1 -- 172.21.15.5:0/297916620 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10441a7950 con 0x7f1044108f20 2024-05-09T00:47:17.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:17.658+0000 7f104b5f4700 1 -- 172.21.15.5:0/297916620 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10440512c0 con 0x7f1044108f20 2024-05-09T00:47:17.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:17.660+0000 7f103a7fc700 1 -- 172.21.15.5:0/297916620 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f1034010610 con 0x7f1044108f20 2024-05-09T00:47:17.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:17.661+0000 7f103a7fc700 1 --2- 172.21.15.5:0/297916620 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f10300b2900 0x7f10300b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:17.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:17.661+0000 7f1049390700 1 --2- 172.21.15.5:0/297916620 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f10300b2900 0x7f10300b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:17.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:17.661+0000 7f103a7fc700 1 -- 172.21.15.5:0/297916620 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f10340e1c10 con 0x7f1044108f20 2024-05-09T00:47:17.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:17.661+0000 7f1049390700 1 --2- 172.21.15.5:0/297916620 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f10300b2900 0x7f10300b4e10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1040009990 tx=0x7f10400036e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:17.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:17.664+0000 7f103a7fc700 1 -- 172.21.15.5:0/297916620 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f10340a9980 con 0x7f1044108f20 2024-05-09T00:47:17.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:47:17 smithi157 ceph-mon[34294]: pgmap v399: 97 pgs: 97 active+clean; 651 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 9 op/s 2024-05-09T00:47:17.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:17.886+0000 7f104b5f4700 1 -- 172.21.15.5:0/297916620 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f104402e5f0 con 0x7f1044108f20 2024-05-09T00:47:17.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:17.887+0000 7f103a7fc700 1 -- 172.21.15.5:0/297916620 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22745 (secure 0 0 0) 0x7f10340a9280 con 0x7f1044108f20 2024-05-09T00:47:17.889 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:47:17.889 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":82,"fsid":"d85814c6-0d9b-11ef-bc97-c7b262605968","created":"2024-05-09T00:35:25.623377+0000","modified":"2024-05-09T00:45:23.926493+0000","last_up_change":"2024-05-09T00:42:38.525335+0000","last_in_change":"2024-05-09T00:42:17.990219+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T00:38:47.844734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T00:43:39.452106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T00:43:42.708106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f8d484bb-2477-4377-ae68-5bce571333bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2963385132},{"type":"v1","addr":"172.21.15.5:6803","nonce":2963385132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2963385132},{"type":"v1","addr":"172.21.15.5:6805","nonce":2963385132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2963385132},{"type":"v1","addr":"172.21.15.5:6809","nonce":2963385132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2963385132},{"type":"v1","addr":"172.21.15.5:6807","nonce":2963385132}]},"public_addr":"172.21.15.5:6803/2963385132","cluster_addr":"172.21.15.5:6805/2963385132","heartbeat_back_addr":"172.21.15.5:6809/2963385132","heartbeat_front_addr":"172.21.15.5:6807/2963385132","state":["exists","up"]},{"osd":1,"uuid":"1c123676-7144-4a67-93a6-fd49e36a5c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3837683163},{"type":"v1","addr":"172.21.15.131:6801","nonce":3837683163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3837683163},{"type":"v1","addr":"172.21.15.131:6803","nonce":3837683163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3837683163},{"type":"v1","addr":"172.21.15.131:6807","nonce":3837683163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3837683163},{"type":"v1","addr":"172.21.15.131:6805","nonce":3837683163}]},"public_addr":"172.21.15.131:6801/3837683163","cluster_addr":"172.21.15.131:6803/3837683163","heartbeat_back_addr":"172.21.15.131:6807/3837683163","heartbeat_front_addr":"172.21.15.131:6805/3837683163","state":["exists","up"]},{"osd":2,"uuid":"95ca51f0-b48b-485c-a5f7-b15c01e754b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":484818139},{"type":"v1","addr":"172.21.15.157:6801","nonce":484818139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":484818139},{"type":"v1","addr":"172.21.15.157:6803","nonce":484818139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":484818139},{"type":"v1","addr":"172.21.15.157:6807","nonce":484818139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":484818139},{"type":"v1","addr":"172.21.15.157:6805","nonce":484818139}]},"public_addr":"172.21.15.157:6801/484818139","cluster_addr":"172.21.15.157:6803/484818139","heartbeat_back_addr":"172.21.15.157:6807/484818139","heartbeat_front_addr":"172.21.15.157:6805/484818139","state":["exists","up"]},{"osd":3,"uuid":"ed7577fb-3676-4a08-b40d-5474eb88974f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":2357377277},{"type":"v1","addr":"172.21.15.5:6811","nonce":2357377277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":2357377277},{"type":"v1","addr":"172.21.15.5:6813","nonce":2357377277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":2357377277},{"type":"v1","addr":"172.21.15.5:6817","nonce":2357377277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":2357377277},{"type":"v1","addr":"172.21.15.5:6815","nonce":2357377277}]},"public_addr":"172.21.15.5:6811/2357377277","cluster_addr":"172.21.15.5:6813/2357377277","heartbeat_back_addr":"172.21.15.5:6817/2357377277","heartbeat_front_addr":"172.21.15.5:6815/2357377277","state":["exists","up"]},{"osd":4,"uuid":"fb5dcf79-a6f1-4b1a-9e3a-087fb5132472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":2915418378},{"type":"v1","addr":"172.21.15.131:6809","nonce":2915418378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":2915418378},{"type":"v1","addr":"172.21.15.131:6811","nonce":2915418378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":2915418378},{"type":"v1","addr":"172.21.15.131:6815","nonce":2915418378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":2915418378},{"type":"v1","addr":"172.21.15.131:6813","nonce":2915418378}]},"public_addr":"172.21.15.131:6809/2915418378","cluster_addr":"172.21.15.131:6811/2915418378","heartbeat_back_addr":"172.21.15.131:6815/2915418378","heartbeat_front_addr":"172.21.15.131:6813/2915418378","state":["exists","up"]},{"osd":5,"uuid":"f01ed5d3-8066-4d25-a113-280c83e75ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4188818663},{"type":"v1","addr":"172.21.15.157:6809","nonce":4188818663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":4188818663},{"type":"v1","addr":"172.21.15.157:6811","nonce":4188818663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":4188818663},{"type":"v1","addr":"172.21.15.157:6815","nonce":4188818663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":4188818663},{"type":"v1","addr":"172.21.15.157:6813","nonce":4188818663}]},"public_addr":"172.21.15.157:6809/4188818663","cluster_addr":"172.21.15.157:6811/4188818663","heartbeat_back_addr":"172.21.15.157:6815/4188818663","heartbeat_front_addr":"172.21.15.157:6813/4188818663","state":["exists","up"]},{"osd":6,"uuid":"932379a8-c97b-4acc-8665-81f7d9cbce65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2930397813},{"type":"v1","addr":"172.21.15.5:6819","nonce":2930397813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2930397813},{"type":"v1","addr":"172.21.15.5:6821","nonce":2930397813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2930397813},{"type":"v1","addr":"172.21.15.5:6825","nonce":2930397813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2930397813},{"type":"v1","addr":"172.21.15.5:6823","nonce":2930397813}]},"public_addr":"172.21.15.5:6819/2930397813","cluster_addr":"172.21.15.5:6821/2930397813","heartbeat_back_addr":"172.21.15.5:6825/2930397813","heartbeat_front_addr":"172.21.15.5:6823/2930397813","state":["exists","up"]},{"osd":7,"uuid":"3c0a0573-6465-4bf6-b70f-310d49857a6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":4125045219},{"type":"v1","addr":"172.21.15.131:6817","nonce":4125045219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":4125045219},{"type":"v1","addr":"172.21.15.131:6819","nonce":4125045219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":4125045219},{"type":"v1","addr":"172.21.15.131:6823","nonce":4125045219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":4125045219},{"type":"v1","addr":"172.21.15.131:6821","nonce":4125045219}]},"public_addr":"172.21.15.131:6817/4125045219","cluster_addr":"172.21.15.131:6819/4125045219","heartbeat_back_addr":"172.21.15.131:6823/4125045219","heartbeat_front_addr":"172.21.15.131:6821/4125045219","state":["exists","up"]},{"osd":8,"uuid":"c49f0a00-f5ae-40ed-8f38-7b953ba1c17c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2001626768},{"type":"v1","addr":"172.21.15.157:6817","nonce":2001626768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2001626768},{"type":"v1","addr":"172.21.15.157:6819","nonce":2001626768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2001626768},{"type":"v1","addr":"172.21.15.157:6823","nonce":2001626768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2001626768},{"type":"v1","addr":"172.21.15.157:6821","nonce":2001626768}]},"public_addr":"172.21.15.157:6817/2001626768","cluster_addr":"172.21.15.157:6819/2001626768","heartbeat_back_addr":"172.21.15.157:6823/2001626768","heartbeat_front_addr":"172.21.15.157:6821/2001626768","state":["exists","up"]},{"osd":9,"uuid":"1927dfe7-c1af-4a15-bb71-d58d97c8734a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":4287220015},{"type":"v1","addr":"172.21.15.5:6827","nonce":4287220015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":4287220015},{"type":"v1","addr":"172.21.15.5:6829","nonce":4287220015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":4287220015},{"type":"v1","addr":"172.21.15.5:6833","nonce":4287220015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":4287220015},{"type":"v1","addr":"172.21.15.5:6831","nonce":4287220015}]},"public_addr":"172.21.15.5:6827/4287220015","cluster_addr":"172.21.15.5:6829/4287220015","heartbeat_back_addr":"172.21.15.5:6833/4287220015","heartbeat_front_addr":"172.21.15.5:6831/4287220015","state":["exists","up"]},{"osd":10,"uuid":"fbb4f3e3-eb7d-4ecb-a30b-55b8910375ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1571783985},{"type":"v1","addr":"172.21.15.131:6825","nonce":1571783985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1571783985},{"type":"v1","addr":"172.21.15.131:6827","nonce":1571783985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1571783985},{"type":"v1","addr":"172.21.15.131:6831","nonce":1571783985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1571783985},{"type":"v1","addr":"172.21.15.131:6829","nonce":1571783985}]},"public_addr":"172.21.15.131:6825/1571783985","cluster_addr":"172.21.15.131:6827/1571783985","heartbeat_back_addr":"172.21.15.131:6831/1571783985","heartbeat_front_addr":"172.21.15.131:6829/1571783985","state":["exists","up"]},{"osd":11,"uuid":"b858453f-eda1-4e25-89c8-94b54c830938","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3261151188},{"type":"v1","addr":"172.21.15.157:6825","nonce":3261151188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":3261151188},{"type":"v1","addr":"172.21.15.157:6827","nonce":3261151188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":3261151188},{"type":"v1","addr":"172.21.15.157:6831","nonce":3261151188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":3261151188},{"type":"v1","addr":"172.21.15.157:6829","nonce":3261151188}]},"public_addr":"172.21.15.157:6825/3261151188","cluster_addr":"172.21.15.157:6827/3261151188","heartbeat_back_addr":"172.21.15.157:6831/3261151188","heartbeat_front_addr":"172.21.15.157:6829/3261151188","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:37:57.528483+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:38:21.477482+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:38:45.780266+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:39:10.938791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:39:35.734857+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:40:00.169040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:40:26.494635+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:40:51.881718+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:41:17.697112+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:41:44.826479+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:42:10.504768+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:42:36.605447+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6},{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":6}]},{"pgid":"3.19","mappings":[{"from":3,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":7}]},{"pgid":"3.35","mappings":[{"from":1,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/3405015786":"2024-05-10T00:36:21.742363+0000","172.21.15.5:0/2669115557":"2024-05-10T00:36:21.742363+0000","172.21.15.5:6801/287842266":"2024-05-10T00:36:21.742363+0000","172.21.15.5:6800/752513054":"2024-05-10T00:35:53.128955+0000","172.21.15.5:6800/287842266":"2024-05-10T00:36:21.742363+0000","172.21.15.5:6801/752513054":"2024-05-10T00:35:53.128955+0000","172.21.15.5:0/1954876662":"2024-05-10T00:35:53.128955+0000","172.21.15.5:0/2545175121":"2024-05-10T00:36:21.742363+0000","172.21.15.5:0/3754653663":"2024-05-10T00:35:53.128955+0000","172.21.15.5:0/651514359":"2024-05-10T00:35:53.128955+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T00:47:17.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:17.890+0000 7f104b5f4700 1 -- 172.21.15.5:0/297916620 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f10300b2900 msgr2=0x7f10300b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:17.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:17.890+0000 7f104b5f4700 1 --2- 172.21.15.5:0/297916620 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f10300b2900 0x7f10300b4e10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1040009990 tx=0x7f10400036e0 comp rx=0 tx=0).stop 2024-05-09T00:47:17.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:17.890+0000 7f104b5f4700 1 -- 172.21.15.5:0/297916620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1044108f20 msgr2=0x7f104419b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:17.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:17.890+0000 7f104b5f4700 1 --2- 172.21.15.5:0/297916620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1044108f20 0x7f104419b410 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f103400d980 tx=0x7f103400dc90 comp rx=0 tx=0).stop 2024-05-09T00:47:17.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:17.890+0000 7f104b5f4700 1 -- 172.21.15.5:0/297916620 shutdown_connections 2024-05-09T00:47:17.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:17.890+0000 7f104b5f4700 1 --2- 172.21.15.5:0/297916620 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f10300b2900 0x7f10300b4e10 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:17.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:17.890+0000 7f104b5f4700 1 --2- 172.21.15.5:0/297916620 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1044109870 0x7f104419f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:17.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:17.890+0000 7f104b5f4700 1 --2- 172.21.15.5:0/297916620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10441045f0 0x7f104419ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:17.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:17.890+0000 7f104b5f4700 1 --2- 172.21.15.5:0/297916620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1044108f20 0x7f104419b410 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:17.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:17.890+0000 7f104b5f4700 1 -- 172.21.15.5:0/297916620 >> 172.21.15.5:0/297916620 conn(0x7f1044100040 msgr2=0x7f1044108690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:17.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:17.890+0000 7f104b5f4700 1 -- 172.21.15.5:0/297916620 shutdown_connections 2024-05-09T00:47:17.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:17.890+0000 7f104b5f4700 1 -- 172.21.15.5:0/297916620 wait complete. 2024-05-09T00:47:18.511 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:47:18 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/297916620' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:47:18.541 DEBUG:tasks.fwd_scrub:Yielding 2024-05-09T00:47:18.542 INFO:teuthology.run_tasks:Running task background_exec... 2024-05-09T00:47:18.552 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi005.front.sepia.ceph.com 2024-05-09T00:47:18.553 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-05-09T00:47:18.553 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-05-09T00:47:18.553 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T00:47:18.553 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T00:47:18.555 INFO:teuthology.run_tasks:Running task workunit... 2024-05-09T00:47:18.605 INFO:tasks.workunit:Pulling workunits from ref c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T00:47:18.605 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-09T00:47:18.605 INFO:tasks.workunit:timeout=3h 2024-05-09T00:47:18.605 INFO:tasks.workunit:cleanup=True 2024-05-09T00:47:18.606 DEBUG:teuthology.orchestra.run.smithi005:> stat -- /home/ubuntu/cephtest/mnt.0 2024-05-09T00:47:18.607 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:47:18 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/297916620' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:47:18.623 INFO:teuthology.orchestra.run.smithi005.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-05-09T00:47:18.624 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-05-09T00:47:18.624 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 72h/114d Inode: 1099511628283 Links: 2 2024-05-09T00:47:18.624 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T00:47:18.624 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-05-09T00:47:18.624 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-05-09 00:44:19.093082475 +0000 2024-05-09T00:47:18.624 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-05-09 00:44:19.093082475 +0000 2024-05-09T00:47:18.624 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-05-09 00:47:06.236554818 +0000 2024-05-09T00:47:18.624 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2024-05-09 00:44:19.093082475 +0000 2024-05-09T00:47:18.624 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-05-09T00:47:18.624 DEBUG:teuthology.orchestra.run.smithi005:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-05-09T00:47:18.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:47:18 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/297916620' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:47:18.699 DEBUG:teuthology.orchestra.run.smithi005:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T00:47:18.756 INFO:tasks.workunit.client.0.smithi005.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-05-09T00:47:18.813 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:47:19.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:47:19 smithi005 ceph-mon[30365]: pgmap v400: 97 pgs: 97 active+clean; 651 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 10 op/s 2024-05-09T00:47:19.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:47:19 smithi131 ceph-mon[34547]: pgmap v400: 97 pgs: 97 active+clean; 651 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 10 op/s 2024-05-09T00:47:19.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:47:19 smithi157 ceph-mon[34294]: pgmap v400: 97 pgs: 97 active+clean; 651 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 10 op/s 2024-05-09T00:47:20.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.460+0000 7f9c4508e700 1 -- 172.21.15.5:0/2716426250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c40102790 msgr2=0x7f9c40102ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:20.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.460+0000 7f9c4508e700 1 --2- 172.21.15.5:0/2716426250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c40102790 0x7f9c40102ba0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f9c34009990 tx=0x7f9c34009ca0 comp rx=0 tx=0).stop 2024-05-09T00:47:20.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.460+0000 7f9c4508e700 1 -- 172.21.15.5:0/2716426250 shutdown_connections 2024-05-09T00:47:20.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.460+0000 7f9c4508e700 1 --2- 172.21.15.5:0/2716426250 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c401030e0 0x7f9c40111170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:20.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.460+0000 7f9c4508e700 1 --2- 172.21.15.5:0/2716426250 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c40109300 0x7f9c40109710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:20.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.460+0000 7f9c4508e700 1 --2- 172.21.15.5:0/2716426250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c40102790 0x7f9c40102ba0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:20.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.460+0000 7f9c4508e700 1 -- 172.21.15.5:0/2716426250 >> 172.21.15.5:0/2716426250 conn(0x7f9c400fe4e0 msgr2=0x7f9c40100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:20.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.460+0000 7f9c4508e700 1 -- 172.21.15.5:0/2716426250 shutdown_connections 2024-05-09T00:47:20.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.460+0000 7f9c4508e700 1 -- 172.21.15.5:0/2716426250 wait complete. 2024-05-09T00:47:20.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.461+0000 7f9c4508e700 1 Processor -- start 2024-05-09T00:47:20.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.461+0000 7f9c4508e700 1 -- start start 2024-05-09T00:47:20.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.461+0000 7f9c4508e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c40102790 0x7f9c4019af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:20.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.461+0000 7f9c4508e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c401030e0 0x7f9c4019b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:20.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.461+0000 7f9c4508e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c40109300 0x7f9c4019f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:20.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.462+0000 7f9c4508e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9c401145a0 con 0x7f9c40102790 2024-05-09T00:47:20.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.462+0000 7f9c4508e700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9c40114420 con 0x7f9c401030e0 2024-05-09T00:47:20.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.462+0000 7f9c4508e700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9c40114720 con 0x7f9c40109300 2024-05-09T00:47:20.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.462+0000 7f9c3ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c40102790 0x7f9c4019af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:20.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.462+0000 7f9c3ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c40102790 0x7f9c4019af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39076/0 (socket says 172.21.15.5:39076) 2024-05-09T00:47:20.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.462+0000 7f9c3ed9d700 1 -- 172.21.15.5:0/2779364949 learned_addr learned my addr 172.21.15.5:0/2779364949 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:47:20.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.462+0000 7f9c3f59e700 1 --2- 172.21.15.5:0/2779364949 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c40109300 0x7f9c4019f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:20.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.462+0000 7f9c3e59c700 1 --2- 172.21.15.5:0/2779364949 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c401030e0 0x7f9c4019b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:20.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.462+0000 7f9c3ed9d700 1 -- 172.21.15.5:0/2779364949 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c40109300 msgr2=0x7f9c4019f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:20.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.462+0000 7f9c3ed9d700 1 --2- 172.21.15.5:0/2779364949 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c40109300 0x7f9c4019f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:20.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.462+0000 7f9c3ed9d700 1 -- 172.21.15.5:0/2779364949 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c401030e0 msgr2=0x7f9c4019b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:20.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.462+0000 7f9c3ed9d700 1 --2- 172.21.15.5:0/2779364949 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c401030e0 0x7f9c4019b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:20.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.462+0000 7f9c3ed9d700 1 -- 172.21.15.5:0/2779364949 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c4019fe20 con 0x7f9c40102790 2024-05-09T00:47:20.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.462+0000 7f9c3ed9d700 1 --2- 172.21.15.5:0/2779364949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c40102790 0x7f9c4019af50 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9c3000c970 tx=0x7f9c3000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:20.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.462+0000 7f9c27fff700 1 -- 172.21.15.5:0/2779364949 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c30004220 con 0x7f9c40102790 2024-05-09T00:47:20.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.462+0000 7f9c4508e700 1 -- 172.21.15.5:0/2779364949 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c401a0110 con 0x7f9c40102790 2024-05-09T00:47:20.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.462+0000 7f9c27fff700 1 -- 172.21.15.5:0/2779364949 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c300043c0 con 0x7f9c40102790 2024-05-09T00:47:20.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.463+0000 7f9c27fff700 1 -- 172.21.15.5:0/2779364949 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c30003be0 con 0x7f9c40102790 2024-05-09T00:47:20.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.463+0000 7f9c4508e700 1 -- 172.21.15.5:0/2779364949 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c401a7a30 con 0x7f9c40102790 2024-05-09T00:47:20.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.466+0000 7f9c27fff700 1 -- 172.21.15.5:0/2779364949 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f9c300076d0 con 0x7f9c40102790 2024-05-09T00:47:20.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.466+0000 7f9c27fff700 1 --2- 172.21.15.5:0/2779364949 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f9c280b2880 0x7f9c280b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:20.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.467+0000 7f9c3e59c700 1 --2- 172.21.15.5:0/2779364949 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f9c280b2880 0x7f9c280b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:20.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.467+0000 7f9c27fff700 1 -- 172.21.15.5:0/2779364949 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f9c300e1f70 con 0x7f9c40102790 2024-05-09T00:47:20.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.467+0000 7f9c53aa4b80 1 Processor -- start 2024-05-09T00:47:20.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.467+0000 7f9c53aa4b80 1 -- start start 2024-05-09T00:47:20.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.467+0000 7f9c3e59c700 1 --2- 172.21.15.5:0/2779364949 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f9c280b2880 0x7f9c280b4d90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9c34006010 tx=0x7f9c34005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:20.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.467+0000 7f9c53aa4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5618c3ab91e0 0x5618c3b52570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:20.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.467+0000 7f9c53aa4b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618c39cf520 0x5618c3b52ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:20.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.467+0000 7f9c53aa4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618c3b4c720 0x5618c3b4cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:20.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.467+0000 7f9c53aa4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5618c3a6f070 con 0x5618c3b4c720 2024-05-09T00:47:20.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.467+0000 7f9c53aa4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5618c3a0d870 con 0x5618c3ab91e0 2024-05-09T00:47:20.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.467+0000 7f9c53aa4b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5618c3a6b030 con 0x5618c39cf520 2024-05-09T00:47:20.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.467+0000 7f9c3f59e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618c3b4c720 0x5618c3b4cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:20.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.467+0000 7f9c3f59e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618c3b4c720 0x5618c3b4cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39092/0 (socket says 172.21.15.5:39092) 2024-05-09T00:47:20.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.467+0000 7f9c3f59e700 1 -- 172.21.15.5:0/1454701731 learned_addr learned my addr 172.21.15.5:0/1454701731 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:47:20.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.468+0000 7f9c3e59c700 1 --2- 172.21.15.5:0/1454701731 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618c39cf520 0x5618c3b52ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:20.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.468+0000 7f9c3ed9d700 1 --2- 172.21.15.5:0/1454701731 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5618c3ab91e0 0x5618c3b52570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:20.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.468+0000 7f9c3f59e700 1 -- 172.21.15.5:0/1454701731 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618c39cf520 msgr2=0x5618c3b52ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:47:20.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.468+0000 7f9c3f59e700 1 --2- 172.21.15.5:0/1454701731 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618c39cf520 0x5618c3b52ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:20.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.468+0000 7f9c3f59e700 1 -- 172.21.15.5:0/1454701731 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5618c3ab91e0 msgr2=0x5618c3b52570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:47:20.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.468+0000 7f9c3f59e700 1 --2- 172.21.15.5:0/1454701731 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5618c3ab91e0 0x5618c3b52570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:20.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.468+0000 7f9c3f59e700 1 -- 172.21.15.5:0/1454701731 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618c3a053f0 con 0x5618c3b4c720 2024-05-09T00:47:20.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.468+0000 7f9c3f59e700 1 --2- 172.21.15.5:0/1454701731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618c3b4c720 0x5618c3b4cb50 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9c380098f0 tx=0x7f9c38009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98191caa50eff058 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:20.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.468+0000 7f9c257fa700 1 -- 172.21.15.5:0/1454701731 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c38004bb0 con 0x5618c3b4c720 2024-05-09T00:47:20.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.468+0000 7f9c257fa700 1 -- 172.21.15.5:0/1454701731 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c38004d50 con 0x5618c3b4c720 2024-05-09T00:47:20.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.468+0000 7f9c257fa700 1 -- 172.21.15.5:0/1454701731 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c38015480 con 0x5618c3b4c720 2024-05-09T00:47:20.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.469+0000 7f9c53aa4b80 1 -- 172.21.15.5:0/1454701731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618c3b4c720 msgr2=0x5618c3b4cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:20.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.469+0000 7f9c53aa4b80 1 --2- 172.21.15.5:0/1454701731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618c3b4c720 0x5618c3b4cb50 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9c380098f0 tx=0x7f9c38009c00 comp rx=0 tx=0).stop 2024-05-09T00:47:20.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.469+0000 7f9c53aa4b80 1 -- 172.21.15.5:0/1454701731 shutdown_connections 2024-05-09T00:47:20.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.469+0000 7f9c53aa4b80 1 --2- 172.21.15.5:0/1454701731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618c3b4c720 0x5618c3b4cb50 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:20.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.469+0000 7f9c53aa4b80 1 --2- 172.21.15.5:0/1454701731 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5618c3ab91e0 0x5618c3b52570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:20.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.469+0000 7f9c53aa4b80 1 --2- 172.21.15.5:0/1454701731 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618c39cf520 0x5618c3b52ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:20.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.469+0000 7f9c53aa4b80 1 -- 172.21.15.5:0/1454701731 >> 172.21.15.5:0/1454701731 conn(0x5618c3a2f720 msgr2=0x5618c3b47480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:20.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.469+0000 7f9c53aa4b80 1 -- 172.21.15.5:0/1454701731 shutdown_connections 2024-05-09T00:47:20.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.469+0000 7f9c53aa4b80 1 -- 172.21.15.5:0/1454701731 wait complete. 2024-05-09T00:47:20.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.469+0000 7f9c53aa4b80 1 -- start start 2024-05-09T00:47:20.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.470+0000 7f9c53aa4b80 1 Processor -- start 2024-05-09T00:47:20.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.470+0000 7f9c53aa4b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618c39cf520 0x5618c3b56230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:20.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.470+0000 7f9c53aa4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5618c3ab91e0 0x5618c3b56770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:20.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.470+0000 7f9c53aa4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618c3b4c720 0x5618c3b56cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:20.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.470+0000 7f9c53aa4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5618c38eac20 con 0x5618c3b4c720 2024-05-09T00:47:20.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.470+0000 7f9c53aa4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5618c3a291d0 con 0x5618c3ab91e0 2024-05-09T00:47:20.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.470+0000 7f9c53aa4b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5618c3b59250 con 0x5618c39cf520 2024-05-09T00:47:20.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.470+0000 7f9c53aa4b80 10 client.0 _ll_register_callbacks cb 0x5618c38e0740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:47:20.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.470+0000 7f9c3f59e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618c3b4c720 0x5618c3b56cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:20.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.470+0000 7f9c3f59e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618c3b4c720 0x5618c3b56cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39104/0 (socket says 172.21.15.5:39104) 2024-05-09T00:47:20.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.470+0000 7f9c3f59e700 1 -- 172.21.15.5:0/3947652644 learned_addr learned my addr 172.21.15.5:0/3947652644 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:47:20.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.470+0000 7f9c3ed9d700 1 --2- 172.21.15.5:0/3947652644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618c39cf520 0x5618c3b56230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:20.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.470+0000 7f9c3e59c700 1 --2- 172.21.15.5:0/3947652644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5618c3ab91e0 0x5618c3b56770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:20.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.470+0000 7f9c3f59e700 1 -- 172.21.15.5:0/3947652644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618c39cf520 msgr2=0x5618c3b56230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:47:20.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.470+0000 7f9c3f59e700 1 --2- 172.21.15.5:0/3947652644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618c39cf520 0x5618c3b56230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:20.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.470+0000 7f9c3f59e700 1 -- 172.21.15.5:0/3947652644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5618c3ab91e0 msgr2=0x5618c3b56770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:47:20.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.470+0000 7f9c3f59e700 1 --2- 172.21.15.5:0/3947652644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5618c3ab91e0 0x5618c3b56770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:20.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.470+0000 7f9c3f59e700 1 -- 172.21.15.5:0/3947652644 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618c38ea510 con 0x5618c3b4c720 2024-05-09T00:47:20.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.471+0000 7f9c3f59e700 1 --2- 172.21.15.5:0/3947652644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618c3b4c720 0x5618c3b56cb0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f9c38003bf0 tx=0x7f9c38007580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdd39ab047562a7d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:20.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.471+0000 7f9c22ff5700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:47:20.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.471+0000 7f9c22ff5700 1 -- 172.21.15.5:0/3947652644 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c38022070 con 0x5618c3b4c720 2024-05-09T00:47:20.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.471+0000 7f9c22ff5700 1 -- 172.21.15.5:0/3947652644 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f9c38004790 con 0x5618c3b4c720 2024-05-09T00:47:20.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.471+0000 7f9c53aa4b80 1 -- 172.21.15.5:0/3947652644 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c38004d50 con 0x5618c3b4c720 2024-05-09T00:47:20.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.471+0000 7f9c22ff5700 1 -- 172.21.15.5:0/3947652644 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9c3802e050 con 0x5618c3b4c720 2024-05-09T00:47:20.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.471+0000 7f9c53aa4b80 10 client.15276 fetch_fsmap learned FSMap version 29 2024-05-09T00:47:20.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.471+0000 7f9c53aa4b80 1 -- 172.21.15.5:0/3947652644 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5618c3b63c10 con 0x5618c3b4c720 2024-05-09T00:47:20.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.471+0000 7f9c22ff5700 1 -- 172.21.15.5:0/3947652644 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c38004d50 con 0x5618c3b4c720 2024-05-09T00:47:20.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.471+0000 7f9c22ff5700 1 -- 172.21.15.5:0/3947652644 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f9c3801e410 con 0x5618c3b4c720 2024-05-09T00:47:20.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.472+0000 7f9c53aa4b80 10 client.15276 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:47:20.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.472+0000 7f9c53aa4b80 10 client.15276 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:47:20.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.472+0000 7f9c53aa4b80 20 client.15276 populate_metadata read hostname 'smithi005' 2024-05-09T00:47:20.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.472+0000 7f9c53aa4b80 1 --2- 172.21.15.5:0/3947652644 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5618c3b660e0 0x5618c3b68500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:20.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.472+0000 7f9c53aa4b80 4 client.15276 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:47:20.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.472+0000 7f9c53aa4b80 1 -- 172.21.15.5:0/3947652644 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5618c3a7ca50 con 0x5618c3b660e0 2024-05-09T00:47:20.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.472+0000 7f9c3ed9d700 1 --2- 172.21.15.5:0/3947652644 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5618c3b660e0 0x5618c3b68500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:20.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.472+0000 7f9c3ed9d700 1 --2- 172.21.15.5:0/3947652644 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5618c3b660e0 0x5618c3b68500 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bb11184dd26dff9 server_cookie=404a7a87b23c0a36 in_seq=0 out_seq=0 2024-05-09T00:47:20.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.472+0000 7f9c22ff5700 10 client.15276 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:47:20.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.473+0000 7f9c22ff5700 1 -- 172.21.15.5:0/3947652644 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5618c3a7ca50 con 0x5618c3b660e0 2024-05-09T00:47:20.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.473+0000 7f9c22ff5700 10 client.15276 handle_command_reply: tid=0 2024-05-09T00:47:20.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.473+0000 7f9c53aa4b80 1 client.15276 shutdown 2024-05-09T00:47:20.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.473+0000 7f9c53aa4b80 1 -- 172.21.15.5:0/3947652644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618c3b4c720 msgr2=0x5618c3b56cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:20.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.473+0000 7f9c53aa4b80 1 --2- 172.21.15.5:0/3947652644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618c3b4c720 0x5618c3b56cb0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f9c38003bf0 tx=0x7f9c38007580 comp rx=0 tx=0).stop 2024-05-09T00:47:20.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.473+0000 7f9c53aa4b80 1 -- 172.21.15.5:0/3947652644 shutdown_connections 2024-05-09T00:47:20.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.473+0000 7f9c53aa4b80 1 --2- 172.21.15.5:0/3947652644 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5618c3b660e0 0x5618c3b68500 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:20.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.473+0000 7f9c53aa4b80 1 --2- 172.21.15.5:0/3947652644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618c3b4c720 0x5618c3b56cb0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:20.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.473+0000 7f9c53aa4b80 1 --2- 172.21.15.5:0/3947652644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618c39cf520 0x5618c3b56230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:20.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.473+0000 7f9c53aa4b80 1 --2- 172.21.15.5:0/3947652644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5618c3ab91e0 0x5618c3b56770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:20.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.473+0000 7f9c22ff5700 0 client.15276 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:47:20.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.473+0000 7f9c53aa4b80 1 -- 172.21.15.5:0/3947652644 >> 172.21.15.5:0/3947652644 conn(0x5618c3a2f720 msgr2=0x5618c3b47240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:20.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.473+0000 7f9c53aa4b80 1 -- 172.21.15.5:0/3947652644 shutdown_connections 2024-05-09T00:47:20.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.473+0000 7f9c53aa4b80 1 -- 172.21.15.5:0/3947652644 wait complete. 2024-05-09T00:47:20.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.474+0000 7f9c53aa4b80 20 client.15276 trim_cache size 0 max 16384 2024-05-09T00:47:20.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.494+0000 7f9c53aa4b80 1 Processor -- start 2024-05-09T00:47:20.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.494+0000 7f9c53aa4b80 1 -- start start 2024-05-09T00:47:20.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.494+0000 7f9c53aa4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5618c39cf520 0x5618c3b58310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:20.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.494+0000 7f9c53aa4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618c3ab91e0 0x5618c3b4ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:20.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.494+0000 7f9c53aa4b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618c3b4c720 0x5618c3b4f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:20.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.494+0000 7f9c53aa4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9c401aedd0 con 0x5618c3ab91e0 2024-05-09T00:47:20.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.494+0000 7f9c53aa4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9c401b0910 con 0x5618c39cf520 2024-05-09T00:47:20.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.494+0000 7f9c53aa4b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9c401af4a0 con 0x5618c3b4c720 2024-05-09T00:47:20.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.495+0000 7f9c3e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618c3ab91e0 0x5618c3b4ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:20.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.495+0000 7f9c3ed9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5618c39cf520 0x5618c3b58310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:20.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.495+0000 7f9c3e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618c3ab91e0 0x5618c3b4ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39114/0 (socket says 172.21.15.5:39114) 2024-05-09T00:47:20.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.495+0000 7f9c3e59c700 1 -- 172.21.15.5:0/591968036 learned_addr learned my addr 172.21.15.5:0/591968036 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:47:20.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.495+0000 7f9c3ed9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5618c39cf520 0x5618c3b58310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:42784/0 (socket says 172.21.15.5:42784) 2024-05-09T00:47:20.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.495+0000 7f9c3f59e700 1 --2- 172.21.15.5:0/591968036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618c3b4c720 0x5618c3b4f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:20.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.495+0000 7f9c3e59c700 1 -- 172.21.15.5:0/591968036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618c3b4c720 msgr2=0x5618c3b4f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:47:20.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.495+0000 7f9c3e59c700 1 --2- 172.21.15.5:0/591968036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618c3b4c720 0x5618c3b4f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:20.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.495+0000 7f9c3e59c700 1 -- 172.21.15.5:0/591968036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5618c39cf520 msgr2=0x5618c3b58310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:47:20.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.495+0000 7f9c3e59c700 1 --2- 172.21.15.5:0/591968036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5618c39cf520 0x5618c3b58310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:20.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.495+0000 7f9c3e59c700 1 -- 172.21.15.5:0/591968036 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618c3b48d80 con 0x5618c3ab91e0 2024-05-09T00:47:20.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.495+0000 7f9c3e59c700 1 --2- 172.21.15.5:0/591968036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618c3ab91e0 0x5618c3b4ee80 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9c4006ccb0 tx=0x7f9c3401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13c50c0a8c1f3077 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:20.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.496+0000 7f9c24ff9700 1 -- 172.21.15.5:0/591968036 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c34003ea0 con 0x5618c3ab91e0 2024-05-09T00:47:20.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.496+0000 7f9c24ff9700 1 -- 172.21.15.5:0/591968036 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c3400be80 con 0x5618c3ab91e0 2024-05-09T00:47:20.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.496+0000 7f9c24ff9700 1 -- 172.21.15.5:0/591968036 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c3400fe60 con 0x5618c3ab91e0 2024-05-09T00:47:20.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.496+0000 7f9c53aa4b80 1 -- 172.21.15.5:0/591968036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618c3ab91e0 msgr2=0x5618c3b4ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:20.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.496+0000 7f9c53aa4b80 1 --2- 172.21.15.5:0/591968036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618c3ab91e0 0x5618c3b4ee80 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9c4006ccb0 tx=0x7f9c3401e000 comp rx=0 tx=0).stop 2024-05-09T00:47:20.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.497+0000 7f9c53aa4b80 1 -- 172.21.15.5:0/591968036 shutdown_connections 2024-05-09T00:47:20.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.497+0000 7f9c53aa4b80 1 --2- 172.21.15.5:0/591968036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618c3b4c720 0x5618c3b4f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:20.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.497+0000 7f9c53aa4b80 1 --2- 172.21.15.5:0/591968036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5618c39cf520 0x5618c3b58310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:20.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.497+0000 7f9c53aa4b80 1 --2- 172.21.15.5:0/591968036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618c3ab91e0 0x5618c3b4ee80 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:20.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.497+0000 7f9c53aa4b80 1 -- 172.21.15.5:0/591968036 >> 172.21.15.5:0/591968036 conn(0x5618c3a2f720 msgr2=0x5618c3b4bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:20.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.497+0000 7f9c53aa4b80 1 -- 172.21.15.5:0/591968036 shutdown_connections 2024-05-09T00:47:20.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.497+0000 7f9c53aa4b80 1 -- 172.21.15.5:0/591968036 wait complete. 2024-05-09T00:47:20.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.497+0000 7f9c53aa4b80 1 -- start start 2024-05-09T00:47:20.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.497+0000 7f9c53aa4b80 1 Processor -- start 2024-05-09T00:47:20.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.498+0000 7f9c53aa4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618c39cf520 0x5618c3b6a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:20.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.498+0000 7f9c53aa4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5618c3ab91e0 0x5618c3b6a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:20.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.498+0000 7f9c53aa4b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618c3b4c720 0x5618c3b6ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:20.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.498+0000 7f9c53aa4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9c401b0240 con 0x5618c39cf520 2024-05-09T00:47:20.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.498+0000 7f9c53aa4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9c401afb70 con 0x5618c3ab91e0 2024-05-09T00:47:20.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.498+0000 7f9c53aa4b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9c401acd70 con 0x5618c3b4c720 2024-05-09T00:47:20.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.498+0000 7f9c53aa4b80 10 client.0 _ll_register_callbacks cb 0x5618c38e0740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:47:20.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.498+0000 7f9c3ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618c39cf520 0x5618c3b6a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:20.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.498+0000 7f9c3ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618c39cf520 0x5618c3b6a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39120/0 (socket says 172.21.15.5:39120) 2024-05-09T00:47:20.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.498+0000 7f9c3ed9d700 1 -- 172.21.15.5:0/2684469343 learned_addr learned my addr 172.21.15.5:0/2684469343 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:47:20.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.498+0000 7f9c3e59c700 1 --2- 172.21.15.5:0/2684469343 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5618c3ab91e0 0x5618c3b6a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:20.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.498+0000 7f9c3ed9d700 1 -- 172.21.15.5:0/2684469343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618c3b4c720 msgr2=0x5618c3b6ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:47:20.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.498+0000 7f9c3ed9d700 1 --2- 172.21.15.5:0/2684469343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618c3b4c720 0x5618c3b6ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:20.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.498+0000 7f9c3ed9d700 1 -- 172.21.15.5:0/2684469343 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5618c3ab91e0 msgr2=0x5618c3b6a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:47:20.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.498+0000 7f9c3ed9d700 1 --2- 172.21.15.5:0/2684469343 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5618c3ab91e0 0x5618c3b6a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:20.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.498+0000 7f9c3ed9d700 1 -- 172.21.15.5:0/2684469343 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618c3bd4a50 con 0x5618c39cf520 2024-05-09T00:47:20.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.498+0000 7f9c3ed9d700 1 --2- 172.21.15.5:0/2684469343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618c39cf520 0x5618c3b6a0a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f9c300a9260 tx=0x7f9c3000a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe0d2c18ad6360f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:20.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.498+0000 7f9c22ff5700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:47:20.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.499+0000 7f9c22ff5700 1 -- 172.21.15.5:0/2684469343 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c3001d040 con 0x5618c39cf520 2024-05-09T00:47:20.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.499+0000 7f9c22ff5700 1 -- 172.21.15.5:0/2684469343 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f9c300910b0 con 0x5618c39cf520 2024-05-09T00:47:20.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.499+0000 7f9c53aa4b80 1 -- 172.21.15.5:0/2684469343 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5618c3a76af0 con 0x5618c39cf520 2024-05-09T00:47:20.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.499+0000 7f9c22ff5700 1 -- 172.21.15.5:0/2684469343 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9c300ad910 con 0x5618c39cf520 2024-05-09T00:47:20.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.499+0000 7f9c53aa4b80 10 client.15282 fetch_fsmap learned FSMap version 29 2024-05-09T00:47:20.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.499+0000 7f9c53aa4b80 1 -- 172.21.15.5:0/2684469343 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5618c3bd5cd0 con 0x5618c39cf520 2024-05-09T00:47:20.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.499+0000 7f9c22ff5700 1 -- 172.21.15.5:0/2684469343 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5618c3a76af0 con 0x5618c39cf520 2024-05-09T00:47:20.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.499+0000 7f9c22ff5700 1 -- 172.21.15.5:0/2684469343 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f9c300a97c0 con 0x5618c39cf520 2024-05-09T00:47:20.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.499+0000 7f9c53aa4b80 10 client.15282 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:47:20.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.499+0000 7f9c53aa4b80 10 client.15282 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:47:20.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.499+0000 7f9c53aa4b80 20 client.15282 populate_metadata read hostname 'smithi005' 2024-05-09T00:47:20.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.499+0000 7f9c53aa4b80 1 --2- 172.21.15.5:0/2684469343 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5618c3bd6360 0x5618c3bd8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:20.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.499+0000 7f9c53aa4b80 4 client.15282 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T00:47:20.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.499+0000 7f9c53aa4b80 1 -- 172.21.15.5:0/2684469343 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5618c38ea9f0 con 0x5618c3bd6360 2024-05-09T00:47:20.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.500+0000 7f9c3e59c700 1 --2- 172.21.15.5:0/2684469343 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5618c3bd6360 0x5618c3bd8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:20.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.500+0000 7f9c22ff5700 10 client.15282 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:47:20.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.500+0000 7f9c3e59c700 1 --2- 172.21.15.5:0/2684469343 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5618c3bd6360 0x5618c3bd8780 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fec8f039f906f63f server_cookie=fffc8e133bb8e4da in_seq=0 out_seq=0 2024-05-09T00:47:20.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.501+0000 7f9c22ff5700 1 -- 172.21.15.5:0/2684469343 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5618c38ea9f0 con 0x5618c3bd6360 2024-05-09T00:47:20.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.501+0000 7f9c22ff5700 10 client.15282 handle_command_reply: tid=0 2024-05-09T00:47:20.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.501+0000 7f9c53aa4b80 1 client.15282 shutdown 2024-05-09T00:47:20.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.502+0000 7f9c53aa4b80 1 -- 172.21.15.5:0/2684469343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618c39cf520 msgr2=0x5618c3b6a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:20.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.502+0000 7f9c53aa4b80 1 --2- 172.21.15.5:0/2684469343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618c39cf520 0x5618c3b6a0a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f9c300a9260 tx=0x7f9c3000a5d0 comp rx=0 tx=0).stop 2024-05-09T00:47:20.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.502+0000 7f9c53aa4b80 1 -- 172.21.15.5:0/2684469343 shutdown_connections 2024-05-09T00:47:20.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.502+0000 7f9c53aa4b80 1 --2- 172.21.15.5:0/2684469343 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5618c3bd6360 0x5618c3bd8780 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:20.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.502+0000 7f9c53aa4b80 1 --2- 172.21.15.5:0/2684469343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618c3b4c720 0x5618c3b6ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:20.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.502+0000 7f9c53aa4b80 1 --2- 172.21.15.5:0/2684469343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618c39cf520 0x5618c3b6a0a0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:20.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.502+0000 7f9c53aa4b80 1 --2- 172.21.15.5:0/2684469343 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5618c3ab91e0 0x5618c3b6a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:20.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.502+0000 7f9c53aa4b80 1 -- 172.21.15.5:0/2684469343 >> 172.21.15.5:0/2684469343 conn(0x5618c3a2f720 msgr2=0x5618c3b4fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:20.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.502+0000 7f9c22ff5700 0 client.15282 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:47:20.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.502+0000 7f9c53aa4b80 1 -- 172.21.15.5:0/2684469343 shutdown_connections 2024-05-09T00:47:20.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.502+0000 7f9c53aa4b80 1 -- 172.21.15.5:0/2684469343 wait complete. 2024-05-09T00:47:20.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.502+0000 7f9c53aa4b80 20 client.15282 trim_cache size 0 max 16384 2024-05-09T00:47:20.505 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-09T00:47:20.505 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-05-09T00:47:20.505 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "827cc3b6-e2c4-486e-8d9d-36bf9577121c", 2024-05-09T00:47:20.505 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-05-09T00:47:20.505 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-09T00:47:20.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.503+0000 7f9c4508e700 1 -- 172.21.15.5:0/2779364949 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f9c280b2880 msgr2=0x7f9c280b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:20.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.503+0000 7f9c4508e700 1 --2- 172.21.15.5:0/2779364949 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f9c280b2880 0x7f9c280b4d90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9c34006010 tx=0x7f9c34005960 comp rx=0 tx=0).stop 2024-05-09T00:47:20.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.503+0000 7f9c4508e700 1 -- 172.21.15.5:0/2779364949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c40102790 msgr2=0x7f9c4019af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:20.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.503+0000 7f9c4508e700 1 --2- 172.21.15.5:0/2779364949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c40102790 0x7f9c4019af50 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9c3000c970 tx=0x7f9c3000cc80 comp rx=0 tx=0).stop 2024-05-09T00:47:20.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.503+0000 7f9c4508e700 1 -- 172.21.15.5:0/2779364949 shutdown_connections 2024-05-09T00:47:20.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.503+0000 7f9c4508e700 1 --2- 172.21.15.5:0/2779364949 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f9c280b2880 0x7f9c280b4d90 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:20.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.503+0000 7f9c4508e700 1 --2- 172.21.15.5:0/2779364949 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c40109300 0x7f9c4019f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:20.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.503+0000 7f9c4508e700 1 --2- 172.21.15.5:0/2779364949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c40102790 0x7f9c4019af50 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:20.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.503+0000 7f9c4508e700 1 --2- 172.21.15.5:0/2779364949 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c401030e0 0x7f9c4019b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:20.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.503+0000 7f9c4508e700 1 -- 172.21.15.5:0/2779364949 >> 172.21.15.5:0/2779364949 conn(0x7f9c400fe4e0 msgr2=0x7f9c40107110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:20.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.503+0000 7f9c4508e700 1 -- 172.21.15.5:0/2779364949 shutdown_connections 2024-05-09T00:47:20.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:20.503+0000 7f9c4508e700 1 -- 172.21.15.5:0/2779364949 wait complete. 2024-05-09T00:47:21.199 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T00:47:21.504 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:47:21.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:47:21 smithi005 ceph-mon[30365]: pgmap v401: 97 pgs: 97 active+clean; 651 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 853 B/s wr, 11 op/s 2024-05-09T00:47:21.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:47:21 smithi005 ceph-mon[30365]: scrub queued for path: / 2024-05-09T00:47:21.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:47:21 smithi005 ceph-mon[30365]: scrub summary: idle+waiting paths [/] 2024-05-09T00:47:21.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:47:21 smithi005 ceph-mon[30365]: scrub summary: active paths [/] 2024-05-09T00:47:21.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:47:21 smithi005 ceph-mon[30365]: scrub summary: idle+waiting paths [/] 2024-05-09T00:47:21.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:47:21 smithi131 ceph-mon[34547]: pgmap v401: 97 pgs: 97 active+clean; 651 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 853 B/s wr, 11 op/s 2024-05-09T00:47:21.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:47:21 smithi131 ceph-mon[34547]: scrub queued for path: / 2024-05-09T00:47:21.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:47:21 smithi131 ceph-mon[34547]: scrub summary: idle+waiting paths [/] 2024-05-09T00:47:21.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:47:21 smithi131 ceph-mon[34547]: scrub summary: active paths [/] 2024-05-09T00:47:21.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:47:21 smithi131 ceph-mon[34547]: scrub summary: idle+waiting paths [/] 2024-05-09T00:47:21.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:47:21 smithi157 ceph-mon[34294]: pgmap v401: 97 pgs: 97 active+clean; 651 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 853 B/s wr, 11 op/s 2024-05-09T00:47:21.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:47:21 smithi157 ceph-mon[34294]: scrub queued for path: / 2024-05-09T00:47:21.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:47:21 smithi157 ceph-mon[34294]: scrub summary: idle+waiting paths [/] 2024-05-09T00:47:21.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:47:21 smithi157 ceph-mon[34294]: scrub summary: active paths [/] 2024-05-09T00:47:21.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:47:21 smithi157 ceph-mon[34294]: scrub summary: idle+waiting paths [/] 2024-05-09T00:47:23.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.230+0000 7fa010642700 1 -- 172.21.15.5:0/4293991590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa008109870 msgr2=0x7fa0081111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:23.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.230+0000 7fa010642700 1 --2- 172.21.15.5:0/4293991590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa008109870 0x7fa0081111d0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fa0000099c0 tx=0x7fa000009cd0 comp rx=0 tx=0).stop 2024-05-09T00:47:23.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.230+0000 7fa010642700 1 -- 172.21.15.5:0/4293991590 shutdown_connections 2024-05-09T00:47:23.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.230+0000 7fa010642700 1 --2- 172.21.15.5:0/4293991590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa008109870 0x7fa0081111d0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:23.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.230+0000 7fa010642700 1 --2- 172.21.15.5:0/4293991590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0081045f0 0x7fa008104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:23.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.230+0000 7fa010642700 1 --2- 172.21.15.5:0/4293991590 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa008108f20 0x7fa008109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:23.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.230+0000 7fa010642700 1 -- 172.21.15.5:0/4293991590 >> 172.21.15.5:0/4293991590 conn(0x7fa008100040 msgr2=0x7fa008102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:23.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.230+0000 7fa010642700 1 -- 172.21.15.5:0/4293991590 shutdown_connections 2024-05-09T00:47:23.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.230+0000 7fa010642700 1 -- 172.21.15.5:0/4293991590 wait complete. 2024-05-09T00:47:23.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.231+0000 7fa010642700 1 Processor -- start 2024-05-09T00:47:23.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.231+0000 7fa010642700 1 -- start start 2024-05-09T00:47:23.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.231+0000 7fa010642700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0081045f0 0x7fa0081a2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:23.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.231+0000 7fa010642700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa008108f20 0x7fa0081a3210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:23.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.231+0000 7fa010642700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa008109870 0x7fa00819ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:23.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.231+0000 7fa010642700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa008076150 con 0x7fa008108f20 2024-05-09T00:47:23.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.231+0000 7fa010642700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa008075fd0 con 0x7fa008109870 2024-05-09T00:47:23.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.231+0000 7fa010642700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa0080762d0 con 0x7fa0081045f0 2024-05-09T00:47:23.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.231+0000 7fa00dbdd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa008108f20 0x7fa0081a3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:23.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.231+0000 7fa00dbdd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa008108f20 0x7fa0081a3210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39144/0 (socket says 172.21.15.5:39144) 2024-05-09T00:47:23.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.231+0000 7fa00dbdd700 1 -- 172.21.15.5:0/1187675447 learned_addr learned my addr 172.21.15.5:0/1187675447 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:47:23.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.231+0000 7fa00e3de700 1 --2- 172.21.15.5:0/1187675447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0081045f0 0x7fa0081a2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:23.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.231+0000 7fa00ebdf700 1 --2- 172.21.15.5:0/1187675447 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa008109870 0x7fa00819ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:23.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.231+0000 7fa00dbdd700 1 -- 172.21.15.5:0/1187675447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0081045f0 msgr2=0x7fa0081a2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:23.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.231+0000 7fa00dbdd700 1 --2- 172.21.15.5:0/1187675447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0081045f0 0x7fa0081a2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:23.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.232+0000 7fa00dbdd700 1 -- 172.21.15.5:0/1187675447 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa008109870 msgr2=0x7fa00819ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:23.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.232+0000 7fa00dbdd700 1 --2- 172.21.15.5:0/1187675447 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa008109870 0x7fa00819ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:23.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.232+0000 7fa00dbdd700 1 -- 172.21.15.5:0/1187675447 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa00819d520 con 0x7fa008108f20 2024-05-09T00:47:23.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.232+0000 7fa00ebdf700 1 --2- 172.21.15.5:0/1187675447 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa008109870 0x7fa00819ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:47:23.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.232+0000 7fa00dbdd700 1 --2- 172.21.15.5:0/1187675447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa008108f20 0x7fa0081a3210 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f9ff800d980 tx=0x7f9ff800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:23.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.232+0000 7f9fff7fe700 1 -- 172.21.15.5:0/1187675447 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ff8017070 con 0x7fa008108f20 2024-05-09T00:47:23.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.232+0000 7f9fff7fe700 1 -- 172.21.15.5:0/1187675447 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ff8009de0 con 0x7fa008108f20 2024-05-09T00:47:23.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.232+0000 7fa010642700 1 -- 172.21.15.5:0/1187675447 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa00819d810 con 0x7fa008108f20 2024-05-09T00:47:23.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.232+0000 7f9fff7fe700 1 -- 172.21.15.5:0/1187675447 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ff800f3c0 con 0x7fa008108f20 2024-05-09T00:47:23.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.232+0000 7fa010642700 1 -- 172.21.15.5:0/1187675447 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0081a9af0 con 0x7fa008108f20 2024-05-09T00:47:23.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.235+0000 7f9fff7fe700 1 -- 172.21.15.5:0/1187675447 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f9ff8009930 con 0x7fa008108f20 2024-05-09T00:47:23.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.236+0000 7f9fff7fe700 1 --2- 172.21.15.5:0/1187675447 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f9ff40b2900 0x7f9ff40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:23.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.236+0000 7f9fff7fe700 1 -- 172.21.15.5:0/1187675447 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f9ff80e1c80 con 0x7fa008108f20 2024-05-09T00:47:23.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.236+0000 7fa00e3de700 1 --2- 172.21.15.5:0/1187675447 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f9ff40b2900 0x7f9ff40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:23.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.236+0000 7fa01f058b80 1 Processor -- start 2024-05-09T00:47:23.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.236+0000 7fa01f058b80 1 -- start start 2024-05-09T00:47:23.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.236+0000 7fa01f058b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e49910160 0x558e499a94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:23.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.236+0000 7fa01f058b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e498264a0 0x558e499a9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:23.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.236+0000 7fa01f058b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e499a36a0 0x558e499a3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:23.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.236+0000 7fa01f058b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558e498c5ff0 con 0x558e49910160 2024-05-09T00:47:23.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.236+0000 7fa01f058b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x558e498647f0 con 0x558e498264a0 2024-05-09T00:47:23.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.236+0000 7fa01f058b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x558e498c1fb0 con 0x558e499a36a0 2024-05-09T00:47:23.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.237+0000 7fa00e3de700 1 --2- 172.21.15.5:0/1187675447 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f9ff40b2900 0x7f9ff40b4e10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa00400f440 tx=0x7fa0040079e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:23.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.237+0000 7fa00e3de700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e49910160 0x558e499a94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:23.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.237+0000 7fa00e3de700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e49910160 0x558e499a94f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39146/0 (socket says 172.21.15.5:39146) 2024-05-09T00:47:23.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.237+0000 7fa00e3de700 1 -- 172.21.15.5:0/1760679690 learned_addr learned my addr 172.21.15.5:0/1760679690 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:47:23.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.237+0000 7fa00ebdf700 1 --2- 172.21.15.5:0/1760679690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e499a36a0 0x558e499a3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:23.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.237+0000 7fa00dbdd700 1 --2- 172.21.15.5:0/1760679690 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e498264a0 0x558e499a9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:23.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.237+0000 7fa00e3de700 1 -- 172.21.15.5:0/1760679690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e499a36a0 msgr2=0x558e499a3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:47:23.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.237+0000 7fa00e3de700 1 --2- 172.21.15.5:0/1760679690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e499a36a0 0x558e499a3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:23.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.237+0000 7fa00e3de700 1 -- 172.21.15.5:0/1760679690 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e498264a0 msgr2=0x558e499a9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:47:23.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.237+0000 7fa00e3de700 1 --2- 172.21.15.5:0/1760679690 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e498264a0 0x558e499a9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:23.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.237+0000 7fa00e3de700 1 -- 172.21.15.5:0/1760679690 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e4985c370 con 0x558e49910160 2024-05-09T00:47:23.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.237+0000 7fa00e3de700 1 --2- 172.21.15.5:0/1760679690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e49910160 0x558e499a94f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa004014770 tx=0x7fa004012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d94a59d9db92043 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:23.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.237+0000 7f9ffcff9700 1 -- 172.21.15.5:0/1760679690 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa00401b050 con 0x558e49910160 2024-05-09T00:47:23.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.237+0000 7f9ffcff9700 1 -- 172.21.15.5:0/1760679690 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa004025070 con 0x558e49910160 2024-05-09T00:47:23.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.237+0000 7f9ffcff9700 1 -- 172.21.15.5:0/1760679690 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa004015660 con 0x558e49910160 2024-05-09T00:47:23.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.238+0000 7fa01f058b80 1 -- 172.21.15.5:0/1760679690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e49910160 msgr2=0x558e499a94f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:23.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.238+0000 7fa01f058b80 1 --2- 172.21.15.5:0/1760679690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e49910160 0x558e499a94f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa004014770 tx=0x7fa004012000 comp rx=0 tx=0).stop 2024-05-09T00:47:23.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.238+0000 7fa01f058b80 1 -- 172.21.15.5:0/1760679690 shutdown_connections 2024-05-09T00:47:23.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.238+0000 7fa01f058b80 1 --2- 172.21.15.5:0/1760679690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e499a36a0 0x558e499a3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:23.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.238+0000 7fa01f058b80 1 --2- 172.21.15.5:0/1760679690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e49910160 0x558e499a94f0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:23.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.238+0000 7fa01f058b80 1 --2- 172.21.15.5:0/1760679690 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e498264a0 0x558e499a9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:23.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.238+0000 7fa01f058b80 1 -- 172.21.15.5:0/1760679690 >> 172.21.15.5:0/1760679690 conn(0x558e498866a0 msgr2=0x558e4999e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:23.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.239+0000 7fa01f058b80 1 -- 172.21.15.5:0/1760679690 shutdown_connections 2024-05-09T00:47:23.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.239+0000 7fa01f058b80 1 -- 172.21.15.5:0/1760679690 wait complete. 2024-05-09T00:47:23.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.239+0000 7fa01f058b80 1 -- start start 2024-05-09T00:47:23.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.239+0000 7fa01f058b80 1 Processor -- start 2024-05-09T00:47:23.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.239+0000 7fa01f058b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e498264a0 0x558e499ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:23.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.239+0000 7fa01f058b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e49910160 0x558e499ad770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:23.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.239+0000 7fa01f058b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e499a36a0 0x558e499adcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:23.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.239+0000 7fa01f058b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558e49741ba0 con 0x558e49910160 2024-05-09T00:47:23.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.239+0000 7fa01f058b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x558e49880150 con 0x558e499a36a0 2024-05-09T00:47:23.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.239+0000 7fa01f058b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x558e499b01d0 con 0x558e498264a0 2024-05-09T00:47:23.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.240+0000 7fa01f058b80 10 client.0 _ll_register_callbacks cb 0x558e497376c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:47:23.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.240+0000 7fa00dbdd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e49910160 0x558e499ad770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:23.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.240+0000 7fa00dbdd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e49910160 0x558e499ad770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39148/0 (socket says 172.21.15.5:39148) 2024-05-09T00:47:23.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.240+0000 7fa00dbdd700 1 -- 172.21.15.5:0/4103990049 learned_addr learned my addr 172.21.15.5:0/4103990049 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:47:23.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.240+0000 7fa00ebdf700 1 --2- 172.21.15.5:0/4103990049 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e499a36a0 0x558e499adcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:23.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.240+0000 7fa00e3de700 1 --2- 172.21.15.5:0/4103990049 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e498264a0 0x558e499ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:23.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.240+0000 7fa00dbdd700 1 -- 172.21.15.5:0/4103990049 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e498264a0 msgr2=0x558e499ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:47:23.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.240+0000 7fa00dbdd700 1 --2- 172.21.15.5:0/4103990049 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e498264a0 0x558e499ad230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:23.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.240+0000 7fa00dbdd700 1 -- 172.21.15.5:0/4103990049 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e499a36a0 msgr2=0x558e499adcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:47:23.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.240+0000 7fa00dbdd700 1 --2- 172.21.15.5:0/4103990049 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e499a36a0 0x558e499adcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:23.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.240+0000 7fa00dbdd700 1 -- 172.21.15.5:0/4103990049 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e49741490 con 0x558e49910160 2024-05-09T00:47:23.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.240+0000 7fa00dbdd700 1 --2- 172.21.15.5:0/4103990049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e49910160 0x558e499ad770 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f9ff80a5290 tx=0x7f9ff8009d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de6c4c54b6610436 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:23.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.240+0000 7f9ff1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:47:23.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.240+0000 7f9ff1ffb700 1 -- 172.21.15.5:0/4103990049 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ff8020030 con 0x558e49910160 2024-05-09T00:47:23.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.240+0000 7f9ff1ffb700 1 -- 172.21.15.5:0/4103990049 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f9ff80ad170 con 0x558e49910160 2024-05-09T00:47:23.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.240+0000 7fa01f058b80 1 -- 172.21.15.5:0/4103990049 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa004025070 con 0x558e49910160 2024-05-09T00:47:23.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.241+0000 7f9ff1ffb700 1 -- 172.21.15.5:0/4103990049 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9ff80a9210 con 0x558e49910160 2024-05-09T00:47:23.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.241+0000 7fa01f058b80 10 client.15294 fetch_fsmap learned FSMap version 29 2024-05-09T00:47:23.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.241+0000 7fa01f058b80 1 -- 172.21.15.5:0/4103990049 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558e499bac10 con 0x558e49910160 2024-05-09T00:47:23.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.241+0000 7f9ff1ffb700 1 -- 172.21.15.5:0/4103990049 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa004025070 con 0x558e49910160 2024-05-09T00:47:23.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.241+0000 7f9ff1ffb700 1 -- 172.21.15.5:0/4103990049 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f9ff80dfbb0 con 0x558e49910160 2024-05-09T00:47:23.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.241+0000 7fa01f058b80 10 client.15294 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:47:23.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.241+0000 7fa01f058b80 10 client.15294 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:47:23.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.241+0000 7fa01f058b80 20 client.15294 populate_metadata read hostname 'smithi005' 2024-05-09T00:47:23.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.241+0000 7fa01f058b80 1 --2- 172.21.15.5:0/4103990049 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x558e499bd250 0x558e499bf670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:23.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.241+0000 7fa01f058b80 4 client.15294 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:47:23.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.241+0000 7fa01f058b80 1 -- 172.21.15.5:0/4103990049 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e498d39d0 con 0x558e499bd250 2024-05-09T00:47:23.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.241+0000 7fa00e3de700 1 --2- 172.21.15.5:0/4103990049 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x558e499bd250 0x558e499bf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:23.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.242+0000 7f9ff1ffb700 10 client.15294 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:47:23.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.242+0000 7fa00e3de700 1 --2- 172.21.15.5:0/4103990049 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x558e499bd250 0x558e499bf670 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7692c1f35fe5d77 server_cookie=e1d8fe7ed2f805dd in_seq=0 out_seq=0 2024-05-09T00:47:23.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.243+0000 7f9ff1ffb700 1 -- 172.21.15.5:0/4103990049 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558e498d39d0 con 0x558e499bd250 2024-05-09T00:47:23.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.243+0000 7f9ff1ffb700 10 client.15294 handle_command_reply: tid=0 2024-05-09T00:47:23.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.243+0000 7fa01f058b80 1 client.15294 shutdown 2024-05-09T00:47:23.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.243+0000 7fa01f058b80 1 -- 172.21.15.5:0/4103990049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e49910160 msgr2=0x558e499ad770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:23.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.243+0000 7fa01f058b80 1 --2- 172.21.15.5:0/4103990049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e49910160 0x558e499ad770 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f9ff80a5290 tx=0x7f9ff8009d60 comp rx=0 tx=0).stop 2024-05-09T00:47:23.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.243+0000 7fa01f058b80 1 -- 172.21.15.5:0/4103990049 shutdown_connections 2024-05-09T00:47:23.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.243+0000 7fa01f058b80 1 --2- 172.21.15.5:0/4103990049 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x558e499bd250 0x558e499bf670 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:23.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.243+0000 7f9ff1ffb700 0 client.15294 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:47:23.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.243+0000 7fa01f058b80 1 --2- 172.21.15.5:0/4103990049 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e499a36a0 0x558e499adcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:23.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.243+0000 7fa01f058b80 1 --2- 172.21.15.5:0/4103990049 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e498264a0 0x558e499ad230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:23.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.243+0000 7fa01f058b80 1 --2- 172.21.15.5:0/4103990049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e49910160 0x558e499ad770 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:23.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.243+0000 7fa01f058b80 1 -- 172.21.15.5:0/4103990049 >> 172.21.15.5:0/4103990049 conn(0x558e498866a0 msgr2=0x558e4999e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:23.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.243+0000 7fa01f058b80 1 -- 172.21.15.5:0/4103990049 shutdown_connections 2024-05-09T00:47:23.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.243+0000 7fa01f058b80 1 -- 172.21.15.5:0/4103990049 wait complete. 2024-05-09T00:47:23.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.243+0000 7fa01f058b80 20 client.15294 trim_cache size 0 max 16384 2024-05-09T00:47:23.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.264+0000 7fa01f058b80 1 Processor -- start 2024-05-09T00:47:23.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.264+0000 7fa01f058b80 1 -- start start 2024-05-09T00:47:23.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.264+0000 7fa01f058b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e498264a0 0x558e499a30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:23.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.264+0000 7fa01f058b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e49910160 0x558e499adaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:23.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.264+0000 7fa01f058b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e499a36a0 0x558e4999d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:23.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.264+0000 7fa01f058b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa0081b0fa0 con 0x558e499a36a0 2024-05-09T00:47:23.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.264+0000 7fa01f058b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa0081b2ae0 con 0x558e498264a0 2024-05-09T00:47:23.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.264+0000 7fa01f058b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa0081b1670 con 0x558e49910160 2024-05-09T00:47:23.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.264+0000 7fa00ebdf700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e499a36a0 0x558e4999d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:23.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.264+0000 7fa00ebdf700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e499a36a0 0x558e4999d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39164/0 (socket says 172.21.15.5:39164) 2024-05-09T00:47:23.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.264+0000 7fa00ebdf700 1 -- 172.21.15.5:0/2132535525 learned_addr learned my addr 172.21.15.5:0/2132535525 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:47:23.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.264+0000 7fa00dbdd700 1 --2- 172.21.15.5:0/2132535525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e49910160 0x558e499adaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:23.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.264+0000 7fa00e3de700 1 --2- 172.21.15.5:0/2132535525 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e498264a0 0x558e499a30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:23.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.265+0000 7fa00ebdf700 1 -- 172.21.15.5:0/2132535525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e49910160 msgr2=0x558e499adaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:47:23.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.265+0000 7fa00ebdf700 1 --2- 172.21.15.5:0/2132535525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e49910160 0x558e499adaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:23.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.265+0000 7fa00ebdf700 1 -- 172.21.15.5:0/2132535525 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e498264a0 msgr2=0x558e499a30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:47:23.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.265+0000 7fa00ebdf700 1 --2- 172.21.15.5:0/2132535525 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e498264a0 0x558e499a30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:23.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.265+0000 7fa00ebdf700 1 -- 172.21.15.5:0/2132535525 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e49997f60 con 0x558e499a36a0 2024-05-09T00:47:23.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.265+0000 7fa00ebdf700 1 --2- 172.21.15.5:0/2132535525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e499a36a0 0x558e4999d8e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa0000099c0 tx=0x7fa0000096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63e0616bb0b7da44 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:23.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.265+0000 7f9ff3fff700 1 -- 172.21.15.5:0/2132535525 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa000028070 con 0x558e499a36a0 2024-05-09T00:47:23.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.265+0000 7f9ff3fff700 1 -- 172.21.15.5:0/2132535525 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0000050c0 con 0x558e499a36a0 2024-05-09T00:47:23.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.265+0000 7f9ff3fff700 1 -- 172.21.15.5:0/2132535525 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa00000bc90 con 0x558e499a36a0 2024-05-09T00:47:23.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.266+0000 7fa01f058b80 1 -- 172.21.15.5:0/2132535525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e499a36a0 msgr2=0x558e4999d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:23.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.266+0000 7fa01f058b80 1 --2- 172.21.15.5:0/2132535525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e499a36a0 0x558e4999d8e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa0000099c0 tx=0x7fa0000096a0 comp rx=0 tx=0).stop 2024-05-09T00:47:23.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.266+0000 7fa01f058b80 1 -- 172.21.15.5:0/2132535525 shutdown_connections 2024-05-09T00:47:23.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.266+0000 7fa01f058b80 1 --2- 172.21.15.5:0/2132535525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e499a36a0 0x558e4999d8e0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:23.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.266+0000 7fa01f058b80 1 --2- 172.21.15.5:0/2132535525 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e498264a0 0x558e499a30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:23.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.266+0000 7fa01f058b80 1 --2- 172.21.15.5:0/2132535525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e49910160 0x558e499adaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:23.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.266+0000 7fa01f058b80 1 -- 172.21.15.5:0/2132535525 >> 172.21.15.5:0/2132535525 conn(0x558e498866a0 msgr2=0x558e499abb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:23.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.266+0000 7fa01f058b80 1 -- 172.21.15.5:0/2132535525 shutdown_connections 2024-05-09T00:47:23.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.266+0000 7fa01f058b80 1 -- 172.21.15.5:0/2132535525 wait complete. 2024-05-09T00:47:23.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.266+0000 7fa01f058b80 1 -- start start 2024-05-09T00:47:23.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.267+0000 7fa01f058b80 1 Processor -- start 2024-05-09T00:47:23.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.267+0000 7fa01f058b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e498264a0 0x558e499b5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:23.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.267+0000 7fa01f058b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e49910160 0x558e499b5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:23.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.267+0000 7fa01f058b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e499a36a0 0x558e499b22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:23.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.267+0000 7fa01f058b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa0081b2410 con 0x558e498264a0 2024-05-09T00:47:23.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.267+0000 7fa01f058b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa0081b1d40 con 0x558e49910160 2024-05-09T00:47:23.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.267+0000 7fa01f058b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa0081aef40 con 0x558e499a36a0 2024-05-09T00:47:23.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.267+0000 7fa01f058b80 10 client.0 _ll_register_callbacks cb 0x558e497376c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:47:23.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.267+0000 7fa00e3de700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e498264a0 0x558e499b5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:23.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.267+0000 7fa00e3de700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e498264a0 0x558e499b5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39168/0 (socket says 172.21.15.5:39168) 2024-05-09T00:47:23.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.267+0000 7fa00e3de700 1 -- 172.21.15.5:0/2514149475 learned_addr learned my addr 172.21.15.5:0/2514149475 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:47:23.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.267+0000 7fa00ebdf700 1 --2- 172.21.15.5:0/2514149475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e499a36a0 0x558e499b22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:23.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.267+0000 7fa00dbdd700 1 --2- 172.21.15.5:0/2514149475 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e49910160 0x558e499b5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:23.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.267+0000 7fa00e3de700 1 -- 172.21.15.5:0/2514149475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e499a36a0 msgr2=0x558e499b22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:47:23.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.267+0000 7fa00e3de700 1 --2- 172.21.15.5:0/2514149475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e499a36a0 0x558e499b22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:23.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.267+0000 7fa00e3de700 1 -- 172.21.15.5:0/2514149475 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e49910160 msgr2=0x558e499b5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:47:23.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.267+0000 7fa00e3de700 1 --2- 172.21.15.5:0/2514149475 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e49910160 0x558e499b5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:23.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.267+0000 7fa00e3de700 1 -- 172.21.15.5:0/2514149475 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e499b28b0 con 0x558e498264a0 2024-05-09T00:47:23.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.268+0000 7fa00e3de700 1 --2- 172.21.15.5:0/2514149475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e498264a0 0x558e499b5320 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa00400fb90 tx=0x7fa004004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9a652952658d947 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:23.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.268+0000 7f9ff1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:47:23.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.268+0000 7f9ff1ffb700 1 -- 172.21.15.5:0/2514149475 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa004038060 con 0x558e498264a0 2024-05-09T00:47:23.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.268+0000 7f9ff1ffb700 1 -- 172.21.15.5:0/2514149475 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fa004033750 con 0x558e498264a0 2024-05-09T00:47:23.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.268+0000 7fa01f058b80 1 -- 172.21.15.5:0/2514149475 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0000050c0 con 0x558e498264a0 2024-05-09T00:47:23.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.268+0000 7f9ff1ffb700 1 -- 172.21.15.5:0/2514149475 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa00401b050 con 0x558e498264a0 2024-05-09T00:47:23.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.268+0000 7fa01f058b80 10 client.15300 fetch_fsmap learned FSMap version 29 2024-05-09T00:47:23.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.268+0000 7fa01f058b80 1 -- 172.21.15.5:0/2514149475 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558e49a281d0 con 0x558e498264a0 2024-05-09T00:47:23.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.268+0000 7f9ff1ffb700 1 -- 172.21.15.5:0/2514149475 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0000050c0 con 0x558e498264a0 2024-05-09T00:47:23.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.268+0000 7f9ff1ffb700 1 -- 172.21.15.5:0/2514149475 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fa004005800 con 0x558e498264a0 2024-05-09T00:47:23.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.269+0000 7fa01f058b80 10 client.15300 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:47:23.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.269+0000 7fa01f058b80 10 client.15300 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:47:23.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.269+0000 7fa01f058b80 20 client.15300 populate_metadata read hostname 'smithi005' 2024-05-09T00:47:23.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.269+0000 7fa01f058b80 1 --2- 172.21.15.5:0/2514149475 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x558e499bd250 0x558e49a2a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:23.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.269+0000 7fa01f058b80 4 client.15300 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:47:23.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.269+0000 7fa01f058b80 1 -- 172.21.15.5:0/2514149475 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558e498cda70 con 0x558e499bd250 2024-05-09T00:47:23.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.269+0000 7fa00dbdd700 1 --2- 172.21.15.5:0/2514149475 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x558e499bd250 0x558e49a2a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:23.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.269+0000 7fa00dbdd700 1 --2- 172.21.15.5:0/2514149475 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x558e499bd250 0x558e49a2a610 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcab7e694b465c4f server_cookie=d6f341f81d1666a9 in_seq=0 out_seq=0 2024-05-09T00:47:23.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.269+0000 7f9ff1ffb700 10 client.15300 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:47:23.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.269+0000 7f9ff1ffb700 1 -- 172.21.15.5:0/2514149475 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x558e498cda70 con 0x558e499bd250 2024-05-09T00:47:23.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.269+0000 7f9ff1ffb700 10 client.15300 handle_command_reply: tid=0 2024-05-09T00:47:23.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.270+0000 7fa01f058b80 1 client.15300 shutdown 2024-05-09T00:47:23.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.270+0000 7fa01f058b80 1 -- 172.21.15.5:0/2514149475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e498264a0 msgr2=0x558e499b5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:23.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.270+0000 7fa01f058b80 1 --2- 172.21.15.5:0/2514149475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e498264a0 0x558e499b5320 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa00400fb90 tx=0x7fa004004b50 comp rx=0 tx=0).stop 2024-05-09T00:47:23.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.270+0000 7fa01f058b80 1 -- 172.21.15.5:0/2514149475 shutdown_connections 2024-05-09T00:47:23.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.270+0000 7fa01f058b80 1 --2- 172.21.15.5:0/2514149475 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x558e499bd250 0x558e49a2a610 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:23.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.270+0000 7fa01f058b80 1 --2- 172.21.15.5:0/2514149475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558e499a36a0 0x558e499b22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:23.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.270+0000 7fa01f058b80 1 --2- 172.21.15.5:0/2514149475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e498264a0 0x558e499b5320 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:23.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.270+0000 7fa01f058b80 1 --2- 172.21.15.5:0/2514149475 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e49910160 0x558e499b5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:23.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.270+0000 7fa01f058b80 1 -- 172.21.15.5:0/2514149475 >> 172.21.15.5:0/2514149475 conn(0x558e498866a0 msgr2=0x558e499abb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:23.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.270+0000 7f9ff1ffb700 0 client.15300 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:47:23.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.270+0000 7fa01f058b80 1 -- 172.21.15.5:0/2514149475 shutdown_connections 2024-05-09T00:47:23.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.270+0000 7fa01f058b80 1 -- 172.21.15.5:0/2514149475 wait complete. 2024-05-09T00:47:23.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.270+0000 7fa01f058b80 20 client.15300 trim_cache size 0 max 16384 2024-05-09T00:47:23.276 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-09T00:47:23.276 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T00:47:23.276 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-05-09T00:47:23.276 INFO:teuthology.orchestra.run.smithi005.stdout: "827cc3b6-e2c4-486e-8d9d-36bf9577121c": { 2024-05-09T00:47:23.276 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-05-09T00:47:23.276 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "827cc3b6-e2c4-486e-8d9d-36bf9577121c", 2024-05-09T00:47:23.276 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-05-09T00:47:23.276 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T00:47:23.276 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T00:47:23.276 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-09T00:47:23.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.270+0000 7fa010642700 1 -- 172.21.15.5:0/1187675447 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f9ff40b2900 msgr2=0x7f9ff40b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:23.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.270+0000 7fa010642700 1 --2- 172.21.15.5:0/1187675447 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f9ff40b2900 0x7f9ff40b4e10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa00400f440 tx=0x7fa0040079e0 comp rx=0 tx=0).stop 2024-05-09T00:47:23.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.270+0000 7fa010642700 1 -- 172.21.15.5:0/1187675447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa008108f20 msgr2=0x7fa0081a3210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:23.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.270+0000 7fa010642700 1 --2- 172.21.15.5:0/1187675447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa008108f20 0x7fa0081a3210 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f9ff800d980 tx=0x7f9ff800dc90 comp rx=0 tx=0).stop 2024-05-09T00:47:23.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.271+0000 7fa010642700 1 -- 172.21.15.5:0/1187675447 shutdown_connections 2024-05-09T00:47:23.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.271+0000 7fa010642700 1 --2- 172.21.15.5:0/1187675447 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f9ff40b2900 0x7f9ff40b4e10 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:23.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.271+0000 7fa010642700 1 --2- 172.21.15.5:0/1187675447 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa008109870 0x7fa00819ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:23.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.271+0000 7fa010642700 1 --2- 172.21.15.5:0/1187675447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0081045f0 0x7fa0081a2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:23.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.271+0000 7fa010642700 1 --2- 172.21.15.5:0/1187675447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa008108f20 0x7fa0081a3210 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:23.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.271+0000 7fa010642700 1 -- 172.21.15.5:0/1187675447 >> 172.21.15.5:0/1187675447 conn(0x7fa008100040 msgr2=0x7fa008102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:23.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.271+0000 7fa010642700 1 -- 172.21.15.5:0/1187675447 shutdown_connections 2024-05-09T00:47:23.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:23.271+0000 7fa010642700 1 -- 172.21.15.5:0/1187675447 wait complete. 2024-05-09T00:47:23.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:47:23 smithi005 ceph-mon[30365]: pgmap v402: 97 pgs: 97 active+clean; 651 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 511 B/s wr, 9 op/s 2024-05-09T00:47:23.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:47:23 smithi131 ceph-mon[34547]: pgmap v402: 97 pgs: 97 active+clean; 651 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 511 B/s wr, 9 op/s 2024-05-09T00:47:23.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:47:23 smithi157 ceph-mon[34294]: pgmap v402: 97 pgs: 97 active+clean; 651 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 511 B/s wr, 9 op/s 2024-05-09T00:47:24.007 INFO:tasks.cephfs.filesystem:scrub status for tag:827cc3b6-e2c4-486e-8d9d-36bf9577121c - {'path': '/', 'tag': '827cc3b6-e2c4-486e-8d9d-36bf9577121c', 'options': 'recursive,force'} 2024-05-09T00:47:25.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:47:25 smithi005 ceph-mon[30365]: pgmap v403: 97 pgs: 97 active+clean; 656 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-05-09T00:47:25.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:47:25 smithi131 ceph-mon[34547]: pgmap v403: 97 pgs: 97 active+clean; 656 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-05-09T00:47:25.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:47:25 smithi157 ceph-mon[34294]: pgmap v403: 97 pgs: 97 active+clean; 656 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-05-09T00:47:27.040 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:47:26 smithi005 ceph-mon[30365]: pgmap v404: 97 pgs: 97 active+clean; 656 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 853 B/s wr, 9 op/s 2024-05-09T00:47:27.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:47:26 smithi131 ceph-mon[34547]: pgmap v404: 97 pgs: 97 active+clean; 656 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 853 B/s wr, 9 op/s 2024-05-09T00:47:27.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:47:26 smithi157 ceph-mon[34294]: pgmap v404: 97 pgs: 97 active+clean; 656 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 853 B/s wr, 9 op/s 2024-05-09T00:47:29.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:47:28 smithi005 ceph-mon[30365]: pgmap v405: 97 pgs: 97 active+clean; 656 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 853 B/s wr, 10 op/s 2024-05-09T00:47:29.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:47:28 smithi131 ceph-mon[34547]: pgmap v405: 97 pgs: 97 active+clean; 656 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 853 B/s wr, 10 op/s 2024-05-09T00:47:29.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:47:28 smithi157 ceph-mon[34294]: pgmap v405: 97 pgs: 97 active+clean; 656 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 853 B/s wr, 10 op/s 2024-05-09T00:47:31.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:47:30 smithi005 ceph-mon[30365]: pgmap v406: 97 pgs: 97 active+clean; 656 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 853 B/s wr, 11 op/s 2024-05-09T00:47:31.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:47:30 smithi131 ceph-mon[34547]: pgmap v406: 97 pgs: 97 active+clean; 656 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 853 B/s wr, 11 op/s 2024-05-09T00:47:31.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:47:30 smithi157 ceph-mon[34294]: pgmap v406: 97 pgs: 97 active+clean; 656 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 853 B/s wr, 11 op/s 2024-05-09T00:47:31.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.717+0000 7ff597284640 1 -- 172.21.15.5:0/256116236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff590103d90 msgr2=0x7ff5901041a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:31.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.717+0000 7ff597284640 1 --2- 172.21.15.5:0/256116236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff590103d90 0x7ff5901041a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff584009900 tx=0x7ff58402a410 comp rx=0 tx=0).stop 2024-05-09T00:47:31.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.717+0000 7ff597284640 1 -- 172.21.15.5:0/256116236 shutdown_connections 2024-05-09T00:47:31.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.717+0000 7ff597284640 1 --2- 172.21.15.5:0/256116236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff590108a90 0x7ff59010aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:31.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.717+0000 7ff597284640 1 --2- 172.21.15.5:0/256116236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff590103d90 0x7ff5901041a0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:31.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.717+0000 7ff597284640 1 --2- 172.21.15.5:0/256116236 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff590102b50 0x7ff590103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:31.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.717+0000 7ff597284640 1 -- 172.21.15.5:0/256116236 >> 172.21.15.5:0/256116236 conn(0x7ff5900fdd40 msgr2=0x7ff5901001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:31.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.717+0000 7ff597284640 1 -- 172.21.15.5:0/256116236 shutdown_connections 2024-05-09T00:47:31.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.717+0000 7ff597284640 1 -- 172.21.15.5:0/256116236 wait complete. 2024-05-09T00:47:31.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.718+0000 7ff597284640 1 Processor -- start 2024-05-09T00:47:31.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.718+0000 7ff597284640 1 -- start start 2024-05-09T00:47:31.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.718+0000 7ff597284640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff590102b50 0x7ff59019a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:31.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.718+0000 7ff597284640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff590103d90 0x7ff59019a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:31.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.718+0000 7ff597284640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff590108a90 0x7ff5901a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:31.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.719+0000 7ff597284640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff59010f780 con 0x7ff590102b50 2024-05-09T00:47:31.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.719+0000 7ff597284640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff59010f600 con 0x7ff590103d90 2024-05-09T00:47:31.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.719+0000 7ff597284640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff59010f900 con 0x7ff590108a90 2024-05-09T00:47:31.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.719+0000 7ff595020640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff590102b50 0x7ff59019a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.719+0000 7ff595020640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff590102b50 0x7ff59019a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40460/0 (socket says 172.21.15.5:40460) 2024-05-09T00:47:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.719+0000 7ff595020640 1 -- 172.21.15.5:0/2671754324 learned_addr learned my addr 172.21.15.5:0/2671754324 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:47:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.719+0000 7ff595821640 1 --2- 172.21.15.5:0/2671754324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff590108a90 0x7ff5901a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.719+0000 7ff59481f640 1 --2- 172.21.15.5:0/2671754324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff590103d90 0x7ff59019a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.719+0000 7ff595020640 1 -- 172.21.15.5:0/2671754324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff590108a90 msgr2=0x7ff5901a1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.719+0000 7ff595020640 1 --2- 172.21.15.5:0/2671754324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff590108a90 0x7ff5901a1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.719+0000 7ff595020640 1 -- 172.21.15.5:0/2671754324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff590103d90 msgr2=0x7ff59019a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.719+0000 7ff595020640 1 --2- 172.21.15.5:0/2671754324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff590103d90 0x7ff59019a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.719+0000 7ff595020640 1 -- 172.21.15.5:0/2671754324 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5901a1ed0 con 0x7ff590102b50 2024-05-09T00:47:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.719+0000 7ff595821640 1 --2- 172.21.15.5:0/2671754324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff590108a90 0x7ff5901a1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:47:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.719+0000 7ff59481f640 1 --2- 172.21.15.5:0/2671754324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff590103d90 0x7ff59019a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:47:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.719+0000 7ff595020640 1 --2- 172.21.15.5:0/2671754324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff590102b50 0x7ff59019a390 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff588005d00 tx=0x7ff58800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.719+0000 7ff58e7fc640 1 -- 172.21.15.5:0/2671754324 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff588017070 con 0x7ff590102b50 2024-05-09T00:47:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.719+0000 7ff597284640 1 -- 172.21.15.5:0/2671754324 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5901a21c0 con 0x7ff590102b50 2024-05-09T00:47:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.719+0000 7ff58e7fc640 1 -- 172.21.15.5:0/2671754324 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff588009c30 con 0x7ff590102b50 2024-05-09T00:47:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.719+0000 7ff58e7fc640 1 -- 172.21.15.5:0/2671754324 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff58800f620 con 0x7ff590102b50 2024-05-09T00:47:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.719+0000 7ff597284640 1 -- 172.21.15.5:0/2671754324 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5901a26c0 con 0x7ff590102b50 2024-05-09T00:47:31.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.722+0000 7ff58e7fc640 1 -- 172.21.15.5:0/2671754324 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7ff58801a070 con 0x7ff590102b50 2024-05-09T00:47:31.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.722+0000 7ff59e20f740 1 Processor -- start 2024-05-09T00:47:31.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.723+0000 7ff59e20f740 1 -- start start 2024-05-09T00:47:31.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.723+0000 7ff59e20f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560af0e121b0 0x560af11b6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:31.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.723+0000 7ff59e20f740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560af0de92f0 0x560af11b6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:31.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.723+0000 7ff59e20f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af0df14d0 0x560af11bd980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:31.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.723+0000 7ff59e20f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560af111a7c0 con 0x560af0df14d0 2024-05-09T00:47:31.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.723+0000 7ff59e20f740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x560af10b60e0 con 0x560af0de92f0 2024-05-09T00:47:31.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.723+0000 7ff59e20f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x560af1116780 con 0x560af0e121b0 2024-05-09T00:47:31.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.723+0000 7ff59481f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af0df14d0 0x560af11bd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:31.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.723+0000 7ff59481f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af0df14d0 0x560af11bd980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40474/0 (socket says 172.21.15.5:40474) 2024-05-09T00:47:31.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.723+0000 7ff59481f640 1 -- 172.21.15.5:0/1807426370 learned_addr learned my addr 172.21.15.5:0/1807426370 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:47:31.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.723+0000 7ff595821640 1 --2- 172.21.15.5:0/1807426370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560af0e121b0 0x560af11b6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:31.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.723+0000 7ff595020640 1 --2- 172.21.15.5:0/1807426370 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560af0de92f0 0x560af11b6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:31.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.723+0000 7ff58e7fc640 1 --2- 172.21.15.5:0/2671754324 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ff5780b27e0 0x7ff5780b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:31.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.723+0000 7ff59481f640 1 -- 172.21.15.5:0/1807426370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560af0e121b0 msgr2=0x560af11b6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:47:31.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.723+0000 7ff59481f640 1 --2- 172.21.15.5:0/1807426370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560af0e121b0 0x560af11b6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:31.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.723+0000 7ff59481f640 1 -- 172.21.15.5:0/1807426370 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560af0de92f0 msgr2=0x560af11b6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:47:31.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.723+0000 7ff59481f640 1 --2- 172.21.15.5:0/1807426370 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560af0de92f0 0x560af11b6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:31.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.723+0000 7ff59481f640 1 -- 172.21.15.5:0/1807426370 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560af10df7c0 con 0x560af0df14d0 2024-05-09T00:47:31.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.723+0000 7ff595020640 1 --2- 172.21.15.5:0/1807426370 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560af0de92f0 0x560af11b6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:47:31.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.723+0000 7ff595821640 1 --2- 172.21.15.5:0/2671754324 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ff5780b27e0 0x7ff5780b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:31.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.724+0000 7ff58e7fc640 1 -- 172.21.15.5:0/2671754324 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7ff5880e1dd0 con 0x7ff590102b50 2024-05-09T00:47:31.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.724+0000 7ff59481f640 1 --2- 172.21.15.5:0/1807426370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af0df14d0 0x560af11bd980 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff584031440 tx=0x7ff584032bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a07ea3ab325e823f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.724+0000 7ff595821640 1 --2- 172.21.15.5:0/2671754324 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ff5780b27e0 0x7ff5780b4cf0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x560af11b76d0 tx=0x7ff580004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.724+0000 7ff5777fe640 1 -- 172.21.15.5:0/1807426370 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff58403f070 con 0x560af0df14d0 2024-05-09T00:47:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.724+0000 7ff5777fe640 1 -- 172.21.15.5:0/1807426370 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5840317a0 con 0x560af0df14d0 2024-05-09T00:47:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.724+0000 7ff5777fe640 1 -- 172.21.15.5:0/1807426370 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff584035070 con 0x560af0df14d0 2024-05-09T00:47:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.725+0000 7ff59e20f740 1 -- 172.21.15.5:0/1807426370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af0df14d0 msgr2=0x560af11bd980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.725+0000 7ff59e20f740 1 --2- 172.21.15.5:0/1807426370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af0df14d0 0x560af11bd980 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff584031440 tx=0x7ff584032bc0 comp rx=0 tx=0).stop 2024-05-09T00:47:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.725+0000 7ff59e20f740 1 -- 172.21.15.5:0/1807426370 shutdown_connections 2024-05-09T00:47:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.725+0000 7ff59e20f740 1 --2- 172.21.15.5:0/1807426370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af0df14d0 0x560af11bd980 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.725+0000 7ff59e20f740 1 --2- 172.21.15.5:0/1807426370 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560af0de92f0 0x560af11b6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.725+0000 7ff59e20f740 1 --2- 172.21.15.5:0/1807426370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560af0e121b0 0x560af11b6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.725+0000 7ff59e20f740 1 -- 172.21.15.5:0/1807426370 >> 172.21.15.5:0/1807426370 conn(0x560af0e06300 msgr2=0x560af11b1e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.725+0000 7ff59e20f740 1 -- 172.21.15.5:0/1807426370 shutdown_connections 2024-05-09T00:47:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.725+0000 7ff59e20f740 1 -- 172.21.15.5:0/1807426370 wait complete. 2024-05-09T00:47:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.725+0000 7ff59e20f740 1 -- start start 2024-05-09T00:47:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.726+0000 7ff59e20f740 1 Processor -- start 2024-05-09T00:47:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.726+0000 7ff59e20f740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560af0de92f0 0x560af11c01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.726+0000 7ff59e20f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af0df14d0 0x560af11c0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.726+0000 7ff59e20f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560af0e121b0 0x560af11c7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.726+0000 7ff59e20f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560af11be370 con 0x560af0df14d0 2024-05-09T00:47:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.726+0000 7ff59e20f740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x560af10a19c0 con 0x560af0de92f0 2024-05-09T00:47:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.726+0000 7ff59e20f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x560af11be4f0 con 0x560af0e121b0 2024-05-09T00:47:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.726+0000 7ff59e20f740 10 client.0 _ll_register_callbacks cb 0x560af10ce020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:47:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.726+0000 7ff59481f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af0df14d0 0x560af11c0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.726+0000 7ff59481f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af0df14d0 0x560af11c0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40478/0 (socket says 172.21.15.5:40478) 2024-05-09T00:47:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.726+0000 7ff59481f640 1 -- 172.21.15.5:0/444362816 learned_addr learned my addr 172.21.15.5:0/444362816 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:47:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.726+0000 7ff595020640 1 --2- 172.21.15.5:0/444362816 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560af0de92f0 0x560af11c01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.726+0000 7ff595821640 1 --2- 172.21.15.5:0/444362816 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560af0e121b0 0x560af11c7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.727+0000 7ff59481f640 1 -- 172.21.15.5:0/444362816 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560af0e121b0 msgr2=0x560af11c7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:47:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.727+0000 7ff59481f640 1 --2- 172.21.15.5:0/444362816 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560af0e121b0 0x560af11c7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.727+0000 7ff59481f640 1 -- 172.21.15.5:0/444362816 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560af0de92f0 msgr2=0x560af11c01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:47:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.727+0000 7ff59481f640 1 --2- 172.21.15.5:0/444362816 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560af0de92f0 0x560af11c01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.727+0000 7ff59481f640 1 -- 172.21.15.5:0/444362816 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560af10c81f0 con 0x560af0df14d0 2024-05-09T00:47:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.727+0000 7ff595020640 1 --2- 172.21.15.5:0/444362816 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560af0de92f0 0x560af11c01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:47:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.727+0000 7ff59481f640 1 --2- 172.21.15.5:0/444362816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af0df14d0 0x560af11c0710 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff5840319d0 tx=0x7ff584031a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d79132c21cf55faf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.727+0000 7ff574ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:47:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.727+0000 7ff574ff9640 1 -- 172.21.15.5:0/444362816 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff58403f070 con 0x560af0df14d0 2024-05-09T00:47:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.727+0000 7ff574ff9640 1 -- 172.21.15.5:0/444362816 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7ff5840035f0 con 0x560af0df14d0 2024-05-09T00:47:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.727+0000 7ff59e20f740 1 -- 172.21.15.5:0/444362816 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5840317a0 con 0x560af0df14d0 2024-05-09T00:47:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.727+0000 7ff574ff9640 1 -- 172.21.15.5:0/444362816 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff584035040 con 0x560af0df14d0 2024-05-09T00:47:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.727+0000 7ff59e20f740 10 client.15312 fetch_fsmap learned FSMap version 29 2024-05-09T00:47:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.727+0000 7ff59e20f740 1 -- 172.21.15.5:0/444362816 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560af0e7d790 con 0x560af0df14d0 2024-05-09T00:47:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.727+0000 7ff574ff9640 1 -- 172.21.15.5:0/444362816 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5840317a0 con 0x560af0df14d0 2024-05-09T00:47:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.728+0000 7ff574ff9640 1 -- 172.21.15.5:0/444362816 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ff584044960 con 0x560af0df14d0 2024-05-09T00:47:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.728+0000 7ff59e20f740 10 client.15312 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:47:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.728+0000 7ff59e20f740 10 client.15312 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:47:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.728+0000 7ff59e20f740 20 client.15312 populate_metadata read hostname 'smithi005' 2024-05-09T00:47:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.728+0000 7ff59e20f740 1 --2- 172.21.15.5:0/444362816 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x560af0dd0d80 0x560af11cd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.728+0000 7ff59e20f740 4 client.15312 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:47:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.728+0000 7ff59e20f740 1 -- 172.21.15.5:0/444362816 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560af11283d0 con 0x560af0dd0d80 2024-05-09T00:47:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.728+0000 7ff595020640 1 --2- 172.21.15.5:0/444362816 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x560af0dd0d80 0x560af11cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:31.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.728+0000 7ff595020640 1 --2- 172.21.15.5:0/444362816 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x560af0dd0d80 0x560af11cd730 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e30342d24a320be server_cookie=52cecefcc3b4a7ac in_seq=0 out_seq=0 2024-05-09T00:47:31.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.728+0000 7ff574ff9640 10 client.15312 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:47:31.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.729+0000 7ff574ff9640 1 -- 172.21.15.5:0/444362816 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560af11283d0 con 0x560af0dd0d80 2024-05-09T00:47:31.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.729+0000 7ff574ff9640 10 client.15312 handle_command_reply: tid=0 2024-05-09T00:47:31.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.729+0000 7ff59e20f740 1 client.15312 shutdown 2024-05-09T00:47:31.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.729+0000 7ff59e20f740 1 -- 172.21.15.5:0/444362816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af0df14d0 msgr2=0x560af11c0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:31.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.729+0000 7ff59e20f740 1 --2- 172.21.15.5:0/444362816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af0df14d0 0x560af11c0710 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff5840319d0 tx=0x7ff584031a00 comp rx=0 tx=0).stop 2024-05-09T00:47:31.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.730+0000 7ff59e20f740 1 -- 172.21.15.5:0/444362816 shutdown_connections 2024-05-09T00:47:31.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.730+0000 7ff59e20f740 1 --2- 172.21.15.5:0/444362816 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x560af0dd0d80 0x560af11cd730 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:31.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.730+0000 7ff59e20f740 1 --2- 172.21.15.5:0/444362816 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560af0e121b0 0x560af11c7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:31.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.730+0000 7ff574ff9640 0 client.15312 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:47:31.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.730+0000 7ff59e20f740 1 --2- 172.21.15.5:0/444362816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af0df14d0 0x560af11c0710 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:31.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.730+0000 7ff59e20f740 1 --2- 172.21.15.5:0/444362816 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560af0de92f0 0x560af11c01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.730+0000 7ff59e20f740 1 -- 172.21.15.5:0/444362816 >> 172.21.15.5:0/444362816 conn(0x560af0e06300 msgr2=0x560af11b4da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.730+0000 7ff59e20f740 1 -- 172.21.15.5:0/444362816 shutdown_connections 2024-05-09T00:47:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.730+0000 7ff59e20f740 1 -- 172.21.15.5:0/444362816 wait complete. 2024-05-09T00:47:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.730+0000 7ff59e20f740 20 client.15312 trim_cache size 0 max 16384 2024-05-09T00:47:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.745+0000 7ff59e20f740 1 Processor -- start 2024-05-09T00:47:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.745+0000 7ff59e20f740 1 -- start start 2024-05-09T00:47:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.745+0000 7ff59e20f740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560af0de92f0 0x560af11bacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.745+0000 7ff59e20f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af0df14d0 0x560af11b53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.745+0000 7ff59e20f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560af0e121b0 0x560af11b58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.745+0000 7ff59e20f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff5901ab420 con 0x560af0df14d0 2024-05-09T00:47:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.745+0000 7ff59e20f740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff5901acf60 con 0x560af0de92f0 2024-05-09T00:47:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.745+0000 7ff59e20f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff5901abaf0 con 0x560af0e121b0 2024-05-09T00:47:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.745+0000 7ff59481f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af0df14d0 0x560af11b53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.745+0000 7ff59481f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af0df14d0 0x560af11b53a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40492/0 (socket says 172.21.15.5:40492) 2024-05-09T00:47:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.745+0000 7ff59481f640 1 -- 172.21.15.5:0/4197738626 learned_addr learned my addr 172.21.15.5:0/4197738626 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:47:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.746+0000 7ff595020640 1 --2- 172.21.15.5:0/4197738626 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560af0de92f0 0x560af11bacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.746+0000 7ff595821640 1 --2- 172.21.15.5:0/4197738626 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560af0e121b0 0x560af11b58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.746+0000 7ff59481f640 1 -- 172.21.15.5:0/4197738626 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560af0e121b0 msgr2=0x560af11b58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:47:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.746+0000 7ff59481f640 1 --2- 172.21.15.5:0/4197738626 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560af0e121b0 0x560af11b58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.746+0000 7ff59481f640 1 -- 172.21.15.5:0/4197738626 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560af0de92f0 msgr2=0x560af11bacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:47:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.746+0000 7ff59481f640 1 --2- 172.21.15.5:0/4197738626 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560af0de92f0 0x560af11bacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.746+0000 7ff59481f640 1 -- 172.21.15.5:0/4197738626 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560af0ef45f0 con 0x560af0df14d0 2024-05-09T00:47:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.746+0000 7ff595020640 1 --2- 172.21.15.5:0/4197738626 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560af0de92f0 0x560af11bacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:47:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.746+0000 7ff59481f640 1 --2- 172.21.15.5:0/4197738626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af0df14d0 0x560af11b53a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff58402a930 tx=0x7ff584005a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=328a6fd4f33f09b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.746+0000 7ff576ffd640 1 -- 172.21.15.5:0/4197738626 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff58403f070 con 0x560af0df14d0 2024-05-09T00:47:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.746+0000 7ff576ffd640 1 -- 172.21.15.5:0/4197738626 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff58404d070 con 0x560af0df14d0 2024-05-09T00:47:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.746+0000 7ff576ffd640 1 -- 172.21.15.5:0/4197738626 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff584044a90 con 0x560af0df14d0 2024-05-09T00:47:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.747+0000 7ff59e20f740 1 -- 172.21.15.5:0/4197738626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af0df14d0 msgr2=0x560af11b53a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.747+0000 7ff59e20f740 1 --2- 172.21.15.5:0/4197738626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af0df14d0 0x560af11b53a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff58402a930 tx=0x7ff584005a60 comp rx=0 tx=0).stop 2024-05-09T00:47:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.747+0000 7ff59e20f740 1 -- 172.21.15.5:0/4197738626 shutdown_connections 2024-05-09T00:47:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.747+0000 7ff59e20f740 1 --2- 172.21.15.5:0/4197738626 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560af0e121b0 0x560af11b58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.747+0000 7ff59e20f740 1 --2- 172.21.15.5:0/4197738626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af0df14d0 0x560af11b53a0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.747+0000 7ff59e20f740 1 --2- 172.21.15.5:0/4197738626 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560af0de92f0 0x560af11bacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.747+0000 7ff59e20f740 1 -- 172.21.15.5:0/4197738626 >> 172.21.15.5:0/4197738626 conn(0x560af0e06300 msgr2=0x560af11b4c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.747+0000 7ff59e20f740 1 -- 172.21.15.5:0/4197738626 shutdown_connections 2024-05-09T00:47:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.747+0000 7ff59e20f740 1 -- 172.21.15.5:0/4197738626 wait complete. 2024-05-09T00:47:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.747+0000 7ff59e20f740 1 -- start start 2024-05-09T00:47:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.747+0000 7ff59e20f740 1 Processor -- start 2024-05-09T00:47:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.748+0000 7ff59e20f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560af0de92f0 0x560af11c4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.748+0000 7ff59e20f740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560af0df14d0 0x560af11c52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.748+0000 7ff59e20f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af0e121b0 0x560af1233a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.748+0000 7ff59e20f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff5901ac890 con 0x560af0e121b0 2024-05-09T00:47:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.748+0000 7ff59e20f740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff5901ac1c0 con 0x560af0df14d0 2024-05-09T00:47:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.748+0000 7ff59e20f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff5901a9400 con 0x560af0de92f0 2024-05-09T00:47:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.748+0000 7ff59e20f740 10 client.0 _ll_register_callbacks cb 0x7ff5880e56e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:47:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.748+0000 7ff595821640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af0e121b0 0x560af1233a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.748+0000 7ff595821640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af0e121b0 0x560af1233a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40498/0 (socket says 172.21.15.5:40498) 2024-05-09T00:47:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.748+0000 7ff595821640 1 -- 172.21.15.5:0/1407001772 learned_addr learned my addr 172.21.15.5:0/1407001772 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:47:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.748+0000 7ff595020640 1 --2- 172.21.15.5:0/1407001772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560af0de92f0 0x560af11c4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.748+0000 7ff59481f640 1 --2- 172.21.15.5:0/1407001772 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560af0df14d0 0x560af11c52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.748+0000 7ff595821640 1 -- 172.21.15.5:0/1407001772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560af0de92f0 msgr2=0x560af11c4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:47:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.748+0000 7ff595821640 1 --2- 172.21.15.5:0/1407001772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560af0de92f0 0x560af11c4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.748+0000 7ff595821640 1 -- 172.21.15.5:0/1407001772 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560af0df14d0 msgr2=0x560af11c52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:47:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.748+0000 7ff595821640 1 --2- 172.21.15.5:0/1407001772 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560af0df14d0 0x560af11c52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.748+0000 7ff595821640 1 -- 172.21.15.5:0/1407001772 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560af10deed0 con 0x560af0e121b0 2024-05-09T00:47:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.749+0000 7ff595821640 1 --2- 172.21.15.5:0/1407001772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af0e121b0 0x560af1233a10 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff580010b80 tx=0x7ff580004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5789e78b5f5ab137 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.749+0000 7ff574ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:47:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.749+0000 7ff574ff9640 1 -- 172.21.15.5:0/1407001772 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff580011b10 con 0x560af0e121b0 2024-05-09T00:47:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.749+0000 7ff574ff9640 1 -- 172.21.15.5:0/1407001772 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7ff580011cb0 con 0x560af0e121b0 2024-05-09T00:47:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.749+0000 7ff59e20f740 1 -- 172.21.15.5:0/1407001772 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff58404d070 con 0x560af0e121b0 2024-05-09T00:47:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.749+0000 7ff574ff9640 1 -- 172.21.15.5:0/1407001772 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff58001e440 con 0x560af0e121b0 2024-05-09T00:47:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.749+0000 7ff59e20f740 10 client.15318 fetch_fsmap learned FSMap version 29 2024-05-09T00:47:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.749+0000 7ff59e20f740 1 -- 172.21.15.5:0/1407001772 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560af1235020 con 0x560af0e121b0 2024-05-09T00:47:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.749+0000 7ff574ff9640 1 -- 172.21.15.5:0/1407001772 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff58404d070 con 0x560af0e121b0 2024-05-09T00:47:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.749+0000 7ff574ff9640 1 -- 172.21.15.5:0/1407001772 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ff580033070 con 0x560af0e121b0 2024-05-09T00:47:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.749+0000 7ff59e20f740 10 client.15318 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:47:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.749+0000 7ff59e20f740 10 client.15318 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:47:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.749+0000 7ff59e20f740 20 client.15318 populate_metadata read hostname 'smithi005' 2024-05-09T00:47:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.749+0000 7ff59e20f740 1 --2- 172.21.15.5:0/1407001772 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x560af0da61c0 0x560af12379b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.749+0000 7ff59e20f740 4 client.15318 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:47:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.749+0000 7ff59e20f740 1 -- 172.21.15.5:0/1407001772 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560af1122240 con 0x560af0da61c0 2024-05-09T00:47:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.750+0000 7ff595020640 1 --2- 172.21.15.5:0/1407001772 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x560af0da61c0 0x560af12379b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.750+0000 7ff574ff9640 10 client.15318 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:47:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.750+0000 7ff595020640 1 --2- 172.21.15.5:0/1407001772 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x560af0da61c0 0x560af12379b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ecd3d4a2db19ac1 server_cookie=7b3bb8e12ee09c56 in_seq=0 out_seq=0 2024-05-09T00:47:31.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.763+0000 7ff574ff9640 1 -- 172.21.15.5:0/1407001772 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x560af1122240 con 0x560af0da61c0 2024-05-09T00:47:31.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.763+0000 7ff574ff9640 10 client.15318 handle_command_reply: tid=0 2024-05-09T00:47:31.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.763+0000 7ff59e20f740 1 client.15318 shutdown 2024-05-09T00:47:31.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.763+0000 7ff59e20f740 1 -- 172.21.15.5:0/1407001772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af0e121b0 msgr2=0x560af1233a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:31.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.763+0000 7ff59e20f740 1 --2- 172.21.15.5:0/1407001772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af0e121b0 0x560af1233a10 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff580010b80 tx=0x7ff580004ab0 comp rx=0 tx=0).stop 2024-05-09T00:47:31.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.763+0000 7ff59e20f740 1 -- 172.21.15.5:0/1407001772 shutdown_connections 2024-05-09T00:47:31.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.763+0000 7ff59e20f740 1 --2- 172.21.15.5:0/1407001772 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x560af0da61c0 0x560af12379b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:31.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.763+0000 7ff59e20f740 1 --2- 172.21.15.5:0/1407001772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af0e121b0 0x560af1233a10 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:31.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.763+0000 7ff59e20f740 1 --2- 172.21.15.5:0/1407001772 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560af0df14d0 0x560af11c52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:31.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.763+0000 7ff59e20f740 1 --2- 172.21.15.5:0/1407001772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560af0de92f0 0x560af11c4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:31.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.763+0000 7ff574ff9640 0 client.15318 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:47:31.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.763+0000 7ff59e20f740 1 -- 172.21.15.5:0/1407001772 >> 172.21.15.5:0/1407001772 conn(0x560af0e06300 msgr2=0x560af11b56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:31.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.763+0000 7ff59e20f740 1 -- 172.21.15.5:0/1407001772 shutdown_connections 2024-05-09T00:47:31.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.763+0000 7ff59e20f740 1 -- 172.21.15.5:0/1407001772 wait complete. 2024-05-09T00:47:31.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.763+0000 7ff59e20f740 20 client.15318 trim_cache size 0 max 16384 2024-05-09T00:47:31.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-09T00:47:31.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-09T00:47:31.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-09T00:47:31.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.764+0000 7ff597284640 1 -- 172.21.15.5:0/2671754324 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ff5780b27e0 msgr2=0x7ff5780b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:31.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.764+0000 7ff597284640 1 --2- 172.21.15.5:0/2671754324 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ff5780b27e0 0x7ff5780b4cf0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x560af11b76d0 tx=0x7ff580004a40 comp rx=0 tx=0).stop 2024-05-09T00:47:31.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.764+0000 7ff597284640 1 -- 172.21.15.5:0/2671754324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff590102b50 msgr2=0x7ff59019a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:31.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.764+0000 7ff597284640 1 --2- 172.21.15.5:0/2671754324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff590102b50 0x7ff59019a390 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff588005d00 tx=0x7ff58800dde0 comp rx=0 tx=0).stop 2024-05-09T00:47:31.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.764+0000 7ff597284640 1 -- 172.21.15.5:0/2671754324 shutdown_connections 2024-05-09T00:47:31.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.764+0000 7ff597284640 1 --2- 172.21.15.5:0/2671754324 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ff5780b27e0 0x7ff5780b4cf0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:31.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.764+0000 7ff597284640 1 --2- 172.21.15.5:0/2671754324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff590108a90 0x7ff5901a1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:31.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.764+0000 7ff597284640 1 --2- 172.21.15.5:0/2671754324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff590103d90 0x7ff59019a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:31.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.764+0000 7ff597284640 1 --2- 172.21.15.5:0/2671754324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff590102b50 0x7ff59019a390 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:31.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.764+0000 7ff597284640 1 -- 172.21.15.5:0/2671754324 >> 172.21.15.5:0/2671754324 conn(0x7ff5900fdd40 msgr2=0x7ff590104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:31.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.764+0000 7ff597284640 1 -- 172.21.15.5:0/2671754324 shutdown_connections 2024-05-09T00:47:31.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:31.764+0000 7ff597284640 1 -- 172.21.15.5:0/2671754324 wait complete. 2024-05-09T00:47:32.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:47:31 smithi005 ceph-mon[30365]: scrub summary: idle 2024-05-09T00:47:32.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:47:31 smithi131 ceph-mon[34547]: scrub summary: idle 2024-05-09T00:47:32.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:47:31 smithi157 ceph-mon[34294]: scrub summary: idle 2024-05-09T00:47:33.256 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:47:32 smithi131 ceph-mon[34547]: pgmap v407: 97 pgs: 97 active+clean; 656 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 9 op/s 2024-05-09T00:47:33.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:47:32 smithi005 ceph-mon[30365]: pgmap v407: 97 pgs: 97 active+clean; 656 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 9 op/s 2024-05-09T00:47:33.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:47:32 smithi157 ceph-mon[34294]: pgmap v407: 97 pgs: 97 active+clean; 656 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 9 op/s 2024-05-09T00:47:35.238 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:47:34 smithi157 ceph-mon[34294]: pgmap v408: 97 pgs: 97 active+clean; 657 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.0 KiB/s wr, 16 op/s 2024-05-09T00:47:35.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:47:34 smithi005 ceph-mon[30365]: pgmap v408: 97 pgs: 97 active+clean; 657 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.0 KiB/s wr, 16 op/s 2024-05-09T00:47:35.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:47:34 smithi131 ceph-mon[34547]: pgmap v408: 97 pgs: 97 active+clean; 657 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.0 KiB/s wr, 16 op/s 2024-05-09T00:47:37.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:47:36 smithi005 ceph-mon[30365]: pgmap v409: 97 pgs: 97 active+clean; 657 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-05-09T00:47:37.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:47:36 smithi131 ceph-mon[34547]: pgmap v409: 97 pgs: 97 active+clean; 657 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-05-09T00:47:37.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:47:36 smithi157 ceph-mon[34294]: pgmap v409: 97 pgs: 97 active+clean; 657 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-05-09T00:47:39.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:47:38 smithi005 ceph-mon[30365]: pgmap v410: 97 pgs: 97 active+clean; 657 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-05-09T00:47:39.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:47:38 smithi131 ceph-mon[34547]: pgmap v410: 97 pgs: 97 active+clean; 657 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-05-09T00:47:39.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:47:38 smithi157 ceph-mon[34294]: pgmap v410: 97 pgs: 97 active+clean; 657 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-05-09T00:47:41.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:47:40 smithi005 ceph-mon[30365]: pgmap v411: 97 pgs: 97 active+clean; 657 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.5 KiB/s wr, 15 op/s 2024-05-09T00:47:41.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:47:40 smithi131 ceph-mon[34547]: pgmap v411: 97 pgs: 97 active+clean; 657 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.5 KiB/s wr, 15 op/s 2024-05-09T00:47:41.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:47:40 smithi157 ceph-mon[34294]: pgmap v411: 97 pgs: 97 active+clean; 657 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.5 KiB/s wr, 15 op/s 2024-05-09T00:47:43.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:47:42 smithi005 ceph-mon[30365]: pgmap v412: 97 pgs: 97 active+clean; 657 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-05-09T00:47:43.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:47:42 smithi131 ceph-mon[34547]: pgmap v412: 97 pgs: 97 active+clean; 657 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-05-09T00:47:43.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:47:42 smithi157 ceph-mon[34294]: pgmap v412: 97 pgs: 97 active+clean; 657 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-05-09T00:47:44.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.879+0000 7f892c35e640 1 -- 172.21.15.5:0/2388160921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8924103a90 msgr2=0x7f8924103f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:44.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.879+0000 7f892c35e640 1 --2- 172.21.15.5:0/2388160921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8924103a90 0x7f8924103f40 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f8914009900 tx=0x7f891402a410 comp rx=0 tx=0).stop 2024-05-09T00:47:44.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.879+0000 7f892c35e640 1 -- 172.21.15.5:0/2388160921 shutdown_connections 2024-05-09T00:47:44.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.879+0000 7f892c35e640 1 --2- 172.21.15.5:0/2388160921 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89241045d0 0x7f892410ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:44.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.879+0000 7f892c35e640 1 --2- 172.21.15.5:0/2388160921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8924103a90 0x7f8924103f40 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:44.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.879+0000 7f892c35e640 1 --2- 172.21.15.5:0/2388160921 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89241030b0 0x7f89241034c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:44.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.879+0000 7f892c35e640 1 -- 172.21.15.5:0/2388160921 >> 172.21.15.5:0/2388160921 conn(0x7f89240fd4d0 msgr2=0x7f89240ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:44.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.879+0000 7f892c35e640 1 -- 172.21.15.5:0/2388160921 shutdown_connections 2024-05-09T00:47:44.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.879+0000 7f892c35e640 1 -- 172.21.15.5:0/2388160921 wait complete. 2024-05-09T00:47:44.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.880+0000 7f892c35e640 1 Processor -- start 2024-05-09T00:47:44.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.880+0000 7f892c35e640 1 -- start start 2024-05-09T00:47:44.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.880+0000 7f892c35e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89241030b0 0x7f8924102350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:44.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.880+0000 7f892c35e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8924103a90 0x7f892419f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:44.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.880+0000 7f892c35e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89241045d0 0x7f89241a1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:44.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.880+0000 7f892c35e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f892410e680 con 0x7f8924103a90 2024-05-09T00:47:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.880+0000 7f892c35e640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f892410e500 con 0x7f89241030b0 2024-05-09T00:47:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.880+0000 7f892c35e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f892410e800 con 0x7f89241045d0 2024-05-09T00:47:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.881+0000 7f89298f9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8924103a90 0x7f892419f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.881+0000 7f89298f9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8924103a90 0x7f892419f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48000/0 (socket says 172.21.15.5:48000) 2024-05-09T00:47:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.881+0000 7f89298f9640 1 -- 172.21.15.5:0/3408738996 learned_addr learned my addr 172.21.15.5:0/3408738996 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:47:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.881+0000 7f892a0fa640 1 --2- 172.21.15.5:0/3408738996 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89241030b0 0x7f8924102350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.881+0000 7f892a8fb640 1 --2- 172.21.15.5:0/3408738996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89241045d0 0x7f89241a1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.881+0000 7f89298f9640 1 -- 172.21.15.5:0/3408738996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89241045d0 msgr2=0x7f89241a1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.881+0000 7f89298f9640 1 --2- 172.21.15.5:0/3408738996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89241045d0 0x7f89241a1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.881+0000 7f89298f9640 1 -- 172.21.15.5:0/3408738996 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89241030b0 msgr2=0x7f8924102350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.881+0000 7f89298f9640 1 --2- 172.21.15.5:0/3408738996 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89241030b0 0x7f8924102350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.881+0000 7f89298f9640 1 -- 172.21.15.5:0/3408738996 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89241a1e50 con 0x7f8924103a90 2024-05-09T00:47:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.881+0000 7f89298f9640 1 --2- 172.21.15.5:0/3408738996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8924103a90 0x7f892419f0e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f8914006010 tx=0x7f8914031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:44.885 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.881+0000 7f891b7fe640 1 -- 172.21.15.5:0/3408738996 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f891403f030 con 0x7f8924103a90 2024-05-09T00:47:44.885 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.881+0000 7f892c35e640 1 -- 172.21.15.5:0/3408738996 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89241a20e0 con 0x7f8924103a90 2024-05-09T00:47:44.885 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.881+0000 7f891b7fe640 1 -- 172.21.15.5:0/3408738996 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8914003aa0 con 0x7f8924103a90 2024-05-09T00:47:44.885 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.881+0000 7f891b7fe640 1 -- 172.21.15.5:0/3408738996 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8914035070 con 0x7f8924103a90 2024-05-09T00:47:44.885 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.881+0000 7f892c35e640 1 -- 172.21.15.5:0/3408738996 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89241a2620 con 0x7f8924103a90 2024-05-09T00:47:44.891 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.884+0000 7f8933210740 1 Processor -- start 2024-05-09T00:47:44.891 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.885+0000 7f8933210740 1 -- start start 2024-05-09T00:47:44.891 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.885+0000 7f891b7fe640 1 -- 172.21.15.5:0/3408738996 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f8914042030 con 0x7f8924103a90 2024-05-09T00:47:44.891 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.885+0000 7f8933210740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560a17341b0 0x5560a1adb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:44.891 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.885+0000 7f8933210740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560a170b2f0 0x5560a1adbe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:44.891 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.885+0000 7f8933210740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560a17134d0 0x5560a1ad8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:44.891 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.885+0000 7f8933210740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5560a1a3c7c0 con 0x5560a17341b0 2024-05-09T00:47:44.892 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.885+0000 7f8933210740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5560a19d80e0 con 0x5560a170b2f0 2024-05-09T00:47:44.892 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.885+0000 7f8933210740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5560a1a38780 con 0x5560a17134d0 2024-05-09T00:47:44.892 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.885+0000 7f892a8fb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560a17341b0 0x5560a1adb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:44.892 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.885+0000 7f892a8fb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560a17341b0 0x5560a1adb8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48006/0 (socket says 172.21.15.5:48006) 2024-05-09T00:47:44.892 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.885+0000 7f892a8fb640 1 -- 172.21.15.5:0/955619670 learned_addr learned my addr 172.21.15.5:0/955619670 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:47:44.892 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.885+0000 7f892a0fa640 1 --2- 172.21.15.5:0/955619670 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560a170b2f0 0x5560a1adbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:44.892 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.885+0000 7f89298f9640 1 --2- 172.21.15.5:0/955619670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560a17134d0 0x5560a1ad8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:44.892 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.885+0000 7f892a8fb640 1 -- 172.21.15.5:0/955619670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560a17134d0 msgr2=0x5560a1ad8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:47:44.892 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.885+0000 7f892a8fb640 1 --2- 172.21.15.5:0/955619670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560a17134d0 0x5560a1ad8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:44.892 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.885+0000 7f892a8fb640 1 -- 172.21.15.5:0/955619670 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560a170b2f0 msgr2=0x5560a1adbe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:47:44.892 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.885+0000 7f892a8fb640 1 --2- 172.21.15.5:0/955619670 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560a170b2f0 0x5560a1adbe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:44.892 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.885+0000 7f892a8fb640 1 -- 172.21.15.5:0/955619670 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560a1a017c0 con 0x5560a17341b0 2024-05-09T00:47:44.892 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.885+0000 7f892a0fa640 1 --2- 172.21.15.5:0/955619670 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560a170b2f0 0x5560a1adbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:47:44.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.886+0000 7f892a8fb640 1 --2- 172.21.15.5:0/955619670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560a17341b0 0x5560a1adb8e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f891c009ee0 tx=0x7f891c010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79dbaf08a413f8d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:44.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.886+0000 7f891b7fe640 1 --2- 172.21.15.5:0/3408738996 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f89100b27e0 0x7f89100b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:44.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.886+0000 7f8918bfe640 1 -- 172.21.15.5:0/955619670 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f891c016070 con 0x5560a17341b0 2024-05-09T00:47:44.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.886+0000 7f8918bfe640 1 -- 172.21.15.5:0/955619670 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f891c004d20 con 0x5560a17341b0 2024-05-09T00:47:44.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.886+0000 7f8918bfe640 1 -- 172.21.15.5:0/955619670 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f891c012730 con 0x5560a17341b0 2024-05-09T00:47:44.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.886+0000 7f892a0fa640 1 --2- 172.21.15.5:0/3408738996 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f89100b27e0 0x7f89100b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:44.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.886+0000 7f891b7fe640 1 -- 172.21.15.5:0/3408738996 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f8914102930 con 0x7f8924103a90 2024-05-09T00:47:44.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.887+0000 7f8933210740 1 -- 172.21.15.5:0/955619670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560a17341b0 msgr2=0x5560a1adb8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:44.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.887+0000 7f8933210740 1 --2- 172.21.15.5:0/955619670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560a17341b0 0x5560a1adb8e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f891c009ee0 tx=0x7f891c010740 comp rx=0 tx=0).stop 2024-05-09T00:47:44.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.887+0000 7f892a0fa640 1 --2- 172.21.15.5:0/3408738996 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f89100b27e0 0x7f89100b4cf0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x5560a1ad9860 tx=0x7f8920005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:44.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.887+0000 7f8933210740 1 -- 172.21.15.5:0/955619670 shutdown_connections 2024-05-09T00:47:44.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.887+0000 7f8933210740 1 --2- 172.21.15.5:0/955619670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560a17134d0 0x5560a1ad8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:44.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.887+0000 7f8933210740 1 --2- 172.21.15.5:0/955619670 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560a170b2f0 0x5560a1adbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:44.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.887+0000 7f8933210740 1 --2- 172.21.15.5:0/955619670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560a17341b0 0x5560a1adb8e0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:44.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.887+0000 7f8933210740 1 -- 172.21.15.5:0/955619670 >> 172.21.15.5:0/955619670 conn(0x5560a1728300 msgr2=0x5560a1ad3e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:44.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.887+0000 7f8933210740 1 -- 172.21.15.5:0/955619670 shutdown_connections 2024-05-09T00:47:44.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.887+0000 7f8933210740 1 -- 172.21.15.5:0/955619670 wait complete. 2024-05-09T00:47:44.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.888+0000 7f8933210740 1 -- start start 2024-05-09T00:47:44.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.888+0000 7f8933210740 1 Processor -- start 2024-05-09T00:47:44.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.888+0000 7f8933210740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560a170b2f0 0x5560a1aea9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:44.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.888+0000 7f8933210740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560a17134d0 0x5560a1aeaef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:44.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.888+0000 7f8933210740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560a17341b0 0x5560a1af1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:44.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.888+0000 7f8933210740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5560a19c39c0 con 0x5560a17341b0 2024-05-09T00:47:44.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.888+0000 7f8933210740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5560a1ae8d90 con 0x5560a17134d0 2024-05-09T00:47:44.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.888+0000 7f8933210740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5560a1af2390 con 0x5560a170b2f0 2024-05-09T00:47:44.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.888+0000 7f8933210740 10 client.0 _ll_register_callbacks cb 0x5560a19f0020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:47:44.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.888+0000 7f892a8fb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560a17341b0 0x5560a1af1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:44.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.888+0000 7f892a8fb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560a17341b0 0x5560a1af1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48016/0 (socket says 172.21.15.5:48016) 2024-05-09T00:47:44.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.888+0000 7f892a8fb640 1 -- 172.21.15.5:0/2437502785 learned_addr learned my addr 172.21.15.5:0/2437502785 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:47:44.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.888+0000 7f892a0fa640 1 --2- 172.21.15.5:0/2437502785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560a170b2f0 0x5560a1aea9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:44.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.888+0000 7f89298f9640 1 --2- 172.21.15.5:0/2437502785 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560a17134d0 0x5560a1aeaef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:44.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.888+0000 7f892a8fb640 1 -- 172.21.15.5:0/2437502785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560a170b2f0 msgr2=0x5560a1aea9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:47:44.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.888+0000 7f892a8fb640 1 --2- 172.21.15.5:0/2437502785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560a170b2f0 0x5560a1aea9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:44.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.888+0000 7f892a8fb640 1 -- 172.21.15.5:0/2437502785 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560a17134d0 msgr2=0x5560a1aeaef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:47:44.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.888+0000 7f892a8fb640 1 --2- 172.21.15.5:0/2437502785 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560a17134d0 0x5560a1aeaef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:44.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.888+0000 7f892a8fb640 1 -- 172.21.15.5:0/2437502785 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560a19ea1f0 con 0x5560a17341b0 2024-05-09T00:47:44.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.888+0000 7f892a0fa640 1 --2- 172.21.15.5:0/2437502785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560a170b2f0 0x5560a1aea9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:47:44.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.888+0000 7f89298f9640 1 --2- 172.21.15.5:0/2437502785 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560a17134d0 0x5560a1aeaef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:47:44.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.889+0000 7f892a8fb640 1 --2- 172.21.15.5:0/2437502785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560a17341b0 0x5560a1af1e50 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f891c005f40 tx=0x7f891c005c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6223b0a8a1e2823 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.889+0000 7f890dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:47:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.889+0000 7f890dffb640 1 -- 172.21.15.5:0/2437502785 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f891c028060 con 0x5560a17341b0 2024-05-09T00:47:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.889+0000 7f890dffb640 1 -- 172.21.15.5:0/2437502785 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f891c005420 con 0x5560a17341b0 2024-05-09T00:47:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.889+0000 7f8933210740 1 -- 172.21.15.5:0/2437502785 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f891c004d20 con 0x5560a17341b0 2024-05-09T00:47:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.889+0000 7f890dffb640 1 -- 172.21.15.5:0/2437502785 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f891c016040 con 0x5560a17341b0 2024-05-09T00:47:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.889+0000 7f8933210740 10 client.15330 fetch_fsmap learned FSMap version 29 2024-05-09T00:47:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.889+0000 7f8933210740 1 -- 172.21.15.5:0/2437502785 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5560a179f790 con 0x5560a17341b0 2024-05-09T00:47:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.889+0000 7f890dffb640 1 -- 172.21.15.5:0/2437502785 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f891c004d20 con 0x5560a17341b0 2024-05-09T00:47:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.890+0000 7f890dffb640 1 -- 172.21.15.5:0/2437502785 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f891c022030 con 0x5560a17341b0 2024-05-09T00:47:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.890+0000 7f8933210740 10 client.15330 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:47:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.890+0000 7f8933210740 10 client.15330 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:47:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.890+0000 7f8933210740 20 client.15330 populate_metadata read hostname 'smithi005' 2024-05-09T00:47:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.890+0000 7f8933210740 1 --2- 172.21.15.5:0/2437502785 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5560a16f2d80 0x5560a1af81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:44.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.890+0000 7f8933210740 4 client.15330 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:47:44.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.890+0000 7f8933210740 1 -- 172.21.15.5:0/2437502785 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5560a1a4a3d0 con 0x5560a16f2d80 2024-05-09T00:47:44.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.890+0000 7f892a0fa640 1 --2- 172.21.15.5:0/2437502785 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5560a16f2d80 0x5560a1af81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:44.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.891+0000 7f890dffb640 10 client.15330 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:47:44.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.891+0000 7f892a0fa640 1 --2- 172.21.15.5:0/2437502785 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5560a16f2d80 0x5560a1af81a0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c51bede6c4d312cd server_cookie=ddf289b444969fd4 in_seq=0 out_seq=0 2024-05-09T00:47:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.891+0000 7f890dffb640 1 -- 172.21.15.5:0/2437502785 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5560a1a4a3d0 con 0x5560a16f2d80 2024-05-09T00:47:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.891+0000 7f890dffb640 10 client.15330 handle_command_reply: tid=0 2024-05-09T00:47:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.892+0000 7f8933210740 1 client.15330 shutdown 2024-05-09T00:47:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.892+0000 7f8933210740 1 -- 172.21.15.5:0/2437502785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560a17341b0 msgr2=0x5560a1af1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.892+0000 7f8933210740 1 --2- 172.21.15.5:0/2437502785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560a17341b0 0x5560a1af1e50 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f891c005f40 tx=0x7f891c005c80 comp rx=0 tx=0).stop 2024-05-09T00:47:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.892+0000 7f8933210740 1 -- 172.21.15.5:0/2437502785 shutdown_connections 2024-05-09T00:47:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.892+0000 7f8933210740 1 --2- 172.21.15.5:0/2437502785 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5560a16f2d80 0x5560a1af81a0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.892+0000 7f8933210740 1 --2- 172.21.15.5:0/2437502785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560a17341b0 0x5560a1af1e50 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.892+0000 7f8933210740 1 --2- 172.21.15.5:0/2437502785 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560a17134d0 0x5560a1aeaef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.892+0000 7f890dffb640 0 client.15330 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:47:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.892+0000 7f8933210740 1 --2- 172.21.15.5:0/2437502785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560a170b2f0 0x5560a1aea9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.892+0000 7f8933210740 1 -- 172.21.15.5:0/2437502785 >> 172.21.15.5:0/2437502785 conn(0x5560a1728300 msgr2=0x5560a1ad4d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.892+0000 7f8933210740 1 -- 172.21.15.5:0/2437502785 shutdown_connections 2024-05-09T00:47:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.892+0000 7f8933210740 1 -- 172.21.15.5:0/2437502785 wait complete. 2024-05-09T00:47:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.892+0000 7f8933210740 20 client.15330 trim_cache size 0 max 16384 2024-05-09T00:47:44.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.908+0000 7f8933210740 1 Processor -- start 2024-05-09T00:47:44.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.908+0000 7f8933210740 1 -- start start 2024-05-09T00:47:44.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.908+0000 7f8933210740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560a1728300 0x5560a1aeb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:44.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.908+0000 7f8933210740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560a16f2d80 0x5560a1afd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:44.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.908+0000 7f8933210740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560a170b2f0 0x5560a1afdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:44.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.908+0000 7f8933210740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f89241a9290 con 0x5560a1728300 2024-05-09T00:47:44.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.908+0000 7f8933210740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f89241aadd0 con 0x5560a170b2f0 2024-05-09T00:47:44.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.908+0000 7f8933210740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f89241a9960 con 0x5560a16f2d80 2024-05-09T00:47:44.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.908+0000 7f892a0fa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560a1728300 0x5560a1aeb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:44.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.908+0000 7f892a0fa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560a1728300 0x5560a1aeb0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48032/0 (socket says 172.21.15.5:48032) 2024-05-09T00:47:44.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.908+0000 7f892a0fa640 1 -- 172.21.15.5:0/2900427837 learned_addr learned my addr 172.21.15.5:0/2900427837 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:47:44.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.908+0000 7f89298f9640 1 --2- 172.21.15.5:0/2900427837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560a16f2d80 0x5560a1afd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:44.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.908+0000 7f892a8fb640 1 --2- 172.21.15.5:0/2900427837 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560a170b2f0 0x5560a1afdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:44.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.908+0000 7f892a0fa640 1 -- 172.21.15.5:0/2900427837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560a16f2d80 msgr2=0x5560a1afd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:47:44.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.908+0000 7f892a0fa640 1 --2- 172.21.15.5:0/2900427837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560a16f2d80 0x5560a1afd850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:44.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.908+0000 7f892a0fa640 1 -- 172.21.15.5:0/2900427837 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560a170b2f0 msgr2=0x5560a1afdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:47:44.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.908+0000 7f892a0fa640 1 --2- 172.21.15.5:0/2900427837 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560a170b2f0 0x5560a1afdd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:44.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.908+0000 7f892a0fa640 1 -- 172.21.15.5:0/2900427837 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560a18165f0 con 0x5560a1728300 2024-05-09T00:47:44.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.908+0000 7f892a8fb640 1 --2- 172.21.15.5:0/2900427837 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560a170b2f0 0x5560a1afdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:47:44.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.908+0000 7f89298f9640 1 --2- 172.21.15.5:0/2900427837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560a16f2d80 0x5560a1afd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:47:44.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.909+0000 7f892a0fa640 1 --2- 172.21.15.5:0/2900427837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560a1728300 0x5560a1aeb0c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f892406b650 tx=0x7f89200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=471a153be66926e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:44.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.909+0000 7f890ffff640 1 -- 172.21.15.5:0/2900427837 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8920003c40 con 0x5560a1728300 2024-05-09T00:47:44.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.909+0000 7f890ffff640 1 -- 172.21.15.5:0/2900427837 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8920030070 con 0x5560a1728300 2024-05-09T00:47:44.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.909+0000 7f890ffff640 1 -- 172.21.15.5:0/2900427837 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892001fa00 con 0x5560a1728300 2024-05-09T00:47:44.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.910+0000 7f8933210740 1 -- 172.21.15.5:0/2900427837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560a1728300 msgr2=0x5560a1aeb0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:44.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.910+0000 7f8933210740 1 --2- 172.21.15.5:0/2900427837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560a1728300 0x5560a1aeb0c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f892406b650 tx=0x7f89200046c0 comp rx=0 tx=0).stop 2024-05-09T00:47:44.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.910+0000 7f8933210740 1 -- 172.21.15.5:0/2900427837 shutdown_connections 2024-05-09T00:47:44.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.910+0000 7f8933210740 1 --2- 172.21.15.5:0/2900427837 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560a170b2f0 0x5560a1afdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:44.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.910+0000 7f8933210740 1 --2- 172.21.15.5:0/2900427837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560a16f2d80 0x5560a1afd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:44.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.910+0000 7f8933210740 1 --2- 172.21.15.5:0/2900427837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560a1728300 0x5560a1aeb0c0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:44.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.910+0000 7f8933210740 1 -- 172.21.15.5:0/2900427837 >> 172.21.15.5:0/2900427837 conn(0x5560a1ae5dc0 msgr2=0x5560a1aea9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:44.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.910+0000 7f8933210740 1 -- 172.21.15.5:0/2900427837 shutdown_connections 2024-05-09T00:47:44.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.910+0000 7f8933210740 1 -- 172.21.15.5:0/2900427837 wait complete. 2024-05-09T00:47:44.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.910+0000 7f8933210740 1 -- start start 2024-05-09T00:47:44.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.910+0000 7f8933210740 1 Processor -- start 2024-05-09T00:47:44.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.910+0000 7f8933210740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560a16f2d80 0x5560a1aef690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:44.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.910+0000 7f8933210740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560a170b2f0 0x5560a1aefbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:44.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.910+0000 7f8933210740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560a1728300 0x5560a1af0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:44.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.910+0000 7f8933210740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f89241aa700 con 0x5560a1728300 2024-05-09T00:47:44.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.910+0000 7f8933210740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f89241aa030 con 0x5560a170b2f0 2024-05-09T00:47:44.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.910+0000 7f8933210740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f89241a7270 con 0x5560a16f2d80 2024-05-09T00:47:44.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.910+0000 7f8933210740 10 client.0 _ll_register_callbacks cb 0x7f8920009590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:47:44.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.911+0000 7f892a8fb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560a1728300 0x5560a1af0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:44.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.911+0000 7f892a0fa640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560a16f2d80 0x5560a1aef690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:44.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.911+0000 7f892a8fb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560a1728300 0x5560a1af0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48044/0 (socket says 172.21.15.5:48044) 2024-05-09T00:47:44.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.911+0000 7f892a8fb640 1 -- 172.21.15.5:0/2079128765 learned_addr learned my addr 172.21.15.5:0/2079128765 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:47:44.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.911+0000 7f89298f9640 1 --2- 172.21.15.5:0/2079128765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560a170b2f0 0x5560a1aefbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:44.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.911+0000 7f892a8fb640 1 -- 172.21.15.5:0/2079128765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560a16f2d80 msgr2=0x5560a1aef690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:47:44.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.911+0000 7f892a8fb640 1 --2- 172.21.15.5:0/2079128765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560a16f2d80 0x5560a1aef690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:44.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.911+0000 7f892a8fb640 1 -- 172.21.15.5:0/2079128765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560a170b2f0 msgr2=0x5560a1aefbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:47:44.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.911+0000 7f892a8fb640 1 --2- 172.21.15.5:0/2079128765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560a170b2f0 0x5560a1aefbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:44.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.911+0000 7f892a8fb640 1 -- 172.21.15.5:0/2079128765 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560a1a00ed0 con 0x5560a1728300 2024-05-09T00:47:44.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.911+0000 7f892a0fa640 1 --2- 172.21.15.5:0/2079128765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560a16f2d80 0x5560a1aef690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:47:44.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.911+0000 7f892a8fb640 1 --2- 172.21.15.5:0/2079128765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560a1728300 0x5560a1af0990 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f891c009b10 tx=0x7f891c004e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dbfdecda87ae640 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:44.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.911+0000 7f890dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:47:44.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.911+0000 7f890dffb640 1 -- 172.21.15.5:0/2079128765 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f891c028070 con 0x5560a1728300 2024-05-09T00:47:44.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.911+0000 7f890dffb640 1 -- 172.21.15.5:0/2079128765 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f891c005220 con 0x5560a1728300 2024-05-09T00:47:44.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.911+0000 7f8933210740 1 -- 172.21.15.5:0/2079128765 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8920030070 con 0x5560a1728300 2024-05-09T00:47:44.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.912+0000 7f890dffb640 1 -- 172.21.15.5:0/2079128765 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f891c03c050 con 0x5560a1728300 2024-05-09T00:47:44.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.912+0000 7f8933210740 10 client.15336 fetch_fsmap learned FSMap version 29 2024-05-09T00:47:44.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.912+0000 7f8933210740 1 -- 172.21.15.5:0/2079128765 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5560a1b60ae0 con 0x5560a1728300 2024-05-09T00:47:44.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.912+0000 7f890dffb640 1 -- 172.21.15.5:0/2079128765 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8920030070 con 0x5560a1728300 2024-05-09T00:47:44.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.912+0000 7f890dffb640 1 -- 172.21.15.5:0/2079128765 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f891c035060 con 0x5560a1728300 2024-05-09T00:47:44.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.912+0000 7f8933210740 10 client.15336 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:47:44.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.912+0000 7f8933210740 10 client.15336 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:47:44.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.912+0000 7f8933210740 20 client.15336 populate_metadata read hostname 'smithi005' 2024-05-09T00:47:44.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.912+0000 7f8933210740 1 --2- 172.21.15.5:0/2079128765 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5560a17134d0 0x5560a1b634d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:44.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.912+0000 7f8933210740 4 client.15336 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:47:44.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.912+0000 7f8933210740 1 -- 172.21.15.5:0/2079128765 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5560a1a44240 con 0x5560a17134d0 2024-05-09T00:47:44.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.912+0000 7f892a0fa640 1 --2- 172.21.15.5:0/2079128765 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5560a17134d0 0x5560a1b634d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:44.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.913+0000 7f892a0fa640 1 --2- 172.21.15.5:0/2079128765 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5560a17134d0 0x5560a1b634d0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d97ce7cf48bda00f server_cookie=6e5e18e60df29df0 in_seq=0 out_seq=0 2024-05-09T00:47:44.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.913+0000 7f890dffb640 10 client.15336 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:47:44.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.917+0000 7f890dffb640 1 -- 172.21.15.5:0/2079128765 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5560a1a44240 con 0x5560a17134d0 2024-05-09T00:47:44.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.917+0000 7f890dffb640 10 client.15336 handle_command_reply: tid=0 2024-05-09T00:47:44.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.917+0000 7f8933210740 1 client.15336 shutdown 2024-05-09T00:47:44.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.917+0000 7f8933210740 1 -- 172.21.15.5:0/2079128765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560a1728300 msgr2=0x5560a1af0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:44.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.917+0000 7f8933210740 1 --2- 172.21.15.5:0/2079128765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560a1728300 0x5560a1af0990 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f891c009b10 tx=0x7f891c004e80 comp rx=0 tx=0).stop 2024-05-09T00:47:44.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.917+0000 7f8933210740 1 -- 172.21.15.5:0/2079128765 shutdown_connections 2024-05-09T00:47:44.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.917+0000 7f8933210740 1 --2- 172.21.15.5:0/2079128765 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5560a17134d0 0x5560a1b634d0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:44.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.917+0000 7f8933210740 1 --2- 172.21.15.5:0/2079128765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560a1728300 0x5560a1af0990 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:44.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.917+0000 7f890dffb640 0 client.15336 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:47:44.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.917+0000 7f8933210740 1 --2- 172.21.15.5:0/2079128765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5560a170b2f0 0x5560a1aefbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:44.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.917+0000 7f8933210740 1 --2- 172.21.15.5:0/2079128765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560a16f2d80 0x5560a1aef690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:44.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.917+0000 7f8933210740 1 -- 172.21.15.5:0/2079128765 >> 172.21.15.5:0/2079128765 conn(0x5560a1ae5dc0 msgr2=0x5560a1ad6c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:44.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.917+0000 7f8933210740 1 -- 172.21.15.5:0/2079128765 shutdown_connections 2024-05-09T00:47:44.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.917+0000 7f8933210740 1 -- 172.21.15.5:0/2079128765 wait complete. 2024-05-09T00:47:44.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.917+0000 7f8933210740 20 client.15336 trim_cache size 0 max 16384 2024-05-09T00:47:44.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-09T00:47:44.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-09T00:47:44.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-09T00:47:44.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.918+0000 7f892c35e640 1 -- 172.21.15.5:0/3408738996 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f89100b27e0 msgr2=0x7f89100b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:44.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.918+0000 7f892c35e640 1 --2- 172.21.15.5:0/3408738996 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f89100b27e0 0x7f89100b4cf0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x5560a1ad9860 tx=0x7f8920005f90 comp rx=0 tx=0).stop 2024-05-09T00:47:44.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.918+0000 7f892c35e640 1 -- 172.21.15.5:0/3408738996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8924103a90 msgr2=0x7f892419f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:44.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.918+0000 7f892c35e640 1 --2- 172.21.15.5:0/3408738996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8924103a90 0x7f892419f0e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f8914006010 tx=0x7f8914031a50 comp rx=0 tx=0).stop 2024-05-09T00:47:44.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.918+0000 7f892c35e640 1 -- 172.21.15.5:0/3408738996 shutdown_connections 2024-05-09T00:47:44.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.918+0000 7f892c35e640 1 --2- 172.21.15.5:0/3408738996 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f89100b27e0 0x7f89100b4cf0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:44.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.918+0000 7f892c35e640 1 --2- 172.21.15.5:0/3408738996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89241045d0 0x7f89241a1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:44.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.918+0000 7f892c35e640 1 --2- 172.21.15.5:0/3408738996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8924103a90 0x7f892419f0e0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:44.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.918+0000 7f892c35e640 1 --2- 172.21.15.5:0/3408738996 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89241030b0 0x7f8924102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:44.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.918+0000 7f892c35e640 1 -- 172.21.15.5:0/3408738996 >> 172.21.15.5:0/3408738996 conn(0x7f89240fd4d0 msgr2=0x7f8924104c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:44.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.918+0000 7f892c35e640 1 -- 172.21.15.5:0/3408738996 shutdown_connections 2024-05-09T00:47:44.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:44.918+0000 7f892c35e640 1 -- 172.21.15.5:0/3408738996 wait complete. 2024-05-09T00:47:45.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:47:45 smithi005 ceph-mon[30365]: pgmap v413: 97 pgs: 97 active+clean; 657 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.5 KiB/s wr, 16 op/s 2024-05-09T00:47:45.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:47:45 smithi131 ceph-mon[34547]: pgmap v413: 97 pgs: 97 active+clean; 657 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.5 KiB/s wr, 16 op/s 2024-05-09T00:47:45.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:47:45 smithi157 ceph-mon[34294]: pgmap v413: 97 pgs: 97 active+clean; 657 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.5 KiB/s wr, 16 op/s 2024-05-09T00:47:47.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:47:47 smithi005 ceph-mon[30365]: pgmap v414: 97 pgs: 97 active+clean; 657 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T00:47:47.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:47:47 smithi131 ceph-mon[34547]: pgmap v414: 97 pgs: 97 active+clean; 657 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T00:47:47.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:47:47 smithi157 ceph-mon[34294]: pgmap v414: 97 pgs: 97 active+clean; 657 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T00:47:49.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:47:49 smithi005 ceph-mon[30365]: pgmap v415: 97 pgs: 97 active+clean; 657 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-09T00:47:49.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:47:49 smithi131 ceph-mon[34547]: pgmap v415: 97 pgs: 97 active+clean; 657 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-09T00:47:49.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:47:49 smithi157 ceph-mon[34294]: pgmap v415: 97 pgs: 97 active+clean; 657 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-09T00:47:51.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:47:51 smithi005 ceph-mon[30365]: pgmap v416: 97 pgs: 97 active+clean; 658 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-09T00:47:51.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:47:51 smithi131 ceph-mon[34547]: pgmap v416: 97 pgs: 97 active+clean; 658 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-09T00:47:51.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:47:51 smithi157 ceph-mon[34294]: pgmap v416: 97 pgs: 97 active+clean; 658 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-09T00:47:52.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:47:52 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:47:52.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:47:52 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:47:52.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:47:52 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:47:53.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:47:53 smithi005 ceph-mon[30365]: pgmap v417: 97 pgs: 97 active+clean; 658 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T00:47:53.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:47:53 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:47:53.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:47:53 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:47:53.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:47:53 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:47:53.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:47:53 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:47:53.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:47:53 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:47:53.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:47:53 smithi131 ceph-mon[34547]: pgmap v417: 97 pgs: 97 active+clean; 658 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T00:47:53.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:47:53 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:47:53.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:47:53 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:47:53.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:47:53 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:47:53.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:47:53 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:47:53.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:47:53 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:47:53.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:47:53 smithi157 ceph-mon[34294]: pgmap v417: 97 pgs: 97 active+clean; 658 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T00:47:53.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:47:53 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:47:53.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:47:53 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:47:53.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:47:53 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:47:53.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:47:53 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:47:53.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:47:53 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:47:54.008 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T00:47:54.277 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:47:55.535 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:47:55 smithi005 ceph-mon[30365]: pgmap v418: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-05-09T00:47:55.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:47:55 smithi131 ceph-mon[34547]: pgmap v418: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-05-09T00:47:55.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:47:55 smithi157 ceph-mon[34294]: pgmap v418: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-05-09T00:47:56.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.116+0000 7ff2db5e4700 1 -- 172.21.15.5:0/2165825479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2d410e900 msgr2=0x7ff2d4110dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:56.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.116+0000 7ff2db5e4700 1 --2- 172.21.15.5:0/2165825479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2d410e900 0x7ff2d4110dc0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc00b0a0 tx=0x7ff2cc00b3b0 comp rx=0 tx=0).stop 2024-05-09T00:47:56.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.116+0000 7ff2db5e4700 1 -- 172.21.15.5:0/2165825479 shutdown_connections 2024-05-09T00:47:56.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.116+0000 7ff2db5e4700 1 --2- 172.21.15.5:0/2165825479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2d410e900 0x7ff2d4110dc0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:56.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.116+0000 7ff2db5e4700 1 --2- 172.21.15.5:0/2165825479 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff2d4105520 0x7ff2d4105930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:56.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.116+0000 7ff2db5e4700 1 --2- 172.21.15.5:0/2165825479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2d4105e70 0x7ff2d410e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:56.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.116+0000 7ff2db5e4700 1 -- 172.21.15.5:0/2165825479 >> 172.21.15.5:0/2165825479 conn(0x7ff2d40fedc0 msgr2=0x7ff2d41011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:56.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.116+0000 7ff2db5e4700 1 -- 172.21.15.5:0/2165825479 shutdown_connections 2024-05-09T00:47:56.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.116+0000 7ff2db5e4700 1 -- 172.21.15.5:0/2165825479 wait complete. 2024-05-09T00:47:56.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.116+0000 7ff2db5e4700 1 Processor -- start 2024-05-09T00:47:56.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.117+0000 7ff2db5e4700 1 -- start start 2024-05-09T00:47:56.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.117+0000 7ff2db5e4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2d4105520 0x7ff2d419ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:56.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.117+0000 7ff2db5e4700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2d4105e70 0x7ff2d419b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:56.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.117+0000 7ff2db5e4700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff2d410e900 0x7ff2d419f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:56.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.117+0000 7ff2db5e4700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d4114440 con 0x7ff2d4105520 2024-05-09T00:47:56.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.117+0000 7ff2db5e4700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d41142c0 con 0x7ff2d410e900 2024-05-09T00:47:56.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.117+0000 7ff2db5e4700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d41145c0 con 0x7ff2d4105e70 2024-05-09T00:47:56.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.117+0000 7ff2d9380700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2d4105520 0x7ff2d419ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:56.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.117+0000 7ff2d9380700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2d4105520 0x7ff2d419ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32958/0 (socket says 172.21.15.5:32958) 2024-05-09T00:47:56.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.117+0000 7ff2d9380700 1 -- 172.21.15.5:0/353997410 learned_addr learned my addr 172.21.15.5:0/353997410 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:47:56.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.117+0000 7ff2d8b7f700 1 --2- 172.21.15.5:0/353997410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2d4105e70 0x7ff2d419b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:56.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.117+0000 7ff2d9b81700 1 --2- 172.21.15.5:0/353997410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff2d410e900 0x7ff2d419f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:56.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.117+0000 7ff2d9380700 1 -- 172.21.15.5:0/353997410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2d4105e70 msgr2=0x7ff2d419b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:56.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.117+0000 7ff2d9380700 1 --2- 172.21.15.5:0/353997410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2d4105e70 0x7ff2d419b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:56.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.117+0000 7ff2d9380700 1 -- 172.21.15.5:0/353997410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff2d410e900 msgr2=0x7ff2d419f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:56.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.117+0000 7ff2d9380700 1 --2- 172.21.15.5:0/353997410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff2d410e900 0x7ff2d419f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:56.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.117+0000 7ff2d9380700 1 -- 172.21.15.5:0/353997410 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2d419fb50 con 0x7ff2d4105520 2024-05-09T00:47:56.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.117+0000 7ff2d8b7f700 1 --2- 172.21.15.5:0/353997410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2d4105e70 0x7ff2d419b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:47:56.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.118+0000 7ff2d9380700 1 --2- 172.21.15.5:0/353997410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2d4105520 0x7ff2d419ad30 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff2d000c9b0 tx=0x7ff2d000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:56.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.118+0000 7ff2ca7fc700 1 -- 172.21.15.5:0/353997410 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d0007b00 con 0x7ff2d4105520 2024-05-09T00:47:56.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.118+0000 7ff2db5e4700 1 -- 172.21.15.5:0/353997410 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2d419fe40 con 0x7ff2d4105520 2024-05-09T00:47:56.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.118+0000 7ff2ca7fc700 1 -- 172.21.15.5:0/353997410 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2d0007ca0 con 0x7ff2d4105520 2024-05-09T00:47:56.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.118+0000 7ff2ca7fc700 1 -- 172.21.15.5:0/353997410 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d0018730 con 0x7ff2d4105520 2024-05-09T00:47:56.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.118+0000 7ff2db5e4700 1 -- 172.21.15.5:0/353997410 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2d41a7810 con 0x7ff2d4105520 2024-05-09T00:47:56.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.122+0000 7ff2ca7fc700 1 -- 172.21.15.5:0/353997410 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7ff2d00188d0 con 0x7ff2d4105520 2024-05-09T00:47:56.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.123+0000 7ff2e9ffab80 1 Processor -- start 2024-05-09T00:47:56.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.123+0000 7ff2e9ffab80 1 -- start start 2024-05-09T00:47:56.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.123+0000 7ff2ca7fc700 1 --2- 172.21.15.5:0/353997410 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ff2c00b2900 0x7ff2c00b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:56.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.123+0000 7ff2e9ffab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5586f07fb160 0x5586f0892d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:56.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.123+0000 7ff2e9ffab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586f07114a0 0x5586f08932f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:56.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.123+0000 7ff2e9ffab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586f0897400 0x5586f0897810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:56.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.123+0000 7ff2e9ffab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5586f07b0ff0 con 0x5586f0897400 2024-05-09T00:47:56.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.123+0000 7ff2e9ffab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5586f074f7f0 con 0x5586f07fb160 2024-05-09T00:47:56.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.123+0000 7ff2e9ffab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5586f07acfb0 con 0x5586f07114a0 2024-05-09T00:47:56.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.123+0000 7ff2d9b81700 1 --2- 172.21.15.5:0/353997410 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ff2c00b2900 0x7ff2c00b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:56.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.123+0000 7ff2ca7fc700 1 -- 172.21.15.5:0/353997410 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7ff2d00e2300 con 0x7ff2d4105520 2024-05-09T00:47:56.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.123+0000 7ff2d8b7f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586f07114a0 0x5586f08932f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:56.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.123+0000 7ff2d9b81700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586f0897400 0x5586f0897810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:56.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.123+0000 7ff2d8b7f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586f07114a0 0x5586f08932f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:39188/0 (socket says 172.21.15.5:39188) 2024-05-09T00:47:56.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.123+0000 7ff2d8b7f700 1 -- 172.21.15.5:0/1304592386 learned_addr learned my addr 172.21.15.5:0/1304592386 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:47:56.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.124+0000 7ff2d9380700 1 --2- 172.21.15.5:0/1304592386 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5586f07fb160 0x5586f0892d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:56.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.124+0000 7ff2d8b7f700 1 -- 172.21.15.5:0/1304592386 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5586f07fb160 msgr2=0x5586f0892d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:47:56.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.124+0000 7ff2d8b7f700 1 --2- 172.21.15.5:0/1304592386 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5586f07fb160 0x5586f0892d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:56.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.124+0000 7ff2d8b7f700 1 -- 172.21.15.5:0/1304592386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586f0897400 msgr2=0x5586f0897810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:47:56.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.124+0000 7ff2d8b7f700 1 --2- 172.21.15.5:0/1304592386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586f0897400 0x5586f0897810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:56.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.124+0000 7ff2d8b7f700 1 -- 172.21.15.5:0/1304592386 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586f0747370 con 0x5586f07114a0 2024-05-09T00:47:56.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.124+0000 7ff2d9380700 1 --2- 172.21.15.5:0/1304592386 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5586f07fb160 0x5586f0892d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:47:56.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.124+0000 7ff2d9b81700 1 --2- 172.21.15.5:0/353997410 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ff2c00b2900 0x7ff2c00b4e10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc009e60 tx=0x7ff2cc009150 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:56.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.124+0000 7ff2d8b7f700 1 --2- 172.21.15.5:0/1304592386 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586f07114a0 0x5586f08932f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff2c400dc90 tx=0x7ff2c400f360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e203b1a0b8fa33dd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:56.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.124+0000 7ff2bf7fe700 1 -- 172.21.15.5:0/1304592386 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c4015070 con 0x5586f07114a0 2024-05-09T00:47:56.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.124+0000 7ff2bf7fe700 1 -- 172.21.15.5:0/1304592386 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2c4011440 con 0x5586f07114a0 2024-05-09T00:47:56.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.124+0000 7ff2bf7fe700 1 -- 172.21.15.5:0/1304592386 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c4010630 con 0x5586f07114a0 2024-05-09T00:47:56.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.126+0000 7ff2e9ffab80 1 -- 172.21.15.5:0/1304592386 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586f07114a0 msgr2=0x5586f08932f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:56.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.126+0000 7ff2e9ffab80 1 --2- 172.21.15.5:0/1304592386 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586f07114a0 0x5586f08932f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff2c400dc90 tx=0x7ff2c400f360 comp rx=0 tx=0).stop 2024-05-09T00:47:56.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.126+0000 7ff2e9ffab80 1 -- 172.21.15.5:0/1304592386 shutdown_connections 2024-05-09T00:47:56.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.126+0000 7ff2e9ffab80 1 --2- 172.21.15.5:0/1304592386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586f0897400 0x5586f0897810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:56.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.126+0000 7ff2e9ffab80 1 --2- 172.21.15.5:0/1304592386 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5586f07fb160 0x5586f0892d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:56.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.126+0000 7ff2e9ffab80 1 --2- 172.21.15.5:0/1304592386 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586f07114a0 0x5586f08932f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:56.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.126+0000 7ff2e9ffab80 1 -- 172.21.15.5:0/1304592386 >> 172.21.15.5:0/1304592386 conn(0x5586f07716a0 msgr2=0x5586f0889400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:56.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.126+0000 7ff2e9ffab80 1 -- 172.21.15.5:0/1304592386 shutdown_connections 2024-05-09T00:47:56.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.126+0000 7ff2e9ffab80 1 -- 172.21.15.5:0/1304592386 wait complete. 2024-05-09T00:47:56.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.126+0000 7ff2e9ffab80 1 -- start start 2024-05-09T00:47:56.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.127+0000 7ff2e9ffab80 1 Processor -- start 2024-05-09T00:47:56.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.127+0000 7ff2e9ffab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586f07114a0 0x5586f089c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:56.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.127+0000 7ff2e9ffab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5586f07fb160 0x5586f089cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:56.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.127+0000 7ff2e9ffab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586f0897400 0x5586f089d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:56.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.127+0000 7ff2e9ffab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5586f062cba0 con 0x5586f0897400 2024-05-09T00:47:56.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.127+0000 7ff2e9ffab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5586f076b150 con 0x5586f07fb160 2024-05-09T00:47:56.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.127+0000 7ff2e9ffab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5586f08981b0 con 0x5586f07114a0 2024-05-09T00:47:56.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.127+0000 7ff2d9b81700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586f0897400 0x5586f089d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:56.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.127+0000 7ff2d9b81700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586f0897400 0x5586f089d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32972/0 (socket says 172.21.15.5:32972) 2024-05-09T00:47:56.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.127+0000 7ff2d9b81700 1 -- 172.21.15.5:0/1944338296 learned_addr learned my addr 172.21.15.5:0/1944338296 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:47:56.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.127+0000 7ff2e9ffab80 10 client.0 _ll_register_callbacks cb 0x5586f06226c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:47:56.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.128+0000 7ff2d8b7f700 1 --2- 172.21.15.5:0/1944338296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5586f07fb160 0x5586f089cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:56.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.128+0000 7ff2d9380700 1 --2- 172.21.15.5:0/1944338296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586f07114a0 0x5586f089c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:56.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.128+0000 7ff2d9b81700 1 -- 172.21.15.5:0/1944338296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586f07114a0 msgr2=0x5586f089c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:47:56.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.128+0000 7ff2d9b81700 1 --2- 172.21.15.5:0/1944338296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586f07114a0 0x5586f089c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:56.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.128+0000 7ff2d9b81700 1 -- 172.21.15.5:0/1944338296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5586f07fb160 msgr2=0x5586f089cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:47:56.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.128+0000 7ff2d9b81700 1 --2- 172.21.15.5:0/1944338296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5586f07fb160 0x5586f089cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:56.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.128+0000 7ff2d9b81700 1 -- 172.21.15.5:0/1944338296 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586f062c490 con 0x5586f0897400 2024-05-09T00:47:56.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.128+0000 7ff2d9b81700 1 --2- 172.21.15.5:0/1944338296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586f0897400 0x5586f089d100 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc009040 tx=0x7ff2cc025000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13908b8f9f331bb3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:56.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.128+0000 7ff2bcff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:47:56.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.128+0000 7ff2bcff9700 1 -- 172.21.15.5:0/1944338296 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc032060 con 0x5586f0897400 2024-05-09T00:47:56.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.128+0000 7ff2bcff9700 1 -- 172.21.15.5:0/1944338296 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7ff2cc009360 con 0x5586f0897400 2024-05-09T00:47:56.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.128+0000 7ff2e9ffab80 1 -- 172.21.15.5:0/1944338296 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2c4011440 con 0x5586f0897400 2024-05-09T00:47:56.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.129+0000 7ff2bcff9700 1 -- 172.21.15.5:0/1944338296 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff2cc020660 con 0x5586f0897400 2024-05-09T00:47:56.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.129+0000 7ff2e9ffab80 10 client.15348 fetch_fsmap learned FSMap version 29 2024-05-09T00:47:56.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.129+0000 7ff2e9ffab80 1 -- 172.21.15.5:0/1944338296 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5586f08a9f90 con 0x5586f0897400 2024-05-09T00:47:56.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.129+0000 7ff2bcff9700 1 -- 172.21.15.5:0/1944338296 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2c4011440 con 0x5586f0897400 2024-05-09T00:47:56.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.129+0000 7ff2bcff9700 1 -- 172.21.15.5:0/1944338296 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ff2cc011070 con 0x5586f0897400 2024-05-09T00:47:56.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.129+0000 7ff2e9ffab80 10 client.15348 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:47:56.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.129+0000 7ff2e9ffab80 10 client.15348 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:47:56.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.129+0000 7ff2e9ffab80 20 client.15348 populate_metadata read hostname 'smithi005' 2024-05-09T00:47:56.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.129+0000 7ff2e9ffab80 1 --2- 172.21.15.5:0/1944338296 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5586f08ac620 0x5586f08aea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:56.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.129+0000 7ff2e9ffab80 4 client.15348 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:47:56.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.129+0000 7ff2e9ffab80 1 -- 172.21.15.5:0/1944338296 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586f07be9d0 con 0x5586f08ac620 2024-05-09T00:47:56.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.129+0000 7ff2d9380700 1 --2- 172.21.15.5:0/1944338296 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5586f08ac620 0x5586f08aea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:56.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.130+0000 7ff2d9380700 1 --2- 172.21.15.5:0/1944338296 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5586f08ac620 0x5586f08aea40 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf9f470be5687f37 server_cookie=9817c54fa0d960ae in_seq=0 out_seq=0 2024-05-09T00:47:56.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.130+0000 7ff2bcff9700 10 client.15348 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:47:56.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.131+0000 7ff2bcff9700 1 -- 172.21.15.5:0/1944338296 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5586f07be9d0 con 0x5586f08ac620 2024-05-09T00:47:56.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.131+0000 7ff2bcff9700 10 client.15348 handle_command_reply: tid=0 2024-05-09T00:47:56.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.131+0000 7ff2e9ffab80 1 client.15348 shutdown 2024-05-09T00:47:56.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.131+0000 7ff2e9ffab80 1 -- 172.21.15.5:0/1944338296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586f0897400 msgr2=0x5586f089d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:56.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.131+0000 7ff2e9ffab80 1 --2- 172.21.15.5:0/1944338296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586f0897400 0x5586f089d100 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc009040 tx=0x7ff2cc025000 comp rx=0 tx=0).stop 2024-05-09T00:47:56.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.131+0000 7ff2e9ffab80 1 -- 172.21.15.5:0/1944338296 shutdown_connections 2024-05-09T00:47:56.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.131+0000 7ff2e9ffab80 1 --2- 172.21.15.5:0/1944338296 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5586f08ac620 0x5586f08aea40 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:56.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.131+0000 7ff2e9ffab80 1 --2- 172.21.15.5:0/1944338296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586f0897400 0x5586f089d100 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:56.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.131+0000 7ff2bcff9700 0 client.15348 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:47:56.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.131+0000 7ff2e9ffab80 1 --2- 172.21.15.5:0/1944338296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586f07114a0 0x5586f089c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:56.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.131+0000 7ff2e9ffab80 1 --2- 172.21.15.5:0/1944338296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5586f07fb160 0x5586f089cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:56.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.131+0000 7ff2e9ffab80 1 -- 172.21.15.5:0/1944338296 >> 172.21.15.5:0/1944338296 conn(0x5586f07716a0 msgr2=0x5586f08891c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:56.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.131+0000 7ff2e9ffab80 1 -- 172.21.15.5:0/1944338296 shutdown_connections 2024-05-09T00:47:56.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.131+0000 7ff2e9ffab80 1 -- 172.21.15.5:0/1944338296 wait complete. 2024-05-09T00:47:56.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.131+0000 7ff2e9ffab80 20 client.15348 trim_cache size 0 max 16384 2024-05-09T00:47:56.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.176+0000 7ff2e9ffab80 1 Processor -- start 2024-05-09T00:47:56.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.176+0000 7ff2e9ffab80 1 -- start start 2024-05-09T00:47:56.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.176+0000 7ff2e9ffab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586f07716a0 0x5586f088e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:56.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.176+0000 7ff2e9ffab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5586f07114a0 0x5586f089bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:56.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.176+0000 7ff2e9ffab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586f07fb160 0x5586f08a8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:56.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.176+0000 7ff2e9ffab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d41ac9a0 con 0x5586f07fb160 2024-05-09T00:47:56.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.176+0000 7ff2e9ffab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d41ae4e0 con 0x5586f07114a0 2024-05-09T00:47:56.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.176+0000 7ff2e9ffab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d41ad070 con 0x5586f07716a0 2024-05-09T00:47:56.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.177+0000 7ff2d9b81700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586f07fb160 0x5586f08a8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:56.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.177+0000 7ff2d9380700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586f07716a0 0x5586f088e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:56.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.177+0000 7ff2d9b81700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586f07fb160 0x5586f08a8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32986/0 (socket says 172.21.15.5:32986) 2024-05-09T00:47:56.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.177+0000 7ff2d9380700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586f07716a0 0x5586f088e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:39208/0 (socket says 172.21.15.5:39208) 2024-05-09T00:47:56.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.177+0000 7ff2d9b81700 1 -- 172.21.15.5:0/2301221912 learned_addr learned my addr 172.21.15.5:0/2301221912 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:47:56.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.177+0000 7ff2d8b7f700 1 --2- 172.21.15.5:0/2301221912 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5586f07114a0 0x5586f089bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:56.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.177+0000 7ff2d9380700 1 -- 172.21.15.5:0/2301221912 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5586f07114a0 msgr2=0x5586f089bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:47:56.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.177+0000 7ff2d9380700 1 --2- 172.21.15.5:0/2301221912 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5586f07114a0 0x5586f089bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:56.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.177+0000 7ff2d9380700 1 -- 172.21.15.5:0/2301221912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586f07fb160 msgr2=0x5586f08a8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:47:56.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.177+0000 7ff2d9380700 1 --2- 172.21.15.5:0/2301221912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586f07fb160 0x5586f08a8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:56.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.177+0000 7ff2d9380700 1 -- 172.21.15.5:0/2301221912 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586f0882f60 con 0x5586f07716a0 2024-05-09T00:47:56.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.177+0000 7ff2d9b81700 1 --2- 172.21.15.5:0/2301221912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586f07fb160 0x5586f08a8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:47:56.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.177+0000 7ff2d8b7f700 1 --2- 172.21.15.5:0/2301221912 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5586f07114a0 0x5586f089bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:47:56.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.177+0000 7ff2d9380700 1 --2- 172.21.15.5:0/2301221912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586f07716a0 0x5586f088e0c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff2d00074a0 tx=0x7ff2d00dfc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cedf78742755a00 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:56.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.178+0000 7ff2beffd700 1 -- 172.21.15.5:0/2301221912 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d00a9890 con 0x5586f07716a0 2024-05-09T00:47:56.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.178+0000 7ff2beffd700 1 -- 172.21.15.5:0/2301221912 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2d00aabc0 con 0x5586f07716a0 2024-05-09T00:47:56.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.178+0000 7ff2beffd700 1 -- 172.21.15.5:0/2301221912 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d00a5b20 con 0x5586f07716a0 2024-05-09T00:47:56.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.178+0000 7ff2e9ffab80 1 -- 172.21.15.5:0/2301221912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586f07716a0 msgr2=0x5586f088e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:56.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.179+0000 7ff2e9ffab80 1 --2- 172.21.15.5:0/2301221912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586f07716a0 0x5586f088e0c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff2d00074a0 tx=0x7ff2d00dfc40 comp rx=0 tx=0).stop 2024-05-09T00:47:56.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.179+0000 7ff2e9ffab80 1 -- 172.21.15.5:0/2301221912 shutdown_connections 2024-05-09T00:47:56.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.179+0000 7ff2e9ffab80 1 --2- 172.21.15.5:0/2301221912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586f07fb160 0x5586f08a8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:56.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.179+0000 7ff2e9ffab80 1 --2- 172.21.15.5:0/2301221912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586f07716a0 0x5586f088e0c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:56.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.179+0000 7ff2e9ffab80 1 --2- 172.21.15.5:0/2301221912 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5586f07114a0 0x5586f089bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:56.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.179+0000 7ff2e9ffab80 1 -- 172.21.15.5:0/2301221912 >> 172.21.15.5:0/2301221912 conn(0x5586f088af40 msgr2=0x5586f089ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:56.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.179+0000 7ff2e9ffab80 1 -- 172.21.15.5:0/2301221912 shutdown_connections 2024-05-09T00:47:56.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.179+0000 7ff2e9ffab80 1 -- 172.21.15.5:0/2301221912 wait complete. 2024-05-09T00:47:56.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.179+0000 7ff2e9ffab80 1 -- start start 2024-05-09T00:47:56.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.179+0000 7ff2e9ffab80 1 Processor -- start 2024-05-09T00:47:56.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.180+0000 7ff2e9ffab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586f07114a0 0x5586f08a4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:56.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.180+0000 7ff2e9ffab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5586f07716a0 0x5586f0895380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:56.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.180+0000 7ff2e9ffab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586f07fb160 0x5586f08958c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:56.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.180+0000 7ff2e9ffab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d41ade10 con 0x5586f07fb160 2024-05-09T00:47:56.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.180+0000 7ff2e9ffab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d41ad740 con 0x5586f07716a0 2024-05-09T00:47:56.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.180+0000 7ff2e9ffab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d41aa940 con 0x5586f07114a0 2024-05-09T00:47:56.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.180+0000 7ff2e9ffab80 10 client.0 _ll_register_callbacks cb 0x5586f06226c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:47:56.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.180+0000 7ff2d9b81700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586f07fb160 0x5586f08958c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:56.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.180+0000 7ff2d9b81700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586f07fb160 0x5586f08958c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32990/0 (socket says 172.21.15.5:32990) 2024-05-09T00:47:56.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.180+0000 7ff2d9380700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586f07114a0 0x5586f08a4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:56.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.180+0000 7ff2d9b81700 1 -- 172.21.15.5:0/1986316224 learned_addr learned my addr 172.21.15.5:0/1986316224 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:47:56.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.180+0000 7ff2d8b7f700 1 --2- 172.21.15.5:0/1986316224 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5586f07716a0 0x5586f0895380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:56.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.180+0000 7ff2d9380700 1 -- 172.21.15.5:0/1986316224 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5586f07716a0 msgr2=0x5586f0895380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:47:56.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.180+0000 7ff2d9380700 1 --2- 172.21.15.5:0/1986316224 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5586f07716a0 0x5586f0895380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:56.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.180+0000 7ff2d9380700 1 -- 172.21.15.5:0/1986316224 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586f07fb160 msgr2=0x5586f08958c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:47:56.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.180+0000 7ff2d9380700 1 --2- 172.21.15.5:0/1986316224 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586f07fb160 0x5586f08958c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:56.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.180+0000 7ff2d9380700 1 -- 172.21.15.5:0/1986316224 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586f0895f60 con 0x5586f07114a0 2024-05-09T00:47:56.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.180+0000 7ff2d9b81700 1 --2- 172.21.15.5:0/1986316224 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586f07fb160 0x5586f08958c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:47:56.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.180+0000 7ff2d9380700 1 --2- 172.21.15.5:0/1986316224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586f07114a0 0x5586f08a4c40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x5586f088a320 tx=0x7ff2d00adc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6dc75de640af8f55 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:56.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.180+0000 7ff2bcff9700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-09T00:47:56.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.181+0000 7ff2bcff9700 1 -- 172.21.15.5:0/1986316224 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d00be070 con 0x5586f07114a0 2024-05-09T00:47:56.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.181+0000 7ff2bcff9700 1 -- 172.21.15.5:0/1986316224 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7ff2d00a5e80 con 0x5586f07114a0 2024-05-09T00:47:56.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.181+0000 7ff2e9ffab80 1 -- 172.21.15.5:0/1986316224 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2d00aabc0 con 0x5586f07114a0 2024-05-09T00:47:56.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.181+0000 7ff2bcff9700 1 -- 172.21.15.5:0/1986316224 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff2d00a9890 con 0x5586f07114a0 2024-05-09T00:47:56.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.181+0000 7ff2e9ffab80 10 client.25123 fetch_fsmap learned FSMap version 29 2024-05-09T00:47:56.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.181+0000 7ff2e9ffab80 1 -- 172.21.15.5:0/1986316224 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5586f0917de0 con 0x5586f07114a0 2024-05-09T00:47:56.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.181+0000 7ff2bcff9700 1 -- 172.21.15.5:0/1986316224 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2d00aabc0 con 0x5586f07114a0 2024-05-09T00:47:56.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.181+0000 7ff2bcff9700 1 -- 172.21.15.5:0/1986316224 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ff2d00bece0 con 0x5586f07114a0 2024-05-09T00:47:56.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.181+0000 7ff2e9ffab80 10 client.25123 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:47:56.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.181+0000 7ff2e9ffab80 10 client.25123 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:47:56.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.181+0000 7ff2e9ffab80 20 client.25123 populate_metadata read hostname 'smithi005' 2024-05-09T00:47:56.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.181+0000 7ff2e9ffab80 1 --2- 172.21.15.5:0/1986316224 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5586f0897400 0x5586f091a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:56.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.181+0000 7ff2e9ffab80 4 client.25123 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:47:56.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.181+0000 7ff2e9ffab80 1 -- 172.21.15.5:0/1986316224 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5586f0897210 con 0x5586f0897400 2024-05-09T00:47:56.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.182+0000 7ff2d8b7f700 1 --2- 172.21.15.5:0/1986316224 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5586f0897400 0x5586f091a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:56.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.182+0000 7ff2bcff9700 10 client.25123 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:47:56.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.182+0000 7ff2d8b7f700 1 --2- 172.21.15.5:0/1986316224 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5586f0897400 0x5586f091a380 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=449d5aa35adee918 server_cookie=7656f121328799d1 in_seq=0 out_seq=0 2024-05-09T00:47:56.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.182+0000 7ff2bcff9700 1 -- 172.21.15.5:0/1986316224 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5586f0897210 con 0x5586f0897400 2024-05-09T00:47:56.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.182+0000 7ff2bcff9700 10 client.25123 handle_command_reply: tid=0 2024-05-09T00:47:56.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.182+0000 7ff2e9ffab80 1 client.25123 shutdown 2024-05-09T00:47:56.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.182+0000 7ff2e9ffab80 1 -- 172.21.15.5:0/1986316224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586f07114a0 msgr2=0x5586f08a4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:56.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.182+0000 7ff2e9ffab80 1 --2- 172.21.15.5:0/1986316224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586f07114a0 0x5586f08a4c40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x5586f088a320 tx=0x7ff2d00adc40 comp rx=0 tx=0).stop 2024-05-09T00:47:56.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.183+0000 7ff2e9ffab80 1 -- 172.21.15.5:0/1986316224 shutdown_connections 2024-05-09T00:47:56.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.183+0000 7ff2e9ffab80 1 --2- 172.21.15.5:0/1986316224 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5586f0897400 0x5586f091a380 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:56.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.183+0000 7ff2e9ffab80 1 --2- 172.21.15.5:0/1986316224 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586f07fb160 0x5586f08958c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:56.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.183+0000 7ff2e9ffab80 1 --2- 172.21.15.5:0/1986316224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586f07114a0 0x5586f08a4c40 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:56.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.183+0000 7ff2e9ffab80 1 --2- 172.21.15.5:0/1986316224 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5586f07716a0 0x5586f0895380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:56.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.183+0000 7ff2e9ffab80 1 -- 172.21.15.5:0/1986316224 >> 172.21.15.5:0/1986316224 conn(0x5586f088af40 msgr2=0x5586f0898410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:56.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.183+0000 7ff2bcff9700 0 client.25123 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:47:56.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.183+0000 7ff2e9ffab80 1 -- 172.21.15.5:0/1986316224 shutdown_connections 2024-05-09T00:47:56.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.183+0000 7ff2e9ffab80 1 -- 172.21.15.5:0/1986316224 wait complete. 2024-05-09T00:47:56.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.183+0000 7ff2e9ffab80 20 client.25123 trim_cache size 0 max 16384 2024-05-09T00:47:56.187 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-09T00:47:56.188 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-05-09T00:47:56.188 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-05-09T00:47:56.188 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-09T00:47:56.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.183+0000 7ff2db5e4700 1 -- 172.21.15.5:0/353997410 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ff2c00b2900 msgr2=0x7ff2c00b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:56.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.183+0000 7ff2db5e4700 1 --2- 172.21.15.5:0/353997410 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ff2c00b2900 0x7ff2c00b4e10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc009e60 tx=0x7ff2cc009150 comp rx=0 tx=0).stop 2024-05-09T00:47:56.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.183+0000 7ff2db5e4700 1 -- 172.21.15.5:0/353997410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2d4105520 msgr2=0x7ff2d419ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:56.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.183+0000 7ff2db5e4700 1 --2- 172.21.15.5:0/353997410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2d4105520 0x7ff2d419ad30 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff2d000c9b0 tx=0x7ff2d000cd70 comp rx=0 tx=0).stop 2024-05-09T00:47:56.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.183+0000 7ff2db5e4700 1 -- 172.21.15.5:0/353997410 shutdown_connections 2024-05-09T00:47:56.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.183+0000 7ff2db5e4700 1 --2- 172.21.15.5:0/353997410 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ff2c00b2900 0x7ff2c00b4e10 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:56.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.183+0000 7ff2db5e4700 1 --2- 172.21.15.5:0/353997410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff2d410e900 0x7ff2d419f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:56.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.183+0000 7ff2db5e4700 1 --2- 172.21.15.5:0/353997410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2d4105520 0x7ff2d419ad30 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:56.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.183+0000 7ff2db5e4700 1 --2- 172.21.15.5:0/353997410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2d4105e70 0x7ff2d419b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:56.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.184+0000 7ff2db5e4700 1 -- 172.21.15.5:0/353997410 >> 172.21.15.5:0/353997410 conn(0x7ff2d40fedc0 msgr2=0x7ff2d40ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:56.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.184+0000 7ff2db5e4700 1 -- 172.21.15.5:0/353997410 shutdown_connections 2024-05-09T00:47:56.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:56.184+0000 7ff2db5e4700 1 -- 172.21.15.5:0/353997410 wait complete. 2024-05-09T00:47:56.874 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T00:47:56.874 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T00:47:57.131 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:47:57.523 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:47:57 smithi005 ceph-mon[30365]: pgmap v419: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-05-09T00:47:57.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:47:57 smithi131 ceph-mon[34547]: pgmap v419: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-05-09T00:47:57.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:47:57 smithi157 ceph-mon[34294]: pgmap v419: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-05-09T00:47:58.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.026+0000 7f9fad00c640 1 -- 172.21.15.5:0/3824471192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fa8108a80 msgr2=0x7f9fa810aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:58.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.026+0000 7f9fad00c640 1 --2- 172.21.15.5:0/3824471192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fa8108a80 0x7f9fa810aea0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f9fa0009ca0 tx=0x7f9fa002a8c0 comp rx=0 tx=0).stop 2024-05-09T00:47:58.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.026+0000 7f9fad00c640 1 -- 172.21.15.5:0/3824471192 shutdown_connections 2024-05-09T00:47:58.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.026+0000 7f9fad00c640 1 --2- 172.21.15.5:0/3824471192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fa8108a80 0x7f9fa810aea0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:58.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.026+0000 7f9fad00c640 1 --2- 172.21.15.5:0/3824471192 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9fa8103d80 0x7f9fa8104190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:58.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.026+0000 7f9fad00c640 1 --2- 172.21.15.5:0/3824471192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9fa8102b40 0x7f9fa8103840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:58.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.026+0000 7f9fad00c640 1 -- 172.21.15.5:0/3824471192 >> 172.21.15.5:0/3824471192 conn(0x7f9fa80fdd30 msgr2=0x7f9fa81001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:58.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.026+0000 7f9fad00c640 1 -- 172.21.15.5:0/3824471192 shutdown_connections 2024-05-09T00:47:58.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.026+0000 7f9fad00c640 1 -- 172.21.15.5:0/3824471192 wait complete. 2024-05-09T00:47:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.027+0000 7f9fad00c640 1 Processor -- start 2024-05-09T00:47:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.027+0000 7f9fad00c640 1 -- start start 2024-05-09T00:47:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.027+0000 7f9fad00c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9fa8102b40 0x7f9fa819a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.027+0000 7f9fad00c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fa8103d80 0x7f9fa819a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.027+0000 7f9fad00c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9fa8108a80 0x7f9fa81a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.027+0000 7f9fad00c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa810e650 con 0x7f9fa8103d80 2024-05-09T00:47:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.027+0000 7f9fad00c640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa810e4d0 con 0x7f9fa8108a80 2024-05-09T00:47:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.027+0000 7f9fad00c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa810e7d0 con 0x7f9fa8102b40 2024-05-09T00:47:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.028+0000 7f9fa659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fa8103d80 0x7f9fa819a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.028+0000 7f9fa659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fa8103d80 0x7f9fa819a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33004/0 (socket says 172.21.15.5:33004) 2024-05-09T00:47:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.028+0000 7f9fa659c640 1 -- 172.21.15.5:0/455138127 learned_addr learned my addr 172.21.15.5:0/455138127 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:47:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.028+0000 7f9fa759e640 1 --2- 172.21.15.5:0/455138127 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9fa8108a80 0x7f9fa81a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.028+0000 7f9fa6d9d640 1 --2- 172.21.15.5:0/455138127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9fa8102b40 0x7f9fa819a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.028+0000 7f9fa659c640 1 -- 172.21.15.5:0/455138127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9fa8102b40 msgr2=0x7f9fa819a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.028+0000 7f9fa659c640 1 --2- 172.21.15.5:0/455138127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9fa8102b40 0x7f9fa819a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.028+0000 7f9fa659c640 1 -- 172.21.15.5:0/455138127 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9fa8108a80 msgr2=0x7f9fa81a17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.028+0000 7f9fa659c640 1 --2- 172.21.15.5:0/455138127 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9fa8108a80 0x7f9fa81a17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.028+0000 7f9fa659c640 1 -- 172.21.15.5:0/455138127 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fa81a1e50 con 0x7f9fa8103d80 2024-05-09T00:47:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.028+0000 7f9fa759e640 1 --2- 172.21.15.5:0/455138127 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9fa8108a80 0x7f9fa81a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:47:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.028+0000 7f9fa659c640 1 --2- 172.21.15.5:0/455138127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fa8103d80 0x7f9fa819a850 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c00d950 tx=0x7f9f9c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.028+0000 7f9f8ffff640 1 -- 172.21.15.5:0/455138127 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9c017070 con 0x7f9fa8103d80 2024-05-09T00:47:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.028+0000 7f9fad00c640 1 -- 172.21.15.5:0/455138127 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fa81a2140 con 0x7f9fa8103d80 2024-05-09T00:47:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.028+0000 7f9f8ffff640 1 -- 172.21.15.5:0/455138127 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f9c009c30 con 0x7f9fa8103d80 2024-05-09T00:47:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.028+0000 7f9f8ffff640 1 -- 172.21.15.5:0/455138127 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9c00f620 con 0x7f9fa8103d80 2024-05-09T00:47:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.028+0000 7f9fad00c640 1 -- 172.21.15.5:0/455138127 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fa81a26e0 con 0x7f9fa8103d80 2024-05-09T00:47:58.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.031+0000 7f9fb3f97740 1 Processor -- start 2024-05-09T00:47:58.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.031+0000 7f9fb3f97740 1 -- start start 2024-05-09T00:47:58.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.031+0000 7f9f8ffff640 1 -- 172.21.15.5:0/455138127 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f9f9c010460 con 0x7f9fa8103d80 2024-05-09T00:47:58.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.031+0000 7f9fb3f97740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56259b4621b0 0x56259b806500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:58.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.031+0000 7f9fb3f97740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56259b4392f0 0x56259b806a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:58.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.031+0000 7f9fb3f97740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56259b4414d0 0x56259b80d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:58.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.031+0000 7f9fb3f97740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56259b76a7c0 con 0x56259b4414d0 2024-05-09T00:47:58.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.031+0000 7f9fb3f97740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x56259b7060e0 con 0x56259b4392f0 2024-05-09T00:47:58.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.031+0000 7f9fb3f97740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56259b766780 con 0x56259b4621b0 2024-05-09T00:47:58.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.031+0000 7f9fa659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56259b4414d0 0x56259b80d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:58.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.031+0000 7f9fa659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56259b4414d0 0x56259b80d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33010/0 (socket says 172.21.15.5:33010) 2024-05-09T00:47:58.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.031+0000 7f9fa659c640 1 -- 172.21.15.5:0/3422375249 learned_addr learned my addr 172.21.15.5:0/3422375249 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:47:58.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.032+0000 7f9fa6d9d640 1 --2- 172.21.15.5:0/3422375249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56259b4392f0 0x56259b806a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:58.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.032+0000 7f9fa759e640 1 --2- 172.21.15.5:0/3422375249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56259b4621b0 0x56259b806500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:58.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.032+0000 7f9fa659c640 1 -- 172.21.15.5:0/3422375249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56259b4621b0 msgr2=0x56259b806500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:47:58.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.032+0000 7f9fa659c640 1 --2- 172.21.15.5:0/3422375249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56259b4621b0 0x56259b806500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:58.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.032+0000 7f9fa659c640 1 -- 172.21.15.5:0/3422375249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56259b4392f0 msgr2=0x56259b806a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:47:58.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.032+0000 7f9fa659c640 1 --2- 172.21.15.5:0/3422375249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56259b4392f0 0x56259b806a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:58.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.032+0000 7f9fa659c640 1 -- 172.21.15.5:0/3422375249 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56259b72f7c0 con 0x56259b4414d0 2024-05-09T00:47:58.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.032+0000 7f9fa6d9d640 1 --2- 172.21.15.5:0/3422375249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56259b4392f0 0x56259b806a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:47:58.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.032+0000 7f9fa759e640 1 --2- 172.21.15.5:0/3422375249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56259b4621b0 0x56259b806500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:47:58.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.032+0000 7f9fa659c640 1 --2- 172.21.15.5:0/3422375249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56259b4414d0 0x56259b80d980 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c0ea0e0 tx=0x7f9f9c0ea5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85a95cddcd7c852c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:58.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.032+0000 7f9f8ffff640 1 --2- 172.21.15.5:0/455138127 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f9f900b27e0 0x7f9f900b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:58.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.032+0000 7f9fa759e640 1 --2- 172.21.15.5:0/455138127 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f9f900b27e0 0x7f9f900b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:58.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.032+0000 7f9f8d7fa640 1 -- 172.21.15.5:0/3422375249 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9c0f5070 con 0x56259b4414d0 2024-05-09T00:47:58.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.032+0000 7f9f8d7fa640 1 -- 172.21.15.5:0/3422375249 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f9c0a8fd0 con 0x56259b4414d0 2024-05-09T00:47:58.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.032+0000 7f9f8d7fa640 1 -- 172.21.15.5:0/3422375249 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9c09d0c0 con 0x56259b4414d0 2024-05-09T00:47:58.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.033+0000 7f9f8ffff640 1 -- 172.21.15.5:0/455138127 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f9f9c0e1ed0 con 0x7f9fa8103d80 2024-05-09T00:47:58.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.033+0000 7f9fb3f97740 1 -- 172.21.15.5:0/3422375249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56259b4414d0 msgr2=0x56259b80d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:58.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.033+0000 7f9fa759e640 1 --2- 172.21.15.5:0/455138127 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f9f900b27e0 0x7f9f900b4cf0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9fa0031b50 tx=0x7f9fa0009360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:58.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.033+0000 7f9fb3f97740 1 --2- 172.21.15.5:0/3422375249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56259b4414d0 0x56259b80d980 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c0ea0e0 tx=0x7f9f9c0ea5a0 comp rx=0 tx=0).stop 2024-05-09T00:47:58.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.033+0000 7f9fb3f97740 1 -- 172.21.15.5:0/3422375249 shutdown_connections 2024-05-09T00:47:58.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.033+0000 7f9fb3f97740 1 --2- 172.21.15.5:0/3422375249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56259b4414d0 0x56259b80d980 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:58.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.033+0000 7f9fb3f97740 1 --2- 172.21.15.5:0/3422375249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56259b4392f0 0x56259b806a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:58.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.033+0000 7f9fb3f97740 1 --2- 172.21.15.5:0/3422375249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56259b4621b0 0x56259b806500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:58.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.033+0000 7f9fb3f97740 1 -- 172.21.15.5:0/3422375249 >> 172.21.15.5:0/3422375249 conn(0x56259b456300 msgr2=0x56259b801e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:58.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.033+0000 7f9fb3f97740 1 -- 172.21.15.5:0/3422375249 shutdown_connections 2024-05-09T00:47:58.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.033+0000 7f9fb3f97740 1 -- 172.21.15.5:0/3422375249 wait complete. 2024-05-09T00:47:58.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.034+0000 7f9fb3f97740 1 -- start start 2024-05-09T00:47:58.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.034+0000 7f9fb3f97740 1 Processor -- start 2024-05-09T00:47:58.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.034+0000 7f9fb3f97740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56259b4392f0 0x56259b810230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:58.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.034+0000 7f9fb3f97740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56259b4414d0 0x56259b810770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:58.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.034+0000 7f9fb3f97740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56259b4621b0 0x56259b8176d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.034+0000 7f9fb3f97740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56259b80e3d0 con 0x56259b4414d0 2024-05-09T00:47:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.034+0000 7f9fb3f97740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x56259b6f19c0 con 0x56259b4621b0 2024-05-09T00:47:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.034+0000 7f9fb3f97740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56259b80e550 con 0x56259b4392f0 2024-05-09T00:47:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.034+0000 7f9fb3f97740 10 client.0 _ll_register_callbacks cb 0x56259b71e020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:47:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.034+0000 7f9fa659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56259b4414d0 0x56259b810770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.034+0000 7f9fa659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56259b4414d0 0x56259b810770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33016/0 (socket says 172.21.15.5:33016) 2024-05-09T00:47:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.034+0000 7f9fa659c640 1 -- 172.21.15.5:0/1439953893 learned_addr learned my addr 172.21.15.5:0/1439953893 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:47:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.035+0000 7f9fa6d9d640 1 --2- 172.21.15.5:0/1439953893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56259b4392f0 0x56259b810230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.035+0000 7f9fa659c640 1 -- 172.21.15.5:0/1439953893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56259b4392f0 msgr2=0x56259b810230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:47:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.035+0000 7f9fa659c640 1 --2- 172.21.15.5:0/1439953893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56259b4392f0 0x56259b810230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.035+0000 7f9fa659c640 1 -- 172.21.15.5:0/1439953893 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56259b4621b0 msgr2=0x56259b8176d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:47:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.035+0000 7f9fa759e640 1 --2- 172.21.15.5:0/1439953893 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56259b4621b0 0x56259b8176d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.035+0000 7f9fa659c640 1 --2- 172.21.15.5:0/1439953893 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56259b4621b0 0x56259b8176d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.035+0000 7f9fa659c640 1 -- 172.21.15.5:0/1439953893 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56259b7181f0 con 0x56259b4414d0 2024-05-09T00:47:58.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.035+0000 7f9fa759e640 1 --2- 172.21.15.5:0/1439953893 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56259b4621b0 0x56259b8176d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:47:58.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.035+0000 7f9fa659c640 1 --2- 172.21.15.5:0/1439953893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56259b4414d0 0x56259b810770 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c0a91c0 tx=0x7f9f9c00bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=585006127a3e9a08 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:58.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.035+0000 7f9f8aff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:47:58.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.035+0000 7f9f8aff5640 1 -- 172.21.15.5:0/1439953893 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9c09dbf0 con 0x56259b4414d0 2024-05-09T00:47:58.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.035+0000 7f9f8aff5640 1 -- 172.21.15.5:0/1439953893 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f9f9c0e5b00 con 0x56259b4414d0 2024-05-09T00:47:58.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.035+0000 7f9fb3f97740 1 -- 172.21.15.5:0/1439953893 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f9c0a8fd0 con 0x56259b4414d0 2024-05-09T00:47:58.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.035+0000 7f9f8aff5640 1 -- 172.21.15.5:0/1439953893 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9f9c0dfe60 con 0x56259b4414d0 2024-05-09T00:47:58.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.035+0000 7f9fb3f97740 10 client.15366 fetch_fsmap learned FSMap version 29 2024-05-09T00:47:58.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.035+0000 7f9fb3f97740 1 -- 172.21.15.5:0/1439953893 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56259b4cd790 con 0x56259b4414d0 2024-05-09T00:47:58.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.035+0000 7f9f8aff5640 1 -- 172.21.15.5:0/1439953893 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f9c0a8fd0 con 0x56259b4414d0 2024-05-09T00:47:58.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.035+0000 7f9f8aff5640 1 -- 172.21.15.5:0/1439953893 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f9f9c0a5750 con 0x56259b4414d0 2024-05-09T00:47:58.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.036+0000 7f9fb3f97740 10 client.15366 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:47:58.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.036+0000 7f9fb3f97740 10 client.15366 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:47:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.036+0000 7f9fb3f97740 20 client.15366 populate_metadata read hostname 'smithi005' 2024-05-09T00:47:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.036+0000 7f9fb3f97740 1 --2- 172.21.15.5:0/1439953893 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56259b420d80 0x56259b81d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.036+0000 7f9fb3f97740 4 client.15366 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:47:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.036+0000 7f9fb3f97740 1 -- 172.21.15.5:0/1439953893 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56259b7783d0 con 0x56259b420d80 2024-05-09T00:47:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.036+0000 7f9fa6d9d640 1 --2- 172.21.15.5:0/1439953893 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56259b420d80 0x56259b81d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.036+0000 7f9f8aff5640 10 client.15366 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:47:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.036+0000 7f9fa6d9d640 1 --2- 172.21.15.5:0/1439953893 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56259b420d80 0x56259b81d790 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9757f033bf5871d1 server_cookie=b577574f062f712a in_seq=0 out_seq=0 2024-05-09T00:47:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.037+0000 7f9f8aff5640 1 -- 172.21.15.5:0/1439953893 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56259b7783d0 con 0x56259b420d80 2024-05-09T00:47:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.037+0000 7f9f8aff5640 10 client.15366 handle_command_reply: tid=0 2024-05-09T00:47:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.037+0000 7f9fb3f97740 1 client.15366 shutdown 2024-05-09T00:47:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.037+0000 7f9fb3f97740 1 -- 172.21.15.5:0/1439953893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56259b4414d0 msgr2=0x56259b810770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.037+0000 7f9fb3f97740 1 --2- 172.21.15.5:0/1439953893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56259b4414d0 0x56259b810770 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c0a91c0 tx=0x7f9f9c00bf60 comp rx=0 tx=0).stop 2024-05-09T00:47:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.037+0000 7f9fb3f97740 1 -- 172.21.15.5:0/1439953893 shutdown_connections 2024-05-09T00:47:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.037+0000 7f9fb3f97740 1 --2- 172.21.15.5:0/1439953893 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56259b420d80 0x56259b81d790 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.037+0000 7f9fb3f97740 1 --2- 172.21.15.5:0/1439953893 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56259b4621b0 0x56259b8176d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.037+0000 7f9fb3f97740 1 --2- 172.21.15.5:0/1439953893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56259b4414d0 0x56259b810770 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.037+0000 7f9fb3f97740 1 --2- 172.21.15.5:0/1439953893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56259b4392f0 0x56259b810230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.037+0000 7f9f8aff5640 0 client.15366 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:47:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.037+0000 7f9fb3f97740 1 -- 172.21.15.5:0/1439953893 >> 172.21.15.5:0/1439953893 conn(0x56259b456300 msgr2=0x56259b804da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.037+0000 7f9fb3f97740 1 -- 172.21.15.5:0/1439953893 shutdown_connections 2024-05-09T00:47:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.037+0000 7f9fb3f97740 1 -- 172.21.15.5:0/1439953893 wait complete. 2024-05-09T00:47:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.037+0000 7f9fb3f97740 20 client.15366 trim_cache size 0 max 16384 2024-05-09T00:47:58.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.052+0000 7f9fb3f97740 1 Processor -- start 2024-05-09T00:47:58.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.052+0000 7f9fb3f97740 1 -- start start 2024-05-09T00:47:58.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.053+0000 7f9fb3f97740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56259b4392f0 0x56259b80f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:58.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.053+0000 7f9fb3f97740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56259b4414d0 0x56259b8053a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:58.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.053+0000 7f9fb3f97740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56259b4621b0 0x56259b8058e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:58.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.053+0000 7f9fb3f97740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa81ab3e0 con 0x56259b4414d0 2024-05-09T00:47:58.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.053+0000 7f9fb3f97740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa81acf20 con 0x56259b4392f0 2024-05-09T00:47:58.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.053+0000 7f9fb3f97740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa81abab0 con 0x56259b4621b0 2024-05-09T00:47:58.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.053+0000 7f9fa659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56259b4414d0 0x56259b8053a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:58.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.053+0000 7f9fa659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56259b4414d0 0x56259b8053a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33026/0 (socket says 172.21.15.5:33026) 2024-05-09T00:47:58.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.053+0000 7f9fa659c640 1 -- 172.21.15.5:0/3680934512 learned_addr learned my addr 172.21.15.5:0/3680934512 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:47:58.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.053+0000 7f9fa6d9d640 1 --2- 172.21.15.5:0/3680934512 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56259b4392f0 0x56259b80f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:58.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.053+0000 7f9fa759e640 1 --2- 172.21.15.5:0/3680934512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56259b4621b0 0x56259b8058e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:58.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.053+0000 7f9fa659c640 1 -- 172.21.15.5:0/3680934512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56259b4621b0 msgr2=0x56259b8058e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:47:58.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.053+0000 7f9fa659c640 1 --2- 172.21.15.5:0/3680934512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56259b4621b0 0x56259b8058e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:58.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.053+0000 7f9fa659c640 1 -- 172.21.15.5:0/3680934512 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56259b4392f0 msgr2=0x56259b80f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:47:58.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.053+0000 7f9fa659c640 1 --2- 172.21.15.5:0/3680934512 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56259b4392f0 0x56259b80f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:58.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.053+0000 7f9fa659c640 1 -- 172.21.15.5:0/3680934512 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56259b5445f0 con 0x56259b4414d0 2024-05-09T00:47:58.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.053+0000 7f9fa759e640 1 --2- 172.21.15.5:0/3680934512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56259b4621b0 0x56259b8058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:47:58.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.053+0000 7f9fa6d9d640 1 --2- 172.21.15.5:0/3680934512 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56259b4392f0 0x56259b80f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:47:58.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.053+0000 7f9fa659c640 1 --2- 172.21.15.5:0/3680934512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56259b4414d0 0x56259b8053a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c0a5540 tx=0x7f9f9c0ea5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5983710822d76860 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:58.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.053+0000 7f9f8cff9640 1 -- 172.21.15.5:0/3680934512 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9c0e5e60 con 0x56259b4414d0 2024-05-09T00:47:58.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.053+0000 7f9f8cff9640 1 -- 172.21.15.5:0/3680934512 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f9c0e5780 con 0x56259b4414d0 2024-05-09T00:47:58.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.053+0000 7f9f8cff9640 1 -- 172.21.15.5:0/3680934512 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9c09de00 con 0x56259b4414d0 2024-05-09T00:47:58.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.054+0000 7f9fb3f97740 1 -- 172.21.15.5:0/3680934512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56259b4414d0 msgr2=0x56259b8053a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:58.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.054+0000 7f9fb3f97740 1 --2- 172.21.15.5:0/3680934512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56259b4414d0 0x56259b8053a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c0a5540 tx=0x7f9f9c0ea5a0 comp rx=0 tx=0).stop 2024-05-09T00:47:58.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.054+0000 7f9fb3f97740 1 -- 172.21.15.5:0/3680934512 shutdown_connections 2024-05-09T00:47:58.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.054+0000 7f9fb3f97740 1 --2- 172.21.15.5:0/3680934512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56259b4621b0 0x56259b8058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:58.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.054+0000 7f9fb3f97740 1 --2- 172.21.15.5:0/3680934512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56259b4414d0 0x56259b8053a0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:58.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.054+0000 7f9fb3f97740 1 --2- 172.21.15.5:0/3680934512 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56259b4392f0 0x56259b80f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:58.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.054+0000 7f9fb3f97740 1 -- 172.21.15.5:0/3680934512 >> 172.21.15.5:0/3680934512 conn(0x56259b456300 msgr2=0x56259b804c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:58.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.054+0000 7f9fb3f97740 1 -- 172.21.15.5:0/3680934512 shutdown_connections 2024-05-09T00:47:58.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.054+0000 7f9fb3f97740 1 -- 172.21.15.5:0/3680934512 wait complete. 2024-05-09T00:47:58.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.054+0000 7f9fb3f97740 1 -- start start 2024-05-09T00:47:58.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.055+0000 7f9fb3f97740 1 Processor -- start 2024-05-09T00:47:58.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.055+0000 7f9fb3f97740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56259b4392f0 0x56259b813f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:58.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.055+0000 7f9fb3f97740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56259b4414d0 0x56259b8144c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:58.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.055+0000 7f9fb3f97740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56259b4621b0 0x56259b8153a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:58.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.055+0000 7f9fb3f97740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa81ac850 con 0x56259b4414d0 2024-05-09T00:47:58.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.055+0000 7f9fb3f97740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa81ac180 con 0x56259b4621b0 2024-05-09T00:47:58.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.055+0000 7f9fb3f97740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa81a93c0 con 0x56259b4392f0 2024-05-09T00:47:58.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.055+0000 7f9fb3f97740 10 client.0 _ll_register_callbacks cb 0x7f9f9c0f3070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:47:58.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.055+0000 7f9fa659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56259b4414d0 0x56259b8144c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:58.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.055+0000 7f9fa659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56259b4414d0 0x56259b8144c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33032/0 (socket says 172.21.15.5:33032) 2024-05-09T00:47:58.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.055+0000 7f9fa659c640 1 -- 172.21.15.5:0/3022297097 learned_addr learned my addr 172.21.15.5:0/3022297097 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:47:58.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.055+0000 7f9fa6d9d640 1 --2- 172.21.15.5:0/3022297097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56259b4392f0 0x56259b813f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:58.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.055+0000 7f9fa659c640 1 -- 172.21.15.5:0/3022297097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56259b4392f0 msgr2=0x56259b813f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:47:58.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.055+0000 7f9fa659c640 1 --2- 172.21.15.5:0/3022297097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56259b4392f0 0x56259b813f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:58.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.055+0000 7f9fa659c640 1 -- 172.21.15.5:0/3022297097 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56259b4621b0 msgr2=0x56259b8153a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:47:58.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.055+0000 7f9fa659c640 1 --2- 172.21.15.5:0/3022297097 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56259b4621b0 0x56259b8153a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:58.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.055+0000 7f9fa659c640 1 -- 172.21.15.5:0/3022297097 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56259b72eed0 con 0x56259b4414d0 2024-05-09T00:47:58.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.055+0000 7f9fa6d9d640 1 --2- 172.21.15.5:0/3022297097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56259b4392f0 0x56259b813f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:47:58.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.055+0000 7f9fa659c640 1 --2- 172.21.15.5:0/3022297097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56259b4414d0 0x56259b8144c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c0a6730 tx=0x7f9f9c0e5d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d18c82fcf9a8bb91 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:58.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.055+0000 7f9f8aff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:47:58.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.056+0000 7f9f8aff5640 1 -- 172.21.15.5:0/3022297097 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9c0e5e60 con 0x56259b4414d0 2024-05-09T00:47:58.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.056+0000 7f9f8aff5640 1 -- 172.21.15.5:0/3022297097 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f9f9c0a3be0 con 0x56259b4414d0 2024-05-09T00:47:58.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.056+0000 7f9fb3f97740 1 -- 172.21.15.5:0/3022297097 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f9c0e5780 con 0x56259b4414d0 2024-05-09T00:47:58.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.056+0000 7f9f8aff5640 1 -- 172.21.15.5:0/3022297097 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9f9c09dff0 con 0x56259b4414d0 2024-05-09T00:47:58.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.056+0000 7f9fb3f97740 10 client.15372 fetch_fsmap learned FSMap version 29 2024-05-09T00:47:58.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.056+0000 7f9fb3f97740 1 -- 172.21.15.5:0/3022297097 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56259b816a20 con 0x56259b4414d0 2024-05-09T00:47:58.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.056+0000 7f9f8aff5640 1 -- 172.21.15.5:0/3022297097 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f9c0e5780 con 0x56259b4414d0 2024-05-09T00:47:58.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.056+0000 7f9f8aff5640 1 -- 172.21.15.5:0/3022297097 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f9f9c0ca300 con 0x56259b4414d0 2024-05-09T00:47:58.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.056+0000 7f9fb3f97740 10 client.15372 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:47:58.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.056+0000 7f9fb3f97740 10 client.15372 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:47:58.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.056+0000 7f9fb3f97740 20 client.15372 populate_metadata read hostname 'smithi005' 2024-05-09T00:47:58.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.056+0000 7f9fb3f97740 1 --2- 172.21.15.5:0/3022297097 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56259b3f61c0 0x56259b8879b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:58.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.056+0000 7f9fb3f97740 4 client.15372 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:47:58.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.056+0000 7f9fb3f97740 1 -- 172.21.15.5:0/3022297097 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56259b772240 con 0x56259b3f61c0 2024-05-09T00:47:58.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.057+0000 7f9fa6d9d640 1 --2- 172.21.15.5:0/3022297097 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56259b3f61c0 0x56259b8879b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:58.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.057+0000 7f9fa6d9d640 1 --2- 172.21.15.5:0/3022297097 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56259b3f61c0 0x56259b8879b0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18f7894d62d0b42a server_cookie=9647b080a00d2c56 in_seq=0 out_seq=0 2024-05-09T00:47:58.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.057+0000 7f9f8aff5640 10 client.15372 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:47:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.065+0000 7f9f8aff5640 1 -- 172.21.15.5:0/3022297097 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56259b772240 con 0x56259b3f61c0 2024-05-09T00:47:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.065+0000 7f9f8aff5640 10 client.15372 handle_command_reply: tid=0 2024-05-09T00:47:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.065+0000 7f9fb3f97740 1 client.15372 shutdown 2024-05-09T00:47:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.065+0000 7f9fb3f97740 1 -- 172.21.15.5:0/3022297097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56259b4414d0 msgr2=0x56259b8144c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.065+0000 7f9fb3f97740 1 --2- 172.21.15.5:0/3022297097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56259b4414d0 0x56259b8144c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c0a6730 tx=0x7f9f9c0e5d10 comp rx=0 tx=0).stop 2024-05-09T00:47:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.065+0000 7f9fb3f97740 1 -- 172.21.15.5:0/3022297097 shutdown_connections 2024-05-09T00:47:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.065+0000 7f9fb3f97740 1 --2- 172.21.15.5:0/3022297097 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56259b3f61c0 0x56259b8879b0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.065+0000 7f9f8aff5640 0 client.15372 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:47:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.065+0000 7f9fb3f97740 1 --2- 172.21.15.5:0/3022297097 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56259b4621b0 0x56259b8153a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.065+0000 7f9fb3f97740 1 --2- 172.21.15.5:0/3022297097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56259b4414d0 0x56259b8144c0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.065+0000 7f9fb3f97740 1 --2- 172.21.15.5:0/3022297097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56259b4392f0 0x56259b813f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.065+0000 7f9fb3f97740 1 -- 172.21.15.5:0/3022297097 >> 172.21.15.5:0/3022297097 conn(0x56259b456300 msgr2=0x56259b8056c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.065+0000 7f9fb3f97740 1 -- 172.21.15.5:0/3022297097 shutdown_connections 2024-05-09T00:47:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.065+0000 7f9fb3f97740 1 -- 172.21.15.5:0/3022297097 wait complete. 2024-05-09T00:47:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.065+0000 7f9fb3f97740 20 client.15372 trim_cache size 0 max 16384 2024-05-09T00:47:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-09T00:47:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-09T00:47:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-09T00:47:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.066+0000 7f9fad00c640 1 -- 172.21.15.5:0/455138127 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f9f900b27e0 msgr2=0x7f9f900b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.066+0000 7f9fad00c640 1 --2- 172.21.15.5:0/455138127 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f9f900b27e0 0x7f9f900b4cf0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9fa0031b50 tx=0x7f9fa0009360 comp rx=0 tx=0).stop 2024-05-09T00:47:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.066+0000 7f9fad00c640 1 -- 172.21.15.5:0/455138127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fa8103d80 msgr2=0x7f9fa819a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.066+0000 7f9fad00c640 1 --2- 172.21.15.5:0/455138127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fa8103d80 0x7f9fa819a850 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c00d950 tx=0x7f9f9c00de10 comp rx=0 tx=0).stop 2024-05-09T00:47:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.066+0000 7f9fad00c640 1 -- 172.21.15.5:0/455138127 shutdown_connections 2024-05-09T00:47:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.066+0000 7f9fad00c640 1 --2- 172.21.15.5:0/455138127 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f9f900b27e0 0x7f9f900b4cf0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.066+0000 7f9fad00c640 1 --2- 172.21.15.5:0/455138127 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9fa8108a80 0x7f9fa81a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.066+0000 7f9fad00c640 1 --2- 172.21.15.5:0/455138127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fa8103d80 0x7f9fa819a850 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.066+0000 7f9fad00c640 1 --2- 172.21.15.5:0/455138127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9fa8102b40 0x7f9fa819a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:58.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.066+0000 7f9fad00c640 1 -- 172.21.15.5:0/455138127 >> 172.21.15.5:0/455138127 conn(0x7f9fa80fdd30 msgr2=0x7f9fa8101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:58.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.066+0000 7f9fad00c640 1 -- 172.21.15.5:0/455138127 shutdown_connections 2024-05-09T00:47:58.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:47:58.066+0000 7f9fad00c640 1 -- 172.21.15.5:0/455138127 wait complete. 2024-05-09T00:47:59.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:59.073+0000 7efc1de94700 1 -- 172.21.15.5:0/2015512105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc18108f20 msgr2=0x7efc18109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:59.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:59.073+0000 7efc1de94700 1 --2- 172.21.15.5:0/2015512105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc18108f20 0x7efc18109330 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7efc000099c0 tx=0x7efc00009cd0 comp rx=0 tx=0).stop 2024-05-09T00:47:59.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:59.074+0000 7efc1de94700 1 -- 172.21.15.5:0/2015512105 shutdown_connections 2024-05-09T00:47:59.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:59.074+0000 7efc1de94700 1 --2- 172.21.15.5:0/2015512105 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efc18109870 0x7efc181111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:59.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:59.074+0000 7efc1de94700 1 --2- 172.21.15.5:0/2015512105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc181045f0 0x7efc18104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:59.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:59.074+0000 7efc1de94700 1 --2- 172.21.15.5:0/2015512105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc18108f20 0x7efc18109330 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:59.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:59.074+0000 7efc1de94700 1 -- 172.21.15.5:0/2015512105 >> 172.21.15.5:0/2015512105 conn(0x7efc18100040 msgr2=0x7efc18102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:59.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:59.074+0000 7efc1de94700 1 -- 172.21.15.5:0/2015512105 shutdown_connections 2024-05-09T00:47:59.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:59.074+0000 7efc1de94700 1 -- 172.21.15.5:0/2015512105 wait complete. 2024-05-09T00:47:59.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:59.074+0000 7efc1de94700 1 Processor -- start 2024-05-09T00:47:59.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:59.074+0000 7efc1de94700 1 -- start start 2024-05-09T00:47:59.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:59.075+0000 7efc1de94700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc181045f0 0x7efc1819aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:59.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:59.075+0000 7efc1de94700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc18108f20 0x7efc1819b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:59.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:59.075+0000 7efc1de94700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efc18109870 0x7efc1819f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:59.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:59.075+0000 7efc1de94700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efc18114480 con 0x7efc18108f20 2024-05-09T00:47:59.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:59.075+0000 7efc1de94700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7efc18114300 con 0x7efc18109870 2024-05-09T00:47:59.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:59.075+0000 7efc1de94700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7efc18114600 con 0x7efc181045f0 2024-05-09T00:47:59.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:59.075+0000 7efc0effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc18108f20 0x7efc1819b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:59.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:59.075+0000 7efc0effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc18108f20 0x7efc1819b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33046/0 (socket says 172.21.15.5:33046) 2024-05-09T00:47:59.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:59.075+0000 7efc0effd700 1 -- 172.21.15.5:0/3916931935 learned_addr learned my addr 172.21.15.5:0/3916931935 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:47:59.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:59.075+0000 7efc177fe700 1 --2- 172.21.15.5:0/3916931935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc181045f0 0x7efc1819aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:59.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:59.075+0000 7efc17fff700 1 --2- 172.21.15.5:0/3916931935 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efc18109870 0x7efc1819f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:59.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:59.075+0000 7efc0effd700 1 -- 172.21.15.5:0/3916931935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc181045f0 msgr2=0x7efc1819aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:59.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:59.075+0000 7efc0effd700 1 --2- 172.21.15.5:0/3916931935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc181045f0 0x7efc1819aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:59.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:59.075+0000 7efc0effd700 1 -- 172.21.15.5:0/3916931935 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efc18109870 msgr2=0x7efc1819f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:59.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:59.075+0000 7efc0effd700 1 --2- 172.21.15.5:0/3916931935 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efc18109870 0x7efc1819f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:59.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:59.075+0000 7efc0effd700 1 -- 172.21.15.5:0/3916931935 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc1819fcd0 con 0x7efc18108f20 2024-05-09T00:47:59.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:59.076+0000 7efc0effd700 1 --2- 172.21.15.5:0/3916931935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc18108f20 0x7efc1819b450 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7efc00006010 tx=0x7efc000047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:59.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:59.076+0000 7efc157fa700 1 -- 172.21.15.5:0/3916931935 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc00027070 con 0x7efc18108f20 2024-05-09T00:47:59.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:59.076+0000 7efc157fa700 1 -- 172.21.15.5:0/3916931935 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc00003f20 con 0x7efc18108f20 2024-05-09T00:47:59.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:59.076+0000 7efc1de94700 1 -- 172.21.15.5:0/3916931935 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc1819ff60 con 0x7efc18108f20 2024-05-09T00:47:59.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:59.076+0000 7efc157fa700 1 -- 172.21.15.5:0/3916931935 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc00022cd0 con 0x7efc18108f20 2024-05-09T00:47:59.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:59.076+0000 7efc1de94700 1 -- 172.21.15.5:0/3916931935 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc181a0440 con 0x7efc18108f20 2024-05-09T00:47:59.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:59.077+0000 7efc1de94700 1 -- 172.21.15.5:0/3916931935 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc180512c0 con 0x7efc18108f20 2024-05-09T00:47:59.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:59.078+0000 7efc157fa700 1 -- 172.21.15.5:0/3916931935 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7efc0002a070 con 0x7efc18108f20 2024-05-09T00:47:59.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:59.079+0000 7efc157fa700 1 --2- 172.21.15.5:0/3916931935 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7efc040b2880 0x7efc040b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:47:59.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:59.079+0000 7efc157fa700 1 -- 172.21.15.5:0/3916931935 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7efc000e3580 con 0x7efc18108f20 2024-05-09T00:47:59.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:59.079+0000 7efc177fe700 1 --2- 172.21.15.5:0/3916931935 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7efc040b2880 0x7efc040b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:47:59.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:59.080+0000 7efc177fe700 1 --2- 172.21.15.5:0/3916931935 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7efc040b2880 0x7efc040b4d90 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7efc08005950 tx=0x7efc0800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:47:59.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:59.083+0000 7efc157fa700 1 -- 172.21.15.5:0/3916931935 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efc00013070 con 0x7efc18108f20 2024-05-09T00:47:59.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:59.351+0000 7efc1de94700 1 -- 172.21.15.5:0/3916931935 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efc1802e5f0 con 0x7efc18108f20 2024-05-09T00:47:59.352 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:47:59 smithi005 ceph-mon[30365]: pgmap v420: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-05-09T00:47:59.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:59.352+0000 7efc157fa700 1 -- 172.21.15.5:0/3916931935 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11926 (secure 0 0 0) 0x7efc000cbe20 con 0x7efc18108f20 2024-05-09T00:47:59.352 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:47:59.353 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2685771887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2685771887},{"type":"v1","addr":"172.21.15.5:6835","nonce":2685771887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/254017535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":254017535},{"type":"v1","addr":"172.21.15.5:6837","nonce":254017535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T00:43:45.856066+0000","modified":"2024-05-09T00:47:06.980541+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24508,"mds_1":24499,"mds_2":24494,"mds_3":24493,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14598":{"gid":14598,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2109255228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2109255228},{"type":"v1","addr":"172.21.15.5:6839","nonce":2109255228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/3809966329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3809966329},{"type":"v1","addr":"172.21.15.5:6841","nonce":3809966329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6833/3272897742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3272897742},{"type":"v1","addr":"172.21.15.131:6833","nonce":3272897742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1283093898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1283093898},{"type":"v1","addr":"172.21.15.131:6835","nonce":1283093898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/2809525455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2809525455},{"type":"v1","addr":"172.21.15.157:6833","nonce":2809525455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.131:6837/1633006186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":1633006186},{"type":"v1","addr":"172.21.15.131:6837","nonce":1633006186}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.157:6835/832898273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":832898273},{"type":"v1","addr":"172.21.15.157:6835","nonce":832898273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.131:6839/2748669631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2748669631},{"type":"v1","addr":"172.21.15.131:6839","nonce":2748669631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.157:6837/744863009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":744863009},{"type":"v1","addr":"172.21.15.157:6837","nonce":744863009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.157:6839/2779860399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":2779860399},{"type":"v1","addr":"172.21.15.157:6839","nonce":2779860399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24494,24508,24499,24467,24493]},"id":1}]} 2024-05-09T00:47:59.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:59.354+0000 7efc1de94700 1 -- 172.21.15.5:0/3916931935 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7efc040b2880 msgr2=0x7efc040b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:59.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:59.354+0000 7efc1de94700 1 --2- 172.21.15.5:0/3916931935 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7efc040b2880 0x7efc040b4d90 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7efc08005950 tx=0x7efc0800b410 comp rx=0 tx=0).stop 2024-05-09T00:47:59.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:59.354+0000 7efc1de94700 1 -- 172.21.15.5:0/3916931935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc18108f20 msgr2=0x7efc1819b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:47:59.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:59.354+0000 7efc1de94700 1 --2- 172.21.15.5:0/3916931935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc18108f20 0x7efc1819b450 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7efc00006010 tx=0x7efc000047b0 comp rx=0 tx=0).stop 2024-05-09T00:47:59.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:59.354+0000 7efc1de94700 1 -- 172.21.15.5:0/3916931935 shutdown_connections 2024-05-09T00:47:59.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:59.354+0000 7efc1de94700 1 --2- 172.21.15.5:0/3916931935 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7efc040b2880 0x7efc040b4d90 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:59.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:59.354+0000 7efc1de94700 1 --2- 172.21.15.5:0/3916931935 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efc18109870 0x7efc1819f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:59.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:59.354+0000 7efc1de94700 1 --2- 172.21.15.5:0/3916931935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc181045f0 0x7efc1819aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:59.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:59.354+0000 7efc1de94700 1 --2- 172.21.15.5:0/3916931935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc18108f20 0x7efc1819b450 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:47:59.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:59.354+0000 7efc1de94700 1 -- 172.21.15.5:0/3916931935 >> 172.21.15.5:0/3916931935 conn(0x7efc18100040 msgr2=0x7efc1810a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:47:59.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:59.354+0000 7efc1de94700 1 -- 172.21.15.5:0/3916931935 shutdown_connections 2024-05-09T00:47:59.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:47:59.354+0000 7efc1de94700 1 -- 172.21.15.5:0/3916931935 wait complete. 2024-05-09T00:47:59.356 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 29 2024-05-09T00:47:59.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:47:59 smithi131 ceph-mon[34547]: pgmap v420: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-05-09T00:47:59.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:47:59 smithi157 ceph-mon[34294]: pgmap v420: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-05-09T00:48:00.141 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-09T00:48:00.405 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:48:00.527 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:48:00 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/3916931935' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:48:00.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:48:00 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/3916931935' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:48:00.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:48:00 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/3916931935' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:48:01.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:48:01 smithi005 ceph-mon[30365]: pgmap v421: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-05-09T00:48:01.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:48:01 smithi131 ceph-mon[34547]: pgmap v421: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-05-09T00:48:01.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:48:01 smithi157 ceph-mon[34294]: pgmap v421: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-05-09T00:48:02.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.216+0000 7f6bf0f1a700 1 -- 172.21.15.5:0/4291779327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bec107e30 msgr2=0x7f6bec108240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:02.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.216+0000 7f6bf0f1a700 1 --2- 172.21.15.5:0/4291779327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bec107e30 0x7f6bec108240 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f6be00099c0 tx=0x7f6be0009cd0 comp rx=0 tx=0).stop 2024-05-09T00:48:02.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.216+0000 7f6bf0f1a700 1 -- 172.21.15.5:0/4291779327 shutdown_connections 2024-05-09T00:48:02.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.216+0000 7f6bf0f1a700 1 --2- 172.21.15.5:0/4291779327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6bec108780 0x7f6bec1100e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:02.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.216+0000 7f6bf0f1a700 1 --2- 172.21.15.5:0/4291779327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bec103550 0x7f6bec103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:02.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.216+0000 7f6bf0f1a700 1 --2- 172.21.15.5:0/4291779327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bec107e30 0x7f6bec108240 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:02.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.216+0000 7f6bf0f1a700 1 -- 172.21.15.5:0/4291779327 >> 172.21.15.5:0/4291779327 conn(0x7f6bec078e00 msgr2=0x7f6bec079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:02.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.216+0000 7f6bf0f1a700 1 -- 172.21.15.5:0/4291779327 shutdown_connections 2024-05-09T00:48:02.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.216+0000 7f6bf0f1a700 1 -- 172.21.15.5:0/4291779327 wait complete. 2024-05-09T00:48:02.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.217+0000 7f6bf0f1a700 1 Processor -- start 2024-05-09T00:48:02.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.217+0000 7f6bf0f1a700 1 -- start start 2024-05-09T00:48:02.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.218+0000 7f6bf0f1a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6bec103550 0x7f6bec19bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:02.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.218+0000 7f6bf0f1a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bec107e30 0x7f6bec19c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:02.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.218+0000 7f6bf0f1a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bec108780 0x7f6bec19c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:02.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.218+0000 7f6bf0f1a700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6bec113400 con 0x7f6bec107e30 2024-05-09T00:48:02.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.218+0000 7f6bf0f1a700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6bec113280 con 0x7f6bec103550 2024-05-09T00:48:02.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.218+0000 7f6bf0f1a700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6bec113580 con 0x7f6bec108780 2024-05-09T00:48:02.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.218+0000 7f6be9d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bec107e30 0x7f6bec19c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:02.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.218+0000 7f6be9d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bec107e30 0x7f6bec19c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33074/0 (socket says 172.21.15.5:33074) 2024-05-09T00:48:02.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.218+0000 7f6be9d9b700 1 -- 172.21.15.5:0/2363421725 learned_addr learned my addr 172.21.15.5:0/2363421725 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:02.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.218+0000 7f6bea59c700 1 --2- 172.21.15.5:0/2363421725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6bec103550 0x7f6bec19bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:02.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.218+0000 7f6bead9d700 1 --2- 172.21.15.5:0/2363421725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bec108780 0x7f6bec19c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:02.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.218+0000 7f6be9d9b700 1 -- 172.21.15.5:0/2363421725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bec108780 msgr2=0x7f6bec19c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:02.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.218+0000 7f6be9d9b700 1 --2- 172.21.15.5:0/2363421725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bec108780 0x7f6bec19c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:02.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.218+0000 7f6be9d9b700 1 -- 172.21.15.5:0/2363421725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6bec103550 msgr2=0x7f6bec19bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:02.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.218+0000 7f6be9d9b700 1 --2- 172.21.15.5:0/2363421725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6bec103550 0x7f6bec19bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:02.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.218+0000 7f6be9d9b700 1 -- 172.21.15.5:0/2363421725 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bec1a7070 con 0x7f6bec107e30 2024-05-09T00:48:02.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.219+0000 7f6be9d9b700 1 --2- 172.21.15.5:0/2363421725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bec107e30 0x7f6bec19c260 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6be0006010 tx=0x7f6be00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:02.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.219+0000 7f6bdb7fe700 1 -- 172.21.15.5:0/2363421725 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be0027070 con 0x7f6bec107e30 2024-05-09T00:48:02.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.219+0000 7f6bdb7fe700 1 -- 172.21.15.5:0/2363421725 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6be0003f20 con 0x7f6bec107e30 2024-05-09T00:48:02.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.219+0000 7f6bdb7fe700 1 -- 172.21.15.5:0/2363421725 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be0022cd0 con 0x7f6bec107e30 2024-05-09T00:48:02.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.219+0000 7f6bf0f1a700 1 -- 172.21.15.5:0/2363421725 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bec1a7300 con 0x7f6bec107e30 2024-05-09T00:48:02.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.219+0000 7f6bf0f1a700 1 -- 172.21.15.5:0/2363421725 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bec1a77b0 con 0x7f6bec107e30 2024-05-09T00:48:02.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.222+0000 7f6bdb7fe700 1 -- 172.21.15.5:0/2363421725 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f6be002a070 con 0x7f6bec107e30 2024-05-09T00:48:02.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.223+0000 7f6bff930b80 1 Processor -- start 2024-05-09T00:48:02.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.223+0000 7f6bff930b80 1 -- start start 2024-05-09T00:48:02.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.223+0000 7f6bff930b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55898e068160 0x55898e0ffd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:02.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.223+0000 7f6bff930b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55898df7e4a0 0x55898e1002f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:02.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.223+0000 7f6bff930b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55898e104400 0x55898e104810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:02.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.223+0000 7f6bff930b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55898e01dff0 con 0x55898df7e4a0 2024-05-09T00:48:02.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.223+0000 7f6bdb7fe700 1 --2- 172.21.15.5:0/2363421725 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f6bd40b2900 0x7f6bd40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:02.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.223+0000 7f6bff930b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55898dfbc7f0 con 0x55898e068160 2024-05-09T00:48:02.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.223+0000 7f6bff930b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55898e019fb0 con 0x55898e104400 2024-05-09T00:48:02.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.223+0000 7f6bea59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55898df7e4a0 0x55898e1002f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:02.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.223+0000 7f6bea59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55898df7e4a0 0x55898e1002f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33076/0 (socket says 172.21.15.5:33076) 2024-05-09T00:48:02.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.223+0000 7f6bea59c700 1 -- 172.21.15.5:0/597301479 learned_addr learned my addr 172.21.15.5:0/597301479 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:02.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.223+0000 7f6bead9d700 1 --2- 172.21.15.5:0/2363421725 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f6bd40b2900 0x7f6bd40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:02.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.224+0000 7f6bdb7fe700 1 -- 172.21.15.5:0/2363421725 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f6be00e30e0 con 0x7f6bec107e30 2024-05-09T00:48:02.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.224+0000 7f6bead9d700 1 --2- 172.21.15.5:0/597301479 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55898e068160 0x55898e0ffd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:02.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.224+0000 7f6bea59c700 1 -- 172.21.15.5:0/597301479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55898e104400 msgr2=0x55898e104810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:02.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.224+0000 7f6bea59c700 1 --2- 172.21.15.5:0/597301479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55898e104400 0x55898e104810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:02.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.224+0000 7f6bea59c700 1 -- 172.21.15.5:0/597301479 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55898e068160 msgr2=0x55898e0ffd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:02.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.224+0000 7f6bea59c700 1 --2- 172.21.15.5:0/597301479 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55898e068160 0x55898e0ffd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:02.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.224+0000 7f6bea59c700 1 -- 172.21.15.5:0/597301479 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55898dfb4370 con 0x55898df7e4a0 2024-05-09T00:48:02.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.224+0000 7f6bead9d700 1 --2- 172.21.15.5:0/2363421725 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f6bd40b2900 0x7f6bd40b4e10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6be4009fd0 tx=0x7f6be40090f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:02.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.224+0000 7f6bea59c700 1 --2- 172.21.15.5:0/597301479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55898df7e4a0 0x55898e1002f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc009af0 tx=0x7f6bdc009e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d868d1c031a5919e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:02.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.224+0000 7f6bd8ff9700 1 -- 172.21.15.5:0/597301479 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc017070 con 0x55898df7e4a0 2024-05-09T00:48:02.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.224+0000 7f6bd8ff9700 1 -- 172.21.15.5:0/597301479 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6bdc00ed80 con 0x55898df7e4a0 2024-05-09T00:48:02.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.224+0000 7f6bd8ff9700 1 -- 172.21.15.5:0/597301479 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc013e20 con 0x55898df7e4a0 2024-05-09T00:48:02.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.225+0000 7f6bff930b80 1 -- 172.21.15.5:0/597301479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55898df7e4a0 msgr2=0x55898e1002f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:02.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.225+0000 7f6bff930b80 1 --2- 172.21.15.5:0/597301479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55898df7e4a0 0x55898e1002f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc009af0 tx=0x7f6bdc009e00 comp rx=0 tx=0).stop 2024-05-09T00:48:02.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.225+0000 7f6bff930b80 1 -- 172.21.15.5:0/597301479 shutdown_connections 2024-05-09T00:48:02.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.225+0000 7f6bff930b80 1 --2- 172.21.15.5:0/597301479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55898e104400 0x55898e104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:02.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.225+0000 7f6bff930b80 1 --2- 172.21.15.5:0/597301479 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55898e068160 0x55898e0ffd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:02.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.225+0000 7f6bff930b80 1 --2- 172.21.15.5:0/597301479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55898df7e4a0 0x55898e1002f0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:02.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.225+0000 7f6bff930b80 1 -- 172.21.15.5:0/597301479 >> 172.21.15.5:0/597301479 conn(0x55898dfde6a0 msgr2=0x55898e0f6400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:02.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.225+0000 7f6bff930b80 1 -- 172.21.15.5:0/597301479 shutdown_connections 2024-05-09T00:48:02.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.226+0000 7f6bff930b80 1 -- 172.21.15.5:0/597301479 wait complete. 2024-05-09T00:48:02.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.226+0000 7f6bff930b80 1 -- start start 2024-05-09T00:48:02.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.226+0000 7f6bff930b80 1 Processor -- start 2024-05-09T00:48:02.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.226+0000 7f6bff930b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55898df7e4a0 0x55898e1096f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:02.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.226+0000 7f6bff930b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55898e068160 0x55898e109c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:02.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.226+0000 7f6bff930b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55898e104400 0x55898e10a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:02.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.226+0000 7f6bff930b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55898de99ba0 con 0x55898e068160 2024-05-09T00:48:02.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.226+0000 7f6bff930b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55898dfd8150 con 0x55898df7e4a0 2024-05-09T00:48:02.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.226+0000 7f6bff930b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55898e105200 con 0x55898e104400 2024-05-09T00:48:02.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.226+0000 7f6bff930b80 10 client.0 _ll_register_callbacks cb 0x55898de8f6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:48:02.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.227+0000 7f6be9d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55898e068160 0x55898e109c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:02.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.227+0000 7f6be9d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55898e068160 0x55898e109c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33090/0 (socket says 172.21.15.5:33090) 2024-05-09T00:48:02.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.227+0000 7f6be9d9b700 1 -- 172.21.15.5:0/2048125902 learned_addr learned my addr 172.21.15.5:0/2048125902 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:02.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.227+0000 7f6bead9d700 1 --2- 172.21.15.5:0/2048125902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55898e104400 0x55898e10a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:02.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.227+0000 7f6bea59c700 1 --2- 172.21.15.5:0/2048125902 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55898df7e4a0 0x55898e1096f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:02.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.227+0000 7f6be9d9b700 1 -- 172.21.15.5:0/2048125902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55898e104400 msgr2=0x55898e10a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:02.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.227+0000 7f6be9d9b700 1 --2- 172.21.15.5:0/2048125902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55898e104400 0x55898e10a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:02.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.227+0000 7f6be9d9b700 1 -- 172.21.15.5:0/2048125902 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55898df7e4a0 msgr2=0x55898e1096f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:02.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.227+0000 7f6be9d9b700 1 --2- 172.21.15.5:0/2048125902 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55898df7e4a0 0x55898e1096f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:02.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.227+0000 7f6be9d9b700 1 -- 172.21.15.5:0/2048125902 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55898de99490 con 0x55898e068160 2024-05-09T00:48:02.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.227+0000 7f6bea59c700 1 --2- 172.21.15.5:0/2048125902 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55898df7e4a0 0x55898e1096f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:48:02.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.227+0000 7f6be9d9b700 1 --2- 172.21.15.5:0/2048125902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55898e068160 0x55898e109c30 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f6be00aa6f0 tx=0x7f6be00a9ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb9f43555b846ac server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:02.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.227+0000 7f6bd1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:48:02.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.229+0000 7f6bd1ffb700 1 -- 172.21.15.5:0/2048125902 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be00095c0 con 0x55898e068160 2024-05-09T00:48:02.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.229+0000 7f6bd1ffb700 1 -- 172.21.15.5:0/2048125902 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f6be00a6d50 con 0x55898e068160 2024-05-09T00:48:02.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.229+0000 7f6bff930b80 1 -- 172.21.15.5:0/2048125902 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6bdc00ed80 con 0x55898e068160 2024-05-09T00:48:02.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.229+0000 7f6bd1ffb700 1 -- 172.21.15.5:0/2048125902 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6be00aa270 con 0x55898e068160 2024-05-09T00:48:02.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.229+0000 7f6bff930b80 10 client.15390 fetch_fsmap learned FSMap version 29 2024-05-09T00:48:02.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.229+0000 7f6bff930b80 1 -- 172.21.15.5:0/2048125902 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55898e116fd0 con 0x55898e068160 2024-05-09T00:48:02.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.229+0000 7f6bd1ffb700 1 -- 172.21.15.5:0/2048125902 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6bdc00ed80 con 0x55898e068160 2024-05-09T00:48:02.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.229+0000 7f6bd1ffb700 1 -- 172.21.15.5:0/2048125902 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f6be00920b0 con 0x55898e068160 2024-05-09T00:48:02.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.229+0000 7f6bff930b80 10 client.15390 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:48:02.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.230+0000 7f6bff930b80 10 client.15390 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T00:48:02.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.230+0000 7f6bff930b80 20 client.15390 populate_metadata read hostname 'smithi005' 2024-05-09T00:48:02.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.230+0000 7f6bff930b80 1 --2- 172.21.15.5:0/2048125902 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x55898e1194a0 0x55898e11b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:02.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.230+0000 7f6bff930b80 4 client.15390 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:48:02.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.230+0000 7f6bff930b80 1 -- 172.21.15.5:0/2048125902 --> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55898e02b9d0 con 0x55898e1194a0 2024-05-09T00:48:02.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.230+0000 7f6bea59c700 1 --2- 172.21.15.5:0/2048125902 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x55898e1194a0 0x55898e11b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:02.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.230+0000 7f6bea59c700 1 --2- 172.21.15.5:0/2048125902 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x55898e1194a0 0x55898e11b8c0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=57e29d625e0e125 server_cookie=eaee7bb572416615 in_seq=0 out_seq=0 2024-05-09T00:48:02.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.230+0000 7f6bd1ffb700 10 client.15390 ms_handle_connect on v2:172.21.15.131:6836/1633006186 2024-05-09T00:48:02.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.231+0000 7f6bd1ffb700 1 -- 172.21.15.5:0/2048125902 <== mds.4 v2:172.21.15.131:6836/1633006186 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55898e02b9d0 con 0x55898e1194a0 2024-05-09T00:48:02.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.231+0000 7f6bd1ffb700 10 client.15390 handle_command_reply: tid=0 2024-05-09T00:48:02.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.231+0000 7f6bff930b80 1 client.15390 shutdown 2024-05-09T00:48:02.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.231+0000 7f6bff930b80 1 -- 172.21.15.5:0/2048125902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55898e068160 msgr2=0x55898e109c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:02.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.231+0000 7f6bff930b80 1 --2- 172.21.15.5:0/2048125902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55898e068160 0x55898e109c30 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f6be00aa6f0 tx=0x7f6be00a9ed0 comp rx=0 tx=0).stop 2024-05-09T00:48:02.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.231+0000 7f6bff930b80 1 -- 172.21.15.5:0/2048125902 shutdown_connections 2024-05-09T00:48:02.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.231+0000 7f6bff930b80 1 --2- 172.21.15.5:0/2048125902 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x55898e1194a0 0x55898e11b8c0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:02.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.231+0000 7f6bd1ffb700 0 client.15390 ms_handle_reset on v2:172.21.15.131:6836/1633006186 2024-05-09T00:48:02.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.231+0000 7f6bff930b80 1 --2- 172.21.15.5:0/2048125902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55898e104400 0x55898e10a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:02.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.231+0000 7f6bff930b80 1 --2- 172.21.15.5:0/2048125902 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55898df7e4a0 0x55898e1096f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:02.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.231+0000 7f6bff930b80 1 --2- 172.21.15.5:0/2048125902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55898e068160 0x55898e109c30 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:02.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.231+0000 7f6bff930b80 1 -- 172.21.15.5:0/2048125902 >> 172.21.15.5:0/2048125902 conn(0x55898dfde6a0 msgr2=0x55898e0f61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:02.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.232+0000 7f6bff930b80 1 -- 172.21.15.5:0/2048125902 shutdown_connections 2024-05-09T00:48:02.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.232+0000 7f6bff930b80 1 -- 172.21.15.5:0/2048125902 wait complete. 2024-05-09T00:48:02.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.232+0000 7f6bff930b80 20 client.15390 trim_cache size 0 max 16384 2024-05-09T00:48:02.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.248+0000 7f6bff930b80 1 Processor -- start 2024-05-09T00:48:02.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.248+0000 7f6bff930b80 1 -- start start 2024-05-09T00:48:02.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.248+0000 7f6bff930b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55898df7e4a0 0x55898e108a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:02.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.248+0000 7f6bff930b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55898e068160 0x55898e109fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:02.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.248+0000 7f6bff930b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55898e104400 0x55898e0fada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:02.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.249+0000 7f6bff930b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6bec1aca50 con 0x55898e068160 2024-05-09T00:48:02.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.249+0000 7f6bff930b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6bec1ae590 con 0x55898e104400 2024-05-09T00:48:02.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.249+0000 7f6bff930b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6bec1ad120 con 0x55898df7e4a0 2024-05-09T00:48:02.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.249+0000 7f6be9d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55898e068160 0x55898e109fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:02.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.249+0000 7f6be9d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55898e068160 0x55898e109fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33100/0 (socket says 172.21.15.5:33100) 2024-05-09T00:48:02.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.249+0000 7f6be9d9b700 1 -- 172.21.15.5:0/1118086991 learned_addr learned my addr 172.21.15.5:0/1118086991 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:02.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.249+0000 7f6bea59c700 1 --2- 172.21.15.5:0/1118086991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55898df7e4a0 0x55898e108a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:02.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.249+0000 7f6be9d9b700 1 -- 172.21.15.5:0/1118086991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55898df7e4a0 msgr2=0x55898e108a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:02.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.249+0000 7f6be9d9b700 1 --2- 172.21.15.5:0/1118086991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55898df7e4a0 0x55898e108a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:02.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.249+0000 7f6be9d9b700 1 -- 172.21.15.5:0/1118086991 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55898e104400 msgr2=0x55898e0fada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:02.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.249+0000 7f6be9d9b700 1 --2- 172.21.15.5:0/1118086991 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55898e104400 0x55898e0fada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:02.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.249+0000 7f6be9d9b700 1 -- 172.21.15.5:0/1118086991 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55898e0f7ec0 con 0x55898e068160 2024-05-09T00:48:02.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.249+0000 7f6bea59c700 1 --2- 172.21.15.5:0/1118086991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55898df7e4a0 0x55898e108a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:48:02.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.249+0000 7f6be9d9b700 1 --2- 172.21.15.5:0/1118086991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55898e068160 0x55898e109fb0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f6be00a67e0 tx=0x7f6be00e3490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f4a22129ade04a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:02.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.249+0000 7f6bd3fff700 1 -- 172.21.15.5:0/1118086991 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be00a6620 con 0x55898e068160 2024-05-09T00:48:02.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.249+0000 7f6bd3fff700 1 -- 172.21.15.5:0/1118086991 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6be00095c0 con 0x55898e068160 2024-05-09T00:48:02.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.249+0000 7f6bd3fff700 1 -- 172.21.15.5:0/1118086991 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be00aeb10 con 0x55898e068160 2024-05-09T00:48:02.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.250+0000 7f6bff930b80 1 -- 172.21.15.5:0/1118086991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55898e068160 msgr2=0x55898e109fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:02.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.250+0000 7f6bff930b80 1 --2- 172.21.15.5:0/1118086991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55898e068160 0x55898e109fb0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f6be00a67e0 tx=0x7f6be00e3490 comp rx=0 tx=0).stop 2024-05-09T00:48:02.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.250+0000 7f6bff930b80 1 -- 172.21.15.5:0/1118086991 shutdown_connections 2024-05-09T00:48:02.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.250+0000 7f6bff930b80 1 --2- 172.21.15.5:0/1118086991 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55898e104400 0x55898e0fada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:02.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.250+0000 7f6bff930b80 1 --2- 172.21.15.5:0/1118086991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55898df7e4a0 0x55898e108a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:02.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.250+0000 7f6bff930b80 1 --2- 172.21.15.5:0/1118086991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55898e068160 0x55898e109fb0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:02.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.250+0000 7f6bff930b80 1 -- 172.21.15.5:0/1118086991 >> 172.21.15.5:0/1118086991 conn(0x55898dfde6a0 msgr2=0x55898e0fa690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:02.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.250+0000 7f6bff930b80 1 -- 172.21.15.5:0/1118086991 shutdown_connections 2024-05-09T00:48:02.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.250+0000 7f6bff930b80 1 -- 172.21.15.5:0/1118086991 wait complete. 2024-05-09T00:48:02.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.251+0000 7f6bff930b80 1 -- start start 2024-05-09T00:48:02.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.251+0000 7f6bff930b80 1 Processor -- start 2024-05-09T00:48:02.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.251+0000 7f6bff930b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55898df7e4a0 0x55898e111c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:02.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.251+0000 7f6bff930b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55898e068160 0x55898e10cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:02.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.251+0000 7f6bff930b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55898e104400 0x55898e10d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:02.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.251+0000 7f6bff930b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6bec1adec0 con 0x55898e104400 2024-05-09T00:48:02.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.251+0000 7f6bff930b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6bec1ad7f0 con 0x55898e068160 2024-05-09T00:48:02.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.251+0000 7f6bff930b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6bec1aaa10 con 0x55898df7e4a0 2024-05-09T00:48:02.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.251+0000 7f6bff930b80 10 client.0 _ll_register_callbacks cb 0x55898de8f6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:48:02.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.252+0000 7f6bead9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55898e104400 0x55898e10d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:02.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.252+0000 7f6bead9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55898e104400 0x55898e10d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33106/0 (socket says 172.21.15.5:33106) 2024-05-09T00:48:02.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.252+0000 7f6bead9d700 1 -- 172.21.15.5:0/2673339591 learned_addr learned my addr 172.21.15.5:0/2673339591 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:02.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.252+0000 7f6bea59c700 1 --2- 172.21.15.5:0/2673339591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55898df7e4a0 0x55898e111c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:02.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.252+0000 7f6be9d9b700 1 --2- 172.21.15.5:0/2673339591 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55898e068160 0x55898e10cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:02.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.252+0000 7f6bead9d700 1 -- 172.21.15.5:0/2673339591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55898df7e4a0 msgr2=0x55898e111c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:02.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.252+0000 7f6bead9d700 1 --2- 172.21.15.5:0/2673339591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55898df7e4a0 0x55898e111c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:02.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.252+0000 7f6bead9d700 1 -- 172.21.15.5:0/2673339591 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55898e068160 msgr2=0x55898e10cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:02.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.252+0000 7f6bead9d700 1 --2- 172.21.15.5:0/2673339591 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55898e068160 0x55898e10cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:02.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.252+0000 7f6bead9d700 1 -- 172.21.15.5:0/2673339591 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55898e0eff60 con 0x55898e104400 2024-05-09T00:48:02.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.252+0000 7f6bead9d700 1 --2- 172.21.15.5:0/2673339591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55898e104400 0x55898e10d4f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f6bec19cd00 tx=0x7f6be4009c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eb4cf0046337592 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:02.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.252+0000 7f6bd1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:48:02.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.252+0000 7f6bd1ffb700 1 -- 172.21.15.5:0/2673339591 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be402a030 con 0x55898e104400 2024-05-09T00:48:02.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.252+0000 7f6bff930b80 1 -- 172.21.15.5:0/2673339591 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55898e025a70 con 0x55898e104400 2024-05-09T00:48:02.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.252+0000 7f6bd1ffb700 1 -- 172.21.15.5:0/2673339591 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f6be4012040 con 0x55898e104400 2024-05-09T00:48:02.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.253+0000 7f6bd1ffb700 1 -- 172.21.15.5:0/2673339591 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6be401aa10 con 0x55898e104400 2024-05-09T00:48:02.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.253+0000 7f6bff930b80 10 client.15396 fetch_fsmap learned FSMap version 29 2024-05-09T00:48:02.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.253+0000 7f6bff930b80 1 -- 172.21.15.5:0/2673339591 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55898e183890 con 0x55898e104400 2024-05-09T00:48:02.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.253+0000 7f6bd1ffb700 1 -- 172.21.15.5:0/2673339591 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55898e025a70 con 0x55898e104400 2024-05-09T00:48:02.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.253+0000 7f6bd1ffb700 1 -- 172.21.15.5:0/2673339591 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f6be4026c20 con 0x55898e104400 2024-05-09T00:48:02.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.253+0000 7f6bff930b80 10 client.15396 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:48:02.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.253+0000 7f6bff930b80 10 client.15396 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T00:48:02.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.253+0000 7f6bff930b80 20 client.15396 populate_metadata read hostname 'smithi005' 2024-05-09T00:48:02.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.253+0000 7f6bff930b80 1 --2- 172.21.15.5:0/2673339591 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x55898e1194a0 0x55898e186030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:02.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.253+0000 7f6bff930b80 4 client.15396 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:48:02.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.253+0000 7f6bff930b80 1 -- 172.21.15.5:0/2673339591 --> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55898de99970 con 0x55898e1194a0 2024-05-09T00:48:02.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.253+0000 7f6bea59c700 1 --2- 172.21.15.5:0/2673339591 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x55898e1194a0 0x55898e186030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:02.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.254+0000 7f6bea59c700 1 --2- 172.21.15.5:0/2673339591 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x55898e1194a0 0x55898e186030 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a7a644398afbde47 server_cookie=e379807d6de2d699 in_seq=0 out_seq=0 2024-05-09T00:48:02.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.254+0000 7f6bd1ffb700 10 client.15396 ms_handle_connect on v2:172.21.15.131:6836/1633006186 2024-05-09T00:48:02.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.254+0000 7f6bd1ffb700 1 -- 172.21.15.5:0/2673339591 <== mds.4 v2:172.21.15.131:6836/1633006186 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55898de99970 con 0x55898e1194a0 2024-05-09T00:48:02.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.254+0000 7f6bd1ffb700 10 client.15396 handle_command_reply: tid=0 2024-05-09T00:48:02.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.254+0000 7f6bff930b80 1 client.15396 shutdown 2024-05-09T00:48:02.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.254+0000 7f6bff930b80 1 -- 172.21.15.5:0/2673339591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55898e104400 msgr2=0x55898e10d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:02.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.254+0000 7f6bff930b80 1 --2- 172.21.15.5:0/2673339591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55898e104400 0x55898e10d4f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f6bec19cd00 tx=0x7f6be4009c40 comp rx=0 tx=0).stop 2024-05-09T00:48:02.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.254+0000 7f6bff930b80 1 -- 172.21.15.5:0/2673339591 shutdown_connections 2024-05-09T00:48:02.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.254+0000 7f6bff930b80 1 --2- 172.21.15.5:0/2673339591 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x55898e1194a0 0x55898e186030 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:02.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.254+0000 7f6bff930b80 1 --2- 172.21.15.5:0/2673339591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55898e104400 0x55898e10d4f0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:02.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.254+0000 7f6bff930b80 1 --2- 172.21.15.5:0/2673339591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55898df7e4a0 0x55898e111c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:02.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.254+0000 7f6bff930b80 1 --2- 172.21.15.5:0/2673339591 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55898e068160 0x55898e10cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:02.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.254+0000 7f6bd1ffb700 0 client.15396 ms_handle_reset on v2:172.21.15.131:6836/1633006186 2024-05-09T00:48:02.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.254+0000 7f6bff930b80 1 -- 172.21.15.5:0/2673339591 >> 172.21.15.5:0/2673339591 conn(0x55898dfde6a0 msgr2=0x55898e108790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:02.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.254+0000 7f6bff930b80 1 -- 172.21.15.5:0/2673339591 shutdown_connections 2024-05-09T00:48:02.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.254+0000 7f6bff930b80 1 -- 172.21.15.5:0/2673339591 wait complete. 2024-05-09T00:48:02.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.254+0000 7f6bff930b80 20 client.15396 trim_cache size 0 max 16384 2024-05-09T00:48:02.259 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-09T00:48:02.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.255+0000 7f6bf0f1a700 1 -- 172.21.15.5:0/2363421725 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f6bd40b2900 msgr2=0x7f6bd40b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:02.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.255+0000 7f6bf0f1a700 1 --2- 172.21.15.5:0/2363421725 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f6bd40b2900 0x7f6bd40b4e10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6be4009fd0 tx=0x7f6be40090f0 comp rx=0 tx=0).stop 2024-05-09T00:48:02.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.255+0000 7f6bf0f1a700 1 -- 172.21.15.5:0/2363421725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bec107e30 msgr2=0x7f6bec19c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:02.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.255+0000 7f6bf0f1a700 1 --2- 172.21.15.5:0/2363421725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bec107e30 0x7f6bec19c260 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6be0006010 tx=0x7f6be00047b0 comp rx=0 tx=0).stop 2024-05-09T00:48:02.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.255+0000 7f6bf0f1a700 1 -- 172.21.15.5:0/2363421725 shutdown_connections 2024-05-09T00:48:02.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.255+0000 7f6bf0f1a700 1 --2- 172.21.15.5:0/2363421725 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f6bd40b2900 0x7f6bd40b4e10 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:02.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.255+0000 7f6bf0f1a700 1 --2- 172.21.15.5:0/2363421725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bec108780 0x7f6bec19c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:02.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.255+0000 7f6bf0f1a700 1 --2- 172.21.15.5:0/2363421725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6bec103550 0x7f6bec19bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:02.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.255+0000 7f6bf0f1a700 1 --2- 172.21.15.5:0/2363421725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bec107e30 0x7f6bec19c260 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:02.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.255+0000 7f6bf0f1a700 1 -- 172.21.15.5:0/2363421725 >> 172.21.15.5:0/2363421725 conn(0x7f6bec078e00 msgr2=0x7f6bec100f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:02.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.255+0000 7f6bf0f1a700 1 -- 172.21.15.5:0/2363421725 shutdown_connections 2024-05-09T00:48:02.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:02.255+0000 7f6bf0f1a700 1 -- 172.21.15.5:0/2363421725 wait complete. 2024-05-09T00:48:03.032 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-09T00:48:03.293 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:48:03.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:48:03 smithi005 ceph-mon[30365]: pgmap v422: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-05-09T00:48:03.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:48:03 smithi131 ceph-mon[34547]: pgmap v422: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-05-09T00:48:03.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:48:03 smithi157 ceph-mon[34294]: pgmap v422: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-05-09T00:48:05.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.198+0000 7fa9bead7700 1 -- 172.21.15.5:0/513906191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9b8108f20 msgr2=0x7fa9b8109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:05.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.198+0000 7fa9bead7700 1 --2- 172.21.15.5:0/513906191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9b8108f20 0x7fa9b8109330 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa9a40099c0 tx=0x7fa9a4009cd0 comp rx=0 tx=0).stop 2024-05-09T00:48:05.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.198+0000 7fa9bead7700 1 -- 172.21.15.5:0/513906191 shutdown_connections 2024-05-09T00:48:05.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.198+0000 7fa9bead7700 1 --2- 172.21.15.5:0/513906191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9b8109870 0x7fa9b81111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:05.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.198+0000 7fa9bead7700 1 --2- 172.21.15.5:0/513906191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9b81045f0 0x7fa9b8104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:05.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.198+0000 7fa9bead7700 1 --2- 172.21.15.5:0/513906191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9b8108f20 0x7fa9b8109330 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:05.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.198+0000 7fa9bead7700 1 -- 172.21.15.5:0/513906191 >> 172.21.15.5:0/513906191 conn(0x7fa9b8100040 msgr2=0x7fa9b8102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:05.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.198+0000 7fa9bead7700 1 -- 172.21.15.5:0/513906191 shutdown_connections 2024-05-09T00:48:05.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.198+0000 7fa9bead7700 1 -- 172.21.15.5:0/513906191 wait complete. 2024-05-09T00:48:05.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.199+0000 7fa9bead7700 1 Processor -- start 2024-05-09T00:48:05.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.199+0000 7fa9bead7700 1 -- start start 2024-05-09T00:48:05.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.199+0000 7fa9bead7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9b81045f0 0x7fa9b819aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:05.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.199+0000 7fa9bead7700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9b8108f20 0x7fa9b819b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:05.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.199+0000 7fa9bead7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9b8109870 0x7fa9b819f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:05.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.199+0000 7fa9bead7700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b81144f0 con 0x7fa9b8109870 2024-05-09T00:48:05.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.199+0000 7fa9bead7700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b8114370 con 0x7fa9b81045f0 2024-05-09T00:48:05.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.199+0000 7fa9bead7700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b8114670 con 0x7fa9b8108f20 2024-05-09T00:48:05.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.199+0000 7fa9bd074700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9b8109870 0x7fa9b819f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:05.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.199+0000 7fa9bd074700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9b8109870 0x7fa9b819f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33780/0 (socket says 172.21.15.5:33780) 2024-05-09T00:48:05.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.199+0000 7fa9bd074700 1 -- 172.21.15.5:0/2438066843 learned_addr learned my addr 172.21.15.5:0/2438066843 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:05.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.199+0000 7fa9bc873700 1 --2- 172.21.15.5:0/2438066843 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9b81045f0 0x7fa9b819aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:05.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.199+0000 7fa9b7fff700 1 --2- 172.21.15.5:0/2438066843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9b8108f20 0x7fa9b819b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:05.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.199+0000 7fa9bd074700 1 -- 172.21.15.5:0/2438066843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9b8108f20 msgr2=0x7fa9b819b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:05.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.199+0000 7fa9bd074700 1 --2- 172.21.15.5:0/2438066843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9b8108f20 0x7fa9b819b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:05.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.199+0000 7fa9bd074700 1 -- 172.21.15.5:0/2438066843 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9b81045f0 msgr2=0x7fa9b819aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:05.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.199+0000 7fa9bd074700 1 --2- 172.21.15.5:0/2438066843 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9b81045f0 0x7fa9b819aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:05.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.199+0000 7fa9bd074700 1 -- 172.21.15.5:0/2438066843 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9b819fd70 con 0x7fa9b8109870 2024-05-09T00:48:05.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.199+0000 7fa9bc873700 1 --2- 172.21.15.5:0/2438066843 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9b81045f0 0x7fa9b819aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:48:05.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.200+0000 7fa9bd074700 1 --2- 172.21.15.5:0/2438066843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9b8109870 0x7fa9b819f6d0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa9b0007ee0 tx=0x7fa9b000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:05.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.200+0000 7fa9b5ffb700 1 -- 172.21.15.5:0/2438066843 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b000dbd0 con 0x7fa9b8109870 2024-05-09T00:48:05.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.200+0000 7fa9bead7700 1 -- 172.21.15.5:0/2438066843 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9b81a0060 con 0x7fa9b8109870 2024-05-09T00:48:05.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.200+0000 7fa9b5ffb700 1 -- 172.21.15.5:0/2438066843 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9b000dd70 con 0x7fa9b8109870 2024-05-09T00:48:05.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.200+0000 7fa9b5ffb700 1 -- 172.21.15.5:0/2438066843 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b0005990 con 0x7fa9b8109870 2024-05-09T00:48:05.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.200+0000 7fa9bead7700 1 -- 172.21.15.5:0/2438066843 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9b81a7980 con 0x7fa9b8109870 2024-05-09T00:48:05.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.203+0000 7fa9b5ffb700 1 -- 172.21.15.5:0/2438066843 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fa9b001a020 con 0x7fa9b8109870 2024-05-09T00:48:05.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.204+0000 7fa9b5ffb700 1 --2- 172.21.15.5:0/2438066843 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fa9a80b2880 0x7fa9a80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:05.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.204+0000 7fa9cd4edb80 1 Processor -- start 2024-05-09T00:48:05.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.204+0000 7fa9bc873700 1 --2- 172.21.15.5:0/2438066843 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fa9a80b2880 0x7fa9a80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:05.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.204+0000 7fa9b5ffb700 1 -- 172.21.15.5:0/2438066843 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fa9b00e1b20 con 0x7fa9b8109870 2024-05-09T00:48:05.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.204+0000 7fa9cd4edb80 1 -- start start 2024-05-09T00:48:05.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.205+0000 7fa9cd4edb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653beb66160 0x5653bec03900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:05.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.205+0000 7fa9cd4edb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653bea7c4a0 0x5653bec03e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:05.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.205+0000 7fa9cd4edb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653bebfdab0 0x5653bebfdee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:05.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.205+0000 7fa9cd4edb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5653beb1bff0 con 0x5653bebfdab0 2024-05-09T00:48:05.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.205+0000 7fa9cd4edb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5653beaba7f0 con 0x5653bea7c4a0 2024-05-09T00:48:05.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.205+0000 7fa9cd4edb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5653beb17fb0 con 0x5653beb66160 2024-05-09T00:48:05.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.205+0000 7fa9bc873700 1 --2- 172.21.15.5:0/2438066843 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fa9a80b2880 0x7fa9a80b4d90 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac0099d0 tx=0x7fa9ac008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:05.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.205+0000 7fa9bd074700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653bebfdab0 0x5653bebfdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:05.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.205+0000 7fa9bd074700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653bebfdab0 0x5653bebfdee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33796/0 (socket says 172.21.15.5:33796) 2024-05-09T00:48:05.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.205+0000 7fa9bd074700 1 -- 172.21.15.5:0/2157236222 learned_addr learned my addr 172.21.15.5:0/2157236222 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:05.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.205+0000 7fa9b7fff700 1 --2- 172.21.15.5:0/2157236222 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653bea7c4a0 0x5653bec03e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:05.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.205+0000 7fa9bc873700 1 --2- 172.21.15.5:0/2157236222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653beb66160 0x5653bec03900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:05.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.205+0000 7fa9bd074700 1 -- 172.21.15.5:0/2157236222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653beb66160 msgr2=0x5653bec03900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:05.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.205+0000 7fa9bd074700 1 --2- 172.21.15.5:0/2157236222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653beb66160 0x5653bec03900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:05.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.205+0000 7fa9bd074700 1 -- 172.21.15.5:0/2157236222 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653bea7c4a0 msgr2=0x5653bec03e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:05.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.205+0000 7fa9bd074700 1 --2- 172.21.15.5:0/2157236222 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653bea7c4a0 0x5653bec03e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:05.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.205+0000 7fa9bd074700 1 -- 172.21.15.5:0/2157236222 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653beab2370 con 0x5653bebfdab0 2024-05-09T00:48:05.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.205+0000 7fa9bd074700 1 --2- 172.21.15.5:0/2157236222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653bebfdab0 0x5653bebfdee0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fa9b00a6320 tx=0x7fa9b00a8820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d97f922726c6422 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:05.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.205+0000 7fa9a2ffd700 1 -- 172.21.15.5:0/2157236222 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b00a90b0 con 0x5653bebfdab0 2024-05-09T00:48:05.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.206+0000 7fa9a2ffd700 1 -- 172.21.15.5:0/2157236222 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9b00ad460 con 0x5653bebfdab0 2024-05-09T00:48:05.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.206+0000 7fa9a2ffd700 1 -- 172.21.15.5:0/2157236222 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b0004cb0 con 0x5653bebfdab0 2024-05-09T00:48:05.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.206+0000 7fa9cd4edb80 1 -- 172.21.15.5:0/2157236222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653bebfdab0 msgr2=0x5653bebfdee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:05.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.206+0000 7fa9cd4edb80 1 --2- 172.21.15.5:0/2157236222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653bebfdab0 0x5653bebfdee0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fa9b00a6320 tx=0x7fa9b00a8820 comp rx=0 tx=0).stop 2024-05-09T00:48:05.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.206+0000 7fa9cd4edb80 1 -- 172.21.15.5:0/2157236222 shutdown_connections 2024-05-09T00:48:05.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.206+0000 7fa9cd4edb80 1 --2- 172.21.15.5:0/2157236222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653bebfdab0 0x5653bebfdee0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:05.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.206+0000 7fa9cd4edb80 1 --2- 172.21.15.5:0/2157236222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653beb66160 0x5653bec03900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:05.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.206+0000 7fa9cd4edb80 1 --2- 172.21.15.5:0/2157236222 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653bea7c4a0 0x5653bec03e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:05.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.207+0000 7fa9cd4edb80 1 -- 172.21.15.5:0/2157236222 >> 172.21.15.5:0/2157236222 conn(0x5653beadc6a0 msgr2=0x5653bebf4400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:05.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.207+0000 7fa9cd4edb80 1 -- 172.21.15.5:0/2157236222 shutdown_connections 2024-05-09T00:48:05.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.207+0000 7fa9cd4edb80 1 -- 172.21.15.5:0/2157236222 wait complete. 2024-05-09T00:48:05.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.207+0000 7fa9cd4edb80 1 -- start start 2024-05-09T00:48:05.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.207+0000 7fa9cd4edb80 1 Processor -- start 2024-05-09T00:48:05.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.208+0000 7fa9cd4edb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653bea7c4a0 0x5653bec07650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:05.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.208+0000 7fa9cd4edb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653beb66160 0x5653bec07b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:05.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.208+0000 7fa9cd4edb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653bebfdab0 0x5653bec080d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:05.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.208+0000 7fa9cd4edb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5653be997ba0 con 0x5653bea7c4a0 2024-05-09T00:48:05.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.208+0000 7fa9cd4edb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5653bead6150 con 0x5653beb66160 2024-05-09T00:48:05.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.208+0000 7fa9cd4edb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5653bec0a5e0 con 0x5653bebfdab0 2024-05-09T00:48:05.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.208+0000 7fa9cd4edb80 10 client.0 _ll_register_callbacks cb 0x5653be98d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:48:05.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.208+0000 7fa9bc873700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653bea7c4a0 0x5653bec07650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:05.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.208+0000 7fa9bc873700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653bea7c4a0 0x5653bec07650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33800/0 (socket says 172.21.15.5:33800) 2024-05-09T00:48:05.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.208+0000 7fa9bc873700 1 -- 172.21.15.5:0/662048301 learned_addr learned my addr 172.21.15.5:0/662048301 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:05.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.208+0000 7fa9b7fff700 1 --2- 172.21.15.5:0/662048301 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653beb66160 0x5653bec07b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:05.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.208+0000 7fa9bd074700 1 --2- 172.21.15.5:0/662048301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653bebfdab0 0x5653bec080d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:05.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.208+0000 7fa9bc873700 1 -- 172.21.15.5:0/662048301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653bebfdab0 msgr2=0x5653bec080d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:05.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.208+0000 7fa9bc873700 1 --2- 172.21.15.5:0/662048301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653bebfdab0 0x5653bec080d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:05.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.208+0000 7fa9bc873700 1 -- 172.21.15.5:0/662048301 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653beb66160 msgr2=0x5653bec07b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:05.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.208+0000 7fa9bc873700 1 --2- 172.21.15.5:0/662048301 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653beb66160 0x5653bec07b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:05.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.208+0000 7fa9bc873700 1 -- 172.21.15.5:0/662048301 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653be997490 con 0x5653bea7c4a0 2024-05-09T00:48:05.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.208+0000 7fa9bc873700 1 --2- 172.21.15.5:0/662048301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653bea7c4a0 0x5653bec07650 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac00df20 tx=0x7fa9ac004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b378987e96f08f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:05.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.208+0000 7fa997fff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:48:05.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.209+0000 7fa997fff700 1 -- 172.21.15.5:0/662048301 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac024020 con 0x5653bea7c4a0 2024-05-09T00:48:05.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.209+0000 7fa997fff700 1 -- 172.21.15.5:0/662048301 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fa9ac00f040 con 0x5653bea7c4a0 2024-05-09T00:48:05.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.209+0000 7fa9cd4edb80 1 -- 172.21.15.5:0/662048301 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9b00ad460 con 0x5653bea7c4a0 2024-05-09T00:48:05.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.209+0000 7fa997fff700 1 -- 172.21.15.5:0/662048301 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa9ac02f050 con 0x5653bea7c4a0 2024-05-09T00:48:05.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.209+0000 7fa9cd4edb80 10 client.15408 fetch_fsmap learned FSMap version 29 2024-05-09T00:48:05.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.209+0000 7fa9cd4edb80 1 -- 172.21.15.5:0/662048301 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5653bec150d0 con 0x5653bea7c4a0 2024-05-09T00:48:05.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.209+0000 7fa997fff700 1 -- 172.21.15.5:0/662048301 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9b00ad460 con 0x5653bea7c4a0 2024-05-09T00:48:05.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.209+0000 7fa997fff700 1 -- 172.21.15.5:0/662048301 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fa9ac013070 con 0x5653bea7c4a0 2024-05-09T00:48:05.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.209+0000 7fa9cd4edb80 10 client.15408 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:48:05.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.209+0000 7fa9cd4edb80 10 client.15408 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T00:48:05.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.210+0000 7fa9cd4edb80 20 client.15408 populate_metadata read hostname 'smithi005' 2024-05-09T00:48:05.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.210+0000 7fa9cd4edb80 1 --2- 172.21.15.5:0/662048301 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x5653bec17760 0x5653bec19b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:05.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.210+0000 7fa9cd4edb80 4 client.15408 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:48:05.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.210+0000 7fa9cd4edb80 1 -- 172.21.15.5:0/662048301 --> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653beb299d0 con 0x5653bec17760 2024-05-09T00:48:05.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.210+0000 7fa9b7fff700 1 --2- 172.21.15.5:0/662048301 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x5653bec17760 0x5653bec19b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:05.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.210+0000 7fa9b7fff700 1 --2- 172.21.15.5:0/662048301 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x5653bec17760 0x5653bec19b80 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6b09ef0b8633b1dd server_cookie=cb0a1e20de5bfff in_seq=0 out_seq=0 2024-05-09T00:48:05.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.210+0000 7fa997fff700 10 client.15408 ms_handle_connect on v2:172.21.15.157:6834/832898273 2024-05-09T00:48:05.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.211+0000 7fa997fff700 1 -- 172.21.15.5:0/662048301 <== mds.3 v2:172.21.15.157:6834/832898273 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5653beb299d0 con 0x5653bec17760 2024-05-09T00:48:05.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.211+0000 7fa997fff700 10 client.15408 handle_command_reply: tid=0 2024-05-09T00:48:05.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.211+0000 7fa9cd4edb80 1 client.15408 shutdown 2024-05-09T00:48:05.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.211+0000 7fa9cd4edb80 1 -- 172.21.15.5:0/662048301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653bea7c4a0 msgr2=0x5653bec07650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:05.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.211+0000 7fa9cd4edb80 1 --2- 172.21.15.5:0/662048301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653bea7c4a0 0x5653bec07650 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac00df20 tx=0x7fa9ac004860 comp rx=0 tx=0).stop 2024-05-09T00:48:05.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.211+0000 7fa9cd4edb80 1 -- 172.21.15.5:0/662048301 shutdown_connections 2024-05-09T00:48:05.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.211+0000 7fa9cd4edb80 1 --2- 172.21.15.5:0/662048301 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x5653bec17760 0x5653bec19b80 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:05.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.211+0000 7fa9cd4edb80 1 --2- 172.21.15.5:0/662048301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653bebfdab0 0x5653bec080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:05.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.211+0000 7fa9cd4edb80 1 --2- 172.21.15.5:0/662048301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653bea7c4a0 0x5653bec07650 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:05.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.211+0000 7fa9cd4edb80 1 --2- 172.21.15.5:0/662048301 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653beb66160 0x5653bec07b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:05.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.211+0000 7fa9cd4edb80 1 -- 172.21.15.5:0/662048301 >> 172.21.15.5:0/662048301 conn(0x5653beadc6a0 msgr2=0x5653bebf41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:05.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.211+0000 7fa997fff700 0 client.15408 ms_handle_reset on v2:172.21.15.157:6834/832898273 2024-05-09T00:48:05.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.212+0000 7fa9cd4edb80 1 -- 172.21.15.5:0/662048301 shutdown_connections 2024-05-09T00:48:05.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.212+0000 7fa9cd4edb80 1 -- 172.21.15.5:0/662048301 wait complete. 2024-05-09T00:48:05.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.212+0000 7fa9cd4edb80 20 client.15408 trim_cache size 0 max 16384 2024-05-09T00:48:05.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.228+0000 7fa9cd4edb80 1 Processor -- start 2024-05-09T00:48:05.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.228+0000 7fa9cd4edb80 1 -- start start 2024-05-09T00:48:05.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.228+0000 7fa9cd4edb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653bea7c4a0 0x5653bec07f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:05.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.229+0000 7fa9cd4edb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653beb66160 0x5653bec133e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:05.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.229+0000 7fa9cd4edb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653bebfdab0 0x5653bec1f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:05.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.229+0000 7fa9cd4edb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b81acb00 con 0x5653beb66160 2024-05-09T00:48:05.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.229+0000 7fa9cd4edb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b81ae640 con 0x5653bea7c4a0 2024-05-09T00:48:05.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.229+0000 7fa9cd4edb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b81ad1d0 con 0x5653bebfdab0 2024-05-09T00:48:05.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.229+0000 7fa9b7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653beb66160 0x5653bec133e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:05.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.229+0000 7fa9b7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653beb66160 0x5653bec133e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33816/0 (socket says 172.21.15.5:33816) 2024-05-09T00:48:05.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.229+0000 7fa9b7fff700 1 -- 172.21.15.5:0/63389210 learned_addr learned my addr 172.21.15.5:0/63389210 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:05.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.229+0000 7fa9bc873700 1 --2- 172.21.15.5:0/63389210 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653bea7c4a0 0x5653bec07f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:05.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.229+0000 7fa9bd074700 1 --2- 172.21.15.5:0/63389210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653bebfdab0 0x5653bec1f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:05.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.229+0000 7fa9b7fff700 1 -- 172.21.15.5:0/63389210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653bebfdab0 msgr2=0x5653bec1f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:05.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.229+0000 7fa9b7fff700 1 --2- 172.21.15.5:0/63389210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653bebfdab0 0x5653bec1f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:05.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.229+0000 7fa9b7fff700 1 -- 172.21.15.5:0/63389210 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653bea7c4a0 msgr2=0x5653bec07f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:05.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.229+0000 7fa9b7fff700 1 --2- 172.21.15.5:0/63389210 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653bea7c4a0 0x5653bec07f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:05.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.229+0000 7fa9b7fff700 1 -- 172.21.15.5:0/63389210 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653bec09ef0 con 0x5653beb66160 2024-05-09T00:48:05.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.229+0000 7fa9bd074700 1 --2- 172.21.15.5:0/63389210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653bebfdab0 0x5653bec1f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:48:05.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.230+0000 7fa9b7fff700 1 --2- 172.21.15.5:0/63389210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653beb66160 0x5653bec133e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4006010 tx=0x7fa9a4003af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf18969ff232cf2b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:05.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.233+0000 7fa9a27fc700 1 -- 172.21.15.5:0/63389210 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a40050c0 con 0x5653beb66160 2024-05-09T00:48:05.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.233+0000 7fa9a27fc700 1 -- 172.21.15.5:0/63389210 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9a400b730 con 0x5653beb66160 2024-05-09T00:48:05.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.233+0000 7fa9a27fc700 1 -- 172.21.15.5:0/63389210 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a400f6c0 con 0x5653beb66160 2024-05-09T00:48:05.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.234+0000 7fa9cd4edb80 1 -- 172.21.15.5:0/63389210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653beb66160 msgr2=0x5653bec133e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:05.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.234+0000 7fa9cd4edb80 1 --2- 172.21.15.5:0/63389210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653beb66160 0x5653bec133e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4006010 tx=0x7fa9a4003af0 comp rx=0 tx=0).stop 2024-05-09T00:48:05.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.234+0000 7fa9cd4edb80 1 -- 172.21.15.5:0/63389210 shutdown_connections 2024-05-09T00:48:05.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.234+0000 7fa9cd4edb80 1 --2- 172.21.15.5:0/63389210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653bebfdab0 0x5653bec1f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:05.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.234+0000 7fa9cd4edb80 1 --2- 172.21.15.5:0/63389210 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653bea7c4a0 0x5653bec07f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:05.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.234+0000 7fa9cd4edb80 1 --2- 172.21.15.5:0/63389210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653beb66160 0x5653bec133e0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:05.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.234+0000 7fa9cd4edb80 1 -- 172.21.15.5:0/63389210 >> 172.21.15.5:0/63389210 conn(0x5653beadc6a0 msgr2=0x5653bec05f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:05.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.234+0000 7fa9cd4edb80 1 -- 172.21.15.5:0/63389210 shutdown_connections 2024-05-09T00:48:05.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.234+0000 7fa9cd4edb80 1 -- 172.21.15.5:0/63389210 wait complete. 2024-05-09T00:48:05.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.235+0000 7fa9cd4edb80 1 -- start start 2024-05-09T00:48:05.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.235+0000 7fa9cd4edb80 1 Processor -- start 2024-05-09T00:48:05.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.235+0000 7fa9cd4edb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653bea7c4a0 0x5653bec0f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:05.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.235+0000 7fa9cd4edb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653beb66160 0x5653bec0fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:05.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.235+0000 7fa9cd4edb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653bebfdab0 0x5653bec1e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:05.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.235+0000 7fa9cd4edb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b81adf70 con 0x5653bea7c4a0 2024-05-09T00:48:05.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.235+0000 7fa9cd4edb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b81ad8a0 con 0x5653bebfdab0 2024-05-09T00:48:05.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.235+0000 7fa9cd4edb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b81aaae0 con 0x5653beb66160 2024-05-09T00:48:05.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.235+0000 7fa9cd4edb80 10 client.0 _ll_register_callbacks cb 0x5653be98d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:48:05.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.235+0000 7fa9bc873700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653bea7c4a0 0x5653bec0f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:05.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.236+0000 7fa9bc873700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653bea7c4a0 0x5653bec0f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33832/0 (socket says 172.21.15.5:33832) 2024-05-09T00:48:05.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.236+0000 7fa9bc873700 1 -- 172.21.15.5:0/2936610384 learned_addr learned my addr 172.21.15.5:0/2936610384 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:05.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.236+0000 7fa9bd074700 1 --2- 172.21.15.5:0/2936610384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653bebfdab0 0x5653bec1e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:05.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.236+0000 7fa9b7fff700 1 --2- 172.21.15.5:0/2936610384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653beb66160 0x5653bec0fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:05.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.236+0000 7fa9bc873700 1 -- 172.21.15.5:0/2936610384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653beb66160 msgr2=0x5653bec0fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:05.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.236+0000 7fa9bc873700 1 --2- 172.21.15.5:0/2936610384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653beb66160 0x5653bec0fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:05.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.236+0000 7fa9bc873700 1 -- 172.21.15.5:0/2936610384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653bebfdab0 msgr2=0x5653bec1e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:05.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.236+0000 7fa9bc873700 1 --2- 172.21.15.5:0/2936610384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653bebfdab0 0x5653bec1e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:05.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.236+0000 7fa9bc873700 1 -- 172.21.15.5:0/2936610384 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653bec1e840 con 0x5653bea7c4a0 2024-05-09T00:48:05.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.236+0000 7fa9bc873700 1 --2- 172.21.15.5:0/2936610384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653bea7c4a0 0x5653bec0f620 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac000c00 tx=0x7fa9ac0127d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dfbe50bd10722f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:05.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.236+0000 7fa997fff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:48:05.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.236+0000 7fa997fff700 1 -- 172.21.15.5:0/2936610384 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac024040 con 0x5653bea7c4a0 2024-05-09T00:48:05.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.236+0000 7fa997fff700 1 -- 172.21.15.5:0/2936610384 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fa9ac012c40 con 0x5653bea7c4a0 2024-05-09T00:48:05.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.236+0000 7fa9cd4edb80 1 -- 172.21.15.5:0/2936610384 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9a400b730 con 0x5653bea7c4a0 2024-05-09T00:48:05.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.237+0000 7fa997fff700 1 -- 172.21.15.5:0/2936610384 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa9ac012e40 con 0x5653bea7c4a0 2024-05-09T00:48:05.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.237+0000 7fa9cd4edb80 10 client.15414 fetch_fsmap learned FSMap version 29 2024-05-09T00:48:05.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.237+0000 7fa9cd4edb80 1 -- 172.21.15.5:0/2936610384 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5653bec81540 con 0x5653bea7c4a0 2024-05-09T00:48:05.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.237+0000 7fa997fff700 1 -- 172.21.15.5:0/2936610384 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9a400b730 con 0x5653bea7c4a0 2024-05-09T00:48:05.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.237+0000 7fa997fff700 1 -- 172.21.15.5:0/2936610384 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fa9ac0336a0 con 0x5653bea7c4a0 2024-05-09T00:48:05.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.237+0000 7fa9cd4edb80 10 client.15414 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:48:05.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.237+0000 7fa9cd4edb80 10 client.15414 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T00:48:05.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.237+0000 7fa9cd4edb80 20 client.15414 populate_metadata read hostname 'smithi005' 2024-05-09T00:48:05.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.237+0000 7fa9cd4edb80 1 --2- 172.21.15.5:0/2936610384 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x5653bec81c40 0x5653bec84060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:05.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.237+0000 7fa9cd4edb80 4 client.15414 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:48:05.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.237+0000 7fa9cd4edb80 1 -- 172.21.15.5:0/2936610384 --> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5653beb23a70 con 0x5653bec81c40 2024-05-09T00:48:05.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.237+0000 7fa9b7fff700 1 --2- 172.21.15.5:0/2936610384 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x5653bec81c40 0x5653bec84060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:05.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.238+0000 7fa997fff700 10 client.15414 ms_handle_connect on v2:172.21.15.157:6834/832898273 2024-05-09T00:48:05.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.238+0000 7fa9b7fff700 1 --2- 172.21.15.5:0/2936610384 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x5653bec81c40 0x5653bec84060 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bd7d2ba3d38658f7 server_cookie=e288624dc5bdcd67 in_seq=0 out_seq=0 2024-05-09T00:48:05.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.238+0000 7fa997fff700 1 -- 172.21.15.5:0/2936610384 <== mds.3 v2:172.21.15.157:6834/832898273 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5653beb23a70 con 0x5653bec81c40 2024-05-09T00:48:05.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.238+0000 7fa997fff700 10 client.15414 handle_command_reply: tid=0 2024-05-09T00:48:05.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.238+0000 7fa9cd4edb80 1 client.15414 shutdown 2024-05-09T00:48:05.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.239+0000 7fa9cd4edb80 1 -- 172.21.15.5:0/2936610384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653bea7c4a0 msgr2=0x5653bec0f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:05.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.239+0000 7fa9cd4edb80 1 --2- 172.21.15.5:0/2936610384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653bea7c4a0 0x5653bec0f620 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac000c00 tx=0x7fa9ac0127d0 comp rx=0 tx=0).stop 2024-05-09T00:48:05.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.239+0000 7fa9cd4edb80 1 -- 172.21.15.5:0/2936610384 shutdown_connections 2024-05-09T00:48:05.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.239+0000 7fa9cd4edb80 1 --2- 172.21.15.5:0/2936610384 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x5653bec81c40 0x5653bec84060 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:05.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.239+0000 7fa9cd4edb80 1 --2- 172.21.15.5:0/2936610384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653bebfdab0 0x5653bec1e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:05.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.239+0000 7fa9cd4edb80 1 --2- 172.21.15.5:0/2936610384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653bea7c4a0 0x5653bec0f620 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:05.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.239+0000 7fa9cd4edb80 1 --2- 172.21.15.5:0/2936610384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653beb66160 0x5653bec0fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:05.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.239+0000 7fa9cd4edb80 1 -- 172.21.15.5:0/2936610384 >> 172.21.15.5:0/2936610384 conn(0x5653beadc6a0 msgr2=0x5653bec05b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:05.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.239+0000 7fa997fff700 0 client.15414 ms_handle_reset on v2:172.21.15.157:6834/832898273 2024-05-09T00:48:05.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.239+0000 7fa9cd4edb80 1 -- 172.21.15.5:0/2936610384 shutdown_connections 2024-05-09T00:48:05.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.239+0000 7fa9cd4edb80 1 -- 172.21.15.5:0/2936610384 wait complete. 2024-05-09T00:48:05.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.239+0000 7fa9cd4edb80 20 client.15414 trim_cache size 0 max 16384 2024-05-09T00:48:05.245 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-09T00:48:05.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.240+0000 7fa9bead7700 1 -- 172.21.15.5:0/2438066843 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fa9a80b2880 msgr2=0x7fa9a80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:05.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.240+0000 7fa9bead7700 1 --2- 172.21.15.5:0/2438066843 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fa9a80b2880 0x7fa9a80b4d90 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac0099d0 tx=0x7fa9ac008040 comp rx=0 tx=0).stop 2024-05-09T00:48:05.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.240+0000 7fa9bead7700 1 -- 172.21.15.5:0/2438066843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9b8109870 msgr2=0x7fa9b819f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:05.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.240+0000 7fa9bead7700 1 --2- 172.21.15.5:0/2438066843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9b8109870 0x7fa9b819f6d0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa9b0007ee0 tx=0x7fa9b000d3b0 comp rx=0 tx=0).stop 2024-05-09T00:48:05.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.240+0000 7fa9bead7700 1 -- 172.21.15.5:0/2438066843 shutdown_connections 2024-05-09T00:48:05.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.240+0000 7fa9bead7700 1 --2- 172.21.15.5:0/2438066843 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fa9a80b2880 0x7fa9a80b4d90 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:05.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.240+0000 7fa9bead7700 1 --2- 172.21.15.5:0/2438066843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9b8109870 0x7fa9b819f6d0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:05.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.240+0000 7fa9bead7700 1 --2- 172.21.15.5:0/2438066843 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9b81045f0 0x7fa9b819aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:05.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.240+0000 7fa9bead7700 1 --2- 172.21.15.5:0/2438066843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9b8108f20 0x7fa9b819b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:05.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.240+0000 7fa9bead7700 1 -- 172.21.15.5:0/2438066843 >> 172.21.15.5:0/2438066843 conn(0x7fa9b8100040 msgr2=0x7fa9b810a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:05.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.240+0000 7fa9bead7700 1 -- 172.21.15.5:0/2438066843 shutdown_connections 2024-05-09T00:48:05.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:05.240+0000 7fa9bead7700 1 -- 172.21.15.5:0/2438066843 wait complete. 2024-05-09T00:48:05.488 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:48:05 smithi157 ceph-mon[34294]: pgmap v423: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-05-09T00:48:05.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:48:05 smithi005 ceph-mon[30365]: pgmap v423: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-05-09T00:48:05.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:48:05 smithi131 ceph-mon[34547]: pgmap v423: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-05-09T00:48:05.971 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T00:48:06.243 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:48:07.486 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:48:07 smithi005 ceph-mon[30365]: pgmap v424: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-05-09T00:48:07.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:48:07 smithi131 ceph-mon[34547]: pgmap v424: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-05-09T00:48:07.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:48:07 smithi157 ceph-mon[34294]: pgmap v424: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-05-09T00:48:08.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.084+0000 7f1c4dbde700 1 -- 172.21.15.5:0/3179633925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c481045f0 msgr2=0x7f1c48104a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:08.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.084+0000 7f1c4dbde700 1 --2- 172.21.15.5:0/3179633925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c481045f0 0x7f1c48104a00 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f1c380099c0 tx=0x7f1c38009cd0 comp rx=0 tx=0).stop 2024-05-09T00:48:08.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.084+0000 7f1c4dbde700 1 -- 172.21.15.5:0/3179633925 shutdown_connections 2024-05-09T00:48:08.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.084+0000 7f1c4dbde700 1 --2- 172.21.15.5:0/3179633925 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1c48109870 0x7f1c481111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:08.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.084+0000 7f1c4dbde700 1 --2- 172.21.15.5:0/3179633925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c481045f0 0x7f1c48104a00 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:08.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.084+0000 7f1c4dbde700 1 --2- 172.21.15.5:0/3179633925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c48108f20 0x7f1c48109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:08.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.084+0000 7f1c4dbde700 1 -- 172.21.15.5:0/3179633925 >> 172.21.15.5:0/3179633925 conn(0x7f1c48100040 msgr2=0x7f1c48102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:08.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.084+0000 7f1c4dbde700 1 -- 172.21.15.5:0/3179633925 shutdown_connections 2024-05-09T00:48:08.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.084+0000 7f1c4dbde700 1 -- 172.21.15.5:0/3179633925 wait complete. 2024-05-09T00:48:08.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.085+0000 7f1c4dbde700 1 Processor -- start 2024-05-09T00:48:08.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.085+0000 7f1c4dbde700 1 -- start start 2024-05-09T00:48:08.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.086+0000 7f1c4dbde700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1c481045f0 0x7f1c48075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:08.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.086+0000 7f1c4dbde700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c48108f20 0x7f1c48076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:08.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.086+0000 7f1c4dbde700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c48109870 0x7f1c48076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:08.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.086+0000 7f1c4dbde700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1c481144a0 con 0x7f1c48109870 2024-05-09T00:48:08.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.086+0000 7f1c4dbde700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1c48114320 con 0x7f1c481045f0 2024-05-09T00:48:08.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.086+0000 7f1c4dbde700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1c48114620 con 0x7f1c48108f20 2024-05-09T00:48:08.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.086+0000 7f1c47fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c48109870 0x7f1c48076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:08.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.086+0000 7f1c47fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c48109870 0x7f1c48076540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33862/0 (socket says 172.21.15.5:33862) 2024-05-09T00:48:08.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.086+0000 7f1c47fff700 1 -- 172.21.15.5:0/1826805236 learned_addr learned my addr 172.21.15.5:0/1826805236 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:08.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.086+0000 7f1c477fe700 1 --2- 172.21.15.5:0/1826805236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1c481045f0 0x7f1c48075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:08.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.086+0000 7f1c3effd700 1 --2- 172.21.15.5:0/1826805236 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c48108f20 0x7f1c48076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:08.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.086+0000 7f1c47fff700 1 -- 172.21.15.5:0/1826805236 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c48108f20 msgr2=0x7f1c48076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:08.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.086+0000 7f1c47fff700 1 --2- 172.21.15.5:0/1826805236 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c48108f20 0x7f1c48076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:08.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.086+0000 7f1c47fff700 1 -- 172.21.15.5:0/1826805236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1c481045f0 msgr2=0x7f1c48075ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:08.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.086+0000 7f1c47fff700 1 --2- 172.21.15.5:0/1826805236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1c481045f0 0x7f1c48075ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:08.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.086+0000 7f1c47fff700 1 -- 172.21.15.5:0/1826805236 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c481a9160 con 0x7f1c48109870 2024-05-09T00:48:08.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.086+0000 7f1c47fff700 1 --2- 172.21.15.5:0/1826805236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c48109870 0x7f1c48076540 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f1c40007d70 tx=0x7f1c4000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:08.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.087+0000 7f1c457fa700 1 -- 172.21.15.5:0/1826805236 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c40016020 con 0x7f1c48109870 2024-05-09T00:48:08.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.087+0000 7f1c457fa700 1 -- 172.21.15.5:0/1826805236 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c4000ac90 con 0x7f1c48109870 2024-05-09T00:48:08.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.087+0000 7f1c4dbde700 1 -- 172.21.15.5:0/1826805236 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c481a9450 con 0x7f1c48109870 2024-05-09T00:48:08.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.087+0000 7f1c457fa700 1 -- 172.21.15.5:0/1826805236 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c400115e0 con 0x7f1c48109870 2024-05-09T00:48:08.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.087+0000 7f1c4dbde700 1 -- 172.21.15.5:0/1826805236 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c481a9940 con 0x7f1c48109870 2024-05-09T00:48:08.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.091+0000 7f1c457fa700 1 -- 172.21.15.5:0/1826805236 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f1c40019020 con 0x7f1c48109870 2024-05-09T00:48:08.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.092+0000 7f1c5c5f4b80 1 Processor -- start 2024-05-09T00:48:08.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.092+0000 7f1c5c5f4b80 1 -- start start 2024-05-09T00:48:08.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.092+0000 7f1c457fa700 1 --2- 172.21.15.5:0/1826805236 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f1c340b2880 0x7f1c340b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:08.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.092+0000 7f1c5c5f4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f0b621d160 0x55f0b62b64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:08.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.092+0000 7f1c5c5f4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0b61334a0 0x55f0b62b6a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:08.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.092+0000 7f1c5c5f4b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f0b62b06a0 0x55f0b62b0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:08.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.092+0000 7f1c5c5f4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f0b61d2ff0 con 0x55f0b61334a0 2024-05-09T00:48:08.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.092+0000 7f1c5c5f4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55f0b61717f0 con 0x55f0b621d160 2024-05-09T00:48:08.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.092+0000 7f1c5c5f4b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55f0b61cefb0 con 0x55f0b62b06a0 2024-05-09T00:48:08.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.092+0000 7f1c3effd700 1 --2- 172.21.15.5:0/1826805236 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f1c340b2880 0x7f1c340b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:08.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.092+0000 7f1c457fa700 1 -- 172.21.15.5:0/1826805236 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f1c400e2170 con 0x7f1c48109870 2024-05-09T00:48:08.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.092+0000 7f1c3effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0b61334a0 0x55f0b62b6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:08.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.092+0000 7f1c3effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0b61334a0 0x55f0b62b6a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33864/0 (socket says 172.21.15.5:33864) 2024-05-09T00:48:08.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.092+0000 7f1c3effd700 1 -- 172.21.15.5:0/3959817550 learned_addr learned my addr 172.21.15.5:0/3959817550 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:08.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.092+0000 7f1c47fff700 1 --2- 172.21.15.5:0/3959817550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f0b62b06a0 0x55f0b62b0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:08.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.092+0000 7f1c477fe700 1 --2- 172.21.15.5:0/3959817550 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f0b621d160 0x55f0b62b64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:08.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.093+0000 7f1c3effd700 1 -- 172.21.15.5:0/3959817550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f0b62b06a0 msgr2=0x55f0b62b0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:08.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.093+0000 7f1c3effd700 1 --2- 172.21.15.5:0/3959817550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f0b62b06a0 0x55f0b62b0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:08.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.093+0000 7f1c3effd700 1 -- 172.21.15.5:0/3959817550 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f0b621d160 msgr2=0x55f0b62b64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:08.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.093+0000 7f1c3effd700 1 --2- 172.21.15.5:0/3959817550 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f0b621d160 0x55f0b62b64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:08.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.093+0000 7f1c3effd700 1 -- 172.21.15.5:0/3959817550 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0b6169370 con 0x55f0b61334a0 2024-05-09T00:48:08.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.093+0000 7f1c3effd700 1 --2- 172.21.15.5:0/1826805236 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f1c340b2880 0x7f1c340b4d90 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1c4806c680 tx=0x7f1c3000bbf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:08.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.093+0000 7f1c3effd700 1 --2- 172.21.15.5:0/3959817550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0b61334a0 0x55f0b62b6a30 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1c30005b00 tx=0x7f1c3001c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85f740babf9a7a5e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:08.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.093+0000 7f1c3dffb700 1 -- 172.21.15.5:0/3959817550 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c30025070 con 0x55f0b61334a0 2024-05-09T00:48:08.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.093+0000 7f1c3dffb700 1 -- 172.21.15.5:0/3959817550 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c30010dd0 con 0x55f0b61334a0 2024-05-09T00:48:08.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.093+0000 7f1c3dffb700 1 -- 172.21.15.5:0/3959817550 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c30003d70 con 0x55f0b61334a0 2024-05-09T00:48:08.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.094+0000 7f1c5c5f4b80 1 -- 172.21.15.5:0/3959817550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0b61334a0 msgr2=0x55f0b62b6a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:08.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.094+0000 7f1c5c5f4b80 1 --2- 172.21.15.5:0/3959817550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0b61334a0 0x55f0b62b6a30 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1c30005b00 tx=0x7f1c3001c000 comp rx=0 tx=0).stop 2024-05-09T00:48:08.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.094+0000 7f1c5c5f4b80 1 -- 172.21.15.5:0/3959817550 shutdown_connections 2024-05-09T00:48:08.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.094+0000 7f1c5c5f4b80 1 --2- 172.21.15.5:0/3959817550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f0b62b06a0 0x55f0b62b0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:08.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.094+0000 7f1c5c5f4b80 1 --2- 172.21.15.5:0/3959817550 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f0b621d160 0x55f0b62b64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:08.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.094+0000 7f1c5c5f4b80 1 --2- 172.21.15.5:0/3959817550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0b61334a0 0x55f0b62b6a30 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:08.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.094+0000 7f1c5c5f4b80 1 -- 172.21.15.5:0/3959817550 >> 172.21.15.5:0/3959817550 conn(0x55f0b61936a0 msgr2=0x55f0b62ab400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:08.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.094+0000 7f1c5c5f4b80 1 -- 172.21.15.5:0/3959817550 shutdown_connections 2024-05-09T00:48:08.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.094+0000 7f1c5c5f4b80 1 -- 172.21.15.5:0/3959817550 wait complete. 2024-05-09T00:48:08.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.094+0000 7f1c5c5f4b80 1 -- start start 2024-05-09T00:48:08.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.095+0000 7f1c5c5f4b80 1 Processor -- start 2024-05-09T00:48:08.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.095+0000 7f1c5c5f4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0b61334a0 0x55f0b62ba200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:08.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.095+0000 7f1c5c5f4b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f0b621d160 0x55f0b62ba740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:08.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.095+0000 7f1c5c5f4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f0b62b06a0 0x55f0b62bac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:08.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.095+0000 7f1c5c5f4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f0b604eba0 con 0x55f0b61334a0 2024-05-09T00:48:08.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.095+0000 7f1c5c5f4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55f0b618d150 con 0x55f0b62b06a0 2024-05-09T00:48:08.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.095+0000 7f1c5c5f4b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55f0b62bd1d0 con 0x55f0b621d160 2024-05-09T00:48:08.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.095+0000 7f1c5c5f4b80 10 client.0 _ll_register_callbacks cb 0x55f0b60446c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:48:08.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.095+0000 7f1c477fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0b61334a0 0x55f0b62ba200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:08.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.095+0000 7f1c3effd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f0b621d160 0x55f0b62ba740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:08.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.095+0000 7f1c477fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0b61334a0 0x55f0b62ba200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33868/0 (socket says 172.21.15.5:33868) 2024-05-09T00:48:08.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.095+0000 7f1c477fe700 1 -- 172.21.15.5:0/3557208997 learned_addr learned my addr 172.21.15.5:0/3557208997 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:08.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.095+0000 7f1c47fff700 1 --2- 172.21.15.5:0/3557208997 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f0b62b06a0 0x55f0b62bac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:08.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.096+0000 7f1c477fe700 1 -- 172.21.15.5:0/3557208997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f0b621d160 msgr2=0x55f0b62ba740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:08.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.096+0000 7f1c477fe700 1 --2- 172.21.15.5:0/3557208997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f0b621d160 0x55f0b62ba740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:08.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.096+0000 7f1c477fe700 1 -- 172.21.15.5:0/3557208997 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f0b62b06a0 msgr2=0x55f0b62bac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:08.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.096+0000 7f1c477fe700 1 --2- 172.21.15.5:0/3557208997 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f0b62b06a0 0x55f0b62bac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:08.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.096+0000 7f1c477fe700 1 -- 172.21.15.5:0/3557208997 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0b604e490 con 0x55f0b61334a0 2024-05-09T00:48:08.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.096+0000 7f1c477fe700 1 --2- 172.21.15.5:0/3557208997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0b61334a0 0x55f0b62ba200 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f1c38015000 tx=0x7f1c38009550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e8bcace3d32674e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:08.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.096+0000 7f1c26ffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:48:08.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.096+0000 7f1c26ffd700 1 -- 172.21.15.5:0/3557208997 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c38005580 con 0x55f0b61334a0 2024-05-09T00:48:08.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.096+0000 7f1c26ffd700 1 -- 172.21.15.5:0/3557208997 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f1c3800b790 con 0x55f0b61334a0 2024-05-09T00:48:08.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.096+0000 7f1c5c5f4b80 1 -- 172.21.15.5:0/3557208997 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c30010dd0 con 0x55f0b61334a0 2024-05-09T00:48:08.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.096+0000 7f1c26ffd700 1 -- 172.21.15.5:0/3557208997 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1c3800b990 con 0x55f0b61334a0 2024-05-09T00:48:08.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.096+0000 7f1c5c5f4b80 10 client.15426 fetch_fsmap learned FSMap version 29 2024-05-09T00:48:08.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.096+0000 7f1c5c5f4b80 1 -- 172.21.15.5:0/3557208997 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f0b62c7b50 con 0x55f0b61334a0 2024-05-09T00:48:08.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.097+0000 7f1c26ffd700 1 -- 172.21.15.5:0/3557208997 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c30010dd0 con 0x55f0b61334a0 2024-05-09T00:48:08.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.097+0000 7f1c26ffd700 1 -- 172.21.15.5:0/3557208997 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f1c38023ae0 con 0x55f0b61334a0 2024-05-09T00:48:08.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.097+0000 7f1c5c5f4b80 10 client.15426 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:48:08.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.097+0000 7f1c5c5f4b80 10 client.15426 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-05-09T00:48:08.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.097+0000 7f1c5c5f4b80 20 client.15426 populate_metadata read hostname 'smithi005' 2024-05-09T00:48:08.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.097+0000 7f1c5c5f4b80 1 --2- 172.21.15.5:0/3557208997 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x55f0b62ca190 0x55f0b62cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:08.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.097+0000 7f1c5c5f4b80 4 client.15426 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:48:08.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.097+0000 7f1c5c5f4b80 1 -- 172.21.15.5:0/3557208997 --> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f0b61e09d0 con 0x55f0b62ca190 2024-05-09T00:48:08.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.097+0000 7f1c3effd700 1 --2- 172.21.15.5:0/3557208997 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x55f0b62ca190 0x55f0b62cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:08.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.098+0000 7f1c3effd700 1 --2- 172.21.15.5:0/3557208997 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x55f0b62ca190 0x55f0b62cc5b0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e015a32458c04c65 server_cookie=805c5d6d5e8417b7 in_seq=0 out_seq=0 2024-05-09T00:48:08.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.098+0000 7f1c26ffd700 10 client.15426 ms_handle_connect on v2:172.21.15.131:6838/2748669631 2024-05-09T00:48:08.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.099+0000 7f1c26ffd700 1 -- 172.21.15.5:0/3557208997 <== mds.2 v2:172.21.15.131:6838/2748669631 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f0b61e09d0 con 0x55f0b62ca190 2024-05-09T00:48:08.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.099+0000 7f1c26ffd700 10 client.15426 handle_command_reply: tid=0 2024-05-09T00:48:08.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.099+0000 7f1c5c5f4b80 1 client.15426 shutdown 2024-05-09T00:48:08.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.099+0000 7f1c5c5f4b80 1 -- 172.21.15.5:0/3557208997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0b61334a0 msgr2=0x55f0b62ba200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:08.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.099+0000 7f1c5c5f4b80 1 --2- 172.21.15.5:0/3557208997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0b61334a0 0x55f0b62ba200 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f1c38015000 tx=0x7f1c38009550 comp rx=0 tx=0).stop 2024-05-09T00:48:08.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.099+0000 7f1c5c5f4b80 1 -- 172.21.15.5:0/3557208997 shutdown_connections 2024-05-09T00:48:08.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.099+0000 7f1c5c5f4b80 1 --2- 172.21.15.5:0/3557208997 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x55f0b62ca190 0x55f0b62cc5b0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:08.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.099+0000 7f1c5c5f4b80 1 --2- 172.21.15.5:0/3557208997 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f0b62b06a0 0x55f0b62bac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:08.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.099+0000 7f1c5c5f4b80 1 --2- 172.21.15.5:0/3557208997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0b61334a0 0x55f0b62ba200 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:08.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.099+0000 7f1c5c5f4b80 1 --2- 172.21.15.5:0/3557208997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f0b621d160 0x55f0b62ba740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:08.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.099+0000 7f1c5c5f4b80 1 -- 172.21.15.5:0/3557208997 >> 172.21.15.5:0/3557208997 conn(0x55f0b61936a0 msgr2=0x55f0b62ab1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:08.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.099+0000 7f1c26ffd700 0 client.15426 ms_handle_reset on v2:172.21.15.131:6838/2748669631 2024-05-09T00:48:08.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.099+0000 7f1c5c5f4b80 1 -- 172.21.15.5:0/3557208997 shutdown_connections 2024-05-09T00:48:08.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.099+0000 7f1c5c5f4b80 1 -- 172.21.15.5:0/3557208997 wait complete. 2024-05-09T00:48:08.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.099+0000 7f1c5c5f4b80 20 client.15426 trim_cache size 0 max 16384 2024-05-09T00:48:08.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.115+0000 7f1c5c5f4b80 1 Processor -- start 2024-05-09T00:48:08.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.116+0000 7f1c5c5f4b80 1 -- start start 2024-05-09T00:48:08.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.116+0000 7f1c5c5f4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f0b61334a0 0x55f0b62baac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:08.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.116+0000 7f1c5c5f4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0b621d160 0x55f0b62bbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:08.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.116+0000 7f1c5c5f4b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f0b62b06a0 0x55f0b62c5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:08.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.116+0000 7f1c5c5f4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1c481b0d50 con 0x55f0b621d160 2024-05-09T00:48:08.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.116+0000 7f1c5c5f4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1c481b2890 con 0x55f0b61334a0 2024-05-09T00:48:08.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.116+0000 7f1c5c5f4b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1c481b1420 con 0x55f0b62b06a0 2024-05-09T00:48:08.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.116+0000 7f1c3effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0b621d160 0x55f0b62bbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:08.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.116+0000 7f1c3effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0b621d160 0x55f0b62bbf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33878/0 (socket says 172.21.15.5:33878) 2024-05-09T00:48:08.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.116+0000 7f1c3effd700 1 -- 172.21.15.5:0/3221295676 learned_addr learned my addr 172.21.15.5:0/3221295676 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:08.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.116+0000 7f1c47fff700 1 --2- 172.21.15.5:0/3221295676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f0b62b06a0 0x55f0b62c5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:08.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.116+0000 7f1c477fe700 1 --2- 172.21.15.5:0/3221295676 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f0b61334a0 0x55f0b62baac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:08.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.116+0000 7f1c3effd700 1 -- 172.21.15.5:0/3221295676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f0b62b06a0 msgr2=0x55f0b62c5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:08.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.116+0000 7f1c3effd700 1 --2- 172.21.15.5:0/3221295676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f0b62b06a0 0x55f0b62c5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:08.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.116+0000 7f1c3effd700 1 -- 172.21.15.5:0/3221295676 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f0b61334a0 msgr2=0x55f0b62baac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:08.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.116+0000 7f1c3effd700 1 --2- 172.21.15.5:0/3221295676 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f0b61334a0 0x55f0b62baac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:08.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.116+0000 7f1c3effd700 1 -- 172.21.15.5:0/3221295676 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0b62accc0 con 0x55f0b621d160 2024-05-09T00:48:08.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.116+0000 7f1c47fff700 1 --2- 172.21.15.5:0/3221295676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f0b62b06a0 0x55f0b62c5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:48:08.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.117+0000 7f1c3effd700 1 --2- 172.21.15.5:0/3221295676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0b621d160 0x55f0b62bbf70 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f1c3000b6c0 tx=0x7f1c3000b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15735a813aa8c597 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:08.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.117+0000 7f1c3d7fa700 1 -- 172.21.15.5:0/3221295676 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c30025040 con 0x55f0b621d160 2024-05-09T00:48:08.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.117+0000 7f1c3d7fa700 1 -- 172.21.15.5:0/3221295676 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c30003e50 con 0x55f0b621d160 2024-05-09T00:48:08.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.117+0000 7f1c3d7fa700 1 -- 172.21.15.5:0/3221295676 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c30005690 con 0x55f0b621d160 2024-05-09T00:48:08.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.118+0000 7f1c5c5f4b80 1 -- 172.21.15.5:0/3221295676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0b621d160 msgr2=0x55f0b62bbf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:08.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.118+0000 7f1c5c5f4b80 1 --2- 172.21.15.5:0/3221295676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0b621d160 0x55f0b62bbf70 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f1c3000b6c0 tx=0x7f1c3000b410 comp rx=0 tx=0).stop 2024-05-09T00:48:08.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.118+0000 7f1c5c5f4b80 1 -- 172.21.15.5:0/3221295676 shutdown_connections 2024-05-09T00:48:08.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.118+0000 7f1c5c5f4b80 1 --2- 172.21.15.5:0/3221295676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f0b62b06a0 0x55f0b62c5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:08.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.118+0000 7f1c5c5f4b80 1 --2- 172.21.15.5:0/3221295676 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f0b61334a0 0x55f0b62baac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:08.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.118+0000 7f1c5c5f4b80 1 --2- 172.21.15.5:0/3221295676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0b621d160 0x55f0b62bbf70 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:08.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.118+0000 7f1c5c5f4b80 1 -- 172.21.15.5:0/3221295676 >> 172.21.15.5:0/3221295676 conn(0x55f0b61936a0 msgr2=0x55f0b62bb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:08.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.118+0000 7f1c5c5f4b80 1 -- 172.21.15.5:0/3221295676 shutdown_connections 2024-05-09T00:48:08.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.118+0000 7f1c5c5f4b80 1 -- 172.21.15.5:0/3221295676 wait complete. 2024-05-09T00:48:08.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.118+0000 7f1c5c5f4b80 1 -- start start 2024-05-09T00:48:08.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.118+0000 7f1c5c5f4b80 1 Processor -- start 2024-05-09T00:48:08.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.119+0000 7f1c5c5f4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0b61334a0 0x55f0b62c2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:08.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.119+0000 7f1c5c5f4b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f0b621d160 0x55f0b62c2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:08.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.119+0000 7f1c5c5f4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f0b62b06a0 0x55f0b62d0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:08.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.119+0000 7f1c5c5f4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1c481b21c0 con 0x55f0b61334a0 2024-05-09T00:48:08.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.119+0000 7f1c5c5f4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1c481b1af0 con 0x55f0b62b06a0 2024-05-09T00:48:08.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.119+0000 7f1c5c5f4b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1c481aecf0 con 0x55f0b621d160 2024-05-09T00:48:08.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.119+0000 7f1c5c5f4b80 10 client.0 _ll_register_callbacks cb 0x55f0b60446c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:48:08.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.119+0000 7f1c477fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0b61334a0 0x55f0b62c2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:08.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.119+0000 7f1c477fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0b61334a0 0x55f0b62c2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33884/0 (socket says 172.21.15.5:33884) 2024-05-09T00:48:08.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.119+0000 7f1c477fe700 1 -- 172.21.15.5:0/1799137795 learned_addr learned my addr 172.21.15.5:0/1799137795 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:08.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.119+0000 7f1c3effd700 1 --2- 172.21.15.5:0/1799137795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f0b621d160 0x55f0b62c2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:08.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.119+0000 7f1c47fff700 1 --2- 172.21.15.5:0/1799137795 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f0b62b06a0 0x55f0b62d0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:08.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.119+0000 7f1c477fe700 1 -- 172.21.15.5:0/1799137795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f0b621d160 msgr2=0x55f0b62c2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:08.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.119+0000 7f1c477fe700 1 --2- 172.21.15.5:0/1799137795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f0b621d160 0x55f0b62c2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:08.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.119+0000 7f1c477fe700 1 -- 172.21.15.5:0/1799137795 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f0b62b06a0 msgr2=0x55f0b62d0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:08.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.119+0000 7f1c477fe700 1 --2- 172.21.15.5:0/1799137795 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f0b62b06a0 0x55f0b62d0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:08.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.119+0000 7f1c477fe700 1 -- 172.21.15.5:0/1799137795 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0b62a4f60 con 0x55f0b61334a0 2024-05-09T00:48:08.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.119+0000 7f1c477fe700 1 --2- 172.21.15.5:0/1799137795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0b61334a0 0x55f0b62c2350 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1c38023f50 tx=0x7f1c3800b850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=720c1139244ee2b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:08.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.119+0000 7f1c26ffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:48:08.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.120+0000 7f1c26ffd700 1 -- 172.21.15.5:0/1799137795 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c3803d060 con 0x55f0b61334a0 2024-05-09T00:48:08.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.120+0000 7f1c26ffd700 1 -- 172.21.15.5:0/1799137795 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f1c38005380 con 0x55f0b61334a0 2024-05-09T00:48:08.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.120+0000 7f1c5c5f4b80 1 -- 172.21.15.5:0/1799137795 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c30003e50 con 0x55f0b61334a0 2024-05-09T00:48:08.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.120+0000 7f1c26ffd700 1 -- 172.21.15.5:0/1799137795 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1c380095c0 con 0x55f0b61334a0 2024-05-09T00:48:08.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.120+0000 7f1c5c5f4b80 10 client.15432 fetch_fsmap learned FSMap version 29 2024-05-09T00:48:08.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.120+0000 7f1c5c5f4b80 1 -- 172.21.15.5:0/1799137795 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f0b6333fb0 con 0x55f0b61334a0 2024-05-09T00:48:08.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.120+0000 7f1c26ffd700 1 -- 172.21.15.5:0/1799137795 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c30003e50 con 0x55f0b61334a0 2024-05-09T00:48:08.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.120+0000 7f1c26ffd700 1 -- 172.21.15.5:0/1799137795 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f1c380236c0 con 0x55f0b61334a0 2024-05-09T00:48:08.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.120+0000 7f1c5c5f4b80 10 client.15432 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:48:08.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.120+0000 7f1c5c5f4b80 10 client.15432 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-05-09T00:48:08.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.120+0000 7f1c5c5f4b80 20 client.15432 populate_metadata read hostname 'smithi005' 2024-05-09T00:48:08.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.120+0000 7f1c5c5f4b80 1 --2- 172.21.15.5:0/1799137795 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x55f0b6334630 0x55f0b6336a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:08.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.120+0000 7f1c5c5f4b80 4 client.15432 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:48:08.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.120+0000 7f1c5c5f4b80 1 -- 172.21.15.5:0/1799137795 --> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f0b61daa70 con 0x55f0b6334630 2024-05-09T00:48:08.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.121+0000 7f1c3effd700 1 --2- 172.21.15.5:0/1799137795 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x55f0b6334630 0x55f0b6336a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:08.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.121+0000 7f1c26ffd700 10 client.15432 ms_handle_connect on v2:172.21.15.131:6838/2748669631 2024-05-09T00:48:08.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.121+0000 7f1c3effd700 1 --2- 172.21.15.5:0/1799137795 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x55f0b6334630 0x55f0b6336a50 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=18efcb6dea6a7307 server_cookie=fe9e4b9f9a789390 in_seq=0 out_seq=0 2024-05-09T00:48:08.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.121+0000 7f1c26ffd700 1 -- 172.21.15.5:0/1799137795 <== mds.2 v2:172.21.15.131:6838/2748669631 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f0b61daa70 con 0x55f0b6334630 2024-05-09T00:48:08.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.121+0000 7f1c26ffd700 10 client.15432 handle_command_reply: tid=0 2024-05-09T00:48:08.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.121+0000 7f1c5c5f4b80 1 client.15432 shutdown 2024-05-09T00:48:08.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.121+0000 7f1c5c5f4b80 1 -- 172.21.15.5:0/1799137795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0b61334a0 msgr2=0x55f0b62c2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:08.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.121+0000 7f1c5c5f4b80 1 --2- 172.21.15.5:0/1799137795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0b61334a0 0x55f0b62c2350 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1c38023f50 tx=0x7f1c3800b850 comp rx=0 tx=0).stop 2024-05-09T00:48:08.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.121+0000 7f1c5c5f4b80 1 -- 172.21.15.5:0/1799137795 shutdown_connections 2024-05-09T00:48:08.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.121+0000 7f1c5c5f4b80 1 --2- 172.21.15.5:0/1799137795 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x55f0b6334630 0x55f0b6336a50 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:08.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.121+0000 7f1c5c5f4b80 1 --2- 172.21.15.5:0/1799137795 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f0b62b06a0 0x55f0b62d0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:08.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.121+0000 7f1c5c5f4b80 1 --2- 172.21.15.5:0/1799137795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0b61334a0 0x55f0b62c2350 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:08.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.121+0000 7f1c5c5f4b80 1 --2- 172.21.15.5:0/1799137795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f0b621d160 0x55f0b62c2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:08.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.121+0000 7f1c5c5f4b80 1 -- 172.21.15.5:0/1799137795 >> 172.21.15.5:0/1799137795 conn(0x55f0b61936a0 msgr2=0x55f0b62b3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:08.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.121+0000 7f1c26ffd700 0 client.15432 ms_handle_reset on v2:172.21.15.131:6838/2748669631 2024-05-09T00:48:08.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.122+0000 7f1c5c5f4b80 1 -- 172.21.15.5:0/1799137795 shutdown_connections 2024-05-09T00:48:08.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.122+0000 7f1c5c5f4b80 1 -- 172.21.15.5:0/1799137795 wait complete. 2024-05-09T00:48:08.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.122+0000 7f1c5c5f4b80 20 client.15432 trim_cache size 0 max 16384 2024-05-09T00:48:08.127 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-09T00:48:08.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.122+0000 7f1c4dbde700 1 -- 172.21.15.5:0/1826805236 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f1c340b2880 msgr2=0x7f1c340b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:08.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.122+0000 7f1c4dbde700 1 --2- 172.21.15.5:0/1826805236 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f1c340b2880 0x7f1c340b4d90 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1c4806c680 tx=0x7f1c3000bbf0 comp rx=0 tx=0).stop 2024-05-09T00:48:08.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.122+0000 7f1c4dbde700 1 -- 172.21.15.5:0/1826805236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c48109870 msgr2=0x7f1c48076540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:08.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.122+0000 7f1c4dbde700 1 --2- 172.21.15.5:0/1826805236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c48109870 0x7f1c48076540 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f1c40007d70 tx=0x7f1c4000a340 comp rx=0 tx=0).stop 2024-05-09T00:48:08.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.123+0000 7f1c4dbde700 1 -- 172.21.15.5:0/1826805236 shutdown_connections 2024-05-09T00:48:08.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.123+0000 7f1c4dbde700 1 --2- 172.21.15.5:0/1826805236 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f1c340b2880 0x7f1c340b4d90 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:08.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.123+0000 7f1c4dbde700 1 --2- 172.21.15.5:0/1826805236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c48109870 0x7f1c48076540 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:08.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.123+0000 7f1c4dbde700 1 --2- 172.21.15.5:0/1826805236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1c481045f0 0x7f1c48075ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:08.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.123+0000 7f1c4dbde700 1 --2- 172.21.15.5:0/1826805236 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c48108f20 0x7f1c48076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:08.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.123+0000 7f1c4dbde700 1 -- 172.21.15.5:0/1826805236 >> 172.21.15.5:0/1826805236 conn(0x7f1c48100040 msgr2=0x7f1c4810a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:08.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.123+0000 7f1c4dbde700 1 -- 172.21.15.5:0/1826805236 shutdown_connections 2024-05-09T00:48:08.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:08.123+0000 7f1c4dbde700 1 -- 172.21.15.5:0/1826805236 wait complete. 2024-05-09T00:48:08.808 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T00:48:09.074 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:48:09.466 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:48:09 smithi005 ceph-mon[30365]: pgmap v425: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-05-09T00:48:09.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:48:09 smithi131 ceph-mon[34547]: pgmap v425: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-05-09T00:48:09.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:48:09 smithi157 ceph-mon[34294]: pgmap v425: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-05-09T00:48:10.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.940+0000 7fccff701700 1 -- 172.21.15.5:0/889648222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccf8107df0 msgr2=0x7fccf8108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:10.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.940+0000 7fccff701700 1 --2- 172.21.15.5:0/889648222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccf8107df0 0x7fccf8108200 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fcce80099c0 tx=0x7fcce8009cd0 comp rx=0 tx=0).stop 2024-05-09T00:48:10.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.940+0000 7fccff701700 1 -- 172.21.15.5:0/889648222 shutdown_connections 2024-05-09T00:48:10.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.940+0000 7fccff701700 1 --2- 172.21.15.5:0/889648222 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccf8108740 0x7fccf81100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:10.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.940+0000 7fccff701700 1 --2- 172.21.15.5:0/889648222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccf81034c0 0x7fccf81038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:10.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.940+0000 7fccff701700 1 --2- 172.21.15.5:0/889648222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccf8107df0 0x7fccf8108200 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:10.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.940+0000 7fccff701700 1 -- 172.21.15.5:0/889648222 >> 172.21.15.5:0/889648222 conn(0x7fccf80fedc0 msgr2=0x7fccf81011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:10.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.940+0000 7fccff701700 1 -- 172.21.15.5:0/889648222 shutdown_connections 2024-05-09T00:48:10.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.940+0000 7fccff701700 1 -- 172.21.15.5:0/889648222 wait complete. 2024-05-09T00:48:10.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.941+0000 7fccff701700 1 Processor -- start 2024-05-09T00:48:10.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.941+0000 7fccff701700 1 -- start start 2024-05-09T00:48:10.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.942+0000 7fccff701700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccf81034c0 0x7fccf819d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:10.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.942+0000 7fccff701700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccf8107df0 0x7fccf819d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:10.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.942+0000 7fccff701700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccf8108740 0x7fccf81a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:10.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.942+0000 7fccff701700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fccf81133a0 con 0x7fccf8108740 2024-05-09T00:48:10.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.942+0000 7fccff701700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fccf8113220 con 0x7fccf8107df0 2024-05-09T00:48:10.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.942+0000 7fccff701700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fccf8113520 con 0x7fccf81034c0 2024-05-09T00:48:10.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.942+0000 7fccfdc9e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccf8108740 0x7fccf81a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:10.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.942+0000 7fccfd49d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccf81034c0 0x7fccf819d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:10.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.942+0000 7fccfdc9e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccf8108740 0x7fccf81a1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33902/0 (socket says 172.21.15.5:33902) 2024-05-09T00:48:10.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.942+0000 7fccfdc9e700 1 -- 172.21.15.5:0/2389912880 learned_addr learned my addr 172.21.15.5:0/2389912880 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:10.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.942+0000 7fccfcc9c700 1 --2- 172.21.15.5:0/2389912880 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccf8107df0 0x7fccf819d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:10.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.942+0000 7fccfdc9e700 1 -- 172.21.15.5:0/2389912880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccf81034c0 msgr2=0x7fccf819d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:10.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.942+0000 7fccfdc9e700 1 --2- 172.21.15.5:0/2389912880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccf81034c0 0x7fccf819d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:10.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.942+0000 7fccfdc9e700 1 -- 172.21.15.5:0/2389912880 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccf8107df0 msgr2=0x7fccf819d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:10.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.942+0000 7fccfdc9e700 1 --2- 172.21.15.5:0/2389912880 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccf8107df0 0x7fccf819d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:10.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.943+0000 7fccfdc9e700 1 -- 172.21.15.5:0/2389912880 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccf81a1f00 con 0x7fccf8108740 2024-05-09T00:48:10.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.943+0000 7fccfd49d700 1 --2- 172.21.15.5:0/2389912880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccf81034c0 0x7fccf819d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:48:10.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.943+0000 7fccfdc9e700 1 --2- 172.21.15.5:0/2389912880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccf8108740 0x7fccf81a1860 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fccf000ef30 tx=0x7fccf000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:10.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.943+0000 7fccee7fc700 1 -- 172.21.15.5:0/2389912880 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf00092e0 con 0x7fccf8108740 2024-05-09T00:48:10.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.943+0000 7fccff701700 1 -- 172.21.15.5:0/2389912880 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccf81a2190 con 0x7fccf8108740 2024-05-09T00:48:10.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.943+0000 7fccee7fc700 1 -- 172.21.15.5:0/2389912880 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccf0004bb0 con 0x7fccf8108740 2024-05-09T00:48:10.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.943+0000 7fccee7fc700 1 -- 172.21.15.5:0/2389912880 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf0004020 con 0x7fccf8108740 2024-05-09T00:48:10.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.943+0000 7fccff701700 1 -- 172.21.15.5:0/2389912880 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccf81a26a0 con 0x7fccf8108740 2024-05-09T00:48:10.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.946+0000 7fccee7fc700 1 -- 172.21.15.5:0/2389912880 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fccf001b030 con 0x7fccf8108740 2024-05-09T00:48:10.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.946+0000 7fccee7fc700 1 --2- 172.21.15.5:0/2389912880 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fcce40b2880 0x7fcce40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:10.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.947+0000 7fccfd49d700 1 --2- 172.21.15.5:0/2389912880 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fcce40b2880 0x7fcce40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:10.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.947+0000 7fccee7fc700 1 -- 172.21.15.5:0/2389912880 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fccf00e1c70 con 0x7fccf8108740 2024-05-09T00:48:10.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.947+0000 7fccfd49d700 1 --2- 172.21.15.5:0/2389912880 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fcce40b2880 0x7fcce40b4d90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fccf4011490 tx=0x7fccf4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:10.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.948+0000 7fcd0e117b80 1 Processor -- start 2024-05-09T00:48:10.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.948+0000 7fcd0e117b80 1 -- start start 2024-05-09T00:48:10.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.949+0000 7fcd0e117b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4012f4d00 0x55e4013fd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:10.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.949+0000 7fcd0e117b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e40127a4a0 0x55e4013fd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:10.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.949+0000 7fcd0e117b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e4013f7400 0x55e4013f7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:10.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.949+0000 7fcd0e117b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e40131a0a0 con 0x55e4012f4d00 2024-05-09T00:48:10.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.949+0000 7fcd0e117b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55e4012bf9a0 con 0x55e40127a4a0 2024-05-09T00:48:10.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.949+0000 7fcd0e117b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55e401316060 con 0x55e4013f7400 2024-05-09T00:48:10.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.949+0000 7fccfd49d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4012f4d00 0x55e4013fd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:10.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.949+0000 7fccfd49d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4012f4d00 0x55e4013fd240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33916/0 (socket says 172.21.15.5:33916) 2024-05-09T00:48:10.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.949+0000 7fccfd49d700 1 -- 172.21.15.5:0/3574560866 learned_addr learned my addr 172.21.15.5:0/3574560866 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:10.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.949+0000 7fccfcc9c700 1 --2- 172.21.15.5:0/3574560866 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e40127a4a0 0x55e4013fd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:10.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.949+0000 7fccfdc9e700 1 --2- 172.21.15.5:0/3574560866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e4013f7400 0x55e4013f7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:10.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.949+0000 7fccfd49d700 1 -- 172.21.15.5:0/3574560866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e4013f7400 msgr2=0x55e4013f7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:10.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.949+0000 7fccfd49d700 1 --2- 172.21.15.5:0/3574560866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e4013f7400 0x55e4013f7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:10.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.949+0000 7fccfd49d700 1 -- 172.21.15.5:0/3574560866 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e40127a4a0 msgr2=0x55e4013fd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:10.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.949+0000 7fccfd49d700 1 --2- 172.21.15.5:0/3574560866 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e40127a4a0 0x55e4013fd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:10.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.949+0000 7fccfd49d700 1 -- 172.21.15.5:0/3574560866 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4012b0500 con 0x55e4012f4d00 2024-05-09T00:48:10.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.949+0000 7fccfdc9e700 1 --2- 172.21.15.5:0/3574560866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e4013f7400 0x55e4013f7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:48:10.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.950+0000 7fccfd49d700 1 --2- 172.21.15.5:0/3574560866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4012f4d00 0x55e4013fd240 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fccf40043c0 tx=0x7fccf4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4158f03366d397d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:10.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.950+0000 7fcce37fe700 1 -- 172.21.15.5:0/3574560866 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf4003d10 con 0x55e4012f4d00 2024-05-09T00:48:10.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.950+0000 7fcce37fe700 1 -- 172.21.15.5:0/3574560866 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccf4011e50 con 0x55e4012f4d00 2024-05-09T00:48:10.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.950+0000 7fcce37fe700 1 -- 172.21.15.5:0/3574560866 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf401e6e0 con 0x55e4012f4d00 2024-05-09T00:48:10.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.951+0000 7fcd0e117b80 1 -- 172.21.15.5:0/3574560866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4012f4d00 msgr2=0x55e4013fd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:10.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.951+0000 7fcd0e117b80 1 --2- 172.21.15.5:0/3574560866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4012f4d00 0x55e4013fd240 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fccf40043c0 tx=0x7fccf4014000 comp rx=0 tx=0).stop 2024-05-09T00:48:10.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.951+0000 7fcd0e117b80 1 -- 172.21.15.5:0/3574560866 shutdown_connections 2024-05-09T00:48:10.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.951+0000 7fcd0e117b80 1 --2- 172.21.15.5:0/3574560866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e4013f7400 0x55e4013f7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:10.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.951+0000 7fcd0e117b80 1 --2- 172.21.15.5:0/3574560866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4012f4d00 0x55e4013fd240 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:10.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.951+0000 7fcd0e117b80 1 --2- 172.21.15.5:0/3574560866 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e40127a4a0 0x55e4013fd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.951+0000 7fcd0e117b80 1 -- 172.21.15.5:0/3574560866 >> 172.21.15.5:0/3574560866 conn(0x55e40127df70 msgr2=0x55e4013f2220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.951+0000 7fcd0e117b80 1 -- 172.21.15.5:0/3574560866 shutdown_connections 2024-05-09T00:48:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.951+0000 7fcd0e117b80 1 -- 172.21.15.5:0/3574560866 wait complete. 2024-05-09T00:48:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.952+0000 7fcd0e117b80 1 -- start start 2024-05-09T00:48:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.952+0000 7fcd0e117b80 1 Processor -- start 2024-05-09T00:48:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.953+0000 7fcd0e117b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e40127a4a0 0x55e401400ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.953+0000 7fcd0e117b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e4012f4d00 0x55e401401530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.953+0000 7fcd0e117b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4013f7400 0x55e401401a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.953+0000 7fcd0e117b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e4013f8160 con 0x55e4013f7400 2024-05-09T00:48:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.953+0000 7fccfcc9c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e4012f4d00 0x55e401401530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.953+0000 7fccfdc9e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4013f7400 0x55e401401a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.953+0000 7fccfcc9c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e4012f4d00 0x55e401401530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:56592/0 (socket says 172.21.15.5:56592) 2024-05-09T00:48:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.953+0000 7fccfcc9c700 1 -- 172.21.15.5:0/1741825169 learned_addr learned my addr 172.21.15.5:0/1741825169 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.953+0000 7fcd0e117b80 1 -- 172.21.15.5:0/1741825169 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55e4012b4b20 con 0x55e40127a4a0 2024-05-09T00:48:10.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.953+0000 7fccfd49d700 1 --2- 172.21.15.5:0/1741825169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e40127a4a0 0x55e401400ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:10.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.953+0000 7fcd0e117b80 1 -- 172.21.15.5:0/1741825169 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55e401404020 con 0x55e4012f4d00 2024-05-09T00:48:10.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.953+0000 7fcd0e117b80 10 client.0 _ll_register_callbacks cb 0x55e40118b6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:48:10.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.953+0000 7fccfdc9e700 1 -- 172.21.15.5:0/1741825169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e4012f4d00 msgr2=0x55e401401530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:10.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.953+0000 7fccfdc9e700 1 --2- 172.21.15.5:0/1741825169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e4012f4d00 0x55e401401530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:10.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.953+0000 7fccfdc9e700 1 -- 172.21.15.5:0/1741825169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e40127a4a0 msgr2=0x55e401400ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:10.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.953+0000 7fccfdc9e700 1 --2- 172.21.15.5:0/1741825169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e40127a4a0 0x55e401400ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:10.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.953+0000 7fccfdc9e700 1 -- 172.21.15.5:0/1741825169 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e401195490 con 0x55e4013f7400 2024-05-09T00:48:10.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.954+0000 7fccfdc9e700 1 --2- 172.21.15.5:0/1741825169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4013f7400 0x55e401401a70 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fccf0009510 tx=0x7fccf0004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=203f4b7635e9d67c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:10.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.954+0000 7fcce0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:48:10.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.954+0000 7fcce0ff9700 1 -- 172.21.15.5:0/1741825169 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf00c3060 con 0x55e4013f7400 2024-05-09T00:48:10.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.954+0000 7fcce0ff9700 1 -- 172.21.15.5:0/1741825169 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fccf00a8760 con 0x55e4013f7400 2024-05-09T00:48:10.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.954+0000 7fcd0e117b80 1 -- 172.21.15.5:0/1741825169 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccf4011e50 con 0x55e4013f7400 2024-05-09T00:48:10.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.954+0000 7fcce0ff9700 1 -- 172.21.15.5:0/1741825169 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fccf00a99f0 con 0x55e4013f7400 2024-05-09T00:48:10.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.954+0000 7fcd0e117b80 10 client.15444 fetch_fsmap learned FSMap version 29 2024-05-09T00:48:10.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.954+0000 7fcd0e117b80 1 -- 172.21.15.5:0/1741825169 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e40140e970 con 0x55e4013f7400 2024-05-09T00:48:10.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.954+0000 7fcce0ff9700 1 -- 172.21.15.5:0/1741825169 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccf4011e50 con 0x55e4013f7400 2024-05-09T00:48:10.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.954+0000 7fcce0ff9700 1 -- 172.21.15.5:0/1741825169 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fccf00a4c60 con 0x55e4013f7400 2024-05-09T00:48:10.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.955+0000 7fcd0e117b80 10 client.15444 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:48:10.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.955+0000 7fcd0e117b80 10 client.15444 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-05-09T00:48:10.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.955+0000 7fcd0e117b80 20 client.15444 populate_metadata read hostname 'smithi005' 2024-05-09T00:48:10.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.955+0000 7fcd0e117b80 1 --2- 172.21.15.5:0/1741825169 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x55e401410f00 0x55e401413360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:10.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.955+0000 7fcd0e117b80 4 client.15444 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:48:10.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.955+0000 7fcd0e117b80 1 -- 172.21.15.5:0/1741825169 --> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e401327a80 con 0x55e401410f00 2024-05-09T00:48:10.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.955+0000 7fccfd49d700 1 --2- 172.21.15.5:0/1741825169 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x55e401410f00 0x55e401413360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:10.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.955+0000 7fcce0ff9700 10 client.15444 ms_handle_connect on v2:172.21.15.157:6836/744863009 2024-05-09T00:48:10.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.956+0000 7fccfd49d700 1 --2- 172.21.15.5:0/1741825169 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x55e401410f00 0x55e401413360 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b1c3c5ae90c23379 server_cookie=31e1115196ec6c85 in_seq=0 out_seq=0 2024-05-09T00:48:10.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.956+0000 7fcce0ff9700 1 -- 172.21.15.5:0/1741825169 <== mds.1 v2:172.21.15.157:6836/744863009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e401327a80 con 0x55e401410f00 2024-05-09T00:48:10.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.956+0000 7fcce0ff9700 10 client.15444 handle_command_reply: tid=0 2024-05-09T00:48:10.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.956+0000 7fcd0e117b80 1 client.15444 shutdown 2024-05-09T00:48:10.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.957+0000 7fcd0e117b80 1 -- 172.21.15.5:0/1741825169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4013f7400 msgr2=0x55e401401a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:10.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.957+0000 7fcd0e117b80 1 --2- 172.21.15.5:0/1741825169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4013f7400 0x55e401401a70 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fccf0009510 tx=0x7fccf0004f90 comp rx=0 tx=0).stop 2024-05-09T00:48:10.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.957+0000 7fcd0e117b80 1 -- 172.21.15.5:0/1741825169 shutdown_connections 2024-05-09T00:48:10.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.957+0000 7fcd0e117b80 1 --2- 172.21.15.5:0/1741825169 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x55e401410f00 0x55e401413360 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:10.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.957+0000 7fcd0e117b80 1 --2- 172.21.15.5:0/1741825169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4013f7400 0x55e401401a70 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:10.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.957+0000 7fcd0e117b80 1 --2- 172.21.15.5:0/1741825169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e40127a4a0 0x55e401400ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:10.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.957+0000 7fcd0e117b80 1 --2- 172.21.15.5:0/1741825169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e4012f4d00 0x55e401401530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:10.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.957+0000 7fcd0e117b80 1 -- 172.21.15.5:0/1741825169 >> 172.21.15.5:0/1741825169 conn(0x55e40127df70 msgr2=0x55e4013ebff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:10.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.957+0000 7fcce0ff9700 0 client.15444 ms_handle_reset on v2:172.21.15.157:6836/744863009 2024-05-09T00:48:10.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.957+0000 7fcd0e117b80 1 -- 172.21.15.5:0/1741825169 shutdown_connections 2024-05-09T00:48:10.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.957+0000 7fcd0e117b80 1 -- 172.21.15.5:0/1741825169 wait complete. 2024-05-09T00:48:10.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.957+0000 7fcd0e117b80 20 client.15444 trim_cache size 0 max 16384 2024-05-09T00:48:10.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.973+0000 7fcd0e117b80 1 Processor -- start 2024-05-09T00:48:10.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.973+0000 7fcd0e117b80 1 -- start start 2024-05-09T00:48:10.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.973+0000 7fcd0e117b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e40127a4a0 0x55e4013fb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:10.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.973+0000 7fcd0e117b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e4012f4d00 0x55e40140ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:10.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.973+0000 7fcd0e117b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e4013f7400 0x55e401418b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:10.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.973+0000 7fcd0e117b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fccf81b0f90 con 0x55e40127a4a0 2024-05-09T00:48:10.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.973+0000 7fcd0e117b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fccf81b2ad0 con 0x55e4012f4d00 2024-05-09T00:48:10.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.973+0000 7fcd0e117b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fccf81b1660 con 0x55e4013f7400 2024-05-09T00:48:10.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.973+0000 7fccfd49d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e40127a4a0 0x55e4013fb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:10.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.974+0000 7fccfd49d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e40127a4a0 0x55e4013fb3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33934/0 (socket says 172.21.15.5:33934) 2024-05-09T00:48:10.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.974+0000 7fccfd49d700 1 -- 172.21.15.5:0/1017069697 learned_addr learned my addr 172.21.15.5:0/1017069697 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:10.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.974+0000 7fccfcc9c700 1 --2- 172.21.15.5:0/1017069697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e4012f4d00 0x55e40140ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:10.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.974+0000 7fccfdc9e700 1 --2- 172.21.15.5:0/1017069697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e4013f7400 0x55e401418b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:10.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.974+0000 7fccfd49d700 1 -- 172.21.15.5:0/1017069697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e4013f7400 msgr2=0x55e401418b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:10.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.974+0000 7fccfd49d700 1 --2- 172.21.15.5:0/1017069697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e4013f7400 0x55e401418b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:10.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.974+0000 7fccfd49d700 1 -- 172.21.15.5:0/1017069697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e4012f4d00 msgr2=0x55e40140ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:10.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.974+0000 7fccfd49d700 1 --2- 172.21.15.5:0/1017069697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e4012f4d00 0x55e40140ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:10.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.974+0000 7fccfd49d700 1 -- 172.21.15.5:0/1017069697 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4013f2cf0 con 0x55e40127a4a0 2024-05-09T00:48:10.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.974+0000 7fccfcc9c700 1 --2- 172.21.15.5:0/1017069697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e4012f4d00 0x55e40140ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:48:10.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.974+0000 7fccfd49d700 1 --2- 172.21.15.5:0/1017069697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e40127a4a0 0x55e4013fb3e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fccf4026ee0 tx=0x7fccf4028040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca8b8c8975445c7f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:10.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.974+0000 7fcce2ffd700 1 -- 172.21.15.5:0/1017069697 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf403a070 con 0x55e40127a4a0 2024-05-09T00:48:10.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.974+0000 7fcce2ffd700 1 -- 172.21.15.5:0/1017069697 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccf40268b0 con 0x55e40127a4a0 2024-05-09T00:48:10.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.974+0000 7fcce2ffd700 1 -- 172.21.15.5:0/1017069697 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf4012e60 con 0x55e40127a4a0 2024-05-09T00:48:10.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.975+0000 7fcd0e117b80 1 -- 172.21.15.5:0/1017069697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e40127a4a0 msgr2=0x55e4013fb3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:10.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.975+0000 7fcd0e117b80 1 --2- 172.21.15.5:0/1017069697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e40127a4a0 0x55e4013fb3e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fccf4026ee0 tx=0x7fccf4028040 comp rx=0 tx=0).stop 2024-05-09T00:48:10.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.975+0000 7fcd0e117b80 1 -- 172.21.15.5:0/1017069697 shutdown_connections 2024-05-09T00:48:10.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.975+0000 7fcd0e117b80 1 --2- 172.21.15.5:0/1017069697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e4013f7400 0x55e401418b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:10.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.975+0000 7fcd0e117b80 1 --2- 172.21.15.5:0/1017069697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e40127a4a0 0x55e4013fb3e0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:10.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.975+0000 7fcd0e117b80 1 --2- 172.21.15.5:0/1017069697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e4012f4d00 0x55e40140ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:10.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.975+0000 7fcd0e117b80 1 -- 172.21.15.5:0/1017069697 >> 172.21.15.5:0/1017069697 conn(0x55e40127df70 msgr2=0x55e4013facd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:10.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.975+0000 7fcd0e117b80 1 -- 172.21.15.5:0/1017069697 shutdown_connections 2024-05-09T00:48:10.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.975+0000 7fcd0e117b80 1 -- 172.21.15.5:0/1017069697 wait complete. 2024-05-09T00:48:10.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.976+0000 7fcd0e117b80 1 -- start start 2024-05-09T00:48:10.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.976+0000 7fcd0e117b80 1 Processor -- start 2024-05-09T00:48:10.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.977+0000 7fcd0e117b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e40127a4a0 0x55e4014091a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:10.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.977+0000 7fcd0e117b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e4012f4d00 0x55e4014096e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:10.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.977+0000 7fcd0e117b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e4013f7400 0x55e401417b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:10.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.977+0000 7fcd0e117b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fccf81b2400 con 0x55e40127a4a0 2024-05-09T00:48:10.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.977+0000 7fccfd49d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e40127a4a0 0x55e4014091a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:10.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.977+0000 7fccfd49d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e40127a4a0 0x55e4014091a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33936/0 (socket says 172.21.15.5:33936) 2024-05-09T00:48:10.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.977+0000 7fccfd49d700 1 -- 172.21.15.5:0/3308062417 learned_addr learned my addr 172.21.15.5:0/3308062417 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:10.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.977+0000 7fcd0e117b80 1 -- 172.21.15.5:0/3308062417 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fccf81b1d30 con 0x55e4012f4d00 2024-05-09T00:48:10.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.977+0000 7fcd0e117b80 1 -- 172.21.15.5:0/3308062417 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fccf81aef30 con 0x55e4013f7400 2024-05-09T00:48:10.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.977+0000 7fccfdc9e700 1 --2- 172.21.15.5:0/3308062417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e4013f7400 0x55e401417b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:10.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.977+0000 7fcd0e117b80 10 client.0 _ll_register_callbacks cb 0x55e4013fccf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:48:10.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.977+0000 7fccfcc9c700 1 --2- 172.21.15.5:0/3308062417 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e4012f4d00 0x55e4014096e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:10.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.977+0000 7fccfd49d700 1 -- 172.21.15.5:0/3308062417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e4013f7400 msgr2=0x55e401417b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:10.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.977+0000 7fccfd49d700 1 --2- 172.21.15.5:0/3308062417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e4013f7400 0x55e401417b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:10.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.977+0000 7fccfd49d700 1 -- 172.21.15.5:0/3308062417 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e4012f4d00 msgr2=0x55e4014096e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:10.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.977+0000 7fccfd49d700 1 --2- 172.21.15.5:0/3308062417 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e4012f4d00 0x55e4014096e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:10.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.977+0000 7fccfd49d700 1 -- 172.21.15.5:0/3308062417 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e401419520 con 0x55e40127a4a0 2024-05-09T00:48:10.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.977+0000 7fccfd49d700 1 --2- 172.21.15.5:0/3308062417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e40127a4a0 0x55e4014091a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fccf4026eb0 tx=0x7fccf4003f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=206787cf586cd35f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:10.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.977+0000 7fcce0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:48:10.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.978+0000 7fcce0ff9700 1 -- 172.21.15.5:0/3308062417 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf403a030 con 0x55e40127a4a0 2024-05-09T00:48:10.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.978+0000 7fcd0e117b80 1 -- 172.21.15.5:0/3308062417 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccf40268b0 con 0x55e40127a4a0 2024-05-09T00:48:10.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.978+0000 7fcce0ff9700 1 -- 172.21.15.5:0/3308062417 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fccf4026a60 con 0x55e40127a4a0 2024-05-09T00:48:10.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.978+0000 7fcce0ff9700 1 -- 172.21.15.5:0/3308062417 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fccf4026c60 con 0x55e40127a4a0 2024-05-09T00:48:10.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.978+0000 7fcd0e117b80 10 client.15450 fetch_fsmap learned FSMap version 29 2024-05-09T00:48:10.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.978+0000 7fcd0e117b80 1 -- 172.21.15.5:0/3308062417 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e40147ac00 con 0x55e40127a4a0 2024-05-09T00:48:10.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.978+0000 7fcce0ff9700 1 -- 172.21.15.5:0/3308062417 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccf40268b0 con 0x55e40127a4a0 2024-05-09T00:48:10.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.978+0000 7fcce0ff9700 1 -- 172.21.15.5:0/3308062417 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fccf4033ce0 con 0x55e40127a4a0 2024-05-09T00:48:10.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.978+0000 7fcd0e117b80 10 client.15450 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:48:10.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.978+0000 7fcd0e117b80 10 client.15450 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-05-09T00:48:10.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.978+0000 7fcd0e117b80 20 client.15450 populate_metadata read hostname 'smithi005' 2024-05-09T00:48:10.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.978+0000 7fcd0e117b80 1 --2- 172.21.15.5:0/3308062417 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x55e40147af20 0x55e40147d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:10.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.978+0000 7fcd0e117b80 4 client.15450 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:48:10.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.978+0000 7fcd0e117b80 1 -- 172.21.15.5:0/3308062417 --> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e401321b20 con 0x55e40147af20 2024-05-09T00:48:10.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.979+0000 7fccfcc9c700 1 --2- 172.21.15.5:0/3308062417 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x55e40147af20 0x55e40147d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:10.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.979+0000 7fcce0ff9700 10 client.15450 ms_handle_connect on v2:172.21.15.157:6836/744863009 2024-05-09T00:48:10.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.979+0000 7fccfcc9c700 1 --2- 172.21.15.5:0/3308062417 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x55e40147af20 0x55e40147d340 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4d5f5c0ea9ad5da7 server_cookie=b714fd08a98f8897 in_seq=0 out_seq=0 2024-05-09T00:48:10.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.979+0000 7fcce0ff9700 1 -- 172.21.15.5:0/3308062417 <== mds.1 v2:172.21.15.157:6836/744863009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e401321b20 con 0x55e40147af20 2024-05-09T00:48:10.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.979+0000 7fcce0ff9700 10 client.15450 handle_command_reply: tid=0 2024-05-09T00:48:10.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.979+0000 7fcd0e117b80 1 client.15450 shutdown 2024-05-09T00:48:10.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.980+0000 7fcd0e117b80 1 -- 172.21.15.5:0/3308062417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e40127a4a0 msgr2=0x55e4014091a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:10.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.980+0000 7fcd0e117b80 1 --2- 172.21.15.5:0/3308062417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e40127a4a0 0x55e4014091a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fccf4026eb0 tx=0x7fccf4003f70 comp rx=0 tx=0).stop 2024-05-09T00:48:10.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.980+0000 7fcd0e117b80 1 -- 172.21.15.5:0/3308062417 shutdown_connections 2024-05-09T00:48:10.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.980+0000 7fcd0e117b80 1 --2- 172.21.15.5:0/3308062417 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x55e40147af20 0x55e40147d340 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:10.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.980+0000 7fcd0e117b80 1 --2- 172.21.15.5:0/3308062417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e4013f7400 0x55e401417b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:10.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.980+0000 7fcd0e117b80 1 --2- 172.21.15.5:0/3308062417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e40127a4a0 0x55e4014091a0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:10.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.980+0000 7fcd0e117b80 1 --2- 172.21.15.5:0/3308062417 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e4012f4d00 0x55e4014096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:10.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.980+0000 7fcce0ff9700 0 client.15450 ms_handle_reset on v2:172.21.15.157:6836/744863009 2024-05-09T00:48:10.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.980+0000 7fcd0e117b80 1 -- 172.21.15.5:0/3308062417 >> 172.21.15.5:0/3308062417 conn(0x55e40127df70 msgr2=0x55e4013ebff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:10.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.980+0000 7fcd0e117b80 1 -- 172.21.15.5:0/3308062417 shutdown_connections 2024-05-09T00:48:10.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.980+0000 7fcd0e117b80 1 -- 172.21.15.5:0/3308062417 wait complete. 2024-05-09T00:48:10.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.980+0000 7fcd0e117b80 20 client.15450 trim_cache size 0 max 16384 2024-05-09T00:48:10.985 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-09T00:48:10.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.980+0000 7fccff701700 1 -- 172.21.15.5:0/2389912880 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fcce40b2880 msgr2=0x7fcce40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:10.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.980+0000 7fccff701700 1 --2- 172.21.15.5:0/2389912880 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fcce40b2880 0x7fcce40b4d90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fccf4011490 tx=0x7fccf4009260 comp rx=0 tx=0).stop 2024-05-09T00:48:10.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.980+0000 7fccff701700 1 -- 172.21.15.5:0/2389912880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccf8108740 msgr2=0x7fccf81a1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:10.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.980+0000 7fccff701700 1 --2- 172.21.15.5:0/2389912880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccf8108740 0x7fccf81a1860 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fccf000ef30 tx=0x7fccf000c3a0 comp rx=0 tx=0).stop 2024-05-09T00:48:10.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.981+0000 7fccff701700 1 -- 172.21.15.5:0/2389912880 shutdown_connections 2024-05-09T00:48:10.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.981+0000 7fccff701700 1 --2- 172.21.15.5:0/2389912880 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fcce40b2880 0x7fcce40b4d90 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:10.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.981+0000 7fccff701700 1 --2- 172.21.15.5:0/2389912880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccf8108740 0x7fccf81a1860 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:10.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.981+0000 7fccff701700 1 --2- 172.21.15.5:0/2389912880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccf81034c0 0x7fccf819d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:10.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.981+0000 7fccff701700 1 --2- 172.21.15.5:0/2389912880 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccf8107df0 0x7fccf819d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:10.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.981+0000 7fccff701700 1 -- 172.21.15.5:0/2389912880 >> 172.21.15.5:0/2389912880 conn(0x7fccf80fedc0 msgr2=0x7fccf810ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:10.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.981+0000 7fccff701700 1 -- 172.21.15.5:0/2389912880 shutdown_connections 2024-05-09T00:48:10.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:10.981+0000 7fccff701700 1 -- 172.21.15.5:0/2389912880 wait complete. 2024-05-09T00:48:11.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.168+0000 7f537b95e640 1 -- 172.21.15.5:0/3032347886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5374108a90 msgr2=0x7f537410aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:11.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.168+0000 7f537b95e640 1 --2- 172.21.15.5:0/3032347886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5374108a90 0x7f537410aeb0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f536c009ca0 tx=0x7f536c02a8c0 comp rx=0 tx=0).stop 2024-05-09T00:48:11.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.168+0000 7f537b95e640 1 -- 172.21.15.5:0/3032347886 shutdown_connections 2024-05-09T00:48:11.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.168+0000 7f537b95e640 1 --2- 172.21.15.5:0/3032347886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5374108a90 0x7f537410aeb0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:11.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.168+0000 7f537b95e640 1 --2- 172.21.15.5:0/3032347886 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5374103d90 0x7f53741041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:11.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.168+0000 7f537b95e640 1 --2- 172.21.15.5:0/3032347886 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5374102b50 0x7f5374103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:11.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.168+0000 7f537b95e640 1 -- 172.21.15.5:0/3032347886 >> 172.21.15.5:0/3032347886 conn(0x7f53740fdd40 msgr2=0x7f53741001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:11.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.169+0000 7f537b95e640 1 -- 172.21.15.5:0/3032347886 shutdown_connections 2024-05-09T00:48:11.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.169+0000 7f537b95e640 1 -- 172.21.15.5:0/3032347886 wait complete. 2024-05-09T00:48:11.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.169+0000 7f537b95e640 1 Processor -- start 2024-05-09T00:48:11.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.169+0000 7f537b95e640 1 -- start start 2024-05-09T00:48:11.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.169+0000 7f537b95e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5374102b50 0x7f5374199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:11.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.169+0000 7f537b95e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5374103d90 0x7f5374199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:11.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.169+0000 7f537b95e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5374108a90 0x7f53741a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:11.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.170+0000 7f537b95e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f537410e660 con 0x7f5374102b50 2024-05-09T00:48:11.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.170+0000 7f537b95e640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f537410e4e0 con 0x7f5374108a90 2024-05-09T00:48:11.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.170+0000 7f537b95e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f537410e7e0 con 0x7f5374103d90 2024-05-09T00:48:11.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.170+0000 7f53796fa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5374102b50 0x7f5374199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:11.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.170+0000 7f53796fa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5374102b50 0x7f5374199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33964/0 (socket says 172.21.15.5:33964) 2024-05-09T00:48:11.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.170+0000 7f53796fa640 1 -- 172.21.15.5:0/159725654 learned_addr learned my addr 172.21.15.5:0/159725654 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:11.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.170+0000 7f5379efb640 1 --2- 172.21.15.5:0/159725654 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5374108a90 0x7f53741a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:11.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.170+0000 7f5378ef9640 1 --2- 172.21.15.5:0/159725654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5374103d90 0x7f5374199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:11.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.170+0000 7f53796fa640 1 -- 172.21.15.5:0/159725654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5374103d90 msgr2=0x7f5374199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:11.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.170+0000 7f53796fa640 1 --2- 172.21.15.5:0/159725654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5374103d90 0x7f5374199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:11.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.170+0000 7f53796fa640 1 -- 172.21.15.5:0/159725654 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5374108a90 msgr2=0x7f53741a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:11.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.170+0000 7f53796fa640 1 --2- 172.21.15.5:0/159725654 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5374108a90 0x7f53741a06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:11.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.170+0000 7f53796fa640 1 -- 172.21.15.5:0/159725654 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53741a0d30 con 0x7f5374102b50 2024-05-09T00:48:11.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.170+0000 7f53796fa640 1 --2- 172.21.15.5:0/159725654 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5374102b50 0x7f5374199240 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f537000d740 tx=0x7f537000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:11.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.170+0000 7f536a7fc640 1 -- 172.21.15.5:0/159725654 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5370017070 con 0x7f5374102b50 2024-05-09T00:48:11.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.170+0000 7f536a7fc640 1 -- 172.21.15.5:0/159725654 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f537000de50 con 0x7f5374102b50 2024-05-09T00:48:11.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.170+0000 7f537b95e640 1 -- 172.21.15.5:0/159725654 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53741a0fc0 con 0x7f5374102b50 2024-05-09T00:48:11.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.170+0000 7f536a7fc640 1 -- 172.21.15.5:0/159725654 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5370003c20 con 0x7f5374102b50 2024-05-09T00:48:11.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.171+0000 7f537b95e640 1 -- 172.21.15.5:0/159725654 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53741a1480 con 0x7f5374102b50 2024-05-09T00:48:11.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.173+0000 7f5382814740 1 Processor -- start 2024-05-09T00:48:11.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.173+0000 7f5382814740 1 -- start start 2024-05-09T00:48:11.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.174+0000 7f5382814740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e93d65d1b0 0x55e93da03710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:11.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.174+0000 7f5382814740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e93d6342f0 0x55e93da03c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:11.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.174+0000 7f5382814740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e93d63c4d0 0x55e93da0ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:11.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.174+0000 7f5382814740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e93d9657c0 con 0x55e93d63c4d0 2024-05-09T00:48:11.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.174+0000 7f5382814740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55e93d9010e0 con 0x55e93d6342f0 2024-05-09T00:48:11.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.174+0000 7f5382814740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55e93d961780 con 0x55e93d65d1b0 2024-05-09T00:48:11.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.174+0000 7f536a7fc640 1 -- 172.21.15.5:0/159725654 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f5370003dc0 con 0x7f5374102b50 2024-05-09T00:48:11.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.174+0000 7f5378ef9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e93d63c4d0 0x55e93da0ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:11.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.174+0000 7f5378ef9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e93d63c4d0 0x55e93da0ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33980/0 (socket says 172.21.15.5:33980) 2024-05-09T00:48:11.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.174+0000 7f5378ef9640 1 -- 172.21.15.5:0/2386622219 learned_addr learned my addr 172.21.15.5:0/2386622219 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:11.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.174+0000 7f5379efb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e93d65d1b0 0x55e93da03710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:11.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.174+0000 7f53796fa640 1 --2- 172.21.15.5:0/2386622219 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e93d6342f0 0x55e93da03c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:11.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.174+0000 7f5378ef9640 1 -- 172.21.15.5:0/2386622219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e93d65d1b0 msgr2=0x55e93da03710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:11.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.174+0000 7f5378ef9640 1 --2- 172.21.15.5:0/2386622219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e93d65d1b0 0x55e93da03710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:11.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.174+0000 7f5378ef9640 1 -- 172.21.15.5:0/2386622219 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e93d6342f0 msgr2=0x55e93da03c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:11.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.174+0000 7f5378ef9640 1 --2- 172.21.15.5:0/2386622219 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e93d6342f0 0x55e93da03c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:11.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.174+0000 7f5378ef9640 1 -- 172.21.15.5:0/2386622219 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e93d92a7c0 con 0x55e93d63c4d0 2024-05-09T00:48:11.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.174+0000 7f5379efb640 1 --2- 172.21.15.5:0/2386622219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e93d65d1b0 0x55e93da03710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:48:11.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.175+0000 7f5378ef9640 1 --2- 172.21.15.5:0/2386622219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e93d63c4d0 0x55e93da0ab90 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f536400bc60 tx=0x7f536400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f44681fc30e99531 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:11.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.175+0000 7f536a7fc640 1 --2- 172.21.15.5:0/159725654 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f53600b27e0 0x7f53600b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:11.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.175+0000 7f535f7fe640 1 -- 172.21.15.5:0/2386622219 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5364015070 con 0x55e93d63c4d0 2024-05-09T00:48:11.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.175+0000 7f535f7fe640 1 -- 172.21.15.5:0/2386622219 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f536401a910 con 0x55e93d63c4d0 2024-05-09T00:48:11.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.175+0000 7f535f7fe640 1 -- 172.21.15.5:0/2386622219 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5364011960 con 0x55e93d63c4d0 2024-05-09T00:48:11.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.175+0000 7f5379efb640 1 --2- 172.21.15.5:0/159725654 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f53600b27e0 0x7f53600b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:11.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.175+0000 7f536a7fc640 1 -- 172.21.15.5:0/159725654 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f53700e1ca0 con 0x7f5374102b50 2024-05-09T00:48:11.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.176+0000 7f5382814740 1 -- 172.21.15.5:0/2386622219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e93d63c4d0 msgr2=0x55e93da0ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:11.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.176+0000 7f5379efb640 1 --2- 172.21.15.5:0/159725654 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f53600b27e0 0x7f53600b4cf0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x55e93da048e0 tx=0x7f536c02f4a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:11.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.176+0000 7f5382814740 1 --2- 172.21.15.5:0/2386622219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e93d63c4d0 0x55e93da0ab90 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f536400bc60 tx=0x7f536400f730 comp rx=0 tx=0).stop 2024-05-09T00:48:11.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.176+0000 7f5382814740 1 -- 172.21.15.5:0/2386622219 shutdown_connections 2024-05-09T00:48:11.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.176+0000 7f5382814740 1 --2- 172.21.15.5:0/2386622219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e93d63c4d0 0x55e93da0ab90 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:11.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.176+0000 7f5382814740 1 --2- 172.21.15.5:0/2386622219 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e93d6342f0 0x55e93da03c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:11.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.176+0000 7f5382814740 1 --2- 172.21.15.5:0/2386622219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e93d65d1b0 0x55e93da03710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:11.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.176+0000 7f5382814740 1 -- 172.21.15.5:0/2386622219 >> 172.21.15.5:0/2386622219 conn(0x55e93d651300 msgr2=0x55e93d9fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:11.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.176+0000 7f5382814740 1 -- 172.21.15.5:0/2386622219 shutdown_connections 2024-05-09T00:48:11.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.176+0000 7f5382814740 1 -- 172.21.15.5:0/2386622219 wait complete. 2024-05-09T00:48:11.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.177+0000 7f5382814740 1 -- start start 2024-05-09T00:48:11.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.177+0000 7f5382814740 1 Processor -- start 2024-05-09T00:48:11.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.177+0000 7f5382814740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e93d6342f0 0x55e93da0fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:11.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.177+0000 7f5382814740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e93d63c4d0 0x55e93da0ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:11.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.177+0000 7f5382814740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e93d65d1b0 0x55e93da16b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:11.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.177+0000 7f5382814740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e93da012c0 con 0x55e93d6342f0 2024-05-09T00:48:11.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.177+0000 7f5382814740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55e93d8ec9c0 con 0x55e93d65d1b0 2024-05-09T00:48:11.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.177+0000 7f5382814740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55e93da01440 con 0x55e93d63c4d0 2024-05-09T00:48:11.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.177+0000 7f5382814740 10 client.0 _ll_register_callbacks cb 0x55e93d919020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:48:11.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.177+0000 7f53796fa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e93d6342f0 0x55e93da0fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:11.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.177+0000 7f53796fa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e93d6342f0 0x55e93da0fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33994/0 (socket says 172.21.15.5:33994) 2024-05-09T00:48:11.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.177+0000 7f53796fa640 1 -- 172.21.15.5:0/3078764872 learned_addr learned my addr 172.21.15.5:0/3078764872 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:11.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.177+0000 7f5379efb640 1 --2- 172.21.15.5:0/3078764872 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e93d65d1b0 0x55e93da16b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.178+0000 7f5378ef9640 1 --2- 172.21.15.5:0/3078764872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e93d63c4d0 0x55e93da0ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.178+0000 7f53796fa640 1 -- 172.21.15.5:0/3078764872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e93d63c4d0 msgr2=0x55e93da0ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.178+0000 7f53796fa640 1 --2- 172.21.15.5:0/3078764872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e93d63c4d0 0x55e93da0ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.178+0000 7f53796fa640 1 -- 172.21.15.5:0/3078764872 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e93d65d1b0 msgr2=0x55e93da16b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.178+0000 7f53796fa640 1 --2- 172.21.15.5:0/3078764872 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e93d65d1b0 0x55e93da16b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.178+0000 7f53796fa640 1 -- 172.21.15.5:0/3078764872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e93d9131f0 con 0x55e93d6342f0 2024-05-09T00:48:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.178+0000 7f5378ef9640 1 --2- 172.21.15.5:0/3078764872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e93d63c4d0 0x55e93da0ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:48:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.178+0000 7f53796fa640 1 --2- 172.21.15.5:0/3078764872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e93d6342f0 0x55e93da0fa40 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f5370005d00 tx=0x7f53700a5210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd144b228a54354 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.178+0000 7f535cff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:48:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.178+0000 7f535cff9640 1 -- 172.21.15.5:0/3078764872 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5370020070 con 0x55e93d6342f0 2024-05-09T00:48:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.178+0000 7f535cff9640 1 -- 172.21.15.5:0/3078764872 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f53700a5520 con 0x55e93d6342f0 2024-05-09T00:48:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.178+0000 7f5382814740 1 -- 172.21.15.5:0/3078764872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f536401a910 con 0x55e93d6342f0 2024-05-09T00:48:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.178+0000 7f535cff9640 1 -- 172.21.15.5:0/3078764872 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f53700c3050 con 0x55e93d6342f0 2024-05-09T00:48:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.178+0000 7f5382814740 10 client.15462 fetch_fsmap learned FSMap version 29 2024-05-09T00:48:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.178+0000 7f5382814740 1 -- 172.21.15.5:0/3078764872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e93d6c8790 con 0x55e93d6342f0 2024-05-09T00:48:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.178+0000 7f535cff9640 1 -- 172.21.15.5:0/3078764872 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f536401a910 con 0x55e93d6342f0 2024-05-09T00:48:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.179+0000 7f535cff9640 1 -- 172.21.15.5:0/3078764872 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f53700b1c60 con 0x55e93d6342f0 2024-05-09T00:48:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.179+0000 7f5382814740 10 client.15462 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:48:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.179+0000 7f5382814740 10 client.15462 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:48:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.179+0000 7f5382814740 20 client.15462 populate_metadata read hostname 'smithi005' 2024-05-09T00:48:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.179+0000 7f5382814740 1 --2- 172.21.15.5:0/3078764872 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55e93d61bd80 0x55e93da1d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.179+0000 7f5382814740 4 client.15462 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:48:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.179+0000 7f5382814740 1 -- 172.21.15.5:0/3078764872 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e93d9733d0 con 0x55e93d61bd80 2024-05-09T00:48:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.179+0000 7f5378ef9640 1 --2- 172.21.15.5:0/3078764872 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55e93d61bd80 0x55e93da1d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.179+0000 7f535cff9640 10 client.15462 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:48:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.179+0000 7f5378ef9640 1 --2- 172.21.15.5:0/3078764872 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55e93d61bd80 0x55e93da1d090 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=971ef3db05bdf4dc server_cookie=6d46b9b247797bc9 in_seq=0 out_seq=0 2024-05-09T00:48:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.180+0000 7f535cff9640 1 -- 172.21.15.5:0/3078764872 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e93d9733d0 con 0x55e93d61bd80 2024-05-09T00:48:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.180+0000 7f535cff9640 10 client.15462 handle_command_reply: tid=0 2024-05-09T00:48:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.180+0000 7f5382814740 1 client.15462 shutdown 2024-05-09T00:48:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.180+0000 7f5382814740 1 -- 172.21.15.5:0/3078764872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e93d6342f0 msgr2=0x55e93da0fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.180+0000 7f5382814740 1 --2- 172.21.15.5:0/3078764872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e93d6342f0 0x55e93da0fa40 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f5370005d00 tx=0x7f53700a5210 comp rx=0 tx=0).stop 2024-05-09T00:48:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.180+0000 7f5382814740 1 -- 172.21.15.5:0/3078764872 shutdown_connections 2024-05-09T00:48:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.180+0000 7f5382814740 1 --2- 172.21.15.5:0/3078764872 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55e93d61bd80 0x55e93da1d090 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.180+0000 7f5382814740 1 --2- 172.21.15.5:0/3078764872 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e93d65d1b0 0x55e93da16b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.180+0000 7f5382814740 1 --2- 172.21.15.5:0/3078764872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e93d63c4d0 0x55e93da0ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.180+0000 7f5382814740 1 --2- 172.21.15.5:0/3078764872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e93d6342f0 0x55e93da0fa40 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.180+0000 7f5382814740 1 -- 172.21.15.5:0/3078764872 >> 172.21.15.5:0/3078764872 conn(0x55e93d651300 msgr2=0x55e93d9ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.180+0000 7f535cff9640 0 client.15462 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:48:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.181+0000 7f5382814740 1 -- 172.21.15.5:0/3078764872 shutdown_connections 2024-05-09T00:48:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.181+0000 7f5382814740 1 -- 172.21.15.5:0/3078764872 wait complete. 2024-05-09T00:48:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.181+0000 7f5382814740 20 client.15462 trim_cache size 0 max 16384 2024-05-09T00:48:11.198 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.196+0000 7f5382814740 1 Processor -- start 2024-05-09T00:48:11.198 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.196+0000 7f5382814740 1 -- start start 2024-05-09T00:48:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.196+0000 7f5382814740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e93d6342f0 0x55e93da028f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.196+0000 7f5382814740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e93d63c4d0 0x55e93da0f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.196+0000 7f5382814740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e93d65d1b0 0x55e93da0fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.196+0000 7f5382814740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f53741a9320 con 0x55e93d63c4d0 2024-05-09T00:48:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.196+0000 7f5382814740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f53741aae60 con 0x55e93d6342f0 2024-05-09T00:48:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.196+0000 7f5382814740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f53741a99f0 con 0x55e93d65d1b0 2024-05-09T00:48:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.196+0000 7f5378ef9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e93d63c4d0 0x55e93da0f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.196+0000 7f5378ef9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e93d63c4d0 0x55e93da0f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34008/0 (socket says 172.21.15.5:34008) 2024-05-09T00:48:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.196+0000 7f5378ef9640 1 -- 172.21.15.5:0/1581042759 learned_addr learned my addr 172.21.15.5:0/1581042759 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.196+0000 7f53796fa640 1 --2- 172.21.15.5:0/1581042759 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e93d6342f0 0x55e93da028f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.196+0000 7f5378ef9640 1 -- 172.21.15.5:0/1581042759 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e93d65d1b0 msgr2=0x55e93da0fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.196+0000 7f5379efb640 1 --2- 172.21.15.5:0/1581042759 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e93d65d1b0 0x55e93da0fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.196+0000 7f5378ef9640 1 --2- 172.21.15.5:0/1581042759 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e93d65d1b0 0x55e93da0fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.196+0000 7f5378ef9640 1 -- 172.21.15.5:0/1581042759 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e93d6342f0 msgr2=0x55e93da028f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.196+0000 7f5378ef9640 1 --2- 172.21.15.5:0/1581042759 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e93d6342f0 0x55e93da028f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.196+0000 7f5378ef9640 1 -- 172.21.15.5:0/1581042759 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e93d73f5f0 con 0x55e93d63c4d0 2024-05-09T00:48:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.196+0000 7f53796fa640 1 --2- 172.21.15.5:0/1581042759 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e93d6342f0 0x55e93da028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:48:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.197+0000 7f5378ef9640 1 --2- 172.21.15.5:0/1581042759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e93d63c4d0 0x55e93da0f7b0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f536400ffd0 tx=0x7f536400ff60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2e87e42efd43132 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.197+0000 7f535effd640 1 -- 172.21.15.5:0/1581042759 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5364015040 con 0x55e93d63c4d0 2024-05-09T00:48:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.197+0000 7f535effd640 1 -- 172.21.15.5:0/1581042759 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5364005520 con 0x55e93d63c4d0 2024-05-09T00:48:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.197+0000 7f535effd640 1 -- 172.21.15.5:0/1581042759 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53640048e0 con 0x55e93d63c4d0 2024-05-09T00:48:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.198+0000 7f5382814740 1 -- 172.21.15.5:0/1581042759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e93d63c4d0 msgr2=0x55e93da0f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.198+0000 7f5382814740 1 --2- 172.21.15.5:0/1581042759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e93d63c4d0 0x55e93da0f7b0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f536400ffd0 tx=0x7f536400ff60 comp rx=0 tx=0).stop 2024-05-09T00:48:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.198+0000 7f5382814740 1 -- 172.21.15.5:0/1581042759 shutdown_connections 2024-05-09T00:48:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.198+0000 7f5382814740 1 --2- 172.21.15.5:0/1581042759 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e93d65d1b0 0x55e93da0fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.198+0000 7f5382814740 1 --2- 172.21.15.5:0/1581042759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e93d63c4d0 0x55e93da0f7b0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.198+0000 7f5382814740 1 --2- 172.21.15.5:0/1581042759 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e93d6342f0 0x55e93da028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:11.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.198+0000 7f5382814740 1 -- 172.21.15.5:0/1581042759 >> 172.21.15.5:0/1581042759 conn(0x55e93d651300 msgr2=0x55e93d9ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:11.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.198+0000 7f5382814740 1 -- 172.21.15.5:0/1581042759 shutdown_connections 2024-05-09T00:48:11.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.198+0000 7f5382814740 1 -- 172.21.15.5:0/1581042759 wait complete. 2024-05-09T00:48:11.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.198+0000 7f5382814740 1 -- start start 2024-05-09T00:48:11.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.198+0000 7f5382814740 1 Processor -- start 2024-05-09T00:48:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.199+0000 7f5382814740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e93d6342f0 0x55e93da12f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.199+0000 7f5382814740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e93d63c4d0 0x55e93da134c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.199+0000 7f5382814740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e93d65d1b0 0x55e93da83480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.199+0000 7f5382814740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f53741aa790 con 0x55e93d63c4d0 2024-05-09T00:48:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.199+0000 7f5382814740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f53741aa0c0 con 0x55e93d65d1b0 2024-05-09T00:48:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.199+0000 7f5382814740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f53741a7300 con 0x55e93d6342f0 2024-05-09T00:48:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.199+0000 7f5382814740 10 client.0 _ll_register_callbacks cb 0x7f53700e5470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:48:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.199+0000 7f5378ef9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e93d63c4d0 0x55e93da134c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.199+0000 7f5378ef9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e93d63c4d0 0x55e93da134c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34024/0 (socket says 172.21.15.5:34024) 2024-05-09T00:48:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.199+0000 7f5378ef9640 1 -- 172.21.15.5:0/2113638098 learned_addr learned my addr 172.21.15.5:0/2113638098 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.199+0000 7f5379efb640 1 --2- 172.21.15.5:0/2113638098 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e93d65d1b0 0x55e93da83480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.199+0000 7f53796fa640 1 --2- 172.21.15.5:0/2113638098 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e93d6342f0 0x55e93da12f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.199+0000 7f5378ef9640 1 -- 172.21.15.5:0/2113638098 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e93d6342f0 msgr2=0x55e93da12f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.199+0000 7f5378ef9640 1 --2- 172.21.15.5:0/2113638098 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e93d6342f0 0x55e93da12f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.199+0000 7f5378ef9640 1 -- 172.21.15.5:0/2113638098 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e93d65d1b0 msgr2=0x55e93da83480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.199+0000 7f5378ef9640 1 --2- 172.21.15.5:0/2113638098 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e93d65d1b0 0x55e93da83480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.199+0000 7f5378ef9640 1 -- 172.21.15.5:0/2113638098 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e93d929ed0 con 0x55e93d63c4d0 2024-05-09T00:48:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.199+0000 7f53796fa640 1 --2- 172.21.15.5:0/2113638098 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e93d6342f0 0x55e93da12f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:48:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.200+0000 7f5378ef9640 1 --2- 172.21.15.5:0/2113638098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e93d63c4d0 0x55e93da134c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f536400ba20 tx=0x7f5364005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50a1f4322ec9930 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.200+0000 7f535cff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:48:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.200+0000 7f535cff9640 1 -- 172.21.15.5:0/2113638098 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5364015030 con 0x55e93d63c4d0 2024-05-09T00:48:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.200+0000 7f535cff9640 1 -- 172.21.15.5:0/2113638098 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f536401a4e0 con 0x55e93d63c4d0 2024-05-09T00:48:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.200+0000 7f5382814740 1 -- 172.21.15.5:0/2113638098 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5364005520 con 0x55e93d63c4d0 2024-05-09T00:48:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.200+0000 7f535cff9640 1 -- 172.21.15.5:0/2113638098 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5364041050 con 0x55e93d63c4d0 2024-05-09T00:48:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.200+0000 7f5382814740 10 client.15468 fetch_fsmap learned FSMap version 29 2024-05-09T00:48:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.200+0000 7f5382814740 1 -- 172.21.15.5:0/2113638098 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e93da847e0 con 0x55e93d63c4d0 2024-05-09T00:48:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.200+0000 7f535cff9640 1 -- 172.21.15.5:0/2113638098 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5364005520 con 0x55e93d63c4d0 2024-05-09T00:48:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.200+0000 7f535cff9640 1 -- 172.21.15.5:0/2113638098 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f5364004b30 con 0x55e93d63c4d0 2024-05-09T00:48:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.200+0000 7f5382814740 10 client.15468 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:48:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.200+0000 7f5382814740 10 client.15468 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:48:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.200+0000 7f5382814740 20 client.15468 populate_metadata read hostname 'smithi005' 2024-05-09T00:48:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.200+0000 7f5382814740 1 --2- 172.21.15.5:0/2113638098 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55e93d5f11c0 0x55e93da87010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.200+0000 7f5382814740 4 client.15468 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:48:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.200+0000 7f5382814740 1 -- 172.21.15.5:0/2113638098 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e93d96d240 con 0x55e93d5f11c0 2024-05-09T00:48:11.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.201+0000 7f53796fa640 1 --2- 172.21.15.5:0/2113638098 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55e93d5f11c0 0x55e93da87010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:11.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.201+0000 7f535cff9640 10 client.15468 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:48:11.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.201+0000 7f53796fa640 1 --2- 172.21.15.5:0/2113638098 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55e93d5f11c0 0x55e93da87010 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a184f5bfba56e33 server_cookie=846ed98a015ed347 in_seq=0 out_seq=0 2024-05-09T00:48:11.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.205+0000 7f535cff9640 1 -- 172.21.15.5:0/2113638098 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e93d96d240 con 0x55e93d5f11c0 2024-05-09T00:48:11.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.205+0000 7f535cff9640 10 client.15468 handle_command_reply: tid=0 2024-05-09T00:48:11.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.205+0000 7f5382814740 1 client.15468 shutdown 2024-05-09T00:48:11.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.205+0000 7f5382814740 1 -- 172.21.15.5:0/2113638098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e93d63c4d0 msgr2=0x55e93da134c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:11.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.205+0000 7f5382814740 1 --2- 172.21.15.5:0/2113638098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e93d63c4d0 0x55e93da134c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f536400ba20 tx=0x7f5364005e60 comp rx=0 tx=0).stop 2024-05-09T00:48:11.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.206+0000 7f5382814740 1 -- 172.21.15.5:0/2113638098 shutdown_connections 2024-05-09T00:48:11.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.206+0000 7f5382814740 1 --2- 172.21.15.5:0/2113638098 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55e93d5f11c0 0x55e93da87010 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:11.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.206+0000 7f5382814740 1 --2- 172.21.15.5:0/2113638098 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e93d65d1b0 0x55e93da83480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:11.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.206+0000 7f535cff9640 0 client.15468 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:48:11.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.206+0000 7f5382814740 1 --2- 172.21.15.5:0/2113638098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e93d63c4d0 0x55e93da134c0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:11.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.206+0000 7f5382814740 1 --2- 172.21.15.5:0/2113638098 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e93d6342f0 0x55e93da12f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:11.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.206+0000 7f5382814740 1 -- 172.21.15.5:0/2113638098 >> 172.21.15.5:0/2113638098 conn(0x55e93d651300 msgr2=0x55e93da0f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:11.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.206+0000 7f5382814740 1 -- 172.21.15.5:0/2113638098 shutdown_connections 2024-05-09T00:48:11.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.206+0000 7f5382814740 1 -- 172.21.15.5:0/2113638098 wait complete. 2024-05-09T00:48:11.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.206+0000 7f5382814740 20 client.15468 trim_cache size 0 max 16384 2024-05-09T00:48:11.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-09T00:48:11.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-09T00:48:11.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-09T00:48:11.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.206+0000 7f537b95e640 1 -- 172.21.15.5:0/159725654 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f53600b27e0 msgr2=0x7f53600b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:11.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.206+0000 7f537b95e640 1 --2- 172.21.15.5:0/159725654 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f53600b27e0 0x7f53600b4cf0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x55e93da048e0 tx=0x7f536c02f4a0 comp rx=0 tx=0).stop 2024-05-09T00:48:11.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.206+0000 7f537b95e640 1 -- 172.21.15.5:0/159725654 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5374102b50 msgr2=0x7f5374199240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:11.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.206+0000 7f537b95e640 1 --2- 172.21.15.5:0/159725654 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5374102b50 0x7f5374199240 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f537000d740 tx=0x7f537000dc00 comp rx=0 tx=0).stop 2024-05-09T00:48:11.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.206+0000 7f537b95e640 1 -- 172.21.15.5:0/159725654 shutdown_connections 2024-05-09T00:48:11.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.206+0000 7f537b95e640 1 --2- 172.21.15.5:0/159725654 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f53600b27e0 0x7f53600b4cf0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:11.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.206+0000 7f537b95e640 1 --2- 172.21.15.5:0/159725654 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5374108a90 0x7f53741a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:11.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.206+0000 7f537b95e640 1 --2- 172.21.15.5:0/159725654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5374103d90 0x7f5374199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:11.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.206+0000 7f537b95e640 1 --2- 172.21.15.5:0/159725654 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5374102b50 0x7f5374199240 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:11.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.207+0000 7f537b95e640 1 -- 172.21.15.5:0/159725654 >> 172.21.15.5:0/159725654 conn(0x7f53740fdd40 msgr2=0x7f5374104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:11.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.207+0000 7f537b95e640 1 -- 172.21.15.5:0/159725654 shutdown_connections 2024-05-09T00:48:11.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:11.207+0000 7f537b95e640 1 -- 172.21.15.5:0/159725654 wait complete. 2024-05-09T00:48:11.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:48:11 smithi005 ceph-mon[30365]: pgmap v426: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 767 B/s wr, 12 op/s 2024-05-09T00:48:11.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:48:11 smithi131 ceph-mon[34547]: pgmap v426: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 767 B/s wr, 12 op/s 2024-05-09T00:48:11.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:48:11 smithi157 ceph-mon[34294]: pgmap v426: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 767 B/s wr, 12 op/s 2024-05-09T00:48:11.811 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T00:48:12.078 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:48:13.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:48:13 smithi005 ceph-mon[30365]: pgmap v427: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 9 op/s 2024-05-09T00:48:13.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:48:13 smithi131 ceph-mon[34547]: pgmap v427: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 9 op/s 2024-05-09T00:48:13.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:48:13 smithi157 ceph-mon[34294]: pgmap v427: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 9 op/s 2024-05-09T00:48:13.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.884+0000 7f7d9fc27700 1 -- 172.21.15.5:0/2174750544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d9810a3e0 msgr2=0x7f7d9810a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:13.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.884+0000 7f7d9fc27700 1 --2- 172.21.15.5:0/2174750544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d9810a3e0 0x7f7d9810a7f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f7d94009990 tx=0x7f7d94009ca0 comp rx=0 tx=0).stop 2024-05-09T00:48:13.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.884+0000 7f7d9fc27700 1 -- 172.21.15.5:0/2174750544 shutdown_connections 2024-05-09T00:48:13.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.884+0000 7f7d9fc27700 1 --2- 172.21.15.5:0/2174750544 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d98103130 0x7f7d98111120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:13.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.884+0000 7f7d9fc27700 1 --2- 172.21.15.5:0/2174750544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d9810a3e0 0x7f7d9810a7f0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:13.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.884+0000 7f7d9fc27700 1 --2- 172.21.15.5:0/2174750544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d981027e0 0x7f7d98102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:13.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.884+0000 7f7d9fc27700 1 -- 172.21.15.5:0/2174750544 >> 172.21.15.5:0/2174750544 conn(0x7f7d980fe6f0 msgr2=0x7f7d98100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:13.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.884+0000 7f7d9fc27700 1 -- 172.21.15.5:0/2174750544 shutdown_connections 2024-05-09T00:48:13.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.884+0000 7f7d9fc27700 1 -- 172.21.15.5:0/2174750544 wait complete. 2024-05-09T00:48:13.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.885+0000 7f7d9fc27700 1 Processor -- start 2024-05-09T00:48:13.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.885+0000 7f7d9fc27700 1 -- start start 2024-05-09T00:48:13.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.885+0000 7f7d9fc27700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d981027e0 0x7f7d9819aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:13.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.885+0000 7f7d9fc27700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d98103130 0x7f7d9819b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:13.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.885+0000 7f7d9fc27700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d9810a3e0 0x7f7d9819f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:13.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.885+0000 7f7d9fc27700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7d98114550 con 0x7f7d9810a3e0 2024-05-09T00:48:13.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.885+0000 7f7d9fc27700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7d981143d0 con 0x7f7d981027e0 2024-05-09T00:48:13.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.885+0000 7f7d9fc27700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7d981146d0 con 0x7f7d98103130 2024-05-09T00:48:13.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.885+0000 7f7d9e1c4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d9810a3e0 0x7f7d9819f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:13.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.885+0000 7f7d9e1c4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d9810a3e0 0x7f7d9819f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46834/0 (socket says 172.21.15.5:46834) 2024-05-09T00:48:13.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.885+0000 7f7d9e1c4700 1 -- 172.21.15.5:0/437456443 learned_addr learned my addr 172.21.15.5:0/437456443 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:13.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.886+0000 7f7d9d9c3700 1 --2- 172.21.15.5:0/437456443 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d981027e0 0x7f7d9819aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:13.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.886+0000 7f7d9e1c4700 1 -- 172.21.15.5:0/437456443 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d98103130 msgr2=0x7f7d9819b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:13.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.886+0000 7f7d9e1c4700 1 --2- 172.21.15.5:0/437456443 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d98103130 0x7f7d9819b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:13.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.886+0000 7f7d9e1c4700 1 -- 172.21.15.5:0/437456443 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d981027e0 msgr2=0x7f7d9819aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:13.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.886+0000 7f7d9e1c4700 1 --2- 172.21.15.5:0/437456443 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d981027e0 0x7f7d9819aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:13.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.886+0000 7f7d9e1c4700 1 -- 172.21.15.5:0/437456443 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d9819fdb0 con 0x7f7d9810a3e0 2024-05-09T00:48:13.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.886+0000 7f7d9d9c3700 1 --2- 172.21.15.5:0/437456443 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d981027e0 0x7f7d9819aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:48:13.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.886+0000 7f7d9e1c4700 1 --2- 172.21.15.5:0/437456443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d9810a3e0 0x7f7d9819f710 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f7d9000b970 tx=0x7f7d9000bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:13.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.886+0000 7f7d8effd700 1 -- 172.21.15.5:0/437456443 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d90019070 con 0x7f7d9810a3e0 2024-05-09T00:48:13.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.886+0000 7f7d8effd700 1 -- 172.21.15.5:0/437456443 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d90014440 con 0x7f7d9810a3e0 2024-05-09T00:48:13.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.886+0000 7f7d9fc27700 1 -- 172.21.15.5:0/437456443 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d981a00a0 con 0x7f7d9810a3e0 2024-05-09T00:48:13.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.886+0000 7f7d8effd700 1 -- 172.21.15.5:0/437456443 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d9000ce60 con 0x7f7d9810a3e0 2024-05-09T00:48:13.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.886+0000 7f7d9fc27700 1 -- 172.21.15.5:0/437456443 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d981a79c0 con 0x7f7d9810a3e0 2024-05-09T00:48:13.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.890+0000 7f7d8effd700 1 -- 172.21.15.5:0/437456443 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f7d9001c070 con 0x7f7d9810a3e0 2024-05-09T00:48:13.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.890+0000 7f7dae63db80 1 Processor -- start 2024-05-09T00:48:13.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.891+0000 7f7d8effd700 1 --2- 172.21.15.5:0/437456443 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f7d840b2900 0x7f7d840b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:13.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.891+0000 7f7dae63db80 1 -- start start 2024-05-09T00:48:13.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.891+0000 7f7d9d1c2700 1 --2- 172.21.15.5:0/437456443 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f7d840b2900 0x7f7d840b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:13.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.891+0000 7f7d8effd700 1 -- 172.21.15.5:0/437456443 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f7d900e1680 con 0x7f7d9810a3e0 2024-05-09T00:48:13.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.891+0000 7f7dae63db80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c593914d00 0x55c593a1d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:13.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.891+0000 7f7dae63db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c59389a4a0 0x55c593a1d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:13.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.891+0000 7f7dae63db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c593a17400 0x55c593a17810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:13.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.891+0000 7f7dae63db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c59393a0a0 con 0x55c593a17400 2024-05-09T00:48:13.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.891+0000 7f7dae63db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55c5938df9a0 con 0x55c59389a4a0 2024-05-09T00:48:13.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.891+0000 7f7dae63db80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55c593936060 con 0x55c593914d00 2024-05-09T00:48:13.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.891+0000 7f7d9d9c3700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c593914d00 0x55c593a1d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:13.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.891+0000 7f7d9d9c3700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c593914d00 0x55c593a1d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:53288/0 (socket says 172.21.15.5:53288) 2024-05-09T00:48:13.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.891+0000 7f7d9d9c3700 1 -- 172.21.15.5:0/3902027435 learned_addr learned my addr 172.21.15.5:0/3902027435 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:13.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.891+0000 7f7d9e1c4700 1 --2- 172.21.15.5:0/3902027435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c593a17400 0x55c593a17810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:13.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.892+0000 7f7d9d1c2700 1 --2- 172.21.15.5:0/3902027435 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c59389a4a0 0x55c593a1d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:13.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.892+0000 7f7d9d1c2700 1 --2- 172.21.15.5:0/437456443 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f7d840b2900 0x7f7d840b4e10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7d88005950 tx=0x7f7d880058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:13.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.892+0000 7f7d9d9c3700 1 -- 172.21.15.5:0/3902027435 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c59389a4a0 msgr2=0x55c593a1d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:13.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.892+0000 7f7d9d9c3700 1 --2- 172.21.15.5:0/3902027435 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c59389a4a0 0x55c593a1d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:13.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.892+0000 7f7d9d9c3700 1 -- 172.21.15.5:0/3902027435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c593a17400 msgr2=0x55c593a17810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:13.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.892+0000 7f7d9d9c3700 1 --2- 172.21.15.5:0/3902027435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c593a17400 0x55c593a17810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:13.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.892+0000 7f7d9d9c3700 1 -- 172.21.15.5:0/3902027435 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5938d0500 con 0x55c593914d00 2024-05-09T00:48:13.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.892+0000 7f7d9d1c2700 1 --2- 172.21.15.5:0/3902027435 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c59389a4a0 0x55c593a1d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:48:13.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.892+0000 7f7d9e1c4700 1 --2- 172.21.15.5:0/3902027435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c593a17400 0x55c593a17810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:48:13.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.892+0000 7f7d9d9c3700 1 --2- 172.21.15.5:0/3902027435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c593914d00 0x55c593a1d240 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7d94009510 tx=0x7f7d94009540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9eef4cf565904e09 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:13.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.892+0000 7f7d83fff700 1 -- 172.21.15.5:0/3902027435 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d94003a60 con 0x55c593914d00 2024-05-09T00:48:13.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.892+0000 7f7d83fff700 1 -- 172.21.15.5:0/3902027435 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d94005580 con 0x55c593914d00 2024-05-09T00:48:13.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.892+0000 7f7d83fff700 1 -- 172.21.15.5:0/3902027435 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d9400bd50 con 0x55c593914d00 2024-05-09T00:48:13.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.893+0000 7f7dae63db80 1 -- 172.21.15.5:0/3902027435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c593914d00 msgr2=0x55c593a1d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:13.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.893+0000 7f7dae63db80 1 --2- 172.21.15.5:0/3902027435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c593914d00 0x55c593a1d240 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7d94009510 tx=0x7f7d94009540 comp rx=0 tx=0).stop 2024-05-09T00:48:13.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.893+0000 7f7dae63db80 1 -- 172.21.15.5:0/3902027435 shutdown_connections 2024-05-09T00:48:13.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.893+0000 7f7dae63db80 1 --2- 172.21.15.5:0/3902027435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c593a17400 0x55c593a17810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:13.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.893+0000 7f7dae63db80 1 --2- 172.21.15.5:0/3902027435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c593914d00 0x55c593a1d240 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:13.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.893+0000 7f7dae63db80 1 --2- 172.21.15.5:0/3902027435 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c59389a4a0 0x55c593a1d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:13.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.893+0000 7f7dae63db80 1 -- 172.21.15.5:0/3902027435 >> 172.21.15.5:0/3902027435 conn(0x55c59389df70 msgr2=0x55c593a12220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:13.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.894+0000 7f7dae63db80 1 -- 172.21.15.5:0/3902027435 shutdown_connections 2024-05-09T00:48:13.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.894+0000 7f7dae63db80 1 -- 172.21.15.5:0/3902027435 wait complete. 2024-05-09T00:48:13.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.894+0000 7f7dae63db80 1 -- start start 2024-05-09T00:48:13.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.894+0000 7f7dae63db80 1 Processor -- start 2024-05-09T00:48:13.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.895+0000 7f7dae63db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c59389a4a0 0x55c593a21090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:13.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.895+0000 7f7dae63db80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c593914d00 0x55c593a215d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:13.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.895+0000 7f7dae63db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c593a17400 0x55c593a21b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:13.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.895+0000 7f7dae63db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c593a180a0 con 0x55c593a17400 2024-05-09T00:48:13.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.895+0000 7f7dae63db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55c5938d4b20 con 0x55c59389a4a0 2024-05-09T00:48:13.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.895+0000 7f7dae63db80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55c593a18220 con 0x55c593914d00 2024-05-09T00:48:13.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.895+0000 7f7dae63db80 10 client.0 _ll_register_callbacks cb 0x55c5937ab6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:48:13.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.895+0000 7f7d9e1c4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c593a17400 0x55c593a21b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:13.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.895+0000 7f7d9e1c4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c593a17400 0x55c593a21b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46852/0 (socket says 172.21.15.5:46852) 2024-05-09T00:48:13.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.895+0000 7f7d9e1c4700 1 -- 172.21.15.5:0/4134712470 learned_addr learned my addr 172.21.15.5:0/4134712470 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:13.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.895+0000 7f7d9d9c3700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c59389a4a0 0x55c593a21090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:13.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.895+0000 7f7d9d1c2700 1 --2- 172.21.15.5:0/4134712470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c593914d00 0x55c593a215d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:13.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.895+0000 7f7d9e1c4700 1 -- 172.21.15.5:0/4134712470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c593914d00 msgr2=0x55c593a215d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:13.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.895+0000 7f7d9e1c4700 1 --2- 172.21.15.5:0/4134712470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c593914d00 0x55c593a215d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:13.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.895+0000 7f7d9e1c4700 1 -- 172.21.15.5:0/4134712470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c59389a4a0 msgr2=0x55c593a21090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:13.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.895+0000 7f7d9e1c4700 1 --2- 172.21.15.5:0/4134712470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c59389a4a0 0x55c593a21090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:13.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.895+0000 7f7d9e1c4700 1 -- 172.21.15.5:0/4134712470 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5937b5490 con 0x55c593a17400 2024-05-09T00:48:13.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.895+0000 7f7d9d1c2700 1 --2- 172.21.15.5:0/4134712470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c593914d00 0x55c593a215d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:48:13.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.896+0000 7f7d9e1c4700 1 --2- 172.21.15.5:0/4134712470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c593a17400 0x55c593a21b10 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f7d9000be50 tx=0x7f7d900b1520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df8135e3b9f0b5a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:13.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.896+0000 7f7d817fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:48:13.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.896+0000 7f7d817fa700 1 -- 172.21.15.5:0/4134712470 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d900a8c70 con 0x55c593a17400 2024-05-09T00:48:13.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.896+0000 7f7dae63db80 1 -- 172.21.15.5:0/4134712470 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d94005580 con 0x55c593a17400 2024-05-09T00:48:13.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.896+0000 7f7d817fa700 1 -- 172.21.15.5:0/4134712470 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f7d900b1640 con 0x55c593a17400 2024-05-09T00:48:13.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.896+0000 7f7d817fa700 1 -- 172.21.15.5:0/4134712470 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7d900ad020 con 0x55c593a17400 2024-05-09T00:48:13.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.896+0000 7f7dae63db80 10 client.15480 fetch_fsmap learned FSMap version 29 2024-05-09T00:48:13.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.896+0000 7f7dae63db80 1 -- 172.21.15.5:0/4134712470 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c593a2e970 con 0x55c593a17400 2024-05-09T00:48:13.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.897+0000 7f7d817fa700 1 -- 172.21.15.5:0/4134712470 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d94005580 con 0x55c593a17400 2024-05-09T00:48:13.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.897+0000 7f7d817fa700 1 -- 172.21.15.5:0/4134712470 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f7d900a7ff0 con 0x55c593a17400 2024-05-09T00:48:13.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.897+0000 7f7dae63db80 10 client.15480 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:48:13.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.897+0000 7f7dae63db80 10 client.15480 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:48:13.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.897+0000 7f7dae63db80 20 client.15480 populate_metadata read hostname 'smithi005' 2024-05-09T00:48:13.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.897+0000 7f7dae63db80 1 --2- 172.21.15.5:0/4134712470 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55c593a30f00 0x55c593a33360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:13.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.897+0000 7f7dae63db80 4 client.15480 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:48:13.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.897+0000 7f7dae63db80 1 -- 172.21.15.5:0/4134712470 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c593947a80 con 0x55c593a30f00 2024-05-09T00:48:13.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.897+0000 7f7d9d9c3700 1 --2- 172.21.15.5:0/4134712470 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55c593a30f00 0x55c593a33360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:13.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.898+0000 7f7d9d9c3700 1 --2- 172.21.15.5:0/4134712470 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55c593a30f00 0x55c593a33360 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee402c603545ba45 server_cookie=7459276940f22096 in_seq=0 out_seq=0 2024-05-09T00:48:13.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.898+0000 7f7d817fa700 10 client.15480 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:48:13.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.899+0000 7f7d817fa700 1 -- 172.21.15.5:0/4134712470 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c593947a80 con 0x55c593a30f00 2024-05-09T00:48:13.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.899+0000 7f7d817fa700 10 client.15480 handle_command_reply: tid=0 2024-05-09T00:48:13.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.899+0000 7f7dae63db80 1 client.15480 shutdown 2024-05-09T00:48:13.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.899+0000 7f7dae63db80 1 -- 172.21.15.5:0/4134712470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c593a17400 msgr2=0x55c593a21b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:13.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.899+0000 7f7dae63db80 1 --2- 172.21.15.5:0/4134712470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c593a17400 0x55c593a21b10 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f7d9000be50 tx=0x7f7d900b1520 comp rx=0 tx=0).stop 2024-05-09T00:48:13.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.899+0000 7f7dae63db80 1 -- 172.21.15.5:0/4134712470 shutdown_connections 2024-05-09T00:48:13.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.899+0000 7f7dae63db80 1 --2- 172.21.15.5:0/4134712470 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55c593a30f00 0x55c593a33360 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:13.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.899+0000 7f7dae63db80 1 --2- 172.21.15.5:0/4134712470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c593a17400 0x55c593a21b10 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:13.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.899+0000 7f7dae63db80 1 --2- 172.21.15.5:0/4134712470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c59389a4a0 0x55c593a21090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:13.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.899+0000 7f7dae63db80 1 --2- 172.21.15.5:0/4134712470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c593914d00 0x55c593a215d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:13.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.899+0000 7f7d817fa700 0 client.15480 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:48:13.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.899+0000 7f7dae63db80 1 -- 172.21.15.5:0/4134712470 >> 172.21.15.5:0/4134712470 conn(0x55c59389df70 msgr2=0x55c593a0bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:13.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.899+0000 7f7dae63db80 1 -- 172.21.15.5:0/4134712470 shutdown_connections 2024-05-09T00:48:13.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.899+0000 7f7dae63db80 1 -- 172.21.15.5:0/4134712470 wait complete. 2024-05-09T00:48:13.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.899+0000 7f7dae63db80 20 client.15480 trim_cache size 0 max 16384 2024-05-09T00:48:13.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.916+0000 7f7dae63db80 1 Processor -- start 2024-05-09T00:48:13.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.916+0000 7f7dae63db80 1 -- start start 2024-05-09T00:48:13.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.916+0000 7f7dae63db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c59389a4a0 0x55c593a0ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:13.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.916+0000 7f7dae63db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c593914d00 0x55c593a209b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:13.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.916+0000 7f7dae63db80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c593a17400 0x55c593a21760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:13.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.916+0000 7f7dae63db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7d981aec80 con 0x55c593914d00 2024-05-09T00:48:13.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.916+0000 7f7dae63db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7d981b07c0 con 0x55c59389a4a0 2024-05-09T00:48:13.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.916+0000 7f7dae63db80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7d981af350 con 0x55c593a17400 2024-05-09T00:48:13.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.917+0000 7f7d9d1c2700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c593914d00 0x55c593a209b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:13.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.917+0000 7f7d9d1c2700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c593914d00 0x55c593a209b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46864/0 (socket says 172.21.15.5:46864) 2024-05-09T00:48:13.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.917+0000 7f7d9d1c2700 1 -- 172.21.15.5:0/1694223732 learned_addr learned my addr 172.21.15.5:0/1694223732 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:13.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.917+0000 7f7d9d9c3700 1 --2- 172.21.15.5:0/1694223732 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c59389a4a0 0x55c593a0ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:13.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.917+0000 7f7d9e1c4700 1 --2- 172.21.15.5:0/1694223732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c593a17400 0x55c593a21760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:13.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.917+0000 7f7d9d1c2700 1 -- 172.21.15.5:0/1694223732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c593a17400 msgr2=0x55c593a21760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:13.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.917+0000 7f7d9d1c2700 1 --2- 172.21.15.5:0/1694223732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c593a17400 0x55c593a21760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:13.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.917+0000 7f7d9d1c2700 1 -- 172.21.15.5:0/1694223732 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c59389a4a0 msgr2=0x55c593a0ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:13.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.917+0000 7f7d9d1c2700 1 --2- 172.21.15.5:0/1694223732 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c59389a4a0 0x55c593a0ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:13.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.917+0000 7f7d9d1c2700 1 -- 172.21.15.5:0/1694223732 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c593a0e510 con 0x55c593914d00 2024-05-09T00:48:13.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.917+0000 7f7d9d9c3700 1 --2- 172.21.15.5:0/1694223732 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c59389a4a0 0x55c593a0ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:48:13.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.917+0000 7f7d9d1c2700 1 --2- 172.21.15.5:0/1694223732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c593914d00 0x55c593a209b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f7d9810acd0 tx=0x7f7d88010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=947795bc1c3321e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:13.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.918+0000 7f7d837fe700 1 -- 172.21.15.5:0/1694223732 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d88010ba0 con 0x55c593914d00 2024-05-09T00:48:13.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.918+0000 7f7d837fe700 1 -- 172.21.15.5:0/1694223732 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d88016040 con 0x55c593914d00 2024-05-09T00:48:13.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.918+0000 7f7d837fe700 1 -- 172.21.15.5:0/1694223732 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8800f5c0 con 0x55c593914d00 2024-05-09T00:48:13.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.919+0000 7f7dae63db80 1 -- 172.21.15.5:0/1694223732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c593914d00 msgr2=0x55c593a209b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:13.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.919+0000 7f7dae63db80 1 --2- 172.21.15.5:0/1694223732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c593914d00 0x55c593a209b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f7d9810acd0 tx=0x7f7d88010000 comp rx=0 tx=0).stop 2024-05-09T00:48:13.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.919+0000 7f7dae63db80 1 -- 172.21.15.5:0/1694223732 shutdown_connections 2024-05-09T00:48:13.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.919+0000 7f7dae63db80 1 --2- 172.21.15.5:0/1694223732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c593a17400 0x55c593a21760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:13.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.919+0000 7f7dae63db80 1 --2- 172.21.15.5:0/1694223732 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c59389a4a0 0x55c593a0ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:13.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.919+0000 7f7dae63db80 1 --2- 172.21.15.5:0/1694223732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c593914d00 0x55c593a209b0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:13.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.919+0000 7f7dae63db80 1 -- 172.21.15.5:0/1694223732 >> 172.21.15.5:0/1694223732 conn(0x55c59389df70 msgr2=0x55c593a21050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:13.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.919+0000 7f7dae63db80 1 -- 172.21.15.5:0/1694223732 shutdown_connections 2024-05-09T00:48:13.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.919+0000 7f7dae63db80 1 -- 172.21.15.5:0/1694223732 wait complete. 2024-05-09T00:48:13.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.919+0000 7f7dae63db80 1 -- start start 2024-05-09T00:48:13.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.919+0000 7f7dae63db80 1 Processor -- start 2024-05-09T00:48:13.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.920+0000 7f7dae63db80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c59389a4a0 0x55c593a29510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:13.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.920+0000 7f7dae63db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c593914d00 0x55c593a1f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:13.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.920+0000 7f7dae63db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c593a17400 0x55c593a1fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:13.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.920+0000 7f7dae63db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7d981b00f0 con 0x55c593a17400 2024-05-09T00:48:13.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.920+0000 7f7dae63db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7d981afa20 con 0x55c593914d00 2024-05-09T00:48:13.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.920+0000 7f7dae63db80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7d981acc20 con 0x55c59389a4a0 2024-05-09T00:48:13.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.920+0000 7f7dae63db80 10 client.0 _ll_register_callbacks cb 0x55c5937ab6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:48:13.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.920+0000 7f7d9e1c4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c593a17400 0x55c593a1fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:13.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.920+0000 7f7d9e1c4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c593a17400 0x55c593a1fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46874/0 (socket says 172.21.15.5:46874) 2024-05-09T00:48:13.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.920+0000 7f7d9e1c4700 1 -- 172.21.15.5:0/1130445493 learned_addr learned my addr 172.21.15.5:0/1130445493 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:13.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.920+0000 7f7d9d9c3700 1 --2- 172.21.15.5:0/1130445493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c59389a4a0 0x55c593a29510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:13.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.920+0000 7f7d9d1c2700 1 --2- 172.21.15.5:0/1130445493 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c593914d00 0x55c593a1f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:13.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.920+0000 7f7d9e1c4700 1 -- 172.21.15.5:0/1130445493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c59389a4a0 msgr2=0x55c593a29510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:13.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.920+0000 7f7d9e1c4700 1 --2- 172.21.15.5:0/1130445493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c59389a4a0 0x55c593a29510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:13.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.920+0000 7f7d9e1c4700 1 -- 172.21.15.5:0/1130445493 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c593914d00 msgr2=0x55c593a1f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:13.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.920+0000 7f7d9e1c4700 1 --2- 172.21.15.5:0/1130445493 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c593914d00 0x55c593a1f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:13.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.920+0000 7f7d9e1c4700 1 -- 172.21.15.5:0/1130445493 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c593a204a0 con 0x55c593a17400 2024-05-09T00:48:13.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.921+0000 7f7d9e1c4700 1 --2- 172.21.15.5:0/1130445493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c593a17400 0x55c593a1fee0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f7d900a4ad0 tx=0x7f7d900a8b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81b5550348ed6c75 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:13.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.921+0000 7f7d817fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:48:13.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.921+0000 7f7d817fa700 1 -- 172.21.15.5:0/1130445493 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d900a8c70 con 0x55c593a17400 2024-05-09T00:48:13.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.921+0000 7f7d817fa700 1 -- 172.21.15.5:0/1130445493 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f7d9000c9b0 con 0x55c593a17400 2024-05-09T00:48:13.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.921+0000 7f7dae63db80 1 -- 172.21.15.5:0/1130445493 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d88016040 con 0x55c593a17400 2024-05-09T00:48:13.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.921+0000 7f7d817fa700 1 -- 172.21.15.5:0/1130445493 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7d900b6050 con 0x55c593a17400 2024-05-09T00:48:13.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.921+0000 7f7dae63db80 10 client.15486 fetch_fsmap learned FSMap version 29 2024-05-09T00:48:13.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.921+0000 7f7dae63db80 1 -- 172.21.15.5:0/1130445493 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c593a9f850 con 0x55c593a17400 2024-05-09T00:48:13.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.921+0000 7f7d817fa700 1 -- 172.21.15.5:0/1130445493 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d900c32f0 con 0x55c593a17400 2024-05-09T00:48:13.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.921+0000 7f7d817fa700 1 -- 172.21.15.5:0/1130445493 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f7d900a7940 con 0x55c593a17400 2024-05-09T00:48:13.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.921+0000 7f7dae63db80 10 client.15486 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:48:13.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.921+0000 7f7dae63db80 10 client.15486 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:48:13.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.921+0000 7f7dae63db80 20 client.15486 populate_metadata read hostname 'smithi005' 2024-05-09T00:48:13.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.921+0000 7f7dae63db80 1 --2- 172.21.15.5:0/1130445493 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55c593a9ff50 0x55c593aa2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:13.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.922+0000 7f7dae63db80 4 client.15486 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:48:13.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.922+0000 7f7dae63db80 1 -- 172.21.15.5:0/1130445493 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f7d88016040 con 0x55c593a9ff50 2024-05-09T00:48:13.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.922+0000 7f7d9d9c3700 1 --2- 172.21.15.5:0/1130445493 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55c593a9ff50 0x55c593aa2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:13.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.922+0000 7f7d817fa700 10 client.15486 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:48:13.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.922+0000 7f7d9d9c3700 1 --2- 172.21.15.5:0/1130445493 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55c593a9ff50 0x55c593aa2370 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=607cea3b4ff99145 server_cookie=4a1cc6b62601e53a in_seq=0 out_seq=0 2024-05-09T00:48:13.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.923+0000 7f7d817fa700 1 -- 172.21.15.5:0/1130445493 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f7d88016040 con 0x55c593a9ff50 2024-05-09T00:48:13.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.923+0000 7f7d817fa700 10 client.15486 handle_command_reply: tid=0 2024-05-09T00:48:13.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.923+0000 7f7dae63db80 1 client.15486 shutdown 2024-05-09T00:48:13.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.923+0000 7f7dae63db80 1 -- 172.21.15.5:0/1130445493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c593a17400 msgr2=0x55c593a1fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:13.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.923+0000 7f7dae63db80 1 --2- 172.21.15.5:0/1130445493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c593a17400 0x55c593a1fee0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f7d900a4ad0 tx=0x7f7d900a8b70 comp rx=0 tx=0).stop 2024-05-09T00:48:13.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.923+0000 7f7dae63db80 1 -- 172.21.15.5:0/1130445493 shutdown_connections 2024-05-09T00:48:13.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.923+0000 7f7dae63db80 1 --2- 172.21.15.5:0/1130445493 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55c593a9ff50 0x55c593aa2370 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:13.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.923+0000 7f7dae63db80 1 --2- 172.21.15.5:0/1130445493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c593a17400 0x55c593a1fee0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:13.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.923+0000 7f7dae63db80 1 --2- 172.21.15.5:0/1130445493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c59389a4a0 0x55c593a29510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:13.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.923+0000 7f7dae63db80 1 --2- 172.21.15.5:0/1130445493 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c593914d00 0x55c593a1f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:13.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.923+0000 7f7d817fa700 0 client.15486 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:48:13.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.923+0000 7f7dae63db80 1 -- 172.21.15.5:0/1130445493 >> 172.21.15.5:0/1130445493 conn(0x55c59389df70 msgr2=0x55c593a0c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:13.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.923+0000 7f7dae63db80 1 -- 172.21.15.5:0/1130445493 shutdown_connections 2024-05-09T00:48:13.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.923+0000 7f7dae63db80 1 -- 172.21.15.5:0/1130445493 wait complete. 2024-05-09T00:48:13.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.923+0000 7f7dae63db80 20 client.15486 trim_cache size 0 max 16384 2024-05-09T00:48:13.928 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-09T00:48:13.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.924+0000 7f7d9fc27700 1 -- 172.21.15.5:0/437456443 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f7d840b2900 msgr2=0x7f7d840b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:13.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.924+0000 7f7d9fc27700 1 --2- 172.21.15.5:0/437456443 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f7d840b2900 0x7f7d840b4e10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7d88005950 tx=0x7f7d880058e0 comp rx=0 tx=0).stop 2024-05-09T00:48:13.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.924+0000 7f7d9fc27700 1 -- 172.21.15.5:0/437456443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d9810a3e0 msgr2=0x7f7d9819f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:13.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.924+0000 7f7d9fc27700 1 --2- 172.21.15.5:0/437456443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d9810a3e0 0x7f7d9819f710 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f7d9000b970 tx=0x7f7d9000bd30 comp rx=0 tx=0).stop 2024-05-09T00:48:13.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.924+0000 7f7d9fc27700 1 -- 172.21.15.5:0/437456443 shutdown_connections 2024-05-09T00:48:13.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.924+0000 7f7d9fc27700 1 --2- 172.21.15.5:0/437456443 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f7d840b2900 0x7f7d840b4e10 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:13.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.924+0000 7f7d9fc27700 1 --2- 172.21.15.5:0/437456443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d9810a3e0 0x7f7d9819f710 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:13.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.924+0000 7f7d9fc27700 1 --2- 172.21.15.5:0/437456443 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d981027e0 0x7f7d9819aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:13.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.924+0000 7f7d9fc27700 1 --2- 172.21.15.5:0/437456443 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d98103130 0x7f7d9819b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:13.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.924+0000 7f7d9fc27700 1 -- 172.21.15.5:0/437456443 >> 172.21.15.5:0/437456443 conn(0x7f7d980fe6f0 msgr2=0x7f7d98100bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:13.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.924+0000 7f7d9fc27700 1 -- 172.21.15.5:0/437456443 shutdown_connections 2024-05-09T00:48:13.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:13.924+0000 7f7d9fc27700 1 -- 172.21.15.5:0/437456443 wait complete. 2024-05-09T00:48:15.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:48:15 smithi005 ceph-mon[30365]: pgmap v428: 97 pgs: 97 active+clean; 669 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 597 B/s wr, 13 op/s 2024-05-09T00:48:15.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:48:15 smithi131 ceph-mon[34547]: pgmap v428: 97 pgs: 97 active+clean; 669 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 597 B/s wr, 13 op/s 2024-05-09T00:48:15.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:48:15 smithi157 ceph-mon[34294]: pgmap v428: 97 pgs: 97 active+clean; 669 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 597 B/s wr, 13 op/s 2024-05-09T00:48:15.700 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T00:48:15.701 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T00:48:16.029 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:48:17.470 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:48:17 smithi005 ceph-mon[30365]: pgmap v429: 97 pgs: 97 active+clean; 669 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T00:48:17.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:48:17 smithi131 ceph-mon[34547]: pgmap v429: 97 pgs: 97 active+clean; 669 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T00:48:17.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:48:17 smithi157 ceph-mon[34294]: pgmap v429: 97 pgs: 97 active+clean; 669 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T00:48:17.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.895+0000 7f8ae361d700 1 -- 172.21.15.5:0/1217651132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8adc105480 msgr2=0x7f8adc105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:17.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.895+0000 7f8ae361d700 1 --2- 172.21.15.5:0/1217651132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8adc105480 0x7f8adc105890 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8009990 tx=0x7f8ad8009ca0 comp rx=0 tx=0).stop 2024-05-09T00:48:17.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.895+0000 7f8ae361d700 1 -- 172.21.15.5:0/1217651132 shutdown_connections 2024-05-09T00:48:17.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.895+0000 7f8ae361d700 1 --2- 172.21.15.5:0/1217651132 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8adc10e8f0 0x7f8adc110dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:17.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.895+0000 7f8ae361d700 1 --2- 172.21.15.5:0/1217651132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8adc105480 0x7f8adc105890 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:17.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.895+0000 7f8ae361d700 1 --2- 172.21.15.5:0/1217651132 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8adc105dd0 0x7f8adc10e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:17.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.895+0000 7f8ae361d700 1 -- 172.21.15.5:0/1217651132 >> 172.21.15.5:0/1217651132 conn(0x7f8adc0fef30 msgr2=0x7f8adc101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:17.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.895+0000 7f8ae361d700 1 -- 172.21.15.5:0/1217651132 shutdown_connections 2024-05-09T00:48:17.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.895+0000 7f8ae361d700 1 -- 172.21.15.5:0/1217651132 wait complete. 2024-05-09T00:48:17.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.896+0000 7f8ae361d700 1 Processor -- start 2024-05-09T00:48:17.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.896+0000 7f8ae361d700 1 -- start start 2024-05-09T00:48:17.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.896+0000 7f8ae361d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8adc105480 0x7f8adc103170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:17.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.896+0000 7f8ae361d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8adc105dd0 0x7f8adc1036b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:17.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.896+0000 7f8ae361d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8adc10e8f0 0x7f8adc103bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:17.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.896+0000 7f8ae361d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8adc114440 con 0x7f8adc105dd0 2024-05-09T00:48:17.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.896+0000 7f8ae361d700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8adc1142c0 con 0x7f8adc10e8f0 2024-05-09T00:48:17.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.896+0000 7f8ae361d700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8adc1145c0 con 0x7f8adc105480 2024-05-09T00:48:17.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.897+0000 7f8ae0bb8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8adc105dd0 0x7f8adc1036b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:17.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.897+0000 7f8ae0bb8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8adc105dd0 0x7f8adc1036b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46890/0 (socket says 172.21.15.5:46890) 2024-05-09T00:48:17.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.897+0000 7f8ae0bb8700 1 -- 172.21.15.5:0/248161141 learned_addr learned my addr 172.21.15.5:0/248161141 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:17.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.897+0000 7f8ae13b9700 1 --2- 172.21.15.5:0/248161141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8adc105480 0x7f8adc103170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:17.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.897+0000 7f8ae1bba700 1 --2- 172.21.15.5:0/248161141 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8adc10e8f0 0x7f8adc103bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:17.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.897+0000 7f8ae0bb8700 1 -- 172.21.15.5:0/248161141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8adc105480 msgr2=0x7f8adc103170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:17.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.897+0000 7f8ae0bb8700 1 --2- 172.21.15.5:0/248161141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8adc105480 0x7f8adc103170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:17.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.897+0000 7f8ae0bb8700 1 -- 172.21.15.5:0/248161141 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8adc10e8f0 msgr2=0x7f8adc103bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:17.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.897+0000 7f8ae0bb8700 1 --2- 172.21.15.5:0/248161141 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8adc10e8f0 0x7f8adc103bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:17.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.897+0000 7f8ae0bb8700 1 -- 172.21.15.5:0/248161141 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8adc1a91c0 con 0x7f8adc105dd0 2024-05-09T00:48:17.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.897+0000 7f8ae13b9700 1 --2- 172.21.15.5:0/248161141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8adc105480 0x7f8adc103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:48:17.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.897+0000 7f8ae0bb8700 1 --2- 172.21.15.5:0/248161141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8adc105dd0 0x7f8adc1036b0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f8acc00d950 tx=0x7f8acc00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:17.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.898+0000 7f8ad27fc700 1 -- 172.21.15.5:0/248161141 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8acc016070 con 0x7f8adc105dd0 2024-05-09T00:48:17.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.898+0000 7f8ad27fc700 1 -- 172.21.15.5:0/248161141 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8acc010460 con 0x7f8adc105dd0 2024-05-09T00:48:17.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.898+0000 7f8ae361d700 1 -- 172.21.15.5:0/248161141 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8adc1a94b0 con 0x7f8adc105dd0 2024-05-09T00:48:17.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.899+0000 7f8ad27fc700 1 -- 172.21.15.5:0/248161141 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8acc00f620 con 0x7f8adc105dd0 2024-05-09T00:48:17.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.899+0000 7f8ae361d700 1 -- 172.21.15.5:0/248161141 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8adc1a99f0 con 0x7f8adc105dd0 2024-05-09T00:48:17.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.901+0000 7f8ad27fc700 1 -- 172.21.15.5:0/248161141 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f8acc019070 con 0x7f8adc105dd0 2024-05-09T00:48:17.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.902+0000 7f8ad27fc700 1 --2- 172.21.15.5:0/248161141 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f8ac80b2820 0x7f8ac80b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:17.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.902+0000 7f8ae13b9700 1 --2- 172.21.15.5:0/248161141 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f8ac80b2820 0x7f8ac80b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:17.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.903+0000 7f8ad27fc700 1 -- 172.21.15.5:0/248161141 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f8acc0e1820 con 0x7f8adc105dd0 2024-05-09T00:48:17.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.903+0000 7f8ae13b9700 1 --2- 172.21.15.5:0/248161141 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f8ac80b2820 0x7f8ac80b4d30 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8ad800b2e0 tx=0x7f8ad8005940 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:17.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.904+0000 7f8af2033b80 1 Processor -- start 2024-05-09T00:48:17.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.904+0000 7f8af2033b80 1 -- start start 2024-05-09T00:48:17.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.904+0000 7f8af2033b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dea5d4bd80 0x55dea5e542c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:17.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.904+0000 7f8af2033b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dea5cd1520 0x55dea5e54800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:17.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.904+0000 7f8af2033b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dea5e4e480 0x55dea5e4e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:17.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.905+0000 7f8af2033b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dea5d71120 con 0x55dea5e4e480 2024-05-09T00:48:17.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.905+0000 7f8af2033b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55dea5d16a20 con 0x55dea5d4bd80 2024-05-09T00:48:17.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.905+0000 7f8af2033b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55dea5d6d0e0 con 0x55dea5cd1520 2024-05-09T00:48:17.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.905+0000 7f8ae1bba700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dea5e4e480 0x55dea5e4e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:17.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.905+0000 7f8ae13b9700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dea5d4bd80 0x55dea5e542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:17.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.905+0000 7f8ae1bba700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dea5e4e480 0x55dea5e4e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46902/0 (socket says 172.21.15.5:46902) 2024-05-09T00:48:17.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.905+0000 7f8ae1bba700 1 -- 172.21.15.5:0/1816248191 learned_addr learned my addr 172.21.15.5:0/1816248191 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:17.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.905+0000 7f8ae0bb8700 1 --2- 172.21.15.5:0/1816248191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dea5cd1520 0x55dea5e54800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:17.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.905+0000 7f8ae13b9700 1 -- 172.21.15.5:0/1816248191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dea5cd1520 msgr2=0x55dea5e54800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:17.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.905+0000 7f8ae13b9700 1 --2- 172.21.15.5:0/1816248191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dea5cd1520 0x55dea5e54800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:17.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.905+0000 7f8ae13b9700 1 -- 172.21.15.5:0/1816248191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dea5e4e480 msgr2=0x55dea5e4e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:17.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.905+0000 7f8ae13b9700 1 --2- 172.21.15.5:0/1816248191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dea5e4e480 0x55dea5e4e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:17.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.905+0000 7f8ae13b9700 1 -- 172.21.15.5:0/1816248191 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dea5d07580 con 0x55dea5d4bd80 2024-05-09T00:48:17.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.905+0000 7f8ae13b9700 1 --2- 172.21.15.5:0/1816248191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dea5d4bd80 0x55dea5e542c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8ad80059d0 tx=0x7f8ad801d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72497c6023e4d139 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:17.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.906+0000 7f8ac77fe700 1 -- 172.21.15.5:0/1816248191 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad800be10 con 0x55dea5d4bd80 2024-05-09T00:48:17.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.906+0000 7f8ac77fe700 1 -- 172.21.15.5:0/1816248191 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ad8004c60 con 0x55dea5d4bd80 2024-05-09T00:48:17.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.907+0000 7f8ac77fe700 1 -- 172.21.15.5:0/1816248191 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad800fb90 con 0x55dea5d4bd80 2024-05-09T00:48:17.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.907+0000 7f8af2033b80 1 -- 172.21.15.5:0/1816248191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dea5d4bd80 msgr2=0x55dea5e542c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:17.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.907+0000 7f8af2033b80 1 --2- 172.21.15.5:0/1816248191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dea5d4bd80 0x55dea5e542c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8ad80059d0 tx=0x7f8ad801d000 comp rx=0 tx=0).stop 2024-05-09T00:48:17.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.907+0000 7f8af2033b80 1 -- 172.21.15.5:0/1816248191 shutdown_connections 2024-05-09T00:48:17.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.907+0000 7f8af2033b80 1 --2- 172.21.15.5:0/1816248191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dea5e4e480 0x55dea5e4e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:17.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.907+0000 7f8af2033b80 1 --2- 172.21.15.5:0/1816248191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dea5d4bd80 0x55dea5e542c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:17.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.907+0000 7f8af2033b80 1 --2- 172.21.15.5:0/1816248191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dea5cd1520 0x55dea5e54800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:17.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.907+0000 7f8af2033b80 1 -- 172.21.15.5:0/1816248191 >> 172.21.15.5:0/1816248191 conn(0x55dea5cd4ff0 msgr2=0x55dea5e492a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:17.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.907+0000 7f8af2033b80 1 -- 172.21.15.5:0/1816248191 shutdown_connections 2024-05-09T00:48:17.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.907+0000 7f8af2033b80 1 -- 172.21.15.5:0/1816248191 wait complete. 2024-05-09T00:48:17.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.908+0000 7f8af2033b80 1 -- start start 2024-05-09T00:48:17.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.908+0000 7f8af2033b80 1 Processor -- start 2024-05-09T00:48:17.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.908+0000 7f8af2033b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dea5cd1520 0x55dea5e580a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:17.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.908+0000 7f8af2033b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dea5d4bd80 0x55dea5e585e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:17.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.909+0000 7f8af2033b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dea5e4e480 0x55dea5e58b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:17.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.909+0000 7f8af2033b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dea5e4f1d0 con 0x55dea5cd1520 2024-05-09T00:48:17.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.909+0000 7f8ae13b9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dea5cd1520 0x55dea5e580a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:17.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.909+0000 7f8af2033b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55dea5d0bba0 con 0x55dea5d4bd80 2024-05-09T00:48:17.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.909+0000 7f8af2033b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55dea5e5b0a0 con 0x55dea5e4e480 2024-05-09T00:48:17.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.909+0000 7f8af2033b80 10 client.0 _ll_register_callbacks cb 0x55dea5be2740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:48:17.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.909+0000 7f8ae13b9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dea5cd1520 0x55dea5e580a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46912/0 (socket says 172.21.15.5:46912) 2024-05-09T00:48:17.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.909+0000 7f8ae13b9700 1 -- 172.21.15.5:0/2539208952 learned_addr learned my addr 172.21.15.5:0/2539208952 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:17.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.909+0000 7f8ae0bb8700 1 --2- 172.21.15.5:0/2539208952 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dea5d4bd80 0x55dea5e585e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:17.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.909+0000 7f8ae1bba700 1 --2- 172.21.15.5:0/2539208952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dea5e4e480 0x55dea5e58b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:17.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.909+0000 7f8ae1bba700 1 -- 172.21.15.5:0/2539208952 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dea5d4bd80 msgr2=0x55dea5e585e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:17.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.909+0000 7f8ae1bba700 1 --2- 172.21.15.5:0/2539208952 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dea5d4bd80 0x55dea5e585e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:17.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.909+0000 7f8ae1bba700 1 -- 172.21.15.5:0/2539208952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dea5cd1520 msgr2=0x55dea5e580a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:17.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.909+0000 7f8ae1bba700 1 --2- 172.21.15.5:0/2539208952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dea5cd1520 0x55dea5e580a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:17.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.909+0000 7f8ae1bba700 1 -- 172.21.15.5:0/2539208952 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dea5bec510 con 0x55dea5e4e480 2024-05-09T00:48:17.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.909+0000 7f8ae13b9700 1 --2- 172.21.15.5:0/2539208952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dea5cd1520 0x55dea5e580a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:48:17.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.910+0000 7f8ae1bba700 1 --2- 172.21.15.5:0/2539208952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dea5e4e480 0x55dea5e58b20 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4007bc0 tx=0x7f8ad4009510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eaa953687aa4588c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:17.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.910+0000 7f8ac4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-09T00:48:17.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.910+0000 7f8ac4ff9700 1 -- 172.21.15.5:0/2539208952 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad401a030 con 0x55dea5e4e480 2024-05-09T00:48:17.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.910+0000 7f8af2033b80 1 -- 172.21.15.5:0/2539208952 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ad8004c60 con 0x55dea5e4e480 2024-05-09T00:48:17.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.910+0000 7f8ac4ff9700 1 -- 172.21.15.5:0/2539208952 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f8ad4003a00 con 0x55dea5e4e480 2024-05-09T00:48:17.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.910+0000 7f8ac4ff9700 1 -- 172.21.15.5:0/2539208952 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8ad4026050 con 0x55dea5e4e480 2024-05-09T00:48:17.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.910+0000 7f8af2033b80 10 client.25243 fetch_fsmap learned FSMap version 29 2024-05-09T00:48:17.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.910+0000 7f8af2033b80 1 -- 172.21.15.5:0/2539208952 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55dea5e65b50 con 0x55dea5e4e480 2024-05-09T00:48:17.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.911+0000 7f8ac4ff9700 1 -- 172.21.15.5:0/2539208952 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ad8004c60 con 0x55dea5e4e480 2024-05-09T00:48:17.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.911+0000 7f8ac4ff9700 1 -- 172.21.15.5:0/2539208952 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f8ad4016810 con 0x55dea5e4e480 2024-05-09T00:48:17.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.911+0000 7f8af2033b80 10 client.25243 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:48:17.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.911+0000 7f8af2033b80 10 client.25243 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:48:17.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.911+0000 7f8af2033b80 20 client.25243 populate_metadata read hostname 'smithi005' 2024-05-09T00:48:17.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.911+0000 7f8af2033b80 1 --2- 172.21.15.5:0/2539208952 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55dea5e68540 0x55dea5e6a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:17.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.911+0000 7f8af2033b80 4 client.25243 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:48:17.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.911+0000 7f8af2033b80 1 -- 172.21.15.5:0/2539208952 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dea5d7eb00 con 0x55dea5e68540 2024-05-09T00:48:17.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.911+0000 7f8ae13b9700 1 --2- 172.21.15.5:0/2539208952 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55dea5e68540 0x55dea5e6a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:17.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.912+0000 7f8ac4ff9700 10 client.25243 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:48:17.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.912+0000 7f8ae13b9700 1 --2- 172.21.15.5:0/2539208952 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55dea5e68540 0x55dea5e6a9f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26aa3ff24beffce9 server_cookie=ed700033727e2f13 in_seq=0 out_seq=0 2024-05-09T00:48:17.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.912+0000 7f8ac4ff9700 1 -- 172.21.15.5:0/2539208952 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dea5d7eb00 con 0x55dea5e68540 2024-05-09T00:48:17.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.912+0000 7f8ac4ff9700 10 client.25243 handle_command_reply: tid=0 2024-05-09T00:48:17.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.913+0000 7f8af2033b80 1 client.25243 shutdown 2024-05-09T00:48:17.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.913+0000 7f8af2033b80 1 -- 172.21.15.5:0/2539208952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dea5e4e480 msgr2=0x55dea5e58b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:17.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.913+0000 7f8af2033b80 1 --2- 172.21.15.5:0/2539208952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dea5e4e480 0x55dea5e58b20 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4007bc0 tx=0x7f8ad4009510 comp rx=0 tx=0).stop 2024-05-09T00:48:17.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.913+0000 7f8af2033b80 1 -- 172.21.15.5:0/2539208952 shutdown_connections 2024-05-09T00:48:17.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.913+0000 7f8af2033b80 1 --2- 172.21.15.5:0/2539208952 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55dea5e68540 0x55dea5e6a9f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:17.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.913+0000 7f8af2033b80 1 --2- 172.21.15.5:0/2539208952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dea5e4e480 0x55dea5e58b20 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:17.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.913+0000 7f8ac4ff9700 0 client.25243 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:48:17.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.913+0000 7f8af2033b80 1 --2- 172.21.15.5:0/2539208952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dea5cd1520 0x55dea5e580a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:17.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.913+0000 7f8af2033b80 1 --2- 172.21.15.5:0/2539208952 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dea5d4bd80 0x55dea5e585e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:17.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.913+0000 7f8af2033b80 1 -- 172.21.15.5:0/2539208952 >> 172.21.15.5:0/2539208952 conn(0x55dea5cd4ff0 msgr2=0x55dea5e43070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:17.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.913+0000 7f8af2033b80 1 -- 172.21.15.5:0/2539208952 shutdown_connections 2024-05-09T00:48:17.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.913+0000 7f8af2033b80 1 -- 172.21.15.5:0/2539208952 wait complete. 2024-05-09T00:48:17.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.914+0000 7f8af2033b80 20 client.25243 trim_cache size 0 max 16384 2024-05-09T00:48:17.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.953+0000 7f8af2033b80 1 Processor -- start 2024-05-09T00:48:17.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.953+0000 7f8af2033b80 1 -- start start 2024-05-09T00:48:17.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.954+0000 7f8af2033b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dea5cd1520 0x55dea5e43aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:17.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.954+0000 7f8af2033b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dea5d4bd80 0x55dea5e58610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:17.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.954+0000 7f8af2033b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dea5e4e480 0x55dea5e701f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:17.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.954+0000 7f8af2033b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8adc1b0c00 con 0x55dea5cd1520 2024-05-09T00:48:17.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.954+0000 7f8af2033b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8adc1b2740 con 0x55dea5e4e480 2024-05-09T00:48:17.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.954+0000 7f8af2033b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8adc1b12d0 con 0x55dea5d4bd80 2024-05-09T00:48:17.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.954+0000 7f8ae13b9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dea5cd1520 0x55dea5e43aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:17.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.954+0000 7f8ae13b9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dea5cd1520 0x55dea5e43aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46918/0 (socket says 172.21.15.5:46918) 2024-05-09T00:48:17.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.954+0000 7f8ae13b9700 1 -- 172.21.15.5:0/1638188002 learned_addr learned my addr 172.21.15.5:0/1638188002 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:17.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.954+0000 7f8ae0bb8700 1 --2- 172.21.15.5:0/1638188002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dea5d4bd80 0x55dea5e58610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:17.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.954+0000 7f8ae1bba700 1 --2- 172.21.15.5:0/1638188002 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dea5e4e480 0x55dea5e701f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:17.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.954+0000 7f8ae13b9700 1 -- 172.21.15.5:0/1638188002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dea5d4bd80 msgr2=0x55dea5e58610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:17.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.954+0000 7f8ae13b9700 1 --2- 172.21.15.5:0/1638188002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dea5d4bd80 0x55dea5e58610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:17.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.954+0000 7f8ae13b9700 1 -- 172.21.15.5:0/1638188002 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dea5e4e480 msgr2=0x55dea5e701f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:17.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.954+0000 7f8ae13b9700 1 --2- 172.21.15.5:0/1638188002 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dea5e4e480 0x55dea5e701f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:17.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.954+0000 7f8ae13b9700 1 -- 172.21.15.5:0/1638188002 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dea5e4aee0 con 0x55dea5cd1520 2024-05-09T00:48:17.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.955+0000 7f8ae13b9700 1 --2- 172.21.15.5:0/1638188002 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dea5cd1520 0x55dea5e43aa0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8009990 tx=0x7f8ad80094b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c990abf94c2169 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:17.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.955+0000 7f8ac6ffd700 1 -- 172.21.15.5:0/1638188002 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad800f7e0 con 0x55dea5cd1520 2024-05-09T00:48:17.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.955+0000 7f8ac6ffd700 1 -- 172.21.15.5:0/1638188002 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ad800fe30 con 0x55dea5cd1520 2024-05-09T00:48:17.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.955+0000 7f8ac6ffd700 1 -- 172.21.15.5:0/1638188002 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad803b330 con 0x55dea5cd1520 2024-05-09T00:48:17.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.956+0000 7f8af2033b80 1 -- 172.21.15.5:0/1638188002 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dea5cd1520 msgr2=0x55dea5e43aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:17.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.956+0000 7f8af2033b80 1 --2- 172.21.15.5:0/1638188002 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dea5cd1520 0x55dea5e43aa0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8009990 tx=0x7f8ad80094b0 comp rx=0 tx=0).stop 2024-05-09T00:48:17.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.956+0000 7f8af2033b80 1 -- 172.21.15.5:0/1638188002 shutdown_connections 2024-05-09T00:48:17.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.956+0000 7f8af2033b80 1 --2- 172.21.15.5:0/1638188002 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dea5e4e480 0x55dea5e701f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:17.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.956+0000 7f8af2033b80 1 --2- 172.21.15.5:0/1638188002 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dea5cd1520 0x55dea5e43aa0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:17.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.956+0000 7f8af2033b80 1 --2- 172.21.15.5:0/1638188002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dea5d4bd80 0x55dea5e58610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:17.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.956+0000 7f8af2033b80 1 -- 172.21.15.5:0/1638188002 >> 172.21.15.5:0/1638188002 conn(0x55dea5cd4ff0 msgr2=0x55dea5e523c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:17.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.956+0000 7f8af2033b80 1 -- 172.21.15.5:0/1638188002 shutdown_connections 2024-05-09T00:48:17.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.956+0000 7f8af2033b80 1 -- 172.21.15.5:0/1638188002 wait complete. 2024-05-09T00:48:17.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.956+0000 7f8af2033b80 1 -- start start 2024-05-09T00:48:17.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.957+0000 7f8af2033b80 1 Processor -- start 2024-05-09T00:48:17.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.957+0000 7f8af2033b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dea5cd1520 0x55dea5e60110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:17.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.957+0000 7f8af2033b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dea5d4bd80 0x55dea5e5b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:17.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.958+0000 7f8ae13b9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dea5cd1520 0x55dea5e60110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:17.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.958+0000 7f8ae13b9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dea5cd1520 0x55dea5e60110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46934/0 (socket says 172.21.15.5:46934) 2024-05-09T00:48:17.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.958+0000 7f8af2033b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dea5e4e480 0x55dea5e5ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:17.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.958+0000 7f8af2033b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8adc1b2070 con 0x55dea5cd1520 2024-05-09T00:48:17.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.958+0000 7f8af2033b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8adc1b19a0 con 0x55dea5e4e480 2024-05-09T00:48:17.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.958+0000 7f8af2033b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8adc1aeba0 con 0x55dea5d4bd80 2024-05-09T00:48:17.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.958+0000 7f8af2033b80 10 client.0 _ll_register_callbacks cb 0x55dea5be2740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:48:17.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.958+0000 7f8ae1bba700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dea5e4e480 0x55dea5e5ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:17.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.958+0000 7f8ae1bba700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dea5e4e480 0x55dea5e5ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:40594/0 (socket says 172.21.15.5:40594) 2024-05-09T00:48:17.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.958+0000 7f8ae1bba700 1 -- 172.21.15.5:0/1574466094 learned_addr learned my addr 172.21.15.5:0/1574466094 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:17.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.958+0000 7f8ae0bb8700 1 --2- 172.21.15.5:0/1574466094 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dea5d4bd80 0x55dea5e5b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:17.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.958+0000 7f8ae1bba700 1 -- 172.21.15.5:0/1574466094 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dea5d4bd80 msgr2=0x55dea5e5b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:17.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.958+0000 7f8ae1bba700 1 --2- 172.21.15.5:0/1574466094 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dea5d4bd80 0x55dea5e5b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:17.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.958+0000 7f8ae1bba700 1 -- 172.21.15.5:0/1574466094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dea5cd1520 msgr2=0x55dea5e60110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:17.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.958+0000 7f8ae1bba700 1 --2- 172.21.15.5:0/1574466094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dea5cd1520 0x55dea5e60110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:17.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.958+0000 7f8ae1bba700 1 -- 172.21.15.5:0/1574466094 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dea5e5bf90 con 0x55dea5e4e480 2024-05-09T00:48:17.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.959+0000 7f8ae1bba700 1 --2- 172.21.15.5:0/1574466094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dea5e4e480 0x55dea5e5ba20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4017f10 tx=0x7f8ad4017f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f18518136bceba03 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:17.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.959+0000 7f8ac4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-09T00:48:17.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.959+0000 7f8ac4ff9700 1 -- 172.21.15.5:0/1574466094 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad401a050 con 0x55dea5e4e480 2024-05-09T00:48:17.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.959+0000 7f8af2033b80 1 -- 172.21.15.5:0/1574466094 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ad800fe30 con 0x55dea5e4e480 2024-05-09T00:48:17.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.959+0000 7f8ac4ff9700 1 -- 172.21.15.5:0/1574466094 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f8ad4016630 con 0x55dea5e4e480 2024-05-09T00:48:17.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.959+0000 7f8ac4ff9700 1 -- 172.21.15.5:0/1574466094 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8ad4004e50 con 0x55dea5e4e480 2024-05-09T00:48:17.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.959+0000 7f8af2033b80 10 client.25211 fetch_fsmap learned FSMap version 29 2024-05-09T00:48:17.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.959+0000 7f8af2033b80 1 -- 172.21.15.5:0/1574466094 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55dea5edb300 con 0x55dea5e4e480 2024-05-09T00:48:17.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.959+0000 7f8ac4ff9700 1 -- 172.21.15.5:0/1574466094 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ad401b040 con 0x55dea5e4e480 2024-05-09T00:48:17.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.960+0000 7f8ac4ff9700 1 -- 172.21.15.5:0/1574466094 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f8ad401f550 con 0x55dea5e4e480 2024-05-09T00:48:17.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.960+0000 7f8af2033b80 10 client.25211 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:48:17.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.960+0000 7f8af2033b80 10 client.25211 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:48:17.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.960+0000 7f8af2033b80 20 client.25211 populate_metadata read hostname 'smithi005' 2024-05-09T00:48:17.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.960+0000 7f8af2033b80 1 --2- 172.21.15.5:0/1574466094 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55dea5e68540 0x55dea5eddb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:17.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.960+0000 7f8af2033b80 4 client.25211 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T00:48:17.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.960+0000 7f8af2033b80 1 -- 172.21.15.5:0/1574466094 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f8ad800fe30 con 0x55dea5e68540 2024-05-09T00:48:17.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.960+0000 7f8ae13b9700 1 --2- 172.21.15.5:0/1574466094 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55dea5e68540 0x55dea5eddb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:17.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.961+0000 7f8ae13b9700 1 --2- 172.21.15.5:0/1574466094 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55dea5e68540 0x55dea5eddb50 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bc440197389ff31 server_cookie=3e333e6633cbd2 in_seq=0 out_seq=0 2024-05-09T00:48:17.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.961+0000 7f8ac4ff9700 10 client.25211 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:48:17.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.962+0000 7f8ac4ff9700 1 -- 172.21.15.5:0/1574466094 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f8ad800fe30 con 0x55dea5e68540 2024-05-09T00:48:17.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.962+0000 7f8ac4ff9700 10 client.25211 handle_command_reply: tid=0 2024-05-09T00:48:17.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.962+0000 7f8af2033b80 1 client.25211 shutdown 2024-05-09T00:48:17.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.962+0000 7f8af2033b80 1 -- 172.21.15.5:0/1574466094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dea5e4e480 msgr2=0x55dea5e5ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:17.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.962+0000 7f8af2033b80 1 --2- 172.21.15.5:0/1574466094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dea5e4e480 0x55dea5e5ba20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4017f10 tx=0x7f8ad4017f40 comp rx=0 tx=0).stop 2024-05-09T00:48:17.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.962+0000 7f8af2033b80 1 -- 172.21.15.5:0/1574466094 shutdown_connections 2024-05-09T00:48:17.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.962+0000 7f8af2033b80 1 --2- 172.21.15.5:0/1574466094 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55dea5e68540 0x55dea5eddb50 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:17.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.962+0000 7f8af2033b80 1 --2- 172.21.15.5:0/1574466094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dea5e4e480 0x55dea5e5ba20 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:17.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.962+0000 7f8af2033b80 1 --2- 172.21.15.5:0/1574466094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dea5cd1520 0x55dea5e60110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:17.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.962+0000 7f8af2033b80 1 --2- 172.21.15.5:0/1574466094 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dea5d4bd80 0x55dea5e5b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:17.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.962+0000 7f8af2033b80 1 -- 172.21.15.5:0/1574466094 >> 172.21.15.5:0/1574466094 conn(0x55dea5cd4ff0 msgr2=0x55dea5e4d470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:17.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.962+0000 7f8ac4ff9700 0 client.25211 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:48:17.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.963+0000 7f8af2033b80 1 -- 172.21.15.5:0/1574466094 shutdown_connections 2024-05-09T00:48:17.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.963+0000 7f8af2033b80 1 -- 172.21.15.5:0/1574466094 wait complete. 2024-05-09T00:48:17.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.963+0000 7f8af2033b80 20 client.25211 trim_cache size 0 max 16384 2024-05-09T00:48:17.966 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-09T00:48:17.966 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-05-09T00:48:17.966 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "267f9ded-7c47-427e-8592-6756fa65aa2e", 2024-05-09T00:48:17.966 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-05-09T00:48:17.966 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-09T00:48:17.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.964+0000 7f8ae361d700 1 -- 172.21.15.5:0/248161141 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f8ac80b2820 msgr2=0x7f8ac80b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:17.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.964+0000 7f8ae361d700 1 --2- 172.21.15.5:0/248161141 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f8ac80b2820 0x7f8ac80b4d30 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8ad800b2e0 tx=0x7f8ad8005940 comp rx=0 tx=0).stop 2024-05-09T00:48:17.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.964+0000 7f8ae361d700 1 -- 172.21.15.5:0/248161141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8adc105dd0 msgr2=0x7f8adc1036b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:17.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.964+0000 7f8ae361d700 1 --2- 172.21.15.5:0/248161141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8adc105dd0 0x7f8adc1036b0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f8acc00d950 tx=0x7f8acc00dd10 comp rx=0 tx=0).stop 2024-05-09T00:48:17.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.964+0000 7f8ae361d700 1 -- 172.21.15.5:0/248161141 shutdown_connections 2024-05-09T00:48:17.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.964+0000 7f8ae361d700 1 --2- 172.21.15.5:0/248161141 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f8ac80b2820 0x7f8ac80b4d30 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:17.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.964+0000 7f8ae361d700 1 --2- 172.21.15.5:0/248161141 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8adc10e8f0 0x7f8adc103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:17.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.964+0000 7f8ae361d700 1 --2- 172.21.15.5:0/248161141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8adc105480 0x7f8adc103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:17.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.964+0000 7f8ae361d700 1 --2- 172.21.15.5:0/248161141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8adc105dd0 0x7f8adc1036b0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:17.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.964+0000 7f8ae361d700 1 -- 172.21.15.5:0/248161141 >> 172.21.15.5:0/248161141 conn(0x7f8adc0fef30 msgr2=0x7f8adc0ff580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:17.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.964+0000 7f8ae361d700 1 -- 172.21.15.5:0/248161141 shutdown_connections 2024-05-09T00:48:17.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:17.965+0000 7f8ae361d700 1 -- 172.21.15.5:0/248161141 wait complete. 2024-05-09T00:48:18.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:48:18 smithi005 ceph-mon[30365]: scrub queued for path: / 2024-05-09T00:48:18.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:48:18 smithi005 ceph-mon[30365]: scrub summary: idle+waiting paths [/] 2024-05-09T00:48:18.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:48:18 smithi005 ceph-mon[30365]: scrub summary: active paths [/] 2024-05-09T00:48:18.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:48:18 smithi005 ceph-mon[30365]: scrub summary: idle+waiting paths [/] 2024-05-09T00:48:18.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:48:18 smithi131 ceph-mon[34547]: scrub queued for path: / 2024-05-09T00:48:18.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:48:18 smithi131 ceph-mon[34547]: scrub summary: idle+waiting paths [/] 2024-05-09T00:48:18.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:48:18 smithi131 ceph-mon[34547]: scrub summary: active paths [/] 2024-05-09T00:48:18.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:48:18 smithi131 ceph-mon[34547]: scrub summary: idle+waiting paths [/] 2024-05-09T00:48:18.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:48:18 smithi157 ceph-mon[34294]: scrub queued for path: / 2024-05-09T00:48:18.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:48:18 smithi157 ceph-mon[34294]: scrub summary: idle+waiting paths [/] 2024-05-09T00:48:18.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:48:18 smithi157 ceph-mon[34294]: scrub summary: active paths [/] 2024-05-09T00:48:18.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:48:18 smithi157 ceph-mon[34294]: scrub summary: idle+waiting paths [/] 2024-05-09T00:48:18.783 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T00:48:19.073 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:48:19.461 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:48:19 smithi005 ceph-mon[30365]: pgmap v430: 97 pgs: 97 active+clean; 669 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T00:48:19.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:48:19 smithi131 ceph-mon[34547]: pgmap v430: 97 pgs: 97 active+clean; 669 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T00:48:19.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:48:19 smithi157 ceph-mon[34294]: pgmap v430: 97 pgs: 97 active+clean; 669 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T00:48:20.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:20.997+0000 7f2b63fff700 1 -- 172.21.15.5:0/3854560995 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b6410e750 msgr2=0x7f2b64110c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:20.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:20.997+0000 7f2b63fff700 1 --2- 172.21.15.5:0/3854560995 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b6410e750 0x7f2b64110c10 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c009990 tx=0x7f2b5c009ca0 comp rx=0 tx=0).stop 2024-05-09T00:48:20.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:20.997+0000 7f2b63fff700 1 -- 172.21.15.5:0/3854560995 shutdown_connections 2024-05-09T00:48:20.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:20.997+0000 7f2b63fff700 1 --2- 172.21.15.5:0/3854560995 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b6410e750 0x7f2b64110c10 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:20.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:20.997+0000 7f2b63fff700 1 --2- 172.21.15.5:0/3854560995 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2b64105370 0x7f2b64105780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:20.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:20.997+0000 7f2b63fff700 1 --2- 172.21.15.5:0/3854560995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b64105cc0 0x7f2b6410e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:20.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:20.997+0000 7f2b63fff700 1 -- 172.21.15.5:0/3854560995 >> 172.21.15.5:0/3854560995 conn(0x7f2b64078c40 msgr2=0x7f2b64079080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:20.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:20.997+0000 7f2b63fff700 1 -- 172.21.15.5:0/3854560995 shutdown_connections 2024-05-09T00:48:20.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:20.998+0000 7f2b63fff700 1 -- 172.21.15.5:0/3854560995 wait complete. 2024-05-09T00:48:20.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:20.998+0000 7f2b63fff700 1 Processor -- start 2024-05-09T00:48:20.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:20.999+0000 7f2b63fff700 1 -- start start 2024-05-09T00:48:21.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:20.999+0000 7f2b63fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b64105370 0x7f2b641a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:21.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:20.999+0000 7f2b63fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b64105cc0 0x7f2b641a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:21.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:20.999+0000 7f2b63fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2b6410e750 0x7f2b6419a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:21.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:20.999+0000 7f2b63fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2b64114290 con 0x7f2b64105cc0 2024-05-09T00:48:21.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:20.999+0000 7f2b63fff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2b64114110 con 0x7f2b6410e750 2024-05-09T00:48:21.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:20.999+0000 7f2b63fff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2b64114410 con 0x7f2b64105370 2024-05-09T00:48:21.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:20.999+0000 7f2b627fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b64105cc0 0x7f2b641a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:21.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:20.999+0000 7f2b627fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b64105cc0 0x7f2b641a0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46954/0 (socket says 172.21.15.5:46954) 2024-05-09T00:48:21.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:20.999+0000 7f2b627fc700 1 -- 172.21.15.5:0/3178531345 learned_addr learned my addr 172.21.15.5:0/3178531345 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:21.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:20.999+0000 7f2b62ffd700 1 --2- 172.21.15.5:0/3178531345 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b64105370 0x7f2b641a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:21.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.000+0000 7f2b637fe700 1 --2- 172.21.15.5:0/3178531345 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2b6410e750 0x7f2b6419a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:21.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.000+0000 7f2b62ffd700 1 -- 172.21.15.5:0/3178531345 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2b6410e750 msgr2=0x7f2b6419a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:21.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.000+0000 7f2b62ffd700 1 --2- 172.21.15.5:0/3178531345 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2b6410e750 0x7f2b6419a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:21.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.000+0000 7f2b62ffd700 1 -- 172.21.15.5:0/3178531345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b64105cc0 msgr2=0x7f2b641a0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:21.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.000+0000 7f2b62ffd700 1 --2- 172.21.15.5:0/3178531345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b64105cc0 0x7f2b641a0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:21.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.000+0000 7f2b62ffd700 1 -- 172.21.15.5:0/3178531345 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b6419aff0 con 0x7f2b64105370 2024-05-09T00:48:21.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.000+0000 7f2b627fc700 1 --2- 172.21.15.5:0/3178531345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b64105cc0 0x7f2b641a0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:48:21.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.000+0000 7f2b62ffd700 1 --2- 172.21.15.5:0/3178531345 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b64105370 0x7f2b641a0790 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2b54007d60 tx=0x7f2b5400a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:21.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.001+0000 7f2b68800700 1 -- 172.21.15.5:0/3178531345 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b54016020 con 0x7f2b64105370 2024-05-09T00:48:21.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.001+0000 7f2b63fff700 1 -- 172.21.15.5:0/3178531345 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b6419b2e0 con 0x7f2b64105370 2024-05-09T00:48:21.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.001+0000 7f2b68800700 1 -- 172.21.15.5:0/3178531345 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b5400ac90 con 0x7f2b64105370 2024-05-09T00:48:21.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.001+0000 7f2b63fff700 1 -- 172.21.15.5:0/3178531345 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b641a7620 con 0x7f2b64105370 2024-05-09T00:48:21.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.001+0000 7f2b68800700 1 -- 172.21.15.5:0/3178531345 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b540115e0 con 0x7f2b64105370 2024-05-09T00:48:21.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.004+0000 7f2b68800700 1 -- 172.21.15.5:0/3178531345 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f2b54019020 con 0x7f2b64105370 2024-05-09T00:48:21.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.005+0000 7f2b68800700 1 --2- 172.21.15.5:0/3178531345 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f2b4c0b6d10 0x7f2b4c0b9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:21.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.005+0000 7f2b627fc700 1 --2- 172.21.15.5:0/3178531345 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f2b4c0b6d10 0x7f2b4c0b9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:21.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.005+0000 7f2b68800700 1 -- 172.21.15.5:0/3178531345 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f2b540e1af0 con 0x7f2b64105370 2024-05-09T00:48:21.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.006+0000 7f2b627fc700 1 --2- 172.21.15.5:0/3178531345 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f2b4c0b6d10 0x7f2b4c0b9220 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2b64102470 tx=0x7f2b5800b500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:21.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.006+0000 7f2b78218b80 1 Processor -- start 2024-05-09T00:48:21.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.006+0000 7f2b78218b80 1 -- start start 2024-05-09T00:48:21.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.007+0000 7f2b78218b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff164c5d00 0x55ff165c8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:21.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.007+0000 7f2b78218b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff1644b4a0 0x55ff165c8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:21.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.007+0000 7f2b78218b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff165d2540 0x55ff165d4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:21.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.007+0000 7f2b78218b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ff164eb0a0 con 0x55ff165d2540 2024-05-09T00:48:21.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.007+0000 7f2b78218b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55ff164909a0 con 0x55ff164c5d00 2024-05-09T00:48:21.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.007+0000 7f2b78218b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ff164e7060 con 0x55ff1644b4a0 2024-05-09T00:48:21.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.007+0000 7f2b62ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff164c5d00 0x55ff165c8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:21.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.007+0000 7f2b627fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff1644b4a0 0x55ff165c8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:21.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.007+0000 7f2b627fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff1644b4a0 0x55ff165c8f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:53406/0 (socket says 172.21.15.5:53406) 2024-05-09T00:48:21.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.007+0000 7f2b627fc700 1 -- 172.21.15.5:0/2551753355 learned_addr learned my addr 172.21.15.5:0/2551753355 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:21.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.008+0000 7f2b637fe700 1 --2- 172.21.15.5:0/2551753355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff165d2540 0x55ff165d4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:21.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.008+0000 7f2b627fc700 1 -- 172.21.15.5:0/2551753355 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff164c5d00 msgr2=0x55ff165c8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:21.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.008+0000 7f2b627fc700 1 --2- 172.21.15.5:0/2551753355 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff164c5d00 0x55ff165c8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:21.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.008+0000 7f2b627fc700 1 -- 172.21.15.5:0/2551753355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff165d2540 msgr2=0x55ff165d4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:21.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.008+0000 7f2b627fc700 1 --2- 172.21.15.5:0/2551753355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff165d2540 0x55ff165d4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:21.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.008+0000 7f2b627fc700 1 -- 172.21.15.5:0/2551753355 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff16481500 con 0x55ff1644b4a0 2024-05-09T00:48:21.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.008+0000 7f2b627fc700 1 --2- 172.21.15.5:0/2551753355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff1644b4a0 0x55ff165c8f70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2b58011790 tx=0x7f2b58013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5ad97d218e1c45f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:21.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.008+0000 7f2b49ffb700 1 -- 172.21.15.5:0/2551753355 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5801e620 con 0x55ff1644b4a0 2024-05-09T00:48:21.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.008+0000 7f2b49ffb700 1 -- 172.21.15.5:0/2551753355 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b5801ec70 con 0x55ff1644b4a0 2024-05-09T00:48:21.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.008+0000 7f2b49ffb700 1 -- 172.21.15.5:0/2551753355 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5801cbd0 con 0x55ff1644b4a0 2024-05-09T00:48:21.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.009+0000 7f2b78218b80 1 -- 172.21.15.5:0/2551753355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff1644b4a0 msgr2=0x55ff165c8f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:21.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.009+0000 7f2b78218b80 1 --2- 172.21.15.5:0/2551753355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff1644b4a0 0x55ff165c8f70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2b58011790 tx=0x7f2b58013000 comp rx=0 tx=0).stop 2024-05-09T00:48:21.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.009+0000 7f2b78218b80 1 -- 172.21.15.5:0/2551753355 shutdown_connections 2024-05-09T00:48:21.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.009+0000 7f2b78218b80 1 --2- 172.21.15.5:0/2551753355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff165d2540 0x55ff165d4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:21.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.009+0000 7f2b78218b80 1 --2- 172.21.15.5:0/2551753355 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff164c5d00 0x55ff165c8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:21.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.009+0000 7f2b78218b80 1 --2- 172.21.15.5:0/2551753355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff1644b4a0 0x55ff165c8f70 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:21.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.009+0000 7f2b78218b80 1 -- 172.21.15.5:0/2551753355 >> 172.21.15.5:0/2551753355 conn(0x55ff1644ef70 msgr2=0x55ff165c3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:21.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.009+0000 7f2b78218b80 1 -- 172.21.15.5:0/2551753355 shutdown_connections 2024-05-09T00:48:21.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.009+0000 7f2b78218b80 1 -- 172.21.15.5:0/2551753355 wait complete. 2024-05-09T00:48:21.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.010+0000 7f2b78218b80 1 -- start start 2024-05-09T00:48:21.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.010+0000 7f2b78218b80 1 Processor -- start 2024-05-09T00:48:21.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.010+0000 7f2b78218b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff1644b4a0 0x55ff165cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:21.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.010+0000 7f2b78218b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff164c5d00 0x55ff165cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:21.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.010+0000 7f2b78218b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff165d2540 0x55ff165d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:21.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.011+0000 7f2b78218b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ff165d4f60 con 0x55ff165d2540 2024-05-09T00:48:21.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.011+0000 7f2b78218b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55ff16485b20 con 0x55ff164c5d00 2024-05-09T00:48:21.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.011+0000 7f2b637fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff165d2540 0x55ff165d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:21.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.011+0000 7f2b62ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff1644b4a0 0x55ff165cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:21.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.011+0000 7f2b637fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff165d2540 0x55ff165d00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46964/0 (socket says 172.21.15.5:46964) 2024-05-09T00:48:21.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.011+0000 7f2b637fe700 1 -- 172.21.15.5:0/3506738094 learned_addr learned my addr 172.21.15.5:0/3506738094 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:21.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.011+0000 7f2b78218b80 1 -- 172.21.15.5:0/3506738094 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ff165d50e0 con 0x55ff1644b4a0 2024-05-09T00:48:21.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.011+0000 7f2b78218b80 10 client.0 _ll_register_callbacks cb 0x55ff1635c6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:48:21.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.011+0000 7f2b627fc700 1 --2- 172.21.15.5:0/3506738094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff164c5d00 0x55ff165cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:21.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.011+0000 7f2b637fe700 1 -- 172.21.15.5:0/3506738094 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff1644b4a0 msgr2=0x55ff165cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:21.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.011+0000 7f2b637fe700 1 --2- 172.21.15.5:0/3506738094 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff1644b4a0 0x55ff165cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:21.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.011+0000 7f2b637fe700 1 -- 172.21.15.5:0/3506738094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff164c5d00 msgr2=0x55ff165cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:21.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.011+0000 7f2b637fe700 1 --2- 172.21.15.5:0/3506738094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff164c5d00 0x55ff165cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:21.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.011+0000 7f2b637fe700 1 -- 172.21.15.5:0/3506738094 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff16366490 con 0x55ff165d2540 2024-05-09T00:48:21.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.011+0000 7f2b627fc700 1 --2- 172.21.15.5:0/3506738094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff164c5d00 0x55ff165cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:48:21.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.011+0000 7f2b637fe700 1 --2- 172.21.15.5:0/3506738094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff165d2540 0x55ff165d00c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c015000 tx=0x7f2b5c003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bf3e228213be099 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:21.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.011+0000 7f2b42ffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:48:21.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.012+0000 7f2b42ffd700 1 -- 172.21.15.5:0/3506738094 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c0054c0 con 0x55ff165d2540 2024-05-09T00:48:21.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.012+0000 7f2b42ffd700 1 -- 172.21.15.5:0/3506738094 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f2b5c005660 con 0x55ff165d2540 2024-05-09T00:48:21.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.012+0000 7f2b78218b80 1 -- 172.21.15.5:0/3506738094 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b5801ec70 con 0x55ff165d2540 2024-05-09T00:48:21.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.012+0000 7f2b42ffd700 1 -- 172.21.15.5:0/3506738094 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2b5c00b7d0 con 0x55ff165d2540 2024-05-09T00:48:21.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.012+0000 7f2b78218b80 10 client.15516 fetch_fsmap learned FSMap version 29 2024-05-09T00:48:21.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.012+0000 7f2b78218b80 1 -- 172.21.15.5:0/3506738094 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ff165dfca0 con 0x55ff165d2540 2024-05-09T00:48:21.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.012+0000 7f2b42ffd700 1 -- 172.21.15.5:0/3506738094 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b5c027070 con 0x55ff165d2540 2024-05-09T00:48:21.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.013+0000 7f2b42ffd700 1 -- 172.21.15.5:0/3506738094 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f2b5c00f9f0 con 0x55ff165d2540 2024-05-09T00:48:21.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.013+0000 7f2b78218b80 10 client.15516 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:48:21.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.013+0000 7f2b78218b80 10 client.15516 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:48:21.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.013+0000 7f2b78218b80 20 client.15516 populate_metadata read hostname 'smithi005' 2024-05-09T00:48:21.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.013+0000 7f2b78218b80 1 --2- 172.21.15.5:0/3506738094 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55ff165e2730 0x55ff165e4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:21.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.013+0000 7f2b78218b80 4 client.15516 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:48:21.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.013+0000 7f2b78218b80 1 -- 172.21.15.5:0/3506738094 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2b5801ec70 con 0x55ff165e2730 2024-05-09T00:48:21.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.014+0000 7f2b62ffd700 1 --2- 172.21.15.5:0/3506738094 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55ff165e2730 0x55ff165e4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:21.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.014+0000 7f2b42ffd700 10 client.15516 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:48:21.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.014+0000 7f2b62ffd700 1 --2- 172.21.15.5:0/3506738094 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55ff165e2730 0x55ff165e4bc0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3918c5522e57913 server_cookie=5103eda7edc10443 in_seq=0 out_seq=0 2024-05-09T00:48:21.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.015+0000 7f2b42ffd700 1 -- 172.21.15.5:0/3506738094 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f2b5801ec70 con 0x55ff165e2730 2024-05-09T00:48:21.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.015+0000 7f2b42ffd700 10 client.15516 handle_command_reply: tid=0 2024-05-09T00:48:21.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.015+0000 7f2b78218b80 1 client.15516 shutdown 2024-05-09T00:48:21.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.016+0000 7f2b78218b80 1 -- 172.21.15.5:0/3506738094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff165d2540 msgr2=0x55ff165d00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:21.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.016+0000 7f2b78218b80 1 --2- 172.21.15.5:0/3506738094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff165d2540 0x55ff165d00c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c015000 tx=0x7f2b5c003f90 comp rx=0 tx=0).stop 2024-05-09T00:48:21.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.016+0000 7f2b78218b80 1 -- 172.21.15.5:0/3506738094 shutdown_connections 2024-05-09T00:48:21.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.016+0000 7f2b78218b80 1 --2- 172.21.15.5:0/3506738094 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55ff165e2730 0x55ff165e4bc0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:21.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.016+0000 7f2b78218b80 1 --2- 172.21.15.5:0/3506738094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff165d2540 0x55ff165d00c0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:21.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.016+0000 7f2b78218b80 1 --2- 172.21.15.5:0/3506738094 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff1644b4a0 0x55ff165cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:21.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.016+0000 7f2b78218b80 1 --2- 172.21.15.5:0/3506738094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff164c5d00 0x55ff165cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:21.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.016+0000 7f2b78218b80 1 -- 172.21.15.5:0/3506738094 >> 172.21.15.5:0/3506738094 conn(0x55ff1644ef70 msgr2=0x55ff165c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:21.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.016+0000 7f2b42ffd700 0 client.15516 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:48:21.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.016+0000 7f2b78218b80 1 -- 172.21.15.5:0/3506738094 shutdown_connections 2024-05-09T00:48:21.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.016+0000 7f2b78218b80 1 -- 172.21.15.5:0/3506738094 wait complete. 2024-05-09T00:48:21.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.016+0000 7f2b78218b80 20 client.15516 trim_cache size 0 max 16384 2024-05-09T00:48:21.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.044+0000 7f2b78218b80 1 Processor -- start 2024-05-09T00:48:21.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.044+0000 7f2b78218b80 1 -- start start 2024-05-09T00:48:21.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.044+0000 7f2b78218b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff1644b4a0 0x55ff165d1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:21.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.044+0000 7f2b78218b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff164c5d00 0x55ff165c7b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:21.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.044+0000 7f2b78218b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff165d2540 0x55ff165cff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:21.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.044+0000 7f2b78218b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2b641ac870 con 0x55ff164c5d00 2024-05-09T00:48:21.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.044+0000 7f2b78218b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2b641ae3b0 con 0x55ff1644b4a0 2024-05-09T00:48:21.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.044+0000 7f2b78218b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2b641acf40 con 0x55ff165d2540 2024-05-09T00:48:21.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.044+0000 7f2b627fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff164c5d00 0x55ff165c7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:21.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.044+0000 7f2b627fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff164c5d00 0x55ff165c7b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46974/0 (socket says 172.21.15.5:46974) 2024-05-09T00:48:21.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.044+0000 7f2b627fc700 1 -- 172.21.15.5:0/78140245 learned_addr learned my addr 172.21.15.5:0/78140245 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:21.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.045+0000 7f2b62ffd700 1 --2- 172.21.15.5:0/78140245 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff1644b4a0 0x55ff165d1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:21.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.045+0000 7f2b637fe700 1 --2- 172.21.15.5:0/78140245 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff165d2540 0x55ff165cff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:21.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.045+0000 7f2b627fc700 1 -- 172.21.15.5:0/78140245 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff165d2540 msgr2=0x55ff165cff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:21.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.045+0000 7f2b627fc700 1 --2- 172.21.15.5:0/78140245 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff165d2540 0x55ff165cff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:21.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.045+0000 7f2b627fc700 1 -- 172.21.15.5:0/78140245 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff1644b4a0 msgr2=0x55ff165d1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:21.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.045+0000 7f2b627fc700 1 --2- 172.21.15.5:0/78140245 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff1644b4a0 0x55ff165d1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:21.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.045+0000 7f2b627fc700 1 -- 172.21.15.5:0/78140245 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff165c4050 con 0x55ff164c5d00 2024-05-09T00:48:21.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.045+0000 7f2b62ffd700 1 --2- 172.21.15.5:0/78140245 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff1644b4a0 0x55ff165d1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:48:21.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.045+0000 7f2b627fc700 1 --2- 172.21.15.5:0/78140245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff164c5d00 0x55ff165c7b00 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f2b58011d80 tx=0x7f2b58005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baf531920340d72a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:21.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.046+0000 7f2b497fa700 1 -- 172.21.15.5:0/78140245 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b58005330 con 0x55ff164c5d00 2024-05-09T00:48:21.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.046+0000 7f2b497fa700 1 -- 172.21.15.5:0/78140245 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b580054d0 con 0x55ff164c5d00 2024-05-09T00:48:21.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.046+0000 7f2b497fa700 1 -- 172.21.15.5:0/78140245 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5801e870 con 0x55ff164c5d00 2024-05-09T00:48:21.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.047+0000 7f2b78218b80 1 -- 172.21.15.5:0/78140245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff164c5d00 msgr2=0x55ff165c7b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:21.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.047+0000 7f2b78218b80 1 --2- 172.21.15.5:0/78140245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff164c5d00 0x55ff165c7b00 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f2b58011d80 tx=0x7f2b58005040 comp rx=0 tx=0).stop 2024-05-09T00:48:21.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.047+0000 7f2b78218b80 1 -- 172.21.15.5:0/78140245 shutdown_connections 2024-05-09T00:48:21.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.047+0000 7f2b78218b80 1 --2- 172.21.15.5:0/78140245 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff165d2540 0x55ff165cff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:21.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.047+0000 7f2b78218b80 1 --2- 172.21.15.5:0/78140245 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff1644b4a0 0x55ff165d1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:21.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.047+0000 7f2b78218b80 1 --2- 172.21.15.5:0/78140245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff164c5d00 0x55ff165c7b00 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:21.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.047+0000 7f2b78218b80 1 -- 172.21.15.5:0/78140245 >> 172.21.15.5:0/78140245 conn(0x55ff1644ef70 msgr2=0x55ff165c73f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:21.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.047+0000 7f2b78218b80 1 -- 172.21.15.5:0/78140245 shutdown_connections 2024-05-09T00:48:21.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.047+0000 7f2b78218b80 1 -- 172.21.15.5:0/78140245 wait complete. 2024-05-09T00:48:21.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.047+0000 7f2b78218b80 1 -- start start 2024-05-09T00:48:21.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.048+0000 7f2b78218b80 1 Processor -- start 2024-05-09T00:48:21.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.048+0000 7f2b78218b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff1644b4a0 0x55ff165d5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:21.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.048+0000 7f2b78218b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff164c5d00 0x55ff165d5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:21.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.048+0000 7f2b62ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff1644b4a0 0x55ff165d5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:21.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.048+0000 7f2b78218b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff165d2540 0x55ff165d5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:21.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.048+0000 7f2b78218b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2b641adce0 con 0x55ff1644b4a0 2024-05-09T00:48:21.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.048+0000 7f2b78218b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2b641ad610 con 0x55ff164c5d00 2024-05-09T00:48:21.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.048+0000 7f2b62ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff1644b4a0 0x55ff165d5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46982/0 (socket says 172.21.15.5:46982) 2024-05-09T00:48:21.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.048+0000 7f2b78218b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2b641aa830 con 0x55ff165d2540 2024-05-09T00:48:21.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.048+0000 7f2b62ffd700 1 -- 172.21.15.5:0/3100044294 learned_addr learned my addr 172.21.15.5:0/3100044294 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:21.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.049+0000 7f2b78218b80 10 client.0 _ll_register_callbacks cb 0x55ff1635c6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:48:21.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.049+0000 7f2b627fc700 1 --2- 172.21.15.5:0/3100044294 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff164c5d00 0x55ff165d5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:21.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.049+0000 7f2b62ffd700 1 -- 172.21.15.5:0/3100044294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff165d2540 msgr2=0x55ff165d5dd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:48:21.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.049+0000 7f2b62ffd700 1 --2- 172.21.15.5:0/3100044294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff165d2540 0x55ff165d5dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:21.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.049+0000 7f2b62ffd700 1 -- 172.21.15.5:0/3100044294 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff164c5d00 msgr2=0x55ff165d5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:21.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.049+0000 7f2b62ffd700 1 --2- 172.21.15.5:0/3100044294 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff164c5d00 0x55ff165d5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:21.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.049+0000 7f2b62ffd700 1 -- 172.21.15.5:0/3100044294 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff165c4c70 con 0x55ff1644b4a0 2024-05-09T00:48:21.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.049+0000 7f2b62ffd700 1 --2- 172.21.15.5:0/3100044294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff1644b4a0 0x55ff165d5330 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f2b540a9070 tx=0x7f2b540060e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e49fd66060ead465 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:21.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.049+0000 7f2b42ffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:48:21.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.050+0000 7f2b42ffd700 1 -- 172.21.15.5:0/3100044294 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b540ad490 con 0x55ff1644b4a0 2024-05-09T00:48:21.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.050+0000 7f2b42ffd700 1 -- 172.21.15.5:0/3100044294 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f2b540a9550 con 0x55ff1644b4a0 2024-05-09T00:48:21.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.050+0000 7f2b78218b80 1 -- 172.21.15.5:0/3100044294 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b580054d0 con 0x55ff1644b4a0 2024-05-09T00:48:21.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.050+0000 7f2b42ffd700 1 -- 172.21.15.5:0/3100044294 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2b540b1f40 con 0x55ff1644b4a0 2024-05-09T00:48:21.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.050+0000 7f2b78218b80 10 client.15522 fetch_fsmap learned FSMap version 29 2024-05-09T00:48:21.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.050+0000 7f2b78218b80 1 -- 172.21.15.5:0/3100044294 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ff1664c2a0 con 0x55ff1644b4a0 2024-05-09T00:48:21.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.050+0000 7f2b42ffd700 1 -- 172.21.15.5:0/3100044294 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b580054d0 con 0x55ff1644b4a0 2024-05-09T00:48:21.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.051+0000 7f2b42ffd700 1 -- 172.21.15.5:0/3100044294 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f2b540a7db0 con 0x55ff1644b4a0 2024-05-09T00:48:21.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.051+0000 7f2b78218b80 10 client.15522 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:48:21.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.051+0000 7f2b78218b80 10 client.15522 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:48:21.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.051+0000 7f2b78218b80 20 client.15522 populate_metadata read hostname 'smithi005' 2024-05-09T00:48:21.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.051+0000 7f2b78218b80 1 --2- 172.21.15.5:0/3100044294 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55ff1664c900 0x55ff1664ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:21.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.051+0000 7f2b78218b80 4 client.15522 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:48:21.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.051+0000 7f2b78218b80 1 -- 172.21.15.5:0/3100044294 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ff164f8a80 con 0x55ff1664c900 2024-05-09T00:48:21.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.051+0000 7f2b627fc700 1 --2- 172.21.15.5:0/3100044294 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55ff1664c900 0x55ff1664ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:21.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.052+0000 7f2b42ffd700 10 client.15522 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:48:21.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.052+0000 7f2b627fc700 1 --2- 172.21.15.5:0/3100044294 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55ff1664c900 0x55ff1664ed20 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c1ebf67dcd5ea8f server_cookie=c396c43730003603 in_seq=0 out_seq=0 2024-05-09T00:48:21.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.052+0000 7f2b42ffd700 1 -- 172.21.15.5:0/3100044294 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ff164f8a80 con 0x55ff1664c900 2024-05-09T00:48:21.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.052+0000 7f2b42ffd700 10 client.15522 handle_command_reply: tid=0 2024-05-09T00:48:21.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.052+0000 7f2b78218b80 1 client.15522 shutdown 2024-05-09T00:48:21.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.052+0000 7f2b78218b80 1 -- 172.21.15.5:0/3100044294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff1644b4a0 msgr2=0x55ff165d5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:21.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.052+0000 7f2b78218b80 1 --2- 172.21.15.5:0/3100044294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff1644b4a0 0x55ff165d5330 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f2b540a9070 tx=0x7f2b540060e0 comp rx=0 tx=0).stop 2024-05-09T00:48:21.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.053+0000 7f2b78218b80 1 -- 172.21.15.5:0/3100044294 shutdown_connections 2024-05-09T00:48:21.057 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-09T00:48:21.057 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-05-09T00:48:21.057 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-05-09T00:48:21.057 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-09T00:48:21.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.053+0000 7f2b78218b80 1 --2- 172.21.15.5:0/3100044294 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55ff1664c900 0x55ff1664ed20 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:21.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.053+0000 7f2b78218b80 1 --2- 172.21.15.5:0/3100044294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff165d2540 0x55ff165d5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:21.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.053+0000 7f2b78218b80 1 --2- 172.21.15.5:0/3100044294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff1644b4a0 0x55ff165d5330 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:21.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.053+0000 7f2b42ffd700 0 client.15522 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:48:21.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.053+0000 7f2b78218b80 1 --2- 172.21.15.5:0/3100044294 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ff164c5d00 0x55ff165d5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:21.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.053+0000 7f2b78218b80 1 -- 172.21.15.5:0/3100044294 >> 172.21.15.5:0/3100044294 conn(0x55ff1644ef70 msgr2=0x55ff165ca300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:21.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.053+0000 7f2b78218b80 1 -- 172.21.15.5:0/3100044294 shutdown_connections 2024-05-09T00:48:21.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.053+0000 7f2b78218b80 1 -- 172.21.15.5:0/3100044294 wait complete. 2024-05-09T00:48:21.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.053+0000 7f2b78218b80 20 client.15522 trim_cache size 0 max 16384 2024-05-09T00:48:21.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.054+0000 7f2b63fff700 1 -- 172.21.15.5:0/3178531345 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f2b4c0b6d10 msgr2=0x7f2b4c0b9220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:21.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.054+0000 7f2b63fff700 1 --2- 172.21.15.5:0/3178531345 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f2b4c0b6d10 0x7f2b4c0b9220 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2b64102470 tx=0x7f2b5800b500 comp rx=0 tx=0).stop 2024-05-09T00:48:21.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.054+0000 7f2b63fff700 1 -- 172.21.15.5:0/3178531345 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b64105370 msgr2=0x7f2b641a0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:21.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.054+0000 7f2b63fff700 1 --2- 172.21.15.5:0/3178531345 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b64105370 0x7f2b641a0790 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2b54007d60 tx=0x7f2b5400a340 comp rx=0 tx=0).stop 2024-05-09T00:48:21.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.054+0000 7f2b63fff700 1 -- 172.21.15.5:0/3178531345 shutdown_connections 2024-05-09T00:48:21.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.054+0000 7f2b63fff700 1 --2- 172.21.15.5:0/3178531345 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f2b4c0b6d10 0x7f2b4c0b9220 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:21.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.054+0000 7f2b63fff700 1 --2- 172.21.15.5:0/3178531345 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2b6410e750 0x7f2b6419a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:21.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.054+0000 7f2b63fff700 1 --2- 172.21.15.5:0/3178531345 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b64105370 0x7f2b641a0790 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:21.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.054+0000 7f2b63fff700 1 --2- 172.21.15.5:0/3178531345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b64105cc0 0x7f2b641a0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:21.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.054+0000 7f2b63fff700 1 -- 172.21.15.5:0/3178531345 >> 172.21.15.5:0/3178531345 conn(0x7f2b64078c40 msgr2=0x7f2b640ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:21.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.054+0000 7f2b63fff700 1 -- 172.21.15.5:0/3178531345 shutdown_connections 2024-05-09T00:48:21.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:21.054+0000 7f2b63fff700 1 -- 172.21.15.5:0/3178531345 wait complete. 2024-05-09T00:48:21.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:48:21 smithi005 ceph-mon[30365]: pgmap v431: 97 pgs: 97 active+clean; 669 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-09T00:48:21.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:48:21 smithi005 ceph-mon[30365]: scrub summary: idle 2024-05-09T00:48:21.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:48:21 smithi131 ceph-mon[34547]: pgmap v431: 97 pgs: 97 active+clean; 669 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-09T00:48:21.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:48:21 smithi131 ceph-mon[34547]: scrub summary: idle 2024-05-09T00:48:21.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:48:21 smithi157 ceph-mon[34294]: pgmap v431: 97 pgs: 97 active+clean; 669 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-09T00:48:21.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:48:21 smithi157 ceph-mon[34294]: scrub summary: idle 2024-05-09T00:48:21.872 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T00:48:21.873 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T00:48:22.141 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:48:23.521 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:48:23 smithi005 ceph-mon[30365]: pgmap v432: 97 pgs: 97 active+clean; 669 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T00:48:23.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:48:23 smithi131 ceph-mon[34547]: pgmap v432: 97 pgs: 97 active+clean; 669 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T00:48:23.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:48:23 smithi157 ceph-mon[34294]: pgmap v432: 97 pgs: 97 active+clean; 669 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T00:48:24.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:24.115+0000 7f89b27c3700 1 -- 172.21.15.5:0/474018456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89ac108760 msgr2=0x7f89ac1100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:24.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:24.115+0000 7f89b27c3700 1 --2- 172.21.15.5:0/474018456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89ac108760 0x7f89ac1100c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f89a400b0d0 tx=0x7f89a400b3e0 comp rx=0 tx=0).stop 2024-05-09T00:48:24.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:24.115+0000 7f89b27c3700 1 -- 172.21.15.5:0/474018456 shutdown_connections 2024-05-09T00:48:24.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:24.115+0000 7f89b27c3700 1 --2- 172.21.15.5:0/474018456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89ac108760 0x7f89ac1100c0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:24.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:24.115+0000 7f89b27c3700 1 --2- 172.21.15.5:0/474018456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89ac1034e0 0x7f89ac1038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:24.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:24.115+0000 7f89b27c3700 1 --2- 172.21.15.5:0/474018456 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89ac107e10 0x7f89ac108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:24.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:24.115+0000 7f89b27c3700 1 -- 172.21.15.5:0/474018456 >> 172.21.15.5:0/474018456 conn(0x7f89ac0fef30 msgr2=0x7f89ac101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:24.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:24.116+0000 7f89b27c3700 1 -- 172.21.15.5:0/474018456 shutdown_connections 2024-05-09T00:48:24.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:24.116+0000 7f89b27c3700 1 -- 172.21.15.5:0/474018456 wait complete. 2024-05-09T00:48:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:24.117+0000 7f89b27c3700 1 Processor -- start 2024-05-09T00:48:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:24.118+0000 7f89b27c3700 1 -- start start 2024-05-09T00:48:24.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:24.118+0000 7f89b27c3700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89ac1034e0 0x7f89ac110d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:24.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:24.118+0000 7f89b27c3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89ac107e10 0x7f89ac111270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:24.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:24.118+0000 7f89b27c3700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89ac108760 0x7f89ac1117b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:24.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:24.118+0000 7f89b27c3700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f89ac1144b0 con 0x7f89ac107e10 2024-05-09T00:48:24.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:24.118+0000 7f89b27c3700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f89ac114330 con 0x7f89ac1034e0 2024-05-09T00:48:24.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:24.118+0000 7f89b27c3700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f89ac114630 con 0x7f89ac108760 2024-05-09T00:48:24.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:24.118+0000 7f89abfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89ac1034e0 0x7f89ac110d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:24.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:24.118+0000 7f89a3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89ac107e10 0x7f89ac111270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:24.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:24.118+0000 7f89abfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89ac1034e0 0x7f89ac110d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:52990/0 (socket says 172.21.15.5:52990) 2024-05-09T00:48:24.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:24.118+0000 7f89abfff700 1 -- 172.21.15.5:0/2897820871 learned_addr learned my addr 172.21.15.5:0/2897820871 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:24.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:24.118+0000 7f89b0d60700 1 --2- 172.21.15.5:0/2897820871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89ac108760 0x7f89ac1117b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:24.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:24.119+0000 7f89a3fff700 1 -- 172.21.15.5:0/2897820871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89ac108760 msgr2=0x7f89ac1117b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:24.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:24.119+0000 7f89a3fff700 1 --2- 172.21.15.5:0/2897820871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89ac108760 0x7f89ac1117b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:24.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:24.119+0000 7f89a3fff700 1 -- 172.21.15.5:0/2897820871 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89ac1034e0 msgr2=0x7f89ac110d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:24.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:24.119+0000 7f89a3fff700 1 --2- 172.21.15.5:0/2897820871 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89ac1034e0 0x7f89ac110d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:24.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:24.119+0000 7f89a3fff700 1 -- 172.21.15.5:0/2897820871 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89ac1a9260 con 0x7f89ac107e10 2024-05-09T00:48:24.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:24.119+0000 7f89a3fff700 1 --2- 172.21.15.5:0/2897820871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89ac107e10 0x7f89ac111270 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f899400d720 tx=0x7f899400da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:24.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:24.119+0000 7f89a9ffb700 1 -- 172.21.15.5:0/2897820871 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8994014030 con 0x7f89ac107e10 2024-05-09T00:48:24.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:24.119+0000 7f89a9ffb700 1 -- 172.21.15.5:0/2897820871 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89940044d0 con 0x7f89ac107e10 2024-05-09T00:48:24.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:24.119+0000 7f89b27c3700 1 -- 172.21.15.5:0/2897820871 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89ac1a9550 con 0x7f89ac107e10 2024-05-09T00:48:24.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:24.119+0000 7f89a9ffb700 1 -- 172.21.15.5:0/2897820871 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8994005400 con 0x7f89ac107e10 2024-05-09T00:48:24.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:24.119+0000 7f89b27c3700 1 -- 172.21.15.5:0/2897820871 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89ac1a9ae0 con 0x7f89ac107e10 2024-05-09T00:48:24.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:24.120+0000 7f89b27c3700 1 -- 172.21.15.5:0/2897820871 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89ac0512c0 con 0x7f89ac107e10 2024-05-09T00:48:24.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:24.122+0000 7f89a9ffb700 1 -- 172.21.15.5:0/2897820871 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f89940055a0 con 0x7f89ac107e10 2024-05-09T00:48:24.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:24.123+0000 7f89a9ffb700 1 --2- 172.21.15.5:0/2897820871 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f89980b2900 0x7f89980b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:24.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:24.123+0000 7f89a9ffb700 1 -- 172.21.15.5:0/2897820871 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f89940e1d40 con 0x7f89ac107e10 2024-05-09T00:48:24.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:24.123+0000 7f89abfff700 1 --2- 172.21.15.5:0/2897820871 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f89980b2900 0x7f89980b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:24.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:24.124+0000 7f89abfff700 1 --2- 172.21.15.5:0/2897820871 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f89980b2900 0x7f89980b4e10 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f899c00bd30 tx=0x7f899c00b480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:24.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:24.132+0000 7f89a9ffb700 1 -- 172.21.15.5:0/2897820871 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f89940a9a30 con 0x7f89ac107e10 2024-05-09T00:48:24.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.311+0000 7f322f9d5640 1 -- 172.21.15.5:0/2278851040 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32281030c0 msgr2=0x7f32281034d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:24.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.311+0000 7f322f9d5640 1 --2- 172.21.15.5:0/2278851040 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32281030c0 0x7f32281034d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3224009860 tx=0x7f322402a340 comp rx=0 tx=0).stop 2024-05-09T00:48:24.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.311+0000 7f322f9d5640 1 -- 172.21.15.5:0/2278851040 shutdown_connections 2024-05-09T00:48:24.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.311+0000 7f322f9d5640 1 --2- 172.21.15.5:0/2278851040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32281045e0 0x7f322810ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:24.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.311+0000 7f322f9d5640 1 --2- 172.21.15.5:0/2278851040 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3228103aa0 0x7f3228103f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:24.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.311+0000 7f322f9d5640 1 --2- 172.21.15.5:0/2278851040 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32281030c0 0x7f32281034d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:24.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.311+0000 7f322f9d5640 1 -- 172.21.15.5:0/2278851040 >> 172.21.15.5:0/2278851040 conn(0x7f32280fd4c0 msgr2=0x7f32280ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:24.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.312+0000 7f322f9d5640 1 -- 172.21.15.5:0/2278851040 shutdown_connections 2024-05-09T00:48:24.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.312+0000 7f322f9d5640 1 -- 172.21.15.5:0/2278851040 wait complete. 2024-05-09T00:48:24.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.312+0000 7f322f9d5640 1 Processor -- start 2024-05-09T00:48:24.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.313+0000 7f322f9d5640 1 -- start start 2024-05-09T00:48:24.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.313+0000 7f322f9d5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32281030c0 0x7f322819a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:24.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.313+0000 7f322f9d5640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3228103aa0 0x7f322819a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:24.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.313+0000 7f322f9d5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32281045e0 0x7f32281a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:24.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.313+0000 7f322f9d5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f322810f7d0 con 0x7f32281030c0 2024-05-09T00:48:24.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.313+0000 7f322f9d5640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f322810f650 con 0x7f3228103aa0 2024-05-09T00:48:24.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.313+0000 7f322f9d5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f322810f950 con 0x7f32281045e0 2024-05-09T00:48:24.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.313+0000 7f322d771640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32281030c0 0x7f322819a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:24.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.313+0000 7f322d771640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32281030c0 0x7f322819a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59974/0 (socket says 172.21.15.5:59974) 2024-05-09T00:48:24.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.313+0000 7f322d771640 1 -- 172.21.15.5:0/1578855844 learned_addr learned my addr 172.21.15.5:0/1578855844 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:24.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.313+0000 7f322df72640 1 --2- 172.21.15.5:0/1578855844 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32281045e0 0x7f32281a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:24.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.313+0000 7f322cf70640 1 --2- 172.21.15.5:0/1578855844 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3228103aa0 0x7f322819a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:24.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.314+0000 7f322d771640 1 -- 172.21.15.5:0/1578855844 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32281045e0 msgr2=0x7f32281a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:24.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.314+0000 7f322d771640 1 --2- 172.21.15.5:0/1578855844 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32281045e0 0x7f32281a18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:24.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.314+0000 7f322d771640 1 -- 172.21.15.5:0/1578855844 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3228103aa0 msgr2=0x7f322819a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:24.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.314+0000 7f322d771640 1 --2- 172.21.15.5:0/1578855844 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3228103aa0 0x7f322819a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:24.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.314+0000 7f322d771640 1 -- 172.21.15.5:0/1578855844 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32281a1f40 con 0x7f32281030c0 2024-05-09T00:48:24.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.314+0000 7f322d771640 1 --2- 172.21.15.5:0/1578855844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32281030c0 0x7f322819a400 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f322402a860 tx=0x7f3224032d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:24.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.314+0000 7f321e7fc640 1 -- 172.21.15.5:0/1578855844 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3224040020 con 0x7f32281030c0 2024-05-09T00:48:24.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.314+0000 7f321e7fc640 1 -- 172.21.15.5:0/1578855844 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3224030e10 con 0x7f32281030c0 2024-05-09T00:48:24.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.314+0000 7f322f9d5640 1 -- 172.21.15.5:0/1578855844 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32281a21d0 con 0x7f32281030c0 2024-05-09T00:48:24.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.315+0000 7f321e7fc640 1 -- 172.21.15.5:0/1578855844 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3224037040 con 0x7f32281030c0 2024-05-09T00:48:24.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.315+0000 7f322f9d5640 1 -- 172.21.15.5:0/1578855844 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32281a2710 con 0x7f32281030c0 2024-05-09T00:48:24.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.318+0000 7f3236963740 1 Processor -- start 2024-05-09T00:48:24.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.319+0000 7f3236963740 1 -- start start 2024-05-09T00:48:24.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.319+0000 7f3236963740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ffda4f1b0 0x555ffddf68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:24.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.319+0000 7f3236963740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ffda262f0 0x555ffddf6e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:24.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.319+0000 7f3236963740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ffda2e4d0 0x555ffddf3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:24.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.319+0000 7f3236963740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555ffdd577c0 con 0x555ffda2e4d0 2024-05-09T00:48:24.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.319+0000 7f3236963740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x555ffdcf30e0 con 0x555ffda262f0 2024-05-09T00:48:24.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.319+0000 7f3236963740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x555ffdd53780 con 0x555ffda4f1b0 2024-05-09T00:48:24.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.319+0000 7f322cf70640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ffda2e4d0 0x555ffddf3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:24.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.319+0000 7f322cf70640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ffda2e4d0 0x555ffddf3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59990/0 (socket says 172.21.15.5:59990) 2024-05-09T00:48:24.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.319+0000 7f322cf70640 1 -- 172.21.15.5:0/2469619778 learned_addr learned my addr 172.21.15.5:0/2469619778 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:24.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.319+0000 7f322df72640 1 --2- 172.21.15.5:0/2469619778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ffda4f1b0 0x555ffddf68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:24.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.320+0000 7f322cf70640 1 -- 172.21.15.5:0/2469619778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ffda4f1b0 msgr2=0x555ffddf68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:24.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.320+0000 7f322cf70640 1 --2- 172.21.15.5:0/2469619778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ffda4f1b0 0x555ffddf68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:24.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.320+0000 7f322cf70640 1 -- 172.21.15.5:0/2469619778 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ffda262f0 msgr2=0x555ffddf6e60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:48:24.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.320+0000 7f322cf70640 1 --2- 172.21.15.5:0/2469619778 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ffda262f0 0x555ffddf6e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:24.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.320+0000 7f322cf70640 1 -- 172.21.15.5:0/2469619778 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ffdd1c7c0 con 0x555ffda2e4d0 2024-05-09T00:48:24.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.320+0000 7f322cf70640 1 --2- 172.21.15.5:0/2469619778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ffda2e4d0 0x555ffddf3590 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f321800b7f0 tx=0x7f321800bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22f25061992dc7d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:24.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.320+0000 7f32137fe640 1 -- 172.21.15.5:0/2469619778 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f321800d950 con 0x555ffda2e4d0 2024-05-09T00:48:24.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.320+0000 7f32137fe640 1 -- 172.21.15.5:0/2469619778 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3218018070 con 0x555ffda2e4d0 2024-05-09T00:48:24.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.320+0000 7f32137fe640 1 -- 172.21.15.5:0/2469619778 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f321800d950 con 0x555ffda2e4d0 2024-05-09T00:48:24.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.321+0000 7f321e7fc640 1 -- 172.21.15.5:0/1578855844 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f3224043020 con 0x7f32281030c0 2024-05-09T00:48:24.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.322+0000 7f3236963740 1 -- 172.21.15.5:0/2469619778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ffda2e4d0 msgr2=0x555ffddf3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:24.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.322+0000 7f3236963740 1 --2- 172.21.15.5:0/2469619778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ffda2e4d0 0x555ffddf3590 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f321800b7f0 tx=0x7f321800bcb0 comp rx=0 tx=0).stop 2024-05-09T00:48:24.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.322+0000 7f321e7fc640 1 --2- 172.21.15.5:0/1578855844 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f32140b28e0 0x7f32140b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:24.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.322+0000 7f322df72640 1 --2- 172.21.15.5:0/1578855844 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f32140b28e0 0x7f32140b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:24.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.322+0000 7f321e7fc640 1 -- 172.21.15.5:0/1578855844 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f3224101e20 con 0x7f32281030c0 2024-05-09T00:48:24.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.322+0000 7f3236963740 1 -- 172.21.15.5:0/2469619778 shutdown_connections 2024-05-09T00:48:24.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.322+0000 7f3236963740 1 --2- 172.21.15.5:0/2469619778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ffda2e4d0 0x555ffddf3590 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:24.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.322+0000 7f3236963740 1 --2- 172.21.15.5:0/2469619778 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ffda262f0 0x555ffddf6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:24.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.322+0000 7f3236963740 1 --2- 172.21.15.5:0/2469619778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ffda4f1b0 0x555ffddf68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:24.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.322+0000 7f3236963740 1 -- 172.21.15.5:0/2469619778 >> 172.21.15.5:0/2469619778 conn(0x555ffda43300 msgr2=0x555ffddeee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:24.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.322+0000 7f322df72640 1 --2- 172.21.15.5:0/1578855844 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f32140b28e0 0x7f32140b4df0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x555ffddf4650 tx=0x7f3220010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:24.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.323+0000 7f3236963740 1 -- 172.21.15.5:0/2469619778 shutdown_connections 2024-05-09T00:48:24.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.323+0000 7f3236963740 1 -- 172.21.15.5:0/2469619778 wait complete. 2024-05-09T00:48:24.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.323+0000 7f3236963740 1 -- start start 2024-05-09T00:48:24.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.323+0000 7f3236963740 1 Processor -- start 2024-05-09T00:48:24.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.323+0000 7f3236963740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ffda262f0 0x555ffddfd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:24.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.324+0000 7f3236963740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ffda2e4d0 0x555ffddfd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:24.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.324+0000 7f3236963740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ffda4f1b0 0x555ffde02530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:24.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.324+0000 7f3236963740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555ffdcde9c0 con 0x555ffda2e4d0 2024-05-09T00:48:24.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.324+0000 7f3236963740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x555ffddf7560 con 0x555ffda262f0 2024-05-09T00:48:24.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.324+0000 7f3236963740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x555ffde02a70 con 0x555ffda4f1b0 2024-05-09T00:48:24.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.324+0000 7f3236963740 10 client.0 _ll_register_callbacks cb 0x555ffdd0b020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:48:24.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.324+0000 7f322d771640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ffda262f0 0x555ffddfd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:24.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.324+0000 7f322cf70640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ffda2e4d0 0x555ffddfd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:24.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.324+0000 7f322d771640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ffda262f0 0x555ffddfd0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:53010/0 (socket says 172.21.15.5:53010) 2024-05-09T00:48:24.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.324+0000 7f322d771640 1 -- 172.21.15.5:0/2415432325 learned_addr learned my addr 172.21.15.5:0/2415432325 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:24.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.324+0000 7f322cf70640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ffda2e4d0 0x555ffddfd5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59996/0 (socket says 172.21.15.5:59996) 2024-05-09T00:48:24.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.324+0000 7f322df72640 1 --2- 172.21.15.5:0/2415432325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ffda4f1b0 0x555ffde02530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:24.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.324+0000 7f322d771640 1 -- 172.21.15.5:0/2415432325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ffda4f1b0 msgr2=0x555ffde02530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:24.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.324+0000 7f322d771640 1 --2- 172.21.15.5:0/2415432325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ffda4f1b0 0x555ffde02530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:24.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.324+0000 7f322d771640 1 -- 172.21.15.5:0/2415432325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ffda2e4d0 msgr2=0x555ffddfd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:24.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.324+0000 7f322d771640 1 --2- 172.21.15.5:0/2415432325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ffda2e4d0 0x555ffddfd5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:24.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.324+0000 7f322d771640 1 -- 172.21.15.5:0/2415432325 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ffdd051f0 con 0x555ffda262f0 2024-05-09T00:48:24.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.324+0000 7f322cf70640 1 --2- 172.21.15.5:0/2415432325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ffda2e4d0 0x555ffddfd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:48:24.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.325+0000 7f322d771640 1 --2- 172.21.15.5:0/2415432325 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ffda262f0 0x555ffddfd0a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f32240c5430 tx=0x7f32240c92b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1cc3a650eea9e592 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:24.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.325+0000 7f3210ff9640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-09T00:48:24.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.325+0000 7f3210ff9640 1 -- 172.21.15.5:0/2415432325 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32240e3060 con 0x555ffda262f0 2024-05-09T00:48:24.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.325+0000 7f3236963740 1 -- 172.21.15.5:0/2415432325 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3218018070 con 0x555ffda262f0 2024-05-09T00:48:24.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.325+0000 7f3210ff9640 1 -- 172.21.15.5:0/2415432325 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f32240d1f90 con 0x555ffda262f0 2024-05-09T00:48:24.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.325+0000 7f3210ff9640 1 -- 172.21.15.5:0/2415432325 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f32240ea050 con 0x555ffda262f0 2024-05-09T00:48:24.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.325+0000 7f3236963740 10 client.25235 fetch_fsmap learned FSMap version 29 2024-05-09T00:48:24.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.325+0000 7f3236963740 1 -- 172.21.15.5:0/2415432325 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555ffdaba790 con 0x555ffda262f0 2024-05-09T00:48:24.330 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.325+0000 7f3210ff9640 1 -- 172.21.15.5:0/2415432325 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3218018070 con 0x555ffda262f0 2024-05-09T00:48:24.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.327+0000 7f3210ff9640 1 -- 172.21.15.5:0/2415432325 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f32240ea1f0 con 0x555ffda262f0 2024-05-09T00:48:24.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.327+0000 7f3236963740 10 client.25235 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:48:24.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.327+0000 7f3236963740 10 client.25235 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:48:24.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.327+0000 7f3236963740 20 client.25235 populate_metadata read hostname 'smithi005' 2024-05-09T00:48:24.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.327+0000 7f3236963740 1 --2- 172.21.15.5:0/2415432325 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x555ffda0dd80 0x555ffde1bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:24.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.327+0000 7f3236963740 4 client.25235 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:48:24.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.327+0000 7f3236963740 1 -- 172.21.15.5:0/2415432325 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ffdd653d0 con 0x555ffda0dd80 2024-05-09T00:48:24.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.327+0000 7f322cf70640 1 --2- 172.21.15.5:0/2415432325 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x555ffda0dd80 0x555ffde1bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:24.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.328+0000 7f3210ff9640 10 client.25235 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:48:24.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.328+0000 7f322cf70640 1 --2- 172.21.15.5:0/2415432325 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x555ffda0dd80 0x555ffde1bae0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55b88e9db8557233 server_cookie=f15263ad6ad66253 in_seq=0 out_seq=0 2024-05-09T00:48:24.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.329+0000 7f3210ff9640 1 -- 172.21.15.5:0/2415432325 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555ffdd653d0 con 0x555ffda0dd80 2024-05-09T00:48:24.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.329+0000 7f3210ff9640 10 client.25235 handle_command_reply: tid=0 2024-05-09T00:48:24.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.329+0000 7f3236963740 1 client.25235 shutdown 2024-05-09T00:48:24.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.329+0000 7f3236963740 1 -- 172.21.15.5:0/2415432325 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ffda262f0 msgr2=0x555ffddfd0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:24.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.329+0000 7f3236963740 1 --2- 172.21.15.5:0/2415432325 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ffda262f0 0x555ffddfd0a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f32240c5430 tx=0x7f32240c92b0 comp rx=0 tx=0).stop 2024-05-09T00:48:24.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.329+0000 7f3236963740 1 -- 172.21.15.5:0/2415432325 shutdown_connections 2024-05-09T00:48:24.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.329+0000 7f3236963740 1 --2- 172.21.15.5:0/2415432325 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x555ffda0dd80 0x555ffde1bae0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:24.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.329+0000 7f3210ff9640 0 client.25235 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:48:24.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.329+0000 7f3236963740 1 --2- 172.21.15.5:0/2415432325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ffda4f1b0 0x555ffde02530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:24.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.329+0000 7f3236963740 1 --2- 172.21.15.5:0/2415432325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ffda2e4d0 0x555ffddfd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:24.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.329+0000 7f3236963740 1 --2- 172.21.15.5:0/2415432325 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ffda262f0 0x555ffddfd0a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:24.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.329+0000 7f3236963740 1 -- 172.21.15.5:0/2415432325 >> 172.21.15.5:0/2415432325 conn(0x555ffda43300 msgr2=0x555ffddf3480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:24.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.329+0000 7f3236963740 1 -- 172.21.15.5:0/2415432325 shutdown_connections 2024-05-09T00:48:24.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.329+0000 7f3236963740 1 -- 172.21.15.5:0/2415432325 wait complete. 2024-05-09T00:48:24.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.329+0000 7f3236963740 20 client.25235 trim_cache size 0 max 16384 2024-05-09T00:48:24.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.345+0000 7f3236963740 1 Processor -- start 2024-05-09T00:48:24.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.345+0000 7f3236963740 1 -- start start 2024-05-09T00:48:24.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.345+0000 7f3236963740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ffda262f0 0x555ffddfd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:24.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.345+0000 7f3236963740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ffda2e4d0 0x555ffddf6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:24.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.345+0000 7f3236963740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ffda4f1b0 0x555ffdded700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:24.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.345+0000 7f3236963740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f32281a9300 con 0x555ffda262f0 2024-05-09T00:48:24.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.345+0000 7f3236963740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f32281aae40 con 0x555ffda2e4d0 2024-05-09T00:48:24.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.345+0000 7f322d771640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ffda262f0 0x555ffddfd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:24.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.345+0000 7f322d771640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ffda262f0 0x555ffddfd6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60010/0 (socket says 172.21.15.5:60010) 2024-05-09T00:48:24.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.345+0000 7f322d771640 1 -- 172.21.15.5:0/1847404307 learned_addr learned my addr 172.21.15.5:0/1847404307 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:24.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.345+0000 7f3236963740 1 -- 172.21.15.5:0/1847404307 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f32281a99d0 con 0x555ffda4f1b0 2024-05-09T00:48:24.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.345+0000 7f322cf70640 1 --2- 172.21.15.5:0/1847404307 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ffda2e4d0 0x555ffddf6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:24.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.346+0000 7f322df72640 1 --2- 172.21.15.5:0/1847404307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ffda4f1b0 0x555ffdded700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:24.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.346+0000 7f322d771640 1 -- 172.21.15.5:0/1847404307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ffda4f1b0 msgr2=0x555ffdded700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:24.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.346+0000 7f322d771640 1 --2- 172.21.15.5:0/1847404307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ffda4f1b0 0x555ffdded700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:24.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.346+0000 7f322d771640 1 -- 172.21.15.5:0/1847404307 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ffda2e4d0 msgr2=0x555ffddf6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:24.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.346+0000 7f322d771640 1 --2- 172.21.15.5:0/1847404307 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ffda2e4d0 0x555ffddf6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:24.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.346+0000 7f322d771640 1 -- 172.21.15.5:0/1847404307 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ffdb315f0 con 0x555ffda262f0 2024-05-09T00:48:24.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.346+0000 7f322df72640 1 --2- 172.21.15.5:0/1847404307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ffda4f1b0 0x555ffdded700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:48:24.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.346+0000 7f322d771640 1 --2- 172.21.15.5:0/1847404307 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ffda262f0 0x555ffddfd6a0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f3224000c00 tx=0x7f32240c9430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ab0ad62bfcc350f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:24.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.346+0000 7f3212ffd640 1 -- 172.21.15.5:0/1847404307 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32240e3040 con 0x555ffda262f0 2024-05-09T00:48:24.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.346+0000 7f3212ffd640 1 -- 172.21.15.5:0/1847404307 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f322404a070 con 0x555ffda262f0 2024-05-09T00:48:24.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.346+0000 7f3212ffd640 1 -- 172.21.15.5:0/1847404307 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32240c4f80 con 0x555ffda262f0 2024-05-09T00:48:24.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.348+0000 7f3236963740 1 -- 172.21.15.5:0/1847404307 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ffda262f0 msgr2=0x555ffddfd6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:24.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.348+0000 7f3236963740 1 --2- 172.21.15.5:0/1847404307 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ffda262f0 0x555ffddfd6a0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f3224000c00 tx=0x7f32240c9430 comp rx=0 tx=0).stop 2024-05-09T00:48:24.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.348+0000 7f3236963740 1 -- 172.21.15.5:0/1847404307 shutdown_connections 2024-05-09T00:48:24.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.348+0000 7f3236963740 1 --2- 172.21.15.5:0/1847404307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ffda4f1b0 0x555ffdded700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:24.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.348+0000 7f3236963740 1 --2- 172.21.15.5:0/1847404307 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ffda2e4d0 0x555ffddf6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:24.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.348+0000 7f3236963740 1 --2- 172.21.15.5:0/1847404307 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ffda262f0 0x555ffddfd6a0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:24.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.348+0000 7f3236963740 1 -- 172.21.15.5:0/1847404307 >> 172.21.15.5:0/1847404307 conn(0x555ffda43300 msgr2=0x555ffddfb6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:24.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.348+0000 7f3236963740 1 -- 172.21.15.5:0/1847404307 shutdown_connections 2024-05-09T00:48:24.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.348+0000 7f3236963740 1 -- 172.21.15.5:0/1847404307 wait complete. 2024-05-09T00:48:24.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.349+0000 7f3236963740 1 -- start start 2024-05-09T00:48:24.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.349+0000 7f3236963740 1 Processor -- start 2024-05-09T00:48:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.350+0000 7f3236963740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ffda262f0 0x555ffde2b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.350+0000 7f3236963740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ffda2e4d0 0x555ffde2b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.350+0000 7f3236963740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ffda4f1b0 0x555ffde2c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.350+0000 7f3236963740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f32281aa770 con 0x555ffda4f1b0 2024-05-09T00:48:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.350+0000 7f322d771640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ffda262f0 0x555ffde2b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.350+0000 7f322df72640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ffda4f1b0 0x555ffde2c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.350+0000 7f3236963740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f32281aa0a0 con 0x555ffda262f0 2024-05-09T00:48:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.350+0000 7f322df72640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ffda4f1b0 0x555ffde2c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60020/0 (socket says 172.21.15.5:60020) 2024-05-09T00:48:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.350+0000 7f322df72640 1 -- 172.21.15.5:0/2370857247 learned_addr learned my addr 172.21.15.5:0/2370857247 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.350+0000 7f322cf70640 1 --2- 172.21.15.5:0/2370857247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ffda2e4d0 0x555ffde2b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.351+0000 7f3236963740 1 -- 172.21.15.5:0/2370857247 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f32281a72e0 con 0x555ffda2e4d0 2024-05-09T00:48:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.351+0000 7f3236963740 10 client.0 _ll_register_callbacks cb 0x7f3224005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:48:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.351+0000 7f322d771640 1 -- 172.21.15.5:0/2370857247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ffda2e4d0 msgr2=0x555ffde2b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.351+0000 7f322d771640 1 --2- 172.21.15.5:0/2370857247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ffda2e4d0 0x555ffde2b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.351+0000 7f322d771640 1 -- 172.21.15.5:0/2370857247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ffda4f1b0 msgr2=0x555ffde2c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.351+0000 7f322d771640 1 --2- 172.21.15.5:0/2370857247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ffda4f1b0 0x555ffde2c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.351+0000 7f322d771640 1 -- 172.21.15.5:0/2370857247 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ffdd1bed0 con 0x555ffda262f0 2024-05-09T00:48:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.351+0000 7f322d771640 1 --2- 172.21.15.5:0/2370857247 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ffda262f0 0x555ffde2b0b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f32240ea5b0 tx=0x7f3224004140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c95c1fe358f5bfa6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.351+0000 7f3210ff9640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-09T00:48:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.351+0000 7f3210ff9640 1 -- 172.21.15.5:0/2370857247 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32240e3070 con 0x555ffda262f0 2024-05-09T00:48:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.352+0000 7f3210ff9640 1 -- 172.21.15.5:0/2370857247 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f32240c4f30 con 0x555ffda262f0 2024-05-09T00:48:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.352+0000 7f3236963740 1 -- 172.21.15.5:0/2370857247 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f322404a070 con 0x555ffda262f0 2024-05-09T00:48:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.352+0000 7f3210ff9640 1 -- 172.21.15.5:0/2370857247 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f32240d2210 con 0x555ffda262f0 2024-05-09T00:48:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.352+0000 7f3236963740 10 client.25241 fetch_fsmap learned FSMap version 29 2024-05-09T00:48:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.352+0000 7f3236963740 1 -- 172.21.15.5:0/2370857247 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555ffdd1bed0 con 0x555ffda262f0 2024-05-09T00:48:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.352+0000 7f3210ff9640 1 -- 172.21.15.5:0/2370857247 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32240c8ec0 con 0x555ffda262f0 2024-05-09T00:48:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.352+0000 7f3210ff9640 1 -- 172.21.15.5:0/2370857247 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f32240c3b60 con 0x555ffda262f0 2024-05-09T00:48:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.352+0000 7f3236963740 10 client.25241 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:48:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.352+0000 7f3236963740 10 client.25241 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:48:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.352+0000 7f3236963740 20 client.25241 populate_metadata read hostname 'smithi005' 2024-05-09T00:48:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.352+0000 7f3236963740 1 --2- 172.21.15.5:0/2370857247 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x555ffd9e31c0 0x555ffde863a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.352+0000 7f3236963740 4 client.25241 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:48:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.352+0000 7f3236963740 1 -- 172.21.15.5:0/2370857247 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f322404a070 con 0x555ffd9e31c0 2024-05-09T00:48:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.353+0000 7f322cf70640 1 --2- 172.21.15.5:0/2370857247 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x555ffd9e31c0 0x555ffde863a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.353+0000 7f3210ff9640 10 client.25241 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:48:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.353+0000 7f322cf70640 1 --2- 172.21.15.5:0/2370857247 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x555ffd9e31c0 0x555ffde863a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d978fb63c5b6544 server_cookie=51fdb2cc85d6745 in_seq=0 out_seq=0 2024-05-09T00:48:24.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.357+0000 7f3210ff9640 1 -- 172.21.15.5:0/2370857247 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7f322404a070 con 0x555ffd9e31c0 2024-05-09T00:48:24.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.357+0000 7f3210ff9640 10 client.25241 handle_command_reply: tid=0 2024-05-09T00:48:24.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.357+0000 7f3236963740 1 client.25241 shutdown 2024-05-09T00:48:24.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.357+0000 7f3236963740 1 -- 172.21.15.5:0/2370857247 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ffda262f0 msgr2=0x555ffde2b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:24.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.357+0000 7f3236963740 1 --2- 172.21.15.5:0/2370857247 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ffda262f0 0x555ffde2b0b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f32240ea5b0 tx=0x7f3224004140 comp rx=0 tx=0).stop 2024-05-09T00:48:24.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.357+0000 7f3236963740 1 -- 172.21.15.5:0/2370857247 shutdown_connections 2024-05-09T00:48:24.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.357+0000 7f3236963740 1 --2- 172.21.15.5:0/2370857247 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x555ffd9e31c0 0x555ffde863a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:24.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.358+0000 7f3210ff9640 0 client.25241 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:48:24.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.358+0000 7f3236963740 1 --2- 172.21.15.5:0/2370857247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ffda4f1b0 0x555ffde2c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:24.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.358+0000 7f3236963740 1 --2- 172.21.15.5:0/2370857247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ffda2e4d0 0x555ffde2b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:24.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.358+0000 7f3236963740 1 --2- 172.21.15.5:0/2370857247 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ffda262f0 0x555ffde2b0b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:24.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.358+0000 7f3236963740 1 -- 172.21.15.5:0/2370857247 >> 172.21.15.5:0/2370857247 conn(0x555ffda43300 msgr2=0x555ffddf1c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:24.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.358+0000 7f3236963740 1 -- 172.21.15.5:0/2370857247 shutdown_connections 2024-05-09T00:48:24.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.358+0000 7f3236963740 1 -- 172.21.15.5:0/2370857247 wait complete. 2024-05-09T00:48:24.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.358+0000 7f3236963740 20 client.25241 trim_cache size 0 max 16384 2024-05-09T00:48:24.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-09T00:48:24.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-09T00:48:24.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-09T00:48:24.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.359+0000 7f322f9d5640 1 -- 172.21.15.5:0/1578855844 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f32140b28e0 msgr2=0x7f32140b4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:24.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.359+0000 7f322f9d5640 1 --2- 172.21.15.5:0/1578855844 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f32140b28e0 0x7f32140b4df0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x555ffddf4650 tx=0x7f3220010040 comp rx=0 tx=0).stop 2024-05-09T00:48:24.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.359+0000 7f322f9d5640 1 -- 172.21.15.5:0/1578855844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32281030c0 msgr2=0x7f322819a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:24.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.359+0000 7f322f9d5640 1 --2- 172.21.15.5:0/1578855844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32281030c0 0x7f322819a400 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f322402a860 tx=0x7f3224032d30 comp rx=0 tx=0).stop 2024-05-09T00:48:24.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.359+0000 7f322f9d5640 1 -- 172.21.15.5:0/1578855844 shutdown_connections 2024-05-09T00:48:24.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.359+0000 7f322f9d5640 1 --2- 172.21.15.5:0/1578855844 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f32140b28e0 0x7f32140b4df0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:24.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.359+0000 7f322f9d5640 1 --2- 172.21.15.5:0/1578855844 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32281045e0 0x7f32281a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:24.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.359+0000 7f322f9d5640 1 --2- 172.21.15.5:0/1578855844 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3228103aa0 0x7f322819a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:24.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.359+0000 7f322f9d5640 1 --2- 172.21.15.5:0/1578855844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32281030c0 0x7f322819a400 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:24.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.359+0000 7f322f9d5640 1 -- 172.21.15.5:0/1578855844 >> 172.21.15.5:0/1578855844 conn(0x7f32280fd4c0 msgr2=0x7f3228104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:24.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.360+0000 7f322f9d5640 1 -- 172.21.15.5:0/1578855844 shutdown_connections 2024-05-09T00:48:24.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:24.360+0000 7f322f9d5640 1 -- 172.21.15.5:0/1578855844 wait complete. 2024-05-09T00:48:24.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:24.659+0000 7f89b27c3700 1 -- 172.21.15.5:0/2897820871 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f89ac112260 con 0x7f89ac107e10 2024-05-09T00:48:24.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:24.660+0000 7f89a9ffb700 1 -- 172.21.15.5:0/2897820871 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11926 (secure 0 0 0) 0x7f89940a9330 con 0x7f89ac107e10 2024-05-09T00:48:24.662 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:48:24.662 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2685771887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2685771887},{"type":"v1","addr":"172.21.15.5:6835","nonce":2685771887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/254017535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":254017535},{"type":"v1","addr":"172.21.15.5:6837","nonce":254017535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T00:43:45.856066+0000","modified":"2024-05-09T00:47:06.980541+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24508,"mds_1":24499,"mds_2":24494,"mds_3":24493,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14598":{"gid":14598,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2109255228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2109255228},{"type":"v1","addr":"172.21.15.5:6839","nonce":2109255228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/3809966329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3809966329},{"type":"v1","addr":"172.21.15.5:6841","nonce":3809966329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6833/3272897742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3272897742},{"type":"v1","addr":"172.21.15.131:6833","nonce":3272897742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1283093898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1283093898},{"type":"v1","addr":"172.21.15.131:6835","nonce":1283093898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/2809525455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2809525455},{"type":"v1","addr":"172.21.15.157:6833","nonce":2809525455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.131:6837/1633006186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":1633006186},{"type":"v1","addr":"172.21.15.131:6837","nonce":1633006186}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.157:6835/832898273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":832898273},{"type":"v1","addr":"172.21.15.157:6835","nonce":832898273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.131:6839/2748669631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2748669631},{"type":"v1","addr":"172.21.15.131:6839","nonce":2748669631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.157:6837/744863009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":744863009},{"type":"v1","addr":"172.21.15.157:6837","nonce":744863009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.157:6839/2779860399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":2779860399},{"type":"v1","addr":"172.21.15.157:6839","nonce":2779860399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24494,24508,24499,24467,24493]},"id":1}]} 2024-05-09T00:48:24.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:24.663+0000 7f89b27c3700 1 -- 172.21.15.5:0/2897820871 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f89980b2900 msgr2=0x7f89980b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:24.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:24.663+0000 7f89b27c3700 1 --2- 172.21.15.5:0/2897820871 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f89980b2900 0x7f89980b4e10 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f899c00bd30 tx=0x7f899c00b480 comp rx=0 tx=0).stop 2024-05-09T00:48:24.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:24.663+0000 7f89b27c3700 1 -- 172.21.15.5:0/2897820871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89ac107e10 msgr2=0x7f89ac111270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:24.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:24.663+0000 7f89b27c3700 1 --2- 172.21.15.5:0/2897820871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89ac107e10 0x7f89ac111270 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f899400d720 tx=0x7f899400da30 comp rx=0 tx=0).stop 2024-05-09T00:48:24.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:24.663+0000 7f89b27c3700 1 -- 172.21.15.5:0/2897820871 shutdown_connections 2024-05-09T00:48:24.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:24.663+0000 7f89b27c3700 1 --2- 172.21.15.5:0/2897820871 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f89980b2900 0x7f89980b4e10 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:24.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:24.663+0000 7f89b27c3700 1 --2- 172.21.15.5:0/2897820871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89ac108760 0x7f89ac1117b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:24.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:24.663+0000 7f89b27c3700 1 --2- 172.21.15.5:0/2897820871 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89ac1034e0 0x7f89ac110d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:24.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:24.663+0000 7f89b27c3700 1 --2- 172.21.15.5:0/2897820871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89ac107e10 0x7f89ac111270 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:24.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:24.663+0000 7f89b27c3700 1 -- 172.21.15.5:0/2897820871 >> 172.21.15.5:0/2897820871 conn(0x7f89ac0fef30 msgr2=0x7f89ac107580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:24.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:24.663+0000 7f89b27c3700 1 -- 172.21.15.5:0/2897820871 shutdown_connections 2024-05-09T00:48:24.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:24.664+0000 7f89b27c3700 1 -- 172.21.15.5:0/2897820871 wait complete. 2024-05-09T00:48:24.666 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 29 2024-05-09T00:48:25.463 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:48:25 smithi005 ceph-mon[30365]: pgmap v433: 97 pgs: 97 active+clean; 669 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-05-09T00:48:25.464 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:48:25 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/2897820871' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:48:25.502 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-09T00:48:25.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:48:25 smithi131 ceph-mon[34547]: pgmap v433: 97 pgs: 97 active+clean; 669 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-05-09T00:48:25.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:48:25 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/2897820871' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:48:25.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:48:25 smithi157 ceph-mon[34294]: pgmap v433: 97 pgs: 97 active+clean; 669 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-05-09T00:48:25.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:48:25 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/2897820871' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:48:25.880 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:48:27.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:48:27 smithi005 ceph-mon[30365]: pgmap v434: 97 pgs: 97 active+clean; 669 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-05-09T00:48:27.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:48:27 smithi131 ceph-mon[34547]: pgmap v434: 97 pgs: 97 active+clean; 669 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-05-09T00:48:27.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:48:27 smithi157 ceph-mon[34294]: pgmap v434: 97 pgs: 97 active+clean; 669 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-05-09T00:48:27.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.899+0000 7f56c7a52700 1 -- 172.21.15.5:0/2439863963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56c0108740 msgr2=0x7f56c01100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:27.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.899+0000 7f56c7a52700 1 --2- 172.21.15.5:0/2439863963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56c0108740 0x7f56c01100a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f56b800b0d0 tx=0x7f56b800b3e0 comp rx=0 tx=0).stop 2024-05-09T00:48:27.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.900+0000 7f56c7a52700 1 -- 172.21.15.5:0/2439863963 shutdown_connections 2024-05-09T00:48:27.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.900+0000 7f56c7a52700 1 --2- 172.21.15.5:0/2439863963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56c0108740 0x7f56c01100a0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:27.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.900+0000 7f56c7a52700 1 --2- 172.21.15.5:0/2439863963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56c01034c0 0x7f56c01038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:27.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.900+0000 7f56c7a52700 1 --2- 172.21.15.5:0/2439863963 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f56c0107df0 0x7f56c0108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:27.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.900+0000 7f56c7a52700 1 -- 172.21.15.5:0/2439863963 >> 172.21.15.5:0/2439863963 conn(0x7f56c00fedc0 msgr2=0x7f56c01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:27.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.900+0000 7f56c7a52700 1 -- 172.21.15.5:0/2439863963 shutdown_connections 2024-05-09T00:48:27.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.900+0000 7f56c7a52700 1 -- 172.21.15.5:0/2439863963 wait complete. 2024-05-09T00:48:27.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.901+0000 7f56c7a52700 1 Processor -- start 2024-05-09T00:48:27.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.902+0000 7f56c7a52700 1 -- start start 2024-05-09T00:48:27.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.902+0000 7f56c7a52700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56c01034c0 0x7f56c019d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:27.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.902+0000 7f56c7a52700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f56c0107df0 0x7f56c019d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:27.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.902+0000 7f56c7a52700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56c0108740 0x7f56c01a1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:27.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.902+0000 7f56c7a52700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f56c0113370 con 0x7f56c0108740 2024-05-09T00:48:27.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.902+0000 7f56c7a52700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f56c01131f0 con 0x7f56c0107df0 2024-05-09T00:48:27.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.902+0000 7f56c7a52700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f56c01134f0 con 0x7f56c01034c0 2024-05-09T00:48:27.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.902+0000 7f56c4fed700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f56c0107df0 0x7f56c019d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:27.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.902+0000 7f56c4fed700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f56c0107df0 0x7f56c019d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:53032/0 (socket says 172.21.15.5:53032) 2024-05-09T00:48:27.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.902+0000 7f56c4fed700 1 -- 172.21.15.5:0/2927997285 learned_addr learned my addr 172.21.15.5:0/2927997285 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:27.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.902+0000 7f56c57ee700 1 --2- 172.21.15.5:0/2927997285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56c01034c0 0x7f56c019d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:27.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.903+0000 7f56c5fef700 1 --2- 172.21.15.5:0/2927997285 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56c0108740 0x7f56c01a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:27.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.903+0000 7f56c5fef700 1 -- 172.21.15.5:0/2927997285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56c01034c0 msgr2=0x7f56c019d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:27.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.903+0000 7f56c5fef700 1 --2- 172.21.15.5:0/2927997285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56c01034c0 0x7f56c019d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:27.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.903+0000 7f56c5fef700 1 -- 172.21.15.5:0/2927997285 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f56c0107df0 msgr2=0x7f56c019d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:27.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.903+0000 7f56c5fef700 1 --2- 172.21.15.5:0/2927997285 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f56c0107df0 0x7f56c019d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:27.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.903+0000 7f56c5fef700 1 -- 172.21.15.5:0/2927997285 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56c01a1ee0 con 0x7f56c0108740 2024-05-09T00:48:27.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.903+0000 7f56c57ee700 1 --2- 172.21.15.5:0/2927997285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56c01034c0 0x7f56c019d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:48:27.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.903+0000 7f56c5fef700 1 --2- 172.21.15.5:0/2927997285 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56c0108740 0x7f56c01a1840 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f56b800b0d0 tx=0x7f56b8007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:27.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.904+0000 7f56b67fc700 1 -- 172.21.15.5:0/2927997285 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b801f070 con 0x7f56c0108740 2024-05-09T00:48:27.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.904+0000 7f56b67fc700 1 -- 172.21.15.5:0/2927997285 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56b8004c50 con 0x7f56c0108740 2024-05-09T00:48:27.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.904+0000 7f56b67fc700 1 -- 172.21.15.5:0/2927997285 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b801b690 con 0x7f56c0108740 2024-05-09T00:48:27.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.904+0000 7f56c7a52700 1 -- 172.21.15.5:0/2927997285 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56c01a2170 con 0x7f56c0108740 2024-05-09T00:48:27.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.904+0000 7f56c7a52700 1 -- 172.21.15.5:0/2927997285 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56c01a2600 con 0x7f56c0108740 2024-05-09T00:48:27.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.909+0000 7f56b67fc700 1 -- 172.21.15.5:0/2927997285 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f56b8022070 con 0x7f56c0108740 2024-05-09T00:48:27.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.909+0000 7f56d6468b80 1 Processor -- start 2024-05-09T00:48:27.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.910+0000 7f56d6468b80 1 -- start start 2024-05-09T00:48:27.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.910+0000 7f56d6468b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc4d78b160 0x55cc4d8244f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:27.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.911+0000 7f56d6468b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc4d6a14a0 0x55cc4d824a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:27.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.911+0000 7f56c4fed700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc4d78b160 0x55cc4d8244f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:27.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.911+0000 7f56c4fed700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc4d78b160 0x55cc4d8244f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60044/0 (socket says 172.21.15.5:60044) 2024-05-09T00:48:27.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.912+0000 7f56d6468b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cc4d81e6a0 0x55cc4d81ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:27.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.912+0000 7f56b67fc700 1 --2- 172.21.15.5:0/2927997285 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f56ac0b2900 0x7f56ac0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:27.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.912+0000 7f56d6468b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55cc4d740ff0 con 0x55cc4d78b160 2024-05-09T00:48:27.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.912+0000 7f56d6468b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55cc4d6df7f0 con 0x55cc4d81e6a0 2024-05-09T00:48:27.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.912+0000 7f56d6468b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55cc4d73cfb0 con 0x55cc4d6a14a0 2024-05-09T00:48:27.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.912+0000 7f56c4fed700 1 -- 172.21.15.5:0/1480108145 learned_addr learned my addr 172.21.15.5:0/1480108145 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:27.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.912+0000 7f56c4fed700 1 --2- 172.21.15.5:0/1480108145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cc4d81e6a0 0x55cc4d81ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:27.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.913+0000 7f56c5fef700 1 --2- 172.21.15.5:0/2927997285 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f56ac0b2900 0x7f56ac0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:27.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.913+0000 7f56c57ee700 1 --2- 172.21.15.5:0/1480108145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc4d6a14a0 0x55cc4d824a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:27.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.913+0000 7f56c4fed700 1 -- 172.21.15.5:0/1480108145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc4d6a14a0 msgr2=0x55cc4d824a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:27.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.913+0000 7f56c4fed700 1 --2- 172.21.15.5:0/1480108145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc4d6a14a0 0x55cc4d824a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:27.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.913+0000 7f56b67fc700 1 -- 172.21.15.5:0/2927997285 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f56b800f020 con 0x7f56c0108740 2024-05-09T00:48:27.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.913+0000 7f56c4fed700 1 -- 172.21.15.5:0/1480108145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cc4d81e6a0 msgr2=0x55cc4d81ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:27.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.913+0000 7f56c4fed700 1 --2- 172.21.15.5:0/1480108145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cc4d81e6a0 0x55cc4d81ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:27.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.913+0000 7f56c4fed700 1 -- 172.21.15.5:0/1480108145 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc4d6d7370 con 0x55cc4d78b160 2024-05-09T00:48:27.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.913+0000 7f56c4fed700 1 --2- 172.21.15.5:0/1480108145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc4d78b160 0x55cc4d8244f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f56b0009880 tx=0x7f56b00099f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e07b15e1dfd0146 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:27.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.914+0000 7f56ab7fe700 1 -- 172.21.15.5:0/1480108145 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b001a070 con 0x55cc4d78b160 2024-05-09T00:48:27.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.914+0000 7f56ab7fe700 1 -- 172.21.15.5:0/1480108145 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56b0003680 con 0x55cc4d78b160 2024-05-09T00:48:27.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.914+0000 7f56ab7fe700 1 -- 172.21.15.5:0/1480108145 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b001a070 con 0x55cc4d78b160 2024-05-09T00:48:27.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.915+0000 7f56d6468b80 1 -- 172.21.15.5:0/1480108145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc4d78b160 msgr2=0x55cc4d8244f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:27.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.915+0000 7f56d6468b80 1 --2- 172.21.15.5:0/1480108145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc4d78b160 0x55cc4d8244f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f56b0009880 tx=0x7f56b00099f0 comp rx=0 tx=0).stop 2024-05-09T00:48:27.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.915+0000 7f56c5fef700 1 --2- 172.21.15.5:0/2927997285 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f56ac0b2900 0x7f56ac0b4e10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f56b80e0cb0 tx=0x7f56b80d9de0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:27.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.915+0000 7f56d6468b80 1 -- 172.21.15.5:0/1480108145 shutdown_connections 2024-05-09T00:48:27.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.915+0000 7f56d6468b80 1 --2- 172.21.15.5:0/1480108145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cc4d81e6a0 0x55cc4d81ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:27.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.915+0000 7f56d6468b80 1 --2- 172.21.15.5:0/1480108145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc4d78b160 0x55cc4d8244f0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:27.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.915+0000 7f56d6468b80 1 --2- 172.21.15.5:0/1480108145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc4d6a14a0 0x55cc4d824a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:27.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.915+0000 7f56d6468b80 1 -- 172.21.15.5:0/1480108145 >> 172.21.15.5:0/1480108145 conn(0x55cc4d7016a0 msgr2=0x55cc4d819400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:27.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.915+0000 7f56d6468b80 1 -- 172.21.15.5:0/1480108145 shutdown_connections 2024-05-09T00:48:27.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.915+0000 7f56d6468b80 1 -- 172.21.15.5:0/1480108145 wait complete. 2024-05-09T00:48:27.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.916+0000 7f56d6468b80 1 -- start start 2024-05-09T00:48:27.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.916+0000 7f56d6468b80 1 Processor -- start 2024-05-09T00:48:27.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.917+0000 7f56d6468b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cc4d6a14a0 0x55cc4d828180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:27.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.917+0000 7f56d6468b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc4d78b160 0x55cc4d8286c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:27.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.918+0000 7f56c57ee700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc4d78b160 0x55cc4d8286c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:27.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.918+0000 7f56c57ee700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc4d78b160 0x55cc4d8286c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60046/0 (socket says 172.21.15.5:60046) 2024-05-09T00:48:27.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.918+0000 7f56c4fed700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cc4d6a14a0 0x55cc4d828180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:27.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.918+0000 7f56c4fed700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cc4d6a14a0 0x55cc4d828180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:53054/0 (socket says 172.21.15.5:53054) 2024-05-09T00:48:27.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.918+0000 7f56c57ee700 1 -- 172.21.15.5:0/2738057367 learned_addr learned my addr 172.21.15.5:0/2738057367 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:27.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.918+0000 7f56d6468b80 1 --2- 172.21.15.5:0/2738057367 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc4d81e6a0 0x55cc4d828c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:27.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.918+0000 7f56d6468b80 1 -- 172.21.15.5:0/2738057367 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55cc4d5bcba0 con 0x55cc4d78b160 2024-05-09T00:48:27.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.918+0000 7f56d6468b80 1 -- 172.21.15.5:0/2738057367 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55cc4d6fb150 con 0x55cc4d6a14a0 2024-05-09T00:48:27.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.918+0000 7f56d6468b80 1 -- 172.21.15.5:0/2738057367 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55cc4d82b1d0 con 0x55cc4d81e6a0 2024-05-09T00:48:27.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.918+0000 7f56d6468b80 10 client.0 _ll_register_callbacks cb 0x55cc4d5b26c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:48:27.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.919+0000 7f56c4fed700 1 --2- 172.21.15.5:0/2738057367 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc4d81e6a0 0x55cc4d828c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:27.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.919+0000 7f56c57ee700 1 -- 172.21.15.5:0/2738057367 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc4d81e6a0 msgr2=0x55cc4d828c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:27.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.919+0000 7f56c57ee700 1 --2- 172.21.15.5:0/2738057367 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc4d81e6a0 0x55cc4d828c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:27.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.919+0000 7f56c57ee700 1 -- 172.21.15.5:0/2738057367 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cc4d6a14a0 msgr2=0x55cc4d828180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:27.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.919+0000 7f56c57ee700 1 --2- 172.21.15.5:0/2738057367 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cc4d6a14a0 0x55cc4d828180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:27.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.919+0000 7f56c57ee700 1 -- 172.21.15.5:0/2738057367 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc4d5bc490 con 0x55cc4d78b160 2024-05-09T00:48:27.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.919+0000 7f56c4fed700 1 --2- 172.21.15.5:0/2738057367 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cc4d6a14a0 0x55cc4d828180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:48:27.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.919+0000 7f56c57ee700 1 --2- 172.21.15.5:0/2738057367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc4d78b160 0x55cc4d8286c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f56bc012780 tx=0x7f56bc012a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb9e41f797847bdd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:27.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.920+0000 7f56a8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:48:27.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.920+0000 7f56a8ff9700 1 -- 172.21.15.5:0/2738057367 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56bc01f060 con 0x55cc4d78b160 2024-05-09T00:48:27.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.920+0000 7f56a8ff9700 1 -- 172.21.15.5:0/2738057367 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f56bc004030 con 0x55cc4d78b160 2024-05-09T00:48:27.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.920+0000 7f56d6468b80 1 -- 172.21.15.5:0/2738057367 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56b0003680 con 0x55cc4d78b160 2024-05-09T00:48:27.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.920+0000 7f56a8ff9700 1 -- 172.21.15.5:0/2738057367 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f56bc004270 con 0x55cc4d78b160 2024-05-09T00:48:27.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.920+0000 7f56d6468b80 10 client.15558 fetch_fsmap learned FSMap version 29 2024-05-09T00:48:27.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.920+0000 7f56d6468b80 1 -- 172.21.15.5:0/2738057367 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cc4d835c30 con 0x55cc4d78b160 2024-05-09T00:48:27.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.921+0000 7f56a8ff9700 1 -- 172.21.15.5:0/2738057367 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56bc004410 con 0x55cc4d78b160 2024-05-09T00:48:27.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.922+0000 7f56a8ff9700 1 -- 172.21.15.5:0/2738057367 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f56bc010980 con 0x55cc4d78b160 2024-05-09T00:48:27.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.922+0000 7f56d6468b80 10 client.15558 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:48:27.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.922+0000 7f56d6468b80 10 client.15558 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T00:48:27.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.922+0000 7f56d6468b80 20 client.15558 populate_metadata read hostname 'smithi005' 2024-05-09T00:48:27.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.922+0000 7f56d6468b80 1 --2- 172.21.15.5:0/2738057367 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x55cc4d838360 0x55cc4d83a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:27.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.922+0000 7f56d6468b80 4 client.15558 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:48:27.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.922+0000 7f56d6468b80 1 -- 172.21.15.5:0/2738057367 --> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f56b0003680 con 0x55cc4d838360 2024-05-09T00:48:27.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.922+0000 7f56c4fed700 1 --2- 172.21.15.5:0/2738057367 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x55cc4d838360 0x55cc4d83a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:27.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.922+0000 7f56a8ff9700 10 client.15558 ms_handle_connect on v2:172.21.15.131:6836/1633006186 2024-05-09T00:48:27.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.923+0000 7f56c4fed700 1 --2- 172.21.15.5:0/2738057367 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x55cc4d838360 0x55cc4d83a800 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5e8ff76b013aad86 server_cookie=ad80a7defd1a629f in_seq=0 out_seq=0 2024-05-09T00:48:27.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.923+0000 7f56a8ff9700 1 -- 172.21.15.5:0/2738057367 <== mds.4 v2:172.21.15.131:6836/1633006186 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f56b0003680 con 0x55cc4d838360 2024-05-09T00:48:27.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.924+0000 7f56a8ff9700 10 client.15558 handle_command_reply: tid=0 2024-05-09T00:48:27.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.924+0000 7f56d6468b80 1 client.15558 shutdown 2024-05-09T00:48:27.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.924+0000 7f56d6468b80 1 -- 172.21.15.5:0/2738057367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc4d78b160 msgr2=0x55cc4d8286c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:27.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.924+0000 7f56d6468b80 1 --2- 172.21.15.5:0/2738057367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc4d78b160 0x55cc4d8286c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f56bc012780 tx=0x7f56bc012a90 comp rx=0 tx=0).stop 2024-05-09T00:48:27.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.924+0000 7f56d6468b80 1 -- 172.21.15.5:0/2738057367 shutdown_connections 2024-05-09T00:48:27.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.924+0000 7f56d6468b80 1 --2- 172.21.15.5:0/2738057367 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x55cc4d838360 0x55cc4d83a800 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:27.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.924+0000 7f56d6468b80 1 --2- 172.21.15.5:0/2738057367 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc4d81e6a0 0x55cc4d828c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:27.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.924+0000 7f56d6468b80 1 --2- 172.21.15.5:0/2738057367 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cc4d6a14a0 0x55cc4d828180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:27.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.924+0000 7f56d6468b80 1 --2- 172.21.15.5:0/2738057367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc4d78b160 0x55cc4d8286c0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:27.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.924+0000 7f56d6468b80 1 -- 172.21.15.5:0/2738057367 >> 172.21.15.5:0/2738057367 conn(0x55cc4d7016a0 msgr2=0x55cc4d8191c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:27.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.924+0000 7f56a8ff9700 0 client.15558 ms_handle_reset on v2:172.21.15.131:6836/1633006186 2024-05-09T00:48:27.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.924+0000 7f56d6468b80 1 -- 172.21.15.5:0/2738057367 shutdown_connections 2024-05-09T00:48:27.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.924+0000 7f56d6468b80 1 -- 172.21.15.5:0/2738057367 wait complete. 2024-05-09T00:48:27.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.924+0000 7f56d6468b80 20 client.15558 trim_cache size 0 max 16384 2024-05-09T00:48:27.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.954+0000 7f56d6468b80 1 Processor -- start 2024-05-09T00:48:27.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.954+0000 7f56d6468b80 1 -- start start 2024-05-09T00:48:27.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.955+0000 7f56d6468b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cc4d6a14a0 0x55cc4d8340a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:27.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.955+0000 7f56d6468b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc4d81e6a0 0x55cc4d840000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:27.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.955+0000 7f56d6468b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc4d8345e0 0x55cc4d840540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:27.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.955+0000 7f56d6468b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f56c01b0e00 con 0x55cc4d8345e0 2024-05-09T00:48:27.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.955+0000 7f56d6468b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f56c01b2940 con 0x55cc4d6a14a0 2024-05-09T00:48:27.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.955+0000 7f56d6468b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f56c01b14d0 con 0x55cc4d81e6a0 2024-05-09T00:48:27.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.955+0000 7f56c4fed700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc4d8345e0 0x55cc4d840540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:27.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.955+0000 7f56c4fed700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc4d8345e0 0x55cc4d840540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60052/0 (socket says 172.21.15.5:60052) 2024-05-09T00:48:27.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.955+0000 7f56c4fed700 1 -- 172.21.15.5:0/3583851269 learned_addr learned my addr 172.21.15.5:0/3583851269 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:27.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.955+0000 7f56c4fed700 1 --2- 172.21.15.5:0/3583851269 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cc4d6a14a0 0x55cc4d8340a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:27.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.955+0000 7f56c57ee700 1 --2- 172.21.15.5:0/3583851269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc4d81e6a0 0x55cc4d840000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:27.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.956+0000 7f56c4fed700 1 -- 172.21.15.5:0/3583851269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc4d81e6a0 msgr2=0x55cc4d840000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:27.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.956+0000 7f56c4fed700 1 --2- 172.21.15.5:0/3583851269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc4d81e6a0 0x55cc4d840000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:27.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.956+0000 7f56c4fed700 1 -- 172.21.15.5:0/3583851269 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cc4d6a14a0 msgr2=0x55cc4d8340a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:27.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.956+0000 7f56c4fed700 1 --2- 172.21.15.5:0/3583851269 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cc4d6a14a0 0x55cc4d8340a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:27.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.956+0000 7f56c4fed700 1 -- 172.21.15.5:0/3583851269 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc4d81eab0 con 0x55cc4d8345e0 2024-05-09T00:48:27.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.956+0000 7f56c57ee700 1 --2- 172.21.15.5:0/3583851269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc4d81e6a0 0x55cc4d840000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:48:27.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.956+0000 7f56c4fed700 1 --2- 172.21.15.5:0/3583851269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc4d8345e0 0x55cc4d840540 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f56b0003f80 tx=0x7f56b0003fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68ce26fbc98baaf2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:27.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.956+0000 7f56aaffd700 1 -- 172.21.15.5:0/3583851269 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b0028040 con 0x55cc4d8345e0 2024-05-09T00:48:27.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.957+0000 7f56aaffd700 1 -- 172.21.15.5:0/3583851269 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56b0029040 con 0x55cc4d8345e0 2024-05-09T00:48:27.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.957+0000 7f56aaffd700 1 -- 172.21.15.5:0/3583851269 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b001a070 con 0x55cc4d8345e0 2024-05-09T00:48:27.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.958+0000 7f56d6468b80 1 -- 172.21.15.5:0/3583851269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc4d8345e0 msgr2=0x55cc4d840540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:27.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.958+0000 7f56d6468b80 1 --2- 172.21.15.5:0/3583851269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc4d8345e0 0x55cc4d840540 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f56b0003f80 tx=0x7f56b0003fb0 comp rx=0 tx=0).stop 2024-05-09T00:48:27.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.958+0000 7f56d6468b80 1 -- 172.21.15.5:0/3583851269 shutdown_connections 2024-05-09T00:48:27.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.958+0000 7f56d6468b80 1 --2- 172.21.15.5:0/3583851269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc4d8345e0 0x55cc4d840540 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:27.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.958+0000 7f56d6468b80 1 --2- 172.21.15.5:0/3583851269 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cc4d6a14a0 0x55cc4d8340a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:27.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.958+0000 7f56d6468b80 1 --2- 172.21.15.5:0/3583851269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc4d81e6a0 0x55cc4d840000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:27.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.958+0000 7f56d6468b80 1 -- 172.21.15.5:0/3583851269 >> 172.21.15.5:0/3583851269 conn(0x55cc4d7016a0 msgr2=0x55cc4d83a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:27.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.958+0000 7f56d6468b80 1 -- 172.21.15.5:0/3583851269 shutdown_connections 2024-05-09T00:48:27.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.958+0000 7f56d6468b80 1 -- 172.21.15.5:0/3583851269 wait complete. 2024-05-09T00:48:27.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.959+0000 7f56d6468b80 1 -- start start 2024-05-09T00:48:27.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.959+0000 7f56d6468b80 1 Processor -- start 2024-05-09T00:48:27.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.960+0000 7f56d6468b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cc4d6a14a0 0x55cc4d830420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:27.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.960+0000 7f56d6468b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc4d81e6a0 0x55cc4d830960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:27.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.960+0000 7f56d6468b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc4d840610 0x55cc4d83eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:27.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.960+0000 7f56c4fed700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cc4d6a14a0 0x55cc4d830420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:27.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.960+0000 7f56c4fed700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc4d840610 0x55cc4d83eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:27.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.960+0000 7f56c4fed700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc4d840610 0x55cc4d83eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60056/0 (socket says 172.21.15.5:60056) 2024-05-09T00:48:27.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.960+0000 7f56c4fed700 1 -- 172.21.15.5:0/1610952018 learned_addr learned my addr 172.21.15.5:0/1610952018 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:27.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.960+0000 7f56c57ee700 1 --2- 172.21.15.5:0/1610952018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc4d81e6a0 0x55cc4d830960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:27.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.960+0000 7f56d6468b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f56c01b2270 con 0x55cc4d840610 2024-05-09T00:48:27.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.960+0000 7f56d6468b80 1 -- 172.21.15.5:0/1610952018 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f56c01b1ba0 con 0x55cc4d6a14a0 2024-05-09T00:48:27.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.961+0000 7f56d6468b80 1 -- 172.21.15.5:0/1610952018 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f56c01aeda0 con 0x55cc4d81e6a0 2024-05-09T00:48:27.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.961+0000 7f56c4fed700 1 -- 172.21.15.5:0/1610952018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc4d81e6a0 msgr2=0x55cc4d830960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:27.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.961+0000 7f56c4fed700 1 --2- 172.21.15.5:0/1610952018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc4d81e6a0 0x55cc4d830960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:27.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.961+0000 7f56c4fed700 1 -- 172.21.15.5:0/1610952018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cc4d6a14a0 msgr2=0x55cc4d830420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:27.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.961+0000 7f56c4fed700 1 --2- 172.21.15.5:0/1610952018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cc4d6a14a0 0x55cc4d830420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:27.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.961+0000 7f56c4fed700 1 -- 172.21.15.5:0/1610952018 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc4d83f660 con 0x55cc4d840610 2024-05-09T00:48:27.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.961+0000 7f56c4fed700 1 --2- 172.21.15.5:0/1610952018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc4d840610 0x55cc4d83eff0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f56b00117e0 tx=0x7f56b0005910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c19362175fcf76b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:27.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.961+0000 7f56a8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:48:27.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.961+0000 7f56d6468b80 10 client.0 _ll_register_callbacks cb 0x55cc4d819010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:48:27.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.961+0000 7f56a8ff9700 1 -- 172.21.15.5:0/1610952018 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b001a030 con 0x55cc4d840610 2024-05-09T00:48:27.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.962+0000 7f56a8ff9700 1 -- 172.21.15.5:0/1610952018 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f56b00052f0 con 0x55cc4d840610 2024-05-09T00:48:27.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.962+0000 7f56d6468b80 1 -- 172.21.15.5:0/1610952018 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56b0029040 con 0x55cc4d840610 2024-05-09T00:48:27.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.962+0000 7f56a8ff9700 1 -- 172.21.15.5:0/1610952018 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f56b00054f0 con 0x55cc4d840610 2024-05-09T00:48:27.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.962+0000 7f56d6468b80 10 client.15564 fetch_fsmap learned FSMap version 29 2024-05-09T00:48:27.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.962+0000 7f56d6468b80 1 -- 172.21.15.5:0/1610952018 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({fsmap=29}) -- 0x55cc4d8a2150 con 0x55cc4d840610 2024-05-09T00:48:27.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.962+0000 7f56a8ff9700 1 -- 172.21.15.5:0/1610952018 <== mon.0 v2:172.21.15.5:3300/0 4 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f56b002e070 con 0x55cc4d840610 2024-05-09T00:48:27.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.963+0000 7f56d6468b80 10 client.15564 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:48:27.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.963+0000 7f56d6468b80 10 client.15564 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T00:48:27.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.963+0000 7f56d6468b80 20 client.15564 populate_metadata read hostname 'smithi005' 2024-05-09T00:48:27.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.963+0000 7f56d6468b80 1 --2- 172.21.15.5:0/1610952018 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x55cc4d8a24e0 0x55cc4d8a4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:27.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.963+0000 7f56d6468b80 4 client.15564 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:48:27.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.963+0000 7f56d6468b80 1 -- 172.21.15.5:0/1610952018 --> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f56b0029040 con 0x55cc4d8a24e0 2024-05-09T00:48:27.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.963+0000 7f56c57ee700 1 --2- 172.21.15.5:0/1610952018 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x55cc4d8a24e0 0x55cc4d8a4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:27.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.963+0000 7f56c57ee700 1 --2- 172.21.15.5:0/1610952018 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x55cc4d8a24e0 0x55cc4d8a4980 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=835a3c3824e86277 server_cookie=a939498c3b34a13d in_seq=0 out_seq=0 2024-05-09T00:48:27.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.963+0000 7f56a8ff9700 10 client.15564 ms_handle_connect on v2:172.21.15.131:6836/1633006186 2024-05-09T00:48:27.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.964+0000 7f56a8ff9700 1 -- 172.21.15.5:0/1610952018 <== mds.4 v2:172.21.15.131:6836/1633006186 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f56b0029040 con 0x55cc4d8a24e0 2024-05-09T00:48:27.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.964+0000 7f56a8ff9700 10 client.15564 handle_command_reply: tid=0 2024-05-09T00:48:27.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.964+0000 7f56d6468b80 1 client.15564 shutdown 2024-05-09T00:48:27.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.964+0000 7f56d6468b80 1 -- 172.21.15.5:0/1610952018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc4d840610 msgr2=0x55cc4d83eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:27.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.964+0000 7f56d6468b80 1 --2- 172.21.15.5:0/1610952018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc4d840610 0x55cc4d83eff0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f56b00117e0 tx=0x7f56b0005910 comp rx=0 tx=0).stop 2024-05-09T00:48:27.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.964+0000 7f56d6468b80 1 -- 172.21.15.5:0/1610952018 shutdown_connections 2024-05-09T00:48:27.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.964+0000 7f56d6468b80 1 --2- 172.21.15.5:0/1610952018 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x55cc4d8a24e0 0x55cc4d8a4980 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:27.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.964+0000 7f56d6468b80 1 --2- 172.21.15.5:0/1610952018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc4d840610 0x55cc4d83eff0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:27.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.964+0000 7f56d6468b80 1 --2- 172.21.15.5:0/1610952018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cc4d6a14a0 0x55cc4d830420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:27.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.964+0000 7f56d6468b80 1 --2- 172.21.15.5:0/1610952018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc4d81e6a0 0x55cc4d830960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:27.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.964+0000 7f56a8ff9700 0 client.15564 ms_handle_reset on v2:172.21.15.131:6836/1633006186 2024-05-09T00:48:27.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.964+0000 7f56d6468b80 1 -- 172.21.15.5:0/1610952018 >> 172.21.15.5:0/1610952018 conn(0x55cc4d7016a0 msgr2=0x55cc4d826a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:27.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.964+0000 7f56d6468b80 1 -- 172.21.15.5:0/1610952018 shutdown_connections 2024-05-09T00:48:27.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.964+0000 7f56d6468b80 1 -- 172.21.15.5:0/1610952018 wait complete. 2024-05-09T00:48:27.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.964+0000 7f56d6468b80 20 client.15564 trim_cache size 0 max 16384 2024-05-09T00:48:27.969 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-09T00:48:27.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.965+0000 7f56c7a52700 1 -- 172.21.15.5:0/2927997285 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f56ac0b2900 msgr2=0x7f56ac0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:27.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.965+0000 7f56c7a52700 1 --2- 172.21.15.5:0/2927997285 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f56ac0b2900 0x7f56ac0b4e10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f56b80e0cb0 tx=0x7f56b80d9de0 comp rx=0 tx=0).stop 2024-05-09T00:48:27.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.965+0000 7f56c7a52700 1 -- 172.21.15.5:0/2927997285 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56c0108740 msgr2=0x7f56c01a1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:27.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.965+0000 7f56c7a52700 1 --2- 172.21.15.5:0/2927997285 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56c0108740 0x7f56c01a1840 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f56b800b0d0 tx=0x7f56b8007a00 comp rx=0 tx=0).stop 2024-05-09T00:48:27.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.965+0000 7f56c7a52700 1 -- 172.21.15.5:0/2927997285 shutdown_connections 2024-05-09T00:48:27.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.965+0000 7f56c7a52700 1 --2- 172.21.15.5:0/2927997285 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f56ac0b2900 0x7f56ac0b4e10 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:27.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.965+0000 7f56c7a52700 1 --2- 172.21.15.5:0/2927997285 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56c0108740 0x7f56c01a1840 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:27.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.965+0000 7f56c7a52700 1 --2- 172.21.15.5:0/2927997285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56c01034c0 0x7f56c019d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:27.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.965+0000 7f56c7a52700 1 --2- 172.21.15.5:0/2927997285 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f56c0107df0 0x7f56c019d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:27.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.965+0000 7f56c7a52700 1 -- 172.21.15.5:0/2927997285 >> 172.21.15.5:0/2927997285 conn(0x7f56c00fedc0 msgr2=0x7f56c01096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:27.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.966+0000 7f56c7a52700 1 -- 172.21.15.5:0/2927997285 shutdown_connections 2024-05-09T00:48:27.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:27.966+0000 7f56c7a52700 1 -- 172.21.15.5:0/2927997285 wait complete. 2024-05-09T00:48:28.675 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-09T00:48:29.081 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:48:29.392 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:48:29 smithi005 ceph-mon[30365]: pgmap v435: 97 pgs: 97 active+clean; 669 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 0 B/s wr, 10 op/s 2024-05-09T00:48:29.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:48:29 smithi131 ceph-mon[34547]: pgmap v435: 97 pgs: 97 active+clean; 669 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 0 B/s wr, 10 op/s 2024-05-09T00:48:29.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:48:29 smithi157 ceph-mon[34294]: pgmap v435: 97 pgs: 97 active+clean; 669 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 0 B/s wr, 10 op/s 2024-05-09T00:48:31.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.076+0000 7f0d9321a700 1 -- 172.21.15.5:0/1428508175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d8c107df0 msgr2=0x7f0d8c108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:31.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.076+0000 7f0d9321a700 1 --2- 172.21.15.5:0/1428508175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d8c107df0 0x7f0d8c108200 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f0d780099c0 tx=0x7f0d78009cd0 comp rx=0 tx=0).stop 2024-05-09T00:48:31.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.077+0000 7f0d9321a700 1 -- 172.21.15.5:0/1428508175 shutdown_connections 2024-05-09T00:48:31.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.077+0000 7f0d9321a700 1 --2- 172.21.15.5:0/1428508175 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d8c108740 0x7f0d8c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:31.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.077+0000 7f0d9321a700 1 --2- 172.21.15.5:0/1428508175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d8c1034c0 0x7f0d8c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:31.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.077+0000 7f0d9321a700 1 --2- 172.21.15.5:0/1428508175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d8c107df0 0x7f0d8c108200 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:31.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.077+0000 7f0d9321a700 1 -- 172.21.15.5:0/1428508175 >> 172.21.15.5:0/1428508175 conn(0x7f0d8c0fedc0 msgr2=0x7f0d8c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:31.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.077+0000 7f0d9321a700 1 -- 172.21.15.5:0/1428508175 shutdown_connections 2024-05-09T00:48:31.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.077+0000 7f0d9321a700 1 -- 172.21.15.5:0/1428508175 wait complete. 2024-05-09T00:48:31.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.077+0000 7f0d9321a700 1 Processor -- start 2024-05-09T00:48:31.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.077+0000 7f0d9321a700 1 -- start start 2024-05-09T00:48:31.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.078+0000 7f0d9321a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d8c1034c0 0x7f0d8c19ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:31.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.078+0000 7f0d9321a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d8c107df0 0x7f0d8c19b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:31.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.078+0000 7f0d9321a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d8c108740 0x7f0d8c19f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:31.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.078+0000 7f0d9321a700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8c1144c0 con 0x7f0d8c108740 2024-05-09T00:48:31.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.078+0000 7f0d9321a700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8c114340 con 0x7f0d8c107df0 2024-05-09T00:48:31.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.078+0000 7f0d9321a700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8c114640 con 0x7f0d8c1034c0 2024-05-09T00:48:31.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.078+0000 7f0d917b7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d8c108740 0x7f0d8c19f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:31.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.078+0000 7f0d917b7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d8c108740 0x7f0d8c19f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60076/0 (socket says 172.21.15.5:60076) 2024-05-09T00:48:31.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.078+0000 7f0d917b7700 1 -- 172.21.15.5:0/2685727671 learned_addr learned my addr 172.21.15.5:0/2685727671 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:31.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.078+0000 7f0d90fb6700 1 --2- 172.21.15.5:0/2685727671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d8c1034c0 0x7f0d8c19ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:31.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.078+0000 7f0d8bfff700 1 --2- 172.21.15.5:0/2685727671 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d8c107df0 0x7f0d8c19b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:31.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.079+0000 7f0d917b7700 1 -- 172.21.15.5:0/2685727671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d8c1034c0 msgr2=0x7f0d8c19ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:31.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.079+0000 7f0d917b7700 1 --2- 172.21.15.5:0/2685727671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d8c1034c0 0x7f0d8c19ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:31.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.079+0000 7f0d917b7700 1 -- 172.21.15.5:0/2685727671 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d8c107df0 msgr2=0x7f0d8c19b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:31.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.079+0000 7f0d917b7700 1 --2- 172.21.15.5:0/2685727671 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d8c107df0 0x7f0d8c19b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:31.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.079+0000 7f0d917b7700 1 -- 172.21.15.5:0/2685727671 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d8c19fd00 con 0x7f0d8c108740 2024-05-09T00:48:31.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.079+0000 7f0d917b7700 1 --2- 172.21.15.5:0/2685727671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d8c108740 0x7f0d8c19f660 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f0d8400ef30 tx=0x7f0d8400c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:31.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.079+0000 7f0d8bfff700 1 --2- 172.21.15.5:0/2685727671 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d8c107df0 0x7f0d8c19b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:48:31.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.079+0000 7f0d89ffb700 1 -- 172.21.15.5:0/2685727671 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d840092e0 con 0x7f0d8c108740 2024-05-09T00:48:31.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.079+0000 7f0d89ffb700 1 -- 172.21.15.5:0/2685727671 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d84004bb0 con 0x7f0d8c108740 2024-05-09T00:48:31.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.079+0000 7f0d90fb6700 1 --2- 172.21.15.5:0/2685727671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d8c1034c0 0x7f0d8c19ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:48:31.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.080+0000 7f0d9321a700 1 -- 172.21.15.5:0/2685727671 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d8c19ff90 con 0x7f0d8c108740 2024-05-09T00:48:31.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.080+0000 7f0d89ffb700 1 -- 172.21.15.5:0/2685727671 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d84004280 con 0x7f0d8c108740 2024-05-09T00:48:31.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.080+0000 7f0d9321a700 1 -- 172.21.15.5:0/2685727671 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d8c1a7910 con 0x7f0d8c108740 2024-05-09T00:48:31.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.083+0000 7f0d89ffb700 1 -- 172.21.15.5:0/2685727671 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f0d8401b030 con 0x7f0d8c108740 2024-05-09T00:48:31.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.084+0000 7f0d89ffb700 1 --2- 172.21.15.5:0/2685727671 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f0d7c0b2a50 0x7f0d7c0b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:31.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.084+0000 7f0d89ffb700 1 -- 172.21.15.5:0/2685727671 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f0d840e1e30 con 0x7f0d8c108740 2024-05-09T00:48:31.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.084+0000 7f0d90fb6700 1 --2- 172.21.15.5:0/2685727671 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f0d7c0b2a50 0x7f0d7c0b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:31.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.085+0000 7f0d90fb6700 1 --2- 172.21.15.5:0/2685727671 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f0d7c0b2a50 0x7f0d7c0b4f60 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0d80009be0 tx=0x7f0d80009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:31.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.086+0000 7f0da1c30b80 1 Processor -- start 2024-05-09T00:48:31.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.086+0000 7f0da1c30b80 1 -- start start 2024-05-09T00:48:31.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.087+0000 7f0da1c30b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b679378d00 0x55b67947fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:31.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.087+0000 7f0da1c30b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b6792fe4a0 0x55b679480030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:31.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.087+0000 7f0da1c30b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6794841e0 0x55b6794845f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:31.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.087+0000 7f0d90fb6700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b679378d00 0x55b67947fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:31.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.087+0000 7f0d90fb6700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b679378d00 0x55b67947fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:49798/0 (socket says 172.21.15.5:49798) 2024-05-09T00:48:31.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.087+0000 7f0d8bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b6792fe4a0 0x55b679480030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:31.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.087+0000 7f0d8bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b6792fe4a0 0x55b679480030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:53104/0 (socket says 172.21.15.5:53104) 2024-05-09T00:48:31.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.087+0000 7f0d90fb6700 1 -- 172.21.15.5:0/111900854 learned_addr learned my addr 172.21.15.5:0/111900854 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:31.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.088+0000 7f0d917b7700 1 --2- 172.21.15.5:0/111900854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6794841e0 0x55b6794845f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:31.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.088+0000 7f0da1c30b80 1 -- 172.21.15.5:0/111900854 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b67939e0a0 con 0x55b6794841e0 2024-05-09T00:48:31.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.088+0000 7f0da1c30b80 1 -- 172.21.15.5:0/111900854 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55b6793439a0 con 0x55b6792fe4a0 2024-05-09T00:48:31.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.088+0000 7f0da1c30b80 1 -- 172.21.15.5:0/111900854 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b67939a060 con 0x55b679378d00 2024-05-09T00:48:31.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.088+0000 7f0d917b7700 1 -- 172.21.15.5:0/111900854 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b679378d00 msgr2=0x55b67947fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:31.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.088+0000 7f0d917b7700 1 --2- 172.21.15.5:0/111900854 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b679378d00 0x55b67947fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:31.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.088+0000 7f0d917b7700 1 -- 172.21.15.5:0/111900854 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b6792fe4a0 msgr2=0x55b679480030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:31.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.088+0000 7f0d917b7700 1 --2- 172.21.15.5:0/111900854 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b6792fe4a0 0x55b679480030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:31.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.088+0000 7f0d917b7700 1 -- 172.21.15.5:0/111900854 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b679334500 con 0x55b6794841e0 2024-05-09T00:48:31.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.088+0000 7f0d90fb6700 1 --2- 172.21.15.5:0/111900854 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b679378d00 0x55b67947fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:48:31.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.089+0000 7f0d917b7700 1 --2- 172.21.15.5:0/111900854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6794841e0 0x55b6794845f0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f0d840df8a0 tx=0x7f0d840a8fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20d4e1986cfaa2e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:31.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.089+0000 7f0d8bfff700 1 --2- 172.21.15.5:0/111900854 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b6792fe4a0 0x55b679480030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:48:31.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.089+0000 7f0d76ffd700 1 -- 172.21.15.5:0/111900854 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d840ad800 con 0x55b6794841e0 2024-05-09T00:48:31.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.089+0000 7f0d76ffd700 1 -- 172.21.15.5:0/111900854 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d840b2220 con 0x55b6794841e0 2024-05-09T00:48:31.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.089+0000 7f0d76ffd700 1 -- 172.21.15.5:0/111900854 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d840a3b70 con 0x55b6794841e0 2024-05-09T00:48:31.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.090+0000 7f0da1c30b80 1 -- 172.21.15.5:0/111900854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6794841e0 msgr2=0x55b6794845f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:31.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.090+0000 7f0da1c30b80 1 --2- 172.21.15.5:0/111900854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6794841e0 0x55b6794845f0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f0d840df8a0 tx=0x7f0d840a8fe0 comp rx=0 tx=0).stop 2024-05-09T00:48:31.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.090+0000 7f0da1c30b80 1 -- 172.21.15.5:0/111900854 shutdown_connections 2024-05-09T00:48:31.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.090+0000 7f0da1c30b80 1 --2- 172.21.15.5:0/111900854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6794841e0 0x55b6794845f0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:31.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.090+0000 7f0da1c30b80 1 --2- 172.21.15.5:0/111900854 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b679378d00 0x55b67947fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:31.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.090+0000 7f0da1c30b80 1 --2- 172.21.15.5:0/111900854 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b6792fe4a0 0x55b679480030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:31.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.090+0000 7f0da1c30b80 1 -- 172.21.15.5:0/111900854 >> 172.21.15.5:0/111900854 conn(0x55b679301f70 msgr2=0x55b679476220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:31.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.090+0000 7f0da1c30b80 1 -- 172.21.15.5:0/111900854 shutdown_connections 2024-05-09T00:48:31.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.090+0000 7f0da1c30b80 1 -- 172.21.15.5:0/111900854 wait complete. 2024-05-09T00:48:31.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.091+0000 7f0da1c30b80 1 -- start start 2024-05-09T00:48:31.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.091+0000 7f0da1c30b80 1 Processor -- start 2024-05-09T00:48:31.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.092+0000 7f0da1c30b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b6792fe4a0 0x55b67947f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:31.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.092+0000 7f0da1c30b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b679378d00 0x55b679478480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:31.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.092+0000 7f0da1c30b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b6794841e0 0x55b6794789c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:31.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.092+0000 7f0d8bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b679378d00 0x55b679478480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:31.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.092+0000 7f0da1c30b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b679484f00 con 0x55b679378d00 2024-05-09T00:48:31.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.092+0000 7f0d8bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b679378d00 0x55b679478480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60098/0 (socket says 172.21.15.5:60098) 2024-05-09T00:48:31.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.092+0000 7f0d8bfff700 1 -- 172.21.15.5:0/4131046442 learned_addr learned my addr 172.21.15.5:0/4131046442 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:31.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.092+0000 7f0da1c30b80 1 -- 172.21.15.5:0/4131046442 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55b679338b20 con 0x55b6792fe4a0 2024-05-09T00:48:31.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.092+0000 7f0da1c30b80 1 -- 172.21.15.5:0/4131046442 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b67948c470 con 0x55b6794841e0 2024-05-09T00:48:31.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.092+0000 7f0d90fb6700 1 --2- 172.21.15.5:0/4131046442 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b6792fe4a0 0x55b67947f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:31.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.092+0000 7f0da1c30b80 10 client.0 _ll_register_callbacks cb 0x55b67920f6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:48:31.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.092+0000 7f0d917b7700 1 --2- 172.21.15.5:0/4131046442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b6794841e0 0x55b6794789c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:31.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.092+0000 7f0d8bfff700 1 -- 172.21.15.5:0/4131046442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b6794841e0 msgr2=0x55b6794789c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:31.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.092+0000 7f0d8bfff700 1 --2- 172.21.15.5:0/4131046442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b6794841e0 0x55b6794789c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:31.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.092+0000 7f0d8bfff700 1 -- 172.21.15.5:0/4131046442 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b6792fe4a0 msgr2=0x55b67947f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:31.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.092+0000 7f0d8bfff700 1 --2- 172.21.15.5:0/4131046442 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b6792fe4a0 0x55b67947f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:31.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.092+0000 7f0d8bfff700 1 -- 172.21.15.5:0/4131046442 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b679219490 con 0x55b679378d00 2024-05-09T00:48:31.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.092+0000 7f0d917b7700 1 --2- 172.21.15.5:0/4131046442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b6794841e0 0x55b6794789c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:48:31.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.092+0000 7f0d90fb6700 1 --2- 172.21.15.5:0/4131046442 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b6792fe4a0 0x55b67947f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:48:31.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.092+0000 7f0d8bfff700 1 --2- 172.21.15.5:0/4131046442 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b679378d00 0x55b679478480 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f0d78006010 tx=0x7f0d78005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9f5b4833c68e893 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:31.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.092+0000 7f0d6bfff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:48:31.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.093+0000 7f0d6bfff700 1 -- 172.21.15.5:0/4131046442 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d78027060 con 0x55b679378d00 2024-05-09T00:48:31.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.093+0000 7f0d6bfff700 1 -- 172.21.15.5:0/4131046442 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f0d780039c0 con 0x55b679378d00 2024-05-09T00:48:31.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.093+0000 7f0da1c30b80 1 -- 172.21.15.5:0/4131046442 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d840b2220 con 0x55b679378d00 2024-05-09T00:48:31.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.093+0000 7f0d6bfff700 1 -- 172.21.15.5:0/4131046442 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0d78003bc0 con 0x55b679378d00 2024-05-09T00:48:31.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.093+0000 7f0da1c30b80 10 client.15576 fetch_fsmap learned FSMap version 29 2024-05-09T00:48:31.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.093+0000 7f0da1c30b80 1 -- 172.21.15.5:0/4131046442 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b679492b10 con 0x55b679378d00 2024-05-09T00:48:31.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.093+0000 7f0d6bfff700 1 -- 172.21.15.5:0/4131046442 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d840b2220 con 0x55b679378d00 2024-05-09T00:48:31.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.093+0000 7f0d6bfff700 1 -- 172.21.15.5:0/4131046442 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f0d78021070 con 0x55b679378d00 2024-05-09T00:48:31.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.094+0000 7f0da1c30b80 10 client.15576 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:48:31.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.094+0000 7f0da1c30b80 10 client.15576 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T00:48:31.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.094+0000 7f0da1c30b80 20 client.15576 populate_metadata read hostname 'smithi005' 2024-05-09T00:48:31.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.094+0000 7f0da1c30b80 1 --2- 172.21.15.5:0/4131046442 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x55b679493fd0 0x55b6794943e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:31.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.094+0000 7f0da1c30b80 4 client.15576 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:48:31.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.094+0000 7f0da1c30b80 1 -- 172.21.15.5:0/4131046442 --> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6793aba80 con 0x55b679493fd0 2024-05-09T00:48:31.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.094+0000 7f0d90fb6700 1 --2- 172.21.15.5:0/4131046442 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x55b679493fd0 0x55b6794943e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:31.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.095+0000 7f0d6bfff700 10 client.15576 ms_handle_connect on v2:172.21.15.157:6834/832898273 2024-05-09T00:48:31.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.095+0000 7f0d90fb6700 1 --2- 172.21.15.5:0/4131046442 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x55b679493fd0 0x55b6794943e0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e2b1868b5f2dbfd8 server_cookie=fc2b76f6a2749d33 in_seq=0 out_seq=0 2024-05-09T00:48:31.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.095+0000 7f0d6bfff700 1 -- 172.21.15.5:0/4131046442 <== mds.3 v2:172.21.15.157:6834/832898273 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b6793aba80 con 0x55b679493fd0 2024-05-09T00:48:31.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.095+0000 7f0d6bfff700 10 client.15576 handle_command_reply: tid=0 2024-05-09T00:48:31.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.095+0000 7f0da1c30b80 1 client.15576 shutdown 2024-05-09T00:48:31.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.096+0000 7f0da1c30b80 1 -- 172.21.15.5:0/4131046442 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b679378d00 msgr2=0x55b679478480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:31.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.096+0000 7f0da1c30b80 1 --2- 172.21.15.5:0/4131046442 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b679378d00 0x55b679478480 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f0d78006010 tx=0x7f0d78005770 comp rx=0 tx=0).stop 2024-05-09T00:48:31.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.096+0000 7f0da1c30b80 1 -- 172.21.15.5:0/4131046442 shutdown_connections 2024-05-09T00:48:31.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.096+0000 7f0da1c30b80 1 --2- 172.21.15.5:0/4131046442 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x55b679493fd0 0x55b6794943e0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:31.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.096+0000 7f0da1c30b80 1 --2- 172.21.15.5:0/4131046442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b6794841e0 0x55b6794789c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:31.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.096+0000 7f0da1c30b80 1 --2- 172.21.15.5:0/4131046442 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b6792fe4a0 0x55b67947f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:31.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.096+0000 7f0d6bfff700 0 client.15576 ms_handle_reset on v2:172.21.15.157:6834/832898273 2024-05-09T00:48:31.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.096+0000 7f0da1c30b80 1 --2- 172.21.15.5:0/4131046442 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b679378d00 0x55b679478480 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:31.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.096+0000 7f0da1c30b80 1 -- 172.21.15.5:0/4131046442 >> 172.21.15.5:0/4131046442 conn(0x55b679301f70 msgr2=0x55b67946fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:31.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.096+0000 7f0da1c30b80 1 -- 172.21.15.5:0/4131046442 shutdown_connections 2024-05-09T00:48:31.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.096+0000 7f0da1c30b80 1 -- 172.21.15.5:0/4131046442 wait complete. 2024-05-09T00:48:31.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.096+0000 7f0da1c30b80 20 client.15576 trim_cache size 0 max 16384 2024-05-09T00:48:31.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.125+0000 7f0da1c30b80 1 Processor -- start 2024-05-09T00:48:31.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.125+0000 7f0da1c30b80 1 -- start start 2024-05-09T00:48:31.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.125+0000 7f0da1c30b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6792fe4a0 0x55b6794befe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:31.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.125+0000 7f0da1c30b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b6794841e0 0x55b6794bf520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:31.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.125+0000 7f0da1c30b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b679493fd0 0x55b679475cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:31.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.125+0000 7f0da1c30b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8c0721e0 con 0x55b6792fe4a0 2024-05-09T00:48:31.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.125+0000 7f0da1c30b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8c073d20 con 0x55b6794841e0 2024-05-09T00:48:31.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.125+0000 7f0da1c30b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8c0728b0 con 0x55b679493fd0 2024-05-09T00:48:31.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.126+0000 7f0d90fb6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6792fe4a0 0x55b6794befe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:31.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.126+0000 7f0d8bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b6794841e0 0x55b6794bf520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:31.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.126+0000 7f0d8bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b6794841e0 0x55b6794bf520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:53134/0 (socket says 172.21.15.5:53134) 2024-05-09T00:48:31.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.126+0000 7f0d8bfff700 1 -- 172.21.15.5:0/3742616852 learned_addr learned my addr 172.21.15.5:0/3742616852 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:31.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.126+0000 7f0d917b7700 1 --2- 172.21.15.5:0/3742616852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b679493fd0 0x55b679475cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:31.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.126+0000 7f0d8bfff700 1 -- 172.21.15.5:0/3742616852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b679493fd0 msgr2=0x55b679475cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:31.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.126+0000 7f0d8bfff700 1 --2- 172.21.15.5:0/3742616852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b679493fd0 0x55b679475cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:31.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.126+0000 7f0d8bfff700 1 -- 172.21.15.5:0/3742616852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6792fe4a0 msgr2=0x55b6794befe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:31.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.126+0000 7f0d8bfff700 1 --2- 172.21.15.5:0/3742616852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6792fe4a0 0x55b6794befe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:31.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.126+0000 7f0d8bfff700 1 -- 172.21.15.5:0/3742616852 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b679476d90 con 0x55b6794841e0 2024-05-09T00:48:31.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.126+0000 7f0d8bfff700 1 --2- 172.21.15.5:0/3742616852 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b6794841e0 0x55b6794bf520 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0d7800b580 tx=0x7f0d78005990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c9db32ef8483e2f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:31.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.127+0000 7f0d767fc700 1 -- 172.21.15.5:0/3742616852 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d78027040 con 0x55b6794841e0 2024-05-09T00:48:31.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.127+0000 7f0d767fc700 1 -- 172.21.15.5:0/3742616852 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d7800fa60 con 0x55b6794841e0 2024-05-09T00:48:31.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.127+0000 7f0d767fc700 1 -- 172.21.15.5:0/3742616852 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d7802d670 con 0x55b6794841e0 2024-05-09T00:48:31.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.128+0000 7f0da1c30b80 1 -- 172.21.15.5:0/3742616852 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b6794841e0 msgr2=0x55b6794bf520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:31.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.128+0000 7f0da1c30b80 1 --2- 172.21.15.5:0/3742616852 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b6794841e0 0x55b6794bf520 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0d7800b580 tx=0x7f0d78005990 comp rx=0 tx=0).stop 2024-05-09T00:48:31.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.128+0000 7f0da1c30b80 1 -- 172.21.15.5:0/3742616852 shutdown_connections 2024-05-09T00:48:31.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.128+0000 7f0da1c30b80 1 --2- 172.21.15.5:0/3742616852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b679493fd0 0x55b679475cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:31.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.128+0000 7f0da1c30b80 1 --2- 172.21.15.5:0/3742616852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6792fe4a0 0x55b6794befe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:31.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.128+0000 7f0da1c30b80 1 --2- 172.21.15.5:0/3742616852 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b6794841e0 0x55b6794bf520 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:31.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.128+0000 7f0da1c30b80 1 -- 172.21.15.5:0/3742616852 >> 172.21.15.5:0/3742616852 conn(0x55b679301f70 msgr2=0x55b679478440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:31.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.128+0000 7f0da1c30b80 1 -- 172.21.15.5:0/3742616852 shutdown_connections 2024-05-09T00:48:31.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.128+0000 7f0da1c30b80 1 -- 172.21.15.5:0/3742616852 wait complete. 2024-05-09T00:48:31.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.128+0000 7f0da1c30b80 1 -- start start 2024-05-09T00:48:31.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.128+0000 7f0da1c30b80 1 Processor -- start 2024-05-09T00:48:31.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.129+0000 7f0da1c30b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6792fe4a0 0x55b67948cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:31.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.129+0000 7f0da1c30b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b6794841e0 0x55b67948d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:31.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.129+0000 7f0da1c30b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b679493fd0 0x55b67951fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:31.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.129+0000 7f0da1c30b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8c073650 con 0x55b6792fe4a0 2024-05-09T00:48:31.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.129+0000 7f0da1c30b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8c072f80 con 0x55b6794841e0 2024-05-09T00:48:31.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.129+0000 7f0da1c30b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8c070180 con 0x55b679493fd0 2024-05-09T00:48:31.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.129+0000 7f0da1c30b80 10 client.0 _ll_register_callbacks cb 0x55b67920f6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:48:31.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.129+0000 7f0d90fb6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6792fe4a0 0x55b67948cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:31.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.129+0000 7f0d90fb6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6792fe4a0 0x55b67948cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60118/0 (socket says 172.21.15.5:60118) 2024-05-09T00:48:31.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.129+0000 7f0d90fb6700 1 -- 172.21.15.5:0/2139672054 learned_addr learned my addr 172.21.15.5:0/2139672054 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:31.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.129+0000 7f0d8bfff700 1 --2- 172.21.15.5:0/2139672054 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b6794841e0 0x55b67948d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:31.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.129+0000 7f0d917b7700 1 --2- 172.21.15.5:0/2139672054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b679493fd0 0x55b67951fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:31.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.130+0000 7f0d90fb6700 1 -- 172.21.15.5:0/2139672054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b679493fd0 msgr2=0x55b67951fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:31.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.130+0000 7f0d90fb6700 1 --2- 172.21.15.5:0/2139672054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b679493fd0 0x55b67951fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:31.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.130+0000 7f0d90fb6700 1 -- 172.21.15.5:0/2139672054 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b6794841e0 msgr2=0x55b67948d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:31.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.130+0000 7f0d90fb6700 1 --2- 172.21.15.5:0/2139672054 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b6794841e0 0x55b67948d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:31.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.130+0000 7f0d90fb6700 1 -- 172.21.15.5:0/2139672054 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b679520110 con 0x55b6792fe4a0 2024-05-09T00:48:31.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.130+0000 7f0d90fb6700 1 --2- 172.21.15.5:0/2139672054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6792fe4a0 0x55b67948cd70 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f0d80010700 tx=0x7f0d800047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc52366ea1a1a983 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:31.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.130+0000 7f0d6bfff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:48:31.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.130+0000 7f0d6bfff700 1 -- 172.21.15.5:0/2139672054 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d80004220 con 0x55b6792fe4a0 2024-05-09T00:48:31.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.130+0000 7f0d6bfff700 1 -- 172.21.15.5:0/2139672054 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f0d800043c0 con 0x55b6792fe4a0 2024-05-09T00:48:31.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.130+0000 7f0da1c30b80 1 -- 172.21.15.5:0/2139672054 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d7800fa60 con 0x55b6792fe4a0 2024-05-09T00:48:31.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.131+0000 7f0d6bfff700 1 -- 172.21.15.5:0/2139672054 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0d800036a0 con 0x55b6792fe4a0 2024-05-09T00:48:31.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.131+0000 7f0da1c30b80 10 client.15582 fetch_fsmap learned FSMap version 29 2024-05-09T00:48:31.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.131+0000 7f0da1c30b80 1 -- 172.21.15.5:0/2139672054 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b6795213d0 con 0x55b6792fe4a0 2024-05-09T00:48:31.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.131+0000 7f0d6bfff700 1 -- 172.21.15.5:0/2139672054 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d7800fa60 con 0x55b6792fe4a0 2024-05-09T00:48:31.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.131+0000 7f0d6bfff700 1 -- 172.21.15.5:0/2139672054 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f0d80030030 con 0x55b6792fe4a0 2024-05-09T00:48:31.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.131+0000 7f0da1c30b80 10 client.15582 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:48:31.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.131+0000 7f0da1c30b80 10 client.15582 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T00:48:31.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.131+0000 7f0da1c30b80 20 client.15582 populate_metadata read hostname 'smithi005' 2024-05-09T00:48:31.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.131+0000 7f0da1c30b80 1 --2- 172.21.15.5:0/2139672054 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x55b679521b60 0x55b679523f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:31.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.131+0000 7f0da1c30b80 4 client.15582 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:48:31.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.131+0000 7f0da1c30b80 1 -- 172.21.15.5:0/2139672054 --> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b6793a5b20 con 0x55b679521b60 2024-05-09T00:48:31.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.132+0000 7f0d8bfff700 1 --2- 172.21.15.5:0/2139672054 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x55b679521b60 0x55b679523f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:31.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.132+0000 7f0d6bfff700 10 client.15582 ms_handle_connect on v2:172.21.15.157:6834/832898273 2024-05-09T00:48:31.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.132+0000 7f0d8bfff700 1 --2- 172.21.15.5:0/2139672054 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x55b679521b60 0x55b679523f80 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9392d2beca4ddd5 server_cookie=b1d375ec1c7a9dd0 in_seq=0 out_seq=0 2024-05-09T00:48:31.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.133+0000 7f0d6bfff700 1 -- 172.21.15.5:0/2139672054 <== mds.3 v2:172.21.15.157:6834/832898273 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b6793a5b20 con 0x55b679521b60 2024-05-09T00:48:31.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.133+0000 7f0d6bfff700 10 client.15582 handle_command_reply: tid=0 2024-05-09T00:48:31.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.133+0000 7f0da1c30b80 1 client.15582 shutdown 2024-05-09T00:48:31.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.133+0000 7f0da1c30b80 1 -- 172.21.15.5:0/2139672054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6792fe4a0 msgr2=0x55b67948cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:31.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.133+0000 7f0da1c30b80 1 --2- 172.21.15.5:0/2139672054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6792fe4a0 0x55b67948cd70 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f0d80010700 tx=0x7f0d800047c0 comp rx=0 tx=0).stop 2024-05-09T00:48:31.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.133+0000 7f0da1c30b80 1 -- 172.21.15.5:0/2139672054 shutdown_connections 2024-05-09T00:48:31.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.133+0000 7f0da1c30b80 1 --2- 172.21.15.5:0/2139672054 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x55b679521b60 0x55b679523f80 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:31.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.133+0000 7f0da1c30b80 1 --2- 172.21.15.5:0/2139672054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b679493fd0 0x55b67951fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:31.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.133+0000 7f0d6bfff700 0 client.15582 ms_handle_reset on v2:172.21.15.157:6834/832898273 2024-05-09T00:48:31.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.133+0000 7f0da1c30b80 1 --2- 172.21.15.5:0/2139672054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6792fe4a0 0x55b67948cd70 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:31.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.133+0000 7f0da1c30b80 1 --2- 172.21.15.5:0/2139672054 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b6794841e0 0x55b67948d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:31.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.133+0000 7f0da1c30b80 1 -- 172.21.15.5:0/2139672054 >> 172.21.15.5:0/2139672054 conn(0x55b679301f70 msgr2=0x55b679478420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:31.138 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-09T00:48:31.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.133+0000 7f0da1c30b80 1 -- 172.21.15.5:0/2139672054 shutdown_connections 2024-05-09T00:48:31.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.133+0000 7f0da1c30b80 1 -- 172.21.15.5:0/2139672054 wait complete. 2024-05-09T00:48:31.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.133+0000 7f0da1c30b80 20 client.15582 trim_cache size 0 max 16384 2024-05-09T00:48:31.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.134+0000 7f0d9321a700 1 -- 172.21.15.5:0/2685727671 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f0d7c0b2a50 msgr2=0x7f0d7c0b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:31.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.134+0000 7f0d9321a700 1 --2- 172.21.15.5:0/2685727671 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f0d7c0b2a50 0x7f0d7c0b4f60 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0d80009be0 tx=0x7f0d80009260 comp rx=0 tx=0).stop 2024-05-09T00:48:31.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.134+0000 7f0d9321a700 1 -- 172.21.15.5:0/2685727671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d8c108740 msgr2=0x7f0d8c19f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:31.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.134+0000 7f0d9321a700 1 --2- 172.21.15.5:0/2685727671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d8c108740 0x7f0d8c19f660 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f0d8400ef30 tx=0x7f0d8400c450 comp rx=0 tx=0).stop 2024-05-09T00:48:31.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.134+0000 7f0d9321a700 1 -- 172.21.15.5:0/2685727671 shutdown_connections 2024-05-09T00:48:31.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.134+0000 7f0d9321a700 1 --2- 172.21.15.5:0/2685727671 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f0d7c0b2a50 0x7f0d7c0b4f60 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:31.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.134+0000 7f0d9321a700 1 --2- 172.21.15.5:0/2685727671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d8c108740 0x7f0d8c19f660 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:31.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.134+0000 7f0d9321a700 1 --2- 172.21.15.5:0/2685727671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d8c1034c0 0x7f0d8c19ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:31.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.134+0000 7f0d9321a700 1 --2- 172.21.15.5:0/2685727671 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d8c107df0 0x7f0d8c19b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:31.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.134+0000 7f0d9321a700 1 -- 172.21.15.5:0/2685727671 >> 172.21.15.5:0/2685727671 conn(0x7f0d8c0fedc0 msgr2=0x7f0d8c1096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:31.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.134+0000 7f0d9321a700 1 -- 172.21.15.5:0/2685727671 shutdown_connections 2024-05-09T00:48:31.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:31.134+0000 7f0d9321a700 1 -- 172.21.15.5:0/2685727671 wait complete. 2024-05-09T00:48:31.523 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:48:31 smithi005 ceph-mon[30365]: pgmap v436: 97 pgs: 97 active+clean; 670 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-09T00:48:31.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:48:31 smithi131 ceph-mon[34547]: pgmap v436: 97 pgs: 97 active+clean; 670 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-09T00:48:31.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:48:31 smithi157 ceph-mon[34294]: pgmap v436: 97 pgs: 97 active+clean; 670 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-09T00:48:31.977 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T00:48:32.363 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:48:33.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:48:33 smithi005 ceph-mon[30365]: pgmap v437: 97 pgs: 97 active+clean; 670 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T00:48:33.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:48:33 smithi131 ceph-mon[34547]: pgmap v437: 97 pgs: 97 active+clean; 670 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T00:48:33.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:48:33 smithi157 ceph-mon[34294]: pgmap v437: 97 pgs: 97 active+clean; 670 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T00:48:35.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:48:35 smithi005 ceph-mon[30365]: pgmap v438: 97 pgs: 97 active+clean; 670 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-09T00:48:35.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:48:35 smithi131 ceph-mon[34547]: pgmap v438: 97 pgs: 97 active+clean; 670 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-09T00:48:35.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:48:35 smithi157 ceph-mon[34294]: pgmap v438: 97 pgs: 97 active+clean; 670 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-09T00:48:37.353 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:48:37 smithi005 ceph-mon[30365]: pgmap v439: 97 pgs: 97 active+clean; 670 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T00:48:37.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.371+0000 7f033391c700 1 -- 172.21.15.5:0/3290272876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f032c109860 msgr2=0x7f032c1111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:37.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.371+0000 7f033391c700 1 --2- 172.21.15.5:0/3290272876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f032c109860 0x7f032c1111c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f032400b0d0 tx=0x7f032400b3e0 comp rx=0 tx=0).stop 2024-05-09T00:48:37.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.371+0000 7f033391c700 1 -- 172.21.15.5:0/3290272876 shutdown_connections 2024-05-09T00:48:37.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.371+0000 7f033391c700 1 --2- 172.21.15.5:0/3290272876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f032c109860 0x7f032c1111c0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:37.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.371+0000 7f033391c700 1 --2- 172.21.15.5:0/3290272876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f032c1045e0 0x7f032c1049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:37.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.371+0000 7f033391c700 1 --2- 172.21.15.5:0/3290272876 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f032c108f10 0x7f032c109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:37.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.371+0000 7f033391c700 1 -- 172.21.15.5:0/3290272876 >> 172.21.15.5:0/3290272876 conn(0x7f032c0ffe70 msgr2=0x7f032c1022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:37.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.371+0000 7f033391c700 1 -- 172.21.15.5:0/3290272876 shutdown_connections 2024-05-09T00:48:37.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.371+0000 7f033391c700 1 -- 172.21.15.5:0/3290272876 wait complete. 2024-05-09T00:48:37.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.372+0000 7f033391c700 1 Processor -- start 2024-05-09T00:48:37.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.372+0000 7f033391c700 1 -- start start 2024-05-09T00:48:37.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.373+0000 7f033391c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f032c1045e0 0x7f032c075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:37.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.373+0000 7f033391c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f032c108f10 0x7f032c076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:37.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.373+0000 7f033391c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f032c109860 0x7f032c076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:37.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.373+0000 7f033391c700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f032c114490 con 0x7f032c109860 2024-05-09T00:48:37.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.373+0000 7f033391c700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f032c114310 con 0x7f032c1045e0 2024-05-09T00:48:37.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.373+0000 7f033391c700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f032c114610 con 0x7f032c108f10 2024-05-09T00:48:37.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.373+0000 7f0330eb7700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f032c108f10 0x7f032c076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:37.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.373+0000 7f03316b8700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f032c1045e0 0x7f032c075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:37.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.373+0000 7f0330eb7700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f032c108f10 0x7f032c076000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:33476/0 (socket says 172.21.15.5:33476) 2024-05-09T00:48:37.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.373+0000 7f03316b8700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f032c1045e0 0x7f032c075ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:35442/0 (socket says 172.21.15.5:35442) 2024-05-09T00:48:37.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.373+0000 7f0330eb7700 1 -- 172.21.15.5:0/2722794658 learned_addr learned my addr 172.21.15.5:0/2722794658 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:37.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.373+0000 7f0331eb9700 1 --2- 172.21.15.5:0/2722794658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f032c109860 0x7f032c076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:37.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.374+0000 7f03316b8700 1 -- 172.21.15.5:0/2722794658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f032c108f10 msgr2=0x7f032c076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:37.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.374+0000 7f03316b8700 1 --2- 172.21.15.5:0/2722794658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f032c108f10 0x7f032c076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:37.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.374+0000 7f03316b8700 1 -- 172.21.15.5:0/2722794658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f032c109860 msgr2=0x7f032c076540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:37.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.374+0000 7f03316b8700 1 --2- 172.21.15.5:0/2722794658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f032c109860 0x7f032c076540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:37.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.374+0000 7f03316b8700 1 -- 172.21.15.5:0/2722794658 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f032c1a9120 con 0x7f032c1045e0 2024-05-09T00:48:37.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.374+0000 7f0330eb7700 1 --2- 172.21.15.5:0/2722794658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f032c108f10 0x7f032c076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:48:37.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.374+0000 7f03316b8700 1 --2- 172.21.15.5:0/2722794658 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f032c1045e0 0x7f032c075ac0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f032800d950 tx=0x7f032800dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:37.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.374+0000 7f0331eb9700 1 --2- 172.21.15.5:0/2722794658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f032c109860 0x7f032c076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:48:37.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.374+0000 7f03227fc700 1 -- 172.21.15.5:0/2722794658 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0328017070 con 0x7f032c1045e0 2024-05-09T00:48:37.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.374+0000 7f03227fc700 1 -- 172.21.15.5:0/2722794658 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0328009de0 con 0x7f032c1045e0 2024-05-09T00:48:37.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.375+0000 7f03227fc700 1 -- 172.21.15.5:0/2722794658 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0328010e00 con 0x7f032c1045e0 2024-05-09T00:48:37.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.375+0000 7f033391c700 1 -- 172.21.15.5:0/2722794658 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f032c1a9410 con 0x7f032c1045e0 2024-05-09T00:48:37.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.375+0000 7f033391c700 1 -- 172.21.15.5:0/2722794658 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f032c1a99a0 con 0x7f032c1045e0 2024-05-09T00:48:37.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.380+0000 7f03227fc700 1 -- 172.21.15.5:0/2722794658 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f032801a070 con 0x7f032c1045e0 2024-05-09T00:48:37.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.381+0000 7f03227fc700 1 --2- 172.21.15.5:0/2722794658 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f03180b2980 0x7f03180b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:37.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.382+0000 7f0342332b80 1 Processor -- start 2024-05-09T00:48:37.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.382+0000 7f03227fc700 1 -- 172.21.15.5:0/2722794658 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f03280e1660 con 0x7f032c1045e0 2024-05-09T00:48:37.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.382+0000 7f0330eb7700 1 --2- 172.21.15.5:0/2722794658 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f03180b2980 0x7f03180b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:37.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.382+0000 7f0342332b80 1 -- start start 2024-05-09T00:48:37.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.382+0000 7f0330eb7700 1 --2- 172.21.15.5:0/2722794658 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f03180b2980 0x7f03180b4e90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f032c103930 tx=0x7f031c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:37.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.383+0000 7f0342332b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644ea187160 0x5644ea2204f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:37.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.383+0000 7f0342332b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644ea09d4a0 0x5644ea220a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:37.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.383+0000 7f0342332b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644ea21a6a0 0x5644ea21aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:37.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.383+0000 7f0342332b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5644ea13cff0 con 0x5644ea21a6a0 2024-05-09T00:48:37.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.383+0000 7f0342332b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5644ea0db7f0 con 0x5644ea187160 2024-05-09T00:48:37.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.383+0000 7f0342332b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5644ea138fb0 con 0x5644ea09d4a0 2024-05-09T00:48:37.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.383+0000 7f0331eb9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644ea21a6a0 0x5644ea21aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:37.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.383+0000 7f0331eb9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644ea21a6a0 0x5644ea21aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38756/0 (socket says 172.21.15.5:38756) 2024-05-09T00:48:37.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.383+0000 7f0331eb9700 1 -- 172.21.15.5:0/33095405 learned_addr learned my addr 172.21.15.5:0/33095405 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:37.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.383+0000 7f0330eb7700 1 --2- 172.21.15.5:0/33095405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644ea09d4a0 0x5644ea220a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:37.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.384+0000 7f0330eb7700 1 -- 172.21.15.5:0/33095405 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644ea187160 msgr2=0x5644ea2204f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:37.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.384+0000 7f0330eb7700 1 --2- 172.21.15.5:0/33095405 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644ea187160 0x5644ea2204f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:37.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.384+0000 7f0330eb7700 1 -- 172.21.15.5:0/33095405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644ea21a6a0 msgr2=0x5644ea21aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:37.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.384+0000 7f0330eb7700 1 --2- 172.21.15.5:0/33095405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644ea21a6a0 0x5644ea21aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:37.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.384+0000 7f0330eb7700 1 -- 172.21.15.5:0/33095405 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644ea0d3370 con 0x5644ea09d4a0 2024-05-09T00:48:37.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.384+0000 7f0331eb9700 1 --2- 172.21.15.5:0/33095405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644ea21a6a0 0x5644ea21aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:48:37.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.384+0000 7f0330eb7700 1 --2- 172.21.15.5:0/33095405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644ea09d4a0 0x5644ea220a30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f031c005950 tx=0x7f031c013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c78b4aea759d0475 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:37.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.384+0000 7f03177fe700 1 -- 172.21.15.5:0/33095405 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031c011db0 con 0x5644ea09d4a0 2024-05-09T00:48:37.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.385+0000 7f03177fe700 1 -- 172.21.15.5:0/33095405 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f031c009a80 con 0x5644ea09d4a0 2024-05-09T00:48:37.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.386+0000 7f03177fe700 1 -- 172.21.15.5:0/33095405 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031c01c730 con 0x5644ea09d4a0 2024-05-09T00:48:37.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.386+0000 7f0342332b80 1 -- 172.21.15.5:0/33095405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644ea09d4a0 msgr2=0x5644ea220a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:37.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.386+0000 7f0342332b80 1 --2- 172.21.15.5:0/33095405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644ea09d4a0 0x5644ea220a30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f031c005950 tx=0x7f031c013000 comp rx=0 tx=0).stop 2024-05-09T00:48:37.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.386+0000 7f0342332b80 1 -- 172.21.15.5:0/33095405 shutdown_connections 2024-05-09T00:48:37.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.386+0000 7f0342332b80 1 --2- 172.21.15.5:0/33095405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644ea21a6a0 0x5644ea21aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:37.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.386+0000 7f0342332b80 1 --2- 172.21.15.5:0/33095405 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644ea187160 0x5644ea2204f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:37.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.386+0000 7f0342332b80 1 --2- 172.21.15.5:0/33095405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644ea09d4a0 0x5644ea220a30 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:37.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.386+0000 7f0342332b80 1 -- 172.21.15.5:0/33095405 >> 172.21.15.5:0/33095405 conn(0x5644ea0fd6a0 msgr2=0x5644ea215400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:37.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.387+0000 7f0342332b80 1 -- 172.21.15.5:0/33095405 shutdown_connections 2024-05-09T00:48:37.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.387+0000 7f0342332b80 1 -- 172.21.15.5:0/33095405 wait complete. 2024-05-09T00:48:37.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.387+0000 7f0342332b80 1 -- start start 2024-05-09T00:48:37.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.388+0000 7f0342332b80 1 Processor -- start 2024-05-09T00:48:37.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.389+0000 7f0342332b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644ea09d4a0 0x5644ea224210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:37.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.389+0000 7f0342332b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644ea187160 0x5644ea224750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:37.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.389+0000 7f0342332b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644ea21a6a0 0x5644ea224c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:37.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.389+0000 7f03316b8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644ea09d4a0 0x5644ea224210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:37.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.389+0000 7f03316b8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644ea09d4a0 0x5644ea224210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38766/0 (socket says 172.21.15.5:38766) 2024-05-09T00:48:37.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.389+0000 7f0330eb7700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644ea187160 0x5644ea224750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:37.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.389+0000 7f0330eb7700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644ea187160 0x5644ea224750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:33494/0 (socket says 172.21.15.5:33494) 2024-05-09T00:48:37.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.389+0000 7f03316b8700 1 -- 172.21.15.5:0/1793908565 learned_addr learned my addr 172.21.15.5:0/1793908565 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:37.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.389+0000 7f0342332b80 1 -- 172.21.15.5:0/1793908565 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5644e9fb8ba0 con 0x5644ea09d4a0 2024-05-09T00:48:37.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.389+0000 7f0342332b80 1 -- 172.21.15.5:0/1793908565 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5644ea0f7150 con 0x5644ea21a6a0 2024-05-09T00:48:37.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.390+0000 7f0342332b80 1 -- 172.21.15.5:0/1793908565 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5644ea2271d0 con 0x5644ea187160 2024-05-09T00:48:37.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.390+0000 7f0342332b80 10 client.0 _ll_register_callbacks cb 0x5644e9fae6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:48:37.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.390+0000 7f03316b8700 1 -- 172.21.15.5:0/1793908565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644ea187160 msgr2=0x5644ea224750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:37.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.390+0000 7f03316b8700 1 --2- 172.21.15.5:0/1793908565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644ea187160 0x5644ea224750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:37.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.390+0000 7f03316b8700 1 -- 172.21.15.5:0/1793908565 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644ea21a6a0 msgr2=0x5644ea224c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:37.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.390+0000 7f0331eb9700 1 --2- 172.21.15.5:0/1793908565 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644ea21a6a0 0x5644ea224c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:37.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.390+0000 7f03316b8700 1 --2- 172.21.15.5:0/1793908565 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644ea21a6a0 0x5644ea224c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:37.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.390+0000 7f03316b8700 1 -- 172.21.15.5:0/1793908565 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644e9fb8490 con 0x5644ea09d4a0 2024-05-09T00:48:37.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.390+0000 7f0331eb9700 1 --2- 172.21.15.5:0/1793908565 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644ea21a6a0 0x5644ea224c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:48:37.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.390+0000 7f0342332b80 1 -- 172.21.15.5:0/1793908565 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5644ea14a9d0 con 0x5644ea09d4a0 2024-05-09T00:48:37.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.390+0000 7f03316b8700 1 --2- 172.21.15.5:0/1793908565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644ea09d4a0 0x5644ea224210 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f03280a4c70 tx=0x7f03280a8360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b017e6e92227b63 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:37.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.390+0000 7f0314ff9700 10 client.15594 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:48:37.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.391+0000 7f0314ff9700 1 -- 172.21.15.5:0/1793908565 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03280c3060 con 0x5644ea09d4a0 2024-05-09T00:48:37.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.391+0000 7f0314ff9700 1 -- 172.21.15.5:0/1793908565 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f03280c3200 con 0x5644ea09d4a0 2024-05-09T00:48:37.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.391+0000 7f0314ff9700 1 -- 172.21.15.5:0/1793908565 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f03280a8700 con 0x5644ea09d4a0 2024-05-09T00:48:37.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.391+0000 7f0342332b80 10 client.15594 fetch_fsmap learned FSMap version 29 2024-05-09T00:48:37.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.391+0000 7f0342332b80 1 -- 172.21.15.5:0/1793908565 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({fsmap=29}) -- 0x5644ea231a40 con 0x5644ea09d4a0 2024-05-09T00:48:37.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.392+0000 7f0314ff9700 1 -- 172.21.15.5:0/1793908565 <== mon.0 v2:172.21.15.5:3300/0 4 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f03280b15c0 con 0x5644ea09d4a0 2024-05-09T00:48:37.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.392+0000 7f0342332b80 10 client.15594 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:48:37.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.392+0000 7f0342332b80 10 client.15594 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-05-09T00:48:37.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.392+0000 7f0342332b80 20 client.15594 populate_metadata read hostname 'smithi005' 2024-05-09T00:48:37.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.392+0000 7f0342332b80 1 --2- 172.21.15.5:0/1793908565 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x5644ea2343e0 0x5644ea236830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:37.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.392+0000 7f0342332b80 4 client.15594 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:48:37.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.392+0000 7f0342332b80 1 -- 172.21.15.5:0/1793908565 --> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644ea144a70 con 0x5644ea2343e0 2024-05-09T00:48:37.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.392+0000 7f0330eb7700 1 --2- 172.21.15.5:0/1793908565 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x5644ea2343e0 0x5644ea236830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:37.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.393+0000 7f0330eb7700 1 --2- 172.21.15.5:0/1793908565 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x5644ea2343e0 0x5644ea236830 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=21f0a618748ef10c server_cookie=12b3e82af7cf7244 in_seq=0 out_seq=0 2024-05-09T00:48:37.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.393+0000 7f0314ff9700 10 client.15594 ms_handle_connect on v2:172.21.15.131:6838/2748669631 2024-05-09T00:48:37.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.394+0000 7f0314ff9700 1 -- 172.21.15.5:0/1793908565 <== mds.2 v2:172.21.15.131:6838/2748669631 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5644ea144a70 con 0x5644ea2343e0 2024-05-09T00:48:37.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.394+0000 7f0314ff9700 10 client.15594 handle_command_reply: tid=0 2024-05-09T00:48:37.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.394+0000 7f0342332b80 1 client.15594 shutdown 2024-05-09T00:48:37.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.394+0000 7f0342332b80 1 -- 172.21.15.5:0/1793908565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644ea09d4a0 msgr2=0x5644ea224210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:37.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.394+0000 7f0342332b80 1 --2- 172.21.15.5:0/1793908565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644ea09d4a0 0x5644ea224210 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f03280a4c70 tx=0x7f03280a8360 comp rx=0 tx=0).stop 2024-05-09T00:48:37.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.394+0000 7f0342332b80 1 -- 172.21.15.5:0/1793908565 shutdown_connections 2024-05-09T00:48:37.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.394+0000 7f0342332b80 1 --2- 172.21.15.5:0/1793908565 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x5644ea2343e0 0x5644ea236830 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:37.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.394+0000 7f0342332b80 1 --2- 172.21.15.5:0/1793908565 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644ea21a6a0 0x5644ea224c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:37.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.394+0000 7f0342332b80 1 --2- 172.21.15.5:0/1793908565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644ea09d4a0 0x5644ea224210 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:37.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.394+0000 7f0342332b80 1 --2- 172.21.15.5:0/1793908565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644ea187160 0x5644ea224750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:37.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.394+0000 7f0342332b80 1 -- 172.21.15.5:0/1793908565 >> 172.21.15.5:0/1793908565 conn(0x5644ea0fd6a0 msgr2=0x5644ea2151c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:37.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.394+0000 7f0314ff9700 0 client.15594 ms_handle_reset on v2:172.21.15.131:6838/2748669631 2024-05-09T00:48:37.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.394+0000 7f0342332b80 1 -- 172.21.15.5:0/1793908565 shutdown_connections 2024-05-09T00:48:37.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.395+0000 7f0342332b80 1 -- 172.21.15.5:0/1793908565 wait complete. 2024-05-09T00:48:37.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.395+0000 7f0342332b80 20 client.15594 trim_cache size 0 max 16384 2024-05-09T00:48:37.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.425+0000 7f0342332b80 1 Processor -- start 2024-05-09T00:48:37.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.425+0000 7f0342332b80 1 -- start start 2024-05-09T00:48:37.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.426+0000 7f0342332b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644ea09d4a0 0x5644ea2241d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:37.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.426+0000 7f0342332b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644ea187160 0x5644ea224710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:37.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.426+0000 7f0342332b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644ea21a6a0 0x5644ea224c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:37.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.426+0000 7f0342332b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f032c1b0f30 con 0x5644ea187160 2024-05-09T00:48:37.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.426+0000 7f0342332b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f032c1b2a70 con 0x5644ea09d4a0 2024-05-09T00:48:37.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.426+0000 7f0342332b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f032c1b1600 con 0x5644ea21a6a0 2024-05-09T00:48:37.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.426+0000 7f0330eb7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644ea187160 0x5644ea224710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:37.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.426+0000 7f0330eb7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644ea187160 0x5644ea224710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38776/0 (socket says 172.21.15.5:38776) 2024-05-09T00:48:37.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.426+0000 7f0330eb7700 1 -- 172.21.15.5:0/955215357 learned_addr learned my addr 172.21.15.5:0/955215357 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:37.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.426+0000 7f03316b8700 1 --2- 172.21.15.5:0/955215357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644ea09d4a0 0x5644ea2241d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:37.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.426+0000 7f0331eb9700 1 --2- 172.21.15.5:0/955215357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644ea21a6a0 0x5644ea224c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:37.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.427+0000 7f0331eb9700 1 -- 172.21.15.5:0/955215357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644ea09d4a0 msgr2=0x5644ea2241d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:37.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.427+0000 7f0331eb9700 1 --2- 172.21.15.5:0/955215357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644ea09d4a0 0x5644ea2241d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:37.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.427+0000 7f0331eb9700 1 -- 172.21.15.5:0/955215357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644ea187160 msgr2=0x5644ea224710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:37.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.427+0000 7f0331eb9700 1 --2- 172.21.15.5:0/955215357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644ea187160 0x5644ea224710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:37.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.427+0000 7f0331eb9700 1 -- 172.21.15.5:0/955215357 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644ea223fe0 con 0x5644ea21a6a0 2024-05-09T00:48:37.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.427+0000 7f0331eb9700 1 --2- 172.21.15.5:0/955215357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644ea21a6a0 0x5644ea224c50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x5644ea2210c0 tx=0x7f03240092e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d216ef32772236d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:37.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.427+0000 7f0316ffd700 1 -- 172.21.15.5:0/955215357 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0324003c00 con 0x5644ea21a6a0 2024-05-09T00:48:37.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.428+0000 7f0316ffd700 1 -- 172.21.15.5:0/955215357 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0324003da0 con 0x5644ea21a6a0 2024-05-09T00:48:37.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.428+0000 7f0316ffd700 1 -- 172.21.15.5:0/955215357 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0324026980 con 0x5644ea21a6a0 2024-05-09T00:48:37.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.429+0000 7f0342332b80 1 -- 172.21.15.5:0/955215357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644ea21a6a0 msgr2=0x5644ea224c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:37.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.429+0000 7f0342332b80 1 --2- 172.21.15.5:0/955215357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644ea21a6a0 0x5644ea224c50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x5644ea2210c0 tx=0x7f03240092e0 comp rx=0 tx=0).stop 2024-05-09T00:48:37.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.429+0000 7f0342332b80 1 -- 172.21.15.5:0/955215357 shutdown_connections 2024-05-09T00:48:37.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.429+0000 7f0342332b80 1 --2- 172.21.15.5:0/955215357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644ea21a6a0 0x5644ea224c50 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:37.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.429+0000 7f0342332b80 1 --2- 172.21.15.5:0/955215357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644ea09d4a0 0x5644ea2241d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:37.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.429+0000 7f0342332b80 1 --2- 172.21.15.5:0/955215357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644ea187160 0x5644ea224710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:37.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.429+0000 7f0342332b80 1 -- 172.21.15.5:0/955215357 >> 172.21.15.5:0/955215357 conn(0x5644ea0fd6a0 msgr2=0x5644ea222b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:37.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.429+0000 7f0342332b80 1 -- 172.21.15.5:0/955215357 shutdown_connections 2024-05-09T00:48:37.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.429+0000 7f0342332b80 1 -- 172.21.15.5:0/955215357 wait complete. 2024-05-09T00:48:37.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.430+0000 7f0342332b80 1 -- start start 2024-05-09T00:48:37.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.430+0000 7f0342332b80 1 Processor -- start 2024-05-09T00:48:37.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.430+0000 7f0342332b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644ea09d4a0 0x5644ea22c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:37.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.431+0000 7f0342332b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644ea187160 0x5644ea22c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:37.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.431+0000 7f0342332b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644ea21a6a0 0x5644ea23ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:37.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.431+0000 7f0331eb9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644ea21a6a0 0x5644ea23ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:37.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.431+0000 7f0331eb9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644ea21a6a0 0x5644ea23ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38782/0 (socket says 172.21.15.5:38782) 2024-05-09T00:48:37.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.431+0000 7f0331eb9700 1 -- 172.21.15.5:0/1650898456 learned_addr learned my addr 172.21.15.5:0/1650898456 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:37.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.431+0000 7f0342332b80 1 -- 172.21.15.5:0/1650898456 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f032c1b23a0 con 0x5644ea21a6a0 2024-05-09T00:48:37.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.431+0000 7f0342332b80 1 -- 172.21.15.5:0/1650898456 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f032c1b1cd0 con 0x5644ea187160 2024-05-09T00:48:37.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.431+0000 7f0342332b80 1 -- 172.21.15.5:0/1650898456 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f032c1aeed0 con 0x5644ea09d4a0 2024-05-09T00:48:37.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.431+0000 7f03316b8700 1 --2- 172.21.15.5:0/1650898456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644ea09d4a0 0x5644ea22c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:37.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.431+0000 7f0342332b80 10 client.0 _ll_register_callbacks cb 0x5644e9fae6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:48:37.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.431+0000 7f0331eb9700 1 -- 172.21.15.5:0/1650898456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644ea09d4a0 msgr2=0x5644ea22c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:37.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.431+0000 7f0331eb9700 1 --2- 172.21.15.5:0/1650898456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644ea09d4a0 0x5644ea22c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:37.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.432+0000 7f0331eb9700 1 -- 172.21.15.5:0/1650898456 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644ea187160 msgr2=0x5644ea22c9b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:48:37.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.432+0000 7f0331eb9700 1 --2- 172.21.15.5:0/1650898456 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644ea187160 0x5644ea22c9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:37.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.432+0000 7f0331eb9700 1 -- 172.21.15.5:0/1650898456 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644ea23b350 con 0x5644ea21a6a0 2024-05-09T00:48:37.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.432+0000 7f0331eb9700 1 --2- 172.21.15.5:0/1650898456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644ea21a6a0 0x5644ea23ae10 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f032400b0d0 tx=0x7f03240075c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3291217fe218e62b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:37.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.432+0000 7f0314ff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:48:37.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.432+0000 7f0314ff9700 1 -- 172.21.15.5:0/1650898456 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0324007ab0 con 0x5644ea21a6a0 2024-05-09T00:48:37.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.432+0000 7f0314ff9700 1 -- 172.21.15.5:0/1650898456 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f0324007c50 con 0x5644ea21a6a0 2024-05-09T00:48:37.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.432+0000 7f0342332b80 1 -- 172.21.15.5:0/1650898456 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0324003da0 con 0x5644ea21a6a0 2024-05-09T00:48:37.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.433+0000 7f0314ff9700 1 -- 172.21.15.5:0/1650898456 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0324035050 con 0x5644ea21a6a0 2024-05-09T00:48:37.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.433+0000 7f0342332b80 10 client.15600 fetch_fsmap learned FSMap version 29 2024-05-09T00:48:37.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.433+0000 7f0342332b80 1 -- 172.21.15.5:0/1650898456 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5644ea29e050 con 0x5644ea21a6a0 2024-05-09T00:48:37.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.433+0000 7f0314ff9700 1 -- 172.21.15.5:0/1650898456 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0324026ab0 con 0x5644ea21a6a0 2024-05-09T00:48:37.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.434+0000 7f0314ff9700 1 -- 172.21.15.5:0/1650898456 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f0324011070 con 0x5644ea21a6a0 2024-05-09T00:48:37.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.434+0000 7f0342332b80 10 client.15600 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:48:37.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.434+0000 7f0342332b80 10 client.15600 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-05-09T00:48:37.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.434+0000 7f0342332b80 20 client.15600 populate_metadata read hostname 'smithi005' 2024-05-09T00:48:37.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.435+0000 7f0342332b80 1 --2- 172.21.15.5:0/1650898456 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x5644ea29e600 0x5644ea2a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:37.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.435+0000 7f0342332b80 4 client.15600 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:48:37.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.435+0000 7f0342332b80 1 -- 172.21.15.5:0/1650898456 --> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f0324003da0 con 0x5644ea29e600 2024-05-09T00:48:37.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.435+0000 7f03316b8700 1 --2- 172.21.15.5:0/1650898456 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x5644ea29e600 0x5644ea2a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:37.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.435+0000 7f03316b8700 1 --2- 172.21.15.5:0/1650898456 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x5644ea29e600 0x5644ea2a0a20 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7766c8281c241648 server_cookie=cfb74b70d69a0738 in_seq=0 out_seq=0 2024-05-09T00:48:37.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.435+0000 7f0314ff9700 10 client.15600 ms_handle_connect on v2:172.21.15.131:6838/2748669631 2024-05-09T00:48:37.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.436+0000 7f0314ff9700 1 -- 172.21.15.5:0/1650898456 <== mds.2 v2:172.21.15.131:6838/2748669631 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f0324003da0 con 0x5644ea29e600 2024-05-09T00:48:37.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.436+0000 7f0314ff9700 10 client.15600 handle_command_reply: tid=0 2024-05-09T00:48:37.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.436+0000 7f0342332b80 1 client.15600 shutdown 2024-05-09T00:48:37.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.436+0000 7f0342332b80 1 -- 172.21.15.5:0/1650898456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644ea21a6a0 msgr2=0x5644ea23ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:37.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.436+0000 7f0342332b80 1 --2- 172.21.15.5:0/1650898456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644ea21a6a0 0x5644ea23ae10 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f032400b0d0 tx=0x7f03240075c0 comp rx=0 tx=0).stop 2024-05-09T00:48:37.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.436+0000 7f0342332b80 1 -- 172.21.15.5:0/1650898456 shutdown_connections 2024-05-09T00:48:37.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.436+0000 7f0342332b80 1 --2- 172.21.15.5:0/1650898456 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x5644ea29e600 0x5644ea2a0a20 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:37.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.436+0000 7f0342332b80 1 --2- 172.21.15.5:0/1650898456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644ea21a6a0 0x5644ea23ae10 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:37.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.436+0000 7f0342332b80 1 --2- 172.21.15.5:0/1650898456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5644ea09d4a0 0x5644ea22c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:37.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.436+0000 7f0342332b80 1 --2- 172.21.15.5:0/1650898456 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644ea187160 0x5644ea22c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:37.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.436+0000 7f0342332b80 1 -- 172.21.15.5:0/1650898456 >> 172.21.15.5:0/1650898456 conn(0x5644ea0fd6a0 msgr2=0x5644ea2241b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:37.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.436+0000 7f0314ff9700 0 client.15600 ms_handle_reset on v2:172.21.15.131:6838/2748669631 2024-05-09T00:48:37.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.437+0000 7f0342332b80 1 -- 172.21.15.5:0/1650898456 shutdown_connections 2024-05-09T00:48:37.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.437+0000 7f0342332b80 1 -- 172.21.15.5:0/1650898456 wait complete. 2024-05-09T00:48:37.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.437+0000 7f0342332b80 20 client.15600 trim_cache size 0 max 16384 2024-05-09T00:48:37.439 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-09T00:48:37.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.438+0000 7f033391c700 1 -- 172.21.15.5:0/2722794658 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f03180b2980 msgr2=0x7f03180b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:37.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.438+0000 7f033391c700 1 --2- 172.21.15.5:0/2722794658 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f03180b2980 0x7f03180b4e90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f032c103930 tx=0x7f031c009260 comp rx=0 tx=0).stop 2024-05-09T00:48:37.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.438+0000 7f033391c700 1 -- 172.21.15.5:0/2722794658 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f032c1045e0 msgr2=0x7f032c075ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:37.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.438+0000 7f033391c700 1 --2- 172.21.15.5:0/2722794658 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f032c1045e0 0x7f032c075ac0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f032800d950 tx=0x7f032800dc60 comp rx=0 tx=0).stop 2024-05-09T00:48:37.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.438+0000 7f033391c700 1 -- 172.21.15.5:0/2722794658 shutdown_connections 2024-05-09T00:48:37.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.438+0000 7f033391c700 1 --2- 172.21.15.5:0/2722794658 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f03180b2980 0x7f03180b4e90 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:37.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.438+0000 7f033391c700 1 --2- 172.21.15.5:0/2722794658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f032c109860 0x7f032c076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:37.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.438+0000 7f033391c700 1 --2- 172.21.15.5:0/2722794658 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f032c1045e0 0x7f032c075ac0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:37.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.438+0000 7f033391c700 1 --2- 172.21.15.5:0/2722794658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f032c108f10 0x7f032c076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:37.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.438+0000 7f033391c700 1 -- 172.21.15.5:0/2722794658 >> 172.21.15.5:0/2722794658 conn(0x7f032c0ffe70 msgr2=0x7f032c108650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:37.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.439+0000 7f033391c700 1 -- 172.21.15.5:0/2722794658 shutdown_connections 2024-05-09T00:48:37.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:37.439+0000 7f033391c700 1 -- 172.21.15.5:0/2722794658 wait complete. 2024-05-09T00:48:37.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.505+0000 7f05b7c3b640 1 -- 172.21.15.5:0/3204195184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05b0108230 msgr2=0x7f05b010a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:37.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.505+0000 7f05b7c3b640 1 --2- 172.21.15.5:0/3204195184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05b0108230 0x7f05b010a650 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f05a800adb0 tx=0x7f05a802a690 comp rx=0 tx=0).stop 2024-05-09T00:48:37.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.505+0000 7f05b7c3b640 1 -- 172.21.15.5:0/3204195184 shutdown_connections 2024-05-09T00:48:37.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.505+0000 7f05b7c3b640 1 --2- 172.21.15.5:0/3204195184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05b0108230 0x7f05b010a650 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:37.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.505+0000 7f05b7c3b640 1 --2- 172.21.15.5:0/3204195184 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05b0103530 0x7f05b0103940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:37.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.505+0000 7f05b7c3b640 1 --2- 172.21.15.5:0/3204195184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05b01022f0 0x7f05b0102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:37.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.505+0000 7f05b7c3b640 1 -- 172.21.15.5:0/3204195184 >> 172.21.15.5:0/3204195184 conn(0x7f05b00fd4c0 msgr2=0x7f05b00ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:37.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.506+0000 7f05b7c3b640 1 -- 172.21.15.5:0/3204195184 shutdown_connections 2024-05-09T00:48:37.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.506+0000 7f05b7c3b640 1 -- 172.21.15.5:0/3204195184 wait complete. 2024-05-09T00:48:37.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.507+0000 7f05b7c3b640 1 Processor -- start 2024-05-09T00:48:37.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.507+0000 7f05b7c3b640 1 -- start start 2024-05-09T00:48:37.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.507+0000 7f05b7c3b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05b01022f0 0x7f05b01980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:37.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.507+0000 7f05b7c3b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05b0103530 0x7f05b0198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:37.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.507+0000 7f05b7c3b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05b0108230 0x7f05b019f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:37.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.507+0000 7f05b7c3b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f05b010d580 con 0x7f05b0108230 2024-05-09T00:48:37.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.507+0000 7f05b7c3b640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f05b010d400 con 0x7f05b01022f0 2024-05-09T00:48:37.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.507+0000 7f05b7c3b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f05b010d700 con 0x7f05b0103530 2024-05-09T00:48:37.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.508+0000 7f05b59d7640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05b01022f0 0x7f05b01980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:37.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.508+0000 7f05b59d7640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05b01022f0 0x7f05b01980e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:35506/0 (socket says 172.21.15.5:35506) 2024-05-09T00:48:37.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.508+0000 7f05b59d7640 1 -- 172.21.15.5:0/1822057936 learned_addr learned my addr 172.21.15.5:0/1822057936 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:37.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.508+0000 7f05b61d8640 1 --2- 172.21.15.5:0/1822057936 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05b0108230 0x7f05b019f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:37.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.508+0000 7f05b51d6640 1 --2- 172.21.15.5:0/1822057936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05b0103530 0x7f05b0198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:37.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.508+0000 7f05b59d7640 1 -- 172.21.15.5:0/1822057936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05b0103530 msgr2=0x7f05b0198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:37.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.508+0000 7f05b59d7640 1 --2- 172.21.15.5:0/1822057936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05b0103530 0x7f05b0198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:37.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.508+0000 7f05b59d7640 1 -- 172.21.15.5:0/1822057936 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05b0108230 msgr2=0x7f05b019f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:37.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.508+0000 7f05b59d7640 1 --2- 172.21.15.5:0/1822057936 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05b0108230 0x7f05b019f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:37.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.508+0000 7f05b59d7640 1 -- 172.21.15.5:0/1822057936 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05b019fc20 con 0x7f05b01022f0 2024-05-09T00:48:37.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.508+0000 7f05b59d7640 1 --2- 172.21.15.5:0/1822057936 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05b01022f0 0x7f05b01980e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f05ac00df70 tx=0x7f05ac009f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:37.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.508+0000 7f05a6ffd640 1 -- 172.21.15.5:0/1822057936 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ac01a070 con 0x7f05b01022f0 2024-05-09T00:48:37.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.509+0000 7f05a6ffd640 1 -- 172.21.15.5:0/1822057936 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05ac00d230 con 0x7f05b01022f0 2024-05-09T00:48:37.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.509+0000 7f05b7c3b640 1 -- 172.21.15.5:0/1822057936 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05b019ff10 con 0x7f05b01022f0 2024-05-09T00:48:37.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.509+0000 7f05a6ffd640 1 -- 172.21.15.5:0/1822057936 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ac00c420 con 0x7f05b01022f0 2024-05-09T00:48:37.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.509+0000 7f05b7c3b640 1 -- 172.21.15.5:0/1822057936 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05b01a0310 con 0x7f05b01022f0 2024-05-09T00:48:37.512 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.512+0000 7f05bebcb740 1 Processor -- start 2024-05-09T00:48:37.512 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.512+0000 7f05bebcb740 1 -- start start 2024-05-09T00:48:37.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.512+0000 7f05bebcb740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dacbae01b0 0x55dacbe8cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:37.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.512+0000 7f05bebcb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dacbab72f0 0x55dacbe8d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:37.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.512+0000 7f05bebcb740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dacbabf4d0 0x55dacbe94190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:37.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.512+0000 7f05bebcb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dacbde87c0 con 0x55dacbab72f0 2024-05-09T00:48:37.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.512+0000 7f05bebcb740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55dacbd840e0 con 0x55dacbae01b0 2024-05-09T00:48:37.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.512+0000 7f05bebcb740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55dacbde4780 con 0x55dacbabf4d0 2024-05-09T00:48:37.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.513+0000 7f05b61d8640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dacbae01b0 0x55dacbe8cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:37.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.513+0000 7f05b61d8640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dacbae01b0 0x55dacbe8cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:35512/0 (socket says 172.21.15.5:35512) 2024-05-09T00:48:37.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.513+0000 7f05b61d8640 1 -- 172.21.15.5:0/1770912544 learned_addr learned my addr 172.21.15.5:0/1770912544 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:37.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.513+0000 7f05b51d6640 1 --2- 172.21.15.5:0/1770912544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dacbabf4d0 0x55dacbe94190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:37.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.513+0000 7f05b61d8640 1 -- 172.21.15.5:0/1770912544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dacbabf4d0 msgr2=0x55dacbe94190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:37.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.513+0000 7f05b61d8640 1 --2- 172.21.15.5:0/1770912544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dacbabf4d0 0x55dacbe94190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:37.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.513+0000 7f05b61d8640 1 -- 172.21.15.5:0/1770912544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dacbab72f0 msgr2=0x55dacbe8d290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:48:37.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.513+0000 7f05b61d8640 1 --2- 172.21.15.5:0/1770912544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dacbab72f0 0x55dacbe8d290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:37.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.513+0000 7f05b61d8640 1 -- 172.21.15.5:0/1770912544 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dacbdad7c0 con 0x55dacbae01b0 2024-05-09T00:48:37.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.513+0000 7f05b51d6640 1 --2- 172.21.15.5:0/1770912544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dacbabf4d0 0x55dacbe94190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:48:37.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.513+0000 7f05b61d8640 1 --2- 172.21.15.5:0/1770912544 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dacbae01b0 0x55dacbe8cd10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f05a8034000 tx=0x7f05a8004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68470d5a806268d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:37.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.513+0000 7f05a6ffd640 1 -- 172.21.15.5:0/1822057936 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f05ac01d070 con 0x7f05b01022f0 2024-05-09T00:48:37.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.513+0000 7f059bfff640 1 -- 172.21.15.5:0/1770912544 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a8031400 con 0x55dacbae01b0 2024-05-09T00:48:37.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.514+0000 7f059bfff640 1 -- 172.21.15.5:0/1770912544 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05a8003e20 con 0x55dacbae01b0 2024-05-09T00:48:37.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.514+0000 7f059bfff640 1 -- 172.21.15.5:0/1770912544 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a8004570 con 0x55dacbae01b0 2024-05-09T00:48:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.514+0000 7f05bebcb740 1 -- 172.21.15.5:0/1770912544 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dacbae01b0 msgr2=0x55dacbe8cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.514+0000 7f05bebcb740 1 --2- 172.21.15.5:0/1770912544 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dacbae01b0 0x55dacbe8cd10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f05a8034000 tx=0x7f05a8004ab0 comp rx=0 tx=0).stop 2024-05-09T00:48:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.515+0000 7f05a6ffd640 1 --2- 172.21.15.5:0/1822057936 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f059c0b28e0 0x7f059c0b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.515+0000 7f05bebcb740 1 -- 172.21.15.5:0/1770912544 shutdown_connections 2024-05-09T00:48:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.515+0000 7f05bebcb740 1 --2- 172.21.15.5:0/1770912544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dacbabf4d0 0x55dacbe94190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.515+0000 7f05bebcb740 1 --2- 172.21.15.5:0/1770912544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dacbab72f0 0x55dacbe8d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.515+0000 7f05bebcb740 1 --2- 172.21.15.5:0/1770912544 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dacbae01b0 0x55dacbe8cd10 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.515+0000 7f05bebcb740 1 -- 172.21.15.5:0/1770912544 >> 172.21.15.5:0/1770912544 conn(0x55dacbad4300 msgr2=0x55dacbe886a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.515+0000 7f05a6ffd640 1 -- 172.21.15.5:0/1822057936 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f05ac0e1570 con 0x7f05b01022f0 2024-05-09T00:48:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.515+0000 7f05bebcb740 1 -- 172.21.15.5:0/1770912544 shutdown_connections 2024-05-09T00:48:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.515+0000 7f05b61d8640 1 --2- 172.21.15.5:0/1822057936 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f059c0b28e0 0x7f059c0b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.515+0000 7f05bebcb740 1 -- 172.21.15.5:0/1770912544 wait complete. 2024-05-09T00:48:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.515+0000 7f05bebcb740 1 -- start start 2024-05-09T00:48:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.516+0000 7f05bebcb740 1 Processor -- start 2024-05-09T00:48:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.516+0000 7f05b61d8640 1 --2- 172.21.15.5:0/1822057936 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f059c0b28e0 0x7f059c0b4df0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f05a8006010 tx=0x7f05a803e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:37.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.516+0000 7f05bebcb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dacbab72f0 0x55dacbe96ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:37.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.516+0000 7f05bebcb740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dacbabf4d0 0x55dacbe96ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:37.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.516+0000 7f05bebcb740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dacbae01b0 0x55dacbe9df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:37.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.517+0000 7f05bebcb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dacbe94b80 con 0x55dacbab72f0 2024-05-09T00:48:37.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.517+0000 7f05bebcb740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55dacbd6f9c0 con 0x55dacbabf4d0 2024-05-09T00:48:37.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.517+0000 7f05bebcb740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55dacbe94d00 con 0x55dacbae01b0 2024-05-09T00:48:37.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.517+0000 7f05bebcb740 10 client.0 _ll_register_callbacks cb 0x55dacbd9c020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:48:37.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.517+0000 7f05b59d7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dacbab72f0 0x55dacbe96ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:37.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.517+0000 7f05b59d7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dacbab72f0 0x55dacbe96ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38798/0 (socket says 172.21.15.5:38798) 2024-05-09T00:48:37.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.517+0000 7f05b59d7640 1 -- 172.21.15.5:0/694227025 learned_addr learned my addr 172.21.15.5:0/694227025 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:37.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.517+0000 7f05b51d6640 1 --2- 172.21.15.5:0/694227025 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dacbabf4d0 0x55dacbe96ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:37.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.517+0000 7f05b61d8640 1 --2- 172.21.15.5:0/694227025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dacbae01b0 0x55dacbe9df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:37.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.517+0000 7f05b59d7640 1 -- 172.21.15.5:0/694227025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dacbae01b0 msgr2=0x55dacbe9df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:37.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.517+0000 7f05b59d7640 1 --2- 172.21.15.5:0/694227025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dacbae01b0 0x55dacbe9df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:37.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.517+0000 7f05b59d7640 1 -- 172.21.15.5:0/694227025 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dacbabf4d0 msgr2=0x55dacbe96ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:37.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.517+0000 7f05b59d7640 1 --2- 172.21.15.5:0/694227025 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dacbabf4d0 0x55dacbe96ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:37.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.517+0000 7f05b59d7640 1 -- 172.21.15.5:0/694227025 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dacbd961f0 con 0x55dacbab72f0 2024-05-09T00:48:37.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.517+0000 7f05b59d7640 1 --2- 172.21.15.5:0/694227025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dacbab72f0 0x55dacbe96ab0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f05ac0a8800 tx=0x7f05ac0a8750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4aaeace6f92f5d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:37.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.517+0000 7f05997fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:48:37.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.518+0000 7f05997fa640 1 -- 172.21.15.5:0/694227025 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ac0aceb0 con 0x55dacbab72f0 2024-05-09T00:48:37.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.518+0000 7f05997fa640 1 -- 172.21.15.5:0/694227025 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f05ac0165e0 con 0x55dacbab72f0 2024-05-09T00:48:37.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.518+0000 7f05bebcb740 1 -- 172.21.15.5:0/694227025 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05a8003e20 con 0x55dacbab72f0 2024-05-09T00:48:37.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.518+0000 7f05997fa640 1 -- 172.21.15.5:0/694227025 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f05ac0b1960 con 0x55dacbab72f0 2024-05-09T00:48:37.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.518+0000 7f05bebcb740 10 client.15609 fetch_fsmap learned FSMap version 29 2024-05-09T00:48:37.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.518+0000 7f05bebcb740 1 -- 172.21.15.5:0/694227025 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55dacbb4b790 con 0x55dacbab72f0 2024-05-09T00:48:37.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.518+0000 7f05997fa640 1 -- 172.21.15.5:0/694227025 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05a8003e20 con 0x55dacbab72f0 2024-05-09T00:48:37.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.519+0000 7f05997fa640 1 -- 172.21.15.5:0/694227025 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f05ac0cb0e0 con 0x55dacbab72f0 2024-05-09T00:48:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.519+0000 7f05bebcb740 10 client.15609 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:48:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.519+0000 7f05bebcb740 10 client.15609 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:48:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.519+0000 7f05bebcb740 20 client.15609 populate_metadata read hostname 'smithi005' 2024-05-09T00:48:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.519+0000 7f05bebcb740 1 --2- 172.21.15.5:0/694227025 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55dacba9ed80 0x55dacbea4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.519+0000 7f05bebcb740 4 client.15609 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:48:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.519+0000 7f05bebcb740 1 -- 172.21.15.5:0/694227025 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dacbdf63d0 con 0x55dacba9ed80 2024-05-09T00:48:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.519+0000 7f05b51d6640 1 --2- 172.21.15.5:0/694227025 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55dacba9ed80 0x55dacbea4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.520+0000 7f05997fa640 10 client.15609 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:48:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.520+0000 7f05b51d6640 1 --2- 172.21.15.5:0/694227025 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55dacba9ed80 0x55dacbea4390 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adfb97b6095ebc43 server_cookie=504d43fac0207086 in_seq=0 out_seq=0 2024-05-09T00:48:37.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.521+0000 7f05997fa640 1 -- 172.21.15.5:0/694227025 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dacbdf63d0 con 0x55dacba9ed80 2024-05-09T00:48:37.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.521+0000 7f05997fa640 10 client.15609 handle_command_reply: tid=0 2024-05-09T00:48:37.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.521+0000 7f05bebcb740 1 client.15609 shutdown 2024-05-09T00:48:37.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.521+0000 7f05bebcb740 1 -- 172.21.15.5:0/694227025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dacbab72f0 msgr2=0x55dacbe96ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:37.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.521+0000 7f05bebcb740 1 --2- 172.21.15.5:0/694227025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dacbab72f0 0x55dacbe96ab0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f05ac0a8800 tx=0x7f05ac0a8750 comp rx=0 tx=0).stop 2024-05-09T00:48:37.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.521+0000 7f05bebcb740 1 -- 172.21.15.5:0/694227025 shutdown_connections 2024-05-09T00:48:37.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.521+0000 7f05bebcb740 1 --2- 172.21.15.5:0/694227025 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55dacba9ed80 0x55dacbea4390 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:37.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.521+0000 7f05997fa640 0 client.15609 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:48:37.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.521+0000 7f05bebcb740 1 --2- 172.21.15.5:0/694227025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dacbae01b0 0x55dacbe9df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:37.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.521+0000 7f05bebcb740 1 --2- 172.21.15.5:0/694227025 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dacbabf4d0 0x55dacbe96ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:37.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.521+0000 7f05bebcb740 1 --2- 172.21.15.5:0/694227025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dacbab72f0 0x55dacbe96ab0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:37.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.521+0000 7f05bebcb740 1 -- 172.21.15.5:0/694227025 >> 172.21.15.5:0/694227025 conn(0x55dacbad4300 msgr2=0x55dacbe8b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:37.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.521+0000 7f05bebcb740 1 -- 172.21.15.5:0/694227025 shutdown_connections 2024-05-09T00:48:37.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.521+0000 7f05bebcb740 1 -- 172.21.15.5:0/694227025 wait complete. 2024-05-09T00:48:37.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.521+0000 7f05bebcb740 20 client.15609 trim_cache size 0 max 16384 2024-05-09T00:48:37.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.538+0000 7f05bebcb740 1 Processor -- start 2024-05-09T00:48:37.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.538+0000 7f05bebcb740 1 -- start start 2024-05-09T00:48:37.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.538+0000 7f05bebcb740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dacbab72f0 0x55dacbe95740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:37.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.538+0000 7f05bebcb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dacbabf4d0 0x55dacbe88640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:37.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.538+0000 7f05bebcb740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dacbae01b0 0x55dacbe96680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:37.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.538+0000 7f05bebcb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f05b01a9180 con 0x55dacbabf4d0 2024-05-09T00:48:37.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.538+0000 7f05bebcb740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f05b01aacc0 con 0x55dacbab72f0 2024-05-09T00:48:37.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.538+0000 7f05bebcb740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f05b01a9850 con 0x55dacbae01b0 2024-05-09T00:48:37.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.538+0000 7f05b51d6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dacbabf4d0 0x55dacbe88640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:37.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.538+0000 7f05b51d6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dacbabf4d0 0x55dacbe88640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38802/0 (socket says 172.21.15.5:38802) 2024-05-09T00:48:37.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.538+0000 7f05b51d6640 1 -- 172.21.15.5:0/1250442698 learned_addr learned my addr 172.21.15.5:0/1250442698 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:37.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.538+0000 7f05b59d7640 1 --2- 172.21.15.5:0/1250442698 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dacbab72f0 0x55dacbe95740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:37.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.539+0000 7f05b61d8640 1 --2- 172.21.15.5:0/1250442698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dacbae01b0 0x55dacbe96680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:37.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.539+0000 7f05b59d7640 1 -- 172.21.15.5:0/1250442698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dacbae01b0 msgr2=0x55dacbe96680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:37.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.539+0000 7f05b59d7640 1 --2- 172.21.15.5:0/1250442698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dacbae01b0 0x55dacbe96680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:37.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.539+0000 7f05b59d7640 1 -- 172.21.15.5:0/1250442698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dacbabf4d0 msgr2=0x55dacbe88640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:37.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.539+0000 7f05b59d7640 1 --2- 172.21.15.5:0/1250442698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dacbabf4d0 0x55dacbe88640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:37.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.539+0000 7f05b59d7640 1 -- 172.21.15.5:0/1250442698 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dacbbc25f0 con 0x55dacbab72f0 2024-05-09T00:48:37.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.539+0000 7f05b51d6640 1 --2- 172.21.15.5:0/1250442698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dacbabf4d0 0x55dacbe88640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:48:37.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.539+0000 7f05b59d7640 1 --2- 172.21.15.5:0/1250442698 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dacbab72f0 0x55dacbe95740 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f05ac0058c0 tx=0x7f05ac016f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70674d7eda861f41 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:37.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.539+0000 7f059b7fe640 1 -- 172.21.15.5:0/1250442698 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ac0aceb0 con 0x55dacbab72f0 2024-05-09T00:48:37.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.539+0000 7f059b7fe640 1 -- 172.21.15.5:0/1250442698 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05ac0c2070 con 0x55dacbab72f0 2024-05-09T00:48:37.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.539+0000 7f059b7fe640 1 -- 172.21.15.5:0/1250442698 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ac0e1070 con 0x55dacbab72f0 2024-05-09T00:48:37.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.540+0000 7f05bebcb740 1 -- 172.21.15.5:0/1250442698 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dacbab72f0 msgr2=0x55dacbe95740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:37.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.540+0000 7f05bebcb740 1 --2- 172.21.15.5:0/1250442698 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dacbab72f0 0x55dacbe95740 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f05ac0058c0 tx=0x7f05ac016f60 comp rx=0 tx=0).stop 2024-05-09T00:48:37.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.541+0000 7f05bebcb740 1 -- 172.21.15.5:0/1250442698 shutdown_connections 2024-05-09T00:48:37.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.541+0000 7f05bebcb740 1 --2- 172.21.15.5:0/1250442698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dacbae01b0 0x55dacbe96680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:37.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.541+0000 7f05bebcb740 1 --2- 172.21.15.5:0/1250442698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dacbabf4d0 0x55dacbe88640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:37.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.541+0000 7f05bebcb740 1 --2- 172.21.15.5:0/1250442698 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dacbab72f0 0x55dacbe95740 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:37.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.541+0000 7f05bebcb740 1 -- 172.21.15.5:0/1250442698 >> 172.21.15.5:0/1250442698 conn(0x55dacbad4300 msgr2=0x55dacbe87f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:37.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.541+0000 7f05bebcb740 1 -- 172.21.15.5:0/1250442698 shutdown_connections 2024-05-09T00:48:37.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.541+0000 7f05bebcb740 1 -- 172.21.15.5:0/1250442698 wait complete. 2024-05-09T00:48:37.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.541+0000 7f05bebcb740 1 -- start start 2024-05-09T00:48:37.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.541+0000 7f05bebcb740 1 Processor -- start 2024-05-09T00:48:37.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.542+0000 7f05bebcb740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dacbab72f0 0x55dacbe9a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:37.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.542+0000 7f05bebcb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dacbabf4d0 0x55dacbe9ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:37.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.542+0000 7f05bebcb740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dacbae01b0 0x55dacbe9bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:37.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.542+0000 7f05bebcb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f05b01aa5f0 con 0x55dacbabf4d0 2024-05-09T00:48:37.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.542+0000 7f05bebcb740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f05b01a9f20 con 0x55dacbae01b0 2024-05-09T00:48:37.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.542+0000 7f05bebcb740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f05b01a7160 con 0x55dacbab72f0 2024-05-09T00:48:37.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.542+0000 7f05bebcb740 10 client.0 _ll_register_callbacks cb 0x7f05ac0a49c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:48:37.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.542+0000 7f05b51d6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dacbabf4d0 0x55dacbe9ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:37.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.542+0000 7f05b59d7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dacbab72f0 0x55dacbe9a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:37.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.542+0000 7f05b51d6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dacbabf4d0 0x55dacbe9ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38812/0 (socket says 172.21.15.5:38812) 2024-05-09T00:48:37.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.542+0000 7f05b51d6640 1 -- 172.21.15.5:0/804526407 learned_addr learned my addr 172.21.15.5:0/804526407 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:37.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.542+0000 7f05b61d8640 1 --2- 172.21.15.5:0/804526407 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dacbae01b0 0x55dacbe9bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:37.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.542+0000 7f05b59d7640 1 -- 172.21.15.5:0/804526407 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dacbae01b0 msgr2=0x55dacbe9bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:37.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.542+0000 7f05b59d7640 1 --2- 172.21.15.5:0/804526407 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dacbae01b0 0x55dacbe9bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:37.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.542+0000 7f05b59d7640 1 -- 172.21.15.5:0/804526407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dacbabf4d0 msgr2=0x55dacbe9ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:37.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.542+0000 7f05b59d7640 1 --2- 172.21.15.5:0/804526407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dacbabf4d0 0x55dacbe9ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:37.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.542+0000 7f05b59d7640 1 -- 172.21.15.5:0/804526407 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dacbdaced0 con 0x55dacbab72f0 2024-05-09T00:48:37.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.542+0000 7f05b51d6640 1 --2- 172.21.15.5:0/804526407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dacbabf4d0 0x55dacbe9ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:48:37.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.543+0000 7f05b59d7640 1 --2- 172.21.15.5:0/804526407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dacbab72f0 0x55dacbe9a8a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f05ac0058c0 tx=0x7f05ac00d3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c5a7212fa871e6a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:37.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.543+0000 7f05997fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-09T00:48:37.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.543+0000 7f05997fa640 1 -- 172.21.15.5:0/804526407 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ac0aceb0 con 0x55dacbab72f0 2024-05-09T00:48:37.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.543+0000 7f05997fa640 1 -- 172.21.15.5:0/804526407 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f05ac0cb0d0 con 0x55dacbab72f0 2024-05-09T00:48:37.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.543+0000 7f05bebcb740 1 -- 172.21.15.5:0/804526407 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05ac0c2070 con 0x55dacbab72f0 2024-05-09T00:48:37.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.543+0000 7f05997fa640 1 -- 172.21.15.5:0/804526407 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f05ac08d050 con 0x55dacbab72f0 2024-05-09T00:48:37.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.543+0000 7f05bebcb740 10 client.25351 fetch_fsmap learned FSMap version 29 2024-05-09T00:48:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.543+0000 7f05bebcb740 1 -- 172.21.15.5:0/804526407 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55dacbdaced0 con 0x55dacbab72f0 2024-05-09T00:48:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.543+0000 7f05997fa640 1 -- 172.21.15.5:0/804526407 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05ac0c2070 con 0x55dacbab72f0 2024-05-09T00:48:37.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.544+0000 7f05997fa640 1 -- 172.21.15.5:0/804526407 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f05ac0a77d0 con 0x55dacbab72f0 2024-05-09T00:48:37.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.544+0000 7f05bebcb740 10 client.25351 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:48:37.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.544+0000 7f05bebcb740 10 client.25351 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:48:37.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.544+0000 7f05bebcb740 20 client.25351 populate_metadata read hostname 'smithi005' 2024-05-09T00:48:37.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.544+0000 7f05bebcb740 1 --2- 172.21.15.5:0/804526407 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55dacba741c0 0x55dacbf0e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:37.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.544+0000 7f05bebcb740 4 client.25351 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:48:37.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.544+0000 7f05bebcb740 1 -- 172.21.15.5:0/804526407 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dacbdf0240 con 0x55dacba741c0 2024-05-09T00:48:37.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.544+0000 7f05b51d6640 1 --2- 172.21.15.5:0/804526407 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55dacba741c0 0x55dacbf0e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:37.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.545+0000 7f05997fa640 10 client.25351 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:48:37.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.545+0000 7f05b51d6640 1 --2- 172.21.15.5:0/804526407 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55dacba741c0 0x55dacbf0e6a0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1717179d7defe78 server_cookie=31b4cb58e2520903 in_seq=0 out_seq=0 2024-05-09T00:48:37.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.549+0000 7f05997fa640 1 -- 172.21.15.5:0/804526407 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55dacbdf0240 con 0x55dacba741c0 2024-05-09T00:48:37.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.549+0000 7f05997fa640 10 client.25351 handle_command_reply: tid=0 2024-05-09T00:48:37.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.549+0000 7f05bebcb740 1 client.25351 shutdown 2024-05-09T00:48:37.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.549+0000 7f05bebcb740 1 -- 172.21.15.5:0/804526407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dacbab72f0 msgr2=0x55dacbe9a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:37.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.549+0000 7f05bebcb740 1 --2- 172.21.15.5:0/804526407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dacbab72f0 0x55dacbe9a8a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f05ac0058c0 tx=0x7f05ac00d3e0 comp rx=0 tx=0).stop 2024-05-09T00:48:37.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.549+0000 7f05bebcb740 1 -- 172.21.15.5:0/804526407 shutdown_connections 2024-05-09T00:48:37.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.549+0000 7f05bebcb740 1 --2- 172.21.15.5:0/804526407 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55dacba741c0 0x55dacbf0e6a0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:37.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.549+0000 7f05997fa640 0 client.25351 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:48:37.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.549+0000 7f05bebcb740 1 --2- 172.21.15.5:0/804526407 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dacbae01b0 0x55dacbe9bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:37.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.549+0000 7f05bebcb740 1 --2- 172.21.15.5:0/804526407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dacbabf4d0 0x55dacbe9ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:37.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.549+0000 7f05bebcb740 1 --2- 172.21.15.5:0/804526407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dacbab72f0 0x55dacbe9a8a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:37.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.549+0000 7f05bebcb740 1 -- 172.21.15.5:0/804526407 >> 172.21.15.5:0/804526407 conn(0x55dacbad4300 msgr2=0x55dacbe7e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:37.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.549+0000 7f05bebcb740 1 -- 172.21.15.5:0/804526407 shutdown_connections 2024-05-09T00:48:37.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.549+0000 7f05bebcb740 1 -- 172.21.15.5:0/804526407 wait complete. 2024-05-09T00:48:37.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.549+0000 7f05bebcb740 20 client.25351 trim_cache size 0 max 16384 2024-05-09T00:48:37.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-09T00:48:37.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-09T00:48:37.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-09T00:48:37.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.550+0000 7f05b7c3b640 1 -- 172.21.15.5:0/1822057936 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f059c0b28e0 msgr2=0x7f059c0b4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:37.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.550+0000 7f05b7c3b640 1 --2- 172.21.15.5:0/1822057936 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f059c0b28e0 0x7f059c0b4df0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f05a8006010 tx=0x7f05a803e040 comp rx=0 tx=0).stop 2024-05-09T00:48:37.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.550+0000 7f05b7c3b640 1 -- 172.21.15.5:0/1822057936 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05b01022f0 msgr2=0x7f05b01980e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:37.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.550+0000 7f05b7c3b640 1 --2- 172.21.15.5:0/1822057936 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05b01022f0 0x7f05b01980e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f05ac00df70 tx=0x7f05ac009f30 comp rx=0 tx=0).stop 2024-05-09T00:48:37.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.550+0000 7f05b7c3b640 1 -- 172.21.15.5:0/1822057936 shutdown_connections 2024-05-09T00:48:37.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.550+0000 7f05b7c3b640 1 --2- 172.21.15.5:0/1822057936 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f059c0b28e0 0x7f059c0b4df0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:37.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.550+0000 7f05b7c3b640 1 --2- 172.21.15.5:0/1822057936 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05b0108230 0x7f05b019f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:37.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.550+0000 7f05b7c3b640 1 --2- 172.21.15.5:0/1822057936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05b0103530 0x7f05b0198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:37.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.550+0000 7f05b7c3b640 1 --2- 172.21.15.5:0/1822057936 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05b01022f0 0x7f05b01980e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:37.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.550+0000 7f05b7c3b640 1 -- 172.21.15.5:0/1822057936 >> 172.21.15.5:0/1822057936 conn(0x7f05b00fd4c0 msgr2=0x7f05b0104210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:37.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.550+0000 7f05b7c3b640 1 -- 172.21.15.5:0/1822057936 shutdown_connections 2024-05-09T00:48:37.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:37.550+0000 7f05b7c3b640 1 -- 172.21.15.5:0/1822057936 wait complete. 2024-05-09T00:48:37.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:48:37 smithi131 ceph-mon[34547]: pgmap v439: 97 pgs: 97 active+clean; 670 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T00:48:37.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:48:37 smithi157 ceph-mon[34294]: pgmap v439: 97 pgs: 97 active+clean; 670 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T00:48:38.315 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T00:48:38.658 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:48:39.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:48:39 smithi005 ceph-mon[30365]: pgmap v440: 97 pgs: 97 active+clean; 670 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-09T00:48:39.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:48:39 smithi131 ceph-mon[34547]: pgmap v440: 97 pgs: 97 active+clean; 670 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-09T00:48:39.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:48:39 smithi157 ceph-mon[34294]: pgmap v440: 97 pgs: 97 active+clean; 670 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-09T00:48:40.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.605+0000 7fa6dd3ce700 1 -- 172.21.15.5:0/2529667068 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6d81034c0 msgr2=0x7fa6d81038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:40.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.605+0000 7fa6dd3ce700 1 --2- 172.21.15.5:0/2529667068 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6d81034c0 0x7fa6d81038d0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fa6c8009990 tx=0x7fa6c8009ca0 comp rx=0 tx=0).stop 2024-05-09T00:48:40.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.605+0000 7fa6dd3ce700 1 -- 172.21.15.5:0/2529667068 shutdown_connections 2024-05-09T00:48:40.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.605+0000 7fa6dd3ce700 1 --2- 172.21.15.5:0/2529667068 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6d8108740 0x7fa6d81100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:40.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.605+0000 7fa6dd3ce700 1 --2- 172.21.15.5:0/2529667068 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6d81034c0 0x7fa6d81038d0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:40.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.605+0000 7fa6dd3ce700 1 --2- 172.21.15.5:0/2529667068 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6d8107df0 0x7fa6d8108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:40.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.605+0000 7fa6dd3ce700 1 -- 172.21.15.5:0/2529667068 >> 172.21.15.5:0/2529667068 conn(0x7fa6d80fedc0 msgr2=0x7fa6d81011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:40.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.606+0000 7fa6dd3ce700 1 -- 172.21.15.5:0/2529667068 shutdown_connections 2024-05-09T00:48:40.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.606+0000 7fa6dd3ce700 1 -- 172.21.15.5:0/2529667068 wait complete. 2024-05-09T00:48:40.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.607+0000 7fa6dd3ce700 1 Processor -- start 2024-05-09T00:48:40.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.608+0000 7fa6dd3ce700 1 -- start start 2024-05-09T00:48:40.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.608+0000 7fa6dd3ce700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6d81034c0 0x7fa6d819ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:40.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.608+0000 7fa6dd3ce700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6d8107df0 0x7fa6d819b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:40.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.608+0000 7fa6dd3ce700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6d8108740 0x7fa6d819f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:40.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.608+0000 7fa6dd3ce700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d8114510 con 0x7fa6d8107df0 2024-05-09T00:48:40.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.608+0000 7fa6dd3ce700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d8114390 con 0x7fa6d8108740 2024-05-09T00:48:40.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.608+0000 7fa6dd3ce700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d8114690 con 0x7fa6d81034c0 2024-05-09T00:48:40.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.608+0000 7fa6d77fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6d8108740 0x7fa6d819f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:40.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.608+0000 7fa6d77fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6d8108740 0x7fa6d819f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:35568/0 (socket says 172.21.15.5:35568) 2024-05-09T00:48:40.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.609+0000 7fa6d77fe700 1 -- 172.21.15.5:0/1349103339 learned_addr learned my addr 172.21.15.5:0/1349103339 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:40.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.609+0000 7fa6d6ffd700 1 --2- 172.21.15.5:0/1349103339 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6d81034c0 0x7fa6d819ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:40.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.609+0000 7fa6d67fc700 1 --2- 172.21.15.5:0/1349103339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6d8107df0 0x7fa6d819b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:40.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.609+0000 7fa6d6ffd700 1 -- 172.21.15.5:0/1349103339 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6d8108740 msgr2=0x7fa6d819f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:40.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.609+0000 7fa6d6ffd700 1 --2- 172.21.15.5:0/1349103339 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6d8108740 0x7fa6d819f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:40.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.609+0000 7fa6d6ffd700 1 -- 172.21.15.5:0/1349103339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6d8107df0 msgr2=0x7fa6d819b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:40.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.609+0000 7fa6d6ffd700 1 --2- 172.21.15.5:0/1349103339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6d8107df0 0x7fa6d819b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:40.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.609+0000 7fa6d6ffd700 1 -- 172.21.15.5:0/1349103339 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6d819fd30 con 0x7fa6d81034c0 2024-05-09T00:48:40.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.609+0000 7fa6d77fe700 1 --2- 172.21.15.5:0/1349103339 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6d8108740 0x7fa6d819f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:48:40.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.610+0000 7fa6d6ffd700 1 --2- 172.21.15.5:0/1349103339 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6d81034c0 0x7fa6d819ae60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa6c80098e0 tx=0x7fa6c80057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:40.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.610+0000 7fa6bffff700 1 -- 172.21.15.5:0/1349103339 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c8026070 con 0x7fa6d81034c0 2024-05-09T00:48:40.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.610+0000 7fa6bffff700 1 -- 172.21.15.5:0/1349103339 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6c8003680 con 0x7fa6d81034c0 2024-05-09T00:48:40.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.610+0000 7fa6dd3ce700 1 -- 172.21.15.5:0/1349103339 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6d819ffc0 con 0x7fa6d81034c0 2024-05-09T00:48:40.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.610+0000 7fa6bffff700 1 -- 172.21.15.5:0/1349103339 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c800fb10 con 0x7fa6d81034c0 2024-05-09T00:48:40.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.611+0000 7fa6dd3ce700 1 -- 172.21.15.5:0/1349103339 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6d81a7940 con 0x7fa6d81034c0 2024-05-09T00:48:40.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.615+0000 7fa6bffff700 1 -- 172.21.15.5:0/1349103339 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fa6c8029070 con 0x7fa6d81034c0 2024-05-09T00:48:40.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.616+0000 7fa6bffff700 1 --2- 172.21.15.5:0/1349103339 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fa6c00b2980 0x7fa6c00b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:40.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.617+0000 7fa6d67fc700 1 --2- 172.21.15.5:0/1349103339 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fa6c00b2980 0x7fa6c00b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:40.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.617+0000 7fa6bffff700 1 -- 172.21.15.5:0/1349103339 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fa6c80e2a40 con 0x7fa6d81034c0 2024-05-09T00:48:40.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.617+0000 7fa6ebde4b80 1 Processor -- start 2024-05-09T00:48:40.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.617+0000 7fa6ebde4b80 1 -- start start 2024-05-09T00:48:40.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.617+0000 7fa6d67fc700 1 --2- 172.21.15.5:0/1349103339 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fa6c00b2980 0x7fa6c00b4e90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa6d8107d70 tx=0x7fa6cc0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:40.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.618+0000 7fa6ebde4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5610a3184d00 0x5610a328bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:40.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.618+0000 7fa6ebde4b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610a310a4a0 0x5610a328c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:40.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.618+0000 7fa6ebde4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610a32901b0 0x5610a32905c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:40.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.618+0000 7fa6ebde4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5610a31aa0a0 con 0x5610a3184d00 2024-05-09T00:48:40.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.618+0000 7fa6ebde4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5610a314f9a0 con 0x5610a32901b0 2024-05-09T00:48:40.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.618+0000 7fa6d67fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610a310a4a0 0x5610a328c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:40.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.618+0000 7fa6d6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5610a3184d00 0x5610a328bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:40.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.618+0000 7fa6d67fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610a310a4a0 0x5610a328c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:33600/0 (socket says 172.21.15.5:33600) 2024-05-09T00:48:40.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.618+0000 7fa6d67fc700 1 -- 172.21.15.5:0/1908510725 learned_addr learned my addr 172.21.15.5:0/1908510725 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:40.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.618+0000 7fa6ebde4b80 1 -- 172.21.15.5:0/1908510725 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5610a31a6060 con 0x5610a310a4a0 2024-05-09T00:48:40.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.618+0000 7fa6d77fe700 1 --2- 172.21.15.5:0/1908510725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610a32901b0 0x5610a32905c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:40.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.619+0000 7fa6d67fc700 1 -- 172.21.15.5:0/1908510725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610a32901b0 msgr2=0x5610a32905c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:40.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.619+0000 7fa6d67fc700 1 --2- 172.21.15.5:0/1908510725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610a32901b0 0x5610a32905c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:40.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.619+0000 7fa6d67fc700 1 -- 172.21.15.5:0/1908510725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5610a3184d00 msgr2=0x5610a328bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:40.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.619+0000 7fa6d67fc700 1 --2- 172.21.15.5:0/1908510725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5610a3184d00 0x5610a328bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:40.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.619+0000 7fa6d67fc700 1 -- 172.21.15.5:0/1908510725 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610a3140500 con 0x5610a310a4a0 2024-05-09T00:48:40.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.619+0000 7fa6d67fc700 1 --2- 172.21.15.5:0/1908510725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610a310a4a0 0x5610a328c050 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc004500 tx=0x7fa6cc013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3cdf3eb3559ebc20 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:40.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.619+0000 7fa6bd7fa700 1 -- 172.21.15.5:0/1908510725 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc024070 con 0x5610a310a4a0 2024-05-09T00:48:40.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.619+0000 7fa6bd7fa700 1 -- 172.21.15.5:0/1908510725 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6cc015070 con 0x5610a310a4a0 2024-05-09T00:48:40.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.619+0000 7fa6bd7fa700 1 -- 172.21.15.5:0/1908510725 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc011830 con 0x5610a310a4a0 2024-05-09T00:48:40.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.620+0000 7fa6ebde4b80 1 -- 172.21.15.5:0/1908510725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610a310a4a0 msgr2=0x5610a328c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:40.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.620+0000 7fa6ebde4b80 1 --2- 172.21.15.5:0/1908510725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610a310a4a0 0x5610a328c050 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc004500 tx=0x7fa6cc013000 comp rx=0 tx=0).stop 2024-05-09T00:48:40.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.621+0000 7fa6ebde4b80 1 -- 172.21.15.5:0/1908510725 shutdown_connections 2024-05-09T00:48:40.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.621+0000 7fa6ebde4b80 1 --2- 172.21.15.5:0/1908510725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610a32901b0 0x5610a32905c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:40.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.621+0000 7fa6ebde4b80 1 --2- 172.21.15.5:0/1908510725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5610a3184d00 0x5610a328bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:40.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.621+0000 7fa6ebde4b80 1 --2- 172.21.15.5:0/1908510725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610a310a4a0 0x5610a328c050 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:40.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.621+0000 7fa6ebde4b80 1 -- 172.21.15.5:0/1908510725 >> 172.21.15.5:0/1908510725 conn(0x5610a310df70 msgr2=0x5610a3282220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:40.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.621+0000 7fa6ebde4b80 1 -- 172.21.15.5:0/1908510725 shutdown_connections 2024-05-09T00:48:40.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.621+0000 7fa6ebde4b80 1 -- 172.21.15.5:0/1908510725 wait complete. 2024-05-09T00:48:40.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.621+0000 7fa6ebde4b80 1 -- start start 2024-05-09T00:48:40.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.622+0000 7fa6ebde4b80 1 Processor -- start 2024-05-09T00:48:40.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.622+0000 7fa6ebde4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610a310a4a0 0x5610a32954c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:40.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.622+0000 7fa6ebde4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5610a3184d00 0x5610a3295a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:40.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.622+0000 7fa6ebde4b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610a32901b0 0x5610a3295f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:40.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.622+0000 7fa6ebde4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5610a3290ed0 con 0x5610a3184d00 2024-05-09T00:48:40.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.622+0000 7fa6ebde4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5610a3144b20 con 0x5610a310a4a0 2024-05-09T00:48:40.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.622+0000 7fa6d67fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5610a3184d00 0x5610a3295a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:40.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.622+0000 7fa6ebde4b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5610a3298460 con 0x5610a32901b0 2024-05-09T00:48:40.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.622+0000 7fa6d67fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5610a3184d00 0x5610a3295a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38844/0 (socket says 172.21.15.5:38844) 2024-05-09T00:48:40.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.622+0000 7fa6d67fc700 1 -- 172.21.15.5:0/1546721311 learned_addr learned my addr 172.21.15.5:0/1546721311 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:40.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.622+0000 7fa6ebde4b80 10 client.0 _ll_register_callbacks cb 0x5610a301b6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:48:40.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.623+0000 7fa6d6ffd700 1 --2- 172.21.15.5:0/1546721311 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610a310a4a0 0x5610a32954c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:40.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.623+0000 7fa6d77fe700 1 --2- 172.21.15.5:0/1546721311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610a32901b0 0x5610a3295f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:40.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.623+0000 7fa6d6ffd700 1 -- 172.21.15.5:0/1546721311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610a32901b0 msgr2=0x5610a3295f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:40.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.623+0000 7fa6d6ffd700 1 --2- 172.21.15.5:0/1546721311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610a32901b0 0x5610a3295f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:40.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.623+0000 7fa6d6ffd700 1 -- 172.21.15.5:0/1546721311 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5610a3184d00 msgr2=0x5610a3295a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:40.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.623+0000 7fa6d6ffd700 1 --2- 172.21.15.5:0/1546721311 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5610a3184d00 0x5610a3295a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:40.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.623+0000 7fa6d6ffd700 1 -- 172.21.15.5:0/1546721311 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610a3025490 con 0x5610a310a4a0 2024-05-09T00:48:40.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.623+0000 7fa6d67fc700 1 --2- 172.21.15.5:0/1546721311 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5610a3184d00 0x5610a3295a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:48:40.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.623+0000 7fa6d6ffd700 1 --2- 172.21.15.5:0/1546721311 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610a310a4a0 0x5610a32954c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa6c80a9ff0 tx=0x7fa6c800f5b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9dcc8ef84a4ceabd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:40.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.623+0000 7fa6baff5700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-09T00:48:40.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.624+0000 7fa6baff5700 1 -- 172.21.15.5:0/1546721311 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c80c4060 con 0x5610a310a4a0 2024-05-09T00:48:40.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.624+0000 7fa6baff5700 1 -- 172.21.15.5:0/1546721311 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fa6c80aa4d0 con 0x5610a310a4a0 2024-05-09T00:48:40.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.624+0000 7fa6ebde4b80 1 -- 172.21.15.5:0/1546721311 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6cc015070 con 0x5610a310a4a0 2024-05-09T00:48:40.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.624+0000 7fa6baff5700 1 -- 172.21.15.5:0/1546721311 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa6c80aa060 con 0x5610a310a4a0 2024-05-09T00:48:40.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.624+0000 7fa6ebde4b80 10 client.25313 fetch_fsmap learned FSMap version 29 2024-05-09T00:48:40.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.624+0000 7fa6ebde4b80 1 -- 172.21.15.5:0/1546721311 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5610a3287b30 con 0x5610a310a4a0 2024-05-09T00:48:40.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.624+0000 7fa6baff5700 1 -- 172.21.15.5:0/1546721311 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6c80cb050 con 0x5610a310a4a0 2024-05-09T00:48:40.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.626+0000 7fa6baff5700 1 -- 172.21.15.5:0/1546721311 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fa6c80b2a30 con 0x5610a310a4a0 2024-05-09T00:48:40.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.626+0000 7fa6ebde4b80 10 client.25313 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:48:40.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.626+0000 7fa6ebde4b80 10 client.25313 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-05-09T00:48:40.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.626+0000 7fa6ebde4b80 20 client.25313 populate_metadata read hostname 'smithi005' 2024-05-09T00:48:40.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.626+0000 7fa6ebde4b80 1 --2- 172.21.15.5:0/1546721311 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x5610a328a3d0 0x5610a328a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:40.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.626+0000 7fa6ebde4b80 4 client.25313 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:48:40.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.626+0000 7fa6ebde4b80 1 -- 172.21.15.5:0/1546721311 --> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa6cc015070 con 0x5610a328a3d0 2024-05-09T00:48:40.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.626+0000 7fa6d67fc700 1 --2- 172.21.15.5:0/1546721311 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x5610a328a3d0 0x5610a328a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:40.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.627+0000 7fa6baff5700 10 client.25313 ms_handle_connect on v2:172.21.15.157:6836/744863009 2024-05-09T00:48:40.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.627+0000 7fa6d67fc700 1 --2- 172.21.15.5:0/1546721311 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x5610a328a3d0 0x5610a328a810 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6d5dca3a9b9a1cd6 server_cookie=5970b109852c6b96 in_seq=0 out_seq=0 2024-05-09T00:48:40.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.627+0000 7fa6baff5700 1 -- 172.21.15.5:0/1546721311 <== mds.1 v2:172.21.15.157:6836/744863009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fa6cc015070 con 0x5610a328a3d0 2024-05-09T00:48:40.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.628+0000 7fa6baff5700 10 client.25313 handle_command_reply: tid=0 2024-05-09T00:48:40.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.628+0000 7fa6ebde4b80 1 client.25313 shutdown 2024-05-09T00:48:40.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.628+0000 7fa6ebde4b80 1 -- 172.21.15.5:0/1546721311 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610a310a4a0 msgr2=0x5610a32954c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:40.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.628+0000 7fa6ebde4b80 1 --2- 172.21.15.5:0/1546721311 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610a310a4a0 0x5610a32954c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa6c80a9ff0 tx=0x7fa6c800f5b0 comp rx=0 tx=0).stop 2024-05-09T00:48:40.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.628+0000 7fa6ebde4b80 1 -- 172.21.15.5:0/1546721311 shutdown_connections 2024-05-09T00:48:40.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.628+0000 7fa6ebde4b80 1 --2- 172.21.15.5:0/1546721311 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x5610a328a3d0 0x5610a328a810 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:40.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.628+0000 7fa6ebde4b80 1 --2- 172.21.15.5:0/1546721311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610a32901b0 0x5610a3295f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:40.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.628+0000 7fa6ebde4b80 1 --2- 172.21.15.5:0/1546721311 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610a310a4a0 0x5610a32954c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:40.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.628+0000 7fa6ebde4b80 1 --2- 172.21.15.5:0/1546721311 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5610a3184d00 0x5610a3295a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:40.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.628+0000 7fa6ebde4b80 1 -- 172.21.15.5:0/1546721311 >> 172.21.15.5:0/1546721311 conn(0x5610a310df70 msgr2=0x5610a32863f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:40.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.628+0000 7fa6baff5700 0 client.25313 ms_handle_reset on v2:172.21.15.157:6836/744863009 2024-05-09T00:48:40.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.628+0000 7fa6ebde4b80 1 -- 172.21.15.5:0/1546721311 shutdown_connections 2024-05-09T00:48:40.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.628+0000 7fa6ebde4b80 1 -- 172.21.15.5:0/1546721311 wait complete. 2024-05-09T00:48:40.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.628+0000 7fa6ebde4b80 20 client.25313 trim_cache size 0 max 16384 2024-05-09T00:48:40.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.658+0000 7fa6ebde4b80 1 Processor -- start 2024-05-09T00:48:40.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.659+0000 7fa6ebde4b80 1 -- start start 2024-05-09T00:48:40.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.659+0000 7fa6ebde4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610a310a4a0 0x5610a327ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:40.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.659+0000 7fa6ebde4b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610a3184d00 0x5610a32a0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:40.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.659+0000 7fa6ebde4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5610a32901b0 0x5610a32a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:40.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.659+0000 7fa6ebde4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d81ac990 con 0x5610a32901b0 2024-05-09T00:48:40.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.659+0000 7fa6ebde4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d81ae4d0 con 0x5610a310a4a0 2024-05-09T00:48:40.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.659+0000 7fa6ebde4b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d81ad060 con 0x5610a3184d00 2024-05-09T00:48:40.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.659+0000 7fa6d77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5610a32901b0 0x5610a32a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:40.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.659+0000 7fa6d67fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610a3184d00 0x5610a32a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:40.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.659+0000 7fa6d77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5610a32901b0 0x5610a32a0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38850/0 (socket says 172.21.15.5:38850) 2024-05-09T00:48:40.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.659+0000 7fa6d77fe700 1 -- 172.21.15.5:0/547268899 learned_addr learned my addr 172.21.15.5:0/547268899 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:40.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.659+0000 7fa6d6ffd700 1 --2- 172.21.15.5:0/547268899 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610a310a4a0 0x5610a327ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:40.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.660+0000 7fa6d6ffd700 1 -- 172.21.15.5:0/547268899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610a3184d00 msgr2=0x5610a32a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:40.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.660+0000 7fa6d6ffd700 1 --2- 172.21.15.5:0/547268899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610a3184d00 0x5610a32a0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:40.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.660+0000 7fa6d6ffd700 1 -- 172.21.15.5:0/547268899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5610a32901b0 msgr2=0x5610a32a0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:40.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.660+0000 7fa6d6ffd700 1 --2- 172.21.15.5:0/547268899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5610a32901b0 0x5610a32a0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:40.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.660+0000 7fa6d6ffd700 1 -- 172.21.15.5:0/547268899 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610a3292160 con 0x5610a310a4a0 2024-05-09T00:48:40.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.660+0000 7fa6d6ffd700 1 --2- 172.21.15.5:0/547268899 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610a310a4a0 0x5610a327ca20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa6c80a9d60 tx=0x7fa6c800ff90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c88865fdc3bbfbc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:40.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.660+0000 7fa6bcff9700 1 -- 172.21.15.5:0/547268899 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c8088070 con 0x5610a310a4a0 2024-05-09T00:48:40.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.661+0000 7fa6bcff9700 1 -- 172.21.15.5:0/547268899 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6c80ae410 con 0x5610a310a4a0 2024-05-09T00:48:40.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.661+0000 7fa6bcff9700 1 -- 172.21.15.5:0/547268899 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c80cb040 con 0x5610a310a4a0 2024-05-09T00:48:40.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.662+0000 7fa6ebde4b80 1 -- 172.21.15.5:0/547268899 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610a310a4a0 msgr2=0x5610a327ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:40.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.662+0000 7fa6ebde4b80 1 --2- 172.21.15.5:0/547268899 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610a310a4a0 0x5610a327ca20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa6c80a9d60 tx=0x7fa6c800ff90 comp rx=0 tx=0).stop 2024-05-09T00:48:40.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.662+0000 7fa6ebde4b80 1 -- 172.21.15.5:0/547268899 shutdown_connections 2024-05-09T00:48:40.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.662+0000 7fa6ebde4b80 1 --2- 172.21.15.5:0/547268899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5610a32901b0 0x5610a32a0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:40.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.662+0000 7fa6ebde4b80 1 --2- 172.21.15.5:0/547268899 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610a310a4a0 0x5610a327ca20 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:40.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.662+0000 7fa6ebde4b80 1 --2- 172.21.15.5:0/547268899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610a3184d00 0x5610a32a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:40.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.662+0000 7fa6ebde4b80 1 -- 172.21.15.5:0/547268899 >> 172.21.15.5:0/547268899 conn(0x5610a310df70 msgr2=0x5610a3294420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:40.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.662+0000 7fa6ebde4b80 1 -- 172.21.15.5:0/547268899 shutdown_connections 2024-05-09T00:48:40.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.662+0000 7fa6ebde4b80 1 -- 172.21.15.5:0/547268899 wait complete. 2024-05-09T00:48:40.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.663+0000 7fa6ebde4b80 1 -- start start 2024-05-09T00:48:40.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.663+0000 7fa6ebde4b80 1 Processor -- start 2024-05-09T00:48:40.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.663+0000 7fa6ebde4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5610a310a4a0 0x5610a32a4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:40.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.664+0000 7fa6d6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5610a310a4a0 0x5610a32a4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:40.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.664+0000 7fa6d6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5610a310a4a0 0x5610a32a4ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38852/0 (socket says 172.21.15.5:38852) 2024-05-09T00:48:40.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.664+0000 7fa6d6ffd700 1 -- 172.21.15.5:0/429846508 learned_addr learned my addr 172.21.15.5:0/429846508 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:40.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.664+0000 7fa6ebde4b80 1 --2- 172.21.15.5:0/429846508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610a3184d00 0x5610a32a5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:40.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.664+0000 7fa6ebde4b80 1 --2- 172.21.15.5:0/429846508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610a32901b0 0x5610a330a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:40.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.664+0000 7fa6ebde4b80 1 -- 172.21.15.5:0/429846508 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d81ade00 con 0x5610a310a4a0 2024-05-09T00:48:40.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.664+0000 7fa6ebde4b80 1 -- 172.21.15.5:0/429846508 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d81ad730 con 0x5610a3184d00 2024-05-09T00:48:40.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.664+0000 7fa6ebde4b80 1 -- 172.21.15.5:0/429846508 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d81aa950 con 0x5610a32901b0 2024-05-09T00:48:40.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.664+0000 7fa6d67fc700 1 --2- 172.21.15.5:0/429846508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610a3184d00 0x5610a32a5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:40.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.664+0000 7fa6d77fe700 1 --2- 172.21.15.5:0/429846508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610a32901b0 0x5610a330a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:40.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.665+0000 7fa6d6ffd700 1 -- 172.21.15.5:0/429846508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610a32901b0 msgr2=0x5610a330a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:40.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.665+0000 7fa6d6ffd700 1 --2- 172.21.15.5:0/429846508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610a32901b0 0x5610a330a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:40.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.665+0000 7fa6ebde4b80 10 client.0 _ll_register_callbacks cb 0x5610a301b6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:48:40.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.665+0000 7fa6d6ffd700 1 -- 172.21.15.5:0/429846508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610a3184d00 msgr2=0x5610a32a5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:40.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.665+0000 7fa6d6ffd700 1 --2- 172.21.15.5:0/429846508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610a3184d00 0x5610a32a5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:40.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.665+0000 7fa6d6ffd700 1 -- 172.21.15.5:0/429846508 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610a330a880 con 0x5610a310a4a0 2024-05-09T00:48:40.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.665+0000 7fa6ebde4b80 1 -- 172.21.15.5:0/429846508 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6c80ae410 con 0x5610a310a4a0 2024-05-09T00:48:40.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.665+0000 7fa6d6ffd700 1 --2- 172.21.15.5:0/429846508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5610a310a4a0 0x5610a32a4ac0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa6c80a6090 tx=0x7fa6c8009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dd5dc7d4fa6b781 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:40.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.665+0000 7fa6baff5700 10 client.15633 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:48:40.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.666+0000 7fa6baff5700 1 -- 172.21.15.5:0/429846508 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c8088020 con 0x5610a310a4a0 2024-05-09T00:48:40.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.666+0000 7fa6baff5700 1 -- 172.21.15.5:0/429846508 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fa6c80a92c0 con 0x5610a310a4a0 2024-05-09T00:48:40.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.666+0000 7fa6baff5700 1 -- 172.21.15.5:0/429846508 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa6c80cb040 con 0x5610a310a4a0 2024-05-09T00:48:40.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.666+0000 7fa6ebde4b80 10 client.15633 fetch_fsmap learned FSMap version 29 2024-05-09T00:48:40.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.666+0000 7fa6ebde4b80 1 -- 172.21.15.5:0/429846508 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({fsmap=29}) -- 0x5610a330a880 con 0x5610a310a4a0 2024-05-09T00:48:40.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.667+0000 7fa6baff5700 1 -- 172.21.15.5:0/429846508 <== mon.0 v2:172.21.15.5:3300/0 4 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fa6c80a8d30 con 0x5610a310a4a0 2024-05-09T00:48:40.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.667+0000 7fa6ebde4b80 10 client.15633 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:48:40.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.667+0000 7fa6ebde4b80 10 client.15633 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-05-09T00:48:40.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.667+0000 7fa6ebde4b80 20 client.15633 populate_metadata read hostname 'smithi005' 2024-05-09T00:48:40.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.667+0000 7fa6ebde4b80 1 --2- 172.21.15.5:0/429846508 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x5610a330bb80 0x5610a330dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:40.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.667+0000 7fa6ebde4b80 4 client.15633 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:48:40.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.667+0000 7fa6ebde4b80 1 -- 172.21.15.5:0/429846508 --> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5610a31b7a80 con 0x5610a330bb80 2024-05-09T00:48:40.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.667+0000 7fa6d67fc700 1 --2- 172.21.15.5:0/429846508 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x5610a330bb80 0x5610a330dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:40.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.668+0000 7fa6baff5700 10 client.15633 ms_handle_connect on v2:172.21.15.157:6836/744863009 2024-05-09T00:48:40.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.668+0000 7fa6d67fc700 1 --2- 172.21.15.5:0/429846508 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x5610a330bb80 0x5610a330dfa0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7122bb6479f054d8 server_cookie=9aa33bbe3ac42abb in_seq=0 out_seq=0 2024-05-09T00:48:40.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.668+0000 7fa6baff5700 1 -- 172.21.15.5:0/429846508 <== mds.1 v2:172.21.15.157:6836/744863009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5610a31b7a80 con 0x5610a330bb80 2024-05-09T00:48:40.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.668+0000 7fa6baff5700 10 client.15633 handle_command_reply: tid=0 2024-05-09T00:48:40.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.668+0000 7fa6ebde4b80 1 client.15633 shutdown 2024-05-09T00:48:40.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.668+0000 7fa6ebde4b80 1 -- 172.21.15.5:0/429846508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5610a310a4a0 msgr2=0x5610a32a4ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:40.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.668+0000 7fa6ebde4b80 1 --2- 172.21.15.5:0/429846508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5610a310a4a0 0x5610a32a4ac0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa6c80a6090 tx=0x7fa6c8009310 comp rx=0 tx=0).stop 2024-05-09T00:48:40.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.668+0000 7fa6ebde4b80 1 -- 172.21.15.5:0/429846508 shutdown_connections 2024-05-09T00:48:40.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.668+0000 7fa6ebde4b80 1 --2- 172.21.15.5:0/429846508 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x5610a330bb80 0x5610a330dfa0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:40.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.668+0000 7fa6ebde4b80 1 --2- 172.21.15.5:0/429846508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5610a32901b0 0x5610a330a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:40.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.668+0000 7fa6ebde4b80 1 --2- 172.21.15.5:0/429846508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5610a310a4a0 0x5610a32a4ac0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:40.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.668+0000 7fa6ebde4b80 1 --2- 172.21.15.5:0/429846508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5610a3184d00 0x5610a32a5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:40.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.668+0000 7fa6ebde4b80 1 -- 172.21.15.5:0/429846508 >> 172.21.15.5:0/429846508 conn(0x5610a310df70 msgr2=0x5610a327c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:40.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.668+0000 7fa6baff5700 0 client.15633 ms_handle_reset on v2:172.21.15.157:6836/744863009 2024-05-09T00:48:40.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.668+0000 7fa6ebde4b80 1 -- 172.21.15.5:0/429846508 shutdown_connections 2024-05-09T00:48:40.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.668+0000 7fa6ebde4b80 1 -- 172.21.15.5:0/429846508 wait complete. 2024-05-09T00:48:40.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.669+0000 7fa6ebde4b80 20 client.15633 trim_cache size 0 max 16384 2024-05-09T00:48:40.674 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-09T00:48:40.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.670+0000 7fa6dd3ce700 1 -- 172.21.15.5:0/1349103339 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fa6c00b2980 msgr2=0x7fa6c00b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:40.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.670+0000 7fa6dd3ce700 1 --2- 172.21.15.5:0/1349103339 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fa6c00b2980 0x7fa6c00b4e90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa6d8107d70 tx=0x7fa6cc0073d0 comp rx=0 tx=0).stop 2024-05-09T00:48:40.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.670+0000 7fa6dd3ce700 1 -- 172.21.15.5:0/1349103339 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6d81034c0 msgr2=0x7fa6d819ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:40.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.670+0000 7fa6dd3ce700 1 --2- 172.21.15.5:0/1349103339 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6d81034c0 0x7fa6d819ae60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa6c80098e0 tx=0x7fa6c80057c0 comp rx=0 tx=0).stop 2024-05-09T00:48:40.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.670+0000 7fa6dd3ce700 1 -- 172.21.15.5:0/1349103339 shutdown_connections 2024-05-09T00:48:40.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.670+0000 7fa6dd3ce700 1 --2- 172.21.15.5:0/1349103339 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fa6c00b2980 0x7fa6c00b4e90 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:40.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.670+0000 7fa6dd3ce700 1 --2- 172.21.15.5:0/1349103339 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6d8108740 0x7fa6d819f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:40.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.670+0000 7fa6dd3ce700 1 --2- 172.21.15.5:0/1349103339 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6d81034c0 0x7fa6d819ae60 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:40.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.670+0000 7fa6dd3ce700 1 --2- 172.21.15.5:0/1349103339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6d8107df0 0x7fa6d819b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:40.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.670+0000 7fa6dd3ce700 1 -- 172.21.15.5:0/1349103339 >> 172.21.15.5:0/1349103339 conn(0x7fa6d80fedc0 msgr2=0x7fa6d81011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:40.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.670+0000 7fa6dd3ce700 1 -- 172.21.15.5:0/1349103339 shutdown_connections 2024-05-09T00:48:40.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:40.670+0000 7fa6dd3ce700 1 -- 172.21.15.5:0/1349103339 wait complete. 2024-05-09T00:48:41.450 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T00:48:41.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:48:41 smithi005 ceph-mon[30365]: pgmap v441: 97 pgs: 97 active+clean; 671 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 511 B/s wr, 11 op/s 2024-05-09T00:48:41.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:48:41 smithi131 ceph-mon[34547]: pgmap v441: 97 pgs: 97 active+clean; 671 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 511 B/s wr, 11 op/s 2024-05-09T00:48:41.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:48:41 smithi157 ceph-mon[34294]: pgmap v441: 97 pgs: 97 active+clean; 671 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 511 B/s wr, 11 op/s 2024-05-09T00:48:41.751 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:48:43.427 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:48:43 smithi005 ceph-mon[30365]: pgmap v442: 97 pgs: 97 active+clean; 671 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T00:48:43.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:48:43 smithi131 ceph-mon[34547]: pgmap v442: 97 pgs: 97 active+clean; 671 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T00:48:43.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:48:43 smithi157 ceph-mon[34294]: pgmap v442: 97 pgs: 97 active+clean; 671 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T00:48:43.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.697+0000 7ff451376700 1 -- 172.21.15.5:0/1872273618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff44c108740 msgr2=0x7ff44c1100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:43.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.697+0000 7ff451376700 1 --2- 172.21.15.5:0/1872273618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff44c108740 0x7ff44c1100a0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ff44400b0d0 tx=0x7ff44400b3e0 comp rx=0 tx=0).stop 2024-05-09T00:48:43.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.697+0000 7ff451376700 1 -- 172.21.15.5:0/1872273618 shutdown_connections 2024-05-09T00:48:43.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.697+0000 7ff451376700 1 --2- 172.21.15.5:0/1872273618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff44c108740 0x7ff44c1100a0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:43.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.697+0000 7ff451376700 1 --2- 172.21.15.5:0/1872273618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff44c1034c0 0x7ff44c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:43.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.697+0000 7ff451376700 1 --2- 172.21.15.5:0/1872273618 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff44c107df0 0x7ff44c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:43.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.697+0000 7ff451376700 1 -- 172.21.15.5:0/1872273618 >> 172.21.15.5:0/1872273618 conn(0x7ff44c0fedc0 msgr2=0x7ff44c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:43.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.697+0000 7ff451376700 1 -- 172.21.15.5:0/1872273618 shutdown_connections 2024-05-09T00:48:43.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.697+0000 7ff451376700 1 -- 172.21.15.5:0/1872273618 wait complete. 2024-05-09T00:48:43.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.698+0000 7ff451376700 1 Processor -- start 2024-05-09T00:48:43.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.698+0000 7ff451376700 1 -- start start 2024-05-09T00:48:43.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.698+0000 7ff451376700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff44c1034c0 0x7ff44c19bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:43.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.699+0000 7ff451376700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff44c107df0 0x7ff44c19c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:43.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.699+0000 7ff451376700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff44c108740 0x7ff44c19c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:43.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.699+0000 7ff44a7fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff44c107df0 0x7ff44c19c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:43.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.699+0000 7ff44b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff44c108740 0x7ff44c19c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:43.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.699+0000 7ff44b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff44c108740 0x7ff44c19c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35460/0 (socket says 172.21.15.5:35460) 2024-05-09T00:48:43.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.699+0000 7ff44a7fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff44c107df0 0x7ff44c19c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:39458/0 (socket says 172.21.15.5:39458) 2024-05-09T00:48:43.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.699+0000 7ff44b7fe700 1 -- 172.21.15.5:0/2360803076 learned_addr learned my addr 172.21.15.5:0/2360803076 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:43.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.699+0000 7ff451376700 1 -- 172.21.15.5:0/2360803076 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff44c113370 con 0x7ff44c108740 2024-05-09T00:48:43.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.699+0000 7ff451376700 1 -- 172.21.15.5:0/2360803076 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff44c1131f0 con 0x7ff44c107df0 2024-05-09T00:48:43.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.699+0000 7ff451376700 1 -- 172.21.15.5:0/2360803076 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff44c1134f0 con 0x7ff44c1034c0 2024-05-09T00:48:43.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.699+0000 7ff44affd700 1 --2- 172.21.15.5:0/2360803076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff44c1034c0 0x7ff44c19bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:43.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.699+0000 7ff44affd700 1 -- 172.21.15.5:0/2360803076 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff44c107df0 msgr2=0x7ff44c19c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:43.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.699+0000 7ff44affd700 1 --2- 172.21.15.5:0/2360803076 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff44c107df0 0x7ff44c19c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:43.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.699+0000 7ff44affd700 1 -- 172.21.15.5:0/2360803076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff44c108740 msgr2=0x7ff44c19c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:43.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.699+0000 7ff44affd700 1 --2- 172.21.15.5:0/2360803076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff44c108740 0x7ff44c19c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:43.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.699+0000 7ff44affd700 1 -- 172.21.15.5:0/2360803076 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff44c1a6f00 con 0x7ff44c1034c0 2024-05-09T00:48:43.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.699+0000 7ff44b7fe700 1 --2- 172.21.15.5:0/2360803076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff44c108740 0x7ff44c19c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:48:43.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.700+0000 7ff44affd700 1 --2- 172.21.15.5:0/2360803076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff44c1034c0 0x7ff44c19bb90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff43c009c50 tx=0x7ff43c009f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:43.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.700+0000 7ff433fff700 1 -- 172.21.15.5:0/2360803076 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff43c01e070 con 0x7ff44c1034c0 2024-05-09T00:48:43.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.700+0000 7ff451376700 1 -- 172.21.15.5:0/2360803076 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff44c1999e0 con 0x7ff44c1034c0 2024-05-09T00:48:43.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.700+0000 7ff451376700 1 -- 172.21.15.5:0/2360803076 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff44c199f70 con 0x7ff44c1034c0 2024-05-09T00:48:43.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.700+0000 7ff433fff700 1 -- 172.21.15.5:0/2360803076 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff43c019410 con 0x7ff44c1034c0 2024-05-09T00:48:43.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.700+0000 7ff433fff700 1 -- 172.21.15.5:0/2360803076 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff43c018610 con 0x7ff44c1034c0 2024-05-09T00:48:43.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.704+0000 7ff433fff700 1 -- 172.21.15.5:0/2360803076 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7ff43c021070 con 0x7ff44c1034c0 2024-05-09T00:48:43.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.705+0000 7ff433fff700 1 --2- 172.21.15.5:0/2360803076 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ff4340b2880 0x7ff4340b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:43.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.705+0000 7ff433fff700 1 -- 172.21.15.5:0/2360803076 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7ff43c0e5490 con 0x7ff44c1034c0 2024-05-09T00:48:43.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.705+0000 7ff44a7fc700 1 --2- 172.21.15.5:0/2360803076 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ff4340b2880 0x7ff4340b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:43.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.706+0000 7ff45fd8cb80 1 Processor -- start 2024-05-09T00:48:43.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.706+0000 7ff44a7fc700 1 --2- 172.21.15.5:0/2360803076 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ff4340b2880 0x7ff4340b4d90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff44c106750 tx=0x7ff440009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:43.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.706+0000 7ff45fd8cb80 1 -- start start 2024-05-09T00:48:43.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.707+0000 7ff45fd8cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c922633d00 0x55c92273c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:43.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.707+0000 7ff45fd8cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c9225b94a0 0x55c92273c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:43.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.707+0000 7ff45fd8cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c922736400 0x55c922736810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:43.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.707+0000 7ff44affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c922633d00 0x55c92273c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:43.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.707+0000 7ff45fd8cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c9226590a0 con 0x55c922633d00 2024-05-09T00:48:43.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.707+0000 7ff44affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c922633d00 0x55c92273c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35474/0 (socket says 172.21.15.5:35474) 2024-05-09T00:48:43.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.707+0000 7ff45fd8cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55c9225fe9a0 con 0x55c9225b94a0 2024-05-09T00:48:43.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.707+0000 7ff44affd700 1 -- 172.21.15.5:0/1227539463 learned_addr learned my addr 172.21.15.5:0/1227539463 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:43.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.707+0000 7ff45fd8cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55c922655060 con 0x55c922736400 2024-05-09T00:48:43.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.708+0000 7ff44b7fe700 1 --2- 172.21.15.5:0/1227539463 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c922736400 0x55c922736810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:43.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.708+0000 7ff44a7fc700 1 --2- 172.21.15.5:0/1227539463 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c9225b94a0 0x55c92273c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:43.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.708+0000 7ff44affd700 1 -- 172.21.15.5:0/1227539463 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c922736400 msgr2=0x55c922736810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:43.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.708+0000 7ff44affd700 1 --2- 172.21.15.5:0/1227539463 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c922736400 0x55c922736810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:43.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.708+0000 7ff44affd700 1 -- 172.21.15.5:0/1227539463 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c9225b94a0 msgr2=0x55c92273c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:43.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.708+0000 7ff44affd700 1 --2- 172.21.15.5:0/1227539463 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c9225b94a0 0x55c92273c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:43.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.708+0000 7ff44affd700 1 -- 172.21.15.5:0/1227539463 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9225ef500 con 0x55c922633d00 2024-05-09T00:48:43.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.709+0000 7ff44affd700 1 --2- 172.21.15.5:0/1227539463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c922633d00 0x55c92273c240 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7ff43c0ac720 tx=0x7ff43c092f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=800550cfb9e869e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:43.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.709+0000 7ff4317fa700 1 -- 172.21.15.5:0/1227539463 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff43c0ad120 con 0x55c922633d00 2024-05-09T00:48:43.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.709+0000 7ff4317fa700 1 -- 172.21.15.5:0/1227539463 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff43c0aca20 con 0x55c922633d00 2024-05-09T00:48:43.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.709+0000 7ff4317fa700 1 -- 172.21.15.5:0/1227539463 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff43c0abee0 con 0x55c922633d00 2024-05-09T00:48:43.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.710+0000 7ff45fd8cb80 1 -- 172.21.15.5:0/1227539463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c922633d00 msgr2=0x55c92273c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:43.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.710+0000 7ff45fd8cb80 1 --2- 172.21.15.5:0/1227539463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c922633d00 0x55c92273c240 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7ff43c0ac720 tx=0x7ff43c092f90 comp rx=0 tx=0).stop 2024-05-09T00:48:43.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.710+0000 7ff45fd8cb80 1 -- 172.21.15.5:0/1227539463 shutdown_connections 2024-05-09T00:48:43.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.710+0000 7ff45fd8cb80 1 --2- 172.21.15.5:0/1227539463 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c922736400 0x55c922736810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:43.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.710+0000 7ff45fd8cb80 1 --2- 172.21.15.5:0/1227539463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c922633d00 0x55c92273c240 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:43.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.710+0000 7ff45fd8cb80 1 --2- 172.21.15.5:0/1227539463 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c9225b94a0 0x55c92273c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:43.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.710+0000 7ff45fd8cb80 1 -- 172.21.15.5:0/1227539463 >> 172.21.15.5:0/1227539463 conn(0x55c9225bcf70 msgr2=0x55c922731220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:43.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.710+0000 7ff45fd8cb80 1 -- 172.21.15.5:0/1227539463 shutdown_connections 2024-05-09T00:48:43.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.710+0000 7ff45fd8cb80 1 -- 172.21.15.5:0/1227539463 wait complete. 2024-05-09T00:48:43.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.711+0000 7ff45fd8cb80 1 -- start start 2024-05-09T00:48:43.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.711+0000 7ff45fd8cb80 1 Processor -- start 2024-05-09T00:48:43.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.711+0000 7ff45fd8cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c9225b94a0 0x55c922740210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:43.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.711+0000 7ff45fd8cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c922736400 0x55c922740750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:43.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.711+0000 7ff45fd8cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c922740c90 0x55c922754a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:43.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.711+0000 7ff45fd8cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c922737100 con 0x55c9225b94a0 2024-05-09T00:48:43.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.711+0000 7ff45fd8cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55c9225f3b20 con 0x55c922740c90 2024-05-09T00:48:43.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.711+0000 7ff45fd8cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55c922755060 con 0x55c922736400 2024-05-09T00:48:43.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.711+0000 7ff45fd8cb80 10 client.0 _ll_register_callbacks cb 0x55c9224ca6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:48:43.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.711+0000 7ff44affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c9225b94a0 0x55c922740210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:43.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.711+0000 7ff44affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c9225b94a0 0x55c922740210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35478/0 (socket says 172.21.15.5:35478) 2024-05-09T00:48:43.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.711+0000 7ff44affd700 1 -- 172.21.15.5:0/1437109935 learned_addr learned my addr 172.21.15.5:0/1437109935 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:43.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.711+0000 7ff44a7fc700 1 --2- 172.21.15.5:0/1437109935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c922736400 0x55c922740750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:43.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.712+0000 7ff44b7fe700 1 --2- 172.21.15.5:0/1437109935 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c922740c90 0x55c922754a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:43.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.712+0000 7ff44affd700 1 -- 172.21.15.5:0/1437109935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c922736400 msgr2=0x55c922740750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:43.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.712+0000 7ff44affd700 1 --2- 172.21.15.5:0/1437109935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c922736400 0x55c922740750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:43.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.712+0000 7ff44affd700 1 -- 172.21.15.5:0/1437109935 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c922740c90 msgr2=0x55c922754a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:43.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.712+0000 7ff44affd700 1 --2- 172.21.15.5:0/1437109935 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c922740c90 0x55c922754a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:43.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.712+0000 7ff44affd700 1 -- 172.21.15.5:0/1437109935 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9224d4490 con 0x55c9225b94a0 2024-05-09T00:48:43.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.712+0000 7ff44a7fc700 1 --2- 172.21.15.5:0/1437109935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c922736400 0x55c922740750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:48:43.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.712+0000 7ff44b7fe700 1 --2- 172.21.15.5:0/1437109935 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c922740c90 0x55c922754a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:48:43.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.712+0000 7ff44affd700 1 --2- 172.21.15.5:0/1437109935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c9225b94a0 0x55c922740210 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ff43c0a8aa0 tx=0x7ff43c00be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2c171d1c92c7351 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:43.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.712+0000 7ff42eff5700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:48:43.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.712+0000 7ff42eff5700 1 -- 172.21.15.5:0/1437109935 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff43c0a88e0 con 0x55c9225b94a0 2024-05-09T00:48:43.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.712+0000 7ff42eff5700 1 -- 172.21.15.5:0/1437109935 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7ff43c072220 con 0x55c9225b94a0 2024-05-09T00:48:43.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.712+0000 7ff45fd8cb80 1 -- 172.21.15.5:0/1437109935 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff43c0aca20 con 0x55c9225b94a0 2024-05-09T00:48:43.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.713+0000 7ff42eff5700 1 -- 172.21.15.5:0/1437109935 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff43c0b5880 con 0x55c9225b94a0 2024-05-09T00:48:43.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.713+0000 7ff45fd8cb80 10 client.15645 fetch_fsmap learned FSMap version 29 2024-05-09T00:48:43.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.713+0000 7ff45fd8cb80 1 -- 172.21.15.5:0/1437109935 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c9227569e0 con 0x55c9225b94a0 2024-05-09T00:48:43.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.713+0000 7ff42eff5700 1 -- 172.21.15.5:0/1437109935 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff43c0aca20 con 0x55c9225b94a0 2024-05-09T00:48:43.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.713+0000 7ff42eff5700 1 -- 172.21.15.5:0/1437109935 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ff43c0a8390 con 0x55c9225b94a0 2024-05-09T00:48:43.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.713+0000 7ff45fd8cb80 10 client.15645 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:48:43.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.713+0000 7ff45fd8cb80 10 client.15645 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:48:43.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.713+0000 7ff45fd8cb80 20 client.15645 populate_metadata read hostname 'smithi005' 2024-05-09T00:48:43.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.713+0000 7ff45fd8cb80 1 --2- 172.21.15.5:0/1437109935 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55c922758eb0 0x55c92275b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:43.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.713+0000 7ff45fd8cb80 4 client.15645 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:48:43.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.713+0000 7ff45fd8cb80 1 -- 172.21.15.5:0/1437109935 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c922666a80 con 0x55c922758eb0 2024-05-09T00:48:43.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.713+0000 7ff44a7fc700 1 --2- 172.21.15.5:0/1437109935 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55c922758eb0 0x55c92275b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:43.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.714+0000 7ff42eff5700 10 client.15645 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:48:43.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.714+0000 7ff44a7fc700 1 --2- 172.21.15.5:0/1437109935 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55c922758eb0 0x55c92275b2d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f1161571fd7b461 server_cookie=9dc5c362ce2890ff in_seq=0 out_seq=0 2024-05-09T00:48:43.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.715+0000 7ff42eff5700 1 -- 172.21.15.5:0/1437109935 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c922666a80 con 0x55c922758eb0 2024-05-09T00:48:43.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.715+0000 7ff42eff5700 10 client.15645 handle_command_reply: tid=0 2024-05-09T00:48:43.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.715+0000 7ff45fd8cb80 1 client.15645 shutdown 2024-05-09T00:48:43.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.715+0000 7ff45fd8cb80 1 -- 172.21.15.5:0/1437109935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c9225b94a0 msgr2=0x55c922740210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:43.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.715+0000 7ff45fd8cb80 1 --2- 172.21.15.5:0/1437109935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c9225b94a0 0x55c922740210 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ff43c0a8aa0 tx=0x7ff43c00be60 comp rx=0 tx=0).stop 2024-05-09T00:48:43.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.715+0000 7ff45fd8cb80 1 -- 172.21.15.5:0/1437109935 shutdown_connections 2024-05-09T00:48:43.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.715+0000 7ff45fd8cb80 1 --2- 172.21.15.5:0/1437109935 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55c922758eb0 0x55c92275b2d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:43.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.715+0000 7ff45fd8cb80 1 --2- 172.21.15.5:0/1437109935 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c922740c90 0x55c922754a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:43.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.715+0000 7ff45fd8cb80 1 --2- 172.21.15.5:0/1437109935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c9225b94a0 0x55c922740210 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:43.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.715+0000 7ff45fd8cb80 1 --2- 172.21.15.5:0/1437109935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c922736400 0x55c922740750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:43.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.715+0000 7ff45fd8cb80 1 -- 172.21.15.5:0/1437109935 >> 172.21.15.5:0/1437109935 conn(0x55c9225bcf70 msgr2=0x55c92272aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:43.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.715+0000 7ff42eff5700 0 client.15645 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:48:43.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.715+0000 7ff45fd8cb80 1 -- 172.21.15.5:0/1437109935 shutdown_connections 2024-05-09T00:48:43.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.715+0000 7ff45fd8cb80 1 -- 172.21.15.5:0/1437109935 wait complete. 2024-05-09T00:48:43.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.715+0000 7ff45fd8cb80 20 client.15645 trim_cache size 0 max 16384 2024-05-09T00:48:43.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.731+0000 7ff45fd8cb80 1 Processor -- start 2024-05-09T00:48:43.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.731+0000 7ff45fd8cb80 1 -- start start 2024-05-09T00:48:43.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.732+0000 7ff45fd8cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c922736400 0x55c92273a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:43.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.732+0000 7ff45fd8cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c922740c90 0x55c92273f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:43.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.732+0000 7ff45fd8cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c922754a30 0x55c92273dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:43.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.732+0000 7ff45fd8cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff43800d3e0 con 0x55c922740c90 2024-05-09T00:48:43.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.732+0000 7ff45fd8cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff43800ef20 con 0x55c922754a30 2024-05-09T00:48:43.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.732+0000 7ff45fd8cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff43800dab0 con 0x55c922736400 2024-05-09T00:48:43.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.732+0000 7ff44a7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c922740c90 0x55c92273f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:43.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.732+0000 7ff44a7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c922740c90 0x55c92273f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35492/0 (socket says 172.21.15.5:35492) 2024-05-09T00:48:43.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.732+0000 7ff44a7fc700 1 -- 172.21.15.5:0/4100201358 learned_addr learned my addr 172.21.15.5:0/4100201358 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:43.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.732+0000 7ff44b7fe700 1 --2- 172.21.15.5:0/4100201358 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c922754a30 0x55c92273dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:43.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.732+0000 7ff44b7fe700 1 -- 172.21.15.5:0/4100201358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c922736400 msgr2=0x55c92273a900 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:48:43.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.732+0000 7ff44b7fe700 1 --2- 172.21.15.5:0/4100201358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c922736400 0x55c92273a900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:43.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.732+0000 7ff44b7fe700 1 -- 172.21.15.5:0/4100201358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c922740c90 msgr2=0x55c92273f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:43.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.732+0000 7ff44b7fe700 1 --2- 172.21.15.5:0/4100201358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c922740c90 0x55c92273f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:43.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.732+0000 7ff44b7fe700 1 -- 172.21.15.5:0/4100201358 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c92272d9d0 con 0x55c922754a30 2024-05-09T00:48:43.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.733+0000 7ff44b7fe700 1 --2- 172.21.15.5:0/4100201358 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c922754a30 0x55c92273dd90 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x55c922743050 tx=0x7ff44400b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e89894cdd043ec9f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:43.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.733+0000 7ff430ff9700 1 -- 172.21.15.5:0/4100201358 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4440092e0 con 0x55c922754a30 2024-05-09T00:48:43.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.733+0000 7ff430ff9700 1 -- 172.21.15.5:0/4100201358 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff444009480 con 0x55c922754a30 2024-05-09T00:48:43.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.733+0000 7ff430ff9700 1 -- 172.21.15.5:0/4100201358 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4440050b0 con 0x55c922754a30 2024-05-09T00:48:43.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.734+0000 7ff45fd8cb80 1 -- 172.21.15.5:0/4100201358 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c922754a30 msgr2=0x55c92273dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:43.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.734+0000 7ff45fd8cb80 1 --2- 172.21.15.5:0/4100201358 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c922754a30 0x55c92273dd90 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x55c922743050 tx=0x7ff44400b040 comp rx=0 tx=0).stop 2024-05-09T00:48:43.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.734+0000 7ff45fd8cb80 1 -- 172.21.15.5:0/4100201358 shutdown_connections 2024-05-09T00:48:43.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.734+0000 7ff45fd8cb80 1 --2- 172.21.15.5:0/4100201358 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c922754a30 0x55c92273dd90 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:43.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.734+0000 7ff45fd8cb80 1 --2- 172.21.15.5:0/4100201358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c922736400 0x55c92273a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:43.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.734+0000 7ff45fd8cb80 1 --2- 172.21.15.5:0/4100201358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c922740c90 0x55c92273f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:43.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.734+0000 7ff45fd8cb80 1 -- 172.21.15.5:0/4100201358 >> 172.21.15.5:0/4100201358 conn(0x55c9225bcf70 msgr2=0x55c92272aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:43.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.735+0000 7ff45fd8cb80 1 -- 172.21.15.5:0/4100201358 shutdown_connections 2024-05-09T00:48:43.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.735+0000 7ff45fd8cb80 1 -- 172.21.15.5:0/4100201358 wait complete. 2024-05-09T00:48:43.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.735+0000 7ff45fd8cb80 1 -- start start 2024-05-09T00:48:43.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.735+0000 7ff45fd8cb80 1 Processor -- start 2024-05-09T00:48:43.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.736+0000 7ff45fd8cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c922736400 0x55c9227592d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:43.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.736+0000 7ff45fd8cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c922740c90 0x55c922759810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:43.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.736+0000 7ff45fd8cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c922754a30 0x55c9227bb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:43.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.736+0000 7ff45fd8cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff43800e850 con 0x55c922736400 2024-05-09T00:48:43.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.736+0000 7ff45fd8cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff43800e180 con 0x55c922754a30 2024-05-09T00:48:43.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.736+0000 7ff45fd8cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff43800b380 con 0x55c922740c90 2024-05-09T00:48:43.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.736+0000 7ff45fd8cb80 10 client.0 _ll_register_callbacks cb 0x55c9224ca6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:48:43.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.736+0000 7ff44a7fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c922740c90 0x55c922759810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:43.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.736+0000 7ff44affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c922736400 0x55c9227592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:43.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.736+0000 7ff44a7fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c922740c90 0x55c922759810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:41714/0 (socket says 172.21.15.5:41714) 2024-05-09T00:48:43.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.736+0000 7ff44a7fc700 1 -- 172.21.15.5:0/1808230522 learned_addr learned my addr 172.21.15.5:0/1808230522 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:43.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.736+0000 7ff44b7fe700 1 --2- 172.21.15.5:0/1808230522 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c922754a30 0x55c9227bb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:43.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.736+0000 7ff44b7fe700 1 -- 172.21.15.5:0/1808230522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c922740c90 msgr2=0x55c922759810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:43.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.736+0000 7ff44b7fe700 1 --2- 172.21.15.5:0/1808230522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c922740c90 0x55c922759810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:43.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.736+0000 7ff44b7fe700 1 -- 172.21.15.5:0/1808230522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c922736400 msgr2=0x55c9227592d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:43.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.737+0000 7ff44b7fe700 1 --2- 172.21.15.5:0/1808230522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c922736400 0x55c9227592d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:43.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.737+0000 7ff44b7fe700 1 -- 172.21.15.5:0/1808230522 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9227bb920 con 0x55c922754a30 2024-05-09T00:48:43.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.737+0000 7ff44affd700 1 --2- 172.21.15.5:0/1808230522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c922736400 0x55c9227592d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:48:43.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.737+0000 7ff44b7fe700 1 --2- 172.21.15.5:0/1808230522 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c922754a30 0x55c9227bb3b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff444005010 tx=0x7ff4440077f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=544cb7b3a03099c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:43.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.737+0000 7ff42eff5700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-09T00:48:43.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.737+0000 7ff42eff5700 1 -- 172.21.15.5:0/1808230522 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff444003a00 con 0x55c922754a30 2024-05-09T00:48:43.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.737+0000 7ff45fd8cb80 1 -- 172.21.15.5:0/1808230522 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c922660b20 con 0x55c922754a30 2024-05-09T00:48:43.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.737+0000 7ff42eff5700 1 -- 172.21.15.5:0/1808230522 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7ff444003ba0 con 0x55c922754a30 2024-05-09T00:48:43.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.737+0000 7ff42eff5700 1 -- 172.21.15.5:0/1808230522 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff444003da0 con 0x55c922754a30 2024-05-09T00:48:43.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.737+0000 7ff45fd8cb80 10 client.25334 fetch_fsmap learned FSMap version 29 2024-05-09T00:48:43.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.737+0000 7ff45fd8cb80 1 -- 172.21.15.5:0/1808230522 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c9227c2c00 con 0x55c922754a30 2024-05-09T00:48:43.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.738+0000 7ff42eff5700 1 -- 172.21.15.5:0/1808230522 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55c922660b20 con 0x55c922754a30 2024-05-09T00:48:43.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.738+0000 7ff42eff5700 1 -- 172.21.15.5:0/1808230522 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ff44403a070 con 0x55c922754a30 2024-05-09T00:48:43.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.738+0000 7ff45fd8cb80 10 client.25334 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:48:43.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.738+0000 7ff45fd8cb80 10 client.25334 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:48:43.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.738+0000 7ff45fd8cb80 20 client.25334 populate_metadata read hostname 'smithi005' 2024-05-09T00:48:43.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.738+0000 7ff45fd8cb80 1 --2- 172.21.15.5:0/1808230522 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55c9227c3270 0x55c9227c5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:43.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.738+0000 7ff45fd8cb80 4 client.25334 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:48:43.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.738+0000 7ff45fd8cb80 1 -- 172.21.15.5:0/1808230522 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c9224d4970 con 0x55c9227c3270 2024-05-09T00:48:43.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.739+0000 7ff44affd700 1 --2- 172.21.15.5:0/1808230522 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55c9227c3270 0x55c9227c5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:43.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.739+0000 7ff42eff5700 10 client.25334 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:48:43.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.739+0000 7ff44affd700 1 --2- 172.21.15.5:0/1808230522 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55c9227c3270 0x55c9227c5690 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adbd57691c78bec1 server_cookie=f167b8335b416325 in_seq=0 out_seq=0 2024-05-09T00:48:43.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.739+0000 7ff42eff5700 1 -- 172.21.15.5:0/1808230522 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c9224d4970 con 0x55c9227c3270 2024-05-09T00:48:43.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.739+0000 7ff42eff5700 10 client.25334 handle_command_reply: tid=0 2024-05-09T00:48:43.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.739+0000 7ff45fd8cb80 1 client.25334 shutdown 2024-05-09T00:48:43.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.740+0000 7ff45fd8cb80 1 -- 172.21.15.5:0/1808230522 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c922754a30 msgr2=0x55c9227bb3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:43.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.740+0000 7ff45fd8cb80 1 --2- 172.21.15.5:0/1808230522 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c922754a30 0x55c9227bb3b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff444005010 tx=0x7ff4440077f0 comp rx=0 tx=0).stop 2024-05-09T00:48:43.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.740+0000 7ff45fd8cb80 1 -- 172.21.15.5:0/1808230522 shutdown_connections 2024-05-09T00:48:43.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.740+0000 7ff45fd8cb80 1 --2- 172.21.15.5:0/1808230522 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55c9227c3270 0x55c9227c5690 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:43.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.740+0000 7ff45fd8cb80 1 --2- 172.21.15.5:0/1808230522 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c922754a30 0x55c9227bb3b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:43.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.740+0000 7ff45fd8cb80 1 --2- 172.21.15.5:0/1808230522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c922736400 0x55c9227592d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:43.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.740+0000 7ff45fd8cb80 1 --2- 172.21.15.5:0/1808230522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c922740c90 0x55c922759810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:43.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.740+0000 7ff45fd8cb80 1 -- 172.21.15.5:0/1808230522 >> 172.21.15.5:0/1808230522 conn(0x55c9225bcf70 msgr2=0x55c9227353f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:43.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.740+0000 7ff42eff5700 0 client.25334 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:48:43.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.740+0000 7ff45fd8cb80 1 -- 172.21.15.5:0/1808230522 shutdown_connections 2024-05-09T00:48:43.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.740+0000 7ff45fd8cb80 1 -- 172.21.15.5:0/1808230522 wait complete. 2024-05-09T00:48:43.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.740+0000 7ff45fd8cb80 20 client.25334 trim_cache size 0 max 16384 2024-05-09T00:48:43.752 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-09T00:48:43.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.741+0000 7ff451376700 1 -- 172.21.15.5:0/2360803076 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ff4340b2880 msgr2=0x7ff4340b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:43.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.741+0000 7ff451376700 1 --2- 172.21.15.5:0/2360803076 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ff4340b2880 0x7ff4340b4d90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff44c106750 tx=0x7ff440009260 comp rx=0 tx=0).stop 2024-05-09T00:48:43.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.741+0000 7ff451376700 1 -- 172.21.15.5:0/2360803076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff44c1034c0 msgr2=0x7ff44c19bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:43.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.741+0000 7ff451376700 1 --2- 172.21.15.5:0/2360803076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff44c1034c0 0x7ff44c19bb90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff43c009c50 tx=0x7ff43c009f60 comp rx=0 tx=0).stop 2024-05-09T00:48:43.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.741+0000 7ff451376700 1 -- 172.21.15.5:0/2360803076 shutdown_connections 2024-05-09T00:48:43.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.741+0000 7ff451376700 1 --2- 172.21.15.5:0/2360803076 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ff4340b2880 0x7ff4340b4d90 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:43.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.741+0000 7ff451376700 1 --2- 172.21.15.5:0/2360803076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff44c108740 0x7ff44c19c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:43.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.741+0000 7ff451376700 1 --2- 172.21.15.5:0/2360803076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff44c1034c0 0x7ff44c19bb90 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:43.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.741+0000 7ff451376700 1 --2- 172.21.15.5:0/2360803076 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff44c107df0 0x7ff44c19c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:43.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.741+0000 7ff451376700 1 -- 172.21.15.5:0/2360803076 >> 172.21.15.5:0/2360803076 conn(0x7ff44c0fedc0 msgr2=0x7ff44c107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:43.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.741+0000 7ff451376700 1 -- 172.21.15.5:0/2360803076 shutdown_connections 2024-05-09T00:48:43.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:43.741+0000 7ff451376700 1 -- 172.21.15.5:0/2360803076 wait complete. 2024-05-09T00:48:45.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:48:45 smithi005 ceph-mon[30365]: pgmap v443: 97 pgs: 97 active+clean; 671 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-09T00:48:45.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:48:45 smithi131 ceph-mon[34547]: pgmap v443: 97 pgs: 97 active+clean; 671 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-09T00:48:45.621 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T00:48:45.621 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T00:48:45.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:48:45 smithi157 ceph-mon[34294]: pgmap v443: 97 pgs: 97 active+clean; 671 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-09T00:48:45.886 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:48:47.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:48:47 smithi131 ceph-mon[34547]: pgmap v444: 97 pgs: 97 active+clean; 671 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T00:48:47.672 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:48:47 smithi005 ceph-mon[30365]: pgmap v444: 97 pgs: 97 active+clean; 671 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T00:48:47.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:48:47 smithi157 ceph-mon[34294]: pgmap v444: 97 pgs: 97 active+clean; 671 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T00:48:48.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.130+0000 7f3fe5a2a700 1 -- 172.21.15.5:0/1868519667 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fe0108740 msgr2=0x7f3fe01100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:48.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.130+0000 7f3fe5a2a700 1 --2- 172.21.15.5:0/1868519667 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fe0108740 0x7f3fe01100a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f3fd800b0d0 tx=0x7f3fd800b3e0 comp rx=0 tx=0).stop 2024-05-09T00:48:48.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.130+0000 7f3fe5a2a700 1 -- 172.21.15.5:0/1868519667 shutdown_connections 2024-05-09T00:48:48.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.130+0000 7f3fe5a2a700 1 --2- 172.21.15.5:0/1868519667 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fe0108740 0x7f3fe01100a0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:48.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.130+0000 7f3fe5a2a700 1 --2- 172.21.15.5:0/1868519667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fe01034c0 0x7f3fe01038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:48.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.130+0000 7f3fe5a2a700 1 --2- 172.21.15.5:0/1868519667 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fe0107df0 0x7f3fe0108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:48.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.130+0000 7f3fe5a2a700 1 -- 172.21.15.5:0/1868519667 >> 172.21.15.5:0/1868519667 conn(0x7f3fe00fedc0 msgr2=0x7f3fe01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:48.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.130+0000 7f3fe5a2a700 1 -- 172.21.15.5:0/1868519667 shutdown_connections 2024-05-09T00:48:48.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.130+0000 7f3fe5a2a700 1 -- 172.21.15.5:0/1868519667 wait complete. 2024-05-09T00:48:48.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.131+0000 7f3fe5a2a700 1 Processor -- start 2024-05-09T00:48:48.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.131+0000 7f3fe5a2a700 1 -- start start 2024-05-09T00:48:48.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.131+0000 7f3fe5a2a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fe01034c0 0x7f3fe019bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:48.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.131+0000 7f3fe5a2a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fe0107df0 0x7f3fe019c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:48.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.131+0000 7f3fe5a2a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fe0108740 0x7f3fe019c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:48.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.131+0000 7f3fe5a2a700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe0113370 con 0x7f3fe0107df0 2024-05-09T00:48:48.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.131+0000 7f3fe5a2a700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe01131f0 con 0x7f3fe01034c0 2024-05-09T00:48:48.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.131+0000 7f3fe5a2a700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe01134f0 con 0x7f3fe0108740 2024-05-09T00:48:48.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.131+0000 7f3fde7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fe0107df0 0x7f3fe019c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:48.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.131+0000 7f3fde7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fe0107df0 0x7f3fe019c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35516/0 (socket says 172.21.15.5:35516) 2024-05-09T00:48:48.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.131+0000 7f3fde7fc700 1 -- 172.21.15.5:0/99617074 learned_addr learned my addr 172.21.15.5:0/99617074 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:48.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.132+0000 7f3fdeffd700 1 --2- 172.21.15.5:0/99617074 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fe01034c0 0x7f3fe019bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:48.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.132+0000 7f3fdf7fe700 1 --2- 172.21.15.5:0/99617074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fe0108740 0x7f3fe019c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:48.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.132+0000 7f3fde7fc700 1 -- 172.21.15.5:0/99617074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fe0108740 msgr2=0x7f3fe019c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:48.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.132+0000 7f3fde7fc700 1 --2- 172.21.15.5:0/99617074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fe0108740 0x7f3fe019c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:48.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.132+0000 7f3fde7fc700 1 -- 172.21.15.5:0/99617074 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fe01034c0 msgr2=0x7f3fe019bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:48.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.132+0000 7f3fde7fc700 1 --2- 172.21.15.5:0/99617074 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fe01034c0 0x7f3fe019bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:48.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.132+0000 7f3fde7fc700 1 -- 172.21.15.5:0/99617074 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fe01a7000 con 0x7f3fe0107df0 2024-05-09T00:48:48.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.132+0000 7f3fde7fc700 1 --2- 172.21.15.5:0/99617074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fe0107df0 0x7f3fe019c1d0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f3fd400d980 tx=0x7f3fd400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:48.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.132+0000 7f3fe4a28700 1 -- 172.21.15.5:0/99617074 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd4017070 con 0x7f3fe0107df0 2024-05-09T00:48:48.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.132+0000 7f3fe4a28700 1 -- 172.21.15.5:0/99617074 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fd400fd40 con 0x7f3fe0107df0 2024-05-09T00:48:48.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.132+0000 7f3fe4a28700 1 -- 172.21.15.5:0/99617074 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd400e3b0 con 0x7f3fe0107df0 2024-05-09T00:48:48.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.132+0000 7f3fe5a2a700 1 -- 172.21.15.5:0/99617074 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fe01a72f0 con 0x7f3fe0107df0 2024-05-09T00:48:48.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.132+0000 7f3fe5a2a700 1 -- 172.21.15.5:0/99617074 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fe01a7880 con 0x7f3fe0107df0 2024-05-09T00:48:48.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.135+0000 7f3fe4a28700 1 -- 172.21.15.5:0/99617074 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f3fd4010610 con 0x7f3fe0107df0 2024-05-09T00:48:48.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.136+0000 7f3ff4440b80 1 Processor -- start 2024-05-09T00:48:48.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.136+0000 7f3fe4a28700 1 --2- 172.21.15.5:0/99617074 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f3fc80b2900 0x7f3fc80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:48.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.136+0000 7f3ff4440b80 1 -- start start 2024-05-09T00:48:48.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.137+0000 7f3fe4a28700 1 -- 172.21.15.5:0/99617074 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f3fd40e1ba0 con 0x7f3fe0107df0 2024-05-09T00:48:48.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.137+0000 7f3fdf7fe700 1 --2- 172.21.15.5:0/99617074 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f3fc80b2900 0x7f3fc80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:48.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.137+0000 7f3ff4440b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d383fc6d80 0x55d3840cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:48.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.137+0000 7f3ff4440b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d383f4c520 0x55d3840ce0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:48.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.137+0000 7f3ff4440b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3840d2260 0x55d3840d2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:48.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.137+0000 7f3ff4440b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d383fec120 con 0x55d383f4c520 2024-05-09T00:48:48.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.137+0000 7f3ff4440b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55d383f91a20 con 0x55d3840d2260 2024-05-09T00:48:48.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.137+0000 7f3ff4440b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d383fe80e0 con 0x55d383fc6d80 2024-05-09T00:48:48.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.137+0000 7f3fdf7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3840d2260 0x55d3840d2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:48.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.137+0000 7f3fdf7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3840d2260 0x55d3840d2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:39500/0 (socket says 172.21.15.5:39500) 2024-05-09T00:48:48.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.137+0000 7f3fdf7fe700 1 -- 172.21.15.5:0/2564371394 learned_addr learned my addr 172.21.15.5:0/2564371394 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:48.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.137+0000 7f3fde7fc700 1 --2- 172.21.15.5:0/2564371394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d383f4c520 0x55d3840ce0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:48.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.137+0000 7f3fdf7fe700 1 --2- 172.21.15.5:0/99617074 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f3fc80b2900 0x7f3fc80b4e10 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8009900 tx=0x7f3fd8009c80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:48.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.137+0000 7f3fdf7fe700 1 -- 172.21.15.5:0/2564371394 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d383fc6d80 msgr2=0x55d3840cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:48.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.137+0000 7f3fdf7fe700 1 --2- 172.21.15.5:0/2564371394 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d383fc6d80 0x55d3840cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:48.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.137+0000 7f3fdf7fe700 1 -- 172.21.15.5:0/2564371394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d383f4c520 msgr2=0x55d3840ce0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:48.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.137+0000 7f3fdf7fe700 1 --2- 172.21.15.5:0/2564371394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d383f4c520 0x55d3840ce0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:48.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.138+0000 7f3fdf7fe700 1 -- 172.21.15.5:0/2564371394 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d383f82580 con 0x55d3840d2260 2024-05-09T00:48:48.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.138+0000 7f3fdf7fe700 1 --2- 172.21.15.5:0/2564371394 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3840d2260 0x55d3840d2670 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8009460 tx=0x7f3fd8025000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33e490cc762b3924 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:48.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.138+0000 7f3fc5ffb700 1 -- 172.21.15.5:0/2564371394 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd8007c50 con 0x55d3840d2260 2024-05-09T00:48:48.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.138+0000 7f3fc5ffb700 1 -- 172.21.15.5:0/2564371394 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fd8007df0 con 0x55d3840d2260 2024-05-09T00:48:48.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.138+0000 7f3fc5ffb700 1 -- 172.21.15.5:0/2564371394 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd80286d0 con 0x55d3840d2260 2024-05-09T00:48:48.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.139+0000 7f3ff4440b80 1 -- 172.21.15.5:0/2564371394 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3840d2260 msgr2=0x55d3840d2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:48.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.139+0000 7f3ff4440b80 1 --2- 172.21.15.5:0/2564371394 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3840d2260 0x55d3840d2670 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8009460 tx=0x7f3fd8025000 comp rx=0 tx=0).stop 2024-05-09T00:48:48.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.139+0000 7f3ff4440b80 1 -- 172.21.15.5:0/2564371394 shutdown_connections 2024-05-09T00:48:48.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.139+0000 7f3ff4440b80 1 --2- 172.21.15.5:0/2564371394 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3840d2260 0x55d3840d2670 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:48.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.139+0000 7f3ff4440b80 1 --2- 172.21.15.5:0/2564371394 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d383fc6d80 0x55d3840cdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:48.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.139+0000 7f3ff4440b80 1 --2- 172.21.15.5:0/2564371394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d383f4c520 0x55d3840ce0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:48.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.139+0000 7f3ff4440b80 1 -- 172.21.15.5:0/2564371394 >> 172.21.15.5:0/2564371394 conn(0x55d383f4fff0 msgr2=0x55d3840c42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:48.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.139+0000 7f3ff4440b80 1 -- 172.21.15.5:0/2564371394 shutdown_connections 2024-05-09T00:48:48.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.139+0000 7f3ff4440b80 1 -- 172.21.15.5:0/2564371394 wait complete. 2024-05-09T00:48:48.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.140+0000 7f3ff4440b80 1 -- start start 2024-05-09T00:48:48.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.140+0000 7f3ff4440b80 1 Processor -- start 2024-05-09T00:48:48.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.141+0000 7f3ff4440b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d383f4c520 0x55d3840d7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:48.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.141+0000 7f3ff4440b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d383fc6d80 0x55d3840d7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:48.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.141+0000 7f3ff4440b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3840d2260 0x55d3840d7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:48.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.141+0000 7f3ff4440b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d3840d2f20 con 0x55d3840d2260 2024-05-09T00:48:48.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.141+0000 7f3ff4440b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55d383f86ba0 con 0x55d383f4c520 2024-05-09T00:48:48.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.141+0000 7f3ff4440b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d3840d30a0 con 0x55d383fc6d80 2024-05-09T00:48:48.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.141+0000 7f3ff4440b80 10 client.0 _ll_register_callbacks cb 0x55d383e5d740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:48:48.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.141+0000 7f3fdf7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3840d2260 0x55d3840d7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:48.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.141+0000 7f3fdf7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3840d2260 0x55d3840d7ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35538/0 (socket says 172.21.15.5:35538) 2024-05-09T00:48:48.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.141+0000 7f3fdf7fe700 1 -- 172.21.15.5:0/3940725426 learned_addr learned my addr 172.21.15.5:0/3940725426 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:48.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.141+0000 7f3fdeffd700 1 --2- 172.21.15.5:0/3940725426 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d383f4c520 0x55d3840d7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:48.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.141+0000 7f3fde7fc700 1 --2- 172.21.15.5:0/3940725426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d383fc6d80 0x55d3840d7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:48.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.141+0000 7f3fdf7fe700 1 -- 172.21.15.5:0/3940725426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d383fc6d80 msgr2=0x55d3840d7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:48.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.141+0000 7f3fdf7fe700 1 --2- 172.21.15.5:0/3940725426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d383fc6d80 0x55d3840d7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:48.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.141+0000 7f3fdf7fe700 1 -- 172.21.15.5:0/3940725426 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d383f4c520 msgr2=0x55d3840d7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:48.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.141+0000 7f3fdf7fe700 1 --2- 172.21.15.5:0/3940725426 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d383f4c520 0x55d3840d7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:48.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.141+0000 7f3fdf7fe700 1 -- 172.21.15.5:0/3940725426 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d383e67510 con 0x55d3840d2260 2024-05-09T00:48:48.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.141+0000 7f3fde7fc700 1 --2- 172.21.15.5:0/3940725426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d383fc6d80 0x55d3840d7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:48:48.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.142+0000 7f3fdf7fe700 1 --2- 172.21.15.5:0/3940725426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3840d2260 0x55d3840d7ec0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8020010 tx=0x7f3fd801e770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d80de69b9109a98e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:48.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.142+0000 7f3fbeffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:48:48.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.142+0000 7f3fbeffd700 1 -- 172.21.15.5:0/3940725426 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd801ece0 con 0x55d3840d2260 2024-05-09T00:48:48.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.142+0000 7f3fbeffd700 1 -- 172.21.15.5:0/3940725426 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f3fd8028420 con 0x55d3840d2260 2024-05-09T00:48:48.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.142+0000 7f3ff4440b80 1 -- 172.21.15.5:0/3940725426 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fd8007df0 con 0x55d3840d2260 2024-05-09T00:48:48.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.142+0000 7f3fbeffd700 1 -- 172.21.15.5:0/3940725426 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3fd8028620 con 0x55d3840d2260 2024-05-09T00:48:48.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.142+0000 7f3ff4440b80 10 client.15663 fetch_fsmap learned FSMap version 29 2024-05-09T00:48:48.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.142+0000 7f3ff4440b80 1 -- 172.21.15.5:0/3940725426 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d3840e4da0 con 0x55d3840d2260 2024-05-09T00:48:48.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.142+0000 7f3fbeffd700 1 -- 172.21.15.5:0/3940725426 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fd8007df0 con 0x55d3840d2260 2024-05-09T00:48:48.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.143+0000 7f3fbeffd700 1 -- 172.21.15.5:0/3940725426 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f3fd8043020 con 0x55d3840d2260 2024-05-09T00:48:48.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.143+0000 7f3ff4440b80 10 client.15663 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:48:48.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.143+0000 7f3ff4440b80 10 client.15663 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:48:48.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.143+0000 7f3ff4440b80 20 client.15663 populate_metadata read hostname 'smithi005' 2024-05-09T00:48:48.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.143+0000 7f3ff4440b80 1 --2- 172.21.15.5:0/3940725426 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55d3840e7270 0x55d3840e9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:48.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.143+0000 7f3ff4440b80 4 client.15663 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:48:48.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.143+0000 7f3ff4440b80 1 -- 172.21.15.5:0/3940725426 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d383ff9b00 con 0x55d3840e7270 2024-05-09T00:48:48.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.143+0000 7f3fdeffd700 1 --2- 172.21.15.5:0/3940725426 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55d3840e7270 0x55d3840e9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:48.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.144+0000 7f3fbeffd700 10 client.15663 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:48:48.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.144+0000 7f3fdeffd700 1 --2- 172.21.15.5:0/3940725426 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55d3840e7270 0x55d3840e9690 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41a800c5f2f69388 server_cookie=d7d8a6bc41f615e2 in_seq=0 out_seq=0 2024-05-09T00:48:48.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.144+0000 7f3fbeffd700 1 -- 172.21.15.5:0/3940725426 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d383ff9b00 con 0x55d3840e7270 2024-05-09T00:48:48.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.144+0000 7f3fbeffd700 10 client.15663 handle_command_reply: tid=0 2024-05-09T00:48:48.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.144+0000 7f3ff4440b80 1 client.15663 shutdown 2024-05-09T00:48:48.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.145+0000 7f3ff4440b80 1 -- 172.21.15.5:0/3940725426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3840d2260 msgr2=0x55d3840d7ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:48.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.145+0000 7f3ff4440b80 1 --2- 172.21.15.5:0/3940725426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3840d2260 0x55d3840d7ec0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8020010 tx=0x7f3fd801e770 comp rx=0 tx=0).stop 2024-05-09T00:48:48.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.145+0000 7f3ff4440b80 1 -- 172.21.15.5:0/3940725426 shutdown_connections 2024-05-09T00:48:48.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.145+0000 7f3ff4440b80 1 --2- 172.21.15.5:0/3940725426 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55d3840e7270 0x55d3840e9690 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:48.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.145+0000 7f3ff4440b80 1 --2- 172.21.15.5:0/3940725426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3840d2260 0x55d3840d7ec0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:48.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.145+0000 7f3ff4440b80 1 --2- 172.21.15.5:0/3940725426 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d383f4c520 0x55d3840d7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:48.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.145+0000 7f3fbeffd700 0 client.15663 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:48:48.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.145+0000 7f3ff4440b80 1 --2- 172.21.15.5:0/3940725426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d383fc6d80 0x55d3840d7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:48.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.145+0000 7f3ff4440b80 1 -- 172.21.15.5:0/3940725426 >> 172.21.15.5:0/3940725426 conn(0x55d383f4fff0 msgr2=0x55d3840c8470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:48.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.145+0000 7f3ff4440b80 1 -- 172.21.15.5:0/3940725426 shutdown_connections 2024-05-09T00:48:48.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.145+0000 7f3ff4440b80 1 -- 172.21.15.5:0/3940725426 wait complete. 2024-05-09T00:48:48.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.145+0000 7f3ff4440b80 20 client.15663 trim_cache size 0 max 16384 2024-05-09T00:48:48.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.166+0000 7f3ff4440b80 1 Processor -- start 2024-05-09T00:48:48.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.166+0000 7f3ff4440b80 1 -- start start 2024-05-09T00:48:48.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.166+0000 7f3ff4440b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d383f4c520 0x55d3840c8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:48.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.166+0000 7f3ff4440b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d383fc6d80 0x55d3840d4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:48.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.166+0000 7f3ff4440b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d3840d2260 0x55d3840d4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:48.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.166+0000 7f3ff4440b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe01acac0 con 0x55d383f4c520 2024-05-09T00:48:48.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.166+0000 7f3ff4440b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe01ae600 con 0x55d383fc6d80 2024-05-09T00:48:48.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.166+0000 7f3ff4440b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe01ad190 con 0x55d3840d2260 2024-05-09T00:48:48.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.167+0000 7f3fdeffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d383f4c520 0x55d3840c8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:48.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.167+0000 7f3fdeffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d383f4c520 0x55d3840c8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35544/0 (socket says 172.21.15.5:35544) 2024-05-09T00:48:48.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.167+0000 7f3fdeffd700 1 -- 172.21.15.5:0/3615672970 learned_addr learned my addr 172.21.15.5:0/3615672970 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:48.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.167+0000 7f3fde7fc700 1 --2- 172.21.15.5:0/3615672970 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d383fc6d80 0x55d3840d4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:48.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.167+0000 7f3fdf7fe700 1 --2- 172.21.15.5:0/3615672970 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d3840d2260 0x55d3840d4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:48.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.167+0000 7f3fdeffd700 1 -- 172.21.15.5:0/3615672970 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d3840d2260 msgr2=0x55d3840d4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:48.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.167+0000 7f3fdeffd700 1 --2- 172.21.15.5:0/3615672970 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d3840d2260 0x55d3840d4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:48.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.167+0000 7f3fdeffd700 1 -- 172.21.15.5:0/3615672970 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d383fc6d80 msgr2=0x55d3840d4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:48.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.167+0000 7f3fdeffd700 1 --2- 172.21.15.5:0/3615672970 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d383fc6d80 0x55d3840d4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:48.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.167+0000 7f3fdeffd700 1 -- 172.21.15.5:0/3615672970 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3840d59b0 con 0x55d383f4c520 2024-05-09T00:48:48.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.167+0000 7f3fdeffd700 1 --2- 172.21.15.5:0/3615672970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d383f4c520 0x55d3840c8ea0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f3fd000b6c0 tx=0x7f3fd000bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30bfa5a7a92b4e38 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:48.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.167+0000 7f3fc57fa700 1 -- 172.21.15.5:0/3615672970 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd0004830 con 0x55d383f4c520 2024-05-09T00:48:48.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.167+0000 7f3fc57fa700 1 -- 172.21.15.5:0/3615672970 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fd0005020 con 0x55d383f4c520 2024-05-09T00:48:48.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.167+0000 7f3fc57fa700 1 -- 172.21.15.5:0/3615672970 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd0003b20 con 0x55d383f4c520 2024-05-09T00:48:48.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.168+0000 7f3ff4440b80 1 -- 172.21.15.5:0/3615672970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d383f4c520 msgr2=0x55d3840c8ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:48.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.168+0000 7f3ff4440b80 1 --2- 172.21.15.5:0/3615672970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d383f4c520 0x55d3840c8ea0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f3fd000b6c0 tx=0x7f3fd000bc30 comp rx=0 tx=0).stop 2024-05-09T00:48:48.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.168+0000 7f3ff4440b80 1 -- 172.21.15.5:0/3615672970 shutdown_connections 2024-05-09T00:48:48.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.168+0000 7f3ff4440b80 1 --2- 172.21.15.5:0/3615672970 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d3840d2260 0x55d3840d4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:48.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.168+0000 7f3ff4440b80 1 --2- 172.21.15.5:0/3615672970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d383f4c520 0x55d3840c8ea0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:48.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.168+0000 7f3ff4440b80 1 --2- 172.21.15.5:0/3615672970 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d383fc6d80 0x55d3840d4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:48.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.168+0000 7f3ff4440b80 1 -- 172.21.15.5:0/3615672970 >> 172.21.15.5:0/3615672970 conn(0x55d383f4fff0 msgr2=0x55d3840be6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:48.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.168+0000 7f3ff4440b80 1 -- 172.21.15.5:0/3615672970 shutdown_connections 2024-05-09T00:48:48.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.168+0000 7f3ff4440b80 1 -- 172.21.15.5:0/3615672970 wait complete. 2024-05-09T00:48:48.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.169+0000 7f3ff4440b80 1 -- start start 2024-05-09T00:48:48.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.169+0000 7f3ff4440b80 1 Processor -- start 2024-05-09T00:48:48.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.169+0000 7f3ff4440b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d383f4c520 0x55d384149c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:48.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.169+0000 7f3ff4440b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d383fc6d80 0x55d38414a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:48.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.169+0000 7f3ff4440b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d3840d2260 0x55d384145750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:48.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.170+0000 7f3ff4440b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe01adf30 con 0x55d383fc6d80 2024-05-09T00:48:48.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.170+0000 7f3ff4440b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe01ad860 con 0x55d383f4c520 2024-05-09T00:48:48.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.170+0000 7f3ff4440b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe01aaa60 con 0x55d3840d2260 2024-05-09T00:48:48.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.170+0000 7f3fde7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d383fc6d80 0x55d38414a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:48.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.170+0000 7f3ff4440b80 10 client.0 _ll_register_callbacks cb 0x55d383e5d740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:48:48.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.170+0000 7f3fde7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d383fc6d80 0x55d38414a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35554/0 (socket says 172.21.15.5:35554) 2024-05-09T00:48:48.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.170+0000 7f3fde7fc700 1 -- 172.21.15.5:0/819176634 learned_addr learned my addr 172.21.15.5:0/819176634 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:48.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.170+0000 7f3fdeffd700 1 --2- 172.21.15.5:0/819176634 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d383f4c520 0x55d384149c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:48.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.170+0000 7f3fdf7fe700 1 --2- 172.21.15.5:0/819176634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d3840d2260 0x55d384145750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:48.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.170+0000 7f3fdeffd700 1 -- 172.21.15.5:0/819176634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d3840d2260 msgr2=0x55d384145750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:48.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.170+0000 7f3fdeffd700 1 --2- 172.21.15.5:0/819176634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d3840d2260 0x55d384145750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:48.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.170+0000 7f3fdeffd700 1 -- 172.21.15.5:0/819176634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d383fc6d80 msgr2=0x55d38414a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:48.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.170+0000 7f3fdeffd700 1 --2- 172.21.15.5:0/819176634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d383fc6d80 0x55d38414a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:48.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.170+0000 7f3fdeffd700 1 -- 172.21.15.5:0/819176634 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d384145d90 con 0x55d383f4c520 2024-05-09T00:48:48.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.170+0000 7f3fdeffd700 1 --2- 172.21.15.5:0/819176634 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d383f4c520 0x55d384149c20 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0004fd0 tx=0x7f3fd00053c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=636a712c34afe8f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:48.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.170+0000 7f3fbeffd700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-09T00:48:48.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.170+0000 7f3fbeffd700 1 -- 172.21.15.5:0/819176634 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd0004770 con 0x55d383f4c520 2024-05-09T00:48:48.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.171+0000 7f3ff4440b80 1 -- 172.21.15.5:0/819176634 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fd0005020 con 0x55d383f4c520 2024-05-09T00:48:48.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.171+0000 7f3fbeffd700 1 -- 172.21.15.5:0/819176634 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f3fd0023040 con 0x55d383f4c520 2024-05-09T00:48:48.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.171+0000 7f3fbeffd700 1 -- 172.21.15.5:0/819176634 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3fd0037050 con 0x55d383f4c520 2024-05-09T00:48:48.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.171+0000 7f3ff4440b80 10 client.25352 fetch_fsmap learned FSMap version 29 2024-05-09T00:48:48.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.171+0000 7f3ff4440b80 1 -- 172.21.15.5:0/819176634 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d3841523c0 con 0x55d383f4c520 2024-05-09T00:48:48.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.171+0000 7f3fbeffd700 1 -- 172.21.15.5:0/819176634 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fd0005020 con 0x55d383f4c520 2024-05-09T00:48:48.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.171+0000 7f3fbeffd700 1 -- 172.21.15.5:0/819176634 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f3fd00115b0 con 0x55d383f4c520 2024-05-09T00:48:48.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.171+0000 7f3ff4440b80 10 client.25352 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:48:48.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.171+0000 7f3ff4440b80 10 client.25352 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:48:48.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.171+0000 7f3ff4440b80 20 client.25352 populate_metadata read hostname 'smithi005' 2024-05-09T00:48:48.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.171+0000 7f3ff4440b80 1 --2- 172.21.15.5:0/819176634 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55d384152a70 0x55d384154e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:48.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.171+0000 7f3ff4440b80 4 client.25352 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T00:48:48.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.171+0000 7f3ff4440b80 1 -- 172.21.15.5:0/819176634 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d3840d8250 con 0x55d384152a70 2024-05-09T00:48:48.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.172+0000 7f3fde7fc700 1 --2- 172.21.15.5:0/819176634 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55d384152a70 0x55d384154e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:48.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.172+0000 7f3fde7fc700 1 --2- 172.21.15.5:0/819176634 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55d384152a70 0x55d384154e90 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5149309df346c6b server_cookie=ae1a144a2bee7b2a in_seq=0 out_seq=0 2024-05-09T00:48:48.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.172+0000 7f3fbeffd700 10 client.25352 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:48:48.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.173+0000 7f3fbeffd700 1 -- 172.21.15.5:0/819176634 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d3840d8250 con 0x55d384152a70 2024-05-09T00:48:48.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.173+0000 7f3fbeffd700 10 client.25352 handle_command_reply: tid=0 2024-05-09T00:48:48.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.173+0000 7f3ff4440b80 1 client.25352 shutdown 2024-05-09T00:48:48.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.173+0000 7f3ff4440b80 1 -- 172.21.15.5:0/819176634 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d383f4c520 msgr2=0x55d384149c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:48.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.173+0000 7f3ff4440b80 1 --2- 172.21.15.5:0/819176634 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d383f4c520 0x55d384149c20 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0004fd0 tx=0x7f3fd00053c0 comp rx=0 tx=0).stop 2024-05-09T00:48:48.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.173+0000 7f3ff4440b80 1 -- 172.21.15.5:0/819176634 shutdown_connections 2024-05-09T00:48:48.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.173+0000 7f3ff4440b80 1 --2- 172.21.15.5:0/819176634 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55d384152a70 0x55d384154e90 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:48.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.173+0000 7f3ff4440b80 1 --2- 172.21.15.5:0/819176634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d3840d2260 0x55d384145750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:48.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.173+0000 7f3ff4440b80 1 --2- 172.21.15.5:0/819176634 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d383f4c520 0x55d384149c20 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:48.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.173+0000 7f3ff4440b80 1 --2- 172.21.15.5:0/819176634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d383fc6d80 0x55d38414a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:48.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.173+0000 7f3ff4440b80 1 -- 172.21.15.5:0/819176634 >> 172.21.15.5:0/819176634 conn(0x55d383f4fff0 msgr2=0x55d3840d8a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:48.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.173+0000 7f3fbeffd700 0 client.25352 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:48:48.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.174+0000 7f3ff4440b80 1 -- 172.21.15.5:0/819176634 shutdown_connections 2024-05-09T00:48:48.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.174+0000 7f3ff4440b80 1 -- 172.21.15.5:0/819176634 wait complete. 2024-05-09T00:48:48.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.174+0000 7f3ff4440b80 20 client.25352 trim_cache size 0 max 16384 2024-05-09T00:48:48.178 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-09T00:48:48.178 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-05-09T00:48:48.178 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "6d127baf-7d53-40c1-9465-fe399baafd70", 2024-05-09T00:48:48.178 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-05-09T00:48:48.178 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-09T00:48:48.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.174+0000 7f3fe5a2a700 1 -- 172.21.15.5:0/99617074 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f3fc80b2900 msgr2=0x7f3fc80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:48.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.174+0000 7f3fe5a2a700 1 --2- 172.21.15.5:0/99617074 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f3fc80b2900 0x7f3fc80b4e10 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8009900 tx=0x7f3fd8009c80 comp rx=0 tx=0).stop 2024-05-09T00:48:48.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.174+0000 7f3fe5a2a700 1 -- 172.21.15.5:0/99617074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fe0107df0 msgr2=0x7f3fe019c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:48.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.174+0000 7f3fe5a2a700 1 --2- 172.21.15.5:0/99617074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fe0107df0 0x7f3fe019c1d0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f3fd400d980 tx=0x7f3fd400dc90 comp rx=0 tx=0).stop 2024-05-09T00:48:48.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.174+0000 7f3fe5a2a700 1 -- 172.21.15.5:0/99617074 shutdown_connections 2024-05-09T00:48:48.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.174+0000 7f3fe5a2a700 1 --2- 172.21.15.5:0/99617074 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f3fc80b2900 0x7f3fc80b4e10 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:48.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.174+0000 7f3fe5a2a700 1 --2- 172.21.15.5:0/99617074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fe0108740 0x7f3fe019c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:48.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.174+0000 7f3fe5a2a700 1 --2- 172.21.15.5:0/99617074 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fe01034c0 0x7f3fe019bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:48.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.174+0000 7f3fe5a2a700 1 --2- 172.21.15.5:0/99617074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fe0107df0 0x7f3fe019c1d0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:48.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.174+0000 7f3fe5a2a700 1 -- 172.21.15.5:0/99617074 >> 172.21.15.5:0/99617074 conn(0x7f3fe00fedc0 msgr2=0x7f3fe0107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:48.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.175+0000 7f3fe5a2a700 1 -- 172.21.15.5:0/99617074 shutdown_connections 2024-05-09T00:48:48.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:48.175+0000 7f3fe5a2a700 1 -- 172.21.15.5:0/99617074 wait complete. 2024-05-09T00:48:48.436 INFO:tasks.workunit.client.0.smithi005.stderr:Updating files: 89% (10930/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-05-09T00:48:49.095 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T00:48:49.150 INFO:tasks.workunit.client.0.smithi005.stderr:Note: switching to 'c9da4049f638c0f84bdd9b4e33c42e86046ac554'. 2024-05-09T00:48:49.151 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-05-09T00:48:49.151 INFO:tasks.workunit.client.0.smithi005.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-09T00:48:49.151 INFO:tasks.workunit.client.0.smithi005.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-09T00:48:49.151 INFO:tasks.workunit.client.0.smithi005.stderr:state without impacting any branches by switching back to a branch. 2024-05-09T00:48:49.151 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-05-09T00:48:49.151 INFO:tasks.workunit.client.0.smithi005.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-09T00:48:49.151 INFO:tasks.workunit.client.0.smithi005.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-09T00:48:49.151 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-05-09T00:48:49.151 INFO:tasks.workunit.client.0.smithi005.stderr: git switch -c 2024-05-09T00:48:49.151 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-05-09T00:48:49.151 INFO:tasks.workunit.client.0.smithi005.stderr:Or undo this operation with: 2024-05-09T00:48:49.152 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-05-09T00:48:49.152 INFO:tasks.workunit.client.0.smithi005.stderr: git switch - 2024-05-09T00:48:49.152 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-05-09T00:48:49.152 INFO:tasks.workunit.client.0.smithi005.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-09T00:48:49.152 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-05-09T00:48:49.152 INFO:tasks.workunit.client.0.smithi005.stderr:HEAD is now at c9da4049f63 Merge PR #57081 into wip-pdonnell-testing-20240508.183908-debug 2024-05-09T00:48:49.157 DEBUG:teuthology.orchestra.run.smithi005:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-05-09T00:48:49.178 INFO:tasks.workunit.client.0.smithi005.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-09T00:48:49.181 INFO:tasks.workunit.client.0.smithi005.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-09T00:48:49.181 INFO:tasks.workunit.client.0.smithi005.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-09T00:48:49.351 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:48:49.475 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:48:49 smithi005 ceph-mon[30365]: pgmap v445: 97 pgs: 97 active+clean; 671 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T00:48:49.475 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:48:49 smithi005 ceph-mon[30365]: scrub queued for path: / 2024-05-09T00:48:49.475 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:48:49 smithi005 ceph-mon[30365]: scrub summary: idle+waiting paths [/] 2024-05-09T00:48:49.475 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:48:49 smithi005 ceph-mon[30365]: scrub summary: active paths [/] 2024-05-09T00:48:49.475 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:48:49 smithi005 ceph-mon[30365]: scrub summary: idle+waiting paths [/] 2024-05-09T00:48:49.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:48:49 smithi131 ceph-mon[34547]: pgmap v445: 97 pgs: 97 active+clean; 671 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T00:48:49.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:48:49 smithi131 ceph-mon[34547]: scrub queued for path: / 2024-05-09T00:48:49.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:48:49 smithi131 ceph-mon[34547]: scrub summary: idle+waiting paths [/] 2024-05-09T00:48:49.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:48:49 smithi131 ceph-mon[34547]: scrub summary: active paths [/] 2024-05-09T00:48:49.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:48:49 smithi131 ceph-mon[34547]: scrub summary: idle+waiting paths [/] 2024-05-09T00:48:49.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:48:49 smithi157 ceph-mon[34294]: pgmap v445: 97 pgs: 97 active+clean; 671 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T00:48:49.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:48:49 smithi157 ceph-mon[34294]: scrub queued for path: / 2024-05-09T00:48:49.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:48:49 smithi157 ceph-mon[34294]: scrub summary: idle+waiting paths [/] 2024-05-09T00:48:49.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:48:49 smithi157 ceph-mon[34294]: scrub summary: active paths [/] 2024-05-09T00:48:49.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:48:49 smithi157 ceph-mon[34294]: scrub summary: idle+waiting paths [/] 2024-05-09T00:48:50.664 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.661+0000 7f4f9b4f9640 1 -- 172.21.15.5:0/1192416314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f94102e70 msgr2=0x7f4f94103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:50.664 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.661+0000 7f4f9b4f9640 1 --2- 172.21.15.5:0/1192416314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f94102e70 0x7f4f94103280 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f4f84009930 tx=0x7f4f8402a420 comp rx=0 tx=0).stop 2024-05-09T00:48:50.664 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.661+0000 7f4f9b4f9640 1 -- 172.21.15.5:0/1192416314 shutdown_connections 2024-05-09T00:48:50.664 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.661+0000 7f4f9b4f9640 1 --2- 172.21.15.5:0/1192416314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f941037c0 0x7f4f94103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:50.664 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.661+0000 7f4f9b4f9640 1 --2- 172.21.15.5:0/1192416314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f94102e70 0x7f4f94103280 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:50.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.661+0000 7f4f9b4f9640 1 --2- 172.21.15.5:0/1192416314 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f94102520 0x7f4f94102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:50.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.661+0000 7f4f9b4f9640 1 -- 172.21.15.5:0/1192416314 >> 172.21.15.5:0/1192416314 conn(0x7f4f940fdd30 msgr2=0x7f4f941001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:50.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.661+0000 7f4f9b4f9640 1 -- 172.21.15.5:0/1192416314 shutdown_connections 2024-05-09T00:48:50.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.661+0000 7f4f9b4f9640 1 -- 172.21.15.5:0/1192416314 wait complete. 2024-05-09T00:48:50.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.662+0000 7f4f9b4f9640 1 Processor -- start 2024-05-09T00:48:50.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.662+0000 7f4f9b4f9640 1 -- start start 2024-05-09T00:48:50.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.662+0000 7f4f9b4f9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f94102520 0x7f4f9419c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:50.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.662+0000 7f4f9b4f9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f94102e70 0x7f4f9419cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:50.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.662+0000 7f4f9b4f9640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f941037c0 0x7f4f941a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:50.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.662+0000 7f4f9b4f9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4f94111970 con 0x7f4f94102520 2024-05-09T00:48:50.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.662+0000 7f4f9b4f9640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4f941117f0 con 0x7f4f941037c0 2024-05-09T00:48:50.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.662+0000 7f4f9b4f9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4f94111af0 con 0x7f4f94102e70 2024-05-09T00:48:50.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.662+0000 7f4f99295640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f94102520 0x7f4f9419c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:50.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.662+0000 7f4f99295640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f94102520 0x7f4f9419c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35570/0 (socket says 172.21.15.5:35570) 2024-05-09T00:48:50.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.662+0000 7f4f99295640 1 -- 172.21.15.5:0/2983815253 learned_addr learned my addr 172.21.15.5:0/2983815253 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:50.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.662+0000 7f4f99a96640 1 --2- 172.21.15.5:0/2983815253 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f941037c0 0x7f4f941a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:50.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.662+0000 7f4f99295640 1 -- 172.21.15.5:0/2983815253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f94102e70 msgr2=0x7f4f9419cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:50.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.662+0000 7f4f98a94640 1 --2- 172.21.15.5:0/2983815253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f94102e70 0x7f4f9419cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:50.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.662+0000 7f4f99295640 1 --2- 172.21.15.5:0/2983815253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f94102e70 0x7f4f9419cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:50.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.662+0000 7f4f99295640 1 -- 172.21.15.5:0/2983815253 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f941037c0 msgr2=0x7f4f941a3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:50.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.662+0000 7f4f99295640 1 --2- 172.21.15.5:0/2983815253 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f941037c0 0x7f4f941a3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:50.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.663+0000 7f4f99295640 1 -- 172.21.15.5:0/2983815253 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f941a4110 con 0x7f4f94102520 2024-05-09T00:48:50.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.663+0000 7f4f99a96640 1 --2- 172.21.15.5:0/2983815253 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f941037c0 0x7f4f941a3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:48:50.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.663+0000 7f4f99295640 1 --2- 172.21.15.5:0/2983815253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f94102520 0x7f4f9419c5d0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f4f9000e9c0 tx=0x7f4f9000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:50.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.663+0000 7f4f8a7fc640 1 -- 172.21.15.5:0/2983815253 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f9000cd80 con 0x7f4f94102520 2024-05-09T00:48:50.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.663+0000 7f4f9b4f9640 1 -- 172.21.15.5:0/2983815253 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f941a4400 con 0x7f4f94102520 2024-05-09T00:48:50.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.663+0000 7f4f8a7fc640 1 -- 172.21.15.5:0/2983815253 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f900054d0 con 0x7f4f94102520 2024-05-09T00:48:50.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.663+0000 7f4f8a7fc640 1 -- 172.21.15.5:0/2983815253 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f90010430 con 0x7f4f94102520 2024-05-09T00:48:50.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.663+0000 7f4f9b4f9640 1 -- 172.21.15.5:0/2983815253 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f941a4950 con 0x7f4f94102520 2024-05-09T00:48:50.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.666+0000 7f4fa23aa740 1 Processor -- start 2024-05-09T00:48:50.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.666+0000 7f4fa23aa740 1 -- start start 2024-05-09T00:48:50.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.666+0000 7f4f8a7fc640 1 -- 172.21.15.5:0/2983815253 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f4f90004fb0 con 0x7f4f94102520 2024-05-09T00:48:50.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.666+0000 7f4fa23aa740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b15c4e1b0 0x559b15ff2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:50.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.666+0000 7f4fa23aa740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b15c252f0 0x559b15ff2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:50.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.666+0000 7f4fa23aa740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559b15c2d4d0 0x559b15ff9980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:50.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.666+0000 7f4fa23aa740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559b15f567c0 con 0x559b15c4e1b0 2024-05-09T00:48:50.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.666+0000 7f4fa23aa740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x559b15ef20e0 con 0x559b15c2d4d0 2024-05-09T00:48:50.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.666+0000 7f4fa23aa740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x559b15f52780 con 0x559b15c252f0 2024-05-09T00:48:50.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.666+0000 7f4f99a96640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b15c4e1b0 0x559b15ff2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:50.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.666+0000 7f4f99a96640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b15c4e1b0 0x559b15ff2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35574/0 (socket says 172.21.15.5:35574) 2024-05-09T00:48:50.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.666+0000 7f4f99a96640 1 -- 172.21.15.5:0/780835543 learned_addr learned my addr 172.21.15.5:0/780835543 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:50.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.666+0000 7f4f99295640 1 --2- 172.21.15.5:0/780835543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b15c252f0 0x559b15ff2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:50.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.666+0000 7f4f98a94640 1 --2- 172.21.15.5:0/780835543 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559b15c2d4d0 0x559b15ff9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:50.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.667+0000 7f4f99a96640 1 -- 172.21.15.5:0/780835543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b15c252f0 msgr2=0x559b15ff2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:50.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.667+0000 7f4f99a96640 1 --2- 172.21.15.5:0/780835543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b15c252f0 0x559b15ff2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:50.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.667+0000 7f4f99a96640 1 -- 172.21.15.5:0/780835543 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559b15c2d4d0 msgr2=0x559b15ff9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:50.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.667+0000 7f4f99a96640 1 --2- 172.21.15.5:0/780835543 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559b15c2d4d0 0x559b15ff9980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:50.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.667+0000 7f4f99a96640 1 -- 172.21.15.5:0/780835543 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b15f1b7c0 con 0x559b15c4e1b0 2024-05-09T00:48:50.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.667+0000 7f4f99295640 1 --2- 172.21.15.5:0/780835543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b15c252f0 0x559b15ff2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:48:50.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.667+0000 7f4f99a96640 1 --2- 172.21.15.5:0/780835543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b15c4e1b0 0x559b15ff2500 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c009e70 tx=0x7f4f8c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f4f6de01230bab3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:50.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.667+0000 7f4f8a7fc640 1 --2- 172.21.15.5:0/2983815253 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f4f800b27e0 0x7f4f800b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:50.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.667+0000 7f4f99295640 1 --2- 172.21.15.5:0/2983815253 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f4f800b27e0 0x7f4f800b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:50.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.667+0000 7f4f7f7fe640 1 -- 172.21.15.5:0/780835543 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c004d20 con 0x559b15c4e1b0 2024-05-09T00:48:50.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.667+0000 7f4f7f7fe640 1 -- 172.21.15.5:0/780835543 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f8c0054d0 con 0x559b15c4e1b0 2024-05-09T00:48:50.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.667+0000 7f4f7f7fe640 1 -- 172.21.15.5:0/780835543 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c0117c0 con 0x559b15c4e1b0 2024-05-09T00:48:50.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.667+0000 7f4f8a7fc640 1 -- 172.21.15.5:0/2983815253 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f4f900e27d0 con 0x7f4f94102520 2024-05-09T00:48:50.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.668+0000 7f4fa23aa740 1 -- 172.21.15.5:0/780835543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b15c4e1b0 msgr2=0x559b15ff2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:50.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.668+0000 7f4fa23aa740 1 --2- 172.21.15.5:0/780835543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b15c4e1b0 0x559b15ff2500 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c009e70 tx=0x7f4f8c00f730 comp rx=0 tx=0).stop 2024-05-09T00:48:50.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.668+0000 7f4f99295640 1 --2- 172.21.15.5:0/2983815253 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f4f800b27e0 0x7f4f800b4cf0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x559b15ff38e0 tx=0x7f4f9001fce0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:50.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.668+0000 7f4fa23aa740 1 -- 172.21.15.5:0/780835543 shutdown_connections 2024-05-09T00:48:50.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.668+0000 7f4fa23aa740 1 --2- 172.21.15.5:0/780835543 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559b15c2d4d0 0x559b15ff9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:50.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.668+0000 7f4fa23aa740 1 --2- 172.21.15.5:0/780835543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b15c252f0 0x559b15ff2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:50.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.668+0000 7f4fa23aa740 1 --2- 172.21.15.5:0/780835543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b15c4e1b0 0x559b15ff2500 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:50.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.668+0000 7f4fa23aa740 1 -- 172.21.15.5:0/780835543 >> 172.21.15.5:0/780835543 conn(0x559b15c42300 msgr2=0x559b15fede90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:50.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.668+0000 7f4fa23aa740 1 -- 172.21.15.5:0/780835543 shutdown_connections 2024-05-09T00:48:50.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.668+0000 7f4fa23aa740 1 -- 172.21.15.5:0/780835543 wait complete. 2024-05-09T00:48:50.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.668+0000 7f4fa23aa740 1 -- start start 2024-05-09T00:48:50.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.669+0000 7f4fa23aa740 1 Processor -- start 2024-05-09T00:48:50.678 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.669+0000 7f4fa23aa740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559b15c252f0 0x559b15ffc290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:50.678 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.669+0000 7f4fa23aa740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b15c2d4d0 0x559b15ffc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:50.678 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.669+0000 7f4fa23aa740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b15c4e1b0 0x559b16003730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:50.678 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.669+0000 7f4fa23aa740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559b15ffa3d0 con 0x559b15c2d4d0 2024-05-09T00:48:50.678 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.669+0000 7f4fa23aa740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x559b15edd9c0 con 0x559b15c252f0 2024-05-09T00:48:50.678 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.669+0000 7f4fa23aa740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x559b15ffa550 con 0x559b15c4e1b0 2024-05-09T00:48:50.678 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.669+0000 7f4fa23aa740 10 client.0 _ll_register_callbacks cb 0x559b15f0a020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:48:50.678 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.669+0000 7f4f98a94640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b15c2d4d0 0x559b15ffc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:50.678 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.669+0000 7f4f98a94640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b15c2d4d0 0x559b15ffc7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35590/0 (socket says 172.21.15.5:35590) 2024-05-09T00:48:50.678 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.669+0000 7f4f98a94640 1 -- 172.21.15.5:0/1414332196 learned_addr learned my addr 172.21.15.5:0/1414332196 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:50.678 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.669+0000 7f4f99a96640 1 --2- 172.21.15.5:0/1414332196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559b15c252f0 0x559b15ffc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:50.678 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.669+0000 7f4f99a96640 1 --2- 172.21.15.5:0/1414332196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b15c4e1b0 0x559b16003730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:50.678 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.670+0000 7f4f98a94640 1 -- 172.21.15.5:0/1414332196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b15c4e1b0 msgr2=0x559b16003730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:50.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.670+0000 7f4f98a94640 1 --2- 172.21.15.5:0/1414332196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b15c4e1b0 0x559b16003730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:50.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.670+0000 7f4f98a94640 1 -- 172.21.15.5:0/1414332196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559b15c252f0 msgr2=0x559b15ffc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:50.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.670+0000 7f4f98a94640 1 --2- 172.21.15.5:0/1414332196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559b15c252f0 0x559b15ffc290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:50.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.670+0000 7f4f98a94640 1 -- 172.21.15.5:0/1414332196 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b15f041f0 con 0x559b15c2d4d0 2024-05-09T00:48:50.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.670+0000 7f4f99a96640 1 --2- 172.21.15.5:0/1414332196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b15c4e1b0 0x559b16003730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:48:50.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.670+0000 7f4f98a94640 1 --2- 172.21.15.5:0/1414332196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b15c2d4d0 0x559b15ffc7d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f4f9419d670 tx=0x7f4f840315a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6de9d009eb0cf9e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:50.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.670+0000 7f4f7cff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:48:50.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.670+0000 7f4f7cff9640 1 -- 172.21.15.5:0/1414332196 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8403f070 con 0x559b15c2d4d0 2024-05-09T00:48:50.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.670+0000 7f4f7cff9640 1 -- 172.21.15.5:0/1414332196 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f4f84004240 con 0x559b15c2d4d0 2024-05-09T00:48:50.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.670+0000 7f4fa23aa740 1 -- 172.21.15.5:0/1414332196 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f8c0054d0 con 0x559b15c2d4d0 2024-05-09T00:48:50.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.670+0000 7f4f7cff9640 1 -- 172.21.15.5:0/1414332196 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4f84035070 con 0x559b15c2d4d0 2024-05-09T00:48:50.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.670+0000 7f4fa23aa740 10 client.15681 fetch_fsmap learned FSMap version 29 2024-05-09T00:48:50.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.670+0000 7f4fa23aa740 1 -- 172.21.15.5:0/1414332196 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559b15cb9790 con 0x559b15c2d4d0 2024-05-09T00:48:50.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.670+0000 7f4f7cff9640 1 -- 172.21.15.5:0/1414332196 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f8c0054d0 con 0x559b15c2d4d0 2024-05-09T00:48:50.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.671+0000 7f4f7cff9640 1 -- 172.21.15.5:0/1414332196 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f4f8404c070 con 0x559b15c2d4d0 2024-05-09T00:48:50.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.671+0000 7f4fa23aa740 10 client.15681 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:48:50.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.671+0000 7f4fa23aa740 10 client.15681 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:48:50.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.671+0000 7f4fa23aa740 20 client.15681 populate_metadata read hostname 'smithi005' 2024-05-09T00:48:50.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.671+0000 7f4fa23aa740 1 --2- 172.21.15.5:0/1414332196 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x559b15c0cd80 0x559b16009bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:50.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.671+0000 7f4fa23aa740 4 client.15681 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:48:50.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.671+0000 7f4fa23aa740 1 -- 172.21.15.5:0/1414332196 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b15f643d0 con 0x559b15c0cd80 2024-05-09T00:48:50.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.671+0000 7f4f99a96640 1 --2- 172.21.15.5:0/1414332196 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x559b15c0cd80 0x559b16009bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:50.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.671+0000 7f4f99a96640 1 --2- 172.21.15.5:0/1414332196 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x559b15c0cd80 0x559b16009bf0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0a35d9d08241ef1 server_cookie=85718bf7ad747077 in_seq=0 out_seq=0 2024-05-09T00:48:50.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.671+0000 7f4f7cff9640 10 client.15681 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:48:50.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.672+0000 7f4f7cff9640 1 -- 172.21.15.5:0/1414332196 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559b15f643d0 con 0x559b15c0cd80 2024-05-09T00:48:50.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.672+0000 7f4f7cff9640 10 client.15681 handle_command_reply: tid=0 2024-05-09T00:48:50.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.672+0000 7f4fa23aa740 1 client.15681 shutdown 2024-05-09T00:48:50.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.672+0000 7f4fa23aa740 1 -- 172.21.15.5:0/1414332196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b15c2d4d0 msgr2=0x559b15ffc7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:50.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.672+0000 7f4fa23aa740 1 --2- 172.21.15.5:0/1414332196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b15c2d4d0 0x559b15ffc7d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f4f9419d670 tx=0x7f4f840315a0 comp rx=0 tx=0).stop 2024-05-09T00:48:50.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.672+0000 7f4fa23aa740 1 -- 172.21.15.5:0/1414332196 shutdown_connections 2024-05-09T00:48:50.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.672+0000 7f4fa23aa740 1 --2- 172.21.15.5:0/1414332196 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x559b15c0cd80 0x559b16009bf0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:50.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.672+0000 7f4fa23aa740 1 --2- 172.21.15.5:0/1414332196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b15c4e1b0 0x559b16003730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:50.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.673+0000 7f4fa23aa740 1 --2- 172.21.15.5:0/1414332196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b15c2d4d0 0x559b15ffc7d0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:50.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.673+0000 7f4f7cff9640 0 client.15681 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:48:50.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.673+0000 7f4fa23aa740 1 --2- 172.21.15.5:0/1414332196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559b15c252f0 0x559b15ffc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:50.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.673+0000 7f4fa23aa740 1 -- 172.21.15.5:0/1414332196 >> 172.21.15.5:0/1414332196 conn(0x559b15c42300 msgr2=0x559b15ff0da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:50.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.673+0000 7f4fa23aa740 1 -- 172.21.15.5:0/1414332196 shutdown_connections 2024-05-09T00:48:50.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.673+0000 7f4fa23aa740 1 -- 172.21.15.5:0/1414332196 wait complete. 2024-05-09T00:48:50.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.673+0000 7f4fa23aa740 20 client.15681 trim_cache size 0 max 16384 2024-05-09T00:48:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.688+0000 7f4fa23aa740 1 Processor -- start 2024-05-09T00:48:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.688+0000 7f4fa23aa740 1 -- start start 2024-05-09T00:48:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.688+0000 7f4fa23aa740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b15c42300 0x559b15ff13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.688+0000 7f4fa23aa740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b15c252f0 0x559b15ff18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.688+0000 7f4fa23aa740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559b15c2d4d0 0x559b1600f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.688+0000 7f4fa23aa740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4f941ad760 con 0x559b15c252f0 2024-05-09T00:48:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.688+0000 7f4fa23aa740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4f941af2a0 con 0x559b15c2d4d0 2024-05-09T00:48:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.688+0000 7f4fa23aa740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4f941ade30 con 0x559b15c42300 2024-05-09T00:48:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.688+0000 7f4f98a94640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b15c252f0 0x559b15ff18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.688+0000 7f4f98a94640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b15c252f0 0x559b15ff18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35602/0 (socket says 172.21.15.5:35602) 2024-05-09T00:48:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.688+0000 7f4f98a94640 1 -- 172.21.15.5:0/2824604682 learned_addr learned my addr 172.21.15.5:0/2824604682 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.689+0000 7f4f99a96640 1 --2- 172.21.15.5:0/2824604682 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b15c42300 0x559b15ff13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.689+0000 7f4f98a94640 1 -- 172.21.15.5:0/2824604682 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b15c42300 msgr2=0x559b15ff13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.689+0000 7f4f98a94640 1 --2- 172.21.15.5:0/2824604682 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b15c42300 0x559b15ff13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.689+0000 7f4f98a94640 1 -- 172.21.15.5:0/2824604682 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559b15c2d4d0 msgr2=0x559b1600f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.689+0000 7f4f98a94640 1 --2- 172.21.15.5:0/2824604682 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559b15c2d4d0 0x559b1600f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.689+0000 7f4f98a94640 1 -- 172.21.15.5:0/2824604682 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b15d305f0 con 0x559b15c252f0 2024-05-09T00:48:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.689+0000 7f4f99a96640 1 --2- 172.21.15.5:0/2824604682 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b15c42300 0x559b15ff13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:48:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.689+0000 7f4f98a94640 1 --2- 172.21.15.5:0/2824604682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b15c252f0 0x559b15ff18e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f4f840096c0 tx=0x7f4f8402a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50313717dd6c8dcb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.689+0000 7f4f7effd640 1 -- 172.21.15.5:0/2824604682 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8403f040 con 0x559b15c252f0 2024-05-09T00:48:50.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.689+0000 7f4f7effd640 1 -- 172.21.15.5:0/2824604682 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f84049070 con 0x559b15c252f0 2024-05-09T00:48:50.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.689+0000 7f4f7effd640 1 -- 172.21.15.5:0/2824604682 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f84032070 con 0x559b15c252f0 2024-05-09T00:48:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.690+0000 7f4fa23aa740 1 -- 172.21.15.5:0/2824604682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b15c252f0 msgr2=0x559b15ff18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.690+0000 7f4fa23aa740 1 --2- 172.21.15.5:0/2824604682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b15c252f0 0x559b15ff18e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f4f840096c0 tx=0x7f4f8402a420 comp rx=0 tx=0).stop 2024-05-09T00:48:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.690+0000 7f4fa23aa740 1 -- 172.21.15.5:0/2824604682 shutdown_connections 2024-05-09T00:48:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.690+0000 7f4fa23aa740 1 --2- 172.21.15.5:0/2824604682 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559b15c2d4d0 0x559b1600f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:50.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.690+0000 7f4fa23aa740 1 --2- 172.21.15.5:0/2824604682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b15c252f0 0x559b15ff18e0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:50.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.690+0000 7f4fa23aa740 1 --2- 172.21.15.5:0/2824604682 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b15c42300 0x559b15ff13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:50.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.690+0000 7f4fa23aa740 1 -- 172.21.15.5:0/2824604682 >> 172.21.15.5:0/2824604682 conn(0x559b15ff6ef0 msgr2=0x559b15ff0c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:50.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.690+0000 7f4fa23aa740 1 -- 172.21.15.5:0/2824604682 shutdown_connections 2024-05-09T00:48:50.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.690+0000 7f4fa23aa740 1 -- 172.21.15.5:0/2824604682 wait complete. 2024-05-09T00:48:50.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.690+0000 7f4fa23aa740 1 -- start start 2024-05-09T00:48:50.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.690+0000 7f4fa23aa740 1 Processor -- start 2024-05-09T00:48:50.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.691+0000 7f4fa23aa740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b15c252f0 0x559b16000ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:50.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.691+0000 7f4fa23aa740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b15c2d4d0 0x559b160011e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.691+0000 7f4fa23aa740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559b15c42300 0x559b1606ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.691+0000 7f4fa23aa740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4f941aebd0 con 0x559b15c2d4d0 2024-05-09T00:48:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.691+0000 7f4fa23aa740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4f941ae500 con 0x559b15c42300 2024-05-09T00:48:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.691+0000 7f4fa23aa740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4f941ab740 con 0x559b15c252f0 2024-05-09T00:48:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.691+0000 7f4fa23aa740 10 client.0 _ll_register_callbacks cb 0x7f4f90005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:48:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.691+0000 7f4f98a94640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b15c2d4d0 0x559b160011e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.691+0000 7f4f98a94640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b15c2d4d0 0x559b160011e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35616/0 (socket says 172.21.15.5:35616) 2024-05-09T00:48:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.691+0000 7f4f98a94640 1 -- 172.21.15.5:0/3327297001 learned_addr learned my addr 172.21.15.5:0/3327297001 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.691+0000 7f4f99a96640 1 --2- 172.21.15.5:0/3327297001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b15c252f0 0x559b16000ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.691+0000 7f4f99a96640 1 --2- 172.21.15.5:0/3327297001 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559b15c42300 0x559b1606ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.691+0000 7f4f98a94640 1 -- 172.21.15.5:0/3327297001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b15c252f0 msgr2=0x559b16000ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.691+0000 7f4f98a94640 1 --2- 172.21.15.5:0/3327297001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b15c252f0 0x559b16000ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.691+0000 7f4f98a94640 1 -- 172.21.15.5:0/3327297001 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559b15c42300 msgr2=0x559b1606ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.691+0000 7f4f98a94640 1 --2- 172.21.15.5:0/3327297001 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559b15c42300 0x559b1606ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.691+0000 7f4f98a94640 1 -- 172.21.15.5:0/3327297001 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b15f1aed0 con 0x559b15c2d4d0 2024-05-09T00:48:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.691+0000 7f4f99a96640 1 --2- 172.21.15.5:0/3327297001 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559b15c42300 0x559b1606ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:48:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.691+0000 7f4f98a94640 1 --2- 172.21.15.5:0/3327297001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b15c2d4d0 0x559b160011e0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f4f8402a940 tx=0x7f4f84031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38a3fbd449468ea server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.691+0000 7f4f7cff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:48:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.692+0000 7f4f7cff9640 1 -- 172.21.15.5:0/3327297001 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8405d060 con 0x559b15c2d4d0 2024-05-09T00:48:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.692+0000 7f4f7cff9640 1 -- 172.21.15.5:0/3327297001 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f4f8404e490 con 0x559b15c2d4d0 2024-05-09T00:48:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.692+0000 7f4fa23aa740 1 -- 172.21.15.5:0/3327297001 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f84049070 con 0x559b15c2d4d0 2024-05-09T00:48:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.692+0000 7f4f7cff9640 1 -- 172.21.15.5:0/3327297001 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4f8403f030 con 0x559b15c2d4d0 2024-05-09T00:48:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.692+0000 7f4fa23aa740 10 client.15687 fetch_fsmap learned FSMap version 29 2024-05-09T00:48:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.692+0000 7f4fa23aa740 1 -- 172.21.15.5:0/3327297001 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559b160712b0 con 0x559b15c2d4d0 2024-05-09T00:48:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.692+0000 7f4f7cff9640 1 -- 172.21.15.5:0/3327297001 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f84049070 con 0x559b15c2d4d0 2024-05-09T00:48:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.692+0000 7f4f7cff9640 1 -- 172.21.15.5:0/3327297001 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f4f8404c070 con 0x559b15c2d4d0 2024-05-09T00:48:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.692+0000 7f4fa23aa740 10 client.15687 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:48:50.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.692+0000 7f4fa23aa740 10 client.15687 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:48:50.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.692+0000 7f4fa23aa740 20 client.15687 populate_metadata read hostname 'smithi005' 2024-05-09T00:48:50.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.692+0000 7f4fa23aa740 1 --2- 172.21.15.5:0/3327297001 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x559b15c4e1b0 0x559b16073bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:50.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.692+0000 7f4fa23aa740 4 client.15687 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:48:50.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.692+0000 7f4fa23aa740 1 -- 172.21.15.5:0/3327297001 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559b15f5e240 con 0x559b15c4e1b0 2024-05-09T00:48:50.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.693+0000 7f4f99a96640 1 --2- 172.21.15.5:0/3327297001 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x559b15c4e1b0 0x559b16073bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:50.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.693+0000 7f4f99a96640 1 --2- 172.21.15.5:0/3327297001 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x559b15c4e1b0 0x559b16073bc0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c65c4c071198eafb server_cookie=bc8df1432380169d in_seq=0 out_seq=0 2024-05-09T00:48:50.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.693+0000 7f4f7cff9640 10 client.15687 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:48:50.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.697+0000 7f4f7cff9640 1 -- 172.21.15.5:0/3327297001 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x559b15f5e240 con 0x559b15c4e1b0 2024-05-09T00:48:50.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.697+0000 7f4f7cff9640 10 client.15687 handle_command_reply: tid=0 2024-05-09T00:48:50.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.697+0000 7f4fa23aa740 1 client.15687 shutdown 2024-05-09T00:48:50.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.697+0000 7f4fa23aa740 1 -- 172.21.15.5:0/3327297001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b15c2d4d0 msgr2=0x559b160011e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:50.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.697+0000 7f4fa23aa740 1 --2- 172.21.15.5:0/3327297001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b15c2d4d0 0x559b160011e0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f4f8402a940 tx=0x7f4f84031400 comp rx=0 tx=0).stop 2024-05-09T00:48:50.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.697+0000 7f4fa23aa740 1 -- 172.21.15.5:0/3327297001 shutdown_connections 2024-05-09T00:48:50.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.697+0000 7f4fa23aa740 1 --2- 172.21.15.5:0/3327297001 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x559b15c4e1b0 0x559b16073bc0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:50.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.697+0000 7f4fa23aa740 1 --2- 172.21.15.5:0/3327297001 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559b15c42300 0x559b1606ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:50.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.697+0000 7f4fa23aa740 1 --2- 172.21.15.5:0/3327297001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b15c2d4d0 0x559b160011e0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:50.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.697+0000 7f4fa23aa740 1 --2- 172.21.15.5:0/3327297001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b15c252f0 0x559b16000ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:50.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.697+0000 7f4f7cff9640 0 client.15687 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:48:50.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.697+0000 7f4fa23aa740 1 -- 172.21.15.5:0/3327297001 >> 172.21.15.5:0/3327297001 conn(0x559b15ff6ef0 msgr2=0x559b15ff16c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:50.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.697+0000 7f4fa23aa740 1 -- 172.21.15.5:0/3327297001 shutdown_connections 2024-05-09T00:48:50.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.697+0000 7f4fa23aa740 1 -- 172.21.15.5:0/3327297001 wait complete. 2024-05-09T00:48:50.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.697+0000 7f4fa23aa740 20 client.15687 trim_cache size 0 max 16384 2024-05-09T00:48:50.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-09T00:48:50.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-09T00:48:50.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-09T00:48:50.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.698+0000 7f4f9b4f9640 1 -- 172.21.15.5:0/2983815253 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f4f800b27e0 msgr2=0x7f4f800b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:50.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.698+0000 7f4f9b4f9640 1 --2- 172.21.15.5:0/2983815253 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f4f800b27e0 0x7f4f800b4cf0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x559b15ff38e0 tx=0x7f4f9001fce0 comp rx=0 tx=0).stop 2024-05-09T00:48:50.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.698+0000 7f4f9b4f9640 1 -- 172.21.15.5:0/2983815253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f94102520 msgr2=0x7f4f9419c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:50.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.698+0000 7f4f9b4f9640 1 --2- 172.21.15.5:0/2983815253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f94102520 0x7f4f9419c5d0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f4f9000e9c0 tx=0x7f4f9000ee80 comp rx=0 tx=0).stop 2024-05-09T00:48:50.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.698+0000 7f4f9b4f9640 1 -- 172.21.15.5:0/2983815253 shutdown_connections 2024-05-09T00:48:50.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.698+0000 7f4f9b4f9640 1 --2- 172.21.15.5:0/2983815253 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f4f800b27e0 0x7f4f800b4cf0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:50.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.698+0000 7f4f9b4f9640 1 --2- 172.21.15.5:0/2983815253 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f941037c0 0x7f4f941a3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:50.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.698+0000 7f4f9b4f9640 1 --2- 172.21.15.5:0/2983815253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f94102e70 0x7f4f9419cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:50.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.698+0000 7f4f9b4f9640 1 --2- 172.21.15.5:0/2983815253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f94102520 0x7f4f9419c5d0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:50.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.698+0000 7f4f9b4f9640 1 -- 172.21.15.5:0/2983815253 >> 172.21.15.5:0/2983815253 conn(0x7f4f940fdd30 msgr2=0x7f4f9410a730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:50.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.698+0000 7f4f9b4f9640 1 -- 172.21.15.5:0/2983815253 shutdown_connections 2024-05-09T00:48:50.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:48:50.698+0000 7f4f9b4f9640 1 -- 172.21.15.5:0/2983815253 wait complete. 2024-05-09T00:48:51.136 INFO:tasks.workunit.client.0.smithi005.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-09T00:48:51.227 INFO:tasks.workunit.client.0.smithi005.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-09T00:48:51.269 INFO:tasks.workunit.client.0.smithi005.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-09T00:48:51.270 INFO:tasks.workunit.client.0.smithi005.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-09T00:48:51.271 INFO:tasks.workunit.client.0.smithi005.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-09T00:48:51.311 INFO:tasks.workunit.client.0.smithi005.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-09T00:48:51.315 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-09T00:48:51.315 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-05-09T00:48:51.373 INFO:tasks.workunit:Running workunits matching suites/iogen.sh on client.0... 2024-05-09T00:48:51.374 INFO:tasks.workunit:Running workunit suites/iogen.sh... 2024-05-09T00:48:51.374 DEBUG:teuthology.orchestra.run.smithi005:workunit test suites/iogen.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=c9da4049f638c0f84bdd9b4e33c42e86046ac554 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iogen.sh 2024-05-09T00:48:51.437 INFO:tasks.workunit.client.0.smithi005.stdout:getting iogen 2024-05-09T00:48:51.437 INFO:tasks.workunit.client.0.smithi005.stderr:+ echo 'getting iogen' 2024-05-09T00:48:51.437 INFO:tasks.workunit.client.0.smithi005.stderr:+ wget http://download.ceph.com/qa/iogen_3.1p0.tar 2024-05-09T00:48:51.468 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:48:51 smithi005 ceph-mon[30365]: pgmap v446: 97 pgs: 97 active+clean; 671 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-09T00:48:51.469 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:48:51 smithi005 ceph-mon[30365]: scrub summary: idle 2024-05-09T00:48:51.481 INFO:tasks.workunit.client.0.smithi005.stderr:--2024-05-09 00:48:51-- http://download.ceph.com/qa/iogen_3.1p0.tar 2024-05-09T00:48:51.481 INFO:tasks.workunit.client.0.smithi005.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-05-09T00:48:51.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.490+0000 7fbfcd920700 1 -- 172.21.15.5:0/3962033133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfc81034c0 msgr2=0x7fbfc81038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:51.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.490+0000 7fbfcd920700 1 --2- 172.21.15.5:0/3962033133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfc81034c0 0x7fbfc81038d0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fbfb80099c0 tx=0x7fbfb8009cd0 comp rx=0 tx=0).stop 2024-05-09T00:48:51.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.490+0000 7fbfcd920700 1 -- 172.21.15.5:0/3962033133 shutdown_connections 2024-05-09T00:48:51.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.490+0000 7fbfcd920700 1 --2- 172.21.15.5:0/3962033133 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfc8108740 0x7fbfc81100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:51.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.490+0000 7fbfcd920700 1 --2- 172.21.15.5:0/3962033133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfc81034c0 0x7fbfc81038d0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:51.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.490+0000 7fbfcd920700 1 --2- 172.21.15.5:0/3962033133 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbfc8107df0 0x7fbfc8108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:51.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.490+0000 7fbfcd920700 1 -- 172.21.15.5:0/3962033133 >> 172.21.15.5:0/3962033133 conn(0x7fbfc80fedf0 msgr2=0x7fbfc8101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:51.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.490+0000 7fbfcd920700 1 -- 172.21.15.5:0/3962033133 shutdown_connections 2024-05-09T00:48:51.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.490+0000 7fbfcd920700 1 -- 172.21.15.5:0/3962033133 wait complete. 2024-05-09T00:48:51.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.491+0000 7fbfcd920700 1 Processor -- start 2024-05-09T00:48:51.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.491+0000 7fbfcd920700 1 -- start start 2024-05-09T00:48:51.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.491+0000 7fbfcd920700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfc81034c0 0x7fbfc819aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:51.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.491+0000 7fbfcd920700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfc8107df0 0x7fbfc819b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:51.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.491+0000 7fbfcd920700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbfc8108740 0x7fbfc819f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:51.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.491+0000 7fbfcd920700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc81144c0 con 0x7fbfc81034c0 2024-05-09T00:48:51.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.491+0000 7fbfcd920700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc8114340 con 0x7fbfc8108740 2024-05-09T00:48:51.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.491+0000 7fbfcd920700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc8114640 con 0x7fbfc8107df0 2024-05-09T00:48:51.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.491+0000 7fbfc6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfc81034c0 0x7fbfc819aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:51.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.491+0000 7fbfc6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfc81034c0 0x7fbfc819aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35638/0 (socket says 172.21.15.5:35638) 2024-05-09T00:48:51.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.491+0000 7fbfc6ffd700 1 -- 172.21.15.5:0/3594419694 learned_addr learned my addr 172.21.15.5:0/3594419694 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:51.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.491+0000 7fbfc77fe700 1 --2- 172.21.15.5:0/3594419694 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbfc8108740 0x7fbfc819f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:51.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.491+0000 7fbfc67fc700 1 --2- 172.21.15.5:0/3594419694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfc8107df0 0x7fbfc819b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:51.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.492+0000 7fbfc6ffd700 1 -- 172.21.15.5:0/3594419694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfc8107df0 msgr2=0x7fbfc819b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:51.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.492+0000 7fbfc6ffd700 1 --2- 172.21.15.5:0/3594419694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfc8107df0 0x7fbfc819b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:51.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.492+0000 7fbfc6ffd700 1 -- 172.21.15.5:0/3594419694 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbfc8108740 msgr2=0x7fbfc819f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:51.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.492+0000 7fbfc6ffd700 1 --2- 172.21.15.5:0/3594419694 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbfc8108740 0x7fbfc819f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:51.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.492+0000 7fbfc6ffd700 1 -- 172.21.15.5:0/3594419694 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfc819fd80 con 0x7fbfc81034c0 2024-05-09T00:48:51.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.492+0000 7fbfc67fc700 1 --2- 172.21.15.5:0/3594419694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfc8107df0 0x7fbfc819b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:48:51.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.492+0000 7fbfc6ffd700 1 --2- 172.21.15.5:0/3594419694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfc81034c0 0x7fbfc819aeb0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fbfb800b580 tx=0x7fbfb80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:51.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.492+0000 7fbfcc91e700 1 -- 172.21.15.5:0/3594419694 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb8027070 con 0x7fbfc81034c0 2024-05-09T00:48:51.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.492+0000 7fbfcc91e700 1 -- 172.21.15.5:0/3594419694 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfb8003f20 con 0x7fbfc81034c0 2024-05-09T00:48:51.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.492+0000 7fbfcd920700 1 -- 172.21.15.5:0/3594419694 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfc81a0010 con 0x7fbfc81034c0 2024-05-09T00:48:51.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.492+0000 7fbfcc91e700 1 -- 172.21.15.5:0/3594419694 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb8022d70 con 0x7fbfc81034c0 2024-05-09T00:48:51.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.492+0000 7fbfcd920700 1 -- 172.21.15.5:0/3594419694 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfc81a04f0 con 0x7fbfc81034c0 2024-05-09T00:48:51.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.495+0000 7fbfcc91e700 1 -- 172.21.15.5:0/3594419694 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fbfb8022500 con 0x7fbfc81034c0 2024-05-09T00:48:51.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.496+0000 7fbfcc91e700 1 --2- 172.21.15.5:0/3594419694 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fbfb00b2880 0x7fbfb00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:51.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.496+0000 7fbfc67fc700 1 --2- 172.21.15.5:0/3594419694 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fbfb00b2880 0x7fbfb00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:51.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.496+0000 7fbfcc91e700 1 -- 172.21.15.5:0/3594419694 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fbfb80e2f40 con 0x7fbfc81034c0 2024-05-09T00:48:51.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.496+0000 7fbfdc336b80 1 Processor -- start 2024-05-09T00:48:51.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.496+0000 7fbfdc336b80 1 -- start start 2024-05-09T00:48:51.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.496+0000 7fbfc67fc700 1 --2- 172.21.15.5:0/3594419694 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fbfb00b2880 0x7fbfb00b4d90 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc005950 tx=0x7fbfbc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:51.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.496+0000 7fbfdc336b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558532f0e160 0x558532fa74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:51.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.496+0000 7fbfdc336b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558532e244a0 0x558532fa7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:51.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.496+0000 7fbfdc336b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558532fa16a0 0x558532fa1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:51.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.496+0000 7fbfdc336b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558532ec3ff0 con 0x558532f0e160 2024-05-09T00:48:51.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.496+0000 7fbfdc336b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x558532e627f0 con 0x558532e244a0 2024-05-09T00:48:51.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.496+0000 7fbfdc336b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x558532ebffb0 con 0x558532fa16a0 2024-05-09T00:48:51.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.496+0000 7fbfc6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558532f0e160 0x558532fa74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:51.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.496+0000 7fbfc6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558532f0e160 0x558532fa74f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35640/0 (socket says 172.21.15.5:35640) 2024-05-09T00:48:51.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.496+0000 7fbfc6ffd700 1 -- 172.21.15.5:0/1657092219 learned_addr learned my addr 172.21.15.5:0/1657092219 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:51.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.497+0000 7fbfc67fc700 1 --2- 172.21.15.5:0/1657092219 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558532e244a0 0x558532fa7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:51.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.497+0000 7fbfc77fe700 1 --2- 172.21.15.5:0/1657092219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558532fa16a0 0x558532fa1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:51.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.497+0000 7fbfc6ffd700 1 -- 172.21.15.5:0/1657092219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558532fa16a0 msgr2=0x558532fa1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:51.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.497+0000 7fbfc6ffd700 1 --2- 172.21.15.5:0/1657092219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558532fa16a0 0x558532fa1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:51.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.497+0000 7fbfc6ffd700 1 -- 172.21.15.5:0/1657092219 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558532e244a0 msgr2=0x558532fa7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:51.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.497+0000 7fbfc6ffd700 1 --2- 172.21.15.5:0/1657092219 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558532e244a0 0x558532fa7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:51.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.497+0000 7fbfc6ffd700 1 -- 172.21.15.5:0/1657092219 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558532e5a370 con 0x558532f0e160 2024-05-09T00:48:51.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.497+0000 7fbfc77fe700 1 --2- 172.21.15.5:0/1657092219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558532fa16a0 0x558532fa1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:48:51.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.497+0000 7fbfc6ffd700 1 --2- 172.21.15.5:0/1657092219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558532f0e160 0x558532fa74f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8000c00 tx=0x7fbfb8009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32cab349e85768ca server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:51.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.497+0000 7fbfadffb700 1 -- 172.21.15.5:0/1657092219 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb80ae910 con 0x558532f0e160 2024-05-09T00:48:51.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.497+0000 7fbfadffb700 1 -- 172.21.15.5:0/1657092219 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfb80b3330 con 0x558532f0e160 2024-05-09T00:48:51.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.497+0000 7fbfadffb700 1 -- 172.21.15.5:0/1657092219 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb80a9a20 con 0x558532f0e160 2024-05-09T00:48:51.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.498+0000 7fbfdc336b80 1 -- 172.21.15.5:0/1657092219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558532f0e160 msgr2=0x558532fa74f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:51.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.498+0000 7fbfdc336b80 1 --2- 172.21.15.5:0/1657092219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558532f0e160 0x558532fa74f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8000c00 tx=0x7fbfb8009cd0 comp rx=0 tx=0).stop 2024-05-09T00:48:51.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.498+0000 7fbfdc336b80 1 -- 172.21.15.5:0/1657092219 shutdown_connections 2024-05-09T00:48:51.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.498+0000 7fbfdc336b80 1 --2- 172.21.15.5:0/1657092219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558532fa16a0 0x558532fa1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:51.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.498+0000 7fbfdc336b80 1 --2- 172.21.15.5:0/1657092219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558532f0e160 0x558532fa74f0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:51.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.498+0000 7fbfdc336b80 1 --2- 172.21.15.5:0/1657092219 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558532e244a0 0x558532fa7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:51.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.498+0000 7fbfdc336b80 1 -- 172.21.15.5:0/1657092219 >> 172.21.15.5:0/1657092219 conn(0x558532e846a0 msgr2=0x558532f9c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:51.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.498+0000 7fbfdc336b80 1 -- 172.21.15.5:0/1657092219 shutdown_connections 2024-05-09T00:48:51.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.498+0000 7fbfdc336b80 1 -- 172.21.15.5:0/1657092219 wait complete. 2024-05-09T00:48:51.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.499+0000 7fbfdc336b80 1 -- start start 2024-05-09T00:48:51.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.499+0000 7fbfdc336b80 1 Processor -- start 2024-05-09T00:48:51.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.499+0000 7fbfdc336b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558532e244a0 0x558532fb0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:51.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.499+0000 7fbfdc336b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558532f0e160 0x558532fb12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:51.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.499+0000 7fbfdc336b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558532fa16a0 0x558532fab120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:51.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.499+0000 7fbfdc336b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558532d3fba0 con 0x558532fa16a0 2024-05-09T00:48:51.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.499+0000 7fbfdc336b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x558532e7e150 con 0x558532e244a0 2024-05-09T00:48:51.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.499+0000 7fbfdc336b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x558532fa2360 con 0x558532f0e160 2024-05-09T00:48:51.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.499+0000 7fbfdc336b80 10 client.0 _ll_register_callbacks cb 0x558532d356c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:48:51.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.499+0000 7fbfc77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558532fa16a0 0x558532fab120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:51.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.500+0000 7fbfc6ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558532e244a0 0x558532fb0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:51.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.500+0000 7fbfc77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558532fa16a0 0x558532fab120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35646/0 (socket says 172.21.15.5:35646) 2024-05-09T00:48:51.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.500+0000 7fbfc77fe700 1 -- 172.21.15.5:0/3244301537 learned_addr learned my addr 172.21.15.5:0/3244301537 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:51.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.500+0000 7fbfc67fc700 1 --2- 172.21.15.5:0/3244301537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558532f0e160 0x558532fb12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:51.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.500+0000 7fbfc77fe700 1 -- 172.21.15.5:0/3244301537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558532f0e160 msgr2=0x558532fb12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:51.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.500+0000 7fbfc77fe700 1 --2- 172.21.15.5:0/3244301537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558532f0e160 0x558532fb12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:51.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.500+0000 7fbfc77fe700 1 -- 172.21.15.5:0/3244301537 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558532e244a0 msgr2=0x558532fb0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:51.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.500+0000 7fbfc77fe700 1 --2- 172.21.15.5:0/3244301537 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558532e244a0 0x558532fb0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:51.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.500+0000 7fbfc77fe700 1 -- 172.21.15.5:0/3244301537 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558532d3f490 con 0x558532fa16a0 2024-05-09T00:48:51.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.500+0000 7fbfc6ffd700 1 --2- 172.21.15.5:0/3244301537 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558532e244a0 0x558532fb0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:48:51.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.500+0000 7fbfc77fe700 1 --2- 172.21.15.5:0/3244301537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558532fa16a0 0x558532fab120 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0009f60 tx=0x7fbfc0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b42d2d0133619d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:51.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.500+0000 7fbfa6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:48:51.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.500+0000 7fbfa6ffd700 1 -- 172.21.15.5:0/3244301537 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc0019020 con 0x558532fa16a0 2024-05-09T00:48:51.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.500+0000 7fbfa6ffd700 1 -- 172.21.15.5:0/3244301537 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fbfc0007710 con 0x558532fa16a0 2024-05-09T00:48:51.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.500+0000 7fbfdc336b80 1 -- 172.21.15.5:0/3244301537 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfb80b3330 con 0x558532fa16a0 2024-05-09T00:48:51.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.501+0000 7fbfa6ffd700 1 -- 172.21.15.5:0/3244301537 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbfc0004bb0 con 0x558532fa16a0 2024-05-09T00:48:51.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.501+0000 7fbfdc336b80 10 client.15699 fetch_fsmap learned FSMap version 29 2024-05-09T00:48:51.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.501+0000 7fbfdc336b80 1 -- 172.21.15.5:0/3244301537 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558532fb8910 con 0x558532fa16a0 2024-05-09T00:48:51.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.501+0000 7fbfa6ffd700 1 -- 172.21.15.5:0/3244301537 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfb80b3330 con 0x558532fa16a0 2024-05-09T00:48:51.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.501+0000 7fbfa6ffd700 1 -- 172.21.15.5:0/3244301537 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fbfc00052d0 con 0x558532fa16a0 2024-05-09T00:48:51.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.501+0000 7fbfdc336b80 10 client.15699 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:48:51.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.501+0000 7fbfdc336b80 10 client.15699 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:48:51.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.501+0000 7fbfdc336b80 20 client.15699 populate_metadata read hostname 'smithi005' 2024-05-09T00:48:51.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.501+0000 7fbfdc336b80 1 --2- 172.21.15.5:0/3244301537 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x558532fbafa0 0x558532fbd400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:51.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.501+0000 7fbfdc336b80 4 client.15699 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:48:51.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.501+0000 7fbfdc336b80 1 -- 172.21.15.5:0/3244301537 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558532ed19d0 con 0x558532fbafa0 2024-05-09T00:48:51.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.501+0000 7fbfc6ffd700 1 --2- 172.21.15.5:0/3244301537 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x558532fbafa0 0x558532fbd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:51.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.502+0000 7fbfa6ffd700 10 client.15699 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:48:51.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.502+0000 7fbfc6ffd700 1 --2- 172.21.15.5:0/3244301537 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x558532fbafa0 0x558532fbd400 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3deb001fa6ff2c10 server_cookie=cb03161943368ef4 in_seq=0 out_seq=0 2024-05-09T00:48:51.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.502+0000 7fbfa6ffd700 1 -- 172.21.15.5:0/3244301537 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558532ed19d0 con 0x558532fbafa0 2024-05-09T00:48:51.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.502+0000 7fbfa6ffd700 10 client.15699 handle_command_reply: tid=0 2024-05-09T00:48:51.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.503+0000 7fbfdc336b80 1 client.15699 shutdown 2024-05-09T00:48:51.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.503+0000 7fbfdc336b80 1 -- 172.21.15.5:0/3244301537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558532fa16a0 msgr2=0x558532fab120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:51.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.503+0000 7fbfdc336b80 1 --2- 172.21.15.5:0/3244301537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558532fa16a0 0x558532fab120 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0009f60 tx=0x7fbfc0009f90 comp rx=0 tx=0).stop 2024-05-09T00:48:51.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.503+0000 7fbfdc336b80 1 -- 172.21.15.5:0/3244301537 shutdown_connections 2024-05-09T00:48:51.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.503+0000 7fbfdc336b80 1 --2- 172.21.15.5:0/3244301537 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x558532fbafa0 0x558532fbd400 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:51.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.503+0000 7fbfdc336b80 1 --2- 172.21.15.5:0/3244301537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558532fa16a0 0x558532fab120 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:51.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.503+0000 7fbfdc336b80 1 --2- 172.21.15.5:0/3244301537 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558532e244a0 0x558532fb0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:51.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.503+0000 7fbfdc336b80 1 --2- 172.21.15.5:0/3244301537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558532f0e160 0x558532fb12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:51.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.503+0000 7fbfdc336b80 1 -- 172.21.15.5:0/3244301537 >> 172.21.15.5:0/3244301537 conn(0x558532e846a0 msgr2=0x558532f9c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:51.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.503+0000 7fbfa6ffd700 0 client.15699 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:48:51.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.503+0000 7fbfdc336b80 1 -- 172.21.15.5:0/3244301537 shutdown_connections 2024-05-09T00:48:51.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.503+0000 7fbfdc336b80 1 -- 172.21.15.5:0/3244301537 wait complete. 2024-05-09T00:48:51.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.503+0000 7fbfdc336b80 20 client.15699 trim_cache size 0 max 16384 2024-05-09T00:48:51.517 INFO:tasks.workunit.client.0.smithi005.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-05-09T00:48:51.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.524+0000 7fbfdc336b80 1 Processor -- start 2024-05-09T00:48:51.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.524+0000 7fbfdc336b80 1 -- start start 2024-05-09T00:48:51.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.524+0000 7fbfdc336b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558532e244a0 0x558532fc2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:51.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.524+0000 7fbfdc336b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558532f0e160 0x558532fc3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:51.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.524+0000 7fbfdc336b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558532fa16a0 0x558532faa9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:51.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.524+0000 7fbfdc336b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc81aca00 con 0x558532f0e160 2024-05-09T00:48:51.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.524+0000 7fbfdc336b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc81ae540 con 0x558532e244a0 2024-05-09T00:48:51.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.524+0000 7fbfdc336b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc81ad0d0 con 0x558532fa16a0 2024-05-09T00:48:51.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.524+0000 7fbfc67fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558532f0e160 0x558532fc3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:51.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.524+0000 7fbfc67fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558532f0e160 0x558532fc3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35662/0 (socket says 172.21.15.5:35662) 2024-05-09T00:48:51.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.524+0000 7fbfc67fc700 1 -- 172.21.15.5:0/3027876287 learned_addr learned my addr 172.21.15.5:0/3027876287 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:51.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.524+0000 7fbfc77fe700 1 --2- 172.21.15.5:0/3027876287 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558532fa16a0 0x558532faa9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:51.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.524+0000 7fbfc6ffd700 1 --2- 172.21.15.5:0/3027876287 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558532e244a0 0x558532fc2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:51.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.524+0000 7fbfc67fc700 1 -- 172.21.15.5:0/3027876287 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558532fa16a0 msgr2=0x558532faa9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:51.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.524+0000 7fbfc67fc700 1 --2- 172.21.15.5:0/3027876287 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558532fa16a0 0x558532faa9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:51.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.524+0000 7fbfc67fc700 1 -- 172.21.15.5:0/3027876287 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558532e244a0 msgr2=0x558532fc2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:51.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.524+0000 7fbfc67fc700 1 --2- 172.21.15.5:0/3027876287 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558532e244a0 0x558532fc2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:51.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.525+0000 7fbfc67fc700 1 -- 172.21.15.5:0/3027876287 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558532f9dd00 con 0x558532f0e160 2024-05-09T00:48:51.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.525+0000 7fbfc67fc700 1 --2- 172.21.15.5:0/3027876287 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558532f0e160 0x558532fc3140 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fbfc81066e0 tx=0x7fbfbc012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b476050b8eb51e1f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:51.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.525+0000 7fbfad7fa700 1 -- 172.21.15.5:0/3027876287 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc003880 con 0x558532f0e160 2024-05-09T00:48:51.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.525+0000 7fbfad7fa700 1 -- 172.21.15.5:0/3027876287 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfbc011440 con 0x558532f0e160 2024-05-09T00:48:51.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.525+0000 7fbfad7fa700 1 -- 172.21.15.5:0/3027876287 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc020670 con 0x558532f0e160 2024-05-09T00:48:51.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.526+0000 7fbfdc336b80 1 -- 172.21.15.5:0/3027876287 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558532f0e160 msgr2=0x558532fc3140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:51.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.526+0000 7fbfdc336b80 1 --2- 172.21.15.5:0/3027876287 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558532f0e160 0x558532fc3140 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fbfc81066e0 tx=0x7fbfbc012000 comp rx=0 tx=0).stop 2024-05-09T00:48:51.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.526+0000 7fbfdc336b80 1 -- 172.21.15.5:0/3027876287 shutdown_connections 2024-05-09T00:48:51.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.526+0000 7fbfdc336b80 1 --2- 172.21.15.5:0/3027876287 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558532fa16a0 0x558532faa9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:51.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.526+0000 7fbfdc336b80 1 --2- 172.21.15.5:0/3027876287 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558532e244a0 0x558532fc2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:51.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.526+0000 7fbfdc336b80 1 --2- 172.21.15.5:0/3027876287 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558532f0e160 0x558532fc3140 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:51.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.526+0000 7fbfdc336b80 1 -- 172.21.15.5:0/3027876287 >> 172.21.15.5:0/3027876287 conn(0x558532e846a0 msgr2=0x558532fa9a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:51.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.526+0000 7fbfdc336b80 1 -- 172.21.15.5:0/3027876287 shutdown_connections 2024-05-09T00:48:51.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.526+0000 7fbfdc336b80 1 -- 172.21.15.5:0/3027876287 wait complete. 2024-05-09T00:48:51.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.526+0000 7fbfdc336b80 1 -- start start 2024-05-09T00:48:51.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.527+0000 7fbfdc336b80 1 Processor -- start 2024-05-09T00:48:51.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.527+0000 7fbfdc336b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558532e244a0 0x558532fb4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:51.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.527+0000 7fbfdc336b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558532f0e160 0x558532fb5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:51.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.527+0000 7fbfdc336b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558532fa16a0 0x558532fb58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:51.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.527+0000 7fbfdc336b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc81ade70 con 0x558532f0e160 2024-05-09T00:48:51.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.527+0000 7fbfdc336b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc81ad7a0 con 0x558532fa16a0 2024-05-09T00:48:51.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.527+0000 7fbfdc336b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc81aa9a0 con 0x558532e244a0 2024-05-09T00:48:51.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.527+0000 7fbfdc336b80 10 client.0 _ll_register_callbacks cb 0x558532d356c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:48:51.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.527+0000 7fbfc67fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558532f0e160 0x558532fb5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:51.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.527+0000 7fbfc6ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558532e244a0 0x558532fb4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:51.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.527+0000 7fbfc67fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558532f0e160 0x558532fb5360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35664/0 (socket says 172.21.15.5:35664) 2024-05-09T00:48:51.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.527+0000 7fbfc67fc700 1 -- 172.21.15.5:0/2825338470 learned_addr learned my addr 172.21.15.5:0/2825338470 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:48:51.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.527+0000 7fbfc77fe700 1 --2- 172.21.15.5:0/2825338470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558532fa16a0 0x558532fb58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:51.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.527+0000 7fbfc67fc700 1 -- 172.21.15.5:0/2825338470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558532e244a0 msgr2=0x558532fb4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:51.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.527+0000 7fbfc67fc700 1 --2- 172.21.15.5:0/2825338470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558532e244a0 0x558532fb4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:51.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.527+0000 7fbfc67fc700 1 -- 172.21.15.5:0/2825338470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558532fa16a0 msgr2=0x558532fb58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:48:51.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.527+0000 7fbfc67fc700 1 --2- 172.21.15.5:0/2825338470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558532fa16a0 0x558532fb58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:51.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.527+0000 7fbfc67fc700 1 -- 172.21.15.5:0/2825338470 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585330247f0 con 0x558532f0e160 2024-05-09T00:48:51.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.528+0000 7fbfc67fc700 1 --2- 172.21.15.5:0/2825338470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558532f0e160 0x558532fb5360 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc011820 tx=0x7fbfbc011850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baaeb9a05f819693 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:48:51.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.528+0000 7fbfa6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:48:51.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.528+0000 7fbfa6ffd700 1 -- 172.21.15.5:0/2825338470 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc003890 con 0x558532f0e160 2024-05-09T00:48:51.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.528+0000 7fbfa6ffd700 1 -- 172.21.15.5:0/2825338470 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fbfbc003a30 con 0x558532f0e160 2024-05-09T00:48:51.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.528+0000 7fbfdc336b80 1 -- 172.21.15.5:0/2825338470 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfbc011440 con 0x558532f0e160 2024-05-09T00:48:51.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.528+0000 7fbfa6ffd700 1 -- 172.21.15.5:0/2825338470 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbfbc003c70 con 0x558532f0e160 2024-05-09T00:48:51.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.528+0000 7fbfdc336b80 10 client.15705 fetch_fsmap learned FSMap version 29 2024-05-09T00:48:51.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.528+0000 7fbfdc336b80 1 -- 172.21.15.5:0/2825338470 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558533025a40 con 0x558532f0e160 2024-05-09T00:48:51.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.528+0000 7fbfa6ffd700 1 -- 172.21.15.5:0/2825338470 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfbc011440 con 0x558532f0e160 2024-05-09T00:48:51.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.528+0000 7fbfa6ffd700 1 -- 172.21.15.5:0/2825338470 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fbfbc040ae0 con 0x558532f0e160 2024-05-09T00:48:51.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.528+0000 7fbfdc336b80 10 client.15705 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:48:51.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.529+0000 7fbfdc336b80 10 client.15705 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:48:51.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.529+0000 7fbfdc336b80 20 client.15705 populate_metadata read hostname 'smithi005' 2024-05-09T00:48:51.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.529+0000 7fbfdc336b80 1 --2- 172.21.15.5:0/2825338470 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x558533026040 0x558533028460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:48:51.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.529+0000 7fbfdc336b80 4 client.15705 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:48:51.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.529+0000 7fbfdc336b80 1 -- 172.21.15.5:0/2825338470 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558532ecba70 con 0x558533026040 2024-05-09T00:48:51.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.529+0000 7fbfc6ffd700 1 --2- 172.21.15.5:0/2825338470 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x558533026040 0x558533028460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:48:51.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.529+0000 7fbfc6ffd700 1 --2- 172.21.15.5:0/2825338470 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x558533026040 0x558533028460 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fb8f913b2ef591a server_cookie=115188933a7506e8 in_seq=0 out_seq=0 2024-05-09T00:48:51.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.529+0000 7fbfa6ffd700 10 client.15705 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:48:51.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.529+0000 7fbfa6ffd700 1 -- 172.21.15.5:0/2825338470 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558532ecba70 con 0x558533026040 2024-05-09T00:48:51.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.529+0000 7fbfa6ffd700 10 client.15705 handle_command_reply: tid=0 2024-05-09T00:48:51.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.529+0000 7fbfdc336b80 1 client.15705 shutdown 2024-05-09T00:48:51.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.530+0000 7fbfdc336b80 1 -- 172.21.15.5:0/2825338470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558532f0e160 msgr2=0x558532fb5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:51.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.530+0000 7fbfdc336b80 1 --2- 172.21.15.5:0/2825338470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558532f0e160 0x558532fb5360 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc011820 tx=0x7fbfbc011850 comp rx=0 tx=0).stop 2024-05-09T00:48:51.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.530+0000 7fbfdc336b80 1 -- 172.21.15.5:0/2825338470 shutdown_connections 2024-05-09T00:48:51.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.530+0000 7fbfdc336b80 1 --2- 172.21.15.5:0/2825338470 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x558533026040 0x558533028460 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:51.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.530+0000 7fbfdc336b80 1 --2- 172.21.15.5:0/2825338470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558532fa16a0 0x558532fb58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:51.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.530+0000 7fbfdc336b80 1 --2- 172.21.15.5:0/2825338470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558532e244a0 0x558532fb4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:51.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.530+0000 7fbfdc336b80 1 --2- 172.21.15.5:0/2825338470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558532f0e160 0x558532fb5360 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:51.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.530+0000 7fbfdc336b80 1 -- 172.21.15.5:0/2825338470 >> 172.21.15.5:0/2825338470 conn(0x558532e846a0 msgr2=0x558532faa9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:51.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.530+0000 7fbfa6ffd700 0 client.15705 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:48:51.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.530+0000 7fbfdc336b80 1 -- 172.21.15.5:0/2825338470 shutdown_connections 2024-05-09T00:48:51.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.530+0000 7fbfdc336b80 1 -- 172.21.15.5:0/2825338470 wait complete. 2024-05-09T00:48:51.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.530+0000 7fbfdc336b80 20 client.15705 trim_cache size 0 max 16384 2024-05-09T00:48:51.535 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-09T00:48:51.535 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-05-09T00:48:51.536 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-05-09T00:48:51.536 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-09T00:48:51.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.530+0000 7fbfcd920700 1 -- 172.21.15.5:0/3594419694 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fbfb00b2880 msgr2=0x7fbfb00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:51.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.530+0000 7fbfcd920700 1 --2- 172.21.15.5:0/3594419694 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fbfb00b2880 0x7fbfb00b4d90 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc005950 tx=0x7fbfbc0058e0 comp rx=0 tx=0).stop 2024-05-09T00:48:51.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.531+0000 7fbfcd920700 1 -- 172.21.15.5:0/3594419694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfc81034c0 msgr2=0x7fbfc819aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:48:51.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.531+0000 7fbfcd920700 1 --2- 172.21.15.5:0/3594419694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfc81034c0 0x7fbfc819aeb0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fbfb800b580 tx=0x7fbfb80047b0 comp rx=0 tx=0).stop 2024-05-09T00:48:51.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.531+0000 7fbfcd920700 1 -- 172.21.15.5:0/3594419694 shutdown_connections 2024-05-09T00:48:51.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.531+0000 7fbfcd920700 1 --2- 172.21.15.5:0/3594419694 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fbfb00b2880 0x7fbfb00b4d90 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:51.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.531+0000 7fbfcd920700 1 --2- 172.21.15.5:0/3594419694 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbfc8108740 0x7fbfc819f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:51.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.531+0000 7fbfcd920700 1 --2- 172.21.15.5:0/3594419694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfc81034c0 0x7fbfc819aeb0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:51.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.531+0000 7fbfcd920700 1 --2- 172.21.15.5:0/3594419694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfc8107df0 0x7fbfc819b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:48:51.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.531+0000 7fbfcd920700 1 -- 172.21.15.5:0/3594419694 >> 172.21.15.5:0/3594419694 conn(0x7fbfc80fedf0 msgr2=0x7fbfc81096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:48:51.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.531+0000 7fbfcd920700 1 -- 172.21.15.5:0/3594419694 shutdown_connections 2024-05-09T00:48:51.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:48:51.531+0000 7fbfcd920700 1 -- 172.21.15.5:0/3594419694 wait complete. 2024-05-09T00:48:51.553 INFO:tasks.workunit.client.0.smithi005.stderr:HTTP request sent, awaiting response... 200 OK 2024-05-09T00:48:51.553 INFO:tasks.workunit.client.0.smithi005.stderr:Length: 8071 (7.9K) [application/octet-stream] 2024-05-09T00:48:51.555 INFO:tasks.workunit.client.0.smithi005.stderr:Saving to: ‘iogen_3.1p0.tar’ 2024-05-09T00:48:51.558 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-05-09T00:48:51.558 INFO:tasks.workunit.client.0.smithi005.stderr: 0K ....... 100% 1017M=0s 2024-05-09T00:48:51.558 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-05-09T00:48:51.558 INFO:tasks.workunit.client.0.smithi005.stderr:2024-05-09 00:48:51 (1017 MB/s) - ‘iogen_3.1p0.tar’ saved [8071/8071] 2024-05-09T00:48:51.558 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-05-09T00:48:51.558 INFO:tasks.workunit.client.0.smithi005.stderr:+ tar -xvzf iogen_3.1p0.tar 2024-05-09T00:48:51.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:48:51 smithi131 ceph-mon[34547]: pgmap v446: 97 pgs: 97 active+clean; 671 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-09T00:48:51.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:48:51 smithi131 ceph-mon[34547]: scrub summary: idle 2024-05-09T00:48:51.620 INFO:tasks.workunit.client.0.smithi005.stdout:iogen_3.1p0 2024-05-09T00:48:51.621 INFO:tasks.workunit.client.0.smithi005.stdout:iogen_3.1p0/src 2024-05-09T00:48:51.622 INFO:tasks.workunit.client.0.smithi005.stdout:iogen_3.1p0/src/iogen.8 2024-05-09T00:48:51.624 INFO:tasks.workunit.client.0.smithi005.stdout:iogen_3.1p0/src/iogen.c 2024-05-09T00:48:51.626 INFO:tasks.workunit.client.0.smithi005.stdout:iogen_3.1p0/README 2024-05-09T00:48:51.628 INFO:tasks.workunit.client.0.smithi005.stdout:iogen_3.1p0/Makefile.bsd 2024-05-09T00:48:51.629 INFO:tasks.workunit.client.0.smithi005.stdout:iogen_3.1p0/Makefile 2024-05-09T00:48:51.631 INFO:tasks.workunit.client.0.smithi005.stderr:+ cd iogen_3.1p0 2024-05-09T00:48:51.631 INFO:tasks.workunit.client.0.smithi005.stdout:making iogen 2024-05-09T00:48:51.631 INFO:tasks.workunit.client.0.smithi005.stderr:+ echo 'making iogen' 2024-05-09T00:48:51.631 INFO:tasks.workunit.client.0.smithi005.stderr:+ make 2024-05-09T00:48:51.640 INFO:tasks.workunit.client.0.smithi005.stdout:gcc -D_FILE_OFFSET_BITS=64 -DVERSION=\"3.1p0\" -Dstrlcpy=strncpy -Darc4random=rand -O2 src/iogen.c -o iogen 2024-05-09T00:48:51.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:48:51 smithi157 ceph-mon[34294]: pgmap v446: 97 pgs: 97 active+clean; 671 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-09T00:48:51.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:48:51 smithi157 ceph-mon[34294]: scrub summary: idle 2024-05-09T00:48:52.085 INFO:tasks.workunit.client.0.smithi005.stdout:nroff -mandoc -Tascii src/iogen.8 > iogen.cat8 2024-05-09T00:48:52.462 INFO:tasks.workunit.client.0.smithi005.stdout:running iogen 2024-05-09T00:48:52.462 INFO:tasks.workunit.client.0.smithi005.stderr:+ echo 'running iogen' 2024-05-09T00:48:52.462 INFO:tasks.workunit.client.0.smithi005.stderr:+ ./iogen -n 5 -s 2g 2024-05-09T00:48:52.463 INFO:tasks.workunit.client.0.smithi005.stderr:child spawned 92416 2024-05-09T00:48:52.463 INFO:tasks.workunit.client.0.smithi005.stderr:child spawned 92417 2024-05-09T00:48:52.463 INFO:tasks.workunit.client.0.smithi005.stderr:child spawned 92418 2024-05-09T00:48:52.463 INFO:tasks.workunit.client.0.smithi005.stderr:child spawned 92419 2024-05-09T00:48:52.463 INFO:tasks.workunit.client.0.smithi005.stderr:child spawned 92420 2024-05-09T00:48:52.464 INFO:tasks.workunit.client.0.smithi005.stdout:sleep for 10 min 2024-05-09T00:48:52.464 INFO:tasks.workunit.client.0.smithi005.stderr:+ echo 'sleep for 10 min' 2024-05-09T00:48:52.464 INFO:tasks.workunit.client.0.smithi005.stderr:+ sleep 600 2024-05-09T00:48:52.475 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T00:48:52.475 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T00:48:53.298 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:48:53.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:48:53 smithi131 ceph-mon[34547]: pgmap v447: 97 pgs: 97 active+clean; 671 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-05-09T00:48:53.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:48:53 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:48:53.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:48:53 smithi157 ceph-mon[34294]: pgmap v447: 97 pgs: 97 active+clean; 671 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-05-09T00:48:53.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:48:53 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:48:53.732 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:48:53 smithi005 ceph-mon[30365]: pgmap v447: 97 pgs: 97 active+clean; 671 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-05-09T00:48:53.732 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:48:53 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:48:55.584 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:48:55 smithi005 ceph-mon[30365]: pgmap v448: 97 pgs: 97 active+clean; 145 MiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 12 MiB/s wr, 29 op/s 2024-05-09T00:48:55.584 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:48:55 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:48:55.584 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:48:55 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:48:55.584 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:48:55 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:48:55.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:48:55 smithi131 ceph-mon[34547]: pgmap v448: 97 pgs: 97 active+clean; 145 MiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 12 MiB/s wr, 29 op/s 2024-05-09T00:48:55.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:48:55 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:48:55.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:48:55 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:48:55.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:48:55 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:48:55.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:48:55 smithi157 ceph-mon[34294]: pgmap v448: 97 pgs: 97 active+clean; 145 MiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 12 MiB/s wr, 29 op/s 2024-05-09T00:48:55.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:48:55 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:48:55.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:48:55 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:48:55.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:48:55 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:48:57.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:48:57 smithi131 ceph-mon[34547]: pgmap v449: 97 pgs: 97 active+clean; 934 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 78 MiB/s wr, 42 op/s 2024-05-09T00:48:57.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:48:57 smithi157 ceph-mon[34294]: pgmap v449: 97 pgs: 97 active+clean; 934 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 78 MiB/s wr, 42 op/s 2024-05-09T00:48:57.790 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:48:57 smithi005 ceph-mon[30365]: pgmap v449: 97 pgs: 97 active+clean; 934 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 78 MiB/s wr, 42 op/s 2024-05-09T00:48:59.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:48:59 smithi131 ceph-mon[34547]: pgmap v450: 97 pgs: 97 active+clean; 1.5 GiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 126 MiB/s wr, 55 op/s 2024-05-09T00:48:59.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:48:59 smithi157 ceph-mon[34294]: pgmap v450: 97 pgs: 97 active+clean; 1.5 GiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 126 MiB/s wr, 55 op/s 2024-05-09T00:48:59.790 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:48:59 smithi005 ceph-mon[30365]: pgmap v450: 97 pgs: 97 active+clean; 1.5 GiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 126 MiB/s wr, 55 op/s 2024-05-09T00:49:01.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:49:01 smithi131 ceph-mon[34547]: pgmap v451: 97 pgs: 97 active+clean; 4.1 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 812 KiB/s rd, 352 MiB/s wr, 343 op/s 2024-05-09T00:49:01.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:49:01 smithi157 ceph-mon[34294]: pgmap v451: 97 pgs: 97 active+clean; 4.1 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 812 KiB/s rd, 352 MiB/s wr, 343 op/s 2024-05-09T00:49:01.723 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:49:01 smithi005 ceph-mon[30365]: pgmap v451: 97 pgs: 97 active+clean; 4.1 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 812 KiB/s rd, 352 MiB/s wr, 343 op/s 2024-05-09T00:49:01.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:01.720+0000 7f51c3fff700 1 -- 172.21.15.5:0/3299854343 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51bc003c00 con 0x7f51cc10a7c0 2024-05-09T00:49:01.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:01.722+0000 7f51d243c700 1 -- 172.21.15.5:0/3299854343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51cc10a7c0 msgr2=0x7f51cc10abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:01.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:01.722+0000 7f51d243c700 1 --2- 172.21.15.5:0/3299854343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51cc10a7c0 0x7f51cc10abd0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f51bc009990 tx=0x7f51bc009ca0 comp rx=0 tx=0).stop 2024-05-09T00:49:01.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:01.723+0000 7f51d243c700 1 -- 172.21.15.5:0/3299854343 shutdown_connections 2024-05-09T00:49:01.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:01.723+0000 7f51d243c700 1 --2- 172.21.15.5:0/3299854343 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51cc10b110 0x7f51cc117620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:01.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:01.723+0000 7f51d243c700 1 --2- 172.21.15.5:0/3299854343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51cc10f6c0 0x7f51cc10fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:01.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:01.723+0000 7f51d243c700 1 --2- 172.21.15.5:0/3299854343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51cc10a7c0 0x7f51cc10abd0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:01.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:01.723+0000 7f51d243c700 1 -- 172.21.15.5:0/3299854343 >> 172.21.15.5:0/3299854343 conn(0x7f51cc076b00 msgr2=0x7f51cc076f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:49:01.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:01.723+0000 7f51d243c700 1 -- 172.21.15.5:0/3299854343 shutdown_connections 2024-05-09T00:49:01.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:01.723+0000 7f51d243c700 1 -- 172.21.15.5:0/3299854343 wait complete. 2024-05-09T00:49:01.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:01.725+0000 7f51d243c700 1 Processor -- start 2024-05-09T00:49:01.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:01.725+0000 7f51d243c700 1 -- start start 2024-05-09T00:49:01.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:01.726+0000 7f51d243c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51cc10a7c0 0x7f51cc1a13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:01.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:01.726+0000 7f51d243c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51cc10b110 0x7f51cc1a1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:01.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:01.726+0000 7f51d243c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51cc10f6c0 0x7f51cc1a5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:01.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:01.726+0000 7f51d243c700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f51cc11a9e0 con 0x7f51cc10b110 2024-05-09T00:49:01.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:01.726+0000 7f51d243c700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f51cc11a860 con 0x7f51cc10a7c0 2024-05-09T00:49:01.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:01.726+0000 7f51d243c700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f51cc11ab60 con 0x7f51cc10f6c0 2024-05-09T00:49:01.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:01.726+0000 7f51d0c39700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51cc10b110 0x7f51cc1a1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:01.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:01.727+0000 7f51d0c39700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51cc10b110 0x7f51cc1a1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54678/0 (socket says 172.21.15.5:54678) 2024-05-09T00:49:01.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:01.727+0000 7f51d0c39700 1 -- 172.21.15.5:0/3185022054 learned_addr learned my addr 172.21.15.5:0/3185022054 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:49:01.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:01.727+0000 7f51d0c39700 1 -- 172.21.15.5:0/3185022054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51cc10f6c0 msgr2=0x7f51cc1a5c20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T00:49:01.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:01.727+0000 7f51d0c39700 1 --2- 172.21.15.5:0/3185022054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51cc10f6c0 0x7f51cc1a5c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:01.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:01.727+0000 7f51d0c39700 1 -- 172.21.15.5:0/3185022054 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51cc10a7c0 msgr2=0x7f51cc1a13f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T00:49:01.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:01.727+0000 7f51d0c39700 1 --2- 172.21.15.5:0/3185022054 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51cc10a7c0 0x7f51cc1a13f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:01.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:01.727+0000 7f51d0c39700 1 -- 172.21.15.5:0/3185022054 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51cc1a62c0 con 0x7f51cc10b110 2024-05-09T00:49:01.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:01.728+0000 7f51d0c39700 1 --2- 172.21.15.5:0/3185022054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51cc10b110 0x7f51cc1a1990 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f51bc0057f0 tx=0x7f51bc005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:49:01.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:01.729+0000 7f51c27fc700 1 -- 172.21.15.5:0/3185022054 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51bc027070 con 0x7f51cc10b110 2024-05-09T00:49:01.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:01.729+0000 7f51c27fc700 1 -- 172.21.15.5:0/3185022054 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51bc004100 con 0x7f51cc10b110 2024-05-09T00:49:01.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:01.729+0000 7f51d243c700 1 -- 172.21.15.5:0/3185022054 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51cc1a6550 con 0x7f51cc10b110 2024-05-09T00:49:01.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:01.729+0000 7f51c27fc700 1 -- 172.21.15.5:0/3185022054 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51bc022e00 con 0x7f51cc10b110 2024-05-09T00:49:01.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:01.729+0000 7f51d243c700 1 -- 172.21.15.5:0/3185022054 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51cc1aded0 con 0x7f51cc10b110 2024-05-09T00:49:01.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:01.731+0000 7f51d243c700 1 -- 172.21.15.5:0/3185022054 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51cc10beb0 con 0x7f51cc10b110 2024-05-09T00:49:01.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:01.737+0000 7f51c27fc700 1 -- 172.21.15.5:0/3185022054 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f51bc022590 con 0x7f51cc10b110 2024-05-09T00:49:01.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:01.739+0000 7f51c27fc700 1 --2- 172.21.15.5:0/3185022054 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f51b80b2980 0x7f51b80b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:01.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:01.740+0000 7f51d143a700 1 --2- 172.21.15.5:0/3185022054 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f51b80b2980 0x7f51b80b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:01.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:01.740+0000 7f51c27fc700 1 -- 172.21.15.5:0/3185022054 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f51bc0e3b90 con 0x7f51cc10b110 2024-05-09T00:49:01.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:01.741+0000 7f51d143a700 1 --2- 172.21.15.5:0/3185022054 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f51b80b2980 0x7f51b80b4e90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f51c8009710 tx=0x7f51c8006c60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:49:01.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:01.744+0000 7f51c27fc700 1 -- 172.21.15.5:0/3185022054 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f51bc013070 con 0x7f51cc10b110 2024-05-09T00:49:02.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:02.361+0000 7f51d243c700 1 -- 172.21.15.5:0/3185022054 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f51cc1a2750 con 0x7f51cc10b110 2024-05-09T00:49:02.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:02.363+0000 7f51c27fc700 1 -- 172.21.15.5:0/3185022054 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11926 (secure 0 0 0) 0x7f51bc0ab8b0 con 0x7f51cc10b110 2024-05-09T00:49:02.369 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:49:02.370 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2685771887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2685771887},{"type":"v1","addr":"172.21.15.5:6835","nonce":2685771887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/254017535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":254017535},{"type":"v1","addr":"172.21.15.5:6837","nonce":254017535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T00:43:45.856066+0000","modified":"2024-05-09T00:47:06.980541+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24508,"mds_1":24499,"mds_2":24494,"mds_3":24493,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14598":{"gid":14598,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2109255228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2109255228},{"type":"v1","addr":"172.21.15.5:6839","nonce":2109255228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/3809966329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3809966329},{"type":"v1","addr":"172.21.15.5:6841","nonce":3809966329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6833/3272897742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3272897742},{"type":"v1","addr":"172.21.15.131:6833","nonce":3272897742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1283093898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1283093898},{"type":"v1","addr":"172.21.15.131:6835","nonce":1283093898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/2809525455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2809525455},{"type":"v1","addr":"172.21.15.157:6833","nonce":2809525455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.131:6837/1633006186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":1633006186},{"type":"v1","addr":"172.21.15.131:6837","nonce":1633006186}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.157:6835/832898273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":832898273},{"type":"v1","addr":"172.21.15.157:6835","nonce":832898273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.131:6839/2748669631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2748669631},{"type":"v1","addr":"172.21.15.131:6839","nonce":2748669631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.157:6837/744863009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":744863009},{"type":"v1","addr":"172.21.15.157:6837","nonce":744863009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.157:6839/2779860399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":2779860399},{"type":"v1","addr":"172.21.15.157:6839","nonce":2779860399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24494,24508,24499,24467,24493]},"id":1}]} 2024-05-09T00:49:02.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:02.374+0000 7f51d243c700 1 -- 172.21.15.5:0/3185022054 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f51b80b2980 msgr2=0x7f51b80b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:02.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:02.374+0000 7f51d243c700 1 --2- 172.21.15.5:0/3185022054 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f51b80b2980 0x7f51b80b4e90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f51c8009710 tx=0x7f51c8006c60 comp rx=0 tx=0).stop 2024-05-09T00:49:02.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:02.374+0000 7f51d243c700 1 -- 172.21.15.5:0/3185022054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51cc10b110 msgr2=0x7f51cc1a1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:02.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:02.374+0000 7f51d243c700 1 --2- 172.21.15.5:0/3185022054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51cc10b110 0x7f51cc1a1990 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f51bc0057f0 tx=0x7f51bc005e50 comp rx=0 tx=0).stop 2024-05-09T00:49:02.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:02.374+0000 7f51d243c700 1 -- 172.21.15.5:0/3185022054 shutdown_connections 2024-05-09T00:49:02.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:02.374+0000 7f51d243c700 1 --2- 172.21.15.5:0/3185022054 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f51b80b2980 0x7f51b80b4e90 secure :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f51c8009710 tx=0x7f51c8006c60 comp rx=0 tx=0).stop 2024-05-09T00:49:02.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:02.374+0000 7f51d243c700 1 --2- 172.21.15.5:0/3185022054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51cc10f6c0 0x7f51cc1a5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:02.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:02.374+0000 7f51d243c700 1 --2- 172.21.15.5:0/3185022054 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51cc10a7c0 0x7f51cc1a13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:02.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:02.374+0000 7f51d243c700 1 --2- 172.21.15.5:0/3185022054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51cc10b110 0x7f51cc1a1990 secure :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f51bc0057f0 tx=0x7f51bc005e50 comp rx=0 tx=0).stop 2024-05-09T00:49:02.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:02.374+0000 7f51d243c700 1 -- 172.21.15.5:0/3185022054 >> 172.21.15.5:0/3185022054 conn(0x7f51cc076b00 msgr2=0x7f51cc1089c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:49:02.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:02.375+0000 7f51d243c700 1 -- 172.21.15.5:0/3185022054 shutdown_connections 2024-05-09T00:49:02.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:02.376+0000 7f51d243c700 1 -- 172.21.15.5:0/3185022054 wait complete. 2024-05-09T00:49:02.383 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 29 2024-05-09T00:49:03.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:49:03 smithi005 ceph-mon[30365]: pgmap v452: 97 pgs: 97 active+clean; 5.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 809 KiB/s rd, 443 MiB/s wr, 389 op/s 2024-05-09T00:49:03.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:49:03 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/3185022054' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:49:03.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:49:03 smithi131 ceph-mon[34547]: pgmap v452: 97 pgs: 97 active+clean; 5.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 809 KiB/s rd, 443 MiB/s wr, 389 op/s 2024-05-09T00:49:03.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:49:03 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/3185022054' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:49:03.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:49:03 smithi157 ceph-mon[34294]: pgmap v452: 97 pgs: 97 active+clean; 5.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 809 KiB/s rd, 443 MiB/s wr, 389 op/s 2024-05-09T00:49:03.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:49:03 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/3185022054' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:49:03.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.864+0000 7fea1e7fc640 1 -- 172.21.15.5:0/4023011391 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea18040c00 con 0x7fea201101a0 2024-05-09T00:49:03.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.864+0000 7fea25e25640 1 -- 172.21.15.5:0/4023011391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea201101a0 msgr2=0x7fea201105b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:03.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.864+0000 7fea25e25640 1 --2- 172.21.15.5:0/4023011391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea201101a0 0x7fea201105b0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fea1801acb0 tx=0x7fea1803b900 comp rx=0 tx=0).stop 2024-05-09T00:49:03.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.864+0000 7fea25e25640 1 -- 172.21.15.5:0/4023011391 shutdown_connections 2024-05-09T00:49:03.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.864+0000 7fea25e25640 1 --2- 172.21.15.5:0/4023011391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea201101a0 0x7fea201105b0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:03.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.864+0000 7fea25e25640 1 --2- 172.21.15.5:0/4023011391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea2010b480 0x7fea2010b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:03.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.864+0000 7fea25e25640 1 --2- 172.21.15.5:0/4023011391 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea2010a550 0x7fea2010af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:03.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.864+0000 7fea25e25640 1 -- 172.21.15.5:0/4023011391 >> 172.21.15.5:0/4023011391 conn(0x7fea20070670 msgr2=0x7fea20072b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:49:03.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.865+0000 7fea25e25640 1 -- 172.21.15.5:0/4023011391 shutdown_connections 2024-05-09T00:49:03.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.865+0000 7fea25e25640 1 -- 172.21.15.5:0/4023011391 wait complete. 2024-05-09T00:49:03.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.866+0000 7fea25e25640 1 Processor -- start 2024-05-09T00:49:03.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.866+0000 7fea25e25640 1 -- start start 2024-05-09T00:49:03.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.866+0000 7fea25e25640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea2010a550 0x7fea201a0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:03.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.866+0000 7fea25e25640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea2010b480 0x7fea201a0f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:03.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.866+0000 7fea25e25640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea201101a0 0x7fea201a7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:03.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.866+0000 7fea25e25640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fea20113e90 con 0x7fea2010a550 2024-05-09T00:49:03.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.866+0000 7fea25e25640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fea20113d10 con 0x7fea2010b480 2024-05-09T00:49:03.868 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.866+0000 7fea25e25640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fea20114010 con 0x7fea201101a0 2024-05-09T00:49:03.868 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.867+0000 7fea1f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea2010a550 0x7fea201a0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:03.868 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.867+0000 7fea1f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea2010a550 0x7fea201a0a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54564/0 (socket says 172.21.15.5:54564) 2024-05-09T00:49:03.868 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.867+0000 7fea1f7fe640 1 -- 172.21.15.5:0/1566995599 learned_addr learned my addr 172.21.15.5:0/1566995599 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:49:03.868 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.867+0000 7fea1ffff640 1 --2- 172.21.15.5:0/1566995599 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea201101a0 0x7fea201a7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:03.868 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.867+0000 7fea1effd640 1 --2- 172.21.15.5:0/1566995599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea2010b480 0x7fea201a0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:03.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.868+0000 7fea1f7fe640 1 -- 172.21.15.5:0/1566995599 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea201101a0 msgr2=0x7fea201a7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:03.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.868+0000 7fea1f7fe640 1 --2- 172.21.15.5:0/1566995599 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea201101a0 0x7fea201a7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:03.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.868+0000 7fea1f7fe640 1 -- 172.21.15.5:0/1566995599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea2010b480 msgr2=0x7fea201a0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:03.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.868+0000 7fea1f7fe640 1 --2- 172.21.15.5:0/1566995599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea2010b480 0x7fea201a0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:03.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.868+0000 7fea1f7fe640 1 -- 172.21.15.5:0/1566995599 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea201a8490 con 0x7fea2010a550 2024-05-09T00:49:03.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.869+0000 7fea1f7fe640 1 --2- 172.21.15.5:0/1566995599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea2010a550 0x7fea201a0a00 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fea1000b470 tx=0x7fea1000b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:49:03.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.869+0000 7fea1cff9640 1 -- 172.21.15.5:0/1566995599 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea10015020 con 0x7fea2010a550 2024-05-09T00:49:03.870 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.869+0000 7fea25e25640 1 -- 172.21.15.5:0/1566995599 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea201a8780 con 0x7fea2010a550 2024-05-09T00:49:03.870 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.869+0000 7fea25e25640 1 -- 172.21.15.5:0/1566995599 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea201a8cd0 con 0x7fea2010a550 2024-05-09T00:49:03.870 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.870+0000 7fea1cff9640 1 -- 172.21.15.5:0/1566995599 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea10011d40 con 0x7fea2010a550 2024-05-09T00:49:03.870 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.870+0000 7fea1cff9640 1 -- 172.21.15.5:0/1566995599 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1000f340 con 0x7fea2010a550 2024-05-09T00:49:03.888 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.877+0000 7fea1cff9640 1 -- 172.21.15.5:0/1566995599 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fea10018020 con 0x7fea2010a550 2024-05-09T00:49:03.890 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.889+0000 7fea1cff9640 1 --2- 172.21.15.5:0/1566995599 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fea080b2860 0x7fea080b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:03.890 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.889+0000 7fea1cff9640 1 -- 172.21.15.5:0/1566995599 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fea100e2df0 con 0x7fea2010a550 2024-05-09T00:49:03.891 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.890+0000 7fea2cdaa740 1 Processor -- start 2024-05-09T00:49:03.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.892+0000 7fea2cdaa740 1 -- start start 2024-05-09T00:49:03.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.893+0000 7fea2cdaa740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623959f01b0 0x562395d94500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:03.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.893+0000 7fea2cdaa740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623959c72f0 0x562395d94a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:03.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.893+0000 7fea2cdaa740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623959cf4d0 0x562395d9b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:03.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.893+0000 7fea2cdaa740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562395cf87c0 con 0x5623959cf4d0 2024-05-09T00:49:03.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.893+0000 7fea2cdaa740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x562395c940e0 con 0x5623959c72f0 2024-05-09T00:49:03.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.893+0000 7fea2cdaa740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x562395cf4780 con 0x5623959f01b0 2024-05-09T00:49:03.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.893+0000 7fea1f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623959f01b0 0x562395d94500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:03.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.894+0000 7fea1ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623959cf4d0 0x562395d9b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:03.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.894+0000 7fea1ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623959cf4d0 0x562395d9b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54578/0 (socket says 172.21.15.5:54578) 2024-05-09T00:49:03.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.894+0000 7fea1ffff640 1 -- 172.21.15.5:0/2149639236 learned_addr learned my addr 172.21.15.5:0/2149639236 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:49:03.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.895+0000 7fea1effd640 1 --2- 172.21.15.5:0/1566995599 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fea080b2860 0x7fea080b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:03.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.899+0000 7fea1ffff640 1 -- 172.21.15.5:0/2149639236 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623959f01b0 msgr2=0x562395d94500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:49:03.900 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.899+0000 7fea1ffff640 1 --2- 172.21.15.5:0/2149639236 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623959f01b0 0x562395d94500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:03.900 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.899+0000 7fea1ffff640 1 -- 172.21.15.5:0/2149639236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623959c72f0 msgr2=0x562395d94a80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:49:03.900 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.899+0000 7fea1ffff640 1 --2- 172.21.15.5:0/2149639236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623959c72f0 0x562395d94a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:03.900 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.899+0000 7fea1ffff640 1 -- 172.21.15.5:0/2149639236 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562395cbd7c0 con 0x5623959cf4d0 2024-05-09T00:49:03.900 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.899+0000 7fea1f7fe640 1 --2- 172.21.15.5:0/2149639236 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623959f01b0 0x562395d94500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:49:03.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.905+0000 7fea1ffff640 1 --2- 172.21.15.5:0/2149639236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623959cf4d0 0x562395d9b980 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fea18005cc0 tx=0x7fea18041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c60896c3a3bb4a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:49:03.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.910+0000 7fea05ffb640 1 -- 172.21.15.5:0/2149639236 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea18045040 con 0x5623959cf4d0 2024-05-09T00:49:03.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.910+0000 7fea05ffb640 1 -- 172.21.15.5:0/2149639236 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea180403d0 con 0x5623959cf4d0 2024-05-09T00:49:03.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.912+0000 7fea2cdaa740 1 -- 172.21.15.5:0/2149639236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623959cf4d0 msgr2=0x562395d9b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:03.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.912+0000 7fea2cdaa740 1 --2- 172.21.15.5:0/2149639236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623959cf4d0 0x562395d9b980 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fea18005cc0 tx=0x7fea18041000 comp rx=0 tx=0).stop 2024-05-09T00:49:03.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.920+0000 7fea2cdaa740 1 -- 172.21.15.5:0/2149639236 shutdown_connections 2024-05-09T00:49:03.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.920+0000 7fea2cdaa740 1 --2- 172.21.15.5:0/2149639236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623959cf4d0 0x562395d9b980 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:03.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.920+0000 7fea2cdaa740 1 --2- 172.21.15.5:0/2149639236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623959c72f0 0x562395d94a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:03.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.920+0000 7fea2cdaa740 1 --2- 172.21.15.5:0/2149639236 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623959f01b0 0x562395d94500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:03.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.920+0000 7fea2cdaa740 1 -- 172.21.15.5:0/2149639236 >> 172.21.15.5:0/2149639236 conn(0x5623959e4300 msgr2=0x562395d8fe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:49:03.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.925+0000 7fea1effd640 1 --2- 172.21.15.5:0/1566995599 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fea080b2860 0x7fea080b4d70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x562395d958e0 tx=0x7fea14005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:49:03.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.925+0000 7fea2cdaa740 1 -- 172.21.15.5:0/2149639236 shutdown_connections 2024-05-09T00:49:03.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.925+0000 7fea2cdaa740 1 -- 172.21.15.5:0/2149639236 wait complete. 2024-05-09T00:49:03.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.926+0000 7fea2cdaa740 1 -- start start 2024-05-09T00:49:03.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.926+0000 7fea2cdaa740 1 Processor -- start 2024-05-09T00:49:03.928 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.927+0000 7fea2cdaa740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623959c72f0 0x562395d9e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:03.928 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.927+0000 7fea2cdaa740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623959cf4d0 0x562395d9e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:03.928 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.927+0000 7fea2cdaa740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623959f01b0 0x562395da5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:03.928 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.927+0000 7fea2cdaa740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562395d9c3c0 con 0x5623959c72f0 2024-05-09T00:49:03.928 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.927+0000 7fea2cdaa740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x562395c7f9c0 con 0x5623959f01b0 2024-05-09T00:49:03.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.927+0000 7fea2cdaa740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x562395d9c540 con 0x5623959cf4d0 2024-05-09T00:49:03.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.927+0000 7fea2cdaa740 10 client.0 _ll_register_callbacks cb 0x562395cac020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:49:03.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.928+0000 7fea1effd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623959cf4d0 0x562395d9e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:03.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.928+0000 7fea1effd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623959cf4d0 0x562395d9e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:42642/0 (socket says 172.21.15.5:42642) 2024-05-09T00:49:03.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.928+0000 7fea1effd640 1 -- 172.21.15.5:0/2596751251 learned_addr learned my addr 172.21.15.5:0/2596751251 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:49:03.930 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.929+0000 7fea1effd640 1 -- 172.21.15.5:0/2596751251 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623959f01b0 msgr2=0x562395da5680 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:49:03.930 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.929+0000 7fea1effd640 1 --2- 172.21.15.5:0/2596751251 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623959f01b0 0x562395da5680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:03.930 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.929+0000 7fea1effd640 1 -- 172.21.15.5:0/2596751251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623959c72f0 msgr2=0x562395d9e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:49:03.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.929+0000 7fea1effd640 1 --2- 172.21.15.5:0/2596751251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623959c72f0 0x562395d9e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:03.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.929+0000 7fea1effd640 1 -- 172.21.15.5:0/2596751251 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562395ca61f0 con 0x5623959cf4d0 2024-05-09T00:49:03.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.932+0000 7fea1effd640 1 --2- 172.21.15.5:0/2596751251 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623959cf4d0 0x562395d9e720 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fea140109b0 tx=0x7fea140046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ce9c0d6f941290d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:49:03.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.943+0000 7fe9feffd640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-09T00:49:03.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.943+0000 7fe9feffd640 1 -- 172.21.15.5:0/2596751251 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea14021070 con 0x5623959cf4d0 2024-05-09T00:49:03.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.943+0000 7fe9feffd640 1 -- 172.21.15.5:0/2596751251 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fea1400fd50 con 0x5623959cf4d0 2024-05-09T00:49:03.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.943+0000 7fea2cdaa740 1 -- 172.21.15.5:0/2596751251 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea180403d0 con 0x5623959cf4d0 2024-05-09T00:49:03.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.956+0000 7fe9feffd640 1 -- 172.21.15.5:0/2596751251 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fea1402d050 con 0x5623959cf4d0 2024-05-09T00:49:03.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.956+0000 7fea2cdaa740 10 client.25432 fetch_fsmap learned FSMap version 29 2024-05-09T00:49:03.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.957+0000 7fea2cdaa740 1 -- 172.21.15.5:0/2596751251 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562395a5b790 con 0x5623959cf4d0 2024-05-09T00:49:03.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.957+0000 7fe9feffd640 1 -- 172.21.15.5:0/2596751251 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea180403d0 con 0x5623959cf4d0 2024-05-09T00:49:03.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.958+0000 7fe9feffd640 1 -- 172.21.15.5:0/2596751251 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fea140253b0 con 0x5623959cf4d0 2024-05-09T00:49:03.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.958+0000 7fea2cdaa740 10 client.25432 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:49:03.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.958+0000 7fea2cdaa740 10 client.25432 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:49:03.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.958+0000 7fea2cdaa740 20 client.25432 populate_metadata read hostname 'smithi005' 2024-05-09T00:49:03.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.958+0000 7fea2cdaa740 1 --2- 172.21.15.5:0/2596751251 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5623959aed80 0x562395dac180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:03.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.958+0000 7fea2cdaa740 4 client.25432 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:49:03.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.958+0000 7fea2cdaa740 1 -- 172.21.15.5:0/2596751251 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562395d063d0 con 0x5623959aed80 2024-05-09T00:49:03.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.960+0000 7fea1f7fe640 1 --2- 172.21.15.5:0/2596751251 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5623959aed80 0x562395dac180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:03.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.961+0000 7fea1f7fe640 1 --2- 172.21.15.5:0/2596751251 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5623959aed80 0x562395dac180 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b08d083f3418831 server_cookie=f4665f63086b0872 in_seq=0 out_seq=0 2024-05-09T00:49:03.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.962+0000 7fe9feffd640 10 client.25432 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:49:03.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.964+0000 7fe9feffd640 1 -- 172.21.15.5:0/2596751251 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562395d063d0 con 0x5623959aed80 2024-05-09T00:49:03.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.964+0000 7fe9feffd640 10 client.25432 handle_command_reply: tid=0 2024-05-09T00:49:03.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.964+0000 7fea2cdaa740 1 client.25432 shutdown 2024-05-09T00:49:03.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.969+0000 7fea2cdaa740 1 -- 172.21.15.5:0/2596751251 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623959cf4d0 msgr2=0x562395d9e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:03.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.969+0000 7fea2cdaa740 1 --2- 172.21.15.5:0/2596751251 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623959cf4d0 0x562395d9e720 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fea140109b0 tx=0x7fea140046c0 comp rx=0 tx=0).stop 2024-05-09T00:49:03.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.976+0000 7fea2cdaa740 1 -- 172.21.15.5:0/2596751251 shutdown_connections 2024-05-09T00:49:03.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.976+0000 7fea2cdaa740 1 --2- 172.21.15.5:0/2596751251 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5623959aed80 0x562395dac180 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:03.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.976+0000 7fea2cdaa740 1 --2- 172.21.15.5:0/2596751251 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623959f01b0 0x562395da5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:03.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.976+0000 7fea2cdaa740 1 --2- 172.21.15.5:0/2596751251 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623959cf4d0 0x562395d9e720 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:03.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.976+0000 7fea2cdaa740 1 --2- 172.21.15.5:0/2596751251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623959c72f0 0x562395d9e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:03.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.976+0000 7fea2cdaa740 1 -- 172.21.15.5:0/2596751251 >> 172.21.15.5:0/2596751251 conn(0x5623959e4300 msgr2=0x562395d92da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:49:03.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.977+0000 7fe9feffd640 0 client.25432 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:49:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.980+0000 7fea2cdaa740 1 -- 172.21.15.5:0/2596751251 shutdown_connections 2024-05-09T00:49:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.983+0000 7fea2cdaa740 1 -- 172.21.15.5:0/2596751251 wait complete. 2024-05-09T00:49:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:03.983+0000 7fea2cdaa740 20 client.25432 trim_cache size 0 max 16384 2024-05-09T00:49:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.036+0000 7fea2cdaa740 1 Processor -- start 2024-05-09T00:49:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.036+0000 7fea2cdaa740 1 -- start start 2024-05-09T00:49:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.037+0000 7fea2cdaa740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623959c72f0 0x562395d9d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.037+0000 7fea2cdaa740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623959cf4d0 0x562395d94480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.037+0000 7fea2cdaa740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623959f01b0 0x562395d949c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.038+0000 7fea2cdaa740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fea0c00e640 con 0x5623959c72f0 2024-05-09T00:49:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.038+0000 7fea2cdaa740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fea0c010180 con 0x5623959cf4d0 2024-05-09T00:49:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.038+0000 7fea2cdaa740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fea0c00ed10 con 0x5623959f01b0 2024-05-09T00:49:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.038+0000 7fea1f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623959c72f0 0x562395d9d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.038+0000 7fea1f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623959c72f0 0x562395d9d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54594/0 (socket says 172.21.15.5:54594) 2024-05-09T00:49:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.038+0000 7fea1f7fe640 1 -- 172.21.15.5:0/1299113222 learned_addr learned my addr 172.21.15.5:0/1299113222 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:49:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.038+0000 7fea1effd640 1 --2- 172.21.15.5:0/1299113222 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623959cf4d0 0x562395d94480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.038+0000 7fea1ffff640 1 --2- 172.21.15.5:0/1299113222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623959f01b0 0x562395d949c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.039+0000 7fea1f7fe640 1 -- 172.21.15.5:0/1299113222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623959f01b0 msgr2=0x562395d949c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:49:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.039+0000 7fea1f7fe640 1 --2- 172.21.15.5:0/1299113222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623959f01b0 0x562395d949c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.039+0000 7fea1f7fe640 1 -- 172.21.15.5:0/1299113222 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623959cf4d0 msgr2=0x562395d94480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:49:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.039+0000 7fea1f7fe640 1 --2- 172.21.15.5:0/1299113222 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623959cf4d0 0x562395d94480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.039+0000 7fea1f7fe640 1 -- 172.21.15.5:0/1299113222 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562395dac750 con 0x5623959c72f0 2024-05-09T00:49:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.040+0000 7fea1f7fe640 1 --2- 172.21.15.5:0/1299113222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623959c72f0 0x562395d9d7c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fea10006fd0 tx=0x7fea10010f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=110c1b0d1f57cbe5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:49:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.040+0000 7fea057fa640 1 -- 172.21.15.5:0/1299113222 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea100e0970 con 0x5623959c72f0 2024-05-09T00:49:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.041+0000 7fea057fa640 1 -- 172.21.15.5:0/1299113222 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea100a6610 con 0x5623959c72f0 2024-05-09T00:49:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.041+0000 7fea057fa640 1 -- 172.21.15.5:0/1299113222 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea100a9840 con 0x5623959c72f0 2024-05-09T00:49:04.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.043+0000 7fea2cdaa740 1 -- 172.21.15.5:0/1299113222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623959c72f0 msgr2=0x562395d9d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:04.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.043+0000 7fea2cdaa740 1 --2- 172.21.15.5:0/1299113222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623959c72f0 0x562395d9d7c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fea10006fd0 tx=0x7fea10010f60 comp rx=0 tx=0).stop 2024-05-09T00:49:04.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.044+0000 7fea2cdaa740 1 -- 172.21.15.5:0/1299113222 shutdown_connections 2024-05-09T00:49:04.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.044+0000 7fea2cdaa740 1 --2- 172.21.15.5:0/1299113222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623959f01b0 0x562395d949c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:04.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.044+0000 7fea2cdaa740 1 --2- 172.21.15.5:0/1299113222 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623959cf4d0 0x562395d94480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:04.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.045+0000 7fea2cdaa740 1 --2- 172.21.15.5:0/1299113222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623959c72f0 0x562395d9d7c0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:04.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.045+0000 7fea2cdaa740 1 -- 172.21.15.5:0/1299113222 >> 172.21.15.5:0/1299113222 conn(0x5623959e4300 msgr2=0x562395d9e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:49:04.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.048+0000 7fea2cdaa740 1 -- 172.21.15.5:0/1299113222 shutdown_connections 2024-05-09T00:49:04.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.048+0000 7fea2cdaa740 1 -- 172.21.15.5:0/1299113222 wait complete. 2024-05-09T00:49:04.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.049+0000 7fea2cdaa740 1 -- start start 2024-05-09T00:49:04.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.050+0000 7fea2cdaa740 1 Processor -- start 2024-05-09T00:49:04.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.051+0000 7fea2cdaa740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623959c72f0 0x562395da2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.055+0000 7fea2cdaa740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623959cf4d0 0x562395da3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:04.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.055+0000 7fea2cdaa740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623959f01b0 0x562395e12520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:04.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.055+0000 7fea2cdaa740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fea0c00fab0 con 0x5623959f01b0 2024-05-09T00:49:04.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.055+0000 7fea2cdaa740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fea0c00f3e0 con 0x5623959cf4d0 2024-05-09T00:49:04.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.055+0000 7fea2cdaa740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fea0c00c620 con 0x5623959c72f0 2024-05-09T00:49:04.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.055+0000 7fea2cdaa740 10 client.0 _ll_register_callbacks cb 0x7fea14014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:49:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.056+0000 7fea1ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623959f01b0 0x562395e12520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.057+0000 7fea1ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623959f01b0 0x562395e12520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54602/0 (socket says 172.21.15.5:54602) 2024-05-09T00:49:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.057+0000 7fea1ffff640 1 -- 172.21.15.5:0/2980493581 learned_addr learned my addr 172.21.15.5:0/2980493581 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:49:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.057+0000 7fea1f7fe640 1 --2- 172.21.15.5:0/2980493581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623959c72f0 0x562395da2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.057+0000 7fea1ffff640 1 -- 172.21.15.5:0/2980493581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623959c72f0 msgr2=0x562395da2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:49:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.057+0000 7fea1ffff640 1 --2- 172.21.15.5:0/2980493581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623959c72f0 0x562395da2b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.057+0000 7fea1ffff640 1 -- 172.21.15.5:0/2980493581 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623959cf4d0 msgr2=0x562395da3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:49:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.057+0000 7fea1ffff640 1 --2- 172.21.15.5:0/2980493581 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623959cf4d0 0x562395da3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.057+0000 7fea1ffff640 1 -- 172.21.15.5:0/2980493581 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562395ad25f0 con 0x5623959f01b0 2024-05-09T00:49:04.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.059+0000 7fea1ffff640 1 --2- 172.21.15.5:0/2980493581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623959f01b0 0x562395e12520 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fea180407a0 tx=0x7fea18040600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96a82d119baf61a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:49:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.059+0000 7fe9feffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:49:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.060+0000 7fe9feffd640 1 -- 172.21.15.5:0/2980493581 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea18063060 con 0x5623959f01b0 2024-05-09T00:49:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.061+0000 7fe9feffd640 1 -- 172.21.15.5:0/2980493581 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fea18019040 con 0x5623959f01b0 2024-05-09T00:49:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.061+0000 7fea2cdaa740 1 -- 172.21.15.5:0/2980493581 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea100a6610 con 0x5623959f01b0 2024-05-09T00:49:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.062+0000 7fe9feffd640 1 -- 172.21.15.5:0/2980493581 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fea18045040 con 0x5623959f01b0 2024-05-09T00:49:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.062+0000 7fea2cdaa740 10 client.15726 fetch_fsmap learned FSMap version 29 2024-05-09T00:49:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.062+0000 7fea2cdaa740 1 -- 172.21.15.5:0/2980493581 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562395cbced0 con 0x5623959f01b0 2024-05-09T00:49:04.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.063+0000 7fe9feffd640 1 -- 172.21.15.5:0/2980493581 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea18040b10 con 0x5623959f01b0 2024-05-09T00:49:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.065+0000 7fe9feffd640 1 -- 172.21.15.5:0/2980493581 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fea18053070 con 0x5623959f01b0 2024-05-09T00:49:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.065+0000 7fea2cdaa740 10 client.15726 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:49:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.065+0000 7fea2cdaa740 10 client.15726 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:49:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.065+0000 7fea2cdaa740 20 client.15726 populate_metadata read hostname 'smithi005' 2024-05-09T00:49:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.066+0000 7fea2cdaa740 1 --2- 172.21.15.5:0/2980493581 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5623959841c0 0x562395e15c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.066+0000 7fea2cdaa740 4 client.15726 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:49:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.066+0000 7fea2cdaa740 1 -- 172.21.15.5:0/2980493581 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fea100a6610 con 0x5623959841c0 2024-05-09T00:49:04.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.072+0000 7fea1f7fe640 1 --2- 172.21.15.5:0/2980493581 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5623959841c0 0x562395e15c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:04.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.086+0000 7fe9feffd640 10 client.15726 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:49:04.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:04.086+0000 7fea1f7fe640 1 --2- 172.21.15.5:0/2980493581 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5623959841c0 0x562395e15c10 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f80b0f055c2a30 server_cookie=cf401182317dfaa7 in_seq=0 out_seq=0 2024-05-09T00:49:04.484 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-09T00:49:05.275 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:49:05.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:49:05 smithi131 ceph-mon[34547]: pgmap v453: 97 pgs: 97 active+clean; 6.8 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 585 MiB/s wr, 479 op/s 2024-05-09T00:49:05.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:49:05 smithi157 ceph-mon[34294]: pgmap v453: 97 pgs: 97 active+clean; 6.8 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 585 MiB/s wr, 479 op/s 2024-05-09T00:49:05.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:49:05 smithi005 ceph-mon[30365]: pgmap v453: 97 pgs: 97 active+clean; 6.8 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 585 MiB/s wr, 479 op/s 2024-05-09T00:49:07.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:49:07 smithi131 ceph-mon[34547]: pgmap v454: 97 pgs: 97 active+clean; 9.4 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 806 MiB/s wr, 629 op/s 2024-05-09T00:49:07.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:49:07 smithi157 ceph-mon[34294]: pgmap v454: 97 pgs: 97 active+clean; 9.4 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 806 MiB/s wr, 629 op/s 2024-05-09T00:49:07.790 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:49:07 smithi005 ceph-mon[30365]: pgmap v454: 97 pgs: 97 active+clean; 9.4 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 806 MiB/s wr, 629 op/s 2024-05-09T00:49:09.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:49:09 smithi157 ceph-mon[34294]: pgmap v455: 97 pgs: 97 active+clean; 9.6 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 791 MiB/s wr, 756 op/s 2024-05-09T00:49:09.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:49:09 smithi005 ceph-mon[30365]: pgmap v455: 97 pgs: 97 active+clean; 9.6 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 791 MiB/s wr, 756 op/s 2024-05-09T00:49:09.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:49:09 smithi131 ceph-mon[34547]: pgmap v455: 97 pgs: 97 active+clean; 9.6 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 791 MiB/s wr, 756 op/s 2024-05-09T00:49:10.376 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:10.374+0000 7fe9feffd640 1 -- 172.21.15.5:0/2980493581 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7fea100a6610 con 0x5623959841c0 2024-05-09T00:49:10.376 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:10.374+0000 7fe9feffd640 10 client.15726 handle_command_reply: tid=0 2024-05-09T00:49:10.376 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:10.374+0000 7fea2cdaa740 1 client.15726 shutdown 2024-05-09T00:49:10.377 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:10.375+0000 7fea2cdaa740 1 -- 172.21.15.5:0/2980493581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623959f01b0 msgr2=0x562395e12520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:10.377 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:10.375+0000 7fea2cdaa740 1 --2- 172.21.15.5:0/2980493581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623959f01b0 0x562395e12520 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fea180407a0 tx=0x7fea18040600 comp rx=0 tx=0).stop 2024-05-09T00:49:10.377 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:10.375+0000 7fea2cdaa740 1 -- 172.21.15.5:0/2980493581 shutdown_connections 2024-05-09T00:49:10.377 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:10.375+0000 7fea2cdaa740 1 --2- 172.21.15.5:0/2980493581 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5623959841c0 0x562395e15c10 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:10.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:10.375+0000 7fea2cdaa740 1 --2- 172.21.15.5:0/2980493581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623959f01b0 0x562395e12520 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:10.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:10.375+0000 7fea2cdaa740 1 --2- 172.21.15.5:0/2980493581 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623959cf4d0 0x562395da3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:10.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:10.375+0000 7fea2cdaa740 1 --2- 172.21.15.5:0/2980493581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623959c72f0 0x562395da2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:10.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:10.375+0000 7fea2cdaa740 1 -- 172.21.15.5:0/2980493581 >> 172.21.15.5:0/2980493581 conn(0x5623959e4300 msgr2=0x562395d94480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:49:10.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:10.375+0000 7fe9feffd640 0 client.15726 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:49:10.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:10.375+0000 7fea2cdaa740 1 -- 172.21.15.5:0/2980493581 shutdown_connections 2024-05-09T00:49:10.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:10.375+0000 7fea2cdaa740 1 -- 172.21.15.5:0/2980493581 wait complete. 2024-05-09T00:49:10.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:10.376+0000 7fea2cdaa740 20 client.15726 trim_cache size 0 max 16384 2024-05-09T00:49:10.380 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-09T00:49:10.380 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-09T00:49:10.380 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-09T00:49:10.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.343+0000 7f26655c0700 1 -- 172.21.15.5:0/1339905893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2660105e10 msgr2=0x7f266010e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:10.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.343+0000 7f26655c0700 1 --2- 172.21.15.5:0/1339905893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2660105e10 0x7f266010e360 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f2654009990 tx=0x7f2654009ca0 comp rx=0 tx=0).stop 2024-05-09T00:49:10.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.343+0000 7f26655c0700 1 -- 172.21.15.5:0/1339905893 shutdown_connections 2024-05-09T00:49:10.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.343+0000 7f26655c0700 1 --2- 172.21.15.5:0/1339905893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f266010e8f0 0x7f2660110dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:10.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.343+0000 7f26655c0700 1 --2- 172.21.15.5:0/1339905893 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26601054c0 0x7f26601058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:10.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.343+0000 7f26655c0700 1 --2- 172.21.15.5:0/1339905893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2660105e10 0x7f266010e360 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:10.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.343+0000 7f26655c0700 1 -- 172.21.15.5:0/1339905893 >> 172.21.15.5:0/1339905893 conn(0x7f26600fedf0 msgr2=0x7f2660101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:49:10.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.343+0000 7f26655c0700 1 -- 172.21.15.5:0/1339905893 shutdown_connections 2024-05-09T00:49:10.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.343+0000 7f26655c0700 1 -- 172.21.15.5:0/1339905893 wait complete. 2024-05-09T00:49:10.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.344+0000 7f26655c0700 1 Processor -- start 2024-05-09T00:49:10.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.344+0000 7f26655c0700 1 -- start start 2024-05-09T00:49:10.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.345+0000 7f26655c0700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26601054c0 0x7f266019afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:10.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.345+0000 7f26655c0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2660105e10 0x7f266019b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:10.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.345+0000 7f26655c0700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f266010e8f0 0x7f266019f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:10.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.345+0000 7f26655c0700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2660114560 con 0x7f2660105e10 2024-05-09T00:49:10.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.345+0000 7f26655c0700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f26601143e0 con 0x7f266010e8f0 2024-05-09T00:49:10.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.345+0000 7f26655c0700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f26601146e0 con 0x7f26601054c0 2024-05-09T00:49:10.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.346+0000 7f265e7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2660105e10 0x7f266019b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:10.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.346+0000 7f265e7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2660105e10 0x7f266019b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54620/0 (socket says 172.21.15.5:54620) 2024-05-09T00:49:10.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.346+0000 7f265e7fc700 1 -- 172.21.15.5:0/982810468 learned_addr learned my addr 172.21.15.5:0/982810468 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:49:10.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.346+0000 7f265f7fe700 1 --2- 172.21.15.5:0/982810468 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f266010e8f0 0x7f266019f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:10.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.347+0000 7f265e7fc700 1 -- 172.21.15.5:0/982810468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26601054c0 msgr2=0x7f266019afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:10.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.347+0000 7f265e7fc700 1 --2- 172.21.15.5:0/982810468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26601054c0 0x7f266019afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:10.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.347+0000 7f265e7fc700 1 -- 172.21.15.5:0/982810468 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f266010e8f0 msgr2=0x7f266019f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:10.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.347+0000 7f265e7fc700 1 --2- 172.21.15.5:0/982810468 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f266010e8f0 0x7f266019f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:10.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.347+0000 7f265e7fc700 1 -- 172.21.15.5:0/982810468 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f266019fe80 con 0x7f2660105e10 2024-05-09T00:49:10.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.347+0000 7f265e7fc700 1 --2- 172.21.15.5:0/982810468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2660105e10 0x7f266019b550 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f2654000c00 tx=0x7f26540047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:49:10.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.347+0000 7f2647fff700 1 -- 172.21.15.5:0/982810468 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2654027070 con 0x7f2660105e10 2024-05-09T00:49:10.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.347+0000 7f2647fff700 1 -- 172.21.15.5:0/982810468 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26540041d0 con 0x7f2660105e10 2024-05-09T00:49:10.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.347+0000 7f2647fff700 1 -- 172.21.15.5:0/982810468 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f265400f460 con 0x7f2660105e10 2024-05-09T00:49:10.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.348+0000 7f26655c0700 1 -- 172.21.15.5:0/982810468 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26601a0110 con 0x7f2660105e10 2024-05-09T00:49:10.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.348+0000 7f26655c0700 1 -- 172.21.15.5:0/982810468 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26601a05a0 con 0x7f2660105e10 2024-05-09T00:49:10.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.355+0000 7f2647fff700 1 -- 172.21.15.5:0/982810468 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f2654022670 con 0x7f2660105e10 2024-05-09T00:49:10.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.357+0000 7f2647fff700 1 --2- 172.21.15.5:0/982810468 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f26480b2900 0x7f26480b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:10.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.357+0000 7f2647fff700 1 -- 172.21.15.5:0/982810468 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f26540e3330 con 0x7f2660105e10 2024-05-09T00:49:10.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.358+0000 7f265effd700 1 --2- 172.21.15.5:0/982810468 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f26480b2900 0x7f26480b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:10.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.359+0000 7f265effd700 1 --2- 172.21.15.5:0/982810468 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f26480b2900 0x7f26480b4e10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f266006d150 tx=0x7f2650006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:49:10.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.360+0000 7f2673fd6b80 1 Processor -- start 2024-05-09T00:49:10.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.360+0000 7f2673fd6b80 1 -- start start 2024-05-09T00:49:10.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.360+0000 7f2673fd6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a6b6f3d00 0x563a6b7fc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:10.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.360+0000 7f2673fd6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a6b6794a0 0x563a6b7fc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:10.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.360+0000 7f2673fd6b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a6b7f6400 0x563a6b7f6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:10.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.360+0000 7f2673fd6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563a6b7190a0 con 0x563a6b6f3d00 2024-05-09T00:49:10.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.360+0000 7f2673fd6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x563a6b6be9a0 con 0x563a6b6794a0 2024-05-09T00:49:10.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.360+0000 7f2673fd6b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x563a6b715060 con 0x563a6b7f6400 2024-05-09T00:49:10.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.360+0000 7f265effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a6b6f3d00 0x563a6b7fc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:10.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.361+0000 7f265effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a6b6f3d00 0x563a6b7fc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54624/0 (socket says 172.21.15.5:54624) 2024-05-09T00:49:10.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.361+0000 7f265effd700 1 -- 172.21.15.5:0/2100916562 learned_addr learned my addr 172.21.15.5:0/2100916562 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:49:10.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.361+0000 7f265e7fc700 1 --2- 172.21.15.5:0/2100916562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a6b6794a0 0x563a6b7fc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:10.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.361+0000 7f265effd700 1 -- 172.21.15.5:0/2100916562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a6b7f6400 msgr2=0x563a6b7f6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:49:10.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.361+0000 7f265effd700 1 --2- 172.21.15.5:0/2100916562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a6b7f6400 0x563a6b7f6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:10.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.361+0000 7f265effd700 1 -- 172.21.15.5:0/2100916562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a6b6794a0 msgr2=0x563a6b7fc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:49:10.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.361+0000 7f265effd700 1 --2- 172.21.15.5:0/2100916562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a6b6794a0 0x563a6b7fc780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:10.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.361+0000 7f265effd700 1 -- 172.21.15.5:0/2100916562 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a6b6af500 con 0x563a6b6f3d00 2024-05-09T00:49:10.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.361+0000 7f265effd700 1 --2- 172.21.15.5:0/2100916562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a6b6f3d00 0x563a6b7fc240 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f26500158b0 tx=0x7f2650006f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44b6c46d635a5ef server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:49:10.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.365+0000 7f26457fa700 1 -- 172.21.15.5:0/2100916562 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2650023070 con 0x563a6b6f3d00 2024-05-09T00:49:10.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.365+0000 7f26457fa700 1 -- 172.21.15.5:0/2100916562 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26500152e0 con 0x563a6b6f3d00 2024-05-09T00:49:10.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.365+0000 7f26457fa700 1 -- 172.21.15.5:0/2100916562 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2650023070 con 0x563a6b6f3d00 2024-05-09T00:49:10.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.366+0000 7f2673fd6b80 1 -- 172.21.15.5:0/2100916562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a6b6f3d00 msgr2=0x563a6b7fc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:10.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.366+0000 7f2673fd6b80 1 --2- 172.21.15.5:0/2100916562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a6b6f3d00 0x563a6b7fc240 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f26500158b0 tx=0x7f2650006f90 comp rx=0 tx=0).stop 2024-05-09T00:49:10.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.366+0000 7f2673fd6b80 1 -- 172.21.15.5:0/2100916562 shutdown_connections 2024-05-09T00:49:10.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.366+0000 7f2673fd6b80 1 --2- 172.21.15.5:0/2100916562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a6b7f6400 0x563a6b7f6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:10.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.366+0000 7f2673fd6b80 1 --2- 172.21.15.5:0/2100916562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a6b6f3d00 0x563a6b7fc240 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:10.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.366+0000 7f2673fd6b80 1 --2- 172.21.15.5:0/2100916562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a6b6794a0 0x563a6b7fc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:10.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.366+0000 7f2673fd6b80 1 -- 172.21.15.5:0/2100916562 >> 172.21.15.5:0/2100916562 conn(0x563a6b67cf70 msgr2=0x563a6b7f1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:49:10.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.366+0000 7f2673fd6b80 1 -- 172.21.15.5:0/2100916562 shutdown_connections 2024-05-09T00:49:10.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.366+0000 7f2673fd6b80 1 -- 172.21.15.5:0/2100916562 wait complete. 2024-05-09T00:49:10.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.367+0000 7f2673fd6b80 1 -- start start 2024-05-09T00:49:10.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.367+0000 7f2673fd6b80 1 Processor -- start 2024-05-09T00:49:10.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.368+0000 7f2673fd6b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a6b6794a0 0x563a6b7ffff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:10.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.368+0000 7f2673fd6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a6b6f3d00 0x563a6b800530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:10.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.368+0000 7f2673fd6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a6b7f6400 0x563a6b800a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:10.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.368+0000 7f2673fd6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563a6b7f71b0 con 0x563a6b7f6400 2024-05-09T00:49:10.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.368+0000 7f2673fd6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x563a6b6b3b20 con 0x563a6b6f3d00 2024-05-09T00:49:10.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.368+0000 7f2673fd6b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x563a6b803020 con 0x563a6b6794a0 2024-05-09T00:49:10.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.368+0000 7f2673fd6b80 10 client.0 _ll_register_callbacks cb 0x563a6b58a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:49:10.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.368+0000 7f265f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a6b7f6400 0x563a6b800a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:10.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.368+0000 7f265f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a6b7f6400 0x563a6b800a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54638/0 (socket says 172.21.15.5:54638) 2024-05-09T00:49:10.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.368+0000 7f265f7fe700 1 -- 172.21.15.5:0/384052286 learned_addr learned my addr 172.21.15.5:0/384052286 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:49:10.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.368+0000 7f265e7fc700 1 --2- 172.21.15.5:0/384052286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a6b6f3d00 0x563a6b800530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:10.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.368+0000 7f265f7fe700 1 -- 172.21.15.5:0/384052286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a6b6794a0 msgr2=0x563a6b7ffff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:49:10.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.368+0000 7f265f7fe700 1 --2- 172.21.15.5:0/384052286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a6b6794a0 0x563a6b7ffff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:10.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.368+0000 7f265f7fe700 1 -- 172.21.15.5:0/384052286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a6b6f3d00 msgr2=0x563a6b800530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:49:10.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.368+0000 7f265f7fe700 1 --2- 172.21.15.5:0/384052286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a6b6f3d00 0x563a6b800530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:10.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.368+0000 7f265f7fe700 1 -- 172.21.15.5:0/384052286 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a6b594490 con 0x563a6b7f6400 2024-05-09T00:49:10.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.369+0000 7f265f7fe700 1 --2- 172.21.15.5:0/384052286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a6b7f6400 0x563a6b800a70 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f265800ca40 tx=0x7f265800cb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6eed0f475f83c99 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:49:10.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.369+0000 7f2642ff5700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:49:10.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.369+0000 7f2642ff5700 1 -- 172.21.15.5:0/384052286 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f265801a060 con 0x563a6b7f6400 2024-05-09T00:49:10.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.369+0000 7f2642ff5700 1 -- 172.21.15.5:0/384052286 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f26580074b0 con 0x563a6b7f6400 2024-05-09T00:49:10.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.369+0000 7f2673fd6b80 1 -- 172.21.15.5:0/384052286 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26500152e0 con 0x563a6b7f6400 2024-05-09T00:49:10.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.369+0000 7f2642ff5700 1 -- 172.21.15.5:0/384052286 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f26580076b0 con 0x563a6b7f6400 2024-05-09T00:49:10.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.369+0000 7f2673fd6b80 10 client.15738 fetch_fsmap learned FSMap version 29 2024-05-09T00:49:10.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.369+0000 7f2673fd6b80 1 -- 172.21.15.5:0/384052286 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563a6b811e70 con 0x563a6b7f6400 2024-05-09T00:49:10.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.370+0000 7f2642ff5700 1 -- 172.21.15.5:0/384052286 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2658007d00 con 0x563a6b7f6400 2024-05-09T00:49:10.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.370+0000 7f2642ff5700 1 -- 172.21.15.5:0/384052286 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f2658020a50 con 0x563a6b7f6400 2024-05-09T00:49:10.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.370+0000 7f2673fd6b80 10 client.15738 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:49:10.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.370+0000 7f2673fd6b80 10 client.15738 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T00:49:10.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.370+0000 7f2673fd6b80 20 client.15738 populate_metadata read hostname 'smithi005' 2024-05-09T00:49:10.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.370+0000 7f2673fd6b80 1 --2- 172.21.15.5:0/384052286 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x563a6b8144b0 0x563a6b816910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:10.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.370+0000 7f2673fd6b80 4 client.15738 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:49:10.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.370+0000 7f2673fd6b80 1 -- 172.21.15.5:0/384052286 --> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f26500152e0 con 0x563a6b8144b0 2024-05-09T00:49:10.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.371+0000 7f265effd700 1 --2- 172.21.15.5:0/384052286 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x563a6b8144b0 0x563a6b816910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:10.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.374+0000 7f2642ff5700 10 client.15738 ms_handle_connect on v2:172.21.15.131:6836/1633006186 2024-05-09T00:49:10.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.374+0000 7f265effd700 1 --2- 172.21.15.5:0/384052286 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x563a6b8144b0 0x563a6b816910 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5aae3702dc83d37b server_cookie=84c9e6a3b364a983 in_seq=0 out_seq=0 2024-05-09T00:49:10.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.375+0000 7f2642ff5700 1 -- 172.21.15.5:0/384052286 <== mds.4 v2:172.21.15.131:6836/1633006186 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f26500152e0 con 0x563a6b8144b0 2024-05-09T00:49:10.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.375+0000 7f2642ff5700 10 client.15738 handle_command_reply: tid=0 2024-05-09T00:49:10.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.376+0000 7f2673fd6b80 1 client.15738 shutdown 2024-05-09T00:49:10.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.376+0000 7f2673fd6b80 1 -- 172.21.15.5:0/384052286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a6b7f6400 msgr2=0x563a6b800a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:10.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.376+0000 7f2673fd6b80 1 --2- 172.21.15.5:0/384052286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a6b7f6400 0x563a6b800a70 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f265800ca40 tx=0x7f265800cb20 comp rx=0 tx=0).stop 2024-05-09T00:49:10.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.376+0000 7f2673fd6b80 1 -- 172.21.15.5:0/384052286 shutdown_connections 2024-05-09T00:49:10.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.376+0000 7f2673fd6b80 1 --2- 172.21.15.5:0/384052286 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x563a6b8144b0 0x563a6b816910 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:10.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.376+0000 7f2673fd6b80 1 --2- 172.21.15.5:0/384052286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a6b7f6400 0x563a6b800a70 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:10.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.376+0000 7f2673fd6b80 1 --2- 172.21.15.5:0/384052286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a6b6794a0 0x563a6b7ffff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:10.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.376+0000 7f2673fd6b80 1 --2- 172.21.15.5:0/384052286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a6b6f3d00 0x563a6b800530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:10.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.376+0000 7f2673fd6b80 1 -- 172.21.15.5:0/384052286 >> 172.21.15.5:0/384052286 conn(0x563a6b67cf70 msgr2=0x563a6b7eaff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:49:10.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.376+0000 7f2642ff5700 0 client.15738 ms_handle_reset on v2:172.21.15.131:6836/1633006186 2024-05-09T00:49:10.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.377+0000 7f2673fd6b80 1 -- 172.21.15.5:0/384052286 shutdown_connections 2024-05-09T00:49:10.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.377+0000 7f2673fd6b80 1 -- 172.21.15.5:0/384052286 wait complete. 2024-05-09T00:49:10.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.377+0000 7f2673fd6b80 20 client.15738 trim_cache size 0 max 16384 2024-05-09T00:49:10.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:10.381+0000 7fea25e25640 1 -- 172.21.15.5:0/1566995599 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fea080b2860 msgr2=0x7fea080b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:10.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:10.381+0000 7fea25e25640 1 --2- 172.21.15.5:0/1566995599 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fea080b2860 0x7fea080b4d70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x562395d958e0 tx=0x7fea14005da0 comp rx=0 tx=0).stop 2024-05-09T00:49:10.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:10.381+0000 7fea25e25640 1 -- 172.21.15.5:0/1566995599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea2010a550 msgr2=0x7fea201a0a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:10.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:10.381+0000 7fea25e25640 1 --2- 172.21.15.5:0/1566995599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea2010a550 0x7fea201a0a00 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fea1000b470 tx=0x7fea1000b930 comp rx=0 tx=0).stop 2024-05-09T00:49:10.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:10.382+0000 7fea25e25640 1 -- 172.21.15.5:0/1566995599 shutdown_connections 2024-05-09T00:49:10.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:10.382+0000 7fea25e25640 1 --2- 172.21.15.5:0/1566995599 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fea080b2860 0x7fea080b4d70 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:10.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:10.382+0000 7fea25e25640 1 --2- 172.21.15.5:0/1566995599 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea201101a0 0x7fea201a7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:10.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:10.382+0000 7fea25e25640 1 --2- 172.21.15.5:0/1566995599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea2010b480 0x7fea201a0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:10.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:10.382+0000 7fea25e25640 1 --2- 172.21.15.5:0/1566995599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea2010a550 0x7fea201a0a00 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:10.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:10.382+0000 7fea25e25640 1 -- 172.21.15.5:0/1566995599 >> 172.21.15.5:0/1566995599 conn(0x7fea20070670 msgr2=0x7fea2010e190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:49:10.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:10.382+0000 7fea25e25640 1 -- 172.21.15.5:0/1566995599 shutdown_connections 2024-05-09T00:49:10.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:10.382+0000 7fea25e25640 1 -- 172.21.15.5:0/1566995599 wait complete. 2024-05-09T00:49:10.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.399+0000 7f2673fd6b80 1 Processor -- start 2024-05-09T00:49:10.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.399+0000 7f2673fd6b80 1 -- start start 2024-05-09T00:49:10.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.399+0000 7f2673fd6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a6b6794a0 0x563a6b802490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:10.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.399+0000 7f2673fd6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a6b6f3d00 0x563a6b8008b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:10.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.399+0000 7f2673fd6b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a6b7f6400 0x563a6b7eb700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:10.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.399+0000 7f2673fd6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f26601acb10 con 0x563a6b6f3d00 2024-05-09T00:49:10.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.399+0000 7f2673fd6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f26601ae650 con 0x563a6b6794a0 2024-05-09T00:49:10.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.399+0000 7f2673fd6b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f26601ad1e0 con 0x563a6b7f6400 2024-05-09T00:49:10.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.400+0000 7f265e7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a6b6f3d00 0x563a6b8008b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:10.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.400+0000 7f265e7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a6b6f3d00 0x563a6b8008b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54652/0 (socket says 172.21.15.5:54652) 2024-05-09T00:49:10.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.400+0000 7f265e7fc700 1 -- 172.21.15.5:0/3799193028 learned_addr learned my addr 172.21.15.5:0/3799193028 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:49:10.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.400+0000 7f265e7fc700 1 -- 172.21.15.5:0/3799193028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a6b7f6400 msgr2=0x563a6b7eb700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:49:10.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.400+0000 7f265e7fc700 1 --2- 172.21.15.5:0/3799193028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a6b7f6400 0x563a6b7eb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:10.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.400+0000 7f265e7fc700 1 -- 172.21.15.5:0/3799193028 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a6b6794a0 msgr2=0x563a6b802490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:49:10.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.400+0000 7f265e7fc700 1 --2- 172.21.15.5:0/3799193028 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a6b6794a0 0x563a6b802490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:10.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.400+0000 7f265e7fc700 1 -- 172.21.15.5:0/3799193028 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a6b8010b0 con 0x563a6b6f3d00 2024-05-09T00:49:10.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.401+0000 7f265e7fc700 1 --2- 172.21.15.5:0/3799193028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a6b6f3d00 0x563a6b8008b0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f26540e0e00 tx=0x7f26540aa230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d385bc0893ec93d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:49:10.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.401+0000 7f2644ff9700 1 -- 172.21.15.5:0/3799193028 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26540c4070 con 0x563a6b6f3d00 2024-05-09T00:49:10.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.401+0000 7f2644ff9700 1 -- 172.21.15.5:0/3799193028 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26540aa4c0 con 0x563a6b6f3d00 2024-05-09T00:49:10.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.401+0000 7f2644ff9700 1 -- 172.21.15.5:0/3799193028 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26540b10c0 con 0x563a6b6f3d00 2024-05-09T00:49:10.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.402+0000 7f2673fd6b80 1 -- 172.21.15.5:0/3799193028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a6b6f3d00 msgr2=0x563a6b8008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:10.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.402+0000 7f2673fd6b80 1 --2- 172.21.15.5:0/3799193028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a6b6f3d00 0x563a6b8008b0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f26540e0e00 tx=0x7f26540aa230 comp rx=0 tx=0).stop 2024-05-09T00:49:10.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.403+0000 7f2673fd6b80 1 -- 172.21.15.5:0/3799193028 shutdown_connections 2024-05-09T00:49:10.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.404+0000 7f2673fd6b80 1 --2- 172.21.15.5:0/3799193028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a6b7f6400 0x563a6b7eb700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:10.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.404+0000 7f2673fd6b80 1 --2- 172.21.15.5:0/3799193028 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a6b6794a0 0x563a6b802490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:10.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.404+0000 7f2673fd6b80 1 --2- 172.21.15.5:0/3799193028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a6b6f3d00 0x563a6b8008b0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:10.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.404+0000 7f2673fd6b80 1 -- 172.21.15.5:0/3799193028 >> 172.21.15.5:0/3799193028 conn(0x563a6b67cf70 msgr2=0x563a6b7eaff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:49:10.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.404+0000 7f2673fd6b80 1 -- 172.21.15.5:0/3799193028 shutdown_connections 2024-05-09T00:49:10.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.404+0000 7f2673fd6b80 1 -- 172.21.15.5:0/3799193028 wait complete. 2024-05-09T00:49:10.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.405+0000 7f2673fd6b80 1 -- start start 2024-05-09T00:49:10.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.405+0000 7f2673fd6b80 1 Processor -- start 2024-05-09T00:49:10.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.406+0000 7f2673fd6b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a6b6794a0 0x563a6b81b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:10.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.406+0000 7f2673fd6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a6b6f3d00 0x563a6b81b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:10.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.406+0000 7f2673fd6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a6b7f6400 0x563a6b81bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:10.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.407+0000 7f265f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a6b7f6400 0x563a6b81bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:10.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.407+0000 7f265e7fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a6b6f3d00 0x563a6b81b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:10.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.407+0000 7f265e7fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a6b6f3d00 0x563a6b81b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:40166/0 (socket says 172.21.15.5:40166) 2024-05-09T00:49:10.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.407+0000 7f265e7fc700 1 -- 172.21.15.5:0/3728564044 learned_addr learned my addr 172.21.15.5:0/3728564044 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:49:10.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.407+0000 7f2673fd6b80 1 -- 172.21.15.5:0/3728564044 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f26601adf80 con 0x563a6b7f6400 2024-05-09T00:49:10.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.407+0000 7f2673fd6b80 1 -- 172.21.15.5:0/3728564044 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f26601ad8b0 con 0x563a6b6f3d00 2024-05-09T00:49:10.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.407+0000 7f2673fd6b80 1 -- 172.21.15.5:0/3728564044 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f26601aaab0 con 0x563a6b6794a0 2024-05-09T00:49:10.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.407+0000 7f265effd700 1 --2- 172.21.15.5:0/3728564044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a6b6794a0 0x563a6b81b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:10.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.408+0000 7f2673fd6b80 10 client.0 _ll_register_callbacks cb 0x563a6b7fbd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:49:10.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.408+0000 7f265f7fe700 1 -- 172.21.15.5:0/3728564044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a6b6794a0 msgr2=0x563a6b81b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:49:10.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.408+0000 7f265f7fe700 1 --2- 172.21.15.5:0/3728564044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a6b6794a0 0x563a6b81b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:10.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.408+0000 7f265f7fe700 1 -- 172.21.15.5:0/3728564044 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a6b6f3d00 msgr2=0x563a6b81b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:49:10.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.408+0000 7f265f7fe700 1 --2- 172.21.15.5:0/3728564044 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a6b6f3d00 0x563a6b81b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:10.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.408+0000 7f265f7fe700 1 -- 172.21.15.5:0/3728564044 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a6b81cb00 con 0x563a6b7f6400 2024-05-09T00:49:10.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.408+0000 7f265e7fc700 1 --2- 172.21.15.5:0/3728564044 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a6b6f3d00 0x563a6b81b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:49:10.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.408+0000 7f265effd700 1 --2- 172.21.15.5:0/3728564044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a6b6794a0 0x563a6b81b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:49:10.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.409+0000 7f265f7fe700 1 --2- 172.21.15.5:0/3728564044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a6b7f6400 0x563a6b81bb40 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f2658020e70 tx=0x7f265800c2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40e70d5d7ea1feb3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:49:10.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.409+0000 7f2642ff5700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:49:10.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.409+0000 7f2642ff5700 1 -- 172.21.15.5:0/3728564044 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2658015ac0 con 0x563a6b7f6400 2024-05-09T00:49:10.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.409+0000 7f2642ff5700 1 -- 172.21.15.5:0/3728564044 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f2658015c60 con 0x563a6b7f6400 2024-05-09T00:49:10.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.410+0000 7f2673fd6b80 1 -- 172.21.15.5:0/3728564044 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26540aa4c0 con 0x563a6b7f6400 2024-05-09T00:49:10.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.410+0000 7f2642ff5700 1 -- 172.21.15.5:0/3728564044 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2658007850 con 0x563a6b7f6400 2024-05-09T00:49:10.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.410+0000 7f2673fd6b80 10 client.15744 fetch_fsmap learned FSMap version 29 2024-05-09T00:49:10.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.410+0000 7f2673fd6b80 1 -- 172.21.15.5:0/3728564044 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563a6b8824c0 con 0x563a6b7f6400 2024-05-09T00:49:10.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.411+0000 7f2642ff5700 1 -- 172.21.15.5:0/3728564044 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2658020410 con 0x563a6b7f6400 2024-05-09T00:49:10.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.411+0000 7f2642ff5700 1 -- 172.21.15.5:0/3728564044 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f2658027070 con 0x563a6b7f6400 2024-05-09T00:49:10.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.411+0000 7f2673fd6b80 10 client.15744 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:49:10.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.411+0000 7f2673fd6b80 10 client.15744 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T00:49:10.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.411+0000 7f2673fd6b80 20 client.15744 populate_metadata read hostname 'smithi005' 2024-05-09T00:49:10.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.411+0000 7f2673fd6b80 1 --2- 172.21.15.5:0/3728564044 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x563a6b882bd0 0x563a6b884ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:10.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.411+0000 7f2673fd6b80 4 client.15744 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:49:10.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.411+0000 7f2673fd6b80 1 -- 172.21.15.5:0/3728564044 --> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f26540aa4c0 con 0x563a6b882bd0 2024-05-09T00:49:10.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.413+0000 7f265effd700 1 --2- 172.21.15.5:0/3728564044 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x563a6b882bd0 0x563a6b884ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:10.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.416+0000 7f265effd700 1 --2- 172.21.15.5:0/3728564044 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x563a6b882bd0 0x563a6b884ff0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e3bbc800eba3cd60 server_cookie=5bd3b0825e853203 in_seq=0 out_seq=0 2024-05-09T00:49:10.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.416+0000 7f2642ff5700 10 client.15744 ms_handle_connect on v2:172.21.15.131:6836/1633006186 2024-05-09T00:49:10.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.417+0000 7f2642ff5700 1 -- 172.21.15.5:0/3728564044 <== mds.4 v2:172.21.15.131:6836/1633006186 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f26540aa4c0 con 0x563a6b882bd0 2024-05-09T00:49:10.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.417+0000 7f2642ff5700 10 client.15744 handle_command_reply: tid=0 2024-05-09T00:49:10.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.417+0000 7f2673fd6b80 1 client.15744 shutdown 2024-05-09T00:49:10.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.419+0000 7f2673fd6b80 1 -- 172.21.15.5:0/3728564044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a6b7f6400 msgr2=0x563a6b81bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:10.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.419+0000 7f2673fd6b80 1 --2- 172.21.15.5:0/3728564044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a6b7f6400 0x563a6b81bb40 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f2658020e70 tx=0x7f265800c2e0 comp rx=0 tx=0).stop 2024-05-09T00:49:10.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.421+0000 7f2673fd6b80 1 -- 172.21.15.5:0/3728564044 shutdown_connections 2024-05-09T00:49:10.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.421+0000 7f2673fd6b80 1 --2- 172.21.15.5:0/3728564044 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x563a6b882bd0 0x563a6b884ff0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:10.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.421+0000 7f2642ff5700 0 client.15744 ms_handle_reset on v2:172.21.15.131:6836/1633006186 2024-05-09T00:49:10.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.421+0000 7f2673fd6b80 1 --2- 172.21.15.5:0/3728564044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a6b7f6400 0x563a6b81bb40 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:10.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.421+0000 7f2673fd6b80 1 --2- 172.21.15.5:0/3728564044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a6b6794a0 0x563a6b81b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:10.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.421+0000 7f2673fd6b80 1 --2- 172.21.15.5:0/3728564044 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563a6b6f3d00 0x563a6b81b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:10.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.421+0000 7f2673fd6b80 1 -- 172.21.15.5:0/3728564044 >> 172.21.15.5:0/3728564044 conn(0x563a6b67cf70 msgr2=0x563a6b7f9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:49:10.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.422+0000 7f2673fd6b80 1 -- 172.21.15.5:0/3728564044 shutdown_connections 2024-05-09T00:49:10.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.422+0000 7f2673fd6b80 1 -- 172.21.15.5:0/3728564044 wait complete. 2024-05-09T00:49:10.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.422+0000 7f2673fd6b80 20 client.15744 trim_cache size 0 max 16384 2024-05-09T00:49:10.425 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-09T00:49:10.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.423+0000 7f26655c0700 1 -- 172.21.15.5:0/982810468 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f26480b2900 msgr2=0x7f26480b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:10.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.423+0000 7f26655c0700 1 --2- 172.21.15.5:0/982810468 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f26480b2900 0x7f26480b4e10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f266006d150 tx=0x7f2650006d20 comp rx=0 tx=0).stop 2024-05-09T00:49:10.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.424+0000 7f26655c0700 1 -- 172.21.15.5:0/982810468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2660105e10 msgr2=0x7f266019b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:10.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.424+0000 7f26655c0700 1 --2- 172.21.15.5:0/982810468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2660105e10 0x7f266019b550 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f2654000c00 tx=0x7f26540047b0 comp rx=0 tx=0).stop 2024-05-09T00:49:10.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.424+0000 7f26655c0700 1 -- 172.21.15.5:0/982810468 shutdown_connections 2024-05-09T00:49:10.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.424+0000 7f26655c0700 1 --2- 172.21.15.5:0/982810468 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f26480b2900 0x7f26480b4e10 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:10.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.424+0000 7f26655c0700 1 --2- 172.21.15.5:0/982810468 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f266010e8f0 0x7f266019f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:10.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.424+0000 7f26655c0700 1 --2- 172.21.15.5:0/982810468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26601054c0 0x7f266019afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:10.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.424+0000 7f26655c0700 1 --2- 172.21.15.5:0/982810468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2660105e10 0x7f266019b550 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:10.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.424+0000 7f26655c0700 1 -- 172.21.15.5:0/982810468 >> 172.21.15.5:0/982810468 conn(0x7f26600fedf0 msgr2=0x7f26600ff540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:49:10.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.440+0000 7f26655c0700 1 -- 172.21.15.5:0/982810468 shutdown_connections 2024-05-09T00:49:10.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:10.442+0000 7f26655c0700 1 -- 172.21.15.5:0/982810468 wait complete. 2024-05-09T00:49:11.656 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:49:11 smithi005 ceph-mon[30365]: pgmap v456: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 970 MiB/s wr, 1.44k op/s 2024-05-09T00:49:11.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:49:11 smithi157 ceph-mon[34294]: pgmap v456: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 970 MiB/s wr, 1.44k op/s 2024-05-09T00:49:11.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:49:11 smithi131 ceph-mon[34547]: pgmap v456: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 970 MiB/s wr, 1.44k op/s 2024-05-09T00:49:12.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:49:12 smithi157 ceph-mon[34294]: pgmap v457: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 837 MiB/s wr, 1.44k op/s 2024-05-09T00:49:12.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:49:12 smithi005 ceph-mon[30365]: pgmap v457: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 837 MiB/s wr, 1.44k op/s 2024-05-09T00:49:12.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:49:12 smithi131 ceph-mon[34547]: pgmap v457: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 837 MiB/s wr, 1.44k op/s 2024-05-09T00:49:12.931 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-09T00:49:13.683 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:49:15.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:49:15 smithi005 ceph-mon[30365]: pgmap v458: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 890 MiB/s wr, 1.75k op/s 2024-05-09T00:49:15.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:49:15 smithi131 ceph-mon[34547]: pgmap v458: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 890 MiB/s wr, 1.75k op/s 2024-05-09T00:49:15.435 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:49:15 smithi157 ceph-mon[34294]: pgmap v458: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 890 MiB/s wr, 1.75k op/s 2024-05-09T00:49:17.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:49:17 smithi005 ceph-mon[30365]: pgmap v459: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 989 MiB/s wr, 2.28k op/s 2024-05-09T00:49:17.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:49:17 smithi131 ceph-mon[34547]: pgmap v459: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 989 MiB/s wr, 2.28k op/s 2024-05-09T00:49:17.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:49:17 smithi157 ceph-mon[34294]: pgmap v459: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 989 MiB/s wr, 2.28k op/s 2024-05-09T00:49:19.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.144+0000 7f10b8b9e700 1 -- 172.21.15.5:0/1114221931 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b4076440 con 0x7f10ac0b8330 2024-05-09T00:49:19.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:49:19 smithi005 ceph-mon[30365]: pgmap v460: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 809 MiB/s wr, 2.23k op/s 2024-05-09T00:49:19.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.145+0000 7f10bbe04700 1 -- 172.21.15.5:0/1114221931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10ac0b8330 msgr2=0x7f10ac0ba750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:19.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.145+0000 7f10bbe04700 1 --2- 172.21.15.5:0/1114221931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10ac0b8330 0x7f10ac0ba750 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f10b40528b0 tx=0x7f10b406a490 comp rx=0 tx=0).stop 2024-05-09T00:49:19.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.145+0000 7f10bbe04700 1 -- 172.21.15.5:0/1114221931 shutdown_connections 2024-05-09T00:49:19.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.145+0000 7f10bbe04700 1 --2- 172.21.15.5:0/1114221931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10ac0b8330 0x7f10ac0ba750 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:19.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.145+0000 7f10bbe04700 1 --2- 172.21.15.5:0/1114221931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10ac0a53d0 0x7f10ac0a57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:19.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.145+0000 7f10bbe04700 1 --2- 172.21.15.5:0/1114221931 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f10ac0a5d20 0x7f10ac0b7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:19.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.145+0000 7f10bbe04700 1 -- 172.21.15.5:0/1114221931 >> 172.21.15.5:0/1114221931 conn(0x7f10ac01af30 msgr2=0x7f10ac01b370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:49:19.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.146+0000 7f10bbe04700 1 -- 172.21.15.5:0/1114221931 shutdown_connections 2024-05-09T00:49:19.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.147+0000 7f10bbe04700 1 -- 172.21.15.5:0/1114221931 wait complete. 2024-05-09T00:49:19.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.148+0000 7f10bbe04700 1 Processor -- start 2024-05-09T00:49:19.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.148+0000 7f10bbe04700 1 -- start start 2024-05-09T00:49:19.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.148+0000 7f10bbe04700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f10ac0a53d0 0x7f10ac144660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:19.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.148+0000 7f10bbe04700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10ac0a5d20 0x7f10ac144c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:19.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.148+0000 7f10bbe04700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10ac0b8330 0x7f10ac148e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:19.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.148+0000 7f10bbe04700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f10ac0bdcf0 con 0x7f10ac0a5d20 2024-05-09T00:49:19.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.148+0000 7f10bbe04700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f10ac0bdb70 con 0x7f10ac0a53d0 2024-05-09T00:49:19.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.148+0000 7f10bbe04700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f10ac0bde70 con 0x7f10ac0b8330 2024-05-09T00:49:19.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.149+0000 7f10b939f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10ac0a5d20 0x7f10ac144c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:19.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.149+0000 7f10b939f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10ac0a5d20 0x7f10ac144c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55550/0 (socket says 172.21.15.5:55550) 2024-05-09T00:49:19.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.149+0000 7f10b939f700 1 -- 172.21.15.5:0/1843332407 learned_addr learned my addr 172.21.15.5:0/1843332407 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:49:19.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.149+0000 7f10ba3a1700 1 --2- 172.21.15.5:0/1843332407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10ac0b8330 0x7f10ac148e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:19.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.149+0000 7f10b9ba0700 1 --2- 172.21.15.5:0/1843332407 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f10ac0a53d0 0x7f10ac144660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:19.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.150+0000 7f10ba3a1700 1 -- 172.21.15.5:0/1843332407 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f10ac0a53d0 msgr2=0x7f10ac144660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:19.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.150+0000 7f10ba3a1700 1 --2- 172.21.15.5:0/1843332407 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f10ac0a53d0 0x7f10ac144660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:19.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.150+0000 7f10ba3a1700 1 -- 172.21.15.5:0/1843332407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10ac0a5d20 msgr2=0x7f10ac144c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:19.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.150+0000 7f10ba3a1700 1 --2- 172.21.15.5:0/1843332407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10ac0a5d20 0x7f10ac144c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:19.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.150+0000 7f10ba3a1700 1 -- 172.21.15.5:0/1843332407 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10ac149530 con 0x7f10ac0b8330 2024-05-09T00:49:19.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.150+0000 7f10ba3a1700 1 --2- 172.21.15.5:0/1843332407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10ac0b8330 0x7f10ac148e90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f10b4069820 tx=0x7f10b406c340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:49:19.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.151+0000 7f10aaffd700 1 -- 172.21.15.5:0/1843332407 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b4086020 con 0x7f10ac0b8330 2024-05-09T00:49:19.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.151+0000 7f10bbe04700 1 -- 172.21.15.5:0/1843332407 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10ac1497c0 con 0x7f10ac0b8330 2024-05-09T00:49:19.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.151+0000 7f10bbe04700 1 -- 172.21.15.5:0/1843332407 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10ac149cc0 con 0x7f10ac0b8330 2024-05-09T00:49:19.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.152+0000 7f10aaffd700 1 -- 172.21.15.5:0/1843332407 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10b406ac10 con 0x7f10ac0b8330 2024-05-09T00:49:19.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.152+0000 7f10aaffd700 1 -- 172.21.15.5:0/1843332407 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b4082690 con 0x7f10ac0b8330 2024-05-09T00:49:19.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.157+0000 7f10ca81ab80 1 Processor -- start 2024-05-09T00:49:19.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.157+0000 7f10aaffd700 1 -- 172.21.15.5:0/1843332407 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f10b407e070 con 0x7f10ac0b8330 2024-05-09T00:49:19.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.158+0000 7f10ca81ab80 1 -- start start 2024-05-09T00:49:19.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.159+0000 7f10aaffd700 1 --2- 172.21.15.5:0/1843332407 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f10a00b2900 0x7f10a00b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:19.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.159+0000 7f10ca81ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ebb0604160 0x55ebb069d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:19.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.159+0000 7f10aaffd700 1 -- 172.21.15.5:0/1843332407 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f10b4148620 con 0x7f10ac0b8330 2024-05-09T00:49:19.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.159+0000 7f10ca81ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebb051a4a0 0x55ebb069db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:19.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.159+0000 7f10b9ba0700 1 --2- 172.21.15.5:0/1843332407 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f10a00b2900 0x7f10a00b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:19.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.160+0000 7f10ca81ab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ebb0697800 0x55ebb0697c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:19.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.160+0000 7f10ca81ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ebb05b9ff0 con 0x55ebb051a4a0 2024-05-09T00:49:19.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.160+0000 7f10ca81ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55ebb05587f0 con 0x55ebb0604160 2024-05-09T00:49:19.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.160+0000 7f10ca81ab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ebb05b5fb0 con 0x55ebb0697800 2024-05-09T00:49:19.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.160+0000 7f10b939f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebb051a4a0 0x55ebb069db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:19.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.161+0000 7f10b939f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebb051a4a0 0x55ebb069db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55554/0 (socket says 172.21.15.5:55554) 2024-05-09T00:49:19.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.161+0000 7f10b939f700 1 -- 172.21.15.5:0/1741538783 learned_addr learned my addr 172.21.15.5:0/1741538783 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:49:19.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.161+0000 7f10b939f700 1 --2- 172.21.15.5:0/1741538783 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ebb0604160 0x55ebb069d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:19.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.161+0000 7f10ba3a1700 1 --2- 172.21.15.5:0/1741538783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ebb0697800 0x55ebb0697c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:19.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.162+0000 7f10b939f700 1 -- 172.21.15.5:0/1741538783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ebb0697800 msgr2=0x55ebb0697c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:49:19.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.162+0000 7f10b939f700 1 --2- 172.21.15.5:0/1741538783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ebb0697800 0x55ebb0697c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:19.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.162+0000 7f10b939f700 1 -- 172.21.15.5:0/1741538783 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ebb0604160 msgr2=0x55ebb069d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:49:19.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.162+0000 7f10b939f700 1 --2- 172.21.15.5:0/1741538783 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ebb0604160 0x55ebb069d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:19.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.162+0000 7f10b939f700 1 -- 172.21.15.5:0/1741538783 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebb0550370 con 0x55ebb051a4a0 2024-05-09T00:49:19.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.162+0000 7f10b939f700 1 --2- 172.21.15.5:0/1741538783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebb051a4a0 0x55ebb069db80 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f10b000bfc0 tx=0x7f10b000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67489217fdbe3520 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:49:19.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.162+0000 7f109ffff700 1 -- 172.21.15.5:0/1741538783 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b000f5f0 con 0x55ebb051a4a0 2024-05-09T00:49:19.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.163+0000 7f109ffff700 1 -- 172.21.15.5:0/1741538783 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10b000fc40 con 0x55ebb051a4a0 2024-05-09T00:49:19.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.163+0000 7f109ffff700 1 -- 172.21.15.5:0/1741538783 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b000ebe0 con 0x55ebb051a4a0 2024-05-09T00:49:19.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.164+0000 7f10ca81ab80 1 -- 172.21.15.5:0/1741538783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebb051a4a0 msgr2=0x55ebb069db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:19.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.164+0000 7f10ca81ab80 1 --2- 172.21.15.5:0/1741538783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebb051a4a0 0x55ebb069db80 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f10b000bfc0 tx=0x7f10b000b8f0 comp rx=0 tx=0).stop 2024-05-09T00:49:19.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.165+0000 7f10b9ba0700 1 --2- 172.21.15.5:0/1843332407 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f10a00b2900 0x7f10a00b4e10 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f10a4009680 tx=0x7f10a4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:49:19.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.165+0000 7f10ca81ab80 1 -- 172.21.15.5:0/1741538783 shutdown_connections 2024-05-09T00:49:19.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.165+0000 7f10ca81ab80 1 --2- 172.21.15.5:0/1741538783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ebb0697800 0x55ebb0697c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:19.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.165+0000 7f10ca81ab80 1 --2- 172.21.15.5:0/1741538783 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ebb0604160 0x55ebb069d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:19.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.165+0000 7f10ca81ab80 1 --2- 172.21.15.5:0/1741538783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebb051a4a0 0x55ebb069db80 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:19.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.165+0000 7f10ca81ab80 1 -- 172.21.15.5:0/1741538783 >> 172.21.15.5:0/1741538783 conn(0x55ebb057a6a0 msgr2=0x55ebb0692620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:49:19.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.166+0000 7f10ca81ab80 1 -- 172.21.15.5:0/1741538783 shutdown_connections 2024-05-09T00:49:19.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.166+0000 7f10ca81ab80 1 -- 172.21.15.5:0/1741538783 wait complete. 2024-05-09T00:49:19.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.167+0000 7f10ca81ab80 1 -- start start 2024-05-09T00:49:19.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.167+0000 7f10ca81ab80 1 Processor -- start 2024-05-09T00:49:19.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.168+0000 7f10ca81ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ebb051a4a0 0x55ebb06a13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:19.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.168+0000 7f10ca81ab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ebb0604160 0x55ebb06a1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:19.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.168+0000 7f10ca81ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebb0697800 0x55ebb06a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:19.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.169+0000 7f10ca81ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ebb0435ba0 con 0x55ebb0697800 2024-05-09T00:49:19.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.169+0000 7f10ca81ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55ebb0574150 con 0x55ebb051a4a0 2024-05-09T00:49:19.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.169+0000 7f10ca81ab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ebb06985f0 con 0x55ebb0604160 2024-05-09T00:49:19.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.169+0000 7f10b9ba0700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ebb051a4a0 0x55ebb06a13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:19.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.169+0000 7f10b9ba0700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ebb051a4a0 0x55ebb06a13f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:36422/0 (socket says 172.21.15.5:36422) 2024-05-09T00:49:19.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.169+0000 7f10b9ba0700 1 -- 172.21.15.5:0/1525205431 learned_addr learned my addr 172.21.15.5:0/1525205431 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:49:19.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.169+0000 7f10b939f700 1 --2- 172.21.15.5:0/1525205431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ebb0604160 0x55ebb06a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:19.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.170+0000 7f10b9ba0700 1 -- 172.21.15.5:0/1525205431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ebb0604160 msgr2=0x55ebb06a1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:49:19.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.170+0000 7f10b9ba0700 1 --2- 172.21.15.5:0/1525205431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ebb0604160 0x55ebb06a1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:19.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.170+0000 7f10b9ba0700 1 -- 172.21.15.5:0/1525205431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebb0697800 msgr2=0x55ebb06a1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:49:19.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.170+0000 7f10b9ba0700 1 --2- 172.21.15.5:0/1525205431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebb0697800 0x55ebb06a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:19.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.170+0000 7f10b9ba0700 1 -- 172.21.15.5:0/1525205431 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebb0435490 con 0x55ebb051a4a0 2024-05-09T00:49:19.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.171+0000 7f10b9ba0700 1 --2- 172.21.15.5:0/1525205431 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ebb051a4a0 0x55ebb06a13f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f10a40127d0 tx=0x7f10a40047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70e666b8092b953d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:49:19.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.172+0000 7f10ca81ab80 10 client.0 _ll_register_callbacks cb 0x55ebb042b6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:49:19.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.172+0000 7f10ca81ab80 1 -- 172.21.15.5:0/1525205431 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10b000fc40 con 0x55ebb051a4a0 2024-05-09T00:49:19.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.172+0000 7f109d7fa700 10 client.25409 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-09T00:49:19.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.172+0000 7f109d7fa700 1 -- 172.21.15.5:0/1525205431 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a4004a30 con 0x55ebb051a4a0 2024-05-09T00:49:19.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.173+0000 7f109d7fa700 1 -- 172.21.15.5:0/1525205431 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f10a4004bd0 con 0x55ebb051a4a0 2024-05-09T00:49:19.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.173+0000 7f109d7fa700 1 -- 172.21.15.5:0/1525205431 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f10a402e050 con 0x55ebb051a4a0 2024-05-09T00:49:19.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.173+0000 7f10ca81ab80 10 client.25409 fetch_fsmap learned FSMap version 29 2024-05-09T00:49:19.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.173+0000 7f10ca81ab80 1 -- 172.21.15.5:0/1525205431 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({fsmap=29}) -- 0x55ebb06aea40 con 0x55ebb051a4a0 2024-05-09T00:49:19.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.174+0000 7f109d7fa700 1 -- 172.21.15.5:0/1525205431 <== mon.2 v2:172.21.15.131:3300/0 4 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f10a4005020 con 0x55ebb051a4a0 2024-05-09T00:49:19.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.175+0000 7f10ca81ab80 10 client.25409 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:49:19.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.175+0000 7f10ca81ab80 10 client.25409 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T00:49:19.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.175+0000 7f10ca81ab80 20 client.25409 populate_metadata read hostname 'smithi005' 2024-05-09T00:49:19.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.175+0000 7f10ca81ab80 1 --2- 172.21.15.5:0/1525205431 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x55ebb06b0f70 0x55ebb06b33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:19.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.175+0000 7f10ca81ab80 4 client.25409 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:49:19.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.175+0000 7f10ca81ab80 1 -- 172.21.15.5:0/1525205431 --> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f10b000fc40 con 0x55ebb06b0f70 2024-05-09T00:49:19.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.176+0000 7f10b939f700 1 --2- 172.21.15.5:0/1525205431 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x55ebb06b0f70 0x55ebb06b33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:19.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.180+0000 7f10b939f700 1 --2- 172.21.15.5:0/1525205431 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x55ebb06b0f70 0x55ebb06b33b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=26591d98afe84237 server_cookie=e85ddc6532ed24ae in_seq=0 out_seq=0 2024-05-09T00:49:19.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.180+0000 7f109d7fa700 10 client.25409 ms_handle_connect on v2:172.21.15.157:6834/832898273 2024-05-09T00:49:19.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.182+0000 7f109d7fa700 1 -- 172.21.15.5:0/1525205431 <== mds.3 v2:172.21.15.157:6834/832898273 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f10b000fc40 con 0x55ebb06b0f70 2024-05-09T00:49:19.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.182+0000 7f109d7fa700 10 client.25409 handle_command_reply: tid=0 2024-05-09T00:49:19.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.184+0000 7f10ca81ab80 1 client.25409 shutdown 2024-05-09T00:49:19.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.185+0000 7f10ca81ab80 1 -- 172.21.15.5:0/1525205431 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ebb051a4a0 msgr2=0x55ebb06a13f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:19.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.185+0000 7f10ca81ab80 1 --2- 172.21.15.5:0/1525205431 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ebb051a4a0 0x55ebb06a13f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f10a40127d0 tx=0x7f10a40047c0 comp rx=0 tx=0).stop 2024-05-09T00:49:19.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.185+0000 7f10ca81ab80 1 -- 172.21.15.5:0/1525205431 shutdown_connections 2024-05-09T00:49:19.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.185+0000 7f10ca81ab80 1 --2- 172.21.15.5:0/1525205431 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x55ebb06b0f70 0x55ebb06b33b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:19.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.185+0000 7f10ca81ab80 1 --2- 172.21.15.5:0/1525205431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebb0697800 0x55ebb06a1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:19.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.185+0000 7f10ca81ab80 1 --2- 172.21.15.5:0/1525205431 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ebb051a4a0 0x55ebb06a13f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:19.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.185+0000 7f10ca81ab80 1 --2- 172.21.15.5:0/1525205431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ebb0604160 0x55ebb06a1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:19.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.185+0000 7f10ca81ab80 1 -- 172.21.15.5:0/1525205431 >> 172.21.15.5:0/1525205431 conn(0x55ebb057a6a0 msgr2=0x55ebb0692380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:49:19.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.186+0000 7f109d7fa700 0 client.25409 ms_handle_reset on v2:172.21.15.157:6834/832898273 2024-05-09T00:49:19.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.186+0000 7f10ca81ab80 1 -- 172.21.15.5:0/1525205431 shutdown_connections 2024-05-09T00:49:19.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.187+0000 7f10ca81ab80 1 -- 172.21.15.5:0/1525205431 wait complete. 2024-05-09T00:49:19.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.187+0000 7f10ca81ab80 20 client.25409 trim_cache size 0 max 16384 2024-05-09T00:49:19.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.220+0000 7f10ca81ab80 1 Processor -- start 2024-05-09T00:49:19.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.220+0000 7f10ca81ab80 1 -- start start 2024-05-09T00:49:19.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.221+0000 7f10ca81ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ebb051a4a0 0x55ebb069cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:19.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.221+0000 7f10ca81ab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ebb0604160 0x55ebb06ad1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:19.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.221+0000 7f10ca81ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebb0697800 0x55ebb06a13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:19.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.221+0000 7f10ca81ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f109800d4f0 con 0x55ebb0697800 2024-05-09T00:49:19.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.221+0000 7f10ca81ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f109800f030 con 0x55ebb051a4a0 2024-05-09T00:49:19.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.222+0000 7f10ca81ab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f109800dbc0 con 0x55ebb0604160 2024-05-09T00:49:19.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.222+0000 7f10ba3a1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebb0697800 0x55ebb06a13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:19.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.222+0000 7f10ba3a1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebb0697800 0x55ebb06a13b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55576/0 (socket says 172.21.15.5:55576) 2024-05-09T00:49:19.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.222+0000 7f10ba3a1700 1 -- 172.21.15.5:0/3373133283 learned_addr learned my addr 172.21.15.5:0/3373133283 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:49:19.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.222+0000 7f10ba3a1700 1 -- 172.21.15.5:0/3373133283 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ebb0604160 msgr2=0x55ebb06ad1a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:49:19.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.222+0000 7f10ba3a1700 1 --2- 172.21.15.5:0/3373133283 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ebb0604160 0x55ebb06ad1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:19.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.222+0000 7f10ba3a1700 1 -- 172.21.15.5:0/3373133283 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ebb051a4a0 msgr2=0x55ebb069cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:49:19.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.222+0000 7f10ba3a1700 1 --2- 172.21.15.5:0/3373133283 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ebb051a4a0 0x55ebb069cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:19.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.222+0000 7f10ba3a1700 1 -- 172.21.15.5:0/3373133283 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebb068c0c0 con 0x55ebb0697800 2024-05-09T00:49:19.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.223+0000 7f10ba3a1700 1 --2- 172.21.15.5:0/3373133283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebb0697800 0x55ebb06a13b0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f10b410cf10 tx=0x7f10b4064c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4812e486806832e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:49:19.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.223+0000 7f109f7fe700 1 -- 172.21.15.5:0/3373133283 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b410f5f0 con 0x55ebb0697800 2024-05-09T00:49:19.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.224+0000 7f109f7fe700 1 -- 172.21.15.5:0/3373133283 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10b4076e40 con 0x55ebb0697800 2024-05-09T00:49:19.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.224+0000 7f109f7fe700 1 -- 172.21.15.5:0/3373133283 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b410f410 con 0x55ebb0697800 2024-05-09T00:49:19.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.225+0000 7f10ca81ab80 1 -- 172.21.15.5:0/3373133283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebb0697800 msgr2=0x55ebb06a13b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:19.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.226+0000 7f10ca81ab80 1 --2- 172.21.15.5:0/3373133283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebb0697800 0x55ebb06a13b0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f10b410cf10 tx=0x7f10b4064c50 comp rx=0 tx=0).stop 2024-05-09T00:49:19.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.226+0000 7f10ca81ab80 1 -- 172.21.15.5:0/3373133283 shutdown_connections 2024-05-09T00:49:19.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.226+0000 7f10ca81ab80 1 --2- 172.21.15.5:0/3373133283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebb0697800 0x55ebb06a13b0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:19.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.226+0000 7f10ca81ab80 1 --2- 172.21.15.5:0/3373133283 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ebb051a4a0 0x55ebb069cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:19.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.226+0000 7f10ca81ab80 1 --2- 172.21.15.5:0/3373133283 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ebb0604160 0x55ebb06ad1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:19.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.226+0000 7f10ca81ab80 1 -- 172.21.15.5:0/3373133283 >> 172.21.15.5:0/3373133283 conn(0x55ebb057a6a0 msgr2=0x55ebb069c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:49:19.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.226+0000 7f10ca81ab80 1 -- 172.21.15.5:0/3373133283 shutdown_connections 2024-05-09T00:49:19.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.226+0000 7f10ca81ab80 1 -- 172.21.15.5:0/3373133283 wait complete. 2024-05-09T00:49:19.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.227+0000 7f10ca81ab80 1 -- start start 2024-05-09T00:49:19.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.228+0000 7f10ca81ab80 1 Processor -- start 2024-05-09T00:49:19.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.229+0000 7f10ca81ab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ebb051a4a0 0x55ebb06b91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:19.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.229+0000 7f10ca81ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebb0604160 0x55ebb06a9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:19.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.229+0000 7f10ca81ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ebb0697800 0x55ebb06a9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:19.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.229+0000 7f10ca81ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f109800e960 con 0x55ebb0604160 2024-05-09T00:49:19.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.229+0000 7f10ca81ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f109800e290 con 0x55ebb0697800 2024-05-09T00:49:19.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.229+0000 7f10ca81ab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f109800b490 con 0x55ebb051a4a0 2024-05-09T00:49:19.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.229+0000 7f10ca81ab80 10 client.0 _ll_register_callbacks cb 0x55ebb0692200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:49:19.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.230+0000 7f10b939f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebb0604160 0x55ebb06a9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:19.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.230+0000 7f10b939f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebb0604160 0x55ebb06a9520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55578/0 (socket says 172.21.15.5:55578) 2024-05-09T00:49:19.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.230+0000 7f10b939f700 1 -- 172.21.15.5:0/1806440125 learned_addr learned my addr 172.21.15.5:0/1806440125 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:49:19.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.231+0000 7f10b939f700 1 -- 172.21.15.5:0/1806440125 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ebb051a4a0 msgr2=0x55ebb06b91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:49:19.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.231+0000 7f10b939f700 1 --2- 172.21.15.5:0/1806440125 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ebb051a4a0 0x55ebb06b91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:19.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.231+0000 7f10b939f700 1 -- 172.21.15.5:0/1806440125 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ebb0697800 msgr2=0x55ebb06a9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:49:19.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.231+0000 7f10b939f700 1 --2- 172.21.15.5:0/1806440125 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ebb0697800 0x55ebb06a9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:19.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.231+0000 7f10b939f700 1 -- 172.21.15.5:0/1806440125 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebb06b7bf0 con 0x55ebb0604160 2024-05-09T00:49:19.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.231+0000 7f10b939f700 1 --2- 172.21.15.5:0/1806440125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebb0604160 0x55ebb06a9520 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f10b0001eb0 tx=0x7f10b00058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcd26317fd895fef server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:49:19.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.231+0000 7f109d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:49:19.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.232+0000 7f109d7fa700 1 -- 172.21.15.5:0/1806440125 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b0005030 con 0x55ebb0604160 2024-05-09T00:49:19.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.232+0000 7f10ca81ab80 1 -- 172.21.15.5:0/1806440125 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ebb05c79d0 con 0x55ebb0604160 2024-05-09T00:49:19.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.232+0000 7f109d7fa700 1 -- 172.21.15.5:0/1806440125 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f10b00051d0 con 0x55ebb0604160 2024-05-09T00:49:19.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.233+0000 7f109d7fa700 1 -- 172.21.15.5:0/1806440125 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f10b00053d0 con 0x55ebb0604160 2024-05-09T00:49:19.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.233+0000 7f10ca81ab80 10 client.15759 fetch_fsmap learned FSMap version 29 2024-05-09T00:49:19.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.233+0000 7f10ca81ab80 1 -- 172.21.15.5:0/1806440125 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ebb071aa20 con 0x55ebb0604160 2024-05-09T00:49:19.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.234+0000 7f109d7fa700 1 -- 172.21.15.5:0/1806440125 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55ebb05c79d0 con 0x55ebb0604160 2024-05-09T00:49:19.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.234+0000 7f109d7fa700 1 -- 172.21.15.5:0/1806440125 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f10b0004990 con 0x55ebb0604160 2024-05-09T00:49:19.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.234+0000 7f10ca81ab80 10 client.15759 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:49:19.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.234+0000 7f10ca81ab80 10 client.15759 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T00:49:19.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.234+0000 7f10ca81ab80 20 client.15759 populate_metadata read hostname 'smithi005' 2024-05-09T00:49:19.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.234+0000 7f10ca81ab80 1 --2- 172.21.15.5:0/1806440125 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x55ebb071af10 0x55ebb071d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:19.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.234+0000 7f10ca81ab80 4 client.15759 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:49:19.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.234+0000 7f10ca81ab80 1 -- 172.21.15.5:0/1806440125 --> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ebb05c1a70 con 0x55ebb071af10 2024-05-09T00:49:19.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.238+0000 7f10b9ba0700 1 --2- 172.21.15.5:0/1806440125 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x55ebb071af10 0x55ebb071d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:19.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.244+0000 7f10b9ba0700 1 --2- 172.21.15.5:0/1806440125 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x55ebb071af10 0x55ebb071d330 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=74f2b0fb7eb99b67 server_cookie=375cb98f73c2ce11 in_seq=0 out_seq=0 2024-05-09T00:49:19.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.244+0000 7f109d7fa700 10 client.15759 ms_handle_connect on v2:172.21.15.157:6834/832898273 2024-05-09T00:49:19.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.245+0000 7f109d7fa700 1 -- 172.21.15.5:0/1806440125 <== mds.3 v2:172.21.15.157:6834/832898273 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ebb05c1a70 con 0x55ebb071af10 2024-05-09T00:49:19.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.245+0000 7f109d7fa700 10 client.15759 handle_command_reply: tid=0 2024-05-09T00:49:19.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.245+0000 7f10ca81ab80 1 client.15759 shutdown 2024-05-09T00:49:19.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.246+0000 7f10ca81ab80 1 -- 172.21.15.5:0/1806440125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebb0604160 msgr2=0x55ebb06a9520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:19.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.246+0000 7f10ca81ab80 1 --2- 172.21.15.5:0/1806440125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebb0604160 0x55ebb06a9520 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f10b0001eb0 tx=0x7f10b00058e0 comp rx=0 tx=0).stop 2024-05-09T00:49:19.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.246+0000 7f10ca81ab80 1 -- 172.21.15.5:0/1806440125 shutdown_connections 2024-05-09T00:49:19.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.246+0000 7f10ca81ab80 1 --2- 172.21.15.5:0/1806440125 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x55ebb071af10 0x55ebb071d330 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:19.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.246+0000 7f10ca81ab80 1 --2- 172.21.15.5:0/1806440125 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ebb0697800 0x55ebb06a9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:19.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.246+0000 7f10ca81ab80 1 --2- 172.21.15.5:0/1806440125 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ebb051a4a0 0x55ebb06b91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:19.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.246+0000 7f10ca81ab80 1 --2- 172.21.15.5:0/1806440125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebb0604160 0x55ebb06a9520 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:19.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.246+0000 7f10ca81ab80 1 -- 172.21.15.5:0/1806440125 >> 172.21.15.5:0/1806440125 conn(0x55ebb057a6a0 msgr2=0x55ebb069b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:49:19.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.246+0000 7f109d7fa700 0 client.15759 ms_handle_reset on v2:172.21.15.157:6834/832898273 2024-05-09T00:49:19.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.247+0000 7f10ca81ab80 1 -- 172.21.15.5:0/1806440125 shutdown_connections 2024-05-09T00:49:19.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.247+0000 7f10ca81ab80 1 -- 172.21.15.5:0/1806440125 wait complete. 2024-05-09T00:49:19.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.247+0000 7f10ca81ab80 20 client.15759 trim_cache size 0 max 16384 2024-05-09T00:49:19.250 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-09T00:49:19.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.248+0000 7f10bbe04700 1 -- 172.21.15.5:0/1843332407 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f10a00b2900 msgr2=0x7f10a00b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:19.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.248+0000 7f10bbe04700 1 --2- 172.21.15.5:0/1843332407 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f10a00b2900 0x7f10a00b4e10 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f10a4009680 tx=0x7f10a4009260 comp rx=0 tx=0).stop 2024-05-09T00:49:19.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.248+0000 7f10bbe04700 1 -- 172.21.15.5:0/1843332407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10ac0b8330 msgr2=0x7f10ac148e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:19.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.248+0000 7f10bbe04700 1 --2- 172.21.15.5:0/1843332407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10ac0b8330 0x7f10ac148e90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f10b4069820 tx=0x7f10b406c340 comp rx=0 tx=0).stop 2024-05-09T00:49:19.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.249+0000 7f10bbe04700 1 -- 172.21.15.5:0/1843332407 shutdown_connections 2024-05-09T00:49:19.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.249+0000 7f10bbe04700 1 --2- 172.21.15.5:0/1843332407 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f10a00b2900 0x7f10a00b4e10 secure :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f10a4009680 tx=0x7f10a4009260 comp rx=0 tx=0).stop 2024-05-09T00:49:19.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.249+0000 7f10bbe04700 1 --2- 172.21.15.5:0/1843332407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10ac0b8330 0x7f10ac148e90 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:19.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.249+0000 7f10bbe04700 1 --2- 172.21.15.5:0/1843332407 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f10ac0a53d0 0x7f10ac144660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:19.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.249+0000 7f10bbe04700 1 --2- 172.21.15.5:0/1843332407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10ac0a5d20 0x7f10ac144c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:19.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.249+0000 7f10bbe04700 1 -- 172.21.15.5:0/1843332407 >> 172.21.15.5:0/1843332407 conn(0x7f10ac01af30 msgr2=0x7f10ac0b5de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:49:19.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.250+0000 7f10bbe04700 1 -- 172.21.15.5:0/1843332407 shutdown_connections 2024-05-09T00:49:19.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:19.250+0000 7f10bbe04700 1 -- 172.21.15.5:0/1843332407 wait complete. 2024-05-09T00:49:19.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:49:19 smithi131 ceph-mon[34547]: pgmap v460: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 809 MiB/s wr, 2.23k op/s 2024-05-09T00:49:19.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:49:19 smithi157 ceph-mon[34294]: pgmap v460: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 809 MiB/s wr, 2.23k op/s 2024-05-09T00:49:21.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:49:21 smithi131 ceph-mon[34547]: pgmap v461: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 972 MiB/s wr, 2.74k op/s 2024-05-09T00:49:21.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:49:21 smithi157 ceph-mon[34294]: pgmap v461: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 972 MiB/s wr, 2.74k op/s 2024-05-09T00:49:21.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:49:21 smithi005 ceph-mon[30365]: pgmap v461: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 972 MiB/s wr, 2.74k op/s 2024-05-09T00:49:22.642 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T00:49:23.292 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:49:23.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:49:23 smithi131 ceph-mon[34547]: pgmap v462: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 838 MiB/s wr, 2.35k op/s 2024-05-09T00:49:23.431 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:49:23 smithi005 ceph-mon[30365]: pgmap v462: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 838 MiB/s wr, 2.35k op/s 2024-05-09T00:49:23.435 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:49:23 smithi157 ceph-mon[34294]: pgmap v462: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 838 MiB/s wr, 2.35k op/s 2024-05-09T00:49:23.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.696+0000 7fe47723d640 1 -- 172.21.15.5:0/3822811603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4701102f0 msgr2=0x7fe470112710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:23.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.696+0000 7fe47723d640 1 --2- 172.21.15.5:0/3822811603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4701102f0 0x7fe470112710 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fe46c009900 tx=0x7fe46c02a440 comp rx=0 tx=0).stop 2024-05-09T00:49:23.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.700+0000 7fe47723d640 1 -- 172.21.15.5:0/3822811603 shutdown_connections 2024-05-09T00:49:23.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.700+0000 7fe47723d640 1 --2- 172.21.15.5:0/3822811603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4701102f0 0x7fe470112710 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:23.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.700+0000 7fe47723d640 1 --2- 172.21.15.5:0/3822811603 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe470076b20 0x7fe47010fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:23.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.700+0000 7fe47723d640 1 --2- 172.21.15.5:0/3822811603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4700761d0 0x7fe4700765e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:23.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.700+0000 7fe47723d640 1 -- 172.21.15.5:0/3822811603 >> 172.21.15.5:0/3822811603 conn(0x7fe4700fed80 msgr2=0x7fe470101230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:49:23.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.702+0000 7fe47723d640 1 -- 172.21.15.5:0/3822811603 shutdown_connections 2024-05-09T00:49:23.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.703+0000 7fe47723d640 1 -- 172.21.15.5:0/3822811603 wait complete. 2024-05-09T00:49:23.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.705+0000 7fe47723d640 1 Processor -- start 2024-05-09T00:49:23.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.705+0000 7fe47723d640 1 -- start start 2024-05-09T00:49:23.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.706+0000 7fe47723d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4700761d0 0x7fe4701a0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:23.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.707+0000 7fe47723d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe470076b20 0x7fe4701a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:23.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.707+0000 7fe47723d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4701102f0 0x7fe4701a7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:23.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.707+0000 7fe47723d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe470115bf0 con 0x7fe4701102f0 2024-05-09T00:49:23.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.707+0000 7fe475a3a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe470076b20 0x7fe4701a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:23.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.708+0000 7fe475a3a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe470076b20 0x7fe4701a0c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:54302/0 (socket says 172.21.15.5:54302) 2024-05-09T00:49:23.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.708+0000 7fe475a3a640 1 -- 172.21.15.5:0/1426074982 learned_addr learned my addr 172.21.15.5:0/1426074982 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:49:23.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.708+0000 7fe47723d640 1 -- 172.21.15.5:0/1426074982 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe470115a70 con 0x7fe470076b20 2024-05-09T00:49:23.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.708+0000 7fe47723d640 1 -- 172.21.15.5:0/1426074982 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe470115d70 con 0x7fe4700761d0 2024-05-09T00:49:23.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.708+0000 7fe476a3c640 1 --2- 172.21.15.5:0/1426074982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4701102f0 0x7fe4701a7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:23.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.709+0000 7fe475a3a640 1 -- 172.21.15.5:0/1426074982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4700761d0 msgr2=0x7fe4701a0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:23.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.709+0000 7fe475a3a640 1 --2- 172.21.15.5:0/1426074982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4700761d0 0x7fe4701a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:23.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.709+0000 7fe475a3a640 1 -- 172.21.15.5:0/1426074982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4701102f0 msgr2=0x7fe4701a7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:23.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.709+0000 7fe475a3a640 1 --2- 172.21.15.5:0/1426074982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4701102f0 0x7fe4701a7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:23.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.709+0000 7fe475a3a640 1 -- 172.21.15.5:0/1426074982 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4701a8260 con 0x7fe470076b20 2024-05-09T00:49:23.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.710+0000 7fe475a3a640 1 --2- 172.21.15.5:0/1426074982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe470076b20 0x7fe4701a0c60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe46800eab0 tx=0x7fe46800ef70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:49:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.714+0000 7fe4677fe640 1 -- 172.21.15.5:0/1426074982 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe46800ce40 con 0x7fe470076b20 2024-05-09T00:49:23.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.715+0000 7fe47723d640 1 -- 172.21.15.5:0/1426074982 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4701a8550 con 0x7fe470076b20 2024-05-09T00:49:23.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.715+0000 7fe4677fe640 1 -- 172.21.15.5:0/1426074982 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe468011920 con 0x7fe470076b20 2024-05-09T00:49:23.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.715+0000 7fe47723d640 1 -- 172.21.15.5:0/1426074982 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4701a8aa0 con 0x7fe470076b20 2024-05-09T00:49:23.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.716+0000 7fe4677fe640 1 -- 172.21.15.5:0/1426074982 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe46801a990 con 0x7fe470076b20 2024-05-09T00:49:23.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.722+0000 7fe47f42e740 1 Processor -- start 2024-05-09T00:49:23.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.723+0000 7fe47f42e740 1 -- start start 2024-05-09T00:49:23.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.723+0000 7fe47f42e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2c0acd1b0 0x55b2c0e716c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:23.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.723+0000 7fe47f42e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2c0aa42f0 0x55b2c0e71c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:23.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.723+0000 7fe47f42e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2c0aac4d0 0x55b2c0e78b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:23.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.723+0000 7fe47f42e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b2c0dd57c0 con 0x55b2c0aa42f0 2024-05-09T00:49:23.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.723+0000 7fe47f42e740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55b2c0d710e0 con 0x55b2c0acd1b0 2024-05-09T00:49:23.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.723+0000 7fe47f42e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b2c0dd1780 con 0x55b2c0aac4d0 2024-05-09T00:49:23.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.724+0000 7fe476a3c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2c0acd1b0 0x55b2c0e716c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:23.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.725+0000 7fe47623b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2c0aa42f0 0x55b2c0e71c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:23.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.725+0000 7fe476a3c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2c0acd1b0 0x55b2c0e716c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:54312/0 (socket says 172.21.15.5:54312) 2024-05-09T00:49:23.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.725+0000 7fe476a3c640 1 -- 172.21.15.5:0/158144934 learned_addr learned my addr 172.21.15.5:0/158144934 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:49:23.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.725+0000 7fe4677fe640 1 -- 172.21.15.5:0/1426074982 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fe46801ac50 con 0x7fe470076b20 2024-05-09T00:49:23.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.726+0000 7fe4677fe640 1 --2- 172.21.15.5:0/1426074982 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe45c0b29f0 0x7fe45c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:23.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.726+0000 7fe4677fe640 1 -- 172.21.15.5:0/1426074982 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fe4680e2a70 con 0x7fe470076b20 2024-05-09T00:49:23.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.728+0000 7fe47623b640 1 -- 172.21.15.5:0/158144934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2c0aac4d0 msgr2=0x55b2c0e78b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:49:23.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.728+0000 7fe47623b640 1 --2- 172.21.15.5:0/158144934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2c0aac4d0 0x55b2c0e78b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:23.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.728+0000 7fe47623b640 1 -- 172.21.15.5:0/158144934 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2c0acd1b0 msgr2=0x55b2c0e716c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:49:23.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.728+0000 7fe47623b640 1 --2- 172.21.15.5:0/158144934 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2c0acd1b0 0x55b2c0e716c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:23.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.728+0000 7fe47623b640 1 -- 172.21.15.5:0/158144934 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2c0d9a7c0 con 0x55b2c0aa42f0 2024-05-09T00:49:23.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.728+0000 7fe476a3c640 1 --2- 172.21.15.5:0/158144934 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2c0acd1b0 0x55b2c0e716c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:49:23.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.728+0000 7fe476a3c640 1 --2- 172.21.15.5:0/1426074982 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe45c0b29f0 0x7fe45c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:23.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.729+0000 7fe476a3c640 1 --2- 172.21.15.5:0/1426074982 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe45c0b29f0 0x7fe45c0b4f00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe46c005f70 tx=0x7fe46c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:49:23.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.729+0000 7fe47623b640 1 --2- 172.21.15.5:0/158144934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2c0aa42f0 0x55b2c0e71c40 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe46000a900 tx=0x7fe46000adc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d47661d5fe5f2e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:49:23.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.730+0000 7fe464bfe640 1 -- 172.21.15.5:0/158144934 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe46000cd80 con 0x55b2c0aa42f0 2024-05-09T00:49:23.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.731+0000 7fe464bfe640 1 -- 172.21.15.5:0/158144934 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe46000f900 con 0x55b2c0aa42f0 2024-05-09T00:49:23.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.731+0000 7fe464bfe640 1 -- 172.21.15.5:0/158144934 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe460018830 con 0x55b2c0aa42f0 2024-05-09T00:49:23.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.732+0000 7fe47f42e740 1 -- 172.21.15.5:0/158144934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2c0aa42f0 msgr2=0x55b2c0e71c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:23.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.732+0000 7fe47f42e740 1 --2- 172.21.15.5:0/158144934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2c0aa42f0 0x55b2c0e71c40 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe46000a900 tx=0x7fe46000adc0 comp rx=0 tx=0).stop 2024-05-09T00:49:23.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.733+0000 7fe47f42e740 1 -- 172.21.15.5:0/158144934 shutdown_connections 2024-05-09T00:49:23.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.733+0000 7fe47f42e740 1 --2- 172.21.15.5:0/158144934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2c0aac4d0 0x55b2c0e78b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:23.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.733+0000 7fe47f42e740 1 --2- 172.21.15.5:0/158144934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2c0aa42f0 0x55b2c0e71c40 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:23.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.733+0000 7fe47f42e740 1 --2- 172.21.15.5:0/158144934 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2c0acd1b0 0x55b2c0e716c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:23.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.733+0000 7fe47f42e740 1 -- 172.21.15.5:0/158144934 >> 172.21.15.5:0/158144934 conn(0x55b2c0ac1300 msgr2=0x55b2c0e6d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:49:23.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.733+0000 7fe47f42e740 1 -- 172.21.15.5:0/158144934 shutdown_connections 2024-05-09T00:49:23.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.733+0000 7fe47f42e740 1 -- 172.21.15.5:0/158144934 wait complete. 2024-05-09T00:49:23.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.734+0000 7fe47f42e740 1 -- start start 2024-05-09T00:49:23.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.734+0000 7fe47f42e740 1 Processor -- start 2024-05-09T00:49:23.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.735+0000 7fe47f42e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2c0aa42f0 0x55b2c0e8c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:23.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.735+0000 7fe47f42e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2c0aac4d0 0x55b2c0e8ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:23.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.735+0000 7fe47f42e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2c0acd1b0 0x55b2c0e939d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:23.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.735+0000 7fe47f42e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b2c0e8a640 con 0x55b2c0aac4d0 2024-05-09T00:49:23.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.735+0000 7fe47f42e740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55b2c0d5c9c0 con 0x55b2c0acd1b0 2024-05-09T00:49:23.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.735+0000 7fe47f42e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b2c0e8a7c0 con 0x55b2c0aa42f0 2024-05-09T00:49:23.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.735+0000 7fe47f42e740 10 client.0 _ll_register_callbacks cb 0x55b2c0d89020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:49:23.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.735+0000 7fe475a3a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2c0aac4d0 0x55b2c0e8ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:23.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.735+0000 7fe475a3a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2c0aac4d0 0x55b2c0e8ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41426/0 (socket says 172.21.15.5:41426) 2024-05-09T00:49:23.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.735+0000 7fe475a3a640 1 -- 172.21.15.5:0/494152653 learned_addr learned my addr 172.21.15.5:0/494152653 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:49:23.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.736+0000 7fe475a3a640 1 -- 172.21.15.5:0/494152653 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2c0aa42f0 msgr2=0x55b2c0e8c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:49:23.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.736+0000 7fe475a3a640 1 --2- 172.21.15.5:0/494152653 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2c0aa42f0 0x55b2c0e8c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:23.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.736+0000 7fe475a3a640 1 -- 172.21.15.5:0/494152653 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2c0acd1b0 msgr2=0x55b2c0e939d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:49:23.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.736+0000 7fe475a3a640 1 --2- 172.21.15.5:0/494152653 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2c0acd1b0 0x55b2c0e939d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:23.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.736+0000 7fe475a3a640 1 -- 172.21.15.5:0/494152653 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2c0d831f0 con 0x55b2c0aac4d0 2024-05-09T00:49:23.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.736+0000 7fe475a3a640 1 --2- 172.21.15.5:0/494152653 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2c0aac4d0 0x55b2c0e8ca70 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fe46800c940 tx=0x7fe4680aa550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=867428674e42b83b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:49:23.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.736+0000 7fe459ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:49:23.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.736+0000 7fe459ffb640 1 -- 172.21.15.5:0/494152653 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4680c3030 con 0x55b2c0aac4d0 2024-05-09T00:49:23.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.736+0000 7fe459ffb640 1 -- 172.21.15.5:0/494152653 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fe4680a65f0 con 0x55b2c0aac4d0 2024-05-09T00:49:23.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.737+0000 7fe47f42e740 1 -- 172.21.15.5:0/494152653 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe46000f900 con 0x55b2c0aac4d0 2024-05-09T00:49:23.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.737+0000 7fe459ffb640 1 -- 172.21.15.5:0/494152653 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe4680ca030 con 0x55b2c0aac4d0 2024-05-09T00:49:23.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.737+0000 7fe47f42e740 10 client.15771 fetch_fsmap learned FSMap version 29 2024-05-09T00:49:23.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.737+0000 7fe47f42e740 1 -- 172.21.15.5:0/494152653 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b2c0b38790 con 0x55b2c0aac4d0 2024-05-09T00:49:23.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.737+0000 7fe459ffb640 1 -- 172.21.15.5:0/494152653 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe46000f900 con 0x55b2c0aac4d0 2024-05-09T00:49:23.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.737+0000 7fe459ffb640 1 -- 172.21.15.5:0/494152653 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fe468022070 con 0x55b2c0aac4d0 2024-05-09T00:49:23.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.737+0000 7fe47f42e740 10 client.15771 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:49:23.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.737+0000 7fe47f42e740 10 client.15771 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:49:23.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.737+0000 7fe47f42e740 20 client.15771 populate_metadata read hostname 'smithi005' 2024-05-09T00:49:23.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.738+0000 7fe47f42e740 1 --2- 172.21.15.5:0/494152653 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55b2c0a8bd80 0x55b2c0e99f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:23.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.738+0000 7fe47f42e740 4 client.15771 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:49:23.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.738+0000 7fe47f42e740 1 -- 172.21.15.5:0/494152653 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b2c0de33d0 con 0x55b2c0a8bd80 2024-05-09T00:49:23.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.738+0000 7fe47623b640 1 --2- 172.21.15.5:0/494152653 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55b2c0a8bd80 0x55b2c0e99f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:23.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.740+0000 7fe47623b640 1 --2- 172.21.15.5:0/494152653 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55b2c0a8bd80 0x55b2c0e99f40 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=450babb7ee40189c server_cookie=7d30020840ecb89d in_seq=0 out_seq=0 2024-05-09T00:49:23.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.740+0000 7fe459ffb640 10 client.15771 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:49:23.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.744+0000 7fe459ffb640 1 -- 172.21.15.5:0/494152653 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b2c0de33d0 con 0x55b2c0a8bd80 2024-05-09T00:49:23.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.744+0000 7fe459ffb640 10 client.15771 handle_command_reply: tid=0 2024-05-09T00:49:23.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.745+0000 7fe47f42e740 1 client.15771 shutdown 2024-05-09T00:49:23.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.745+0000 7fe47f42e740 1 -- 172.21.15.5:0/494152653 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2c0aac4d0 msgr2=0x55b2c0e8ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:23.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.745+0000 7fe47f42e740 1 --2- 172.21.15.5:0/494152653 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2c0aac4d0 0x55b2c0e8ca70 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fe46800c940 tx=0x7fe4680aa550 comp rx=0 tx=0).stop 2024-05-09T00:49:23.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.745+0000 7fe47f42e740 1 -- 172.21.15.5:0/494152653 shutdown_connections 2024-05-09T00:49:23.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.745+0000 7fe47f42e740 1 --2- 172.21.15.5:0/494152653 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55b2c0a8bd80 0x55b2c0e99f40 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:23.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.745+0000 7fe47f42e740 1 --2- 172.21.15.5:0/494152653 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2c0acd1b0 0x55b2c0e939d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:23.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.745+0000 7fe47f42e740 1 --2- 172.21.15.5:0/494152653 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2c0aac4d0 0x55b2c0e8ca70 secure :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fe46800c940 tx=0x7fe4680aa550 comp rx=0 tx=0).stop 2024-05-09T00:49:23.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.745+0000 7fe47f42e740 1 --2- 172.21.15.5:0/494152653 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2c0aa42f0 0x55b2c0e8c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:23.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.745+0000 7fe47f42e740 1 -- 172.21.15.5:0/494152653 >> 172.21.15.5:0/494152653 conn(0x55b2c0ac1300 msgr2=0x55b2c0e6b860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:49:23.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.745+0000 7fe459ffb640 0 client.15771 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:49:23.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.745+0000 7fe47f42e740 1 -- 172.21.15.5:0/494152653 shutdown_connections 2024-05-09T00:49:23.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.745+0000 7fe47f42e740 1 -- 172.21.15.5:0/494152653 wait complete. 2024-05-09T00:49:23.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.746+0000 7fe47f42e740 20 client.15771 trim_cache size 0 max 16384 2024-05-09T00:49:23.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.782+0000 7fe47f42e740 1 Processor -- start 2024-05-09T00:49:23.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.782+0000 7fe47f42e740 1 -- start start 2024-05-09T00:49:23.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.782+0000 7fe47f42e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2c0aa42f0 0x55b2c0e8abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.782+0000 7fe47f42e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2c0acd1b0 0x55b2c0e8b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.783+0000 7fe47f42e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2c0a611c0 0x55b2c0e8b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.783+0000 7fe47f42e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe45400e590 con 0x55b2c0a611c0 2024-05-09T00:49:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.783+0000 7fe476a3c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2c0a611c0 0x55b2c0e8b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.783+0000 7fe476a3c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2c0a611c0 0x55b2c0e8b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41432/0 (socket says 172.21.15.5:41432) 2024-05-09T00:49:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.783+0000 7fe476a3c640 1 -- 172.21.15.5:0/10997148 learned_addr learned my addr 172.21.15.5:0/10997148 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:49:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.783+0000 7fe47f42e740 1 -- 172.21.15.5:0/10997148 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe4540100d0 con 0x55b2c0acd1b0 2024-05-09T00:49:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.783+0000 7fe47f42e740 1 -- 172.21.15.5:0/10997148 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe45400ec60 con 0x55b2c0aa42f0 2024-05-09T00:49:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.784+0000 7fe476a3c640 1 -- 172.21.15.5:0/10997148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2c0aa42f0 msgr2=0x55b2c0e8abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:49:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.784+0000 7fe476a3c640 1 --2- 172.21.15.5:0/10997148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2c0aa42f0 0x55b2c0e8abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.784+0000 7fe475a3a640 1 --2- 172.21.15.5:0/10997148 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2c0acd1b0 0x55b2c0e8b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.784+0000 7fe476a3c640 1 -- 172.21.15.5:0/10997148 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2c0acd1b0 msgr2=0x55b2c0e8b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:49:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.784+0000 7fe476a3c640 1 --2- 172.21.15.5:0/10997148 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2c0acd1b0 0x55b2c0e8b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.784+0000 7fe476a3c640 1 -- 172.21.15.5:0/10997148 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2c0baf5f0 con 0x55b2c0a611c0 2024-05-09T00:49:23.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.784+0000 7fe476a3c640 1 --2- 172.21.15.5:0/10997148 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2c0a611c0 0x55b2c0e8b670 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fe46c033040 tx=0x7fe46c040070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=793cce40292064c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:49:23.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.784+0000 7fe45bfff640 1 -- 172.21.15.5:0/10997148 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe46c04e070 con 0x55b2c0a611c0 2024-05-09T00:49:23.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.785+0000 7fe45bfff640 1 -- 172.21.15.5:0/10997148 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe46c042070 con 0x55b2c0a611c0 2024-05-09T00:49:23.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.785+0000 7fe45bfff640 1 -- 172.21.15.5:0/10997148 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe46c0316a0 con 0x55b2c0a611c0 2024-05-09T00:49:23.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.786+0000 7fe47f42e740 1 -- 172.21.15.5:0/10997148 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2c0a611c0 msgr2=0x55b2c0e8b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:23.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.786+0000 7fe47f42e740 1 --2- 172.21.15.5:0/10997148 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2c0a611c0 0x55b2c0e8b670 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fe46c033040 tx=0x7fe46c040070 comp rx=0 tx=0).stop 2024-05-09T00:49:23.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.786+0000 7fe47f42e740 1 -- 172.21.15.5:0/10997148 shutdown_connections 2024-05-09T00:49:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.786+0000 7fe47f42e740 1 --2- 172.21.15.5:0/10997148 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2c0a611c0 0x55b2c0e8b670 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.786+0000 7fe47f42e740 1 --2- 172.21.15.5:0/10997148 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2c0acd1b0 0x55b2c0e8b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.786+0000 7fe47f42e740 1 --2- 172.21.15.5:0/10997148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2c0aa42f0 0x55b2c0e8abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.786+0000 7fe47f42e740 1 -- 172.21.15.5:0/10997148 >> 172.21.15.5:0/10997148 conn(0x55b2c0ac1300 msgr2=0x55b2c0e75840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:49:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.787+0000 7fe47f42e740 1 -- 172.21.15.5:0/10997148 shutdown_connections 2024-05-09T00:49:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.787+0000 7fe47f42e740 1 -- 172.21.15.5:0/10997148 wait complete. 2024-05-09T00:49:23.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.787+0000 7fe47f42e740 1 -- start start 2024-05-09T00:49:23.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.788+0000 7fe47f42e740 1 Processor -- start 2024-05-09T00:49:23.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.788+0000 7fe47f42e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2c0a611c0 0x55b2c0e900f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.788+0000 7fe47f42e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2c0aa42f0 0x55b2c0e90630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.788+0000 7fe47f42e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2c0acd1b0 0x55b2c0f002a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.789+0000 7fe47f42e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe45400fa00 con 0x55b2c0aa42f0 2024-05-09T00:49:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.789+0000 7fe47f42e740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe45400f330 con 0x55b2c0acd1b0 2024-05-09T00:49:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.789+0000 7fe47f42e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe45400c570 con 0x55b2c0a611c0 2024-05-09T00:49:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.789+0000 7fe475a3a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2c0aa42f0 0x55b2c0e90630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.789+0000 7fe475a3a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2c0aa42f0 0x55b2c0e90630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41444/0 (socket says 172.21.15.5:41444) 2024-05-09T00:49:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.789+0000 7fe475a3a640 1 -- 172.21.15.5:0/1626329240 learned_addr learned my addr 172.21.15.5:0/1626329240 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:49:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.789+0000 7fe47f42e740 10 client.0 _ll_register_callbacks cb 0x7fe4680a9dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:49:23.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.789+0000 7fe475a3a640 1 -- 172.21.15.5:0/1626329240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2c0a611c0 msgr2=0x55b2c0e900f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:49:23.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.789+0000 7fe475a3a640 1 --2- 172.21.15.5:0/1626329240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2c0a611c0 0x55b2c0e900f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:23.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.790+0000 7fe475a3a640 1 -- 172.21.15.5:0/1626329240 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2c0acd1b0 msgr2=0x55b2c0f002a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:49:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.790+0000 7fe475a3a640 1 --2- 172.21.15.5:0/1626329240 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2c0acd1b0 0x55b2c0f002a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.790+0000 7fe475a3a640 1 -- 172.21.15.5:0/1626329240 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2c0d99ed0 con 0x55b2c0aa42f0 2024-05-09T00:49:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.790+0000 7fe475a3a640 1 --2- 172.21.15.5:0/1626329240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2c0aa42f0 0x55b2c0e90630 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fe4680a9d00 tx=0x7fe4680e2cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab19e04833426982 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:49:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.790+0000 7fe459ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:49:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.790+0000 7fe459ffb640 1 -- 172.21.15.5:0/1626329240 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4680c3070 con 0x55b2c0aa42f0 2024-05-09T00:49:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.790+0000 7fe459ffb640 1 -- 172.21.15.5:0/1626329240 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fe4680a65f0 con 0x55b2c0aa42f0 2024-05-09T00:49:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.790+0000 7fe47f42e740 1 -- 172.21.15.5:0/1626329240 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe46c042070 con 0x55b2c0aa42f0 2024-05-09T00:49:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.791+0000 7fe459ffb640 1 -- 172.21.15.5:0/1626329240 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe4680a9950 con 0x55b2c0aa42f0 2024-05-09T00:49:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.791+0000 7fe47f42e740 10 client.15777 fetch_fsmap learned FSMap version 29 2024-05-09T00:49:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.791+0000 7fe47f42e740 1 -- 172.21.15.5:0/1626329240 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b2c0f01850 con 0x55b2c0aa42f0 2024-05-09T00:49:23.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.791+0000 7fe459ffb640 1 -- 172.21.15.5:0/1626329240 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4680aa700 con 0x55b2c0aa42f0 2024-05-09T00:49:23.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.792+0000 7fe459ffb640 1 -- 172.21.15.5:0/1626329240 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fe468022070 con 0x55b2c0aa42f0 2024-05-09T00:49:23.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.792+0000 7fe47f42e740 10 client.15777 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:49:23.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.793+0000 7fe47f42e740 10 client.15777 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:49:23.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.793+0000 7fe47f42e740 20 client.15777 populate_metadata read hostname 'smithi005' 2024-05-09T00:49:23.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.793+0000 7fe47f42e740 1 --2- 172.21.15.5:0/1626329240 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55b2c0f02200 0x55b2c0f04650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:23.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.793+0000 7fe47f42e740 4 client.15777 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:49:23.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.793+0000 7fe47f42e740 1 -- 172.21.15.5:0/1626329240 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fe46c042070 con 0x55b2c0f02200 2024-05-09T00:49:23.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.795+0000 7fe47623b640 1 --2- 172.21.15.5:0/1626329240 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55b2c0f02200 0x55b2c0f04650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:23.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.800+0000 7fe459ffb640 10 client.15777 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:49:23.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:23.800+0000 7fe47623b640 1 --2- 172.21.15.5:0/1626329240 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55b2c0f02200 0x55b2c0f04650 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94675d5d4aaa394c server_cookie=d385c485183e46e7 in_seq=0 out_seq=0 2024-05-09T00:49:25.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:49:25 smithi131 ceph-mon[34547]: pgmap v463: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 885 MiB/s wr, 2.73k op/s 2024-05-09T00:49:25.378 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:49:25 smithi005 ceph-mon[30365]: pgmap v463: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 885 MiB/s wr, 2.73k op/s 2024-05-09T00:49:25.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:49:25 smithi157 ceph-mon[34294]: pgmap v463: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 885 MiB/s wr, 2.73k op/s 2024-05-09T00:49:27.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:49:27 smithi131 ceph-mon[34547]: pgmap v464: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 975 MiB/s wr, 3.29k op/s 2024-05-09T00:49:27.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:49:27 smithi157 ceph-mon[34294]: pgmap v464: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 975 MiB/s wr, 3.29k op/s 2024-05-09T00:49:27.478 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:49:27 smithi005 ceph-mon[30365]: pgmap v464: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 975 MiB/s wr, 3.29k op/s 2024-05-09T00:49:27.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.784+0000 7f2fbfe1f700 1 -- 172.21.15.5:0/674510258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fb8108740 msgr2=0x7f2fb806b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:27.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.784+0000 7f2fbfe1f700 1 --2- 172.21.15.5:0/674510258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fb8108740 0x7f2fb806b610 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f2fb4009990 tx=0x7f2fb4009ca0 comp rx=0 tx=0).stop 2024-05-09T00:49:27.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.785+0000 7f2fbfe1f700 1 -- 172.21.15.5:0/674510258 shutdown_connections 2024-05-09T00:49:27.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.785+0000 7f2fbfe1f700 1 --2- 172.21.15.5:0/674510258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fb8108740 0x7f2fb806b610 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:27.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.785+0000 7f2fbfe1f700 1 --2- 172.21.15.5:0/674510258 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fb81034c0 0x7f2fb81038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:27.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.785+0000 7f2fbfe1f700 1 --2- 172.21.15.5:0/674510258 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2fb8107df0 0x7f2fb8108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:27.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.785+0000 7f2fbfe1f700 1 -- 172.21.15.5:0/674510258 >> 172.21.15.5:0/674510258 conn(0x7f2fb80fedc0 msgr2=0x7f2fb81011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:49:27.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.787+0000 7f2fbfe1f700 1 -- 172.21.15.5:0/674510258 shutdown_connections 2024-05-09T00:49:27.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.791+0000 7f2fbfe1f700 1 -- 172.21.15.5:0/674510258 wait complete. 2024-05-09T00:49:27.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.792+0000 7f2fbfe1f700 1 Processor -- start 2024-05-09T00:49:27.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.792+0000 7f2fbfe1f700 1 -- start start 2024-05-09T00:49:27.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.793+0000 7f2fbfe1f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fb81034c0 0x7f2fb8118c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:27.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.793+0000 7f2fbfe1f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fb8107df0 0x7f2fb811e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:27.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.793+0000 7f2fbfe1f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2fb8108740 0x7f2fb81192e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:27.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.793+0000 7f2fbfe1f700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb8112430 con 0x7f2fb81034c0 2024-05-09T00:49:27.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.793+0000 7f2fbfe1f700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb81122b0 con 0x7f2fb8108740 2024-05-09T00:49:27.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.793+0000 7f2fbfe1f700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb81125b0 con 0x7f2fb8107df0 2024-05-09T00:49:27.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.797+0000 7f2fbd3ba700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fb8107df0 0x7f2fb811e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:27.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.797+0000 7f2fbe3bc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2fb8108740 0x7f2fb81192e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:27.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.798+0000 7f2fbd3ba700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fb8107df0 0x7f2fb811e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:45726/0 (socket says 172.21.15.5:45726) 2024-05-09T00:49:27.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.798+0000 7f2fbd3ba700 1 -- 172.21.15.5:0/1187835531 learned_addr learned my addr 172.21.15.5:0/1187835531 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:49:27.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.799+0000 7f2fbdbbb700 1 --2- 172.21.15.5:0/1187835531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fb81034c0 0x7f2fb8118c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:27.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.800+0000 7f2fbdbbb700 1 -- 172.21.15.5:0/1187835531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fb8107df0 msgr2=0x7f2fb811e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:27.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.800+0000 7f2fbdbbb700 1 --2- 172.21.15.5:0/1187835531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fb8107df0 0x7f2fb811e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:27.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.800+0000 7f2fbdbbb700 1 -- 172.21.15.5:0/1187835531 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2fb8108740 msgr2=0x7f2fb81192e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:27.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.800+0000 7f2fbdbbb700 1 --2- 172.21.15.5:0/1187835531 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2fb8108740 0x7f2fb81192e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:27.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.800+0000 7f2fbdbbb700 1 -- 172.21.15.5:0/1187835531 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fb8114080 con 0x7f2fb81034c0 2024-05-09T00:49:27.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.801+0000 7f2fbdbbb700 1 --2- 172.21.15.5:0/1187835531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fb81034c0 0x7f2fb8118c20 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f2fa800d950 tx=0x7f2fa800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:49:27.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.802+0000 7f2faeffd700 1 -- 172.21.15.5:0/1187835531 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa8016070 con 0x7f2fb81034c0 2024-05-09T00:49:27.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.802+0000 7f2fbfe1f700 1 -- 172.21.15.5:0/1187835531 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fb8114370 con 0x7f2fb81034c0 2024-05-09T00:49:27.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.802+0000 7f2fbfe1f700 1 -- 172.21.15.5:0/1187835531 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fb81148b0 con 0x7f2fb81034c0 2024-05-09T00:49:27.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.802+0000 7f2faeffd700 1 -- 172.21.15.5:0/1187835531 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fa800fd40 con 0x7f2fb81034c0 2024-05-09T00:49:27.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.802+0000 7f2faeffd700 1 -- 172.21.15.5:0/1187835531 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa800e3b0 con 0x7f2fb81034c0 2024-05-09T00:49:27.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.805+0000 7f2fbe3bc700 1 --2- 172.21.15.5:0/1187835531 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2fb8108740 0x7f2fb81192e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:49:27.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.816+0000 7f2fce835b80 1 Processor -- start 2024-05-09T00:49:27.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.818+0000 7f2fce835b80 1 -- start start 2024-05-09T00:49:27.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.818+0000 7f2fce835b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f83fd56d00 0x55f83fe5f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:27.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.818+0000 7f2fce835b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f83fcdc4a0 0x55f83fe5f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:27.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.818+0000 7f2fce835b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f83fe59400 0x55f83fe59810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:27.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.818+0000 7f2fce835b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f83fd7c0a0 con 0x55f83fe59400 2024-05-09T00:49:27.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.818+0000 7f2fce835b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55f83fd219a0 con 0x55f83fd56d00 2024-05-09T00:49:27.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.818+0000 7f2fce835b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55f83fd78060 con 0x55f83fcdc4a0 2024-05-09T00:49:27.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.819+0000 7f2faeffd700 1 -- 172.21.15.5:0/1187835531 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f2fa8019070 con 0x7f2fb81034c0 2024-05-09T00:49:27.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.820+0000 7f2fbe3bc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f83fd56d00 0x55f83fe5f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:27.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.820+0000 7f2fbe3bc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f83fd56d00 0x55f83fe5f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:54370/0 (socket says 172.21.15.5:54370) 2024-05-09T00:49:27.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.820+0000 7f2fbe3bc700 1 -- 172.21.15.5:0/1578663450 learned_addr learned my addr 172.21.15.5:0/1578663450 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:49:27.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.820+0000 7f2fbd3ba700 1 --2- 172.21.15.5:0/1578663450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f83fe59400 0x55f83fe59810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:27.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.821+0000 7f2fbd3ba700 1 -- 172.21.15.5:0/1578663450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f83fcdc4a0 msgr2=0x55f83fe5f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:49:27.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.821+0000 7f2fbd3ba700 1 --2- 172.21.15.5:0/1578663450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f83fcdc4a0 0x55f83fe5f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:27.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.821+0000 7f2fbd3ba700 1 -- 172.21.15.5:0/1578663450 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f83fd56d00 msgr2=0x55f83fe5f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:49:27.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.821+0000 7f2fbd3ba700 1 --2- 172.21.15.5:0/1578663450 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f83fd56d00 0x55f83fe5f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:27.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.821+0000 7f2fbd3ba700 1 -- 172.21.15.5:0/1578663450 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f83fd12500 con 0x55f83fe59400 2024-05-09T00:49:27.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.823+0000 7f2faeffd700 1 --2- 172.21.15.5:0/1187835531 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f2fa40b2a50 0x7f2fa40b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:27.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.823+0000 7f2faeffd700 1 -- 172.21.15.5:0/1187835531 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f2fa80e1a90 con 0x7f2fb81034c0 2024-05-09T00:49:27.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.824+0000 7f2fbd3ba700 1 --2- 172.21.15.5:0/1578663450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f83fe59400 0x55f83fe59810 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f2fb000e140 tx=0x7f2fb0009760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85101945b397af6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:49:27.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.825+0000 7f2fa3fff700 1 -- 172.21.15.5:0/1578663450 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb0018020 con 0x55f83fe59400 2024-05-09T00:49:27.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.825+0000 7f2fa3fff700 1 -- 172.21.15.5:0/1578663450 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fb0014930 con 0x55f83fe59400 2024-05-09T00:49:27.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.826+0000 7f2fbe3bc700 1 --2- 172.21.15.5:0/1187835531 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f2fa40b2a50 0x7f2fa40b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:27.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.828+0000 7f2fce835b80 1 -- 172.21.15.5:0/1578663450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f83fe59400 msgr2=0x55f83fe59810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:27.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.828+0000 7f2fce835b80 1 --2- 172.21.15.5:0/1578663450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f83fe59400 0x55f83fe59810 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f2fb000e140 tx=0x7f2fb0009760 comp rx=0 tx=0).stop 2024-05-09T00:49:27.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.828+0000 7f2fbe3bc700 1 --2- 172.21.15.5:0/1187835531 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f2fa40b2a50 0x7f2fa40b4f60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2fb4009750 tx=0x7f2fb4005040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:49:27.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.837+0000 7f2fce835b80 1 -- 172.21.15.5:0/1578663450 shutdown_connections 2024-05-09T00:49:27.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.837+0000 7f2fce835b80 1 --2- 172.21.15.5:0/1578663450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f83fe59400 0x55f83fe59810 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:27.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.837+0000 7f2fce835b80 1 --2- 172.21.15.5:0/1578663450 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f83fd56d00 0x55f83fe5f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:27.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.837+0000 7f2fce835b80 1 --2- 172.21.15.5:0/1578663450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f83fcdc4a0 0x55f83fe5f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:27.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.837+0000 7f2fce835b80 1 -- 172.21.15.5:0/1578663450 >> 172.21.15.5:0/1578663450 conn(0x55f83fcdff70 msgr2=0x55f83fe54220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:49:27.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.837+0000 7f2fce835b80 1 -- 172.21.15.5:0/1578663450 shutdown_connections 2024-05-09T00:49:27.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.838+0000 7f2fce835b80 1 -- 172.21.15.5:0/1578663450 wait complete. 2024-05-09T00:49:27.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.839+0000 7f2fce835b80 1 -- start start 2024-05-09T00:49:27.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.840+0000 7f2fce835b80 1 Processor -- start 2024-05-09T00:49:27.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.841+0000 7f2fce835b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f83fcdc4a0 0x55f83fe62f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:27.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.841+0000 7f2fce835b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f83fd56d00 0x55f83fe634b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:27.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.841+0000 7f2fce835b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f83fe59400 0x55f83fe639f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:27.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.841+0000 7f2fce835b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f83fe5a1b0 con 0x55f83fe59400 2024-05-09T00:49:27.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.841+0000 7f2fce835b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55f83fd16b20 con 0x55f83fd56d00 2024-05-09T00:49:27.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.841+0000 7f2fce835b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55f83fe66020 con 0x55f83fcdc4a0 2024-05-09T00:49:27.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.841+0000 7f2fce835b80 10 client.0 _ll_register_callbacks cb 0x55f83fbed6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:49:27.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.842+0000 7f2fbd3ba700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f83fd56d00 0x55f83fe634b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:27.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.843+0000 7f2fbe3bc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f83fe59400 0x55f83fe639f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:27.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.844+0000 7f2fbe3bc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f83fe59400 0x55f83fe639f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41470/0 (socket says 172.21.15.5:41470) 2024-05-09T00:49:27.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.844+0000 7f2fbe3bc700 1 -- 172.21.15.5:0/1042679501 learned_addr learned my addr 172.21.15.5:0/1042679501 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:49:27.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.844+0000 7f2fbdbbb700 1 --2- 172.21.15.5:0/1042679501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f83fcdc4a0 0x55f83fe62f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:27.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.845+0000 7f2fbe3bc700 1 -- 172.21.15.5:0/1042679501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f83fcdc4a0 msgr2=0x55f83fe62f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:49:27.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.845+0000 7f2fbe3bc700 1 --2- 172.21.15.5:0/1042679501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f83fcdc4a0 0x55f83fe62f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:27.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.845+0000 7f2fbe3bc700 1 -- 172.21.15.5:0/1042679501 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f83fd56d00 msgr2=0x55f83fe634b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:49:27.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.845+0000 7f2fbe3bc700 1 --2- 172.21.15.5:0/1042679501 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f83fd56d00 0x55f83fe634b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:27.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.845+0000 7f2fbe3bc700 1 -- 172.21.15.5:0/1042679501 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f83fbf7490 con 0x55f83fe59400 2024-05-09T00:49:27.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.846+0000 7f2fbe3bc700 1 --2- 172.21.15.5:0/1042679501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f83fe59400 0x55f83fe639f0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f2fb40092c0 tx=0x7f2fb4009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ec90aaee69556e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:49:27.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.846+0000 7f2fa17fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:49:27.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.846+0000 7f2fa17fa700 1 -- 172.21.15.5:0/1042679501 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb4034060 con 0x55f83fe59400 2024-05-09T00:49:27.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.846+0000 7f2fce835b80 1 -- 172.21.15.5:0/1042679501 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fb0014930 con 0x55f83fe59400 2024-05-09T00:49:27.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.847+0000 7f2fa17fa700 1 -- 172.21.15.5:0/1042679501 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f2fb400f870 con 0x55f83fe59400 2024-05-09T00:49:27.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.847+0000 7f2fa17fa700 1 -- 172.21.15.5:0/1042679501 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2fb400bea0 con 0x55f83fe59400 2024-05-09T00:49:27.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.847+0000 7f2fce835b80 10 client.15789 fetch_fsmap learned FSMap version 29 2024-05-09T00:49:27.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.847+0000 7f2fce835b80 1 -- 172.21.15.5:0/1042679501 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f83fe70a50 con 0x55f83fe59400 2024-05-09T00:49:27.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.848+0000 7f2fa17fa700 1 -- 172.21.15.5:0/1042679501 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fb0014930 con 0x55f83fe59400 2024-05-09T00:49:27.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.850+0000 7f2fa17fa700 1 -- 172.21.15.5:0/1042679501 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f2fb403abc0 con 0x55f83fe59400 2024-05-09T00:49:27.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.850+0000 7f2fce835b80 10 client.15789 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:49:27.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.850+0000 7f2fce835b80 10 client.15789 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-05-09T00:49:27.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.850+0000 7f2fce835b80 20 client.15789 populate_metadata read hostname 'smithi005' 2024-05-09T00:49:27.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.850+0000 7f2fce835b80 1 --2- 172.21.15.5:0/1042679501 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x55f83fe73480 0x55f83fe75910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:27.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.850+0000 7f2fce835b80 4 client.15789 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:49:27.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.850+0000 7f2fce835b80 1 -- 172.21.15.5:0/1042679501 --> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f83fd89a80 con 0x55f83fe73480 2024-05-09T00:49:27.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.854+0000 7f2fbdbbb700 1 --2- 172.21.15.5:0/1042679501 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x55f83fe73480 0x55f83fe75910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:27.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.866+0000 7f2fbdbbb700 1 --2- 172.21.15.5:0/1042679501 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x55f83fe73480 0x55f83fe75910 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=784fe0be271a3639 server_cookie=875fc827a51e6852 in_seq=0 out_seq=0 2024-05-09T00:49:27.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.866+0000 7f2fa17fa700 10 client.15789 ms_handle_connect on v2:172.21.15.131:6838/2748669631 2024-05-09T00:49:27.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.870+0000 7f2fa17fa700 1 -- 172.21.15.5:0/1042679501 <== mds.2 v2:172.21.15.131:6838/2748669631 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f83fd89a80 con 0x55f83fe73480 2024-05-09T00:49:27.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.870+0000 7f2fa17fa700 10 client.15789 handle_command_reply: tid=0 2024-05-09T00:49:27.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.870+0000 7f2fce835b80 1 client.15789 shutdown 2024-05-09T00:49:27.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.871+0000 7f2fce835b80 1 -- 172.21.15.5:0/1042679501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f83fe59400 msgr2=0x55f83fe639f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:27.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.871+0000 7f2fce835b80 1 --2- 172.21.15.5:0/1042679501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f83fe59400 0x55f83fe639f0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f2fb40092c0 tx=0x7f2fb4009ca0 comp rx=0 tx=0).stop 2024-05-09T00:49:27.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.872+0000 7f2fce835b80 1 -- 172.21.15.5:0/1042679501 shutdown_connections 2024-05-09T00:49:27.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.872+0000 7f2fce835b80 1 --2- 172.21.15.5:0/1042679501 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x55f83fe73480 0x55f83fe75910 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:27.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.872+0000 7f2fce835b80 1 --2- 172.21.15.5:0/1042679501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f83fe59400 0x55f83fe639f0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:27.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.872+0000 7f2fce835b80 1 --2- 172.21.15.5:0/1042679501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f83fcdc4a0 0x55f83fe62f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:27.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.872+0000 7f2fce835b80 1 --2- 172.21.15.5:0/1042679501 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f83fd56d00 0x55f83fe634b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:27.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.872+0000 7f2fce835b80 1 -- 172.21.15.5:0/1042679501 >> 172.21.15.5:0/1042679501 conn(0x55f83fcdff70 msgr2=0x55f83fe4dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:49:27.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.872+0000 7f2fa17fa700 0 client.15789 ms_handle_reset on v2:172.21.15.131:6838/2748669631 2024-05-09T00:49:27.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.876+0000 7f2fce835b80 1 -- 172.21.15.5:0/1042679501 shutdown_connections 2024-05-09T00:49:27.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.876+0000 7f2fce835b80 1 -- 172.21.15.5:0/1042679501 wait complete. 2024-05-09T00:49:27.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.876+0000 7f2fce835b80 20 client.15789 trim_cache size 0 max 16384 2024-05-09T00:49:27.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.926+0000 7f2fce835b80 1 Processor -- start 2024-05-09T00:49:27.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.926+0000 7f2fce835b80 1 -- start start 2024-05-09T00:49:27.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.926+0000 7f2fce835b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f83fcdc4a0 0x55f83fe61ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:27.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.926+0000 7f2fce835b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f83fd56d00 0x55f83fe5e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:27.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.927+0000 7f2fce835b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f83fe59400 0x55f83fe6eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:27.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.927+0000 7f2fce835b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2f9c00d440 con 0x55f83fd56d00 2024-05-09T00:49:27.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.927+0000 7f2fce835b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2f9c00ef80 con 0x55f83fe59400 2024-05-09T00:49:27.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.927+0000 7f2fce835b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2f9c00db10 con 0x55f83fcdc4a0 2024-05-09T00:49:27.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.927+0000 7f2fbd3ba700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f83fd56d00 0x55f83fe5e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:27.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.927+0000 7f2fbd3ba700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f83fd56d00 0x55f83fe5e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41484/0 (socket says 172.21.15.5:41484) 2024-05-09T00:49:27.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.927+0000 7f2fbd3ba700 1 -- 172.21.15.5:0/2072711785 learned_addr learned my addr 172.21.15.5:0/2072711785 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:49:27.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.929+0000 7f2fbd3ba700 1 -- 172.21.15.5:0/2072711785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f83fcdc4a0 msgr2=0x55f83fe61ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:49:27.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.929+0000 7f2fbdbbb700 1 --2- 172.21.15.5:0/2072711785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f83fcdc4a0 0x55f83fe61ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:27.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.930+0000 7f2fbd3ba700 1 --2- 172.21.15.5:0/2072711785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f83fcdc4a0 0x55f83fe61ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:27.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.930+0000 7f2fbd3ba700 1 -- 172.21.15.5:0/2072711785 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f83fe59400 msgr2=0x55f83fe6eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:49:27.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.930+0000 7f2fbd3ba700 1 --2- 172.21.15.5:0/2072711785 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f83fe59400 0x55f83fe6eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:27.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.930+0000 7f2fbd3ba700 1 -- 172.21.15.5:0/2072711785 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f83fe5a600 con 0x55f83fd56d00 2024-05-09T00:49:27.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.932+0000 7f2fbd3ba700 1 --2- 172.21.15.5:0/2072711785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f83fd56d00 0x55f83fe5e770 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0013450 tx=0x7f2fb000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83eedb5b5d4d4c2d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:49:27.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.936+0000 7f2fa37fe700 1 -- 172.21.15.5:0/2072711785 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb0018070 con 0x55f83fd56d00 2024-05-09T00:49:27.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.936+0000 7f2fa37fe700 1 -- 172.21.15.5:0/2072711785 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fb0003ca0 con 0x55f83fd56d00 2024-05-09T00:49:27.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.938+0000 7f2fce835b80 1 -- 172.21.15.5:0/2072711785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f83fd56d00 msgr2=0x55f83fe5e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:27.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.938+0000 7f2fce835b80 1 --2- 172.21.15.5:0/2072711785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f83fd56d00 0x55f83fe5e770 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0013450 tx=0x7f2fb000ef20 comp rx=0 tx=0).stop 2024-05-09T00:49:27.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.942+0000 7f2fce835b80 1 -- 172.21.15.5:0/2072711785 shutdown_connections 2024-05-09T00:49:27.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.942+0000 7f2fce835b80 1 --2- 172.21.15.5:0/2072711785 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f83fe59400 0x55f83fe6eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:27.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.942+0000 7f2fce835b80 1 --2- 172.21.15.5:0/2072711785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f83fcdc4a0 0x55f83fe61ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:27.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.942+0000 7f2fce835b80 1 --2- 172.21.15.5:0/2072711785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f83fd56d00 0x55f83fe5e770 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:27.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.942+0000 7f2fce835b80 1 -- 172.21.15.5:0/2072711785 >> 172.21.15.5:0/2072711785 conn(0x55f83fcdff70 msgr2=0x55f83fe5e060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:49:27.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.947+0000 7f2fce835b80 1 -- 172.21.15.5:0/2072711785 shutdown_connections 2024-05-09T00:49:27.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.948+0000 7f2fce835b80 1 -- 172.21.15.5:0/2072711785 wait complete. 2024-05-09T00:49:27.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.948+0000 7f2fce835b80 1 -- start start 2024-05-09T00:49:27.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.949+0000 7f2fce835b80 1 Processor -- start 2024-05-09T00:49:27.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.950+0000 7f2fce835b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f83fcdc4a0 0x55f83fe7b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:27.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.950+0000 7f2fce835b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f83fd56d00 0x55f83fe6b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:27.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.950+0000 7f2fce835b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f83fe59400 0x55f83fe6b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:27.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.950+0000 7f2fce835b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2f9c00e8b0 con 0x55f83fcdc4a0 2024-05-09T00:49:27.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.950+0000 7f2fce835b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2f9c00e1e0 con 0x55f83fd56d00 2024-05-09T00:49:27.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.950+0000 7f2fce835b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2f9c00b490 con 0x55f83fe59400 2024-05-09T00:49:27.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.950+0000 7f2fce835b80 10 client.0 _ll_register_callbacks cb 0x55f83fbed6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:49:27.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.952+0000 7f2fbd3ba700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f83fd56d00 0x55f83fe6b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:27.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.953+0000 7f2fbd3ba700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f83fd56d00 0x55f83fe6b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:54406/0 (socket says 172.21.15.5:54406) 2024-05-09T00:49:27.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.953+0000 7f2fbd3ba700 1 -- 172.21.15.5:0/1837282798 learned_addr learned my addr 172.21.15.5:0/1837282798 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:49:27.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.953+0000 7f2fbdbbb700 1 --2- 172.21.15.5:0/1837282798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f83fcdc4a0 0x55f83fe7b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:27.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.953+0000 7f2fbe3bc700 1 --2- 172.21.15.5:0/1837282798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f83fe59400 0x55f83fe6b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:27.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.955+0000 7f2fbd3ba700 1 -- 172.21.15.5:0/1837282798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f83fe59400 msgr2=0x55f83fe6b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:49:27.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.955+0000 7f2fbd3ba700 1 --2- 172.21.15.5:0/1837282798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f83fe59400 0x55f83fe6b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:27.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.955+0000 7f2fbd3ba700 1 -- 172.21.15.5:0/1837282798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f83fcdc4a0 msgr2=0x55f83fe7b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:49:27.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.955+0000 7f2fbd3ba700 1 --2- 172.21.15.5:0/1837282798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f83fcdc4a0 0x55f83fe7b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:27.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.955+0000 7f2fbd3ba700 1 -- 172.21.15.5:0/1837282798 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f83fe7a100 con 0x55f83fd56d00 2024-05-09T00:49:27.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.956+0000 7f2fbd3ba700 1 --2- 172.21.15.5:0/1837282798 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f83fd56d00 0x55f83fe6b270 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2fb00134c0 tx=0x7f2fb0013de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90842b7f68793123 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:49:27.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.957+0000 7f2fa17fa700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-09T00:49:27.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.961+0000 7f2fa17fa700 1 -- 172.21.15.5:0/1837282798 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb0018070 con 0x55f83fd56d00 2024-05-09T00:49:27.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.961+0000 7f2fa17fa700 1 -- 172.21.15.5:0/1837282798 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f2fb0049660 con 0x55f83fd56d00 2024-05-09T00:49:27.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.963+0000 7f2fce835b80 1 -- 172.21.15.5:0/1837282798 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fb0003ca0 con 0x55f83fd56d00 2024-05-09T00:49:27.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.964+0000 7f2fa17fa700 1 -- 172.21.15.5:0/1837282798 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2fb0014930 con 0x55f83fd56d00 2024-05-09T00:49:27.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.967+0000 7f2fce835b80 10 client.25430 fetch_fsmap learned FSMap version 29 2024-05-09T00:49:27.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.967+0000 7f2fce835b80 1 -- 172.21.15.5:0/1837282798 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f83fedcdb0 con 0x55f83fd56d00 2024-05-09T00:49:27.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.971+0000 7f2fa17fa700 1 -- 172.21.15.5:0/1837282798 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fb0003ca0 con 0x55f83fd56d00 2024-05-09T00:49:27.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.971+0000 7f2fa17fa700 1 -- 172.21.15.5:0/1837282798 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f2fb0047020 con 0x55f83fd56d00 2024-05-09T00:49:27.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.971+0000 7f2fce835b80 10 client.25430 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:49:27.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.971+0000 7f2fce835b80 10 client.25430 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-05-09T00:49:27.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.971+0000 7f2fce835b80 20 client.25430 populate_metadata read hostname 'smithi005' 2024-05-09T00:49:27.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.971+0000 7f2fce835b80 1 --2- 172.21.15.5:0/1837282798 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x55f83fedd1e0 0x55f83fedf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:27.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.971+0000 7f2fce835b80 4 client.25430 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:49:27.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.971+0000 7f2fce835b80 1 -- 172.21.15.5:0/1837282798 --> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f83fd83b20 con 0x55f83fedd1e0 2024-05-09T00:49:27.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.978+0000 7f2fbdbbb700 1 --2- 172.21.15.5:0/1837282798 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x55f83fedd1e0 0x55f83fedf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:27.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.986+0000 7f2fa17fa700 10 client.25430 ms_handle_connect on v2:172.21.15.131:6838/2748669631 2024-05-09T00:49:27.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.986+0000 7f2fbdbbb700 1 --2- 172.21.15.5:0/1837282798 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x55f83fedd1e0 0x55f83fedf600 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8b96bcd07c33a07f server_cookie=4b0e1c77eaa6f646 in_seq=0 out_seq=0 2024-05-09T00:49:27.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.987+0000 7f2fa17fa700 1 -- 172.21.15.5:0/1837282798 <== mds.2 v2:172.21.15.131:6838/2748669631 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f83fd83b20 con 0x55f83fedd1e0 2024-05-09T00:49:27.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.987+0000 7f2fa17fa700 10 client.25430 handle_command_reply: tid=0 2024-05-09T00:49:27.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.987+0000 7f2fce835b80 1 client.25430 shutdown 2024-05-09T00:49:27.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.989+0000 7f2fce835b80 1 -- 172.21.15.5:0/1837282798 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f83fd56d00 msgr2=0x55f83fe6b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:27.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.989+0000 7f2fce835b80 1 --2- 172.21.15.5:0/1837282798 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f83fd56d00 0x55f83fe6b270 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2fb00134c0 tx=0x7f2fb0013de0 comp rx=0 tx=0).stop 2024-05-09T00:49:27.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.990+0000 7f2fce835b80 1 -- 172.21.15.5:0/1837282798 shutdown_connections 2024-05-09T00:49:27.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.990+0000 7f2fce835b80 1 --2- 172.21.15.5:0/1837282798 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x55f83fedd1e0 0x55f83fedf600 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:27.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.990+0000 7f2fce835b80 1 --2- 172.21.15.5:0/1837282798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f83fe59400 0x55f83fe6b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:27.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.990+0000 7f2fce835b80 1 --2- 172.21.15.5:0/1837282798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f83fcdc4a0 0x55f83fe7b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:27.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.990+0000 7f2fce835b80 1 --2- 172.21.15.5:0/1837282798 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f83fd56d00 0x55f83fe6b270 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:27.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.990+0000 7f2fa17fa700 0 client.25430 ms_handle_reset on v2:172.21.15.131:6838/2748669631 2024-05-09T00:49:27.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.990+0000 7f2fce835b80 1 -- 172.21.15.5:0/1837282798 >> 172.21.15.5:0/1837282798 conn(0x55f83fcdff70 msgr2=0x55f83fe4dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:49:27.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.991+0000 7f2fce835b80 1 -- 172.21.15.5:0/1837282798 shutdown_connections 2024-05-09T00:49:27.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.991+0000 7f2fce835b80 1 -- 172.21.15.5:0/1837282798 wait complete. 2024-05-09T00:49:27.995 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-09T00:49:27.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.991+0000 7f2fce835b80 20 client.25430 trim_cache size 0 max 16384 2024-05-09T00:49:27.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.993+0000 7f2fbfe1f700 1 -- 172.21.15.5:0/1187835531 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f2fa40b2a50 msgr2=0x7f2fa40b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:27.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.993+0000 7f2fbfe1f700 1 --2- 172.21.15.5:0/1187835531 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f2fa40b2a50 0x7f2fa40b4f60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2fb4009750 tx=0x7f2fb4005040 comp rx=0 tx=0).stop 2024-05-09T00:49:27.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.993+0000 7f2fbfe1f700 1 -- 172.21.15.5:0/1187835531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fb81034c0 msgr2=0x7f2fb8118c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:27.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:27.993+0000 7f2fbfe1f700 1 --2- 172.21.15.5:0/1187835531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fb81034c0 0x7f2fb8118c20 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f2fa800d950 tx=0x7f2fa800dc60 comp rx=0 tx=0).stop 2024-05-09T00:49:28.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:28.000+0000 7f2fbfe1f700 1 -- 172.21.15.5:0/1187835531 shutdown_connections 2024-05-09T00:49:28.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:28.000+0000 7f2fbfe1f700 1 --2- 172.21.15.5:0/1187835531 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f2fa40b2a50 0x7f2fa40b4f60 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:28.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:28.000+0000 7f2fbfe1f700 1 --2- 172.21.15.5:0/1187835531 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2fb8108740 0x7f2fb81192e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:28.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:28.000+0000 7f2fbfe1f700 1 --2- 172.21.15.5:0/1187835531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fb81034c0 0x7f2fb8118c20 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:28.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:28.000+0000 7f2fbfe1f700 1 --2- 172.21.15.5:0/1187835531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fb8107df0 0x7f2fb811e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:28.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:28.000+0000 7f2fbfe1f700 1 -- 172.21.15.5:0/1187835531 >> 172.21.15.5:0/1187835531 conn(0x7f2fb80fedc0 msgr2=0x7f2fb81113a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:49:28.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:28.002+0000 7f2fbfe1f700 1 -- 172.21.15.5:0/1187835531 shutdown_connections 2024-05-09T00:49:28.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:28.002+0000 7f2fbfe1f700 1 -- 172.21.15.5:0/1187835531 wait complete. 2024-05-09T00:49:29.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:49:29 smithi157 ceph-mon[34294]: pgmap v465: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 782 MiB/s wr, 2.76k op/s 2024-05-09T00:49:29.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:49:29 smithi005 ceph-mon[30365]: pgmap v465: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 782 MiB/s wr, 2.76k op/s 2024-05-09T00:49:29.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:49:29 smithi131 ceph-mon[34547]: pgmap v465: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 782 MiB/s wr, 2.76k op/s 2024-05-09T00:49:30.252 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T00:49:30.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:30.700+0000 7fe459ffb640 1 -- 172.21.15.5:0/1626329240 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7fe46c042070 con 0x55b2c0f02200 2024-05-09T00:49:30.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:30.700+0000 7fe459ffb640 10 client.15777 handle_command_reply: tid=0 2024-05-09T00:49:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:30.733+0000 7fe47f42e740 1 client.15777 shutdown 2024-05-09T00:49:30.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:30.740+0000 7fe47f42e740 1 -- 172.21.15.5:0/1626329240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2c0aa42f0 msgr2=0x55b2c0e90630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:30.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:30.740+0000 7fe47f42e740 1 --2- 172.21.15.5:0/1626329240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2c0aa42f0 0x55b2c0e90630 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fe4680a9d00 tx=0x7fe4680e2cc0 comp rx=0 tx=0).stop 2024-05-09T00:49:30.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:30.740+0000 7fe47f42e740 1 -- 172.21.15.5:0/1626329240 shutdown_connections 2024-05-09T00:49:30.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:30.740+0000 7fe47f42e740 1 --2- 172.21.15.5:0/1626329240 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55b2c0f02200 0x55b2c0f04650 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:30.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:30.740+0000 7fe47f42e740 1 --2- 172.21.15.5:0/1626329240 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2c0acd1b0 0x55b2c0f002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:30.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:30.740+0000 7fe47f42e740 1 --2- 172.21.15.5:0/1626329240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2c0aa42f0 0x55b2c0e90630 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:30.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:30.740+0000 7fe47f42e740 1 --2- 172.21.15.5:0/1626329240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2c0a611c0 0x55b2c0e900f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:30.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:30.740+0000 7fe47f42e740 1 -- 172.21.15.5:0/1626329240 >> 172.21.15.5:0/1626329240 conn(0x55b2c0ac1300 msgr2=0x55b2c0e6fe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:49:30.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:30.740+0000 7fe459ffb640 0 client.15777 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:49:30.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:30.743+0000 7fe47f42e740 1 -- 172.21.15.5:0/1626329240 shutdown_connections 2024-05-09T00:49:30.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:30.743+0000 7fe47f42e740 1 -- 172.21.15.5:0/1626329240 wait complete. 2024-05-09T00:49:30.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:30.743+0000 7fe47f42e740 20 client.15777 trim_cache size 0 max 16384 2024-05-09T00:49:30.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-09T00:49:30.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-09T00:49:30.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-09T00:49:30.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:30.748+0000 7fe47723d640 1 -- 172.21.15.5:0/1426074982 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe45c0b29f0 msgr2=0x7fe45c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:30.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:30.749+0000 7fe47723d640 1 --2- 172.21.15.5:0/1426074982 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe45c0b29f0 0x7fe45c0b4f00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe46c005f70 tx=0x7fe46c009260 comp rx=0 tx=0).stop 2024-05-09T00:49:30.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:30.749+0000 7fe47723d640 1 -- 172.21.15.5:0/1426074982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe470076b20 msgr2=0x7fe4701a0c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:30.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:30.749+0000 7fe47723d640 1 --2- 172.21.15.5:0/1426074982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe470076b20 0x7fe4701a0c60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe46800eab0 tx=0x7fe46800ef70 comp rx=0 tx=0).stop 2024-05-09T00:49:30.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:30.752+0000 7fe47723d640 1 -- 172.21.15.5:0/1426074982 shutdown_connections 2024-05-09T00:49:30.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:30.752+0000 7fe47723d640 1 --2- 172.21.15.5:0/1426074982 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe45c0b29f0 0x7fe45c0b4f00 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:30.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:30.752+0000 7fe47723d640 1 --2- 172.21.15.5:0/1426074982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4701102f0 0x7fe4701a7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:30.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:30.752+0000 7fe47723d640 1 --2- 172.21.15.5:0/1426074982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe470076b20 0x7fe4701a0c60 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:30.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:30.752+0000 7fe47723d640 1 --2- 172.21.15.5:0/1426074982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4700761d0 0x7fe4701a0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:30.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:30.753+0000 7fe47723d640 1 -- 172.21.15.5:0/1426074982 >> 172.21.15.5:0/1426074982 conn(0x7fe4700fed80 msgr2=0x7fe470103040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:49:30.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:30.756+0000 7fe47723d640 1 -- 172.21.15.5:0/1426074982 shutdown_connections 2024-05-09T00:49:30.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:30.756+0000 7fe47723d640 1 -- 172.21.15.5:0/1426074982 wait complete. 2024-05-09T00:49:31.124 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:49:31.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:49:31 smithi157 ceph-mon[34294]: pgmap v466: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 971 MiB/s wr, 3.14k op/s 2024-05-09T00:49:31.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:49:31 smithi005 ceph-mon[30365]: pgmap v466: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 971 MiB/s wr, 3.14k op/s 2024-05-09T00:49:31.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:49:31 smithi131 ceph-mon[34547]: pgmap v466: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 971 MiB/s wr, 3.14k op/s 2024-05-09T00:49:33.406 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:49:33 smithi131 ceph-mon[34547]: pgmap v467: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 847 MiB/s wr, 2.74k op/s 2024-05-09T00:49:33.435 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:49:33 smithi157 ceph-mon[34294]: pgmap v467: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 847 MiB/s wr, 2.74k op/s 2024-05-09T00:49:33.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:49:33 smithi005 ceph-mon[30365]: pgmap v467: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 847 MiB/s wr, 2.74k op/s 2024-05-09T00:49:35.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:49:35 smithi005 ceph-mon[30365]: pgmap v468: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 896 MiB/s wr, 2.96k op/s 2024-05-09T00:49:35.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:49:35 smithi157 ceph-mon[34294]: pgmap v468: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 896 MiB/s wr, 2.96k op/s 2024-05-09T00:49:35.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:49:35 smithi131 ceph-mon[34547]: pgmap v468: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 896 MiB/s wr, 2.96k op/s 2024-05-09T00:49:36.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.441+0000 7fed71aad700 1 -- 172.21.15.5:0/882599762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed6c10b280 msgr2=0x7fed6c11d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:36.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.441+0000 7fed71aad700 1 --2- 172.21.15.5:0/882599762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed6c10b280 0x7fed6c11d3f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fed600098f0 tx=0x7fed60009c00 comp rx=0 tx=0).stop 2024-05-09T00:49:36.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.442+0000 7fed71aad700 1 -- 172.21.15.5:0/882599762 shutdown_connections 2024-05-09T00:49:36.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.442+0000 7fed71aad700 1 --2- 172.21.15.5:0/882599762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed6c11d930 0x7fed6c11fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:36.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.442+0000 7fed71aad700 1 --2- 172.21.15.5:0/882599762 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed6c10a930 0x7fed6c10ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:36.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.442+0000 7fed71aad700 1 --2- 172.21.15.5:0/882599762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed6c10b280 0x7fed6c11d3f0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:36.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.442+0000 7fed71aad700 1 -- 172.21.15.5:0/882599762 >> 172.21.15.5:0/882599762 conn(0x7fed6c06f4d0 msgr2=0x7fed6c06f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:49:36.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.442+0000 7fed71aad700 1 -- 172.21.15.5:0/882599762 shutdown_connections 2024-05-09T00:49:36.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.443+0000 7fed71aad700 1 -- 172.21.15.5:0/882599762 wait complete. 2024-05-09T00:49:36.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.445+0000 7fed71aad700 1 Processor -- start 2024-05-09T00:49:36.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.446+0000 7fed71aad700 1 -- start start 2024-05-09T00:49:36.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.446+0000 7fed71aad700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed6c10a930 0x7fed6c11d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:36.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.446+0000 7fed71aad700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed6c10b280 0x7fed6c117530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:36.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.446+0000 7fed71aad700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed6c11d930 0x7fed6c117c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:36.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.446+0000 7fed71aad700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fed6c1134d0 con 0x7fed6c11d930 2024-05-09T00:49:36.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.446+0000 7fed71aad700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fed6c113350 con 0x7fed6c10b280 2024-05-09T00:49:36.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.446+0000 7fed71aad700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fed6c113650 con 0x7fed6c10a930 2024-05-09T00:49:36.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.447+0000 7fed6bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed6c11d930 0x7fed6c117c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:36.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.447+0000 7fed6bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed6c11d930 0x7fed6c117c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53714/0 (socket says 172.21.15.5:53714) 2024-05-09T00:49:36.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.447+0000 7fed6bfff700 1 -- 172.21.15.5:0/1626167041 learned_addr learned my addr 172.21.15.5:0/1626167041 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:49:36.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.449+0000 7fed6b7fe700 1 --2- 172.21.15.5:0/1626167041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed6c10a930 0x7fed6c11d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:36.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.449+0000 7fed6bfff700 1 -- 172.21.15.5:0/1626167041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed6c10a930 msgr2=0x7fed6c11d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:36.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.449+0000 7fed6affd700 1 --2- 172.21.15.5:0/1626167041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed6c10b280 0x7fed6c117530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:36.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.449+0000 7fed6bfff700 1 --2- 172.21.15.5:0/1626167041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed6c10a930 0x7fed6c11d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:36.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.449+0000 7fed6bfff700 1 -- 172.21.15.5:0/1626167041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed6c10b280 msgr2=0x7fed6c117530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:36.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.449+0000 7fed6bfff700 1 --2- 172.21.15.5:0/1626167041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed6c10b280 0x7fed6c117530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:36.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.449+0000 7fed6bfff700 1 -- 172.21.15.5:0/1626167041 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed6c1be790 con 0x7fed6c11d930 2024-05-09T00:49:36.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.450+0000 7fed6bfff700 1 --2- 172.21.15.5:0/1626167041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed6c11d930 0x7fed6c117c20 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fed6400ef90 tx=0x7fed6400c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:49:36.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.454+0000 7fed68ff9700 1 -- 172.21.15.5:0/1626167041 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6400cd10 con 0x7fed6c11d930 2024-05-09T00:49:36.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.454+0000 7fed68ff9700 1 -- 172.21.15.5:0/1626167041 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed64004bb0 con 0x7fed6c11d930 2024-05-09T00:49:36.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.454+0000 7fed71aad700 1 -- 172.21.15.5:0/1626167041 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed6c1be9c0 con 0x7fed6c11d930 2024-05-09T00:49:36.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.454+0000 7fed71aad700 1 -- 172.21.15.5:0/1626167041 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed6c1bef00 con 0x7fed6c11d930 2024-05-09T00:49:36.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.456+0000 7fed68ff9700 1 -- 172.21.15.5:0/1626167041 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed64004280 con 0x7fed6c11d930 2024-05-09T00:49:36.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.466+0000 7fed68ff9700 1 -- 172.21.15.5:0/1626167041 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fed6401b030 con 0x7fed6c11d930 2024-05-09T00:49:36.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.468+0000 7fed68ff9700 1 --2- 172.21.15.5:0/1626167041 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fed540b2860 0x7fed540b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:36.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.469+0000 7fed68ff9700 1 -- 172.21.15.5:0/1626167041 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fed640e1dd0 con 0x7fed6c11d930 2024-05-09T00:49:36.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.470+0000 7fed804c3b80 1 Processor -- start 2024-05-09T00:49:36.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.475+0000 7fed6b7fe700 1 --2- 172.21.15.5:0/1626167041 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fed540b2860 0x7fed540b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:36.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.475+0000 7fed804c3b80 1 -- start start 2024-05-09T00:49:36.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.475+0000 7fed804c3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627f134f160 0x5627f13e84f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:36.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.476+0000 7fed804c3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627f12654a0 0x5627f13e8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:36.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.476+0000 7fed804c3b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627f13e26a0 0x5627f13e2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:36.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.476+0000 7fed804c3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5627f1304ff0 con 0x5627f12654a0 2024-05-09T00:49:36.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.476+0000 7fed804c3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5627f12a37f0 con 0x5627f134f160 2024-05-09T00:49:36.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.476+0000 7fed804c3b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5627f1300fb0 con 0x5627f13e26a0 2024-05-09T00:49:36.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.477+0000 7fed6b7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627f134f160 0x5627f13e84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:36.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.478+0000 7fed6b7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627f134f160 0x5627f13e84f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:55294/0 (socket says 172.21.15.5:55294) 2024-05-09T00:49:36.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.478+0000 7fed6b7fe700 1 -- 172.21.15.5:0/248832015 learned_addr learned my addr 172.21.15.5:0/248832015 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:49:36.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.478+0000 7fed6bfff700 1 --2- 172.21.15.5:0/248832015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627f13e26a0 0x5627f13e2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:36.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.481+0000 7fed6affd700 1 --2- 172.21.15.5:0/248832015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627f12654a0 0x5627f13e8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:36.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.481+0000 7fed6b7fe700 1 -- 172.21.15.5:0/248832015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627f13e26a0 msgr2=0x5627f13e2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:49:36.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.481+0000 7fed6b7fe700 1 --2- 172.21.15.5:0/248832015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627f13e26a0 0x5627f13e2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:36.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.481+0000 7fed6b7fe700 1 -- 172.21.15.5:0/248832015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627f12654a0 msgr2=0x5627f13e8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:49:36.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.481+0000 7fed6b7fe700 1 --2- 172.21.15.5:0/248832015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627f12654a0 0x5627f13e8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:36.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.481+0000 7fed6b7fe700 1 -- 172.21.15.5:0/248832015 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627f129b370 con 0x5627f134f160 2024-05-09T00:49:36.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.483+0000 7fed6b7fe700 1 --2- 172.21.15.5:0/248832015 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627f134f160 0x5627f13e84f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fed5c015740 tx=0x7fed5c015a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b8e6992d7332899 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:49:36.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.484+0000 7fed51ffb700 1 -- 172.21.15.5:0/248832015 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed5c022070 con 0x5627f134f160 2024-05-09T00:49:36.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.484+0000 7fed51ffb700 1 -- 172.21.15.5:0/248832015 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed5c013cf0 con 0x5627f134f160 2024-05-09T00:49:36.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.484+0000 7fed51ffb700 1 -- 172.21.15.5:0/248832015 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed5c01d3f0 con 0x5627f134f160 2024-05-09T00:49:36.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.484+0000 7fed6affd700 1 --2- 172.21.15.5:0/248832015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627f12654a0 0x5627f13e8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:49:36.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.486+0000 7fed804c3b80 1 -- 172.21.15.5:0/248832015 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627f134f160 msgr2=0x5627f13e84f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:36.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.486+0000 7fed804c3b80 1 --2- 172.21.15.5:0/248832015 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627f134f160 0x5627f13e84f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fed5c015740 tx=0x7fed5c015a50 comp rx=0 tx=0).stop 2024-05-09T00:49:36.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.489+0000 7fed6b7fe700 1 --2- 172.21.15.5:0/1626167041 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fed540b2860 0x7fed540b4d70 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fed5c015740 tx=0x7fed5c015a50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:49:36.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.489+0000 7fed804c3b80 1 -- 172.21.15.5:0/248832015 shutdown_connections 2024-05-09T00:49:36.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.489+0000 7fed804c3b80 1 --2- 172.21.15.5:0/248832015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627f13e26a0 0x5627f13e2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:36.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.489+0000 7fed804c3b80 1 --2- 172.21.15.5:0/248832015 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627f134f160 0x5627f13e84f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:36.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.489+0000 7fed804c3b80 1 --2- 172.21.15.5:0/248832015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627f12654a0 0x5627f13e8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:36.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.489+0000 7fed804c3b80 1 -- 172.21.15.5:0/248832015 >> 172.21.15.5:0/248832015 conn(0x5627f12c56a0 msgr2=0x5627f13dd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:49:36.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.490+0000 7fed804c3b80 1 -- 172.21.15.5:0/248832015 shutdown_connections 2024-05-09T00:49:36.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.491+0000 7fed804c3b80 1 -- 172.21.15.5:0/248832015 wait complete. 2024-05-09T00:49:36.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.492+0000 7fed804c3b80 1 -- start start 2024-05-09T00:49:36.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.492+0000 7fed804c3b80 1 Processor -- start 2024-05-09T00:49:36.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.493+0000 7fed804c3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627f12654a0 0x5627f13ec140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:36.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.496+0000 7fed804c3b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627f134f160 0x5627f13ec680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:36.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.497+0000 7fed804c3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627f13e26a0 0x5627f13ecbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:36.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.497+0000 7fed804c3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5627f1180ba0 con 0x5627f12654a0 2024-05-09T00:49:36.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.497+0000 7fed804c3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5627f12bf150 con 0x5627f13e26a0 2024-05-09T00:49:36.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.497+0000 7fed804c3b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5627f13ef1d0 con 0x5627f134f160 2024-05-09T00:49:36.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.497+0000 7fed804c3b80 10 client.0 _ll_register_callbacks cb 0x5627f11766c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:49:36.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.499+0000 7fed6bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627f13e26a0 0x5627f13ecbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:36.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.499+0000 7fed6affd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627f134f160 0x5627f13ec680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:36.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.499+0000 7fed6affd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627f134f160 0x5627f13ec680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:45556/0 (socket says 172.21.15.5:45556) 2024-05-09T00:49:36.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.499+0000 7fed6affd700 1 -- 172.21.15.5:0/141876605 learned_addr learned my addr 172.21.15.5:0/141876605 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:49:36.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.499+0000 7fed6b7fe700 1 --2- 172.21.15.5:0/141876605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627f12654a0 0x5627f13ec140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:36.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.505+0000 7fed6b7fe700 1 -- 172.21.15.5:0/141876605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627f134f160 msgr2=0x5627f13ec680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:49:36.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.505+0000 7fed6b7fe700 1 --2- 172.21.15.5:0/141876605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627f134f160 0x5627f13ec680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:36.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.505+0000 7fed6b7fe700 1 -- 172.21.15.5:0/141876605 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627f13e26a0 msgr2=0x5627f13ecbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:49:36.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.505+0000 7fed6b7fe700 1 --2- 172.21.15.5:0/141876605 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627f13e26a0 0x5627f13ecbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:36.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.505+0000 7fed6b7fe700 1 -- 172.21.15.5:0/141876605 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627f1180490 con 0x5627f12654a0 2024-05-09T00:49:36.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.506+0000 7fed6b7fe700 1 --2- 172.21.15.5:0/141876605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627f12654a0 0x5627f13ec140 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fed5c015710 tx=0x7fed5c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ecbd61d557eb6ec server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:49:36.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.506+0000 7fed4affd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:49:36.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.507+0000 7fed4affd700 1 -- 172.21.15.5:0/141876605 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed5c022040 con 0x5627f12654a0 2024-05-09T00:49:36.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.507+0000 7fed4affd700 1 -- 172.21.15.5:0/141876605 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fed5c0139a0 con 0x5627f12654a0 2024-05-09T00:49:36.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.507+0000 7fed804c3b80 1 -- 172.21.15.5:0/141876605 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed5c013cf0 con 0x5627f12654a0 2024-05-09T00:49:36.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.509+0000 7fed4affd700 1 -- 172.21.15.5:0/141876605 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fed5c01d3f0 con 0x5627f12654a0 2024-05-09T00:49:36.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.509+0000 7fed804c3b80 10 client.15804 fetch_fsmap learned FSMap version 29 2024-05-09T00:49:36.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.510+0000 7fed804c3b80 1 -- 172.21.15.5:0/141876605 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5627f13f9940 con 0x5627f12654a0 2024-05-09T00:49:36.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.511+0000 7fed4affd700 1 -- 172.21.15.5:0/141876605 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed5c01db60 con 0x5627f12654a0 2024-05-09T00:49:36.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.511+0000 7fed4affd700 1 -- 172.21.15.5:0/141876605 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fed5c01c030 con 0x5627f12654a0 2024-05-09T00:49:36.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.512+0000 7fed804c3b80 10 client.15804 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:49:36.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.512+0000 7fed804c3b80 10 client.15804 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-05-09T00:49:36.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.512+0000 7fed804c3b80 20 client.15804 populate_metadata read hostname 'smithi005' 2024-05-09T00:49:36.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.512+0000 7fed804c3b80 1 --2- 172.21.15.5:0/141876605 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x5627f13fbf20 0x5627f13fe340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:36.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.512+0000 7fed804c3b80 4 client.15804 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:49:36.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.512+0000 7fed804c3b80 1 -- 172.21.15.5:0/141876605 --> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fed5c013cf0 con 0x5627f13fbf20 2024-05-09T00:49:36.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.515+0000 7fed6affd700 1 --2- 172.21.15.5:0/141876605 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x5627f13fbf20 0x5627f13fe340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:36.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.516+0000 7fed6affd700 1 --2- 172.21.15.5:0/141876605 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x5627f13fbf20 0x5627f13fe340 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7b834bf1e3d290e4 server_cookie=120ce729d208768 in_seq=0 out_seq=0 2024-05-09T00:49:36.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.516+0000 7fed4affd700 10 client.15804 ms_handle_connect on v2:172.21.15.157:6836/744863009 2024-05-09T00:49:36.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.518+0000 7fed4affd700 1 -- 172.21.15.5:0/141876605 <== mds.1 v2:172.21.15.157:6836/744863009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fed5c013cf0 con 0x5627f13fbf20 2024-05-09T00:49:36.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.518+0000 7fed4affd700 10 client.15804 handle_command_reply: tid=0 2024-05-09T00:49:36.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.519+0000 7fed804c3b80 1 client.15804 shutdown 2024-05-09T00:49:36.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.520+0000 7fed804c3b80 1 -- 172.21.15.5:0/141876605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627f12654a0 msgr2=0x5627f13ec140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:36.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.520+0000 7fed804c3b80 1 --2- 172.21.15.5:0/141876605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627f12654a0 0x5627f13ec140 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fed5c015710 tx=0x7fed5c004750 comp rx=0 tx=0).stop 2024-05-09T00:49:36.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.522+0000 7fed804c3b80 1 -- 172.21.15.5:0/141876605 shutdown_connections 2024-05-09T00:49:36.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.522+0000 7fed804c3b80 1 --2- 172.21.15.5:0/141876605 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x5627f13fbf20 0x5627f13fe340 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:36.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.523+0000 7fed804c3b80 1 --2- 172.21.15.5:0/141876605 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627f13e26a0 0x5627f13ecbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:36.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.523+0000 7fed804c3b80 1 --2- 172.21.15.5:0/141876605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627f12654a0 0x5627f13ec140 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:36.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.523+0000 7fed804c3b80 1 --2- 172.21.15.5:0/141876605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627f134f160 0x5627f13ec680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:36.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.523+0000 7fed804c3b80 1 -- 172.21.15.5:0/141876605 >> 172.21.15.5:0/141876605 conn(0x5627f12c56a0 msgr2=0x5627f13dd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:49:36.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.527+0000 7fed4affd700 0 client.15804 ms_handle_reset on v2:172.21.15.157:6836/744863009 2024-05-09T00:49:36.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.528+0000 7fed804c3b80 1 -- 172.21.15.5:0/141876605 shutdown_connections 2024-05-09T00:49:36.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.529+0000 7fed804c3b80 1 -- 172.21.15.5:0/141876605 wait complete. 2024-05-09T00:49:36.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.531+0000 7fed804c3b80 20 client.15804 trim_cache size 0 max 16384 2024-05-09T00:49:36.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.571+0000 7fed804c3b80 1 Processor -- start 2024-05-09T00:49:36.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.575+0000 7fed804c3b80 1 -- start start 2024-05-09T00:49:36.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.576+0000 7fed804c3b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627f12654a0 0x5627f13e9d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:36.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.576+0000 7fed804c3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627f134f160 0x5627f13e7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:36.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.576+0000 7fed804c3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627f13e26a0 0x5627f13f80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:36.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.576+0000 7fed804c3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fed5800d530 con 0x5627f134f160 2024-05-09T00:49:36.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.576+0000 7fed804c3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fed5800f070 con 0x5627f13e26a0 2024-05-09T00:49:36.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.576+0000 7fed804c3b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fed5800dc00 con 0x5627f12654a0 2024-05-09T00:49:36.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.577+0000 7fed6bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627f13e26a0 0x5627f13f80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:36.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.577+0000 7fed6bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627f13e26a0 0x5627f13f80a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:55312/0 (socket says 172.21.15.5:55312) 2024-05-09T00:49:36.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.577+0000 7fed6bfff700 1 -- 172.21.15.5:0/3363273999 learned_addr learned my addr 172.21.15.5:0/3363273999 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:49:36.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.577+0000 7fed6b7fe700 1 --2- 172.21.15.5:0/3363273999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627f12654a0 0x5627f13e9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:36.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.578+0000 7fed6affd700 1 --2- 172.21.15.5:0/3363273999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627f134f160 0x5627f13e7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:36.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.579+0000 7fed6affd700 1 -- 172.21.15.5:0/3363273999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627f12654a0 msgr2=0x5627f13e9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:49:36.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.579+0000 7fed6affd700 1 --2- 172.21.15.5:0/3363273999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627f12654a0 0x5627f13e9d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:36.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.579+0000 7fed6affd700 1 -- 172.21.15.5:0/3363273999 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627f13e26a0 msgr2=0x5627f13f80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:49:36.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.579+0000 7fed6affd700 1 --2- 172.21.15.5:0/3363273999 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627f13e26a0 0x5627f13f80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:36.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.579+0000 7fed6affd700 1 -- 172.21.15.5:0/3363273999 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627f13d6f60 con 0x5627f134f160 2024-05-09T00:49:36.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.581+0000 7fed6affd700 1 --2- 172.21.15.5:0/3363273999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627f134f160 0x5627f13e7880 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fed6000a010 tx=0x7fed600094b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e07e1e256dfec15 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:49:36.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.584+0000 7fed517fa700 1 -- 172.21.15.5:0/3363273999 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6004d020 con 0x5627f134f160 2024-05-09T00:49:36.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.584+0000 7fed517fa700 1 -- 172.21.15.5:0/3363273999 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed60005040 con 0x5627f134f160 2024-05-09T00:49:36.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.584+0000 7fed517fa700 1 -- 172.21.15.5:0/3363273999 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed600049e0 con 0x5627f134f160 2024-05-09T00:49:36.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.586+0000 7fed804c3b80 1 -- 172.21.15.5:0/3363273999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627f134f160 msgr2=0x5627f13e7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:36.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.586+0000 7fed804c3b80 1 --2- 172.21.15.5:0/3363273999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627f134f160 0x5627f13e7880 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fed6000a010 tx=0x7fed600094b0 comp rx=0 tx=0).stop 2024-05-09T00:49:36.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.588+0000 7fed804c3b80 1 -- 172.21.15.5:0/3363273999 shutdown_connections 2024-05-09T00:49:36.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.588+0000 7fed804c3b80 1 --2- 172.21.15.5:0/3363273999 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627f13e26a0 0x5627f13f80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:36.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.588+0000 7fed804c3b80 1 --2- 172.21.15.5:0/3363273999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627f12654a0 0x5627f13e9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:36.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.588+0000 7fed804c3b80 1 --2- 172.21.15.5:0/3363273999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627f134f160 0x5627f13e7880 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:36.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.588+0000 7fed804c3b80 1 -- 172.21.15.5:0/3363273999 >> 172.21.15.5:0/3363273999 conn(0x5627f12c56a0 msgr2=0x5627f13e7170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:49:36.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.594+0000 7fed804c3b80 1 -- 172.21.15.5:0/3363273999 shutdown_connections 2024-05-09T00:49:36.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.595+0000 7fed804c3b80 1 -- 172.21.15.5:0/3363273999 wait complete. 2024-05-09T00:49:36.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.596+0000 7fed804c3b80 1 -- start start 2024-05-09T00:49:36.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.598+0000 7fed804c3b80 1 Processor -- start 2024-05-09T00:49:36.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.599+0000 7fed804c3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627f12654a0 0x5627f13f4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:36.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.599+0000 7fed804c3b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627f134f160 0x5627f13f4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:36.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.599+0000 7fed804c3b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627f13e26a0 0x5627f1402a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:36.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.600+0000 7fed804c3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fed5800e9a0 con 0x5627f12654a0 2024-05-09T00:49:36.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.600+0000 7fed804c3b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fed5800e2d0 con 0x5627f13e26a0 2024-05-09T00:49:36.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.600+0000 7fed804c3b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fed5800b4d0 con 0x5627f134f160 2024-05-09T00:49:36.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.600+0000 7fed804c3b80 10 client.0 _ll_register_callbacks cb 0x5627f11766c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:49:36.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.601+0000 7fed6bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627f13e26a0 0x5627f1402a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:36.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.602+0000 7fed6affd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627f134f160 0x5627f13f4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:36.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.602+0000 7fed6bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627f13e26a0 0x5627f1402a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:55328/0 (socket says 172.21.15.5:55328) 2024-05-09T00:49:36.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.602+0000 7fed6bfff700 1 -- 172.21.15.5:0/2677845925 learned_addr learned my addr 172.21.15.5:0/2677845925 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:49:36.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.602+0000 7fed6b7fe700 1 --2- 172.21.15.5:0/2677845925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627f12654a0 0x5627f13f4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:36.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.604+0000 7fed6bfff700 1 -- 172.21.15.5:0/2677845925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627f134f160 msgr2=0x5627f13f4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:49:36.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.604+0000 7fed6bfff700 1 --2- 172.21.15.5:0/2677845925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627f134f160 0x5627f13f4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:36.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.604+0000 7fed6bfff700 1 -- 172.21.15.5:0/2677845925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627f12654a0 msgr2=0x5627f13f4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:49:36.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.604+0000 7fed6bfff700 1 --2- 172.21.15.5:0/2677845925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627f12654a0 0x5627f13f4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:36.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.604+0000 7fed6bfff700 1 -- 172.21.15.5:0/2677845925 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627f1404470 con 0x5627f13e26a0 2024-05-09T00:49:36.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.605+0000 7fed6bfff700 1 --2- 172.21.15.5:0/2677845925 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627f13e26a0 0x5627f1402a50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fed64009fd0 tx=0x7fed64004f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d28cca10e81c8460 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:49:36.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.606+0000 7fed4affd700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-09T00:49:36.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.606+0000 7fed4affd700 1 -- 172.21.15.5:0/2677845925 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed640a93f0 con 0x5627f13e26a0 2024-05-09T00:49:36.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.606+0000 7fed4affd700 1 -- 172.21.15.5:0/2677845925 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fed640b1f40 con 0x5627f13e26a0 2024-05-09T00:49:36.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.610+0000 7fed804c3b80 1 -- 172.21.15.5:0/2677845925 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed60005040 con 0x5627f13e26a0 2024-05-09T00:49:36.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.613+0000 7fed4affd700 1 -- 172.21.15.5:0/2677845925 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fed640ad7a0 con 0x5627f13e26a0 2024-05-09T00:49:36.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.614+0000 7fed804c3b80 10 client.25445 fetch_fsmap learned FSMap version 29 2024-05-09T00:49:36.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.614+0000 7fed804c3b80 1 -- 172.21.15.5:0/2677845925 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5627f1465be0 con 0x5627f13e26a0 2024-05-09T00:49:36.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.623+0000 7fed4affd700 1 -- 172.21.15.5:0/2677845925 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed640b21c0 con 0x5627f13e26a0 2024-05-09T00:49:36.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.623+0000 7fed4affd700 1 -- 172.21.15.5:0/2677845925 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fed640a3b10 con 0x5627f13e26a0 2024-05-09T00:49:36.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.627+0000 7fed804c3b80 10 client.25445 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:49:36.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.627+0000 7fed804c3b80 10 client.25445 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-05-09T00:49:36.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.627+0000 7fed804c3b80 20 client.25445 populate_metadata read hostname 'smithi005' 2024-05-09T00:49:36.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.627+0000 7fed804c3b80 1 --2- 172.21.15.5:0/2677845925 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x5627f14661f0 0x5627f1468610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:36.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.627+0000 7fed804c3b80 4 client.25445 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:49:36.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.627+0000 7fed804c3b80 1 -- 172.21.15.5:0/2677845925 --> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fed60005040 con 0x5627f14661f0 2024-05-09T00:49:36.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.629+0000 7fed6b7fe700 1 --2- 172.21.15.5:0/2677845925 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x5627f14661f0 0x5627f1468610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:36.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.636+0000 7fed6b7fe700 1 --2- 172.21.15.5:0/2677845925 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x5627f14661f0 0x5627f1468610 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c0990ebefecd5642 server_cookie=34937cbd2b777edc in_seq=0 out_seq=0 2024-05-09T00:49:36.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.636+0000 7fed4affd700 10 client.25445 ms_handle_connect on v2:172.21.15.157:6836/744863009 2024-05-09T00:49:36.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.637+0000 7fed4affd700 1 -- 172.21.15.5:0/2677845925 <== mds.1 v2:172.21.15.157:6836/744863009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fed60005040 con 0x5627f14661f0 2024-05-09T00:49:36.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.637+0000 7fed4affd700 10 client.25445 handle_command_reply: tid=0 2024-05-09T00:49:36.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.637+0000 7fed804c3b80 1 client.25445 shutdown 2024-05-09T00:49:36.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.643+0000 7fed804c3b80 1 -- 172.21.15.5:0/2677845925 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627f13e26a0 msgr2=0x5627f1402a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:36.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.643+0000 7fed804c3b80 1 --2- 172.21.15.5:0/2677845925 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627f13e26a0 0x5627f1402a50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fed64009fd0 tx=0x7fed64004f90 comp rx=0 tx=0).stop 2024-05-09T00:49:36.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.645+0000 7fed804c3b80 1 -- 172.21.15.5:0/2677845925 shutdown_connections 2024-05-09T00:49:36.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.645+0000 7fed804c3b80 1 --2- 172.21.15.5:0/2677845925 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x5627f14661f0 0x5627f1468610 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:36.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.645+0000 7fed804c3b80 1 --2- 172.21.15.5:0/2677845925 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5627f13e26a0 0x5627f1402a50 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:36.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.645+0000 7fed4affd700 0 client.25445 ms_handle_reset on v2:172.21.15.157:6836/744863009 2024-05-09T00:49:36.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.645+0000 7fed804c3b80 1 --2- 172.21.15.5:0/2677845925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627f12654a0 0x5627f13f4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:36.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.645+0000 7fed804c3b80 1 --2- 172.21.15.5:0/2677845925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627f134f160 0x5627f13f4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:36.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.645+0000 7fed804c3b80 1 -- 172.21.15.5:0/2677845925 >> 172.21.15.5:0/2677845925 conn(0x5627f12c56a0 msgr2=0x5627f13e7170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:49:36.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.646+0000 7fed804c3b80 1 -- 172.21.15.5:0/2677845925 shutdown_connections 2024-05-09T00:49:36.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.646+0000 7fed804c3b80 1 -- 172.21.15.5:0/2677845925 wait complete. 2024-05-09T00:49:36.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.647+0000 7fed804c3b80 20 client.25445 trim_cache size 0 max 16384 2024-05-09T00:49:36.651 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-09T00:49:36.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.652+0000 7fed71aad700 1 -- 172.21.15.5:0/1626167041 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fed540b2860 msgr2=0x7fed540b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:36.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.652+0000 7fed71aad700 1 --2- 172.21.15.5:0/1626167041 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fed540b2860 0x7fed540b4d70 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fed5c015740 tx=0x7fed5c015a50 comp rx=0 tx=0).stop 2024-05-09T00:49:36.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.652+0000 7fed71aad700 1 -- 172.21.15.5:0/1626167041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed6c11d930 msgr2=0x7fed6c117c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:36.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.652+0000 7fed71aad700 1 --2- 172.21.15.5:0/1626167041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed6c11d930 0x7fed6c117c20 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fed6400ef90 tx=0x7fed6400c450 comp rx=0 tx=0).stop 2024-05-09T00:49:36.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.653+0000 7fed71aad700 1 -- 172.21.15.5:0/1626167041 shutdown_connections 2024-05-09T00:49:36.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.653+0000 7fed71aad700 1 --2- 172.21.15.5:0/1626167041 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fed540b2860 0x7fed540b4d70 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:36.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.653+0000 7fed71aad700 1 --2- 172.21.15.5:0/1626167041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed6c11d930 0x7fed6c117c20 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:36.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.653+0000 7fed71aad700 1 --2- 172.21.15.5:0/1626167041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed6c10a930 0x7fed6c11d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:36.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.653+0000 7fed71aad700 1 --2- 172.21.15.5:0/1626167041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed6c10b280 0x7fed6c117530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:36.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.653+0000 7fed71aad700 1 -- 172.21.15.5:0/1626167041 >> 172.21.15.5:0/1626167041 conn(0x7fed6c06f4d0 msgr2=0x7fed6c11dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:49:36.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.654+0000 7fed71aad700 1 -- 172.21.15.5:0/1626167041 shutdown_connections 2024-05-09T00:49:36.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:36.654+0000 7fed71aad700 1 -- 172.21.15.5:0/1626167041 wait complete. 2024-05-09T00:49:37.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:49:37 smithi157 ceph-mon[34294]: pgmap v469: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 981 MiB/s wr, 3.07k op/s 2024-05-09T00:49:37.529 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:49:37 smithi005 ceph-mon[30365]: pgmap v469: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 981 MiB/s wr, 3.07k op/s 2024-05-09T00:49:37.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:49:37 smithi131 ceph-mon[34547]: pgmap v469: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 981 MiB/s wr, 3.07k op/s 2024-05-09T00:49:38.525 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T00:49:39.316 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:49:39.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:49:39 smithi005 ceph-mon[30365]: pgmap v470: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 797 MiB/s wr, 2.26k op/s 2024-05-09T00:49:39.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:49:39 smithi131 ceph-mon[34547]: pgmap v470: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 797 MiB/s wr, 2.26k op/s 2024-05-09T00:49:39.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:49:39 smithi157 ceph-mon[34294]: pgmap v470: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 797 MiB/s wr, 2.26k op/s 2024-05-09T00:49:41.544 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:49:41 smithi005 ceph-mon[30365]: pgmap v471: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 986 MiB/s wr, 2.52k op/s 2024-05-09T00:49:41.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:49:41 smithi131 ceph-mon[34547]: pgmap v471: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 986 MiB/s wr, 2.52k op/s 2024-05-09T00:49:41.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:49:41 smithi157 ceph-mon[34294]: pgmap v471: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 986 MiB/s wr, 2.52k op/s 2024-05-09T00:49:43.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:49:43 smithi005 ceph-mon[30365]: pgmap v472: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 840 MiB/s wr, 2.19k op/s 2024-05-09T00:49:43.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:49:43 smithi131 ceph-mon[34547]: pgmap v472: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 840 MiB/s wr, 2.19k op/s 2024-05-09T00:49:43.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:49:43 smithi157 ceph-mon[34294]: pgmap v472: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 840 MiB/s wr, 2.19k op/s 2024-05-09T00:49:44.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.092+0000 7fe7b759e640 1 -- 172.21.15.5:0/951042437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7b81103f0 msgr2=0x7fe7b8112810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:44.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.092+0000 7fe7b759e640 1 --2- 172.21.15.5:0/951042437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7b81103f0 0x7fe7b8112810 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8009ca0 tx=0x7fe7a802a8f0 comp rx=0 tx=0).stop 2024-05-09T00:49:44.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.095+0000 7fe7b759e640 1 -- 172.21.15.5:0/951042437 shutdown_connections 2024-05-09T00:49:44.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.095+0000 7fe7b759e640 1 --2- 172.21.15.5:0/951042437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7b81103f0 0x7fe7b8112810 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:44.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.095+0000 7fe7b759e640 1 --2- 172.21.15.5:0/951042437 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7b810b6f0 0x7fe7b810bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:44.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.095+0000 7fe7b759e640 1 --2- 172.21.15.5:0/951042437 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7b810a510 0x7fe7b810b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:44.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.095+0000 7fe7b759e640 1 -- 172.21.15.5:0/951042437 >> 172.21.15.5:0/951042437 conn(0x7fe7b80708d0 msgr2=0x7fe7b8072d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:49:44.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.106+0000 7fe7b759e640 1 -- 172.21.15.5:0/951042437 shutdown_connections 2024-05-09T00:49:44.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.106+0000 7fe7b759e640 1 -- 172.21.15.5:0/951042437 wait complete. 2024-05-09T00:49:44.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.108+0000 7fe7b759e640 1 Processor -- start 2024-05-09T00:49:44.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.108+0000 7fe7b759e640 1 -- start start 2024-05-09T00:49:44.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.109+0000 7fe7b759e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7b810a510 0x7fe7b81a9130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:44.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.109+0000 7fe7b759e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7b810b6f0 0x7fe7b81a9670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:44.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.109+0000 7fe7b759e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7b81103f0 0x7fe7b81b0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:44.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.109+0000 7fe7b759e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b81160b0 con 0x7fe7b810b6f0 2024-05-09T00:49:44.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.109+0000 7fe7b759e640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b8115f30 con 0x7fe7b81103f0 2024-05-09T00:49:44.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.109+0000 7fe7b759e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b8116230 con 0x7fe7b810a510 2024-05-09T00:49:44.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.110+0000 7fe7b659c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7b810a510 0x7fe7b81a9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:44.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.110+0000 7fe7b659c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7b810a510 0x7fe7b81a9130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:47398/0 (socket says 172.21.15.5:47398) 2024-05-09T00:49:44.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.110+0000 7fe7b659c640 1 -- 172.21.15.5:0/1924688466 learned_addr learned my addr 172.21.15.5:0/1924688466 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:49:44.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.110+0000 7fe7b6d9d640 1 --2- 172.21.15.5:0/1924688466 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7b81103f0 0x7fe7b81b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:44.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.112+0000 7fe7b5d9b640 1 --2- 172.21.15.5:0/1924688466 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7b810b6f0 0x7fe7b81a9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:44.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.116+0000 7fe7b659c640 1 -- 172.21.15.5:0/1924688466 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7b81103f0 msgr2=0x7fe7b81b0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:44.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.116+0000 7fe7b659c640 1 --2- 172.21.15.5:0/1924688466 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7b81103f0 0x7fe7b81b0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:44.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.116+0000 7fe7b659c640 1 -- 172.21.15.5:0/1924688466 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7b810b6f0 msgr2=0x7fe7b81a9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:44.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.116+0000 7fe7b659c640 1 --2- 172.21.15.5:0/1924688466 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7b810b6f0 0x7fe7b81a9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:44.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.116+0000 7fe7b659c640 1 -- 172.21.15.5:0/1924688466 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7b81b0c70 con 0x7fe7b810a510 2024-05-09T00:49:44.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.119+0000 7fe7b659c640 1 --2- 172.21.15.5:0/1924688466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7b810a510 0x7fe7b81a9130 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0006fd0 tx=0x7fe7b000b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:49:44.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.121+0000 7fe7af7fe640 1 -- 172.21.15.5:0/1924688466 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b0015020 con 0x7fe7b810a510 2024-05-09T00:49:44.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.121+0000 7fe7af7fe640 1 -- 172.21.15.5:0/1924688466 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7b0011ce0 con 0x7fe7b810a510 2024-05-09T00:49:44.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.121+0000 7fe7af7fe640 1 -- 172.21.15.5:0/1924688466 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b0010d10 con 0x7fe7b810a510 2024-05-09T00:49:44.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.122+0000 7fe7b759e640 1 -- 172.21.15.5:0/1924688466 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7b81b0f60 con 0x7fe7b810a510 2024-05-09T00:49:44.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.122+0000 7fe7b759e640 1 -- 172.21.15.5:0/1924688466 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7b81b1310 con 0x7fe7b810a510 2024-05-09T00:49:44.141 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.133+0000 7fe7c39b9740 1 Processor -- start 2024-05-09T00:49:44.141 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.138+0000 7fe7c39b9740 1 -- start start 2024-05-09T00:49:44.141 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.139+0000 7fe7c39b9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55699f6b91b0 0x55699fa5d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:44.141 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.140+0000 7fe7c39b9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55699f6902f0 0x55699fa5dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:44.141 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.140+0000 7fe7c39b9740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55699f6984d0 0x55699fa64b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:44.141 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.140+0000 7fe7c39b9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55699f9c17c0 con 0x55699f6902f0 2024-05-09T00:49:44.141 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.140+0000 7fe7c39b9740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55699f95d0e0 con 0x55699f6984d0 2024-05-09T00:49:44.141 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.140+0000 7fe7c39b9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55699f9bd780 con 0x55699f6b91b0 2024-05-09T00:49:44.141 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.140+0000 7fe7af7fe640 1 -- 172.21.15.5:0/1924688466 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fe7b0018020 con 0x7fe7b810a510 2024-05-09T00:49:44.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.141+0000 7fe7b6d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55699f6b91b0 0x55699fa5d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:44.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.142+0000 7fe7b6d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55699f6b91b0 0x55699fa5d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:47414/0 (socket says 172.21.15.5:47414) 2024-05-09T00:49:44.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.142+0000 7fe7b6d9d640 1 -- 172.21.15.5:0/980755031 learned_addr learned my addr 172.21.15.5:0/980755031 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:49:44.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.142+0000 7fe7af7fe640 1 --2- 172.21.15.5:0/1924688466 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe7a40b2930 0x7fe7a40b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:44.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.143+0000 7fe7b659c640 1 --2- 172.21.15.5:0/980755031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55699f6902f0 0x55699fa5dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:44.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.143+0000 7fe7b6d9d640 1 -- 172.21.15.5:0/980755031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55699f6984d0 msgr2=0x55699fa64b40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:49:44.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.143+0000 7fe7b6d9d640 1 --2- 172.21.15.5:0/980755031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55699f6984d0 0x55699fa64b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:44.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.143+0000 7fe7b6d9d640 1 -- 172.21.15.5:0/980755031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55699f6902f0 msgr2=0x55699fa5dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:49:44.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.143+0000 7fe7b6d9d640 1 --2- 172.21.15.5:0/980755031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55699f6902f0 0x55699fa5dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:44.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.143+0000 7fe7b6d9d640 1 -- 172.21.15.5:0/980755031 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55699f9867c0 con 0x55699f6b91b0 2024-05-09T00:49:44.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.144+0000 7fe7b659c640 1 --2- 172.21.15.5:0/980755031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55699f6902f0 0x55699fa5dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:49:44.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.144+0000 7fe7af7fe640 1 -- 172.21.15.5:0/1924688466 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fe7b00e17c0 con 0x7fe7b810a510 2024-05-09T00:49:44.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.144+0000 7fe7b6d9d640 1 --2- 172.21.15.5:0/980755031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55699f6b91b0 0x55699fa5d6c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8009bf0 tx=0x7fe7a8031580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38881e5b7ee2563c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:49:44.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.147+0000 7fe7b6d9d640 1 --2- 172.21.15.5:0/1924688466 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe7a40b2930 0x7fe7a40b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:44.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.152+0000 7fe7b6d9d640 1 --2- 172.21.15.5:0/1924688466 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe7a40b2930 0x7fe7a40b4e40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8048000 tx=0x7fe7a802f8b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:49:44.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.160+0000 7fe7acff9640 1 -- 172.21.15.5:0/980755031 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a804b070 con 0x55699f6b91b0 2024-05-09T00:49:44.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.160+0000 7fe7acff9640 1 -- 172.21.15.5:0/980755031 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7a802d070 con 0x55699f6b91b0 2024-05-09T00:49:44.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.163+0000 7fe7c39b9740 1 -- 172.21.15.5:0/980755031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55699f6b91b0 msgr2=0x55699fa5d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:44.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.163+0000 7fe7c39b9740 1 --2- 172.21.15.5:0/980755031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55699f6b91b0 0x55699fa5d6c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8009bf0 tx=0x7fe7a8031580 comp rx=0 tx=0).stop 2024-05-09T00:49:44.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.163+0000 7fe7c39b9740 1 -- 172.21.15.5:0/980755031 shutdown_connections 2024-05-09T00:49:44.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.163+0000 7fe7c39b9740 1 --2- 172.21.15.5:0/980755031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55699f6984d0 0x55699fa64b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:44.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.163+0000 7fe7c39b9740 1 --2- 172.21.15.5:0/980755031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55699f6902f0 0x55699fa5dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:44.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.163+0000 7fe7c39b9740 1 --2- 172.21.15.5:0/980755031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55699f6b91b0 0x55699fa5d6c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:44.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.163+0000 7fe7c39b9740 1 -- 172.21.15.5:0/980755031 >> 172.21.15.5:0/980755031 conn(0x55699f6ad300 msgr2=0x55699fa59050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:49:44.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.163+0000 7fe7c39b9740 1 -- 172.21.15.5:0/980755031 shutdown_connections 2024-05-09T00:49:44.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.163+0000 7fe7c39b9740 1 -- 172.21.15.5:0/980755031 wait complete. 2024-05-09T00:49:44.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.165+0000 7fe7c39b9740 1 -- start start 2024-05-09T00:49:44.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.166+0000 7fe7c39b9740 1 Processor -- start 2024-05-09T00:49:44.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.168+0000 7fe7c39b9740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55699f6902f0 0x55699fa67250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:44.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.168+0000 7fe7c39b9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55699f6984d0 0x55699fa67790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:44.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.168+0000 7fe7c39b9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55699f677d80 0x55699fa6e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:44.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.168+0000 7fe7b5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55699f6984d0 0x55699fa67790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:44.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.169+0000 7fe7b5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55699f6984d0 0x55699fa67790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50866/0 (socket says 172.21.15.5:50866) 2024-05-09T00:49:44.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.169+0000 7fe7b5d9b640 1 -- 172.21.15.5:0/1368165229 learned_addr learned my addr 172.21.15.5:0/1368165229 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:49:44.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.169+0000 7fe7c39b9740 1 -- 172.21.15.5:0/1368165229 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55699fa65580 con 0x55699f6984d0 2024-05-09T00:49:44.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.169+0000 7fe7c39b9740 1 -- 172.21.15.5:0/1368165229 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55699f9489c0 con 0x55699f6902f0 2024-05-09T00:49:44.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.169+0000 7fe7c39b9740 1 -- 172.21.15.5:0/1368165229 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55699fa65700 con 0x55699f677d80 2024-05-09T00:49:44.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.169+0000 7fe7c39b9740 10 client.0 _ll_register_callbacks cb 0x55699f975020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:49:44.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.170+0000 7fe7b659c640 1 --2- 172.21.15.5:0/1368165229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55699f6902f0 0x55699fa67250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:44.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.172+0000 7fe7b6d9d640 1 --2- 172.21.15.5:0/1368165229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55699f677d80 0x55699fa6e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:44.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.173+0000 7fe7b5d9b640 1 -- 172.21.15.5:0/1368165229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55699f677d80 msgr2=0x55699fa6e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:49:44.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.173+0000 7fe7b5d9b640 1 --2- 172.21.15.5:0/1368165229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55699f677d80 0x55699fa6e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:44.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.173+0000 7fe7b5d9b640 1 -- 172.21.15.5:0/1368165229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55699f6902f0 msgr2=0x55699fa67250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:49:44.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.173+0000 7fe7b5d9b640 1 --2- 172.21.15.5:0/1368165229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55699f6902f0 0x55699fa67250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:44.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.173+0000 7fe7b5d9b640 1 -- 172.21.15.5:0/1368165229 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55699f96f1f0 con 0x55699f6984d0 2024-05-09T00:49:44.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.184+0000 7fe7b5d9b640 1 --2- 172.21.15.5:0/1368165229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55699f6984d0 0x55699fa67790 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fe7a000e480 tx=0x7fe7a000e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=515afb95130b521e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:49:44.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.184+0000 7fe79dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:49:44.189 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.186+0000 7fe79dffb640 1 -- 172.21.15.5:0/1368165229 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a0017070 con 0x55699f6984d0 2024-05-09T00:49:44.189 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.186+0000 7fe79dffb640 1 -- 172.21.15.5:0/1368165229 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fe7a0013680 con 0x55699f6984d0 2024-05-09T00:49:44.189 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.186+0000 7fe7c39b9740 1 -- 172.21.15.5:0/1368165229 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55699f9cf3d0 con 0x55699f6984d0 2024-05-09T00:49:44.189 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.187+0000 7fe79dffb640 1 -- 172.21.15.5:0/1368165229 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe7a0021050 con 0x55699f6984d0 2024-05-09T00:49:44.189 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.188+0000 7fe7c39b9740 10 client.15819 fetch_fsmap learned FSMap version 29 2024-05-09T00:49:44.189 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.188+0000 7fe7c39b9740 1 -- 172.21.15.5:0/1368165229 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55699f724790 con 0x55699f6984d0 2024-05-09T00:49:44.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.202+0000 7fe79dffb640 1 -- 172.21.15.5:0/1368165229 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55699f9cf3d0 con 0x55699f6984d0 2024-05-09T00:49:44.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.202+0000 7fe79dffb640 1 -- 172.21.15.5:0/1368165229 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fe7a001ca10 con 0x55699f6984d0 2024-05-09T00:49:44.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.202+0000 7fe7c39b9740 10 client.15819 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:49:44.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.202+0000 7fe7c39b9740 10 client.15819 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:49:44.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.202+0000 7fe7c39b9740 20 client.15819 populate_metadata read hostname 'smithi005' 2024-05-09T00:49:44.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.202+0000 7fe7c39b9740 1 --2- 172.21.15.5:0/1368165229 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55699f64d1c0 0x55699fa74cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:44.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.202+0000 7fe7c39b9740 4 client.15819 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:49:44.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.202+0000 7fe7c39b9740 1 -- 172.21.15.5:0/1368165229 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55699f9c9240 con 0x55699f64d1c0 2024-05-09T00:49:44.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.208+0000 7fe7b659c640 1 --2- 172.21.15.5:0/1368165229 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55699f64d1c0 0x55699fa74cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:44.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.217+0000 7fe7b659c640 1 --2- 172.21.15.5:0/1368165229 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55699f64d1c0 0x55699fa74cb0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5eb9dae4fcbddcf5 server_cookie=4087acf114b89974 in_seq=0 out_seq=0 2024-05-09T00:49:44.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.217+0000 7fe79dffb640 10 client.15819 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:49:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.225+0000 7fe79dffb640 1 -- 172.21.15.5:0/1368165229 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55699f9c9240 con 0x55699f64d1c0 2024-05-09T00:49:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.225+0000 7fe79dffb640 10 client.15819 handle_command_reply: tid=0 2024-05-09T00:49:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.225+0000 7fe7c39b9740 1 client.15819 shutdown 2024-05-09T00:49:44.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.228+0000 7fe7c39b9740 1 -- 172.21.15.5:0/1368165229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55699f6984d0 msgr2=0x55699fa67790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:44.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.228+0000 7fe7c39b9740 1 --2- 172.21.15.5:0/1368165229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55699f6984d0 0x55699fa67790 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fe7a000e480 tx=0x7fe7a000e940 comp rx=0 tx=0).stop 2024-05-09T00:49:44.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.228+0000 7fe7c39b9740 1 -- 172.21.15.5:0/1368165229 shutdown_connections 2024-05-09T00:49:44.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.228+0000 7fe7c39b9740 1 --2- 172.21.15.5:0/1368165229 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55699f64d1c0 0x55699fa74cb0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:44.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.228+0000 7fe7c39b9740 1 --2- 172.21.15.5:0/1368165229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55699f677d80 0x55699fa6e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:44.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.228+0000 7fe7c39b9740 1 --2- 172.21.15.5:0/1368165229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55699f6984d0 0x55699fa67790 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:44.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.228+0000 7fe7c39b9740 1 --2- 172.21.15.5:0/1368165229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55699f6902f0 0x55699fa67250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:44.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.228+0000 7fe7c39b9740 1 -- 172.21.15.5:0/1368165229 >> 172.21.15.5:0/1368165229 conn(0x55699f6ad300 msgr2=0x55699fa58290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:49:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.230+0000 7fe79dffb640 0 client.15819 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:49:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.230+0000 7fe7c39b9740 1 -- 172.21.15.5:0/1368165229 shutdown_connections 2024-05-09T00:49:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.230+0000 7fe7c39b9740 1 -- 172.21.15.5:0/1368165229 wait complete. 2024-05-09T00:49:44.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.234+0000 7fe7c39b9740 20 client.15819 trim_cache size 0 max 16384 2024-05-09T00:49:44.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.308+0000 7fe7c39b9740 1 Processor -- start 2024-05-09T00:49:44.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.308+0000 7fe7c39b9740 1 -- start start 2024-05-09T00:49:44.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.309+0000 7fe7c39b9740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55699f677d80 0x55699fa590d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:44.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.309+0000 7fe7c39b9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55699f6902f0 0x55699fa6e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:44.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.310+0000 7fe7c39b9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55699f6984d0 0x55699fa6ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:44.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.310+0000 7fe7c39b9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b81ba240 con 0x55699f6902f0 2024-05-09T00:49:44.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.310+0000 7fe7c39b9740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b81bbd80 con 0x55699f677d80 2024-05-09T00:49:44.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.310+0000 7fe7c39b9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b81ba910 con 0x55699f6984d0 2024-05-09T00:49:44.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.313+0000 7fe7b6d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55699f6984d0 0x55699fa6ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:44.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.313+0000 7fe7b659c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55699f677d80 0x55699fa590d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:44.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.313+0000 7fe7b659c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55699f677d80 0x55699fa590d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:56364/0 (socket says 172.21.15.5:56364) 2024-05-09T00:49:44.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.313+0000 7fe7b659c640 1 -- 172.21.15.5:0/3539798678 learned_addr learned my addr 172.21.15.5:0/3539798678 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:49:44.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.316+0000 7fe7b5d9b640 1 --2- 172.21.15.5:0/3539798678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55699f6902f0 0x55699fa6e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:44.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.318+0000 7fe7b5d9b640 1 -- 172.21.15.5:0/3539798678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55699f6984d0 msgr2=0x55699fa6ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:49:44.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.318+0000 7fe7b5d9b640 1 --2- 172.21.15.5:0/3539798678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55699f6984d0 0x55699fa6ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:44.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.318+0000 7fe7b5d9b640 1 -- 172.21.15.5:0/3539798678 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55699f677d80 msgr2=0x55699fa590d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:49:44.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.318+0000 7fe7b5d9b640 1 --2- 172.21.15.5:0/3539798678 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55699f677d80 0x55699fa590d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:44.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.318+0000 7fe7b5d9b640 1 -- 172.21.15.5:0/3539798678 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55699f79b5f0 con 0x55699f6902f0 2024-05-09T00:49:44.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.319+0000 7fe7b5d9b640 1 --2- 172.21.15.5:0/3539798678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55699f6902f0 0x55699fa6e6b0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe7a001cde0 tx=0x7fe7a0012620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a6ba2f85783fabd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:49:44.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.321+0000 7fe79ffff640 1 -- 172.21.15.5:0/3539798678 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a0017070 con 0x55699f6902f0 2024-05-09T00:49:44.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.321+0000 7fe79ffff640 1 -- 172.21.15.5:0/3539798678 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7a0012c40 con 0x55699f6902f0 2024-05-09T00:49:44.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.323+0000 7fe79ffff640 1 -- 172.21.15.5:0/3539798678 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a001c440 con 0x55699f6902f0 2024-05-09T00:49:44.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.325+0000 7fe7c39b9740 1 -- 172.21.15.5:0/3539798678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55699f6902f0 msgr2=0x55699fa6e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:44.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.325+0000 7fe7c39b9740 1 --2- 172.21.15.5:0/3539798678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55699f6902f0 0x55699fa6e6b0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe7a001cde0 tx=0x7fe7a0012620 comp rx=0 tx=0).stop 2024-05-09T00:49:44.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.328+0000 7fe7c39b9740 1 -- 172.21.15.5:0/3539798678 shutdown_connections 2024-05-09T00:49:44.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.328+0000 7fe7c39b9740 1 --2- 172.21.15.5:0/3539798678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55699f6984d0 0x55699fa6ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:44.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.328+0000 7fe7c39b9740 1 --2- 172.21.15.5:0/3539798678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55699f6902f0 0x55699fa6e6b0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:44.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.328+0000 7fe7c39b9740 1 --2- 172.21.15.5:0/3539798678 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55699f677d80 0x55699fa590d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:44.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.328+0000 7fe7c39b9740 1 -- 172.21.15.5:0/3539798678 >> 172.21.15.5:0/3539798678 conn(0x55699f6ad300 msgr2=0x55699fa589c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:49:44.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.332+0000 7fe7c39b9740 1 -- 172.21.15.5:0/3539798678 shutdown_connections 2024-05-09T00:49:44.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.335+0000 7fe7c39b9740 1 -- 172.21.15.5:0/3539798678 wait complete. 2024-05-09T00:49:44.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.335+0000 7fe7c39b9740 1 -- start start 2024-05-09T00:49:44.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.335+0000 7fe7c39b9740 1 Processor -- start 2024-05-09T00:49:44.342 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.340+0000 7fe7c39b9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55699f677d80 0x55699fa69fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:44.342 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.340+0000 7fe7c39b9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55699f6902f0 0x55699fa6a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:44.343 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.340+0000 7fe7c39b9740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55699f6984d0 0x55699fa6d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:44.343 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.340+0000 7fe7c39b9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b81bb6b0 con 0x55699f6902f0 2024-05-09T00:49:44.343 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.340+0000 7fe7c39b9740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b81bafe0 con 0x55699f6984d0 2024-05-09T00:49:44.343 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.340+0000 7fe7c39b9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b81b8220 con 0x55699f677d80 2024-05-09T00:49:44.343 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.340+0000 7fe7c39b9740 10 client.0 _ll_register_callbacks cb 0x7fe7a0009420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:49:44.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.344+0000 7fe7b6d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55699f6984d0 0x55699fa6d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:44.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.345+0000 7fe7b5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55699f6902f0 0x55699fa6a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:44.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.345+0000 7fe7b5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55699f6902f0 0x55699fa6a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50896/0 (socket says 172.21.15.5:50896) 2024-05-09T00:49:44.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.345+0000 7fe7b5d9b640 1 -- 172.21.15.5:0/2280865453 learned_addr learned my addr 172.21.15.5:0/2280865453 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:49:44.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.347+0000 7fe7b659c640 1 --2- 172.21.15.5:0/2280865453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55699f677d80 0x55699fa69fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:44.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.348+0000 7fe7b5d9b640 1 -- 172.21.15.5:0/2280865453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55699f677d80 msgr2=0x55699fa69fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:49:44.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.348+0000 7fe7b5d9b640 1 --2- 172.21.15.5:0/2280865453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55699f677d80 0x55699fa69fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:44.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.348+0000 7fe7b5d9b640 1 -- 172.21.15.5:0/2280865453 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55699f6984d0 msgr2=0x55699fa6d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:49:44.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.348+0000 7fe7b5d9b640 1 --2- 172.21.15.5:0/2280865453 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55699f6984d0 0x55699fa6d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:44.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.348+0000 7fe7b5d9b640 1 -- 172.21.15.5:0/2280865453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55699f985ed0 con 0x55699f6902f0 2024-05-09T00:49:44.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.349+0000 7fe7b5d9b640 1 --2- 172.21.15.5:0/2280865453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55699f6902f0 0x55699fa6a520 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0001eb0 tx=0x7fe7a00047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a1050ad02018e0c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:49:44.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.350+0000 7fe79dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:49:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.351+0000 7fe79dffb640 1 -- 172.21.15.5:0/2280865453 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a0033060 con 0x55699f6902f0 2024-05-09T00:49:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.351+0000 7fe7c39b9740 1 -- 172.21.15.5:0/2280865453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7a0012c40 con 0x55699f6902f0 2024-05-09T00:49:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.351+0000 7fe79dffb640 1 -- 172.21.15.5:0/2280865453 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fe7a001c440 con 0x55699f6902f0 2024-05-09T00:49:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.352+0000 7fe79dffb640 1 -- 172.21.15.5:0/2280865453 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe7a002a070 con 0x55699f6902f0 2024-05-09T00:49:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.352+0000 7fe7c39b9740 10 client.15825 fetch_fsmap learned FSMap version 29 2024-05-09T00:49:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.352+0000 7fe7c39b9740 1 -- 172.21.15.5:0/2280865453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55699fa6ea70 con 0x55699f6902f0 2024-05-09T00:49:44.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.355+0000 7fe79dffb640 1 -- 172.21.15.5:0/2280865453 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7a0012c40 con 0x55699f6902f0 2024-05-09T00:49:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.355+0000 7fe79dffb640 1 -- 172.21.15.5:0/2280865453 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fe7a0013b10 con 0x55699f6902f0 2024-05-09T00:49:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.355+0000 7fe7c39b9740 10 client.15825 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:49:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.355+0000 7fe7c39b9740 10 client.15825 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:49:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.355+0000 7fe7c39b9740 20 client.15825 populate_metadata read hostname 'smithi005' 2024-05-09T00:49:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.355+0000 7fe7c39b9740 1 --2- 172.21.15.5:0/2280865453 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55699faee210 0x55699faf0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.355+0000 7fe7c39b9740 4 client.15825 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:49:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.356+0000 7fe7c39b9740 1 -- 172.21.15.5:0/2280865453 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55699f944350 con 0x55699faee210 2024-05-09T00:49:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.357+0000 7fe7b659c640 1 --2- 172.21.15.5:0/2280865453 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55699faee210 0x55699faf0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:44.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.363+0000 7fe79dffb640 10 client.15825 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:49:44.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:44.363+0000 7fe7b659c640 1 --2- 172.21.15.5:0/2280865453 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55699faee210 0x55699faf0630 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffc374f1f35b7c70 server_cookie=dac38ce14da2539 in_seq=0 out_seq=0 2024-05-09T00:49:45.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.375+0000 7ff1ebfff700 1 -- 172.21.15.5:0/3246677843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1ec1032c0 msgr2=0x7ff1ec1036d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:45.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.375+0000 7ff1ebfff700 1 --2- 172.21.15.5:0/3246677843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1ec1032c0 0x7ff1ec1036d0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc009990 tx=0x7ff1dc009ca0 comp rx=0 tx=0).stop 2024-05-09T00:49:45.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.379+0000 7ff1ebfff700 1 -- 172.21.15.5:0/3246677843 shutdown_connections 2024-05-09T00:49:45.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.379+0000 7ff1ebfff700 1 --2- 172.21.15.5:0/3246677843 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff1ec108540 0x7ff1ec10fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:45.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.379+0000 7ff1ebfff700 1 --2- 172.21.15.5:0/3246677843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1ec1032c0 0x7ff1ec1036d0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:45.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.379+0000 7ff1ebfff700 1 --2- 172.21.15.5:0/3246677843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1ec107bf0 0x7ff1ec108000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:45.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.379+0000 7ff1ebfff700 1 -- 172.21.15.5:0/3246677843 >> 172.21.15.5:0/3246677843 conn(0x7ff1ec078bf0 msgr2=0x7ff1ec079030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:49:45.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.383+0000 7ff1ebfff700 1 -- 172.21.15.5:0/3246677843 shutdown_connections 2024-05-09T00:49:45.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.384+0000 7ff1ebfff700 1 -- 172.21.15.5:0/3246677843 wait complete. 2024-05-09T00:49:45.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.385+0000 7ff1ebfff700 1 Processor -- start 2024-05-09T00:49:45.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.388+0000 7ff1ebfff700 1 -- start start 2024-05-09T00:49:45.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.389+0000 7ff1ebfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1ec1032c0 0x7ff1ec118b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:45.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.389+0000 7ff1ebfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1ec107bf0 0x7ff1ec119080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:45.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.389+0000 7ff1ebfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff1ec108540 0x7ff1ec11e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:45.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.389+0000 7ff1ebfff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ec112200 con 0x7ff1ec1032c0 2024-05-09T00:49:45.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.389+0000 7ff1ebfff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ec112080 con 0x7ff1ec108540 2024-05-09T00:49:45.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.389+0000 7ff1ebfff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ec112380 con 0x7ff1ec107bf0 2024-05-09T00:49:45.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.391+0000 7ff1ea7fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1ec107bf0 0x7ff1ec119080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:45.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.392+0000 7ff1ea7fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1ec107bf0 0x7ff1ec119080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:47448/0 (socket says 172.21.15.5:47448) 2024-05-09T00:49:45.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.392+0000 7ff1ea7fc700 1 -- 172.21.15.5:0/2817137694 learned_addr learned my addr 172.21.15.5:0/2817137694 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:49:45.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.393+0000 7ff1eaffd700 1 --2- 172.21.15.5:0/2817137694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1ec1032c0 0x7ff1ec118b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:45.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.393+0000 7ff1eb7fe700 1 --2- 172.21.15.5:0/2817137694 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff1ec108540 0x7ff1ec11e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:45.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.394+0000 7ff1eaffd700 1 -- 172.21.15.5:0/2817137694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1ec107bf0 msgr2=0x7ff1ec119080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:45.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.394+0000 7ff1eaffd700 1 --2- 172.21.15.5:0/2817137694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1ec107bf0 0x7ff1ec119080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:45.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.394+0000 7ff1eaffd700 1 -- 172.21.15.5:0/2817137694 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff1ec108540 msgr2=0x7ff1ec11e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:45.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.394+0000 7ff1eaffd700 1 --2- 172.21.15.5:0/2817137694 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff1ec108540 0x7ff1ec11e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:45.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.394+0000 7ff1eaffd700 1 -- 172.21.15.5:0/2817137694 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1ec113ce0 con 0x7ff1ec1032c0 2024-05-09T00:49:45.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.395+0000 7ff1eaffd700 1 --2- 172.21.15.5:0/2817137694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1ec1032c0 0x7ff1ec118b20 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc000c00 tx=0x7ff1dc0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:49:45.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.397+0000 7ff1d3fff700 1 -- 172.21.15.5:0/2817137694 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc026070 con 0x7ff1ec1032c0 2024-05-09T00:49:45.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.398+0000 7ff1ebfff700 1 -- 172.21.15.5:0/2817137694 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1ec113f70 con 0x7ff1ec1032c0 2024-05-09T00:49:45.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.398+0000 7ff1ebfff700 1 -- 172.21.15.5:0/2817137694 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1ec114450 con 0x7ff1ec1032c0 2024-05-09T00:49:45.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.403+0000 7ff1d3fff700 1 -- 172.21.15.5:0/2817137694 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1dc003f40 con 0x7ff1ec1032c0 2024-05-09T00:49:45.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.403+0000 7ff1d3fff700 1 -- 172.21.15.5:0/2817137694 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc00fd30 con 0x7ff1ec1032c0 2024-05-09T00:49:45.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.410+0000 7ff20008fb80 1 Processor -- start 2024-05-09T00:49:45.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.410+0000 7ff20008fb80 1 -- start start 2024-05-09T00:49:45.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.411+0000 7ff20008fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562426f31d00 0x562427034a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:45.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.411+0000 7ff20008fb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562426eb74a0 0x562427034f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:45.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.411+0000 7ff20008fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56242703e540 0x562427040960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:45.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.411+0000 7ff20008fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562426f570a0 con 0x56242703e540 2024-05-09T00:49:45.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.411+0000 7ff20008fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x562426efc9a0 con 0x562426f31d00 2024-05-09T00:49:45.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.411+0000 7ff20008fb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x562426f53060 con 0x562426eb74a0 2024-05-09T00:49:45.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.411+0000 7ff1d3fff700 1 -- 172.21.15.5:0/2817137694 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7ff1dc029070 con 0x7ff1ec1032c0 2024-05-09T00:49:45.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.412+0000 7ff1eaffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562426eb74a0 0x562427034f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:45.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.412+0000 7ff1eb7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562426f31d00 0x562427034a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:45.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.412+0000 7ff1eaffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562426eb74a0 0x562427034f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:47452/0 (socket says 172.21.15.5:47452) 2024-05-09T00:49:45.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.412+0000 7ff1eaffd700 1 -- 172.21.15.5:0/1246761103 learned_addr learned my addr 172.21.15.5:0/1246761103 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:49:45.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.412+0000 7ff1ea7fc700 1 --2- 172.21.15.5:0/1246761103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56242703e540 0x562427040960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:45.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.416+0000 7ff1d3fff700 1 --2- 172.21.15.5:0/2817137694 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ff1d40b2a90 0x7ff1d40b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:45.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.417+0000 7ff1eaffd700 1 -- 172.21.15.5:0/1246761103 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562426f31d00 msgr2=0x562427034a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:49:45.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.417+0000 7ff1eaffd700 1 --2- 172.21.15.5:0/1246761103 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562426f31d00 0x562427034a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:45.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.417+0000 7ff1eaffd700 1 -- 172.21.15.5:0/1246761103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56242703e540 msgr2=0x562427040960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:49:45.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.417+0000 7ff1eaffd700 1 --2- 172.21.15.5:0/1246761103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56242703e540 0x562427040960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:45.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.417+0000 7ff1eaffd700 1 -- 172.21.15.5:0/1246761103 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562426eed500 con 0x562426eb74a0 2024-05-09T00:49:45.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.418+0000 7ff1eaffd700 1 --2- 172.21.15.5:0/1246761103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562426eb74a0 0x562427034f70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc0a8640 tx=0x7ff1dc0a6270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37a236161691a0b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:49:45.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.418+0000 7ff1eb7fe700 1 --2- 172.21.15.5:0/2817137694 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ff1d40b2a90 0x7ff1d40b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:45.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.418+0000 7ff1d3fff700 1 -- 172.21.15.5:0/2817137694 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7ff1dc0e2ee0 con 0x7ff1ec1032c0 2024-05-09T00:49:45.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.419+0000 7ff1d17fa700 1 -- 172.21.15.5:0/1246761103 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc0aa2d0 con 0x562426eb74a0 2024-05-09T00:49:45.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.419+0000 7ff1d17fa700 1 -- 172.21.15.5:0/1246761103 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1dc0a8450 con 0x562426eb74a0 2024-05-09T00:49:45.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.420+0000 7ff20008fb80 1 -- 172.21.15.5:0/1246761103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562426eb74a0 msgr2=0x562427034f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:45.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.420+0000 7ff20008fb80 1 --2- 172.21.15.5:0/1246761103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562426eb74a0 0x562427034f70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc0a8640 tx=0x7ff1dc0a6270 comp rx=0 tx=0).stop 2024-05-09T00:49:45.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.421+0000 7ff20008fb80 1 -- 172.21.15.5:0/1246761103 shutdown_connections 2024-05-09T00:49:45.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.421+0000 7ff20008fb80 1 --2- 172.21.15.5:0/1246761103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56242703e540 0x562427040960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:45.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.421+0000 7ff20008fb80 1 --2- 172.21.15.5:0/1246761103 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562426f31d00 0x562427034a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:45.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.421+0000 7ff20008fb80 1 --2- 172.21.15.5:0/1246761103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562426eb74a0 0x562427034f70 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:45.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.421+0000 7ff20008fb80 1 -- 172.21.15.5:0/1246761103 >> 172.21.15.5:0/1246761103 conn(0x562426ebaf70 msgr2=0x56242702f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:49:45.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.422+0000 7ff1eb7fe700 1 --2- 172.21.15.5:0/2817137694 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ff1d40b2a90 0x7ff1d40b4fa0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x562427030110 tx=0x7ff1e0004800 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:49:45.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.422+0000 7ff20008fb80 1 -- 172.21.15.5:0/1246761103 shutdown_connections 2024-05-09T00:49:45.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.422+0000 7ff20008fb80 1 -- 172.21.15.5:0/1246761103 wait complete. 2024-05-09T00:49:45.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.422+0000 7ff20008fb80 1 -- start start 2024-05-09T00:49:45.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.423+0000 7ff20008fb80 1 Processor -- start 2024-05-09T00:49:45.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.423+0000 7ff20008fb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562426eb74a0 0x56242703b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:45.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.423+0000 7ff20008fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562426f31d00 0x56242703b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:45.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.424+0000 7ff20008fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56242703e540 0x56242703bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:45.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.424+0000 7ff20008fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562426f53060 con 0x562426f31d00 2024-05-09T00:49:45.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.424+0000 7ff20008fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x562426ef1b20 con 0x56242703e540 2024-05-09T00:49:45.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.424+0000 7ff20008fb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x562427051eb0 con 0x562426eb74a0 2024-05-09T00:49:45.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.424+0000 7ff20008fb80 10 client.0 _ll_register_callbacks cb 0x562426dc86c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:49:45.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.424+0000 7ff1eaffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562426eb74a0 0x56242703b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:45.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.424+0000 7ff1eaffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562426eb74a0 0x56242703b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:47466/0 (socket says 172.21.15.5:47466) 2024-05-09T00:49:45.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.424+0000 7ff1eaffd700 1 -- 172.21.15.5:0/1381503629 learned_addr learned my addr 172.21.15.5:0/1381503629 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:49:45.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.424+0000 7ff1ea7fc700 1 --2- 172.21.15.5:0/1381503629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562426f31d00 0x56242703b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:45.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.425+0000 7ff1ea7fc700 1 -- 172.21.15.5:0/1381503629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562426eb74a0 msgr2=0x56242703b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:49:45.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.425+0000 7ff1ea7fc700 1 --2- 172.21.15.5:0/1381503629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562426eb74a0 0x56242703b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:45.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.425+0000 7ff1ea7fc700 1 -- 172.21.15.5:0/1381503629 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56242703e540 msgr2=0x56242703bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:49:45.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.425+0000 7ff1ea7fc700 1 --2- 172.21.15.5:0/1381503629 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56242703e540 0x56242703bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:45.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.425+0000 7ff1ea7fc700 1 -- 172.21.15.5:0/1381503629 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562426dd2490 con 0x562426f31d00 2024-05-09T00:49:45.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.426+0000 7ff1ea7fc700 1 --2- 172.21.15.5:0/1381503629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562426f31d00 0x56242703b9b0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7ff1e4007d80 tx=0x7ff1e40114b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f3740296bcf82d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:49:45.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.427+0000 7ff1ceff5700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:49:45.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.427+0000 7ff1ceff5700 1 -- 172.21.15.5:0/1381503629 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e401a020 con 0x562426f31d00 2024-05-09T00:49:45.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.428+0000 7ff20008fb80 1 -- 172.21.15.5:0/1381503629 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562426f64a80 con 0x562426f31d00 2024-05-09T00:49:45.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.428+0000 7ff1ceff5700 1 -- 172.21.15.5:0/1381503629 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7ff1e4011a40 con 0x562426f31d00 2024-05-09T00:49:45.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.430+0000 7ff1ceff5700 1 -- 172.21.15.5:0/1381503629 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff1e4025050 con 0x562426f31d00 2024-05-09T00:49:45.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.431+0000 7ff20008fb80 10 client.15837 fetch_fsmap learned FSMap version 29 2024-05-09T00:49:45.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.431+0000 7ff20008fb80 1 -- 172.21.15.5:0/1381503629 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56242705c870 con 0x562426f31d00 2024-05-09T00:49:45.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.433+0000 7ff1ceff5700 1 -- 172.21.15.5:0/1381503629 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x562426f64a80 con 0x562426f31d00 2024-05-09T00:49:45.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.433+0000 7ff1ceff5700 1 -- 172.21.15.5:0/1381503629 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ff1e40153d0 con 0x562426f31d00 2024-05-09T00:49:45.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.433+0000 7ff20008fb80 10 client.15837 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:49:45.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.433+0000 7ff20008fb80 10 client.15837 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:49:45.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.433+0000 7ff20008fb80 20 client.15837 populate_metadata read hostname 'smithi005' 2024-05-09T00:49:45.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.433+0000 7ff20008fb80 1 --2- 172.21.15.5:0/1381503629 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56242705f2f0 0x562427061760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:45.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.433+0000 7ff20008fb80 4 client.15837 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:49:45.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.433+0000 7ff20008fb80 1 -- 172.21.15.5:0/1381503629 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562426f5eb20 con 0x56242705f2f0 2024-05-09T00:49:45.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.436+0000 7ff1eaffd700 1 --2- 172.21.15.5:0/1381503629 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56242705f2f0 0x562427061760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:45.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.442+0000 7ff1eaffd700 1 --2- 172.21.15.5:0/1381503629 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56242705f2f0 0x562427061760 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d824e7d55d59ac2 server_cookie=6545ac0b6d2f4df3 in_seq=0 out_seq=0 2024-05-09T00:49:45.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.443+0000 7ff1ceff5700 10 client.15837 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:49:45.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.446+0000 7ff1ceff5700 1 -- 172.21.15.5:0/1381503629 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562426f5eb20 con 0x56242705f2f0 2024-05-09T00:49:45.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.447+0000 7ff1ceff5700 10 client.15837 handle_command_reply: tid=0 2024-05-09T00:49:45.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.447+0000 7ff20008fb80 1 client.15837 shutdown 2024-05-09T00:49:45.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.447+0000 7ff20008fb80 1 -- 172.21.15.5:0/1381503629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562426f31d00 msgr2=0x56242703b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:45.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.447+0000 7ff20008fb80 1 --2- 172.21.15.5:0/1381503629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562426f31d00 0x56242703b9b0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7ff1e4007d80 tx=0x7ff1e40114b0 comp rx=0 tx=0).stop 2024-05-09T00:49:45.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.447+0000 7ff20008fb80 1 -- 172.21.15.5:0/1381503629 shutdown_connections 2024-05-09T00:49:45.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.447+0000 7ff20008fb80 1 --2- 172.21.15.5:0/1381503629 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56242705f2f0 0x562427061760 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:45.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.447+0000 7ff20008fb80 1 --2- 172.21.15.5:0/1381503629 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56242703e540 0x56242703bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:45.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.447+0000 7ff1ceff5700 0 client.15837 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:49:45.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.447+0000 7ff20008fb80 1 --2- 172.21.15.5:0/1381503629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562426eb74a0 0x56242703b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:45.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.447+0000 7ff20008fb80 1 --2- 172.21.15.5:0/1381503629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562426f31d00 0x56242703b9b0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:45.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.447+0000 7ff20008fb80 1 -- 172.21.15.5:0/1381503629 >> 172.21.15.5:0/1381503629 conn(0x562426ebaf70 msgr2=0x56242702f250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:49:45.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.448+0000 7ff20008fb80 1 -- 172.21.15.5:0/1381503629 shutdown_connections 2024-05-09T00:49:45.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.448+0000 7ff20008fb80 1 -- 172.21.15.5:0/1381503629 wait complete. 2024-05-09T00:49:45.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.448+0000 7ff20008fb80 20 client.15837 trim_cache size 0 max 16384 2024-05-09T00:49:45.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.488+0000 7ff20008fb80 1 Processor -- start 2024-05-09T00:49:45.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.489+0000 7ff20008fb80 1 -- start start 2024-05-09T00:49:45.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.489+0000 7ff20008fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562426eb74a0 0x562427033b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:45.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.490+0000 7ff20008fb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562426f31d00 0x56242703b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:45.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.490+0000 7ff20008fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56242703e540 0x56242705ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:45.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.490+0000 7ff1eaffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562426eb74a0 0x562427033b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:45.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.490+0000 7ff20008fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d800d590 con 0x56242703e540 2024-05-09T00:49:45.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.490+0000 7ff1eaffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562426eb74a0 0x562427033b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:56404/0 (socket says 172.21.15.5:56404) 2024-05-09T00:49:45.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.490+0000 7ff1eaffd700 1 -- 172.21.15.5:0/2417375405 learned_addr learned my addr 172.21.15.5:0/2417375405 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:49:45.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.490+0000 7ff1eb7fe700 1 --2- 172.21.15.5:0/2417375405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56242703e540 0x56242705ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:45.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.491+0000 7ff1ea7fc700 1 --2- 172.21.15.5:0/2417375405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562426f31d00 0x56242703b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:45.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.491+0000 7ff20008fb80 1 -- 172.21.15.5:0/2417375405 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d800f0d0 con 0x562426eb74a0 2024-05-09T00:49:45.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.491+0000 7ff20008fb80 1 -- 172.21.15.5:0/2417375405 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d800dc60 con 0x562426f31d00 2024-05-09T00:49:45.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.492+0000 7ff1ea7fc700 1 -- 172.21.15.5:0/2417375405 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562426eb74a0 msgr2=0x562427033b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:49:45.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.492+0000 7ff1ea7fc700 1 --2- 172.21.15.5:0/2417375405 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562426eb74a0 0x562427033b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:45.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.492+0000 7ff1ea7fc700 1 -- 172.21.15.5:0/2417375405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56242703e540 msgr2=0x56242705ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:49:45.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.492+0000 7ff1ea7fc700 1 --2- 172.21.15.5:0/2417375405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56242703e540 0x56242705ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:45.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.492+0000 7ff1ea7fc700 1 -- 172.21.15.5:0/2417375405 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56242702b970 con 0x562426f31d00 2024-05-09T00:49:45.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.493+0000 7ff1ea7fc700 1 --2- 172.21.15.5:0/2417375405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562426f31d00 0x56242703b9e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff1e40157c0 tx=0x7ff1e4009980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d931a511afd55c7b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:49:45.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.493+0000 7ff1d0ff9700 1 -- 172.21.15.5:0/2417375405 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e401a070 con 0x562426f31d00 2024-05-09T00:49:45.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.494+0000 7ff1d0ff9700 1 -- 172.21.15.5:0/2417375405 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1e402e070 con 0x562426f31d00 2024-05-09T00:49:45.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.494+0000 7ff1d0ff9700 1 -- 172.21.15.5:0/2417375405 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e4011e60 con 0x562426f31d00 2024-05-09T00:49:45.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.495+0000 7ff20008fb80 1 -- 172.21.15.5:0/2417375405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562426f31d00 msgr2=0x56242703b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:45.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.496+0000 7ff20008fb80 1 --2- 172.21.15.5:0/2417375405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562426f31d00 0x56242703b9e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff1e40157c0 tx=0x7ff1e4009980 comp rx=0 tx=0).stop 2024-05-09T00:49:45.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.497+0000 7ff20008fb80 1 -- 172.21.15.5:0/2417375405 shutdown_connections 2024-05-09T00:49:45.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.497+0000 7ff20008fb80 1 --2- 172.21.15.5:0/2417375405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56242703e540 0x56242705ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:45.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.497+0000 7ff20008fb80 1 --2- 172.21.15.5:0/2417375405 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562426eb74a0 0x562427033b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:45.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.498+0000 7ff20008fb80 1 --2- 172.21.15.5:0/2417375405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562426f31d00 0x56242703b9e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:45.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.498+0000 7ff20008fb80 1 -- 172.21.15.5:0/2417375405 >> 172.21.15.5:0/2417375405 conn(0x562426ebaf70 msgr2=0x5624270333f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:49:45.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.498+0000 7ff20008fb80 1 -- 172.21.15.5:0/2417375405 shutdown_connections 2024-05-09T00:49:45.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.498+0000 7ff20008fb80 1 -- 172.21.15.5:0/2417375405 wait complete. 2024-05-09T00:49:45.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.500+0000 7ff20008fb80 1 -- start start 2024-05-09T00:49:45.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.500+0000 7ff20008fb80 1 Processor -- start 2024-05-09T00:49:45.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.501+0000 7ff20008fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562426eb74a0 0x562427057550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:45.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.501+0000 7ff20008fb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562426f31d00 0x562427051eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:45.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.501+0000 7ff20008fb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56242703e540 0x562427052500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:45.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.501+0000 7ff20008fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d800ea00 con 0x562426eb74a0 2024-05-09T00:49:45.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.501+0000 7ff20008fb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d800e330 con 0x56242703e540 2024-05-09T00:49:45.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.501+0000 7ff20008fb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d800b530 con 0x562426f31d00 2024-05-09T00:49:45.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.501+0000 7ff20008fb80 10 client.0 _ll_register_callbacks cb 0x562426dc86c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:49:45.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.501+0000 7ff1eaffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562426eb74a0 0x562427057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:45.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.501+0000 7ff1eaffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562426eb74a0 0x562427057550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50952/0 (socket says 172.21.15.5:50952) 2024-05-09T00:49:45.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.501+0000 7ff1eaffd700 1 -- 172.21.15.5:0/965756453 learned_addr learned my addr 172.21.15.5:0/965756453 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:49:45.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.502+0000 7ff1eaffd700 1 -- 172.21.15.5:0/965756453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562426f31d00 msgr2=0x562427051eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:49:45.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.502+0000 7ff1eaffd700 1 --2- 172.21.15.5:0/965756453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562426f31d00 0x562427051eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:45.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.502+0000 7ff1eaffd700 1 -- 172.21.15.5:0/965756453 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56242703e540 msgr2=0x562427052500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:49:45.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.502+0000 7ff1eaffd700 1 --2- 172.21.15.5:0/965756453 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56242703e540 0x562427052500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:45.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.502+0000 7ff1eaffd700 1 -- 172.21.15.5:0/965756453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562427052cb0 con 0x562426eb74a0 2024-05-09T00:49:45.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.502+0000 7ff1eaffd700 1 --2- 172.21.15.5:0/965756453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562426eb74a0 0x562427057550 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc0a6580 tx=0x7ff1dc0e6440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=978d526fa00b49ba server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:49:45.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.502+0000 7ff1ceff5700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:49:45.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.503+0000 7ff1ceff5700 1 -- 172.21.15.5:0/965756453 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc0a7d90 con 0x562426eb74a0 2024-05-09T00:49:45.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.503+0000 7ff1ceff5700 1 -- 172.21.15.5:0/965756453 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7ff1dc0e09d0 con 0x562426eb74a0 2024-05-09T00:49:45.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.503+0000 7ff20008fb80 1 -- 172.21.15.5:0/965756453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1e402e070 con 0x562426eb74a0 2024-05-09T00:49:45.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.503+0000 7ff1ceff5700 1 -- 172.21.15.5:0/965756453 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff1dc0a81f0 con 0x562426eb74a0 2024-05-09T00:49:45.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.503+0000 7ff20008fb80 10 client.15843 fetch_fsmap learned FSMap version 29 2024-05-09T00:49:45.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.503+0000 7ff20008fb80 1 -- 172.21.15.5:0/965756453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5624270c8bf0 con 0x562426eb74a0 2024-05-09T00:49:45.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.503+0000 7ff1ceff5700 1 -- 172.21.15.5:0/965756453 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1dc0e6670 con 0x562426eb74a0 2024-05-09T00:49:45.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.504+0000 7ff1ceff5700 1 -- 172.21.15.5:0/965756453 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ff1dc0aa760 con 0x562426eb74a0 2024-05-09T00:49:45.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.504+0000 7ff20008fb80 10 client.15843 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:49:45.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.504+0000 7ff20008fb80 10 client.15843 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:49:45.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.504+0000 7ff20008fb80 20 client.15843 populate_metadata read hostname 'smithi005' 2024-05-09T00:49:45.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.504+0000 7ff20008fb80 1 --2- 172.21.15.5:0/965756453 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56242705f2f0 0x5624270cb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:45.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.505+0000 7ff20008fb80 4 client.15843 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:49:45.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.505+0000 7ff20008fb80 1 -- 172.21.15.5:0/965756453 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7ff1e402e070 con 0x56242705f2f0 2024-05-09T00:49:45.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.505+0000 7ff1ea7fc700 1 --2- 172.21.15.5:0/965756453 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56242705f2f0 0x5624270cb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:45.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.507+0000 7ff1ea7fc700 1 --2- 172.21.15.5:0/965756453 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56242705f2f0 0x5624270cb2b0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d8e4a949b4c9083 server_cookie=b22fd340faf017c1 in_seq=0 out_seq=0 2024-05-09T00:49:45.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.507+0000 7ff1ceff5700 10 client.15843 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:49:45.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.512+0000 7ff1ceff5700 1 -- 172.21.15.5:0/965756453 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7ff1e402e070 con 0x56242705f2f0 2024-05-09T00:49:45.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.512+0000 7ff1ceff5700 10 client.15843 handle_command_reply: tid=0 2024-05-09T00:49:45.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.513+0000 7ff20008fb80 1 client.15843 shutdown 2024-05-09T00:49:45.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.514+0000 7ff20008fb80 1 -- 172.21.15.5:0/965756453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562426eb74a0 msgr2=0x562427057550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:45.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.514+0000 7ff20008fb80 1 --2- 172.21.15.5:0/965756453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562426eb74a0 0x562427057550 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc0a6580 tx=0x7ff1dc0e6440 comp rx=0 tx=0).stop 2024-05-09T00:49:45.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.514+0000 7ff20008fb80 1 -- 172.21.15.5:0/965756453 shutdown_connections 2024-05-09T00:49:45.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.514+0000 7ff20008fb80 1 --2- 172.21.15.5:0/965756453 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56242705f2f0 0x5624270cb2b0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:45.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.515+0000 7ff20008fb80 1 --2- 172.21.15.5:0/965756453 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56242703e540 0x562427052500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:45.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.515+0000 7ff1ceff5700 0 client.15843 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:49:45.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.515+0000 7ff20008fb80 1 --2- 172.21.15.5:0/965756453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562426eb74a0 0x562427057550 secure :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc0a6580 tx=0x7ff1dc0e6440 comp rx=0 tx=0).stop 2024-05-09T00:49:45.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.515+0000 7ff20008fb80 1 --2- 172.21.15.5:0/965756453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562426f31d00 0x562427051eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:45.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.515+0000 7ff20008fb80 1 -- 172.21.15.5:0/965756453 >> 172.21.15.5:0/965756453 conn(0x562426ebaf70 msgr2=0x562427039950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:49:45.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.522+0000 7ff20008fb80 1 -- 172.21.15.5:0/965756453 shutdown_connections 2024-05-09T00:49:45.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.522+0000 7ff20008fb80 1 -- 172.21.15.5:0/965756453 wait complete. 2024-05-09T00:49:45.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.523+0000 7ff20008fb80 20 client.15843 trim_cache size 0 max 16384 2024-05-09T00:49:45.531 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-09T00:49:45.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.536+0000 7ff1ebfff700 1 -- 172.21.15.5:0/2817137694 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ff1d40b2a90 msgr2=0x7ff1d40b4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:45.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.536+0000 7ff1ebfff700 1 --2- 172.21.15.5:0/2817137694 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ff1d40b2a90 0x7ff1d40b4fa0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x562427030110 tx=0x7ff1e0004800 comp rx=0 tx=0).stop 2024-05-09T00:49:45.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.536+0000 7ff1ebfff700 1 -- 172.21.15.5:0/2817137694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1ec1032c0 msgr2=0x7ff1ec118b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:45.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.536+0000 7ff1ebfff700 1 --2- 172.21.15.5:0/2817137694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1ec1032c0 0x7ff1ec118b20 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc000c00 tx=0x7ff1dc0057c0 comp rx=0 tx=0).stop 2024-05-09T00:49:45.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.536+0000 7ff1ebfff700 1 -- 172.21.15.5:0/2817137694 shutdown_connections 2024-05-09T00:49:45.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.536+0000 7ff1ebfff700 1 --2- 172.21.15.5:0/2817137694 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ff1d40b2a90 0x7ff1d40b4fa0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:45.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.536+0000 7ff1ebfff700 1 --2- 172.21.15.5:0/2817137694 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff1ec108540 0x7ff1ec11e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:45.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.536+0000 7ff1ebfff700 1 --2- 172.21.15.5:0/2817137694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1ec1032c0 0x7ff1ec118b20 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:45.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.536+0000 7ff1ebfff700 1 --2- 172.21.15.5:0/2817137694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1ec107bf0 0x7ff1ec119080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:45.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.536+0000 7ff1ebfff700 1 -- 172.21.15.5:0/2817137694 >> 172.21.15.5:0/2817137694 conn(0x7ff1ec078bf0 msgr2=0x7ff1ec1006d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:49:45.539 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:49:45 smithi005 ceph-mon[30365]: pgmap v473: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 851 MiB/s wr, 2.17k op/s 2024-05-09T00:49:45.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.537+0000 7ff1ebfff700 1 -- 172.21.15.5:0/2817137694 shutdown_connections 2024-05-09T00:49:45.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:45.537+0000 7ff1ebfff700 1 -- 172.21.15.5:0/2817137694 wait complete. 2024-05-09T00:49:45.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:49:45 smithi131 ceph-mon[34547]: pgmap v473: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 851 MiB/s wr, 2.17k op/s 2024-05-09T00:49:45.685 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:49:45 smithi157 ceph-mon[34294]: pgmap v473: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 851 MiB/s wr, 2.17k op/s 2024-05-09T00:49:47.440 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:49:47 smithi005 ceph-mon[30365]: pgmap v474: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 937 MiB/s wr, 2.18k op/s 2024-05-09T00:49:47.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:49:47 smithi131 ceph-mon[34547]: pgmap v474: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 937 MiB/s wr, 2.18k op/s 2024-05-09T00:49:47.685 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:49:47 smithi157 ceph-mon[34294]: pgmap v474: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 937 MiB/s wr, 2.18k op/s 2024-05-09T00:49:48.445 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T00:49:48.445 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T00:49:49.229 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:49:49.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:49:49 smithi131 ceph-mon[34547]: pgmap v475: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 807 MiB/s wr, 1.57k op/s 2024-05-09T00:49:49.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:49:49 smithi157 ceph-mon[34294]: pgmap v475: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 807 MiB/s wr, 1.57k op/s 2024-05-09T00:49:49.728 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:49:49 smithi005 ceph-mon[30365]: pgmap v475: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 807 MiB/s wr, 1.57k op/s 2024-05-09T00:49:51.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:49:51 smithi131 ceph-mon[34547]: pgmap v476: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 988 MiB/s wr, 1.90k op/s 2024-05-09T00:49:51.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:49:51 smithi157 ceph-mon[34294]: pgmap v476: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 988 MiB/s wr, 1.90k op/s 2024-05-09T00:49:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:51.770+0000 7fe79dffb640 1 -- 172.21.15.5:0/2280865453 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55699f944350 con 0x55699faee210 2024-05-09T00:49:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:51.770+0000 7fe79dffb640 10 client.15825 handle_command_reply: tid=0 2024-05-09T00:49:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:51.770+0000 7fe7c39b9740 1 client.15825 shutdown 2024-05-09T00:49:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:51.771+0000 7fe7c39b9740 1 -- 172.21.15.5:0/2280865453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55699f6902f0 msgr2=0x55699fa6a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:51.771+0000 7fe7c39b9740 1 --2- 172.21.15.5:0/2280865453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55699f6902f0 0x55699fa6a520 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0001eb0 tx=0x7fe7a00047d0 comp rx=0 tx=0).stop 2024-05-09T00:49:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:51.771+0000 7fe7c39b9740 1 -- 172.21.15.5:0/2280865453 shutdown_connections 2024-05-09T00:49:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:51.771+0000 7fe7c39b9740 1 --2- 172.21.15.5:0/2280865453 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55699faee210 0x55699faf0630 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:51.771+0000 7fe7c39b9740 1 --2- 172.21.15.5:0/2280865453 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55699f6984d0 0x55699fa6d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:51.771+0000 7fe7c39b9740 1 --2- 172.21.15.5:0/2280865453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55699f6902f0 0x55699fa6a520 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:51.771+0000 7fe7c39b9740 1 --2- 172.21.15.5:0/2280865453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55699f677d80 0x55699fa69fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:51.771+0000 7fe7c39b9740 1 -- 172.21.15.5:0/2280865453 >> 172.21.15.5:0/2280865453 conn(0x55699f6ad300 msgr2=0x55699fa794a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:49:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:51.771+0000 7fe79dffb640 0 client.15825 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:49:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:51.771+0000 7fe7c39b9740 1 -- 172.21.15.5:0/2280865453 shutdown_connections 2024-05-09T00:49:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:51.771+0000 7fe7c39b9740 1 -- 172.21.15.5:0/2280865453 wait complete. 2024-05-09T00:49:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:51.772+0000 7fe7c39b9740 20 client.15825 trim_cache size 0 max 16384 2024-05-09T00:49:51.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-09T00:49:51.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-09T00:49:51.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-09T00:49:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:51.775+0000 7fe7b759e640 1 -- 172.21.15.5:0/1924688466 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe7a40b2930 msgr2=0x7fe7a40b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:51.775+0000 7fe7b759e640 1 --2- 172.21.15.5:0/1924688466 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe7a40b2930 0x7fe7a40b4e40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8048000 tx=0x7fe7a802f8b0 comp rx=0 tx=0).stop 2024-05-09T00:49:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:51.775+0000 7fe7b759e640 1 -- 172.21.15.5:0/1924688466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7b810a510 msgr2=0x7fe7b81a9130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:51.775+0000 7fe7b759e640 1 --2- 172.21.15.5:0/1924688466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7b810a510 0x7fe7b81a9130 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0006fd0 tx=0x7fe7b000b820 comp rx=0 tx=0).stop 2024-05-09T00:49:51.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:51.778+0000 7fe7b759e640 1 -- 172.21.15.5:0/1924688466 shutdown_connections 2024-05-09T00:49:51.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:51.778+0000 7fe7b759e640 1 --2- 172.21.15.5:0/1924688466 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe7a40b2930 0x7fe7a40b4e40 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:51.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:51.778+0000 7fe7b759e640 1 --2- 172.21.15.5:0/1924688466 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7b81103f0 0x7fe7b81b0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:51.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:51.778+0000 7fe7b759e640 1 --2- 172.21.15.5:0/1924688466 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7b810b6f0 0x7fe7b81a9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:51.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:51.778+0000 7fe7b759e640 1 --2- 172.21.15.5:0/1924688466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7b810a510 0x7fe7b81a9130 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:51.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:51.778+0000 7fe7b759e640 1 -- 172.21.15.5:0/1924688466 >> 172.21.15.5:0/1924688466 conn(0x7fe7b80708d0 msgr2=0x7fe7b8072cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:49:51.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:51.780+0000 7fe7b759e640 1 -- 172.21.15.5:0/1924688466 shutdown_connections 2024-05-09T00:49:51.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:49:51.780+0000 7fe7b759e640 1 -- 172.21.15.5:0/1924688466 wait complete. 2024-05-09T00:49:51.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:49:51 smithi005 ceph-mon[30365]: pgmap v476: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 988 MiB/s wr, 1.90k op/s 2024-05-09T00:49:53.782 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:49:53 smithi005 ceph-mon[30365]: pgmap v477: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 846 MiB/s wr, 1.75k op/s 2024-05-09T00:49:53.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:49:53 smithi131 ceph-mon[34547]: pgmap v477: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 846 MiB/s wr, 1.75k op/s 2024-05-09T00:49:53.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:49:53 smithi157 ceph-mon[34294]: pgmap v477: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 846 MiB/s wr, 1.75k op/s 2024-05-09T00:49:54.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.082+0000 7fe56f144700 1 -- 172.21.15.5:0/2855594914 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5681084d0 msgr2=0x7fe5681089a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:54.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.082+0000 7fe56f144700 1 --2- 172.21.15.5:0/2855594914 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5681084d0 0x7fe5681089a0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fe55800b0a0 tx=0x7fe55800b3b0 comp rx=0 tx=0).stop 2024-05-09T00:49:54.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.082+0000 7fe56f144700 1 -- 172.21.15.5:0/2855594914 shutdown_connections 2024-05-09T00:49:54.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.082+0000 7fe56f144700 1 --2- 172.21.15.5:0/2855594914 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5681084d0 0x7fe5681089a0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:54.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.082+0000 7fe56f144700 1 --2- 172.21.15.5:0/2855594914 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe568103080 0x7fe568103490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:54.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.082+0000 7fe56f144700 1 --2- 172.21.15.5:0/2855594914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5681039d0 0x7fe568107f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:54.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.082+0000 7fe56f144700 1 -- 172.21.15.5:0/2855594914 >> 172.21.15.5:0/2855594914 conn(0x7fe568078ac0 msgr2=0x7fe568078f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:49:54.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.088+0000 7fe56f144700 1 -- 172.21.15.5:0/2855594914 shutdown_connections 2024-05-09T00:49:54.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.091+0000 7fe56f144700 1 -- 172.21.15.5:0/2855594914 wait complete. 2024-05-09T00:49:54.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.093+0000 7fe56f144700 1 Processor -- start 2024-05-09T00:49:54.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.093+0000 7fe56f144700 1 -- start start 2024-05-09T00:49:54.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.094+0000 7fe56f144700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe568103080 0x7fe568111530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:54.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.094+0000 7fe56f144700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5681039d0 0x7fe5681095a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:54.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.094+0000 7fe56f144700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe5681084d0 0x7fe568109ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:54.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.095+0000 7fe56f144700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe56811a8b0 con 0x7fe568103080 2024-05-09T00:49:54.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.095+0000 7fe56f144700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe56811a730 con 0x7fe5681084d0 2024-05-09T00:49:54.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.095+0000 7fe56f144700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe56811aa30 con 0x7fe5681039d0 2024-05-09T00:49:54.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.096+0000 7fe56e943700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe5681084d0 0x7fe568109ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:54.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.096+0000 7fe56e943700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe5681084d0 0x7fe568109ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:51822/0 (socket says 172.21.15.5:51822) 2024-05-09T00:49:54.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.096+0000 7fe56e943700 1 -- 172.21.15.5:0/88647321 learned_addr learned my addr 172.21.15.5:0/88647321 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:49:54.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.098+0000 7fe56e943700 1 -- 172.21.15.5:0/88647321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5681039d0 msgr2=0x7fe5681095a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:54.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.098+0000 7fe56e943700 1 --2- 172.21.15.5:0/88647321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5681039d0 0x7fe5681095a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:54.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.098+0000 7fe56e943700 1 -- 172.21.15.5:0/88647321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe568103080 msgr2=0x7fe568111530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:54.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.098+0000 7fe56e943700 1 --2- 172.21.15.5:0/88647321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe568103080 0x7fe568111530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:54.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.098+0000 7fe56e943700 1 -- 172.21.15.5:0/88647321 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe56810a1e0 con 0x7fe5681084d0 2024-05-09T00:49:54.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.099+0000 7fe56e943700 1 --2- 172.21.15.5:0/88647321 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe5681084d0 0x7fe568109ae0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe558004fb0 tx=0x7fe55800cf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:49:54.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.101+0000 7fe5677fe700 1 -- 172.21.15.5:0/88647321 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe558021020 con 0x7fe5681084d0 2024-05-09T00:49:54.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.101+0000 7fe56f144700 1 -- 172.21.15.5:0/88647321 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5681b60b0 con 0x7fe5681084d0 2024-05-09T00:49:54.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.101+0000 7fe56f144700 1 -- 172.21.15.5:0/88647321 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5681b64b0 con 0x7fe5681084d0 2024-05-09T00:49:54.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.102+0000 7fe5677fe700 1 -- 172.21.15.5:0/88647321 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe558007d40 con 0x7fe5681084d0 2024-05-09T00:49:54.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.102+0000 7fe5677fe700 1 -- 172.21.15.5:0/88647321 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe55801b840 con 0x7fe5681084d0 2024-05-09T00:49:54.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.108+0000 7fe57edbcb80 1 Processor -- start 2024-05-09T00:49:54.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.111+0000 7fe5677fe700 1 -- 172.21.15.5:0/88647321 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fe558024020 con 0x7fe5681084d0 2024-05-09T00:49:54.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.111+0000 7fe57edbcb80 1 -- start start 2024-05-09T00:49:54.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.112+0000 7fe57edbcb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea0f320d80 0x55ea0f429510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:54.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.112+0000 7fe57edbcb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea0f2a6520 0x55ea0f429a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:54.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.112+0000 7fe57edbcb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea0f423660 0x55ea0f423a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:54.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.112+0000 7fe57edbcb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ea0f346120 con 0x55ea0f423660 2024-05-09T00:49:54.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.112+0000 7fe57edbcb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55ea0f2eba20 con 0x55ea0f2a6520 2024-05-09T00:49:54.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.112+0000 7fe57edbcb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ea0f3420e0 con 0x55ea0f320d80 2024-05-09T00:49:54.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.114+0000 7fe56e142700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea0f2a6520 0x55ea0f429a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:54.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.114+0000 7fe56e142700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea0f2a6520 0x55ea0f429a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:51828/0 (socket says 172.21.15.5:51828) 2024-05-09T00:49:54.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.114+0000 7fe56e142700 1 -- 172.21.15.5:0/1497632939 learned_addr learned my addr 172.21.15.5:0/1497632939 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:49:54.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.114+0000 7fe5677fe700 1 --2- 172.21.15.5:0/88647321 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe5500b2a90 0x7fe5500b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:54.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.115+0000 7fe5677fe700 1 -- 172.21.15.5:0/88647321 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fe5580e3790 con 0x7fe5681084d0 2024-05-09T00:49:54.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.116+0000 7fe56d941700 1 --2- 172.21.15.5:0/1497632939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea0f423660 0x55ea0f423a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:54.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.116+0000 7fe56e142700 1 -- 172.21.15.5:0/1497632939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea0f320d80 msgr2=0x55ea0f429510 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:49:54.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.116+0000 7fe56e142700 1 --2- 172.21.15.5:0/1497632939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea0f320d80 0x55ea0f429510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:54.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.116+0000 7fe56e142700 1 -- 172.21.15.5:0/1497632939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea0f423660 msgr2=0x55ea0f423a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:49:54.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.117+0000 7fe56e142700 1 --2- 172.21.15.5:0/1497632939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea0f423660 0x55ea0f423a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:54.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.117+0000 7fe56e142700 1 -- 172.21.15.5:0/1497632939 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea0f2dc580 con 0x55ea0f2a6520 2024-05-09T00:49:54.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.118+0000 7fe56e142700 1 --2- 172.21.15.5:0/1497632939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea0f2a6520 0x55ea0f429a50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe55c00bb90 tx=0x7fe55c00bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88eb6cc08cb64922 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:49:54.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.118+0000 7fe56d941700 1 --2- 172.21.15.5:0/1497632939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea0f423660 0x55ea0f423a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:49:54.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.119+0000 7fe564d47700 1 -- 172.21.15.5:0/1497632939 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe55c016070 con 0x55ea0f2a6520 2024-05-09T00:49:54.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.119+0000 7fe564d47700 1 -- 172.21.15.5:0/1497632939 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe55c004830 con 0x55ea0f2a6520 2024-05-09T00:49:54.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.119+0000 7fe56e943700 1 --2- 172.21.15.5:0/88647321 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe5500b2a90 0x7fe5500b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:54.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.121+0000 7fe57edbcb80 1 -- 172.21.15.5:0/1497632939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea0f2a6520 msgr2=0x55ea0f429a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:54.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.121+0000 7fe57edbcb80 1 --2- 172.21.15.5:0/1497632939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea0f2a6520 0x55ea0f429a50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe55c00bb90 tx=0x7fe55c00bea0 comp rx=0 tx=0).stop 2024-05-09T00:49:54.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.121+0000 7fe56e943700 1 --2- 172.21.15.5:0/88647321 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe5500b2a90 0x7fe5500b4fa0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe55800b3f0 tx=0x7fe558009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:49:54.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.127+0000 7fe57edbcb80 1 -- 172.21.15.5:0/1497632939 shutdown_connections 2024-05-09T00:49:54.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.127+0000 7fe57edbcb80 1 --2- 172.21.15.5:0/1497632939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea0f423660 0x55ea0f423a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:54.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.127+0000 7fe57edbcb80 1 --2- 172.21.15.5:0/1497632939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea0f320d80 0x55ea0f429510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:54.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.127+0000 7fe57edbcb80 1 --2- 172.21.15.5:0/1497632939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea0f2a6520 0x55ea0f429a50 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:54.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.127+0000 7fe57edbcb80 1 -- 172.21.15.5:0/1497632939 >> 172.21.15.5:0/1497632939 conn(0x55ea0f2a9ff0 msgr2=0x55ea0f41e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:49:54.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.128+0000 7fe57edbcb80 1 -- 172.21.15.5:0/1497632939 shutdown_connections 2024-05-09T00:49:54.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.128+0000 7fe57edbcb80 1 -- 172.21.15.5:0/1497632939 wait complete. 2024-05-09T00:49:54.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.129+0000 7fe57edbcb80 1 -- start start 2024-05-09T00:49:54.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.129+0000 7fe57edbcb80 1 Processor -- start 2024-05-09T00:49:54.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.130+0000 7fe57edbcb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea0f2a6520 0x55ea0f42d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:54.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.130+0000 7fe57edbcb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea0f320d80 0x55ea0f42d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:54.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.130+0000 7fe57edbcb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea0f423660 0x55ea0f42dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:54.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.130+0000 7fe57edbcb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ea0f424460 con 0x55ea0f320d80 2024-05-09T00:49:54.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.130+0000 7fe57edbcb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55ea0f2e0ba0 con 0x55ea0f2a6520 2024-05-09T00:49:54.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.130+0000 7fe57edbcb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ea0f430280 con 0x55ea0f423660 2024-05-09T00:49:54.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.130+0000 7fe57edbcb80 10 client.0 _ll_register_callbacks cb 0x55ea0f1b7740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:49:54.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.131+0000 7fe56d941700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea0f2a6520 0x55ea0f42d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:54.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.131+0000 7fe56d941700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea0f423660 0x55ea0f42dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:54.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.131+0000 7fe56e142700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea0f320d80 0x55ea0f42d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:54.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.131+0000 7fe56e142700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea0f320d80 0x55ea0f42d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58832/0 (socket says 172.21.15.5:58832) 2024-05-09T00:49:54.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.131+0000 7fe56e142700 1 -- 172.21.15.5:0/235700120 learned_addr learned my addr 172.21.15.5:0/235700120 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:49:54.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.136+0000 7fe56d941700 1 -- 172.21.15.5:0/235700120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea0f423660 msgr2=0x55ea0f42dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:49:54.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.136+0000 7fe56d941700 1 --2- 172.21.15.5:0/235700120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea0f423660 0x55ea0f42dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:54.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.136+0000 7fe56d941700 1 -- 172.21.15.5:0/235700120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea0f320d80 msgr2=0x55ea0f42d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:49:54.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.137+0000 7fe56d941700 1 --2- 172.21.15.5:0/235700120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea0f320d80 0x55ea0f42d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:54.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.137+0000 7fe56d941700 1 -- 172.21.15.5:0/235700120 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea0f1c1510 con 0x55ea0f2a6520 2024-05-09T00:49:54.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.140+0000 7fe56d941700 1 --2- 172.21.15.5:0/235700120 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea0f2a6520 0x55ea0f42d130 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe560005fd0 tx=0x7fe560005d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e19f4fdfbe0328b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:49:54.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.140+0000 7fe54dffb700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-09T00:49:54.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.142+0000 7fe54dffb700 1 -- 172.21.15.5:0/235700120 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe560015230 con 0x55ea0f2a6520 2024-05-09T00:49:54.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.142+0000 7fe54dffb700 1 -- 172.21.15.5:0/235700120 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fe560004260 con 0x55ea0f2a6520 2024-05-09T00:49:54.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.143+0000 7fe57edbcb80 1 -- 172.21.15.5:0/235700120 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe55c004830 con 0x55ea0f2a6520 2024-05-09T00:49:54.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.145+0000 7fe54dffb700 1 -- 172.21.15.5:0/235700120 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe560004460 con 0x55ea0f2a6520 2024-05-09T00:49:54.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.147+0000 7fe57edbcb80 10 client.25475 fetch_fsmap learned FSMap version 29 2024-05-09T00:49:54.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.147+0000 7fe57edbcb80 1 -- 172.21.15.5:0/235700120 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ea0f44bc70 con 0x55ea0f2a6520 2024-05-09T00:49:54.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.150+0000 7fe54dffb700 1 -- 172.21.15.5:0/235700120 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe55c004830 con 0x55ea0f2a6520 2024-05-09T00:49:54.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.150+0000 7fe54dffb700 1 -- 172.21.15.5:0/235700120 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fe56001e030 con 0x55ea0f2a6520 2024-05-09T00:49:54.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.150+0000 7fe57edbcb80 10 client.25475 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:49:54.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.150+0000 7fe57edbcb80 10 client.25475 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:49:54.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.150+0000 7fe57edbcb80 20 client.25475 populate_metadata read hostname 'smithi005' 2024-05-09T00:49:54.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.150+0000 7fe57edbcb80 1 --2- 172.21.15.5:0/235700120 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55ea0f44e710 0x55ea0f450b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:54.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.150+0000 7fe57edbcb80 4 client.25475 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:49:54.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.151+0000 7fe57edbcb80 1 -- 172.21.15.5:0/235700120 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea0f353b00 con 0x55ea0f44e710 2024-05-09T00:49:54.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.152+0000 7fe56e142700 1 --2- 172.21.15.5:0/235700120 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55ea0f44e710 0x55ea0f450b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:54.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.160+0000 7fe56e142700 1 --2- 172.21.15.5:0/235700120 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55ea0f44e710 0x55ea0f450b60 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0affb6800879614 server_cookie=23192e5407fd0795 in_seq=0 out_seq=0 2024-05-09T00:49:54.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.160+0000 7fe54dffb700 10 client.25475 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:49:54.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.164+0000 7fe54dffb700 1 -- 172.21.15.5:0/235700120 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ea0f353b00 con 0x55ea0f44e710 2024-05-09T00:49:54.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.164+0000 7fe54dffb700 10 client.25475 handle_command_reply: tid=0 2024-05-09T00:49:54.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.165+0000 7fe57edbcb80 1 client.25475 shutdown 2024-05-09T00:49:54.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.168+0000 7fe57edbcb80 1 -- 172.21.15.5:0/235700120 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea0f2a6520 msgr2=0x55ea0f42d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:54.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.168+0000 7fe57edbcb80 1 --2- 172.21.15.5:0/235700120 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea0f2a6520 0x55ea0f42d130 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe560005fd0 tx=0x7fe560005d70 comp rx=0 tx=0).stop 2024-05-09T00:49:54.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.174+0000 7fe57edbcb80 1 -- 172.21.15.5:0/235700120 shutdown_connections 2024-05-09T00:49:54.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.174+0000 7fe57edbcb80 1 --2- 172.21.15.5:0/235700120 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55ea0f44e710 0x55ea0f450b60 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:54.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.174+0000 7fe57edbcb80 1 --2- 172.21.15.5:0/235700120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea0f423660 0x55ea0f42dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:54.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.174+0000 7fe57edbcb80 1 --2- 172.21.15.5:0/235700120 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea0f2a6520 0x55ea0f42d130 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:54.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.174+0000 7fe57edbcb80 1 --2- 172.21.15.5:0/235700120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea0f320d80 0x55ea0f42d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:54.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.174+0000 7fe57edbcb80 1 -- 172.21.15.5:0/235700120 >> 172.21.15.5:0/235700120 conn(0x55ea0f2a9ff0 msgr2=0x55ea0f41e350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:49:54.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.174+0000 7fe54dffb700 0 client.25475 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:49:54.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.174+0000 7fe57edbcb80 1 -- 172.21.15.5:0/235700120 shutdown_connections 2024-05-09T00:49:54.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.174+0000 7fe57edbcb80 1 -- 172.21.15.5:0/235700120 wait complete. 2024-05-09T00:49:54.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.175+0000 7fe57edbcb80 20 client.25475 trim_cache size 0 max 16384 2024-05-09T00:49:54.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.222+0000 7fe57edbcb80 1 Processor -- start 2024-05-09T00:49:54.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.222+0000 7fe57edbcb80 1 -- start start 2024-05-09T00:49:54.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.223+0000 7fe57edbcb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea0f2a6520 0x55ea0f42c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:54.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.223+0000 7fe57edbcb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea0f320d80 0x55ea0f42d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:54.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.223+0000 7fe57edbcb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea0f423660 0x55ea0f4562e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:54.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.224+0000 7fe56e142700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea0f320d80 0x55ea0f42d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:54.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.224+0000 7fe57edbcb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe55400d590 con 0x55ea0f320d80 2024-05-09T00:49:54.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.224+0000 7fe57edbcb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe55400f0d0 con 0x55ea0f423660 2024-05-09T00:49:54.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.224+0000 7fe57edbcb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe55400dc60 con 0x55ea0f2a6520 2024-05-09T00:49:54.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.224+0000 7fe56d941700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea0f2a6520 0x55ea0f42c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:54.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.224+0000 7fe56d941700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea0f2a6520 0x55ea0f42c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:54978/0 (socket says 172.21.15.5:54978) 2024-05-09T00:49:54.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.224+0000 7fe56d941700 1 -- 172.21.15.5:0/3441981115 learned_addr learned my addr 172.21.15.5:0/3441981115 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:49:54.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.226+0000 7fe56d941700 1 --2- 172.21.15.5:0/3441981115 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea0f423660 0x55ea0f4562e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:54.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.226+0000 7fe56d941700 1 -- 172.21.15.5:0/3441981115 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea0f423660 msgr2=0x55ea0f4562e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:49:54.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.226+0000 7fe56d941700 1 --2- 172.21.15.5:0/3441981115 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea0f423660 0x55ea0f4562e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:54.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.226+0000 7fe56d941700 1 -- 172.21.15.5:0/3441981115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea0f320d80 msgr2=0x55ea0f42d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:49:54.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.226+0000 7fe56d941700 1 --2- 172.21.15.5:0/3441981115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea0f320d80 0x55ea0f42d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:54.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.226+0000 7fe56d941700 1 -- 172.21.15.5:0/3441981115 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea0f428ea0 con 0x55ea0f2a6520 2024-05-09T00:49:54.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.233+0000 7fe56d941700 1 --2- 172.21.15.5:0/3441981115 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea0f2a6520 0x55ea0f42c450 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe560009c20 tx=0x7fe560009f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1841f9f6c638c7a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:49:54.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.236+0000 7fe54ffff700 1 -- 172.21.15.5:0/3441981115 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe560003680 con 0x55ea0f2a6520 2024-05-09T00:49:54.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.236+0000 7fe54ffff700 1 -- 172.21.15.5:0/3441981115 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe560004260 con 0x55ea0f2a6520 2024-05-09T00:49:54.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.236+0000 7fe54ffff700 1 -- 172.21.15.5:0/3441981115 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe560003d00 con 0x55ea0f2a6520 2024-05-09T00:49:54.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.241+0000 7fe57edbcb80 1 -- 172.21.15.5:0/3441981115 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea0f2a6520 msgr2=0x55ea0f42c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:54.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.241+0000 7fe57edbcb80 1 --2- 172.21.15.5:0/3441981115 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea0f2a6520 0x55ea0f42c450 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe560009c20 tx=0x7fe560009f50 comp rx=0 tx=0).stop 2024-05-09T00:49:54.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.243+0000 7fe57edbcb80 1 -- 172.21.15.5:0/3441981115 shutdown_connections 2024-05-09T00:49:54.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.243+0000 7fe57edbcb80 1 --2- 172.21.15.5:0/3441981115 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea0f423660 0x55ea0f4562e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:54.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.243+0000 7fe57edbcb80 1 --2- 172.21.15.5:0/3441981115 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea0f2a6520 0x55ea0f42c450 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:54.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.243+0000 7fe57edbcb80 1 --2- 172.21.15.5:0/3441981115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea0f320d80 0x55ea0f42d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:54.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.243+0000 7fe57edbcb80 1 -- 172.21.15.5:0/3441981115 >> 172.21.15.5:0/3441981115 conn(0x55ea0f2a9ff0 msgr2=0x55ea0f42d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:49:54.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.245+0000 7fe57edbcb80 1 -- 172.21.15.5:0/3441981115 shutdown_connections 2024-05-09T00:49:54.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.245+0000 7fe57edbcb80 1 -- 172.21.15.5:0/3441981115 wait complete. 2024-05-09T00:49:54.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.246+0000 7fe57edbcb80 1 -- start start 2024-05-09T00:49:54.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.246+0000 7fe57edbcb80 1 Processor -- start 2024-05-09T00:49:54.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.247+0000 7fe57edbcb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea0f2a6520 0x55ea0f435560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:54.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.247+0000 7fe57edbcb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea0f320d80 0x55ea0f435aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:54.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.247+0000 7fe57edbcb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea0f423660 0x55ea0f432530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:54.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.247+0000 7fe57edbcb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe55400ea00 con 0x55ea0f423660 2024-05-09T00:49:54.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.247+0000 7fe57edbcb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe55400e330 con 0x55ea0f2a6520 2024-05-09T00:49:54.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.248+0000 7fe56d941700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea0f423660 0x55ea0f432530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:54.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.248+0000 7fe56d941700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea0f423660 0x55ea0f432530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58856/0 (socket says 172.21.15.5:58856) 2024-05-09T00:49:54.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.248+0000 7fe56d941700 1 -- 172.21.15.5:0/117499315 learned_addr learned my addr 172.21.15.5:0/117499315 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:49:54.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.248+0000 7fe57edbcb80 1 -- 172.21.15.5:0/117499315 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe55400b530 con 0x55ea0f320d80 2024-05-09T00:49:54.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.248+0000 7fe56d941700 1 --2- 172.21.15.5:0/117499315 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea0f2a6520 0x55ea0f435560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:54.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.248+0000 7fe57edbcb80 10 client.0 _ll_register_callbacks cb 0x55ea0f1b7740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:49:54.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.248+0000 7fe56e142700 1 --2- 172.21.15.5:0/117499315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea0f320d80 0x55ea0f435aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:54.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.249+0000 7fe56d941700 1 -- 172.21.15.5:0/117499315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea0f320d80 msgr2=0x55ea0f435aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:49:54.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.249+0000 7fe56d941700 1 --2- 172.21.15.5:0/117499315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea0f320d80 0x55ea0f435aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:54.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.250+0000 7fe56d941700 1 -- 172.21.15.5:0/117499315 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea0f2a6520 msgr2=0x55ea0f435560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:49:54.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.250+0000 7fe56d941700 1 --2- 172.21.15.5:0/117499315 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea0f2a6520 0x55ea0f435560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:54.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.250+0000 7fe56d941700 1 -- 172.21.15.5:0/117499315 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea0f432a70 con 0x55ea0f423660 2024-05-09T00:49:54.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.251+0000 7fe56d941700 1 --2- 172.21.15.5:0/117499315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea0f423660 0x55ea0f432530 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fe560009e90 tx=0x7fe560005d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66c221320b1be01a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:49:54.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.252+0000 7fe54dffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:49:54.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.252+0000 7fe54dffb700 1 -- 172.21.15.5:0/117499315 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5600108d0 con 0x55ea0f423660 2024-05-09T00:49:54.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.253+0000 7fe57edbcb80 1 -- 172.21.15.5:0/117499315 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe560004260 con 0x55ea0f423660 2024-05-09T00:49:54.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.253+0000 7fe54dffb700 1 -- 172.21.15.5:0/117499315 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fe560010a70 con 0x55ea0f423660 2024-05-09T00:49:54.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.256+0000 7fe54dffb700 1 -- 172.21.15.5:0/117499315 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe560010c70 con 0x55ea0f423660 2024-05-09T00:49:54.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.256+0000 7fe57edbcb80 10 client.15852 fetch_fsmap learned FSMap version 29 2024-05-09T00:49:54.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.256+0000 7fe57edbcb80 1 -- 172.21.15.5:0/117499315 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ea0f4b95c0 con 0x55ea0f423660 2024-05-09T00:49:54.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.257+0000 7fe54dffb700 1 -- 172.21.15.5:0/117499315 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe560004260 con 0x55ea0f423660 2024-05-09T00:49:54.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.257+0000 7fe54dffb700 1 -- 172.21.15.5:0/117499315 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fe56001e050 con 0x55ea0f423660 2024-05-09T00:49:54.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.257+0000 7fe57edbcb80 10 client.15852 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:49:54.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.257+0000 7fe57edbcb80 10 client.15852 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:49:54.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.257+0000 7fe57edbcb80 20 client.15852 populate_metadata read hostname 'smithi005' 2024-05-09T00:49:54.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.257+0000 7fe57edbcb80 1 --2- 172.21.15.5:0/117499315 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55ea0f4b9b00 0x55ea0f4bbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:49:54.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.258+0000 7fe57edbcb80 4 client.15852 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T00:49:54.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.258+0000 7fe57edbcb80 1 -- 172.21.15.5:0/117499315 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ea0f34dba0 con 0x55ea0f4b9b00 2024-05-09T00:49:54.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.263+0000 7fe56e142700 1 --2- 172.21.15.5:0/117499315 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55ea0f4b9b00 0x55ea0f4bbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:49:54.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.274+0000 7fe54dffb700 10 client.15852 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:49:54.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:54.274+0000 7fe56e142700 1 --2- 172.21.15.5:0/117499315 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55ea0f4b9b00 0x55ea0f4bbf20 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5b73faf2c91acc2 server_cookie=44c1a22ff5f8bf00 in_seq=0 out_seq=0 2024-05-09T00:49:54.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:49:54 smithi131 ceph-mon[34547]: pgmap v478: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 848 MiB/s wr, 1.78k op/s 2024-05-09T00:49:54.935 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:49:54 smithi157 ceph-mon[34294]: pgmap v478: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 848 MiB/s wr, 1.78k op/s 2024-05-09T00:49:55.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:49:54 smithi005 ceph-mon[30365]: pgmap v478: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 848 MiB/s wr, 1.78k op/s 2024-05-09T00:49:55.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:49:55 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:49:55.935 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:49:55 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:49:56.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:56.034+0000 7fe54dffb700 1 -- 172.21.15.5:0/117499315 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ea0f34dba0 con 0x55ea0f4b9b00 2024-05-09T00:49:56.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:56.034+0000 7fe54dffb700 10 client.15852 handle_command_reply: tid=0 2024-05-09T00:49:56.035 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:49:55 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:49:56.037 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-09T00:49:56.038 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-05-09T00:49:56.038 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "333ca816-f4a1-453d-83f5-cb5fd271b1a5", 2024-05-09T00:49:56.038 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-05-09T00:49:56.038 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-09T00:49:56.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:56.034+0000 7fe57edbcb80 1 client.15852 shutdown 2024-05-09T00:49:56.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:56.034+0000 7fe57edbcb80 1 -- 172.21.15.5:0/117499315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea0f423660 msgr2=0x55ea0f432530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:56.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:56.034+0000 7fe57edbcb80 1 --2- 172.21.15.5:0/117499315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea0f423660 0x55ea0f432530 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fe560009e90 tx=0x7fe560005d20 comp rx=0 tx=0).stop 2024-05-09T00:49:56.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:56.034+0000 7fe57edbcb80 1 -- 172.21.15.5:0/117499315 shutdown_connections 2024-05-09T00:49:56.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:56.035+0000 7fe57edbcb80 1 --2- 172.21.15.5:0/117499315 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55ea0f4b9b00 0x55ea0f4bbf20 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:56.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:56.035+0000 7fe57edbcb80 1 --2- 172.21.15.5:0/117499315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea0f423660 0x55ea0f432530 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:56.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:56.035+0000 7fe57edbcb80 1 --2- 172.21.15.5:0/117499315 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea0f2a6520 0x55ea0f435560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:56.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:56.035+0000 7fe57edbcb80 1 --2- 172.21.15.5:0/117499315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea0f320d80 0x55ea0f435aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:56.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:56.035+0000 7fe57edbcb80 1 -- 172.21.15.5:0/117499315 >> 172.21.15.5:0/117499315 conn(0x55ea0f2a9ff0 msgr2=0x55ea0f42c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:49:56.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:56.035+0000 7fe54dffb700 0 client.15852 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:49:56.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:56.035+0000 7fe57edbcb80 1 -- 172.21.15.5:0/117499315 shutdown_connections 2024-05-09T00:49:56.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:56.035+0000 7fe57edbcb80 1 -- 172.21.15.5:0/117499315 wait complete. 2024-05-09T00:49:56.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:56.035+0000 7fe57edbcb80 20 client.15852 trim_cache size 0 max 16384 2024-05-09T00:49:56.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:56.042+0000 7fe56f144700 1 -- 172.21.15.5:0/88647321 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe5500b2a90 msgr2=0x7fe5500b4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:56.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:56.043+0000 7fe56f144700 1 --2- 172.21.15.5:0/88647321 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe5500b2a90 0x7fe5500b4fa0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe55800b3f0 tx=0x7fe558009040 comp rx=0 tx=0).stop 2024-05-09T00:49:56.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:56.043+0000 7fe56f144700 1 -- 172.21.15.5:0/88647321 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe5681084d0 msgr2=0x7fe568109ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:49:56.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:56.043+0000 7fe56f144700 1 --2- 172.21.15.5:0/88647321 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe5681084d0 0x7fe568109ae0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe558004fb0 tx=0x7fe55800cf20 comp rx=0 tx=0).stop 2024-05-09T00:49:56.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:56.043+0000 7fe56f144700 1 -- 172.21.15.5:0/88647321 shutdown_connections 2024-05-09T00:49:56.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:56.043+0000 7fe56f144700 1 --2- 172.21.15.5:0/88647321 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe5500b2a90 0x7fe5500b4fa0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:56.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:56.043+0000 7fe56f144700 1 --2- 172.21.15.5:0/88647321 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe5681084d0 0x7fe568109ae0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:56.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:56.043+0000 7fe56f144700 1 --2- 172.21.15.5:0/88647321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe568103080 0x7fe568111530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:56.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:56.044+0000 7fe56f144700 1 --2- 172.21.15.5:0/88647321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5681039d0 0x7fe5681095a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:49:56.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:56.044+0000 7fe56f144700 1 -- 172.21.15.5:0/88647321 >> 172.21.15.5:0/88647321 conn(0x7fe568078ac0 msgr2=0x7fe568105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:49:56.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:56.047+0000 7fe56f144700 1 -- 172.21.15.5:0/88647321 shutdown_connections 2024-05-09T00:49:56.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:49:56.047+0000 7fe56f144700 1 -- 172.21.15.5:0/88647321 wait complete. 2024-05-09T00:49:56.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:49:56 smithi131 ceph-mon[34547]: pgmap v479: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 980 MiB/s wr, 1.98k op/s 2024-05-09T00:49:56.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:49:56 smithi131 ceph-mon[34547]: scrub queued for path: / 2024-05-09T00:49:56.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:49:56 smithi131 ceph-mon[34547]: scrub summary: idle+waiting paths [/] 2024-05-09T00:49:56.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:49:56 smithi131 ceph-mon[34547]: scrub summary: active paths [/] 2024-05-09T00:49:56.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:49:56 smithi157 ceph-mon[34294]: pgmap v479: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 980 MiB/s wr, 1.98k op/s 2024-05-09T00:49:56.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:49:56 smithi157 ceph-mon[34294]: scrub queued for path: / 2024-05-09T00:49:56.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:49:56 smithi157 ceph-mon[34294]: scrub summary: idle+waiting paths [/] 2024-05-09T00:49:56.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:49:56 smithi157 ceph-mon[34294]: scrub summary: active paths [/] 2024-05-09T00:49:57.043 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:49:56 smithi005 ceph-mon[30365]: pgmap v479: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 980 MiB/s wr, 1.98k op/s 2024-05-09T00:49:57.043 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:49:56 smithi005 ceph-mon[30365]: scrub queued for path: / 2024-05-09T00:49:57.043 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:49:56 smithi005 ceph-mon[30365]: scrub summary: idle+waiting paths [/] 2024-05-09T00:49:57.043 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:49:56 smithi005 ceph-mon[30365]: scrub summary: active paths [/] 2024-05-09T00:49:57.761 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:49:57 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:49:57.761 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:49:57 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:49:57.761 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:49:57 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:49:58.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:49:57 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:49:58.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:49:57 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:49:58.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:49:57 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:49:58.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:49:57 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:49:58.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:49:57 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:49:58.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:49:57 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:49:58.438 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T00:49:59.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:49:58 smithi005 ceph-mon[30365]: pgmap v480: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 843 MiB/s wr, 1.67k op/s 2024-05-09T00:49:59.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:49:58 smithi131 ceph-mon[34547]: pgmap v480: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 843 MiB/s wr, 1.67k op/s 2024-05-09T00:49:59.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:49:58 smithi157 ceph-mon[34294]: pgmap v480: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 843 MiB/s wr, 1.67k op/s 2024-05-09T00:49:59.197 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:50:00.210 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:50:00 smithi005 ceph-mon[30365]: overall HEALTH_OK 2024-05-09T00:50:00.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:50:00 smithi131 ceph-mon[34547]: overall HEALTH_OK 2024-05-09T00:50:00.435 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:50:00 smithi157 ceph-mon[34294]: overall HEALTH_OK 2024-05-09T00:50:01.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:50:01 smithi131 ceph-mon[34547]: pgmap v481: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 983 MiB/s wr, 2.36k op/s 2024-05-09T00:50:01.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:50:01 smithi157 ceph-mon[34294]: pgmap v481: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 983 MiB/s wr, 2.36k op/s 2024-05-09T00:50:01.479 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:50:01 smithi005 ceph-mon[30365]: pgmap v481: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 983 MiB/s wr, 2.36k op/s 2024-05-09T00:50:02.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:50:02 smithi131 ceph-mon[34547]: scrub summary: idle+waiting paths [/] 2024-05-09T00:50:02.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:50:02 smithi157 ceph-mon[34294]: scrub summary: idle+waiting paths [/] 2024-05-09T00:50:02.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:50:02 smithi005 ceph-mon[30365]: scrub summary: idle+waiting paths [/] 2024-05-09T00:50:03.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:50:03 smithi131 ceph-mon[34547]: pgmap v482: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 849 MiB/s wr, 2.26k op/s 2024-05-09T00:50:03.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:50:03 smithi157 ceph-mon[34294]: pgmap v482: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 849 MiB/s wr, 2.26k op/s 2024-05-09T00:50:03.543 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:50:03 smithi005 ceph-mon[30365]: pgmap v482: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 849 MiB/s wr, 2.26k op/s 2024-05-09T00:50:04.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.514+0000 7fc001f71700 1 -- 172.21.15.5:0/2239368930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbffc107c10 msgr2=0x7fbffc108020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:50:04.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.514+0000 7fc001f71700 1 --2- 172.21.15.5:0/2239368930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbffc107c10 0x7fbffc108020 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fbff400b0a0 tx=0x7fbff400b3b0 comp rx=0 tx=0).stop 2024-05-09T00:50:04.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.520+0000 7fc001f71700 1 -- 172.21.15.5:0/2239368930 shutdown_connections 2024-05-09T00:50:04.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.520+0000 7fc001f71700 1 --2- 172.21.15.5:0/2239368930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbffc108560 0x7fbffc10fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:04.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.520+0000 7fc001f71700 1 --2- 172.21.15.5:0/2239368930 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbffc1032e0 0x7fbffc1036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:04.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.520+0000 7fc001f71700 1 --2- 172.21.15.5:0/2239368930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbffc107c10 0x7fbffc108020 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:04.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.520+0000 7fc001f71700 1 -- 172.21.15.5:0/2239368930 >> 172.21.15.5:0/2239368930 conn(0x7fbffc078ac0 msgr2=0x7fbffc078f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:50:04.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.523+0000 7fc001f71700 1 -- 172.21.15.5:0/2239368930 shutdown_connections 2024-05-09T00:50:04.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.523+0000 7fc001f71700 1 -- 172.21.15.5:0/2239368930 wait complete. 2024-05-09T00:50:04.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.525+0000 7fc001f71700 1 Processor -- start 2024-05-09T00:50:04.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.525+0000 7fc001f71700 1 -- start start 2024-05-09T00:50:04.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.526+0000 7fc001f71700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbffc1032e0 0x7fbffc112070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:04.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.526+0000 7fc001f71700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbffc107c10 0x7fbffc1104d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:04.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.526+0000 7fc001f71700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbffc108560 0x7fbffc110ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:04.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.526+0000 7fc001f71700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc114400 con 0x7fbffc107c10 2024-05-09T00:50:04.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.526+0000 7fc001f71700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc114280 con 0x7fbffc1032e0 2024-05-09T00:50:04.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.526+0000 7fc001f71700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc114580 con 0x7fbffc108560 2024-05-09T00:50:04.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.527+0000 7fc000f6f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbffc1032e0 0x7fbffc112070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:04.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.527+0000 7fc000f6f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbffc1032e0 0x7fbffc112070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:46904/0 (socket says 172.21.15.5:46904) 2024-05-09T00:50:04.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.527+0000 7fc000f6f700 1 -- 172.21.15.5:0/118268331 learned_addr learned my addr 172.21.15.5:0/118268331 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:50:04.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.527+0000 7fbffbfff700 1 --2- 172.21.15.5:0/118268331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbffc107c10 0x7fbffc1104d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:04.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.527+0000 7fc001770700 1 --2- 172.21.15.5:0/118268331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbffc108560 0x7fbffc110ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:04.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.528+0000 7fc000f6f700 1 -- 172.21.15.5:0/118268331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbffc108560 msgr2=0x7fbffc110ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:50:04.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.528+0000 7fc000f6f700 1 --2- 172.21.15.5:0/118268331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbffc108560 0x7fbffc110ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:04.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.528+0000 7fc000f6f700 1 -- 172.21.15.5:0/118268331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbffc107c10 msgr2=0x7fbffc1104d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:50:04.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.528+0000 7fc000f6f700 1 --2- 172.21.15.5:0/118268331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbffc107c10 0x7fbffc1104d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:04.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.528+0000 7fc000f6f700 1 -- 172.21.15.5:0/118268331 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbffc1112e0 con 0x7fbffc1032e0 2024-05-09T00:50:04.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.529+0000 7fc000f6f700 1 --2- 172.21.15.5:0/118268331 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbffc1032e0 0x7fbffc112070 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbfec00b530 tx=0x7fbfec00b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:50:04.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.529+0000 7fbff9ffb700 1 -- 172.21.15.5:0/118268331 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfec015020 con 0x7fbffc1032e0 2024-05-09T00:50:04.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.529+0000 7fbff9ffb700 1 -- 172.21.15.5:0/118268331 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfec011ce0 con 0x7fbffc1032e0 2024-05-09T00:50:04.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.529+0000 7fc001f71700 1 -- 172.21.15.5:0/118268331 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbffc1ad7c0 con 0x7fbffc1032e0 2024-05-09T00:50:04.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.529+0000 7fbff9ffb700 1 -- 172.21.15.5:0/118268331 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfec010e40 con 0x7fbffc1032e0 2024-05-09T00:50:04.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.530+0000 7fc001f71700 1 -- 172.21.15.5:0/118268331 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbffc1add20 con 0x7fbffc1032e0 2024-05-09T00:50:04.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.538+0000 7fc011be9b80 1 Processor -- start 2024-05-09T00:50:04.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.538+0000 7fc011be9b80 1 -- start start 2024-05-09T00:50:04.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.539+0000 7fc011be9b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b90cafad00 0x55b90cc03490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:04.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.539+0000 7fc011be9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b90ca804a0 0x55b90cc039d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:04.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.539+0000 7fc011be9b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b90cbfd5e0 0x55b90cbfd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:04.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.539+0000 7fc011be9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b90cb200a0 con 0x55b90ca804a0 2024-05-09T00:50:04.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.539+0000 7fc011be9b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55b90cac59a0 con 0x55b90cafad00 2024-05-09T00:50:04.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.539+0000 7fc011be9b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b90cb1c060 con 0x55b90cbfd5e0 2024-05-09T00:50:04.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.539+0000 7fbff9ffb700 1 -- 172.21.15.5:0/118268331 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fbfec018020 con 0x7fbffc1032e0 2024-05-09T00:50:04.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.539+0000 7fc000f6f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b90ca804a0 0x55b90cc039d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:04.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.539+0000 7fc001770700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b90cafad00 0x55b90cc03490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:04.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.539+0000 7fc001770700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b90cafad00 0x55b90cc03490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:46910/0 (socket says 172.21.15.5:46910) 2024-05-09T00:50:04.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.539+0000 7fc001770700 1 -- 172.21.15.5:0/2669194913 learned_addr learned my addr 172.21.15.5:0/2669194913 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:50:04.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.540+0000 7fc000f6f700 1 -- 172.21.15.5:0/2669194913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b90cbfd5e0 msgr2=0x55b90cbfd9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:50:04.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.540+0000 7fc000f6f700 1 --2- 172.21.15.5:0/2669194913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b90cbfd5e0 0x55b90cbfd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:04.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.540+0000 7fc000f6f700 1 -- 172.21.15.5:0/2669194913 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b90cafad00 msgr2=0x55b90cc03490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:50:04.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.540+0000 7fc000f6f700 1 --2- 172.21.15.5:0/2669194913 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b90cafad00 0x55b90cc03490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:04.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.540+0000 7fc000f6f700 1 -- 172.21.15.5:0/2669194913 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b90cab6500 con 0x55b90ca804a0 2024-05-09T00:50:04.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.540+0000 7fc000f6f700 1 --2- 172.21.15.5:0/2669194913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b90ca804a0 0x55b90cc039d0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fbfec0df450 tx=0x7fbfec0e7ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb00fc4caf677455 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:50:04.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.541+0000 7fbfe2ffd700 1 -- 172.21.15.5:0/2669194913 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfec0f0070 con 0x55b90ca804a0 2024-05-09T00:50:04.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.541+0000 7fbff9ffb700 1 --2- 172.21.15.5:0/118268331 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fbfe40b29b0 0x7fbfe40b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:04.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.541+0000 7fbfe2ffd700 1 -- 172.21.15.5:0/2669194913 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfec0a8ae0 con 0x55b90ca804a0 2024-05-09T00:50:04.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.541+0000 7fbfe2ffd700 1 -- 172.21.15.5:0/2669194913 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfec0a96d0 con 0x55b90ca804a0 2024-05-09T00:50:04.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.542+0000 7fbff9ffb700 1 -- 172.21.15.5:0/118268331 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fbfec0e1a40 con 0x7fbffc1032e0 2024-05-09T00:50:04.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.542+0000 7fc001770700 1 --2- 172.21.15.5:0/118268331 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fbfe40b29b0 0x7fbfe40b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:04.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.542+0000 7fc011be9b80 1 -- 172.21.15.5:0/2669194913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b90ca804a0 msgr2=0x55b90cc039d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:50:04.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.542+0000 7fc011be9b80 1 --2- 172.21.15.5:0/2669194913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b90ca804a0 0x55b90cc039d0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fbfec0df450 tx=0x7fbfec0e7ed0 comp rx=0 tx=0).stop 2024-05-09T00:50:04.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.543+0000 7fc011be9b80 1 -- 172.21.15.5:0/2669194913 shutdown_connections 2024-05-09T00:50:04.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.543+0000 7fc011be9b80 1 --2- 172.21.15.5:0/2669194913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b90cbfd5e0 0x55b90cbfd9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:04.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.543+0000 7fc011be9b80 1 --2- 172.21.15.5:0/2669194913 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b90cafad00 0x55b90cc03490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:04.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.543+0000 7fc011be9b80 1 --2- 172.21.15.5:0/2669194913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b90ca804a0 0x55b90cc039d0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:04.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.543+0000 7fc011be9b80 1 -- 172.21.15.5:0/2669194913 >> 172.21.15.5:0/2669194913 conn(0x55b90ca83f70 msgr2=0x55b90cbf87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:50:04.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.543+0000 7fc011be9b80 1 -- 172.21.15.5:0/2669194913 shutdown_connections 2024-05-09T00:50:04.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.543+0000 7fc011be9b80 1 -- 172.21.15.5:0/2669194913 wait complete. 2024-05-09T00:50:04.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.544+0000 7fc001770700 1 --2- 172.21.15.5:0/118268331 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fbfe40b29b0 0x7fbfe40b4ec0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x55b90cbfe7d0 tx=0x7fbff0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:50:04.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.544+0000 7fc011be9b80 1 -- start start 2024-05-09T00:50:04.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.545+0000 7fc011be9b80 1 Processor -- start 2024-05-09T00:50:04.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.546+0000 7fc011be9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b90ca804a0 0x55b90cc07110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:04.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.546+0000 7fc011be9b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b90cafad00 0x55b90cc07650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:04.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.546+0000 7fc011be9b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b90cbfd5e0 0x55b90cc07b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:04.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.546+0000 7fc011be9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b90cbfe390 con 0x55b90ca804a0 2024-05-09T00:50:04.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.546+0000 7fc011be9b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55b90cabab20 con 0x55b90cafad00 2024-05-09T00:50:04.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.546+0000 7fc011be9b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b90cc0a200 con 0x55b90cbfd5e0 2024-05-09T00:50:04.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.546+0000 7fc011be9b80 10 client.0 _ll_register_callbacks cb 0x55b90c9916c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:50:04.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.546+0000 7fc000f6f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b90ca804a0 0x55b90cc07110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:04.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.546+0000 7fc000f6f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b90ca804a0 0x55b90cc07110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46812/0 (socket says 172.21.15.5:46812) 2024-05-09T00:50:04.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.546+0000 7fc000f6f700 1 -- 172.21.15.5:0/1417842092 learned_addr learned my addr 172.21.15.5:0/1417842092 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:50:04.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.547+0000 7fc000f6f700 1 -- 172.21.15.5:0/1417842092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b90cbfd5e0 msgr2=0x55b90cc07b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:50:04.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.547+0000 7fc000f6f700 1 --2- 172.21.15.5:0/1417842092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b90cbfd5e0 0x55b90cc07b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:04.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.547+0000 7fc000f6f700 1 -- 172.21.15.5:0/1417842092 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b90cafad00 msgr2=0x55b90cc07650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:50:04.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.547+0000 7fc000f6f700 1 --2- 172.21.15.5:0/1417842092 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b90cafad00 0x55b90cc07650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:04.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.548+0000 7fc000f6f700 1 -- 172.21.15.5:0/1417842092 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b90c99b490 con 0x55b90ca804a0 2024-05-09T00:50:04.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.549+0000 7fbfdbfff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:50:04.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.549+0000 7fc000f6f700 1 --2- 172.21.15.5:0/1417842092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b90ca804a0 0x55b90cc07110 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fbfec00bc40 tx=0x7fbfec0e7780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce1f25de0564f017 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:50:04.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.550+0000 7fbfdbfff700 1 -- 172.21.15.5:0/1417842092 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfec0f0070 con 0x55b90ca804a0 2024-05-09T00:50:04.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.550+0000 7fbfdbfff700 1 -- 172.21.15.5:0/1417842092 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fbfec0a55c0 con 0x55b90ca804a0 2024-05-09T00:50:04.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.551+0000 7fc011be9b80 1 -- 172.21.15.5:0/1417842092 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfec0a8ae0 con 0x55b90ca804a0 2024-05-09T00:50:04.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.552+0000 7fbfdbfff700 1 -- 172.21.15.5:0/1417842092 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbfec0f0070 con 0x55b90ca804a0 2024-05-09T00:50:04.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.552+0000 7fc011be9b80 10 client.15864 fetch_fsmap learned FSMap version 29 2024-05-09T00:50:04.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.552+0000 7fc011be9b80 1 -- 172.21.15.5:0/1417842092 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b90cc14b70 con 0x55b90ca804a0 2024-05-09T00:50:04.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.553+0000 7fbfdbfff700 1 -- 172.21.15.5:0/1417842092 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfec0a8ae0 con 0x55b90ca804a0 2024-05-09T00:50:04.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.554+0000 7fbfdbfff700 1 -- 172.21.15.5:0/1417842092 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fbfec0a5260 con 0x55b90ca804a0 2024-05-09T00:50:04.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.554+0000 7fc011be9b80 10 client.15864 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:50:04.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.554+0000 7fc011be9b80 10 client.15864 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:50:04.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.554+0000 7fc011be9b80 20 client.15864 populate_metadata read hostname 'smithi005' 2024-05-09T00:50:04.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.554+0000 7fc011be9b80 1 --2- 172.21.15.5:0/1417842092 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55b90cc17180 0x55b90cc19650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:04.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.554+0000 7fc011be9b80 4 client.15864 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:50:04.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.554+0000 7fc011be9b80 1 -- 172.21.15.5:0/1417842092 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b90cb2da80 con 0x55b90cc17180 2024-05-09T00:50:04.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.555+0000 7fbffbfff700 1 --2- 172.21.15.5:0/1417842092 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55b90cc17180 0x55b90cc19650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:04.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.557+0000 7fbffbfff700 1 --2- 172.21.15.5:0/1417842092 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55b90cc17180 0x55b90cc19650 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3a06af8631e88b7 server_cookie=46cdf7b45d4c0541 in_seq=0 out_seq=0 2024-05-09T00:50:04.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.558+0000 7fbfdbfff700 10 client.15864 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:50:04.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.566+0000 7fbfdbfff700 1 -- 172.21.15.5:0/1417842092 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b90cb2da80 con 0x55b90cc17180 2024-05-09T00:50:04.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.566+0000 7fbfdbfff700 10 client.15864 handle_command_reply: tid=0 2024-05-09T00:50:04.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.566+0000 7fc011be9b80 1 client.15864 shutdown 2024-05-09T00:50:04.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.566+0000 7fc011be9b80 1 -- 172.21.15.5:0/1417842092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b90ca804a0 msgr2=0x55b90cc07110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:50:04.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.566+0000 7fc011be9b80 1 --2- 172.21.15.5:0/1417842092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b90ca804a0 0x55b90cc07110 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fbfec00bc40 tx=0x7fbfec0e7780 comp rx=0 tx=0).stop 2024-05-09T00:50:04.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.566+0000 7fc011be9b80 1 -- 172.21.15.5:0/1417842092 shutdown_connections 2024-05-09T00:50:04.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.566+0000 7fc011be9b80 1 --2- 172.21.15.5:0/1417842092 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55b90cc17180 0x55b90cc19650 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:04.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.566+0000 7fc011be9b80 1 --2- 172.21.15.5:0/1417842092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b90cbfd5e0 0x55b90cc07b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:04.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.566+0000 7fc011be9b80 1 --2- 172.21.15.5:0/1417842092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b90ca804a0 0x55b90cc07110 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:04.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.566+0000 7fc011be9b80 1 --2- 172.21.15.5:0/1417842092 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b90cafad00 0x55b90cc07650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:04.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.566+0000 7fc011be9b80 1 -- 172.21.15.5:0/1417842092 >> 172.21.15.5:0/1417842092 conn(0x55b90ca83f70 msgr2=0x55b90cbf82d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:50:04.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.566+0000 7fbfdbfff700 0 client.15864 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:50:04.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.567+0000 7fc011be9b80 1 -- 172.21.15.5:0/1417842092 shutdown_connections 2024-05-09T00:50:04.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.567+0000 7fc011be9b80 1 -- 172.21.15.5:0/1417842092 wait complete. 2024-05-09T00:50:04.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.567+0000 7fc011be9b80 20 client.15864 trim_cache size 0 max 16384 2024-05-09T00:50:04.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.640+0000 7fc011be9b80 1 Processor -- start 2024-05-09T00:50:04.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.642+0000 7fc011be9b80 1 -- start start 2024-05-09T00:50:04.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.642+0000 7fc011be9b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b90ca804a0 0x55b90cbfce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:04.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.642+0000 7fc011be9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b90cafad00 0x55b90cc130d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:04.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.642+0000 7fc011be9b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b90cbfd5e0 0x55b90cbffde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:04.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.642+0000 7fc011be9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc1b53f0 con 0x55b90cafad00 2024-05-09T00:50:04.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.642+0000 7fc011be9b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc1b6f30 con 0x55b90ca804a0 2024-05-09T00:50:04.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.642+0000 7fc011be9b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc1b5ac0 con 0x55b90cbfd5e0 2024-05-09T00:50:04.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.647+0000 7fbffbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b90cafad00 0x55b90cc130d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:04.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.650+0000 7fc000f6f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b90ca804a0 0x55b90cbfce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:04.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.650+0000 7fc000f6f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b90ca804a0 0x55b90cbfce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:46920/0 (socket says 172.21.15.5:46920) 2024-05-09T00:50:04.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.650+0000 7fc000f6f700 1 -- 172.21.15.5:0/807445699 learned_addr learned my addr 172.21.15.5:0/807445699 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:50:04.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.651+0000 7fc001770700 1 --2- 172.21.15.5:0/807445699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b90cbfd5e0 0x55b90cbffde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:04.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.655+0000 7fc000f6f700 1 -- 172.21.15.5:0/807445699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b90cbfd5e0 msgr2=0x55b90cbffde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:50:04.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.655+0000 7fc000f6f700 1 --2- 172.21.15.5:0/807445699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b90cbfd5e0 0x55b90cbffde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:04.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.655+0000 7fc000f6f700 1 -- 172.21.15.5:0/807445699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b90cafad00 msgr2=0x55b90cc130d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:50:04.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.655+0000 7fc000f6f700 1 --2- 172.21.15.5:0/807445699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b90cafad00 0x55b90cc130d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:04.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.655+0000 7fc000f6f700 1 -- 172.21.15.5:0/807445699 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b90cbf90d0 con 0x55b90ca804a0 2024-05-09T00:50:04.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.660+0000 7fc000f6f700 1 --2- 172.21.15.5:0/807445699 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b90ca804a0 0x55b90cbfce20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbfec0a6240 tx=0x7fbfec0e7ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f30cc15f48a78f8d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:50:04.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.664+0000 7fbfe27fc700 1 -- 172.21.15.5:0/807445699 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfec0f0070 con 0x55b90ca804a0 2024-05-09T00:50:04.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.664+0000 7fbfe27fc700 1 -- 172.21.15.5:0/807445699 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfec0a46a0 con 0x55b90ca804a0 2024-05-09T00:50:04.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.664+0000 7fbfe27fc700 1 -- 172.21.15.5:0/807445699 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfec0a8ae0 con 0x55b90ca804a0 2024-05-09T00:50:04.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.666+0000 7fc011be9b80 1 -- 172.21.15.5:0/807445699 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b90ca804a0 msgr2=0x55b90cbfce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:50:04.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.666+0000 7fc011be9b80 1 --2- 172.21.15.5:0/807445699 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b90ca804a0 0x55b90cbfce20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbfec0a6240 tx=0x7fbfec0e7ed0 comp rx=0 tx=0).stop 2024-05-09T00:50:04.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.669+0000 7fc011be9b80 1 -- 172.21.15.5:0/807445699 shutdown_connections 2024-05-09T00:50:04.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.669+0000 7fc011be9b80 1 --2- 172.21.15.5:0/807445699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b90cbfd5e0 0x55b90cbffde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:04.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.669+0000 7fc011be9b80 1 --2- 172.21.15.5:0/807445699 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b90ca804a0 0x55b90cbfce20 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:04.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.669+0000 7fc011be9b80 1 --2- 172.21.15.5:0/807445699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b90cafad00 0x55b90cc130d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:04.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.669+0000 7fc011be9b80 1 -- 172.21.15.5:0/807445699 >> 172.21.15.5:0/807445699 conn(0x55b90ca83f70 msgr2=0x55b90cbf8270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:50:04.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.671+0000 7fc011be9b80 1 -- 172.21.15.5:0/807445699 shutdown_connections 2024-05-09T00:50:04.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.673+0000 7fc011be9b80 1 -- 172.21.15.5:0/807445699 wait complete. 2024-05-09T00:50:04.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.674+0000 7fc011be9b80 1 -- start start 2024-05-09T00:50:04.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.674+0000 7fc011be9b80 1 Processor -- start 2024-05-09T00:50:04.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.677+0000 7fc011be9b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b90ca804a0 0x55b90cc0b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:04.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.677+0000 7fc011be9b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b90cafad00 0x55b90cc0b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:04.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.677+0000 7fc011be9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b90cbfd5e0 0x55b90cc0bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:04.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.679+0000 7fc011be9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc1b6860 con 0x55b90cbfd5e0 2024-05-09T00:50:04.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.679+0000 7fc011be9b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc1b6190 con 0x55b90ca804a0 2024-05-09T00:50:04.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.679+0000 7fc011be9b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc1b3330 con 0x55b90cafad00 2024-05-09T00:50:04.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.679+0000 7fc011be9b80 10 client.0 _ll_register_callbacks cb 0x55b90c9916c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:50:04.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.680+0000 7fc001770700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b90cbfd5e0 0x55b90cc0bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:04.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.680+0000 7fbffbfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b90cafad00 0x55b90cc0b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:04.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.680+0000 7fc001770700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b90cbfd5e0 0x55b90cc0bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46824/0 (socket says 172.21.15.5:46824) 2024-05-09T00:50:04.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.680+0000 7fc001770700 1 -- 172.21.15.5:0/4277434378 learned_addr learned my addr 172.21.15.5:0/4277434378 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:50:04.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.682+0000 7fc001770700 1 -- 172.21.15.5:0/4277434378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b90cafad00 msgr2=0x55b90cc0b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:50:04.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.682+0000 7fc001770700 1 --2- 172.21.15.5:0/4277434378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b90cafad00 0x55b90cc0b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:04.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.682+0000 7fc001770700 1 -- 172.21.15.5:0/4277434378 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b90ca804a0 msgr2=0x55b90cc0b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:50:04.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.682+0000 7fc001770700 1 --2- 172.21.15.5:0/4277434378 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b90ca804a0 0x55b90cc0b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:04.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.682+0000 7fc001770700 1 -- 172.21.15.5:0/4277434378 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b90cc80a50 con 0x55b90cbfd5e0 2024-05-09T00:50:04.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.685+0000 7fc001770700 1 --2- 172.21.15.5:0/4277434378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b90cbfd5e0 0x55b90cc0bde0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fbff0015700 tx=0x7fbff00157e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff2b4cea6ae41329 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:50:04.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.687+0000 7fbfdbfff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:50:04.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.687+0000 7fbfdbfff700 1 -- 172.21.15.5:0/4277434378 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff0024020 con 0x55b90cbfd5e0 2024-05-09T00:50:04.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.687+0000 7fc011be9b80 1 -- 172.21.15.5:0/4277434378 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b90cb27b20 con 0x55b90cbfd5e0 2024-05-09T00:50:04.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.687+0000 7fbfdbfff700 1 -- 172.21.15.5:0/4277434378 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fbff0020440 con 0x55b90cbfd5e0 2024-05-09T00:50:04.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.689+0000 7fbfdbfff700 1 -- 172.21.15.5:0/4277434378 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbff0005a50 con 0x55b90cbfd5e0 2024-05-09T00:50:04.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.689+0000 7fc011be9b80 10 client.15870 fetch_fsmap learned FSMap version 29 2024-05-09T00:50:04.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.689+0000 7fc011be9b80 1 -- 172.21.15.5:0/4277434378 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b90cc81cc0 con 0x55b90cbfd5e0 2024-05-09T00:50:04.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.691+0000 7fbfdbfff700 1 -- 172.21.15.5:0/4277434378 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b90cb27b20 con 0x55b90cbfd5e0 2024-05-09T00:50:04.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.691+0000 7fbfdbfff700 1 -- 172.21.15.5:0/4277434378 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fbff0006020 con 0x55b90cbfd5e0 2024-05-09T00:50:04.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.693+0000 7fc011be9b80 10 client.15870 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:50:04.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.693+0000 7fc011be9b80 10 client.15870 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:50:04.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.693+0000 7fc011be9b80 20 client.15870 populate_metadata read hostname 'smithi005' 2024-05-09T00:50:04.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.693+0000 7fc011be9b80 1 --2- 172.21.15.5:0/4277434378 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55b90cc823a0 0x55b90cc847c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:04.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.693+0000 7fc011be9b80 4 client.15870 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:50:04.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.693+0000 7fc011be9b80 1 -- 172.21.15.5:0/4277434378 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b90c99b970 con 0x55b90cc823a0 2024-05-09T00:50:04.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.698+0000 7fc000f6f700 1 --2- 172.21.15.5:0/4277434378 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55b90cc823a0 0x55b90cc847c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:04.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.706+0000 7fc000f6f700 1 --2- 172.21.15.5:0/4277434378 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55b90cc823a0 0x55b90cc847c0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e99201853b18a46 server_cookie=6f9f55204c9ff7a3 in_seq=0 out_seq=0 2024-05-09T00:50:04.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.707+0000 7fbfdbfff700 10 client.15870 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:50:04.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.711+0000 7fbfdbfff700 1 -- 172.21.15.5:0/4277434378 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b90c99b970 con 0x55b90cc823a0 2024-05-09T00:50:04.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.711+0000 7fbfdbfff700 10 client.15870 handle_command_reply: tid=0 2024-05-09T00:50:04.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.711+0000 7fc011be9b80 1 client.15870 shutdown 2024-05-09T00:50:04.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.712+0000 7fc011be9b80 1 -- 172.21.15.5:0/4277434378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b90cbfd5e0 msgr2=0x55b90cc0bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:50:04.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.712+0000 7fc011be9b80 1 --2- 172.21.15.5:0/4277434378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b90cbfd5e0 0x55b90cc0bde0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fbff0015700 tx=0x7fbff00157e0 comp rx=0 tx=0).stop 2024-05-09T00:50:04.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.712+0000 7fc011be9b80 1 -- 172.21.15.5:0/4277434378 shutdown_connections 2024-05-09T00:50:04.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.712+0000 7fc011be9b80 1 --2- 172.21.15.5:0/4277434378 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55b90cc823a0 0x55b90cc847c0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:04.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.712+0000 7fc011be9b80 1 --2- 172.21.15.5:0/4277434378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b90cbfd5e0 0x55b90cc0bde0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:04.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.712+0000 7fc011be9b80 1 --2- 172.21.15.5:0/4277434378 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b90ca804a0 0x55b90cc0b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:04.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.712+0000 7fc011be9b80 1 --2- 172.21.15.5:0/4277434378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b90cafad00 0x55b90cc0b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:04.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.712+0000 7fbfdbfff700 0 client.15870 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:50:04.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.712+0000 7fc011be9b80 1 -- 172.21.15.5:0/4277434378 >> 172.21.15.5:0/4277434378 conn(0x55b90ca83f70 msgr2=0x55b90cc00de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:50:04.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.713+0000 7fc011be9b80 1 -- 172.21.15.5:0/4277434378 shutdown_connections 2024-05-09T00:50:04.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.713+0000 7fc011be9b80 1 -- 172.21.15.5:0/4277434378 wait complete. 2024-05-09T00:50:04.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.714+0000 7fc011be9b80 20 client.15870 trim_cache size 0 max 16384 2024-05-09T00:50:04.717 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-09T00:50:04.717 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T00:50:04.717 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-05-09T00:50:04.717 INFO:teuthology.orchestra.run.smithi005.stdout: "333ca816-f4a1-453d-83f5-cb5fd271b1a5": { 2024-05-09T00:50:04.717 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-05-09T00:50:04.717 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "333ca816-f4a1-453d-83f5-cb5fd271b1a5", 2024-05-09T00:50:04.717 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-05-09T00:50:04.717 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T00:50:04.717 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T00:50:04.717 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-09T00:50:04.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.716+0000 7fc001f71700 1 -- 172.21.15.5:0/118268331 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fbfe40b29b0 msgr2=0x7fbfe40b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:50:04.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.716+0000 7fc001f71700 1 --2- 172.21.15.5:0/118268331 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fbfe40b29b0 0x7fbfe40b4ec0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x55b90cbfe7d0 tx=0x7fbff0008040 comp rx=0 tx=0).stop 2024-05-09T00:50:04.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.716+0000 7fc001f71700 1 -- 172.21.15.5:0/118268331 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbffc1032e0 msgr2=0x7fbffc112070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:50:04.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.716+0000 7fc001f71700 1 --2- 172.21.15.5:0/118268331 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbffc1032e0 0x7fbffc112070 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbfec00b530 tx=0x7fbfec00b840 comp rx=0 tx=0).stop 2024-05-09T00:50:04.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.716+0000 7fc001f71700 1 -- 172.21.15.5:0/118268331 shutdown_connections 2024-05-09T00:50:04.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.716+0000 7fc001f71700 1 --2- 172.21.15.5:0/118268331 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fbfe40b29b0 0x7fbfe40b4ec0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:04.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.716+0000 7fc001f71700 1 --2- 172.21.15.5:0/118268331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbffc108560 0x7fbffc110ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:04.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.716+0000 7fc001f71700 1 --2- 172.21.15.5:0/118268331 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbffc1032e0 0x7fbffc112070 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:04.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.716+0000 7fc001f71700 1 --2- 172.21.15.5:0/118268331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbffc107c10 0x7fbffc1104d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:04.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.716+0000 7fc001f71700 1 -- 172.21.15.5:0/118268331 >> 172.21.15.5:0/118268331 conn(0x7fbffc078ac0 msgr2=0x7fbffc100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:50:04.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.717+0000 7fc001f71700 1 -- 172.21.15.5:0/118268331 shutdown_connections 2024-05-09T00:50:04.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:04.717+0000 7fc001f71700 1 -- 172.21.15.5:0/118268331 wait complete. 2024-05-09T00:50:05.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.069+0000 7f08d3fff640 1 -- 172.21.15.5:0/3129161749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08d410a470 msgr2=0x7f08d410ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:50:05.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.069+0000 7f08d3fff640 1 --2- 172.21.15.5:0/3129161749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08d410a470 0x7f08d410ad80 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f08cc009ca0 tx=0x7f08cc02a8f0 comp rx=0 tx=0).stop 2024-05-09T00:50:05.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.069+0000 7f08d1ffb640 1 -- 172.21.15.5:0/3129161749 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08cc02f340 con 0x7f08d410a470 2024-05-09T00:50:05.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.072+0000 7f08d3fff640 1 -- 172.21.15.5:0/3129161749 shutdown_connections 2024-05-09T00:50:05.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.072+0000 7f08d3fff640 1 --2- 172.21.15.5:0/3129161749 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08d410ffe0 0x7f08d41103f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:05.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.072+0000 7f08d3fff640 1 --2- 172.21.15.5:0/3129161749 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08d410b2c0 0x7f08d410b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:05.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.072+0000 7f08d3fff640 1 --2- 172.21.15.5:0/3129161749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08d410a470 0x7f08d410ad80 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:05.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.072+0000 7f08d3fff640 1 -- 172.21.15.5:0/3129161749 >> 172.21.15.5:0/3129161749 conn(0x7f08d4070460 msgr2=0x7f08d4072910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:50:05.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.075+0000 7f08d3fff640 1 -- 172.21.15.5:0/3129161749 shutdown_connections 2024-05-09T00:50:05.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.077+0000 7f08d3fff640 1 -- 172.21.15.5:0/3129161749 wait complete. 2024-05-09T00:50:05.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.079+0000 7f08d3fff640 1 Processor -- start 2024-05-09T00:50:05.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.080+0000 7f08d3fff640 1 -- start start 2024-05-09T00:50:05.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.080+0000 7f08d3fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08d410a470 0x7f08d41a9020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:05.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.080+0000 7f08d3fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08d410b2c0 0x7f08d41a9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:05.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.080+0000 7f08d3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08d410ffe0 0x7f08d41b04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:05.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.080+0000 7f08d3fff640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f08d411e420 con 0x7f08d410ffe0 2024-05-09T00:50:05.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.080+0000 7f08d3fff640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f08d411e2a0 con 0x7f08d410a470 2024-05-09T00:50:05.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.080+0000 7f08d3fff640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f08d411e5a0 con 0x7f08d410b2c0 2024-05-09T00:50:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.081+0000 7f08d2ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08d410a470 0x7f08d41a9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.081+0000 7f08d2ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08d410a470 0x7f08d41a9020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:46948/0 (socket says 172.21.15.5:46948) 2024-05-09T00:50:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.081+0000 7f08d2ffd640 1 -- 172.21.15.5:0/519390726 learned_addr learned my addr 172.21.15.5:0/519390726 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:50:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.084+0000 7f08d27fc640 1 --2- 172.21.15.5:0/519390726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08d410b2c0 0x7f08d41a9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.084+0000 7f08d2ffd640 1 -- 172.21.15.5:0/519390726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08d410b2c0 msgr2=0x7f08d41a9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:50:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.084+0000 7f08d2ffd640 1 --2- 172.21.15.5:0/519390726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08d410b2c0 0x7f08d41a9560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.084+0000 7f08d2ffd640 1 -- 172.21.15.5:0/519390726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08d410ffe0 msgr2=0x7f08d41b04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:50:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.084+0000 7f08d2ffd640 1 --2- 172.21.15.5:0/519390726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08d410ffe0 0x7f08d41b04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.084+0000 7f08d2ffd640 1 -- 172.21.15.5:0/519390726 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08d41b0b60 con 0x7f08d410a470 2024-05-09T00:50:05.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.086+0000 7f08d2ffd640 1 --2- 172.21.15.5:0/519390726 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08d410a470 0x7f08d41a9020 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f08cc031af0 tx=0x7f08cc003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:50:05.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.100+0000 7f08bbfff640 1 -- 172.21.15.5:0/519390726 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08cc02f3b0 con 0x7f08d410a470 2024-05-09T00:50:05.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.101+0000 7f08d3fff640 1 -- 172.21.15.5:0/519390726 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08d41b0df0 con 0x7f08d410a470 2024-05-09T00:50:05.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.101+0000 7f08d3fff640 1 -- 172.21.15.5:0/519390726 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08d41b12d0 con 0x7f08d410a470 2024-05-09T00:50:05.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.101+0000 7f08bbfff640 1 -- 172.21.15.5:0/519390726 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08cc02d070 con 0x7f08d410a470 2024-05-09T00:50:05.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.101+0000 7f08bbfff640 1 -- 172.21.15.5:0/519390726 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08cc02fa20 con 0x7f08d410a470 2024-05-09T00:50:05.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.109+0000 7f08bbfff640 1 -- 172.21.15.5:0/519390726 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f08cc02fca0 con 0x7f08d410a470 2024-05-09T00:50:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.112+0000 7f08e03e7740 1 Processor -- start 2024-05-09T00:50:05.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.118+0000 7f08e03e7740 1 -- start start 2024-05-09T00:50:05.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.118+0000 7f08e03e7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a6ddfc51b0 0x55a6de3696c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:05.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.118+0000 7f08e03e7740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6ddf9c2f0 0x55a6de369c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:05.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.118+0000 7f08e03e7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6ddfa44d0 0x55a6de370b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:05.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.118+0000 7f08e03e7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a6de2cd7c0 con 0x55a6ddfa44d0 2024-05-09T00:50:05.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.118+0000 7f08e03e7740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55a6de2690e0 con 0x55a6ddf9c2f0 2024-05-09T00:50:05.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.118+0000 7f08e03e7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55a6de2c9780 con 0x55a6ddfc51b0 2024-05-09T00:50:05.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.120+0000 7f08d2ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6ddf9c2f0 0x55a6de369c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:05.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.120+0000 7f08d2ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6ddf9c2f0 0x55a6de369c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:46950/0 (socket says 172.21.15.5:46950) 2024-05-09T00:50:05.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.120+0000 7f08d2ffd640 1 -- 172.21.15.5:0/3088216123 learned_addr learned my addr 172.21.15.5:0/3088216123 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:50:05.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.121+0000 7f08d2ffd640 1 -- 172.21.15.5:0/3088216123 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a6ddfc51b0 msgr2=0x55a6de3696c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:50:05.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.121+0000 7f08d2ffd640 1 --2- 172.21.15.5:0/3088216123 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a6ddfc51b0 0x55a6de3696c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:05.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.121+0000 7f08d2ffd640 1 -- 172.21.15.5:0/3088216123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6ddfa44d0 msgr2=0x55a6de370b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:50:05.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.121+0000 7f08d2ffd640 1 --2- 172.21.15.5:0/3088216123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6ddfa44d0 0x55a6de370b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:05.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.121+0000 7f08d2ffd640 1 -- 172.21.15.5:0/3088216123 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6de2927c0 con 0x55a6ddf9c2f0 2024-05-09T00:50:05.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.122+0000 7f08d2ffd640 1 --2- 172.21.15.5:0/3088216123 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6ddf9c2f0 0x55a6de369c40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f08cc109b00 tx=0x7f08cc109bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f378352ee40d742 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:50:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.124+0000 7f08bbfff640 1 --2- 172.21.15.5:0/519390726 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f08bc0b29f0 0x7f08bc0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.125+0000 7f08b97fa640 1 -- 172.21.15.5:0/3088216123 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08cc10c8c0 con 0x55a6ddf9c2f0 2024-05-09T00:50:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.125+0000 7f08b97fa640 1 -- 172.21.15.5:0/3088216123 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08cc110030 con 0x55a6ddf9c2f0 2024-05-09T00:50:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.125+0000 7f08b97fa640 1 -- 172.21.15.5:0/3088216123 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08cc1153e0 con 0x55a6ddf9c2f0 2024-05-09T00:50:05.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.130+0000 7f08d37fe640 1 --2- 172.21.15.5:0/519390726 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f08bc0b29f0 0x7f08bc0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:05.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.131+0000 7f08e03e7740 1 -- 172.21.15.5:0/3088216123 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6ddf9c2f0 msgr2=0x55a6de369c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:50:05.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.131+0000 7f08e03e7740 1 --2- 172.21.15.5:0/3088216123 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6ddf9c2f0 0x55a6de369c40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f08cc109b00 tx=0x7f08cc109bf0 comp rx=0 tx=0).stop 2024-05-09T00:50:05.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.131+0000 7f08bbfff640 1 -- 172.21.15.5:0/519390726 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f08cc103960 con 0x7f08d410a470 2024-05-09T00:50:05.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.132+0000 7f08e03e7740 1 -- 172.21.15.5:0/3088216123 shutdown_connections 2024-05-09T00:50:05.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.132+0000 7f08e03e7740 1 --2- 172.21.15.5:0/3088216123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6ddfa44d0 0x55a6de370b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:05.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.132+0000 7f08e03e7740 1 --2- 172.21.15.5:0/3088216123 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6ddf9c2f0 0x55a6de369c40 secure :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f08cc109b00 tx=0x7f08cc109bf0 comp rx=0 tx=0).stop 2024-05-09T00:50:05.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.132+0000 7f08e03e7740 1 --2- 172.21.15.5:0/3088216123 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a6ddfc51b0 0x55a6de3696c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:05.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.132+0000 7f08e03e7740 1 -- 172.21.15.5:0/3088216123 >> 172.21.15.5:0/3088216123 conn(0x55a6ddfb9300 msgr2=0x55a6de365050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:50:05.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.139+0000 7f08e03e7740 1 -- 172.21.15.5:0/3088216123 shutdown_connections 2024-05-09T00:50:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.141+0000 7f08e03e7740 1 -- 172.21.15.5:0/3088216123 wait complete. 2024-05-09T00:50:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.142+0000 7f08e03e7740 1 -- start start 2024-05-09T00:50:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.142+0000 7f08e03e7740 1 Processor -- start 2024-05-09T00:50:05.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.148+0000 7f08d37fe640 1 --2- 172.21.15.5:0/519390726 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f08bc0b29f0 0x7f08bc0b4f00 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x55a6de36a890 tx=0x7f08c8006cb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:50:05.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.148+0000 7f08e03e7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a6ddfa44d0 0x55a6de373400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:05.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.148+0000 7f08e03e7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6ddfc51b0 0x55a6de373940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:05.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.148+0000 7f08e03e7740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6ddf83d80 0x55a6de37a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:05.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.148+0000 7f08e03e7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a6de371350 con 0x55a6ddfc51b0 2024-05-09T00:50:05.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.148+0000 7f08e03e7740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55a6de2549c0 con 0x55a6ddf83d80 2024-05-09T00:50:05.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.148+0000 7f08e03e7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55a6de3714d0 con 0x55a6ddfa44d0 2024-05-09T00:50:05.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.148+0000 7f08e03e7740 10 client.0 _ll_register_callbacks cb 0x55a6de281020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:50:05.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.149+0000 7f08d27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6ddfc51b0 0x55a6de373940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:05.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.149+0000 7f08d27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6ddfc51b0 0x55a6de373940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46858/0 (socket says 172.21.15.5:46858) 2024-05-09T00:50:05.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.149+0000 7f08d27fc640 1 -- 172.21.15.5:0/3549917438 learned_addr learned my addr 172.21.15.5:0/3549917438 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:50:05.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.152+0000 7f08d2ffd640 1 --2- 172.21.15.5:0/3549917438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a6ddfa44d0 0x55a6de373400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:05.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.154+0000 7f08d37fe640 1 --2- 172.21.15.5:0/3549917438 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6ddf83d80 0x55a6de37a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.154+0000 7f08d37fe640 1 -- 172.21.15.5:0/3549917438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a6ddfa44d0 msgr2=0x55a6de373400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:50:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.154+0000 7f08d37fe640 1 --2- 172.21.15.5:0/3549917438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a6ddfa44d0 0x55a6de373400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.154+0000 7f08d37fe640 1 -- 172.21.15.5:0/3549917438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6ddfc51b0 msgr2=0x55a6de373940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:50:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.154+0000 7f08d37fe640 1 --2- 172.21.15.5:0/3549917438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6ddfc51b0 0x55a6de373940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.154+0000 7f08d37fe640 1 -- 172.21.15.5:0/3549917438 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6de27b1f0 con 0x55a6ddf83d80 2024-05-09T00:50:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.155+0000 7f08d37fe640 1 --2- 172.21.15.5:0/3549917438 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6ddf83d80 0x55a6de37a830 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f08c8009e50 tx=0x7f08c80047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe45bfac31428452 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:50:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.156+0000 7f08b6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-09T00:50:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.156+0000 7f08b6ff5640 1 -- 172.21.15.5:0/3549917438 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c801d5f0 con 0x55a6ddf83d80 2024-05-09T00:50:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.156+0000 7f08e03e7740 1 -- 172.21.15.5:0/3549917438 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08cc110030 con 0x55a6ddf83d80 2024-05-09T00:50:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.156+0000 7f08b6ff5640 1 -- 172.21.15.5:0/3549917438 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f08c801d790 con 0x55a6ddf83d80 2024-05-09T00:50:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.158+0000 7f08b6ff5640 1 -- 172.21.15.5:0/3549917438 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f08c801d990 con 0x55a6ddf83d80 2024-05-09T00:50:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.158+0000 7f08e03e7740 10 client.25496 fetch_fsmap learned FSMap version 29 2024-05-09T00:50:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.158+0000 7f08e03e7740 1 -- 172.21.15.5:0/3549917438 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a6de030790 con 0x55a6ddf83d80 2024-05-09T00:50:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.159+0000 7f08b6ff5640 1 -- 172.21.15.5:0/3549917438 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08cc110030 con 0x55a6ddf83d80 2024-05-09T00:50:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.161+0000 7f08b6ff5640 1 -- 172.21.15.5:0/3549917438 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f08c8012070 con 0x55a6ddf83d80 2024-05-09T00:50:05.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.166+0000 7f08e03e7740 10 client.25496 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:50:05.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.166+0000 7f08e03e7740 10 client.25496 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:50:05.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.166+0000 7f08e03e7740 20 client.25496 populate_metadata read hostname 'smithi005' 2024-05-09T00:50:05.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.166+0000 7f08e03e7740 1 --2- 172.21.15.5:0/3549917438 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55a6ddf591c0 0x55a6de392230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:05.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.166+0000 7f08e03e7740 4 client.25496 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:50:05.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.166+0000 7f08e03e7740 1 -- 172.21.15.5:0/3549917438 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6de2db3d0 con 0x55a6ddf591c0 2024-05-09T00:50:05.169 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.167+0000 7f08d2ffd640 1 --2- 172.21.15.5:0/3549917438 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55a6ddf591c0 0x55a6de392230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:05.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.175+0000 7f08d2ffd640 1 --2- 172.21.15.5:0/3549917438 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55a6ddf591c0 0x55a6de392230 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de0c3a7571767ab8 server_cookie=7b9f5b0947a7f44c in_seq=0 out_seq=0 2024-05-09T00:50:05.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.175+0000 7f08b6ff5640 10 client.25496 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:50:05.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.180+0000 7f08b6ff5640 1 -- 172.21.15.5:0/3549917438 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a6de2db3d0 con 0x55a6ddf591c0 2024-05-09T00:50:05.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.180+0000 7f08b6ff5640 10 client.25496 handle_command_reply: tid=0 2024-05-09T00:50:05.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.180+0000 7f08e03e7740 1 client.25496 shutdown 2024-05-09T00:50:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.183+0000 7f08e03e7740 1 -- 172.21.15.5:0/3549917438 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6ddf83d80 msgr2=0x55a6de37a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:50:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.183+0000 7f08e03e7740 1 --2- 172.21.15.5:0/3549917438 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6ddf83d80 0x55a6de37a830 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f08c8009e50 tx=0x7f08c80047c0 comp rx=0 tx=0).stop 2024-05-09T00:50:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.183+0000 7f08e03e7740 1 -- 172.21.15.5:0/3549917438 shutdown_connections 2024-05-09T00:50:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.183+0000 7f08e03e7740 1 --2- 172.21.15.5:0/3549917438 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55a6ddf591c0 0x55a6de392230 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.183+0000 7f08e03e7740 1 --2- 172.21.15.5:0/3549917438 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6ddf83d80 0x55a6de37a830 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.183+0000 7f08e03e7740 1 --2- 172.21.15.5:0/3549917438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6ddfc51b0 0x55a6de373940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.183+0000 7f08e03e7740 1 --2- 172.21.15.5:0/3549917438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a6ddfa44d0 0x55a6de373400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.183+0000 7f08e03e7740 1 -- 172.21.15.5:0/3549917438 >> 172.21.15.5:0/3549917438 conn(0x55a6ddfb9300 msgr2=0x55a6de363860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:50:05.188 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.186+0000 7f08b6ff5640 0 client.25496 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:50:05.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.189+0000 7f08e03e7740 1 -- 172.21.15.5:0/3549917438 shutdown_connections 2024-05-09T00:50:05.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.192+0000 7f08e03e7740 1 -- 172.21.15.5:0/3549917438 wait complete. 2024-05-09T00:50:05.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.194+0000 7f08e03e7740 20 client.25496 trim_cache size 0 max 16384 2024-05-09T00:50:05.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.271+0000 7f08e03e7740 1 Processor -- start 2024-05-09T00:50:05.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.271+0000 7f08e03e7740 1 -- start start 2024-05-09T00:50:05.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.271+0000 7f08e03e7740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6ddf83d80 0x55a6de372730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:05.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.271+0000 7f08e03e7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a6ddfa44d0 0x55a6de372c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:05.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.271+0000 7f08e03e7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6ddfc51b0 0x55a6de364920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:05.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.271+0000 7f08e03e7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f08c000e4c0 con 0x55a6ddfc51b0 2024-05-09T00:50:05.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.271+0000 7f08e03e7740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f08c0010000 con 0x55a6ddf83d80 2024-05-09T00:50:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.271+0000 7f08e03e7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f08c000eb90 con 0x55a6ddfa44d0 2024-05-09T00:50:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.273+0000 7f08d2ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6ddf83d80 0x55a6de372730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.273+0000 7f08d2ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6ddf83d80 0x55a6de372730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:46976/0 (socket says 172.21.15.5:46976) 2024-05-09T00:50:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.273+0000 7f08d2ffd640 1 -- 172.21.15.5:0/464972385 learned_addr learned my addr 172.21.15.5:0/464972385 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:50:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.273+0000 7f08d27fc640 1 --2- 172.21.15.5:0/464972385 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a6ddfa44d0 0x55a6de372c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.274+0000 7f08d37fe640 1 --2- 172.21.15.5:0/464972385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6ddfc51b0 0x55a6de364920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.274+0000 7f08d2ffd640 1 -- 172.21.15.5:0/464972385 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a6ddfa44d0 msgr2=0x55a6de372c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:50:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.274+0000 7f08d2ffd640 1 --2- 172.21.15.5:0/464972385 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a6ddfa44d0 0x55a6de372c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.274+0000 7f08d2ffd640 1 -- 172.21.15.5:0/464972385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6ddfc51b0 msgr2=0x55a6de364920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:50:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.274+0000 7f08d2ffd640 1 --2- 172.21.15.5:0/464972385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6ddfc51b0 0x55a6de364920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.274+0000 7f08d2ffd640 1 -- 172.21.15.5:0/464972385 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6de0a75f0 con 0x55a6ddf83d80 2024-05-09T00:50:05.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.275+0000 7f08d2ffd640 1 --2- 172.21.15.5:0/464972385 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6ddf83d80 0x55a6de372730 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f08cc0cbcd0 tx=0x7f08cc10b6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb825e860a1145f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:50:05.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.277+0000 7f08b8ff9640 1 -- 172.21.15.5:0/464972385 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08cc0cbb10 con 0x55a6ddf83d80 2024-05-09T00:50:05.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.277+0000 7f08b8ff9640 1 -- 172.21.15.5:0/464972385 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08cc036070 con 0x55a6ddf83d80 2024-05-09T00:50:05.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.277+0000 7f08b8ff9640 1 -- 172.21.15.5:0/464972385 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08cc0c7440 con 0x55a6ddf83d80 2024-05-09T00:50:05.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.280+0000 7f08e03e7740 1 -- 172.21.15.5:0/464972385 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6ddf83d80 msgr2=0x55a6de372730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:50:05.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.280+0000 7f08e03e7740 1 --2- 172.21.15.5:0/464972385 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6ddf83d80 0x55a6de372730 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f08cc0cbcd0 tx=0x7f08cc10b6f0 comp rx=0 tx=0).stop 2024-05-09T00:50:05.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.280+0000 7f08e03e7740 1 -- 172.21.15.5:0/464972385 shutdown_connections 2024-05-09T00:50:05.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.280+0000 7f08e03e7740 1 --2- 172.21.15.5:0/464972385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6ddfc51b0 0x55a6de364920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:05.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.281+0000 7f08e03e7740 1 --2- 172.21.15.5:0/464972385 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a6ddfa44d0 0x55a6de372c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:05.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.281+0000 7f08e03e7740 1 --2- 172.21.15.5:0/464972385 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6ddf83d80 0x55a6de372730 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:05.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.281+0000 7f08e03e7740 1 -- 172.21.15.5:0/464972385 >> 172.21.15.5:0/464972385 conn(0x55a6ddfb9300 msgr2=0x55a6de371a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:50:05.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.281+0000 7f08e03e7740 1 -- 172.21.15.5:0/464972385 shutdown_connections 2024-05-09T00:50:05.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.281+0000 7f08e03e7740 1 -- 172.21.15.5:0/464972385 wait complete. 2024-05-09T00:50:05.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.283+0000 7f08e03e7740 1 -- start start 2024-05-09T00:50:05.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.283+0000 7f08e03e7740 1 Processor -- start 2024-05-09T00:50:05.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.284+0000 7f08e03e7740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6ddf83d80 0x55a6de376c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:05.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.284+0000 7f08e03e7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6ddfa44d0 0x55a6de377140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:05.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.285+0000 7f08e03e7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a6ddfc51b0 0x55a6de3f85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:05.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.285+0000 7f08e03e7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f08c000f930 con 0x55a6ddfa44d0 2024-05-09T00:50:05.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.285+0000 7f08e03e7740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f08c000f260 con 0x55a6ddf83d80 2024-05-09T00:50:05.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.285+0000 7f08e03e7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f08c000c550 con 0x55a6ddfc51b0 2024-05-09T00:50:05.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.285+0000 7f08d27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6ddfa44d0 0x55a6de377140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:05.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.285+0000 7f08d27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6ddfa44d0 0x55a6de377140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46876/0 (socket says 172.21.15.5:46876) 2024-05-09T00:50:05.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.285+0000 7f08d27fc640 1 -- 172.21.15.5:0/2627800189 learned_addr learned my addr 172.21.15.5:0/2627800189 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:50:05.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.286+0000 7f08e03e7740 10 client.0 _ll_register_callbacks cb 0x7f08c8009280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:50:05.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.288+0000 7f08d37fe640 1 --2- 172.21.15.5:0/2627800189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a6ddfc51b0 0x55a6de3f85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.288+0000 7f08d27fc640 1 -- 172.21.15.5:0/2627800189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a6ddfc51b0 msgr2=0x55a6de3f85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:50:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.288+0000 7f08d27fc640 1 --2- 172.21.15.5:0/2627800189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a6ddfc51b0 0x55a6de3f85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.288+0000 7f08d27fc640 1 -- 172.21.15.5:0/2627800189 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6ddf83d80 msgr2=0x55a6de376c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:50:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.288+0000 7f08d2ffd640 1 --2- 172.21.15.5:0/2627800189 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6ddf83d80 0x55a6de376c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.288+0000 7f08d27fc640 1 --2- 172.21.15.5:0/2627800189 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6ddf83d80 0x55a6de376c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.288+0000 7f08d27fc640 1 -- 172.21.15.5:0/2627800189 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6de291ed0 con 0x55a6ddfa44d0 2024-05-09T00:50:05.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.288+0000 7f08d2ffd640 1 --2- 172.21.15.5:0/2627800189 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6ddf83d80 0x55a6de376c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:50:05.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.288+0000 7f08d27fc640 1 --2- 172.21.15.5:0/2627800189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6ddfa44d0 0x55a6de377140 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f08c4009c30 tx=0x7f08c40127c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4de2948d75bc55a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:50:05.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.289+0000 7f08b6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:50:05.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:50:05 smithi005 ceph-mon[30365]: pgmap v483: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 852 MiB/s wr, 2.47k op/s 2024-05-09T00:50:05.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.289+0000 7f08b6ff5640 1 -- 172.21.15.5:0/2627800189 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c401f060 con 0x55a6ddfa44d0 2024-05-09T00:50:05.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.289+0000 7f08b6ff5640 1 -- 172.21.15.5:0/2627800189 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f08c401a5f0 con 0x55a6ddfa44d0 2024-05-09T00:50:05.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.289+0000 7f08e03e7740 1 -- 172.21.15.5:0/2627800189 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08cc036070 con 0x55a6ddfa44d0 2024-05-09T00:50:05.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.290+0000 7f08b6ff5640 1 -- 172.21.15.5:0/2627800189 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f08c401a7f0 con 0x55a6ddfa44d0 2024-05-09T00:50:05.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.290+0000 7f08e03e7740 10 client.15882 fetch_fsmap learned FSMap version 29 2024-05-09T00:50:05.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.290+0000 7f08e03e7740 1 -- 172.21.15.5:0/2627800189 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a6de3f9970 con 0x55a6ddfa44d0 2024-05-09T00:50:05.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.290+0000 7f08b6ff5640 1 -- 172.21.15.5:0/2627800189 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08cc036070 con 0x55a6ddfa44d0 2024-05-09T00:50:05.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.291+0000 7f08b6ff5640 1 -- 172.21.15.5:0/2627800189 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f08c4019030 con 0x55a6ddfa44d0 2024-05-09T00:50:05.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.291+0000 7f08e03e7740 10 client.15882 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T00:50:05.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.291+0000 7f08e03e7740 10 client.15882 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:50:05.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.291+0000 7f08e03e7740 20 client.15882 populate_metadata read hostname 'smithi005' 2024-05-09T00:50:05.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.291+0000 7f08e03e7740 1 --2- 172.21.15.5:0/2627800189 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55a6de3fa330 0x55a6de3fc750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:05.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.291+0000 7f08e03e7740 4 client.15882 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:50:05.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.291+0000 7f08e03e7740 1 -- 172.21.15.5:0/2627800189 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a6de2d5240 con 0x55a6de3fa330 2024-05-09T00:50:05.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.292+0000 7f08d2ffd640 1 --2- 172.21.15.5:0/2627800189 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55a6de3fa330 0x55a6de3fc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:05.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.295+0000 7f08b6ff5640 10 client.15882 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:50:05.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:05.295+0000 7f08d2ffd640 1 --2- 172.21.15.5:0/2627800189 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55a6de3fa330 0x55a6de3fc750 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e82a55e8c6368efc server_cookie=ce3d2632e83c87c8 in_seq=0 out_seq=0 2024-05-09T00:50:05.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:50:05 smithi131 ceph-mon[34547]: pgmap v483: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 852 MiB/s wr, 2.47k op/s 2024-05-09T00:50:05.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:50:05 smithi157 ceph-mon[34294]: pgmap v483: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 852 MiB/s wr, 2.47k op/s 2024-05-09T00:50:06.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:50:06 smithi005 ceph-mon[30365]: scrub summary: idle 2024-05-09T00:50:06.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:50:06 smithi157 ceph-mon[34294]: scrub summary: idle 2024-05-09T00:50:06.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:50:06 smithi131 ceph-mon[34547]: scrub summary: idle 2024-05-09T00:50:06.993 INFO:tasks.cephfs.filesystem:scrub status for tag:333ca816-f4a1-453d-83f5-cb5fd271b1a5 - {'path': '/', 'tag': '333ca816-f4a1-453d-83f5-cb5fd271b1a5', 'options': 'recursive,force'} 2024-05-09T00:50:07.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:50:07 smithi005 ceph-mon[30365]: pgmap v484: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 976 MiB/s wr, 2.82k op/s 2024-05-09T00:50:07.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:50:07 smithi005 ceph-mon[30365]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T00:50:07.435 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:50:07 smithi157 ceph-mon[34294]: pgmap v484: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 976 MiB/s wr, 2.82k op/s 2024-05-09T00:50:07.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:50:07 smithi157 ceph-mon[34294]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T00:50:07.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:50:07 smithi131 ceph-mon[34547]: pgmap v484: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 976 MiB/s wr, 2.82k op/s 2024-05-09T00:50:07.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:50:07 smithi131 ceph-mon[34547]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T00:50:09.140 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:50:09 smithi005 ceph-mon[30365]: pgmap v485: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 833 MiB/s wr, 2.58k op/s 2024-05-09T00:50:09.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:50:09 smithi157 ceph-mon[34294]: pgmap v485: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 833 MiB/s wr, 2.58k op/s 2024-05-09T00:50:09.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:50:09 smithi131 ceph-mon[34547]: pgmap v485: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 833 MiB/s wr, 2.58k op/s 2024-05-09T00:50:11.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:50:11 smithi157 ceph-mon[34294]: pgmap v486: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 965 MiB/s wr, 2.99k op/s 2024-05-09T00:50:11.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:50:11 smithi005 ceph-mon[30365]: pgmap v486: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 965 MiB/s wr, 2.99k op/s 2024-05-09T00:50:11.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:50:11 smithi131 ceph-mon[34547]: pgmap v486: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 965 MiB/s wr, 2.99k op/s 2024-05-09T00:50:13.202 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:13.201+0000 7f08b6ff5640 1 -- 172.21.15.5:0/2627800189 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a6de2d5240 con 0x55a6de3fa330 2024-05-09T00:50:13.202 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:13.201+0000 7f08b6ff5640 10 client.15882 handle_command_reply: tid=0 2024-05-09T00:50:13.202 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:13.201+0000 7f08e03e7740 1 client.15882 shutdown 2024-05-09T00:50:13.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:13.202+0000 7f08e03e7740 1 -- 172.21.15.5:0/2627800189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6ddfa44d0 msgr2=0x55a6de377140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:50:13.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:13.202+0000 7f08e03e7740 1 --2- 172.21.15.5:0/2627800189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6ddfa44d0 0x55a6de377140 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f08c4009c30 tx=0x7f08c40127c0 comp rx=0 tx=0).stop 2024-05-09T00:50:13.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:13.202+0000 7f08e03e7740 1 -- 172.21.15.5:0/2627800189 shutdown_connections 2024-05-09T00:50:13.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:13.202+0000 7f08e03e7740 1 --2- 172.21.15.5:0/2627800189 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55a6de3fa330 0x55a6de3fc750 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:13.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:13.202+0000 7f08e03e7740 1 --2- 172.21.15.5:0/2627800189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a6ddfc51b0 0x55a6de3f85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:13.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:13.202+0000 7f08e03e7740 1 --2- 172.21.15.5:0/2627800189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6ddfa44d0 0x55a6de377140 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:13.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:13.202+0000 7f08e03e7740 1 --2- 172.21.15.5:0/2627800189 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6ddf83d80 0x55a6de376c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:13.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:13.202+0000 7f08e03e7740 1 -- 172.21.15.5:0/2627800189 >> 172.21.15.5:0/2627800189 conn(0x55a6ddfb9300 msgr2=0x55a6de3719e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:50:13.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:13.202+0000 7f08b6ff5640 0 client.15882 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:50:13.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:13.202+0000 7f08e03e7740 1 -- 172.21.15.5:0/2627800189 shutdown_connections 2024-05-09T00:50:13.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:13.202+0000 7f08e03e7740 1 -- 172.21.15.5:0/2627800189 wait complete. 2024-05-09T00:50:13.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:13.203+0000 7f08e03e7740 20 client.15882 trim_cache size 0 max 16384 2024-05-09T00:50:13.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-09T00:50:13.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-09T00:50:13.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-09T00:50:13.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:13.205+0000 7f08d3fff640 1 -- 172.21.15.5:0/519390726 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f08bc0b29f0 msgr2=0x7f08bc0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:50:13.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:13.205+0000 7f08d3fff640 1 --2- 172.21.15.5:0/519390726 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f08bc0b29f0 0x7f08bc0b4f00 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x55a6de36a890 tx=0x7f08c8006cb0 comp rx=0 tx=0).stop 2024-05-09T00:50:13.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:13.205+0000 7f08d3fff640 1 -- 172.21.15.5:0/519390726 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08d410a470 msgr2=0x7f08d41a9020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:50:13.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:13.205+0000 7f08d3fff640 1 --2- 172.21.15.5:0/519390726 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08d410a470 0x7f08d41a9020 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f08cc031af0 tx=0x7f08cc003ba0 comp rx=0 tx=0).stop 2024-05-09T00:50:13.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:13.206+0000 7f08d3fff640 1 -- 172.21.15.5:0/519390726 shutdown_connections 2024-05-09T00:50:13.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:13.206+0000 7f08d3fff640 1 --2- 172.21.15.5:0/519390726 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f08bc0b29f0 0x7f08bc0b4f00 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:13.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:13.206+0000 7f08d3fff640 1 --2- 172.21.15.5:0/519390726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08d410ffe0 0x7f08d41b04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:13.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:13.206+0000 7f08d3fff640 1 --2- 172.21.15.5:0/519390726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08d410b2c0 0x7f08d41a9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:13.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:13.206+0000 7f08d3fff640 1 --2- 172.21.15.5:0/519390726 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08d410a470 0x7f08d41a9020 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:13.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:13.206+0000 7f08d3fff640 1 -- 172.21.15.5:0/519390726 >> 172.21.15.5:0/519390726 conn(0x7f08d4070460 msgr2=0x7f08d410bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:50:13.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:13.207+0000 7f08d3fff640 1 -- 172.21.15.5:0/519390726 shutdown_connections 2024-05-09T00:50:13.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:13.207+0000 7f08d3fff640 1 -- 172.21.15.5:0/519390726 wait complete. 2024-05-09T00:50:13.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:50:13 smithi157 ceph-mon[34294]: pgmap v487: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 812 MiB/s wr, 2.43k op/s 2024-05-09T00:50:13.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:50:13 smithi005 ceph-mon[30365]: pgmap v487: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 812 MiB/s wr, 2.43k op/s 2024-05-09T00:50:13.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:50:13 smithi131 ceph-mon[34547]: pgmap v487: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 812 MiB/s wr, 2.43k op/s 2024-05-09T00:50:15.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:50:15 smithi157 ceph-mon[34294]: pgmap v488: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 822 MiB/s wr, 2.33k op/s 2024-05-09T00:50:15.544 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:50:15 smithi005 ceph-mon[30365]: pgmap v488: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 822 MiB/s wr, 2.33k op/s 2024-05-09T00:50:15.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:50:15 smithi131 ceph-mon[34547]: pgmap v488: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 822 MiB/s wr, 2.33k op/s 2024-05-09T00:50:17.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:50:17 smithi005 ceph-mon[30365]: pgmap v489: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 955 MiB/s wr, 2.40k op/s 2024-05-09T00:50:17.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:50:17 smithi131 ceph-mon[34547]: pgmap v489: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 955 MiB/s wr, 2.40k op/s 2024-05-09T00:50:17.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:50:17 smithi157 ceph-mon[34294]: pgmap v489: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 955 MiB/s wr, 2.40k op/s 2024-05-09T00:50:19.296 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:50:19 smithi005 ceph-mon[30365]: pgmap v490: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 832 MiB/s wr, 1.99k op/s 2024-05-09T00:50:19.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:50:19 smithi131 ceph-mon[34547]: pgmap v490: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 832 MiB/s wr, 1.99k op/s 2024-05-09T00:50:19.685 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:50:19 smithi157 ceph-mon[34294]: pgmap v490: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 832 MiB/s wr, 1.99k op/s 2024-05-09T00:50:21.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:50:21 smithi005 ceph-mon[30365]: pgmap v491: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1023 MiB/s wr, 2.64k op/s 2024-05-09T00:50:21.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:50:21 smithi131 ceph-mon[34547]: pgmap v491: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1023 MiB/s wr, 2.64k op/s 2024-05-09T00:50:21.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:50:21 smithi157 ceph-mon[34294]: pgmap v491: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1023 MiB/s wr, 2.64k op/s 2024-05-09T00:50:23.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:50:23 smithi005 ceph-mon[30365]: pgmap v492: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 840 MiB/s wr, 2.13k op/s 2024-05-09T00:50:23.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:50:23 smithi131 ceph-mon[34547]: pgmap v492: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 840 MiB/s wr, 2.13k op/s 2024-05-09T00:50:23.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:50:23 smithi157 ceph-mon[34294]: pgmap v492: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 840 MiB/s wr, 2.13k op/s 2024-05-09T00:50:25.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:50:25 smithi005 ceph-mon[30365]: pgmap v493: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 855 MiB/s wr, 2.07k op/s 2024-05-09T00:50:25.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:50:25 smithi131 ceph-mon[34547]: pgmap v493: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 855 MiB/s wr, 2.07k op/s 2024-05-09T00:50:25.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:50:25 smithi157 ceph-mon[34294]: pgmap v493: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 855 MiB/s wr, 2.07k op/s 2024-05-09T00:50:26.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.539+0000 7f32a955e640 1 -- 172.21.15.5:0/1692930794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32a410a550 msgr2=0x7f32a410af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:50:26.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.539+0000 7f32a955e640 1 --2- 172.21.15.5:0/1692930794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32a410a550 0x7f32a410af40 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f329c009ca0 tx=0x7f329c02a8f0 comp rx=0 tx=0).stop 2024-05-09T00:50:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.544+0000 7f32a955e640 1 -- 172.21.15.5:0/1692930794 shutdown_connections 2024-05-09T00:50:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.544+0000 7f32a955e640 1 --2- 172.21.15.5:0/1692930794 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32a41101a0 0x7f32a41105b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.544+0000 7f32a955e640 1 --2- 172.21.15.5:0/1692930794 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32a410b480 0x7f32a410b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.544+0000 7f32a955e640 1 --2- 172.21.15.5:0/1692930794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32a410a550 0x7f32a410af40 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.544+0000 7f32a955e640 1 -- 172.21.15.5:0/1692930794 >> 172.21.15.5:0/1692930794 conn(0x7f32a4070670 msgr2=0x7f32a4072b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:50:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.547+0000 7f32a955e640 1 -- 172.21.15.5:0/1692930794 shutdown_connections 2024-05-09T00:50:26.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.548+0000 7f32a955e640 1 -- 172.21.15.5:0/1692930794 wait complete. 2024-05-09T00:50:26.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.549+0000 7f32a955e640 1 Processor -- start 2024-05-09T00:50:26.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.550+0000 7f32a955e640 1 -- start start 2024-05-09T00:50:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.550+0000 7f32a955e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32a410a550 0x7f32a41a92f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.550+0000 7f32a955e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32a410b480 0x7f32a41a9830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.550+0000 7f32a955e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32a41101a0 0x7f32a41b0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.550+0000 7f32a955e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f32a411e690 con 0x7f32a41101a0 2024-05-09T00:50:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.550+0000 7f32a955e640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f32a411e510 con 0x7f32a410b480 2024-05-09T00:50:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.550+0000 7f32a955e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f32a411e810 con 0x7f32a410a550 2024-05-09T00:50:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.551+0000 7f32a27fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32a410b480 0x7f32a41a9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.551+0000 7f32a27fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32a410b480 0x7f32a41a9830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:57962/0 (socket says 172.21.15.5:57962) 2024-05-09T00:50:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.551+0000 7f32a27fc640 1 -- 172.21.15.5:0/3217381916 learned_addr learned my addr 172.21.15.5:0/3217381916 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:50:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.552+0000 7f32a37fe640 1 --2- 172.21.15.5:0/3217381916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32a41101a0 0x7f32a41b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.552+0000 7f32a27fc640 1 -- 172.21.15.5:0/3217381916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32a410a550 msgr2=0x7f32a41a92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:50:26.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.552+0000 7f32a27fc640 1 --2- 172.21.15.5:0/3217381916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32a410a550 0x7f32a41a92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:26.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.552+0000 7f32a27fc640 1 -- 172.21.15.5:0/3217381916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32a41101a0 msgr2=0x7f32a41b0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:50:26.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.552+0000 7f32a27fc640 1 --2- 172.21.15.5:0/3217381916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32a41101a0 0x7f32a41b0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:26.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.552+0000 7f32a27fc640 1 -- 172.21.15.5:0/3217381916 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32a41b0de0 con 0x7f32a410b480 2024-05-09T00:50:26.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.553+0000 7f32a27fc640 1 --2- 172.21.15.5:0/3217381916 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32a410b480 0x7f32a41a9830 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f329400b4a0 tx=0x7f329400b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:50:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.554+0000 7f328bfff640 1 -- 172.21.15.5:0/3217381916 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3294014070 con 0x7f32a410b480 2024-05-09T00:50:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.554+0000 7f32a37fe640 1 --2- 172.21.15.5:0/3217381916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32a41101a0 0x7f32a41b0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:50:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.554+0000 7f32a955e640 1 -- 172.21.15.5:0/3217381916 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32a41b10d0 con 0x7f32a410b480 2024-05-09T00:50:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.554+0000 7f32a955e640 1 -- 172.21.15.5:0/3217381916 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32a41b16a0 con 0x7f32a410b480 2024-05-09T00:50:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.555+0000 7f328bfff640 1 -- 172.21.15.5:0/3217381916 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3294010ac0 con 0x7f32a410b480 2024-05-09T00:50:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.557+0000 7f328bfff640 1 -- 172.21.15.5:0/3217381916 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3294017c00 con 0x7f32a410b480 2024-05-09T00:50:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.559+0000 7f32b0412740 1 Processor -- start 2024-05-09T00:50:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.559+0000 7f32b0412740 1 -- start start 2024-05-09T00:50:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.560+0000 7f32b0412740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d090e21b0 0x557d09486500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.560+0000 7f32b0412740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d090b92f0 0x557d09486a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.561+0000 7f32b0412740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d090c14d0 0x557d0948d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:26.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.561+0000 7f32b0412740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557d093ea7c0 con 0x557d090c14d0 2024-05-09T00:50:26.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.561+0000 7f32b0412740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x557d093860e0 con 0x557d090b92f0 2024-05-09T00:50:26.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.561+0000 7f32a37fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d090e21b0 0x557d09486500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:26.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.561+0000 7f32a37fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d090e21b0 0x557d09486500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:49950/0 (socket says 172.21.15.5:49950) 2024-05-09T00:50:26.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.561+0000 7f32a37fe640 1 -- 172.21.15.5:0/3795493447 learned_addr learned my addr 172.21.15.5:0/3795493447 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:50:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.561+0000 7f32b0412740 1 -- 172.21.15.5:0/3795493447 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x557d093e6780 con 0x557d090e21b0 2024-05-09T00:50:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.563+0000 7f32a37fe640 1 -- 172.21.15.5:0/3795493447 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d090b92f0 msgr2=0x557d09486a80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:50:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.563+0000 7f32a37fe640 1 --2- 172.21.15.5:0/3795493447 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d090b92f0 0x557d09486a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.563+0000 7f32a37fe640 1 -- 172.21.15.5:0/3795493447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d090c14d0 msgr2=0x557d0948d980 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:50:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.563+0000 7f32a37fe640 1 --2- 172.21.15.5:0/3795493447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d090c14d0 0x557d0948d980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.563+0000 7f32a37fe640 1 -- 172.21.15.5:0/3795493447 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d093af7c0 con 0x557d090e21b0 2024-05-09T00:50:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.564+0000 7f32a37fe640 1 --2- 172.21.15.5:0/3795493447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d090e21b0 0x557d09486500 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f329800bae0 tx=0x7f329800d340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec4ca15904baf33f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:50:26.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.569+0000 7f32897fa640 1 -- 172.21.15.5:0/3795493447 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3298017420 con 0x557d090e21b0 2024-05-09T00:50:26.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.570+0000 7f32897fa640 1 -- 172.21.15.5:0/3795493447 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3298017a70 con 0x557d090e21b0 2024-05-09T00:50:26.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.570+0000 7f32897fa640 1 -- 172.21.15.5:0/3795493447 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329800fa80 con 0x557d090e21b0 2024-05-09T00:50:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.571+0000 7f328bfff640 1 -- 172.21.15.5:0/3217381916 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f3294017070 con 0x7f32a410b480 2024-05-09T00:50:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.573+0000 7f328bfff640 1 --2- 172.21.15.5:0/3217381916 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f328c0b28d0 0x7f328c0b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.574+0000 7f328bfff640 1 -- 172.21.15.5:0/3217381916 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f32940e1490 con 0x7f32a410b480 2024-05-09T00:50:26.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.576+0000 7f32b0412740 1 -- 172.21.15.5:0/3795493447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d090e21b0 msgr2=0x557d09486500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:50:26.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.577+0000 7f32b0412740 1 --2- 172.21.15.5:0/3795493447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d090e21b0 0x557d09486500 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f329800bae0 tx=0x7f329800d340 comp rx=0 tx=0).stop 2024-05-09T00:50:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.580+0000 7f32a2ffd640 1 --2- 172.21.15.5:0/3217381916 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f328c0b28d0 0x7f328c0b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:26.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.599+0000 7f32b0412740 1 -- 172.21.15.5:0/3795493447 shutdown_connections 2024-05-09T00:50:26.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.599+0000 7f32b0412740 1 --2- 172.21.15.5:0/3795493447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d090c14d0 0x557d0948d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:26.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.599+0000 7f32b0412740 1 --2- 172.21.15.5:0/3795493447 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d090b92f0 0x557d09486a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:26.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.599+0000 7f32b0412740 1 --2- 172.21.15.5:0/3795493447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d090e21b0 0x557d09486500 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:26.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.599+0000 7f32b0412740 1 -- 172.21.15.5:0/3795493447 >> 172.21.15.5:0/3795493447 conn(0x557d090d6300 msgr2=0x557d09481e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:50:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.604+0000 7f32a2ffd640 1 --2- 172.21.15.5:0/3217381916 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f328c0b28d0 0x7f328c0b4de0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f329c02aee0 tx=0x7f329c000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:50:26.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.605+0000 7f32b0412740 1 -- 172.21.15.5:0/3795493447 shutdown_connections 2024-05-09T00:50:26.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.608+0000 7f32b0412740 1 -- 172.21.15.5:0/3795493447 wait complete. 2024-05-09T00:50:26.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.609+0000 7f32b0412740 1 -- start start 2024-05-09T00:50:26.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.609+0000 7f32b0412740 1 Processor -- start 2024-05-09T00:50:26.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.612+0000 7f32b0412740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d090b92f0 0x557d09490400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:26.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.612+0000 7f32b0412740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d090c14d0 0x557d09490940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:26.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.612+0000 7f32b0412740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d090e21b0 0x557d094978a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:26.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.612+0000 7f32b0412740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557d0948e3d0 con 0x557d090c14d0 2024-05-09T00:50:26.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.612+0000 7f32b0412740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x557d093719c0 con 0x557d090e21b0 2024-05-09T00:50:26.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.612+0000 7f32b0412740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x557d0948e550 con 0x557d090b92f0 2024-05-09T00:50:26.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.612+0000 7f32b0412740 10 client.0 _ll_register_callbacks cb 0x557d0939e020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:50:26.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.614+0000 7f32a27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d090c14d0 0x557d09490940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:26.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.614+0000 7f32a27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d090c14d0 0x557d09490940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54386/0 (socket says 172.21.15.5:54386) 2024-05-09T00:50:26.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.614+0000 7f32a27fc640 1 -- 172.21.15.5:0/3956327180 learned_addr learned my addr 172.21.15.5:0/3956327180 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:50:26.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.614+0000 7f32a2ffd640 1 --2- 172.21.15.5:0/3956327180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d090b92f0 0x557d09490400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:26.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.615+0000 7f32a27fc640 1 -- 172.21.15.5:0/3956327180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d090b92f0 msgr2=0x557d09490400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:50:26.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.615+0000 7f32a27fc640 1 --2- 172.21.15.5:0/3956327180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d090b92f0 0x557d09490400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:26.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.615+0000 7f32a27fc640 1 -- 172.21.15.5:0/3956327180 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d090e21b0 msgr2=0x557d094978a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:50:26.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.615+0000 7f32a27fc640 1 --2- 172.21.15.5:0/3956327180 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d090e21b0 0x557d094978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:26.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.615+0000 7f32a27fc640 1 -- 172.21.15.5:0/3956327180 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d093981f0 con 0x557d090c14d0 2024-05-09T00:50:26.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.616+0000 7f32a27fc640 1 --2- 172.21.15.5:0/3956327180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d090c14d0 0x557d09490940 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f32940a5d10 tx=0x7f32940a86e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c61fc5108709c3fc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:50:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.616+0000 7f3286ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:50:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.617+0000 7f3286ff5640 1 -- 172.21.15.5:0/3956327180 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32940ace50 con 0x557d090c14d0 2024-05-09T00:50:26.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.617+0000 7f3286ff5640 1 -- 172.21.15.5:0/3956327180 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f32940b1040 con 0x557d090c14d0 2024-05-09T00:50:26.622 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.620+0000 7f32b0412740 1 -- 172.21.15.5:0/3956327180 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3298017a70 con 0x557d090c14d0 2024-05-09T00:50:26.623 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.621+0000 7f3286ff5640 1 -- 172.21.15.5:0/3956327180 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f32940b1900 con 0x557d090c14d0 2024-05-09T00:50:26.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.623+0000 7f32b0412740 10 client.15888 fetch_fsmap learned FSMap version 30 2024-05-09T00:50:26.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.623+0000 7f32b0412740 1 -- 172.21.15.5:0/3956327180 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557d0914d790 con 0x557d090c14d0 2024-05-09T00:50:26.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.628+0000 7f3286ff5640 1 -- 172.21.15.5:0/3956327180 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3298017a70 con 0x557d090c14d0 2024-05-09T00:50:26.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.628+0000 7f3286ff5640 1 -- 172.21.15.5:0/3956327180 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f3294010c70 con 0x557d090c14d0 2024-05-09T00:50:26.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.629+0000 7f32b0412740 10 client.15888 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T00:50:26.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.629+0000 7f32b0412740 10 client.15888 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:50:26.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.629+0000 7f32b0412740 20 client.15888 populate_metadata read hostname 'smithi005' 2024-05-09T00:50:26.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.629+0000 7f32b0412740 1 --2- 172.21.15.5:0/3956327180 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x557d090a0d80 0x557d094af140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:26.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.629+0000 7f32b0412740 4 client.15888 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:50:26.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.629+0000 7f32b0412740 1 -- 172.21.15.5:0/3956327180 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d093f83d0 con 0x557d090a0d80 2024-05-09T00:50:26.635 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.634+0000 7f32a2ffd640 1 --2- 172.21.15.5:0/3956327180 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x557d090a0d80 0x557d094af140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:26.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.649+0000 7f32a2ffd640 1 --2- 172.21.15.5:0/3956327180 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x557d090a0d80 0x557d094af140 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=124e49275a10b059 server_cookie=5560d40cba8150ef in_seq=0 out_seq=0 2024-05-09T00:50:26.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.650+0000 7f3286ff5640 10 client.15888 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:50:26.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.654+0000 7f3286ff5640 1 -- 172.21.15.5:0/3956327180 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557d093f83d0 con 0x557d090a0d80 2024-05-09T00:50:26.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.654+0000 7f3286ff5640 10 client.15888 handle_command_reply: tid=0 2024-05-09T00:50:26.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.654+0000 7f32b0412740 1 client.15888 shutdown 2024-05-09T00:50:26.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.654+0000 7f32b0412740 1 -- 172.21.15.5:0/3956327180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d090c14d0 msgr2=0x557d09490940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:50:26.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.654+0000 7f32b0412740 1 --2- 172.21.15.5:0/3956327180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d090c14d0 0x557d09490940 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f32940a5d10 tx=0x7f32940a86e0 comp rx=0 tx=0).stop 2024-05-09T00:50:26.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.655+0000 7f32b0412740 1 -- 172.21.15.5:0/3956327180 shutdown_connections 2024-05-09T00:50:26.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.655+0000 7f32b0412740 1 --2- 172.21.15.5:0/3956327180 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x557d090a0d80 0x557d094af140 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:26.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.655+0000 7f32b0412740 1 --2- 172.21.15.5:0/3956327180 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d090e21b0 0x557d094978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:26.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.655+0000 7f32b0412740 1 --2- 172.21.15.5:0/3956327180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d090c14d0 0x557d09490940 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:26.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.655+0000 7f32b0412740 1 --2- 172.21.15.5:0/3956327180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d090b92f0 0x557d09490400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:26.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.655+0000 7f3286ff5640 0 client.15888 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:50:26.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.655+0000 7f32b0412740 1 -- 172.21.15.5:0/3956327180 >> 172.21.15.5:0/3956327180 conn(0x557d090d6300 msgr2=0x557d09484da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:50:26.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.655+0000 7f32b0412740 1 -- 172.21.15.5:0/3956327180 shutdown_connections 2024-05-09T00:50:26.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.655+0000 7f32b0412740 1 -- 172.21.15.5:0/3956327180 wait complete. 2024-05-09T00:50:26.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.658+0000 7f32b0412740 20 client.15888 trim_cache size 0 max 16384 2024-05-09T00:50:26.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.700+0000 7f32b0412740 1 Processor -- start 2024-05-09T00:50:26.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.700+0000 7f32b0412740 1 -- start start 2024-05-09T00:50:26.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.700+0000 7f32b0412740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d090a0d80 0x557d09481710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:26.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.701+0000 7f32b0412740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d090b92f0 0x557d0948a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:26.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.701+0000 7f32a2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d090a0d80 0x557d09481710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:26.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.701+0000 7f32a2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d090a0d80 0x557d09481710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54398/0 (socket says 172.21.15.5:54398) 2024-05-09T00:50:26.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.701+0000 7f32a2ffd640 1 -- 172.21.15.5:0/517991695 learned_addr learned my addr 172.21.15.5:0/517991695 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:50:26.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.701+0000 7f32b0412740 1 --2- 172.21.15.5:0/517991695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d090c14d0 0x557d0948ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:26.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.701+0000 7f32b0412740 1 -- 172.21.15.5:0/517991695 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f329000e5b0 con 0x557d090a0d80 2024-05-09T00:50:26.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.701+0000 7f32b0412740 1 -- 172.21.15.5:0/517991695 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3290010120 con 0x557d090b92f0 2024-05-09T00:50:26.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.701+0000 7f32b0412740 1 -- 172.21.15.5:0/517991695 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f329000ec80 con 0x557d090c14d0 2024-05-09T00:50:26.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.701+0000 7f32a2ffd640 1 -- 172.21.15.5:0/517991695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d090c14d0 msgr2=0x557d0948ace0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:50:26.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.701+0000 7f32a2ffd640 1 --2- 172.21.15.5:0/517991695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d090c14d0 0x557d0948ace0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:26.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.701+0000 7f32a2ffd640 1 -- 172.21.15.5:0/517991695 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d090b92f0 msgr2=0x557d0948a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:50:26.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.701+0000 7f32a2ffd640 1 --2- 172.21.15.5:0/517991695 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d090b92f0 0x557d0948a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:26.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.701+0000 7f32a2ffd640 1 -- 172.21.15.5:0/517991695 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d091c45f0 con 0x557d090a0d80 2024-05-09T00:50:26.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.702+0000 7f32a2ffd640 1 --2- 172.21.15.5:0/517991695 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d090a0d80 0x557d09481710 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f329c009ca0 tx=0x7f329c040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d267522fb0ef788a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:50:26.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.702+0000 7f3288ff9640 1 -- 172.21.15.5:0/517991695 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c0505b0 con 0x557d090a0d80 2024-05-09T00:50:26.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.702+0000 7f3288ff9640 1 -- 172.21.15.5:0/517991695 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f329c056070 con 0x557d090a0d80 2024-05-09T00:50:26.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.702+0000 7f3288ff9640 1 -- 172.21.15.5:0/517991695 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c047420 con 0x557d090a0d80 2024-05-09T00:50:26.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.703+0000 7f32b0412740 1 -- 172.21.15.5:0/517991695 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d090a0d80 msgr2=0x557d09481710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:50:26.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.703+0000 7f32b0412740 1 --2- 172.21.15.5:0/517991695 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d090a0d80 0x557d09481710 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f329c009ca0 tx=0x7f329c040000 comp rx=0 tx=0).stop 2024-05-09T00:50:26.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.703+0000 7f32b0412740 1 -- 172.21.15.5:0/517991695 shutdown_connections 2024-05-09T00:50:26.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.703+0000 7f32b0412740 1 --2- 172.21.15.5:0/517991695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d090c14d0 0x557d0948ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:26.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.703+0000 7f32b0412740 1 --2- 172.21.15.5:0/517991695 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d090b92f0 0x557d0948a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:26.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.703+0000 7f32b0412740 1 --2- 172.21.15.5:0/517991695 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d090a0d80 0x557d09481710 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:26.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.703+0000 7f32b0412740 1 -- 172.21.15.5:0/517991695 >> 172.21.15.5:0/517991695 conn(0x557d090d6300 msgr2=0x557d09490400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:50:26.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.703+0000 7f32b0412740 1 -- 172.21.15.5:0/517991695 shutdown_connections 2024-05-09T00:50:26.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.703+0000 7f32b0412740 1 -- 172.21.15.5:0/517991695 wait complete. 2024-05-09T00:50:26.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.704+0000 7f32b0412740 1 -- start start 2024-05-09T00:50:26.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.704+0000 7f32b0412740 1 Processor -- start 2024-05-09T00:50:26.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.705+0000 7f32b0412740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d090a0d80 0x557d09493b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:26.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.705+0000 7f32b0412740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d090b92f0 0x557d094940b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:26.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.705+0000 7f32b0412740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d090c14d0 0x557d095154e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:26.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.705+0000 7f32a27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d090b92f0 0x557d094940b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:26.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.705+0000 7f32a27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d090b92f0 0x557d094940b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54402/0 (socket says 172.21.15.5:54402) 2024-05-09T00:50:26.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.705+0000 7f32a27fc640 1 -- 172.21.15.5:0/3805249111 learned_addr learned my addr 172.21.15.5:0/3805249111 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:50:26.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.705+0000 7f32b0412740 1 -- 172.21.15.5:0/3805249111 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f329000fa20 con 0x557d090b92f0 2024-05-09T00:50:26.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.705+0000 7f32b0412740 1 -- 172.21.15.5:0/3805249111 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f329000f350 con 0x557d090c14d0 2024-05-09T00:50:26.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.705+0000 7f32b0412740 1 -- 172.21.15.5:0/3805249111 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f329000c590 con 0x557d090a0d80 2024-05-09T00:50:26.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.705+0000 7f32b0412740 10 client.0 _ll_register_callbacks cb 0x7f32940a8870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:50:26.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.705+0000 7f32a37fe640 1 --2- 172.21.15.5:0/3805249111 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d090c14d0 0x557d095154e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:26.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.705+0000 7f32a27fc640 1 -- 172.21.15.5:0/3805249111 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d090a0d80 msgr2=0x557d09493b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:50:26.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.705+0000 7f32a27fc640 1 --2- 172.21.15.5:0/3805249111 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d090a0d80 0x557d09493b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:26.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.705+0000 7f32a27fc640 1 -- 172.21.15.5:0/3805249111 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d090c14d0 msgr2=0x557d095154e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:50:26.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.705+0000 7f32a27fc640 1 --2- 172.21.15.5:0/3805249111 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d090c14d0 0x557d095154e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:26.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.705+0000 7f32a27fc640 1 -- 172.21.15.5:0/3805249111 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d093aeed0 con 0x557d090b92f0 2024-05-09T00:50:26.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.706+0000 7f32a37fe640 1 --2- 172.21.15.5:0/3805249111 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d090c14d0 0x557d095154e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:50:26.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.706+0000 7f32a27fc640 1 --2- 172.21.15.5:0/3805249111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d090b92f0 0x557d094940b0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f32940a7d00 tx=0x7f32940a86e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa88d70271984158 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:50:26.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.706+0000 7f3286ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:50:26.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.706+0000 7f3286ff5640 1 -- 172.21.15.5:0/3805249111 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32940b7070 con 0x557d090b92f0 2024-05-09T00:50:26.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.706+0000 7f3286ff5640 1 -- 172.21.15.5:0/3805249111 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f32940a5090 con 0x557d090b92f0 2024-05-09T00:50:26.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.706+0000 7f32b0412740 1 -- 172.21.15.5:0/3805249111 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f329c056070 con 0x557d090b92f0 2024-05-09T00:50:26.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.706+0000 7f3286ff5640 1 -- 172.21.15.5:0/3805249111 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f32940a83f0 con 0x557d090b92f0 2024-05-09T00:50:26.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.707+0000 7f32b0412740 10 client.15894 fetch_fsmap learned FSMap version 30 2024-05-09T00:50:26.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.707+0000 7f32b0412740 1 -- 172.21.15.5:0/3805249111 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557d09516690 con 0x557d090b92f0 2024-05-09T00:50:26.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.707+0000 7f3286ff5640 1 -- 172.21.15.5:0/3805249111 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32940b1240 con 0x557d090b92f0 2024-05-09T00:50:26.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.707+0000 7f3286ff5640 1 -- 172.21.15.5:0/3805249111 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f32940ba030 con 0x557d090b92f0 2024-05-09T00:50:26.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.707+0000 7f32b0412740 10 client.15894 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T00:50:26.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.707+0000 7f32b0412740 10 client.15894 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:50:26.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.707+0000 7f32b0412740 20 client.15894 populate_metadata read hostname 'smithi005' 2024-05-09T00:50:26.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.707+0000 7f32b0412740 1 --2- 172.21.15.5:0/3805249111 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x557d090e21b0 0x557d09518f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:26.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.707+0000 7f32b0412740 4 client.15894 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:50:26.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.707+0000 7f32b0412740 1 -- 172.21.15.5:0/3805249111 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f329c056070 con 0x557d090e21b0 2024-05-09T00:50:26.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.711+0000 7f32a2ffd640 1 --2- 172.21.15.5:0/3805249111 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x557d090e21b0 0x557d09518f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:26.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.721+0000 7f32a2ffd640 1 --2- 172.21.15.5:0/3805249111 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x557d090e21b0 0x557d09518f20 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5752ddfd258e43f3 server_cookie=64c9f7797c066540 in_seq=0 out_seq=0 2024-05-09T00:50:26.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:26.721+0000 7f3286ff5640 10 client.15894 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:50:27.543 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:50:27 smithi005 ceph-mon[30365]: pgmap v494: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 975 MiB/s wr, 2.34k op/s 2024-05-09T00:50:27.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:50:27 smithi131 ceph-mon[34547]: pgmap v494: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 975 MiB/s wr, 2.34k op/s 2024-05-09T00:50:27.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:50:27 smithi157 ceph-mon[34294]: pgmap v494: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 975 MiB/s wr, 2.34k op/s 2024-05-09T00:50:29.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:50:29 smithi005 ceph-mon[30365]: pgmap v495: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 831 MiB/s wr, 2.08k op/s 2024-05-09T00:50:29.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:50:29 smithi131 ceph-mon[34547]: pgmap v495: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 831 MiB/s wr, 2.08k op/s 2024-05-09T00:50:29.685 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:50:29 smithi157 ceph-mon[34294]: pgmap v495: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 831 MiB/s wr, 2.08k op/s 2024-05-09T00:50:31.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:50:31 smithi005 ceph-mon[30365]: pgmap v496: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1024 MiB/s wr, 2.37k op/s 2024-05-09T00:50:31.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:50:31 smithi131 ceph-mon[34547]: pgmap v496: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1024 MiB/s wr, 2.37k op/s 2024-05-09T00:50:31.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:50:31 smithi157 ceph-mon[34294]: pgmap v496: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1024 MiB/s wr, 2.37k op/s 2024-05-09T00:50:33.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:50:33 smithi005 ceph-mon[30365]: pgmap v497: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 789 MiB/s wr, 1.58k op/s 2024-05-09T00:50:33.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:50:33 smithi131 ceph-mon[34547]: pgmap v497: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 789 MiB/s wr, 1.58k op/s 2024-05-09T00:50:33.685 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:50:33 smithi157 ceph-mon[34294]: pgmap v497: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 789 MiB/s wr, 1.58k op/s 2024-05-09T00:50:34.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:34.389+0000 7f3286ff5640 1 -- 172.21.15.5:0/3805249111 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7f329c056070 con 0x557d090e21b0 2024-05-09T00:50:34.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:34.389+0000 7f3286ff5640 10 client.15894 handle_command_reply: tid=0 2024-05-09T00:50:34.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:34.389+0000 7f32b0412740 1 client.15894 shutdown 2024-05-09T00:50:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:34.391+0000 7f32b0412740 1 -- 172.21.15.5:0/3805249111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d090b92f0 msgr2=0x557d094940b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:50:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:34.391+0000 7f32b0412740 1 --2- 172.21.15.5:0/3805249111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d090b92f0 0x557d094940b0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f32940a7d00 tx=0x7f32940a86e0 comp rx=0 tx=0).stop 2024-05-09T00:50:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:34.392+0000 7f32b0412740 1 -- 172.21.15.5:0/3805249111 shutdown_connections 2024-05-09T00:50:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:34.392+0000 7f32b0412740 1 --2- 172.21.15.5:0/3805249111 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x557d090e21b0 0x557d09518f20 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:34.392+0000 7f32b0412740 1 --2- 172.21.15.5:0/3805249111 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d090c14d0 0x557d095154e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:34.392+0000 7f32b0412740 1 --2- 172.21.15.5:0/3805249111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d090b92f0 0x557d094940b0 secure :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f32940a7d00 tx=0x7f32940a86e0 comp rx=0 tx=0).stop 2024-05-09T00:50:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:34.392+0000 7f32b0412740 1 --2- 172.21.15.5:0/3805249111 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d090a0d80 0x557d09493b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:34.392+0000 7f32b0412740 1 -- 172.21.15.5:0/3805249111 >> 172.21.15.5:0/3805249111 conn(0x557d090d6300 msgr2=0x557d094903a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:50:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:34.392+0000 7f3286ff5640 0 client.15894 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:50:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:34.393+0000 7f32b0412740 1 -- 172.21.15.5:0/3805249111 shutdown_connections 2024-05-09T00:50:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:34.394+0000 7f32b0412740 1 -- 172.21.15.5:0/3805249111 wait complete. 2024-05-09T00:50:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:34.394+0000 7f32b0412740 20 client.15894 trim_cache size 0 max 16384 2024-05-09T00:50:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-09T00:50:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-09T00:50:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-09T00:50:34.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:34.401+0000 7f32a955e640 1 -- 172.21.15.5:0/3217381916 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f328c0b28d0 msgr2=0x7f328c0b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:50:34.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:34.401+0000 7f32a955e640 1 --2- 172.21.15.5:0/3217381916 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f328c0b28d0 0x7f328c0b4de0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f329c02aee0 tx=0x7f329c000f40 comp rx=0 tx=0).stop 2024-05-09T00:50:34.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:34.401+0000 7f32a955e640 1 -- 172.21.15.5:0/3217381916 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32a410b480 msgr2=0x7f32a41a9830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:50:34.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:34.401+0000 7f32a955e640 1 --2- 172.21.15.5:0/3217381916 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32a410b480 0x7f32a41a9830 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f329400b4a0 tx=0x7f329400b960 comp rx=0 tx=0).stop 2024-05-09T00:50:34.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:34.402+0000 7f32a955e640 1 -- 172.21.15.5:0/3217381916 shutdown_connections 2024-05-09T00:50:34.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:34.402+0000 7f32a955e640 1 --2- 172.21.15.5:0/3217381916 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f328c0b28d0 0x7f328c0b4de0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:34.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:34.402+0000 7f32a955e640 1 --2- 172.21.15.5:0/3217381916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32a41101a0 0x7f32a41b0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:34.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:34.402+0000 7f32a955e640 1 --2- 172.21.15.5:0/3217381916 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32a410b480 0x7f32a41a9830 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:34.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:34.402+0000 7f32a955e640 1 --2- 172.21.15.5:0/3217381916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32a410a550 0x7f32a41a92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:34.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:34.402+0000 7f32a955e640 1 -- 172.21.15.5:0/3217381916 >> 172.21.15.5:0/3217381916 conn(0x7f32a4070670 msgr2=0x7f32a410c180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:50:34.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:34.404+0000 7f32a955e640 1 -- 172.21.15.5:0/3217381916 shutdown_connections 2024-05-09T00:50:34.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:34.405+0000 7f32a955e640 1 -- 172.21.15.5:0/3217381916 wait complete. 2024-05-09T00:50:35.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:50:35 smithi005 ceph-mon[30365]: pgmap v498: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 857 MiB/s wr, 1.66k op/s 2024-05-09T00:50:35.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:50:35 smithi131 ceph-mon[34547]: pgmap v498: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 857 MiB/s wr, 1.66k op/s 2024-05-09T00:50:35.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:50:35 smithi157 ceph-mon[34294]: pgmap v498: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 857 MiB/s wr, 1.66k op/s 2024-05-09T00:50:36.994 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T00:50:37.544 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:50:37 smithi005 ceph-mon[30365]: pgmap v499: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 980 MiB/s wr, 1.82k op/s 2024-05-09T00:50:37.590 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:50:37.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:50:37 smithi131 ceph-mon[34547]: pgmap v499: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 980 MiB/s wr, 1.82k op/s 2024-05-09T00:50:37.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:50:37 smithi157 ceph-mon[34294]: pgmap v499: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 980 MiB/s wr, 1.82k op/s 2024-05-09T00:50:39.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:50:39 smithi131 ceph-mon[34547]: pgmap v500: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 847 MiB/s wr, 1.47k op/s 2024-05-09T00:50:39.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:50:39 smithi157 ceph-mon[34294]: pgmap v500: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 847 MiB/s wr, 1.47k op/s 2024-05-09T00:50:39.790 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:50:39 smithi005 ceph-mon[30365]: pgmap v500: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 847 MiB/s wr, 1.47k op/s 2024-05-09T00:50:41.531 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:50:41 smithi005 ceph-mon[30365]: pgmap v501: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.0 GiB/s wr, 1.68k op/s 2024-05-09T00:50:41.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:50:41 smithi131 ceph-mon[34547]: pgmap v501: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.0 GiB/s wr, 1.68k op/s 2024-05-09T00:50:41.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:50:41 smithi157 ceph-mon[34294]: pgmap v501: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.0 GiB/s wr, 1.68k op/s 2024-05-09T00:50:42.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.513+0000 7f5f7c9e4700 1 -- 172.21.15.5:0/3293870336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f78108760 msgr2=0x7f5f781100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:50:42.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.513+0000 7f5f7c9e4700 1 --2- 172.21.15.5:0/3293870336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f78108760 0x7f5f781100c0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f5f7000b0d0 tx=0x7f5f7000b3e0 comp rx=0 tx=0).stop 2024-05-09T00:50:42.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.514+0000 7f5f7c9e4700 1 -- 172.21.15.5:0/3293870336 shutdown_connections 2024-05-09T00:50:42.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.514+0000 7f5f7c9e4700 1 --2- 172.21.15.5:0/3293870336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f78108760 0x7f5f781100c0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:42.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.514+0000 7f5f7c9e4700 1 --2- 172.21.15.5:0/3293870336 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f781034e0 0x7f5f781038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:42.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.514+0000 7f5f7c9e4700 1 --2- 172.21.15.5:0/3293870336 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f78107e10 0x7f5f78108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:42.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.514+0000 7f5f7c9e4700 1 -- 172.21.15.5:0/3293870336 >> 172.21.15.5:0/3293870336 conn(0x7f5f780fef30 msgr2=0x7f5f78101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:50:42.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.514+0000 7f5f7c9e4700 1 -- 172.21.15.5:0/3293870336 shutdown_connections 2024-05-09T00:50:42.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.514+0000 7f5f7c9e4700 1 -- 172.21.15.5:0/3293870336 wait complete. 2024-05-09T00:50:42.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.515+0000 7f5f7c9e4700 1 Processor -- start 2024-05-09T00:50:42.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.515+0000 7f5f7c9e4700 1 -- start start 2024-05-09T00:50:42.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.516+0000 7f5f7c9e4700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f781034e0 0x7f5f78199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:42.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.516+0000 7f5f7c9e4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f78107e10 0x7f5f7819a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:42.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.516+0000 7f5f7c9e4700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f78108760 0x7f5f7819e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:42.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.516+0000 7f5f7c9e4700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5f781134d0 con 0x7f5f78107e10 2024-05-09T00:50:42.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.516+0000 7f5f7c9e4700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5f78113350 con 0x7f5f781034e0 2024-05-09T00:50:42.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.516+0000 7f5f7c9e4700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5f78113650 con 0x7f5f78108760 2024-05-09T00:50:42.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.517+0000 7f5f75d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f78107e10 0x7f5f7819a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:42.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.517+0000 7f5f75d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f78107e10 0x7f5f7819a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48940/0 (socket says 172.21.15.5:48940) 2024-05-09T00:50:42.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.517+0000 7f5f75d9b700 1 -- 172.21.15.5:0/4275063806 learned_addr learned my addr 172.21.15.5:0/4275063806 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:50:42.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.517+0000 7f5f76d9d700 1 --2- 172.21.15.5:0/4275063806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f78108760 0x7f5f7819e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:42.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.517+0000 7f5f75d9b700 1 -- 172.21.15.5:0/4275063806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f78108760 msgr2=0x7f5f7819e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:50:42.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.517+0000 7f5f75d9b700 1 --2- 172.21.15.5:0/4275063806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f78108760 0x7f5f7819e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:42.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.517+0000 7f5f75d9b700 1 -- 172.21.15.5:0/4275063806 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f781034e0 msgr2=0x7f5f78199ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:50:42.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.517+0000 7f5f75d9b700 1 --2- 172.21.15.5:0/4275063806 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f781034e0 0x7f5f78199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:42.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.517+0000 7f5f75d9b700 1 -- 172.21.15.5:0/4275063806 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f7819ed70 con 0x7f5f78107e10 2024-05-09T00:50:42.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.518+0000 7f5f75d9b700 1 --2- 172.21.15.5:0/4275063806 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f78107e10 0x7f5f7819a440 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c00b550 tx=0x7f5f6c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:50:42.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.519+0000 7f5f677fe700 1 -- 172.21.15.5:0/4275063806 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c015020 con 0x7f5f78107e10 2024-05-09T00:50:42.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.519+0000 7f5f677fe700 1 -- 172.21.15.5:0/4275063806 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f6c011ce0 con 0x7f5f78107e10 2024-05-09T00:50:42.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.519+0000 7f5f677fe700 1 -- 172.21.15.5:0/4275063806 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c010d60 con 0x7f5f78107e10 2024-05-09T00:50:42.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.519+0000 7f5f7c9e4700 1 -- 172.21.15.5:0/4275063806 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f7819f060 con 0x7f5f78107e10 2024-05-09T00:50:42.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.519+0000 7f5f7c9e4700 1 -- 172.21.15.5:0/4275063806 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f7806c040 con 0x7f5f78107e10 2024-05-09T00:50:42.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.528+0000 7f5f8b3fab80 1 Processor -- start 2024-05-09T00:50:42.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.528+0000 7f5f677fe700 1 -- 172.21.15.5:0/4275063806 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f5f6c018020 con 0x7f5f78107e10 2024-05-09T00:50:42.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.529+0000 7f5f8b3fab80 1 -- start start 2024-05-09T00:50:42.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.529+0000 7f5f8b3fab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618608c0d00 0x5618609c9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:42.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.529+0000 7f5f8b3fab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618608464a0 0x5618609c9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:42.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.530+0000 7f5f8b3fab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5618609c3400 0x5618609c3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:42.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.530+0000 7f5f8b3fab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5618608e60a0 con 0x5618608464a0 2024-05-09T00:50:42.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.530+0000 7f5f8b3fab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x56186088b9a0 con 0x5618609c3400 2024-05-09T00:50:42.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.530+0000 7f5f8b3fab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5618608e2060 con 0x5618608c0d00 2024-05-09T00:50:42.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.530+0000 7f5f7659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618608464a0 0x5618609c9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:42.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.530+0000 7f5f7659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618608464a0 0x5618609c9780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48948/0 (socket says 172.21.15.5:48948) 2024-05-09T00:50:42.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.530+0000 7f5f7659c700 1 -- 172.21.15.5:0/1833930958 learned_addr learned my addr 172.21.15.5:0/1833930958 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:50:42.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.530+0000 7f5f76d9d700 1 --2- 172.21.15.5:0/1833930958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618608c0d00 0x5618609c9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:42.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.531+0000 7f5f677fe700 1 --2- 172.21.15.5:0/4275063806 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f5f600b6d90 0x7f5f600b92a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:42.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.531+0000 7f5f7659c700 1 -- 172.21.15.5:0/1833930958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618608c0d00 msgr2=0x5618609c9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:50:42.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.531+0000 7f5f7659c700 1 --2- 172.21.15.5:0/1833930958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618608c0d00 0x5618609c9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:42.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.531+0000 7f5f7659c700 1 -- 172.21.15.5:0/1833930958 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5618609c3400 msgr2=0x5618609c3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:50:42.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.531+0000 7f5f7659c700 1 --2- 172.21.15.5:0/1833930958 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5618609c3400 0x5618609c3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:42.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.531+0000 7f5f7659c700 1 -- 172.21.15.5:0/1833930958 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56186087c500 con 0x5618608464a0 2024-05-09T00:50:42.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.531+0000 7f5f76d9d700 1 --2- 172.21.15.5:0/4275063806 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f5f600b6d90 0x7f5f600b92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:42.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.531+0000 7f5f677fe700 1 -- 172.21.15.5:0/4275063806 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f5f6c0e1910 con 0x7f5f78107e10 2024-05-09T00:50:42.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.532+0000 7f5f76d9d700 1 --2- 172.21.15.5:0/4275063806 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f5f600b6d90 0x7f5f600b92a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5f7000b820 tx=0x7f5f7000b3e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:50:42.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.532+0000 7f5f7659c700 1 --2- 172.21.15.5:0/1833930958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618608464a0 0x5618609c9780 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f5f680098a0 tx=0x7f5f68009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e7211fbec2dea32 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:50:42.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.532+0000 7f5f64ff9700 1 -- 172.21.15.5:0/1833930958 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f68015020 con 0x5618608464a0 2024-05-09T00:50:42.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.532+0000 7f5f64ff9700 1 -- 172.21.15.5:0/1833930958 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f6800bd20 con 0x5618608464a0 2024-05-09T00:50:42.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.532+0000 7f5f64ff9700 1 -- 172.21.15.5:0/1833930958 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f68011e50 con 0x5618608464a0 2024-05-09T00:50:42.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.533+0000 7f5f8b3fab80 1 -- 172.21.15.5:0/1833930958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618608464a0 msgr2=0x5618609c9780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:50:42.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.533+0000 7f5f8b3fab80 1 --2- 172.21.15.5:0/1833930958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618608464a0 0x5618609c9780 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f5f680098a0 tx=0x7f5f68009bb0 comp rx=0 tx=0).stop 2024-05-09T00:50:42.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.533+0000 7f5f8b3fab80 1 -- 172.21.15.5:0/1833930958 shutdown_connections 2024-05-09T00:50:42.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.533+0000 7f5f8b3fab80 1 --2- 172.21.15.5:0/1833930958 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5618609c3400 0x5618609c3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:42.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.534+0000 7f5f8b3fab80 1 --2- 172.21.15.5:0/1833930958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618608c0d00 0x5618609c9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:42.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.534+0000 7f5f8b3fab80 1 --2- 172.21.15.5:0/1833930958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618608464a0 0x5618609c9780 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:42.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.534+0000 7f5f8b3fab80 1 -- 172.21.15.5:0/1833930958 >> 172.21.15.5:0/1833930958 conn(0x561860849f70 msgr2=0x5618609be220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:50:42.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.534+0000 7f5f8b3fab80 1 -- 172.21.15.5:0/1833930958 shutdown_connections 2024-05-09T00:50:42.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.534+0000 7f5f8b3fab80 1 -- 172.21.15.5:0/1833930958 wait complete. 2024-05-09T00:50:42.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.534+0000 7f5f8b3fab80 1 -- start start 2024-05-09T00:50:42.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.535+0000 7f5f8b3fab80 1 Processor -- start 2024-05-09T00:50:42.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.535+0000 7f5f8b3fab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618608464a0 0x5618609cd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:42.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.535+0000 7f5f8b3fab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5618608c0d00 0x5618609cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:42.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.535+0000 7f5f8b3fab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618609c3400 0x5618609cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:42.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.535+0000 7f5f8b3fab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5618609c4160 con 0x5618608464a0 2024-05-09T00:50:42.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.535+0000 7f5f8b3fab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x561860880b20 con 0x5618608c0d00 2024-05-09T00:50:42.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.535+0000 7f5f8b3fab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5618609d0020 con 0x5618609c3400 2024-05-09T00:50:42.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.535+0000 7f5f8b3fab80 10 client.0 _ll_register_callbacks cb 0x5618607576c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:50:42.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.535+0000 7f5f7659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618608464a0 0x5618609cd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:42.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.535+0000 7f5f7659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618608464a0 0x5618609cd020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48950/0 (socket says 172.21.15.5:48950) 2024-05-09T00:50:42.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.535+0000 7f5f7659c700 1 -- 172.21.15.5:0/3765961676 learned_addr learned my addr 172.21.15.5:0/3765961676 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:50:42.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.536+0000 7f5f76d9d700 1 --2- 172.21.15.5:0/3765961676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618609c3400 0x5618609cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:42.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.536+0000 7f5f7659c700 1 -- 172.21.15.5:0/3765961676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618609c3400 msgr2=0x5618609cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:50:42.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.536+0000 7f5f7659c700 1 --2- 172.21.15.5:0/3765961676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618609c3400 0x5618609cdaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:42.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.536+0000 7f5f7659c700 1 -- 172.21.15.5:0/3765961676 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5618608c0d00 msgr2=0x5618609cd560 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:50:42.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.536+0000 7f5f7659c700 1 --2- 172.21.15.5:0/3765961676 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5618608c0d00 0x5618609cd560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:42.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.536+0000 7f5f7659c700 1 -- 172.21.15.5:0/3765961676 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561860761490 con 0x5618608464a0 2024-05-09T00:50:42.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.536+0000 7f5f7659c700 1 --2- 172.21.15.5:0/3765961676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618608464a0 0x5618609cd020 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f5f680119d0 tx=0x7f5f68011700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1209cbab3a0df63b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:50:42.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.536+0000 7f5f5dffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:50:42.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.537+0000 7f5f5dffb700 1 -- 172.21.15.5:0/3765961676 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6800b670 con 0x5618608464a0 2024-05-09T00:50:42.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.537+0000 7f5f5dffb700 1 -- 172.21.15.5:0/3765961676 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f5f6800b810 con 0x5618608464a0 2024-05-09T00:50:42.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.537+0000 7f5f8b3fab80 1 -- 172.21.15.5:0/3765961676 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f6800bd20 con 0x5618608464a0 2024-05-09T00:50:42.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.538+0000 7f5f5dffb700 1 -- 172.21.15.5:0/3765961676 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5f6800ba30 con 0x5618608464a0 2024-05-09T00:50:42.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.538+0000 7f5f8b3fab80 10 client.15906 fetch_fsmap learned FSMap version 30 2024-05-09T00:50:42.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.538+0000 7f5f8b3fab80 1 -- 172.21.15.5:0/3765961676 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5618609e3290 con 0x5618608464a0 2024-05-09T00:50:42.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.538+0000 7f5f5dffb700 1 -- 172.21.15.5:0/3765961676 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f6800bd20 con 0x5618608464a0 2024-05-09T00:50:42.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.539+0000 7f5f5dffb700 1 -- 172.21.15.5:0/3765961676 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f5f6801a430 con 0x5618608464a0 2024-05-09T00:50:42.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.539+0000 7f5f8b3fab80 10 client.15906 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T00:50:42.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.539+0000 7f5f8b3fab80 10 client.15906 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:50:42.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.539+0000 7f5f8b3fab80 20 client.15906 populate_metadata read hostname 'smithi005' 2024-05-09T00:50:42.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.539+0000 7f5f8b3fab80 1 --2- 172.21.15.5:0/3765961676 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5618609e5810 0x5618609e7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:42.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.540+0000 7f5f8b3fab80 4 client.15906 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:50:42.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.540+0000 7f5f8b3fab80 1 -- 172.21.15.5:0/3765961676 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5618608f3a80 con 0x5618609e5810 2024-05-09T00:50:42.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.541+0000 7f5f75d9b700 1 --2- 172.21.15.5:0/3765961676 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5618609e5810 0x5618609e7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:42.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.546+0000 7f5f75d9b700 1 --2- 172.21.15.5:0/3765961676 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5618609e5810 0x5618609e7c30 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a47f1ff99a1ea08b server_cookie=57463290a62c3109 in_seq=0 out_seq=0 2024-05-09T00:50:42.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.546+0000 7f5f5dffb700 10 client.15906 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:50:42.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.550+0000 7f5f5dffb700 1 -- 172.21.15.5:0/3765961676 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5618608f3a80 con 0x5618609e5810 2024-05-09T00:50:42.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.550+0000 7f5f5dffb700 10 client.15906 handle_command_reply: tid=0 2024-05-09T00:50:42.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.550+0000 7f5f8b3fab80 1 client.15906 shutdown 2024-05-09T00:50:42.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.550+0000 7f5f8b3fab80 1 -- 172.21.15.5:0/3765961676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618608464a0 msgr2=0x5618609cd020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:50:42.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.550+0000 7f5f8b3fab80 1 --2- 172.21.15.5:0/3765961676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618608464a0 0x5618609cd020 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f5f680119d0 tx=0x7f5f68011700 comp rx=0 tx=0).stop 2024-05-09T00:50:42.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.550+0000 7f5f8b3fab80 1 -- 172.21.15.5:0/3765961676 shutdown_connections 2024-05-09T00:50:42.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.550+0000 7f5f8b3fab80 1 --2- 172.21.15.5:0/3765961676 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5618609e5810 0x5618609e7c30 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:42.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.550+0000 7f5f8b3fab80 1 --2- 172.21.15.5:0/3765961676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618609c3400 0x5618609cdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:42.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.550+0000 7f5f8b3fab80 1 --2- 172.21.15.5:0/3765961676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618608464a0 0x5618609cd020 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:42.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.550+0000 7f5f8b3fab80 1 --2- 172.21.15.5:0/3765961676 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5618608c0d00 0x5618609cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:42.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.550+0000 7f5f5dffb700 0 client.15906 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:50:42.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.550+0000 7f5f8b3fab80 1 -- 172.21.15.5:0/3765961676 >> 172.21.15.5:0/3765961676 conn(0x561860849f70 msgr2=0x5618609b7ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:50:42.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.550+0000 7f5f8b3fab80 1 -- 172.21.15.5:0/3765961676 shutdown_connections 2024-05-09T00:50:42.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.550+0000 7f5f8b3fab80 1 -- 172.21.15.5:0/3765961676 wait complete. 2024-05-09T00:50:42.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.551+0000 7f5f8b3fab80 20 client.15906 trim_cache size 0 max 16384 2024-05-09T00:50:42.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.586+0000 7f5f8b3fab80 1 Processor -- start 2024-05-09T00:50:42.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.587+0000 7f5f8b3fab80 1 -- start start 2024-05-09T00:50:42.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.587+0000 7f5f8b3fab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5618608464a0 0x5618609cd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:42.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.587+0000 7f5f8b3fab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618608c0d00 0x5618609e1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:42.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.588+0000 7f5f8b3fab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618609c3400 0x5618609b7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:42.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.588+0000 7f5f8b3fab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5f5800d4f0 con 0x5618609c3400 2024-05-09T00:50:42.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.588+0000 7f5f76d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618609c3400 0x5618609b7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:42.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.588+0000 7f5f76d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618609c3400 0x5618609b7ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48962/0 (socket says 172.21.15.5:48962) 2024-05-09T00:50:42.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.588+0000 7f5f76d9d700 1 -- 172.21.15.5:0/2558842197 learned_addr learned my addr 172.21.15.5:0/2558842197 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:50:42.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.588+0000 7f5f7659c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5618608464a0 0x5618609cd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:42.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.588+0000 7f5f8b3fab80 1 -- 172.21.15.5:0/2558842197 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5f5800f060 con 0x5618608464a0 2024-05-09T00:50:42.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.588+0000 7f5f8b3fab80 1 -- 172.21.15.5:0/2558842197 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5f5800dbc0 con 0x5618608c0d00 2024-05-09T00:50:42.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.589+0000 7f5f75d9b700 1 --2- 172.21.15.5:0/2558842197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618608c0d00 0x5618609e1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:42.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.589+0000 7f5f76d9d700 1 -- 172.21.15.5:0/2558842197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618608c0d00 msgr2=0x5618609e1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:50:42.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.589+0000 7f5f76d9d700 1 --2- 172.21.15.5:0/2558842197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618608c0d00 0x5618609e1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:42.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.589+0000 7f5f76d9d700 1 -- 172.21.15.5:0/2558842197 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5618608464a0 msgr2=0x5618609cd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:50:42.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.589+0000 7f5f76d9d700 1 --2- 172.21.15.5:0/2558842197 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5618608464a0 0x5618609cd8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:42.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.590+0000 7f5f76d9d700 1 -- 172.21.15.5:0/2558842197 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618609ba9d0 con 0x5618609c3400 2024-05-09T00:50:42.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.592+0000 7f5f76d9d700 1 --2- 172.21.15.5:0/2558842197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618609c3400 0x5618609b7ff0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f5f70003d10 tx=0x7f5f70020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24c46cc3a4702406 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:50:42.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.593+0000 7f5f5ffff700 1 -- 172.21.15.5:0/2558842197 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f70003d40 con 0x5618609c3400 2024-05-09T00:50:42.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.593+0000 7f5f5ffff700 1 -- 172.21.15.5:0/2558842197 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f7002e480 con 0x5618609c3400 2024-05-09T00:50:42.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.593+0000 7f5f5ffff700 1 -- 172.21.15.5:0/2558842197 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f700206e0 con 0x5618609c3400 2024-05-09T00:50:42.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.594+0000 7f5f8b3fab80 1 -- 172.21.15.5:0/2558842197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618609c3400 msgr2=0x5618609b7ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:50:42.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.594+0000 7f5f8b3fab80 1 --2- 172.21.15.5:0/2558842197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618609c3400 0x5618609b7ff0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f5f70003d10 tx=0x7f5f70020000 comp rx=0 tx=0).stop 2024-05-09T00:50:42.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.598+0000 7f5f8b3fab80 1 -- 172.21.15.5:0/2558842197 shutdown_connections 2024-05-09T00:50:42.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.598+0000 7f5f8b3fab80 1 --2- 172.21.15.5:0/2558842197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618609c3400 0x5618609b7ff0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:42.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.598+0000 7f5f8b3fab80 1 --2- 172.21.15.5:0/2558842197 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5618608464a0 0x5618609cd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:42.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.598+0000 7f5f8b3fab80 1 --2- 172.21.15.5:0/2558842197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618608c0d00 0x5618609e1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:42.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.598+0000 7f5f8b3fab80 1 -- 172.21.15.5:0/2558842197 >> 172.21.15.5:0/2558842197 conn(0x561860849f70 msgr2=0x5618609bde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:50:42.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.599+0000 7f5f8b3fab80 1 -- 172.21.15.5:0/2558842197 shutdown_connections 2024-05-09T00:50:42.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.599+0000 7f5f8b3fab80 1 -- 172.21.15.5:0/2558842197 wait complete. 2024-05-09T00:50:42.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.600+0000 7f5f8b3fab80 1 -- start start 2024-05-09T00:50:42.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.601+0000 7f5f8b3fab80 1 Processor -- start 2024-05-09T00:50:42.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.602+0000 7f5f8b3fab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5618608464a0 0x5618609ddac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:42.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.602+0000 7f5f8b3fab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618608c0d00 0x5618609de000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:42.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.602+0000 7f5f8b3fab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618609c3400 0x5618609ec1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:42.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.603+0000 7f5f8b3fab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5f5800e960 con 0x5618609c3400 2024-05-09T00:50:42.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.603+0000 7f5f8b3fab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5f5800e290 con 0x5618608464a0 2024-05-09T00:50:42.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.603+0000 7f5f7659c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5618608464a0 0x5618609ddac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:42.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.603+0000 7f5f7659c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5618608464a0 0x5618609ddac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:42550/0 (socket says 172.21.15.5:42550) 2024-05-09T00:50:42.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.603+0000 7f5f7659c700 1 -- 172.21.15.5:0/437745663 learned_addr learned my addr 172.21.15.5:0/437745663 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:50:42.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.603+0000 7f5f8b3fab80 1 -- 172.21.15.5:0/437745663 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5f5800b490 con 0x5618608c0d00 2024-05-09T00:50:42.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.603+0000 7f5f75d9b700 1 --2- 172.21.15.5:0/437745663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618608c0d00 0x5618609de000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:42.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.604+0000 7f5f8b3fab80 10 client.0 _ll_register_callbacks cb 0x5618607576c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:50:42.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.606+0000 7f5f76d9d700 1 --2- 172.21.15.5:0/437745663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618609c3400 0x5618609ec1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:42.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.608+0000 7f5f76d9d700 1 -- 172.21.15.5:0/437745663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618608c0d00 msgr2=0x5618609de000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:50:42.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.608+0000 7f5f76d9d700 1 --2- 172.21.15.5:0/437745663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618608c0d00 0x5618609de000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:42.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.608+0000 7f5f76d9d700 1 -- 172.21.15.5:0/437745663 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5618608464a0 msgr2=0x5618609ddac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:50:42.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.608+0000 7f5f76d9d700 1 --2- 172.21.15.5:0/437745663 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5618608464a0 0x5618609ddac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:42.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.608+0000 7f5f76d9d700 1 -- 172.21.15.5:0/437745663 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618609ec840 con 0x5618609c3400 2024-05-09T00:50:42.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.609+0000 7f5f76d9d700 1 --2- 172.21.15.5:0/437745663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618609c3400 0x5618609ec1c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f5f7000d010 tx=0x7f5f70007c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0bc3e55aa532bd0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:50:42.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.610+0000 7f5f5dffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:50:42.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.614+0000 7f5f5dffb700 1 -- 172.21.15.5:0/437745663 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f70032070 con 0x5618609c3400 2024-05-09T00:50:42.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.614+0000 7f5f5dffb700 1 -- 172.21.15.5:0/437745663 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f5f7002ea40 con 0x5618609c3400 2024-05-09T00:50:42.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.614+0000 7f5f8b3fab80 1 -- 172.21.15.5:0/437745663 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f7002e480 con 0x5618609c3400 2024-05-09T00:50:42.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.618+0000 7f5f5dffb700 1 -- 172.21.15.5:0/437745663 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5f7003c050 con 0x5618609c3400 2024-05-09T00:50:42.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.623+0000 7f5f8b3fab80 10 client.15912 fetch_fsmap learned FSMap version 30 2024-05-09T00:50:42.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.623+0000 7f5f8b3fab80 1 -- 172.21.15.5:0/437745663 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561860a4f650 con 0x5618609c3400 2024-05-09T00:50:42.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.625+0000 7f5f5dffb700 1 -- 172.21.15.5:0/437745663 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f7002e480 con 0x5618609c3400 2024-05-09T00:50:42.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.625+0000 7f5f5dffb700 1 -- 172.21.15.5:0/437745663 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f5f70011070 con 0x5618609c3400 2024-05-09T00:50:42.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.627+0000 7f5f8b3fab80 10 client.15912 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T00:50:42.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.627+0000 7f5f8b3fab80 10 client.15912 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:50:42.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.627+0000 7f5f8b3fab80 20 client.15912 populate_metadata read hostname 'smithi005' 2024-05-09T00:50:42.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.627+0000 7f5f8b3fab80 1 --2- 172.21.15.5:0/437745663 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x561860a4fbd0 0x561860a51ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:42.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.627+0000 7f5f8b3fab80 4 client.15912 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:50:42.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.627+0000 7f5f8b3fab80 1 -- 172.21.15.5:0/437745663 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5618608edb20 con 0x561860a4fbd0 2024-05-09T00:50:42.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.630+0000 7f5f7659c700 1 --2- 172.21.15.5:0/437745663 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x561860a4fbd0 0x561860a51ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:42.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.640+0000 7f5f7659c700 1 --2- 172.21.15.5:0/437745663 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x561860a4fbd0 0x561860a51ff0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a71fd44398f0925 server_cookie=6d8284bcf1a8ce35 in_seq=0 out_seq=0 2024-05-09T00:50:42.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.642+0000 7f5f5dffb700 10 client.15912 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:50:42.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.646+0000 7f5f5dffb700 1 -- 172.21.15.5:0/437745663 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5618608edb20 con 0x561860a4fbd0 2024-05-09T00:50:42.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.646+0000 7f5f5dffb700 10 client.15912 handle_command_reply: tid=0 2024-05-09T00:50:42.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.646+0000 7f5f8b3fab80 1 client.15912 shutdown 2024-05-09T00:50:42.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.654+0000 7f5f8b3fab80 1 -- 172.21.15.5:0/437745663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618609c3400 msgr2=0x5618609ec1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:50:42.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.654+0000 7f5f8b3fab80 1 --2- 172.21.15.5:0/437745663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618609c3400 0x5618609ec1c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f5f7000d010 tx=0x7f5f70007c80 comp rx=0 tx=0).stop 2024-05-09T00:50:42.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.654+0000 7f5f8b3fab80 1 -- 172.21.15.5:0/437745663 shutdown_connections 2024-05-09T00:50:42.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.654+0000 7f5f8b3fab80 1 --2- 172.21.15.5:0/437745663 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x561860a4fbd0 0x561860a51ff0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:42.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.654+0000 7f5f8b3fab80 1 --2- 172.21.15.5:0/437745663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618609c3400 0x5618609ec1c0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:42.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.654+0000 7f5f8b3fab80 1 --2- 172.21.15.5:0/437745663 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5618608464a0 0x5618609ddac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:42.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.654+0000 7f5f8b3fab80 1 --2- 172.21.15.5:0/437745663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618608c0d00 0x5618609de000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:42.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.654+0000 7f5f8b3fab80 1 -- 172.21.15.5:0/437745663 >> 172.21.15.5:0/437745663 conn(0x561860849f70 msgr2=0x5618609b7ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:50:42.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.654+0000 7f5f5dffb700 0 client.15912 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:50:42.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.654+0000 7f5f8b3fab80 1 -- 172.21.15.5:0/437745663 shutdown_connections 2024-05-09T00:50:42.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.655+0000 7f5f8b3fab80 1 -- 172.21.15.5:0/437745663 wait complete. 2024-05-09T00:50:42.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.655+0000 7f5f8b3fab80 20 client.15912 trim_cache size 0 max 16384 2024-05-09T00:50:42.658 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-09T00:50:42.658 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-05-09T00:50:42.658 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-05-09T00:50:42.658 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-09T00:50:42.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.662+0000 7f5f7c9e4700 1 -- 172.21.15.5:0/4275063806 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f5f600b6d90 msgr2=0x7f5f600b92a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:50:42.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.662+0000 7f5f7c9e4700 1 --2- 172.21.15.5:0/4275063806 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f5f600b6d90 0x7f5f600b92a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5f7000b820 tx=0x7f5f7000b3e0 comp rx=0 tx=0).stop 2024-05-09T00:50:42.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.662+0000 7f5f7c9e4700 1 -- 172.21.15.5:0/4275063806 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f78107e10 msgr2=0x7f5f7819a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:50:42.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.662+0000 7f5f7c9e4700 1 --2- 172.21.15.5:0/4275063806 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f78107e10 0x7f5f7819a440 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c00b550 tx=0x7f5f6c00b910 comp rx=0 tx=0).stop 2024-05-09T00:50:42.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.663+0000 7f5f7c9e4700 1 -- 172.21.15.5:0/4275063806 shutdown_connections 2024-05-09T00:50:42.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.663+0000 7f5f7c9e4700 1 --2- 172.21.15.5:0/4275063806 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f5f600b6d90 0x7f5f600b92a0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:42.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.663+0000 7f5f7c9e4700 1 --2- 172.21.15.5:0/4275063806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f78108760 0x7f5f7819e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:42.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.663+0000 7f5f7c9e4700 1 --2- 172.21.15.5:0/4275063806 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f781034e0 0x7f5f78199ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:42.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.663+0000 7f5f7c9e4700 1 --2- 172.21.15.5:0/4275063806 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f78107e10 0x7f5f7819a440 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:42.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.663+0000 7f5f7c9e4700 1 -- 172.21.15.5:0/4275063806 >> 172.21.15.5:0/4275063806 conn(0x7f5f780fef30 msgr2=0x7f5f781096c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:50:42.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.665+0000 7f5f7c9e4700 1 -- 172.21.15.5:0/4275063806 shutdown_connections 2024-05-09T00:50:42.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:42.666+0000 7f5f7c9e4700 1 -- 172.21.15.5:0/4275063806 wait complete. 2024-05-09T00:50:43.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:50:43 smithi131 ceph-mon[34547]: pgmap v502: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 794 MiB/s wr, 1.31k op/s 2024-05-09T00:50:43.685 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:50:43 smithi157 ceph-mon[34294]: pgmap v502: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 794 MiB/s wr, 1.31k op/s 2024-05-09T00:50:43.790 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:50:43 smithi005 ceph-mon[30365]: pgmap v502: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 794 MiB/s wr, 1.31k op/s 2024-05-09T00:50:44.597 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T00:50:44.597 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T00:50:45.469 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:50:45.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:50:45 smithi131 ceph-mon[34547]: pgmap v503: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 850 MiB/s wr, 1.39k op/s 2024-05-09T00:50:45.674 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:50:45 smithi005 ceph-mon[30365]: pgmap v503: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 850 MiB/s wr, 1.39k op/s 2024-05-09T00:50:45.685 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:50:45 smithi157 ceph-mon[34294]: pgmap v503: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 850 MiB/s wr, 1.39k op/s 2024-05-09T00:50:47.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:50:47 smithi005 ceph-mon[30365]: pgmap v504: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 966 MiB/s wr, 1.53k op/s 2024-05-09T00:50:47.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:50:47 smithi157 ceph-mon[34294]: pgmap v504: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 966 MiB/s wr, 1.53k op/s 2024-05-09T00:50:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.706+0000 7f95c77fe640 1 -- 172.21.15.5:0/99490941 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95bc0318a0 con 0x7f95c810e640 2024-05-09T00:50:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.707+0000 7f95cecf9640 1 -- 172.21.15.5:0/99490941 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95c810e640 msgr2=0x7f95c810f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:50:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.707+0000 7f95cecf9640 1 --2- 172.21.15.5:0/99490941 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95c810e640 0x7f95c810f340 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f95bc009900 tx=0x7f95bc02a410 comp rx=0 tx=0).stop 2024-05-09T00:50:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.707+0000 7f95cecf9640 1 -- 172.21.15.5:0/99490941 shutdown_connections 2024-05-09T00:50:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.707+0000 7f95cecf9640 1 --2- 172.21.15.5:0/99490941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95c8114580 0x7f95c81169a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.707+0000 7f95cecf9640 1 --2- 172.21.15.5:0/99490941 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95c810f880 0x7f95c810fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.707+0000 7f95cecf9640 1 --2- 172.21.15.5:0/99490941 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95c810e640 0x7f95c810f340 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.707+0000 7f95cecf9640 1 -- 172.21.15.5:0/99490941 >> 172.21.15.5:0/99490941 conn(0x7f95c806e700 msgr2=0x7f95c810bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:50:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.707+0000 7f95cecf9640 1 -- 172.21.15.5:0/99490941 shutdown_connections 2024-05-09T00:50:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.707+0000 7f95cecf9640 1 -- 172.21.15.5:0/99490941 wait complete. 2024-05-09T00:50:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.708+0000 7f95cecf9640 1 Processor -- start 2024-05-09T00:50:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.708+0000 7f95cecf9640 1 -- start start 2024-05-09T00:50:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.709+0000 7f95cecf9640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95c810e640 0x7f95c81a4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.709+0000 7f95cecf9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95c810f880 0x7f95c81a5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.709+0000 7f95cecf9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95c8114580 0x7f95c81ac2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.709+0000 7f95cecf9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f95c811a1f0 con 0x7f95c810f880 2024-05-09T00:50:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.709+0000 7f95cecf9640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f95c811a070 con 0x7f95c810e640 2024-05-09T00:50:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.709+0000 7f95cecf9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f95c811a370 con 0x7f95c8114580 2024-05-09T00:50:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.709+0000 7f95c7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95c810f880 0x7f95c81a5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.709+0000 7f95c7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95c810f880 0x7f95c81a5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35822/0 (socket says 172.21.15.5:35822) 2024-05-09T00:50:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.709+0000 7f95c7fff640 1 -- 172.21.15.5:0/1934753935 learned_addr learned my addr 172.21.15.5:0/1934753935 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:50:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.710+0000 7f95cd296640 1 --2- 172.21.15.5:0/1934753935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95c8114580 0x7f95c81ac2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.710+0000 7f95cca95640 1 --2- 172.21.15.5:0/1934753935 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95c810e640 0x7f95c81a4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.710+0000 7f95c7fff640 1 -- 172.21.15.5:0/1934753935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95c8114580 msgr2=0x7f95c81ac2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:50:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.710+0000 7f95c7fff640 1 --2- 172.21.15.5:0/1934753935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95c8114580 0x7f95c81ac2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.710+0000 7f95c7fff640 1 -- 172.21.15.5:0/1934753935 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95c810e640 msgr2=0x7f95c81a4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:50:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.710+0000 7f95c7fff640 1 --2- 172.21.15.5:0/1934753935 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95c810e640 0x7f95c81a4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.710+0000 7f95c7fff640 1 -- 172.21.15.5:0/1934753935 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95c81ac940 con 0x7f95c810f880 2024-05-09T00:50:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.710+0000 7f95c7fff640 1 --2- 172.21.15.5:0/1934753935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95c810f880 0x7f95c81a5340 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f95b400d950 tx=0x7f95b400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:50:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.711+0000 7f95c5ffb640 1 -- 172.21.15.5:0/1934753935 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b4017070 con 0x7f95c810f880 2024-05-09T00:50:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.711+0000 7f95c5ffb640 1 -- 172.21.15.5:0/1934753935 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95b4009c30 con 0x7f95c810f880 2024-05-09T00:50:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.711+0000 7f95c5ffb640 1 -- 172.21.15.5:0/1934753935 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b400f620 con 0x7f95c810f880 2024-05-09T00:50:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.711+0000 7f95cecf9640 1 -- 172.21.15.5:0/1934753935 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95c81acc30 con 0x7f95c810f880 2024-05-09T00:50:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.711+0000 7f95cecf9640 1 -- 172.21.15.5:0/1934753935 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95c81ad1d0 con 0x7f95c810f880 2024-05-09T00:50:47.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.714+0000 7f95d5bac740 1 Processor -- start 2024-05-09T00:50:47.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.715+0000 7f95d5bac740 1 -- start start 2024-05-09T00:50:47.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.715+0000 7f95d5bac740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa709881b0 0x55fa70d2c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:47.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.715+0000 7f95d5bac740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa7095f2f0 0x55fa70d2ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:47.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.715+0000 7f95d5bac740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa709674d0 0x55fa70d33980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:47.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.715+0000 7f95d5bac740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fa70c907c0 con 0x55fa7095f2f0 2024-05-09T00:50:47.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.715+0000 7f95d5bac740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55fa70c2c0e0 con 0x55fa709881b0 2024-05-09T00:50:47.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.715+0000 7f95d5bac740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55fa70c8c780 con 0x55fa709674d0 2024-05-09T00:50:47.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.715+0000 7f95cca95640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa7095f2f0 0x55fa70d2ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:47.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.716+0000 7f95cca95640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa7095f2f0 0x55fa70d2ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35832/0 (socket says 172.21.15.5:35832) 2024-05-09T00:50:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.716+0000 7f95cca95640 1 -- 172.21.15.5:0/4185751981 learned_addr learned my addr 172.21.15.5:0/4185751981 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:50:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.716+0000 7f95cca95640 1 -- 172.21.15.5:0/4185751981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa709674d0 msgr2=0x55fa70d33980 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:50:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.716+0000 7f95cca95640 1 --2- 172.21.15.5:0/4185751981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa709674d0 0x55fa70d33980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.716+0000 7f95cca95640 1 -- 172.21.15.5:0/4185751981 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa709881b0 msgr2=0x55fa70d2c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:50:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.716+0000 7f95cca95640 1 --2- 172.21.15.5:0/4185751981 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa709881b0 0x55fa70d2c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.716+0000 7f95cca95640 1 -- 172.21.15.5:0/4185751981 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa70c557c0 con 0x55fa7095f2f0 2024-05-09T00:50:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.716+0000 7f95cca95640 1 --2- 172.21.15.5:0/4185751981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa7095f2f0 0x55fa70d2ca80 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f95bc009900 tx=0x7f95bc004200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bfe532f3c17c5c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:50:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.717+0000 7f95b2ffd640 1 -- 172.21.15.5:0/4185751981 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95bc035030 con 0x55fa7095f2f0 2024-05-09T00:50:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.717+0000 7f95b2ffd640 1 -- 172.21.15.5:0/4185751981 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95bc0044b0 con 0x55fa7095f2f0 2024-05-09T00:50:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.717+0000 7f95b2ffd640 1 -- 172.21.15.5:0/4185751981 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95bc003850 con 0x55fa7095f2f0 2024-05-09T00:50:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.717+0000 7f95c5ffb640 1 -- 172.21.15.5:0/1934753935 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f95b4010460 con 0x7f95c810f880 2024-05-09T00:50:47.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.718+0000 7f95d5bac740 1 -- 172.21.15.5:0/4185751981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa7095f2f0 msgr2=0x55fa70d2ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:50:47.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.718+0000 7f95d5bac740 1 --2- 172.21.15.5:0/4185751981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa7095f2f0 0x55fa70d2ca80 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f95bc009900 tx=0x7f95bc004200 comp rx=0 tx=0).stop 2024-05-09T00:50:47.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.719+0000 7f95d5bac740 1 -- 172.21.15.5:0/4185751981 shutdown_connections 2024-05-09T00:50:47.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.719+0000 7f95d5bac740 1 --2- 172.21.15.5:0/4185751981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa709674d0 0x55fa70d33980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:47.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.719+0000 7f95d5bac740 1 --2- 172.21.15.5:0/4185751981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa7095f2f0 0x55fa70d2ca80 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:47.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.719+0000 7f95d5bac740 1 --2- 172.21.15.5:0/4185751981 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa709881b0 0x55fa70d2c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:47.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.719+0000 7f95d5bac740 1 -- 172.21.15.5:0/4185751981 >> 172.21.15.5:0/4185751981 conn(0x55fa7097c300 msgr2=0x55fa70d27e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:50:47.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.719+0000 7f95d5bac740 1 -- 172.21.15.5:0/4185751981 shutdown_connections 2024-05-09T00:50:47.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.719+0000 7f95c5ffb640 1 --2- 172.21.15.5:0/1934753935 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f95b80b2770 0x7f95b80b4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:47.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.719+0000 7f95d5bac740 1 -- 172.21.15.5:0/4185751981 wait complete. 2024-05-09T00:50:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.719+0000 7f95cca95640 1 --2- 172.21.15.5:0/1934753935 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f95b80b2770 0x7f95b80b4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.720+0000 7f95c5ffb640 1 -- 172.21.15.5:0/1934753935 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f95b40e1e70 con 0x7f95c810f880 2024-05-09T00:50:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.720+0000 7f95d5bac740 1 -- start start 2024-05-09T00:50:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.720+0000 7f95d5bac740 1 Processor -- start 2024-05-09T00:50:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.720+0000 7f95cca95640 1 --2- 172.21.15.5:0/1934753935 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f95b80b2770 0x7f95b80b4c80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f95bc000c00 tx=0x7f95bc02a410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:50:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.721+0000 7f95d5bac740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa7095f2f0 0x55fa70d36240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.721+0000 7f95d5bac740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa709674d0 0x55fa70d36780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.721+0000 7f95d5bac740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa709881b0 0x55fa70d3d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.721+0000 7f95d5bac740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fa70d34370 con 0x55fa709881b0 2024-05-09T00:50:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.721+0000 7f95d5bac740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55fa70c179c0 con 0x55fa709674d0 2024-05-09T00:50:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.721+0000 7f95d5bac740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55fa70d344f0 con 0x55fa7095f2f0 2024-05-09T00:50:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.721+0000 7f95d5bac740 10 client.0 _ll_register_callbacks cb 0x55fa70c44020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:50:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.721+0000 7f95cd296640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa709881b0 0x55fa70d3d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.721+0000 7f95cd296640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa709881b0 0x55fa70d3d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35840/0 (socket says 172.21.15.5:35840) 2024-05-09T00:50:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.721+0000 7f95cd296640 1 -- 172.21.15.5:0/401925240 learned_addr learned my addr 172.21.15.5:0/401925240 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:50:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.721+0000 7f95cd296640 1 -- 172.21.15.5:0/401925240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa7095f2f0 msgr2=0x55fa70d36240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:50:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.721+0000 7f95cd296640 1 --2- 172.21.15.5:0/401925240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa7095f2f0 0x55fa70d36240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.721+0000 7f95cd296640 1 -- 172.21.15.5:0/401925240 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa709674d0 msgr2=0x55fa70d36780 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:50:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.721+0000 7f95cd296640 1 --2- 172.21.15.5:0/401925240 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa709674d0 0x55fa70d36780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.721+0000 7f95cd296640 1 -- 172.21.15.5:0/401925240 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa70c3e1f0 con 0x55fa709881b0 2024-05-09T00:50:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.721+0000 7f95cd296640 1 --2- 172.21.15.5:0/401925240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa709881b0 0x55fa70d3d6e0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f95c000c9d0 tx=0x7f95c000ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c1e6923d099878e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:50:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.721+0000 7f95a7fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:50:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.722+0000 7f95a7fff640 1 -- 172.21.15.5:0/401925240 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c001a060 con 0x55fa709881b0 2024-05-09T00:50:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.722+0000 7f95d5bac740 1 -- 172.21.15.5:0/401925240 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95bc0044b0 con 0x55fa709881b0 2024-05-09T00:50:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.722+0000 7f95a7fff640 1 -- 172.21.15.5:0/401925240 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f95c00079b0 con 0x55fa709881b0 2024-05-09T00:50:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.722+0000 7f95a7fff640 1 -- 172.21.15.5:0/401925240 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f95c0007bb0 con 0x55fa709881b0 2024-05-09T00:50:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.722+0000 7f95d5bac740 10 client.15924 fetch_fsmap learned FSMap version 30 2024-05-09T00:50:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.722+0000 7f95d5bac740 1 -- 172.21.15.5:0/401925240 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fa709f3790 con 0x55fa709881b0 2024-05-09T00:50:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.722+0000 7f95a7fff640 1 -- 172.21.15.5:0/401925240 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95bc0044b0 con 0x55fa709881b0 2024-05-09T00:50:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.722+0000 7f95a7fff640 1 -- 172.21.15.5:0/401925240 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f95c001e3a0 con 0x55fa709881b0 2024-05-09T00:50:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.722+0000 7f95d5bac740 10 client.15924 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T00:50:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.722+0000 7f95d5bac740 10 client.15924 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:50:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.722+0000 7f95d5bac740 20 client.15924 populate_metadata read hostname 'smithi005' 2024-05-09T00:50:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.722+0000 7f95d5bac740 1 --2- 172.21.15.5:0/401925240 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55fa70946d80 0x55fa70d43d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.722+0000 7f95d5bac740 4 client.15924 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:50:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.723+0000 7f95d5bac740 1 -- 172.21.15.5:0/401925240 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa70c9e3d0 con 0x55fa70946d80 2024-05-09T00:50:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.724+0000 7f95cca95640 1 --2- 172.21.15.5:0/401925240 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55fa70946d80 0x55fa70d43d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.725+0000 7f95a7fff640 10 client.15924 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:50:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.725+0000 7f95cca95640 1 --2- 172.21.15.5:0/401925240 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55fa70946d80 0x55fa70d43d30 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29ce60e2313c5582 server_cookie=cc76b857daa9b3a8 in_seq=0 out_seq=0 2024-05-09T00:50:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.730+0000 7f95a7fff640 1 -- 172.21.15.5:0/401925240 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fa70c9e3d0 con 0x55fa70946d80 2024-05-09T00:50:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.730+0000 7f95a7fff640 10 client.15924 handle_command_reply: tid=0 2024-05-09T00:50:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.731+0000 7f95d5bac740 1 client.15924 shutdown 2024-05-09T00:50:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.731+0000 7f95d5bac740 1 -- 172.21.15.5:0/401925240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa709881b0 msgr2=0x55fa70d3d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:50:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.731+0000 7f95d5bac740 1 --2- 172.21.15.5:0/401925240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa709881b0 0x55fa70d3d6e0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f95c000c9d0 tx=0x7f95c000ce90 comp rx=0 tx=0).stop 2024-05-09T00:50:47.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.731+0000 7f95d5bac740 1 -- 172.21.15.5:0/401925240 shutdown_connections 2024-05-09T00:50:47.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.731+0000 7f95d5bac740 1 --2- 172.21.15.5:0/401925240 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55fa70946d80 0x55fa70d43d30 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:47.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.732+0000 7f95d5bac740 1 --2- 172.21.15.5:0/401925240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa709881b0 0x55fa70d3d6e0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:47.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.732+0000 7f95a7fff640 0 client.15924 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:50:47.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.732+0000 7f95d5bac740 1 --2- 172.21.15.5:0/401925240 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa709674d0 0x55fa70d36780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:47.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.732+0000 7f95d5bac740 1 --2- 172.21.15.5:0/401925240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa7095f2f0 0x55fa70d36240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:47.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.732+0000 7f95d5bac740 1 -- 172.21.15.5:0/401925240 >> 172.21.15.5:0/401925240 conn(0x55fa7097c300 msgr2=0x55fa70d26700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:50:47.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.732+0000 7f95d5bac740 1 -- 172.21.15.5:0/401925240 shutdown_connections 2024-05-09T00:50:47.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.732+0000 7f95d5bac740 1 -- 172.21.15.5:0/401925240 wait complete. 2024-05-09T00:50:47.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.732+0000 7f95d5bac740 20 client.15924 trim_cache size 0 max 16384 2024-05-09T00:50:47.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.759+0000 7f95d5bac740 1 Processor -- start 2024-05-09T00:50:47.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.759+0000 7f95d5bac740 1 -- start start 2024-05-09T00:50:47.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.759+0000 7f95d5bac740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa70946d80 0x55fa70d30d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:47.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.759+0000 7f95d5bac740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa7095f2f0 0x55fa70d34fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:47.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.759+0000 7f95d5bac740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa709674d0 0x55fa70d3d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:47.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.759+0000 7f95d5bac740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f95c81b5f30 con 0x55fa7095f2f0 2024-05-09T00:50:47.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.759+0000 7f95d5bac740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f95c81b7aa0 con 0x55fa709674d0 2024-05-09T00:50:47.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.759+0000 7f95d5bac740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f95c81b6600 con 0x55fa70946d80 2024-05-09T00:50:47.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.760+0000 7f95c7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa7095f2f0 0x55fa70d34fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:47.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.760+0000 7f95c7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa7095f2f0 0x55fa70d34fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35846/0 (socket says 172.21.15.5:35846) 2024-05-09T00:50:47.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.760+0000 7f95c7fff640 1 -- 172.21.15.5:0/384998308 learned_addr learned my addr 172.21.15.5:0/384998308 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:50:47.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.760+0000 7f95cd296640 1 --2- 172.21.15.5:0/384998308 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa709674d0 0x55fa70d3d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:47.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.760+0000 7f95c7fff640 1 -- 172.21.15.5:0/384998308 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa70946d80 msgr2=0x55fa70d30d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:50:47.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.760+0000 7f95c7fff640 1 --2- 172.21.15.5:0/384998308 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa70946d80 0x55fa70d30d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:47.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.760+0000 7f95c7fff640 1 -- 172.21.15.5:0/384998308 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa709674d0 msgr2=0x55fa70d3d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:50:47.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.760+0000 7f95c7fff640 1 --2- 172.21.15.5:0/384998308 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa709674d0 0x55fa70d3d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:47.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.760+0000 7f95c7fff640 1 -- 172.21.15.5:0/384998308 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa70a6a5f0 con 0x55fa7095f2f0 2024-05-09T00:50:47.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.760+0000 7f95c7fff640 1 --2- 172.21.15.5:0/384998308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa7095f2f0 0x55fa70d34fd0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f95b4000c00 tx=0x7f95b40a8cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2b7493d3378cb43 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:50:47.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.761+0000 7f95b27fc640 1 -- 172.21.15.5:0/384998308 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b4020070 con 0x55fa7095f2f0 2024-05-09T00:50:47.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.761+0000 7f95b27fc640 1 -- 172.21.15.5:0/384998308 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95b4009de0 con 0x55fa7095f2f0 2024-05-09T00:50:47.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.761+0000 7f95b27fc640 1 -- 172.21.15.5:0/384998308 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b40b0b90 con 0x55fa7095f2f0 2024-05-09T00:50:47.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.762+0000 7f95d5bac740 1 -- 172.21.15.5:0/384998308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa7095f2f0 msgr2=0x55fa70d34fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:50:47.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.762+0000 7f95d5bac740 1 --2- 172.21.15.5:0/384998308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa7095f2f0 0x55fa70d34fd0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f95b4000c00 tx=0x7f95b40a8cc0 comp rx=0 tx=0).stop 2024-05-09T00:50:47.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.762+0000 7f95d5bac740 1 -- 172.21.15.5:0/384998308 shutdown_connections 2024-05-09T00:50:47.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.762+0000 7f95d5bac740 1 --2- 172.21.15.5:0/384998308 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa709674d0 0x55fa70d3d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:47.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.762+0000 7f95d5bac740 1 --2- 172.21.15.5:0/384998308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa7095f2f0 0x55fa70d34fd0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:47.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.762+0000 7f95d5bac740 1 --2- 172.21.15.5:0/384998308 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa70946d80 0x55fa70d30d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:47.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.762+0000 7f95d5bac740 1 -- 172.21.15.5:0/384998308 >> 172.21.15.5:0/384998308 conn(0x55fa7097c300 msgr2=0x55fa70d348c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:50:47.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.762+0000 7f95d5bac740 1 -- 172.21.15.5:0/384998308 shutdown_connections 2024-05-09T00:50:47.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.763+0000 7f95d5bac740 1 -- 172.21.15.5:0/384998308 wait complete. 2024-05-09T00:50:47.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.763+0000 7f95d5bac740 1 -- start start 2024-05-09T00:50:47.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.763+0000 7f95d5bac740 1 Processor -- start 2024-05-09T00:50:47.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.764+0000 7f95d5bac740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa70946d80 0x55fa70d390e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:47.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.764+0000 7f95d5bac740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa7095f2f0 0x55fa70d39620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:47.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.764+0000 7f95d5bac740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa709674d0 0x55fa70d3c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:47.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.764+0000 7f95d5bac740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f95c81b73a0 con 0x55fa70946d80 2024-05-09T00:50:47.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.764+0000 7f95d5bac740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f95c81b6cd0 con 0x55fa7095f2f0 2024-05-09T00:50:47.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.764+0000 7f95d5bac740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f95c81b3f10 con 0x55fa709674d0 2024-05-09T00:50:47.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.764+0000 7f95cca95640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa70946d80 0x55fa70d390e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:47.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.764+0000 7f95d5bac740 10 client.0 _ll_register_callbacks cb 0x7f95c0009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:50:47.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.764+0000 7f95cca95640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa70946d80 0x55fa70d390e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35858/0 (socket says 172.21.15.5:35858) 2024-05-09T00:50:47.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.764+0000 7f95cca95640 1 -- 172.21.15.5:0/4257189051 learned_addr learned my addr 172.21.15.5:0/4257189051 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:50:47.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.764+0000 7f95c7fff640 1 --2- 172.21.15.5:0/4257189051 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa7095f2f0 0x55fa70d39620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:47.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.764+0000 7f95cca95640 1 -- 172.21.15.5:0/4257189051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa709674d0 msgr2=0x55fa70d3c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:50:47.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.764+0000 7f95cca95640 1 --2- 172.21.15.5:0/4257189051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa709674d0 0x55fa70d3c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:47.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.764+0000 7f95cca95640 1 -- 172.21.15.5:0/4257189051 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa7095f2f0 msgr2=0x55fa70d39620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:50:47.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.764+0000 7f95cca95640 1 --2- 172.21.15.5:0/4257189051 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa7095f2f0 0x55fa70d39620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:47.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.764+0000 7f95cca95640 1 -- 172.21.15.5:0/4257189051 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa70c54ed0 con 0x55fa70946d80 2024-05-09T00:50:47.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.764+0000 7f95c7fff640 1 --2- 172.21.15.5:0/4257189051 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa7095f2f0 0x55fa70d39620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:50:47.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.765+0000 7f95cca95640 1 --2- 172.21.15.5:0/4257189051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa70946d80 0x55fa70d390e0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x55fa70d2d8e0 tx=0x7f95bc004370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca3661a4c7b2c6f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:50:47.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.765+0000 7f95a7fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:50:47.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.765+0000 7f95a7fff640 1 -- 172.21.15.5:0/4257189051 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95bc05d030 con 0x55fa70946d80 2024-05-09T00:50:47.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.765+0000 7f95a7fff640 1 -- 172.21.15.5:0/4257189051 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f95bc059330 con 0x55fa70946d80 2024-05-09T00:50:47.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.765+0000 7f95d5bac740 1 -- 172.21.15.5:0/4257189051 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95b4009de0 con 0x55fa70946d80 2024-05-09T00:50:47.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.765+0000 7f95a7fff640 1 -- 172.21.15.5:0/4257189051 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f95bc035040 con 0x55fa70946d80 2024-05-09T00:50:47.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.765+0000 7f95d5bac740 10 client.15930 fetch_fsmap learned FSMap version 30 2024-05-09T00:50:47.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.765+0000 7f95d5bac740 1 -- 172.21.15.5:0/4257189051 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fa70d3db70 con 0x55fa70946d80 2024-05-09T00:50:47.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.765+0000 7f95a7fff640 1 -- 172.21.15.5:0/4257189051 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95b4009de0 con 0x55fa70946d80 2024-05-09T00:50:47.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.766+0000 7f95a7fff640 1 -- 172.21.15.5:0/4257189051 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f95bc003ad0 con 0x55fa70946d80 2024-05-09T00:50:47.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.766+0000 7f95d5bac740 10 client.15930 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T00:50:47.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.766+0000 7f95d5bac740 10 client.15930 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:50:47.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.766+0000 7f95d5bac740 20 client.15930 populate_metadata read hostname 'smithi005' 2024-05-09T00:50:47.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.766+0000 7f95d5bac740 1 --2- 172.21.15.5:0/4257189051 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55fa709881b0 0x55fa70db6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:47.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.766+0000 7f95d5bac740 4 client.15930 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:50:47.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.766+0000 7f95d5bac740 1 -- 172.21.15.5:0/4257189051 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fa70c98240 con 0x55fa709881b0 2024-05-09T00:50:47.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.766+0000 7f95c7fff640 1 --2- 172.21.15.5:0/4257189051 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55fa709881b0 0x55fa70db6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:47.771 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.769+0000 7f95a7fff640 10 client.15930 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:50:47.771 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:47.769+0000 7f95c7fff640 1 --2- 172.21.15.5:0/4257189051 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55fa709881b0 0x55fa70db6900 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bffb467513b35095 server_cookie=d97a32eae564687f in_seq=0 out_seq=0 2024-05-09T00:50:47.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:50:47 smithi131 ceph-mon[34547]: pgmap v504: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 966 MiB/s wr, 1.53k op/s 2024-05-09T00:50:49.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:50:49 smithi157 ceph-mon[34294]: pgmap v505: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 837 MiB/s wr, 1.26k op/s 2024-05-09T00:50:49.790 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:50:49 smithi005 ceph-mon[30365]: pgmap v505: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 837 MiB/s wr, 1.26k op/s 2024-05-09T00:50:49.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:50:49 smithi131 ceph-mon[34547]: pgmap v505: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 837 MiB/s wr, 1.26k op/s 2024-05-09T00:50:50.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:50.583+0000 7fd48bfff700 1 -- 172.21.15.5:0/1857292133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd48c102cf0 msgr2=0x7fd48c103100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:50:50.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:50.583+0000 7fd48bfff700 1 --2- 172.21.15.5:0/1857292133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd48c102cf0 0x7fd48c103100 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fd47c009990 tx=0x7fd47c009ca0 comp rx=0 tx=0).stop 2024-05-09T00:50:50.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:50.584+0000 7fd48bfff700 1 -- 172.21.15.5:0/1857292133 shutdown_connections 2024-05-09T00:50:50.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:50.584+0000 7fd48bfff700 1 --2- 172.21.15.5:0/1857292133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd48c102cf0 0x7fd48c103100 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:50.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:50.584+0000 7fd48bfff700 1 --2- 172.21.15.5:0/1857292133 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd48c06ad80 0x7fd48c1087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:50.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:50.584+0000 7fd48bfff700 1 --2- 172.21.15.5:0/1857292133 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd48c108d20 0x7fd48c1027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:50.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:50.584+0000 7fd48bfff700 1 -- 172.21.15.5:0/1857292133 >> 172.21.15.5:0/1857292133 conn(0x7fd48c078370 msgr2=0x7fd48c0787b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:50:50.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:50.585+0000 7fd48bfff700 1 -- 172.21.15.5:0/1857292133 shutdown_connections 2024-05-09T00:50:50.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:50.585+0000 7fd48bfff700 1 -- 172.21.15.5:0/1857292133 wait complete. 2024-05-09T00:50:50.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:50.586+0000 7fd48bfff700 1 Processor -- start 2024-05-09T00:50:50.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:50.587+0000 7fd48bfff700 1 -- start start 2024-05-09T00:50:50.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:50.588+0000 7fd48bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd48c06ad80 0x7fd48c0727a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:50.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:50.588+0000 7fd48bfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd48c102cf0 0x7fd48c072ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:50.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:50.588+0000 7fd48bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd48c108d20 0x7fd48c073220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:50.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:50.589+0000 7fd48affd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd48c06ad80 0x7fd48c0727a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:50.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:50.589+0000 7fd48affd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd48c06ad80 0x7fd48c0727a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:34990/0 (socket says 172.21.15.5:34990) 2024-05-09T00:50:50.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:50.589+0000 7fd48a7fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd48c102cf0 0x7fd48c072ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:50.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:50.589+0000 7fd48a7fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd48c102cf0 0x7fd48c072ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:42472/0 (socket says 172.21.15.5:42472) 2024-05-09T00:50:50.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:50.589+0000 7fd48a7fc700 1 -- 172.21.15.5:0/69006333 learned_addr learned my addr 172.21.15.5:0/69006333 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:50:50.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:50.589+0000 7fd48bfff700 1 -- 172.21.15.5:0/69006333 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c10c090 con 0x7fd48c108d20 2024-05-09T00:50:50.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:50.589+0000 7fd48bfff700 1 -- 172.21.15.5:0/69006333 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c10bf10 con 0x7fd48c06ad80 2024-05-09T00:50:50.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:50.590+0000 7fd48bfff700 1 -- 172.21.15.5:0/69006333 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c10c210 con 0x7fd48c102cf0 2024-05-09T00:50:50.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:50.591+0000 7fd48b7fe700 1 --2- 172.21.15.5:0/69006333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd48c108d20 0x7fd48c073220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:50.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:50.593+0000 7fd48b7fe700 1 -- 172.21.15.5:0/69006333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd48c102cf0 msgr2=0x7fd48c072ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:50:50.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:50.593+0000 7fd48b7fe700 1 --2- 172.21.15.5:0/69006333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd48c102cf0 0x7fd48c072ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:50.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:50.593+0000 7fd48b7fe700 1 -- 172.21.15.5:0/69006333 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd48c06ad80 msgr2=0x7fd48c0727a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:50:50.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:50.593+0000 7fd48b7fe700 1 --2- 172.21.15.5:0/69006333 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd48c06ad80 0x7fd48c0727a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:50.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:50.593+0000 7fd48b7fe700 1 -- 172.21.15.5:0/69006333 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd48c1be370 con 0x7fd48c108d20 2024-05-09T00:50:50.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:50.595+0000 7fd48a7fc700 1 --2- 172.21.15.5:0/69006333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd48c102cf0 0x7fd48c072ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:50:50.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:50.595+0000 7fd48affd700 1 --2- 172.21.15.5:0/69006333 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd48c06ad80 0x7fd48c0727a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:50:50.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:50.603+0000 7fd48b7fe700 1 --2- 172.21.15.5:0/69006333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd48c108d20 0x7fd48c073220 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fd47c009990 tx=0x7fd47c009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:50:50.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:50.608+0000 7fd483fff700 1 -- 172.21.15.5:0/69006333 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47c026070 con 0x7fd48c108d20 2024-05-09T00:50:50.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:50.608+0000 7fd483fff700 1 -- 172.21.15.5:0/69006333 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd47c00fba0 con 0x7fd48c108d20 2024-05-09T00:50:50.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:50.608+0000 7fd48bfff700 1 -- 172.21.15.5:0/69006333 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd48c1be600 con 0x7fd48c108d20 2024-05-09T00:50:50.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:50.608+0000 7fd483fff700 1 -- 172.21.15.5:0/69006333 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47c004150 con 0x7fd48c108d20 2024-05-09T00:50:50.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:50.608+0000 7fd48bfff700 1 -- 172.21.15.5:0/69006333 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd48c1bea50 con 0x7fd48c108d20 2024-05-09T00:50:50.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:50.621+0000 7fd481ffb700 1 -- 172.21.15.5:0/69006333 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd48c103ef0 con 0x7fd48c108d20 2024-05-09T00:50:50.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:50.621+0000 7fd483fff700 1 -- 172.21.15.5:0/69006333 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fd47c029070 con 0x7fd48c108d20 2024-05-09T00:50:50.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:50.623+0000 7fd483fff700 1 --2- 172.21.15.5:0/69006333 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fd4780b2a90 0x7fd4780b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:50:50.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:50.625+0000 7fd483fff700 1 -- 172.21.15.5:0/69006333 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fd47c0e2bb0 con 0x7fd48c108d20 2024-05-09T00:50:50.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:50.627+0000 7fd48affd700 1 --2- 172.21.15.5:0/69006333 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fd4780b2a90 0x7fd4780b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:50:50.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:50.635+0000 7fd48affd700 1 --2- 172.21.15.5:0/69006333 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fd4780b2a90 0x7fd4780b4fa0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd4840060b0 tx=0x7fd4840076a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:50:50.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:50.643+0000 7fd483fff700 1 -- 172.21.15.5:0/69006333 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd47c013070 con 0x7fd48c108d20 2024-05-09T00:50:51.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:51.325+0000 7fd481ffb700 1 -- 172.21.15.5:0/69006333 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd48c073c80 con 0x7fd48c108d20 2024-05-09T00:50:51.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:51.328+0000 7fd483fff700 1 -- 172.21.15.5:0/69006333 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11927 (secure 0 0 0) 0x7fd47c0aa8d0 con 0x7fd48c108d20 2024-05-09T00:50:51.336 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:50:51.336 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2685771887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2685771887},{"type":"v1","addr":"172.21.15.5:6835","nonce":2685771887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/254017535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":254017535},{"type":"v1","addr":"172.21.15.5:6837","nonce":254017535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T00:43:45.856066+0000","modified":"2024-05-09T00:50:05.997715+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24508,"mds_1":24499,"mds_2":24494,"mds_3":24493,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14598":{"gid":14598,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2109255228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2109255228},{"type":"v1","addr":"172.21.15.5:6839","nonce":2109255228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/3809966329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3809966329},{"type":"v1","addr":"172.21.15.5:6841","nonce":3809966329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6833/3272897742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3272897742},{"type":"v1","addr":"172.21.15.131:6833","nonce":3272897742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1283093898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1283093898},{"type":"v1","addr":"172.21.15.131:6835","nonce":1283093898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/2809525455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2809525455},{"type":"v1","addr":"172.21.15.157:6833","nonce":2809525455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.131:6837/1633006186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":1633006186},{"type":"v1","addr":"172.21.15.131:6837","nonce":1633006186}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.157:6835/832898273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":832898273},{"type":"v1","addr":"172.21.15.157:6835","nonce":832898273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.131:6839/2748669631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2748669631},{"type":"v1","addr":"172.21.15.131:6839","nonce":2748669631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.157:6837/744863009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":744863009},{"type":"v1","addr":"172.21.15.157:6837","nonce":744863009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.157:6839/2779860399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":2779860399},{"type":"v1","addr":"172.21.15.157:6839","nonce":2779860399}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24493,24467,24499,24494,24508]},"id":1}]} 2024-05-09T00:50:51.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:51.340+0000 7fd481ffb700 1 -- 172.21.15.5:0/69006333 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fd4780b2a90 msgr2=0x7fd4780b4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:50:51.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:51.340+0000 7fd481ffb700 1 --2- 172.21.15.5:0/69006333 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fd4780b2a90 0x7fd4780b4fa0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd4840060b0 tx=0x7fd4840076a0 comp rx=0 tx=0).stop 2024-05-09T00:50:51.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:51.340+0000 7fd481ffb700 1 -- 172.21.15.5:0/69006333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd48c108d20 msgr2=0x7fd48c073220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:50:51.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:51.340+0000 7fd481ffb700 1 --2- 172.21.15.5:0/69006333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd48c108d20 0x7fd48c073220 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fd47c009990 tx=0x7fd47c009f70 comp rx=0 tx=0).stop 2024-05-09T00:50:51.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:51.340+0000 7fd481ffb700 1 -- 172.21.15.5:0/69006333 shutdown_connections 2024-05-09T00:50:51.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:51.340+0000 7fd481ffb700 1 --2- 172.21.15.5:0/69006333 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fd4780b2a90 0x7fd4780b4fa0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:51.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:51.341+0000 7fd481ffb700 1 --2- 172.21.15.5:0/69006333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd48c108d20 0x7fd48c073220 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:51.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:51.341+0000 7fd481ffb700 1 --2- 172.21.15.5:0/69006333 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd48c06ad80 0x7fd48c0727a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:51.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:51.341+0000 7fd481ffb700 1 --2- 172.21.15.5:0/69006333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd48c102cf0 0x7fd48c072ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:51.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:51.341+0000 7fd481ffb700 1 -- 172.21.15.5:0/69006333 >> 172.21.15.5:0/69006333 conn(0x7fd48c078370 msgr2=0x7fd48c11d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:50:51.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:51.344+0000 7fd481ffb700 1 -- 172.21.15.5:0/69006333 shutdown_connections 2024-05-09T00:50:51.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:50:51.344+0000 7fd481ffb700 1 -- 172.21.15.5:0/69006333 wait complete. 2024-05-09T00:50:51.349 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 30 2024-05-09T00:50:51.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:50:51 smithi157 ceph-mon[34294]: pgmap v506: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.0 GiB/s wr, 1.59k op/s 2024-05-09T00:50:51.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:50:51 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/69006333' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:50:51.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:50:51 smithi005 ceph-mon[30365]: pgmap v506: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.0 GiB/s wr, 1.59k op/s 2024-05-09T00:50:51.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:50:51 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/69006333' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:50:51.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:50:51 smithi131 ceph-mon[34547]: pgmap v506: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.0 GiB/s wr, 1.59k op/s 2024-05-09T00:50:51.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:50:51 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/69006333' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:50:53.448 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-09T00:50:53.569 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:50:53 smithi005 ceph-mon[30365]: pgmap v507: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 788 MiB/s wr, 1.23k op/s 2024-05-09T00:50:53.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:50:53 smithi157 ceph-mon[34294]: pgmap v507: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 788 MiB/s wr, 1.23k op/s 2024-05-09T00:50:53.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:50:53 smithi131 ceph-mon[34547]: pgmap v507: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 788 MiB/s wr, 1.23k op/s 2024-05-09T00:50:54.232 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:50:55.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:50:55 smithi005 ceph-mon[30365]: pgmap v508: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 846 MiB/s wr, 1.47k op/s 2024-05-09T00:50:55.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:50:55 smithi131 ceph-mon[34547]: pgmap v508: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 846 MiB/s wr, 1.47k op/s 2024-05-09T00:50:55.935 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:50:55 smithi157 ceph-mon[34294]: pgmap v508: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 846 MiB/s wr, 1.47k op/s 2024-05-09T00:50:56.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:56.429+0000 7f95a7fff640 1 -- 172.21.15.5:0/4257189051 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55fa70c98240 con 0x55fa709881b0 2024-05-09T00:50:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:56.429+0000 7f95a7fff640 10 client.15930 handle_command_reply: tid=0 2024-05-09T00:50:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:56.429+0000 7f95d5bac740 1 client.15930 shutdown 2024-05-09T00:50:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:56.429+0000 7f95d5bac740 1 -- 172.21.15.5:0/4257189051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa70946d80 msgr2=0x55fa70d390e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:50:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:56.429+0000 7f95d5bac740 1 --2- 172.21.15.5:0/4257189051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa70946d80 0x55fa70d390e0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x55fa70d2d8e0 tx=0x7f95bc004370 comp rx=0 tx=0).stop 2024-05-09T00:50:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:56.430+0000 7f95d5bac740 1 -- 172.21.15.5:0/4257189051 shutdown_connections 2024-05-09T00:50:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:56.430+0000 7f95d5bac740 1 --2- 172.21.15.5:0/4257189051 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55fa709881b0 0x55fa70db6900 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:56.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:56.430+0000 7f95d5bac740 1 --2- 172.21.15.5:0/4257189051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa709674d0 0x55fa70d3c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:56.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:56.430+0000 7f95d5bac740 1 --2- 172.21.15.5:0/4257189051 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa7095f2f0 0x55fa70d39620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:56.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:56.430+0000 7f95d5bac740 1 --2- 172.21.15.5:0/4257189051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa70946d80 0x55fa70d390e0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:56.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:56.430+0000 7f95d5bac740 1 -- 172.21.15.5:0/4257189051 >> 172.21.15.5:0/4257189051 conn(0x55fa7097c300 msgr2=0x55fa70d26700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:50:56.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:56.430+0000 7f95a7fff640 0 client.15930 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:50:56.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:56.430+0000 7f95d5bac740 1 -- 172.21.15.5:0/4257189051 shutdown_connections 2024-05-09T00:50:56.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:56.430+0000 7f95d5bac740 1 -- 172.21.15.5:0/4257189051 wait complete. 2024-05-09T00:50:56.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:56.430+0000 7f95d5bac740 20 client.15930 trim_cache size 0 max 16384 2024-05-09T00:50:56.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-09T00:50:56.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-09T00:50:56.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-09T00:50:56.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:56.433+0000 7f95cecf9640 1 -- 172.21.15.5:0/1934753935 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f95b80b2770 msgr2=0x7f95b80b4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:50:56.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:56.433+0000 7f95cecf9640 1 --2- 172.21.15.5:0/1934753935 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f95b80b2770 0x7f95b80b4c80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f95bc000c00 tx=0x7f95bc02a410 comp rx=0 tx=0).stop 2024-05-09T00:50:56.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:56.433+0000 7f95cecf9640 1 -- 172.21.15.5:0/1934753935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95c810f880 msgr2=0x7f95c81a5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:50:56.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:56.433+0000 7f95cecf9640 1 --2- 172.21.15.5:0/1934753935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95c810f880 0x7f95c81a5340 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f95b400d950 tx=0x7f95b400de10 comp rx=0 tx=0).stop 2024-05-09T00:50:56.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:56.434+0000 7f95cecf9640 1 -- 172.21.15.5:0/1934753935 shutdown_connections 2024-05-09T00:50:56.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:56.434+0000 7f95cecf9640 1 --2- 172.21.15.5:0/1934753935 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f95b80b2770 0x7f95b80b4c80 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:56.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:56.434+0000 7f95cecf9640 1 --2- 172.21.15.5:0/1934753935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95c8114580 0x7f95c81ac2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:56.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:56.434+0000 7f95cecf9640 1 --2- 172.21.15.5:0/1934753935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95c810f880 0x7f95c81a5340 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:56.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:56.434+0000 7f95cecf9640 1 --2- 172.21.15.5:0/1934753935 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95c810e640 0x7f95c81a4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:50:56.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:56.434+0000 7f95cecf9640 1 -- 172.21.15.5:0/1934753935 >> 172.21.15.5:0/1934753935 conn(0x7f95c806e700 msgr2=0x7f95c8110560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:50:56.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:56.435+0000 7f95cecf9640 1 -- 172.21.15.5:0/1934753935 shutdown_connections 2024-05-09T00:50:56.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:50:56.435+0000 7f95cecf9640 1 -- 172.21.15.5:0/1934753935 wait complete. 2024-05-09T00:50:57.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:50:57 smithi005 ceph-mon[30365]: pgmap v509: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 975 MiB/s wr, 2.10k op/s 2024-05-09T00:50:57.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:50:57 smithi131 ceph-mon[34547]: pgmap v509: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 975 MiB/s wr, 2.10k op/s 2024-05-09T00:50:57.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:50:57 smithi157 ceph-mon[34294]: pgmap v509: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 975 MiB/s wr, 2.10k op/s 2024-05-09T00:50:58.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:50:58 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:50:58.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:50:58 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:50:58.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:50:58 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:50:59.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:50:59 smithi005 ceph-mon[30365]: pgmap v510: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 838 MiB/s wr, 2.21k op/s 2024-05-09T00:50:59.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:50:59 smithi131 ceph-mon[34547]: pgmap v510: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 838 MiB/s wr, 2.21k op/s 2024-05-09T00:50:59.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:50:59 smithi157 ceph-mon[34294]: pgmap v510: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 838 MiB/s wr, 2.21k op/s 2024-05-09T00:51:00.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.051+0000 7f41ebfff700 1 -- 172.21.15.5:0/3886280404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41ec111870 msgr2=0x7f41ec111c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:00.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.051+0000 7f41ebfff700 1 --2- 172.21.15.5:0/3886280404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41ec111870 0x7f41ec111c80 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f41dc009990 tx=0x7f41dc009ca0 comp rx=0 tx=0).stop 2024-05-09T00:51:00.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.052+0000 7f41ebfff700 1 -- 172.21.15.5:0/3886280404 shutdown_connections 2024-05-09T00:51:00.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.052+0000 7f41ebfff700 1 --2- 172.21.15.5:0/3886280404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41ec10b700 0x7f41ec110540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:00.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.052+0000 7f41ebfff700 1 --2- 172.21.15.5:0/3886280404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41ec111870 0x7f41ec111c80 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:00.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.052+0000 7f41ebfff700 1 --2- 172.21.15.5:0/3886280404 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f41ec10adb0 0x7f41ec10b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:00.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.052+0000 7f41ebfff700 1 -- 172.21.15.5:0/3886280404 >> 172.21.15.5:0/3886280404 conn(0x7f41ec074dc0 msgr2=0x7f41ec075200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:51:00.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.054+0000 7f41ebfff700 1 -- 172.21.15.5:0/3886280404 shutdown_connections 2024-05-09T00:51:00.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.056+0000 7f41ebfff700 1 -- 172.21.15.5:0/3886280404 wait complete. 2024-05-09T00:51:00.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.058+0000 7f41ebfff700 1 Processor -- start 2024-05-09T00:51:00.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.058+0000 7f41ebfff700 1 -- start start 2024-05-09T00:51:00.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.059+0000 7f41ebfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41ec10adb0 0x7f41ec1af800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:00.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.059+0000 7f41ebfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41ec10b700 0x7f41ec1afd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:00.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.059+0000 7f41ebfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f41ec111870 0x7f41ec1a9a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:00.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.059+0000 7f41ebfff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f41ec123260 con 0x7f41ec10adb0 2024-05-09T00:51:00.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.059+0000 7f41ebfff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f41ec1230e0 con 0x7f41ec111870 2024-05-09T00:51:00.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.059+0000 7f41ebfff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f41ec1233e0 con 0x7f41ec10b700 2024-05-09T00:51:00.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.064+0000 7f41eaffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41ec10adb0 0x7f41ec1af800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:00.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.064+0000 7f41eaffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41ec10adb0 0x7f41ec1af800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33546/0 (socket says 172.21.15.5:33546) 2024-05-09T00:51:00.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.064+0000 7f41eaffd700 1 -- 172.21.15.5:0/485738013 learned_addr learned my addr 172.21.15.5:0/485738013 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:51:00.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.065+0000 7f41ea7fc700 1 --2- 172.21.15.5:0/485738013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41ec10b700 0x7f41ec1afd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:00.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.066+0000 7f41eb7fe700 1 --2- 172.21.15.5:0/485738013 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f41ec111870 0x7f41ec1a9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:00.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.067+0000 7f41eaffd700 1 -- 172.21.15.5:0/485738013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41ec10b700 msgr2=0x7f41ec1afd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:00.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.067+0000 7f41eaffd700 1 --2- 172.21.15.5:0/485738013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41ec10b700 0x7f41ec1afd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:00.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.067+0000 7f41eaffd700 1 -- 172.21.15.5:0/485738013 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f41ec111870 msgr2=0x7f41ec1a9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:00.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.067+0000 7f41eaffd700 1 --2- 172.21.15.5:0/485738013 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f41ec111870 0x7f41ec1a9a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:00.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.067+0000 7f41eaffd700 1 -- 172.21.15.5:0/485738013 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41ec1aa0d0 con 0x7f41ec10adb0 2024-05-09T00:51:00.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.068+0000 7f41ea7fc700 1 --2- 172.21.15.5:0/485738013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41ec10b700 0x7f41ec1afd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:51:00.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.070+0000 7f41eaffd700 1 --2- 172.21.15.5:0/485738013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41ec10adb0 0x7f41ec1af800 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f41dc0098e0 tx=0x7f41dc003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:51:00.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.073+0000 7f41d3fff700 1 -- 172.21.15.5:0/485738013 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41dc026070 con 0x7f41ec10adb0 2024-05-09T00:51:00.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.073+0000 7f41d3fff700 1 -- 172.21.15.5:0/485738013 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41dc003fc0 con 0x7f41ec10adb0 2024-05-09T00:51:00.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.073+0000 7f41d3fff700 1 -- 172.21.15.5:0/485738013 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41dc00fde0 con 0x7f41ec10adb0 2024-05-09T00:51:00.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.074+0000 7f41ebfff700 1 -- 172.21.15.5:0/485738013 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41ec1aa360 con 0x7f41ec10adb0 2024-05-09T00:51:00.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.074+0000 7f41ebfff700 1 -- 172.21.15.5:0/485738013 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41ec1b6690 con 0x7f41ec10adb0 2024-05-09T00:51:00.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.081+0000 7f41d3fff700 1 -- 172.21.15.5:0/485738013 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f41dc029070 con 0x7f41ec10adb0 2024-05-09T00:51:00.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.085+0000 7f41d3fff700 1 --2- 172.21.15.5:0/485738013 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f41d40b2a90 0x7f41d40b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:00.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.086+0000 7f41d3fff700 1 -- 172.21.15.5:0/485738013 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f41dc0e2dc0 con 0x7f41ec10adb0 2024-05-09T00:51:00.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.088+0000 7f41ea7fc700 1 --2- 172.21.15.5:0/485738013 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f41d40b2a90 0x7f41d40b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:00.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.090+0000 7f41ea7fc700 1 --2- 172.21.15.5:0/485738013 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f41d40b2a90 0x7f41d40b4fa0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f41ec06f8a0 tx=0x7f41e4009550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:51:00.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.094+0000 7f41ffda8b80 1 Processor -- start 2024-05-09T00:51:00.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.097+0000 7f41ffda8b80 1 -- start start 2024-05-09T00:51:00.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.097+0000 7f41ffda8b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556993ec3d00 0x556993fcc490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:00.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.097+0000 7f41ffda8b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556993e494a0 0x556993fcc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:00.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.097+0000 7f41ffda8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556993fc65e0 0x556993fc69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:00.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.097+0000 7f41ffda8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556993ee90a0 con 0x556993fc65e0 2024-05-09T00:51:00.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.097+0000 7f41ffda8b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x556993e8e9a0 con 0x556993ec3d00 2024-05-09T00:51:00.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.097+0000 7f41ffda8b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x556993ee5060 con 0x556993e494a0 2024-05-09T00:51:00.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.098+0000 7f41eb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556993fc65e0 0x556993fc69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:00.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.098+0000 7f41eb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556993fc65e0 0x556993fc69f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33562/0 (socket says 172.21.15.5:33562) 2024-05-09T00:51:00.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.098+0000 7f41eb7fe700 1 -- 172.21.15.5:0/941557910 learned_addr learned my addr 172.21.15.5:0/941557910 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:51:00.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.098+0000 7f41eaffd700 1 --2- 172.21.15.5:0/941557910 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556993ec3d00 0x556993fcc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:00.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.099+0000 7f41eb7fe700 1 -- 172.21.15.5:0/941557910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556993e494a0 msgr2=0x556993fcc9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:00.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.099+0000 7f41eb7fe700 1 --2- 172.21.15.5:0/941557910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556993e494a0 0x556993fcc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:00.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.099+0000 7f41eb7fe700 1 -- 172.21.15.5:0/941557910 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556993ec3d00 msgr2=0x556993fcc490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:00.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.099+0000 7f41eb7fe700 1 --2- 172.21.15.5:0/941557910 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556993ec3d00 0x556993fcc490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:00.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.099+0000 7f41eb7fe700 1 -- 172.21.15.5:0/941557910 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556993e7f500 con 0x556993fc65e0 2024-05-09T00:51:00.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.099+0000 7f41eb7fe700 1 --2- 172.21.15.5:0/941557910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556993fc65e0 0x556993fc69f0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f41e000b820 tx=0x7f41e000bb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d04769ca3fca9a2b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:51:00.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.111+0000 7f41d17fa700 1 -- 172.21.15.5:0/941557910 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e00125f0 con 0x556993fc65e0 2024-05-09T00:51:00.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.111+0000 7f41d17fa700 1 -- 172.21.15.5:0/941557910 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41e0019070 con 0x556993fc65e0 2024-05-09T00:51:00.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.113+0000 7f41ffda8b80 1 -- 172.21.15.5:0/941557910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556993fc65e0 msgr2=0x556993fc69f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:00.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.113+0000 7f41ffda8b80 1 --2- 172.21.15.5:0/941557910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556993fc65e0 0x556993fc69f0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f41e000b820 tx=0x7f41e000bb30 comp rx=0 tx=0).stop 2024-05-09T00:51:00.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.119+0000 7f41ffda8b80 1 -- 172.21.15.5:0/941557910 shutdown_connections 2024-05-09T00:51:00.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.119+0000 7f41ffda8b80 1 --2- 172.21.15.5:0/941557910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556993fc65e0 0x556993fc69f0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:00.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.119+0000 7f41ffda8b80 1 --2- 172.21.15.5:0/941557910 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556993ec3d00 0x556993fcc490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:00.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.119+0000 7f41ffda8b80 1 --2- 172.21.15.5:0/941557910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556993e494a0 0x556993fcc9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:00.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.119+0000 7f41ffda8b80 1 -- 172.21.15.5:0/941557910 >> 172.21.15.5:0/941557910 conn(0x556993e4cf70 msgr2=0x556993fc17e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:51:00.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.121+0000 7f41ffda8b80 1 -- 172.21.15.5:0/941557910 shutdown_connections 2024-05-09T00:51:00.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.122+0000 7f41ffda8b80 1 -- 172.21.15.5:0/941557910 wait complete. 2024-05-09T00:51:00.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.122+0000 7f41ffda8b80 1 -- start start 2024-05-09T00:51:00.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.122+0000 7f41ffda8b80 1 Processor -- start 2024-05-09T00:51:00.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.123+0000 7f41ffda8b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556993e494a0 0x556993fd0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:00.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.123+0000 7f41ffda8b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556993ec3d00 0x556993fd05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:00.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.123+0000 7f41ffda8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556993fc65e0 0x556993fd0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:00.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.124+0000 7f41ffda8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556993fc73e0 con 0x556993fc65e0 2024-05-09T00:51:00.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.124+0000 7f41ffda8b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x556993e83b20 con 0x556993e494a0 2024-05-09T00:51:00.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.124+0000 7f41ffda8b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x556993fd3200 con 0x556993ec3d00 2024-05-09T00:51:00.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.124+0000 7f41ffda8b80 10 client.0 _ll_register_callbacks cb 0x556993d5a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:51:00.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.126+0000 7f41eb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556993fc65e0 0x556993fd0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:00.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.126+0000 7f41eb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556993fc65e0 0x556993fd0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33572/0 (socket says 172.21.15.5:33572) 2024-05-09T00:51:00.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.126+0000 7f41eb7fe700 1 -- 172.21.15.5:0/3471949062 learned_addr learned my addr 172.21.15.5:0/3471949062 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:51:00.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.126+0000 7f41eaffd700 1 --2- 172.21.15.5:0/3471949062 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556993e494a0 0x556993fd0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:00.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.127+0000 7f41ea7fc700 1 --2- 172.21.15.5:0/3471949062 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556993ec3d00 0x556993fd05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:00.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.128+0000 7f41eb7fe700 1 -- 172.21.15.5:0/3471949062 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556993ec3d00 msgr2=0x556993fd05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:00.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.128+0000 7f41eb7fe700 1 --2- 172.21.15.5:0/3471949062 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556993ec3d00 0x556993fd05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:00.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.128+0000 7f41eb7fe700 1 -- 172.21.15.5:0/3471949062 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556993e494a0 msgr2=0x556993fd0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:00.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.128+0000 7f41eb7fe700 1 --2- 172.21.15.5:0/3471949062 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556993e494a0 0x556993fd0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:00.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.128+0000 7f41eb7fe700 1 -- 172.21.15.5:0/3471949062 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556993d64490 con 0x556993fc65e0 2024-05-09T00:51:00.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.129+0000 7f41eb7fe700 1 --2- 172.21.15.5:0/3471949062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556993fc65e0 0x556993fd0b10 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f41e0014610 tx=0x7f41e0012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a25d23949823fa29 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:51:00.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.129+0000 7f41ceff5700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:51:00.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.129+0000 7f41ceff5700 1 -- 172.21.15.5:0/3471949062 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e0023020 con 0x556993fc65e0 2024-05-09T00:51:00.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.129+0000 7f41ffda8b80 1 -- 172.21.15.5:0/3471949062 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41e0019070 con 0x556993fc65e0 2024-05-09T00:51:00.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.130+0000 7f41ceff5700 1 -- 172.21.15.5:0/3471949062 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f41e0012cd0 con 0x556993fc65e0 2024-05-09T00:51:00.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.131+0000 7f41ceff5700 1 -- 172.21.15.5:0/3471949062 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f41e002e050 con 0x556993fc65e0 2024-05-09T00:51:00.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.132+0000 7f41ffda8b80 10 client.15948 fetch_fsmap learned FSMap version 30 2024-05-09T00:51:00.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.132+0000 7f41ffda8b80 1 -- 172.21.15.5:0/3471949062 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556993fddb30 con 0x556993fc65e0 2024-05-09T00:51:00.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.135+0000 7f41ceff5700 1 -- 172.21.15.5:0/3471949062 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41e0019070 con 0x556993fc65e0 2024-05-09T00:51:00.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.136+0000 7f41ceff5700 1 -- 172.21.15.5:0/3471949062 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f41e0013070 con 0x556993fc65e0 2024-05-09T00:51:00.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.137+0000 7f41ffda8b80 10 client.15948 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T00:51:00.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.137+0000 7f41ffda8b80 10 client.15948 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T00:51:00.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.137+0000 7f41ffda8b80 20 client.15948 populate_metadata read hostname 'smithi005' 2024-05-09T00:51:00.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.137+0000 7f41ffda8b80 1 --2- 172.21.15.5:0/3471949062 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x556993fe0110 0x556993fe2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:00.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.137+0000 7f41ffda8b80 4 client.15948 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:51:00.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.137+0000 7f41ffda8b80 1 -- 172.21.15.5:0/3471949062 --> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556993ef6a80 con 0x556993fe0110 2024-05-09T00:51:00.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.140+0000 7f41eaffd700 1 --2- 172.21.15.5:0/3471949062 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x556993fe0110 0x556993fe2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:00.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.143+0000 7f41eaffd700 1 --2- 172.21.15.5:0/3471949062 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x556993fe0110 0x556993fe2530 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5bb930564745a130 server_cookie=3523af599f0c362d in_seq=0 out_seq=0 2024-05-09T00:51:00.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.143+0000 7f41ceff5700 10 client.15948 ms_handle_connect on v2:172.21.15.131:6836/1633006186 2024-05-09T00:51:00.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.144+0000 7f41ceff5700 1 -- 172.21.15.5:0/3471949062 <== mds.4 v2:172.21.15.131:6836/1633006186 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556993ef6a80 con 0x556993fe0110 2024-05-09T00:51:00.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.144+0000 7f41ceff5700 10 client.15948 handle_command_reply: tid=0 2024-05-09T00:51:00.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.144+0000 7f41ffda8b80 1 client.15948 shutdown 2024-05-09T00:51:00.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.145+0000 7f41ffda8b80 1 -- 172.21.15.5:0/3471949062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556993fc65e0 msgr2=0x556993fd0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:00.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.145+0000 7f41ffda8b80 1 --2- 172.21.15.5:0/3471949062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556993fc65e0 0x556993fd0b10 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f41e0014610 tx=0x7f41e0012820 comp rx=0 tx=0).stop 2024-05-09T00:51:00.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.145+0000 7f41ffda8b80 1 -- 172.21.15.5:0/3471949062 shutdown_connections 2024-05-09T00:51:00.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.145+0000 7f41ffda8b80 1 --2- 172.21.15.5:0/3471949062 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x556993fe0110 0x556993fe2530 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:00.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.146+0000 7f41ffda8b80 1 --2- 172.21.15.5:0/3471949062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556993fc65e0 0x556993fd0b10 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:00.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.146+0000 7f41ffda8b80 1 --2- 172.21.15.5:0/3471949062 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556993e494a0 0x556993fd0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:00.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.146+0000 7f41ffda8b80 1 --2- 172.21.15.5:0/3471949062 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556993ec3d00 0x556993fd05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:00.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.146+0000 7f41ceff5700 0 client.15948 ms_handle_reset on v2:172.21.15.131:6836/1633006186 2024-05-09T00:51:00.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.146+0000 7f41ffda8b80 1 -- 172.21.15.5:0/3471949062 >> 172.21.15.5:0/3471949062 conn(0x556993e4cf70 msgr2=0x556993fc12d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:51:00.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.146+0000 7f41ffda8b80 1 -- 172.21.15.5:0/3471949062 shutdown_connections 2024-05-09T00:51:00.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.146+0000 7f41ffda8b80 1 -- 172.21.15.5:0/3471949062 wait complete. 2024-05-09T00:51:00.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.146+0000 7f41ffda8b80 20 client.15948 trim_cache size 0 max 16384 2024-05-09T00:51:00.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.191+0000 7f41ffda8b80 1 Processor -- start 2024-05-09T00:51:00.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.191+0000 7f41ffda8b80 1 -- start start 2024-05-09T00:51:00.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.191+0000 7f41ffda8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556993e494a0 0x556993fd0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:00.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.192+0000 7f41ffda8b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556993ec3d00 0x556993fcf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:00.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.192+0000 7f41ffda8b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556993fc65e0 0x556993fc0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:00.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.192+0000 7f41ffda8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f41ec1bd990 con 0x556993e494a0 2024-05-09T00:51:00.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.192+0000 7f41ffda8b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f41ec1bf500 con 0x556993ec3d00 2024-05-09T00:51:00.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.192+0000 7f41ffda8b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f41ec1be060 con 0x556993fc65e0 2024-05-09T00:51:00.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.194+0000 7f41eb7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556993fc65e0 0x556993fc0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:00.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.194+0000 7f41eb7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556993fc65e0 0x556993fc0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:35470/0 (socket says 172.21.15.5:35470) 2024-05-09T00:51:00.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.194+0000 7f41eb7fe700 1 -- 172.21.15.5:0/3874716321 learned_addr learned my addr 172.21.15.5:0/3874716321 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:51:00.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.195+0000 7f41eb7fe700 1 -- 172.21.15.5:0/3874716321 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556993ec3d00 msgr2=0x556993fcf0b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:51:00.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.195+0000 7f41eb7fe700 1 --2- 172.21.15.5:0/3874716321 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556993ec3d00 0x556993fcf0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:00.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.195+0000 7f41eb7fe700 1 -- 172.21.15.5:0/3874716321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556993e494a0 msgr2=0x556993fd0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:00.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.195+0000 7f41eb7fe700 1 --2- 172.21.15.5:0/3874716321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556993e494a0 0x556993fd0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:00.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.195+0000 7f41eb7fe700 1 -- 172.21.15.5:0/3874716321 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556993fd3070 con 0x556993fc65e0 2024-05-09T00:51:00.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.196+0000 7f41eb7fe700 1 --2- 172.21.15.5:0/3874716321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556993fc65e0 0x556993fc0d60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f41e000b740 tx=0x7f41e00147e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d033bdc929161d27 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:51:00.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.197+0000 7f41d0ff9700 1 -- 172.21.15.5:0/3874716321 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e0023070 con 0x556993fc65e0 2024-05-09T00:51:00.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.197+0000 7f41d0ff9700 1 -- 172.21.15.5:0/3874716321 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41e0019040 con 0x556993fc65e0 2024-05-09T00:51:00.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.199+0000 7f41ffda8b80 1 -- 172.21.15.5:0/3874716321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556993fc65e0 msgr2=0x556993fc0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:00.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.199+0000 7f41ffda8b80 1 --2- 172.21.15.5:0/3874716321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556993fc65e0 0x556993fc0d60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f41e000b740 tx=0x7f41e00147e0 comp rx=0 tx=0).stop 2024-05-09T00:51:00.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.203+0000 7f41ffda8b80 1 -- 172.21.15.5:0/3874716321 shutdown_connections 2024-05-09T00:51:00.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.203+0000 7f41ffda8b80 1 --2- 172.21.15.5:0/3874716321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556993fc65e0 0x556993fc0d60 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:00.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.203+0000 7f41ffda8b80 1 --2- 172.21.15.5:0/3874716321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556993e494a0 0x556993fd0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:00.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.203+0000 7f41ffda8b80 1 --2- 172.21.15.5:0/3874716321 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556993ec3d00 0x556993fcf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:00.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.203+0000 7f41ffda8b80 1 -- 172.21.15.5:0/3874716321 >> 172.21.15.5:0/3874716321 conn(0x556993e4cf70 msgr2=0x556993fce9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:51:00.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.203+0000 7f41ffda8b80 1 -- 172.21.15.5:0/3874716321 shutdown_connections 2024-05-09T00:51:00.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.204+0000 7f41ffda8b80 1 -- 172.21.15.5:0/3874716321 wait complete. 2024-05-09T00:51:00.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.204+0000 7f41ffda8b80 1 -- start start 2024-05-09T00:51:00.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.204+0000 7f41ffda8b80 1 Processor -- start 2024-05-09T00:51:00.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.206+0000 7f41ffda8b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556993e494a0 0x556994042a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:00.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.206+0000 7f41ffda8b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556993ec3d00 0x556994042fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:00.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.206+0000 7f41ffda8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556993fc65e0 0x556993fe2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:00.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.206+0000 7f41ffda8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f41ec1bee00 con 0x556993fc65e0 2024-05-09T00:51:00.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.206+0000 7f41ffda8b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f41ec1be730 con 0x556993ec3d00 2024-05-09T00:51:00.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.206+0000 7f41ffda8b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f41ec1bb9e0 con 0x556993e494a0 2024-05-09T00:51:00.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.206+0000 7f41ffda8b80 10 client.0 _ll_register_callbacks cb 0x556993d5a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:51:00.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.207+0000 7f41eb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556993fc65e0 0x556993fe2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:00.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.207+0000 7f41eb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556993fc65e0 0x556993fe2970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33590/0 (socket says 172.21.15.5:33590) 2024-05-09T00:51:00.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.207+0000 7f41eb7fe700 1 -- 172.21.15.5:0/1738342839 learned_addr learned my addr 172.21.15.5:0/1738342839 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:51:00.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.208+0000 7f41eaffd700 1 --2- 172.21.15.5:0/1738342839 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556993e494a0 0x556994042a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:00.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.209+0000 7f41eaffd700 1 -- 172.21.15.5:0/1738342839 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556993ec3d00 msgr2=0x556994042fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:00.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.209+0000 7f41eaffd700 1 --2- 172.21.15.5:0/1738342839 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556993ec3d00 0x556994042fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:00.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.209+0000 7f41eaffd700 1 -- 172.21.15.5:0/1738342839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556993fc65e0 msgr2=0x556993fe2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:00.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.209+0000 7f41eaffd700 1 --2- 172.21.15.5:0/1738342839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556993fc65e0 0x556993fe2970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:00.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.209+0000 7f41eaffd700 1 -- 172.21.15.5:0/1738342839 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556993fe2fb0 con 0x556993e494a0 2024-05-09T00:51:00.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.210+0000 7f41eaffd700 1 --2- 172.21.15.5:0/1738342839 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556993e494a0 0x556994042a90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f41dc0e0830 tx=0x7f41dc005d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0018e264c33de6c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:51:00.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.210+0000 7f41ceff5700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-09T00:51:00.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.211+0000 7f41ceff5700 1 -- 172.21.15.5:0/1738342839 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41dc0aaae0 con 0x556993e494a0 2024-05-09T00:51:00.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.211+0000 7f41ffda8b80 1 -- 172.21.15.5:0/1738342839 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41e0019040 con 0x556993e494a0 2024-05-09T00:51:00.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.211+0000 7f41ceff5700 1 -- 172.21.15.5:0/1738342839 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f41dc0b2f30 con 0x556993e494a0 2024-05-09T00:51:00.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.224+0000 7f41ceff5700 1 -- 172.21.15.5:0/1738342839 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f41dc004170 con 0x556993e494a0 2024-05-09T00:51:00.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.225+0000 7f41ffda8b80 10 client.25549 fetch_fsmap learned FSMap version 30 2024-05-09T00:51:00.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.225+0000 7f41ffda8b80 1 -- 172.21.15.5:0/1738342839 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55699404a090 con 0x556993e494a0 2024-05-09T00:51:00.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.227+0000 7f41ceff5700 1 -- 172.21.15.5:0/1738342839 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41e0019040 con 0x556993e494a0 2024-05-09T00:51:00.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.227+0000 7f41ceff5700 1 -- 172.21.15.5:0/1738342839 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f41dc0a9640 con 0x556993e494a0 2024-05-09T00:51:00.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.228+0000 7f41ffda8b80 10 client.25549 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T00:51:00.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.228+0000 7f41ffda8b80 10 client.25549 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T00:51:00.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.228+0000 7f41ffda8b80 20 client.25549 populate_metadata read hostname 'smithi005' 2024-05-09T00:51:00.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.228+0000 7f41ffda8b80 1 --2- 172.21.15.5:0/1738342839 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x55699404a720 0x55699404cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:00.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.228+0000 7f41ffda8b80 4 client.25549 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:51:00.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.228+0000 7f41ffda8b80 1 -- 172.21.15.5:0/1738342839 --> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556993fe8530 con 0x55699404a720 2024-05-09T00:51:00.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.231+0000 7f41ea7fc700 1 --2- 172.21.15.5:0/1738342839 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x55699404a720 0x55699404cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:00.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.241+0000 7f41ea7fc700 1 --2- 172.21.15.5:0/1738342839 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x55699404a720 0x55699404cb40 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a95b8a31cc3001b7 server_cookie=b6e8f5fc1539d7a7 in_seq=0 out_seq=0 2024-05-09T00:51:00.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.241+0000 7f41ceff5700 10 client.25549 ms_handle_connect on v2:172.21.15.131:6836/1633006186 2024-05-09T00:51:00.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.247+0000 7f41ceff5700 1 -- 172.21.15.5:0/1738342839 <== mds.4 v2:172.21.15.131:6836/1633006186 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556993fe8530 con 0x55699404a720 2024-05-09T00:51:00.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.247+0000 7f41ceff5700 10 client.25549 handle_command_reply: tid=0 2024-05-09T00:51:00.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.247+0000 7f41ffda8b80 1 client.25549 shutdown 2024-05-09T00:51:00.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.249+0000 7f41ffda8b80 1 -- 172.21.15.5:0/1738342839 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556993e494a0 msgr2=0x556994042a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:00.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.249+0000 7f41ffda8b80 1 --2- 172.21.15.5:0/1738342839 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556993e494a0 0x556994042a90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f41dc0e0830 tx=0x7f41dc005d40 comp rx=0 tx=0).stop 2024-05-09T00:51:00.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.249+0000 7f41ffda8b80 1 -- 172.21.15.5:0/1738342839 shutdown_connections 2024-05-09T00:51:00.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.249+0000 7f41ffda8b80 1 --2- 172.21.15.5:0/1738342839 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x55699404a720 0x55699404cb40 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:00.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.249+0000 7f41ffda8b80 1 --2- 172.21.15.5:0/1738342839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556993fc65e0 0x556993fe2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:00.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.249+0000 7f41ffda8b80 1 --2- 172.21.15.5:0/1738342839 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556993e494a0 0x556994042a90 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:00.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.249+0000 7f41ffda8b80 1 --2- 172.21.15.5:0/1738342839 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556993ec3d00 0x556994042fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:00.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.249+0000 7f41ffda8b80 1 -- 172.21.15.5:0/1738342839 >> 172.21.15.5:0/1738342839 conn(0x556993e4cf70 msgr2=0x556993fca640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:51:00.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.250+0000 7f41ceff5700 0 client.25549 ms_handle_reset on v2:172.21.15.131:6836/1633006186 2024-05-09T00:51:00.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.250+0000 7f41ffda8b80 1 -- 172.21.15.5:0/1738342839 shutdown_connections 2024-05-09T00:51:00.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.250+0000 7f41ffda8b80 1 -- 172.21.15.5:0/1738342839 wait complete. 2024-05-09T00:51:00.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.252+0000 7f41ffda8b80 20 client.25549 trim_cache size 0 max 16384 2024-05-09T00:51:00.255 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-09T00:51:00.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.255+0000 7f41ebfff700 1 -- 172.21.15.5:0/485738013 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f41d40b2a90 msgr2=0x7f41d40b4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:00.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.255+0000 7f41ebfff700 1 --2- 172.21.15.5:0/485738013 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f41d40b2a90 0x7f41d40b4fa0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f41ec06f8a0 tx=0x7f41e4009550 comp rx=0 tx=0).stop 2024-05-09T00:51:00.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.255+0000 7f41ebfff700 1 -- 172.21.15.5:0/485738013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41ec10adb0 msgr2=0x7f41ec1af800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:00.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.255+0000 7f41ebfff700 1 --2- 172.21.15.5:0/485738013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41ec10adb0 0x7f41ec1af800 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f41dc0098e0 tx=0x7f41dc003680 comp rx=0 tx=0).stop 2024-05-09T00:51:00.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.256+0000 7f41ebfff700 1 -- 172.21.15.5:0/485738013 shutdown_connections 2024-05-09T00:51:00.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.256+0000 7f41ebfff700 1 --2- 172.21.15.5:0/485738013 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f41d40b2a90 0x7f41d40b4fa0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:00.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.256+0000 7f41ebfff700 1 --2- 172.21.15.5:0/485738013 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f41ec111870 0x7f41ec1a9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:00.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.256+0000 7f41ebfff700 1 --2- 172.21.15.5:0/485738013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41ec10adb0 0x7f41ec1af800 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:00.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.256+0000 7f41ebfff700 1 --2- 172.21.15.5:0/485738013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41ec10b700 0x7f41ec1afd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:00.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.256+0000 7f41ebfff700 1 -- 172.21.15.5:0/485738013 >> 172.21.15.5:0/485738013 conn(0x7f41ec074dc0 msgr2=0x7f41ec10e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:51:00.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.256+0000 7f41ebfff700 1 -- 172.21.15.5:0/485738013 shutdown_connections 2024-05-09T00:51:00.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:00.256+0000 7f41ebfff700 1 -- 172.21.15.5:0/485738013 wait complete. 2024-05-09T00:51:00.508 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:51:00 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:51:00.508 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:51:00 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:51:00.508 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:51:00 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:51:00.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:51:00 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:51:00.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:51:00 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:51:00.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:51:00 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:51:00.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:51:00 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:51:00.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:51:00 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:51:00.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:51:00 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:51:01.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:51:01 smithi005 ceph-mon[30365]: pgmap v511: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1009 MiB/s wr, 3.02k op/s 2024-05-09T00:51:01.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:51:01 smithi131 ceph-mon[34547]: pgmap v511: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1009 MiB/s wr, 3.02k op/s 2024-05-09T00:51:01.935 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:51:01 smithi157 ceph-mon[34294]: pgmap v511: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1009 MiB/s wr, 3.02k op/s 2024-05-09T00:51:02.516 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-09T00:51:02.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:51:02 smithi005 ceph-mon[30365]: pgmap v512: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 766 MiB/s wr, 2.62k op/s 2024-05-09T00:51:02.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:51:02 smithi131 ceph-mon[34547]: pgmap v512: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 766 MiB/s wr, 2.62k op/s 2024-05-09T00:51:02.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:51:02 smithi157 ceph-mon[34294]: pgmap v512: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 766 MiB/s wr, 2.62k op/s 2024-05-09T00:51:03.447 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:51:05.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:51:05 smithi005 ceph-mon[30365]: pgmap v513: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 871 MiB/s wr, 2.89k op/s 2024-05-09T00:51:05.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:51:05 smithi131 ceph-mon[34547]: pgmap v513: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 871 MiB/s wr, 2.89k op/s 2024-05-09T00:51:05.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:51:05 smithi157 ceph-mon[34294]: pgmap v513: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 871 MiB/s wr, 2.89k op/s 2024-05-09T00:51:07.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:51:07 smithi005 ceph-mon[30365]: pgmap v514: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 964 MiB/s wr, 3.15k op/s 2024-05-09T00:51:07.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:51:07 smithi131 ceph-mon[34547]: pgmap v514: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 964 MiB/s wr, 3.15k op/s 2024-05-09T00:51:07.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:51:07 smithi157 ceph-mon[34294]: pgmap v514: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 964 MiB/s wr, 3.15k op/s 2024-05-09T00:51:08.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.553+0000 7f2997d72700 1 -- 172.21.15.5:0/484986709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f299010b280 msgr2=0x7f299011d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:08.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.553+0000 7f2997d72700 1 --2- 172.21.15.5:0/484986709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f299010b280 0x7f299011d3f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f2984009990 tx=0x7f2984009ca0 comp rx=0 tx=0).stop 2024-05-09T00:51:08.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.555+0000 7f2997d72700 1 -- 172.21.15.5:0/484986709 shutdown_connections 2024-05-09T00:51:08.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.555+0000 7f2997d72700 1 --2- 172.21.15.5:0/484986709 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f299011d930 0x7f299011fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:08.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.555+0000 7f2997d72700 1 --2- 172.21.15.5:0/484986709 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f299010a930 0x7f299010ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:08.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.555+0000 7f2997d72700 1 --2- 172.21.15.5:0/484986709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f299010b280 0x7f299011d3f0 secure :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f2984009990 tx=0x7f2984009ca0 comp rx=0 tx=0).stop 2024-05-09T00:51:08.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.555+0000 7f2997d72700 1 -- 172.21.15.5:0/484986709 >> 172.21.15.5:0/484986709 conn(0x7f299006f4d0 msgr2=0x7f299006f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:51:08.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.555+0000 7f2997d72700 1 -- 172.21.15.5:0/484986709 shutdown_connections 2024-05-09T00:51:08.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.555+0000 7f2997d72700 1 -- 172.21.15.5:0/484986709 wait complete. 2024-05-09T00:51:08.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.557+0000 7f2997d72700 1 Processor -- start 2024-05-09T00:51:08.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.557+0000 7f2997d72700 1 -- start start 2024-05-09T00:51:08.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.558+0000 7f2997d72700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f299010a930 0x7f2990114ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:08.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.558+0000 7f2997d72700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f299011d930 0x7f2990115530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:08.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.558+0000 7f2997d72700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2990115a70 0x7f29901199f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:08.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.558+0000 7f2997d72700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f29901233d0 con 0x7f299010a930 2024-05-09T00:51:08.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.558+0000 7f2997d72700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2990123250 con 0x7f2990115a70 2024-05-09T00:51:08.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.558+0000 7f2997d72700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2990123550 con 0x7f299011d930 2024-05-09T00:51:08.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.558+0000 7f299630f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2990115a70 0x7f29901199f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:08.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.559+0000 7f299630f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2990115a70 0x7f29901199f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:36220/0 (socket says 172.21.15.5:36220) 2024-05-09T00:51:08.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.559+0000 7f299630f700 1 -- 172.21.15.5:0/1629185256 learned_addr learned my addr 172.21.15.5:0/1629185256 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:51:08.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.559+0000 7f299530d700 1 --2- 172.21.15.5:0/1629185256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f299011d930 0x7f2990115530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:08.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.559+0000 7f2995b0e700 1 --2- 172.21.15.5:0/1629185256 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f299010a930 0x7f2990114ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:08.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.559+0000 7f299630f700 1 -- 172.21.15.5:0/1629185256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f299011d930 msgr2=0x7f2990115530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:08.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.559+0000 7f299630f700 1 --2- 172.21.15.5:0/1629185256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f299011d930 0x7f2990115530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:08.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.560+0000 7f299630f700 1 -- 172.21.15.5:0/1629185256 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f299010a930 msgr2=0x7f2990114ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:08.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.560+0000 7f299630f700 1 --2- 172.21.15.5:0/1629185256 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f299010a930 0x7f2990114ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:08.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.560+0000 7f299630f700 1 -- 172.21.15.5:0/1629185256 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2990119f30 con 0x7f2990115a70 2024-05-09T00:51:08.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.562+0000 7f299630f700 1 --2- 172.21.15.5:0/1629185256 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2990115a70 0x7f29901199f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f29800094f0 tx=0x7f298000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:51:08.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.562+0000 7f298effd700 1 -- 172.21.15.5:0/1629185256 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298000ddd0 con 0x7f2990115a70 2024-05-09T00:51:08.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.563+0000 7f298effd700 1 -- 172.21.15.5:0/1629185256 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2980009e20 con 0x7f2990115a70 2024-05-09T00:51:08.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.563+0000 7f2997d72700 1 -- 172.21.15.5:0/1629185256 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f299011a220 con 0x7f2990115a70 2024-05-09T00:51:08.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.563+0000 7f298effd700 1 -- 172.21.15.5:0/1629185256 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2980015400 con 0x7f2990115a70 2024-05-09T00:51:08.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.563+0000 7f2997d72700 1 -- 172.21.15.5:0/1629185256 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f299011a730 con 0x7f2990115a70 2024-05-09T00:51:08.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.572+0000 7f29a6788b80 1 Processor -- start 2024-05-09T00:51:08.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.573+0000 7f298effd700 1 -- 172.21.15.5:0/1629185256 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f29800155a0 con 0x7f2990115a70 2024-05-09T00:51:08.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.574+0000 7f29a6788b80 1 -- start start 2024-05-09T00:51:08.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.575+0000 7f29a6788b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc0510bd00 0x55fc05214240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:08.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.575+0000 7f29a6788b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc050914a0 0x55fc05214780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:08.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.575+0000 7f29a6788b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc0520e400 0x55fc0520e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:08.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.575+0000 7f29a6788b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fc051310a0 con 0x55fc0510bd00 2024-05-09T00:51:08.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.575+0000 7f29a6788b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55fc050d69a0 con 0x55fc0520e400 2024-05-09T00:51:08.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.575+0000 7f29a6788b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55fc0512d060 con 0x55fc050914a0 2024-05-09T00:51:08.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.576+0000 7f299530d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc0520e400 0x55fc0520e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:08.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.576+0000 7f299530d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc0520e400 0x55fc0520e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:36234/0 (socket says 172.21.15.5:36234) 2024-05-09T00:51:08.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.576+0000 7f299530d700 1 -- 172.21.15.5:0/1813939607 learned_addr learned my addr 172.21.15.5:0/1813939607 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:51:08.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.576+0000 7f299530d700 1 --2- 172.21.15.5:0/1813939607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc0510bd00 0x55fc05214240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:08.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.576+0000 7f2995b0e700 1 --2- 172.21.15.5:0/1813939607 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc050914a0 0x55fc05214780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:08.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.577+0000 7f298effd700 1 --2- 172.21.15.5:0/1629185256 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f29780b2a10 0x7f29780b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:08.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.577+0000 7f2995b0e700 1 -- 172.21.15.5:0/1813939607 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc0520e400 msgr2=0x55fc0520e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:08.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.577+0000 7f2995b0e700 1 --2- 172.21.15.5:0/1813939607 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc0520e400 0x55fc0520e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:08.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.577+0000 7f2995b0e700 1 -- 172.21.15.5:0/1813939607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc0510bd00 msgr2=0x55fc05214240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:08.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.577+0000 7f2995b0e700 1 --2- 172.21.15.5:0/1813939607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc0510bd00 0x55fc05214240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:08.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.577+0000 7f2995b0e700 1 -- 172.21.15.5:0/1813939607 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc050c7500 con 0x55fc050914a0 2024-05-09T00:51:08.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.577+0000 7f298effd700 1 -- 172.21.15.5:0/1629185256 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f29800e2ae0 con 0x7f2990115a70 2024-05-09T00:51:08.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.577+0000 7f299630f700 1 --2- 172.21.15.5:0/1629185256 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f29780b2a10 0x7f29780b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:08.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.578+0000 7f2995b0e700 1 --2- 172.21.15.5:0/1813939607 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc050914a0 0x55fc05214780 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f298800b7b0 tx=0x7f298800bac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd05f979f26781d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:51:08.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.578+0000 7f2977fff700 1 -- 172.21.15.5:0/1813939607 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29880125f0 con 0x55fc050914a0 2024-05-09T00:51:08.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.578+0000 7f2977fff700 1 -- 172.21.15.5:0/1813939607 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2988018070 con 0x55fc050914a0 2024-05-09T00:51:08.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.580+0000 7f29a6788b80 1 -- 172.21.15.5:0/1813939607 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc050914a0 msgr2=0x55fc05214780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:08.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.580+0000 7f29a6788b80 1 --2- 172.21.15.5:0/1813939607 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc050914a0 0x55fc05214780 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f298800b7b0 tx=0x7f298800bac0 comp rx=0 tx=0).stop 2024-05-09T00:51:08.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.580+0000 7f29a6788b80 1 -- 172.21.15.5:0/1813939607 shutdown_connections 2024-05-09T00:51:08.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.580+0000 7f29a6788b80 1 --2- 172.21.15.5:0/1813939607 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc0520e400 0x55fc0520e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:08.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.580+0000 7f29a6788b80 1 --2- 172.21.15.5:0/1813939607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc0510bd00 0x55fc05214240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:08.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.580+0000 7f29a6788b80 1 --2- 172.21.15.5:0/1813939607 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc050914a0 0x55fc05214780 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:08.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.580+0000 7f29a6788b80 1 -- 172.21.15.5:0/1813939607 >> 172.21.15.5:0/1813939607 conn(0x55fc05094f70 msgr2=0x55fc05209220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:51:08.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.580+0000 7f29a6788b80 1 -- 172.21.15.5:0/1813939607 shutdown_connections 2024-05-09T00:51:08.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.582+0000 7f29a6788b80 1 -- 172.21.15.5:0/1813939607 wait complete. 2024-05-09T00:51:08.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.582+0000 7f29a6788b80 1 -- start start 2024-05-09T00:51:08.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.583+0000 7f29a6788b80 1 Processor -- start 2024-05-09T00:51:08.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.584+0000 7f29a6788b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc0510bd00 0x55fc0521a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:08.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.584+0000 7f29a6788b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc0520e400 0x55fc05217c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:08.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.584+0000 7f29a6788b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc05218160 0x55fc05218570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:08.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.584+0000 7f29a6788b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fc0520f200 con 0x55fc0510bd00 2024-05-09T00:51:08.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.584+0000 7f29a6788b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55fc050cbb20 con 0x55fc05218160 2024-05-09T00:51:08.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.584+0000 7f29a6788b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55fc0521b020 con 0x55fc0520e400 2024-05-09T00:51:08.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.584+0000 7f29a6788b80 10 client.0 _ll_register_callbacks cb 0x55fc04fa26c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:51:08.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.585+0000 7f299530d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc0510bd00 0x55fc0521a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:08.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.585+0000 7f299530d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc05218160 0x55fc05218570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:08.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.585+0000 7f299530d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc05218160 0x55fc05218570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:36240/0 (socket says 172.21.15.5:36240) 2024-05-09T00:51:08.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.585+0000 7f299530d700 1 -- 172.21.15.5:0/4129076309 learned_addr learned my addr 172.21.15.5:0/4129076309 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:51:08.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.586+0000 7f2995b0e700 1 --2- 172.21.15.5:0/4129076309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc0520e400 0x55fc05217c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:08.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.586+0000 7f299630f700 1 --2- 172.21.15.5:0/1629185256 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f29780b2a10 0x7f29780b4f20 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f29800a9dd0 tx=0x7f298000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:51:08.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.587+0000 7f2995b0e700 1 -- 172.21.15.5:0/4129076309 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc05218160 msgr2=0x55fc05218570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:08.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.587+0000 7f2995b0e700 1 --2- 172.21.15.5:0/4129076309 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc05218160 0x55fc05218570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:08.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.587+0000 7f2995b0e700 1 -- 172.21.15.5:0/4129076309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc0510bd00 msgr2=0x55fc0521a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:08.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.587+0000 7f2995b0e700 1 --2- 172.21.15.5:0/4129076309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc0510bd00 0x55fc0521a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:08.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.587+0000 7f2995b0e700 1 -- 172.21.15.5:0/4129076309 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc04fac490 con 0x55fc0520e400 2024-05-09T00:51:08.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.587+0000 7f2995b0e700 1 --2- 172.21.15.5:0/4129076309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc0520e400 0x55fc05217c20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2988008010 tx=0x7f29880137f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dcc8710363c8e878 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:51:08.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.587+0000 7f29757fa700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-09T00:51:08.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.590+0000 7f29757fa700 1 -- 172.21.15.5:0/4129076309 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2988022070 con 0x55fc0520e400 2024-05-09T00:51:08.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.590+0000 7f29a6788b80 1 -- 172.21.15.5:0/4129076309 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55fc0513ea80 con 0x55fc0520e400 2024-05-09T00:51:08.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.590+0000 7f29757fa700 1 -- 172.21.15.5:0/4129076309 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f2988004550 con 0x55fc0520e400 2024-05-09T00:51:08.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.592+0000 7f29757fa700 1 -- 172.21.15.5:0/4129076309 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2988003740 con 0x55fc0520e400 2024-05-09T00:51:08.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.594+0000 7f29a6788b80 10 client.25561 fetch_fsmap learned FSMap version 30 2024-05-09T00:51:08.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.594+0000 7f29a6788b80 1 -- 172.21.15.5:0/4129076309 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fc05225c20 con 0x55fc0520e400 2024-05-09T00:51:08.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.598+0000 7f29757fa700 1 -- 172.21.15.5:0/4129076309 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55fc0513ea80 con 0x55fc0520e400 2024-05-09T00:51:08.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.598+0000 7f29757fa700 1 -- 172.21.15.5:0/4129076309 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f2988012b70 con 0x55fc0520e400 2024-05-09T00:51:08.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.600+0000 7f29a6788b80 10 client.25561 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T00:51:08.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.600+0000 7f29a6788b80 10 client.25561 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T00:51:08.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.600+0000 7f29a6788b80 20 client.25561 populate_metadata read hostname 'smithi005' 2024-05-09T00:51:08.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.600+0000 7f29a6788b80 1 --2- 172.21.15.5:0/4129076309 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x55fc052284d0 0x55fc0522a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:08.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.600+0000 7f29a6788b80 4 client.25561 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:51:08.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.600+0000 7f29a6788b80 1 -- 172.21.15.5:0/4129076309 --> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc05138b20 con 0x55fc052284d0 2024-05-09T00:51:08.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.604+0000 7f299530d700 1 --2- 172.21.15.5:0/4129076309 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x55fc052284d0 0x55fc0522a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:08.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.615+0000 7f299530d700 1 --2- 172.21.15.5:0/4129076309 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x55fc052284d0 0x55fc0522a930 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b2421999da37f88e server_cookie=7a66ddf985549c6b in_seq=0 out_seq=0 2024-05-09T00:51:08.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.615+0000 7f29757fa700 10 client.25561 ms_handle_connect on v2:172.21.15.157:6834/832898273 2024-05-09T00:51:08.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.618+0000 7f29757fa700 1 -- 172.21.15.5:0/4129076309 <== mds.3 v2:172.21.15.157:6834/832898273 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fc05138b20 con 0x55fc052284d0 2024-05-09T00:51:08.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.618+0000 7f29757fa700 10 client.25561 handle_command_reply: tid=0 2024-05-09T00:51:08.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.618+0000 7f29a6788b80 1 client.25561 shutdown 2024-05-09T00:51:08.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.619+0000 7f29a6788b80 1 -- 172.21.15.5:0/4129076309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc0520e400 msgr2=0x55fc05217c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:08.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.619+0000 7f29a6788b80 1 --2- 172.21.15.5:0/4129076309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc0520e400 0x55fc05217c20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2988008010 tx=0x7f29880137f0 comp rx=0 tx=0).stop 2024-05-09T00:51:08.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.630+0000 7f29a6788b80 1 -- 172.21.15.5:0/4129076309 shutdown_connections 2024-05-09T00:51:08.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.630+0000 7f29a6788b80 1 --2- 172.21.15.5:0/4129076309 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x55fc052284d0 0x55fc0522a930 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:08.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.630+0000 7f29a6788b80 1 --2- 172.21.15.5:0/4129076309 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc05218160 0x55fc05218570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:08.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.630+0000 7f29a6788b80 1 --2- 172.21.15.5:0/4129076309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc0510bd00 0x55fc0521a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:08.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.630+0000 7f29a6788b80 1 --2- 172.21.15.5:0/4129076309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc0520e400 0x55fc05217c20 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:08.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.630+0000 7f29a6788b80 1 -- 172.21.15.5:0/4129076309 >> 172.21.15.5:0/4129076309 conn(0x55fc05094f70 msgr2=0x55fc05202ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:51:08.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.630+0000 7f29757fa700 0 client.25561 ms_handle_reset on v2:172.21.15.157:6834/832898273 2024-05-09T00:51:08.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.632+0000 7f29a6788b80 1 -- 172.21.15.5:0/4129076309 shutdown_connections 2024-05-09T00:51:08.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.632+0000 7f29a6788b80 1 -- 172.21.15.5:0/4129076309 wait complete. 2024-05-09T00:51:08.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.632+0000 7f29a6788b80 20 client.25561 trim_cache size 0 max 16384 2024-05-09T00:51:08.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.689+0000 7f29a6788b80 1 Processor -- start 2024-05-09T00:51:08.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.691+0000 7f29a6788b80 1 -- start start 2024-05-09T00:51:08.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.692+0000 7f29a6788b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc0510bd00 0x55fc05208d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:08.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.693+0000 7f29a6788b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc0520e400 0x55fc05209260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:08.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.693+0000 7f29a6788b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc05218160 0x55fc05216510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:08.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.695+0000 7f29a6788b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f29901b5b00 con 0x55fc0520e400 2024-05-09T00:51:08.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.695+0000 7f29a6788b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f29901b7670 con 0x55fc0510bd00 2024-05-09T00:51:08.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.695+0000 7f29a6788b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f29901b61d0 con 0x55fc05218160 2024-05-09T00:51:08.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.698+0000 7f2995b0e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc0520e400 0x55fc05209260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:08.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.698+0000 7f2995b0e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc0520e400 0x55fc05209260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34304/0 (socket says 172.21.15.5:34304) 2024-05-09T00:51:08.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.698+0000 7f2995b0e700 1 -- 172.21.15.5:0/1589435161 learned_addr learned my addr 172.21.15.5:0/1589435161 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:51:08.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.699+0000 7f299530d700 1 --2- 172.21.15.5:0/1589435161 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc0510bd00 0x55fc05208d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:08.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.699+0000 7f299530d700 1 --2- 172.21.15.5:0/1589435161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc05218160 0x55fc05216510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:08.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.709+0000 7f2995b0e700 1 -- 172.21.15.5:0/1589435161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc05218160 msgr2=0x55fc05216510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:08.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.709+0000 7f2995b0e700 1 --2- 172.21.15.5:0/1589435161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc05218160 0x55fc05216510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:08.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.709+0000 7f2995b0e700 1 -- 172.21.15.5:0/1589435161 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc0510bd00 msgr2=0x55fc05208d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:08.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.709+0000 7f2995b0e700 1 --2- 172.21.15.5:0/1589435161 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc0510bd00 0x55fc05208d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:08.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.709+0000 7f2995b0e700 1 -- 172.21.15.5:0/1589435161 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc052059d0 con 0x55fc0520e400 2024-05-09T00:51:08.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.714+0000 7f2995b0e700 1 --2- 172.21.15.5:0/1589435161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc0520e400 0x55fc05209260 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f2988012f70 tx=0x7f298801e740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=316437208f2e46db server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:51:08.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.716+0000 7f29777fe700 1 -- 172.21.15.5:0/1589435161 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2988022040 con 0x55fc0520e400 2024-05-09T00:51:08.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.716+0000 7f29777fe700 1 -- 172.21.15.5:0/1589435161 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2988018040 con 0x55fc0520e400 2024-05-09T00:51:08.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.716+0000 7f29777fe700 1 -- 172.21.15.5:0/1589435161 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29880038e0 con 0x55fc0520e400 2024-05-09T00:51:08.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.718+0000 7f29a6788b80 1 -- 172.21.15.5:0/1589435161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc0520e400 msgr2=0x55fc05209260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:08.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.718+0000 7f29a6788b80 1 --2- 172.21.15.5:0/1589435161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc0520e400 0x55fc05209260 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f2988012f70 tx=0x7f298801e740 comp rx=0 tx=0).stop 2024-05-09T00:51:08.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.721+0000 7f29a6788b80 1 -- 172.21.15.5:0/1589435161 shutdown_connections 2024-05-09T00:51:08.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.721+0000 7f29a6788b80 1 --2- 172.21.15.5:0/1589435161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc05218160 0x55fc05216510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:08.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.721+0000 7f29a6788b80 1 --2- 172.21.15.5:0/1589435161 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc0510bd00 0x55fc05208d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:08.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.721+0000 7f29a6788b80 1 --2- 172.21.15.5:0/1589435161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc0520e400 0x55fc05209260 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:08.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.721+0000 7f29a6788b80 1 -- 172.21.15.5:0/1589435161 >> 172.21.15.5:0/1589435161 conn(0x55fc05094f70 msgr2=0x55fc0522a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:51:08.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.723+0000 7f29a6788b80 1 -- 172.21.15.5:0/1589435161 shutdown_connections 2024-05-09T00:51:08.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.723+0000 7f29a6788b80 1 -- 172.21.15.5:0/1589435161 wait complete. 2024-05-09T00:51:08.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.724+0000 7f29a6788b80 1 -- start start 2024-05-09T00:51:08.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.724+0000 7f29a6788b80 1 Processor -- start 2024-05-09T00:51:08.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.725+0000 7f29a6788b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc0510bd00 0x55fc05230340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:08.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.725+0000 7f29a6788b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc0520e400 0x55fc05220250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:08.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.726+0000 7f29a6788b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc05218160 0x55fc05220790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:08.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.726+0000 7f29a6788b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f29901b6f70 con 0x55fc0520e400 2024-05-09T00:51:08.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.726+0000 7f29a6788b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f29901b68a0 con 0x55fc0510bd00 2024-05-09T00:51:08.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.726+0000 7f29a6788b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f29901b3aa0 con 0x55fc05218160 2024-05-09T00:51:08.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.726+0000 7f29a6788b80 10 client.0 _ll_register_callbacks cb 0x55fc04fa26c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:51:08.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.727+0000 7f299530d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc0510bd00 0x55fc05230340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:08.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.727+0000 7f299530d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc0510bd00 0x55fc05230340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:36254/0 (socket says 172.21.15.5:36254) 2024-05-09T00:51:08.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.727+0000 7f299530d700 1 -- 172.21.15.5:0/3010530660 learned_addr learned my addr 172.21.15.5:0/3010530660 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:51:08.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.727+0000 7f2995b0e700 1 --2- 172.21.15.5:0/3010530660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc0520e400 0x55fc05220250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:08.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.728+0000 7f299530d700 1 --2- 172.21.15.5:0/3010530660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc05218160 0x55fc05220790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:08.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.729+0000 7f2995b0e700 1 -- 172.21.15.5:0/3010530660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc05218160 msgr2=0x55fc05220790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:08.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.729+0000 7f2995b0e700 1 --2- 172.21.15.5:0/3010530660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc05218160 0x55fc05220790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:08.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.729+0000 7f2995b0e700 1 -- 172.21.15.5:0/3010530660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc0510bd00 msgr2=0x55fc05230340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:08.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.729+0000 7f2995b0e700 1 --2- 172.21.15.5:0/3010530660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc0510bd00 0x55fc05230340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:08.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.729+0000 7f2995b0e700 1 -- 172.21.15.5:0/3010530660 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc05230a60 con 0x55fc0520e400 2024-05-09T00:51:08.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.730+0000 7f2995b0e700 1 --2- 172.21.15.5:0/3010530660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc0520e400 0x55fc05220250 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x55fc0521af00 tx=0x7f298801eae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cd72a5890de3860 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:51:08.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.731+0000 7f29757fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:51:08.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.731+0000 7f29757fa700 1 -- 172.21.15.5:0/3010530660 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2988022040 con 0x55fc0520e400 2024-05-09T00:51:08.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.731+0000 7f29757fa700 1 -- 172.21.15.5:0/3010530660 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f2988004560 con 0x55fc0520e400 2024-05-09T00:51:08.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.731+0000 7f29a6788b80 1 -- 172.21.15.5:0/3010530660 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2988018040 con 0x55fc0520e400 2024-05-09T00:51:08.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.732+0000 7f29757fa700 1 -- 172.21.15.5:0/3010530660 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f29880038e0 con 0x55fc0520e400 2024-05-09T00:51:08.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.732+0000 7f29a6788b80 10 client.15969 fetch_fsmap learned FSMap version 30 2024-05-09T00:51:08.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.733+0000 7f29a6788b80 1 -- 172.21.15.5:0/3010530660 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fc05291c80 con 0x55fc0520e400 2024-05-09T00:51:08.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.734+0000 7f29757fa700 1 -- 172.21.15.5:0/3010530660 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29880125f0 con 0x55fc0520e400 2024-05-09T00:51:08.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.734+0000 7f29757fa700 1 -- 172.21.15.5:0/3010530660 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f2988027440 con 0x55fc0520e400 2024-05-09T00:51:08.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.734+0000 7f29a6788b80 10 client.15969 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T00:51:08.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.736+0000 7f29a6788b80 10 client.15969 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T00:51:08.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.736+0000 7f29a6788b80 20 client.15969 populate_metadata read hostname 'smithi005' 2024-05-09T00:51:08.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.736+0000 7f29a6788b80 1 --2- 172.21.15.5:0/3010530660 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x55fc05292120 0x55fc05294540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:08.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.736+0000 7f29a6788b80 4 client.15969 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:51:08.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.736+0000 7f29a6788b80 1 -- 172.21.15.5:0/3010530660 --> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f2988018040 con 0x55fc05292120 2024-05-09T00:51:08.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.737+0000 7f299530d700 1 --2- 172.21.15.5:0/3010530660 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x55fc05292120 0x55fc05294540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:08.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.739+0000 7f299530d700 1 --2- 172.21.15.5:0/3010530660 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x55fc05292120 0x55fc05294540 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2f12a2b924a8680f server_cookie=e9f8456dad4edc48 in_seq=0 out_seq=0 2024-05-09T00:51:08.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.740+0000 7f29757fa700 10 client.15969 ms_handle_connect on v2:172.21.15.157:6834/832898273 2024-05-09T00:51:08.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.740+0000 7f29757fa700 1 -- 172.21.15.5:0/3010530660 <== mds.3 v2:172.21.15.157:6834/832898273 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f2988018040 con 0x55fc05292120 2024-05-09T00:51:08.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.740+0000 7f29757fa700 10 client.15969 handle_command_reply: tid=0 2024-05-09T00:51:08.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.740+0000 7f29a6788b80 1 client.15969 shutdown 2024-05-09T00:51:08.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.740+0000 7f29a6788b80 1 -- 172.21.15.5:0/3010530660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc0520e400 msgr2=0x55fc05220250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:08.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.740+0000 7f29a6788b80 1 --2- 172.21.15.5:0/3010530660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc0520e400 0x55fc05220250 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x55fc0521af00 tx=0x7f298801eae0 comp rx=0 tx=0).stop 2024-05-09T00:51:08.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.741+0000 7f29a6788b80 1 -- 172.21.15.5:0/3010530660 shutdown_connections 2024-05-09T00:51:08.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.741+0000 7f29a6788b80 1 --2- 172.21.15.5:0/3010530660 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x55fc05292120 0x55fc05294540 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:08.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.741+0000 7f29a6788b80 1 --2- 172.21.15.5:0/3010530660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc05218160 0x55fc05220790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:08.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.741+0000 7f29a6788b80 1 --2- 172.21.15.5:0/3010530660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc0510bd00 0x55fc05230340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:08.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.741+0000 7f29a6788b80 1 --2- 172.21.15.5:0/3010530660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc0520e400 0x55fc05220250 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:08.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.741+0000 7f29a6788b80 1 -- 172.21.15.5:0/3010530660 >> 172.21.15.5:0/3010530660 conn(0x55fc05094f70 msgr2=0x55fc05216420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:51:08.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.741+0000 7f29757fa700 0 client.15969 ms_handle_reset on v2:172.21.15.157:6834/832898273 2024-05-09T00:51:08.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.741+0000 7f29a6788b80 1 -- 172.21.15.5:0/3010530660 shutdown_connections 2024-05-09T00:51:08.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.742+0000 7f29a6788b80 1 -- 172.21.15.5:0/3010530660 wait complete. 2024-05-09T00:51:08.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.742+0000 7f29a6788b80 20 client.15969 trim_cache size 0 max 16384 2024-05-09T00:51:08.743 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-09T00:51:08.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.745+0000 7f2997d72700 1 -- 172.21.15.5:0/1629185256 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f29780b2a10 msgr2=0x7f29780b4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:08.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.746+0000 7f2997d72700 1 --2- 172.21.15.5:0/1629185256 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f29780b2a10 0x7f29780b4f20 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f29800a9dd0 tx=0x7f298000b040 comp rx=0 tx=0).stop 2024-05-09T00:51:08.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.746+0000 7f2997d72700 1 -- 172.21.15.5:0/1629185256 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2990115a70 msgr2=0x7f29901199f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:08.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.746+0000 7f2997d72700 1 --2- 172.21.15.5:0/1629185256 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2990115a70 0x7f29901199f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f29800094f0 tx=0x7f298000d3b0 comp rx=0 tx=0).stop 2024-05-09T00:51:08.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.750+0000 7f2997d72700 1 -- 172.21.15.5:0/1629185256 shutdown_connections 2024-05-09T00:51:08.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.750+0000 7f2997d72700 1 --2- 172.21.15.5:0/1629185256 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f29780b2a10 0x7f29780b4f20 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:08.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.750+0000 7f2997d72700 1 --2- 172.21.15.5:0/1629185256 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2990115a70 0x7f29901199f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:08.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.750+0000 7f2997d72700 1 --2- 172.21.15.5:0/1629185256 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f299010a930 0x7f2990114ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:08.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.750+0000 7f2997d72700 1 --2- 172.21.15.5:0/1629185256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f299011d930 0x7f2990115530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:08.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.750+0000 7f2997d72700 1 -- 172.21.15.5:0/1629185256 >> 172.21.15.5:0/1629185256 conn(0x7f299006f4d0 msgr2=0x7f2990072790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:51:08.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.753+0000 7f2997d72700 1 -- 172.21.15.5:0/1629185256 shutdown_connections 2024-05-09T00:51:08.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:08.753+0000 7f2997d72700 1 -- 172.21.15.5:0/1629185256 wait complete. 2024-05-09T00:51:09.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:51:09 smithi157 ceph-mon[34294]: pgmap v515: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 819 MiB/s wr, 2.60k op/s 2024-05-09T00:51:09.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:51:09 smithi005 ceph-mon[30365]: pgmap v515: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 819 MiB/s wr, 2.60k op/s 2024-05-09T00:51:09.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:51:09 smithi131 ceph-mon[34547]: pgmap v515: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 819 MiB/s wr, 2.60k op/s 2024-05-09T00:51:09.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.769+0000 7f7ad495e640 1 -- 172.21.15.5:0/2113126443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ad0102b50 msgr2=0x7f7ad0103850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:09.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.769+0000 7f7ad495e640 1 --2- 172.21.15.5:0/2113126443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ad0102b50 0x7f7ad0103850 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f7ac000d910 tx=0x7f7ac002c370 comp rx=0 tx=0).stop 2024-05-09T00:51:09.771 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.769+0000 7f7ad495e640 1 -- 172.21.15.5:0/2113126443 shutdown_connections 2024-05-09T00:51:09.771 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.769+0000 7f7ad495e640 1 --2- 172.21.15.5:0/2113126443 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ad0108a90 0x7f7ad010aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:09.771 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.769+0000 7f7ad495e640 1 --2- 172.21.15.5:0/2113126443 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ad0103d90 0x7f7ad01041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:09.771 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.770+0000 7f7ad495e640 1 --2- 172.21.15.5:0/2113126443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ad0102b50 0x7f7ad0103850 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:09.771 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.770+0000 7f7ad495e640 1 -- 172.21.15.5:0/2113126443 >> 172.21.15.5:0/2113126443 conn(0x7f7ad00fdd40 msgr2=0x7f7ad01001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:51:09.771 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.770+0000 7f7ad495e640 1 -- 172.21.15.5:0/2113126443 shutdown_connections 2024-05-09T00:51:09.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.770+0000 7f7ad495e640 1 -- 172.21.15.5:0/2113126443 wait complete. 2024-05-09T00:51:09.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.772+0000 7f7ad495e640 1 Processor -- start 2024-05-09T00:51:09.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.772+0000 7f7ad495e640 1 -- start start 2024-05-09T00:51:09.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.773+0000 7f7ad495e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ad0102b50 0x7f7ad01a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:09.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.773+0000 7f7ad495e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ad0103d90 0x7f7ad01a1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:09.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.773+0000 7f7ad495e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ad0108a90 0x7f7ad01a8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:09.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.773+0000 7f7ad495e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad010d630 con 0x7f7ad0103d90 2024-05-09T00:51:09.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.773+0000 7f7ad495e640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad010d4b0 con 0x7f7ad0102b50 2024-05-09T00:51:09.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.773+0000 7f7ad495e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad010d7b0 con 0x7f7ad0108a90 2024-05-09T00:51:09.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.773+0000 7f7ac5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ad0103d90 0x7f7ad01a1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:09.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.773+0000 7f7ac5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ad0103d90 0x7f7ad01a1070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34334/0 (socket says 172.21.15.5:34334) 2024-05-09T00:51:09.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.773+0000 7f7ac5d9b640 1 -- 172.21.15.5:0/2723498005 learned_addr learned my addr 172.21.15.5:0/2723498005 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:51:09.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.774+0000 7f7ac5d9b640 1 -- 172.21.15.5:0/2723498005 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ad0108a90 msgr2=0x7f7ad01a8000 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T00:51:09.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.774+0000 7f7ac5d9b640 1 --2- 172.21.15.5:0/2723498005 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ad0108a90 0x7f7ad01a8000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:09.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.774+0000 7f7ac5d9b640 1 -- 172.21.15.5:0/2723498005 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ad0102b50 msgr2=0x7f7ad01a0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:09.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.774+0000 7f7ac5d9b640 1 --2- 172.21.15.5:0/2723498005 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ad0102b50 0x7f7ad01a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:09.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.774+0000 7f7ac5d9b640 1 -- 172.21.15.5:0/2723498005 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ad01a8670 con 0x7f7ad0103d90 2024-05-09T00:51:09.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.775+0000 7f7ac5d9b640 1 --2- 172.21.15.5:0/2723498005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ad0103d90 0x7f7ad01a1070 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f7ab800b4a0 tx=0x7f7ab800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:51:09.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.775+0000 7f7ac7fff640 1 -- 172.21.15.5:0/2723498005 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab8014070 con 0x7f7ad0103d90 2024-05-09T00:51:09.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.775+0000 7f7ac7fff640 1 -- 172.21.15.5:0/2723498005 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ab8010ac0 con 0x7f7ad0103d90 2024-05-09T00:51:09.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.775+0000 7f7ad495e640 1 -- 172.21.15.5:0/2723498005 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ad01a8960 con 0x7f7ad0103d90 2024-05-09T00:51:09.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.775+0000 7f7ac7fff640 1 -- 172.21.15.5:0/2723498005 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab8017c30 con 0x7f7ad0103d90 2024-05-09T00:51:09.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.775+0000 7f7ad495e640 1 -- 172.21.15.5:0/2723498005 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ad01a8ee0 con 0x7f7ad0103d90 2024-05-09T00:51:09.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.781+0000 7f7adb815740 1 Processor -- start 2024-05-09T00:51:09.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.782+0000 7f7adb815740 1 -- start start 2024-05-09T00:51:09.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.783+0000 7f7adb815740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6f1bdb1b0 0x55d6f1f7f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:09.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.783+0000 7f7adb815740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d6f1bb22f0 0x55d6f1f7fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:09.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.783+0000 7f7adb815740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6f1bba4d0 0x55d6f1f86980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:09.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.783+0000 7f7adb815740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d6f1ee37c0 con 0x55d6f1bba4d0 2024-05-09T00:51:09.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.783+0000 7f7adb815740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55d6f1e7f0e0 con 0x55d6f1bb22f0 2024-05-09T00:51:09.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.783+0000 7f7adb815740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d6f1edf780 con 0x55d6f1bdb1b0 2024-05-09T00:51:09.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.784+0000 7f7ac7fff640 1 -- 172.21.15.5:0/2723498005 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f7ab8017070 con 0x7f7ad0103d90 2024-05-09T00:51:09.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.784+0000 7f7aced9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6f1bdb1b0 0x55d6f1f7f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:09.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.784+0000 7f7ac5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6f1bba4d0 0x55d6f1f86980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:09.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.784+0000 7f7aced9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6f1bdb1b0 0x55d6f1f7f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:43142/0 (socket says 172.21.15.5:43142) 2024-05-09T00:51:09.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.784+0000 7f7aced9d640 1 -- 172.21.15.5:0/3200795655 learned_addr learned my addr 172.21.15.5:0/3200795655 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:51:09.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.785+0000 7f7aced9d640 1 -- 172.21.15.5:0/3200795655 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d6f1bb22f0 msgr2=0x55d6f1f7fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:09.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.785+0000 7f7aced9d640 1 --2- 172.21.15.5:0/3200795655 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d6f1bb22f0 0x55d6f1f7fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:09.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.785+0000 7f7aced9d640 1 -- 172.21.15.5:0/3200795655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6f1bba4d0 msgr2=0x55d6f1f86980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:09.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.785+0000 7f7aced9d640 1 --2- 172.21.15.5:0/3200795655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6f1bba4d0 0x55d6f1f86980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:09.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.785+0000 7f7aced9d640 1 -- 172.21.15.5:0/3200795655 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6f1ea87c0 con 0x55d6f1bdb1b0 2024-05-09T00:51:09.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.785+0000 7f7ac5d9b640 1 --2- 172.21.15.5:0/3200795655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6f1bba4d0 0x55d6f1f86980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:51:09.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.787+0000 7f7ac7fff640 1 --2- 172.21.15.5:0/2723498005 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f7abc0b27e0 0x7f7abc0b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:09.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.787+0000 7f7ac7fff640 1 -- 172.21.15.5:0/2723498005 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f7ab80e1a50 con 0x7f7ad0103d90 2024-05-09T00:51:09.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.787+0000 7f7ace59c640 1 --2- 172.21.15.5:0/2723498005 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f7abc0b27e0 0x7f7abc0b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:09.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.788+0000 7f7aced9d640 1 --2- 172.21.15.5:0/3200795655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6f1bdb1b0 0x55d6f1f7f500 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7ac8009d00 tx=0x7f7ac800b420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=914823dae935cedf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:51:09.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.788+0000 7f7ace59c640 1 --2- 172.21.15.5:0/2723498005 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f7abc0b27e0 0x7f7abc0b4cf0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7ac00048e0 tx=0x7f7ac000d570 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:51:09.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.789+0000 7f7ac4d99640 1 -- 172.21.15.5:0/3200795655 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac800bd40 con 0x55d6f1bdb1b0 2024-05-09T00:51:09.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.789+0000 7f7ac4d99640 1 -- 172.21.15.5:0/3200795655 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ac8017070 con 0x55d6f1bdb1b0 2024-05-09T00:51:09.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.790+0000 7f7ac4d99640 1 -- 172.21.15.5:0/3200795655 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac80116c0 con 0x55d6f1bdb1b0 2024-05-09T00:51:09.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.791+0000 7f7adb815740 1 -- 172.21.15.5:0/3200795655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6f1bdb1b0 msgr2=0x55d6f1f7f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:09.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.791+0000 7f7adb815740 1 --2- 172.21.15.5:0/3200795655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6f1bdb1b0 0x55d6f1f7f500 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7ac8009d00 tx=0x7f7ac800b420 comp rx=0 tx=0).stop 2024-05-09T00:51:09.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.791+0000 7f7adb815740 1 -- 172.21.15.5:0/3200795655 shutdown_connections 2024-05-09T00:51:09.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.791+0000 7f7adb815740 1 --2- 172.21.15.5:0/3200795655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6f1bba4d0 0x55d6f1f86980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:09.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.791+0000 7f7adb815740 1 --2- 172.21.15.5:0/3200795655 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d6f1bb22f0 0x55d6f1f7fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:09.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.792+0000 7f7adb815740 1 --2- 172.21.15.5:0/3200795655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6f1bdb1b0 0x55d6f1f7f500 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:09.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.792+0000 7f7adb815740 1 -- 172.21.15.5:0/3200795655 >> 172.21.15.5:0/3200795655 conn(0x55d6f1bcf300 msgr2=0x55d6f1f7ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:51:09.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.793+0000 7f7adb815740 1 -- 172.21.15.5:0/3200795655 shutdown_connections 2024-05-09T00:51:09.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.794+0000 7f7adb815740 1 -- 172.21.15.5:0/3200795655 wait complete. 2024-05-09T00:51:09.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.795+0000 7f7adb815740 1 -- start start 2024-05-09T00:51:09.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.795+0000 7f7adb815740 1 Processor -- start 2024-05-09T00:51:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.796+0000 7f7adb815740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6f1bb22f0 0x55d6f1f893f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.796+0000 7f7adb815740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d6f1bba4d0 0x55d6f1f89930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.796+0000 7f7adb815740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6f1bdb1b0 0x55d6f1f90890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.796+0000 7f7adb815740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d6f1f87420 con 0x55d6f1bdb1b0 2024-05-09T00:51:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.796+0000 7f7adb815740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55d6f1e6a9c0 con 0x55d6f1bba4d0 2024-05-09T00:51:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.796+0000 7f7adb815740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d6f1f875a0 con 0x55d6f1bb22f0 2024-05-09T00:51:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.796+0000 7f7aced9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6f1bdb1b0 0x55d6f1f90890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.796+0000 7f7aced9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6f1bdb1b0 0x55d6f1f90890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34350/0 (socket says 172.21.15.5:34350) 2024-05-09T00:51:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.796+0000 7f7aced9d640 1 -- 172.21.15.5:0/1284759808 learned_addr learned my addr 172.21.15.5:0/1284759808 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:51:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.796+0000 7f7adb815740 10 client.0 _ll_register_callbacks cb 0x55d6f1e97020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:51:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.797+0000 7f7ace59c640 1 --2- 172.21.15.5:0/1284759808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6f1bb22f0 0x55d6f1f893f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.797+0000 7f7aced9d640 1 -- 172.21.15.5:0/1284759808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6f1bb22f0 msgr2=0x55d6f1f893f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.797+0000 7f7aced9d640 1 --2- 172.21.15.5:0/1284759808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6f1bb22f0 0x55d6f1f893f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.797+0000 7f7aced9d640 1 -- 172.21.15.5:0/1284759808 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d6f1bba4d0 msgr2=0x55d6f1f89930 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:51:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.797+0000 7f7aced9d640 1 --2- 172.21.15.5:0/1284759808 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d6f1bba4d0 0x55d6f1f89930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.797+0000 7f7aced9d640 1 -- 172.21.15.5:0/1284759808 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6f1e911f0 con 0x55d6f1bdb1b0 2024-05-09T00:51:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.797+0000 7f7aced9d640 1 --2- 172.21.15.5:0/1284759808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6f1bdb1b0 0x55d6f1f90890 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f7ac800d010 tx=0x7f7ac8012660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbee0e5aa42b8544 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:51:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.797+0000 7f7ab5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:51:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.798+0000 7f7ab5ffb640 1 -- 172.21.15.5:0/1284759808 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac80255f0 con 0x55d6f1bdb1b0 2024-05-09T00:51:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.798+0000 7f7ab5ffb640 1 -- 172.21.15.5:0/1284759808 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f7ac801b3f0 con 0x55d6f1bdb1b0 2024-05-09T00:51:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.798+0000 7f7adb815740 1 -- 172.21.15.5:0/1284759808 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ac8017070 con 0x55d6f1bdb1b0 2024-05-09T00:51:09.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.798+0000 7f7ab5ffb640 1 -- 172.21.15.5:0/1284759808 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7ac8012a60 con 0x55d6f1bdb1b0 2024-05-09T00:51:09.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.798+0000 7f7adb815740 10 client.15981 fetch_fsmap learned FSMap version 30 2024-05-09T00:51:09.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.798+0000 7f7adb815740 1 -- 172.21.15.5:0/1284759808 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d6f1c46790 con 0x55d6f1bdb1b0 2024-05-09T00:51:09.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.799+0000 7f7ab5ffb640 1 -- 172.21.15.5:0/1284759808 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ac8017070 con 0x55d6f1bdb1b0 2024-05-09T00:51:09.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.799+0000 7f7ab5ffb640 1 -- 172.21.15.5:0/1284759808 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f7ac8011070 con 0x55d6f1bdb1b0 2024-05-09T00:51:09.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.800+0000 7f7adb815740 10 client.15981 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T00:51:09.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.800+0000 7f7adb815740 10 client.15981 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:51:09.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.800+0000 7f7adb815740 20 client.15981 populate_metadata read hostname 'smithi005' 2024-05-09T00:51:09.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.800+0000 7f7adb815740 1 --2- 172.21.15.5:0/1284759808 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55d6f1b99d80 0x55d6f1f96a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:09.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.800+0000 7f7adb815740 4 client.15981 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:51:09.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.800+0000 7f7adb815740 1 -- 172.21.15.5:0/1284759808 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d6f1ef13d0 con 0x55d6f1b99d80 2024-05-09T00:51:09.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.800+0000 7f7ace59c640 1 --2- 172.21.15.5:0/1284759808 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55d6f1b99d80 0x55d6f1f96a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:09.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.803+0000 7f7ace59c640 1 --2- 172.21.15.5:0/1284759808 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55d6f1b99d80 0x55d6f1f96a10 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d2b4f1317fd3c4e server_cookie=80b22602dcf493c6 in_seq=0 out_seq=0 2024-05-09T00:51:09.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.803+0000 7f7ab5ffb640 10 client.15981 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:51:09.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.809+0000 7f7ab5ffb640 1 -- 172.21.15.5:0/1284759808 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d6f1ef13d0 con 0x55d6f1b99d80 2024-05-09T00:51:09.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.809+0000 7f7ab5ffb640 10 client.15981 handle_command_reply: tid=0 2024-05-09T00:51:09.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.809+0000 7f7adb815740 1 client.15981 shutdown 2024-05-09T00:51:09.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.810+0000 7f7adb815740 1 -- 172.21.15.5:0/1284759808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6f1bdb1b0 msgr2=0x55d6f1f90890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:09.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.810+0000 7f7adb815740 1 --2- 172.21.15.5:0/1284759808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6f1bdb1b0 0x55d6f1f90890 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f7ac800d010 tx=0x7f7ac8012660 comp rx=0 tx=0).stop 2024-05-09T00:51:09.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.810+0000 7f7adb815740 1 -- 172.21.15.5:0/1284759808 shutdown_connections 2024-05-09T00:51:09.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.810+0000 7f7adb815740 1 --2- 172.21.15.5:0/1284759808 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55d6f1b99d80 0x55d6f1f96a10 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.810+0000 7f7ab5ffb640 0 client.15981 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:51:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.810+0000 7f7adb815740 1 --2- 172.21.15.5:0/1284759808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6f1bdb1b0 0x55d6f1f90890 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.810+0000 7f7adb815740 1 --2- 172.21.15.5:0/1284759808 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d6f1bba4d0 0x55d6f1f89930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.810+0000 7f7adb815740 1 --2- 172.21.15.5:0/1284759808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6f1bb22f0 0x55d6f1f893f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.810+0000 7f7adb815740 1 -- 172.21.15.5:0/1284759808 >> 172.21.15.5:0/1284759808 conn(0x55d6f1bcf300 msgr2=0x55d6f1f79700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:51:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.810+0000 7f7adb815740 1 -- 172.21.15.5:0/1284759808 shutdown_connections 2024-05-09T00:51:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.811+0000 7f7adb815740 1 -- 172.21.15.5:0/1284759808 wait complete. 2024-05-09T00:51:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.811+0000 7f7adb815740 20 client.15981 trim_cache size 0 max 16384 2024-05-09T00:51:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.841+0000 7f7adb815740 1 Processor -- start 2024-05-09T00:51:09.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.842+0000 7f7adb815740 1 -- start start 2024-05-09T00:51:09.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.842+0000 7f7adb815740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6f1bcf300 0x55d6f1f88100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:09.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.842+0000 7f7adb815740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6f1b99d80 0x55d6f1f907b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:09.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.842+0000 7f7adb815740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d6f1bb22f0 0x55d6f1f90cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:09.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.842+0000 7f7adb815740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad01b1ce0 con 0x55d6f1bcf300 2024-05-09T00:51:09.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.842+0000 7f7adb815740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad01b3850 con 0x55d6f1bb22f0 2024-05-09T00:51:09.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.842+0000 7f7adb815740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad01b23b0 con 0x55d6f1b99d80 2024-05-09T00:51:09.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.843+0000 7f7ac5d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6f1b99d80 0x55d6f1f907b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:09.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.843+0000 7f7ac5d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6f1b99d80 0x55d6f1f907b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:43156/0 (socket says 172.21.15.5:43156) 2024-05-09T00:51:09.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.843+0000 7f7ac5d9b640 1 -- 172.21.15.5:0/2482186983 learned_addr learned my addr 172.21.15.5:0/2482186983 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:51:09.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.843+0000 7f7ace59c640 1 --2- 172.21.15.5:0/2482186983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6f1bcf300 0x55d6f1f88100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:09.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.844+0000 7f7aced9d640 1 --2- 172.21.15.5:0/2482186983 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d6f1bb22f0 0x55d6f1f90cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:09.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.844+0000 7f7ac5d9b640 1 -- 172.21.15.5:0/2482186983 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d6f1bb22f0 msgr2=0x55d6f1f90cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:09.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.844+0000 7f7ac5d9b640 1 --2- 172.21.15.5:0/2482186983 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d6f1bb22f0 0x55d6f1f90cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:09.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.844+0000 7f7ac5d9b640 1 -- 172.21.15.5:0/2482186983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6f1bcf300 msgr2=0x55d6f1f88100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:09.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.844+0000 7f7ac5d9b640 1 --2- 172.21.15.5:0/2482186983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6f1bcf300 0x55d6f1f88100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:09.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.844+0000 7f7ac5d9b640 1 -- 172.21.15.5:0/2482186983 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6f1cbd5f0 con 0x55d6f1b99d80 2024-05-09T00:51:09.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.845+0000 7f7ac5d9b640 1 --2- 172.21.15.5:0/2482186983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6f1b99d80 0x55d6f1f907b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7ab800be60 tx=0x7f7ab80e7610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf0670e303a39bc0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:51:09.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.845+0000 7f7ace59c640 1 --2- 172.21.15.5:0/2482186983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6f1bcf300 0x55d6f1f88100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:51:09.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.845+0000 7f7aced9d640 1 --2- 172.21.15.5:0/2482186983 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d6f1bb22f0 0x55d6f1f90cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:51:09.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.848+0000 7f7ab7fff640 1 -- 172.21.15.5:0/2482186983 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab801e070 con 0x55d6f1b99d80 2024-05-09T00:51:09.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.848+0000 7f7ab7fff640 1 -- 172.21.15.5:0/2482186983 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ab8017360 con 0x55d6f1b99d80 2024-05-09T00:51:09.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.848+0000 7f7ab7fff640 1 -- 172.21.15.5:0/2482186983 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab80b1e40 con 0x55d6f1b99d80 2024-05-09T00:51:09.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.850+0000 7f7adb815740 1 -- 172.21.15.5:0/2482186983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6f1b99d80 msgr2=0x55d6f1f907b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:09.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.850+0000 7f7adb815740 1 --2- 172.21.15.5:0/2482186983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6f1b99d80 0x55d6f1f907b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7ab800be60 tx=0x7f7ab80e7610 comp rx=0 tx=0).stop 2024-05-09T00:51:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.853+0000 7f7adb815740 1 -- 172.21.15.5:0/2482186983 shutdown_connections 2024-05-09T00:51:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.853+0000 7f7adb815740 1 --2- 172.21.15.5:0/2482186983 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d6f1bb22f0 0x55d6f1f90cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.853+0000 7f7adb815740 1 --2- 172.21.15.5:0/2482186983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6f1b99d80 0x55d6f1f907b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.853+0000 7f7adb815740 1 --2- 172.21.15.5:0/2482186983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6f1bcf300 0x55d6f1f88100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.853+0000 7f7adb815740 1 -- 172.21.15.5:0/2482186983 >> 172.21.15.5:0/2482186983 conn(0x55d6f1f89bc0 msgr2=0x55d6f1f879f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:51:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.853+0000 7f7adb815740 1 -- 172.21.15.5:0/2482186983 shutdown_connections 2024-05-09T00:51:09.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.853+0000 7f7adb815740 1 -- 172.21.15.5:0/2482186983 wait complete. 2024-05-09T00:51:09.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.854+0000 7f7adb815740 1 -- start start 2024-05-09T00:51:09.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.854+0000 7f7adb815740 1 Processor -- start 2024-05-09T00:51:09.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.855+0000 7f7adb815740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6f1b99d80 0x55d6f1f8dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:09.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.855+0000 7f7adb815740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d6f1bb22f0 0x55d6f1f8e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:09.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.855+0000 7f7adb815740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6f1bcf300 0x55d6f1ffcd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:09.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.855+0000 7f7adb815740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad01b3150 con 0x55d6f1b99d80 2024-05-09T00:51:09.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.855+0000 7f7adb815740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad01b2a80 con 0x55d6f1bb22f0 2024-05-09T00:51:09.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.855+0000 7f7adb815740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad01afcc0 con 0x55d6f1bcf300 2024-05-09T00:51:09.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.855+0000 7f7adb815740 10 client.0 _ll_register_callbacks cb 0x7f7ab80e74e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:51:09.858 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.856+0000 7f7ace59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6f1b99d80 0x55d6f1f8dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:09.858 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.856+0000 7f7ace59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6f1b99d80 0x55d6f1f8dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34364/0 (socket says 172.21.15.5:34364) 2024-05-09T00:51:09.858 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.856+0000 7f7ace59c640 1 -- 172.21.15.5:0/1608025470 learned_addr learned my addr 172.21.15.5:0/1608025470 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:51:09.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.856+0000 7f7aced9d640 1 --2- 172.21.15.5:0/1608025470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6f1bcf300 0x55d6f1ffcd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:09.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.857+0000 7f7ac5d9b640 1 --2- 172.21.15.5:0/1608025470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d6f1bb22f0 0x55d6f1f8e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:09.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.857+0000 7f7ace59c640 1 -- 172.21.15.5:0/1608025470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6f1bcf300 msgr2=0x55d6f1ffcd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:09.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.857+0000 7f7ace59c640 1 --2- 172.21.15.5:0/1608025470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6f1bcf300 0x55d6f1ffcd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:09.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.857+0000 7f7ace59c640 1 -- 172.21.15.5:0/1608025470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d6f1bb22f0 msgr2=0x55d6f1f8e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:09.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.857+0000 7f7ace59c640 1 --2- 172.21.15.5:0/1608025470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d6f1bb22f0 0x55d6f1f8e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:09.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.857+0000 7f7ace59c640 1 -- 172.21.15.5:0/1608025470 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6f1ea7ed0 con 0x55d6f1b99d80 2024-05-09T00:51:09.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.858+0000 7f7ace59c640 1 --2- 172.21.15.5:0/1608025470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6f1b99d80 0x55d6f1f8dfb0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f7ac000d910 tx=0x7f7ac0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74158e5c96163b8e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:51:09.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.858+0000 7f7ab5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:51:09.861 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.859+0000 7f7ab5ffb640 1 -- 172.21.15.5:0/1608025470 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac004f070 con 0x55d6f1b99d80 2024-05-09T00:51:09.861 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.859+0000 7f7ab5ffb640 1 -- 172.21.15.5:0/1608025470 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f7ac004b630 con 0x55d6f1b99d80 2024-05-09T00:51:09.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.859+0000 7f7adb815740 1 -- 172.21.15.5:0/1608025470 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d6f1eeb240 con 0x55d6f1b99d80 2024-05-09T00:51:09.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.860+0000 7f7ab5ffb640 1 -- 172.21.15.5:0/1608025470 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7ac0059050 con 0x55d6f1b99d80 2024-05-09T00:51:09.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.860+0000 7f7adb815740 10 client.15987 fetch_fsmap learned FSMap version 30 2024-05-09T00:51:09.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.860+0000 7f7adb815740 1 -- 172.21.15.5:0/1608025470 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d6f1ffe290 con 0x55d6f1b99d80 2024-05-09T00:51:09.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.861+0000 7f7ab5ffb640 1 -- 172.21.15.5:0/1608025470 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ac004be50 con 0x55d6f1b99d80 2024-05-09T00:51:09.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.861+0000 7f7ab5ffb640 1 -- 172.21.15.5:0/1608025470 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f7ac0054ce0 con 0x55d6f1b99d80 2024-05-09T00:51:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.863+0000 7f7adb815740 10 client.15987 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T00:51:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.863+0000 7f7adb815740 10 client.15987 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:51:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.863+0000 7f7adb815740 20 client.15987 populate_metadata read hostname 'smithi005' 2024-05-09T00:51:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.863+0000 7f7adb815740 1 --2- 172.21.15.5:0/1608025470 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55d6f1bba4d0 0x55d6f2000c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.863+0000 7f7adb815740 4 client.15987 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:51:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.863+0000 7f7adb815740 1 -- 172.21.15.5:0/1608025470 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d6f1eeb240 con 0x55d6f1bba4d0 2024-05-09T00:51:09.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.866+0000 7f7ac5d9b640 1 --2- 172.21.15.5:0/1608025470 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55d6f1bba4d0 0x55d6f2000c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:09.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.871+0000 7f7ac5d9b640 1 --2- 172.21.15.5:0/1608025470 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55d6f1bba4d0 0x55d6f2000c10 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9f7149830f8c987 server_cookie=d75c39ecf25072e7 in_seq=0 out_seq=0 2024-05-09T00:51:09.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:09.871+0000 7f7ab5ffb640 10 client.15987 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:51:10.817 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T00:51:11.435 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:51:11 smithi157 ceph-mon[34294]: pgmap v516: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1007 MiB/s wr, 2.85k op/s 2024-05-09T00:51:11.517 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:51:11.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:51:11 smithi005 ceph-mon[30365]: pgmap v516: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1007 MiB/s wr, 2.85k op/s 2024-05-09T00:51:11.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:51:11 smithi131 ceph-mon[34547]: pgmap v516: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1007 MiB/s wr, 2.85k op/s 2024-05-09T00:51:13.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:51:13 smithi157 ceph-mon[34294]: pgmap v517: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 789 MiB/s wr, 2.01k op/s 2024-05-09T00:51:13.543 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:51:13 smithi005 ceph-mon[30365]: pgmap v517: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 789 MiB/s wr, 2.01k op/s 2024-05-09T00:51:13.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:51:13 smithi131 ceph-mon[34547]: pgmap v517: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 789 MiB/s wr, 2.01k op/s 2024-05-09T00:51:15.435 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:51:15 smithi157 ceph-mon[34294]: pgmap v518: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 872 MiB/s wr, 2.06k op/s 2024-05-09T00:51:15.537 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:51:15 smithi005 ceph-mon[30365]: pgmap v518: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 872 MiB/s wr, 2.06k op/s 2024-05-09T00:51:15.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:51:15 smithi131 ceph-mon[34547]: pgmap v518: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 872 MiB/s wr, 2.06k op/s 2024-05-09T00:51:16.355 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-09T00:51:16.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.283+0000 7f12e5852700 1 -- 172.21.15.5:0/3361110651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12d8098790 msgr2=0x7f12d8098ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:16.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.283+0000 7f12e5852700 1 --2- 172.21.15.5:0/3361110651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12d8098790 0x7f12d8098ba0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f12d0009990 tx=0x7f12d0009ca0 comp rx=0 tx=0).stop 2024-05-09T00:51:16.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.283+0000 7f12e5852700 1 -- 172.21.15.5:0/3361110651 shutdown_connections 2024-05-09T00:51:16.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.283+0000 7f12e5852700 1 --2- 172.21.15.5:0/3361110651 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12d80990e0 0x7f12d80a5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:16.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.283+0000 7f12e5852700 1 --2- 172.21.15.5:0/3361110651 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12d809e7c0 0x7f12d809ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:16.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.283+0000 7f12e5852700 1 --2- 172.21.15.5:0/3361110651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12d8098790 0x7f12d8098ba0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:16.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.283+0000 7f12e5852700 1 -- 172.21.15.5:0/3361110651 >> 172.21.15.5:0/3361110651 conn(0x7f12d80942b0 msgr2=0x7f12d8096700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:51:16.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.283+0000 7f12e5852700 1 -- 172.21.15.5:0/3361110651 shutdown_connections 2024-05-09T00:51:16.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.283+0000 7f12e5852700 1 -- 172.21.15.5:0/3361110651 wait complete. 2024-05-09T00:51:16.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.285+0000 7f12e5852700 1 Processor -- start 2024-05-09T00:51:16.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.285+0000 7f12e5852700 1 -- start start 2024-05-09T00:51:16.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.286+0000 7f12e5852700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12d8098790 0x7f12d8009360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:16.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.286+0000 7f12e5852700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12d80990e0 0x7f12d80098a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:16.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.286+0000 7f12e5852700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12d809e7c0 0x7f12d8009de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:16.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.287+0000 7f12e5852700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f12d80a8a60 con 0x7f12d8098790 2024-05-09T00:51:16.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.287+0000 7f12e5852700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f12d80a88e0 con 0x7f12d80990e0 2024-05-09T00:51:16.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.287+0000 7f12e5852700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f12d80a8be0 con 0x7f12d809e7c0 2024-05-09T00:51:16.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.287+0000 7f12e4850700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12d8098790 0x7f12d8009360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:16.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.287+0000 7f12e4850700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12d8098790 0x7f12d8009360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48830/0 (socket says 172.21.15.5:48830) 2024-05-09T00:51:16.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.287+0000 7f12e4850700 1 -- 172.21.15.5:0/772217067 learned_addr learned my addr 172.21.15.5:0/772217067 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:51:16.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.287+0000 7f12dffff700 1 --2- 172.21.15.5:0/772217067 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12d80990e0 0x7f12d80098a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:16.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.288+0000 7f12dffff700 1 -- 172.21.15.5:0/772217067 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12d809e7c0 msgr2=0x7f12d8009de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:16.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.288+0000 7f12dffff700 1 --2- 172.21.15.5:0/772217067 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12d809e7c0 0x7f12d8009de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:16.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.288+0000 7f12dffff700 1 -- 172.21.15.5:0/772217067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12d8098790 msgr2=0x7f12d8009360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:16.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.288+0000 7f12dffff700 1 --2- 172.21.15.5:0/772217067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12d8098790 0x7f12d8009360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:16.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.288+0000 7f12dffff700 1 -- 172.21.15.5:0/772217067 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12d813fa70 con 0x7f12d80990e0 2024-05-09T00:51:16.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.289+0000 7f12dffff700 1 --2- 172.21.15.5:0/772217067 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12d80990e0 0x7f12d80098a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f12d00098e0 tx=0x7f12d00047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:51:16.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.290+0000 7f12ddffb700 1 -- 172.21.15.5:0/772217067 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d0026070 con 0x7f12d80990e0 2024-05-09T00:51:16.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.290+0000 7f12e5852700 1 -- 172.21.15.5:0/772217067 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12d813fca0 con 0x7f12d80990e0 2024-05-09T00:51:16.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.290+0000 7f12ddffb700 1 -- 172.21.15.5:0/772217067 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12d0004060 con 0x7f12d80990e0 2024-05-09T00:51:16.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.290+0000 7f12ddffb700 1 -- 172.21.15.5:0/772217067 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d000fd10 con 0x7f12d80990e0 2024-05-09T00:51:16.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.291+0000 7f12e5852700 1 -- 172.21.15.5:0/772217067 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12d8140040 con 0x7f12d80990e0 2024-05-09T00:51:16.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.293+0000 7f12ddffb700 1 -- 172.21.15.5:0/772217067 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f12d0029070 con 0x7f12d80990e0 2024-05-09T00:51:16.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.295+0000 7f12ddffb700 1 --2- 172.21.15.5:0/772217067 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f12c80b2940 0x7f12c80b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:16.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.296+0000 7f12e4850700 1 --2- 172.21.15.5:0/772217067 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f12c80b2940 0x7f12c80b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:16.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.296+0000 7f12ddffb700 1 -- 172.21.15.5:0/772217067 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f12d00e2ec0 con 0x7f12d80990e0 2024-05-09T00:51:16.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.297+0000 7f12e4850700 1 --2- 172.21.15.5:0/772217067 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f12c80b2940 0x7f12c80b4e50 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f12d4009aa0 tx=0x7f12d4009240 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:51:16.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.297+0000 7f12f54cab80 1 Processor -- start 2024-05-09T00:51:16.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.298+0000 7f12f54cab80 1 -- start start 2024-05-09T00:51:16.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.298+0000 7f12f54cab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634df373d00 0x5634df476a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:16.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.298+0000 7f12f54cab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5634df2f94a0 0x5634df476f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:16.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.298+0000 7f12f54cab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5634df480540 0x5634df482960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:16.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.298+0000 7f12f54cab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5634df3990a0 con 0x5634df373d00 2024-05-09T00:51:16.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.298+0000 7f12e4850700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634df373d00 0x5634df476a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:16.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.298+0000 7f12e4850700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634df373d00 0x5634df476a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48836/0 (socket says 172.21.15.5:48836) 2024-05-09T00:51:16.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.298+0000 7f12e4850700 1 -- 172.21.15.5:0/1145943795 learned_addr learned my addr 172.21.15.5:0/1145943795 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:51:16.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.298+0000 7f12f54cab80 1 -- 172.21.15.5:0/1145943795 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5634df33e9a0 con 0x5634df480540 2024-05-09T00:51:16.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.299+0000 7f12f54cab80 1 -- 172.21.15.5:0/1145943795 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5634df395060 con 0x5634df2f94a0 2024-05-09T00:51:16.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.299+0000 7f12e5051700 1 --2- 172.21.15.5:0/1145943795 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5634df480540 0x5634df482960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:16.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.299+0000 7f12dffff700 1 --2- 172.21.15.5:0/1145943795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5634df2f94a0 0x5634df476f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:16.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.299+0000 7f12e4850700 1 -- 172.21.15.5:0/1145943795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5634df2f94a0 msgr2=0x5634df476f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:16.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.299+0000 7f12e4850700 1 --2- 172.21.15.5:0/1145943795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5634df2f94a0 0x5634df476f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:16.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.299+0000 7f12e4850700 1 -- 172.21.15.5:0/1145943795 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5634df480540 msgr2=0x5634df482960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:16.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.299+0000 7f12e4850700 1 --2- 172.21.15.5:0/1145943795 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5634df480540 0x5634df482960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:16.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.299+0000 7f12e4850700 1 -- 172.21.15.5:0/1145943795 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634df32f500 con 0x5634df373d00 2024-05-09T00:51:16.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.299+0000 7f12e5051700 1 --2- 172.21.15.5:0/1145943795 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5634df480540 0x5634df482960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:51:16.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.299+0000 7f12e4850700 1 --2- 172.21.15.5:0/1145943795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634df373d00 0x5634df476a30 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f12d400ead0 tx=0x7f12d4011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fea3eee1fea48aa9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:51:16.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.300+0000 7f12c6ffd700 1 -- 172.21.15.5:0/1145943795 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d401b040 con 0x5634df373d00 2024-05-09T00:51:16.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.300+0000 7f12c6ffd700 1 -- 172.21.15.5:0/1145943795 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12d4024070 con 0x5634df373d00 2024-05-09T00:51:16.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.300+0000 7f12c6ffd700 1 -- 172.21.15.5:0/1145943795 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d4013920 con 0x5634df373d00 2024-05-09T00:51:16.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.301+0000 7f12f54cab80 1 -- 172.21.15.5:0/1145943795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634df373d00 msgr2=0x5634df476a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:16.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.301+0000 7f12f54cab80 1 --2- 172.21.15.5:0/1145943795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634df373d00 0x5634df476a30 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f12d400ead0 tx=0x7f12d4011000 comp rx=0 tx=0).stop 2024-05-09T00:51:16.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.301+0000 7f12f54cab80 1 -- 172.21.15.5:0/1145943795 shutdown_connections 2024-05-09T00:51:16.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.301+0000 7f12f54cab80 1 --2- 172.21.15.5:0/1145943795 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5634df480540 0x5634df482960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:16.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.301+0000 7f12f54cab80 1 --2- 172.21.15.5:0/1145943795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634df373d00 0x5634df476a30 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:16.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.302+0000 7f12f54cab80 1 --2- 172.21.15.5:0/1145943795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5634df2f94a0 0x5634df476f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:16.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.302+0000 7f12f54cab80 1 -- 172.21.15.5:0/1145943795 >> 172.21.15.5:0/1145943795 conn(0x5634df2fcf70 msgr2=0x5634df471760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:51:16.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.302+0000 7f12f54cab80 1 -- 172.21.15.5:0/1145943795 shutdown_connections 2024-05-09T00:51:16.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.302+0000 7f12f54cab80 1 -- 172.21.15.5:0/1145943795 wait complete. 2024-05-09T00:51:16.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.302+0000 7f12f54cab80 1 -- start start 2024-05-09T00:51:16.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.303+0000 7f12f54cab80 1 Processor -- start 2024-05-09T00:51:16.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.303+0000 7f12f54cab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634df2f94a0 0x5634df47d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:16.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.303+0000 7f12f54cab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5634df373d00 0x5634df47d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:16.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.304+0000 7f12f54cab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5634df480540 0x5634df47df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:16.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.304+0000 7f12f54cab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5634df482f60 con 0x5634df2f94a0 2024-05-09T00:51:16.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.304+0000 7f12f54cab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5634df333b20 con 0x5634df373d00 2024-05-09T00:51:16.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.304+0000 7f12f54cab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5634df4830e0 con 0x5634df480540 2024-05-09T00:51:16.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.304+0000 7f12f54cab80 10 client.0 _ll_register_callbacks cb 0x5634df20a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:51:16.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.304+0000 7f12e4850700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634df2f94a0 0x5634df47d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:16.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.304+0000 7f12e4850700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634df2f94a0 0x5634df47d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48848/0 (socket says 172.21.15.5:48848) 2024-05-09T00:51:16.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.304+0000 7f12e4850700 1 -- 172.21.15.5:0/3613718448 learned_addr learned my addr 172.21.15.5:0/3613718448 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:51:16.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.304+0000 7f12dffff700 1 --2- 172.21.15.5:0/3613718448 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5634df373d00 0x5634df47d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:16.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.304+0000 7f12e4850700 1 -- 172.21.15.5:0/3613718448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5634df480540 msgr2=0x5634df47df10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:51:16.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.304+0000 7f12e4850700 1 --2- 172.21.15.5:0/3613718448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5634df480540 0x5634df47df10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:16.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.304+0000 7f12e4850700 1 -- 172.21.15.5:0/3613718448 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5634df373d00 msgr2=0x5634df47d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:16.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.304+0000 7f12e4850700 1 --2- 172.21.15.5:0/3613718448 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5634df373d00 0x5634df47d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:16.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.304+0000 7f12e4850700 1 -- 172.21.15.5:0/3613718448 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634df214490 con 0x5634df2f94a0 2024-05-09T00:51:16.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.305+0000 7f12e4850700 1 --2- 172.21.15.5:0/3613718448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634df2f94a0 0x5634df47d490 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f12d400eaa0 tx=0x7f12d4013a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d10076d9a6f70e3e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:51:16.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.305+0000 7f12bffff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:51:16.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.305+0000 7f12bffff700 1 -- 172.21.15.5:0/3613718448 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d4015680 con 0x5634df2f94a0 2024-05-09T00:51:16.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.305+0000 7f12f54cab80 1 -- 172.21.15.5:0/3613718448 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12d4024070 con 0x5634df2f94a0 2024-05-09T00:51:16.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.305+0000 7f12bffff700 1 -- 172.21.15.5:0/3613718448 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f12d4015820 con 0x5634df2f94a0 2024-05-09T00:51:16.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.305+0000 7f12bffff700 1 -- 172.21.15.5:0/3613718448 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f12d4015a20 con 0x5634df2f94a0 2024-05-09T00:51:16.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.305+0000 7f12f54cab80 10 client.15999 fetch_fsmap learned FSMap version 30 2024-05-09T00:51:16.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.305+0000 7f12f54cab80 1 -- 172.21.15.5:0/3613718448 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5634df48dba0 con 0x5634df2f94a0 2024-05-09T00:51:16.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.306+0000 7f12bffff700 1 -- 172.21.15.5:0/3613718448 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12d4024070 con 0x5634df2f94a0 2024-05-09T00:51:16.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.306+0000 7f12bffff700 1 -- 172.21.15.5:0/3613718448 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f12d401e030 con 0x5634df2f94a0 2024-05-09T00:51:16.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.306+0000 7f12f54cab80 10 client.15999 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T00:51:16.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.306+0000 7f12f54cab80 10 client.15999 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-05-09T00:51:16.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.306+0000 7f12f54cab80 20 client.15999 populate_metadata read hostname 'smithi005' 2024-05-09T00:51:16.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.306+0000 7f12f54cab80 1 --2- 172.21.15.5:0/3613718448 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x5634df4901e0 0x5634df492600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:16.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.307+0000 7f12f54cab80 4 client.15999 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:51:16.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.307+0000 7f12f54cab80 1 -- 172.21.15.5:0/3613718448 --> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634df3a6a80 con 0x5634df4901e0 2024-05-09T00:51:16.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.307+0000 7f12dffff700 1 --2- 172.21.15.5:0/3613718448 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x5634df4901e0 0x5634df492600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:16.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.308+0000 7f12bffff700 10 client.15999 ms_handle_connect on v2:172.21.15.131:6838/2748669631 2024-05-09T00:51:16.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.308+0000 7f12dffff700 1 --2- 172.21.15.5:0/3613718448 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x5634df4901e0 0x5634df492600 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2495ec69cb50fbd2 server_cookie=6e0464498c9852d7 in_seq=0 out_seq=0 2024-05-09T00:51:16.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.309+0000 7f12bffff700 1 -- 172.21.15.5:0/3613718448 <== mds.2 v2:172.21.15.131:6838/2748669631 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5634df3a6a80 con 0x5634df4901e0 2024-05-09T00:51:16.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.309+0000 7f12bffff700 10 client.15999 handle_command_reply: tid=0 2024-05-09T00:51:16.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.309+0000 7f12f54cab80 1 client.15999 shutdown 2024-05-09T00:51:16.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.310+0000 7f12f54cab80 1 -- 172.21.15.5:0/3613718448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634df2f94a0 msgr2=0x5634df47d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:16.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.310+0000 7f12f54cab80 1 --2- 172.21.15.5:0/3613718448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634df2f94a0 0x5634df47d490 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f12d400eaa0 tx=0x7f12d4013a70 comp rx=0 tx=0).stop 2024-05-09T00:51:16.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.310+0000 7f12f54cab80 1 -- 172.21.15.5:0/3613718448 shutdown_connections 2024-05-09T00:51:16.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.310+0000 7f12f54cab80 1 --2- 172.21.15.5:0/3613718448 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x5634df4901e0 0x5634df492600 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:16.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.310+0000 7f12f54cab80 1 --2- 172.21.15.5:0/3613718448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5634df480540 0x5634df47df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:16.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.310+0000 7f12bffff700 0 client.15999 ms_handle_reset on v2:172.21.15.131:6838/2748669631 2024-05-09T00:51:16.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.310+0000 7f12f54cab80 1 --2- 172.21.15.5:0/3613718448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634df2f94a0 0x5634df47d490 secure :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f12d400eaa0 tx=0x7f12d4013a70 comp rx=0 tx=0).stop 2024-05-09T00:51:16.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.310+0000 7f12f54cab80 1 --2- 172.21.15.5:0/3613718448 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5634df373d00 0x5634df47d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:16.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.310+0000 7f12f54cab80 1 -- 172.21.15.5:0/3613718448 >> 172.21.15.5:0/3613718448 conn(0x5634df2fcf70 msgr2=0x5634df471250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:51:16.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.310+0000 7f12f54cab80 1 -- 172.21.15.5:0/3613718448 shutdown_connections 2024-05-09T00:51:16.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.310+0000 7f12f54cab80 1 -- 172.21.15.5:0/3613718448 wait complete. 2024-05-09T00:51:16.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.311+0000 7f12f54cab80 20 client.15999 trim_cache size 0 max 16384 2024-05-09T00:51:16.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.329+0000 7f12f54cab80 1 Processor -- start 2024-05-09T00:51:16.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.329+0000 7f12f54cab80 1 -- start start 2024-05-09T00:51:16.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.329+0000 7f12f54cab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634df373d00 0x5634df478b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:16.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.329+0000 7f12f54cab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5634df480540 0x5634df48c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:16.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.330+0000 7f12f54cab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5634df48c5c0 0x5634df497c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:16.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.330+0000 7f12dffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5634df480540 0x5634df48c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:16.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.330+0000 7f12dffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5634df480540 0x5634df48c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:39466/0 (socket says 172.21.15.5:39466) 2024-05-09T00:51:16.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.330+0000 7f12dffff700 1 -- 172.21.15.5:0/2707584371 learned_addr learned my addr 172.21.15.5:0/2707584371 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:51:16.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.330+0000 7f12f54cab80 1 -- 172.21.15.5:0/2707584371 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f12d8147520 con 0x5634df373d00 2024-05-09T00:51:16.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.330+0000 7f12e4850700 1 --2- 172.21.15.5:0/2707584371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634df373d00 0x5634df478b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:16.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.330+0000 7f12f54cab80 1 -- 172.21.15.5:0/2707584371 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f12d8149090 con 0x5634df480540 2024-05-09T00:51:16.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.330+0000 7f12f54cab80 1 -- 172.21.15.5:0/2707584371 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f12d8147bf0 con 0x5634df48c5c0 2024-05-09T00:51:16.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.330+0000 7f12e5051700 1 --2- 172.21.15.5:0/2707584371 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5634df48c5c0 0x5634df497c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:16.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.331+0000 7f12dffff700 1 -- 172.21.15.5:0/2707584371 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5634df48c5c0 msgr2=0x5634df497c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:16.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.331+0000 7f12dffff700 1 --2- 172.21.15.5:0/2707584371 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5634df48c5c0 0x5634df497c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:16.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.331+0000 7f12dffff700 1 -- 172.21.15.5:0/2707584371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634df373d00 msgr2=0x5634df478b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:16.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.331+0000 7f12dffff700 1 --2- 172.21.15.5:0/2707584371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634df373d00 0x5634df478b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:16.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.331+0000 7f12dffff700 1 -- 172.21.15.5:0/2707584371 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634df43bca0 con 0x5634df480540 2024-05-09T00:51:16.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.331+0000 7f12dffff700 1 --2- 172.21.15.5:0/2707584371 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5634df480540 0x5634df48c080 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f12d0000c00 tx=0x7f12d001a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed2b020f90804c64 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:51:16.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.331+0000 7f12c67fc700 1 -- 172.21.15.5:0/2707584371 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d00920c0 con 0x5634df480540 2024-05-09T00:51:16.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.332+0000 7f12c67fc700 1 -- 172.21.15.5:0/2707584371 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12d00b32b0 con 0x5634df480540 2024-05-09T00:51:16.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.332+0000 7f12c67fc700 1 -- 172.21.15.5:0/2707584371 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d00b2e20 con 0x5634df480540 2024-05-09T00:51:16.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.333+0000 7f12f54cab80 1 -- 172.21.15.5:0/2707584371 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5634df480540 msgr2=0x5634df48c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:16.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.333+0000 7f12f54cab80 1 --2- 172.21.15.5:0/2707584371 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5634df480540 0x5634df48c080 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f12d0000c00 tx=0x7f12d001a040 comp rx=0 tx=0).stop 2024-05-09T00:51:16.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.333+0000 7f12f54cab80 1 -- 172.21.15.5:0/2707584371 shutdown_connections 2024-05-09T00:51:16.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.333+0000 7f12f54cab80 1 --2- 172.21.15.5:0/2707584371 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5634df48c5c0 0x5634df497c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:16.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.333+0000 7f12f54cab80 1 --2- 172.21.15.5:0/2707584371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634df373d00 0x5634df478b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:16.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.333+0000 7f12f54cab80 1 --2- 172.21.15.5:0/2707584371 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5634df480540 0x5634df48c080 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:16.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.333+0000 7f12f54cab80 1 -- 172.21.15.5:0/2707584371 >> 172.21.15.5:0/2707584371 conn(0x5634df2fcf70 msgr2=0x5634df478410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:51:16.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.333+0000 7f12f54cab80 1 -- 172.21.15.5:0/2707584371 shutdown_connections 2024-05-09T00:51:16.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.333+0000 7f12f54cab80 1 -- 172.21.15.5:0/2707584371 wait complete. 2024-05-09T00:51:16.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.334+0000 7f12f54cab80 1 -- start start 2024-05-09T00:51:16.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.334+0000 7f12f54cab80 1 Processor -- start 2024-05-09T00:51:16.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.334+0000 7f12f54cab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5634df373d00 0x5634df488730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:16.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.334+0000 7f12f54cab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634df480540 0x5634df483ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:16.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.334+0000 7f12f54cab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5634df48c5c0 0x5634df484020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:16.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.334+0000 7f12f54cab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f12d8148990 con 0x5634df480540 2024-05-09T00:51:16.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.334+0000 7f12f54cab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f12d81482c0 con 0x5634df373d00 2024-05-09T00:51:16.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.334+0000 7f12f54cab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f12d81454c0 con 0x5634df48c5c0 2024-05-09T00:51:16.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.335+0000 7f12f54cab80 10 client.0 _ll_register_callbacks cb 0x5634df20a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:51:16.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.335+0000 7f12e4850700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5634df373d00 0x5634df488730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:16.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.335+0000 7f12e4850700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5634df373d00 0x5634df488730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:39478/0 (socket says 172.21.15.5:39478) 2024-05-09T00:51:16.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.335+0000 7f12e4850700 1 -- 172.21.15.5:0/3020494390 learned_addr learned my addr 172.21.15.5:0/3020494390 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:51:16.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.335+0000 7f12dffff700 1 --2- 172.21.15.5:0/3020494390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634df480540 0x5634df483ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:16.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.335+0000 7f12e4850700 1 -- 172.21.15.5:0/3020494390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5634df48c5c0 msgr2=0x5634df484020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:16.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.335+0000 7f12e4850700 1 --2- 172.21.15.5:0/3020494390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5634df48c5c0 0x5634df484020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:16.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.335+0000 7f12e4850700 1 -- 172.21.15.5:0/3020494390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634df480540 msgr2=0x5634df483ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:16.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.335+0000 7f12e4850700 1 --2- 172.21.15.5:0/3020494390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634df480540 0x5634df483ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:16.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.335+0000 7f12e4850700 1 -- 172.21.15.5:0/3020494390 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634df484560 con 0x5634df373d00 2024-05-09T00:51:16.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.335+0000 7f12dffff700 1 --2- 172.21.15.5:0/3020494390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634df480540 0x5634df483ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:51:16.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.335+0000 7f12e4850700 1 --2- 172.21.15.5:0/3020494390 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5634df373d00 0x5634df488730 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f12d4006be0 tx=0x7f12d403ce80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=921d81b9d5ec0f30 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:51:16.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.335+0000 7f12bffff700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-09T00:51:16.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.336+0000 7f12bffff700 1 -- 172.21.15.5:0/3020494390 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d4015480 con 0x5634df373d00 2024-05-09T00:51:16.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.336+0000 7f12bffff700 1 -- 172.21.15.5:0/3020494390 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f12d4015620 con 0x5634df373d00 2024-05-09T00:51:16.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.336+0000 7f12f54cab80 1 -- 172.21.15.5:0/3020494390 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12d00b32b0 con 0x5634df373d00 2024-05-09T00:51:16.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.336+0000 7f12bffff700 1 -- 172.21.15.5:0/3020494390 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f12d405c050 con 0x5634df373d00 2024-05-09T00:51:16.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.336+0000 7f12f54cab80 10 client.25556 fetch_fsmap learned FSMap version 30 2024-05-09T00:51:16.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.336+0000 7f12f54cab80 1 -- 172.21.15.5:0/3020494390 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5634df4fa4e0 con 0x5634df373d00 2024-05-09T00:51:16.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.336+0000 7f12bffff700 1 -- 172.21.15.5:0/3020494390 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12d00b32b0 con 0x5634df373d00 2024-05-09T00:51:16.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.336+0000 7f12bffff700 1 -- 172.21.15.5:0/3020494390 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f12d401e040 con 0x5634df373d00 2024-05-09T00:51:16.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.336+0000 7f12f54cab80 10 client.25556 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T00:51:16.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.336+0000 7f12f54cab80 10 client.25556 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-05-09T00:51:16.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.336+0000 7f12f54cab80 20 client.25556 populate_metadata read hostname 'smithi005' 2024-05-09T00:51:16.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.337+0000 7f12f54cab80 1 --2- 172.21.15.5:0/3020494390 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x5634df50bad0 0x5634df50df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:16.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.337+0000 7f12f54cab80 4 client.25556 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:51:16.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.337+0000 7f12f54cab80 1 -- 172.21.15.5:0/3020494390 --> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5634df3a0b20 con 0x5634df50bad0 2024-05-09T00:51:16.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.337+0000 7f12dffff700 1 --2- 172.21.15.5:0/3020494390 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x5634df50bad0 0x5634df50df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:16.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.338+0000 7f12dffff700 1 --2- 172.21.15.5:0/3020494390 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x5634df50bad0 0x5634df50df60 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f5cb9e393ed6f762 server_cookie=5f16534669128e03 in_seq=0 out_seq=0 2024-05-09T00:51:16.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.338+0000 7f12bffff700 10 client.25556 ms_handle_connect on v2:172.21.15.131:6838/2748669631 2024-05-09T00:51:16.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.338+0000 7f12bffff700 1 -- 172.21.15.5:0/3020494390 <== mds.2 v2:172.21.15.131:6838/2748669631 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5634df3a0b20 con 0x5634df50bad0 2024-05-09T00:51:16.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.338+0000 7f12bffff700 10 client.25556 handle_command_reply: tid=0 2024-05-09T00:51:16.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.338+0000 7f12f54cab80 1 client.25556 shutdown 2024-05-09T00:51:16.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.338+0000 7f12f54cab80 1 -- 172.21.15.5:0/3020494390 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5634df373d00 msgr2=0x5634df488730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:16.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.338+0000 7f12f54cab80 1 --2- 172.21.15.5:0/3020494390 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5634df373d00 0x5634df488730 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f12d4006be0 tx=0x7f12d403ce80 comp rx=0 tx=0).stop 2024-05-09T00:51:16.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.338+0000 7f12f54cab80 1 -- 172.21.15.5:0/3020494390 shutdown_connections 2024-05-09T00:51:16.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.338+0000 7f12f54cab80 1 --2- 172.21.15.5:0/3020494390 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x5634df50bad0 0x5634df50df60 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:16.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.338+0000 7f12bffff700 0 client.25556 ms_handle_reset on v2:172.21.15.131:6838/2748669631 2024-05-09T00:51:16.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.338+0000 7f12f54cab80 1 --2- 172.21.15.5:0/3020494390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5634df48c5c0 0x5634df484020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:16.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.338+0000 7f12f54cab80 1 --2- 172.21.15.5:0/3020494390 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5634df373d00 0x5634df488730 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:16.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.338+0000 7f12f54cab80 1 --2- 172.21.15.5:0/3020494390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634df480540 0x5634df483ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:16.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.338+0000 7f12f54cab80 1 -- 172.21.15.5:0/3020494390 >> 172.21.15.5:0/3020494390 conn(0x5634df2fcf70 msgr2=0x5634df47bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:51:16.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.339+0000 7f12f54cab80 1 -- 172.21.15.5:0/3020494390 shutdown_connections 2024-05-09T00:51:16.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.339+0000 7f12f54cab80 1 -- 172.21.15.5:0/3020494390 wait complete. 2024-05-09T00:51:16.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.339+0000 7f12f54cab80 20 client.25556 trim_cache size 0 max 16384 2024-05-09T00:51:16.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.339+0000 7f12e5852700 1 -- 172.21.15.5:0/772217067 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f12c80b2940 msgr2=0x7f12c80b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:16.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.339+0000 7f12e5852700 1 --2- 172.21.15.5:0/772217067 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f12c80b2940 0x7f12c80b4e50 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f12d4009aa0 tx=0x7f12d4009240 comp rx=0 tx=0).stop 2024-05-09T00:51:16.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.339+0000 7f12e5852700 1 -- 172.21.15.5:0/772217067 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12d80990e0 msgr2=0x7f12d80098a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:16.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.339+0000 7f12e5852700 1 --2- 172.21.15.5:0/772217067 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12d80990e0 0x7f12d80098a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f12d00098e0 tx=0x7f12d00047b0 comp rx=0 tx=0).stop 2024-05-09T00:51:16.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.340+0000 7f12e5852700 1 -- 172.21.15.5:0/772217067 shutdown_connections 2024-05-09T00:51:16.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.340+0000 7f12e5852700 1 --2- 172.21.15.5:0/772217067 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f12c80b2940 0x7f12c80b4e50 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:16.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.340+0000 7f12e5852700 1 --2- 172.21.15.5:0/772217067 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12d809e7c0 0x7f12d8009de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:16.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.340+0000 7f12e5852700 1 --2- 172.21.15.5:0/772217067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12d8098790 0x7f12d8009360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:16.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.340+0000 7f12e5852700 1 --2- 172.21.15.5:0/772217067 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12d80990e0 0x7f12d80098a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:16.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.340+0000 7f12e5852700 1 -- 172.21.15.5:0/772217067 >> 172.21.15.5:0/772217067 conn(0x7f12d80942b0 msgr2=0x7f12d80966d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:51:16.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.340+0000 7f12e5852700 1 -- 172.21.15.5:0/772217067 shutdown_connections 2024-05-09T00:51:16.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:16.340+0000 7f12e5852700 1 -- 172.21.15.5:0/772217067 wait complete. 2024-05-09T00:51:17.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:51:17 smithi157 ceph-mon[34294]: pgmap v519: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 922 MiB/s wr, 1.94k op/s 2024-05-09T00:51:17.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:51:17 smithi005 ceph-mon[30365]: pgmap v519: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 922 MiB/s wr, 1.94k op/s 2024-05-09T00:51:17.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:51:17 smithi131 ceph-mon[34547]: pgmap v519: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 922 MiB/s wr, 1.94k op/s 2024-05-09T00:51:18.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:18.450+0000 7f7ab5ffb640 1 -- 172.21.15.5:0/1608025470 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55d6f1eeb240 con 0x55d6f1bba4d0 2024-05-09T00:51:18.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:18.450+0000 7f7ab5ffb640 10 client.15987 handle_command_reply: tid=0 2024-05-09T00:51:18.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:18.450+0000 7f7adb815740 1 client.15987 shutdown 2024-05-09T00:51:18.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:18.453+0000 7f7adb815740 1 -- 172.21.15.5:0/1608025470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6f1b99d80 msgr2=0x55d6f1f8dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:18.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:18.453+0000 7f7adb815740 1 --2- 172.21.15.5:0/1608025470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6f1b99d80 0x55d6f1f8dfb0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f7ac000d910 tx=0x7f7ac0000bc0 comp rx=0 tx=0).stop 2024-05-09T00:51:18.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:18.453+0000 7f7adb815740 1 -- 172.21.15.5:0/1608025470 shutdown_connections 2024-05-09T00:51:18.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:18.453+0000 7f7adb815740 1 --2- 172.21.15.5:0/1608025470 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55d6f1bba4d0 0x55d6f2000c10 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:18.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:18.453+0000 7f7adb815740 1 --2- 172.21.15.5:0/1608025470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6f1bcf300 0x55d6f1ffcd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:18.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:18.453+0000 7f7adb815740 1 --2- 172.21.15.5:0/1608025470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d6f1bb22f0 0x55d6f1f8e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:18.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:18.453+0000 7f7adb815740 1 --2- 172.21.15.5:0/1608025470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6f1b99d80 0x55d6f1f8dfb0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:18.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:18.453+0000 7f7adb815740 1 -- 172.21.15.5:0/1608025470 >> 172.21.15.5:0/1608025470 conn(0x55d6f1f89bc0 msgr2=0x55d6f1f7a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:51:18.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:18.453+0000 7f7ab5ffb640 0 client.15987 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:51:18.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:18.454+0000 7f7adb815740 1 -- 172.21.15.5:0/1608025470 shutdown_connections 2024-05-09T00:51:18.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:18.454+0000 7f7adb815740 1 -- 172.21.15.5:0/1608025470 wait complete. 2024-05-09T00:51:18.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:18.455+0000 7f7adb815740 20 client.15987 trim_cache size 0 max 16384 2024-05-09T00:51:18.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-09T00:51:18.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-09T00:51:18.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-09T00:51:18.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:18.468+0000 7f7ad495e640 1 -- 172.21.15.5:0/2723498005 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f7abc0b27e0 msgr2=0x7f7abc0b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:18.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:18.468+0000 7f7ad495e640 1 --2- 172.21.15.5:0/2723498005 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f7abc0b27e0 0x7f7abc0b4cf0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7ac00048e0 tx=0x7f7ac000d570 comp rx=0 tx=0).stop 2024-05-09T00:51:18.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:18.468+0000 7f7ad495e640 1 -- 172.21.15.5:0/2723498005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ad0103d90 msgr2=0x7f7ad01a1070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:18.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:18.468+0000 7f7ad495e640 1 --2- 172.21.15.5:0/2723498005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ad0103d90 0x7f7ad01a1070 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f7ab800b4a0 tx=0x7f7ab800b960 comp rx=0 tx=0).stop 2024-05-09T00:51:18.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:18.468+0000 7f7ad495e640 1 -- 172.21.15.5:0/2723498005 shutdown_connections 2024-05-09T00:51:18.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:18.468+0000 7f7ad495e640 1 --2- 172.21.15.5:0/2723498005 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f7abc0b27e0 0x7f7abc0b4cf0 secure :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7ac00048e0 tx=0x7f7ac000d570 comp rx=0 tx=0).stop 2024-05-09T00:51:18.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:18.468+0000 7f7ad495e640 1 --2- 172.21.15.5:0/2723498005 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ad0108a90 0x7f7ad01a8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:18.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:18.468+0000 7f7ad495e640 1 --2- 172.21.15.5:0/2723498005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ad0103d90 0x7f7ad01a1070 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:18.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:18.468+0000 7f7ad495e640 1 --2- 172.21.15.5:0/2723498005 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ad0102b50 0x7f7ad01a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:18.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:18.468+0000 7f7ad495e640 1 -- 172.21.15.5:0/2723498005 >> 172.21.15.5:0/2723498005 conn(0x7f7ad00fdd40 msgr2=0x7f7ad0104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:51:18.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:18.470+0000 7f7ad495e640 1 -- 172.21.15.5:0/2723498005 shutdown_connections 2024-05-09T00:51:18.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:18.470+0000 7f7ad495e640 1 -- 172.21.15.5:0/2723498005 wait complete. 2024-05-09T00:51:18.523 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T00:51:19.288 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:51:19.465 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:51:19 smithi005 ceph-mon[30365]: pgmap v520: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 822 MiB/s wr, 1.51k op/s 2024-05-09T00:51:19.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:51:19 smithi131 ceph-mon[34547]: pgmap v520: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 822 MiB/s wr, 1.51k op/s 2024-05-09T00:51:19.685 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:51:19 smithi157 ceph-mon[34294]: pgmap v520: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 822 MiB/s wr, 1.51k op/s 2024-05-09T00:51:21.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:51:21 smithi005 ceph-mon[30365]: pgmap v521: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 976 MiB/s wr, 1.89k op/s 2024-05-09T00:51:21.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:51:21 smithi131 ceph-mon[34547]: pgmap v521: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 976 MiB/s wr, 1.89k op/s 2024-05-09T00:51:21.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:51:21 smithi157 ceph-mon[34294]: pgmap v521: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 976 MiB/s wr, 1.89k op/s 2024-05-09T00:51:23.473 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:51:23 smithi005 ceph-mon[30365]: pgmap v522: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 787 MiB/s wr, 1.38k op/s 2024-05-09T00:51:23.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:51:23 smithi131 ceph-mon[34547]: pgmap v522: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 787 MiB/s wr, 1.38k op/s 2024-05-09T00:51:23.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:51:23 smithi157 ceph-mon[34294]: pgmap v522: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 787 MiB/s wr, 1.38k op/s 2024-05-09T00:51:24.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.939+0000 7fb0b48df700 1 -- 172.21.15.5:0/2002384759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0b010b280 msgr2=0x7fb0b01198b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:24.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.939+0000 7fb0b48df700 1 --2- 172.21.15.5:0/2002384759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0b010b280 0x7fb0b01198b0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fb0a800b0a0 tx=0x7fb0a800b3b0 comp rx=0 tx=0).stop 2024-05-09T00:51:24.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.940+0000 7fb0b48df700 1 -- 172.21.15.5:0/2002384759 shutdown_connections 2024-05-09T00:51:24.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.940+0000 7fb0b48df700 1 --2- 172.21.15.5:0/2002384759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0b010b280 0x7fb0b01198b0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:24.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.940+0000 7fb0b48df700 1 --2- 172.21.15.5:0/2002384759 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0b0111bd0 0x7fb0b0111fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:24.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.940+0000 7fb0b48df700 1 --2- 172.21.15.5:0/2002384759 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb0b010a930 0x7fb0b010ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:24.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.940+0000 7fb0b48df700 1 -- 172.21.15.5:0/2002384759 >> 172.21.15.5:0/2002384759 conn(0x7fb0b006f4d0 msgr2=0x7fb0b006f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:51:24.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.946+0000 7fb0b48df700 1 -- 172.21.15.5:0/2002384759 shutdown_connections 2024-05-09T00:51:24.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.947+0000 7fb0b48df700 1 -- 172.21.15.5:0/2002384759 wait complete. 2024-05-09T00:51:24.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.948+0000 7fb0b48df700 1 Processor -- start 2024-05-09T00:51:24.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.950+0000 7fb0b48df700 1 -- start start 2024-05-09T00:51:24.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.950+0000 7fb0b48df700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb0b010a930 0x7fb0b0116cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:24.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.950+0000 7fb0b48df700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0b010b280 0x7fb0b0117230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:24.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.950+0000 7fb0b48df700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0b0111bd0 0x7fb0b0117770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:24.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.950+0000 7fb0b48df700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b011ef60 con 0x7fb0b0111bd0 2024-05-09T00:51:24.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.950+0000 7fb0b48df700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b011ede0 con 0x7fb0b010a930 2024-05-09T00:51:24.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.950+0000 7fb0b48df700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b011f0e0 con 0x7fb0b010b280 2024-05-09T00:51:24.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.951+0000 7fb0aed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0b0111bd0 0x7fb0b0117770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:24.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.951+0000 7fb0ae59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb0b010a930 0x7fb0b0116cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:24.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.951+0000 7fb0aed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0b0111bd0 0x7fb0b0117770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60282/0 (socket says 172.21.15.5:60282) 2024-05-09T00:51:24.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.951+0000 7fb0aed9d700 1 -- 172.21.15.5:0/1012341649 learned_addr learned my addr 172.21.15.5:0/1012341649 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:51:24.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.954+0000 7fb0add9b700 1 --2- 172.21.15.5:0/1012341649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0b010b280 0x7fb0b0117230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:24.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.955+0000 7fb0ae59c700 1 -- 172.21.15.5:0/1012341649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0b010b280 msgr2=0x7fb0b0117230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:24.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.955+0000 7fb0ae59c700 1 --2- 172.21.15.5:0/1012341649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0b010b280 0x7fb0b0117230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:24.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.955+0000 7fb0ae59c700 1 -- 172.21.15.5:0/1012341649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0b0111bd0 msgr2=0x7fb0b0117770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:24.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.955+0000 7fb0ae59c700 1 --2- 172.21.15.5:0/1012341649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0b0111bd0 0x7fb0b0117770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:24.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.955+0000 7fb0ae59c700 1 -- 172.21.15.5:0/1012341649 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0b01a6560 con 0x7fb0b010a930 2024-05-09T00:51:24.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.957+0000 7fb0ae59c700 1 --2- 172.21.15.5:0/1012341649 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb0b010a930 0x7fb0b0116cf0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb0a400e9c0 tx=0x7fb0a400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:51:24.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.962+0000 7fb09f7fe700 1 -- 172.21.15.5:0/1012341649 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a400cbd0 con 0x7fb0b010a930 2024-05-09T00:51:24.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.962+0000 7fb09f7fe700 1 -- 172.21.15.5:0/1012341649 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0a40044d0 con 0x7fb0b010a930 2024-05-09T00:51:24.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.962+0000 7fb09f7fe700 1 -- 172.21.15.5:0/1012341649 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a4003df0 con 0x7fb0b010a930 2024-05-09T00:51:24.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.962+0000 7fb0b48df700 1 -- 172.21.15.5:0/1012341649 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0b01a6850 con 0x7fb0b010a930 2024-05-09T00:51:24.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.962+0000 7fb0b48df700 1 -- 172.21.15.5:0/1012341649 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0b01a6d40 con 0x7fb0b010a930 2024-05-09T00:51:24.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.970+0000 7fb0c32f5b80 1 Processor -- start 2024-05-09T00:51:24.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.971+0000 7fb0c32f5b80 1 -- start start 2024-05-09T00:51:24.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.973+0000 7fb0c32f5b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649fdcd6160 0x5649fdd6f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:24.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.973+0000 7fb0c32f5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649fdbec4a0 0x5649fdd6fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:24.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.973+0000 7fb0c32f5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649fdd696a0 0x5649fdd69ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:24.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.973+0000 7fb0c32f5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5649fdc8bff0 con 0x5649fdbec4a0 2024-05-09T00:51:24.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.973+0000 7fb0c32f5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5649fdc2a7f0 con 0x5649fdd696a0 2024-05-09T00:51:24.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.973+0000 7fb0c32f5b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5649fdc87fb0 con 0x5649fdcd6160 2024-05-09T00:51:24.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.975+0000 7fb0add9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649fdd696a0 0x5649fdd69ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:24.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.975+0000 7fb0add9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649fdd696a0 0x5649fdd69ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:42922/0 (socket says 172.21.15.5:42922) 2024-05-09T00:51:24.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.975+0000 7fb0aed9d700 1 --2- 172.21.15.5:0/806725175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649fdcd6160 0x5649fdd6f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:24.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.975+0000 7fb0add9b700 1 -- 172.21.15.5:0/806725175 learned_addr learned my addr 172.21.15.5:0/806725175 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:51:24.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.976+0000 7fb09f7fe700 1 -- 172.21.15.5:0/1012341649 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fb0a4004020 con 0x7fb0b010a930 2024-05-09T00:51:24.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.976+0000 7fb0aed9d700 1 -- 172.21.15.5:0/806725175 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649fdd696a0 msgr2=0x5649fdd69ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:24.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.976+0000 7fb0aed9d700 1 --2- 172.21.15.5:0/806725175 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649fdd696a0 0x5649fdd69ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:24.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.976+0000 7fb0aed9d700 1 -- 172.21.15.5:0/806725175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649fdbec4a0 msgr2=0x5649fdd6fa30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:51:24.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.976+0000 7fb0aed9d700 1 --2- 172.21.15.5:0/806725175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649fdbec4a0 0x5649fdd6fa30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:24.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.976+0000 7fb0aed9d700 1 -- 172.21.15.5:0/806725175 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649fdc22370 con 0x5649fdcd6160 2024-05-09T00:51:24.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.977+0000 7fb0aed9d700 1 --2- 172.21.15.5:0/806725175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649fdcd6160 0x5649fdd6f4f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8009fd0 tx=0x7fb0a8009390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=406694eb9bdcd73b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:51:24.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.978+0000 7fb09cff9700 1 -- 172.21.15.5:0/806725175 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a800bcb0 con 0x5649fdcd6160 2024-05-09T00:51:24.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.978+0000 7fb09cff9700 1 -- 172.21.15.5:0/806725175 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0a80075c0 con 0x5649fdcd6160 2024-05-09T00:51:24.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.978+0000 7fb09cff9700 1 -- 172.21.15.5:0/806725175 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a80052a0 con 0x5649fdcd6160 2024-05-09T00:51:24.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.980+0000 7fb0c32f5b80 1 -- 172.21.15.5:0/806725175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649fdcd6160 msgr2=0x5649fdd6f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:24.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.980+0000 7fb0c32f5b80 1 --2- 172.21.15.5:0/806725175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649fdcd6160 0x5649fdd6f4f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8009fd0 tx=0x7fb0a8009390 comp rx=0 tx=0).stop 2024-05-09T00:51:24.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.981+0000 7fb09f7fe700 1 --2- 172.21.15.5:0/1012341649 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fb0980b2a10 0x7fb0980b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:24.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.981+0000 7fb0aed9d700 1 --2- 172.21.15.5:0/1012341649 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fb0980b2a10 0x7fb0980b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:24.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.982+0000 7fb0c32f5b80 1 -- 172.21.15.5:0/806725175 shutdown_connections 2024-05-09T00:51:24.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.982+0000 7fb0c32f5b80 1 --2- 172.21.15.5:0/806725175 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649fdd696a0 0x5649fdd69ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:24.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.982+0000 7fb0c32f5b80 1 --2- 172.21.15.5:0/806725175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649fdcd6160 0x5649fdd6f4f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:24.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.982+0000 7fb0c32f5b80 1 --2- 172.21.15.5:0/806725175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649fdbec4a0 0x5649fdd6fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:24.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.982+0000 7fb0c32f5b80 1 -- 172.21.15.5:0/806725175 >> 172.21.15.5:0/806725175 conn(0x5649fdc4c6a0 msgr2=0x5649fdd64400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:51:24.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.982+0000 7fb09f7fe700 1 -- 172.21.15.5:0/1012341649 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fb0a40e23e0 con 0x7fb0b010a930 2024-05-09T00:51:24.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.983+0000 7fb0c32f5b80 1 -- 172.21.15.5:0/806725175 shutdown_connections 2024-05-09T00:51:24.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.983+0000 7fb0c32f5b80 1 -- 172.21.15.5:0/806725175 wait complete. 2024-05-09T00:51:24.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.983+0000 7fb0c32f5b80 1 -- start start 2024-05-09T00:51:24.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.984+0000 7fb0c32f5b80 1 Processor -- start 2024-05-09T00:51:24.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.985+0000 7fb0c32f5b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649fdcd6160 0x5649fdd73260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:24.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.985+0000 7fb0c32f5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649fdd696a0 0x5649fdd737a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:24.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.985+0000 7fb0c32f5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649fdd73ce0 0x5649fdd7f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:24.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.985+0000 7fb0c32f5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5649fdb07ba0 con 0x5649fdd73ce0 2024-05-09T00:51:24.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.985+0000 7fb0c32f5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5649fdc46150 con 0x5649fdd696a0 2024-05-09T00:51:24.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.985+0000 7fb0c32f5b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5649fdd761d0 con 0x5649fdcd6160 2024-05-09T00:51:24.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.985+0000 7fb0c32f5b80 10 client.0 _ll_register_callbacks cb 0x5649fdafd6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:51:24.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.985+0000 7fb0aed9d700 1 --2- 172.21.15.5:0/1012341649 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fb0980b2a10 0x7fb0980b4f20 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb0a800b3f0 tx=0x7fb0a8028040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:51:24.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.986+0000 7fb0ae59c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649fdcd6160 0x5649fdd73260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:24.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.987+0000 7fb0add9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649fdd696a0 0x5649fdd737a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:24.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.987+0000 7fb0add9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649fdd696a0 0x5649fdd737a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:42936/0 (socket says 172.21.15.5:42936) 2024-05-09T00:51:24.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.987+0000 7fb0add9b700 1 -- 172.21.15.5:0/2673805337 learned_addr learned my addr 172.21.15.5:0/2673805337 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:51:24.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.988+0000 7fb0aed9d700 1 --2- 172.21.15.5:0/2673805337 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649fdd73ce0 0x5649fdd7f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:24.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.988+0000 7fb0ae59c700 1 -- 172.21.15.5:0/2673805337 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649fdd696a0 msgr2=0x5649fdd737a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:24.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.988+0000 7fb0ae59c700 1 --2- 172.21.15.5:0/2673805337 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649fdd696a0 0x5649fdd737a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:24.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.988+0000 7fb0ae59c700 1 -- 172.21.15.5:0/2673805337 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649fdd73ce0 msgr2=0x5649fdd7f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:24.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.988+0000 7fb0ae59c700 1 --2- 172.21.15.5:0/2673805337 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649fdd73ce0 0x5649fdd7f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:24.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.988+0000 7fb0ae59c700 1 -- 172.21.15.5:0/2673805337 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649fdb07490 con 0x5649fdcd6160 2024-05-09T00:51:24.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.988+0000 7fb0add9b700 1 --2- 172.21.15.5:0/2673805337 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649fdd696a0 0x5649fdd737a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:51:24.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.988+0000 7fb0aed9d700 1 --2- 172.21.15.5:0/2673805337 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649fdd73ce0 0x5649fdd7f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:51:24.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.989+0000 7fb0ae59c700 1 --2- 172.21.15.5:0/2673805337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649fdcd6160 0x5649fdd73260 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x5649fdd65ed0 tx=0x7fb0a40a9140 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de218d99612bef36 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:51:24.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.989+0000 7fb095ffb700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-09T00:51:24.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.990+0000 7fb095ffb700 1 -- 172.21.15.5:0/2673805337 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a400ce00 con 0x5649fdcd6160 2024-05-09T00:51:24.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.990+0000 7fb0c32f5b80 1 -- 172.21.15.5:0/2673805337 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0a80075c0 con 0x5649fdcd6160 2024-05-09T00:51:24.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.990+0000 7fb095ffb700 1 -- 172.21.15.5:0/2673805337 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fb0a40a5fc0 con 0x5649fdcd6160 2024-05-09T00:51:24.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.990+0000 7fb095ffb700 1 -- 172.21.15.5:0/2673805337 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb0a40a94e0 con 0x5649fdcd6160 2024-05-09T00:51:24.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.992+0000 7fb0c32f5b80 10 client.25591 fetch_fsmap learned FSMap version 30 2024-05-09T00:51:24.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.992+0000 7fb0c32f5b80 1 -- 172.21.15.5:0/2673805337 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5649fdd810a0 con 0x5649fdcd6160 2024-05-09T00:51:24.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.993+0000 7fb095ffb700 1 -- 172.21.15.5:0/2673805337 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0a80075c0 con 0x5649fdcd6160 2024-05-09T00:51:24.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.993+0000 7fb095ffb700 1 -- 172.21.15.5:0/2673805337 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7fb0a40a9c30 con 0x5649fdcd6160 2024-05-09T00:51:24.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.993+0000 7fb0c32f5b80 10 client.25591 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T00:51:24.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.993+0000 7fb0c32f5b80 10 client.25591 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-05-09T00:51:24.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.993+0000 7fb0c32f5b80 20 client.25591 populate_metadata read hostname 'smithi005' 2024-05-09T00:51:24.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.993+0000 7fb0c32f5b80 1 --2- 172.21.15.5:0/2673805337 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x5649fdd83aa0 0x5649fdd85f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:24.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.993+0000 7fb0c32f5b80 4 client.25591 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:51:24.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.993+0000 7fb0c32f5b80 1 -- 172.21.15.5:0/2673805337 --> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649fdc999d0 con 0x5649fdd83aa0 2024-05-09T00:51:24.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.995+0000 7fb0add9b700 1 --2- 172.21.15.5:0/2673805337 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x5649fdd83aa0 0x5649fdd85f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:24.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.997+0000 7fb0add9b700 1 --2- 172.21.15.5:0/2673805337 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x5649fdd83aa0 0x5649fdd85f00 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e4fd0591ab00cd94 server_cookie=219b7afc46e1269d in_seq=0 out_seq=0 2024-05-09T00:51:24.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.997+0000 7fb095ffb700 10 client.25591 ms_handle_connect on v2:172.21.15.157:6836/744863009 2024-05-09T00:51:25.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.999+0000 7fb095ffb700 1 -- 172.21.15.5:0/2673805337 <== mds.1 v2:172.21.15.157:6836/744863009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5649fdc999d0 con 0x5649fdd83aa0 2024-05-09T00:51:25.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.999+0000 7fb095ffb700 10 client.25591 handle_command_reply: tid=0 2024-05-09T00:51:25.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:24.999+0000 7fb0c32f5b80 1 client.25591 shutdown 2024-05-09T00:51:25.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.002+0000 7fb0c32f5b80 1 -- 172.21.15.5:0/2673805337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649fdcd6160 msgr2=0x5649fdd73260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:25.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.002+0000 7fb0c32f5b80 1 --2- 172.21.15.5:0/2673805337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649fdcd6160 0x5649fdd73260 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x5649fdd65ed0 tx=0x7fb0a40a9140 comp rx=0 tx=0).stop 2024-05-09T00:51:25.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.003+0000 7fb0c32f5b80 1 -- 172.21.15.5:0/2673805337 shutdown_connections 2024-05-09T00:51:25.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.003+0000 7fb0c32f5b80 1 --2- 172.21.15.5:0/2673805337 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x5649fdd83aa0 0x5649fdd85f00 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:25.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.003+0000 7fb0c32f5b80 1 --2- 172.21.15.5:0/2673805337 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649fdd73ce0 0x5649fdd7f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:25.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.003+0000 7fb0c32f5b80 1 --2- 172.21.15.5:0/2673805337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649fdcd6160 0x5649fdd73260 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:25.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.003+0000 7fb0c32f5b80 1 --2- 172.21.15.5:0/2673805337 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649fdd696a0 0x5649fdd737a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:25.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.003+0000 7fb0c32f5b80 1 -- 172.21.15.5:0/2673805337 >> 172.21.15.5:0/2673805337 conn(0x5649fdc4c6a0 msgr2=0x5649fdd641c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:51:25.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.003+0000 7fb095ffb700 0 client.25591 ms_handle_reset on v2:172.21.15.157:6836/744863009 2024-05-09T00:51:25.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.004+0000 7fb0c32f5b80 1 -- 172.21.15.5:0/2673805337 shutdown_connections 2024-05-09T00:51:25.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.004+0000 7fb0c32f5b80 1 -- 172.21.15.5:0/2673805337 wait complete. 2024-05-09T00:51:25.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.004+0000 7fb0c32f5b80 20 client.25591 trim_cache size 0 max 16384 2024-05-09T00:51:25.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.067+0000 7fb0c32f5b80 1 Processor -- start 2024-05-09T00:51:25.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.072+0000 7fb0c32f5b80 1 -- start start 2024-05-09T00:51:25.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.073+0000 7fb0c32f5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649fdcd6160 0x5649fdd8b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:25.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.074+0000 7fb0c32f5b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649fdd696a0 0x5649fdd8bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:25.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.074+0000 7fb0c32f5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649fdd73ce0 0x5649fdd6e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:25.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.075+0000 7fb0c32f5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b01c6310 con 0x5649fdcd6160 2024-05-09T00:51:25.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.075+0000 7fb0c32f5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b01c7e80 con 0x5649fdd73ce0 2024-05-09T00:51:25.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.075+0000 7fb0add9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649fdd696a0 0x5649fdd8bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:25.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.075+0000 7fb0add9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649fdd696a0 0x5649fdd8bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:44228/0 (socket says 172.21.15.5:44228) 2024-05-09T00:51:25.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.075+0000 7fb0add9b700 1 -- 172.21.15.5:0/1276960739 learned_addr learned my addr 172.21.15.5:0/1276960739 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:51:25.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.075+0000 7fb0c32f5b80 1 -- 172.21.15.5:0/1276960739 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b01c69e0 con 0x5649fdd696a0 2024-05-09T00:51:25.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.077+0000 7fb0aed9d700 1 --2- 172.21.15.5:0/1276960739 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649fdd73ce0 0x5649fdd6e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:25.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.079+0000 7fb0add9b700 1 -- 172.21.15.5:0/1276960739 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649fdd73ce0 msgr2=0x5649fdd6e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:25.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.079+0000 7fb0add9b700 1 --2- 172.21.15.5:0/1276960739 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649fdd73ce0 0x5649fdd6e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:25.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.079+0000 7fb0add9b700 1 -- 172.21.15.5:0/1276960739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649fdcd6160 msgr2=0x5649fdd8b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:25.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.079+0000 7fb0add9b700 1 --2- 172.21.15.5:0/1276960739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649fdcd6160 0x5649fdd8b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:25.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.079+0000 7fb0add9b700 1 -- 172.21.15.5:0/1276960739 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649fdd6a640 con 0x5649fdd696a0 2024-05-09T00:51:25.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.084+0000 7fb0add9b700 1 --2- 172.21.15.5:0/1276960739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649fdd696a0 0x5649fdd8bc00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb0a0009350 tx=0x7fb0a0014750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=741c906f2e2c4c6e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:51:25.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.087+0000 7fb097fff700 1 -- 172.21.15.5:0/1276960739 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a0014c80 con 0x5649fdd696a0 2024-05-09T00:51:25.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.087+0000 7fb097fff700 1 -- 172.21.15.5:0/1276960739 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0a0014e20 con 0x5649fdd696a0 2024-05-09T00:51:25.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.087+0000 7fb097fff700 1 -- 172.21.15.5:0/1276960739 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a0012450 con 0x5649fdd696a0 2024-05-09T00:51:25.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.089+0000 7fb0c32f5b80 1 -- 172.21.15.5:0/1276960739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649fdd696a0 msgr2=0x5649fdd8bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:25.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.089+0000 7fb0c32f5b80 1 --2- 172.21.15.5:0/1276960739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649fdd696a0 0x5649fdd8bc00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb0a0009350 tx=0x7fb0a0014750 comp rx=0 tx=0).stop 2024-05-09T00:51:25.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.089+0000 7fb0c32f5b80 1 -- 172.21.15.5:0/1276960739 shutdown_connections 2024-05-09T00:51:25.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.089+0000 7fb0c32f5b80 1 --2- 172.21.15.5:0/1276960739 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649fdd73ce0 0x5649fdd6e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:25.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.089+0000 7fb0c32f5b80 1 --2- 172.21.15.5:0/1276960739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649fdcd6160 0x5649fdd8b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:25.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.089+0000 7fb0c32f5b80 1 --2- 172.21.15.5:0/1276960739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649fdd696a0 0x5649fdd8bc00 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:25.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.089+0000 7fb0c32f5b80 1 -- 172.21.15.5:0/1276960739 >> 172.21.15.5:0/1276960739 conn(0x5649fdc4c6a0 msgr2=0x5649fdd85ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:51:25.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.093+0000 7fb0c32f5b80 1 -- 172.21.15.5:0/1276960739 shutdown_connections 2024-05-09T00:51:25.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.097+0000 7fb0c32f5b80 1 -- 172.21.15.5:0/1276960739 wait complete. 2024-05-09T00:51:25.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.098+0000 7fb0c32f5b80 1 -- start start 2024-05-09T00:51:25.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.098+0000 7fb0c32f5b80 1 Processor -- start 2024-05-09T00:51:25.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.099+0000 7fb0c32f5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649fdcd6160 0x5649fdd78120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:25.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.099+0000 7fb0c32f5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649fdd696a0 0x5649fdd78660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:25.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.099+0000 7fb0c32f5b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649fdd73ce0 0x5649fdd78ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:25.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.099+0000 7fb0c32f5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b01c7780 con 0x5649fdd696a0 2024-05-09T00:51:25.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.100+0000 7fb0c32f5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b01c70b0 con 0x5649fdcd6160 2024-05-09T00:51:25.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.100+0000 7fb0c32f5b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b01ac250 con 0x5649fdd73ce0 2024-05-09T00:51:25.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.100+0000 7fb0c32f5b80 10 client.0 _ll_register_callbacks cb 0x5649fdafd6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:51:25.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.100+0000 7fb0add9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649fdd696a0 0x5649fdd78660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:25.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.100+0000 7fb0add9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649fdd696a0 0x5649fdd78660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60322/0 (socket says 172.21.15.5:60322) 2024-05-09T00:51:25.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.101+0000 7fb0add9b700 1 -- 172.21.15.5:0/2737736644 learned_addr learned my addr 172.21.15.5:0/2737736644 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:51:25.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.102+0000 7fb0add9b700 1 -- 172.21.15.5:0/2737736644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649fdd73ce0 msgr2=0x5649fdd78ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:25.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.102+0000 7fb0add9b700 1 --2- 172.21.15.5:0/2737736644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649fdd73ce0 0x5649fdd78ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:25.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.102+0000 7fb0add9b700 1 -- 172.21.15.5:0/2737736644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649fdcd6160 msgr2=0x5649fdd78120 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:51:25.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.102+0000 7fb0add9b700 1 --2- 172.21.15.5:0/2737736644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649fdcd6160 0x5649fdd78120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:25.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.102+0000 7fb0add9b700 1 -- 172.21.15.5:0/2737736644 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649fdd5df60 con 0x5649fdd696a0 2024-05-09T00:51:25.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.103+0000 7fb0add9b700 1 --2- 172.21.15.5:0/2737736644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649fdd696a0 0x5649fdd78660 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fb0a0014d30 tx=0x7fb0a00141d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef66f70a6431903d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:51:25.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.103+0000 7fb095ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:51:25.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.103+0000 7fb095ffb700 1 -- 172.21.15.5:0/2737736644 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a00047c0 con 0x5649fdd696a0 2024-05-09T00:51:25.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.104+0000 7fb095ffb700 1 -- 172.21.15.5:0/2737736644 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fb0a0004960 con 0x5649fdd696a0 2024-05-09T00:51:25.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.109+0000 7fb0c32f5b80 1 -- 172.21.15.5:0/2737736644 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0a0014e20 con 0x5649fdd696a0 2024-05-09T00:51:25.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.123+0000 7fb095ffb700 1 -- 172.21.15.5:0/2737736644 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb0a0035050 con 0x5649fdd696a0 2024-05-09T00:51:25.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.127+0000 7fb0c32f5b80 10 client.16014 fetch_fsmap learned FSMap version 30 2024-05-09T00:51:25.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.127+0000 7fb0c32f5b80 1 -- 172.21.15.5:0/2737736644 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5649fdd65cc0 con 0x5649fdd696a0 2024-05-09T00:51:25.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.131+0000 7fb095ffb700 1 -- 172.21.15.5:0/2737736644 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0a002b070 con 0x5649fdd696a0 2024-05-09T00:51:25.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.133+0000 7fb095ffb700 1 -- 172.21.15.5:0/2737736644 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7fb0a000f040 con 0x5649fdd696a0 2024-05-09T00:51:25.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.135+0000 7fb0c32f5b80 10 client.16014 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T00:51:25.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.135+0000 7fb0c32f5b80 10 client.16014 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-05-09T00:51:25.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.135+0000 7fb0c32f5b80 20 client.16014 populate_metadata read hostname 'smithi005' 2024-05-09T00:51:25.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.135+0000 7fb0c32f5b80 1 --2- 172.21.15.5:0/2737736644 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x5649fdded870 0x5649fddefc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:25.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.135+0000 7fb0c32f5b80 4 client.16014 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:51:25.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.135+0000 7fb0c32f5b80 1 -- 172.21.15.5:0/2737736644 --> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fb0a0014e20 con 0x5649fdded870 2024-05-09T00:51:25.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.140+0000 7fb0ae59c700 1 --2- 172.21.15.5:0/2737736644 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x5649fdded870 0x5649fddefc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:25.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.156+0000 7fb0ae59c700 1 --2- 172.21.15.5:0/2737736644 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x5649fdded870 0x5649fddefc90 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a9bcf37bc1394f7f server_cookie=17ad125de86dc31c in_seq=0 out_seq=0 2024-05-09T00:51:25.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.156+0000 7fb095ffb700 10 client.16014 ms_handle_connect on v2:172.21.15.157:6836/744863009 2024-05-09T00:51:25.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.160+0000 7fb095ffb700 1 -- 172.21.15.5:0/2737736644 <== mds.1 v2:172.21.15.157:6836/744863009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fb0a0014e20 con 0x5649fdded870 2024-05-09T00:51:25.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.160+0000 7fb095ffb700 10 client.16014 handle_command_reply: tid=0 2024-05-09T00:51:25.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.164+0000 7fb0c32f5b80 1 client.16014 shutdown 2024-05-09T00:51:25.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.164+0000 7fb0c32f5b80 1 -- 172.21.15.5:0/2737736644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649fdd696a0 msgr2=0x5649fdd78660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:25.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.164+0000 7fb0c32f5b80 1 --2- 172.21.15.5:0/2737736644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649fdd696a0 0x5649fdd78660 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fb0a0014d30 tx=0x7fb0a00141d0 comp rx=0 tx=0).stop 2024-05-09T00:51:25.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.165+0000 7fb0c32f5b80 1 -- 172.21.15.5:0/2737736644 shutdown_connections 2024-05-09T00:51:25.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.165+0000 7fb0c32f5b80 1 --2- 172.21.15.5:0/2737736644 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x5649fdded870 0x5649fddefc90 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:25.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.165+0000 7fb0c32f5b80 1 --2- 172.21.15.5:0/2737736644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5649fdd73ce0 0x5649fdd78ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:25.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.165+0000 7fb0c32f5b80 1 --2- 172.21.15.5:0/2737736644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5649fdcd6160 0x5649fdd78120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:25.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.165+0000 7fb0c32f5b80 1 --2- 172.21.15.5:0/2737736644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649fdd696a0 0x5649fdd78660 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:25.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.165+0000 7fb0c32f5b80 1 -- 172.21.15.5:0/2737736644 >> 172.21.15.5:0/2737736644 conn(0x5649fdc4c6a0 msgr2=0x5649fdd6e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:51:25.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.165+0000 7fb095ffb700 0 client.16014 ms_handle_reset on v2:172.21.15.157:6836/744863009 2024-05-09T00:51:25.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.169+0000 7fb0c32f5b80 1 -- 172.21.15.5:0/2737736644 shutdown_connections 2024-05-09T00:51:25.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.169+0000 7fb0c32f5b80 1 -- 172.21.15.5:0/2737736644 wait complete. 2024-05-09T00:51:25.172 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-09T00:51:25.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.170+0000 7fb0c32f5b80 20 client.16014 trim_cache size 0 max 16384 2024-05-09T00:51:25.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.172+0000 7fb0b48df700 1 -- 172.21.15.5:0/1012341649 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fb0980b2a10 msgr2=0x7fb0980b4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:25.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.172+0000 7fb0b48df700 1 --2- 172.21.15.5:0/1012341649 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fb0980b2a10 0x7fb0980b4f20 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb0a800b3f0 tx=0x7fb0a8028040 comp rx=0 tx=0).stop 2024-05-09T00:51:25.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.172+0000 7fb0b48df700 1 -- 172.21.15.5:0/1012341649 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb0b010a930 msgr2=0x7fb0b0116cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:25.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.172+0000 7fb0b48df700 1 --2- 172.21.15.5:0/1012341649 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb0b010a930 0x7fb0b0116cf0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb0a400e9c0 tx=0x7fb0a400ecd0 comp rx=0 tx=0).stop 2024-05-09T00:51:25.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.177+0000 7fb0b48df700 1 -- 172.21.15.5:0/1012341649 shutdown_connections 2024-05-09T00:51:25.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.177+0000 7fb0b48df700 1 --2- 172.21.15.5:0/1012341649 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fb0980b2a10 0x7fb0980b4f20 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:25.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.177+0000 7fb0b48df700 1 --2- 172.21.15.5:0/1012341649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0b0111bd0 0x7fb0b0117770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:25.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.177+0000 7fb0b48df700 1 --2- 172.21.15.5:0/1012341649 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb0b010a930 0x7fb0b0116cf0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:25.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.177+0000 7fb0b48df700 1 --2- 172.21.15.5:0/1012341649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0b010b280 0x7fb0b0117230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:25.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.177+0000 7fb0b48df700 1 -- 172.21.15.5:0/1012341649 >> 172.21.15.5:0/1012341649 conn(0x7fb0b006f4d0 msgr2=0x7fb0b011b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:51:25.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.179+0000 7fb0b48df700 1 -- 172.21.15.5:0/1012341649 shutdown_connections 2024-05-09T00:51:25.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:25.179+0000 7fb0b48df700 1 -- 172.21.15.5:0/1012341649 wait complete. 2024-05-09T00:51:25.439 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:51:25 smithi005 ceph-mon[30365]: pgmap v523: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 869 MiB/s wr, 1.49k op/s 2024-05-09T00:51:25.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:51:25 smithi131 ceph-mon[34547]: pgmap v523: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 869 MiB/s wr, 1.49k op/s 2024-05-09T00:51:25.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:51:25 smithi157 ceph-mon[34294]: pgmap v523: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 869 MiB/s wr, 1.49k op/s 2024-05-09T00:51:27.061 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T00:51:27.185 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:51:26 smithi157 ceph-mon[34294]: pgmap v524: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 935 MiB/s wr, 1.56k op/s 2024-05-09T00:51:27.207 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:51:26 smithi005 ceph-mon[30365]: pgmap v524: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 935 MiB/s wr, 1.56k op/s 2024-05-09T00:51:27.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:51:26 smithi131 ceph-mon[34547]: pgmap v524: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 935 MiB/s wr, 1.56k op/s 2024-05-09T00:51:28.066 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:51:29.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:51:29 smithi157 ceph-mon[34294]: pgmap v525: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 837 MiB/s wr, 1.49k op/s 2024-05-09T00:51:29.545 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:51:29 smithi005 ceph-mon[30365]: pgmap v525: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 837 MiB/s wr, 1.49k op/s 2024-05-09T00:51:29.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:51:29 smithi131 ceph-mon[34547]: pgmap v525: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 837 MiB/s wr, 1.49k op/s 2024-05-09T00:51:31.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:51:31 smithi157 ceph-mon[34294]: pgmap v526: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 980 MiB/s wr, 1.80k op/s 2024-05-09T00:51:31.543 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:51:31 smithi005 ceph-mon[30365]: pgmap v526: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 980 MiB/s wr, 1.80k op/s 2024-05-09T00:51:31.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:51:31 smithi131 ceph-mon[34547]: pgmap v526: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 980 MiB/s wr, 1.80k op/s 2024-05-09T00:51:31.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.711+0000 7feb0ddc7640 1 -- 172.21.15.5:0/3370906336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb080799d0 msgr2=0x7feb08079de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:31.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.711+0000 7feb0ddc7640 1 --2- 172.21.15.5:0/3370906336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb080799d0 0x7feb08079de0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7feb00009ca0 tx=0x7feb0002a8c0 comp rx=0 tx=0).stop 2024-05-09T00:51:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.711+0000 7feb0ddc7640 1 -- 172.21.15.5:0/3370906336 shutdown_connections 2024-05-09T00:51:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.711+0000 7feb0ddc7640 1 --2- 172.21.15.5:0/3370906336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb080799d0 0x7feb08079de0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.711+0000 7feb0ddc7640 1 --2- 172.21.15.5:0/3370906336 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb08078cd0 0x7feb080790e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.711+0000 7feb0ddc7640 1 --2- 172.21.15.5:0/3370906336 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb08077a40 0x7feb08078790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.711+0000 7feb0ddc7640 1 -- 172.21.15.5:0/3370906336 >> 172.21.15.5:0/3370906336 conn(0x7feb08101060 msgr2=0x7feb08103530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:51:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.712+0000 7feb0ddc7640 1 -- 172.21.15.5:0/3370906336 shutdown_connections 2024-05-09T00:51:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.712+0000 7feb0ddc7640 1 -- 172.21.15.5:0/3370906336 wait complete. 2024-05-09T00:51:31.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.713+0000 7feb0ddc7640 1 Processor -- start 2024-05-09T00:51:31.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.713+0000 7feb0ddc7640 1 -- start start 2024-05-09T00:51:31.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.713+0000 7feb0ddc7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb08077a40 0x7feb081a0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:31.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.713+0000 7feb0ddc7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb08078cd0 0x7feb081a0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:31.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.714+0000 7feb0ddc7640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb080799d0 0x7feb081a7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:31.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.714+0000 7feb0ddc7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7feb08115e70 con 0x7feb08077a40 2024-05-09T00:51:31.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.714+0000 7feb077fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb08077a40 0x7feb081a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:31.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.714+0000 7feb0ddc7640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7feb08115cf0 con 0x7feb080799d0 2024-05-09T00:51:31.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.714+0000 7feb0ddc7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7feb08115ff0 con 0x7feb08078cd0 2024-05-09T00:51:31.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.714+0000 7feb077fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb08077a40 0x7feb081a0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60340/0 (socket says 172.21.15.5:60340) 2024-05-09T00:51:31.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.714+0000 7feb077fe640 1 -- 172.21.15.5:0/1125462730 learned_addr learned my addr 172.21.15.5:0/1125462730 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:51:31.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.714+0000 7feb077fe640 1 -- 172.21.15.5:0/1125462730 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb08078cd0 msgr2=0x7feb081a0ec0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T00:51:31.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.714+0000 7feb077fe640 1 --2- 172.21.15.5:0/1125462730 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb08078cd0 0x7feb081a0ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:31.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.714+0000 7feb077fe640 1 -- 172.21.15.5:0/1125462730 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb080799d0 msgr2=0x7feb081a7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:31.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.714+0000 7feb077fe640 1 --2- 172.21.15.5:0/1125462730 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb080799d0 0x7feb081a7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:31.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.714+0000 7feb077fe640 1 -- 172.21.15.5:0/1125462730 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb081a84c0 con 0x7feb08077a40 2024-05-09T00:51:31.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.715+0000 7feb077fe640 1 --2- 172.21.15.5:0/1125462730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb08077a40 0x7feb081a0980 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7feaf800d740 tx=0x7feaf800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:51:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.715+0000 7feb04ff9640 1 -- 172.21.15.5:0/1125462730 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf8017070 con 0x7feb08077a40 2024-05-09T00:51:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.716+0000 7feb04ff9640 1 -- 172.21.15.5:0/1125462730 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feaf800de50 con 0x7feb08077a40 2024-05-09T00:51:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.716+0000 7feb04ff9640 1 -- 172.21.15.5:0/1125462730 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf8004ce0 con 0x7feb08077a40 2024-05-09T00:51:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.716+0000 7feb0ddc7640 1 -- 172.21.15.5:0/1125462730 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb081a87b0 con 0x7feb08077a40 2024-05-09T00:51:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.716+0000 7feb0ddc7640 1 -- 172.21.15.5:0/1125462730 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb081a8d30 con 0x7feb08077a40 2024-05-09T00:51:31.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.723+0000 7feb04ff9640 1 -- 172.21.15.5:0/1125462730 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7feaf8004e80 con 0x7feb08077a40 2024-05-09T00:51:31.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.724+0000 7feb14c78740 1 Processor -- start 2024-05-09T00:51:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.724+0000 7feb04ff9640 1 --2- 172.21.15.5:0/1125462730 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7feaf40b28e0 0x7feaf40b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.724+0000 7feb14c78740 1 -- start start 2024-05-09T00:51:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.725+0000 7feb04ff9640 1 -- 172.21.15.5:0/1125462730 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7feaf80e1c70 con 0x7feb08077a40 2024-05-09T00:51:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.725+0000 7feb07fff640 1 --2- 172.21.15.5:0/1125462730 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7feaf40b28e0 0x7feaf40b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.725+0000 7feb14c78740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f236d941b0 0x55f237138500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.725+0000 7feb14c78740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f236d6b2f0 0x55f237138a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.725+0000 7feb14c78740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f236d734d0 0x55f23713f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.725+0000 7feb14c78740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f23709c7c0 con 0x55f236d941b0 2024-05-09T00:51:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.725+0000 7feb14c78740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55f2370380e0 con 0x55f236d6b2f0 2024-05-09T00:51:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.725+0000 7feb14c78740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55f237098780 con 0x55f236d734d0 2024-05-09T00:51:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.726+0000 7feb077fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f236d941b0 0x55f237138500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.726+0000 7feb077fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f236d941b0 0x55f237138500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60350/0 (socket says 172.21.15.5:60350) 2024-05-09T00:51:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.726+0000 7feb077fe640 1 -- 172.21.15.5:0/2744605092 learned_addr learned my addr 172.21.15.5:0/2744605092 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:51:31.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.726+0000 7feb077fe640 1 -- 172.21.15.5:0/2744605092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f236d734d0 msgr2=0x55f23713f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:31.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.726+0000 7feb077fe640 1 --2- 172.21.15.5:0/2744605092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f236d734d0 0x55f23713f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:31.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.726+0000 7feb077fe640 1 -- 172.21.15.5:0/2744605092 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f236d6b2f0 msgr2=0x55f237138a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:31.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.726+0000 7feb077fe640 1 --2- 172.21.15.5:0/2744605092 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f236d6b2f0 0x55f237138a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:31.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.726+0000 7feb077fe640 1 -- 172.21.15.5:0/2744605092 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2370617c0 con 0x55f236d941b0 2024-05-09T00:51:31.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.726+0000 7feb07fff640 1 --2- 172.21.15.5:0/1125462730 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7feaf40b28e0 0x7feaf40b4df0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7feb0002aeb0 tx=0x7feb00009360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:51:31.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.726+0000 7feb077fe640 1 --2- 172.21.15.5:0/2744605092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f236d941b0 0x55f237138500 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7feaf80a9190 tx=0x7feaf80a8a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4db0475ba9b2759 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:51:31.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.727+0000 7feafdffb640 1 -- 172.21.15.5:0/2744605092 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf8020040 con 0x55f236d941b0 2024-05-09T00:51:31.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.727+0000 7feafdffb640 1 -- 172.21.15.5:0/2744605092 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feaf8009c30 con 0x55f236d941b0 2024-05-09T00:51:31.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.727+0000 7feafdffb640 1 -- 172.21.15.5:0/2744605092 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf8020040 con 0x55f236d941b0 2024-05-09T00:51:31.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.728+0000 7feb14c78740 1 -- 172.21.15.5:0/2744605092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f236d941b0 msgr2=0x55f237138500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:31.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.728+0000 7feb14c78740 1 --2- 172.21.15.5:0/2744605092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f236d941b0 0x55f237138500 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7feaf80a9190 tx=0x7feaf80a8a80 comp rx=0 tx=0).stop 2024-05-09T00:51:31.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.729+0000 7feb14c78740 1 -- 172.21.15.5:0/2744605092 shutdown_connections 2024-05-09T00:51:31.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.729+0000 7feb14c78740 1 --2- 172.21.15.5:0/2744605092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f236d734d0 0x55f23713f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:31.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.729+0000 7feb14c78740 1 --2- 172.21.15.5:0/2744605092 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f236d6b2f0 0x55f237138a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:31.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.729+0000 7feb14c78740 1 --2- 172.21.15.5:0/2744605092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f236d941b0 0x55f237138500 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:31.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.729+0000 7feb14c78740 1 -- 172.21.15.5:0/2744605092 >> 172.21.15.5:0/2744605092 conn(0x55f236d88300 msgr2=0x55f237133e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:51:31.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.729+0000 7feb14c78740 1 -- 172.21.15.5:0/2744605092 shutdown_connections 2024-05-09T00:51:31.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.729+0000 7feb14c78740 1 -- 172.21.15.5:0/2744605092 wait complete. 2024-05-09T00:51:31.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.729+0000 7feb14c78740 1 -- start start 2024-05-09T00:51:31.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.730+0000 7feb14c78740 1 Processor -- start 2024-05-09T00:51:31.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.731+0000 7feb14c78740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f236d6b2f0 0x55f237142260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:31.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.731+0000 7feb14c78740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f236d734d0 0x55f2371427a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:31.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.731+0000 7feb14c78740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f236d941b0 0x55f237149700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:31.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.731+0000 7feb14c78740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f237140370 con 0x55f236d941b0 2024-05-09T00:51:31.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.731+0000 7feb07fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f236d941b0 0x55f237149700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:31.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.731+0000 7feb14c78740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55f2370239c0 con 0x55f236d6b2f0 2024-05-09T00:51:31.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.731+0000 7feb14c78740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55f2371404f0 con 0x55f236d734d0 2024-05-09T00:51:31.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.731+0000 7feb07fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f236d941b0 0x55f237149700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60366/0 (socket says 172.21.15.5:60366) 2024-05-09T00:51:31.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.731+0000 7feb07fff640 1 -- 172.21.15.5:0/3279487289 learned_addr learned my addr 172.21.15.5:0/3279487289 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:51:31.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.732+0000 7feb14c78740 10 client.0 _ll_register_callbacks cb 0x55f237050020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:51:31.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.732+0000 7feb07fff640 1 -- 172.21.15.5:0/3279487289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f236d734d0 msgr2=0x55f2371427a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:31.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.732+0000 7feb07fff640 1 --2- 172.21.15.5:0/3279487289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f236d734d0 0x55f2371427a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:31.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.732+0000 7feb07fff640 1 -- 172.21.15.5:0/3279487289 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f236d6b2f0 msgr2=0x55f237142260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:31.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.732+0000 7feb07fff640 1 --2- 172.21.15.5:0/3279487289 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f236d6b2f0 0x55f237142260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:31.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.732+0000 7feb07fff640 1 -- 172.21.15.5:0/3279487289 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f23704a1f0 con 0x55f236d941b0 2024-05-09T00:51:31.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.733+0000 7feb07fff640 1 --2- 172.21.15.5:0/3279487289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f236d941b0 0x55f237149700 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7feb000316b0 tx=0x7feb000095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6a40d225e278e12 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:51:31.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.733+0000 7feae6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:51:31.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.733+0000 7feae6ffd640 1 -- 172.21.15.5:0/3279487289 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb00004540 con 0x55f236d941b0 2024-05-09T00:51:31.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.733+0000 7feae6ffd640 1 -- 172.21.15.5:0/3279487289 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7feb000046e0 con 0x55f236d941b0 2024-05-09T00:51:31.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.733+0000 7feb14c78740 1 -- 172.21.15.5:0/3279487289 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feaf8009c30 con 0x55f236d941b0 2024-05-09T00:51:31.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.734+0000 7feae6ffd640 1 -- 172.21.15.5:0/3279487289 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7feb0002f610 con 0x55f236d941b0 2024-05-09T00:51:31.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.734+0000 7feb14c78740 10 client.16026 fetch_fsmap learned FSMap version 30 2024-05-09T00:51:31.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.734+0000 7feb14c78740 1 -- 172.21.15.5:0/3279487289 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f236dff790 con 0x55f236d941b0 2024-05-09T00:51:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.734+0000 7feae6ffd640 1 -- 172.21.15.5:0/3279487289 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feaf8009c30 con 0x55f236d941b0 2024-05-09T00:51:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.735+0000 7feae6ffd640 1 -- 172.21.15.5:0/3279487289 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7feb00059070 con 0x55f236d941b0 2024-05-09T00:51:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.735+0000 7feb14c78740 10 client.16026 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T00:51:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.735+0000 7feb14c78740 10 client.16026 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:51:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.735+0000 7feb14c78740 20 client.16026 populate_metadata read hostname 'smithi005' 2024-05-09T00:51:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.735+0000 7feb14c78740 1 --2- 172.21.15.5:0/3279487289 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55f236d52d80 0x55f23714fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.735+0000 7feb14c78740 4 client.16026 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:51:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.735+0000 7feb14c78740 1 -- 172.21.15.5:0/3279487289 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f2370aa3d0 con 0x55f236d52d80 2024-05-09T00:51:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.738+0000 7feb077fe640 1 --2- 172.21.15.5:0/3279487289 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55f236d52d80 0x55f23714fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.745+0000 7feae6ffd640 10 client.16026 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:51:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.745+0000 7feb077fe640 1 --2- 172.21.15.5:0/3279487289 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55f236d52d80 0x55f23714fcd0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a58319c5ee187c33 server_cookie=52d3ea41e222d8a3 in_seq=0 out_seq=0 2024-05-09T00:51:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.757+0000 7feae6ffd640 1 -- 172.21.15.5:0/3279487289 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f2370aa3d0 con 0x55f236d52d80 2024-05-09T00:51:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.757+0000 7feae6ffd640 10 client.16026 handle_command_reply: tid=0 2024-05-09T00:51:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.757+0000 7feb14c78740 1 client.16026 shutdown 2024-05-09T00:51:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.758+0000 7feb14c78740 1 -- 172.21.15.5:0/3279487289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f236d941b0 msgr2=0x55f237149700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.758+0000 7feb14c78740 1 --2- 172.21.15.5:0/3279487289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f236d941b0 0x55f237149700 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7feb000316b0 tx=0x7feb000095c0 comp rx=0 tx=0).stop 2024-05-09T00:51:31.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.759+0000 7feb14c78740 1 -- 172.21.15.5:0/3279487289 shutdown_connections 2024-05-09T00:51:31.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.759+0000 7feb14c78740 1 --2- 172.21.15.5:0/3279487289 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55f236d52d80 0x55f23714fcd0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:31.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.759+0000 7feb14c78740 1 --2- 172.21.15.5:0/3279487289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f236d941b0 0x55f237149700 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:31.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.759+0000 7feb14c78740 1 --2- 172.21.15.5:0/3279487289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f236d734d0 0x55f2371427a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:31.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.759+0000 7feb14c78740 1 --2- 172.21.15.5:0/3279487289 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f236d6b2f0 0x55f237142260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:31.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.759+0000 7feb14c78740 1 -- 172.21.15.5:0/3279487289 >> 172.21.15.5:0/3279487289 conn(0x55f236d88300 msgr2=0x55f237132700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:51:31.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.759+0000 7feae6ffd640 0 client.16026 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:51:31.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.760+0000 7feb14c78740 1 -- 172.21.15.5:0/3279487289 shutdown_connections 2024-05-09T00:51:31.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.760+0000 7feb14c78740 1 -- 172.21.15.5:0/3279487289 wait complete. 2024-05-09T00:51:31.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.760+0000 7feb14c78740 20 client.16026 trim_cache size 0 max 16384 2024-05-09T00:51:31.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.807+0000 7feb14c78740 1 Processor -- start 2024-05-09T00:51:31.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.807+0000 7feb14c78740 1 -- start start 2024-05-09T00:51:31.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.808+0000 7feb14c78740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f236d6b2f0 0x55f237133ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:31.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.808+0000 7feb14c78740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f236d734d0 0x55f237149620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:31.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.808+0000 7feb14c78740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f236d941b0 0x55f237149b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:31.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.808+0000 7feb14c78740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7feb081b1c40 con 0x55f236d6b2f0 2024-05-09T00:51:31.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.808+0000 7feb14c78740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7feb081b37b0 con 0x55f236d734d0 2024-05-09T00:51:31.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.808+0000 7feb14c78740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7feb081b2310 con 0x55f236d941b0 2024-05-09T00:51:31.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.809+0000 7feb077fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f236d6b2f0 0x55f237133ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:31.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.809+0000 7feb077fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f236d6b2f0 0x55f237133ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60378/0 (socket says 172.21.15.5:60378) 2024-05-09T00:51:31.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.809+0000 7feb077fe640 1 -- 172.21.15.5:0/703735778 learned_addr learned my addr 172.21.15.5:0/703735778 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:51:31.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.809+0000 7feb06ffd640 1 --2- 172.21.15.5:0/703735778 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f236d734d0 0x55f237149620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:31.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.810+0000 7feb077fe640 1 -- 172.21.15.5:0/703735778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f236d941b0 msgr2=0x55f237149b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:31.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.810+0000 7feb077fe640 1 --2- 172.21.15.5:0/703735778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f236d941b0 0x55f237149b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:31.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.810+0000 7feb077fe640 1 -- 172.21.15.5:0/703735778 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f236d734d0 msgr2=0x55f237149620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:31.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.810+0000 7feb077fe640 1 --2- 172.21.15.5:0/703735778 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f236d734d0 0x55f237149620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:31.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.810+0000 7feb077fe640 1 -- 172.21.15.5:0/703735778 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f236e765f0 con 0x55f236d6b2f0 2024-05-09T00:51:31.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.811+0000 7feb077fe640 1 --2- 172.21.15.5:0/703735778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f236d6b2f0 0x55f237133ec0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7feaf8009610 tx=0x7feaf80a9380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b209ef330c49018e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:51:31.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.811+0000 7feafd7fa640 1 -- 172.21.15.5:0/703735778 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf80c8070 con 0x55f236d6b2f0 2024-05-09T00:51:31.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.811+0000 7feafd7fa640 1 -- 172.21.15.5:0/703735778 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feaf80a5490 con 0x55f236d6b2f0 2024-05-09T00:51:31.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.811+0000 7feafd7fa640 1 -- 172.21.15.5:0/703735778 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf8020040 con 0x55f236d6b2f0 2024-05-09T00:51:31.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.813+0000 7feb14c78740 1 -- 172.21.15.5:0/703735778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f236d6b2f0 msgr2=0x55f237133ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:31.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.813+0000 7feb14c78740 1 --2- 172.21.15.5:0/703735778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f236d6b2f0 0x55f237133ec0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7feaf8009610 tx=0x7feaf80a9380 comp rx=0 tx=0).stop 2024-05-09T00:51:31.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.814+0000 7feb14c78740 1 -- 172.21.15.5:0/703735778 shutdown_connections 2024-05-09T00:51:31.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.814+0000 7feb14c78740 1 --2- 172.21.15.5:0/703735778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f236d941b0 0x55f237149b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:31.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.814+0000 7feb14c78740 1 --2- 172.21.15.5:0/703735778 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f236d734d0 0x55f237149620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:31.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.814+0000 7feb14c78740 1 --2- 172.21.15.5:0/703735778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f236d6b2f0 0x55f237133ec0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:31.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.814+0000 7feb14c78740 1 -- 172.21.15.5:0/703735778 >> 172.21.15.5:0/703735778 conn(0x55f236d88300 msgr2=0x55f2371337b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:51:31.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.814+0000 7feb14c78740 1 -- 172.21.15.5:0/703735778 shutdown_connections 2024-05-09T00:51:31.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.814+0000 7feb14c78740 1 -- 172.21.15.5:0/703735778 wait complete. 2024-05-09T00:51:31.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.815+0000 7feb14c78740 1 -- start start 2024-05-09T00:51:31.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.815+0000 7feb14c78740 1 Processor -- start 2024-05-09T00:51:31.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.816+0000 7feb14c78740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f236d6b2f0 0x55f237146d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:31.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.816+0000 7feb14c78740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f236d734d0 0x55f2371472d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.816+0000 7feb14c78740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f236d941b0 0x55f2371b6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.816+0000 7feb14c78740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7feb081b30b0 con 0x55f236d734d0 2024-05-09T00:51:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.816+0000 7feb14c78740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7feb081b29e0 con 0x55f236d941b0 2024-05-09T00:51:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.816+0000 7feb14c78740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7feb081afc20 con 0x55f236d6b2f0 2024-05-09T00:51:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.816+0000 7feb14c78740 10 client.0 _ll_register_callbacks cb 0x7feb00009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:51:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.816+0000 7feb06ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f236d734d0 0x55f2371472d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.816+0000 7feb06ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f236d734d0 0x55f2371472d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60394/0 (socket says 172.21.15.5:60394) 2024-05-09T00:51:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.816+0000 7feb06ffd640 1 -- 172.21.15.5:0/2070611964 learned_addr learned my addr 172.21.15.5:0/2070611964 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:51:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.816+0000 7feb06ffd640 1 -- 172.21.15.5:0/2070611964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f236d6b2f0 msgr2=0x55f237146d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.816+0000 7feb06ffd640 1 --2- 172.21.15.5:0/2070611964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f236d6b2f0 0x55f237146d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.817+0000 7feb06ffd640 1 -- 172.21.15.5:0/2070611964 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f236d941b0 msgr2=0x55f2371b6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.817+0000 7feb06ffd640 1 --2- 172.21.15.5:0/2070611964 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f236d941b0 0x55f2371b6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.817+0000 7feb06ffd640 1 -- 172.21.15.5:0/2070611964 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f237060ed0 con 0x55f236d734d0 2024-05-09T00:51:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.817+0000 7feb06ffd640 1 --2- 172.21.15.5:0/2070611964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f236d734d0 0x55f2371472d0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x55f237134c20 tx=0x7feaf000f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceea0a3412e2f3c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:51:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.817+0000 7feae6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:51:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.817+0000 7feae6ffd640 1 -- 172.21.15.5:0/2070611964 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf0019020 con 0x55f236d734d0 2024-05-09T00:51:31.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.817+0000 7feae6ffd640 1 -- 172.21.15.5:0/2070611964 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7feaf0014460 con 0x55f236d734d0 2024-05-09T00:51:31.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.817+0000 7feb14c78740 1 -- 172.21.15.5:0/2070611964 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feaf80a5490 con 0x55f236d734d0 2024-05-09T00:51:31.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.818+0000 7feae6ffd640 1 -- 172.21.15.5:0/2070611964 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7feaf0024050 con 0x55f236d734d0 2024-05-09T00:51:31.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.818+0000 7feb14c78740 10 client.16032 fetch_fsmap learned FSMap version 30 2024-05-09T00:51:31.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.818+0000 7feb14c78740 1 -- 172.21.15.5:0/2070611964 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f2371c85c0 con 0x55f236d734d0 2024-05-09T00:51:31.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.819+0000 7feae6ffd640 1 -- 172.21.15.5:0/2070611964 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feaf80a5490 con 0x55f236d734d0 2024-05-09T00:51:31.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.820+0000 7feae6ffd640 1 -- 172.21.15.5:0/2070611964 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7feaf00156d0 con 0x55f236d734d0 2024-05-09T00:51:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.820+0000 7feb14c78740 10 client.16032 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T00:51:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.820+0000 7feb14c78740 10 client.16032 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:51:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.820+0000 7feb14c78740 20 client.16032 populate_metadata read hostname 'smithi005' 2024-05-09T00:51:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.820+0000 7feb14c78740 1 --2- 172.21.15.5:0/2070611964 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55f236d281c0 0x55f2371cb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.820+0000 7feb14c78740 4 client.16032 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:51:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.820+0000 7feb14c78740 1 -- 172.21.15.5:0/2070611964 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f2370a4240 con 0x55f236d281c0 2024-05-09T00:51:31.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.827+0000 7feb077fe640 1 --2- 172.21.15.5:0/2070611964 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55f236d281c0 0x55f2371cb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:31.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.840+0000 7feb077fe640 1 --2- 172.21.15.5:0/2070611964 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55f236d281c0 0x55f2371cb0b0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdfca7a2b4d3ae30 server_cookie=8c390e253abd6112 in_seq=0 out_seq=0 2024-05-09T00:51:31.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:31.840+0000 7feae6ffd640 10 client.16032 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:51:33.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:51:33 smithi157 ceph-mon[34294]: pgmap v527: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 826 MiB/s wr, 1.34k op/s 2024-05-09T00:51:33.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.463+0000 7fa8dbfff700 1 -- 172.21.15.5:0/1923819704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8dc11d930 msgr2=0x7fa8dc11fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:33.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.463+0000 7fa8dbfff700 1 --2- 172.21.15.5:0/1923819704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8dc11d930 0x7fa8dc11fd50 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fa8d400b0a0 tx=0x7fa8d400b3b0 comp rx=0 tx=0).stop 2024-05-09T00:51:33.466 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:51:33 smithi005 ceph-mon[30365]: pgmap v527: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 826 MiB/s wr, 1.34k op/s 2024-05-09T00:51:33.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.464+0000 7fa8dbfff700 1 -- 172.21.15.5:0/1923819704 shutdown_connections 2024-05-09T00:51:33.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.464+0000 7fa8dbfff700 1 --2- 172.21.15.5:0/1923819704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8dc11d930 0x7fa8dc11fd50 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:33.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.464+0000 7fa8dbfff700 1 --2- 172.21.15.5:0/1923819704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8dc10a7c0 0x7fa8dc10abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:33.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.464+0000 7fa8dbfff700 1 --2- 172.21.15.5:0/1923819704 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8dc10b110 0x7fa8dc11d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:33.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.464+0000 7fa8dbfff700 1 -- 172.21.15.5:0/1923819704 >> 172.21.15.5:0/1923819704 conn(0x7fa8dc06f290 msgr2=0x7fa8dc06f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:51:33.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.465+0000 7fa8dbfff700 1 -- 172.21.15.5:0/1923819704 shutdown_connections 2024-05-09T00:51:33.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.467+0000 7fa8dbfff700 1 -- 172.21.15.5:0/1923819704 wait complete. 2024-05-09T00:51:33.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.469+0000 7fa8dbfff700 1 Processor -- start 2024-05-09T00:51:33.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.469+0000 7fa8dbfff700 1 -- start start 2024-05-09T00:51:33.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.470+0000 7fa8dbfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8dc10a7c0 0x7fa8dc11bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:33.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.470+0000 7fa8dbfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8dc10b110 0x7fa8dc114e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:33.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.470+0000 7fa8dbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8dc11d930 0x7fa8dc115430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:33.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.470+0000 7fa8dbfff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa8dc1230b0 con 0x7fa8dc11d930 2024-05-09T00:51:33.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.470+0000 7fa8dbfff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa8dc122f30 con 0x7fa8dc10b110 2024-05-09T00:51:33.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.470+0000 7fa8dbfff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa8dc123230 con 0x7fa8dc10a7c0 2024-05-09T00:51:33.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.471+0000 7fa8db7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8dc11d930 0x7fa8dc115430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:33.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.471+0000 7fa8db7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8dc11d930 0x7fa8dc115430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60420/0 (socket says 172.21.15.5:60420) 2024-05-09T00:51:33.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.472+0000 7fa8db7fe700 1 -- 172.21.15.5:0/4240023738 learned_addr learned my addr 172.21.15.5:0/4240023738 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:51:33.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.472+0000 7fa8da7fc700 1 --2- 172.21.15.5:0/4240023738 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8dc10b110 0x7fa8dc114e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:33.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.472+0000 7fa8db7fe700 1 -- 172.21.15.5:0/4240023738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8dc10a7c0 msgr2=0x7fa8dc11bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:33.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.472+0000 7fa8db7fe700 1 --2- 172.21.15.5:0/4240023738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8dc10a7c0 0x7fa8dc11bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:33.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.472+0000 7fa8db7fe700 1 -- 172.21.15.5:0/4240023738 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8dc10b110 msgr2=0x7fa8dc114e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:33.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.472+0000 7fa8db7fe700 1 --2- 172.21.15.5:0/4240023738 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8dc10b110 0x7fa8dc114e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:33.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.472+0000 7fa8db7fe700 1 -- 172.21.15.5:0/4240023738 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8dc115c90 con 0x7fa8dc11d930 2024-05-09T00:51:33.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.473+0000 7fa8db7fe700 1 --2- 172.21.15.5:0/4240023738 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8dc11d930 0x7fa8dc115430 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fa8d4009fd0 tx=0x7fa8d4004e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:51:33.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.473+0000 7fa8d3fff700 1 -- 172.21.15.5:0/4240023738 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d4021070 con 0x7fa8dc11d930 2024-05-09T00:51:33.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.473+0000 7fa8dbfff700 1 -- 172.21.15.5:0/4240023738 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8dc119930 con 0x7fa8dc11d930 2024-05-09T00:51:33.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.474+0000 7fa8dbfff700 1 -- 172.21.15.5:0/4240023738 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8dc119e70 con 0x7fa8dc11d930 2024-05-09T00:51:33.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.480+0000 7fa8efed5b80 1 Processor -- start 2024-05-09T00:51:33.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.481+0000 7fa8efed5b80 1 -- start start 2024-05-09T00:51:33.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.481+0000 7fa8efed5b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557996933160 0x5579969cc740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:33.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.481+0000 7fa8efed5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579968494a0 0x5579969ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:33.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.481+0000 7fa8efed5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579969c6800 0x5579969c6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:33.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.481+0000 7fa8efed5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5579968e8ff0 con 0x5579968494a0 2024-05-09T00:51:33.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.481+0000 7fa8efed5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5579968877f0 con 0x5579969c6800 2024-05-09T00:51:33.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.481+0000 7fa8efed5b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5579968e4fb0 con 0x557996933160 2024-05-09T00:51:33.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.482+0000 7fa8d3fff700 1 -- 172.21.15.5:0/4240023738 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8d4007c00 con 0x7fa8dc11d930 2024-05-09T00:51:33.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.482+0000 7fa8d3fff700 1 -- 172.21.15.5:0/4240023738 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d4005400 con 0x7fa8dc11d930 2024-05-09T00:51:33.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.486+0000 7fa8da7fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557996933160 0x5579969cc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:33.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.486+0000 7fa8da7fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579969c6800 0x5579969c6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:33.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.489+0000 7fa8daffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579968494a0 0x5579969ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:33.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.489+0000 7fa8da7fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579969c6800 0x5579969c6c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:43038/0 (socket says 172.21.15.5:43038) 2024-05-09T00:51:33.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.489+0000 7fa8da7fc700 1 -- 172.21.15.5:0/3804296760 learned_addr learned my addr 172.21.15.5:0/3804296760 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:51:33.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.490+0000 7fa8d3fff700 1 -- 172.21.15.5:0/4240023738 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fa8d4024070 con 0x7fa8dc11d930 2024-05-09T00:51:33.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.491+0000 7fa8d3fff700 1 --2- 172.21.15.5:0/4240023738 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fa8c80b2a10 0x7fa8c80b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:33.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.496+0000 7fa8db7fe700 1 --2- 172.21.15.5:0/4240023738 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fa8c80b2a10 0x7fa8c80b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:33.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.496+0000 7fa8da7fc700 1 -- 172.21.15.5:0/3804296760 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557996933160 msgr2=0x5579969cc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:33.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.496+0000 7fa8da7fc700 1 --2- 172.21.15.5:0/3804296760 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557996933160 0x5579969cc740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:33.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.496+0000 7fa8da7fc700 1 -- 172.21.15.5:0/3804296760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579968494a0 msgr2=0x5579969ccc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:33.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.496+0000 7fa8da7fc700 1 --2- 172.21.15.5:0/3804296760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579968494a0 0x5579969ccc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:33.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.496+0000 7fa8da7fc700 1 -- 172.21.15.5:0/3804296760 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55799687f370 con 0x5579969c6800 2024-05-09T00:51:33.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.496+0000 7fa8d3fff700 1 -- 172.21.15.5:0/4240023738 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fa8d40a6f10 con 0x7fa8dc11d930 2024-05-09T00:51:33.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.497+0000 7fa8db7fe700 1 --2- 172.21.15.5:0/4240023738 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fa8c80b2a10 0x7fa8c80b4f20 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa8d40aa620 tx=0x7fa8d40a6040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:51:33.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.501+0000 7fa8da7fc700 1 --2- 172.21.15.5:0/3804296760 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579969c6800 0x5579969c6c30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc013950 tx=0x7fa8cc0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f478a7ea68f74a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:51:33.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.506+0000 7fa8d17fa700 1 -- 172.21.15.5:0/3804296760 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc011640 con 0x5579969c6800 2024-05-09T00:51:33.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.507+0000 7fa8d17fa700 1 -- 172.21.15.5:0/3804296760 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8cc00e9b0 con 0x5579969c6800 2024-05-09T00:51:33.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.508+0000 7fa8efed5b80 1 -- 172.21.15.5:0/3804296760 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579969c6800 msgr2=0x5579969c6c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:33.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.508+0000 7fa8efed5b80 1 --2- 172.21.15.5:0/3804296760 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579969c6800 0x5579969c6c30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc013950 tx=0x7fa8cc0047c0 comp rx=0 tx=0).stop 2024-05-09T00:51:33.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.509+0000 7fa8efed5b80 1 -- 172.21.15.5:0/3804296760 shutdown_connections 2024-05-09T00:51:33.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.509+0000 7fa8efed5b80 1 --2- 172.21.15.5:0/3804296760 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579969c6800 0x5579969c6c30 secure :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc013950 tx=0x7fa8cc0047c0 comp rx=0 tx=0).stop 2024-05-09T00:51:33.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.509+0000 7fa8efed5b80 1 --2- 172.21.15.5:0/3804296760 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557996933160 0x5579969cc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:33.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.509+0000 7fa8efed5b80 1 --2- 172.21.15.5:0/3804296760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579968494a0 0x5579969ccc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:33.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.509+0000 7fa8efed5b80 1 -- 172.21.15.5:0/3804296760 >> 172.21.15.5:0/3804296760 conn(0x5579968a96a0 msgr2=0x5579969c16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:51:33.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.512+0000 7fa8efed5b80 1 -- 172.21.15.5:0/3804296760 shutdown_connections 2024-05-09T00:51:33.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.512+0000 7fa8efed5b80 1 -- 172.21.15.5:0/3804296760 wait complete. 2024-05-09T00:51:33.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.513+0000 7fa8efed5b80 1 -- start start 2024-05-09T00:51:33.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.513+0000 7fa8efed5b80 1 Processor -- start 2024-05-09T00:51:33.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.515+0000 7fa8efed5b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5579968494a0 0x5579969d0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:33.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.515+0000 7fa8efed5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557996933160 0x5579969d09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:33.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.516+0000 7fa8efed5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579969d0f00 0x5579969dc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:33.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.516+0000 7fa8efed5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557996764ba0 con 0x557996933160 2024-05-09T00:51:33.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.516+0000 7fa8efed5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5579968a3150 con 0x5579969d0f00 2024-05-09T00:51:33.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.516+0000 7fa8efed5b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5579969d3420 con 0x5579968494a0 2024-05-09T00:51:33.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.516+0000 7fa8efed5b80 10 client.0 _ll_register_callbacks cb 0x55799675a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:51:33.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.520+0000 7fa8da7fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579969d0f00 0x5579969dc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:33.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.520+0000 7fa8da7fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5579968494a0 0x5579969d0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:33.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.520+0000 7fa8daffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557996933160 0x5579969d09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:33.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.520+0000 7fa8daffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557996933160 0x5579969d09c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60442/0 (socket says 172.21.15.5:60442) 2024-05-09T00:51:33.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.520+0000 7fa8daffd700 1 -- 172.21.15.5:0/1666627145 learned_addr learned my addr 172.21.15.5:0/1666627145 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:51:33.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.523+0000 7fa8daffd700 1 -- 172.21.15.5:0/1666627145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5579968494a0 msgr2=0x5579969d0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:33.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.523+0000 7fa8daffd700 1 --2- 172.21.15.5:0/1666627145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5579968494a0 0x5579969d0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:33.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.523+0000 7fa8daffd700 1 -- 172.21.15.5:0/1666627145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579969d0f00 msgr2=0x5579969dc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:33.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.523+0000 7fa8daffd700 1 --2- 172.21.15.5:0/1666627145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579969d0f00 0x5579969dc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:33.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.523+0000 7fa8daffd700 1 -- 172.21.15.5:0/1666627145 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557996764490 con 0x557996933160 2024-05-09T00:51:33.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.529+0000 7fa8daffd700 1 --2- 172.21.15.5:0/1666627145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557996933160 0x5579969d09c0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4003930 tx=0x7fa8c4003cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf8a46d5203a8be1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:51:33.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.529+0000 7fa8c27fc700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:51:33.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.534+0000 7fa8c27fc700 1 -- 172.21.15.5:0/1666627145 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c4003e10 con 0x557996933160 2024-05-09T00:51:33.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.534+0000 7fa8c27fc700 1 -- 172.21.15.5:0/1666627145 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fa8c401a470 con 0x557996933160 2024-05-09T00:51:33.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.534+0000 7fa8efed5b80 1 -- 172.21.15.5:0/1666627145 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5579968f69d0 con 0x557996933160 2024-05-09T00:51:33.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.536+0000 7fa8c27fc700 1 -- 172.21.15.5:0/1666627145 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa8c4020030 con 0x557996933160 2024-05-09T00:51:33.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.541+0000 7fa8efed5b80 10 client.16044 fetch_fsmap learned FSMap version 30 2024-05-09T00:51:33.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.541+0000 7fa8efed5b80 1 -- 172.21.15.5:0/1666627145 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5579969de530 con 0x557996933160 2024-05-09T00:51:33.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.544+0000 7fa8c27fc700 1 -- 172.21.15.5:0/1666627145 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8c4015070 con 0x557996933160 2024-05-09T00:51:33.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.544+0000 7fa8c27fc700 1 -- 172.21.15.5:0/1666627145 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7fa8c4025640 con 0x557996933160 2024-05-09T00:51:33.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.545+0000 7fa8efed5b80 10 client.16044 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T00:51:33.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.545+0000 7fa8efed5b80 10 client.16044 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:51:33.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.545+0000 7fa8efed5b80 20 client.16044 populate_metadata read hostname 'smithi005' 2024-05-09T00:51:33.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.545+0000 7fa8efed5b80 1 --2- 172.21.15.5:0/1666627145 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5579969e0b70 0x5579969e2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:33.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.545+0000 7fa8efed5b80 4 client.16044 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:51:33.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.545+0000 7fa8efed5b80 1 -- 172.21.15.5:0/1666627145 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5579968f69d0 con 0x5579969e0b70 2024-05-09T00:51:33.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.549+0000 7fa8da7fc700 1 --2- 172.21.15.5:0/1666627145 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5579969e0b70 0x5579969e2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:33.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.552+0000 7fa8da7fc700 1 --2- 172.21.15.5:0/1666627145 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5579969e0b70 0x5579969e2fc0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c057cdd907bae183 server_cookie=597300ed34b92f81 in_seq=0 out_seq=0 2024-05-09T00:51:33.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.553+0000 7fa8c27fc700 10 client.16044 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:51:33.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.559+0000 7fa8c27fc700 1 -- 172.21.15.5:0/1666627145 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5579968f69d0 con 0x5579969e0b70 2024-05-09T00:51:33.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.559+0000 7fa8c27fc700 10 client.16044 handle_command_reply: tid=0 2024-05-09T00:51:33.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.560+0000 7fa8efed5b80 1 client.16044 shutdown 2024-05-09T00:51:33.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.560+0000 7fa8efed5b80 1 -- 172.21.15.5:0/1666627145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557996933160 msgr2=0x5579969d09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:33.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.560+0000 7fa8efed5b80 1 --2- 172.21.15.5:0/1666627145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557996933160 0x5579969d09c0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4003930 tx=0x7fa8c4003cf0 comp rx=0 tx=0).stop 2024-05-09T00:51:33.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.560+0000 7fa8efed5b80 1 -- 172.21.15.5:0/1666627145 shutdown_connections 2024-05-09T00:51:33.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.560+0000 7fa8efed5b80 1 --2- 172.21.15.5:0/1666627145 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5579969e0b70 0x5579969e2fc0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:33.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.560+0000 7fa8efed5b80 1 --2- 172.21.15.5:0/1666627145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579969d0f00 0x5579969dc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:33.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.560+0000 7fa8efed5b80 1 --2- 172.21.15.5:0/1666627145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5579968494a0 0x5579969d0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:33.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.561+0000 7fa8c27fc700 0 client.16044 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:51:33.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.561+0000 7fa8efed5b80 1 --2- 172.21.15.5:0/1666627145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557996933160 0x5579969d09c0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:33.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.561+0000 7fa8efed5b80 1 -- 172.21.15.5:0/1666627145 >> 172.21.15.5:0/1666627145 conn(0x5579968a96a0 msgr2=0x5579969c13d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:51:33.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.565+0000 7fa8efed5b80 1 -- 172.21.15.5:0/1666627145 shutdown_connections 2024-05-09T00:51:33.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.565+0000 7fa8efed5b80 1 -- 172.21.15.5:0/1666627145 wait complete. 2024-05-09T00:51:33.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.565+0000 7fa8efed5b80 20 client.16044 trim_cache size 0 max 16384 2024-05-09T00:51:33.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:51:33 smithi131 ceph-mon[34547]: pgmap v527: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 826 MiB/s wr, 1.34k op/s 2024-05-09T00:51:33.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.607+0000 7fa8efed5b80 1 Processor -- start 2024-05-09T00:51:33.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.610+0000 7fa8efed5b80 1 -- start start 2024-05-09T00:51:33.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.611+0000 7fa8efed5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579968a96a0 0x5579969ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:33.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.611+0000 7fa8efed5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579968494a0 0x5579969c6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:33.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.611+0000 7fa8efed5b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557996933160 0x5579969e8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:33.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.611+0000 7fa8efed5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa8bc00d540 con 0x5579968a96a0 2024-05-09T00:51:33.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.611+0000 7fa8efed5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa8bc00f0b0 con 0x5579968494a0 2024-05-09T00:51:33.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.611+0000 7fa8efed5b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa8bc00dc10 con 0x557996933160 2024-05-09T00:51:33.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.613+0000 7fa8daffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579968494a0 0x5579969c6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:33.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.613+0000 7fa8daffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579968494a0 0x5579969c6220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:36670/0 (socket says 172.21.15.5:36670) 2024-05-09T00:51:33.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.613+0000 7fa8daffd700 1 -- 172.21.15.5:0/1261595343 learned_addr learned my addr 172.21.15.5:0/1261595343 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:51:33.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.613+0000 7fa8da7fc700 1 --2- 172.21.15.5:0/1261595343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579968a96a0 0x5579969ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:33.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.613+0000 7fa8da7fc700 1 --2- 172.21.15.5:0/1261595343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557996933160 0x5579969e8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:33.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.617+0000 7fa8daffd700 1 -- 172.21.15.5:0/1261595343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557996933160 msgr2=0x5579969e8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:33.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.617+0000 7fa8daffd700 1 --2- 172.21.15.5:0/1261595343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557996933160 0x5579969e8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:33.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.617+0000 7fa8daffd700 1 -- 172.21.15.5:0/1261595343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579968a96a0 msgr2=0x5579969ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:33.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.617+0000 7fa8daffd700 1 --2- 172.21.15.5:0/1261595343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579968a96a0 0x5579969ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:33.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.617+0000 7fa8daffd700 1 -- 172.21.15.5:0/1261595343 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579969c78e0 con 0x5579968494a0 2024-05-09T00:51:33.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.618+0000 7fa8daffd700 1 --2- 172.21.15.5:0/1261595343 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579968494a0 0x5579969c6220 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4025bd0 tx=0x7fa8c400b430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41661247a2dd28d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:51:33.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.621+0000 7fa8d0ff9700 1 -- 172.21.15.5:0/1261595343 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c400bd20 con 0x5579968494a0 2024-05-09T00:51:33.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.621+0000 7fa8d0ff9700 1 -- 172.21.15.5:0/1261595343 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8c4015070 con 0x5579968494a0 2024-05-09T00:51:33.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.621+0000 7fa8d0ff9700 1 -- 172.21.15.5:0/1261595343 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c401a9b0 con 0x5579968494a0 2024-05-09T00:51:33.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.623+0000 7fa8efed5b80 1 -- 172.21.15.5:0/1261595343 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579968494a0 msgr2=0x5579969c6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:33.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.623+0000 7fa8efed5b80 1 --2- 172.21.15.5:0/1261595343 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579968494a0 0x5579969c6220 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4025bd0 tx=0x7fa8c400b430 comp rx=0 tx=0).stop 2024-05-09T00:51:33.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.626+0000 7fa8efed5b80 1 -- 172.21.15.5:0/1261595343 shutdown_connections 2024-05-09T00:51:33.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.626+0000 7fa8efed5b80 1 --2- 172.21.15.5:0/1261595343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557996933160 0x5579969e8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:33.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.626+0000 7fa8efed5b80 1 --2- 172.21.15.5:0/1261595343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579968a96a0 0x5579969ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:33.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.626+0000 7fa8efed5b80 1 --2- 172.21.15.5:0/1261595343 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579968494a0 0x5579969c6220 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:33.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.626+0000 7fa8efed5b80 1 -- 172.21.15.5:0/1261595343 >> 172.21.15.5:0/1261595343 conn(0x5579969cabe0 msgr2=0x5579969cbae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:51:33.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.628+0000 7fa8efed5b80 1 -- 172.21.15.5:0/1261595343 shutdown_connections 2024-05-09T00:51:33.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.629+0000 7fa8efed5b80 1 -- 172.21.15.5:0/1261595343 wait complete. 2024-05-09T00:51:33.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.629+0000 7fa8efed5b80 1 -- start start 2024-05-09T00:51:33.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.630+0000 7fa8efed5b80 1 Processor -- start 2024-05-09T00:51:33.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.630+0000 7fa8efed5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579968494a0 0x5579969d6280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:33.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.630+0000 7fa8efed5b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5579968a96a0 0x5579969d67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:33.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.630+0000 7fa8efed5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557996933160 0x5579969d6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:33.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.630+0000 7fa8efed5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa8bc00e9b0 con 0x5579968494a0 2024-05-09T00:51:33.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.630+0000 7fa8efed5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa8bc00e2e0 con 0x557996933160 2024-05-09T00:51:33.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.630+0000 7fa8efed5b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa8bc00b4e0 con 0x5579968a96a0 2024-05-09T00:51:33.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.630+0000 7fa8efed5b80 10 client.0 _ll_register_callbacks cb 0x55799675a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:51:33.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.631+0000 7fa8daffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5579968a96a0 0x5579969d67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:33.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.631+0000 7fa8daffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5579968a96a0 0x5579969d67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:59964/0 (socket says 172.21.15.5:59964) 2024-05-09T00:51:33.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.631+0000 7fa8daffd700 1 -- 172.21.15.5:0/104784212 learned_addr learned my addr 172.21.15.5:0/104784212 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:51:33.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.638+0000 7fa8da7fc700 1 --2- 172.21.15.5:0/104784212 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557996933160 0x5579969d6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:33.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.638+0000 7fa8da7fc700 1 --2- 172.21.15.5:0/104784212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579968494a0 0x5579969d6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:33.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.639+0000 7fa8daffd700 1 -- 172.21.15.5:0/104784212 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557996933160 msgr2=0x5579969d6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:33.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.639+0000 7fa8daffd700 1 --2- 172.21.15.5:0/104784212 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557996933160 0x5579969d6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:33.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.639+0000 7fa8daffd700 1 -- 172.21.15.5:0/104784212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579968494a0 msgr2=0x5579969d6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:33.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.639+0000 7fa8daffd700 1 --2- 172.21.15.5:0/104784212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579968494a0 0x5579969d6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:33.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.639+0000 7fa8daffd700 1 -- 172.21.15.5:0/104784212 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557996a4a340 con 0x5579968a96a0 2024-05-09T00:51:33.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.643+0000 7fa8daffd700 1 --2- 172.21.15.5:0/104784212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5579968a96a0 0x5579969d67c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4027010 tx=0x7fa8c401a520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=495bb4faa21fd1c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:51:33.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.644+0000 7fa8c27fc700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-09T00:51:33.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.647+0000 7fa8c27fc700 1 -- 172.21.15.5:0/104784212 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c401a5f0 con 0x5579968a96a0 2024-05-09T00:51:33.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.647+0000 7fa8efed5b80 1 -- 172.21.15.5:0/104784212 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8c4015070 con 0x5579968a96a0 2024-05-09T00:51:33.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.647+0000 7fa8c27fc700 1 -- 172.21.15.5:0/104784212 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fa8c4021070 con 0x5579968a96a0 2024-05-09T00:51:33.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.652+0000 7fa8c27fc700 1 -- 172.21.15.5:0/104784212 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa8c401a790 con 0x5579968a96a0 2024-05-09T00:51:33.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.653+0000 7fa8efed5b80 10 client.25612 fetch_fsmap learned FSMap version 30 2024-05-09T00:51:33.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.653+0000 7fa8efed5b80 1 -- 172.21.15.5:0/104784212 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557996a4b520 con 0x5579968a96a0 2024-05-09T00:51:33.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.657+0000 7fa8c27fc700 1 -- 172.21.15.5:0/104784212 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8c4015070 con 0x5579968a96a0 2024-05-09T00:51:33.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.657+0000 7fa8c27fc700 1 -- 172.21.15.5:0/104784212 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7fa8c4040a50 con 0x5579968a96a0 2024-05-09T00:51:33.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.657+0000 7fa8efed5b80 10 client.25612 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T00:51:33.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.657+0000 7fa8efed5b80 10 client.25612 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:51:33.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.657+0000 7fa8efed5b80 20 client.25612 populate_metadata read hostname 'smithi005' 2024-05-09T00:51:33.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.657+0000 7fa8efed5b80 1 --2- 172.21.15.5:0/104784212 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5579969d0f00 0x557996a4da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:33.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.658+0000 7fa8efed5b80 4 client.25612 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:51:33.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.658+0000 7fa8efed5b80 1 -- 172.21.15.5:0/104784212 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5579968f0a70 con 0x5579969d0f00 2024-05-09T00:51:33.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.660+0000 7fa8da7fc700 1 --2- 172.21.15.5:0/104784212 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5579969d0f00 0x557996a4da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:33.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.672+0000 7fa8da7fc700 1 --2- 172.21.15.5:0/104784212 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5579969d0f00 0x557996a4da70 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c684d9ad5df8e5 server_cookie=55e7d0ea74ebc200 in_seq=0 out_seq=0 2024-05-09T00:51:33.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.672+0000 7fa8c27fc700 10 client.25612 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:51:33.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.677+0000 7fa8c27fc700 1 -- 172.21.15.5:0/104784212 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5579968f0a70 con 0x5579969d0f00 2024-05-09T00:51:33.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.677+0000 7fa8c27fc700 10 client.25612 handle_command_reply: tid=0 2024-05-09T00:51:33.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.677+0000 7fa8efed5b80 1 client.25612 shutdown 2024-05-09T00:51:33.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.677+0000 7fa8efed5b80 1 -- 172.21.15.5:0/104784212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5579968a96a0 msgr2=0x5579969d67c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:33.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.677+0000 7fa8efed5b80 1 --2- 172.21.15.5:0/104784212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5579968a96a0 0x5579969d67c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4027010 tx=0x7fa8c401a520 comp rx=0 tx=0).stop 2024-05-09T00:51:33.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.678+0000 7fa8efed5b80 1 -- 172.21.15.5:0/104784212 shutdown_connections 2024-05-09T00:51:33.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.678+0000 7fa8efed5b80 1 --2- 172.21.15.5:0/104784212 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5579969d0f00 0x557996a4da70 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:33.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.678+0000 7fa8efed5b80 1 --2- 172.21.15.5:0/104784212 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557996933160 0x5579969d6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:33.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.678+0000 7fa8efed5b80 1 --2- 172.21.15.5:0/104784212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579968494a0 0x5579969d6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:33.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.678+0000 7fa8efed5b80 1 --2- 172.21.15.5:0/104784212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5579968a96a0 0x5579969d67c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:33.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.678+0000 7fa8efed5b80 1 -- 172.21.15.5:0/104784212 >> 172.21.15.5:0/104784212 conn(0x5579969cabe0 msgr2=0x5579969d1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:51:33.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.678+0000 7fa8c27fc700 0 client.25612 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:51:33.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.679+0000 7fa8efed5b80 1 -- 172.21.15.5:0/104784212 shutdown_connections 2024-05-09T00:51:33.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.679+0000 7fa8efed5b80 1 -- 172.21.15.5:0/104784212 wait complete. 2024-05-09T00:51:33.683 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-09T00:51:33.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.682+0000 7fa8efed5b80 20 client.25612 trim_cache size 0 max 16384 2024-05-09T00:51:33.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.683+0000 7fa8dbfff700 1 -- 172.21.15.5:0/4240023738 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fa8c80b2a10 msgr2=0x7fa8c80b4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:33.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.683+0000 7fa8dbfff700 1 --2- 172.21.15.5:0/4240023738 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fa8c80b2a10 0x7fa8c80b4f20 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa8d40aa620 tx=0x7fa8d40a6040 comp rx=0 tx=0).stop 2024-05-09T00:51:33.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.684+0000 7fa8dbfff700 1 -- 172.21.15.5:0/4240023738 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8dc11d930 msgr2=0x7fa8dc115430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:33.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.684+0000 7fa8dbfff700 1 --2- 172.21.15.5:0/4240023738 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8dc11d930 0x7fa8dc115430 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fa8d4009fd0 tx=0x7fa8d4004e90 comp rx=0 tx=0).stop 2024-05-09T00:51:33.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.686+0000 7fa8dbfff700 1 -- 172.21.15.5:0/4240023738 shutdown_connections 2024-05-09T00:51:33.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.686+0000 7fa8dbfff700 1 --2- 172.21.15.5:0/4240023738 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fa8c80b2a10 0x7fa8c80b4f20 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:33.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.686+0000 7fa8dbfff700 1 --2- 172.21.15.5:0/4240023738 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8dc11d930 0x7fa8dc115430 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:33.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.686+0000 7fa8dbfff700 1 --2- 172.21.15.5:0/4240023738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8dc10a7c0 0x7fa8dc11bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:33.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.686+0000 7fa8dbfff700 1 --2- 172.21.15.5:0/4240023738 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8dc10b110 0x7fa8dc114e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:33.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.686+0000 7fa8dbfff700 1 -- 172.21.15.5:0/4240023738 >> 172.21.15.5:0/4240023738 conn(0x7fa8dc06f290 msgr2=0x7fa8dc072510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:51:33.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.687+0000 7fa8dbfff700 1 -- 172.21.15.5:0/4240023738 shutdown_connections 2024-05-09T00:51:33.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:33.687+0000 7fa8dbfff700 1 -- 172.21.15.5:0/4240023738 wait complete. 2024-05-09T00:51:35.407 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:51:35 smithi131 ceph-mon[34547]: pgmap v528: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 876 MiB/s wr, 1.38k op/s 2024-05-09T00:51:35.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:51:35 smithi157 ceph-mon[34294]: pgmap v528: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 876 MiB/s wr, 1.38k op/s 2024-05-09T00:51:35.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:51:35 smithi005 ceph-mon[30365]: pgmap v528: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 876 MiB/s wr, 1.38k op/s 2024-05-09T00:51:36.883 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T00:51:36.884 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T00:51:37.293 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:51:37 smithi005 ceph-mon[30365]: pgmap v529: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 948 MiB/s wr, 1.48k op/s 2024-05-09T00:51:37.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:51:37 smithi157 ceph-mon[34294]: pgmap v529: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 948 MiB/s wr, 1.48k op/s 2024-05-09T00:51:37.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:51:37 smithi131 ceph-mon[34547]: pgmap v529: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 948 MiB/s wr, 1.48k op/s 2024-05-09T00:51:37.812 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:51:39.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:51:39 smithi005 ceph-mon[30365]: pgmap v530: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 835 MiB/s wr, 1.41k op/s 2024-05-09T00:51:39.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:51:39 smithi157 ceph-mon[34294]: pgmap v530: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 835 MiB/s wr, 1.41k op/s 2024-05-09T00:51:39.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:51:39 smithi131 ceph-mon[34547]: pgmap v530: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 835 MiB/s wr, 1.41k op/s 2024-05-09T00:51:39.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:39.945+0000 7feae6ffd640 1 -- 172.21.15.5:0/2070611964 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f2370a4240 con 0x55f236d281c0 2024-05-09T00:51:39.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:39.945+0000 7feae6ffd640 10 client.16032 handle_command_reply: tid=0 2024-05-09T00:51:39.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:39.945+0000 7feb14c78740 1 client.16032 shutdown 2024-05-09T00:51:39.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:39.946+0000 7feb14c78740 1 -- 172.21.15.5:0/2070611964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f236d734d0 msgr2=0x55f2371472d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:39.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:39.946+0000 7feb14c78740 1 --2- 172.21.15.5:0/2070611964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f236d734d0 0x55f2371472d0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x55f237134c20 tx=0x7feaf000f5b0 comp rx=0 tx=0).stop 2024-05-09T00:51:39.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:39.946+0000 7feb14c78740 1 -- 172.21.15.5:0/2070611964 shutdown_connections 2024-05-09T00:51:39.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:39.946+0000 7feb14c78740 1 --2- 172.21.15.5:0/2070611964 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55f236d281c0 0x55f2371cb0b0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:39.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:39.946+0000 7feb14c78740 1 --2- 172.21.15.5:0/2070611964 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f236d941b0 0x55f2371b6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:39.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:39.946+0000 7feb14c78740 1 --2- 172.21.15.5:0/2070611964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f236d734d0 0x55f2371472d0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:39.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:39.946+0000 7feb14c78740 1 --2- 172.21.15.5:0/2070611964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f236d6b2f0 0x55f237146d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:39.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:39.946+0000 7feb14c78740 1 -- 172.21.15.5:0/2070611964 >> 172.21.15.5:0/2070611964 conn(0x55f236d88300 msgr2=0x55f237140890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:51:39.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:39.946+0000 7feae6ffd640 0 client.16032 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:51:39.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:39.946+0000 7feb14c78740 1 -- 172.21.15.5:0/2070611964 shutdown_connections 2024-05-09T00:51:39.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:39.946+0000 7feb14c78740 1 -- 172.21.15.5:0/2070611964 wait complete. 2024-05-09T00:51:39.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:39.947+0000 7feb14c78740 20 client.16032 trim_cache size 0 max 16384 2024-05-09T00:51:39.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-09T00:51:39.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-09T00:51:39.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-09T00:51:39.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:39.951+0000 7feb0ddc7640 1 -- 172.21.15.5:0/1125462730 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7feaf40b28e0 msgr2=0x7feaf40b4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:39.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:39.951+0000 7feb0ddc7640 1 --2- 172.21.15.5:0/1125462730 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7feaf40b28e0 0x7feaf40b4df0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7feb0002aeb0 tx=0x7feb00009360 comp rx=0 tx=0).stop 2024-05-09T00:51:39.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:39.951+0000 7feb0ddc7640 1 -- 172.21.15.5:0/1125462730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb08077a40 msgr2=0x7feb081a0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:39.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:39.951+0000 7feb0ddc7640 1 --2- 172.21.15.5:0/1125462730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb08077a40 0x7feb081a0980 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7feaf800d740 tx=0x7feaf800dc00 comp rx=0 tx=0).stop 2024-05-09T00:51:39.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:39.951+0000 7feb0ddc7640 1 -- 172.21.15.5:0/1125462730 shutdown_connections 2024-05-09T00:51:39.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:39.951+0000 7feb0ddc7640 1 --2- 172.21.15.5:0/1125462730 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7feaf40b28e0 0x7feaf40b4df0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:39.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:39.951+0000 7feb0ddc7640 1 --2- 172.21.15.5:0/1125462730 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb080799d0 0x7feb081a7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:39.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:39.951+0000 7feb0ddc7640 1 --2- 172.21.15.5:0/1125462730 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb08078cd0 0x7feb081a0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:39.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:39.951+0000 7feb0ddc7640 1 --2- 172.21.15.5:0/1125462730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb08077a40 0x7feb081a0980 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:39.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:39.951+0000 7feb0ddc7640 1 -- 172.21.15.5:0/1125462730 >> 172.21.15.5:0/1125462730 conn(0x7feb08101060 msgr2=0x7feb0810fd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:51:39.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:39.952+0000 7feb0ddc7640 1 -- 172.21.15.5:0/1125462730 shutdown_connections 2024-05-09T00:51:39.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:39.952+0000 7feb0ddc7640 1 -- 172.21.15.5:0/1125462730 wait complete. 2024-05-09T00:51:41.403 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:51:41 smithi005 ceph-mon[30365]: pgmap v531: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1000 MiB/s wr, 1.69k op/s 2024-05-09T00:51:41.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:51:41 smithi131 ceph-mon[34547]: pgmap v531: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1000 MiB/s wr, 1.69k op/s 2024-05-09T00:51:41.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:51:41 smithi157 ceph-mon[34294]: pgmap v531: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1000 MiB/s wr, 1.69k op/s 2024-05-09T00:51:42.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.768+0000 7fb62143c700 1 -- 172.21.15.5:0/36209839 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb60c003b80 con 0x7fb61c070060 2024-05-09T00:51:42.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.769+0000 7fb6246a2700 1 -- 172.21.15.5:0/36209839 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb61c070060 msgr2=0x7fb61c11d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:42.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.769+0000 7fb6246a2700 1 --2- 172.21.15.5:0/36209839 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb61c070060 0x7fb61c11d4a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb60c01a9a0 tx=0x7fb60c01acb0 comp rx=0 tx=0).stop 2024-05-09T00:51:42.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.771+0000 7fb6246a2700 1 -- 172.21.15.5:0/36209839 shutdown_connections 2024-05-09T00:51:42.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.771+0000 7fb6246a2700 1 --2- 172.21.15.5:0/36209839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb61c11d9e0 0x7fb61c11fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:42.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.771+0000 7fb6246a2700 1 --2- 172.21.15.5:0/36209839 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb61c06f710 0x7fb61c06fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:42.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.771+0000 7fb6246a2700 1 --2- 172.21.15.5:0/36209839 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb61c070060 0x7fb61c11d4a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:42.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.771+0000 7fb6246a2700 1 -- 172.21.15.5:0/36209839 >> 172.21.15.5:0/36209839 conn(0x7fb61c10ab60 msgr2=0x7fb61c10cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:51:42.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.773+0000 7fb6246a2700 1 -- 172.21.15.5:0/36209839 shutdown_connections 2024-05-09T00:51:42.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.773+0000 7fb6246a2700 1 -- 172.21.15.5:0/36209839 wait complete. 2024-05-09T00:51:42.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.775+0000 7fb6246a2700 1 Processor -- start 2024-05-09T00:51:42.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.775+0000 7fb6246a2700 1 -- start start 2024-05-09T00:51:42.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.776+0000 7fb6246a2700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb61c06f710 0x7fb61c11d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:42.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.776+0000 7fb6246a2700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb61c070060 0x7fb61c119400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:42.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.776+0000 7fb6246a2700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb61c11d9e0 0x7fb61c119940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:42.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.776+0000 7fb6246a2700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb61c111470 con 0x7fb61c070060 2024-05-09T00:51:42.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.776+0000 7fb6246a2700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb61c1112f0 con 0x7fb61c11d9e0 2024-05-09T00:51:42.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.776+0000 7fb6246a2700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb61c1115f0 con 0x7fb61c06f710 2024-05-09T00:51:42.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.777+0000 7fb62243e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb61c06f710 0x7fb61c11d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:42.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.777+0000 7fb62243e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb61c06f710 0x7fb61c11d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:59982/0 (socket says 172.21.15.5:59982) 2024-05-09T00:51:42.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.777+0000 7fb62243e700 1 -- 172.21.15.5:0/3335073540 learned_addr learned my addr 172.21.15.5:0/3335073540 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:51:42.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.778+0000 7fb621c3d700 1 --2- 172.21.15.5:0/3335073540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb61c070060 0x7fb61c119400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:42.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.778+0000 7fb621c3d700 1 -- 172.21.15.5:0/3335073540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb61c06f710 msgr2=0x7fb61c11d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:42.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.778+0000 7fb621c3d700 1 --2- 172.21.15.5:0/3335073540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb61c06f710 0x7fb61c11d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:42.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.778+0000 7fb621c3d700 1 -- 172.21.15.5:0/3335073540 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb61c11d9e0 msgr2=0x7fb61c119940 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T00:51:42.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.778+0000 7fb621c3d700 1 --2- 172.21.15.5:0/3335073540 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb61c11d9e0 0x7fb61c119940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:42.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.779+0000 7fb621c3d700 1 -- 172.21.15.5:0/3335073540 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb61c11a0f0 con 0x7fb61c070060 2024-05-09T00:51:42.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.779+0000 7fb621c3d700 1 --2- 172.21.15.5:0/3335073540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb61c070060 0x7fb61c119400 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fb60c01a8f0 tx=0x7fb60c029a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:51:42.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.781+0000 7fb6137fe700 1 -- 172.21.15.5:0/3335073540 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb60c036070 con 0x7fb61c070060 2024-05-09T00:51:42.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.781+0000 7fb6137fe700 1 -- 172.21.15.5:0/3335073540 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb60c0318c0 con 0x7fb61c070060 2024-05-09T00:51:42.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.781+0000 7fb6137fe700 1 -- 172.21.15.5:0/3335073540 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb60c01f3f0 con 0x7fb61c070060 2024-05-09T00:51:42.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.781+0000 7fb6246a2700 1 -- 172.21.15.5:0/3335073540 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb61c1ad8b0 con 0x7fb61c070060 2024-05-09T00:51:42.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.781+0000 7fb6246a2700 1 -- 172.21.15.5:0/3335073540 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb61c1adc50 con 0x7fb61c070060 2024-05-09T00:51:42.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.792+0000 7fb6330b8b80 1 Processor -- start 2024-05-09T00:51:42.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.792+0000 7fb6137fe700 1 -- 172.21.15.5:0/3335073540 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fb60c031410 con 0x7fb61c070060 2024-05-09T00:51:42.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.795+0000 7fb6330b8b80 1 -- start start 2024-05-09T00:51:42.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.795+0000 7fb6330b8b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55621bfc31e0 0x55621c05c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:42.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.795+0000 7fb6330b8b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55621bed9520 0x55621c05cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:42.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.795+0000 7fb6330b8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55621c056720 0x55621c056b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:42.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.795+0000 7fb6330b8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55621bf79070 con 0x55621c056720 2024-05-09T00:51:42.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.795+0000 7fb6330b8b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55621bf17870 con 0x55621bed9520 2024-05-09T00:51:42.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.795+0000 7fb6330b8b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55621bf75030 con 0x55621bfc31e0 2024-05-09T00:51:42.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.796+0000 7fb6137fe700 1 --2- 172.21.15.5:0/3335073540 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fb6080b2a50 0x7fb6080b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:42.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.796+0000 7fb621c3d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55621c056720 0x55621c056b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:42.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.796+0000 7fb621c3d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55621c056720 0x55621c056b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39220/0 (socket says 172.21.15.5:39220) 2024-05-09T00:51:42.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.796+0000 7fb621c3d700 1 -- 172.21.15.5:0/383908911 learned_addr learned my addr 172.21.15.5:0/383908911 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:51:42.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.796+0000 7fb621c3d700 1 -- 172.21.15.5:0/383908911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55621bfc31e0 msgr2=0x55621c05c570 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:51:42.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.796+0000 7fb621c3d700 1 --2- 172.21.15.5:0/383908911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55621bfc31e0 0x55621c05c570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:42.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.796+0000 7fb621c3d700 1 -- 172.21.15.5:0/383908911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55621bed9520 msgr2=0x55621c05cab0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:51:42.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.796+0000 7fb621c3d700 1 --2- 172.21.15.5:0/383908911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55621bed9520 0x55621c05cab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:42.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.796+0000 7fb621c3d700 1 -- 172.21.15.5:0/383908911 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55621bf0f3f0 con 0x55621c056720 2024-05-09T00:51:42.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.797+0000 7fb622c3f700 1 --2- 172.21.15.5:0/3335073540 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fb6080b2a50 0x7fb6080b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:42.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.797+0000 7fb621c3d700 1 --2- 172.21.15.5:0/383908911 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55621c056720 0x55621c056b50 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fb60c0f7ef0 tx=0x7fb60c0f7f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edc611dd9669488f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:51:42.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.797+0000 7fb6137fe700 1 -- 172.21.15.5:0/3335073540 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fb60c0f3ec0 con 0x7fb61c070060 2024-05-09T00:51:42.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.798+0000 7fb610ff9700 1 -- 172.21.15.5:0/383908911 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb60c0b73a0 con 0x55621c056720 2024-05-09T00:51:42.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.798+0000 7fb622c3f700 1 --2- 172.21.15.5:0/3335073540 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fb6080b2a50 0x7fb6080b4f60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb6140060b0 tx=0x7fb61400d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:51:42.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.798+0000 7fb610ff9700 1 -- 172.21.15.5:0/383908911 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb60c01a5e0 con 0x55621c056720 2024-05-09T00:51:42.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.798+0000 7fb610ff9700 1 -- 172.21.15.5:0/383908911 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb60c0bac50 con 0x55621c056720 2024-05-09T00:51:42.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.799+0000 7fb6330b8b80 1 -- 172.21.15.5:0/383908911 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55621c056720 msgr2=0x55621c056b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:42.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.799+0000 7fb6330b8b80 1 --2- 172.21.15.5:0/383908911 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55621c056720 0x55621c056b50 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fb60c0f7ef0 tx=0x7fb60c0f7f20 comp rx=0 tx=0).stop 2024-05-09T00:51:42.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.800+0000 7fb6330b8b80 1 -- 172.21.15.5:0/383908911 shutdown_connections 2024-05-09T00:51:42.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.800+0000 7fb6330b8b80 1 --2- 172.21.15.5:0/383908911 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55621c056720 0x55621c056b50 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:42.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.800+0000 7fb6330b8b80 1 --2- 172.21.15.5:0/383908911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55621bfc31e0 0x55621c05c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:42.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.800+0000 7fb6330b8b80 1 --2- 172.21.15.5:0/383908911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55621bed9520 0x55621c05cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:42.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.800+0000 7fb6330b8b80 1 -- 172.21.15.5:0/383908911 >> 172.21.15.5:0/383908911 conn(0x55621bf39720 msgr2=0x55621c051480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:51:42.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.800+0000 7fb6330b8b80 1 -- 172.21.15.5:0/383908911 shutdown_connections 2024-05-09T00:51:42.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.800+0000 7fb6330b8b80 1 -- 172.21.15.5:0/383908911 wait complete. 2024-05-09T00:51:42.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.801+0000 7fb6330b8b80 1 -- start start 2024-05-09T00:51:42.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.802+0000 7fb6330b8b80 1 Processor -- start 2024-05-09T00:51:42.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.802+0000 7fb6330b8b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55621bed9520 0x55621c065c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:42.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.802+0000 7fb6330b8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55621bfc31e0 0x55621c066150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:42.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.802+0000 7fb6330b8b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55621c056720 0x55621c060010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:42.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.802+0000 7fb6330b8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55621bdf4c20 con 0x55621bfc31e0 2024-05-09T00:51:42.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.802+0000 7fb6330b8b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55621bf331d0 con 0x55621c056720 2024-05-09T00:51:42.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.802+0000 7fb6330b8b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55621bf79070 con 0x55621bed9520 2024-05-09T00:51:42.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.803+0000 7fb6330b8b80 10 client.0 _ll_register_callbacks cb 0x55621bdea740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:51:42.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.806+0000 7fb621c3d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55621bfc31e0 0x55621c066150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:42.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.806+0000 7fb621c3d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55621bfc31e0 0x55621c066150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39230/0 (socket says 172.21.15.5:39230) 2024-05-09T00:51:42.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.806+0000 7fb621c3d700 1 -- 172.21.15.5:0/3461040829 learned_addr learned my addr 172.21.15.5:0/3461040829 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:51:42.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.806+0000 7fb622c3f700 1 --2- 172.21.15.5:0/3461040829 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55621c056720 0x55621c060010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:42.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.808+0000 7fb622c3f700 1 -- 172.21.15.5:0/3461040829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55621bed9520 msgr2=0x55621c065c10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:51:42.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.808+0000 7fb622c3f700 1 --2- 172.21.15.5:0/3461040829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55621bed9520 0x55621c065c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:42.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.808+0000 7fb622c3f700 1 -- 172.21.15.5:0/3461040829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55621bfc31e0 msgr2=0x55621c066150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:42.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.808+0000 7fb622c3f700 1 --2- 172.21.15.5:0/3461040829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55621bfc31e0 0x55621c066150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:42.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.808+0000 7fb622c3f700 1 -- 172.21.15.5:0/3461040829 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55621bdf4510 con 0x55621c056720 2024-05-09T00:51:42.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.814+0000 7fb622c3f700 1 --2- 172.21.15.5:0/3461040829 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55621c056720 0x55621c060010 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb614016ad0 tx=0x7fb614004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a483f97516da760 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:51:42.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.815+0000 7fb605ffb700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-09T00:51:42.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.816+0000 7fb605ffb700 1 -- 172.21.15.5:0/3461040829 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb614003a00 con 0x55621c056720 2024-05-09T00:51:42.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.816+0000 7fb605ffb700 1 -- 172.21.15.5:0/3461040829 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fb614003ba0 con 0x55621c056720 2024-05-09T00:51:42.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.816+0000 7fb6330b8b80 1 -- 172.21.15.5:0/3461040829 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb60c01a5e0 con 0x55621c056720 2024-05-09T00:51:42.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.818+0000 7fb605ffb700 1 -- 172.21.15.5:0/3461040829 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb614003da0 con 0x55621c056720 2024-05-09T00:51:42.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.818+0000 7fb6330b8b80 10 client.25598 fetch_fsmap learned FSMap version 30 2024-05-09T00:51:42.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.819+0000 7fb6330b8b80 1 -- 172.21.15.5:0/3461040829 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55621c06d8e0 con 0x55621c056720 2024-05-09T00:51:42.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.821+0000 7fb605ffb700 1 -- 172.21.15.5:0/3461040829 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb60c01a5e0 con 0x55621c056720 2024-05-09T00:51:42.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.823+0000 7fb605ffb700 1 -- 172.21.15.5:0/3461040829 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7fb614004610 con 0x55621c056720 2024-05-09T00:51:42.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.824+0000 7fb6330b8b80 10 client.25598 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T00:51:42.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.824+0000 7fb6330b8b80 10 client.25598 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:51:42.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.824+0000 7fb6330b8b80 20 client.25598 populate_metadata read hostname 'smithi005' 2024-05-09T00:51:42.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.824+0000 7fb6330b8b80 1 --2- 172.21.15.5:0/3461040829 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55621c06ff70 0x55621c0723b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:42.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.824+0000 7fb6330b8b80 4 client.25598 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:51:42.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.824+0000 7fb6330b8b80 1 -- 172.21.15.5:0/3461040829 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55621bf86a50 con 0x55621c06ff70 2024-05-09T00:51:42.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.825+0000 7fb62243e700 1 --2- 172.21.15.5:0/3461040829 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55621c06ff70 0x55621c0723b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:42.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.826+0000 7fb62243e700 1 --2- 172.21.15.5:0/3461040829 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55621c06ff70 0x55621c0723b0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=176051de0bbc9dcd server_cookie=aac7b7db10d2f2ac in_seq=0 out_seq=0 2024-05-09T00:51:42.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.826+0000 7fb605ffb700 10 client.25598 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:51:42.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.832+0000 7fb605ffb700 1 -- 172.21.15.5:0/3461040829 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55621bf86a50 con 0x55621c06ff70 2024-05-09T00:51:42.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.832+0000 7fb605ffb700 10 client.25598 handle_command_reply: tid=0 2024-05-09T00:51:42.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.832+0000 7fb6330b8b80 1 client.25598 shutdown 2024-05-09T00:51:42.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.833+0000 7fb6330b8b80 1 -- 172.21.15.5:0/3461040829 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55621c056720 msgr2=0x55621c060010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:42.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.833+0000 7fb6330b8b80 1 --2- 172.21.15.5:0/3461040829 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55621c056720 0x55621c060010 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb614016ad0 tx=0x7fb614004ad0 comp rx=0 tx=0).stop 2024-05-09T00:51:42.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.833+0000 7fb6330b8b80 1 -- 172.21.15.5:0/3461040829 shutdown_connections 2024-05-09T00:51:42.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.833+0000 7fb6330b8b80 1 --2- 172.21.15.5:0/3461040829 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55621c06ff70 0x55621c0723b0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:42.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.834+0000 7fb6330b8b80 1 --2- 172.21.15.5:0/3461040829 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55621c056720 0x55621c060010 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:42.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.834+0000 7fb6330b8b80 1 --2- 172.21.15.5:0/3461040829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55621bed9520 0x55621c065c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:42.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.834+0000 7fb6330b8b80 1 --2- 172.21.15.5:0/3461040829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55621bfc31e0 0x55621c066150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:42.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.834+0000 7fb6330b8b80 1 -- 172.21.15.5:0/3461040829 >> 172.21.15.5:0/3461040829 conn(0x55621bf39720 msgr2=0x55621c051240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:51:42.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.834+0000 7fb605ffb700 0 client.25598 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:51:42.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.834+0000 7fb6330b8b80 1 -- 172.21.15.5:0/3461040829 shutdown_connections 2024-05-09T00:51:42.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.834+0000 7fb6330b8b80 1 -- 172.21.15.5:0/3461040829 wait complete. 2024-05-09T00:51:42.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.834+0000 7fb6330b8b80 20 client.25598 trim_cache size 0 max 16384 2024-05-09T00:51:42.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.869+0000 7fb6330b8b80 1 Processor -- start 2024-05-09T00:51:42.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.870+0000 7fb6330b8b80 1 -- start start 2024-05-09T00:51:42.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.871+0000 7fb6330b8b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55621bed9520 0x55621c05ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:42.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.871+0000 7fb6330b8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55621bfc31e0 0x55621c05a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:42.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.871+0000 7fb6330b8b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55621c056720 0x55621c060580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:42.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.871+0000 7fb6330b8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb61c1b5110 con 0x55621bfc31e0 2024-05-09T00:51:42.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.871+0000 7fb6330b8b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb61c1b6c80 con 0x55621bed9520 2024-05-09T00:51:42.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.871+0000 7fb6330b8b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb61c1b57e0 con 0x55621c056720 2024-05-09T00:51:42.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.873+0000 7fb621c3d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55621bfc31e0 0x55621c05a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:42.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.873+0000 7fb621c3d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55621bfc31e0 0x55621c05a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39232/0 (socket says 172.21.15.5:39232) 2024-05-09T00:51:42.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.873+0000 7fb62243e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55621bed9520 0x55621c05ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:42.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.873+0000 7fb621c3d700 1 -- 172.21.15.5:0/1464298935 learned_addr learned my addr 172.21.15.5:0/1464298935 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:51:42.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.873+0000 7fb622c3f700 1 --2- 172.21.15.5:0/1464298935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55621c056720 0x55621c060580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:42.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.873+0000 7fb621c3d700 1 -- 172.21.15.5:0/1464298935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55621c056720 msgr2=0x55621c060580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:42.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.873+0000 7fb621c3d700 1 --2- 172.21.15.5:0/1464298935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55621c056720 0x55621c060580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:42.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.873+0000 7fb621c3d700 1 -- 172.21.15.5:0/1464298935 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55621bed9520 msgr2=0x55621c05ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:42.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.873+0000 7fb621c3d700 1 --2- 172.21.15.5:0/1464298935 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55621bed9520 0x55621c05ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:42.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.874+0000 7fb621c3d700 1 -- 172.21.15.5:0/1464298935 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55621c0576c0 con 0x55621bfc31e0 2024-05-09T00:51:42.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.875+0000 7fb621c3d700 1 --2- 172.21.15.5:0/1464298935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55621bfc31e0 0x55621c05a4f0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fb60c0f7ef0 tx=0x7fb60c01acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef3e1156ead44fb0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:51:42.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.876+0000 7fb607fff700 1 -- 172.21.15.5:0/1464298935 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb60c0bae30 con 0x55621bfc31e0 2024-05-09T00:51:42.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.876+0000 7fb607fff700 1 -- 172.21.15.5:0/1464298935 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb60c01a5e0 con 0x55621bfc31e0 2024-05-09T00:51:42.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.876+0000 7fb607fff700 1 -- 172.21.15.5:0/1464298935 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb60c0bb2b0 con 0x55621bfc31e0 2024-05-09T00:51:42.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.877+0000 7fb6330b8b80 1 -- 172.21.15.5:0/1464298935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55621bfc31e0 msgr2=0x55621c05a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:42.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.877+0000 7fb6330b8b80 1 --2- 172.21.15.5:0/1464298935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55621bfc31e0 0x55621c05a4f0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fb60c0f7ef0 tx=0x7fb60c01acb0 comp rx=0 tx=0).stop 2024-05-09T00:51:42.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.878+0000 7fb6330b8b80 1 -- 172.21.15.5:0/1464298935 shutdown_connections 2024-05-09T00:51:42.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.878+0000 7fb6330b8b80 1 --2- 172.21.15.5:0/1464298935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55621c056720 0x55621c060580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:42.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.878+0000 7fb6330b8b80 1 --2- 172.21.15.5:0/1464298935 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55621bed9520 0x55621c05ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:42.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.878+0000 7fb6330b8b80 1 --2- 172.21.15.5:0/1464298935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55621bfc31e0 0x55621c05a4f0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:42.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.878+0000 7fb6330b8b80 1 -- 172.21.15.5:0/1464298935 >> 172.21.15.5:0/1464298935 conn(0x55621bf39720 msgr2=0x55621c05d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:51:42.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.879+0000 7fb6330b8b80 1 -- 172.21.15.5:0/1464298935 shutdown_connections 2024-05-09T00:51:42.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.881+0000 7fb6330b8b80 1 -- 172.21.15.5:0/1464298935 wait complete. 2024-05-09T00:51:42.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.882+0000 7fb6330b8b80 1 -- start start 2024-05-09T00:51:42.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.883+0000 7fb6330b8b80 1 Processor -- start 2024-05-09T00:51:42.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.884+0000 7fb6330b8b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55621bed9520 0x55621c069ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:42.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.884+0000 7fb6330b8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55621c056720 0x55621c06a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:42.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.884+0000 7fb6330b8b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55621c06a960 0x55621c0d9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:42.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.884+0000 7fb62243e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55621bed9520 0x55621c069ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:42.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.884+0000 7fb62243e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55621bed9520 0x55621c069ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:60000/0 (socket says 172.21.15.5:60000) 2024-05-09T00:51:42.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.884+0000 7fb62243e700 1 -- 172.21.15.5:0/31065059 learned_addr learned my addr 172.21.15.5:0/31065059 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:51:42.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.884+0000 7fb6330b8b80 1 -- 172.21.15.5:0/31065059 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb61c1b6580 con 0x55621c056720 2024-05-09T00:51:42.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.885+0000 7fb6330b8b80 1 -- 172.21.15.5:0/31065059 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb61c1b5eb0 con 0x55621c06a960 2024-05-09T00:51:42.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.885+0000 7fb6330b8b80 1 -- 172.21.15.5:0/31065059 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb61c1b3160 con 0x55621bed9520 2024-05-09T00:51:42.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.886+0000 7fb62243e700 1 -- 172.21.15.5:0/31065059 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55621c06a960 msgr2=0x55621c0d9770 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:51:42.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.886+0000 7fb62243e700 1 --2- 172.21.15.5:0/31065059 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55621c06a960 0x55621c0d9770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:42.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.886+0000 7fb62243e700 1 -- 172.21.15.5:0/31065059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55621c056720 msgr2=0x55621c06a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:42.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.886+0000 7fb62243e700 1 --2- 172.21.15.5:0/31065059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55621c056720 0x55621c06a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:42.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.886+0000 7fb62243e700 1 -- 172.21.15.5:0/31065059 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55621c0d9db0 con 0x55621bed9520 2024-05-09T00:51:42.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.887+0000 7fb62243e700 1 --2- 172.21.15.5:0/31065059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55621bed9520 0x55621c069ee0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x55621c052f50 tx=0x7fb618010710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4cc134b30e0805 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:51:42.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.888+0000 7fb6330b8b80 10 client.0 _ll_register_callbacks cb 0x55621bdea740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:51:42.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.888+0000 7fb6330b8b80 1 -- 172.21.15.5:0/31065059 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb60c01a5e0 con 0x55621bed9520 2024-05-09T00:51:42.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.888+0000 7fb605ffb700 10 client.25624 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-09T00:51:42.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.888+0000 7fb605ffb700 1 -- 172.21.15.5:0/31065059 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb61801d070 con 0x55621bed9520 2024-05-09T00:51:42.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.889+0000 7fb605ffb700 1 -- 172.21.15.5:0/31065059 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fb618019490 con 0x55621bed9520 2024-05-09T00:51:42.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.889+0000 7fb605ffb700 1 -- 172.21.15.5:0/31065059 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb618028030 con 0x55621bed9520 2024-05-09T00:51:42.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.889+0000 7fb6330b8b80 10 client.25624 fetch_fsmap learned FSMap version 30 2024-05-09T00:51:42.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.889+0000 7fb6330b8b80 1 -- 172.21.15.5:0/31065059 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({fsmap=30}) -- 0x55621c0dafb0 con 0x55621bed9520 2024-05-09T00:51:42.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.890+0000 7fb605ffb700 1 -- 172.21.15.5:0/31065059 <== mon.1 v2:172.21.15.157:3300/0 4 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7fb618019690 con 0x55621bed9520 2024-05-09T00:51:42.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.890+0000 7fb6330b8b80 10 client.25624 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T00:51:42.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.890+0000 7fb6330b8b80 10 client.25624 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:51:42.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.891+0000 7fb6330b8b80 20 client.25624 populate_metadata read hostname 'smithi005' 2024-05-09T00:51:42.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.891+0000 7fb6330b8b80 1 --2- 172.21.15.5:0/31065059 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55621c0db610 0x55621c0dda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:42.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.891+0000 7fb6330b8b80 4 client.25624 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T00:51:42.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.891+0000 7fb6330b8b80 1 -- 172.21.15.5:0/31065059 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fb60c01a5e0 con 0x55621c0db610 2024-05-09T00:51:42.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.891+0000 7fb621c3d700 1 --2- 172.21.15.5:0/31065059 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55621c0db610 0x55621c0dda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:42.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.892+0000 7fb621c3d700 1 --2- 172.21.15.5:0/31065059 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55621c0db610 0x55621c0dda90 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=401856717b461801 server_cookie=4488a15dba130c2 in_seq=0 out_seq=0 2024-05-09T00:51:42.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:42.892+0000 7fb605ffb700 10 client.25624 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:51:43.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:51:43 smithi005 ceph-mon[30365]: pgmap v532: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 845 MiB/s wr, 1.38k op/s 2024-05-09T00:51:43.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:51:43 smithi131 ceph-mon[34547]: pgmap v532: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 845 MiB/s wr, 1.38k op/s 2024-05-09T00:51:43.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:51:43 smithi157 ceph-mon[34294]: pgmap v532: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 845 MiB/s wr, 1.38k op/s 2024-05-09T00:51:44.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:44.441+0000 7fb605ffb700 1 -- 172.21.15.5:0/31065059 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fb60c01a5e0 con 0x55621c0db610 2024-05-09T00:51:44.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:44.442+0000 7fb605ffb700 10 client.25624 handle_command_reply: tid=0 2024-05-09T00:51:44.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:44.442+0000 7fb6330b8b80 1 client.25624 shutdown 2024-05-09T00:51:44.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:44.442+0000 7fb6330b8b80 1 -- 172.21.15.5:0/31065059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55621bed9520 msgr2=0x55621c069ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:44.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:44.442+0000 7fb6330b8b80 1 --2- 172.21.15.5:0/31065059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55621bed9520 0x55621c069ee0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x55621c052f50 tx=0x7fb618010710 comp rx=0 tx=0).stop 2024-05-09T00:51:44.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:44.442+0000 7fb6330b8b80 1 -- 172.21.15.5:0/31065059 shutdown_connections 2024-05-09T00:51:44.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:44.442+0000 7fb6330b8b80 1 --2- 172.21.15.5:0/31065059 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55621c0db610 0x55621c0dda90 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:44.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:44.443+0000 7fb6330b8b80 1 --2- 172.21.15.5:0/31065059 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55621c06a960 0x55621c0d9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:44.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:44.443+0000 7fb605ffb700 0 client.25624 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:51:44.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:44.443+0000 7fb6330b8b80 1 --2- 172.21.15.5:0/31065059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55621bed9520 0x55621c069ee0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:44.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:44.443+0000 7fb6330b8b80 1 --2- 172.21.15.5:0/31065059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55621c056720 0x55621c06a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:44.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:44.443+0000 7fb6330b8b80 1 -- 172.21.15.5:0/31065059 >> 172.21.15.5:0/31065059 conn(0x55621bf39720 msgr2=0x55621c05b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:51:44.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:44.444+0000 7fb6330b8b80 1 -- 172.21.15.5:0/31065059 shutdown_connections 2024-05-09T00:51:44.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:44.444+0000 7fb6330b8b80 1 -- 172.21.15.5:0/31065059 wait complete. 2024-05-09T00:51:44.446 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-09T00:51:44.447 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-05-09T00:51:44.447 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "92d24500-4efe-4208-8190-01302b541b47", 2024-05-09T00:51:44.447 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-05-09T00:51:44.447 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-09T00:51:44.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:44.444+0000 7fb6330b8b80 20 client.25624 trim_cache size 0 max 16384 2024-05-09T00:51:44.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:44.447+0000 7fb6246a2700 1 -- 172.21.15.5:0/3335073540 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fb6080b2a50 msgr2=0x7fb6080b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:44.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:44.447+0000 7fb6246a2700 1 --2- 172.21.15.5:0/3335073540 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fb6080b2a50 0x7fb6080b4f60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb6140060b0 tx=0x7fb61400d040 comp rx=0 tx=0).stop 2024-05-09T00:51:44.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:44.447+0000 7fb6246a2700 1 -- 172.21.15.5:0/3335073540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb61c070060 msgr2=0x7fb61c119400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:44.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:44.447+0000 7fb6246a2700 1 --2- 172.21.15.5:0/3335073540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb61c070060 0x7fb61c119400 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fb60c01a8f0 tx=0x7fb60c029a20 comp rx=0 tx=0).stop 2024-05-09T00:51:44.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:44.450+0000 7fb6246a2700 1 -- 172.21.15.5:0/3335073540 shutdown_connections 2024-05-09T00:51:44.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:44.450+0000 7fb6246a2700 1 --2- 172.21.15.5:0/3335073540 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fb6080b2a50 0x7fb6080b4f60 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:44.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:44.450+0000 7fb6246a2700 1 --2- 172.21.15.5:0/3335073540 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb61c11d9e0 0x7fb61c119940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:44.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:44.450+0000 7fb6246a2700 1 --2- 172.21.15.5:0/3335073540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb61c06f710 0x7fb61c11d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:44.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:44.450+0000 7fb6246a2700 1 --2- 172.21.15.5:0/3335073540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb61c070060 0x7fb61c119400 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:44.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:44.450+0000 7fb6246a2700 1 -- 172.21.15.5:0/3335073540 >> 172.21.15.5:0/3335073540 conn(0x7fb61c10ab60 msgr2=0x7fb61c0729a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:51:44.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:44.452+0000 7fb6246a2700 1 -- 172.21.15.5:0/3335073540 shutdown_connections 2024-05-09T00:51:44.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:44.452+0000 7fb6246a2700 1 -- 172.21.15.5:0/3335073540 wait complete. 2024-05-09T00:51:45.527 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:51:45 smithi005 ceph-mon[30365]: pgmap v533: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 895 MiB/s wr, 1.61k op/s 2024-05-09T00:51:45.527 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:51:45 smithi005 ceph-mon[30365]: scrub queued for path: / 2024-05-09T00:51:45.527 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:51:45 smithi005 ceph-mon[30365]: scrub summary: idle+waiting paths [/] 2024-05-09T00:51:45.527 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:51:45 smithi005 ceph-mon[30365]: scrub summary: active paths [/] 2024-05-09T00:51:45.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:51:45 smithi131 ceph-mon[34547]: pgmap v533: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 895 MiB/s wr, 1.61k op/s 2024-05-09T00:51:45.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:51:45 smithi131 ceph-mon[34547]: scrub queued for path: / 2024-05-09T00:51:45.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:51:45 smithi131 ceph-mon[34547]: scrub summary: idle+waiting paths [/] 2024-05-09T00:51:45.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:51:45 smithi131 ceph-mon[34547]: scrub summary: active paths [/] 2024-05-09T00:51:45.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:51:45 smithi157 ceph-mon[34294]: pgmap v533: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 895 MiB/s wr, 1.61k op/s 2024-05-09T00:51:45.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:51:45 smithi157 ceph-mon[34294]: scrub queued for path: / 2024-05-09T00:51:45.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:51:45 smithi157 ceph-mon[34294]: scrub summary: idle+waiting paths [/] 2024-05-09T00:51:45.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:51:45 smithi157 ceph-mon[34294]: scrub summary: active paths [/] 2024-05-09T00:51:46.436 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T00:51:47.089 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:51:47.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:51:47 smithi005 ceph-mon[30365]: pgmap v534: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 950 MiB/s wr, 1.88k op/s 2024-05-09T00:51:47.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:51:47 smithi131 ceph-mon[34547]: pgmap v534: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 950 MiB/s wr, 1.88k op/s 2024-05-09T00:51:47.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:51:47 smithi157 ceph-mon[34294]: pgmap v534: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 950 MiB/s wr, 1.88k op/s 2024-05-09T00:51:49.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:51:49 smithi005 ceph-mon[30365]: scrub summary: active paths [/] 2024-05-09T00:51:49.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:51:49 smithi005 ceph-mon[30365]: scrub summary: idle+waiting paths [/] 2024-05-09T00:51:49.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:51:49 smithi005 ceph-mon[30365]: pgmap v535: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 827 MiB/s wr, 1.68k op/s 2024-05-09T00:51:49.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:51:49 smithi131 ceph-mon[34547]: scrub summary: active paths [/] 2024-05-09T00:51:49.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:51:49 smithi131 ceph-mon[34547]: scrub summary: idle+waiting paths [/] 2024-05-09T00:51:49.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:51:49 smithi131 ceph-mon[34547]: pgmap v535: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 827 MiB/s wr, 1.68k op/s 2024-05-09T00:51:49.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:51:49 smithi157 ceph-mon[34294]: scrub summary: active paths [/] 2024-05-09T00:51:49.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:51:49 smithi157 ceph-mon[34294]: scrub summary: idle+waiting paths [/] 2024-05-09T00:51:49.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:51:49 smithi157 ceph-mon[34294]: pgmap v535: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 827 MiB/s wr, 1.68k op/s 2024-05-09T00:51:51.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:51:51 smithi005 ceph-mon[30365]: pgmap v536: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1004 MiB/s wr, 2.05k op/s 2024-05-09T00:51:51.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:51:51 smithi131 ceph-mon[34547]: pgmap v536: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1004 MiB/s wr, 2.05k op/s 2024-05-09T00:51:51.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:51:51 smithi157 ceph-mon[34294]: pgmap v536: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1004 MiB/s wr, 2.05k op/s 2024-05-09T00:51:52.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.617+0000 7fdbd759e700 1 -- 172.21.15.5:0/2450443328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbd8108d10 msgr2=0x7fdbd8109120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:52.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.617+0000 7fdbd759e700 1 --2- 172.21.15.5:0/2450443328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbd8108d10 0x7fdbd8109120 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc009990 tx=0x7fdbcc009ca0 comp rx=0 tx=0).stop 2024-05-09T00:51:52.619 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:51:52 smithi005 ceph-mon[30365]: scrub summary: idle+waiting paths [/] 2024-05-09T00:51:52.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.619+0000 7fdbd759e700 1 -- 172.21.15.5:0/2450443328 shutdown_connections 2024-05-09T00:51:52.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.620+0000 7fdbd759e700 1 --2- 172.21.15.5:0/2450443328 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdbd8109660 0x7fdbd8110fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:52.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.620+0000 7fdbd759e700 1 --2- 172.21.15.5:0/2450443328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbd81043e0 0x7fdbd81047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:52.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.620+0000 7fdbd759e700 1 --2- 172.21.15.5:0/2450443328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbd8108d10 0x7fdbd8109120 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:52.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.620+0000 7fdbd759e700 1 -- 172.21.15.5:0/2450443328 >> 172.21.15.5:0/2450443328 conn(0x7fdbd8079d10 msgr2=0x7fdbd807a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:51:52.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.621+0000 7fdbd759e700 1 -- 172.21.15.5:0/2450443328 shutdown_connections 2024-05-09T00:51:52.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.621+0000 7fdbd759e700 1 -- 172.21.15.5:0/2450443328 wait complete. 2024-05-09T00:51:52.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.623+0000 7fdbd759e700 1 Processor -- start 2024-05-09T00:51:52.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.624+0000 7fdbd759e700 1 -- start start 2024-05-09T00:51:52.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.625+0000 7fdbd759e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbd81043e0 0x7fdbd819d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:52.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.625+0000 7fdbd759e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbd8108d10 0x7fdbd819d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:52.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.625+0000 7fdbd759e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdbd8109660 0x7fdbd81a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:52.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.625+0000 7fdbd759e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd8116610 con 0x7fdbd81043e0 2024-05-09T00:51:52.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.625+0000 7fdbd759e700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd8116490 con 0x7fdbd8109660 2024-05-09T00:51:52.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.625+0000 7fdbd759e700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd8116790 con 0x7fdbd8108d10 2024-05-09T00:51:52.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.626+0000 7fdbd659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbd81043e0 0x7fdbd819d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:52.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.626+0000 7fdbd659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbd81043e0 0x7fdbd819d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46256/0 (socket says 172.21.15.5:46256) 2024-05-09T00:51:52.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.626+0000 7fdbd659c700 1 -- 172.21.15.5:0/1708613109 learned_addr learned my addr 172.21.15.5:0/1708613109 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:51:52.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.626+0000 7fdbd6d9d700 1 --2- 172.21.15.5:0/1708613109 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdbd8109660 0x7fdbd81a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:52.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.626+0000 7fdbd659c700 1 -- 172.21.15.5:0/1708613109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbd8108d10 msgr2=0x7fdbd819d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:52.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.627+0000 7fdbd659c700 1 --2- 172.21.15.5:0/1708613109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbd8108d10 0x7fdbd819d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:52.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.627+0000 7fdbd659c700 1 -- 172.21.15.5:0/1708613109 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdbd8109660 msgr2=0x7fdbd81a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:52.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.627+0000 7fdbd659c700 1 --2- 172.21.15.5:0/1708613109 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdbd8109660 0x7fdbd81a1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:52.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.627+0000 7fdbd659c700 1 -- 172.21.15.5:0/1708613109 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbd81a1ed0 con 0x7fdbd81043e0 2024-05-09T00:51:52.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.627+0000 7fdbd659c700 1 --2- 172.21.15.5:0/1708613109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbd81043e0 0x7fdbd819d000 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fdbc800b530 tx=0x7fdbc800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:51:52.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.627+0000 7fdbc77fe700 1 -- 172.21.15.5:0/1708613109 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc8015020 con 0x7fdbd81043e0 2024-05-09T00:51:52.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.628+0000 7fdbc77fe700 1 -- 172.21.15.5:0/1708613109 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbc8011ce0 con 0x7fdbd81043e0 2024-05-09T00:51:52.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.628+0000 7fdbd759e700 1 -- 172.21.15.5:0/1708613109 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbd81a21c0 con 0x7fdbd81043e0 2024-05-09T00:51:52.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.628+0000 7fdbd759e700 1 -- 172.21.15.5:0/1708613109 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbd81a9ae0 con 0x7fdbd81043e0 2024-05-09T00:51:52.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.628+0000 7fdbc77fe700 1 -- 172.21.15.5:0/1708613109 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc8010d20 con 0x7fdbd81043e0 2024-05-09T00:51:52.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.638+0000 7fdbc77fe700 1 -- 172.21.15.5:0/1708613109 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fdbc8018020 con 0x7fdbd81043e0 2024-05-09T00:51:52.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.641+0000 7fdbc77fe700 1 --2- 172.21.15.5:0/1708613109 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fdbc00b2940 0x7fdbc00b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:52.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.642+0000 7fdbd5d9b700 1 --2- 172.21.15.5:0/1708613109 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fdbc00b2940 0x7fdbc00b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:52.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.642+0000 7fdbc77fe700 1 -- 172.21.15.5:0/1708613109 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fdbc80e19c0 con 0x7fdbd81043e0 2024-05-09T00:51:52.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.642+0000 7fdbeb9ddb80 1 Processor -- start 2024-05-09T00:51:52.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.643+0000 7fdbeb9ddb80 1 -- start start 2024-05-09T00:51:52.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.643+0000 7fdbeb9ddb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5566b5dbad00 0x5566b5ebda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:52.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.643+0000 7fdbeb9ddb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566b5d404a0 0x5566b5ebdf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:52.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.643+0000 7fdbeb9ddb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566b5ec7540 0x5566b5ec9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:52.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.644+0000 7fdbeb9ddb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5566b5de00a0 con 0x5566b5ec7540 2024-05-09T00:51:52.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.644+0000 7fdbeb9ddb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5566b5d859a0 con 0x5566b5dbad00 2024-05-09T00:51:52.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.644+0000 7fdbeb9ddb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5566b5ddc060 con 0x5566b5d404a0 2024-05-09T00:51:52.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.645+0000 7fdbd6d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566b5ec7540 0x5566b5ec9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:52.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.645+0000 7fdbd6d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566b5ec7540 0x5566b5ec9960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46260/0 (socket says 172.21.15.5:46260) 2024-05-09T00:51:52.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.645+0000 7fdbd6d9d700 1 -- 172.21.15.5:0/1200536693 learned_addr learned my addr 172.21.15.5:0/1200536693 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:51:52.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.646+0000 7fdbd5d9b700 1 --2- 172.21.15.5:0/1708613109 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fdbc00b2940 0x7fdbc00b4e50 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc00b5c0 tx=0x7fdbcc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:51:52.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.646+0000 7fdbd659c700 1 --2- 172.21.15.5:0/1200536693 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5566b5dbad00 0x5566b5ebda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:52.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.647+0000 7fdbd5d9b700 1 --2- 172.21.15.5:0/1200536693 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566b5d404a0 0x5566b5ebdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:52.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.647+0000 7fdbd6d9d700 1 -- 172.21.15.5:0/1200536693 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566b5d404a0 msgr2=0x5566b5ebdf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:52.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.647+0000 7fdbd6d9d700 1 --2- 172.21.15.5:0/1200536693 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566b5d404a0 0x5566b5ebdf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:52.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.648+0000 7fdbd6d9d700 1 -- 172.21.15.5:0/1200536693 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5566b5dbad00 msgr2=0x5566b5ebda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:52.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.648+0000 7fdbd6d9d700 1 --2- 172.21.15.5:0/1200536693 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5566b5dbad00 0x5566b5ebda30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:52.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.648+0000 7fdbd6d9d700 1 -- 172.21.15.5:0/1200536693 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566b5d76500 con 0x5566b5ec7540 2024-05-09T00:51:52.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.649+0000 7fdbd659c700 1 --2- 172.21.15.5:0/1200536693 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5566b5dbad00 0x5566b5ebda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:51:52.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.650+0000 7fdbd6d9d700 1 --2- 172.21.15.5:0/1200536693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566b5ec7540 0x5566b5ec9960 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0009f40 tx=0x7fdbd000c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdd5b986bdbb8c88 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:51:52.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.650+0000 7fdbc4ff9700 1 -- 172.21.15.5:0/1200536693 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd001a070 con 0x5566b5ec7540 2024-05-09T00:51:52.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.651+0000 7fdbc4ff9700 1 -- 172.21.15.5:0/1200536693 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbd00158c0 con 0x5566b5ec7540 2024-05-09T00:51:52.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.651+0000 7fdbc4ff9700 1 -- 172.21.15.5:0/1200536693 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd001a070 con 0x5566b5ec7540 2024-05-09T00:51:52.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.652+0000 7fdbeb9ddb80 1 -- 172.21.15.5:0/1200536693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566b5ec7540 msgr2=0x5566b5ec9960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:52.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.652+0000 7fdbeb9ddb80 1 --2- 172.21.15.5:0/1200536693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566b5ec7540 0x5566b5ec9960 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0009f40 tx=0x7fdbd000c6e0 comp rx=0 tx=0).stop 2024-05-09T00:51:52.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.653+0000 7fdbeb9ddb80 1 -- 172.21.15.5:0/1200536693 shutdown_connections 2024-05-09T00:51:52.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.653+0000 7fdbeb9ddb80 1 --2- 172.21.15.5:0/1200536693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566b5ec7540 0x5566b5ec9960 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:52.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.653+0000 7fdbeb9ddb80 1 --2- 172.21.15.5:0/1200536693 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5566b5dbad00 0x5566b5ebda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:52.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.653+0000 7fdbeb9ddb80 1 --2- 172.21.15.5:0/1200536693 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566b5d404a0 0x5566b5ebdf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:52.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.653+0000 7fdbeb9ddb80 1 -- 172.21.15.5:0/1200536693 >> 172.21.15.5:0/1200536693 conn(0x5566b5d43f70 msgr2=0x5566b5eb8760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:51:52.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.653+0000 7fdbeb9ddb80 1 -- 172.21.15.5:0/1200536693 shutdown_connections 2024-05-09T00:51:52.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.653+0000 7fdbeb9ddb80 1 -- 172.21.15.5:0/1200536693 wait complete. 2024-05-09T00:51:52.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.655+0000 7fdbeb9ddb80 1 -- start start 2024-05-09T00:51:52.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.655+0000 7fdbeb9ddb80 1 Processor -- start 2024-05-09T00:51:52.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.656+0000 7fdbeb9ddb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566b5d404a0 0x5566b5ec43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:52.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.656+0000 7fdbeb9ddb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566b5dbad00 0x5566b5ec48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:52.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.656+0000 7fdbeb9ddb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5566b5ec7540 0x5566b5ec4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:52.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.656+0000 7fdbeb9ddb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5566b5ec9fc0 con 0x5566b5dbad00 2024-05-09T00:51:52.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.656+0000 7fdbeb9ddb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5566b5d7ab20 con 0x5566b5ec7540 2024-05-09T00:51:52.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.656+0000 7fdbeb9ddb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5566b5eca140 con 0x5566b5d404a0 2024-05-09T00:51:52.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.657+0000 7fdbeb9ddb80 10 client.0 _ll_register_callbacks cb 0x5566b5c516c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:51:52.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.659+0000 7fdbd659c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566b5d404a0 0x5566b5ec43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:52.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.659+0000 7fdbd659c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566b5d404a0 0x5566b5ec43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:48054/0 (socket says 172.21.15.5:48054) 2024-05-09T00:51:52.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.659+0000 7fdbd659c700 1 -- 172.21.15.5:0/3602866237 learned_addr learned my addr 172.21.15.5:0/3602866237 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:51:52.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.659+0000 7fdbd5d9b700 1 --2- 172.21.15.5:0/3602866237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566b5dbad00 0x5566b5ec48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:52.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.660+0000 7fdbd5d9b700 1 -- 172.21.15.5:0/3602866237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566b5d404a0 msgr2=0x5566b5ec43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:52.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.660+0000 7fdbd5d9b700 1 --2- 172.21.15.5:0/3602866237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566b5d404a0 0x5566b5ec43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:52.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.660+0000 7fdbd5d9b700 1 -- 172.21.15.5:0/3602866237 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5566b5ec7540 msgr2=0x5566b5ec4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:52.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.660+0000 7fdbd6d9d700 1 --2- 172.21.15.5:0/3602866237 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5566b5ec7540 0x5566b5ec4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:52.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.660+0000 7fdbd5d9b700 1 --2- 172.21.15.5:0/3602866237 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5566b5ec7540 0x5566b5ec4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:52.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.660+0000 7fdbd5d9b700 1 -- 172.21.15.5:0/3602866237 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566b5c5b490 con 0x5566b5dbad00 2024-05-09T00:51:52.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.661+0000 7fdbd6d9d700 1 --2- 172.21.15.5:0/3602866237 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5566b5ec7540 0x5566b5ec4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:51:52.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.662+0000 7fdbd5d9b700 1 --2- 172.21.15.5:0/3602866237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566b5dbad00 0x5566b5ec48e0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc0098e0 tx=0x7fdbcc009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e0f8c5a45525927 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:51:52.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.663+0000 7fdbbdffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:51:52.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.663+0000 7fdbbdffb700 1 -- 172.21.15.5:0/3602866237 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc00bdc0 con 0x5566b5dbad00 2024-05-09T00:51:52.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.663+0000 7fdbbdffb700 1 -- 172.21.15.5:0/3602866237 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fdbcc01e070 con 0x5566b5dbad00 2024-05-09T00:51:52.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.663+0000 7fdbeb9ddb80 1 -- 172.21.15.5:0/3602866237 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbd00158c0 con 0x5566b5dbad00 2024-05-09T00:51:52.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.665+0000 7fdbbdffb700 1 -- 172.21.15.5:0/3602866237 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdbcc02f420 con 0x5566b5dbad00 2024-05-09T00:51:52.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.665+0000 7fdbeb9ddb80 10 client.16077 fetch_fsmap learned FSMap version 30 2024-05-09T00:51:52.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.665+0000 7fdbeb9ddb80 1 -- 172.21.15.5:0/3602866237 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5566b5eee2a0 con 0x5566b5dbad00 2024-05-09T00:51:52.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.666+0000 7fdbbdffb700 1 -- 172.21.15.5:0/3602866237 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbd00158c0 con 0x5566b5dbad00 2024-05-09T00:51:52.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.666+0000 7fdbbdffb700 1 -- 172.21.15.5:0/3602866237 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7fdbcc038020 con 0x5566b5dbad00 2024-05-09T00:51:52.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.667+0000 7fdbeb9ddb80 10 client.16077 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T00:51:52.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.667+0000 7fdbeb9ddb80 10 client.16077 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:51:52.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.667+0000 7fdbeb9ddb80 20 client.16077 populate_metadata read hostname 'smithi005' 2024-05-09T00:51:52.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.667+0000 7fdbeb9ddb80 1 --2- 172.21.15.5:0/3602866237 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5566b5ef08e0 0x5566b5ef2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:52.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.667+0000 7fdbeb9ddb80 4 client.16077 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:51:52.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.667+0000 7fdbeb9ddb80 1 -- 172.21.15.5:0/3602866237 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566b5deda80 con 0x5566b5ef08e0 2024-05-09T00:51:52.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.670+0000 7fdbd659c700 1 --2- 172.21.15.5:0/3602866237 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5566b5ef08e0 0x5566b5ef2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:52.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.673+0000 7fdbd659c700 1 --2- 172.21.15.5:0/3602866237 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5566b5ef08e0 0x5566b5ef2d00 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fdb584ef3995cc2 server_cookie=173a61b416dfb382 in_seq=0 out_seq=0 2024-05-09T00:51:52.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.674+0000 7fdbbdffb700 10 client.16077 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:51:52.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.678+0000 7fdbbdffb700 1 -- 172.21.15.5:0/3602866237 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5566b5deda80 con 0x5566b5ef08e0 2024-05-09T00:51:52.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.678+0000 7fdbbdffb700 10 client.16077 handle_command_reply: tid=0 2024-05-09T00:51:52.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.678+0000 7fdbeb9ddb80 1 client.16077 shutdown 2024-05-09T00:51:52.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.678+0000 7fdbeb9ddb80 1 -- 172.21.15.5:0/3602866237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566b5dbad00 msgr2=0x5566b5ec48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:52.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.678+0000 7fdbeb9ddb80 1 --2- 172.21.15.5:0/3602866237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566b5dbad00 0x5566b5ec48e0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc0098e0 tx=0x7fdbcc009310 comp rx=0 tx=0).stop 2024-05-09T00:51:52.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.679+0000 7fdbeb9ddb80 1 -- 172.21.15.5:0/3602866237 shutdown_connections 2024-05-09T00:51:52.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.679+0000 7fdbeb9ddb80 1 --2- 172.21.15.5:0/3602866237 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5566b5ef08e0 0x5566b5ef2d00 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:52.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.679+0000 7fdbeb9ddb80 1 --2- 172.21.15.5:0/3602866237 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5566b5ec7540 0x5566b5ec4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:52.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.679+0000 7fdbeb9ddb80 1 --2- 172.21.15.5:0/3602866237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566b5d404a0 0x5566b5ec43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:52.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.679+0000 7fdbeb9ddb80 1 --2- 172.21.15.5:0/3602866237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566b5dbad00 0x5566b5ec48e0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:52.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.679+0000 7fdbeb9ddb80 1 -- 172.21.15.5:0/3602866237 >> 172.21.15.5:0/3602866237 conn(0x5566b5d43f70 msgr2=0x5566b5eb8250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:51:52.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.679+0000 7fdbbdffb700 0 client.16077 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:51:52.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.679+0000 7fdbeb9ddb80 1 -- 172.21.15.5:0/3602866237 shutdown_connections 2024-05-09T00:51:52.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.679+0000 7fdbeb9ddb80 1 -- 172.21.15.5:0/3602866237 wait complete. 2024-05-09T00:51:52.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.679+0000 7fdbeb9ddb80 20 client.16077 trim_cache size 0 max 16384 2024-05-09T00:51:52.685 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:51:52 smithi157 ceph-mon[34294]: scrub summary: idle+waiting paths [/] 2024-05-09T00:51:52.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.732+0000 7fdbeb9ddb80 1 Processor -- start 2024-05-09T00:51:52.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.732+0000 7fdbeb9ddb80 1 -- start start 2024-05-09T00:51:52.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.733+0000 7fdbeb9ddb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566b5d404a0 0x5566b5ec4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:52.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.733+0000 7fdbeb9ddb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5566b5dbad00 0x5566b5ef84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:52.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.734+0000 7fdbeb9ddb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566b5ec7540 0x5566b5ef8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:52.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.734+0000 7fdbeb9ddb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb800d4f0 con 0x5566b5ec7540 2024-05-09T00:51:52.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.734+0000 7fdbeb9ddb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb800f060 con 0x5566b5dbad00 2024-05-09T00:51:52.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.734+0000 7fdbeb9ddb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb800dbc0 con 0x5566b5d404a0 2024-05-09T00:51:52.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.735+0000 7fdbd5d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5566b5dbad00 0x5566b5ef84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:52.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.735+0000 7fdbd5d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5566b5dbad00 0x5566b5ef84e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:59306/0 (socket says 172.21.15.5:59306) 2024-05-09T00:51:52.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.735+0000 7fdbd5d9b700 1 -- 172.21.15.5:0/3842169020 learned_addr learned my addr 172.21.15.5:0/3842169020 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:51:52.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.737+0000 7fdbd659c700 1 --2- 172.21.15.5:0/3842169020 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566b5d404a0 0x5566b5ec4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:52.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.738+0000 7fdbd6d9d700 1 --2- 172.21.15.5:0/3842169020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566b5ec7540 0x5566b5ef8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:52.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.738+0000 7fdbd5d9b700 1 -- 172.21.15.5:0/3842169020 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566b5d404a0 msgr2=0x5566b5ec4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:52.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.738+0000 7fdbd5d9b700 1 --2- 172.21.15.5:0/3842169020 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566b5d404a0 0x5566b5ec4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:52.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.738+0000 7fdbd5d9b700 1 -- 172.21.15.5:0/3842169020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566b5ec7540 msgr2=0x5566b5ef8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:52.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.738+0000 7fdbd5d9b700 1 --2- 172.21.15.5:0/3842169020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566b5ec7540 0x5566b5ef8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:52.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.738+0000 7fdbd5d9b700 1 -- 172.21.15.5:0/3842169020 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566b5eb9050 con 0x5566b5dbad00 2024-05-09T00:51:52.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.740+0000 7fdbd5d9b700 1 --2- 172.21.15.5:0/3842169020 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5566b5dbad00 0x5566b5ef84e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc005d70 tx=0x7fdbcc0227e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9f0b8c9ee289ae7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:51:52.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.744+0000 7fdbbffff700 1 -- 172.21.15.5:0/3842169020 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc03d020 con 0x5566b5dbad00 2024-05-09T00:51:52.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.745+0000 7fdbbffff700 1 -- 172.21.15.5:0/3842169020 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbcc035070 con 0x5566b5dbad00 2024-05-09T00:51:52.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.746+0000 7fdbeb9ddb80 1 -- 172.21.15.5:0/3842169020 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5566b5dbad00 msgr2=0x5566b5ef84e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:52.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.746+0000 7fdbeb9ddb80 1 --2- 172.21.15.5:0/3842169020 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5566b5dbad00 0x5566b5ef84e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc005d70 tx=0x7fdbcc0227e0 comp rx=0 tx=0).stop 2024-05-09T00:51:52.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.748+0000 7fdbeb9ddb80 1 -- 172.21.15.5:0/3842169020 shutdown_connections 2024-05-09T00:51:52.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.748+0000 7fdbeb9ddb80 1 --2- 172.21.15.5:0/3842169020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566b5ec7540 0x5566b5ef8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:52.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.748+0000 7fdbeb9ddb80 1 --2- 172.21.15.5:0/3842169020 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566b5d404a0 0x5566b5ec4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:52.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.748+0000 7fdbeb9ddb80 1 --2- 172.21.15.5:0/3842169020 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5566b5dbad00 0x5566b5ef84e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:52.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.748+0000 7fdbeb9ddb80 1 -- 172.21.15.5:0/3842169020 >> 172.21.15.5:0/3842169020 conn(0x5566b5d43f70 msgr2=0x5566b5ec2c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:51:52.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.753+0000 7fdbeb9ddb80 1 -- 172.21.15.5:0/3842169020 shutdown_connections 2024-05-09T00:51:52.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.757+0000 7fdbeb9ddb80 1 -- 172.21.15.5:0/3842169020 wait complete. 2024-05-09T00:51:52.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.757+0000 7fdbeb9ddb80 1 -- start start 2024-05-09T00:51:52.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.758+0000 7fdbeb9ddb80 1 Processor -- start 2024-05-09T00:51:52.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.759+0000 7fdbeb9ddb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566b5d404a0 0x5566b5ecfdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:52.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.759+0000 7fdbeb9ddb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5566b5dbad00 0x5566b5ed0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:52.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.759+0000 7fdbeb9ddb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566b5ec7540 0x5566b5ed7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:52.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.760+0000 7fdbd659c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566b5d404a0 0x5566b5ecfdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:52.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.760+0000 7fdbd659c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566b5d404a0 0x5566b5ecfdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:48064/0 (socket says 172.21.15.5:48064) 2024-05-09T00:51:52.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.760+0000 7fdbd5d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5566b5dbad00 0x5566b5ed0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:52.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.760+0000 7fdbd5d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5566b5dbad00 0x5566b5ed0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:59314/0 (socket says 172.21.15.5:59314) 2024-05-09T00:51:52.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.760+0000 7fdbd659c700 1 -- 172.21.15.5:0/2041817219 learned_addr learned my addr 172.21.15.5:0/2041817219 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:51:52.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.760+0000 7fdbeb9ddb80 1 -- 172.21.15.5:0/2041817219 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb800e960 con 0x5566b5ec7540 2024-05-09T00:51:52.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.760+0000 7fdbeb9ddb80 1 -- 172.21.15.5:0/2041817219 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb800e290 con 0x5566b5dbad00 2024-05-09T00:51:52.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.760+0000 7fdbeb9ddb80 1 -- 172.21.15.5:0/2041817219 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb800b490 con 0x5566b5d404a0 2024-05-09T00:51:52.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.761+0000 7fdbeb9ddb80 10 client.0 _ll_register_callbacks cb 0x5566b5c516c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:51:52.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.761+0000 7fdbd659c700 1 -- 172.21.15.5:0/2041817219 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5566b5dbad00 msgr2=0x5566b5ed0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:52.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.761+0000 7fdbd659c700 1 --2- 172.21.15.5:0/2041817219 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5566b5dbad00 0x5566b5ed0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:52.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.761+0000 7fdbd659c700 1 -- 172.21.15.5:0/2041817219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566b5ec7540 msgr2=0x5566b5ed7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:52.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.761+0000 7fdbd659c700 1 --2- 172.21.15.5:0/2041817219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566b5ec7540 0x5566b5ed7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:52.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.761+0000 7fdbd659c700 1 -- 172.21.15.5:0/2041817219 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566b5e82ca0 con 0x5566b5d404a0 2024-05-09T00:51:52.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.765+0000 7fdbd659c700 1 --2- 172.21.15.5:0/2041817219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566b5d404a0 0x5566b5ecfdd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdbc80a5030 tx=0x7fdbc8004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f7516b622796d2e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:51:52.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.765+0000 7fdbbdffb700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-09T00:51:52.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.770+0000 7fdbbdffb700 1 -- 172.21.15.5:0/2041817219 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc80b1e10 con 0x5566b5d404a0 2024-05-09T00:51:52.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.771+0000 7fdbeb9ddb80 1 -- 172.21.15.5:0/2041817219 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbcc035070 con 0x5566b5d404a0 2024-05-09T00:51:52.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.771+0000 7fdbbdffb700 1 -- 172.21.15.5:0/2041817219 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fdbc80a9420 con 0x5566b5d404a0 2024-05-09T00:51:52.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.772+0000 7fdbbdffb700 1 -- 172.21.15.5:0/2041817219 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdbc80b1980 con 0x5566b5d404a0 2024-05-09T00:51:52.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.772+0000 7fdbeb9ddb80 10 client.25636 fetch_fsmap learned FSMap version 30 2024-05-09T00:51:52.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.772+0000 7fdbeb9ddb80 1 -- 172.21.15.5:0/2041817219 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5566b5ecbcc0 con 0x5566b5d404a0 2024-05-09T00:51:52.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.777+0000 7fdbbdffb700 1 -- 172.21.15.5:0/2041817219 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbcc035070 con 0x5566b5d404a0 2024-05-09T00:51:52.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.777+0000 7fdbbdffb700 1 -- 172.21.15.5:0/2041817219 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7fdbc80a4a30 con 0x5566b5d404a0 2024-05-09T00:51:52.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.777+0000 7fdbeb9ddb80 10 client.25636 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T00:51:52.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.777+0000 7fdbeb9ddb80 10 client.25636 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:51:52.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.777+0000 7fdbeb9ddb80 20 client.25636 populate_metadata read hostname 'smithi005' 2024-05-09T00:51:52.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.777+0000 7fdbeb9ddb80 1 --2- 172.21.15.5:0/2041817219 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5566b5f642a0 0x5566b5f666c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:52.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.777+0000 7fdbeb9ddb80 4 client.25636 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:51:52.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.777+0000 7fdbeb9ddb80 1 -- 172.21.15.5:0/2041817219 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5566b5de7b20 con 0x5566b5f642a0 2024-05-09T00:51:52.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.780+0000 7fdbd5d9b700 1 --2- 172.21.15.5:0/2041817219 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5566b5f642a0 0x5566b5f666c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:52.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.784+0000 7fdbd5d9b700 1 --2- 172.21.15.5:0/2041817219 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5566b5f642a0 0x5566b5f666c0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f08200560bcd60b server_cookie=43b6b61b1201c0dc in_seq=0 out_seq=0 2024-05-09T00:51:52.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.784+0000 7fdbbdffb700 10 client.25636 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:51:52.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.789+0000 7fdbbdffb700 1 -- 172.21.15.5:0/2041817219 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5566b5de7b20 con 0x5566b5f642a0 2024-05-09T00:51:52.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.789+0000 7fdbbdffb700 10 client.25636 handle_command_reply: tid=0 2024-05-09T00:51:52.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.789+0000 7fdbeb9ddb80 1 client.25636 shutdown 2024-05-09T00:51:52.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.790+0000 7fdbeb9ddb80 1 -- 172.21.15.5:0/2041817219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566b5d404a0 msgr2=0x5566b5ecfdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:52.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.790+0000 7fdbeb9ddb80 1 --2- 172.21.15.5:0/2041817219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566b5d404a0 0x5566b5ecfdd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdbc80a5030 tx=0x7fdbc8004750 comp rx=0 tx=0).stop 2024-05-09T00:51:52.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.791+0000 7fdbeb9ddb80 1 -- 172.21.15.5:0/2041817219 shutdown_connections 2024-05-09T00:51:52.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.791+0000 7fdbeb9ddb80 1 --2- 172.21.15.5:0/2041817219 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5566b5f642a0 0x5566b5f666c0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:52.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.791+0000 7fdbeb9ddb80 1 --2- 172.21.15.5:0/2041817219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566b5ec7540 0x5566b5ed7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:52.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.791+0000 7fdbeb9ddb80 1 --2- 172.21.15.5:0/2041817219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566b5d404a0 0x5566b5ecfdd0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:52.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.791+0000 7fdbeb9ddb80 1 --2- 172.21.15.5:0/2041817219 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5566b5dbad00 0x5566b5ed0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:52.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.791+0000 7fdbeb9ddb80 1 -- 172.21.15.5:0/2041817219 >> 172.21.15.5:0/2041817219 conn(0x5566b5d43f70 msgr2=0x5566b5ec5940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:51:52.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.791+0000 7fdbbdffb700 0 client.25636 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:51:52.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.795+0000 7fdbeb9ddb80 1 -- 172.21.15.5:0/2041817219 shutdown_connections 2024-05-09T00:51:52.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.795+0000 7fdbeb9ddb80 1 -- 172.21.15.5:0/2041817219 wait complete. 2024-05-09T00:51:52.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.796+0000 7fdbeb9ddb80 20 client.25636 trim_cache size 0 max 16384 2024-05-09T00:51:52.799 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-09T00:51:52.799 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T00:51:52.799 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-05-09T00:51:52.799 INFO:teuthology.orchestra.run.smithi005.stdout: "92d24500-4efe-4208-8190-01302b541b47": { 2024-05-09T00:51:52.800 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-05-09T00:51:52.800 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "92d24500-4efe-4208-8190-01302b541b47", 2024-05-09T00:51:52.800 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-05-09T00:51:52.800 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T00:51:52.800 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T00:51:52.800 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-09T00:51:52.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.799+0000 7fdbd759e700 1 -- 172.21.15.5:0/1708613109 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fdbc00b2940 msgr2=0x7fdbc00b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:52.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.799+0000 7fdbd759e700 1 --2- 172.21.15.5:0/1708613109 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fdbc00b2940 0x7fdbc00b4e50 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc00b5c0 tx=0x7fdbcc005fb0 comp rx=0 tx=0).stop 2024-05-09T00:51:52.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.799+0000 7fdbd759e700 1 -- 172.21.15.5:0/1708613109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbd81043e0 msgr2=0x7fdbd819d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:52.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.799+0000 7fdbd759e700 1 --2- 172.21.15.5:0/1708613109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbd81043e0 0x7fdbd819d000 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fdbc800b530 tx=0x7fdbc800b8f0 comp rx=0 tx=0).stop 2024-05-09T00:51:52.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.800+0000 7fdbd759e700 1 -- 172.21.15.5:0/1708613109 shutdown_connections 2024-05-09T00:51:52.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.800+0000 7fdbd759e700 1 --2- 172.21.15.5:0/1708613109 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fdbc00b2940 0x7fdbc00b4e50 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:52.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.800+0000 7fdbd759e700 1 --2- 172.21.15.5:0/1708613109 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdbd8109660 0x7fdbd81a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:52.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.800+0000 7fdbd759e700 1 --2- 172.21.15.5:0/1708613109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbd81043e0 0x7fdbd819d000 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:52.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.800+0000 7fdbd759e700 1 --2- 172.21.15.5:0/1708613109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbd8108d10 0x7fdbd819d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:52.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.800+0000 7fdbd759e700 1 -- 172.21.15.5:0/1708613109 >> 172.21.15.5:0/1708613109 conn(0x7fdbd8079d10 msgr2=0x7fdbd8101700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:51:52.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.800+0000 7fdbd759e700 1 -- 172.21.15.5:0/1708613109 shutdown_connections 2024-05-09T00:51:52.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:51:52.800+0000 7fdbd759e700 1 -- 172.21.15.5:0/1708613109 wait complete. 2024-05-09T00:51:52.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:51:52 smithi131 ceph-mon[34547]: scrub summary: idle+waiting paths [/] 2024-05-09T00:51:53.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.209+0000 7fcceb7fe640 1 -- 172.21.15.5:0/1189631088 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce4030d70 con 0x7fccec10b2c0 2024-05-09T00:51:53.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.211+0000 7fccf184f640 1 -- 172.21.15.5:0/1189631088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccec10b2c0 msgr2=0x7fccec10b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:53.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.211+0000 7fccf184f640 1 --2- 172.21.15.5:0/1189631088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccec10b2c0 0x7fccec10b6d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fcce4009be0 tx=0x7fcce402a6f0 comp rx=0 tx=0).stop 2024-05-09T00:51:53.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.217+0000 7fccf184f640 1 -- 172.21.15.5:0/1189631088 shutdown_connections 2024-05-09T00:51:53.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.217+0000 7fccf184f640 1 --2- 172.21.15.5:0/1189631088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccec10ffe0 0x7fccec1103f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:53.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.217+0000 7fccf184f640 1 --2- 172.21.15.5:0/1189631088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccec10b2c0 0x7fccec10b6d0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:53.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.217+0000 7fccf184f640 1 --2- 172.21.15.5:0/1189631088 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccec10a470 0x7fccec10ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:53.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.217+0000 7fccf184f640 1 -- 172.21.15.5:0/1189631088 >> 172.21.15.5:0/1189631088 conn(0x7fccec070460 msgr2=0x7fccec072910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:51:53.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.227+0000 7fccf184f640 1 -- 172.21.15.5:0/1189631088 shutdown_connections 2024-05-09T00:51:53.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.231+0000 7fccf184f640 1 -- 172.21.15.5:0/1189631088 wait complete. 2024-05-09T00:51:53.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.233+0000 7fccf184f640 1 Processor -- start 2024-05-09T00:51:53.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.233+0000 7fccf184f640 1 -- start start 2024-05-09T00:51:53.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.234+0000 7fccf184f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccec10a470 0x7fccec1a9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:53.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.234+0000 7fccf184f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccec10b2c0 0x7fccec1a95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:53.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.234+0000 7fccf184f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccec10ffe0 0x7fccec1b0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:53.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.234+0000 7fccf184f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fccec11e420 con 0x7fccec10a470 2024-05-09T00:51:53.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.234+0000 7fccf184f640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fccec11e2a0 con 0x7fccec10b2c0 2024-05-09T00:51:53.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.234+0000 7fccf184f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fccec11e5a0 con 0x7fccec10ffe0 2024-05-09T00:51:53.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.237+0000 7fccf104e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccec10ffe0 0x7fccec1b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:53.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.238+0000 7fccebfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccec10b2c0 0x7fccec1a95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:53.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.238+0000 7fccf104e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccec10ffe0 0x7fccec1b0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:48080/0 (socket says 172.21.15.5:48080) 2024-05-09T00:51:53.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.238+0000 7fccf104e640 1 -- 172.21.15.5:0/571471886 learned_addr learned my addr 172.21.15.5:0/571471886 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:51:53.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.238+0000 7fccf084d640 1 --2- 172.21.15.5:0/571471886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccec10a470 0x7fccec1a9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:53.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.241+0000 7fccf104e640 1 -- 172.21.15.5:0/571471886 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccec10b2c0 msgr2=0x7fccec1a95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:53.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.241+0000 7fccf104e640 1 --2- 172.21.15.5:0/571471886 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccec10b2c0 0x7fccec1a95a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:53.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.241+0000 7fccf104e640 1 -- 172.21.15.5:0/571471886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccec10a470 msgr2=0x7fccec1a9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:53.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.241+0000 7fccf104e640 1 --2- 172.21.15.5:0/571471886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccec10a470 0x7fccec1a9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:53.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.241+0000 7fccf104e640 1 -- 172.21.15.5:0/571471886 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccec111dc0 con 0x7fccec10ffe0 2024-05-09T00:51:53.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.243+0000 7fccf104e640 1 --2- 172.21.15.5:0/571471886 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccec10ffe0 0x7fccec1b0530 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fccdc00d750 tx=0x7fccdc00dc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:51:53.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.245+0000 7fcce9ffb640 1 -- 172.21.15.5:0/571471886 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccdc017070 con 0x7fccec10ffe0 2024-05-09T00:51:53.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.245+0000 7fcce9ffb640 1 -- 172.21.15.5:0/571471886 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccdc0054d0 con 0x7fccec10ffe0 2024-05-09T00:51:53.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.245+0000 7fcce9ffb640 1 -- 172.21.15.5:0/571471886 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccdc004ce0 con 0x7fccec10ffe0 2024-05-09T00:51:53.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.245+0000 7fccf184f640 1 -- 172.21.15.5:0/571471886 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccec1120b0 con 0x7fccec10ffe0 2024-05-09T00:51:53.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.246+0000 7fccf184f640 1 -- 172.21.15.5:0/571471886 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccec112630 con 0x7fccec10ffe0 2024-05-09T00:51:53.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.262+0000 7fccf9967740 1 Processor -- start 2024-05-09T00:51:53.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.264+0000 7fccf9967740 1 -- start start 2024-05-09T00:51:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.265+0000 7fccf9967740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561be8ab51b0 0x561be8e596c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.265+0000 7fccf9967740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561be8a8c2f0 0x561be8e59c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.265+0000 7fccf9967740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561be8a944d0 0x561be8e60b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.266+0000 7fccf9967740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561be8dbd7c0 con 0x561be8ab51b0 2024-05-09T00:51:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.266+0000 7fccf9967740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x561be8d590e0 con 0x561be8a8c2f0 2024-05-09T00:51:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.267+0000 7fccebfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561be8ab51b0 0x561be8e596c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.272+0000 7fccebfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561be8ab51b0 0x561be8e596c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46308/0 (socket says 172.21.15.5:46308) 2024-05-09T00:51:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.272+0000 7fccebfff640 1 -- 172.21.15.5:0/1767589792 learned_addr learned my addr 172.21.15.5:0/1767589792 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:51:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.272+0000 7fccf9967740 1 -- 172.21.15.5:0/1767589792 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x561be8db9780 con 0x561be8a944d0 2024-05-09T00:51:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.272+0000 7fcce9ffb640 1 -- 172.21.15.5:0/571471886 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fccdc044ab0 con 0x7fccec10ffe0 2024-05-09T00:51:53.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.273+0000 7fccebfff640 1 --2- 172.21.15.5:0/1767589792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561be8a944d0 0x561be8e60b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:53.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.274+0000 7fccf084d640 1 --2- 172.21.15.5:0/1767589792 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561be8a8c2f0 0x561be8e59c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:53.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.274+0000 7fcce9ffb640 1 --2- 172.21.15.5:0/571471886 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fccd40b29f0 0x7fccd40b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.275+0000 7fccebfff640 1 -- 172.21.15.5:0/1767589792 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561be8a8c2f0 msgr2=0x561be8e59c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.275+0000 7fccebfff640 1 --2- 172.21.15.5:0/1767589792 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561be8a8c2f0 0x561be8e59c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.275+0000 7fccebfff640 1 -- 172.21.15.5:0/1767589792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561be8ab51b0 msgr2=0x561be8e596c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.275+0000 7fccebfff640 1 --2- 172.21.15.5:0/1767589792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561be8ab51b0 0x561be8e596c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.275+0000 7fccebfff640 1 -- 172.21.15.5:0/1767589792 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561be8d827c0 con 0x561be8a944d0 2024-05-09T00:51:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.276+0000 7fccebfff640 1 --2- 172.21.15.5:0/1767589792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561be8a944d0 0x561be8e60b40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fcce40309d0 tx=0x7fcce4000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6f396800870d072 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:51:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.277+0000 7fcce9ffb640 1 -- 172.21.15.5:0/571471886 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fccdc0e1270 con 0x7fccec10ffe0 2024-05-09T00:51:53.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.279+0000 7fccd2ffd640 1 -- 172.21.15.5:0/1767589792 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce4043020 con 0x561be8a944d0 2024-05-09T00:51:53.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.279+0000 7fccd2ffd640 1 -- 172.21.15.5:0/1767589792 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcce4003e20 con 0x561be8a944d0 2024-05-09T00:51:53.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.279+0000 7fccd2ffd640 1 -- 172.21.15.5:0/1767589792 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce4036040 con 0x561be8a944d0 2024-05-09T00:51:53.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.284+0000 7fccf104e640 1 --2- 172.21.15.5:0/571471886 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fccd40b29f0 0x7fccd40b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:53.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.286+0000 7fccf9967740 1 -- 172.21.15.5:0/1767589792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561be8a944d0 msgr2=0x561be8e60b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:53.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.286+0000 7fccf9967740 1 --2- 172.21.15.5:0/1767589792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561be8a944d0 0x561be8e60b40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fcce40309d0 tx=0x7fcce4000f40 comp rx=0 tx=0).stop 2024-05-09T00:51:53.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.286+0000 7fccf9967740 1 -- 172.21.15.5:0/1767589792 shutdown_connections 2024-05-09T00:51:53.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.286+0000 7fccf9967740 1 --2- 172.21.15.5:0/1767589792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561be8a944d0 0x561be8e60b40 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:53.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.286+0000 7fccf9967740 1 --2- 172.21.15.5:0/1767589792 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561be8a8c2f0 0x561be8e59c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:53.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.286+0000 7fccf9967740 1 --2- 172.21.15.5:0/1767589792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561be8ab51b0 0x561be8e596c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:53.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.286+0000 7fccf9967740 1 -- 172.21.15.5:0/1767589792 >> 172.21.15.5:0/1767589792 conn(0x561be8aa9300 msgr2=0x561be8e55050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:51:53.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.289+0000 7fccf9967740 1 -- 172.21.15.5:0/1767589792 shutdown_connections 2024-05-09T00:51:53.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.289+0000 7fccf9967740 1 -- 172.21.15.5:0/1767589792 wait complete. 2024-05-09T00:51:53.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.290+0000 7fccf9967740 1 -- start start 2024-05-09T00:51:53.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.290+0000 7fccf9967740 1 Processor -- start 2024-05-09T00:51:53.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.294+0000 7fccf104e640 1 --2- 172.21.15.5:0/571471886 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fccd40b29f0 0x7fccd40b4f00 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fccdc0a4880 tx=0x7fccdc0a4040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:51:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.296+0000 7fccf9967740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561be8a8c2f0 0x561be8e63480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.296+0000 7fccf9967740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561be8a944d0 0x561be8e639c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.296+0000 7fccf9967740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561be8ab51b0 0x561be8e6a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.296+0000 7fccf9967740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561be8e61530 con 0x561be8a944d0 2024-05-09T00:51:53.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.297+0000 7fccebfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561be8a8c2f0 0x561be8e63480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:53.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.298+0000 7fccebfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561be8a8c2f0 0x561be8e63480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:59360/0 (socket says 172.21.15.5:59360) 2024-05-09T00:51:53.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.298+0000 7fccebfff640 1 -- 172.21.15.5:0/2107010512 learned_addr learned my addr 172.21.15.5:0/2107010512 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:51:53.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.299+0000 7fccf9967740 1 -- 172.21.15.5:0/2107010512 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x561be8d449c0 con 0x561be8a8c2f0 2024-05-09T00:51:53.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.299+0000 7fccf9967740 1 -- 172.21.15.5:0/2107010512 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x561be8e616b0 con 0x561be8ab51b0 2024-05-09T00:51:53.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.299+0000 7fccf084d640 1 --2- 172.21.15.5:0/2107010512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561be8a944d0 0x561be8e639c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:53.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.299+0000 7fccf9967740 10 client.0 _ll_register_callbacks cb 0x561be8d71020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:51:53.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.300+0000 7fccebfff640 1 --2- 172.21.15.5:0/2107010512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561be8ab51b0 0x561be8e6a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:53.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.301+0000 7fccf084d640 1 -- 172.21.15.5:0/2107010512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561be8ab51b0 msgr2=0x561be8e6a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:53.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.301+0000 7fccf084d640 1 --2- 172.21.15.5:0/2107010512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561be8ab51b0 0x561be8e6a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:53.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.301+0000 7fccf084d640 1 -- 172.21.15.5:0/2107010512 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561be8a8c2f0 msgr2=0x561be8e63480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:53.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.301+0000 7fccf084d640 1 --2- 172.21.15.5:0/2107010512 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561be8a8c2f0 0x561be8e63480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:53.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.301+0000 7fccf084d640 1 -- 172.21.15.5:0/2107010512 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561be8d6b1f0 con 0x561be8a944d0 2024-05-09T00:51:53.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.309+0000 7fccf084d640 1 --2- 172.21.15.5:0/2107010512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561be8a944d0 0x561be8e639c0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x561be8e5aaa0 tx=0x7fcce000f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3091f7f0bc47042 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:51:53.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.309+0000 7fcccbfff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:51:53.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.316+0000 7fcccbfff640 1 -- 172.21.15.5:0/2107010512 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce0018020 con 0x561be8a944d0 2024-05-09T00:51:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.316+0000 7fccf9967740 1 -- 172.21.15.5:0/2107010512 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcce4003e20 con 0x561be8a944d0 2024-05-09T00:51:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.316+0000 7fcccbfff640 1 -- 172.21.15.5:0/2107010512 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fcce0014660 con 0x561be8a944d0 2024-05-09T00:51:53.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.321+0000 7fcccbfff640 1 -- 172.21.15.5:0/2107010512 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcce0023050 con 0x561be8a944d0 2024-05-09T00:51:53.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.323+0000 7fccf9967740 10 client.16092 fetch_fsmap learned FSMap version 30 2024-05-09T00:51:53.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.323+0000 7fccf9967740 1 -- 172.21.15.5:0/2107010512 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561be8b20790 con 0x561be8a944d0 2024-05-09T00:51:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.329+0000 7fcccbfff640 1 -- 172.21.15.5:0/2107010512 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcce4003e20 con 0x561be8a944d0 2024-05-09T00:51:53.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.331+0000 7fcccbfff640 1 -- 172.21.15.5:0/2107010512 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7fcce0013a00 con 0x561be8a944d0 2024-05-09T00:51:53.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.331+0000 7fccf9967740 10 client.16092 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T00:51:53.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.331+0000 7fccf9967740 10 client.16092 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:51:53.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.331+0000 7fccf9967740 20 client.16092 populate_metadata read hostname 'smithi005' 2024-05-09T00:51:53.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.331+0000 7fccf9967740 1 --2- 172.21.15.5:0/2107010512 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x561be8a73d80 0x561be8e71170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:53.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.331+0000 7fccf9967740 4 client.16092 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:51:53.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.332+0000 7fccf9967740 1 -- 172.21.15.5:0/2107010512 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561be8dcb3d0 con 0x561be8a73d80 2024-05-09T00:51:53.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.337+0000 7fccebfff640 1 --2- 172.21.15.5:0/2107010512 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x561be8a73d80 0x561be8e71170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:53.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.341+0000 7fccebfff640 1 --2- 172.21.15.5:0/2107010512 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x561be8a73d80 0x561be8e71170 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=116ab18d21b2cd46 server_cookie=b297937a304ad3a5 in_seq=0 out_seq=0 2024-05-09T00:51:53.342 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.341+0000 7fcccbfff640 10 client.16092 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:51:53.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.344+0000 7fcccbfff640 1 -- 172.21.15.5:0/2107010512 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561be8dcb3d0 con 0x561be8a73d80 2024-05-09T00:51:53.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.344+0000 7fcccbfff640 10 client.16092 handle_command_reply: tid=0 2024-05-09T00:51:53.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.344+0000 7fccf9967740 1 client.16092 shutdown 2024-05-09T00:51:53.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.345+0000 7fccf9967740 1 -- 172.21.15.5:0/2107010512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561be8a944d0 msgr2=0x561be8e639c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:53.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.345+0000 7fccf9967740 1 --2- 172.21.15.5:0/2107010512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561be8a944d0 0x561be8e639c0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x561be8e5aaa0 tx=0x7fcce000f5b0 comp rx=0 tx=0).stop 2024-05-09T00:51:53.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.349+0000 7fccf9967740 1 -- 172.21.15.5:0/2107010512 shutdown_connections 2024-05-09T00:51:53.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.349+0000 7fccf9967740 1 --2- 172.21.15.5:0/2107010512 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x561be8a73d80 0x561be8e71170 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:53.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.349+0000 7fccf9967740 1 --2- 172.21.15.5:0/2107010512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561be8ab51b0 0x561be8e6a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:53.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.349+0000 7fccf9967740 1 --2- 172.21.15.5:0/2107010512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561be8a944d0 0x561be8e639c0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:53.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.349+0000 7fccf9967740 1 --2- 172.21.15.5:0/2107010512 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561be8a8c2f0 0x561be8e63480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:53.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.349+0000 7fccf9967740 1 -- 172.21.15.5:0/2107010512 >> 172.21.15.5:0/2107010512 conn(0x561be8aa9300 msgr2=0x561be8e54290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:51:53.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.349+0000 7fcccbfff640 0 client.16092 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:51:53.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.355+0000 7fccf9967740 1 -- 172.21.15.5:0/2107010512 shutdown_connections 2024-05-09T00:51:53.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.355+0000 7fccf9967740 1 -- 172.21.15.5:0/2107010512 wait complete. 2024-05-09T00:51:53.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.356+0000 7fccf9967740 20 client.16092 trim_cache size 0 max 16384 2024-05-09T00:51:53.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.420+0000 7fccf9967740 1 Processor -- start 2024-05-09T00:51:53.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.420+0000 7fccf9967740 1 -- start start 2024-05-09T00:51:53.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.421+0000 7fccf9967740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561be8a8c2f0 0x561be8e55100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:53.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.421+0000 7fccf9967740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561be8a944d0 0x561be8e6a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:53.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.421+0000 7fccf9967740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561be8ab51b0 0x561be8e6ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:53.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.421+0000 7fccf9967740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fccd800e4c0 con 0x561be8a944d0 2024-05-09T00:51:53.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.421+0000 7fccf9967740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fccd8010030 con 0x561be8a8c2f0 2024-05-09T00:51:53.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.421+0000 7fccf9967740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fccd800eb90 con 0x561be8ab51b0 2024-05-09T00:51:53.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.426+0000 7fccf084d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561be8a944d0 0x561be8e6a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:53.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.426+0000 7fccf084d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561be8a944d0 0x561be8e6a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46320/0 (socket says 172.21.15.5:46320) 2024-05-09T00:51:53.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.426+0000 7fccf084d640 1 -- 172.21.15.5:0/450128494 learned_addr learned my addr 172.21.15.5:0/450128494 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:51:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.426+0000 7fccebfff640 1 --2- 172.21.15.5:0/450128494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561be8ab51b0 0x561be8e6ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.426+0000 7fccebfff640 1 --2- 172.21.15.5:0/450128494 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561be8a8c2f0 0x561be8e55100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.427+0000 7fccebfff640 1 -- 172.21.15.5:0/450128494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561be8ab51b0 msgr2=0x561be8e6ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.427+0000 7fccebfff640 1 --2- 172.21.15.5:0/450128494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561be8ab51b0 0x561be8e6ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.427+0000 7fccebfff640 1 -- 172.21.15.5:0/450128494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561be8a944d0 msgr2=0x561be8e6a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.427+0000 7fccebfff640 1 --2- 172.21.15.5:0/450128494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561be8a944d0 0x561be8e6a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.427+0000 7fccebfff640 1 -- 172.21.15.5:0/450128494 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561be8b975f0 con 0x561be8a8c2f0 2024-05-09T00:51:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.427+0000 7fccebfff640 1 --2- 172.21.15.5:0/450128494 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561be8a8c2f0 0x561be8e55100 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcce40305a0 tx=0x7fcce4009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6cb59172d5eb9bff server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:51:53.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.430+0000 7fccd27fc640 1 -- 172.21.15.5:0/450128494 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce4043040 con 0x561be8a8c2f0 2024-05-09T00:51:53.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.431+0000 7fccd27fc640 1 -- 172.21.15.5:0/450128494 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcce40317e0 con 0x561be8a8c2f0 2024-05-09T00:51:53.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.431+0000 7fccd27fc640 1 -- 172.21.15.5:0/450128494 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce4036040 con 0x561be8a8c2f0 2024-05-09T00:51:53.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.434+0000 7fccf9967740 1 -- 172.21.15.5:0/450128494 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561be8a8c2f0 msgr2=0x561be8e55100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.434+0000 7fccf9967740 1 --2- 172.21.15.5:0/450128494 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561be8a8c2f0 0x561be8e55100 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcce40305a0 tx=0x7fcce4009500 comp rx=0 tx=0).stop 2024-05-09T00:51:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.434+0000 7fccf9967740 1 -- 172.21.15.5:0/450128494 shutdown_connections 2024-05-09T00:51:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.434+0000 7fccf9967740 1 --2- 172.21.15.5:0/450128494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561be8ab51b0 0x561be8e6ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.434+0000 7fccf9967740 1 --2- 172.21.15.5:0/450128494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561be8a944d0 0x561be8e6a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.434+0000 7fccf9967740 1 --2- 172.21.15.5:0/450128494 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561be8a8c2f0 0x561be8e55100 secure :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcce40305a0 tx=0x7fcce4009500 comp rx=0 tx=0).stop 2024-05-09T00:51:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.434+0000 7fccf9967740 1 -- 172.21.15.5:0/450128494 >> 172.21.15.5:0/450128494 conn(0x561be8aa9300 msgr2=0x561be8e549f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:51:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.435+0000 7fccf9967740 1 -- 172.21.15.5:0/450128494 shutdown_connections 2024-05-09T00:51:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.438+0000 7fccf9967740 1 -- 172.21.15.5:0/450128494 wait complete. 2024-05-09T00:51:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.438+0000 7fccf9967740 1 -- start start 2024-05-09T00:51:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.439+0000 7fccf9967740 1 Processor -- start 2024-05-09T00:51:53.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.448+0000 7fccf9967740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561be8a944d0 0x561be8e67d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:53.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.448+0000 7fccf9967740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561be8ab51b0 0x561be8e682c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:53.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.448+0000 7fccf9967740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561be8a491c0 0x561be8ed74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:53.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.449+0000 7fccf9967740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fccd800f930 con 0x561be8ab51b0 2024-05-09T00:51:53.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.449+0000 7fccf9967740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fccd800f260 con 0x561be8a944d0 2024-05-09T00:51:53.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.449+0000 7fccf9967740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fccd800c550 con 0x561be8a491c0 2024-05-09T00:51:53.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.449+0000 7fccf9967740 10 client.0 _ll_register_callbacks cb 0x7fcce000b260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:51:53.454 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.454+0000 7fccf084d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561be8ab51b0 0x561be8e682c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:53.454 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.454+0000 7fccebfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561be8a944d0 0x561be8e67d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:53.454 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.454+0000 7fccf084d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561be8ab51b0 0x561be8e682c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46326/0 (socket says 172.21.15.5:46326) 2024-05-09T00:51:53.454 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.454+0000 7fccebfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561be8a491c0 0x561be8ed74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:53.454 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.454+0000 7fccf084d640 1 -- 172.21.15.5:0/2634585430 learned_addr learned my addr 172.21.15.5:0/2634585430 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:51:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.458+0000 7fccf084d640 1 -- 172.21.15.5:0/2634585430 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561be8a491c0 msgr2=0x561be8ed74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.458+0000 7fccf084d640 1 --2- 172.21.15.5:0/2634585430 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561be8a491c0 0x561be8ed74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.458+0000 7fccf084d640 1 -- 172.21.15.5:0/2634585430 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561be8a944d0 msgr2=0x561be8e67d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:51:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.458+0000 7fccf084d640 1 --2- 172.21.15.5:0/2634585430 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561be8a944d0 0x561be8e67d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.458+0000 7fccf084d640 1 -- 172.21.15.5:0/2634585430 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561be8d81ed0 con 0x561be8ab51b0 2024-05-09T00:51:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.459+0000 7fccf084d640 1 --2- 172.21.15.5:0/2634585430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561be8ab51b0 0x561be8e682c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fcce0004270 tx=0x7fcce00042a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bc377cb4ccf8a85 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:51:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.459+0000 7fcccbfff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:51:53.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.461+0000 7fcccbfff640 1 -- 172.21.15.5:0/2634585430 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce0018040 con 0x561be8ab51b0 2024-05-09T00:51:53.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.461+0000 7fcccbfff640 1 -- 172.21.15.5:0/2634585430 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fcce0003890 con 0x561be8ab51b0 2024-05-09T00:51:53.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.461+0000 7fccf9967740 1 -- 172.21.15.5:0/2634585430 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcce40317e0 con 0x561be8ab51b0 2024-05-09T00:51:53.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.468+0000 7fcccbfff640 1 -- 172.21.15.5:0/2634585430 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcce0003a90 con 0x561be8ab51b0 2024-05-09T00:51:53.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.468+0000 7fccf9967740 10 client.16098 fetch_fsmap learned FSMap version 30 2024-05-09T00:51:53.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.468+0000 7fccf9967740 1 -- 172.21.15.5:0/2634585430 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561be8ed87e0 con 0x561be8ab51b0 2024-05-09T00:51:53.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.469+0000 7fcccbfff640 1 -- 172.21.15.5:0/2634585430 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcce002c070 con 0x561be8ab51b0 2024-05-09T00:51:53.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.470+0000 7fcccbfff640 1 -- 172.21.15.5:0/2634585430 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7fcce0014460 con 0x561be8ab51b0 2024-05-09T00:51:53.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.470+0000 7fccf9967740 10 client.16098 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T00:51:53.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.470+0000 7fccf9967740 10 client.16098 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:51:53.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.470+0000 7fccf9967740 20 client.16098 populate_metadata read hostname 'smithi005' 2024-05-09T00:51:53.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.470+0000 7fccf9967740 1 --2- 172.21.15.5:0/2634585430 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x561be8ed9240 0x561be8edb660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:51:53.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.470+0000 7fccf9967740 4 client.16098 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:51:53.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.470+0000 7fccf9967740 1 -- 172.21.15.5:0/2634585430 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fcce40317e0 con 0x561be8ed9240 2024-05-09T00:51:53.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.477+0000 7fccebfff640 1 --2- 172.21.15.5:0/2634585430 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x561be8ed9240 0x561be8edb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:51:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.493+0000 7fccebfff640 1 --2- 172.21.15.5:0/2634585430 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x561be8ed9240 0x561be8edb660 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6970acd32304eab server_cookie=5dabdc6585770c85 in_seq=0 out_seq=0 2024-05-09T00:51:53.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:53.493+0000 7fcccbfff640 10 client.16098 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:51:53.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:51:53 smithi157 ceph-mon[34294]: pgmap v537: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 782 MiB/s wr, 1.73k op/s 2024-05-09T00:51:53.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:51:53 smithi005 ceph-mon[30365]: pgmap v537: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 782 MiB/s wr, 1.73k op/s 2024-05-09T00:51:53.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:51:53 smithi131 ceph-mon[34547]: pgmap v537: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 782 MiB/s wr, 1.73k op/s 2024-05-09T00:51:55.627 INFO:tasks.cephfs.filesystem:scrub status for tag:92d24500-4efe-4208-8190-01302b541b47 - {'path': '/', 'tag': '92d24500-4efe-4208-8190-01302b541b47', 'options': 'recursive,force'} 2024-05-09T00:51:55.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:51:55 smithi157 ceph-mon[34294]: pgmap v538: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 877 MiB/s wr, 2.25k op/s 2024-05-09T00:51:55.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:51:55 smithi005 ceph-mon[30365]: pgmap v538: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 877 MiB/s wr, 2.25k op/s 2024-05-09T00:51:55.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:51:55 smithi131 ceph-mon[34547]: pgmap v538: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 877 MiB/s wr, 2.25k op/s 2024-05-09T00:51:56.685 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:51:56 smithi157 ceph-mon[34294]: scrub summary: idle 2024-05-09T00:51:56.790 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:51:56 smithi005 ceph-mon[30365]: scrub summary: idle 2024-05-09T00:51:56.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:51:56 smithi131 ceph-mon[34547]: scrub summary: idle 2024-05-09T00:51:57.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:51:57 smithi157 ceph-mon[34294]: pgmap v539: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 934 MiB/s wr, 2.47k op/s 2024-05-09T00:51:57.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:51:57 smithi157 ceph-mon[34294]: scrub summary: idle 2024-05-09T00:51:57.790 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:51:57 smithi005 ceph-mon[30365]: pgmap v539: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 934 MiB/s wr, 2.47k op/s 2024-05-09T00:51:57.790 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:51:57 smithi005 ceph-mon[30365]: scrub summary: idle 2024-05-09T00:51:57.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:51:57 smithi131 ceph-mon[34547]: pgmap v539: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 934 MiB/s wr, 2.47k op/s 2024-05-09T00:51:57.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:51:57 smithi131 ceph-mon[34547]: scrub summary: idle 2024-05-09T00:51:59.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:59.203+0000 7fcccbfff640 1 -- 172.21.15.5:0/2634585430 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7fcce40317e0 con 0x561be8ed9240 2024-05-09T00:51:59.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:59.203+0000 7fcccbfff640 10 client.16098 handle_command_reply: tid=0 2024-05-09T00:51:59.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:59.203+0000 7fccf9967740 1 client.16098 shutdown 2024-05-09T00:51:59.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:59.204+0000 7fccf9967740 1 -- 172.21.15.5:0/2634585430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561be8ab51b0 msgr2=0x561be8e682c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:59.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:59.204+0000 7fccf9967740 1 --2- 172.21.15.5:0/2634585430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561be8ab51b0 0x561be8e682c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fcce0004270 tx=0x7fcce00042a0 comp rx=0 tx=0).stop 2024-05-09T00:51:59.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:59.204+0000 7fccf9967740 1 -- 172.21.15.5:0/2634585430 shutdown_connections 2024-05-09T00:51:59.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:59.204+0000 7fccf9967740 1 --2- 172.21.15.5:0/2634585430 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x561be8ed9240 0x561be8edb660 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:59.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:59.205+0000 7fccf9967740 1 --2- 172.21.15.5:0/2634585430 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561be8a491c0 0x561be8ed74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:59.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:59.205+0000 7fccf9967740 1 --2- 172.21.15.5:0/2634585430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561be8ab51b0 0x561be8e682c0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:59.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:59.205+0000 7fccf9967740 1 --2- 172.21.15.5:0/2634585430 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561be8a944d0 0x561be8e67d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:59.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:59.205+0000 7fccf9967740 1 -- 172.21.15.5:0/2634585430 >> 172.21.15.5:0/2634585430 conn(0x561be8aa9300 msgr2=0x561be8e57f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:51:59.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:59.205+0000 7fcccbfff640 0 client.16098 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:51:59.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:59.205+0000 7fccf9967740 1 -- 172.21.15.5:0/2634585430 shutdown_connections 2024-05-09T00:51:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:59.206+0000 7fccf9967740 1 -- 172.21.15.5:0/2634585430 wait complete. 2024-05-09T00:51:59.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:59.207+0000 7fccf9967740 20 client.16098 trim_cache size 0 max 16384 2024-05-09T00:51:59.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-09T00:51:59.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-09T00:51:59.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-09T00:51:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:59.212+0000 7fccf184f640 1 -- 172.21.15.5:0/571471886 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fccd40b29f0 msgr2=0x7fccd40b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:59.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:59.212+0000 7fccf184f640 1 --2- 172.21.15.5:0/571471886 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fccd40b29f0 0x7fccd40b4f00 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fccdc0a4880 tx=0x7fccdc0a4040 comp rx=0 tx=0).stop 2024-05-09T00:51:59.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:59.212+0000 7fccf184f640 1 -- 172.21.15.5:0/571471886 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccec10ffe0 msgr2=0x7fccec1b0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:51:59.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:59.212+0000 7fccf184f640 1 --2- 172.21.15.5:0/571471886 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccec10ffe0 0x7fccec1b0530 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fccdc00d750 tx=0x7fccdc00dc10 comp rx=0 tx=0).stop 2024-05-09T00:51:59.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:59.213+0000 7fccf184f640 1 -- 172.21.15.5:0/571471886 shutdown_connections 2024-05-09T00:51:59.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:59.213+0000 7fccf184f640 1 --2- 172.21.15.5:0/571471886 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fccd40b29f0 0x7fccd40b4f00 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:59.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:59.213+0000 7fccf184f640 1 --2- 172.21.15.5:0/571471886 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccec10ffe0 0x7fccec1b0530 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:59.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:59.213+0000 7fccf184f640 1 --2- 172.21.15.5:0/571471886 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccec10b2c0 0x7fccec1a95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:59.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:59.213+0000 7fccf184f640 1 --2- 172.21.15.5:0/571471886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccec10a470 0x7fccec1a9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:51:59.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:59.213+0000 7fccf184f640 1 -- 172.21.15.5:0/571471886 >> 172.21.15.5:0/571471886 conn(0x7fccec070460 msgr2=0x7fccec10bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:51:59.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:59.213+0000 7fccf184f640 1 -- 172.21.15.5:0/571471886 shutdown_connections 2024-05-09T00:51:59.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:51:59.213+0000 7fccf184f640 1 -- 172.21.15.5:0/571471886 wait complete. 2024-05-09T00:51:59.790 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:51:59 smithi005 ceph-mon[30365]: pgmap v540: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 818 MiB/s wr, 2.22k op/s 2024-05-09T00:51:59.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:51:59 smithi131 ceph-mon[34547]: pgmap v540: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 818 MiB/s wr, 2.22k op/s 2024-05-09T00:51:59.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:51:59 smithi157 ceph-mon[34294]: pgmap v540: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 818 MiB/s wr, 2.22k op/s 2024-05-09T00:52:01.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:52:01 smithi005 ceph-mon[30365]: pgmap v541: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 971 MiB/s wr, 2.55k op/s 2024-05-09T00:52:01.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:52:01 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:52:01.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:52:01 smithi131 ceph-mon[34547]: pgmap v541: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 971 MiB/s wr, 2.55k op/s 2024-05-09T00:52:01.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:52:01 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:52:01.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:52:01 smithi157 ceph-mon[34294]: pgmap v541: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 971 MiB/s wr, 2.55k op/s 2024-05-09T00:52:01.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:52:01 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:52:03.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:52:03 smithi005 ceph-mon[30365]: pgmap v542: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 787 MiB/s wr, 2.17k op/s 2024-05-09T00:52:03.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:52:03 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:52:03.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:52:03 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:52:03.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:52:03 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:52:03.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:52:03 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:52:03.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:52:03 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:52:03.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:52:03 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:52:03.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:52:03 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:52:03.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:52:03 smithi131 ceph-mon[34547]: pgmap v542: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 787 MiB/s wr, 2.17k op/s 2024-05-09T00:52:03.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:52:03 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:52:03.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:52:03 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:52:03.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:52:03 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:52:03.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:52:03 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:52:03.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:52:03 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:52:03.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:52:03 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:52:03.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:52:03 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:52:03.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:52:03 smithi157 ceph-mon[34294]: pgmap v542: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 787 MiB/s wr, 2.17k op/s 2024-05-09T00:52:03.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:52:03 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:52:03.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:52:03 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:52:03.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:52:03 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:52:03.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:52:03 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:52:03.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:52:03 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:52:03.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:52:03 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:52:03.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:52:03 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:52:04.796 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:52:04 smithi005 ceph-mon[30365]: Adjusting osd_memory_target on smithi005 to 177.2M 2024-05-09T00:52:04.796 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:52:04 smithi005 ceph-mon[30365]: Unable to set osd_memory_target on smithi005 to 185878425: error parsing value: Value '185878425' is below minimum 939524096 2024-05-09T00:52:04.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:52:04 smithi131 ceph-mon[34547]: Adjusting osd_memory_target on smithi005 to 177.2M 2024-05-09T00:52:04.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:52:04 smithi131 ceph-mon[34547]: Unable to set osd_memory_target on smithi005 to 185878425: error parsing value: Value '185878425' is below minimum 939524096 2024-05-09T00:52:04.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:52:04 smithi157 ceph-mon[34294]: Adjusting osd_memory_target on smithi005 to 177.2M 2024-05-09T00:52:04.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:52:04 smithi157 ceph-mon[34294]: Unable to set osd_memory_target on smithi005 to 185878425: error parsing value: Value '185878425' is below minimum 939524096 2024-05-09T00:52:05.790 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:52:05 smithi005 ceph-mon[30365]: pgmap v543: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 938 MiB/s wr, 2.39k op/s 2024-05-09T00:52:05.836 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:52:05 smithi157 ceph-mon[34294]: pgmap v543: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 938 MiB/s wr, 2.39k op/s 2024-05-09T00:52:05.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:52:05 smithi131 ceph-mon[34547]: pgmap v543: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 938 MiB/s wr, 2.39k op/s 2024-05-09T00:52:07.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:52:07 smithi005 ceph-mon[30365]: pgmap v544: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 950 MiB/s wr, 2.08k op/s 2024-05-09T00:52:07.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:52:07 smithi131 ceph-mon[34547]: pgmap v544: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 950 MiB/s wr, 2.08k op/s 2024-05-09T00:52:07.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:52:07 smithi157 ceph-mon[34294]: pgmap v544: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 950 MiB/s wr, 2.08k op/s 2024-05-09T00:52:09.790 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:52:09 smithi005 ceph-mon[30365]: pgmap v545: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 832 MiB/s wr, 1.67k op/s 2024-05-09T00:52:09.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:52:09 smithi131 ceph-mon[34547]: pgmap v545: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 832 MiB/s wr, 1.67k op/s 2024-05-09T00:52:09.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:52:09 smithi157 ceph-mon[34294]: pgmap v545: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 832 MiB/s wr, 1.67k op/s 2024-05-09T00:52:11.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:52:11 smithi131 ceph-mon[34547]: pgmap v546: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 976 MiB/s wr, 2.01k op/s 2024-05-09T00:52:11.935 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:52:11 smithi157 ceph-mon[34294]: pgmap v546: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 976 MiB/s wr, 2.01k op/s 2024-05-09T00:52:12.044 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:52:11 smithi005 ceph-mon[30365]: pgmap v546: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 976 MiB/s wr, 2.01k op/s 2024-05-09T00:52:12.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.516+0000 7f06609ce640 1 -- 172.21.15.5:0/3972146120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f065c10b8d0 msgr2=0x7f065c10bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:52:12.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.516+0000 7f06609ce640 1 --2- 172.21.15.5:0/3972146120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f065c10b8d0 0x7f065c10bce0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f0654005fd0 tx=0x7f065402e6e0 comp rx=0 tx=0).stop 2024-05-09T00:52:12.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.517+0000 7f06609ce640 1 -- 172.21.15.5:0/3972146120 shutdown_connections 2024-05-09T00:52:12.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.517+0000 7f06609ce640 1 --2- 172.21.15.5:0/3972146120 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f065c1105d0 0x7f065c1129f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:12.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.517+0000 7f06609ce640 1 --2- 172.21.15.5:0/3972146120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f065c10b8d0 0x7f065c10bce0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:12.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.517+0000 7f06609ce640 1 --2- 172.21.15.5:0/3972146120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f065c10a690 0x7f065c10b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:12.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.517+0000 7f06609ce640 1 -- 172.21.15.5:0/3972146120 >> 172.21.15.5:0/3972146120 conn(0x7f065c070670 msgr2=0x7f065c072b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:52:12.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.517+0000 7f06609ce640 1 -- 172.21.15.5:0/3972146120 shutdown_connections 2024-05-09T00:52:12.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.517+0000 7f06609ce640 1 -- 172.21.15.5:0/3972146120 wait complete. 2024-05-09T00:52:12.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.519+0000 7f06609ce640 1 Processor -- start 2024-05-09T00:52:12.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.520+0000 7f06609ce640 1 -- start start 2024-05-09T00:52:12.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.520+0000 7f06609ce640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f065c10a690 0x7f065c1a0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:12.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.520+0000 7f06609ce640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f065c10b8d0 0x7f065c1a0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:12.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.521+0000 7f06609ce640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f065c1105d0 0x7f065c1a7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:12.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.521+0000 7f06609ce640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f065c115d70 con 0x7f065c1105d0 2024-05-09T00:52:12.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.521+0000 7f06609ce640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f065c115bf0 con 0x7f065c10b8d0 2024-05-09T00:52:12.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.521+0000 7f0659d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f065c10b8d0 0x7f065c1a0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:12.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.521+0000 7f0659d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f065c10b8d0 0x7f065c1a0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:53378/0 (socket says 172.21.15.5:53378) 2024-05-09T00:52:12.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.521+0000 7f0659d9b640 1 -- 172.21.15.5:0/1591472761 learned_addr learned my addr 172.21.15.5:0/1591472761 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:52:12.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.521+0000 7f06609ce640 1 -- 172.21.15.5:0/1591472761 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f065c115ef0 con 0x7f065c10a690 2024-05-09T00:52:12.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.522+0000 7f065ad9d640 1 --2- 172.21.15.5:0/1591472761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f065c1105d0 0x7f065c1a7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:12.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.524+0000 7f065a59c640 1 --2- 172.21.15.5:0/1591472761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f065c10a690 0x7f065c1a0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:12.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.524+0000 7f0659d9b640 1 -- 172.21.15.5:0/1591472761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f065c10a690 msgr2=0x7f065c1a0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:52:12.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.524+0000 7f0659d9b640 1 --2- 172.21.15.5:0/1591472761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f065c10a690 0x7f065c1a0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:12.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.524+0000 7f0659d9b640 1 -- 172.21.15.5:0/1591472761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f065c1105d0 msgr2=0x7f065c1a7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:52:12.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.524+0000 7f0659d9b640 1 --2- 172.21.15.5:0/1591472761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f065c1105d0 0x7f065c1a7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:12.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.524+0000 7f0659d9b640 1 -- 172.21.15.5:0/1591472761 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f065c1a8420 con 0x7f065c10b8d0 2024-05-09T00:52:12.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.526+0000 7f0659d9b640 1 --2- 172.21.15.5:0/1591472761 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f065c10b8d0 0x7f065c1a0ed0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0654035780 tx=0x7f0654035a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:52:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.529+0000 7f064b7fe640 1 -- 172.21.15.5:0/1591472761 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0654044020 con 0x7f065c10b8d0 2024-05-09T00:52:12.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.530+0000 7f06609ce640 1 -- 172.21.15.5:0/1591472761 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f065c1a86b0 con 0x7f065c10b8d0 2024-05-09T00:52:12.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.530+0000 7f06609ce640 1 -- 172.21.15.5:0/1591472761 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f065c1a8c00 con 0x7f065c10b8d0 2024-05-09T00:52:12.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.530+0000 7f064b7fe640 1 -- 172.21.15.5:0/1591472761 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0654003e20 con 0x7f065c10b8d0 2024-05-09T00:52:12.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.530+0000 7f064b7fe640 1 -- 172.21.15.5:0/1591472761 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065403a040 con 0x7f065c10b8d0 2024-05-09T00:52:12.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.539+0000 7f0667883740 1 Processor -- start 2024-05-09T00:52:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.539+0000 7f064b7fe640 1 -- 172.21.15.5:0/1591472761 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f0654047020 con 0x7f065c10b8d0 2024-05-09T00:52:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.540+0000 7f0667883740 1 -- start start 2024-05-09T00:52:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.540+0000 7f0667883740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5628dde671b0 0x5628de213d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.540+0000 7f0667883740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628dde3e2f0 0x5628de214290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.540+0000 7f0667883740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5628dde464d0 0x5628de21b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.540+0000 7f0667883740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5628de16f7c0 con 0x5628dde3e2f0 2024-05-09T00:52:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.540+0000 7f0667883740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5628de10b0e0 con 0x5628dde464d0 2024-05-09T00:52:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.540+0000 7f0667883740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5628de16b780 con 0x5628dde671b0 2024-05-09T00:52:12.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.541+0000 7f0659d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5628dde464d0 0x5628de21b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:12.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.541+0000 7f065ad9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5628dde671b0 0x5628de213d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:12.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.541+0000 7f065ad9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5628dde671b0 0x5628de213d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:39880/0 (socket says 172.21.15.5:39880) 2024-05-09T00:52:12.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.541+0000 7f065ad9d640 1 -- 172.21.15.5:0/4011088948 learned_addr learned my addr 172.21.15.5:0/4011088948 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:52:12.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.543+0000 7f065a59c640 1 --2- 172.21.15.5:0/4011088948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628dde3e2f0 0x5628de214290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:12.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.544+0000 7f065ad9d640 1 -- 172.21.15.5:0/4011088948 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5628dde464d0 msgr2=0x5628de21b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:52:12.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.544+0000 7f065ad9d640 1 --2- 172.21.15.5:0/4011088948 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5628dde464d0 0x5628de21b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:12.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.544+0000 7f065ad9d640 1 -- 172.21.15.5:0/4011088948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628dde3e2f0 msgr2=0x5628de214290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:52:12.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.544+0000 7f065ad9d640 1 --2- 172.21.15.5:0/4011088948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628dde3e2f0 0x5628de214290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:12.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.544+0000 7f065ad9d640 1 -- 172.21.15.5:0/4011088948 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628de1347c0 con 0x5628dde671b0 2024-05-09T00:52:12.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.545+0000 7f065ad9d640 1 --2- 172.21.15.5:0/4011088948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5628dde671b0 0x5628de213d10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f065000cbe0 tx=0x7f065000e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9262c278da4cb9e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:52:12.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.546+0000 7f0648ff9640 1 -- 172.21.15.5:0/4011088948 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0650005220 con 0x5628dde671b0 2024-05-09T00:52:12.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.546+0000 7f0648ff9640 1 -- 172.21.15.5:0/4011088948 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0650004c50 con 0x5628dde671b0 2024-05-09T00:52:12.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.546+0000 7f0648ff9640 1 -- 172.21.15.5:0/4011088948 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0650010790 con 0x5628dde671b0 2024-05-09T00:52:12.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.548+0000 7f0667883740 1 -- 172.21.15.5:0/4011088948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5628dde671b0 msgr2=0x5628de213d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:52:12.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.548+0000 7f0667883740 1 --2- 172.21.15.5:0/4011088948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5628dde671b0 0x5628de213d10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f065000cbe0 tx=0x7f065000e720 comp rx=0 tx=0).stop 2024-05-09T00:52:12.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.549+0000 7f064b7fe640 1 --2- 172.21.15.5:0/1591472761 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f06440b2930 0x7f06440b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:12.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.549+0000 7f0667883740 1 -- 172.21.15.5:0/4011088948 shutdown_connections 2024-05-09T00:52:12.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.549+0000 7f0667883740 1 --2- 172.21.15.5:0/4011088948 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5628dde464d0 0x5628de21b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:12.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.549+0000 7f0667883740 1 --2- 172.21.15.5:0/4011088948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628dde3e2f0 0x5628de214290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:12.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.549+0000 7f0667883740 1 --2- 172.21.15.5:0/4011088948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5628dde671b0 0x5628de213d10 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:12.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.549+0000 7f0667883740 1 -- 172.21.15.5:0/4011088948 >> 172.21.15.5:0/4011088948 conn(0x5628dde5b300 msgr2=0x5628de20f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:52:12.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.550+0000 7f064b7fe640 1 -- 172.21.15.5:0/1591472761 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f0654106e50 con 0x7f065c10b8d0 2024-05-09T00:52:12.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.551+0000 7f065ad9d640 1 --2- 172.21.15.5:0/1591472761 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f06440b2930 0x7f06440b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:12.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.553+0000 7f065ad9d640 1 --2- 172.21.15.5:0/1591472761 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f06440b2930 0x7f06440b4e40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f065000cbe0 tx=0x7f0650000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:52:12.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.554+0000 7f0667883740 1 -- 172.21.15.5:0/4011088948 shutdown_connections 2024-05-09T00:52:12.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.554+0000 7f0667883740 1 -- 172.21.15.5:0/4011088948 wait complete. 2024-05-09T00:52:12.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.554+0000 7f0667883740 1 -- start start 2024-05-09T00:52:12.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.555+0000 7f0667883740 1 Processor -- start 2024-05-09T00:52:12.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.555+0000 7f0667883740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628dde3e2f0 0x5628de208b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.555+0000 7f0667883740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5628dde464d0 0x5628de209080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.555+0000 7f0667883740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5628dde671b0 0x5628de20dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.555+0000 7f0667883740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5628de206b20 con 0x5628dde3e2f0 2024-05-09T00:52:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.555+0000 7f0667883740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5628de0f69c0 con 0x5628dde464d0 2024-05-09T00:52:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.555+0000 7f0667883740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5628de206ca0 con 0x5628dde671b0 2024-05-09T00:52:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.555+0000 7f0667883740 10 client.0 _ll_register_callbacks cb 0x5628de123020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:52:12.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.556+0000 7f0659d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5628dde464d0 0x5628de209080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.556+0000 7f0659d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5628dde464d0 0x5628de209080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:53402/0 (socket says 172.21.15.5:53402) 2024-05-09T00:52:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.556+0000 7f0659d9b640 1 -- 172.21.15.5:0/3376696957 learned_addr learned my addr 172.21.15.5:0/3376696957 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:52:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.556+0000 7f065a59c640 1 --2- 172.21.15.5:0/3376696957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628dde3e2f0 0x5628de208b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.557+0000 7f0659d9b640 1 -- 172.21.15.5:0/3376696957 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5628dde671b0 msgr2=0x5628de20dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:52:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.557+0000 7f0659d9b640 1 --2- 172.21.15.5:0/3376696957 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5628dde671b0 0x5628de20dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.557+0000 7f0659d9b640 1 -- 172.21.15.5:0/3376696957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628dde3e2f0 msgr2=0x5628de208b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:52:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.557+0000 7f0659d9b640 1 --2- 172.21.15.5:0/3376696957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628dde3e2f0 0x5628de208b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.557+0000 7f0659d9b640 1 -- 172.21.15.5:0/3376696957 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628de11d1f0 con 0x5628dde464d0 2024-05-09T00:52:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.558+0000 7f0659d9b640 1 --2- 172.21.15.5:0/3376696957 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5628dde464d0 0x5628de209080 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0654005e30 tx=0x7f065400a6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e80d65c2eef5f388 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:52:12.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.581+0000 7f0641ffb640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-09T00:52:12.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.581+0000 7f0641ffb640 1 -- 172.21.15.5:0/3376696957 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0654109020 con 0x5628dde464d0 2024-05-09T00:52:12.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.581+0000 7f0641ffb640 1 -- 172.21.15.5:0/3376696957 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f0654004040 con 0x5628dde464d0 2024-05-09T00:52:12.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.581+0000 7f0667883740 1 -- 172.21.15.5:0/3376696957 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0650004c50 con 0x5628dde464d0 2024-05-09T00:52:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.606+0000 7f0641ffb640 1 -- 172.21.15.5:0/3376696957 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f06540ca2a0 con 0x5628dde464d0 2024-05-09T00:52:12.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.606+0000 7f0667883740 10 client.25637 fetch_fsmap learned FSMap version 31 2024-05-09T00:52:12.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.606+0000 7f0667883740 1 -- 172.21.15.5:0/3376696957 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5628dded2790 con 0x5628dde464d0 2024-05-09T00:52:12.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.609+0000 7f0641ffb640 1 -- 172.21.15.5:0/3376696957 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0650004c50 con 0x5628dde464d0 2024-05-09T00:52:12.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.610+0000 7f0641ffb640 1 -- 172.21.15.5:0/3376696957 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f065403d030 con 0x5628dde464d0 2024-05-09T00:52:12.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.611+0000 7f0667883740 10 client.25637 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:52:12.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.611+0000 7f0667883740 10 client.25637 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:52:12.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.611+0000 7f0667883740 20 client.25637 populate_metadata read hostname 'smithi005' 2024-05-09T00:52:12.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.611+0000 7f0667883740 1 --2- 172.21.15.5:0/3376696957 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5628dde25d80 0x5628de233ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:12.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.611+0000 7f0667883740 4 client.25637 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:52:12.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.611+0000 7f0667883740 1 -- 172.21.15.5:0/3376696957 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5628de17d3d0 con 0x5628dde25d80 2024-05-09T00:52:12.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.614+0000 7f065a59c640 1 --2- 172.21.15.5:0/3376696957 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5628dde25d80 0x5628de233ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:12.623 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.622+0000 7f0641ffb640 10 client.25637 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:52:12.623 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.622+0000 7f065a59c640 1 --2- 172.21.15.5:0/3376696957 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5628dde25d80 0x5628de233ec0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc3460b374a340c0 server_cookie=83872d3831390ca5 in_seq=0 out_seq=0 2024-05-09T00:52:12.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.626+0000 7f0641ffb640 1 -- 172.21.15.5:0/3376696957 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5628de17d3d0 con 0x5628dde25d80 2024-05-09T00:52:12.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.626+0000 7f0641ffb640 10 client.25637 handle_command_reply: tid=0 2024-05-09T00:52:12.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.626+0000 7f0667883740 1 client.25637 shutdown 2024-05-09T00:52:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.627+0000 7f0667883740 1 -- 172.21.15.5:0/3376696957 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5628dde464d0 msgr2=0x5628de209080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:52:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.627+0000 7f0667883740 1 --2- 172.21.15.5:0/3376696957 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5628dde464d0 0x5628de209080 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0654005e30 tx=0x7f065400a6c0 comp rx=0 tx=0).stop 2024-05-09T00:52:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.627+0000 7f0667883740 1 -- 172.21.15.5:0/3376696957 shutdown_connections 2024-05-09T00:52:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.627+0000 7f0667883740 1 --2- 172.21.15.5:0/3376696957 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5628dde25d80 0x5628de233ec0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.627+0000 7f0667883740 1 --2- 172.21.15.5:0/3376696957 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5628dde671b0 0x5628de20dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.627+0000 7f0667883740 1 --2- 172.21.15.5:0/3376696957 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5628dde464d0 0x5628de209080 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.627+0000 7f0667883740 1 --2- 172.21.15.5:0/3376696957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628dde3e2f0 0x5628de208b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.627+0000 7f0641ffb640 0 client.25637 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:52:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.627+0000 7f0667883740 1 -- 172.21.15.5:0/3376696957 >> 172.21.15.5:0/3376696957 conn(0x5628dde5b300 msgr2=0x5628de205700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:52:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.628+0000 7f0667883740 1 -- 172.21.15.5:0/3376696957 shutdown_connections 2024-05-09T00:52:12.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.628+0000 7f0667883740 1 -- 172.21.15.5:0/3376696957 wait complete. 2024-05-09T00:52:12.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.628+0000 7f0667883740 20 client.25637 trim_cache size 0 max 16384 2024-05-09T00:52:12.678 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.677+0000 7f0667883740 1 Processor -- start 2024-05-09T00:52:12.678 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.677+0000 7f0667883740 1 -- start start 2024-05-09T00:52:12.678 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.678+0000 7f0667883740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5628dde3e2f0 0x5628de217fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:12.678 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.678+0000 7f0667883740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628dde464d0 0x5628de2184e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:12.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.678+0000 7f0667883740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5628dde671b0 0x5628de205700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:12.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.679+0000 7f0667883740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f063c00e4b0 con 0x5628dde464d0 2024-05-09T00:52:12.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.679+0000 7f0667883740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f063c00fff0 con 0x5628dde671b0 2024-05-09T00:52:12.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.679+0000 7f0667883740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f063c00eb80 con 0x5628dde3e2f0 2024-05-09T00:52:12.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.679+0000 7f065a59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5628dde3e2f0 0x5628de217fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:12.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.679+0000 7f065a59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5628dde3e2f0 0x5628de217fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:39898/0 (socket says 172.21.15.5:39898) 2024-05-09T00:52:12.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.679+0000 7f065a59c640 1 -- 172.21.15.5:0/127320242 learned_addr learned my addr 172.21.15.5:0/127320242 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:52:12.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.680+0000 7f065ad9d640 1 --2- 172.21.15.5:0/127320242 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5628dde671b0 0x5628de205700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:12.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.681+0000 7f0659d9b640 1 --2- 172.21.15.5:0/127320242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628dde464d0 0x5628de2184e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:12.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.681+0000 7f065ad9d640 1 -- 172.21.15.5:0/127320242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5628dde3e2f0 msgr2=0x5628de217fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:52:12.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.681+0000 7f065ad9d640 1 --2- 172.21.15.5:0/127320242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5628dde3e2f0 0x5628de217fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:12.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.681+0000 7f065ad9d640 1 -- 172.21.15.5:0/127320242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628dde464d0 msgr2=0x5628de2184e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:52:12.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.681+0000 7f065ad9d640 1 --2- 172.21.15.5:0/127320242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628dde464d0 0x5628de2184e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:12.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.681+0000 7f065ad9d640 1 -- 172.21.15.5:0/127320242 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628ddf495f0 con 0x5628dde671b0 2024-05-09T00:52:12.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.682+0000 7f065ad9d640 1 --2- 172.21.15.5:0/127320242 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5628dde671b0 0x5628de205700 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f065000c970 tx=0x7f065000c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96d1e10efed82195 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:52:12.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.703+0000 7f0643fff640 1 -- 172.21.15.5:0/127320242 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0650004050 con 0x5628dde671b0 2024-05-09T00:52:12.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.703+0000 7f0643fff640 1 -- 172.21.15.5:0/127320242 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06500035f0 con 0x5628dde671b0 2024-05-09T00:52:12.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.703+0000 7f0643fff640 1 -- 172.21.15.5:0/127320242 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0650028680 con 0x5628dde671b0 2024-05-09T00:52:12.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.707+0000 7f0667883740 1 -- 172.21.15.5:0/127320242 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5628dde671b0 msgr2=0x5628de205700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:52:12.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.707+0000 7f0667883740 1 --2- 172.21.15.5:0/127320242 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5628dde671b0 0x5628de205700 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f065000c970 tx=0x7f065000c4d0 comp rx=0 tx=0).stop 2024-05-09T00:52:12.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.707+0000 7f0667883740 1 -- 172.21.15.5:0/127320242 shutdown_connections 2024-05-09T00:52:12.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.707+0000 7f0667883740 1 --2- 172.21.15.5:0/127320242 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5628dde671b0 0x5628de205700 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:12.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.707+0000 7f0667883740 1 --2- 172.21.15.5:0/127320242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628dde464d0 0x5628de2184e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:12.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.707+0000 7f0667883740 1 --2- 172.21.15.5:0/127320242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5628dde3e2f0 0x5628de217fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:12.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.707+0000 7f0667883740 1 -- 172.21.15.5:0/127320242 >> 172.21.15.5:0/127320242 conn(0x5628dde5b300 msgr2=0x5628de207190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:52:12.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.709+0000 7f0667883740 1 -- 172.21.15.5:0/127320242 shutdown_connections 2024-05-09T00:52:12.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.709+0000 7f0667883740 1 -- 172.21.15.5:0/127320242 wait complete. 2024-05-09T00:52:12.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.710+0000 7f0667883740 1 -- start start 2024-05-09T00:52:12.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.710+0000 7f0667883740 1 Processor -- start 2024-05-09T00:52:12.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.711+0000 7f0667883740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628dde3e2f0 0x5628de242e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:12.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.711+0000 7f0667883740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5628dde464d0 0x5628de243390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:12.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.711+0000 7f0667883740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5628dde671b0 0x5628de244270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:12.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.711+0000 7f0667883740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f063c00f920 con 0x5628dde3e2f0 2024-05-09T00:52:12.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.711+0000 7f0667883740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f063c00f250 con 0x5628dde671b0 2024-05-09T00:52:12.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.711+0000 7f0667883740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f063c00c540 con 0x5628dde464d0 2024-05-09T00:52:12.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.711+0000 7f0667883740 10 client.0 _ll_register_callbacks cb 0x7f065400ad60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:52:12.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.712+0000 7f0659d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5628dde464d0 0x5628de243390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:12.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.712+0000 7f0659d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5628dde464d0 0x5628de243390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:39900/0 (socket says 172.21.15.5:39900) 2024-05-09T00:52:12.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.712+0000 7f0659d9b640 1 -- 172.21.15.5:0/1455804308 learned_addr learned my addr 172.21.15.5:0/1455804308 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:52:12.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.713+0000 7f065a59c640 1 --2- 172.21.15.5:0/1455804308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628dde3e2f0 0x5628de242e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:12.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.713+0000 7f0659d9b640 1 -- 172.21.15.5:0/1455804308 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5628dde671b0 msgr2=0x5628de244270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:52:12.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.713+0000 7f065ad9d640 1 --2- 172.21.15.5:0/1455804308 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5628dde671b0 0x5628de244270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:12.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.714+0000 7f0659d9b640 1 --2- 172.21.15.5:0/1455804308 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5628dde671b0 0x5628de244270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:12.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.714+0000 7f0659d9b640 1 -- 172.21.15.5:0/1455804308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628dde3e2f0 msgr2=0x5628de242e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:52:12.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.714+0000 7f0659d9b640 1 --2- 172.21.15.5:0/1455804308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628dde3e2f0 0x5628de242e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:12.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.714+0000 7f0659d9b640 1 -- 172.21.15.5:0/1455804308 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628de133ed0 con 0x5628dde464d0 2024-05-09T00:52:12.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.714+0000 7f065a59c640 1 --2- 172.21.15.5:0/1455804308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628dde3e2f0 0x5628de242e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:52:12.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.714+0000 7f065ad9d640 1 --2- 172.21.15.5:0/1455804308 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5628dde671b0 0x5628de244270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:52:12.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.715+0000 7f0659d9b640 1 --2- 172.21.15.5:0/1455804308 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5628dde464d0 0x5628de243390 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0654036f90 tx=0x7f065400e4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2768ce6e05ef24e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:52:12.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.715+0000 7f0641ffb640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-09T00:52:12.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.716+0000 7f0641ffb640 1 -- 172.21.15.5:0/1455804308 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0654109070 con 0x5628dde464d0 2024-05-09T00:52:12.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.716+0000 7f0667883740 1 -- 172.21.15.5:0/1455804308 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06500035f0 con 0x5628dde464d0 2024-05-09T00:52:12.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.716+0000 7f0641ffb640 1 -- 172.21.15.5:0/1455804308 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f06540ca9d0 con 0x5628dde464d0 2024-05-09T00:52:12.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.717+0000 7f0641ffb640 1 -- 172.21.15.5:0/1455804308 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0654004240 con 0x5628dde464d0 2024-05-09T00:52:12.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.717+0000 7f0667883740 10 client.25669 fetch_fsmap learned FSMap version 31 2024-05-09T00:52:12.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.717+0000 7f0667883740 1 -- 172.21.15.5:0/1455804308 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5628de2458f0 con 0x5628dde464d0 2024-05-09T00:52:12.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.723+0000 7f0641ffb640 1 -- 172.21.15.5:0/1455804308 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06500035f0 con 0x5628dde464d0 2024-05-09T00:52:12.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.723+0000 7f0641ffb640 1 -- 172.21.15.5:0/1455804308 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f06540d5c20 con 0x5628dde464d0 2024-05-09T00:52:12.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.724+0000 7f0667883740 10 client.25669 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:52:12.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.724+0000 7f0667883740 10 client.25669 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:52:12.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.724+0000 7f0667883740 20 client.25669 populate_metadata read hostname 'smithi005' 2024-05-09T00:52:12.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.724+0000 7f0667883740 1 --2- 172.21.15.5:0/1455804308 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5628dddfb1c0 0x5628de29e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:12.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.724+0000 7f0667883740 4 client.25669 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:52:12.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.724+0000 7f0667883740 1 -- 172.21.15.5:0/1455804308 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5628de177240 con 0x5628dddfb1c0 2024-05-09T00:52:12.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.728+0000 7f065a59c640 1 --2- 172.21.15.5:0/1455804308 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5628dddfb1c0 0x5628de29e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:12.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.731+0000 7f065a59c640 1 --2- 172.21.15.5:0/1455804308 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5628dddfb1c0 0x5628de29e320 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8e62b6c9ac3cd4d server_cookie=18e3f03b55d23ee2 in_seq=0 out_seq=0 2024-05-09T00:52:12.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:12.731+0000 7f0641ffb640 10 client.25669 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:52:12.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:52:12 smithi131 ceph-mon[34547]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T00:52:12.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:52:12 smithi131 ceph-mon[34547]: pgmap v547: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 815 MiB/s wr, 1.72k op/s 2024-05-09T00:52:12.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:52:12 smithi157 ceph-mon[34294]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T00:52:12.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:52:12 smithi157 ceph-mon[34294]: pgmap v547: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 815 MiB/s wr, 1.72k op/s 2024-05-09T00:52:13.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:52:12 smithi005 ceph-mon[30365]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T00:52:13.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:52:12 smithi005 ceph-mon[30365]: pgmap v547: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 815 MiB/s wr, 1.72k op/s 2024-05-09T00:52:15.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:52:15 smithi157 ceph-mon[34294]: pgmap v548: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 931 MiB/s wr, 2.00k op/s 2024-05-09T00:52:15.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:52:15 smithi005 ceph-mon[30365]: pgmap v548: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 931 MiB/s wr, 2.00k op/s 2024-05-09T00:52:15.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:52:15 smithi131 ceph-mon[34547]: pgmap v548: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 931 MiB/s wr, 2.00k op/s 2024-05-09T00:52:17.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:52:17 smithi157 ceph-mon[34294]: pgmap v549: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 931 MiB/s wr, 2.13k op/s 2024-05-09T00:52:17.543 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:52:17 smithi005 ceph-mon[30365]: pgmap v549: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 931 MiB/s wr, 2.13k op/s 2024-05-09T00:52:17.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:52:17 smithi131 ceph-mon[34547]: pgmap v549: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 931 MiB/s wr, 2.13k op/s 2024-05-09T00:52:19.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:52:19 smithi157 ceph-mon[34294]: pgmap v550: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 813 MiB/s wr, 1.88k op/s 2024-05-09T00:52:19.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:52:19 smithi005 ceph-mon[30365]: pgmap v550: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 813 MiB/s wr, 1.88k op/s 2024-05-09T00:52:19.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:52:19 smithi131 ceph-mon[34547]: pgmap v550: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 813 MiB/s wr, 1.88k op/s 2024-05-09T00:52:21.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:21.421+0000 7f0641ffb640 1 -- 172.21.15.5:0/1455804308 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5628de177240 con 0x5628dddfb1c0 2024-05-09T00:52:21.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:21.422+0000 7f0641ffb640 10 client.25669 handle_command_reply: tid=0 2024-05-09T00:52:21.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:21.422+0000 7f0667883740 1 client.25669 shutdown 2024-05-09T00:52:21.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:21.424+0000 7f0667883740 1 -- 172.21.15.5:0/1455804308 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5628dde464d0 msgr2=0x5628de243390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:52:21.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:21.424+0000 7f0667883740 1 --2- 172.21.15.5:0/1455804308 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5628dde464d0 0x5628de243390 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0654036f90 tx=0x7f065400e4b0 comp rx=0 tx=0).stop 2024-05-09T00:52:21.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:21.425+0000 7f0667883740 1 -- 172.21.15.5:0/1455804308 shutdown_connections 2024-05-09T00:52:21.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:21.425+0000 7f0667883740 1 --2- 172.21.15.5:0/1455804308 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5628dddfb1c0 0x5628de29e320 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:21.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:21.425+0000 7f0667883740 1 --2- 172.21.15.5:0/1455804308 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5628dde671b0 0x5628de244270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:21.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:21.425+0000 7f0667883740 1 --2- 172.21.15.5:0/1455804308 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5628dde464d0 0x5628de243390 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:21.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:21.425+0000 7f0667883740 1 --2- 172.21.15.5:0/1455804308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628dde3e2f0 0x5628de242e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:21.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:21.425+0000 7f0667883740 1 -- 172.21.15.5:0/1455804308 >> 172.21.15.5:0/1455804308 conn(0x5628dde5b300 msgr2=0x5628de2070d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:52:21.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:21.425+0000 7f0641ffb640 0 client.25669 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:52:21.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:21.425+0000 7f0667883740 1 -- 172.21.15.5:0/1455804308 shutdown_connections 2024-05-09T00:52:21.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:21.425+0000 7f0667883740 1 -- 172.21.15.5:0/1455804308 wait complete. 2024-05-09T00:52:21.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:21.426+0000 7f0667883740 20 client.25669 trim_cache size 0 max 16384 2024-05-09T00:52:21.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-09T00:52:21.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-09T00:52:21.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-09T00:52:21.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:21.429+0000 7f06609ce640 1 -- 172.21.15.5:0/1591472761 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f06440b2930 msgr2=0x7f06440b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:52:21.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:21.430+0000 7f06609ce640 1 --2- 172.21.15.5:0/1591472761 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f06440b2930 0x7f06440b4e40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f065000cbe0 tx=0x7f0650000bc0 comp rx=0 tx=0).stop 2024-05-09T00:52:21.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:21.430+0000 7f06609ce640 1 -- 172.21.15.5:0/1591472761 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f065c10b8d0 msgr2=0x7f065c1a0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:52:21.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:21.430+0000 7f06609ce640 1 --2- 172.21.15.5:0/1591472761 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f065c10b8d0 0x7f065c1a0ed0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0654035780 tx=0x7f0654035a50 comp rx=0 tx=0).stop 2024-05-09T00:52:21.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:21.430+0000 7f06609ce640 1 -- 172.21.15.5:0/1591472761 shutdown_connections 2024-05-09T00:52:21.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:21.431+0000 7f06609ce640 1 --2- 172.21.15.5:0/1591472761 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f06440b2930 0x7f06440b4e40 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:21.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:21.431+0000 7f06609ce640 1 --2- 172.21.15.5:0/1591472761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f065c1105d0 0x7f065c1a7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:21.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:21.431+0000 7f06609ce640 1 --2- 172.21.15.5:0/1591472761 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f065c10b8d0 0x7f065c1a0ed0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:21.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:21.431+0000 7f06609ce640 1 --2- 172.21.15.5:0/1591472761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f065c10a690 0x7f065c1a0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:21.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:21.431+0000 7f06609ce640 1 -- 172.21.15.5:0/1591472761 >> 172.21.15.5:0/1591472761 conn(0x7f065c070670 msgr2=0x7f065c10c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:52:21.435 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:52:21 smithi157 ceph-mon[34294]: pgmap v551: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 965 MiB/s wr, 1.99k op/s 2024-05-09T00:52:21.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:21.436+0000 7f06609ce640 1 -- 172.21.15.5:0/1591472761 shutdown_connections 2024-05-09T00:52:21.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:21.436+0000 7f06609ce640 1 -- 172.21.15.5:0/1591472761 wait complete. 2024-05-09T00:52:21.543 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:52:21 smithi005 ceph-mon[30365]: pgmap v551: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 965 MiB/s wr, 1.99k op/s 2024-05-09T00:52:21.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:52:21 smithi131 ceph-mon[34547]: pgmap v551: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 965 MiB/s wr, 1.99k op/s 2024-05-09T00:52:23.547 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:52:23 smithi005 ceph-mon[30365]: pgmap v552: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 818 MiB/s wr, 1.51k op/s 2024-05-09T00:52:23.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:52:23 smithi131 ceph-mon[34547]: pgmap v552: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 818 MiB/s wr, 1.51k op/s 2024-05-09T00:52:23.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:52:23 smithi157 ceph-mon[34294]: pgmap v552: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 818 MiB/s wr, 1.51k op/s 2024-05-09T00:52:25.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:52:25 smithi005 ceph-mon[30365]: pgmap v553: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 930 MiB/s wr, 1.63k op/s 2024-05-09T00:52:25.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:52:25 smithi131 ceph-mon[34547]: pgmap v553: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 930 MiB/s wr, 1.63k op/s 2024-05-09T00:52:25.628 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T00:52:25.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:52:25 smithi157 ceph-mon[34294]: pgmap v553: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 930 MiB/s wr, 1.63k op/s 2024-05-09T00:52:26.560 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:52:27.535 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:52:27 smithi005 ceph-mon[30365]: pgmap v554: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 943 MiB/s wr, 1.53k op/s 2024-05-09T00:52:27.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:52:27 smithi131 ceph-mon[34547]: pgmap v554: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 943 MiB/s wr, 1.53k op/s 2024-05-09T00:52:27.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:52:27 smithi157 ceph-mon[34294]: pgmap v554: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 943 MiB/s wr, 1.53k op/s 2024-05-09T00:52:29.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:52:29 smithi005 ceph-mon[30365]: pgmap v555: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 828 MiB/s wr, 1.20k op/s 2024-05-09T00:52:29.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:52:29 smithi131 ceph-mon[34547]: pgmap v555: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 828 MiB/s wr, 1.20k op/s 2024-05-09T00:52:29.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:52:29 smithi157 ceph-mon[34294]: pgmap v555: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 828 MiB/s wr, 1.20k op/s 2024-05-09T00:52:31.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:52:31 smithi005 ceph-mon[30365]: pgmap v556: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 973 MiB/s wr, 1.44k op/s 2024-05-09T00:52:31.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:52:31 smithi131 ceph-mon[34547]: pgmap v556: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 973 MiB/s wr, 1.44k op/s 2024-05-09T00:52:31.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:52:31 smithi157 ceph-mon[34294]: pgmap v556: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 973 MiB/s wr, 1.44k op/s 2024-05-09T00:52:32.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:31.990+0000 7f8146310700 1 -- 172.21.15.5:0/2204963894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8140108740 msgr2=0x7f81401100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:52:32.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:31.990+0000 7f8146310700 1 --2- 172.21.15.5:0/2204963894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8140108740 0x7f81401100a0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f813800b0d0 tx=0x7f813800b3e0 comp rx=0 tx=0).stop 2024-05-09T00:52:32.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:31.990+0000 7f8146310700 1 -- 172.21.15.5:0/2204963894 shutdown_connections 2024-05-09T00:52:32.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:31.990+0000 7f8146310700 1 --2- 172.21.15.5:0/2204963894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8140108740 0x7f81401100a0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:32.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:31.990+0000 7f8146310700 1 --2- 172.21.15.5:0/2204963894 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81401034c0 0x7f81401038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:32.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:31.990+0000 7f8146310700 1 --2- 172.21.15.5:0/2204963894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8140107df0 0x7f8140108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:32.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:31.990+0000 7f8146310700 1 -- 172.21.15.5:0/2204963894 >> 172.21.15.5:0/2204963894 conn(0x7f81400fedf0 msgr2=0x7f8140101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:52:32.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:31.991+0000 7f8146310700 1 -- 172.21.15.5:0/2204963894 shutdown_connections 2024-05-09T00:52:32.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:31.991+0000 7f8146310700 1 -- 172.21.15.5:0/2204963894 wait complete. 2024-05-09T00:52:32.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:31.993+0000 7f8146310700 1 Processor -- start 2024-05-09T00:52:32.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:31.993+0000 7f8146310700 1 -- start start 2024-05-09T00:52:32.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:31.994+0000 7f8146310700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81401034c0 0x7f814019d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:32.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:31.994+0000 7f8146310700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8140107df0 0x7f814019d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:32.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:31.994+0000 7f813ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81401034c0 0x7f814019d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:32.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:31.995+0000 7f8146310700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8140108740 0x7f81401a1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:32.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:31.995+0000 7f8146310700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8140113440 con 0x7f81401034c0 2024-05-09T00:52:32.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:31.995+0000 7f8146310700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f81401132c0 con 0x7f8140108740 2024-05-09T00:52:32.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:31.995+0000 7f8146310700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f81401135c0 con 0x7f8140107df0 2024-05-09T00:52:32.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:31.994+0000 7f813ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81401034c0 0x7f814019d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41186/0 (socket says 172.21.15.5:41186) 2024-05-09T00:52:32.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:31.995+0000 7f813ffff700 1 -- 172.21.15.5:0/2764818895 learned_addr learned my addr 172.21.15.5:0/2764818895 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:52:32.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:31.996+0000 7f813ffff700 1 -- 172.21.15.5:0/2764818895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8140107df0 msgr2=0x7f814019d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:52:32.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:31.996+0000 7f813ffff700 1 --2- 172.21.15.5:0/2764818895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8140107df0 0x7f814019d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:32.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:31.996+0000 7f813ffff700 1 -- 172.21.15.5:0/2764818895 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8140108740 msgr2=0x7f81401a1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:52:32.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:31.996+0000 7f813ffff700 1 --2- 172.21.15.5:0/2764818895 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8140108740 0x7f81401a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:32.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:31.996+0000 7f813ffff700 1 -- 172.21.15.5:0/2764818895 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81401a1f30 con 0x7f81401034c0 2024-05-09T00:52:32.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:31.996+0000 7f813ffff700 1 --2- 172.21.15.5:0/2764818895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81401034c0 0x7f814019d060 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f813000b530 tx=0x7f813000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:52:32.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:31.996+0000 7f813dffb700 1 -- 172.21.15.5:0/2764818895 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8130015020 con 0x7f81401034c0 2024-05-09T00:52:32.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:31.997+0000 7f8146310700 1 -- 172.21.15.5:0/2764818895 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81401a2220 con 0x7f81401034c0 2024-05-09T00:52:32.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:31.997+0000 7f813dffb700 1 -- 172.21.15.5:0/2764818895 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8130011ce0 con 0x7f81401034c0 2024-05-09T00:52:32.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:31.997+0000 7f813dffb700 1 -- 172.21.15.5:0/2764818895 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8130010d20 con 0x7f81401034c0 2024-05-09T00:52:32.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:31.997+0000 7f8146310700 1 -- 172.21.15.5:0/2764818895 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81401a9b40 con 0x7f81401034c0 2024-05-09T00:52:32.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.003+0000 7f813dffb700 1 -- 172.21.15.5:0/2764818895 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f8130018020 con 0x7f81401034c0 2024-05-09T00:52:32.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.004+0000 7f8154d26b80 1 Processor -- start 2024-05-09T00:52:32.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.004+0000 7f8154d26b80 1 -- start start 2024-05-09T00:52:32.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.005+0000 7f8154d26b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c40721d00 0x555c40832a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:32.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.005+0000 7f8154d26b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555c406a74a0 0x555c40832fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:32.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.005+0000 7f8154d26b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c4082cc10 0x555c4082d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:32.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.005+0000 7f8154d26b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555c407470a0 con 0x555c4082cc10 2024-05-09T00:52:32.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.005+0000 7f8154d26b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x555c406ec9a0 con 0x555c406a74a0 2024-05-09T00:52:32.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.005+0000 7f8154d26b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x555c40743060 con 0x555c40721d00 2024-05-09T00:52:32.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.006+0000 7f813ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555c406a74a0 0x555c40832fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:32.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.007+0000 7f813dffb700 1 --2- 172.21.15.5:0/2764818895 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f812c0b2900 0x7f812c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:32.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.007+0000 7f813ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555c406a74a0 0x555c40832fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:57980/0 (socket says 172.21.15.5:57980) 2024-05-09T00:52:32.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.007+0000 7f813ffff700 1 -- 172.21.15.5:0/80213073 learned_addr learned my addr 172.21.15.5:0/80213073 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:52:32.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.007+0000 7f81448ad700 1 --2- 172.21.15.5:0/80213073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c40721d00 0x555c40832a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:32.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.007+0000 7f813dffb700 1 -- 172.21.15.5:0/2764818895 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f81300e19f0 con 0x7f81401034c0 2024-05-09T00:52:32.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.008+0000 7f81448ad700 1 --2- 172.21.15.5:0/2764818895 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f812c0b2900 0x7f812c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:32.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.008+0000 7f8137fff700 1 --2- 172.21.15.5:0/80213073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c4082cc10 0x555c4082d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:32.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.008+0000 7f813ffff700 1 -- 172.21.15.5:0/80213073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c40721d00 msgr2=0x555c40832a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:52:32.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.008+0000 7f813ffff700 1 --2- 172.21.15.5:0/80213073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c40721d00 0x555c40832a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:32.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.008+0000 7f813ffff700 1 -- 172.21.15.5:0/80213073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c4082cc10 msgr2=0x555c4082d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:52:32.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.008+0000 7f813ffff700 1 --2- 172.21.15.5:0/80213073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c4082cc10 0x555c4082d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:32.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.008+0000 7f813ffff700 1 -- 172.21.15.5:0/80213073 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c406dd500 con 0x555c406a74a0 2024-05-09T00:52:32.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.009+0000 7f81448ad700 1 --2- 172.21.15.5:0/2764818895 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f812c0b2900 0x7f812c0b4e10 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f813800d010 tx=0x7f8138005b60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:52:32.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.009+0000 7f813ffff700 1 --2- 172.21.15.5:0/80213073 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555c406a74a0 0x555c40832fc0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f81300a6250 tx=0x7f8130011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5eda868d3e06f70 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:52:32.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.010+0000 7f81367fc700 1 -- 172.21.15.5:0/80213073 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81300a9440 con 0x555c406a74a0 2024-05-09T00:52:32.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.010+0000 7f81367fc700 1 -- 172.21.15.5:0/80213073 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81300a96e0 con 0x555c406a74a0 2024-05-09T00:52:32.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.010+0000 7f81367fc700 1 -- 172.21.15.5:0/80213073 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81300a8240 con 0x555c406a74a0 2024-05-09T00:52:32.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.011+0000 7f8154d26b80 1 -- 172.21.15.5:0/80213073 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555c406a74a0 msgr2=0x555c40832fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:52:32.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.011+0000 7f8154d26b80 1 --2- 172.21.15.5:0/80213073 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555c406a74a0 0x555c40832fc0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f81300a6250 tx=0x7f8130011000 comp rx=0 tx=0).stop 2024-05-09T00:52:32.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.018+0000 7f8154d26b80 1 -- 172.21.15.5:0/80213073 shutdown_connections 2024-05-09T00:52:32.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.018+0000 7f8154d26b80 1 --2- 172.21.15.5:0/80213073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c4082cc10 0x555c4082d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:32.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.018+0000 7f8154d26b80 1 --2- 172.21.15.5:0/80213073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c40721d00 0x555c40832a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:32.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.018+0000 7f8154d26b80 1 --2- 172.21.15.5:0/80213073 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555c406a74a0 0x555c40832fc0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:32.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.018+0000 7f8154d26b80 1 -- 172.21.15.5:0/80213073 >> 172.21.15.5:0/80213073 conn(0x555c406aaf70 msgr2=0x555c4081f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:52:32.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.030+0000 7f8154d26b80 1 -- 172.21.15.5:0/80213073 shutdown_connections 2024-05-09T00:52:32.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.030+0000 7f8154d26b80 1 -- 172.21.15.5:0/80213073 wait complete. 2024-05-09T00:52:32.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.031+0000 7f8154d26b80 1 -- start start 2024-05-09T00:52:32.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.032+0000 7f8154d26b80 1 Processor -- start 2024-05-09T00:52:32.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.033+0000 7f8154d26b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c406a74a0 0x555c408368e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:32.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.033+0000 7f8154d26b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555c40721d00 0x555c40836e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:32.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.033+0000 7f8154d26b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c4082cc10 0x555c40837360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:32.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.033+0000 7f813ffff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c406a74a0 0x555c408368e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:32.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.034+0000 7f813ffff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c406a74a0 0x555c408368e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:46662/0 (socket says 172.21.15.5:46662) 2024-05-09T00:52:32.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.034+0000 7f813ffff700 1 -- 172.21.15.5:0/2293449884 learned_addr learned my addr 172.21.15.5:0/2293449884 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:52:32.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.034+0000 7f81448ad700 1 --2- 172.21.15.5:0/2293449884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c4082cc10 0x555c40837360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:32.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.034+0000 7f8154d26b80 1 -- 172.21.15.5:0/2293449884 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555c4082d970 con 0x555c4082cc10 2024-05-09T00:52:32.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.034+0000 7f8154d26b80 1 -- 172.21.15.5:0/2293449884 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x555c406e1b20 con 0x555c40721d00 2024-05-09T00:52:32.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.034+0000 7f8154d26b80 1 -- 172.21.15.5:0/2293449884 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x555c40839820 con 0x555c406a74a0 2024-05-09T00:52:32.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.035+0000 7f8154d26b80 10 client.0 _ll_register_callbacks cb 0x555c405b86c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:52:32.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.035+0000 7f8137fff700 1 --2- 172.21.15.5:0/2293449884 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555c40721d00 0x555c40836e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:32.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.036+0000 7f81448ad700 1 -- 172.21.15.5:0/2293449884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c406a74a0 msgr2=0x555c408368e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:52:32.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.036+0000 7f81448ad700 1 --2- 172.21.15.5:0/2293449884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c406a74a0 0x555c408368e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:32.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.036+0000 7f81448ad700 1 -- 172.21.15.5:0/2293449884 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555c40721d00 msgr2=0x555c40836e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:52:32.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.036+0000 7f81448ad700 1 --2- 172.21.15.5:0/2293449884 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555c40721d00 0x555c40836e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:32.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.036+0000 7f81448ad700 1 -- 172.21.15.5:0/2293449884 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c405c2490 con 0x555c4082cc10 2024-05-09T00:52:32.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.038+0000 7f81448ad700 1 --2- 172.21.15.5:0/2293449884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c4082cc10 0x555c40837360 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f8138009900 tx=0x7f8138006210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55f7c648a417c93c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:52:32.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.039+0000 7f811f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:52:32.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.040+0000 7f811f7fe700 1 -- 172.21.15.5:0/2293449884 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8138030060 con 0x555c4082cc10 2024-05-09T00:52:32.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.041+0000 7f8154d26b80 1 -- 172.21.15.5:0/2293449884 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81300a96e0 con 0x555c4082cc10 2024-05-09T00:52:32.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.041+0000 7f811f7fe700 1 -- 172.21.15.5:0/2293449884 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f8138005040 con 0x555c4082cc10 2024-05-09T00:52:32.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.042+0000 7f811f7fe700 1 -- 172.21.15.5:0/2293449884 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8138005240 con 0x555c4082cc10 2024-05-09T00:52:32.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.042+0000 7f8154d26b80 10 client.16125 fetch_fsmap learned FSMap version 31 2024-05-09T00:52:32.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.042+0000 7f8154d26b80 1 -- 172.21.15.5:0/2293449884 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x555c40844240 con 0x555c4082cc10 2024-05-09T00:52:32.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.045+0000 7f811f7fe700 1 -- 172.21.15.5:0/2293449884 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81300a96e0 con 0x555c4082cc10 2024-05-09T00:52:32.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.047+0000 7f811f7fe700 1 -- 172.21.15.5:0/2293449884 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f8138011070 con 0x555c4082cc10 2024-05-09T00:52:32.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.047+0000 7f8154d26b80 10 client.16125 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:52:32.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.047+0000 7f8154d26b80 10 client.16125 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:52:32.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.047+0000 7f8154d26b80 20 client.16125 populate_metadata read hostname 'smithi005' 2024-05-09T00:52:32.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.047+0000 7f8154d26b80 1 --2- 172.21.15.5:0/2293449884 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x555c40846b80 0x555c40848fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:32.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.047+0000 7f8154d26b80 4 client.16125 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:52:32.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.047+0000 7f8154d26b80 1 -- 172.21.15.5:0/2293449884 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c40754a80 con 0x555c40846b80 2024-05-09T00:52:32.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.053+0000 7f813ffff700 1 --2- 172.21.15.5:0/2293449884 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x555c40846b80 0x555c40848fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:32.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.062+0000 7f813ffff700 1 --2- 172.21.15.5:0/2293449884 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x555c40846b80 0x555c40848fe0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32d74a235d2c5036 server_cookie=cc519e8053b6f4b7 in_seq=0 out_seq=0 2024-05-09T00:52:32.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.063+0000 7f811f7fe700 10 client.16125 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:52:32.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.067+0000 7f811f7fe700 1 -- 172.21.15.5:0/2293449884 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555c40754a80 con 0x555c40846b80 2024-05-09T00:52:32.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.067+0000 7f811f7fe700 10 client.16125 handle_command_reply: tid=0 2024-05-09T00:52:32.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.068+0000 7f8154d26b80 1 client.16125 shutdown 2024-05-09T00:52:32.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.068+0000 7f8154d26b80 1 -- 172.21.15.5:0/2293449884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c4082cc10 msgr2=0x555c40837360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:52:32.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.068+0000 7f8154d26b80 1 --2- 172.21.15.5:0/2293449884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c4082cc10 0x555c40837360 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f8138009900 tx=0x7f8138006210 comp rx=0 tx=0).stop 2024-05-09T00:52:32.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.069+0000 7f8154d26b80 1 -- 172.21.15.5:0/2293449884 shutdown_connections 2024-05-09T00:52:32.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.069+0000 7f8154d26b80 1 --2- 172.21.15.5:0/2293449884 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x555c40846b80 0x555c40848fe0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:32.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.070+0000 7f8154d26b80 1 --2- 172.21.15.5:0/2293449884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c4082cc10 0x555c40837360 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:32.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.070+0000 7f8154d26b80 1 --2- 172.21.15.5:0/2293449884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c406a74a0 0x555c408368e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:32.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.070+0000 7f8154d26b80 1 --2- 172.21.15.5:0/2293449884 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555c40721d00 0x555c40836e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:32.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.070+0000 7f8154d26b80 1 -- 172.21.15.5:0/2293449884 >> 172.21.15.5:0/2293449884 conn(0x555c406aaf70 msgr2=0x555c408233f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:52:32.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.070+0000 7f811f7fe700 0 client.16125 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:52:32.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.070+0000 7f8154d26b80 1 -- 172.21.15.5:0/2293449884 shutdown_connections 2024-05-09T00:52:32.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.070+0000 7f8154d26b80 1 -- 172.21.15.5:0/2293449884 wait complete. 2024-05-09T00:52:32.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.075+0000 7f8154d26b80 20 client.16125 trim_cache size 0 max 16384 2024-05-09T00:52:32.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.125+0000 7f8154d26b80 1 Processor -- start 2024-05-09T00:52:32.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.126+0000 7f8154d26b80 1 -- start start 2024-05-09T00:52:32.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.126+0000 7f8154d26b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c406a74a0 0x555c40842480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:32.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.126+0000 7f8154d26b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555c40721d00 0x555c408371a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:32.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.127+0000 7f8154d26b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c4082cc10 0x555c40818ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:32.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.127+0000 7f813ffff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c406a74a0 0x555c40842480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:32.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.127+0000 7f813ffff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c406a74a0 0x555c40842480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:46676/0 (socket says 172.21.15.5:46676) 2024-05-09T00:52:32.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.127+0000 7f813ffff700 1 -- 172.21.15.5:0/2107755957 learned_addr learned my addr 172.21.15.5:0/2107755957 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:52:32.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.127+0000 7f8137fff700 1 --2- 172.21.15.5:0/2107755957 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555c40721d00 0x555c408371a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:32.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.128+0000 7f81448ad700 1 --2- 172.21.15.5:0/2107755957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c4082cc10 0x555c40818ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:32.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.128+0000 7f8154d26b80 1 -- 172.21.15.5:0/2107755957 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f81401acaf0 con 0x555c4082cc10 2024-05-09T00:52:32.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.128+0000 7f8154d26b80 1 -- 172.21.15.5:0/2107755957 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f81401ae630 con 0x555c40721d00 2024-05-09T00:52:32.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.128+0000 7f8154d26b80 1 -- 172.21.15.5:0/2107755957 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f81401ad1c0 con 0x555c406a74a0 2024-05-09T00:52:32.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.129+0000 7f8137fff700 1 -- 172.21.15.5:0/2107755957 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c406a74a0 msgr2=0x555c40842480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:52:32.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.129+0000 7f8137fff700 1 --2- 172.21.15.5:0/2107755957 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c406a74a0 0x555c40842480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:32.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.129+0000 7f8137fff700 1 -- 172.21.15.5:0/2107755957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c4082cc10 msgr2=0x555c40818ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:52:32.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.129+0000 7f8137fff700 1 --2- 172.21.15.5:0/2107755957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c4082cc10 0x555c40818ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:32.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.129+0000 7f8137fff700 1 -- 172.21.15.5:0/2107755957 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c40820ee0 con 0x555c40721d00 2024-05-09T00:52:32.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.130+0000 7f8137fff700 1 --2- 172.21.15.5:0/2107755957 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555c40721d00 0x555c408371a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x555c407e9df0 tx=0x7f812800beb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78df4f7fb63d3f3b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:52:32.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.132+0000 7f8135ffb700 1 -- 172.21.15.5:0/2107755957 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8128012c90 con 0x555c40721d00 2024-05-09T00:52:32.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.132+0000 7f8135ffb700 1 -- 172.21.15.5:0/2107755957 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8128012e30 con 0x555c40721d00 2024-05-09T00:52:32.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.132+0000 7f8135ffb700 1 -- 172.21.15.5:0/2107755957 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8128005280 con 0x555c40721d00 2024-05-09T00:52:32.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.135+0000 7f8154d26b80 1 -- 172.21.15.5:0/2107755957 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555c40721d00 msgr2=0x555c408371a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:52:32.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.135+0000 7f8154d26b80 1 --2- 172.21.15.5:0/2107755957 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555c40721d00 0x555c408371a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x555c407e9df0 tx=0x7f812800beb0 comp rx=0 tx=0).stop 2024-05-09T00:52:32.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.137+0000 7f8154d26b80 1 -- 172.21.15.5:0/2107755957 shutdown_connections 2024-05-09T00:52:32.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.138+0000 7f8154d26b80 1 --2- 172.21.15.5:0/2107755957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c4082cc10 0x555c40818ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:32.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.138+0000 7f8154d26b80 1 --2- 172.21.15.5:0/2107755957 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c406a74a0 0x555c40842480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:32.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.138+0000 7f8154d26b80 1 --2- 172.21.15.5:0/2107755957 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555c40721d00 0x555c408371a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:32.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.138+0000 7f8154d26b80 1 -- 172.21.15.5:0/2107755957 >> 172.21.15.5:0/2107755957 conn(0x555c406aaf70 msgr2=0x555c408351d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:52:32.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.141+0000 7f8154d26b80 1 -- 172.21.15.5:0/2107755957 shutdown_connections 2024-05-09T00:52:32.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.141+0000 7f8154d26b80 1 -- 172.21.15.5:0/2107755957 wait complete. 2024-05-09T00:52:32.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.142+0000 7f8154d26b80 1 -- start start 2024-05-09T00:52:32.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.143+0000 7f8154d26b80 1 Processor -- start 2024-05-09T00:52:32.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.144+0000 7f8154d26b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c406a74a0 0x555c4084e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:32.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.144+0000 7f8154d26b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c40721d00 0x555c4084e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:32.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.145+0000 7f813ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c406a74a0 0x555c4084e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:32.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.145+0000 7f813ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c406a74a0 0x555c4084e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41222/0 (socket says 172.21.15.5:41222) 2024-05-09T00:52:32.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.145+0000 7f813ffff700 1 -- 172.21.15.5:0/3198662577 learned_addr learned my addr 172.21.15.5:0/3198662577 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:52:32.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.145+0000 7f8154d26b80 1 --2- 172.21.15.5:0/3198662577 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555c4082cc10 0x555c4084ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:32.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.145+0000 7f8137fff700 1 --2- 172.21.15.5:0/3198662577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c40721d00 0x555c4084e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:32.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.145+0000 7f8154d26b80 1 -- 172.21.15.5:0/3198662577 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f81401adf60 con 0x555c406a74a0 2024-05-09T00:52:32.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.145+0000 7f8154d26b80 1 -- 172.21.15.5:0/3198662577 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f81401ad890 con 0x555c4082cc10 2024-05-09T00:52:32.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.145+0000 7f8154d26b80 1 -- 172.21.15.5:0/3198662577 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f814010e080 con 0x555c40721d00 2024-05-09T00:52:32.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.146+0000 7f81448ad700 1 --2- 172.21.15.5:0/3198662577 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555c4082cc10 0x555c4084ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:32.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.146+0000 7f8154d26b80 10 client.0 _ll_register_callbacks cb 0x555c405b86c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:52:32.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.147+0000 7f8137fff700 1 -- 172.21.15.5:0/3198662577 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555c4082cc10 msgr2=0x555c4084ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:52:32.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.147+0000 7f8137fff700 1 --2- 172.21.15.5:0/3198662577 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555c4082cc10 0x555c4084ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:32.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.147+0000 7f8137fff700 1 -- 172.21.15.5:0/3198662577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c406a74a0 msgr2=0x555c4084e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:52:32.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.147+0000 7f8137fff700 1 --2- 172.21.15.5:0/3198662577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c406a74a0 0x555c4084e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:32.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.147+0000 7f8137fff700 1 -- 172.21.15.5:0/3198662577 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c4081b9d0 con 0x555c40721d00 2024-05-09T00:52:32.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.147+0000 7f8137fff700 1 --2- 172.21.15.5:0/3198662577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c40721d00 0x555c4084e680 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x555c408326e0 tx=0x7f8128012cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95b1916522bd1b49 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:52:32.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.147+0000 7f811f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-09T00:52:32.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.148+0000 7f811f7fe700 1 -- 172.21.15.5:0/3198662577 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8128029060 con 0x555c40721d00 2024-05-09T00:52:32.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.148+0000 7f811f7fe700 1 -- 172.21.15.5:0/3198662577 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f8128004050 con 0x555c40721d00 2024-05-09T00:52:32.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.148+0000 7f8154d26b80 1 -- 172.21.15.5:0/3198662577 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8128012e30 con 0x555c40721d00 2024-05-09T00:52:32.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.148+0000 7f811f7fe700 1 -- 172.21.15.5:0/3198662577 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8128004250 con 0x555c40721d00 2024-05-09T00:52:32.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.149+0000 7f8154d26b80 10 client.25684 fetch_fsmap learned FSMap version 31 2024-05-09T00:52:32.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.149+0000 7f8154d26b80 1 -- 172.21.15.5:0/3198662577 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x555c408b13e0 con 0x555c40721d00 2024-05-09T00:52:32.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.153+0000 7f811f7fe700 1 -- 172.21.15.5:0/3198662577 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81280043f0 con 0x555c40721d00 2024-05-09T00:52:32.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.153+0000 7f811f7fe700 1 -- 172.21.15.5:0/3198662577 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f8128019030 con 0x555c40721d00 2024-05-09T00:52:32.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.156+0000 7f8154d26b80 10 client.25684 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:52:32.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.156+0000 7f8154d26b80 10 client.25684 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:52:32.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.156+0000 7f8154d26b80 20 client.25684 populate_metadata read hostname 'smithi005' 2024-05-09T00:52:32.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.156+0000 7f8154d26b80 1 --2- 172.21.15.5:0/3198662577 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x555c408b18d0 0x555c408b3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:32.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.156+0000 7f8154d26b80 4 client.25684 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:52:32.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.156+0000 7f8154d26b80 1 -- 172.21.15.5:0/3198662577 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f8128012e30 con 0x555c408b18d0 2024-05-09T00:52:32.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.160+0000 7f813ffff700 1 --2- 172.21.15.5:0/3198662577 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x555c408b18d0 0x555c408b3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:32.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.167+0000 7f813ffff700 1 --2- 172.21.15.5:0/3198662577 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x555c408b18d0 0x555c408b3cf0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72fc15a73bf4b71c server_cookie=4381f8a505cf9a8a in_seq=0 out_seq=0 2024-05-09T00:52:32.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.167+0000 7f811f7fe700 10 client.25684 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:52:32.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.171+0000 7f811f7fe700 1 -- 172.21.15.5:0/3198662577 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f8128012e30 con 0x555c408b18d0 2024-05-09T00:52:32.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.172+0000 7f811f7fe700 10 client.25684 handle_command_reply: tid=0 2024-05-09T00:52:32.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.174+0000 7f8154d26b80 1 client.25684 shutdown 2024-05-09T00:52:32.188 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-09T00:52:32.188 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-05-09T00:52:32.188 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-05-09T00:52:32.188 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-09T00:52:32.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.178+0000 7f8154d26b80 1 -- 172.21.15.5:0/3198662577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c40721d00 msgr2=0x555c4084e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:52:32.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.178+0000 7f8154d26b80 1 --2- 172.21.15.5:0/3198662577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c40721d00 0x555c4084e680 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x555c408326e0 tx=0x7f8128012cc0 comp rx=0 tx=0).stop 2024-05-09T00:52:32.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.178+0000 7f8154d26b80 1 -- 172.21.15.5:0/3198662577 shutdown_connections 2024-05-09T00:52:32.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.178+0000 7f8154d26b80 1 --2- 172.21.15.5:0/3198662577 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x555c408b18d0 0x555c408b3cf0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:32.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.178+0000 7f8154d26b80 1 --2- 172.21.15.5:0/3198662577 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555c4082cc10 0x555c4084ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:32.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.178+0000 7f811f7fe700 0 client.25684 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:52:32.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.178+0000 7f8154d26b80 1 --2- 172.21.15.5:0/3198662577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c406a74a0 0x555c4084e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:32.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.178+0000 7f8154d26b80 1 --2- 172.21.15.5:0/3198662577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c40721d00 0x555c4084e680 secure :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0x555c408326e0 tx=0x7f8128012cc0 comp rx=0 tx=0).stop 2024-05-09T00:52:32.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.178+0000 7f8154d26b80 1 -- 172.21.15.5:0/3198662577 >> 172.21.15.5:0/3198662577 conn(0x555c406aaf70 msgr2=0x555c40835190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:52:32.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.179+0000 7f8154d26b80 1 -- 172.21.15.5:0/3198662577 shutdown_connections 2024-05-09T00:52:32.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.179+0000 7f8154d26b80 1 -- 172.21.15.5:0/3198662577 wait complete. 2024-05-09T00:52:32.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.179+0000 7f8154d26b80 20 client.25684 trim_cache size 0 max 16384 2024-05-09T00:52:32.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.182+0000 7f8146310700 1 -- 172.21.15.5:0/2764818895 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f812c0b2900 msgr2=0x7f812c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:52:32.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.182+0000 7f8146310700 1 --2- 172.21.15.5:0/2764818895 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f812c0b2900 0x7f812c0b4e10 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f813800d010 tx=0x7f8138005b60 comp rx=0 tx=0).stop 2024-05-09T00:52:32.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.182+0000 7f8146310700 1 -- 172.21.15.5:0/2764818895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81401034c0 msgr2=0x7f814019d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:52:32.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.182+0000 7f8146310700 1 --2- 172.21.15.5:0/2764818895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81401034c0 0x7f814019d060 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f813000b530 tx=0x7f813000b8f0 comp rx=0 tx=0).stop 2024-05-09T00:52:32.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.182+0000 7f8146310700 1 -- 172.21.15.5:0/2764818895 shutdown_connections 2024-05-09T00:52:32.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.182+0000 7f8146310700 1 --2- 172.21.15.5:0/2764818895 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f812c0b2900 0x7f812c0b4e10 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:32.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.182+0000 7f8146310700 1 --2- 172.21.15.5:0/2764818895 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8140108740 0x7f81401a1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:32.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.182+0000 7f8146310700 1 --2- 172.21.15.5:0/2764818895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81401034c0 0x7f814019d060 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:32.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.182+0000 7f8146310700 1 --2- 172.21.15.5:0/2764818895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8140107df0 0x7f814019d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:32.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.182+0000 7f8146310700 1 -- 172.21.15.5:0/2764818895 >> 172.21.15.5:0/2764818895 conn(0x7f81400fedf0 msgr2=0x7f814010ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:52:32.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.183+0000 7f8146310700 1 -- 172.21.15.5:0/2764818895 shutdown_connections 2024-05-09T00:52:32.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:32.183+0000 7f8146310700 1 -- 172.21.15.5:0/2764818895 wait complete. 2024-05-09T00:52:33.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:52:33 smithi005 ceph-mon[30365]: pgmap v557: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 818 MiB/s wr, 1.26k op/s 2024-05-09T00:52:33.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:52:33 smithi131 ceph-mon[34547]: pgmap v557: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 818 MiB/s wr, 1.26k op/s 2024-05-09T00:52:33.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:52:33 smithi157 ceph-mon[34294]: pgmap v557: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 818 MiB/s wr, 1.26k op/s 2024-05-09T00:52:34.231 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T00:52:34.231 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T00:52:34.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.838+0000 7f655c5a8640 1 -- 172.21.15.5:0/433501449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6554110020 msgr2=0x7f6554110430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:52:34.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.838+0000 7f655c5a8640 1 --2- 172.21.15.5:0/433501449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6554110020 0x7f6554110430 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f6544009900 tx=0x7f654402a410 comp rx=0 tx=0).stop 2024-05-09T00:52:34.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.842+0000 7f655c5a8640 1 -- 172.21.15.5:0/433501449 shutdown_connections 2024-05-09T00:52:34.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.842+0000 7f655c5a8640 1 --2- 172.21.15.5:0/433501449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6554110020 0x7f6554110430 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:34.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.842+0000 7f655c5a8640 1 --2- 172.21.15.5:0/433501449 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f655410b300 0x7f655410b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:34.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.842+0000 7f655c5a8640 1 --2- 172.21.15.5:0/433501449 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f655410a480 0x7f655410adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:34.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.842+0000 7f655c5a8640 1 -- 172.21.15.5:0/433501449 >> 172.21.15.5:0/433501449 conn(0x7f65540704c0 msgr2=0x7f6554072950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:52:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.848+0000 7f655c5a8640 1 -- 172.21.15.5:0/433501449 shutdown_connections 2024-05-09T00:52:34.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.851+0000 7f655c5a8640 1 -- 172.21.15.5:0/433501449 wait complete. 2024-05-09T00:52:34.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.852+0000 7f655c5a8640 1 Processor -- start 2024-05-09T00:52:34.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.853+0000 7f655c5a8640 1 -- start start 2024-05-09T00:52:34.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.853+0000 7f655c5a8640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f655410a480 0x7f65541a8e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:34.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.853+0000 7f655c5a8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f655410b300 0x7f65541a9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:34.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.853+0000 7f655c5a8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6554110020 0x7f655407af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.853+0000 7f655c5a8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f655411e3a0 con 0x7f655410b300 2024-05-09T00:52:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.853+0000 7f655c5a8640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f655411e220 con 0x7f655410a480 2024-05-09T00:52:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.853+0000 7f655c5a8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f655411e520 con 0x7f6554110020 2024-05-09T00:52:34.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.858+0000 7f655ab45640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6554110020 0x7f655407af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:34.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.859+0000 7f655ab45640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6554110020 0x7f655407af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:54388/0 (socket says 172.21.15.5:54388) 2024-05-09T00:52:34.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.859+0000 7f655ab45640 1 -- 172.21.15.5:0/3146459229 learned_addr learned my addr 172.21.15.5:0/3146459229 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:52:34.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.864+0000 7f655a344640 1 --2- 172.21.15.5:0/3146459229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f655410a480 0x7f65541a8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:34.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.864+0000 7f655ab45640 1 -- 172.21.15.5:0/3146459229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f655410a480 msgr2=0x7f65541a8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:52:34.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.864+0000 7f655ab45640 1 --2- 172.21.15.5:0/3146459229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f655410a480 0x7f65541a8e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:34.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.864+0000 7f655ab45640 1 -- 172.21.15.5:0/3146459229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f655410b300 msgr2=0x7f65541a9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:52:34.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.864+0000 7f655ab45640 1 --2- 172.21.15.5:0/3146459229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f655410b300 0x7f65541a9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:34.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.864+0000 7f655ab45640 1 -- 172.21.15.5:0/3146459229 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f655407b5a0 con 0x7f6554110020 2024-05-09T00:52:34.868 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.867+0000 7f655ab45640 1 --2- 172.21.15.5:0/3146459229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6554110020 0x7f655407af30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f654402aa00 tx=0x7f6544003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:52:34.877 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.876+0000 7f654b7fe640 1 -- 172.21.15.5:0/3146459229 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f654403f070 con 0x7f6554110020 2024-05-09T00:52:34.877 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.876+0000 7f654b7fe640 1 -- 172.21.15.5:0/3146459229 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6544031c50 con 0x7f6554110020 2024-05-09T00:52:34.877 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.876+0000 7f654b7fe640 1 -- 172.21.15.5:0/3146459229 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6544035030 con 0x7f6554110020 2024-05-09T00:52:34.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.876+0000 7f655c5a8640 1 -- 172.21.15.5:0/3146459229 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f655407b830 con 0x7f6554110020 2024-05-09T00:52:34.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.877+0000 7f655c5a8640 1 -- 172.21.15.5:0/3146459229 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f655407bcd0 con 0x7f6554110020 2024-05-09T00:52:34.890 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.889+0000 7f656345c740 1 Processor -- start 2024-05-09T00:52:34.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.894+0000 7f656345c740 1 -- start start 2024-05-09T00:52:34.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.895+0000 7f656345c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf9d8b71b0 0x55cf9dc5b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.895+0000 7f656345c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf9d88e2f0 0x55cf9dc5bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.895+0000 7f656345c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cf9d8964d0 0x55cf9dc62b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.895+0000 7f656345c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55cf9dbbf7c0 con 0x55cf9d88e2f0 2024-05-09T00:52:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.895+0000 7f656345c740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55cf9db5b0e0 con 0x55cf9d8964d0 2024-05-09T00:52:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.895+0000 7f656345c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55cf9dbbb780 con 0x55cf9d8b71b0 2024-05-09T00:52:34.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.902+0000 7f6559b43640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf9d8b71b0 0x55cf9dc5b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:34.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.902+0000 7f6559b43640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf9d8b71b0 0x55cf9dc5b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:54402/0 (socket says 172.21.15.5:54402) 2024-05-09T00:52:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.902+0000 7f6559b43640 1 -- 172.21.15.5:0/2725719238 learned_addr learned my addr 172.21.15.5:0/2725719238 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:52:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.902+0000 7f6559b43640 1 --2- 172.21.15.5:0/2725719238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cf9d8964d0 0x55cf9dc62b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.903+0000 7f6559b43640 1 -- 172.21.15.5:0/2725719238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cf9d8964d0 msgr2=0x55cf9dc62b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:52:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.903+0000 7f6559b43640 1 --2- 172.21.15.5:0/2725719238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cf9d8964d0 0x55cf9dc62b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:34.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.903+0000 7f6559b43640 1 -- 172.21.15.5:0/2725719238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf9d88e2f0 msgr2=0x55cf9dc5bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:52:34.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.903+0000 7f6559b43640 1 --2- 172.21.15.5:0/2725719238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf9d88e2f0 0x55cf9dc5bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:34.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.903+0000 7f6559b43640 1 -- 172.21.15.5:0/2725719238 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf9db847c0 con 0x55cf9d8b71b0 2024-05-09T00:52:34.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.905+0000 7f6559b43640 1 --2- 172.21.15.5:0/2725719238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf9d8b71b0 0x55cf9dc5b6c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f654c013bb0 tx=0x7f654c009a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fa750f7a4cd6658 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:52:34.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.905+0000 7f6548bfe640 1 -- 172.21.15.5:0/2725719238 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f654c013d60 con 0x55cf9d8b71b0 2024-05-09T00:52:34.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.905+0000 7f6548bfe640 1 -- 172.21.15.5:0/2725719238 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f654c003970 con 0x55cf9d8b71b0 2024-05-09T00:52:34.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.909+0000 7f656345c740 1 -- 172.21.15.5:0/2725719238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf9d8b71b0 msgr2=0x55cf9dc5b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:52:34.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.909+0000 7f656345c740 1 --2- 172.21.15.5:0/2725719238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf9d8b71b0 0x55cf9dc5b6c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f654c013bb0 tx=0x7f654c009a40 comp rx=0 tx=0).stop 2024-05-09T00:52:34.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.910+0000 7f654b7fe640 1 -- 172.21.15.5:0/3146459229 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f6544042070 con 0x7f6554110020 2024-05-09T00:52:34.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.913+0000 7f654b7fe640 1 --2- 172.21.15.5:0/3146459229 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f65400b29a0 0x7f65400b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:34.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.915+0000 7f654b7fe640 1 -- 172.21.15.5:0/3146459229 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f65440c6190 con 0x7f6554110020 2024-05-09T00:52:34.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.917+0000 7f656345c740 1 -- 172.21.15.5:0/2725719238 shutdown_connections 2024-05-09T00:52:34.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.917+0000 7f656345c740 1 --2- 172.21.15.5:0/2725719238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cf9d8964d0 0x55cf9dc62b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:34.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.917+0000 7f656345c740 1 --2- 172.21.15.5:0/2725719238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf9d88e2f0 0x55cf9dc5bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:34.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.917+0000 7f656345c740 1 --2- 172.21.15.5:0/2725719238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf9d8b71b0 0x55cf9dc5b6c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:34.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.917+0000 7f656345c740 1 -- 172.21.15.5:0/2725719238 >> 172.21.15.5:0/2725719238 conn(0x55cf9d8ab300 msgr2=0x55cf9dc57050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:52:34.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.919+0000 7f6559b43640 1 --2- 172.21.15.5:0/3146459229 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f65400b29a0 0x7f65400b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:34.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.919+0000 7f656345c740 1 -- 172.21.15.5:0/2725719238 shutdown_connections 2024-05-09T00:52:34.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.919+0000 7f656345c740 1 -- 172.21.15.5:0/2725719238 wait complete. 2024-05-09T00:52:34.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.921+0000 7f656345c740 1 -- start start 2024-05-09T00:52:34.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.921+0000 7f656345c740 1 Processor -- start 2024-05-09T00:52:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.925+0000 7f656345c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf9d88e2f0 0x55cf9dc65370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.925+0000 7f656345c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf9d8964d0 0x55cf9dc658b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.926+0000 7f656345c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cf9d8b71b0 0x55cf9dc6c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.926+0000 7f656345c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55cf9dc63580 con 0x55cf9d8964d0 2024-05-09T00:52:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.926+0000 7f656345c740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55cf9db469c0 con 0x55cf9d8b71b0 2024-05-09T00:52:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.926+0000 7f656345c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55cf9dc63700 con 0x55cf9d88e2f0 2024-05-09T00:52:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.926+0000 7f656345c740 10 client.0 _ll_register_callbacks cb 0x55cf9db73020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:52:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.928+0000 7f655a344640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf9d88e2f0 0x55cf9dc65370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.928+0000 7f655a344640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf9d88e2f0 0x55cf9dc65370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:54404/0 (socket says 172.21.15.5:54404) 2024-05-09T00:52:34.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.928+0000 7f655a344640 1 -- 172.21.15.5:0/2141788269 learned_addr learned my addr 172.21.15.5:0/2141788269 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:52:34.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.928+0000 7f655ab45640 1 --2- 172.21.15.5:0/2141788269 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cf9d8b71b0 0x55cf9dc6c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:34.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.930+0000 7f655a344640 1 -- 172.21.15.5:0/2141788269 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cf9d8b71b0 msgr2=0x55cf9dc6c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:52:34.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.930+0000 7f655a344640 1 --2- 172.21.15.5:0/2141788269 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cf9d8b71b0 0x55cf9dc6c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:34.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.930+0000 7f655a344640 1 -- 172.21.15.5:0/2141788269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf9d8964d0 msgr2=0x55cf9dc658b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:52:34.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.930+0000 7f655a344640 1 --2- 172.21.15.5:0/2141788269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf9d8964d0 0x55cf9dc658b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:34.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.930+0000 7f655a344640 1 -- 172.21.15.5:0/2141788269 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf9db6d1f0 con 0x55cf9d88e2f0 2024-05-09T00:52:34.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.931+0000 7f655a344640 1 --2- 172.21.15.5:0/2141788269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf9d88e2f0 0x55cf9dc65370 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6550009fa0 tx=0x7f6550010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7fa88758242f015 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:52:34.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.931+0000 7f653dffb640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-09T00:52:34.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.933+0000 7f653dffb640 1 -- 172.21.15.5:0/2141788269 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6550005220 con 0x55cf9d88e2f0 2024-05-09T00:52:34.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.933+0000 7f656345c740 1 -- 172.21.15.5:0/2141788269 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f654c003970 con 0x55cf9d88e2f0 2024-05-09T00:52:34.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.933+0000 7f653dffb640 1 -- 172.21.15.5:0/2141788269 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f65500053c0 con 0x55cf9d88e2f0 2024-05-09T00:52:34.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.935+0000 7f653dffb640 1 -- 172.21.15.5:0/2141788269 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6550020030 con 0x55cf9d88e2f0 2024-05-09T00:52:34.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.936+0000 7f6559b43640 1 --2- 172.21.15.5:0/3146459229 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f65400b29a0 0x7f65400b4eb0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f654c0139c0 tx=0x7f654c004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:52:34.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.937+0000 7f656345c740 10 client.25696 fetch_fsmap learned FSMap version 31 2024-05-09T00:52:34.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.937+0000 7f656345c740 1 -- 172.21.15.5:0/2141788269 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55cf9d922790 con 0x55cf9d88e2f0 2024-05-09T00:52:34.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.939+0000 7f653dffb640 1 -- 172.21.15.5:0/2141788269 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f654c003970 con 0x55cf9d88e2f0 2024-05-09T00:52:34.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.939+0000 7f653dffb640 1 -- 172.21.15.5:0/2141788269 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f655000f040 con 0x55cf9d88e2f0 2024-05-09T00:52:34.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.939+0000 7f656345c740 10 client.25696 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:52:34.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.939+0000 7f656345c740 10 client.25696 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:52:34.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.939+0000 7f656345c740 20 client.25696 populate_metadata read hostname 'smithi005' 2024-05-09T00:52:34.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.939+0000 7f656345c740 1 --2- 172.21.15.5:0/2141788269 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55cf9d875d80 0x55cf9dc73040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:34.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.939+0000 7f656345c740 4 client.25696 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:52:34.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.939+0000 7f656345c740 1 -- 172.21.15.5:0/2141788269 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf9dbcd3d0 con 0x55cf9d875d80 2024-05-09T00:52:34.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.940+0000 7f6559b43640 1 --2- 172.21.15.5:0/2141788269 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55cf9d875d80 0x55cf9dc73040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:34.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.942+0000 7f6559b43640 1 --2- 172.21.15.5:0/2141788269 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55cf9d875d80 0x55cf9dc73040 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e0acc7375bffd6b server_cookie=5e9f5304618ab475 in_seq=0 out_seq=0 2024-05-09T00:52:34.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.942+0000 7f653dffb640 10 client.25696 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:52:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.946+0000 7f653dffb640 1 -- 172.21.15.5:0/2141788269 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55cf9dbcd3d0 con 0x55cf9d875d80 2024-05-09T00:52:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.946+0000 7f653dffb640 10 client.25696 handle_command_reply: tid=0 2024-05-09T00:52:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.946+0000 7f656345c740 1 client.25696 shutdown 2024-05-09T00:52:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.946+0000 7f656345c740 1 -- 172.21.15.5:0/2141788269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf9d88e2f0 msgr2=0x55cf9dc65370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:52:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.946+0000 7f656345c740 1 --2- 172.21.15.5:0/2141788269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf9d88e2f0 0x55cf9dc65370 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6550009fa0 tx=0x7f6550010800 comp rx=0 tx=0).stop 2024-05-09T00:52:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.946+0000 7f656345c740 1 -- 172.21.15.5:0/2141788269 shutdown_connections 2024-05-09T00:52:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.946+0000 7f656345c740 1 --2- 172.21.15.5:0/2141788269 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55cf9d875d80 0x55cf9dc73040 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.947+0000 7f656345c740 1 --2- 172.21.15.5:0/2141788269 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cf9d8b71b0 0x55cf9dc6c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.947+0000 7f656345c740 1 --2- 172.21.15.5:0/2141788269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf9d8964d0 0x55cf9dc658b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.947+0000 7f656345c740 1 --2- 172.21.15.5:0/2141788269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf9d88e2f0 0x55cf9dc65370 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:34.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.947+0000 7f656345c740 1 -- 172.21.15.5:0/2141788269 >> 172.21.15.5:0/2141788269 conn(0x55cf9d8ab300 msgr2=0x55cf9dc55860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:52:34.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.947+0000 7f653dffb640 0 client.25696 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:52:34.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.947+0000 7f656345c740 1 -- 172.21.15.5:0/2141788269 shutdown_connections 2024-05-09T00:52:34.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.947+0000 7f656345c740 1 -- 172.21.15.5:0/2141788269 wait complete. 2024-05-09T00:52:34.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:34.947+0000 7f656345c740 20 client.25696 trim_cache size 0 max 16384 2024-05-09T00:52:35.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:35.029+0000 7f656345c740 1 Processor -- start 2024-05-09T00:52:35.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:35.029+0000 7f656345c740 1 -- start start 2024-05-09T00:52:35.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:35.029+0000 7f656345c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf9d8964d0 0x55cf9dc63470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:35.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:35.029+0000 7f656345c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cf9d8b71b0 0x55cf9dc63b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:35.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:35.029+0000 7f656345c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf9d84b1c0 0x55cf9dc640c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:35.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:35.029+0000 7f656345c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f653800e560 con 0x55cf9d8964d0 2024-05-09T00:52:35.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:35.030+0000 7f656345c740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f65380100a0 con 0x55cf9d8b71b0 2024-05-09T00:52:35.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:35.030+0000 7f656345c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f653800ec30 con 0x55cf9d84b1c0 2024-05-09T00:52:35.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:35.030+0000 7f655ab45640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf9d84b1c0 0x55cf9dc640c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:35.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:35.030+0000 7f655ab45640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf9d84b1c0 0x55cf9dc640c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:54420/0 (socket says 172.21.15.5:54420) 2024-05-09T00:52:35.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:35.030+0000 7f655ab45640 1 -- 172.21.15.5:0/3778774414 learned_addr learned my addr 172.21.15.5:0/3778774414 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:52:35.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:35.030+0000 7f6559b43640 1 --2- 172.21.15.5:0/3778774414 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cf9d8b71b0 0x55cf9dc63b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:35.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:35.031+0000 7f655ab45640 1 -- 172.21.15.5:0/3778774414 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cf9d8b71b0 msgr2=0x55cf9dc63b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:52:35.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:35.031+0000 7f655ab45640 1 --2- 172.21.15.5:0/3778774414 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cf9d8b71b0 0x55cf9dc63b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:35.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:35.031+0000 7f655ab45640 1 -- 172.21.15.5:0/3778774414 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf9d8964d0 msgr2=0x55cf9dc63470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:52:35.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:35.031+0000 7f655ab45640 1 --2- 172.21.15.5:0/3778774414 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf9d8964d0 0x55cf9dc63470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:35.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:35.031+0000 7f655ab45640 1 -- 172.21.15.5:0/3778774414 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf9d9995f0 con 0x55cf9d84b1c0 2024-05-09T00:52:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:35.036+0000 7f655ab45640 1 --2- 172.21.15.5:0/3778774414 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf9d84b1c0 0x55cf9dc640c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6544031440 tx=0x7f65440cf070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4925a3d367b7e744 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:52:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:35.037+0000 7f653ffff640 1 -- 172.21.15.5:0/3778774414 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65440c6390 con 0x55cf9d84b1c0 2024-05-09T00:52:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:35.037+0000 7f653ffff640 1 -- 172.21.15.5:0/3778774414 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65440eae50 con 0x55cf9d84b1c0 2024-05-09T00:52:35.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:35.039+0000 7f656345c740 1 -- 172.21.15.5:0/3778774414 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf9d84b1c0 msgr2=0x55cf9dc640c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:52:35.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:35.039+0000 7f656345c740 1 --2- 172.21.15.5:0/3778774414 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf9d84b1c0 0x55cf9dc640c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6544031440 tx=0x7f65440cf070 comp rx=0 tx=0).stop 2024-05-09T00:52:35.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:35.039+0000 7f656345c740 1 -- 172.21.15.5:0/3778774414 shutdown_connections 2024-05-09T00:52:35.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:35.039+0000 7f656345c740 1 --2- 172.21.15.5:0/3778774414 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf9d84b1c0 0x55cf9dc640c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:35.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:35.039+0000 7f656345c740 1 --2- 172.21.15.5:0/3778774414 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cf9d8b71b0 0x55cf9dc63b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:35.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:35.039+0000 7f656345c740 1 --2- 172.21.15.5:0/3778774414 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf9d8964d0 0x55cf9dc63470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:35.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:35.039+0000 7f656345c740 1 -- 172.21.15.5:0/3778774414 >> 172.21.15.5:0/3778774414 conn(0x55cf9d8ab300 msgr2=0x55cf9dc5f720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:52:35.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:35.039+0000 7f656345c740 1 -- 172.21.15.5:0/3778774414 shutdown_connections 2024-05-09T00:52:35.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:35.039+0000 7f656345c740 1 -- 172.21.15.5:0/3778774414 wait complete. 2024-05-09T00:52:35.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:35.040+0000 7f656345c740 1 -- start start 2024-05-09T00:52:35.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:35.040+0000 7f656345c740 1 Processor -- start 2024-05-09T00:52:35.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:35.040+0000 7f656345c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf9d8964d0 0x55cf9dc817f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:35.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:35.040+0000 7f656345c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf9d8b71b0 0x55cf9dc81d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:35.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:35.040+0000 7f656345c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cf9dcd9050 0x55cf9dcdb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:35.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:35.040+0000 7f656345c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f653800f9d0 con 0x55cf9d8b71b0 2024-05-09T00:52:35.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:35.040+0000 7f656345c740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f653800f300 con 0x55cf9dcd9050 2024-05-09T00:52:35.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:35.040+0000 7f656345c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f653800c540 con 0x55cf9d8964d0 2024-05-09T00:52:35.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:35.040+0000 7f656345c740 10 client.0 _ll_register_callbacks cb 0x7f65500093b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:52:35.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:35.042+0000 7f655ab45640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cf9dcd9050 0x55cf9dcdb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:35.042+0000 7f655a344640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf9d8964d0 0x55cf9dc817f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:35.042+0000 7f6559b43640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf9d8b71b0 0x55cf9dc81d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:35.042+0000 7f6559b43640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf9d8b71b0 0x55cf9dc81d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50744/0 (socket says 172.21.15.5:50744) 2024-05-09T00:52:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:35.042+0000 7f6559b43640 1 -- 172.21.15.5:0/2703119375 learned_addr learned my addr 172.21.15.5:0/2703119375 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:52:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:35.043+0000 7f6559b43640 1 -- 172.21.15.5:0/2703119375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf9d8964d0 msgr2=0x55cf9dc817f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:52:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:35.043+0000 7f6559b43640 1 --2- 172.21.15.5:0/2703119375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf9d8964d0 0x55cf9dc817f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:35.043+0000 7f6559b43640 1 -- 172.21.15.5:0/2703119375 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cf9dcd9050 msgr2=0x55cf9dcdb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:52:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:35.043+0000 7f6559b43640 1 --2- 172.21.15.5:0/2703119375 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cf9dcd9050 0x55cf9dcdb470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:35.043+0000 7f6559b43640 1 -- 172.21.15.5:0/2703119375 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf9db83ed0 con 0x55cf9d8b71b0 2024-05-09T00:52:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:35.044+0000 7f6559b43640 1 --2- 172.21.15.5:0/2703119375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf9d8b71b0 0x55cf9dc81d30 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f654c013b50 tx=0x7f654c00b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58217277973f9e0a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:52:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:35.044+0000 7f653dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:52:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:35.045+0000 7f653dffb640 1 -- 172.21.15.5:0/2703119375 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f654c037230 con 0x55cf9d8b71b0 2024-05-09T00:52:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:35.045+0000 7f656345c740 1 -- 172.21.15.5:0/2703119375 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55cf9dbc7240 con 0x55cf9d8b71b0 2024-05-09T00:52:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:35.045+0000 7f653dffb640 1 -- 172.21.15.5:0/2703119375 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f654c0373d0 con 0x55cf9d8b71b0 2024-05-09T00:52:35.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:35.046+0000 7f653dffb640 1 -- 172.21.15.5:0/2703119375 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f654c0375d0 con 0x55cf9d8b71b0 2024-05-09T00:52:35.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:35.047+0000 7f656345c740 10 client.16137 fetch_fsmap learned FSMap version 31 2024-05-09T00:52:35.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:35.047+0000 7f656345c740 1 -- 172.21.15.5:0/2703119375 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55cf9dcdca00 con 0x55cf9d8b71b0 2024-05-09T00:52:35.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:35.048+0000 7f653dffb640 1 -- 172.21.15.5:0/2703119375 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55cf9dbc7240 con 0x55cf9d8b71b0 2024-05-09T00:52:35.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:35.048+0000 7f653dffb640 1 -- 172.21.15.5:0/2703119375 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f654c048070 con 0x55cf9d8b71b0 2024-05-09T00:52:35.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:35.048+0000 7f656345c740 10 client.16137 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:52:35.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:35.048+0000 7f656345c740 10 client.16137 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:52:35.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:35.048+0000 7f656345c740 20 client.16137 populate_metadata read hostname 'smithi005' 2024-05-09T00:52:35.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:35.048+0000 7f656345c740 1 --2- 172.21.15.5:0/2703119375 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55cf9dcdd320 0x55cf9dcdf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:35.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:35.048+0000 7f656345c740 4 client.16137 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:52:35.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:35.048+0000 7f656345c740 1 -- 172.21.15.5:0/2703119375 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cf9db42350 con 0x55cf9dcdd320 2024-05-09T00:52:35.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:35.053+0000 7f655a344640 1 --2- 172.21.15.5:0/2703119375 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55cf9dcdd320 0x55cf9dcdf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:35.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:35.063+0000 7f655a344640 1 --2- 172.21.15.5:0/2703119375 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55cf9dcdd320 0x55cf9dcdf740 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adbc5a691ab89cc0 server_cookie=11fa84566422d899 in_seq=0 out_seq=0 2024-05-09T00:52:35.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:35.063+0000 7f653dffb640 10 client.16137 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:52:35.158 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:52:35.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:52:35 smithi005 ceph-mon[30365]: pgmap v558: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 929 MiB/s wr, 1.77k op/s 2024-05-09T00:52:35.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:52:35 smithi131 ceph-mon[34547]: pgmap v558: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 929 MiB/s wr, 1.77k op/s 2024-05-09T00:52:35.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:52:35 smithi157 ceph-mon[34294]: pgmap v558: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 929 MiB/s wr, 1.77k op/s 2024-05-09T00:52:37.546 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:52:37 smithi005 ceph-mon[30365]: pgmap v559: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 940 MiB/s wr, 2.43k op/s 2024-05-09T00:52:37.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:52:37 smithi131 ceph-mon[34547]: pgmap v559: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 940 MiB/s wr, 2.43k op/s 2024-05-09T00:52:37.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:52:37 smithi157 ceph-mon[34294]: pgmap v559: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 940 MiB/s wr, 2.43k op/s 2024-05-09T00:52:39.377 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:52:39 smithi005 ceph-mon[30365]: pgmap v560: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 810 MiB/s wr, 2.42k op/s 2024-05-09T00:52:39.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:52:39 smithi131 ceph-mon[34547]: pgmap v560: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 810 MiB/s wr, 2.42k op/s 2024-05-09T00:52:39.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:52:39 smithi157 ceph-mon[34294]: pgmap v560: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 810 MiB/s wr, 2.42k op/s 2024-05-09T00:52:40.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:39.972+0000 7f0cf77fe700 1 -- 172.21.15.5:0/1109802777 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cec017400 con 0x7f0cf8107e40 2024-05-09T00:52:40.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:39.973+0000 7f0cfef0c700 1 -- 172.21.15.5:0/1109802777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cf8107e40 msgr2=0x7f0cf8108250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:52:40.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:39.973+0000 7f0cfef0c700 1 --2- 172.21.15.5:0/1109802777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cf8107e40 0x7f0cf8108250 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f0cec009920 tx=0x7f0cec009c30 comp rx=0 tx=0).stop 2024-05-09T00:52:40.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:39.974+0000 7f0cfef0c700 1 -- 172.21.15.5:0/1109802777 shutdown_connections 2024-05-09T00:52:40.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:39.974+0000 7f0cfef0c700 1 --2- 172.21.15.5:0/1109802777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0cf8108790 0x7f0cf81100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:40.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:39.974+0000 7f0cfef0c700 1 --2- 172.21.15.5:0/1109802777 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0cf8103f40 0x7f0cf8104350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:40.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:39.974+0000 7f0cfef0c700 1 --2- 172.21.15.5:0/1109802777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cf8107e40 0x7f0cf8108250 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:40.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:39.974+0000 7f0cfef0c700 1 -- 172.21.15.5:0/1109802777 >> 172.21.15.5:0/1109802777 conn(0x7f0cf80fe0d0 msgr2=0x7f0cf81004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:52:40.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:39.975+0000 7f0cfef0c700 1 -- 172.21.15.5:0/1109802777 shutdown_connections 2024-05-09T00:52:40.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:39.975+0000 7f0cfef0c700 1 -- 172.21.15.5:0/1109802777 wait complete. 2024-05-09T00:52:40.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:39.976+0000 7f0cfef0c700 1 Processor -- start 2024-05-09T00:52:40.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:39.976+0000 7f0cfef0c700 1 -- start start 2024-05-09T00:52:40.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:39.977+0000 7f0cfef0c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cf8103f40 0x7f0cf810c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:40.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:39.977+0000 7f0cfef0c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0cf8107e40 0x7f0cf810cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:40.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:39.977+0000 7f0cfef0c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0cf8108790 0x7f0cf81ad110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:40.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:39.977+0000 7f0cfef0c700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf8112310 con 0x7f0cf8103f40 2024-05-09T00:52:40.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:39.977+0000 7f0cfef0c700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf8112190 con 0x7f0cf8108790 2024-05-09T00:52:40.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:39.977+0000 7f0cfef0c700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf8112490 con 0x7f0cf8107e40 2024-05-09T00:52:40.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:39.977+0000 7f0cf7fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0cf8107e40 0x7f0cf810cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:40.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:39.977+0000 7f0cf7fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0cf8107e40 0x7f0cf810cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:54456/0 (socket says 172.21.15.5:54456) 2024-05-09T00:52:40.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:39.977+0000 7f0cf7fff700 1 -- 172.21.15.5:0/284374706 learned_addr learned my addr 172.21.15.5:0/284374706 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:52:40.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:39.977+0000 7f0cfd4a9700 1 --2- 172.21.15.5:0/284374706 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0cf8108790 0x7f0cf81ad110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:40.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:39.978+0000 7f0cfcca8700 1 --2- 172.21.15.5:0/284374706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cf8103f40 0x7f0cf810c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:40.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:39.979+0000 7f0cfcca8700 1 -- 172.21.15.5:0/284374706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0cf8107e40 msgr2=0x7f0cf810cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:52:40.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:39.979+0000 7f0cfcca8700 1 --2- 172.21.15.5:0/284374706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0cf8107e40 0x7f0cf810cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:40.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:39.979+0000 7f0cfcca8700 1 -- 172.21.15.5:0/284374706 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0cf8108790 msgr2=0x7f0cf81ad110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:52:40.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:39.979+0000 7f0cfcca8700 1 --2- 172.21.15.5:0/284374706 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0cf8108790 0x7f0cf81ad110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:40.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:39.979+0000 7f0cfcca8700 1 -- 172.21.15.5:0/284374706 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cf81ad650 con 0x7f0cf8103f40 2024-05-09T00:52:40.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:39.979+0000 7f0cfcca8700 1 --2- 172.21.15.5:0/284374706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cf8103f40 0x7f0cf810c9c0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8005fd0 tx=0x7f0ce800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:52:40.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:39.980+0000 7f0cf5ffb700 1 -- 172.21.15.5:0/284374706 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce800c9c0 con 0x7f0cf8103f40 2024-05-09T00:52:40.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:39.980+0000 7f0cf5ffb700 1 -- 172.21.15.5:0/284374706 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ce800cb60 con 0x7f0cf8103f40 2024-05-09T00:52:40.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:39.980+0000 7f0cf5ffb700 1 -- 172.21.15.5:0/284374706 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce8010e60 con 0x7f0cf8103f40 2024-05-09T00:52:40.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:39.980+0000 7f0cfef0c700 1 -- 172.21.15.5:0/284374706 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cf81ad8e0 con 0x7f0cf8103f40 2024-05-09T00:52:40.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:39.981+0000 7f0cfef0c700 1 -- 172.21.15.5:0/284374706 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cf81addd0 con 0x7f0cf8103f40 2024-05-09T00:52:40.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:39.982+0000 7f0cfef0c700 1 -- 172.21.15.5:0/284374706 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cf8102860 con 0x7f0cf8103f40 2024-05-09T00:52:40.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:39.986+0000 7f0cf5ffb700 1 -- 172.21.15.5:0/284374706 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f0ce8010430 con 0x7f0cf8103f40 2024-05-09T00:52:40.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:39.988+0000 7f0cf5ffb700 1 --2- 172.21.15.5:0/284374706 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f0ce00b2a50 0x7f0ce00b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:40.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:39.988+0000 7f0cf5ffb700 1 -- 172.21.15.5:0/284374706 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f0ce80e2740 con 0x7f0cf8103f40 2024-05-09T00:52:40.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:39.989+0000 7f0cf7fff700 1 --2- 172.21.15.5:0/284374706 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f0ce00b2a50 0x7f0ce00b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:40.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:39.991+0000 7f0cf7fff700 1 --2- 172.21.15.5:0/284374706 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f0ce00b2a50 0x7f0ce00b4f60 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0cec009680 tx=0x7f0cec019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:52:40.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:39.993+0000 7f0cf5ffb700 1 -- 172.21.15.5:0/284374706 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0ce80aa450 con 0x7f0cf8103f40 2024-05-09T00:52:40.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:40.628+0000 7f0cfef0c700 1 -- 172.21.15.5:0/284374706 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0cf802e5f0 con 0x7f0cf8103f40 2024-05-09T00:52:40.635 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:52:40.635 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2685771887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2685771887},{"type":"v1","addr":"172.21.15.5:6835","nonce":2685771887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/254017535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":254017535},{"type":"v1","addr":"172.21.15.5:6837","nonce":254017535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T00:43:45.856066+0000","modified":"2024-05-09T00:52:11.233619+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24508,"mds_1":24499,"mds_2":24494,"mds_3":24493,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14598":{"gid":14598,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2109255228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2109255228},{"type":"v1","addr":"172.21.15.5:6839","nonce":2109255228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/3809966329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3809966329},{"type":"v1","addr":"172.21.15.5:6841","nonce":3809966329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6833/3272897742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3272897742},{"type":"v1","addr":"172.21.15.131:6833","nonce":3272897742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1283093898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1283093898},{"type":"v1","addr":"172.21.15.131:6835","nonce":1283093898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/2809525455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2809525455},{"type":"v1","addr":"172.21.15.157:6833","nonce":2809525455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.131:6837/1633006186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":1633006186},{"type":"v1","addr":"172.21.15.131:6837","nonce":1633006186}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.157:6835/832898273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":832898273},{"type":"v1","addr":"172.21.15.157:6835","nonce":832898273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.131:6839/2748669631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2748669631},{"type":"v1","addr":"172.21.15.131:6839","nonce":2748669631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.157:6837/744863009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":744863009},{"type":"v1","addr":"172.21.15.157:6837","nonce":744863009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.157:6839/2779860399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":2779860399},{"type":"v1","addr":"172.21.15.157:6839","nonce":2779860399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24508,24494,24499,24493,24467]},"id":1}]} 2024-05-09T00:52:40.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:40.634+0000 7f0cf5ffb700 1 -- 172.21.15.5:0/284374706 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11926 (secure 0 0 0) 0x7f0ce80a9d50 con 0x7f0cf8103f40 2024-05-09T00:52:40.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:40.643+0000 7f0cfef0c700 1 -- 172.21.15.5:0/284374706 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f0ce00b2a50 msgr2=0x7f0ce00b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:52:40.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:40.643+0000 7f0cfef0c700 1 --2- 172.21.15.5:0/284374706 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f0ce00b2a50 0x7f0ce00b4f60 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0cec009680 tx=0x7f0cec019040 comp rx=0 tx=0).stop 2024-05-09T00:52:40.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:40.643+0000 7f0cfef0c700 1 -- 172.21.15.5:0/284374706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cf8103f40 msgr2=0x7f0cf810c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:52:40.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:40.643+0000 7f0cfef0c700 1 --2- 172.21.15.5:0/284374706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cf8103f40 0x7f0cf810c9c0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8005fd0 tx=0x7f0ce800eca0 comp rx=0 tx=0).stop 2024-05-09T00:52:40.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:40.649+0000 7f0cfef0c700 1 -- 172.21.15.5:0/284374706 shutdown_connections 2024-05-09T00:52:40.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:40.649+0000 7f0cfef0c700 1 --2- 172.21.15.5:0/284374706 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f0ce00b2a50 0x7f0ce00b4f60 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:40.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:40.649+0000 7f0cfef0c700 1 --2- 172.21.15.5:0/284374706 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0cf8108790 0x7f0cf81ad110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:40.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:40.649+0000 7f0cfef0c700 1 --2- 172.21.15.5:0/284374706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cf8103f40 0x7f0cf810c9c0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:40.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:40.649+0000 7f0cfef0c700 1 --2- 172.21.15.5:0/284374706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0cf8107e40 0x7f0cf810cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:40.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:40.649+0000 7f0cfef0c700 1 -- 172.21.15.5:0/284374706 >> 172.21.15.5:0/284374706 conn(0x7f0cf80fe0d0 msgr2=0x7f0cf81004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:52:40.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:40.653+0000 7f0cfef0c700 1 -- 172.21.15.5:0/284374706 shutdown_connections 2024-05-09T00:52:40.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:40.655+0000 7f0cfef0c700 1 -- 172.21.15.5:0/284374706 wait complete. 2024-05-09T00:52:40.658 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 31 2024-05-09T00:52:41.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:52:41 smithi005 ceph-mon[30365]: pgmap v561: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 939 MiB/s wr, 3.08k op/s 2024-05-09T00:52:41.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:52:41 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/284374706' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:52:41.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:52:41 smithi131 ceph-mon[34547]: pgmap v561: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 939 MiB/s wr, 3.08k op/s 2024-05-09T00:52:41.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:52:41 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/284374706' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:52:41.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:52:41 smithi157 ceph-mon[34294]: pgmap v561: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 939 MiB/s wr, 3.08k op/s 2024-05-09T00:52:41.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:52:41 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/284374706' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:52:42.617 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-09T00:52:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:43.530+0000 7f653dffb640 1 -- 172.21.15.5:0/2703119375 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55cf9db42350 con 0x55cf9dcdd320 2024-05-09T00:52:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:43.534+0000 7f653dffb640 10 client.16137 handle_command_reply: tid=0 2024-05-09T00:52:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:43.534+0000 7f656345c740 1 client.16137 shutdown 2024-05-09T00:52:43.538 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:52:43.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:43.538+0000 7f656345c740 1 -- 172.21.15.5:0/2703119375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf9d8b71b0 msgr2=0x55cf9dc81d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:52:43.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:43.538+0000 7f656345c740 1 --2- 172.21.15.5:0/2703119375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf9d8b71b0 0x55cf9dc81d30 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f654c013b50 tx=0x7f654c00b000 comp rx=0 tx=0).stop 2024-05-09T00:52:43.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:43.541+0000 7f656345c740 1 -- 172.21.15.5:0/2703119375 shutdown_connections 2024-05-09T00:52:43.543 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:43.541+0000 7f656345c740 1 --2- 172.21.15.5:0/2703119375 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55cf9dcdd320 0x55cf9dcdf740 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:43.543 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:43.541+0000 7f656345c740 1 --2- 172.21.15.5:0/2703119375 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cf9dcd9050 0x55cf9dcdb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:43.543 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:43.541+0000 7f656345c740 1 --2- 172.21.15.5:0/2703119375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf9d8b71b0 0x55cf9dc81d30 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:43.543 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:43.541+0000 7f656345c740 1 --2- 172.21.15.5:0/2703119375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cf9d8964d0 0x55cf9dc817f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:43.543 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:43.541+0000 7f656345c740 1 -- 172.21.15.5:0/2703119375 >> 172.21.15.5:0/2703119375 conn(0x55cf9d8ab300 msgr2=0x55cf9dc63b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:52:43.543 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:43.541+0000 7f653dffb640 0 client.16137 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:52:43.543 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:43.542+0000 7f656345c740 1 -- 172.21.15.5:0/2703119375 shutdown_connections 2024-05-09T00:52:43.543 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:43.542+0000 7f656345c740 1 -- 172.21.15.5:0/2703119375 wait complete. 2024-05-09T00:52:43.543 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:43.542+0000 7f656345c740 20 client.16137 trim_cache size 0 max 16384 2024-05-09T00:52:43.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-09T00:52:43.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-09T00:52:43.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-09T00:52:43.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:43.545+0000 7f655c5a8640 1 -- 172.21.15.5:0/3146459229 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f65400b29a0 msgr2=0x7f65400b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:52:43.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:43.545+0000 7f655c5a8640 1 --2- 172.21.15.5:0/3146459229 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f65400b29a0 0x7f65400b4eb0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f654c0139c0 tx=0x7f654c004a40 comp rx=0 tx=0).stop 2024-05-09T00:52:43.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:43.545+0000 7f655c5a8640 1 -- 172.21.15.5:0/3146459229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6554110020 msgr2=0x7f655407af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:52:43.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:43.545+0000 7f655c5a8640 1 --2- 172.21.15.5:0/3146459229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6554110020 0x7f655407af30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f654402aa00 tx=0x7f6544003b90 comp rx=0 tx=0).stop 2024-05-09T00:52:43.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:43.545+0000 7f655c5a8640 1 -- 172.21.15.5:0/3146459229 shutdown_connections 2024-05-09T00:52:43.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:43.545+0000 7f655c5a8640 1 --2- 172.21.15.5:0/3146459229 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f65400b29a0 0x7f65400b4eb0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:43.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:43.545+0000 7f655c5a8640 1 --2- 172.21.15.5:0/3146459229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6554110020 0x7f655407af30 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:43.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:43.545+0000 7f655c5a8640 1 --2- 172.21.15.5:0/3146459229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f655410b300 0x7f65541a9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:43.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:43.545+0000 7f655c5a8640 1 --2- 172.21.15.5:0/3146459229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f655410a480 0x7f65541a8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:43.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:43.545+0000 7f655c5a8640 1 -- 172.21.15.5:0/3146459229 >> 172.21.15.5:0/3146459229 conn(0x7f65540704c0 msgr2=0x7f655410c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:52:43.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:43.545+0000 7f655c5a8640 1 -- 172.21.15.5:0/3146459229 shutdown_connections 2024-05-09T00:52:43.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:43.545+0000 7f655c5a8640 1 -- 172.21.15.5:0/3146459229 wait complete. 2024-05-09T00:52:43.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:52:43 smithi131 ceph-mon[34547]: pgmap v562: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 797 MiB/s wr, 2.92k op/s 2024-05-09T00:52:43.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:52:43 smithi157 ceph-mon[34294]: pgmap v562: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 797 MiB/s wr, 2.92k op/s 2024-05-09T00:52:43.765 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:52:43 smithi005 ceph-mon[30365]: pgmap v562: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 797 MiB/s wr, 2.92k op/s 2024-05-09T00:52:45.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:52:45 smithi131 ceph-mon[34547]: pgmap v563: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 905 MiB/s wr, 3.29k op/s 2024-05-09T00:52:45.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:52:45 smithi157 ceph-mon[34294]: pgmap v563: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 905 MiB/s wr, 3.29k op/s 2024-05-09T00:52:45.790 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:52:45 smithi005 ceph-mon[30365]: pgmap v563: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 905 MiB/s wr, 3.29k op/s 2024-05-09T00:52:47.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:52:47 smithi157 ceph-mon[34294]: pgmap v564: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 925 MiB/s wr, 3.37k op/s 2024-05-09T00:52:47.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:52:47 smithi005 ceph-mon[30365]: pgmap v564: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 925 MiB/s wr, 3.37k op/s 2024-05-09T00:52:47.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:52:47 smithi131 ceph-mon[34547]: pgmap v564: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 925 MiB/s wr, 3.37k op/s 2024-05-09T00:52:49.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.214+0000 7f21ebb03700 1 -- 172.21.15.5:0/1621529199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f21e410a930 msgr2=0x7f21e410ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:52:49.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.214+0000 7f21ebb03700 1 --2- 172.21.15.5:0/1621529199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f21e410a930 0x7f21e410ad40 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f21e0009990 tx=0x7f21e0009ca0 comp rx=0 tx=0).stop 2024-05-09T00:52:49.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.216+0000 7f21ebb03700 1 -- 172.21.15.5:0/1621529199 shutdown_connections 2024-05-09T00:52:49.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.216+0000 7f21ebb03700 1 --2- 172.21.15.5:0/1621529199 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21e411d930 0x7f21e411fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:49.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.216+0000 7f21ebb03700 1 --2- 172.21.15.5:0/1621529199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f21e410a930 0x7f21e410ad40 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:49.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.216+0000 7f21ebb03700 1 --2- 172.21.15.5:0/1621529199 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21e410b280 0x7f21e411d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:49.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.216+0000 7f21ebb03700 1 -- 172.21.15.5:0/1621529199 >> 172.21.15.5:0/1621529199 conn(0x7f21e406f4d0 msgr2=0x7f21e406f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:52:49.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.224+0000 7f21ebb03700 1 -- 172.21.15.5:0/1621529199 shutdown_connections 2024-05-09T00:52:49.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.227+0000 7f21ebb03700 1 -- 172.21.15.5:0/1621529199 wait complete. 2024-05-09T00:52:49.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.228+0000 7f21ebb03700 1 Processor -- start 2024-05-09T00:52:49.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.231+0000 7f21ebb03700 1 -- start start 2024-05-09T00:52:49.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.231+0000 7f21ebb03700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21e410a930 0x7f21e411bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:49.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.231+0000 7f21ebb03700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21e410b280 0x7f21e4114e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:49.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.231+0000 7f21ebb03700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f21e411d930 0x7f21e4115490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:49.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.231+0000 7f21ebb03700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f21e4123330 con 0x7f21e411d930 2024-05-09T00:52:49.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.231+0000 7f21ebb03700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f21e41231b0 con 0x7f21e410a930 2024-05-09T00:52:49.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.231+0000 7f21ebb03700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f21e41234b0 con 0x7f21e410b280 2024-05-09T00:52:49.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.232+0000 7f21ea0a0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f21e411d930 0x7f21e4115490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:49.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.232+0000 7f21ea0a0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f21e411d930 0x7f21e4115490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55474/0 (socket says 172.21.15.5:55474) 2024-05-09T00:52:49.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.232+0000 7f21ea0a0700 1 -- 172.21.15.5:0/698828158 learned_addr learned my addr 172.21.15.5:0/698828158 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:52:49.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.234+0000 7f21e989f700 1 --2- 172.21.15.5:0/698828158 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21e410a930 0x7f21e411bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:49.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.235+0000 7f21e909e700 1 --2- 172.21.15.5:0/698828158 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21e410b280 0x7f21e4114e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:49.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.236+0000 7f21e989f700 1 -- 172.21.15.5:0/698828158 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21e410b280 msgr2=0x7f21e4114e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:52:49.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.236+0000 7f21e989f700 1 --2- 172.21.15.5:0/698828158 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21e410b280 0x7f21e4114e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:49.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.236+0000 7f21e989f700 1 -- 172.21.15.5:0/698828158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f21e411d930 msgr2=0x7f21e4115490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:52:49.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.236+0000 7f21e989f700 1 --2- 172.21.15.5:0/698828158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f21e411d930 0x7f21e4115490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:49.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.236+0000 7f21e989f700 1 -- 172.21.15.5:0/698828158 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21e4115c30 con 0x7f21e410a930 2024-05-09T00:52:49.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.238+0000 7f21e989f700 1 --2- 172.21.15.5:0/698828158 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21e410a930 0x7f21e411bd60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f21e0005950 tx=0x7f21e00037c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:52:49.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.241+0000 7f21daffd700 1 -- 172.21.15.5:0/698828158 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e0004830 con 0x7f21e410a930 2024-05-09T00:52:49.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.241+0000 7f21ebb03700 1 -- 172.21.15.5:0/698828158 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21e41198f0 con 0x7f21e410a930 2024-05-09T00:52:49.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.241+0000 7f21ebb03700 1 -- 172.21.15.5:0/698828158 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21e4119d70 con 0x7f21e410a930 2024-05-09T00:52:49.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.249+0000 7f21fa519b80 1 Processor -- start 2024-05-09T00:52:49.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.250+0000 7f21fa519b80 1 -- start start 2024-05-09T00:52:49.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.253+0000 7f21daffd700 1 -- 172.21.15.5:0/698828158 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21e0003ba0 con 0x7f21e410a930 2024-05-09T00:52:49.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.253+0000 7f21daffd700 1 -- 172.21.15.5:0/698828158 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e0005260 con 0x7f21e410a930 2024-05-09T00:52:49.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.253+0000 7f21daffd700 1 -- 172.21.15.5:0/698828158 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f21e00054c0 con 0x7f21e410a930 2024-05-09T00:52:49.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.255+0000 7f21daffd700 1 --2- 172.21.15.5:0/698828158 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f21d00b2ac0 0x7f21d00b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:49.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.258+0000 7f21fa519b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55575afdad00 0x55575b0e3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:49.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.258+0000 7f21fa519b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55575af604a0 0x55575b0e3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:49.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.258+0000 7f21fa519b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55575b0dd400 0x55575b0dd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:49.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.258+0000 7f21fa519b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55575b0000a0 con 0x55575af604a0 2024-05-09T00:52:49.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.258+0000 7f21fa519b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55575afa59a0 con 0x55575afdad00 2024-05-09T00:52:49.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.258+0000 7f21fa519b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55575affc060 con 0x55575b0dd400 2024-05-09T00:52:49.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.259+0000 7f21daffd700 1 -- 172.21.15.5:0/698828158 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f21e00e4050 con 0x7f21e410a930 2024-05-09T00:52:49.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.260+0000 7f21e989f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55575afdad00 0x55575b0e3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:49.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.261+0000 7f21e909e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55575af604a0 0x55575b0e3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:49.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.261+0000 7f21e989f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55575afdad00 0x55575b0e3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:56376/0 (socket says 172.21.15.5:56376) 2024-05-09T00:52:49.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.261+0000 7f21e989f700 1 -- 172.21.15.5:0/1267403671 learned_addr learned my addr 172.21.15.5:0/1267403671 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:52:49.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.262+0000 7f21ea0a0700 1 --2- 172.21.15.5:0/1267403671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55575b0dd400 0x55575b0dd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:49.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.262+0000 7f21e909e700 1 -- 172.21.15.5:0/1267403671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55575b0dd400 msgr2=0x55575b0dd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:52:49.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.262+0000 7f21e909e700 1 --2- 172.21.15.5:0/1267403671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55575b0dd400 0x55575b0dd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:49.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.262+0000 7f21e909e700 1 -- 172.21.15.5:0/1267403671 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55575afdad00 msgr2=0x55575b0e3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:52:49.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.262+0000 7f21e909e700 1 --2- 172.21.15.5:0/1267403671 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55575afdad00 0x55575b0e3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:49.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.262+0000 7f21e909e700 1 -- 172.21.15.5:0/1267403671 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55575af96500 con 0x55575af604a0 2024-05-09T00:52:49.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.265+0000 7f21e909e700 1 --2- 172.21.15.5:0/1267403671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55575af604a0 0x55575b0e3780 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f21d4005950 tx=0x7f21d400aeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cedbe8c653b2418e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:52:49.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.266+0000 7f21ea0a0700 1 --2- 172.21.15.5:0/698828158 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f21d00b2ac0 0x7f21d00b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:49.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.266+0000 7f21cffff700 1 -- 172.21.15.5:0/1267403671 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d400fdb0 con 0x55575af604a0 2024-05-09T00:52:49.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.266+0000 7f21cffff700 1 -- 172.21.15.5:0/1267403671 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21d40044d0 con 0x55575af604a0 2024-05-09T00:52:49.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.266+0000 7f21cffff700 1 -- 172.21.15.5:0/1267403671 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d40186a0 con 0x55575af604a0 2024-05-09T00:52:49.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.268+0000 7f21fa519b80 1 -- 172.21.15.5:0/1267403671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55575af604a0 msgr2=0x55575b0e3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:52:49.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.268+0000 7f21fa519b80 1 --2- 172.21.15.5:0/1267403671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55575af604a0 0x55575b0e3780 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f21d4005950 tx=0x7f21d400aeb0 comp rx=0 tx=0).stop 2024-05-09T00:52:49.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.273+0000 7f21fa519b80 1 -- 172.21.15.5:0/1267403671 shutdown_connections 2024-05-09T00:52:49.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.274+0000 7f21fa519b80 1 --2- 172.21.15.5:0/1267403671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55575b0dd400 0x55575b0dd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:49.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.274+0000 7f21fa519b80 1 --2- 172.21.15.5:0/1267403671 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55575afdad00 0x55575b0e3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:49.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.274+0000 7f21fa519b80 1 --2- 172.21.15.5:0/1267403671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55575af604a0 0x55575b0e3780 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:49.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.274+0000 7f21fa519b80 1 -- 172.21.15.5:0/1267403671 >> 172.21.15.5:0/1267403671 conn(0x55575af63f70 msgr2=0x55575b0d8220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:52:49.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.275+0000 7f21ea0a0700 1 --2- 172.21.15.5:0/698828158 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f21d00b2ac0 0x7f21d00b4fd0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f21dc007fd0 tx=0x7f21dc007c60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:52:49.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.276+0000 7f21fa519b80 1 -- 172.21.15.5:0/1267403671 shutdown_connections 2024-05-09T00:52:49.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.277+0000 7f21fa519b80 1 -- 172.21.15.5:0/1267403671 wait complete. 2024-05-09T00:52:49.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.278+0000 7f21fa519b80 1 -- start start 2024-05-09T00:52:49.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.278+0000 7f21fa519b80 1 Processor -- start 2024-05-09T00:52:49.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.280+0000 7f21fa519b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55575af604a0 0x55575b0e6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:49.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.280+0000 7f21fa519b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55575afdad00 0x55575b0e74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:49.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.281+0000 7f21fa519b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55575b0dd400 0x55575b0e7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:49.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.281+0000 7f21fa519b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55575b0de160 con 0x55575b0dd400 2024-05-09T00:52:49.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.281+0000 7f21fa519b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55575af9ab20 con 0x55575af604a0 2024-05-09T00:52:49.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.281+0000 7f21fa519b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55575b0ea020 con 0x55575afdad00 2024-05-09T00:52:49.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.281+0000 7f21fa519b80 10 client.0 _ll_register_callbacks cb 0x55575ae716c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:52:49.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.283+0000 7f21e989f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55575af604a0 0x55575b0e6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:49.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.283+0000 7f21e989f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55575af604a0 0x55575b0e6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:56378/0 (socket says 172.21.15.5:56378) 2024-05-09T00:52:49.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.283+0000 7f21e909e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55575afdad00 0x55575b0e74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:49.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.284+0000 7f21e989f700 1 -- 172.21.15.5:0/1430035667 learned_addr learned my addr 172.21.15.5:0/1430035667 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:52:49.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.288+0000 7f21e909e700 1 -- 172.21.15.5:0/1430035667 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55575af604a0 msgr2=0x55575b0e6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:52:49.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.288+0000 7f21e909e700 1 --2- 172.21.15.5:0/1430035667 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55575af604a0 0x55575b0e6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:49.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.288+0000 7f21e909e700 1 -- 172.21.15.5:0/1430035667 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55575b0dd400 msgr2=0x55575b0e7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:52:49.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.288+0000 7f21e909e700 1 --2- 172.21.15.5:0/1430035667 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55575b0dd400 0x55575b0e7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:49.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.288+0000 7f21e909e700 1 -- 172.21.15.5:0/1430035667 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55575ae7b490 con 0x55575afdad00 2024-05-09T00:52:49.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.291+0000 7f21e909e700 1 --2- 172.21.15.5:0/1430035667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55575afdad00 0x55575b0e74d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x55575b0d9a00 tx=0x7f21d4003940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0d787624896fa80 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:52:49.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.291+0000 7f21cd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-09T00:52:49.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.308+0000 7f21cd7fa700 1 -- 172.21.15.5:0/1430035667 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d4026020 con 0x55575afdad00 2024-05-09T00:52:49.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.308+0000 7f21cd7fa700 1 -- 172.21.15.5:0/1430035667 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f21d40041d0 con 0x55575afdad00 2024-05-09T00:52:49.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.309+0000 7f21fa519b80 1 -- 172.21.15.5:0/1430035667 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21d40044d0 con 0x55575afdad00 2024-05-09T00:52:49.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.314+0000 7f21cd7fa700 1 -- 172.21.15.5:0/1430035667 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f21d4004750 con 0x55575afdad00 2024-05-09T00:52:49.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.315+0000 7f21fa519b80 10 client.25717 fetch_fsmap learned FSMap version 31 2024-05-09T00:52:49.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.315+0000 7f21fa519b80 1 -- 172.21.15.5:0/1430035667 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55575b0f49f0 con 0x55575afdad00 2024-05-09T00:52:49.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.320+0000 7f21cd7fa700 1 -- 172.21.15.5:0/1430035667 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21d4004da0 con 0x55575afdad00 2024-05-09T00:52:49.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.320+0000 7f21cd7fa700 1 -- 172.21.15.5:0/1430035667 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f21d4022460 con 0x55575afdad00 2024-05-09T00:52:49.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.320+0000 7f21fa519b80 10 client.25717 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:52:49.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.320+0000 7f21fa519b80 10 client.25717 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T00:52:49.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.320+0000 7f21fa519b80 20 client.25717 populate_metadata read hostname 'smithi005' 2024-05-09T00:52:49.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.320+0000 7f21fa519b80 1 --2- 172.21.15.5:0/1430035667 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x55575b0f6ec0 0x55575b0f92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:49.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.320+0000 7f21fa519b80 4 client.25717 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:52:49.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.320+0000 7f21fa519b80 1 -- 172.21.15.5:0/1430035667 --> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f21d40044d0 con 0x55575b0f6ec0 2024-05-09T00:52:49.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.322+0000 7f21e989f700 1 --2- 172.21.15.5:0/1430035667 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x55575b0f6ec0 0x55575b0f92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:49.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.332+0000 7f21cd7fa700 10 client.25717 ms_handle_connect on v2:172.21.15.131:6836/1633006186 2024-05-09T00:52:49.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.332+0000 7f21e989f700 1 --2- 172.21.15.5:0/1430035667 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x55575b0f6ec0 0x55575b0f92e0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2551490761de5a99 server_cookie=e199ba12e0aafa64 in_seq=0 out_seq=0 2024-05-09T00:52:49.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.339+0000 7f21cd7fa700 1 -- 172.21.15.5:0/1430035667 <== mds.4 v2:172.21.15.131:6836/1633006186 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f21d40044d0 con 0x55575b0f6ec0 2024-05-09T00:52:49.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.339+0000 7f21cd7fa700 10 client.25717 handle_command_reply: tid=0 2024-05-09T00:52:49.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.339+0000 7f21fa519b80 1 client.25717 shutdown 2024-05-09T00:52:49.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.340+0000 7f21fa519b80 1 -- 172.21.15.5:0/1430035667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55575afdad00 msgr2=0x55575b0e74d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:52:49.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.340+0000 7f21fa519b80 1 --2- 172.21.15.5:0/1430035667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55575afdad00 0x55575b0e74d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x55575b0d9a00 tx=0x7f21d4003940 comp rx=0 tx=0).stop 2024-05-09T00:52:49.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.340+0000 7f21fa519b80 1 -- 172.21.15.5:0/1430035667 shutdown_connections 2024-05-09T00:52:49.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.340+0000 7f21fa519b80 1 --2- 172.21.15.5:0/1430035667 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x55575b0f6ec0 0x55575b0f92e0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:49.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.341+0000 7f21fa519b80 1 --2- 172.21.15.5:0/1430035667 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55575b0dd400 0x55575b0e7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:49.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.341+0000 7f21fa519b80 1 --2- 172.21.15.5:0/1430035667 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55575af604a0 0x55575b0e6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:49.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.341+0000 7f21fa519b80 1 --2- 172.21.15.5:0/1430035667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55575afdad00 0x55575b0e74d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:49.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.341+0000 7f21fa519b80 1 -- 172.21.15.5:0/1430035667 >> 172.21.15.5:0/1430035667 conn(0x55575af63f70 msgr2=0x55575b0d1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:52:49.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.341+0000 7f21cd7fa700 0 client.25717 ms_handle_reset on v2:172.21.15.131:6836/1633006186 2024-05-09T00:52:49.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.341+0000 7f21fa519b80 1 -- 172.21.15.5:0/1430035667 shutdown_connections 2024-05-09T00:52:49.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.342+0000 7f21fa519b80 1 -- 172.21.15.5:0/1430035667 wait complete. 2024-05-09T00:52:49.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.343+0000 7f21fa519b80 20 client.25717 trim_cache size 0 max 16384 2024-05-09T00:52:49.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.398+0000 7f21fa519b80 1 Processor -- start 2024-05-09T00:52:49.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.398+0000 7f21fa519b80 1 -- start start 2024-05-09T00:52:49.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.399+0000 7f21fa519b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55575af604a0 0x55575b0f3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:49.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.399+0000 7f21fa519b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55575afdad00 0x55575b0e2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:49.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.399+0000 7f21fa519b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55575b0dd400 0x55575b0e7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:49.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.399+0000 7f21fa519b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f21c800d500 con 0x55575b0dd400 2024-05-09T00:52:49.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.399+0000 7f21fa519b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f21c800f040 con 0x55575afdad00 2024-05-09T00:52:49.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.399+0000 7f21fa519b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f21c800dbd0 con 0x55575af604a0 2024-05-09T00:52:49.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.400+0000 7f21e989f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55575af604a0 0x55575b0f3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:49.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.400+0000 7f21e989f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55575af604a0 0x55575b0f3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:53690/0 (socket says 172.21.15.5:53690) 2024-05-09T00:52:49.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.400+0000 7f21e989f700 1 -- 172.21.15.5:0/1961841492 learned_addr learned my addr 172.21.15.5:0/1961841492 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:52:49.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.404+0000 7f21e989f700 1 -- 172.21.15.5:0/1961841492 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55575afdad00 msgr2=0x55575b0e2800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:52:49.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.404+0000 7f21e989f700 1 --2- 172.21.15.5:0/1961841492 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55575afdad00 0x55575b0e2800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:49.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.404+0000 7f21e989f700 1 -- 172.21.15.5:0/1961841492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55575b0dd400 msgr2=0x55575b0e7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:52:49.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.404+0000 7f21e989f700 1 --2- 172.21.15.5:0/1961841492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55575b0dd400 0x55575b0e7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:49.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.404+0000 7f21e989f700 1 -- 172.21.15.5:0/1961841492 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55575b0de600 con 0x55575af604a0 2024-05-09T00:52:49.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.405+0000 7f21e989f700 1 --2- 172.21.15.5:0/1961841492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55575af604a0 0x55575b0f3280 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f21e00a8960 tx=0x7f21e001a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffb521501a6bab33 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:52:49.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.406+0000 7f21cf7fe700 1 -- 172.21.15.5:0/1961841492 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e00c4070 con 0x55575af604a0 2024-05-09T00:52:49.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.406+0000 7f21cf7fe700 1 -- 172.21.15.5:0/1961841492 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21e00abdf0 con 0x55575af604a0 2024-05-09T00:52:49.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.406+0000 7f21cf7fe700 1 -- 172.21.15.5:0/1961841492 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e0004cd0 con 0x55575af604a0 2024-05-09T00:52:49.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.409+0000 7f21fa519b80 1 -- 172.21.15.5:0/1961841492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55575af604a0 msgr2=0x55575b0f3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:52:49.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.409+0000 7f21fa519b80 1 --2- 172.21.15.5:0/1961841492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55575af604a0 0x55575b0f3280 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f21e00a8960 tx=0x7f21e001a040 comp rx=0 tx=0).stop 2024-05-09T00:52:49.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.409+0000 7f21fa519b80 1 -- 172.21.15.5:0/1961841492 shutdown_connections 2024-05-09T00:52:49.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.409+0000 7f21fa519b80 1 --2- 172.21.15.5:0/1961841492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55575b0dd400 0x55575b0e7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:49.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.409+0000 7f21fa519b80 1 --2- 172.21.15.5:0/1961841492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55575af604a0 0x55575b0f3280 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:49.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.409+0000 7f21fa519b80 1 --2- 172.21.15.5:0/1961841492 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55575afdad00 0x55575b0e2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:49.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.409+0000 7f21fa519b80 1 -- 172.21.15.5:0/1961841492 >> 172.21.15.5:0/1961841492 conn(0x55575af63f70 msgr2=0x55575b0e20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:52:49.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.409+0000 7f21fa519b80 1 -- 172.21.15.5:0/1961841492 shutdown_connections 2024-05-09T00:52:49.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.409+0000 7f21fa519b80 1 -- 172.21.15.5:0/1961841492 wait complete. 2024-05-09T00:52:49.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.410+0000 7f21fa519b80 1 -- start start 2024-05-09T00:52:49.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.410+0000 7f21fa519b80 1 Processor -- start 2024-05-09T00:52:49.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.412+0000 7f21fa519b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55575af604a0 0x55575b0f8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:49.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.412+0000 7f21fa519b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55575afdad00 0x55575b0f9410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:49.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.413+0000 7f21fa519b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55575b0dd400 0x55575b15f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:49.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.413+0000 7f21fa519b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f21c800e970 con 0x55575af604a0 2024-05-09T00:52:49.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.413+0000 7f21fa519b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f21c800e2a0 con 0x55575b0dd400 2024-05-09T00:52:49.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.413+0000 7f21fa519b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f21c800b4a0 con 0x55575afdad00 2024-05-09T00:52:49.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.414+0000 7f21fa519b80 10 client.0 _ll_register_callbacks cb 0x55575ae716c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:52:49.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.415+0000 7f21e909e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55575afdad00 0x55575b0f9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:49.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.415+0000 7f21e909e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55575afdad00 0x55575b0f9410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:53696/0 (socket says 172.21.15.5:53696) 2024-05-09T00:52:49.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.415+0000 7f21e909e700 1 -- 172.21.15.5:0/171305182 learned_addr learned my addr 172.21.15.5:0/171305182 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:52:49.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.415+0000 7f21ea0a0700 1 --2- 172.21.15.5:0/171305182 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55575b0dd400 0x55575b15f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:49.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.419+0000 7f21e909e700 1 -- 172.21.15.5:0/171305182 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55575b0dd400 msgr2=0x55575b15f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:52:49.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.419+0000 7f21e909e700 1 --2- 172.21.15.5:0/171305182 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55575b0dd400 0x55575b15f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:49.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.419+0000 7f21e909e700 1 -- 172.21.15.5:0/171305182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55575af604a0 msgr2=0x55575b0f8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:52:49.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.419+0000 7f21e909e700 1 --2- 172.21.15.5:0/171305182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55575af604a0 0x55575b0f8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:49.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.420+0000 7f21e909e700 1 -- 172.21.15.5:0/171305182 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55575b0f9ca0 con 0x55575afdad00 2024-05-09T00:52:49.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.421+0000 7f21e909e700 1 --2- 172.21.15.5:0/171305182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55575afdad00 0x55575b0f9410 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f21d400ff90 tx=0x7f21d400a570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=694a665001de375f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:52:49.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.421+0000 7f21cd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-09T00:52:49.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.423+0000 7f21cd7fa700 1 -- 172.21.15.5:0/171305182 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d4037020 con 0x55575afdad00 2024-05-09T00:52:49.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.424+0000 7f21fa519b80 1 -- 172.21.15.5:0/171305182 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21e00abdf0 con 0x55575afdad00 2024-05-09T00:52:49.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.424+0000 7f21cd7fa700 1 -- 172.21.15.5:0/171305182 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f21d40043e0 con 0x55575afdad00 2024-05-09T00:52:49.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.425+0000 7f21cd7fa700 1 -- 172.21.15.5:0/171305182 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f21d4026070 con 0x55575afdad00 2024-05-09T00:52:49.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.428+0000 7f21fa519b80 10 client.25723 fetch_fsmap learned FSMap version 31 2024-05-09T00:52:49.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.428+0000 7f21fa519b80 1 -- 172.21.15.5:0/171305182 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55575b160960 con 0x55575afdad00 2024-05-09T00:52:49.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.430+0000 7f21cd7fa700 1 -- 172.21.15.5:0/171305182 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21e00abdf0 con 0x55575afdad00 2024-05-09T00:52:49.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.431+0000 7f21cd7fa700 1 -- 172.21.15.5:0/171305182 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f21d4010430 con 0x55575afdad00 2024-05-09T00:52:49.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.431+0000 7f21fa519b80 10 client.25723 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:52:49.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.431+0000 7f21fa519b80 10 client.25723 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T00:52:49.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.431+0000 7f21fa519b80 20 client.25723 populate_metadata read hostname 'smithi005' 2024-05-09T00:52:49.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.431+0000 7f21fa519b80 1 --2- 172.21.15.5:0/171305182 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x55575b161070 0x55575b163490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:49.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.431+0000 7f21fa519b80 4 client.25723 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:52:49.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.431+0000 7f21fa519b80 1 -- 172.21.15.5:0/171305182 --> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55575b00da80 con 0x55575b161070 2024-05-09T00:52:49.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.433+0000 7f21e989f700 1 --2- 172.21.15.5:0/171305182 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x55575b161070 0x55575b163490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:49.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.440+0000 7f21cd7fa700 10 client.25723 ms_handle_connect on v2:172.21.15.131:6836/1633006186 2024-05-09T00:52:49.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.440+0000 7f21e989f700 1 --2- 172.21.15.5:0/171305182 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x55575b161070 0x55575b163490 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=889fd126463b3eab server_cookie=b5f5e452eced40d4 in_seq=0 out_seq=0 2024-05-09T00:52:49.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.442+0000 7f21cd7fa700 1 -- 172.21.15.5:0/171305182 <== mds.4 v2:172.21.15.131:6836/1633006186 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55575b00da80 con 0x55575b161070 2024-05-09T00:52:49.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.442+0000 7f21cd7fa700 10 client.25723 handle_command_reply: tid=0 2024-05-09T00:52:49.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.442+0000 7f21fa519b80 1 client.25723 shutdown 2024-05-09T00:52:49.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.443+0000 7f21fa519b80 1 -- 172.21.15.5:0/171305182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55575afdad00 msgr2=0x55575b0f9410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:52:49.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.443+0000 7f21fa519b80 1 --2- 172.21.15.5:0/171305182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55575afdad00 0x55575b0f9410 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f21d400ff90 tx=0x7f21d400a570 comp rx=0 tx=0).stop 2024-05-09T00:52:49.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.443+0000 7f21fa519b80 1 -- 172.21.15.5:0/171305182 shutdown_connections 2024-05-09T00:52:49.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.443+0000 7f21fa519b80 1 --2- 172.21.15.5:0/171305182 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x55575b161070 0x55575b163490 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:49.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.443+0000 7f21fa519b80 1 --2- 172.21.15.5:0/171305182 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55575b0dd400 0x55575b15f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:49.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.443+0000 7f21fa519b80 1 --2- 172.21.15.5:0/171305182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55575af604a0 0x55575b0f8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:49.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.443+0000 7f21fa519b80 1 --2- 172.21.15.5:0/171305182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55575afdad00 0x55575b0f9410 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:49.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.443+0000 7f21fa519b80 1 -- 172.21.15.5:0/171305182 >> 172.21.15.5:0/171305182 conn(0x55575af63f70 msgr2=0x55575b0d1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:52:49.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.443+0000 7f21cd7fa700 0 client.25723 ms_handle_reset on v2:172.21.15.131:6836/1633006186 2024-05-09T00:52:49.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.444+0000 7f21fa519b80 1 -- 172.21.15.5:0/171305182 shutdown_connections 2024-05-09T00:52:49.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.444+0000 7f21fa519b80 1 -- 172.21.15.5:0/171305182 wait complete. 2024-05-09T00:52:49.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.444+0000 7f21fa519b80 20 client.25723 trim_cache size 0 max 16384 2024-05-09T00:52:49.445 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-09T00:52:49.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.446+0000 7f21ebb03700 1 -- 172.21.15.5:0/698828158 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f21d00b2ac0 msgr2=0x7f21d00b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:52:49.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.446+0000 7f21ebb03700 1 --2- 172.21.15.5:0/698828158 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f21d00b2ac0 0x7f21d00b4fd0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f21dc007fd0 tx=0x7f21dc007c60 comp rx=0 tx=0).stop 2024-05-09T00:52:49.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.446+0000 7f21ebb03700 1 -- 172.21.15.5:0/698828158 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21e410a930 msgr2=0x7f21e411bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:52:49.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.446+0000 7f21ebb03700 1 --2- 172.21.15.5:0/698828158 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21e410a930 0x7f21e411bd60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f21e0005950 tx=0x7f21e00037c0 comp rx=0 tx=0).stop 2024-05-09T00:52:49.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.447+0000 7f21ebb03700 1 -- 172.21.15.5:0/698828158 shutdown_connections 2024-05-09T00:52:49.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.447+0000 7f21ebb03700 1 --2- 172.21.15.5:0/698828158 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f21d00b2ac0 0x7f21d00b4fd0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:49.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.447+0000 7f21ebb03700 1 --2- 172.21.15.5:0/698828158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f21e411d930 0x7f21e4115490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:49.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.447+0000 7f21ebb03700 1 --2- 172.21.15.5:0/698828158 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21e410a930 0x7f21e411bd60 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:49.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.447+0000 7f21ebb03700 1 --2- 172.21.15.5:0/698828158 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21e410b280 0x7f21e4114e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:49.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.447+0000 7f21ebb03700 1 -- 172.21.15.5:0/698828158 >> 172.21.15.5:0/698828158 conn(0x7f21e406f4d0 msgr2=0x7f21e4072790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:52:49.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.448+0000 7f21ebb03700 1 -- 172.21.15.5:0/698828158 shutdown_connections 2024-05-09T00:52:49.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:49.449+0000 7f21ebb03700 1 -- 172.21.15.5:0/698828158 wait complete. 2024-05-09T00:52:49.471 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:52:49 smithi005 ceph-mon[30365]: pgmap v565: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 811 MiB/s wr, 2.68k op/s 2024-05-09T00:52:49.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:52:49 smithi157 ceph-mon[34294]: pgmap v565: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 811 MiB/s wr, 2.68k op/s 2024-05-09T00:52:49.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:52:49 smithi131 ceph-mon[34547]: pgmap v565: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 811 MiB/s wr, 2.68k op/s 2024-05-09T00:52:51.262 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-09T00:52:51.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:52:51 smithi157 ceph-mon[34294]: pgmap v566: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 970 MiB/s wr, 2.99k op/s 2024-05-09T00:52:51.790 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:52:51 smithi005 ceph-mon[30365]: pgmap v566: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 970 MiB/s wr, 2.99k op/s 2024-05-09T00:52:51.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:52:51 smithi131 ceph-mon[34547]: pgmap v566: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 970 MiB/s wr, 2.99k op/s 2024-05-09T00:52:52.022 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:52:53.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:52:53 smithi157 ceph-mon[34294]: pgmap v567: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 842 MiB/s wr, 2.40k op/s 2024-05-09T00:52:53.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:52:53 smithi005 ceph-mon[30365]: pgmap v567: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 842 MiB/s wr, 2.40k op/s 2024-05-09T00:52:53.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:52:53 smithi131 ceph-mon[34547]: pgmap v567: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 842 MiB/s wr, 2.40k op/s 2024-05-09T00:52:55.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:52:55 smithi157 ceph-mon[34294]: pgmap v568: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 945 MiB/s wr, 2.46k op/s 2024-05-09T00:52:55.790 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:52:55 smithi005 ceph-mon[30365]: pgmap v568: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 945 MiB/s wr, 2.46k op/s 2024-05-09T00:52:55.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:52:55 smithi131 ceph-mon[34547]: pgmap v568: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 945 MiB/s wr, 2.46k op/s 2024-05-09T00:52:56.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.875+0000 7f5ceefb2640 1 -- 172.21.15.5:0/1398543022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ce810b2c0 msgr2=0x7f5ce810b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:52:56.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.875+0000 7f5ceefb2640 1 --2- 172.21.15.5:0/1398543022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ce810b2c0 0x7f5ce810b6d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0009be0 tx=0x7f5ce002a6f0 comp rx=0 tx=0).stop 2024-05-09T00:52:56.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.880+0000 7f5ceefb2640 1 -- 172.21.15.5:0/1398543022 shutdown_connections 2024-05-09T00:52:56.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.880+0000 7f5ceefb2640 1 --2- 172.21.15.5:0/1398543022 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ce810ffe0 0x7f5ce81103f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:56.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.880+0000 7f5ceefb2640 1 --2- 172.21.15.5:0/1398543022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ce810b2c0 0x7f5ce810b6d0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:56.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.880+0000 7f5ceefb2640 1 --2- 172.21.15.5:0/1398543022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ce810a470 0x7f5ce810ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:56.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.880+0000 7f5ceefb2640 1 -- 172.21.15.5:0/1398543022 >> 172.21.15.5:0/1398543022 conn(0x7f5ce8070460 msgr2=0x7f5ce8072910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:52:56.886 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.885+0000 7f5ceefb2640 1 -- 172.21.15.5:0/1398543022 shutdown_connections 2024-05-09T00:52:56.890 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.889+0000 7f5ceefb2640 1 -- 172.21.15.5:0/1398543022 wait complete. 2024-05-09T00:52:56.891 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.890+0000 7f5ceefb2640 1 Processor -- start 2024-05-09T00:52:56.891 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.890+0000 7f5ceefb2640 1 -- start start 2024-05-09T00:52:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.891+0000 7f5ceefb2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ce810a470 0x7f5ce81a9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.891+0000 7f5ceefb2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ce810b2c0 0x7f5ce81a9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.891+0000 7f5ceefb2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ce810ffe0 0x7f5ce81b0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.891+0000 7f5ceefb2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce811e440 con 0x7f5ce810ffe0 2024-05-09T00:52:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.891+0000 7f5ceefb2640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce811e2c0 con 0x7f5ce810b2c0 2024-05-09T00:52:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.891+0000 7f5ceefb2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce811e5c0 con 0x7f5ce810a470 2024-05-09T00:52:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.892+0000 7f5cee7b1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ce810ffe0 0x7f5ce81b0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.893+0000 7f5cedfb0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ce810a470 0x7f5ce81a9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.893+0000 7f5cee7b1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ce810ffe0 0x7f5ce81b0510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60104/0 (socket says 172.21.15.5:60104) 2024-05-09T00:52:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.893+0000 7f5cee7b1640 1 -- 172.21.15.5:0/1662722036 learned_addr learned my addr 172.21.15.5:0/1662722036 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:52:56.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.894+0000 7f5ced7af640 1 --2- 172.21.15.5:0/1662722036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ce810b2c0 0x7f5ce81a9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.895+0000 7f5cee7b1640 1 -- 172.21.15.5:0/1662722036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ce810a470 msgr2=0x7f5ce81a9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:52:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.895+0000 7f5cee7b1640 1 --2- 172.21.15.5:0/1662722036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ce810a470 0x7f5ce81a9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.895+0000 7f5cee7b1640 1 -- 172.21.15.5:0/1662722036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ce810b2c0 msgr2=0x7f5ce81a9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:52:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.895+0000 7f5cee7b1640 1 --2- 172.21.15.5:0/1662722036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ce810b2c0 0x7f5ce81a9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.895+0000 7f5cee7b1640 1 -- 172.21.15.5:0/1662722036 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ce81b0b80 con 0x7f5ce810ffe0 2024-05-09T00:52:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.900+0000 7f5cee7b1640 1 --2- 172.21.15.5:0/1662722036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ce810ffe0 0x7f5ce81b0510 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc00b3f0 tx=0x7f5cdc00b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:52:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.901+0000 7f5cdaffd640 1 -- 172.21.15.5:0/1662722036 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc015020 con 0x7f5ce810ffe0 2024-05-09T00:52:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.901+0000 7f5ceefb2640 1 -- 172.21.15.5:0/1662722036 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ce81b0e70 con 0x7f5ce810ffe0 2024-05-09T00:52:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.901+0000 7f5ceefb2640 1 -- 172.21.15.5:0/1662722036 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ce81b1300 con 0x7f5ce810ffe0 2024-05-09T00:52:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.907+0000 7f5cdaffd640 1 -- 172.21.15.5:0/1662722036 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cdc011ce0 con 0x7f5ce810ffe0 2024-05-09T00:52:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.907+0000 7f5cdaffd640 1 -- 172.21.15.5:0/1662722036 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc010da0 con 0x7f5ce810ffe0 2024-05-09T00:52:56.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.910+0000 7f5cf719b740 1 Processor -- start 2024-05-09T00:52:56.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.920+0000 7f5cdaffd640 1 -- 172.21.15.5:0/1662722036 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f5cdc018020 con 0x7f5ce810ffe0 2024-05-09T00:52:56.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.921+0000 7f5cdaffd640 1 --2- 172.21.15.5:0/1662722036 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f5cd40b2830 0x7f5cd40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:56.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.922+0000 7f5cdaffd640 1 -- 172.21.15.5:0/1662722036 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f5cdc0e1a20 con 0x7f5ce810ffe0 2024-05-09T00:52:56.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.922+0000 7f5cf719b740 1 -- start start 2024-05-09T00:52:56.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.923+0000 7f5cf719b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1c21af1b0 0x55d1c25536c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:56.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.923+0000 7f5cf719b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1c21862f0 0x55d1c2553c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:56.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.923+0000 7f5cf719b740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1c218e4d0 0x55d1c255ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:56.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.923+0000 7f5cf719b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d1c24b77c0 con 0x55d1c21862f0 2024-05-09T00:52:56.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.923+0000 7f5cf719b740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55d1c24530e0 con 0x55d1c218e4d0 2024-05-09T00:52:56.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.923+0000 7f5cf719b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d1c24b3780 con 0x55d1c21af1b0 2024-05-09T00:52:56.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.924+0000 7f5cedfb0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1c21af1b0 0x55d1c25536c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:56.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.924+0000 7f5cee7b1640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1c218e4d0 0x55d1c255ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:56.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.924+0000 7f5cee7b1640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1c218e4d0 0x55d1c255ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:42080/0 (socket says 172.21.15.5:42080) 2024-05-09T00:52:56.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.924+0000 7f5cee7b1640 1 -- 172.21.15.5:0/404360485 learned_addr learned my addr 172.21.15.5:0/404360485 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:52:56.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.924+0000 7f5ced7af640 1 --2- 172.21.15.5:0/1662722036 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f5cd40b2830 0x7f5cd40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:56.928 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.925+0000 7f5ced7af640 1 --2- 172.21.15.5:0/404360485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1c21862f0 0x55d1c2553c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:56.928 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.926+0000 7f5cee7b1640 1 -- 172.21.15.5:0/404360485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1c21af1b0 msgr2=0x55d1c25536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:52:56.928 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.926+0000 7f5cee7b1640 1 --2- 172.21.15.5:0/404360485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1c21af1b0 0x55d1c25536c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:56.928 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.926+0000 7f5cee7b1640 1 -- 172.21.15.5:0/404360485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1c21862f0 msgr2=0x55d1c2553c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:52:56.928 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.926+0000 7f5cee7b1640 1 --2- 172.21.15.5:0/404360485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1c21862f0 0x55d1c2553c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:56.928 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.926+0000 7f5cee7b1640 1 -- 172.21.15.5:0/404360485 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1c247c7c0 con 0x55d1c218e4d0 2024-05-09T00:52:56.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.931+0000 7f5cee7b1640 1 --2- 172.21.15.5:0/404360485 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1c218e4d0 0x55d1c255ab40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc0a8cb0 tx=0x7f5cdc0a8ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e110aedc98542e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:52:56.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.931+0000 7f5ced7af640 1 --2- 172.21.15.5:0/1662722036 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f5cd40b2830 0x7f5cd40b4d40 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0006010 tx=0x7f5ce00303e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:52:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.932+0000 7f5cd3fff640 1 -- 172.21.15.5:0/404360485 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc0a96b0 con 0x55d1c218e4d0 2024-05-09T00:52:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.932+0000 7f5cd3fff640 1 -- 172.21.15.5:0/404360485 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cdc0a8fb0 con 0x55d1c218e4d0 2024-05-09T00:52:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.934+0000 7f5cf719b740 1 -- 172.21.15.5:0/404360485 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1c218e4d0 msgr2=0x55d1c255ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:52:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.934+0000 7f5cf719b740 1 --2- 172.21.15.5:0/404360485 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1c218e4d0 0x55d1c255ab40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc0a8cb0 tx=0x7f5cdc0a8ac0 comp rx=0 tx=0).stop 2024-05-09T00:52:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.935+0000 7f5cf719b740 1 -- 172.21.15.5:0/404360485 shutdown_connections 2024-05-09T00:52:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.935+0000 7f5cf719b740 1 --2- 172.21.15.5:0/404360485 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1c218e4d0 0x55d1c255ab40 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.935+0000 7f5cf719b740 1 --2- 172.21.15.5:0/404360485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1c21862f0 0x55d1c2553c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.935+0000 7f5cf719b740 1 --2- 172.21.15.5:0/404360485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1c21af1b0 0x55d1c25536c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:56.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.935+0000 7f5cf719b740 1 -- 172.21.15.5:0/404360485 >> 172.21.15.5:0/404360485 conn(0x55d1c21a3300 msgr2=0x55d1c254f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:52:56.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.935+0000 7f5cf719b740 1 -- 172.21.15.5:0/404360485 shutdown_connections 2024-05-09T00:52:56.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.935+0000 7f5cf719b740 1 -- 172.21.15.5:0/404360485 wait complete. 2024-05-09T00:52:56.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.936+0000 7f5cf719b740 1 -- start start 2024-05-09T00:52:56.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.936+0000 7f5cf719b740 1 Processor -- start 2024-05-09T00:52:56.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.937+0000 7f5cf719b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1c21862f0 0x55d1c255d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:56.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.937+0000 7f5cf719b740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1c218e4d0 0x55d1c255d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:56.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.937+0000 7f5cf719b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1c21af1b0 0x55d1c25647e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:56.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.937+0000 7f5cf719b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d1c255b570 con 0x55d1c21862f0 2024-05-09T00:52:56.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.937+0000 7f5cf719b740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55d1c243e9c0 con 0x55d1c218e4d0 2024-05-09T00:52:56.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.937+0000 7f5cf719b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d1c255b6f0 con 0x55d1c21af1b0 2024-05-09T00:52:56.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.938+0000 7f5cf719b740 10 client.0 _ll_register_callbacks cb 0x55d1c246b020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:52:56.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.938+0000 7f5cedfb0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1c21862f0 0x55d1c255d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:56.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.939+0000 7f5ced7af640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1c218e4d0 0x55d1c255d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:56.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.940+0000 7f5ced7af640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1c218e4d0 0x55d1c255d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:42090/0 (socket says 172.21.15.5:42090) 2024-05-09T00:52:56.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.940+0000 7f5ced7af640 1 -- 172.21.15.5:0/4236749852 learned_addr learned my addr 172.21.15.5:0/4236749852 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:52:56.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.941+0000 7f5cee7b1640 1 --2- 172.21.15.5:0/4236749852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1c21af1b0 0x55d1c25647e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:56.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.942+0000 7f5cee7b1640 1 -- 172.21.15.5:0/4236749852 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1c218e4d0 msgr2=0x55d1c255d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:52:56.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.942+0000 7f5cee7b1640 1 --2- 172.21.15.5:0/4236749852 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1c218e4d0 0x55d1c255d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:56.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.942+0000 7f5cee7b1640 1 -- 172.21.15.5:0/4236749852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1c21862f0 msgr2=0x55d1c255d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:52:56.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.942+0000 7f5cee7b1640 1 --2- 172.21.15.5:0/4236749852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1c21862f0 0x55d1c255d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:56.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.942+0000 7f5cee7b1640 1 -- 172.21.15.5:0/4236749852 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1c24651f0 con 0x55d1c21af1b0 2024-05-09T00:52:56.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.949+0000 7f5cee7b1640 1 --2- 172.21.15.5:0/4236749852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1c21af1b0 0x55d1c25647e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc00b3c0 tx=0x7f5cdc0b1e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b4b1e8ed45a5ffc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:52:56.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.949+0000 7f5cd17fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-09T00:52:56.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.953+0000 7f5cd17fa640 1 -- 172.21.15.5:0/4236749852 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc06c070 con 0x55d1c21af1b0 2024-05-09T00:52:56.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.953+0000 7f5cf719b740 1 -- 172.21.15.5:0/4236749852 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cdc0a8fb0 con 0x55d1c21af1b0 2024-05-09T00:52:56.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.953+0000 7f5cd17fa640 1 -- 172.21.15.5:0/4236749852 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f5cdc011a40 con 0x55d1c21af1b0 2024-05-09T00:52:56.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.954+0000 7f5cd17fa640 1 -- 172.21.15.5:0/4236749852 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5cdc0a96b0 con 0x55d1c21af1b0 2024-05-09T00:52:56.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.954+0000 7f5cf719b740 10 client.25732 fetch_fsmap learned FSMap version 31 2024-05-09T00:52:56.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.954+0000 7f5cf719b740 1 -- 172.21.15.5:0/4236749852 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d1c221a790 con 0x55d1c21af1b0 2024-05-09T00:52:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.963+0000 7f5cd17fa640 1 -- 172.21.15.5:0/4236749852 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cdc0a8fb0 con 0x55d1c21af1b0 2024-05-09T00:52:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.964+0000 7f5cd17fa640 1 -- 172.21.15.5:0/4236749852 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f5cdc0ae4f0 con 0x55d1c21af1b0 2024-05-09T00:52:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.964+0000 7f5cf719b740 10 client.25732 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:52:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.964+0000 7f5cf719b740 10 client.25732 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:52:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.964+0000 7f5cf719b740 20 client.25732 populate_metadata read hostname 'smithi005' 2024-05-09T00:52:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.965+0000 7f5cf719b740 1 --2- 172.21.15.5:0/4236749852 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55d1c216dd80 0x55d1c257bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.965+0000 7f5cf719b740 4 client.25732 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:52:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.965+0000 7f5cf719b740 1 -- 172.21.15.5:0/4236749852 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1c24c53d0 con 0x55d1c216dd80 2024-05-09T00:52:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.971+0000 7f5cedfb0640 1 --2- 172.21.15.5:0/4236749852 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55d1c216dd80 0x55d1c257bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.980+0000 7f5cedfb0640 1 --2- 172.21.15.5:0/4236749852 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55d1c216dd80 0x55d1c257bc90 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a0ef2338e43aff3 server_cookie=160b516fa9f232a4 in_seq=0 out_seq=0 2024-05-09T00:52:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.981+0000 7f5cd17fa640 10 client.25732 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:52:56.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.986+0000 7f5cd17fa640 1 -- 172.21.15.5:0/4236749852 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d1c24c53d0 con 0x55d1c216dd80 2024-05-09T00:52:56.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.986+0000 7f5cd17fa640 10 client.25732 handle_command_reply: tid=0 2024-05-09T00:52:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.990+0000 7f5cf719b740 1 client.25732 shutdown 2024-05-09T00:52:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.996+0000 7f5cf719b740 1 -- 172.21.15.5:0/4236749852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1c21af1b0 msgr2=0x55d1c25647e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:52:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.996+0000 7f5cf719b740 1 --2- 172.21.15.5:0/4236749852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1c21af1b0 0x55d1c25647e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc00b3c0 tx=0x7f5cdc0b1e80 comp rx=0 tx=0).stop 2024-05-09T00:52:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.996+0000 7f5cf719b740 1 -- 172.21.15.5:0/4236749852 shutdown_connections 2024-05-09T00:52:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.996+0000 7f5cf719b740 1 --2- 172.21.15.5:0/4236749852 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55d1c216dd80 0x55d1c257bc90 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.996+0000 7f5cf719b740 1 --2- 172.21.15.5:0/4236749852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1c21af1b0 0x55d1c25647e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.996+0000 7f5cf719b740 1 --2- 172.21.15.5:0/4236749852 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1c218e4d0 0x55d1c255d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.996+0000 7f5cf719b740 1 --2- 172.21.15.5:0/4236749852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1c21862f0 0x55d1c255d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.996+0000 7f5cf719b740 1 -- 172.21.15.5:0/4236749852 >> 172.21.15.5:0/4236749852 conn(0x55d1c21a3300 msgr2=0x55d1c254e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:52:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.997+0000 7f5cd17fa640 0 client.25732 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:52:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.997+0000 7f5cf719b740 1 -- 172.21.15.5:0/4236749852 shutdown_connections 2024-05-09T00:52:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.997+0000 7f5cf719b740 1 -- 172.21.15.5:0/4236749852 wait complete. 2024-05-09T00:52:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:56.997+0000 7f5cf719b740 20 client.25732 trim_cache size 0 max 16384 2024-05-09T00:52:57.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.039+0000 7f5cf719b740 1 Processor -- start 2024-05-09T00:52:57.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.039+0000 7f5cf719b740 1 -- start start 2024-05-09T00:52:57.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.040+0000 7f5cf719b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1c21862f0 0x55d1c2556a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:57.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.040+0000 7f5cf719b740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1c218e4d0 0x55d1c2564700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:57.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.040+0000 7f5cf719b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1c21af1b0 0x55d1c254ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:57.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.041+0000 7f5cf719b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5ccc00e500 con 0x55d1c21862f0 2024-05-09T00:52:57.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.041+0000 7f5cf719b740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5ccc010040 con 0x55d1c218e4d0 2024-05-09T00:52:57.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.041+0000 7f5cf719b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5ccc00ebd0 con 0x55d1c21af1b0 2024-05-09T00:52:57.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.041+0000 7f5cedfb0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1c21862f0 0x55d1c2556a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:57.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.041+0000 7f5cedfb0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1c21862f0 0x55d1c2556a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60136/0 (socket says 172.21.15.5:60136) 2024-05-09T00:52:57.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.041+0000 7f5cedfb0640 1 -- 172.21.15.5:0/1532022086 learned_addr learned my addr 172.21.15.5:0/1532022086 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:52:57.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.042+0000 7f5cee7b1640 1 --2- 172.21.15.5:0/1532022086 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1c21af1b0 0x55d1c254ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:57.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.042+0000 7f5ced7af640 1 --2- 172.21.15.5:0/1532022086 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1c218e4d0 0x55d1c2564700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:57.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.042+0000 7f5cedfb0640 1 -- 172.21.15.5:0/1532022086 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1c21af1b0 msgr2=0x55d1c254ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:52:57.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.043+0000 7f5cedfb0640 1 --2- 172.21.15.5:0/1532022086 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1c21af1b0 0x55d1c254ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:57.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.043+0000 7f5cedfb0640 1 -- 172.21.15.5:0/1532022086 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1c218e4d0 msgr2=0x55d1c2564700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:52:57.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.043+0000 7f5cedfb0640 1 --2- 172.21.15.5:0/1532022086 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1c218e4d0 0x55d1c2564700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:57.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.043+0000 7f5cedfb0640 1 -- 172.21.15.5:0/1532022086 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1c22915f0 con 0x55d1c21862f0 2024-05-09T00:52:57.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.043+0000 7f5cedfb0640 1 --2- 172.21.15.5:0/1532022086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1c21862f0 0x55d1c2556a00 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x55d1c2554890 tx=0x7f5ce4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=167a41d60f07e291 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:52:57.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.047+0000 7f5cd37fe640 1 -- 172.21.15.5:0/1532022086 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce4026070 con 0x55d1c21862f0 2024-05-09T00:52:57.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.047+0000 7f5cd37fe640 1 -- 172.21.15.5:0/1532022086 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ce40035f0 con 0x55d1c21862f0 2024-05-09T00:52:57.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.049+0000 7f5cf719b740 1 -- 172.21.15.5:0/1532022086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1c21862f0 msgr2=0x55d1c2556a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:52:57.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.049+0000 7f5cf719b740 1 --2- 172.21.15.5:0/1532022086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1c21862f0 0x55d1c2556a00 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x55d1c2554890 tx=0x7f5ce4009260 comp rx=0 tx=0).stop 2024-05-09T00:52:57.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.049+0000 7f5cf719b740 1 -- 172.21.15.5:0/1532022086 shutdown_connections 2024-05-09T00:52:57.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.049+0000 7f5cf719b740 1 --2- 172.21.15.5:0/1532022086 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1c21af1b0 0x55d1c254ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:57.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.049+0000 7f5cf719b740 1 --2- 172.21.15.5:0/1532022086 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1c218e4d0 0x55d1c2564700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:57.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.049+0000 7f5cf719b740 1 --2- 172.21.15.5:0/1532022086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1c21862f0 0x55d1c2556a00 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:57.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.049+0000 7f5cf719b740 1 -- 172.21.15.5:0/1532022086 >> 172.21.15.5:0/1532022086 conn(0x55d1c21a3300 msgr2=0x55d1c2557740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:52:57.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.050+0000 7f5cf719b740 1 -- 172.21.15.5:0/1532022086 shutdown_connections 2024-05-09T00:52:57.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.050+0000 7f5cf719b740 1 -- 172.21.15.5:0/1532022086 wait complete. 2024-05-09T00:52:57.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.050+0000 7f5cf719b740 1 -- start start 2024-05-09T00:52:57.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.050+0000 7f5cf719b740 1 Processor -- start 2024-05-09T00:52:57.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.051+0000 7f5cf719b740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1c218e4d0 0x55d1c25614f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:57.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.051+0000 7f5cf719b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1c21af1b0 0x55d1c2561a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:57.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.051+0000 7f5cf719b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1c21431c0 0x55d1c25e2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:57.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.051+0000 7f5cf719b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5ccc00f970 con 0x55d1c21af1b0 2024-05-09T00:52:57.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.051+0000 7f5cf719b740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5ccc00f2a0 con 0x55d1c218e4d0 2024-05-09T00:52:57.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.051+0000 7f5cf719b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5ccc00c590 con 0x55d1c21431c0 2024-05-09T00:52:57.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.051+0000 7f5cf719b740 10 client.0 _ll_register_callbacks cb 0x7f5ce0034070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:52:57.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.053+0000 7f5cee7b1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1c21431c0 0x55d1c25e2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:57.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.053+0000 7f5cee7b1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1c21431c0 0x55d1c25e2030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:52424/0 (socket says 172.21.15.5:52424) 2024-05-09T00:52:57.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.053+0000 7f5cee7b1640 1 -- 172.21.15.5:0/3463246133 learned_addr learned my addr 172.21.15.5:0/3463246133 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:52:57.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.054+0000 7f5ced7af640 1 --2- 172.21.15.5:0/3463246133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1c21af1b0 0x55d1c2561a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:57.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.054+0000 7f5cedfb0640 1 --2- 172.21.15.5:0/3463246133 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1c218e4d0 0x55d1c25614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:57.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.055+0000 7f5cee7b1640 1 -- 172.21.15.5:0/3463246133 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1c218e4d0 msgr2=0x55d1c25614f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:52:57.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.055+0000 7f5cee7b1640 1 --2- 172.21.15.5:0/3463246133 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1c218e4d0 0x55d1c25614f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:57.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.055+0000 7f5cee7b1640 1 -- 172.21.15.5:0/3463246133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1c21af1b0 msgr2=0x55d1c2561a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:52:57.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.055+0000 7f5cee7b1640 1 --2- 172.21.15.5:0/3463246133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1c21af1b0 0x55d1c2561a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:57.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.055+0000 7f5cee7b1640 1 -- 172.21.15.5:0/3463246133 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1c247bed0 con 0x55d1c21431c0 2024-05-09T00:52:57.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.058+0000 7f5cee7b1640 1 --2- 172.21.15.5:0/3463246133 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1c21431c0 0x55d1c25e2030 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc068a70 tx=0x7f5cdc011c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6802ce9180ba2dd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:52:57.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.058+0000 7f5cd17fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-09T00:52:57.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.059+0000 7f5cd17fa640 1 -- 172.21.15.5:0/3463246133 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc06c070 con 0x55d1c21431c0 2024-05-09T00:52:57.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.059+0000 7f5cd17fa640 1 -- 172.21.15.5:0/3463246133 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f5cdc068af0 con 0x55d1c21431c0 2024-05-09T00:52:57.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.060+0000 7f5cf719b740 1 -- 172.21.15.5:0/3463246133 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d1c24bf240 con 0x55d1c21431c0 2024-05-09T00:52:57.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.062+0000 7f5cd17fa640 1 -- 172.21.15.5:0/3463246133 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5cdc0a8fb0 con 0x55d1c21431c0 2024-05-09T00:52:57.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.062+0000 7f5cf719b740 10 client.25738 fetch_fsmap learned FSMap version 31 2024-05-09T00:52:57.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.062+0000 7f5cf719b740 1 -- 172.21.15.5:0/3463246133 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d1c25e33e0 con 0x55d1c21431c0 2024-05-09T00:52:57.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.063+0000 7f5cd17fa640 1 -- 172.21.15.5:0/3463246133 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55d1c24bf240 con 0x55d1c21431c0 2024-05-09T00:52:57.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.063+0000 7f5cd17fa640 1 -- 172.21.15.5:0/3463246133 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f5cdc0a7c80 con 0x55d1c21431c0 2024-05-09T00:52:57.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.065+0000 7f5cf719b740 10 client.25738 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:52:57.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.065+0000 7f5cf719b740 10 client.25738 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:52:57.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.065+0000 7f5cf719b740 20 client.25738 populate_metadata read hostname 'smithi005' 2024-05-09T00:52:57.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.065+0000 7f5cf719b740 1 --2- 172.21.15.5:0/3463246133 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55d1c25e3bd0 0x55d1c25e5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:57.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.065+0000 7f5cf719b740 4 client.25738 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:52:57.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.065+0000 7f5cf719b740 1 -- 172.21.15.5:0/3463246133 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d1c243a350 con 0x55d1c25e3bd0 2024-05-09T00:52:57.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.067+0000 7f5cedfb0640 1 --2- 172.21.15.5:0/3463246133 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55d1c25e3bd0 0x55d1c25e5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:57.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.068+0000 7f5cedfb0640 1 --2- 172.21.15.5:0/3463246133 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55d1c25e3bd0 0x55d1c25e5ff0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1870b19ebbb4d66 server_cookie=c2633d5d7919ed69 in_seq=0 out_seq=0 2024-05-09T00:52:57.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:52:57.069+0000 7f5cd17fa640 10 client.25738 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:52:57.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.437+0000 7f5095b08700 1 -- 172.21.15.5:0/2294140556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f509010b280 msgr2=0x7f509011d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:52:57.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.437+0000 7f5095b08700 1 --2- 172.21.15.5:0/2294140556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f509010b280 0x7f509011d3f0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f5080009990 tx=0x7f5080009ca0 comp rx=0 tx=0).stop 2024-05-09T00:52:57.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.442+0000 7f5095b08700 1 -- 172.21.15.5:0/2294140556 shutdown_connections 2024-05-09T00:52:57.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.443+0000 7f5095b08700 1 --2- 172.21.15.5:0/2294140556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f509011d930 0x7f509011fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:57.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.443+0000 7f5095b08700 1 --2- 172.21.15.5:0/2294140556 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f509010a930 0x7f509010ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:57.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.443+0000 7f5095b08700 1 --2- 172.21.15.5:0/2294140556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f509010b280 0x7f509011d3f0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:57.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.443+0000 7f5095b08700 1 -- 172.21.15.5:0/2294140556 >> 172.21.15.5:0/2294140556 conn(0x7f509006f4d0 msgr2=0x7f509006f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:52:57.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.443+0000 7f5095b08700 1 -- 172.21.15.5:0/2294140556 shutdown_connections 2024-05-09T00:52:57.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.444+0000 7f5095b08700 1 -- 172.21.15.5:0/2294140556 wait complete. 2024-05-09T00:52:57.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.446+0000 7f5095b08700 1 Processor -- start 2024-05-09T00:52:57.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.447+0000 7f5095b08700 1 -- start start 2024-05-09T00:52:57.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.447+0000 7f5095b08700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f509010a930 0x7f50901a9df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:57.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.447+0000 7f5095b08700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f509010b280 0x7f50901aa390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:57.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.447+0000 7f5095b08700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f509011d930 0x7f50901ae620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:57.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.448+0000 7f5095b08700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5090123420 con 0x7f509011d930 2024-05-09T00:52:57.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.448+0000 7f5095b08700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f50901232a0 con 0x7f509010a930 2024-05-09T00:52:57.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.448+0000 7f5095b08700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f50901235a0 con 0x7f509010b280 2024-05-09T00:52:57.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.452+0000 7f508ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f509011d930 0x7f50901ae620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:57.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.452+0000 7f508ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f509011d930 0x7f50901ae620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60162/0 (socket says 172.21.15.5:60162) 2024-05-09T00:52:57.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.452+0000 7f508ffff700 1 -- 172.21.15.5:0/3576449612 learned_addr learned my addr 172.21.15.5:0/3576449612 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:52:57.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.452+0000 7f508effd700 1 --2- 172.21.15.5:0/3576449612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f509010b280 0x7f50901aa390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:57.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.452+0000 7f508f7fe700 1 --2- 172.21.15.5:0/3576449612 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f509010a930 0x7f50901a9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:57.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.458+0000 7f508ffff700 1 -- 172.21.15.5:0/3576449612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f509010b280 msgr2=0x7f50901aa390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:52:57.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.458+0000 7f508ffff700 1 --2- 172.21.15.5:0/3576449612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f509010b280 0x7f50901aa390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:57.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.458+0000 7f508ffff700 1 -- 172.21.15.5:0/3576449612 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f509010a930 msgr2=0x7f50901a9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:52:57.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.458+0000 7f508ffff700 1 --2- 172.21.15.5:0/3576449612 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f509010a930 0x7f50901a9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:57.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.458+0000 7f508ffff700 1 -- 172.21.15.5:0/3576449612 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50901aecc0 con 0x7f509011d930 2024-05-09T00:52:57.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.464+0000 7f508ffff700 1 --2- 172.21.15.5:0/3576449612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f509011d930 0x7f50901ae620 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f508400b450 tx=0x7f508400b760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:52:57.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.467+0000 7f508cff9700 1 -- 172.21.15.5:0/3576449612 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5084015020 con 0x7f509011d930 2024-05-09T00:52:57.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.467+0000 7f5095b08700 1 -- 172.21.15.5:0/3576449612 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50901aefb0 con 0x7f509011d930 2024-05-09T00:52:57.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.467+0000 7f5095b08700 1 -- 172.21.15.5:0/3576449612 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50901b68d0 con 0x7f509011d930 2024-05-09T00:52:57.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.468+0000 7f508cff9700 1 -- 172.21.15.5:0/3576449612 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5084011ce0 con 0x7f509011d930 2024-05-09T00:52:57.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.468+0000 7f508cff9700 1 -- 172.21.15.5:0/3576449612 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5084010d40 con 0x7f509011d930 2024-05-09T00:52:57.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.479+0000 7f50a451eb80 1 Processor -- start 2024-05-09T00:52:57.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.484+0000 7f50a451eb80 1 -- start start 2024-05-09T00:52:57.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.485+0000 7f508cff9700 1 -- 172.21.15.5:0/3576449612 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f5084018020 con 0x7f509011d930 2024-05-09T00:52:57.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.486+0000 7f508cff9700 1 --2- 172.21.15.5:0/3576449612 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f50780b2a90 0x7f50780b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:57.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.487+0000 7f508cff9700 1 -- 172.21.15.5:0/3576449612 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f50840e17f0 con 0x7f509011d930 2024-05-09T00:52:57.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.489+0000 7f508effd700 1 --2- 172.21.15.5:0/3576449612 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f50780b2a90 0x7f50780b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:57.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.493+0000 7f508effd700 1 --2- 172.21.15.5:0/3576449612 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f50780b2a90 0x7f50780b4fa0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5080009750 tx=0x7f508001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:52:57.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.494+0000 7f50a451eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4f2985d00 0x55e4f2a8e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:57.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.494+0000 7f50a451eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e4f290b4a0 0x55e4f2a8e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:57.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.494+0000 7f50a451eb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e4f2a88400 0x55e4f2a88810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:57.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.494+0000 7f50a451eb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e4f29ab0a0 con 0x55e4f2985d00 2024-05-09T00:52:57.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.494+0000 7f50a451eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55e4f29509a0 con 0x55e4f290b4a0 2024-05-09T00:52:57.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.494+0000 7f50a451eb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55e4f29a7060 con 0x55e4f2a88400 2024-05-09T00:52:57.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.494+0000 7f508f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4f2985d00 0x55e4f2a8e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:57.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.494+0000 7f508f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4f2985d00 0x55e4f2a8e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60166/0 (socket says 172.21.15.5:60166) 2024-05-09T00:52:57.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.494+0000 7f508f7fe700 1 -- 172.21.15.5:0/1380783589 learned_addr learned my addr 172.21.15.5:0/1380783589 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:52:57.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.498+0000 7f508effd700 1 --2- 172.21.15.5:0/1380783589 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e4f290b4a0 0x55e4f2a8e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:57.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.499+0000 7f508ffff700 1 --2- 172.21.15.5:0/1380783589 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e4f2a88400 0x55e4f2a88810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:57.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.499+0000 7f508f7fe700 1 -- 172.21.15.5:0/1380783589 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e4f2a88400 msgr2=0x55e4f2a88810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:52:57.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.499+0000 7f508f7fe700 1 --2- 172.21.15.5:0/1380783589 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e4f2a88400 0x55e4f2a88810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:57.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.499+0000 7f508f7fe700 1 -- 172.21.15.5:0/1380783589 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e4f290b4a0 msgr2=0x55e4f2a8e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:52:57.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.499+0000 7f508f7fe700 1 --2- 172.21.15.5:0/1380783589 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e4f290b4a0 0x55e4f2a8e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:57.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.499+0000 7f508f7fe700 1 -- 172.21.15.5:0/1380783589 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4f2941500 con 0x55e4f2985d00 2024-05-09T00:52:57.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.500+0000 7f508f7fe700 1 --2- 172.21.15.5:0/1380783589 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4f2985d00 0x55e4f2a8e240 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f5088009fa0 tx=0x7f508800c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c6996abdea1492b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:52:57.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.500+0000 7f5075ffb700 1 -- 172.21.15.5:0/1380783589 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f508801a070 con 0x55e4f2985d00 2024-05-09T00:52:57.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.501+0000 7f5075ffb700 1 -- 172.21.15.5:0/1380783589 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5088015410 con 0x55e4f2985d00 2024-05-09T00:52:57.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.502+0000 7f50a451eb80 1 -- 172.21.15.5:0/1380783589 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4f2985d00 msgr2=0x55e4f2a8e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:52:57.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.502+0000 7f50a451eb80 1 --2- 172.21.15.5:0/1380783589 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4f2985d00 0x55e4f2a8e240 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f5088009fa0 tx=0x7f508800c420 comp rx=0 tx=0).stop 2024-05-09T00:52:57.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.502+0000 7f50a451eb80 1 -- 172.21.15.5:0/1380783589 shutdown_connections 2024-05-09T00:52:57.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.502+0000 7f50a451eb80 1 --2- 172.21.15.5:0/1380783589 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e4f2a88400 0x55e4f2a88810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:57.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.502+0000 7f50a451eb80 1 --2- 172.21.15.5:0/1380783589 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4f2985d00 0x55e4f2a8e240 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:57.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.502+0000 7f50a451eb80 1 --2- 172.21.15.5:0/1380783589 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e4f290b4a0 0x55e4f2a8e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:57.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.502+0000 7f50a451eb80 1 -- 172.21.15.5:0/1380783589 >> 172.21.15.5:0/1380783589 conn(0x55e4f290ef70 msgr2=0x55e4f2a83220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:52:57.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.504+0000 7f50a451eb80 1 -- 172.21.15.5:0/1380783589 shutdown_connections 2024-05-09T00:52:57.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.504+0000 7f50a451eb80 1 -- 172.21.15.5:0/1380783589 wait complete. 2024-05-09T00:52:57.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.505+0000 7f50a451eb80 1 -- start start 2024-05-09T00:52:57.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.508+0000 7f50a451eb80 1 Processor -- start 2024-05-09T00:52:57.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.508+0000 7f50a451eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e4f290b4a0 0x55e4f2a94ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:57.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.509+0000 7f50a451eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4f2a88400 0x55e4f2a91d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:57.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.509+0000 7f50a451eb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e4f2a92350 0x55e4f2a92820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:57.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.509+0000 7f50a451eb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e4f2a891b0 con 0x55e4f2a88400 2024-05-09T00:52:57.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.509+0000 7f50a451eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55e4f2945b20 con 0x55e4f290b4a0 2024-05-09T00:52:57.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.509+0000 7f50a451eb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55e4f2a95020 con 0x55e4f2a92350 2024-05-09T00:52:57.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.509+0000 7f50a451eb80 10 client.0 _ll_register_callbacks cb 0x55e4f281c6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:52:57.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.511+0000 7f508f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e4f290b4a0 0x55e4f2a94ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:57.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.511+0000 7f508f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e4f290b4a0 0x55e4f2a94ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:42134/0 (socket says 172.21.15.5:42134) 2024-05-09T00:52:57.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.511+0000 7f508f7fe700 1 -- 172.21.15.5:0/293665595 learned_addr learned my addr 172.21.15.5:0/293665595 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:52:57.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.512+0000 7f508ffff700 1 --2- 172.21.15.5:0/293665595 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e4f2a92350 0x55e4f2a92820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:57.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.512+0000 7f508effd700 1 --2- 172.21.15.5:0/293665595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4f2a88400 0x55e4f2a91d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:57.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.513+0000 7f508effd700 1 -- 172.21.15.5:0/293665595 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e4f2a92350 msgr2=0x55e4f2a92820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:52:57.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.513+0000 7f508effd700 1 --2- 172.21.15.5:0/293665595 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e4f2a92350 0x55e4f2a92820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:57.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.513+0000 7f508effd700 1 -- 172.21.15.5:0/293665595 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e4f290b4a0 msgr2=0x55e4f2a94ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:52:57.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.513+0000 7f508effd700 1 --2- 172.21.15.5:0/293665595 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e4f290b4a0 0x55e4f2a94ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:57.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.513+0000 7f508effd700 1 -- 172.21.15.5:0/293665595 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4f2826490 con 0x55e4f2a88400 2024-05-09T00:52:57.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.514+0000 7f508effd700 1 --2- 172.21.15.5:0/293665595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4f2a88400 0x55e4f2a91d70 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f50800098e0 tx=0x7f5080009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a4b9f61ac6c497d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:52:57.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.519+0000 7f506effd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:52:57.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.519+0000 7f506effd700 1 -- 172.21.15.5:0/293665595 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5080032060 con 0x55e4f2a88400 2024-05-09T00:52:57.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.520+0000 7f506effd700 1 -- 172.21.15.5:0/293665595 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f5080005480 con 0x55e4f2a88400 2024-05-09T00:52:57.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.524+0000 7f50a451eb80 1 -- 172.21.15.5:0/293665595 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e4f29b8a80 con 0x55e4f2a88400 2024-05-09T00:52:57.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.525+0000 7f506effd700 1 -- 172.21.15.5:0/293665595 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5080005680 con 0x55e4f2a88400 2024-05-09T00:52:57.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.525+0000 7f50a451eb80 10 client.16182 fetch_fsmap learned FSMap version 31 2024-05-09T00:52:57.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.525+0000 7f50a451eb80 1 -- 172.21.15.5:0/293665595 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e4f2ab0bb0 con 0x55e4f2a88400 2024-05-09T00:52:57.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.526+0000 7f506effd700 1 -- 172.21.15.5:0/293665595 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f508000f910 con 0x55e4f2a88400 2024-05-09T00:52:57.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.527+0000 7f506effd700 1 -- 172.21.15.5:0/293665595 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f508002d6f0 con 0x55e4f2a88400 2024-05-09T00:52:57.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.528+0000 7f50a451eb80 10 client.16182 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:52:57.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.528+0000 7f50a451eb80 10 client.16182 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T00:52:57.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.528+0000 7f50a451eb80 20 client.16182 populate_metadata read hostname 'smithi005' 2024-05-09T00:52:57.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.528+0000 7f50a451eb80 1 --2- 172.21.15.5:0/293665595 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x55e4f2ab3140 0x55e4f2ab5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:57.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.528+0000 7f50a451eb80 4 client.16182 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:52:57.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.528+0000 7f50a451eb80 1 -- 172.21.15.5:0/293665595 --> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4f29b8a80 con 0x55e4f2ab3140 2024-05-09T00:52:57.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.529+0000 7f508f7fe700 1 --2- 172.21.15.5:0/293665595 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x55e4f2ab3140 0x55e4f2ab5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:57.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.530+0000 7f508f7fe700 1 --2- 172.21.15.5:0/293665595 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x55e4f2ab3140 0x55e4f2ab5590 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cf663a42436043f4 server_cookie=981b137352ca2174 in_seq=0 out_seq=0 2024-05-09T00:52:57.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.530+0000 7f506effd700 10 client.16182 ms_handle_connect on v2:172.21.15.157:6834/832898273 2024-05-09T00:52:57.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.533+0000 7f506effd700 1 -- 172.21.15.5:0/293665595 <== mds.3 v2:172.21.15.157:6834/832898273 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e4f29b8a80 con 0x55e4f2ab3140 2024-05-09T00:52:57.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.533+0000 7f506effd700 10 client.16182 handle_command_reply: tid=0 2024-05-09T00:52:57.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.535+0000 7f50a451eb80 1 client.16182 shutdown 2024-05-09T00:52:57.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.537+0000 7f50a451eb80 1 -- 172.21.15.5:0/293665595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4f2a88400 msgr2=0x55e4f2a91d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:52:57.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.537+0000 7f50a451eb80 1 --2- 172.21.15.5:0/293665595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4f2a88400 0x55e4f2a91d70 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f50800098e0 tx=0x7f5080009310 comp rx=0 tx=0).stop 2024-05-09T00:52:57.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.543+0000 7f50a451eb80 1 -- 172.21.15.5:0/293665595 shutdown_connections 2024-05-09T00:52:57.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.543+0000 7f50a451eb80 1 --2- 172.21.15.5:0/293665595 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x55e4f2ab3140 0x55e4f2ab5590 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:57.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.543+0000 7f50a451eb80 1 --2- 172.21.15.5:0/293665595 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e4f2a92350 0x55e4f2a92820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:57.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.543+0000 7f50a451eb80 1 --2- 172.21.15.5:0/293665595 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e4f290b4a0 0x55e4f2a94ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:57.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.543+0000 7f50a451eb80 1 --2- 172.21.15.5:0/293665595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4f2a88400 0x55e4f2a91d70 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:57.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.543+0000 7f50a451eb80 1 -- 172.21.15.5:0/293665595 >> 172.21.15.5:0/293665595 conn(0x55e4f290ef70 msgr2=0x55e4f2a7cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:52:57.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.543+0000 7f506effd700 0 client.16182 ms_handle_reset on v2:172.21.15.157:6834/832898273 2024-05-09T00:52:57.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.547+0000 7f50a451eb80 1 -- 172.21.15.5:0/293665595 shutdown_connections 2024-05-09T00:52:57.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.549+0000 7f50a451eb80 1 -- 172.21.15.5:0/293665595 wait complete. 2024-05-09T00:52:57.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.550+0000 7f50a451eb80 20 client.16182 trim_cache size 0 max 16384 2024-05-09T00:52:57.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.585+0000 7f50a451eb80 1 Processor -- start 2024-05-09T00:52:57.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.589+0000 7f50a451eb80 1 -- start start 2024-05-09T00:52:57.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.590+0000 7f50a451eb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e4f290b4a0 0x55e4f2a9def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:57.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.590+0000 7f50a451eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e4f2a88400 0x55e4f2a90660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:57.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.590+0000 7f50a451eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4f2a92350 0x55e4f2a90ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:57.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.590+0000 7f50a451eb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f507c00d5c0 con 0x55e4f2a92350 2024-05-09T00:52:57.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.590+0000 7f50a451eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f507c00f100 con 0x55e4f2a88400 2024-05-09T00:52:57.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.590+0000 7f50a451eb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f507c00dc90 con 0x55e4f290b4a0 2024-05-09T00:52:57.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.593+0000 7f508f7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e4f290b4a0 0x55e4f2a9def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:57.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.593+0000 7f508effd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e4f2a88400 0x55e4f2a90660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:57.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.594+0000 7f508f7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e4f290b4a0 0x55e4f2a9def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:52446/0 (socket says 172.21.15.5:52446) 2024-05-09T00:52:57.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.594+0000 7f508f7fe700 1 -- 172.21.15.5:0/1513335921 learned_addr learned my addr 172.21.15.5:0/1513335921 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:52:57.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.596+0000 7f508f7fe700 1 -- 172.21.15.5:0/1513335921 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e4f2a88400 msgr2=0x55e4f2a90660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:52:57.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.596+0000 7f508f7fe700 1 --2- 172.21.15.5:0/1513335921 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e4f2a88400 0x55e4f2a90660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:57.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.596+0000 7f508f7fe700 1 -- 172.21.15.5:0/1513335921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4f2a92350 msgr2=0x55e4f2a90ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:52:57.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.596+0000 7f508f7fe700 1 --2- 172.21.15.5:0/1513335921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4f2a92350 0x55e4f2a90ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:57.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.596+0000 7f508f7fe700 1 -- 172.21.15.5:0/1513335921 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4f2a94610 con 0x55e4f290b4a0 2024-05-09T00:52:57.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.599+0000 7f508f7fe700 1 --2- 172.21.15.5:0/1513335921 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e4f290b4a0 0x55e4f2a9def0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5088009440 tx=0x7f5088009760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=133ba2ae937ca752 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:52:57.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.602+0000 7f50757fa700 1 -- 172.21.15.5:0/1513335921 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f508801a040 con 0x55e4f290b4a0 2024-05-09T00:52:57.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.602+0000 7f50757fa700 1 -- 172.21.15.5:0/1513335921 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f508802a070 con 0x55e4f290b4a0 2024-05-09T00:52:57.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.603+0000 7f50757fa700 1 -- 172.21.15.5:0/1513335921 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50880217d0 con 0x55e4f290b4a0 2024-05-09T00:52:57.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.604+0000 7f50a451eb80 1 -- 172.21.15.5:0/1513335921 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e4f290b4a0 msgr2=0x55e4f2a9def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:52:57.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.605+0000 7f50a451eb80 1 --2- 172.21.15.5:0/1513335921 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e4f290b4a0 0x55e4f2a9def0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5088009440 tx=0x7f5088009760 comp rx=0 tx=0).stop 2024-05-09T00:52:57.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.608+0000 7f50a451eb80 1 -- 172.21.15.5:0/1513335921 shutdown_connections 2024-05-09T00:52:57.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.608+0000 7f50a451eb80 1 --2- 172.21.15.5:0/1513335921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4f2a92350 0x55e4f2a90ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:57.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.608+0000 7f50a451eb80 1 --2- 172.21.15.5:0/1513335921 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e4f290b4a0 0x55e4f2a9def0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:57.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.609+0000 7f50a451eb80 1 --2- 172.21.15.5:0/1513335921 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e4f2a88400 0x55e4f2a90660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:57.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.609+0000 7f50a451eb80 1 -- 172.21.15.5:0/1513335921 >> 172.21.15.5:0/1513335921 conn(0x55e4f290ef70 msgr2=0x55e4f2ab5150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:52:57.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.619+0000 7f50a451eb80 1 -- 172.21.15.5:0/1513335921 shutdown_connections 2024-05-09T00:52:57.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.619+0000 7f50a451eb80 1 -- 172.21.15.5:0/1513335921 wait complete. 2024-05-09T00:52:57.633 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:52:57 smithi005 ceph-mon[30365]: pgmap v569: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 967 MiB/s wr, 2.31k op/s 2024-05-09T00:52:57.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.620+0000 7f50a451eb80 1 -- start start 2024-05-09T00:52:57.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.621+0000 7f50a451eb80 1 Processor -- start 2024-05-09T00:52:57.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.621+0000 7f50a451eb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e4f290b4a0 0x55e4f2abb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:57.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.622+0000 7f50a451eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4f2a88400 0x55e4f2a9a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:57.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.622+0000 7f50a451eb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e4f2a92350 0x55e4f2a9a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:57.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.622+0000 7f50a451eb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f507c00ea30 con 0x55e4f2a88400 2024-05-09T00:52:57.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.622+0000 7f50a451eb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f507c00e360 con 0x55e4f2a92350 2024-05-09T00:52:57.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.622+0000 7f50a451eb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f507c00b560 con 0x55e4f290b4a0 2024-05-09T00:52:57.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.623+0000 7f508ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e4f2a92350 0x55e4f2a9a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:57.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.623+0000 7f508ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e4f2a92350 0x55e4f2a9a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:42156/0 (socket says 172.21.15.5:42156) 2024-05-09T00:52:57.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.623+0000 7f508ffff700 1 -- 172.21.15.5:0/1905430246 learned_addr learned my addr 172.21.15.5:0/1905430246 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:52:57.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.623+0000 7f508f7fe700 1 --2- 172.21.15.5:0/1905430246 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e4f290b4a0 0x55e4f2abb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:57.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.623+0000 7f50a451eb80 10 client.0 _ll_register_callbacks cb 0x55e4f281c6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:52:57.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.629+0000 7f508ffff700 1 -- 172.21.15.5:0/1905430246 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e4f290b4a0 msgr2=0x55e4f2abb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:52:57.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.629+0000 7f508ffff700 1 --2- 172.21.15.5:0/1905430246 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e4f290b4a0 0x55e4f2abb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:57.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.629+0000 7f508ffff700 1 -- 172.21.15.5:0/1905430246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4f2a88400 msgr2=0x55e4f2a9a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:52:57.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.629+0000 7f508ffff700 1 --2- 172.21.15.5:0/1905430246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4f2a88400 0x55e4f2a9a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:57.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.629+0000 7f508ffff700 1 -- 172.21.15.5:0/1905430246 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4f2ab9dc0 con 0x55e4f2a92350 2024-05-09T00:52:57.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.638+0000 7f508ffff700 1 --2- 172.21.15.5:0/1905430246 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e4f2a92350 0x55e4f2a9a7b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f50840a91e0 tx=0x7f50840a8af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7de91b9eaeb0726 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:52:57.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.638+0000 7f506effd700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-09T00:52:57.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.644+0000 7f506effd700 1 -- 172.21.15.5:0/1905430246 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50840c2020 con 0x55e4f2a92350 2024-05-09T00:52:57.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.645+0000 7f506effd700 1 -- 172.21.15.5:0/1905430246 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f50840ad3a0 con 0x55e4f2a92350 2024-05-09T00:52:57.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.646+0000 7f50a451eb80 1 -- 172.21.15.5:0/1905430246 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f508802a070 con 0x55e4f2a92350 2024-05-09T00:52:57.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.649+0000 7f506effd700 1 -- 172.21.15.5:0/1905430246 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f50840a9250 con 0x55e4f2a92350 2024-05-09T00:52:57.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.650+0000 7f50a451eb80 10 client.25715 fetch_fsmap learned FSMap version 31 2024-05-09T00:52:57.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.650+0000 7f50a451eb80 1 -- 172.21.15.5:0/1905430246 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e4f2b1cbd0 con 0x55e4f2a92350 2024-05-09T00:52:57.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.653+0000 7f506effd700 1 -- 172.21.15.5:0/1905430246 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f508802a070 con 0x55e4f2a92350 2024-05-09T00:52:57.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.654+0000 7f506effd700 1 -- 172.21.15.5:0/1905430246 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f50840b1d20 con 0x55e4f2a92350 2024-05-09T00:52:57.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.655+0000 7f50a451eb80 10 client.25715 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:52:57.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.656+0000 7f50a451eb80 10 client.25715 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T00:52:57.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.656+0000 7f50a451eb80 20 client.25715 populate_metadata read hostname 'smithi005' 2024-05-09T00:52:57.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.656+0000 7f50a451eb80 1 --2- 172.21.15.5:0/1905430246 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x55e4f2b1d230 0x55e4f2b1f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:52:57.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.656+0000 7f50a451eb80 4 client.25715 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:52:57.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.656+0000 7f50a451eb80 1 -- 172.21.15.5:0/1905430246 --> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e4f29b2b20 con 0x55e4f2b1d230 2024-05-09T00:52:57.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.658+0000 7f508f7fe700 1 --2- 172.21.15.5:0/1905430246 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x55e4f2b1d230 0x55e4f2b1f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:52:57.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.663+0000 7f508f7fe700 1 --2- 172.21.15.5:0/1905430246 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x55e4f2b1d230 0x55e4f2b1f650 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=587bcd92edfd06da server_cookie=9d0706907c6643e2 in_seq=0 out_seq=0 2024-05-09T00:52:57.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.664+0000 7f506effd700 10 client.25715 ms_handle_connect on v2:172.21.15.157:6834/832898273 2024-05-09T00:52:57.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.664+0000 7f506effd700 1 -- 172.21.15.5:0/1905430246 <== mds.3 v2:172.21.15.157:6834/832898273 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e4f29b2b20 con 0x55e4f2b1d230 2024-05-09T00:52:57.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.665+0000 7f506effd700 10 client.25715 handle_command_reply: tid=0 2024-05-09T00:52:57.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.665+0000 7f50a451eb80 1 client.25715 shutdown 2024-05-09T00:52:57.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.669+0000 7f50a451eb80 1 -- 172.21.15.5:0/1905430246 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e4f2a92350 msgr2=0x55e4f2a9a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:52:57.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.669+0000 7f50a451eb80 1 --2- 172.21.15.5:0/1905430246 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e4f2a92350 0x55e4f2a9a7b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f50840a91e0 tx=0x7f50840a8af0 comp rx=0 tx=0).stop 2024-05-09T00:52:57.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.669+0000 7f50a451eb80 1 -- 172.21.15.5:0/1905430246 shutdown_connections 2024-05-09T00:52:57.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.669+0000 7f50a451eb80 1 --2- 172.21.15.5:0/1905430246 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x55e4f2b1d230 0x55e4f2b1f650 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:57.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.670+0000 7f50a451eb80 1 --2- 172.21.15.5:0/1905430246 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e4f2a92350 0x55e4f2a9a7b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:57.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.670+0000 7f50a451eb80 1 --2- 172.21.15.5:0/1905430246 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e4f290b4a0 0x55e4f2abb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:57.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.670+0000 7f50a451eb80 1 --2- 172.21.15.5:0/1905430246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4f2a88400 0x55e4f2a9a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:57.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.670+0000 7f50a451eb80 1 -- 172.21.15.5:0/1905430246 >> 172.21.15.5:0/1905430246 conn(0x55e4f290ef70 msgr2=0x55e4f2ab5150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:52:57.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.670+0000 7f506effd700 0 client.25715 ms_handle_reset on v2:172.21.15.157:6834/832898273 2024-05-09T00:52:57.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.672+0000 7f50a451eb80 1 -- 172.21.15.5:0/1905430246 shutdown_connections 2024-05-09T00:52:57.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.672+0000 7f50a451eb80 1 -- 172.21.15.5:0/1905430246 wait complete. 2024-05-09T00:52:57.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.673+0000 7f50a451eb80 20 client.25715 trim_cache size 0 max 16384 2024-05-09T00:52:57.675 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-09T00:52:57.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.679+0000 7f5095b08700 1 -- 172.21.15.5:0/3576449612 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f50780b2a90 msgr2=0x7f50780b4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:52:57.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.679+0000 7f5095b08700 1 --2- 172.21.15.5:0/3576449612 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f50780b2a90 0x7f50780b4fa0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5080009750 tx=0x7f508001a040 comp rx=0 tx=0).stop 2024-05-09T00:52:57.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.679+0000 7f5095b08700 1 -- 172.21.15.5:0/3576449612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f509011d930 msgr2=0x7f50901ae620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:52:57.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.679+0000 7f5095b08700 1 --2- 172.21.15.5:0/3576449612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f509011d930 0x7f50901ae620 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f508400b450 tx=0x7f508400b760 comp rx=0 tx=0).stop 2024-05-09T00:52:57.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.679+0000 7f5095b08700 1 -- 172.21.15.5:0/3576449612 shutdown_connections 2024-05-09T00:52:57.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.679+0000 7f5095b08700 1 --2- 172.21.15.5:0/3576449612 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f50780b2a90 0x7f50780b4fa0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:57.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.679+0000 7f5095b08700 1 --2- 172.21.15.5:0/3576449612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f509011d930 0x7f50901ae620 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:57.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.679+0000 7f5095b08700 1 --2- 172.21.15.5:0/3576449612 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f509010a930 0x7f50901a9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:57.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.679+0000 7f5095b08700 1 --2- 172.21.15.5:0/3576449612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f509010b280 0x7f50901aa390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:52:57.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.679+0000 7f5095b08700 1 -- 172.21.15.5:0/3576449612 >> 172.21.15.5:0/3576449612 conn(0x7f509006f4d0 msgr2=0x7f509011b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:52:57.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.681+0000 7f5095b08700 1 -- 172.21.15.5:0/3576449612 shutdown_connections 2024-05-09T00:52:57.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:52:57.681+0000 7f5095b08700 1 -- 172.21.15.5:0/3576449612 wait complete. 2024-05-09T00:52:57.685 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:52:57 smithi157 ceph-mon[34294]: pgmap v569: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 967 MiB/s wr, 2.31k op/s 2024-05-09T00:52:57.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:52:57 smithi131 ceph-mon[34547]: pgmap v569: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 967 MiB/s wr, 2.31k op/s 2024-05-09T00:52:59.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:52:59 smithi157 ceph-mon[34294]: pgmap v570: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 838 MiB/s wr, 1.77k op/s 2024-05-09T00:52:59.790 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:52:59 smithi005 ceph-mon[30365]: pgmap v570: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 838 MiB/s wr, 1.77k op/s 2024-05-09T00:52:59.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:52:59 smithi131 ceph-mon[34547]: pgmap v570: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 838 MiB/s wr, 1.77k op/s 2024-05-09T00:53:00.421 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T00:53:01.327 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:53:01.776 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:53:01 smithi005 ceph-mon[30365]: pgmap v571: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 994 MiB/s wr, 2.09k op/s 2024-05-09T00:53:01.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:53:01 smithi131 ceph-mon[34547]: pgmap v571: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 994 MiB/s wr, 2.09k op/s 2024-05-09T00:53:01.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:53:01 smithi157 ceph-mon[34294]: pgmap v571: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 994 MiB/s wr, 2.09k op/s 2024-05-09T00:53:03.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:53:03 smithi005 ceph-mon[30365]: pgmap v572: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 860 MiB/s wr, 1.76k op/s 2024-05-09T00:53:03.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:53:03 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:53:03.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:53:03 smithi131 ceph-mon[34547]: pgmap v572: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 860 MiB/s wr, 1.76k op/s 2024-05-09T00:53:03.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:53:03 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:53:03.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:53:03 smithi157 ceph-mon[34294]: pgmap v572: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 860 MiB/s wr, 1.76k op/s 2024-05-09T00:53:03.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:53:03 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:53:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:04.007+0000 7f5cd17fa640 1 -- 172.21.15.5:0/3463246133 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55d1c243a350 con 0x55d1c25e3bd0 2024-05-09T00:53:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:04.007+0000 7f5cd17fa640 10 client.25738 handle_command_reply: tid=0 2024-05-09T00:53:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:04.007+0000 7f5cf719b740 1 client.25738 shutdown 2024-05-09T00:53:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:04.009+0000 7f5cf719b740 1 -- 172.21.15.5:0/3463246133 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1c21431c0 msgr2=0x55d1c25e2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:53:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:04.009+0000 7f5cf719b740 1 --2- 172.21.15.5:0/3463246133 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1c21431c0 0x55d1c25e2030 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc068a70 tx=0x7f5cdc011c70 comp rx=0 tx=0).stop 2024-05-09T00:53:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:04.009+0000 7f5cf719b740 1 -- 172.21.15.5:0/3463246133 shutdown_connections 2024-05-09T00:53:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:04.009+0000 7f5cf719b740 1 --2- 172.21.15.5:0/3463246133 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55d1c25e3bd0 0x55d1c25e5ff0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:04.009+0000 7f5cf719b740 1 --2- 172.21.15.5:0/3463246133 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1c21431c0 0x55d1c25e2030 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:04.009+0000 7f5cf719b740 1 --2- 172.21.15.5:0/3463246133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1c21af1b0 0x55d1c2561a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:04.009+0000 7f5cf719b740 1 --2- 172.21.15.5:0/3463246133 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1c218e4d0 0x55d1c25614f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:04.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:04.009+0000 7f5cf719b740 1 -- 172.21.15.5:0/3463246133 >> 172.21.15.5:0/3463246133 conn(0x55d1c21a3300 msgr2=0x55d1c2551e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:53:04.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:04.009+0000 7f5cd17fa640 0 client.25738 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:53:04.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:04.009+0000 7f5cf719b740 1 -- 172.21.15.5:0/3463246133 shutdown_connections 2024-05-09T00:53:04.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:04.009+0000 7f5cf719b740 1 -- 172.21.15.5:0/3463246133 wait complete. 2024-05-09T00:53:04.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:04.010+0000 7f5cf719b740 20 client.25738 trim_cache size 0 max 16384 2024-05-09T00:53:04.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-09T00:53:04.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-09T00:53:04.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-09T00:53:04.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:04.018+0000 7f5ceefb2640 1 -- 172.21.15.5:0/1662722036 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f5cd40b2830 msgr2=0x7f5cd40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:53:04.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:04.018+0000 7f5ceefb2640 1 --2- 172.21.15.5:0/1662722036 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f5cd40b2830 0x7f5cd40b4d40 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0006010 tx=0x7f5ce00303e0 comp rx=0 tx=0).stop 2024-05-09T00:53:04.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:04.018+0000 7f5ceefb2640 1 -- 172.21.15.5:0/1662722036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ce810ffe0 msgr2=0x7f5ce81b0510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:53:04.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:04.018+0000 7f5ceefb2640 1 --2- 172.21.15.5:0/1662722036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ce810ffe0 0x7f5ce81b0510 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc00b3f0 tx=0x7f5cdc00b7f0 comp rx=0 tx=0).stop 2024-05-09T00:53:04.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:04.018+0000 7f5ceefb2640 1 -- 172.21.15.5:0/1662722036 shutdown_connections 2024-05-09T00:53:04.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:04.018+0000 7f5ceefb2640 1 --2- 172.21.15.5:0/1662722036 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f5cd40b2830 0x7f5cd40b4d40 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:04.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:04.018+0000 7f5ceefb2640 1 --2- 172.21.15.5:0/1662722036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ce810ffe0 0x7f5ce81b0510 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:04.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:04.018+0000 7f5ceefb2640 1 --2- 172.21.15.5:0/1662722036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ce810b2c0 0x7f5ce81a9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:04.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:04.018+0000 7f5ceefb2640 1 --2- 172.21.15.5:0/1662722036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ce810a470 0x7f5ce81a9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:04.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:04.018+0000 7f5ceefb2640 1 -- 172.21.15.5:0/1662722036 >> 172.21.15.5:0/1662722036 conn(0x7f5ce8070460 msgr2=0x7f5ce810bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:53:04.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:04.018+0000 7f5ceefb2640 1 -- 172.21.15.5:0/1662722036 shutdown_connections 2024-05-09T00:53:04.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:04.018+0000 7f5ceefb2640 1 -- 172.21.15.5:0/1662722036 wait complete. 2024-05-09T00:53:04.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:53:04 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:53:04.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:53:04 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:53:04.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:53:04 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:53:04.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:53:04 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:53:04.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:53:04 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:53:04.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:53:04 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:53:04.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:53:04 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:53:04.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:53:04 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:53:04.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:53:04 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:53:04.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:53:04 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:53:04.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:53:04 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:53:04.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:53:04 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:53:04.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:53:04 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:53:04.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:53:04 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:53:04.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:53:04 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:53:04.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:53:04 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:53:04.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:53:04 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:53:04.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:53:04 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:53:04.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:53:04 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:53:04.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:53:04 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:53:04.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:53:04 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:53:04.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:53:04 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:53:04.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:53:04 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:53:04.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:53:04 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:53:05.694 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:53:05 smithi005 ceph-mon[30365]: Adjusting osd_memory_target on smithi131 to 176.2M 2024-05-09T00:53:05.694 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:53:05 smithi005 ceph-mon[30365]: Unable to set osd_memory_target on smithi131 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-05-09T00:53:05.694 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:53:05 smithi005 ceph-mon[30365]: pgmap v573: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 911 MiB/s wr, 2.09k op/s 2024-05-09T00:53:05.694 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:53:05 smithi005 ceph-mon[30365]: Adjusting osd_memory_target on smithi157 to 176.2M 2024-05-09T00:53:05.694 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:53:05 smithi005 ceph-mon[30365]: Unable to set osd_memory_target on smithi157 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-05-09T00:53:05.755 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:53:05 smithi131 ceph-mon[34547]: Adjusting osd_memory_target on smithi131 to 176.2M 2024-05-09T00:53:05.756 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:53:05 smithi131 ceph-mon[34547]: Unable to set osd_memory_target on smithi131 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-05-09T00:53:05.756 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:53:05 smithi131 ceph-mon[34547]: pgmap v573: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 911 MiB/s wr, 2.09k op/s 2024-05-09T00:53:05.756 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:53:05 smithi131 ceph-mon[34547]: Adjusting osd_memory_target on smithi157 to 176.2M 2024-05-09T00:53:05.756 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:53:05 smithi131 ceph-mon[34547]: Unable to set osd_memory_target on smithi157 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-05-09T00:53:05.837 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:53:05 smithi157 ceph-mon[34294]: Adjusting osd_memory_target on smithi131 to 176.2M 2024-05-09T00:53:05.837 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:53:05 smithi157 ceph-mon[34294]: Unable to set osd_memory_target on smithi131 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-05-09T00:53:05.837 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:53:05 smithi157 ceph-mon[34294]: pgmap v573: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 911 MiB/s wr, 2.09k op/s 2024-05-09T00:53:05.837 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:53:05 smithi157 ceph-mon[34294]: Adjusting osd_memory_target on smithi157 to 176.2M 2024-05-09T00:53:05.837 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:53:05 smithi157 ceph-mon[34294]: Unable to set osd_memory_target on smithi157 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-05-09T00:53:06.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.676+0000 7fd06c787700 1 -- 172.21.15.5:0/1194130735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd06410a990 msgr2=0x7fd06410ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:53:06.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.676+0000 7fd06c787700 1 --2- 172.21.15.5:0/1194130735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd06410a990 0x7fd06410ada0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd05c00b0a0 tx=0x7fd05c00b3b0 comp rx=0 tx=0).stop 2024-05-09T00:53:06.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.676+0000 7fd06c787700 1 -- 172.21.15.5:0/1194130735 shutdown_connections 2024-05-09T00:53:06.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.676+0000 7fd06c787700 1 --2- 172.21.15.5:0/1194130735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd06410b2e0 0x7fd064117810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:06.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.676+0000 7fd06c787700 1 --2- 172.21.15.5:0/1194130735 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd064110970 0x7fd064110d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:06.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.676+0000 7fd06c787700 1 --2- 172.21.15.5:0/1194130735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd06410a990 0x7fd06410ada0 secure :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd05c00b0a0 tx=0x7fd05c00b3b0 comp rx=0 tx=0).stop 2024-05-09T00:53:06.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.676+0000 7fd06c787700 1 -- 172.21.15.5:0/1194130735 >> 172.21.15.5:0/1194130735 conn(0x7fd064077c80 msgr2=0x7fd0640780e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:53:06.679 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:53:06 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:53:06.679 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:53:06 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:53:06.679 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:53:06 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:53:06.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.681+0000 7fd06c787700 1 -- 172.21.15.5:0/1194130735 shutdown_connections 2024-05-09T00:53:06.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.681+0000 7fd06c787700 1 -- 172.21.15.5:0/1194130735 wait complete. 2024-05-09T00:53:06.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.683+0000 7fd06c787700 1 Processor -- start 2024-05-09T00:53:06.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.683+0000 7fd06c787700 1 -- start start 2024-05-09T00:53:06.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.686+0000 7fd06c787700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd06410b2e0 0x7fd0641a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:06.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.686+0000 7fd06c787700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd064110970 0x7fd0641a1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:06.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.686+0000 7fd06c787700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0641a5e90 0x7fd0641a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:06.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.686+0000 7fd06c787700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd06411ab90 con 0x7fd06410b2e0 2024-05-09T00:53:06.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.686+0000 7fd06c787700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd06411aa10 con 0x7fd064110970 2024-05-09T00:53:06.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.686+0000 7fd06c787700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd06411ad10 con 0x7fd0641a5e90 2024-05-09T00:53:06.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.688+0000 7fd069d22700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd064110970 0x7fd0641a1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:06.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.688+0000 7fd069d22700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd064110970 0x7fd0641a1cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:34546/0 (socket says 172.21.15.5:34546) 2024-05-09T00:53:06.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.688+0000 7fd069d22700 1 -- 172.21.15.5:0/4213833099 learned_addr learned my addr 172.21.15.5:0/4213833099 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:53:06.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.687+0000 7fd06a523700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd06410b2e0 0x7fd0641a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:06.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.689+0000 7fd06a523700 1 -- 172.21.15.5:0/4213833099 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0641a5e90 msgr2=0x7fd0641a62a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T00:53:06.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.689+0000 7fd06a523700 1 --2- 172.21.15.5:0/4213833099 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0641a5e90 0x7fd0641a62a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:06.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.689+0000 7fd06a523700 1 -- 172.21.15.5:0/4213833099 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd064110970 msgr2=0x7fd0641a1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:53:06.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.689+0000 7fd06a523700 1 --2- 172.21.15.5:0/4213833099 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd064110970 0x7fd0641a1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:06.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.689+0000 7fd06a523700 1 -- 172.21.15.5:0/4213833099 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0641a69e0 con 0x7fd06410b2e0 2024-05-09T00:53:06.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.689+0000 7fd069d22700 1 --2- 172.21.15.5:0/4213833099 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd064110970 0x7fd0641a1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:53:06.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.690+0000 7fd06a523700 1 --2- 172.21.15.5:0/4213833099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd06410b2e0 0x7fd0641a1720 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fd06000e9c0 tx=0x7fd06000ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:53:06.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.695+0000 7fd05b7fe700 1 -- 172.21.15.5:0/4213833099 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd06000cbd0 con 0x7fd06410b2e0 2024-05-09T00:53:06.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.695+0000 7fd05b7fe700 1 -- 172.21.15.5:0/4213833099 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd06000cd70 con 0x7fd06410b2e0 2024-05-09T00:53:06.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.695+0000 7fd05b7fe700 1 -- 172.21.15.5:0/4213833099 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd060018710 con 0x7fd06410b2e0 2024-05-09T00:53:06.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.696+0000 7fd06c787700 1 -- 172.21.15.5:0/4213833099 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0641a6cd0 con 0x7fd06410b2e0 2024-05-09T00:53:06.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.697+0000 7fd06c787700 1 -- 172.21.15.5:0/4213833099 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd06406f4e0 con 0x7fd06410b2e0 2024-05-09T00:53:06.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.708+0000 7fd07b19db80 1 Processor -- start 2024-05-09T00:53:06.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.718+0000 7fd07b19db80 1 -- start start 2024-05-09T00:53:06.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.719+0000 7fd07b19db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b33f829d00 0x55b33f932240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:06.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.719+0000 7fd07b19db80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b33f7af4a0 0x55b33f932780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:06.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.719+0000 7fd07b19db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b33f92c400 0x55b33f92c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:06.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.720+0000 7fd07b19db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b33f84f0a0 con 0x55b33f829d00 2024-05-09T00:53:06.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.720+0000 7fd07b19db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55b33f7f49a0 con 0x55b33f92c400 2024-05-09T00:53:06.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.720+0000 7fd07b19db80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b33f84b060 con 0x55b33f7af4a0 2024-05-09T00:53:06.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.721+0000 7fd06ad24700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b33f829d00 0x55b33f932240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:06.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.721+0000 7fd06ad24700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b33f829d00 0x55b33f932240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52038/0 (socket says 172.21.15.5:52038) 2024-05-09T00:53:06.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.721+0000 7fd06ad24700 1 -- 172.21.15.5:0/1372151703 learned_addr learned my addr 172.21.15.5:0/1372151703 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:53:06.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.723+0000 7fd06a523700 1 --2- 172.21.15.5:0/1372151703 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b33f7af4a0 0x55b33f932780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:06.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.726+0000 7fd069d22700 1 --2- 172.21.15.5:0/1372151703 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b33f92c400 0x55b33f92c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:06.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.728+0000 7fd05b7fe700 1 -- 172.21.15.5:0/4213833099 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fd060010430 con 0x7fd06410b2e0 2024-05-09T00:53:06.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.729+0000 7fd05b7fe700 1 --2- 172.21.15.5:0/4213833099 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fd0500b2a90 0x7fd0500b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:06.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.730+0000 7fd069d22700 1 -- 172.21.15.5:0/1372151703 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b33f7af4a0 msgr2=0x55b33f932780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:53:06.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.730+0000 7fd069d22700 1 --2- 172.21.15.5:0/1372151703 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b33f7af4a0 0x55b33f932780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:06.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.730+0000 7fd069d22700 1 -- 172.21.15.5:0/1372151703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b33f829d00 msgr2=0x55b33f932240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:53:06.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.730+0000 7fd069d22700 1 --2- 172.21.15.5:0/1372151703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b33f829d00 0x55b33f932240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:06.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.730+0000 7fd069d22700 1 -- 172.21.15.5:0/1372151703 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b33f7e5500 con 0x55b33f92c400 2024-05-09T00:53:06.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.730+0000 7fd05b7fe700 1 -- 172.21.15.5:0/4213833099 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fd0600e5a90 con 0x7fd06410b2e0 2024-05-09T00:53:06.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.731+0000 7fd069d22700 1 --2- 172.21.15.5:0/1372151703 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b33f92c400 0x55b33f92c810 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd05c009040 tx=0x7fd05c0093a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=131ee9e2d89f19e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:53:06.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.731+0000 7fd06a523700 1 --2- 172.21.15.5:0/1372151703 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b33f7af4a0 0x55b33f932780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:53:06.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.731+0000 7fd058ff9700 1 -- 172.21.15.5:0/1372151703 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05c021070 con 0x55b33f92c400 2024-05-09T00:53:06.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.731+0000 7fd058ff9700 1 -- 172.21.15.5:0/1372151703 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd05c00bcb0 con 0x55b33f92c400 2024-05-09T00:53:06.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.733+0000 7fd07b19db80 1 -- 172.21.15.5:0/1372151703 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b33f92c400 msgr2=0x55b33f92c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:53:06.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.733+0000 7fd07b19db80 1 --2- 172.21.15.5:0/1372151703 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b33f92c400 0x55b33f92c810 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd05c009040 tx=0x7fd05c0093a0 comp rx=0 tx=0).stop 2024-05-09T00:53:06.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.735+0000 7fd06ad24700 1 --2- 172.21.15.5:0/4213833099 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fd0500b2a90 0x7fd0500b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:06.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.735+0000 7fd07b19db80 1 -- 172.21.15.5:0/1372151703 shutdown_connections 2024-05-09T00:53:06.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.735+0000 7fd07b19db80 1 --2- 172.21.15.5:0/1372151703 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b33f92c400 0x55b33f92c810 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:06.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.735+0000 7fd07b19db80 1 --2- 172.21.15.5:0/1372151703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b33f829d00 0x55b33f932240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:06.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.735+0000 7fd07b19db80 1 --2- 172.21.15.5:0/1372151703 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b33f7af4a0 0x55b33f932780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:06.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.735+0000 7fd07b19db80 1 -- 172.21.15.5:0/1372151703 >> 172.21.15.5:0/1372151703 conn(0x55b33f7b2f70 msgr2=0x55b33f927220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:53:06.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.739+0000 7fd07b19db80 1 -- 172.21.15.5:0/1372151703 shutdown_connections 2024-05-09T00:53:06.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.744+0000 7fd06ad24700 1 --2- 172.21.15.5:0/4213833099 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fd0500b2a90 0x7fd0500b4fa0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd054009a30 tx=0x7fd054008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:53:06.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.744+0000 7fd07b19db80 1 -- 172.21.15.5:0/1372151703 wait complete. 2024-05-09T00:53:06.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.745+0000 7fd07b19db80 1 -- start start 2024-05-09T00:53:06.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.745+0000 7fd07b19db80 1 Processor -- start 2024-05-09T00:53:06.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.750+0000 7fd07b19db80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b33f7af4a0 0x55b33f935ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:06.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.750+0000 7fd07b19db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b33f829d00 0x55b33f936410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:06.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.750+0000 7fd07b19db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b33f92c400 0x55b33f936950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:06.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.750+0000 7fd07b19db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b33f92d100 con 0x55b33f829d00 2024-05-09T00:53:06.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.750+0000 7fd07b19db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55b33f7e9b20 con 0x55b33f92c400 2024-05-09T00:53:06.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.750+0000 7fd07b19db80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b33f938e60 con 0x55b33f7af4a0 2024-05-09T00:53:06.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.750+0000 7fd07b19db80 10 client.0 _ll_register_callbacks cb 0x55b33f6c06c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:53:06.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.754+0000 7fd069d22700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b33f829d00 0x55b33f936410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:06.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.754+0000 7fd069d22700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b33f829d00 0x55b33f936410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52050/0 (socket says 172.21.15.5:52050) 2024-05-09T00:53:06.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.754+0000 7fd069d22700 1 -- 172.21.15.5:0/2816851540 learned_addr learned my addr 172.21.15.5:0/2816851540 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:53:06.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.754+0000 7fd06ad24700 1 --2- 172.21.15.5:0/2816851540 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b33f92c400 0x55b33f936950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:06.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.754+0000 7fd06a523700 1 --2- 172.21.15.5:0/2816851540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b33f7af4a0 0x55b33f935ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:06.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.759+0000 7fd069d22700 1 -- 172.21.15.5:0/2816851540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b33f7af4a0 msgr2=0x55b33f935ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:53:06.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.759+0000 7fd069d22700 1 --2- 172.21.15.5:0/2816851540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b33f7af4a0 0x55b33f935ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:06.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.759+0000 7fd069d22700 1 -- 172.21.15.5:0/2816851540 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b33f92c400 msgr2=0x55b33f936950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:53:06.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.759+0000 7fd069d22700 1 --2- 172.21.15.5:0/2816851540 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b33f92c400 0x55b33f936950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:06.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.759+0000 7fd069d22700 1 -- 172.21.15.5:0/2816851540 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b33f6ca490 con 0x55b33f829d00 2024-05-09T00:53:06.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.760+0000 7fd069d22700 1 --2- 172.21.15.5:0/2816851540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b33f829d00 0x55b33f936410 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fd05c00d010 tx=0x7fd05c0061f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1618b245068fb41 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:53:06.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.760+0000 7fd04dffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:53:06.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.760+0000 7fd04dffb700 1 -- 172.21.15.5:0/2816851540 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05c021040 con 0x55b33f829d00 2024-05-09T00:53:06.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.760+0000 7fd07b19db80 1 -- 172.21.15.5:0/2816851540 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd05c00bcb0 con 0x55b33f829d00 2024-05-09T00:53:06.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.761+0000 7fd04dffb700 1 -- 172.21.15.5:0/2816851540 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fd05c02e450 con 0x55b33f829d00 2024-05-09T00:53:06.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.763+0000 7fd04dffb700 1 -- 172.21.15.5:0/2816851540 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fd05c034030 con 0x55b33f829d00 2024-05-09T00:53:06.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.765+0000 7fd07b19db80 10 client.16191 fetch_fsmap learned FSMap version 31 2024-05-09T00:53:06.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.765+0000 7fd07b19db80 1 -- 172.21.15.5:0/2816851540 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b33f954a20 con 0x55b33f829d00 2024-05-09T00:53:06.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.767+0000 7fd04dffb700 1 -- 172.21.15.5:0/2816851540 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd05c00bcb0 con 0x55b33f829d00 2024-05-09T00:53:06.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.768+0000 7fd04dffb700 1 -- 172.21.15.5:0/2816851540 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fd05c011070 con 0x55b33f829d00 2024-05-09T00:53:06.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.770+0000 7fd07b19db80 10 client.16191 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:53:06.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.770+0000 7fd07b19db80 10 client.16191 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-05-09T00:53:06.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.770+0000 7fd07b19db80 20 client.16191 populate_metadata read hostname 'smithi005' 2024-05-09T00:53:06.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.770+0000 7fd07b19db80 1 --2- 172.21.15.5:0/2816851540 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x55b33f956ef0 0x55b33f959310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:06.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.770+0000 7fd07b19db80 4 client.16191 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:53:06.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.770+0000 7fd07b19db80 1 -- 172.21.15.5:0/2816851540 --> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b33f85ca80 con 0x55b33f956ef0 2024-05-09T00:53:06.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.771+0000 7fd06a523700 1 --2- 172.21.15.5:0/2816851540 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x55b33f956ef0 0x55b33f959310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:06.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.774+0000 7fd06a523700 1 --2- 172.21.15.5:0/2816851540 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x55b33f956ef0 0x55b33f959310 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6bc7c21ee015b0c3 server_cookie=1a8dff3cd6a85b86 in_seq=0 out_seq=0 2024-05-09T00:53:06.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.774+0000 7fd04dffb700 10 client.16191 ms_handle_connect on v2:172.21.15.131:6838/2748669631 2024-05-09T00:53:06.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.776+0000 7fd04dffb700 1 -- 172.21.15.5:0/2816851540 <== mds.2 v2:172.21.15.131:6838/2748669631 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b33f85ca80 con 0x55b33f956ef0 2024-05-09T00:53:06.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.776+0000 7fd04dffb700 10 client.16191 handle_command_reply: tid=0 2024-05-09T00:53:06.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.782+0000 7fd07b19db80 1 client.16191 shutdown 2024-05-09T00:53:06.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.790+0000 7fd07b19db80 1 -- 172.21.15.5:0/2816851540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b33f829d00 msgr2=0x55b33f936410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:53:06.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.790+0000 7fd07b19db80 1 --2- 172.21.15.5:0/2816851540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b33f829d00 0x55b33f936410 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fd05c00d010 tx=0x7fd05c0061f0 comp rx=0 tx=0).stop 2024-05-09T00:53:06.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.791+0000 7fd07b19db80 1 -- 172.21.15.5:0/2816851540 shutdown_connections 2024-05-09T00:53:06.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.791+0000 7fd07b19db80 1 --2- 172.21.15.5:0/2816851540 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x55b33f956ef0 0x55b33f959310 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:06.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.791+0000 7fd07b19db80 1 --2- 172.21.15.5:0/2816851540 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b33f92c400 0x55b33f936950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:06.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.791+0000 7fd07b19db80 1 --2- 172.21.15.5:0/2816851540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b33f7af4a0 0x55b33f935ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:06.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.791+0000 7fd07b19db80 1 --2- 172.21.15.5:0/2816851540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b33f829d00 0x55b33f936410 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:06.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.791+0000 7fd07b19db80 1 -- 172.21.15.5:0/2816851540 >> 172.21.15.5:0/2816851540 conn(0x55b33f7b2f70 msgr2=0x55b33f920ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:53:06.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.791+0000 7fd04dffb700 0 client.16191 ms_handle_reset on v2:172.21.15.131:6838/2748669631 2024-05-09T00:53:06.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.798+0000 7fd07b19db80 1 -- 172.21.15.5:0/2816851540 shutdown_connections 2024-05-09T00:53:06.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.798+0000 7fd07b19db80 1 -- 172.21.15.5:0/2816851540 wait complete. 2024-05-09T00:53:06.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.799+0000 7fd07b19db80 20 client.16191 trim_cache size 0 max 16384 2024-05-09T00:53:06.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.838+0000 7fd07b19db80 1 Processor -- start 2024-05-09T00:53:06.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.838+0000 7fd07b19db80 1 -- start start 2024-05-09T00:53:06.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.838+0000 7fd07b19db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b33f7af4a0 0x55b33f930340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:06.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.838+0000 7fd07b19db80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b33f829d00 0x55b33f936790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:06.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.838+0000 7fd07b19db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b33f92c400 0x55b33f934ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:06.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.838+0000 7fd07b19db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd06407e210 con 0x55b33f7af4a0 2024-05-09T00:53:06.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.838+0000 7fd07b19db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd06407fd50 con 0x55b33f92c400 2024-05-09T00:53:06.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.838+0000 7fd07b19db80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd06407e8e0 con 0x55b33f829d00 2024-05-09T00:53:06.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.839+0000 7fd069d22700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b33f829d00 0x55b33f936790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:06.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.839+0000 7fd06a523700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b33f7af4a0 0x55b33f930340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:06.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.839+0000 7fd06a523700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b33f7af4a0 0x55b33f930340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52060/0 (socket says 172.21.15.5:52060) 2024-05-09T00:53:06.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.839+0000 7fd06a523700 1 -- 172.21.15.5:0/2412044396 learned_addr learned my addr 172.21.15.5:0/2412044396 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:53:06.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.839+0000 7fd06ad24700 1 --2- 172.21.15.5:0/2412044396 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b33f92c400 0x55b33f934ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:06.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.840+0000 7fd06a523700 1 -- 172.21.15.5:0/2412044396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b33f829d00 msgr2=0x55b33f936790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:53:06.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.840+0000 7fd06a523700 1 --2- 172.21.15.5:0/2412044396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b33f829d00 0x55b33f936790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:06.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.840+0000 7fd06a523700 1 -- 172.21.15.5:0/2412044396 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b33f92c400 msgr2=0x55b33f934ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:53:06.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.840+0000 7fd06a523700 1 --2- 172.21.15.5:0/2412044396 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b33f92c400 0x55b33f934ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:06.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.840+0000 7fd06a523700 1 -- 172.21.15.5:0/2412044396 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b33f938770 con 0x55b33f7af4a0 2024-05-09T00:53:06.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.841+0000 7fd06a523700 1 --2- 172.21.15.5:0/2412044396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b33f7af4a0 0x55b33f930340 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fd0600e3500 tx=0x7fd0600e33d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dc2a93636dd3407 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:53:06.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.845+0000 7fd04ffff700 1 -- 172.21.15.5:0/2412044396 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0600ad020 con 0x55b33f7af4a0 2024-05-09T00:53:06.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.845+0000 7fd04ffff700 1 -- 172.21.15.5:0/2412044396 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0600c7070 con 0x55b33f7af4a0 2024-05-09T00:53:06.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.845+0000 7fd04ffff700 1 -- 172.21.15.5:0/2412044396 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0600b10d0 con 0x55b33f7af4a0 2024-05-09T00:53:06.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.846+0000 7fd07b19db80 1 -- 172.21.15.5:0/2412044396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b33f7af4a0 msgr2=0x55b33f930340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:53:06.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.847+0000 7fd07b19db80 1 --2- 172.21.15.5:0/2412044396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b33f7af4a0 0x55b33f930340 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fd0600e3500 tx=0x7fd0600e33d0 comp rx=0 tx=0).stop 2024-05-09T00:53:06.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.847+0000 7fd07b19db80 1 -- 172.21.15.5:0/2412044396 shutdown_connections 2024-05-09T00:53:06.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.847+0000 7fd07b19db80 1 --2- 172.21.15.5:0/2412044396 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b33f92c400 0x55b33f934ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:06.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.847+0000 7fd07b19db80 1 --2- 172.21.15.5:0/2412044396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b33f7af4a0 0x55b33f930340 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:06.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.847+0000 7fd07b19db80 1 --2- 172.21.15.5:0/2412044396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b33f829d00 0x55b33f936790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:06.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.847+0000 7fd07b19db80 1 -- 172.21.15.5:0/2412044396 >> 172.21.15.5:0/2412044396 conn(0x55b33f7b2f70 msgr2=0x55b33f9347c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:53:06.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.849+0000 7fd07b19db80 1 -- 172.21.15.5:0/2412044396 shutdown_connections 2024-05-09T00:53:06.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.849+0000 7fd07b19db80 1 -- 172.21.15.5:0/2412044396 wait complete. 2024-05-09T00:53:06.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.850+0000 7fd07b19db80 1 -- start start 2024-05-09T00:53:06.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.850+0000 7fd07b19db80 1 Processor -- start 2024-05-09T00:53:06.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.851+0000 7fd07b19db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b33f7af4a0 0x55b33f95c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:06.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.851+0000 7fd07b19db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b33f829d00 0x55b33f959d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:06.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:53:06 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:53:06.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:53:06 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:53:06.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:53:06 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:53:06.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.852+0000 7fd07b19db80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b33f92c400 0x55b33f95a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:06.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.852+0000 7fd07b19db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd06407f680 con 0x55b33f829d00 2024-05-09T00:53:06.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.852+0000 7fd07b19db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd06407efb0 con 0x55b33f7af4a0 2024-05-09T00:53:06.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.852+0000 7fd07b19db80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd064074ab0 con 0x55b33f92c400 2024-05-09T00:53:06.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.852+0000 7fd07b19db80 10 client.0 _ll_register_callbacks cb 0x55b33f6c06c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:53:06.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.852+0000 7fd06a523700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b33f7af4a0 0x55b33f95c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:06.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.852+0000 7fd06ad24700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b33f92c400 0x55b33f95a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:06.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.852+0000 7fd06ad24700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b33f92c400 0x55b33f95a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:48726/0 (socket says 172.21.15.5:48726) 2024-05-09T00:53:06.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.852+0000 7fd06ad24700 1 -- 172.21.15.5:0/1977919315 learned_addr learned my addr 172.21.15.5:0/1977919315 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:53:06.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.852+0000 7fd069d22700 1 --2- 172.21.15.5:0/1977919315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b33f829d00 0x55b33f959d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:06.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.853+0000 7fd069d22700 1 -- 172.21.15.5:0/1977919315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b33f92c400 msgr2=0x55b33f95a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:53:06.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.853+0000 7fd069d22700 1 --2- 172.21.15.5:0/1977919315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b33f92c400 0x55b33f95a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:06.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.853+0000 7fd069d22700 1 -- 172.21.15.5:0/1977919315 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b33f7af4a0 msgr2=0x55b33f95c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:53:06.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.853+0000 7fd069d22700 1 --2- 172.21.15.5:0/1977919315 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b33f7af4a0 0x55b33f95c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:06.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.853+0000 7fd069d22700 1 -- 172.21.15.5:0/1977919315 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b33f95ab20 con 0x55b33f829d00 2024-05-09T00:53:06.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.854+0000 7fd069d22700 1 --2- 172.21.15.5:0/1977919315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b33f829d00 0x55b33f959d50 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fd05c005bd0 tx=0x7fd05c00bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e03538fc0baf331 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:53:06.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.854+0000 7fd04dffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:53:06.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.854+0000 7fd04dffb700 1 -- 172.21.15.5:0/1977919315 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05c03d070 con 0x55b33f829d00 2024-05-09T00:53:06.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.854+0000 7fd04dffb700 1 -- 172.21.15.5:0/1977919315 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fd05c025540 con 0x55b33f829d00 2024-05-09T00:53:06.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.855+0000 7fd07b19db80 1 -- 172.21.15.5:0/1977919315 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0600c7070 con 0x55b33f829d00 2024-05-09T00:53:06.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.855+0000 7fd04dffb700 1 -- 172.21.15.5:0/1977919315 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fd05c021040 con 0x55b33f829d00 2024-05-09T00:53:06.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.855+0000 7fd07b19db80 10 client.16197 fetch_fsmap learned FSMap version 31 2024-05-09T00:53:06.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.855+0000 7fd07b19db80 1 -- 172.21.15.5:0/1977919315 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b33f9c1460 con 0x55b33f829d00 2024-05-09T00:53:06.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.856+0000 7fd04dffb700 1 -- 172.21.15.5:0/1977919315 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0600c7070 con 0x55b33f829d00 2024-05-09T00:53:06.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.856+0000 7fd04dffb700 1 -- 172.21.15.5:0/1977919315 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fd05c040020 con 0x55b33f829d00 2024-05-09T00:53:06.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.856+0000 7fd07b19db80 10 client.16197 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:53:06.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.856+0000 7fd07b19db80 10 client.16197 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-05-09T00:53:06.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.856+0000 7fd07b19db80 20 client.16197 populate_metadata read hostname 'smithi005' 2024-05-09T00:53:06.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.856+0000 7fd07b19db80 1 --2- 172.21.15.5:0/1977919315 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x55b33f9c1a60 0x55b33f9c3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:06.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.856+0000 7fd07b19db80 4 client.16197 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:53:06.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.856+0000 7fd07b19db80 1 -- 172.21.15.5:0/1977919315 --> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b33f856b20 con 0x55b33f9c1a60 2024-05-09T00:53:06.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.857+0000 7fd06a523700 1 --2- 172.21.15.5:0/1977919315 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x55b33f9c1a60 0x55b33f9c3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:06.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.859+0000 7fd06a523700 1 --2- 172.21.15.5:0/1977919315 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x55b33f9c1a60 0x55b33f9c3e80 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f1ef70b275e5b566 server_cookie=82e1c76a5e51346f in_seq=0 out_seq=0 2024-05-09T00:53:06.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.859+0000 7fd04dffb700 10 client.16197 ms_handle_connect on v2:172.21.15.131:6838/2748669631 2024-05-09T00:53:06.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.860+0000 7fd04dffb700 1 -- 172.21.15.5:0/1977919315 <== mds.2 v2:172.21.15.131:6838/2748669631 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b33f856b20 con 0x55b33f9c1a60 2024-05-09T00:53:06.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.860+0000 7fd04dffb700 10 client.16197 handle_command_reply: tid=0 2024-05-09T00:53:06.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.860+0000 7fd07b19db80 1 client.16197 shutdown 2024-05-09T00:53:06.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.874+0000 7fd07b19db80 1 -- 172.21.15.5:0/1977919315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b33f829d00 msgr2=0x55b33f959d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:53:06.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.874+0000 7fd07b19db80 1 --2- 172.21.15.5:0/1977919315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b33f829d00 0x55b33f959d50 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fd05c005bd0 tx=0x7fd05c00bee0 comp rx=0 tx=0).stop 2024-05-09T00:53:06.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.874+0000 7fd07b19db80 1 -- 172.21.15.5:0/1977919315 shutdown_connections 2024-05-09T00:53:06.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.874+0000 7fd07b19db80 1 --2- 172.21.15.5:0/1977919315 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x55b33f9c1a60 0x55b33f9c3e80 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:06.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.874+0000 7fd07b19db80 1 --2- 172.21.15.5:0/1977919315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b33f92c400 0x55b33f95a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:06.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.874+0000 7fd07b19db80 1 --2- 172.21.15.5:0/1977919315 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b33f7af4a0 0x55b33f95c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:06.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.874+0000 7fd07b19db80 1 --2- 172.21.15.5:0/1977919315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b33f829d00 0x55b33f959d50 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:06.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.874+0000 7fd07b19db80 1 -- 172.21.15.5:0/1977919315 >> 172.21.15.5:0/1977919315 conn(0x55b33f7b2f70 msgr2=0x55b33f931750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:53:06.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.874+0000 7fd04dffb700 0 client.16197 ms_handle_reset on v2:172.21.15.131:6838/2748669631 2024-05-09T00:53:06.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.875+0000 7fd07b19db80 1 -- 172.21.15.5:0/1977919315 shutdown_connections 2024-05-09T00:53:06.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.875+0000 7fd07b19db80 1 -- 172.21.15.5:0/1977919315 wait complete. 2024-05-09T00:53:06.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.875+0000 7fd07b19db80 20 client.16197 trim_cache size 0 max 16384 2024-05-09T00:53:06.878 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-09T00:53:06.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.879+0000 7fd06c787700 1 -- 172.21.15.5:0/4213833099 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fd0500b2a90 msgr2=0x7fd0500b4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:53:06.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.879+0000 7fd06c787700 1 --2- 172.21.15.5:0/4213833099 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fd0500b2a90 0x7fd0500b4fa0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd054009a30 tx=0x7fd054008040 comp rx=0 tx=0).stop 2024-05-09T00:53:06.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.879+0000 7fd06c787700 1 -- 172.21.15.5:0/4213833099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd06410b2e0 msgr2=0x7fd0641a1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:53:06.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.879+0000 7fd06c787700 1 --2- 172.21.15.5:0/4213833099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd06410b2e0 0x7fd0641a1720 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fd06000e9c0 tx=0x7fd06000ed80 comp rx=0 tx=0).stop 2024-05-09T00:53:06.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.879+0000 7fd06c787700 1 -- 172.21.15.5:0/4213833099 shutdown_connections 2024-05-09T00:53:06.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.879+0000 7fd06c787700 1 --2- 172.21.15.5:0/4213833099 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fd0500b2a90 0x7fd0500b4fa0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:06.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.879+0000 7fd06c787700 1 --2- 172.21.15.5:0/4213833099 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0641a5e90 0x7fd0641a62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:06.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.879+0000 7fd06c787700 1 --2- 172.21.15.5:0/4213833099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd06410b2e0 0x7fd0641a1720 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:06.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.879+0000 7fd06c787700 1 --2- 172.21.15.5:0/4213833099 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd064110970 0x7fd0641a1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:06.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.879+0000 7fd06c787700 1 -- 172.21.15.5:0/4213833099 >> 172.21.15.5:0/4213833099 conn(0x7fd064077c80 msgr2=0x7fd06407b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:53:06.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.879+0000 7fd06c787700 1 -- 172.21.15.5:0/4213833099 shutdown_connections 2024-05-09T00:53:06.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:06.879+0000 7fd06c787700 1 -- 172.21.15.5:0/4213833099 wait complete. 2024-05-09T00:53:06.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:53:06 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:53:06.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:53:06 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:53:06.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:53:06 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:53:07.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:53:07 smithi005 ceph-mon[30365]: pgmap v574: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 923 MiB/s wr, 2.33k op/s 2024-05-09T00:53:07.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:53:07 smithi131 ceph-mon[34547]: pgmap v574: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 923 MiB/s wr, 2.33k op/s 2024-05-09T00:53:07.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:53:07 smithi157 ceph-mon[34294]: pgmap v574: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 923 MiB/s wr, 2.33k op/s 2024-05-09T00:53:09.291 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T00:53:09.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:53:09 smithi005 ceph-mon[30365]: pgmap v575: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 790 MiB/s wr, 2.12k op/s 2024-05-09T00:53:09.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:53:09 smithi131 ceph-mon[34547]: pgmap v575: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 790 MiB/s wr, 2.12k op/s 2024-05-09T00:53:09.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:53:09 smithi157 ceph-mon[34294]: pgmap v575: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 790 MiB/s wr, 2.12k op/s 2024-05-09T00:53:10.054 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:53:11.795 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:53:11 smithi005 ceph-mon[30365]: pgmap v576: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 939 MiB/s wr, 2.30k op/s 2024-05-09T00:53:11.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:53:11 smithi131 ceph-mon[34547]: pgmap v576: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 939 MiB/s wr, 2.30k op/s 2024-05-09T00:53:11.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:53:11 smithi157 ceph-mon[34294]: pgmap v576: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 939 MiB/s wr, 2.30k op/s 2024-05-09T00:53:13.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:53:13 smithi005 ceph-mon[30365]: pgmap v577: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 819 MiB/s wr, 1.97k op/s 2024-05-09T00:53:13.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:53:13 smithi131 ceph-mon[34547]: pgmap v577: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 819 MiB/s wr, 1.97k op/s 2024-05-09T00:53:13.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:53:13 smithi157 ceph-mon[34294]: pgmap v577: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 819 MiB/s wr, 1.97k op/s 2024-05-09T00:53:15.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.467+0000 7f4efbc6c700 1 -- 172.21.15.5:0/3997954343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ef4103ba0 msgr2=0x7f4ef41080d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:53:15.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.467+0000 7f4efbc6c700 1 --2- 172.21.15.5:0/3997954343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ef4103ba0 0x7f4ef41080d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f4ef00098f0 tx=0x7f4ef0009c00 comp rx=0 tx=0).stop 2024-05-09T00:53:15.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.467+0000 7f4efbc6c700 1 -- 172.21.15.5:0/3997954343 shutdown_connections 2024-05-09T00:53:15.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.467+0000 7f4efbc6c700 1 --2- 172.21.15.5:0/3997954343 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ef41086a0 0x7f4ef4108ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:15.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.467+0000 7f4efbc6c700 1 --2- 172.21.15.5:0/3997954343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ef4103250 0x7f4ef4103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:15.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.467+0000 7f4efbc6c700 1 --2- 172.21.15.5:0/3997954343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ef4103ba0 0x7f4ef41080d0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:15.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.467+0000 7f4efbc6c700 1 -- 172.21.15.5:0/3997954343 >> 172.21.15.5:0/3997954343 conn(0x7f4ef40fedf0 msgr2=0x7f4ef4101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:53:15.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.467+0000 7f4efbc6c700 1 -- 172.21.15.5:0/3997954343 shutdown_connections 2024-05-09T00:53:15.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.468+0000 7f4efbc6c700 1 -- 172.21.15.5:0/3997954343 wait complete. 2024-05-09T00:53:15.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.469+0000 7f4efbc6c700 1 Processor -- start 2024-05-09T00:53:15.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.470+0000 7f4efbc6c700 1 -- start start 2024-05-09T00:53:15.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.470+0000 7f4efbc6c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ef4103250 0x7f4ef41a14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:15.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.470+0000 7f4efbc6c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ef4103ba0 0x7f4ef41a1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:15.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.470+0000 7f4efbc6c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ef41086a0 0x7f4ef41a5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:15.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.470+0000 7f4efbc6c700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef411aaf0 con 0x7f4ef4103250 2024-05-09T00:53:15.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.470+0000 7f4efbc6c700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef411a970 con 0x7f4ef4103ba0 2024-05-09T00:53:15.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.470+0000 7f4efbc6c700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef411ac70 con 0x7f4ef41086a0 2024-05-09T00:53:15.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.471+0000 7f4ef9a08700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ef4103250 0x7f4ef41a14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:15.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.471+0000 7f4ef9a08700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ef4103250 0x7f4ef41a14e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57592/0 (socket says 172.21.15.5:57592) 2024-05-09T00:53:15.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.471+0000 7f4ef9a08700 1 -- 172.21.15.5:0/2998496031 learned_addr learned my addr 172.21.15.5:0/2998496031 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:53:15.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.471+0000 7f4ef9a08700 1 -- 172.21.15.5:0/2998496031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ef41086a0 msgr2=0x7f4ef41a5d10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T00:53:15.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.471+0000 7f4ef9a08700 1 --2- 172.21.15.5:0/2998496031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ef41086a0 0x7f4ef41a5d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:15.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.471+0000 7f4ef9a08700 1 -- 172.21.15.5:0/2998496031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ef4103ba0 msgr2=0x7f4ef41a1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:53:15.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.471+0000 7f4ef9a08700 1 --2- 172.21.15.5:0/2998496031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ef4103ba0 0x7f4ef41a1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:15.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.471+0000 7f4ef9a08700 1 -- 172.21.15.5:0/2998496031 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ef41a63b0 con 0x7f4ef4103250 2024-05-09T00:53:15.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.472+0000 7f4ef9a08700 1 --2- 172.21.15.5:0/2998496031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ef4103250 0x7f4ef41a14e0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f4ee400b500 tx=0x7f4ee400b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:53:15.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.473+0000 7f4eeaffd700 1 -- 172.21.15.5:0/2998496031 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee4015020 con 0x7f4ef4103250 2024-05-09T00:53:15.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.473+0000 7f4efbc6c700 1 -- 172.21.15.5:0/2998496031 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ef41a66a0 con 0x7f4ef4103250 2024-05-09T00:53:15.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.473+0000 7f4efbc6c700 1 -- 172.21.15.5:0/2998496031 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ef41adfc0 con 0x7f4ef4103250 2024-05-09T00:53:15.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.474+0000 7f4eeaffd700 1 -- 172.21.15.5:0/2998496031 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ee4011ce0 con 0x7f4ef4103250 2024-05-09T00:53:15.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.474+0000 7f4eeaffd700 1 -- 172.21.15.5:0/2998496031 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee4010d60 con 0x7f4ef4103250 2024-05-09T00:53:15.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.486+0000 7f4eeaffd700 1 -- 172.21.15.5:0/2998496031 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f4ee4018020 con 0x7f4ef4103250 2024-05-09T00:53:15.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.488+0000 7f4eeaffd700 1 --2- 172.21.15.5:0/2998496031 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f4ee00b2a50 0x7f4ee00b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:15.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.489+0000 7f4eeaffd700 1 -- 172.21.15.5:0/2998496031 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f4ee40e2950 con 0x7f4ef4103250 2024-05-09T00:53:15.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.490+0000 7f4ef9207700 1 --2- 172.21.15.5:0/2998496031 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f4ee00b2a50 0x7f4ee00b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:15.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.491+0000 7f4f0a682b80 1 Processor -- start 2024-05-09T00:53:15.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.492+0000 7f4ef9207700 1 --2- 172.21.15.5:0/2998496031 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f4ee00b2a50 0x7f4ee00b4f60 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0009810 tx=0x7f4ef0005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:53:15.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.492+0000 7f4f0a682b80 1 -- start start 2024-05-09T00:53:15.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.492+0000 7f4f0a682b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d00e0a1d00 0x55d00e1aa240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:15.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.492+0000 7f4f0a682b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d00e0274a0 0x55d00e1aa780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:15.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.492+0000 7f4f0a682b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d00e1a4400 0x55d00e1a4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:15.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.492+0000 7f4f0a682b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d00e0c70a0 con 0x55d00e0a1d00 2024-05-09T00:53:15.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.492+0000 7f4f0a682b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55d00e06c9a0 con 0x55d00e0274a0 2024-05-09T00:53:15.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.492+0000 7f4f0a682b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d00e0c3060 con 0x55d00e1a4400 2024-05-09T00:53:15.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.493+0000 7f4ef9a08700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d00e0a1d00 0x55d00e1aa240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:15.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.493+0000 7f4ef9a08700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d00e0a1d00 0x55d00e1aa240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57606/0 (socket says 172.21.15.5:57606) 2024-05-09T00:53:15.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.494+0000 7f4ef9a08700 1 -- 172.21.15.5:0/1173876864 learned_addr learned my addr 172.21.15.5:0/1173876864 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:53:15.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.494+0000 7f4ef9207700 1 --2- 172.21.15.5:0/1173876864 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d00e0274a0 0x55d00e1aa780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:15.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.495+0000 7f4ef9207700 1 -- 172.21.15.5:0/1173876864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d00e1a4400 msgr2=0x55d00e1a4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:53:15.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.495+0000 7f4efa209700 1 --2- 172.21.15.5:0/1173876864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d00e1a4400 0x55d00e1a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:15.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.496+0000 7f4ef9207700 1 --2- 172.21.15.5:0/1173876864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d00e1a4400 0x55d00e1a4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:15.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.496+0000 7f4ef9207700 1 -- 172.21.15.5:0/1173876864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d00e0a1d00 msgr2=0x55d00e1aa240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:53:15.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.496+0000 7f4ef9207700 1 --2- 172.21.15.5:0/1173876864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d00e0a1d00 0x55d00e1aa240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:15.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.496+0000 7f4ef9207700 1 -- 172.21.15.5:0/1173876864 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d00e05d500 con 0x55d00e0274a0 2024-05-09T00:53:15.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.496+0000 7f4ef9207700 1 --2- 172.21.15.5:0/1173876864 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d00e0274a0 0x55d00e1aa780 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4ef000b2e0 tx=0x7f4ef001b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=538e488d7642bdb5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:53:15.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.497+0000 7f4edffff700 1 -- 172.21.15.5:0/1173876864 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef000b810 con 0x55d00e0274a0 2024-05-09T00:53:15.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.497+0000 7f4edffff700 1 -- 172.21.15.5:0/1173876864 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ef0027070 con 0x55d00e0274a0 2024-05-09T00:53:15.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.497+0000 7f4edffff700 1 -- 172.21.15.5:0/1173876864 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef002c470 con 0x55d00e0274a0 2024-05-09T00:53:15.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.499+0000 7f4f0a682b80 1 -- 172.21.15.5:0/1173876864 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d00e0274a0 msgr2=0x55d00e1aa780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:53:15.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.499+0000 7f4f0a682b80 1 --2- 172.21.15.5:0/1173876864 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d00e0274a0 0x55d00e1aa780 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4ef000b2e0 tx=0x7f4ef001b000 comp rx=0 tx=0).stop 2024-05-09T00:53:15.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.500+0000 7f4f0a682b80 1 -- 172.21.15.5:0/1173876864 shutdown_connections 2024-05-09T00:53:15.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.500+0000 7f4f0a682b80 1 --2- 172.21.15.5:0/1173876864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d00e1a4400 0x55d00e1a4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:15.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.500+0000 7f4f0a682b80 1 --2- 172.21.15.5:0/1173876864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d00e0a1d00 0x55d00e1aa240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:15.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.500+0000 7f4f0a682b80 1 --2- 172.21.15.5:0/1173876864 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d00e0274a0 0x55d00e1aa780 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:15.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.500+0000 7f4f0a682b80 1 -- 172.21.15.5:0/1173876864 >> 172.21.15.5:0/1173876864 conn(0x55d00e02af70 msgr2=0x55d00e19f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:53:15.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.501+0000 7f4f0a682b80 1 -- 172.21.15.5:0/1173876864 shutdown_connections 2024-05-09T00:53:15.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.501+0000 7f4f0a682b80 1 -- 172.21.15.5:0/1173876864 wait complete. 2024-05-09T00:53:15.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.502+0000 7f4f0a682b80 1 -- start start 2024-05-09T00:53:15.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.502+0000 7f4f0a682b80 1 Processor -- start 2024-05-09T00:53:15.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.504+0000 7f4f0a682b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d00e0274a0 0x55d00e1ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:15.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.505+0000 7f4f0a682b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d00e0a1d00 0x55d00e1ae540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:15.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.505+0000 7f4f0a682b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d00e1a4400 0x55d00e1aea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:15.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.505+0000 7f4f0a682b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d00e1a5100 con 0x55d00e1a4400 2024-05-09T00:53:15.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.505+0000 7f4efa209700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d00e1a4400 0x55d00e1aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:15.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.505+0000 7f4efa209700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d00e1a4400 0x55d00e1aea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57618/0 (socket says 172.21.15.5:57618) 2024-05-09T00:53:15.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.505+0000 7f4efa209700 1 -- 172.21.15.5:0/1087449800 learned_addr learned my addr 172.21.15.5:0/1087449800 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:53:15.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.505+0000 7f4f0a682b80 1 -- 172.21.15.5:0/1087449800 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55d00e061b20 con 0x55d00e0a1d00 2024-05-09T00:53:15.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.505+0000 7f4f0a682b80 1 -- 172.21.15.5:0/1087449800 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d00e1ba230 con 0x55d00e0274a0 2024-05-09T00:53:15.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.506+0000 7f4ef9a08700 1 --2- 172.21.15.5:0/1087449800 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d00e0274a0 0x55d00e1ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:15.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.506+0000 7f4ef9207700 1 --2- 172.21.15.5:0/1087449800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d00e0a1d00 0x55d00e1ae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:15.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.506+0000 7f4f0a682b80 10 client.0 _ll_register_callbacks cb 0x55d00df386c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:53:15.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.507+0000 7f4efa209700 1 -- 172.21.15.5:0/1087449800 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d00e0274a0 msgr2=0x55d00e1ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:53:15.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.507+0000 7f4efa209700 1 --2- 172.21.15.5:0/1087449800 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d00e0274a0 0x55d00e1ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:15.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.507+0000 7f4efa209700 1 -- 172.21.15.5:0/1087449800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d00e0a1d00 msgr2=0x55d00e1ae540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:53:15.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.507+0000 7f4efa209700 1 --2- 172.21.15.5:0/1087449800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d00e0a1d00 0x55d00e1ae540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:15.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.507+0000 7f4efa209700 1 -- 172.21.15.5:0/1087449800 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d00df42490 con 0x55d00e1a4400 2024-05-09T00:53:15.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.507+0000 7f4ef9207700 1 --2- 172.21.15.5:0/1087449800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d00e0a1d00 0x55d00e1ae540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:53:15.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.508+0000 7f4efa209700 1 --2- 172.21.15.5:0/1087449800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d00e1a4400 0x55d00e1aea80 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f4eec007870 tx=0x7f4eec0095d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc2593aa9d6ffe9d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:53:15.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.508+0000 7f4edd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:53:15.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.508+0000 7f4edd7fa700 1 -- 172.21.15.5:0/1087449800 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eec003a00 con 0x55d00e1a4400 2024-05-09T00:53:15.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.508+0000 7f4f0a682b80 1 -- 172.21.15.5:0/1087449800 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ef0027070 con 0x55d00e1a4400 2024-05-09T00:53:15.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.509+0000 7f4edd7fa700 1 -- 172.21.15.5:0/1087449800 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f4eec003ba0 con 0x55d00e1a4400 2024-05-09T00:53:15.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.509+0000 7f4edd7fa700 1 -- 172.21.15.5:0/1087449800 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f4eec003da0 con 0x55d00e1a4400 2024-05-09T00:53:15.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.509+0000 7f4f0a682b80 10 client.16209 fetch_fsmap learned FSMap version 31 2024-05-09T00:53:15.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.509+0000 7f4f0a682b80 1 -- 172.21.15.5:0/1087449800 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d00e1bb980 con 0x55d00e1a4400 2024-05-09T00:53:15.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.510+0000 7f4edd7fa700 1 -- 172.21.15.5:0/1087449800 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ef0027070 con 0x55d00e1a4400 2024-05-09T00:53:15.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.510+0000 7f4edd7fa700 1 -- 172.21.15.5:0/1087449800 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f4eec013800 con 0x55d00e1a4400 2024-05-09T00:53:15.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.510+0000 7f4f0a682b80 10 client.16209 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:53:15.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.510+0000 7f4f0a682b80 10 client.16209 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-05-09T00:53:15.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.510+0000 7f4f0a682b80 20 client.16209 populate_metadata read hostname 'smithi005' 2024-05-09T00:53:15.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.510+0000 7f4f0a682b80 1 --2- 172.21.15.5:0/1087449800 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x55d00e1be060 0x55d00e1c04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:15.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.510+0000 7f4f0a682b80 4 client.16209 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:53:15.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.510+0000 7f4f0a682b80 1 -- 172.21.15.5:0/1087449800 --> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d00e0d4a80 con 0x55d00e1be060 2024-05-09T00:53:15.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.511+0000 7f4ef9a08700 1 --2- 172.21.15.5:0/1087449800 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x55d00e1be060 0x55d00e1c04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:15.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.516+0000 7f4ef9a08700 1 --2- 172.21.15.5:0/1087449800 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x55d00e1be060 0x55d00e1c04b0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=daff38246784592d server_cookie=3d7d5748e9e65879 in_seq=0 out_seq=0 2024-05-09T00:53:15.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.516+0000 7f4edd7fa700 10 client.16209 ms_handle_connect on v2:172.21.15.157:6836/744863009 2024-05-09T00:53:15.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.518+0000 7f4edd7fa700 1 -- 172.21.15.5:0/1087449800 <== mds.1 v2:172.21.15.157:6836/744863009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d00e0d4a80 con 0x55d00e1be060 2024-05-09T00:53:15.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.518+0000 7f4edd7fa700 10 client.16209 handle_command_reply: tid=0 2024-05-09T00:53:15.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.518+0000 7f4f0a682b80 1 client.16209 shutdown 2024-05-09T00:53:15.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.518+0000 7f4f0a682b80 1 -- 172.21.15.5:0/1087449800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d00e1a4400 msgr2=0x55d00e1aea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:53:15.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.518+0000 7f4f0a682b80 1 --2- 172.21.15.5:0/1087449800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d00e1a4400 0x55d00e1aea80 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f4eec007870 tx=0x7f4eec0095d0 comp rx=0 tx=0).stop 2024-05-09T00:53:15.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.519+0000 7f4f0a682b80 1 -- 172.21.15.5:0/1087449800 shutdown_connections 2024-05-09T00:53:15.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.519+0000 7f4f0a682b80 1 --2- 172.21.15.5:0/1087449800 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x55d00e1be060 0x55d00e1c04b0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:15.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.519+0000 7f4f0a682b80 1 --2- 172.21.15.5:0/1087449800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d00e1a4400 0x55d00e1aea80 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:15.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.519+0000 7f4f0a682b80 1 --2- 172.21.15.5:0/1087449800 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d00e0274a0 0x55d00e1ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:15.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.519+0000 7f4f0a682b80 1 --2- 172.21.15.5:0/1087449800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d00e0a1d00 0x55d00e1ae540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:15.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.519+0000 7f4f0a682b80 1 -- 172.21.15.5:0/1087449800 >> 172.21.15.5:0/1087449800 conn(0x55d00e02af70 msgr2=0x55d00e198ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:53:15.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.519+0000 7f4edd7fa700 0 client.16209 ms_handle_reset on v2:172.21.15.157:6836/744863009 2024-05-09T00:53:15.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.520+0000 7f4f0a682b80 1 -- 172.21.15.5:0/1087449800 shutdown_connections 2024-05-09T00:53:15.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.520+0000 7f4f0a682b80 1 -- 172.21.15.5:0/1087449800 wait complete. 2024-05-09T00:53:15.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.520+0000 7f4f0a682b80 20 client.16209 trim_cache size 0 max 16384 2024-05-09T00:53:15.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.558+0000 7f4f0a682b80 1 Processor -- start 2024-05-09T00:53:15.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.558+0000 7f4f0a682b80 1 -- start start 2024-05-09T00:53:15.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.559+0000 7f4f0a682b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d00e0274a0 0x55d00e1ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:15.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.559+0000 7f4f0a682b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d00e0a1d00 0x55d00e1a9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:15.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.559+0000 7f4f0a682b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d00e1a4400 0x55d00e1a96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:15.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.559+0000 7f4f0a682b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed800d420 con 0x55d00e1a4400 2024-05-09T00:53:15.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.559+0000 7f4f0a682b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed800ef60 con 0x55d00e0a1d00 2024-05-09T00:53:15.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.560+0000 7f4f0a682b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed800daf0 con 0x55d00e0274a0 2024-05-09T00:53:15.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.560+0000 7f4ef9a08700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d00e0274a0 0x55d00e1ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:15.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.560+0000 7f4ef9a08700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d00e0274a0 0x55d00e1ae8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:55802/0 (socket says 172.21.15.5:55802) 2024-05-09T00:53:15.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.560+0000 7f4ef9a08700 1 -- 172.21.15.5:0/3960908722 learned_addr learned my addr 172.21.15.5:0/3960908722 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:53:15.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.562+0000 7f4ef9a08700 1 -- 172.21.15.5:0/3960908722 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d00e0a1d00 msgr2=0x55d00e1a9160 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:53:15.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.562+0000 7f4ef9a08700 1 --2- 172.21.15.5:0/3960908722 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d00e0a1d00 0x55d00e1a9160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:15.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.562+0000 7f4ef9a08700 1 -- 172.21.15.5:0/3960908722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d00e1a4400 msgr2=0x55d00e1a96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:53:15.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.562+0000 7f4ef9a08700 1 --2- 172.21.15.5:0/3960908722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d00e1a4400 0x55d00e1a96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:15.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.562+0000 7f4ef9a08700 1 -- 172.21.15.5:0/3960908722 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d00e1aee00 con 0x55d00e0274a0 2024-05-09T00:53:15.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.563+0000 7f4ef9a08700 1 --2- 172.21.15.5:0/3960908722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d00e0274a0 0x55d00e1ae8c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4ee40a7150 tx=0x7f4ee4010f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b823c5f788a7aa1d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:53:15.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.565+0000 7f4edf7fe700 1 -- 172.21.15.5:0/3960908722 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee40e04d0 con 0x55d00e0274a0 2024-05-09T00:53:15.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.565+0000 7f4edf7fe700 1 -- 172.21.15.5:0/3960908722 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ee40b28b0 con 0x55d00e0274a0 2024-05-09T00:53:15.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.567+0000 7f4f0a682b80 1 -- 172.21.15.5:0/3960908722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d00e0274a0 msgr2=0x55d00e1ae8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:53:15.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.567+0000 7f4f0a682b80 1 --2- 172.21.15.5:0/3960908722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d00e0274a0 0x55d00e1ae8c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4ee40a7150 tx=0x7f4ee4010f50 comp rx=0 tx=0).stop 2024-05-09T00:53:15.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.574+0000 7f4f0a682b80 1 -- 172.21.15.5:0/3960908722 shutdown_connections 2024-05-09T00:53:15.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.574+0000 7f4f0a682b80 1 --2- 172.21.15.5:0/3960908722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d00e1a4400 0x55d00e1a96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:15.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.574+0000 7f4f0a682b80 1 --2- 172.21.15.5:0/3960908722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d00e0274a0 0x55d00e1ae8c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:15.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.574+0000 7f4f0a682b80 1 --2- 172.21.15.5:0/3960908722 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d00e0a1d00 0x55d00e1a9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:15.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.574+0000 7f4f0a682b80 1 -- 172.21.15.5:0/3960908722 >> 172.21.15.5:0/3960908722 conn(0x55d00e02af70 msgr2=0x55d00e1ac910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:53:15.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.578+0000 7f4f0a682b80 1 -- 172.21.15.5:0/3960908722 shutdown_connections 2024-05-09T00:53:15.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.580+0000 7f4f0a682b80 1 -- 172.21.15.5:0/3960908722 wait complete. 2024-05-09T00:53:15.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.581+0000 7f4f0a682b80 1 -- start start 2024-05-09T00:53:15.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.581+0000 7f4f0a682b80 1 Processor -- start 2024-05-09T00:53:15.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.584+0000 7f4f0a682b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d00e0274a0 0x55d00e1b6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:15.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.584+0000 7f4f0a682b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d00e0a1d00 0x55d00e1b66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:15.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.584+0000 7f4f0a682b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d00e1a4400 0x55d00e1ad8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:15.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.585+0000 7f4f0a682b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed800e890 con 0x55d00e0a1d00 2024-05-09T00:53:15.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.585+0000 7f4f0a682b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed800e1c0 con 0x55d00e1a4400 2024-05-09T00:53:15.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.585+0000 7f4f0a682b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed800b3c0 con 0x55d00e0274a0 2024-05-09T00:53:15.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.585+0000 7f4f0a682b80 10 client.0 _ll_register_callbacks cb 0x55d00df386c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:53:15.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.587+0000 7f4ef9207700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d00e0a1d00 0x55d00e1b66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:15.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.588+0000 7f4ef9a08700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d00e0274a0 0x55d00e1b6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:15.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.589+0000 7f4ef9a08700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d00e0274a0 0x55d00e1b6190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:55814/0 (socket says 172.21.15.5:55814) 2024-05-09T00:53:15.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.589+0000 7f4ef9a08700 1 -- 172.21.15.5:0/1291209264 learned_addr learned my addr 172.21.15.5:0/1291209264 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:53:15.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.589+0000 7f4efa209700 1 --2- 172.21.15.5:0/1291209264 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d00e1a4400 0x55d00e1ad8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:15.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.592+0000 7f4ef9a08700 1 -- 172.21.15.5:0/1291209264 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d00e1a4400 msgr2=0x55d00e1ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:53:15.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.592+0000 7f4ef9a08700 1 --2- 172.21.15.5:0/1291209264 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d00e1a4400 0x55d00e1ad8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:15.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.592+0000 7f4ef9a08700 1 -- 172.21.15.5:0/1291209264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d00e0a1d00 msgr2=0x55d00e1b66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:53:15.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.592+0000 7f4ef9a08700 1 --2- 172.21.15.5:0/1291209264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d00e0a1d00 0x55d00e1b66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:15.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.592+0000 7f4ef9a08700 1 -- 172.21.15.5:0/1291209264 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d00e1ade20 con 0x55d00e0274a0 2024-05-09T00:53:15.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.594+0000 7f4ef9a08700 1 --2- 172.21.15.5:0/1291209264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d00e0274a0 0x55d00e1b6190 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4ee40a5eb0 tx=0x7f4ee40a9de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0c1872e002155fa server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:53:15.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.594+0000 7f4edd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-09T00:53:15.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.595+0000 7f4edd7fa700 1 -- 172.21.15.5:0/1291209264 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee40c8070 con 0x55d00e0274a0 2024-05-09T00:53:15.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.595+0000 7f4f0a682b80 1 -- 172.21.15.5:0/1291209264 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ee40b28b0 con 0x55d00e0274a0 2024-05-09T00:53:15.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.599+0000 7f4edd7fa700 1 -- 172.21.15.5:0/1291209264 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f4ee4011830 con 0x55d00e0274a0 2024-05-09T00:53:15.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.599+0000 7f4edd7fa700 1 -- 172.21.15.5:0/1291209264 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f4ee4011a70 con 0x55d00e0274a0 2024-05-09T00:53:15.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.599+0000 7f4f0a682b80 10 client.25768 fetch_fsmap learned FSMap version 31 2024-05-09T00:53:15.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.599+0000 7f4f0a682b80 1 -- 172.21.15.5:0/1291209264 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d00e238f70 con 0x55d00e0274a0 2024-05-09T00:53:15.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.604+0000 7f4edd7fa700 1 -- 172.21.15.5:0/1291209264 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ee40b28b0 con 0x55d00e0274a0 2024-05-09T00:53:15.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.604+0000 7f4edd7fa700 1 -- 172.21.15.5:0/1291209264 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f4ee40af420 con 0x55d00e0274a0 2024-05-09T00:53:15.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.604+0000 7f4f0a682b80 10 client.25768 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:53:15.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.604+0000 7f4f0a682b80 10 client.25768 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-05-09T00:53:15.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.604+0000 7f4f0a682b80 20 client.25768 populate_metadata read hostname 'smithi005' 2024-05-09T00:53:15.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.604+0000 7f4f0a682b80 1 --2- 172.21.15.5:0/1291209264 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x55d00e239670 0x55d00e23ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:15.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.605+0000 7f4f0a682b80 4 client.25768 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:53:15.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.605+0000 7f4f0a682b80 1 -- 172.21.15.5:0/1291209264 --> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d00e0ceb20 con 0x55d00e239670 2024-05-09T00:53:15.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.612+0000 7f4ef9207700 1 --2- 172.21.15.5:0/1291209264 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x55d00e239670 0x55d00e23ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:15.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.626+0000 7f4edd7fa700 10 client.25768 ms_handle_connect on v2:172.21.15.157:6836/744863009 2024-05-09T00:53:15.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.626+0000 7f4ef9207700 1 --2- 172.21.15.5:0/1291209264 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x55d00e239670 0x55d00e23ba90 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=689b7c74cda34ab8 server_cookie=ea40f365f3574fe8 in_seq=0 out_seq=0 2024-05-09T00:53:15.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.629+0000 7f4edd7fa700 1 -- 172.21.15.5:0/1291209264 <== mds.1 v2:172.21.15.157:6836/744863009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d00e0ceb20 con 0x55d00e239670 2024-05-09T00:53:15.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.630+0000 7f4edd7fa700 10 client.25768 handle_command_reply: tid=0 2024-05-09T00:53:15.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.630+0000 7f4f0a682b80 1 client.25768 shutdown 2024-05-09T00:53:15.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.633+0000 7f4f0a682b80 1 -- 172.21.15.5:0/1291209264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d00e0274a0 msgr2=0x55d00e1b6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:53:15.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.633+0000 7f4f0a682b80 1 --2- 172.21.15.5:0/1291209264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d00e0274a0 0x55d00e1b6190 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4ee40a5eb0 tx=0x7f4ee40a9de0 comp rx=0 tx=0).stop 2024-05-09T00:53:15.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.634+0000 7f4f0a682b80 1 -- 172.21.15.5:0/1291209264 shutdown_connections 2024-05-09T00:53:15.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.634+0000 7f4f0a682b80 1 --2- 172.21.15.5:0/1291209264 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x55d00e239670 0x55d00e23ba90 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:15.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.634+0000 7f4f0a682b80 1 --2- 172.21.15.5:0/1291209264 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d00e1a4400 0x55d00e1ad8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:15.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.634+0000 7f4f0a682b80 1 --2- 172.21.15.5:0/1291209264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d00e0274a0 0x55d00e1b6190 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:15.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.634+0000 7f4f0a682b80 1 --2- 172.21.15.5:0/1291209264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d00e0a1d00 0x55d00e1b66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:15.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.634+0000 7f4f0a682b80 1 -- 172.21.15.5:0/1291209264 >> 172.21.15.5:0/1291209264 conn(0x55d00e02af70 msgr2=0x55d00e1a9160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:53:15.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.634+0000 7f4edd7fa700 0 client.25768 ms_handle_reset on v2:172.21.15.157:6836/744863009 2024-05-09T00:53:15.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.634+0000 7f4f0a682b80 1 -- 172.21.15.5:0/1291209264 shutdown_connections 2024-05-09T00:53:15.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.634+0000 7f4f0a682b80 1 -- 172.21.15.5:0/1291209264 wait complete. 2024-05-09T00:53:15.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.634+0000 7f4f0a682b80 20 client.25768 trim_cache size 0 max 16384 2024-05-09T00:53:15.639 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-09T00:53:15.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.637+0000 7f4efbc6c700 1 -- 172.21.15.5:0/2998496031 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f4ee00b2a50 msgr2=0x7f4ee00b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:53:15.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.637+0000 7f4efbc6c700 1 --2- 172.21.15.5:0/2998496031 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f4ee00b2a50 0x7f4ee00b4f60 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0009810 tx=0x7f4ef0005da0 comp rx=0 tx=0).stop 2024-05-09T00:53:15.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.637+0000 7f4efbc6c700 1 -- 172.21.15.5:0/2998496031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ef4103250 msgr2=0x7f4ef41a14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:53:15.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.637+0000 7f4efbc6c700 1 --2- 172.21.15.5:0/2998496031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ef4103250 0x7f4ef41a14e0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f4ee400b500 tx=0x7f4ee400b8c0 comp rx=0 tx=0).stop 2024-05-09T00:53:15.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.640+0000 7f4efbc6c700 1 -- 172.21.15.5:0/2998496031 shutdown_connections 2024-05-09T00:53:15.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.640+0000 7f4efbc6c700 1 --2- 172.21.15.5:0/2998496031 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f4ee00b2a50 0x7f4ee00b4f60 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:15.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.640+0000 7f4efbc6c700 1 --2- 172.21.15.5:0/2998496031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ef41086a0 0x7f4ef41a5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:15.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.640+0000 7f4efbc6c700 1 --2- 172.21.15.5:0/2998496031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ef4103250 0x7f4ef41a14e0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:15.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.640+0000 7f4efbc6c700 1 --2- 172.21.15.5:0/2998496031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ef4103ba0 0x7f4ef41a1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:15.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.640+0000 7f4efbc6c700 1 -- 172.21.15.5:0/2998496031 >> 172.21.15.5:0/2998496031 conn(0x7f4ef40fedf0 msgr2=0x7f4ef41060a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:53:15.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.643+0000 7f4efbc6c700 1 -- 172.21.15.5:0/2998496031 shutdown_connections 2024-05-09T00:53:15.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:15.643+0000 7f4efbc6c700 1 -- 172.21.15.5:0/2998496031 wait complete. 2024-05-09T00:53:15.698 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:53:15 smithi005 ceph-mon[30365]: pgmap v578: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 863 MiB/s wr, 2.14k op/s 2024-05-09T00:53:15.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:53:15 smithi131 ceph-mon[34547]: pgmap v578: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 863 MiB/s wr, 2.14k op/s 2024-05-09T00:53:15.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:53:15 smithi157 ceph-mon[34294]: pgmap v578: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 863 MiB/s wr, 2.14k op/s 2024-05-09T00:53:17.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.526+0000 7f5869b3f640 1 -- 172.21.15.5:0/2115515823 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5864103d90 msgr2=0x7f58641041a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:53:17.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.526+0000 7f5869b3f640 1 --2- 172.21.15.5:0/2115515823 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5864103d90 0x7f58641041a0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f5858009900 tx=0x7f585802a410 comp rx=0 tx=0).stop 2024-05-09T00:53:17.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.527+0000 7f5869b3f640 1 -- 172.21.15.5:0/2115515823 shutdown_connections 2024-05-09T00:53:17.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.527+0000 7f5869b3f640 1 --2- 172.21.15.5:0/2115515823 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5864108a90 0x7f586410aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:17.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.527+0000 7f5869b3f640 1 --2- 172.21.15.5:0/2115515823 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5864103d90 0x7f58641041a0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:17.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.527+0000 7f5869b3f640 1 --2- 172.21.15.5:0/2115515823 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5864102b50 0x7f5864103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:17.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.527+0000 7f5869b3f640 1 -- 172.21.15.5:0/2115515823 >> 172.21.15.5:0/2115515823 conn(0x7f58640fdd40 msgr2=0x7f58641001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:53:17.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.527+0000 7f5869b3f640 1 -- 172.21.15.5:0/2115515823 shutdown_connections 2024-05-09T00:53:17.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.527+0000 7f5869b3f640 1 -- 172.21.15.5:0/2115515823 wait complete. 2024-05-09T00:53:17.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.528+0000 7f5869b3f640 1 Processor -- start 2024-05-09T00:53:17.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.528+0000 7f5869b3f640 1 -- start start 2024-05-09T00:53:17.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.528+0000 7f5869b3f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5864102b50 0x7f586410d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:17.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.528+0000 7f5869b3f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5864103d90 0x7f586410b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:17.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.528+0000 7f5869b3f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5864108a90 0x7f586410be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:17.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.528+0000 7f5869b3f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f586410f7c0 con 0x7f5864102b50 2024-05-09T00:53:17.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.528+0000 7f5869b3f640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f586410f640 con 0x7f5864103d90 2024-05-09T00:53:17.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.528+0000 7f5869b3f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f586410f940 con 0x7f5864108a90 2024-05-09T00:53:17.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.529+0000 7f58637fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5864102b50 0x7f586410d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:17.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.529+0000 7f58637fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5864102b50 0x7f586410d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57652/0 (socket says 172.21.15.5:57652) 2024-05-09T00:53:17.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.529+0000 7f58637fe640 1 -- 172.21.15.5:0/1626436755 learned_addr learned my addr 172.21.15.5:0/1626436755 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:53:17.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.529+0000 7f5863fff640 1 --2- 172.21.15.5:0/1626436755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5864108a90 0x7f586410be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:17.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.529+0000 7f58637fe640 1 -- 172.21.15.5:0/1626436755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5864108a90 msgr2=0x7f586410be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:53:17.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.529+0000 7f5862ffd640 1 --2- 172.21.15.5:0/1626436755 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5864103d90 0x7f586410b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:17.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.529+0000 7f58637fe640 1 --2- 172.21.15.5:0/1626436755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5864108a90 0x7f586410be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:17.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.529+0000 7f58637fe640 1 -- 172.21.15.5:0/1626436755 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5864103d90 msgr2=0x7f586410b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:53:17.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.529+0000 7f58637fe640 1 --2- 172.21.15.5:0/1626436755 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5864103d90 0x7f586410b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:17.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.530+0000 7f58637fe640 1 -- 172.21.15.5:0/1626436755 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f586410c530 con 0x7f5864102b50 2024-05-09T00:53:17.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.530+0000 7f58637fe640 1 --2- 172.21.15.5:0/1626436755 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5864102b50 0x7f586410d220 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f585400d740 tx=0x7f585400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:53:17.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.530+0000 7f5860ff9640 1 -- 172.21.15.5:0/1626436755 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5854017070 con 0x7f5864102b50 2024-05-09T00:53:17.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.530+0000 7f5860ff9640 1 -- 172.21.15.5:0/1626436755 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f585400de50 con 0x7f5864102b50 2024-05-09T00:53:17.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.530+0000 7f5869b3f640 1 -- 172.21.15.5:0/1626436755 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58641a4400 con 0x7f5864102b50 2024-05-09T00:53:17.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.530+0000 7f5860ff9640 1 -- 172.21.15.5:0/1626436755 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5854003c20 con 0x7f5864102b50 2024-05-09T00:53:17.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.530+0000 7f5869b3f640 1 -- 172.21.15.5:0/1626436755 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58641a48c0 con 0x7f5864102b50 2024-05-09T00:53:17.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.533+0000 7f5860ff9640 1 -- 172.21.15.5:0/1626436755 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f5854010460 con 0x7f5864102b50 2024-05-09T00:53:17.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.534+0000 7f5860ff9640 1 --2- 172.21.15.5:0/1626436755 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f584c0b27e0 0x7f584c0b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:17.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.534+0000 7f5860ff9640 1 -- 172.21.15.5:0/1626436755 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f58540e2c40 con 0x7f5864102b50 2024-05-09T00:53:17.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.534+0000 7f5862ffd640 1 --2- 172.21.15.5:0/1626436755 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f584c0b27e0 0x7f584c0b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:17.634 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.633+0000 7f5862ffd640 1 --2- 172.21.15.5:0/1626436755 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f584c0b27e0 0x7f584c0b4cf0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5858005d90 tx=0x7f5858031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:53:17.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.634+0000 7f58709f6740 1 Processor -- start 2024-05-09T00:53:17.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.634+0000 7f58709f6740 1 -- start start 2024-05-09T00:53:17.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.634+0000 7f58709f6740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623b62d51b0 0x5623b6679500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:17.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.634+0000 7f58709f6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623b62ac2f0 0x5623b6679a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:17.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.634+0000 7f58709f6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623b62b44d0 0x5623b6680980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:17.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.634+0000 7f58709f6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5623b65dd7c0 con 0x5623b62b44d0 2024-05-09T00:53:17.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.634+0000 7f58709f6740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5623b65790e0 con 0x5623b62d51b0 2024-05-09T00:53:17.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.634+0000 7f58709f6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5623b65d9780 con 0x5623b62ac2f0 2024-05-09T00:53:17.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.635+0000 7f5863fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623b62b44d0 0x5623b6680980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:17.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.635+0000 7f5863fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623b62b44d0 0x5623b6680980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57654/0 (socket says 172.21.15.5:57654) 2024-05-09T00:53:17.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.635+0000 7f5863fff640 1 -- 172.21.15.5:0/2592254584 learned_addr learned my addr 172.21.15.5:0/2592254584 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:53:17.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.635+0000 7f5862ffd640 1 --2- 172.21.15.5:0/2592254584 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623b62ac2f0 0x5623b6679a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:17.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.635+0000 7f58637fe640 1 --2- 172.21.15.5:0/2592254584 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623b62d51b0 0x5623b6679500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:17.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.635+0000 7f5863fff640 1 -- 172.21.15.5:0/2592254584 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623b62ac2f0 msgr2=0x5623b6679a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:53:17.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.635+0000 7f5863fff640 1 --2- 172.21.15.5:0/2592254584 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623b62ac2f0 0x5623b6679a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:17.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.635+0000 7f5863fff640 1 -- 172.21.15.5:0/2592254584 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623b62d51b0 msgr2=0x5623b6679500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:53:17.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.635+0000 7f5863fff640 1 --2- 172.21.15.5:0/2592254584 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623b62d51b0 0x5623b6679500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:17.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.635+0000 7f5863fff640 1 -- 172.21.15.5:0/2592254584 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623b65a27c0 con 0x5623b62b44d0 2024-05-09T00:53:17.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.635+0000 7f5862ffd640 1 --2- 172.21.15.5:0/2592254584 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623b62ac2f0 0x5623b6679a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:53:17.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.636+0000 7f5863fff640 1 --2- 172.21.15.5:0/2592254584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623b62b44d0 0x5623b6680980 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f585c009ee0 tx=0x7f585c010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea098d7b8a2cda66 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:53:17.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.636+0000 7f5849ffb640 1 -- 172.21.15.5:0/2592254584 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585c016070 con 0x5623b62b44d0 2024-05-09T00:53:17.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.636+0000 7f5849ffb640 1 -- 172.21.15.5:0/2592254584 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f585c004d20 con 0x5623b62b44d0 2024-05-09T00:53:17.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.636+0000 7f5849ffb640 1 -- 172.21.15.5:0/2592254584 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585c012780 con 0x5623b62b44d0 2024-05-09T00:53:17.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.637+0000 7f58709f6740 1 -- 172.21.15.5:0/2592254584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623b62b44d0 msgr2=0x5623b6680980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:53:17.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.637+0000 7f58709f6740 1 --2- 172.21.15.5:0/2592254584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623b62b44d0 0x5623b6680980 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f585c009ee0 tx=0x7f585c010740 comp rx=0 tx=0).stop 2024-05-09T00:53:17.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.637+0000 7f58709f6740 1 -- 172.21.15.5:0/2592254584 shutdown_connections 2024-05-09T00:53:17.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.637+0000 7f58709f6740 1 --2- 172.21.15.5:0/2592254584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623b62b44d0 0x5623b6680980 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:17.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.637+0000 7f58709f6740 1 --2- 172.21.15.5:0/2592254584 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623b62ac2f0 0x5623b6679a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:17.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.637+0000 7f58709f6740 1 --2- 172.21.15.5:0/2592254584 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623b62d51b0 0x5623b6679500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:17.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.637+0000 7f58709f6740 1 -- 172.21.15.5:0/2592254584 >> 172.21.15.5:0/2592254584 conn(0x5623b62c9300 msgr2=0x5623b6674e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:53:17.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.637+0000 7f58709f6740 1 -- 172.21.15.5:0/2592254584 shutdown_connections 2024-05-09T00:53:17.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.637+0000 7f58709f6740 1 -- 172.21.15.5:0/2592254584 wait complete. 2024-05-09T00:53:17.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.638+0000 7f58709f6740 1 -- start start 2024-05-09T00:53:17.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.638+0000 7f58709f6740 1 Processor -- start 2024-05-09T00:53:17.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.639+0000 7f58709f6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623b62ac2f0 0x5623b6683230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:17.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.639+0000 7f58709f6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623b62b44d0 0x5623b6683770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:17.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.639+0000 7f58709f6740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623b62d51b0 0x5623b668a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:17.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.639+0000 7f58709f6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5623b66813d0 con 0x5623b62ac2f0 2024-05-09T00:53:17.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.639+0000 7f58709f6740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5623b65649c0 con 0x5623b62d51b0 2024-05-09T00:53:17.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.639+0000 7f58709f6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5623b6681550 con 0x5623b62b44d0 2024-05-09T00:53:17.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.639+0000 7f58637fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623b62ac2f0 0x5623b6683230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:17.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.639+0000 7f58637fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623b62ac2f0 0x5623b6683230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57664/0 (socket says 172.21.15.5:57664) 2024-05-09T00:53:17.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.639+0000 7f58637fe640 1 -- 172.21.15.5:0/1334102515 learned_addr learned my addr 172.21.15.5:0/1334102515 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:53:17.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.639+0000 7f58709f6740 10 client.0 _ll_register_callbacks cb 0x5623b6591020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:53:17.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.639+0000 7f5862ffd640 1 --2- 172.21.15.5:0/1334102515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623b62b44d0 0x5623b6683770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:17.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.639+0000 7f5863fff640 1 --2- 172.21.15.5:0/1334102515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623b62d51b0 0x5623b668a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:17.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.640+0000 7f58637fe640 1 -- 172.21.15.5:0/1334102515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623b62b44d0 msgr2=0x5623b6683770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:53:17.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.640+0000 7f58637fe640 1 --2- 172.21.15.5:0/1334102515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623b62b44d0 0x5623b6683770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:17.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.640+0000 7f58637fe640 1 -- 172.21.15.5:0/1334102515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623b62d51b0 msgr2=0x5623b668a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:53:17.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.640+0000 7f58637fe640 1 --2- 172.21.15.5:0/1334102515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623b62d51b0 0x5623b668a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:17.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.640+0000 7f58637fe640 1 -- 172.21.15.5:0/1334102515 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623b658b1f0 con 0x5623b62ac2f0 2024-05-09T00:53:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.640+0000 7f58637fe640 1 --2- 172.21.15.5:0/1334102515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623b62ac2f0 0x5623b6683230 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f58540aa1c0 tx=0x7f58540a9ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3595b183971c54c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:53:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.640+0000 7f5842ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:53:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.640+0000 7f5842ffd640 1 -- 172.21.15.5:0/1334102515 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5854020040 con 0x5623b62ac2f0 2024-05-09T00:53:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.641+0000 7f5842ffd640 1 -- 172.21.15.5:0/1334102515 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f58540e0bd0 con 0x5623b62ac2f0 2024-05-09T00:53:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.641+0000 7f58709f6740 1 -- 172.21.15.5:0/1334102515 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f585c004d20 con 0x5623b62ac2f0 2024-05-09T00:53:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.641+0000 7f5842ffd640 1 -- 172.21.15.5:0/1334102515 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f58540b3090 con 0x5623b62ac2f0 2024-05-09T00:53:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.641+0000 7f58709f6740 10 client.16224 fetch_fsmap learned FSMap version 31 2024-05-09T00:53:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.641+0000 7f58709f6740 1 -- 172.21.15.5:0/1334102515 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5623b6340790 con 0x5623b62ac2f0 2024-05-09T00:53:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.641+0000 7f5842ffd640 1 -- 172.21.15.5:0/1334102515 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f585c004d20 con 0x5623b62ac2f0 2024-05-09T00:53:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.642+0000 7f5842ffd640 1 -- 172.21.15.5:0/1334102515 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f58540a5ba0 con 0x5623b62ac2f0 2024-05-09T00:53:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.642+0000 7f58709f6740 10 client.16224 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:53:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.642+0000 7f58709f6740 10 client.16224 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:53:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.642+0000 7f58709f6740 20 client.16224 populate_metadata read hostname 'smithi005' 2024-05-09T00:53:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.642+0000 7f58709f6740 1 --2- 172.21.15.5:0/1334102515 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5623b6293d80 0x5623b6690bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.642+0000 7f58709f6740 4 client.16224 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:53:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.642+0000 7f58709f6740 1 -- 172.21.15.5:0/1334102515 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5623b65eb3d0 con 0x5623b6293d80 2024-05-09T00:53:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.642+0000 7f5862ffd640 1 --2- 172.21.15.5:0/1334102515 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5623b6293d80 0x5623b6690bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.644+0000 7f5842ffd640 10 client.16224 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:53:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.644+0000 7f5862ffd640 1 --2- 172.21.15.5:0/1334102515 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5623b6293d80 0x5623b6690bb0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90e145eca7e89c04 server_cookie=57a814e75949e1ee in_seq=0 out_seq=0 2024-05-09T00:53:17.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.647+0000 7f5842ffd640 1 -- 172.21.15.5:0/1334102515 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5623b65eb3d0 con 0x5623b6293d80 2024-05-09T00:53:17.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.647+0000 7f5842ffd640 10 client.16224 handle_command_reply: tid=0 2024-05-09T00:53:17.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.647+0000 7f58709f6740 1 client.16224 shutdown 2024-05-09T00:53:17.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.647+0000 7f58709f6740 1 -- 172.21.15.5:0/1334102515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623b62ac2f0 msgr2=0x5623b6683230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:53:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.647+0000 7f58709f6740 1 --2- 172.21.15.5:0/1334102515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623b62ac2f0 0x5623b6683230 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f58540aa1c0 tx=0x7f58540a9ea0 comp rx=0 tx=0).stop 2024-05-09T00:53:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.647+0000 7f58709f6740 1 -- 172.21.15.5:0/1334102515 shutdown_connections 2024-05-09T00:53:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.647+0000 7f58709f6740 1 --2- 172.21.15.5:0/1334102515 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5623b6293d80 0x5623b6690bb0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.647+0000 7f5842ffd640 0 client.16224 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:53:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.647+0000 7f58709f6740 1 --2- 172.21.15.5:0/1334102515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623b62d51b0 0x5623b668a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.647+0000 7f58709f6740 1 --2- 172.21.15.5:0/1334102515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623b62b44d0 0x5623b6683770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.647+0000 7f58709f6740 1 --2- 172.21.15.5:0/1334102515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623b62ac2f0 0x5623b6683230 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.647+0000 7f58709f6740 1 -- 172.21.15.5:0/1334102515 >> 172.21.15.5:0/1334102515 conn(0x5623b62c9300 msgr2=0x5623b6677da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:53:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.648+0000 7f58709f6740 1 -- 172.21.15.5:0/1334102515 shutdown_connections 2024-05-09T00:53:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.648+0000 7f58709f6740 1 -- 172.21.15.5:0/1334102515 wait complete. 2024-05-09T00:53:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.648+0000 7f58709f6740 20 client.16224 trim_cache size 0 max 16384 2024-05-09T00:53:17.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.680+0000 7f58709f6740 1 Processor -- start 2024-05-09T00:53:17.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.681+0000 7f58709f6740 1 -- start start 2024-05-09T00:53:17.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.681+0000 7f58709f6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623b62ac2f0 0x5623b667dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:17.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.681+0000 7f58709f6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623b62b44d0 0x5623b6683510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:17.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.681+0000 7f58709f6740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623b62d51b0 0x5623b6673700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:17.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.682+0000 7f5862ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623b62b44d0 0x5623b6683510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:17.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.682+0000 7f5862ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623b62b44d0 0x5623b6683510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57676/0 (socket says 172.21.15.5:57676) 2024-05-09T00:53:17.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.682+0000 7f5862ffd640 1 -- 172.21.15.5:0/331028634 learned_addr learned my addr 172.21.15.5:0/331028634 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:53:17.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.682+0000 7f58709f6740 1 -- 172.21.15.5:0/331028634 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f58641ad800 con 0x5623b62b44d0 2024-05-09T00:53:17.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.682+0000 7f58709f6740 1 -- 172.21.15.5:0/331028634 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f58641af340 con 0x5623b62d51b0 2024-05-09T00:53:17.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.682+0000 7f58709f6740 1 -- 172.21.15.5:0/331028634 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f58641aded0 con 0x5623b62ac2f0 2024-05-09T00:53:17.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.683+0000 7f5863fff640 1 --2- 172.21.15.5:0/331028634 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623b62d51b0 0x5623b6673700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:17.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.683+0000 7f58637fe640 1 --2- 172.21.15.5:0/331028634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623b62ac2f0 0x5623b667dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:17.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.683+0000 7f5862ffd640 1 -- 172.21.15.5:0/331028634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623b62ac2f0 msgr2=0x5623b667dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:53:17.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.683+0000 7f5862ffd640 1 --2- 172.21.15.5:0/331028634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623b62ac2f0 0x5623b667dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:17.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.683+0000 7f5862ffd640 1 -- 172.21.15.5:0/331028634 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623b62d51b0 msgr2=0x5623b6673700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:53:17.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.683+0000 7f5862ffd640 1 --2- 172.21.15.5:0/331028634 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623b62d51b0 0x5623b6673700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:17.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.683+0000 7f5862ffd640 1 -- 172.21.15.5:0/331028634 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623b63b75f0 con 0x5623b62b44d0 2024-05-09T00:53:17.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.684+0000 7f5862ffd640 1 --2- 172.21.15.5:0/331028634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623b62b44d0 0x5623b6683510 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f5858004650 tx=0x7f585803b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe04ddae6adc36a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:53:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.692+0000 7f58497fa640 1 -- 172.21.15.5:0/331028634 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5858057070 con 0x5623b62b44d0 2024-05-09T00:53:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.692+0000 7f58497fa640 1 -- 172.21.15.5:0/331028634 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5858031500 con 0x5623b62b44d0 2024-05-09T00:53:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.694+0000 7f58709f6740 1 -- 172.21.15.5:0/331028634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623b62b44d0 msgr2=0x5623b6683510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:53:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.694+0000 7f58709f6740 1 --2- 172.21.15.5:0/331028634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623b62b44d0 0x5623b6683510 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f5858004650 tx=0x7f585803b000 comp rx=0 tx=0).stop 2024-05-09T00:53:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.694+0000 7f58709f6740 1 -- 172.21.15.5:0/331028634 shutdown_connections 2024-05-09T00:53:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.694+0000 7f58709f6740 1 --2- 172.21.15.5:0/331028634 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623b62d51b0 0x5623b6673700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:17.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.694+0000 7f58709f6740 1 --2- 172.21.15.5:0/331028634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623b62b44d0 0x5623b6683510 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:17.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.694+0000 7f58709f6740 1 --2- 172.21.15.5:0/331028634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623b62ac2f0 0x5623b667dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:17.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.694+0000 7f58709f6740 1 -- 172.21.15.5:0/331028634 >> 172.21.15.5:0/331028634 conn(0x5623b62c9300 msgr2=0x5623b6682e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:53:17.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.695+0000 7f58709f6740 1 -- 172.21.15.5:0/331028634 shutdown_connections 2024-05-09T00:53:17.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.695+0000 7f58709f6740 1 -- 172.21.15.5:0/331028634 wait complete. 2024-05-09T00:53:17.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.696+0000 7f58709f6740 1 -- start start 2024-05-09T00:53:17.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.696+0000 7f58709f6740 1 Processor -- start 2024-05-09T00:53:17.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.697+0000 7f58709f6740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623b62ac2f0 0x5623b6686f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:17.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.698+0000 7f58709f6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623b62d51b0 0x5623b6687440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:17.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.698+0000 7f58709f6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623b62691c0 0x5623b6688320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:17.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.698+0000 7f58709f6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f58641aec70 con 0x5623b62691c0 2024-05-09T00:53:17.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.698+0000 7f58709f6740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f58641ae5a0 con 0x5623b62ac2f0 2024-05-09T00:53:17.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.698+0000 7f58709f6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f58641ab7e0 con 0x5623b62d51b0 2024-05-09T00:53:17.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.698+0000 7f58637fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623b62ac2f0 0x5623b6686f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:17.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.698+0000 7f58637fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623b62ac2f0 0x5623b6686f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:43790/0 (socket says 172.21.15.5:43790) 2024-05-09T00:53:17.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.698+0000 7f58637fe640 1 -- 172.21.15.5:0/2245406626 learned_addr learned my addr 172.21.15.5:0/2245406626 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:53:17.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.699+0000 7f58709f6740 10 client.0 _ll_register_callbacks cb 0x7f58540a60f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:53:17.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.700+0000 7f5863fff640 1 --2- 172.21.15.5:0/2245406626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623b62691c0 0x5623b6688320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:17.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.700+0000 7f58637fe640 1 -- 172.21.15.5:0/2245406626 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623b62d51b0 msgr2=0x5623b6687440 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:53:17.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.700+0000 7f58637fe640 1 --2- 172.21.15.5:0/2245406626 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623b62d51b0 0x5623b6687440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:17.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.700+0000 7f58637fe640 1 -- 172.21.15.5:0/2245406626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623b62691c0 msgr2=0x5623b6688320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:53:17.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.700+0000 7f58637fe640 1 --2- 172.21.15.5:0/2245406626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623b62691c0 0x5623b6688320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:17.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.700+0000 7f58637fe640 1 -- 172.21.15.5:0/2245406626 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623b65a1ed0 con 0x5623b62ac2f0 2024-05-09T00:53:17.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.700+0000 7f5863fff640 1 --2- 172.21.15.5:0/2245406626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623b62691c0 0x5623b6688320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:53:17.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.701+0000 7f58637fe640 1 --2- 172.21.15.5:0/2245406626 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623b62ac2f0 0x5623b6686f00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f58540a74a0 tx=0x7f58540a9130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f421f383f1b5b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:53:17.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.711+0000 7f5842ffd640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-09T00:53:17.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.711+0000 7f5842ffd640 1 -- 172.21.15.5:0/2245406626 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5854020070 con 0x5623b62ac2f0 2024-05-09T00:53:17.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.711+0000 7f58709f6740 1 -- 172.21.15.5:0/2245406626 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5623b65e5240 con 0x5623b62ac2f0 2024-05-09T00:53:17.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.711+0000 7f5842ffd640 1 -- 172.21.15.5:0/2245406626 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f58540e0bd0 con 0x5623b62ac2f0 2024-05-09T00:53:17.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.718+0000 7f5842ffd640 1 -- 172.21.15.5:0/2245406626 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f58540aa000 con 0x5623b62ac2f0 2024-05-09T00:53:17.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.718+0000 7f58709f6740 10 client.25757 fetch_fsmap learned FSMap version 31 2024-05-09T00:53:17.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.718+0000 7f58709f6740 1 -- 172.21.15.5:0/2245406626 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5623b6689aa0 con 0x5623b62ac2f0 2024-05-09T00:53:17.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.725+0000 7f5842ffd640 1 -- 172.21.15.5:0/2245406626 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5623b65e5240 con 0x5623b62ac2f0 2024-05-09T00:53:17.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.726+0000 7f5842ffd640 1 -- 172.21.15.5:0/2245406626 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f58540af770 con 0x5623b62ac2f0 2024-05-09T00:53:17.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.734+0000 7f58709f6740 10 client.25757 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:53:17.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.734+0000 7f58709f6740 10 client.25757 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:53:17.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.734+0000 7f58709f6740 20 client.25757 populate_metadata read hostname 'smithi005' 2024-05-09T00:53:17.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.734+0000 7f58709f6740 1 --2- 172.21.15.5:0/2245406626 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5623b66f90f0 0x5623b66fb550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:17.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.734+0000 7f58709f6740 4 client.25757 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:53:17.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.734+0000 7f58709f6740 1 -- 172.21.15.5:0/2245406626 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5623b6560350 con 0x5623b66f90f0 2024-05-09T00:53:17.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.740+0000 7f5862ffd640 1 --2- 172.21.15.5:0/2245406626 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5623b66f90f0 0x5623b66fb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:17.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.752+0000 7f5842ffd640 10 client.25757 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:53:17.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:17.752+0000 7f5862ffd640 1 --2- 172.21.15.5:0/2245406626 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5623b66f90f0 0x5623b66fb550 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2b105cffd2886bb server_cookie=9fb2602df0c9c6a6 in_seq=0 out_seq=0 2024-05-09T00:53:17.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:53:17 smithi005 ceph-mon[30365]: pgmap v579: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 926 MiB/s wr, 2.33k op/s 2024-05-09T00:53:17.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:53:17 smithi131 ceph-mon[34547]: pgmap v579: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 926 MiB/s wr, 2.33k op/s 2024-05-09T00:53:17.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:53:17 smithi157 ceph-mon[34294]: pgmap v579: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 926 MiB/s wr, 2.33k op/s 2024-05-09T00:53:18.013 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T00:53:18.901 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:53:19.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:53:19 smithi005 ceph-mon[30365]: pgmap v580: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 809 MiB/s wr, 2.07k op/s 2024-05-09T00:53:19.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:53:19 smithi131 ceph-mon[34547]: pgmap v580: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 809 MiB/s wr, 2.07k op/s 2024-05-09T00:53:19.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:53:19 smithi157 ceph-mon[34294]: pgmap v580: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 809 MiB/s wr, 2.07k op/s 2024-05-09T00:53:21.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:53:21 smithi131 ceph-mon[34547]: pgmap v581: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 971 MiB/s wr, 2.30k op/s 2024-05-09T00:53:21.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:53:21 smithi157 ceph-mon[34294]: pgmap v581: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 971 MiB/s wr, 2.30k op/s 2024-05-09T00:53:22.040 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:53:21 smithi005 ceph-mon[30365]: pgmap v581: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 971 MiB/s wr, 2.30k op/s 2024-05-09T00:53:23.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:53:23 smithi005 ceph-mon[30365]: pgmap v582: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 862 MiB/s wr, 2.13k op/s 2024-05-09T00:53:23.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:53:23 smithi131 ceph-mon[34547]: pgmap v582: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 862 MiB/s wr, 2.13k op/s 2024-05-09T00:53:23.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:53:23 smithi157 ceph-mon[34294]: pgmap v582: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 862 MiB/s wr, 2.13k op/s 2024-05-09T00:53:24.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.128+0000 7faee990c700 1 -- 172.21.15.5:0/3027457868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faee41084d0 msgr2=0x7faee41089a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:53:24.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.128+0000 7faee990c700 1 --2- 172.21.15.5:0/3027457868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faee41084d0 0x7faee41089a0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7faecc009990 tx=0x7faecc009ca0 comp rx=0 tx=0).stop 2024-05-09T00:53:24.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.128+0000 7faee990c700 1 -- 172.21.15.5:0/3027457868 shutdown_connections 2024-05-09T00:53:24.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.128+0000 7faee990c700 1 --2- 172.21.15.5:0/3027457868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faee41084d0 0x7faee41089a0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:24.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.128+0000 7faee990c700 1 --2- 172.21.15.5:0/3027457868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faee4103080 0x7faee4103490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:24.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.128+0000 7faee990c700 1 --2- 172.21.15.5:0/3027457868 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faee41039d0 0x7faee4107f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:24.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.128+0000 7faee990c700 1 -- 172.21.15.5:0/3027457868 >> 172.21.15.5:0/3027457868 conn(0x7faee4078ac0 msgr2=0x7faee4078f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:53:24.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.139+0000 7faee990c700 1 -- 172.21.15.5:0/3027457868 shutdown_connections 2024-05-09T00:53:24.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.140+0000 7faee990c700 1 -- 172.21.15.5:0/3027457868 wait complete. 2024-05-09T00:53:24.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.141+0000 7faee990c700 1 Processor -- start 2024-05-09T00:53:24.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.142+0000 7faee990c700 1 -- start start 2024-05-09T00:53:24.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.143+0000 7faee990c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faee4103080 0x7faee41a1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:24.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.143+0000 7faee990c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faee41039d0 0x7faee41a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:24.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.143+0000 7faee990c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faee41084d0 0x7faee41a5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:24.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.143+0000 7faee990c700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7faee411a920 con 0x7faee41084d0 2024-05-09T00:53:24.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.143+0000 7faee990c700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7faee411a7a0 con 0x7faee4103080 2024-05-09T00:53:24.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.143+0000 7faee990c700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7faee411aaa0 con 0x7faee41039d0 2024-05-09T00:53:24.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.144+0000 7faee910b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faee41084d0 0x7faee41a5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:24.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.144+0000 7faee910b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faee41084d0 0x7faee41a5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54406/0 (socket says 172.21.15.5:54406) 2024-05-09T00:53:24.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.144+0000 7faee910b700 1 -- 172.21.15.5:0/552664247 learned_addr learned my addr 172.21.15.5:0/552664247 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:53:24.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.145+0000 7faee3fff700 1 --2- 172.21.15.5:0/552664247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faee41039d0 0x7faee41a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:24.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.145+0000 7faee890a700 1 --2- 172.21.15.5:0/552664247 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faee4103080 0x7faee41a1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:24.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.148+0000 7faee3fff700 1 -- 172.21.15.5:0/552664247 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faee4103080 msgr2=0x7faee41a1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:53:24.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.148+0000 7faee3fff700 1 --2- 172.21.15.5:0/552664247 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faee4103080 0x7faee41a1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:24.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.148+0000 7faee3fff700 1 -- 172.21.15.5:0/552664247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faee41084d0 msgr2=0x7faee41a5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:53:24.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.148+0000 7faee3fff700 1 --2- 172.21.15.5:0/552664247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faee41084d0 0x7faee41a5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:24.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.148+0000 7faee3fff700 1 -- 172.21.15.5:0/552664247 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faee41a61e0 con 0x7faee41039d0 2024-05-09T00:53:24.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.149+0000 7faee3fff700 1 --2- 172.21.15.5:0/552664247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faee41039d0 0x7faee41a18b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7faed400bd70 tx=0x7faed400bda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:53:24.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.151+0000 7faee1ffb700 1 -- 172.21.15.5:0/552664247 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed4019070 con 0x7faee41039d0 2024-05-09T00:53:24.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.151+0000 7faee1ffb700 1 -- 172.21.15.5:0/552664247 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faed400cc90 con 0x7faee41039d0 2024-05-09T00:53:24.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.151+0000 7faee1ffb700 1 -- 172.21.15.5:0/552664247 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed40125e0 con 0x7faee41039d0 2024-05-09T00:53:24.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.151+0000 7faee990c700 1 -- 172.21.15.5:0/552664247 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faee41a64d0 con 0x7faee41039d0 2024-05-09T00:53:24.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.151+0000 7faee990c700 1 -- 172.21.15.5:0/552664247 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faee41addf0 con 0x7faee41039d0 2024-05-09T00:53:24.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.158+0000 7faef9584b80 1 Processor -- start 2024-05-09T00:53:24.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.158+0000 7faef9584b80 1 -- start start 2024-05-09T00:53:24.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.158+0000 7faef9584b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5647e225b160 0x5647e22edcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:24.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.158+0000 7faef9584b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5647e21714a0 0x5647e22ee230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:24.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.158+0000 7faef9584b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647e2309a10 0x5647e230be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:24.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.158+0000 7faef9584b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5647e2210ff0 con 0x5647e2309a10 2024-05-09T00:53:24.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.158+0000 7faef9584b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5647e21af7f0 con 0x5647e21714a0 2024-05-09T00:53:24.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.158+0000 7faef9584b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5647e220cfb0 con 0x5647e225b160 2024-05-09T00:53:24.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.159+0000 7faee910b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5647e225b160 0x5647e22edcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:24.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.163+0000 7faee910b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5647e225b160 0x5647e22edcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:60834/0 (socket says 172.21.15.5:60834) 2024-05-09T00:53:24.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.163+0000 7faee910b700 1 -- 172.21.15.5:0/4282287691 learned_addr learned my addr 172.21.15.5:0/4282287691 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:53:24.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.165+0000 7faee910b700 1 -- 172.21.15.5:0/4282287691 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5647e21714a0 msgr2=0x5647e22ee230 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:53:24.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.165+0000 7faee910b700 1 --2- 172.21.15.5:0/4282287691 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5647e21714a0 0x5647e22ee230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:24.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.165+0000 7faee910b700 1 -- 172.21.15.5:0/4282287691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647e2309a10 msgr2=0x5647e230be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:53:24.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.165+0000 7faee910b700 1 --2- 172.21.15.5:0/4282287691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647e2309a10 0x5647e230be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:24.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.165+0000 7faee910b700 1 -- 172.21.15.5:0/4282287691 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647e21a7370 con 0x5647e225b160 2024-05-09T00:53:24.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.167+0000 7faee1ffb700 1 -- 172.21.15.5:0/552664247 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7faed401c070 con 0x7faee41039d0 2024-05-09T00:53:24.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.169+0000 7faee1ffb700 1 --2- 172.21.15.5:0/552664247 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7faed00b2a90 0x7faed00b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:24.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.169+0000 7faee1ffb700 1 -- 172.21.15.5:0/552664247 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7faed40e39d0 con 0x7faee41039d0 2024-05-09T00:53:24.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.169+0000 7faee910b700 1 --2- 172.21.15.5:0/4282287691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5647e225b160 0x5647e22edcf0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7faecc005950 tx=0x7faecc0096d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40ef91555ca04e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:53:24.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.171+0000 7faee890a700 1 --2- 172.21.15.5:0/552664247 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7faed00b2a90 0x7faed00b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:24.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.171+0000 7faeda7fc700 1 -- 172.21.15.5:0/4282287691 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faecc028070 con 0x5647e225b160 2024-05-09T00:53:24.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.172+0000 7faeda7fc700 1 -- 172.21.15.5:0/4282287691 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faecc005040 con 0x5647e225b160 2024-05-09T00:53:24.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.172+0000 7faee890a700 1 --2- 172.21.15.5:0/552664247 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7faed00b2a90 0x7faed00b4fa0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x5647e22eb330 tx=0x7faedc00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:53:24.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.172+0000 7faeda7fc700 1 -- 172.21.15.5:0/4282287691 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faecc00bac0 con 0x5647e225b160 2024-05-09T00:53:24.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.174+0000 7faef9584b80 1 -- 172.21.15.5:0/4282287691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5647e225b160 msgr2=0x5647e22edcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:53:24.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.174+0000 7faef9584b80 1 --2- 172.21.15.5:0/4282287691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5647e225b160 0x5647e22edcf0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7faecc005950 tx=0x7faecc0096d0 comp rx=0 tx=0).stop 2024-05-09T00:53:24.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.175+0000 7faef9584b80 1 -- 172.21.15.5:0/4282287691 shutdown_connections 2024-05-09T00:53:24.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.175+0000 7faef9584b80 1 --2- 172.21.15.5:0/4282287691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647e2309a10 0x5647e230be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:24.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.175+0000 7faef9584b80 1 --2- 172.21.15.5:0/4282287691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5647e225b160 0x5647e22edcf0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:24.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.175+0000 7faef9584b80 1 --2- 172.21.15.5:0/4282287691 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5647e21714a0 0x5647e22ee230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:24.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.175+0000 7faef9584b80 1 -- 172.21.15.5:0/4282287691 >> 172.21.15.5:0/4282287691 conn(0x5647e21d16a0 msgr2=0x5647e22e9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:53:24.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.176+0000 7faef9584b80 1 -- 172.21.15.5:0/4282287691 shutdown_connections 2024-05-09T00:53:24.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.176+0000 7faef9584b80 1 -- 172.21.15.5:0/4282287691 wait complete. 2024-05-09T00:53:24.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.178+0000 7faef9584b80 1 -- start start 2024-05-09T00:53:24.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.178+0000 7faef9584b80 1 Processor -- start 2024-05-09T00:53:24.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.179+0000 7faef9584b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647e21714a0 0x5647e23068e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:24.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.180+0000 7faef9584b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5647e225b160 0x5647e2306e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:24.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.180+0000 7faef9584b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5647e2309a10 0x5647e2307360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:24.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.180+0000 7faef9584b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5647e208cba0 con 0x5647e21714a0 2024-05-09T00:53:24.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.181+0000 7faef9584b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5647e21cb150 con 0x5647e225b160 2024-05-09T00:53:24.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.181+0000 7faef9584b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5647e230c460 con 0x5647e2309a10 2024-05-09T00:53:24.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.181+0000 7faee3fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5647e225b160 0x5647e2306e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:24.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.181+0000 7faee3fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5647e225b160 0x5647e2306e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:37080/0 (socket says 172.21.15.5:37080) 2024-05-09T00:53:24.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.181+0000 7faee3fff700 1 -- 172.21.15.5:0/1003859712 learned_addr learned my addr 172.21.15.5:0/1003859712 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:53:24.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.181+0000 7faef9584b80 10 client.0 _ll_register_callbacks cb 0x5647e20826c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:53:24.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.181+0000 7faee910b700 1 --2- 172.21.15.5:0/1003859712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5647e2309a10 0x5647e2307360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:24.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.182+0000 7faee890a700 1 --2- 172.21.15.5:0/1003859712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647e21714a0 0x5647e23068e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:24.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.182+0000 7faee3fff700 1 -- 172.21.15.5:0/1003859712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5647e2309a10 msgr2=0x5647e2307360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:53:24.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.182+0000 7faee3fff700 1 --2- 172.21.15.5:0/1003859712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5647e2309a10 0x5647e2307360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:24.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.182+0000 7faee3fff700 1 -- 172.21.15.5:0/1003859712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647e21714a0 msgr2=0x5647e23068e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:53:24.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.182+0000 7faee3fff700 1 --2- 172.21.15.5:0/1003859712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647e21714a0 0x5647e23068e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:24.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.182+0000 7faee3fff700 1 -- 172.21.15.5:0/1003859712 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647e208c490 con 0x5647e225b160 2024-05-09T00:53:24.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.182+0000 7faee910b700 1 --2- 172.21.15.5:0/1003859712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5647e2309a10 0x5647e2307360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:53:24.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.185+0000 7faee3fff700 1 --2- 172.21.15.5:0/1003859712 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5647e225b160 0x5647e2306e20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7faed40aacf0 tx=0x7faed4014c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa61ded9d5567354 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:53:24.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.189+0000 7faec3fff700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-09T00:53:24.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.189+0000 7faec3fff700 1 -- 172.21.15.5:0/1003859712 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed40ab6f0 con 0x5647e225b160 2024-05-09T00:53:24.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.189+0000 7faec3fff700 1 -- 172.21.15.5:0/1003859712 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7faed40b3b40 con 0x5647e225b160 2024-05-09T00:53:24.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.192+0000 7faef9584b80 1 -- 172.21.15.5:0/1003859712 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faecc005040 con 0x5647e225b160 2024-05-09T00:53:24.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.194+0000 7faec3fff700 1 -- 172.21.15.5:0/1003859712 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7faed40aaff0 con 0x5647e225b160 2024-05-09T00:53:24.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.198+0000 7faef9584b80 10 client.25763 fetch_fsmap learned FSMap version 31 2024-05-09T00:53:24.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.198+0000 7faef9584b80 1 -- 172.21.15.5:0/1003859712 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5647e2316ea0 con 0x5647e225b160 2024-05-09T00:53:24.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.203+0000 7faec3fff700 1 -- 172.21.15.5:0/1003859712 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faed40af3a0 con 0x5647e225b160 2024-05-09T00:53:24.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.203+0000 7faec3fff700 1 -- 172.21.15.5:0/1003859712 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7faed40a6b00 con 0x5647e225b160 2024-05-09T00:53:24.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.203+0000 7faef9584b80 10 client.25763 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:53:24.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.203+0000 7faef9584b80 10 client.25763 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:53:24.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.203+0000 7faef9584b80 20 client.25763 populate_metadata read hostname 'smithi005' 2024-05-09T00:53:24.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.203+0000 7faef9584b80 1 --2- 172.21.15.5:0/1003859712 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5647e2319430 0x5647e231b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:24.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.204+0000 7faef9584b80 4 client.25763 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:53:24.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.204+0000 7faef9584b80 1 -- 172.21.15.5:0/1003859712 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7faecc005040 con 0x5647e2319430 2024-05-09T00:53:24.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.204+0000 7faee890a700 1 --2- 172.21.15.5:0/1003859712 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5647e2319430 0x5647e231b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:24.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.206+0000 7faec3fff700 10 client.25763 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:53:24.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.206+0000 7faee890a700 1 --2- 172.21.15.5:0/1003859712 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5647e2319430 0x5647e231b850 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89b580242f0bcf34 server_cookie=9bd7a09363825089 in_seq=0 out_seq=0 2024-05-09T00:53:24.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.208+0000 7faec3fff700 1 -- 172.21.15.5:0/1003859712 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7faecc005040 con 0x5647e2319430 2024-05-09T00:53:24.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.208+0000 7faec3fff700 10 client.25763 handle_command_reply: tid=0 2024-05-09T00:53:24.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.208+0000 7faef9584b80 1 client.25763 shutdown 2024-05-09T00:53:24.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.211+0000 7faef9584b80 1 -- 172.21.15.5:0/1003859712 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5647e225b160 msgr2=0x5647e2306e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:53:24.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.211+0000 7faef9584b80 1 --2- 172.21.15.5:0/1003859712 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5647e225b160 0x5647e2306e20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7faed40aacf0 tx=0x7faed4014c80 comp rx=0 tx=0).stop 2024-05-09T00:53:24.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.212+0000 7faef9584b80 1 -- 172.21.15.5:0/1003859712 shutdown_connections 2024-05-09T00:53:24.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.212+0000 7faef9584b80 1 --2- 172.21.15.5:0/1003859712 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5647e2319430 0x5647e231b850 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:24.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.213+0000 7faef9584b80 1 --2- 172.21.15.5:0/1003859712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5647e2309a10 0x5647e2307360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:24.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.213+0000 7faef9584b80 1 --2- 172.21.15.5:0/1003859712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647e21714a0 0x5647e23068e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:24.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.213+0000 7faef9584b80 1 --2- 172.21.15.5:0/1003859712 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5647e225b160 0x5647e2306e20 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:24.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.213+0000 7faef9584b80 1 -- 172.21.15.5:0/1003859712 >> 172.21.15.5:0/1003859712 conn(0x5647e21d16a0 msgr2=0x5647e22e98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:53:24.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.213+0000 7faec3fff700 0 client.25763 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:53:24.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.214+0000 7faef9584b80 1 -- 172.21.15.5:0/1003859712 shutdown_connections 2024-05-09T00:53:24.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.214+0000 7faef9584b80 1 -- 172.21.15.5:0/1003859712 wait complete. 2024-05-09T00:53:24.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.215+0000 7faef9584b80 20 client.25763 trim_cache size 0 max 16384 2024-05-09T00:53:24.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.248+0000 7faef9584b80 1 Processor -- start 2024-05-09T00:53:24.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.248+0000 7faef9584b80 1 -- start start 2024-05-09T00:53:24.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.249+0000 7faef9584b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5647e21714a0 0x5647e23020c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:24.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.249+0000 7faef9584b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647e225b160 0x5647e23153a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:24.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.250+0000 7faef9584b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5647e2309a10 0x5647e2320fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:24.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.250+0000 7faef9584b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7faee4111740 con 0x5647e225b160 2024-05-09T00:53:24.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.251+0000 7faef9584b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7faee41ae3f0 con 0x5647e2309a10 2024-05-09T00:53:24.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.251+0000 7faef9584b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7faee4111e10 con 0x5647e21714a0 2024-05-09T00:53:24.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.252+0000 7faee910b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5647e2309a10 0x5647e2320fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:24.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.252+0000 7faee910b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5647e2309a10 0x5647e2320fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:37090/0 (socket says 172.21.15.5:37090) 2024-05-09T00:53:24.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.252+0000 7faee910b700 1 -- 172.21.15.5:0/3367941649 learned_addr learned my addr 172.21.15.5:0/3367941649 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:53:24.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.253+0000 7faee3fff700 1 --2- 172.21.15.5:0/3367941649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647e225b160 0x5647e23153a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:24.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.253+0000 7faee910b700 1 -- 172.21.15.5:0/3367941649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5647e21714a0 msgr2=0x5647e23020c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:53:24.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.253+0000 7faee910b700 1 --2- 172.21.15.5:0/3367941649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5647e21714a0 0x5647e23020c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:24.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.253+0000 7faee910b700 1 -- 172.21.15.5:0/3367941649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647e225b160 msgr2=0x5647e23153a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:53:24.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.254+0000 7faee910b700 1 --2- 172.21.15.5:0/3367941649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647e225b160 0x5647e23153a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:24.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.254+0000 7faee910b700 1 -- 172.21.15.5:0/3367941649 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647e2287810 con 0x5647e2309a10 2024-05-09T00:53:24.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.255+0000 7faee910b700 1 --2- 172.21.15.5:0/3367941649 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5647e2309a10 0x5647e2320fa0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7faecc026040 tx=0x7faecc004cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd3365e53e03583a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:53:24.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.256+0000 7faed9ffb700 1 -- 172.21.15.5:0/3367941649 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faecc032070 con 0x5647e2309a10 2024-05-09T00:53:24.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.256+0000 7faed9ffb700 1 -- 172.21.15.5:0/3367941649 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faecc00bc80 con 0x5647e2309a10 2024-05-09T00:53:24.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.256+0000 7faed9ffb700 1 -- 172.21.15.5:0/3367941649 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faecc028070 con 0x5647e2309a10 2024-05-09T00:53:24.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.258+0000 7faef9584b80 1 -- 172.21.15.5:0/3367941649 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5647e2309a10 msgr2=0x5647e2320fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:53:24.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.258+0000 7faef9584b80 1 --2- 172.21.15.5:0/3367941649 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5647e2309a10 0x5647e2320fa0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7faecc026040 tx=0x7faecc004cd0 comp rx=0 tx=0).stop 2024-05-09T00:53:24.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.259+0000 7faef9584b80 1 -- 172.21.15.5:0/3367941649 shutdown_connections 2024-05-09T00:53:24.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.259+0000 7faef9584b80 1 --2- 172.21.15.5:0/3367941649 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5647e2309a10 0x5647e2320fa0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:24.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.259+0000 7faef9584b80 1 --2- 172.21.15.5:0/3367941649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5647e21714a0 0x5647e23020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:24.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.259+0000 7faef9584b80 1 --2- 172.21.15.5:0/3367941649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647e225b160 0x5647e23153a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:24.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.259+0000 7faef9584b80 1 -- 172.21.15.5:0/3367941649 >> 172.21.15.5:0/3367941649 conn(0x5647e21d16a0 msgr2=0x5647e23019b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:53:24.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.259+0000 7faef9584b80 1 -- 172.21.15.5:0/3367941649 shutdown_connections 2024-05-09T00:53:24.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.259+0000 7faef9584b80 1 -- 172.21.15.5:0/3367941649 wait complete. 2024-05-09T00:53:24.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.260+0000 7faef9584b80 1 -- start start 2024-05-09T00:53:24.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.260+0000 7faef9584b80 1 Processor -- start 2024-05-09T00:53:24.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.261+0000 7faef9584b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647e21714a0 0x5647e2309e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:24.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.261+0000 7faef9584b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5647e225b160 0x5647e230a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:24.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.262+0000 7faef9584b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5647e2309a10 0x5647e230a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:24.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.262+0000 7faef9584b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7faee4112bb0 con 0x5647e21714a0 2024-05-09T00:53:24.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.262+0000 7faef9584b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7faee41124e0 con 0x5647e2309a10 2024-05-09T00:53:24.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.262+0000 7faef9584b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7faee406bee0 con 0x5647e225b160 2024-05-09T00:53:24.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.262+0000 7faef9584b80 10 client.0 _ll_register_callbacks cb 0x5647e20826c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:53:24.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.262+0000 7faee890a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647e21714a0 0x5647e2309e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:24.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.262+0000 7faee890a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647e21714a0 0x5647e2309e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54460/0 (socket says 172.21.15.5:54460) 2024-05-09T00:53:24.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.262+0000 7faee890a700 1 -- 172.21.15.5:0/2950095010 learned_addr learned my addr 172.21.15.5:0/2950095010 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:53:24.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.263+0000 7faee910b700 1 --2- 172.21.15.5:0/2950095010 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5647e2309a10 0x5647e230a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:24.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.264+0000 7faee910b700 1 -- 172.21.15.5:0/2950095010 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5647e225b160 msgr2=0x5647e230a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:53:24.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.264+0000 7faee910b700 1 --2- 172.21.15.5:0/2950095010 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5647e225b160 0x5647e230a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:24.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.264+0000 7faee910b700 1 -- 172.21.15.5:0/2950095010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647e21714a0 msgr2=0x5647e2309e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:53:24.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.264+0000 7faee910b700 1 --2- 172.21.15.5:0/2950095010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647e21714a0 0x5647e2309e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:24.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.264+0000 7faee910b700 1 -- 172.21.15.5:0/2950095010 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647e2310720 con 0x5647e2309a10 2024-05-09T00:53:24.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.269+0000 7faee910b700 1 --2- 172.21.15.5:0/2950095010 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5647e2309a10 0x5647e230a8a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x5647e2309840 tx=0x7faecc00b7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4edf66fb1e478d0e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:53:24.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.269+0000 7faec3fff700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-09T00:53:24.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.270+0000 7faec3fff700 1 -- 172.21.15.5:0/2950095010 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faecc032070 con 0x5647e2309a10 2024-05-09T00:53:24.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.270+0000 7faec3fff700 1 -- 172.21.15.5:0/2950095010 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7faecc005410 con 0x5647e2309a10 2024-05-09T00:53:24.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.271+0000 7faef9584b80 1 -- 172.21.15.5:0/2950095010 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faecc00bc80 con 0x5647e2309a10 2024-05-09T00:53:24.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.272+0000 7faec3fff700 1 -- 172.21.15.5:0/2950095010 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7faecc028070 con 0x5647e2309a10 2024-05-09T00:53:24.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.272+0000 7faef9584b80 10 client.25769 fetch_fsmap learned FSMap version 31 2024-05-09T00:53:24.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.273+0000 7faef9584b80 1 -- 172.21.15.5:0/2950095010 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5647e2311980 con 0x5647e2309a10 2024-05-09T00:53:24.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.277+0000 7faec3fff700 1 -- 172.21.15.5:0/2950095010 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faecc005610 con 0x5647e2309a10 2024-05-09T00:53:24.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.277+0000 7faec3fff700 1 -- 172.21.15.5:0/2950095010 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7faecc00f690 con 0x5647e2309a10 2024-05-09T00:53:24.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.277+0000 7faef9584b80 10 client.25769 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:53:24.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.278+0000 7faef9584b80 10 client.25769 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:53:24.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.278+0000 7faef9584b80 20 client.25769 populate_metadata read hostname 'smithi005' 2024-05-09T00:53:24.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.278+0000 7faef9584b80 1 --2- 172.21.15.5:0/2950095010 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5647e2311bb0 0x5647e2385b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:24.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.278+0000 7faef9584b80 4 client.25769 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:53:24.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.278+0000 7faef9584b80 1 -- 172.21.15.5:0/2950095010 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7faecc00bc80 con 0x5647e2311bb0 2024-05-09T00:53:24.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.283+0000 7faee890a700 1 --2- 172.21.15.5:0/2950095010 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5647e2311bb0 0x5647e2385b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:24.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.289+0000 7faec3fff700 10 client.25769 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:53:24.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.289+0000 7faee890a700 1 --2- 172.21.15.5:0/2950095010 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5647e2311bb0 0x5647e2385b60 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c13e67064ac53071 server_cookie=5b5eed27fccd69e2 in_seq=0 out_seq=0 2024-05-09T00:53:24.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.293+0000 7faec3fff700 1 -- 172.21.15.5:0/2950095010 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7faecc00bc80 con 0x5647e2311bb0 2024-05-09T00:53:24.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.293+0000 7faec3fff700 10 client.25769 handle_command_reply: tid=0 2024-05-09T00:53:24.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.293+0000 7faef9584b80 1 client.25769 shutdown 2024-05-09T00:53:24.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.293+0000 7faef9584b80 1 -- 172.21.15.5:0/2950095010 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5647e2309a10 msgr2=0x5647e230a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:53:24.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.293+0000 7faef9584b80 1 --2- 172.21.15.5:0/2950095010 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5647e2309a10 0x5647e230a8a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x5647e2309840 tx=0x7faecc00b7d0 comp rx=0 tx=0).stop 2024-05-09T00:53:24.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.294+0000 7faef9584b80 1 -- 172.21.15.5:0/2950095010 shutdown_connections 2024-05-09T00:53:24.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.294+0000 7faef9584b80 1 --2- 172.21.15.5:0/2950095010 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5647e2311bb0 0x5647e2385b60 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:24.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.294+0000 7faec3fff700 0 client.25769 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:53:24.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.294+0000 7faef9584b80 1 --2- 172.21.15.5:0/2950095010 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5647e2309a10 0x5647e230a8a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:24.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.294+0000 7faef9584b80 1 --2- 172.21.15.5:0/2950095010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647e21714a0 0x5647e2309e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:24.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.294+0000 7faef9584b80 1 --2- 172.21.15.5:0/2950095010 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5647e225b160 0x5647e230a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:24.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.294+0000 7faef9584b80 1 -- 172.21.15.5:0/2950095010 >> 172.21.15.5:0/2950095010 conn(0x5647e21d16a0 msgr2=0x5647e22ef7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:53:24.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.294+0000 7faef9584b80 1 -- 172.21.15.5:0/2950095010 shutdown_connections 2024-05-09T00:53:24.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.295+0000 7faef9584b80 1 -- 172.21.15.5:0/2950095010 wait complete. 2024-05-09T00:53:24.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.295+0000 7faef9584b80 20 client.25769 trim_cache size 0 max 16384 2024-05-09T00:53:24.297 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-09T00:53:24.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.297+0000 7faedb7fe700 1 -- 172.21.15.5:0/552664247 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7faed00b2a90 msgr2=0x7faed00b4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:53:24.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.297+0000 7faedb7fe700 1 --2- 172.21.15.5:0/552664247 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7faed00b2a90 0x7faed00b4fa0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x5647e22eb330 tx=0x7faedc00a300 comp rx=0 tx=0).stop 2024-05-09T00:53:24.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.297+0000 7faedb7fe700 1 -- 172.21.15.5:0/552664247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faee41039d0 msgr2=0x7faee41a18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:53:24.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.297+0000 7faedb7fe700 1 --2- 172.21.15.5:0/552664247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faee41039d0 0x7faee41a18b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7faed400bd70 tx=0x7faed400bda0 comp rx=0 tx=0).stop 2024-05-09T00:53:24.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.297+0000 7faedb7fe700 1 -- 172.21.15.5:0/552664247 shutdown_connections 2024-05-09T00:53:24.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.297+0000 7faedb7fe700 1 --2- 172.21.15.5:0/552664247 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7faed00b2a90 0x7faed00b4fa0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:24.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.297+0000 7faedb7fe700 1 --2- 172.21.15.5:0/552664247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faee41084d0 0x7faee41a5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:24.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.297+0000 7faedb7fe700 1 --2- 172.21.15.5:0/552664247 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faee4103080 0x7faee41a1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:24.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.297+0000 7faedb7fe700 1 --2- 172.21.15.5:0/552664247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faee41039d0 0x7faee41a18b0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:24.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.297+0000 7faedb7fe700 1 -- 172.21.15.5:0/552664247 >> 172.21.15.5:0/552664247 conn(0x7faee4078ac0 msgr2=0x7faee4105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:53:24.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.298+0000 7faedb7fe700 1 -- 172.21.15.5:0/552664247 shutdown_connections 2024-05-09T00:53:24.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:24.298+0000 7faedb7fe700 1 -- 172.21.15.5:0/552664247 wait complete. 2024-05-09T00:53:25.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:53:25 smithi005 ceph-mon[30365]: pgmap v583: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 882 MiB/s wr, 2.26k op/s 2024-05-09T00:53:25.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:53:25 smithi131 ceph-mon[34547]: pgmap v583: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 882 MiB/s wr, 2.26k op/s 2024-05-09T00:53:25.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:53:25 smithi157 ceph-mon[34294]: pgmap v583: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 882 MiB/s wr, 2.26k op/s 2024-05-09T00:53:26.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:26.672+0000 7f5842ffd640 1 -- 172.21.15.5:0/2245406626 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5623b6560350 con 0x5623b66f90f0 2024-05-09T00:53:26.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:26.672+0000 7f5842ffd640 10 client.25757 handle_command_reply: tid=0 2024-05-09T00:53:26.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:26.672+0000 7f58709f6740 1 client.25757 shutdown 2024-05-09T00:53:26.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:26.673+0000 7f58709f6740 1 -- 172.21.15.5:0/2245406626 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623b62ac2f0 msgr2=0x5623b6686f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:53:26.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:26.673+0000 7f58709f6740 1 --2- 172.21.15.5:0/2245406626 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623b62ac2f0 0x5623b6686f00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f58540a74a0 tx=0x7f58540a9130 comp rx=0 tx=0).stop 2024-05-09T00:53:26.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:26.675+0000 7f58709f6740 1 -- 172.21.15.5:0/2245406626 shutdown_connections 2024-05-09T00:53:26.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:26.675+0000 7f58709f6740 1 --2- 172.21.15.5:0/2245406626 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5623b66f90f0 0x5623b66fb550 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:26.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:26.675+0000 7f58709f6740 1 --2- 172.21.15.5:0/2245406626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623b62691c0 0x5623b6688320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:26.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:26.675+0000 7f58709f6740 1 --2- 172.21.15.5:0/2245406626 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623b62d51b0 0x5623b6687440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:26.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:26.675+0000 7f58709f6740 1 --2- 172.21.15.5:0/2245406626 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5623b62ac2f0 0x5623b6686f00 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:26.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:26.675+0000 7f5842ffd640 0 client.25757 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:53:26.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:26.675+0000 7f58709f6740 1 -- 172.21.15.5:0/2245406626 >> 172.21.15.5:0/2245406626 conn(0x5623b62c9300 msgr2=0x5623b6673700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:53:26.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:26.676+0000 7f58709f6740 1 -- 172.21.15.5:0/2245406626 shutdown_connections 2024-05-09T00:53:26.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:26.676+0000 7f58709f6740 1 -- 172.21.15.5:0/2245406626 wait complete. 2024-05-09T00:53:26.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:26.676+0000 7f58709f6740 20 client.25757 trim_cache size 0 max 16384 2024-05-09T00:53:26.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-09T00:53:26.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-09T00:53:26.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-09T00:53:26.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:26.682+0000 7f5869b3f640 1 -- 172.21.15.5:0/1626436755 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f584c0b27e0 msgr2=0x7f584c0b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:53:26.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:26.682+0000 7f5869b3f640 1 --2- 172.21.15.5:0/1626436755 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f584c0b27e0 0x7f584c0b4cf0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5858005d90 tx=0x7f5858031400 comp rx=0 tx=0).stop 2024-05-09T00:53:26.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:26.683+0000 7f5869b3f640 1 -- 172.21.15.5:0/1626436755 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5864102b50 msgr2=0x7f586410d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:53:26.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:26.683+0000 7f5869b3f640 1 --2- 172.21.15.5:0/1626436755 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5864102b50 0x7f586410d220 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f585400d740 tx=0x7f585400dc00 comp rx=0 tx=0).stop 2024-05-09T00:53:26.688 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:26.687+0000 7f5869b3f640 1 -- 172.21.15.5:0/1626436755 shutdown_connections 2024-05-09T00:53:26.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:26.687+0000 7f5869b3f640 1 --2- 172.21.15.5:0/1626436755 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f584c0b27e0 0x7f584c0b4cf0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:26.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:26.687+0000 7f5869b3f640 1 --2- 172.21.15.5:0/1626436755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5864108a90 0x7f586410be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:26.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:26.687+0000 7f5869b3f640 1 --2- 172.21.15.5:0/1626436755 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5864103d90 0x7f586410b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:26.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:26.687+0000 7f5869b3f640 1 --2- 172.21.15.5:0/1626436755 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5864102b50 0x7f586410d220 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:26.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:26.687+0000 7f5869b3f640 1 -- 172.21.15.5:0/1626436755 >> 172.21.15.5:0/1626436755 conn(0x7f58640fdd40 msgr2=0x7f5864104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:53:26.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:26.689+0000 7f5869b3f640 1 -- 172.21.15.5:0/1626436755 shutdown_connections 2024-05-09T00:53:26.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:26.689+0000 7f5869b3f640 1 -- 172.21.15.5:0/1626436755 wait complete. 2024-05-09T00:53:27.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:53:27 smithi157 ceph-mon[34294]: pgmap v584: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 900 MiB/s wr, 2.22k op/s 2024-05-09T00:53:27.943 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T00:53:27.944 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T00:53:27.991 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:53:27 smithi005 ceph-mon[30365]: pgmap v584: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 900 MiB/s wr, 2.22k op/s 2024-05-09T00:53:28.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:53:27 smithi131 ceph-mon[34547]: pgmap v584: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 900 MiB/s wr, 2.22k op/s 2024-05-09T00:53:28.792 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:53:29.770 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:53:29 smithi005 ceph-mon[30365]: pgmap v585: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 820 MiB/s wr, 1.75k op/s 2024-05-09T00:53:29.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:53:29 smithi157 ceph-mon[34294]: pgmap v585: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 820 MiB/s wr, 1.75k op/s 2024-05-09T00:53:30.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:53:29 smithi131 ceph-mon[34547]: pgmap v585: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 820 MiB/s wr, 1.75k op/s 2024-05-09T00:53:30.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:53:30 smithi157 ceph-mon[34294]: pgmap v586: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 991 MiB/s wr, 1.87k op/s 2024-05-09T00:53:31.040 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:53:30 smithi005 ceph-mon[30365]: pgmap v586: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 991 MiB/s wr, 1.87k op/s 2024-05-09T00:53:31.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:53:30 smithi131 ceph-mon[34547]: pgmap v586: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 991 MiB/s wr, 1.87k op/s 2024-05-09T00:53:33.543 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:53:33 smithi005 ceph-mon[30365]: pgmap v587: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 878 MiB/s wr, 1.69k op/s 2024-05-09T00:53:33.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:53:33 smithi131 ceph-mon[34547]: pgmap v587: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 878 MiB/s wr, 1.69k op/s 2024-05-09T00:53:33.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:53:33 smithi157 ceph-mon[34294]: pgmap v587: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 878 MiB/s wr, 1.69k op/s 2024-05-09T00:53:33.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.685+0000 7f5489130700 1 -- 172.21.15.5:0/52977550 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5480003c00 con 0x7f547c0b8320 2024-05-09T00:53:33.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.686+0000 7f548b134700 1 -- 172.21.15.5:0/52977550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f547c0b8320 msgr2=0x7f547c0ba740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:53:33.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.686+0000 7f548b134700 1 --2- 172.21.15.5:0/52977550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f547c0b8320 0x7f547c0ba740 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f5480009990 tx=0x7f5480009ca0 comp rx=0 tx=0).stop 2024-05-09T00:53:33.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.686+0000 7f548b134700 1 -- 172.21.15.5:0/52977550 shutdown_connections 2024-05-09T00:53:33.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.686+0000 7f548b134700 1 --2- 172.21.15.5:0/52977550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f547c0b8320 0x7f547c0ba740 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:33.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.686+0000 7f548b134700 1 --2- 172.21.15.5:0/52977550 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f547c0a53c0 0x7f547c0a57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:33.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.686+0000 7f548b134700 1 --2- 172.21.15.5:0/52977550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f547c0a5d10 0x7f547c0b7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:33.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.686+0000 7f548b134700 1 -- 172.21.15.5:0/52977550 >> 172.21.15.5:0/52977550 conn(0x7f547c01af20 msgr2=0x7f547c01b360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:53:33.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.688+0000 7f548b134700 1 -- 172.21.15.5:0/52977550 shutdown_connections 2024-05-09T00:53:33.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.688+0000 7f548b134700 1 -- 172.21.15.5:0/52977550 wait complete. 2024-05-09T00:53:33.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.689+0000 7f548b134700 1 Processor -- start 2024-05-09T00:53:33.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.689+0000 7f548b134700 1 -- start start 2024-05-09T00:53:33.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.690+0000 7f548b134700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f547c0a53c0 0x7f547c144680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:33.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.690+0000 7f548b134700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f547c0a5d10 0x7f547c144c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:33.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.690+0000 7f548b134700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f547c0b8320 0x7f547c148eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:33.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.690+0000 7f548b134700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f547c0bdda0 con 0x7f547c0b8320 2024-05-09T00:53:33.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.690+0000 7f548b134700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f547c0bdc20 con 0x7f547c0a5d10 2024-05-09T00:53:33.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.690+0000 7f548b134700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f547c0bdf20 con 0x7f547c0a53c0 2024-05-09T00:53:33.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.691+0000 7f5489931700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f547c0a5d10 0x7f547c144c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:33.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.691+0000 7f548a933700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f547c0b8320 0x7f547c148eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:33.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.691+0000 7f5489931700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f547c0a5d10 0x7f547c144c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:38354/0 (socket says 172.21.15.5:38354) 2024-05-09T00:53:33.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.691+0000 7f5489931700 1 -- 172.21.15.5:0/2583898608 learned_addr learned my addr 172.21.15.5:0/2583898608 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:53:33.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.691+0000 7f548a132700 1 --2- 172.21.15.5:0/2583898608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f547c0a53c0 0x7f547c144680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:33.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.691+0000 7f5489931700 1 -- 172.21.15.5:0/2583898608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f547c0a53c0 msgr2=0x7f547c144680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:53:33.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.691+0000 7f5489931700 1 --2- 172.21.15.5:0/2583898608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f547c0a53c0 0x7f547c144680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:33.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.691+0000 7f5489931700 1 -- 172.21.15.5:0/2583898608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f547c0b8320 msgr2=0x7f547c148eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:53:33.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.691+0000 7f5489931700 1 --2- 172.21.15.5:0/2583898608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f547c0b8320 0x7f547c148eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:33.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.691+0000 7f5489931700 1 -- 172.21.15.5:0/2583898608 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f547c149550 con 0x7f547c0a5d10 2024-05-09T00:53:33.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.692+0000 7f5489931700 1 --2- 172.21.15.5:0/2583898608 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f547c0a5d10 0x7f547c144c20 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f547400d980 tx=0x7f547400dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:53:33.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.692+0000 7f547b7fe700 1 -- 172.21.15.5:0/2583898608 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5474017070 con 0x7f547c0a5d10 2024-05-09T00:53:33.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.693+0000 7f547b7fe700 1 -- 172.21.15.5:0/2583898608 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5474009de0 con 0x7f547c0a5d10 2024-05-09T00:53:33.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.693+0000 7f548b134700 1 -- 172.21.15.5:0/2583898608 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f547c149840 con 0x7f547c0a5d10 2024-05-09T00:53:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.693+0000 7f547b7fe700 1 -- 172.21.15.5:0/2583898608 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f547400f3c0 con 0x7f547c0a5d10 2024-05-09T00:53:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.693+0000 7f548b134700 1 -- 172.21.15.5:0/2583898608 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f547c151160 con 0x7f547c0a5d10 2024-05-09T00:53:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.700+0000 7f549adacb80 1 Processor -- start 2024-05-09T00:53:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.701+0000 7f549adacb80 1 -- start start 2024-05-09T00:53:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.702+0000 7f549adacb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556f4e08c1e0 0x556f4e1256c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.702+0000 7f549adacb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f4dfa2520 0x556f4e125c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.702+0000 7f549adacb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f4e11f880 0x556f4e11fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.702+0000 7f549adacb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556f4e042070 con 0x556f4e11f880 2024-05-09T00:53:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.703+0000 7f549adacb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x556f4dfe0870 con 0x556f4dfa2520 2024-05-09T00:53:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.703+0000 7f547b7fe700 1 -- 172.21.15.5:0/2583898608 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f547400f560 con 0x7f547c0a5d10 2024-05-09T00:53:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.703+0000 7f549adacb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x556f4e03e030 con 0x556f4e08c1e0 2024-05-09T00:53:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.703+0000 7f5489931700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f4e11f880 0x556f4e11fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.703+0000 7f548a132700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f4dfa2520 0x556f4e125c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:33.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.703+0000 7f5489931700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f4e11f880 0x556f4e11fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38032/0 (socket says 172.21.15.5:38032) 2024-05-09T00:53:33.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.703+0000 7f5489931700 1 -- 172.21.15.5:0/3545510998 learned_addr learned my addr 172.21.15.5:0/3545510998 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:53:33.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.703+0000 7f5489931700 1 -- 172.21.15.5:0/3545510998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556f4e08c1e0 msgr2=0x556f4e1256c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:53:33.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.703+0000 7f5489931700 1 --2- 172.21.15.5:0/3545510998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556f4e08c1e0 0x556f4e1256c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:33.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.703+0000 7f5489931700 1 -- 172.21.15.5:0/3545510998 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f4dfa2520 msgr2=0x556f4e125c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:53:33.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.703+0000 7f5489931700 1 --2- 172.21.15.5:0/3545510998 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f4dfa2520 0x556f4e125c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:33.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.703+0000 7f5489931700 1 -- 172.21.15.5:0/3545510998 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f4dfd83f0 con 0x556f4e11f880 2024-05-09T00:53:33.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.704+0000 7f5489931700 1 --2- 172.21.15.5:0/3545510998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f4e11f880 0x556f4e11fc90 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f54740e5dc0 tx=0x7f54740ea5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aea2247c738d01cf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:53:33.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.704+0000 7f5478d47700 1 -- 172.21.15.5:0/3545510998 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54740f4600 con 0x556f4e11f880 2024-05-09T00:53:33.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.704+0000 7f5478d47700 1 -- 172.21.15.5:0/3545510998 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54740eae10 con 0x556f4e11f880 2024-05-09T00:53:33.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.704+0000 7f5478d47700 1 -- 172.21.15.5:0/3545510998 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54740ec970 con 0x556f4e11f880 2024-05-09T00:53:33.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.706+0000 7f549adacb80 1 -- 172.21.15.5:0/3545510998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f4e11f880 msgr2=0x556f4e11fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:53:33.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.706+0000 7f549adacb80 1 --2- 172.21.15.5:0/3545510998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f4e11f880 0x556f4e11fc90 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f54740e5dc0 tx=0x7f54740ea5f0 comp rx=0 tx=0).stop 2024-05-09T00:53:33.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.706+0000 7f547b7fe700 1 --2- 172.21.15.5:0/2583898608 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f54700b29c0 0x7f54700b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:33.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.706+0000 7f549adacb80 1 -- 172.21.15.5:0/3545510998 shutdown_connections 2024-05-09T00:53:33.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.706+0000 7f549adacb80 1 --2- 172.21.15.5:0/3545510998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f4e11f880 0x556f4e11fc90 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:33.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.706+0000 7f549adacb80 1 --2- 172.21.15.5:0/3545510998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556f4e08c1e0 0x556f4e1256c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:33.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.706+0000 7f549adacb80 1 --2- 172.21.15.5:0/3545510998 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f4dfa2520 0x556f4e125c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:33.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.706+0000 7f549adacb80 1 -- 172.21.15.5:0/3545510998 >> 172.21.15.5:0/3545510998 conn(0x556f4e002720 msgr2=0x556f4e11a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:53:33.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.707+0000 7f548a933700 1 --2- 172.21.15.5:0/2583898608 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f54700b29c0 0x7f54700b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:33.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.707+0000 7f549adacb80 1 -- 172.21.15.5:0/3545510998 shutdown_connections 2024-05-09T00:53:33.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.707+0000 7f549adacb80 1 -- 172.21.15.5:0/3545510998 wait complete. 2024-05-09T00:53:33.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.707+0000 7f547b7fe700 1 -- 172.21.15.5:0/2583898608 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f54740e1d00 con 0x7f547c0a5d10 2024-05-09T00:53:33.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.708+0000 7f549adacb80 1 -- start start 2024-05-09T00:53:33.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.708+0000 7f549adacb80 1 Processor -- start 2024-05-09T00:53:33.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.708+0000 7f548a933700 1 --2- 172.21.15.5:0/2583898608 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f54700b29c0 0x7f54700b4ed0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f548000b5c0 tx=0x7f5480003680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:53:33.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.709+0000 7f549adacb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f4dfa2520 0x556f4e1293f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:33.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.709+0000 7f549adacb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f4e08c1e0 0x556f4e129930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:33.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.709+0000 7f549adacb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556f4e11f880 0x556f4e129e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:33.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.709+0000 7f549adacb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556f4debdc20 con 0x556f4dfa2520 2024-05-09T00:53:33.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.709+0000 7f549adacb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x556f4dffc1d0 con 0x556f4e08c1e0 2024-05-09T00:53:33.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.709+0000 7f549adacb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x556f4e120670 con 0x556f4e11f880 2024-05-09T00:53:33.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.709+0000 7f549adacb80 10 client.0 _ll_register_callbacks cb 0x556f4deb3740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:53:33.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.709+0000 7f548a132700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f4dfa2520 0x556f4e1293f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:33.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.709+0000 7f548a132700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f4dfa2520 0x556f4e1293f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38042/0 (socket says 172.21.15.5:38042) 2024-05-09T00:53:33.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.709+0000 7f548a132700 1 -- 172.21.15.5:0/3700131166 learned_addr learned my addr 172.21.15.5:0/3700131166 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:53:33.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.709+0000 7f5489931700 1 --2- 172.21.15.5:0/3700131166 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f4e08c1e0 0x556f4e129930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:33.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.710+0000 7f548a132700 1 -- 172.21.15.5:0/3700131166 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556f4e11f880 msgr2=0x556f4e129e70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:53:33.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.710+0000 7f548a132700 1 --2- 172.21.15.5:0/3700131166 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556f4e11f880 0x556f4e129e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:33.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.710+0000 7f548a132700 1 -- 172.21.15.5:0/3700131166 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f4e08c1e0 msgr2=0x556f4e129930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:53:33.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.710+0000 7f548a132700 1 --2- 172.21.15.5:0/3700131166 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f4e08c1e0 0x556f4e129930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:33.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.710+0000 7f548a132700 1 -- 172.21.15.5:0/3700131166 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f4debd510 con 0x556f4dfa2520 2024-05-09T00:53:33.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.711+0000 7f548a132700 1 --2- 172.21.15.5:0/3700131166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f4dfa2520 0x556f4e1293f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f5484052c60 tx=0x7f548406bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad8ac89faeca5bce server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:53:33.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.711+0000 7f546dffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:53:33.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.712+0000 7f546dffb700 1 -- 172.21.15.5:0/3700131166 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5484077be0 con 0x556f4dfa2520 2024-05-09T00:53:33.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.712+0000 7f546dffb700 1 -- 172.21.15.5:0/3700131166 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f5484077d80 con 0x556f4dfa2520 2024-05-09T00:53:33.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.713+0000 7f549adacb80 1 -- 172.21.15.5:0/3700131166 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54740eae10 con 0x556f4dfa2520 2024-05-09T00:53:33.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.715+0000 7f546dffb700 1 -- 172.21.15.5:0/3700131166 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f548406a8e0 con 0x556f4dfa2520 2024-05-09T00:53:33.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.716+0000 7f549adacb80 10 client.16254 fetch_fsmap learned FSMap version 31 2024-05-09T00:53:33.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.716+0000 7f549adacb80 1 -- 172.21.15.5:0/3700131166 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556f4e136b60 con 0x556f4dfa2520 2024-05-09T00:53:33.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.716+0000 7f546dffb700 1 -- 172.21.15.5:0/3700131166 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f548406b020 con 0x556f4dfa2520 2024-05-09T00:53:33.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.716+0000 7f546dffb700 1 -- 172.21.15.5:0/3700131166 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f5484076040 con 0x556f4dfa2520 2024-05-09T00:53:33.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.716+0000 7f549adacb80 10 client.16254 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:53:33.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.716+0000 7f549adacb80 10 client.16254 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:53:33.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.716+0000 7f549adacb80 20 client.16254 populate_metadata read hostname 'smithi005' 2024-05-09T00:53:33.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.716+0000 7f549adacb80 1 --2- 172.21.15.5:0/3700131166 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x556f4e139140 0x556f4e13b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:33.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.717+0000 7f549adacb80 4 client.16254 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:53:33.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.717+0000 7f549adacb80 1 -- 172.21.15.5:0/3700131166 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f54740eae10 con 0x556f4e139140 2024-05-09T00:53:33.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.719+0000 7f5489931700 1 --2- 172.21.15.5:0/3700131166 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x556f4e139140 0x556f4e13b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:33.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.724+0000 7f546dffb700 10 client.16254 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:53:33.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.724+0000 7f5489931700 1 --2- 172.21.15.5:0/3700131166 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x556f4e139140 0x556f4e13b5c0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c6f9efe96a1341 server_cookie=42e23cc51a7fab26 in_seq=0 out_seq=0 2024-05-09T00:53:33.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.727+0000 7f546dffb700 1 -- 172.21.15.5:0/3700131166 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f54740eae10 con 0x556f4e139140 2024-05-09T00:53:33.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.727+0000 7f546dffb700 10 client.16254 handle_command_reply: tid=0 2024-05-09T00:53:33.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.728+0000 7f549adacb80 1 client.16254 shutdown 2024-05-09T00:53:33.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.728+0000 7f549adacb80 1 -- 172.21.15.5:0/3700131166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f4dfa2520 msgr2=0x556f4e1293f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:53:33.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.728+0000 7f549adacb80 1 --2- 172.21.15.5:0/3700131166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f4dfa2520 0x556f4e1293f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f5484052c60 tx=0x7f548406bc20 comp rx=0 tx=0).stop 2024-05-09T00:53:33.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.728+0000 7f549adacb80 1 -- 172.21.15.5:0/3700131166 shutdown_connections 2024-05-09T00:53:33.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.728+0000 7f549adacb80 1 --2- 172.21.15.5:0/3700131166 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x556f4e139140 0x556f4e13b5c0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:33.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.728+0000 7f546dffb700 0 client.16254 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:53:33.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.728+0000 7f549adacb80 1 --2- 172.21.15.5:0/3700131166 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556f4e11f880 0x556f4e129e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:33.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.728+0000 7f549adacb80 1 --2- 172.21.15.5:0/3700131166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f4dfa2520 0x556f4e1293f0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:33.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.728+0000 7f549adacb80 1 --2- 172.21.15.5:0/3700131166 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f4e08c1e0 0x556f4e129930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:33.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.728+0000 7f549adacb80 1 -- 172.21.15.5:0/3700131166 >> 172.21.15.5:0/3700131166 conn(0x556f4e002720 msgr2=0x556f4e11a400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:53:33.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.729+0000 7f549adacb80 1 -- 172.21.15.5:0/3700131166 shutdown_connections 2024-05-09T00:53:33.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.729+0000 7f549adacb80 1 -- 172.21.15.5:0/3700131166 wait complete. 2024-05-09T00:53:33.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.729+0000 7f549adacb80 20 client.16254 trim_cache size 0 max 16384 2024-05-09T00:53:33.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.772+0000 7f549adacb80 1 Processor -- start 2024-05-09T00:53:33.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.773+0000 7f549adacb80 1 -- start start 2024-05-09T00:53:33.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.773+0000 7f549adacb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556f4dfa2520 0x556f4e11f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:33.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.773+0000 7f549adacb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f4e08c1e0 0x556f4e129cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:33.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.773+0000 7f549adacb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f4e11f880 0x556f4e1351f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:33.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.773+0000 7f549adacb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f547c158650 con 0x556f4e11f880 2024-05-09T00:53:33.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.773+0000 7f549adacb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f547c15a190 con 0x556f4e08c1e0 2024-05-09T00:53:33.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.773+0000 7f549adacb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f547c158d20 con 0x556f4dfa2520 2024-05-09T00:53:33.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.775+0000 7f5489931700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f4e08c1e0 0x556f4e129cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:33.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.775+0000 7f5489931700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f4e08c1e0 0x556f4e129cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:38390/0 (socket says 172.21.15.5:38390) 2024-05-09T00:53:33.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.775+0000 7f5489931700 1 -- 172.21.15.5:0/1250904952 learned_addr learned my addr 172.21.15.5:0/1250904952 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:53:33.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.775+0000 7f548a132700 1 --2- 172.21.15.5:0/1250904952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556f4dfa2520 0x556f4e11f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:33.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.781+0000 7f548a132700 1 -- 172.21.15.5:0/1250904952 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f4e08c1e0 msgr2=0x556f4e129cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:53:33.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.781+0000 7f548a132700 1 --2- 172.21.15.5:0/1250904952 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f4e08c1e0 0x556f4e129cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:33.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.781+0000 7f548a132700 1 -- 172.21.15.5:0/1250904952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f4e11f880 msgr2=0x556f4e1351f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:53:33.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.781+0000 7f548a132700 1 --2- 172.21.15.5:0/1250904952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f4e11f880 0x556f4e1351f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:33.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.781+0000 7f548a132700 1 -- 172.21.15.5:0/1250904952 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f4e125f70 con 0x556f4dfa2520 2024-05-09T00:53:33.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.784+0000 7f548a132700 1 --2- 172.21.15.5:0/1250904952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556f4dfa2520 0x556f4e11f2a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f548402ec10 tx=0x7f548406a730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f1ca1bf650c83e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:53:33.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.785+0000 7f546ffff700 1 -- 172.21.15.5:0/1250904952 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5484079880 con 0x556f4dfa2520 2024-05-09T00:53:33.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.785+0000 7f546ffff700 1 -- 172.21.15.5:0/1250904952 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54840779e0 con 0x556f4dfa2520 2024-05-09T00:53:33.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.787+0000 7f549adacb80 1 -- 172.21.15.5:0/1250904952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556f4dfa2520 msgr2=0x556f4e11f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:53:33.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.787+0000 7f549adacb80 1 --2- 172.21.15.5:0/1250904952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556f4dfa2520 0x556f4e11f2a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f548402ec10 tx=0x7f548406a730 comp rx=0 tx=0).stop 2024-05-09T00:53:33.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.787+0000 7f549adacb80 1 -- 172.21.15.5:0/1250904952 shutdown_connections 2024-05-09T00:53:33.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.787+0000 7f549adacb80 1 --2- 172.21.15.5:0/1250904952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f4e11f880 0x556f4e1351f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:33.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.787+0000 7f549adacb80 1 --2- 172.21.15.5:0/1250904952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556f4dfa2520 0x556f4e11f2a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:33.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.787+0000 7f549adacb80 1 --2- 172.21.15.5:0/1250904952 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f4e08c1e0 0x556f4e129cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:33.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.787+0000 7f549adacb80 1 -- 172.21.15.5:0/1250904952 >> 172.21.15.5:0/1250904952 conn(0x556f4e002720 msgr2=0x556f4e123790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:53:33.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.789+0000 7f549adacb80 1 -- 172.21.15.5:0/1250904952 shutdown_connections 2024-05-09T00:53:33.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.790+0000 7f549adacb80 1 -- 172.21.15.5:0/1250904952 wait complete. 2024-05-09T00:53:33.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.791+0000 7f549adacb80 1 -- start start 2024-05-09T00:53:33.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.791+0000 7f549adacb80 1 Processor -- start 2024-05-09T00:53:33.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.792+0000 7f549adacb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f4dfa2520 0x556f4e131930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:33.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.792+0000 7f549adacb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556f4e08c1e0 0x556f4e12e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:33.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.792+0000 7f549adacb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f4e11f880 0x556f4e12ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:33.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.792+0000 7f549adacb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f547c159ac0 con 0x556f4e11f880 2024-05-09T00:53:33.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.792+0000 7f549adacb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f547c1593f0 con 0x556f4dfa2520 2024-05-09T00:53:33.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.792+0000 7f549adacb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f547c1565f0 con 0x556f4e08c1e0 2024-05-09T00:53:33.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.792+0000 7f549adacb80 10 client.0 _ll_register_callbacks cb 0x556f4deb3740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:53:33.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.793+0000 7f548a933700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f4e11f880 0x556f4e12ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:33.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.793+0000 7f548a933700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f4e11f880 0x556f4e12ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38060/0 (socket says 172.21.15.5:38060) 2024-05-09T00:53:33.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.793+0000 7f548a933700 1 -- 172.21.15.5:0/3774611922 learned_addr learned my addr 172.21.15.5:0/3774611922 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:53:33.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.794+0000 7f548a933700 1 -- 172.21.15.5:0/3774611922 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556f4e08c1e0 msgr2=0x556f4e12e540 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:53:33.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.794+0000 7f548a933700 1 --2- 172.21.15.5:0/3774611922 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556f4e08c1e0 0x556f4e12e540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:33.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.794+0000 7f548a933700 1 -- 172.21.15.5:0/3774611922 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f4dfa2520 msgr2=0x556f4e131930 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:53:33.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.794+0000 7f548a933700 1 --2- 172.21.15.5:0/3774611922 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f4dfa2520 0x556f4e131930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:33.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.794+0000 7f548a933700 1 -- 172.21.15.5:0/3774611922 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f4e114140 con 0x556f4e11f880 2024-05-09T00:53:33.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.794+0000 7f548a933700 1 --2- 172.21.15.5:0/3774611922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f4e11f880 0x556f4e12ea80 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f5480003fc0 tx=0x7f5480009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e668e214462a835 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:53:33.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.795+0000 7f546dffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:53:33.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.795+0000 7f546dffb700 1 -- 172.21.15.5:0/3774611922 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5480033060 con 0x556f4e11f880 2024-05-09T00:53:33.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.795+0000 7f549adacb80 1 -- 172.21.15.5:0/3774611922 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54840779e0 con 0x556f4e11f880 2024-05-09T00:53:33.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.796+0000 7f546dffb700 1 -- 172.21.15.5:0/3774611922 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f548000b9e0 con 0x556f4e11f880 2024-05-09T00:53:33.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.796+0000 7f546dffb700 1 -- 172.21.15.5:0/3774611922 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f548000bc20 con 0x556f4e11f880 2024-05-09T00:53:33.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.797+0000 7f549adacb80 10 client.16257 fetch_fsmap learned FSMap version 31 2024-05-09T00:53:33.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.797+0000 7f549adacb80 1 -- 172.21.15.5:0/3774611922 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556f4e115490 con 0x556f4e11f880 2024-05-09T00:53:33.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.798+0000 7f546dffb700 1 -- 172.21.15.5:0/3774611922 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54840779e0 con 0x556f4e11f880 2024-05-09T00:53:33.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.798+0000 7f546dffb700 1 -- 172.21.15.5:0/3774611922 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f5480022070 con 0x556f4e11f880 2024-05-09T00:53:33.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.798+0000 7f549adacb80 10 client.16257 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:53:33.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.798+0000 7f549adacb80 10 client.16257 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:53:33.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.798+0000 7f549adacb80 20 client.16257 populate_metadata read hostname 'smithi005' 2024-05-09T00:53:33.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.799+0000 7f549adacb80 1 --2- 172.21.15.5:0/3774611922 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x556f4e139140 0x556f4e1c8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:33.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.799+0000 7f549adacb80 4 client.16257 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T00:53:33.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.799+0000 7f549adacb80 1 -- 172.21.15.5:0/3774611922 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556f4e04fa50 con 0x556f4e139140 2024-05-09T00:53:33.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.808+0000 7f548a132700 1 --2- 172.21.15.5:0/3774611922 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x556f4e139140 0x556f4e1c8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:33.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.818+0000 7f546dffb700 10 client.16257 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:53:33.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:33.818+0000 7f548a132700 1 --2- 172.21.15.5:0/3774611922 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x556f4e139140 0x556f4e1c8ae0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc3dfb9e21b0ac31 server_cookie=120e48d60766e408 in_seq=0 out_seq=0 2024-05-09T00:53:34.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:34.762+0000 7f546dffb700 1 -- 172.21.15.5:0/3774611922 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556f4e04fa50 con 0x556f4e139140 2024-05-09T00:53:34.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:34.762+0000 7f546dffb700 10 client.16257 handle_command_reply: tid=0 2024-05-09T00:53:34.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:34.762+0000 7f549adacb80 1 client.16257 shutdown 2024-05-09T00:53:34.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:34.762+0000 7f549adacb80 1 -- 172.21.15.5:0/3774611922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f4e11f880 msgr2=0x556f4e12ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:53:34.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:34.762+0000 7f549adacb80 1 --2- 172.21.15.5:0/3774611922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f4e11f880 0x556f4e12ea80 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f5480003fc0 tx=0x7f5480009ca0 comp rx=0 tx=0).stop 2024-05-09T00:53:34.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:34.762+0000 7f549adacb80 1 -- 172.21.15.5:0/3774611922 shutdown_connections 2024-05-09T00:53:34.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:34.762+0000 7f549adacb80 1 --2- 172.21.15.5:0/3774611922 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x556f4e139140 0x556f4e1c8ae0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:34.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:34.762+0000 7f549adacb80 1 --2- 172.21.15.5:0/3774611922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f4e11f880 0x556f4e12ea80 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:34.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:34.762+0000 7f549adacb80 1 --2- 172.21.15.5:0/3774611922 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f4dfa2520 0x556f4e131930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:34.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:34.762+0000 7f549adacb80 1 --2- 172.21.15.5:0/3774611922 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556f4e08c1e0 0x556f4e12e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:34.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:34.763+0000 7f549adacb80 1 -- 172.21.15.5:0/3774611922 >> 172.21.15.5:0/3774611922 conn(0x556f4e002720 msgr2=0x556f4e119ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:53:34.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:34.763+0000 7f546dffb700 0 client.16257 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:53:34.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:34.763+0000 7f549adacb80 1 -- 172.21.15.5:0/3774611922 shutdown_connections 2024-05-09T00:53:34.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:34.763+0000 7f549adacb80 1 -- 172.21.15.5:0/3774611922 wait complete. 2024-05-09T00:53:34.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:34.763+0000 7f549adacb80 20 client.16257 trim_cache size 0 max 16384 2024-05-09T00:53:34.765 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-09T00:53:34.765 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-05-09T00:53:34.766 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "2499ed0b-1624-4dbc-aba7-2efa8d4e624e", 2024-05-09T00:53:34.766 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-05-09T00:53:34.766 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-09T00:53:34.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:34.764+0000 7f548b134700 1 -- 172.21.15.5:0/2583898608 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f54700b29c0 msgr2=0x7f54700b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:53:34.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:34.764+0000 7f548b134700 1 --2- 172.21.15.5:0/2583898608 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f54700b29c0 0x7f54700b4ed0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f548000b5c0 tx=0x7f5480003680 comp rx=0 tx=0).stop 2024-05-09T00:53:34.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:34.764+0000 7f548b134700 1 -- 172.21.15.5:0/2583898608 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f547c0a5d10 msgr2=0x7f547c144c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:53:34.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:34.764+0000 7f548b134700 1 --2- 172.21.15.5:0/2583898608 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f547c0a5d10 0x7f547c144c20 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f547400d980 tx=0x7f547400dc90 comp rx=0 tx=0).stop 2024-05-09T00:53:34.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:34.764+0000 7f548b134700 1 -- 172.21.15.5:0/2583898608 shutdown_connections 2024-05-09T00:53:34.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:34.764+0000 7f548b134700 1 --2- 172.21.15.5:0/2583898608 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f54700b29c0 0x7f54700b4ed0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:34.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:34.764+0000 7f548b134700 1 --2- 172.21.15.5:0/2583898608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f547c0b8320 0x7f547c148eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:34.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:34.764+0000 7f548b134700 1 --2- 172.21.15.5:0/2583898608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f547c0a53c0 0x7f547c144680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:34.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:34.764+0000 7f548b134700 1 --2- 172.21.15.5:0/2583898608 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f547c0a5d10 0x7f547c144c20 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:34.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:34.764+0000 7f548b134700 1 -- 172.21.15.5:0/2583898608 >> 172.21.15.5:0/2583898608 conn(0x7f547c01af20 msgr2=0x7f547c0b5dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:53:34.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:34.765+0000 7f548b134700 1 -- 172.21.15.5:0/2583898608 shutdown_connections 2024-05-09T00:53:34.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:34.765+0000 7f548b134700 1 -- 172.21.15.5:0/2583898608 wait complete. 2024-05-09T00:53:35.544 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:53:35 smithi005 ceph-mon[30365]: pgmap v588: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 896 MiB/s wr, 1.76k op/s 2024-05-09T00:53:35.545 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:53:35 smithi005 ceph-mon[30365]: scrub queued for path: / 2024-05-09T00:53:35.545 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:53:35 smithi005 ceph-mon[30365]: scrub summary: idle+waiting paths [/] 2024-05-09T00:53:35.545 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:53:35 smithi005 ceph-mon[30365]: scrub summary: active paths [/] 2024-05-09T00:53:35.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:53:35 smithi131 ceph-mon[34547]: pgmap v588: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 896 MiB/s wr, 1.76k op/s 2024-05-09T00:53:35.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:53:35 smithi131 ceph-mon[34547]: scrub queued for path: / 2024-05-09T00:53:35.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:53:35 smithi131 ceph-mon[34547]: scrub summary: idle+waiting paths [/] 2024-05-09T00:53:35.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:53:35 smithi131 ceph-mon[34547]: scrub summary: active paths [/] 2024-05-09T00:53:35.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:53:35 smithi157 ceph-mon[34294]: pgmap v588: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 896 MiB/s wr, 1.76k op/s 2024-05-09T00:53:35.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:53:35 smithi157 ceph-mon[34294]: scrub queued for path: / 2024-05-09T00:53:35.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:53:35 smithi157 ceph-mon[34294]: scrub summary: idle+waiting paths [/] 2024-05-09T00:53:35.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:53:35 smithi157 ceph-mon[34294]: scrub summary: active paths [/] 2024-05-09T00:53:36.634 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T00:53:37.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:53:36 smithi005 ceph-mon[30365]: pgmap v589: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 912 MiB/s wr, 1.66k op/s 2024-05-09T00:53:37.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:53:36 smithi131 ceph-mon[34547]: pgmap v589: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 912 MiB/s wr, 1.66k op/s 2024-05-09T00:53:37.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:53:36 smithi157 ceph-mon[34294]: pgmap v589: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 912 MiB/s wr, 1.66k op/s 2024-05-09T00:53:37.887 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:53:38.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:53:38 smithi005 ceph-mon[30365]: scrub summary: active paths [/] 2024-05-09T00:53:38.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:53:38 smithi005 ceph-mon[30365]: scrub summary: idle+waiting paths [/] 2024-05-09T00:53:38.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:53:38 smithi131 ceph-mon[34547]: scrub summary: active paths [/] 2024-05-09T00:53:38.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:53:38 smithi131 ceph-mon[34547]: scrub summary: idle+waiting paths [/] 2024-05-09T00:53:38.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:53:38 smithi157 ceph-mon[34294]: scrub summary: active paths [/] 2024-05-09T00:53:38.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:53:38 smithi157 ceph-mon[34294]: scrub summary: idle+waiting paths [/] 2024-05-09T00:53:39.172 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:53:39 smithi005 ceph-mon[30365]: pgmap v590: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 863 MiB/s wr, 1.42k op/s 2024-05-09T00:53:39.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:53:39 smithi131 ceph-mon[34547]: pgmap v590: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 863 MiB/s wr, 1.42k op/s 2024-05-09T00:53:39.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:53:39 smithi157 ceph-mon[34294]: pgmap v590: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 863 MiB/s wr, 1.42k op/s 2024-05-09T00:53:40.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.139+0000 7ff95a376640 1 -- 172.21.15.5:0/1331766473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff94c09b820 msgr2=0x7ff94c09dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:53:40.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.139+0000 7ff95a376640 1 --2- 172.21.15.5:0/1331766473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff94c09b820 0x7ff94c09dc40 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7ff954051dc0 tx=0x7ff954099950 comp rx=0 tx=0).stop 2024-05-09T00:53:40.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.143+0000 7ff95a376640 1 -- 172.21.15.5:0/1331766473 shutdown_connections 2024-05-09T00:53:40.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.143+0000 7ff95a376640 1 --2- 172.21.15.5:0/1331766473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff94c09b820 0x7ff94c09dc40 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:40.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.143+0000 7ff95a376640 1 --2- 172.21.15.5:0/1331766473 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff94c096b20 0x7ff94c096f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:40.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.143+0000 7ff95a376640 1 --2- 172.21.15.5:0/1331766473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff94c0958e0 0x7ff94c0965e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:40.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.143+0000 7ff95a376640 1 -- 172.21.15.5:0/1331766473 >> 172.21.15.5:0/1331766473 conn(0x7ff94c090ab0 msgr2=0x7ff94c092f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:53:40.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.143+0000 7ff95a376640 1 -- 172.21.15.5:0/1331766473 shutdown_connections 2024-05-09T00:53:40.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.143+0000 7ff95a376640 1 -- 172.21.15.5:0/1331766473 wait complete. 2024-05-09T00:53:40.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.149+0000 7ff95a376640 1 Processor -- start 2024-05-09T00:53:40.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.149+0000 7ff95a376640 1 -- start start 2024-05-09T00:53:40.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.150+0000 7ff95a376640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff94c0958e0 0x7ff94c0a67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:40.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.150+0000 7ff95a376640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff94c096b20 0x7ff94c0a4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:40.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.150+0000 7ff95a376640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff94c09b820 0x7ff94c0a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:40.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.150+0000 7ff95a376640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c09f1e0 con 0x7ff94c0958e0 2024-05-09T00:53:40.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.150+0000 7ff95a376640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c09f060 con 0x7ff94c09b820 2024-05-09T00:53:40.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.150+0000 7ff95a376640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c09f360 con 0x7ff94c096b20 2024-05-09T00:53:40.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.151+0000 7ff953fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff94c0958e0 0x7ff94c0a67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:40.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.151+0000 7ff953fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff94c0958e0 0x7ff94c0a67e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38078/0 (socket says 172.21.15.5:38078) 2024-05-09T00:53:40.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.151+0000 7ff953fff640 1 -- 172.21.15.5:0/1257626955 learned_addr learned my addr 172.21.15.5:0/1257626955 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:53:40.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.152+0000 7ff958913640 1 --2- 172.21.15.5:0/1257626955 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff94c09b820 0x7ff94c0a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:40.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.152+0000 7ff9537fe640 1 --2- 172.21.15.5:0/1257626955 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff94c096b20 0x7ff94c0a4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:40.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.156+0000 7ff9537fe640 1 -- 172.21.15.5:0/1257626955 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff94c09b820 msgr2=0x7ff94c0a53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:53:40.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.156+0000 7ff9537fe640 1 --2- 172.21.15.5:0/1257626955 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff94c09b820 0x7ff94c0a53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:40.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.156+0000 7ff9537fe640 1 -- 172.21.15.5:0/1257626955 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff94c0958e0 msgr2=0x7ff94c0a67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:53:40.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.156+0000 7ff9537fe640 1 --2- 172.21.15.5:0/1257626955 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff94c0958e0 0x7ff94c0a67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:40.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.156+0000 7ff9537fe640 1 -- 172.21.15.5:0/1257626955 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff94c0a5900 con 0x7ff94c096b20 2024-05-09T00:53:40.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.164+0000 7ff9537fe640 1 --2- 172.21.15.5:0/1257626955 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff94c096b20 0x7ff94c0a4e80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff94800c970 tx=0x7ff94800ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:53:40.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.172+0000 7ff9517fa640 1 -- 172.21.15.5:0/1257626955 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff948007bb0 con 0x7ff94c096b20 2024-05-09T00:53:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.172+0000 7ff95a376640 1 -- 172.21.15.5:0/1257626955 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff94c0a5bf0 con 0x7ff94c096b20 2024-05-09T00:53:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.172+0000 7ff95a376640 1 -- 172.21.15.5:0/1257626955 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff94c1441d0 con 0x7ff94c096b20 2024-05-09T00:53:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.173+0000 7ff9517fa640 1 -- 172.21.15.5:0/1257626955 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff948007d50 con 0x7ff94c096b20 2024-05-09T00:53:40.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.173+0000 7ff9517fa640 1 -- 172.21.15.5:0/1257626955 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff948004df0 con 0x7ff94c096b20 2024-05-09T00:53:40.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.181+0000 7ff9517fa640 1 -- 172.21.15.5:0/1257626955 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7ff94800f510 con 0x7ff94c096b20 2024-05-09T00:53:40.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.184+0000 7ff96122b740 1 Processor -- start 2024-05-09T00:53:40.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.184+0000 7ff96122b740 1 -- start start 2024-05-09T00:53:40.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.185+0000 7ff96122b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56425e7de1b0 0x56425eb826c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:40.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.186+0000 7ff96122b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56425e7b52f0 0x56425eb82c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:40.189 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.186+0000 7ff96122b740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56425e7bd4d0 0x56425eb89b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:40.189 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.186+0000 7ff96122b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56425eae67c0 con 0x56425e7b52f0 2024-05-09T00:53:40.189 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.186+0000 7ff96122b740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x56425ea820e0 con 0x56425e7bd4d0 2024-05-09T00:53:40.190 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.186+0000 7ff96122b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56425eae2780 con 0x56425e7de1b0 2024-05-09T00:53:40.190 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.186+0000 7ff9517fa640 1 --2- 172.21.15.5:0/1257626955 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ff9440b2a00 0x7ff9440b4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:40.190 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.188+0000 7ff9517fa640 1 -- 172.21.15.5:0/1257626955 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7ff9480e1dc0 con 0x7ff94c096b20 2024-05-09T00:53:40.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.190+0000 7ff958913640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56425e7de1b0 0x56425eb826c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:40.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.190+0000 7ff958913640 1 --2- 172.21.15.5:0/1257626955 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ff9440b2a00 0x7ff9440b4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:40.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.190+0000 7ff9537fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56425e7bd4d0 0x56425eb89b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:40.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.193+0000 7ff953fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56425e7b52f0 0x56425eb82c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:40.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.193+0000 7ff953fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56425e7b52f0 0x56425eb82c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38092/0 (socket says 172.21.15.5:38092) 2024-05-09T00:53:40.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.193+0000 7ff953fff640 1 -- 172.21.15.5:0/3571533553 learned_addr learned my addr 172.21.15.5:0/3571533553 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:53:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.198+0000 7ff9537fe640 1 -- 172.21.15.5:0/3571533553 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56425e7de1b0 msgr2=0x56425eb826c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:53:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.198+0000 7ff9537fe640 1 --2- 172.21.15.5:0/3571533553 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56425e7de1b0 0x56425eb826c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.198+0000 7ff9537fe640 1 -- 172.21.15.5:0/3571533553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56425e7b52f0 msgr2=0x56425eb82c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:53:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.198+0000 7ff9537fe640 1 --2- 172.21.15.5:0/3571533553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56425e7b52f0 0x56425eb82c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.198+0000 7ff9537fe640 1 -- 172.21.15.5:0/3571533553 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56425eaab7c0 con 0x56425e7bd4d0 2024-05-09T00:53:40.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.203+0000 7ff958913640 1 --2- 172.21.15.5:0/1257626955 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ff9440b2a00 0x7ff9440b4f10 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff954052170 tx=0x7ff954068f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:53:40.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.203+0000 7ff9537fe640 1 --2- 172.21.15.5:0/3571533553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56425e7bd4d0 0x56425eb89b40 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff9480a9390 tx=0x7ff9480a53f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=385dba8ec18fd774 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:53:40.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.205+0000 7ff93e7fc640 1 -- 172.21.15.5:0/3571533553 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff94801d040 con 0x56425e7bd4d0 2024-05-09T00:53:40.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.206+0000 7ff93e7fc640 1 -- 172.21.15.5:0/3571533553 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9480ad700 con 0x56425e7bd4d0 2024-05-09T00:53:40.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.208+0000 7ff96122b740 1 -- 172.21.15.5:0/3571533553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56425e7bd4d0 msgr2=0x56425eb89b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:53:40.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.208+0000 7ff96122b740 1 --2- 172.21.15.5:0/3571533553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56425e7bd4d0 0x56425eb89b40 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff9480a9390 tx=0x7ff9480a53f0 comp rx=0 tx=0).stop 2024-05-09T00:53:40.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.209+0000 7ff96122b740 1 -- 172.21.15.5:0/3571533553 shutdown_connections 2024-05-09T00:53:40.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.209+0000 7ff96122b740 1 --2- 172.21.15.5:0/3571533553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56425e7bd4d0 0x56425eb89b40 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:40.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.209+0000 7ff96122b740 1 --2- 172.21.15.5:0/3571533553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56425e7b52f0 0x56425eb82c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:40.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.209+0000 7ff96122b740 1 --2- 172.21.15.5:0/3571533553 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56425e7de1b0 0x56425eb826c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:40.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.209+0000 7ff96122b740 1 -- 172.21.15.5:0/3571533553 >> 172.21.15.5:0/3571533553 conn(0x56425e7d2300 msgr2=0x56425eb7e050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:53:40.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.210+0000 7ff96122b740 1 -- 172.21.15.5:0/3571533553 shutdown_connections 2024-05-09T00:53:40.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.211+0000 7ff96122b740 1 -- 172.21.15.5:0/3571533553 wait complete. 2024-05-09T00:53:40.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.211+0000 7ff96122b740 1 -- start start 2024-05-09T00:53:40.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.212+0000 7ff96122b740 1 Processor -- start 2024-05-09T00:53:40.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.212+0000 7ff96122b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56425e7b52f0 0x56425eb8c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:40.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.212+0000 7ff96122b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56425e7bd4d0 0x56425eb8cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:40.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.212+0000 7ff96122b740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56425e7de1b0 0x56425eb93a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:40.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.212+0000 7ff96122b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56425eb8a5d0 con 0x56425e7b52f0 2024-05-09T00:53:40.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.212+0000 7ff96122b740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x56425ea6d9c0 con 0x56425e7de1b0 2024-05-09T00:53:40.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.213+0000 7ff96122b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56425eb8a750 con 0x56425e7bd4d0 2024-05-09T00:53:40.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.213+0000 7ff96122b740 10 client.0 _ll_register_callbacks cb 0x56425ea9a020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:53:40.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.214+0000 7ff953fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56425e7b52f0 0x56425eb8c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:40.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.214+0000 7ff953fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56425e7b52f0 0x56425eb8c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38108/0 (socket says 172.21.15.5:38108) 2024-05-09T00:53:40.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.214+0000 7ff953fff640 1 -- 172.21.15.5:0/1065666536 learned_addr learned my addr 172.21.15.5:0/1065666536 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:53:40.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.215+0000 7ff953fff640 1 -- 172.21.15.5:0/1065666536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56425e7bd4d0 msgr2=0x56425eb8cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:53:40.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.215+0000 7ff953fff640 1 --2- 172.21.15.5:0/1065666536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56425e7bd4d0 0x56425eb8cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:40.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.215+0000 7ff953fff640 1 -- 172.21.15.5:0/1065666536 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56425e7de1b0 msgr2=0x56425eb93a70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:53:40.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.215+0000 7ff953fff640 1 --2- 172.21.15.5:0/1065666536 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56425e7de1b0 0x56425eb93a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:40.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.215+0000 7ff953fff640 1 -- 172.21.15.5:0/1065666536 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56425ea941f0 con 0x56425e7b52f0 2024-05-09T00:53:40.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.216+0000 7ff953fff640 1 --2- 172.21.15.5:0/1065666536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56425e7b52f0 0x56425eb8c5d0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7ff940012840 tx=0x7ff940012d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8037afb78f06009b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:53:40.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.216+0000 7ff9337fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:53:40.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.217+0000 7ff9337fe640 1 -- 172.21.15.5:0/1065666536 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff94000f920 con 0x56425e7b52f0 2024-05-09T00:53:40.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.217+0000 7ff96122b740 1 -- 172.21.15.5:0/1065666536 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56425eaf43d0 con 0x56425e7b52f0 2024-05-09T00:53:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.222+0000 7ff9337fe640 1 -- 172.21.15.5:0/1065666536 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7ff94000fac0 con 0x56425e7b52f0 2024-05-09T00:53:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.230+0000 7ff9337fe640 1 -- 172.21.15.5:0/1065666536 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff94000fcc0 con 0x56425e7b52f0 2024-05-09T00:53:40.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.231+0000 7ff96122b740 10 client.16269 fetch_fsmap learned FSMap version 31 2024-05-09T00:53:40.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.231+0000 7ff96122b740 1 -- 172.21.15.5:0/1065666536 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56425e849790 con 0x56425e7b52f0 2024-05-09T00:53:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.238+0000 7ff9337fe640 1 -- 172.21.15.5:0/1065666536 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56425eaf43d0 con 0x56425e7b52f0 2024-05-09T00:53:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.239+0000 7ff9337fe640 1 -- 172.21.15.5:0/1065666536 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7ff940025070 con 0x56425e7b52f0 2024-05-09T00:53:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.240+0000 7ff96122b740 10 client.16269 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:53:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.240+0000 7ff96122b740 10 client.16269 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:53:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.240+0000 7ff96122b740 20 client.16269 populate_metadata read hostname 'smithi005' 2024-05-09T00:53:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.240+0000 7ff96122b740 1 --2- 172.21.15.5:0/1065666536 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56425e79cd80 0x56425ebab560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.240+0000 7ff96122b740 4 client.16269 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:53:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.240+0000 7ff96122b740 1 -- 172.21.15.5:0/1065666536 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56425eaee240 con 0x56425e79cd80 2024-05-09T00:53:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.243+0000 7ff9537fe640 1 --2- 172.21.15.5:0/1065666536 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56425e79cd80 0x56425ebab560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.258+0000 7ff9537fe640 1 --2- 172.21.15.5:0/1065666536 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56425e79cd80 0x56425ebab560 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8531219ea46d3b5 server_cookie=3605fb3fad4d1b37 in_seq=0 out_seq=0 2024-05-09T00:53:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.258+0000 7ff9337fe640 10 client.16269 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:53:40.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.265+0000 7ff9337fe640 1 -- 172.21.15.5:0/1065666536 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56425eaee240 con 0x56425e79cd80 2024-05-09T00:53:40.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.265+0000 7ff9337fe640 10 client.16269 handle_command_reply: tid=0 2024-05-09T00:53:40.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.265+0000 7ff96122b740 1 client.16269 shutdown 2024-05-09T00:53:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.266+0000 7ff96122b740 1 -- 172.21.15.5:0/1065666536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56425e7b52f0 msgr2=0x56425eb8c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:53:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.266+0000 7ff96122b740 1 --2- 172.21.15.5:0/1065666536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56425e7b52f0 0x56425eb8c5d0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7ff940012840 tx=0x7ff940012d00 comp rx=0 tx=0).stop 2024-05-09T00:53:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.267+0000 7ff96122b740 1 -- 172.21.15.5:0/1065666536 shutdown_connections 2024-05-09T00:53:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.267+0000 7ff96122b740 1 --2- 172.21.15.5:0/1065666536 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56425e79cd80 0x56425ebab560 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.267+0000 7ff96122b740 1 --2- 172.21.15.5:0/1065666536 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56425e7de1b0 0x56425eb93a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.267+0000 7ff96122b740 1 --2- 172.21.15.5:0/1065666536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56425e7bd4d0 0x56425eb8cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.267+0000 7ff96122b740 1 --2- 172.21.15.5:0/1065666536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56425e7b52f0 0x56425eb8c5d0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.267+0000 7ff96122b740 1 -- 172.21.15.5:0/1065666536 >> 172.21.15.5:0/1065666536 conn(0x56425e7d2300 msgr2=0x56425eb7d290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:53:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.268+0000 7ff9337fe640 0 client.16269 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:53:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.268+0000 7ff96122b740 1 -- 172.21.15.5:0/1065666536 shutdown_connections 2024-05-09T00:53:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.269+0000 7ff96122b740 1 -- 172.21.15.5:0/1065666536 wait complete. 2024-05-09T00:53:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.269+0000 7ff96122b740 20 client.16269 trim_cache size 0 max 16384 2024-05-09T00:53:40.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.321+0000 7ff96122b740 1 Processor -- start 2024-05-09T00:53:40.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.321+0000 7ff96122b740 1 -- start start 2024-05-09T00:53:40.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.322+0000 7ff96122b740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56425e7b52f0 0x56425eb8ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:40.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.322+0000 7ff96122b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56425e7bd4d0 0x56425eb8b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:40.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.322+0000 7ff96122b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56425e7de1b0 0x56425eb8c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:40.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.322+0000 7ff96122b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff93800e500 con 0x56425e7bd4d0 2024-05-09T00:53:40.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.322+0000 7ff96122b740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff938010040 con 0x56425e7b52f0 2024-05-09T00:53:40.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.322+0000 7ff96122b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff93800ebd0 con 0x56425e7de1b0 2024-05-09T00:53:40.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.322+0000 7ff9537fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56425e7bd4d0 0x56425eb8b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:40.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.323+0000 7ff9537fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56425e7bd4d0 0x56425eb8b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38120/0 (socket says 172.21.15.5:38120) 2024-05-09T00:53:40.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.323+0000 7ff9537fe640 1 -- 172.21.15.5:0/1665447305 learned_addr learned my addr 172.21.15.5:0/1665447305 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:53:40.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.323+0000 7ff9537fe640 1 -- 172.21.15.5:0/1665447305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56425e7de1b0 msgr2=0x56425eb8c1a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:53:40.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.323+0000 7ff9537fe640 1 --2- 172.21.15.5:0/1665447305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56425e7de1b0 0x56425eb8c1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:40.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.323+0000 7ff9537fe640 1 -- 172.21.15.5:0/1665447305 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56425e7b52f0 msgr2=0x56425eb8ab70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:53:40.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.323+0000 7ff9537fe640 1 --2- 172.21.15.5:0/1665447305 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56425e7b52f0 0x56425eb8ab70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:40.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.323+0000 7ff9537fe640 1 -- 172.21.15.5:0/1665447305 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56425e8c05f0 con 0x56425e7bd4d0 2024-05-09T00:53:40.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.324+0000 7ff9537fe640 1 --2- 172.21.15.5:0/1665447305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56425e7bd4d0 0x56425eb8b0b0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ff948000c00 tx=0x7ff9480a52c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c6a1a36e2170de1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:53:40.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.325+0000 7ff93dffb640 1 -- 172.21.15.5:0/1665447305 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9480c7020 con 0x56425e7bd4d0 2024-05-09T00:53:40.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.325+0000 7ff93dffb640 1 -- 172.21.15.5:0/1665447305 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9480b21b0 con 0x56425e7bd4d0 2024-05-09T00:53:40.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.325+0000 7ff93dffb640 1 -- 172.21.15.5:0/1665447305 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff94801d040 con 0x56425e7bd4d0 2024-05-09T00:53:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.327+0000 7ff96122b740 1 -- 172.21.15.5:0/1665447305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56425e7bd4d0 msgr2=0x56425eb8b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:53:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.327+0000 7ff96122b740 1 --2- 172.21.15.5:0/1665447305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56425e7bd4d0 0x56425eb8b0b0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ff948000c00 tx=0x7ff9480a52c0 comp rx=0 tx=0).stop 2024-05-09T00:53:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.328+0000 7ff96122b740 1 -- 172.21.15.5:0/1665447305 shutdown_connections 2024-05-09T00:53:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.328+0000 7ff96122b740 1 --2- 172.21.15.5:0/1665447305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56425e7de1b0 0x56425eb8c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.328+0000 7ff96122b740 1 --2- 172.21.15.5:0/1665447305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56425e7bd4d0 0x56425eb8b0b0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.328+0000 7ff96122b740 1 --2- 172.21.15.5:0/1665447305 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56425e7b52f0 0x56425eb8ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:40.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.328+0000 7ff96122b740 1 -- 172.21.15.5:0/1665447305 >> 172.21.15.5:0/1665447305 conn(0x56425e7d2300 msgr2=0x56425eb82640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:53:40.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.329+0000 7ff96122b740 1 -- 172.21.15.5:0/1665447305 shutdown_connections 2024-05-09T00:53:40.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.329+0000 7ff96122b740 1 -- 172.21.15.5:0/1665447305 wait complete. 2024-05-09T00:53:40.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.330+0000 7ff96122b740 1 -- start start 2024-05-09T00:53:40.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.331+0000 7ff96122b740 1 Processor -- start 2024-05-09T00:53:40.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.331+0000 7ff96122b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56425e7b52f0 0x56425eb8fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:40.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.331+0000 7ff96122b740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56425e7bd4d0 0x56425eb903b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:40.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.332+0000 7ff96122b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56425e7de1b0 0x56425ec11900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:40.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.332+0000 7ff96122b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff93800f970 con 0x56425e7b52f0 2024-05-09T00:53:40.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.332+0000 7ff96122b740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff93800f2a0 con 0x56425e7bd4d0 2024-05-09T00:53:40.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.332+0000 7ff96122b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff93800c590 con 0x56425e7de1b0 2024-05-09T00:53:40.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.332+0000 7ff96122b740 10 client.0 _ll_register_callbacks cb 0x7ff9400096c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:53:40.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.332+0000 7ff953fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56425e7b52f0 0x56425eb8fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:40.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.332+0000 7ff953fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56425e7b52f0 0x56425eb8fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38128/0 (socket says 172.21.15.5:38128) 2024-05-09T00:53:40.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.332+0000 7ff953fff640 1 -- 172.21.15.5:0/2537716282 learned_addr learned my addr 172.21.15.5:0/2537716282 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:53:40.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.333+0000 7ff953fff640 1 -- 172.21.15.5:0/2537716282 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56425e7de1b0 msgr2=0x56425ec11900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:53:40.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.333+0000 7ff953fff640 1 --2- 172.21.15.5:0/2537716282 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56425e7de1b0 0x56425ec11900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:40.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.333+0000 7ff953fff640 1 -- 172.21.15.5:0/2537716282 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56425e7bd4d0 msgr2=0x56425eb903b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:53:40.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.333+0000 7ff953fff640 1 --2- 172.21.15.5:0/2537716282 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56425e7bd4d0 0x56425eb903b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:40.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.333+0000 7ff953fff640 1 -- 172.21.15.5:0/2537716282 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56425eaaaed0 con 0x56425e7b52f0 2024-05-09T00:53:40.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.334+0000 7ff953fff640 1 --2- 172.21.15.5:0/2537716282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56425e7b52f0 0x56425eb8fe70 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7ff9400054f0 tx=0x7ff94000ffc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36da56141529d958 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:53:40.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.334+0000 7ff9337fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:53:40.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.335+0000 7ff9337fe640 1 -- 172.21.15.5:0/2537716282 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9400123e0 con 0x56425e7b52f0 2024-05-09T00:53:40.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.336+0000 7ff9337fe640 1 -- 172.21.15.5:0/2537716282 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7ff940005000 con 0x56425e7b52f0 2024-05-09T00:53:40.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.336+0000 7ff96122b740 1 -- 172.21.15.5:0/2537716282 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9480b21b0 con 0x56425e7b52f0 2024-05-09T00:53:40.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.336+0000 7ff9337fe640 1 -- 172.21.15.5:0/2537716282 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff940005260 con 0x56425e7b52f0 2024-05-09T00:53:40.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.337+0000 7ff96122b740 10 client.16275 fetch_fsmap learned FSMap version 31 2024-05-09T00:53:40.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.337+0000 7ff96122b740 1 -- 172.21.15.5:0/2537716282 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56425ec12c90 con 0x56425e7b52f0 2024-05-09T00:53:40.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.338+0000 7ff9337fe640 1 -- 172.21.15.5:0/2537716282 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9480b21b0 con 0x56425e7b52f0 2024-05-09T00:53:40.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.341+0000 7ff9337fe640 1 -- 172.21.15.5:0/2537716282 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7ff940025040 con 0x56425e7b52f0 2024-05-09T00:53:40.344 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.341+0000 7ff96122b740 10 client.16275 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:53:40.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.341+0000 7ff96122b740 10 client.16275 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:53:40.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.342+0000 7ff96122b740 20 client.16275 populate_metadata read hostname 'smithi005' 2024-05-09T00:53:40.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.342+0000 7ff96122b740 1 --2- 172.21.15.5:0/2537716282 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56425e7721c0 0x56425ec15680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:40.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.342+0000 7ff96122b740 4 client.16275 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:53:40.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.342+0000 7ff96122b740 1 -- 172.21.15.5:0/2537716282 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56425ea69350 con 0x56425e7721c0 2024-05-09T00:53:40.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.348+0000 7ff9537fe640 1 --2- 172.21.15.5:0/2537716282 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56425e7721c0 0x56425ec15680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.361+0000 7ff9337fe640 10 client.16275 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:53:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:40.361+0000 7ff9537fe640 1 --2- 172.21.15.5:0/2537716282 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56425e7721c0 0x56425ec15680 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b83f1c068d98378f server_cookie=742aa80768522812 in_seq=0 out_seq=0 2024-05-09T00:53:41.538 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:53:41 smithi005 ceph-mon[30365]: pgmap v591: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 990 MiB/s wr, 1.57k op/s 2024-05-09T00:53:41.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:53:41 smithi131 ceph-mon[34547]: pgmap v591: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 990 MiB/s wr, 1.57k op/s 2024-05-09T00:53:41.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:53:41 smithi157 ceph-mon[34294]: pgmap v591: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 990 MiB/s wr, 1.57k op/s 2024-05-09T00:53:42.487 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:53:42 smithi005 ceph-mon[30365]: scrub summary: idle+waiting paths [/] 2024-05-09T00:53:42.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:53:42 smithi131 ceph-mon[34547]: scrub summary: idle+waiting paths [/] 2024-05-09T00:53:42.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:53:42 smithi157 ceph-mon[34294]: scrub summary: idle+waiting paths [/] 2024-05-09T00:53:42.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.900+0000 7f6161a3a700 1 -- 172.21.15.5:0/1222077052 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f615c109660 msgr2=0x7f615c110fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:53:42.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.900+0000 7f6161a3a700 1 --2- 172.21.15.5:0/1222077052 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f615c109660 0x7f615c110fc0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6154009f90 tx=0x7f615400b330 comp rx=0 tx=0).stop 2024-05-09T00:53:42.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.908+0000 7f6161a3a700 1 -- 172.21.15.5:0/1222077052 shutdown_connections 2024-05-09T00:53:42.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.908+0000 7f6161a3a700 1 --2- 172.21.15.5:0/1222077052 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f615c109660 0x7f615c110fc0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:42.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.908+0000 7f6161a3a700 1 --2- 172.21.15.5:0/1222077052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f615c1043e0 0x7f615c1047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:42.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.908+0000 7f6161a3a700 1 --2- 172.21.15.5:0/1222077052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f615c108d10 0x7f615c109120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:42.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.908+0000 7f6161a3a700 1 -- 172.21.15.5:0/1222077052 >> 172.21.15.5:0/1222077052 conn(0x7f615c079d10 msgr2=0x7f615c07a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:53:42.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.914+0000 7f6161a3a700 1 -- 172.21.15.5:0/1222077052 shutdown_connections 2024-05-09T00:53:42.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.915+0000 7f6161a3a700 1 -- 172.21.15.5:0/1222077052 wait complete. 2024-05-09T00:53:42.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.916+0000 7f6161a3a700 1 Processor -- start 2024-05-09T00:53:42.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.919+0000 7f6161a3a700 1 -- start start 2024-05-09T00:53:42.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.919+0000 7f6161a3a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f615c1043e0 0x7f615c0763f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:42.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.919+0000 7f6161a3a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f615c108d10 0x7f615c06e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:42.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.919+0000 7f6161a3a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f615c109660 0x7f615c06eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:42.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.919+0000 7f6161a3a700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f615c06dae0 con 0x7f615c108d10 2024-05-09T00:53:42.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.919+0000 7f6161a3a700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f615c06d960 con 0x7f615c1043e0 2024-05-09T00:53:42.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.919+0000 7f6161a3a700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f615c06dc60 con 0x7f615c109660 2024-05-09T00:53:42.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.922+0000 7f6160a38700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f615c1043e0 0x7f615c0763f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:42.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.922+0000 7f6160a38700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f615c1043e0 0x7f615c0763f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:38458/0 (socket says 172.21.15.5:38458) 2024-05-09T00:53:42.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.922+0000 7f6160a38700 1 -- 172.21.15.5:0/3374073698 learned_addr learned my addr 172.21.15.5:0/3374073698 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:53:42.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.922+0000 7f6161239700 1 --2- 172.21.15.5:0/3374073698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f615c109660 0x7f615c06eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:42.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.923+0000 7f6161239700 1 -- 172.21.15.5:0/3374073698 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f615c1043e0 msgr2=0x7f615c0763f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:53:42.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.923+0000 7f6161239700 1 --2- 172.21.15.5:0/3374073698 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f615c1043e0 0x7f615c0763f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:42.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.923+0000 7f6161239700 1 -- 172.21.15.5:0/3374073698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f615c108d10 msgr2=0x7f615c06e5d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T00:53:42.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.923+0000 7f6161239700 1 --2- 172.21.15.5:0/3374073698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f615c108d10 0x7f615c06e5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:42.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.923+0000 7f6161239700 1 -- 172.21.15.5:0/3374073698 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f615c06f320 con 0x7f615c109660 2024-05-09T00:53:42.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.923+0000 7f6160a38700 1 --2- 172.21.15.5:0/3374073698 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f615c1043e0 0x7f615c0763f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:53:42.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.925+0000 7f6161239700 1 --2- 172.21.15.5:0/3374073698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f615c109660 0x7f615c06eb10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6154009f90 tx=0x7f615400cd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:53:42.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.927+0000 7f6159ffb700 1 -- 172.21.15.5:0/3374073698 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6154021070 con 0x7f615c109660 2024-05-09T00:53:42.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.928+0000 7f6159ffb700 1 -- 172.21.15.5:0/3374073698 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f615401d450 con 0x7f615c109660 2024-05-09T00:53:42.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.928+0000 7f6159ffb700 1 -- 172.21.15.5:0/3374073698 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61540052a0 con 0x7f615c109660 2024-05-09T00:53:42.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.928+0000 7f6161a3a700 1 -- 172.21.15.5:0/3374073698 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f615c1be7b0 con 0x7f615c109660 2024-05-09T00:53:42.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.928+0000 7f6161a3a700 1 -- 172.21.15.5:0/3374073698 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f615c1bec10 con 0x7f615c109660 2024-05-09T00:53:42.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.939+0000 7f61716b2b80 1 Processor -- start 2024-05-09T00:53:42.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.939+0000 7f61716b2b80 1 -- start start 2024-05-09T00:53:42.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.940+0000 7f61716b2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b831acfd00 0x55b831bd2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:42.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.940+0000 7f61716b2b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b831a554a0 0x55b831bd2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:42.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.940+0000 7f61716b2b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b831bdc540 0x55b831bde960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:42.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.940+0000 7f61716b2b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b831af50a0 con 0x55b831bdc540 2024-05-09T00:53:42.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.940+0000 7f61716b2b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55b831a9a9a0 con 0x55b831acfd00 2024-05-09T00:53:42.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.940+0000 7f61716b2b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b831af1060 con 0x55b831a554a0 2024-05-09T00:53:42.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.942+0000 7f6160a38700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b831a554a0 0x55b831bd2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:42.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.943+0000 7f6160a38700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b831a554a0 0x55b831bd2f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:60836/0 (socket says 172.21.15.5:60836) 2024-05-09T00:53:42.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.943+0000 7f6160a38700 1 -- 172.21.15.5:0/3228291815 learned_addr learned my addr 172.21.15.5:0/3228291815 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:53:42.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.945+0000 7f6160a38700 1 -- 172.21.15.5:0/3228291815 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b831acfd00 msgr2=0x55b831bd2a30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:53:42.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.945+0000 7f6160a38700 1 --2- 172.21.15.5:0/3228291815 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b831acfd00 0x55b831bd2a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:42.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.945+0000 7f6160a38700 1 -- 172.21.15.5:0/3228291815 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b831bdc540 msgr2=0x55b831bde960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:53:42.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.945+0000 7f6160a38700 1 --2- 172.21.15.5:0/3228291815 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b831bdc540 0x55b831bde960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:42.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.945+0000 7f6160a38700 1 -- 172.21.15.5:0/3228291815 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b831a8b500 con 0x55b831a554a0 2024-05-09T00:53:42.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.947+0000 7f6159ffb700 1 -- 172.21.15.5:0/3374073698 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f6154024070 con 0x7f615c109660 2024-05-09T00:53:42.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.947+0000 7f6160a38700 1 --2- 172.21.15.5:0/3228291815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b831a554a0 0x55b831bd2f70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f614c009fb0 tx=0x7f614c00e4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c51e3e1974af138 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:53:42.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.948+0000 7f6142ffd700 1 -- 172.21.15.5:0/3228291815 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f614c017070 con 0x55b831a554a0 2024-05-09T00:53:42.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.948+0000 7f6142ffd700 1 -- 172.21.15.5:0/3228291815 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f614c00edf0 con 0x55b831a554a0 2024-05-09T00:53:42.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.948+0000 7f6142ffd700 1 -- 172.21.15.5:0/3228291815 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f614c0126c0 con 0x55b831a554a0 2024-05-09T00:53:42.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.950+0000 7f6159ffb700 1 --2- 172.21.15.5:0/3374073698 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f61440b2a90 0x7f61440b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:42.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.950+0000 7f61716b2b80 1 -- 172.21.15.5:0/3228291815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b831a554a0 msgr2=0x55b831bd2f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:53:42.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.950+0000 7f61716b2b80 1 --2- 172.21.15.5:0/3228291815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b831a554a0 0x55b831bd2f70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f614c009fb0 tx=0x7f614c00e4b0 comp rx=0 tx=0).stop 2024-05-09T00:53:42.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.950+0000 7f6159ffb700 1 -- 172.21.15.5:0/3374073698 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f615400f020 con 0x7f615c109660 2024-05-09T00:53:42.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.951+0000 7f61716b2b80 1 -- 172.21.15.5:0/3228291815 shutdown_connections 2024-05-09T00:53:42.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.951+0000 7f61716b2b80 1 --2- 172.21.15.5:0/3228291815 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b831bdc540 0x55b831bde960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:42.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.951+0000 7f61716b2b80 1 --2- 172.21.15.5:0/3228291815 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b831acfd00 0x55b831bd2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:42.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.951+0000 7f61716b2b80 1 --2- 172.21.15.5:0/3228291815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b831a554a0 0x55b831bd2f70 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:42.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.951+0000 7f61716b2b80 1 -- 172.21.15.5:0/3228291815 >> 172.21.15.5:0/3228291815 conn(0x55b831a58f70 msgr2=0x55b831bcd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:53:42.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.956+0000 7f61716b2b80 1 -- 172.21.15.5:0/3228291815 shutdown_connections 2024-05-09T00:53:42.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.960+0000 7f615bfff700 1 --2- 172.21.15.5:0/3374073698 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f61440b2a90 0x7f61440b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:42.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.961+0000 7f61716b2b80 1 -- 172.21.15.5:0/3228291815 wait complete. 2024-05-09T00:53:42.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.962+0000 7f61716b2b80 1 -- start start 2024-05-09T00:53:42.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.963+0000 7f61716b2b80 1 Processor -- start 2024-05-09T00:53:42.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.963+0000 7f61716b2b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b831a554a0 0x55b831bd94e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:42.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.963+0000 7f61716b2b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b831acfd00 0x55b831bd9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:42.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.963+0000 7f61716b2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b831bdc540 0x55b831bd9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:42.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.963+0000 7f61716b2b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b831bdef60 con 0x55b831acfd00 2024-05-09T00:53:42.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.963+0000 7f61716b2b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55b831a8fb20 con 0x55b831bdc540 2024-05-09T00:53:42.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.963+0000 7f61716b2b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b831bdf0e0 con 0x55b831a554a0 2024-05-09T00:53:42.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.963+0000 7f61716b2b80 10 client.0 _ll_register_callbacks cb 0x55b8319666c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:53:42.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.964+0000 7f6160a38700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b831a554a0 0x55b831bd94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:42.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.965+0000 7f6160a38700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b831a554a0 0x55b831bd94e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:60850/0 (socket says 172.21.15.5:60850) 2024-05-09T00:53:42.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.965+0000 7f6160a38700 1 -- 172.21.15.5:0/3879251450 learned_addr learned my addr 172.21.15.5:0/3879251450 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:53:42.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.969+0000 7f6160a38700 1 -- 172.21.15.5:0/3879251450 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b831bdc540 msgr2=0x55b831bd9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:53:42.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.969+0000 7f6160a38700 1 --2- 172.21.15.5:0/3879251450 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b831bdc540 0x55b831bd9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:42.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.969+0000 7f6160a38700 1 -- 172.21.15.5:0/3879251450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b831acfd00 msgr2=0x55b831bd9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:53:42.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.969+0000 7f6160a38700 1 --2- 172.21.15.5:0/3879251450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b831acfd00 0x55b831bd9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:42.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.969+0000 7f6160a38700 1 -- 172.21.15.5:0/3879251450 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b831970490 con 0x55b831a554a0 2024-05-09T00:53:42.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.969+0000 7f6160a38700 1 --2- 172.21.15.5:0/3879251450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b831a554a0 0x55b831bd94e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f614c019000 tx=0x7f614c00efa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fa6c3299c146144 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:53:42.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.970+0000 7f613bfff700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-09T00:53:42.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.970+0000 7f613bfff700 1 -- 172.21.15.5:0/3879251450 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f614c017040 con 0x55b831a554a0 2024-05-09T00:53:42.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.970+0000 7f61716b2b80 1 -- 172.21.15.5:0/3879251450 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f614c00edf0 con 0x55b831a554a0 2024-05-09T00:53:42.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.970+0000 7f613bfff700 1 -- 172.21.15.5:0/3879251450 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f614c01c9e0 con 0x55b831a554a0 2024-05-09T00:53:42.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.971+0000 7f613bfff700 1 -- 172.21.15.5:0/3879251450 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f614c01cbe0 con 0x55b831a554a0 2024-05-09T00:53:42.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.977+0000 7f61716b2b80 10 client.25819 fetch_fsmap learned FSMap version 31 2024-05-09T00:53:42.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.977+0000 7f61716b2b80 1 -- 172.21.15.5:0/3879251450 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b831be9a60 con 0x55b831a554a0 2024-05-09T00:53:42.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.980+0000 7f613bfff700 1 -- 172.21.15.5:0/3879251450 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f614c00edf0 con 0x55b831a554a0 2024-05-09T00:53:42.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.980+0000 7f613bfff700 1 -- 172.21.15.5:0/3879251450 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f614c02e070 con 0x55b831a554a0 2024-05-09T00:53:42.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.980+0000 7f61716b2b80 10 client.25819 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:53:42.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.980+0000 7f61716b2b80 10 client.25819 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:53:42.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.980+0000 7f61716b2b80 20 client.25819 populate_metadata read hostname 'smithi005' 2024-05-09T00:53:42.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.980+0000 7f61716b2b80 1 --2- 172.21.15.5:0/3879251450 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55b831bec210 0x55b831bee6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:42.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.980+0000 7f61716b2b80 4 client.25819 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:53:42.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.980+0000 7f61716b2b80 1 -- 172.21.15.5:0/3879251450 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b831b02a80 con 0x55b831bec210 2024-05-09T00:53:42.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.982+0000 7f615bfff700 1 --2- 172.21.15.5:0/3374073698 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f61440b2a90 0x7f61440b4fa0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6150005850 tx=0x7f6150009ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:53:42.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.984+0000 7f615bfff700 1 --2- 172.21.15.5:0/3879251450 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55b831bec210 0x55b831bee6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:42.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.991+0000 7f613bfff700 10 client.25819 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:53:42.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.991+0000 7f615bfff700 1 --2- 172.21.15.5:0/3879251450 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55b831bec210 0x55b831bee6c0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49378ca9970ab732 server_cookie=7f7560a46863e809 in_seq=0 out_seq=0 2024-05-09T00:53:42.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.996+0000 7f613bfff700 1 -- 172.21.15.5:0/3879251450 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b831b02a80 con 0x55b831bec210 2024-05-09T00:53:42.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.996+0000 7f613bfff700 10 client.25819 handle_command_reply: tid=0 2024-05-09T00:53:42.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.996+0000 7f61716b2b80 1 client.25819 shutdown 2024-05-09T00:53:42.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.997+0000 7f61716b2b80 1 -- 172.21.15.5:0/3879251450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b831a554a0 msgr2=0x55b831bd94e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:53:42.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:42.997+0000 7f61716b2b80 1 --2- 172.21.15.5:0/3879251450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b831a554a0 0x55b831bd94e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f614c019000 tx=0x7f614c00efa0 comp rx=0 tx=0).stop 2024-05-09T00:53:43.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.001+0000 7f61716b2b80 1 -- 172.21.15.5:0/3879251450 shutdown_connections 2024-05-09T00:53:43.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.001+0000 7f61716b2b80 1 --2- 172.21.15.5:0/3879251450 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55b831bec210 0x55b831bee6c0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:43.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.001+0000 7f61716b2b80 1 --2- 172.21.15.5:0/3879251450 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b831bdc540 0x55b831bd9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:43.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.001+0000 7f61716b2b80 1 --2- 172.21.15.5:0/3879251450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b831a554a0 0x55b831bd94e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:43.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.001+0000 7f61716b2b80 1 --2- 172.21.15.5:0/3879251450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b831acfd00 0x55b831bd9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:43.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.001+0000 7f61716b2b80 1 -- 172.21.15.5:0/3879251450 >> 172.21.15.5:0/3879251450 conn(0x55b831a58f70 msgr2=0x55b831bcd250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:53:43.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.001+0000 7f613bfff700 0 client.25819 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:53:43.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.002+0000 7f61716b2b80 1 -- 172.21.15.5:0/3879251450 shutdown_connections 2024-05-09T00:53:43.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.002+0000 7f61716b2b80 1 -- 172.21.15.5:0/3879251450 wait complete. 2024-05-09T00:53:43.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.002+0000 7f61716b2b80 20 client.25819 trim_cache size 0 max 16384 2024-05-09T00:53:43.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.072+0000 7f61716b2b80 1 Processor -- start 2024-05-09T00:53:43.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.072+0000 7f61716b2b80 1 -- start start 2024-05-09T00:53:43.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.073+0000 7f61716b2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b831a554a0 0x55b831bdb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:43.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.073+0000 7f61716b2b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b831acfd00 0x55b831be8080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:43.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.073+0000 7f61716b2b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b831bdc540 0x55b831bf3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:43.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.074+0000 7f6160a38700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b831a554a0 0x55b831bdb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:43.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.074+0000 7f6160a38700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b831a554a0 0x55b831bdb980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:38472/0 (socket says 172.21.15.5:38472) 2024-05-09T00:53:43.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.074+0000 7f6160a38700 1 -- 172.21.15.5:0/3046678003 learned_addr learned my addr 172.21.15.5:0/3046678003 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:53:43.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.074+0000 7f61716b2b80 1 -- 172.21.15.5:0/3046678003 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f615c1c6040 con 0x55b831acfd00 2024-05-09T00:53:43.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.074+0000 7f61716b2b80 1 -- 172.21.15.5:0/3046678003 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f615c1c7b80 con 0x55b831a554a0 2024-05-09T00:53:43.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.074+0000 7f61716b2b80 1 -- 172.21.15.5:0/3046678003 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f615c1c6710 con 0x55b831bdc540 2024-05-09T00:53:43.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.075+0000 7f6161239700 1 --2- 172.21.15.5:0/3046678003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b831bdc540 0x55b831bf3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:43.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.077+0000 7f6160a38700 1 -- 172.21.15.5:0/3046678003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b831bdc540 msgr2=0x55b831bf3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:53:43.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.077+0000 7f6160a38700 1 --2- 172.21.15.5:0/3046678003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b831bdc540 0x55b831bf3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:43.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.077+0000 7f6160a38700 1 -- 172.21.15.5:0/3046678003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b831acfd00 msgr2=0x55b831be8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:53:43.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.077+0000 7f6160a38700 1 --2- 172.21.15.5:0/3046678003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b831acfd00 0x55b831be8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:43.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.077+0000 7f6160a38700 1 -- 172.21.15.5:0/3046678003 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b831bc9970 con 0x55b831a554a0 2024-05-09T00:53:43.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.082+0000 7f6160a38700 1 --2- 172.21.15.5:0/3046678003 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b831a554a0 0x55b831bdb980 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f614c009130 tx=0x7f614c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7f667b1c44ddff0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:53:43.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.083+0000 7f61427fc700 1 -- 172.21.15.5:0/3046678003 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f614c017040 con 0x55b831a554a0 2024-05-09T00:53:43.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.083+0000 7f61427fc700 1 -- 172.21.15.5:0/3046678003 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f614c02b070 con 0x55b831a554a0 2024-05-09T00:53:43.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.083+0000 7f61427fc700 1 -- 172.21.15.5:0/3046678003 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f614c027400 con 0x55b831a554a0 2024-05-09T00:53:43.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.085+0000 7f61716b2b80 1 -- 172.21.15.5:0/3046678003 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b831a554a0 msgr2=0x55b831bdb980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:53:43.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.085+0000 7f61716b2b80 1 --2- 172.21.15.5:0/3046678003 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b831a554a0 0x55b831bdb980 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f614c009130 tx=0x7f614c004750 comp rx=0 tx=0).stop 2024-05-09T00:53:43.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.085+0000 7f61716b2b80 1 -- 172.21.15.5:0/3046678003 shutdown_connections 2024-05-09T00:53:43.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.085+0000 7f61716b2b80 1 --2- 172.21.15.5:0/3046678003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b831bdc540 0x55b831bf3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:43.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.085+0000 7f61716b2b80 1 --2- 172.21.15.5:0/3046678003 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b831a554a0 0x55b831bdb980 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:43.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.085+0000 7f61716b2b80 1 --2- 172.21.15.5:0/3046678003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b831acfd00 0x55b831be8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:43.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.089+0000 7f61716b2b80 1 -- 172.21.15.5:0/3046678003 >> 172.21.15.5:0/3046678003 conn(0x55b831a58f70 msgr2=0x55b831bd2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:53:43.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.090+0000 7f61716b2b80 1 -- 172.21.15.5:0/3046678003 shutdown_connections 2024-05-09T00:53:43.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.090+0000 7f61716b2b80 1 -- 172.21.15.5:0/3046678003 wait complete. 2024-05-09T00:53:43.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.091+0000 7f61716b2b80 1 -- start start 2024-05-09T00:53:43.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.092+0000 7f61716b2b80 1 Processor -- start 2024-05-09T00:53:43.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.093+0000 7f61716b2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b831acfd00 0x55b831bdf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:43.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.093+0000 7f61716b2b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b831bdc540 0x55b831bdf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:43.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.093+0000 7f61716b2b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b831be4c00 0x55b831c54a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:43.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.099+0000 7f6160a38700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b831acfd00 0x55b831bdf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:43.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.100+0000 7f6160a38700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b831acfd00 0x55b831bdf3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:38484/0 (socket says 172.21.15.5:38484) 2024-05-09T00:53:43.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.100+0000 7f6161239700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b831be4c00 0x55b831c54a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:43.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.100+0000 7f6161239700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b831be4c00 0x55b831c54a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38162/0 (socket says 172.21.15.5:38162) 2024-05-09T00:53:43.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.100+0000 7f615bfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b831bdc540 0x55b831bdf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:43.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.100+0000 7f6160a38700 1 -- 172.21.15.5:0/1652868483 learned_addr learned my addr 172.21.15.5:0/1652868483 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:53:43.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.101+0000 7f61716b2b80 1 -- 172.21.15.5:0/1652868483 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f615c1c74b0 con 0x55b831be4c00 2024-05-09T00:53:43.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.101+0000 7f61716b2b80 1 -- 172.21.15.5:0/1652868483 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f615c1c6de0 con 0x55b831acfd00 2024-05-09T00:53:43.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.101+0000 7f61716b2b80 1 -- 172.21.15.5:0/1652868483 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f615c1c3fe0 con 0x55b831bdc540 2024-05-09T00:53:43.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.105+0000 7f61716b2b80 10 client.0 _ll_register_callbacks cb 0x55b8319666c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:53:43.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.106+0000 7f6160a38700 1 -- 172.21.15.5:0/1652868483 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b831bdc540 msgr2=0x55b831bdf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:53:43.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.106+0000 7f6160a38700 1 --2- 172.21.15.5:0/1652868483 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b831bdc540 0x55b831bdf900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:43.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.106+0000 7f6160a38700 1 -- 172.21.15.5:0/1652868483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b831be4c00 msgr2=0x55b831c54a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:53:43.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.106+0000 7f6160a38700 1 --2- 172.21.15.5:0/1652868483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b831be4c00 0x55b831c54a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:43.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.106+0000 7f6160a38700 1 -- 172.21.15.5:0/1652868483 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b831be0150 con 0x55b831acfd00 2024-05-09T00:53:43.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.108+0000 7f6160a38700 1 --2- 172.21.15.5:0/1652868483 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b831acfd00 0x55b831bdf3c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f614c009a50 tx=0x7f614c009290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c36b7865fbb0586d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:53:43.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.108+0000 7f613bfff700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-09T00:53:43.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.109+0000 7f613bfff700 1 -- 172.21.15.5:0/1652868483 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f614c017070 con 0x55b831acfd00 2024-05-09T00:53:43.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.110+0000 7f613bfff700 1 -- 172.21.15.5:0/1652868483 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f614c027320 con 0x55b831acfd00 2024-05-09T00:53:43.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.110+0000 7f61716b2b80 1 -- 172.21.15.5:0/1652868483 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f614c02b070 con 0x55b831acfd00 2024-05-09T00:53:43.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.111+0000 7f613bfff700 1 -- 172.21.15.5:0/1652868483 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f614c027870 con 0x55b831acfd00 2024-05-09T00:53:43.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.116+0000 7f61716b2b80 10 client.25802 fetch_fsmap learned FSMap version 31 2024-05-09T00:53:43.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.116+0000 7f61716b2b80 1 -- 172.21.15.5:0/1652868483 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b831c55f10 con 0x55b831acfd00 2024-05-09T00:53:43.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.117+0000 7f613bfff700 1 -- 172.21.15.5:0/1652868483 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f614c041070 con 0x55b831acfd00 2024-05-09T00:53:43.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.117+0000 7f613bfff700 1 -- 172.21.15.5:0/1652868483 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f614c044070 con 0x55b831acfd00 2024-05-09T00:53:43.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.122+0000 7f61716b2b80 10 client.25802 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:53:43.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.122+0000 7f61716b2b80 10 client.25802 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:53:43.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.122+0000 7f61716b2b80 20 client.25802 populate_metadata read hostname 'smithi005' 2024-05-09T00:53:43.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.122+0000 7f61716b2b80 1 --2- 172.21.15.5:0/1652868483 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55b831c564b0 0x55b831c588d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:53:43.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.122+0000 7f61716b2b80 4 client.25802 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:53:43.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.122+0000 7f61716b2b80 1 -- 172.21.15.5:0/1652868483 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f614c02b070 con 0x55b831c564b0 2024-05-09T00:53:43.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.128+0000 7f615bfff700 1 --2- 172.21.15.5:0/1652868483 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55b831c564b0 0x55b831c588d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:53:43.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.143+0000 7f615bfff700 1 --2- 172.21.15.5:0/1652868483 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55b831c564b0 0x55b831c588d0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b884b124bd03ee28 server_cookie=bf1f0ff524280eb3 in_seq=0 out_seq=0 2024-05-09T00:53:43.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.143+0000 7f613bfff700 10 client.25802 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:53:43.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.148+0000 7f613bfff700 1 -- 172.21.15.5:0/1652868483 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f614c02b070 con 0x55b831c564b0 2024-05-09T00:53:43.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.148+0000 7f613bfff700 10 client.25802 handle_command_reply: tid=0 2024-05-09T00:53:43.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.148+0000 7f61716b2b80 1 client.25802 shutdown 2024-05-09T00:53:43.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.148+0000 7f61716b2b80 1 -- 172.21.15.5:0/1652868483 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b831acfd00 msgr2=0x55b831bdf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:53:43.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.148+0000 7f61716b2b80 1 --2- 172.21.15.5:0/1652868483 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b831acfd00 0x55b831bdf3c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f614c009a50 tx=0x7f614c009290 comp rx=0 tx=0).stop 2024-05-09T00:53:43.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.149+0000 7f61716b2b80 1 -- 172.21.15.5:0/1652868483 shutdown_connections 2024-05-09T00:53:43.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.149+0000 7f61716b2b80 1 --2- 172.21.15.5:0/1652868483 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55b831c564b0 0x55b831c588d0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:43.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.149+0000 7f61716b2b80 1 --2- 172.21.15.5:0/1652868483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b831be4c00 0x55b831c54a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:43.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.149+0000 7f61716b2b80 1 --2- 172.21.15.5:0/1652868483 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b831acfd00 0x55b831bdf3c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:43.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.149+0000 7f61716b2b80 1 --2- 172.21.15.5:0/1652868483 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b831bdc540 0x55b831bdf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:43.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.149+0000 7f613bfff700 0 client.25802 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:53:43.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.149+0000 7f61716b2b80 1 -- 172.21.15.5:0/1652868483 >> 172.21.15.5:0/1652868483 conn(0x55b831a58f70 msgr2=0x55b831bd2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:53:43.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.150+0000 7f61716b2b80 1 -- 172.21.15.5:0/1652868483 shutdown_connections 2024-05-09T00:53:43.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.150+0000 7f61716b2b80 1 -- 172.21.15.5:0/1652868483 wait complete. 2024-05-09T00:53:43.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.151+0000 7f61716b2b80 20 client.25802 trim_cache size 0 max 16384 2024-05-09T00:53:43.157 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-09T00:53:43.158 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T00:53:43.158 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-05-09T00:53:43.158 INFO:teuthology.orchestra.run.smithi005.stdout: "2499ed0b-1624-4dbc-aba7-2efa8d4e624e": { 2024-05-09T00:53:43.158 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-05-09T00:53:43.158 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "2499ed0b-1624-4dbc-aba7-2efa8d4e624e", 2024-05-09T00:53:43.158 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-05-09T00:53:43.158 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T00:53:43.158 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T00:53:43.158 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-09T00:53:43.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.158+0000 7f6161a3a700 1 -- 172.21.15.5:0/3374073698 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f61440b2a90 msgr2=0x7f61440b4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:53:43.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.158+0000 7f6161a3a700 1 --2- 172.21.15.5:0/3374073698 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f61440b2a90 0x7f61440b4fa0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6150005850 tx=0x7f6150009ee0 comp rx=0 tx=0).stop 2024-05-09T00:53:43.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.158+0000 7f6161a3a700 1 -- 172.21.15.5:0/3374073698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f615c109660 msgr2=0x7f615c06eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:53:43.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.158+0000 7f6161a3a700 1 --2- 172.21.15.5:0/3374073698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f615c109660 0x7f615c06eb10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6154009f90 tx=0x7f615400cd70 comp rx=0 tx=0).stop 2024-05-09T00:53:43.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.160+0000 7f6161a3a700 1 -- 172.21.15.5:0/3374073698 shutdown_connections 2024-05-09T00:53:43.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.160+0000 7f6161a3a700 1 --2- 172.21.15.5:0/3374073698 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f61440b2a90 0x7f61440b4fa0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:43.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.160+0000 7f6161a3a700 1 --2- 172.21.15.5:0/3374073698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f615c109660 0x7f615c06eb10 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:43.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.160+0000 7f6161a3a700 1 --2- 172.21.15.5:0/3374073698 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f615c1043e0 0x7f615c0763f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:43.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.160+0000 7f6161a3a700 1 --2- 172.21.15.5:0/3374073698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f615c108d10 0x7f615c06e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:43.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.160+0000 7f6161a3a700 1 -- 172.21.15.5:0/3374073698 >> 172.21.15.5:0/3374073698 conn(0x7f615c079d10 msgr2=0x7f615c0774e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:53:43.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.161+0000 7f6161a3a700 1 -- 172.21.15.5:0/3374073698 shutdown_connections 2024-05-09T00:53:43.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:53:43.161+0000 7f6161a3a700 1 -- 172.21.15.5:0/3374073698 wait complete. 2024-05-09T00:53:43.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:53:43 smithi005 ceph-mon[30365]: pgmap v592: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 860 MiB/s wr, 1.42k op/s 2024-05-09T00:53:43.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:53:43 smithi131 ceph-mon[34547]: pgmap v592: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 860 MiB/s wr, 1.42k op/s 2024-05-09T00:53:43.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:53:43 smithi157 ceph-mon[34294]: pgmap v592: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 860 MiB/s wr, 1.42k op/s 2024-05-09T00:53:45.481 INFO:tasks.cephfs.filesystem:scrub status for tag:2499ed0b-1624-4dbc-aba7-2efa8d4e624e - {'path': '/', 'tag': '2499ed0b-1624-4dbc-aba7-2efa8d4e624e', 'options': 'recursive,force'} 2024-05-09T00:53:45.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:53:45 smithi131 ceph-mon[34547]: pgmap v593: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 886 MiB/s wr, 1.48k op/s 2024-05-09T00:53:45.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:53:45 smithi157 ceph-mon[34294]: pgmap v593: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 886 MiB/s wr, 1.48k op/s 2024-05-09T00:53:45.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:53:45 smithi005 ceph-mon[30365]: pgmap v593: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 886 MiB/s wr, 1.48k op/s 2024-05-09T00:53:46.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:53:46 smithi131 ceph-mon[34547]: scrub summary: idle 2024-05-09T00:53:46.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:53:46 smithi157 ceph-mon[34294]: scrub summary: idle 2024-05-09T00:53:46.798 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:53:46 smithi005 ceph-mon[30365]: scrub summary: idle 2024-05-09T00:53:47.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:53:47 smithi131 ceph-mon[34547]: pgmap v594: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 909 MiB/s wr, 1.52k op/s 2024-05-09T00:53:47.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:53:47 smithi131 ceph-mon[34547]: scrub summary: idle 2024-05-09T00:53:47.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:53:47 smithi157 ceph-mon[34294]: pgmap v594: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 909 MiB/s wr, 1.52k op/s 2024-05-09T00:53:47.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:53:47 smithi157 ceph-mon[34294]: scrub summary: idle 2024-05-09T00:53:47.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:53:47 smithi005 ceph-mon[30365]: pgmap v594: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 909 MiB/s wr, 1.52k op/s 2024-05-09T00:53:47.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:53:47 smithi005 ceph-mon[30365]: scrub summary: idle 2024-05-09T00:53:49.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:53:49 smithi131 ceph-mon[34547]: pgmap v595: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 861 MiB/s wr, 1.45k op/s 2024-05-09T00:53:49.685 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:53:49 smithi157 ceph-mon[34294]: pgmap v595: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 861 MiB/s wr, 1.45k op/s 2024-05-09T00:53:49.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:53:49 smithi005 ceph-mon[30365]: pgmap v595: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 861 MiB/s wr, 1.45k op/s 2024-05-09T00:53:50.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:50.728+0000 7ff9337fe640 1 -- 172.21.15.5:0/2537716282 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56425ea69350 con 0x56425e7721c0 2024-05-09T00:53:50.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:50.728+0000 7ff9337fe640 10 client.16275 handle_command_reply: tid=0 2024-05-09T00:53:50.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:50.728+0000 7ff96122b740 1 client.16275 shutdown 2024-05-09T00:53:50.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:50.732+0000 7ff96122b740 1 -- 172.21.15.5:0/2537716282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56425e7b52f0 msgr2=0x56425eb8fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:53:50.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:50.732+0000 7ff96122b740 1 --2- 172.21.15.5:0/2537716282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56425e7b52f0 0x56425eb8fe70 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7ff9400054f0 tx=0x7ff94000ffc0 comp rx=0 tx=0).stop 2024-05-09T00:53:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:50.739+0000 7ff96122b740 1 -- 172.21.15.5:0/2537716282 shutdown_connections 2024-05-09T00:53:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:50.739+0000 7ff96122b740 1 --2- 172.21.15.5:0/2537716282 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56425e7721c0 0x56425ec15680 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:50.739+0000 7ff96122b740 1 --2- 172.21.15.5:0/2537716282 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56425e7de1b0 0x56425ec11900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:50.739+0000 7ff96122b740 1 --2- 172.21.15.5:0/2537716282 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56425e7bd4d0 0x56425eb903b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:50.739+0000 7ff96122b740 1 --2- 172.21.15.5:0/2537716282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56425e7b52f0 0x56425eb8fe70 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:50.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:50.739+0000 7ff96122b740 1 -- 172.21.15.5:0/2537716282 >> 172.21.15.5:0/2537716282 conn(0x56425e7d2300 msgr2=0x56425eb82640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:53:50.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:50.739+0000 7ff9337fe640 0 client.16275 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:53:50.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:50.741+0000 7ff96122b740 1 -- 172.21.15.5:0/2537716282 shutdown_connections 2024-05-09T00:53:50.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:50.742+0000 7ff96122b740 1 -- 172.21.15.5:0/2537716282 wait complete. 2024-05-09T00:53:50.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:50.745+0000 7ff96122b740 20 client.16275 trim_cache size 0 max 16384 2024-05-09T00:53:50.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-09T00:53:50.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-09T00:53:50.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-09T00:53:50.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:50.752+0000 7ff95a376640 1 -- 172.21.15.5:0/1257626955 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ff9440b2a00 msgr2=0x7ff9440b4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:53:50.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:50.752+0000 7ff95a376640 1 --2- 172.21.15.5:0/1257626955 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ff9440b2a00 0x7ff9440b4f10 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff954052170 tx=0x7ff954068f90 comp rx=0 tx=0).stop 2024-05-09T00:53:50.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:50.752+0000 7ff95a376640 1 -- 172.21.15.5:0/1257626955 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff94c096b20 msgr2=0x7ff94c0a4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:53:50.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:50.752+0000 7ff95a376640 1 --2- 172.21.15.5:0/1257626955 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff94c096b20 0x7ff94c0a4e80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff94800c970 tx=0x7ff94800ce30 comp rx=0 tx=0).stop 2024-05-09T00:53:50.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:50.756+0000 7ff95a376640 1 -- 172.21.15.5:0/1257626955 shutdown_connections 2024-05-09T00:53:50.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:50.756+0000 7ff95a376640 1 --2- 172.21.15.5:0/1257626955 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ff9440b2a00 0x7ff9440b4f10 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:50.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:50.756+0000 7ff95a376640 1 --2- 172.21.15.5:0/1257626955 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff94c09b820 0x7ff94c0a53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:50.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:50.756+0000 7ff95a376640 1 --2- 172.21.15.5:0/1257626955 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff94c096b20 0x7ff94c0a4e80 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:50.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:50.756+0000 7ff95a376640 1 --2- 172.21.15.5:0/1257626955 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff94c0958e0 0x7ff94c0a67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:53:50.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:50.756+0000 7ff95a376640 1 -- 172.21.15.5:0/1257626955 >> 172.21.15.5:0/1257626955 conn(0x7ff94c090ab0 msgr2=0x7ff94c097800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:53:50.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:50.764+0000 7ff95a376640 1 -- 172.21.15.5:0/1257626955 shutdown_connections 2024-05-09T00:53:50.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:53:50.764+0000 7ff95a376640 1 -- 172.21.15.5:0/1257626955 wait complete. 2024-05-09T00:53:51.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:53:51 smithi157 ceph-mon[34294]: pgmap v596: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 981 MiB/s wr, 1.91k op/s 2024-05-09T00:53:51.804 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:53:51 smithi005 ceph-mon[30365]: pgmap v596: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 981 MiB/s wr, 1.91k op/s 2024-05-09T00:53:51.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:53:51 smithi131 ceph-mon[34547]: pgmap v596: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 981 MiB/s wr, 1.91k op/s 2024-05-09T00:53:53.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:53:53 smithi157 ceph-mon[34294]: pgmap v597: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 852 MiB/s wr, 1.79k op/s 2024-05-09T00:53:53.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:53:53 smithi005 ceph-mon[30365]: pgmap v597: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 852 MiB/s wr, 1.79k op/s 2024-05-09T00:53:53.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:53:53 smithi131 ceph-mon[34547]: pgmap v597: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 852 MiB/s wr, 1.79k op/s 2024-05-09T00:53:55.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:53:55 smithi157 ceph-mon[34294]: pgmap v598: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 878 MiB/s wr, 1.83k op/s 2024-05-09T00:53:55.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:53:55 smithi005 ceph-mon[30365]: pgmap v598: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 878 MiB/s wr, 1.83k op/s 2024-05-09T00:53:55.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:53:55 smithi131 ceph-mon[34547]: pgmap v598: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 878 MiB/s wr, 1.83k op/s 2024-05-09T00:53:57.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:53:57 smithi157 ceph-mon[34294]: pgmap v599: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 899 MiB/s wr, 1.79k op/s 2024-05-09T00:53:57.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:53:57 smithi005 ceph-mon[30365]: pgmap v599: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 899 MiB/s wr, 1.79k op/s 2024-05-09T00:53:57.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:53:57 smithi131 ceph-mon[34547]: pgmap v599: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 899 MiB/s wr, 1.79k op/s 2024-05-09T00:53:59.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:53:59 smithi157 ceph-mon[34294]: pgmap v600: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 849 MiB/s wr, 1.75k op/s 2024-05-09T00:53:59.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:53:59 smithi005 ceph-mon[30365]: pgmap v600: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 849 MiB/s wr, 1.75k op/s 2024-05-09T00:53:59.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:53:59 smithi131 ceph-mon[34547]: pgmap v600: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 849 MiB/s wr, 1.75k op/s 2024-05-09T00:54:01.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:54:01 smithi157 ceph-mon[34294]: pgmap v601: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 984 MiB/s wr, 2.14k op/s 2024-05-09T00:54:01.790 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:54:01 smithi005 ceph-mon[30365]: pgmap v601: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 984 MiB/s wr, 2.14k op/s 2024-05-09T00:54:01.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:54:01 smithi131 ceph-mon[34547]: pgmap v601: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 984 MiB/s wr, 2.14k op/s 2024-05-09T00:54:03.685 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:54:03 smithi157 ceph-mon[34294]: pgmap v602: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 866 MiB/s wr, 1.69k op/s 2024-05-09T00:54:03.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:54:03 smithi005 ceph-mon[30365]: pgmap v602: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 866 MiB/s wr, 1.69k op/s 2024-05-09T00:54:03.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:54:03 smithi131 ceph-mon[34547]: pgmap v602: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 866 MiB/s wr, 1.69k op/s 2024-05-09T00:54:04.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.150+0000 7fdee2ad2640 1 -- 172.21.15.5:0/1141316426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fded40a5c30 msgr2=0x7fded40a6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:54:04.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.150+0000 7fdee2ad2640 1 --2- 172.21.15.5:0/1141316426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fded40a5c30 0x7fded40a6930 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fded80074e0 tx=0x7fded802b080 comp rx=0 tx=0).stop 2024-05-09T00:54:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.153+0000 7fdee2ad2640 1 -- 172.21.15.5:0/1141316426 shutdown_connections 2024-05-09T00:54:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.153+0000 7fdee2ad2640 1 --2- 172.21.15.5:0/1141316426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fded40abb70 0x7fded40adf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.153+0000 7fdee2ad2640 1 --2- 172.21.15.5:0/1141316426 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fded40a6e70 0x7fded40a7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:04.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.153+0000 7fdee2ad2640 1 --2- 172.21.15.5:0/1141316426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fded40a5c30 0x7fded40a6930 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:04.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.153+0000 7fdee2ad2640 1 -- 172.21.15.5:0/1141316426 >> 172.21.15.5:0/1141316426 conn(0x7fded40a0e00 msgr2=0x7fded40a32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:54:04.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.153+0000 7fdee2ad2640 1 -- 172.21.15.5:0/1141316426 shutdown_connections 2024-05-09T00:54:04.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.154+0000 7fdee2ad2640 1 -- 172.21.15.5:0/1141316426 wait complete. 2024-05-09T00:54:04.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.155+0000 7fdee2ad2640 1 Processor -- start 2024-05-09T00:54:04.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.155+0000 7fdee2ad2640 1 -- start start 2024-05-09T00:54:04.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.156+0000 7fdee2ad2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fded40a5c30 0x7fded413bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:04.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.156+0000 7fdee2ad2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fded40a6e70 0x7fded413c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:04.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.156+0000 7fdee2ad2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fded40abb70 0x7fded41545e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:04.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.156+0000 7fdee2ad2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fded40b12d0 con 0x7fded40a6e70 2024-05-09T00:54:04.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.156+0000 7fdee2ad2640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fded40b1150 con 0x7fded40abb70 2024-05-09T00:54:04.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.156+0000 7fdee2ad2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fded40b1450 con 0x7fded40a5c30 2024-05-09T00:54:04.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.156+0000 7fdee1ad0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fded40a5c30 0x7fded413bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:04.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.159+0000 7fdee12cf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fded40a6e70 0x7fded413c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:04.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.159+0000 7fdee1ad0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fded40a5c30 0x7fded413bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:45802/0 (socket says 172.21.15.5:45802) 2024-05-09T00:54:04.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.159+0000 7fdee12cf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fded40a6e70 0x7fded413c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43072/0 (socket says 172.21.15.5:43072) 2024-05-09T00:54:04.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.159+0000 7fdee1ad0640 1 -- 172.21.15.5:0/1249479705 learned_addr learned my addr 172.21.15.5:0/1249479705 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:54:04.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.160+0000 7fdee22d1640 1 --2- 172.21.15.5:0/1249479705 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fded40abb70 0x7fded41545e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:04.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.163+0000 7fdee12cf640 1 -- 172.21.15.5:0/1249479705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fded40a5c30 msgr2=0x7fded413bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:54:04.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.163+0000 7fdee12cf640 1 --2- 172.21.15.5:0/1249479705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fded40a5c30 0x7fded413bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:04.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.163+0000 7fdee12cf640 1 -- 172.21.15.5:0/1249479705 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fded40abb70 msgr2=0x7fded41545e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:54:04.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.163+0000 7fdee12cf640 1 --2- 172.21.15.5:0/1249479705 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fded40abb70 0x7fded41545e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:04.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.163+0000 7fdee12cf640 1 -- 172.21.15.5:0/1249479705 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fded4154b20 con 0x7fded40a6e70 2024-05-09T00:54:04.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.165+0000 7fdee12cf640 1 --2- 172.21.15.5:0/1249479705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fded40a6e70 0x7fded413c440 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fdecc00e9f0 tx=0x7fdecc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:54:04.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.168+0000 7fded2ffd640 1 -- 172.21.15.5:0/1249479705 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdecc00ccc0 con 0x7fded40a6e70 2024-05-09T00:54:04.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.169+0000 7fded2ffd640 1 -- 172.21.15.5:0/1249479705 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdecc0108e0 con 0x7fded40a6e70 2024-05-09T00:54:04.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.169+0000 7fdee2ad2640 1 -- 172.21.15.5:0/1249479705 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fded4154d50 con 0x7fded40a6e70 2024-05-09T00:54:04.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.169+0000 7fdee2ad2640 1 -- 172.21.15.5:0/1249479705 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fded4155190 con 0x7fded40a6e70 2024-05-09T00:54:04.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.169+0000 7fded2ffd640 1 -- 172.21.15.5:0/1249479705 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdecc0188e0 con 0x7fded40a6e70 2024-05-09T00:54:04.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.180+0000 7fded2ffd640 1 -- 172.21.15.5:0/1249479705 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fdecc010430 con 0x7fded40a6e70 2024-05-09T00:54:04.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.185+0000 7fdeeabe4740 1 Processor -- start 2024-05-09T00:54:04.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.186+0000 7fded2ffd640 1 --2- 172.21.15.5:0/1249479705 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fdec80b29a0 0x7fdec80b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:04.188 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.186+0000 7fded2ffd640 1 -- 172.21.15.5:0/1249479705 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fdecc0e2600 con 0x7fded40a6e70 2024-05-09T00:54:04.190 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.188+0000 7fdee1ad0640 1 --2- 172.21.15.5:0/1249479705 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fdec80b29a0 0x7fdec80b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:04.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.189+0000 7fdeeabe4740 1 -- start start 2024-05-09T00:54:04.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.190+0000 7fdeeabe4740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56198982f1b0 0x561989bd36c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:04.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.190+0000 7fdeeabe4740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619898062f0 0x561989bd3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:04.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.190+0000 7fdeeabe4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56198980e4d0 0x561989bdab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:04.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.190+0000 7fdeeabe4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561989b377c0 con 0x56198980e4d0 2024-05-09T00:54:04.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.190+0000 7fdeeabe4740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x561989ad30e0 con 0x5619898062f0 2024-05-09T00:54:04.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.190+0000 7fdeeabe4740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x561989b33780 con 0x56198982f1b0 2024-05-09T00:54:04.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.191+0000 7fdee12cf640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619898062f0 0x561989bd3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:04.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.191+0000 7fdee12cf640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619898062f0 0x561989bd3c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:48208/0 (socket says 172.21.15.5:48208) 2024-05-09T00:54:04.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.191+0000 7fdee12cf640 1 -- 172.21.15.5:0/3470909569 learned_addr learned my addr 172.21.15.5:0/3470909569 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:54:04.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.192+0000 7fdee1ad0640 1 --2- 172.21.15.5:0/3470909569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56198982f1b0 0x561989bd36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.193+0000 7fdee1ad0640 1 -- 172.21.15.5:0/3470909569 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619898062f0 msgr2=0x561989bd3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:54:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.193+0000 7fdee1ad0640 1 --2- 172.21.15.5:0/3470909569 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619898062f0 0x561989bd3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.193+0000 7fdee1ad0640 1 -- 172.21.15.5:0/3470909569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56198980e4d0 msgr2=0x561989bdab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:54:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.193+0000 7fdee1ad0640 1 --2- 172.21.15.5:0/3470909569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56198980e4d0 0x561989bdab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.193+0000 7fdee1ad0640 1 -- 172.21.15.5:0/3470909569 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561989afc7c0 con 0x56198982f1b0 2024-05-09T00:54:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.194+0000 7fdee1ad0640 1 --2- 172.21.15.5:0/1249479705 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fdec80b29a0 0x7fdec80b4eb0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fded802b5a0 tx=0x7fded8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:54:04.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.194+0000 7fdee1ad0640 1 --2- 172.21.15.5:0/3470909569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56198982f1b0 0x561989bd36c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fded8041000 tx=0x7fded8042000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=961f279956476467 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:54:04.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.199+0000 7fdec7fff640 1 -- 172.21.15.5:0/3470909569 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded804d070 con 0x56198982f1b0 2024-05-09T00:54:04.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.199+0000 7fdec7fff640 1 -- 172.21.15.5:0/3470909569 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fded800a580 con 0x56198982f1b0 2024-05-09T00:54:04.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.201+0000 7fdeeabe4740 1 -- 172.21.15.5:0/3470909569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56198982f1b0 msgr2=0x561989bd36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:54:04.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.201+0000 7fdeeabe4740 1 --2- 172.21.15.5:0/3470909569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56198982f1b0 0x561989bd36c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fded8041000 tx=0x7fded8042000 comp rx=0 tx=0).stop 2024-05-09T00:54:04.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.201+0000 7fdeeabe4740 1 -- 172.21.15.5:0/3470909569 shutdown_connections 2024-05-09T00:54:04.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.201+0000 7fdeeabe4740 1 --2- 172.21.15.5:0/3470909569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56198980e4d0 0x561989bdab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:04.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.201+0000 7fdeeabe4740 1 --2- 172.21.15.5:0/3470909569 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619898062f0 0x561989bd3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:04.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.201+0000 7fdeeabe4740 1 --2- 172.21.15.5:0/3470909569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56198982f1b0 0x561989bd36c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:04.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.201+0000 7fdeeabe4740 1 -- 172.21.15.5:0/3470909569 >> 172.21.15.5:0/3470909569 conn(0x561989823300 msgr2=0x561989bcf050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:54:04.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.202+0000 7fdeeabe4740 1 -- 172.21.15.5:0/3470909569 shutdown_connections 2024-05-09T00:54:04.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.202+0000 7fdeeabe4740 1 -- 172.21.15.5:0/3470909569 wait complete. 2024-05-09T00:54:04.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.203+0000 7fdeeabe4740 1 -- start start 2024-05-09T00:54:04.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.203+0000 7fdeeabe4740 1 Processor -- start 2024-05-09T00:54:04.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.204+0000 7fdeeabe4740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619898062f0 0x561989bdd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:04.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.205+0000 7fdeeabe4740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56198980e4d0 0x561989bdd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:04.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.205+0000 7fdeeabe4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56198982f1b0 0x561989be4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:04.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.205+0000 7fdeeabe4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561989bdb580 con 0x56198982f1b0 2024-05-09T00:54:04.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.205+0000 7fdeeabe4740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x561989abe9c0 con 0x5619898062f0 2024-05-09T00:54:04.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.205+0000 7fdeeabe4740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x561989bdb700 con 0x56198980e4d0 2024-05-09T00:54:04.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.205+0000 7fdee22d1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56198982f1b0 0x561989be4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:04.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.206+0000 7fdee22d1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56198982f1b0 0x561989be4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43098/0 (socket says 172.21.15.5:43098) 2024-05-09T00:54:04.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.206+0000 7fdee22d1640 1 -- 172.21.15.5:0/1943111198 learned_addr learned my addr 172.21.15.5:0/1943111198 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:54:04.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.206+0000 7fdeeabe4740 10 client.0 _ll_register_callbacks cb 0x561989aeb020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:54:04.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.206+0000 7fdee12cf640 1 --2- 172.21.15.5:0/1943111198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56198980e4d0 0x561989bdd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:04.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.206+0000 7fdee22d1640 1 -- 172.21.15.5:0/1943111198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56198980e4d0 msgr2=0x561989bdd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:54:04.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.206+0000 7fdee22d1640 1 --2- 172.21.15.5:0/1943111198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56198980e4d0 0x561989bdd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:04.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.206+0000 7fdee22d1640 1 -- 172.21.15.5:0/1943111198 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619898062f0 msgr2=0x561989bdd3d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:54:04.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.206+0000 7fdee22d1640 1 --2- 172.21.15.5:0/1943111198 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619898062f0 0x561989bdd3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:04.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.206+0000 7fdee22d1640 1 -- 172.21.15.5:0/1943111198 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561989ae51f0 con 0x56198982f1b0 2024-05-09T00:54:04.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.207+0000 7fdee22d1640 1 --2- 172.21.15.5:0/1943111198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56198982f1b0 0x561989be4870 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fdedc051620 tx=0x7fdedc0746c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24bcaecfabff5f76 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:54:04.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.207+0000 7fdec57fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:54:04.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.208+0000 7fdec57fa640 1 -- 172.21.15.5:0/1943111198 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdedc07b070 con 0x56198982f1b0 2024-05-09T00:54:04.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.208+0000 7fdec57fa640 1 -- 172.21.15.5:0/1943111198 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fdedc077470 con 0x56198982f1b0 2024-05-09T00:54:04.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.209+0000 7fdeeabe4740 1 -- 172.21.15.5:0/1943111198 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fded800a580 con 0x56198982f1b0 2024-05-09T00:54:04.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.210+0000 7fdec57fa640 1 -- 172.21.15.5:0/1943111198 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fdedc084050 con 0x56198982f1b0 2024-05-09T00:54:04.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.210+0000 7fdeeabe4740 10 client.16287 fetch_fsmap learned FSMap version 31 2024-05-09T00:54:04.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.210+0000 7fdeeabe4740 1 -- 172.21.15.5:0/1943111198 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56198989a790 con 0x56198982f1b0 2024-05-09T00:54:04.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.213+0000 7fdec57fa640 1 -- 172.21.15.5:0/1943111198 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fded800a580 con 0x56198982f1b0 2024-05-09T00:54:04.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.213+0000 7fdec57fa640 1 -- 172.21.15.5:0/1943111198 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fdedc087020 con 0x56198982f1b0 2024-05-09T00:54:04.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.215+0000 7fdeeabe4740 10 client.16287 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:54:04.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.215+0000 7fdeeabe4740 10 client.16287 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:54:04.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.215+0000 7fdeeabe4740 20 client.16287 populate_metadata read hostname 'smithi005' 2024-05-09T00:54:04.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.215+0000 7fdeeabe4740 1 --2- 172.21.15.5:0/1943111198 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5619897edd80 0x561989beada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:04.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.215+0000 7fdeeabe4740 4 client.16287 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:54:04.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.215+0000 7fdeeabe4740 1 -- 172.21.15.5:0/1943111198 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561989b453d0 con 0x5619897edd80 2024-05-09T00:54:04.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.217+0000 7fdee1ad0640 1 --2- 172.21.15.5:0/1943111198 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5619897edd80 0x561989beada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:04.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.224+0000 7fdee1ad0640 1 --2- 172.21.15.5:0/1943111198 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5619897edd80 0x561989beada0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91afd90a18e73bdf server_cookie=d7fbd855ce83f8e5 in_seq=0 out_seq=0 2024-05-09T00:54:04.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.224+0000 7fdec57fa640 10 client.16287 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:54:04.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.234+0000 7fdec57fa640 1 -- 172.21.15.5:0/1943111198 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561989b453d0 con 0x5619897edd80 2024-05-09T00:54:04.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.234+0000 7fdec57fa640 10 client.16287 handle_command_reply: tid=0 2024-05-09T00:54:04.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.234+0000 7fdeeabe4740 1 client.16287 shutdown 2024-05-09T00:54:04.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.235+0000 7fdeeabe4740 1 -- 172.21.15.5:0/1943111198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56198982f1b0 msgr2=0x561989be4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:54:04.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.235+0000 7fdeeabe4740 1 --2- 172.21.15.5:0/1943111198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56198982f1b0 0x561989be4870 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fdedc051620 tx=0x7fdedc0746c0 comp rx=0 tx=0).stop 2024-05-09T00:54:04.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.236+0000 7fdeeabe4740 1 -- 172.21.15.5:0/1943111198 shutdown_connections 2024-05-09T00:54:04.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.236+0000 7fdeeabe4740 1 --2- 172.21.15.5:0/1943111198 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5619897edd80 0x561989beada0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:04.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.236+0000 7fdeeabe4740 1 --2- 172.21.15.5:0/1943111198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56198982f1b0 0x561989be4870 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:04.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.236+0000 7fdeeabe4740 1 --2- 172.21.15.5:0/1943111198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56198980e4d0 0x561989bdd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:04.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.236+0000 7fdeeabe4740 1 --2- 172.21.15.5:0/1943111198 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619898062f0 0x561989bdd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:04.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.236+0000 7fdeeabe4740 1 -- 172.21.15.5:0/1943111198 >> 172.21.15.5:0/1943111198 conn(0x561989823300 msgr2=0x561989bce290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:54:04.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.236+0000 7fdec57fa640 0 client.16287 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:54:04.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.236+0000 7fdeeabe4740 1 -- 172.21.15.5:0/1943111198 shutdown_connections 2024-05-09T00:54:04.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.237+0000 7fdeeabe4740 1 -- 172.21.15.5:0/1943111198 wait complete. 2024-05-09T00:54:04.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.238+0000 7fdeeabe4740 20 client.16287 trim_cache size 0 max 16384 2024-05-09T00:54:04.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.278+0000 7fdeeabe4740 1 Processor -- start 2024-05-09T00:54:04.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.278+0000 7fdeeabe4740 1 -- start start 2024-05-09T00:54:04.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.278+0000 7fdeeabe4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5619898062f0 0x561989bd2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:04.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.278+0000 7fdeeabe4740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56198980e4d0 0x561989bd2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:04.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.278+0000 7fdeeabe4740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56198982f1b0 0x561989bef590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:04.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.278+0000 7fdeeabe4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdec000e690 con 0x5619898062f0 2024-05-09T00:54:04.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.278+0000 7fdeeabe4740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fdec00101d0 con 0x56198980e4d0 2024-05-09T00:54:04.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.278+0000 7fdeeabe4740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdec000ed60 con 0x56198982f1b0 2024-05-09T00:54:04.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.279+0000 7fdee22d1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56198982f1b0 0x561989bef590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:04.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.279+0000 7fdee22d1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56198982f1b0 0x561989bef590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:45810/0 (socket says 172.21.15.5:45810) 2024-05-09T00:54:04.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.279+0000 7fdee22d1640 1 -- 172.21.15.5:0/3251093674 learned_addr learned my addr 172.21.15.5:0/3251093674 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:54:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.280+0000 7fdee22d1640 1 -- 172.21.15.5:0/3251093674 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56198980e4d0 msgr2=0x561989bd2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:54:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.280+0000 7fdee22d1640 1 --2- 172.21.15.5:0/3251093674 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56198980e4d0 0x561989bd2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.280+0000 7fdee22d1640 1 -- 172.21.15.5:0/3251093674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5619898062f0 msgr2=0x561989bd2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:54:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.280+0000 7fdee22d1640 1 --2- 172.21.15.5:0/3251093674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5619898062f0 0x561989bd2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.280+0000 7fdee22d1640 1 -- 172.21.15.5:0/3251093674 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619899115f0 con 0x56198982f1b0 2024-05-09T00:54:04.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.281+0000 7fdee22d1640 1 --2- 172.21.15.5:0/3251093674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56198982f1b0 0x561989bef590 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdedc065d90 tx=0x7fdedc06bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ae45435fd194d07 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:54:04.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.282+0000 7fdec77fe640 1 -- 172.21.15.5:0/3251093674 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdedc07b050 con 0x56198982f1b0 2024-05-09T00:54:04.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.282+0000 7fdec77fe640 1 -- 172.21.15.5:0/3251093674 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdedc07fd30 con 0x56198982f1b0 2024-05-09T00:54:04.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.283+0000 7fdec77fe640 1 -- 172.21.15.5:0/3251093674 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdedc077cc0 con 0x56198982f1b0 2024-05-09T00:54:04.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.284+0000 7fdeeabe4740 1 -- 172.21.15.5:0/3251093674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56198982f1b0 msgr2=0x561989bef590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:54:04.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.284+0000 7fdeeabe4740 1 --2- 172.21.15.5:0/3251093674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56198982f1b0 0x561989bef590 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdedc065d90 tx=0x7fdedc06bfd0 comp rx=0 tx=0).stop 2024-05-09T00:54:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.284+0000 7fdeeabe4740 1 -- 172.21.15.5:0/3251093674 shutdown_connections 2024-05-09T00:54:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.284+0000 7fdeeabe4740 1 --2- 172.21.15.5:0/3251093674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56198982f1b0 0x561989bef590 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.284+0000 7fdeeabe4740 1 --2- 172.21.15.5:0/3251093674 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56198980e4d0 0x561989bd2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.284+0000 7fdeeabe4740 1 --2- 172.21.15.5:0/3251093674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5619898062f0 0x561989bd2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.284+0000 7fdeeabe4740 1 -- 172.21.15.5:0/3251093674 >> 172.21.15.5:0/3251093674 conn(0x561989823300 msgr2=0x561989bd1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:54:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.284+0000 7fdeeabe4740 1 -- 172.21.15.5:0/3251093674 shutdown_connections 2024-05-09T00:54:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.285+0000 7fdeeabe4740 1 -- 172.21.15.5:0/3251093674 wait complete. 2024-05-09T00:54:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.286+0000 7fdeeabe4740 1 -- start start 2024-05-09T00:54:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.286+0000 7fdeeabe4740 1 Processor -- start 2024-05-09T00:54:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.288+0000 7fdeeabe4740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619898062f0 0x561989be0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.288+0000 7fdeeabe4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56198980e4d0 0x561989be10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.288+0000 7fdeeabe4740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56198982f1b0 0x561989c51140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.288+0000 7fdeeabe4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdec000fb00 con 0x56198980e4d0 2024-05-09T00:54:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.288+0000 7fdeeabe4740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fdec000f430 con 0x5619898062f0 2024-05-09T00:54:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.288+0000 7fdeeabe4740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdec000c670 con 0x56198982f1b0 2024-05-09T00:54:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.288+0000 7fdeeabe4740 10 client.0 _ll_register_callbacks cb 0x7fdedc068d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:54:04.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.290+0000 7fdee22d1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56198982f1b0 0x561989c51140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:04.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.290+0000 7fdee22d1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56198982f1b0 0x561989c51140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:45812/0 (socket says 172.21.15.5:45812) 2024-05-09T00:54:04.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.290+0000 7fdee22d1640 1 -- 172.21.15.5:0/967505403 learned_addr learned my addr 172.21.15.5:0/967505403 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:54:04.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.290+0000 7fdee12cf640 1 --2- 172.21.15.5:0/967505403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56198980e4d0 0x561989be10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:04.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.290+0000 7fdee1ad0640 1 --2- 172.21.15.5:0/967505403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619898062f0 0x561989be0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.291+0000 7fdee22d1640 1 -- 172.21.15.5:0/967505403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619898062f0 msgr2=0x561989be0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:54:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.291+0000 7fdee22d1640 1 --2- 172.21.15.5:0/967505403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619898062f0 0x561989be0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.291+0000 7fdee22d1640 1 -- 172.21.15.5:0/967505403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56198980e4d0 msgr2=0x561989be10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:54:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.291+0000 7fdee22d1640 1 --2- 172.21.15.5:0/967505403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56198980e4d0 0x561989be10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.291+0000 7fdee22d1640 1 -- 172.21.15.5:0/967505403 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561989afbed0 con 0x56198982f1b0 2024-05-09T00:54:04.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.292+0000 7fdee22d1640 1 --2- 172.21.15.5:0/967505403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56198982f1b0 0x561989c51140 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fdedc052220 tx=0x7fdedc050ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbcf70db30ba63e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:54:04.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.293+0000 7fdec57fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-09T00:54:04.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.293+0000 7fdec57fa640 1 -- 172.21.15.5:0/967505403 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdedc07b040 con 0x56198982f1b0 2024-05-09T00:54:04.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.293+0000 7fdeeabe4740 1 -- 172.21.15.5:0/967505403 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdedc07fd30 con 0x56198982f1b0 2024-05-09T00:54:04.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.293+0000 7fdec57fa640 1 -- 172.21.15.5:0/967505403 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fdedc077dd0 con 0x56198982f1b0 2024-05-09T00:54:04.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.294+0000 7fdec57fa640 1 -- 172.21.15.5:0/967505403 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fdedc09f030 con 0x56198982f1b0 2024-05-09T00:54:04.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.294+0000 7fdeeabe4740 10 client.25843 fetch_fsmap learned FSMap version 31 2024-05-09T00:54:04.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.294+0000 7fdeeabe4740 1 -- 172.21.15.5:0/967505403 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561989c523d0 con 0x56198982f1b0 2024-05-09T00:54:04.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.295+0000 7fdec57fa640 1 -- 172.21.15.5:0/967505403 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdedc07fd30 con 0x56198982f1b0 2024-05-09T00:54:04.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.296+0000 7fdec57fa640 1 -- 172.21.15.5:0/967505403 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fdedc08f070 con 0x56198982f1b0 2024-05-09T00:54:04.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.297+0000 7fdeeabe4740 10 client.25843 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:54:04.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.297+0000 7fdeeabe4740 10 client.25843 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:54:04.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.297+0000 7fdeeabe4740 20 client.25843 populate_metadata read hostname 'smithi005' 2024-05-09T00:54:04.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.297+0000 7fdeeabe4740 1 --2- 172.21.15.5:0/967505403 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5619897c31c0 0x561989c54d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:04.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.297+0000 7fdeeabe4740 4 client.25843 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:54:04.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.297+0000 7fdeeabe4740 1 -- 172.21.15.5:0/967505403 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561989b3f240 con 0x5619897c31c0 2024-05-09T00:54:04.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.298+0000 7fdee1ad0640 1 --2- 172.21.15.5:0/967505403 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5619897c31c0 0x561989c54d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.301+0000 7fdee1ad0640 1 --2- 172.21.15.5:0/967505403 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5619897c31c0 0x561989c54d70 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=806e5644160c0af0 server_cookie=3617591f155b5abe in_seq=0 out_seq=0 2024-05-09T00:54:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:04.301+0000 7fdec57fa640 10 client.25843 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:54:05.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:54:05 smithi157 ceph-mon[34294]: pgmap v603: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 894 MiB/s wr, 1.63k op/s 2024-05-09T00:54:05.790 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:54:05 smithi005 ceph-mon[30365]: pgmap v603: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 894 MiB/s wr, 1.63k op/s 2024-05-09T00:54:05.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:54:05 smithi131 ceph-mon[34547]: pgmap v603: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 894 MiB/s wr, 1.63k op/s 2024-05-09T00:54:06.795 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:54:06 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:54:06.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:54:06 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:54:06.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:54:06 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:54:07.790 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:54:07 smithi005 ceph-mon[30365]: pgmap v604: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 920 MiB/s wr, 1.73k op/s 2024-05-09T00:54:07.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:54:07 smithi131 ceph-mon[34547]: pgmap v604: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 920 MiB/s wr, 1.73k op/s 2024-05-09T00:54:07.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:54:07 smithi157 ceph-mon[34294]: pgmap v604: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 920 MiB/s wr, 1.73k op/s 2024-05-09T00:54:09.801 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:54:09 smithi005 ceph-mon[30365]: pgmap v605: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 866 MiB/s wr, 1.85k op/s 2024-05-09T00:54:09.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:54:09 smithi131 ceph-mon[34547]: pgmap v605: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 866 MiB/s wr, 1.85k op/s 2024-05-09T00:54:09.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:54:09 smithi157 ceph-mon[34294]: pgmap v605: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 866 MiB/s wr, 1.85k op/s 2024-05-09T00:54:11.790 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:54:11 smithi005 ceph-mon[30365]: pgmap v606: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 973 MiB/s wr, 2.44k op/s 2024-05-09T00:54:11.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:54:11 smithi131 ceph-mon[34547]: pgmap v606: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 973 MiB/s wr, 2.44k op/s 2024-05-09T00:54:11.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:54:11 smithi157 ceph-mon[34294]: pgmap v606: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 973 MiB/s wr, 2.44k op/s 2024-05-09T00:54:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:12.963+0000 7fdec57fa640 1 -- 172.21.15.5:0/967505403 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x561989b3f240 con 0x5619897c31c0 2024-05-09T00:54:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:12.963+0000 7fdec57fa640 10 client.25843 handle_command_reply: tid=0 2024-05-09T00:54:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:12.963+0000 7fdeeabe4740 1 client.25843 shutdown 2024-05-09T00:54:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:12.963+0000 7fdeeabe4740 1 -- 172.21.15.5:0/967505403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56198982f1b0 msgr2=0x561989c51140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:54:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:12.963+0000 7fdeeabe4740 1 --2- 172.21.15.5:0/967505403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56198982f1b0 0x561989c51140 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fdedc052220 tx=0x7fdedc050ea0 comp rx=0 tx=0).stop 2024-05-09T00:54:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:12.963+0000 7fdeeabe4740 1 -- 172.21.15.5:0/967505403 shutdown_connections 2024-05-09T00:54:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:12.963+0000 7fdeeabe4740 1 --2- 172.21.15.5:0/967505403 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5619897c31c0 0x561989c54d70 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:12.963+0000 7fdeeabe4740 1 --2- 172.21.15.5:0/967505403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56198982f1b0 0x561989c51140 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:12.963+0000 7fdeeabe4740 1 --2- 172.21.15.5:0/967505403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56198980e4d0 0x561989be10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:12.963+0000 7fdeeabe4740 1 --2- 172.21.15.5:0/967505403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619898062f0 0x561989be0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:12.963+0000 7fdec57fa640 0 client.25843 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:54:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:12.963+0000 7fdeeabe4740 1 -- 172.21.15.5:0/967505403 >> 172.21.15.5:0/967505403 conn(0x561989823300 msgr2=0x561989bd2880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:54:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:12.964+0000 7fdeeabe4740 1 -- 172.21.15.5:0/967505403 shutdown_connections 2024-05-09T00:54:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:12.964+0000 7fdeeabe4740 1 -- 172.21.15.5:0/967505403 wait complete. 2024-05-09T00:54:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:12.964+0000 7fdeeabe4740 20 client.25843 trim_cache size 0 max 16384 2024-05-09T00:54:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-09T00:54:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-09T00:54:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-09T00:54:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:12.965+0000 7fdee2ad2640 1 -- 172.21.15.5:0/1249479705 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fdec80b29a0 msgr2=0x7fdec80b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:54:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:12.965+0000 7fdee2ad2640 1 --2- 172.21.15.5:0/1249479705 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fdec80b29a0 0x7fdec80b4eb0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fded802b5a0 tx=0x7fded8000bc0 comp rx=0 tx=0).stop 2024-05-09T00:54:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:12.965+0000 7fdee2ad2640 1 -- 172.21.15.5:0/1249479705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fded40a6e70 msgr2=0x7fded413c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:54:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:12.965+0000 7fdee2ad2640 1 --2- 172.21.15.5:0/1249479705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fded40a6e70 0x7fded413c440 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fdecc00e9f0 tx=0x7fdecc00eeb0 comp rx=0 tx=0).stop 2024-05-09T00:54:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:12.966+0000 7fdee2ad2640 1 -- 172.21.15.5:0/1249479705 shutdown_connections 2024-05-09T00:54:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:12.966+0000 7fdee2ad2640 1 --2- 172.21.15.5:0/1249479705 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fdec80b29a0 0x7fdec80b4eb0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:12.966+0000 7fdee2ad2640 1 --2- 172.21.15.5:0/1249479705 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fded40abb70 0x7fded41545e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:12.966+0000 7fdee2ad2640 1 --2- 172.21.15.5:0/1249479705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fded40a6e70 0x7fded413c440 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:12.966+0000 7fdee2ad2640 1 --2- 172.21.15.5:0/1249479705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fded40a5c30 0x7fded413bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:12.966+0000 7fdee2ad2640 1 -- 172.21.15.5:0/1249479705 >> 172.21.15.5:0/1249479705 conn(0x7fded40a0e00 msgr2=0x7fded40a7b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:54:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:12.966+0000 7fdee2ad2640 1 -- 172.21.15.5:0/1249479705 shutdown_connections 2024-05-09T00:54:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:12.966+0000 7fdee2ad2640 1 -- 172.21.15.5:0/1249479705 wait complete. 2024-05-09T00:54:13.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:54:13 smithi005 ceph-mon[30365]: pgmap v607: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 838 MiB/s wr, 2.26k op/s 2024-05-09T00:54:13.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:54:13 smithi131 ceph-mon[34547]: pgmap v607: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 838 MiB/s wr, 2.26k op/s 2024-05-09T00:54:13.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:54:13 smithi157 ceph-mon[34294]: pgmap v607: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 838 MiB/s wr, 2.26k op/s 2024-05-09T00:54:14.797 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:54:14 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:54:14.798 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:54:14 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:54:14.798 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:54:14 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:54:14.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:54:14 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:54:14.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:54:14 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:54:14.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:54:14 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:54:14.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:54:14 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:54:14.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:54:14 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:54:14.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:54:14 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:54:15.482 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T00:54:15.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:54:15 smithi005 ceph-mon[30365]: pgmap v608: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 879 MiB/s wr, 2.46k op/s 2024-05-09T00:54:15.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:54:15 smithi131 ceph-mon[34547]: pgmap v608: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 879 MiB/s wr, 2.46k op/s 2024-05-09T00:54:15.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:54:15 smithi157 ceph-mon[34294]: pgmap v608: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 879 MiB/s wr, 2.46k op/s 2024-05-09T00:54:16.247 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:54:17.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:54:17 smithi005 ceph-mon[30365]: pgmap v609: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 863 MiB/s wr, 2.52k op/s 2024-05-09T00:54:17.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:54:17 smithi131 ceph-mon[34547]: pgmap v609: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 863 MiB/s wr, 2.52k op/s 2024-05-09T00:54:17.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:54:17 smithi157 ceph-mon[34294]: pgmap v609: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 863 MiB/s wr, 2.52k op/s 2024-05-09T00:54:19.790 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:54:19 smithi005 ceph-mon[30365]: pgmap v610: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 813 MiB/s wr, 2.37k op/s 2024-05-09T00:54:19.857 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:54:19 smithi131 ceph-mon[34547]: pgmap v610: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 813 MiB/s wr, 2.37k op/s 2024-05-09T00:54:19.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:54:19 smithi157 ceph-mon[34294]: pgmap v610: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 813 MiB/s wr, 2.37k op/s 2024-05-09T00:54:21.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.506+0000 7f436977f700 1 -- 172.21.15.5:0/2710121927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f436410a930 msgr2=0x7f436410ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:54:21.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.506+0000 7f436977f700 1 --2- 172.21.15.5:0/2710121927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f436410a930 0x7f436410ad40 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f4358009990 tx=0x7f4358009ca0 comp rx=0 tx=0).stop 2024-05-09T00:54:21.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.507+0000 7f436977f700 1 -- 172.21.15.5:0/2710121927 shutdown_connections 2024-05-09T00:54:21.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.507+0000 7f436977f700 1 --2- 172.21.15.5:0/2710121927 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f436410b280 0x7f436407ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:21.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.507+0000 7f436977f700 1 --2- 172.21.15.5:0/2710121927 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4364110960 0x7f4364110d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:21.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.507+0000 7f436977f700 1 --2- 172.21.15.5:0/2710121927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f436410a930 0x7f436410ad40 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:21.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.507+0000 7f436977f700 1 -- 172.21.15.5:0/2710121927 >> 172.21.15.5:0/2710121927 conn(0x7f436406f4d0 msgr2=0x7f436406f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:54:21.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.515+0000 7f436977f700 1 -- 172.21.15.5:0/2710121927 shutdown_connections 2024-05-09T00:54:21.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.516+0000 7f436977f700 1 -- 172.21.15.5:0/2710121927 wait complete. 2024-05-09T00:54:21.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.517+0000 7f436977f700 1 Processor -- start 2024-05-09T00:54:21.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.518+0000 7f436977f700 1 -- start start 2024-05-09T00:54:21.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.518+0000 7f436977f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f436410b280 0x7f436407b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:21.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.518+0000 7f436977f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4364110960 0x7f4364075940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:21.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.518+0000 7f436977f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4364075ed0 0x7f43640763b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:21.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.518+0000 7f436977f700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f436407ed70 con 0x7f436410b280 2024-05-09T00:54:21.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.518+0000 7f436977f700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f436407ebf0 con 0x7f4364110960 2024-05-09T00:54:21.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.518+0000 7f436977f700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f436407eef0 con 0x7f4364075ed0 2024-05-09T00:54:21.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.518+0000 7f4362ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f436410b280 0x7f436407b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:21.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.518+0000 7f4362ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f436410b280 0x7f436407b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60330/0 (socket says 172.21.15.5:60330) 2024-05-09T00:54:21.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.518+0000 7f4362ffd700 1 -- 172.21.15.5:0/2850674326 learned_addr learned my addr 172.21.15.5:0/2850674326 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:54:21.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.521+0000 7f43637fe700 1 --2- 172.21.15.5:0/2850674326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4364075ed0 0x7f43640763b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:21.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.522+0000 7f43627fc700 1 --2- 172.21.15.5:0/2850674326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4364110960 0x7f4364075940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:21.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.525+0000 7f4362ffd700 1 -- 172.21.15.5:0/2850674326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4364075ed0 msgr2=0x7f43640763b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:54:21.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.525+0000 7f4362ffd700 1 --2- 172.21.15.5:0/2850674326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4364075ed0 0x7f43640763b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:21.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.525+0000 7f4362ffd700 1 -- 172.21.15.5:0/2850674326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4364110960 msgr2=0x7f4364075940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:54:21.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.525+0000 7f4362ffd700 1 --2- 172.21.15.5:0/2850674326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4364110960 0x7f4364075940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:21.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.525+0000 7f4362ffd700 1 -- 172.21.15.5:0/2850674326 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4364139ad0 con 0x7f436410b280 2024-05-09T00:54:21.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.534+0000 7f4362ffd700 1 --2- 172.21.15.5:0/2850674326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f436410b280 0x7f436407b8e0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f4354007950 tx=0x7f4354007c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:54:21.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.538+0000 7f434bfff700 1 -- 172.21.15.5:0/2850674326 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4354019070 con 0x7f436410b280 2024-05-09T00:54:21.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.539+0000 7f436977f700 1 -- 172.21.15.5:0/2850674326 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4364139dc0 con 0x7f436410b280 2024-05-09T00:54:21.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.539+0000 7f436977f700 1 -- 172.21.15.5:0/2850674326 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f436413a300 con 0x7f436410b280 2024-05-09T00:54:21.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.545+0000 7f434bfff700 1 -- 172.21.15.5:0/2850674326 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f435400ae00 con 0x7f436410b280 2024-05-09T00:54:21.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.545+0000 7f434bfff700 1 -- 172.21.15.5:0/2850674326 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4354014e60 con 0x7f436410b280 2024-05-09T00:54:21.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.545+0000 7f434bfff700 1 -- 172.21.15.5:0/2850674326 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f435401c070 con 0x7f436410b280 2024-05-09T00:54:21.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.546+0000 7f434bfff700 1 --2- 172.21.15.5:0/2850674326 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f434c0b2b10 0x7f434c0b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:21.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.552+0000 7f434bfff700 1 -- 172.21.15.5:0/2850674326 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f43540e21e0 con 0x7f436410b280 2024-05-09T00:54:21.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.552+0000 7f4378195b80 1 Processor -- start 2024-05-09T00:54:21.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.552+0000 7f43627fc700 1 --2- 172.21.15.5:0/2850674326 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f434c0b2b10 0x7f434c0b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:21.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.553+0000 7f4378195b80 1 -- start start 2024-05-09T00:54:21.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.553+0000 7f4378195b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56355e6b0d00 0x56355e7b9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:21.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.553+0000 7f4378195b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56355e6364a0 0x56355e7b9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:21.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.553+0000 7f4378195b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56355e7b3400 0x56355e7b3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:21.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.553+0000 7f4378195b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56355e6d60a0 con 0x56355e6364a0 2024-05-09T00:54:21.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.553+0000 7f4378195b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x56355e67b9a0 con 0x56355e6b0d00 2024-05-09T00:54:21.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.553+0000 7f4378195b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56355e6d2060 con 0x56355e7b3400 2024-05-09T00:54:21.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.554+0000 7f43627fc700 1 --2- 172.21.15.5:0/2850674326 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f434c0b2b10 0x7f434c0b5020 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4358009480 tx=0x7f4358005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:54:21.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.555+0000 7f43637fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56355e7b3400 0x56355e7b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:21.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.555+0000 7f4362ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56355e6b0d00 0x56355e7b9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:21.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.556+0000 7f43637fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56355e7b3400 0x56355e7b3810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:47908/0 (socket says 172.21.15.5:47908) 2024-05-09T00:54:21.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.556+0000 7f43637fe700 1 -- 172.21.15.5:0/3456727625 learned_addr learned my addr 172.21.15.5:0/3456727625 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:54:21.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.556+0000 7f43627fc700 1 --2- 172.21.15.5:0/3456727625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56355e6364a0 0x56355e7b9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:21.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.557+0000 7f43637fe700 1 -- 172.21.15.5:0/3456727625 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56355e6b0d00 msgr2=0x56355e7b9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:54:21.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.557+0000 7f43637fe700 1 --2- 172.21.15.5:0/3456727625 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56355e6b0d00 0x56355e7b9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:21.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.557+0000 7f43637fe700 1 -- 172.21.15.5:0/3456727625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56355e6364a0 msgr2=0x56355e7b9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:54:21.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.557+0000 7f43637fe700 1 --2- 172.21.15.5:0/3456727625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56355e6364a0 0x56355e7b9780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:21.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.557+0000 7f43637fe700 1 -- 172.21.15.5:0/3456727625 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56355e66c500 con 0x56355e7b3400 2024-05-09T00:54:21.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.564+0000 7f43637fe700 1 --2- 172.21.15.5:0/3456727625 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56355e7b3400 0x56355e7b3810 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f435c004360 tx=0x7f435c008930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=909e0a5e2cff790e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:54:21.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.564+0000 7f43497fa700 1 -- 172.21.15.5:0/3456727625 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f435c01d070 con 0x56355e7b3400 2024-05-09T00:54:21.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.564+0000 7f43497fa700 1 -- 172.21.15.5:0/3456727625 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f435c010ac0 con 0x56355e7b3400 2024-05-09T00:54:21.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.564+0000 7f43497fa700 1 -- 172.21.15.5:0/3456727625 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f435c018c50 con 0x56355e7b3400 2024-05-09T00:54:21.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.566+0000 7f4378195b80 1 -- 172.21.15.5:0/3456727625 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56355e7b3400 msgr2=0x56355e7b3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:54:21.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.566+0000 7f4378195b80 1 --2- 172.21.15.5:0/3456727625 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56355e7b3400 0x56355e7b3810 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f435c004360 tx=0x7f435c008930 comp rx=0 tx=0).stop 2024-05-09T00:54:21.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.566+0000 7f4378195b80 1 -- 172.21.15.5:0/3456727625 shutdown_connections 2024-05-09T00:54:21.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.566+0000 7f4378195b80 1 --2- 172.21.15.5:0/3456727625 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56355e7b3400 0x56355e7b3810 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:21.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.566+0000 7f4378195b80 1 --2- 172.21.15.5:0/3456727625 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56355e6b0d00 0x56355e7b9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:21.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.566+0000 7f4378195b80 1 --2- 172.21.15.5:0/3456727625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56355e6364a0 0x56355e7b9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:21.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.566+0000 7f4378195b80 1 -- 172.21.15.5:0/3456727625 >> 172.21.15.5:0/3456727625 conn(0x56355e639f70 msgr2=0x56355e7ae220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:54:21.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.571+0000 7f4378195b80 1 -- 172.21.15.5:0/3456727625 shutdown_connections 2024-05-09T00:54:21.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.571+0000 7f4378195b80 1 -- 172.21.15.5:0/3456727625 wait complete. 2024-05-09T00:54:21.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.572+0000 7f4378195b80 1 -- start start 2024-05-09T00:54:21.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.572+0000 7f4378195b80 1 Processor -- start 2024-05-09T00:54:21.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.573+0000 7f4378195b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56355e6364a0 0x56355e7bd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:21.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.573+0000 7f4378195b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56355e6b0d00 0x56355e7bd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:21.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.573+0000 7f4378195b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56355e7b3400 0x56355e7bdb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:21.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.573+0000 7f4378195b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56355e7b4160 con 0x56355e6364a0 2024-05-09T00:54:21.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.573+0000 7f4378195b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x56355e670b20 con 0x56355e6b0d00 2024-05-09T00:54:21.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.573+0000 7f4378195b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56355e7c0020 con 0x56355e7b3400 2024-05-09T00:54:21.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.574+0000 7f4378195b80 10 client.0 _ll_register_callbacks cb 0x56355e5476c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:54:21.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.574+0000 7f4362ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56355e6364a0 0x56355e7bd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:21.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.574+0000 7f4362ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56355e6364a0 0x56355e7bd0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60362/0 (socket says 172.21.15.5:60362) 2024-05-09T00:54:21.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.574+0000 7f4362ffd700 1 -- 172.21.15.5:0/1683099795 learned_addr learned my addr 172.21.15.5:0/1683099795 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:54:21.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.575+0000 7f4362ffd700 1 -- 172.21.15.5:0/1683099795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56355e7b3400 msgr2=0x56355e7bdb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:54:21.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.575+0000 7f4362ffd700 1 --2- 172.21.15.5:0/1683099795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56355e7b3400 0x56355e7bdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:21.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.575+0000 7f4362ffd700 1 -- 172.21.15.5:0/1683099795 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56355e6b0d00 msgr2=0x56355e7bd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:54:21.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.575+0000 7f4362ffd700 1 --2- 172.21.15.5:0/1683099795 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56355e6b0d00 0x56355e7bd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:21.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.575+0000 7f4362ffd700 1 -- 172.21.15.5:0/1683099795 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56355e551490 con 0x56355e6364a0 2024-05-09T00:54:21.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.577+0000 7f4362ffd700 1 --2- 172.21.15.5:0/1683099795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56355e6364a0 0x56355e7bd0e0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f43540a5820 tx=0x7f43540cc280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73ec0498c8955e61 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:54:21.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.578+0000 7f4346ff5700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:54:21.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.581+0000 7f4346ff5700 1 -- 172.21.15.5:0/1683099795 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43540a97f0 con 0x56355e6364a0 2024-05-09T00:54:21.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.581+0000 7f4346ff5700 1 -- 172.21.15.5:0/1683099795 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f435400a950 con 0x56355e6364a0 2024-05-09T00:54:21.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.582+0000 7f4378195b80 1 -- 172.21.15.5:0/1683099795 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f435c010ac0 con 0x56355e6364a0 2024-05-09T00:54:21.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.584+0000 7f4346ff5700 1 -- 172.21.15.5:0/1683099795 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f43540adba0 con 0x56355e6364a0 2024-05-09T00:54:21.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.587+0000 7f4378195b80 10 client.16302 fetch_fsmap learned FSMap version 31 2024-05-09T00:54:21.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.587+0000 7f4378195b80 1 -- 172.21.15.5:0/1683099795 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56355e7ca940 con 0x56355e6364a0 2024-05-09T00:54:21.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.590+0000 7f4346ff5700 1 -- 172.21.15.5:0/1683099795 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43540c3070 con 0x56355e6364a0 2024-05-09T00:54:21.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.592+0000 7f4346ff5700 1 -- 172.21.15.5:0/1683099795 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f43540a5160 con 0x56355e6364a0 2024-05-09T00:54:21.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.592+0000 7f4378195b80 10 client.16302 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:54:21.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.592+0000 7f4378195b80 10 client.16302 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:54:21.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.592+0000 7f4378195b80 20 client.16302 populate_metadata read hostname 'smithi005' 2024-05-09T00:54:21.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.592+0000 7f4378195b80 1 --2- 172.21.15.5:0/1683099795 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56355e7ccf20 0x56355e7cf400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:21.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.593+0000 7f4378195b80 4 client.16302 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:54:21.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.593+0000 7f4378195b80 1 -- 172.21.15.5:0/1683099795 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f435c010ac0 con 0x56355e7ccf20 2024-05-09T00:54:21.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.597+0000 7f43627fc700 1 --2- 172.21.15.5:0/1683099795 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56355e7ccf20 0x56355e7cf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:21.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.613+0000 7f4346ff5700 10 client.16302 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:54:21.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.613+0000 7f43627fc700 1 --2- 172.21.15.5:0/1683099795 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56355e7ccf20 0x56355e7cf400 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2322ac68e66a471 server_cookie=54828c12bb6b93c7 in_seq=0 out_seq=0 2024-05-09T00:54:21.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.622+0000 7f4346ff5700 1 -- 172.21.15.5:0/1683099795 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f435c010ac0 con 0x56355e7ccf20 2024-05-09T00:54:21.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.622+0000 7f4346ff5700 10 client.16302 handle_command_reply: tid=0 2024-05-09T00:54:21.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.622+0000 7f4378195b80 1 client.16302 shutdown 2024-05-09T00:54:21.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.624+0000 7f4378195b80 1 -- 172.21.15.5:0/1683099795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56355e6364a0 msgr2=0x56355e7bd0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:54:21.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.624+0000 7f4378195b80 1 --2- 172.21.15.5:0/1683099795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56355e6364a0 0x56355e7bd0e0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f43540a5820 tx=0x7f43540cc280 comp rx=0 tx=0).stop 2024-05-09T00:54:21.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.625+0000 7f4378195b80 1 -- 172.21.15.5:0/1683099795 shutdown_connections 2024-05-09T00:54:21.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.625+0000 7f4378195b80 1 --2- 172.21.15.5:0/1683099795 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56355e7ccf20 0x56355e7cf400 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:21.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.625+0000 7f4378195b80 1 --2- 172.21.15.5:0/1683099795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56355e7b3400 0x56355e7bdb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:21.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.625+0000 7f4346ff5700 0 client.16302 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:54:21.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.625+0000 7f4378195b80 1 --2- 172.21.15.5:0/1683099795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56355e6364a0 0x56355e7bd0e0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:21.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.625+0000 7f4378195b80 1 --2- 172.21.15.5:0/1683099795 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56355e6b0d00 0x56355e7bd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:21.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.626+0000 7f4378195b80 1 -- 172.21.15.5:0/1683099795 >> 172.21.15.5:0/1683099795 conn(0x56355e639f70 msgr2=0x56355e7a7ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:54:21.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.630+0000 7f4378195b80 1 -- 172.21.15.5:0/1683099795 shutdown_connections 2024-05-09T00:54:21.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.630+0000 7f4378195b80 1 -- 172.21.15.5:0/1683099795 wait complete. 2024-05-09T00:54:21.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.630+0000 7f4378195b80 20 client.16302 trim_cache size 0 max 16384 2024-05-09T00:54:21.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.682+0000 7f4378195b80 1 Processor -- start 2024-05-09T00:54:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.683+0000 7f4378195b80 1 -- start start 2024-05-09T00:54:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.683+0000 7f4378195b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56355e6364a0 0x56355e7bc400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.684+0000 7f4378195b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56355e6b0d00 0x56355e7bf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.684+0000 7f4378195b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56355e7b3400 0x56355e7a8700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.684+0000 7f4378195b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f435000d240 con 0x56355e6b0d00 2024-05-09T00:54:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.684+0000 7f4378195b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f435000ed80 con 0x56355e6364a0 2024-05-09T00:54:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.684+0000 7f4378195b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f435000d910 con 0x56355e7b3400 2024-05-09T00:54:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.684+0000 7f4362ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56355e6364a0 0x56355e7bc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.685+0000 7f4362ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56355e6364a0 0x56355e7bc400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:56086/0 (socket says 172.21.15.5:56086) 2024-05-09T00:54:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.685+0000 7f4362ffd700 1 -- 172.21.15.5:0/2955278376 learned_addr learned my addr 172.21.15.5:0/2955278376 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:54:21.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.686+0000 7f43637fe700 1 --2- 172.21.15.5:0/2955278376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56355e7b3400 0x56355e7a8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:21.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.687+0000 7f43627fc700 1 --2- 172.21.15.5:0/2955278376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56355e6b0d00 0x56355e7bf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:21.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.687+0000 7f4362ffd700 1 -- 172.21.15.5:0/2955278376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56355e7b3400 msgr2=0x56355e7a8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:54:21.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.687+0000 7f4362ffd700 1 --2- 172.21.15.5:0/2955278376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56355e7b3400 0x56355e7a8700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:21.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.687+0000 7f4362ffd700 1 -- 172.21.15.5:0/2955278376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56355e6b0d00 msgr2=0x56355e7bf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:54:21.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.687+0000 7f4362ffd700 1 --2- 172.21.15.5:0/2955278376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56355e6b0d00 0x56355e7bf580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:21.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.687+0000 7f4362ffd700 1 -- 172.21.15.5:0/2955278376 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56355e7af190 con 0x56355e6364a0 2024-05-09T00:54:21.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.688+0000 7f4362ffd700 1 --2- 172.21.15.5:0/2955278376 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56355e6364a0 0x56355e7bc400 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f43540a5d90 tx=0x7f43540a9410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7687c7563e42a455 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:54:21.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.688+0000 7f4348ff9700 1 -- 172.21.15.5:0/2955278376 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43540a97f0 con 0x56355e6364a0 2024-05-09T00:54:21.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.688+0000 7f4348ff9700 1 -- 172.21.15.5:0/2955278376 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43540c3070 con 0x56355e6364a0 2024-05-09T00:54:21.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.688+0000 7f4348ff9700 1 -- 172.21.15.5:0/2955278376 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43540a8cb0 con 0x56355e6364a0 2024-05-09T00:54:21.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.690+0000 7f4378195b80 1 -- 172.21.15.5:0/2955278376 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56355e6364a0 msgr2=0x56355e7bc400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:54:21.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.690+0000 7f4378195b80 1 --2- 172.21.15.5:0/2955278376 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56355e6364a0 0x56355e7bc400 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f43540a5d90 tx=0x7f43540a9410 comp rx=0 tx=0).stop 2024-05-09T00:54:21.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.690+0000 7f4378195b80 1 -- 172.21.15.5:0/2955278376 shutdown_connections 2024-05-09T00:54:21.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.690+0000 7f4378195b80 1 --2- 172.21.15.5:0/2955278376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56355e7b3400 0x56355e7a8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:21.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.690+0000 7f4378195b80 1 --2- 172.21.15.5:0/2955278376 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56355e6364a0 0x56355e7bc400 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:21.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.690+0000 7f4378195b80 1 --2- 172.21.15.5:0/2955278376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56355e6b0d00 0x56355e7bf580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:21.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.690+0000 7f4378195b80 1 -- 172.21.15.5:0/2955278376 >> 172.21.15.5:0/2955278376 conn(0x56355e639f70 msgr2=0x56355e7a7ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:54:21.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.690+0000 7f4378195b80 1 -- 172.21.15.5:0/2955278376 shutdown_connections 2024-05-09T00:54:21.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.691+0000 7f4378195b80 1 -- 172.21.15.5:0/2955278376 wait complete. 2024-05-09T00:54:21.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.691+0000 7f4378195b80 1 -- start start 2024-05-09T00:54:21.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.691+0000 7f4378195b80 1 Processor -- start 2024-05-09T00:54:21.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.692+0000 7f4378195b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56355e6364a0 0x56355e7d4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:21.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.692+0000 7f4378195b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56355e6b0d00 0x56355e7d4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:21.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.692+0000 7f4378195b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56355e7b3400 0x56355e7d4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:21.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.692+0000 7f4378195b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f435000e6b0 con 0x56355e6b0d00 2024-05-09T00:54:21.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.692+0000 7f4378195b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f435000dfe0 con 0x56355e7b3400 2024-05-09T00:54:21.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.692+0000 7f4378195b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f435000b290 con 0x56355e6364a0 2024-05-09T00:54:21.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.692+0000 7f4378195b80 10 client.0 _ll_register_callbacks cb 0x56355e5476c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:54:21.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.693+0000 7f43637fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56355e7b3400 0x56355e7d4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:21.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.693+0000 7f43637fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56355e7b3400 0x56355e7d4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:56088/0 (socket says 172.21.15.5:56088) 2024-05-09T00:54:21.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.693+0000 7f43637fe700 1 -- 172.21.15.5:0/395689543 learned_addr learned my addr 172.21.15.5:0/395689543 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:54:21.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.694+0000 7f43627fc700 1 --2- 172.21.15.5:0/395689543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56355e6b0d00 0x56355e7d4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:21.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.695+0000 7f43637fe700 1 -- 172.21.15.5:0/395689543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56355e6364a0 msgr2=0x56355e7d4520 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:54:21.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.695+0000 7f43637fe700 1 --2- 172.21.15.5:0/395689543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56355e6364a0 0x56355e7d4520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:21.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.695+0000 7f43637fe700 1 -- 172.21.15.5:0/395689543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56355e6b0d00 msgr2=0x56355e7d4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:54:21.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.695+0000 7f43637fe700 1 --2- 172.21.15.5:0/395689543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56355e6b0d00 0x56355e7d4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:21.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.695+0000 7f43637fe700 1 -- 172.21.15.5:0/395689543 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56355e8367a0 con 0x56355e7b3400 2024-05-09T00:54:21.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.698+0000 7f43637fe700 1 --2- 172.21.15.5:0/395689543 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56355e7b3400 0x56355e7d4fa0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f435c004360 tx=0x7f435c004f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5842740684963e82 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:54:21.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.698+0000 7f4346ff5700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-09T00:54:21.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.700+0000 7f4346ff5700 1 -- 172.21.15.5:0/395689543 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f435c025020 con 0x56355e7b3400 2024-05-09T00:54:21.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.700+0000 7f4378195b80 1 -- 172.21.15.5:0/395689543 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43540c3070 con 0x56355e7b3400 2024-05-09T00:54:21.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.700+0000 7f4346ff5700 1 -- 172.21.15.5:0/395689543 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f435c005370 con 0x56355e7b3400 2024-05-09T00:54:21.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.702+0000 7f4346ff5700 1 -- 172.21.15.5:0/395689543 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f435c030050 con 0x56355e7b3400 2024-05-09T00:54:21.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.703+0000 7f4378195b80 10 client.25823 fetch_fsmap learned FSMap version 31 2024-05-09T00:54:21.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.703+0000 7f4378195b80 1 -- 172.21.15.5:0/395689543 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56355e837a10 con 0x56355e7b3400 2024-05-09T00:54:21.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.705+0000 7f4346ff5700 1 -- 172.21.15.5:0/395689543 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43540c3070 con 0x56355e7b3400 2024-05-09T00:54:21.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.705+0000 7f4346ff5700 1 -- 172.21.15.5:0/395689543 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f435c020050 con 0x56355e7b3400 2024-05-09T00:54:21.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.708+0000 7f4378195b80 10 client.25823 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:54:21.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.708+0000 7f4378195b80 10 client.25823 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:54:21.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.708+0000 7f4378195b80 20 client.25823 populate_metadata read hostname 'smithi005' 2024-05-09T00:54:21.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.709+0000 7f4378195b80 1 --2- 172.21.15.5:0/395689543 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56355e7ccf20 0x56355e839cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:21.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.709+0000 7f4378195b80 4 client.25823 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:54:21.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.709+0000 7f4378195b80 1 -- 172.21.15.5:0/395689543 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56355e6e3a80 con 0x56355e7ccf20 2024-05-09T00:54:21.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.710+0000 7f4362ffd700 1 --2- 172.21.15.5:0/395689543 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56355e7ccf20 0x56355e839cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:21.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.720+0000 7f4362ffd700 1 --2- 172.21.15.5:0/395689543 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56355e7ccf20 0x56355e839cf0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad36b21d8ef753e4 server_cookie=5ba5079d709b30a1 in_seq=0 out_seq=0 2024-05-09T00:54:21.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.721+0000 7f4346ff5700 10 client.25823 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:54:21.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.725+0000 7f4346ff5700 1 -- 172.21.15.5:0/395689543 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56355e6e3a80 con 0x56355e7ccf20 2024-05-09T00:54:21.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.725+0000 7f4346ff5700 10 client.25823 handle_command_reply: tid=0 2024-05-09T00:54:21.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.728+0000 7f4378195b80 1 client.25823 shutdown 2024-05-09T00:54:21.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.731+0000 7f4378195b80 1 -- 172.21.15.5:0/395689543 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56355e7b3400 msgr2=0x56355e7d4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:54:21.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.731+0000 7f4378195b80 1 --2- 172.21.15.5:0/395689543 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56355e7b3400 0x56355e7d4fa0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f435c004360 tx=0x7f435c004f80 comp rx=0 tx=0).stop 2024-05-09T00:54:21.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.732+0000 7f4378195b80 1 -- 172.21.15.5:0/395689543 shutdown_connections 2024-05-09T00:54:21.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.732+0000 7f4378195b80 1 --2- 172.21.15.5:0/395689543 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56355e7ccf20 0x56355e839cf0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:21.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.732+0000 7f4378195b80 1 --2- 172.21.15.5:0/395689543 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56355e7b3400 0x56355e7d4fa0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:21.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.732+0000 7f4378195b80 1 --2- 172.21.15.5:0/395689543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56355e6364a0 0x56355e7d4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:21.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.732+0000 7f4378195b80 1 --2- 172.21.15.5:0/395689543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56355e6b0d00 0x56355e7d4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:21.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.733+0000 7f4378195b80 1 -- 172.21.15.5:0/395689543 >> 172.21.15.5:0/395689543 conn(0x56355e639f70 msgr2=0x56355e7bc160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:54:21.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.733+0000 7f4346ff5700 0 client.25823 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:54:21.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.737+0000 7f4378195b80 1 -- 172.21.15.5:0/395689543 shutdown_connections 2024-05-09T00:54:21.742 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-09T00:54:21.742 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-05-09T00:54:21.743 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-05-09T00:54:21.743 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-09T00:54:21.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.737+0000 7f4378195b80 1 -- 172.21.15.5:0/395689543 wait complete. 2024-05-09T00:54:21.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.737+0000 7f4378195b80 20 client.25823 trim_cache size 0 max 16384 2024-05-09T00:54:21.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.743+0000 7f436977f700 1 -- 172.21.15.5:0/2850674326 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f434c0b2b10 msgr2=0x7f434c0b5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:54:21.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.743+0000 7f436977f700 1 --2- 172.21.15.5:0/2850674326 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f434c0b2b10 0x7f434c0b5020 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4358009480 tx=0x7f4358005fb0 comp rx=0 tx=0).stop 2024-05-09T00:54:21.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.743+0000 7f436977f700 1 -- 172.21.15.5:0/2850674326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f436410b280 msgr2=0x7f436407b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:54:21.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.743+0000 7f436977f700 1 --2- 172.21.15.5:0/2850674326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f436410b280 0x7f436407b8e0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f4354007950 tx=0x7f4354007c60 comp rx=0 tx=0).stop 2024-05-09T00:54:21.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.746+0000 7f436977f700 1 -- 172.21.15.5:0/2850674326 shutdown_connections 2024-05-09T00:54:21.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.746+0000 7f436977f700 1 --2- 172.21.15.5:0/2850674326 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f434c0b2b10 0x7f434c0b5020 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:21.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.746+0000 7f436977f700 1 --2- 172.21.15.5:0/2850674326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4364075ed0 0x7f43640763b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:21.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.746+0000 7f436977f700 1 --2- 172.21.15.5:0/2850674326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f436410b280 0x7f436407b8e0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:21.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.746+0000 7f436977f700 1 --2- 172.21.15.5:0/2850674326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4364110960 0x7f4364075940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:21.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.746+0000 7f436977f700 1 -- 172.21.15.5:0/2850674326 >> 172.21.15.5:0/2850674326 conn(0x7f436406f4d0 msgr2=0x7f436410caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:54:21.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.749+0000 7f436977f700 1 -- 172.21.15.5:0/2850674326 shutdown_connections 2024-05-09T00:54:21.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:21.751+0000 7f436977f700 1 -- 172.21.15.5:0/2850674326 wait complete. 2024-05-09T00:54:21.794 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:54:21 smithi005 ceph-mon[30365]: pgmap v611: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 966 MiB/s wr, 2.59k op/s 2024-05-09T00:54:21.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:54:21 smithi131 ceph-mon[34547]: pgmap v611: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 966 MiB/s wr, 2.59k op/s 2024-05-09T00:54:21.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:54:21 smithi157 ceph-mon[34294]: pgmap v611: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 966 MiB/s wr, 2.59k op/s 2024-05-09T00:54:23.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:54:23 smithi005 ceph-mon[30365]: pgmap v612: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 823 MiB/s wr, 1.93k op/s 2024-05-09T00:54:23.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:54:23 smithi131 ceph-mon[34547]: pgmap v612: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 823 MiB/s wr, 1.93k op/s 2024-05-09T00:54:23.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:54:23 smithi157 ceph-mon[34294]: pgmap v612: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 823 MiB/s wr, 1.93k op/s 2024-05-09T00:54:23.975 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T00:54:23.975 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T00:54:24.694 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:54:25.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:54:25 smithi131 ceph-mon[34547]: pgmap v613: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 899 MiB/s wr, 1.86k op/s 2024-05-09T00:54:25.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:54:25 smithi005 ceph-mon[30365]: pgmap v613: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 899 MiB/s wr, 1.86k op/s 2024-05-09T00:54:25.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:54:25 smithi157 ceph-mon[34294]: pgmap v613: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 899 MiB/s wr, 1.86k op/s 2024-05-09T00:54:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.286+0000 7f47537ce640 1 -- 172.21.15.5:0/873083370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f474c101990 msgr2=0x7f474c101da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:54:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.286+0000 7f47537ce640 1 --2- 172.21.15.5:0/873083370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f474c101990 0x7f474c101da0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f4744009900 tx=0x7f474402a410 comp rx=0 tx=0).stop 2024-05-09T00:54:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.286+0000 7f47537ce640 1 -- 172.21.15.5:0/873083370 shutdown_connections 2024-05-09T00:54:26.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.286+0000 7f47537ce640 1 --2- 172.21.15.5:0/873083370 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f474c109900 0x7f474c10bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:26.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.286+0000 7f47537ce640 1 --2- 172.21.15.5:0/873083370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f474c106f50 0x7f474c109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:26.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.286+0000 7f47537ce640 1 --2- 172.21.15.5:0/873083370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f474c101990 0x7f474c101da0 secure :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f4744009900 tx=0x7f474402a410 comp rx=0 tx=0).stop 2024-05-09T00:54:26.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.286+0000 7f47537ce640 1 -- 172.21.15.5:0/873083370 >> 172.21.15.5:0/873083370 conn(0x7f474c0fd080 msgr2=0x7f474c0ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:54:26.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.291+0000 7f47537ce640 1 -- 172.21.15.5:0/873083370 shutdown_connections 2024-05-09T00:54:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.292+0000 7f47537ce640 1 -- 172.21.15.5:0/873083370 wait complete. 2024-05-09T00:54:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.294+0000 7f47537ce640 1 Processor -- start 2024-05-09T00:54:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.295+0000 7f47537ce640 1 -- start start 2024-05-09T00:54:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.295+0000 7f47537ce640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f474c106f50 0x7f474c1047b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.295+0000 7f47537ce640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f474c109900 0x7f474c102e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.295+0000 7f47537ce640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f474c103390 0x7f474c1037a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.295+0000 7f47537ce640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f474c10f820 con 0x7f474c103390 2024-05-09T00:54:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.295+0000 7f47537ce640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f474c10f6a0 con 0x7f474c109900 2024-05-09T00:54:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.296+0000 7f47537ce640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f474c10f9a0 con 0x7f474c106f50 2024-05-09T00:54:26.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.297+0000 7f4751d6b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f474c103390 0x7f474c1037a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:26.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.297+0000 7f4751d6b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f474c103390 0x7f474c1037a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59334/0 (socket says 172.21.15.5:59334) 2024-05-09T00:54:26.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.297+0000 7f4751d6b640 1 -- 172.21.15.5:0/973497681 learned_addr learned my addr 172.21.15.5:0/973497681 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:54:26.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.297+0000 7f475156a640 1 --2- 172.21.15.5:0/973497681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f474c106f50 0x7f474c1047b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:26.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.298+0000 7f475156a640 1 -- 172.21.15.5:0/973497681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f474c109900 msgr2=0x7f474c102e50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T00:54:26.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.298+0000 7f475156a640 1 --2- 172.21.15.5:0/973497681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f474c109900 0x7f474c102e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:26.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.298+0000 7f475156a640 1 -- 172.21.15.5:0/973497681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f474c103390 msgr2=0x7f474c1037a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:54:26.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.298+0000 7f475156a640 1 --2- 172.21.15.5:0/973497681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f474c103390 0x7f474c1037a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:26.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.298+0000 7f475156a640 1 -- 172.21.15.5:0/973497681 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f474c1aa090 con 0x7f474c106f50 2024-05-09T00:54:26.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.299+0000 7f475156a640 1 --2- 172.21.15.5:0/973497681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f474c106f50 0x7f474c1047b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f474c069b10 tx=0x7f4744004b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:54:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.299+0000 7f474a7fc640 1 -- 172.21.15.5:0/973497681 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4744040070 con 0x7f474c106f50 2024-05-09T00:54:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.300+0000 7f474a7fc640 1 -- 172.21.15.5:0/973497681 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4744031b80 con 0x7f474c106f50 2024-05-09T00:54:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.300+0000 7f474a7fc640 1 -- 172.21.15.5:0/973497681 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4744035020 con 0x7f474c106f50 2024-05-09T00:54:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.300+0000 7f47537ce640 1 -- 172.21.15.5:0/973497681 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f474c1aa320 con 0x7f474c106f50 2024-05-09T00:54:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.300+0000 7f47537ce640 1 -- 172.21.15.5:0/973497681 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f474c1aa810 con 0x7f474c106f50 2024-05-09T00:54:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.308+0000 7f475a67e740 1 Processor -- start 2024-05-09T00:54:26.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.309+0000 7f475a67e740 1 -- start start 2024-05-09T00:54:26.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.312+0000 7f475a67e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ece3611b0 0x557ece705500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:26.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.312+0000 7f475a67e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ece3382f0 0x557ece705a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:26.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.312+0000 7f475a67e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ece3404d0 0x557ece70c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:26.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.312+0000 7f475a67e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557ece6697c0 con 0x557ece3382f0 2024-05-09T00:54:26.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.312+0000 7f475a67e740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x557ece6050e0 con 0x557ece3611b0 2024-05-09T00:54:26.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.312+0000 7f475a67e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x557ece665780 con 0x557ece3404d0 2024-05-09T00:54:26.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.313+0000 7f4750d69640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ece3404d0 0x557ece70c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:26.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.314+0000 7f4751d6b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ece3611b0 0x557ece705500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:26.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.314+0000 7f4750d69640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ece3404d0 0x557ece70c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:57722/0 (socket says 172.21.15.5:57722) 2024-05-09T00:54:26.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.314+0000 7f4750d69640 1 -- 172.21.15.5:0/603834369 learned_addr learned my addr 172.21.15.5:0/603834369 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:54:26.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.314+0000 7f474a7fc640 1 -- 172.21.15.5:0/973497681 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f4744003b30 con 0x7f474c106f50 2024-05-09T00:54:26.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.317+0000 7f474a7fc640 1 --2- 172.21.15.5:0/973497681 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f47340b2930 0x7f47340b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:26.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.317+0000 7f4750d69640 1 -- 172.21.15.5:0/603834369 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ece3611b0 msgr2=0x557ece705500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:54:26.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.317+0000 7f4750d69640 1 --2- 172.21.15.5:0/603834369 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ece3611b0 0x557ece705500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:26.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.317+0000 7f4750d69640 1 -- 172.21.15.5:0/603834369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ece3382f0 msgr2=0x557ece705a80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:54:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.317+0000 7f4750d69640 1 --2- 172.21.15.5:0/603834369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ece3382f0 0x557ece705a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.317+0000 7f4750d69640 1 -- 172.21.15.5:0/603834369 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ece62e7c0 con 0x557ece3404d0 2024-05-09T00:54:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.318+0000 7f4751d6b640 1 --2- 172.21.15.5:0/973497681 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f47340b2930 0x7f47340b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.319+0000 7f474a7fc640 1 -- 172.21.15.5:0/973497681 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f47440c7620 con 0x7f474c106f50 2024-05-09T00:54:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.320+0000 7f4751d6b640 1 --2- 172.21.15.5:0/973497681 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f47340b2930 0x7f47340b4e40 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f473c009fd0 tx=0x7f473c004b50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:54:26.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.320+0000 7f4750d69640 1 --2- 172.21.15.5:0/603834369 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ece3404d0 0x557ece70c980 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4740009780 tx=0x7f4740009c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1af791aa7b55b34 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:54:26.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.321+0000 7f47337fe640 1 -- 172.21.15.5:0/603834369 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4740015020 con 0x557ece3404d0 2024-05-09T00:54:26.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.321+0000 7f47337fe640 1 -- 172.21.15.5:0/603834369 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4740011420 con 0x557ece3404d0 2024-05-09T00:54:26.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.323+0000 7f475a67e740 1 -- 172.21.15.5:0/603834369 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ece3404d0 msgr2=0x557ece70c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:54:26.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.323+0000 7f475a67e740 1 --2- 172.21.15.5:0/603834369 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ece3404d0 0x557ece70c980 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4740009780 tx=0x7f4740009c40 comp rx=0 tx=0).stop 2024-05-09T00:54:26.330 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.330+0000 7f475a67e740 1 -- 172.21.15.5:0/603834369 shutdown_connections 2024-05-09T00:54:26.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.330+0000 7f475a67e740 1 --2- 172.21.15.5:0/603834369 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ece3404d0 0x557ece70c980 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:26.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.330+0000 7f475a67e740 1 --2- 172.21.15.5:0/603834369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ece3382f0 0x557ece705a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:26.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.330+0000 7f475a67e740 1 --2- 172.21.15.5:0/603834369 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ece3611b0 0x557ece705500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:26.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.330+0000 7f475a67e740 1 -- 172.21.15.5:0/603834369 >> 172.21.15.5:0/603834369 conn(0x557ece355300 msgr2=0x557ece700e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:54:26.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.331+0000 7f475a67e740 1 -- 172.21.15.5:0/603834369 shutdown_connections 2024-05-09T00:54:26.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.331+0000 7f475a67e740 1 -- 172.21.15.5:0/603834369 wait complete. 2024-05-09T00:54:26.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.331+0000 7f475a67e740 1 -- start start 2024-05-09T00:54:26.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.331+0000 7f475a67e740 1 Processor -- start 2024-05-09T00:54:26.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.332+0000 7f475a67e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ece3382f0 0x557ece70f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:26.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.332+0000 7f475a67e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ece3404d0 0x557ece70f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:26.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.333+0000 7f475a67e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ece3611b0 0x557ece7166c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:26.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.333+0000 7f475a67e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557ece70d420 con 0x557ece3611b0 2024-05-09T00:54:26.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.333+0000 7f475a67e740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x557ece5f09c0 con 0x557ece3404d0 2024-05-09T00:54:26.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.333+0000 7f475a67e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x557ece70d5a0 con 0x557ece3382f0 2024-05-09T00:54:26.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.333+0000 7f475a67e740 10 client.0 _ll_register_callbacks cb 0x557ece61d020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:54:26.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.333+0000 7f4751d6b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ece3611b0 0x557ece7166c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:26.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.333+0000 7f4751d6b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ece3611b0 0x557ece7166c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59338/0 (socket says 172.21.15.5:59338) 2024-05-09T00:54:26.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.333+0000 7f4751d6b640 1 -- 172.21.15.5:0/2789534064 learned_addr learned my addr 172.21.15.5:0/2789534064 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:54:26.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.334+0000 7f475156a640 1 --2- 172.21.15.5:0/2789534064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ece3382f0 0x557ece70f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:26.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.334+0000 7f4751d6b640 1 -- 172.21.15.5:0/2789534064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ece3382f0 msgr2=0x557ece70f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:54:26.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.334+0000 7f4751d6b640 1 --2- 172.21.15.5:0/2789534064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ece3382f0 0x557ece70f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:26.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.334+0000 7f4751d6b640 1 -- 172.21.15.5:0/2789534064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ece3404d0 msgr2=0x557ece70f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:54:26.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.334+0000 7f4751d6b640 1 --2- 172.21.15.5:0/2789534064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ece3404d0 0x557ece70f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:26.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.334+0000 7f4751d6b640 1 -- 172.21.15.5:0/2789534064 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ece6171f0 con 0x557ece3611b0 2024-05-09T00:54:26.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.334+0000 7f4751d6b640 1 --2- 172.21.15.5:0/2789534064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ece3611b0 0x557ece7166c0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f473c016ea0 tx=0x7f473c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d68961d850cb7ea2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:54:26.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.334+0000 7f4730ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:54:26.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.335+0000 7f4730ff9640 1 -- 172.21.15.5:0/2789534064 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f473c022450 con 0x557ece3611b0 2024-05-09T00:54:26.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.335+0000 7f475a67e740 1 -- 172.21.15.5:0/2789534064 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4740011420 con 0x557ece3611b0 2024-05-09T00:54:26.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.335+0000 7f4730ff9640 1 -- 172.21.15.5:0/2789534064 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f473c0225f0 con 0x557ece3611b0 2024-05-09T00:54:26.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.335+0000 7f4730ff9640 1 -- 172.21.15.5:0/2789534064 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f473c022830 con 0x557ece3611b0 2024-05-09T00:54:26.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.336+0000 7f475a67e740 10 client.16314 fetch_fsmap learned FSMap version 31 2024-05-09T00:54:26.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.336+0000 7f475a67e740 1 -- 172.21.15.5:0/2789534064 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x557ece3cc790 con 0x557ece3611b0 2024-05-09T00:54:26.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.340+0000 7f4730ff9640 1 -- 172.21.15.5:0/2789534064 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f473c027070 con 0x557ece3611b0 2024-05-09T00:54:26.342 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.341+0000 7f4730ff9640 1 -- 172.21.15.5:0/2789534064 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f473c0213b0 con 0x557ece3611b0 2024-05-09T00:54:26.342 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.341+0000 7f475a67e740 10 client.16314 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:54:26.343 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.341+0000 7f475a67e740 10 client.16314 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:54:26.343 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.341+0000 7f475a67e740 20 client.16314 populate_metadata read hostname 'smithi005' 2024-05-09T00:54:26.343 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.341+0000 7f475a67e740 1 --2- 172.21.15.5:0/2789534064 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x557ece31fd80 0x557ece71cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:26.343 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.341+0000 7f475a67e740 4 client.16314 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:54:26.343 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.341+0000 7f475a67e740 1 -- 172.21.15.5:0/2789534064 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4740011420 con 0x557ece31fd80 2024-05-09T00:54:26.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.358+0000 7f475156a640 1 --2- 172.21.15.5:0/2789534064 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x557ece31fd80 0x557ece71cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:26.379 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.375+0000 7f475156a640 1 --2- 172.21.15.5:0/2789534064 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x557ece31fd80 0x557ece71cb40 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26ff2d6ada014b95 server_cookie=5846679b0b5fa4f3 in_seq=0 out_seq=0 2024-05-09T00:54:26.379 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.375+0000 7f4730ff9640 10 client.16314 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:54:26.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.379+0000 7f4730ff9640 1 -- 172.21.15.5:0/2789534064 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f4740011420 con 0x557ece31fd80 2024-05-09T00:54:26.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.379+0000 7f4730ff9640 10 client.16314 handle_command_reply: tid=0 2024-05-09T00:54:26.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.379+0000 7f475a67e740 1 client.16314 shutdown 2024-05-09T00:54:26.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.384+0000 7f475a67e740 1 -- 172.21.15.5:0/2789534064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ece3611b0 msgr2=0x557ece7166c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:54:26.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.384+0000 7f475a67e740 1 --2- 172.21.15.5:0/2789534064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ece3611b0 0x557ece7166c0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f473c016ea0 tx=0x7f473c00b040 comp rx=0 tx=0).stop 2024-05-09T00:54:26.388 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.387+0000 7f475a67e740 1 -- 172.21.15.5:0/2789534064 shutdown_connections 2024-05-09T00:54:26.388 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.387+0000 7f475a67e740 1 --2- 172.21.15.5:0/2789534064 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x557ece31fd80 0x557ece71cb40 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:26.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.387+0000 7f475a67e740 1 --2- 172.21.15.5:0/2789534064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ece3611b0 0x557ece7166c0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:26.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.387+0000 7f475a67e740 1 --2- 172.21.15.5:0/2789534064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ece3404d0 0x557ece70f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:26.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.387+0000 7f475a67e740 1 --2- 172.21.15.5:0/2789534064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ece3382f0 0x557ece70f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:26.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.388+0000 7f475a67e740 1 -- 172.21.15.5:0/2789534064 >> 172.21.15.5:0/2789534064 conn(0x557ece355300 msgr2=0x557ece703da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:54:26.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.395+0000 7f4730ff9640 0 client.16314 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:54:26.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.399+0000 7f475a67e740 1 -- 172.21.15.5:0/2789534064 shutdown_connections 2024-05-09T00:54:26.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.404+0000 7f475a67e740 1 -- 172.21.15.5:0/2789534064 wait complete. 2024-05-09T00:54:26.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.404+0000 7f475a67e740 20 client.16314 trim_cache size 0 max 16384 2024-05-09T00:54:26.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.458+0000 7f475a67e740 1 Processor -- start 2024-05-09T00:54:26.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.458+0000 7f475a67e740 1 -- start start 2024-05-09T00:54:26.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.459+0000 7f475a67e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ece3382f0 0x557ece70ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:26.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.459+0000 7f475a67e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ece3404d0 0x557ece7165e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:26.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.459+0000 7f475a67e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ece3611b0 0x557ece716b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:26.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.460+0000 7f475a67e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f473800e5c0 con 0x557ece3404d0 2024-05-09T00:54:26.464 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.460+0000 7f475a67e740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4738010100 con 0x557ece3382f0 2024-05-09T00:54:26.464 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.460+0000 7f475a67e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f473800ec90 con 0x557ece3611b0 2024-05-09T00:54:26.464 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.460+0000 7f4750d69640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ece3404d0 0x557ece7165e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:26.464 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.460+0000 7f4750d69640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ece3404d0 0x557ece7165e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59340/0 (socket says 172.21.15.5:59340) 2024-05-09T00:54:26.464 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.460+0000 7f4750d69640 1 -- 172.21.15.5:0/2486566725 learned_addr learned my addr 172.21.15.5:0/2486566725 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:54:26.464 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.461+0000 7f4750d69640 1 -- 172.21.15.5:0/2486566725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ece3611b0 msgr2=0x557ece716b20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:54:26.464 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.461+0000 7f4750d69640 1 --2- 172.21.15.5:0/2486566725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ece3611b0 0x557ece716b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:26.464 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.461+0000 7f4750d69640 1 -- 172.21.15.5:0/2486566725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ece3382f0 msgr2=0x557ece70ebe0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:54:26.464 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.461+0000 7f4750d69640 1 --2- 172.21.15.5:0/2486566725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ece3382f0 0x557ece70ebe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:26.464 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.461+0000 7f4750d69640 1 -- 172.21.15.5:0/2486566725 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ece70e2c0 con 0x557ece3404d0 2024-05-09T00:54:26.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.471+0000 7f4750d69640 1 --2- 172.21.15.5:0/2486566725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ece3404d0 0x557ece7165e0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f4740006be0 tx=0x7f4740004640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89e734cd2cdc8341 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:54:26.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.475+0000 7f4732ffd640 1 -- 172.21.15.5:0/2486566725 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4740015050 con 0x557ece3404d0 2024-05-09T00:54:26.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.475+0000 7f4732ffd640 1 -- 172.21.15.5:0/2486566725 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f474001e070 con 0x557ece3404d0 2024-05-09T00:54:26.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.475+0000 7f4732ffd640 1 -- 172.21.15.5:0/2486566725 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474001a680 con 0x557ece3404d0 2024-05-09T00:54:26.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.477+0000 7f475a67e740 1 -- 172.21.15.5:0/2486566725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ece3404d0 msgr2=0x557ece7165e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:54:26.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.477+0000 7f475a67e740 1 --2- 172.21.15.5:0/2486566725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ece3404d0 0x557ece7165e0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f4740006be0 tx=0x7f4740004640 comp rx=0 tx=0).stop 2024-05-09T00:54:26.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.477+0000 7f475a67e740 1 -- 172.21.15.5:0/2486566725 shutdown_connections 2024-05-09T00:54:26.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.477+0000 7f475a67e740 1 --2- 172.21.15.5:0/2486566725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ece3611b0 0x557ece716b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:26.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.477+0000 7f475a67e740 1 --2- 172.21.15.5:0/2486566725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ece3404d0 0x557ece7165e0 secure :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f4740006be0 tx=0x7f4740004640 comp rx=0 tx=0).stop 2024-05-09T00:54:26.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.477+0000 7f475a67e740 1 --2- 172.21.15.5:0/2486566725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ece3382f0 0x557ece70ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:26.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.477+0000 7f475a67e740 1 -- 172.21.15.5:0/2486566725 >> 172.21.15.5:0/2486566725 conn(0x557ece355300 msgr2=0x557ece70e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:54:26.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.480+0000 7f475a67e740 1 -- 172.21.15.5:0/2486566725 shutdown_connections 2024-05-09T00:54:26.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.481+0000 7f475a67e740 1 -- 172.21.15.5:0/2486566725 wait complete. 2024-05-09T00:54:26.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.482+0000 7f475a67e740 1 -- start start 2024-05-09T00:54:26.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.482+0000 7f475a67e740 1 Processor -- start 2024-05-09T00:54:26.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.483+0000 7f475a67e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ece3382f0 0x557ece714040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:26.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.483+0000 7f475a67e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ece3611b0 0x557ece714580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:26.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.483+0000 7f475a67e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ece2f51c0 0x557ece7154b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:26.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.483+0000 7f475a67e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f473800fa30 con 0x557ece2f51c0 2024-05-09T00:54:26.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.483+0000 7f475a67e740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f473800f360 con 0x557ece3382f0 2024-05-09T00:54:26.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.483+0000 7f475a67e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f473800c5a0 con 0x557ece3611b0 2024-05-09T00:54:26.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.483+0000 7f475a67e740 10 client.0 _ll_register_callbacks cb 0x7f473c009cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:54:26.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.486+0000 7f4751d6b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ece2f51c0 0x557ece7154b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:26.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.486+0000 7f4751d6b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ece2f51c0 0x557ece7154b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59344/0 (socket says 172.21.15.5:59344) 2024-05-09T00:54:26.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.486+0000 7f4751d6b640 1 -- 172.21.15.5:0/275313458 learned_addr learned my addr 172.21.15.5:0/275313458 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:54:26.493 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.492+0000 7f4750d69640 1 --2- 172.21.15.5:0/275313458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ece3611b0 0x557ece714580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:26.493 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.492+0000 7f475156a640 1 --2- 172.21.15.5:0/275313458 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ece3382f0 0x557ece714040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:26.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.498+0000 7f4751d6b640 1 -- 172.21.15.5:0/275313458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ece3611b0 msgr2=0x557ece714580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:54:26.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.498+0000 7f4751d6b640 1 --2- 172.21.15.5:0/275313458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ece3611b0 0x557ece714580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:26.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.498+0000 7f4751d6b640 1 -- 172.21.15.5:0/275313458 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ece3382f0 msgr2=0x557ece714040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:54:26.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.498+0000 7f4751d6b640 1 --2- 172.21.15.5:0/275313458 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ece3382f0 0x557ece714040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:26.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.498+0000 7f4751d6b640 1 -- 172.21.15.5:0/275313458 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ece4435f0 con 0x557ece2f51c0 2024-05-09T00:54:26.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.507+0000 7f4751d6b640 1 --2- 172.21.15.5:0/275313458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ece2f51c0 0x557ece7154b0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f473c0219c0 tx=0x7f473c015ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccd59e9005e6e839 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:54:26.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.507+0000 7f4730ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:54:26.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.513+0000 7f4730ff9640 1 -- 172.21.15.5:0/275313458 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f473c022650 con 0x557ece2f51c0 2024-05-09T00:54:26.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.513+0000 7f4730ff9640 1 -- 172.21.15.5:0/275313458 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f473c008000 con 0x557ece2f51c0 2024-05-09T00:54:26.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.514+0000 7f475a67e740 1 -- 172.21.15.5:0/275313458 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f474001e070 con 0x557ece2f51c0 2024-05-09T00:54:26.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.515+0000 7f4730ff9640 1 -- 172.21.15.5:0/275313458 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f473c008200 con 0x557ece2f51c0 2024-05-09T00:54:26.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.516+0000 7f475a67e740 10 client.16320 fetch_fsmap learned FSMap version 31 2024-05-09T00:54:26.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.516+0000 7f475a67e740 1 -- 172.21.15.5:0/275313458 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x557ece62ded0 con 0x557ece2f51c0 2024-05-09T00:54:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.517+0000 7f4730ff9640 1 -- 172.21.15.5:0/275313458 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f474001e070 con 0x557ece2f51c0 2024-05-09T00:54:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.517+0000 7f4730ff9640 1 -- 172.21.15.5:0/275313458 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f473c034070 con 0x557ece2f51c0 2024-05-09T00:54:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.518+0000 7f475a67e740 10 client.16320 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:54:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.518+0000 7f475a67e740 10 client.16320 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:54:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.518+0000 7f475a67e740 20 client.16320 populate_metadata read hostname 'smithi005' 2024-05-09T00:54:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.518+0000 7f475a67e740 1 --2- 172.21.15.5:0/275313458 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x557ece784ef0 0x557ece787310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.518+0000 7f475a67e740 4 client.16320 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:54:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.518+0000 7f475a67e740 1 -- 172.21.15.5:0/275313458 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557ece6773d0 con 0x557ece784ef0 2024-05-09T00:54:26.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.521+0000 7f475156a640 1 --2- 172.21.15.5:0/275313458 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x557ece784ef0 0x557ece787310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.526+0000 7f475156a640 1 --2- 172.21.15.5:0/275313458 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x557ece784ef0 0x557ece787310 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b671ecb909363dc server_cookie=1539c9a6b4e516b3 in_seq=0 out_seq=0 2024-05-09T00:54:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:26.526+0000 7f4730ff9640 10 client.16320 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:54:27.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:54:27 smithi131 ceph-mon[34547]: pgmap v614: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 867 MiB/s wr, 1.66k op/s 2024-05-09T00:54:27.903 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:54:27 smithi005 ceph-mon[30365]: pgmap v614: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 867 MiB/s wr, 1.66k op/s 2024-05-09T00:54:27.935 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:54:27 smithi157 ceph-mon[34294]: pgmap v614: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 867 MiB/s wr, 1.66k op/s 2024-05-09T00:54:29.790 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:54:29 smithi005 ceph-mon[30365]: pgmap v615: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 855 MiB/s wr, 1.56k op/s 2024-05-09T00:54:29.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:54:29 smithi131 ceph-mon[34547]: pgmap v615: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 855 MiB/s wr, 1.56k op/s 2024-05-09T00:54:29.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:54:29 smithi157 ceph-mon[34294]: pgmap v615: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 855 MiB/s wr, 1.56k op/s 2024-05-09T00:54:30.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:30.446+0000 7f4cef59e700 1 -- 172.21.15.5:0/1231255481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4cf010b110 msgr2=0x7f4cf011d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:54:30.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:30.446+0000 7f4cef59e700 1 --2- 172.21.15.5:0/1231255481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4cf010b110 0x7f4cf011d3f0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f4ce800b0a0 tx=0x7f4ce800b3b0 comp rx=0 tx=0).stop 2024-05-09T00:54:30.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:30.448+0000 7f4cef59e700 1 -- 172.21.15.5:0/1231255481 shutdown_connections 2024-05-09T00:54:30.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:30.448+0000 7f4cef59e700 1 --2- 172.21.15.5:0/1231255481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4cf011d930 0x7f4cf011fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:30.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:30.448+0000 7f4cef59e700 1 --2- 172.21.15.5:0/1231255481 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4cf010a7c0 0x7f4cf010abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:30.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:30.448+0000 7f4cef59e700 1 --2- 172.21.15.5:0/1231255481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4cf010b110 0x7f4cf011d3f0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:30.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:30.448+0000 7f4cef59e700 1 -- 172.21.15.5:0/1231255481 >> 172.21.15.5:0/1231255481 conn(0x7f4cf006f290 msgr2=0x7f4cf006f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:54:30.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:30.449+0000 7f4cef59e700 1 -- 172.21.15.5:0/1231255481 shutdown_connections 2024-05-09T00:54:30.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:30.450+0000 7f4cef59e700 1 -- 172.21.15.5:0/1231255481 wait complete. 2024-05-09T00:54:30.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:30.452+0000 7f4cef59e700 1 Processor -- start 2024-05-09T00:54:30.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:30.452+0000 7f4cef59e700 1 -- start start 2024-05-09T00:54:30.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:30.453+0000 7f4cef59e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4cf010a7c0 0x7f4cf01af7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:30.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:30.453+0000 7f4cef59e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4cf010b110 0x7f4cf01afcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:30.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:30.453+0000 7f4cef59e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4cf011d930 0x7f4cf01a99d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:30.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:30.453+0000 7f4cef59e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf0123150 con 0x7f4cf010b110 2024-05-09T00:54:30.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:30.453+0000 7f4cef59e700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf0122fd0 con 0x7f4cf010a7c0 2024-05-09T00:54:30.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:30.453+0000 7f4cef59e700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf01232d0 con 0x7f4cf011d930 2024-05-09T00:54:30.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:30.454+0000 7f4cedd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4cf010b110 0x7f4cf01afcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:30.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:30.454+0000 7f4cee59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4cf010a7c0 0x7f4cf01af7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:30.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:30.454+0000 7f4cedd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4cf010b110 0x7f4cf01afcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59358/0 (socket says 172.21.15.5:59358) 2024-05-09T00:54:30.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:30.454+0000 7f4cedd9b700 1 -- 172.21.15.5:0/2343604275 learned_addr learned my addr 172.21.15.5:0/2343604275 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:54:30.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:30.455+0000 7f4ceed9d700 1 --2- 172.21.15.5:0/2343604275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4cf011d930 0x7f4cf01a99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:30.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:30.455+0000 7f4cedd9b700 1 -- 172.21.15.5:0/2343604275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4cf011d930 msgr2=0x7f4cf01a99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:54:30.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:30.455+0000 7f4cedd9b700 1 --2- 172.21.15.5:0/2343604275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4cf011d930 0x7f4cf01a99d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:30.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:30.455+0000 7f4cedd9b700 1 -- 172.21.15.5:0/2343604275 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4cf010a7c0 msgr2=0x7f4cf01af7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:54:30.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:30.455+0000 7f4cedd9b700 1 --2- 172.21.15.5:0/2343604275 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4cf010a7c0 0x7f4cf01af7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:30.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:30.456+0000 7f4cedd9b700 1 -- 172.21.15.5:0/2343604275 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cf01aa080 con 0x7f4cf010b110 2024-05-09T00:54:30.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:30.456+0000 7f4cedd9b700 1 --2- 172.21.15.5:0/2343604275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4cf010b110 0x7f4cf01afcf0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f4ce80062a0 tx=0x7f4ce8004f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:54:30.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:30.461+0000 7f4cdf7fe700 1 -- 172.21.15.5:0/2343604275 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce8021070 con 0x7f4cf010b110 2024-05-09T00:54:30.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:30.461+0000 7f4cdf7fe700 1 -- 172.21.15.5:0/2343604275 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ce8007e30 con 0x7f4cf010b110 2024-05-09T00:54:30.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:30.461+0000 7f4cdf7fe700 1 -- 172.21.15.5:0/2343604275 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce801d450 con 0x7f4cf010b110 2024-05-09T00:54:30.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:30.461+0000 7f4cef59e700 1 -- 172.21.15.5:0/2343604275 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cf01aa310 con 0x7f4cf010b110 2024-05-09T00:54:30.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:30.461+0000 7f4cef59e700 1 -- 172.21.15.5:0/2343604275 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cf01aa800 con 0x7f4cf010b110 2024-05-09T00:54:30.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:30.464+0000 7f4cef59e700 1 -- 172.21.15.5:0/2343604275 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cf00512c0 con 0x7f4cf010b110 2024-05-09T00:54:30.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:30.469+0000 7f4cdf7fe700 1 -- 172.21.15.5:0/2343604275 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f4ce8004000 con 0x7f4cf010b110 2024-05-09T00:54:30.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:30.471+0000 7f4cdf7fe700 1 --2- 172.21.15.5:0/2343604275 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f4cd80b2a50 0x7f4cd80b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:30.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:30.471+0000 7f4cdf7fe700 1 -- 172.21.15.5:0/2343604275 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f4ce800f020 con 0x7f4cf010b110 2024-05-09T00:54:30.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:30.471+0000 7f4cee59c700 1 --2- 172.21.15.5:0/2343604275 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f4cd80b2a50 0x7f4cd80b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:30.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:30.472+0000 7f4cee59c700 1 --2- 172.21.15.5:0/2343604275 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f4cd80b2a50 0x7f4cd80b4f60 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4005fd0 tx=0x7f4ce4005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:54:30.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:30.477+0000 7f4cdf7fe700 1 -- 172.21.15.5:0/2343604275 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4cf01aa800 con 0x7f4cf010b110 2024-05-09T00:54:31.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:31.122+0000 7f4cef59e700 1 -- 172.21.15.5:0/2343604275 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4cf01b6790 con 0x7f4cf010b110 2024-05-09T00:54:31.130 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:54:31.130 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2685771887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2685771887},{"type":"v1","addr":"172.21.15.5:6835","nonce":2685771887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/254017535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":254017535},{"type":"v1","addr":"172.21.15.5:6837","nonce":254017535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T00:43:45.856066+0000","modified":"2024-05-09T00:52:11.233619+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24508,"mds_1":24499,"mds_2":24494,"mds_3":24493,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14598":{"gid":14598,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2109255228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2109255228},{"type":"v1","addr":"172.21.15.5:6839","nonce":2109255228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/3809966329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3809966329},{"type":"v1","addr":"172.21.15.5:6841","nonce":3809966329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6833/3272897742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3272897742},{"type":"v1","addr":"172.21.15.131:6833","nonce":3272897742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1283093898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1283093898},{"type":"v1","addr":"172.21.15.131:6835","nonce":1283093898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/2809525455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2809525455},{"type":"v1","addr":"172.21.15.157:6833","nonce":2809525455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.131:6837/1633006186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":1633006186},{"type":"v1","addr":"172.21.15.131:6837","nonce":1633006186}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.157:6835/832898273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":832898273},{"type":"v1","addr":"172.21.15.157:6835","nonce":832898273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.131:6839/2748669631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2748669631},{"type":"v1","addr":"172.21.15.131:6839","nonce":2748669631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.157:6837/744863009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":744863009},{"type":"v1","addr":"172.21.15.157:6837","nonce":744863009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.157:6839/2779860399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":2779860399},{"type":"v1","addr":"172.21.15.157:6839","nonce":2779860399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24508,24494,24499,24493,24467]},"id":1}]} 2024-05-09T00:54:31.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:31.128+0000 7f4cdf7fe700 1 -- 172.21.15.5:0/2343604275 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11926 (secure 0 0 0) 0x7f4ce802a020 con 0x7f4cf010b110 2024-05-09T00:54:31.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:31.134+0000 7f4cef59e700 1 -- 172.21.15.5:0/2343604275 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f4cd80b2a50 msgr2=0x7f4cd80b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:54:31.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:31.134+0000 7f4cef59e700 1 --2- 172.21.15.5:0/2343604275 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f4cd80b2a50 0x7f4cd80b4f60 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4005fd0 tx=0x7f4ce4005f60 comp rx=0 tx=0).stop 2024-05-09T00:54:31.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:31.134+0000 7f4cef59e700 1 -- 172.21.15.5:0/2343604275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4cf010b110 msgr2=0x7f4cf01afcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:54:31.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:31.134+0000 7f4cef59e700 1 --2- 172.21.15.5:0/2343604275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4cf010b110 0x7f4cf01afcf0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f4ce80062a0 tx=0x7f4ce8004f10 comp rx=0 tx=0).stop 2024-05-09T00:54:31.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:31.134+0000 7f4cef59e700 1 -- 172.21.15.5:0/2343604275 shutdown_connections 2024-05-09T00:54:31.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:31.134+0000 7f4cef59e700 1 --2- 172.21.15.5:0/2343604275 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f4cd80b2a50 0x7f4cd80b4f60 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:31.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:31.134+0000 7f4cef59e700 1 --2- 172.21.15.5:0/2343604275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4cf011d930 0x7f4cf01a99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:31.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:31.135+0000 7f4cef59e700 1 --2- 172.21.15.5:0/2343604275 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4cf010a7c0 0x7f4cf01af7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:31.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:31.135+0000 7f4cef59e700 1 --2- 172.21.15.5:0/2343604275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4cf010b110 0x7f4cf01afcf0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:31.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:31.135+0000 7f4cef59e700 1 -- 172.21.15.5:0/2343604275 >> 172.21.15.5:0/2343604275 conn(0x7f4cf006f290 msgr2=0x7f4cf011b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:54:31.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:31.136+0000 7f4cef59e700 1 -- 172.21.15.5:0/2343604275 shutdown_connections 2024-05-09T00:54:31.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:31.137+0000 7f4cef59e700 1 -- 172.21.15.5:0/2343604275 wait complete. 2024-05-09T00:54:31.160 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 31 2024-05-09T00:54:31.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:54:31 smithi131 ceph-mon[34547]: pgmap v616: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 995 MiB/s wr, 2.06k op/s 2024-05-09T00:54:31.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:54:31 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/2343604275' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:54:31.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:54:31 smithi157 ceph-mon[34294]: pgmap v616: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 995 MiB/s wr, 2.06k op/s 2024-05-09T00:54:31.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:54:31 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/2343604275' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:54:32.043 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:54:31 smithi005 ceph-mon[30365]: pgmap v616: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 995 MiB/s wr, 2.06k op/s 2024-05-09T00:54:32.043 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:54:31 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/2343604275' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:54:32.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:32.699+0000 7f4730ff9640 1 -- 172.21.15.5:0/275313458 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x557ece6773d0 con 0x557ece784ef0 2024-05-09T00:54:32.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:32.699+0000 7f4730ff9640 10 client.16320 handle_command_reply: tid=0 2024-05-09T00:54:32.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:32.699+0000 7f475a67e740 1 client.16320 shutdown 2024-05-09T00:54:32.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:32.700+0000 7f475a67e740 1 -- 172.21.15.5:0/275313458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ece2f51c0 msgr2=0x557ece7154b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:54:32.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:32.700+0000 7f475a67e740 1 --2- 172.21.15.5:0/275313458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ece2f51c0 0x557ece7154b0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f473c0219c0 tx=0x7f473c015ce0 comp rx=0 tx=0).stop 2024-05-09T00:54:32.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:32.700+0000 7f475a67e740 1 -- 172.21.15.5:0/275313458 shutdown_connections 2024-05-09T00:54:32.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:32.700+0000 7f475a67e740 1 --2- 172.21.15.5:0/275313458 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x557ece784ef0 0x557ece787310 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:32.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:32.700+0000 7f475a67e740 1 --2- 172.21.15.5:0/275313458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ece2f51c0 0x557ece7154b0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:32.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:32.700+0000 7f475a67e740 1 --2- 172.21.15.5:0/275313458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ece3611b0 0x557ece714580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:32.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:32.700+0000 7f475a67e740 1 --2- 172.21.15.5:0/275313458 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ece3382f0 0x557ece714040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:32.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:32.700+0000 7f475a67e740 1 -- 172.21.15.5:0/275313458 >> 172.21.15.5:0/275313458 conn(0x557ece355300 msgr2=0x557ece703d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:54:32.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:32.701+0000 7f4730ff9640 0 client.16320 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:54:32.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:32.701+0000 7f475a67e740 1 -- 172.21.15.5:0/275313458 shutdown_connections 2024-05-09T00:54:32.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:32.701+0000 7f475a67e740 1 -- 172.21.15.5:0/275313458 wait complete. 2024-05-09T00:54:32.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:32.701+0000 7f475a67e740 20 client.16320 trim_cache size 0 max 16384 2024-05-09T00:54:32.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-09T00:54:32.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-09T00:54:32.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-09T00:54:32.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:32.704+0000 7f47537ce640 1 -- 172.21.15.5:0/973497681 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f47340b2930 msgr2=0x7f47340b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:54:32.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:32.704+0000 7f47537ce640 1 --2- 172.21.15.5:0/973497681 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f47340b2930 0x7f47340b4e40 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f473c009fd0 tx=0x7f473c004b50 comp rx=0 tx=0).stop 2024-05-09T00:54:32.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:32.704+0000 7f47537ce640 1 -- 172.21.15.5:0/973497681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f474c106f50 msgr2=0x7f474c1047b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:54:32.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:32.704+0000 7f47537ce640 1 --2- 172.21.15.5:0/973497681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f474c106f50 0x7f474c1047b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f474c069b10 tx=0x7f4744004b80 comp rx=0 tx=0).stop 2024-05-09T00:54:32.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:32.704+0000 7f47537ce640 1 -- 172.21.15.5:0/973497681 shutdown_connections 2024-05-09T00:54:32.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:32.705+0000 7f47537ce640 1 --2- 172.21.15.5:0/973497681 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f47340b2930 0x7f47340b4e40 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:32.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:32.705+0000 7f47537ce640 1 --2- 172.21.15.5:0/973497681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f474c103390 0x7f474c1037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:32.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:32.705+0000 7f47537ce640 1 --2- 172.21.15.5:0/973497681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f474c109900 0x7f474c102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:32.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:32.705+0000 7f47537ce640 1 --2- 172.21.15.5:0/973497681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f474c106f50 0x7f474c1047b0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:32.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:32.705+0000 7f47537ce640 1 -- 172.21.15.5:0/973497681 >> 172.21.15.5:0/973497681 conn(0x7f474c0fd080 msgr2=0x7f474c107f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:54:32.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:32.705+0000 7f47537ce640 1 -- 172.21.15.5:0/973497681 shutdown_connections 2024-05-09T00:54:32.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:32.705+0000 7f47537ce640 1 -- 172.21.15.5:0/973497681 wait complete. 2024-05-09T00:54:33.636 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-09T00:54:33.790 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:54:33 smithi005 ceph-mon[30365]: pgmap v617: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 805 MiB/s wr, 1.72k op/s 2024-05-09T00:54:33.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:54:33 smithi131 ceph-mon[34547]: pgmap v617: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 805 MiB/s wr, 1.72k op/s 2024-05-09T00:54:33.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:54:33 smithi157 ceph-mon[34294]: pgmap v617: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 805 MiB/s wr, 1.72k op/s 2024-05-09T00:54:34.338 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:54:35.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:54:35 smithi005 ceph-mon[30365]: pgmap v618: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 874 MiB/s wr, 1.83k op/s 2024-05-09T00:54:35.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:54:35 smithi131 ceph-mon[34547]: pgmap v618: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 874 MiB/s wr, 1.83k op/s 2024-05-09T00:54:35.935 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:54:35 smithi157 ceph-mon[34294]: pgmap v618: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 874 MiB/s wr, 1.83k op/s 2024-05-09T00:54:37.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:54:37 smithi005 ceph-mon[30365]: pgmap v619: 97 pgs: 97 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 852 MiB/s wr, 1.72k op/s 2024-05-09T00:54:37.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:54:37 smithi157 ceph-mon[34294]: pgmap v619: 97 pgs: 97 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 852 MiB/s wr, 1.72k op/s 2024-05-09T00:54:38.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:54:37 smithi131 ceph-mon[34547]: pgmap v619: 97 pgs: 97 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 852 MiB/s wr, 1.72k op/s 2024-05-09T00:54:39.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:54:39 smithi157 ceph-mon[34294]: pgmap v620: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 848 MiB/s wr, 1.74k op/s 2024-05-09T00:54:39.977 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:54:39 smithi005 ceph-mon[30365]: pgmap v620: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 848 MiB/s wr, 1.74k op/s 2024-05-09T00:54:40.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:54:39 smithi131 ceph-mon[34547]: pgmap v620: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 848 MiB/s wr, 1.74k op/s 2024-05-09T00:54:41.600 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-09T00:54:41.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.111+0000 7f3d0eb9f700 1 -- 172.21.15.5:0/1275682909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d08108760 msgr2=0x7f3d081100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:54:41.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.111+0000 7f3d0eb9f700 1 --2- 172.21.15.5:0/1275682909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d08108760 0x7f3d081100c0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f3d0000b0d0 tx=0x7f3d0000b3e0 comp rx=0 tx=0).stop 2024-05-09T00:54:41.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.112+0000 7f3d0eb9f700 1 -- 172.21.15.5:0/1275682909 shutdown_connections 2024-05-09T00:54:41.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.112+0000 7f3d0eb9f700 1 --2- 172.21.15.5:0/1275682909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d08108760 0x7f3d081100c0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:41.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.112+0000 7f3d0eb9f700 1 --2- 172.21.15.5:0/1275682909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d081034e0 0x7f3d081038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:41.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.112+0000 7f3d0eb9f700 1 --2- 172.21.15.5:0/1275682909 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d08107e10 0x7f3d08108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:41.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.112+0000 7f3d0eb9f700 1 -- 172.21.15.5:0/1275682909 >> 172.21.15.5:0/1275682909 conn(0x7f3d080fef30 msgr2=0x7f3d08101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:54:41.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.112+0000 7f3d0eb9f700 1 -- 172.21.15.5:0/1275682909 shutdown_connections 2024-05-09T00:54:41.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.112+0000 7f3d0eb9f700 1 -- 172.21.15.5:0/1275682909 wait complete. 2024-05-09T00:54:41.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.113+0000 7f3d0eb9f700 1 Processor -- start 2024-05-09T00:54:41.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.113+0000 7f3d0eb9f700 1 -- start start 2024-05-09T00:54:41.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.113+0000 7f3d0eb9f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d081034e0 0x7f3d0819be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:41.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.113+0000 7f3d0eb9f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d08107e10 0x7f3d0819c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:41.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.113+0000 7f3d0eb9f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d08108760 0x7f3d0819c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:41.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.114+0000 7f3d0eb9f700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3d081134f0 con 0x7f3d08108760 2024-05-09T00:54:41.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.114+0000 7f3d0eb9f700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3d08113370 con 0x7f3d08107e10 2024-05-09T00:54:41.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.114+0000 7f3d0eb9f700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3d08113670 con 0x7f3d081034e0 2024-05-09T00:54:41.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.114+0000 7f3d0d13c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d08108760 0x7f3d0819c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:41.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.114+0000 7f3d0d13c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d08108760 0x7f3d0819c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49322/0 (socket says 172.21.15.5:49322) 2024-05-09T00:54:41.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.114+0000 7f3d0d13c700 1 -- 172.21.15.5:0/1493823221 learned_addr learned my addr 172.21.15.5:0/1493823221 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:54:41.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.114+0000 7f3d07fff700 1 --2- 172.21.15.5:0/1493823221 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d08107e10 0x7f3d0819c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:41.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.115+0000 7f3d0d13c700 1 -- 172.21.15.5:0/1493823221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d081034e0 msgr2=0x7f3d0819be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:54:41.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.115+0000 7f3d0d13c700 1 --2- 172.21.15.5:0/1493823221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d081034e0 0x7f3d0819be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:41.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.115+0000 7f3d0d13c700 1 -- 172.21.15.5:0/1493823221 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d08107e10 msgr2=0x7f3d0819c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:54:41.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.115+0000 7f3d0d13c700 1 --2- 172.21.15.5:0/1493823221 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d08107e10 0x7f3d0819c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:41.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.115+0000 7f3d0d13c700 1 -- 172.21.15.5:0/1493823221 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d081a71c0 con 0x7f3d08108760 2024-05-09T00:54:41.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.115+0000 7f3d0d13c700 1 --2- 172.21.15.5:0/1493823221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d08108760 0x7f3d0819c910 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f3d00009bb0 tx=0x7f3d00007ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:54:41.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.115+0000 7f3d05ffb700 1 -- 172.21.15.5:0/1493823221 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d00021070 con 0x7f3d08108760 2024-05-09T00:54:41.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.115+0000 7f3d0eb9f700 1 -- 172.21.15.5:0/1493823221 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d081a7450 con 0x7f3d08108760 2024-05-09T00:54:41.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.115+0000 7f3d05ffb700 1 -- 172.21.15.5:0/1493823221 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d0001db00 con 0x7f3d08108760 2024-05-09T00:54:41.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.115+0000 7f3d05ffb700 1 -- 172.21.15.5:0/1493823221 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d0001bb70 con 0x7f3d08108760 2024-05-09T00:54:41.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.116+0000 7f3d0eb9f700 1 -- 172.21.15.5:0/1493823221 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d081a7900 con 0x7f3d08108760 2024-05-09T00:54:41.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.121+0000 7f3d05ffb700 1 -- 172.21.15.5:0/1493823221 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f3d00024070 con 0x7f3d08108760 2024-05-09T00:54:41.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.123+0000 7f3d05ffb700 1 --2- 172.21.15.5:0/1493823221 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f3cf80b2940 0x7f3cf80b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:41.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.124+0000 7f3d05ffb700 1 -- 172.21.15.5:0/1493823221 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f3d0000f020 con 0x7f3d08108760 2024-05-09T00:54:41.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.124+0000 7f3d1d5b5b80 1 Processor -- start 2024-05-09T00:54:41.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.124+0000 7f3d1d5b5b80 1 -- start start 2024-05-09T00:54:41.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.125+0000 7f3d0c93b700 1 --2- 172.21.15.5:0/1493823221 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f3cf80b2940 0x7f3cf80b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:41.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.125+0000 7f3d1d5b5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56421310cd00 0x564213215240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:41.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.125+0000 7f3d1d5b5b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642130924a0 0x564213215780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:41.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.125+0000 7f3d1d5b5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56421320f400 0x56421320f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:41.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.125+0000 7f3d1d5b5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5642131320a0 con 0x56421320f400 2024-05-09T00:54:41.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.125+0000 7f3d1d5b5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5642130d79a0 con 0x56421310cd00 2024-05-09T00:54:41.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.125+0000 7f3d1d5b5b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56421312e060 con 0x5642130924a0 2024-05-09T00:54:41.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.125+0000 7f3d0c93b700 1 --2- 172.21.15.5:0/1493823221 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f3cf80b2940 0x7f3cf80b4e50 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc009770 tx=0x7f3cfc006cd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:54:41.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.126+0000 7f3d0d13c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56421320f400 0x56421320f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:41.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.126+0000 7f3d0d13c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56421320f400 0x56421320f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49334/0 (socket says 172.21.15.5:49334) 2024-05-09T00:54:41.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.126+0000 7f3d0d13c700 1 -- 172.21.15.5:0/4277312879 learned_addr learned my addr 172.21.15.5:0/4277312879 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:54:41.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.126+0000 7f3d0d13c700 1 -- 172.21.15.5:0/4277312879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642130924a0 msgr2=0x564213215780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:54:41.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.126+0000 7f3d0d13c700 1 --2- 172.21.15.5:0/4277312879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642130924a0 0x564213215780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:41.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.126+0000 7f3d0d13c700 1 -- 172.21.15.5:0/4277312879 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56421310cd00 msgr2=0x564213215240 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:54:41.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.126+0000 7f3d0d13c700 1 --2- 172.21.15.5:0/4277312879 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56421310cd00 0x564213215240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:41.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.126+0000 7f3d0d13c700 1 -- 172.21.15.5:0/4277312879 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642130c8500 con 0x56421320f400 2024-05-09T00:54:41.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.126+0000 7f3d0d13c700 1 --2- 172.21.15.5:0/4277312879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56421320f400 0x56421320f810 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f3d000aac90 tx=0x7f3d00006000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f9592450b17f5c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:54:41.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.127+0000 7f3cf2ffd700 1 -- 172.21.15.5:0/4277312879 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d000aaf90 con 0x56421320f400 2024-05-09T00:54:41.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.127+0000 7f3cf2ffd700 1 -- 172.21.15.5:0/4277312879 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d000af340 con 0x56421320f400 2024-05-09T00:54:41.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.127+0000 7f3cf2ffd700 1 -- 172.21.15.5:0/4277312879 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d000ab450 con 0x56421320f400 2024-05-09T00:54:41.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.128+0000 7f3d1d5b5b80 1 -- 172.21.15.5:0/4277312879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56421320f400 msgr2=0x56421320f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:54:41.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.128+0000 7f3d1d5b5b80 1 --2- 172.21.15.5:0/4277312879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56421320f400 0x56421320f810 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f3d000aac90 tx=0x7f3d00006000 comp rx=0 tx=0).stop 2024-05-09T00:54:41.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.128+0000 7f3d1d5b5b80 1 -- 172.21.15.5:0/4277312879 shutdown_connections 2024-05-09T00:54:41.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.128+0000 7f3d1d5b5b80 1 --2- 172.21.15.5:0/4277312879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56421320f400 0x56421320f810 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:41.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.128+0000 7f3d1d5b5b80 1 --2- 172.21.15.5:0/4277312879 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56421310cd00 0x564213215240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:41.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.128+0000 7f3d1d5b5b80 1 --2- 172.21.15.5:0/4277312879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642130924a0 0x564213215780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:41.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.128+0000 7f3d1d5b5b80 1 -- 172.21.15.5:0/4277312879 >> 172.21.15.5:0/4277312879 conn(0x564213095f70 msgr2=0x56421320a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:54:41.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.128+0000 7f3d1d5b5b80 1 -- 172.21.15.5:0/4277312879 shutdown_connections 2024-05-09T00:54:41.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.128+0000 7f3d1d5b5b80 1 -- 172.21.15.5:0/4277312879 wait complete. 2024-05-09T00:54:41.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.129+0000 7f3d1d5b5b80 1 -- start start 2024-05-09T00:54:41.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.129+0000 7f3d1d5b5b80 1 Processor -- start 2024-05-09T00:54:41.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.130+0000 7f3d1d5b5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642130924a0 0x56421321fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:41.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.130+0000 7f3d1d5b5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56421310cd00 0x564213218f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:41.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.130+0000 7f3d1d5b5b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56421320f400 0x5642132194d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:41.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.130+0000 7f3d1d5b5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56421320ffb0 con 0x56421310cd00 2024-05-09T00:54:41.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.130+0000 7f3d1d5b5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5642130ccb20 con 0x5642130924a0 2024-05-09T00:54:41.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.130+0000 7f3d1d5b5b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x564213210130 con 0x56421320f400 2024-05-09T00:54:41.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.130+0000 7f3d1d5b5b80 10 client.0 _ll_register_callbacks cb 0x564212fa36c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:54:41.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.130+0000 7f3d0c93b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642130924a0 0x56421321fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:41.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.130+0000 7f3d0c93b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642130924a0 0x56421321fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:42108/0 (socket says 172.21.15.5:42108) 2024-05-09T00:54:41.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.130+0000 7f3d0c93b700 1 -- 172.21.15.5:0/3863203267 learned_addr learned my addr 172.21.15.5:0/3863203267 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:54:41.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.131+0000 7f3d0c93b700 1 -- 172.21.15.5:0/3863203267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56421320f400 msgr2=0x5642132194d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:54:41.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.131+0000 7f3d0c93b700 1 --2- 172.21.15.5:0/3863203267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56421320f400 0x5642132194d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:41.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.131+0000 7f3d0c93b700 1 -- 172.21.15.5:0/3863203267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56421310cd00 msgr2=0x564213218f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:54:41.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.131+0000 7f3d0c93b700 1 --2- 172.21.15.5:0/3863203267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56421310cd00 0x564213218f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:41.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.131+0000 7f3d0c93b700 1 -- 172.21.15.5:0/3863203267 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564212fad490 con 0x5642130924a0 2024-05-09T00:54:41.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.132+0000 7f3d0c93b700 1 --2- 172.21.15.5:0/3863203267 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642130924a0 0x56421321fc50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc012420 tx=0x7f3cfc0047e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ae849c084e42694 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:54:41.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.132+0000 7f3ce7fff700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-09T00:54:41.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.132+0000 7f3ce7fff700 1 -- 172.21.15.5:0/3863203267 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc023060 con 0x5642130924a0 2024-05-09T00:54:41.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.132+0000 7f3ce7fff700 1 -- 172.21.15.5:0/3863203267 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f3cfc0150d0 con 0x5642130924a0 2024-05-09T00:54:41.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.132+0000 7f3d1d5b5b80 1 -- 172.21.15.5:0/3863203267 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d000af340 con 0x5642130924a0 2024-05-09T00:54:41.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.133+0000 7f3ce7fff700 1 -- 172.21.15.5:0/3863203267 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3cfc012e30 con 0x5642130924a0 2024-05-09T00:54:41.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.133+0000 7f3d1d5b5b80 10 client.25841 fetch_fsmap learned FSMap version 31 2024-05-09T00:54:41.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.133+0000 7f3d1d5b5b80 1 -- 172.21.15.5:0/3863203267 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56421322f160 con 0x5642130924a0 2024-05-09T00:54:41.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.134+0000 7f3ce7fff700 1 -- 172.21.15.5:0/3863203267 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d000af340 con 0x5642130924a0 2024-05-09T00:54:41.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.134+0000 7f3ce7fff700 1 -- 172.21.15.5:0/3863203267 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f3cfc029410 con 0x5642130924a0 2024-05-09T00:54:41.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.134+0000 7f3d1d5b5b80 10 client.25841 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:54:41.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.134+0000 7f3d1d5b5b80 10 client.25841 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T00:54:41.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.134+0000 7f3d1d5b5b80 20 client.25841 populate_metadata read hostname 'smithi005' 2024-05-09T00:54:41.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.134+0000 7f3d1d5b5b80 1 --2- 172.21.15.5:0/3863203267 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x564213231740 0x564213233be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:41.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.134+0000 7f3d1d5b5b80 4 client.25841 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:54:41.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.134+0000 7f3d1d5b5b80 1 -- 172.21.15.5:0/3863203267 --> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56421313fa80 con 0x564213231740 2024-05-09T00:54:41.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.134+0000 7f3d07fff700 1 --2- 172.21.15.5:0/3863203267 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x564213231740 0x564213233be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:41.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.136+0000 7f3d07fff700 1 --2- 172.21.15.5:0/3863203267 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x564213231740 0x564213233be0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8776350e2553d68a server_cookie=d687883ddde88877 in_seq=0 out_seq=0 2024-05-09T00:54:41.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.136+0000 7f3ce7fff700 10 client.25841 ms_handle_connect on v2:172.21.15.131:6836/1633006186 2024-05-09T00:54:41.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.138+0000 7f3ce7fff700 1 -- 172.21.15.5:0/3863203267 <== mds.4 v2:172.21.15.131:6836/1633006186 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56421313fa80 con 0x564213231740 2024-05-09T00:54:41.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.138+0000 7f3ce7fff700 10 client.25841 handle_command_reply: tid=0 2024-05-09T00:54:41.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.138+0000 7f3d1d5b5b80 1 client.25841 shutdown 2024-05-09T00:54:41.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.139+0000 7f3d1d5b5b80 1 -- 172.21.15.5:0/3863203267 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642130924a0 msgr2=0x56421321fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:54:41.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.139+0000 7f3d1d5b5b80 1 --2- 172.21.15.5:0/3863203267 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642130924a0 0x56421321fc50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc012420 tx=0x7f3cfc0047e0 comp rx=0 tx=0).stop 2024-05-09T00:54:41.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.139+0000 7f3d1d5b5b80 1 -- 172.21.15.5:0/3863203267 shutdown_connections 2024-05-09T00:54:41.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.139+0000 7f3d1d5b5b80 1 --2- 172.21.15.5:0/3863203267 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x564213231740 0x564213233be0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:41.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.139+0000 7f3ce7fff700 0 client.25841 ms_handle_reset on v2:172.21.15.131:6836/1633006186 2024-05-09T00:54:41.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.139+0000 7f3d1d5b5b80 1 --2- 172.21.15.5:0/3863203267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56421320f400 0x5642132194d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:41.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.139+0000 7f3d1d5b5b80 1 --2- 172.21.15.5:0/3863203267 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642130924a0 0x56421321fc50 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:41.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.139+0000 7f3d1d5b5b80 1 --2- 172.21.15.5:0/3863203267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56421310cd00 0x564213218f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:41.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.139+0000 7f3d1d5b5b80 1 -- 172.21.15.5:0/3863203267 >> 172.21.15.5:0/3863203267 conn(0x564213095f70 msgr2=0x564213203ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:54:41.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.139+0000 7f3d1d5b5b80 1 -- 172.21.15.5:0/3863203267 shutdown_connections 2024-05-09T00:54:41.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.139+0000 7f3d1d5b5b80 1 -- 172.21.15.5:0/3863203267 wait complete. 2024-05-09T00:54:41.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.140+0000 7f3d1d5b5b80 20 client.25841 trim_cache size 0 max 16384 2024-05-09T00:54:41.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.185+0000 7f3d1d5b5b80 1 Processor -- start 2024-05-09T00:54:41.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.185+0000 7f3d1d5b5b80 1 -- start start 2024-05-09T00:54:41.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.186+0000 7f3d1d5b5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642130924a0 0x564213217ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:41.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.186+0000 7f3d1d5b5b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56421310cd00 0x5642132147f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:41.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.186+0000 7f3d1d5b5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56421320f400 0x564213203ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:41.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.186+0000 7f3d1d5b5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3d081acb00 con 0x5642130924a0 2024-05-09T00:54:41.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.186+0000 7f3d1d5b5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3d081ae640 con 0x56421320f400 2024-05-09T00:54:41.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.186+0000 7f3d1d5b5b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3d081ad1d0 con 0x56421310cd00 2024-05-09T00:54:41.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.189+0000 7f3d0d13c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56421320f400 0x564213203ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:41.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.191+0000 7f3d07fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56421310cd00 0x5642132147f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:41.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.191+0000 7f3d07fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56421310cd00 0x5642132147f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:34030/0 (socket says 172.21.15.5:34030) 2024-05-09T00:54:41.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.191+0000 7f3d07fff700 1 -- 172.21.15.5:0/4294159865 learned_addr learned my addr 172.21.15.5:0/4294159865 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:54:41.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.195+0000 7f3d07fff700 1 -- 172.21.15.5:0/4294159865 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56421320f400 msgr2=0x564213203ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:54:41.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.195+0000 7f3d07fff700 1 --2- 172.21.15.5:0/4294159865 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56421320f400 0x564213203ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:41.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.195+0000 7f3d07fff700 1 -- 172.21.15.5:0/4294159865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642130924a0 msgr2=0x564213217ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:54:41.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.195+0000 7f3d07fff700 1 --2- 172.21.15.5:0/4294159865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642130924a0 0x564213217ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:41.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.195+0000 7f3d07fff700 1 -- 172.21.15.5:0/4294159865 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564213234170 con 0x56421310cd00 2024-05-09T00:54:41.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.198+0000 7f3d07fff700 1 --2- 172.21.15.5:0/4294159865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56421310cd00 0x5642132147f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3cf4000c00 tx=0x7f3cf4009d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=839c79571d54eab1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:54:41.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.201+0000 7f3cf27fc700 1 -- 172.21.15.5:0/4294159865 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf4009e90 con 0x56421310cd00 2024-05-09T00:54:41.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.202+0000 7f3cf27fc700 1 -- 172.21.15.5:0/4294159865 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cf401d070 con 0x56421310cd00 2024-05-09T00:54:41.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.203+0000 7f3d1d5b5b80 1 -- 172.21.15.5:0/4294159865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56421310cd00 msgr2=0x5642132147f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:54:41.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.203+0000 7f3d1d5b5b80 1 --2- 172.21.15.5:0/4294159865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56421310cd00 0x5642132147f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3cf4000c00 tx=0x7f3cf4009d60 comp rx=0 tx=0).stop 2024-05-09T00:54:41.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.211+0000 7f3d1d5b5b80 1 -- 172.21.15.5:0/4294159865 shutdown_connections 2024-05-09T00:54:41.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.211+0000 7f3d1d5b5b80 1 --2- 172.21.15.5:0/4294159865 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56421320f400 0x564213203ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:41.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.211+0000 7f3d1d5b5b80 1 --2- 172.21.15.5:0/4294159865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642130924a0 0x564213217ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:41.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.211+0000 7f3d1d5b5b80 1 --2- 172.21.15.5:0/4294159865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56421310cd00 0x5642132147f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:41.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.211+0000 7f3d1d5b5b80 1 -- 172.21.15.5:0/4294159865 >> 172.21.15.5:0/4294159865 conn(0x564213095f70 msgr2=0x5642132140e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:54:41.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.211+0000 7f3d1d5b5b80 1 -- 172.21.15.5:0/4294159865 shutdown_connections 2024-05-09T00:54:41.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.215+0000 7f3d1d5b5b80 1 -- 172.21.15.5:0/4294159865 wait complete. 2024-05-09T00:54:41.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.215+0000 7f3d1d5b5b80 1 -- start start 2024-05-09T00:54:41.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.215+0000 7f3d1d5b5b80 1 Processor -- start 2024-05-09T00:54:41.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.217+0000 7f3d1d5b5b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642130924a0 0x564213238b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:41.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.217+0000 7f3d1d5b5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56421310cd00 0x5642132390f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:41.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.217+0000 7f3d1d5b5b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56421320f400 0x5642132254b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:41.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.217+0000 7f3d1d5b5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3d081adf70 con 0x56421310cd00 2024-05-09T00:54:41.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.217+0000 7f3d1d5b5b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3d081ad8a0 con 0x5642130924a0 2024-05-09T00:54:41.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.217+0000 7f3d1d5b5b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3d081aaae0 con 0x56421320f400 2024-05-09T00:54:41.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.217+0000 7f3d1d5b5b80 10 client.0 _ll_register_callbacks cb 0x564212fa36c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:54:41.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.220+0000 7f3d0d13c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56421320f400 0x5642132254b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:41.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.220+0000 7f3d0c93b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642130924a0 0x564213238b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:41.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.222+0000 7f3d0c93b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642130924a0 0x564213238b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:42134/0 (socket says 172.21.15.5:42134) 2024-05-09T00:54:41.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.222+0000 7f3d0c93b700 1 -- 172.21.15.5:0/655976392 learned_addr learned my addr 172.21.15.5:0/655976392 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:54:41.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.226+0000 7f3d0c93b700 1 -- 172.21.15.5:0/655976392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56421320f400 msgr2=0x5642132254b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:54:41.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.226+0000 7f3d0c93b700 1 --2- 172.21.15.5:0/655976392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56421320f400 0x5642132254b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:41.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.226+0000 7f3d0c93b700 1 -- 172.21.15.5:0/655976392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56421310cd00 msgr2=0x5642132390f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:54:41.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.226+0000 7f3d0c93b700 1 --2- 172.21.15.5:0/655976392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56421310cd00 0x5642132390f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:41.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.226+0000 7f3d0c93b700 1 -- 172.21.15.5:0/655976392 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564213225af0 con 0x5642130924a0 2024-05-09T00:54:41.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.233+0000 7f3d0c93b700 1 --2- 172.21.15.5:0/655976392 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642130924a0 0x564213238b90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc029a20 tx=0x7f3cfc029640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41e7a07494114960 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:54:41.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.233+0000 7f3ce7fff700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-09T00:54:41.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.234+0000 7f3ce7fff700 1 -- 172.21.15.5:0/655976392 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc004020 con 0x5642130924a0 2024-05-09T00:54:41.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.235+0000 7f3d1d5b5b80 1 -- 172.21.15.5:0/655976392 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cf401d070 con 0x5642130924a0 2024-05-09T00:54:41.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.235+0000 7f3ce7fff700 1 -- 172.21.15.5:0/655976392 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f3cfc0041c0 con 0x5642130924a0 2024-05-09T00:54:41.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.236+0000 7f3ce7fff700 1 -- 172.21.15.5:0/655976392 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3cfc037030 con 0x5642130924a0 2024-05-09T00:54:41.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.236+0000 7f3d1d5b5b80 10 client.25847 fetch_fsmap learned FSMap version 31 2024-05-09T00:54:41.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.236+0000 7f3d1d5b5b80 1 -- 172.21.15.5:0/655976392 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56421322dfb0 con 0x5642130924a0 2024-05-09T00:54:41.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.238+0000 7f3ce7fff700 1 -- 172.21.15.5:0/655976392 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cf401d070 con 0x5642130924a0 2024-05-09T00:54:41.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.239+0000 7f3ce7fff700 1 -- 172.21.15.5:0/655976392 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f3cfc030030 con 0x5642130924a0 2024-05-09T00:54:41.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.239+0000 7f3d1d5b5b80 10 client.25847 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:54:41.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.239+0000 7f3d1d5b5b80 10 client.25847 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T00:54:41.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.239+0000 7f3d1d5b5b80 20 client.25847 populate_metadata read hostname 'smithi005' 2024-05-09T00:54:41.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.239+0000 7f3d1d5b5b80 1 --2- 172.21.15.5:0/655976392 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x564213231740 0x56421322e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:41.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.239+0000 7f3d1d5b5b80 4 client.25847 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:54:41.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.239+0000 7f3d1d5b5b80 1 -- 172.21.15.5:0/655976392 --> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564213139b20 con 0x564213231740 2024-05-09T00:54:41.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.242+0000 7f3d07fff700 1 --2- 172.21.15.5:0/655976392 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x564213231740 0x56421322e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:41.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.245+0000 7f3d07fff700 1 --2- 172.21.15.5:0/655976392 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x564213231740 0x56421322e720 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=44f6229de2de291 server_cookie=926e02709f0ea7c3 in_seq=0 out_seq=0 2024-05-09T00:54:41.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.246+0000 7f3ce7fff700 10 client.25847 ms_handle_connect on v2:172.21.15.131:6836/1633006186 2024-05-09T00:54:41.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.247+0000 7f3ce7fff700 1 -- 172.21.15.5:0/655976392 <== mds.4 v2:172.21.15.131:6836/1633006186 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564213139b20 con 0x564213231740 2024-05-09T00:54:41.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.247+0000 7f3ce7fff700 10 client.25847 handle_command_reply: tid=0 2024-05-09T00:54:41.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.247+0000 7f3d1d5b5b80 1 client.25847 shutdown 2024-05-09T00:54:41.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.249+0000 7f3d1d5b5b80 1 -- 172.21.15.5:0/655976392 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642130924a0 msgr2=0x564213238b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:54:41.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.249+0000 7f3d1d5b5b80 1 --2- 172.21.15.5:0/655976392 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642130924a0 0x564213238b90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc029a20 tx=0x7f3cfc029640 comp rx=0 tx=0).stop 2024-05-09T00:54:41.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.250+0000 7f3d1d5b5b80 1 -- 172.21.15.5:0/655976392 shutdown_connections 2024-05-09T00:54:41.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.250+0000 7f3d1d5b5b80 1 --2- 172.21.15.5:0/655976392 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x564213231740 0x56421322e720 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:41.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.250+0000 7f3d1d5b5b80 1 --2- 172.21.15.5:0/655976392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56421320f400 0x5642132254b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:41.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.250+0000 7f3d1d5b5b80 1 --2- 172.21.15.5:0/655976392 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5642130924a0 0x564213238b90 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:41.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.250+0000 7f3d1d5b5b80 1 --2- 172.21.15.5:0/655976392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56421310cd00 0x5642132390f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:41.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.250+0000 7f3d1d5b5b80 1 -- 172.21.15.5:0/655976392 >> 172.21.15.5:0/655976392 conn(0x564213095f70 msgr2=0x564213203ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:54:41.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.250+0000 7f3ce7fff700 0 client.25847 ms_handle_reset on v2:172.21.15.131:6836/1633006186 2024-05-09T00:54:41.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.255+0000 7f3d1d5b5b80 1 -- 172.21.15.5:0/655976392 shutdown_connections 2024-05-09T00:54:41.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.257+0000 7f3d1d5b5b80 1 -- 172.21.15.5:0/655976392 wait complete. 2024-05-09T00:54:41.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.257+0000 7f3d1d5b5b80 20 client.25847 trim_cache size 0 max 16384 2024-05-09T00:54:41.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.264+0000 7f3d0eb9f700 1 -- 172.21.15.5:0/1493823221 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f3cf80b2940 msgr2=0x7f3cf80b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:54:41.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.264+0000 7f3d0eb9f700 1 --2- 172.21.15.5:0/1493823221 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f3cf80b2940 0x7f3cf80b4e50 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc009770 tx=0x7f3cfc006cd0 comp rx=0 tx=0).stop 2024-05-09T00:54:41.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.264+0000 7f3d0eb9f700 1 -- 172.21.15.5:0/1493823221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d08108760 msgr2=0x7f3d0819c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:54:41.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.264+0000 7f3d0eb9f700 1 --2- 172.21.15.5:0/1493823221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d08108760 0x7f3d0819c910 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f3d00009bb0 tx=0x7f3d00007ae0 comp rx=0 tx=0).stop 2024-05-09T00:54:41.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.265+0000 7f3d0eb9f700 1 -- 172.21.15.5:0/1493823221 shutdown_connections 2024-05-09T00:54:41.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.265+0000 7f3d0eb9f700 1 --2- 172.21.15.5:0/1493823221 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f3cf80b2940 0x7f3cf80b4e50 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:41.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.265+0000 7f3d0eb9f700 1 --2- 172.21.15.5:0/1493823221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d08108760 0x7f3d0819c910 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:41.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.265+0000 7f3d0eb9f700 1 --2- 172.21.15.5:0/1493823221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d081034e0 0x7f3d0819be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:41.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.265+0000 7f3d0eb9f700 1 --2- 172.21.15.5:0/1493823221 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d08107e10 0x7f3d0819c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:41.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.265+0000 7f3d0eb9f700 1 -- 172.21.15.5:0/1493823221 >> 172.21.15.5:0/1493823221 conn(0x7f3d080fef30 msgr2=0x7f3d081096c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:54:41.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.265+0000 7f3d0eb9f700 1 -- 172.21.15.5:0/1493823221 shutdown_connections 2024-05-09T00:54:41.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:41.265+0000 7f3d0eb9f700 1 -- 172.21.15.5:0/1493823221 wait complete. 2024-05-09T00:54:41.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:54:41 smithi157 ceph-mon[34294]: pgmap v621: 97 pgs: 97 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 998 MiB/s wr, 2.36k op/s 2024-05-09T00:54:42.040 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:54:41 smithi005 ceph-mon[30365]: pgmap v621: 97 pgs: 97 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 998 MiB/s wr, 2.36k op/s 2024-05-09T00:54:42.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:54:41 smithi131 ceph-mon[34547]: pgmap v621: 97 pgs: 97 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 998 MiB/s wr, 2.36k op/s 2024-05-09T00:54:43.966 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:54:43 smithi005 ceph-mon[30365]: pgmap v622: 97 pgs: 97 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 830 MiB/s wr, 1.92k op/s 2024-05-09T00:54:43.970 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-09T00:54:44.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:54:43 smithi131 ceph-mon[34547]: pgmap v622: 97 pgs: 97 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 830 MiB/s wr, 1.92k op/s 2024-05-09T00:54:44.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:54:43 smithi157 ceph-mon[34294]: pgmap v622: 97 pgs: 97 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 830 MiB/s wr, 1.92k op/s 2024-05-09T00:54:44.754 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:54:45.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:54:45 smithi005 ceph-mon[30365]: pgmap v623: 97 pgs: 97 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 928 MiB/s wr, 1.93k op/s 2024-05-09T00:54:46.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.068+0000 7fe86b6b8640 1 -- 172.21.15.5:0/2626013310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe86410b480 msgr2=0x7fe86410b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:54:46.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.068+0000 7fe86b6b8640 1 --2- 172.21.15.5:0/2626013310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe86410b480 0x7fe86410b890 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fe854009860 tx=0x7fe85402a360 comp rx=0 tx=0).stop 2024-05-09T00:54:46.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.087+0000 7fe86b6b8640 1 -- 172.21.15.5:0/2626013310 shutdown_connections 2024-05-09T00:54:46.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.087+0000 7fe86b6b8640 1 --2- 172.21.15.5:0/2626013310 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe86410c180 0x7fe86410c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:46.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.087+0000 7fe86b6b8640 1 --2- 172.21.15.5:0/2626013310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe86410b480 0x7fe86410b890 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:46.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.087+0000 7fe86b6b8640 1 --2- 172.21.15.5:0/2626013310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe86410a550 0x7fe86410af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:46.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.087+0000 7fe86b6b8640 1 -- 172.21.15.5:0/2626013310 >> 172.21.15.5:0/2626013310 conn(0x7fe864070670 msgr2=0x7fe864072b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:54:46.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.093+0000 7fe86b6b8640 1 -- 172.21.15.5:0/2626013310 shutdown_connections 2024-05-09T00:54:46.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.093+0000 7fe86b6b8640 1 -- 172.21.15.5:0/2626013310 wait complete. 2024-05-09T00:54:46.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:54:45 smithi131 ceph-mon[34547]: pgmap v623: 97 pgs: 97 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 928 MiB/s wr, 1.93k op/s 2024-05-09T00:54:46.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.107+0000 7fe86b6b8640 1 Processor -- start 2024-05-09T00:54:46.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.111+0000 7fe86b6b8640 1 -- start start 2024-05-09T00:54:46.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.112+0000 7fe86b6b8640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe86410a550 0x7fe8641a4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:46.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.112+0000 7fe86b6b8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe86410b480 0x7fe8641a5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:46.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.112+0000 7fe86b6b8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe86410c180 0x7fe8641ac3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:46.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.113+0000 7fe86b6b8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe86411a2b0 con 0x7fe86410c180 2024-05-09T00:54:46.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.113+0000 7fe86b6b8640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe86411a130 con 0x7fe86410a550 2024-05-09T00:54:46.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.113+0000 7fe869454640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe86410a550 0x7fe8641a4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:46.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.113+0000 7fe869454640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe86410a550 0x7fe8641a4ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:40176/0 (socket says 172.21.15.5:40176) 2024-05-09T00:54:46.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.113+0000 7fe869454640 1 -- 172.21.15.5:0/4132144451 learned_addr learned my addr 172.21.15.5:0/4132144451 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:54:46.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.113+0000 7fe86b6b8640 1 -- 172.21.15.5:0/4132144451 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe86411a430 con 0x7fe86410b480 2024-05-09T00:54:46.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.114+0000 7fe868c53640 1 --2- 172.21.15.5:0/4132144451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe86410b480 0x7fe8641a5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:46.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.115+0000 7fe869454640 1 -- 172.21.15.5:0/4132144451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe86410b480 msgr2=0x7fe8641a5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:54:46.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.115+0000 7fe869454640 1 --2- 172.21.15.5:0/4132144451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe86410b480 0x7fe8641a5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:46.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.115+0000 7fe869454640 1 -- 172.21.15.5:0/4132144451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe86410c180 msgr2=0x7fe8641ac3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:54:46.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.115+0000 7fe869454640 1 --2- 172.21.15.5:0/4132144451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe86410c180 0x7fe8641ac3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:46.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.115+0000 7fe869454640 1 -- 172.21.15.5:0/4132144451 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8641aca30 con 0x7fe86410a550 2024-05-09T00:54:46.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.117+0000 7fe869454640 1 --2- 172.21.15.5:0/4132144451 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe86410a550 0x7fe8641a4ef0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe85c00b9e0 tx=0x7fe85c00bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:54:46.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.119+0000 7fe85a7fc640 1 -- 172.21.15.5:0/4132144451 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe85c019070 con 0x7fe86410a550 2024-05-09T00:54:46.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.119+0000 7fe86b6b8640 1 -- 172.21.15.5:0/4132144451 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8641acd20 con 0x7fe86410a550 2024-05-09T00:54:46.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.119+0000 7fe85a7fc640 1 -- 172.21.15.5:0/4132144451 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe85c00cba0 con 0x7fe86410a550 2024-05-09T00:54:46.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.119+0000 7fe86b6b8640 1 -- 172.21.15.5:0/4132144451 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8641ad1c0 con 0x7fe86410a550 2024-05-09T00:54:46.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.120+0000 7fe85a7fc640 1 -- 172.21.15.5:0/4132144451 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe85c0125a0 con 0x7fe86410a550 2024-05-09T00:54:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.134+0000 7fe85a7fc640 1 -- 172.21.15.5:0/4132144451 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fe85c01c070 con 0x7fe86410a550 2024-05-09T00:54:46.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.137+0000 7fe87256c740 1 Processor -- start 2024-05-09T00:54:46.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.137+0000 7fe87256c740 1 -- start start 2024-05-09T00:54:46.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.138+0000 7fe87256c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccd6e8f1b0 0x55ccd7233500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:46.140 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.138+0000 7fe87256c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ccd6e662f0 0x55ccd7233a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:46.140 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.139+0000 7fe87256c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ccd6e6e4d0 0x55ccd723a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:46.140 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.139+0000 7fe87256c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ccd71977c0 con 0x55ccd6e8f1b0 2024-05-09T00:54:46.140 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.139+0000 7fe87256c740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55ccd71330e0 con 0x55ccd6e6e4d0 2024-05-09T00:54:46.140 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.139+0000 7fe87256c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ccd7193780 con 0x55ccd6e662f0 2024-05-09T00:54:46.140 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.139+0000 7fe85a7fc640 1 --2- 172.21.15.5:0/4132144451 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe8500b2970 0x7fe8500b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:46.141 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.140+0000 7fe85a7fc640 1 -- 172.21.15.5:0/4132144451 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fe85c0144a0 con 0x7fe86410a550 2024-05-09T00:54:46.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.141+0000 7fe868c53640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ccd6e6e4d0 0x55ccd723a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:46.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.141+0000 7fe868c53640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ccd6e6e4d0 0x55ccd723a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:40178/0 (socket says 172.21.15.5:40178) 2024-05-09T00:54:46.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.141+0000 7fe868c53640 1 -- 172.21.15.5:0/1481005954 learned_addr learned my addr 172.21.15.5:0/1481005954 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:54:46.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.141+0000 7fe869454640 1 --2- 172.21.15.5:0/1481005954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ccd6e662f0 0x55ccd7233a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:46.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.142+0000 7fe868c53640 1 -- 172.21.15.5:0/1481005954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ccd6e662f0 msgr2=0x55ccd7233a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:54:46.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.142+0000 7fe868c53640 1 --2- 172.21.15.5:0/1481005954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ccd6e662f0 0x55ccd7233a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:46.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.142+0000 7fe868c53640 1 -- 172.21.15.5:0/1481005954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccd6e8f1b0 msgr2=0x55ccd7233500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:54:46.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.142+0000 7fe868c53640 1 --2- 172.21.15.5:0/1481005954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccd6e8f1b0 0x55ccd7233500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:46.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.142+0000 7fe868c53640 1 -- 172.21.15.5:0/1481005954 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccd715c7c0 con 0x55ccd6e6e4d0 2024-05-09T00:54:46.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.149+0000 7fe868c53640 1 --2- 172.21.15.5:0/1481005954 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ccd6e6e4d0 0x55ccd723a980 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe854009780 tx=0x7fe8540095b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62eae84bccc77608 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:54:46.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.150+0000 7fe869c55640 1 --2- 172.21.15.5:0/4132144451 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe8500b2970 0x7fe8500b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:46.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.150+0000 7fe84f7fe640 1 -- 172.21.15.5:0/1481005954 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe854040020 con 0x55ccd6e6e4d0 2024-05-09T00:54:46.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.151+0000 7fe84f7fe640 1 -- 172.21.15.5:0/1481005954 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe854030bc0 con 0x55ccd6e6e4d0 2024-05-09T00:54:46.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.152+0000 7fe87256c740 1 -- 172.21.15.5:0/1481005954 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ccd6e6e4d0 msgr2=0x55ccd723a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:54:46.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.152+0000 7fe87256c740 1 --2- 172.21.15.5:0/1481005954 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ccd6e6e4d0 0x55ccd723a980 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe854009780 tx=0x7fe8540095b0 comp rx=0 tx=0).stop 2024-05-09T00:54:46.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.154+0000 7fe87256c740 1 -- 172.21.15.5:0/1481005954 shutdown_connections 2024-05-09T00:54:46.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.154+0000 7fe87256c740 1 --2- 172.21.15.5:0/1481005954 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ccd6e6e4d0 0x55ccd723a980 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:46.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.154+0000 7fe87256c740 1 --2- 172.21.15.5:0/1481005954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ccd6e662f0 0x55ccd7233a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:46.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.154+0000 7fe87256c740 1 --2- 172.21.15.5:0/1481005954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccd6e8f1b0 0x55ccd7233500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:46.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.154+0000 7fe87256c740 1 -- 172.21.15.5:0/1481005954 >> 172.21.15.5:0/1481005954 conn(0x55ccd6e83300 msgr2=0x55ccd722ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:54:46.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.159+0000 7fe87256c740 1 -- 172.21.15.5:0/1481005954 shutdown_connections 2024-05-09T00:54:46.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.159+0000 7fe869c55640 1 --2- 172.21.15.5:0/4132144451 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe8500b2970 0x7fe8500b4e80 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe860009510 tx=0x7fe860009480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:54:46.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.160+0000 7fe87256c740 1 -- 172.21.15.5:0/1481005954 wait complete. 2024-05-09T00:54:46.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.161+0000 7fe87256c740 1 -- start start 2024-05-09T00:54:46.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.161+0000 7fe87256c740 1 Processor -- start 2024-05-09T00:54:46.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.161+0000 7fe87256c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ccd6e662f0 0x55ccd7245bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:46.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.161+0000 7fe87256c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccd6e6e4d0 0x55ccd7246130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:46.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.161+0000 7fe87256c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ccd6e8f1b0 0x55ccd724d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:46.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.161+0000 7fe87256c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ccd723b410 con 0x55ccd6e6e4d0 2024-05-09T00:54:46.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.161+0000 7fe87256c740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55ccd711e9c0 con 0x55ccd6e8f1b0 2024-05-09T00:54:46.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.162+0000 7fe87256c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ccd723b590 con 0x55ccd6e662f0 2024-05-09T00:54:46.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.162+0000 7fe87256c740 10 client.0 _ll_register_callbacks cb 0x55ccd714b020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:54:46.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.162+0000 7fe868c53640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccd6e6e4d0 0x55ccd7246130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:46.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.163+0000 7fe868c53640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccd6e6e4d0 0x55ccd7246130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33372/0 (socket says 172.21.15.5:33372) 2024-05-09T00:54:46.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.163+0000 7fe868c53640 1 -- 172.21.15.5:0/1001711043 learned_addr learned my addr 172.21.15.5:0/1001711043 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:54:46.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.163+0000 7fe869c55640 1 --2- 172.21.15.5:0/1001711043 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ccd6e8f1b0 0x55ccd724d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:46.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.164+0000 7fe869454640 1 --2- 172.21.15.5:0/1001711043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ccd6e662f0 0x55ccd7245bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:46.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.164+0000 7fe868c53640 1 -- 172.21.15.5:0/1001711043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ccd6e662f0 msgr2=0x55ccd7245bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:54:46.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.164+0000 7fe868c53640 1 --2- 172.21.15.5:0/1001711043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ccd6e662f0 0x55ccd7245bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:46.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.164+0000 7fe868c53640 1 -- 172.21.15.5:0/1001711043 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ccd6e8f1b0 msgr2=0x55ccd724d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:54:46.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.164+0000 7fe868c53640 1 --2- 172.21.15.5:0/1001711043 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ccd6e8f1b0 0x55ccd724d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:46.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.164+0000 7fe868c53640 1 -- 172.21.15.5:0/1001711043 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccd71451f0 con 0x55ccd6e6e4d0 2024-05-09T00:54:46.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.165+0000 7fe868c53640 1 --2- 172.21.15.5:0/1001711043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccd6e6e4d0 0x55ccd7246130 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fe854030520 tx=0x7fe854005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac2a0f704a118429 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:54:46.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.166+0000 7fe84cff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:54:46.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.170+0000 7fe84cff9640 1 -- 172.21.15.5:0/1001711043 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe854040070 con 0x55ccd6e6e4d0 2024-05-09T00:54:46.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.170+0000 7fe87256c740 1 -- 172.21.15.5:0/1001711043 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ccd71a53d0 con 0x55ccd6e6e4d0 2024-05-09T00:54:46.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.170+0000 7fe84cff9640 1 -- 172.21.15.5:0/1001711043 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fe854005240 con 0x55ccd6e6e4d0 2024-05-09T00:54:46.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.171+0000 7fe84cff9640 1 -- 172.21.15.5:0/1001711043 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe85402ce80 con 0x55ccd6e6e4d0 2024-05-09T00:54:46.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.174+0000 7fe87256c740 10 client.16341 fetch_fsmap learned FSMap version 31 2024-05-09T00:54:46.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.174+0000 7fe87256c740 1 -- 172.21.15.5:0/1001711043 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55ccd6efa790 con 0x55ccd6e6e4d0 2024-05-09T00:54:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.180+0000 7fe84cff9640 1 -- 172.21.15.5:0/1001711043 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55ccd71a53d0 con 0x55ccd6e6e4d0 2024-05-09T00:54:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.180+0000 7fe84cff9640 1 -- 172.21.15.5:0/1001711043 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fe854004950 con 0x55ccd6e6e4d0 2024-05-09T00:54:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.181+0000 7fe87256c740 10 client.16341 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:54:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.181+0000 7fe87256c740 10 client.16341 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:54:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.181+0000 7fe87256c740 20 client.16341 populate_metadata read hostname 'smithi005' 2024-05-09T00:54:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.181+0000 7fe87256c740 1 --2- 172.21.15.5:0/1001711043 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55ccd6e4dd80 0x55ccd7241bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.181+0000 7fe87256c740 4 client.16341 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:54:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.181+0000 7fe87256c740 1 -- 172.21.15.5:0/1001711043 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ccd719f240 con 0x55ccd6e4dd80 2024-05-09T00:54:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.182+0000 7fe869454640 1 --2- 172.21.15.5:0/1001711043 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55ccd6e4dd80 0x55ccd7241bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:46.185 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:54:45 smithi157 ceph-mon[34294]: pgmap v623: 97 pgs: 97 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 928 MiB/s wr, 1.93k op/s 2024-05-09T00:54:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.189+0000 7fe869454640 1 --2- 172.21.15.5:0/1001711043 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55ccd6e4dd80 0x55ccd7241bc0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=902fafea14d13f6 server_cookie=983ec57c21363333 in_seq=0 out_seq=0 2024-05-09T00:54:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.189+0000 7fe84cff9640 10 client.16341 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:54:46.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.192+0000 7fe84cff9640 1 -- 172.21.15.5:0/1001711043 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ccd719f240 con 0x55ccd6e4dd80 2024-05-09T00:54:46.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.192+0000 7fe84cff9640 10 client.16341 handle_command_reply: tid=0 2024-05-09T00:54:46.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.192+0000 7fe87256c740 1 client.16341 shutdown 2024-05-09T00:54:46.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.195+0000 7fe87256c740 1 -- 172.21.15.5:0/1001711043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccd6e6e4d0 msgr2=0x55ccd7246130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:54:46.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.195+0000 7fe87256c740 1 --2- 172.21.15.5:0/1001711043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccd6e6e4d0 0x55ccd7246130 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fe854030520 tx=0x7fe854005f30 comp rx=0 tx=0).stop 2024-05-09T00:54:46.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.198+0000 7fe87256c740 1 -- 172.21.15.5:0/1001711043 shutdown_connections 2024-05-09T00:54:46.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.198+0000 7fe87256c740 1 --2- 172.21.15.5:0/1001711043 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55ccd6e4dd80 0x55ccd7241bc0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:46.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.198+0000 7fe87256c740 1 --2- 172.21.15.5:0/1001711043 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ccd6e8f1b0 0x55ccd724d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:46.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.198+0000 7fe87256c740 1 --2- 172.21.15.5:0/1001711043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccd6e6e4d0 0x55ccd7246130 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:46.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.198+0000 7fe87256c740 1 --2- 172.21.15.5:0/1001711043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ccd6e662f0 0x55ccd7245bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:46.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.199+0000 7fe87256c740 1 -- 172.21.15.5:0/1001711043 >> 172.21.15.5:0/1001711043 conn(0x55ccd6e83300 msgr2=0x55ccd7231e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:54:46.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.199+0000 7fe84cff9640 0 client.16341 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:54:46.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.199+0000 7fe87256c740 1 -- 172.21.15.5:0/1001711043 shutdown_connections 2024-05-09T00:54:46.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.200+0000 7fe87256c740 1 -- 172.21.15.5:0/1001711043 wait complete. 2024-05-09T00:54:46.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.204+0000 7fe87256c740 20 client.16341 trim_cache size 0 max 16384 2024-05-09T00:54:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.274+0000 7fe87256c740 1 Processor -- start 2024-05-09T00:54:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.280+0000 7fe87256c740 1 -- start start 2024-05-09T00:54:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.281+0000 7fe87256c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ccd6e662f0 0x55ccd72335a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.281+0000 7fe87256c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccd6e6e4d0 0x55ccd7236710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.281+0000 7fe87256c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ccd6e8f1b0 0x55ccd7236c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.281+0000 7fe87256c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe8641b1be0 con 0x55ccd6e6e4d0 2024-05-09T00:54:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.281+0000 7fe87256c740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe8641b3720 con 0x55ccd6e662f0 2024-05-09T00:54:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.281+0000 7fe87256c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe8641b22b0 con 0x55ccd6e8f1b0 2024-05-09T00:54:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.281+0000 7fe868c53640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccd6e6e4d0 0x55ccd7236710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.282+0000 7fe868c53640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccd6e6e4d0 0x55ccd7236710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33378/0 (socket says 172.21.15.5:33378) 2024-05-09T00:54:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.282+0000 7fe868c53640 1 -- 172.21.15.5:0/297208663 learned_addr learned my addr 172.21.15.5:0/297208663 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:54:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.283+0000 7fe868c53640 1 -- 172.21.15.5:0/297208663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ccd6e8f1b0 msgr2=0x55ccd7236c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:54:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.283+0000 7fe868c53640 1 --2- 172.21.15.5:0/297208663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ccd6e8f1b0 0x55ccd7236c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.283+0000 7fe868c53640 1 -- 172.21.15.5:0/297208663 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ccd6e662f0 msgr2=0x55ccd72335a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:54:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.283+0000 7fe868c53640 1 --2- 172.21.15.5:0/297208663 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ccd6e662f0 0x55ccd72335a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.283+0000 7fe868c53640 1 -- 172.21.15.5:0/297208663 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccd6f715f0 con 0x55ccd6e6e4d0 2024-05-09T00:54:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.288+0000 7fe868c53640 1 --2- 172.21.15.5:0/297208663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccd6e6e4d0 0x55ccd7236710 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fe854004da0 tx=0x7fe854030420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=846f463ab91c8fcc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:54:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.289+0000 7fe84effd640 1 -- 172.21.15.5:0/297208663 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe854040040 con 0x55ccd6e6e4d0 2024-05-09T00:54:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.289+0000 7fe84effd640 1 -- 172.21.15.5:0/297208663 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8540307d0 con 0x55ccd6e6e4d0 2024-05-09T00:54:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.289+0000 7fe84effd640 1 -- 172.21.15.5:0/297208663 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8540326a0 con 0x55ccd6e6e4d0 2024-05-09T00:54:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.291+0000 7fe87256c740 1 -- 172.21.15.5:0/297208663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccd6e6e4d0 msgr2=0x55ccd7236710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:54:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.291+0000 7fe87256c740 1 --2- 172.21.15.5:0/297208663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccd6e6e4d0 0x55ccd7236710 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fe854004da0 tx=0x7fe854030420 comp rx=0 tx=0).stop 2024-05-09T00:54:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.291+0000 7fe87256c740 1 -- 172.21.15.5:0/297208663 shutdown_connections 2024-05-09T00:54:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.292+0000 7fe87256c740 1 --2- 172.21.15.5:0/297208663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ccd6e8f1b0 0x55ccd7236c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.292+0000 7fe87256c740 1 --2- 172.21.15.5:0/297208663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccd6e6e4d0 0x55ccd7236710 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.292+0000 7fe87256c740 1 --2- 172.21.15.5:0/297208663 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ccd6e662f0 0x55ccd72335a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.292+0000 7fe87256c740 1 -- 172.21.15.5:0/297208663 >> 172.21.15.5:0/297208663 conn(0x55ccd6e83300 msgr2=0x55ccd72377b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:54:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.292+0000 7fe87256c740 1 -- 172.21.15.5:0/297208663 shutdown_connections 2024-05-09T00:54:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.292+0000 7fe87256c740 1 -- 172.21.15.5:0/297208663 wait complete. 2024-05-09T00:54:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.294+0000 7fe87256c740 1 -- start start 2024-05-09T00:54:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.295+0000 7fe87256c740 1 Processor -- start 2024-05-09T00:54:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.296+0000 7fe87256c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ccd6e662f0 0x55ccd724c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.297+0000 7fe87256c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ccd6e6e4d0 0x55ccd7249010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.297+0000 7fe87256c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccd6e8f1b0 0x55ccd7249550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.298+0000 7fe87256c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe8641b3050 con 0x55ccd6e8f1b0 2024-05-09T00:54:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.298+0000 7fe87256c740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe8641b2980 con 0x55ccd6e6e4d0 2024-05-09T00:54:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.298+0000 7fe87256c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe8641afbc0 con 0x55ccd6e662f0 2024-05-09T00:54:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.298+0000 7fe87256c740 10 client.0 _ll_register_callbacks cb 0x7fe85c0a3920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:54:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.298+0000 7fe868c53640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ccd6e6e4d0 0x55ccd7249010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.298+0000 7fe868c53640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ccd6e6e4d0 0x55ccd7249010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:40210/0 (socket says 172.21.15.5:40210) 2024-05-09T00:54:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.299+0000 7fe868c53640 1 -- 172.21.15.5:0/1642443073 learned_addr learned my addr 172.21.15.5:0/1642443073 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:54:46.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.299+0000 7fe869454640 1 --2- 172.21.15.5:0/1642443073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ccd6e662f0 0x55ccd724c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:46.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.300+0000 7fe869c55640 1 --2- 172.21.15.5:0/1642443073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccd6e8f1b0 0x55ccd7249550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:46.305 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.303+0000 7fe869454640 1 -- 172.21.15.5:0/1642443073 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ccd6e6e4d0 msgr2=0x55ccd7249010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:54:46.305 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.303+0000 7fe869454640 1 --2- 172.21.15.5:0/1642443073 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ccd6e6e4d0 0x55ccd7249010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:46.305 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.303+0000 7fe869454640 1 -- 172.21.15.5:0/1642443073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccd6e8f1b0 msgr2=0x55ccd7249550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:54:46.305 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.303+0000 7fe869454640 1 --2- 172.21.15.5:0/1642443073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccd6e8f1b0 0x55ccd7249550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:46.305 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.303+0000 7fe869454640 1 -- 172.21.15.5:0/1642443073 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccd715bed0 con 0x55ccd6e662f0 2024-05-09T00:54:46.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.308+0000 7fe869454640 1 --2- 172.21.15.5:0/1642443073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ccd6e662f0 0x55ccd724c3d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe85c0df7d0 tx=0x7fe85c009a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c045fa9d6380388 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:54:46.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.310+0000 7fe84cff9640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-09T00:54:46.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.314+0000 7fe84cff9640 1 -- 172.21.15.5:0/1642443073 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe85c0a5610 con 0x55ccd6e662f0 2024-05-09T00:54:46.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.314+0000 7fe87256c740 1 -- 172.21.15.5:0/1642443073 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8540307d0 con 0x55ccd6e662f0 2024-05-09T00:54:46.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.314+0000 7fe84cff9640 1 -- 172.21.15.5:0/1642443073 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fe85c0a85e0 con 0x55ccd6e662f0 2024-05-09T00:54:46.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.317+0000 7fe84cff9640 1 -- 172.21.15.5:0/1642443073 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe85c0a57b0 con 0x55ccd6e662f0 2024-05-09T00:54:46.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.317+0000 7fe87256c740 10 client.25882 fetch_fsmap learned FSMap version 31 2024-05-09T00:54:46.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.317+0000 7fe87256c740 1 -- 172.21.15.5:0/1642443073 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55ccd72c30e0 con 0x55ccd6e662f0 2024-05-09T00:54:46.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.318+0000 7fe84cff9640 1 -- 172.21.15.5:0/1642443073 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8540307d0 con 0x55ccd6e662f0 2024-05-09T00:54:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.321+0000 7fe84cff9640 1 -- 172.21.15.5:0/1642443073 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fe85c0dfba0 con 0x55ccd6e662f0 2024-05-09T00:54:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.321+0000 7fe87256c740 10 client.25882 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:54:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.321+0000 7fe87256c740 10 client.25882 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:54:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.321+0000 7fe87256c740 20 client.25882 populate_metadata read hostname 'smithi005' 2024-05-09T00:54:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.321+0000 7fe87256c740 1 --2- 172.21.15.5:0/1642443073 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55ccd6e231c0 0x55ccd72c5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.321+0000 7fe87256c740 4 client.25882 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:54:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.321+0000 7fe87256c740 1 -- 172.21.15.5:0/1642443073 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ccd711a350 con 0x55ccd6e231c0 2024-05-09T00:54:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.327+0000 7fe868c53640 1 --2- 172.21.15.5:0/1642443073 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55ccd6e231c0 0x55ccd72c5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.331+0000 7fe84cff9640 10 client.25882 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:54:46.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:46.331+0000 7fe868c53640 1 --2- 172.21.15.5:0/1642443073 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55ccd6e231c0 0x55ccd72c5ab0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=533d9c1e16b3d6a7 server_cookie=4dab4636187143e0 in_seq=0 out_seq=0 2024-05-09T00:54:48.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:54:47 smithi005 ceph-mon[30365]: pgmap v624: 97 pgs: 97 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 914 MiB/s wr, 1.81k op/s 2024-05-09T00:54:48.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:54:47 smithi131 ceph-mon[34547]: pgmap v624: 97 pgs: 97 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 914 MiB/s wr, 1.81k op/s 2024-05-09T00:54:48.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:54:47 smithi157 ceph-mon[34294]: pgmap v624: 97 pgs: 97 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 914 MiB/s wr, 1.81k op/s 2024-05-09T00:54:49.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:54:48 smithi005 ceph-mon[30365]: pgmap v625: 97 pgs: 97 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 868 MiB/s wr, 1.81k op/s 2024-05-09T00:54:49.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:54:48 smithi131 ceph-mon[34547]: pgmap v625: 97 pgs: 97 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 868 MiB/s wr, 1.81k op/s 2024-05-09T00:54:49.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:54:48 smithi157 ceph-mon[34294]: pgmap v625: 97 pgs: 97 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 868 MiB/s wr, 1.81k op/s 2024-05-09T00:54:50.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.750+0000 7ffb4c6da700 1 -- 172.21.15.5:0/1265348382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb441045d0 msgr2=0x7ffb441049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:54:50.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.750+0000 7ffb4c6da700 1 --2- 172.21.15.5:0/1265348382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb441045d0 0x7ffb441049e0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7ffb400099c0 tx=0x7ffb40009cd0 comp rx=0 tx=0).stop 2024-05-09T00:54:50.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.750+0000 7ffb4c6da700 1 -- 172.21.15.5:0/1265348382 shutdown_connections 2024-05-09T00:54:50.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.750+0000 7ffb4c6da700 1 --2- 172.21.15.5:0/1265348382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb44109850 0x7ffb441111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:50.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.750+0000 7ffb4c6da700 1 --2- 172.21.15.5:0/1265348382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb441045d0 0x7ffb441049e0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:50.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.750+0000 7ffb4c6da700 1 --2- 172.21.15.5:0/1265348382 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb44108f00 0x7ffb44109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:50.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.750+0000 7ffb4c6da700 1 -- 172.21.15.5:0/1265348382 >> 172.21.15.5:0/1265348382 conn(0x7ffb440ffe80 msgr2=0x7ffb441022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:54:50.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.751+0000 7ffb4c6da700 1 -- 172.21.15.5:0/1265348382 shutdown_connections 2024-05-09T00:54:50.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.751+0000 7ffb4c6da700 1 -- 172.21.15.5:0/1265348382 wait complete. 2024-05-09T00:54:50.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.752+0000 7ffb4c6da700 1 Processor -- start 2024-05-09T00:54:50.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.752+0000 7ffb4c6da700 1 -- start start 2024-05-09T00:54:50.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.753+0000 7ffb4c6da700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb441045d0 0x7ffb4419aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:50.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.753+0000 7ffb4c6da700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb44108f00 0x7ffb4419b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:50.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.753+0000 7ffb4c6da700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb44109850 0x7ffb4419f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:50.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.753+0000 7ffb4c6da700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffb44114570 con 0x7ffb44109850 2024-05-09T00:54:50.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.754+0000 7ffb4c6da700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ffb441143f0 con 0x7ffb44108f00 2024-05-09T00:54:50.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.754+0000 7ffb4c6da700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ffb441146f0 con 0x7ffb441045d0 2024-05-09T00:54:50.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.754+0000 7ffb4ac77700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb44109850 0x7ffb4419f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:50.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.754+0000 7ffb4ac77700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb44109850 0x7ffb4419f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33398/0 (socket says 172.21.15.5:33398) 2024-05-09T00:54:50.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.754+0000 7ffb4ac77700 1 -- 172.21.15.5:0/752019490 learned_addr learned my addr 172.21.15.5:0/752019490 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:54:50.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.754+0000 7ffb4ac77700 1 -- 172.21.15.5:0/752019490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb441045d0 msgr2=0x7ffb4419aec0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T00:54:50.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.754+0000 7ffb4ac77700 1 --2- 172.21.15.5:0/752019490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb441045d0 0x7ffb4419aec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:50.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.754+0000 7ffb4ac77700 1 -- 172.21.15.5:0/752019490 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb44108f00 msgr2=0x7ffb4419b460 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T00:54:50.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.754+0000 7ffb4ac77700 1 --2- 172.21.15.5:0/752019490 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb44108f00 0x7ffb4419b460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:50.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.754+0000 7ffb4ac77700 1 -- 172.21.15.5:0/752019490 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb4419fd90 con 0x7ffb44109850 2024-05-09T00:54:50.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.755+0000 7ffb4ac77700 1 --2- 172.21.15.5:0/752019490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb44109850 0x7ffb4419f6f0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c00b970 tx=0x7ffb3c00bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:54:50.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.755+0000 7ffb3b7fe700 1 -- 172.21.15.5:0/752019490 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c019070 con 0x7ffb44109850 2024-05-09T00:54:50.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.755+0000 7ffb3b7fe700 1 -- 172.21.15.5:0/752019490 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb3c014440 con 0x7ffb44109850 2024-05-09T00:54:50.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.755+0000 7ffb4c6da700 1 -- 172.21.15.5:0/752019490 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb441a0080 con 0x7ffb44109850 2024-05-09T00:54:50.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.756+0000 7ffb3b7fe700 1 -- 172.21.15.5:0/752019490 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c00ce60 con 0x7ffb44109850 2024-05-09T00:54:50.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.756+0000 7ffb4c6da700 1 -- 172.21.15.5:0/752019490 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb441a79a0 con 0x7ffb44109850 2024-05-09T00:54:50.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.763+0000 7ffb3b7fe700 1 -- 172.21.15.5:0/752019490 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7ffb3c01c070 con 0x7ffb44109850 2024-05-09T00:54:50.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.763+0000 7ffb5b0f0b80 1 Processor -- start 2024-05-09T00:54:50.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.765+0000 7ffb3b7fe700 1 --2- 172.21.15.5:0/752019490 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ffb300b2980 0x7ffb300b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:50.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.766+0000 7ffb49c75700 1 --2- 172.21.15.5:0/752019490 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ffb300b2980 0x7ffb300b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:50.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.766+0000 7ffb3b7fe700 1 -- 172.21.15.5:0/752019490 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7ffb3c0e1670 con 0x7ffb44109850 2024-05-09T00:54:50.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.766+0000 7ffb5b0f0b80 1 -- start start 2024-05-09T00:54:50.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.767+0000 7ffb5b0f0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f22cf1160 0x563f22d8a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:50.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.768+0000 7ffb5b0f0b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f22c074a0 0x563f22d8aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:50.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.768+0000 7ffb5b0f0b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f22d846a0 0x563f22d84ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:50.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.768+0000 7ffb5b0f0b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563f22ca6ff0 con 0x563f22d846a0 2024-05-09T00:54:50.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.768+0000 7ffb5b0f0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x563f22c457f0 con 0x563f22cf1160 2024-05-09T00:54:50.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.769+0000 7ffb4ac77700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f22d846a0 0x563f22d84ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:50.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.769+0000 7ffb49c75700 1 --2- 172.21.15.5:0/752019490 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ffb300b2980 0x7ffb300b4e90 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ffb44103420 tx=0x7ffb34006cb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:54:50.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.769+0000 7ffb4ac77700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f22d846a0 0x563f22d84ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33406/0 (socket says 172.21.15.5:33406) 2024-05-09T00:54:50.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.769+0000 7ffb4ac77700 1 -- 172.21.15.5:0/265764193 learned_addr learned my addr 172.21.15.5:0/265764193 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:54:50.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.769+0000 7ffb5b0f0b80 1 -- 172.21.15.5:0/265764193 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x563f22ca2fb0 con 0x563f22c074a0 2024-05-09T00:54:50.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.769+0000 7ffb49c75700 1 --2- 172.21.15.5:0/265764193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f22c074a0 0x563f22d8aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:50.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.769+0000 7ffb4a476700 1 --2- 172.21.15.5:0/265764193 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f22cf1160 0x563f22d8a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:50.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.770+0000 7ffb4ac77700 1 -- 172.21.15.5:0/265764193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f22c074a0 msgr2=0x563f22d8aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:54:50.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.770+0000 7ffb4ac77700 1 --2- 172.21.15.5:0/265764193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f22c074a0 0x563f22d8aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:50.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.770+0000 7ffb4ac77700 1 -- 172.21.15.5:0/265764193 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f22cf1160 msgr2=0x563f22d8a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:54:50.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.770+0000 7ffb4ac77700 1 --2- 172.21.15.5:0/265764193 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f22cf1160 0x563f22d8a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:50.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.770+0000 7ffb4ac77700 1 -- 172.21.15.5:0/265764193 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f22c3d370 con 0x563f22d846a0 2024-05-09T00:54:50.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.770+0000 7ffb4ac77700 1 --2- 172.21.15.5:0/265764193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f22d846a0 0x563f22d84ad0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c0a8b90 tx=0x7ffb3c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71525f5468998869 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:54:50.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.771+0000 7ffb38ff9700 1 -- 172.21.15.5:0/265764193 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c0a8c00 con 0x563f22d846a0 2024-05-09T00:54:50.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.771+0000 7ffb38ff9700 1 -- 172.21.15.5:0/265764193 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb3c0acfb0 con 0x563f22d846a0 2024-05-09T00:54:50.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.775+0000 7ffb5b0f0b80 1 -- 172.21.15.5:0/265764193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f22d846a0 msgr2=0x563f22d84ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:54:50.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.775+0000 7ffb5b0f0b80 1 --2- 172.21.15.5:0/265764193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f22d846a0 0x563f22d84ad0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c0a8b90 tx=0x7ffb3c004ad0 comp rx=0 tx=0).stop 2024-05-09T00:54:50.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.776+0000 7ffb5b0f0b80 1 -- 172.21.15.5:0/265764193 shutdown_connections 2024-05-09T00:54:50.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.776+0000 7ffb5b0f0b80 1 --2- 172.21.15.5:0/265764193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f22d846a0 0x563f22d84ad0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:50.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.776+0000 7ffb5b0f0b80 1 --2- 172.21.15.5:0/265764193 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f22cf1160 0x563f22d8a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:50.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.776+0000 7ffb5b0f0b80 1 --2- 172.21.15.5:0/265764193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f22c074a0 0x563f22d8aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:50.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.776+0000 7ffb5b0f0b80 1 -- 172.21.15.5:0/265764193 >> 172.21.15.5:0/265764193 conn(0x563f22c676a0 msgr2=0x563f22d7f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:54:50.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.778+0000 7ffb5b0f0b80 1 -- 172.21.15.5:0/265764193 shutdown_connections 2024-05-09T00:54:50.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.778+0000 7ffb5b0f0b80 1 -- 172.21.15.5:0/265764193 wait complete. 2024-05-09T00:54:50.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.779+0000 7ffb5b0f0b80 1 -- start start 2024-05-09T00:54:50.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.779+0000 7ffb5b0f0b80 1 Processor -- start 2024-05-09T00:54:50.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.780+0000 7ffb5b0f0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f22c074a0 0x563f22d8e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:50.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.781+0000 7ffb5b0f0b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f22cf1160 0x563f22d8e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:50.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.781+0000 7ffb5b0f0b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f22d846a0 0x563f22d8ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:50.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.781+0000 7ffb5b0f0b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563f22b22ba0 con 0x563f22cf1160 2024-05-09T00:54:50.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.781+0000 7ffb5b0f0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x563f22c61150 con 0x563f22c074a0 2024-05-09T00:54:50.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.781+0000 7ffb5b0f0b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x563f22d911d0 con 0x563f22d846a0 2024-05-09T00:54:50.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.781+0000 7ffb5b0f0b80 10 client.0 _ll_register_callbacks cb 0x563f22b186c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:54:50.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.783+0000 7ffb4a476700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f22c074a0 0x563f22d8e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:50.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.783+0000 7ffb4ac77700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f22d846a0 0x563f22d8ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:50.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.783+0000 7ffb4ac77700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f22d846a0 0x563f22d8ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:38306/0 (socket says 172.21.15.5:38306) 2024-05-09T00:54:50.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.783+0000 7ffb4ac77700 1 -- 172.21.15.5:0/265460712 learned_addr learned my addr 172.21.15.5:0/265460712 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:54:50.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.785+0000 7ffb49c75700 1 --2- 172.21.15.5:0/265460712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f22cf1160 0x563f22d8e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:50.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.787+0000 7ffb49c75700 1 -- 172.21.15.5:0/265460712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f22d846a0 msgr2=0x563f22d8ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:54:50.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.787+0000 7ffb49c75700 1 --2- 172.21.15.5:0/265460712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f22d846a0 0x563f22d8ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:50.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.787+0000 7ffb49c75700 1 -- 172.21.15.5:0/265460712 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f22c074a0 msgr2=0x563f22d8e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:54:50.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.787+0000 7ffb49c75700 1 --2- 172.21.15.5:0/265460712 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f22c074a0 0x563f22d8e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:50.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.787+0000 7ffb49c75700 1 -- 172.21.15.5:0/265460712 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f22b22490 con 0x563f22cf1160 2024-05-09T00:54:50.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.788+0000 7ffb49c75700 1 --2- 172.21.15.5:0/265460712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f22cf1160 0x563f22d8e6c0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x563f22d80ed0 tx=0x7ffb34004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=986443f5dbc4eef8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:54:50.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.788+0000 7ffb2dffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:54:50.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.790+0000 7ffb2dffb700 1 -- 172.21.15.5:0/265460712 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb3402d060 con 0x563f22cf1160 2024-05-09T00:54:50.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.791+0000 7ffb5b0f0b80 1 -- 172.21.15.5:0/265460712 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb3c0acfb0 con 0x563f22cf1160 2024-05-09T00:54:50.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.791+0000 7ffb2dffb700 1 -- 172.21.15.5:0/265460712 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7ffb340149c0 con 0x563f22cf1160 2024-05-09T00:54:50.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.791+0000 7ffb2dffb700 1 -- 172.21.15.5:0/265460712 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ffb34015e40 con 0x563f22cf1160 2024-05-09T00:54:50.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.791+0000 7ffb5b0f0b80 10 client.16359 fetch_fsmap learned FSMap version 31 2024-05-09T00:54:50.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.792+0000 7ffb5b0f0b80 1 -- 172.21.15.5:0/265460712 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563f22d9ba90 con 0x563f22cf1160 2024-05-09T00:54:50.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.793+0000 7ffb2dffb700 1 -- 172.21.15.5:0/265460712 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb3c0acfb0 con 0x563f22cf1160 2024-05-09T00:54:50.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.793+0000 7ffb2dffb700 1 -- 172.21.15.5:0/265460712 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7ffb340123b0 con 0x563f22cf1160 2024-05-09T00:54:50.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.793+0000 7ffb5b0f0b80 10 client.16359 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:54:50.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.793+0000 7ffb5b0f0b80 10 client.16359 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T00:54:50.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.793+0000 7ffb5b0f0b80 20 client.16359 populate_metadata read hostname 'smithi005' 2024-05-09T00:54:50.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.793+0000 7ffb5b0f0b80 1 --2- 172.21.15.5:0/265460712 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x563f22d9e480 0x563f22da08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:50.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.793+0000 7ffb5b0f0b80 4 client.16359 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:54:50.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.793+0000 7ffb5b0f0b80 1 -- 172.21.15.5:0/265460712 --> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f22cb49d0 con 0x563f22d9e480 2024-05-09T00:54:50.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.794+0000 7ffb4a476700 1 --2- 172.21.15.5:0/265460712 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x563f22d9e480 0x563f22da08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:50.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.802+0000 7ffb4a476700 1 --2- 172.21.15.5:0/265460712 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x563f22d9e480 0x563f22da08f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5cf33e7dd74bf05d server_cookie=6941dc7420dac9be in_seq=0 out_seq=0 2024-05-09T00:54:50.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.802+0000 7ffb2dffb700 10 client.16359 ms_handle_connect on v2:172.21.15.157:6834/832898273 2024-05-09T00:54:50.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.807+0000 7ffb2dffb700 1 -- 172.21.15.5:0/265460712 <== mds.3 v2:172.21.15.157:6834/832898273 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563f22cb49d0 con 0x563f22d9e480 2024-05-09T00:54:50.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.807+0000 7ffb2dffb700 10 client.16359 handle_command_reply: tid=0 2024-05-09T00:54:50.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.808+0000 7ffb5b0f0b80 1 client.16359 shutdown 2024-05-09T00:54:50.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.815+0000 7ffb5b0f0b80 1 -- 172.21.15.5:0/265460712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f22cf1160 msgr2=0x563f22d8e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:54:50.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.815+0000 7ffb5b0f0b80 1 --2- 172.21.15.5:0/265460712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f22cf1160 0x563f22d8e6c0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x563f22d80ed0 tx=0x7ffb34004750 comp rx=0 tx=0).stop 2024-05-09T00:54:50.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.816+0000 7ffb5b0f0b80 1 -- 172.21.15.5:0/265460712 shutdown_connections 2024-05-09T00:54:50.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.816+0000 7ffb5b0f0b80 1 --2- 172.21.15.5:0/265460712 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x563f22d9e480 0x563f22da08f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:50.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.816+0000 7ffb5b0f0b80 1 --2- 172.21.15.5:0/265460712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f22d846a0 0x563f22d8ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:50.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.816+0000 7ffb5b0f0b80 1 --2- 172.21.15.5:0/265460712 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f22c074a0 0x563f22d8e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:50.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.816+0000 7ffb5b0f0b80 1 --2- 172.21.15.5:0/265460712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f22cf1160 0x563f22d8e6c0 secure :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0x563f22d80ed0 tx=0x7ffb34004750 comp rx=0 tx=0).stop 2024-05-09T00:54:50.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.816+0000 7ffb5b0f0b80 1 -- 172.21.15.5:0/265460712 >> 172.21.15.5:0/265460712 conn(0x563f22c676a0 msgr2=0x563f22d7f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:54:50.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.816+0000 7ffb2dffb700 0 client.16359 ms_handle_reset on v2:172.21.15.157:6834/832898273 2024-05-09T00:54:50.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.822+0000 7ffb5b0f0b80 1 -- 172.21.15.5:0/265460712 shutdown_connections 2024-05-09T00:54:50.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.824+0000 7ffb5b0f0b80 1 -- 172.21.15.5:0/265460712 wait complete. 2024-05-09T00:54:50.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.825+0000 7ffb5b0f0b80 20 client.16359 trim_cache size 0 max 16384 2024-05-09T00:54:50.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.872+0000 7ffb5b0f0b80 1 Processor -- start 2024-05-09T00:54:50.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.872+0000 7ffb5b0f0b80 1 -- start start 2024-05-09T00:54:50.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.873+0000 7ffb5b0f0b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f22c074a0 0x563f22d8d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:50.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.873+0000 7ffb5b0f0b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f22d846a0 0x563f22da0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:50.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.874+0000 7ffb5b0f0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f22d89980 0x563f22d9a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:50.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.874+0000 7ffb5b0f0b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffb441acc50 con 0x563f22c074a0 2024-05-09T00:54:50.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.874+0000 7ffb5b0f0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ffb441ae790 con 0x563f22d89980 2024-05-09T00:54:50.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.874+0000 7ffb5b0f0b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ffb441ad320 con 0x563f22d846a0 2024-05-09T00:54:50.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.877+0000 7ffb4ac77700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f22d89980 0x563f22d9a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:50.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.877+0000 7ffb49c75700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f22d846a0 0x563f22da0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:50.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.878+0000 7ffb4ac77700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f22d89980 0x563f22d9a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:40258/0 (socket says 172.21.15.5:40258) 2024-05-09T00:54:50.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.878+0000 7ffb4ac77700 1 -- 172.21.15.5:0/568717463 learned_addr learned my addr 172.21.15.5:0/568717463 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:54:50.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.878+0000 7ffb4a476700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f22c074a0 0x563f22d8d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:50.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.881+0000 7ffb4ac77700 1 -- 172.21.15.5:0/568717463 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f22d846a0 msgr2=0x563f22da0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:54:50.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.881+0000 7ffb4ac77700 1 --2- 172.21.15.5:0/568717463 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f22d846a0 0x563f22da0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:50.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.881+0000 7ffb4ac77700 1 -- 172.21.15.5:0/568717463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f22c074a0 msgr2=0x563f22d8d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:54:50.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.881+0000 7ffb4ac77700 1 --2- 172.21.15.5:0/568717463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f22c074a0 0x563f22d8d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:50.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.881+0000 7ffb4ac77700 1 -- 172.21.15.5:0/568717463 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f22d84fd0 con 0x563f22d89980 2024-05-09T00:54:50.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.884+0000 7ffb4ac77700 1 --2- 172.21.15.5:0/568717463 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f22d89980 0x563f22d9a0a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c0a8900 tx=0x7ffb3c0a8370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e9f9107e1596227 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:54:50.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.885+0000 7ffb2ffff700 1 -- 172.21.15.5:0/568717463 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c0c8070 con 0x563f22d89980 2024-05-09T00:54:50.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.885+0000 7ffb2ffff700 1 -- 172.21.15.5:0/568717463 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb3c0acfb0 con 0x563f22d89980 2024-05-09T00:54:50.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.887+0000 7ffb5b0f0b80 1 -- 172.21.15.5:0/568717463 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f22d89980 msgr2=0x563f22d9a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:54:50.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.887+0000 7ffb5b0f0b80 1 --2- 172.21.15.5:0/568717463 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f22d89980 0x563f22d9a0a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c0a8900 tx=0x7ffb3c0a8370 comp rx=0 tx=0).stop 2024-05-09T00:54:50.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.887+0000 7ffb5b0f0b80 1 -- 172.21.15.5:0/568717463 shutdown_connections 2024-05-09T00:54:50.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.887+0000 7ffb5b0f0b80 1 --2- 172.21.15.5:0/568717463 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f22d89980 0x563f22d9a0a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:50.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.887+0000 7ffb5b0f0b80 1 --2- 172.21.15.5:0/568717463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f22c074a0 0x563f22d8d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:50.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.887+0000 7ffb5b0f0b80 1 --2- 172.21.15.5:0/568717463 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f22d846a0 0x563f22da0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:50.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.887+0000 7ffb5b0f0b80 1 -- 172.21.15.5:0/568717463 >> 172.21.15.5:0/568717463 conn(0x563f22c676a0 msgr2=0x563f22d89270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:54:50.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.888+0000 7ffb5b0f0b80 1 -- 172.21.15.5:0/568717463 shutdown_connections 2024-05-09T00:54:50.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.888+0000 7ffb5b0f0b80 1 -- 172.21.15.5:0/568717463 wait complete. 2024-05-09T00:54:50.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.889+0000 7ffb5b0f0b80 1 -- start start 2024-05-09T00:54:50.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.889+0000 7ffb5b0f0b80 1 Processor -- start 2024-05-09T00:54:50.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.890+0000 7ffb5b0f0b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f22c074a0 0x563f22d96420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:50.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.890+0000 7ffb5b0f0b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f22d846a0 0x563f22d96960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:50.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.891+0000 7ffb5b0f0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f22d8b720 0x563f22d8bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:50.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.891+0000 7ffb5b0f0b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffb441ae0c0 con 0x563f22d846a0 2024-05-09T00:54:50.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.891+0000 7ffb5b0f0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ffb441ad9f0 con 0x563f22d8b720 2024-05-09T00:54:50.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.891+0000 7ffb5b0f0b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ffb441aabf0 con 0x563f22c074a0 2024-05-09T00:54:50.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.891+0000 7ffb4a476700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f22c074a0 0x563f22d96420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:50.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.891+0000 7ffb4a476700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f22c074a0 0x563f22d96420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:38322/0 (socket says 172.21.15.5:38322) 2024-05-09T00:54:50.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.891+0000 7ffb4a476700 1 -- 172.21.15.5:0/2817506422 learned_addr learned my addr 172.21.15.5:0/2817506422 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:54:50.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.891+0000 7ffb5b0f0b80 10 client.0 _ll_register_callbacks cb 0x563f22b186c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:54:50.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.894+0000 7ffb4ac77700 1 --2- 172.21.15.5:0/2817506422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f22d8b720 0x563f22d8bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:50.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.894+0000 7ffb49c75700 1 --2- 172.21.15.5:0/2817506422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f22d846a0 0x563f22d96960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:50.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.896+0000 7ffb4a476700 1 -- 172.21.15.5:0/2817506422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f22d8b720 msgr2=0x563f22d8bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:54:50.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.896+0000 7ffb4a476700 1 --2- 172.21.15.5:0/2817506422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f22d8b720 0x563f22d8bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:50.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.896+0000 7ffb4a476700 1 -- 172.21.15.5:0/2817506422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f22d846a0 msgr2=0x563f22d96960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:54:50.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.896+0000 7ffb4a476700 1 --2- 172.21.15.5:0/2817506422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f22d846a0 0x563f22d96960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:50.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.896+0000 7ffb4a476700 1 -- 172.21.15.5:0/2817506422 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f22d8c230 con 0x563f22c074a0 2024-05-09T00:54:50.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.901+0000 7ffb4a476700 1 --2- 172.21.15.5:0/2817506422 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f22c074a0 0x563f22d96420 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ffb400098e0 tx=0x7ffb4000bb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a02ea2862aa83e3b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:54:50.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.901+0000 7ffb2dffb700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-09T00:54:50.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.902+0000 7ffb2dffb700 1 -- 172.21.15.5:0/2817506422 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb40003a60 con 0x563f22c074a0 2024-05-09T00:54:50.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.902+0000 7ffb5b0f0b80 1 -- 172.21.15.5:0/2817506422 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563f22caea70 con 0x563f22c074a0 2024-05-09T00:54:50.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.902+0000 7ffb2dffb700 1 -- 172.21.15.5:0/2817506422 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7ffb40003c00 con 0x563f22c074a0 2024-05-09T00:54:50.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.903+0000 7ffb2dffb700 1 -- 172.21.15.5:0/2817506422 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ffb40003e00 con 0x563f22c074a0 2024-05-09T00:54:50.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.903+0000 7ffb5b0f0b80 10 client.25894 fetch_fsmap learned FSMap version 31 2024-05-09T00:54:50.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.903+0000 7ffb5b0f0b80 1 -- 172.21.15.5:0/2817506422 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563f22e07d60 con 0x563f22c074a0 2024-05-09T00:54:50.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.906+0000 7ffb2dffb700 1 -- 172.21.15.5:0/2817506422 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x563f22caea70 con 0x563f22c074a0 2024-05-09T00:54:50.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.906+0000 7ffb2dffb700 1 -- 172.21.15.5:0/2817506422 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7ffb40021040 con 0x563f22c074a0 2024-05-09T00:54:50.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.906+0000 7ffb5b0f0b80 10 client.25894 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:54:50.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.906+0000 7ffb5b0f0b80 10 client.25894 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T00:54:50.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.906+0000 7ffb5b0f0b80 20 client.25894 populate_metadata read hostname 'smithi005' 2024-05-09T00:54:50.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.906+0000 7ffb5b0f0b80 1 --2- 172.21.15.5:0/2817506422 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x563f22e084f0 0x563f22e0a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:50.161 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-09T00:54:50.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.906+0000 7ffb5b0f0b80 4 client.25894 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:54:50.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.906+0000 7ffb5b0f0b80 1 -- 172.21.15.5:0/2817506422 --> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563f22b22970 con 0x563f22e084f0 2024-05-09T00:54:50.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.910+0000 7ffb49c75700 1 --2- 172.21.15.5:0/2817506422 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x563f22e084f0 0x563f22e0a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:50.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.915+0000 7ffb49c75700 1 --2- 172.21.15.5:0/2817506422 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x563f22e084f0 0x563f22e0a910 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f36ea9ef7ef67991 server_cookie=11316e1743a02c95 in_seq=0 out_seq=0 2024-05-09T00:54:50.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.915+0000 7ffb2dffb700 10 client.25894 ms_handle_connect on v2:172.21.15.157:6834/832898273 2024-05-09T00:54:50.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.917+0000 7ffb2dffb700 1 -- 172.21.15.5:0/2817506422 <== mds.3 v2:172.21.15.157:6834/832898273 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563f22b22970 con 0x563f22e084f0 2024-05-09T00:54:50.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.917+0000 7ffb2dffb700 10 client.25894 handle_command_reply: tid=0 2024-05-09T00:54:50.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.917+0000 7ffb5b0f0b80 1 client.25894 shutdown 2024-05-09T00:54:50.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.919+0000 7ffb5b0f0b80 1 -- 172.21.15.5:0/2817506422 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f22c074a0 msgr2=0x563f22d96420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:54:50.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.919+0000 7ffb5b0f0b80 1 --2- 172.21.15.5:0/2817506422 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f22c074a0 0x563f22d96420 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ffb400098e0 tx=0x7ffb4000bb90 comp rx=0 tx=0).stop 2024-05-09T00:54:50.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.919+0000 7ffb5b0f0b80 1 -- 172.21.15.5:0/2817506422 shutdown_connections 2024-05-09T00:54:50.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.919+0000 7ffb5b0f0b80 1 --2- 172.21.15.5:0/2817506422 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x563f22e084f0 0x563f22e0a910 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:50.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.920+0000 7ffb5b0f0b80 1 --2- 172.21.15.5:0/2817506422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563f22d8b720 0x563f22d8bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:50.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.920+0000 7ffb5b0f0b80 1 --2- 172.21.15.5:0/2817506422 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f22c074a0 0x563f22d96420 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:50.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.920+0000 7ffb5b0f0b80 1 --2- 172.21.15.5:0/2817506422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f22d846a0 0x563f22d96960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:50.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.920+0000 7ffb5b0f0b80 1 -- 172.21.15.5:0/2817506422 >> 172.21.15.5:0/2817506422 conn(0x563f22c676a0 msgr2=0x563f22d8d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:54:50.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.921+0000 7ffb2dffb700 0 client.25894 ms_handle_reset on v2:172.21.15.157:6834/832898273 2024-05-09T00:54:50.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.921+0000 7ffb5b0f0b80 1 -- 172.21.15.5:0/2817506422 shutdown_connections 2024-05-09T00:54:50.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.921+0000 7ffb5b0f0b80 1 -- 172.21.15.5:0/2817506422 wait complete. 2024-05-09T00:54:50.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.921+0000 7ffb5b0f0b80 20 client.25894 trim_cache size 0 max 16384 2024-05-09T00:54:50.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.931+0000 7ffb4c6da700 1 -- 172.21.15.5:0/752019490 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ffb300b2980 msgr2=0x7ffb300b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:54:50.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.931+0000 7ffb4c6da700 1 --2- 172.21.15.5:0/752019490 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ffb300b2980 0x7ffb300b4e90 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ffb44103420 tx=0x7ffb34006cb0 comp rx=0 tx=0).stop 2024-05-09T00:54:50.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.931+0000 7ffb4c6da700 1 -- 172.21.15.5:0/752019490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb44109850 msgr2=0x7ffb4419f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:54:50.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.931+0000 7ffb4c6da700 1 --2- 172.21.15.5:0/752019490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb44109850 0x7ffb4419f6f0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c00b970 tx=0x7ffb3c00bd30 comp rx=0 tx=0).stop 2024-05-09T00:54:50.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.931+0000 7ffb4c6da700 1 -- 172.21.15.5:0/752019490 shutdown_connections 2024-05-09T00:54:50.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.931+0000 7ffb4c6da700 1 --2- 172.21.15.5:0/752019490 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ffb300b2980 0x7ffb300b4e90 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:50.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.931+0000 7ffb4c6da700 1 --2- 172.21.15.5:0/752019490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb44109850 0x7ffb4419f6f0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:50.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.931+0000 7ffb4c6da700 1 --2- 172.21.15.5:0/752019490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb441045d0 0x7ffb4419aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:50.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.931+0000 7ffb4c6da700 1 --2- 172.21.15.5:0/752019490 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb44108f00 0x7ffb4419b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:50.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.931+0000 7ffb4c6da700 1 -- 172.21.15.5:0/752019490 >> 172.21.15.5:0/752019490 conn(0x7ffb440ffe80 msgr2=0x7ffb4410a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:54:50.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.933+0000 7ffb4c6da700 1 -- 172.21.15.5:0/752019490 shutdown_connections 2024-05-09T00:54:50.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:49.933+0000 7ffb4c6da700 1 -- 172.21.15.5:0/752019490 wait complete. 2024-05-09T00:54:51.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:54:51 smithi131 ceph-mon[34547]: pgmap v626: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1019 MiB/s wr, 2.16k op/s 2024-05-09T00:54:51.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:54:51 smithi157 ceph-mon[34294]: pgmap v626: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1019 MiB/s wr, 2.16k op/s 2024-05-09T00:54:51.741 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:54:51 smithi005 ceph-mon[30365]: pgmap v626: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1019 MiB/s wr, 2.16k op/s 2024-05-09T00:54:52.655 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T00:54:53.513 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:54:53.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:54:53 smithi131 ceph-mon[34547]: pgmap v627: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 852 MiB/s wr, 1.55k op/s 2024-05-09T00:54:53.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:54:53 smithi157 ceph-mon[34294]: pgmap v627: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 852 MiB/s wr, 1.55k op/s 2024-05-09T00:54:53.754 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:54:53 smithi005 ceph-mon[30365]: pgmap v627: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 852 MiB/s wr, 1.55k op/s 2024-05-09T00:54:54.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:54.956+0000 7fe84cff9640 1 -- 172.21.15.5:0/1642443073 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ccd711a350 con 0x55ccd6e231c0 2024-05-09T00:54:54.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:54.956+0000 7fe84cff9640 10 client.25882 handle_command_reply: tid=0 2024-05-09T00:54:54.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:54.956+0000 7fe87256c740 1 client.25882 shutdown 2024-05-09T00:54:54.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:54.956+0000 7fe87256c740 1 -- 172.21.15.5:0/1642443073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ccd6e662f0 msgr2=0x55ccd724c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:54:54.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:54.956+0000 7fe87256c740 1 --2- 172.21.15.5:0/1642443073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ccd6e662f0 0x55ccd724c3d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe85c0df7d0 tx=0x7fe85c009a10 comp rx=0 tx=0).stop 2024-05-09T00:54:54.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:54.957+0000 7fe87256c740 1 -- 172.21.15.5:0/1642443073 shutdown_connections 2024-05-09T00:54:54.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:54.957+0000 7fe87256c740 1 --2- 172.21.15.5:0/1642443073 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55ccd6e231c0 0x55ccd72c5ab0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:54.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:54.957+0000 7fe87256c740 1 --2- 172.21.15.5:0/1642443073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccd6e8f1b0 0x55ccd7249550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:54.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:54.957+0000 7fe87256c740 1 --2- 172.21.15.5:0/1642443073 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ccd6e6e4d0 0x55ccd7249010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:54.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:54.957+0000 7fe87256c740 1 --2- 172.21.15.5:0/1642443073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ccd6e662f0 0x55ccd724c3d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:54.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:54.957+0000 7fe87256c740 1 -- 172.21.15.5:0/1642443073 >> 172.21.15.5:0/1642443073 conn(0x55ccd6e83300 msgr2=0x55ccd7245af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:54:54.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:54.957+0000 7fe84cff9640 0 client.25882 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:54:54.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:54.957+0000 7fe87256c740 1 -- 172.21.15.5:0/1642443073 shutdown_connections 2024-05-09T00:54:54.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:54.957+0000 7fe87256c740 1 -- 172.21.15.5:0/1642443073 wait complete. 2024-05-09T00:54:54.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:54.957+0000 7fe87256c740 20 client.25882 trim_cache size 0 max 16384 2024-05-09T00:54:54.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-09T00:54:54.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-09T00:54:54.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-09T00:54:54.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:54.963+0000 7fe86b6b8640 1 -- 172.21.15.5:0/4132144451 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe8500b2970 msgr2=0x7fe8500b4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:54:54.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:54.963+0000 7fe86b6b8640 1 --2- 172.21.15.5:0/4132144451 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe8500b2970 0x7fe8500b4e80 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe860009510 tx=0x7fe860009480 comp rx=0 tx=0).stop 2024-05-09T00:54:54.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:54.963+0000 7fe86b6b8640 1 -- 172.21.15.5:0/4132144451 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe86410a550 msgr2=0x7fe8641a4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:54:54.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:54.963+0000 7fe86b6b8640 1 --2- 172.21.15.5:0/4132144451 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe86410a550 0x7fe8641a4ef0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe85c00b9e0 tx=0x7fe85c00bea0 comp rx=0 tx=0).stop 2024-05-09T00:54:54.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:54.963+0000 7fe86b6b8640 1 -- 172.21.15.5:0/4132144451 shutdown_connections 2024-05-09T00:54:54.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:54.963+0000 7fe86b6b8640 1 --2- 172.21.15.5:0/4132144451 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe8500b2970 0x7fe8500b4e80 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:54.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:54.963+0000 7fe86b6b8640 1 --2- 172.21.15.5:0/4132144451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe86410c180 0x7fe8641ac3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:54.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:54.963+0000 7fe86b6b8640 1 --2- 172.21.15.5:0/4132144451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe86410b480 0x7fe8641a5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:54.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:54.963+0000 7fe86b6b8640 1 --2- 172.21.15.5:0/4132144451 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe86410a550 0x7fe8641a4ef0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:54.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:54.963+0000 7fe86b6b8640 1 -- 172.21.15.5:0/4132144451 >> 172.21.15.5:0/4132144451 conn(0x7fe864070670 msgr2=0x7fe864113ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:54:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:54.965+0000 7fe86b6b8640 1 -- 172.21.15.5:0/4132144451 shutdown_connections 2024-05-09T00:54:54.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:54:54.970+0000 7fe86b6b8640 1 -- 172.21.15.5:0/4132144451 wait complete. 2024-05-09T00:54:55.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:54:55 smithi131 ceph-mon[34547]: pgmap v628: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 931 MiB/s wr, 1.66k op/s 2024-05-09T00:54:55.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:54:55 smithi157 ceph-mon[34294]: pgmap v628: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 931 MiB/s wr, 1.66k op/s 2024-05-09T00:54:55.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:54:55 smithi005 ceph-mon[30365]: pgmap v628: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 931 MiB/s wr, 1.66k op/s 2024-05-09T00:54:57.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:54:57 smithi131 ceph-mon[34547]: pgmap v629: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 881 MiB/s wr, 1.68k op/s 2024-05-09T00:54:57.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:54:57 smithi157 ceph-mon[34294]: pgmap v629: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 881 MiB/s wr, 1.68k op/s 2024-05-09T00:54:57.742 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:54:57 smithi005 ceph-mon[30365]: pgmap v629: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 881 MiB/s wr, 1.68k op/s 2024-05-09T00:54:58.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.489+0000 7fade759e700 1 -- 172.21.15.5:0/2055593290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fade009b520 msgr2=0x7fade009b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:54:58.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.489+0000 7fade759e700 1 --2- 172.21.15.5:0/2055593290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fade009b520 0x7fade009b930 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fadd8009990 tx=0x7fadd8009ca0 comp rx=0 tx=0).stop 2024-05-09T00:54:58.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.490+0000 7fade759e700 1 -- 172.21.15.5:0/2055593290 shutdown_connections 2024-05-09T00:54:58.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.490+0000 7fade759e700 1 --2- 172.21.15.5:0/2055593290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fade0095e30 0x7fade00a22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:58.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.490+0000 7fade759e700 1 --2- 172.21.15.5:0/2055593290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fade009b520 0x7fade009b930 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:58.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.490+0000 7fade759e700 1 --2- 172.21.15.5:0/2055593290 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fade00954e0 0x7fade00958f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:58.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.490+0000 7fade759e700 1 -- 172.21.15.5:0/2055593290 >> 172.21.15.5:0/2055593290 conn(0x7fade000b020 msgr2=0x7fade000b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:54:58.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.492+0000 7fade759e700 1 -- 172.21.15.5:0/2055593290 shutdown_connections 2024-05-09T00:54:58.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.492+0000 7fade759e700 1 -- 172.21.15.5:0/2055593290 wait complete. 2024-05-09T00:54:58.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.497+0000 7fade759e700 1 Processor -- start 2024-05-09T00:54:58.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.498+0000 7fade759e700 1 -- start start 2024-05-09T00:54:58.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.499+0000 7fade759e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fade00954e0 0x7fade00a55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:58.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.499+0000 7fade759e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fade0095e30 0x7fade00a5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:58.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.499+0000 7fade759e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fade009b520 0x7fade00a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:58.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.499+0000 7fade759e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fade00a3d10 con 0x7fade0095e30 2024-05-09T00:54:58.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.499+0000 7fade759e700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fade00a3b90 con 0x7fade009b520 2024-05-09T00:54:58.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.499+0000 7fade759e700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fade00a3e90 con 0x7fade00954e0 2024-05-09T00:54:58.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.502+0000 7fade659c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fade00954e0 0x7fade00a55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:58.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.503+0000 7fade659c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fade00954e0 0x7fade00a55e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:56154/0 (socket says 172.21.15.5:56154) 2024-05-09T00:54:58.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.503+0000 7fade659c700 1 -- 172.21.15.5:0/4038858042 learned_addr learned my addr 172.21.15.5:0/4038858042 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:54:58.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.506+0000 7fade6d9d700 1 --2- 172.21.15.5:0/4038858042 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fade009b520 0x7fade00a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:58.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.506+0000 7fade5d9b700 1 --2- 172.21.15.5:0/4038858042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fade0095e30 0x7fade00a5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:58.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.508+0000 7fade659c700 1 -- 172.21.15.5:0/4038858042 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fade009b520 msgr2=0x7fade00a6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:54:58.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.509+0000 7fade659c700 1 --2- 172.21.15.5:0/4038858042 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fade009b520 0x7fade00a6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:58.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.509+0000 7fade659c700 1 -- 172.21.15.5:0/4038858042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fade0095e30 msgr2=0x7fade00a5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:54:58.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.509+0000 7fade659c700 1 --2- 172.21.15.5:0/4038858042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fade0095e30 0x7fade00a5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:58.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.509+0000 7fade659c700 1 -- 172.21.15.5:0/4038858042 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fade0147f30 con 0x7fade00954e0 2024-05-09T00:54:58.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.509+0000 7fade5d9b700 1 --2- 172.21.15.5:0/4038858042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fade0095e30 0x7fade00a5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:54:58.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.511+0000 7fade659c700 1 --2- 172.21.15.5:0/4038858042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fade00954e0 0x7fade00a55e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fadd80098e0 tx=0x7fadd8009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:54:58.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.514+0000 7fadd77fe700 1 -- 172.21.15.5:0/4038858042 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd8003ce0 con 0x7fade00954e0 2024-05-09T00:54:58.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.515+0000 7fade759e700 1 -- 172.21.15.5:0/4038858042 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fade01481c0 con 0x7fade00954e0 2024-05-09T00:54:58.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.515+0000 7fade759e700 1 -- 172.21.15.5:0/4038858042 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fade01485b0 con 0x7fade00954e0 2024-05-09T00:54:58.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.515+0000 7fadd77fe700 1 -- 172.21.15.5:0/4038858042 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadd8004830 con 0x7fade00954e0 2024-05-09T00:54:58.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.515+0000 7fadd77fe700 1 -- 172.21.15.5:0/4038858042 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd80054e0 con 0x7fade00954e0 2024-05-09T00:54:58.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.533+0000 7fadd77fe700 1 -- 172.21.15.5:0/4038858042 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fadd8028020 con 0x7fade00954e0 2024-05-09T00:54:58.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.536+0000 7fadd77fe700 1 --2- 172.21.15.5:0/4038858042 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fadd00b2a90 0x7fadd00b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:58.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.536+0000 7fadd77fe700 1 -- 172.21.15.5:0/4038858042 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fadd80e3150 con 0x7fade00954e0 2024-05-09T00:54:58.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.537+0000 7fadfb740b80 1 Processor -- start 2024-05-09T00:54:58.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.538+0000 7fadfb740b80 1 -- start start 2024-05-09T00:54:58.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.538+0000 7fade5d9b700 1 --2- 172.21.15.5:0/4038858042 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fadd00b2a90 0x7fadd00b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:58.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.539+0000 7fadfb740b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4b46e4160 0x55c4b477d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:58.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.539+0000 7fadfb740b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4b45fa4a0 0x55c4b477db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:58.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.539+0000 7fadfb740b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4b4777800 0x55c4b4777c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:58.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.539+0000 7fadfb740b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c4b4699ff0 con 0x55c4b46e4160 2024-05-09T00:54:58.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.539+0000 7fadfb740b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55c4b46387f0 con 0x55c4b45fa4a0 2024-05-09T00:54:58.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.539+0000 7fadfb740b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55c4b4695fb0 con 0x55c4b4777800 2024-05-09T00:54:58.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.540+0000 7fade5d9b700 1 --2- 172.21.15.5:0/4038858042 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fadd00b2a90 0x7fadd00b4fa0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fade00944f0 tx=0x7faddc00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:54:58.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.542+0000 7fade6d9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4b4777800 0x55c4b4777c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:58.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.542+0000 7fade6d9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4b4777800 0x55c4b4777c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:56168/0 (socket says 172.21.15.5:56168) 2024-05-09T00:54:58.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.542+0000 7fade6d9d700 1 -- 172.21.15.5:0/3129293023 learned_addr learned my addr 172.21.15.5:0/3129293023 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:54:58.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.542+0000 7fade659c700 1 --2- 172.21.15.5:0/3129293023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4b46e4160 0x55c4b477d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:58.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.542+0000 7fade5d9b700 1 --2- 172.21.15.5:0/3129293023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4b45fa4a0 0x55c4b477db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:58.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.544+0000 7fade6d9d700 1 -- 172.21.15.5:0/3129293023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4b45fa4a0 msgr2=0x55c4b477db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:54:58.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.544+0000 7fade6d9d700 1 --2- 172.21.15.5:0/3129293023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4b45fa4a0 0x55c4b477db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:58.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.544+0000 7fade6d9d700 1 -- 172.21.15.5:0/3129293023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4b46e4160 msgr2=0x55c4b477d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:54:58.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.544+0000 7fade6d9d700 1 --2- 172.21.15.5:0/3129293023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4b46e4160 0x55c4b477d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:58.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.544+0000 7fade6d9d700 1 -- 172.21.15.5:0/3129293023 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4b4630370 con 0x55c4b4777800 2024-05-09T00:54:58.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.544+0000 7fade5d9b700 1 --2- 172.21.15.5:0/3129293023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4b45fa4a0 0x55c4b477db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:54:58.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.546+0000 7fade6d9d700 1 --2- 172.21.15.5:0/3129293023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4b4777800 0x55c4b4777c10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fade8051590 tx=0x7fade8069f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5c14230eec0c86 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:54:58.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.547+0000 7fadd4ff9700 1 -- 172.21.15.5:0/3129293023 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade8075b70 con 0x55c4b4777800 2024-05-09T00:54:58.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.548+0000 7fadd4ff9700 1 -- 172.21.15.5:0/3129293023 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fade806cf70 con 0x55c4b4777800 2024-05-09T00:54:58.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.548+0000 7fadd4ff9700 1 -- 172.21.15.5:0/3129293023 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade8082920 con 0x55c4b4777800 2024-05-09T00:54:58.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.549+0000 7fadfb740b80 1 -- 172.21.15.5:0/3129293023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4b4777800 msgr2=0x55c4b4777c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:54:58.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.549+0000 7fadfb740b80 1 --2- 172.21.15.5:0/3129293023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4b4777800 0x55c4b4777c10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fade8051590 tx=0x7fade8069f40 comp rx=0 tx=0).stop 2024-05-09T00:54:58.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.550+0000 7fadfb740b80 1 -- 172.21.15.5:0/3129293023 shutdown_connections 2024-05-09T00:54:58.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.550+0000 7fadfb740b80 1 --2- 172.21.15.5:0/3129293023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4b4777800 0x55c4b4777c10 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:58.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.550+0000 7fadfb740b80 1 --2- 172.21.15.5:0/3129293023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4b46e4160 0x55c4b477d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:58.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.550+0000 7fadfb740b80 1 --2- 172.21.15.5:0/3129293023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4b45fa4a0 0x55c4b477db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:58.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.550+0000 7fadfb740b80 1 -- 172.21.15.5:0/3129293023 >> 172.21.15.5:0/3129293023 conn(0x55c4b465a6a0 msgr2=0x55c4b4772620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:54:58.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.554+0000 7fadfb740b80 1 -- 172.21.15.5:0/3129293023 shutdown_connections 2024-05-09T00:54:58.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.554+0000 7fadfb740b80 1 -- 172.21.15.5:0/3129293023 wait complete. 2024-05-09T00:54:58.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.555+0000 7fadfb740b80 1 -- start start 2024-05-09T00:54:58.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.555+0000 7fadfb740b80 1 Processor -- start 2024-05-09T00:54:58.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.556+0000 7fadfb740b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4b45fa4a0 0x55c4b47813b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:58.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.556+0000 7fadfb740b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4b46e4160 0x55c4b47818f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:58.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.557+0000 7fadfb740b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4b4777800 0x55c4b4781e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:58.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.557+0000 7fade5d9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4b46e4160 0x55c4b47818f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:58.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.557+0000 7fade5d9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4b46e4160 0x55c4b47818f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:56182/0 (socket says 172.21.15.5:56182) 2024-05-09T00:54:58.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.557+0000 7fade5d9b700 1 -- 172.21.15.5:0/3906961370 learned_addr learned my addr 172.21.15.5:0/3906961370 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:54:58.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.557+0000 7fadfb740b80 1 -- 172.21.15.5:0/3906961370 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c4b4515ba0 con 0x55c4b4777800 2024-05-09T00:54:58.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.557+0000 7fadfb740b80 1 -- 172.21.15.5:0/3906961370 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55c4b4654150 con 0x55c4b45fa4a0 2024-05-09T00:54:58.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.557+0000 7fade659c700 1 --2- 172.21.15.5:0/3906961370 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4b45fa4a0 0x55c4b47813b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:58.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.558+0000 7fadfb740b80 1 -- 172.21.15.5:0/3906961370 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55c4b47785f0 con 0x55c4b46e4160 2024-05-09T00:54:58.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.558+0000 7fadfb740b80 10 client.0 _ll_register_callbacks cb 0x55c4b450b6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:54:58.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.559+0000 7fade659c700 1 -- 172.21.15.5:0/3906961370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4b46e4160 msgr2=0x55c4b47818f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:54:58.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.559+0000 7fade659c700 1 --2- 172.21.15.5:0/3906961370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4b46e4160 0x55c4b47818f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:58.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.559+0000 7fade659c700 1 -- 172.21.15.5:0/3906961370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4b4777800 msgr2=0x55c4b4781e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:54:58.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.559+0000 7fade659c700 1 --2- 172.21.15.5:0/3906961370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4b4777800 0x55c4b4781e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:58.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.560+0000 7fade659c700 1 -- 172.21.15.5:0/3906961370 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4b4515490 con 0x55c4b45fa4a0 2024-05-09T00:54:58.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.560+0000 7fade659c700 1 --2- 172.21.15.5:0/3906961370 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4b45fa4a0 0x55c4b47813b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fadd8009350 tx=0x7fadd80b4330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ef22963fd2ed65c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:54:58.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.560+0000 7fadcdffb700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-09T00:54:58.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.561+0000 7fadcdffb700 1 -- 172.21.15.5:0/3906961370 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd80b9060 con 0x55c4b45fa4a0 2024-05-09T00:54:58.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.561+0000 7fadcdffb700 1 -- 172.21.15.5:0/3906961370 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fadd80aab80 con 0x55c4b45fa4a0 2024-05-09T00:54:58.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.561+0000 7fadfb740b80 1 -- 172.21.15.5:0/3906961370 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fade806cf70 con 0x55c4b45fa4a0 2024-05-09T00:54:58.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.562+0000 7fadcdffb700 1 -- 172.21.15.5:0/3906961370 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fadd80abbf0 con 0x55c4b45fa4a0 2024-05-09T00:54:58.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.562+0000 7fadfb740b80 10 client.25883 fetch_fsmap learned FSMap version 31 2024-05-09T00:54:58.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.562+0000 7fadfb740b80 1 -- 172.21.15.5:0/3906961370 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c4b478ecc0 con 0x55c4b45fa4a0 2024-05-09T00:54:58.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.564+0000 7fadcdffb700 1 -- 172.21.15.5:0/3906961370 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadd80ab220 con 0x55c4b45fa4a0 2024-05-09T00:54:58.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.564+0000 7fadcdffb700 1 -- 172.21.15.5:0/3906961370 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fadd8003e80 con 0x55c4b45fa4a0 2024-05-09T00:54:58.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.565+0000 7fadfb740b80 10 client.25883 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:54:58.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.565+0000 7fadfb740b80 10 client.25883 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-05-09T00:54:58.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.565+0000 7fadfb740b80 20 client.25883 populate_metadata read hostname 'smithi005' 2024-05-09T00:54:58.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.565+0000 7fadfb740b80 1 --2- 172.21.15.5:0/3906961370 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x55c4b47912d0 0x55c4b47937a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:58.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.566+0000 7fadfb740b80 4 client.25883 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:54:58.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.566+0000 7fadfb740b80 1 -- 172.21.15.5:0/3906961370 --> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fade806cf70 con 0x55c4b47912d0 2024-05-09T00:54:58.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.567+0000 7fade5d9b700 1 --2- 172.21.15.5:0/3906961370 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x55c4b47912d0 0x55c4b47937a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:58.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.569+0000 7fade5d9b700 1 --2- 172.21.15.5:0/3906961370 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x55c4b47912d0 0x55c4b47937a0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fb9dc80e78ff13ec server_cookie=a3e6d41a7f459479 in_seq=0 out_seq=0 2024-05-09T00:54:58.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.569+0000 7fadcdffb700 10 client.25883 ms_handle_connect on v2:172.21.15.131:6838/2748669631 2024-05-09T00:54:58.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.573+0000 7fadcdffb700 1 -- 172.21.15.5:0/3906961370 <== mds.2 v2:172.21.15.131:6838/2748669631 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fade806cf70 con 0x55c4b47912d0 2024-05-09T00:54:58.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.574+0000 7fadcdffb700 10 client.25883 handle_command_reply: tid=0 2024-05-09T00:54:58.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.574+0000 7fadfb740b80 1 client.25883 shutdown 2024-05-09T00:54:58.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.577+0000 7fadfb740b80 1 -- 172.21.15.5:0/3906961370 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4b45fa4a0 msgr2=0x55c4b47813b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:54:58.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.577+0000 7fadfb740b80 1 --2- 172.21.15.5:0/3906961370 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4b45fa4a0 0x55c4b47813b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fadd8009350 tx=0x7fadd80b4330 comp rx=0 tx=0).stop 2024-05-09T00:54:58.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.577+0000 7fadfb740b80 1 -- 172.21.15.5:0/3906961370 shutdown_connections 2024-05-09T00:54:58.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.577+0000 7fadfb740b80 1 --2- 172.21.15.5:0/3906961370 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x55c4b47912d0 0x55c4b47937a0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:58.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.578+0000 7fadfb740b80 1 --2- 172.21.15.5:0/3906961370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4b4777800 0x55c4b4781e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:58.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.578+0000 7fadcdffb700 0 client.25883 ms_handle_reset on v2:172.21.15.131:6838/2748669631 2024-05-09T00:54:58.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.578+0000 7fadfb740b80 1 --2- 172.21.15.5:0/3906961370 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4b45fa4a0 0x55c4b47813b0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:58.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.578+0000 7fadfb740b80 1 --2- 172.21.15.5:0/3906961370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4b46e4160 0x55c4b47818f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:58.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.578+0000 7fadfb740b80 1 -- 172.21.15.5:0/3906961370 >> 172.21.15.5:0/3906961370 conn(0x55c4b465a6a0 msgr2=0x55c4b4772380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:54:58.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.579+0000 7fadfb740b80 1 -- 172.21.15.5:0/3906961370 shutdown_connections 2024-05-09T00:54:58.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.581+0000 7fadfb740b80 1 -- 172.21.15.5:0/3906961370 wait complete. 2024-05-09T00:54:58.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.581+0000 7fadfb740b80 20 client.25883 trim_cache size 0 max 16384 2024-05-09T00:54:58.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.610+0000 7fadfb740b80 1 Processor -- start 2024-05-09T00:54:58.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.610+0000 7fadfb740b80 1 -- start start 2024-05-09T00:54:58.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.610+0000 7fadfb740b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4b45fa4a0 0x55c4b478d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:58.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.610+0000 7fadfb740b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4b46e4160 0x55c4b4798fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:58.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.610+0000 7fadfb740b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4b4777800 0x55c4b47994e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:58.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.613+0000 7fade659c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4b45fa4a0 0x55c4b478d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:58.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.613+0000 7fade659c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4b45fa4a0 0x55c4b478d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:56196/0 (socket says 172.21.15.5:56196) 2024-05-09T00:54:58.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.613+0000 7fade5d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4b46e4160 0x55c4b4798fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:58.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.613+0000 7fade659c700 1 -- 172.21.15.5:0/3158389853 learned_addr learned my addr 172.21.15.5:0/3158389853 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:54:58.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.614+0000 7fadfb740b80 1 -- 172.21.15.5:0/3158389853 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fadc800d500 con 0x55c4b4777800 2024-05-09T00:54:58.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.614+0000 7fadfb740b80 1 -- 172.21.15.5:0/3158389853 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fadc800f040 con 0x55c4b46e4160 2024-05-09T00:54:58.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.614+0000 7fadfb740b80 1 -- 172.21.15.5:0/3158389853 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fadc800dbd0 con 0x55c4b45fa4a0 2024-05-09T00:54:58.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.615+0000 7fade6d9d700 1 --2- 172.21.15.5:0/3158389853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4b4777800 0x55c4b47994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:58.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.615+0000 7fade659c700 1 -- 172.21.15.5:0/3158389853 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4b46e4160 msgr2=0x55c4b4798fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:54:58.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.615+0000 7fade659c700 1 --2- 172.21.15.5:0/3158389853 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4b46e4160 0x55c4b4798fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:58.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.615+0000 7fade659c700 1 -- 172.21.15.5:0/3158389853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4b4777800 msgr2=0x55c4b47994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:54:58.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.615+0000 7fade659c700 1 --2- 172.21.15.5:0/3158389853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4b4777800 0x55c4b47994e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:58.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.615+0000 7fade659c700 1 -- 172.21.15.5:0/3158389853 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4b4783c50 con 0x55c4b45fa4a0 2024-05-09T00:54:58.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.616+0000 7fade659c700 1 --2- 172.21.15.5:0/3158389853 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4b45fa4a0 0x55c4b478d170 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fadd80039a0 tx=0x7fadd80a7700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acf153e8fbb53af0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:54:58.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.616+0000 7fadcffff700 1 -- 172.21.15.5:0/3158389853 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd80abbf0 con 0x55c4b45fa4a0 2024-05-09T00:54:58.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.616+0000 7fadcffff700 1 -- 172.21.15.5:0/3158389853 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadd80afb00 con 0x55c4b45fa4a0 2024-05-09T00:54:58.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.616+0000 7fadcffff700 1 -- 172.21.15.5:0/3158389853 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd80ab710 con 0x55c4b45fa4a0 2024-05-09T00:54:58.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.618+0000 7fadfb740b80 1 -- 172.21.15.5:0/3158389853 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4b45fa4a0 msgr2=0x55c4b478d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:54:58.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.618+0000 7fadfb740b80 1 --2- 172.21.15.5:0/3158389853 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4b45fa4a0 0x55c4b478d170 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fadd80039a0 tx=0x7fadd80a7700 comp rx=0 tx=0).stop 2024-05-09T00:54:58.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.618+0000 7fadfb740b80 1 -- 172.21.15.5:0/3158389853 shutdown_connections 2024-05-09T00:54:58.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.618+0000 7fadfb740b80 1 --2- 172.21.15.5:0/3158389853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4b4777800 0x55c4b47994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:58.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.618+0000 7fadfb740b80 1 --2- 172.21.15.5:0/3158389853 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4b45fa4a0 0x55c4b478d170 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:58.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.618+0000 7fadfb740b80 1 --2- 172.21.15.5:0/3158389853 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4b46e4160 0x55c4b4798fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:58.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.618+0000 7fadfb740b80 1 -- 172.21.15.5:0/3158389853 >> 172.21.15.5:0/3158389853 conn(0x55c4b465a6a0 msgr2=0x55c4b477fca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:54:58.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.618+0000 7fadfb740b80 1 -- 172.21.15.5:0/3158389853 shutdown_connections 2024-05-09T00:54:58.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.619+0000 7fadfb740b80 1 -- 172.21.15.5:0/3158389853 wait complete. 2024-05-09T00:54:58.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.619+0000 7fadfb740b80 1 -- start start 2024-05-09T00:54:58.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.620+0000 7fadfb740b80 1 Processor -- start 2024-05-09T00:54:58.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.621+0000 7fadfb740b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4b45fa4a0 0x55c4b47894f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:58.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.621+0000 7fadfb740b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4b46e4160 0x55c4b4789a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:58.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.621+0000 7fadfb740b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4b4777800 0x55c4b478c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:58.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.622+0000 7fadfb740b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fadc800e970 con 0x55c4b45fa4a0 2024-05-09T00:54:58.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.622+0000 7fadfb740b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fadc800e2a0 con 0x55c4b46e4160 2024-05-09T00:54:58.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.622+0000 7fadfb740b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fadc800b4a0 con 0x55c4b4777800 2024-05-09T00:54:58.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.622+0000 7fade5d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4b46e4160 0x55c4b4789a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:58.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.622+0000 7fade6d9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4b4777800 0x55c4b478c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:58.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.622+0000 7fade6d9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4b4777800 0x55c4b478c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:56208/0 (socket says 172.21.15.5:56208) 2024-05-09T00:54:58.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.622+0000 7fade6d9d700 1 -- 172.21.15.5:0/3850433112 learned_addr learned my addr 172.21.15.5:0/3850433112 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:54:58.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.622+0000 7fade5d9b700 1 --2- 172.21.15.5:0/3850433112 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4b46e4160 0x55c4b4789a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:40800/0 (socket says 172.21.15.5:40800) 2024-05-09T00:54:58.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.623+0000 7fadfb740b80 10 client.0 _ll_register_callbacks cb 0x55c4b450b6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:54:58.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.624+0000 7fade6d9d700 1 -- 172.21.15.5:0/3850433112 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4b46e4160 msgr2=0x55c4b4789a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:54:58.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.624+0000 7fade6d9d700 1 --2- 172.21.15.5:0/3850433112 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4b46e4160 0x55c4b4789a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:58.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.624+0000 7fade6d9d700 1 -- 172.21.15.5:0/3850433112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4b45fa4a0 msgr2=0x55c4b47894f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:54:58.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.624+0000 7fade6d9d700 1 --2- 172.21.15.5:0/3850433112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4b45fa4a0 0x55c4b47894f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:58.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.624+0000 7fade6d9d700 1 -- 172.21.15.5:0/3850433112 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4b478c720 con 0x55c4b4777800 2024-05-09T00:54:58.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.624+0000 7fade6d9d700 1 --2- 172.21.15.5:0/3850433112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4b4777800 0x55c4b478c160 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fade80525d0 tx=0x7fade8051cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c747edaee96e9e7d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:54:58.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.625+0000 7fadcdffb700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-09T00:54:58.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.625+0000 7fadcdffb700 1 -- 172.21.15.5:0/3850433112 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade806cac0 con 0x55c4b4777800 2024-05-09T00:54:58.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.625+0000 7fadcdffb700 1 -- 172.21.15.5:0/3850433112 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fade807e040 con 0x55c4b4777800 2024-05-09T00:54:58.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.625+0000 7fadfb740b80 1 -- 172.21.15.5:0/3850433112 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadd80afb00 con 0x55c4b4777800 2024-05-09T00:54:58.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.626+0000 7fadcdffb700 1 -- 172.21.15.5:0/3850433112 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fade806cc60 con 0x55c4b4777800 2024-05-09T00:54:58.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.628+0000 7fadfb740b80 10 client.25909 fetch_fsmap learned FSMap version 31 2024-05-09T00:54:58.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.628+0000 7fadfb740b80 1 -- 172.21.15.5:0/3850433112 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c4b47fb310 con 0x55c4b4777800 2024-05-09T00:54:58.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.628+0000 7fadcdffb700 1 -- 172.21.15.5:0/3850433112 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fade806ce00 con 0x55c4b4777800 2024-05-09T00:54:58.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.628+0000 7fadcdffb700 1 -- 172.21.15.5:0/3850433112 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fade8082400 con 0x55c4b4777800 2024-05-09T00:54:58.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.633+0000 7fadfb740b80 10 client.25909 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:54:58.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.633+0000 7fadfb740b80 10 client.25909 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-05-09T00:54:58.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.633+0000 7fadfb740b80 20 client.25909 populate_metadata read hostname 'smithi005' 2024-05-09T00:54:58.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.633+0000 7fadfb740b80 1 --2- 172.21.15.5:0/3850433112 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x55c4b47912d0 0x55c4b47fd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:54:58.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.633+0000 7fadfb740b80 4 client.25909 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:54:58.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.633+0000 7fadfb740b80 1 -- 172.21.15.5:0/3850433112 --> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fadd80afb00 con 0x55c4b47912d0 2024-05-09T00:54:58.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.635+0000 7fade659c700 1 --2- 172.21.15.5:0/3850433112 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x55c4b47912d0 0x55c4b47fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:54:58.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.643+0000 7fadcdffb700 10 client.25909 ms_handle_connect on v2:172.21.15.131:6838/2748669631 2024-05-09T00:54:58.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.643+0000 7fade659c700 1 --2- 172.21.15.5:0/3850433112 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x55c4b47912d0 0x55c4b47fd700 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1e0b896616a58b9a server_cookie=d2d41c42fbc9dd1d in_seq=0 out_seq=0 2024-05-09T00:54:58.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.644+0000 7fadcdffb700 1 -- 172.21.15.5:0/3850433112 <== mds.2 v2:172.21.15.131:6838/2748669631 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fadd80afb00 con 0x55c4b47912d0 2024-05-09T00:54:58.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.644+0000 7fadcdffb700 10 client.25909 handle_command_reply: tid=0 2024-05-09T00:54:58.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.644+0000 7fadfb740b80 1 client.25909 shutdown 2024-05-09T00:54:58.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.646+0000 7fadfb740b80 1 -- 172.21.15.5:0/3850433112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4b4777800 msgr2=0x55c4b478c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:54:58.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.646+0000 7fadfb740b80 1 --2- 172.21.15.5:0/3850433112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4b4777800 0x55c4b478c160 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fade80525d0 tx=0x7fade8051cd0 comp rx=0 tx=0).stop 2024-05-09T00:54:58.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.646+0000 7fadfb740b80 1 -- 172.21.15.5:0/3850433112 shutdown_connections 2024-05-09T00:54:58.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.646+0000 7fadfb740b80 1 --2- 172.21.15.5:0/3850433112 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x55c4b47912d0 0x55c4b47fd700 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:58.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.646+0000 7fadfb740b80 1 --2- 172.21.15.5:0/3850433112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4b4777800 0x55c4b478c160 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:58.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.646+0000 7fadcdffb700 0 client.25909 ms_handle_reset on v2:172.21.15.131:6838/2748669631 2024-05-09T00:54:58.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.646+0000 7fadfb740b80 1 --2- 172.21.15.5:0/3850433112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4b45fa4a0 0x55c4b47894f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:58.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.647+0000 7fadfb740b80 1 --2- 172.21.15.5:0/3850433112 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4b46e4160 0x55c4b4789a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:58.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.647+0000 7fadfb740b80 1 -- 172.21.15.5:0/3850433112 >> 172.21.15.5:0/3850433112 conn(0x55c4b465a6a0 msgr2=0x55c4b477c510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:54:58.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.647+0000 7fadfb740b80 1 -- 172.21.15.5:0/3850433112 shutdown_connections 2024-05-09T00:54:58.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.647+0000 7fadfb740b80 1 -- 172.21.15.5:0/3850433112 wait complete. 2024-05-09T00:54:58.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.647+0000 7fadfb740b80 20 client.25909 trim_cache size 0 max 16384 2024-05-09T00:54:58.653 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-09T00:54:58.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.654+0000 7fade759e700 1 -- 172.21.15.5:0/4038858042 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fadd00b2a90 msgr2=0x7fadd00b4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:54:58.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.654+0000 7fade759e700 1 --2- 172.21.15.5:0/4038858042 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fadd00b2a90 0x7fadd00b4fa0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fade00944f0 tx=0x7faddc00a300 comp rx=0 tx=0).stop 2024-05-09T00:54:58.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.654+0000 7fade759e700 1 -- 172.21.15.5:0/4038858042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fade00954e0 msgr2=0x7fade00a55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:54:58.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.654+0000 7fade759e700 1 --2- 172.21.15.5:0/4038858042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fade00954e0 0x7fade00a55e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fadd80098e0 tx=0x7fadd8009f90 comp rx=0 tx=0).stop 2024-05-09T00:54:58.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.654+0000 7fade759e700 1 -- 172.21.15.5:0/4038858042 shutdown_connections 2024-05-09T00:54:58.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.654+0000 7fade759e700 1 --2- 172.21.15.5:0/4038858042 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fadd00b2a90 0x7fadd00b4fa0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:58.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.654+0000 7fade759e700 1 --2- 172.21.15.5:0/4038858042 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fade009b520 0x7fade00a6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:58.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.654+0000 7fade759e700 1 --2- 172.21.15.5:0/4038858042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fade00954e0 0x7fade00a55e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:58.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.654+0000 7fade759e700 1 --2- 172.21.15.5:0/4038858042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fade0095e30 0x7fade00a5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:54:58.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.654+0000 7fade759e700 1 -- 172.21.15.5:0/4038858042 >> 172.21.15.5:0/4038858042 conn(0x7fade000b020 msgr2=0x7fade009c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:54:58.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.655+0000 7fade759e700 1 -- 172.21.15.5:0/4038858042 shutdown_connections 2024-05-09T00:54:58.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:54:58.655+0000 7fade759e700 1 -- 172.21.15.5:0/4038858042 wait complete. 2024-05-09T00:54:59.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:54:59 smithi131 ceph-mon[34547]: pgmap v630: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 863 MiB/s wr, 1.73k op/s 2024-05-09T00:54:59.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:54:59 smithi157 ceph-mon[34294]: pgmap v630: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 863 MiB/s wr, 1.73k op/s 2024-05-09T00:54:59.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:54:59 smithi005 ceph-mon[30365]: pgmap v630: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 863 MiB/s wr, 1.73k op/s 2024-05-09T00:55:01.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:55:01 smithi157 ceph-mon[34294]: pgmap v631: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1011 MiB/s wr, 1.97k op/s 2024-05-09T00:55:01.689 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:55:01 smithi005 ceph-mon[30365]: pgmap v631: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1011 MiB/s wr, 1.97k op/s 2024-05-09T00:55:01.692 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T00:55:01.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:55:01 smithi131 ceph-mon[34547]: pgmap v631: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1011 MiB/s wr, 1.97k op/s 2024-05-09T00:55:02.577 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:55:03.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:55:03 smithi157 ceph-mon[34294]: pgmap v632: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 957 KiB/s rd, 838 MiB/s wr, 1.56k op/s 2024-05-09T00:55:03.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:55:03 smithi005 ceph-mon[30365]: pgmap v632: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 957 KiB/s rd, 838 MiB/s wr, 1.56k op/s 2024-05-09T00:55:03.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:55:03 smithi131 ceph-mon[34547]: pgmap v632: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 957 KiB/s rd, 838 MiB/s wr, 1.56k op/s 2024-05-09T00:55:05.657 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:55:05 smithi131 ceph-mon[34547]: pgmap v633: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 867 MiB/s wr, 1.68k op/s 2024-05-09T00:55:05.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:55:05 smithi157 ceph-mon[34294]: pgmap v633: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 867 MiB/s wr, 1.68k op/s 2024-05-09T00:55:05.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:55:05 smithi005 ceph-mon[30365]: pgmap v633: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 867 MiB/s wr, 1.68k op/s 2024-05-09T00:55:07.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:55:07 smithi157 ceph-mon[34294]: pgmap v634: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 854 MiB/s wr, 1.77k op/s 2024-05-09T00:55:07.761 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:55:07 smithi005 ceph-mon[30365]: pgmap v634: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 854 MiB/s wr, 1.77k op/s 2024-05-09T00:55:07.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:55:07 smithi131 ceph-mon[34547]: pgmap v634: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 854 MiB/s wr, 1.77k op/s 2024-05-09T00:55:08.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.027+0000 7fccf65aa700 1 -- 172.21.15.5:0/4176027204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccf0110fc0 msgr2=0x7fccf010ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:55:08.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.027+0000 7fccf65aa700 1 --2- 172.21.15.5:0/4176027204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccf0110fc0 0x7fccf010ad80 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fccec009990 tx=0x7fccec009ca0 comp rx=0 tx=0).stop 2024-05-09T00:55:08.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.031+0000 7fccf65aa700 1 -- 172.21.15.5:0/4176027204 shutdown_connections 2024-05-09T00:55:08.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.031+0000 7fccf65aa700 1 --2- 172.21.15.5:0/4176027204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccf0110fc0 0x7fccf010ad80 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:08.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.031+0000 7fccf65aa700 1 --2- 172.21.15.5:0/4176027204 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccf0112740 0x7fccf0110500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:08.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.031+0000 7fccf65aa700 1 --2- 172.21.15.5:0/4176027204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccf0110ad0 0x7fccf010a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:08.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.031+0000 7fccf65aa700 1 -- 172.21.15.5:0/4176027204 >> 172.21.15.5:0/4176027204 conn(0x7fccf006f260 msgr2=0x7fccf006f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:55:08.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.033+0000 7fccf65aa700 1 -- 172.21.15.5:0/4176027204 shutdown_connections 2024-05-09T00:55:08.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.033+0000 7fccf65aa700 1 -- 172.21.15.5:0/4176027204 wait complete. 2024-05-09T00:55:08.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.034+0000 7fccf65aa700 1 Processor -- start 2024-05-09T00:55:08.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.035+0000 7fccf65aa700 1 -- start start 2024-05-09T00:55:08.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.036+0000 7fccf65aa700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccf0110ad0 0x7fccf01af5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:08.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.036+0000 7fccf65aa700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccf0110fc0 0x7fccf01afb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:08.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.036+0000 7fccf65aa700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccf0112740 0x7fccf01a97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:08.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.036+0000 7fccf65aa700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fccf0123140 con 0x7fccf0112740 2024-05-09T00:55:08.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.036+0000 7fccf65aa700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fccf0122fc0 con 0x7fccf0110fc0 2024-05-09T00:55:08.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.036+0000 7fccf65aa700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fccf01232c0 con 0x7fccf0110ad0 2024-05-09T00:55:08.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.038+0000 7fccf55a8700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccf0110ad0 0x7fccf01af5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:08.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.038+0000 7fccf4da7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccf0110fc0 0x7fccf01afb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:08.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.038+0000 7fccf4da7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccf0110fc0 0x7fccf01afb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:53564/0 (socket says 172.21.15.5:53564) 2024-05-09T00:55:08.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.038+0000 7fccf4da7700 1 -- 172.21.15.5:0/1946435981 learned_addr learned my addr 172.21.15.5:0/1946435981 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:55:08.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.039+0000 7fccf4da7700 1 -- 172.21.15.5:0/1946435981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccf0110ad0 msgr2=0x7fccf01af5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:55:08.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.039+0000 7fccf4da7700 1 --2- 172.21.15.5:0/1946435981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccf0110ad0 0x7fccf01af5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:08.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.039+0000 7fccf4da7700 1 -- 172.21.15.5:0/1946435981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccf0112740 msgr2=0x7fccf01a97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:55:08.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.039+0000 7fccf4da7700 1 --2- 172.21.15.5:0/1946435981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccf0112740 0x7fccf01a97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:08.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.039+0000 7fccf4da7700 1 -- 172.21.15.5:0/1946435981 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccf01a9f10 con 0x7fccf0110fc0 2024-05-09T00:55:08.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.043+0000 7fccf4da7700 1 --2- 172.21.15.5:0/1946435981 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccf0110fc0 0x7fccf01afb00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcce000c8f0 tx=0x7fcce000cc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:55:08.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.045+0000 7fcce67fc700 1 -- 172.21.15.5:0/1946435981 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce00078c0 con 0x7fccf0110fc0 2024-05-09T00:55:08.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.045+0000 7fcce67fc700 1 -- 172.21.15.5:0/1946435981 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcce000f450 con 0x7fccf0110fc0 2024-05-09T00:55:08.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.045+0000 7fcce67fc700 1 -- 172.21.15.5:0/1946435981 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce0018640 con 0x7fccf0110fc0 2024-05-09T00:55:08.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.046+0000 7fccf65aa700 1 -- 172.21.15.5:0/1946435981 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccf01aa200 con 0x7fccf0110fc0 2024-05-09T00:55:08.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.046+0000 7fccf65aa700 1 -- 172.21.15.5:0/1946435981 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccf01b6400 con 0x7fccf0110fc0 2024-05-09T00:55:08.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.052+0000 7fcd06222b80 1 Processor -- start 2024-05-09T00:55:08.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.052+0000 7fcd06222b80 1 -- start start 2024-05-09T00:55:08.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.053+0000 7fcd06222b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55673e424d00 0x55673e5276e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:08.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.053+0000 7fcd06222b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55673e3aa4a0 0x55673e527c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:08.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.053+0000 7fcd06222b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55673e528200 0x55673e544560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:08.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.053+0000 7fcd06222b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55673e44a0a0 con 0x55673e3aa4a0 2024-05-09T00:55:08.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.053+0000 7fcd06222b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55673e3ef9a0 con 0x55673e528200 2024-05-09T00:55:08.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.053+0000 7fcd06222b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55673e446060 con 0x55673e424d00 2024-05-09T00:55:08.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.054+0000 7fccf5da9700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55673e424d00 0x55673e5276e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:08.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.054+0000 7fccf5da9700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55673e424d00 0x55673e5276e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:33434/0 (socket says 172.21.15.5:33434) 2024-05-09T00:55:08.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.054+0000 7fccf5da9700 1 -- 172.21.15.5:0/102506197 learned_addr learned my addr 172.21.15.5:0/102506197 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:55:08.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.069+0000 7fccf5da9700 1 -- 172.21.15.5:0/102506197 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55673e528200 msgr2=0x55673e544560 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:55:08.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.069+0000 7fccf5da9700 1 --2- 172.21.15.5:0/102506197 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55673e528200 0x55673e544560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:08.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.069+0000 7fccf5da9700 1 -- 172.21.15.5:0/102506197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55673e3aa4a0 msgr2=0x55673e527c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:55:08.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.069+0000 7fccf5da9700 1 --2- 172.21.15.5:0/102506197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55673e3aa4a0 0x55673e527c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:08.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.069+0000 7fccf5da9700 1 -- 172.21.15.5:0/102506197 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55673e3e0500 con 0x55673e424d00 2024-05-09T00:55:08.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.070+0000 7fcce67fc700 1 -- 172.21.15.5:0/1946435981 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fcce000f900 con 0x7fccf0110fc0 2024-05-09T00:55:08.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.072+0000 7fcce67fc700 1 --2- 172.21.15.5:0/1946435981 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fccdc0b29b0 0x7fccdc0b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:08.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.072+0000 7fcce67fc700 1 -- 172.21.15.5:0/1946435981 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fcce00e2050 con 0x7fccf0110fc0 2024-05-09T00:55:08.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.075+0000 7fccf55a8700 1 --2- 172.21.15.5:0/1946435981 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fccdc0b29b0 0x7fccdc0b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:08.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.082+0000 7fccf5da9700 1 --2- 172.21.15.5:0/102506197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55673e424d00 0x55673e5276e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fccec0098e0 tx=0x7fccec003740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4350ee3d5f4f6047 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:55:08.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.082+0000 7fccdb7fe700 1 -- 172.21.15.5:0/102506197 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccec003910 con 0x55673e424d00 2024-05-09T00:55:08.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.083+0000 7fccdb7fe700 1 -- 172.21.15.5:0/102506197 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccec003ab0 con 0x55673e424d00 2024-05-09T00:55:08.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.084+0000 7fcd06222b80 1 -- 172.21.15.5:0/102506197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55673e424d00 msgr2=0x55673e5276e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:55:08.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.084+0000 7fcd06222b80 1 --2- 172.21.15.5:0/102506197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55673e424d00 0x55673e5276e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fccec0098e0 tx=0x7fccec003740 comp rx=0 tx=0).stop 2024-05-09T00:55:08.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.085+0000 7fcd06222b80 1 -- 172.21.15.5:0/102506197 shutdown_connections 2024-05-09T00:55:08.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.085+0000 7fcd06222b80 1 --2- 172.21.15.5:0/102506197 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55673e528200 0x55673e544560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:08.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.085+0000 7fcd06222b80 1 --2- 172.21.15.5:0/102506197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55673e424d00 0x55673e5276e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:08.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.085+0000 7fcd06222b80 1 --2- 172.21.15.5:0/102506197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55673e3aa4a0 0x55673e527c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:08.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.085+0000 7fcd06222b80 1 -- 172.21.15.5:0/102506197 >> 172.21.15.5:0/102506197 conn(0x55673e3adf70 msgr2=0x55673e522760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:55:08.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.085+0000 7fcd06222b80 1 -- 172.21.15.5:0/102506197 shutdown_connections 2024-05-09T00:55:08.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.085+0000 7fccf55a8700 1 --2- 172.21.15.5:0/1946435981 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fccdc0b29b0 0x7fccdc0b4ec0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fcce8007fd0 tx=0x7fcce800c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:55:08.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.085+0000 7fcd06222b80 1 -- 172.21.15.5:0/102506197 wait complete. 2024-05-09T00:55:08.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.086+0000 7fcd06222b80 1 -- start start 2024-05-09T00:55:08.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.086+0000 7fcd06222b80 1 Processor -- start 2024-05-09T00:55:08.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.087+0000 7fcd06222b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55673e3aa4a0 0x55673e5444d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:08.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.087+0000 7fcd06222b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55673e528200 0x55673e541710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:08.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.087+0000 7fcd06222b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55673e541c50 0x55673e542080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:08.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.087+0000 7fcd06222b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55673e544f90 con 0x55673e541c50 2024-05-09T00:55:08.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.087+0000 7fcd06222b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55673e3e4b20 con 0x55673e3aa4a0 2024-05-09T00:55:08.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.087+0000 7fcd06222b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55673e545110 con 0x55673e528200 2024-05-09T00:55:08.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.087+0000 7fcd06222b80 10 client.0 _ll_register_callbacks cb 0x55673e2bb6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:55:08.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.092+0000 7fccf55a8700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55673e3aa4a0 0x55673e5444d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:08.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.093+0000 7fccf4da7700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55673e528200 0x55673e541710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:08.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.093+0000 7fccf55a8700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55673e3aa4a0 0x55673e5444d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:53596/0 (socket says 172.21.15.5:53596) 2024-05-09T00:55:08.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.093+0000 7fccf55a8700 1 -- 172.21.15.5:0/3124844969 learned_addr learned my addr 172.21.15.5:0/3124844969 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:55:08.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.094+0000 7fccf5da9700 1 --2- 172.21.15.5:0/3124844969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55673e541c50 0x55673e542080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:08.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.094+0000 7fccf55a8700 1 -- 172.21.15.5:0/3124844969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55673e528200 msgr2=0x55673e541710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:55:08.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.094+0000 7fccf55a8700 1 --2- 172.21.15.5:0/3124844969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55673e528200 0x55673e541710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:08.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.094+0000 7fccf55a8700 1 -- 172.21.15.5:0/3124844969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55673e541c50 msgr2=0x55673e542080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:55:08.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.094+0000 7fccf55a8700 1 --2- 172.21.15.5:0/3124844969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55673e541c50 0x55673e542080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:08.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.094+0000 7fccf55a8700 1 -- 172.21.15.5:0/3124844969 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55673e2c5490 con 0x55673e3aa4a0 2024-05-09T00:55:08.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.096+0000 7fccf55a8700 1 --2- 172.21.15.5:0/3124844969 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55673e3aa4a0 0x55673e5444d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fcce8004b10 tx=0x7fcce8004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76daa6463e4be60a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:55:08.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.096+0000 7fccd8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-09T00:55:08.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.097+0000 7fccd8ff9700 1 -- 172.21.15.5:0/3124844969 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce8024030 con 0x55673e3aa4a0 2024-05-09T00:55:08.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.097+0000 7fcd06222b80 1 -- 172.21.15.5:0/3124844969 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55673e457a80 con 0x55673e3aa4a0 2024-05-09T00:55:08.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.097+0000 7fccd8ff9700 1 -- 172.21.15.5:0/3124844969 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fcce8014be0 con 0x55673e3aa4a0 2024-05-09T00:55:08.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.098+0000 7fccd8ff9700 1 -- 172.21.15.5:0/3124844969 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fcce801c040 con 0x55673e3aa4a0 2024-05-09T00:55:08.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.099+0000 7fcd06222b80 10 client.25895 fetch_fsmap learned FSMap version 31 2024-05-09T00:55:08.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.099+0000 7fcd06222b80 1 -- 172.21.15.5:0/3124844969 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55673e54fe40 con 0x55673e3aa4a0 2024-05-09T00:55:08.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.102+0000 7fccd8ff9700 1 -- 172.21.15.5:0/3124844969 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55673e457a80 con 0x55673e3aa4a0 2024-05-09T00:55:08.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.103+0000 7fccd8ff9700 1 -- 172.21.15.5:0/3124844969 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fcce801f030 con 0x55673e3aa4a0 2024-05-09T00:55:08.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.104+0000 7fcd06222b80 10 client.25895 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:55:08.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.104+0000 7fcd06222b80 10 client.25895 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-05-09T00:55:08.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.104+0000 7fcd06222b80 20 client.25895 populate_metadata read hostname 'smithi005' 2024-05-09T00:55:08.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.104+0000 7fcd06222b80 1 --2- 172.21.15.5:0/3124844969 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x55673e552800 0x55673e554c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:08.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.104+0000 7fcd06222b80 4 client.25895 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:55:08.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.104+0000 7fcd06222b80 1 -- 172.21.15.5:0/3124844969 --> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55673e451b20 con 0x55673e552800 2024-05-09T00:55:08.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.107+0000 7fccf4da7700 1 --2- 172.21.15.5:0/3124844969 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x55673e552800 0x55673e554c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:08.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.121+0000 7fccf4da7700 1 --2- 172.21.15.5:0/3124844969 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x55673e552800 0x55673e554c90 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d49c1522d3f26fb1 server_cookie=7773ebde5936cc97 in_seq=0 out_seq=0 2024-05-09T00:55:08.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.121+0000 7fccd8ff9700 10 client.25895 ms_handle_connect on v2:172.21.15.157:6836/744863009 2024-05-09T00:55:08.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.126+0000 7fccd8ff9700 1 -- 172.21.15.5:0/3124844969 <== mds.1 v2:172.21.15.157:6836/744863009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55673e451b20 con 0x55673e552800 2024-05-09T00:55:08.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.126+0000 7fccd8ff9700 10 client.25895 handle_command_reply: tid=0 2024-05-09T00:55:08.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.126+0000 7fcd06222b80 1 client.25895 shutdown 2024-05-09T00:55:08.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.127+0000 7fcd06222b80 1 -- 172.21.15.5:0/3124844969 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55673e3aa4a0 msgr2=0x55673e5444d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:55:08.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.127+0000 7fcd06222b80 1 --2- 172.21.15.5:0/3124844969 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55673e3aa4a0 0x55673e5444d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fcce8004b10 tx=0x7fcce8004b40 comp rx=0 tx=0).stop 2024-05-09T00:55:08.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.127+0000 7fcd06222b80 1 -- 172.21.15.5:0/3124844969 shutdown_connections 2024-05-09T00:55:08.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.127+0000 7fcd06222b80 1 --2- 172.21.15.5:0/3124844969 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x55673e552800 0x55673e554c90 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:08.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.127+0000 7fcd06222b80 1 --2- 172.21.15.5:0/3124844969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55673e541c50 0x55673e542080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:08.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.127+0000 7fcd06222b80 1 --2- 172.21.15.5:0/3124844969 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55673e3aa4a0 0x55673e5444d0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:08.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.127+0000 7fcd06222b80 1 --2- 172.21.15.5:0/3124844969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55673e528200 0x55673e541710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:08.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.127+0000 7fcd06222b80 1 -- 172.21.15.5:0/3124844969 >> 172.21.15.5:0/3124844969 conn(0x55673e3adf70 msgr2=0x55673e5222b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:55:08.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.127+0000 7fccd8ff9700 0 client.25895 ms_handle_reset on v2:172.21.15.157:6836/744863009 2024-05-09T00:55:08.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.132+0000 7fcd06222b80 1 -- 172.21.15.5:0/3124844969 shutdown_connections 2024-05-09T00:55:08.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.132+0000 7fcd06222b80 1 -- 172.21.15.5:0/3124844969 wait complete. 2024-05-09T00:55:08.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.132+0000 7fcd06222b80 20 client.25895 trim_cache size 0 max 16384 2024-05-09T00:55:08.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.178+0000 7fcd06222b80 1 Processor -- start 2024-05-09T00:55:08.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.178+0000 7fcd06222b80 1 -- start start 2024-05-09T00:55:08.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.179+0000 7fcd06222b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55673e3aa4a0 0x55673e54e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:08.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.179+0000 7fcd06222b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55673e528200 0x55673e54e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:08.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.179+0000 7fcd06222b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55673e541c50 0x55673e55a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:08.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.179+0000 7fcd06222b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fccf011af30 con 0x55673e528200 2024-05-09T00:55:08.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.179+0000 7fcd06222b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fccf01b6a70 con 0x55673e541c50 2024-05-09T00:55:08.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.180+0000 7fcd06222b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fccf011b600 con 0x55673e3aa4a0 2024-05-09T00:55:08.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.180+0000 7fccf5da9700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55673e541c50 0x55673e55a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:08.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.180+0000 7fccf5da9700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55673e541c50 0x55673e55a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:53610/0 (socket says 172.21.15.5:53610) 2024-05-09T00:55:08.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.180+0000 7fccf5da9700 1 -- 172.21.15.5:0/642344733 learned_addr learned my addr 172.21.15.5:0/642344733 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:55:08.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.181+0000 7fccf4da7700 1 --2- 172.21.15.5:0/642344733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55673e528200 0x55673e54e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:08.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.182+0000 7fccf55a8700 1 --2- 172.21.15.5:0/642344733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55673e3aa4a0 0x55673e54e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:08.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.182+0000 7fccf5da9700 1 -- 172.21.15.5:0/642344733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55673e3aa4a0 msgr2=0x55673e54e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:55:08.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.182+0000 7fccf5da9700 1 --2- 172.21.15.5:0/642344733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55673e3aa4a0 0x55673e54e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:08.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.182+0000 7fccf5da9700 1 -- 172.21.15.5:0/642344733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55673e528200 msgr2=0x55673e54e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:55:08.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.182+0000 7fccf5da9700 1 --2- 172.21.15.5:0/642344733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55673e528200 0x55673e54e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:08.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.182+0000 7fccf5da9700 1 -- 172.21.15.5:0/642344733 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55673e4ecca0 con 0x55673e541c50 2024-05-09T00:55:08.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.183+0000 7fccf5da9700 1 --2- 172.21.15.5:0/642344733 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55673e541c50 0x55673e55a490 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fccec0098e0 tx=0x7fccec005040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a53162f805f0479 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:55:08.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.188+0000 7fccdaffd700 1 -- 172.21.15.5:0/642344733 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccec00b7d0 con 0x55673e541c50 2024-05-09T00:55:08.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.188+0000 7fccdaffd700 1 -- 172.21.15.5:0/642344733 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccec005560 con 0x55673e541c50 2024-05-09T00:55:08.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.190+0000 7fcd06222b80 1 -- 172.21.15.5:0/642344733 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55673e541c50 msgr2=0x55673e55a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:55:08.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.190+0000 7fcd06222b80 1 --2- 172.21.15.5:0/642344733 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55673e541c50 0x55673e55a490 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fccec0098e0 tx=0x7fccec005040 comp rx=0 tx=0).stop 2024-05-09T00:55:08.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.190+0000 7fcd06222b80 1 -- 172.21.15.5:0/642344733 shutdown_connections 2024-05-09T00:55:08.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.190+0000 7fcd06222b80 1 --2- 172.21.15.5:0/642344733 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55673e541c50 0x55673e55a490 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:08.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.190+0000 7fcd06222b80 1 --2- 172.21.15.5:0/642344733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55673e3aa4a0 0x55673e54e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:08.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.190+0000 7fcd06222b80 1 --2- 172.21.15.5:0/642344733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55673e528200 0x55673e54e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:08.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.188+0000 7fccdaffd700 1 -- 172.21.15.5:0/642344733 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccec00f600 con 0x55673e541c50 2024-05-09T00:55:08.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.190+0000 7fcd06222b80 1 -- 172.21.15.5:0/642344733 >> 172.21.15.5:0/642344733 conn(0x55673e3adf70 msgr2=0x55673e540000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:55:08.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.198+0000 7fcd06222b80 1 -- 172.21.15.5:0/642344733 shutdown_connections 2024-05-09T00:55:08.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.198+0000 7fcd06222b80 1 -- 172.21.15.5:0/642344733 wait complete. 2024-05-09T00:55:08.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.199+0000 7fcd06222b80 1 -- start start 2024-05-09T00:55:08.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.199+0000 7fcd06222b80 1 Processor -- start 2024-05-09T00:55:08.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.200+0000 7fcd06222b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55673e3aa4a0 0x55673e54aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:08.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.201+0000 7fcd06222b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55673e528200 0x55673e5453d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:08.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.201+0000 7fcd06222b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55673e545910 0x55673e545d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:08.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.201+0000 7fcd06222b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fccf011c3a0 con 0x55673e528200 2024-05-09T00:55:08.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.201+0000 7fcd06222b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fccf011bcd0 con 0x55673e545910 2024-05-09T00:55:08.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.201+0000 7fcd06222b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fccf010be50 con 0x55673e3aa4a0 2024-05-09T00:55:08.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.201+0000 7fccf55a8700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55673e3aa4a0 0x55673e54aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:08.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.201+0000 7fccf55a8700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55673e3aa4a0 0x55673e54aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:33464/0 (socket says 172.21.15.5:33464) 2024-05-09T00:55:08.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.201+0000 7fccf55a8700 1 -- 172.21.15.5:0/869979263 learned_addr learned my addr 172.21.15.5:0/869979263 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:55:08.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.202+0000 7fcd06222b80 10 client.0 _ll_register_callbacks cb 0x55673e2bb6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:55:08.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.202+0000 7fccf5da9700 1 --2- 172.21.15.5:0/869979263 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55673e545910 0x55673e545d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:08.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.202+0000 7fccf55a8700 1 -- 172.21.15.5:0/869979263 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55673e545910 msgr2=0x55673e545d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:55:08.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.202+0000 7fccf55a8700 1 --2- 172.21.15.5:0/869979263 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55673e545910 0x55673e545d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:08.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.202+0000 7fccf55a8700 1 -- 172.21.15.5:0/869979263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55673e528200 msgr2=0x55673e5453d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:55:08.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.202+0000 7fccf55a8700 1 --2- 172.21.15.5:0/869979263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55673e528200 0x55673e5453d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:08.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.202+0000 7fccf55a8700 1 -- 172.21.15.5:0/869979263 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55673e528030 con 0x55673e3aa4a0 2024-05-09T00:55:08.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.207+0000 7fccf55a8700 1 --2- 172.21.15.5:0/869979263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55673e3aa4a0 0x55673e54aa50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcce800afd0 tx=0x7fcce8005f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=882844c976d955f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:55:08.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.207+0000 7fccd8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-09T00:55:08.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.209+0000 7fccd8ff9700 1 -- 172.21.15.5:0/869979263 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce8038020 con 0x55673e3aa4a0 2024-05-09T00:55:08.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.209+0000 7fcd06222b80 1 -- 172.21.15.5:0/869979263 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccec005560 con 0x55673e3aa4a0 2024-05-09T00:55:08.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.209+0000 7fccd8ff9700 1 -- 172.21.15.5:0/869979263 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fcce8020b00 con 0x55673e3aa4a0 2024-05-09T00:55:08.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.210+0000 7fccd8ff9700 1 -- 172.21.15.5:0/869979263 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fcce8043050 con 0x55673e3aa4a0 2024-05-09T00:55:08.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.212+0000 7fcd06222b80 10 client.25924 fetch_fsmap learned FSMap version 31 2024-05-09T00:55:08.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.212+0000 7fcd06222b80 1 -- 172.21.15.5:0/869979263 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55673e5bc190 con 0x55673e3aa4a0 2024-05-09T00:55:08.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.213+0000 7fccd8ff9700 1 -- 172.21.15.5:0/869979263 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccec005560 con 0x55673e3aa4a0 2024-05-09T00:55:08.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.213+0000 7fccd8ff9700 1 -- 172.21.15.5:0/869979263 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fcce801f040 con 0x55673e3aa4a0 2024-05-09T00:55:08.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.213+0000 7fcd06222b80 10 client.25924 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:55:08.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.213+0000 7fcd06222b80 10 client.25924 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-05-09T00:55:08.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.213+0000 7fcd06222b80 20 client.25924 populate_metadata read hostname 'smithi005' 2024-05-09T00:55:08.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.213+0000 7fcd06222b80 1 --2- 172.21.15.5:0/869979263 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x55673e5bc840 0x55673e5bec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:08.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.213+0000 7fcd06222b80 4 client.25924 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:55:08.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.213+0000 7fcd06222b80 1 -- 172.21.15.5:0/869979263 --> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55673e2c5970 con 0x55673e5bc840 2024-05-09T00:55:08.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.216+0000 7fccf4da7700 1 --2- 172.21.15.5:0/869979263 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x55673e5bc840 0x55673e5bec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:08.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.231+0000 7fccd8ff9700 10 client.25924 ms_handle_connect on v2:172.21.15.157:6836/744863009 2024-05-09T00:55:08.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.231+0000 7fccf4da7700 1 --2- 172.21.15.5:0/869979263 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x55673e5bc840 0x55673e5bec60 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=56c2f504d7bdc20a server_cookie=3bd777aa8e859777 in_seq=0 out_seq=0 2024-05-09T00:55:08.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.233+0000 7fccd8ff9700 1 -- 172.21.15.5:0/869979263 <== mds.1 v2:172.21.15.157:6836/744863009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55673e2c5970 con 0x55673e5bc840 2024-05-09T00:55:08.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.233+0000 7fccd8ff9700 10 client.25924 handle_command_reply: tid=0 2024-05-09T00:55:08.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.233+0000 7fcd06222b80 1 client.25924 shutdown 2024-05-09T00:55:08.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.234+0000 7fcd06222b80 1 -- 172.21.15.5:0/869979263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55673e3aa4a0 msgr2=0x55673e54aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:55:08.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.234+0000 7fcd06222b80 1 --2- 172.21.15.5:0/869979263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55673e3aa4a0 0x55673e54aa50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcce800afd0 tx=0x7fcce8005f40 comp rx=0 tx=0).stop 2024-05-09T00:55:08.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.234+0000 7fcd06222b80 1 -- 172.21.15.5:0/869979263 shutdown_connections 2024-05-09T00:55:08.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.234+0000 7fcd06222b80 1 --2- 172.21.15.5:0/869979263 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x55673e5bc840 0x55673e5bec60 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:08.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.234+0000 7fcd06222b80 1 --2- 172.21.15.5:0/869979263 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55673e545910 0x55673e545d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:08.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.234+0000 7fcd06222b80 1 --2- 172.21.15.5:0/869979263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55673e3aa4a0 0x55673e54aa50 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:08.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.234+0000 7fcd06222b80 1 --2- 172.21.15.5:0/869979263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55673e528200 0x55673e5453d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:08.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.234+0000 7fcd06222b80 1 -- 172.21.15.5:0/869979263 >> 172.21.15.5:0/869979263 conn(0x55673e3adf70 msgr2=0x55673e53c500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:55:08.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.234+0000 7fccd8ff9700 0 client.25924 ms_handle_reset on v2:172.21.15.157:6836/744863009 2024-05-09T00:55:08.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.234+0000 7fcd06222b80 1 -- 172.21.15.5:0/869979263 shutdown_connections 2024-05-09T00:55:08.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.234+0000 7fcd06222b80 1 -- 172.21.15.5:0/869979263 wait complete. 2024-05-09T00:55:08.236 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-09T00:55:08.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.235+0000 7fcd06222b80 20 client.25924 trim_cache size 0 max 16384 2024-05-09T00:55:08.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.236+0000 7fccf65aa700 1 -- 172.21.15.5:0/1946435981 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fccdc0b29b0 msgr2=0x7fccdc0b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:55:08.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.236+0000 7fccf65aa700 1 --2- 172.21.15.5:0/1946435981 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fccdc0b29b0 0x7fccdc0b4ec0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fcce8007fd0 tx=0x7fcce800c040 comp rx=0 tx=0).stop 2024-05-09T00:55:08.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.236+0000 7fccf65aa700 1 -- 172.21.15.5:0/1946435981 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccf0110fc0 msgr2=0x7fccf01afb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:55:08.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.236+0000 7fccf65aa700 1 --2- 172.21.15.5:0/1946435981 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccf0110fc0 0x7fccf01afb00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcce000c8f0 tx=0x7fcce000cc00 comp rx=0 tx=0).stop 2024-05-09T00:55:08.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.236+0000 7fccf65aa700 1 -- 172.21.15.5:0/1946435981 shutdown_connections 2024-05-09T00:55:08.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.236+0000 7fccf65aa700 1 --2- 172.21.15.5:0/1946435981 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fccdc0b29b0 0x7fccdc0b4ec0 secure :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fcce8007fd0 tx=0x7fcce800c040 comp rx=0 tx=0).stop 2024-05-09T00:55:08.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.236+0000 7fccf65aa700 1 --2- 172.21.15.5:0/1946435981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccf0112740 0x7fccf01a97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:08.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.236+0000 7fccf65aa700 1 --2- 172.21.15.5:0/1946435981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccf0110ad0 0x7fccf01af5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:08.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.236+0000 7fccf65aa700 1 --2- 172.21.15.5:0/1946435981 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccf0110fc0 0x7fccf01afb00 secure :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcce000c8f0 tx=0x7fcce000cc00 comp rx=0 tx=0).stop 2024-05-09T00:55:08.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.236+0000 7fccf65aa700 1 -- 172.21.15.5:0/1946435981 >> 172.21.15.5:0/1946435981 conn(0x7fccf006f260 msgr2=0x7fccf00744f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:55:08.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.238+0000 7fccf65aa700 1 -- 172.21.15.5:0/1946435981 shutdown_connections 2024-05-09T00:55:08.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:08.238+0000 7fccf65aa700 1 -- 172.21.15.5:0/1946435981 wait complete. 2024-05-09T00:55:08.412 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.411+0000 7f819030d640 1 -- 172.21.15.5:0/91167319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f818810a670 msgr2=0x7f818810b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:55:08.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.411+0000 7f819030d640 1 --2- 172.21.15.5:0/91167319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f818810a670 0x7f818810b370 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8180009ca0 tx=0x7f818002a8f0 comp rx=0 tx=0).stop 2024-05-09T00:55:08.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.413+0000 7f819030d640 1 -- 172.21.15.5:0/91167319 shutdown_connections 2024-05-09T00:55:08.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.413+0000 7f819030d640 1 --2- 172.21.15.5:0/91167319 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f818806bfb0 0x7f8188112e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:08.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.413+0000 7f819030d640 1 --2- 172.21.15.5:0/91167319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f818810b8b0 0x7f818810bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:08.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.413+0000 7f819030d640 1 --2- 172.21.15.5:0/91167319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f818810a670 0x7f818810b370 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:08.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.413+0000 7f819030d640 1 -- 172.21.15.5:0/91167319 >> 172.21.15.5:0/91167319 conn(0x7f8188070a90 msgr2=0x7f8188072f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:55:08.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.414+0000 7f819030d640 1 -- 172.21.15.5:0/91167319 shutdown_connections 2024-05-09T00:55:08.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.415+0000 7f819030d640 1 -- 172.21.15.5:0/91167319 wait complete. 2024-05-09T00:55:08.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.417+0000 7f819030d640 1 Processor -- start 2024-05-09T00:55:08.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.417+0000 7f819030d640 1 -- start start 2024-05-09T00:55:08.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.417+0000 7f819030d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f818806bfb0 0x7f818811bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:08.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.417+0000 7f819030d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f818810a670 0x7f818811a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:08.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.417+0000 7f819030d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f818810b8b0 0x7f818811ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:08.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.417+0000 7f819030d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8188114a00 con 0x7f818810b8b0 2024-05-09T00:55:08.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.417+0000 7f819030d640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8188114880 con 0x7f818806bfb0 2024-05-09T00:55:08.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.417+0000 7f819030d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8188114b80 con 0x7f818810a670 2024-05-09T00:55:08.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.418+0000 7f818e8aa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f818810b8b0 0x7f818811ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:08.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.418+0000 7f818e8aa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f818810b8b0 0x7f818811ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47168/0 (socket says 172.21.15.5:47168) 2024-05-09T00:55:08.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.418+0000 7f818e8aa640 1 -- 172.21.15.5:0/4156329476 learned_addr learned my addr 172.21.15.5:0/4156329476 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:55:08.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.418+0000 7f818e0a9640 1 --2- 172.21.15.5:0/4156329476 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f818806bfb0 0x7f818811bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:08.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.419+0000 7f818d8a8640 1 --2- 172.21.15.5:0/4156329476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f818810a670 0x7f818811a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:08.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.420+0000 7f818e8aa640 1 -- 172.21.15.5:0/4156329476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f818810a670 msgr2=0x7f818811a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:55:08.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.420+0000 7f818e8aa640 1 --2- 172.21.15.5:0/4156329476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f818810a670 0x7f818811a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:08.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.420+0000 7f818e8aa640 1 -- 172.21.15.5:0/4156329476 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f818806bfb0 msgr2=0x7f818811bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:55:08.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.420+0000 7f818e8aa640 1 --2- 172.21.15.5:0/4156329476 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f818806bfb0 0x7f818811bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:08.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.420+0000 7f818e8aa640 1 -- 172.21.15.5:0/4156329476 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f818811b140 con 0x7f818810b8b0 2024-05-09T00:55:08.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.421+0000 7f818e8aa640 1 --2- 172.21.15.5:0/4156329476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f818810b8b0 0x7f818811ac00 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f818400c970 tx=0x7f818400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:55:08.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.421+0000 7f817f7fe640 1 -- 172.21.15.5:0/4156329476 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8184007bb0 con 0x7f818810b8b0 2024-05-09T00:55:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.422+0000 7f817f7fe640 1 -- 172.21.15.5:0/4156329476 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8184007d50 con 0x7f818810b8b0 2024-05-09T00:55:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.422+0000 7f817f7fe640 1 -- 172.21.15.5:0/4156329476 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8184007bb0 con 0x7f818810b8b0 2024-05-09T00:55:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.422+0000 7f819030d640 1 -- 172.21.15.5:0/4156329476 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f818811b430 con 0x7f818810b8b0 2024-05-09T00:55:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.422+0000 7f819030d640 1 -- 172.21.15.5:0/4156329476 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81881b9cc0 con 0x7f818810b8b0 2024-05-09T00:55:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.429+0000 7f81971c4740 1 Processor -- start 2024-05-09T00:55:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.429+0000 7f817f7fe640 1 -- 172.21.15.5:0/4156329476 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f8184004c70 con 0x7f818810b8b0 2024-05-09T00:55:08.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.430+0000 7f81971c4740 1 -- start start 2024-05-09T00:55:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.431+0000 7f81971c4740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56121f0301b0 0x56121f3d4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.431+0000 7f81971c4740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56121f0072f0 0x56121f3d4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.431+0000 7f81971c4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56121f00f4d0 0x56121f3db980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.431+0000 7f81971c4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56121f3387c0 con 0x56121f00f4d0 2024-05-09T00:55:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.431+0000 7f81971c4740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x56121f2d40e0 con 0x56121f0072f0 2024-05-09T00:55:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.431+0000 7f81971c4740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56121f334780 con 0x56121f0301b0 2024-05-09T00:55:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.431+0000 7f817f7fe640 1 --2- 172.21.15.5:0/4156329476 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f81740b2bd0 0x7f81740b50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.431+0000 7f818d8a8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56121f00f4d0 0x56121f3db980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.431+0000 7f818d8a8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56121f00f4d0 0x56121f3db980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47174/0 (socket says 172.21.15.5:47174) 2024-05-09T00:55:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.431+0000 7f818d8a8640 1 -- 172.21.15.5:0/2659863679 learned_addr learned my addr 172.21.15.5:0/2659863679 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:55:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.431+0000 7f817f7fe640 1 -- 172.21.15.5:0/4156329476 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f81840079b0 con 0x7f818810b8b0 2024-05-09T00:55:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.431+0000 7f818d8a8640 1 --2- 172.21.15.5:0/2659863679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56121f0301b0 0x56121f3d4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.432+0000 7f818d8a8640 1 -- 172.21.15.5:0/2659863679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56121f0301b0 msgr2=0x56121f3d4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:55:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.432+0000 7f818d8a8640 1 --2- 172.21.15.5:0/2659863679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56121f0301b0 0x56121f3d4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.432+0000 7f818d8a8640 1 -- 172.21.15.5:0/2659863679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56121f0072f0 msgr2=0x56121f3d4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:55:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.432+0000 7f818e8aa640 1 --2- 172.21.15.5:0/4156329476 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f81740b2bd0 0x7f81740b50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.432+0000 7f818d8a8640 1 --2- 172.21.15.5:0/2659863679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56121f0072f0 0x56121f3d4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.432+0000 7f818d8a8640 1 -- 172.21.15.5:0/2659863679 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56121f2fd7c0 con 0x56121f00f4d0 2024-05-09T00:55:08.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.432+0000 7f818d8a8640 1 --2- 172.21.15.5:0/2659863679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56121f00f4d0 0x56121f3db980 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f817800bfc0 tx=0x7f817800b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=365b2f90dfb3adf8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:55:08.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.432+0000 7f818e8aa640 1 --2- 172.21.15.5:0/4156329476 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f81740b2bd0 0x7f81740b50e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f81840a8bd0 tx=0x7f8184005d80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:55:08.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.432+0000 7f817cbfe640 1 -- 172.21.15.5:0/2659863679 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8178019030 con 0x56121f00f4d0 2024-05-09T00:55:08.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.432+0000 7f817cbfe640 1 -- 172.21.15.5:0/2659863679 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8178011970 con 0x56121f00f4d0 2024-05-09T00:55:08.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.433+0000 7f817cbfe640 1 -- 172.21.15.5:0/2659863679 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f817801d840 con 0x56121f00f4d0 2024-05-09T00:55:08.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.433+0000 7f81971c4740 1 -- 172.21.15.5:0/2659863679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56121f00f4d0 msgr2=0x56121f3db980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:55:08.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.433+0000 7f81971c4740 1 --2- 172.21.15.5:0/2659863679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56121f00f4d0 0x56121f3db980 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f817800bfc0 tx=0x7f817800b670 comp rx=0 tx=0).stop 2024-05-09T00:55:08.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.433+0000 7f81971c4740 1 -- 172.21.15.5:0/2659863679 shutdown_connections 2024-05-09T00:55:08.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.434+0000 7f81971c4740 1 --2- 172.21.15.5:0/2659863679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56121f00f4d0 0x56121f3db980 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:08.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.434+0000 7f81971c4740 1 --2- 172.21.15.5:0/2659863679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56121f0072f0 0x56121f3d4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:08.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.434+0000 7f81971c4740 1 --2- 172.21.15.5:0/2659863679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56121f0301b0 0x56121f3d4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:08.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.434+0000 7f81971c4740 1 -- 172.21.15.5:0/2659863679 >> 172.21.15.5:0/2659863679 conn(0x56121f024300 msgr2=0x56121f3cfe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:55:08.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.434+0000 7f81971c4740 1 -- 172.21.15.5:0/2659863679 shutdown_connections 2024-05-09T00:55:08.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.434+0000 7f81971c4740 1 -- 172.21.15.5:0/2659863679 wait complete. 2024-05-09T00:55:08.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.434+0000 7f81971c4740 1 -- start start 2024-05-09T00:55:08.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.435+0000 7f81971c4740 1 Processor -- start 2024-05-09T00:55:08.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.435+0000 7f81971c4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56121f0072f0 0x56121f3de280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:08.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.435+0000 7f81971c4740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56121f00f4d0 0x56121f3de7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:08.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.435+0000 7f81971c4740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56121f0301b0 0x56121f3e5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:08.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.435+0000 7f818d8a8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56121f0072f0 0x56121f3de280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:08.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.435+0000 7f818d8a8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56121f0072f0 0x56121f3de280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47180/0 (socket says 172.21.15.5:47180) 2024-05-09T00:55:08.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.435+0000 7f818d8a8640 1 -- 172.21.15.5:0/1309123683 learned_addr learned my addr 172.21.15.5:0/1309123683 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:55:08.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.435+0000 7f81971c4740 1 -- 172.21.15.5:0/1309123683 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56121f3dc470 con 0x56121f0072f0 2024-05-09T00:55:08.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.435+0000 7f81971c4740 1 -- 172.21.15.5:0/1309123683 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x56121f2bf9c0 con 0x56121f00f4d0 2024-05-09T00:55:08.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.435+0000 7f81971c4740 1 -- 172.21.15.5:0/1309123683 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56121f3dc5f0 con 0x56121f0301b0 2024-05-09T00:55:08.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.436+0000 7f81971c4740 10 client.0 _ll_register_callbacks cb 0x56121f2ec020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:55:08.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.436+0000 7f818d8a8640 1 -- 172.21.15.5:0/1309123683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56121f0301b0 msgr2=0x56121f3e5720 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:55:08.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.436+0000 7f818d8a8640 1 --2- 172.21.15.5:0/1309123683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56121f0301b0 0x56121f3e5720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:08.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.436+0000 7f818d8a8640 1 -- 172.21.15.5:0/1309123683 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56121f00f4d0 msgr2=0x56121f3de7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:55:08.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.436+0000 7f818d8a8640 1 --2- 172.21.15.5:0/1309123683 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56121f00f4d0 0x56121f3de7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:08.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.436+0000 7f818d8a8640 1 -- 172.21.15.5:0/1309123683 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56121f2e61f0 con 0x56121f0072f0 2024-05-09T00:55:08.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.436+0000 7f818d8a8640 1 --2- 172.21.15.5:0/1309123683 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56121f0072f0 0x56121f3de280 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f81780117c0 tx=0x7f81780117f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61a5dc3099a023ed server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:55:08.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.436+0000 7f8171ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:55:08.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.436+0000 7f8171ffb640 1 -- 172.21.15.5:0/1309123683 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8178019040 con 0x56121f0072f0 2024-05-09T00:55:08.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.437+0000 7f81971c4740 1 -- 172.21.15.5:0/1309123683 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8178011970 con 0x56121f0072f0 2024-05-09T00:55:08.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.437+0000 7f8171ffb640 1 -- 172.21.15.5:0/1309123683 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f8178011d20 con 0x56121f0072f0 2024-05-09T00:55:08.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.437+0000 7f8171ffb640 1 -- 172.21.15.5:0/1309123683 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8178005040 con 0x56121f0072f0 2024-05-09T00:55:08.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.437+0000 7f81971c4740 10 client.16398 fetch_fsmap learned FSMap version 31 2024-05-09T00:55:08.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.437+0000 7f81971c4740 1 -- 172.21.15.5:0/1309123683 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56121f09b790 con 0x56121f0072f0 2024-05-09T00:55:08.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.437+0000 7f8171ffb640 1 -- 172.21.15.5:0/1309123683 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f817802d070 con 0x56121f0072f0 2024-05-09T00:55:08.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.437+0000 7f8171ffb640 1 -- 172.21.15.5:0/1309123683 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f8178030070 con 0x56121f0072f0 2024-05-09T00:55:08.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.438+0000 7f81971c4740 10 client.16398 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:55:08.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.438+0000 7f81971c4740 10 client.16398 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:55:08.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.438+0000 7f81971c4740 20 client.16398 populate_metadata read hostname 'smithi005' 2024-05-09T00:55:08.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.438+0000 7f81971c4740 1 --2- 172.21.15.5:0/1309123683 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56121efeed80 0x56121f3eb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:08.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.438+0000 7f81971c4740 4 client.16398 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:55:08.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.438+0000 7f81971c4740 1 -- 172.21.15.5:0/1309123683 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8178011970 con 0x56121efeed80 2024-05-09T00:55:08.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.439+0000 7f818e0a9640 1 --2- 172.21.15.5:0/1309123683 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56121efeed80 0x56121f3eb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:08.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.443+0000 7f818e0a9640 1 --2- 172.21.15.5:0/1309123683 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56121efeed80 0x56121f3eb760 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df5304ed09886ab8 server_cookie=d6140ddc24e701ac in_seq=0 out_seq=0 2024-05-09T00:55:08.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.443+0000 7f8171ffb640 10 client.16398 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:55:08.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.454+0000 7f8171ffb640 1 -- 172.21.15.5:0/1309123683 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f8178011970 con 0x56121efeed80 2024-05-09T00:55:08.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.454+0000 7f8171ffb640 10 client.16398 handle_command_reply: tid=0 2024-05-09T00:55:08.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.454+0000 7f81971c4740 1 client.16398 shutdown 2024-05-09T00:55:08.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.454+0000 7f81971c4740 1 -- 172.21.15.5:0/1309123683 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56121f0072f0 msgr2=0x56121f3de280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:55:08.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.454+0000 7f81971c4740 1 --2- 172.21.15.5:0/1309123683 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56121f0072f0 0x56121f3de280 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f81780117c0 tx=0x7f81780117f0 comp rx=0 tx=0).stop 2024-05-09T00:55:08.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.455+0000 7f81971c4740 1 -- 172.21.15.5:0/1309123683 shutdown_connections 2024-05-09T00:55:08.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.455+0000 7f81971c4740 1 --2- 172.21.15.5:0/1309123683 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56121efeed80 0x56121f3eb760 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:08.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.455+0000 7f8171ffb640 0 client.16398 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:55:08.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.455+0000 7f81971c4740 1 --2- 172.21.15.5:0/1309123683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56121f0301b0 0x56121f3e5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:08.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.455+0000 7f81971c4740 1 --2- 172.21.15.5:0/1309123683 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56121f00f4d0 0x56121f3de7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:08.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.455+0000 7f81971c4740 1 --2- 172.21.15.5:0/1309123683 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56121f0072f0 0x56121f3de280 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:08.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.455+0000 7f81971c4740 1 -- 172.21.15.5:0/1309123683 >> 172.21.15.5:0/1309123683 conn(0x56121f024300 msgr2=0x56121f3ce700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:55:08.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.455+0000 7f81971c4740 1 -- 172.21.15.5:0/1309123683 shutdown_connections 2024-05-09T00:55:08.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.455+0000 7f81971c4740 1 -- 172.21.15.5:0/1309123683 wait complete. 2024-05-09T00:55:08.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.455+0000 7f81971c4740 20 client.16398 trim_cache size 0 max 16384 2024-05-09T00:55:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.485+0000 7f81971c4740 1 Processor -- start 2024-05-09T00:55:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.486+0000 7f81971c4740 1 -- start start 2024-05-09T00:55:08.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.486+0000 7f81971c4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56121f024300 0x56121f3d8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:08.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.486+0000 7f81971c4740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56121efeed80 0x56121f3d8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:08.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.487+0000 7f818d8a8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56121f024300 0x56121f3d8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:08.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.487+0000 7f818d8a8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56121f024300 0x56121f3d8640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47190/0 (socket says 172.21.15.5:47190) 2024-05-09T00:55:08.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.487+0000 7f818d8a8640 1 -- 172.21.15.5:0/1520180121 learned_addr learned my addr 172.21.15.5:0/1520180121 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:55:08.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.487+0000 7f81971c4740 1 --2- 172.21.15.5:0/1520180121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56121f0072f0 0x56121f3d7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:08.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.487+0000 7f81971c4740 1 -- 172.21.15.5:0/1520180121 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f81881c2aa0 con 0x56121f024300 2024-05-09T00:55:08.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.487+0000 7f81971c4740 1 -- 172.21.15.5:0/1520180121 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f81881c45e0 con 0x56121efeed80 2024-05-09T00:55:08.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.488+0000 7f81971c4740 1 -- 172.21.15.5:0/1520180121 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f81881c3170 con 0x56121f0072f0 2024-05-09T00:55:08.489 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.488+0000 7f818e0a9640 1 --2- 172.21.15.5:0/1520180121 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56121efeed80 0x56121f3d8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:08.489 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.488+0000 7f818d8a8640 1 -- 172.21.15.5:0/1520180121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56121f0072f0 msgr2=0x56121f3d7570 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:55:08.489 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.488+0000 7f818d8a8640 1 --2- 172.21.15.5:0/1520180121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56121f0072f0 0x56121f3d7570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:08.489 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.488+0000 7f818d8a8640 1 -- 172.21.15.5:0/1520180121 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56121efeed80 msgr2=0x56121f3d8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:55:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.488+0000 7f818d8a8640 1 --2- 172.21.15.5:0/1520180121 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56121efeed80 0x56121f3d8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.489+0000 7f818d8a8640 1 -- 172.21.15.5:0/1520180121 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56121f1125f0 con 0x56121f024300 2024-05-09T00:55:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.489+0000 7f818d8a8640 1 --2- 172.21.15.5:0/1520180121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56121f024300 0x56121f3d8640 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f817800bf90 tx=0x7f817800b680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19aaf8afe8ae4edb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:55:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.490+0000 7f8173fff640 1 -- 172.21.15.5:0/1520180121 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8178019040 con 0x56121f024300 2024-05-09T00:55:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.490+0000 7f8173fff640 1 -- 172.21.15.5:0/1520180121 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f817802d070 con 0x56121f024300 2024-05-09T00:55:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.490+0000 7f8173fff640 1 -- 172.21.15.5:0/1520180121 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8178011c50 con 0x56121f024300 2024-05-09T00:55:08.492 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.491+0000 7f81971c4740 1 -- 172.21.15.5:0/1520180121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56121f024300 msgr2=0x56121f3d8640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:55:08.492 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.491+0000 7f81971c4740 1 --2- 172.21.15.5:0/1520180121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56121f024300 0x56121f3d8640 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f817800bf90 tx=0x7f817800b680 comp rx=0 tx=0).stop 2024-05-09T00:55:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.492+0000 7f81971c4740 1 -- 172.21.15.5:0/1520180121 shutdown_connections 2024-05-09T00:55:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.492+0000 7f81971c4740 1 --2- 172.21.15.5:0/1520180121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56121f0072f0 0x56121f3d7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.492+0000 7f81971c4740 1 --2- 172.21.15.5:0/1520180121 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56121efeed80 0x56121f3d8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.492+0000 7f81971c4740 1 --2- 172.21.15.5:0/1520180121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56121f024300 0x56121f3d8640 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.492+0000 7f81971c4740 1 -- 172.21.15.5:0/1520180121 >> 172.21.15.5:0/1520180121 conn(0x56121f3ebad0 msgr2=0x56121f3dc990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:55:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.492+0000 7f81971c4740 1 -- 172.21.15.5:0/1520180121 shutdown_connections 2024-05-09T00:55:08.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.492+0000 7f81971c4740 1 -- 172.21.15.5:0/1520180121 wait complete. 2024-05-09T00:55:08.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.493+0000 7f81971c4740 1 -- start start 2024-05-09T00:55:08.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.493+0000 7f81971c4740 1 Processor -- start 2024-05-09T00:55:08.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.494+0000 7f81971c4740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56121efeed80 0x56121f3e2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:08.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.494+0000 7f81971c4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56121f0072f0 0x56121f3e2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:08.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.494+0000 7f81971c4740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56121f024300 0x56121f3e35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:08.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.494+0000 7f81971c4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f81881c3f10 con 0x56121f0072f0 2024-05-09T00:55:08.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.494+0000 7f81971c4740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f81881c3840 con 0x56121f024300 2024-05-09T00:55:08.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.494+0000 7f81971c4740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f81881c0a80 con 0x56121efeed80 2024-05-09T00:55:08.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.495+0000 7f818e0a9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56121f0072f0 0x56121f3e2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:08.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.495+0000 7f818e0a9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56121f0072f0 0x56121f3e2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47194/0 (socket says 172.21.15.5:47194) 2024-05-09T00:55:08.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.495+0000 7f818e0a9640 1 -- 172.21.15.5:0/788307479 learned_addr learned my addr 172.21.15.5:0/788307479 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:55:08.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.495+0000 7f81971c4740 10 client.0 _ll_register_callbacks cb 0x7f8178013400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:55:08.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.495+0000 7f818e0a9640 1 -- 172.21.15.5:0/788307479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56121efeed80 msgr2=0x56121f3e2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:55:08.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.495+0000 7f818e0a9640 1 --2- 172.21.15.5:0/788307479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56121efeed80 0x56121f3e2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:08.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.495+0000 7f818e0a9640 1 -- 172.21.15.5:0/788307479 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56121f024300 msgr2=0x56121f3e35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:55:08.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.495+0000 7f818e0a9640 1 --2- 172.21.15.5:0/788307479 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56121f024300 0x56121f3e35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:08.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.495+0000 7f818e0a9640 1 -- 172.21.15.5:0/788307479 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56121f2fced0 con 0x56121f0072f0 2024-05-09T00:55:08.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.496+0000 7f818e0a9640 1 --2- 172.21.15.5:0/788307479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56121f0072f0 0x56121f3e2670 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f8180031a50 tx=0x7f8180031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b13526176bf7b2c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:55:08.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.496+0000 7f8171ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:55:08.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.496+0000 7f8171ffb640 1 -- 172.21.15.5:0/788307479 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8180048030 con 0x56121f0072f0 2024-05-09T00:55:08.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.496+0000 7f81971c4740 1 -- 172.21.15.5:0/788307479 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f817802d070 con 0x56121f0072f0 2024-05-09T00:55:08.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.496+0000 7f8171ffb640 1 -- 172.21.15.5:0/788307479 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f81800305f0 con 0x56121f0072f0 2024-05-09T00:55:08.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.496+0000 7f8171ffb640 1 -- 172.21.15.5:0/788307479 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f81800307f0 con 0x56121f0072f0 2024-05-09T00:55:08.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.496+0000 7f81971c4740 10 client.16404 fetch_fsmap learned FSMap version 31 2024-05-09T00:55:08.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.496+0000 7f81971c4740 1 -- 172.21.15.5:0/788307479 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56121f3e4ca0 con 0x56121f0072f0 2024-05-09T00:55:08.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.497+0000 7f8171ffb640 1 -- 172.21.15.5:0/788307479 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f817802d070 con 0x56121f0072f0 2024-05-09T00:55:08.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.497+0000 7f8171ffb640 1 -- 172.21.15.5:0/788307479 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f8180044350 con 0x56121f0072f0 2024-05-09T00:55:08.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.497+0000 7f81971c4740 10 client.16404 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:55:08.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.497+0000 7f81971c4740 10 client.16404 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:55:08.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.497+0000 7f81971c4740 20 client.16404 populate_metadata read hostname 'smithi005' 2024-05-09T00:55:08.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.497+0000 7f81971c4740 1 --2- 172.21.15.5:0/788307479 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56121f00f4d0 0x56121f455e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:08.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.497+0000 7f81971c4740 4 client.16404 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:55:08.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.497+0000 7f81971c4740 1 -- 172.21.15.5:0/788307479 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56121f3463d0 con 0x56121f00f4d0 2024-05-09T00:55:08.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.499+0000 7f818d8a8640 1 --2- 172.21.15.5:0/788307479 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56121f00f4d0 0x56121f455e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:08.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.508+0000 7f818d8a8640 1 --2- 172.21.15.5:0/788307479 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56121f00f4d0 0x56121f455e40 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a93f336843ae6d2 server_cookie=9dec2be000babc48 in_seq=0 out_seq=0 2024-05-09T00:55:08.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:08.509+0000 7f8171ffb640 10 client.16404 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:55:09.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:55:09 smithi005 ceph-mon[30365]: pgmap v635: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 860 MiB/s wr, 1.85k op/s 2024-05-09T00:55:09.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:55:09 smithi157 ceph-mon[34294]: pgmap v635: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 860 MiB/s wr, 1.85k op/s 2024-05-09T00:55:09.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:55:09 smithi131 ceph-mon[34547]: pgmap v635: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 860 MiB/s wr, 1.85k op/s 2024-05-09T00:55:09.906 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T00:55:10.788 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:55:11.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:55:11 smithi157 ceph-mon[34294]: pgmap v636: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 975 MiB/s wr, 1.94k op/s 2024-05-09T00:55:11.794 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:55:11 smithi005 ceph-mon[30365]: pgmap v636: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 975 MiB/s wr, 1.94k op/s 2024-05-09T00:55:11.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:55:11 smithi131 ceph-mon[34547]: pgmap v636: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 975 MiB/s wr, 1.94k op/s 2024-05-09T00:55:13.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:55:13 smithi005 ceph-mon[30365]: pgmap v637: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 802 MiB/s wr, 1.65k op/s 2024-05-09T00:55:13.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:55:13 smithi131 ceph-mon[34547]: pgmap v637: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 802 MiB/s wr, 1.65k op/s 2024-05-09T00:55:13.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:55:13 smithi157 ceph-mon[34294]: pgmap v637: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 802 MiB/s wr, 1.65k op/s 2024-05-09T00:55:14.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:55:14 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:55:14.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:55:14 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:55:14.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:55:14 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:55:15.762 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:55:15 smithi005 ceph-mon[30365]: pgmap v638: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 842 MiB/s wr, 1.77k op/s 2024-05-09T00:55:15.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:55:15 smithi131 ceph-mon[34547]: pgmap v638: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 842 MiB/s wr, 1.77k op/s 2024-05-09T00:55:15.935 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:55:15 smithi157 ceph-mon[34294]: pgmap v638: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 842 MiB/s wr, 1.77k op/s 2024-05-09T00:55:16.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.932+0000 7fd5fcd7f700 1 -- 172.21.15.5:0/2747071561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5f810a930 msgr2=0x7fd5f810ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:55:16.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.932+0000 7fd5fcd7f700 1 --2- 172.21.15.5:0/2747071561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5f810a930 0x7fd5f810ad40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd5f000b0a0 tx=0x7fd5f000b3b0 comp rx=0 tx=0).stop 2024-05-09T00:55:16.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.943+0000 7fd5fcd7f700 1 -- 172.21.15.5:0/2747071561 shutdown_connections 2024-05-09T00:55:16.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.943+0000 7fd5fcd7f700 1 --2- 172.21.15.5:0/2747071561 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5f811d930 0x7fd5f811fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:16.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.943+0000 7fd5fcd7f700 1 --2- 172.21.15.5:0/2747071561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5f810a930 0x7fd5f810ad40 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:16.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.943+0000 7fd5fcd7f700 1 --2- 172.21.15.5:0/2747071561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5f810b280 0x7fd5f811d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:16.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.943+0000 7fd5fcd7f700 1 -- 172.21.15.5:0/2747071561 >> 172.21.15.5:0/2747071561 conn(0x7fd5f806f4d0 msgr2=0x7fd5f806f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:55:16.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.943+0000 7fd5fcd7f700 1 -- 172.21.15.5:0/2747071561 shutdown_connections 2024-05-09T00:55:16.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.943+0000 7fd5fcd7f700 1 -- 172.21.15.5:0/2747071561 wait complete. 2024-05-09T00:55:16.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.945+0000 7fd5fcd7f700 1 Processor -- start 2024-05-09T00:55:16.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.945+0000 7fd5fcd7f700 1 -- start start 2024-05-09T00:55:16.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.945+0000 7fd5fcd7f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5f810a930 0x7fd5f81a9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:16.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.945+0000 7fd5fcd7f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5f810b280 0x7fd5f81aa3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:16.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.945+0000 7fd5fcd7f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5f811d930 0x7fd5f81ae680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:16.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.945+0000 7fd5fcd7f700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f8123420 con 0x7fd5f811d930 2024-05-09T00:55:16.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.945+0000 7fd5fcd7f700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f81232a0 con 0x7fd5f810b280 2024-05-09T00:55:16.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.945+0000 7fd5fcd7f700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f81235a0 con 0x7fd5f810a930 2024-05-09T00:55:16.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.946+0000 7fd5f659c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5f810a930 0x7fd5f81a9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:16.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.946+0000 7fd5f659c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5f810a930 0x7fd5f81a9e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:52756/0 (socket says 172.21.15.5:52756) 2024-05-09T00:55:16.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.946+0000 7fd5f659c700 1 -- 172.21.15.5:0/3978283734 learned_addr learned my addr 172.21.15.5:0/3978283734 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:55:16.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.948+0000 7fd5f659c700 1 -- 172.21.15.5:0/3978283734 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5f810b280 msgr2=0x7fd5f81aa3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:55:16.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.948+0000 7fd5f659c700 1 --2- 172.21.15.5:0/3978283734 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5f810b280 0x7fd5f81aa3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:16.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.948+0000 7fd5f659c700 1 -- 172.21.15.5:0/3978283734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5f811d930 msgr2=0x7fd5f81ae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:55:16.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.948+0000 7fd5f659c700 1 --2- 172.21.15.5:0/3978283734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5f811d930 0x7fd5f81ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:16.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.948+0000 7fd5f659c700 1 -- 172.21.15.5:0/3978283734 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5f81aed20 con 0x7fd5f810a930 2024-05-09T00:55:16.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.950+0000 7fd5f659c700 1 --2- 172.21.15.5:0/3978283734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5f810a930 0x7fd5f81a9e50 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0009bd0 tx=0x7fd5f000cf00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:55:16.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.951+0000 7fd5e77fe700 1 -- 172.21.15.5:0/3978283734 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f0021070 con 0x7fd5f810a930 2024-05-09T00:55:16.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.951+0000 7fd5e77fe700 1 -- 172.21.15.5:0/3978283734 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5f0007d30 con 0x7fd5f810a930 2024-05-09T00:55:16.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.953+0000 7fd5e77fe700 1 -- 172.21.15.5:0/3978283734 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f001d450 con 0x7fd5f810a930 2024-05-09T00:55:16.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.954+0000 7fd5fcd7f700 1 -- 172.21.15.5:0/3978283734 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5f81aefb0 con 0x7fd5f810a930 2024-05-09T00:55:16.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.954+0000 7fd5fcd7f700 1 -- 172.21.15.5:0/3978283734 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5f81af470 con 0x7fd5f810a930 2024-05-09T00:55:16.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.965+0000 7fd60b795b80 1 Processor -- start 2024-05-09T00:55:16.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.965+0000 7fd60b795b80 1 -- start start 2024-05-09T00:55:16.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.966+0000 7fd60b795b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c50a95a160 0x55c50a9f34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:16.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.966+0000 7fd60b795b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c50a8704a0 0x55c50a9f3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:16.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.966+0000 7fd60b795b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c50a9ed6a0 0x55c50a9edad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:16.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.966+0000 7fd60b795b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c50a90fff0 con 0x55c50a8704a0 2024-05-09T00:55:16.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.966+0000 7fd60b795b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55c50a8ae7f0 con 0x55c50a95a160 2024-05-09T00:55:16.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.966+0000 7fd60b795b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55c50a90bfb0 con 0x55c50a9ed6a0 2024-05-09T00:55:16.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.968+0000 7fd5f5d9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c50a9ed6a0 0x55c50a9edad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:16.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.968+0000 7fd5f5d9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c50a9ed6a0 0x55c50a9edad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:52758/0 (socket says 172.21.15.5:52758) 2024-05-09T00:55:16.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.968+0000 7fd5f5d9b700 1 -- 172.21.15.5:0/3432137235 learned_addr learned my addr 172.21.15.5:0/3432137235 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:55:16.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.968+0000 7fd5f659c700 1 --2- 172.21.15.5:0/3432137235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c50a8704a0 0x55c50a9f3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:16.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.970+0000 7fd5f6d9d700 1 --2- 172.21.15.5:0/3432137235 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c50a95a160 0x55c50a9f34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:16.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.970+0000 7fd5f5d9b700 1 -- 172.21.15.5:0/3432137235 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c50a95a160 msgr2=0x55c50a9f34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:55:16.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.970+0000 7fd5f5d9b700 1 --2- 172.21.15.5:0/3432137235 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c50a95a160 0x55c50a9f34f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:16.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.970+0000 7fd5f5d9b700 1 -- 172.21.15.5:0/3432137235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c50a8704a0 msgr2=0x55c50a9f3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:55:16.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.970+0000 7fd5f5d9b700 1 --2- 172.21.15.5:0/3432137235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c50a8704a0 0x55c50a9f3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:16.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.970+0000 7fd5f5d9b700 1 -- 172.21.15.5:0/3432137235 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c50a8a6370 con 0x55c50a9ed6a0 2024-05-09T00:55:16.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.972+0000 7fd5e77fe700 1 -- 172.21.15.5:0/3978283734 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fd5f0004000 con 0x7fd5f810a930 2024-05-09T00:55:16.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.972+0000 7fd5f5d9b700 1 --2- 172.21.15.5:0/3432137235 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c50a9ed6a0 0x55c50a9edad0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8009870 tx=0x7fd5e8009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dcaa93151295f78e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:55:16.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.973+0000 7fd5e4ff9700 1 -- 172.21.15.5:0/3432137235 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e8015020 con 0x55c50a9ed6a0 2024-05-09T00:55:16.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.974+0000 7fd5e4ff9700 1 -- 172.21.15.5:0/3432137235 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5e8011ce0 con 0x55c50a9ed6a0 2024-05-09T00:55:16.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.974+0000 7fd5e4ff9700 1 -- 172.21.15.5:0/3432137235 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e800be20 con 0x55c50a9ed6a0 2024-05-09T00:55:16.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.974+0000 7fd5e77fe700 1 --2- 172.21.15.5:0/3978283734 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fd5e00b2960 0x7fd5e00b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:16.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.975+0000 7fd5e77fe700 1 -- 172.21.15.5:0/3978283734 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fd5f000f020 con 0x7fd5f810a930 2024-05-09T00:55:16.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.977+0000 7fd60b795b80 1 -- 172.21.15.5:0/3432137235 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c50a9ed6a0 msgr2=0x55c50a9edad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:55:16.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.977+0000 7fd60b795b80 1 --2- 172.21.15.5:0/3432137235 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c50a9ed6a0 0x55c50a9edad0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8009870 tx=0x7fd5e8009b80 comp rx=0 tx=0).stop 2024-05-09T00:55:16.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.978+0000 7fd60b795b80 1 -- 172.21.15.5:0/3432137235 shutdown_connections 2024-05-09T00:55:16.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.978+0000 7fd60b795b80 1 --2- 172.21.15.5:0/3432137235 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c50a9ed6a0 0x55c50a9edad0 secure :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8009870 tx=0x7fd5e8009b80 comp rx=0 tx=0).stop 2024-05-09T00:55:16.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.978+0000 7fd60b795b80 1 --2- 172.21.15.5:0/3432137235 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c50a95a160 0x55c50a9f34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:16.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.978+0000 7fd60b795b80 1 --2- 172.21.15.5:0/3432137235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c50a8704a0 0x55c50a9f3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:16.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.978+0000 7fd60b795b80 1 -- 172.21.15.5:0/3432137235 >> 172.21.15.5:0/3432137235 conn(0x55c50a8d06a0 msgr2=0x55c50a9e8400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:55:16.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.980+0000 7fd5f6d9d700 1 --2- 172.21.15.5:0/3978283734 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fd5e00b2960 0x7fd5e00b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:16.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.986+0000 7fd60b795b80 1 -- 172.21.15.5:0/3432137235 shutdown_connections 2024-05-09T00:55:16.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.987+0000 7fd60b795b80 1 -- 172.21.15.5:0/3432137235 wait complete. 2024-05-09T00:55:16.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.988+0000 7fd60b795b80 1 -- start start 2024-05-09T00:55:16.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.988+0000 7fd60b795b80 1 Processor -- start 2024-05-09T00:55:16.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.989+0000 7fd60b795b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c50a8704a0 0x55c50a9f7290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:16.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.990+0000 7fd60b795b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c50a95a160 0x55c50a9f77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:16.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.990+0000 7fd60b795b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c50a9f7d10 0x55c50aa034c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:16.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.990+0000 7fd60b795b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c50a78bba0 con 0x55c50a95a160 2024-05-09T00:55:16.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.990+0000 7fd60b795b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55c50a8ca150 con 0x55c50a8704a0 2024-05-09T00:55:16.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.990+0000 7fd60b795b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55c50a9fa1d0 con 0x55c50a9f7d10 2024-05-09T00:55:16.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.991+0000 7fd60b795b80 10 client.0 _ll_register_callbacks cb 0x55c50a7816c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:55:16.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.991+0000 7fd5f6d9d700 1 --2- 172.21.15.5:0/3978283734 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fd5e00b2960 0x7fd5e00b4e70 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8112ba0 tx=0x7fd5ec008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:55:16.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.992+0000 7fd5f5d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c50a95a160 0x55c50a9f77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:16.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.993+0000 7fd5f659c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c50a8704a0 0x55c50a9f7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:16.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.993+0000 7fd5f659c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c50a8704a0 0x55c50a9f7290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:40206/0 (socket says 172.21.15.5:40206) 2024-05-09T00:55:16.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.993+0000 7fd5f5d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c50a95a160 0x55c50a9f77d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59868/0 (socket says 172.21.15.5:59868) 2024-05-09T00:55:16.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.993+0000 7fd5f659c700 1 -- 172.21.15.5:0/1532642436 learned_addr learned my addr 172.21.15.5:0/1532642436 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:55:17.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:16.997+0000 7fd5f6d9d700 1 --2- 172.21.15.5:0/1532642436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c50a9f7d10 0x55c50aa034c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:17.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.006+0000 7fd5f5d9b700 1 -- 172.21.15.5:0/1532642436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c50a9f7d10 msgr2=0x55c50aa034c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:55:17.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.006+0000 7fd5f5d9b700 1 --2- 172.21.15.5:0/1532642436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c50a9f7d10 0x55c50aa034c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:17.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.006+0000 7fd5f5d9b700 1 -- 172.21.15.5:0/1532642436 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c50a8704a0 msgr2=0x55c50a9f7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:55:17.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.006+0000 7fd5f5d9b700 1 --2- 172.21.15.5:0/1532642436 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c50a8704a0 0x55c50a9f7290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:17.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.006+0000 7fd5f5d9b700 1 -- 172.21.15.5:0/1532642436 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c50a78b490 con 0x55c50a95a160 2024-05-09T00:55:17.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.006+0000 7fd5f5d9b700 1 --2- 172.21.15.5:0/1532642436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c50a95a160 0x55c50a9f77d0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x55c50a9f4050 tx=0x7fd5e8009410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af779f06eef74e33 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:55:17.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.006+0000 7fd5ddffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:55:17.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.009+0000 7fd5ddffb700 1 -- 172.21.15.5:0/1532642436 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e8015040 con 0x55c50a95a160 2024-05-09T00:55:17.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.009+0000 7fd60b795b80 1 -- 172.21.15.5:0/1532642436 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5e8011ce0 con 0x55c50a95a160 2024-05-09T00:55:17.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.009+0000 7fd5ddffb700 1 -- 172.21.15.5:0/1532642436 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fd5e8011630 con 0x55c50a95a160 2024-05-09T00:55:17.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.019+0000 7fd5ddffb700 1 -- 172.21.15.5:0/1532642436 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fd5e802c030 con 0x55c50a95a160 2024-05-09T00:55:17.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.022+0000 7fd60b795b80 10 client.16410 fetch_fsmap learned FSMap version 31 2024-05-09T00:55:17.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.022+0000 7fd60b795b80 1 -- 172.21.15.5:0/1532642436 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c50aa05220 con 0x55c50a95a160 2024-05-09T00:55:17.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.028+0000 7fd5ddffb700 1 -- 172.21.15.5:0/1532642436 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5e8011ce0 con 0x55c50a95a160 2024-05-09T00:55:17.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.028+0000 7fd5ddffb700 1 -- 172.21.15.5:0/1532642436 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fd5e8026b70 con 0x55c50a95a160 2024-05-09T00:55:17.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.028+0000 7fd60b795b80 10 client.16410 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:55:17.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.028+0000 7fd60b795b80 10 client.16410 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:55:17.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.028+0000 7fd60b795b80 20 client.16410 populate_metadata read hostname 'smithi005' 2024-05-09T00:55:17.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.028+0000 7fd60b795b80 1 --2- 172.21.15.5:0/1532642436 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55c50aa079c0 0x55c50aa09e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:17.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.028+0000 7fd60b795b80 4 client.16410 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:55:17.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.028+0000 7fd60b795b80 1 -- 172.21.15.5:0/1532642436 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c50a91d9d0 con 0x55c50aa079c0 2024-05-09T00:55:17.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.030+0000 7fd5f659c700 1 --2- 172.21.15.5:0/1532642436 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55c50aa079c0 0x55c50aa09e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:17.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.034+0000 7fd5f659c700 1 --2- 172.21.15.5:0/1532642436 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55c50aa079c0 0x55c50aa09e00 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=884622c64813b68f server_cookie=2bd438f7447f8ca7 in_seq=0 out_seq=0 2024-05-09T00:55:17.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.034+0000 7fd5ddffb700 10 client.16410 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:55:17.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.037+0000 7fd5ddffb700 1 -- 172.21.15.5:0/1532642436 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c50a91d9d0 con 0x55c50aa079c0 2024-05-09T00:55:17.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.037+0000 7fd5ddffb700 10 client.16410 handle_command_reply: tid=0 2024-05-09T00:55:17.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.037+0000 7fd60b795b80 1 client.16410 shutdown 2024-05-09T00:55:17.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.041+0000 7fd60b795b80 1 -- 172.21.15.5:0/1532642436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c50a95a160 msgr2=0x55c50a9f77d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:55:17.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.041+0000 7fd60b795b80 1 --2- 172.21.15.5:0/1532642436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c50a95a160 0x55c50a9f77d0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x55c50a9f4050 tx=0x7fd5e8009410 comp rx=0 tx=0).stop 2024-05-09T00:55:17.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.041+0000 7fd60b795b80 1 -- 172.21.15.5:0/1532642436 shutdown_connections 2024-05-09T00:55:17.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.041+0000 7fd60b795b80 1 --2- 172.21.15.5:0/1532642436 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55c50aa079c0 0x55c50aa09e00 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:17.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.041+0000 7fd60b795b80 1 --2- 172.21.15.5:0/1532642436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c50a9f7d10 0x55c50aa034c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:17.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.041+0000 7fd60b795b80 1 --2- 172.21.15.5:0/1532642436 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c50a8704a0 0x55c50a9f7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:17.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.041+0000 7fd60b795b80 1 --2- 172.21.15.5:0/1532642436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c50a95a160 0x55c50a9f77d0 secure :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0x55c50a9f4050 tx=0x7fd5e8009410 comp rx=0 tx=0).stop 2024-05-09T00:55:17.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.041+0000 7fd60b795b80 1 -- 172.21.15.5:0/1532642436 >> 172.21.15.5:0/1532642436 conn(0x55c50a8d06a0 msgr2=0x55c50a9e81c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:55:17.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.042+0000 7fd5ddffb700 0 client.16410 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:55:17.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.042+0000 7fd60b795b80 1 -- 172.21.15.5:0/1532642436 shutdown_connections 2024-05-09T00:55:17.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.044+0000 7fd60b795b80 1 -- 172.21.15.5:0/1532642436 wait complete. 2024-05-09T00:55:17.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.045+0000 7fd60b795b80 20 client.16410 trim_cache size 0 max 16384 2024-05-09T00:55:17.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.081+0000 7fd60b795b80 1 Processor -- start 2024-05-09T00:55:17.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.081+0000 7fd60b795b80 1 -- start start 2024-05-09T00:55:17.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.082+0000 7fd60b795b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c50a8704a0 0x55c50a9f5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:17.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.083+0000 7fd60b795b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c50a9f7d10 0x55c50a9f60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:17.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.083+0000 7fd60b795b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c50aa0a0e0 0x55c50aa0f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:17.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.083+0000 7fd60b795b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f8119c70 con 0x55c50a8704a0 2024-05-09T00:55:17.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.083+0000 7fd60b795b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f81c7d20 con 0x55c50aa0a0e0 2024-05-09T00:55:17.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.083+0000 7fd60b795b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f811a340 con 0x55c50a9f7d10 2024-05-09T00:55:17.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.083+0000 7fd5f5d9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c50a9f7d10 0x55c50a9f60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:17.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.084+0000 7fd5f5d9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c50a9f7d10 0x55c50a9f60c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:52782/0 (socket says 172.21.15.5:52782) 2024-05-09T00:55:17.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.084+0000 7fd5f5d9b700 1 -- 172.21.15.5:0/327169032 learned_addr learned my addr 172.21.15.5:0/327169032 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:55:17.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.084+0000 7fd5f659c700 1 --2- 172.21.15.5:0/327169032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c50a8704a0 0x55c50a9f5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:17.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.084+0000 7fd5f5d9b700 1 -- 172.21.15.5:0/327169032 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c50aa0a0e0 msgr2=0x55c50aa0f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:55:17.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.084+0000 7fd5f5d9b700 1 --2- 172.21.15.5:0/327169032 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c50aa0a0e0 0x55c50aa0f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:17.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.084+0000 7fd5f5d9b700 1 -- 172.21.15.5:0/327169032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c50a8704a0 msgr2=0x55c50a9f5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:55:17.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.084+0000 7fd5f5d9b700 1 --2- 172.21.15.5:0/327169032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c50a8704a0 0x55c50a9f5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:17.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.084+0000 7fd5f5d9b700 1 -- 172.21.15.5:0/327169032 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c50a9f5800 con 0x55c50a9f7d10 2024-05-09T00:55:17.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.085+0000 7fd5f5d9b700 1 --2- 172.21.15.5:0/327169032 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c50a9f7d10 0x55c50a9f60c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd5e801a570 tx=0x7fd5e8012f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47912ee57af84ed9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:55:17.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.087+0000 7fd5dffff700 1 -- 172.21.15.5:0/327169032 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e8035070 con 0x55c50a9f7d10 2024-05-09T00:55:17.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.087+0000 7fd5dffff700 1 -- 172.21.15.5:0/327169032 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5e800b930 con 0x55c50a9f7d10 2024-05-09T00:55:17.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.087+0000 7fd5dffff700 1 -- 172.21.15.5:0/327169032 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e8011890 con 0x55c50a9f7d10 2024-05-09T00:55:17.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.089+0000 7fd60b795b80 1 -- 172.21.15.5:0/327169032 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c50a9f7d10 msgr2=0x55c50a9f60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:55:17.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.089+0000 7fd60b795b80 1 --2- 172.21.15.5:0/327169032 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c50a9f7d10 0x55c50a9f60c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd5e801a570 tx=0x7fd5e8012f50 comp rx=0 tx=0).stop 2024-05-09T00:55:17.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.089+0000 7fd60b795b80 1 -- 172.21.15.5:0/327169032 shutdown_connections 2024-05-09T00:55:17.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.089+0000 7fd60b795b80 1 --2- 172.21.15.5:0/327169032 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c50aa0a0e0 0x55c50aa0f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:17.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.089+0000 7fd60b795b80 1 --2- 172.21.15.5:0/327169032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c50a8704a0 0x55c50a9f5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:17.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.089+0000 7fd60b795b80 1 --2- 172.21.15.5:0/327169032 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c50a9f7d10 0x55c50a9f60c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:17.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.089+0000 7fd60b795b80 1 -- 172.21.15.5:0/327169032 >> 172.21.15.5:0/327169032 conn(0x55c50a8d06a0 msgr2=0x55c50aa099d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:55:17.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.093+0000 7fd60b795b80 1 -- 172.21.15.5:0/327169032 shutdown_connections 2024-05-09T00:55:17.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.093+0000 7fd60b795b80 1 -- 172.21.15.5:0/327169032 wait complete. 2024-05-09T00:55:17.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.094+0000 7fd60b795b80 1 -- start start 2024-05-09T00:55:17.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.094+0000 7fd60b795b80 1 Processor -- start 2024-05-09T00:55:17.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.095+0000 7fd60b795b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c50a8704a0 0x55c50a9fa3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:17.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.095+0000 7fd60b795b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c50a9f7d10 0x55c50a9fa920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:17.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.095+0000 7fd60b795b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c50aa0a0e0 0x55c50a9fae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:17.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.095+0000 7fd60b795b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f811b0e0 con 0x55c50a8704a0 2024-05-09T00:55:17.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.095+0000 7fd60b795b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f811aa10 con 0x55c50a9f7d10 2024-05-09T00:55:17.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.096+0000 7fd60b795b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f8117c10 con 0x55c50aa0a0e0 2024-05-09T00:55:17.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.096+0000 7fd60b795b80 10 client.0 _ll_register_callbacks cb 0x55c50a7816c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:55:17.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.096+0000 7fd5f659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c50a8704a0 0x55c50a9fa3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:17.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.096+0000 7fd5f5d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c50a9f7d10 0x55c50a9fa920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:17.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.097+0000 7fd5f659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c50a8704a0 0x55c50a9fa3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59884/0 (socket says 172.21.15.5:59884) 2024-05-09T00:55:17.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.097+0000 7fd5f659c700 1 -- 172.21.15.5:0/2749947121 learned_addr learned my addr 172.21.15.5:0/2749947121 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:55:17.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.097+0000 7fd5f659c700 1 -- 172.21.15.5:0/2749947121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c50aa0a0e0 msgr2=0x55c50a9fae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:55:17.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.097+0000 7fd5f659c700 1 --2- 172.21.15.5:0/2749947121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c50aa0a0e0 0x55c50a9fae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:17.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.097+0000 7fd5f659c700 1 -- 172.21.15.5:0/2749947121 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c50a9f7d10 msgr2=0x55c50a9fa920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:55:17.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.097+0000 7fd5f659c700 1 --2- 172.21.15.5:0/2749947121 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c50a9f7d10 0x55c50a9fa920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:17.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.097+0000 7fd5f659c700 1 -- 172.21.15.5:0/2749947121 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c50aa703b0 con 0x55c50a8704a0 2024-05-09T00:55:17.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.098+0000 7fd5f659c700 1 --2- 172.21.15.5:0/2749947121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c50a8704a0 0x55c50a9fa3e0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fd5f00abb80 tx=0x7fd5f0029000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d94a0109ae05e81 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:55:17.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.098+0000 7fd5ddffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:55:17.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.098+0000 7fd5ddffb700 1 -- 172.21.15.5:0/2749947121 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f00ab290 con 0x55c50a8704a0 2024-05-09T00:55:17.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.099+0000 7fd5ddffb700 1 -- 172.21.15.5:0/2749947121 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fd5f00e80b0 con 0x55c50a8704a0 2024-05-09T00:55:17.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.100+0000 7fd60b795b80 1 -- 172.21.15.5:0/2749947121 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5e800b930 con 0x55c50a8704a0 2024-05-09T00:55:17.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.101+0000 7fd5ddffb700 1 -- 172.21.15.5:0/2749947121 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fd5f00b8210 con 0x55c50a8704a0 2024-05-09T00:55:17.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.103+0000 7fd60b795b80 10 client.16416 fetch_fsmap learned FSMap version 31 2024-05-09T00:55:17.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.103+0000 7fd60b795b80 1 -- 172.21.15.5:0/2749947121 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c50aa71670 con 0x55c50a8704a0 2024-05-09T00:55:17.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.105+0000 7fd5ddffb700 1 -- 172.21.15.5:0/2749947121 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5e800b930 con 0x55c50a8704a0 2024-05-09T00:55:17.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.105+0000 7fd5ddffb700 1 -- 172.21.15.5:0/2749947121 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fd5f0011070 con 0x55c50a8704a0 2024-05-09T00:55:17.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.105+0000 7fd60b795b80 10 client.16416 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:55:17.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.105+0000 7fd60b795b80 10 client.16416 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:55:17.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.105+0000 7fd60b795b80 20 client.16416 populate_metadata read hostname 'smithi005' 2024-05-09T00:55:17.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.105+0000 7fd60b795b80 1 --2- 172.21.15.5:0/2749947121 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55c50aa71b50 0x55c50aa73f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:17.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.106+0000 7fd60b795b80 4 client.16416 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:55:17.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.106+0000 7fd60b795b80 1 -- 172.21.15.5:0/2749947121 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c50a917a70 con 0x55c50aa71b50 2024-05-09T00:55:17.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.110+0000 7fd5f5d9b700 1 --2- 172.21.15.5:0/2749947121 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55c50aa71b50 0x55c50aa73f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:17.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.124+0000 7fd5f5d9b700 1 --2- 172.21.15.5:0/2749947121 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55c50aa71b50 0x55c50aa73f70 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3eb1bb269225d1e2 server_cookie=d54ffa4116310515 in_seq=0 out_seq=0 2024-05-09T00:55:17.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.125+0000 7fd5ddffb700 10 client.16416 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:55:17.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.128+0000 7fd5ddffb700 1 -- 172.21.15.5:0/2749947121 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c50a917a70 con 0x55c50aa71b50 2024-05-09T00:55:17.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.129+0000 7fd5ddffb700 10 client.16416 handle_command_reply: tid=0 2024-05-09T00:55:17.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.129+0000 7fd60b795b80 1 client.16416 shutdown 2024-05-09T00:55:17.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.130+0000 7fd60b795b80 1 -- 172.21.15.5:0/2749947121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c50a8704a0 msgr2=0x55c50a9fa3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:55:17.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.130+0000 7fd60b795b80 1 --2- 172.21.15.5:0/2749947121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c50a8704a0 0x55c50a9fa3e0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fd5f00abb80 tx=0x7fd5f0029000 comp rx=0 tx=0).stop 2024-05-09T00:55:17.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.131+0000 7fd60b795b80 1 -- 172.21.15.5:0/2749947121 shutdown_connections 2024-05-09T00:55:17.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.131+0000 7fd60b795b80 1 --2- 172.21.15.5:0/2749947121 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55c50aa71b50 0x55c50aa73f70 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:17.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.131+0000 7fd60b795b80 1 --2- 172.21.15.5:0/2749947121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c50aa0a0e0 0x55c50a9fae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:17.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.131+0000 7fd5ddffb700 0 client.16416 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:55:17.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.131+0000 7fd60b795b80 1 --2- 172.21.15.5:0/2749947121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c50a8704a0 0x55c50a9fa3e0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:17.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.131+0000 7fd60b795b80 1 --2- 172.21.15.5:0/2749947121 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c50a9f7d10 0x55c50a9fa920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:17.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.131+0000 7fd60b795b80 1 -- 172.21.15.5:0/2749947121 >> 172.21.15.5:0/2749947121 conn(0x55c50a8d06a0 msgr2=0x55c50aa099d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:55:17.138 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-09T00:55:17.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.134+0000 7fd60b795b80 1 -- 172.21.15.5:0/2749947121 shutdown_connections 2024-05-09T00:55:17.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.135+0000 7fd60b795b80 1 -- 172.21.15.5:0/2749947121 wait complete. 2024-05-09T00:55:17.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.135+0000 7fd60b795b80 20 client.16416 trim_cache size 0 max 16384 2024-05-09T00:55:17.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.138+0000 7fd5fcd7f700 1 -- 172.21.15.5:0/3978283734 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fd5e00b2960 msgr2=0x7fd5e00b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:55:17.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.138+0000 7fd5fcd7f700 1 --2- 172.21.15.5:0/3978283734 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fd5e00b2960 0x7fd5e00b4e70 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8112ba0 tx=0x7fd5ec008040 comp rx=0 tx=0).stop 2024-05-09T00:55:17.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.138+0000 7fd5fcd7f700 1 -- 172.21.15.5:0/3978283734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5f810a930 msgr2=0x7fd5f81a9e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:55:17.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.138+0000 7fd5fcd7f700 1 --2- 172.21.15.5:0/3978283734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5f810a930 0x7fd5f81a9e50 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0009bd0 tx=0x7fd5f000cf00 comp rx=0 tx=0).stop 2024-05-09T00:55:17.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.138+0000 7fd5fcd7f700 1 -- 172.21.15.5:0/3978283734 shutdown_connections 2024-05-09T00:55:17.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.138+0000 7fd5fcd7f700 1 --2- 172.21.15.5:0/3978283734 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fd5e00b2960 0x7fd5e00b4e70 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:17.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.138+0000 7fd5fcd7f700 1 --2- 172.21.15.5:0/3978283734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5f811d930 0x7fd5f81ae680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:17.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.138+0000 7fd5fcd7f700 1 --2- 172.21.15.5:0/3978283734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5f810a930 0x7fd5f81a9e50 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:17.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.138+0000 7fd5fcd7f700 1 --2- 172.21.15.5:0/3978283734 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5f810b280 0x7fd5f81aa3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:17.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.138+0000 7fd5fcd7f700 1 -- 172.21.15.5:0/3978283734 >> 172.21.15.5:0/3978283734 conn(0x7fd5f806f4d0 msgr2=0x7fd5f811b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:55:17.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.138+0000 7fd5fcd7f700 1 -- 172.21.15.5:0/3978283734 shutdown_connections 2024-05-09T00:55:17.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:17.138+0000 7fd5fcd7f700 1 -- 172.21.15.5:0/3978283734 wait complete. 2024-05-09T00:55:17.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:17.761+0000 7f8171ffb640 1 -- 172.21.15.5:0/788307479 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56121f3463d0 con 0x56121f00f4d0 2024-05-09T00:55:17.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:17.761+0000 7f8171ffb640 10 client.16404 handle_command_reply: tid=0 2024-05-09T00:55:17.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:17.761+0000 7f81971c4740 1 client.16404 shutdown 2024-05-09T00:55:17.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:17.762+0000 7f81971c4740 1 -- 172.21.15.5:0/788307479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56121f0072f0 msgr2=0x56121f3e2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:55:17.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:17.762+0000 7f81971c4740 1 --2- 172.21.15.5:0/788307479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56121f0072f0 0x56121f3e2670 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f8180031a50 tx=0x7f8180031630 comp rx=0 tx=0).stop 2024-05-09T00:55:17.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:17.762+0000 7f81971c4740 1 -- 172.21.15.5:0/788307479 shutdown_connections 2024-05-09T00:55:17.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:17.762+0000 7f81971c4740 1 --2- 172.21.15.5:0/788307479 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56121f00f4d0 0x56121f455e40 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:17.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:17.762+0000 7f81971c4740 1 --2- 172.21.15.5:0/788307479 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56121f024300 0x56121f3e35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:17.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:17.762+0000 7f81971c4740 1 --2- 172.21.15.5:0/788307479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56121f0072f0 0x56121f3e2670 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:17.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:17.762+0000 7f81971c4740 1 --2- 172.21.15.5:0/788307479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56121efeed80 0x56121f3e2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:17.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:17.762+0000 7f81971c4740 1 -- 172.21.15.5:0/788307479 >> 172.21.15.5:0/788307479 conn(0x56121f3ebad0 msgr2=0x56121f3dc990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:55:17.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:17.762+0000 7f8171ffb640 0 client.16404 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:55:17.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:17.762+0000 7f81971c4740 1 -- 172.21.15.5:0/788307479 shutdown_connections 2024-05-09T00:55:17.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:17.762+0000 7f81971c4740 1 -- 172.21.15.5:0/788307479 wait complete. 2024-05-09T00:55:17.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:17.762+0000 7f81971c4740 20 client.16404 trim_cache size 0 max 16384 2024-05-09T00:55:17.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-09T00:55:17.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-09T00:55:17.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-09T00:55:17.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:17.763+0000 7f819030d640 1 -- 172.21.15.5:0/4156329476 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f81740b2bd0 msgr2=0x7f81740b50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:55:17.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:17.763+0000 7f819030d640 1 --2- 172.21.15.5:0/4156329476 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f81740b2bd0 0x7f81740b50e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f81840a8bd0 tx=0x7f8184005d80 comp rx=0 tx=0).stop 2024-05-09T00:55:17.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:17.763+0000 7f819030d640 1 -- 172.21.15.5:0/4156329476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f818810b8b0 msgr2=0x7f818811ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:55:17.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:17.763+0000 7f819030d640 1 --2- 172.21.15.5:0/4156329476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f818810b8b0 0x7f818811ac00 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f818400c970 tx=0x7f818400ce30 comp rx=0 tx=0).stop 2024-05-09T00:55:17.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:17.764+0000 7f819030d640 1 -- 172.21.15.5:0/4156329476 shutdown_connections 2024-05-09T00:55:17.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:17.764+0000 7f819030d640 1 --2- 172.21.15.5:0/4156329476 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f81740b2bd0 0x7f81740b50e0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:17.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:17.764+0000 7f819030d640 1 --2- 172.21.15.5:0/4156329476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f818810b8b0 0x7f818811ac00 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:17.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:17.764+0000 7f819030d640 1 --2- 172.21.15.5:0/4156329476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f818810a670 0x7f818811a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:17.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:17.764+0000 7f819030d640 1 --2- 172.21.15.5:0/4156329476 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f818806bfb0 0x7f818811bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:17.764+0000 7f819030d640 1 -- 172.21.15.5:0/4156329476 >> 172.21.15.5:0/4156329476 conn(0x7f8188070a90 msgr2=0x7f818810c590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:55:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:17.764+0000 7f819030d640 1 -- 172.21.15.5:0/4156329476 shutdown_connections 2024-05-09T00:55:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:17.764+0000 7f819030d640 1 -- 172.21.15.5:0/4156329476 wait complete. 2024-05-09T00:55:17.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:55:17 smithi005 ceph-mon[30365]: pgmap v639: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 884 MiB/s wr, 1.85k op/s 2024-05-09T00:55:17.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:55:17 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:55:17.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:55:17 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:55:17.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:55:17 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:55:17.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:55:17 smithi131 ceph-mon[34547]: pgmap v639: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 884 MiB/s wr, 1.85k op/s 2024-05-09T00:55:17.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:55:17 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:55:17.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:55:17 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:55:17.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:55:17 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:55:17.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:55:17 smithi157 ceph-mon[34294]: pgmap v639: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 884 MiB/s wr, 1.85k op/s 2024-05-09T00:55:17.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:55:17 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:55:17.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:55:17 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:55:17.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:55:17 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:55:19.825 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T00:55:19.825 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T00:55:19.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:55:19 smithi131 ceph-mon[34547]: pgmap v640: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 865 MiB/s wr, 1.66k op/s 2024-05-09T00:55:19.937 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:55:19 smithi157 ceph-mon[34294]: pgmap v640: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 865 MiB/s wr, 1.66k op/s 2024-05-09T00:55:19.943 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:55:19 smithi005 ceph-mon[30365]: pgmap v640: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 865 MiB/s wr, 1.66k op/s 2024-05-09T00:55:20.756 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:55:21.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:55:21 smithi131 ceph-mon[34547]: pgmap v641: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 975 MiB/s wr, 1.78k op/s 2024-05-09T00:55:21.925 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:55:21 smithi005 ceph-mon[30365]: pgmap v641: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 975 MiB/s wr, 1.78k op/s 2024-05-09T00:55:21.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:55:21 smithi157 ceph-mon[34294]: pgmap v641: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 975 MiB/s wr, 1.78k op/s 2024-05-09T00:55:23.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:55:23 smithi157 ceph-mon[34294]: pgmap v642: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 848 MiB/s wr, 1.78k op/s 2024-05-09T00:55:24.040 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:55:23 smithi005 ceph-mon[30365]: pgmap v642: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 848 MiB/s wr, 1.78k op/s 2024-05-09T00:55:24.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:55:23 smithi131 ceph-mon[34547]: pgmap v642: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 848 MiB/s wr, 1.78k op/s 2024-05-09T00:55:25.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:55:25 smithi157 ceph-mon[34294]: pgmap v643: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 896 MiB/s wr, 2.00k op/s 2024-05-09T00:55:25.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:25.989+0000 7f87fa3a1700 1 -- 172.21.15.5:0/4158676714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87f4103270 msgr2=0x7f87f4103680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:55:25.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:25.989+0000 7f87fa3a1700 1 --2- 172.21.15.5:0/4158676714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87f4103270 0x7f87f4103680 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f87e80099c0 tx=0x7f87e8009cd0 comp rx=0 tx=0).stop 2024-05-09T00:55:25.998 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:55:25 smithi005 ceph-mon[30365]: pgmap v643: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 896 MiB/s wr, 2.00k op/s 2024-05-09T00:55:25.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:25.994+0000 7f87fa3a1700 1 -- 172.21.15.5:0/4158676714 shutdown_connections 2024-05-09T00:55:25.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:25.994+0000 7f87fa3a1700 1 --2- 172.21.15.5:0/4158676714 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87f4108710 0x7f87f4108bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:25.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:25.994+0000 7f87fa3a1700 1 --2- 172.21.15.5:0/4158676714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87f4103270 0x7f87f4103680 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:25.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:25.994+0000 7f87fa3a1700 1 --2- 172.21.15.5:0/4158676714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87f4103c50 0x7f87f41080f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:25.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:25.994+0000 7f87fa3a1700 1 -- 172.21.15.5:0/4158676714 >> 172.21.15.5:0/4158676714 conn(0x7f87f40fef50 msgr2=0x7f87f4101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:55:25.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:25.996+0000 7f87fa3a1700 1 -- 172.21.15.5:0/4158676714 shutdown_connections 2024-05-09T00:55:25.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:25.996+0000 7f87fa3a1700 1 -- 172.21.15.5:0/4158676714 wait complete. 2024-05-09T00:55:26.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:25.998+0000 7f87fa3a1700 1 Processor -- start 2024-05-09T00:55:26.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:25.998+0000 7f87fa3a1700 1 -- start start 2024-05-09T00:55:26.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:25.999+0000 7f87fa3a1700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87f4103270 0x7f87f41a14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:26.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:25.999+0000 7f87fa3a1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87f4103c50 0x7f87f41a1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:26.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:25.999+0000 7f87fa3a1700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87f4108710 0x7f87f41a5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:26.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:25.999+0000 7f87fa3a1700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f87f411ac40 con 0x7f87f4103c50 2024-05-09T00:55:26.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:25.999+0000 7f87fa3a1700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f87f411aac0 con 0x7f87f4108710 2024-05-09T00:55:26.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:25.999+0000 7f87fa3a1700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f87f411adc0 con 0x7f87f4103270 2024-05-09T00:55:26.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.000+0000 7f87f893e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87f4108710 0x7f87f41a5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:26.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.000+0000 7f87f893e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87f4108710 0x7f87f41a5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:59144/0 (socket says 172.21.15.5:59144) 2024-05-09T00:55:26.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.000+0000 7f87f893e700 1 -- 172.21.15.5:0/782823851 learned_addr learned my addr 172.21.15.5:0/782823851 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:55:26.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.003+0000 7f87f893e700 1 -- 172.21.15.5:0/782823851 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87f4103270 msgr2=0x7f87f41a14f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T00:55:26.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.003+0000 7f87f893e700 1 --2- 172.21.15.5:0/782823851 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87f4103270 0x7f87f41a14f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:26.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.003+0000 7f87f893e700 1 -- 172.21.15.5:0/782823851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87f4103c50 msgr2=0x7f87f41a1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:55:26.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.003+0000 7f87f893e700 1 --2- 172.21.15.5:0/782823851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87f4103c50 0x7f87f41a1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:26.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.003+0000 7f87f893e700 1 -- 172.21.15.5:0/782823851 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87f41a63c0 con 0x7f87f4108710 2024-05-09T00:55:26.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.004+0000 7f87f893e700 1 --2- 172.21.15.5:0/782823851 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87f4108710 0x7f87f41a5d20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f87ec00b970 tx=0x7f87ec00bd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:55:26.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.009+0000 7f87f17fa700 1 -- 172.21.15.5:0/782823851 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ec019070 con 0x7f87f4108710 2024-05-09T00:55:26.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.009+0000 7f87f17fa700 1 -- 172.21.15.5:0/782823851 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87ec014440 con 0x7f87f4108710 2024-05-09T00:55:26.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.011+0000 7f87f17fa700 1 -- 172.21.15.5:0/782823851 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ec00ce60 con 0x7f87f4108710 2024-05-09T00:55:26.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.011+0000 7f87fa3a1700 1 -- 172.21.15.5:0/782823851 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87f41a66b0 con 0x7f87f4108710 2024-05-09T00:55:26.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.011+0000 7f87fa3a1700 1 -- 172.21.15.5:0/782823851 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87f406d1e0 con 0x7f87f4108710 2024-05-09T00:55:26.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.033+0000 7f87f17fa700 1 -- 172.21.15.5:0/782823851 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f87ec00c9b0 con 0x7f87f4108710 2024-05-09T00:55:26.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.035+0000 7f87f17fa700 1 --2- 172.21.15.5:0/782823851 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f87dc0b2960 0x7f87dc0b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:26.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.035+0000 7f87f17fa700 1 -- 172.21.15.5:0/782823851 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f87ec0e1520 con 0x7f87f4108710 2024-05-09T00:55:26.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.036+0000 7f8808db7b80 1 Processor -- start 2024-05-09T00:55:26.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.037+0000 7f8808db7b80 1 -- start start 2024-05-09T00:55:26.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.038+0000 7f8808db7b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c645b32d80 0x55c645c3b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:26.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.038+0000 7f8808db7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c645ab8520 0x55c645c3b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:26.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.038+0000 7f8808db7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c645c35480 0x55c645c35890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:26.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.038+0000 7f8808db7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c645b58120 con 0x55c645ab8520 2024-05-09T00:55:26.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.038+0000 7f87f37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c645ab8520 0x55c645c3b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:26.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.038+0000 7f87f37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c645ab8520 0x55c645c3b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59006/0 (socket says 172.21.15.5:59006) 2024-05-09T00:55:26.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.038+0000 7f87f37fe700 1 -- 172.21.15.5:0/673965642 learned_addr learned my addr 172.21.15.5:0/673965642 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:55:26.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.039+0000 7f87f3fff700 1 --2- 172.21.15.5:0/673965642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c645b32d80 0x55c645c3b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:26.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.039+0000 7f8808db7b80 1 -- 172.21.15.5:0/673965642 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55c645afda20 con 0x55c645c35480 2024-05-09T00:55:26.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.039+0000 7f8808db7b80 1 -- 172.21.15.5:0/673965642 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55c645b540e0 con 0x55c645b32d80 2024-05-09T00:55:26.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.039+0000 7f87f893e700 1 --2- 172.21.15.5:0/673965642 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c645c35480 0x55c645c35890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:26.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.040+0000 7f87f37fe700 1 -- 172.21.15.5:0/673965642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c645b32d80 msgr2=0x55c645c3b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:55:26.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.040+0000 7f87f37fe700 1 --2- 172.21.15.5:0/673965642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c645b32d80 0x55c645c3b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:26.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.040+0000 7f87f37fe700 1 -- 172.21.15.5:0/673965642 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c645c35480 msgr2=0x55c645c35890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:55:26.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.040+0000 7f87f37fe700 1 --2- 172.21.15.5:0/673965642 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c645c35480 0x55c645c35890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:26.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.040+0000 7f87f37fe700 1 -- 172.21.15.5:0/673965642 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c645aee580 con 0x55c645ab8520 2024-05-09T00:55:26.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.040+0000 7f87f893e700 1 --2- 172.21.15.5:0/673965642 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c645c35480 0x55c645c35890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:55:26.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.040+0000 7f87f37fe700 1 --2- 172.21.15.5:0/673965642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c645ab8520 0x55c645c3b800 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f87e4009820 tx=0x7f87e4009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f8b8acd865deb8f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:55:26.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.040+0000 7f87f3fff700 1 --2- 172.21.15.5:0/673965642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c645b32d80 0x55c645c3b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:55:26.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.040+0000 7f87f3fff700 1 --2- 172.21.15.5:0/782823851 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f87dc0b2960 0x7f87dc0b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:26.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.040+0000 7f87da7fc700 1 -- 172.21.15.5:0/673965642 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e4015020 con 0x55c645ab8520 2024-05-09T00:55:26.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.041+0000 7f87da7fc700 1 -- 172.21.15.5:0/673965642 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87e4011ce0 con 0x55c645ab8520 2024-05-09T00:55:26.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.041+0000 7f87da7fc700 1 -- 172.21.15.5:0/673965642 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e400bd80 con 0x55c645ab8520 2024-05-09T00:55:26.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.042+0000 7f87f3fff700 1 --2- 172.21.15.5:0/782823851 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f87dc0b2960 0x7f87dc0b4e70 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f87e8003800 tx=0x7f87e8019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:55:26.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.042+0000 7f8808db7b80 1 -- 172.21.15.5:0/673965642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c645ab8520 msgr2=0x55c645c3b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:55:26.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.042+0000 7f8808db7b80 1 --2- 172.21.15.5:0/673965642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c645ab8520 0x55c645c3b800 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f87e4009820 tx=0x7f87e4009b30 comp rx=0 tx=0).stop 2024-05-09T00:55:26.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.043+0000 7f8808db7b80 1 -- 172.21.15.5:0/673965642 shutdown_connections 2024-05-09T00:55:26.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.043+0000 7f8808db7b80 1 --2- 172.21.15.5:0/673965642 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c645c35480 0x55c645c35890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:26.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.043+0000 7f8808db7b80 1 --2- 172.21.15.5:0/673965642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c645b32d80 0x55c645c3b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:26.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.043+0000 7f8808db7b80 1 --2- 172.21.15.5:0/673965642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c645ab8520 0x55c645c3b800 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:26.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.043+0000 7f8808db7b80 1 -- 172.21.15.5:0/673965642 >> 172.21.15.5:0/673965642 conn(0x55c645abbff0 msgr2=0x55c645c302a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:55:26.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.043+0000 7f8808db7b80 1 -- 172.21.15.5:0/673965642 shutdown_connections 2024-05-09T00:55:26.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.043+0000 7f8808db7b80 1 -- 172.21.15.5:0/673965642 wait complete. 2024-05-09T00:55:26.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.043+0000 7f8808db7b80 1 -- start start 2024-05-09T00:55:26.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.044+0000 7f8808db7b80 1 Processor -- start 2024-05-09T00:55:26.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.050+0000 7f8808db7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c645ab8520 0x55c645c3f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:26.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.050+0000 7f8808db7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c645b32d80 0x55c645c3f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:26.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.050+0000 7f8808db7b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c645c35480 0x55c645c3fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:26.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.050+0000 7f8808db7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c645c361e0 con 0x55c645ab8520 2024-05-09T00:55:26.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.050+0000 7f8808db7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55c645af2ba0 con 0x55c645b32d80 2024-05-09T00:55:26.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.050+0000 7f8808db7b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55c645c420a0 con 0x55c645c35480 2024-05-09T00:55:26.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.050+0000 7f8808db7b80 10 client.0 _ll_register_callbacks cb 0x55c6459c9740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:55:26.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.050+0000 7f87f3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c645ab8520 0x55c645c3f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:26.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.050+0000 7f87f3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c645ab8520 0x55c645c3f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59020/0 (socket says 172.21.15.5:59020) 2024-05-09T00:55:26.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.050+0000 7f87f3fff700 1 -- 172.21.15.5:0/2826490276 learned_addr learned my addr 172.21.15.5:0/2826490276 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:55:26.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.052+0000 7f87f37fe700 1 --2- 172.21.15.5:0/2826490276 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c645b32d80 0x55c645c3f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:26.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.054+0000 7f87f893e700 1 --2- 172.21.15.5:0/2826490276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c645c35480 0x55c645c3fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:26.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.055+0000 7f87f37fe700 1 -- 172.21.15.5:0/2826490276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c645c35480 msgr2=0x55c645c3fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:55:26.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.055+0000 7f87f37fe700 1 --2- 172.21.15.5:0/2826490276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c645c35480 0x55c645c3fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:26.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.055+0000 7f87f37fe700 1 -- 172.21.15.5:0/2826490276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c645ab8520 msgr2=0x55c645c3f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:55:26.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.055+0000 7f87f37fe700 1 --2- 172.21.15.5:0/2826490276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c645ab8520 0x55c645c3f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:26.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.055+0000 7f87f37fe700 1 -- 172.21.15.5:0/2826490276 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6459d3510 con 0x55c645b32d80 2024-05-09T00:55:26.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.068+0000 7f87f37fe700 1 --2- 172.21.15.5:0/2826490276 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c645b32d80 0x55c645c3f5e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f87e40090b0 tx=0x7f87e400bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16f78125caa22032 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:55:26.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.069+0000 7f87d37fe700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-09T00:55:26.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.069+0000 7f87d37fe700 1 -- 172.21.15.5:0/2826490276 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e4015030 con 0x55c645b32d80 2024-05-09T00:55:26.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.069+0000 7f87d37fe700 1 -- 172.21.15.5:0/2826490276 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f87e4011a40 con 0x55c645b32d80 2024-05-09T00:55:26.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.069+0000 7f8808db7b80 1 -- 172.21.15.5:0/2826490276 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87e4011ce0 con 0x55c645b32d80 2024-05-09T00:55:26.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.081+0000 7f87d37fe700 1 -- 172.21.15.5:0/2826490276 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f87e4020070 con 0x55c645b32d80 2024-05-09T00:55:26.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.081+0000 7f8808db7b80 10 client.25922 fetch_fsmap learned FSMap version 31 2024-05-09T00:55:26.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.081+0000 7f8808db7b80 1 -- 172.21.15.5:0/2826490276 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c645c4c7d0 con 0x55c645b32d80 2024-05-09T00:55:26.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.082+0000 7f87d37fe700 1 -- 172.21.15.5:0/2826490276 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87e401e070 con 0x55c645b32d80 2024-05-09T00:55:26.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.084+0000 7f87d37fe700 1 -- 172.21.15.5:0/2826490276 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f87e401a420 con 0x55c645b32d80 2024-05-09T00:55:26.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.084+0000 7f8808db7b80 10 client.25922 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:55:26.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.084+0000 7f8808db7b80 10 client.25922 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:55:26.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.084+0000 7f8808db7b80 20 client.25922 populate_metadata read hostname 'smithi005' 2024-05-09T00:55:26.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.084+0000 7f8808db7b80 1 --2- 172.21.15.5:0/2826490276 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55c645c4ed50 0x55c645c51170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:26.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.084+0000 7f8808db7b80 4 client.25922 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:55:26.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.084+0000 7f8808db7b80 1 -- 172.21.15.5:0/2826490276 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f87e4011ce0 con 0x55c645c4ed50 2024-05-09T00:55:26.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.087+0000 7f87f3fff700 1 --2- 172.21.15.5:0/2826490276 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55c645c4ed50 0x55c645c51170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:26.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.090+0000 7f87d37fe700 10 client.25922 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:55:26.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.090+0000 7f87f3fff700 1 --2- 172.21.15.5:0/2826490276 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55c645c4ed50 0x55c645c51170 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d31d7ce91d7bed9 server_cookie=f1a24ba365a251c1 in_seq=0 out_seq=0 2024-05-09T00:55:26.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.096+0000 7f87d37fe700 1 -- 172.21.15.5:0/2826490276 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f87e4011ce0 con 0x55c645c4ed50 2024-05-09T00:55:26.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.096+0000 7f87d37fe700 10 client.25922 handle_command_reply: tid=0 2024-05-09T00:55:26.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.097+0000 7f8808db7b80 1 client.25922 shutdown 2024-05-09T00:55:26.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.097+0000 7f8808db7b80 1 -- 172.21.15.5:0/2826490276 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c645b32d80 msgr2=0x55c645c3f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:55:26.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.097+0000 7f8808db7b80 1 --2- 172.21.15.5:0/2826490276 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c645b32d80 0x55c645c3f5e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f87e40090b0 tx=0x7f87e400bc30 comp rx=0 tx=0).stop 2024-05-09T00:55:26.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.097+0000 7f8808db7b80 1 -- 172.21.15.5:0/2826490276 shutdown_connections 2024-05-09T00:55:26.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.097+0000 7f8808db7b80 1 --2- 172.21.15.5:0/2826490276 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55c645c4ed50 0x55c645c51170 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:26.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.097+0000 7f8808db7b80 1 --2- 172.21.15.5:0/2826490276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c645c35480 0x55c645c3fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:26.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.097+0000 7f8808db7b80 1 --2- 172.21.15.5:0/2826490276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c645ab8520 0x55c645c3f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:26.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.097+0000 7f87d37fe700 0 client.25922 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:55:26.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.097+0000 7f8808db7b80 1 --2- 172.21.15.5:0/2826490276 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c645b32d80 0x55c645c3f5e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:26.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.097+0000 7f8808db7b80 1 -- 172.21.15.5:0/2826490276 >> 172.21.15.5:0/2826490276 conn(0x55c645abbff0 msgr2=0x55c645c2a070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:55:26.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.097+0000 7f8808db7b80 1 -- 172.21.15.5:0/2826490276 shutdown_connections 2024-05-09T00:55:26.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.097+0000 7f8808db7b80 1 -- 172.21.15.5:0/2826490276 wait complete. 2024-05-09T00:55:26.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.098+0000 7f8808db7b80 20 client.25922 trim_cache size 0 max 16384 2024-05-09T00:55:26.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:55:25 smithi131 ceph-mon[34547]: pgmap v643: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 896 MiB/s wr, 2.00k op/s 2024-05-09T00:55:26.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.141+0000 7f8808db7b80 1 Processor -- start 2024-05-09T00:55:26.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.141+0000 7f8808db7b80 1 -- start start 2024-05-09T00:55:26.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.142+0000 7f8808db7b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c645ab8520 0x55c645c2aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:26.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.142+0000 7f8808db7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c645b32d80 0x55c645c4b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:26.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.142+0000 7f8808db7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c645c35480 0x55c645c40e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:26.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.142+0000 7f8808db7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f87f40741a0 con 0x55c645b32d80 2024-05-09T00:55:26.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.142+0000 7f8808db7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f87f41bf2e0 con 0x55c645c35480 2024-05-09T00:55:26.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.142+0000 7f8808db7b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f87f4074870 con 0x55c645ab8520 2024-05-09T00:55:26.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.142+0000 7f87f37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c645b32d80 0x55c645c4b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:26.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.142+0000 7f87f37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c645b32d80 0x55c645c4b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59032/0 (socket says 172.21.15.5:59032) 2024-05-09T00:55:26.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.142+0000 7f87f37fe700 1 -- 172.21.15.5:0/2934428710 learned_addr learned my addr 172.21.15.5:0/2934428710 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:55:26.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.143+0000 7f87f893e700 1 --2- 172.21.15.5:0/2934428710 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c645c35480 0x55c645c40e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:26.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.143+0000 7f87f37fe700 1 -- 172.21.15.5:0/2934428710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c645ab8520 msgr2=0x55c645c2aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:55:26.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.143+0000 7f87f37fe700 1 --2- 172.21.15.5:0/2934428710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c645ab8520 0x55c645c2aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:26.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.143+0000 7f87f37fe700 1 -- 172.21.15.5:0/2934428710 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c645c35480 msgr2=0x55c645c40e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:55:26.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.143+0000 7f87f37fe700 1 --2- 172.21.15.5:0/2934428710 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c645c35480 0x55c645c40e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:26.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.143+0000 7f87f37fe700 1 -- 172.21.15.5:0/2934428710 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c645c2ca50 con 0x55c645b32d80 2024-05-09T00:55:26.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.144+0000 7f87f37fe700 1 --2- 172.21.15.5:0/2934428710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c645b32d80 0x55c645c4b270 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f87e4009770 tx=0x7f87e40090e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77460e2c5382a0f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:55:26.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.144+0000 7f87d9ffb700 1 -- 172.21.15.5:0/2934428710 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e4015070 con 0x55c645b32d80 2024-05-09T00:55:26.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.144+0000 7f87d9ffb700 1 -- 172.21.15.5:0/2934428710 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87e4033070 con 0x55c645b32d80 2024-05-09T00:55:26.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.144+0000 7f87d9ffb700 1 -- 172.21.15.5:0/2934428710 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e402e680 con 0x55c645b32d80 2024-05-09T00:55:26.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.145+0000 7f8808db7b80 1 -- 172.21.15.5:0/2934428710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c645b32d80 msgr2=0x55c645c4b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:55:26.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.145+0000 7f8808db7b80 1 --2- 172.21.15.5:0/2934428710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c645b32d80 0x55c645c4b270 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f87e4009770 tx=0x7f87e40090e0 comp rx=0 tx=0).stop 2024-05-09T00:55:26.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.146+0000 7f8808db7b80 1 -- 172.21.15.5:0/2934428710 shutdown_connections 2024-05-09T00:55:26.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.146+0000 7f8808db7b80 1 --2- 172.21.15.5:0/2934428710 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c645c35480 0x55c645c40e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:26.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.146+0000 7f8808db7b80 1 --2- 172.21.15.5:0/2934428710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c645ab8520 0x55c645c2aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:26.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.146+0000 7f8808db7b80 1 --2- 172.21.15.5:0/2934428710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c645b32d80 0x55c645c4b270 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:26.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.146+0000 7f8808db7b80 1 -- 172.21.15.5:0/2934428710 >> 172.21.15.5:0/2934428710 conn(0x55c645abbff0 msgr2=0x55c645c40750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:55:26.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.146+0000 7f8808db7b80 1 -- 172.21.15.5:0/2934428710 shutdown_connections 2024-05-09T00:55:26.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.146+0000 7f8808db7b80 1 -- 172.21.15.5:0/2934428710 wait complete. 2024-05-09T00:55:26.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.147+0000 7f8808db7b80 1 -- start start 2024-05-09T00:55:26.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.147+0000 7f8808db7b80 1 Processor -- start 2024-05-09T00:55:26.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.148+0000 7f8808db7b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c645ab8520 0x55c645c54480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:26.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.148+0000 7f8808db7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c645b32d80 0x55c645c52810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:26.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.148+0000 7f8808db7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c645c35480 0x55c645c52d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:26.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.148+0000 7f8808db7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f87f4075610 con 0x55c645c35480 2024-05-09T00:55:26.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.148+0000 7f8808db7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f87f4074f40 con 0x55c645b32d80 2024-05-09T00:55:26.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.148+0000 7f8808db7b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f87f40721f0 con 0x55c645ab8520 2024-05-09T00:55:26.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.148+0000 7f87f3fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c645ab8520 0x55c645c54480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:26.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.148+0000 7f87f3fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c645ab8520 0x55c645c54480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:35748/0 (socket says 172.21.15.5:35748) 2024-05-09T00:55:26.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.148+0000 7f87f3fff700 1 -- 172.21.15.5:0/1264155913 learned_addr learned my addr 172.21.15.5:0/1264155913 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:55:26.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.148+0000 7f87f893e700 1 --2- 172.21.15.5:0/1264155913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c645c35480 0x55c645c52d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:26.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.149+0000 7f87f37fe700 1 --2- 172.21.15.5:0/1264155913 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c645b32d80 0x55c645c52810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:26.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.149+0000 7f8808db7b80 10 client.0 _ll_register_callbacks cb 0x55c645c35e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:55:26.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.150+0000 7f87f3fff700 1 -- 172.21.15.5:0/1264155913 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c645b32d80 msgr2=0x55c645c52810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:55:26.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.150+0000 7f87f3fff700 1 --2- 172.21.15.5:0/1264155913 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c645b32d80 0x55c645c52810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:26.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.150+0000 7f87f3fff700 1 -- 172.21.15.5:0/1264155913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c645c35480 msgr2=0x55c645c52d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:55:26.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.150+0000 7f87f3fff700 1 --2- 172.21.15.5:0/1264155913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c645c35480 0x55c645c52d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:26.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.150+0000 7f87f3fff700 1 -- 172.21.15.5:0/1264155913 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c645c3bb70 con 0x55c645ab8520 2024-05-09T00:55:26.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.150+0000 7f87f893e700 1 --2- 172.21.15.5:0/1264155913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c645c35480 0x55c645c52d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:55:26.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.150+0000 7f8808db7b80 1 -- 172.21.15.5:0/1264155913 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87e4033070 con 0x55c645ab8520 2024-05-09T00:55:26.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.151+0000 7f87f3fff700 1 --2- 172.21.15.5:0/1264155913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c645ab8520 0x55c645c54480 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f87e8009720 tx=0x7f87e8000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b856470e6f4fbff7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:55:26.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.151+0000 7f87d37fe700 10 client.25954 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-09T00:55:26.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.154+0000 7f87d37fe700 1 -- 172.21.15.5:0/1264155913 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e8035060 con 0x55c645ab8520 2024-05-09T00:55:26.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.154+0000 7f87d37fe700 1 -- 172.21.15.5:0/1264155913 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f87e800bc70 con 0x55c645ab8520 2024-05-09T00:55:26.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.155+0000 7f87d37fe700 1 -- 172.21.15.5:0/1264155913 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f87e800be70 con 0x55c645ab8520 2024-05-09T00:55:26.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.155+0000 7f8808db7b80 10 client.25954 fetch_fsmap learned FSMap version 31 2024-05-09T00:55:26.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.155+0000 7f8808db7b80 1 -- 172.21.15.5:0/1264155913 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({fsmap=31}) -- 0x55c645cb9d50 con 0x55c645ab8520 2024-05-09T00:55:26.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.156+0000 7f87d37fe700 1 -- 172.21.15.5:0/1264155913 <== mon.1 v2:172.21.15.157:3300/0 4 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f87e8021040 con 0x55c645ab8520 2024-05-09T00:55:26.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.156+0000 7f8808db7b80 10 client.25954 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:55:26.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.156+0000 7f8808db7b80 10 client.25954 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:55:26.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.157+0000 7f8808db7b80 20 client.25954 populate_metadata read hostname 'smithi005' 2024-05-09T00:55:26.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.157+0000 7f8808db7b80 1 --2- 172.21.15.5:0/1264155913 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55c645c4ed50 0x55c645cbc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:26.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.157+0000 7f8808db7b80 4 client.25954 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T00:55:26.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.157+0000 7f8808db7b80 1 -- 172.21.15.5:0/1264155913 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c645c31f80 con 0x55c645c4ed50 2024-05-09T00:55:26.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.158+0000 7f87f37fe700 1 --2- 172.21.15.5:0/1264155913 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55c645c4ed50 0x55c645cbc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:26.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.162+0000 7f87f37fe700 1 --2- 172.21.15.5:0/1264155913 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55c645c4ed50 0x55c645cbc300 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a24bb355a6c646 server_cookie=c9cb234ec7ccbfab in_seq=0 out_seq=0 2024-05-09T00:55:26.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.162+0000 7f87d37fe700 10 client.25954 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:55:26.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.762+0000 7f87d37fe700 1 -- 172.21.15.5:0/1264155913 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c645c31f80 con 0x55c645c4ed50 2024-05-09T00:55:26.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.763+0000 7f87d37fe700 10 client.25954 handle_command_reply: tid=0 2024-05-09T00:55:26.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.763+0000 7f8808db7b80 1 client.25954 shutdown 2024-05-09T00:55:26.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.763+0000 7f8808db7b80 1 -- 172.21.15.5:0/1264155913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c645ab8520 msgr2=0x55c645c54480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:55:26.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.764+0000 7f8808db7b80 1 --2- 172.21.15.5:0/1264155913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c645ab8520 0x55c645c54480 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f87e8009720 tx=0x7f87e8000bc0 comp rx=0 tx=0).stop 2024-05-09T00:55:26.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.764+0000 7f8808db7b80 1 -- 172.21.15.5:0/1264155913 shutdown_connections 2024-05-09T00:55:26.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.764+0000 7f8808db7b80 1 --2- 172.21.15.5:0/1264155913 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55c645c4ed50 0x55c645cbc300 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:26.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.764+0000 7f8808db7b80 1 --2- 172.21.15.5:0/1264155913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c645c35480 0x55c645c52d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:26.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.764+0000 7f87d37fe700 0 client.25954 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:55:26.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.764+0000 7f8808db7b80 1 --2- 172.21.15.5:0/1264155913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c645ab8520 0x55c645c54480 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:26.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.764+0000 7f8808db7b80 1 --2- 172.21.15.5:0/1264155913 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c645b32d80 0x55c645c52810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:26.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.764+0000 7f8808db7b80 1 -- 172.21.15.5:0/1264155913 >> 172.21.15.5:0/1264155913 conn(0x55c645abbff0 msgr2=0x55c645c2a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:55:26.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.765+0000 7f8808db7b80 1 -- 172.21.15.5:0/1264155913 shutdown_connections 2024-05-09T00:55:26.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.765+0000 7f8808db7b80 1 -- 172.21.15.5:0/1264155913 wait complete. 2024-05-09T00:55:26.767 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-09T00:55:26.767 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-05-09T00:55:26.767 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "176d2d68-dd2a-4069-a614-d6fef3a5f1a9", 2024-05-09T00:55:26.767 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-05-09T00:55:26.767 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-09T00:55:26.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.765+0000 7f8808db7b80 20 client.25954 trim_cache size 0 max 16384 2024-05-09T00:55:26.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.768+0000 7f87fa3a1700 1 -- 172.21.15.5:0/782823851 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f87dc0b2960 msgr2=0x7f87dc0b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:55:26.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.768+0000 7f87fa3a1700 1 --2- 172.21.15.5:0/782823851 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f87dc0b2960 0x7f87dc0b4e70 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f87e8003800 tx=0x7f87e8019040 comp rx=0 tx=0).stop 2024-05-09T00:55:26.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.768+0000 7f87fa3a1700 1 -- 172.21.15.5:0/782823851 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87f4108710 msgr2=0x7f87f41a5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:55:26.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.768+0000 7f87fa3a1700 1 --2- 172.21.15.5:0/782823851 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87f4108710 0x7f87f41a5d20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f87ec00b970 tx=0x7f87ec00bd30 comp rx=0 tx=0).stop 2024-05-09T00:55:26.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.773+0000 7f87fa3a1700 1 -- 172.21.15.5:0/782823851 shutdown_connections 2024-05-09T00:55:26.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.773+0000 7f87fa3a1700 1 --2- 172.21.15.5:0/782823851 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f87dc0b2960 0x7f87dc0b4e70 secure :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f87e8003800 tx=0x7f87e8019040 comp rx=0 tx=0).stop 2024-05-09T00:55:26.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.773+0000 7f87fa3a1700 1 --2- 172.21.15.5:0/782823851 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87f4108710 0x7f87f41a5d20 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:26.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.773+0000 7f87fa3a1700 1 --2- 172.21.15.5:0/782823851 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87f4103270 0x7f87f41a14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:26.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.773+0000 7f87fa3a1700 1 --2- 172.21.15.5:0/782823851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87f4103c50 0x7f87f41a1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:26.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.773+0000 7f87fa3a1700 1 -- 172.21.15.5:0/782823851 >> 172.21.15.5:0/782823851 conn(0x7f87f40fef50 msgr2=0x7f87f4106090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:55:26.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.776+0000 7f87fa3a1700 1 -- 172.21.15.5:0/782823851 shutdown_connections 2024-05-09T00:55:26.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:26.776+0000 7f87fa3a1700 1 -- 172.21.15.5:0/782823851 wait complete. 2024-05-09T00:55:27.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:55:27 smithi131 ceph-mon[34547]: pgmap v644: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 919 MiB/s wr, 2.13k op/s 2024-05-09T00:55:27.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:55:27 smithi157 ceph-mon[34294]: pgmap v644: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 919 MiB/s wr, 2.13k op/s 2024-05-09T00:55:27.543 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:55:27 smithi005 ceph-mon[30365]: pgmap v644: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 919 MiB/s wr, 2.13k op/s 2024-05-09T00:55:28.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:55:28 smithi005 ceph-mon[30365]: scrub queued for path: / 2024-05-09T00:55:28.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:55:28 smithi005 ceph-mon[30365]: scrub summary: idle+waiting paths [/] 2024-05-09T00:55:28.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:55:28 smithi005 ceph-mon[30365]: scrub summary: active paths [/] 2024-05-09T00:55:28.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:55:28 smithi131 ceph-mon[34547]: scrub queued for path: / 2024-05-09T00:55:28.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:55:28 smithi131 ceph-mon[34547]: scrub summary: idle+waiting paths [/] 2024-05-09T00:55:28.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:55:28 smithi131 ceph-mon[34547]: scrub summary: active paths [/] 2024-05-09T00:55:28.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:55:28 smithi157 ceph-mon[34294]: scrub queued for path: / 2024-05-09T00:55:28.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:55:28 smithi157 ceph-mon[34294]: scrub summary: idle+waiting paths [/] 2024-05-09T00:55:28.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:55:28 smithi157 ceph-mon[34294]: scrub summary: active paths [/] 2024-05-09T00:55:28.952 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T00:55:29.117 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:55:29 smithi005 ceph-mon[30365]: pgmap v645: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 887 MiB/s wr, 1.94k op/s 2024-05-09T00:55:29.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:55:29 smithi157 ceph-mon[34294]: pgmap v645: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 887 MiB/s wr, 1.94k op/s 2024-05-09T00:55:29.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:55:29 smithi131 ceph-mon[34547]: pgmap v645: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 887 MiB/s wr, 1.94k op/s 2024-05-09T00:55:29.804 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:55:31.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.095+0000 7f6fa6d9d640 1 -- 172.21.15.5:0/2669560636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f980ab760 msgr2=0x7f6f980abb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:55:31.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.095+0000 7f6fa6d9d640 1 --2- 172.21.15.5:0/2669560636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f980ab760 0x7f6f980abb70 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f6f9001a910 tx=0x7f6f9003b450 comp rx=0 tx=0).stop 2024-05-09T00:55:31.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.100+0000 7f6fa6d9d640 1 -- 172.21.15.5:0/2669560636 shutdown_connections 2024-05-09T00:55:31.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.100+0000 7f6fa6d9d640 1 --2- 172.21.15.5:0/2669560636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f980ab760 0x7f6f980abb70 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:31.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.100+0000 7f6fa6d9d640 1 --2- 172.21.15.5:0/2669560636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f980a6a40 0x7f6f980a6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:31.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.100+0000 7f6fa6d9d640 1 --2- 172.21.15.5:0/2669560636 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f980a5940 0x7f6f980a6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:31.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.100+0000 7f6fa6d9d640 1 -- 172.21.15.5:0/2669560636 >> 172.21.15.5:0/2669560636 conn(0x7f6f980a09b0 msgr2=0x7f6f980a2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:55:31.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.107+0000 7f6fa6d9d640 1 -- 172.21.15.5:0/2669560636 shutdown_connections 2024-05-09T00:55:31.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.116+0000 7f6fa6d9d640 1 -- 172.21.15.5:0/2669560636 wait complete. 2024-05-09T00:55:31.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.121+0000 7f6fa6d9d640 1 Processor -- start 2024-05-09T00:55:31.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.122+0000 7f6fa6d9d640 1 -- start start 2024-05-09T00:55:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.123+0000 7f6fa6d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f980a5940 0x7f6f9813bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.123+0000 7f6fa6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f980a6a40 0x7f6f9813c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.123+0000 7f6fa6d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f980ab760 0x7f6f981433a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.124+0000 7f6fa6d9d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6f980af390 con 0x7f6f980a6a40 2024-05-09T00:55:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.124+0000 7f6fa6d9d640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6f980af210 con 0x7f6f980ab760 2024-05-09T00:55:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.124+0000 7f6fa6d9d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6f980af510 con 0x7f6f980a5940 2024-05-09T00:55:31.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.125+0000 7f6fa659c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f980ab760 0x7f6f981433a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:31.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.126+0000 7f6fa659c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f980ab760 0x7f6f981433a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:59190/0 (socket says 172.21.15.5:59190) 2024-05-09T00:55:31.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.126+0000 7f6fa659c640 1 -- 172.21.15.5:0/4236442311 learned_addr learned my addr 172.21.15.5:0/4236442311 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:55:31.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.129+0000 7f6fa659c640 1 -- 172.21.15.5:0/4236442311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f980a5940 msgr2=0x7f6f9813bed0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T00:55:31.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.129+0000 7f6fa659c640 1 --2- 172.21.15.5:0/4236442311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f980a5940 0x7f6f9813bed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:31.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.129+0000 7f6fa659c640 1 -- 172.21.15.5:0/4236442311 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f980a6a40 msgr2=0x7f6f9813c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:55:31.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.129+0000 7f6fa659c640 1 --2- 172.21.15.5:0/4236442311 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f980a6a40 0x7f6f9813c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:31.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.129+0000 7f6fa659c640 1 -- 172.21.15.5:0/4236442311 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f98143a10 con 0x7f6f980ab760 2024-05-09T00:55:31.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.133+0000 7f6fa659c640 1 --2- 172.21.15.5:0/4236442311 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f980ab760 0x7f6f981433a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6f90004a60 tx=0x7f6f90004a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:55:31.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.135+0000 7f6f96ffd640 1 -- 172.21.15.5:0/4236442311 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9004f070 con 0x7f6f980ab760 2024-05-09T00:55:31.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.135+0000 7f6f96ffd640 1 -- 172.21.15.5:0/4236442311 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f90040bd0 con 0x7f6f980ab760 2024-05-09T00:55:31.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.136+0000 7f6f96ffd640 1 -- 172.21.15.5:0/4236442311 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f90046030 con 0x7f6f980ab760 2024-05-09T00:55:31.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.138+0000 7f6fa6d9d640 1 -- 172.21.15.5:0/4236442311 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f98143ca0 con 0x7f6f980ab760 2024-05-09T00:55:31.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.138+0000 7f6fa6d9d640 1 -- 172.21.15.5:0/4236442311 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f98144050 con 0x7f6f980ab760 2024-05-09T00:55:31.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.156+0000 7f6faef86740 1 Processor -- start 2024-05-09T00:55:31.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.157+0000 7f6faef86740 1 -- start start 2024-05-09T00:55:31.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.157+0000 7f6faef86740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641ad4ea1b0 0x5641ad88e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:31.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.157+0000 7f6faef86740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641ad4c12f0 0x5641ad88ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:31.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.157+0000 7f6faef86740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5641ad4c94d0 0x5641ad895b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:31.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.158+0000 7f6faef86740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5641ad7f27c0 con 0x5641ad4ea1b0 2024-05-09T00:55:31.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.158+0000 7f6faef86740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5641ad78e0e0 con 0x5641ad4c12f0 2024-05-09T00:55:31.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.158+0000 7f6faef86740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5641ad7ee780 con 0x5641ad4c94d0 2024-05-09T00:55:31.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.158+0000 7f6f96ffd640 1 -- 172.21.15.5:0/4236442311 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f6f90052070 con 0x7f6f980ab760 2024-05-09T00:55:31.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.160+0000 7f6f96ffd640 1 --2- 172.21.15.5:0/4236442311 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f6f8c0b2850 0x7f6f8c0b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:31.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.161+0000 7f6fa5d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641ad4c12f0 0x5641ad88ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:31.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.161+0000 7f6fa5d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641ad4c12f0 0x5641ad88ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:59198/0 (socket says 172.21.15.5:59198) 2024-05-09T00:55:31.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.161+0000 7f6fa5d9b640 1 -- 172.21.15.5:0/1178070179 learned_addr learned my addr 172.21.15.5:0/1178070179 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:55:31.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.162+0000 7f6f96ffd640 1 -- 172.21.15.5:0/4236442311 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f6f90113300 con 0x7f6f980ab760 2024-05-09T00:55:31.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.162+0000 7f6fa559a640 1 --2- 172.21.15.5:0/1178070179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641ad4ea1b0 0x5641ad88e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:31.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.162+0000 7f6fa659c640 1 --2- 172.21.15.5:0/4236442311 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f6f8c0b2850 0x7f6f8c0b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:31.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.162+0000 7f6fa559a640 1 --2- 172.21.15.5:0/1178070179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5641ad4c94d0 0x5641ad895b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:31.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.163+0000 7f6fa559a640 1 -- 172.21.15.5:0/1178070179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5641ad4c94d0 msgr2=0x5641ad895b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:55:31.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.163+0000 7f6fa559a640 1 --2- 172.21.15.5:0/1178070179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5641ad4c94d0 0x5641ad895b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:31.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.163+0000 7f6fa559a640 1 -- 172.21.15.5:0/1178070179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641ad4c12f0 msgr2=0x5641ad88ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:55:31.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.163+0000 7f6fa559a640 1 --2- 172.21.15.5:0/1178070179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641ad4c12f0 0x5641ad88ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:31.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.163+0000 7f6fa559a640 1 -- 172.21.15.5:0/1178070179 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641ad7b77c0 con 0x5641ad4ea1b0 2024-05-09T00:55:31.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.164+0000 7f6fa5d9b640 1 --2- 172.21.15.5:0/1178070179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641ad4c12f0 0x5641ad88ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:55:31.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.164+0000 7f6fa559a640 1 --2- 172.21.15.5:0/1178070179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641ad4ea1b0 0x5641ad88e6c0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f6fa0052220 tx=0x7f6fa0064250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16e1b92a9f009ee8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:55:31.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.167+0000 7f6f8bfff640 1 -- 172.21.15.5:0/1178070179 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa006b0d0 con 0x5641ad4ea1b0 2024-05-09T00:55:31.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.167+0000 7f6f8bfff640 1 -- 172.21.15.5:0/1178070179 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fa0076400 con 0x5641ad4ea1b0 2024-05-09T00:55:31.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.168+0000 7f6f8bfff640 1 -- 172.21.15.5:0/1178070179 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa00755c0 con 0x5641ad4ea1b0 2024-05-09T00:55:31.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.169+0000 7f6faef86740 1 -- 172.21.15.5:0/1178070179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641ad4ea1b0 msgr2=0x5641ad88e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:55:31.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.169+0000 7f6faef86740 1 --2- 172.21.15.5:0/1178070179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641ad4ea1b0 0x5641ad88e6c0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f6fa0052220 tx=0x7f6fa0064250 comp rx=0 tx=0).stop 2024-05-09T00:55:31.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.169+0000 7f6fa659c640 1 --2- 172.21.15.5:0/4236442311 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f6f8c0b2850 0x7f6f8c0b4d60 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f6f900054a0 tx=0x7f6f9003b950 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:55:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.170+0000 7f6faef86740 1 -- 172.21.15.5:0/1178070179 shutdown_connections 2024-05-09T00:55:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.170+0000 7f6faef86740 1 --2- 172.21.15.5:0/1178070179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5641ad4c94d0 0x5641ad895b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.170+0000 7f6faef86740 1 --2- 172.21.15.5:0/1178070179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641ad4c12f0 0x5641ad88ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.170+0000 7f6faef86740 1 --2- 172.21.15.5:0/1178070179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641ad4ea1b0 0x5641ad88e6c0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.170+0000 7f6faef86740 1 -- 172.21.15.5:0/1178070179 >> 172.21.15.5:0/1178070179 conn(0x5641ad4de300 msgr2=0x5641ad88a050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:55:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.171+0000 7f6faef86740 1 -- 172.21.15.5:0/1178070179 shutdown_connections 2024-05-09T00:55:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.172+0000 7f6faef86740 1 -- 172.21.15.5:0/1178070179 wait complete. 2024-05-09T00:55:31.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.174+0000 7f6faef86740 1 -- start start 2024-05-09T00:55:31.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.174+0000 7f6faef86740 1 Processor -- start 2024-05-09T00:55:31.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.176+0000 7f6faef86740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641ad4c12f0 0x5641ad898480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.176+0000 7f6faef86740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5641ad4c94d0 0x5641ad8989c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.176+0000 7f6faef86740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641ad4ea1b0 0x5641ad89f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.176+0000 7f6faef86740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5641ad896530 con 0x5641ad4c12f0 2024-05-09T00:55:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.176+0000 7f6faef86740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5641ad7799c0 con 0x5641ad4ea1b0 2024-05-09T00:55:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.176+0000 7f6faef86740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5641ad8966b0 con 0x5641ad4c94d0 2024-05-09T00:55:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.176+0000 7f6faef86740 10 client.0 _ll_register_callbacks cb 0x5641ad7a6020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:55:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.177+0000 7f6fa559a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641ad4c12f0 0x5641ad898480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.177+0000 7f6fa559a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641ad4c12f0 0x5641ad898480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59078/0 (socket says 172.21.15.5:59078) 2024-05-09T00:55:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.177+0000 7f6fa559a640 1 -- 172.21.15.5:0/3421330064 learned_addr learned my addr 172.21.15.5:0/3421330064 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:55:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.177+0000 7f6fa559a640 1 --2- 172.21.15.5:0/3421330064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641ad4ea1b0 0x5641ad89f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.177+0000 7f6fa559a640 1 -- 172.21.15.5:0/3421330064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5641ad4c94d0 msgr2=0x5641ad8989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:55:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.177+0000 7f6fa559a640 1 --2- 172.21.15.5:0/3421330064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5641ad4c94d0 0x5641ad8989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.177+0000 7f6fa559a640 1 -- 172.21.15.5:0/3421330064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641ad4ea1b0 msgr2=0x5641ad89f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:55:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.177+0000 7f6fa559a640 1 --2- 172.21.15.5:0/3421330064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641ad4ea1b0 0x5641ad89f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.178+0000 7f6fa559a640 1 -- 172.21.15.5:0/3421330064 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641ad7a01f0 con 0x5641ad4c12f0 2024-05-09T00:55:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.179+0000 7f6fa559a640 1 --2- 172.21.15.5:0/3421330064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641ad4c12f0 0x5641ad898480 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f6fa0068bf0 tx=0x7f6fa0068cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a80be1c527f9cb82 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:55:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.179+0000 7f6f897fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:55:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.180+0000 7f6f897fa640 1 -- 172.21.15.5:0/3421330064 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa006aa70 con 0x5641ad4c12f0 2024-05-09T00:55:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.180+0000 7f6f897fa640 1 -- 172.21.15.5:0/3421330064 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f6fa006f040 con 0x5641ad4c12f0 2024-05-09T00:55:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.180+0000 7f6faef86740 1 -- 172.21.15.5:0/3421330064 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fa0076400 con 0x5641ad4c12f0 2024-05-09T00:55:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.181+0000 7f6f897fa640 1 -- 172.21.15.5:0/3421330064 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6fa0088420 con 0x5641ad4c12f0 2024-05-09T00:55:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.181+0000 7f6faef86740 10 client.16440 fetch_fsmap learned FSMap version 31 2024-05-09T00:55:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.181+0000 7f6faef86740 1 -- 172.21.15.5:0/3421330064 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5641ad555790 con 0x5641ad4c12f0 2024-05-09T00:55:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.182+0000 7f6f897fa640 1 -- 172.21.15.5:0/3421330064 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fa0087880 con 0x5641ad4c12f0 2024-05-09T00:55:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.182+0000 7f6f897fa640 1 -- 172.21.15.5:0/3421330064 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f6fa0072020 con 0x5641ad4c12f0 2024-05-09T00:55:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.182+0000 7f6faef86740 10 client.16440 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:55:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.182+0000 7f6faef86740 10 client.16440 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:55:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.182+0000 7f6faef86740 20 client.16440 populate_metadata read hostname 'smithi005' 2024-05-09T00:55:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.182+0000 7f6faef86740 1 --2- 172.21.15.5:0/3421330064 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5641ad4a8d80 0x5641ad8a58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.182+0000 7f6faef86740 4 client.16440 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:55:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.183+0000 7f6faef86740 1 -- 172.21.15.5:0/3421330064 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6fa0076400 con 0x5641ad4a8d80 2024-05-09T00:55:31.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.184+0000 7f6fa5d9b640 1 --2- 172.21.15.5:0/3421330064 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5641ad4a8d80 0x5641ad8a58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.187+0000 7f6fa5d9b640 1 --2- 172.21.15.5:0/3421330064 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5641ad4a8d80 0x5641ad8a58d0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d8d3c1530937337 server_cookie=7217772f7cc77f81 in_seq=0 out_seq=0 2024-05-09T00:55:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.187+0000 7f6f897fa640 10 client.16440 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:55:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.196+0000 7f6f897fa640 1 -- 172.21.15.5:0/3421330064 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f6fa0076400 con 0x5641ad4a8d80 2024-05-09T00:55:31.198 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.196+0000 7f6f897fa640 10 client.16440 handle_command_reply: tid=0 2024-05-09T00:55:31.198 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.198+0000 7f6faef86740 1 client.16440 shutdown 2024-05-09T00:55:31.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.202+0000 7f6faef86740 1 -- 172.21.15.5:0/3421330064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641ad4c12f0 msgr2=0x5641ad898480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:55:31.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.202+0000 7f6faef86740 1 --2- 172.21.15.5:0/3421330064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641ad4c12f0 0x5641ad898480 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f6fa0068bf0 tx=0x7f6fa0068cd0 comp rx=0 tx=0).stop 2024-05-09T00:55:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.203+0000 7f6faef86740 1 -- 172.21.15.5:0/3421330064 shutdown_connections 2024-05-09T00:55:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.203+0000 7f6faef86740 1 --2- 172.21.15.5:0/3421330064 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5641ad4a8d80 0x5641ad8a58d0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.204+0000 7f6f897fa640 0 client.16440 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:55:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.204+0000 7f6faef86740 1 --2- 172.21.15.5:0/3421330064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641ad4ea1b0 0x5641ad89f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.204+0000 7f6faef86740 1 --2- 172.21.15.5:0/3421330064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5641ad4c94d0 0x5641ad8989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.204+0000 7f6faef86740 1 --2- 172.21.15.5:0/3421330064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641ad4c12f0 0x5641ad898480 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.204+0000 7f6faef86740 1 -- 172.21.15.5:0/3421330064 >> 172.21.15.5:0/3421330064 conn(0x5641ad4de300 msgr2=0x5641ad888f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:55:31.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.206+0000 7f6faef86740 1 -- 172.21.15.5:0/3421330064 shutdown_connections 2024-05-09T00:55:31.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.208+0000 7f6faef86740 1 -- 172.21.15.5:0/3421330064 wait complete. 2024-05-09T00:55:31.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.208+0000 7f6faef86740 20 client.16440 trim_cache size 0 max 16384 2024-05-09T00:55:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.254+0000 7f6faef86740 1 Processor -- start 2024-05-09T00:55:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.254+0000 7f6faef86740 1 -- start start 2024-05-09T00:55:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.254+0000 7f6faef86740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5641ad4a8d80 0x5641ad8aa0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.254+0000 7f6faef86740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641ad4c12f0 0x5641ad89f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.254+0000 7f6faef86740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641ad4c94d0 0x5641ad89fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.255+0000 7f6faef86740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6f9814ce70 con 0x5641ad4c12f0 2024-05-09T00:55:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.255+0000 7f6faef86740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6f9814e9b0 con 0x5641ad4c94d0 2024-05-09T00:55:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.255+0000 7f6faef86740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6f9814d540 con 0x5641ad4a8d80 2024-05-09T00:55:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.255+0000 7f6fa559a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5641ad4a8d80 0x5641ad8aa0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.255+0000 7f6fa5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641ad4c12f0 0x5641ad89f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.255+0000 7f6fa5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641ad4c12f0 0x5641ad89f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59086/0 (socket says 172.21.15.5:59086) 2024-05-09T00:55:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.255+0000 7f6fa559a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5641ad4a8d80 0x5641ad8aa0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:35802/0 (socket says 172.21.15.5:35802) 2024-05-09T00:55:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.255+0000 7f6fa5d9b640 1 -- 172.21.15.5:0/3792024782 learned_addr learned my addr 172.21.15.5:0/3792024782 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:55:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.255+0000 7f6fa559a640 1 --2- 172.21.15.5:0/3792024782 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641ad4c94d0 0x5641ad89fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.255+0000 7f6fa5d9b640 1 -- 172.21.15.5:0/3792024782 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5641ad4a8d80 msgr2=0x5641ad8aa0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:55:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.255+0000 7f6fa5d9b640 1 --2- 172.21.15.5:0/3792024782 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5641ad4a8d80 0x5641ad8aa0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.255+0000 7f6fa5d9b640 1 -- 172.21.15.5:0/3792024782 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641ad4c94d0 msgr2=0x5641ad89fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:55:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.255+0000 7f6fa5d9b640 1 --2- 172.21.15.5:0/3792024782 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641ad4c94d0 0x5641ad89fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.256+0000 7f6fa5d9b640 1 -- 172.21.15.5:0/3792024782 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641ad5cc5f0 con 0x5641ad4c12f0 2024-05-09T00:55:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.256+0000 7f6fa5d9b640 1 --2- 172.21.15.5:0/3792024782 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641ad4c12f0 0x5641ad89f8b0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x5641ad8994f0 tx=0x7f6f9c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=187716ec8328009c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:55:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.256+0000 7f6f8b7fe640 1 -- 172.21.15.5:0/3792024782 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c01d030 con 0x5641ad4c12f0 2024-05-09T00:55:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.257+0000 7f6f8b7fe640 1 -- 172.21.15.5:0/3792024782 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f9c011df0 con 0x5641ad4c12f0 2024-05-09T00:55:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.257+0000 7f6f8b7fe640 1 -- 172.21.15.5:0/3792024782 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c0035f0 con 0x5641ad4c12f0 2024-05-09T00:55:31.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.258+0000 7f6faef86740 1 -- 172.21.15.5:0/3792024782 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641ad4c12f0 msgr2=0x5641ad89f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:55:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.258+0000 7f6faef86740 1 --2- 172.21.15.5:0/3792024782 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641ad4c12f0 0x5641ad89f8b0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x5641ad8994f0 tx=0x7f6f9c011810 comp rx=0 tx=0).stop 2024-05-09T00:55:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.259+0000 7f6faef86740 1 -- 172.21.15.5:0/3792024782 shutdown_connections 2024-05-09T00:55:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.259+0000 7f6faef86740 1 --2- 172.21.15.5:0/3792024782 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641ad4c94d0 0x5641ad89fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.259+0000 7f6faef86740 1 --2- 172.21.15.5:0/3792024782 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641ad4c12f0 0x5641ad89f8b0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.259+0000 7f6faef86740 1 --2- 172.21.15.5:0/3792024782 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5641ad4a8d80 0x5641ad8aa0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.259+0000 7f6faef86740 1 -- 172.21.15.5:0/3792024782 >> 172.21.15.5:0/3792024782 conn(0x5641ad4de300 msgr2=0x5641ad88d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:55:31.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.259+0000 7f6faef86740 1 -- 172.21.15.5:0/3792024782 shutdown_connections 2024-05-09T00:55:31.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.259+0000 7f6faef86740 1 -- 172.21.15.5:0/3792024782 wait complete. 2024-05-09T00:55:31.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.260+0000 7f6faef86740 1 -- start start 2024-05-09T00:55:31.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.260+0000 7f6faef86740 1 Processor -- start 2024-05-09T00:55:31.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.260+0000 7f6faef86740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5641ad4a8d80 0x5641ad89d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:31.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.260+0000 7f6faef86740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641ad4c12f0 0x5641ad89d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:31.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.260+0000 7f6faef86740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641ad4c94d0 0x5641ad90bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:31.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.260+0000 7f6faef86740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6f9814e2e0 con 0x5641ad4c12f0 2024-05-09T00:55:31.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.260+0000 7f6faef86740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6f9814dc10 con 0x5641ad4c94d0 2024-05-09T00:55:31.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.260+0000 7f6faef86740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6f9814ae50 con 0x5641ad4a8d80 2024-05-09T00:55:31.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.260+0000 7f6faef86740 10 client.0 _ll_register_callbacks cb 0x7f6fa006bb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:55:31.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.260+0000 7f6fa5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641ad4c12f0 0x5641ad89d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:31.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.260+0000 7f6fa5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641ad4c12f0 0x5641ad89d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59094/0 (socket says 172.21.15.5:59094) 2024-05-09T00:55:31.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.260+0000 7f6fa5d9b640 1 -- 172.21.15.5:0/4097065242 learned_addr learned my addr 172.21.15.5:0/4097065242 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:55:31.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.261+0000 7f6fa5d9b640 1 -- 172.21.15.5:0/4097065242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5641ad4a8d80 msgr2=0x5641ad89d390 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:55:31.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.261+0000 7f6fa5d9b640 1 --2- 172.21.15.5:0/4097065242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5641ad4a8d80 0x5641ad89d390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:31.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.261+0000 7f6fa5d9b640 1 -- 172.21.15.5:0/4097065242 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641ad4c94d0 msgr2=0x5641ad90bc70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:55:31.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.261+0000 7f6fa5d9b640 1 --2- 172.21.15.5:0/4097065242 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641ad4c94d0 0x5641ad90bc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:31.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.261+0000 7f6fa5d9b640 1 -- 172.21.15.5:0/4097065242 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641ad7b6ed0 con 0x5641ad4c12f0 2024-05-09T00:55:31.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.261+0000 7f6fa5d9b640 1 --2- 172.21.15.5:0/4097065242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641ad4c12f0 0x5641ad89d8d0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c019ad0 tx=0x7f6f9c019b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c179178be41087c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:55:31.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.261+0000 7f6f897fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:55:31.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.261+0000 7f6f897fa640 1 -- 172.21.15.5:0/4097065242 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c004020 con 0x5641ad4c12f0 2024-05-09T00:55:31.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.261+0000 7f6f897fa640 1 -- 172.21.15.5:0/4097065242 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f6f9c0041c0 con 0x5641ad4c12f0 2024-05-09T00:55:31.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.261+0000 7f6faef86740 1 -- 172.21.15.5:0/4097065242 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f9c011df0 con 0x5641ad4c12f0 2024-05-09T00:55:31.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.262+0000 7f6f897fa640 1 -- 172.21.15.5:0/4097065242 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6f9c0043c0 con 0x5641ad4c12f0 2024-05-09T00:55:31.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.262+0000 7f6faef86740 10 client.16446 fetch_fsmap learned FSMap version 31 2024-05-09T00:55:31.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.262+0000 7f6faef86740 1 -- 172.21.15.5:0/4097065242 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5641ad90cfd0 con 0x5641ad4c12f0 2024-05-09T00:55:31.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.262+0000 7f6f897fa640 1 -- 172.21.15.5:0/4097065242 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f9c011df0 con 0x5641ad4c12f0 2024-05-09T00:55:31.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.262+0000 7f6f897fa640 1 -- 172.21.15.5:0/4097065242 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f6f9c035070 con 0x5641ad4c12f0 2024-05-09T00:55:31.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.262+0000 7f6faef86740 10 client.16446 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:55:31.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.262+0000 7f6faef86740 10 client.16446 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:55:31.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.262+0000 7f6faef86740 20 client.16446 populate_metadata read hostname 'smithi005' 2024-05-09T00:55:31.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.262+0000 7f6faef86740 1 --2- 172.21.15.5:0/4097065242 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5641ad4ea1b0 0x5641ad90f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:31.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.262+0000 7f6faef86740 4 client.16446 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:55:31.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.262+0000 7f6faef86740 1 -- 172.21.15.5:0/4097065242 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5641ad8003d0 con 0x5641ad4ea1b0 2024-05-09T00:55:31.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.265+0000 7f6fa559a640 1 --2- 172.21.15.5:0/4097065242 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5641ad4ea1b0 0x5641ad90f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:31.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.272+0000 7f6fa559a640 1 --2- 172.21.15.5:0/4097065242 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5641ad4ea1b0 0x5641ad90f920 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=336e7ca0e7b72198 server_cookie=4ecf2652c8806e6 in_seq=0 out_seq=0 2024-05-09T00:55:31.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:31.272+0000 7f6f897fa640 10 client.16446 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:55:31.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:55:31 smithi005 ceph-mon[30365]: scrub summary: active paths [/] 2024-05-09T00:55:31.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:55:31 smithi005 ceph-mon[30365]: scrub summary: idle+waiting paths [/] 2024-05-09T00:55:31.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:55:31 smithi005 ceph-mon[30365]: pgmap v646: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1003 MiB/s wr, 2.16k op/s 2024-05-09T00:55:31.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:55:31 smithi131 ceph-mon[34547]: scrub summary: active paths [/] 2024-05-09T00:55:31.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:55:31 smithi131 ceph-mon[34547]: scrub summary: idle+waiting paths [/] 2024-05-09T00:55:31.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:55:31 smithi131 ceph-mon[34547]: pgmap v646: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1003 MiB/s wr, 2.16k op/s 2024-05-09T00:55:31.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:55:31 smithi157 ceph-mon[34294]: scrub summary: active paths [/] 2024-05-09T00:55:31.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:55:31 smithi157 ceph-mon[34294]: scrub summary: idle+waiting paths [/] 2024-05-09T00:55:31.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:55:31 smithi157 ceph-mon[34294]: pgmap v646: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1003 MiB/s wr, 2.16k op/s 2024-05-09T00:55:33.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:55:33 smithi131 ceph-mon[34547]: pgmap v647: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 822 MiB/s wr, 1.82k op/s 2024-05-09T00:55:33.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:55:33 smithi131 ceph-mon[34547]: scrub summary: idle+waiting paths [/] 2024-05-09T00:55:33.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:55:33 smithi157 ceph-mon[34294]: pgmap v647: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 822 MiB/s wr, 1.82k op/s 2024-05-09T00:55:33.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:55:33 smithi157 ceph-mon[34294]: scrub summary: idle+waiting paths [/] 2024-05-09T00:55:33.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:55:33 smithi005 ceph-mon[30365]: pgmap v647: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 822 MiB/s wr, 1.82k op/s 2024-05-09T00:55:33.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:55:33 smithi005 ceph-mon[30365]: scrub summary: idle+waiting paths [/] 2024-05-09T00:55:35.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:35.247+0000 7f6f897fa640 1 -- 172.21.15.5:0/4097065242 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5641ad8003d0 con 0x5641ad4ea1b0 2024-05-09T00:55:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:35.248+0000 7f6f897fa640 10 client.16446 handle_command_reply: tid=0 2024-05-09T00:55:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:35.248+0000 7f6faef86740 1 client.16446 shutdown 2024-05-09T00:55:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:35.248+0000 7f6faef86740 1 -- 172.21.15.5:0/4097065242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641ad4c12f0 msgr2=0x5641ad89d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:55:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:35.248+0000 7f6faef86740 1 --2- 172.21.15.5:0/4097065242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641ad4c12f0 0x5641ad89d8d0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c019ad0 tx=0x7f6f9c019b00 comp rx=0 tx=0).stop 2024-05-09T00:55:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:35.248+0000 7f6faef86740 1 -- 172.21.15.5:0/4097065242 shutdown_connections 2024-05-09T00:55:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:35.248+0000 7f6faef86740 1 --2- 172.21.15.5:0/4097065242 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5641ad4ea1b0 0x5641ad90f920 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:35.248+0000 7f6faef86740 1 --2- 172.21.15.5:0/4097065242 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5641ad4c94d0 0x5641ad90bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:35.248+0000 7f6faef86740 1 --2- 172.21.15.5:0/4097065242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641ad4c12f0 0x5641ad89d8d0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:35.248+0000 7f6faef86740 1 --2- 172.21.15.5:0/4097065242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5641ad4a8d80 0x5641ad89d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:35.248+0000 7f6faef86740 1 -- 172.21.15.5:0/4097065242 >> 172.21.15.5:0/4097065242 conn(0x5641ad4de300 msgr2=0x5641ad88d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:55:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:35.248+0000 7f6f897fa640 0 client.16446 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:55:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:35.249+0000 7f6faef86740 1 -- 172.21.15.5:0/4097065242 shutdown_connections 2024-05-09T00:55:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:35.249+0000 7f6faef86740 1 -- 172.21.15.5:0/4097065242 wait complete. 2024-05-09T00:55:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:35.253+0000 7f6faef86740 20 client.16446 trim_cache size 0 max 16384 2024-05-09T00:55:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-09T00:55:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-09T00:55:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-09T00:55:35.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:35.266+0000 7f6fa6d9d640 1 -- 172.21.15.5:0/4236442311 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f6f8c0b2850 msgr2=0x7f6f8c0b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:55:35.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:35.266+0000 7f6fa6d9d640 1 --2- 172.21.15.5:0/4236442311 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f6f8c0b2850 0x7f6f8c0b4d60 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f6f900054a0 tx=0x7f6f9003b950 comp rx=0 tx=0).stop 2024-05-09T00:55:35.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:35.266+0000 7f6fa6d9d640 1 -- 172.21.15.5:0/4236442311 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f980ab760 msgr2=0x7f6f981433a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:55:35.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:35.266+0000 7f6fa6d9d640 1 --2- 172.21.15.5:0/4236442311 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f980ab760 0x7f6f981433a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6f90004a60 tx=0x7f6f90004a90 comp rx=0 tx=0).stop 2024-05-09T00:55:35.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:35.267+0000 7f6fa6d9d640 1 -- 172.21.15.5:0/4236442311 shutdown_connections 2024-05-09T00:55:35.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:35.267+0000 7f6fa6d9d640 1 --2- 172.21.15.5:0/4236442311 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f6f8c0b2850 0x7f6f8c0b4d60 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:35.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:35.267+0000 7f6fa6d9d640 1 --2- 172.21.15.5:0/4236442311 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f980ab760 0x7f6f981433a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:35.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:35.267+0000 7f6fa6d9d640 1 --2- 172.21.15.5:0/4236442311 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f980a6a40 0x7f6f9813c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:35.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:35.267+0000 7f6fa6d9d640 1 --2- 172.21.15.5:0/4236442311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f980a5940 0x7f6f9813bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:35.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:35.267+0000 7f6fa6d9d640 1 -- 172.21.15.5:0/4236442311 >> 172.21.15.5:0/4236442311 conn(0x7f6f980a09b0 msgr2=0x7f6f980a9750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:55:35.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:35.268+0000 7f6fa6d9d640 1 -- 172.21.15.5:0/4236442311 shutdown_connections 2024-05-09T00:55:35.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:35.269+0000 7f6fa6d9d640 1 -- 172.21.15.5:0/4236442311 wait complete. 2024-05-09T00:55:35.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.384+0000 7f493bfff700 1 -- 172.21.15.5:0/3108575751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f493c110fc0 msgr2=0x7f493c10ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:55:35.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.384+0000 7f493bfff700 1 --2- 172.21.15.5:0/3108575751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f493c110fc0 0x7f493c10ad80 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f493400b0a0 tx=0x7f493400b3b0 comp rx=0 tx=0).stop 2024-05-09T00:55:35.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.389+0000 7f493bfff700 1 -- 172.21.15.5:0/3108575751 shutdown_connections 2024-05-09T00:55:35.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.389+0000 7f493bfff700 1 --2- 172.21.15.5:0/3108575751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f493c110fc0 0x7f493c10ad80 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:35.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.389+0000 7f493bfff700 1 --2- 172.21.15.5:0/3108575751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f493c112740 0x7f493c110500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:35.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.389+0000 7f493bfff700 1 --2- 172.21.15.5:0/3108575751 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f493c110ad0 0x7f493c10a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:35.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.389+0000 7f493bfff700 1 -- 172.21.15.5:0/3108575751 >> 172.21.15.5:0/3108575751 conn(0x7f493c06f260 msgr2=0x7f493c06f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:55:35.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.392+0000 7f493bfff700 1 -- 172.21.15.5:0/3108575751 shutdown_connections 2024-05-09T00:55:35.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.392+0000 7f493bfff700 1 -- 172.21.15.5:0/3108575751 wait complete. 2024-05-09T00:55:35.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.393+0000 7f493bfff700 1 Processor -- start 2024-05-09T00:55:35.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.394+0000 7f493bfff700 1 -- start start 2024-05-09T00:55:35.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.395+0000 7f493bfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f493c110ad0 0x7f493c120d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:35.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.395+0000 7f493bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f493c110fc0 0x7f493c119eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:35.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.395+0000 7f493bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f493c112740 0x7f493c11a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:35.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.396+0000 7f493bfff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f493c1230f0 con 0x7f493c112740 2024-05-09T00:55:35.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.396+0000 7f493affd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f493c110ad0 0x7f493c120d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:35.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.396+0000 7f493affd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f493c110ad0 0x7f493c120d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:32976/0 (socket says 172.21.15.5:32976) 2024-05-09T00:55:35.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.396+0000 7f493affd700 1 -- 172.21.15.5:0/1758911213 learned_addr learned my addr 172.21.15.5:0/1758911213 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:55:35.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.396+0000 7f493bfff700 1 -- 172.21.15.5:0/1758911213 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f493c122f70 con 0x7f493c110fc0 2024-05-09T00:55:35.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.396+0000 7f493bfff700 1 -- 172.21.15.5:0/1758911213 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f493c123270 con 0x7f493c110ad0 2024-05-09T00:55:35.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.398+0000 7f493a7fc700 1 --2- 172.21.15.5:0/1758911213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f493c110fc0 0x7f493c119eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:35.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.399+0000 7f493affd700 1 -- 172.21.15.5:0/1758911213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f493c110fc0 msgr2=0x7f493c119eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:55:35.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.399+0000 7f493affd700 1 --2- 172.21.15.5:0/1758911213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f493c110fc0 0x7f493c119eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:35.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.399+0000 7f493affd700 1 -- 172.21.15.5:0/1758911213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f493c112740 msgr2=0x7f493c11a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:55:35.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.399+0000 7f493affd700 1 --2- 172.21.15.5:0/1758911213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f493c112740 0x7f493c11a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:35.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.399+0000 7f493affd700 1 -- 172.21.15.5:0/1758911213 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f493c11ace0 con 0x7f493c110ad0 2024-05-09T00:55:35.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.401+0000 7f493affd700 1 --2- 172.21.15.5:0/1758911213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f493c110ad0 0x7f493c120d80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f492c00e9a0 tx=0x7f492c00ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:55:35.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.401+0000 7f4923fff700 1 -- 172.21.15.5:0/1758911213 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f492c00cbd0 con 0x7f493c110ad0 2024-05-09T00:55:35.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.402+0000 7f4923fff700 1 -- 172.21.15.5:0/1758911213 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f492c00cd70 con 0x7f493c110ad0 2024-05-09T00:55:35.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.402+0000 7f4923fff700 1 -- 172.21.15.5:0/1758911213 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f492c018680 con 0x7f493c110ad0 2024-05-09T00:55:35.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.410+0000 7f493bfff700 1 -- 172.21.15.5:0/1758911213 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f493c11e9d0 con 0x7f493c110ad0 2024-05-09T00:55:35.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.410+0000 7f493bfff700 1 -- 172.21.15.5:0/1758911213 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f493c11ee90 con 0x7f493c110ad0 2024-05-09T00:55:35.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.419+0000 7f494ff2bb80 1 Processor -- start 2024-05-09T00:55:35.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.421+0000 7f494ff2bb80 1 -- start start 2024-05-09T00:55:35.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.422+0000 7f494ff2bb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560fb6a81d00 0x560fb6b83460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:35.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.422+0000 7f494ff2bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560fb6a074a0 0x560fb6b839a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:35.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.422+0000 7f494ff2bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560fb6b83ee0 0x560fb6b90720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:35.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.423+0000 7f494ff2bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560fb6aa70a0 con 0x560fb6a074a0 2024-05-09T00:55:35.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.423+0000 7f494ff2bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x560fb6a4c9a0 con 0x560fb6b83ee0 2024-05-09T00:55:35.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.423+0000 7f494ff2bb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x560fb6aa3060 con 0x560fb6a81d00 2024-05-09T00:55:35.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.423+0000 7f4923fff700 1 -- 172.21.15.5:0/1758911213 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f492c01f020 con 0x7f493c110ad0 2024-05-09T00:55:35.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.423+0000 7f493b7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560fb6a81d00 0x560fb6b83460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:35.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.424+0000 7f493b7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560fb6a81d00 0x560fb6b83460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:32978/0 (socket says 172.21.15.5:32978) 2024-05-09T00:55:35.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.424+0000 7f493b7fe700 1 -- 172.21.15.5:0/338716550 learned_addr learned my addr 172.21.15.5:0/338716550 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:55:35.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.424+0000 7f493a7fc700 1 --2- 172.21.15.5:0/338716550 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560fb6b83ee0 0x560fb6b90720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:35.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.425+0000 7f4923fff700 1 --2- 172.21.15.5:0/1758911213 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f49240b2a90 0x7f49240b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:35.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.426+0000 7f4923fff700 1 -- 172.21.15.5:0/1758911213 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f492c0e21f0 con 0x7f493c110ad0 2024-05-09T00:55:35.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.426+0000 7f493b7fe700 1 --2- 172.21.15.5:0/1758911213 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f49240b2a90 0x7f49240b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:35.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.427+0000 7f493b7fe700 1 -- 172.21.15.5:0/338716550 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560fb6b83ee0 msgr2=0x560fb6b90720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:55:35.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.427+0000 7f493b7fe700 1 --2- 172.21.15.5:0/338716550 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560fb6b83ee0 0x560fb6b90720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:35.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.427+0000 7f493b7fe700 1 -- 172.21.15.5:0/338716550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560fb6a074a0 msgr2=0x560fb6b839a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:55:35.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.427+0000 7f493b7fe700 1 --2- 172.21.15.5:0/338716550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560fb6a074a0 0x560fb6b839a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:35.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.427+0000 7f493b7fe700 1 -- 172.21.15.5:0/338716550 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fb6a3d500 con 0x560fb6a81d00 2024-05-09T00:55:35.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.428+0000 7f493b7fe700 1 --2- 172.21.15.5:0/338716550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560fb6a81d00 0x560fb6b83460 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f493400b780 tx=0x7f493400bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16326b33b76ef174 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:55:35.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.428+0000 7f493b7fe700 1 --2- 172.21.15.5:0/1758911213 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f49240b2a90 0x7f49240b4fa0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4934009fd0 tx=0x7f4934027000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:55:35.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.428+0000 7f4920ff9700 1 -- 172.21.15.5:0/338716550 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4934003c00 con 0x560fb6a81d00 2024-05-09T00:55:35.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.429+0000 7f4920ff9700 1 -- 172.21.15.5:0/338716550 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4934007e00 con 0x560fb6a81d00 2024-05-09T00:55:35.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.430+0000 7f494ff2bb80 1 -- 172.21.15.5:0/338716550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560fb6a81d00 msgr2=0x560fb6b83460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:55:35.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.430+0000 7f494ff2bb80 1 --2- 172.21.15.5:0/338716550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560fb6a81d00 0x560fb6b83460 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f493400b780 tx=0x7f493400bf60 comp rx=0 tx=0).stop 2024-05-09T00:55:35.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.432+0000 7f494ff2bb80 1 -- 172.21.15.5:0/338716550 shutdown_connections 2024-05-09T00:55:35.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.432+0000 7f494ff2bb80 1 --2- 172.21.15.5:0/338716550 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560fb6b83ee0 0x560fb6b90720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:35.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.432+0000 7f494ff2bb80 1 --2- 172.21.15.5:0/338716550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560fb6a81d00 0x560fb6b83460 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:35.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.432+0000 7f494ff2bb80 1 --2- 172.21.15.5:0/338716550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560fb6a074a0 0x560fb6b839a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:35.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.432+0000 7f494ff2bb80 1 -- 172.21.15.5:0/338716550 >> 172.21.15.5:0/338716550 conn(0x560fb6a0af70 msgr2=0x560fb6b7f980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:55:35.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.435+0000 7f494ff2bb80 1 -- 172.21.15.5:0/338716550 shutdown_connections 2024-05-09T00:55:35.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.435+0000 7f494ff2bb80 1 -- 172.21.15.5:0/338716550 wait complete. 2024-05-09T00:55:35.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.436+0000 7f494ff2bb80 1 -- start start 2024-05-09T00:55:35.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.436+0000 7f494ff2bb80 1 Processor -- start 2024-05-09T00:55:35.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.437+0000 7f494ff2bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560fb6a074a0 0x560fb6b8dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:35.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.437+0000 7f494ff2bb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560fb6a81d00 0x560fb6b8e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:35.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.437+0000 7f494ff2bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560fb6b83ee0 0x560fb6b8e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:35.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.437+0000 7f494ff2bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560fb6b90f80 con 0x560fb6a074a0 2024-05-09T00:55:35.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.437+0000 7f494ff2bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x560fb6a41b20 con 0x560fb6b83ee0 2024-05-09T00:55:35.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.437+0000 7f494ff2bb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x560fb6b91100 con 0x560fb6a81d00 2024-05-09T00:55:35.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.437+0000 7f494ff2bb80 10 client.0 _ll_register_callbacks cb 0x560fb69186c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:55:35.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.438+0000 7f493a7fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560fb6a81d00 0x560fb6b8e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:35.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.438+0000 7f493a7fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560fb6a81d00 0x560fb6b8e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:32992/0 (socket says 172.21.15.5:32992) 2024-05-09T00:55:35.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.438+0000 7f493a7fc700 1 -- 172.21.15.5:0/2303543212 learned_addr learned my addr 172.21.15.5:0/2303543212 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:55:35.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.439+0000 7f493affd700 1 --2- 172.21.15.5:0/2303543212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560fb6a074a0 0x560fb6b8dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:35.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.439+0000 7f493b7fe700 1 --2- 172.21.15.5:0/2303543212 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560fb6b83ee0 0x560fb6b8e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:35.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.440+0000 7f493affd700 1 -- 172.21.15.5:0/2303543212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560fb6a81d00 msgr2=0x560fb6b8e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:55:35.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.440+0000 7f493affd700 1 --2- 172.21.15.5:0/2303543212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560fb6a81d00 0x560fb6b8e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:35.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.440+0000 7f493affd700 1 -- 172.21.15.5:0/2303543212 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560fb6b83ee0 msgr2=0x560fb6b8e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:55:35.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.440+0000 7f493affd700 1 --2- 172.21.15.5:0/2303543212 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560fb6b83ee0 0x560fb6b8e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:35.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.440+0000 7f493affd700 1 -- 172.21.15.5:0/2303543212 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fb6922490 con 0x560fb6a074a0 2024-05-09T00:55:35.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.441+0000 7f493affd700 1 --2- 172.21.15.5:0/2303543212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560fb6a074a0 0x560fb6b8dca0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x560fb6b80dc0 tx=0x7f492c0a9d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9443ea779cc9e25 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:55:35.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.441+0000 7f491eff5700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:55:35.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.444+0000 7f491eff5700 1 -- 172.21.15.5:0/2303543212 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f492c0a9340 con 0x560fb6a074a0 2024-05-09T00:55:35.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.444+0000 7f494ff2bb80 1 -- 172.21.15.5:0/2303543212 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4934007e00 con 0x560fb6a074a0 2024-05-09T00:55:35.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.444+0000 7f491eff5700 1 -- 172.21.15.5:0/2303543212 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f492c0a8ca0 con 0x560fb6a074a0 2024-05-09T00:55:35.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.446+0000 7f491eff5700 1 -- 172.21.15.5:0/2303543212 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f492c0cc0b0 con 0x560fb6a074a0 2024-05-09T00:55:35.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.450+0000 7f494ff2bb80 10 client.16452 fetch_fsmap learned FSMap version 31 2024-05-09T00:55:35.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.450+0000 7f494ff2bb80 1 -- 172.21.15.5:0/2303543212 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560fb6b9bac0 con 0x560fb6a074a0 2024-05-09T00:55:35.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.452+0000 7f491eff5700 1 -- 172.21.15.5:0/2303543212 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4934007e00 con 0x560fb6a074a0 2024-05-09T00:55:35.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.452+0000 7f491eff5700 1 -- 172.21.15.5:0/2303543212 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f492c010430 con 0x560fb6a074a0 2024-05-09T00:55:35.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.452+0000 7f494ff2bb80 10 client.16452 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:55:35.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.452+0000 7f494ff2bb80 10 client.16452 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:55:35.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.452+0000 7f494ff2bb80 20 client.16452 populate_metadata read hostname 'smithi005' 2024-05-09T00:55:35.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.452+0000 7f494ff2bb80 1 --2- 172.21.15.5:0/2303543212 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x560fb6b9e4c0 0x560fb6ba0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:35.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.452+0000 7f494ff2bb80 4 client.16452 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:55:35.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.452+0000 7f494ff2bb80 1 -- 172.21.15.5:0/2303543212 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560fb6ab4a80 con 0x560fb6b9e4c0 2024-05-09T00:55:35.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.475+0000 7f493a7fc700 1 --2- 172.21.15.5:0/2303543212 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x560fb6b9e4c0 0x560fb6ba0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:35.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.482+0000 7f491eff5700 10 client.16452 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:55:35.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.482+0000 7f493a7fc700 1 --2- 172.21.15.5:0/2303543212 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x560fb6b9e4c0 0x560fb6ba0930 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7d204d31b312e54 server_cookie=39fcf13d3290f30f in_seq=0 out_seq=0 2024-05-09T00:55:35.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.485+0000 7f491eff5700 1 -- 172.21.15.5:0/2303543212 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560fb6ab4a80 con 0x560fb6b9e4c0 2024-05-09T00:55:35.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.485+0000 7f491eff5700 10 client.16452 handle_command_reply: tid=0 2024-05-09T00:55:35.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.485+0000 7f494ff2bb80 1 client.16452 shutdown 2024-05-09T00:55:35.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.486+0000 7f494ff2bb80 1 -- 172.21.15.5:0/2303543212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560fb6a074a0 msgr2=0x560fb6b8dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:55:35.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.486+0000 7f494ff2bb80 1 --2- 172.21.15.5:0/2303543212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560fb6a074a0 0x560fb6b8dca0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x560fb6b80dc0 tx=0x7f492c0a9d70 comp rx=0 tx=0).stop 2024-05-09T00:55:35.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.487+0000 7f494ff2bb80 1 -- 172.21.15.5:0/2303543212 shutdown_connections 2024-05-09T00:55:35.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.487+0000 7f494ff2bb80 1 --2- 172.21.15.5:0/2303543212 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x560fb6b9e4c0 0x560fb6ba0930 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:35.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.487+0000 7f494ff2bb80 1 --2- 172.21.15.5:0/2303543212 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560fb6b83ee0 0x560fb6b8e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:35.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.487+0000 7f494ff2bb80 1 --2- 172.21.15.5:0/2303543212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560fb6a074a0 0x560fb6b8dca0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:35.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.487+0000 7f494ff2bb80 1 --2- 172.21.15.5:0/2303543212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560fb6a81d00 0x560fb6b8e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:35.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.487+0000 7f491eff5700 0 client.16452 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:55:35.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.487+0000 7f494ff2bb80 1 -- 172.21.15.5:0/2303543212 >> 172.21.15.5:0/2303543212 conn(0x560fb6a0af70 msgr2=0x560fb6b7f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:55:35.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.487+0000 7f494ff2bb80 1 -- 172.21.15.5:0/2303543212 shutdown_connections 2024-05-09T00:55:35.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.487+0000 7f494ff2bb80 1 -- 172.21.15.5:0/2303543212 wait complete. 2024-05-09T00:55:35.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.487+0000 7f494ff2bb80 20 client.16452 trim_cache size 0 max 16384 2024-05-09T00:55:35.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.546+0000 7f494ff2bb80 1 Processor -- start 2024-05-09T00:55:35.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.547+0000 7f494ff2bb80 1 -- start start 2024-05-09T00:55:35.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.547+0000 7f494ff2bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560fb6a074a0 0x560fb6b90140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:35.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.547+0000 7f494ff2bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560fb6a81d00 0x560fb6b8e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:35.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.548+0000 7f494ff2bb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560fb6b83ee0 0x560fb6ba60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:35.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.548+0000 7f494ff2bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f493c1c60c0 con 0x560fb6a81d00 2024-05-09T00:55:35.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.548+0000 7f494ff2bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f493c1c7c00 con 0x560fb6a074a0 2024-05-09T00:55:35.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.548+0000 7f494ff2bb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f493c1c6790 con 0x560fb6b83ee0 2024-05-09T00:55:35.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.549+0000 7f493a7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560fb6a81d00 0x560fb6b8e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:35.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.549+0000 7f493a7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560fb6a81d00 0x560fb6b8e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58862/0 (socket says 172.21.15.5:58862) 2024-05-09T00:55:35.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.549+0000 7f493a7fc700 1 -- 172.21.15.5:0/2872019968 learned_addr learned my addr 172.21.15.5:0/2872019968 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:55:35.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.550+0000 7f493b7fe700 1 --2- 172.21.15.5:0/2872019968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560fb6b83ee0 0x560fb6ba60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:35.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.551+0000 7f493affd700 1 --2- 172.21.15.5:0/2872019968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560fb6a074a0 0x560fb6b90140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:35.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.551+0000 7f493a7fc700 1 -- 172.21.15.5:0/2872019968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560fb6b83ee0 msgr2=0x560fb6ba60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:55:35.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.551+0000 7f493a7fc700 1 --2- 172.21.15.5:0/2872019968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560fb6b83ee0 0x560fb6ba60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:35.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.551+0000 7f493a7fc700 1 -- 172.21.15.5:0/2872019968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560fb6a074a0 msgr2=0x560fb6b90140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:55:35.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.551+0000 7f493a7fc700 1 --2- 172.21.15.5:0/2872019968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560fb6a074a0 0x560fb6b90140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:35.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.551+0000 7f493a7fc700 1 -- 172.21.15.5:0/2872019968 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fb6b8da70 con 0x560fb6a81d00 2024-05-09T00:55:35.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.555+0000 7f493a7fc700 1 --2- 172.21.15.5:0/2872019968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560fb6a81d00 0x560fb6b8e370 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f4930000c00 tx=0x7f4930009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f58a5c9947651edf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:55:35.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.558+0000 7f493bfff700 1 -- 172.21.15.5:0/2872019968 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4930003be0 con 0x560fb6a81d00 2024-05-09T00:55:35.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.558+0000 7f493bfff700 1 -- 172.21.15.5:0/2872019968 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4930003d80 con 0x560fb6a81d00 2024-05-09T00:55:35.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.558+0000 7f493bfff700 1 -- 172.21.15.5:0/2872019968 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49300127d0 con 0x560fb6a81d00 2024-05-09T00:55:35.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.560+0000 7f494ff2bb80 1 -- 172.21.15.5:0/2872019968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560fb6a81d00 msgr2=0x560fb6b8e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:55:35.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.560+0000 7f494ff2bb80 1 --2- 172.21.15.5:0/2872019968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560fb6a81d00 0x560fb6b8e370 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f4930000c00 tx=0x7f4930009260 comp rx=0 tx=0).stop 2024-05-09T00:55:35.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.560+0000 7f494ff2bb80 1 -- 172.21.15.5:0/2872019968 shutdown_connections 2024-05-09T00:55:35.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.560+0000 7f494ff2bb80 1 --2- 172.21.15.5:0/2872019968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560fb6b83ee0 0x560fb6ba60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:35.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.560+0000 7f494ff2bb80 1 --2- 172.21.15.5:0/2872019968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560fb6a074a0 0x560fb6b90140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:35.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.560+0000 7f494ff2bb80 1 --2- 172.21.15.5:0/2872019968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560fb6a81d00 0x560fb6b8e370 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:35.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.560+0000 7f494ff2bb80 1 -- 172.21.15.5:0/2872019968 >> 172.21.15.5:0/2872019968 conn(0x560fb6a0af70 msgr2=0x560fb6b8dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:55:35.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.563+0000 7f494ff2bb80 1 -- 172.21.15.5:0/2872019968 shutdown_connections 2024-05-09T00:55:35.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.564+0000 7f494ff2bb80 1 -- 172.21.15.5:0/2872019968 wait complete. 2024-05-09T00:55:35.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.564+0000 7f494ff2bb80 1 -- start start 2024-05-09T00:55:35.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.564+0000 7f494ff2bb80 1 Processor -- start 2024-05-09T00:55:35.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.565+0000 7f494ff2bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560fb6a074a0 0x560fb6b96990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:35.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.566+0000 7f494ff2bb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560fb6a81d00 0x560fb6b93d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:35.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.566+0000 7f494ff2bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560fb6b83ee0 0x560fb6b94250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:35.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.566+0000 7f494ff2bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f493c1c7530 con 0x560fb6b83ee0 2024-05-09T00:55:35.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.566+0000 7f494ff2bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f493c1c6e60 con 0x560fb6a074a0 2024-05-09T00:55:35.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.566+0000 7f494ff2bb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f493c1c4060 con 0x560fb6a81d00 2024-05-09T00:55:35.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.566+0000 7f494ff2bb80 10 client.0 _ll_register_callbacks cb 0x560fb69186c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:55:35.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.567+0000 7f493b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560fb6b83ee0 0x560fb6b94250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:35.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.567+0000 7f493b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560fb6b83ee0 0x560fb6b94250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58874/0 (socket says 172.21.15.5:58874) 2024-05-09T00:55:35.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.567+0000 7f493b7fe700 1 -- 172.21.15.5:0/3478216677 learned_addr learned my addr 172.21.15.5:0/3478216677 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:55:35.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.568+0000 7f493b7fe700 1 -- 172.21.15.5:0/3478216677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560fb6a81d00 msgr2=0x560fb6b93d10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:55:35.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.568+0000 7f493b7fe700 1 --2- 172.21.15.5:0/3478216677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560fb6a81d00 0x560fb6b93d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:35.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.568+0000 7f493b7fe700 1 -- 172.21.15.5:0/3478216677 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560fb6a074a0 msgr2=0x560fb6b96990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:55:35.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.568+0000 7f493b7fe700 1 --2- 172.21.15.5:0/3478216677 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560fb6a074a0 0x560fb6b96990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:35.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.568+0000 7f493b7fe700 1 -- 172.21.15.5:0/3478216677 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fb6b949a0 con 0x560fb6b83ee0 2024-05-09T00:55:35.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.569+0000 7f493b7fe700 1 --2- 172.21.15.5:0/3478216677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560fb6b83ee0 0x560fb6b94250 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f4934004130 tx=0x7f4934004020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20b861cccf588f79 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:55:35.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.569+0000 7f491eff5700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:55:35.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.570+0000 7f491eff5700 1 -- 172.21.15.5:0/3478216677 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4934003a00 con 0x560fb6b83ee0 2024-05-09T00:55:35.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.570+0000 7f494ff2bb80 1 -- 172.21.15.5:0/3478216677 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4930003d80 con 0x560fb6b83ee0 2024-05-09T00:55:35.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.570+0000 7f491eff5700 1 -- 172.21.15.5:0/3478216677 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f4934003ba0 con 0x560fb6b83ee0 2024-05-09T00:55:35.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.571+0000 7f491eff5700 1 -- 172.21.15.5:0/3478216677 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f4934041030 con 0x560fb6b83ee0 2024-05-09T00:55:35.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.572+0000 7f494ff2bb80 10 client.16458 fetch_fsmap learned FSMap version 31 2024-05-09T00:55:35.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.572+0000 7f494ff2bb80 1 -- 172.21.15.5:0/3478216677 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560fb6c08be0 con 0x560fb6b83ee0 2024-05-09T00:55:35.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.574+0000 7f491eff5700 1 -- 172.21.15.5:0/3478216677 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4930003d80 con 0x560fb6b83ee0 2024-05-09T00:55:35.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.574+0000 7f491eff5700 1 -- 172.21.15.5:0/3478216677 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f4934011070 con 0x560fb6b83ee0 2024-05-09T00:55:35.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.575+0000 7f494ff2bb80 10 client.16458 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:55:35.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.575+0000 7f494ff2bb80 10 client.16458 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:55:35.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.575+0000 7f494ff2bb80 20 client.16458 populate_metadata read hostname 'smithi005' 2024-05-09T00:55:35.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.575+0000 7f494ff2bb80 1 --2- 172.21.15.5:0/3478216677 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x560fb6c09060 0x560fb6c0b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:35.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.575+0000 7f494ff2bb80 4 client.16458 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:55:35.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.575+0000 7f494ff2bb80 1 -- 172.21.15.5:0/3478216677 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560fb6aaeb20 con 0x560fb6c09060 2024-05-09T00:55:35.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.580+0000 7f493affd700 1 --2- 172.21.15.5:0/3478216677 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x560fb6c09060 0x560fb6c0b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:35.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.589+0000 7f491eff5700 10 client.16458 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:55:35.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.589+0000 7f493affd700 1 --2- 172.21.15.5:0/3478216677 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x560fb6c09060 0x560fb6c0b4c0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d89e21df8920cce server_cookie=82353a88e059160f in_seq=0 out_seq=0 2024-05-09T00:55:35.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.595+0000 7f491eff5700 1 -- 172.21.15.5:0/3478216677 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560fb6aaeb20 con 0x560fb6c09060 2024-05-09T00:55:35.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.595+0000 7f491eff5700 10 client.16458 handle_command_reply: tid=0 2024-05-09T00:55:35.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.595+0000 7f494ff2bb80 1 client.16458 shutdown 2024-05-09T00:55:35.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.595+0000 7f494ff2bb80 1 -- 172.21.15.5:0/3478216677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560fb6b83ee0 msgr2=0x560fb6b94250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:55:35.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.595+0000 7f494ff2bb80 1 --2- 172.21.15.5:0/3478216677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560fb6b83ee0 0x560fb6b94250 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f4934004130 tx=0x7f4934004020 comp rx=0 tx=0).stop 2024-05-09T00:55:35.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.595+0000 7f494ff2bb80 1 -- 172.21.15.5:0/3478216677 shutdown_connections 2024-05-09T00:55:35.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.595+0000 7f494ff2bb80 1 --2- 172.21.15.5:0/3478216677 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x560fb6c09060 0x560fb6c0b4c0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:35.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.595+0000 7f494ff2bb80 1 --2- 172.21.15.5:0/3478216677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560fb6b83ee0 0x560fb6b94250 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:35.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.595+0000 7f494ff2bb80 1 --2- 172.21.15.5:0/3478216677 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560fb6a074a0 0x560fb6b96990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:35.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.595+0000 7f494ff2bb80 1 --2- 172.21.15.5:0/3478216677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560fb6a81d00 0x560fb6b93d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:35.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.595+0000 7f494ff2bb80 1 -- 172.21.15.5:0/3478216677 >> 172.21.15.5:0/3478216677 conn(0x560fb6a0af70 msgr2=0x560fb6b8eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:55:35.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.595+0000 7f491eff5700 0 client.16458 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:55:35.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.597+0000 7f494ff2bb80 1 -- 172.21.15.5:0/3478216677 shutdown_connections 2024-05-09T00:55:35.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.597+0000 7f494ff2bb80 1 -- 172.21.15.5:0/3478216677 wait complete. 2024-05-09T00:55:35.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.597+0000 7f494ff2bb80 20 client.16458 trim_cache size 0 max 16384 2024-05-09T00:55:35.601 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-09T00:55:35.601 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T00:55:35.602 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-05-09T00:55:35.602 INFO:teuthology.orchestra.run.smithi005.stdout: "176d2d68-dd2a-4069-a614-d6fef3a5f1a9": { 2024-05-09T00:55:35.602 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-05-09T00:55:35.602 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "176d2d68-dd2a-4069-a614-d6fef3a5f1a9", 2024-05-09T00:55:35.602 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-05-09T00:55:35.602 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T00:55:35.602 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T00:55:35.602 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-09T00:55:35.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.604+0000 7f4921ffb700 1 -- 172.21.15.5:0/1758911213 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f49240b2a90 msgr2=0x7f49240b4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:55:35.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.604+0000 7f4921ffb700 1 --2- 172.21.15.5:0/1758911213 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f49240b2a90 0x7f49240b4fa0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4934009fd0 tx=0x7f4934027000 comp rx=0 tx=0).stop 2024-05-09T00:55:35.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.604+0000 7f4921ffb700 1 -- 172.21.15.5:0/1758911213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f493c110ad0 msgr2=0x7f493c120d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:55:35.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.604+0000 7f4921ffb700 1 --2- 172.21.15.5:0/1758911213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f493c110ad0 0x7f493c120d80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f492c00e9a0 tx=0x7f492c00ed60 comp rx=0 tx=0).stop 2024-05-09T00:55:35.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.604+0000 7f4921ffb700 1 -- 172.21.15.5:0/1758911213 shutdown_connections 2024-05-09T00:55:35.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.604+0000 7f4921ffb700 1 --2- 172.21.15.5:0/1758911213 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f49240b2a90 0x7f49240b4fa0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:35.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.604+0000 7f4921ffb700 1 --2- 172.21.15.5:0/1758911213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f493c112740 0x7f493c11a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:35.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.604+0000 7f4921ffb700 1 --2- 172.21.15.5:0/1758911213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f493c110ad0 0x7f493c120d80 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:35.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.604+0000 7f4921ffb700 1 --2- 172.21.15.5:0/1758911213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f493c110fc0 0x7f493c119eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:35.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.605+0000 7f4921ffb700 1 -- 172.21.15.5:0/1758911213 >> 172.21.15.5:0/1758911213 conn(0x7f493c06f260 msgr2=0x7f493c0744f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:55:35.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.608+0000 7f4921ffb700 1 -- 172.21.15.5:0/1758911213 shutdown_connections 2024-05-09T00:55:35.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:55:35.609+0000 7f4921ffb700 1 -- 172.21.15.5:0/1758911213 wait complete. 2024-05-09T00:55:35.653 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:55:35 smithi005 ceph-mon[30365]: pgmap v648: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 856 MiB/s wr, 1.78k op/s 2024-05-09T00:55:35.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:55:35 smithi157 ceph-mon[34294]: pgmap v648: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 856 MiB/s wr, 1.78k op/s 2024-05-09T00:55:35.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:55:35 smithi131 ceph-mon[34547]: pgmap v648: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 856 MiB/s wr, 1.78k op/s 2024-05-09T00:55:36.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:55:36 smithi157 ceph-mon[34294]: scrub summary: idle 2024-05-09T00:55:36.794 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:55:36 smithi005 ceph-mon[30365]: scrub summary: idle 2024-05-09T00:55:36.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:55:36 smithi131 ceph-mon[34547]: scrub summary: idle 2024-05-09T00:55:37.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:55:37 smithi157 ceph-mon[34294]: scrub summary: idle 2024-05-09T00:55:37.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:55:37 smithi157 ceph-mon[34294]: pgmap v649: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 902 MiB/s wr, 1.89k op/s 2024-05-09T00:55:37.790 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:55:37 smithi005 ceph-mon[30365]: scrub summary: idle 2024-05-09T00:55:37.790 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:55:37 smithi005 ceph-mon[30365]: pgmap v649: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 902 MiB/s wr, 1.89k op/s 2024-05-09T00:55:37.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:55:37 smithi131 ceph-mon[34547]: scrub summary: idle 2024-05-09T00:55:37.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:55:37 smithi131 ceph-mon[34547]: pgmap v649: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 902 MiB/s wr, 1.89k op/s 2024-05-09T00:55:38.391 INFO:tasks.cephfs.filesystem:scrub status for tag:176d2d68-dd2a-4069-a614-d6fef3a5f1a9 - {'path': '/', 'tag': '176d2d68-dd2a-4069-a614-d6fef3a5f1a9', 'options': 'recursive,force'} 2024-05-09T00:55:39.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:55:39 smithi005 ceph-mon[30365]: pgmap v650: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 849 MiB/s wr, 1.75k op/s 2024-05-09T00:55:39.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:55:39 smithi157 ceph-mon[34294]: pgmap v650: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 849 MiB/s wr, 1.75k op/s 2024-05-09T00:55:39.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:55:39 smithi131 ceph-mon[34547]: pgmap v650: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 849 MiB/s wr, 1.75k op/s 2024-05-09T00:55:41.790 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:55:41 smithi005 ceph-mon[30365]: pgmap v651: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 995 MiB/s wr, 2.01k op/s 2024-05-09T00:55:41.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:55:41 smithi131 ceph-mon[34547]: pgmap v651: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 995 MiB/s wr, 2.01k op/s 2024-05-09T00:55:41.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:55:41 smithi157 ceph-mon[34294]: pgmap v651: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 995 MiB/s wr, 2.01k op/s 2024-05-09T00:55:43.790 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:55:43 smithi005 ceph-mon[30365]: pgmap v652: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 757 MiB/s wr, 1.58k op/s 2024-05-09T00:55:43.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:55:43 smithi131 ceph-mon[34547]: pgmap v652: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 757 MiB/s wr, 1.58k op/s 2024-05-09T00:55:43.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:55:43 smithi157 ceph-mon[34294]: pgmap v652: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 757 MiB/s wr, 1.58k op/s 2024-05-09T00:55:45.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:55:45 smithi005 ceph-mon[30365]: pgmap v653: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 883 MiB/s wr, 1.96k op/s 2024-05-09T00:55:45.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:55:45 smithi131 ceph-mon[34547]: pgmap v653: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 883 MiB/s wr, 1.96k op/s 2024-05-09T00:55:45.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:55:45 smithi157 ceph-mon[34294]: pgmap v653: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 883 MiB/s wr, 1.96k op/s 2024-05-09T00:55:47.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:55:47 smithi005 ceph-mon[30365]: pgmap v654: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 918 MiB/s wr, 2.18k op/s 2024-05-09T00:55:47.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:55:47 smithi131 ceph-mon[34547]: pgmap v654: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 918 MiB/s wr, 2.18k op/s 2024-05-09T00:55:47.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:55:47 smithi157 ceph-mon[34294]: pgmap v654: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 918 MiB/s wr, 2.18k op/s 2024-05-09T00:55:48.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.588+0000 7f34d10eb640 1 -- 172.21.15.5:0/1915872013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34cc10ffe0 msgr2=0x7f34cc1103f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:55:48.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.588+0000 7f34d10eb640 1 --2- 172.21.15.5:0/1915872013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34cc10ffe0 0x7f34cc1103f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f34c4009be0 tx=0x7f34c402a6d0 comp rx=0 tx=0).stop 2024-05-09T00:55:48.597 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.594+0000 7f34d10eb640 1 -- 172.21.15.5:0/1915872013 shutdown_connections 2024-05-09T00:55:48.597 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.594+0000 7f34d10eb640 1 --2- 172.21.15.5:0/1915872013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34cc10ffe0 0x7f34cc1103f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.594+0000 7f34d10eb640 1 --2- 172.21.15.5:0/1915872013 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34cc10b2c0 0x7f34cc10b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.594+0000 7f34d10eb640 1 --2- 172.21.15.5:0/1915872013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34cc10a470 0x7f34cc10ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.594+0000 7f34d10eb640 1 -- 172.21.15.5:0/1915872013 >> 172.21.15.5:0/1915872013 conn(0x7f34cc070460 msgr2=0x7f34cc072910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:55:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.600+0000 7f34d10eb640 1 -- 172.21.15.5:0/1915872013 shutdown_connections 2024-05-09T00:55:48.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.600+0000 7f34d10eb640 1 -- 172.21.15.5:0/1915872013 wait complete. 2024-05-09T00:55:48.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.601+0000 7f34d10eb640 1 Processor -- start 2024-05-09T00:55:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.602+0000 7f34d10eb640 1 -- start start 2024-05-09T00:55:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.602+0000 7f34d10eb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34cc10a470 0x7f34cc1a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.602+0000 7f34d10eb640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34cc10b2c0 0x7f34cc1a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.602+0000 7f34d10eb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34cc10ffe0 0x7f34cc1a7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:48.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.602+0000 7f34d10eb640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f34cc113b40 con 0x7f34cc10a470 2024-05-09T00:55:48.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.602+0000 7f34d10eb640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f34cc1139c0 con 0x7f34cc10b2c0 2024-05-09T00:55:48.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.602+0000 7f34d10eb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f34cc113cc0 con 0x7f34cc10ffe0 2024-05-09T00:55:48.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.603+0000 7f34d08ea640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34cc10ffe0 0x7f34cc1a7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:48.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.603+0000 7f34cb7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34cc10b2c0 0x7f34cc1a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:48.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.603+0000 7f34cbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34cc10a470 0x7f34cc1a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:48.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.603+0000 7f34cbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34cc10a470 0x7f34cc1a0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43514/0 (socket says 172.21.15.5:43514) 2024-05-09T00:55:48.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.603+0000 7f34cbfff640 1 -- 172.21.15.5:0/2482708517 learned_addr learned my addr 172.21.15.5:0/2482708517 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:55:48.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.604+0000 7f34d08ea640 1 -- 172.21.15.5:0/2482708517 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34cc10b2c0 msgr2=0x7f34cc1a0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:55:48.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.604+0000 7f34d08ea640 1 --2- 172.21.15.5:0/2482708517 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34cc10b2c0 0x7f34cc1a0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:48.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.604+0000 7f34d08ea640 1 -- 172.21.15.5:0/2482708517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34cc10a470 msgr2=0x7f34cc1a0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:55:48.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.604+0000 7f34d08ea640 1 --2- 172.21.15.5:0/2482708517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34cc10a470 0x7f34cc1a0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:48.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.604+0000 7f34d08ea640 1 -- 172.21.15.5:0/2482708517 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34cc1a8250 con 0x7f34cc10ffe0 2024-05-09T00:55:48.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.609+0000 7f34d08ea640 1 --2- 172.21.15.5:0/2482708517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34cc10ffe0 0x7f34cc1a7be0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f34c4034040 tx=0x7f34c4030c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:55:48.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.614+0000 7f34c97fa640 1 -- 172.21.15.5:0/2482708517 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c4041020 con 0x7f34cc10ffe0 2024-05-09T00:55:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.614+0000 7f34d10eb640 1 -- 172.21.15.5:0/2482708517 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34cc1a84e0 con 0x7f34cc10ffe0 2024-05-09T00:55:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.614+0000 7f34d10eb640 1 -- 172.21.15.5:0/2482708517 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34cc1a89d0 con 0x7f34cc10ffe0 2024-05-09T00:55:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.615+0000 7f34c97fa640 1 -- 172.21.15.5:0/2482708517 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34c4003e20 con 0x7f34cc10ffe0 2024-05-09T00:55:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.615+0000 7f34c97fa640 1 -- 172.21.15.5:0/2482708517 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c4036040 con 0x7f34cc10ffe0 2024-05-09T00:55:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.623+0000 7f34c97fa640 1 -- 172.21.15.5:0/2482708517 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f34c4044020 con 0x7f34cc10ffe0 2024-05-09T00:55:48.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.627+0000 7f34d9204740 1 Processor -- start 2024-05-09T00:55:48.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.627+0000 7f34c97fa640 1 --2- 172.21.15.5:0/2482708517 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f34b40b2970 0x7f34b40b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.627+0000 7f34d9204740 1 -- start start 2024-05-09T00:55:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.628+0000 7f34d9204740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564eb33501b0 0x564eb36f46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.628+0000 7f34d9204740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564eb33272f0 0x564eb36f4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.628+0000 7f34d9204740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564eb332f4d0 0x564eb36fbb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:48.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.628+0000 7f34d9204740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564eb36587c0 con 0x564eb33501b0 2024-05-09T00:55:48.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.628+0000 7f34d9204740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x564eb35f40e0 con 0x564eb33272f0 2024-05-09T00:55:48.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.628+0000 7f34d9204740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x564eb3654780 con 0x564eb332f4d0 2024-05-09T00:55:48.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.628+0000 7f34cb7fe640 1 --2- 172.21.15.5:0/2482708517 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f34b40b2970 0x7f34b40b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:48.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.629+0000 7f34d08ea640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564eb332f4d0 0x564eb36fbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:48.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.629+0000 7f34d08ea640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564eb332f4d0 0x564eb36fbb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:50286/0 (socket says 172.21.15.5:50286) 2024-05-09T00:55:48.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.629+0000 7f34d08ea640 1 -- 172.21.15.5:0/3646662292 learned_addr learned my addr 172.21.15.5:0/3646662292 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:55:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.629+0000 7f34cb7fe640 1 --2- 172.21.15.5:0/3646662292 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564eb33272f0 0x564eb36f4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.629+0000 7f34c97fa640 1 -- 172.21.15.5:0/2482708517 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f34c41037c0 con 0x7f34cc10ffe0 2024-05-09T00:55:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.630+0000 7f34d08ea640 1 -- 172.21.15.5:0/3646662292 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564eb33272f0 msgr2=0x564eb36f4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:55:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.630+0000 7f34d08ea640 1 --2- 172.21.15.5:0/3646662292 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564eb33272f0 0x564eb36f4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.630+0000 7f34d08ea640 1 -- 172.21.15.5:0/3646662292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564eb33501b0 msgr2=0x564eb36f46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:55:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.630+0000 7f34d08ea640 1 --2- 172.21.15.5:0/3646662292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564eb33501b0 0x564eb36f46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.630+0000 7f34d08ea640 1 -- 172.21.15.5:0/3646662292 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564eb361d7c0 con 0x564eb332f4d0 2024-05-09T00:55:48.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.631+0000 7f34d08ea640 1 --2- 172.21.15.5:0/3646662292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564eb332f4d0 0x564eb36fbb40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f34c40caab0 tx=0x7f34c402abd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ddbe94ca9f4378a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:55:48.634 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.634+0000 7f34b27fc640 1 -- 172.21.15.5:0/3646662292 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c40ca8c0 con 0x564eb332f4d0 2024-05-09T00:55:48.634 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.634+0000 7f34b27fc640 1 -- 172.21.15.5:0/3646662292 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34c40cb4b0 con 0x564eb332f4d0 2024-05-09T00:55:48.635 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.634+0000 7f34b27fc640 1 -- 172.21.15.5:0/3646662292 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c40d3780 con 0x564eb332f4d0 2024-05-09T00:55:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.635+0000 7f34d9204740 1 -- 172.21.15.5:0/3646662292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564eb332f4d0 msgr2=0x564eb36fbb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:55:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.635+0000 7f34d9204740 1 --2- 172.21.15.5:0/3646662292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564eb332f4d0 0x564eb36fbb40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f34c40caab0 tx=0x7f34c402abd0 comp rx=0 tx=0).stop 2024-05-09T00:55:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.635+0000 7f34cb7fe640 1 --2- 172.21.15.5:0/2482708517 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f34b40b2970 0x7f34b40b4e80 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x564eb36f5aa0 tx=0x7f34c0015040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:55:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.636+0000 7f34d9204740 1 -- 172.21.15.5:0/3646662292 shutdown_connections 2024-05-09T00:55:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.636+0000 7f34d9204740 1 --2- 172.21.15.5:0/3646662292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564eb332f4d0 0x564eb36fbb40 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.636+0000 7f34d9204740 1 --2- 172.21.15.5:0/3646662292 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564eb33272f0 0x564eb36f4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.636+0000 7f34d9204740 1 --2- 172.21.15.5:0/3646662292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564eb33501b0 0x564eb36f46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.636+0000 7f34d9204740 1 -- 172.21.15.5:0/3646662292 >> 172.21.15.5:0/3646662292 conn(0x564eb3344300 msgr2=0x564eb36f0050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:55:48.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.638+0000 7f34d9204740 1 -- 172.21.15.5:0/3646662292 shutdown_connections 2024-05-09T00:55:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.638+0000 7f34d9204740 1 -- 172.21.15.5:0/3646662292 wait complete. 2024-05-09T00:55:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.639+0000 7f34d9204740 1 -- start start 2024-05-09T00:55:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.639+0000 7f34d9204740 1 Processor -- start 2024-05-09T00:55:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.640+0000 7f34d9204740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564eb33272f0 0x564eb36fe060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.640+0000 7f34d9204740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564eb332f4d0 0x564eb36fe5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.640+0000 7f34d9204740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564eb33501b0 0x564eb3705490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.641+0000 7f34d9204740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564eb3654780 con 0x564eb33501b0 2024-05-09T00:55:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.641+0000 7f34cbfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564eb33272f0 0x564eb36fe060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.641+0000 7f34cbfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564eb33272f0 0x564eb36fe060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:50290/0 (socket says 172.21.15.5:50290) 2024-05-09T00:55:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.641+0000 7f34cbfff640 1 -- 172.21.15.5:0/1131906685 learned_addr learned my addr 172.21.15.5:0/1131906685 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:55:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.641+0000 7f34cb7fe640 1 --2- 172.21.15.5:0/1131906685 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564eb332f4d0 0x564eb36fe5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.641+0000 7f34d9204740 1 -- 172.21.15.5:0/1131906685 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x564eb35df9c0 con 0x564eb332f4d0 2024-05-09T00:55:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.641+0000 7f34d9204740 1 -- 172.21.15.5:0/1131906685 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x564eb36fc350 con 0x564eb33272f0 2024-05-09T00:55:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.641+0000 7f34d9204740 10 client.0 _ll_register_callbacks cb 0x564eb360c020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:55:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.642+0000 7f34d08ea640 1 --2- 172.21.15.5:0/1131906685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564eb33501b0 0x564eb3705490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.642+0000 7f34cbfff640 1 -- 172.21.15.5:0/1131906685 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564eb332f4d0 msgr2=0x564eb36fe5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:55:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.642+0000 7f34cbfff640 1 --2- 172.21.15.5:0/1131906685 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564eb332f4d0 0x564eb36fe5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.642+0000 7f34cbfff640 1 -- 172.21.15.5:0/1131906685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564eb33501b0 msgr2=0x564eb3705490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:55:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.642+0000 7f34cbfff640 1 --2- 172.21.15.5:0/1131906685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564eb33501b0 0x564eb3705490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.642+0000 7f34cbfff640 1 -- 172.21.15.5:0/1131906685 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564eb36061f0 con 0x564eb33272f0 2024-05-09T00:55:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.643+0000 7f34cbfff640 1 --2- 172.21.15.5:0/1131906685 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564eb33272f0 0x564eb36fe060 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f34bc022470 tx=0x7f34bc022930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5a47d694dbc5692 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:55:48.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.650+0000 7f34afff7640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-09T00:55:48.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.650+0000 7f34afff7640 1 -- 172.21.15.5:0/1131906685 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34bc02f060 con 0x564eb33272f0 2024-05-09T00:55:48.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.650+0000 7f34afff7640 1 -- 172.21.15.5:0/1131906685 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f34bc019040 con 0x564eb33272f0 2024-05-09T00:55:48.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.650+0000 7f34d9204740 1 -- 172.21.15.5:0/1131906685 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34c40cb4b0 con 0x564eb33272f0 2024-05-09T00:55:48.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.651+0000 7f34afff7640 1 -- 172.21.15.5:0/1131906685 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f34bc0206a0 con 0x564eb33272f0 2024-05-09T00:55:48.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.656+0000 7f34d9204740 10 client.25987 fetch_fsmap learned FSMap version 31 2024-05-09T00:55:48.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.656+0000 7f34d9204740 1 -- 172.21.15.5:0/1131906685 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564eb33bb790 con 0x564eb33272f0 2024-05-09T00:55:48.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.657+0000 7f34afff7640 1 -- 172.21.15.5:0/1131906685 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34c40cb4b0 con 0x564eb33272f0 2024-05-09T00:55:48.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.657+0000 7f34afff7640 1 -- 172.21.15.5:0/1131906685 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f34bc035a50 con 0x564eb33272f0 2024-05-09T00:55:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.661+0000 7f34d9204740 10 client.25987 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:55:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.661+0000 7f34d9204740 10 client.25987 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:55:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.661+0000 7f34d9204740 20 client.25987 populate_metadata read hostname 'smithi005' 2024-05-09T00:55:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.661+0000 7f34d9204740 1 --2- 172.21.15.5:0/1131906685 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x564eb330ed80 0x564eb370b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.661+0000 7f34d9204740 4 client.25987 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:55:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.661+0000 7f34d9204740 1 -- 172.21.15.5:0/1131906685 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564eb36663d0 con 0x564eb330ed80 2024-05-09T00:55:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.665+0000 7f34cb7fe640 1 --2- 172.21.15.5:0/1131906685 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x564eb330ed80 0x564eb370b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.676+0000 7f34cb7fe640 1 --2- 172.21.15.5:0/1131906685 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x564eb330ed80 0x564eb370b9e0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e593637cc8ab1bf server_cookie=9eeb747eecaf1844 in_seq=0 out_seq=0 2024-05-09T00:55:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.677+0000 7f34afff7640 10 client.25987 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:55:48.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.688+0000 7f34afff7640 1 -- 172.21.15.5:0/1131906685 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564eb36663d0 con 0x564eb330ed80 2024-05-09T00:55:48.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.688+0000 7f34afff7640 10 client.25987 handle_command_reply: tid=0 2024-05-09T00:55:48.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.688+0000 7f34d9204740 1 client.25987 shutdown 2024-05-09T00:55:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.691+0000 7f34d9204740 1 -- 172.21.15.5:0/1131906685 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564eb33272f0 msgr2=0x564eb36fe060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:55:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.691+0000 7f34d9204740 1 --2- 172.21.15.5:0/1131906685 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564eb33272f0 0x564eb36fe060 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f34bc022470 tx=0x7f34bc022930 comp rx=0 tx=0).stop 2024-05-09T00:55:48.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.696+0000 7f34d9204740 1 -- 172.21.15.5:0/1131906685 shutdown_connections 2024-05-09T00:55:48.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.696+0000 7f34d9204740 1 --2- 172.21.15.5:0/1131906685 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x564eb330ed80 0x564eb370b9e0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:48.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.696+0000 7f34d9204740 1 --2- 172.21.15.5:0/1131906685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564eb33501b0 0x564eb3705490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:48.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.696+0000 7f34d9204740 1 --2- 172.21.15.5:0/1131906685 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564eb332f4d0 0x564eb36fe5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:48.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.696+0000 7f34d9204740 1 --2- 172.21.15.5:0/1131906685 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564eb33272f0 0x564eb36fe060 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:48.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.696+0000 7f34afff7640 0 client.25987 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:55:48.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.696+0000 7f34d9204740 1 -- 172.21.15.5:0/1131906685 >> 172.21.15.5:0/1131906685 conn(0x564eb3344300 msgr2=0x564eb36ef290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:55:48.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.698+0000 7f34d9204740 1 -- 172.21.15.5:0/1131906685 shutdown_connections 2024-05-09T00:55:48.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.701+0000 7f34d9204740 1 -- 172.21.15.5:0/1131906685 wait complete. 2024-05-09T00:55:48.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.702+0000 7f34d9204740 20 client.25987 trim_cache size 0 max 16384 2024-05-09T00:55:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.754+0000 7f34d9204740 1 Processor -- start 2024-05-09T00:55:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.755+0000 7f34d9204740 1 -- start start 2024-05-09T00:55:48.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.755+0000 7f34d9204740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564eb33272f0 0x564eb36f0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:48.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.755+0000 7f34d9204740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564eb332f4d0 0x564eb3705420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:48.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.755+0000 7f34d9204740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564eb33501b0 0x564eb3705960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:48.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.755+0000 7f34d9204740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f34b800e650 con 0x564eb332f4d0 2024-05-09T00:55:48.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.755+0000 7f34d9204740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f34b8010190 con 0x564eb33501b0 2024-05-09T00:55:48.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.755+0000 7f34d9204740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f34b800ed20 con 0x564eb33272f0 2024-05-09T00:55:48.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.756+0000 7f34cb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564eb332f4d0 0x564eb3705420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:48.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.756+0000 7f34cb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564eb332f4d0 0x564eb3705420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43544/0 (socket says 172.21.15.5:43544) 2024-05-09T00:55:48.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.756+0000 7f34cb7fe640 1 -- 172.21.15.5:0/2319405075 learned_addr learned my addr 172.21.15.5:0/2319405075 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:55:48.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.770+0000 7f34d08ea640 1 --2- 172.21.15.5:0/2319405075 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564eb33501b0 0x564eb3705960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:48.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.770+0000 7f34cbfff640 1 --2- 172.21.15.5:0/2319405075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564eb33272f0 0x564eb36f0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:48.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.772+0000 7f34cb7fe640 1 -- 172.21.15.5:0/2319405075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564eb33272f0 msgr2=0x564eb36f0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:55:48.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.772+0000 7f34cb7fe640 1 --2- 172.21.15.5:0/2319405075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564eb33272f0 0x564eb36f0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:48.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.772+0000 7f34cb7fe640 1 -- 172.21.15.5:0/2319405075 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564eb33501b0 msgr2=0x564eb3705960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:55:48.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.772+0000 7f34cb7fe640 1 --2- 172.21.15.5:0/2319405075 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564eb33501b0 0x564eb3705960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:48.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.772+0000 7f34cb7fe640 1 -- 172.21.15.5:0/2319405075 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564eb34325f0 con 0x564eb332f4d0 2024-05-09T00:55:48.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.789+0000 7f34cb7fe640 1 --2- 172.21.15.5:0/2319405075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564eb332f4d0 0x564eb3705420 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f34c000f590 tx=0x7f34c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14712cd552a88cd6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:55:48.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.793+0000 7f34b1ffb640 1 -- 172.21.15.5:0/2319405075 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c0014070 con 0x564eb332f4d0 2024-05-09T00:55:48.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.793+0000 7f34b1ffb640 1 -- 172.21.15.5:0/2319405075 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34c002a260 con 0x564eb332f4d0 2024-05-09T00:55:48.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.793+0000 7f34b1ffb640 1 -- 172.21.15.5:0/2319405075 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c0021610 con 0x564eb332f4d0 2024-05-09T00:55:48.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.794+0000 7f34d9204740 1 -- 172.21.15.5:0/2319405075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564eb332f4d0 msgr2=0x564eb3705420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:55:48.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.794+0000 7f34d9204740 1 --2- 172.21.15.5:0/2319405075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564eb332f4d0 0x564eb3705420 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f34c000f590 tx=0x7f34c00046c0 comp rx=0 tx=0).stop 2024-05-09T00:55:48.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.794+0000 7f34d9204740 1 -- 172.21.15.5:0/2319405075 shutdown_connections 2024-05-09T00:55:48.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.794+0000 7f34d9204740 1 --2- 172.21.15.5:0/2319405075 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564eb33501b0 0x564eb3705960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:48.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.794+0000 7f34d9204740 1 --2- 172.21.15.5:0/2319405075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564eb332f4d0 0x564eb3705420 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:48.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.795+0000 7f34d9204740 1 --2- 172.21.15.5:0/2319405075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564eb33272f0 0x564eb36f0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:48.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.795+0000 7f34d9204740 1 -- 172.21.15.5:0/2319405075 >> 172.21.15.5:0/2319405075 conn(0x564eb3344300 msgr2=0x564eb36ef9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:55:48.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.795+0000 7f34d9204740 1 -- 172.21.15.5:0/2319405075 shutdown_connections 2024-05-09T00:55:48.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.795+0000 7f34d9204740 1 -- 172.21.15.5:0/2319405075 wait complete. 2024-05-09T00:55:48.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.796+0000 7f34d9204740 1 -- start start 2024-05-09T00:55:48.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.796+0000 7f34d9204740 1 Processor -- start 2024-05-09T00:55:48.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.802+0000 7f34d9204740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564eb33272f0 0x564eb3702660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:48.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.802+0000 7f34d9204740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564eb332f4d0 0x564eb3702ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:48.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.802+0000 7f34d9204740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564eb33501b0 0x564eb3771d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:48.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.803+0000 7f34d9204740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f34b800fac0 con 0x564eb33272f0 2024-05-09T00:55:48.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.803+0000 7f34d9204740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f34b800f3f0 con 0x564eb332f4d0 2024-05-09T00:55:48.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.803+0000 7f34d9204740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f34b800c630 con 0x564eb33501b0 2024-05-09T00:55:48.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.803+0000 7f34d9204740 10 client.0 _ll_register_callbacks cb 0x7f34bc01a610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:55:48.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.805+0000 7f34d08ea640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564eb33501b0 0x564eb3771d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:48.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.805+0000 7f34cb7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564eb332f4d0 0x564eb3702ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:48.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.805+0000 7f34cbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564eb33272f0 0x564eb3702660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:48.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.806+0000 7f34cbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564eb33272f0 0x564eb3702660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43548/0 (socket says 172.21.15.5:43548) 2024-05-09T00:55:48.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.806+0000 7f34cbfff640 1 -- 172.21.15.5:0/993293382 learned_addr learned my addr 172.21.15.5:0/993293382 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:55:48.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.810+0000 7f34cbfff640 1 -- 172.21.15.5:0/993293382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564eb33501b0 msgr2=0x564eb3771d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:55:48.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.810+0000 7f34cbfff640 1 --2- 172.21.15.5:0/993293382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564eb33501b0 0x564eb3771d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:48.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.810+0000 7f34cbfff640 1 -- 172.21.15.5:0/993293382 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564eb332f4d0 msgr2=0x564eb3702ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:55:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.810+0000 7f34cbfff640 1 --2- 172.21.15.5:0/993293382 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564eb332f4d0 0x564eb3702ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.810+0000 7f34cbfff640 1 -- 172.21.15.5:0/993293382 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564eb361ced0 con 0x564eb33272f0 2024-05-09T00:55:48.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.834+0000 7f34cbfff640 1 --2- 172.21.15.5:0/993293382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564eb33272f0 0x564eb3702660 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f34bc023fd0 tx=0x7f34bc01a800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f2c3ae424ed5bc6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:55:48.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.834+0000 7f34afff7640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:55:48.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.835+0000 7f34afff7640 1 -- 172.21.15.5:0/993293382 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34bc0052f0 con 0x564eb33272f0 2024-05-09T00:55:48.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.835+0000 7f34afff7640 1 -- 172.21.15.5:0/993293382 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f34bc005490 con 0x564eb33272f0 2024-05-09T00:55:48.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.836+0000 7f34d9204740 1 -- 172.21.15.5:0/993293382 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34c002a260 con 0x564eb33272f0 2024-05-09T00:55:48.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.839+0000 7f34afff7640 1 -- 172.21.15.5:0/993293382 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f34bc02f030 con 0x564eb33272f0 2024-05-09T00:55:48.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.839+0000 7f34d9204740 10 client.16473 fetch_fsmap learned FSMap version 31 2024-05-09T00:55:48.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.839+0000 7f34d9204740 1 -- 172.21.15.5:0/993293382 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564eb3773070 con 0x564eb33272f0 2024-05-09T00:55:48.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.841+0000 7f34afff7640 1 -- 172.21.15.5:0/993293382 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34c002a260 con 0x564eb33272f0 2024-05-09T00:55:48.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.842+0000 7f34afff7640 1 -- 172.21.15.5:0/993293382 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f34bc02a9e0 con 0x564eb33272f0 2024-05-09T00:55:48.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.842+0000 7f34d9204740 10 client.16473 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:55:48.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.842+0000 7f34d9204740 10 client.16473 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:55:48.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.842+0000 7f34d9204740 20 client.16473 populate_metadata read hostname 'smithi005' 2024-05-09T00:55:48.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.842+0000 7f34d9204740 1 --2- 172.21.15.5:0/993293382 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x564eb32e41c0 0x564eb3775a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:55:48.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.842+0000 7f34d9204740 4 client.16473 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:55:48.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.842+0000 7f34d9204740 1 -- 172.21.15.5:0/993293382 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564eb3660240 con 0x564eb32e41c0 2024-05-09T00:55:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.844+0000 7f34cb7fe640 1 --2- 172.21.15.5:0/993293382 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x564eb32e41c0 0x564eb3775a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:55:48.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.851+0000 7f34cb7fe640 1 --2- 172.21.15.5:0/993293382 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x564eb32e41c0 0x564eb3775a40 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a7b006dd8c72915 server_cookie=593b9338c7901b9b in_seq=0 out_seq=0 2024-05-09T00:55:48.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:48.851+0000 7f34afff7640 10 client.16473 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:55:49.789 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:55:49 smithi005 ceph-mon[30365]: pgmap v655: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 833 MiB/s wr, 1.81k op/s 2024-05-09T00:55:49.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:55:49 smithi131 ceph-mon[34547]: pgmap v655: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 833 MiB/s wr, 1.81k op/s 2024-05-09T00:55:49.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:55:49 smithi157 ceph-mon[34294]: pgmap v655: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 833 MiB/s wr, 1.81k op/s 2024-05-09T00:55:51.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:55:51 smithi005 ceph-mon[30365]: pgmap v656: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 994 MiB/s wr, 2.01k op/s 2024-05-09T00:55:51.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:55:51 smithi131 ceph-mon[34547]: pgmap v656: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 994 MiB/s wr, 2.01k op/s 2024-05-09T00:55:51.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:55:51 smithi157 ceph-mon[34294]: pgmap v656: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 994 MiB/s wr, 2.01k op/s 2024-05-09T00:55:53.790 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:55:53 smithi005 ceph-mon[30365]: pgmap v657: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 728 MiB/s wr, 1.60k op/s 2024-05-09T00:55:53.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:55:53 smithi131 ceph-mon[34547]: pgmap v657: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 728 MiB/s wr, 1.60k op/s 2024-05-09T00:55:53.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:55:53 smithi157 ceph-mon[34294]: pgmap v657: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 728 MiB/s wr, 1.60k op/s 2024-05-09T00:55:55.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:55:55 smithi131 ceph-mon[34547]: pgmap v658: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 909 MiB/s wr, 1.93k op/s 2024-05-09T00:55:55.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:55:55 smithi157 ceph-mon[34294]: pgmap v658: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 909 MiB/s wr, 1.93k op/s 2024-05-09T00:55:56.043 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:55:55 smithi005 ceph-mon[30365]: pgmap v658: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 909 MiB/s wr, 1.93k op/s 2024-05-09T00:55:56.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:56.107+0000 7f34afff7640 1 -- 172.21.15.5:0/993293382 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x564eb3660240 con 0x564eb32e41c0 2024-05-09T00:55:56.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:56.107+0000 7f34afff7640 10 client.16473 handle_command_reply: tid=0 2024-05-09T00:55:56.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:56.107+0000 7f34d9204740 1 client.16473 shutdown 2024-05-09T00:55:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:56.109+0000 7f34d9204740 1 -- 172.21.15.5:0/993293382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564eb33272f0 msgr2=0x564eb3702660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:55:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:56.109+0000 7f34d9204740 1 --2- 172.21.15.5:0/993293382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564eb33272f0 0x564eb3702660 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f34bc023fd0 tx=0x7f34bc01a800 comp rx=0 tx=0).stop 2024-05-09T00:55:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:56.110+0000 7f34d9204740 1 -- 172.21.15.5:0/993293382 shutdown_connections 2024-05-09T00:55:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:56.110+0000 7f34d9204740 1 --2- 172.21.15.5:0/993293382 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x564eb32e41c0 0x564eb3775a40 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:56.110+0000 7f34d9204740 1 --2- 172.21.15.5:0/993293382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564eb33501b0 0x564eb3771d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:56.110+0000 7f34d9204740 1 --2- 172.21.15.5:0/993293382 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564eb332f4d0 0x564eb3702ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:56.110+0000 7f34d9204740 1 --2- 172.21.15.5:0/993293382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564eb33272f0 0x564eb3702660 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:56.110+0000 7f34d9204740 1 -- 172.21.15.5:0/993293382 >> 172.21.15.5:0/993293382 conn(0x564eb3344300 msgr2=0x564eb36ee860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:55:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:56.110+0000 7f34afff7640 0 client.16473 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:55:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:56.110+0000 7f34d9204740 1 -- 172.21.15.5:0/993293382 shutdown_connections 2024-05-09T00:55:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:56.110+0000 7f34d9204740 1 -- 172.21.15.5:0/993293382 wait complete. 2024-05-09T00:55:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:56.110+0000 7f34d9204740 20 client.16473 trim_cache size 0 max 16384 2024-05-09T00:55:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-09T00:55:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-09T00:55:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-09T00:55:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:56.112+0000 7f34d10eb640 1 -- 172.21.15.5:0/2482708517 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f34b40b2970 msgr2=0x7f34b40b4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:55:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:56.112+0000 7f34d10eb640 1 --2- 172.21.15.5:0/2482708517 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f34b40b2970 0x7f34b40b4e80 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x564eb36f5aa0 tx=0x7f34c0015040 comp rx=0 tx=0).stop 2024-05-09T00:55:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:56.112+0000 7f34d10eb640 1 -- 172.21.15.5:0/2482708517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34cc10ffe0 msgr2=0x7f34cc1a7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:55:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:56.112+0000 7f34d10eb640 1 --2- 172.21.15.5:0/2482708517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34cc10ffe0 0x7f34cc1a7be0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f34c4034040 tx=0x7f34c4030c80 comp rx=0 tx=0).stop 2024-05-09T00:55:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:56.117+0000 7f34d10eb640 1 -- 172.21.15.5:0/2482708517 shutdown_connections 2024-05-09T00:55:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:56.117+0000 7f34d10eb640 1 --2- 172.21.15.5:0/2482708517 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f34b40b2970 0x7f34b40b4e80 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:56.117+0000 7f34d10eb640 1 --2- 172.21.15.5:0/2482708517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34cc10ffe0 0x7f34cc1a7be0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:56.117+0000 7f34d10eb640 1 --2- 172.21.15.5:0/2482708517 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34cc10b2c0 0x7f34cc1a0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:56.117+0000 7f34d10eb640 1 --2- 172.21.15.5:0/2482708517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34cc10a470 0x7f34cc1a0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:55:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:56.117+0000 7f34d10eb640 1 -- 172.21.15.5:0/2482708517 >> 172.21.15.5:0/2482708517 conn(0x7f34cc070460 msgr2=0x7f34cc10dfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:55:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:56.117+0000 7f34d10eb640 1 -- 172.21.15.5:0/2482708517 shutdown_connections 2024-05-09T00:55:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:55:56.118+0000 7f34d10eb640 1 -- 172.21.15.5:0/2482708517 wait complete. 2024-05-09T00:55:57.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:55:57 smithi131 ceph-mon[34547]: pgmap v659: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 904 MiB/s wr, 1.72k op/s 2024-05-09T00:55:57.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:55:57 smithi157 ceph-mon[34294]: pgmap v659: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 904 MiB/s wr, 1.72k op/s 2024-05-09T00:55:58.054 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:55:57 smithi005 ceph-mon[30365]: pgmap v659: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 904 MiB/s wr, 1.72k op/s 2024-05-09T00:55:59.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:55:59 smithi131 ceph-mon[34547]: pgmap v660: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 847 MiB/s wr, 1.47k op/s 2024-05-09T00:55:59.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:55:59 smithi157 ceph-mon[34294]: pgmap v660: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 847 MiB/s wr, 1.47k op/s 2024-05-09T00:56:00.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:55:59 smithi005 ceph-mon[30365]: pgmap v660: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 847 MiB/s wr, 1.47k op/s 2024-05-09T00:56:01.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:56:01 smithi157 ceph-mon[34294]: pgmap v661: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1015 MiB/s wr, 2.14k op/s 2024-05-09T00:56:02.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:56:01 smithi005 ceph-mon[30365]: pgmap v661: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1015 MiB/s wr, 2.14k op/s 2024-05-09T00:56:02.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:56:01 smithi131 ceph-mon[34547]: pgmap v661: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1015 MiB/s wr, 2.14k op/s 2024-05-09T00:56:03.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:56:03 smithi157 ceph-mon[34294]: pgmap v662: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 763 MiB/s wr, 1.70k op/s 2024-05-09T00:56:04.040 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:56:03 smithi005 ceph-mon[30365]: pgmap v662: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 763 MiB/s wr, 1.70k op/s 2024-05-09T00:56:04.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:56:03 smithi131 ceph-mon[34547]: pgmap v662: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 763 MiB/s wr, 1.70k op/s 2024-05-09T00:56:05.876 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:56:05 smithi005 ceph-mon[30365]: pgmap v663: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 942 MiB/s wr, 2.15k op/s 2024-05-09T00:56:05.905 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:56:05 smithi131 ceph-mon[34547]: pgmap v663: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 942 MiB/s wr, 2.15k op/s 2024-05-09T00:56:05.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:56:05 smithi157 ceph-mon[34294]: pgmap v663: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 942 MiB/s wr, 2.15k op/s 2024-05-09T00:56:07.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:56:07 smithi157 ceph-mon[34294]: pgmap v664: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 926 MiB/s wr, 2.19k op/s 2024-05-09T00:56:08.040 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:56:07 smithi005 ceph-mon[30365]: pgmap v664: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 926 MiB/s wr, 2.19k op/s 2024-05-09T00:56:08.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:56:07 smithi131 ceph-mon[34547]: pgmap v664: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 926 MiB/s wr, 2.19k op/s 2024-05-09T00:56:08.392 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T00:56:09.414 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:56:09.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.525+0000 7f77e0b4c640 1 -- 172.21.15.5:0/3257503083 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d8037070 con 0x7f77dc10a550 2024-05-09T00:56:09.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.528+0000 7f77e3db2640 1 -- 172.21.15.5:0/3257503083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77dc10a550 msgr2=0x7f77dc10af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:56:09.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.528+0000 7f77e3db2640 1 --2- 172.21.15.5:0/3257503083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77dc10a550 0x7f77dc10af40 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f77d8009860 tx=0x7f77d802a380 comp rx=0 tx=0).stop 2024-05-09T00:56:09.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.544+0000 7f77e3db2640 1 -- 172.21.15.5:0/3257503083 shutdown_connections 2024-05-09T00:56:09.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.544+0000 7f77e3db2640 1 --2- 172.21.15.5:0/3257503083 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77dc1101a0 0x7f77dc1105b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:09.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.544+0000 7f77e3db2640 1 --2- 172.21.15.5:0/3257503083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77dc10b480 0x7f77dc10b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:09.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.544+0000 7f77e3db2640 1 --2- 172.21.15.5:0/3257503083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77dc10a550 0x7f77dc10af40 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:09.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.544+0000 7f77e3db2640 1 -- 172.21.15.5:0/3257503083 >> 172.21.15.5:0/3257503083 conn(0x7f77dc070670 msgr2=0x7f77dc072b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:56:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.553+0000 7f77e3db2640 1 -- 172.21.15.5:0/3257503083 shutdown_connections 2024-05-09T00:56:09.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.568+0000 7f77e3db2640 1 -- 172.21.15.5:0/3257503083 wait complete. 2024-05-09T00:56:09.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.569+0000 7f77e3db2640 1 Processor -- start 2024-05-09T00:56:09.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.569+0000 7f77e3db2640 1 -- start start 2024-05-09T00:56:09.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.570+0000 7f77e3db2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77dc10a550 0x7f77dc1171e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:09.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.570+0000 7f77e3db2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77dc10b480 0x7f77dc117720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:09.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.570+0000 7f77e3db2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77dc1101a0 0x7f77dc118680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:09.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.570+0000 7f77e3db2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f77dc11e650 con 0x7f77dc10a550 2024-05-09T00:56:09.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.570+0000 7f77e3db2640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f77dc11e4d0 con 0x7f77dc1101a0 2024-05-09T00:56:09.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.570+0000 7f77e3db2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f77dc11e7d0 con 0x7f77dc10b480 2024-05-09T00:56:09.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.571+0000 7f77e234f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77dc1101a0 0x7f77dc118680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:09.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.571+0000 7f77e234f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77dc1101a0 0x7f77dc118680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:40366/0 (socket says 172.21.15.5:40366) 2024-05-09T00:56:09.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.571+0000 7f77e234f640 1 -- 172.21.15.5:0/2775175531 learned_addr learned my addr 172.21.15.5:0/2775175531 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:56:09.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.572+0000 7f77e234f640 1 -- 172.21.15.5:0/2775175531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77dc10b480 msgr2=0x7f77dc117720 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T00:56:09.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.572+0000 7f77e234f640 1 --2- 172.21.15.5:0/2775175531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77dc10b480 0x7f77dc117720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:09.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.572+0000 7f77e234f640 1 -- 172.21.15.5:0/2775175531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77dc10a550 msgr2=0x7f77dc1171e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:56:09.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.572+0000 7f77e234f640 1 --2- 172.21.15.5:0/2775175531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77dc10a550 0x7f77dc1171e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:09.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.572+0000 7f77e234f640 1 -- 172.21.15.5:0/2775175531 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77dc118cf0 con 0x7f77dc1101a0 2024-05-09T00:56:09.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.574+0000 7f77e234f640 1 --2- 172.21.15.5:0/2775175531 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77dc1101a0 0x7f77dc118680 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f77d4009fd0 tx=0x7f77d400c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:56:09.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.577+0000 7f77d2ffd640 1 -- 172.21.15.5:0/2775175531 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d4004b20 con 0x7f77dc1101a0 2024-05-09T00:56:09.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.577+0000 7f77d2ffd640 1 -- 172.21.15.5:0/2775175531 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77d4003e20 con 0x7f77dc1101a0 2024-05-09T00:56:09.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.577+0000 7f77d2ffd640 1 -- 172.21.15.5:0/2775175531 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d4004820 con 0x7f77dc1101a0 2024-05-09T00:56:09.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.577+0000 7f77e3db2640 1 -- 172.21.15.5:0/2775175531 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77dc118fe0 con 0x7f77dc1101a0 2024-05-09T00:56:09.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.578+0000 7f77e3db2640 1 -- 172.21.15.5:0/2775175531 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77dc1194e0 con 0x7f77dc1101a0 2024-05-09T00:56:09.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.583+0000 7f77eac66740 1 Processor -- start 2024-05-09T00:56:09.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.586+0000 7f77d2ffd640 1 -- 172.21.15.5:0/2775175531 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f77d401d020 con 0x7f77dc1101a0 2024-05-09T00:56:09.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.587+0000 7f77eac66740 1 -- start start 2024-05-09T00:56:09.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.587+0000 7f77eac66740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a1b30a31b0 0x55a1b3447500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:09.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.588+0000 7f77d2ffd640 1 --2- 172.21.15.5:0/2775175531 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f77c80b2930 0x7f77c80b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:09.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.589+0000 7f77e1b4e640 1 --2- 172.21.15.5:0/2775175531 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f77c80b2930 0x7f77c80b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:09.590 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.590+0000 7f77eac66740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a1b307a2f0 0x55a1b3447a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:09.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.590+0000 7f77e134d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a1b30a31b0 0x55a1b3447500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:09.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.590+0000 7f77eac66740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1b30824d0 0x55a1b344e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:09.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.590+0000 7f77eac66740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a1b33ab7c0 con 0x55a1b30824d0 2024-05-09T00:56:09.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.590+0000 7f77eac66740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55a1b33470e0 con 0x55a1b30a31b0 2024-05-09T00:56:09.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.590+0000 7f77eac66740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55a1b33a7780 con 0x55a1b307a2f0 2024-05-09T00:56:09.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.590+0000 7f77e134d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a1b30a31b0 0x55a1b3447500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:40370/0 (socket says 172.21.15.5:40370) 2024-05-09T00:56:09.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.590+0000 7f77e134d640 1 -- 172.21.15.5:0/66281191 learned_addr learned my addr 172.21.15.5:0/66281191 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:56:09.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.591+0000 7f77d2ffd640 1 -- 172.21.15.5:0/2775175531 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f77d40e2260 con 0x7f77dc1101a0 2024-05-09T00:56:09.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.591+0000 7f77e134d640 1 --2- 172.21.15.5:0/66281191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a1b307a2f0 0x55a1b3447a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:09.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.591+0000 7f77e234f640 1 --2- 172.21.15.5:0/66281191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1b30824d0 0x55a1b344e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:09.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.592+0000 7f77e134d640 1 -- 172.21.15.5:0/66281191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a1b307a2f0 msgr2=0x55a1b3447a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:56:09.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.592+0000 7f77e134d640 1 --2- 172.21.15.5:0/66281191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a1b307a2f0 0x55a1b3447a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:09.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.592+0000 7f77e134d640 1 -- 172.21.15.5:0/66281191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1b30824d0 msgr2=0x55a1b344e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:56:09.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.592+0000 7f77e134d640 1 --2- 172.21.15.5:0/66281191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1b30824d0 0x55a1b344e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:09.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.592+0000 7f77e134d640 1 -- 172.21.15.5:0/66281191 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1b33707c0 con 0x55a1b30a31b0 2024-05-09T00:56:09.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.593+0000 7f77e134d640 1 --2- 172.21.15.5:0/66281191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a1b30a31b0 0x55a1b3447500 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f77cc013750 tx=0x7f77cc005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c074451820fa96d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:56:09.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.594+0000 7f77e1b4e640 1 --2- 172.21.15.5:0/2775175531 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f77c80b2930 0x7f77c80b4e40 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f77d80057c0 tx=0x7f77d80305f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:56:09.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.600+0000 7f77c7fff640 1 -- 172.21.15.5:0/66281191 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77cc004220 con 0x55a1b30a31b0 2024-05-09T00:56:09.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.600+0000 7f77c7fff640 1 -- 172.21.15.5:0/66281191 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77cc0043c0 con 0x55a1b30a31b0 2024-05-09T00:56:09.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.602+0000 7f77eac66740 1 -- 172.21.15.5:0/66281191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a1b30a31b0 msgr2=0x55a1b3447500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:56:09.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.602+0000 7f77eac66740 1 --2- 172.21.15.5:0/66281191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a1b30a31b0 0x55a1b3447500 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f77cc013750 tx=0x7f77cc005f90 comp rx=0 tx=0).stop 2024-05-09T00:56:09.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.608+0000 7f77eac66740 1 -- 172.21.15.5:0/66281191 shutdown_connections 2024-05-09T00:56:09.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.608+0000 7f77eac66740 1 --2- 172.21.15.5:0/66281191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1b30824d0 0x55a1b344e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:09.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.608+0000 7f77eac66740 1 --2- 172.21.15.5:0/66281191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a1b307a2f0 0x55a1b3447a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:09.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.608+0000 7f77eac66740 1 --2- 172.21.15.5:0/66281191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a1b30a31b0 0x55a1b3447500 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:09.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.608+0000 7f77eac66740 1 -- 172.21.15.5:0/66281191 >> 172.21.15.5:0/66281191 conn(0x55a1b3097300 msgr2=0x55a1b3442e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:56:09.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.608+0000 7f77eac66740 1 -- 172.21.15.5:0/66281191 shutdown_connections 2024-05-09T00:56:09.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.610+0000 7f77eac66740 1 -- 172.21.15.5:0/66281191 wait complete. 2024-05-09T00:56:09.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.611+0000 7f77eac66740 1 -- start start 2024-05-09T00:56:09.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.611+0000 7f77eac66740 1 Processor -- start 2024-05-09T00:56:09.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.612+0000 7f77eac66740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a1b307a2f0 0x55a1b3451360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:09.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.612+0000 7f77eac66740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1b30824d0 0x55a1b34518a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:09.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.612+0000 7f77eac66740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a1b30a31b0 0x55a1b3458800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:09.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.612+0000 7f77eac66740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a1b344f420 con 0x55a1b30824d0 2024-05-09T00:56:09.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.612+0000 7f77eac66740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55a1b33329c0 con 0x55a1b307a2f0 2024-05-09T00:56:09.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.612+0000 7f77eac66740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55a1b344f5a0 con 0x55a1b30a31b0 2024-05-09T00:56:09.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.612+0000 7f77eac66740 10 client.0 _ll_register_callbacks cb 0x55a1b335f020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:56:09.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.613+0000 7f77e134d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1b30824d0 0x55a1b34518a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:09.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.613+0000 7f77e134d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1b30824d0 0x55a1b34518a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49166/0 (socket says 172.21.15.5:49166) 2024-05-09T00:56:09.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.613+0000 7f77e134d640 1 -- 172.21.15.5:0/2512158353 learned_addr learned my addr 172.21.15.5:0/2512158353 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:56:09.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.613+0000 7f77e1b4e640 1 --2- 172.21.15.5:0/2512158353 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a1b307a2f0 0x55a1b3451360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:09.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.613+0000 7f77e134d640 1 -- 172.21.15.5:0/2512158353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a1b30a31b0 msgr2=0x55a1b3458800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:56:09.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.613+0000 7f77e234f640 1 --2- 172.21.15.5:0/2512158353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a1b30a31b0 0x55a1b3458800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:09.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.614+0000 7f77e134d640 1 --2- 172.21.15.5:0/2512158353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a1b30a31b0 0x55a1b3458800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:09.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.614+0000 7f77e134d640 1 -- 172.21.15.5:0/2512158353 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a1b307a2f0 msgr2=0x55a1b3451360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:56:09.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.614+0000 7f77e134d640 1 --2- 172.21.15.5:0/2512158353 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a1b307a2f0 0x55a1b3451360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:09.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.614+0000 7f77e134d640 1 -- 172.21.15.5:0/2512158353 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1b33591f0 con 0x55a1b30824d0 2024-05-09T00:56:09.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.614+0000 7f77e234f640 1 --2- 172.21.15.5:0/2512158353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a1b30a31b0 0x55a1b3458800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:56:09.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.614+0000 7f77e134d640 1 --2- 172.21.15.5:0/2512158353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1b30824d0 0x55a1b34518a0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f77cc00cb40 tx=0x7f77cc00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd6215392f3a02b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:56:09.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.615+0000 7f77c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:56:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.630+0000 7f77c57fa640 1 -- 172.21.15.5:0/2512158353 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77cc013230 con 0x55a1b30824d0 2024-05-09T00:56:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.630+0000 7f77eac66740 1 -- 172.21.15.5:0/2512158353 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77cc0043c0 con 0x55a1b30824d0 2024-05-09T00:56:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.630+0000 7f77c57fa640 1 -- 172.21.15.5:0/2512158353 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f77cc004a00 con 0x55a1b30824d0 2024-05-09T00:56:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.631+0000 7f77c57fa640 1 -- 172.21.15.5:0/2512158353 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f77cc0133d0 con 0x55a1b30824d0 2024-05-09T00:56:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.631+0000 7f77eac66740 10 client.16482 fetch_fsmap learned FSMap version 31 2024-05-09T00:56:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.631+0000 7f77eac66740 1 -- 172.21.15.5:0/2512158353 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a1b310e790 con 0x55a1b30824d0 2024-05-09T00:56:09.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.632+0000 7f77c57fa640 1 -- 172.21.15.5:0/2512158353 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77cc0043c0 con 0x55a1b30824d0 2024-05-09T00:56:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.634+0000 7f77c57fa640 1 -- 172.21.15.5:0/2512158353 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f77cc02f070 con 0x55a1b30824d0 2024-05-09T00:56:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.634+0000 7f77eac66740 10 client.16482 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:56:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.634+0000 7f77eac66740 10 client.16482 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:56:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.634+0000 7f77eac66740 20 client.16482 populate_metadata read hostname 'smithi005' 2024-05-09T00:56:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.634+0000 7f77eac66740 1 --2- 172.21.15.5:0/2512158353 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55a1b3061d80 0x55a1b345e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.634+0000 7f77eac66740 4 client.16482 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:56:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.634+0000 7f77eac66740 1 -- 172.21.15.5:0/2512158353 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1b33b93d0 con 0x55a1b3061d80 2024-05-09T00:56:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.635+0000 7f77e1b4e640 1 --2- 172.21.15.5:0/2512158353 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55a1b3061d80 0x55a1b345e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.635+0000 7f77e1b4e640 1 --2- 172.21.15.5:0/2512158353 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55a1b3061d80 0x55a1b345e980 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53c29a924b5b025d server_cookie=4d052b18e88d95d4 in_seq=0 out_seq=0 2024-05-09T00:56:09.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.637+0000 7f77c57fa640 10 client.16482 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:56:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.638+0000 7f77c57fa640 1 -- 172.21.15.5:0/2512158353 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a1b33b93d0 con 0x55a1b3061d80 2024-05-09T00:56:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.638+0000 7f77c57fa640 10 client.16482 handle_command_reply: tid=0 2024-05-09T00:56:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.638+0000 7f77eac66740 1 client.16482 shutdown 2024-05-09T00:56:09.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.641+0000 7f77eac66740 1 -- 172.21.15.5:0/2512158353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1b30824d0 msgr2=0x55a1b34518a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:56:09.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.641+0000 7f77eac66740 1 --2- 172.21.15.5:0/2512158353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1b30824d0 0x55a1b34518a0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f77cc00cb40 tx=0x7f77cc00cb70 comp rx=0 tx=0).stop 2024-05-09T00:56:09.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.641+0000 7f77eac66740 1 -- 172.21.15.5:0/2512158353 shutdown_connections 2024-05-09T00:56:09.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.641+0000 7f77eac66740 1 --2- 172.21.15.5:0/2512158353 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55a1b3061d80 0x55a1b345e980 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:09.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.641+0000 7f77eac66740 1 --2- 172.21.15.5:0/2512158353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a1b30a31b0 0x55a1b3458800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:09.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.641+0000 7f77eac66740 1 --2- 172.21.15.5:0/2512158353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1b30824d0 0x55a1b34518a0 secure :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f77cc00cb40 tx=0x7f77cc00cb70 comp rx=0 tx=0).stop 2024-05-09T00:56:09.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.641+0000 7f77eac66740 1 --2- 172.21.15.5:0/2512158353 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a1b307a2f0 0x55a1b3451360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:09.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.641+0000 7f77eac66740 1 -- 172.21.15.5:0/2512158353 >> 172.21.15.5:0/2512158353 conn(0x55a1b3097300 msgr2=0x55a1b3441700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:56:09.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.642+0000 7f77c57fa640 0 client.16482 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:56:09.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.645+0000 7f77eac66740 1 -- 172.21.15.5:0/2512158353 shutdown_connections 2024-05-09T00:56:09.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.645+0000 7f77eac66740 1 -- 172.21.15.5:0/2512158353 wait complete. 2024-05-09T00:56:09.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.646+0000 7f77eac66740 20 client.16482 trim_cache size 0 max 16384 2024-05-09T00:56:09.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.710+0000 7f77eac66740 1 Processor -- start 2024-05-09T00:56:09.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.711+0000 7f77eac66740 1 -- start start 2024-05-09T00:56:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.715+0000 7f77eac66740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1b307a2f0 0x55a1b344a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.715+0000 7f77eac66740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a1b30a31b0 0x55a1b344ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.715+0000 7f77eac66740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a1b30371c0 0x55a1b34587c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.715+0000 7f77eac66740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f77dc1c2a30 con 0x55a1b307a2f0 2024-05-09T00:56:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.715+0000 7f77eac66740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f77dc1c4570 con 0x55a1b30371c0 2024-05-09T00:56:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.715+0000 7f77eac66740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f77dc1c3100 con 0x55a1b30a31b0 2024-05-09T00:56:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.716+0000 7f77e134d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a1b30a31b0 0x55a1b344ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.716+0000 7f77e134d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a1b30a31b0 0x55a1b344ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:50284/0 (socket says 172.21.15.5:50284) 2024-05-09T00:56:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.716+0000 7f77e134d640 1 -- 172.21.15.5:0/455907289 learned_addr learned my addr 172.21.15.5:0/455907289 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:56:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.716+0000 7f77e234f640 1 --2- 172.21.15.5:0/455907289 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a1b30371c0 0x55a1b34587c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.717+0000 7f77e134d640 1 -- 172.21.15.5:0/455907289 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a1b30371c0 msgr2=0x55a1b34587c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:56:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.717+0000 7f77e134d640 1 --2- 172.21.15.5:0/455907289 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a1b30371c0 0x55a1b34587c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.717+0000 7f77e134d640 1 -- 172.21.15.5:0/455907289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1b307a2f0 msgr2=0x55a1b344a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:56:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.717+0000 7f77e134d640 1 --2- 172.21.15.5:0/455907289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1b307a2f0 0x55a1b344a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.717+0000 7f77e134d640 1 -- 172.21.15.5:0/455907289 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1b31855f0 con 0x55a1b30a31b0 2024-05-09T00:56:09.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.718+0000 7f77e134d640 1 --2- 172.21.15.5:0/455907289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a1b30a31b0 0x55a1b344ac90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f77cc00cb10 tx=0x7f77cc00ed70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0a82171b9ba156a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:56:09.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.724+0000 7f77c77fe640 1 -- 172.21.15.5:0/455907289 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77cc036070 con 0x55a1b30a31b0 2024-05-09T00:56:09.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.725+0000 7f77c77fe640 1 -- 172.21.15.5:0/455907289 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77cc02c070 con 0x55a1b30a31b0 2024-05-09T00:56:09.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.725+0000 7f77c77fe640 1 -- 172.21.15.5:0/455907289 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77cc013030 con 0x55a1b30a31b0 2024-05-09T00:56:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.727+0000 7f77eac66740 1 -- 172.21.15.5:0/455907289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a1b30a31b0 msgr2=0x55a1b344ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:56:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.727+0000 7f77eac66740 1 --2- 172.21.15.5:0/455907289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a1b30a31b0 0x55a1b344ac90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f77cc00cb10 tx=0x7f77cc00ed70 comp rx=0 tx=0).stop 2024-05-09T00:56:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.727+0000 7f77eac66740 1 -- 172.21.15.5:0/455907289 shutdown_connections 2024-05-09T00:56:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.727+0000 7f77eac66740 1 --2- 172.21.15.5:0/455907289 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a1b30371c0 0x55a1b34587c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.727+0000 7f77eac66740 1 --2- 172.21.15.5:0/455907289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a1b30a31b0 0x55a1b344ac90 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:09.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.727+0000 7f77eac66740 1 --2- 172.21.15.5:0/455907289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1b307a2f0 0x55a1b344a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:09.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.728+0000 7f77eac66740 1 -- 172.21.15.5:0/455907289 >> 172.21.15.5:0/455907289 conn(0x55a1b3097300 msgr2=0x55a1b344b6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:56:09.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.728+0000 7f77eac66740 1 -- 172.21.15.5:0/455907289 shutdown_connections 2024-05-09T00:56:09.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.729+0000 7f77eac66740 1 -- 172.21.15.5:0/455907289 wait complete. 2024-05-09T00:56:09.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.729+0000 7f77eac66740 1 -- start start 2024-05-09T00:56:09.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.730+0000 7f77eac66740 1 Processor -- start 2024-05-09T00:56:09.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.730+0000 7f77eac66740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1b30371c0 0x55a1b3458f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:09.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.731+0000 7f77eac66740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a1b307a2f0 0x55a1b3455b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:09.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.731+0000 7f77eac66740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a1b30a31b0 0x55a1b3456080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:09.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.731+0000 7f77eac66740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f77dc1c3ea0 con 0x55a1b30371c0 2024-05-09T00:56:09.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.731+0000 7f77eac66740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f77dc1c37d0 con 0x55a1b307a2f0 2024-05-09T00:56:09.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.731+0000 7f77eac66740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f77dc1c0ac0 con 0x55a1b30a31b0 2024-05-09T00:56:09.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.731+0000 7f77eac66740 10 client.0 _ll_register_callbacks cb 0x7f77cc00ec40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:56:09.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.732+0000 7f77e234f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a1b30a31b0 0x55a1b3456080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:09.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.732+0000 7f77e234f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a1b30a31b0 0x55a1b3456080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:50288/0 (socket says 172.21.15.5:50288) 2024-05-09T00:56:09.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.732+0000 7f77e234f640 1 -- 172.21.15.5:0/3140216377 learned_addr learned my addr 172.21.15.5:0/3140216377 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:56:09.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.733+0000 7f77e134d640 1 --2- 172.21.15.5:0/3140216377 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a1b307a2f0 0x55a1b3455b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:09.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.734+0000 7f77e234f640 1 -- 172.21.15.5:0/3140216377 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a1b307a2f0 msgr2=0x55a1b3455b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:56:09.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.734+0000 7f77e234f640 1 --2- 172.21.15.5:0/3140216377 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a1b307a2f0 0x55a1b3455b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:09.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.734+0000 7f77e234f640 1 -- 172.21.15.5:0/3140216377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1b30371c0 msgr2=0x55a1b3458f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:56:09.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.734+0000 7f77e234f640 1 --2- 172.21.15.5:0/3140216377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1b30371c0 0x55a1b3458f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:09.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.734+0000 7f77e234f640 1 -- 172.21.15.5:0/3140216377 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1b336fed0 con 0x55a1b30a31b0 2024-05-09T00:56:09.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.735+0000 7f77e234f640 1 --2- 172.21.15.5:0/3140216377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a1b30a31b0 0x55a1b3456080 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f77d40dfd30 tx=0x7f77d40dfc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f493a10d3e1a436a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:56:09.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.743+0000 7f77c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-09T00:56:09.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.743+0000 7f77c57fa640 1 -- 172.21.15.5:0/3140216377 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d40c2020 con 0x55a1b30a31b0 2024-05-09T00:56:09.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.743+0000 7f77eac66740 1 -- 172.21.15.5:0/3140216377 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77cc02c070 con 0x55a1b30a31b0 2024-05-09T00:56:09.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.743+0000 7f77c57fa640 1 -- 172.21.15.5:0/3140216377 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f77d4004de0 con 0x55a1b30a31b0 2024-05-09T00:56:09.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.745+0000 7f77c57fa640 1 -- 172.21.15.5:0/3140216377 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f77d401a040 con 0x55a1b30a31b0 2024-05-09T00:56:09.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.747+0000 7f77eac66740 10 client.25999 fetch_fsmap learned FSMap version 31 2024-05-09T00:56:09.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.748+0000 7f77eac66740 1 -- 172.21.15.5:0/3140216377 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a1b34c5ca0 con 0x55a1b30a31b0 2024-05-09T00:56:09.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.748+0000 7f77c57fa640 1 -- 172.21.15.5:0/3140216377 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77cc02c070 con 0x55a1b30a31b0 2024-05-09T00:56:09.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.748+0000 7f77c57fa640 1 -- 172.21.15.5:0/3140216377 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f77d400b040 con 0x55a1b30a31b0 2024-05-09T00:56:09.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.748+0000 7f77eac66740 10 client.25999 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:56:09.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.748+0000 7f77eac66740 10 client.25999 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:56:09.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.748+0000 7f77eac66740 20 client.25999 populate_metadata read hostname 'smithi005' 2024-05-09T00:56:09.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.749+0000 7f77eac66740 1 --2- 172.21.15.5:0/3140216377 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55a1b34c6600 0x55a1b34c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:09.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.749+0000 7f77eac66740 4 client.25999 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:56:09.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.749+0000 7f77eac66740 1 -- 172.21.15.5:0/3140216377 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a1b33b3240 con 0x55a1b34c6600 2024-05-09T00:56:09.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.750+0000 7f77e1b4e640 1 --2- 172.21.15.5:0/3140216377 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55a1b34c6600 0x55a1b34c8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:09.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.752+0000 7f77e1b4e640 1 --2- 172.21.15.5:0/3140216377 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55a1b34c6600 0x55a1b34c8a40 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61c80e9b234f3659 server_cookie=ecd713958534eeca in_seq=0 out_seq=0 2024-05-09T00:56:09.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:09.753+0000 7f77c57fa640 10 client.25999 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:56:09.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:56:09 smithi157 ceph-mon[34294]: pgmap v665: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 855 MiB/s wr, 2.19k op/s 2024-05-09T00:56:10.040 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:56:09 smithi005 ceph-mon[30365]: pgmap v665: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 855 MiB/s wr, 2.19k op/s 2024-05-09T00:56:10.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:56:09 smithi131 ceph-mon[34547]: pgmap v665: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 855 MiB/s wr, 2.19k op/s 2024-05-09T00:56:12.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:56:11 smithi005 ceph-mon[30365]: pgmap v666: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1020 MiB/s wr, 2.44k op/s 2024-05-09T00:56:12.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:56:11 smithi131 ceph-mon[34547]: pgmap v666: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1020 MiB/s wr, 2.44k op/s 2024-05-09T00:56:12.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:56:11 smithi157 ceph-mon[34294]: pgmap v666: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1020 MiB/s wr, 2.44k op/s 2024-05-09T00:56:14.040 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:56:13 smithi005 ceph-mon[30365]: pgmap v667: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 767 MiB/s wr, 1.66k op/s 2024-05-09T00:56:14.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:56:13 smithi131 ceph-mon[34547]: pgmap v667: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 767 MiB/s wr, 1.66k op/s 2024-05-09T00:56:14.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:56:13 smithi157 ceph-mon[34294]: pgmap v667: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 767 MiB/s wr, 1.66k op/s 2024-05-09T00:56:15.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:56:15 smithi005 ceph-mon[30365]: pgmap v668: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 908 MiB/s wr, 1.95k op/s 2024-05-09T00:56:16.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.047+0000 7fd50deff700 1 -- 172.21.15.5:0/393760517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd50810a960 msgr2=0x7fd50810ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:56:16.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.047+0000 7fd50deff700 1 --2- 172.21.15.5:0/393760517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd50810a960 0x7fd50810ad70 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fd50000b0a0 tx=0x7fd50000b3b0 comp rx=0 tx=0).stop 2024-05-09T00:56:16.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.047+0000 7fd50deff700 1 -- 172.21.15.5:0/393760517 shutdown_connections 2024-05-09T00:56:16.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.047+0000 7fd50deff700 1 --2- 172.21.15.5:0/393760517 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd50810b2b0 0x7fd508117830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:16.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.047+0000 7fd50deff700 1 --2- 172.21.15.5:0/393760517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd508110990 0x7fd508110da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:16.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.047+0000 7fd50deff700 1 --2- 172.21.15.5:0/393760517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd50810a960 0x7fd50810ad70 secure :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fd50000b0a0 tx=0x7fd50000b3b0 comp rx=0 tx=0).stop 2024-05-09T00:56:16.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.047+0000 7fd50deff700 1 -- 172.21.15.5:0/393760517 >> 172.21.15.5:0/393760517 conn(0x7fd508076d30 msgr2=0x7fd508077140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:56:16.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.051+0000 7fd50deff700 1 -- 172.21.15.5:0/393760517 shutdown_connections 2024-05-09T00:56:16.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.052+0000 7fd50deff700 1 -- 172.21.15.5:0/393760517 wait complete. 2024-05-09T00:56:16.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.054+0000 7fd50deff700 1 Processor -- start 2024-05-09T00:56:16.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.054+0000 7fd50deff700 1 -- start start 2024-05-09T00:56:16.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.055+0000 7fd50deff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd50810b2b0 0x7fd5081a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:16.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.055+0000 7fd50deff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd508110990 0x7fd5081a1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:16.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.055+0000 7fd50deff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5081a5e50 0x7fd5081a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:16.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.055+0000 7fd50deff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd50811abb0 con 0x7fd508110990 2024-05-09T00:56:16.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.055+0000 7fd50deff700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd50811aa30 con 0x7fd5081a5e50 2024-05-09T00:56:16.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.055+0000 7fd50deff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd50811ad30 con 0x7fd50810b2b0 2024-05-09T00:56:16.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.060+0000 7fd507fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5081a5e50 0x7fd5081a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:16.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.060+0000 7fd506ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd508110990 0x7fd5081a1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:16.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.060+0000 7fd506ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd508110990 0x7fd5081a1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34186/0 (socket says 172.21.15.5:34186) 2024-05-09T00:56:16.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.060+0000 7fd506ffd700 1 -- 172.21.15.5:0/1678979870 learned_addr learned my addr 172.21.15.5:0/1678979870 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:56:16.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.063+0000 7fd5077fe700 1 --2- 172.21.15.5:0/1678979870 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd50810b2b0 0x7fd5081a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:16.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.063+0000 7fd506ffd700 1 -- 172.21.15.5:0/1678979870 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd50810b2b0 msgr2=0x7fd5081a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:56:16.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.063+0000 7fd506ffd700 1 --2- 172.21.15.5:0/1678979870 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd50810b2b0 0x7fd5081a16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:16.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.063+0000 7fd506ffd700 1 -- 172.21.15.5:0/1678979870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5081a5e50 msgr2=0x7fd5081a6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:56:16.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.063+0000 7fd506ffd700 1 --2- 172.21.15.5:0/1678979870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5081a5e50 0x7fd5081a6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:16.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.063+0000 7fd506ffd700 1 -- 172.21.15.5:0/1678979870 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5081a69a0 con 0x7fd508110990 2024-05-09T00:56:16.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.064+0000 7fd507fff700 1 --2- 172.21.15.5:0/1678979870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5081a5e50 0x7fd5081a6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:56:16.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.065+0000 7fd506ffd700 1 --2- 172.21.15.5:0/1678979870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd508110990 0x7fd5081a1c80 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fd50000b830 tx=0x7fd500007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:56:16.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.069+0000 7fd504ff9700 1 -- 172.21.15.5:0/1678979870 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd500021030 con 0x7fd508110990 2024-05-09T00:56:16.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.069+0000 7fd504ff9700 1 -- 172.21.15.5:0/1678979870 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd500004e50 con 0x7fd508110990 2024-05-09T00:56:16.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.071+0000 7fd504ff9700 1 -- 172.21.15.5:0/1678979870 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5000048a0 con 0x7fd508110990 2024-05-09T00:56:16.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.071+0000 7fd50deff700 1 -- 172.21.15.5:0/1678979870 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5081a6c30 con 0x7fd508110990 2024-05-09T00:56:16.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.071+0000 7fd50deff700 1 -- 172.21.15.5:0/1678979870 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5081ae330 con 0x7fd508110990 2024-05-09T00:56:16.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.080+0000 7fd51c915b80 1 Processor -- start 2024-05-09T00:56:16.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.080+0000 7fd51c915b80 1 -- start start 2024-05-09T00:56:16.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.081+0000 7fd51c915b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559aee7d6d00 0x559aee8df240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:16.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.081+0000 7fd51c915b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559aee75c4a0 0x559aee8df780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:16.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.081+0000 7fd51c915b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559aee8d9400 0x559aee8d9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:16.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.082+0000 7fd51c915b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559aee7fc0a0 con 0x559aee75c4a0 2024-05-09T00:56:16.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.082+0000 7fd51c915b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x559aee7a19a0 con 0x559aee7d6d00 2024-05-09T00:56:16.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.082+0000 7fd51c915b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x559aee7f8060 con 0x559aee8d9400 2024-05-09T00:56:16.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.083+0000 7fd507fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559aee7d6d00 0x559aee8df240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:16.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.084+0000 7fd507fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559aee7d6d00 0x559aee8df240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:55038/0 (socket says 172.21.15.5:55038) 2024-05-09T00:56:16.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.084+0000 7fd507fff700 1 -- 172.21.15.5:0/1776200080 learned_addr learned my addr 172.21.15.5:0/1776200080 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:56:16.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.085+0000 7fd507fff700 1 -- 172.21.15.5:0/1776200080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559aee8d9400 msgr2=0x559aee8d9810 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:56:16.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.085+0000 7fd507fff700 1 --2- 172.21.15.5:0/1776200080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559aee8d9400 0x559aee8d9810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:16.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.085+0000 7fd507fff700 1 -- 172.21.15.5:0/1776200080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559aee75c4a0 msgr2=0x559aee8df780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:56:16.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.085+0000 7fd507fff700 1 --2- 172.21.15.5:0/1776200080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559aee75c4a0 0x559aee8df780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:16.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.085+0000 7fd507fff700 1 -- 172.21.15.5:0/1776200080 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559aee792500 con 0x559aee7d6d00 2024-05-09T00:56:16.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.086+0000 7fd507fff700 1 --2- 172.21.15.5:0/1776200080 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559aee7d6d00 0x559aee8df240 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd4f800bdb0 tx=0x7fd4f800f640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba8e62bc49b1cc8a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:56:16.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.086+0000 7fd4edffb700 1 -- 172.21.15.5:0/1776200080 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f8015070 con 0x559aee7d6d00 2024-05-09T00:56:16.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.086+0000 7fd4edffb700 1 -- 172.21.15.5:0/1776200080 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4f800fe50 con 0x559aee7d6d00 2024-05-09T00:56:16.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.088+0000 7fd51c915b80 1 -- 172.21.15.5:0/1776200080 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559aee7d6d00 msgr2=0x559aee8df240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:56:16.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.088+0000 7fd51c915b80 1 --2- 172.21.15.5:0/1776200080 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559aee7d6d00 0x559aee8df240 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd4f800bdb0 tx=0x7fd4f800f640 comp rx=0 tx=0).stop 2024-05-09T00:56:16.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.089+0000 7fd51c915b80 1 -- 172.21.15.5:0/1776200080 shutdown_connections 2024-05-09T00:56:16.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.089+0000 7fd51c915b80 1 --2- 172.21.15.5:0/1776200080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559aee8d9400 0x559aee8d9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:16.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.089+0000 7fd51c915b80 1 --2- 172.21.15.5:0/1776200080 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559aee7d6d00 0x559aee8df240 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:16.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.089+0000 7fd51c915b80 1 --2- 172.21.15.5:0/1776200080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559aee75c4a0 0x559aee8df780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:16.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.089+0000 7fd51c915b80 1 -- 172.21.15.5:0/1776200080 >> 172.21.15.5:0/1776200080 conn(0x559aee75ff70 msgr2=0x559aee8d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:56:16.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.090+0000 7fd504ff9700 1 -- 172.21.15.5:0/1678979870 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fd500024070 con 0x7fd508110990 2024-05-09T00:56:16.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.092+0000 7fd504ff9700 1 --2- 172.21.15.5:0/1678979870 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fd4f00b2a00 0x7fd4f00b4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:16.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.093+0000 7fd504ff9700 1 -- 172.21.15.5:0/1678979870 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fd5000e2470 con 0x7fd508110990 2024-05-09T00:56:16.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.096+0000 7fd51c915b80 1 -- 172.21.15.5:0/1776200080 shutdown_connections 2024-05-09T00:56:16.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.096+0000 7fd51c915b80 1 -- 172.21.15.5:0/1776200080 wait complete. 2024-05-09T00:56:16.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.100+0000 7fd51c915b80 1 -- start start 2024-05-09T00:56:16.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.100+0000 7fd51c915b80 1 Processor -- start 2024-05-09T00:56:16.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.102+0000 7fd51c915b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559aee75c4a0 0x559aee8e2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:16.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.102+0000 7fd51c915b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559aee7d6d00 0x559aee8e3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:16.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.102+0000 7fd51c915b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559aee8d9400 0x559aee8e3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:16.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.102+0000 7fd51c915b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559aee8da200 con 0x559aee8d9400 2024-05-09T00:56:16.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.102+0000 7fd51c915b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x559aee796b20 con 0x559aee75c4a0 2024-05-09T00:56:16.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.102+0000 7fd51c915b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x559aee8e6020 con 0x559aee7d6d00 2024-05-09T00:56:16.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.102+0000 7fd51c915b80 10 client.0 _ll_register_callbacks cb 0x559aee66d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:56:16.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:56:15 smithi131 ceph-mon[34547]: pgmap v668: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 908 MiB/s wr, 1.95k op/s 2024-05-09T00:56:16.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.104+0000 7fd507fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559aee8d9400 0x559aee8e3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:16.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.104+0000 7fd507fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559aee8d9400 0x559aee8e3a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34206/0 (socket says 172.21.15.5:34206) 2024-05-09T00:56:16.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.104+0000 7fd507fff700 1 -- 172.21.15.5:0/997351169 learned_addr learned my addr 172.21.15.5:0/997351169 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:56:16.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.106+0000 7fd5077fe700 1 --2- 172.21.15.5:0/1678979870 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fd4f00b2a00 0x7fd4f00b4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:16.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.106+0000 7fd506ffd700 1 --2- 172.21.15.5:0/997351169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559aee7d6d00 0x559aee8e3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:16.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.106+0000 7fd5077fe700 1 --2- 172.21.15.5:0/997351169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559aee75c4a0 0x559aee8e2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:16.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.107+0000 7fd507fff700 1 -- 172.21.15.5:0/997351169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559aee7d6d00 msgr2=0x559aee8e3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:56:16.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.107+0000 7fd507fff700 1 --2- 172.21.15.5:0/997351169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559aee7d6d00 0x559aee8e3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:16.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.107+0000 7fd507fff700 1 -- 172.21.15.5:0/997351169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559aee75c4a0 msgr2=0x559aee8e2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:56:16.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.107+0000 7fd507fff700 1 --2- 172.21.15.5:0/997351169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559aee75c4a0 0x559aee8e2ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:16.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.107+0000 7fd507fff700 1 -- 172.21.15.5:0/997351169 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559aee677490 con 0x559aee8d9400 2024-05-09T00:56:16.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.109+0000 7fd507fff700 1 --2- 172.21.15.5:0/997351169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559aee8d9400 0x559aee8e3a70 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fd4f8005910 tx=0x7fd4f8011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6b136829e21e585 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:56:16.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.110+0000 7fd4e6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:56:16.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.111+0000 7fd4e6ffd700 1 -- 172.21.15.5:0/997351169 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f8015070 con 0x559aee8d9400 2024-05-09T00:56:16.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.111+0000 7fd51c915b80 1 -- 172.21.15.5:0/997351169 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4f800fe50 con 0x559aee8d9400 2024-05-09T00:56:16.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.111+0000 7fd4e6ffd700 1 -- 172.21.15.5:0/997351169 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fd4f8011d60 con 0x559aee8d9400 2024-05-09T00:56:16.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.112+0000 7fd4e6ffd700 1 -- 172.21.15.5:0/997351169 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fd4f802a030 con 0x559aee8d9400 2024-05-09T00:56:16.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.118+0000 7fd51c915b80 10 client.16491 fetch_fsmap learned FSMap version 31 2024-05-09T00:56:16.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.118+0000 7fd51c915b80 1 -- 172.21.15.5:0/997351169 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559aee901af0 con 0x559aee8d9400 2024-05-09T00:56:16.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.120+0000 7fd5077fe700 1 --2- 172.21.15.5:0/1678979870 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fd4f00b2a00 0x7fd4f00b4f10 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x559aee8e2870 tx=0x7fd4fc00a110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:56:16.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.122+0000 7fd4e6ffd700 1 -- 172.21.15.5:0/997351169 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4f800fe50 con 0x559aee8d9400 2024-05-09T00:56:16.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.122+0000 7fd4e6ffd700 1 -- 172.21.15.5:0/997351169 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fd4f80047e0 con 0x559aee8d9400 2024-05-09T00:56:16.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.125+0000 7fd51c915b80 10 client.16491 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:56:16.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.125+0000 7fd51c915b80 10 client.16491 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:56:16.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.125+0000 7fd51c915b80 20 client.16491 populate_metadata read hostname 'smithi005' 2024-05-09T00:56:16.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.125+0000 7fd51c915b80 1 --2- 172.21.15.5:0/997351169 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x559aee9040d0 0x559aee9064f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:16.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.125+0000 7fd51c915b80 4 client.16491 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:56:16.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.125+0000 7fd51c915b80 1 -- 172.21.15.5:0/997351169 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559aee809a80 con 0x559aee9040d0 2024-05-09T00:56:16.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.127+0000 7fd5077fe700 1 --2- 172.21.15.5:0/997351169 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x559aee9040d0 0x559aee9064f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:16.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.139+0000 7fd4e6ffd700 10 client.16491 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:56:16.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.139+0000 7fd5077fe700 1 --2- 172.21.15.5:0/997351169 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x559aee9040d0 0x559aee9064f0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ea9412e594a00d2 server_cookie=962a416c07b392cb in_seq=0 out_seq=0 2024-05-09T00:56:16.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.143+0000 7fd4e6ffd700 1 -- 172.21.15.5:0/997351169 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559aee809a80 con 0x559aee9040d0 2024-05-09T00:56:16.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.143+0000 7fd4e6ffd700 10 client.16491 handle_command_reply: tid=0 2024-05-09T00:56:16.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.146+0000 7fd51c915b80 1 client.16491 shutdown 2024-05-09T00:56:16.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.146+0000 7fd51c915b80 1 -- 172.21.15.5:0/997351169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559aee8d9400 msgr2=0x559aee8e3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:56:16.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.146+0000 7fd51c915b80 1 --2- 172.21.15.5:0/997351169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559aee8d9400 0x559aee8e3a70 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fd4f8005910 tx=0x7fd4f8011900 comp rx=0 tx=0).stop 2024-05-09T00:56:16.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.148+0000 7fd51c915b80 1 -- 172.21.15.5:0/997351169 shutdown_connections 2024-05-09T00:56:16.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.148+0000 7fd51c915b80 1 --2- 172.21.15.5:0/997351169 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x559aee9040d0 0x559aee9064f0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:16.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.148+0000 7fd51c915b80 1 --2- 172.21.15.5:0/997351169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559aee8d9400 0x559aee8e3a70 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:16.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.148+0000 7fd51c915b80 1 --2- 172.21.15.5:0/997351169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559aee75c4a0 0x559aee8e2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:16.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.148+0000 7fd51c915b80 1 --2- 172.21.15.5:0/997351169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559aee7d6d00 0x559aee8e3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:16.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.148+0000 7fd51c915b80 1 -- 172.21.15.5:0/997351169 >> 172.21.15.5:0/997351169 conn(0x559aee75ff70 msgr2=0x559aee8cdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:56:16.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.148+0000 7fd4e6ffd700 0 client.16491 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:56:16.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.150+0000 7fd51c915b80 1 -- 172.21.15.5:0/997351169 shutdown_connections 2024-05-09T00:56:16.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.152+0000 7fd51c915b80 1 -- 172.21.15.5:0/997351169 wait complete. 2024-05-09T00:56:16.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.152+0000 7fd51c915b80 20 client.16491 trim_cache size 0 max 16384 2024-05-09T00:56:16.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:56:15 smithi157 ceph-mon[34294]: pgmap v668: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 908 MiB/s wr, 1.95k op/s 2024-05-09T00:56:16.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.194+0000 7fd51c915b80 1 Processor -- start 2024-05-09T00:56:16.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.195+0000 7fd51c915b80 1 -- start start 2024-05-09T00:56:16.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.195+0000 7fd51c915b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559aee75c4a0 0x559aee8cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:16.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.196+0000 7fd51c915b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559aee7d6d00 0x559aee909510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:16.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.196+0000 7fd51c915b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559aee8d9400 0x559aee909a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:16.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.196+0000 7fd51c915b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd5080830b0 con 0x559aee7d6d00 2024-05-09T00:56:16.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.196+0000 7fd51c915b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd5081aea30 con 0x559aee8d9400 2024-05-09T00:56:16.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.196+0000 7fd51c915b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd508083780 con 0x559aee75c4a0 2024-05-09T00:56:16.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.196+0000 7fd506ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559aee7d6d00 0x559aee909510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:16.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.197+0000 7fd506ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559aee7d6d00 0x559aee909510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34210/0 (socket says 172.21.15.5:34210) 2024-05-09T00:56:16.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.197+0000 7fd506ffd700 1 -- 172.21.15.5:0/1322444969 learned_addr learned my addr 172.21.15.5:0/1322444969 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:56:16.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.197+0000 7fd507fff700 1 --2- 172.21.15.5:0/1322444969 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559aee8d9400 0x559aee909a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:16.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.197+0000 7fd5077fe700 1 --2- 172.21.15.5:0/1322444969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559aee75c4a0 0x559aee8cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:16.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.197+0000 7fd506ffd700 1 -- 172.21.15.5:0/1322444969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559aee75c4a0 msgr2=0x559aee8cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:56:16.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.197+0000 7fd506ffd700 1 --2- 172.21.15.5:0/1322444969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559aee75c4a0 0x559aee8cea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:16.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.197+0000 7fd506ffd700 1 -- 172.21.15.5:0/1322444969 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559aee8d9400 msgr2=0x559aee909a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:56:16.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.197+0000 7fd506ffd700 1 --2- 172.21.15.5:0/1322444969 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559aee8d9400 0x559aee909a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:16.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.197+0000 7fd506ffd700 1 -- 172.21.15.5:0/1322444969 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559aee8d09d0 con 0x559aee7d6d00 2024-05-09T00:56:16.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.198+0000 7fd506ffd700 1 --2- 172.21.15.5:0/1322444969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559aee7d6d00 0x559aee909510 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fd5000a5ba0 tx=0x7fd5000a99c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e6d26e006743a63 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:56:16.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.199+0000 7fd4ed7fa700 1 -- 172.21.15.5:0/1322444969 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5000b2230 con 0x559aee7d6d00 2024-05-09T00:56:16.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.199+0000 7fd4ed7fa700 1 -- 172.21.15.5:0/1322444969 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5000aae50 con 0x559aee7d6d00 2024-05-09T00:56:16.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.199+0000 7fd4ed7fa700 1 -- 172.21.15.5:0/1322444969 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5000b28f0 con 0x559aee7d6d00 2024-05-09T00:56:16.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.200+0000 7fd51c915b80 1 -- 172.21.15.5:0/1322444969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559aee7d6d00 msgr2=0x559aee909510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:56:16.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.201+0000 7fd51c915b80 1 --2- 172.21.15.5:0/1322444969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559aee7d6d00 0x559aee909510 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fd5000a5ba0 tx=0x7fd5000a99c0 comp rx=0 tx=0).stop 2024-05-09T00:56:16.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.201+0000 7fd51c915b80 1 -- 172.21.15.5:0/1322444969 shutdown_connections 2024-05-09T00:56:16.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.201+0000 7fd51c915b80 1 --2- 172.21.15.5:0/1322444969 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559aee8d9400 0x559aee909a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:16.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.201+0000 7fd51c915b80 1 --2- 172.21.15.5:0/1322444969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559aee75c4a0 0x559aee8cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:16.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.201+0000 7fd51c915b80 1 --2- 172.21.15.5:0/1322444969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559aee7d6d00 0x559aee909510 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:16.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.201+0000 7fd51c915b80 1 -- 172.21.15.5:0/1322444969 >> 172.21.15.5:0/1322444969 conn(0x559aee75ff70 msgr2=0x559aee8d3cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:56:16.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.201+0000 7fd51c915b80 1 -- 172.21.15.5:0/1322444969 shutdown_connections 2024-05-09T00:56:16.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.201+0000 7fd51c915b80 1 -- 172.21.15.5:0/1322444969 wait complete. 2024-05-09T00:56:16.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.202+0000 7fd51c915b80 1 -- start start 2024-05-09T00:56:16.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.203+0000 7fd51c915b80 1 Processor -- start 2024-05-09T00:56:16.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.204+0000 7fd51c915b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559aee75c4a0 0x559aee907ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:16.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.204+0000 7fd51c915b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559aee7d6d00 0x559aee908010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:16.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.204+0000 7fd51c915b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559aee8d9400 0x559aee908550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:16.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.204+0000 7fd51c915b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd508084520 con 0x559aee8d9400 2024-05-09T00:56:16.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.204+0000 7fd51c915b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd508083e50 con 0x559aee75c4a0 2024-05-09T00:56:16.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.204+0000 7fd51c915b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd5080810b0 con 0x559aee7d6d00 2024-05-09T00:56:16.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.204+0000 7fd51c915b80 10 client.0 _ll_register_callbacks cb 0x559aee8ded60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:56:16.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.206+0000 7fd506ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559aee7d6d00 0x559aee908010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:16.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.206+0000 7fd507fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559aee8d9400 0x559aee908550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:16.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.206+0000 7fd507fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559aee8d9400 0x559aee908550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34226/0 (socket says 172.21.15.5:34226) 2024-05-09T00:56:16.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.206+0000 7fd507fff700 1 -- 172.21.15.5:0/757661870 learned_addr learned my addr 172.21.15.5:0/757661870 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:56:16.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.207+0000 7fd5077fe700 1 --2- 172.21.15.5:0/757661870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559aee75c4a0 0x559aee907ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:16.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.208+0000 7fd507fff700 1 -- 172.21.15.5:0/757661870 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559aee7d6d00 msgr2=0x559aee908010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:56:16.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.208+0000 7fd507fff700 1 --2- 172.21.15.5:0/757661870 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559aee7d6d00 0x559aee908010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:16.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.208+0000 7fd507fff700 1 -- 172.21.15.5:0/757661870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559aee75c4a0 msgr2=0x559aee907ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:56:16.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.208+0000 7fd507fff700 1 --2- 172.21.15.5:0/757661870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559aee75c4a0 0x559aee907ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:16.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.208+0000 7fd507fff700 1 -- 172.21.15.5:0/757661870 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559aee96d670 con 0x559aee8d9400 2024-05-09T00:56:16.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.209+0000 7fd507fff700 1 --2- 172.21.15.5:0/757661870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559aee8d9400 0x559aee908550 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fd4f8011a20 tx=0x7fd4f8005000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e79064add345a250 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:56:16.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.209+0000 7fd4e6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:56:16.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.209+0000 7fd4e6ffd700 1 -- 172.21.15.5:0/757661870 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f8015040 con 0x559aee8d9400 2024-05-09T00:56:16.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.209+0000 7fd4e6ffd700 1 -- 172.21.15.5:0/757661870 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fd4f802b070 con 0x559aee8d9400 2024-05-09T00:56:16.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.209+0000 7fd51c915b80 1 -- 172.21.15.5:0/757661870 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5000aae50 con 0x559aee8d9400 2024-05-09T00:56:16.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.210+0000 7fd4e6ffd700 1 -- 172.21.15.5:0/757661870 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fd4f803d030 con 0x559aee8d9400 2024-05-09T00:56:16.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.210+0000 7fd51c915b80 10 client.16497 fetch_fsmap learned FSMap version 31 2024-05-09T00:56:16.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.210+0000 7fd51c915b80 1 -- 172.21.15.5:0/757661870 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559aee96e8e0 con 0x559aee8d9400 2024-05-09T00:56:16.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.210+0000 7fd4e6ffd700 1 -- 172.21.15.5:0/757661870 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4f8005220 con 0x559aee8d9400 2024-05-09T00:56:16.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.211+0000 7fd4e6ffd700 1 -- 172.21.15.5:0/757661870 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fd4f802fc90 con 0x559aee8d9400 2024-05-09T00:56:16.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.211+0000 7fd51c915b80 10 client.16497 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:56:16.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.211+0000 7fd51c915b80 10 client.16497 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:56:16.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.211+0000 7fd51c915b80 20 client.16497 populate_metadata read hostname 'smithi005' 2024-05-09T00:56:16.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.211+0000 7fd51c915b80 1 --2- 172.21.15.5:0/757661870 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x559aee96ef60 0x559aee971380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:16.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.211+0000 7fd51c915b80 4 client.16497 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:56:16.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.211+0000 7fd51c915b80 1 -- 172.21.15.5:0/757661870 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fd5000aae50 con 0x559aee96ef60 2024-05-09T00:56:16.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.212+0000 7fd5077fe700 1 --2- 172.21.15.5:0/757661870 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x559aee96ef60 0x559aee971380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:16.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.213+0000 7fd4e6ffd700 10 client.16497 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:56:16.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.213+0000 7fd5077fe700 1 --2- 172.21.15.5:0/757661870 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x559aee96ef60 0x559aee971380 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bcf82e559f8fdac server_cookie=f5f40dc17b99ccdb in_seq=0 out_seq=0 2024-05-09T00:56:16.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.217+0000 7fd4e6ffd700 1 -- 172.21.15.5:0/757661870 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7fd5000aae50 con 0x559aee96ef60 2024-05-09T00:56:16.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.217+0000 7fd4e6ffd700 10 client.16497 handle_command_reply: tid=0 2024-05-09T00:56:16.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.217+0000 7fd51c915b80 1 client.16497 shutdown 2024-05-09T00:56:16.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.217+0000 7fd51c915b80 1 -- 172.21.15.5:0/757661870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559aee8d9400 msgr2=0x559aee908550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:56:16.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.217+0000 7fd51c915b80 1 --2- 172.21.15.5:0/757661870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559aee8d9400 0x559aee908550 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fd4f8011a20 tx=0x7fd4f8005000 comp rx=0 tx=0).stop 2024-05-09T00:56:16.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.218+0000 7fd51c915b80 1 -- 172.21.15.5:0/757661870 shutdown_connections 2024-05-09T00:56:16.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.218+0000 7fd51c915b80 1 --2- 172.21.15.5:0/757661870 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x559aee96ef60 0x559aee971380 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:16.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.218+0000 7fd51c915b80 1 --2- 172.21.15.5:0/757661870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559aee8d9400 0x559aee908550 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:16.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.218+0000 7fd4e6ffd700 0 client.16497 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:56:16.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.218+0000 7fd51c915b80 1 --2- 172.21.15.5:0/757661870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559aee75c4a0 0x559aee907ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:16.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.218+0000 7fd51c915b80 1 --2- 172.21.15.5:0/757661870 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559aee7d6d00 0x559aee908010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:16.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.218+0000 7fd51c915b80 1 -- 172.21.15.5:0/757661870 >> 172.21.15.5:0/757661870 conn(0x559aee75ff70 msgr2=0x559aee8dccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:56:16.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.218+0000 7fd51c915b80 1 -- 172.21.15.5:0/757661870 shutdown_connections 2024-05-09T00:56:16.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.218+0000 7fd51c915b80 1 -- 172.21.15.5:0/757661870 wait complete. 2024-05-09T00:56:16.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.219+0000 7fd51c915b80 20 client.16497 trim_cache size 0 max 16384 2024-05-09T00:56:16.221 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-09T00:56:16.221 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-05-09T00:56:16.221 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-05-09T00:56:16.221 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-09T00:56:16.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.220+0000 7fd50deff700 1 -- 172.21.15.5:0/1678979870 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fd4f00b2a00 msgr2=0x7fd4f00b4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:56:16.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.220+0000 7fd50deff700 1 --2- 172.21.15.5:0/1678979870 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fd4f00b2a00 0x7fd4f00b4f10 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x559aee8e2870 tx=0x7fd4fc00a110 comp rx=0 tx=0).stop 2024-05-09T00:56:16.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.220+0000 7fd50deff700 1 -- 172.21.15.5:0/1678979870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd508110990 msgr2=0x7fd5081a1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:56:16.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.220+0000 7fd50deff700 1 --2- 172.21.15.5:0/1678979870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd508110990 0x7fd5081a1c80 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fd50000b830 tx=0x7fd500007a50 comp rx=0 tx=0).stop 2024-05-09T00:56:16.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.220+0000 7fd50deff700 1 -- 172.21.15.5:0/1678979870 shutdown_connections 2024-05-09T00:56:16.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.220+0000 7fd50deff700 1 --2- 172.21.15.5:0/1678979870 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fd4f00b2a00 0x7fd4f00b4f10 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:16.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.220+0000 7fd50deff700 1 --2- 172.21.15.5:0/1678979870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5081a5e50 0x7fd5081a6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:16.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.220+0000 7fd50deff700 1 --2- 172.21.15.5:0/1678979870 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd50810b2b0 0x7fd5081a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:16.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.220+0000 7fd50deff700 1 --2- 172.21.15.5:0/1678979870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd508110990 0x7fd5081a1c80 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:16.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.220+0000 7fd50deff700 1 -- 172.21.15.5:0/1678979870 >> 172.21.15.5:0/1678979870 conn(0x7fd508076d30 msgr2=0x7fd508108940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:56:16.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.220+0000 7fd50deff700 1 -- 172.21.15.5:0/1678979870 shutdown_connections 2024-05-09T00:56:16.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:16.220+0000 7fd50deff700 1 -- 172.21.15.5:0/1678979870 wait complete. 2024-05-09T00:56:17.713 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:56:17 smithi005 ceph-mon[30365]: pgmap v669: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 923 MiB/s wr, 1.93k op/s 2024-05-09T00:56:17.714 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:56:17 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:56:18.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:56:17 smithi131 ceph-mon[34547]: pgmap v669: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 923 MiB/s wr, 1.93k op/s 2024-05-09T00:56:18.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:56:17 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:56:18.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:56:17 smithi157 ceph-mon[34294]: pgmap v669: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 923 MiB/s wr, 1.93k op/s 2024-05-09T00:56:18.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:56:17 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:56:18.729 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T00:56:18.729 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T00:56:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:19.879+0000 7f77c57fa640 1 -- 172.21.15.5:0/3140216377 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a1b33b3240 con 0x55a1b34c6600 2024-05-09T00:56:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:19.879+0000 7f77c57fa640 10 client.25999 handle_command_reply: tid=0 2024-05-09T00:56:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:19.879+0000 7f77eac66740 1 client.25999 shutdown 2024-05-09T00:56:19.882 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:19.879+0000 7f77eac66740 1 -- 172.21.15.5:0/3140216377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a1b30a31b0 msgr2=0x55a1b3456080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:56:19.882 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:19.879+0000 7f77eac66740 1 --2- 172.21.15.5:0/3140216377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a1b30a31b0 0x55a1b3456080 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f77d40dfd30 tx=0x7f77d40dfc00 comp rx=0 tx=0).stop 2024-05-09T00:56:19.882 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:19.879+0000 7f77eac66740 1 -- 172.21.15.5:0/3140216377 shutdown_connections 2024-05-09T00:56:19.882 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:19.879+0000 7f77eac66740 1 --2- 172.21.15.5:0/3140216377 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55a1b34c6600 0x55a1b34c8a40 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:19.882 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:19.880+0000 7f77eac66740 1 --2- 172.21.15.5:0/3140216377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a1b30a31b0 0x55a1b3456080 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:19.882 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:19.880+0000 7f77eac66740 1 --2- 172.21.15.5:0/3140216377 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a1b307a2f0 0x55a1b3455b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:19.882 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:19.880+0000 7f77c57fa640 0 client.25999 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:56:19.882 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:19.880+0000 7f77eac66740 1 --2- 172.21.15.5:0/3140216377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1b30371c0 0x55a1b3458f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:19.882 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:19.880+0000 7f77eac66740 1 -- 172.21.15.5:0/3140216377 >> 172.21.15.5:0/3140216377 conn(0x55a1b3097300 msgr2=0x55a1b344a750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:56:19.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:19.880+0000 7f77eac66740 1 -- 172.21.15.5:0/3140216377 shutdown_connections 2024-05-09T00:56:19.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:19.880+0000 7f77eac66740 1 -- 172.21.15.5:0/3140216377 wait complete. 2024-05-09T00:56:19.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-09T00:56:19.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-09T00:56:19.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-09T00:56:19.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:19.881+0000 7f77eac66740 20 client.25999 trim_cache size 0 max 16384 2024-05-09T00:56:19.887 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:19.883+0000 7f77e3db2640 1 -- 172.21.15.5:0/2775175531 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f77c80b2930 msgr2=0x7f77c80b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:56:19.887 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:19.883+0000 7f77e3db2640 1 --2- 172.21.15.5:0/2775175531 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f77c80b2930 0x7f77c80b4e40 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f77d80057c0 tx=0x7f77d80305f0 comp rx=0 tx=0).stop 2024-05-09T00:56:19.887 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:19.883+0000 7f77e3db2640 1 -- 172.21.15.5:0/2775175531 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77dc1101a0 msgr2=0x7f77dc118680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:56:19.887 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:19.883+0000 7f77e3db2640 1 --2- 172.21.15.5:0/2775175531 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77dc1101a0 0x7f77dc118680 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f77d4009fd0 tx=0x7f77d400c800 comp rx=0 tx=0).stop 2024-05-09T00:56:19.887 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:19.885+0000 7f77e3db2640 1 -- 172.21.15.5:0/2775175531 shutdown_connections 2024-05-09T00:56:19.888 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:19.885+0000 7f77e3db2640 1 --2- 172.21.15.5:0/2775175531 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f77c80b2930 0x7f77c80b4e40 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:19.888 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:19.885+0000 7f77e3db2640 1 --2- 172.21.15.5:0/2775175531 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77dc1101a0 0x7f77dc118680 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:19.888 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:19.885+0000 7f77e3db2640 1 --2- 172.21.15.5:0/2775175531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77dc10b480 0x7f77dc117720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:19.888 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:19.885+0000 7f77e3db2640 1 --2- 172.21.15.5:0/2775175531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77dc10a550 0x7f77dc1171e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:19.888 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:19.885+0000 7f77e3db2640 1 -- 172.21.15.5:0/2775175531 >> 172.21.15.5:0/2775175531 conn(0x7f77dc070670 msgr2=0x7f77dc10c180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:56:19.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:19.887+0000 7f77e3db2640 1 -- 172.21.15.5:0/2775175531 shutdown_connections 2024-05-09T00:56:19.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:19.888+0000 7f77e3db2640 1 -- 172.21.15.5:0/2775175531 wait complete. 2024-05-09T00:56:20.043 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:56:19 smithi005 ceph-mon[30365]: pgmap v670: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 853 MiB/s wr, 1.71k op/s 2024-05-09T00:56:20.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:56:19 smithi131 ceph-mon[34547]: pgmap v670: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 853 MiB/s wr, 1.71k op/s 2024-05-09T00:56:20.158 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:56:19 smithi157 ceph-mon[34294]: pgmap v670: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 853 MiB/s wr, 1.71k op/s 2024-05-09T00:56:21.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:56:21 smithi157 ceph-mon[34294]: pgmap v671: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1018 MiB/s wr, 1.93k op/s 2024-05-09T00:56:21.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:56:21 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:56:21.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:56:21 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:56:21.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:56:21 smithi005 ceph-mon[30365]: pgmap v671: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1018 MiB/s wr, 1.93k op/s 2024-05-09T00:56:21.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:56:21 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:56:21.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:56:21 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:56:21.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:56:21 smithi131 ceph-mon[34547]: pgmap v671: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1018 MiB/s wr, 1.93k op/s 2024-05-09T00:56:21.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:56:21 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:56:21.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:56:21 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:56:22.373 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:56:22.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:56:22 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:56:22.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:56:22 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:56:23.020 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:56:22 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:56:23.021 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:56:22 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:56:23.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:56:22 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:56:23.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:56:22 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:56:23.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:56:23 smithi157 ceph-mon[34294]: pgmap v672: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 758 MiB/s wr, 1.50k op/s 2024-05-09T00:56:24.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:56:23 smithi005 ceph-mon[30365]: pgmap v672: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 758 MiB/s wr, 1.50k op/s 2024-05-09T00:56:24.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:56:23 smithi131 ceph-mon[34547]: pgmap v672: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 758 MiB/s wr, 1.50k op/s 2024-05-09T00:56:25.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:56:25 smithi157 ceph-mon[34294]: pgmap v673: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 889 MiB/s wr, 1.86k op/s 2024-05-09T00:56:26.043 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:56:25 smithi005 ceph-mon[30365]: pgmap v673: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 889 MiB/s wr, 1.86k op/s 2024-05-09T00:56:26.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:56:25 smithi131 ceph-mon[34547]: pgmap v673: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 889 MiB/s wr, 1.86k op/s 2024-05-09T00:56:27.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:27.495+0000 7ff208d8f700 1 -- 172.21.15.5:0/2621950209 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff204106570 msgr2=0x7ff204106980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:56:27.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:27.495+0000 7ff208d8f700 1 --2- 172.21.15.5:0/2621950209 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff204106570 0x7ff204106980 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff1f8009990 tx=0x7ff1f8009ca0 comp rx=0 tx=0).stop 2024-05-09T00:56:27.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:27.495+0000 7ff208d8f700 1 -- 172.21.15.5:0/2621950209 shutdown_connections 2024-05-09T00:56:27.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:27.495+0000 7ff208d8f700 1 --2- 172.21.15.5:0/2621950209 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff204106ec0 0x7ff204113350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:27.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:27.495+0000 7ff208d8f700 1 --2- 172.21.15.5:0/2621950209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff20410c5b0 0x7ff20410c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:27.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:27.495+0000 7ff208d8f700 1 --2- 172.21.15.5:0/2621950209 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff204106570 0x7ff204106980 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:27.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:27.495+0000 7ff208d8f700 1 -- 172.21.15.5:0/2621950209 >> 172.21.15.5:0/2621950209 conn(0x7ff204102050 msgr2=0x7ff2041044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:56:27.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:27.497+0000 7ff208d8f700 1 -- 172.21.15.5:0/2621950209 shutdown_connections 2024-05-09T00:56:27.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:27.497+0000 7ff208d8f700 1 -- 172.21.15.5:0/2621950209 wait complete. 2024-05-09T00:56:27.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:27.499+0000 7ff208d8f700 1 Processor -- start 2024-05-09T00:56:27.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:27.499+0000 7ff208d8f700 1 -- start start 2024-05-09T00:56:27.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:27.500+0000 7ff208d8f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff204106570 0x7ff204112090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:27.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:27.500+0000 7ff208d8f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff204106ec0 0x7ff204112630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:27.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:27.500+0000 7ff208d8f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff20410c5b0 0x7ff2041a5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:27.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:27.501+0000 7ff208d8f700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff204078cb0 con 0x7ff20410c5b0 2024-05-09T00:56:27.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:27.501+0000 7ff208d8f700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff204078b30 con 0x7ff204106570 2024-05-09T00:56:27.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:27.501+0000 7ff208d8f700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff204078e30 con 0x7ff204106ec0 2024-05-09T00:56:27.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:27.501+0000 7ff201d9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff204106ec0 0x7ff204112630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:27.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:27.501+0000 7ff201d9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff204106ec0 0x7ff204112630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:57308/0 (socket says 172.21.15.5:57308) 2024-05-09T00:56:27.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:27.501+0000 7ff201d9b700 1 -- 172.21.15.5:0/775444658 learned_addr learned my addr 172.21.15.5:0/775444658 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:56:27.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:27.502+0000 7ff202d9d700 1 --2- 172.21.15.5:0/775444658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff20410c5b0 0x7ff2041a5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:27.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:27.502+0000 7ff20259c700 1 --2- 172.21.15.5:0/775444658 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff204106570 0x7ff204112090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:27.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:27.503+0000 7ff202d9d700 1 -- 172.21.15.5:0/775444658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff204106ec0 msgr2=0x7ff204112630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:56:27.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:27.503+0000 7ff202d9d700 1 --2- 172.21.15.5:0/775444658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff204106ec0 0x7ff204112630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:27.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:27.503+0000 7ff202d9d700 1 -- 172.21.15.5:0/775444658 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff204106570 msgr2=0x7ff204112090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:56:27.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:27.503+0000 7ff202d9d700 1 --2- 172.21.15.5:0/775444658 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff204106570 0x7ff204112090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:27.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:27.503+0000 7ff202d9d700 1 -- 172.21.15.5:0/775444658 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2041a62f0 con 0x7ff20410c5b0 2024-05-09T00:56:27.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:27.503+0000 7ff202d9d700 1 --2- 172.21.15.5:0/775444658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff20410c5b0 0x7ff2041a5c50 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc00efc0 tx=0x7ff1fc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:56:27.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:27.504+0000 7ff1f37fe700 1 -- 172.21.15.5:0/775444658 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1fc0092e0 con 0x7ff20410c5b0 2024-05-09T00:56:27.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:27.504+0000 7ff208d8f700 1 -- 172.21.15.5:0/775444658 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2041a6580 con 0x7ff20410c5b0 2024-05-09T00:56:27.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:27.504+0000 7ff208d8f700 1 -- 172.21.15.5:0/775444658 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2041adf00 con 0x7ff20410c5b0 2024-05-09T00:56:27.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:27.505+0000 7ff1f37fe700 1 -- 172.21.15.5:0/775444658 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1fc004bb0 con 0x7ff20410c5b0 2024-05-09T00:56:27.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:27.505+0000 7ff1f37fe700 1 -- 172.21.15.5:0/775444658 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1fc004080 con 0x7ff20410c5b0 2024-05-09T00:56:27.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:27.510+0000 7ff208d8f700 1 -- 172.21.15.5:0/775444658 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1e4005330 con 0x7ff20410c5b0 2024-05-09T00:56:27.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:27.513+0000 7ff1f37fe700 1 -- 172.21.15.5:0/775444658 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7ff1fc01b030 con 0x7ff20410c5b0 2024-05-09T00:56:27.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:27.514+0000 7ff1f37fe700 1 --2- 172.21.15.5:0/775444658 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ff1ec0b2a80 0x7ff1ec0b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:27.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:27.514+0000 7ff1f37fe700 1 -- 172.21.15.5:0/775444658 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7ff1fc0e1cb0 con 0x7ff20410c5b0 2024-05-09T00:56:27.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:27.521+0000 7ff20259c700 1 --2- 172.21.15.5:0/775444658 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ff1ec0b2a80 0x7ff1ec0b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:27.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:27.522+0000 7ff20259c700 1 --2- 172.21.15.5:0/775444658 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ff1ec0b2a80 0x7ff1ec0b4f90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff2041079e0 tx=0x7ff1f4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:56:27.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:27.537+0000 7ff1f37fe700 1 -- 172.21.15.5:0/775444658 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff1fc022020 con 0x7ff20410c5b0 2024-05-09T00:56:27.790 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:56:27 smithi005 ceph-mon[30365]: pgmap v674: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 943 MiB/s wr, 2.12k op/s 2024-05-09T00:56:27.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:56:27 smithi157 ceph-mon[34294]: pgmap v674: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 943 MiB/s wr, 2.12k op/s 2024-05-09T00:56:28.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:56:27 smithi131 ceph-mon[34547]: pgmap v674: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 943 MiB/s wr, 2.12k op/s 2024-05-09T00:56:28.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:28.197+0000 7ff208d8f700 1 -- 172.21.15.5:0/775444658 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff1e4005620 con 0x7ff20410c5b0 2024-05-09T00:56:28.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:28.202+0000 7ff1f37fe700 1 -- 172.21.15.5:0/775444658 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11926 (secure 0 0 0) 0x7ff1fc010040 con 0x7ff20410c5b0 2024-05-09T00:56:28.206 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:56:28.207 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2685771887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2685771887},{"type":"v1","addr":"172.21.15.5:6835","nonce":2685771887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/254017535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":254017535},{"type":"v1","addr":"172.21.15.5:6837","nonce":254017535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T00:43:45.856066+0000","modified":"2024-05-09T00:52:11.233619+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24508,"mds_1":24499,"mds_2":24494,"mds_3":24493,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14598":{"gid":14598,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2109255228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2109255228},{"type":"v1","addr":"172.21.15.5:6839","nonce":2109255228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/3809966329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3809966329},{"type":"v1","addr":"172.21.15.5:6841","nonce":3809966329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6833/3272897742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3272897742},{"type":"v1","addr":"172.21.15.131:6833","nonce":3272897742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1283093898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1283093898},{"type":"v1","addr":"172.21.15.131:6835","nonce":1283093898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/2809525455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2809525455},{"type":"v1","addr":"172.21.15.157:6833","nonce":2809525455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.131:6837/1633006186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":1633006186},{"type":"v1","addr":"172.21.15.131:6837","nonce":1633006186}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.157:6835/832898273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":832898273},{"type":"v1","addr":"172.21.15.157:6835","nonce":832898273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.131:6839/2748669631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2748669631},{"type":"v1","addr":"172.21.15.131:6839","nonce":2748669631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.157:6837/744863009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":744863009},{"type":"v1","addr":"172.21.15.157:6837","nonce":744863009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.157:6839/2779860399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":2779860399},{"type":"v1","addr":"172.21.15.157:6839","nonce":2779860399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24508,24494,24499,24493,24467]},"id":1}]} 2024-05-09T00:56:28.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:28.210+0000 7ff208d8f700 1 -- 172.21.15.5:0/775444658 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ff1ec0b2a80 msgr2=0x7ff1ec0b4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:56:28.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:28.210+0000 7ff208d8f700 1 --2- 172.21.15.5:0/775444658 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ff1ec0b2a80 0x7ff1ec0b4f90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff2041079e0 tx=0x7ff1f4008040 comp rx=0 tx=0).stop 2024-05-09T00:56:28.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:28.210+0000 7ff208d8f700 1 -- 172.21.15.5:0/775444658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff20410c5b0 msgr2=0x7ff2041a5c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:56:28.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:28.210+0000 7ff208d8f700 1 --2- 172.21.15.5:0/775444658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff20410c5b0 0x7ff2041a5c50 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc00efc0 tx=0x7ff1fc00c3a0 comp rx=0 tx=0).stop 2024-05-09T00:56:28.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:28.217+0000 7ff208d8f700 1 -- 172.21.15.5:0/775444658 shutdown_connections 2024-05-09T00:56:28.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:28.217+0000 7ff208d8f700 1 --2- 172.21.15.5:0/775444658 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ff1ec0b2a80 0x7ff1ec0b4f90 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:28.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:28.217+0000 7ff208d8f700 1 --2- 172.21.15.5:0/775444658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff20410c5b0 0x7ff2041a5c50 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:28.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:28.217+0000 7ff208d8f700 1 --2- 172.21.15.5:0/775444658 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff204106570 0x7ff204112090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:28.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:28.217+0000 7ff208d8f700 1 --2- 172.21.15.5:0/775444658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff204106ec0 0x7ff204112630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:28.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:28.217+0000 7ff208d8f700 1 -- 172.21.15.5:0/775444658 >> 172.21.15.5:0/775444658 conn(0x7ff204102050 msgr2=0x7ff204103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:56:28.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:28.219+0000 7ff208d8f700 1 -- 172.21.15.5:0/775444658 shutdown_connections 2024-05-09T00:56:28.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:28.219+0000 7ff208d8f700 1 -- 172.21.15.5:0/775444658 wait complete. 2024-05-09T00:56:28.230 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 31 2024-05-09T00:56:28.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:56:28 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/775444658' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:56:29.040 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:56:28 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/775444658' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:56:29.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:56:28 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/775444658' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:56:30.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:56:29 smithi005 ceph-mon[30365]: pgmap v675: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 839 MiB/s wr, 1.91k op/s 2024-05-09T00:56:30.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:56:29 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:56:30.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:56:29 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:56:30.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:56:29 smithi131 ceph-mon[34547]: pgmap v675: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 839 MiB/s wr, 1.91k op/s 2024-05-09T00:56:30.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:56:29 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:56:30.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:56:29 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:56:30.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:56:29 smithi157 ceph-mon[34294]: pgmap v675: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 839 MiB/s wr, 1.91k op/s 2024-05-09T00:56:30.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:56:29 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:56:30.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:56:29 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:56:30.369 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-09T00:56:31.544 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:56:32.039 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:56:31 smithi005 ceph-mon[30365]: pgmap v676: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 961 MiB/s wr, 2.30k op/s 2024-05-09T00:56:32.039 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:56:31 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:56:32.039 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:56:31 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:56:32.039 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:56:31 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:56:32.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:56:31 smithi131 ceph-mon[34547]: pgmap v676: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 961 MiB/s wr, 2.30k op/s 2024-05-09T00:56:32.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:56:31 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:56:32.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:56:31 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:56:32.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:56:31 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:56:32.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:56:31 smithi157 ceph-mon[34294]: pgmap v676: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 961 MiB/s wr, 2.30k op/s 2024-05-09T00:56:32.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:56:31 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:56:32.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:56:31 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:56:32.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:56:31 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:56:33.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.110+0000 7fe99ec53640 1 -- 172.21.15.5:0/1362031313 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe99810a550 msgr2=0x7fe99810af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:56:33.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.110+0000 7fe99ec53640 1 --2- 172.21.15.5:0/1362031313 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe99810a550 0x7fe99810af40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe990009ca0 tx=0x7fe99002a8f0 comp rx=0 tx=0).stop 2024-05-09T00:56:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.111+0000 7fe99ec53640 1 -- 172.21.15.5:0/1362031313 shutdown_connections 2024-05-09T00:56:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.111+0000 7fe99ec53640 1 --2- 172.21.15.5:0/1362031313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9981101a0 0x7fe9981105b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.111+0000 7fe99ec53640 1 --2- 172.21.15.5:0/1362031313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe99810b480 0x7fe99810b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.111+0000 7fe99ec53640 1 --2- 172.21.15.5:0/1362031313 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe99810a550 0x7fe99810af40 secure :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe990009ca0 tx=0x7fe99002a8f0 comp rx=0 tx=0).stop 2024-05-09T00:56:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.111+0000 7fe99ec53640 1 -- 172.21.15.5:0/1362031313 >> 172.21.15.5:0/1362031313 conn(0x7fe998070670 msgr2=0x7fe998072b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:56:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.112+0000 7fe99ec53640 1 -- 172.21.15.5:0/1362031313 shutdown_connections 2024-05-09T00:56:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.112+0000 7fe99ec53640 1 -- 172.21.15.5:0/1362031313 wait complete. 2024-05-09T00:56:33.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.113+0000 7fe99ec53640 1 Processor -- start 2024-05-09T00:56:33.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.113+0000 7fe99ec53640 1 -- start start 2024-05-09T00:56:33.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.114+0000 7fe99ec53640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe99810b480 0x7fe9981a92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.114+0000 7fe99ec53640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9981101a0 0x7fe9981a9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.114+0000 7fe99c9ef640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe99810b480 0x7fe9981a92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.114+0000 7fe99c9ef640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe99810b480 0x7fe9981a92d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:57324/0 (socket says 172.21.15.5:57324) 2024-05-09T00:56:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.114+0000 7fe99c9ef640 1 -- 172.21.15.5:0/3649167732 learned_addr learned my addr 172.21.15.5:0/3649167732 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:56:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.114+0000 7fe99ec53640 1 --2- 172.21.15.5:0/3649167732 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9981ae790 0x7fe9981b0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:33.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.115+0000 7fe99ec53640 1 -- 172.21.15.5:0/3649167732 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe99811e610 con 0x7fe9981101a0 2024-05-09T00:56:33.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.115+0000 7fe997fff640 1 --2- 172.21.15.5:0/3649167732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9981101a0 0x7fe9981a9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:33.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.115+0000 7fe99d1f0640 1 --2- 172.21.15.5:0/3649167732 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9981ae790 0x7fe9981b0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:33.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.115+0000 7fe99ec53640 1 -- 172.21.15.5:0/3649167732 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe99811e490 con 0x7fe9981ae790 2024-05-09T00:56:33.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.115+0000 7fe99ec53640 1 -- 172.21.15.5:0/3649167732 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe99811e790 con 0x7fe99810b480 2024-05-09T00:56:33.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.115+0000 7fe99c9ef640 1 -- 172.21.15.5:0/3649167732 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9981ae790 msgr2=0x7fe9981b0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:56:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.115+0000 7fe99c9ef640 1 --2- 172.21.15.5:0/3649167732 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9981ae790 0x7fe9981b0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.115+0000 7fe99c9ef640 1 -- 172.21.15.5:0/3649167732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9981101a0 msgr2=0x7fe9981a9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:56:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.115+0000 7fe99c9ef640 1 --2- 172.21.15.5:0/3649167732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9981101a0 0x7fe9981a9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.116+0000 7fe99c9ef640 1 -- 172.21.15.5:0/3649167732 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9981b12e0 con 0x7fe99810b480 2024-05-09T00:56:33.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.116+0000 7fe99c9ef640 1 --2- 172.21.15.5:0/3649167732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe99810b480 0x7fe9981a92d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe99002ae60 tx=0x7fe990032da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:56:33.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.116+0000 7fe995ffb640 1 -- 172.21.15.5:0/3649167732 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe990030c30 con 0x7fe99810b480 2024-05-09T00:56:33.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.116+0000 7fe99ec53640 1 -- 172.21.15.5:0/3649167732 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9981b1570 con 0x7fe99810b480 2024-05-09T00:56:33.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.116+0000 7fe995ffb640 1 -- 172.21.15.5:0/3649167732 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe99002d070 con 0x7fe99810b480 2024-05-09T00:56:33.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.116+0000 7fe995ffb640 1 -- 172.21.15.5:0/3649167732 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9900045d0 con 0x7fe99810b480 2024-05-09T00:56:33.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.117+0000 7fe99ec53640 1 -- 172.21.15.5:0/3649167732 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9981b1b10 con 0x7fe99810b480 2024-05-09T00:56:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.122+0000 7fe9a5be6740 1 Processor -- start 2024-05-09T00:56:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.122+0000 7fe9a5be6740 1 -- start start 2024-05-09T00:56:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.123+0000 7fe995ffb640 1 -- 172.21.15.5:0/3649167732 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fe990003970 con 0x7fe99810b480 2024-05-09T00:56:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.124+0000 7fe995ffb640 1 --2- 172.21.15.5:0/3649167732 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe9800b2930 0x7fe9800b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.125+0000 7fe995ffb640 1 -- 172.21.15.5:0/3649167732 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fe9901034b0 con 0x7fe99810b480 2024-05-09T00:56:33.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.125+0000 7fe99d1f0640 1 --2- 172.21.15.5:0/3649167732 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe9800b2930 0x7fe9800b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:33.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.126+0000 7fe99d1f0640 1 --2- 172.21.15.5:0/3649167732 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe9800b2930 0x7fe9800b4e40 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe98c004630 tx=0x7fe98c00b2e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:56:33.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.126+0000 7fe9a5be6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580d5de81b0 0x5580d618c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:33.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.126+0000 7fe9a5be6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5580d5dbf2f0 0x5580d618ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:33.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.126+0000 7fe9a5be6740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5580d5dc74d0 0x5580d6193950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:33.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.126+0000 7fe9a5be6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5580d60f07c0 con 0x5580d5de81b0 2024-05-09T00:56:33.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.126+0000 7fe9a5be6740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5580d608c0e0 con 0x5580d5dc74d0 2024-05-09T00:56:33.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.126+0000 7fe9a5be6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5580d60ec780 con 0x5580d5dbf2f0 2024-05-09T00:56:33.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.127+0000 7fe997fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5580d5dbf2f0 0x5580d618ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.129+0000 7fe99c9ef640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580d5de81b0 0x5580d618c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.129+0000 7fe99c9ef640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580d5de81b0 0x5580d618c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39230/0 (socket says 172.21.15.5:39230) 2024-05-09T00:56:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.129+0000 7fe99c9ef640 1 -- 172.21.15.5:0/2736506394 learned_addr learned my addr 172.21.15.5:0/2736506394 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:56:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.129+0000 7fe997fff640 1 --2- 172.21.15.5:0/2736506394 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5580d5dbf2f0 0x5580d618ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:57336/0 (socket says 172.21.15.5:57336) 2024-05-09T00:56:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.131+0000 7fe99c9ef640 1 -- 172.21.15.5:0/2736506394 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5580d5dbf2f0 msgr2=0x5580d618ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:56:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.131+0000 7fe99c9ef640 1 --2- 172.21.15.5:0/2736506394 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5580d5dbf2f0 0x5580d618ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.131+0000 7fe99c9ef640 1 -- 172.21.15.5:0/2736506394 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5580d5dc74d0 msgr2=0x5580d6193950 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:56:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.131+0000 7fe99c9ef640 1 --2- 172.21.15.5:0/2736506394 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5580d5dc74d0 0x5580d6193950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.131+0000 7fe99c9ef640 1 -- 172.21.15.5:0/2736506394 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580d60b57c0 con 0x5580d5de81b0 2024-05-09T00:56:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.132+0000 7fe99c9ef640 1 --2- 172.21.15.5:0/2736506394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580d5de81b0 0x5580d618c4d0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fe990005e30 tx=0x7fe990004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3cf3817839d02ec server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:56:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.133+0000 7fe97effd640 1 -- 172.21.15.5:0/2736506394 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9900d3040 con 0x5580d5de81b0 2024-05-09T00:56:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.133+0000 7fe97effd640 1 -- 172.21.15.5:0/2736506394 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe990036070 con 0x5580d5de81b0 2024-05-09T00:56:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.133+0000 7fe97effd640 1 -- 172.21.15.5:0/2736506394 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9900d38a0 con 0x5580d5de81b0 2024-05-09T00:56:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.136+0000 7fe9a5be6740 1 -- 172.21.15.5:0/2736506394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580d5de81b0 msgr2=0x5580d618c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:56:33.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.136+0000 7fe9a5be6740 1 --2- 172.21.15.5:0/2736506394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580d5de81b0 0x5580d618c4d0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fe990005e30 tx=0x7fe990004040 comp rx=0 tx=0).stop 2024-05-09T00:56:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.139+0000 7fe9a5be6740 1 -- 172.21.15.5:0/2736506394 shutdown_connections 2024-05-09T00:56:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.139+0000 7fe9a5be6740 1 --2- 172.21.15.5:0/2736506394 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5580d5dc74d0 0x5580d6193950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.139+0000 7fe9a5be6740 1 --2- 172.21.15.5:0/2736506394 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5580d5dbf2f0 0x5580d618ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.139+0000 7fe9a5be6740 1 --2- 172.21.15.5:0/2736506394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580d5de81b0 0x5580d618c4d0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.139+0000 7fe9a5be6740 1 -- 172.21.15.5:0/2736506394 >> 172.21.15.5:0/2736506394 conn(0x5580d5ddc300 msgr2=0x5580d6187e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:56:33.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.140+0000 7fe9a5be6740 1 -- 172.21.15.5:0/2736506394 shutdown_connections 2024-05-09T00:56:33.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.140+0000 7fe9a5be6740 1 -- 172.21.15.5:0/2736506394 wait complete. 2024-05-09T00:56:33.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.141+0000 7fe9a5be6740 1 -- start start 2024-05-09T00:56:33.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.141+0000 7fe9a5be6740 1 Processor -- start 2024-05-09T00:56:33.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.142+0000 7fe9a5be6740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5580d5dbf2f0 0x5580d6195ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.142+0000 7fe9a5be6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5580d5dc74d0 0x5580d6196530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.142+0000 7fe9a5be6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580d5de81b0 0x5580d619d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.142+0000 7fe9a5be6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5580d6194160 con 0x5580d5de81b0 2024-05-09T00:56:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.142+0000 7fe9a5be6740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5580d60779c0 con 0x5580d5dbf2f0 2024-05-09T00:56:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.142+0000 7fe9a5be6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5580d61942e0 con 0x5580d5dc74d0 2024-05-09T00:56:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.142+0000 7fe9a5be6740 10 client.0 _ll_register_callbacks cb 0x5580d60a4020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:56:33.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.144+0000 7fe99d1f0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580d5de81b0 0x5580d619d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:33.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.144+0000 7fe99d1f0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580d5de81b0 0x5580d619d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39240/0 (socket says 172.21.15.5:39240) 2024-05-09T00:56:33.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.144+0000 7fe99d1f0640 1 -- 172.21.15.5:0/4214965864 learned_addr learned my addr 172.21.15.5:0/4214965864 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:56:33.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.144+0000 7fe99c9ef640 1 --2- 172.21.15.5:0/4214965864 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5580d5dbf2f0 0x5580d6195ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:33.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.144+0000 7fe997fff640 1 --2- 172.21.15.5:0/4214965864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5580d5dc74d0 0x5580d6196530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:33.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.145+0000 7fe99c9ef640 1 -- 172.21.15.5:0/4214965864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5580d5dc74d0 msgr2=0x5580d6196530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:56:33.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.145+0000 7fe99c9ef640 1 --2- 172.21.15.5:0/4214965864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5580d5dc74d0 0x5580d6196530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:33.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.145+0000 7fe99c9ef640 1 -- 172.21.15.5:0/4214965864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580d5de81b0 msgr2=0x5580d619d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:56:33.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.145+0000 7fe99c9ef640 1 --2- 172.21.15.5:0/4214965864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580d5de81b0 0x5580d619d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:33.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.145+0000 7fe99c9ef640 1 -- 172.21.15.5:0/4214965864 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580d609e1f0 con 0x5580d5dbf2f0 2024-05-09T00:56:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.146+0000 7fe99c9ef640 1 --2- 172.21.15.5:0/4214965864 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5580d5dbf2f0 0x5580d6195ff0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe990030960 tx=0x7fe990000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9af35c9b52e63fd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:56:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.146+0000 7fe977fff640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-09T00:56:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.146+0000 7fe977fff640 1 -- 172.21.15.5:0/4214965864 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9900d3040 con 0x5580d5dbf2f0 2024-05-09T00:56:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.146+0000 7fe977fff640 1 -- 172.21.15.5:0/4214965864 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fe990003c60 con 0x5580d5dbf2f0 2024-05-09T00:56:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.148+0000 7fe9a5be6740 1 -- 172.21.15.5:0/4214965864 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe990036070 con 0x5580d5dbf2f0 2024-05-09T00:56:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.163+0000 7fe977fff640 1 -- 172.21.15.5:0/4214965864 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe9900cb140 con 0x5580d5dbf2f0 2024-05-09T00:56:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.164+0000 7fe9a5be6740 10 client.26009 fetch_fsmap learned FSMap version 31 2024-05-09T00:56:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.164+0000 7fe9a5be6740 1 -- 172.21.15.5:0/4214965864 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5580d5e53790 con 0x5580d5dbf2f0 2024-05-09T00:56:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.165+0000 7fe977fff640 1 -- 172.21.15.5:0/4214965864 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe990036070 con 0x5580d5dbf2f0 2024-05-09T00:56:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.165+0000 7fe977fff640 1 -- 172.21.15.5:0/4214965864 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fe9900caca0 con 0x5580d5dbf2f0 2024-05-09T00:56:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.165+0000 7fe9a5be6740 10 client.26009 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:56:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.165+0000 7fe9a5be6740 10 client.26009 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:56:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.165+0000 7fe9a5be6740 20 client.26009 populate_metadata read hostname 'smithi005' 2024-05-09T00:56:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.165+0000 7fe9a5be6740 1 --2- 172.21.15.5:0/4214965864 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5580d5da6d80 0x5580d61a35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.165+0000 7fe9a5be6740 4 client.26009 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:56:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.165+0000 7fe9a5be6740 1 -- 172.21.15.5:0/4214965864 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5580d60fe3d0 con 0x5580d5da6d80 2024-05-09T00:56:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.166+0000 7fe997fff640 1 --2- 172.21.15.5:0/4214965864 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5580d5da6d80 0x5580d61a35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:33.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.168+0000 7fe997fff640 1 --2- 172.21.15.5:0/4214965864 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5580d5da6d80 0x5580d61a35c0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d404a2bda8a5460 server_cookie=9a3dec6445e02c4f in_seq=0 out_seq=0 2024-05-09T00:56:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.168+0000 7fe977fff640 10 client.26009 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:56:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.179+0000 7fe977fff640 1 -- 172.21.15.5:0/4214965864 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5580d60fe3d0 con 0x5580d5da6d80 2024-05-09T00:56:33.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.179+0000 7fe977fff640 10 client.26009 handle_command_reply: tid=0 2024-05-09T00:56:33.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.179+0000 7fe9a5be6740 1 client.26009 shutdown 2024-05-09T00:56:33.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.180+0000 7fe9a5be6740 1 -- 172.21.15.5:0/4214965864 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5580d5dbf2f0 msgr2=0x5580d6195ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:56:33.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.180+0000 7fe9a5be6740 1 --2- 172.21.15.5:0/4214965864 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5580d5dbf2f0 0x5580d6195ff0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe990030960 tx=0x7fe990000f40 comp rx=0 tx=0).stop 2024-05-09T00:56:33.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.194+0000 7fe9a5be6740 1 -- 172.21.15.5:0/4214965864 shutdown_connections 2024-05-09T00:56:33.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.194+0000 7fe9a5be6740 1 --2- 172.21.15.5:0/4214965864 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5580d5da6d80 0x5580d61a35c0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:33.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.194+0000 7fe9a5be6740 1 --2- 172.21.15.5:0/4214965864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580d5de81b0 0x5580d619d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:33.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.194+0000 7fe9a5be6740 1 --2- 172.21.15.5:0/4214965864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5580d5dc74d0 0x5580d6196530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:33.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.194+0000 7fe9a5be6740 1 --2- 172.21.15.5:0/4214965864 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5580d5dbf2f0 0x5580d6195ff0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:33.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.194+0000 7fe9a5be6740 1 -- 172.21.15.5:0/4214965864 >> 172.21.15.5:0/4214965864 conn(0x5580d5ddc300 msgr2=0x5580d6186e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:56:33.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.194+0000 7fe977fff640 0 client.26009 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:56:33.197 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.194+0000 7fe9a5be6740 1 -- 172.21.15.5:0/4214965864 shutdown_connections 2024-05-09T00:56:33.197 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.194+0000 7fe9a5be6740 1 -- 172.21.15.5:0/4214965864 wait complete. 2024-05-09T00:56:33.197 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.195+0000 7fe9a5be6740 20 client.26009 trim_cache size 0 max 16384 2024-05-09T00:56:33.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.246+0000 7fe9a5be6740 1 Processor -- start 2024-05-09T00:56:33.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.246+0000 7fe9a5be6740 1 -- start start 2024-05-09T00:56:33.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.253+0000 7fe9a5be6740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5580d5dbf2f0 0x5580d6190700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:33.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.253+0000 7fe9a5be6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580d5dc74d0 0x5580d6190c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:33.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.253+0000 7fe9a5be6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5580d5de81b0 0x5580d61945d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:33.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.254+0000 7fe9a5be6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe9981ba9e0 con 0x5580d5dc74d0 2024-05-09T00:56:33.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.254+0000 7fe9a5be6740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe9981bc520 con 0x5580d5dbf2f0 2024-05-09T00:56:33.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.254+0000 7fe9a5be6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe9981bb0b0 con 0x5580d5de81b0 2024-05-09T00:56:33.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.254+0000 7fe997fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580d5dc74d0 0x5580d6190c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:33.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.254+0000 7fe997fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580d5dc74d0 0x5580d6190c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39242/0 (socket says 172.21.15.5:39242) 2024-05-09T00:56:33.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.254+0000 7fe997fff640 1 -- 172.21.15.5:0/1329510246 learned_addr learned my addr 172.21.15.5:0/1329510246 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:56:33.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.255+0000 7fe99c9ef640 1 --2- 172.21.15.5:0/1329510246 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5580d5dbf2f0 0x5580d6190700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:33.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.255+0000 7fe997fff640 1 -- 172.21.15.5:0/1329510246 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5580d5de81b0 msgr2=0x5580d61945d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:56:33.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.255+0000 7fe997fff640 1 --2- 172.21.15.5:0/1329510246 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5580d5de81b0 0x5580d61945d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:33.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.255+0000 7fe997fff640 1 -- 172.21.15.5:0/1329510246 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5580d5dbf2f0 msgr2=0x5580d6190700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:56:33.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.255+0000 7fe997fff640 1 --2- 172.21.15.5:0/1329510246 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5580d5dbf2f0 0x5580d6190700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:33.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.255+0000 7fe997fff640 1 -- 172.21.15.5:0/1329510246 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580d5eca5f0 con 0x5580d5dc74d0 2024-05-09T00:56:33.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.255+0000 7fe997fff640 1 --2- 172.21.15.5:0/1329510246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580d5dc74d0 0x5580d6190c40 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x5580d6188bf0 tx=0x7fe98800b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bad678fac1ea3665 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:56:33.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.257+0000 7fe97e7fc640 1 -- 172.21.15.5:0/1329510246 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe988003b00 con 0x5580d5dc74d0 2024-05-09T00:56:33.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.257+0000 7fe97e7fc640 1 -- 172.21.15.5:0/1329510246 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe988024020 con 0x5580d5dc74d0 2024-05-09T00:56:33.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.257+0000 7fe97e7fc640 1 -- 172.21.15.5:0/1329510246 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9880116a0 con 0x5580d5dc74d0 2024-05-09T00:56:33.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.259+0000 7fe9a5be6740 1 -- 172.21.15.5:0/1329510246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580d5dc74d0 msgr2=0x5580d6190c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:56:33.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.259+0000 7fe9a5be6740 1 --2- 172.21.15.5:0/1329510246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580d5dc74d0 0x5580d6190c40 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x5580d6188bf0 tx=0x7fe98800b390 comp rx=0 tx=0).stop 2024-05-09T00:56:33.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.261+0000 7fe9a5be6740 1 -- 172.21.15.5:0/1329510246 shutdown_connections 2024-05-09T00:56:33.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.261+0000 7fe9a5be6740 1 --2- 172.21.15.5:0/1329510246 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5580d5de81b0 0x5580d61945d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:33.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.261+0000 7fe9a5be6740 1 --2- 172.21.15.5:0/1329510246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580d5dc74d0 0x5580d6190c40 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:33.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.261+0000 7fe9a5be6740 1 --2- 172.21.15.5:0/1329510246 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5580d5dbf2f0 0x5580d6190700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:33.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.261+0000 7fe9a5be6740 1 -- 172.21.15.5:0/1329510246 >> 172.21.15.5:0/1329510246 conn(0x5580d5ddc300 msgr2=0x5580d6187530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:56:33.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.265+0000 7fe9a5be6740 1 -- 172.21.15.5:0/1329510246 shutdown_connections 2024-05-09T00:56:33.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.266+0000 7fe9a5be6740 1 -- 172.21.15.5:0/1329510246 wait complete. 2024-05-09T00:56:33.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.266+0000 7fe9a5be6740 1 -- start start 2024-05-09T00:56:33.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.267+0000 7fe9a5be6740 1 Processor -- start 2024-05-09T00:56:33.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.268+0000 7fe9a5be6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580d5dbf2f0 0x5580d619c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:33.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.269+0000 7fe9a5be6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5580d5dc74d0 0x5580d61994f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:33.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.269+0000 7fe9a5be6740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5580d5de81b0 0x5580d6199a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:33.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.269+0000 7fe9a5be6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe9981bbe50 con 0x5580d5dbf2f0 2024-05-09T00:56:33.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.269+0000 7fe9a5be6740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe9981bb780 con 0x5580d5de81b0 2024-05-09T00:56:33.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.269+0000 7fe9a5be6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe9981b89c0 con 0x5580d5dc74d0 2024-05-09T00:56:33.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.269+0000 7fe9a5be6740 10 client.0 _ll_register_callbacks cb 0x7fe98800b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:56:33.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.270+0000 7fe99c9ef640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580d5dbf2f0 0x5580d619c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:33.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.270+0000 7fe997fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5580d5dc74d0 0x5580d61994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:33.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.270+0000 7fe997fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5580d5dc74d0 0x5580d61994f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:57368/0 (socket says 172.21.15.5:57368) 2024-05-09T00:56:33.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.270+0000 7fe997fff640 1 -- 172.21.15.5:0/1028149334 learned_addr learned my addr 172.21.15.5:0/1028149334 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:56:33.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.271+0000 7fe99d1f0640 1 --2- 172.21.15.5:0/1028149334 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5580d5de81b0 0x5580d6199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:33.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.271+0000 7fe99c9ef640 1 -- 172.21.15.5:0/1028149334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5580d5dc74d0 msgr2=0x5580d61994f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:56:33.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.271+0000 7fe99c9ef640 1 --2- 172.21.15.5:0/1028149334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5580d5dc74d0 0x5580d61994f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:33.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.271+0000 7fe99c9ef640 1 -- 172.21.15.5:0/1028149334 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5580d5de81b0 msgr2=0x5580d6199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:56:33.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.271+0000 7fe99c9ef640 1 --2- 172.21.15.5:0/1028149334 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5580d5de81b0 0x5580d6199a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:33.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.271+0000 7fe99c9ef640 1 -- 172.21.15.5:0/1028149334 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580d60b4ed0 con 0x5580d5dbf2f0 2024-05-09T00:56:33.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.272+0000 7fe99c9ef640 1 --2- 172.21.15.5:0/1028149334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580d5dbf2f0 0x5580d619c970 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fe9900ca740 tx=0x7fe990003e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6ab7187e8ab280f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:56:33.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.272+0000 7fe977fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:56:33.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.272+0000 7fe977fff640 1 -- 172.21.15.5:0/1028149334 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9900d3070 con 0x5580d5dbf2f0 2024-05-09T00:56:33.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.272+0000 7fe9a5be6740 1 -- 172.21.15.5:0/1028149334 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe988024020 con 0x5580d5dbf2f0 2024-05-09T00:56:33.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.272+0000 7fe977fff640 1 -- 172.21.15.5:0/1028149334 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fe9900c6cd0 con 0x5580d5dbf2f0 2024-05-09T00:56:33.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.273+0000 7fe977fff640 1 -- 172.21.15.5:0/1028149334 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe9900ca550 con 0x5580d5dbf2f0 2024-05-09T00:56:33.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.273+0000 7fe9a5be6740 10 client.16518 fetch_fsmap learned FSMap version 31 2024-05-09T00:56:33.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.273+0000 7fe9a5be6740 1 -- 172.21.15.5:0/1028149334 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5580d60b4ed0 con 0x5580d5dbf2f0 2024-05-09T00:56:33.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.274+0000 7fe977fff640 1 -- 172.21.15.5:0/1028149334 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe990036070 con 0x5580d5dbf2f0 2024-05-09T00:56:33.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.274+0000 7fe977fff640 1 -- 172.21.15.5:0/1028149334 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fe9900caca0 con 0x5580d5dbf2f0 2024-05-09T00:56:33.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.274+0000 7fe9a5be6740 10 client.16518 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:56:33.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.274+0000 7fe9a5be6740 10 client.16518 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:56:33.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.274+0000 7fe9a5be6740 20 client.16518 populate_metadata read hostname 'smithi005' 2024-05-09T00:56:33.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.276+0000 7fe9a5be6740 1 --2- 172.21.15.5:0/1028149334 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5580d5d7c1c0 0x5580d620cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:33.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.276+0000 7fe9a5be6740 4 client.16518 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:56:33.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.276+0000 7fe9a5be6740 1 -- 172.21.15.5:0/1028149334 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fe988024020 con 0x5580d5d7c1c0 2024-05-09T00:56:33.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.285+0000 7fe997fff640 1 --2- 172.21.15.5:0/1028149334 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5580d5d7c1c0 0x5580d620cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:33.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.299+0000 7fe977fff640 10 client.16518 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:56:33.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:33.299+0000 7fe997fff640 1 --2- 172.21.15.5:0/1028149334 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5580d5d7c1c0 0x5580d620cf10 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e48dd9ce27791c1f server_cookie=cc370a41322b9bc0 in_seq=0 out_seq=0 2024-05-09T00:56:33.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:56:33 smithi005 ceph-mon[30365]: pgmap v677: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 744 MiB/s wr, 1.95k op/s 2024-05-09T00:56:34.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:56:33 smithi131 ceph-mon[34547]: pgmap v677: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 744 MiB/s wr, 1.95k op/s 2024-05-09T00:56:34.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:56:33 smithi157 ceph-mon[34294]: pgmap v677: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 744 MiB/s wr, 1.95k op/s 2024-05-09T00:56:36.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:56:35 smithi005 ceph-mon[30365]: pgmap v678: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 881 MiB/s wr, 2.30k op/s 2024-05-09T00:56:36.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:56:35 smithi131 ceph-mon[34547]: pgmap v678: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 881 MiB/s wr, 2.30k op/s 2024-05-09T00:56:36.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:56:35 smithi157 ceph-mon[34294]: pgmap v678: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 881 MiB/s wr, 2.30k op/s 2024-05-09T00:56:37.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.636+0000 7fc23c6f4700 1 -- 172.21.15.5:0/3618419035 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc23410b6e0 msgr2=0x7fc23410f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:56:37.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.637+0000 7fc23948e700 1 -- 172.21.15.5:0/3618419035 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc230004220 con 0x7fc23410b6e0 2024-05-09T00:56:37.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.637+0000 7fc23c6f4700 1 --2- 172.21.15.5:0/3618419035 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc23410b6e0 0x7fc23410f950 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc230009990 tx=0x7fc230009ca0 comp rx=0 tx=0).stop 2024-05-09T00:56:37.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.640+0000 7fc23c6f4700 1 -- 172.21.15.5:0/3618419035 shutdown_connections 2024-05-09T00:56:37.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.640+0000 7fc23c6f4700 1 --2- 172.21.15.5:0/3618419035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc23410fe90 0x7fc2341102a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:37.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.640+0000 7fc23c6f4700 1 --2- 172.21.15.5:0/3618419035 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc23410ad90 0x7fc23410b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:37.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.640+0000 7fc23c6f4700 1 --2- 172.21.15.5:0/3618419035 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc23410b6e0 0x7fc23410f950 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:37.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.640+0000 7fc23c6f4700 1 -- 172.21.15.5:0/3618419035 >> 172.21.15.5:0/3618419035 conn(0x7fc23406f990 msgr2=0x7fc23406fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:56:37.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.641+0000 7fc23c6f4700 1 -- 172.21.15.5:0/3618419035 shutdown_connections 2024-05-09T00:56:37.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.643+0000 7fc23c6f4700 1 -- 172.21.15.5:0/3618419035 wait complete. 2024-05-09T00:56:37.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.645+0000 7fc23c6f4700 1 Processor -- start 2024-05-09T00:56:37.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.645+0000 7fc23c6f4700 1 -- start start 2024-05-09T00:56:37.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.645+0000 7fc23c6f4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc23410ad90 0x7fc234121440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:37.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.645+0000 7fc23c6f4700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc23410b6e0 0x7fc2341219a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:37.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.646+0000 7fc23c6f4700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc23410fe90 0x7fc234127160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:37.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.646+0000 7fc23c6f4700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc234114b30 con 0x7fc23410ad90 2024-05-09T00:56:37.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.646+0000 7fc23c6f4700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc2341149b0 con 0x7fc23410b6e0 2024-05-09T00:56:37.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.646+0000 7fc23c6f4700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc234114cb0 con 0x7fc23410fe90 2024-05-09T00:56:37.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.647+0000 7fc23ac91700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc23410fe90 0x7fc234127160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:37.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.647+0000 7fc23ac91700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc23410fe90 0x7fc234127160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:45960/0 (socket says 172.21.15.5:45960) 2024-05-09T00:56:37.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.647+0000 7fc23ac91700 1 -- 172.21.15.5:0/3196698982 learned_addr learned my addr 172.21.15.5:0/3196698982 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:56:37.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.649+0000 7fc23ac91700 1 -- 172.21.15.5:0/3196698982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc23410b6e0 msgr2=0x7fc2341219a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T00:56:37.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.649+0000 7fc23ac91700 1 --2- 172.21.15.5:0/3196698982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc23410b6e0 0x7fc2341219a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:37.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.649+0000 7fc23ac91700 1 -- 172.21.15.5:0/3196698982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc23410ad90 msgr2=0x7fc234121440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:56:37.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.649+0000 7fc23ac91700 1 --2- 172.21.15.5:0/3196698982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc23410ad90 0x7fc234121440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:37.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.649+0000 7fc23ac91700 1 -- 172.21.15.5:0/3196698982 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc23411c600 con 0x7fc23410fe90 2024-05-09T00:56:37.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.650+0000 7fc23ac91700 1 --2- 172.21.15.5:0/3196698982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc23410fe90 0x7fc234127160 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc22c00ada0 tx=0x7fc22c00c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:56:37.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.650+0000 7fc22b7fe700 1 -- 172.21.15.5:0/3196698982 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22c017020 con 0x7fc23410fe90 2024-05-09T00:56:37.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.651+0000 7fc23c6f4700 1 -- 172.21.15.5:0/3196698982 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc23411c8f0 con 0x7fc23410fe90 2024-05-09T00:56:37.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.651+0000 7fc23c6f4700 1 -- 172.21.15.5:0/3196698982 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc23411ce30 con 0x7fc23410fe90 2024-05-09T00:56:37.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.657+0000 7fc24b10ab80 1 Processor -- start 2024-05-09T00:56:37.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.658+0000 7fc24b10ab80 1 -- start start 2024-05-09T00:56:37.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.658+0000 7fc24b10ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cfe4b5d00 0x557cfe5be240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:37.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.658+0000 7fc24b10ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557cfe43b4a0 0x557cfe5be780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:37.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.658+0000 7fc24b10ab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cfe5b8400 0x557cfe5b8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:37.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.658+0000 7fc24b10ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557cfe4db0a0 con 0x557cfe4b5d00 2024-05-09T00:56:37.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.658+0000 7fc24b10ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x557cfe4809a0 con 0x557cfe43b4a0 2024-05-09T00:56:37.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.658+0000 7fc24b10ab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x557cfe4d7060 con 0x557cfe5b8400 2024-05-09T00:56:37.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.658+0000 7fc22b7fe700 1 -- 172.21.15.5:0/3196698982 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc22c00ce30 con 0x7fc23410fe90 2024-05-09T00:56:37.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.658+0000 7fc22b7fe700 1 -- 172.21.15.5:0/3196698982 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22c005460 con 0x7fc23410fe90 2024-05-09T00:56:37.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.659+0000 7fc239c8f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cfe4b5d00 0x557cfe5be240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:37.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.659+0000 7fc239c8f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cfe4b5d00 0x557cfe5be240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54030/0 (socket says 172.21.15.5:54030) 2024-05-09T00:56:37.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.659+0000 7fc239c8f700 1 -- 172.21.15.5:0/2994350064 learned_addr learned my addr 172.21.15.5:0/2994350064 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:56:37.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.659+0000 7fc239c8f700 1 -- 172.21.15.5:0/2994350064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cfe5b8400 msgr2=0x557cfe5b8810 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:56:37.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.659+0000 7fc239c8f700 1 --2- 172.21.15.5:0/2994350064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cfe5b8400 0x557cfe5b8810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:37.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.659+0000 7fc239c8f700 1 -- 172.21.15.5:0/2994350064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557cfe43b4a0 msgr2=0x557cfe5be780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:56:37.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.659+0000 7fc239c8f700 1 --2- 172.21.15.5:0/2994350064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557cfe43b4a0 0x557cfe5be780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:37.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.661+0000 7fc239c8f700 1 -- 172.21.15.5:0/2994350064 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cfe471500 con 0x557cfe4b5d00 2024-05-09T00:56:37.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.661+0000 7fc239c8f700 1 --2- 172.21.15.5:0/2994350064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cfe4b5d00 0x557cfe5be240 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fc230006010 tx=0x7fc230009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0ce0facad99239f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:56:37.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.664+0000 7fc22b7fe700 1 -- 172.21.15.5:0/3196698982 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fc22c005600 con 0x7fc23410fe90 2024-05-09T00:56:37.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.668+0000 7fc228ff9700 1 -- 172.21.15.5:0/2994350064 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc230005100 con 0x557cfe4b5d00 2024-05-09T00:56:37.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.670+0000 7fc22b7fe700 1 --2- 172.21.15.5:0/3196698982 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fc2200b29b0 0x7fc2200b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:37.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.671+0000 7fc228ff9700 1 -- 172.21.15.5:0/2994350064 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc230003860 con 0x557cfe4b5d00 2024-05-09T00:56:37.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.671+0000 7fc22b7fe700 1 -- 172.21.15.5:0/3196698982 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fc22c0e1e70 con 0x7fc23410fe90 2024-05-09T00:56:37.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.672+0000 7fc23a490700 1 --2- 172.21.15.5:0/3196698982 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fc2200b29b0 0x7fc2200b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:37.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.673+0000 7fc24b10ab80 1 -- 172.21.15.5:0/2994350064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cfe4b5d00 msgr2=0x557cfe5be240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:56:37.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.673+0000 7fc24b10ab80 1 --2- 172.21.15.5:0/2994350064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cfe4b5d00 0x557cfe5be240 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fc230006010 tx=0x7fc230009560 comp rx=0 tx=0).stop 2024-05-09T00:56:37.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.673+0000 7fc24b10ab80 1 -- 172.21.15.5:0/2994350064 shutdown_connections 2024-05-09T00:56:37.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.673+0000 7fc24b10ab80 1 --2- 172.21.15.5:0/2994350064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cfe5b8400 0x557cfe5b8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:37.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.673+0000 7fc24b10ab80 1 --2- 172.21.15.5:0/2994350064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cfe4b5d00 0x557cfe5be240 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:37.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.673+0000 7fc24b10ab80 1 --2- 172.21.15.5:0/2994350064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557cfe43b4a0 0x557cfe5be780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:37.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.673+0000 7fc24b10ab80 1 -- 172.21.15.5:0/2994350064 >> 172.21.15.5:0/2994350064 conn(0x557cfe43ef70 msgr2=0x557cfe5b3220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:56:37.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.673+0000 7fc23a490700 1 --2- 172.21.15.5:0/3196698982 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fc2200b29b0 0x7fc2200b4ec0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x557cfe5b4a00 tx=0x7fc224006d90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:56:37.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.683+0000 7fc24b10ab80 1 -- 172.21.15.5:0/2994350064 shutdown_connections 2024-05-09T00:56:37.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.684+0000 7fc24b10ab80 1 -- 172.21.15.5:0/2994350064 wait complete. 2024-05-09T00:56:37.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.685+0000 7fc24b10ab80 1 -- start start 2024-05-09T00:56:37.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.685+0000 7fc24b10ab80 1 Processor -- start 2024-05-09T00:56:37.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.686+0000 7fc24b10ab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cfe43b4a0 0x557cfe5c4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:37.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.686+0000 7fc24b10ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cfe5b8400 0x557cfe5c1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:37.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.686+0000 7fc24b10ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557cfe5c2080 0x557cfe5c2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:37.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.686+0000 7fc24b10ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557cfe5b90f0 con 0x557cfe5b8400 2024-05-09T00:56:37.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.686+0000 7fc24b10ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x557cfe475b20 con 0x557cfe5c2080 2024-05-09T00:56:37.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.686+0000 7fc24b10ab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x557cfe5b9270 con 0x557cfe43b4a0 2024-05-09T00:56:37.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.686+0000 7fc24b10ab80 10 client.0 _ll_register_callbacks cb 0x557cfe34c6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:56:37.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.688+0000 7fc23ac91700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557cfe5c2080 0x557cfe5c2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:37.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.688+0000 7fc23ac91700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557cfe5c2080 0x557cfe5c2490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:54458/0 (socket says 172.21.15.5:54458) 2024-05-09T00:56:37.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.688+0000 7fc23ac91700 1 -- 172.21.15.5:0/1249389856 learned_addr learned my addr 172.21.15.5:0/1249389856 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:56:37.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.689+0000 7fc239c8f700 1 --2- 172.21.15.5:0/1249389856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cfe5b8400 0x557cfe5c1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:37.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.691+0000 7fc239c8f700 1 -- 172.21.15.5:0/1249389856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cfe43b4a0 msgr2=0x557cfe5c4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:56:37.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.691+0000 7fc239c8f700 1 --2- 172.21.15.5:0/1249389856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cfe43b4a0 0x557cfe5c4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:37.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.691+0000 7fc239c8f700 1 -- 172.21.15.5:0/1249389856 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557cfe5c2080 msgr2=0x557cfe5c2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:56:37.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.691+0000 7fc239c8f700 1 --2- 172.21.15.5:0/1249389856 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557cfe5c2080 0x557cfe5c2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:37.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.691+0000 7fc239c8f700 1 -- 172.21.15.5:0/1249389856 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557cfe356490 con 0x557cfe5b8400 2024-05-09T00:56:37.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.692+0000 7fc239c8f700 1 --2- 172.21.15.5:0/1249389856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cfe5b8400 0x557cfe5c1b40 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fc230000c00 tx=0x7fc23000f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb9d04fc34f342f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:56:37.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.692+0000 7fc21dffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:56:37.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.693+0000 7fc21dffb700 1 -- 172.21.15.5:0/1249389856 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23000fb20 con 0x557cfe5b8400 2024-05-09T00:56:37.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.693+0000 7fc24b10ab80 1 -- 172.21.15.5:0/1249389856 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557cfe4e8a80 con 0x557cfe5b8400 2024-05-09T00:56:37.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.693+0000 7fc21dffb700 1 -- 172.21.15.5:0/1249389856 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fc23001e040 con 0x557cfe5b8400 2024-05-09T00:56:37.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.694+0000 7fc21dffb700 1 -- 172.21.15.5:0/1249389856 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc23000fcc0 con 0x557cfe5b8400 2024-05-09T00:56:37.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.694+0000 7fc24b10ab80 10 client.16524 fetch_fsmap learned FSMap version 31 2024-05-09T00:56:37.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.694+0000 7fc24b10ab80 1 -- 172.21.15.5:0/1249389856 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x557cfe5cfaf0 con 0x557cfe5b8400 2024-05-09T00:56:37.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.696+0000 7fc21dffb700 1 -- 172.21.15.5:0/1249389856 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x557cfe4e8a80 con 0x557cfe5b8400 2024-05-09T00:56:37.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.697+0000 7fc21dffb700 1 -- 172.21.15.5:0/1249389856 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fc2300054c0 con 0x557cfe5b8400 2024-05-09T00:56:37.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.698+0000 7fc24b10ab80 10 client.16524 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:56:37.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.699+0000 7fc24b10ab80 10 client.16524 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T00:56:37.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.699+0000 7fc24b10ab80 20 client.16524 populate_metadata read hostname 'smithi005' 2024-05-09T00:56:37.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.699+0000 7fc24b10ab80 1 --2- 172.21.15.5:0/1249389856 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x557cfe5d1fc0 0x557cfe5d43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:37.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.699+0000 7fc24b10ab80 4 client.16524 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:56:37.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.699+0000 7fc24b10ab80 1 -- 172.21.15.5:0/1249389856 --> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557cfe4e2b20 con 0x557cfe5d1fc0 2024-05-09T00:56:37.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.700+0000 7fc23a490700 1 --2- 172.21.15.5:0/1249389856 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x557cfe5d1fc0 0x557cfe5d43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:37.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.702+0000 7fc21dffb700 10 client.16524 ms_handle_connect on v2:172.21.15.131:6836/1633006186 2024-05-09T00:56:37.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.702+0000 7fc23a490700 1 --2- 172.21.15.5:0/1249389856 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x557cfe5d1fc0 0x557cfe5d43e0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7513fe5792f00c21 server_cookie=b876a52a08d3068b in_seq=0 out_seq=0 2024-05-09T00:56:37.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.705+0000 7fc21dffb700 1 -- 172.21.15.5:0/1249389856 <== mds.4 v2:172.21.15.131:6836/1633006186 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557cfe4e2b20 con 0x557cfe5d1fc0 2024-05-09T00:56:37.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.705+0000 7fc21dffb700 10 client.16524 handle_command_reply: tid=0 2024-05-09T00:56:37.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.706+0000 7fc24b10ab80 1 client.16524 shutdown 2024-05-09T00:56:37.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.706+0000 7fc24b10ab80 1 -- 172.21.15.5:0/1249389856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cfe5b8400 msgr2=0x557cfe5c1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:56:37.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.706+0000 7fc24b10ab80 1 --2- 172.21.15.5:0/1249389856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cfe5b8400 0x557cfe5c1b40 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fc230000c00 tx=0x7fc23000f670 comp rx=0 tx=0).stop 2024-05-09T00:56:37.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.706+0000 7fc24b10ab80 1 -- 172.21.15.5:0/1249389856 shutdown_connections 2024-05-09T00:56:37.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.706+0000 7fc24b10ab80 1 --2- 172.21.15.5:0/1249389856 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x557cfe5d1fc0 0x557cfe5d43e0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:37.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.706+0000 7fc24b10ab80 1 --2- 172.21.15.5:0/1249389856 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557cfe5c2080 0x557cfe5c2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:37.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.706+0000 7fc24b10ab80 1 --2- 172.21.15.5:0/1249389856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cfe43b4a0 0x557cfe5c4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:37.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.706+0000 7fc24b10ab80 1 --2- 172.21.15.5:0/1249389856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cfe5b8400 0x557cfe5c1b40 secure :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fc230000c00 tx=0x7fc23000f670 comp rx=0 tx=0).stop 2024-05-09T00:56:37.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.706+0000 7fc24b10ab80 1 -- 172.21.15.5:0/1249389856 >> 172.21.15.5:0/1249389856 conn(0x557cfe43ef70 msgr2=0x557cfe5acff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:56:37.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.707+0000 7fc21dffb700 0 client.16524 ms_handle_reset on v2:172.21.15.131:6836/1633006186 2024-05-09T00:56:37.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.707+0000 7fc24b10ab80 1 -- 172.21.15.5:0/1249389856 shutdown_connections 2024-05-09T00:56:37.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.707+0000 7fc24b10ab80 1 -- 172.21.15.5:0/1249389856 wait complete. 2024-05-09T00:56:37.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.709+0000 7fc24b10ab80 20 client.16524 trim_cache size 0 max 16384 2024-05-09T00:56:37.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.757+0000 7fc24b10ab80 1 Processor -- start 2024-05-09T00:56:37.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.757+0000 7fc24b10ab80 1 -- start start 2024-05-09T00:56:37.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.758+0000 7fc24b10ab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cfe43b4a0 0x557cfe5b3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:37.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.758+0000 7fc24b10ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cfe5c2080 0x557cfe5c0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:37.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.759+0000 7fc24b10ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557cfe5c4150 0x557cfe5bf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:37.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.759+0000 7fc24b10ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc21800d590 con 0x557cfe5c2080 2024-05-09T00:56:37.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.759+0000 7fc24b10ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc21800f0d0 con 0x557cfe5c4150 2024-05-09T00:56:37.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.759+0000 7fc24b10ab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc21800dc60 con 0x557cfe43b4a0 2024-05-09T00:56:37.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.761+0000 7fc23ac91700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557cfe5c4150 0x557cfe5bf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:37.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.761+0000 7fc23ac91700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557cfe5c4150 0x557cfe5bf250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:54466/0 (socket says 172.21.15.5:54466) 2024-05-09T00:56:37.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.762+0000 7fc23ac91700 1 -- 172.21.15.5:0/3219380102 learned_addr learned my addr 172.21.15.5:0/3219380102 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:56:37.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.762+0000 7fc239c8f700 1 --2- 172.21.15.5:0/3219380102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cfe5c2080 0x557cfe5c0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:37.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.763+0000 7fc239c8f700 1 -- 172.21.15.5:0/3219380102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cfe43b4a0 msgr2=0x557cfe5b3330 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:56:37.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.763+0000 7fc239c8f700 1 --2- 172.21.15.5:0/3219380102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cfe43b4a0 0x557cfe5b3330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:37.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.763+0000 7fc239c8f700 1 -- 172.21.15.5:0/3219380102 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557cfe5c4150 msgr2=0x557cfe5bf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:56:37.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.763+0000 7fc239c8f700 1 --2- 172.21.15.5:0/3219380102 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557cfe5c4150 0x557cfe5bf250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:37.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.763+0000 7fc239c8f700 1 -- 172.21.15.5:0/3219380102 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cfe5af9d0 con 0x557cfe5c2080 2024-05-09T00:56:37.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.764+0000 7fc239c8f700 1 --2- 172.21.15.5:0/3219380102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cfe5c2080 0x557cfe5c0b40 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x557cfe5c2c00 tx=0x7fc230021870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a49f637926147d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:56:37.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.766+0000 7fc21ffff700 1 -- 172.21.15.5:0/3219380102 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23000fe60 con 0x557cfe5c2080 2024-05-09T00:56:37.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.767+0000 7fc21ffff700 1 -- 172.21.15.5:0/3219380102 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2300058e0 con 0x557cfe5c2080 2024-05-09T00:56:37.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.767+0000 7fc21ffff700 1 -- 172.21.15.5:0/3219380102 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2300039b0 con 0x557cfe5c2080 2024-05-09T00:56:37.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.768+0000 7fc24b10ab80 1 -- 172.21.15.5:0/3219380102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cfe5c2080 msgr2=0x557cfe5c0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:56:37.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.768+0000 7fc24b10ab80 1 --2- 172.21.15.5:0/3219380102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cfe5c2080 0x557cfe5c0b40 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x557cfe5c2c00 tx=0x7fc230021870 comp rx=0 tx=0).stop 2024-05-09T00:56:37.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.769+0000 7fc24b10ab80 1 -- 172.21.15.5:0/3219380102 shutdown_connections 2024-05-09T00:56:37.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.769+0000 7fc24b10ab80 1 --2- 172.21.15.5:0/3219380102 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557cfe5c4150 0x557cfe5bf250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:37.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.769+0000 7fc24b10ab80 1 --2- 172.21.15.5:0/3219380102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cfe43b4a0 0x557cfe5b3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:37.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.769+0000 7fc24b10ab80 1 --2- 172.21.15.5:0/3219380102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cfe5c2080 0x557cfe5c0b40 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:37.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.769+0000 7fc24b10ab80 1 -- 172.21.15.5:0/3219380102 >> 172.21.15.5:0/3219380102 conn(0x557cfe43ef70 msgr2=0x557cfe5c0430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:56:37.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.773+0000 7fc24b10ab80 1 -- 172.21.15.5:0/3219380102 shutdown_connections 2024-05-09T00:56:37.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.774+0000 7fc24b10ab80 1 -- 172.21.15.5:0/3219380102 wait complete. 2024-05-09T00:56:37.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.776+0000 7fc24b10ab80 1 -- start start 2024-05-09T00:56:37.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.776+0000 7fc24b10ab80 1 Processor -- start 2024-05-09T00:56:37.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.777+0000 7fc24b10ab80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cfe43b4a0 0x557cfe5d6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:37.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.777+0000 7fc24b10ab80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557cfe5c2080 0x557cfe5d7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:37.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.777+0000 7fc24b10ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cfe5c4150 0x557cfe5d43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:37.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.778+0000 7fc24b10ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc21800ea00 con 0x557cfe5c4150 2024-05-09T00:56:37.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.778+0000 7fc24b10ab80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc21800e330 con 0x557cfe5c2080 2024-05-09T00:56:37.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.778+0000 7fc24b10ab80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc21800b530 con 0x557cfe43b4a0 2024-05-09T00:56:37.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.778+0000 7fc24b10ab80 10 client.0 _ll_register_callbacks cb 0x557cfe34c6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:56:37.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.778+0000 7fc239c8f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557cfe5c2080 0x557cfe5d7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:37.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.778+0000 7fc239c8f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557cfe5c2080 0x557cfe5d7360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:54476/0 (socket says 172.21.15.5:54476) 2024-05-09T00:56:37.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.778+0000 7fc239c8f700 1 -- 172.21.15.5:0/1565603019 learned_addr learned my addr 172.21.15.5:0/1565603019 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:56:37.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.779+0000 7fc23ac91700 1 --2- 172.21.15.5:0/1565603019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cfe5c4150 0x557cfe5d43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:37.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.779+0000 7fc239c8f700 1 -- 172.21.15.5:0/1565603019 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cfe43b4a0 msgr2=0x557cfe5d6e20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:56:37.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.779+0000 7fc239c8f700 1 --2- 172.21.15.5:0/1565603019 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cfe43b4a0 0x557cfe5d6e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:37.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.779+0000 7fc239c8f700 1 -- 172.21.15.5:0/1565603019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cfe5c4150 msgr2=0x557cfe5d43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:56:37.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.779+0000 7fc239c8f700 1 --2- 172.21.15.5:0/1565603019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cfe5c4150 0x557cfe5d43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:37.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.779+0000 7fc239c8f700 1 -- 172.21.15.5:0/1565603019 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557cfe5d4a10 con 0x557cfe5c2080 2024-05-09T00:56:37.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.781+0000 7fc239c8f700 1 --2- 172.21.15.5:0/1565603019 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557cfe5c2080 0x557cfe5d7360 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc230021420 tx=0x7fc230021bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba17a04ae1dd82c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:56:37.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.782+0000 7fc21dffb700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-09T00:56:37.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.782+0000 7fc21dffb700 1 -- 172.21.15.5:0/1565603019 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23000fe60 con 0x557cfe5c2080 2024-05-09T00:56:37.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.782+0000 7fc24b10ab80 1 -- 172.21.15.5:0/1565603019 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2300058e0 con 0x557cfe5c2080 2024-05-09T00:56:37.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.782+0000 7fc21dffb700 1 -- 172.21.15.5:0/1565603019 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fc23002e040 con 0x557cfe5c2080 2024-05-09T00:56:37.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.794+0000 7fc21dffb700 1 -- 172.21.15.5:0/1565603019 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc230003a70 con 0x557cfe5c2080 2024-05-09T00:56:37.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.794+0000 7fc24b10ab80 10 client.26024 fetch_fsmap learned FSMap version 31 2024-05-09T00:56:37.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.794+0000 7fc24b10ab80 1 -- 172.21.15.5:0/1565603019 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x557cfe63be60 con 0x557cfe5c2080 2024-05-09T00:56:37.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.796+0000 7fc21dffb700 1 -- 172.21.15.5:0/1565603019 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2300058e0 con 0x557cfe5c2080 2024-05-09T00:56:37.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.797+0000 7fc21dffb700 1 -- 172.21.15.5:0/1565603019 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fc23003c070 con 0x557cfe5c2080 2024-05-09T00:56:37.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.798+0000 7fc24b10ab80 10 client.26024 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:56:37.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.798+0000 7fc24b10ab80 10 client.26024 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T00:56:37.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.798+0000 7fc24b10ab80 20 client.26024 populate_metadata read hostname 'smithi005' 2024-05-09T00:56:37.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.798+0000 7fc24b10ab80 1 --2- 172.21.15.5:0/1565603019 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x557cfe63c270 0x557cfe63e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:37.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.798+0000 7fc24b10ab80 4 client.26024 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:56:37.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.798+0000 7fc24b10ab80 1 -- 172.21.15.5:0/1565603019 --> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557cfe356970 con 0x557cfe63c270 2024-05-09T00:56:37.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.803+0000 7fc23a490700 1 --2- 172.21.15.5:0/1565603019 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x557cfe63c270 0x557cfe63e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:37.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.814+0000 7fc23a490700 1 --2- 172.21.15.5:0/1565603019 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x557cfe63c270 0x557cfe63e690 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=57a411ba4a80efe8 server_cookie=72d09a966f697105 in_seq=0 out_seq=0 2024-05-09T00:56:37.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.814+0000 7fc21dffb700 10 client.26024 ms_handle_connect on v2:172.21.15.131:6836/1633006186 2024-05-09T00:56:37.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.815+0000 7fc21dffb700 1 -- 172.21.15.5:0/1565603019 <== mds.4 v2:172.21.15.131:6836/1633006186 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557cfe356970 con 0x557cfe63c270 2024-05-09T00:56:37.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.815+0000 7fc21dffb700 10 client.26024 handle_command_reply: tid=0 2024-05-09T00:56:37.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.815+0000 7fc24b10ab80 1 client.26024 shutdown 2024-05-09T00:56:37.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.815+0000 7fc24b10ab80 1 -- 172.21.15.5:0/1565603019 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557cfe5c2080 msgr2=0x557cfe5d7360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:56:37.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.815+0000 7fc24b10ab80 1 --2- 172.21.15.5:0/1565603019 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557cfe5c2080 0x557cfe5d7360 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc230021420 tx=0x7fc230021bc0 comp rx=0 tx=0).stop 2024-05-09T00:56:37.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.824+0000 7fc24b10ab80 1 -- 172.21.15.5:0/1565603019 shutdown_connections 2024-05-09T00:56:37.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.824+0000 7fc24b10ab80 1 --2- 172.21.15.5:0/1565603019 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x557cfe63c270 0x557cfe63e690 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:37.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.824+0000 7fc24b10ab80 1 --2- 172.21.15.5:0/1565603019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557cfe5c4150 0x557cfe5d43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:37.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.824+0000 7fc24b10ab80 1 --2- 172.21.15.5:0/1565603019 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cfe43b4a0 0x557cfe5d6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:37.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.824+0000 7fc24b10ab80 1 --2- 172.21.15.5:0/1565603019 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557cfe5c2080 0x557cfe5d7360 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:37.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.824+0000 7fc24b10ab80 1 -- 172.21.15.5:0/1565603019 >> 172.21.15.5:0/1565603019 conn(0x557cfe43ef70 msgr2=0x557cfe5acff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:56:37.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.824+0000 7fc21dffb700 0 client.26024 ms_handle_reset on v2:172.21.15.131:6836/1633006186 2024-05-09T00:56:37.828 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-09T00:56:37.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.824+0000 7fc24b10ab80 1 -- 172.21.15.5:0/1565603019 shutdown_connections 2024-05-09T00:56:37.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.824+0000 7fc24b10ab80 1 -- 172.21.15.5:0/1565603019 wait complete. 2024-05-09T00:56:37.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.825+0000 7fc24b10ab80 20 client.26024 trim_cache size 0 max 16384 2024-05-09T00:56:37.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.828+0000 7fc23c6f4700 1 -- 172.21.15.5:0/3196698982 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fc2200b29b0 msgr2=0x7fc2200b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:56:37.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.828+0000 7fc23c6f4700 1 --2- 172.21.15.5:0/3196698982 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fc2200b29b0 0x7fc2200b4ec0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x557cfe5b4a00 tx=0x7fc224006d90 comp rx=0 tx=0).stop 2024-05-09T00:56:37.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.828+0000 7fc23c6f4700 1 -- 172.21.15.5:0/3196698982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc23410fe90 msgr2=0x7fc234127160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:56:37.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.828+0000 7fc23c6f4700 1 --2- 172.21.15.5:0/3196698982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc23410fe90 0x7fc234127160 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc22c00ada0 tx=0x7fc22c00c330 comp rx=0 tx=0).stop 2024-05-09T00:56:37.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.832+0000 7fc23c6f4700 1 -- 172.21.15.5:0/3196698982 shutdown_connections 2024-05-09T00:56:37.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.832+0000 7fc23c6f4700 1 --2- 172.21.15.5:0/3196698982 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fc2200b29b0 0x7fc2200b4ec0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:37.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.832+0000 7fc23c6f4700 1 --2- 172.21.15.5:0/3196698982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc23410fe90 0x7fc234127160 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:37.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.832+0000 7fc23c6f4700 1 --2- 172.21.15.5:0/3196698982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc23410ad90 0x7fc234121440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:37.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.832+0000 7fc23c6f4700 1 --2- 172.21.15.5:0/3196698982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc23410b6e0 0x7fc2341219a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:37.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.832+0000 7fc23c6f4700 1 -- 172.21.15.5:0/3196698982 >> 172.21.15.5:0/3196698982 conn(0x7fc23406f990 msgr2=0x7fc23410e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:56:37.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.832+0000 7fc23c6f4700 1 -- 172.21.15.5:0/3196698982 shutdown_connections 2024-05-09T00:56:37.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:37.833+0000 7fc23c6f4700 1 -- 172.21.15.5:0/3196698982 wait complete. 2024-05-09T00:56:38.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:56:37 smithi005 ceph-mon[30365]: pgmap v679: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 943 MiB/s wr, 2.50k op/s 2024-05-09T00:56:38.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:56:37 smithi131 ceph-mon[34547]: pgmap v679: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 943 MiB/s wr, 2.50k op/s 2024-05-09T00:56:38.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:56:37 smithi157 ceph-mon[34294]: pgmap v679: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 943 MiB/s wr, 2.50k op/s 2024-05-09T00:56:38.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:38.577+0000 7fe977fff640 1 -- 172.21.15.5:0/1028149334 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7fe988024020 con 0x5580d5d7c1c0 2024-05-09T00:56:38.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:38.577+0000 7fe977fff640 10 client.16518 handle_command_reply: tid=0 2024-05-09T00:56:38.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:38.577+0000 7fe9a5be6740 1 client.16518 shutdown 2024-05-09T00:56:38.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:38.577+0000 7fe9a5be6740 1 -- 172.21.15.5:0/1028149334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580d5dbf2f0 msgr2=0x5580d619c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:56:38.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:38.577+0000 7fe9a5be6740 1 --2- 172.21.15.5:0/1028149334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580d5dbf2f0 0x5580d619c970 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fe9900ca740 tx=0x7fe990003e70 comp rx=0 tx=0).stop 2024-05-09T00:56:38.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:38.577+0000 7fe9a5be6740 1 -- 172.21.15.5:0/1028149334 shutdown_connections 2024-05-09T00:56:38.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:38.577+0000 7fe9a5be6740 1 --2- 172.21.15.5:0/1028149334 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5580d5d7c1c0 0x5580d620cf10 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:38.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:38.577+0000 7fe9a5be6740 1 --2- 172.21.15.5:0/1028149334 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5580d5de81b0 0x5580d6199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:38.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:38.577+0000 7fe977fff640 0 client.16518 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:56:38.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:38.577+0000 7fe9a5be6740 1 --2- 172.21.15.5:0/1028149334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5580d5dc74d0 0x5580d61994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:38.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:38.577+0000 7fe9a5be6740 1 --2- 172.21.15.5:0/1028149334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580d5dbf2f0 0x5580d619c970 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:38.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:38.577+0000 7fe9a5be6740 1 -- 172.21.15.5:0/1028149334 >> 172.21.15.5:0/1028149334 conn(0x5580d5ddc300 msgr2=0x5580d6187530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:56:38.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:38.578+0000 7fe9a5be6740 1 -- 172.21.15.5:0/1028149334 shutdown_connections 2024-05-09T00:56:38.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:38.578+0000 7fe9a5be6740 1 -- 172.21.15.5:0/1028149334 wait complete. 2024-05-09T00:56:38.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:38.578+0000 7fe9a5be6740 20 client.16518 trim_cache size 0 max 16384 2024-05-09T00:56:38.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-09T00:56:38.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-09T00:56:38.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-09T00:56:38.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:38.579+0000 7fe99ec53640 1 -- 172.21.15.5:0/3649167732 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe9800b2930 msgr2=0x7fe9800b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:56:38.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:38.579+0000 7fe99ec53640 1 --2- 172.21.15.5:0/3649167732 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe9800b2930 0x7fe9800b4e40 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe98c004630 tx=0x7fe98c00b2e0 comp rx=0 tx=0).stop 2024-05-09T00:56:38.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:38.579+0000 7fe99ec53640 1 -- 172.21.15.5:0/3649167732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe99810b480 msgr2=0x7fe9981a92d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:56:38.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:38.579+0000 7fe99ec53640 1 --2- 172.21.15.5:0/3649167732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe99810b480 0x7fe9981a92d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe99002ae60 tx=0x7fe990032da0 comp rx=0 tx=0).stop 2024-05-09T00:56:38.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:38.579+0000 7fe99ec53640 1 -- 172.21.15.5:0/3649167732 shutdown_connections 2024-05-09T00:56:38.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:38.579+0000 7fe99ec53640 1 --2- 172.21.15.5:0/3649167732 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe9800b2930 0x7fe9800b4e40 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:38.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:38.579+0000 7fe99ec53640 1 --2- 172.21.15.5:0/3649167732 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9981ae790 0x7fe9981b0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:38.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:38.579+0000 7fe99ec53640 1 --2- 172.21.15.5:0/3649167732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9981101a0 0x7fe9981a9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:38.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:38.579+0000 7fe99ec53640 1 --2- 172.21.15.5:0/3649167732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe99810b480 0x7fe9981a92d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:38.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:38.579+0000 7fe99ec53640 1 -- 172.21.15.5:0/3649167732 >> 172.21.15.5:0/3649167732 conn(0x7fe998070670 msgr2=0x7fe99810c180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:56:38.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:38.580+0000 7fe99ec53640 1 -- 172.21.15.5:0/3649167732 shutdown_connections 2024-05-09T00:56:38.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:38.580+0000 7fe99ec53640 1 -- 172.21.15.5:0/3649167732 wait complete. 2024-05-09T00:56:40.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:56:39 smithi131 ceph-mon[34547]: pgmap v680: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 859 MiB/s wr, 2.16k op/s 2024-05-09T00:56:40.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:56:39 smithi157 ceph-mon[34294]: pgmap v680: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 859 MiB/s wr, 2.16k op/s 2024-05-09T00:56:40.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:56:39 smithi005 ceph-mon[30365]: pgmap v680: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 859 MiB/s wr, 2.16k op/s 2024-05-09T00:56:40.850 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-09T00:56:41.755 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:56:42.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:56:41 smithi131 ceph-mon[34547]: pgmap v681: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 962 MiB/s wr, 2.17k op/s 2024-05-09T00:56:42.112 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:56:41 smithi005 ceph-mon[30365]: pgmap v681: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 962 MiB/s wr, 2.17k op/s 2024-05-09T00:56:42.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:56:41 smithi157 ceph-mon[34294]: pgmap v681: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 962 MiB/s wr, 2.17k op/s 2024-05-09T00:56:44.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:56:43 smithi005 ceph-mon[30365]: pgmap v682: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 786 MiB/s wr, 1.68k op/s 2024-05-09T00:56:44.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:56:43 smithi131 ceph-mon[34547]: pgmap v682: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 786 MiB/s wr, 1.68k op/s 2024-05-09T00:56:44.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:56:43 smithi157 ceph-mon[34294]: pgmap v682: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 786 MiB/s wr, 1.68k op/s 2024-05-09T00:56:45.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:56:44 smithi157 ceph-mon[34294]: pgmap v683: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 876 MiB/s wr, 1.92k op/s 2024-05-09T00:56:45.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:56:44 smithi005 ceph-mon[30365]: pgmap v683: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 876 MiB/s wr, 1.92k op/s 2024-05-09T00:56:45.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:56:44 smithi131 ceph-mon[34547]: pgmap v683: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 876 MiB/s wr, 1.92k op/s 2024-05-09T00:56:47.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:56:47 smithi005 ceph-mon[30365]: pgmap v684: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 935 MiB/s wr, 2.05k op/s 2024-05-09T00:56:47.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:56:47 smithi157 ceph-mon[34294]: pgmap v684: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 935 MiB/s wr, 2.05k op/s 2024-05-09T00:56:47.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:56:47 smithi131 ceph-mon[34547]: pgmap v684: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 935 MiB/s wr, 2.05k op/s 2024-05-09T00:56:48.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.019+0000 7fa565865700 1 -- 172.21.15.5:0/1827303966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5580980c0 msgr2=0x7fa5580a0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:56:48.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.019+0000 7fa565865700 1 --2- 172.21.15.5:0/1827303966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5580980c0 0x7fa5580a0610 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fa54c009990 tx=0x7fa54c009ca0 comp rx=0 tx=0).stop 2024-05-09T00:56:48.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.020+0000 7fa565865700 1 -- 172.21.15.5:0/1827303966 shutdown_connections 2024-05-09T00:56:48.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.020+0000 7fa565865700 1 --2- 172.21.15.5:0/1827303966 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5580a0ba0 0x7fa5580a3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:48.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.020+0000 7fa565865700 1 --2- 172.21.15.5:0/1827303966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa558097770 0x7fa558097b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:48.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.020+0000 7fa565865700 1 --2- 172.21.15.5:0/1827303966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5580980c0 0x7fa5580a0610 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:48.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.020+0000 7fa565865700 1 -- 172.21.15.5:0/1827303966 >> 172.21.15.5:0/1827303966 conn(0x7fa55800b040 msgr2=0x7fa55800b480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:56:48.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.020+0000 7fa565865700 1 -- 172.21.15.5:0/1827303966 shutdown_connections 2024-05-09T00:56:48.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.021+0000 7fa565865700 1 -- 172.21.15.5:0/1827303966 wait complete. 2024-05-09T00:56:48.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.022+0000 7fa565865700 1 Processor -- start 2024-05-09T00:56:48.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.022+0000 7fa565865700 1 -- start start 2024-05-09T00:56:48.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.023+0000 7fa565865700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa558097770 0x7fa558095460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:48.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.023+0000 7fa565865700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5580980c0 0x7fa5580959a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:48.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.023+0000 7fa565865700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5580a0ba0 0x7fa558095ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:48.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.024+0000 7fa565865700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa5580a67e0 con 0x7fa5580980c0 2024-05-09T00:56:48.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.024+0000 7fa565865700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa5580a6660 con 0x7fa558097770 2024-05-09T00:56:48.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.024+0000 7fa565865700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa5580a6960 con 0x7fa5580a0ba0 2024-05-09T00:56:48.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.024+0000 7fa55ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5580980c0 0x7fa5580959a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:48.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.024+0000 7fa55ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5580980c0 0x7fa5580959a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36904/0 (socket says 172.21.15.5:36904) 2024-05-09T00:56:48.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.024+0000 7fa55ffff700 1 -- 172.21.15.5:0/1430296054 learned_addr learned my addr 172.21.15.5:0/1430296054 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:56:48.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.025+0000 7fa55ffff700 1 -- 172.21.15.5:0/1430296054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5580a0ba0 msgr2=0x7fa558095ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:56:48.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.025+0000 7fa55ffff700 1 --2- 172.21.15.5:0/1430296054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5580a0ba0 0x7fa558095ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:48.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.025+0000 7fa55ffff700 1 -- 172.21.15.5:0/1430296054 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa558097770 msgr2=0x7fa558095460 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T00:56:48.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.025+0000 7fa564863700 1 --2- 172.21.15.5:0/1430296054 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa558097770 0x7fa558095460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:48.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.026+0000 7fa55ffff700 1 --2- 172.21.15.5:0/1430296054 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa558097770 0x7fa558095460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:48.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.026+0000 7fa55ffff700 1 -- 172.21.15.5:0/1430296054 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa55813b590 con 0x7fa5580980c0 2024-05-09T00:56:48.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.026+0000 7fa55ffff700 1 --2- 172.21.15.5:0/1430296054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5580980c0 0x7fa5580959a0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fa54c005910 tx=0x7fa54c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:56:48.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.027+0000 7fa55dffb700 1 -- 172.21.15.5:0/1430296054 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54c027070 con 0x7fa5580980c0 2024-05-09T00:56:48.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.027+0000 7fa55dffb700 1 -- 172.21.15.5:0/1430296054 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa54c003fe0 con 0x7fa5580980c0 2024-05-09T00:56:48.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.027+0000 7fa565865700 1 -- 172.21.15.5:0/1430296054 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa55813b820 con 0x7fa5580980c0 2024-05-09T00:56:48.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.027+0000 7fa55dffb700 1 -- 172.21.15.5:0/1430296054 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54c022dd0 con 0x7fa5580980c0 2024-05-09T00:56:48.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.027+0000 7fa565865700 1 -- 172.21.15.5:0/1430296054 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa55813bcd0 con 0x7fa5580980c0 2024-05-09T00:56:48.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.041+0000 7fa5754ddb80 1 Processor -- start 2024-05-09T00:56:48.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.041+0000 7fa5754ddb80 1 -- start start 2024-05-09T00:56:48.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.043+0000 7fa5754ddb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a1367c160 0x561a13715740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:48.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.043+0000 7fa5754ddb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a135924a0 0x561a13715c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:48.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.043+0000 7fa5754ddb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a1370f800 0x561a1370fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:48.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.043+0000 7fa5754ddb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561a13631ff0 con 0x561a1370f800 2024-05-09T00:56:48.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.043+0000 7fa5754ddb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x561a135d07f0 con 0x561a1367c160 2024-05-09T00:56:48.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.043+0000 7fa5754ddb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x561a1362dfb0 con 0x561a135924a0 2024-05-09T00:56:48.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.044+0000 7fa55dffb700 1 -- 172.21.15.5:0/1430296054 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fa54c022500 con 0x7fa5580980c0 2024-05-09T00:56:48.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.047+0000 7fa564863700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a135924a0 0x561a13715c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:48.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.047+0000 7fa564863700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a135924a0 0x561a13715c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:37870/0 (socket says 172.21.15.5:37870) 2024-05-09T00:56:48.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.047+0000 7fa55dffb700 1 --2- 172.21.15.5:0/1430296054 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fa5500b2980 0x7fa5500b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:48.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.047+0000 7fa564863700 1 -- 172.21.15.5:0/570829277 learned_addr learned my addr 172.21.15.5:0/570829277 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:56:48.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.048+0000 7fa55dffb700 1 -- 172.21.15.5:0/1430296054 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fa54c0e3230 con 0x7fa5580980c0 2024-05-09T00:56:48.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.051+0000 7fa565064700 1 --2- 172.21.15.5:0/570829277 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a1367c160 0x561a13715740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:48.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.053+0000 7fa565064700 1 --2- 172.21.15.5:0/1430296054 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fa5500b2980 0x7fa5500b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:48.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.055+0000 7fa55ffff700 1 --2- 172.21.15.5:0/570829277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a1370f800 0x561a1370fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:48.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.056+0000 7fa565064700 1 -- 172.21.15.5:0/570829277 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a135924a0 msgr2=0x561a13715c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:56:48.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.056+0000 7fa565064700 1 --2- 172.21.15.5:0/570829277 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a135924a0 0x561a13715c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:48.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.056+0000 7fa565064700 1 -- 172.21.15.5:0/570829277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a1370f800 msgr2=0x561a1370fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:56:48.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.056+0000 7fa565064700 1 --2- 172.21.15.5:0/570829277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a1370f800 0x561a1370fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:48.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.056+0000 7fa565064700 1 -- 172.21.15.5:0/570829277 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a135c8370 con 0x561a1367c160 2024-05-09T00:56:48.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.061+0000 7fa565064700 1 --2- 172.21.15.5:0/1430296054 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fa5500b2980 0x7fa5500b4e90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa560069880 tx=0x7fa56007b000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:56:48.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.063+0000 7fa565064700 1 --2- 172.21.15.5:0/570829277 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a1367c160 0x561a13715740 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa56006a3c0 tx=0x7fa56006cc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=878130c3474e37e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:56:48.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.065+0000 7fa54affd700 1 -- 172.21.15.5:0/570829277 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa560092070 con 0x561a1367c160 2024-05-09T00:56:48.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.065+0000 7fa54affd700 1 -- 172.21.15.5:0/570829277 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa56006a930 con 0x561a1367c160 2024-05-09T00:56:48.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.067+0000 7fa5754ddb80 1 -- 172.21.15.5:0/570829277 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a1367c160 msgr2=0x561a13715740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:56:48.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.067+0000 7fa5754ddb80 1 --2- 172.21.15.5:0/570829277 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a1367c160 0x561a13715740 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa56006a3c0 tx=0x7fa56006cc30 comp rx=0 tx=0).stop 2024-05-09T00:56:48.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.077+0000 7fa5754ddb80 1 -- 172.21.15.5:0/570829277 shutdown_connections 2024-05-09T00:56:48.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.077+0000 7fa5754ddb80 1 --2- 172.21.15.5:0/570829277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a1370f800 0x561a1370fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:48.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.077+0000 7fa5754ddb80 1 --2- 172.21.15.5:0/570829277 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a1367c160 0x561a13715740 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:48.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.077+0000 7fa5754ddb80 1 --2- 172.21.15.5:0/570829277 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a135924a0 0x561a13715c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:48.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.077+0000 7fa5754ddb80 1 -- 172.21.15.5:0/570829277 >> 172.21.15.5:0/570829277 conn(0x561a135f26a0 msgr2=0x561a1370a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:56:48.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.078+0000 7fa5754ddb80 1 -- 172.21.15.5:0/570829277 shutdown_connections 2024-05-09T00:56:48.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.085+0000 7fa5754ddb80 1 -- 172.21.15.5:0/570829277 wait complete. 2024-05-09T00:56:48.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.086+0000 7fa5754ddb80 1 -- start start 2024-05-09T00:56:48.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.086+0000 7fa5754ddb80 1 Processor -- start 2024-05-09T00:56:48.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.089+0000 7fa5754ddb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a135924a0 0x561a13719360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:48.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.089+0000 7fa5754ddb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a1367c160 0x561a137198a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:48.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.090+0000 7fa5754ddb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a1370f800 0x561a13719de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:48.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.090+0000 7fa5754ddb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561a134adba0 con 0x561a1367c160 2024-05-09T00:56:48.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.090+0000 7fa5754ddb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x561a135ec150 con 0x561a1370f800 2024-05-09T00:56:48.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.090+0000 7fa5754ddb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x561a1371c420 con 0x561a135924a0 2024-05-09T00:56:48.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.090+0000 7fa5754ddb80 10 client.0 _ll_register_callbacks cb 0x561a134a36c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:56:48.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.094+0000 7fa564863700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a135924a0 0x561a13719360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:48.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.094+0000 7fa565064700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a1370f800 0x561a13719de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:48.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.095+0000 7fa565064700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a1370f800 0x561a13719de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:34168/0 (socket says 172.21.15.5:34168) 2024-05-09T00:56:48.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.095+0000 7fa565064700 1 -- 172.21.15.5:0/547033290 learned_addr learned my addr 172.21.15.5:0/547033290 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:56:48.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.098+0000 7fa55ffff700 1 --2- 172.21.15.5:0/547033290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a1367c160 0x561a137198a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:48.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.100+0000 7fa565064700 1 -- 172.21.15.5:0/547033290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a135924a0 msgr2=0x561a13719360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:56:48.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.100+0000 7fa565064700 1 --2- 172.21.15.5:0/547033290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a135924a0 0x561a13719360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:48.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.100+0000 7fa565064700 1 -- 172.21.15.5:0/547033290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a1367c160 msgr2=0x561a137198a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:56:48.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.100+0000 7fa565064700 1 --2- 172.21.15.5:0/547033290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a1367c160 0x561a137198a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:48.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.100+0000 7fa565064700 1 -- 172.21.15.5:0/547033290 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a134ad490 con 0x561a1370f800 2024-05-09T00:56:48.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.103+0000 7fa565064700 1 --2- 172.21.15.5:0/547033290 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a1370f800 0x561a13719de0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa560090040 tx=0x7fa560084d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18c9a868db78c46e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:56:48.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.105+0000 7fa53ffff700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-09T00:56:48.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.108+0000 7fa53ffff700 1 -- 172.21.15.5:0/547033290 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa560092020 con 0x561a1370f800 2024-05-09T00:56:48.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.108+0000 7fa53ffff700 1 -- 172.21.15.5:0/547033290 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fa56006c620 con 0x561a1370f800 2024-05-09T00:56:48.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.113+0000 7fa5754ddb80 1 -- 172.21.15.5:0/547033290 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa56006a930 con 0x561a1370f800 2024-05-09T00:56:48.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.118+0000 7fa53ffff700 1 -- 172.21.15.5:0/547033290 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa56006be60 con 0x561a1370f800 2024-05-09T00:56:48.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.121+0000 7fa5754ddb80 10 client.26033 fetch_fsmap learned FSMap version 31 2024-05-09T00:56:48.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.121+0000 7fa5754ddb80 1 -- 172.21.15.5:0/547033290 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561a137405e0 con 0x561a1370f800 2024-05-09T00:56:48.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.125+0000 7fa53ffff700 1 -- 172.21.15.5:0/547033290 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa56006a930 con 0x561a1370f800 2024-05-09T00:56:48.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.127+0000 7fa53ffff700 1 -- 172.21.15.5:0/547033290 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fa560071070 con 0x561a1370f800 2024-05-09T00:56:48.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.127+0000 7fa5754ddb80 10 client.26033 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:56:48.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.127+0000 7fa5754ddb80 10 client.26033 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T00:56:48.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.127+0000 7fa5754ddb80 20 client.26033 populate_metadata read hostname 'smithi005' 2024-05-09T00:56:48.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.127+0000 7fa5754ddb80 1 --2- 172.21.15.5:0/547033290 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x561a13743070 0x561a13745500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:48.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.128+0000 7fa5754ddb80 4 client.26033 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:56:48.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.128+0000 7fa5754ddb80 1 -- 172.21.15.5:0/547033290 --> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a1363f9d0 con 0x561a13743070 2024-05-09T00:56:48.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.138+0000 7fa564863700 1 --2- 172.21.15.5:0/547033290 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x561a13743070 0x561a13745500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:48.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.154+0000 7fa53ffff700 10 client.26033 ms_handle_connect on v2:172.21.15.157:6834/832898273 2024-05-09T00:56:48.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.154+0000 7fa564863700 1 --2- 172.21.15.5:0/547033290 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x561a13743070 0x561a13745500 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=300b71773e5f65fa server_cookie=c90430c6c02f0b8f in_seq=0 out_seq=0 2024-05-09T00:56:48.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.157+0000 7fa53ffff700 1 -- 172.21.15.5:0/547033290 <== mds.3 v2:172.21.15.157:6834/832898273 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561a1363f9d0 con 0x561a13743070 2024-05-09T00:56:48.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.157+0000 7fa53ffff700 10 client.26033 handle_command_reply: tid=0 2024-05-09T00:56:48.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.157+0000 7fa5754ddb80 1 client.26033 shutdown 2024-05-09T00:56:48.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.160+0000 7fa5754ddb80 1 -- 172.21.15.5:0/547033290 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a1370f800 msgr2=0x561a13719de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:56:48.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.160+0000 7fa5754ddb80 1 --2- 172.21.15.5:0/547033290 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a1370f800 0x561a13719de0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa560090040 tx=0x7fa560084d60 comp rx=0 tx=0).stop 2024-05-09T00:56:48.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.160+0000 7fa5754ddb80 1 -- 172.21.15.5:0/547033290 shutdown_connections 2024-05-09T00:56:48.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.161+0000 7fa5754ddb80 1 --2- 172.21.15.5:0/547033290 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x561a13743070 0x561a13745500 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:48.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.161+0000 7fa5754ddb80 1 --2- 172.21.15.5:0/547033290 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a1370f800 0x561a13719de0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:48.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.161+0000 7fa5754ddb80 1 --2- 172.21.15.5:0/547033290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a135924a0 0x561a13719360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:48.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.161+0000 7fa5754ddb80 1 --2- 172.21.15.5:0/547033290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a1367c160 0x561a137198a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:48.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.161+0000 7fa5754ddb80 1 -- 172.21.15.5:0/547033290 >> 172.21.15.5:0/547033290 conn(0x561a135f26a0 msgr2=0x561a1370a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:56:48.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.161+0000 7fa53ffff700 0 client.26033 ms_handle_reset on v2:172.21.15.157:6834/832898273 2024-05-09T00:56:48.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.161+0000 7fa5754ddb80 1 -- 172.21.15.5:0/547033290 shutdown_connections 2024-05-09T00:56:48.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.161+0000 7fa5754ddb80 1 -- 172.21.15.5:0/547033290 wait complete. 2024-05-09T00:56:48.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.161+0000 7fa5754ddb80 20 client.26033 trim_cache size 0 max 16384 2024-05-09T00:56:48.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.202+0000 7fa5754ddb80 1 Processor -- start 2024-05-09T00:56:48.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.202+0000 7fa5754ddb80 1 -- start start 2024-05-09T00:56:48.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.203+0000 7fa5754ddb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a135924a0 0x561a13716f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:48.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.204+0000 7fa5754ddb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a1367c160 0x561a137198d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:48.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.204+0000 7fa5754ddb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a1370f800 0x561a1374acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:48.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.205+0000 7fa565064700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a1370f800 0x561a1374acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:48.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.205+0000 7fa5754ddb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa55813eb50 con 0x561a1370f800 2024-05-09T00:56:48.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.205+0000 7fa5754ddb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa558140690 con 0x561a135924a0 2024-05-09T00:56:48.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.205+0000 7fa5754ddb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa55813f220 con 0x561a1367c160 2024-05-09T00:56:48.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.205+0000 7fa565064700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a1370f800 0x561a1374acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36938/0 (socket says 172.21.15.5:36938) 2024-05-09T00:56:48.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.205+0000 7fa565064700 1 -- 172.21.15.5:0/295392989 learned_addr learned my addr 172.21.15.5:0/295392989 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:56:48.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.206+0000 7fa565064700 1 -- 172.21.15.5:0/295392989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a1367c160 msgr2=0x561a137198d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:56:48.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.206+0000 7fa565064700 1 --2- 172.21.15.5:0/295392989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a1367c160 0x561a137198d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:48.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.206+0000 7fa565064700 1 -- 172.21.15.5:0/295392989 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a135924a0 msgr2=0x561a13716f20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:56:48.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.206+0000 7fa565064700 1 --2- 172.21.15.5:0/295392989 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a135924a0 0x561a13716f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:48.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.206+0000 7fa565064700 1 -- 172.21.15.5:0/295392989 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a137105a0 con 0x561a1370f800 2024-05-09T00:56:48.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.207+0000 7fa565064700 1 --2- 172.21.15.5:0/295392989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a1370f800 0x561a1374acc0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fa560052d10 tx=0x7fa56006cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58dbda123025e298 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:56:48.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.208+0000 7fa54a7fc700 1 -- 172.21.15.5:0/295392989 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa560092070 con 0x561a1370f800 2024-05-09T00:56:48.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.208+0000 7fa54a7fc700 1 -- 172.21.15.5:0/295392989 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa560084e80 con 0x561a1370f800 2024-05-09T00:56:48.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.210+0000 7fa5754ddb80 1 -- 172.21.15.5:0/295392989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a1370f800 msgr2=0x561a1374acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:56:48.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.210+0000 7fa5754ddb80 1 --2- 172.21.15.5:0/295392989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a1370f800 0x561a1374acc0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fa560052d10 tx=0x7fa56006cc30 comp rx=0 tx=0).stop 2024-05-09T00:56:48.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.210+0000 7fa5754ddb80 1 -- 172.21.15.5:0/295392989 shutdown_connections 2024-05-09T00:56:48.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.210+0000 7fa5754ddb80 1 --2- 172.21.15.5:0/295392989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a1370f800 0x561a1374acc0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:48.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.210+0000 7fa5754ddb80 1 --2- 172.21.15.5:0/295392989 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a135924a0 0x561a13716f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:48.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.210+0000 7fa5754ddb80 1 --2- 172.21.15.5:0/295392989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a1367c160 0x561a137198d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:48.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.210+0000 7fa5754ddb80 1 -- 172.21.15.5:0/295392989 >> 172.21.15.5:0/295392989 conn(0x561a135f26a0 msgr2=0x561a13717c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:56:48.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.212+0000 7fa5754ddb80 1 -- 172.21.15.5:0/295392989 shutdown_connections 2024-05-09T00:56:48.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.212+0000 7fa5754ddb80 1 -- 172.21.15.5:0/295392989 wait complete. 2024-05-09T00:56:48.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.214+0000 7fa5754ddb80 1 -- start start 2024-05-09T00:56:48.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.214+0000 7fa5754ddb80 1 Processor -- start 2024-05-09T00:56:48.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.216+0000 7fa5754ddb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a135924a0 0x561a13729f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:48.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.217+0000 7fa5754ddb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a1367c160 0x561a1372a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:48.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.217+0000 7fa5754ddb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a13719040 0x561a13719450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:48.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.217+0000 7fa5754ddb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa55813ffc0 con 0x561a135924a0 2024-05-09T00:56:48.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.217+0000 7fa5754ddb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa55813f8f0 con 0x561a1367c160 2024-05-09T00:56:48.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.217+0000 7fa5754ddb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa558130d00 con 0x561a13719040 2024-05-09T00:56:48.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.217+0000 7fa5754ddb80 10 client.0 _ll_register_callbacks cb 0x561a134a36c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:56:48.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.218+0000 7fa55ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a1367c160 0x561a1372a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:48.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.218+0000 7fa565064700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a13719040 0x561a13719450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:48.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.218+0000 7fa55ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a1367c160 0x561a1372a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:34186/0 (socket says 172.21.15.5:34186) 2024-05-09T00:56:48.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.218+0000 7fa55ffff700 1 -- 172.21.15.5:0/2300912087 learned_addr learned my addr 172.21.15.5:0/2300912087 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:56:48.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.218+0000 7fa564863700 1 --2- 172.21.15.5:0/2300912087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a135924a0 0x561a13729f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:48.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.220+0000 7fa564863700 1 -- 172.21.15.5:0/2300912087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a13719040 msgr2=0x561a13719450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:56:48.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.220+0000 7fa564863700 1 --2- 172.21.15.5:0/2300912087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a13719040 0x561a13719450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:48.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.220+0000 7fa564863700 1 -- 172.21.15.5:0/2300912087 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a1367c160 msgr2=0x561a1372a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:56:48.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.220+0000 7fa564863700 1 --2- 172.21.15.5:0/2300912087 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a1367c160 0x561a1372a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:48.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.220+0000 7fa564863700 1 -- 172.21.15.5:0/2300912087 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a13719990 con 0x561a135924a0 2024-05-09T00:56:48.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.220+0000 7fa564863700 1 --2- 172.21.15.5:0/2300912087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a135924a0 0x561a13729f00 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x561a13745c00 tx=0x7fa554015700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ae5e9396cce345d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:56:48.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.220+0000 7fa53ffff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:56:48.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.221+0000 7fa53ffff700 1 -- 172.21.15.5:0/2300912087 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa554020070 con 0x561a135924a0 2024-05-09T00:56:48.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.221+0000 7fa5754ddb80 1 -- 172.21.15.5:0/2300912087 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561a13639a70 con 0x561a135924a0 2024-05-09T00:56:48.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.221+0000 7fa53ffff700 1 -- 172.21.15.5:0/2300912087 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fa554005a50 con 0x561a135924a0 2024-05-09T00:56:48.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.222+0000 7fa53ffff700 1 -- 172.21.15.5:0/2300912087 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa55402b050 con 0x561a135924a0 2024-05-09T00:56:48.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.230+0000 7fa5754ddb80 10 client.16548 fetch_fsmap learned FSMap version 31 2024-05-09T00:56:48.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.230+0000 7fa5754ddb80 1 -- 172.21.15.5:0/2300912087 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561a137ac450 con 0x561a135924a0 2024-05-09T00:56:48.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.233+0000 7fa53ffff700 1 -- 172.21.15.5:0/2300912087 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x561a13639a70 con 0x561a135924a0 2024-05-09T00:56:48.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.233+0000 7fa53ffff700 1 -- 172.21.15.5:0/2300912087 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fa55401cce0 con 0x561a135924a0 2024-05-09T00:56:48.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.233+0000 7fa5754ddb80 10 client.16548 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:56:48.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.233+0000 7fa5754ddb80 10 client.16548 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T00:56:48.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.233+0000 7fa5754ddb80 20 client.16548 populate_metadata read hostname 'smithi005' 2024-05-09T00:56:48.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.234+0000 7fa5754ddb80 1 --2- 172.21.15.5:0/2300912087 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x561a137acb30 0x561a137aef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:48.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.234+0000 7fa5754ddb80 4 client.16548 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:56:48.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.234+0000 7fa5754ddb80 1 -- 172.21.15.5:0/2300912087 --> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561a134ad970 con 0x561a137acb30 2024-05-09T00:56:48.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.239+0000 7fa55ffff700 1 --2- 172.21.15.5:0/2300912087 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x561a137acb30 0x561a137aef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:48.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.240+0000 7fa55ffff700 1 --2- 172.21.15.5:0/2300912087 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x561a137acb30 0x561a137aef50 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c8bb30d6501cfe3f server_cookie=3eb22e87cd51ceec in_seq=0 out_seq=0 2024-05-09T00:56:48.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.240+0000 7fa53ffff700 10 client.16548 ms_handle_connect on v2:172.21.15.157:6834/832898273 2024-05-09T00:56:48.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.241+0000 7fa53ffff700 1 -- 172.21.15.5:0/2300912087 <== mds.3 v2:172.21.15.157:6834/832898273 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561a134ad970 con 0x561a137acb30 2024-05-09T00:56:48.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.241+0000 7fa53ffff700 10 client.16548 handle_command_reply: tid=0 2024-05-09T00:56:48.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.241+0000 7fa5754ddb80 1 client.16548 shutdown 2024-05-09T00:56:48.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.242+0000 7fa5754ddb80 1 -- 172.21.15.5:0/2300912087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a135924a0 msgr2=0x561a13729f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:56:48.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.242+0000 7fa5754ddb80 1 --2- 172.21.15.5:0/2300912087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a135924a0 0x561a13729f00 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x561a13745c00 tx=0x7fa554015700 comp rx=0 tx=0).stop 2024-05-09T00:56:48.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.242+0000 7fa5754ddb80 1 -- 172.21.15.5:0/2300912087 shutdown_connections 2024-05-09T00:56:48.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.242+0000 7fa5754ddb80 1 --2- 172.21.15.5:0/2300912087 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x561a137acb30 0x561a137aef50 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:48.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.242+0000 7fa5754ddb80 1 --2- 172.21.15.5:0/2300912087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a13719040 0x561a13719450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:48.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.242+0000 7fa5754ddb80 1 --2- 172.21.15.5:0/2300912087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a135924a0 0x561a13729f00 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:48.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.242+0000 7fa5754ddb80 1 --2- 172.21.15.5:0/2300912087 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561a1367c160 0x561a1372a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:48.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.242+0000 7fa5754ddb80 1 -- 172.21.15.5:0/2300912087 >> 172.21.15.5:0/2300912087 conn(0x561a135f26a0 msgr2=0x561a13714360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:56:48.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.242+0000 7fa53ffff700 0 client.16548 ms_handle_reset on v2:172.21.15.157:6834/832898273 2024-05-09T00:56:48.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.242+0000 7fa5754ddb80 1 -- 172.21.15.5:0/2300912087 shutdown_connections 2024-05-09T00:56:48.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.244+0000 7fa5754ddb80 1 -- 172.21.15.5:0/2300912087 wait complete. 2024-05-09T00:56:48.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.248+0000 7fa5754ddb80 20 client.16548 trim_cache size 0 max 16384 2024-05-09T00:56:48.254 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-09T00:56:48.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.252+0000 7fa565865700 1 -- 172.21.15.5:0/1430296054 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fa5500b2980 msgr2=0x7fa5500b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:56:48.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.252+0000 7fa565865700 1 --2- 172.21.15.5:0/1430296054 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fa5500b2980 0x7fa5500b4e90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa560069880 tx=0x7fa56007b000 comp rx=0 tx=0).stop 2024-05-09T00:56:48.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.252+0000 7fa565865700 1 -- 172.21.15.5:0/1430296054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5580980c0 msgr2=0x7fa5580959a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:56:48.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.252+0000 7fa565865700 1 --2- 172.21.15.5:0/1430296054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5580980c0 0x7fa5580959a0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fa54c005910 tx=0x7fa54c009f90 comp rx=0 tx=0).stop 2024-05-09T00:56:48.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.254+0000 7fa565865700 1 -- 172.21.15.5:0/1430296054 shutdown_connections 2024-05-09T00:56:48.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.254+0000 7fa565865700 1 --2- 172.21.15.5:0/1430296054 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fa5500b2980 0x7fa5500b4e90 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:48.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.254+0000 7fa565865700 1 --2- 172.21.15.5:0/1430296054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5580a0ba0 0x7fa558095ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:48.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.254+0000 7fa565865700 1 --2- 172.21.15.5:0/1430296054 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa558097770 0x7fa558095460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:48.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.254+0000 7fa565865700 1 --2- 172.21.15.5:0/1430296054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5580980c0 0x7fa5580959a0 secure :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fa54c005910 tx=0x7fa54c009f90 comp rx=0 tx=0).stop 2024-05-09T00:56:48.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.254+0000 7fa565865700 1 -- 172.21.15.5:0/1430296054 >> 172.21.15.5:0/1430296054 conn(0x7fa55800b040 msgr2=0x7fa5580933e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:56:48.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.257+0000 7fa565865700 1 -- 172.21.15.5:0/1430296054 shutdown_connections 2024-05-09T00:56:48.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:48.257+0000 7fa565865700 1 -- 172.21.15.5:0/1430296054 wait complete. 2024-05-09T00:56:49.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:56:49 smithi157 ceph-mon[34294]: pgmap v685: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 811 MiB/s wr, 1.70k op/s 2024-05-09T00:56:49.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:56:49 smithi005 ceph-mon[30365]: pgmap v685: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 811 MiB/s wr, 1.70k op/s 2024-05-09T00:56:49.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:56:49 smithi131 ceph-mon[34547]: pgmap v685: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 811 MiB/s wr, 1.70k op/s 2024-05-09T00:56:50.431 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T00:56:51.251 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:56:51.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:56:51 smithi157 ceph-mon[34294]: pgmap v686: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 938 MiB/s wr, 2.17k op/s 2024-05-09T00:56:51.750 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:56:51 smithi005 ceph-mon[30365]: pgmap v686: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 938 MiB/s wr, 2.17k op/s 2024-05-09T00:56:51.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.842+0000 7f2d0b0a6640 1 -- 172.21.15.5:0/2968810222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cfc0aad10 msgr2=0x7f2cfc0ad130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:56:51.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.842+0000 7f2d0b0a6640 1 --2- 172.21.15.5:0/2968810222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cfc0aad10 0x7f2cfc0ad130 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f2cf4009860 tx=0x7f2cf402a360 comp rx=0 tx=0).stop 2024-05-09T00:56:51.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.843+0000 7f2d0b0a6640 1 -- 172.21.15.5:0/2968810222 shutdown_connections 2024-05-09T00:56:51.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.843+0000 7f2d0b0a6640 1 --2- 172.21.15.5:0/2968810222 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2cfc0ad6c0 0x7f2cfc0afbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:51.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.843+0000 7f2d0b0a6640 1 --2- 172.21.15.5:0/2968810222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cfc0aad10 0x7f2cfc0ad130 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:51.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.843+0000 7f2d0b0a6640 1 --2- 172.21.15.5:0/2968810222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cfc0a5750 0x7f2cfc0a5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:51.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.843+0000 7f2d0b0a6640 1 -- 172.21.15.5:0/2968810222 >> 172.21.15.5:0/2968810222 conn(0x7f2cfc0a09b0 msgr2=0x7f2cfc0a2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:56:51.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.844+0000 7f2d0b0a6640 1 -- 172.21.15.5:0/2968810222 shutdown_connections 2024-05-09T00:56:51.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.844+0000 7f2d0b0a6640 1 -- 172.21.15.5:0/2968810222 wait complete. 2024-05-09T00:56:51.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.846+0000 7f2d0b0a6640 1 Processor -- start 2024-05-09T00:56:51.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.846+0000 7f2d0b0a6640 1 -- start start 2024-05-09T00:56:51.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.847+0000 7f2d0b0a6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cfc0a5750 0x7f2cfc144830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:51.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.848+0000 7f2d0b0a6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cfc0aad10 0x7f2cfc144d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:51.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.848+0000 7f2d0a0a4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cfc0a5750 0x7f2cfc144830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:51.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.848+0000 7f2d0b0a6640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2cfc0ad6c0 0x7f2cfc14bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:51.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.848+0000 7f2d0a0a4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cfc0a5750 0x7f2cfc144830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36972/0 (socket says 172.21.15.5:36972) 2024-05-09T00:56:51.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.848+0000 7f2d0a0a4640 1 -- 172.21.15.5:0/3809990125 learned_addr learned my addr 172.21.15.5:0/3809990125 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:56:51.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.849+0000 7f2d0b0a6640 1 -- 172.21.15.5:0/3809990125 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2cfc0b1390 con 0x7f2cfc0a5750 2024-05-09T00:56:51.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.849+0000 7f2d0b0a6640 1 -- 172.21.15.5:0/3809990125 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2cfc0b1210 con 0x7f2cfc0ad6c0 2024-05-09T00:56:51.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.849+0000 7f2d0b0a6640 1 -- 172.21.15.5:0/3809990125 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2cfc0b1510 con 0x7f2cfc0aad10 2024-05-09T00:56:51.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.850+0000 7f2d0a0a4640 1 -- 172.21.15.5:0/3809990125 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cfc0aad10 msgr2=0x7f2cfc144d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:56:51.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.850+0000 7f2d0a0a4640 1 --2- 172.21.15.5:0/3809990125 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cfc0aad10 0x7f2cfc144d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:51.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.851+0000 7f2d0a0a4640 1 -- 172.21.15.5:0/3809990125 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2cfc0ad6c0 msgr2=0x7f2cfc14bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:56:51.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.851+0000 7f2d0a0a4640 1 --2- 172.21.15.5:0/3809990125 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2cfc0ad6c0 0x7f2cfc14bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:51.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.851+0000 7f2d0a0a4640 1 -- 172.21.15.5:0/3809990125 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cfc14c2c0 con 0x7f2cfc0a5750 2024-05-09T00:56:51.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.852+0000 7f2d0a0a4640 1 --2- 172.21.15.5:0/3809990125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cfc0a5750 0x7f2cfc144830 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f2d04000b60 tx=0x7f2d04072cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:56:51.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.852+0000 7f2cfb7fe640 1 -- 172.21.15.5:0/3809990125 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d04079070 con 0x7f2cfc0a5750 2024-05-09T00:56:51.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:56:51 smithi131 ceph-mon[34547]: pgmap v686: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 938 MiB/s wr, 2.17k op/s 2024-05-09T00:56:51.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.853+0000 7f2cfb7fe640 1 -- 172.21.15.5:0/3809990125 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d04070800 con 0x7f2cfc0a5750 2024-05-09T00:56:51.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.854+0000 7f2d0b0a6640 1 -- 172.21.15.5:0/3809990125 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cfc14c5b0 con 0x7f2cfc0a5750 2024-05-09T00:56:51.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.854+0000 7f2cfb7fe640 1 -- 172.21.15.5:0/3809990125 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d04079070 con 0x7f2cfc0a5750 2024-05-09T00:56:51.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.854+0000 7f2d0b0a6640 1 -- 172.21.15.5:0/3809990125 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cfc14ca80 con 0x7f2cfc0a5750 2024-05-09T00:56:51.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.858+0000 7f2d131b6740 1 Processor -- start 2024-05-09T00:56:51.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.859+0000 7f2d131b6740 1 -- start start 2024-05-09T00:56:51.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.859+0000 7f2d131b6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c5d5471b0 0x560c5d8eb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:51.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.859+0000 7f2d131b6740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c5d51e2f0 0x560c5d8ebc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:51.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.859+0000 7f2d131b6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c5d5264d0 0x560c5d8f2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:51.861 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.859+0000 7f2d131b6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560c5d84f7c0 con 0x560c5d5264d0 2024-05-09T00:56:51.861 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.859+0000 7f2d131b6740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x560c5d7eb0e0 con 0x560c5d51e2f0 2024-05-09T00:56:51.861 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.859+0000 7f2d131b6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x560c5d84b780 con 0x560c5d5471b0 2024-05-09T00:56:51.861 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.860+0000 7f2d098a3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c5d5264d0 0x560c5d8f2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:51.861 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.860+0000 7f2d098a3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c5d5264d0 0x560c5d8f2b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36986/0 (socket says 172.21.15.5:36986) 2024-05-09T00:56:51.861 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.860+0000 7f2d098a3640 1 -- 172.21.15.5:0/1595894728 learned_addr learned my addr 172.21.15.5:0/1595894728 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:56:51.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.861+0000 7f2d098a3640 1 -- 172.21.15.5:0/1595894728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c5d5471b0 msgr2=0x560c5d8eb6c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:56:51.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.861+0000 7f2d098a3640 1 --2- 172.21.15.5:0/1595894728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c5d5471b0 0x560c5d8eb6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:51.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.861+0000 7f2d098a3640 1 -- 172.21.15.5:0/1595894728 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c5d51e2f0 msgr2=0x560c5d8ebc40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:56:51.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.861+0000 7f2d098a3640 1 --2- 172.21.15.5:0/1595894728 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c5d51e2f0 0x560c5d8ebc40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:51.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.861+0000 7f2d098a3640 1 -- 172.21.15.5:0/1595894728 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c5d8147c0 con 0x560c5d5264d0 2024-05-09T00:56:51.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.861+0000 7f2d098a3640 1 --2- 172.21.15.5:0/1595894728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c5d5264d0 0x560c5d8f2b40 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f2cf4035040 tx=0x7f2cf4030420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bfbaa61def0cb47 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:56:51.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.862+0000 7f2cf8bfe640 1 -- 172.21.15.5:0/1595894728 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf4037040 con 0x560c5d5264d0 2024-05-09T00:56:51.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.862+0000 7f2cf8bfe640 1 -- 172.21.15.5:0/1595894728 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2cf4040070 con 0x560c5d5264d0 2024-05-09T00:56:51.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.863+0000 7f2cf8bfe640 1 -- 172.21.15.5:0/1595894728 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf4030820 con 0x560c5d5264d0 2024-05-09T00:56:51.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.864+0000 7f2d131b6740 1 -- 172.21.15.5:0/1595894728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c5d5264d0 msgr2=0x560c5d8f2b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:56:51.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.864+0000 7f2d131b6740 1 --2- 172.21.15.5:0/1595894728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c5d5264d0 0x560c5d8f2b40 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f2cf4035040 tx=0x7f2cf4030420 comp rx=0 tx=0).stop 2024-05-09T00:56:51.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.864+0000 7f2d131b6740 1 -- 172.21.15.5:0/1595894728 shutdown_connections 2024-05-09T00:56:51.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.864+0000 7f2d131b6740 1 --2- 172.21.15.5:0/1595894728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c5d5264d0 0x560c5d8f2b40 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:51.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.864+0000 7f2d131b6740 1 --2- 172.21.15.5:0/1595894728 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c5d51e2f0 0x560c5d8ebc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:51.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.864+0000 7f2d131b6740 1 --2- 172.21.15.5:0/1595894728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c5d5471b0 0x560c5d8eb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:51.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.864+0000 7f2d131b6740 1 -- 172.21.15.5:0/1595894728 >> 172.21.15.5:0/1595894728 conn(0x560c5d53b300 msgr2=0x560c5d8e7050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:56:51.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.864+0000 7f2cfb7fe640 1 -- 172.21.15.5:0/3809990125 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f2d040709b0 con 0x7f2cfc0a5750 2024-05-09T00:56:51.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.865+0000 7f2d131b6740 1 -- 172.21.15.5:0/1595894728 shutdown_connections 2024-05-09T00:56:51.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.865+0000 7f2d131b6740 1 -- 172.21.15.5:0/1595894728 wait complete. 2024-05-09T00:56:51.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.865+0000 7f2d131b6740 1 -- start start 2024-05-09T00:56:51.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.865+0000 7f2d131b6740 1 Processor -- start 2024-05-09T00:56:51.868 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.866+0000 7f2d131b6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c5d5264d0 0x560c5d8f5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:51.868 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.866+0000 7f2d131b6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c5d5471b0 0x560c5d8f5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:51.868 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.866+0000 7f2d131b6740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c5d505d80 0x560c5d8fc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:51.868 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.866+0000 7f2d131b6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560c5d8f35d0 con 0x560c5d5264d0 2024-05-09T00:56:51.868 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.866+0000 7f2d131b6740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x560c5d7d69c0 con 0x560c5d505d80 2024-05-09T00:56:51.868 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.866+0000 7f2d131b6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x560c5d8f3750 con 0x560c5d5471b0 2024-05-09T00:56:51.868 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.866+0000 7f2d131b6740 10 client.0 _ll_register_callbacks cb 0x560c5d803020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:56:51.868 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.866+0000 7f2cfb7fe640 1 --2- 172.21.15.5:0/3809990125 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f2cf00b2910 0x7f2cf00b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:51.868 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.866+0000 7f2d0a8a5640 1 --2- 172.21.15.5:0/3809990125 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f2cf00b2910 0x7f2cf00b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:51.868 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.866+0000 7f2d0a8a5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c5d5264d0 0x560c5d8f5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:51.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.866+0000 7f2d0a8a5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c5d5264d0 0x560c5d8f5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36998/0 (socket says 172.21.15.5:36998) 2024-05-09T00:56:51.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.866+0000 7f2d0a8a5640 1 -- 172.21.15.5:0/1730259966 learned_addr learned my addr 172.21.15.5:0/1730259966 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:56:51.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.866+0000 7f2cfb7fe640 1 -- 172.21.15.5:0/3809990125 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f2d04145f80 con 0x7f2cfc0a5750 2024-05-09T00:56:51.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.866+0000 7f2d0a0a4640 1 --2- 172.21.15.5:0/1730259966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c5d5471b0 0x560c5d8f5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:51.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.866+0000 7f2d0a8a5640 1 -- 172.21.15.5:0/1730259966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c5d5471b0 msgr2=0x560c5d8f5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:56:51.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.866+0000 7f2d0a8a5640 1 --2- 172.21.15.5:0/1730259966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c5d5471b0 0x560c5d8f5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:51.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.866+0000 7f2d0a8a5640 1 -- 172.21.15.5:0/1730259966 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c5d505d80 msgr2=0x560c5d8fc9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:56:51.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.866+0000 7f2d0a8a5640 1 --2- 172.21.15.5:0/1730259966 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c5d505d80 0x560c5d8fc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:51.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.866+0000 7f2d0a8a5640 1 -- 172.21.15.5:0/1730259966 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c5d7fd1f0 con 0x560c5d5264d0 2024-05-09T00:56:51.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.867+0000 7f2d0a8a5640 1 --2- 172.21.15.5:0/1730259966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c5d5264d0 0x560c5d8f5530 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f2d00011f90 tx=0x7f2d000068b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e2e13a1ed5f4c17 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:56:51.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.867+0000 7f2cedffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:56:51.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.867+0000 7f2d0a8a5640 1 --2- 172.21.15.5:0/3809990125 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f2cf00b2910 0x7f2cf00b4e20 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2d000117a0 tx=0x7f2d0000c980 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:56:51.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.867+0000 7f2cedffb640 1 -- 172.21.15.5:0/1730259966 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0001f070 con 0x560c5d5264d0 2024-05-09T00:56:51.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.867+0000 7f2cedffb640 1 -- 172.21.15.5:0/1730259966 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f2d000060e0 con 0x560c5d5264d0 2024-05-09T00:56:51.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.867+0000 7f2d131b6740 1 -- 172.21.15.5:0/1730259966 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cf4040070 con 0x560c5d5264d0 2024-05-09T00:56:51.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.867+0000 7f2cedffb640 1 -- 172.21.15.5:0/1730259966 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2d00011280 con 0x560c5d5264d0 2024-05-09T00:56:51.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.867+0000 7f2d131b6740 10 client.16560 fetch_fsmap learned FSMap version 31 2024-05-09T00:56:51.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.867+0000 7f2d131b6740 1 -- 172.21.15.5:0/1730259966 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560c5d5b2790 con 0x560c5d5264d0 2024-05-09T00:56:51.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.868+0000 7f2cedffb640 1 -- 172.21.15.5:0/1730259966 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2cf4040070 con 0x560c5d5264d0 2024-05-09T00:56:51.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.869+0000 7f2cedffb640 1 -- 172.21.15.5:0/1730259966 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f2d0002d070 con 0x560c5d5264d0 2024-05-09T00:56:51.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.869+0000 7f2d131b6740 10 client.16560 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:56:51.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.869+0000 7f2d131b6740 10 client.16560 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:56:51.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.869+0000 7f2d131b6740 20 client.16560 populate_metadata read hostname 'smithi005' 2024-05-09T00:56:51.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.869+0000 7f2d131b6740 1 --2- 172.21.15.5:0/1730259966 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x560c5d4db1c0 0x560c5d902ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:51.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.869+0000 7f2d131b6740 4 client.16560 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:56:51.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.869+0000 7f2d131b6740 1 -- 172.21.15.5:0/1730259966 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c5d85d3d0 con 0x560c5d4db1c0 2024-05-09T00:56:51.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.871+0000 7f2d0a0a4640 1 --2- 172.21.15.5:0/1730259966 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x560c5d4db1c0 0x560c5d902ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:51.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.874+0000 7f2d0a0a4640 1 --2- 172.21.15.5:0/1730259966 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x560c5d4db1c0 0x560c5d902ff0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de8cbfb35acbb928 server_cookie=7e70f8005b944c2 in_seq=0 out_seq=0 2024-05-09T00:56:51.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.874+0000 7f2cedffb640 10 client.16560 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:56:51.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.881+0000 7f2cedffb640 1 -- 172.21.15.5:0/1730259966 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560c5d85d3d0 con 0x560c5d4db1c0 2024-05-09T00:56:51.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.881+0000 7f2cedffb640 10 client.16560 handle_command_reply: tid=0 2024-05-09T00:56:51.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.881+0000 7f2d131b6740 1 client.16560 shutdown 2024-05-09T00:56:51.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.882+0000 7f2d131b6740 1 -- 172.21.15.5:0/1730259966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c5d5264d0 msgr2=0x560c5d8f5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:56:51.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.882+0000 7f2d131b6740 1 --2- 172.21.15.5:0/1730259966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c5d5264d0 0x560c5d8f5530 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f2d00011f90 tx=0x7f2d000068b0 comp rx=0 tx=0).stop 2024-05-09T00:56:51.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.882+0000 7f2d131b6740 1 -- 172.21.15.5:0/1730259966 shutdown_connections 2024-05-09T00:56:51.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.882+0000 7f2d131b6740 1 --2- 172.21.15.5:0/1730259966 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x560c5d4db1c0 0x560c5d902ff0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:51.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.882+0000 7f2d131b6740 1 --2- 172.21.15.5:0/1730259966 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c5d505d80 0x560c5d8fc9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:51.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.882+0000 7f2d131b6740 1 --2- 172.21.15.5:0/1730259966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c5d5471b0 0x560c5d8f5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:51.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.882+0000 7f2cedffb640 0 client.16560 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:56:51.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.882+0000 7f2d131b6740 1 --2- 172.21.15.5:0/1730259966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c5d5264d0 0x560c5d8f5530 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:51.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.882+0000 7f2d131b6740 1 -- 172.21.15.5:0/1730259966 >> 172.21.15.5:0/1730259966 conn(0x560c5d53b300 msgr2=0x560c5d8e6290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:56:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.882+0000 7f2d131b6740 1 -- 172.21.15.5:0/1730259966 shutdown_connections 2024-05-09T00:56:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.882+0000 7f2d131b6740 1 -- 172.21.15.5:0/1730259966 wait complete. 2024-05-09T00:56:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.882+0000 7f2d131b6740 20 client.16560 trim_cache size 0 max 16384 2024-05-09T00:56:51.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.905+0000 7f2d131b6740 1 Processor -- start 2024-05-09T00:56:51.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.905+0000 7f2d131b6740 1 -- start start 2024-05-09T00:56:51.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.906+0000 7f2d131b6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c5d505d80 0x560c5d8e70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:51.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.906+0000 7f2d131b6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c5d5264d0 0x560c5d8f5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:51.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.906+0000 7f2d131b6740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c5d5471b0 0x560c5d8f5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:51.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.906+0000 7f2d131b6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2cfc155760 con 0x560c5d5264d0 2024-05-09T00:56:51.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.906+0000 7f2d131b6740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2cfc1572a0 con 0x560c5d5471b0 2024-05-09T00:56:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.906+0000 7f2d131b6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2cfc155e30 con 0x560c5d505d80 2024-05-09T00:56:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.906+0000 7f2d098a3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c5d5264d0 0x560c5d8f5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.906+0000 7f2d098a3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c5d5264d0 0x560c5d8f5100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37006/0 (socket says 172.21.15.5:37006) 2024-05-09T00:56:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.906+0000 7f2d098a3640 1 -- 172.21.15.5:0/1769056817 learned_addr learned my addr 172.21.15.5:0/1769056817 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:56:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.906+0000 7f2d098a3640 1 -- 172.21.15.5:0/1769056817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c5d505d80 msgr2=0x560c5d8e70d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:56:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.906+0000 7f2d098a3640 1 --2- 172.21.15.5:0/1769056817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c5d505d80 0x560c5d8e70d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.906+0000 7f2d098a3640 1 -- 172.21.15.5:0/1769056817 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c5d5471b0 msgr2=0x560c5d8f5640 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:56:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.906+0000 7f2d098a3640 1 --2- 172.21.15.5:0/1769056817 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c5d5471b0 0x560c5d8f5640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.906+0000 7f2d098a3640 1 -- 172.21.15.5:0/1769056817 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c5d6295f0 con 0x560c5d5264d0 2024-05-09T00:56:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.906+0000 7f2d098a3640 1 --2- 172.21.15.5:0/1769056817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c5d5264d0 0x560c5d8f5100 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f2cf4009780 tx=0x7f2cf402aa90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e96411bbac2944f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:56:51.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.907+0000 7f2ceffff640 1 -- 172.21.15.5:0/1769056817 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf4048070 con 0x560c5d5264d0 2024-05-09T00:56:51.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.907+0000 7f2ceffff640 1 -- 172.21.15.5:0/1769056817 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2cf4040040 con 0x560c5d5264d0 2024-05-09T00:56:51.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.907+0000 7f2ceffff640 1 -- 172.21.15.5:0/1769056817 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf4037070 con 0x560c5d5264d0 2024-05-09T00:56:51.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.908+0000 7f2d131b6740 1 -- 172.21.15.5:0/1769056817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c5d5264d0 msgr2=0x560c5d8f5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:56:51.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.908+0000 7f2d131b6740 1 --2- 172.21.15.5:0/1769056817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c5d5264d0 0x560c5d8f5100 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f2cf4009780 tx=0x7f2cf402aa90 comp rx=0 tx=0).stop 2024-05-09T00:56:51.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.908+0000 7f2d131b6740 1 -- 172.21.15.5:0/1769056817 shutdown_connections 2024-05-09T00:56:51.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.908+0000 7f2d131b6740 1 --2- 172.21.15.5:0/1769056817 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c5d5471b0 0x560c5d8f5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:51.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.908+0000 7f2d131b6740 1 --2- 172.21.15.5:0/1769056817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c5d5264d0 0x560c5d8f5100 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:51.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.908+0000 7f2d131b6740 1 --2- 172.21.15.5:0/1769056817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c5d505d80 0x560c5d8e70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:51.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.908+0000 7f2d131b6740 1 -- 172.21.15.5:0/1769056817 >> 172.21.15.5:0/1769056817 conn(0x560c5d53b300 msgr2=0x560c5d8e69c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:56:51.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.908+0000 7f2d131b6740 1 -- 172.21.15.5:0/1769056817 shutdown_connections 2024-05-09T00:56:51.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.908+0000 7f2d131b6740 1 -- 172.21.15.5:0/1769056817 wait complete. 2024-05-09T00:56:51.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.909+0000 7f2d131b6740 1 -- start start 2024-05-09T00:56:51.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.909+0000 7f2d131b6740 1 Processor -- start 2024-05-09T00:56:51.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.910+0000 7f2d131b6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c5d505d80 0x560c5d8f9790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:51.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.911+0000 7f2d131b6740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c5d5264d0 0x560c5d8f9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:51.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.911+0000 7f2d131b6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c5d5471b0 0x560c5d8fab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:51.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.912+0000 7f2d131b6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2cfc156bd0 con 0x560c5d5471b0 2024-05-09T00:56:51.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.912+0000 7f2d131b6740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2cfc156500 con 0x560c5d5264d0 2024-05-09T00:56:51.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.912+0000 7f2d131b6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2cfc153740 con 0x560c5d505d80 2024-05-09T00:56:51.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.912+0000 7f2d131b6740 10 client.0 _ll_register_callbacks cb 0x7f2d0000ce70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:56:51.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.912+0000 7f2d0a8a5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c5d5471b0 0x560c5d8fab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:51.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.912+0000 7f2d0a8a5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c5d5471b0 0x560c5d8fab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37010/0 (socket says 172.21.15.5:37010) 2024-05-09T00:56:51.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.912+0000 7f2d0a8a5640 1 -- 172.21.15.5:0/3510173080 learned_addr learned my addr 172.21.15.5:0/3510173080 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:56:51.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.912+0000 7f2d0a8a5640 1 -- 172.21.15.5:0/3510173080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c5d505d80 msgr2=0x560c5d8f9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:56:51.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.912+0000 7f2d0a8a5640 1 --2- 172.21.15.5:0/3510173080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c5d505d80 0x560c5d8f9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:51.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.912+0000 7f2d0a8a5640 1 -- 172.21.15.5:0/3510173080 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c5d5264d0 msgr2=0x560c5d8f9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:56:51.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.912+0000 7f2d0a8a5640 1 --2- 172.21.15.5:0/3510173080 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c5d5264d0 0x560c5d8f9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:51.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.912+0000 7f2d0a8a5640 1 -- 172.21.15.5:0/3510173080 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c5d813ed0 con 0x560c5d5471b0 2024-05-09T00:56:51.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.913+0000 7f2d0a8a5640 1 --2- 172.21.15.5:0/3510173080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c5d5471b0 0x560c5d8fab60 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f2d00006960 tx=0x7f2d00007e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cb468489d84e5cb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:56:51.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.913+0000 7f2cedffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:56:51.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.914+0000 7f2cedffb640 1 -- 172.21.15.5:0/3510173080 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0001f040 con 0x560c5d5471b0 2024-05-09T00:56:51.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.914+0000 7f2cedffb640 1 -- 172.21.15.5:0/3510173080 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f2d000062b0 con 0x560c5d5471b0 2024-05-09T00:56:51.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.914+0000 7f2d131b6740 1 -- 172.21.15.5:0/3510173080 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cf4040040 con 0x560c5d5471b0 2024-05-09T00:56:51.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.914+0000 7f2cedffb640 1 -- 172.21.15.5:0/3510173080 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2d00004850 con 0x560c5d5471b0 2024-05-09T00:56:51.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.914+0000 7f2d131b6740 10 client.16566 fetch_fsmap learned FSMap version 31 2024-05-09T00:56:51.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.914+0000 7f2d131b6740 1 -- 172.21.15.5:0/3510173080 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560c5d8fc280 con 0x560c5d5471b0 2024-05-09T00:56:51.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.915+0000 7f2cedffb640 1 -- 172.21.15.5:0/3510173080 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2cf4040040 con 0x560c5d5471b0 2024-05-09T00:56:51.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.915+0000 7f2cedffb640 1 -- 172.21.15.5:0/3510173080 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f2d0002d070 con 0x560c5d5471b0 2024-05-09T00:56:51.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.915+0000 7f2d131b6740 10 client.16566 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:56:51.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.915+0000 7f2d131b6740 10 client.16566 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:56:51.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.915+0000 7f2d131b6740 20 client.16566 populate_metadata read hostname 'smithi005' 2024-05-09T00:56:51.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.915+0000 7f2d131b6740 1 --2- 172.21.15.5:0/3510173080 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x560c5d96ba90 0x560c5d96df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:51.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.915+0000 7f2d131b6740 4 client.16566 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:56:51.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.915+0000 7f2d131b6740 1 -- 172.21.15.5:0/3510173080 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560c5d857240 con 0x560c5d96ba90 2024-05-09T00:56:51.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.927+0000 7f2d0a0a4640 1 --2- 172.21.15.5:0/3510173080 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x560c5d96ba90 0x560c5d96df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:51.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.931+0000 7f2d0a0a4640 1 --2- 172.21.15.5:0/3510173080 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x560c5d96ba90 0x560c5d96df20 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b2ae2edcc86c6e9 server_cookie=3c1a9c8f7dcf70ee in_seq=0 out_seq=0 2024-05-09T00:56:51.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:51.932+0000 7f2cedffb640 10 client.16566 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:56:53.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:56:53 smithi005 ceph-mon[30365]: pgmap v687: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 785 MiB/s wr, 2.04k op/s 2024-05-09T00:56:53.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:56:53 smithi131 ceph-mon[34547]: pgmap v687: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 785 MiB/s wr, 2.04k op/s 2024-05-09T00:56:53.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:56:53 smithi157 ceph-mon[34294]: pgmap v687: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 785 MiB/s wr, 2.04k op/s 2024-05-09T00:56:55.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:56:55 smithi005 ceph-mon[30365]: pgmap v688: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 878 MiB/s wr, 2.15k op/s 2024-05-09T00:56:55.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:56:55 smithi131 ceph-mon[34547]: pgmap v688: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 878 MiB/s wr, 2.15k op/s 2024-05-09T00:56:55.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:56:55 smithi157 ceph-mon[34294]: pgmap v688: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 878 MiB/s wr, 2.15k op/s 2024-05-09T00:56:57.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.035+0000 7f4b96a06700 1 -- 172.21.15.5:0/1015094296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b9011d930 msgr2=0x7f4b9011fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:56:57.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.035+0000 7f4b96a06700 1 --2- 172.21.15.5:0/1015094296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b9011d930 0x7f4b9011fd50 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f4b8800b0a0 tx=0x7f4b8800b3b0 comp rx=0 tx=0).stop 2024-05-09T00:56:57.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.036+0000 7f4b96a06700 1 -- 172.21.15.5:0/1015094296 shutdown_connections 2024-05-09T00:56:57.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.036+0000 7f4b96a06700 1 --2- 172.21.15.5:0/1015094296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b9011d930 0x7f4b9011fd50 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:57.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.036+0000 7f4b96a06700 1 --2- 172.21.15.5:0/1015094296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b9010a930 0x7f4b9010ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:57.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.036+0000 7f4b96a06700 1 --2- 172.21.15.5:0/1015094296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b9010b280 0x7f4b9011d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:57.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.036+0000 7f4b96a06700 1 -- 172.21.15.5:0/1015094296 >> 172.21.15.5:0/1015094296 conn(0x7f4b9006f4d0 msgr2=0x7f4b9006f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:56:57.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.037+0000 7f4b96a06700 1 -- 172.21.15.5:0/1015094296 shutdown_connections 2024-05-09T00:56:57.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.040+0000 7f4b96a06700 1 -- 172.21.15.5:0/1015094296 wait complete. 2024-05-09T00:56:57.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.041+0000 7f4b96a06700 1 Processor -- start 2024-05-09T00:56:57.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.041+0000 7f4b96a06700 1 -- start start 2024-05-09T00:56:57.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.042+0000 7f4b96a06700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b9010a930 0x7f4b9011d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:57.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.042+0000 7f4b96a06700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b9010b280 0x7f4b90117450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:57.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.042+0000 7f4b96a06700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b9011d930 0x7f4b90117aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:57.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.042+0000 7f4b96a06700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4b901134d0 con 0x7f4b9010b280 2024-05-09T00:56:57.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.042+0000 7f4b96a06700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4b90113350 con 0x7f4b9010a930 2024-05-09T00:56:57.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.042+0000 7f4b96a06700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4b90113650 con 0x7f4b9011d930 2024-05-09T00:56:57.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.043+0000 7f4b94fa3700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b9011d930 0x7f4b90117aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:57.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.044+0000 7f4b94fa3700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b9011d930 0x7f4b90117aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:56816/0 (socket says 172.21.15.5:56816) 2024-05-09T00:56:57.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.045+0000 7f4b94fa3700 1 -- 172.21.15.5:0/2311142927 learned_addr learned my addr 172.21.15.5:0/2311142927 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:56:57.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.046+0000 7f4b8ffff700 1 --2- 172.21.15.5:0/2311142927 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b9010a930 0x7f4b9011d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:57.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.047+0000 7f4b94fa3700 1 -- 172.21.15.5:0/2311142927 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b9010a930 msgr2=0x7f4b9011d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:56:57.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.047+0000 7f4b94fa3700 1 --2- 172.21.15.5:0/2311142927 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b9010a930 0x7f4b9011d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:57.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.047+0000 7f4b94fa3700 1 -- 172.21.15.5:0/2311142927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b9010b280 msgr2=0x7f4b90117450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:56:57.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.047+0000 7f4b94fa3700 1 --2- 172.21.15.5:0/2311142927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b9010b280 0x7f4b90117450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:57.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.047+0000 7f4b94fa3700 1 -- 172.21.15.5:0/2311142927 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b901be7a0 con 0x7f4b9011d930 2024-05-09T00:56:57.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.052+0000 7f4b94fa3700 1 --2- 172.21.15.5:0/2311142927 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b9011d930 0x7f4b90117aa0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4b88009fd0 tx=0x7f4b8800cf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:56:57.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.055+0000 7f4b8d7fa700 1 -- 172.21.15.5:0/2311142927 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b88021070 con 0x7f4b9011d930 2024-05-09T00:56:57.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.055+0000 7f4b96a06700 1 -- 172.21.15.5:0/2311142927 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b901be970 con 0x7f4b9011d930 2024-05-09T00:56:57.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.055+0000 7f4b96a06700 1 -- 172.21.15.5:0/2311142927 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b901bee20 con 0x7f4b9011d930 2024-05-09T00:56:57.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.055+0000 7f4b8d7fa700 1 -- 172.21.15.5:0/2311142927 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b88007db0 con 0x7f4b9011d930 2024-05-09T00:56:57.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.055+0000 7f4b8d7fa700 1 -- 172.21.15.5:0/2311142927 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b88005420 con 0x7f4b9011d930 2024-05-09T00:56:57.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.065+0000 7f4b8d7fa700 1 -- 172.21.15.5:0/2311142927 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f4b88024070 con 0x7f4b9011d930 2024-05-09T00:56:57.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.066+0000 7f4b8d7fa700 1 --2- 172.21.15.5:0/2311142927 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f4b7c0b29c0 0x7f4b7c0b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:57.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.066+0000 7f4b8d7fa700 1 -- 172.21.15.5:0/2311142927 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f4b880e32f0 con 0x7f4b9011d930 2024-05-09T00:56:57.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.070+0000 7f4b8ffff700 1 --2- 172.21.15.5:0/2311142927 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f4b7c0b29c0 0x7f4b7c0b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:57.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.075+0000 7f4b8ffff700 1 --2- 172.21.15.5:0/2311142927 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f4b7c0b29c0 0x7f4b7c0b4ed0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4b8000a850 tx=0x7f4b80008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:56:57.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.090+0000 7f4ba541cb80 1 Processor -- start 2024-05-09T00:56:57.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.090+0000 7f4ba541cb80 1 -- start start 2024-05-09T00:56:57.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.091+0000 7f4ba541cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e61acd160 0x562e61b664f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:57.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.091+0000 7f4ba541cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e619e34a0 0x562e61b66a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:57.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.091+0000 7f4ba541cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562e61b606a0 0x562e61b60ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:57.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.091+0000 7f4ba541cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562e61a82ff0 con 0x562e61acd160 2024-05-09T00:56:57.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.091+0000 7f4ba541cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x562e61a217f0 con 0x562e61b606a0 2024-05-09T00:56:57.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.091+0000 7f4ba541cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x562e61a7efb0 con 0x562e619e34a0 2024-05-09T00:56:57.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.092+0000 7f4b8ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e61acd160 0x562e61b664f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:57.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.092+0000 7f4b8ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e61acd160 0x562e61b664f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35216/0 (socket says 172.21.15.5:35216) 2024-05-09T00:56:57.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.092+0000 7f4b8ffff700 1 -- 172.21.15.5:0/2632669096 learned_addr learned my addr 172.21.15.5:0/2632669096 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:56:57.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.096+0000 7f4b8ffff700 1 -- 172.21.15.5:0/2632669096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e619e34a0 msgr2=0x562e61b66a30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:56:57.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.096+0000 7f4b94fa3700 1 --2- 172.21.15.5:0/2632669096 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562e61b606a0 0x562e61b60ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:57.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.096+0000 7f4b8ffff700 1 --2- 172.21.15.5:0/2632669096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e619e34a0 0x562e61b66a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:57.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.096+0000 7f4b8ffff700 1 -- 172.21.15.5:0/2632669096 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562e61b606a0 msgr2=0x562e61b60ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:56:57.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.096+0000 7f4b8ffff700 1 --2- 172.21.15.5:0/2632669096 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562e61b606a0 0x562e61b60ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:57.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.096+0000 7f4b8ffff700 1 -- 172.21.15.5:0/2632669096 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e61a19370 con 0x562e61acd160 2024-05-09T00:56:57.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.097+0000 7f4b8ffff700 1 --2- 172.21.15.5:0/2632669096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e61acd160 0x562e61b664f0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f4b800138a0 tx=0x7f4b8000f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f24657f5075b9678 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:56:57.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.100+0000 7f4b94fa3700 1 --2- 172.21.15.5:0/2632669096 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562e61b606a0 0x562e61b60ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:56:57.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.100+0000 7f4b867fc700 1 -- 172.21.15.5:0/2632669096 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b80004220 con 0x562e61acd160 2024-05-09T00:56:57.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.100+0000 7f4b867fc700 1 -- 172.21.15.5:0/2632669096 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b800043c0 con 0x562e61acd160 2024-05-09T00:56:57.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.100+0000 7f4b867fc700 1 -- 172.21.15.5:0/2632669096 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b80011bb0 con 0x562e61acd160 2024-05-09T00:56:57.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.102+0000 7f4ba541cb80 1 -- 172.21.15.5:0/2632669096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e61acd160 msgr2=0x562e61b664f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:56:57.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.102+0000 7f4ba541cb80 1 --2- 172.21.15.5:0/2632669096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e61acd160 0x562e61b664f0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f4b800138a0 tx=0x7f4b8000f000 comp rx=0 tx=0).stop 2024-05-09T00:56:57.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.107+0000 7f4ba541cb80 1 -- 172.21.15.5:0/2632669096 shutdown_connections 2024-05-09T00:56:57.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.107+0000 7f4ba541cb80 1 --2- 172.21.15.5:0/2632669096 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562e61b606a0 0x562e61b60ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:57.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.107+0000 7f4ba541cb80 1 --2- 172.21.15.5:0/2632669096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e61acd160 0x562e61b664f0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:57.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.107+0000 7f4ba541cb80 1 --2- 172.21.15.5:0/2632669096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e619e34a0 0x562e61b66a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:57.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.107+0000 7f4ba541cb80 1 -- 172.21.15.5:0/2632669096 >> 172.21.15.5:0/2632669096 conn(0x562e61a436a0 msgr2=0x562e61b5b400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:56:57.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.110+0000 7f4ba541cb80 1 -- 172.21.15.5:0/2632669096 shutdown_connections 2024-05-09T00:56:57.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.112+0000 7f4ba541cb80 1 -- 172.21.15.5:0/2632669096 wait complete. 2024-05-09T00:56:57.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.113+0000 7f4ba541cb80 1 -- start start 2024-05-09T00:56:57.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.113+0000 7f4ba541cb80 1 Processor -- start 2024-05-09T00:56:57.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.114+0000 7f4ba541cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562e619e34a0 0x562e61b6a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:57.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.114+0000 7f4ba541cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e61acd160 0x562e61b6a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:57.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.114+0000 7f4ba541cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e61b606a0 0x562e61b6ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:57.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.114+0000 7f4ba541cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562e618feba0 con 0x562e61acd160 2024-05-09T00:56:57.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.114+0000 7f4ba541cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x562e61a3d150 con 0x562e619e34a0 2024-05-09T00:56:57.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.114+0000 7f4ba541cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x562e61b6d1d0 con 0x562e61b606a0 2024-05-09T00:56:57.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.114+0000 7f4ba541cb80 10 client.0 _ll_register_callbacks cb 0x562e618f46c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:56:57.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.115+0000 7f4b8f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e61acd160 0x562e61b6a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:57.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.115+0000 7f4b8f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e61acd160 0x562e61b6a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35224/0 (socket says 172.21.15.5:35224) 2024-05-09T00:56:57.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.115+0000 7f4b8f7fe700 1 -- 172.21.15.5:0/3111108950 learned_addr learned my addr 172.21.15.5:0/3111108950 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:56:57.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.126+0000 7f4b94fa3700 1 --2- 172.21.15.5:0/3111108950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e61b606a0 0x562e61b6ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:57.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.126+0000 7f4b8ffff700 1 --2- 172.21.15.5:0/3111108950 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562e619e34a0 0x562e61b6a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:57.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.129+0000 7f4b94fa3700 1 -- 172.21.15.5:0/3111108950 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562e619e34a0 msgr2=0x562e61b6a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:56:57.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.129+0000 7f4b94fa3700 1 --2- 172.21.15.5:0/3111108950 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562e619e34a0 0x562e61b6a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:57.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.129+0000 7f4b94fa3700 1 -- 172.21.15.5:0/3111108950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e61acd160 msgr2=0x562e61b6a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:56:57.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.129+0000 7f4b94fa3700 1 --2- 172.21.15.5:0/3111108950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e61acd160 0x562e61b6a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:57.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.129+0000 7f4b94fa3700 1 -- 172.21.15.5:0/3111108950 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e618fe490 con 0x562e61b606a0 2024-05-09T00:56:57.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.134+0000 7f4b94fa3700 1 --2- 172.21.15.5:0/3111108950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e61b606a0 0x562e61b6ad00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x562e61b670c0 tx=0x7f4b8800b3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95f0b0f70223f1be server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:56:57.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.134+0000 7f4b6f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-09T00:56:57.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.135+0000 7f4b6f7fe700 1 -- 172.21.15.5:0/3111108950 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b8802b070 con 0x562e61b606a0 2024-05-09T00:56:57.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.136+0000 7f4ba541cb80 1 -- 172.21.15.5:0/3111108950 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b800043c0 con 0x562e61b606a0 2024-05-09T00:56:57.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.136+0000 7f4b6f7fe700 1 -- 172.21.15.5:0/3111108950 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f4b880b10b0 con 0x562e61b606a0 2024-05-09T00:56:57.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.138+0000 7f4b6f7fe700 1 -- 172.21.15.5:0/3111108950 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f4b880c6020 con 0x562e61b606a0 2024-05-09T00:56:57.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.138+0000 7f4ba541cb80 10 client.26050 fetch_fsmap learned FSMap version 31 2024-05-09T00:56:57.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.138+0000 7f4ba541cb80 1 -- 172.21.15.5:0/3111108950 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562e61b77c60 con 0x562e61b606a0 2024-05-09T00:56:57.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.142+0000 7f4b6f7fe700 1 -- 172.21.15.5:0/3111108950 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b800043c0 con 0x562e61b606a0 2024-05-09T00:56:57.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.143+0000 7f4b6f7fe700 1 -- 172.21.15.5:0/3111108950 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f4b88011070 con 0x562e61b606a0 2024-05-09T00:56:57.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.144+0000 7f4ba541cb80 10 client.26050 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:56:57.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.144+0000 7f4ba541cb80 10 client.26050 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-05-09T00:56:57.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.144+0000 7f4ba541cb80 20 client.26050 populate_metadata read hostname 'smithi005' 2024-05-09T00:56:57.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.144+0000 7f4ba541cb80 1 --2- 172.21.15.5:0/3111108950 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x562e61b7a460 0x562e61b7c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:57.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.144+0000 7f4ba541cb80 4 client.26050 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:56:57.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.144+0000 7f4ba541cb80 1 -- 172.21.15.5:0/3111108950 --> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e61a909d0 con 0x562e61b7a460 2024-05-09T00:56:57.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.146+0000 7f4b8ffff700 1 --2- 172.21.15.5:0/3111108950 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x562e61b7a460 0x562e61b7c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:57.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.154+0000 7f4b8ffff700 1 --2- 172.21.15.5:0/3111108950 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x562e61b7a460 0x562e61b7c8d0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d8cb6952c1b559dc server_cookie=f91f631b12cd06e in_seq=0 out_seq=0 2024-05-09T00:56:57.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.154+0000 7f4b6f7fe700 10 client.26050 ms_handle_connect on v2:172.21.15.131:6838/2748669631 2024-05-09T00:56:57.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.158+0000 7f4b6f7fe700 1 -- 172.21.15.5:0/3111108950 <== mds.2 v2:172.21.15.131:6838/2748669631 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562e61a909d0 con 0x562e61b7a460 2024-05-09T00:56:57.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.158+0000 7f4b6f7fe700 10 client.26050 handle_command_reply: tid=0 2024-05-09T00:56:57.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.158+0000 7f4ba541cb80 1 client.26050 shutdown 2024-05-09T00:56:57.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.161+0000 7f4ba541cb80 1 -- 172.21.15.5:0/3111108950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e61b606a0 msgr2=0x562e61b6ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:56:57.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.161+0000 7f4ba541cb80 1 --2- 172.21.15.5:0/3111108950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e61b606a0 0x562e61b6ad00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x562e61b670c0 tx=0x7f4b8800b3b0 comp rx=0 tx=0).stop 2024-05-09T00:56:57.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.161+0000 7f4ba541cb80 1 -- 172.21.15.5:0/3111108950 shutdown_connections 2024-05-09T00:56:57.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.161+0000 7f4ba541cb80 1 --2- 172.21.15.5:0/3111108950 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x562e61b7a460 0x562e61b7c8d0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:57.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.161+0000 7f4ba541cb80 1 --2- 172.21.15.5:0/3111108950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e61b606a0 0x562e61b6ad00 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:57.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.161+0000 7f4ba541cb80 1 --2- 172.21.15.5:0/3111108950 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562e619e34a0 0x562e61b6a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:57.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.161+0000 7f4ba541cb80 1 --2- 172.21.15.5:0/3111108950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e61acd160 0x562e61b6a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:57.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.161+0000 7f4ba541cb80 1 -- 172.21.15.5:0/3111108950 >> 172.21.15.5:0/3111108950 conn(0x562e61a436a0 msgr2=0x562e61b5b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:56:57.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.162+0000 7f4b6f7fe700 0 client.26050 ms_handle_reset on v2:172.21.15.131:6838/2748669631 2024-05-09T00:56:57.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.163+0000 7f4ba541cb80 1 -- 172.21.15.5:0/3111108950 shutdown_connections 2024-05-09T00:56:57.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.164+0000 7f4ba541cb80 1 -- 172.21.15.5:0/3111108950 wait complete. 2024-05-09T00:56:57.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.164+0000 7f4ba541cb80 20 client.26050 trim_cache size 0 max 16384 2024-05-09T00:56:57.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.206+0000 7f4ba541cb80 1 Processor -- start 2024-05-09T00:56:57.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.206+0000 7f4ba541cb80 1 -- start start 2024-05-09T00:56:57.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.208+0000 7f4ba541cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e619e34a0 0x562e61b65700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:57.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.208+0000 7f4ba541cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562e61acd160 0x562e61b6ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:57.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.208+0000 7f4ba541cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e61b606a0 0x562e61b6b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:57.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.208+0000 7f4b94fa3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e61b606a0 0x562e61b6b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:57.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.208+0000 7f4b94fa3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e61b606a0 0x562e61b6b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35234/0 (socket says 172.21.15.5:35234) 2024-05-09T00:56:57.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.208+0000 7f4b8ffff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e619e34a0 0x562e61b65700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:57.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.210+0000 7f4b8ffff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e619e34a0 0x562e61b65700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:56856/0 (socket says 172.21.15.5:56856) 2024-05-09T00:56:57.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.210+0000 7f4b94fa3700 1 -- 172.21.15.5:0/682119410 learned_addr learned my addr 172.21.15.5:0/682119410 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:56:57.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.211+0000 7f4ba541cb80 1 -- 172.21.15.5:0/682119410 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4b7000d3f0 con 0x562e61b606a0 2024-05-09T00:56:57.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.211+0000 7f4ba541cb80 1 -- 172.21.15.5:0/682119410 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4b7000ef30 con 0x562e61acd160 2024-05-09T00:56:57.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.211+0000 7f4ba541cb80 1 -- 172.21.15.5:0/682119410 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4b7000dac0 con 0x562e619e34a0 2024-05-09T00:56:57.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.211+0000 7f4b8f7fe700 1 --2- 172.21.15.5:0/682119410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562e61acd160 0x562e61b6ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:57.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.214+0000 7f4b8ffff700 1 -- 172.21.15.5:0/682119410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562e61acd160 msgr2=0x562e61b6ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:56:57.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.214+0000 7f4b8ffff700 1 --2- 172.21.15.5:0/682119410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562e61acd160 0x562e61b6ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:57.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.214+0000 7f4b8ffff700 1 -- 172.21.15.5:0/682119410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e61b606a0 msgr2=0x562e61b6b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:56:57.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.214+0000 7f4b8ffff700 1 --2- 172.21.15.5:0/682119410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e61b606a0 0x562e61b6b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:57.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.214+0000 7f4b8ffff700 1 -- 172.21.15.5:0/682119410 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e61b7cea0 con 0x562e619e34a0 2024-05-09T00:56:57.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.218+0000 7f4b8ffff700 1 --2- 172.21.15.5:0/682119410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e619e34a0 0x562e61b65700 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4b80006b20 tx=0x7f4b80006f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df6b34e87113ba5e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:56:57.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.220+0000 7f4b85ffb700 1 -- 172.21.15.5:0/682119410 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b80011980 con 0x562e619e34a0 2024-05-09T00:56:57.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.220+0000 7f4b85ffb700 1 -- 172.21.15.5:0/682119410 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b80004670 con 0x562e619e34a0 2024-05-09T00:56:57.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.221+0000 7f4ba541cb80 1 -- 172.21.15.5:0/682119410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e619e34a0 msgr2=0x562e61b65700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:56:57.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.221+0000 7f4ba541cb80 1 --2- 172.21.15.5:0/682119410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e619e34a0 0x562e61b65700 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4b80006b20 tx=0x7f4b80006f90 comp rx=0 tx=0).stop 2024-05-09T00:56:57.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.222+0000 7f4ba541cb80 1 -- 172.21.15.5:0/682119410 shutdown_connections 2024-05-09T00:56:57.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.222+0000 7f4ba541cb80 1 --2- 172.21.15.5:0/682119410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e61b606a0 0x562e61b6b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:57.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.222+0000 7f4ba541cb80 1 --2- 172.21.15.5:0/682119410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e619e34a0 0x562e61b65700 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:57.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.222+0000 7f4ba541cb80 1 --2- 172.21.15.5:0/682119410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562e61acd160 0x562e61b6ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:57.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.222+0000 7f4ba541cb80 1 -- 172.21.15.5:0/682119410 >> 172.21.15.5:0/682119410 conn(0x562e61a436a0 msgr2=0x562e61b68b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:56:57.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.223+0000 7f4ba541cb80 1 -- 172.21.15.5:0/682119410 shutdown_connections 2024-05-09T00:56:57.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.223+0000 7f4ba541cb80 1 -- 172.21.15.5:0/682119410 wait complete. 2024-05-09T00:56:57.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.224+0000 7f4ba541cb80 1 -- start start 2024-05-09T00:56:57.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.225+0000 7f4ba541cb80 1 Processor -- start 2024-05-09T00:56:57.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.226+0000 7f4ba541cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e619e34a0 0x562e61b72340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:57.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.226+0000 7f4ba541cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562e61acd160 0x562e61b72880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:57.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.226+0000 7f4ba541cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e61b606a0 0x562e61b81480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:57.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.226+0000 7f4ba541cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4b7000e860 con 0x562e619e34a0 2024-05-09T00:56:57.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.226+0000 7f4ba541cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4b7000e190 con 0x562e61acd160 2024-05-09T00:56:57.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.226+0000 7f4ba541cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4b7000b440 con 0x562e61b606a0 2024-05-09T00:56:57.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.226+0000 7f4ba541cb80 10 client.0 _ll_register_callbacks cb 0x562e618f46c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:56:57.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.227+0000 7f4b8ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e619e34a0 0x562e61b72340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:57.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.227+0000 7f4b8ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e619e34a0 0x562e61b72340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35246/0 (socket says 172.21.15.5:35246) 2024-05-09T00:56:57.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.227+0000 7f4b8ffff700 1 -- 172.21.15.5:0/123105205 learned_addr learned my addr 172.21.15.5:0/123105205 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:56:57.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.227+0000 7f4b94fa3700 1 --2- 172.21.15.5:0/123105205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e61b606a0 0x562e61b81480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:57.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.227+0000 7f4b8f7fe700 1 --2- 172.21.15.5:0/123105205 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562e61acd160 0x562e61b72880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:57.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.228+0000 7f4b8ffff700 1 -- 172.21.15.5:0/123105205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e61b606a0 msgr2=0x562e61b81480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:56:57.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.228+0000 7f4b8ffff700 1 --2- 172.21.15.5:0/123105205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e61b606a0 0x562e61b81480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:57.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.228+0000 7f4b8ffff700 1 -- 172.21.15.5:0/123105205 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562e61acd160 msgr2=0x562e61b72880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:56:57.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.228+0000 7f4b8ffff700 1 --2- 172.21.15.5:0/123105205 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562e61acd160 0x562e61b72880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:57.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.228+0000 7f4b8ffff700 1 -- 172.21.15.5:0/123105205 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e61b81ac0 con 0x562e619e34a0 2024-05-09T00:56:57.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.229+0000 7f4b8ffff700 1 --2- 172.21.15.5:0/123105205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e619e34a0 0x562e61b72340 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f4b80012040 tx=0x7f4b80003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f500304bead32b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:56:57.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.229+0000 7f4b6f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:56:57.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.229+0000 7f4b6f7fe700 1 -- 172.21.15.5:0/123105205 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b80011b60 con 0x562e619e34a0 2024-05-09T00:56:57.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.229+0000 7f4ba541cb80 1 -- 172.21.15.5:0/123105205 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562e61a8aa70 con 0x562e619e34a0 2024-05-09T00:56:57.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.230+0000 7f4b6f7fe700 1 -- 172.21.15.5:0/123105205 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f4b80030410 con 0x562e619e34a0 2024-05-09T00:56:57.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.230+0000 7f4b6f7fe700 1 -- 172.21.15.5:0/123105205 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f4b80004670 con 0x562e619e34a0 2024-05-09T00:56:57.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.231+0000 7f4ba541cb80 10 client.16581 fetch_fsmap learned FSMap version 31 2024-05-09T00:56:57.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.231+0000 7f4ba541cb80 1 -- 172.21.15.5:0/123105205 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562e61be46d0 con 0x562e619e34a0 2024-05-09T00:56:57.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.234+0000 7f4b6f7fe700 1 -- 172.21.15.5:0/123105205 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b80039480 con 0x562e619e34a0 2024-05-09T00:56:57.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.234+0000 7f4b6f7fe700 1 -- 172.21.15.5:0/123105205 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f4b80041030 con 0x562e619e34a0 2024-05-09T00:56:57.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.234+0000 7f4ba541cb80 10 client.16581 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:56:57.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.234+0000 7f4ba541cb80 10 client.16581 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-05-09T00:56:57.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.234+0000 7f4ba541cb80 20 client.16581 populate_metadata read hostname 'smithi005' 2024-05-09T00:56:57.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.234+0000 7f4ba541cb80 1 --2- 172.21.15.5:0/123105205 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x562e61be4d70 0x562e61be7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:56:57.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.235+0000 7f4ba541cb80 4 client.16581 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:56:57.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.235+0000 7f4ba541cb80 1 -- 172.21.15.5:0/123105205 --> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562e61a8aa70 con 0x562e61be4d70 2024-05-09T00:56:57.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.239+0000 7f4b8f7fe700 1 --2- 172.21.15.5:0/123105205 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x562e61be4d70 0x562e61be7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:56:57.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.245+0000 7f4b8f7fe700 1 --2- 172.21.15.5:0/123105205 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x562e61be4d70 0x562e61be7190 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=69f246559a1c4f server_cookie=cda518b56d616865 in_seq=0 out_seq=0 2024-05-09T00:56:57.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.246+0000 7f4b6f7fe700 10 client.16581 ms_handle_connect on v2:172.21.15.131:6838/2748669631 2024-05-09T00:56:57.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.246+0000 7f4b6f7fe700 1 -- 172.21.15.5:0/123105205 <== mds.2 v2:172.21.15.131:6838/2748669631 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562e61a8aa70 con 0x562e61be4d70 2024-05-09T00:56:57.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.246+0000 7f4b6f7fe700 10 client.16581 handle_command_reply: tid=0 2024-05-09T00:56:57.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.246+0000 7f4ba541cb80 1 client.16581 shutdown 2024-05-09T00:56:57.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.250+0000 7f4ba541cb80 1 -- 172.21.15.5:0/123105205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e619e34a0 msgr2=0x562e61b72340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:56:57.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.250+0000 7f4ba541cb80 1 --2- 172.21.15.5:0/123105205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e619e34a0 0x562e61b72340 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f4b80012040 tx=0x7f4b80003f40 comp rx=0 tx=0).stop 2024-05-09T00:56:57.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.251+0000 7f4ba541cb80 1 -- 172.21.15.5:0/123105205 shutdown_connections 2024-05-09T00:56:57.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.251+0000 7f4ba541cb80 1 --2- 172.21.15.5:0/123105205 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x562e61be4d70 0x562e61be7190 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:57.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.251+0000 7f4ba541cb80 1 --2- 172.21.15.5:0/123105205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e61b606a0 0x562e61b81480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:57.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.251+0000 7f4ba541cb80 1 --2- 172.21.15.5:0/123105205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e619e34a0 0x562e61b72340 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:57.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.251+0000 7f4ba541cb80 1 --2- 172.21.15.5:0/123105205 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562e61acd160 0x562e61b72880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:57.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.251+0000 7f4ba541cb80 1 -- 172.21.15.5:0/123105205 >> 172.21.15.5:0/123105205 conn(0x562e61a436a0 msgr2=0x562e61b6b8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:56:57.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.252+0000 7f4b6f7fe700 0 client.16581 ms_handle_reset on v2:172.21.15.131:6838/2748669631 2024-05-09T00:56:57.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.253+0000 7f4ba541cb80 1 -- 172.21.15.5:0/123105205 shutdown_connections 2024-05-09T00:56:57.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.255+0000 7f4ba541cb80 1 -- 172.21.15.5:0/123105205 wait complete. 2024-05-09T00:56:57.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.255+0000 7f4ba541cb80 20 client.16581 trim_cache size 0 max 16384 2024-05-09T00:56:57.261 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-09T00:56:57.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.264+0000 7f4b96a06700 1 -- 172.21.15.5:0/2311142927 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f4b7c0b29c0 msgr2=0x7f4b7c0b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:56:57.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.264+0000 7f4b96a06700 1 --2- 172.21.15.5:0/2311142927 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f4b7c0b29c0 0x7f4b7c0b4ed0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4b8000a850 tx=0x7f4b80008040 comp rx=0 tx=0).stop 2024-05-09T00:56:57.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.264+0000 7f4b96a06700 1 -- 172.21.15.5:0/2311142927 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b9011d930 msgr2=0x7f4b90117aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:56:57.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.264+0000 7f4b96a06700 1 --2- 172.21.15.5:0/2311142927 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b9011d930 0x7f4b90117aa0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4b88009fd0 tx=0x7f4b8800cf50 comp rx=0 tx=0).stop 2024-05-09T00:56:57.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.266+0000 7f4b96a06700 1 -- 172.21.15.5:0/2311142927 shutdown_connections 2024-05-09T00:56:57.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.266+0000 7f4b96a06700 1 --2- 172.21.15.5:0/2311142927 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f4b7c0b29c0 0x7f4b7c0b4ed0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:57.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.266+0000 7f4b96a06700 1 --2- 172.21.15.5:0/2311142927 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b9011d930 0x7f4b90117aa0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:57.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.266+0000 7f4b96a06700 1 --2- 172.21.15.5:0/2311142927 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b9010a930 0x7f4b9011d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:57.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.266+0000 7f4b96a06700 1 --2- 172.21.15.5:0/2311142927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b9010b280 0x7f4b90117450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:57.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.266+0000 7f4b96a06700 1 -- 172.21.15.5:0/2311142927 >> 172.21.15.5:0/2311142927 conn(0x7f4b9006f4d0 msgr2=0x7f4b9011df20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:56:57.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.266+0000 7f4b96a06700 1 -- 172.21.15.5:0/2311142927 shutdown_connections 2024-05-09T00:56:57.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:56:57.266+0000 7f4b96a06700 1 -- 172.21.15.5:0/2311142927 wait complete. 2024-05-09T00:56:57.697 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:56:57 smithi005 ceph-mon[30365]: pgmap v689: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 947 MiB/s wr, 2.20k op/s 2024-05-09T00:56:57.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:56:57 smithi131 ceph-mon[34547]: pgmap v689: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 947 MiB/s wr, 2.20k op/s 2024-05-09T00:56:57.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:56:57 smithi157 ceph-mon[34294]: pgmap v689: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 947 MiB/s wr, 2.20k op/s 2024-05-09T00:56:58.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:58.127+0000 7f2cedffb640 1 -- 172.21.15.5:0/3510173080 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x560c5d857240 con 0x560c5d96ba90 2024-05-09T00:56:58.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:58.127+0000 7f2cedffb640 10 client.16566 handle_command_reply: tid=0 2024-05-09T00:56:58.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:58.127+0000 7f2d131b6740 1 client.16566 shutdown 2024-05-09T00:56:58.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:58.127+0000 7f2d131b6740 1 -- 172.21.15.5:0/3510173080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c5d5471b0 msgr2=0x560c5d8fab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:56:58.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:58.127+0000 7f2d131b6740 1 --2- 172.21.15.5:0/3510173080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c5d5471b0 0x560c5d8fab60 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f2d00006960 tx=0x7f2d00007e30 comp rx=0 tx=0).stop 2024-05-09T00:56:58.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:58.129+0000 7f2d131b6740 1 -- 172.21.15.5:0/3510173080 shutdown_connections 2024-05-09T00:56:58.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:58.129+0000 7f2d131b6740 1 --2- 172.21.15.5:0/3510173080 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x560c5d96ba90 0x560c5d96df20 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:58.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:58.129+0000 7f2d131b6740 1 --2- 172.21.15.5:0/3510173080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c5d5471b0 0x560c5d8fab60 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:58.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:58.129+0000 7f2d131b6740 1 --2- 172.21.15.5:0/3510173080 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c5d5264d0 0x560c5d8f9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:58.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:58.129+0000 7f2d131b6740 1 --2- 172.21.15.5:0/3510173080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c5d505d80 0x560c5d8f9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:58.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:58.129+0000 7f2d131b6740 1 -- 172.21.15.5:0/3510173080 >> 172.21.15.5:0/3510173080 conn(0x560c5d53b300 msgr2=0x560c5d8e9e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:56:58.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:58.129+0000 7f2cedffb640 0 client.16566 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:56:58.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:58.129+0000 7f2d131b6740 1 -- 172.21.15.5:0/3510173080 shutdown_connections 2024-05-09T00:56:58.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:58.129+0000 7f2d131b6740 1 -- 172.21.15.5:0/3510173080 wait complete. 2024-05-09T00:56:58.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:58.130+0000 7f2d131b6740 20 client.16566 trim_cache size 0 max 16384 2024-05-09T00:56:58.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-09T00:56:58.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-09T00:56:58.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-09T00:56:58.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:58.141+0000 7f2d0b0a6640 1 -- 172.21.15.5:0/3809990125 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f2cf00b2910 msgr2=0x7f2cf00b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:56:58.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:58.141+0000 7f2d0b0a6640 1 --2- 172.21.15.5:0/3809990125 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f2cf00b2910 0x7f2cf00b4e20 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2d000117a0 tx=0x7f2d0000c980 comp rx=0 tx=0).stop 2024-05-09T00:56:58.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:58.141+0000 7f2d0b0a6640 1 -- 172.21.15.5:0/3809990125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cfc0a5750 msgr2=0x7f2cfc144830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:56:58.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:58.141+0000 7f2d0b0a6640 1 --2- 172.21.15.5:0/3809990125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cfc0a5750 0x7f2cfc144830 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f2d04000b60 tx=0x7f2d04072cc0 comp rx=0 tx=0).stop 2024-05-09T00:56:58.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:58.146+0000 7f2d0b0a6640 1 -- 172.21.15.5:0/3809990125 shutdown_connections 2024-05-09T00:56:58.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:58.146+0000 7f2d0b0a6640 1 --2- 172.21.15.5:0/3809990125 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f2cf00b2910 0x7f2cf00b4e20 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:58.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:58.146+0000 7f2d0b0a6640 1 --2- 172.21.15.5:0/3809990125 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2cfc0ad6c0 0x7f2cfc14bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:58.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:58.147+0000 7f2d0b0a6640 1 --2- 172.21.15.5:0/3809990125 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cfc0aad10 0x7f2cfc144d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:58.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:58.147+0000 7f2d0b0a6640 1 --2- 172.21.15.5:0/3809990125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cfc0a5750 0x7f2cfc144830 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:56:58.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:58.147+0000 7f2d0b0a6640 1 -- 172.21.15.5:0/3809990125 >> 172.21.15.5:0/3809990125 conn(0x7f2cfc0a09b0 msgr2=0x7f2cfc0abc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:56:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:58.149+0000 7f2d0b0a6640 1 -- 172.21.15.5:0/3809990125 shutdown_connections 2024-05-09T00:56:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:56:58.150+0000 7f2d0b0a6640 1 -- 172.21.15.5:0/3809990125 wait complete. 2024-05-09T00:56:59.128 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T00:56:59.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:56:59 smithi005 ceph-mon[30365]: pgmap v690: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 821 MiB/s wr, 1.93k op/s 2024-05-09T00:56:59.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:56:59 smithi131 ceph-mon[34547]: pgmap v690: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 821 MiB/s wr, 1.93k op/s 2024-05-09T00:56:59.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:56:59 smithi157 ceph-mon[34294]: pgmap v690: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 821 MiB/s wr, 1.93k op/s 2024-05-09T00:57:00.043 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:57:01.546 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:57:01 smithi005 ceph-mon[30365]: pgmap v691: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 987 MiB/s wr, 2.39k op/s 2024-05-09T00:57:01.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:57:01 smithi131 ceph-mon[34547]: pgmap v691: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 987 MiB/s wr, 2.39k op/s 2024-05-09T00:57:01.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:57:01 smithi157 ceph-mon[34294]: pgmap v691: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 987 MiB/s wr, 2.39k op/s 2024-05-09T00:57:03.753 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:57:03 smithi005 ceph-mon[30365]: pgmap v692: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 794 MiB/s wr, 1.77k op/s 2024-05-09T00:57:03.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:57:03 smithi131 ceph-mon[34547]: pgmap v692: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 794 MiB/s wr, 1.77k op/s 2024-05-09T00:57:03.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:57:03 smithi157 ceph-mon[34294]: pgmap v692: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 794 MiB/s wr, 1.77k op/s 2024-05-09T00:57:05.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.022+0000 7f8d9814e700 1 -- 172.21.15.5:0/3292595998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d901054e0 msgr2=0x7f8d901058f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:05.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.022+0000 7f8d9814e700 1 --2- 172.21.15.5:0/3292595998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d901054e0 0x7f8d901058f0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c009990 tx=0x7f8d8c009ca0 comp rx=0 tx=0).stop 2024-05-09T00:57:05.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.022+0000 7f8d9814e700 1 -- 172.21.15.5:0/3292595998 shutdown_connections 2024-05-09T00:57:05.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.022+0000 7f8d9814e700 1 --2- 172.21.15.5:0/3292595998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d9010e910 0x7f8d90110df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:05.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.022+0000 7f8d9814e700 1 --2- 172.21.15.5:0/3292595998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d901054e0 0x7f8d901058f0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:05.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.022+0000 7f8d9814e700 1 --2- 172.21.15.5:0/3292595998 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d90105e30 0x7f8d9010e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:05.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.022+0000 7f8d9814e700 1 -- 172.21.15.5:0/3292595998 >> 172.21.15.5:0/3292595998 conn(0x7f8d900fedc0 msgr2=0x7f8d901011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:57:05.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.022+0000 7f8d9814e700 1 -- 172.21.15.5:0/3292595998 shutdown_connections 2024-05-09T00:57:05.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.023+0000 7f8d9814e700 1 -- 172.21.15.5:0/3292595998 wait complete. 2024-05-09T00:57:05.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.024+0000 7f8d9814e700 1 Processor -- start 2024-05-09T00:57:05.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.024+0000 7f8d9814e700 1 -- start start 2024-05-09T00:57:05.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.024+0000 7f8d9814e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d901054e0 0x7f8d90104d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:05.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.024+0000 7f8d9814e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d90105e30 0x7f8d901031d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:05.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.024+0000 7f8d9814e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d9010e910 0x7f8d901037b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:05.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.024+0000 7f8d956e9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d90105e30 0x7f8d901031d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:05.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.024+0000 7f8d9814e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8d901145a0 con 0x7f8d90105e30 2024-05-09T00:57:05.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.024+0000 7f8d9814e700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8d90114420 con 0x7f8d901054e0 2024-05-09T00:57:05.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.024+0000 7f8d9814e700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8d90114720 con 0x7f8d9010e910 2024-05-09T00:57:05.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.025+0000 7f8d956e9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d90105e30 0x7f8d901031d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50662/0 (socket says 172.21.15.5:50662) 2024-05-09T00:57:05.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.025+0000 7f8d956e9700 1 -- 172.21.15.5:0/3468633712 learned_addr learned my addr 172.21.15.5:0/3468633712 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:57:05.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.025+0000 7f8d966eb700 1 --2- 172.21.15.5:0/3468633712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d9010e910 0x7f8d901037b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:05.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.025+0000 7f8d956e9700 1 -- 172.21.15.5:0/3468633712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d9010e910 msgr2=0x7f8d901037b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:05.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.025+0000 7f8d956e9700 1 --2- 172.21.15.5:0/3468633712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d9010e910 0x7f8d901037b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:05.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.025+0000 7f8d956e9700 1 -- 172.21.15.5:0/3468633712 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d901054e0 msgr2=0x7f8d90104d70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T00:57:05.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.025+0000 7f8d956e9700 1 --2- 172.21.15.5:0/3468633712 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d901054e0 0x7f8d90104d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:05.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.025+0000 7f8d956e9700 1 -- 172.21.15.5:0/3468633712 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d90103fe0 con 0x7f8d90105e30 2024-05-09T00:57:05.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.026+0000 7f8d956e9700 1 --2- 172.21.15.5:0/3468633712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d90105e30 0x7f8d901031d0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f8d8000b530 tx=0x7f8d8000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:57:05.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.028+0000 7f8d86ffd700 1 -- 172.21.15.5:0/3468633712 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d80015020 con 0x7f8d90105e30 2024-05-09T00:57:05.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.028+0000 7f8d86ffd700 1 -- 172.21.15.5:0/3468633712 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d80011ce0 con 0x7f8d90105e30 2024-05-09T00:57:05.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.028+0000 7f8d86ffd700 1 -- 172.21.15.5:0/3468633712 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d80010d20 con 0x7f8d90105e30 2024-05-09T00:57:05.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.029+0000 7f8d9814e700 1 -- 172.21.15.5:0/3468633712 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d901adaa0 con 0x7f8d90105e30 2024-05-09T00:57:05.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.029+0000 7f8d9814e700 1 -- 172.21.15.5:0/3468633712 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d901adf60 con 0x7f8d90105e30 2024-05-09T00:57:05.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.035+0000 7f8da6b64b80 1 Processor -- start 2024-05-09T00:57:05.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.035+0000 7f8da6b64b80 1 -- start start 2024-05-09T00:57:05.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.036+0000 7f8da6b64b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c77c40160 0x558c77cd94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:05.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.036+0000 7f8da6b64b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558c77b564a0 0x558c77cd9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:05.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.036+0000 7f8da6b64b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c77cd36a0 0x558c77cd3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:05.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.036+0000 7f8da6b64b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558c77bf5ff0 con 0x558c77c40160 2024-05-09T00:57:05.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.036+0000 7f8d966eb700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c77c40160 0x558c77cd94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:05.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.036+0000 7f8d966eb700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c77c40160 0x558c77cd94f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50664/0 (socket says 172.21.15.5:50664) 2024-05-09T00:57:05.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.036+0000 7f8d966eb700 1 -- 172.21.15.5:0/2501181240 learned_addr learned my addr 172.21.15.5:0/2501181240 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:57:05.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.036+0000 7f8da6b64b80 1 -- 172.21.15.5:0/2501181240 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x558c77b947f0 con 0x558c77b564a0 2024-05-09T00:57:05.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.036+0000 7f8da6b64b80 1 -- 172.21.15.5:0/2501181240 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x558c77bf1fb0 con 0x558c77cd36a0 2024-05-09T00:57:05.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.037+0000 7f8d95eea700 1 --2- 172.21.15.5:0/2501181240 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558c77b564a0 0x558c77cd9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:05.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.037+0000 7f8d966eb700 1 -- 172.21.15.5:0/2501181240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c77cd36a0 msgr2=0x558c77cd3ad0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:57:05.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.037+0000 7f8d966eb700 1 --2- 172.21.15.5:0/2501181240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c77cd36a0 0x558c77cd3ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:05.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.037+0000 7f8d966eb700 1 -- 172.21.15.5:0/2501181240 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558c77b564a0 msgr2=0x558c77cd9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:57:05.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.037+0000 7f8d966eb700 1 --2- 172.21.15.5:0/2501181240 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558c77b564a0 0x558c77cd9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:05.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.037+0000 7f8d966eb700 1 -- 172.21.15.5:0/2501181240 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c77b8c370 con 0x558c77c40160 2024-05-09T00:57:05.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.037+0000 7f8d86ffd700 1 -- 172.21.15.5:0/3468633712 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f8d80018020 con 0x7f8d90105e30 2024-05-09T00:57:05.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.037+0000 7f8d966eb700 1 --2- 172.21.15.5:0/2501181240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c77c40160 0x558c77cd94f0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f8d88009f60 tx=0x7f8d88009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a997e47e3fd74192 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:57:05.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.038+0000 7f8d7bfff700 1 -- 172.21.15.5:0/2501181240 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d8801a070 con 0x558c77c40160 2024-05-09T00:57:05.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.038+0000 7f8d7bfff700 1 -- 172.21.15.5:0/2501181240 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d8800c0d0 con 0x558c77c40160 2024-05-09T00:57:05.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.038+0000 7f8d7bfff700 1 -- 172.21.15.5:0/2501181240 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d88015d20 con 0x558c77c40160 2024-05-09T00:57:05.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.039+0000 7f8da6b64b80 1 -- 172.21.15.5:0/2501181240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c77c40160 msgr2=0x558c77cd94f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:05.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.039+0000 7f8da6b64b80 1 --2- 172.21.15.5:0/2501181240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c77c40160 0x558c77cd94f0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f8d88009f60 tx=0x7f8d88009f90 comp rx=0 tx=0).stop 2024-05-09T00:57:05.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.039+0000 7f8da6b64b80 1 -- 172.21.15.5:0/2501181240 shutdown_connections 2024-05-09T00:57:05.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.039+0000 7f8da6b64b80 1 --2- 172.21.15.5:0/2501181240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c77cd36a0 0x558c77cd3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:05.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.039+0000 7f8da6b64b80 1 --2- 172.21.15.5:0/2501181240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c77c40160 0x558c77cd94f0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:05.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.039+0000 7f8da6b64b80 1 --2- 172.21.15.5:0/2501181240 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558c77b564a0 0x558c77cd9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:05.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.039+0000 7f8da6b64b80 1 -- 172.21.15.5:0/2501181240 >> 172.21.15.5:0/2501181240 conn(0x558c77bb66a0 msgr2=0x558c77cce400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:57:05.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.039+0000 7f8d86ffd700 1 --2- 172.21.15.5:0/3468633712 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f8d7c0b2900 0x7f8d7c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:05.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.040+0000 7f8da6b64b80 1 -- 172.21.15.5:0/2501181240 shutdown_connections 2024-05-09T00:57:05.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.040+0000 7f8d95eea700 1 --2- 172.21.15.5:0/3468633712 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f8d7c0b2900 0x7f8d7c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:05.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.040+0000 7f8d86ffd700 1 -- 172.21.15.5:0/3468633712 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f8d800e1990 con 0x7f8d90105e30 2024-05-09T00:57:05.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.040+0000 7f8da6b64b80 1 -- 172.21.15.5:0/2501181240 wait complete. 2024-05-09T00:57:05.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.040+0000 7f8da6b64b80 1 -- start start 2024-05-09T00:57:05.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.041+0000 7f8da6b64b80 1 Processor -- start 2024-05-09T00:57:05.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.041+0000 7f8d95eea700 1 --2- 172.21.15.5:0/3468633712 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f8d7c0b2900 0x7f8d7c0b4e10 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c00b5c0 tx=0x7f8d8c019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:57:05.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.042+0000 7f8da6b64b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c77b564a0 0x558c77cdd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:05.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.042+0000 7f8da6b64b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558c77c40160 0x558c77cdd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:05.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.042+0000 7f8da6b64b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c77cd36a0 0x558c77cddd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:05.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.042+0000 7f8da6b64b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558c77a71ba0 con 0x558c77b564a0 2024-05-09T00:57:05.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.042+0000 7f8da6b64b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x558c77bb0150 con 0x558c77c40160 2024-05-09T00:57:05.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.042+0000 7f8da6b64b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x558c77ce01d0 con 0x558c77cd36a0 2024-05-09T00:57:05.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.042+0000 7f8da6b64b80 10 client.0 _ll_register_callbacks cb 0x558c77a676c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:57:05.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.042+0000 7f8d95eea700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c77b564a0 0x558c77cdd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:05.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.042+0000 7f8d95eea700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c77b564a0 0x558c77cdd290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50676/0 (socket says 172.21.15.5:50676) 2024-05-09T00:57:05.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.042+0000 7f8d95eea700 1 -- 172.21.15.5:0/2447912798 learned_addr learned my addr 172.21.15.5:0/2447912798 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:57:05.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.044+0000 7f8d966eb700 1 --2- 172.21.15.5:0/2447912798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c77cd36a0 0x558c77cddd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:05.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.044+0000 7f8d956e9700 1 --2- 172.21.15.5:0/2447912798 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558c77c40160 0x558c77cdd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:05.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.044+0000 7f8d95eea700 1 -- 172.21.15.5:0/2447912798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c77cd36a0 msgr2=0x558c77cddd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:57:05.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.044+0000 7f8d95eea700 1 --2- 172.21.15.5:0/2447912798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c77cd36a0 0x558c77cddd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:05.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.044+0000 7f8d95eea700 1 -- 172.21.15.5:0/2447912798 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558c77c40160 msgr2=0x558c77cdd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:57:05.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.044+0000 7f8d95eea700 1 --2- 172.21.15.5:0/2447912798 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558c77c40160 0x558c77cdd7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:05.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.044+0000 7f8d95eea700 1 -- 172.21.15.5:0/2447912798 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c77a71490 con 0x558c77b564a0 2024-05-09T00:57:05.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.044+0000 7f8d966eb700 1 --2- 172.21.15.5:0/2447912798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c77cd36a0 0x558c77cddd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:57:05.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.045+0000 7f8d95eea700 1 --2- 172.21.15.5:0/2447912798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c77b564a0 0x558c77cdd290 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c009990 tx=0x7f8d8c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de7bb6b19e435a28 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:57:05.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.045+0000 7f8d797fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:57:05.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.045+0000 7f8d797fa700 1 -- 172.21.15.5:0/2447912798 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c00be20 con 0x558c77b564a0 2024-05-09T00:57:05.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.045+0000 7f8d797fa700 1 -- 172.21.15.5:0/2447912798 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f8d8c00f460 con 0x558c77b564a0 2024-05-09T00:57:05.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.045+0000 7f8da6b64b80 1 -- 172.21.15.5:0/2447912798 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d8800c0d0 con 0x558c77b564a0 2024-05-09T00:57:05.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.046+0000 7f8d797fa700 1 -- 172.21.15.5:0/2447912798 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8d8c00f660 con 0x558c77b564a0 2024-05-09T00:57:05.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.046+0000 7f8da6b64b80 10 client.16593 fetch_fsmap learned FSMap version 31 2024-05-09T00:57:05.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.046+0000 7f8da6b64b80 1 -- 172.21.15.5:0/2447912798 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558c77ceaa50 con 0x558c77b564a0 2024-05-09T00:57:05.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.047+0000 7f8d797fa700 1 -- 172.21.15.5:0/2447912798 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d8c00fcb0 con 0x558c77b564a0 2024-05-09T00:57:05.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.047+0000 7f8d797fa700 1 -- 172.21.15.5:0/2447912798 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f8d8c02b070 con 0x558c77b564a0 2024-05-09T00:57:05.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.048+0000 7f8da6b64b80 10 client.16593 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:57:05.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.048+0000 7f8da6b64b80 10 client.16593 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-05-09T00:57:05.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.048+0000 7f8da6b64b80 20 client.16593 populate_metadata read hostname 'smithi005' 2024-05-09T00:57:05.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.048+0000 7f8da6b64b80 1 --2- 172.21.15.5:0/2447912798 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x558c77cecfe0 0x558c77cef400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:05.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.048+0000 7f8da6b64b80 4 client.16593 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:57:05.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.048+0000 7f8da6b64b80 1 -- 172.21.15.5:0/2447912798 --> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8d8800c0d0 con 0x558c77cecfe0 2024-05-09T00:57:05.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.049+0000 7f8d956e9700 1 --2- 172.21.15.5:0/2447912798 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x558c77cecfe0 0x558c77cef400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:05.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.050+0000 7f8d956e9700 1 --2- 172.21.15.5:0/2447912798 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x558c77cecfe0 0x558c77cef400 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=478af5feca0da70c server_cookie=c6dba2f65fe45993 in_seq=0 out_seq=0 2024-05-09T00:57:05.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.050+0000 7f8d797fa700 10 client.16593 ms_handle_connect on v2:172.21.15.157:6836/744863009 2024-05-09T00:57:05.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.052+0000 7f8d797fa700 1 -- 172.21.15.5:0/2447912798 <== mds.1 v2:172.21.15.157:6836/744863009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f8d8800c0d0 con 0x558c77cecfe0 2024-05-09T00:57:05.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.052+0000 7f8d797fa700 10 client.16593 handle_command_reply: tid=0 2024-05-09T00:57:05.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.052+0000 7f8da6b64b80 1 client.16593 shutdown 2024-05-09T00:57:05.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.054+0000 7f8da6b64b80 1 -- 172.21.15.5:0/2447912798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c77b564a0 msgr2=0x558c77cdd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:05.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.055+0000 7f8da6b64b80 1 --2- 172.21.15.5:0/2447912798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c77b564a0 0x558c77cdd290 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c009990 tx=0x7f8d8c009310 comp rx=0 tx=0).stop 2024-05-09T00:57:05.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.055+0000 7f8da6b64b80 1 -- 172.21.15.5:0/2447912798 shutdown_connections 2024-05-09T00:57:05.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.055+0000 7f8da6b64b80 1 --2- 172.21.15.5:0/2447912798 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x558c77cecfe0 0x558c77cef400 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:05.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.055+0000 7f8d797fa700 0 client.16593 ms_handle_reset on v2:172.21.15.157:6836/744863009 2024-05-09T00:57:05.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.055+0000 7f8da6b64b80 1 --2- 172.21.15.5:0/2447912798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c77cd36a0 0x558c77cddd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:05.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.055+0000 7f8da6b64b80 1 --2- 172.21.15.5:0/2447912798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c77b564a0 0x558c77cdd290 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:05.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.055+0000 7f8da6b64b80 1 --2- 172.21.15.5:0/2447912798 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558c77c40160 0x558c77cdd7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:05.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.055+0000 7f8da6b64b80 1 -- 172.21.15.5:0/2447912798 >> 172.21.15.5:0/2447912798 conn(0x558c77bb66a0 msgr2=0x558c77cce1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:57:05.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.056+0000 7f8da6b64b80 1 -- 172.21.15.5:0/2447912798 shutdown_connections 2024-05-09T00:57:05.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.056+0000 7f8da6b64b80 1 -- 172.21.15.5:0/2447912798 wait complete. 2024-05-09T00:57:05.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.056+0000 7f8da6b64b80 20 client.16593 trim_cache size 0 max 16384 2024-05-09T00:57:05.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.101+0000 7f8da6b64b80 1 Processor -- start 2024-05-09T00:57:05.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.101+0000 7f8da6b64b80 1 -- start start 2024-05-09T00:57:05.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.102+0000 7f8da6b64b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558c77b564a0 0x558c77cd8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:05.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.102+0000 7f8da6b64b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c77c40160 0x558c77cf4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:05.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.102+0000 7f8da6b64b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c77cd36a0 0x558c77cf5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:05.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.102+0000 7f8d956e9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c77c40160 0x558c77cf4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:05.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.102+0000 7f8d956e9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c77c40160 0x558c77cf4bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50682/0 (socket says 172.21.15.5:50682) 2024-05-09T00:57:05.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.103+0000 7f8d956e9700 1 -- 172.21.15.5:0/223457399 learned_addr learned my addr 172.21.15.5:0/223457399 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:57:05.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.103+0000 7f8d95eea700 1 --2- 172.21.15.5:0/223457399 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558c77b564a0 0x558c77cd8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:05.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.103+0000 7f8da6b64b80 1 -- 172.21.15.5:0/223457399 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8d901b5400 con 0x558c77c40160 2024-05-09T00:57:05.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.103+0000 7f8da6b64b80 1 -- 172.21.15.5:0/223457399 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8d901b6f40 con 0x558c77b564a0 2024-05-09T00:57:05.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.103+0000 7f8da6b64b80 1 -- 172.21.15.5:0/223457399 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8d901b5ad0 con 0x558c77cd36a0 2024-05-09T00:57:05.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.103+0000 7f8d956e9700 1 -- 172.21.15.5:0/223457399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c77cd36a0 msgr2=0x558c77cf5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:57:05.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.103+0000 7f8d956e9700 1 --2- 172.21.15.5:0/223457399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c77cd36a0 0x558c77cf5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:05.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.103+0000 7f8d956e9700 1 -- 172.21.15.5:0/223457399 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558c77b564a0 msgr2=0x558c77cd8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:57:05.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.103+0000 7f8d956e9700 1 --2- 172.21.15.5:0/223457399 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558c77b564a0 0x558c77cd8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:05.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.104+0000 7f8d956e9700 1 -- 172.21.15.5:0/223457399 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c77cc7f60 con 0x558c77c40160 2024-05-09T00:57:05.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.104+0000 7f8d956e9700 1 --2- 172.21.15.5:0/223457399 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c77c40160 0x558c77cf4bc0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f8d800a8c20 tx=0x7f8d800107c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bb9d7ff1d408c73 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:57:05.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.104+0000 7f8d7b7fe700 1 -- 172.21.15.5:0/223457399 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d800c7020 con 0x558c77c40160 2024-05-09T00:57:05.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.104+0000 7f8d7b7fe700 1 -- 172.21.15.5:0/223457399 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d800ad2d0 con 0x558c77c40160 2024-05-09T00:57:05.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.104+0000 7f8d7b7fe700 1 -- 172.21.15.5:0/223457399 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d800b1b50 con 0x558c77c40160 2024-05-09T00:57:05.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.105+0000 7f8da6b64b80 1 -- 172.21.15.5:0/223457399 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c77c40160 msgr2=0x558c77cf4bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:05.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.105+0000 7f8da6b64b80 1 --2- 172.21.15.5:0/223457399 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c77c40160 0x558c77cf4bc0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f8d800a8c20 tx=0x7f8d800107c0 comp rx=0 tx=0).stop 2024-05-09T00:57:05.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.106+0000 7f8da6b64b80 1 -- 172.21.15.5:0/223457399 shutdown_connections 2024-05-09T00:57:05.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.106+0000 7f8da6b64b80 1 --2- 172.21.15.5:0/223457399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c77cd36a0 0x558c77cf5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:05.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.106+0000 7f8da6b64b80 1 --2- 172.21.15.5:0/223457399 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558c77b564a0 0x558c77cd8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:05.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.106+0000 7f8da6b64b80 1 --2- 172.21.15.5:0/223457399 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c77c40160 0x558c77cf4bc0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:05.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.106+0000 7f8da6b64b80 1 -- 172.21.15.5:0/223457399 >> 172.21.15.5:0/223457399 conn(0x558c77bb66a0 msgr2=0x558c77cd8380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:57:05.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.107+0000 7f8da6b64b80 1 -- 172.21.15.5:0/223457399 shutdown_connections 2024-05-09T00:57:05.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.113+0000 7f8da6b64b80 1 -- 172.21.15.5:0/223457399 wait complete. 2024-05-09T00:57:05.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.114+0000 7f8da6b64b80 1 -- start start 2024-05-09T00:57:05.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.114+0000 7f8da6b64b80 1 Processor -- start 2024-05-09T00:57:05.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.116+0000 7f8da6b64b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c77b564a0 0x558c77ce8060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:05.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.116+0000 7f8da6b64b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c77c40160 0x558c77ce85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:05.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.116+0000 7f8da6b64b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558c77cd36a0 0x558c77ce8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:05.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.116+0000 7f8da6b64b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8d901b6870 con 0x558c77b564a0 2024-05-09T00:57:05.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.116+0000 7f8da6b64b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8d901b61a0 con 0x558c77cd36a0 2024-05-09T00:57:05.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.116+0000 7f8da6b64b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8d901b33a0 con 0x558c77c40160 2024-05-09T00:57:05.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.116+0000 7f8da6b64b80 10 client.0 _ll_register_callbacks cb 0x558c77a676c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:57:05.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.117+0000 7f8d95eea700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c77b564a0 0x558c77ce8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:05.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.117+0000 7f8d95eea700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c77b564a0 0x558c77ce8060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50696/0 (socket says 172.21.15.5:50696) 2024-05-09T00:57:05.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.117+0000 7f8d95eea700 1 -- 172.21.15.5:0/2507855453 learned_addr learned my addr 172.21.15.5:0/2507855453 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:57:05.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.117+0000 7f8d956e9700 1 --2- 172.21.15.5:0/2507855453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c77c40160 0x558c77ce85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:05.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.120+0000 7f8d966eb700 1 --2- 172.21.15.5:0/2507855453 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558c77cd36a0 0x558c77ce8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:05.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.121+0000 7f8d95eea700 1 -- 172.21.15.5:0/2507855453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c77c40160 msgr2=0x558c77ce85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:57:05.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.121+0000 7f8d95eea700 1 --2- 172.21.15.5:0/2507855453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c77c40160 0x558c77ce85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:05.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.121+0000 7f8d95eea700 1 -- 172.21.15.5:0/2507855453 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558c77cd36a0 msgr2=0x558c77ce8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:57:05.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.121+0000 7f8d95eea700 1 --2- 172.21.15.5:0/2507855453 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558c77cd36a0 0x558c77ce8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:05.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.121+0000 7f8d95eea700 1 -- 172.21.15.5:0/2507855453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c77d56770 con 0x558c77b564a0 2024-05-09T00:57:05.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.121+0000 7f8d966eb700 1 --2- 172.21.15.5:0/2507855453 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558c77cd36a0 0x558c77ce8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:57:05.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.125+0000 7f8d95eea700 1 --2- 172.21.15.5:0/2507855453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c77b564a0 0x558c77ce8060 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c006010 tx=0x7f8d8c02da40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ddc0382f765cf9e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:57:05.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.125+0000 7f8d797fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:57:05.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.126+0000 7f8d797fa700 1 -- 172.21.15.5:0/2507855453 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c005980 con 0x558c77b564a0 2024-05-09T00:57:05.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.126+0000 7f8d797fa700 1 -- 172.21.15.5:0/2507855453 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f8d8c00fa10 con 0x558c77b564a0 2024-05-09T00:57:05.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.128+0000 7f8da6b64b80 1 -- 172.21.15.5:0/2507855453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d800ad2d0 con 0x558c77b564a0 2024-05-09T00:57:05.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.130+0000 7f8d797fa700 1 -- 172.21.15.5:0/2507855453 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8d8c046050 con 0x558c77b564a0 2024-05-09T00:57:05.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.131+0000 7f8da6b64b80 10 client.16599 fetch_fsmap learned FSMap version 31 2024-05-09T00:57:05.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.131+0000 7f8da6b64b80 1 -- 172.21.15.5:0/2507855453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558c77d578a0 con 0x558c77b564a0 2024-05-09T00:57:05.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.132+0000 7f8d797fa700 1 -- 172.21.15.5:0/2507855453 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d800ad2d0 con 0x558c77b564a0 2024-05-09T00:57:05.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.132+0000 7f8d797fa700 1 -- 172.21.15.5:0/2507855453 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f8d8c02b040 con 0x558c77b564a0 2024-05-09T00:57:05.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.132+0000 7f8da6b64b80 10 client.16599 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:57:05.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.132+0000 7f8da6b64b80 10 client.16599 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-05-09T00:57:05.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.132+0000 7f8da6b64b80 20 client.16599 populate_metadata read hostname 'smithi005' 2024-05-09T00:57:05.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.132+0000 7f8da6b64b80 1 --2- 172.21.15.5:0/2507855453 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x558c77cecfe0 0x558c77d5a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:05.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.133+0000 7f8da6b64b80 4 client.16599 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:57:05.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.133+0000 7f8da6b64b80 1 -- 172.21.15.5:0/2507855453 --> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558c77c039d0 con 0x558c77cecfe0 2024-05-09T00:57:05.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.135+0000 7f8d956e9700 1 --2- 172.21.15.5:0/2507855453 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x558c77cecfe0 0x558c77d5a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:05.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.170+0000 7f8d797fa700 10 client.16599 ms_handle_connect on v2:172.21.15.157:6836/744863009 2024-05-09T00:57:05.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.170+0000 7f8d956e9700 1 --2- 172.21.15.5:0/2507855453 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x558c77cecfe0 0x558c77d5a020 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4132e357e30cc744 server_cookie=85071c34177f1707 in_seq=0 out_seq=0 2024-05-09T00:57:05.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.171+0000 7f8d797fa700 1 -- 172.21.15.5:0/2507855453 <== mds.1 v2:172.21.15.157:6836/744863009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558c77c039d0 con 0x558c77cecfe0 2024-05-09T00:57:05.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.171+0000 7f8d797fa700 10 client.16599 handle_command_reply: tid=0 2024-05-09T00:57:05.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.172+0000 7f8da6b64b80 1 client.16599 shutdown 2024-05-09T00:57:05.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.176+0000 7f8da6b64b80 1 -- 172.21.15.5:0/2507855453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c77b564a0 msgr2=0x558c77ce8060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:05.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.176+0000 7f8da6b64b80 1 --2- 172.21.15.5:0/2507855453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c77b564a0 0x558c77ce8060 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c006010 tx=0x7f8d8c02da40 comp rx=0 tx=0).stop 2024-05-09T00:57:05.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.183+0000 7f8da6b64b80 1 -- 172.21.15.5:0/2507855453 shutdown_connections 2024-05-09T00:57:05.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.183+0000 7f8da6b64b80 1 --2- 172.21.15.5:0/2507855453 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x558c77cecfe0 0x558c77d5a020 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:05.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.183+0000 7f8da6b64b80 1 --2- 172.21.15.5:0/2507855453 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558c77cd36a0 0x558c77ce8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:05.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.183+0000 7f8da6b64b80 1 --2- 172.21.15.5:0/2507855453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c77b564a0 0x558c77ce8060 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:05.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.183+0000 7f8da6b64b80 1 --2- 172.21.15.5:0/2507855453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c77c40160 0x558c77ce85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:05.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.183+0000 7f8da6b64b80 1 -- 172.21.15.5:0/2507855453 >> 172.21.15.5:0/2507855453 conn(0x558c77bb66a0 msgr2=0x558c77cdbb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:57:05.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.183+0000 7f8d797fa700 0 client.16599 ms_handle_reset on v2:172.21.15.157:6836/744863009 2024-05-09T00:57:05.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.183+0000 7f8da6b64b80 1 -- 172.21.15.5:0/2507855453 shutdown_connections 2024-05-09T00:57:05.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.183+0000 7f8da6b64b80 1 -- 172.21.15.5:0/2507855453 wait complete. 2024-05-09T00:57:05.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.184+0000 7f8da6b64b80 20 client.16599 trim_cache size 0 max 16384 2024-05-09T00:57:05.186 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-09T00:57:05.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.189+0000 7f8d9814e700 1 -- 172.21.15.5:0/3468633712 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f8d7c0b2900 msgr2=0x7f8d7c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:05.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.189+0000 7f8d9814e700 1 --2- 172.21.15.5:0/3468633712 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f8d7c0b2900 0x7f8d7c0b4e10 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c00b5c0 tx=0x7f8d8c019040 comp rx=0 tx=0).stop 2024-05-09T00:57:05.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.189+0000 7f8d9814e700 1 -- 172.21.15.5:0/3468633712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d90105e30 msgr2=0x7f8d901031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:05.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.189+0000 7f8d9814e700 1 --2- 172.21.15.5:0/3468633712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d90105e30 0x7f8d901031d0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f8d8000b530 tx=0x7f8d8000b8f0 comp rx=0 tx=0).stop 2024-05-09T00:57:05.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.190+0000 7f8d9814e700 1 -- 172.21.15.5:0/3468633712 shutdown_connections 2024-05-09T00:57:05.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.190+0000 7f8d9814e700 1 --2- 172.21.15.5:0/3468633712 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f8d7c0b2900 0x7f8d7c0b4e10 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:05.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.190+0000 7f8d9814e700 1 --2- 172.21.15.5:0/3468633712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d9010e910 0x7f8d901037b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:05.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.190+0000 7f8d9814e700 1 --2- 172.21.15.5:0/3468633712 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d901054e0 0x7f8d90104d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:05.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.190+0000 7f8d9814e700 1 --2- 172.21.15.5:0/3468633712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d90105e30 0x7f8d901031d0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:05.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.190+0000 7f8d9814e700 1 -- 172.21.15.5:0/3468633712 >> 172.21.15.5:0/3468633712 conn(0x7f8d900fedc0 msgr2=0x7f8d900ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:57:05.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.195+0000 7f8d9814e700 1 -- 172.21.15.5:0/3468633712 shutdown_connections 2024-05-09T00:57:05.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:05.195+0000 7f8d9814e700 1 -- 172.21.15.5:0/3468633712 wait complete. 2024-05-09T00:57:05.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:57:05 smithi005 ceph-mon[30365]: pgmap v693: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 893 MiB/s wr, 1.94k op/s 2024-05-09T00:57:05.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:57:05 smithi131 ceph-mon[34547]: pgmap v693: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 893 MiB/s wr, 1.94k op/s 2024-05-09T00:57:05.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:57:05 smithi157 ceph-mon[34294]: pgmap v693: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 893 MiB/s wr, 1.94k op/s 2024-05-09T00:57:07.797 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:57:07 smithi005 ceph-mon[30365]: pgmap v694: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 964 MiB/s wr, 2.21k op/s 2024-05-09T00:57:07.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:57:07 smithi131 ceph-mon[34547]: pgmap v694: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 964 MiB/s wr, 2.21k op/s 2024-05-09T00:57:07.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:57:07 smithi157 ceph-mon[34294]: pgmap v694: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 964 MiB/s wr, 2.21k op/s 2024-05-09T00:57:08.077 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T00:57:09.088 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:57:09.797 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:57:09 smithi005 ceph-mon[30365]: pgmap v695: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 837 MiB/s wr, 1.99k op/s 2024-05-09T00:57:09.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:57:09 smithi131 ceph-mon[34547]: pgmap v695: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 837 MiB/s wr, 1.99k op/s 2024-05-09T00:57:09.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:57:09 smithi157 ceph-mon[34294]: pgmap v695: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 837 MiB/s wr, 1.99k op/s 2024-05-09T00:57:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.404+0000 7f001359e640 1 -- 172.21.15.5:0/638202425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f001410a470 msgr2=0x7f001410ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.404+0000 7f001359e640 1 --2- 172.21.15.5:0/638202425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f001410a470 0x7f001410ad80 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f0004009900 tx=0x7f000402a440 comp rx=0 tx=0).stop 2024-05-09T00:57:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.405+0000 7f001359e640 1 -- 172.21.15.5:0/638202425 shutdown_connections 2024-05-09T00:57:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.405+0000 7f001359e640 1 --2- 172.21.15.5:0/638202425 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f001410ffe0 0x7f00141103f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.405+0000 7f001359e640 1 --2- 172.21.15.5:0/638202425 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f001410b2c0 0x7f001410b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.405+0000 7f001359e640 1 --2- 172.21.15.5:0/638202425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f001410a470 0x7f001410ad80 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.405+0000 7f001359e640 1 -- 172.21.15.5:0/638202425 >> 172.21.15.5:0/638202425 conn(0x7f0014070460 msgr2=0x7f0014072910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:57:11.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.409+0000 7f001359e640 1 -- 172.21.15.5:0/638202425 shutdown_connections 2024-05-09T00:57:11.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.410+0000 7f001359e640 1 -- 172.21.15.5:0/638202425 wait complete. 2024-05-09T00:57:11.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.413+0000 7f001359e640 1 Processor -- start 2024-05-09T00:57:11.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.416+0000 7f001359e640 1 -- start start 2024-05-09T00:57:11.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.416+0000 7f001359e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f001410a470 0x7f0014119150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:11.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.416+0000 7f001359e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f001410b2c0 0x7f0014119690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:11.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.416+0000 7f001359e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f001410ffe0 0x7f001411a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:11.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.416+0000 7f001359e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f001411e400 con 0x7f001410a470 2024-05-09T00:57:11.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.416+0000 7f001359e640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f001411e280 con 0x7f001410ffe0 2024-05-09T00:57:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.416+0000 7f001359e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f001411e580 con 0x7f001410b2c0 2024-05-09T00:57:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.418+0000 7f0012d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f001410ffe0 0x7f001411a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.418+0000 7f0012d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f001410ffe0 0x7f001411a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:53668/0 (socket says 172.21.15.5:53668) 2024-05-09T00:57:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.418+0000 7f0012d9d640 1 -- 172.21.15.5:0/1360853298 learned_addr learned my addr 172.21.15.5:0/1360853298 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:57:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.421+0000 7f0011d9b640 1 --2- 172.21.15.5:0/1360853298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f001410b2c0 0x7f0014119690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.421+0000 7f001259c640 1 --2- 172.21.15.5:0/1360853298 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f001410a470 0x7f0014119150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.421+0000 7f0012d9d640 1 -- 172.21.15.5:0/1360853298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f001410b2c0 msgr2=0x7f0014119690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.421+0000 7f0012d9d640 1 --2- 172.21.15.5:0/1360853298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f001410b2c0 0x7f0014119690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.421+0000 7f0012d9d640 1 -- 172.21.15.5:0/1360853298 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f001410a470 msgr2=0x7f0014119150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.421+0000 7f0012d9d640 1 --2- 172.21.15.5:0/1360853298 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f001410a470 0x7f0014119150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.422+0000 7f0012d9d640 1 -- 172.21.15.5:0/1360853298 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f001411ac60 con 0x7f001410ffe0 2024-05-09T00:57:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.423+0000 7f0012d9d640 1 --2- 172.21.15.5:0/1360853298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f001410ffe0 0x7f001411a5f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f000c00b970 tx=0x7f000c00be30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:57:11.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.425+0000 7f00037fe640 1 -- 172.21.15.5:0/1360853298 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000c019070 con 0x7f001410ffe0 2024-05-09T00:57:11.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.425+0000 7f001359e640 1 -- 172.21.15.5:0/1360853298 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f001411af50 con 0x7f001410ffe0 2024-05-09T00:57:11.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.425+0000 7f001359e640 1 -- 172.21.15.5:0/1360853298 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f001411b4a0 con 0x7f001410ffe0 2024-05-09T00:57:11.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.434+0000 7f001f82e740 1 Processor -- start 2024-05-09T00:57:11.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.435+0000 7f001f82e740 1 -- start start 2024-05-09T00:57:11.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.436+0000 7f001f82e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e304f991b0 0x55e30533d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:11.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.436+0000 7f001f82e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e304f702f0 0x55e30533dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:11.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.436+0000 7f001f82e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e304f784d0 0x55e305344b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:11.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.436+0000 7f001f82e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e3052a17c0 con 0x55e304f702f0 2024-05-09T00:57:11.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.436+0000 7f001f82e740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55e30523d0e0 con 0x55e304f784d0 2024-05-09T00:57:11.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.436+0000 7f001f82e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55e30529d780 con 0x55e304f991b0 2024-05-09T00:57:11.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.436+0000 7f00037fe640 1 -- 172.21.15.5:0/1360853298 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f000c014440 con 0x7f001410ffe0 2024-05-09T00:57:11.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.437+0000 7f00037fe640 1 -- 172.21.15.5:0/1360853298 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000c0075d0 con 0x7f001410ffe0 2024-05-09T00:57:11.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.439+0000 7f0011d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e304f784d0 0x55e305344b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:11.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.440+0000 7f0011d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e304f784d0 0x55e305344b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:53682/0 (socket says 172.21.15.5:53682) 2024-05-09T00:57:11.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.440+0000 7f0011d9b640 1 -- 172.21.15.5:0/3854695606 learned_addr learned my addr 172.21.15.5:0/3854695606 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:57:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.441+0000 7f00037fe640 1 -- 172.21.15.5:0/1360853298 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f000c01c070 con 0x7f001410ffe0 2024-05-09T00:57:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.441+0000 7f0011d9b640 1 -- 172.21.15.5:0/3854695606 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e304f991b0 msgr2=0x55e30533d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:57:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.441+0000 7f0011d9b640 1 --2- 172.21.15.5:0/3854695606 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e304f991b0 0x55e30533d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.441+0000 7f0011d9b640 1 -- 172.21.15.5:0/3854695606 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e304f702f0 msgr2=0x55e30533dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:57:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.441+0000 7f0011d9b640 1 --2- 172.21.15.5:0/3854695606 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e304f702f0 0x55e30533dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.441+0000 7f0011d9b640 1 -- 172.21.15.5:0/3854695606 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3052667c0 con 0x55e304f784d0 2024-05-09T00:57:11.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.446+0000 7f0011d9b640 1 --2- 172.21.15.5:0/3854695606 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e304f784d0 0x55e305344b40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f000800bb20 tx=0x7f000800f360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14051b46e66f8db6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:57:11.452 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.448+0000 7f00037fe640 1 --2- 172.21.15.5:0/1360853298 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7efffc0b29f0 0x7efffc0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:11.452 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.448+0000 7f00037fe640 1 -- 172.21.15.5:0/1360853298 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f000c0e27b0 con 0x7f001410ffe0 2024-05-09T00:57:11.452 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.449+0000 7f0000ff9640 1 -- 172.21.15.5:0/3854695606 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0008005020 con 0x55e304f784d0 2024-05-09T00:57:11.452 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.449+0000 7f0000ff9640 1 -- 172.21.15.5:0/3854695606 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00080047a0 con 0x55e304f784d0 2024-05-09T00:57:11.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.452+0000 7f001259c640 1 --2- 172.21.15.5:0/1360853298 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7efffc0b29f0 0x7efffc0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:11.461 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.459+0000 7f001f82e740 1 -- 172.21.15.5:0/3854695606 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e304f784d0 msgr2=0x55e305344b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:11.461 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.459+0000 7f001f82e740 1 --2- 172.21.15.5:0/3854695606 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e304f784d0 0x55e305344b40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f000800bb20 tx=0x7f000800f360 comp rx=0 tx=0).stop 2024-05-09T00:57:11.461 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.459+0000 7f001f82e740 1 -- 172.21.15.5:0/3854695606 shutdown_connections 2024-05-09T00:57:11.461 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.459+0000 7f001f82e740 1 --2- 172.21.15.5:0/3854695606 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e304f784d0 0x55e305344b40 secure :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f000800bb20 tx=0x7f000800f360 comp rx=0 tx=0).stop 2024-05-09T00:57:11.461 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.459+0000 7f001f82e740 1 --2- 172.21.15.5:0/3854695606 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e304f702f0 0x55e30533dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:11.461 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.459+0000 7f001f82e740 1 --2- 172.21.15.5:0/3854695606 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e304f991b0 0x55e30533d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:11.461 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.459+0000 7f001f82e740 1 -- 172.21.15.5:0/3854695606 >> 172.21.15.5:0/3854695606 conn(0x55e304f8d300 msgr2=0x55e305339050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:57:11.464 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.460+0000 7f001f82e740 1 -- 172.21.15.5:0/3854695606 shutdown_connections 2024-05-09T00:57:11.465 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.464+0000 7f001f82e740 1 -- 172.21.15.5:0/3854695606 wait complete. 2024-05-09T00:57:11.466 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.464+0000 7f001f82e740 1 -- start start 2024-05-09T00:57:11.466 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.465+0000 7f001f82e740 1 Processor -- start 2024-05-09T00:57:11.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.465+0000 7f001f82e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e304f702f0 0x55e305347680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:11.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.465+0000 7f001f82e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e304f991b0 0x55e305347bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:11.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.465+0000 7f001f82e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e304f57d80 0x55e30534eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:11.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.465+0000 7f001f82e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e3053455e0 con 0x55e304f991b0 2024-05-09T00:57:11.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.465+0000 7f001f82e740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55e3052289c0 con 0x55e304f57d80 2024-05-09T00:57:11.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.465+0000 7f001f82e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55e305345760 con 0x55e304f702f0 2024-05-09T00:57:11.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.466+0000 7f001f82e740 10 client.0 _ll_register_callbacks cb 0x55e305255020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:57:11.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.466+0000 7f0012d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e304f57d80 0x55e30534eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:11.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.467+0000 7f0012d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e304f57d80 0x55e30534eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:53688/0 (socket says 172.21.15.5:53688) 2024-05-09T00:57:11.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.467+0000 7f0012d9d640 1 -- 172.21.15.5:0/51003416 learned_addr learned my addr 172.21.15.5:0/51003416 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:57:11.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.467+0000 7f0012d9d640 1 -- 172.21.15.5:0/51003416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e304f702f0 msgr2=0x55e305347680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:57:11.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.468+0000 7f0012d9d640 1 --2- 172.21.15.5:0/51003416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e304f702f0 0x55e305347680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:11.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.468+0000 7f0012d9d640 1 -- 172.21.15.5:0/51003416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e304f991b0 msgr2=0x55e305347bc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:57:11.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.468+0000 7f0012d9d640 1 --2- 172.21.15.5:0/51003416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e304f991b0 0x55e305347bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:11.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.468+0000 7f0012d9d640 1 -- 172.21.15.5:0/51003416 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e30524f1f0 con 0x55e304f57d80 2024-05-09T00:57:11.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.468+0000 7f0012d9d640 1 --2- 172.21.15.5:0/51003416 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e304f57d80 0x55e30534eb20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f000c00b940 tx=0x7f000c0a9850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f937238f412687c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:57:11.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.474+0000 7efff9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-09T00:57:11.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.474+0000 7efff9ffb640 1 -- 172.21.15.5:0/51003416 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000c0ae0f0 con 0x55e304f57d80 2024-05-09T00:57:11.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.474+0000 7f001f82e740 1 -- 172.21.15.5:0/51003416 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e3052af3d0 con 0x55e304f57d80 2024-05-09T00:57:11.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.474+0000 7efff9ffb640 1 -- 172.21.15.5:0/51003416 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f000c0c43c0 con 0x55e304f57d80 2024-05-09T00:57:11.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.475+0000 7efff9ffb640 1 -- 172.21.15.5:0/51003416 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f000c0b2ba0 con 0x55e304f57d80 2024-05-09T00:57:11.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.475+0000 7f001f82e740 10 client.26063 fetch_fsmap learned FSMap version 31 2024-05-09T00:57:11.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.475+0000 7f001f82e740 1 -- 172.21.15.5:0/51003416 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e305004790 con 0x55e304f57d80 2024-05-09T00:57:11.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.476+0000 7f001259c640 1 --2- 172.21.15.5:0/1360853298 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7efffc0b29f0 0x7efffc0b4f00 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f000402aa30 tx=0x7f0004004aa0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:57:11.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.476+0000 7efff9ffb640 1 -- 172.21.15.5:0/51003416 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55e3052af3d0 con 0x55e304f57d80 2024-05-09T00:57:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.484+0000 7efff9ffb640 1 -- 172.21.15.5:0/51003416 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f000c0a5740 con 0x55e304f57d80 2024-05-09T00:57:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.484+0000 7f001f82e740 10 client.26063 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:57:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.484+0000 7f001f82e740 10 client.26063 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:57:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.484+0000 7f001f82e740 20 client.26063 populate_metadata read hostname 'smithi005' 2024-05-09T00:57:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.484+0000 7f001f82e740 1 --2- 172.21.15.5:0/51003416 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55e304f2d1c0 0x55e305355700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.484+0000 7f001f82e740 4 client.26063 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:57:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.484+0000 7f001f82e740 1 -- 172.21.15.5:0/51003416 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e3052a9240 con 0x55e304f2d1c0 2024-05-09T00:57:11.491 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.488+0000 7f001259c640 1 --2- 172.21.15.5:0/51003416 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55e304f2d1c0 0x55e305355700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.502+0000 7f001259c640 1 --2- 172.21.15.5:0/51003416 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55e304f2d1c0 0x55e305355700 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce378acc881828a1 server_cookie=5c75811bdb0a0b88 in_seq=0 out_seq=0 2024-05-09T00:57:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.502+0000 7efff9ffb640 10 client.26063 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:57:11.512 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.509+0000 7efff9ffb640 1 -- 172.21.15.5:0/51003416 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e3052a9240 con 0x55e304f2d1c0 2024-05-09T00:57:11.512 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.509+0000 7efff9ffb640 10 client.26063 handle_command_reply: tid=0 2024-05-09T00:57:11.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.510+0000 7f001f82e740 1 client.26063 shutdown 2024-05-09T00:57:11.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.510+0000 7f001f82e740 1 -- 172.21.15.5:0/51003416 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e304f57d80 msgr2=0x55e30534eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:11.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.510+0000 7f001f82e740 1 --2- 172.21.15.5:0/51003416 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e304f57d80 0x55e30534eb20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f000c00b940 tx=0x7f000c0a9850 comp rx=0 tx=0).stop 2024-05-09T00:57:11.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.510+0000 7f001f82e740 1 -- 172.21.15.5:0/51003416 shutdown_connections 2024-05-09T00:57:11.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.510+0000 7f001f82e740 1 --2- 172.21.15.5:0/51003416 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55e304f2d1c0 0x55e305355700 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:11.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.510+0000 7f001f82e740 1 --2- 172.21.15.5:0/51003416 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e304f57d80 0x55e30534eb20 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:11.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.510+0000 7f001f82e740 1 --2- 172.21.15.5:0/51003416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e304f991b0 0x55e305347bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:11.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.510+0000 7f001f82e740 1 --2- 172.21.15.5:0/51003416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e304f702f0 0x55e305347680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:11.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.510+0000 7f001f82e740 1 -- 172.21.15.5:0/51003416 >> 172.21.15.5:0/51003416 conn(0x55e304f8d300 msgr2=0x55e30533bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:57:11.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.511+0000 7efff9ffb640 0 client.26063 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:57:11.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.511+0000 7f001f82e740 1 -- 172.21.15.5:0/51003416 shutdown_connections 2024-05-09T00:57:11.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.511+0000 7f001f82e740 1 -- 172.21.15.5:0/51003416 wait complete. 2024-05-09T00:57:11.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.511+0000 7f001f82e740 20 client.26063 trim_cache size 0 max 16384 2024-05-09T00:57:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.603+0000 7f001f82e740 1 Processor -- start 2024-05-09T00:57:11.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.604+0000 7f001f82e740 1 -- start start 2024-05-09T00:57:11.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.604+0000 7f001f82e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e304f57d80 0x55e305346910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:11.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.604+0000 7f001f82e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e304f702f0 0x55e305346e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:11.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.604+0000 7f001f82e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e304f991b0 0x55e30534eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:11.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.604+0000 7f001f82e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efff400e410 con 0x55e304f702f0 2024-05-09T00:57:11.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.604+0000 7f001f82e740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7efff400ff50 con 0x55e304f57d80 2024-05-09T00:57:11.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.604+0000 7f001f82e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7efff400eae0 con 0x55e304f991b0 2024-05-09T00:57:11.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.606+0000 7f0012d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e304f991b0 0x55e30534eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:11.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.606+0000 7f001259c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e304f57d80 0x55e305346910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:11.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.606+0000 7f001259c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e304f57d80 0x55e305346910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:53700/0 (socket says 172.21.15.5:53700) 2024-05-09T00:57:11.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.606+0000 7f001259c640 1 -- 172.21.15.5:0/2198323380 learned_addr learned my addr 172.21.15.5:0/2198323380 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:57:11.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.607+0000 7f001259c640 1 -- 172.21.15.5:0/2198323380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e304f991b0 msgr2=0x55e30534eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:57:11.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.607+0000 7f001259c640 1 --2- 172.21.15.5:0/2198323380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e304f991b0 0x55e30534eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:11.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.607+0000 7f001259c640 1 -- 172.21.15.5:0/2198323380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e304f702f0 msgr2=0x55e305346e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:57:11.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.607+0000 7f001259c640 1 --2- 172.21.15.5:0/2198323380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e304f702f0 0x55e305346e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:11.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.607+0000 7f001259c640 1 -- 172.21.15.5:0/2198323380 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e30507b5f0 con 0x55e304f57d80 2024-05-09T00:57:11.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.608+0000 7f001259c640 1 --2- 172.21.15.5:0/2198323380 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e304f57d80 0x55e305346910 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f00040096c0 tx=0x7f0004041000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b655e825aaa35535 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:57:11.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.609+0000 7efffbfff640 1 -- 172.21.15.5:0/2198323380 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0004057070 con 0x55e304f57d80 2024-05-09T00:57:11.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.609+0000 7efffbfff640 1 -- 172.21.15.5:0/2198323380 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0004051250 con 0x55e304f57d80 2024-05-09T00:57:11.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.610+0000 7efffbfff640 1 -- 172.21.15.5:0/2198323380 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0004031680 con 0x55e304f57d80 2024-05-09T00:57:11.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.610+0000 7f001f82e740 1 -- 172.21.15.5:0/2198323380 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e304f57d80 msgr2=0x55e305346910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:11.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.610+0000 7f001f82e740 1 --2- 172.21.15.5:0/2198323380 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e304f57d80 0x55e305346910 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f00040096c0 tx=0x7f0004041000 comp rx=0 tx=0).stop 2024-05-09T00:57:11.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.611+0000 7f001f82e740 1 -- 172.21.15.5:0/2198323380 shutdown_connections 2024-05-09T00:57:11.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.611+0000 7f001f82e740 1 --2- 172.21.15.5:0/2198323380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e304f991b0 0x55e30534eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:11.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.611+0000 7f001f82e740 1 --2- 172.21.15.5:0/2198323380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e304f702f0 0x55e305346e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:11.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.611+0000 7f001f82e740 1 --2- 172.21.15.5:0/2198323380 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e304f57d80 0x55e305346910 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:11.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.611+0000 7f001f82e740 1 -- 172.21.15.5:0/2198323380 >> 172.21.15.5:0/2198323380 conn(0x55e304f8d300 msgr2=0x55e305345cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:57:11.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.611+0000 7f001f82e740 1 -- 172.21.15.5:0/2198323380 shutdown_connections 2024-05-09T00:57:11.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.611+0000 7f001f82e740 1 -- 172.21.15.5:0/2198323380 wait complete. 2024-05-09T00:57:11.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.612+0000 7f001f82e740 1 -- start start 2024-05-09T00:57:11.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.612+0000 7f001f82e740 1 Processor -- start 2024-05-09T00:57:11.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.613+0000 7f001f82e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e304f57d80 0x55e30534edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:11.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.613+0000 7f001f82e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e304f702f0 0x55e30534b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:11.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.613+0000 7f001f82e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e304f991b0 0x55e30534be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:11.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.613+0000 7f001f82e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efff400f880 con 0x55e304f702f0 2024-05-09T00:57:11.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.613+0000 7f001f82e740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7efff400f1b0 con 0x55e304f991b0 2024-05-09T00:57:11.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.613+0000 7f001f82e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7efff400c4a0 con 0x55e304f57d80 2024-05-09T00:57:11.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.613+0000 7f001f82e740 10 client.0 _ll_register_callbacks cb 0x7f000c014b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:57:11.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.614+0000 7f0012d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e304f991b0 0x55e30534be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:11.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.614+0000 7f0012d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e304f991b0 0x55e30534be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:53714/0 (socket says 172.21.15.5:53714) 2024-05-09T00:57:11.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.614+0000 7f0012d9d640 1 -- 172.21.15.5:0/2743885473 learned_addr learned my addr 172.21.15.5:0/2743885473 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:57:11.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.614+0000 7f0012d9d640 1 -- 172.21.15.5:0/2743885473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e304f57d80 msgr2=0x55e30534edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:57:11.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.614+0000 7f0012d9d640 1 --2- 172.21.15.5:0/2743885473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e304f57d80 0x55e30534edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:11.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.614+0000 7f0012d9d640 1 -- 172.21.15.5:0/2743885473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e304f702f0 msgr2=0x55e30534b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:57:11.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.614+0000 7f0012d9d640 1 --2- 172.21.15.5:0/2743885473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e304f702f0 0x55e30534b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:11.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.614+0000 7f0012d9d640 1 -- 172.21.15.5:0/2743885473 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e305265ed0 con 0x55e304f991b0 2024-05-09T00:57:11.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.617+0000 7f0012d9d640 1 --2- 172.21.15.5:0/2743885473 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e304f991b0 0x55e30534be90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f000c0a9cd0 tx=0x7f000c014d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf7016b83209cc64 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:57:11.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.617+0000 7efff9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-09T00:57:11.621 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.619+0000 7efff9ffb640 1 -- 172.21.15.5:0/2743885473 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000c0a5c00 con 0x55e304f991b0 2024-05-09T00:57:11.621 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.619+0000 7efff9ffb640 1 -- 172.21.15.5:0/2743885473 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f000c0c43c0 con 0x55e304f991b0 2024-05-09T00:57:11.621 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.619+0000 7f001f82e740 1 -- 172.21.15.5:0/2743885473 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0004051250 con 0x55e304f991b0 2024-05-09T00:57:11.622 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.621+0000 7efff9ffb640 1 -- 172.21.15.5:0/2743885473 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f000c0ae0f0 con 0x55e304f991b0 2024-05-09T00:57:11.622 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.621+0000 7f001f82e740 10 client.26069 fetch_fsmap learned FSMap version 31 2024-05-09T00:57:11.623 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.621+0000 7f001f82e740 1 -- 172.21.15.5:0/2743885473 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e3053bc6d0 con 0x55e304f991b0 2024-05-09T00:57:11.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.622+0000 7efff9ffb640 1 -- 172.21.15.5:0/2743885473 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0004051250 con 0x55e304f991b0 2024-05-09T00:57:11.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.623+0000 7efff9ffb640 1 -- 172.21.15.5:0/2743885473 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f000c0b9020 con 0x55e304f991b0 2024-05-09T00:57:11.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.623+0000 7f001f82e740 10 client.26069 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:57:11.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.623+0000 7f001f82e740 10 client.26069 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:57:11.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.623+0000 7f001f82e740 20 client.26069 populate_metadata read hostname 'smithi005' 2024-05-09T00:57:11.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.623+0000 7f001f82e740 1 --2- 172.21.15.5:0/2743885473 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55e3053bce70 0x55e3053bf300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:11.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.623+0000 7f001f82e740 4 client.26069 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:57:11.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.623+0000 7f001f82e740 1 -- 172.21.15.5:0/2743885473 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e305224350 con 0x55e3053bce70 2024-05-09T00:57:11.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.628+0000 7f001259c640 1 --2- 172.21.15.5:0/2743885473 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55e3053bce70 0x55e3053bf300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:11.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.641+0000 7f001259c640 1 --2- 172.21.15.5:0/2743885473 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55e3053bce70 0x55e3053bf300 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c69a16bec2316ac5 server_cookie=24a9681d493f9849 in_seq=0 out_seq=0 2024-05-09T00:57:11.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:11.641+0000 7efff9ffb640 10 client.26069 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:57:11.790 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:57:11 smithi005 ceph-mon[30365]: pgmap v696: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1002 MiB/s wr, 2.19k op/s 2024-05-09T00:57:11.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:57:11 smithi131 ceph-mon[34547]: pgmap v696: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1002 MiB/s wr, 2.19k op/s 2024-05-09T00:57:11.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:57:11 smithi157 ceph-mon[34294]: pgmap v696: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1002 MiB/s wr, 2.19k op/s 2024-05-09T00:57:13.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:57:13 smithi131 ceph-mon[34547]: pgmap v697: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 810 MiB/s wr, 1.62k op/s 2024-05-09T00:57:13.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:57:13 smithi157 ceph-mon[34294]: pgmap v697: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 810 MiB/s wr, 1.62k op/s 2024-05-09T00:57:14.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:57:13 smithi005 ceph-mon[30365]: pgmap v697: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 810 MiB/s wr, 1.62k op/s 2024-05-09T00:57:14.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.412+0000 7fe36ea35700 1 -- 172.21.15.5:0/1376260037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe368103250 msgr2=0x7fe368103660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:14.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.412+0000 7fe36ea35700 1 --2- 172.21.15.5:0/1376260037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe368103250 0x7fe368103660 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fe360009990 tx=0x7fe360009ca0 comp rx=0 tx=0).stop 2024-05-09T00:57:14.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.412+0000 7fe36ea35700 1 -- 172.21.15.5:0/1376260037 shutdown_connections 2024-05-09T00:57:14.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.412+0000 7fe36ea35700 1 --2- 172.21.15.5:0/1376260037 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3681086a0 0x7fe368108b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:14.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.412+0000 7fe36ea35700 1 --2- 172.21.15.5:0/1376260037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe368103250 0x7fe368103660 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:14.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.412+0000 7fe36ea35700 1 --2- 172.21.15.5:0/1376260037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe368103ba0 0x7fe3681080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:14.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.412+0000 7fe36ea35700 1 -- 172.21.15.5:0/1376260037 >> 172.21.15.5:0/1376260037 conn(0x7fe3680fedc0 msgr2=0x7fe3681011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:57:14.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.412+0000 7fe36ea35700 1 -- 172.21.15.5:0/1376260037 shutdown_connections 2024-05-09T00:57:14.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.412+0000 7fe36ea35700 1 -- 172.21.15.5:0/1376260037 wait complete. 2024-05-09T00:57:14.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.414+0000 7fe36ea35700 1 Processor -- start 2024-05-09T00:57:14.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.414+0000 7fe36ea35700 1 -- start start 2024-05-09T00:57:14.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.414+0000 7fe36ea35700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe368103250 0x7fe3681a1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:14.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.414+0000 7fe36ea35700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe368103ba0 0x7fe3681a1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:14.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.415+0000 7fe36ea35700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3681086a0 0x7fe3681a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:14.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.415+0000 7fe36ea35700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe36811ab70 con 0x7fe368103250 2024-05-09T00:57:14.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.415+0000 7fe36ea35700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe36811a9f0 con 0x7fe3681086a0 2024-05-09T00:57:14.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.415+0000 7fe36ea35700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe36811acf0 con 0x7fe368103ba0 2024-05-09T00:57:14.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.415+0000 7fe367fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe368103250 0x7fe3681a1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:14.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.415+0000 7fe367fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe368103250 0x7fe3681a1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37314/0 (socket says 172.21.15.5:37314) 2024-05-09T00:57:14.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.415+0000 7fe367fff700 1 -- 172.21.15.5:0/1734576932 learned_addr learned my addr 172.21.15.5:0/1734576932 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:57:14.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.415+0000 7fe367fff700 1 -- 172.21.15.5:0/1734576932 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe368103ba0 msgr2=0x7fe3681a1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:14.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.415+0000 7fe367fff700 1 --2- 172.21.15.5:0/1734576932 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe368103ba0 0x7fe3681a1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:14.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.416+0000 7fe367fff700 1 -- 172.21.15.5:0/1734576932 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3681086a0 msgr2=0x7fe3681a5e70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T00:57:14.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.416+0000 7fe367fff700 1 --2- 172.21.15.5:0/1734576932 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3681086a0 0x7fe3681a5e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:14.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.416+0000 7fe367fff700 1 -- 172.21.15.5:0/1734576932 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3681a6510 con 0x7fe368103250 2024-05-09T00:57:14.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.416+0000 7fe367fff700 1 --2- 172.21.15.5:0/1734576932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe368103250 0x7fe3681a1640 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fe360009ee0 tx=0x7fe360005590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:57:14.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.416+0000 7fe3657fa700 1 -- 172.21.15.5:0/1734576932 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe360027070 con 0x7fe368103250 2024-05-09T00:57:14.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.416+0000 7fe3657fa700 1 -- 172.21.15.5:0/1734576932 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe360003fb0 con 0x7fe368103250 2024-05-09T00:57:14.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.416+0000 7fe3657fa700 1 -- 172.21.15.5:0/1734576932 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe360022d70 con 0x7fe368103250 2024-05-09T00:57:14.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.416+0000 7fe36ea35700 1 -- 172.21.15.5:0/1734576932 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3681a67a0 con 0x7fe368103250 2024-05-09T00:57:14.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.417+0000 7fe36ea35700 1 -- 172.21.15.5:0/1734576932 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3681a6b70 con 0x7fe368103250 2024-05-09T00:57:14.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.420+0000 7fe3657fa700 1 -- 172.21.15.5:0/1734576932 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fe36002a070 con 0x7fe368103250 2024-05-09T00:57:14.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.421+0000 7fe3657fa700 1 --2- 172.21.15.5:0/1734576932 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe3540b2980 0x7fe3540b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:14.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.421+0000 7fe3657fa700 1 -- 172.21.15.5:0/1734576932 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fe3600e3080 con 0x7fe368103250 2024-05-09T00:57:14.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.421+0000 7fe3677fe700 1 --2- 172.21.15.5:0/1734576932 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe3540b2980 0x7fe3540b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:14.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.422+0000 7fe37d44bb80 1 Processor -- start 2024-05-09T00:57:14.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.422+0000 7fe37d44bb80 1 -- start start 2024-05-09T00:57:14.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.422+0000 7fe3677fe700 1 --2- 172.21.15.5:0/1734576932 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe3540b2980 0x7fe3540b4e90 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe368100af0 tx=0x7fe358009250 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:57:14.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.422+0000 7fe37d44bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c496d85160 0x55c496e1e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:14.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.422+0000 7fe37d44bb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c496c9b4a0 0x55c496e1ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:14.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.422+0000 7fe37d44bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c496e186a0 0x55c496e18ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:14.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.422+0000 7fe37d44bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c496d3aff0 con 0x55c496e186a0 2024-05-09T00:57:14.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.422+0000 7fe37d44bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55c496cd97f0 con 0x55c496d85160 2024-05-09T00:57:14.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.422+0000 7fe37d44bb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55c496d36fb0 con 0x55c496c9b4a0 2024-05-09T00:57:14.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.422+0000 7fe36cfd2700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c496e186a0 0x55c496e18ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:14.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.422+0000 7fe36cfd2700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c496e186a0 0x55c496e18ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37330/0 (socket says 172.21.15.5:37330) 2024-05-09T00:57:14.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.422+0000 7fe36cfd2700 1 -- 172.21.15.5:0/1130291778 learned_addr learned my addr 172.21.15.5:0/1130291778 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:57:14.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.423+0000 7fe36cfd2700 1 -- 172.21.15.5:0/1130291778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c496c9b4a0 msgr2=0x55c496e1ea30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:57:14.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.423+0000 7fe36cfd2700 1 --2- 172.21.15.5:0/1130291778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c496c9b4a0 0x55c496e1ea30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:14.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.423+0000 7fe36cfd2700 1 -- 172.21.15.5:0/1130291778 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c496d85160 msgr2=0x55c496e1e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:57:14.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.423+0000 7fe36cfd2700 1 --2- 172.21.15.5:0/1130291778 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c496d85160 0x55c496e1e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:14.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.423+0000 7fe36cfd2700 1 -- 172.21.15.5:0/1130291778 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c496cd1370 con 0x55c496e186a0 2024-05-09T00:57:14.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.423+0000 7fe36cfd2700 1 --2- 172.21.15.5:0/1130291778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c496e186a0 0x55c496e18ad0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fe3500097b0 tx=0x7fe350009ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a01bedb512b906be server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:57:14.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.423+0000 7fe35e7fc700 1 -- 172.21.15.5:0/1130291778 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe350015020 con 0x55c496e186a0 2024-05-09T00:57:14.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.424+0000 7fe35e7fc700 1 -- 172.21.15.5:0/1130291778 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe350011ce0 con 0x55c496e186a0 2024-05-09T00:57:14.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.424+0000 7fe35e7fc700 1 -- 172.21.15.5:0/1130291778 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe350010d20 con 0x55c496e186a0 2024-05-09T00:57:14.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.425+0000 7fe37d44bb80 1 -- 172.21.15.5:0/1130291778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c496e186a0 msgr2=0x55c496e18ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:14.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.425+0000 7fe37d44bb80 1 --2- 172.21.15.5:0/1130291778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c496e186a0 0x55c496e18ad0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fe3500097b0 tx=0x7fe350009ac0 comp rx=0 tx=0).stop 2024-05-09T00:57:14.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.425+0000 7fe37d44bb80 1 -- 172.21.15.5:0/1130291778 shutdown_connections 2024-05-09T00:57:14.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.425+0000 7fe37d44bb80 1 --2- 172.21.15.5:0/1130291778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c496e186a0 0x55c496e18ad0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:14.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.425+0000 7fe37d44bb80 1 --2- 172.21.15.5:0/1130291778 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c496d85160 0x55c496e1e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:14.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.425+0000 7fe37d44bb80 1 --2- 172.21.15.5:0/1130291778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c496c9b4a0 0x55c496e1ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:14.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.425+0000 7fe37d44bb80 1 -- 172.21.15.5:0/1130291778 >> 172.21.15.5:0/1130291778 conn(0x55c496cfb6a0 msgr2=0x55c496e13400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:57:14.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.425+0000 7fe37d44bb80 1 -- 172.21.15.5:0/1130291778 shutdown_connections 2024-05-09T00:57:14.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.425+0000 7fe37d44bb80 1 -- 172.21.15.5:0/1130291778 wait complete. 2024-05-09T00:57:14.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.426+0000 7fe37d44bb80 1 -- start start 2024-05-09T00:57:14.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.426+0000 7fe37d44bb80 1 Processor -- start 2024-05-09T00:57:14.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.427+0000 7fe37d44bb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c496c9b4a0 0x55c496e222a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:14.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.427+0000 7fe37d44bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c496d85160 0x55c496e227e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:14.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.427+0000 7fe37d44bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c496e186a0 0x55c496e22d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:14.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.427+0000 7fe37d44bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c496bb6ba0 con 0x55c496e186a0 2024-05-09T00:57:14.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.427+0000 7fe37d44bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55c496cf5150 con 0x55c496d85160 2024-05-09T00:57:14.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.427+0000 7fe37d44bb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55c496e251d0 con 0x55c496c9b4a0 2024-05-09T00:57:14.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.428+0000 7fe37d44bb80 10 client.0 _ll_register_callbacks cb 0x55c496bac6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:57:14.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.428+0000 7fe36cfd2700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c496e186a0 0x55c496e22d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:14.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.428+0000 7fe36cfd2700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c496e186a0 0x55c496e22d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37342/0 (socket says 172.21.15.5:37342) 2024-05-09T00:57:14.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.428+0000 7fe36cfd2700 1 -- 172.21.15.5:0/3084746781 learned_addr learned my addr 172.21.15.5:0/3084746781 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:57:14.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.428+0000 7fe36cfd2700 1 -- 172.21.15.5:0/3084746781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c496c9b4a0 msgr2=0x55c496e222a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:57:14.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.428+0000 7fe36cfd2700 1 --2- 172.21.15.5:0/3084746781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c496c9b4a0 0x55c496e222a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:14.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.428+0000 7fe36cfd2700 1 -- 172.21.15.5:0/3084746781 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c496d85160 msgr2=0x55c496e227e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:57:14.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.428+0000 7fe36cfd2700 1 --2- 172.21.15.5:0/3084746781 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c496d85160 0x55c496e227e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:14.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.429+0000 7fe36cfd2700 1 -- 172.21.15.5:0/3084746781 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c496bb6490 con 0x55c496e186a0 2024-05-09T00:57:14.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.429+0000 7fe36cfd2700 1 --2- 172.21.15.5:0/3084746781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c496e186a0 0x55c496e22d20 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fe350004790 tx=0x7fe350011630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=503fa0bb41a5c19d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:57:14.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.429+0000 7fe3477fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:57:14.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.429+0000 7fe3477fe700 1 -- 172.21.15.5:0/3084746781 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe350027060 con 0x55c496e186a0 2024-05-09T00:57:14.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.430+0000 7fe3477fe700 1 -- 172.21.15.5:0/3084746781 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fe3500109a0 con 0x55c496e186a0 2024-05-09T00:57:14.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.430+0000 7fe37d44bb80 1 -- 172.21.15.5:0/3084746781 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe350011ce0 con 0x55c496e186a0 2024-05-09T00:57:14.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.430+0000 7fe3477fe700 1 -- 172.21.15.5:0/3084746781 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe350010ba0 con 0x55c496e186a0 2024-05-09T00:57:14.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.430+0000 7fe37d44bb80 10 client.16617 fetch_fsmap learned FSMap version 31 2024-05-09T00:57:14.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.430+0000 7fe37d44bb80 1 -- 172.21.15.5:0/3084746781 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c496e2fc70 con 0x55c496e186a0 2024-05-09T00:57:14.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.431+0000 7fe3477fe700 1 -- 172.21.15.5:0/3084746781 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe350010d40 con 0x55c496e186a0 2024-05-09T00:57:14.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.431+0000 7fe3477fe700 1 -- 172.21.15.5:0/3084746781 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fe35002d410 con 0x55c496e186a0 2024-05-09T00:57:14.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.431+0000 7fe37d44bb80 10 client.16617 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:57:14.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.431+0000 7fe37d44bb80 10 client.16617 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:57:14.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.431+0000 7fe37d44bb80 20 client.16617 populate_metadata read hostname 'smithi005' 2024-05-09T00:57:14.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.431+0000 7fe37d44bb80 1 --2- 172.21.15.5:0/3084746781 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55c496e32140 0x55c496e34560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:14.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.431+0000 7fe37d44bb80 4 client.16617 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:57:14.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.431+0000 7fe37d44bb80 1 -- 172.21.15.5:0/3084746781 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe350011ce0 con 0x55c496e32140 2024-05-09T00:57:14.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.432+0000 7fe367fff700 1 --2- 172.21.15.5:0/3084746781 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55c496e32140 0x55c496e34560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:14.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.436+0000 7fe367fff700 1 --2- 172.21.15.5:0/3084746781 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55c496e32140 0x55c496e34560 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b757455ba7bae9d9 server_cookie=e83a5b0afd78b286 in_seq=0 out_seq=0 2024-05-09T00:57:14.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.436+0000 7fe3477fe700 10 client.16617 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:57:14.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.440+0000 7fe3477fe700 1 -- 172.21.15.5:0/3084746781 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fe350011ce0 con 0x55c496e32140 2024-05-09T00:57:14.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.440+0000 7fe3477fe700 10 client.16617 handle_command_reply: tid=0 2024-05-09T00:57:14.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.440+0000 7fe37d44bb80 1 client.16617 shutdown 2024-05-09T00:57:14.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.441+0000 7fe37d44bb80 1 -- 172.21.15.5:0/3084746781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c496e186a0 msgr2=0x55c496e22d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:14.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.441+0000 7fe37d44bb80 1 --2- 172.21.15.5:0/3084746781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c496e186a0 0x55c496e22d20 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fe350004790 tx=0x7fe350011630 comp rx=0 tx=0).stop 2024-05-09T00:57:14.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.443+0000 7fe37d44bb80 1 -- 172.21.15.5:0/3084746781 shutdown_connections 2024-05-09T00:57:14.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.443+0000 7fe37d44bb80 1 --2- 172.21.15.5:0/3084746781 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55c496e32140 0x55c496e34560 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:14.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.444+0000 7fe37d44bb80 1 --2- 172.21.15.5:0/3084746781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c496e186a0 0x55c496e22d20 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:14.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.444+0000 7fe37d44bb80 1 --2- 172.21.15.5:0/3084746781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c496c9b4a0 0x55c496e222a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:14.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.444+0000 7fe37d44bb80 1 --2- 172.21.15.5:0/3084746781 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c496d85160 0x55c496e227e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:14.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.444+0000 7fe37d44bb80 1 -- 172.21.15.5:0/3084746781 >> 172.21.15.5:0/3084746781 conn(0x55c496cfb6a0 msgr2=0x55c496e131c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:57:14.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.444+0000 7fe3477fe700 0 client.16617 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:57:14.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.445+0000 7fe37d44bb80 1 -- 172.21.15.5:0/3084746781 shutdown_connections 2024-05-09T00:57:14.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.445+0000 7fe37d44bb80 1 -- 172.21.15.5:0/3084746781 wait complete. 2024-05-09T00:57:14.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.447+0000 7fe37d44bb80 20 client.16617 trim_cache size 0 max 16384 2024-05-09T00:57:14.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.502+0000 7fe37d44bb80 1 Processor -- start 2024-05-09T00:57:14.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.503+0000 7fe37d44bb80 1 -- start start 2024-05-09T00:57:14.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.504+0000 7fe37d44bb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c496c9b4a0 0x55c496e22b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:14.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.505+0000 7fe37d44bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c496d85160 0x55c496e24060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:14.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.505+0000 7fe37d44bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c496e186a0 0x55c496e39c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:14.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.505+0000 7fe37d44bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe3681b5330 con 0x55c496d85160 2024-05-09T00:57:14.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.505+0000 7fe37d44bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe3681b6e70 con 0x55c496e186a0 2024-05-09T00:57:14.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.505+0000 7fe37d44bb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe3681b5a00 con 0x55c496c9b4a0 2024-05-09T00:57:14.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.507+0000 7fe367fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c496c9b4a0 0x55c496e22b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:14.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.507+0000 7fe36cfd2700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c496e186a0 0x55c496e39c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:14.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.507+0000 7fe36cfd2700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c496e186a0 0x55c496e39c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:45492/0 (socket says 172.21.15.5:45492) 2024-05-09T00:57:14.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.507+0000 7fe36cfd2700 1 -- 172.21.15.5:0/1747975816 learned_addr learned my addr 172.21.15.5:0/1747975816 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:57:14.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.513+0000 7fe36cfd2700 1 -- 172.21.15.5:0/1747975816 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c496c9b4a0 msgr2=0x55c496e22b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:57:14.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.513+0000 7fe36cfd2700 1 --2- 172.21.15.5:0/1747975816 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c496c9b4a0 0x55c496e22b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:14.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.513+0000 7fe36cfd2700 1 -- 172.21.15.5:0/1747975816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c496d85160 msgr2=0x55c496e24060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:57:14.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.513+0000 7fe36cfd2700 1 --2- 172.21.15.5:0/1747975816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c496d85160 0x55c496e24060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:14.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.513+0000 7fe36cfd2700 1 -- 172.21.15.5:0/1747975816 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c496e14d00 con 0x55c496e186a0 2024-05-09T00:57:14.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.519+0000 7fe36cfd2700 1 --2- 172.21.15.5:0/1747975816 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c496e186a0 0x55c496e39c00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe350010f50 tx=0x7fe350000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44c5fd9bb2032e97 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:57:14.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.523+0000 7fe35dffb700 1 -- 172.21.15.5:0/1747975816 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe350015070 con 0x55c496e186a0 2024-05-09T00:57:14.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.523+0000 7fe35dffb700 1 -- 172.21.15.5:0/1747975816 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe35002da50 con 0x55c496e186a0 2024-05-09T00:57:14.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.523+0000 7fe35dffb700 1 -- 172.21.15.5:0/1747975816 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe350027050 con 0x55c496e186a0 2024-05-09T00:57:14.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.526+0000 7fe37d44bb80 1 -- 172.21.15.5:0/1747975816 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c496e186a0 msgr2=0x55c496e39c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:14.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.526+0000 7fe37d44bb80 1 --2- 172.21.15.5:0/1747975816 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c496e186a0 0x55c496e39c00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe350010f50 tx=0x7fe350000bc0 comp rx=0 tx=0).stop 2024-05-09T00:57:14.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.527+0000 7fe37d44bb80 1 -- 172.21.15.5:0/1747975816 shutdown_connections 2024-05-09T00:57:14.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.527+0000 7fe37d44bb80 1 --2- 172.21.15.5:0/1747975816 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c496e186a0 0x55c496e39c00 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:14.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.527+0000 7fe37d44bb80 1 --2- 172.21.15.5:0/1747975816 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c496c9b4a0 0x55c496e22b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:14.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.527+0000 7fe37d44bb80 1 --2- 172.21.15.5:0/1747975816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c496d85160 0x55c496e24060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:14.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.527+0000 7fe37d44bb80 1 -- 172.21.15.5:0/1747975816 >> 172.21.15.5:0/1747975816 conn(0x55c496cfb6a0 msgr2=0x55c496e23950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:57:14.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.528+0000 7fe37d44bb80 1 -- 172.21.15.5:0/1747975816 shutdown_connections 2024-05-09T00:57:14.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.528+0000 7fe37d44bb80 1 -- 172.21.15.5:0/1747975816 wait complete. 2024-05-09T00:57:14.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.530+0000 7fe37d44bb80 1 -- start start 2024-05-09T00:57:14.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.530+0000 7fe37d44bb80 1 Processor -- start 2024-05-09T00:57:14.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.533+0000 7fe37d44bb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c496c9b4a0 0x55c496e25700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:14.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.533+0000 7fe37d44bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c496d85160 0x55c496e25c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:14.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.533+0000 7fe37d44bb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c496e186a0 0x55c496e9a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:14.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.533+0000 7fe37d44bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe3681b67a0 con 0x55c496d85160 2024-05-09T00:57:14.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.533+0000 7fe37d44bb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe3681b60d0 con 0x55c496e186a0 2024-05-09T00:57:14.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.533+0000 7fe37d44bb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe3681b32d0 con 0x55c496c9b4a0 2024-05-09T00:57:14.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.533+0000 7fe37d44bb80 10 client.0 _ll_register_callbacks cb 0x55c496bac6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:57:14.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.535+0000 7fe36cfd2700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c496e186a0 0x55c496e9a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:14.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.535+0000 7fe36cfd2700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c496e186a0 0x55c496e9a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:45508/0 (socket says 172.21.15.5:45508) 2024-05-09T00:57:14.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.535+0000 7fe36cfd2700 1 -- 172.21.15.5:0/44742797 learned_addr learned my addr 172.21.15.5:0/44742797 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:57:14.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.536+0000 7fe36cfd2700 1 -- 172.21.15.5:0/44742797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c496c9b4a0 msgr2=0x55c496e25700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:57:14.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.536+0000 7fe36cfd2700 1 --2- 172.21.15.5:0/44742797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c496c9b4a0 0x55c496e25700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:14.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.536+0000 7fe36cfd2700 1 -- 172.21.15.5:0/44742797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c496d85160 msgr2=0x55c496e25c40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:57:14.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.536+0000 7fe36cfd2700 1 --2- 172.21.15.5:0/44742797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c496d85160 0x55c496e25c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:14.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.536+0000 7fe36cfd2700 1 -- 172.21.15.5:0/44742797 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c496e1ddc0 con 0x55c496e186a0 2024-05-09T00:57:14.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.536+0000 7fe36cfd2700 1 --2- 172.21.15.5:0/44742797 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c496e186a0 0x55c496e9a7a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe35000b010 tx=0x7fe350006c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27efc1c03b73edc3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:57:14.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.537+0000 7fe3477fe700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-09T00:57:14.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.537+0000 7fe3477fe700 1 -- 172.21.15.5:0/44742797 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe350015070 con 0x55c496e186a0 2024-05-09T00:57:14.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.537+0000 7fe37d44bb80 1 -- 172.21.15.5:0/44742797 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe35002da50 con 0x55c496e186a0 2024-05-09T00:57:14.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.537+0000 7fe3477fe700 1 -- 172.21.15.5:0/44742797 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fe350023850 con 0x55c496e186a0 2024-05-09T00:57:14.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.538+0000 7fe3477fe700 1 -- 172.21.15.5:0/44742797 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe350027040 con 0x55c496e186a0 2024-05-09T00:57:14.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.540+0000 7fe37d44bb80 10 client.26075 fetch_fsmap learned FSMap version 31 2024-05-09T00:57:14.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.540+0000 7fe37d44bb80 1 -- 172.21.15.5:0/44742797 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c496eacd90 con 0x55c496e186a0 2024-05-09T00:57:14.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.544+0000 7fe3477fe700 1 -- 172.21.15.5:0/44742797 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe35002da50 con 0x55c496e186a0 2024-05-09T00:57:14.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.546+0000 7fe3477fe700 1 -- 172.21.15.5:0/44742797 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fe350035ae0 con 0x55c496e186a0 2024-05-09T00:57:14.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.546+0000 7fe37d44bb80 10 client.26075 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:57:14.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.546+0000 7fe37d44bb80 10 client.26075 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:57:14.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.546+0000 7fe37d44bb80 20 client.26075 populate_metadata read hostname 'smithi005' 2024-05-09T00:57:14.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.546+0000 7fe37d44bb80 1 --2- 172.21.15.5:0/44742797 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55c496ead320 0x55c496eaf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:14.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.546+0000 7fe37d44bb80 4 client.26075 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:57:14.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.546+0000 7fe37d44bb80 1 -- 172.21.15.5:0/44742797 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c496d489d0 con 0x55c496ead320 2024-05-09T00:57:14.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.567+0000 7fe367fff700 1 --2- 172.21.15.5:0/44742797 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55c496ead320 0x55c496eaf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:14.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.583+0000 7fe367fff700 1 --2- 172.21.15.5:0/44742797 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55c496ead320 0x55c496eaf7c0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25f66ddc1373ffd5 server_cookie=86eac84c245f1fe7 in_seq=0 out_seq=0 2024-05-09T00:57:14.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.587+0000 7fe3477fe700 10 client.26075 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:57:14.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.598+0000 7fe3477fe700 1 -- 172.21.15.5:0/44742797 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c496d489d0 con 0x55c496ead320 2024-05-09T00:57:14.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.598+0000 7fe3477fe700 10 client.26075 handle_command_reply: tid=0 2024-05-09T00:57:14.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.598+0000 7fe37d44bb80 1 client.26075 shutdown 2024-05-09T00:57:14.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.598+0000 7fe37d44bb80 1 -- 172.21.15.5:0/44742797 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c496e186a0 msgr2=0x55c496e9a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:14.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.598+0000 7fe37d44bb80 1 --2- 172.21.15.5:0/44742797 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c496e186a0 0x55c496e9a7a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe35000b010 tx=0x7fe350006c00 comp rx=0 tx=0).stop 2024-05-09T00:57:14.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.598+0000 7fe37d44bb80 1 -- 172.21.15.5:0/44742797 shutdown_connections 2024-05-09T00:57:14.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.598+0000 7fe37d44bb80 1 --2- 172.21.15.5:0/44742797 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55c496ead320 0x55c496eaf7c0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:14.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.599+0000 7fe37d44bb80 1 --2- 172.21.15.5:0/44742797 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c496e186a0 0x55c496e9a7a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:14.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.599+0000 7fe37d44bb80 1 --2- 172.21.15.5:0/44742797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c496c9b4a0 0x55c496e25700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:14.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.599+0000 7fe37d44bb80 1 --2- 172.21.15.5:0/44742797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c496d85160 0x55c496e25c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:14.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.599+0000 7fe37d44bb80 1 -- 172.21.15.5:0/44742797 >> 172.21.15.5:0/44742797 conn(0x55c496cfb6a0 msgr2=0x55c496e128e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:57:14.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.601+0000 7fe3477fe700 0 client.26075 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:57:14.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.604+0000 7fe37d44bb80 1 -- 172.21.15.5:0/44742797 shutdown_connections 2024-05-09T00:57:14.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.605+0000 7fe37d44bb80 1 -- 172.21.15.5:0/44742797 wait complete. 2024-05-09T00:57:14.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.606+0000 7fe37d44bb80 20 client.26075 trim_cache size 0 max 16384 2024-05-09T00:57:14.611 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-09T00:57:14.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.610+0000 7fe36ea35700 1 -- 172.21.15.5:0/1734576932 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe3540b2980 msgr2=0x7fe3540b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:14.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.610+0000 7fe36ea35700 1 --2- 172.21.15.5:0/1734576932 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe3540b2980 0x7fe3540b4e90 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe368100af0 tx=0x7fe358009250 comp rx=0 tx=0).stop 2024-05-09T00:57:14.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.610+0000 7fe36ea35700 1 -- 172.21.15.5:0/1734576932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe368103250 msgr2=0x7fe3681a1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:14.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.610+0000 7fe36ea35700 1 --2- 172.21.15.5:0/1734576932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe368103250 0x7fe3681a1640 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fe360009ee0 tx=0x7fe360005590 comp rx=0 tx=0).stop 2024-05-09T00:57:14.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.612+0000 7fe36ea35700 1 -- 172.21.15.5:0/1734576932 shutdown_connections 2024-05-09T00:57:14.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.612+0000 7fe36ea35700 1 --2- 172.21.15.5:0/1734576932 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe3540b2980 0x7fe3540b4e90 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:14.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.612+0000 7fe36ea35700 1 --2- 172.21.15.5:0/1734576932 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3681086a0 0x7fe3681a5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:14.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.612+0000 7fe36ea35700 1 --2- 172.21.15.5:0/1734576932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe368103250 0x7fe3681a1640 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:14.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.612+0000 7fe36ea35700 1 --2- 172.21.15.5:0/1734576932 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe368103ba0 0x7fe3681a1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:14.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.612+0000 7fe36ea35700 1 -- 172.21.15.5:0/1734576932 >> 172.21.15.5:0/1734576932 conn(0x7fe3680fedc0 msgr2=0x7fe3681060a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:57:14.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.613+0000 7fe36ea35700 1 -- 172.21.15.5:0/1734576932 shutdown_connections 2024-05-09T00:57:14.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:14.613+0000 7fe36ea35700 1 -- 172.21.15.5:0/1734576932 wait complete. 2024-05-09T00:57:15.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:57:15 smithi131 ceph-mon[34547]: pgmap v698: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 901 MiB/s wr, 1.93k op/s 2024-05-09T00:57:15.930 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:57:15 smithi005 ceph-mon[30365]: pgmap v698: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 901 MiB/s wr, 1.93k op/s 2024-05-09T00:57:15.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:57:15 smithi157 ceph-mon[34294]: pgmap v698: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 901 MiB/s wr, 1.93k op/s 2024-05-09T00:57:17.765 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T00:57:17.766 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T00:57:17.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:57:17 smithi131 ceph-mon[34547]: pgmap v699: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 957 MiB/s wr, 2.19k op/s 2024-05-09T00:57:17.912 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:57:17 smithi005 ceph-mon[30365]: pgmap v699: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 957 MiB/s wr, 2.19k op/s 2024-05-09T00:57:17.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:57:17 smithi157 ceph-mon[34294]: pgmap v699: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 957 MiB/s wr, 2.19k op/s 2024-05-09T00:57:18.579 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:57:19.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:57:19 smithi157 ceph-mon[34294]: pgmap v700: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 832 MiB/s wr, 1.87k op/s 2024-05-09T00:57:20.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:57:19 smithi005 ceph-mon[30365]: pgmap v700: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 832 MiB/s wr, 1.87k op/s 2024-05-09T00:57:20.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:57:19 smithi131 ceph-mon[34547]: pgmap v700: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 832 MiB/s wr, 1.87k op/s 2024-05-09T00:57:21.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:57:21 smithi157 ceph-mon[34294]: pgmap v701: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 992 MiB/s wr, 2.03k op/s 2024-05-09T00:57:22.040 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:57:21 smithi005 ceph-mon[30365]: pgmap v701: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 992 MiB/s wr, 2.03k op/s 2024-05-09T00:57:22.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:57:21 smithi131 ceph-mon[34547]: pgmap v701: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 992 MiB/s wr, 2.03k op/s 2024-05-09T00:57:23.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:23.503+0000 7efff9ffb640 1 -- 172.21.15.5:0/2743885473 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e305224350 con 0x55e3053bce70 2024-05-09T00:57:23.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:23.503+0000 7efff9ffb640 10 client.26069 handle_command_reply: tid=0 2024-05-09T00:57:23.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:23.504+0000 7f001f82e740 1 client.26069 shutdown 2024-05-09T00:57:23.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:23.504+0000 7f001f82e740 1 -- 172.21.15.5:0/2743885473 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e304f991b0 msgr2=0x55e30534be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:23.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:23.504+0000 7f001f82e740 1 --2- 172.21.15.5:0/2743885473 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e304f991b0 0x55e30534be90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f000c0a9cd0 tx=0x7f000c014d20 comp rx=0 tx=0).stop 2024-05-09T00:57:23.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:23.504+0000 7f001f82e740 1 -- 172.21.15.5:0/2743885473 shutdown_connections 2024-05-09T00:57:23.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:23.504+0000 7f001f82e740 1 --2- 172.21.15.5:0/2743885473 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55e3053bce70 0x55e3053bf300 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:23.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:23.504+0000 7f001f82e740 1 --2- 172.21.15.5:0/2743885473 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e304f991b0 0x55e30534be90 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:23.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:23.504+0000 7f001f82e740 1 --2- 172.21.15.5:0/2743885473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e304f702f0 0x55e30534b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:23.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:23.504+0000 7f001f82e740 1 --2- 172.21.15.5:0/2743885473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e304f57d80 0x55e30534edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:23.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:23.504+0000 7f001f82e740 1 -- 172.21.15.5:0/2743885473 >> 172.21.15.5:0/2743885473 conn(0x55e304f8d300 msgr2=0x55e305345bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:57:23.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:23.504+0000 7efff9ffb640 0 client.26069 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:57:23.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:23.504+0000 7f001f82e740 1 -- 172.21.15.5:0/2743885473 shutdown_connections 2024-05-09T00:57:23.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:23.504+0000 7f001f82e740 1 -- 172.21.15.5:0/2743885473 wait complete. 2024-05-09T00:57:23.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:23.505+0000 7f001f82e740 20 client.26069 trim_cache size 0 max 16384 2024-05-09T00:57:23.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-09T00:57:23.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-09T00:57:23.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-09T00:57:23.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:23.506+0000 7f001359e640 1 -- 172.21.15.5:0/1360853298 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7efffc0b29f0 msgr2=0x7efffc0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:23.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:23.506+0000 7f001359e640 1 --2- 172.21.15.5:0/1360853298 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7efffc0b29f0 0x7efffc0b4f00 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f000402aa30 tx=0x7f0004004aa0 comp rx=0 tx=0).stop 2024-05-09T00:57:23.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:23.506+0000 7f001359e640 1 -- 172.21.15.5:0/1360853298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f001410ffe0 msgr2=0x7f001411a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:23.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:23.506+0000 7f001359e640 1 --2- 172.21.15.5:0/1360853298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f001410ffe0 0x7f001411a5f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f000c00b970 tx=0x7f000c00be30 comp rx=0 tx=0).stop 2024-05-09T00:57:23.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:23.507+0000 7f001359e640 1 -- 172.21.15.5:0/1360853298 shutdown_connections 2024-05-09T00:57:23.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:23.507+0000 7f001359e640 1 --2- 172.21.15.5:0/1360853298 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7efffc0b29f0 0x7efffc0b4f00 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:23.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:23.507+0000 7f001359e640 1 --2- 172.21.15.5:0/1360853298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f001410ffe0 0x7f001411a5f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:23.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:23.507+0000 7f001359e640 1 --2- 172.21.15.5:0/1360853298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f001410b2c0 0x7f0014119690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:23.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:23.507+0000 7f001359e640 1 --2- 172.21.15.5:0/1360853298 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f001410a470 0x7f0014119150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:23.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:23.507+0000 7f001359e640 1 -- 172.21.15.5:0/1360853298 >> 172.21.15.5:0/1360853298 conn(0x7f0014070460 msgr2=0x7f001410dfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:57:23.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:23.507+0000 7f001359e640 1 -- 172.21.15.5:0/1360853298 shutdown_connections 2024-05-09T00:57:23.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:23.507+0000 7f001359e640 1 -- 172.21.15.5:0/1360853298 wait complete. 2024-05-09T00:57:23.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:57:23 smithi005 ceph-mon[30365]: pgmap v702: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 802 MiB/s wr, 1.68k op/s 2024-05-09T00:57:23.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:57:23 smithi157 ceph-mon[34294]: pgmap v702: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 802 MiB/s wr, 1.68k op/s 2024-05-09T00:57:24.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.015+0000 7f59e2998700 1 -- 172.21.15.5:0/3642658220 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59dc10b020 msgr2=0x7f59dc10b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:24.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.015+0000 7f59e2998700 1 --2- 172.21.15.5:0/3642658220 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59dc10b020 0x7f59dc10b430 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f59cc00ba80 tx=0x7f59cc00bd90 comp rx=0 tx=0).stop 2024-05-09T00:57:24.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.015+0000 7f59e2998700 1 -- 172.21.15.5:0/3642658220 shutdown_connections 2024-05-09T00:57:24.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.015+0000 7f59e2998700 1 --2- 172.21.15.5:0/3642658220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59dc1102a0 0x7f59dc117c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:24.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.015+0000 7f59e2998700 1 --2- 172.21.15.5:0/3642658220 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59dc10b020 0x7f59dc10b430 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:24.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.015+0000 7f59e2998700 1 --2- 172.21.15.5:0/3642658220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59dc10f950 0x7f59dc10fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:24.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.015+0000 7f59e2998700 1 -- 172.21.15.5:0/3642658220 >> 172.21.15.5:0/3642658220 conn(0x7f59dc06f990 msgr2=0x7f59dc06fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:57:24.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.017+0000 7f59e2998700 1 -- 172.21.15.5:0/3642658220 shutdown_connections 2024-05-09T00:57:24.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.018+0000 7f59e2998700 1 -- 172.21.15.5:0/3642658220 wait complete. 2024-05-09T00:57:24.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.019+0000 7f59e2998700 1 Processor -- start 2024-05-09T00:57:24.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.020+0000 7f59e2998700 1 -- start start 2024-05-09T00:57:24.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.021+0000 7f59e2998700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59dc10b020 0x7f59dc1a14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:24.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.021+0000 7f59e2998700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59dc10f950 0x7f59dc1a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:24.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.021+0000 7f59e2998700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59dc1102a0 0x7f59dc1a5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:24.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.021+0000 7f59e2998700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f59dc11aac0 con 0x7f59dc10f950 2024-05-09T00:57:24.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.021+0000 7f59e2998700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f59dc11a940 con 0x7f59dc1102a0 2024-05-09T00:57:24.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.021+0000 7f59e2998700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f59dc11ac40 con 0x7f59dc10b020 2024-05-09T00:57:24.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.021+0000 7f59db7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59dc10f950 0x7f59dc1a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:24.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.021+0000 7f59db7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59dc10f950 0x7f59dc1a1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40238/0 (socket says 172.21.15.5:40238) 2024-05-09T00:57:24.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.021+0000 7f59db7fe700 1 -- 172.21.15.5:0/671996800 learned_addr learned my addr 172.21.15.5:0/671996800 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:57:24.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.023+0000 7f59dbfff700 1 --2- 172.21.15.5:0/671996800 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59dc10b020 0x7f59dc1a14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:24.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.026+0000 7f59db7fe700 1 -- 172.21.15.5:0/671996800 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59dc10b020 msgr2=0x7f59dc1a14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:24.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.026+0000 7f59db7fe700 1 --2- 172.21.15.5:0/671996800 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59dc10b020 0x7f59dc1a14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:24.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.026+0000 7f59db7fe700 1 -- 172.21.15.5:0/671996800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59dc1102a0 msgr2=0x7f59dc1a5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:24.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.026+0000 7f59db7fe700 1 --2- 172.21.15.5:0/671996800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59dc1102a0 0x7f59dc1a5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:24.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.026+0000 7f59db7fe700 1 -- 172.21.15.5:0/671996800 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59dc1a6380 con 0x7f59dc10f950 2024-05-09T00:57:24.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.026+0000 7f59db7fe700 1 --2- 172.21.15.5:0/671996800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59dc10f950 0x7f59dc1a1a50 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f59d000e970 tx=0x7f59d000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:57:24.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.032+0000 7f59d97fa700 1 -- 172.21.15.5:0/671996800 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d000cbd0 con 0x7f59dc10f950 2024-05-09T00:57:24.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.032+0000 7f59d97fa700 1 -- 172.21.15.5:0/671996800 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59d000cd70 con 0x7f59dc10f950 2024-05-09T00:57:24.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.032+0000 7f59d97fa700 1 -- 172.21.15.5:0/671996800 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d0010430 con 0x7f59dc10f950 2024-05-09T00:57:24.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.032+0000 7f59e2998700 1 -- 172.21.15.5:0/671996800 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59dc1a6670 con 0x7f59dc10f950 2024-05-09T00:57:24.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.032+0000 7f59e2998700 1 -- 172.21.15.5:0/671996800 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59dc1a6b00 con 0x7f59dc10f950 2024-05-09T00:57:24.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.048+0000 7f59d97fa700 1 -- 172.21.15.5:0/671996800 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f59d00105d0 con 0x7f59dc10f950 2024-05-09T00:57:24.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.057+0000 7f59f13aeb80 1 Processor -- start 2024-05-09T00:57:24.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.059+0000 7f59d97fa700 1 --2- 172.21.15.5:0/671996800 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f59c40b6e60 0x7f59c40b9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:24.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.060+0000 7f59d97fa700 1 -- 172.21.15.5:0/671996800 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f59d00e3420 con 0x7f59dc10f950 2024-05-09T00:57:24.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.060+0000 7f59f13aeb80 1 -- start start 2024-05-09T00:57:24.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.061+0000 7f59e0f35700 1 --2- 172.21.15.5:0/671996800 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f59c40b6e60 0x7f59c40b9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:24.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.061+0000 7f59f13aeb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe25a6bd80 0x55fe25b742c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:24.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.061+0000 7f59f13aeb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe259f1520 0x55fe25b74800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:24.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.061+0000 7f59f13aeb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe25b6e480 0x55fe25b6e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:24.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.061+0000 7f59dbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe25a6bd80 0x55fe25b742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:24.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.061+0000 7f59dbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe25a6bd80 0x55fe25b742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40254/0 (socket says 172.21.15.5:40254) 2024-05-09T00:57:24.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.061+0000 7f59dbfff700 1 -- 172.21.15.5:0/1712864323 learned_addr learned my addr 172.21.15.5:0/1712864323 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:57:24.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.061+0000 7f59f13aeb80 1 -- 172.21.15.5:0/1712864323 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fe25a91120 con 0x55fe25a6bd80 2024-05-09T00:57:24.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.061+0000 7f59f13aeb80 1 -- 172.21.15.5:0/1712864323 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55fe25a36a20 con 0x55fe259f1520 2024-05-09T00:57:24.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.061+0000 7f59f13aeb80 1 -- 172.21.15.5:0/1712864323 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55fe25a8d0e0 con 0x55fe25b6e480 2024-05-09T00:57:24.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.063+0000 7f59db7fe700 1 --2- 172.21.15.5:0/1712864323 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe259f1520 0x55fe25b74800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:24.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.063+0000 7f59e0f35700 1 --2- 172.21.15.5:0/1712864323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe25b6e480 0x55fe25b6e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:24.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.064+0000 7f59dbfff700 1 -- 172.21.15.5:0/1712864323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe25b6e480 msgr2=0x55fe25b6e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:57:24.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.064+0000 7f59dbfff700 1 --2- 172.21.15.5:0/1712864323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe25b6e480 0x55fe25b6e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:24.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.064+0000 7f59dbfff700 1 -- 172.21.15.5:0/1712864323 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe259f1520 msgr2=0x55fe25b74800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:57:24.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.064+0000 7f59dbfff700 1 --2- 172.21.15.5:0/1712864323 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe259f1520 0x55fe25b74800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:24.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.064+0000 7f59dbfff700 1 -- 172.21.15.5:0/1712864323 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe25a27580 con 0x55fe25a6bd80 2024-05-09T00:57:24.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.065+0000 7f59db7fe700 1 --2- 172.21.15.5:0/1712864323 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe259f1520 0x55fe25b74800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:57:24.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.065+0000 7f59e0f35700 1 --2- 172.21.15.5:0/671996800 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f59c40b6e60 0x7f59c40b9370 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f59d4003cf0 tx=0x7f59d40089d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:57:24.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.065+0000 7f59dbfff700 1 --2- 172.21.15.5:0/1712864323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe25a6bd80 0x55fe25b742c0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f59cc00b850 tx=0x7f59cc005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bf5b52750b9f240 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:57:24.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.071+0000 7f59c27fc700 1 -- 172.21.15.5:0/1712864323 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59cc021070 con 0x55fe25a6bd80 2024-05-09T00:57:24.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.080+0000 7f59c27fc700 1 -- 172.21.15.5:0/1712864323 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59cc019040 con 0x55fe25a6bd80 2024-05-09T00:57:24.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.080+0000 7f59c27fc700 1 -- 172.21.15.5:0/1712864323 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59cc005570 con 0x55fe25a6bd80 2024-05-09T00:57:24.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.083+0000 7f59f13aeb80 1 -- 172.21.15.5:0/1712864323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe25a6bd80 msgr2=0x55fe25b742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:24.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.083+0000 7f59f13aeb80 1 --2- 172.21.15.5:0/1712864323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe25a6bd80 0x55fe25b742c0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f59cc00b850 tx=0x7f59cc005e60 comp rx=0 tx=0).stop 2024-05-09T00:57:24.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.085+0000 7f59f13aeb80 1 -- 172.21.15.5:0/1712864323 shutdown_connections 2024-05-09T00:57:24.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.085+0000 7f59f13aeb80 1 --2- 172.21.15.5:0/1712864323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe25b6e480 0x55fe25b6e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:24.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.085+0000 7f59f13aeb80 1 --2- 172.21.15.5:0/1712864323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe25a6bd80 0x55fe25b742c0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:24.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.085+0000 7f59f13aeb80 1 --2- 172.21.15.5:0/1712864323 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe259f1520 0x55fe25b74800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:24.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.085+0000 7f59f13aeb80 1 -- 172.21.15.5:0/1712864323 >> 172.21.15.5:0/1712864323 conn(0x55fe259f4ff0 msgr2=0x55fe25b692a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:57:24.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.086+0000 7f59f13aeb80 1 -- 172.21.15.5:0/1712864323 shutdown_connections 2024-05-09T00:57:24.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.086+0000 7f59f13aeb80 1 -- 172.21.15.5:0/1712864323 wait complete. 2024-05-09T00:57:24.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.087+0000 7f59f13aeb80 1 -- start start 2024-05-09T00:57:24.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.088+0000 7f59f13aeb80 1 Processor -- start 2024-05-09T00:57:24.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.090+0000 7f59f13aeb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe259f1520 0x55fe25b78050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:24.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.090+0000 7f59f13aeb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe25a6bd80 0x55fe25b78590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:24.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.090+0000 7f59f13aeb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe25b6e480 0x55fe25b78ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:24.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.090+0000 7f59f13aeb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fe25b6f180 con 0x55fe25b6e480 2024-05-09T00:57:24.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.091+0000 7f59f13aeb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55fe25a2bba0 con 0x55fe25a6bd80 2024-05-09T00:57:24.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.091+0000 7f59f13aeb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55fe25b84280 con 0x55fe259f1520 2024-05-09T00:57:24.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.091+0000 7f59f13aeb80 10 client.0 _ll_register_callbacks cb 0x55fe25902740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:57:24.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.092+0000 7f59db7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe25a6bd80 0x55fe25b78590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:24.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.093+0000 7f59dbfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe259f1520 0x55fe25b78050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:24.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.093+0000 7f59dbfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe259f1520 0x55fe25b78050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:55892/0 (socket says 172.21.15.5:55892) 2024-05-09T00:57:24.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.093+0000 7f59dbfff700 1 -- 172.21.15.5:0/3986469595 learned_addr learned my addr 172.21.15.5:0/3986469595 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:57:24.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.093+0000 7f59e0f35700 1 --2- 172.21.15.5:0/3986469595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe25b6e480 0x55fe25b78ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:24.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.096+0000 7f59db7fe700 1 -- 172.21.15.5:0/3986469595 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe259f1520 msgr2=0x55fe25b78050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:57:24.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.096+0000 7f59db7fe700 1 --2- 172.21.15.5:0/3986469595 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe259f1520 0x55fe25b78050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:24.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.096+0000 7f59db7fe700 1 -- 172.21.15.5:0/3986469595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe25b6e480 msgr2=0x55fe25b78ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:57:24.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.096+0000 7f59db7fe700 1 --2- 172.21.15.5:0/3986469595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe25b6e480 0x55fe25b78ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:24.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.096+0000 7f59db7fe700 1 -- 172.21.15.5:0/3986469595 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe2590c510 con 0x55fe25a6bd80 2024-05-09T00:57:24.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.098+0000 7f59db7fe700 1 --2- 172.21.15.5:0/3986469595 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe25a6bd80 0x55fe25b78590 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f59d00e0e90 tx=0x7f59d00aaf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54fbb2c35d3d14c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:57:24.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:57:23 smithi131 ceph-mon[34547]: pgmap v702: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 802 MiB/s wr, 1.68k op/s 2024-05-09T00:57:24.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.101+0000 7f59bb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-09T00:57:24.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.101+0000 7f59bb7fe700 1 -- 172.21.15.5:0/3986469595 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d00c3070 con 0x55fe25a6bd80 2024-05-09T00:57:24.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.101+0000 7f59bb7fe700 1 -- 172.21.15.5:0/3986469595 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f59d00a6fa0 con 0x55fe25a6bd80 2024-05-09T00:57:24.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.102+0000 7f59f13aeb80 1 -- 172.21.15.5:0/3986469595 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59cc019040 con 0x55fe25a6bd80 2024-05-09T00:57:24.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.103+0000 7f59bb7fe700 1 -- 172.21.15.5:0/3986469595 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f59d00aa4c0 con 0x55fe25a6bd80 2024-05-09T00:57:24.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.104+0000 7f59f13aeb80 10 client.26084 fetch_fsmap learned FSMap version 31 2024-05-09T00:57:24.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.104+0000 7f59f13aeb80 1 -- 172.21.15.5:0/3986469595 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55fe25b859c0 con 0x55fe25a6bd80 2024-05-09T00:57:24.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.106+0000 7f59bb7fe700 1 -- 172.21.15.5:0/3986469595 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59d00ab0b0 con 0x55fe25a6bd80 2024-05-09T00:57:24.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.106+0000 7f59bb7fe700 1 -- 172.21.15.5:0/3986469595 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f59d00abce0 con 0x55fe25a6bd80 2024-05-09T00:57:24.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.108+0000 7f59f13aeb80 10 client.26084 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:57:24.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.108+0000 7f59f13aeb80 10 client.26084 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:57:24.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.108+0000 7f59f13aeb80 20 client.26084 populate_metadata read hostname 'smithi005' 2024-05-09T00:57:24.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.108+0000 7f59f13aeb80 1 --2- 172.21.15.5:0/3986469595 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55fe25b88000 0x55fe25b8a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:24.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.108+0000 7f59f13aeb80 4 client.26084 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:57:24.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.108+0000 7f59f13aeb80 1 -- 172.21.15.5:0/3986469595 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f59cc019040 con 0x55fe25b88000 2024-05-09T00:57:24.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.113+0000 7f59dbfff700 1 --2- 172.21.15.5:0/3986469595 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55fe25b88000 0x55fe25b8a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:24.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.120+0000 7f59dbfff700 1 --2- 172.21.15.5:0/3986469595 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55fe25b88000 0x55fe25b8a420 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f948d627d4a83167 server_cookie=6eb313362299e967 in_seq=0 out_seq=0 2024-05-09T00:57:24.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.123+0000 7f59bb7fe700 10 client.26084 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:57:24.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.127+0000 7f59bb7fe700 1 -- 172.21.15.5:0/3986469595 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f59cc019040 con 0x55fe25b88000 2024-05-09T00:57:24.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.127+0000 7f59bb7fe700 10 client.26084 handle_command_reply: tid=0 2024-05-09T00:57:24.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.127+0000 7f59f13aeb80 1 client.26084 shutdown 2024-05-09T00:57:24.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.128+0000 7f59f13aeb80 1 -- 172.21.15.5:0/3986469595 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe25a6bd80 msgr2=0x55fe25b78590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:24.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.128+0000 7f59f13aeb80 1 --2- 172.21.15.5:0/3986469595 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe25a6bd80 0x55fe25b78590 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f59d00e0e90 tx=0x7f59d00aaf10 comp rx=0 tx=0).stop 2024-05-09T00:57:24.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.132+0000 7f59f13aeb80 1 -- 172.21.15.5:0/3986469595 shutdown_connections 2024-05-09T00:57:24.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.132+0000 7f59f13aeb80 1 --2- 172.21.15.5:0/3986469595 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55fe25b88000 0x55fe25b8a420 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:24.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.132+0000 7f59f13aeb80 1 --2- 172.21.15.5:0/3986469595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe25b6e480 0x55fe25b78ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:24.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.132+0000 7f59f13aeb80 1 --2- 172.21.15.5:0/3986469595 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe259f1520 0x55fe25b78050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:24.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.132+0000 7f59f13aeb80 1 --2- 172.21.15.5:0/3986469595 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe25a6bd80 0x55fe25b78590 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:24.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.132+0000 7f59f13aeb80 1 -- 172.21.15.5:0/3986469595 >> 172.21.15.5:0/3986469595 conn(0x55fe259f4ff0 msgr2=0x55fe25b63070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:57:24.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.133+0000 7f59bb7fe700 0 client.26084 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:57:24.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.133+0000 7f59f13aeb80 1 -- 172.21.15.5:0/3986469595 shutdown_connections 2024-05-09T00:57:24.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.134+0000 7f59f13aeb80 1 -- 172.21.15.5:0/3986469595 wait complete. 2024-05-09T00:57:24.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.135+0000 7f59f13aeb80 20 client.26084 trim_cache size 0 max 16384 2024-05-09T00:57:24.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.190+0000 7f59f13aeb80 1 Processor -- start 2024-05-09T00:57:24.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.190+0000 7f59f13aeb80 1 -- start start 2024-05-09T00:57:24.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.190+0000 7f59f13aeb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe259f1520 0x55fe25b6dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:24.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.190+0000 7f59f13aeb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe25a6bd80 0x55fe25b7a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:24.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.190+0000 7f59f13aeb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe25b6e480 0x55fe25b78910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:24.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.191+0000 7f59f13aeb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f59c800d540 con 0x55fe25b6e480 2024-05-09T00:57:24.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.191+0000 7f59f13aeb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f59c800f080 con 0x55fe259f1520 2024-05-09T00:57:24.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.191+0000 7f59f13aeb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f59c800dc10 con 0x55fe25a6bd80 2024-05-09T00:57:24.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.192+0000 7f59e0f35700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe25b6e480 0x55fe25b78910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:24.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.192+0000 7f59e0f35700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe25b6e480 0x55fe25b78910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40266/0 (socket says 172.21.15.5:40266) 2024-05-09T00:57:24.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.192+0000 7f59e0f35700 1 -- 172.21.15.5:0/3186570787 learned_addr learned my addr 172.21.15.5:0/3186570787 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:57:24.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.193+0000 7f59e0f35700 1 -- 172.21.15.5:0/3186570787 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe25a6bd80 msgr2=0x55fe25b7a4c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:57:24.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.193+0000 7f59e0f35700 1 --2- 172.21.15.5:0/3186570787 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe25a6bd80 0x55fe25b7a4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:24.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.193+0000 7f59e0f35700 1 -- 172.21.15.5:0/3186570787 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe259f1520 msgr2=0x55fe25b6dea0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:57:24.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.193+0000 7f59dbfff700 1 --2- 172.21.15.5:0/3186570787 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe259f1520 0x55fe25b6dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:24.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.193+0000 7f59e0f35700 1 --2- 172.21.15.5:0/3186570787 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe259f1520 0x55fe25b6dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:24.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.193+0000 7f59e0f35700 1 -- 172.21.15.5:0/3186570787 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe25b69fc0 con 0x55fe25b6e480 2024-05-09T00:57:24.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.193+0000 7f59e0f35700 1 --2- 172.21.15.5:0/3186570787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe25b6e480 0x55fe25b78910 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f59d4007c40 tx=0x7f59d401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2331b2749ae25f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:57:24.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.194+0000 7f59c1ffb700 1 -- 172.21.15.5:0/3186570787 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d4008600 con 0x55fe25b6e480 2024-05-09T00:57:24.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.194+0000 7f59c1ffb700 1 -- 172.21.15.5:0/3186570787 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59d4026070 con 0x55fe25b6e480 2024-05-09T00:57:24.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.195+0000 7f59f13aeb80 1 -- 172.21.15.5:0/3186570787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe25b6e480 msgr2=0x55fe25b78910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:24.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.195+0000 7f59f13aeb80 1 --2- 172.21.15.5:0/3186570787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe25b6e480 0x55fe25b78910 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f59d4007c40 tx=0x7f59d401b000 comp rx=0 tx=0).stop 2024-05-09T00:57:24.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.196+0000 7f59f13aeb80 1 -- 172.21.15.5:0/3186570787 shutdown_connections 2024-05-09T00:57:24.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.196+0000 7f59f13aeb80 1 --2- 172.21.15.5:0/3186570787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe25b6e480 0x55fe25b78910 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:24.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.196+0000 7f59f13aeb80 1 --2- 172.21.15.5:0/3186570787 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe259f1520 0x55fe25b6dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:24.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.196+0000 7f59f13aeb80 1 --2- 172.21.15.5:0/3186570787 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe25a6bd80 0x55fe25b7a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:24.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.196+0000 7f59f13aeb80 1 -- 172.21.15.5:0/3186570787 >> 172.21.15.5:0/3186570787 conn(0x55fe259f4ff0 msgr2=0x55fe25b636e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:57:24.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.198+0000 7f59f13aeb80 1 -- 172.21.15.5:0/3186570787 shutdown_connections 2024-05-09T00:57:24.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.199+0000 7f59f13aeb80 1 -- 172.21.15.5:0/3186570787 wait complete. 2024-05-09T00:57:24.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.200+0000 7f59f13aeb80 1 -- start start 2024-05-09T00:57:24.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.200+0000 7f59f13aeb80 1 Processor -- start 2024-05-09T00:57:24.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.201+0000 7f59f13aeb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe259f1520 0x55fe25b80540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:24.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.201+0000 7f59f13aeb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe25a6bd80 0x55fe25b7c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:24.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.201+0000 7f59f13aeb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe25b7cdf0 0x55fe25b7d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:24.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.201+0000 7f59f13aeb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f59c800e9b0 con 0x55fe25a6bd80 2024-05-09T00:57:24.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.201+0000 7f59f13aeb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f59c800e2e0 con 0x55fe25b7cdf0 2024-05-09T00:57:24.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.201+0000 7f59f13aeb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f59c800b4e0 con 0x55fe259f1520 2024-05-09T00:57:24.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.201+0000 7f59f13aeb80 10 client.0 _ll_register_callbacks cb 0x55fe25902740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:57:24.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.202+0000 7f59db7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe25a6bd80 0x55fe25b7c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:24.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.202+0000 7f59db7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe25a6bd80 0x55fe25b7c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40278/0 (socket says 172.21.15.5:40278) 2024-05-09T00:57:24.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.202+0000 7f59db7fe700 1 -- 172.21.15.5:0/1883864928 learned_addr learned my addr 172.21.15.5:0/1883864928 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:57:24.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.203+0000 7f59e0f35700 1 --2- 172.21.15.5:0/1883864928 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe25b7cdf0 0x55fe25b7d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:24.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.203+0000 7f59dbfff700 1 --2- 172.21.15.5:0/1883864928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe259f1520 0x55fe25b80540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:24.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.204+0000 7f59e0f35700 1 -- 172.21.15.5:0/1883864928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe259f1520 msgr2=0x55fe25b80540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:57:24.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.204+0000 7f59e0f35700 1 --2- 172.21.15.5:0/1883864928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe259f1520 0x55fe25b80540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:24.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.204+0000 7f59e0f35700 1 -- 172.21.15.5:0/1883864928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe25a6bd80 msgr2=0x55fe25b7c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:57:24.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.204+0000 7f59e0f35700 1 --2- 172.21.15.5:0/1883864928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe25a6bd80 0x55fe25b7c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:24.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.204+0000 7f59e0f35700 1 -- 172.21.15.5:0/1883864928 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe25bf2780 con 0x55fe25b7cdf0 2024-05-09T00:57:24.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.204+0000 7f59e0f35700 1 --2- 172.21.15.5:0/1883864928 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe25b7cdf0 0x55fe25b7d200 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x55fe25b6e450 tx=0x7f59d4004a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54db567f44ab9cff server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:57:24.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.204+0000 7f59bb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-09T00:57:24.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.205+0000 7f59bb7fe700 1 -- 172.21.15.5:0/1883864928 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d402f070 con 0x55fe25b7cdf0 2024-05-09T00:57:24.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.205+0000 7f59f13aeb80 1 -- 172.21.15.5:0/1883864928 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55fe25a9eb00 con 0x55fe25b7cdf0 2024-05-09T00:57:24.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.205+0000 7f59bb7fe700 1 -- 172.21.15.5:0/1883864928 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f59d4004db0 con 0x55fe25b7cdf0 2024-05-09T00:57:24.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.206+0000 7f59bb7fe700 1 -- 172.21.15.5:0/1883864928 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f59d4008770 con 0x55fe25b7cdf0 2024-05-09T00:57:24.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.206+0000 7f59f13aeb80 10 client.26087 fetch_fsmap learned FSMap version 31 2024-05-09T00:57:24.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.206+0000 7f59f13aeb80 1 -- 172.21.15.5:0/1883864928 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55fe25c04960 con 0x55fe25b7cdf0 2024-05-09T00:57:24.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.207+0000 7f59bb7fe700 1 -- 172.21.15.5:0/1883864928 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55fe25a9eb00 con 0x55fe25b7cdf0 2024-05-09T00:57:24.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.207+0000 7f59bb7fe700 1 -- 172.21.15.5:0/1883864928 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f59d4032030 con 0x55fe25b7cdf0 2024-05-09T00:57:24.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.207+0000 7f59f13aeb80 10 client.26087 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:57:24.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.207+0000 7f59f13aeb80 10 client.26087 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:57:24.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.207+0000 7f59f13aeb80 20 client.26087 populate_metadata read hostname 'smithi005' 2024-05-09T00:57:24.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.207+0000 7f59f13aeb80 1 --2- 172.21.15.5:0/1883864928 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55fe25c05060 0x55fe25c07480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:24.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.207+0000 7f59f13aeb80 4 client.26087 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T00:57:24.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.207+0000 7f59f13aeb80 1 -- 172.21.15.5:0/1883864928 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55fe25a98ba0 con 0x55fe25c05060 2024-05-09T00:57:24.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.212+0000 7f59dbfff700 1 --2- 172.21.15.5:0/1883864928 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55fe25c05060 0x55fe25c07480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:24.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.214+0000 7f59dbfff700 1 --2- 172.21.15.5:0/1883864928 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55fe25c05060 0x55fe25c07480 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e586f5f0e8e0e697 server_cookie=e7bde8ca2b08683c in_seq=0 out_seq=0 2024-05-09T00:57:24.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.215+0000 7f59bb7fe700 10 client.26087 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:57:24.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.581+0000 7f59bb7fe700 1 -- 172.21.15.5:0/1883864928 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55fe25a98ba0 con 0x55fe25c05060 2024-05-09T00:57:24.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.581+0000 7f59bb7fe700 10 client.26087 handle_command_reply: tid=0 2024-05-09T00:57:24.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.581+0000 7f59f13aeb80 1 client.26087 shutdown 2024-05-09T00:57:24.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.582+0000 7f59f13aeb80 1 -- 172.21.15.5:0/1883864928 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe25b7cdf0 msgr2=0x55fe25b7d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:24.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.582+0000 7f59f13aeb80 1 --2- 172.21.15.5:0/1883864928 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe25b7cdf0 0x55fe25b7d200 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x55fe25b6e450 tx=0x7f59d4004a20 comp rx=0 tx=0).stop 2024-05-09T00:57:24.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.582+0000 7f59f13aeb80 1 -- 172.21.15.5:0/1883864928 shutdown_connections 2024-05-09T00:57:24.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.582+0000 7f59f13aeb80 1 --2- 172.21.15.5:0/1883864928 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55fe25c05060 0x55fe25c07480 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:24.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.582+0000 7f59f13aeb80 1 --2- 172.21.15.5:0/1883864928 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe25b7cdf0 0x55fe25b7d200 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:24.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.582+0000 7f59f13aeb80 1 --2- 172.21.15.5:0/1883864928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fe259f1520 0x55fe25b80540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:24.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.582+0000 7f59f13aeb80 1 --2- 172.21.15.5:0/1883864928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe25a6bd80 0x55fe25b7c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:24.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.582+0000 7f59f13aeb80 1 -- 172.21.15.5:0/1883864928 >> 172.21.15.5:0/1883864928 conn(0x55fe259f4ff0 msgr2=0x55fe25b71c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:57:24.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.582+0000 7f59bb7fe700 0 client.26087 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:57:24.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.583+0000 7f59f13aeb80 1 -- 172.21.15.5:0/1883864928 shutdown_connections 2024-05-09T00:57:24.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.583+0000 7f59f13aeb80 1 -- 172.21.15.5:0/1883864928 wait complete. 2024-05-09T00:57:24.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.584+0000 7f59f13aeb80 20 client.26087 trim_cache size 0 max 16384 2024-05-09T00:57:24.586 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-09T00:57:24.587 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-05-09T00:57:24.587 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "f47d73fa-4eb2-42d6-8058-05f1f3f5d575", 2024-05-09T00:57:24.587 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-05-09T00:57:24.587 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-09T00:57:24.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.585+0000 7f59e2998700 1 -- 172.21.15.5:0/671996800 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f59c40b6e60 msgr2=0x7f59c40b9370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:24.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.585+0000 7f59e2998700 1 --2- 172.21.15.5:0/671996800 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f59c40b6e60 0x7f59c40b9370 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f59d4003cf0 tx=0x7f59d40089d0 comp rx=0 tx=0).stop 2024-05-09T00:57:24.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.585+0000 7f59e2998700 1 -- 172.21.15.5:0/671996800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59dc10f950 msgr2=0x7f59dc1a1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:24.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.585+0000 7f59e2998700 1 --2- 172.21.15.5:0/671996800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59dc10f950 0x7f59dc1a1a50 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f59d000e970 tx=0x7f59d000ed30 comp rx=0 tx=0).stop 2024-05-09T00:57:24.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.585+0000 7f59e2998700 1 -- 172.21.15.5:0/671996800 shutdown_connections 2024-05-09T00:57:24.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.585+0000 7f59e2998700 1 --2- 172.21.15.5:0/671996800 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f59c40b6e60 0x7f59c40b9370 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:24.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.585+0000 7f59e2998700 1 --2- 172.21.15.5:0/671996800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59dc1102a0 0x7f59dc1a5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:24.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.585+0000 7f59e2998700 1 --2- 172.21.15.5:0/671996800 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59dc10b020 0x7f59dc1a14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:24.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.585+0000 7f59e2998700 1 --2- 172.21.15.5:0/671996800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59dc10f950 0x7f59dc1a1a50 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:24.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.585+0000 7f59e2998700 1 -- 172.21.15.5:0/671996800 >> 172.21.15.5:0/671996800 conn(0x7f59dc06f990 msgr2=0x7f59dc10e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:57:24.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.586+0000 7f59e2998700 1 -- 172.21.15.5:0/671996800 shutdown_connections 2024-05-09T00:57:24.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:24.586+0000 7f59e2998700 1 -- 172.21.15.5:0/671996800 wait complete. 2024-05-09T00:57:25.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:57:25 smithi157 ceph-mon[34294]: pgmap v703: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 896 MiB/s wr, 1.93k op/s 2024-05-09T00:57:25.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:57:25 smithi157 ceph-mon[34294]: scrub queued for path: / 2024-05-09T00:57:25.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:57:25 smithi157 ceph-mon[34294]: scrub summary: idle+waiting paths [/] 2024-05-09T00:57:25.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:57:25 smithi157 ceph-mon[34294]: scrub summary: active paths [/] 2024-05-09T00:57:26.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:57:25 smithi005 ceph-mon[30365]: pgmap v703: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 896 MiB/s wr, 1.93k op/s 2024-05-09T00:57:26.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:57:25 smithi005 ceph-mon[30365]: scrub queued for path: / 2024-05-09T00:57:26.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:57:25 smithi005 ceph-mon[30365]: scrub summary: idle+waiting paths [/] 2024-05-09T00:57:26.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:57:25 smithi005 ceph-mon[30365]: scrub summary: active paths [/] 2024-05-09T00:57:26.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:57:25 smithi131 ceph-mon[34547]: pgmap v703: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 896 MiB/s wr, 1.93k op/s 2024-05-09T00:57:26.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:57:25 smithi131 ceph-mon[34547]: scrub queued for path: / 2024-05-09T00:57:26.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:57:25 smithi131 ceph-mon[34547]: scrub summary: idle+waiting paths [/] 2024-05-09T00:57:26.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:57:25 smithi131 ceph-mon[34547]: scrub summary: active paths [/] 2024-05-09T00:57:27.229 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T00:57:27.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:57:27 smithi157 ceph-mon[34294]: scrub summary: active paths [/] 2024-05-09T00:57:27.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:57:27 smithi157 ceph-mon[34294]: scrub summary: idle+waiting paths [/] 2024-05-09T00:57:27.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:57:27 smithi157 ceph-mon[34294]: pgmap v704: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 920 MiB/s wr, 1.96k op/s 2024-05-09T00:57:27.549 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:57:27 smithi005 ceph-mon[30365]: scrub summary: active paths [/] 2024-05-09T00:57:27.549 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:57:27 smithi005 ceph-mon[30365]: scrub summary: idle+waiting paths [/] 2024-05-09T00:57:27.549 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:57:27 smithi005 ceph-mon[30365]: pgmap v704: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 920 MiB/s wr, 1.96k op/s 2024-05-09T00:57:27.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:57:27 smithi131 ceph-mon[34547]: scrub summary: active paths [/] 2024-05-09T00:57:27.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:57:27 smithi131 ceph-mon[34547]: scrub summary: idle+waiting paths [/] 2024-05-09T00:57:27.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:57:27 smithi131 ceph-mon[34547]: pgmap v704: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 920 MiB/s wr, 1.96k op/s 2024-05-09T00:57:28.260 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:57:29.790 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:57:29 smithi005 ceph-mon[30365]: pgmap v705: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 837 MiB/s wr, 1.77k op/s 2024-05-09T00:57:29.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:57:29 smithi005 ceph-mon[30365]: scrub summary: idle+waiting paths [/] 2024-05-09T00:57:29.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:57:29 smithi131 ceph-mon[34547]: pgmap v705: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 837 MiB/s wr, 1.77k op/s 2024-05-09T00:57:29.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:57:29 smithi131 ceph-mon[34547]: scrub summary: idle+waiting paths [/] 2024-05-09T00:57:29.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:57:29 smithi157 ceph-mon[34294]: pgmap v705: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 837 MiB/s wr, 1.77k op/s 2024-05-09T00:57:29.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:57:29 smithi157 ceph-mon[34294]: scrub summary: idle+waiting paths [/] 2024-05-09T00:57:31.797 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:57:31 smithi005 ceph-mon[30365]: pgmap v706: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 962 MiB/s wr, 2.19k op/s 2024-05-09T00:57:31.797 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:57:31 smithi005 ceph-mon[30365]: scrub summary: idle 2024-05-09T00:57:31.797 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:57:31 smithi005 ceph-mon[30365]: scrub summary: idle 2024-05-09T00:57:31.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:57:31 smithi131 ceph-mon[34547]: pgmap v706: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 962 MiB/s wr, 2.19k op/s 2024-05-09T00:57:31.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:57:31 smithi131 ceph-mon[34547]: scrub summary: idle 2024-05-09T00:57:31.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:57:31 smithi131 ceph-mon[34547]: scrub summary: idle 2024-05-09T00:57:31.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:57:31 smithi157 ceph-mon[34294]: pgmap v706: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 962 MiB/s wr, 2.19k op/s 2024-05-09T00:57:31.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:57:31 smithi157 ceph-mon[34294]: scrub summary: idle 2024-05-09T00:57:31.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:57:31 smithi157 ceph-mon[34294]: scrub summary: idle 2024-05-09T00:57:32.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:57:32 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:57:32.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:57:32 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:57:32.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:57:32 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:57:33.591 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:57:33 smithi005 ceph-mon[30365]: pgmap v707: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 800 MiB/s wr, 1.98k op/s 2024-05-09T00:57:33.591 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:57:33 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:57:33.591 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:57:33 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:57:33.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:57:33 smithi131 ceph-mon[34547]: pgmap v707: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 800 MiB/s wr, 1.98k op/s 2024-05-09T00:57:33.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:57:33 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:57:33.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:57:33 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:57:33.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:57:33 smithi157 ceph-mon[34294]: pgmap v707: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 800 MiB/s wr, 1.98k op/s 2024-05-09T00:57:33.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:57:33 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:57:33.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:57:33 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:57:34.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.726+0000 7f5ba23ae700 1 -- 172.21.15.5:0/1923128668 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b9c116eb0 msgr2=0x7f5b9c1193c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:34.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.726+0000 7f5ba23ae700 1 --2- 172.21.15.5:0/1923128668 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b9c116eb0 0x7f5b9c1193c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5b98009990 tx=0x7f5b98009ca0 comp rx=0 tx=0).stop 2024-05-09T00:57:34.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.726+0000 7f5ba23ae700 1 -- 172.21.15.5:0/1923128668 shutdown_connections 2024-05-09T00:57:34.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.726+0000 7f5ba23ae700 1 --2- 172.21.15.5:0/1923128668 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b9c116eb0 0x7f5b9c1193c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:34.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.726+0000 7f5ba23ae700 1 --2- 172.21.15.5:0/1923128668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b9c10aa50 0x7f5b9c10ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:34.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.726+0000 7f5ba23ae700 1 --2- 172.21.15.5:0/1923128668 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b9c10b3a0 0x7f5b9c116820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:34.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.726+0000 7f5ba23ae700 1 -- 172.21.15.5:0/1923128668 >> 172.21.15.5:0/1923128668 conn(0x7f5b9c06f260 msgr2=0x7f5b9c06f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:57:34.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.730+0000 7f5ba23ae700 1 -- 172.21.15.5:0/1923128668 shutdown_connections 2024-05-09T00:57:34.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.731+0000 7f5ba23ae700 1 -- 172.21.15.5:0/1923128668 wait complete. 2024-05-09T00:57:34.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.733+0000 7f5ba23ae700 1 Processor -- start 2024-05-09T00:57:34.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.733+0000 7f5ba23ae700 1 -- start start 2024-05-09T00:57:34.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.733+0000 7f5ba23ae700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b9c10aa50 0x7f5b9c1aa840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:34.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.733+0000 7f5ba23ae700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b9c10b3a0 0x7f5b9c1aad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:34.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.733+0000 7f5ba23ae700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b9c116eb0 0x7f5b9c1b58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:34.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.733+0000 7f5ba23ae700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c11cb60 con 0x7f5b9c116eb0 2024-05-09T00:57:34.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.733+0000 7f5ba23ae700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c11c9e0 con 0x7f5b9c10b3a0 2024-05-09T00:57:34.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.733+0000 7f5ba23ae700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c11cce0 con 0x7f5b9c10aa50 2024-05-09T00:57:34.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.738+0000 7f5ba1bad700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b9c116eb0 0x7f5b9c1b58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:34.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.738+0000 7f5ba1bad700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b9c116eb0 0x7f5b9c1b58c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38376/0 (socket says 172.21.15.5:38376) 2024-05-09T00:57:34.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.738+0000 7f5ba1bad700 1 -- 172.21.15.5:0/2733774943 learned_addr learned my addr 172.21.15.5:0/2733774943 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:57:34.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.740+0000 7f5ba0bab700 1 --2- 172.21.15.5:0/2733774943 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b9c10b3a0 0x7f5b9c1aad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:34.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.741+0000 7f5ba13ac700 1 --2- 172.21.15.5:0/2733774943 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b9c10aa50 0x7f5b9c1aa840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:34.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.745+0000 7f5ba1bad700 1 -- 172.21.15.5:0/2733774943 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b9c10aa50 msgr2=0x7f5b9c1aa840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:34.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.745+0000 7f5ba1bad700 1 --2- 172.21.15.5:0/2733774943 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b9c10aa50 0x7f5b9c1aa840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:34.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.745+0000 7f5ba1bad700 1 -- 172.21.15.5:0/2733774943 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b9c10b3a0 msgr2=0x7f5b9c1aad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:34.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.745+0000 7f5ba1bad700 1 --2- 172.21.15.5:0/2733774943 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b9c10b3a0 0x7f5b9c1aad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:34.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.745+0000 7f5ba1bad700 1 -- 172.21.15.5:0/2733774943 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b9c1b5e00 con 0x7f5b9c116eb0 2024-05-09T00:57:34.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.749+0000 7f5ba1bad700 1 --2- 172.21.15.5:0/2733774943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b9c116eb0 0x7f5b9c1b58c0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f5b98009fb0 tx=0x7f5b98005590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:57:34.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.750+0000 7f5b927fc700 1 -- 172.21.15.5:0/2733774943 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b98003b00 con 0x7f5b9c116eb0 2024-05-09T00:57:34.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.750+0000 7f5b927fc700 1 -- 172.21.15.5:0/2733774943 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b98004150 con 0x7f5b9c116eb0 2024-05-09T00:57:34.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.750+0000 7f5ba23ae700 1 -- 172.21.15.5:0/2733774943 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b9c1b5fd0 con 0x7f5b9c116eb0 2024-05-09T00:57:34.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.750+0000 7f5b927fc700 1 -- 172.21.15.5:0/2733774943 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b980173f0 con 0x7f5b9c116eb0 2024-05-09T00:57:34.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.750+0000 7f5ba23ae700 1 -- 172.21.15.5:0/2733774943 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b9c1b6370 con 0x7f5b9c116eb0 2024-05-09T00:57:34.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.762+0000 7f5bb2026b80 1 Processor -- start 2024-05-09T00:57:34.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.762+0000 7f5b927fc700 1 -- 172.21.15.5:0/2733774943 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f5b98003ca0 con 0x7f5b9c116eb0 2024-05-09T00:57:34.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.764+0000 7f5b927fc700 1 --2- 172.21.15.5:0/2733774943 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f5b880b29c0 0x7f5b880b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:34.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.764+0000 7f5b927fc700 1 -- 172.21.15.5:0/2733774943 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f5b980e3aa0 con 0x7f5b9c116eb0 2024-05-09T00:57:34.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.764+0000 7f5ba0bab700 1 --2- 172.21.15.5:0/2733774943 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f5b880b29c0 0x7f5b880b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:34.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.765+0000 7f5bb2026b80 1 -- start start 2024-05-09T00:57:34.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.766+0000 7f5bb2026b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560eda6d0d00 0x560eda7d3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:34.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.766+0000 7f5bb2026b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560eda6564a0 0x560eda7d3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:34.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.766+0000 7f5bb2026b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560eda7d4160 0x560eda7f0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:34.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.766+0000 7f5bb2026b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560eda6f60a0 con 0x560eda6564a0 2024-05-09T00:57:34.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.767+0000 7f5ba0bab700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560eda6564a0 0x560eda7d3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:34.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.767+0000 7f5ba0bab700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560eda6564a0 0x560eda7d3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38384/0 (socket says 172.21.15.5:38384) 2024-05-09T00:57:34.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.767+0000 7f5ba0bab700 1 -- 172.21.15.5:0/415430767 learned_addr learned my addr 172.21.15.5:0/415430767 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:57:34.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.770+0000 7f5ba13ac700 1 --2- 172.21.15.5:0/415430767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560eda6d0d00 0x560eda7d3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:34.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.771+0000 7f5ba1bad700 1 --2- 172.21.15.5:0/415430767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560eda7d4160 0x560eda7f0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:34.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.772+0000 7f5bb2026b80 1 -- 172.21.15.5:0/415430767 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x560eda69b9a0 con 0x560eda6d0d00 2024-05-09T00:57:34.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.772+0000 7f5bb2026b80 1 -- 172.21.15.5:0/415430767 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x560eda6f2060 con 0x560eda7d4160 2024-05-09T00:57:34.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.772+0000 7f5ba0bab700 1 --2- 172.21.15.5:0/2733774943 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f5b880b29c0 0x7f5b880b4ed0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5b94009550 tx=0x7f5b940091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:57:34.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.778+0000 7f5ba13ac700 1 -- 172.21.15.5:0/415430767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560eda7d4160 msgr2=0x560eda7f0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:57:34.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.778+0000 7f5ba13ac700 1 --2- 172.21.15.5:0/415430767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560eda7d4160 0x560eda7f0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:34.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.778+0000 7f5ba13ac700 1 -- 172.21.15.5:0/415430767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560eda6564a0 msgr2=0x560eda7d3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:57:34.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.778+0000 7f5ba13ac700 1 --2- 172.21.15.5:0/415430767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560eda6564a0 0x560eda7d3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:34.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.778+0000 7f5ba13ac700 1 -- 172.21.15.5:0/415430767 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560eda68c500 con 0x560eda6d0d00 2024-05-09T00:57:34.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.778+0000 7f5ba0bab700 1 --2- 172.21.15.5:0/415430767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560eda6564a0 0x560eda7d3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:57:34.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.781+0000 7f5ba13ac700 1 --2- 172.21.15.5:0/415430767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560eda6d0d00 0x560eda7d3620 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c00a870 tx=0x7f5b8c00ab80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a388e74053fecac server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:57:34.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.783+0000 7f5b877fe700 1 -- 172.21.15.5:0/415430767 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c015070 con 0x560eda6d0d00 2024-05-09T00:57:34.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.783+0000 7f5b877fe700 1 -- 172.21.15.5:0/415430767 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b8c011cb0 con 0x560eda6d0d00 2024-05-09T00:57:34.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.783+0000 7f5b877fe700 1 -- 172.21.15.5:0/415430767 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c00cdd0 con 0x560eda6d0d00 2024-05-09T00:57:34.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.785+0000 7f5bb2026b80 1 -- 172.21.15.5:0/415430767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560eda6d0d00 msgr2=0x560eda7d3620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:34.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.785+0000 7f5bb2026b80 1 --2- 172.21.15.5:0/415430767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560eda6d0d00 0x560eda7d3620 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c00a870 tx=0x7f5b8c00ab80 comp rx=0 tx=0).stop 2024-05-09T00:57:34.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.786+0000 7f5bb2026b80 1 -- 172.21.15.5:0/415430767 shutdown_connections 2024-05-09T00:57:34.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.786+0000 7f5bb2026b80 1 --2- 172.21.15.5:0/415430767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560eda7d4160 0x560eda7f0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:34.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.786+0000 7f5bb2026b80 1 --2- 172.21.15.5:0/415430767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560eda6d0d00 0x560eda7d3620 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:34.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.786+0000 7f5bb2026b80 1 --2- 172.21.15.5:0/415430767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560eda6564a0 0x560eda7d3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:34.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.786+0000 7f5bb2026b80 1 -- 172.21.15.5:0/415430767 >> 172.21.15.5:0/415430767 conn(0x560eda659f70 msgr2=0x560eda7ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:57:34.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.791+0000 7f5bb2026b80 1 -- 172.21.15.5:0/415430767 shutdown_connections 2024-05-09T00:57:34.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.791+0000 7f5bb2026b80 1 -- 172.21.15.5:0/415430767 wait complete. 2024-05-09T00:57:34.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.792+0000 7f5bb2026b80 1 -- start start 2024-05-09T00:57:34.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.792+0000 7f5bb2026b80 1 Processor -- start 2024-05-09T00:57:34.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.793+0000 7f5bb2026b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560eda6564a0 0x560eda7edb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:34.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.793+0000 7f5bb2026b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560eda6d0d00 0x560eda7ee070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:34.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.793+0000 7f5bb2026b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560eda7d4160 0x560eda7ee5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:34.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.794+0000 7f5bb2026b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560eda7f0f70 con 0x560eda7d4160 2024-05-09T00:57:34.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.794+0000 7f5bb2026b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x560eda690b20 con 0x560eda6d0d00 2024-05-09T00:57:34.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.794+0000 7f5bb2026b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x560eda7f10f0 con 0x560eda6564a0 2024-05-09T00:57:34.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.794+0000 7f5bb2026b80 10 client.0 _ll_register_callbacks cb 0x560eda5676c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:57:34.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.795+0000 7f5ba0bab700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560eda6d0d00 0x560eda7ee070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:34.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.795+0000 7f5ba1bad700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560eda7d4160 0x560eda7ee5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:34.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.795+0000 7f5ba1bad700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560eda7d4160 0x560eda7ee5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38392/0 (socket says 172.21.15.5:38392) 2024-05-09T00:57:34.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.795+0000 7f5ba1bad700 1 -- 172.21.15.5:0/1606998271 learned_addr learned my addr 172.21.15.5:0/1606998271 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:57:34.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.795+0000 7f5ba13ac700 1 --2- 172.21.15.5:0/1606998271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560eda6564a0 0x560eda7edb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:34.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.797+0000 7f5ba1bad700 1 -- 172.21.15.5:0/1606998271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560eda6564a0 msgr2=0x560eda7edb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:57:34.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.797+0000 7f5ba1bad700 1 --2- 172.21.15.5:0/1606998271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560eda6564a0 0x560eda7edb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:34.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.797+0000 7f5ba1bad700 1 -- 172.21.15.5:0/1606998271 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560eda6d0d00 msgr2=0x560eda7ee070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:57:34.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.797+0000 7f5ba1bad700 1 --2- 172.21.15.5:0/1606998271 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560eda6d0d00 0x560eda7ee070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:34.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.797+0000 7f5ba1bad700 1 -- 172.21.15.5:0/1606998271 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560eda571490 con 0x560eda7d4160 2024-05-09T00:57:34.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.801+0000 7f5ba1bad700 1 --2- 172.21.15.5:0/1606998271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560eda7d4160 0x560eda7ee5b0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f5b98005810 tx=0x7f5b9800f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db20163fc2ac9fc2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:57:34.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.801+0000 7f5b84ff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:57:34.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.802+0000 7f5b84ff9700 1 -- 172.21.15.5:0/1606998271 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b980aabd0 con 0x560eda7d4160 2024-05-09T00:57:34.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.803+0000 7f5b84ff9700 1 -- 172.21.15.5:0/1606998271 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f5b980a76b0 con 0x560eda7d4160 2024-05-09T00:57:34.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.803+0000 7f5bb2026b80 1 -- 172.21.15.5:0/1606998271 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b8c011cb0 con 0x560eda7d4160 2024-05-09T00:57:34.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.803+0000 7f5b84ff9700 1 -- 172.21.15.5:0/1606998271 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5b980cc050 con 0x560eda7d4160 2024-05-09T00:57:34.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.803+0000 7f5bb2026b80 10 client.16644 fetch_fsmap learned FSMap version 31 2024-05-09T00:57:34.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.803+0000 7f5bb2026b80 1 -- 172.21.15.5:0/1606998271 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560eda7fbc60 con 0x560eda7d4160 2024-05-09T00:57:34.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.804+0000 7f5b84ff9700 1 -- 172.21.15.5:0/1606998271 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b980ab7c0 con 0x560eda7d4160 2024-05-09T00:57:34.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.804+0000 7f5b84ff9700 1 -- 172.21.15.5:0/1606998271 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f5b9802a070 con 0x560eda7d4160 2024-05-09T00:57:34.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.804+0000 7f5bb2026b80 10 client.16644 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:57:34.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.804+0000 7f5bb2026b80 10 client.16644 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:57:34.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.804+0000 7f5bb2026b80 20 client.16644 populate_metadata read hostname 'smithi005' 2024-05-09T00:57:34.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.805+0000 7f5bb2026b80 1 --2- 172.21.15.5:0/1606998271 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x560eda7fe130 0x560eda8005a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:34.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.805+0000 7f5bb2026b80 4 client.16644 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:57:34.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.805+0000 7f5bb2026b80 1 -- 172.21.15.5:0/1606998271 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5b8c011cb0 con 0x560eda7fe130 2024-05-09T00:57:34.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.806+0000 7f5ba13ac700 1 --2- 172.21.15.5:0/1606998271 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x560eda7fe130 0x560eda8005a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:34.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.808+0000 7f5ba13ac700 1 --2- 172.21.15.5:0/1606998271 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x560eda7fe130 0x560eda8005a0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=142deb8ed4d09ab0 server_cookie=b3d53aee18cd9873 in_seq=0 out_seq=0 2024-05-09T00:57:34.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.808+0000 7f5b84ff9700 10 client.16644 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:57:34.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.815+0000 7f5b84ff9700 1 -- 172.21.15.5:0/1606998271 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f5b8c011cb0 con 0x560eda7fe130 2024-05-09T00:57:34.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.815+0000 7f5b84ff9700 10 client.16644 handle_command_reply: tid=0 2024-05-09T00:57:34.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.815+0000 7f5bb2026b80 1 client.16644 shutdown 2024-05-09T00:57:34.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.816+0000 7f5bb2026b80 1 -- 172.21.15.5:0/1606998271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560eda7d4160 msgr2=0x560eda7ee5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:34.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.816+0000 7f5bb2026b80 1 --2- 172.21.15.5:0/1606998271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560eda7d4160 0x560eda7ee5b0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f5b98005810 tx=0x7f5b9800f840 comp rx=0 tx=0).stop 2024-05-09T00:57:34.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.816+0000 7f5bb2026b80 1 -- 172.21.15.5:0/1606998271 shutdown_connections 2024-05-09T00:57:34.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.816+0000 7f5bb2026b80 1 --2- 172.21.15.5:0/1606998271 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x560eda7fe130 0x560eda8005a0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:34.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.817+0000 7f5bb2026b80 1 --2- 172.21.15.5:0/1606998271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560eda7d4160 0x560eda7ee5b0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:34.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.817+0000 7f5bb2026b80 1 --2- 172.21.15.5:0/1606998271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560eda6564a0 0x560eda7edb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:34.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.817+0000 7f5b84ff9700 0 client.16644 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:57:34.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.817+0000 7f5bb2026b80 1 --2- 172.21.15.5:0/1606998271 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560eda6d0d00 0x560eda7ee070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:34.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.817+0000 7f5bb2026b80 1 -- 172.21.15.5:0/1606998271 >> 172.21.15.5:0/1606998271 conn(0x560eda659f70 msgr2=0x560eda7ce2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:57:34.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.818+0000 7f5bb2026b80 1 -- 172.21.15.5:0/1606998271 shutdown_connections 2024-05-09T00:57:34.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.822+0000 7f5bb2026b80 1 -- 172.21.15.5:0/1606998271 wait complete. 2024-05-09T00:57:34.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.823+0000 7f5bb2026b80 20 client.16644 trim_cache size 0 max 16384 2024-05-09T00:57:34.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.898+0000 7f5bb2026b80 1 Processor -- start 2024-05-09T00:57:34.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.898+0000 7f5bb2026b80 1 -- start start 2024-05-09T00:57:34.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.899+0000 7f5bb2026b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560eda6564a0 0x560eda7cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:34.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.899+0000 7f5bb2026b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560eda6d0d00 0x560eda7d2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:34.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.899+0000 7f5bb2026b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560eda7d4160 0x560eda7ee020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:34.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.900+0000 7f5bb2026b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c1a9190 con 0x560eda6d0d00 2024-05-09T00:57:34.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.900+0000 7f5bb2026b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c1b6af0 con 0x560eda6564a0 2024-05-09T00:57:34.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.900+0000 7f5bb2026b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c1a9860 con 0x560eda7d4160 2024-05-09T00:57:34.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.901+0000 7f5ba0bab700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560eda6d0d00 0x560eda7d2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:34.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.901+0000 7f5ba0bab700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560eda6d0d00 0x560eda7d2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38404/0 (socket says 172.21.15.5:38404) 2024-05-09T00:57:34.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.901+0000 7f5ba0bab700 1 -- 172.21.15.5:0/3752387979 learned_addr learned my addr 172.21.15.5:0/3752387979 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:57:34.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.901+0000 7f5ba13ac700 1 --2- 172.21.15.5:0/3752387979 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560eda6564a0 0x560eda7cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:34.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.901+0000 7f5ba1bad700 1 --2- 172.21.15.5:0/3752387979 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560eda7d4160 0x560eda7ee020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:34.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.903+0000 7f5ba0bab700 1 -- 172.21.15.5:0/3752387979 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560eda7d4160 msgr2=0x560eda7ee020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:57:34.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.903+0000 7f5ba0bab700 1 --2- 172.21.15.5:0/3752387979 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560eda7d4160 0x560eda7ee020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:34.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.903+0000 7f5ba0bab700 1 -- 172.21.15.5:0/3752387979 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560eda6564a0 msgr2=0x560eda7cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:57:34.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.903+0000 7f5ba0bab700 1 --2- 172.21.15.5:0/3752387979 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560eda6564a0 0x560eda7cdef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:34.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.903+0000 7f5ba0bab700 1 -- 172.21.15.5:0/3752387979 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560eda7ca970 con 0x560eda6d0d00 2024-05-09T00:57:34.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.904+0000 7f5ba0bab700 1 --2- 172.21.15.5:0/3752387979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560eda6d0d00 0x560eda7d2e20 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f5b940047b0 tx=0x7f5b94013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b515dbe387d9c133 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:57:34.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.905+0000 7f5b86ffd700 1 -- 172.21.15.5:0/3752387979 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b9401c040 con 0x560eda6d0d00 2024-05-09T00:57:34.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.905+0000 7f5b86ffd700 1 -- 172.21.15.5:0/3752387979 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b9401d040 con 0x560eda6d0d00 2024-05-09T00:57:34.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.905+0000 7f5b86ffd700 1 -- 172.21.15.5:0/3752387979 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b940156a0 con 0x560eda6d0d00 2024-05-09T00:57:34.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.907+0000 7f5bb2026b80 1 -- 172.21.15.5:0/3752387979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560eda6d0d00 msgr2=0x560eda7d2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:34.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.907+0000 7f5bb2026b80 1 --2- 172.21.15.5:0/3752387979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560eda6d0d00 0x560eda7d2e20 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f5b940047b0 tx=0x7f5b94013000 comp rx=0 tx=0).stop 2024-05-09T00:57:34.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.907+0000 7f5bb2026b80 1 -- 172.21.15.5:0/3752387979 shutdown_connections 2024-05-09T00:57:34.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.907+0000 7f5bb2026b80 1 --2- 172.21.15.5:0/3752387979 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560eda7d4160 0x560eda7ee020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:34.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.907+0000 7f5bb2026b80 1 --2- 172.21.15.5:0/3752387979 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560eda6564a0 0x560eda7cdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:34.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.907+0000 7f5bb2026b80 1 --2- 172.21.15.5:0/3752387979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560eda6d0d00 0x560eda7d2e20 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:34.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.907+0000 7f5bb2026b80 1 -- 172.21.15.5:0/3752387979 >> 172.21.15.5:0/3752387979 conn(0x560eda659f70 msgr2=0x560eda7ed910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:57:34.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.908+0000 7f5bb2026b80 1 -- 172.21.15.5:0/3752387979 shutdown_connections 2024-05-09T00:57:34.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.908+0000 7f5bb2026b80 1 -- 172.21.15.5:0/3752387979 wait complete. 2024-05-09T00:57:34.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.909+0000 7f5bb2026b80 1 -- start start 2024-05-09T00:57:34.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.909+0000 7f5bb2026b80 1 Processor -- start 2024-05-09T00:57:34.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.910+0000 7f5bb2026b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560eda6564a0 0x560eda7f6850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:34.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.911+0000 7f5bb2026b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560eda6d0d00 0x560eda7f2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:34.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.911+0000 7f5bb2026b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560eda7d4160 0x560eda7f3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:34.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.911+0000 7f5bb2026b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c1aa600 con 0x560eda6564a0 2024-05-09T00:57:34.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.911+0000 7f5bb2026b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c1a9f30 con 0x560eda6d0d00 2024-05-09T00:57:34.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.911+0000 7f5bb2026b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c1a7130 con 0x560eda7d4160 2024-05-09T00:57:34.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.911+0000 7f5bb2026b80 10 client.0 _ll_register_callbacks cb 0x560eda5676c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:57:34.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.911+0000 7f5ba13ac700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560eda6564a0 0x560eda7f6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:34.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.911+0000 7f5ba13ac700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560eda6564a0 0x560eda7f6850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38416/0 (socket says 172.21.15.5:38416) 2024-05-09T00:57:34.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.911+0000 7f5ba13ac700 1 -- 172.21.15.5:0/1682294500 learned_addr learned my addr 172.21.15.5:0/1682294500 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:57:34.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.912+0000 7f5ba13ac700 1 -- 172.21.15.5:0/1682294500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560eda7d4160 msgr2=0x560eda7f3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:57:34.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.912+0000 7f5ba13ac700 1 --2- 172.21.15.5:0/1682294500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560eda7d4160 0x560eda7f3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:34.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.912+0000 7f5ba13ac700 1 -- 172.21.15.5:0/1682294500 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560eda6d0d00 msgr2=0x560eda7f2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:57:34.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.912+0000 7f5ba13ac700 1 --2- 172.21.15.5:0/1682294500 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560eda6d0d00 0x560eda7f2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:34.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.912+0000 7f5ba13ac700 1 -- 172.21.15.5:0/1682294500 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560eda798ca0 con 0x560eda6564a0 2024-05-09T00:57:34.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.914+0000 7f5ba13ac700 1 --2- 172.21.15.5:0/1682294500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560eda6564a0 0x560eda7f6850 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c00a870 tx=0x7f5b8c00a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a405b45855055c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:57:34.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.914+0000 7f5b84ff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:57:34.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.914+0000 7f5b84ff9700 1 -- 172.21.15.5:0/1682294500 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c015040 con 0x560eda6564a0 2024-05-09T00:57:34.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.915+0000 7f5bb2026b80 1 -- 172.21.15.5:0/1682294500 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b9401d040 con 0x560eda6564a0 2024-05-09T00:57:34.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.915+0000 7f5b84ff9700 1 -- 172.21.15.5:0/1682294500 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f5b8c00ccf0 con 0x560eda6564a0 2024-05-09T00:57:34.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.916+0000 7f5b84ff9700 1 -- 172.21.15.5:0/1682294500 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5b8c011e60 con 0x560eda6564a0 2024-05-09T00:57:34.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.917+0000 7f5bb2026b80 10 client.16650 fetch_fsmap learned FSMap version 31 2024-05-09T00:57:34.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.917+0000 7f5bb2026b80 1 -- 172.21.15.5:0/1682294500 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560eda869650 con 0x560eda6564a0 2024-05-09T00:57:34.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.947+0000 7f5b84ff9700 1 -- 172.21.15.5:0/1682294500 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b9401d040 con 0x560eda6564a0 2024-05-09T00:57:34.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.947+0000 7f5b84ff9700 1 -- 172.21.15.5:0/1682294500 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f5b8c0119e0 con 0x560eda6564a0 2024-05-09T00:57:34.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.947+0000 7f5bb2026b80 10 client.16650 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:57:34.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.947+0000 7f5bb2026b80 10 client.16650 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:57:34.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.947+0000 7f5bb2026b80 20 client.16650 populate_metadata read hostname 'smithi005' 2024-05-09T00:57:34.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.947+0000 7f5bb2026b80 1 --2- 172.21.15.5:0/1682294500 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x560eda869a80 0x560eda86bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:34.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.947+0000 7f5bb2026b80 4 client.16650 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:57:34.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.947+0000 7f5bb2026b80 1 -- 172.21.15.5:0/1682294500 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560eda703a80 con 0x560eda869a80 2024-05-09T00:57:34.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.949+0000 7f5ba0bab700 1 --2- 172.21.15.5:0/1682294500 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x560eda869a80 0x560eda86bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:34.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.953+0000 7f5ba0bab700 1 --2- 172.21.15.5:0/1682294500 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x560eda869a80 0x560eda86bea0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a97afef8ee22fb9 server_cookie=5e3989072dc1d9eb in_seq=0 out_seq=0 2024-05-09T00:57:34.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.954+0000 7f5b84ff9700 10 client.16650 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:57:34.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.962+0000 7f5b84ff9700 1 -- 172.21.15.5:0/1682294500 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560eda703a80 con 0x560eda869a80 2024-05-09T00:57:34.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.962+0000 7f5b84ff9700 10 client.16650 handle_command_reply: tid=0 2024-05-09T00:57:34.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.963+0000 7f5bb2026b80 1 client.16650 shutdown 2024-05-09T00:57:34.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.968+0000 7f5bb2026b80 1 -- 172.21.15.5:0/1682294500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560eda6564a0 msgr2=0x560eda7f6850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:34.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.968+0000 7f5bb2026b80 1 --2- 172.21.15.5:0/1682294500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560eda6564a0 0x560eda7f6850 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c00a870 tx=0x7f5b8c00a230 comp rx=0 tx=0).stop 2024-05-09T00:57:34.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.968+0000 7f5bb2026b80 1 -- 172.21.15.5:0/1682294500 shutdown_connections 2024-05-09T00:57:34.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.968+0000 7f5bb2026b80 1 --2- 172.21.15.5:0/1682294500 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x560eda869a80 0x560eda86bea0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:34.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.968+0000 7f5bb2026b80 1 --2- 172.21.15.5:0/1682294500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560eda7d4160 0x560eda7f3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:34.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.968+0000 7f5bb2026b80 1 --2- 172.21.15.5:0/1682294500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560eda6564a0 0x560eda7f6850 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:34.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.968+0000 7f5bb2026b80 1 --2- 172.21.15.5:0/1682294500 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560eda6d0d00 0x560eda7f2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:34.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.968+0000 7f5b84ff9700 0 client.16650 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:57:34.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.968+0000 7f5bb2026b80 1 -- 172.21.15.5:0/1682294500 >> 172.21.15.5:0/1682294500 conn(0x560eda659f70 msgr2=0x560eda7ee340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:57:34.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.973+0000 7f5bb2026b80 1 -- 172.21.15.5:0/1682294500 shutdown_connections 2024-05-09T00:57:34.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.973+0000 7f5bb2026b80 1 -- 172.21.15.5:0/1682294500 wait complete. 2024-05-09T00:57:34.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.973+0000 7f5bb2026b80 20 client.16650 trim_cache size 0 max 16384 2024-05-09T00:57:34.975 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-09T00:57:34.975 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-05-09T00:57:34.975 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-05-09T00:57:34.975 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-09T00:57:34.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.974+0000 7f5ba23ae700 1 -- 172.21.15.5:0/2733774943 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f5b880b29c0 msgr2=0x7f5b880b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:34.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.974+0000 7f5ba23ae700 1 --2- 172.21.15.5:0/2733774943 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f5b880b29c0 0x7f5b880b4ed0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5b94009550 tx=0x7f5b940091b0 comp rx=0 tx=0).stop 2024-05-09T00:57:34.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.974+0000 7f5ba23ae700 1 -- 172.21.15.5:0/2733774943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b9c116eb0 msgr2=0x7f5b9c1b58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:34.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.974+0000 7f5ba23ae700 1 --2- 172.21.15.5:0/2733774943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b9c116eb0 0x7f5b9c1b58c0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f5b98009fb0 tx=0x7f5b98005590 comp rx=0 tx=0).stop 2024-05-09T00:57:34.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.974+0000 7f5ba23ae700 1 -- 172.21.15.5:0/2733774943 shutdown_connections 2024-05-09T00:57:34.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.974+0000 7f5ba23ae700 1 --2- 172.21.15.5:0/2733774943 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f5b880b29c0 0x7f5b880b4ed0 secure :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5b94009550 tx=0x7f5b940091b0 comp rx=0 tx=0).stop 2024-05-09T00:57:34.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.974+0000 7f5ba23ae700 1 --2- 172.21.15.5:0/2733774943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b9c116eb0 0x7f5b9c1b58c0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:34.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.974+0000 7f5ba23ae700 1 --2- 172.21.15.5:0/2733774943 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b9c10aa50 0x7f5b9c1aa840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:34.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.974+0000 7f5ba23ae700 1 --2- 172.21.15.5:0/2733774943 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b9c10b3a0 0x7f5b9c1aad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:34.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.974+0000 7f5ba23ae700 1 -- 172.21.15.5:0/2733774943 >> 172.21.15.5:0/2733774943 conn(0x7f5b9c06f260 msgr2=0x7f5b9c10c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:57:34.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.979+0000 7f5ba23ae700 1 -- 172.21.15.5:0/2733774943 shutdown_connections 2024-05-09T00:57:34.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:34.979+0000 7f5ba23ae700 1 -- 172.21.15.5:0/2733774943 wait complete. 2024-05-09T00:57:35.294 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:57:35 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:57:35.294 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:57:35 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:57:35.294 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:57:35 smithi005 ceph-mon[30365]: pgmap v708: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 887 MiB/s wr, 2.13k op/s 2024-05-09T00:57:35.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:57:35 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:57:35.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:57:35 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:57:35.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:57:35 smithi131 ceph-mon[34547]: pgmap v708: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 887 MiB/s wr, 2.13k op/s 2024-05-09T00:57:35.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:57:35 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:57:35.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:57:35 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:57:35.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:57:35 smithi157 ceph-mon[34294]: pgmap v708: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 887 MiB/s wr, 2.13k op/s 2024-05-09T00:57:36.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.771+0000 7f489ece0640 1 -- 172.21.15.5:0/1533483296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f489807aae0 msgr2=0x7f489807aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:36.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.771+0000 7f489ece0640 1 --2- 172.21.15.5:0/1533483296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f489807aae0 0x7f489807aef0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f4890009900 tx=0x7f489002a410 comp rx=0 tx=0).stop 2024-05-09T00:57:36.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.771+0000 7f489ece0640 1 -- 172.21.15.5:0/1533483296 shutdown_connections 2024-05-09T00:57:36.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.771+0000 7f489ece0640 1 --2- 172.21.15.5:0/1533483296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f489807aae0 0x7f489807aef0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:36.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.771+0000 7f489ece0640 1 --2- 172.21.15.5:0/1533483296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4898079de0 0x7f489807a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:36.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.771+0000 7f489ece0640 1 --2- 172.21.15.5:0/1533483296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4898078b50 0x7f48980798a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:36.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.771+0000 7f489ece0640 1 -- 172.21.15.5:0/1533483296 >> 172.21.15.5:0/1533483296 conn(0x7f4898101070 msgr2=0x7f4898103520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:57:36.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.771+0000 7f489ece0640 1 -- 172.21.15.5:0/1533483296 shutdown_connections 2024-05-09T00:57:36.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.771+0000 7f489ece0640 1 -- 172.21.15.5:0/1533483296 wait complete. 2024-05-09T00:57:36.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.773+0000 7f489ece0640 1 Processor -- start 2024-05-09T00:57:36.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.773+0000 7f489ece0640 1 -- start start 2024-05-09T00:57:36.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.773+0000 7f489ece0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4898078b50 0x7f48981a0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:36.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.773+0000 7f489ece0640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4898079de0 0x7f48981a0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:36.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.773+0000 7f489ece0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f489807aae0 0x7f48981a7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:36.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.773+0000 7f489ece0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4898115d90 con 0x7f489807aae0 2024-05-09T00:57:36.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.773+0000 7f489ece0640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4898115c10 con 0x7f4898079de0 2024-05-09T00:57:36.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.773+0000 7f489ece0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4898115f10 con 0x7f4898078b50 2024-05-09T00:57:36.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.774+0000 7f489d27d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f489807aae0 0x7f48981a7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:36.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.774+0000 7f489d27d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f489807aae0 0x7f48981a7dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38436/0 (socket says 172.21.15.5:38436) 2024-05-09T00:57:36.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.774+0000 7f489d27d640 1 -- 172.21.15.5:0/2527061901 learned_addr learned my addr 172.21.15.5:0/2527061901 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:57:36.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.774+0000 7f489d27d640 1 -- 172.21.15.5:0/2527061901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4898078b50 msgr2=0x7f48981a0900 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T00:57:36.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.774+0000 7f489d27d640 1 --2- 172.21.15.5:0/2527061901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4898078b50 0x7f48981a0900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:36.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.774+0000 7f489d27d640 1 -- 172.21.15.5:0/2527061901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4898079de0 msgr2=0x7f48981a0e40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T00:57:36.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.775+0000 7f489d27d640 1 --2- 172.21.15.5:0/2527061901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4898079de0 0x7f48981a0e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:36.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.775+0000 7f489d27d640 1 -- 172.21.15.5:0/2527061901 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48981a8440 con 0x7f489807aae0 2024-05-09T00:57:36.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.776+0000 7f489d27d640 1 --2- 172.21.15.5:0/2527061901 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f489807aae0 0x7f48981a7dd0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f4890009230 tx=0x7f4890031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:57:36.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.776+0000 7f4895ffb640 1 -- 172.21.15.5:0/2527061901 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f489003f030 con 0x7f489807aae0 2024-05-09T00:57:36.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.776+0000 7f489ece0640 1 -- 172.21.15.5:0/2527061901 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48981a86d0 con 0x7f489807aae0 2024-05-09T00:57:36.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.776+0000 7f4895ffb640 1 -- 172.21.15.5:0/2527061901 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4890003aa0 con 0x7f489807aae0 2024-05-09T00:57:36.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.777+0000 7f489ece0640 1 -- 172.21.15.5:0/2527061901 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48981a8c10 con 0x7f489807aae0 2024-05-09T00:57:36.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.777+0000 7f4895ffb640 1 -- 172.21.15.5:0/2527061901 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f489003f030 con 0x7f489807aae0 2024-05-09T00:57:36.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.782+0000 7f48a5b9a740 1 Processor -- start 2024-05-09T00:57:36.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.784+0000 7f4895ffb640 1 -- 172.21.15.5:0/2527061901 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f4890042030 con 0x7f489807aae0 2024-05-09T00:57:36.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.784+0000 7f48a5b9a740 1 -- start start 2024-05-09T00:57:36.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.784+0000 7f48a5b9a740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adee1421b0 0x55adee4e6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:36.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.784+0000 7f48a5b9a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adee1192f0 0x55adee4e6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:36.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.784+0000 7f48a5b9a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55adee1214d0 0x55adee4ed980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:36.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.784+0000 7f48a5b9a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55adee44a7c0 con 0x55adee1192f0 2024-05-09T00:57:36.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.784+0000 7f48a5b9a740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55adee3e60e0 con 0x55adee1421b0 2024-05-09T00:57:36.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.784+0000 7f48a5b9a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55adee446780 con 0x55adee1214d0 2024-05-09T00:57:36.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.785+0000 7f489ca7c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adee1192f0 0x55adee4e6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:36.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.785+0000 7f489ca7c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adee1192f0 0x55adee4e6a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38440/0 (socket says 172.21.15.5:38440) 2024-05-09T00:57:36.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.785+0000 7f489ca7c640 1 -- 172.21.15.5:0/626988407 learned_addr learned my addr 172.21.15.5:0/626988407 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:57:36.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.785+0000 7f4897fff640 1 --2- 172.21.15.5:0/626988407 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adee1421b0 0x55adee4e6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:36.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.785+0000 7f4895ffb640 1 --2- 172.21.15.5:0/2527061901 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f48800b29d0 0x7f48800b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:36.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.785+0000 7f4897fff640 1 --2- 172.21.15.5:0/626988407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55adee1214d0 0x55adee4ed980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:36.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.785+0000 7f4895ffb640 1 -- 172.21.15.5:0/2527061901 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f48901024b0 con 0x7f489807aae0 2024-05-09T00:57:36.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.785+0000 7f489d27d640 1 --2- 172.21.15.5:0/2527061901 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f48800b29d0 0x7f48800b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:36.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.785+0000 7f489ca7c640 1 -- 172.21.15.5:0/626988407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55adee1214d0 msgr2=0x55adee4ed980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:57:36.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.785+0000 7f489ca7c640 1 --2- 172.21.15.5:0/626988407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55adee1214d0 0x55adee4ed980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:36.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.785+0000 7f489ca7c640 1 -- 172.21.15.5:0/626988407 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adee1421b0 msgr2=0x55adee4e6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:57:36.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.785+0000 7f489ca7c640 1 --2- 172.21.15.5:0/626988407 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adee1421b0 0x55adee4e6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:36.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.785+0000 7f489ca7c640 1 -- 172.21.15.5:0/626988407 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adee40f7c0 con 0x55adee1192f0 2024-05-09T00:57:36.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.786+0000 7f489ca7c640 1 --2- 172.21.15.5:0/626988407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adee1192f0 0x55adee4e6a80 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f488c00ba50 tx=0x7f488c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=632976bd61c0b797 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:57:36.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.786+0000 7f487effd640 1 -- 172.21.15.5:0/626988407 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488c016070 con 0x55adee1192f0 2024-05-09T00:57:36.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.786+0000 7f489d27d640 1 --2- 172.21.15.5:0/2527061901 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f48800b29d0 0x7f48800b4ee0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f48900c97a0 tx=0x7f48900ec040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:57:36.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.786+0000 7f487effd640 1 -- 172.21.15.5:0/626988407 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f488c0054d0 con 0x55adee1192f0 2024-05-09T00:57:36.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.786+0000 7f487effd640 1 -- 172.21.15.5:0/626988407 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488c004df0 con 0x55adee1192f0 2024-05-09T00:57:36.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.787+0000 7f48a5b9a740 1 -- 172.21.15.5:0/626988407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adee1192f0 msgr2=0x55adee4e6a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:36.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.787+0000 7f48a5b9a740 1 --2- 172.21.15.5:0/626988407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adee1192f0 0x55adee4e6a80 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f488c00ba50 tx=0x7f488c00bf10 comp rx=0 tx=0).stop 2024-05-09T00:57:36.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.787+0000 7f48a5b9a740 1 -- 172.21.15.5:0/626988407 shutdown_connections 2024-05-09T00:57:36.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.787+0000 7f48a5b9a740 1 --2- 172.21.15.5:0/626988407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55adee1214d0 0x55adee4ed980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:36.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.787+0000 7f48a5b9a740 1 --2- 172.21.15.5:0/626988407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adee1192f0 0x55adee4e6a80 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:36.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.787+0000 7f48a5b9a740 1 --2- 172.21.15.5:0/626988407 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adee1421b0 0x55adee4e6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:36.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.787+0000 7f48a5b9a740 1 -- 172.21.15.5:0/626988407 >> 172.21.15.5:0/626988407 conn(0x55adee136300 msgr2=0x55adee4e1e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:57:36.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.787+0000 7f48a5b9a740 1 -- 172.21.15.5:0/626988407 shutdown_connections 2024-05-09T00:57:36.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.787+0000 7f48a5b9a740 1 -- 172.21.15.5:0/626988407 wait complete. 2024-05-09T00:57:36.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.788+0000 7f48a5b9a740 1 -- start start 2024-05-09T00:57:36.792 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.788+0000 7f48a5b9a740 1 Processor -- start 2024-05-09T00:57:36.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.789+0000 7f48a5b9a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adee1192f0 0x55adee4f0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:36.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.789+0000 7f48a5b9a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55adee1214d0 0x55adee4f0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:36.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.789+0000 7f48a5b9a740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adee1421b0 0x55adee4f76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:36.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.789+0000 7f48a5b9a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55adee4ee3d0 con 0x55adee1192f0 2024-05-09T00:57:36.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.789+0000 7f48a5b9a740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55adee3d19c0 con 0x55adee1421b0 2024-05-09T00:57:36.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.789+0000 7f48a5b9a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55adee4ee550 con 0x55adee1214d0 2024-05-09T00:57:36.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.789+0000 7f48a5b9a740 10 client.0 _ll_register_callbacks cb 0x55adee3fe020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:57:36.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.789+0000 7f4897fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adee1192f0 0x55adee4f0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:36.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.789+0000 7f4897fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adee1192f0 0x55adee4f0230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38452/0 (socket says 172.21.15.5:38452) 2024-05-09T00:57:36.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.789+0000 7f4897fff640 1 -- 172.21.15.5:0/1086109225 learned_addr learned my addr 172.21.15.5:0/1086109225 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:57:36.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.790+0000 7f4897fff640 1 -- 172.21.15.5:0/1086109225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55adee1214d0 msgr2=0x55adee4f0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:57:36.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.790+0000 7f4897fff640 1 --2- 172.21.15.5:0/1086109225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55adee1214d0 0x55adee4f0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:36.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.790+0000 7f4897fff640 1 -- 172.21.15.5:0/1086109225 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adee1421b0 msgr2=0x55adee4f76d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:57:36.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.790+0000 7f4897fff640 1 --2- 172.21.15.5:0/1086109225 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adee1421b0 0x55adee4f76d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:36.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.790+0000 7f4897fff640 1 -- 172.21.15.5:0/1086109225 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55adee3f81f0 con 0x55adee1192f0 2024-05-09T00:57:36.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.790+0000 7f4897fff640 1 --2- 172.21.15.5:0/1086109225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adee1192f0 0x55adee4f0230 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f4888013930 tx=0x7f4888009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d889624b65067aed server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:57:36.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.790+0000 7f4877fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:57:36.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.790+0000 7f4877fff640 1 -- 172.21.15.5:0/1086109225 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488800eca0 con 0x55adee1192f0 2024-05-09T00:57:36.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.790+0000 7f4877fff640 1 -- 172.21.15.5:0/1086109225 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f4888003970 con 0x55adee1192f0 2024-05-09T00:57:36.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.790+0000 7f48a5b9a740 1 -- 172.21.15.5:0/1086109225 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f488c0054d0 con 0x55adee1192f0 2024-05-09T00:57:36.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.791+0000 7f4877fff640 1 -- 172.21.15.5:0/1086109225 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f4888022050 con 0x55adee1192f0 2024-05-09T00:57:36.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.791+0000 7f48a5b9a740 10 client.16662 fetch_fsmap learned FSMap version 31 2024-05-09T00:57:36.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.791+0000 7f48a5b9a740 1 -- 172.21.15.5:0/1086109225 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55adee1ad790 con 0x55adee1192f0 2024-05-09T00:57:36.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.791+0000 7f4877fff640 1 -- 172.21.15.5:0/1086109225 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f488c0054d0 con 0x55adee1192f0 2024-05-09T00:57:36.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.791+0000 7f4877fff640 1 -- 172.21.15.5:0/1086109225 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f488800c070 con 0x55adee1192f0 2024-05-09T00:57:36.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.792+0000 7f48a5b9a740 10 client.16662 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:57:36.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.792+0000 7f48a5b9a740 10 client.16662 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:57:36.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.792+0000 7f48a5b9a740 20 client.16662 populate_metadata read hostname 'smithi005' 2024-05-09T00:57:36.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.792+0000 7f48a5b9a740 1 --2- 172.21.15.5:0/1086109225 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55adee100d80 0x55adee4fdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:36.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.792+0000 7f48a5b9a740 4 client.16662 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:57:36.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.792+0000 7f48a5b9a740 1 -- 172.21.15.5:0/1086109225 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55adee4583d0 con 0x55adee100d80 2024-05-09T00:57:36.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.794+0000 7f489ca7c640 1 --2- 172.21.15.5:0/1086109225 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55adee100d80 0x55adee4fdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:36.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.802+0000 7f4877fff640 10 client.16662 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:57:36.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.802+0000 7f489ca7c640 1 --2- 172.21.15.5:0/1086109225 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55adee100d80 0x55adee4fdc20 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27c03692597d2846 server_cookie=d3d4674d6c45eead in_seq=0 out_seq=0 2024-05-09T00:57:36.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.813+0000 7f4877fff640 1 -- 172.21.15.5:0/1086109225 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55adee4583d0 con 0x55adee100d80 2024-05-09T00:57:36.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.813+0000 7f4877fff640 10 client.16662 handle_command_reply: tid=0 2024-05-09T00:57:36.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.813+0000 7f48a5b9a740 1 client.16662 shutdown 2024-05-09T00:57:36.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.813+0000 7f48a5b9a740 1 -- 172.21.15.5:0/1086109225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adee1192f0 msgr2=0x55adee4f0230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:36.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.813+0000 7f48a5b9a740 1 --2- 172.21.15.5:0/1086109225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adee1192f0 0x55adee4f0230 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f4888013930 tx=0x7f4888009b80 comp rx=0 tx=0).stop 2024-05-09T00:57:36.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.813+0000 7f48a5b9a740 1 -- 172.21.15.5:0/1086109225 shutdown_connections 2024-05-09T00:57:36.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.813+0000 7f48a5b9a740 1 --2- 172.21.15.5:0/1086109225 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55adee100d80 0x55adee4fdc20 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:36.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.813+0000 7f48a5b9a740 1 --2- 172.21.15.5:0/1086109225 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adee1421b0 0x55adee4f76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:36.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.813+0000 7f48a5b9a740 1 --2- 172.21.15.5:0/1086109225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55adee1214d0 0x55adee4f0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:36.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.813+0000 7f4877fff640 0 client.16662 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:57:36.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.813+0000 7f48a5b9a740 1 --2- 172.21.15.5:0/1086109225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adee1192f0 0x55adee4f0230 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:36.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.813+0000 7f48a5b9a740 1 -- 172.21.15.5:0/1086109225 >> 172.21.15.5:0/1086109225 conn(0x55adee136300 msgr2=0x55adee4e0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:57:36.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.813+0000 7f48a5b9a740 1 -- 172.21.15.5:0/1086109225 shutdown_connections 2024-05-09T00:57:36.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.814+0000 7f48a5b9a740 1 -- 172.21.15.5:0/1086109225 wait complete. 2024-05-09T00:57:36.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.814+0000 7f48a5b9a740 20 client.16662 trim_cache size 0 max 16384 2024-05-09T00:57:36.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.840+0000 7f48a5b9a740 1 Processor -- start 2024-05-09T00:57:36.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.840+0000 7f48a5b9a740 1 -- start start 2024-05-09T00:57:36.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.840+0000 7f48a5b9a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55adee100d80 0x55adee4e1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:36.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.840+0000 7f48a5b9a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adee1192f0 0x55adee4efe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:36.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.840+0000 7f48a5b9a740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adee1214d0 0x55adee4f0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:36.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.840+0000 7f48a5b9a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f48981b1ad0 con 0x55adee1192f0 2024-05-09T00:57:36.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.840+0000 7f48a5b9a740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f48981b3610 con 0x55adee1214d0 2024-05-09T00:57:36.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.840+0000 7f48a5b9a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f48981b21a0 con 0x55adee100d80 2024-05-09T00:57:36.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.840+0000 7f489ca7c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adee1192f0 0x55adee4efe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:36.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.841+0000 7f489ca7c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adee1192f0 0x55adee4efe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38466/0 (socket says 172.21.15.5:38466) 2024-05-09T00:57:36.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.841+0000 7f489ca7c640 1 -- 172.21.15.5:0/2154891926 learned_addr learned my addr 172.21.15.5:0/2154891926 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:57:36.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.841+0000 7f489ca7c640 1 -- 172.21.15.5:0/2154891926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55adee100d80 msgr2=0x55adee4e1f20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:57:36.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.841+0000 7f489ca7c640 1 --2- 172.21.15.5:0/2154891926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55adee100d80 0x55adee4e1f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:36.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.841+0000 7f489ca7c640 1 -- 172.21.15.5:0/2154891926 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adee1214d0 msgr2=0x55adee4f0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:57:36.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.841+0000 7f489ca7c640 1 --2- 172.21.15.5:0/2154891926 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adee1214d0 0x55adee4f0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:36.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.841+0000 7f489ca7c640 1 -- 172.21.15.5:0/2154891926 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adee2245f0 con 0x55adee1192f0 2024-05-09T00:57:36.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.841+0000 7f489ca7c640 1 --2- 172.21.15.5:0/2154891926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adee1192f0 0x55adee4efe00 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f488c00b9a0 tx=0x7f488c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0c9fdb5573aae08 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:57:36.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.843+0000 7f487e7fc640 1 -- 172.21.15.5:0/2154891926 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488c016040 con 0x55adee1192f0 2024-05-09T00:57:36.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.843+0000 7f487e7fc640 1 -- 172.21.15.5:0/2154891926 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f488c0044f0 con 0x55adee1192f0 2024-05-09T00:57:36.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.843+0000 7f487e7fc640 1 -- 172.21.15.5:0/2154891926 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488c003850 con 0x55adee1192f0 2024-05-09T00:57:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.844+0000 7f48a5b9a740 1 -- 172.21.15.5:0/2154891926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adee1192f0 msgr2=0x55adee4efe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.844+0000 7f48a5b9a740 1 --2- 172.21.15.5:0/2154891926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adee1192f0 0x55adee4efe00 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f488c00b9a0 tx=0x7f488c00dc00 comp rx=0 tx=0).stop 2024-05-09T00:57:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.844+0000 7f48a5b9a740 1 -- 172.21.15.5:0/2154891926 shutdown_connections 2024-05-09T00:57:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.844+0000 7f48a5b9a740 1 --2- 172.21.15.5:0/2154891926 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adee1214d0 0x55adee4f0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.844+0000 7f48a5b9a740 1 --2- 172.21.15.5:0/2154891926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adee1192f0 0x55adee4efe00 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.844+0000 7f48a5b9a740 1 --2- 172.21.15.5:0/2154891926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55adee100d80 0x55adee4e1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.844+0000 7f48a5b9a740 1 -- 172.21.15.5:0/2154891926 >> 172.21.15.5:0/2154891926 conn(0x55adee136300 msgr2=0x55adee4e1810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:57:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.845+0000 7f48a5b9a740 1 -- 172.21.15.5:0/2154891926 shutdown_connections 2024-05-09T00:57:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.845+0000 7f48a5b9a740 1 -- 172.21.15.5:0/2154891926 wait complete. 2024-05-09T00:57:36.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.846+0000 7f48a5b9a740 1 -- start start 2024-05-09T00:57:36.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.846+0000 7f48a5b9a740 1 Processor -- start 2024-05-09T00:57:36.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.847+0000 7f48a5b9a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55adee100d80 0x55adee4f43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:36.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.847+0000 7f48a5b9a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adee1192f0 0x55adee4f4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:36.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.847+0000 7f48a5b9a740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adee1214d0 0x55adee4f5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:36.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.847+0000 7f48a5b9a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f48981b2f40 con 0x55adee1192f0 2024-05-09T00:57:36.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.848+0000 7f48a5b9a740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f48981b2870 con 0x55adee1214d0 2024-05-09T00:57:36.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.848+0000 7f48a5b9a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f48981afab0 con 0x55adee100d80 2024-05-09T00:57:36.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.848+0000 7f489ca7c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adee1192f0 0x55adee4f4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:36.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.848+0000 7f489ca7c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adee1192f0 0x55adee4f4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38470/0 (socket says 172.21.15.5:38470) 2024-05-09T00:57:36.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.848+0000 7f489ca7c640 1 -- 172.21.15.5:0/1565616852 learned_addr learned my addr 172.21.15.5:0/1565616852 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:57:36.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.848+0000 7f48a5b9a740 10 client.0 _ll_register_callbacks cb 0x7f4888009d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:57:36.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.849+0000 7f4897fff640 1 --2- 172.21.15.5:0/1565616852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55adee100d80 0x55adee4f43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:36.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.849+0000 7f489ca7c640 1 -- 172.21.15.5:0/1565616852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55adee100d80 msgr2=0x55adee4f43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:57:36.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.849+0000 7f489ca7c640 1 --2- 172.21.15.5:0/1565616852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55adee100d80 0x55adee4f43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:36.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.849+0000 7f489ca7c640 1 -- 172.21.15.5:0/1565616852 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adee1214d0 msgr2=0x55adee4f5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:57:36.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.849+0000 7f489ca7c640 1 --2- 172.21.15.5:0/1565616852 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adee1214d0 0x55adee4f5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:36.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.849+0000 7f489ca7c640 1 -- 172.21.15.5:0/1565616852 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55adee40eed0 con 0x55adee1192f0 2024-05-09T00:57:36.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.849+0000 7f4897fff640 1 --2- 172.21.15.5:0/1565616852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55adee100d80 0x55adee4f43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:57:36.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.849+0000 7f489ca7c640 1 --2- 172.21.15.5:0/1565616852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adee1192f0 0x55adee4f4930 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f488c026040 tx=0x7f488c023ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a97643df00ed767a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:57:36.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.849+0000 7f4877fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:57:36.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.850+0000 7f4877fff640 1 -- 172.21.15.5:0/1565616852 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488c030070 con 0x55adee1192f0 2024-05-09T00:57:36.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.850+0000 7f4877fff640 1 -- 172.21.15.5:0/1565616852 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f488c023380 con 0x55adee1192f0 2024-05-09T00:57:36.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.850+0000 7f48a5b9a740 1 -- 172.21.15.5:0/1565616852 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f488c0044f0 con 0x55adee1192f0 2024-05-09T00:57:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.850+0000 7f4877fff640 1 -- 172.21.15.5:0/1565616852 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f488c03c050 con 0x55adee1192f0 2024-05-09T00:57:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.851+0000 7f48a5b9a740 10 client.16668 fetch_fsmap learned FSMap version 31 2024-05-09T00:57:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.851+0000 7f48a5b9a740 1 -- 172.21.15.5:0/1565616852 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55adee4f6ee0 con 0x55adee1192f0 2024-05-09T00:57:36.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.851+0000 7f4877fff640 1 -- 172.21.15.5:0/1565616852 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f488c004300 con 0x55adee1192f0 2024-05-09T00:57:36.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.851+0000 7f4877fff640 1 -- 172.21.15.5:0/1565616852 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f488c0052c0 con 0x55adee1192f0 2024-05-09T00:57:36.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.852+0000 7f48a5b9a740 10 client.16668 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:57:36.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.852+0000 7f48a5b9a740 10 client.16668 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:57:36.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.852+0000 7f48a5b9a740 20 client.16668 populate_metadata read hostname 'smithi005' 2024-05-09T00:57:36.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.852+0000 7f48a5b9a740 1 --2- 172.21.15.5:0/1565616852 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55adee1421b0 0x55adee5684a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:36.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.852+0000 7f48a5b9a740 4 client.16668 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:57:36.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.852+0000 7f48a5b9a740 1 -- 172.21.15.5:0/1565616852 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f488c0044f0 con 0x55adee1421b0 2024-05-09T00:57:36.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.864+0000 7f4897fff640 1 --2- 172.21.15.5:0/1565616852 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55adee1421b0 0x55adee5684a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:36.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.882+0000 7f4897fff640 1 --2- 172.21.15.5:0/1565616852 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55adee1421b0 0x55adee5684a0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75abaec8de13220c server_cookie=f07fc81e685b3d25 in_seq=0 out_seq=0 2024-05-09T00:57:36.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:36.884+0000 7f4877fff640 10 client.16668 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:57:37.070 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T00:57:37.070 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T00:57:37.795 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:57:37 smithi005 ceph-mon[30365]: pgmap v709: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 905 MiB/s wr, 2.13k op/s 2024-05-09T00:57:37.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:57:37 smithi131 ceph-mon[34547]: pgmap v709: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 905 MiB/s wr, 2.13k op/s 2024-05-09T00:57:37.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:57:37 smithi157 ceph-mon[34294]: pgmap v709: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 905 MiB/s wr, 2.13k op/s 2024-05-09T00:57:38.163 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:57:38.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:57:38 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:57:38.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:57:38 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:57:38.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:57:38 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:57:38.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:57:38 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:57:38.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:57:38 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:57:38.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:57:38 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:57:38.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:57:38 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:57:38.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:57:38 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:57:38.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:57:38 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:57:40.098 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:57:39 smithi005 ceph-mon[30365]: pgmap v710: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 858 MiB/s wr, 1.99k op/s 2024-05-09T00:57:40.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:57:39 smithi131 ceph-mon[34547]: pgmap v710: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 858 MiB/s wr, 1.99k op/s 2024-05-09T00:57:40.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:57:39 smithi157 ceph-mon[34294]: pgmap v710: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 858 MiB/s wr, 1.99k op/s 2024-05-09T00:57:42.119 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:57:41 smithi005 ceph-mon[30365]: pgmap v711: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 950 MiB/s wr, 2.06k op/s 2024-05-09T00:57:42.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:57:41 smithi157 ceph-mon[34294]: pgmap v711: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 950 MiB/s wr, 2.06k op/s 2024-05-09T00:57:42.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:57:41 smithi131 ceph-mon[34547]: pgmap v711: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 950 MiB/s wr, 2.06k op/s 2024-05-09T00:57:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:43.252+0000 7f4877fff640 1 -- 172.21.15.5:0/1565616852 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7f488c0044f0 con 0x55adee1421b0 2024-05-09T00:57:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:43.252+0000 7f4877fff640 10 client.16668 handle_command_reply: tid=0 2024-05-09T00:57:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:43.252+0000 7f48a5b9a740 1 client.16668 shutdown 2024-05-09T00:57:43.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:43.255+0000 7f48a5b9a740 1 -- 172.21.15.5:0/1565616852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adee1192f0 msgr2=0x55adee4f4930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:43.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:43.255+0000 7f48a5b9a740 1 --2- 172.21.15.5:0/1565616852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adee1192f0 0x55adee4f4930 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f488c026040 tx=0x7f488c023ac0 comp rx=0 tx=0).stop 2024-05-09T00:57:43.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:43.256+0000 7f48a5b9a740 1 -- 172.21.15.5:0/1565616852 shutdown_connections 2024-05-09T00:57:43.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:43.256+0000 7f48a5b9a740 1 --2- 172.21.15.5:0/1565616852 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55adee1421b0 0x55adee5684a0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:43.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:43.256+0000 7f48a5b9a740 1 --2- 172.21.15.5:0/1565616852 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55adee1214d0 0x55adee4f5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:43.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:43.256+0000 7f48a5b9a740 1 --2- 172.21.15.5:0/1565616852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adee1192f0 0x55adee4f4930 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:43.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:43.256+0000 7f48a5b9a740 1 --2- 172.21.15.5:0/1565616852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55adee100d80 0x55adee4f43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:43.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:43.256+0000 7f48a5b9a740 1 -- 172.21.15.5:0/1565616852 >> 172.21.15.5:0/1565616852 conn(0x55adee136300 msgr2=0x55adee4efe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:57:43.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:43.256+0000 7f4877fff640 0 client.16668 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:57:43.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:43.257+0000 7f48a5b9a740 1 -- 172.21.15.5:0/1565616852 shutdown_connections 2024-05-09T00:57:43.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:43.258+0000 7f48a5b9a740 1 -- 172.21.15.5:0/1565616852 wait complete. 2024-05-09T00:57:43.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:43.258+0000 7f48a5b9a740 20 client.16668 trim_cache size 0 max 16384 2024-05-09T00:57:43.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-09T00:57:43.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-09T00:57:43.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-09T00:57:43.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:43.260+0000 7f489ece0640 1 -- 172.21.15.5:0/2527061901 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f48800b29d0 msgr2=0x7f48800b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:43.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:43.260+0000 7f489ece0640 1 --2- 172.21.15.5:0/2527061901 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f48800b29d0 0x7f48800b4ee0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f48900c97a0 tx=0x7f48900ec040 comp rx=0 tx=0).stop 2024-05-09T00:57:43.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:43.260+0000 7f489ece0640 1 -- 172.21.15.5:0/2527061901 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f489807aae0 msgr2=0x7f48981a7dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:43.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:43.260+0000 7f489ece0640 1 --2- 172.21.15.5:0/2527061901 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f489807aae0 0x7f48981a7dd0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f4890009230 tx=0x7f4890031a50 comp rx=0 tx=0).stop 2024-05-09T00:57:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:43.261+0000 7f489ece0640 1 -- 172.21.15.5:0/2527061901 shutdown_connections 2024-05-09T00:57:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:43.261+0000 7f489ece0640 1 --2- 172.21.15.5:0/2527061901 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f48800b29d0 0x7f48800b4ee0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:43.261+0000 7f489ece0640 1 --2- 172.21.15.5:0/2527061901 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f489807aae0 0x7f48981a7dd0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:43.261+0000 7f489ece0640 1 --2- 172.21.15.5:0/2527061901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4898079de0 0x7f48981a0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:43.261+0000 7f489ece0640 1 --2- 172.21.15.5:0/2527061901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4898078b50 0x7f48981a0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:43.261+0000 7f489ece0640 1 -- 172.21.15.5:0/2527061901 >> 172.21.15.5:0/2527061901 conn(0x7f4898101070 msgr2=0x7f489810fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:57:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:43.262+0000 7f489ece0640 1 -- 172.21.15.5:0/2527061901 shutdown_connections 2024-05-09T00:57:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:43.262+0000 7f489ece0640 1 -- 172.21.15.5:0/2527061901 wait complete. 2024-05-09T00:57:43.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:43.597+0000 7efd8cb4c700 1 -- 172.21.15.5:0/1591902271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd8810b2e0 msgr2=0x7efd88117810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:43.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:43.597+0000 7efd8cb4c700 1 --2- 172.21.15.5:0/1591902271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd8810b2e0 0x7efd88117810 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7efd8000b0a0 tx=0x7efd8000b3b0 comp rx=0 tx=0).stop 2024-05-09T00:57:43.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:43.598+0000 7efd8cb4c700 1 -- 172.21.15.5:0/1591902271 shutdown_connections 2024-05-09T00:57:43.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:43.598+0000 7efd8cb4c700 1 --2- 172.21.15.5:0/1591902271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd8810b2e0 0x7efd88117810 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:43.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:43.598+0000 7efd8cb4c700 1 --2- 172.21.15.5:0/1591902271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd88110970 0x7efd88110d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:43.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:43.598+0000 7efd8cb4c700 1 --2- 172.21.15.5:0/1591902271 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd8810a990 0x7efd8810ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:43.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:43.598+0000 7efd8cb4c700 1 -- 172.21.15.5:0/1591902271 >> 172.21.15.5:0/1591902271 conn(0x7efd88077c80 msgr2=0x7efd880780e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:57:43.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:43.598+0000 7efd8cb4c700 1 -- 172.21.15.5:0/1591902271 shutdown_connections 2024-05-09T00:57:43.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:43.599+0000 7efd8cb4c700 1 -- 172.21.15.5:0/1591902271 wait complete. 2024-05-09T00:57:43.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:43.601+0000 7efd8cb4c700 1 Processor -- start 2024-05-09T00:57:43.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:43.601+0000 7efd8cb4c700 1 -- start start 2024-05-09T00:57:43.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:43.601+0000 7efd8cb4c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd8810a990 0x7efd88075240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:43.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:43.601+0000 7efd8cb4c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd8810b2e0 0x7efd8806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:43.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:43.601+0000 7efd8cb4c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd88110970 0x7efd8806e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:43.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:43.601+0000 7efd8cb4c700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efd8811ac30 con 0x7efd8810b2e0 2024-05-09T00:57:43.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:43.601+0000 7efd8cb4c700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7efd8811aab0 con 0x7efd88110970 2024-05-09T00:57:43.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:43.601+0000 7efd8cb4c700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7efd8811adb0 con 0x7efd8810a990 2024-05-09T00:57:43.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:43.610+0000 7efd7dd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd8810b2e0 0x7efd8806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:43.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:43.610+0000 7efd7dd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd8810b2e0 0x7efd8806e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38026/0 (socket says 172.21.15.5:38026) 2024-05-09T00:57:43.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:43.610+0000 7efd7dd9b700 1 -- 172.21.15.5:0/2394753792 learned_addr learned my addr 172.21.15.5:0/2394753792 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:57:43.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:43.612+0000 7efd8659c700 1 --2- 172.21.15.5:0/2394753792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd8810a990 0x7efd88075240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:43.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:43.610+0000 7efd86d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd88110970 0x7efd8806e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:43.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:43.615+0000 7efd7dd9b700 1 -- 172.21.15.5:0/2394753792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd8810a990 msgr2=0x7efd88075240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:43.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:43.615+0000 7efd7dd9b700 1 --2- 172.21.15.5:0/2394753792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd8810a990 0x7efd88075240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:43.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:43.615+0000 7efd7dd9b700 1 -- 172.21.15.5:0/2394753792 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd88110970 msgr2=0x7efd8806e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:43.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:43.615+0000 7efd7dd9b700 1 --2- 172.21.15.5:0/2394753792 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd88110970 0x7efd8806e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:43.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:43.615+0000 7efd7dd9b700 1 -- 172.21.15.5:0/2394753792 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd8806f080 con 0x7efd8810b2e0 2024-05-09T00:57:43.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:43.630+0000 7efd7dd9b700 1 --2- 172.21.15.5:0/2394753792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd8810b2e0 0x7efd8806e370 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7efd7000b550 tx=0x7efd7000b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:57:43.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:43.631+0000 7efd7ffff700 1 -- 172.21.15.5:0/2394753792 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd70015020 con 0x7efd8810b2e0 2024-05-09T00:57:43.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:43.631+0000 7efd7ffff700 1 -- 172.21.15.5:0/2394753792 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd70011ce0 con 0x7efd8810b2e0 2024-05-09T00:57:43.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:43.631+0000 7efd7ffff700 1 -- 172.21.15.5:0/2394753792 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd70010c60 con 0x7efd8810b2e0 2024-05-09T00:57:43.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:43.632+0000 7efd8cb4c700 1 -- 172.21.15.5:0/2394753792 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd88072d70 con 0x7efd8810b2e0 2024-05-09T00:57:43.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:43.632+0000 7efd8cb4c700 1 -- 172.21.15.5:0/2394753792 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd88073260 con 0x7efd8810b2e0 2024-05-09T00:57:43.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:43.633+0000 7efd7d59a700 1 -- 172.21.15.5:0/2394753792 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd6c005330 con 0x7efd8810b2e0 2024-05-09T00:57:43.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:43.640+0000 7efd7ffff700 1 -- 172.21.15.5:0/2394753792 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7efd70018020 con 0x7efd8810b2e0 2024-05-09T00:57:43.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:43.641+0000 7efd7ffff700 1 --2- 172.21.15.5:0/2394753792 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7efd740b29b0 0x7efd740b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:43.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:43.646+0000 7efd8659c700 1 --2- 172.21.15.5:0/2394753792 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7efd740b29b0 0x7efd740b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:43.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:43.646+0000 7efd7ffff700 1 -- 172.21.15.5:0/2394753792 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7efd700e1e20 con 0x7efd8810b2e0 2024-05-09T00:57:43.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:43.650+0000 7efd8659c700 1 --2- 172.21.15.5:0/2394753792 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7efd740b29b0 0x7efd740b4ec0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7efd8806fbc0 tx=0x7efd78009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:57:43.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:43.655+0000 7efd7ffff700 1 -- 172.21.15.5:0/2394753792 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efd7001f020 con 0x7efd8810b2e0 2024-05-09T00:57:44.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:57:43 smithi157 ceph-mon[34294]: pgmap v712: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 827 MiB/s wr, 1.67k op/s 2024-05-09T00:57:44.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:57:43 smithi005 ceph-mon[30365]: pgmap v712: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 827 MiB/s wr, 1.67k op/s 2024-05-09T00:57:44.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:44.305+0000 7efd7d59a700 1 -- 172.21.15.5:0/2394753792 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efd6c005620 con 0x7efd8810b2e0 2024-05-09T00:57:44.308 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:57:44.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:44.307+0000 7efd7ffff700 1 -- 172.21.15.5:0/2394753792 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11926 (secure 0 0 0) 0x7efd700a9ab0 con 0x7efd8810b2e0 2024-05-09T00:57:44.309 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2685771887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2685771887},{"type":"v1","addr":"172.21.15.5:6835","nonce":2685771887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/254017535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":254017535},{"type":"v1","addr":"172.21.15.5:6837","nonce":254017535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T00:43:45.856066+0000","modified":"2024-05-09T00:52:11.233619+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24508,"mds_1":24499,"mds_2":24494,"mds_3":24493,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14598":{"gid":14598,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2109255228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2109255228},{"type":"v1","addr":"172.21.15.5:6839","nonce":2109255228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/3809966329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3809966329},{"type":"v1","addr":"172.21.15.5:6841","nonce":3809966329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6833/3272897742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3272897742},{"type":"v1","addr":"172.21.15.131:6833","nonce":3272897742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1283093898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1283093898},{"type":"v1","addr":"172.21.15.131:6835","nonce":1283093898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/2809525455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2809525455},{"type":"v1","addr":"172.21.15.157:6833","nonce":2809525455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.131:6837/1633006186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":1633006186},{"type":"v1","addr":"172.21.15.131:6837","nonce":1633006186}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.157:6835/832898273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":832898273},{"type":"v1","addr":"172.21.15.157:6835","nonce":832898273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.131:6839/2748669631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2748669631},{"type":"v1","addr":"172.21.15.131:6839","nonce":2748669631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.157:6837/744863009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":744863009},{"type":"v1","addr":"172.21.15.157:6837","nonce":744863009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.157:6839/2779860399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":2779860399},{"type":"v1","addr":"172.21.15.157:6839","nonce":2779860399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24508,24494,24499,24493,24467]},"id":1}]} 2024-05-09T00:57:44.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:44.320+0000 7efd7d59a700 1 -- 172.21.15.5:0/2394753792 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7efd740b29b0 msgr2=0x7efd740b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:44.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:44.320+0000 7efd7d59a700 1 --2- 172.21.15.5:0/2394753792 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7efd740b29b0 0x7efd740b4ec0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7efd8806fbc0 tx=0x7efd78009190 comp rx=0 tx=0).stop 2024-05-09T00:57:44.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:44.320+0000 7efd7d59a700 1 -- 172.21.15.5:0/2394753792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd8810b2e0 msgr2=0x7efd8806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:44.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:44.320+0000 7efd7d59a700 1 --2- 172.21.15.5:0/2394753792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd8810b2e0 0x7efd8806e370 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7efd7000b550 tx=0x7efd7000b860 comp rx=0 tx=0).stop 2024-05-09T00:57:44.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:44.322+0000 7efd7d59a700 1 -- 172.21.15.5:0/2394753792 shutdown_connections 2024-05-09T00:57:44.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:44.322+0000 7efd7d59a700 1 --2- 172.21.15.5:0/2394753792 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7efd740b29b0 0x7efd740b4ec0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:44.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:44.322+0000 7efd7d59a700 1 --2- 172.21.15.5:0/2394753792 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd88110970 0x7efd8806e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:44.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:44.322+0000 7efd7d59a700 1 --2- 172.21.15.5:0/2394753792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd8810a990 0x7efd88075240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:44.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:44.322+0000 7efd7d59a700 1 --2- 172.21.15.5:0/2394753792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd8810b2e0 0x7efd8806e370 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:44.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:44.322+0000 7efd7d59a700 1 -- 172.21.15.5:0/2394753792 >> 172.21.15.5:0/2394753792 conn(0x7efd88077c80 msgr2=0x7efd8807b190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:57:44.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:44.324+0000 7efd7d59a700 1 -- 172.21.15.5:0/2394753792 shutdown_connections 2024-05-09T00:57:44.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:44.325+0000 7efd7d59a700 1 -- 172.21.15.5:0/2394753792 wait complete. 2024-05-09T00:57:44.333 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 31 2024-05-09T00:57:44.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:57:43 smithi131 ceph-mon[34547]: pgmap v712: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 827 MiB/s wr, 1.67k op/s 2024-05-09T00:57:45.084 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:57:44 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/2394753792' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:57:45.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:57:44 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/2394753792' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:57:45.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:57:44 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/2394753792' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:57:46.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:57:45 smithi157 ceph-mon[34294]: pgmap v713: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 890 MiB/s wr, 1.86k op/s 2024-05-09T00:57:46.293 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:57:45 smithi005 ceph-mon[30365]: pgmap v713: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 890 MiB/s wr, 1.86k op/s 2024-05-09T00:57:46.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:57:45 smithi131 ceph-mon[34547]: pgmap v713: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 890 MiB/s wr, 1.86k op/s 2024-05-09T00:57:46.794 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-09T00:57:47.623 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:57:48.116 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:57:47 smithi005 ceph-mon[30365]: pgmap v714: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 919 MiB/s wr, 2.09k op/s 2024-05-09T00:57:48.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:57:47 smithi157 ceph-mon[34294]: pgmap v714: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 919 MiB/s wr, 2.09k op/s 2024-05-09T00:57:48.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:57:47 smithi131 ceph-mon[34547]: pgmap v714: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 919 MiB/s wr, 2.09k op/s 2024-05-09T00:57:50.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:57:49 smithi157 ceph-mon[34294]: pgmap v715: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 876 MiB/s wr, 2.20k op/s 2024-05-09T00:57:50.213 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:57:49 smithi005 ceph-mon[30365]: pgmap v715: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 876 MiB/s wr, 2.20k op/s 2024-05-09T00:57:50.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:57:49 smithi131 ceph-mon[34547]: pgmap v715: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 876 MiB/s wr, 2.20k op/s 2024-05-09T00:57:51.286 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:57:50 smithi005 ceph-mon[30365]: pgmap v716: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 939 MiB/s wr, 2.87k op/s 2024-05-09T00:57:51.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:57:50 smithi131 ceph-mon[34547]: pgmap v716: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 939 MiB/s wr, 2.87k op/s 2024-05-09T00:57:51.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:57:50 smithi157 ceph-mon[34294]: pgmap v716: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 939 MiB/s wr, 2.87k op/s 2024-05-09T00:57:52.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.503+0000 7f4360b6b700 1 -- 172.21.15.5:0/3751203257 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4348003c00 con 0x7f435c0764b0 2024-05-09T00:57:52.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.504+0000 7f4363dd1700 1 -- 172.21.15.5:0/3751203257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f435c0764b0 msgr2=0x7f435c114e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:52.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.504+0000 7f4363dd1700 1 --2- 172.21.15.5:0/3751203257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f435c0764b0 0x7f435c114e30 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f4348009990 tx=0x7f4348009ca0 comp rx=0 tx=0).stop 2024-05-09T00:57:52.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.505+0000 7f4363dd1700 1 -- 172.21.15.5:0/3751203257 shutdown_connections 2024-05-09T00:57:52.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.505+0000 7f4363dd1700 1 --2- 172.21.15.5:0/3751203257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f435c115370 0x7f435c117790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:52.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.505+0000 7f4363dd1700 1 --2- 172.21.15.5:0/3751203257 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f435c075b60 0x7f435c075f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:52.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.505+0000 7f4363dd1700 1 --2- 172.21.15.5:0/3751203257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f435c0764b0 0x7f435c114e30 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:52.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.505+0000 7f4363dd1700 1 -- 172.21.15.5:0/3751203257 >> 172.21.15.5:0/3751203257 conn(0x7f435c100040 msgr2=0x7f435c102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:57:52.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.505+0000 7f4363dd1700 1 -- 172.21.15.5:0/3751203257 shutdown_connections 2024-05-09T00:57:52.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.505+0000 7f4363dd1700 1 -- 172.21.15.5:0/3751203257 wait complete. 2024-05-09T00:57:52.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.507+0000 7f4363dd1700 1 Processor -- start 2024-05-09T00:57:52.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.508+0000 7f4363dd1700 1 -- start start 2024-05-09T00:57:52.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.509+0000 7f4363dd1700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f435c075b60 0x7f435c1a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:52.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.509+0000 7f4363dd1700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f435c0764b0 0x7f435c1a1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:52.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.509+0000 7f4363dd1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f435c115370 0x7f435c1a5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:52.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.509+0000 7f4363dd1700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f435c11ac50 con 0x7f435c115370 2024-05-09T00:57:52.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.509+0000 7f4363dd1700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f435c11aad0 con 0x7f435c0764b0 2024-05-09T00:57:52.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.509+0000 7f4363dd1700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f435c11add0 con 0x7f435c075b60 2024-05-09T00:57:52.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.509+0000 7f436236e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f435c115370 0x7f435c1a5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:52.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.509+0000 7f436236e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f435c115370 0x7f435c1a5f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38044/0 (socket says 172.21.15.5:38044) 2024-05-09T00:57:52.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.509+0000 7f436236e700 1 -- 172.21.15.5:0/3392380901 learned_addr learned my addr 172.21.15.5:0/3392380901 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:57:52.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.510+0000 7f436236e700 1 -- 172.21.15.5:0/3392380901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f435c075b60 msgr2=0x7f435c1a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:52.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.510+0000 7f436236e700 1 --2- 172.21.15.5:0/3392380901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f435c075b60 0x7f435c1a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:52.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.510+0000 7f436236e700 1 -- 172.21.15.5:0/3392380901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f435c0764b0 msgr2=0x7f435c1a1c80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T00:57:52.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.510+0000 7f436236e700 1 --2- 172.21.15.5:0/3392380901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f435c0764b0 0x7f435c1a1c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:52.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.510+0000 7f436236e700 1 -- 172.21.15.5:0/3392380901 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f435c1a65b0 con 0x7f435c115370 2024-05-09T00:57:52.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.510+0000 7f436236e700 1 --2- 172.21.15.5:0/3392380901 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f435c115370 0x7f435c1a5f10 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f435400b4a0 tx=0x7f435400b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:57:52.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.511+0000 7f435affd700 1 -- 172.21.15.5:0/3392380901 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4354015020 con 0x7f435c115370 2024-05-09T00:57:52.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.511+0000 7f435affd700 1 -- 172.21.15.5:0/3392380901 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4354011ce0 con 0x7f435c115370 2024-05-09T00:57:52.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.511+0000 7f435affd700 1 -- 172.21.15.5:0/3392380901 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4354010cb0 con 0x7f435c115370 2024-05-09T00:57:52.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.511+0000 7f4363dd1700 1 -- 172.21.15.5:0/3392380901 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f435c1a68a0 con 0x7f435c115370 2024-05-09T00:57:52.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.512+0000 7f4363dd1700 1 -- 172.21.15.5:0/3392380901 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f435c1ae1c0 con 0x7f435c115370 2024-05-09T00:57:52.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.520+0000 7f435affd700 1 -- 172.21.15.5:0/3392380901 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f4354018020 con 0x7f435c115370 2024-05-09T00:57:52.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.521+0000 7f43727e7b80 1 Processor -- start 2024-05-09T00:57:52.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.522+0000 7f435affd700 1 --2- 172.21.15.5:0/3392380901 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f434c0b2a10 0x7f434c0b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:52.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.522+0000 7f43727e7b80 1 -- start start 2024-05-09T00:57:52.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.523+0000 7f436136c700 1 --2- 172.21.15.5:0/3392380901 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f434c0b2a10 0x7f434c0b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:52.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.523+0000 7f435affd700 1 -- 172.21.15.5:0/3392380901 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f43540e1960 con 0x7f435c115370 2024-05-09T00:57:52.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.524+0000 7f436136c700 1 --2- 172.21.15.5:0/3392380901 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f434c0b2a10 0x7f434c0b4f20 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f4348000c00 tx=0x7f4348005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:57:52.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.524+0000 7f43727e7b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea149bc160 0x55ea14a554f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:52.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.524+0000 7f43727e7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea148d24a0 0x55ea14a55a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:52.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.524+0000 7f43727e7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea14a4f6a0 0x55ea14a4fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:52.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.524+0000 7f43727e7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ea14971ff0 con 0x55ea14a4f6a0 2024-05-09T00:57:52.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.524+0000 7f43727e7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55ea149107f0 con 0x55ea148d24a0 2024-05-09T00:57:52.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.524+0000 7f43727e7b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ea1496dfb0 con 0x55ea149bc160 2024-05-09T00:57:52.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.524+0000 7f436236e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea14a4f6a0 0x55ea14a4fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:52.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.524+0000 7f436236e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea14a4f6a0 0x55ea14a4fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38050/0 (socket says 172.21.15.5:38050) 2024-05-09T00:57:52.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.524+0000 7f436236e700 1 -- 172.21.15.5:0/3859986189 learned_addr learned my addr 172.21.15.5:0/3859986189 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:57:52.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.525+0000 7f436236e700 1 -- 172.21.15.5:0/3859986189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea149bc160 msgr2=0x55ea14a554f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:57:52.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.525+0000 7f436236e700 1 --2- 172.21.15.5:0/3859986189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea149bc160 0x55ea14a554f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:52.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.525+0000 7f436236e700 1 -- 172.21.15.5:0/3859986189 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea148d24a0 msgr2=0x55ea14a55a30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:57:52.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.525+0000 7f436236e700 1 --2- 172.21.15.5:0/3859986189 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea148d24a0 0x55ea14a55a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:52.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.525+0000 7f436236e700 1 -- 172.21.15.5:0/3859986189 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea14908370 con 0x55ea14a4f6a0 2024-05-09T00:57:52.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.525+0000 7f436236e700 1 --2- 172.21.15.5:0/3859986189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea14a4f6a0 0x55ea14a4fad0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f4354000c00 tx=0x7f43540a8df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9268442f2d0a244e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:57:52.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.526+0000 7f4347fff700 1 -- 172.21.15.5:0/3859986189 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43540a8ef0 con 0x55ea14a4f6a0 2024-05-09T00:57:52.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.526+0000 7f4347fff700 1 -- 172.21.15.5:0/3859986189 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43540ad2a0 con 0x55ea14a4f6a0 2024-05-09T00:57:52.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.526+0000 7f4347fff700 1 -- 172.21.15.5:0/3859986189 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43540a83b0 con 0x55ea14a4f6a0 2024-05-09T00:57:52.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.531+0000 7f43727e7b80 1 -- 172.21.15.5:0/3859986189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea14a4f6a0 msgr2=0x55ea14a4fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:52.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.531+0000 7f43727e7b80 1 --2- 172.21.15.5:0/3859986189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea14a4f6a0 0x55ea14a4fad0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f4354000c00 tx=0x7f43540a8df0 comp rx=0 tx=0).stop 2024-05-09T00:57:52.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.532+0000 7f43727e7b80 1 -- 172.21.15.5:0/3859986189 shutdown_connections 2024-05-09T00:57:52.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.532+0000 7f43727e7b80 1 --2- 172.21.15.5:0/3859986189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea14a4f6a0 0x55ea14a4fad0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:52.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.532+0000 7f43727e7b80 1 --2- 172.21.15.5:0/3859986189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea149bc160 0x55ea14a554f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:52.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.533+0000 7f43727e7b80 1 --2- 172.21.15.5:0/3859986189 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea148d24a0 0x55ea14a55a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:52.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.533+0000 7f43727e7b80 1 -- 172.21.15.5:0/3859986189 >> 172.21.15.5:0/3859986189 conn(0x55ea149326a0 msgr2=0x55ea14a4a400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:57:52.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.533+0000 7f43727e7b80 1 -- 172.21.15.5:0/3859986189 shutdown_connections 2024-05-09T00:57:52.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.533+0000 7f43727e7b80 1 -- 172.21.15.5:0/3859986189 wait complete. 2024-05-09T00:57:52.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.534+0000 7f43727e7b80 1 -- start start 2024-05-09T00:57:52.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.534+0000 7f43727e7b80 1 Processor -- start 2024-05-09T00:57:52.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.537+0000 7f43727e7b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea148d24a0 0x55ea14a592d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:52.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.537+0000 7f43727e7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea149bc160 0x55ea14a59810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:52.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.537+0000 7f43727e7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea14a4f6a0 0x55ea14a59d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:52.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.537+0000 7f436136c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea149bc160 0x55ea14a59810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:52.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.537+0000 7f436136c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea149bc160 0x55ea14a59810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38062/0 (socket says 172.21.15.5:38062) 2024-05-09T00:57:52.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.537+0000 7f436136c700 1 -- 172.21.15.5:0/921805027 learned_addr learned my addr 172.21.15.5:0/921805027 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:57:52.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.538+0000 7f4361b6d700 1 --2- 172.21.15.5:0/921805027 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea148d24a0 0x55ea14a592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:52.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.538+0000 7f43727e7b80 1 -- 172.21.15.5:0/921805027 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ea147edba0 con 0x55ea149bc160 2024-05-09T00:57:52.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.538+0000 7f43727e7b80 1 -- 172.21.15.5:0/921805027 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55ea1492c150 con 0x55ea14a4f6a0 2024-05-09T00:57:52.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.538+0000 7f43727e7b80 1 -- 172.21.15.5:0/921805027 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55ea14a5c1d0 con 0x55ea148d24a0 2024-05-09T00:57:52.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.538+0000 7f43727e7b80 10 client.0 _ll_register_callbacks cb 0x55ea147e36c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:57:52.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.539+0000 7f436236e700 1 --2- 172.21.15.5:0/921805027 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea14a4f6a0 0x55ea14a59d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:52.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.541+0000 7f436136c700 1 -- 172.21.15.5:0/921805027 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea148d24a0 msgr2=0x55ea14a592d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:57:52.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.541+0000 7f436136c700 1 --2- 172.21.15.5:0/921805027 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea148d24a0 0x55ea14a592d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:52.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.541+0000 7f436136c700 1 -- 172.21.15.5:0/921805027 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea14a4f6a0 msgr2=0x55ea14a59d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:57:52.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.541+0000 7f436136c700 1 --2- 172.21.15.5:0/921805027 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea14a4f6a0 0x55ea14a59d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:52.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.541+0000 7f436136c700 1 -- 172.21.15.5:0/921805027 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea147ed490 con 0x55ea149bc160 2024-05-09T00:57:52.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.542+0000 7f436136c700 1 --2- 172.21.15.5:0/921805027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea149bc160 0x55ea14a59810 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f435c117c70 tx=0x7f4348009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0c0a2b22af3e68a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:57:52.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.542+0000 7f43457fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:57:52.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.543+0000 7f43457fa700 1 -- 172.21.15.5:0/921805027 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434800b9e0 con 0x55ea149bc160 2024-05-09T00:57:52.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.543+0000 7f43457fa700 1 -- 172.21.15.5:0/921805027 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f434800bb80 con 0x55ea149bc160 2024-05-09T00:57:52.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.543+0000 7f43727e7b80 1 -- 172.21.15.5:0/921805027 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43540ad2a0 con 0x55ea149bc160 2024-05-09T00:57:52.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.543+0000 7f43457fa700 1 -- 172.21.15.5:0/921805027 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f434800bd80 con 0x55ea149bc160 2024-05-09T00:57:52.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.543+0000 7f43727e7b80 10 client.16686 fetch_fsmap learned FSMap version 31 2024-05-09T00:57:52.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.543+0000 7f43727e7b80 1 -- 172.21.15.5:0/921805027 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55ea14a66a90 con 0x55ea149bc160 2024-05-09T00:57:52.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.544+0000 7f43457fa700 1 -- 172.21.15.5:0/921805027 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4348033070 con 0x55ea149bc160 2024-05-09T00:57:52.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.544+0000 7f43457fa700 1 -- 172.21.15.5:0/921805027 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f4348036070 con 0x55ea149bc160 2024-05-09T00:57:52.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.545+0000 7f43727e7b80 10 client.16686 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:57:52.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.545+0000 7f43727e7b80 10 client.16686 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T00:57:52.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.545+0000 7f43727e7b80 20 client.16686 populate_metadata read hostname 'smithi005' 2024-05-09T00:57:52.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.545+0000 7f43727e7b80 1 --2- 172.21.15.5:0/921805027 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x55ea14a690d0 0x55ea14a6b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:52.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.545+0000 7f43727e7b80 4 client.16686 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:57:52.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.546+0000 7f43727e7b80 1 -- 172.21.15.5:0/921805027 --> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f43540ad2a0 con 0x55ea14a690d0 2024-05-09T00:57:52.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.547+0000 7f4361b6d700 1 --2- 172.21.15.5:0/921805027 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x55ea14a690d0 0x55ea14a6b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:52.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.551+0000 7f4361b6d700 1 --2- 172.21.15.5:0/921805027 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x55ea14a690d0 0x55ea14a6b4f0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1ea126b8d1663d8c server_cookie=24e64b488318859b in_seq=0 out_seq=0 2024-05-09T00:57:52.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.551+0000 7f43457fa700 10 client.16686 ms_handle_connect on v2:172.21.15.131:6836/1633006186 2024-05-09T00:57:52.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.552+0000 7f43457fa700 1 -- 172.21.15.5:0/921805027 <== mds.4 v2:172.21.15.131:6836/1633006186 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f43540ad2a0 con 0x55ea14a690d0 2024-05-09T00:57:52.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.552+0000 7f43457fa700 10 client.16686 handle_command_reply: tid=0 2024-05-09T00:57:52.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.552+0000 7f43727e7b80 1 client.16686 shutdown 2024-05-09T00:57:52.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.553+0000 7f43727e7b80 1 -- 172.21.15.5:0/921805027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea149bc160 msgr2=0x55ea14a59810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:52.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.553+0000 7f43727e7b80 1 --2- 172.21.15.5:0/921805027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea149bc160 0x55ea14a59810 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f435c117c70 tx=0x7f4348009ca0 comp rx=0 tx=0).stop 2024-05-09T00:57:52.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.553+0000 7f43727e7b80 1 -- 172.21.15.5:0/921805027 shutdown_connections 2024-05-09T00:57:52.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.553+0000 7f43727e7b80 1 --2- 172.21.15.5:0/921805027 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x55ea14a690d0 0x55ea14a6b4f0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:52.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.553+0000 7f43727e7b80 1 --2- 172.21.15.5:0/921805027 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea14a4f6a0 0x55ea14a59d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:52.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.553+0000 7f43457fa700 0 client.16686 ms_handle_reset on v2:172.21.15.131:6836/1633006186 2024-05-09T00:57:52.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.553+0000 7f43727e7b80 1 --2- 172.21.15.5:0/921805027 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea148d24a0 0x55ea14a592d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:52.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.553+0000 7f43727e7b80 1 --2- 172.21.15.5:0/921805027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea149bc160 0x55ea14a59810 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:52.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.554+0000 7f43727e7b80 1 -- 172.21.15.5:0/921805027 >> 172.21.15.5:0/921805027 conn(0x55ea149326a0 msgr2=0x55ea14a4a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:57:52.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.554+0000 7f43727e7b80 1 -- 172.21.15.5:0/921805027 shutdown_connections 2024-05-09T00:57:52.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.554+0000 7f43727e7b80 1 -- 172.21.15.5:0/921805027 wait complete. 2024-05-09T00:57:52.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.555+0000 7f43727e7b80 20 client.16686 trim_cache size 0 max 16384 2024-05-09T00:57:52.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.609+0000 7f43727e7b80 1 Processor -- start 2024-05-09T00:57:52.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.609+0000 7f43727e7b80 1 -- start start 2024-05-09T00:57:52.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.609+0000 7f43727e7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea148d24a0 0x55ea14a55950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:52.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.610+0000 7f43727e7b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea149bc160 0x55ea14a549e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:52.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.610+0000 7f43727e7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea14a4f6a0 0x55ea14a59860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:52.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.610+0000 7f43727e7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f435c1b5670 con 0x55ea14a4f6a0 2024-05-09T00:57:52.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.610+0000 7f43727e7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f435c1b71b0 con 0x55ea148d24a0 2024-05-09T00:57:52.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.610+0000 7f43727e7b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f435c1b5d40 con 0x55ea149bc160 2024-05-09T00:57:52.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.614+0000 7f436136c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea149bc160 0x55ea14a549e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:52.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.614+0000 7f436136c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea149bc160 0x55ea14a549e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:37886/0 (socket says 172.21.15.5:37886) 2024-05-09T00:57:52.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.614+0000 7f436136c700 1 -- 172.21.15.5:0/3880503918 learned_addr learned my addr 172.21.15.5:0/3880503918 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:57:52.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.614+0000 7f4361b6d700 1 --2- 172.21.15.5:0/3880503918 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea148d24a0 0x55ea14a55950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:52.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.616+0000 7f436136c700 1 -- 172.21.15.5:0/3880503918 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea148d24a0 msgr2=0x55ea14a55950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:57:52.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.616+0000 7f436136c700 1 --2- 172.21.15.5:0/3880503918 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea148d24a0 0x55ea14a55950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:52.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.616+0000 7f436136c700 1 -- 172.21.15.5:0/3880503918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea14a4f6a0 msgr2=0x55ea14a59860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:57:52.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.616+0000 7f436136c700 1 --2- 172.21.15.5:0/3880503918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea14a4f6a0 0x55ea14a59860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:52.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.616+0000 7f436136c700 1 -- 172.21.15.5:0/3880503918 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea14a58610 con 0x55ea149bc160 2024-05-09T00:57:52.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.618+0000 7f436136c700 1 --2- 172.21.15.5:0/3880503918 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea149bc160 0x55ea14a549e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f434802d750 tx=0x7f434800bb70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78136a78a51a7564 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:57:52.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.620+0000 7f43477fe700 1 -- 172.21.15.5:0/3880503918 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434803d070 con 0x55ea149bc160 2024-05-09T00:57:52.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.621+0000 7f43477fe700 1 -- 172.21.15.5:0/3880503918 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4348033040 con 0x55ea149bc160 2024-05-09T00:57:52.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.621+0000 7f43477fe700 1 -- 172.21.15.5:0/3880503918 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434800f460 con 0x55ea149bc160 2024-05-09T00:57:52.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.626+0000 7f43727e7b80 1 -- 172.21.15.5:0/3880503918 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea149bc160 msgr2=0x55ea14a549e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:52.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.626+0000 7f43727e7b80 1 --2- 172.21.15.5:0/3880503918 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea149bc160 0x55ea14a549e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f434802d750 tx=0x7f434800bb70 comp rx=0 tx=0).stop 2024-05-09T00:57:52.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.630+0000 7f43727e7b80 1 -- 172.21.15.5:0/3880503918 shutdown_connections 2024-05-09T00:57:52.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.631+0000 7f43727e7b80 1 --2- 172.21.15.5:0/3880503918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea14a4f6a0 0x55ea14a59860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:52.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.631+0000 7f43727e7b80 1 --2- 172.21.15.5:0/3880503918 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea148d24a0 0x55ea14a55950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:52.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.631+0000 7f43727e7b80 1 --2- 172.21.15.5:0/3880503918 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea149bc160 0x55ea14a549e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:52.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.631+0000 7f43727e7b80 1 -- 172.21.15.5:0/3880503918 >> 172.21.15.5:0/3880503918 conn(0x55ea149326a0 msgr2=0x55ea14a542d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:57:52.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.634+0000 7f43727e7b80 1 -- 172.21.15.5:0/3880503918 shutdown_connections 2024-05-09T00:57:52.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.635+0000 7f43727e7b80 1 -- 172.21.15.5:0/3880503918 wait complete. 2024-05-09T00:57:52.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.635+0000 7f43727e7b80 1 -- start start 2024-05-09T00:57:52.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.636+0000 7f43727e7b80 1 Processor -- start 2024-05-09T00:57:52.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.636+0000 7f43727e7b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea148d24a0 0x55ea14acf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:52.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.637+0000 7f43727e7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea149bc160 0x55ea14a612b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:52.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.637+0000 7f43727e7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea14a4f6a0 0x55ea14a617f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:52.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.637+0000 7f43727e7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f435c1b6ae0 con 0x55ea14a4f6a0 2024-05-09T00:57:52.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.637+0000 7f43727e7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f435c1b6410 con 0x55ea149bc160 2024-05-09T00:57:52.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.637+0000 7f43727e7b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f435c1b3610 con 0x55ea148d24a0 2024-05-09T00:57:52.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.637+0000 7f43727e7b80 10 client.0 _ll_register_callbacks cb 0x55ea147e36c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:57:52.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.643+0000 7f4361b6d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea148d24a0 0x55ea14acf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:52.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.643+0000 7f436136c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea149bc160 0x55ea14a612b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:52.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.643+0000 7f436236e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea14a4f6a0 0x55ea14a617f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:52.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.643+0000 7f436236e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea14a4f6a0 0x55ea14a617f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38084/0 (socket says 172.21.15.5:38084) 2024-05-09T00:57:52.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.643+0000 7f436236e700 1 -- 172.21.15.5:0/1928055793 learned_addr learned my addr 172.21.15.5:0/1928055793 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:57:52.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.652+0000 7f436136c700 1 -- 172.21.15.5:0/1928055793 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea148d24a0 msgr2=0x55ea14acf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:57:52.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.652+0000 7f436136c700 1 --2- 172.21.15.5:0/1928055793 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea148d24a0 0x55ea14acf030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:52.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.652+0000 7f436136c700 1 -- 172.21.15.5:0/1928055793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea14a4f6a0 msgr2=0x55ea14a617f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:57:52.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.652+0000 7f436136c700 1 --2- 172.21.15.5:0/1928055793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea14a4f6a0 0x55ea14a617f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:52.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.652+0000 7f436136c700 1 -- 172.21.15.5:0/1928055793 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea14a70470 con 0x55ea149bc160 2024-05-09T00:57:52.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.657+0000 7f436136c700 1 --2- 172.21.15.5:0/1928055793 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea149bc160 0x55ea14a612b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4348009990 tx=0x7f43480386c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60ff65adbadcd864 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:57:52.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.658+0000 7f43457fa700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-09T00:57:52.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.660+0000 7f43457fa700 1 -- 172.21.15.5:0/1928055793 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4348046060 con 0x55ea149bc160 2024-05-09T00:57:52.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.660+0000 7f43457fa700 1 -- 172.21.15.5:0/1928055793 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f434802d9e0 con 0x55ea149bc160 2024-05-09T00:57:52.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.666+0000 7f43727e7b80 1 -- 172.21.15.5:0/1928055793 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4348033040 con 0x55ea149bc160 2024-05-09T00:57:52.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.666+0000 7f43457fa700 1 -- 172.21.15.5:0/1928055793 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f434803d040 con 0x55ea149bc160 2024-05-09T00:57:52.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.667+0000 7f43727e7b80 10 client.26111 fetch_fsmap learned FSMap version 31 2024-05-09T00:57:52.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.667+0000 7f43727e7b80 1 -- 172.21.15.5:0/1928055793 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55ea14ae4910 con 0x55ea149bc160 2024-05-09T00:57:52.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.667+0000 7f43457fa700 1 -- 172.21.15.5:0/1928055793 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4348033040 con 0x55ea149bc160 2024-05-09T00:57:52.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.668+0000 7f43457fa700 1 -- 172.21.15.5:0/1928055793 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f4348040030 con 0x55ea149bc160 2024-05-09T00:57:52.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.668+0000 7f43727e7b80 10 client.26111 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:57:52.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.668+0000 7f43727e7b80 10 client.26111 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T00:57:52.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.668+0000 7f43727e7b80 20 client.26111 populate_metadata read hostname 'smithi005' 2024-05-09T00:57:52.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.668+0000 7f43727e7b80 1 --2- 172.21.15.5:0/1928055793 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x55ea14a690d0 0x55ea14ae6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:52.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.668+0000 7f43727e7b80 4 client.26111 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:57:52.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.668+0000 7f43727e7b80 1 -- 172.21.15.5:0/1928055793 --> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ea1497f9d0 con 0x55ea14a690d0 2024-05-09T00:57:52.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.669+0000 7f4361b6d700 1 --2- 172.21.15.5:0/1928055793 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x55ea14a690d0 0x55ea14ae6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:52.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.671+0000 7f43457fa700 10 client.26111 ms_handle_connect on v2:172.21.15.131:6836/1633006186 2024-05-09T00:57:52.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.671+0000 7f4361b6d700 1 --2- 172.21.15.5:0/1928055793 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x55ea14a690d0 0x55ea14ae6ea0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=21ee1c4ba74f766d server_cookie=1f7df2e08db1b9ff in_seq=0 out_seq=0 2024-05-09T00:57:52.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.676+0000 7f43457fa700 1 -- 172.21.15.5:0/1928055793 <== mds.4 v2:172.21.15.131:6836/1633006186 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ea1497f9d0 con 0x55ea14a690d0 2024-05-09T00:57:52.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.677+0000 7f43457fa700 10 client.26111 handle_command_reply: tid=0 2024-05-09T00:57:52.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.680+0000 7f43727e7b80 1 client.26111 shutdown 2024-05-09T00:57:52.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.687+0000 7f43727e7b80 1 -- 172.21.15.5:0/1928055793 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea149bc160 msgr2=0x55ea14a612b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:52.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.687+0000 7f43727e7b80 1 --2- 172.21.15.5:0/1928055793 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea149bc160 0x55ea14a612b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4348009990 tx=0x7f43480386c0 comp rx=0 tx=0).stop 2024-05-09T00:57:52.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.688+0000 7f43727e7b80 1 -- 172.21.15.5:0/1928055793 shutdown_connections 2024-05-09T00:57:52.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.688+0000 7f43727e7b80 1 --2- 172.21.15.5:0/1928055793 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x55ea14a690d0 0x55ea14ae6ea0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:52.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.688+0000 7f43727e7b80 1 --2- 172.21.15.5:0/1928055793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea14a4f6a0 0x55ea14a617f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:52.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.689+0000 7f43727e7b80 1 --2- 172.21.15.5:0/1928055793 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ea148d24a0 0x55ea14acf030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:52.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.689+0000 7f43727e7b80 1 --2- 172.21.15.5:0/1928055793 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea149bc160 0x55ea14a612b0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:52.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.689+0000 7f43727e7b80 1 -- 172.21.15.5:0/1928055793 >> 172.21.15.5:0/1928055793 conn(0x55ea149326a0 msgr2=0x55ea14a55950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:57:52.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.689+0000 7f43457fa700 0 client.26111 ms_handle_reset on v2:172.21.15.131:6836/1633006186 2024-05-09T00:57:52.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.689+0000 7f43727e7b80 1 -- 172.21.15.5:0/1928055793 shutdown_connections 2024-05-09T00:57:52.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.691+0000 7f43727e7b80 1 -- 172.21.15.5:0/1928055793 wait complete. 2024-05-09T00:57:52.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.691+0000 7f43727e7b80 20 client.26111 trim_cache size 0 max 16384 2024-05-09T00:57:52.693 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-09T00:57:52.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.697+0000 7f4363dd1700 1 -- 172.21.15.5:0/3392380901 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f434c0b2a10 msgr2=0x7f434c0b4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:52.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.697+0000 7f4363dd1700 1 --2- 172.21.15.5:0/3392380901 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f434c0b2a10 0x7f434c0b4f20 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f4348000c00 tx=0x7f4348005fb0 comp rx=0 tx=0).stop 2024-05-09T00:57:52.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.697+0000 7f4363dd1700 1 -- 172.21.15.5:0/3392380901 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f435c115370 msgr2=0x7f435c1a5f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:52.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.697+0000 7f4363dd1700 1 --2- 172.21.15.5:0/3392380901 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f435c115370 0x7f435c1a5f10 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f435400b4a0 tx=0x7f435400b860 comp rx=0 tx=0).stop 2024-05-09T00:57:52.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.697+0000 7f4363dd1700 1 -- 172.21.15.5:0/3392380901 shutdown_connections 2024-05-09T00:57:52.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.698+0000 7f4363dd1700 1 --2- 172.21.15.5:0/3392380901 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f434c0b2a10 0x7f434c0b4f20 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:52.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.698+0000 7f4363dd1700 1 --2- 172.21.15.5:0/3392380901 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f435c115370 0x7f435c1a5f10 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:52.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.698+0000 7f4363dd1700 1 --2- 172.21.15.5:0/3392380901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f435c075b60 0x7f435c1a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:52.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.698+0000 7f4363dd1700 1 --2- 172.21.15.5:0/3392380901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f435c0764b0 0x7f435c1a1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:52.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.698+0000 7f4363dd1700 1 -- 172.21.15.5:0/3392380901 >> 172.21.15.5:0/3392380901 conn(0x7f435c100040 msgr2=0x7f435c100690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:57:52.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.702+0000 7f4363dd1700 1 -- 172.21.15.5:0/3392380901 shutdown_connections 2024-05-09T00:57:52.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:57:52.702+0000 7f4363dd1700 1 -- 172.21.15.5:0/3392380901 wait complete. 2024-05-09T00:57:53.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:57:53 smithi005 ceph-mon[30365]: pgmap v717: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 798 MiB/s wr, 2.76k op/s 2024-05-09T00:57:53.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:57:53 smithi131 ceph-mon[34547]: pgmap v717: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 798 MiB/s wr, 2.76k op/s 2024-05-09T00:57:53.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:57:53 smithi157 ceph-mon[34294]: pgmap v717: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 798 MiB/s wr, 2.76k op/s 2024-05-09T00:57:55.252 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-09T00:57:55.796 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:57:55 smithi005 ceph-mon[30365]: pgmap v718: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 815 MiB/s wr, 2.74k op/s 2024-05-09T00:57:55.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:57:55 smithi131 ceph-mon[34547]: pgmap v718: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 815 MiB/s wr, 2.74k op/s 2024-05-09T00:57:55.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:57:55 smithi157 ceph-mon[34294]: pgmap v718: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 815 MiB/s wr, 2.74k op/s 2024-05-09T00:57:56.229 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:57:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.685+0000 7facf9f9d640 1 -- 172.21.15.5:0/2544927707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facf4109820 msgr2=0x7facf410bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.685+0000 7facf9f9d640 1 --2- 172.21.15.5:0/2544927707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facf4109820 0x7facf410bc40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7facec009be0 tx=0x7facec02a6f0 comp rx=0 tx=0).stop 2024-05-09T00:57:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.686+0000 7facf9f9d640 1 -- 172.21.15.5:0/2544927707 shutdown_connections 2024-05-09T00:57:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.686+0000 7facf9f9d640 1 --2- 172.21.15.5:0/2544927707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facf4109820 0x7facf410bc40 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.686+0000 7facf9f9d640 1 --2- 172.21.15.5:0/2544927707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7facf4106ec0 0x7facf41092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.686+0000 7facf9f9d640 1 --2- 172.21.15.5:0/2544927707 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7facf4104560 0x7facf4106980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.686+0000 7facf9f9d640 1 -- 172.21.15.5:0/2544927707 >> 172.21.15.5:0/2544927707 conn(0x7facf40fdb60 msgr2=0x7facf4100010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:57:56.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.693+0000 7facf9f9d640 1 -- 172.21.15.5:0/2544927707 shutdown_connections 2024-05-09T00:57:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.693+0000 7facf9f9d640 1 -- 172.21.15.5:0/2544927707 wait complete. 2024-05-09T00:57:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.694+0000 7facf9f9d640 1 Processor -- start 2024-05-09T00:57:56.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.694+0000 7facf9f9d640 1 -- start start 2024-05-09T00:57:56.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.695+0000 7facf9f9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7facf4104560 0x7facf4113240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:56.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.695+0000 7facf9f9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7facf4106ec0 0x7facf4113780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:56.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.695+0000 7facf9f9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facf4109820 0x7facf4118700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:56.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.695+0000 7facf9f9d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7facf4103dd0 con 0x7facf4104560 2024-05-09T00:57:56.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.695+0000 7facf9f9d640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7facf4103c50 con 0x7facf4106ec0 2024-05-09T00:57:56.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.695+0000 7facf9f9d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7facf4103f50 con 0x7facf4109820 2024-05-09T00:57:56.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.695+0000 7facf8f9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7facf4104560 0x7facf4113240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.696+0000 7facf979c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facf4109820 0x7facf4118700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.696+0000 7facf979c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facf4109820 0x7facf4118700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:48966/0 (socket says 172.21.15.5:48966) 2024-05-09T00:57:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.696+0000 7facf979c640 1 -- 172.21.15.5:0/587926727 learned_addr learned my addr 172.21.15.5:0/587926727 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:57:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.696+0000 7facf3fff640 1 --2- 172.21.15.5:0/587926727 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7facf4106ec0 0x7facf4113780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.696+0000 7facf8f9b640 1 -- 172.21.15.5:0/587926727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facf4109820 msgr2=0x7facf4118700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.697+0000 7facf8f9b640 1 --2- 172.21.15.5:0/587926727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facf4109820 0x7facf4118700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.697+0000 7facf8f9b640 1 -- 172.21.15.5:0/587926727 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7facf4106ec0 msgr2=0x7facf4113780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.697+0000 7facf8f9b640 1 --2- 172.21.15.5:0/587926727 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7facf4106ec0 0x7facf4113780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.697+0000 7facf8f9b640 1 -- 172.21.15.5:0/587926727 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facf4118d70 con 0x7facf4104560 2024-05-09T00:57:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.697+0000 7facf979c640 1 --2- 172.21.15.5:0/587926727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facf4109820 0x7facf4118700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:57:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.698+0000 7facf8f9b640 1 --2- 172.21.15.5:0/587926727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7facf4104560 0x7facf4113240 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7face8009fd0 tx=0x7face800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:57:56.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.706+0000 7facf1ffb640 1 -- 172.21.15.5:0/587926727 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face8015020 con 0x7facf4104560 2024-05-09T00:57:56.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.706+0000 7facf9f9d640 1 -- 172.21.15.5:0/587926727 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facf4119060 con 0x7facf4104560 2024-05-09T00:57:56.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.707+0000 7facf9f9d640 1 -- 172.21.15.5:0/587926727 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facf4119630 con 0x7facf4104560 2024-05-09T00:57:56.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.712+0000 7facf1ffb640 1 -- 172.21.15.5:0/587926727 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7face8011ce0 con 0x7facf4104560 2024-05-09T00:57:56.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.712+0000 7facf1ffb640 1 -- 172.21.15.5:0/587926727 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face8010e00 con 0x7facf4104560 2024-05-09T00:57:56.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.721+0000 7fad02187740 1 Processor -- start 2024-05-09T00:57:56.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.722+0000 7fad02187740 1 -- start start 2024-05-09T00:57:56.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.722+0000 7fad02187740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561fd81c31b0 0x561fd85676c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:56.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.723+0000 7fad02187740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fd819a2f0 0x561fd8567c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:56.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.723+0000 7fad02187740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561fd81a24d0 0x561fd856eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:56.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.723+0000 7fad02187740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561fd84cb7c0 con 0x561fd819a2f0 2024-05-09T00:57:56.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.723+0000 7fad02187740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x561fd84670e0 con 0x561fd81c31b0 2024-05-09T00:57:56.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.723+0000 7fad02187740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x561fd84c7780 con 0x561fd81a24d0 2024-05-09T00:57:56.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.723+0000 7facf979c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561fd81c31b0 0x561fd85676c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:56.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.723+0000 7facf979c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561fd81c31b0 0x561fd85676c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:52564/0 (socket says 172.21.15.5:52564) 2024-05-09T00:57:56.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.723+0000 7facf979c640 1 -- 172.21.15.5:0/1023765003 learned_addr learned my addr 172.21.15.5:0/1023765003 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:57:56.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.724+0000 7facf979c640 1 -- 172.21.15.5:0/1023765003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561fd81a24d0 msgr2=0x561fd856eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:57:56.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.724+0000 7facf979c640 1 --2- 172.21.15.5:0/1023765003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561fd81a24d0 0x561fd856eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:56.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.724+0000 7facf979c640 1 -- 172.21.15.5:0/1023765003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fd819a2f0 msgr2=0x561fd8567c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:57:56.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.724+0000 7facf979c640 1 --2- 172.21.15.5:0/1023765003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fd819a2f0 0x561fd8567c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:56.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.724+0000 7facf979c640 1 -- 172.21.15.5:0/1023765003 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fd84907c0 con 0x561fd81c31b0 2024-05-09T00:57:56.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.725+0000 7facf979c640 1 --2- 172.21.15.5:0/1023765003 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561fd81c31b0 0x561fd85676c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7facec009b30 tx=0x7facec030c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab6ad506b7e6a466 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:57:56.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.727+0000 7facdeffd640 1 -- 172.21.15.5:0/1023765003 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facec041030 con 0x561fd81c31b0 2024-05-09T00:57:56.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.727+0000 7facdeffd640 1 -- 172.21.15.5:0/1023765003 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facec003e20 con 0x561fd81c31b0 2024-05-09T00:57:56.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.728+0000 7facf1ffb640 1 -- 172.21.15.5:0/587926727 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7face8018020 con 0x7facf4104560 2024-05-09T00:57:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.729+0000 7fad02187740 1 -- 172.21.15.5:0/1023765003 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561fd81c31b0 msgr2=0x561fd85676c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.729+0000 7fad02187740 1 --2- 172.21.15.5:0/1023765003 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561fd81c31b0 0x561fd85676c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7facec009b30 tx=0x7facec030c20 comp rx=0 tx=0).stop 2024-05-09T00:57:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.730+0000 7facf1ffb640 1 --2- 172.21.15.5:0/587926727 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7face40b29b0 0x7face40b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.730+0000 7facf1ffb640 1 -- 172.21.15.5:0/587926727 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7face80e3aa0 con 0x7facf4104560 2024-05-09T00:57:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.736+0000 7facf979c640 1 --2- 172.21.15.5:0/587926727 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7face40b29b0 0x7face40b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:56.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.745+0000 7fad02187740 1 -- 172.21.15.5:0/1023765003 shutdown_connections 2024-05-09T00:57:56.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.745+0000 7fad02187740 1 --2- 172.21.15.5:0/1023765003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561fd81a24d0 0x561fd856eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:56.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.745+0000 7fad02187740 1 --2- 172.21.15.5:0/1023765003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fd819a2f0 0x561fd8567c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:56.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.745+0000 7fad02187740 1 --2- 172.21.15.5:0/1023765003 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561fd81c31b0 0x561fd85676c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:56.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.745+0000 7fad02187740 1 -- 172.21.15.5:0/1023765003 >> 172.21.15.5:0/1023765003 conn(0x561fd81b7300 msgr2=0x561fd8563050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:57:56.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.750+0000 7facf979c640 1 --2- 172.21.15.5:0/587926727 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7face40b29b0 0x7face40b4ec0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7facec009420 tx=0x7facec009880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:57:56.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.753+0000 7fad02187740 1 -- 172.21.15.5:0/1023765003 shutdown_connections 2024-05-09T00:57:56.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.753+0000 7fad02187740 1 -- 172.21.15.5:0/1023765003 wait complete. 2024-05-09T00:57:56.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.760+0000 7fad02187740 1 -- start start 2024-05-09T00:57:56.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.760+0000 7fad02187740 1 Processor -- start 2024-05-09T00:57:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.761+0000 7fad02187740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561fd819a2f0 0x561fd85713d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.761+0000 7fad02187740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fd81a24d0 0x561fd8571910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.762+0000 7fad02187740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561fd81c31b0 0x561fd8578870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.762+0000 7fad02187740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561fd856f580 con 0x561fd81a24d0 2024-05-09T00:57:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.762+0000 7fad02187740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x561fd84529c0 con 0x561fd81c31b0 2024-05-09T00:57:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.762+0000 7fad02187740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x561fd856f700 con 0x561fd819a2f0 2024-05-09T00:57:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.762+0000 7facf3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fd81a24d0 0x561fd8571910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.763+0000 7facf3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fd81a24d0 0x561fd8571910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37956/0 (socket says 172.21.15.5:37956) 2024-05-09T00:57:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.763+0000 7facf3fff640 1 -- 172.21.15.5:0/1405693202 learned_addr learned my addr 172.21.15.5:0/1405693202 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:57:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.763+0000 7facf979c640 1 --2- 172.21.15.5:0/1405693202 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561fd81c31b0 0x561fd8578870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.763+0000 7fad02187740 10 client.0 _ll_register_callbacks cb 0x561fd847f020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:57:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.765+0000 7facf979c640 1 -- 172.21.15.5:0/1405693202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561fd819a2f0 msgr2=0x561fd85713d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:57:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.765+0000 7facf979c640 1 --2- 172.21.15.5:0/1405693202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561fd819a2f0 0x561fd85713d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.765+0000 7facf979c640 1 -- 172.21.15.5:0/1405693202 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fd81a24d0 msgr2=0x561fd8571910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:57:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.765+0000 7facf979c640 1 --2- 172.21.15.5:0/1405693202 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fd81a24d0 0x561fd8571910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.765+0000 7facf979c640 1 -- 172.21.15.5:0/1405693202 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fd84791f0 con 0x561fd81c31b0 2024-05-09T00:57:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.767+0000 7facf979c640 1 --2- 172.21.15.5:0/1405693202 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561fd81c31b0 0x561fd8578870 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7facec02ac10 tx=0x7facec030b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ea8edcaee85576f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:57:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.768+0000 7facd7fff640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-09T00:57:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.770+0000 7facd7fff640 1 -- 172.21.15.5:0/1405693202 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facec041070 con 0x561fd81c31b0 2024-05-09T00:57:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.771+0000 7facd7fff640 1 -- 172.21.15.5:0/1405693202 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7facec031640 con 0x561fd81c31b0 2024-05-09T00:57:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.771+0000 7fad02187740 1 -- 172.21.15.5:0/1405693202 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facec003e20 con 0x561fd81c31b0 2024-05-09T00:57:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.773+0000 7facd7fff640 1 -- 172.21.15.5:0/1405693202 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7facec031840 con 0x561fd81c31b0 2024-05-09T00:57:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.773+0000 7fad02187740 10 client.26120 fetch_fsmap learned FSMap version 31 2024-05-09T00:57:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.773+0000 7fad02187740 1 -- 172.21.15.5:0/1405693202 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561fd822e790 con 0x561fd81c31b0 2024-05-09T00:57:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.776+0000 7facd7fff640 1 -- 172.21.15.5:0/1405693202 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facec042040 con 0x561fd81c31b0 2024-05-09T00:57:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.776+0000 7facd7fff640 1 -- 172.21.15.5:0/1405693202 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7facec050070 con 0x561fd81c31b0 2024-05-09T00:57:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.776+0000 7fad02187740 10 client.26120 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:57:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.776+0000 7fad02187740 10 client.26120 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:57:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.776+0000 7fad02187740 20 client.26120 populate_metadata read hostname 'smithi005' 2024-05-09T00:57:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.776+0000 7fad02187740 1 --2- 172.21.15.5:0/1405693202 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x561fd8181d80 0x561fd857e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:56.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.777+0000 7fad02187740 4 client.26120 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:57:56.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.777+0000 7fad02187740 1 -- 172.21.15.5:0/1405693202 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7facec003e20 con 0x561fd8181d80 2024-05-09T00:57:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.781+0000 7facf8f9b640 1 --2- 172.21.15.5:0/1405693202 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x561fd8181d80 0x561fd857e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:56.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.809+0000 7facf8f9b640 1 --2- 172.21.15.5:0/1405693202 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x561fd8181d80 0x561fd857e910 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7c820346856e5f8 server_cookie=3e3dc8121b3e0ffe in_seq=0 out_seq=0 2024-05-09T00:57:56.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.820+0000 7facd7fff640 10 client.26120 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:57:56.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.823+0000 7facd7fff640 1 -- 172.21.15.5:0/1405693202 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7facec003e20 con 0x561fd8181d80 2024-05-09T00:57:56.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.823+0000 7facd7fff640 10 client.26120 handle_command_reply: tid=0 2024-05-09T00:57:56.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.823+0000 7fad02187740 1 client.26120 shutdown 2024-05-09T00:57:56.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.827+0000 7fad02187740 1 -- 172.21.15.5:0/1405693202 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561fd81c31b0 msgr2=0x561fd8578870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:56.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.827+0000 7fad02187740 1 --2- 172.21.15.5:0/1405693202 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561fd81c31b0 0x561fd8578870 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7facec02ac10 tx=0x7facec030b00 comp rx=0 tx=0).stop 2024-05-09T00:57:56.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.831+0000 7fad02187740 1 -- 172.21.15.5:0/1405693202 shutdown_connections 2024-05-09T00:57:56.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.831+0000 7fad02187740 1 --2- 172.21.15.5:0/1405693202 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x561fd8181d80 0x561fd857e910 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:56.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.831+0000 7fad02187740 1 --2- 172.21.15.5:0/1405693202 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561fd81c31b0 0x561fd8578870 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:56.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.831+0000 7fad02187740 1 --2- 172.21.15.5:0/1405693202 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fd81a24d0 0x561fd8571910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:56.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.831+0000 7fad02187740 1 --2- 172.21.15.5:0/1405693202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561fd819a2f0 0x561fd85713d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:56.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.831+0000 7fad02187740 1 -- 172.21.15.5:0/1405693202 >> 172.21.15.5:0/1405693202 conn(0x561fd81b7300 msgr2=0x561fd8561860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:57:56.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.831+0000 7facd7fff640 0 client.26120 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:57:56.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.832+0000 7fad02187740 1 -- 172.21.15.5:0/1405693202 shutdown_connections 2024-05-09T00:57:56.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.832+0000 7fad02187740 1 -- 172.21.15.5:0/1405693202 wait complete. 2024-05-09T00:57:56.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.832+0000 7fad02187740 20 client.26120 trim_cache size 0 max 16384 2024-05-09T00:57:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.905+0000 7fad02187740 1 Processor -- start 2024-05-09T00:57:56.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.908+0000 7fad02187740 1 -- start start 2024-05-09T00:57:56.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.911+0000 7fad02187740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561fd819a2f0 0x561fd8570d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:56.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.911+0000 7fad02187740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561fd81a24d0 0x561fd8578790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:56.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.911+0000 7fad02187740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fd81c31b0 0x561fd8583040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.912+0000 7fad02187740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7facd800e680 con 0x561fd81c31b0 2024-05-09T00:57:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.912+0000 7fad02187740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7facd80101c0 con 0x561fd819a2f0 2024-05-09T00:57:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.912+0000 7fad02187740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7facd800ed50 con 0x561fd81a24d0 2024-05-09T00:57:56.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.912+0000 7facf8f9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561fd819a2f0 0x561fd8570d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:56.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.913+0000 7facf8f9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561fd819a2f0 0x561fd8570d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:52580/0 (socket says 172.21.15.5:52580) 2024-05-09T00:57:56.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.913+0000 7facf8f9b640 1 -- 172.21.15.5:0/2467964844 learned_addr learned my addr 172.21.15.5:0/2467964844 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:57:56.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.913+0000 7facf979c640 1 --2- 172.21.15.5:0/2467964844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fd81c31b0 0x561fd8583040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:56.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.915+0000 7facf979c640 1 -- 172.21.15.5:0/2467964844 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561fd81a24d0 msgr2=0x561fd8578790 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:57:56.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.915+0000 7facf979c640 1 --2- 172.21.15.5:0/2467964844 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561fd81a24d0 0x561fd8578790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:56.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.915+0000 7facf979c640 1 -- 172.21.15.5:0/2467964844 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561fd819a2f0 msgr2=0x561fd8570d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:57:56.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.915+0000 7facf979c640 1 --2- 172.21.15.5:0/2467964844 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561fd819a2f0 0x561fd8570d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.915+0000 7facf979c640 1 -- 172.21.15.5:0/2467964844 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fd82a55f0 con 0x561fd81c31b0 2024-05-09T00:57:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.915+0000 7facf8f9b640 1 --2- 172.21.15.5:0/2467964844 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561fd819a2f0 0x561fd8570d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:57:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.917+0000 7facf979c640 1 --2- 172.21.15.5:0/2467964844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fd81c31b0 0x561fd8583040 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7facec060040 tx=0x7facec030990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3386a89f4d3714c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:57:56.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.918+0000 7facde7fc640 1 -- 172.21.15.5:0/2467964844 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facec041070 con 0x561fd81c31b0 2024-05-09T00:57:56.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.918+0000 7facde7fc640 1 -- 172.21.15.5:0/2467964844 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facec004100 con 0x561fd81c31b0 2024-05-09T00:57:56.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.918+0000 7facde7fc640 1 -- 172.21.15.5:0/2467964844 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facec0316a0 con 0x561fd81c31b0 2024-05-09T00:57:56.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.920+0000 7fad02187740 1 -- 172.21.15.5:0/2467964844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fd81c31b0 msgr2=0x561fd8583040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:57:56.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.920+0000 7fad02187740 1 --2- 172.21.15.5:0/2467964844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fd81c31b0 0x561fd8583040 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7facec060040 tx=0x7facec030990 comp rx=0 tx=0).stop 2024-05-09T00:57:56.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.921+0000 7fad02187740 1 -- 172.21.15.5:0/2467964844 shutdown_connections 2024-05-09T00:57:56.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.922+0000 7fad02187740 1 --2- 172.21.15.5:0/2467964844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fd81c31b0 0x561fd8583040 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:56.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.922+0000 7fad02187740 1 --2- 172.21.15.5:0/2467964844 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561fd81a24d0 0x561fd8578790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:56.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.922+0000 7fad02187740 1 --2- 172.21.15.5:0/2467964844 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561fd819a2f0 0x561fd8570d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:56.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.922+0000 7fad02187740 1 -- 172.21.15.5:0/2467964844 >> 172.21.15.5:0/2467964844 conn(0x561fd81b7300 msgr2=0x561fd8570660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:57:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.931+0000 7fad02187740 1 -- 172.21.15.5:0/2467964844 shutdown_connections 2024-05-09T00:57:56.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.931+0000 7fad02187740 1 -- 172.21.15.5:0/2467964844 wait complete. 2024-05-09T00:57:56.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.932+0000 7fad02187740 1 -- start start 2024-05-09T00:57:56.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.932+0000 7fad02187740 1 Processor -- start 2024-05-09T00:57:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.933+0000 7fad02187740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561fd819a2f0 0x561fd8575a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.933+0000 7fad02187740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561fd81a24d0 0x561fd8575f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.934+0000 7fad02187740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fd81c31b0 0x561fd8576e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.934+0000 7fad02187740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7facd800faf0 con 0x561fd81c31b0 2024-05-09T00:57:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.934+0000 7fad02187740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7facd800f420 con 0x561fd819a2f0 2024-05-09T00:57:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.934+0000 7fad02187740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7facd800c660 con 0x561fd81a24d0 2024-05-09T00:57:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.934+0000 7fad02187740 10 client.0 _ll_register_callbacks cb 0x7facec030b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:57:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.934+0000 7facf979c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fd81c31b0 0x561fd8576e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.934+0000 7facf8f9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561fd819a2f0 0x561fd8575a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.934+0000 7facf979c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fd81c31b0 0x561fd8576e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37970/0 (socket says 172.21.15.5:37970) 2024-05-09T00:57:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.934+0000 7facf979c640 1 -- 172.21.15.5:0/3924306597 learned_addr learned my addr 172.21.15.5:0/3924306597 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:57:56.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.935+0000 7facf979c640 1 -- 172.21.15.5:0/3924306597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561fd81a24d0 msgr2=0x561fd8575f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:57:56.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.935+0000 7facf979c640 1 --2- 172.21.15.5:0/3924306597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561fd81a24d0 0x561fd8575f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:56.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.935+0000 7facf979c640 1 -- 172.21.15.5:0/3924306597 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561fd819a2f0 msgr2=0x561fd8575a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:57:56.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.935+0000 7facf979c640 1 --2- 172.21.15.5:0/3924306597 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561fd819a2f0 0x561fd8575a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:57:56.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.935+0000 7facf979c640 1 -- 172.21.15.5:0/3924306597 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fd848fed0 con 0x561fd81c31b0 2024-05-09T00:57:56.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.935+0000 7facf979c640 1 --2- 172.21.15.5:0/3924306597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fd81c31b0 0x561fd8576e90 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7facec000f80 tx=0x7facec003ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9d7a1a1900e2f98 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:57:56.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.935+0000 7facd7fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:57:56.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.935+0000 7facd7fff640 1 -- 172.21.15.5:0/3924306597 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facec041040 con 0x561fd81c31b0 2024-05-09T00:57:56.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.936+0000 7facd7fff640 1 -- 172.21.15.5:0/3924306597 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7facec03d460 con 0x561fd81c31b0 2024-05-09T00:57:56.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.936+0000 7fad02187740 1 -- 172.21.15.5:0/3924306597 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facec004100 con 0x561fd81c31b0 2024-05-09T00:57:56.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.936+0000 7facd7fff640 1 -- 172.21.15.5:0/3924306597 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7facec03daf0 con 0x561fd81c31b0 2024-05-09T00:57:56.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.936+0000 7fad02187740 10 client.16704 fetch_fsmap learned FSMap version 31 2024-05-09T00:57:56.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.936+0000 7fad02187740 1 -- 172.21.15.5:0/3924306597 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561fd848fed0 con 0x561fd81c31b0 2024-05-09T00:57:56.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.937+0000 7facd7fff640 1 -- 172.21.15.5:0/3924306597 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facec004100 con 0x561fd81c31b0 2024-05-09T00:57:56.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.937+0000 7facd7fff640 1 -- 172.21.15.5:0/3924306597 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7facec050070 con 0x561fd81c31b0 2024-05-09T00:57:56.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.937+0000 7fad02187740 10 client.16704 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:57:56.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.937+0000 7fad02187740 10 client.16704 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:57:56.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.937+0000 7fad02187740 20 client.16704 populate_metadata read hostname 'smithi005' 2024-05-09T00:57:56.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.937+0000 7fad02187740 1 --2- 172.21.15.5:0/3924306597 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x561fd81571c0 0x561fd85e8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:57:56.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.937+0000 7fad02187740 4 client.16704 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:57:56.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.937+0000 7fad02187740 1 -- 172.21.15.5:0/3924306597 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561fd84d93d0 con 0x561fd81571c0 2024-05-09T00:57:56.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.938+0000 7facf8f9b640 1 --2- 172.21.15.5:0/3924306597 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x561fd81571c0 0x561fd85e8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:57:56.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.941+0000 7facf8f9b640 1 --2- 172.21.15.5:0/3924306597 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x561fd81571c0 0x561fd85e8c10 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13d90c584e0f3118 server_cookie=49c3d8f48aa32670 in_seq=0 out_seq=0 2024-05-09T00:57:56.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:57:56.941+0000 7facd7fff640 10 client.16704 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:57:57.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:57:57 smithi005 ceph-mon[30365]: pgmap v719: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 885 MiB/s wr, 2.71k op/s 2024-05-09T00:57:57.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:57:57 smithi131 ceph-mon[34547]: pgmap v719: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 885 MiB/s wr, 2.71k op/s 2024-05-09T00:57:57.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:57:57 smithi157 ceph-mon[34294]: pgmap v719: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 885 MiB/s wr, 2.71k op/s 2024-05-09T00:57:59.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:57:59 smithi005 ceph-mon[30365]: pgmap v720: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 840 MiB/s wr, 2.56k op/s 2024-05-09T00:57:59.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:57:59 smithi131 ceph-mon[34547]: pgmap v720: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 840 MiB/s wr, 2.56k op/s 2024-05-09T00:57:59.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:57:59 smithi157 ceph-mon[34294]: pgmap v720: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 840 MiB/s wr, 2.56k op/s 2024-05-09T00:58:01.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.495+0000 7f7c3a227700 1 -- 172.21.15.5:0/2889097188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c3410a930 msgr2=0x7f7c3410ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:01.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.495+0000 7f7c3a227700 1 --2- 172.21.15.5:0/2889097188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c3410a930 0x7f7c3410ad40 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f7c280098f0 tx=0x7f7c28009c00 comp rx=0 tx=0).stop 2024-05-09T00:58:01.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.496+0000 7f7c3a227700 1 -- 172.21.15.5:0/2889097188 shutdown_connections 2024-05-09T00:58:01.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.496+0000 7f7c3a227700 1 --2- 172.21.15.5:0/2889097188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c3411d930 0x7f7c3411fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:01.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.496+0000 7f7c3a227700 1 --2- 172.21.15.5:0/2889097188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c3410a930 0x7f7c3410ad40 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:01.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.496+0000 7f7c3a227700 1 --2- 172.21.15.5:0/2889097188 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c3410b280 0x7f7c3411d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:01.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.496+0000 7f7c3a227700 1 -- 172.21.15.5:0/2889097188 >> 172.21.15.5:0/2889097188 conn(0x7f7c3406f4d0 msgr2=0x7f7c3406f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:58:01.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.501+0000 7f7c3a227700 1 -- 172.21.15.5:0/2889097188 shutdown_connections 2024-05-09T00:58:01.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.504+0000 7f7c3a227700 1 -- 172.21.15.5:0/2889097188 wait complete. 2024-05-09T00:58:01.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.506+0000 7f7c3a227700 1 Processor -- start 2024-05-09T00:58:01.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.506+0000 7f7c3a227700 1 -- start start 2024-05-09T00:58:01.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.507+0000 7f7c3a227700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c3410a930 0x7f7c3411bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:01.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.507+0000 7f7c3a227700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c3410b280 0x7f7c34114ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:01.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.507+0000 7f7c3a227700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c3411d930 0x7f7c341154a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:01.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.507+0000 7f7c3a227700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7c34123330 con 0x7f7c3410a930 2024-05-09T00:58:01.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.507+0000 7f7c3a227700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7c341231b0 con 0x7f7c3410b280 2024-05-09T00:58:01.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.507+0000 7f7c3a227700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7c341234b0 con 0x7f7c3411d930 2024-05-09T00:58:01.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.509+0000 7f7c32ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c3410b280 0x7f7c34114ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:01.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.511+0000 7f7c32ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c3410b280 0x7f7c34114ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:52604/0 (socket says 172.21.15.5:52604) 2024-05-09T00:58:01.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.512+0000 7f7c32ffd700 1 -- 172.21.15.5:0/600964587 learned_addr learned my addr 172.21.15.5:0/600964587 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:58:01.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.512+0000 7f7c337fe700 1 --2- 172.21.15.5:0/600964587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c3410a930 0x7f7c3411bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:01.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.516+0000 7f7c32ffd700 1 -- 172.21.15.5:0/600964587 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c3411d930 msgr2=0x7f7c341154a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:01.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.516+0000 7f7c32ffd700 1 --2- 172.21.15.5:0/600964587 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c3411d930 0x7f7c341154a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:01.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.516+0000 7f7c32ffd700 1 -- 172.21.15.5:0/600964587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c3410a930 msgr2=0x7f7c3411bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:01.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.516+0000 7f7c32ffd700 1 --2- 172.21.15.5:0/600964587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c3410a930 0x7f7c3411bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:01.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.516+0000 7f7c32ffd700 1 -- 172.21.15.5:0/600964587 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c34115d00 con 0x7f7c3410b280 2024-05-09T00:58:01.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.517+0000 7f7c32ffd700 1 --2- 172.21.15.5:0/600964587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c3410b280 0x7f7c34114ec0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7c2400e9c0 tx=0x7f7c2400ed80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:58:01.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.521+0000 7f7c30ff9700 1 -- 172.21.15.5:0/600964587 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c2400cbd0 con 0x7f7c3410b280 2024-05-09T00:58:01.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.521+0000 7f7c30ff9700 1 -- 172.21.15.5:0/600964587 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c2400cd70 con 0x7f7c3410b280 2024-05-09T00:58:01.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.521+0000 7f7c30ff9700 1 -- 172.21.15.5:0/600964587 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c24005580 con 0x7f7c3410b280 2024-05-09T00:58:01.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.525+0000 7f7c3a227700 1 -- 172.21.15.5:0/600964587 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c341199e0 con 0x7f7c3410b280 2024-05-09T00:58:01.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.525+0000 7f7c3a227700 1 -- 172.21.15.5:0/600964587 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c34119e40 con 0x7f7c3410b280 2024-05-09T00:58:01.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.535+0000 7f7c30ff9700 1 -- 172.21.15.5:0/600964587 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f7c24004750 con 0x7f7c3410b280 2024-05-09T00:58:01.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.537+0000 7f7c30ff9700 1 --2- 172.21.15.5:0/600964587 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f7c1c0b2a90 0x7f7c1c0b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:01.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.540+0000 7f7c30ff9700 1 -- 172.21.15.5:0/600964587 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f7c240e24f0 con 0x7f7c3410b280 2024-05-09T00:58:01.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.541+0000 7f7c337fe700 1 --2- 172.21.15.5:0/600964587 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f7c1c0b2a90 0x7f7c1c0b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:01.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.547+0000 7f7c48c3db80 1 Processor -- start 2024-05-09T00:58:01.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.547+0000 7f7c48c3db80 1 -- start start 2024-05-09T00:58:01.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.548+0000 7f7c48c3db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b6a86e7160 0x55b6a8780480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:01.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.548+0000 7f7c48c3db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6a85fd4a0 0x55b6a8780a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:01.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.548+0000 7f7c48c3db80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b6a877a6a0 0x55b6a877aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:01.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.548+0000 7f7c48c3db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b6a869cff0 con 0x55b6a85fd4a0 2024-05-09T00:58:01.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.548+0000 7f7c48c3db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55b6a863b7f0 con 0x55b6a86e7160 2024-05-09T00:58:01.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.548+0000 7f7c48c3db80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b6a8698fb0 con 0x55b6a877a6a0 2024-05-09T00:58:01.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.553+0000 7f7c32ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6a85fd4a0 0x55b6a8780a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:01.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.553+0000 7f7c32ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6a85fd4a0 0x55b6a8780a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37992/0 (socket says 172.21.15.5:37992) 2024-05-09T00:58:01.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.553+0000 7f7c32ffd700 1 -- 172.21.15.5:0/2379458883 learned_addr learned my addr 172.21.15.5:0/2379458883 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:58:01.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.553+0000 7f7c33fff700 1 --2- 172.21.15.5:0/2379458883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b6a877a6a0 0x55b6a877aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:01.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.553+0000 7f7c32ffd700 1 -- 172.21.15.5:0/2379458883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b6a877a6a0 msgr2=0x55b6a877aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:58:01.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.553+0000 7f7c32ffd700 1 --2- 172.21.15.5:0/2379458883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b6a877a6a0 0x55b6a877aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:01.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.553+0000 7f7c32ffd700 1 -- 172.21.15.5:0/2379458883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b6a86e7160 msgr2=0x55b6a8780480 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:58:01.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.553+0000 7f7c32ffd700 1 --2- 172.21.15.5:0/2379458883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b6a86e7160 0x55b6a8780480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:01.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.553+0000 7f7c32ffd700 1 -- 172.21.15.5:0/2379458883 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6a8633370 con 0x55b6a85fd4a0 2024-05-09T00:58:01.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.554+0000 7f7c32ffd700 1 --2- 172.21.15.5:0/2379458883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6a85fd4a0 0x55b6a8780a20 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f7c240a5b00 tx=0x7f7c240097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdb4292da7299324 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:58:01.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.555+0000 7f7c3a227700 1 -- 172.21.15.5:0/2379458883 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c240c3070 con 0x55b6a85fd4a0 2024-05-09T00:58:01.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.555+0000 7f7c3a227700 1 -- 172.21.15.5:0/2379458883 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c240b2230 con 0x55b6a85fd4a0 2024-05-09T00:58:01.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.555+0000 7f7c3a227700 1 -- 172.21.15.5:0/2379458883 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c240b28e0 con 0x55b6a85fd4a0 2024-05-09T00:58:01.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.557+0000 7f7c48c3db80 1 -- 172.21.15.5:0/2379458883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6a85fd4a0 msgr2=0x55b6a8780a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:01.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.557+0000 7f7c48c3db80 1 --2- 172.21.15.5:0/2379458883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6a85fd4a0 0x55b6a8780a20 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f7c240a5b00 tx=0x7f7c240097d0 comp rx=0 tx=0).stop 2024-05-09T00:58:01.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.553+0000 7f7c337fe700 1 --2- 172.21.15.5:0/600964587 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f7c1c0b2a90 0x7f7c1c0b4fa0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7c28009810 tx=0x7f7c28019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:58:01.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.561+0000 7f7c48c3db80 1 -- 172.21.15.5:0/2379458883 shutdown_connections 2024-05-09T00:58:01.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.561+0000 7f7c48c3db80 1 --2- 172.21.15.5:0/2379458883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b6a877a6a0 0x55b6a877aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:01.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.561+0000 7f7c48c3db80 1 --2- 172.21.15.5:0/2379458883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b6a86e7160 0x55b6a8780480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:01.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.562+0000 7f7c48c3db80 1 --2- 172.21.15.5:0/2379458883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6a85fd4a0 0x55b6a8780a20 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:01.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.562+0000 7f7c48c3db80 1 -- 172.21.15.5:0/2379458883 >> 172.21.15.5:0/2379458883 conn(0x55b6a865d6a0 msgr2=0x55b6a8775a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:58:01.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.562+0000 7f7c48c3db80 1 -- 172.21.15.5:0/2379458883 shutdown_connections 2024-05-09T00:58:01.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.562+0000 7f7c48c3db80 1 -- 172.21.15.5:0/2379458883 wait complete. 2024-05-09T00:58:01.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.564+0000 7f7c48c3db80 1 -- start start 2024-05-09T00:58:01.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.565+0000 7f7c48c3db80 1 Processor -- start 2024-05-09T00:58:01.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.566+0000 7f7c48c3db80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b6a85fd4a0 0x55b6a8784120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:01.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.566+0000 7f7c48c3db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6a86e7160 0x55b6a8784660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:01.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.567+0000 7f7c48c3db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b6a877a6a0 0x55b6a8784ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:01.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.567+0000 7f7c48c3db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b6a8518ba0 con 0x55b6a86e7160 2024-05-09T00:58:01.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.567+0000 7f7c48c3db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55b6a8657150 con 0x55b6a877a6a0 2024-05-09T00:58:01.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.568+0000 7f7c48c3db80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b6a877b490 con 0x55b6a85fd4a0 2024-05-09T00:58:01.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.568+0000 7f7c48c3db80 10 client.0 _ll_register_callbacks cb 0x55b6a850e6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:58:01.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.569+0000 7f7c33fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b6a877a6a0 0x55b6a8784ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:01.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.570+0000 7f7c337fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b6a85fd4a0 0x55b6a8784120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:01.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.570+0000 7f7c337fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b6a85fd4a0 0x55b6a8784120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:49016/0 (socket says 172.21.15.5:49016) 2024-05-09T00:58:01.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.570+0000 7f7c337fe700 1 -- 172.21.15.5:0/3446644767 learned_addr learned my addr 172.21.15.5:0/3446644767 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:58:01.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.572+0000 7f7c32ffd700 1 --2- 172.21.15.5:0/3446644767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6a86e7160 0x55b6a8784660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:01.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.573+0000 7f7c337fe700 1 -- 172.21.15.5:0/3446644767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b6a877a6a0 msgr2=0x55b6a8784ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:58:01.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.573+0000 7f7c337fe700 1 --2- 172.21.15.5:0/3446644767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b6a877a6a0 0x55b6a8784ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:01.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.573+0000 7f7c337fe700 1 -- 172.21.15.5:0/3446644767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6a86e7160 msgr2=0x55b6a8784660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:58:01.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.573+0000 7f7c337fe700 1 --2- 172.21.15.5:0/3446644767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6a86e7160 0x55b6a8784660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:01.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.573+0000 7f7c337fe700 1 -- 172.21.15.5:0/3446644767 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6a8518490 con 0x55b6a85fd4a0 2024-05-09T00:58:01.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.574+0000 7f7c337fe700 1 --2- 172.21.15.5:0/3446644767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b6a85fd4a0 0x55b6a8784120 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7c280098f0 tx=0x7f7c28009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77448c128d7fb76e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:58:01.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.576+0000 7f7c12ffd700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-09T00:58:01.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.576+0000 7f7c12ffd700 1 -- 172.21.15.5:0/3446644767 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c28032060 con 0x55b6a85fd4a0 2024-05-09T00:58:01.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.576+0000 7f7c12ffd700 1 -- 172.21.15.5:0/3446644767 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f7c280055b0 con 0x55b6a85fd4a0 2024-05-09T00:58:01.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.577+0000 7f7c48c3db80 1 -- 172.21.15.5:0/3446644767 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c240b2230 con 0x55b6a85fd4a0 2024-05-09T00:58:01.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.577+0000 7f7c12ffd700 1 -- 172.21.15.5:0/3446644767 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7c280046d0 con 0x55b6a85fd4a0 2024-05-09T00:58:01.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.579+0000 7f7c48c3db80 10 client.26122 fetch_fsmap learned FSMap version 31 2024-05-09T00:58:01.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.579+0000 7f7c48c3db80 1 -- 172.21.15.5:0/3446644767 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b6a8791aa0 con 0x55b6a85fd4a0 2024-05-09T00:58:01.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.582+0000 7f7c12ffd700 1 -- 172.21.15.5:0/3446644767 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c28004d20 con 0x55b6a85fd4a0 2024-05-09T00:58:01.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.582+0000 7f7c12ffd700 1 -- 172.21.15.5:0/3446644767 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f7c2800bc70 con 0x55b6a85fd4a0 2024-05-09T00:58:01.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.584+0000 7f7c48c3db80 10 client.26122 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:58:01.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.584+0000 7f7c48c3db80 10 client.26122 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T00:58:01.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.584+0000 7f7c48c3db80 20 client.26122 populate_metadata read hostname 'smithi005' 2024-05-09T00:58:01.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.584+0000 7f7c48c3db80 1 --2- 172.21.15.5:0/3446644767 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x55b6a8794030 0x55b6a8796450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:01.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.584+0000 7f7c48c3db80 4 client.26122 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:58:01.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.584+0000 7f7c48c3db80 1 -- 172.21.15.5:0/3446644767 --> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7c240b2230 con 0x55b6a8794030 2024-05-09T00:58:01.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.590+0000 7f7c32ffd700 1 --2- 172.21.15.5:0/3446644767 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x55b6a8794030 0x55b6a8796450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:01.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.600+0000 7f7c12ffd700 10 client.26122 ms_handle_connect on v2:172.21.15.157:6834/832898273 2024-05-09T00:58:01.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.600+0000 7f7c32ffd700 1 --2- 172.21.15.5:0/3446644767 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x55b6a8794030 0x55b6a8796450 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e7eca6082361d658 server_cookie=af9c186b23a09140 in_seq=0 out_seq=0 2024-05-09T00:58:01.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.602+0000 7f7c12ffd700 1 -- 172.21.15.5:0/3446644767 <== mds.3 v2:172.21.15.157:6834/832898273 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f7c240b2230 con 0x55b6a8794030 2024-05-09T00:58:01.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.602+0000 7f7c12ffd700 10 client.26122 handle_command_reply: tid=0 2024-05-09T00:58:01.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.602+0000 7f7c48c3db80 1 client.26122 shutdown 2024-05-09T00:58:01.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.605+0000 7f7c48c3db80 1 -- 172.21.15.5:0/3446644767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b6a85fd4a0 msgr2=0x55b6a8784120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:01.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.605+0000 7f7c48c3db80 1 --2- 172.21.15.5:0/3446644767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b6a85fd4a0 0x55b6a8784120 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7c280098f0 tx=0x7f7c28009260 comp rx=0 tx=0).stop 2024-05-09T00:58:01.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.605+0000 7f7c48c3db80 1 -- 172.21.15.5:0/3446644767 shutdown_connections 2024-05-09T00:58:01.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.605+0000 7f7c48c3db80 1 --2- 172.21.15.5:0/3446644767 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x55b6a8794030 0x55b6a8796450 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:01.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.605+0000 7f7c48c3db80 1 --2- 172.21.15.5:0/3446644767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b6a877a6a0 0x55b6a8784ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:01.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.605+0000 7f7c48c3db80 1 --2- 172.21.15.5:0/3446644767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b6a85fd4a0 0x55b6a8784120 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:01.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.605+0000 7f7c48c3db80 1 --2- 172.21.15.5:0/3446644767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6a86e7160 0x55b6a8784660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:01.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.605+0000 7f7c12ffd700 0 client.26122 ms_handle_reset on v2:172.21.15.157:6834/832898273 2024-05-09T00:58:01.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.605+0000 7f7c48c3db80 1 -- 172.21.15.5:0/3446644767 >> 172.21.15.5:0/3446644767 conn(0x55b6a865d6a0 msgr2=0x55b6a87754f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:58:01.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.609+0000 7f7c48c3db80 1 -- 172.21.15.5:0/3446644767 shutdown_connections 2024-05-09T00:58:01.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.611+0000 7f7c48c3db80 1 -- 172.21.15.5:0/3446644767 wait complete. 2024-05-09T00:58:01.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.611+0000 7f7c48c3db80 20 client.26122 trim_cache size 0 max 16384 2024-05-09T00:58:01.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.664+0000 7f7c48c3db80 1 Processor -- start 2024-05-09T00:58:01.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.664+0000 7f7c48c3db80 1 -- start start 2024-05-09T00:58:01.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.665+0000 7f7c48c3db80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b6a85fd4a0 0x55b6a877a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:01.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.665+0000 7f7c48c3db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6a86e7160 0x55b6a8775a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:01.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.665+0000 7f7c48c3db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b6a877a6a0 0x55b6a878ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:01.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.666+0000 7f7c48c3db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7c341c65d0 con 0x55b6a86e7160 2024-05-09T00:58:01.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.666+0000 7f7c48c3db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7c341c8110 con 0x55b6a877a6a0 2024-05-09T00:58:01.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.666+0000 7f7c48c3db80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7c341c6ca0 con 0x55b6a85fd4a0 2024-05-09T00:58:01.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.667+0000 7f7c32ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6a86e7160 0x55b6a8775a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:01.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.667+0000 7f7c32ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6a86e7160 0x55b6a8775a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38014/0 (socket says 172.21.15.5:38014) 2024-05-09T00:58:01.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.667+0000 7f7c32ffd700 1 -- 172.21.15.5:0/1556132621 learned_addr learned my addr 172.21.15.5:0/1556132621 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:58:01.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.668+0000 7f7c337fe700 1 --2- 172.21.15.5:0/1556132621 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b6a85fd4a0 0x55b6a877a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:01.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.669+0000 7f7c33fff700 1 --2- 172.21.15.5:0/1556132621 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b6a877a6a0 0x55b6a878ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:01.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.677+0000 7f7c32ffd700 1 -- 172.21.15.5:0/1556132621 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b6a85fd4a0 msgr2=0x55b6a877a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:58:01.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.677+0000 7f7c32ffd700 1 --2- 172.21.15.5:0/1556132621 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b6a85fd4a0 0x55b6a877a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:01.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.677+0000 7f7c32ffd700 1 -- 172.21.15.5:0/1556132621 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b6a877a6a0 msgr2=0x55b6a878ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:58:01.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.677+0000 7f7c32ffd700 1 --2- 172.21.15.5:0/1556132621 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b6a877a6a0 0x55b6a878ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:01.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.677+0000 7f7c32ffd700 1 -- 172.21.15.5:0/1556132621 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6a877b610 con 0x55b6a86e7160 2024-05-09T00:58:01.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.682+0000 7f7c32ffd700 1 --2- 172.21.15.5:0/1556132621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6a86e7160 0x55b6a8775a50 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f7c24009780 tx=0x7f7c240a5a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f28b9f058ed393a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:58:01.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.689+0000 7f7c197fa700 1 -- 172.21.15.5:0/1556132621 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c240c3040 con 0x55b6a86e7160 2024-05-09T00:58:01.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.690+0000 7f7c197fa700 1 -- 172.21.15.5:0/1556132621 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c240b28e0 con 0x55b6a86e7160 2024-05-09T00:58:01.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.691+0000 7f7c48c3db80 1 -- 172.21.15.5:0/1556132621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6a86e7160 msgr2=0x55b6a8775a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:01.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.691+0000 7f7c48c3db80 1 --2- 172.21.15.5:0/1556132621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6a86e7160 0x55b6a8775a50 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f7c24009780 tx=0x7f7c240a5a70 comp rx=0 tx=0).stop 2024-05-09T00:58:01.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.692+0000 7f7c48c3db80 1 -- 172.21.15.5:0/1556132621 shutdown_connections 2024-05-09T00:58:01.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.692+0000 7f7c48c3db80 1 --2- 172.21.15.5:0/1556132621 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b6a877a6a0 0x55b6a878ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:01.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.692+0000 7f7c48c3db80 1 --2- 172.21.15.5:0/1556132621 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b6a85fd4a0 0x55b6a877a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:01.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.692+0000 7f7c48c3db80 1 --2- 172.21.15.5:0/1556132621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6a86e7160 0x55b6a8775a50 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:01.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.692+0000 7f7c48c3db80 1 -- 172.21.15.5:0/1556132621 >> 172.21.15.5:0/1556132621 conn(0x55b6a865d6a0 msgr2=0x55b6a87840e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:58:01.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.700+0000 7f7c48c3db80 1 -- 172.21.15.5:0/1556132621 shutdown_connections 2024-05-09T00:58:01.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.704+0000 7f7c48c3db80 1 -- 172.21.15.5:0/1556132621 wait complete. 2024-05-09T00:58:01.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.704+0000 7f7c48c3db80 1 -- start start 2024-05-09T00:58:01.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.705+0000 7f7c48c3db80 1 Processor -- start 2024-05-09T00:58:01.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.706+0000 7f7c48c3db80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b6a85fd4a0 0x55b6a878c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:01.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.706+0000 7f7c48c3db80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b6a86e7160 0x55b6a879b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:01.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.706+0000 7f7c48c3db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6a877a6a0 0x55b6a879b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:01.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.706+0000 7f7c48c3db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7c341c7a40 con 0x55b6a877a6a0 2024-05-09T00:58:01.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.706+0000 7f7c48c3db80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7c341c7370 con 0x55b6a85fd4a0 2024-05-09T00:58:01.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.706+0000 7f7c48c3db80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7c341c4570 con 0x55b6a86e7160 2024-05-09T00:58:01.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.706+0000 7f7c48c3db80 10 client.0 _ll_register_callbacks cb 0x55b6a850e6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:58:01.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.707+0000 7f7c33fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6a877a6a0 0x55b6a879b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:01.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.707+0000 7f7c33fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6a877a6a0 0x55b6a879b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38022/0 (socket says 172.21.15.5:38022) 2024-05-09T00:58:01.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.707+0000 7f7c33fff700 1 -- 172.21.15.5:0/385836840 learned_addr learned my addr 172.21.15.5:0/385836840 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:58:01.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.709+0000 7f7c32ffd700 1 --2- 172.21.15.5:0/385836840 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b6a86e7160 0x55b6a879b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:01.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.710+0000 7f7c33fff700 1 -- 172.21.15.5:0/385836840 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b6a86e7160 msgr2=0x55b6a879b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:58:01.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.710+0000 7f7c33fff700 1 --2- 172.21.15.5:0/385836840 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b6a86e7160 0x55b6a879b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:01.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.710+0000 7f7c33fff700 1 -- 172.21.15.5:0/385836840 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b6a85fd4a0 msgr2=0x55b6a878c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:58:01.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.710+0000 7f7c33fff700 1 --2- 172.21.15.5:0/385836840 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b6a85fd4a0 0x55b6a878c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:01.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.710+0000 7f7c33fff700 1 -- 172.21.15.5:0/385836840 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6a879bd20 con 0x55b6a877a6a0 2024-05-09T00:58:01.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.713+0000 7f7c33fff700 1 --2- 172.21.15.5:0/385836840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6a877a6a0 0x55b6a879b680 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c007b30 tx=0x7f7c2c007e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cea15fb44ea7cec server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:58:01.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.713+0000 7f7c12ffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:58:01.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.714+0000 7f7c12ffd700 1 -- 172.21.15.5:0/385836840 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c01c020 con 0x55b6a877a6a0 2024-05-09T00:58:01.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.715+0000 7f7c12ffd700 1 -- 172.21.15.5:0/385836840 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f7c2c013910 con 0x55b6a877a6a0 2024-05-09T00:58:01.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.715+0000 7f7c48c3db80 1 -- 172.21.15.5:0/385836840 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c240b28e0 con 0x55b6a877a6a0 2024-05-09T00:58:01.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.717+0000 7f7c12ffd700 1 -- 172.21.15.5:0/385836840 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7c2c027050 con 0x55b6a877a6a0 2024-05-09T00:58:01.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.718+0000 7f7c48c3db80 10 client.16722 fetch_fsmap learned FSMap version 31 2024-05-09T00:58:01.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.718+0000 7f7c48c3db80 1 -- 172.21.15.5:0/385836840 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b6a87fe590 con 0x55b6a877a6a0 2024-05-09T00:58:01.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.723+0000 7f7c12ffd700 1 -- 172.21.15.5:0/385836840 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c240b28e0 con 0x55b6a877a6a0 2024-05-09T00:58:01.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.724+0000 7f7c12ffd700 1 -- 172.21.15.5:0/385836840 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f7c2c0173b0 con 0x55b6a877a6a0 2024-05-09T00:58:01.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.728+0000 7f7c48c3db80 10 client.16722 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:58:01.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.728+0000 7f7c48c3db80 10 client.16722 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T00:58:01.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.728+0000 7f7c48c3db80 20 client.16722 populate_metadata read hostname 'smithi005' 2024-05-09T00:58:01.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.728+0000 7f7c48c3db80 1 --2- 172.21.15.5:0/385836840 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x55b6a87fec90 0x55b6a88010b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:01.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.728+0000 7f7c48c3db80 4 client.16722 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:58:01.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.728+0000 7f7c48c3db80 1 -- 172.21.15.5:0/385836840 --> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b6a86aa9d0 con 0x55b6a87fec90 2024-05-09T00:58:01.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.734+0000 7f7c337fe700 1 --2- 172.21.15.5:0/385836840 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x55b6a87fec90 0x55b6a88010b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:01.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.747+0000 7f7c12ffd700 10 client.16722 ms_handle_connect on v2:172.21.15.157:6834/832898273 2024-05-09T00:58:01.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.747+0000 7f7c337fe700 1 --2- 172.21.15.5:0/385836840 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x55b6a87fec90 0x55b6a88010b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ecb031df94778e06 server_cookie=97c55ab4dc793e8b in_seq=0 out_seq=0 2024-05-09T00:58:01.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.749+0000 7f7c12ffd700 1 -- 172.21.15.5:0/385836840 <== mds.3 v2:172.21.15.157:6834/832898273 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b6a86aa9d0 con 0x55b6a87fec90 2024-05-09T00:58:01.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.749+0000 7f7c12ffd700 10 client.16722 handle_command_reply: tid=0 2024-05-09T00:58:01.752 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:58:01 smithi005 ceph-mon[30365]: pgmap v721: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 933 MiB/s wr, 2.65k op/s 2024-05-09T00:58:01.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.753+0000 7f7c48c3db80 1 client.16722 shutdown 2024-05-09T00:58:01.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.763+0000 7f7c48c3db80 1 -- 172.21.15.5:0/385836840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6a877a6a0 msgr2=0x55b6a879b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:01.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.763+0000 7f7c48c3db80 1 --2- 172.21.15.5:0/385836840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6a877a6a0 0x55b6a879b680 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c007b30 tx=0x7f7c2c007e40 comp rx=0 tx=0).stop 2024-05-09T00:58:01.770 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-09T00:58:01.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.765+0000 7f7c48c3db80 1 -- 172.21.15.5:0/385836840 shutdown_connections 2024-05-09T00:58:01.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.765+0000 7f7c48c3db80 1 --2- 172.21.15.5:0/385836840 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x55b6a87fec90 0x55b6a88010b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:01.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.765+0000 7f7c48c3db80 1 --2- 172.21.15.5:0/385836840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6a877a6a0 0x55b6a879b680 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:01.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.765+0000 7f7c48c3db80 1 --2- 172.21.15.5:0/385836840 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b6a85fd4a0 0x55b6a878c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:01.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.765+0000 7f7c48c3db80 1 --2- 172.21.15.5:0/385836840 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b6a86e7160 0x55b6a879b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:01.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.765+0000 7f7c48c3db80 1 -- 172.21.15.5:0/385836840 >> 172.21.15.5:0/385836840 conn(0x55b6a865d6a0 msgr2=0x55b6a8784b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:58:01.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.765+0000 7f7c12ffd700 0 client.16722 ms_handle_reset on v2:172.21.15.157:6834/832898273 2024-05-09T00:58:01.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.766+0000 7f7c48c3db80 1 -- 172.21.15.5:0/385836840 shutdown_connections 2024-05-09T00:58:01.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.766+0000 7f7c48c3db80 1 -- 172.21.15.5:0/385836840 wait complete. 2024-05-09T00:58:01.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.767+0000 7f7c48c3db80 20 client.16722 trim_cache size 0 max 16384 2024-05-09T00:58:01.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.768+0000 7f7c1a7fc700 1 -- 172.21.15.5:0/600964587 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f7c1c0b2a90 msgr2=0x7f7c1c0b4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:01.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.768+0000 7f7c1a7fc700 1 --2- 172.21.15.5:0/600964587 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f7c1c0b2a90 0x7f7c1c0b4fa0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7c28009810 tx=0x7f7c28019040 comp rx=0 tx=0).stop 2024-05-09T00:58:01.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.769+0000 7f7c1a7fc700 1 -- 172.21.15.5:0/600964587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c3410b280 msgr2=0x7f7c34114ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:01.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.769+0000 7f7c1a7fc700 1 --2- 172.21.15.5:0/600964587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c3410b280 0x7f7c34114ec0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7c2400e9c0 tx=0x7f7c2400ed80 comp rx=0 tx=0).stop 2024-05-09T00:58:01.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.769+0000 7f7c1a7fc700 1 -- 172.21.15.5:0/600964587 shutdown_connections 2024-05-09T00:58:01.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.769+0000 7f7c1a7fc700 1 --2- 172.21.15.5:0/600964587 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f7c1c0b2a90 0x7f7c1c0b4fa0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:01.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.769+0000 7f7c1a7fc700 1 --2- 172.21.15.5:0/600964587 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c3411d930 0x7f7c341154a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:01.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.769+0000 7f7c1a7fc700 1 --2- 172.21.15.5:0/600964587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c3410a930 0x7f7c3411bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:01.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.769+0000 7f7c1a7fc700 1 --2- 172.21.15.5:0/600964587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c3410b280 0x7f7c34114ec0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:01.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.769+0000 7f7c1a7fc700 1 -- 172.21.15.5:0/600964587 >> 172.21.15.5:0/600964587 conn(0x7f7c3406f4d0 msgr2=0x7f7c34072790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:58:01.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.774+0000 7f7c1a7fc700 1 -- 172.21.15.5:0/600964587 shutdown_connections 2024-05-09T00:58:01.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:01.774+0000 7f7c1a7fc700 1 -- 172.21.15.5:0/600964587 wait complete. 2024-05-09T00:58:01.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:58:01 smithi131 ceph-mon[34547]: pgmap v721: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 933 MiB/s wr, 2.65k op/s 2024-05-09T00:58:01.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:58:01 smithi157 ceph-mon[34294]: pgmap v721: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 933 MiB/s wr, 2.65k op/s 2024-05-09T00:58:03.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:58:03 smithi005 ceph-mon[30365]: pgmap v722: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 830 MiB/s wr, 1.86k op/s 2024-05-09T00:58:03.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:58:03 smithi131 ceph-mon[34547]: pgmap v722: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 830 MiB/s wr, 1.86k op/s 2024-05-09T00:58:03.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:58:03 smithi157 ceph-mon[34294]: pgmap v722: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 830 MiB/s wr, 1.86k op/s 2024-05-09T00:58:05.423 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T00:58:05.593 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:58:05 smithi005 ceph-mon[30365]: pgmap v723: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 871 MiB/s wr, 1.80k op/s 2024-05-09T00:58:05.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:58:05 smithi131 ceph-mon[34547]: pgmap v723: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 871 MiB/s wr, 1.80k op/s 2024-05-09T00:58:05.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:58:05 smithi157 ceph-mon[34294]: pgmap v723: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 871 MiB/s wr, 1.80k op/s 2024-05-09T00:58:06.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:06.158+0000 7facd7fff640 1 -- 172.21.15.5:0/3924306597 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x561fd84d93d0 con 0x561fd81571c0 2024-05-09T00:58:06.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:06.158+0000 7facd7fff640 10 client.16704 handle_command_reply: tid=0 2024-05-09T00:58:06.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:06.159+0000 7fad02187740 1 client.16704 shutdown 2024-05-09T00:58:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:06.165+0000 7fad02187740 1 -- 172.21.15.5:0/3924306597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fd81c31b0 msgr2=0x561fd8576e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:06.165+0000 7fad02187740 1 --2- 172.21.15.5:0/3924306597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fd81c31b0 0x561fd8576e90 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7facec000f80 tx=0x7facec003ec0 comp rx=0 tx=0).stop 2024-05-09T00:58:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:06.165+0000 7fad02187740 1 -- 172.21.15.5:0/3924306597 shutdown_connections 2024-05-09T00:58:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:06.165+0000 7fad02187740 1 --2- 172.21.15.5:0/3924306597 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x561fd81571c0 0x561fd85e8c10 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:06.165+0000 7fad02187740 1 --2- 172.21.15.5:0/3924306597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561fd81c31b0 0x561fd8576e90 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:06.165+0000 7fad02187740 1 --2- 172.21.15.5:0/3924306597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561fd81a24d0 0x561fd8575f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:06.165+0000 7fad02187740 1 --2- 172.21.15.5:0/3924306597 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561fd819a2f0 0x561fd8575a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:06.165+0000 7fad02187740 1 -- 172.21.15.5:0/3924306597 >> 172.21.15.5:0/3924306597 conn(0x561fd81b7300 msgr2=0x561fd8565e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:58:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:06.165+0000 7facd7fff640 0 client.16704 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:58:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:06.166+0000 7fad02187740 1 -- 172.21.15.5:0/3924306597 shutdown_connections 2024-05-09T00:58:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:06.166+0000 7fad02187740 1 -- 172.21.15.5:0/3924306597 wait complete. 2024-05-09T00:58:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:06.166+0000 7fad02187740 20 client.16704 trim_cache size 0 max 16384 2024-05-09T00:58:06.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-09T00:58:06.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-09T00:58:06.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-09T00:58:06.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:06.170+0000 7facf9f9d640 1 -- 172.21.15.5:0/587926727 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7face40b29b0 msgr2=0x7face40b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:06.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:06.170+0000 7facf9f9d640 1 --2- 172.21.15.5:0/587926727 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7face40b29b0 0x7face40b4ec0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7facec009420 tx=0x7facec009880 comp rx=0 tx=0).stop 2024-05-09T00:58:06.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:06.170+0000 7facf9f9d640 1 -- 172.21.15.5:0/587926727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7facf4104560 msgr2=0x7facf4113240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:06.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:06.170+0000 7facf9f9d640 1 --2- 172.21.15.5:0/587926727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7facf4104560 0x7facf4113240 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7face8009fd0 tx=0x7face800b870 comp rx=0 tx=0).stop 2024-05-09T00:58:06.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:06.173+0000 7facf9f9d640 1 -- 172.21.15.5:0/587926727 shutdown_connections 2024-05-09T00:58:06.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:06.173+0000 7facf9f9d640 1 --2- 172.21.15.5:0/587926727 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7face40b29b0 0x7face40b4ec0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:06.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:06.173+0000 7facf9f9d640 1 --2- 172.21.15.5:0/587926727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facf4109820 0x7facf4118700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:06.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:06.173+0000 7facf9f9d640 1 --2- 172.21.15.5:0/587926727 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7facf4106ec0 0x7facf4113780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:06.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:06.173+0000 7facf9f9d640 1 --2- 172.21.15.5:0/587926727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7facf4104560 0x7facf4113240 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:06.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:06.173+0000 7facf9f9d640 1 -- 172.21.15.5:0/587926727 >> 172.21.15.5:0/587926727 conn(0x7facf40fdb60 msgr2=0x7facf4105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:58:06.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:06.174+0000 7facf9f9d640 1 -- 172.21.15.5:0/587926727 shutdown_connections 2024-05-09T00:58:06.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:06.174+0000 7facf9f9d640 1 -- 172.21.15.5:0/587926727 wait complete. 2024-05-09T00:58:06.368 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:58:07.795 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:58:07 smithi005 ceph-mon[30365]: pgmap v724: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 965 MiB/s wr, 2.19k op/s 2024-05-09T00:58:07.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:58:07 smithi131 ceph-mon[34547]: pgmap v724: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 965 MiB/s wr, 2.19k op/s 2024-05-09T00:58:07.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:58:07 smithi157 ceph-mon[34294]: pgmap v724: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 965 MiB/s wr, 2.19k op/s 2024-05-09T00:58:09.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:58:09 smithi131 ceph-mon[34547]: pgmap v725: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 867 MiB/s wr, 2.17k op/s 2024-05-09T00:58:09.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:58:09 smithi157 ceph-mon[34294]: pgmap v725: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 867 MiB/s wr, 2.17k op/s 2024-05-09T00:58:09.993 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:58:09 smithi005 ceph-mon[30365]: pgmap v725: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 867 MiB/s wr, 2.17k op/s 2024-05-09T00:58:11.613 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:58:11 smithi005 ceph-mon[30365]: pgmap v726: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 934 MiB/s wr, 2.07k op/s 2024-05-09T00:58:11.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.762+0000 7f55fdb21700 1 -- 172.21.15.5:0/2639289510 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55f810b280 msgr2=0x7f55f811d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:11.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.762+0000 7f55fdb21700 1 --2- 172.21.15.5:0/2639289510 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55f810b280 0x7f55f811d3f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f55f000b0a0 tx=0x7f55f000b3b0 comp rx=0 tx=0).stop 2024-05-09T00:58:11.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.764+0000 7f55fdb21700 1 -- 172.21.15.5:0/2639289510 shutdown_connections 2024-05-09T00:58:11.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.764+0000 7f55fdb21700 1 --2- 172.21.15.5:0/2639289510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55f811d930 0x7f55f811fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:11.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.764+0000 7f55fdb21700 1 --2- 172.21.15.5:0/2639289510 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55f810a930 0x7f55f810ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:11.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.764+0000 7f55fdb21700 1 --2- 172.21.15.5:0/2639289510 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55f810b280 0x7f55f811d3f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:11.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.764+0000 7f55fdb21700 1 -- 172.21.15.5:0/2639289510 >> 172.21.15.5:0/2639289510 conn(0x7f55f806f4d0 msgr2=0x7f55f806f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:58:11.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.771+0000 7f55fdb21700 1 -- 172.21.15.5:0/2639289510 shutdown_connections 2024-05-09T00:58:11.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.773+0000 7f55fdb21700 1 -- 172.21.15.5:0/2639289510 wait complete. 2024-05-09T00:58:11.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.775+0000 7f55fdb21700 1 Processor -- start 2024-05-09T00:58:11.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.775+0000 7f55fdb21700 1 -- start start 2024-05-09T00:58:11.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.776+0000 7f55fdb21700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55f810a930 0x7f55f811bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:11.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.776+0000 7f55fdb21700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55f810b280 0x7f55f8114e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:11.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.776+0000 7f55fdb21700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55f811d930 0x7f55f8115440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:11.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.776+0000 7f55fdb21700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f55f8123340 con 0x7f55f810b280 2024-05-09T00:58:11.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.776+0000 7f55fdb21700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f55f81231c0 con 0x7f55f811d930 2024-05-09T00:58:11.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.776+0000 7f55fdb21700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f55f81234c0 con 0x7f55f810a930 2024-05-09T00:58:11.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.776+0000 7f55f7fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55f811d930 0x7f55f8115440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:11.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.776+0000 7f55f7fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55f811d930 0x7f55f8115440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:42516/0 (socket says 172.21.15.5:42516) 2024-05-09T00:58:11.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.776+0000 7f55f7fff700 1 -- 172.21.15.5:0/524516676 learned_addr learned my addr 172.21.15.5:0/524516676 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:58:11.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.777+0000 7f55f6ffd700 1 --2- 172.21.15.5:0/524516676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55f810b280 0x7f55f8114e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:11.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.777+0000 7f55f6ffd700 1 -- 172.21.15.5:0/524516676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55f810a930 msgr2=0x7f55f811bd50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T00:58:11.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.777+0000 7f55f6ffd700 1 --2- 172.21.15.5:0/524516676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55f810a930 0x7f55f811bd50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:11.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.777+0000 7f55f6ffd700 1 -- 172.21.15.5:0/524516676 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55f811d930 msgr2=0x7f55f8115440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:11.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.777+0000 7f55f6ffd700 1 --2- 172.21.15.5:0/524516676 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55f811d930 0x7f55f8115440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:11.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.777+0000 7f55f6ffd700 1 -- 172.21.15.5:0/524516676 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55f8115ca0 con 0x7f55f810b280 2024-05-09T00:58:11.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.778+0000 7f55f6ffd700 1 --2- 172.21.15.5:0/524516676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55f810b280 0x7f55f8114e80 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f55f000b070 tx=0x7f55f0004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:58:11.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.778+0000 7f55f4ff9700 1 -- 172.21.15.5:0/524516676 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f0021070 con 0x7f55f810b280 2024-05-09T00:58:11.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.778+0000 7f55fdb21700 1 -- 172.21.15.5:0/524516676 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55f8119940 con 0x7f55f810b280 2024-05-09T00:58:11.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.779+0000 7f55fdb21700 1 -- 172.21.15.5:0/524516676 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55f8119e80 con 0x7f55f810b280 2024-05-09T00:58:11.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.779+0000 7f55f4ff9700 1 -- 172.21.15.5:0/524516676 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55f0007c40 con 0x7f55f810b280 2024-05-09T00:58:11.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.779+0000 7f55f4ff9700 1 -- 172.21.15.5:0/524516676 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f00047e0 con 0x7f55f810b280 2024-05-09T00:58:11.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.785+0000 7f55f4ff9700 1 -- 172.21.15.5:0/524516676 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f55f001d450 con 0x7f55f810b280 2024-05-09T00:58:11.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.787+0000 7f55f4ff9700 1 --2- 172.21.15.5:0/524516676 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f55e00b2a90 0x7f55e00b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:11.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.787+0000 7f55f4ff9700 1 -- 172.21.15.5:0/524516676 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f55f00e3030 con 0x7f55f810b280 2024-05-09T00:58:11.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.791+0000 7f560c537b80 1 Processor -- start 2024-05-09T00:58:11.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.792+0000 7f560c537b80 1 -- start start 2024-05-09T00:58:11.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.792+0000 7f560c537b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609bac38160 0x5609bacd14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:11.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.792+0000 7f560c537b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609bab4e4a0 0x5609bacd1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:11.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.792+0000 7f560c537b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609baccb6a0 0x5609baccbad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:11.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.792+0000 7f560c537b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5609babedff0 con 0x5609bab4e4a0 2024-05-09T00:58:11.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.793+0000 7f560c537b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5609bab8c7f0 con 0x5609bac38160 2024-05-09T00:58:11.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.793+0000 7f560c537b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5609babe9fb0 con 0x5609baccb6a0 2024-05-09T00:58:11.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.793+0000 7f55f6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609bab4e4a0 0x5609bacd1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:11.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.793+0000 7f55f6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609bab4e4a0 0x5609bacd1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37756/0 (socket says 172.21.15.5:37756) 2024-05-09T00:58:11.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.793+0000 7f55f6ffd700 1 -- 172.21.15.5:0/4237408010 learned_addr learned my addr 172.21.15.5:0/4237408010 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:58:11.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.794+0000 7f55f7fff700 1 --2- 172.21.15.5:0/4237408010 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609baccb6a0 0x5609baccbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:11.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.794+0000 7f55f6ffd700 1 -- 172.21.15.5:0/4237408010 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609baccb6a0 msgr2=0x5609baccbad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:58:11.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.794+0000 7f55f6ffd700 1 --2- 172.21.15.5:0/4237408010 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609baccb6a0 0x5609baccbad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:11.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.794+0000 7f55f6ffd700 1 -- 172.21.15.5:0/4237408010 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609bac38160 msgr2=0x5609bacd14f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:58:11.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.794+0000 7f55f6ffd700 1 --2- 172.21.15.5:0/4237408010 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609bac38160 0x5609bacd14f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:11.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.795+0000 7f55f6ffd700 1 -- 172.21.15.5:0/4237408010 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609bab84370 con 0x5609bab4e4a0 2024-05-09T00:58:11.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.795+0000 7f55f7fff700 1 --2- 172.21.15.5:0/4237408010 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609baccb6a0 0x5609baccbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:58:11.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.798+0000 7f55f6ffd700 1 --2- 172.21.15.5:0/4237408010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609bab4e4a0 0x5609bacd1a30 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f55f00aa350 tx=0x7f55f000b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c299bbd2ebac4f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:58:11.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.798+0000 7f55ddffb700 1 -- 172.21.15.5:0/4237408010 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f002a030 con 0x5609bab4e4a0 2024-05-09T00:58:11.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.799+0000 7f55ddffb700 1 -- 172.21.15.5:0/4237408010 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55f00040d0 con 0x5609bab4e4a0 2024-05-09T00:58:11.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.799+0000 7f55ddffb700 1 -- 172.21.15.5:0/4237408010 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f00aab10 con 0x5609bab4e4a0 2024-05-09T00:58:11.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.802+0000 7f560c537b80 1 -- 172.21.15.5:0/4237408010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609bab4e4a0 msgr2=0x5609bacd1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:11.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.802+0000 7f560c537b80 1 --2- 172.21.15.5:0/4237408010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609bab4e4a0 0x5609bacd1a30 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f55f00aa350 tx=0x7f55f000b3b0 comp rx=0 tx=0).stop 2024-05-09T00:58:11.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.802+0000 7f560c537b80 1 -- 172.21.15.5:0/4237408010 shutdown_connections 2024-05-09T00:58:11.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.802+0000 7f560c537b80 1 --2- 172.21.15.5:0/4237408010 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609baccb6a0 0x5609baccbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:11.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.802+0000 7f560c537b80 1 --2- 172.21.15.5:0/4237408010 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609bac38160 0x5609bacd14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:11.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.802+0000 7f560c537b80 1 --2- 172.21.15.5:0/4237408010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609bab4e4a0 0x5609bacd1a30 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:11.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.802+0000 7f560c537b80 1 -- 172.21.15.5:0/4237408010 >> 172.21.15.5:0/4237408010 conn(0x5609babae6a0 msgr2=0x5609bacc6400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:58:11.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.803+0000 7f55f77fe700 1 --2- 172.21.15.5:0/524516676 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f55e00b2a90 0x7f55e00b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:11.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.806+0000 7f560c537b80 1 -- 172.21.15.5:0/4237408010 shutdown_connections 2024-05-09T00:58:11.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.807+0000 7f560c537b80 1 -- 172.21.15.5:0/4237408010 wait complete. 2024-05-09T00:58:11.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.808+0000 7f560c537b80 1 -- start start 2024-05-09T00:58:11.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.808+0000 7f560c537b80 1 Processor -- start 2024-05-09T00:58:11.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.809+0000 7f560c537b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609bab4e4a0 0x5609bacdacb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:11.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.809+0000 7f560c537b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609bac38160 0x5609bacdb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:11.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.809+0000 7f560c537b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609baccb6a0 0x5609bacd50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:11.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.809+0000 7f560c537b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5609baa69ba0 con 0x5609baccb6a0 2024-05-09T00:58:11.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.809+0000 7f560c537b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5609baba8150 con 0x5609bac38160 2024-05-09T00:58:11.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.809+0000 7f560c537b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5609baccc360 con 0x5609bab4e4a0 2024-05-09T00:58:11.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.809+0000 7f560c537b80 10 client.0 _ll_register_callbacks cb 0x5609baa5f6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:58:11.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.810+0000 7f55f6ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609bac38160 0x5609bacdb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:11.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.810+0000 7f55f6ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609bac38160 0x5609bacdb1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:42528/0 (socket says 172.21.15.5:42528) 2024-05-09T00:58:11.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.810+0000 7f55f6ffd700 1 -- 172.21.15.5:0/1241630859 learned_addr learned my addr 172.21.15.5:0/1241630859 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:58:11.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.810+0000 7f55f6ffd700 1 -- 172.21.15.5:0/1241630859 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609bab4e4a0 msgr2=0x5609bacdacb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:58:11.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.810+0000 7f55f6ffd700 1 --2- 172.21.15.5:0/1241630859 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609bab4e4a0 0x5609bacdacb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:11.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.810+0000 7f55f6ffd700 1 -- 172.21.15.5:0/1241630859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609baccb6a0 msgr2=0x5609bacd50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:58:11.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.810+0000 7f55f6ffd700 1 --2- 172.21.15.5:0/1241630859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609baccb6a0 0x5609bacd50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:11.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.810+0000 7f55f6ffd700 1 -- 172.21.15.5:0/1241630859 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609baa69490 con 0x5609bac38160 2024-05-09T00:58:11.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.810+0000 7f55f77fe700 1 --2- 172.21.15.5:0/524516676 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f55e00b2a90 0x7f55e00b4fa0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f55e8005fd0 tx=0x7f55e8005b90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:58:11.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.810+0000 7f55f6ffd700 1 --2- 172.21.15.5:0/1241630859 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609bac38160 0x5609bacdb1f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f55f0009fd0 tx=0x7f55f00b3420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f30d2edc80ce2caf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:58:11.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.811+0000 7f55d6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-09T00:58:11.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.815+0000 7f55d6ffd700 1 -- 172.21.15.5:0/1241630859 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f002a040 con 0x5609bac38160 2024-05-09T00:58:11.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.815+0000 7f55d6ffd700 1 -- 172.21.15.5:0/1241630859 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f55f00a6c40 con 0x5609bac38160 2024-05-09T00:58:11.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.816+0000 7f560c537b80 1 -- 172.21.15.5:0/1241630859 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55f00040d0 con 0x5609bac38160 2024-05-09T00:58:11.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.816+0000 7f55d6ffd700 1 -- 172.21.15.5:0/1241630859 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f55f00aaac0 con 0x5609bac38160 2024-05-09T00:58:11.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.817+0000 7f560c537b80 10 client.26147 fetch_fsmap learned FSMap version 31 2024-05-09T00:58:11.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.817+0000 7f560c537b80 1 -- 172.21.15.5:0/1241630859 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5609bace2840 con 0x5609bac38160 2024-05-09T00:58:11.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.820+0000 7f55d6ffd700 1 -- 172.21.15.5:0/1241630859 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55f00c4070 con 0x5609bac38160 2024-05-09T00:58:11.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.820+0000 7f55d6ffd700 1 -- 172.21.15.5:0/1241630859 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f55f0011070 con 0x5609bac38160 2024-05-09T00:58:11.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.821+0000 7f560c537b80 10 client.26147 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:58:11.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.821+0000 7f560c537b80 10 client.26147 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-05-09T00:58:11.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.821+0000 7f560c537b80 20 client.26147 populate_metadata read hostname 'smithi005' 2024-05-09T00:58:11.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.821+0000 7f560c537b80 1 --2- 172.21.15.5:0/1241630859 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x5609bace4d10 0x5609bace7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:11.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.821+0000 7f560c537b80 4 client.26147 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:58:11.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.821+0000 7f560c537b80 1 -- 172.21.15.5:0/1241630859 --> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f55f00040d0 con 0x5609bace4d10 2024-05-09T00:58:11.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.825+0000 7f55f77fe700 1 --2- 172.21.15.5:0/1241630859 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x5609bace4d10 0x5609bace7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:11.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.829+0000 7f55f77fe700 1 --2- 172.21.15.5:0/1241630859 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x5609bace4d10 0x5609bace7130 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b55c8847467ec264 server_cookie=f643ae57d1cb3c2e in_seq=0 out_seq=0 2024-05-09T00:58:11.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.829+0000 7f55d6ffd700 10 client.26147 ms_handle_connect on v2:172.21.15.131:6838/2748669631 2024-05-09T00:58:11.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.830+0000 7f55d6ffd700 1 -- 172.21.15.5:0/1241630859 <== mds.2 v2:172.21.15.131:6838/2748669631 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f55f00040d0 con 0x5609bace4d10 2024-05-09T00:58:11.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.831+0000 7f55d6ffd700 10 client.26147 handle_command_reply: tid=0 2024-05-09T00:58:11.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.831+0000 7f560c537b80 1 client.26147 shutdown 2024-05-09T00:58:11.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.839+0000 7f560c537b80 1 -- 172.21.15.5:0/1241630859 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609bac38160 msgr2=0x5609bacdb1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:11.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.839+0000 7f560c537b80 1 --2- 172.21.15.5:0/1241630859 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609bac38160 0x5609bacdb1f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f55f0009fd0 tx=0x7f55f00b3420 comp rx=0 tx=0).stop 2024-05-09T00:58:11.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.840+0000 7f560c537b80 1 -- 172.21.15.5:0/1241630859 shutdown_connections 2024-05-09T00:58:11.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.840+0000 7f560c537b80 1 --2- 172.21.15.5:0/1241630859 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x5609bace4d10 0x5609bace7130 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:11.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.840+0000 7f560c537b80 1 --2- 172.21.15.5:0/1241630859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609baccb6a0 0x5609bacd50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:11.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.840+0000 7f560c537b80 1 --2- 172.21.15.5:0/1241630859 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609bab4e4a0 0x5609bacdacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:11.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.840+0000 7f560c537b80 1 --2- 172.21.15.5:0/1241630859 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609bac38160 0x5609bacdb1f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:11.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.840+0000 7f560c537b80 1 -- 172.21.15.5:0/1241630859 >> 172.21.15.5:0/1241630859 conn(0x5609babae6a0 msgr2=0x5609bacc61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:58:11.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.841+0000 7f55d6ffd700 0 client.26147 ms_handle_reset on v2:172.21.15.131:6838/2748669631 2024-05-09T00:58:11.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.841+0000 7f560c537b80 1 -- 172.21.15.5:0/1241630859 shutdown_connections 2024-05-09T00:58:11.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.841+0000 7f560c537b80 1 -- 172.21.15.5:0/1241630859 wait complete. 2024-05-09T00:58:11.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.841+0000 7f560c537b80 20 client.26147 trim_cache size 0 max 16384 2024-05-09T00:58:11.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:58:11 smithi131 ceph-mon[34547]: pgmap v726: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 934 MiB/s wr, 2.07k op/s 2024-05-09T00:58:11.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.878+0000 7f560c537b80 1 Processor -- start 2024-05-09T00:58:11.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.878+0000 7f560c537b80 1 -- start start 2024-05-09T00:58:11.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.878+0000 7f560c537b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609bab4e4a0 0x5609bacd39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:11.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.878+0000 7f560c537b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609bac38160 0x5609bacd3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:11.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.878+0000 7f560c537b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609baccb6a0 0x5609bacec870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:11.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.878+0000 7f560c537b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f55e400d580 con 0x5609bac38160 2024-05-09T00:58:11.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.878+0000 7f560c537b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f55e400f0c0 con 0x5609bab4e4a0 2024-05-09T00:58:11.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.878+0000 7f560c537b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f55e400dc50 con 0x5609baccb6a0 2024-05-09T00:58:11.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.879+0000 7f55f77fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609bab4e4a0 0x5609bacd39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:11.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.879+0000 7f55f7fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609baccb6a0 0x5609bacec870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:11.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.879+0000 7f55f77fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609bab4e4a0 0x5609bacd39a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:42538/0 (socket says 172.21.15.5:42538) 2024-05-09T00:58:11.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.879+0000 7f55f77fe700 1 -- 172.21.15.5:0/1087454691 learned_addr learned my addr 172.21.15.5:0/1087454691 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:58:11.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.879+0000 7f55f6ffd700 1 --2- 172.21.15.5:0/1087454691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609bac38160 0x5609bacd3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:11.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.879+0000 7f55f77fe700 1 -- 172.21.15.5:0/1087454691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609baccb6a0 msgr2=0x5609bacec870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:58:11.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.879+0000 7f55f77fe700 1 --2- 172.21.15.5:0/1087454691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609baccb6a0 0x5609bacec870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:11.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.879+0000 7f55f77fe700 1 -- 172.21.15.5:0/1087454691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609bac38160 msgr2=0x5609bacd3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:58:11.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.879+0000 7f55f77fe700 1 --2- 172.21.15.5:0/1087454691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609bac38160 0x5609bacd3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:11.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.879+0000 7f55f77fe700 1 -- 172.21.15.5:0/1087454691 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609bacd6080 con 0x5609bab4e4a0 2024-05-09T00:58:11.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.879+0000 7f55f7fff700 1 --2- 172.21.15.5:0/1087454691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609baccb6a0 0x5609bacec870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:58:11.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.879+0000 7f55f77fe700 1 --2- 172.21.15.5:0/1087454691 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609bab4e4a0 0x5609bacd39a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f55e8005950 tx=0x7f55e800f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17d5ba1f420645b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:58:11.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.880+0000 7f55dd7fa700 1 -- 172.21.15.5:0/1087454691 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e8028030 con 0x5609bab4e4a0 2024-05-09T00:58:11.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.880+0000 7f55dd7fa700 1 -- 172.21.15.5:0/1087454691 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55e8016040 con 0x5609bab4e4a0 2024-05-09T00:58:11.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.880+0000 7f55dd7fa700 1 -- 172.21.15.5:0/1087454691 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e801f5e0 con 0x5609bab4e4a0 2024-05-09T00:58:11.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.881+0000 7f560c537b80 1 -- 172.21.15.5:0/1087454691 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609bab4e4a0 msgr2=0x5609bacd39a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:11.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.881+0000 7f560c537b80 1 --2- 172.21.15.5:0/1087454691 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609bab4e4a0 0x5609bacd39a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f55e8005950 tx=0x7f55e800f000 comp rx=0 tx=0).stop 2024-05-09T00:58:11.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.882+0000 7f560c537b80 1 -- 172.21.15.5:0/1087454691 shutdown_connections 2024-05-09T00:58:11.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.882+0000 7f560c537b80 1 --2- 172.21.15.5:0/1087454691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609baccb6a0 0x5609bacec870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:11.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.882+0000 7f560c537b80 1 --2- 172.21.15.5:0/1087454691 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609bab4e4a0 0x5609bacd39a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:11.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.882+0000 7f560c537b80 1 --2- 172.21.15.5:0/1087454691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609bac38160 0x5609bacd3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:11.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.882+0000 7f560c537b80 1 -- 172.21.15.5:0/1087454691 >> 172.21.15.5:0/1087454691 conn(0x5609babae6a0 msgr2=0x5609bacdacb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:58:11.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.886+0000 7f560c537b80 1 -- 172.21.15.5:0/1087454691 shutdown_connections 2024-05-09T00:58:11.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.886+0000 7f560c537b80 1 -- 172.21.15.5:0/1087454691 wait complete. 2024-05-09T00:58:11.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.888+0000 7f560c537b80 1 -- start start 2024-05-09T00:58:11.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.889+0000 7f560c537b80 1 Processor -- start 2024-05-09T00:58:11.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.889+0000 7f560c537b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609bab4e4a0 0x5609bacddbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:11.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.890+0000 7f560c537b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609bac38160 0x5609bacde110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:11.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.891+0000 7f55f6ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609bac38160 0x5609bacde110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:11.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.891+0000 7f560c537b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609baccb6a0 0x5609bacde650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:11.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.891+0000 7f55f6ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609bac38160 0x5609bacde110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:46118/0 (socket says 172.21.15.5:46118) 2024-05-09T00:58:11.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.891+0000 7f55f6ffd700 1 -- 172.21.15.5:0/832211531 learned_addr learned my addr 172.21.15.5:0/832211531 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:58:11.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.891+0000 7f560c537b80 1 -- 172.21.15.5:0/832211531 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f55e400e9f0 con 0x5609bab4e4a0 2024-05-09T00:58:11.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.891+0000 7f560c537b80 1 -- 172.21.15.5:0/832211531 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f55e400e320 con 0x5609baccb6a0 2024-05-09T00:58:11.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.891+0000 7f560c537b80 1 -- 172.21.15.5:0/832211531 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f55e400b520 con 0x5609bac38160 2024-05-09T00:58:11.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.892+0000 7f55f7fff700 1 --2- 172.21.15.5:0/832211531 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609baccb6a0 0x5609bacde650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:11.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.893+0000 7f560c537b80 10 client.0 _ll_register_callbacks cb 0x5609baa5f6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:58:11.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.893+0000 7f55f6ffd700 1 -- 172.21.15.5:0/832211531 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609baccb6a0 msgr2=0x5609bacde650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:58:11.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.894+0000 7f55f6ffd700 1 --2- 172.21.15.5:0/832211531 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609baccb6a0 0x5609bacde650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:11.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.894+0000 7f55f6ffd700 1 -- 172.21.15.5:0/832211531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609bab4e4a0 msgr2=0x5609bacddbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:58:11.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.894+0000 7f55f6ffd700 1 --2- 172.21.15.5:0/832211531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609bab4e4a0 0x5609bacddbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:11.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.894+0000 7f55f6ffd700 1 -- 172.21.15.5:0/832211531 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609bad4d420 con 0x5609bac38160 2024-05-09T00:58:11.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.895+0000 7f55f6ffd700 1 --2- 172.21.15.5:0/832211531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609bac38160 0x5609bacde110 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f55f000b0a0 tx=0x7f55f00b0e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ee43a4737cdddb2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:58:11.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.895+0000 7f55d6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-09T00:58:11.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.895+0000 7f55d6ffd700 1 -- 172.21.15.5:0/832211531 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f002a060 con 0x5609bac38160 2024-05-09T00:58:11.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.895+0000 7f55d6ffd700 1 -- 172.21.15.5:0/832211531 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f55f00a6c40 con 0x5609bac38160 2024-05-09T00:58:11.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.896+0000 7f560c537b80 1 -- 172.21.15.5:0/832211531 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55e8016040 con 0x5609bac38160 2024-05-09T00:58:11.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.896+0000 7f55d6ffd700 1 -- 172.21.15.5:0/832211531 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f55f00040d0 con 0x5609bac38160 2024-05-09T00:58:11.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.897+0000 7f560c537b80 10 client.26134 fetch_fsmap learned FSMap version 31 2024-05-09T00:58:11.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.897+0000 7f560c537b80 1 -- 172.21.15.5:0/832211531 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5609bad4d420 con 0x5609bac38160 2024-05-09T00:58:11.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.899+0000 7f55d6ffd700 1 -- 172.21.15.5:0/832211531 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55f00c4070 con 0x5609bac38160 2024-05-09T00:58:11.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.899+0000 7f55d6ffd700 1 -- 172.21.15.5:0/832211531 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f55f0011030 con 0x5609bac38160 2024-05-09T00:58:11.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.899+0000 7f560c537b80 10 client.26134 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:58:11.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.899+0000 7f560c537b80 10 client.26134 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-05-09T00:58:11.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.899+0000 7f560c537b80 20 client.26134 populate_metadata read hostname 'smithi005' 2024-05-09T00:58:11.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.899+0000 7f560c537b80 1 --2- 172.21.15.5:0/832211531 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x5609bad4e820 0x5609bad50c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:11.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.899+0000 7f560c537b80 4 client.26134 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:58:11.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.899+0000 7f560c537b80 1 -- 172.21.15.5:0/832211531 --> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f55e8016040 con 0x5609bad4e820 2024-05-09T00:58:11.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.901+0000 7f55f77fe700 1 --2- 172.21.15.5:0/832211531 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x5609bad4e820 0x5609bad50c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:11.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.904+0000 7f55f77fe700 1 --2- 172.21.15.5:0/832211531 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x5609bad4e820 0x5609bad50c40 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4f68311b45409c80 server_cookie=a7704353d8e9e993 in_seq=0 out_seq=0 2024-05-09T00:58:11.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.904+0000 7f55d6ffd700 10 client.26134 ms_handle_connect on v2:172.21.15.131:6838/2748669631 2024-05-09T00:58:11.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.905+0000 7f55d6ffd700 1 -- 172.21.15.5:0/832211531 <== mds.2 v2:172.21.15.131:6838/2748669631 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f55e8016040 con 0x5609bad4e820 2024-05-09T00:58:11.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.905+0000 7f55d6ffd700 10 client.26134 handle_command_reply: tid=0 2024-05-09T00:58:11.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.905+0000 7f560c537b80 1 client.26134 shutdown 2024-05-09T00:58:11.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.905+0000 7f560c537b80 1 -- 172.21.15.5:0/832211531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609bac38160 msgr2=0x5609bacde110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:11.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.905+0000 7f560c537b80 1 --2- 172.21.15.5:0/832211531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609bac38160 0x5609bacde110 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f55f000b0a0 tx=0x7f55f00b0e80 comp rx=0 tx=0).stop 2024-05-09T00:58:11.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.905+0000 7f560c537b80 1 -- 172.21.15.5:0/832211531 shutdown_connections 2024-05-09T00:58:11.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.905+0000 7f560c537b80 1 --2- 172.21.15.5:0/832211531 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x5609bad4e820 0x5609bad50c40 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:11.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.905+0000 7f560c537b80 1 --2- 172.21.15.5:0/832211531 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5609baccb6a0 0x5609bacde650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:11.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.906+0000 7f560c537b80 1 --2- 172.21.15.5:0/832211531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609bab4e4a0 0x5609bacddbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:11.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.906+0000 7f560c537b80 1 --2- 172.21.15.5:0/832211531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609bac38160 0x5609bacde110 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:11.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.906+0000 7f560c537b80 1 -- 172.21.15.5:0/832211531 >> 172.21.15.5:0/832211531 conn(0x5609babae6a0 msgr2=0x5609bacd3800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:58:11.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.906+0000 7f55d6ffd700 0 client.26134 ms_handle_reset on v2:172.21.15.131:6838/2748669631 2024-05-09T00:58:11.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.906+0000 7f560c537b80 1 -- 172.21.15.5:0/832211531 shutdown_connections 2024-05-09T00:58:11.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.906+0000 7f560c537b80 1 -- 172.21.15.5:0/832211531 wait complete. 2024-05-09T00:58:11.910 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-09T00:58:11.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.907+0000 7f560c537b80 20 client.26134 trim_cache size 0 max 16384 2024-05-09T00:58:11.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.909+0000 7f55fdb21700 1 -- 172.21.15.5:0/524516676 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f55e00b2a90 msgr2=0x7f55e00b4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:11.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.909+0000 7f55fdb21700 1 --2- 172.21.15.5:0/524516676 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f55e00b2a90 0x7f55e00b4fa0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f55e8005fd0 tx=0x7f55e8005b90 comp rx=0 tx=0).stop 2024-05-09T00:58:11.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.909+0000 7f55fdb21700 1 -- 172.21.15.5:0/524516676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55f810b280 msgr2=0x7f55f8114e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:11.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.909+0000 7f55fdb21700 1 --2- 172.21.15.5:0/524516676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55f810b280 0x7f55f8114e80 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f55f000b070 tx=0x7f55f0004d10 comp rx=0 tx=0).stop 2024-05-09T00:58:11.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.910+0000 7f55fdb21700 1 -- 172.21.15.5:0/524516676 shutdown_connections 2024-05-09T00:58:11.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.910+0000 7f55fdb21700 1 --2- 172.21.15.5:0/524516676 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f55e00b2a90 0x7f55e00b4fa0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:11.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.910+0000 7f55fdb21700 1 --2- 172.21.15.5:0/524516676 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55f811d930 0x7f55f8115440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:11.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.910+0000 7f55fdb21700 1 --2- 172.21.15.5:0/524516676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55f810a930 0x7f55f811bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:11.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.910+0000 7f55fdb21700 1 --2- 172.21.15.5:0/524516676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55f810b280 0x7f55f8114e80 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:11.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.910+0000 7f55fdb21700 1 -- 172.21.15.5:0/524516676 >> 172.21.15.5:0/524516676 conn(0x7f55f806f4d0 msgr2=0x7f55f8072790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:58:11.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.912+0000 7f55fdb21700 1 -- 172.21.15.5:0/524516676 shutdown_connections 2024-05-09T00:58:11.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:11.912+0000 7f55fdb21700 1 -- 172.21.15.5:0/524516676 wait complete. 2024-05-09T00:58:11.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:58:11 smithi157 ceph-mon[34294]: pgmap v726: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 934 MiB/s wr, 2.07k op/s 2024-05-09T00:58:13.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:58:13 smithi131 ceph-mon[34547]: pgmap v727: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 822 MiB/s wr, 1.73k op/s 2024-05-09T00:58:13.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:58:13 smithi157 ceph-mon[34294]: pgmap v727: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 822 MiB/s wr, 1.73k op/s 2024-05-09T00:58:14.040 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:58:13 smithi005 ceph-mon[30365]: pgmap v727: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 822 MiB/s wr, 1.73k op/s 2024-05-09T00:58:14.567 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T00:58:15.257 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:58:15.790 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:58:15 smithi005 ceph-mon[30365]: pgmap v728: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 844 MiB/s wr, 1.92k op/s 2024-05-09T00:58:15.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:58:15 smithi157 ceph-mon[34294]: pgmap v728: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 844 MiB/s wr, 1.92k op/s 2024-05-09T00:58:16.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:58:15 smithi131 ceph-mon[34547]: pgmap v728: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 844 MiB/s wr, 1.92k op/s 2024-05-09T00:58:17.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:58:17 smithi157 ceph-mon[34294]: pgmap v729: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 929 MiB/s wr, 2.30k op/s 2024-05-09T00:58:18.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:58:17 smithi005 ceph-mon[30365]: pgmap v729: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 929 MiB/s wr, 2.30k op/s 2024-05-09T00:58:18.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:58:17 smithi131 ceph-mon[34547]: pgmap v729: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 929 MiB/s wr, 2.30k op/s 2024-05-09T00:58:19.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.536+0000 7f837e1ff640 1 -- 172.21.15.5:0/2744162090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83781101a0 msgr2=0x7f83781105b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:19.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.537+0000 7f83767fc640 1 -- 172.21.15.5:0/2744162090 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8368004230 con 0x7f83781101a0 2024-05-09T00:58:19.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.537+0000 7f837e1ff640 1 --2- 172.21.15.5:0/2744162090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83781101a0 0x7f83781105b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8368009900 tx=0x7f836802a410 comp rx=0 tx=0).stop 2024-05-09T00:58:19.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.540+0000 7f837e1ff640 1 -- 172.21.15.5:0/2744162090 shutdown_connections 2024-05-09T00:58:19.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.540+0000 7f837e1ff640 1 --2- 172.21.15.5:0/2744162090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83781101a0 0x7f83781105b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:19.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.540+0000 7f837e1ff640 1 --2- 172.21.15.5:0/2744162090 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f837810b480 0x7f837810b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:19.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.540+0000 7f837e1ff640 1 --2- 172.21.15.5:0/2744162090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f837810a550 0x7f837810af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:19.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.540+0000 7f837e1ff640 1 -- 172.21.15.5:0/2744162090 >> 172.21.15.5:0/2744162090 conn(0x7f8378070670 msgr2=0x7f8378072b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:58:19.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.544+0000 7f837e1ff640 1 -- 172.21.15.5:0/2744162090 shutdown_connections 2024-05-09T00:58:19.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.546+0000 7f837e1ff640 1 -- 172.21.15.5:0/2744162090 wait complete. 2024-05-09T00:58:19.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.548+0000 7f837e1ff640 1 Processor -- start 2024-05-09T00:58:19.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.548+0000 7f837e1ff640 1 -- start start 2024-05-09T00:58:19.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.549+0000 7f837e1ff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f837810a550 0x7f83781a9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:19.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.549+0000 7f837e1ff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f837810b480 0x7f83781a9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:19.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.549+0000 7f837e1ff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83781101a0 0x7f83781b06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:19.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.549+0000 7f837e1ff640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f837811e6b0 con 0x7f837810a550 2024-05-09T00:58:19.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.549+0000 7f837e1ff640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f837811e530 con 0x7f837810b480 2024-05-09T00:58:19.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.549+0000 7f837e1ff640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f837811e830 con 0x7f83781101a0 2024-05-09T00:58:19.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.549+0000 7f8377fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83781101a0 0x7f83781b06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:19.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.549+0000 7f8377fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83781101a0 0x7f83781b06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:36424/0 (socket says 172.21.15.5:36424) 2024-05-09T00:58:19.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.550+0000 7f8377fff640 1 -- 172.21.15.5:0/3088868717 learned_addr learned my addr 172.21.15.5:0/3088868717 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:58:19.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.550+0000 7f8377fff640 1 -- 172.21.15.5:0/3088868717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f837810b480 msgr2=0x7f83781a9760 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T00:58:19.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.550+0000 7f8377fff640 1 --2- 172.21.15.5:0/3088868717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f837810b480 0x7f83781a9760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:19.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.551+0000 7f8377fff640 1 -- 172.21.15.5:0/3088868717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f837810a550 msgr2=0x7f83781a9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:19.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.551+0000 7f83777fe640 1 --2- 172.21.15.5:0/3088868717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f837810a550 0x7f83781a9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:19.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.551+0000 7f8377fff640 1 --2- 172.21.15.5:0/3088868717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f837810a550 0x7f83781a9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:19.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.551+0000 7f8377fff640 1 -- 172.21.15.5:0/3088868717 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83781b0d60 con 0x7f83781101a0 2024-05-09T00:58:19.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.551+0000 7f83777fe640 1 --2- 172.21.15.5:0/3088868717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f837810a550 0x7f83781a9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:58:19.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.551+0000 7f8377fff640 1 --2- 172.21.15.5:0/3088868717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83781101a0 0x7f83781b06f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8368033040 tx=0x7f83680048f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:58:19.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.556+0000 7f8374ff9640 1 -- 172.21.15.5:0/3088868717 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8368040030 con 0x7f83781101a0 2024-05-09T00:58:19.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.556+0000 7f837e1ff640 1 -- 172.21.15.5:0/3088868717 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83781b0ff0 con 0x7f83781101a0 2024-05-09T00:58:19.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.556+0000 7f837e1ff640 1 -- 172.21.15.5:0/3088868717 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83781b14d0 con 0x7f83781101a0 2024-05-09T00:58:19.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.557+0000 7f8374ff9640 1 -- 172.21.15.5:0/3088868717 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f836803cb00 con 0x7f83781101a0 2024-05-09T00:58:19.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.557+0000 7f8374ff9640 1 -- 172.21.15.5:0/3088868717 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8368031af0 con 0x7f83781101a0 2024-05-09T00:58:19.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.562+0000 7f8374ff9640 1 -- 172.21.15.5:0/3088868717 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f8368043030 con 0x7f83781101a0 2024-05-09T00:58:19.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.565+0000 7f8374ff9640 1 --2- 172.21.15.5:0/3088868717 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f83600b2a00 0x7f83600b4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:19.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.566+0000 7f8374ff9640 1 -- 172.21.15.5:0/3088868717 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f8368103ae0 con 0x7f83781101a0 2024-05-09T00:58:19.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.567+0000 7f83777fe640 1 --2- 172.21.15.5:0/3088868717 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f83600b2a00 0x7f83600b4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:19.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.571+0000 7f8385187740 1 Processor -- start 2024-05-09T00:58:19.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.572+0000 7f8385187740 1 -- start start 2024-05-09T00:58:19.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.572+0000 7f8385187740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc758611b0 0x55dc75c05500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:19.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.573+0000 7f8385187740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc758382f0 0x55dc75c05a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:19.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.573+0000 7f83777fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc758611b0 0x55dc75c05500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:19.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.573+0000 7f83777fe640 1 --2- 172.21.15.5:0/3088868717 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f83600b2a00 0x7f83600b4f10 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f8378109d90 tx=0x7f83700073c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:58:19.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.574+0000 7f8376ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc758382f0 0x55dc75c05a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:19.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.574+0000 7f83777fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc758611b0 0x55dc75c05500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:36438/0 (socket says 172.21.15.5:36438) 2024-05-09T00:58:19.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.574+0000 7f83777fe640 1 -- 172.21.15.5:0/1155658965 learned_addr learned my addr 172.21.15.5:0/1155658965 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:58:19.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.574+0000 7f8385187740 1 --2- 172.21.15.5:0/1155658965 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc758404d0 0x55dc75c0c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:19.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.574+0000 7f8385187740 1 -- 172.21.15.5:0/1155658965 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dc75b697c0 con 0x55dc758382f0 2024-05-09T00:58:19.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.574+0000 7f8385187740 1 -- 172.21.15.5:0/1155658965 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55dc75b050e0 con 0x55dc758404d0 2024-05-09T00:58:19.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.574+0000 7f8385187740 1 -- 172.21.15.5:0/1155658965 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55dc75b65780 con 0x55dc758611b0 2024-05-09T00:58:19.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.575+0000 7f8377fff640 1 --2- 172.21.15.5:0/1155658965 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc758404d0 0x55dc75c0c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:19.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.575+0000 7f8377fff640 1 -- 172.21.15.5:0/1155658965 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc758611b0 msgr2=0x55dc75c05500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:58:19.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.575+0000 7f8377fff640 1 --2- 172.21.15.5:0/1155658965 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc758611b0 0x55dc75c05500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:19.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.575+0000 7f8377fff640 1 -- 172.21.15.5:0/1155658965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc758382f0 msgr2=0x55dc75c05a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:58:19.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.575+0000 7f8377fff640 1 --2- 172.21.15.5:0/1155658965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc758382f0 0x55dc75c05a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:19.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.575+0000 7f8377fff640 1 -- 172.21.15.5:0/1155658965 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc75b2e7c0 con 0x55dc758404d0 2024-05-09T00:58:19.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.576+0000 7f8377fff640 1 --2- 172.21.15.5:0/1155658965 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc758404d0 0x55dc75c0c980 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f836803cfd0 tx=0x7f83680cacc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0a327418c43a667 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:58:19.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.579+0000 7f835dffb640 1 -- 172.21.15.5:0/1155658965 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83680d3ed0 con 0x55dc758404d0 2024-05-09T00:58:19.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.579+0000 7f835dffb640 1 -- 172.21.15.5:0/1155658965 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8368031440 con 0x55dc758404d0 2024-05-09T00:58:19.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.579+0000 7f835dffb640 1 -- 172.21.15.5:0/1155658965 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f836803cdd0 con 0x55dc758404d0 2024-05-09T00:58:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.581+0000 7f8385187740 1 -- 172.21.15.5:0/1155658965 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc758404d0 msgr2=0x55dc75c0c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.582+0000 7f8385187740 1 --2- 172.21.15.5:0/1155658965 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc758404d0 0x55dc75c0c980 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f836803cfd0 tx=0x7f83680cacc0 comp rx=0 tx=0).stop 2024-05-09T00:58:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.587+0000 7f8385187740 1 -- 172.21.15.5:0/1155658965 shutdown_connections 2024-05-09T00:58:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.587+0000 7f8385187740 1 --2- 172.21.15.5:0/1155658965 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc758404d0 0x55dc75c0c980 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.587+0000 7f8385187740 1 --2- 172.21.15.5:0/1155658965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc758382f0 0x55dc75c05a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.587+0000 7f8385187740 1 --2- 172.21.15.5:0/1155658965 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc758611b0 0x55dc75c05500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.587+0000 7f8385187740 1 -- 172.21.15.5:0/1155658965 >> 172.21.15.5:0/1155658965 conn(0x55dc75855300 msgr2=0x55dc75c00e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:58:19.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.592+0000 7f8385187740 1 -- 172.21.15.5:0/1155658965 shutdown_connections 2024-05-09T00:58:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.592+0000 7f8385187740 1 -- 172.21.15.5:0/1155658965 wait complete. 2024-05-09T00:58:19.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.593+0000 7f8385187740 1 -- start start 2024-05-09T00:58:19.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.593+0000 7f8385187740 1 Processor -- start 2024-05-09T00:58:19.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.594+0000 7f8385187740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc758382f0 0x55dc75c0f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.594+0000 7f8385187740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc758404d0 0x55dc75c0f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.594+0000 7f8385187740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc758611b0 0x55dc75c164e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.594+0000 7f8385187740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dc75c0d190 con 0x55dc758404d0 2024-05-09T00:58:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.594+0000 7f83777fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc758382f0 0x55dc75c0f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.595+0000 7f83777fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc758382f0 0x55dc75c0f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:36440/0 (socket says 172.21.15.5:36440) 2024-05-09T00:58:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.595+0000 7f83777fe640 1 -- 172.21.15.5:0/3856785566 learned_addr learned my addr 172.21.15.5:0/3856785566 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:58:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.595+0000 7f8385187740 1 -- 172.21.15.5:0/3856785566 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55dc75af09c0 con 0x55dc758611b0 2024-05-09T00:58:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.595+0000 7f8385187740 1 -- 172.21.15.5:0/3856785566 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55dc75c0d310 con 0x55dc758382f0 2024-05-09T00:58:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.595+0000 7f8377fff640 1 --2- 172.21.15.5:0/3856785566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc758611b0 0x55dc75c164e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.595+0000 7f8385187740 10 client.0 _ll_register_callbacks cb 0x55dc75b1d020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:58:19.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.597+0000 7f8376ffd640 1 --2- 172.21.15.5:0/3856785566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc758404d0 0x55dc75c0f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:19.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.598+0000 7f83777fe640 1 -- 172.21.15.5:0/3856785566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc758611b0 msgr2=0x55dc75c164e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:58:19.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.598+0000 7f83777fe640 1 --2- 172.21.15.5:0/3856785566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc758611b0 0x55dc75c164e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:19.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.598+0000 7f83777fe640 1 -- 172.21.15.5:0/3856785566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc758404d0 msgr2=0x55dc75c0f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:58:19.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.598+0000 7f83777fe640 1 --2- 172.21.15.5:0/3856785566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc758404d0 0x55dc75c0f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:19.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.598+0000 7f83777fe640 1 -- 172.21.15.5:0/3856785566 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc75b171f0 con 0x55dc758382f0 2024-05-09T00:58:19.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.599+0000 7f83777fe640 1 --2- 172.21.15.5:0/3856785566 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc758382f0 0x55dc75c0f040 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f837000fe10 tx=0x7f8370004b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50b4615da1f856e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:58:19.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.600+0000 7f8356ffd640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-09T00:58:19.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.600+0000 7f8356ffd640 1 -- 172.21.15.5:0/3856785566 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8370021070 con 0x55dc758382f0 2024-05-09T00:58:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.600+0000 7f8385187740 1 -- 172.21.15.5:0/3856785566 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8368031440 con 0x55dc758382f0 2024-05-09T00:58:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.600+0000 7f8356ffd640 1 -- 172.21.15.5:0/3856785566 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f837000ed30 con 0x55dc758382f0 2024-05-09T00:58:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.601+0000 7f8356ffd640 1 -- 172.21.15.5:0/3856785566 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f837002d050 con 0x55dc758382f0 2024-05-09T00:58:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.601+0000 7f8385187740 10 client.26146 fetch_fsmap learned FSMap version 31 2024-05-09T00:58:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.601+0000 7f8385187740 1 -- 172.21.15.5:0/3856785566 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55dc758cc790 con 0x55dc758382f0 2024-05-09T00:58:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.602+0000 7f8356ffd640 1 -- 172.21.15.5:0/3856785566 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8368031440 con 0x55dc758382f0 2024-05-09T00:58:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.603+0000 7f8356ffd640 1 -- 172.21.15.5:0/3856785566 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f83700254c0 con 0x55dc758382f0 2024-05-09T00:58:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.604+0000 7f8385187740 10 client.26146 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:58:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.604+0000 7f8385187740 10 client.26146 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:58:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.604+0000 7f8385187740 20 client.26146 populate_metadata read hostname 'smithi005' 2024-05-09T00:58:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.604+0000 7f8385187740 1 --2- 172.21.15.5:0/3856785566 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55dc7581fd80 0x55dc75c1ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.604+0000 7f8385187740 4 client.26146 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:58:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.604+0000 7f8385187740 1 -- 172.21.15.5:0/3856785566 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc75b773d0 con 0x55dc7581fd80 2024-05-09T00:58:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.607+0000 7f8376ffd640 1 --2- 172.21.15.5:0/3856785566 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55dc7581fd80 0x55dc75c1ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:19.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.614+0000 7f8376ffd640 1 --2- 172.21.15.5:0/3856785566 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55dc7581fd80 0x55dc75c1ccf0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5df4a93d5b81287 server_cookie=9cbac3435fea6852 in_seq=0 out_seq=0 2024-05-09T00:58:19.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.614+0000 7f8356ffd640 10 client.26146 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:58:19.621 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.620+0000 7f8356ffd640 1 -- 172.21.15.5:0/3856785566 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dc75b773d0 con 0x55dc7581fd80 2024-05-09T00:58:19.621 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.621+0000 7f8356ffd640 10 client.26146 handle_command_reply: tid=0 2024-05-09T00:58:19.621 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.621+0000 7f8385187740 1 client.26146 shutdown 2024-05-09T00:58:19.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.629+0000 7f8385187740 1 -- 172.21.15.5:0/3856785566 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc758382f0 msgr2=0x55dc75c0f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:19.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.629+0000 7f8385187740 1 --2- 172.21.15.5:0/3856785566 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc758382f0 0x55dc75c0f040 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f837000fe10 tx=0x7f8370004b50 comp rx=0 tx=0).stop 2024-05-09T00:58:19.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.629+0000 7f8385187740 1 -- 172.21.15.5:0/3856785566 shutdown_connections 2024-05-09T00:58:19.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.629+0000 7f8385187740 1 --2- 172.21.15.5:0/3856785566 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55dc7581fd80 0x55dc75c1ccf0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:19.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.629+0000 7f8385187740 1 --2- 172.21.15.5:0/3856785566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc758611b0 0x55dc75c164e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:19.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.629+0000 7f8385187740 1 --2- 172.21.15.5:0/3856785566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc758404d0 0x55dc75c0f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:19.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.629+0000 7f8385187740 1 --2- 172.21.15.5:0/3856785566 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc758382f0 0x55dc75c0f040 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:19.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.629+0000 7f8385187740 1 -- 172.21.15.5:0/3856785566 >> 172.21.15.5:0/3856785566 conn(0x55dc75855300 msgr2=0x55dc75c03da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:58:19.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.629+0000 7f8356ffd640 0 client.26146 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:58:19.635 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.634+0000 7f8385187740 1 -- 172.21.15.5:0/3856785566 shutdown_connections 2024-05-09T00:58:19.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.636+0000 7f8385187740 1 -- 172.21.15.5:0/3856785566 wait complete. 2024-05-09T00:58:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.636+0000 7f8385187740 20 client.26146 trim_cache size 0 max 16384 2024-05-09T00:58:19.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.685+0000 7f8385187740 1 Processor -- start 2024-05-09T00:58:19.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.689+0000 7f8385187740 1 -- start start 2024-05-09T00:58:19.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.689+0000 7f8385187740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc758382f0 0x55dc75c0d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:19.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.689+0000 7f8385187740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc758404d0 0x55dc75c0db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:19.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.689+0000 7f8385187740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc758611b0 0x55dc75c16470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:19.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.689+0000 7f8385187740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f836400e4c0 con 0x55dc758404d0 2024-05-09T00:58:19.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.689+0000 7f8385187740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8364010000 con 0x55dc758611b0 2024-05-09T00:58:19.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.689+0000 7f8385187740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f836400eb90 con 0x55dc758382f0 2024-05-09T00:58:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.690+0000 7f8376ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc758404d0 0x55dc75c0db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.690+0000 7f8376ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc758404d0 0x55dc75c0db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32948/0 (socket says 172.21.15.5:32948) 2024-05-09T00:58:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.690+0000 7f8376ffd640 1 -- 172.21.15.5:0/1457132874 learned_addr learned my addr 172.21.15.5:0/1457132874 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:58:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.690+0000 7f8377fff640 1 --2- 172.21.15.5:0/1457132874 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc758611b0 0x55dc75c16470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.691+0000 7f83777fe640 1 --2- 172.21.15.5:0/1457132874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc758382f0 0x55dc75c0d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.691+0000 7f8376ffd640 1 -- 172.21.15.5:0/1457132874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc758382f0 msgr2=0x55dc75c0d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:58:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.691+0000 7f8376ffd640 1 --2- 172.21.15.5:0/1457132874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc758382f0 0x55dc75c0d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.691+0000 7f8376ffd640 1 -- 172.21.15.5:0/1457132874 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc758611b0 msgr2=0x55dc75c16470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:58:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.691+0000 7f8376ffd640 1 --2- 172.21.15.5:0/1457132874 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc758611b0 0x55dc75c16470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.691+0000 7f8376ffd640 1 -- 172.21.15.5:0/1457132874 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc759435f0 con 0x55dc758404d0 2024-05-09T00:58:19.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.694+0000 7f8376ffd640 1 --2- 172.21.15.5:0/1457132874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc758404d0 0x55dc75c0db40 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x55dc75c068e0 tx=0x7f836c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35a4c652d320949 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:58:19.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.698+0000 7f835d7fa640 1 -- 172.21.15.5:0/1457132874 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f836c004310 con 0x55dc758404d0 2024-05-09T00:58:19.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.698+0000 7f835d7fa640 1 -- 172.21.15.5:0/1457132874 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f836c0044b0 con 0x55dc758404d0 2024-05-09T00:58:19.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.698+0000 7f835d7fa640 1 -- 172.21.15.5:0/1457132874 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f836c003850 con 0x55dc758404d0 2024-05-09T00:58:19.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.702+0000 7f8385187740 1 -- 172.21.15.5:0/1457132874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc758404d0 msgr2=0x55dc75c0db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:19.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.702+0000 7f8385187740 1 --2- 172.21.15.5:0/1457132874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc758404d0 0x55dc75c0db40 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x55dc75c068e0 tx=0x7f836c005f90 comp rx=0 tx=0).stop 2024-05-09T00:58:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.706+0000 7f8385187740 1 -- 172.21.15.5:0/1457132874 shutdown_connections 2024-05-09T00:58:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.706+0000 7f8385187740 1 --2- 172.21.15.5:0/1457132874 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc758611b0 0x55dc75c16470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.706+0000 7f8385187740 1 --2- 172.21.15.5:0/1457132874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc758404d0 0x55dc75c0db40 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.706+0000 7f8385187740 1 --2- 172.21.15.5:0/1457132874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc758382f0 0x55dc75c0d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.706+0000 7f8385187740 1 -- 172.21.15.5:0/1457132874 >> 172.21.15.5:0/1457132874 conn(0x55dc75855300 msgr2=0x55dc75c09740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:58:19.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.709+0000 7f8385187740 1 -- 172.21.15.5:0/1457132874 shutdown_connections 2024-05-09T00:58:19.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.709+0000 7f8385187740 1 -- 172.21.15.5:0/1457132874 wait complete. 2024-05-09T00:58:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.710+0000 7f8385187740 1 -- start start 2024-05-09T00:58:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.711+0000 7f8385187740 1 Processor -- start 2024-05-09T00:58:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.712+0000 7f8385187740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc758382f0 0x55dc75c12e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.712+0000 7f8385187740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc758404d0 0x55dc75c133a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.712+0000 7f8385187740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc758611b0 0x55dc75c162e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.712+0000 7f8385187740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f836400f930 con 0x55dc758382f0 2024-05-09T00:58:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.712+0000 7f8385187740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f836400f260 con 0x55dc758611b0 2024-05-09T00:58:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.712+0000 7f8385187740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f836400c550 con 0x55dc758404d0 2024-05-09T00:58:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.713+0000 7f8385187740 10 client.0 _ll_register_callbacks cb 0x7f8370007f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:58:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.713+0000 7f83777fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc758382f0 0x55dc75c12e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.713+0000 7f83777fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc758382f0 0x55dc75c12e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32964/0 (socket says 172.21.15.5:32964) 2024-05-09T00:58:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.713+0000 7f83777fe640 1 -- 172.21.15.5:0/2541958212 learned_addr learned my addr 172.21.15.5:0/2541958212 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:58:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.713+0000 7f8377fff640 1 --2- 172.21.15.5:0/2541958212 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc758611b0 0x55dc75c162e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.714+0000 7f8376ffd640 1 --2- 172.21.15.5:0/2541958212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc758404d0 0x55dc75c133a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.715+0000 7f83777fe640 1 -- 172.21.15.5:0/2541958212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc758404d0 msgr2=0x55dc75c133a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:58:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.715+0000 7f83777fe640 1 --2- 172.21.15.5:0/2541958212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc758404d0 0x55dc75c133a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.715+0000 7f83777fe640 1 -- 172.21.15.5:0/2541958212 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc758611b0 msgr2=0x55dc75c162e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:58:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.715+0000 7f83777fe640 1 --2- 172.21.15.5:0/2541958212 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc758611b0 0x55dc75c162e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.715+0000 7f83777fe640 1 -- 172.21.15.5:0/2541958212 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc75b2ded0 con 0x55dc758382f0 2024-05-09T00:58:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.716+0000 7f83777fe640 1 --2- 172.21.15.5:0/2541958212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc758382f0 0x55dc75c12e60 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f8370025a30 tx=0x7f8370007070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73c02d4b4cbc794d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:58:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.716+0000 7f8356ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:58:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.717+0000 7f8356ffd640 1 -- 172.21.15.5:0/2541958212 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8370035070 con 0x55dc758382f0 2024-05-09T00:58:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.717+0000 7f8356ffd640 1 -- 172.21.15.5:0/2541958212 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f837000eb30 con 0x55dc758382f0 2024-05-09T00:58:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.717+0000 7f8385187740 1 -- 172.21.15.5:0/2541958212 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f836c0044b0 con 0x55dc758382f0 2024-05-09T00:58:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.718+0000 7f8356ffd640 1 -- 172.21.15.5:0/2541958212 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8370040030 con 0x55dc758382f0 2024-05-09T00:58:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.718+0000 7f8385187740 10 client.16743 fetch_fsmap learned FSMap version 31 2024-05-09T00:58:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.718+0000 7f8385187740 1 -- 172.21.15.5:0/2541958212 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55dc75c837b0 con 0x55dc758382f0 2024-05-09T00:58:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.719+0000 7f8356ffd640 1 -- 172.21.15.5:0/2541958212 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8370015040 con 0x55dc758382f0 2024-05-09T00:58:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.719+0000 7f8356ffd640 1 -- 172.21.15.5:0/2541958212 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f8370026b00 con 0x55dc758382f0 2024-05-09T00:58:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.719+0000 7f8385187740 10 client.16743 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:58:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.719+0000 7f8385187740 10 client.16743 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:58:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.719+0000 7f8385187740 20 client.16743 populate_metadata read hostname 'smithi005' 2024-05-09T00:58:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.719+0000 7f8385187740 1 --2- 172.21.15.5:0/2541958212 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55dc757f51c0 0x55dc75c86200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.720+0000 7f8385187740 4 client.16743 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:58:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.720+0000 7f8385187740 1 -- 172.21.15.5:0/2541958212 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f836c0044b0 con 0x55dc757f51c0 2024-05-09T00:58:19.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.721+0000 7f8376ffd640 1 --2- 172.21.15.5:0/2541958212 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55dc757f51c0 0x55dc75c86200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:19.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.724+0000 7f8376ffd640 1 --2- 172.21.15.5:0/2541958212 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55dc757f51c0 0x55dc75c86200 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71f5fcf2518cf2bd server_cookie=83b13448befc8915 in_seq=0 out_seq=0 2024-05-09T00:58:19.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:19.725+0000 7f8356ffd640 10 client.16743 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:58:19.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:58:19 smithi005 ceph-mon[30365]: pgmap v730: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 856 MiB/s wr, 1.96k op/s 2024-05-09T00:58:19.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.896+0000 7effe9aef700 1 -- 172.21.15.5:0/2517990885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7effe4103850 msgr2=0x7effe4110570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:19.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.896+0000 7effe9aef700 1 --2- 172.21.15.5:0/2517990885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7effe4103850 0x7effe4110570 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7effcc009990 tx=0x7effcc009ca0 comp rx=0 tx=0).stop 2024-05-09T00:58:19.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.897+0000 7effe9aef700 1 -- 172.21.15.5:0/2517990885 shutdown_connections 2024-05-09T00:58:19.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.897+0000 7effe9aef700 1 --2- 172.21.15.5:0/2517990885 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effe4110ab0 0x7effe4112ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:19.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.897+0000 7effe9aef700 1 --2- 172.21.15.5:0/2517990885 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7effe4102f00 0x7effe4103310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:19.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.897+0000 7effe9aef700 1 --2- 172.21.15.5:0/2517990885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7effe4103850 0x7effe4110570 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:19.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.897+0000 7effe9aef700 1 -- 172.21.15.5:0/2517990885 >> 172.21.15.5:0/2517990885 conn(0x7effe4078bf0 msgr2=0x7effe4079030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:58:19.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.902+0000 7effe9aef700 1 -- 172.21.15.5:0/2517990885 shutdown_connections 2024-05-09T00:58:19.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.908+0000 7effe9aef700 1 -- 172.21.15.5:0/2517990885 wait complete. 2024-05-09T00:58:19.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.910+0000 7effe9aef700 1 Processor -- start 2024-05-09T00:58:19.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.910+0000 7effe9aef700 1 -- start start 2024-05-09T00:58:19.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.911+0000 7effe9aef700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effe4103850 0x7effe406d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:19.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.911+0000 7effe9aef700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7effe4110ab0 0x7effe406deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:19.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.911+0000 7effe9aef700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7effe406e3f0 0x7effe41ad220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:19.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.911+0000 7effe9aef700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7effe41081c0 con 0x7effe406e3f0 2024-05-09T00:58:19.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.911+0000 7effe9aef700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7effe4108040 con 0x7effe4110ab0 2024-05-09T00:58:19.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.911+0000 7effe9aef700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7effe4108340 con 0x7effe4103850 2024-05-09T00:58:19.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.912+0000 7effe92ee700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7effe406e3f0 0x7effe41ad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:19.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.912+0000 7effe92ee700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7effe406e3f0 0x7effe41ad220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32972/0 (socket says 172.21.15.5:32972) 2024-05-09T00:58:19.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.912+0000 7effe92ee700 1 -- 172.21.15.5:0/1407341833 learned_addr learned my addr 172.21.15.5:0/1407341833 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:58:19.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.914+0000 7effe3fff700 1 --2- 172.21.15.5:0/1407341833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7effe4110ab0 0x7effe406deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:19.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.914+0000 7effe8aed700 1 --2- 172.21.15.5:0/1407341833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effe4103850 0x7effe406d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:19.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.919+0000 7effe3fff700 1 -- 172.21.15.5:0/1407341833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effe4103850 msgr2=0x7effe406d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:19.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.919+0000 7effe3fff700 1 --2- 172.21.15.5:0/1407341833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effe4103850 0x7effe406d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:19.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.919+0000 7effe3fff700 1 -- 172.21.15.5:0/1407341833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7effe406e3f0 msgr2=0x7effe41ad220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:19.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.919+0000 7effe3fff700 1 --2- 172.21.15.5:0/1407341833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7effe406e3f0 0x7effe41ad220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:19.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.919+0000 7effe3fff700 1 -- 172.21.15.5:0/1407341833 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effe41ad930 con 0x7effe4110ab0 2024-05-09T00:58:19.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.926+0000 7effe3fff700 1 --2- 172.21.15.5:0/1407341833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7effe4110ab0 0x7effe406deb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7effcc000c00 tx=0x7effcc0057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:58:19.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.930+0000 7effe1ffb700 1 -- 172.21.15.5:0/1407341833 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc038070 con 0x7effe4110ab0 2024-05-09T00:58:19.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.930+0000 7effe9aef700 1 -- 172.21.15.5:0/1407341833 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effe41adbc0 con 0x7effe4110ab0 2024-05-09T00:58:19.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.930+0000 7effe9aef700 1 -- 172.21.15.5:0/1407341833 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effe41ae070 con 0x7effe4110ab0 2024-05-09T00:58:19.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.930+0000 7effe1ffb700 1 -- 172.21.15.5:0/1407341833 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effcc003680 con 0x7effe4110ab0 2024-05-09T00:58:19.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.930+0000 7effe1ffb700 1 -- 172.21.15.5:0/1407341833 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc033af0 con 0x7effe4110ab0 2024-05-09T00:58:19.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:58:19 smithi157 ceph-mon[34294]: pgmap v730: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 856 MiB/s wr, 1.96k op/s 2024-05-09T00:58:19.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.941+0000 7efff9767b80 1 Processor -- start 2024-05-09T00:58:19.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.941+0000 7efff9767b80 1 -- start start 2024-05-09T00:58:19.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.944+0000 7effe1ffb700 1 -- 172.21.15.5:0/1407341833 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7effcc01f3f0 con 0x7effe4110ab0 2024-05-09T00:58:19.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.945+0000 7effe1ffb700 1 --2- 172.21.15.5:0/1407341833 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7effd00b29b0 0x7effd00b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:19.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.946+0000 7effe1ffb700 1 -- 172.21.15.5:0/1407341833 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7effcc0f4020 con 0x7effe4110ab0 2024-05-09T00:58:19.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.947+0000 7efff9767b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559e44a2f160 0x559e44ac8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:19.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.947+0000 7efff9767b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e449454a0 0x559e44ac8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:19.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.947+0000 7efff9767b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e44ac2800 0x559e44ac2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:19.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.947+0000 7efff9767b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559e449e4ff0 con 0x559e44ac2800 2024-05-09T00:58:19.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.947+0000 7efff9767b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x559e449837f0 con 0x559e44a2f160 2024-05-09T00:58:19.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.947+0000 7efff9767b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x559e449e0fb0 con 0x559e449454a0 2024-05-09T00:58:19.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.948+0000 7effe92ee700 1 --2- 172.21.15.5:0/1407341833 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7effd00b29b0 0x7effd00b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:19.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.949+0000 7effe3fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e449454a0 0x559e44ac8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:19.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.949+0000 7effe3fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e449454a0 0x559e44ac8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:36466/0 (socket says 172.21.15.5:36466) 2024-05-09T00:58:19.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.949+0000 7effe3fff700 1 -- 172.21.15.5:0/1618835455 learned_addr learned my addr 172.21.15.5:0/1618835455 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:58:19.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.953+0000 7effe8aed700 1 --2- 172.21.15.5:0/1618835455 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559e44a2f160 0x559e44ac8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:19.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.954+0000 7effe3fff700 1 -- 172.21.15.5:0/1618835455 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559e44a2f160 msgr2=0x559e44ac8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:58:19.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.954+0000 7effe3fff700 1 --2- 172.21.15.5:0/1618835455 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559e44a2f160 0x559e44ac8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:19.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.954+0000 7effe3fff700 1 -- 172.21.15.5:0/1618835455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e44ac2800 msgr2=0x559e44ac2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:58:19.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.954+0000 7effe3fff700 1 --2- 172.21.15.5:0/1618835455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e44ac2800 0x559e44ac2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:19.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.954+0000 7effe3fff700 1 -- 172.21.15.5:0/1618835455 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e4497b370 con 0x559e449454a0 2024-05-09T00:58:19.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.957+0000 7effe3fff700 1 --2- 172.21.15.5:0/1618835455 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e449454a0 0x559e44ac8b80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7effcc0b76c0 tx=0x7effcc009710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8b7ae28c33ad20d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:58:19.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.960+0000 7effdaffd700 1 -- 172.21.15.5:0/1618835455 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc0bbd40 con 0x559e449454a0 2024-05-09T00:58:19.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.961+0000 7effdaffd700 1 -- 172.21.15.5:0/1618835455 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effcc0bb640 con 0x559e449454a0 2024-05-09T00:58:19.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.963+0000 7efff9767b80 1 -- 172.21.15.5:0/1618835455 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e449454a0 msgr2=0x559e44ac8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:19.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.963+0000 7efff9767b80 1 --2- 172.21.15.5:0/1618835455 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e449454a0 0x559e44ac8b80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7effcc0b76c0 tx=0x7effcc009710 comp rx=0 tx=0).stop 2024-05-09T00:58:19.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.966+0000 7effe92ee700 1 --2- 172.21.15.5:0/1407341833 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7effd00b29b0 0x7effd00b4ec0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7effe40759c0 tx=0x7effd400a770 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:58:19.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.967+0000 7efff9767b80 1 -- 172.21.15.5:0/1618835455 shutdown_connections 2024-05-09T00:58:19.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.967+0000 7efff9767b80 1 --2- 172.21.15.5:0/1618835455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e44ac2800 0x559e44ac2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:19.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.967+0000 7efff9767b80 1 --2- 172.21.15.5:0/1618835455 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559e44a2f160 0x559e44ac8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:19.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.967+0000 7efff9767b80 1 --2- 172.21.15.5:0/1618835455 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e449454a0 0x559e44ac8b80 secure :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7effcc0b76c0 tx=0x7effcc009710 comp rx=0 tx=0).stop 2024-05-09T00:58:19.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.967+0000 7efff9767b80 1 -- 172.21.15.5:0/1618835455 >> 172.21.15.5:0/1618835455 conn(0x559e449a56a0 msgr2=0x559e44abd620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:58:19.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.969+0000 7efff9767b80 1 -- 172.21.15.5:0/1618835455 shutdown_connections 2024-05-09T00:58:19.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.971+0000 7efff9767b80 1 -- 172.21.15.5:0/1618835455 wait complete. 2024-05-09T00:58:19.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.971+0000 7efff9767b80 1 -- start start 2024-05-09T00:58:19.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.972+0000 7efff9767b80 1 Processor -- start 2024-05-09T00:58:19.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.972+0000 7efff9767b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559e44a2f160 0x559e44ad1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:19.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.972+0000 7efff9767b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e44ac2800 0x559e44ad2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:19.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.972+0000 7efff9767b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e44acc250 0x559e44acc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:19.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.972+0000 7efff9767b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559e44860ba0 con 0x559e44ac2800 2024-05-09T00:58:19.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.972+0000 7efff9767b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x559e4499f150 con 0x559e44a2f160 2024-05-09T00:58:19.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.972+0000 7efff9767b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x559e44ac33b0 con 0x559e44acc250 2024-05-09T00:58:19.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.972+0000 7efff9767b80 10 client.0 _ll_register_callbacks cb 0x559e448566c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:58:19.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.976+0000 7effe92ee700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e44acc250 0x559e44acc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:19.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.979+0000 7effe92ee700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e44acc250 0x559e44acc660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:36472/0 (socket says 172.21.15.5:36472) 2024-05-09T00:58:19.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.979+0000 7effe92ee700 1 -- 172.21.15.5:0/2624092699 learned_addr learned my addr 172.21.15.5:0/2624092699 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:58:19.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.979+0000 7effe8aed700 1 --2- 172.21.15.5:0/2624092699 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559e44a2f160 0x559e44ad1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:19.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.980+0000 7effe3fff700 1 --2- 172.21.15.5:0/2624092699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e44ac2800 0x559e44ad2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:19.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.982+0000 7effe92ee700 1 -- 172.21.15.5:0/2624092699 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559e44a2f160 msgr2=0x559e44ad1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:58:19.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.982+0000 7effe92ee700 1 --2- 172.21.15.5:0/2624092699 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559e44a2f160 0x559e44ad1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:19.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.982+0000 7effe92ee700 1 -- 172.21.15.5:0/2624092699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e44ac2800 msgr2=0x559e44ad2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:58:19.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.982+0000 7effe92ee700 1 --2- 172.21.15.5:0/2624092699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e44ac2800 0x559e44ad2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:19.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.982+0000 7effe92ee700 1 -- 172.21.15.5:0/2624092699 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e44860490 con 0x559e44acc250 2024-05-09T00:58:19.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.992+0000 7effe92ee700 1 --2- 172.21.15.5:0/2624092699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e44acc250 0x559e44acc660 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7effd40129d0 tx=0x7effd4004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=387b6fb77dddc79 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:58:19.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.992+0000 7effc3fff700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-09T00:58:19.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.992+0000 7effc3fff700 1 -- 172.21.15.5:0/2624092699 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd4022070 con 0x559e44acc250 2024-05-09T00:58:19.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.992+0000 7efff9767b80 1 -- 172.21.15.5:0/2624092699 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effcc0bb640 con 0x559e44acc250 2024-05-09T00:58:19.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.993+0000 7effc3fff700 1 -- 172.21.15.5:0/2624092699 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7effd400ea40 con 0x559e44acc250 2024-05-09T00:58:19.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.995+0000 7effc3fff700 1 -- 172.21.15.5:0/2624092699 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7effd402d030 con 0x559e44acc250 2024-05-09T00:58:19.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.995+0000 7efff9767b80 10 client.26161 fetch_fsmap learned FSMap version 31 2024-05-09T00:58:19.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.995+0000 7efff9767b80 1 -- 172.21.15.5:0/2624092699 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559e44ad9fe0 con 0x559e44acc250 2024-05-09T00:58:19.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.998+0000 7effc3fff700 1 -- 172.21.15.5:0/2624092699 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effcc0bb640 con 0x559e44acc250 2024-05-09T00:58:19.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.998+0000 7effc3fff700 1 -- 172.21.15.5:0/2624092699 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7effd401e420 con 0x559e44acc250 2024-05-09T00:58:19.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.998+0000 7efff9767b80 10 client.26161 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:58:19.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.998+0000 7efff9767b80 10 client.26161 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-05-09T00:58:19.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.998+0000 7efff9767b80 20 client.26161 populate_metadata read hostname 'smithi005' 2024-05-09T00:58:19.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.998+0000 7efff9767b80 1 --2- 172.21.15.5:0/2624092699 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x559e44adc570 0x559e44ade9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:20.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.998+0000 7efff9767b80 4 client.26161 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:58:20.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:19.998+0000 7efff9767b80 1 -- 172.21.15.5:0/2624092699 --> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e449f29d0 con 0x559e44adc570 2024-05-09T00:58:20.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.010+0000 7effe8aed700 1 --2- 172.21.15.5:0/2624092699 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x559e44adc570 0x559e44ade9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:20.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.012+0000 7effe8aed700 1 --2- 172.21.15.5:0/2624092699 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x559e44adc570 0x559e44ade9e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=511eaf4b2693667c server_cookie=ce46f3166eea2c15 in_seq=0 out_seq=0 2024-05-09T00:58:20.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.013+0000 7effc3fff700 10 client.26161 ms_handle_connect on v2:172.21.15.157:6836/744863009 2024-05-09T00:58:20.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.015+0000 7effc3fff700 1 -- 172.21.15.5:0/2624092699 <== mds.1 v2:172.21.15.157:6836/744863009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559e449f29d0 con 0x559e44adc570 2024-05-09T00:58:20.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.015+0000 7effc3fff700 10 client.26161 handle_command_reply: tid=0 2024-05-09T00:58:20.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.015+0000 7efff9767b80 1 client.26161 shutdown 2024-05-09T00:58:20.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.016+0000 7efff9767b80 1 -- 172.21.15.5:0/2624092699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e44acc250 msgr2=0x559e44acc660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:20.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.016+0000 7efff9767b80 1 --2- 172.21.15.5:0/2624092699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e44acc250 0x559e44acc660 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7effd40129d0 tx=0x7effd4004750 comp rx=0 tx=0).stop 2024-05-09T00:58:20.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.016+0000 7efff9767b80 1 -- 172.21.15.5:0/2624092699 shutdown_connections 2024-05-09T00:58:20.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.016+0000 7efff9767b80 1 --2- 172.21.15.5:0/2624092699 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x559e44adc570 0x559e44ade9e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:20.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.016+0000 7efff9767b80 1 --2- 172.21.15.5:0/2624092699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e44acc250 0x559e44acc660 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:20.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.016+0000 7efff9767b80 1 --2- 172.21.15.5:0/2624092699 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559e44a2f160 0x559e44ad1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:20.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.016+0000 7efff9767b80 1 --2- 172.21.15.5:0/2624092699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e44ac2800 0x559e44ad2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:20.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.016+0000 7efff9767b80 1 -- 172.21.15.5:0/2624092699 >> 172.21.15.5:0/2624092699 conn(0x559e449a56a0 msgr2=0x559e44abd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:58:20.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.017+0000 7effc3fff700 0 client.26161 ms_handle_reset on v2:172.21.15.157:6836/744863009 2024-05-09T00:58:20.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.017+0000 7efff9767b80 1 -- 172.21.15.5:0/2624092699 shutdown_connections 2024-05-09T00:58:20.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.019+0000 7efff9767b80 1 -- 172.21.15.5:0/2624092699 wait complete. 2024-05-09T00:58:20.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.023+0000 7efff9767b80 20 client.26161 trim_cache size 0 max 16384 2024-05-09T00:58:20.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.060+0000 7efff9767b80 1 Processor -- start 2024-05-09T00:58:20.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.060+0000 7efff9767b80 1 -- start start 2024-05-09T00:58:20.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.061+0000 7efff9767b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559e44a2f160 0x559e44acc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:20.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.061+0000 7efff9767b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e44ac2800 0x559e44ae4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:20.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.061+0000 7efff9767b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e44acc250 0x559e44ae4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:20.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.061+0000 7efff9767b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7effc400d4d0 con 0x559e44acc250 2024-05-09T00:58:20.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.061+0000 7efff9767b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7effc400f010 con 0x559e44a2f160 2024-05-09T00:58:20.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.061+0000 7efff9767b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7effc400dba0 con 0x559e44ac2800 2024-05-09T00:58:20.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.063+0000 7effe8aed700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559e44a2f160 0x559e44acc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:20.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.063+0000 7effe92ee700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e44acc250 0x559e44ae4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:20.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.063+0000 7effe92ee700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e44acc250 0x559e44ae4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33002/0 (socket says 172.21.15.5:33002) 2024-05-09T00:58:20.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.063+0000 7effe92ee700 1 -- 172.21.15.5:0/3504694465 learned_addr learned my addr 172.21.15.5:0/3504694465 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:58:20.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.063+0000 7effe3fff700 1 --2- 172.21.15.5:0/3504694465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e44ac2800 0x559e44ae4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:20.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.065+0000 7effe92ee700 1 -- 172.21.15.5:0/3504694465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e44ac2800 msgr2=0x559e44ae4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:58:20.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.065+0000 7effe92ee700 1 --2- 172.21.15.5:0/3504694465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e44ac2800 0x559e44ae4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:20.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.065+0000 7effe92ee700 1 -- 172.21.15.5:0/3504694465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559e44a2f160 msgr2=0x559e44acc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:58:20.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.065+0000 7effe92ee700 1 --2- 172.21.15.5:0/3504694465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559e44a2f160 0x559e44acc660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:20.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.065+0000 7effe92ee700 1 -- 172.21.15.5:0/3504694465 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e44ab8410 con 0x559e44acc250 2024-05-09T00:58:20.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.065+0000 7effe8aed700 1 --2- 172.21.15.5:0/3504694465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559e44a2f160 0x559e44acc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:58:20.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.067+0000 7effe92ee700 1 --2- 172.21.15.5:0/3504694465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e44acc250 0x559e44ae4770 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7effd400a470 tx=0x7effd4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e876d4b4013abb4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:58:20.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.068+0000 7effda7fc700 1 -- 172.21.15.5:0/3504694465 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd403f070 con 0x559e44acc250 2024-05-09T00:58:20.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.069+0000 7effe3fff700 1 --2- 172.21.15.5:0/3504694465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e44ac2800 0x559e44ae4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:58:20.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.069+0000 7effda7fc700 1 -- 172.21.15.5:0/3504694465 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effd401e420 con 0x559e44acc250 2024-05-09T00:58:20.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.070+0000 7efff9767b80 1 -- 172.21.15.5:0/3504694465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e44acc250 msgr2=0x559e44ae4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:20.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.070+0000 7efff9767b80 1 --2- 172.21.15.5:0/3504694465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e44acc250 0x559e44ae4770 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7effd400a470 tx=0x7effd4011ad0 comp rx=0 tx=0).stop 2024-05-09T00:58:20.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.071+0000 7efff9767b80 1 -- 172.21.15.5:0/3504694465 shutdown_connections 2024-05-09T00:58:20.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.071+0000 7efff9767b80 1 --2- 172.21.15.5:0/3504694465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e44acc250 0x559e44ae4770 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:20.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.071+0000 7efff9767b80 1 --2- 172.21.15.5:0/3504694465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559e44a2f160 0x559e44acc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:20.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.071+0000 7efff9767b80 1 --2- 172.21.15.5:0/3504694465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e44ac2800 0x559e44ae4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:20.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.071+0000 7efff9767b80 1 -- 172.21.15.5:0/3504694465 >> 172.21.15.5:0/3504694465 conn(0x559e449a56a0 msgr2=0x559e44acab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:58:20.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.074+0000 7efff9767b80 1 -- 172.21.15.5:0/3504694465 shutdown_connections 2024-05-09T00:58:20.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.077+0000 7efff9767b80 1 -- 172.21.15.5:0/3504694465 wait complete. 2024-05-09T00:58:20.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.077+0000 7efff9767b80 1 -- start start 2024-05-09T00:58:20.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.078+0000 7efff9767b80 1 Processor -- start 2024-05-09T00:58:20.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.079+0000 7efff9767b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e44a2f160 0x559e44ad5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:20.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.079+0000 7efff9767b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559e44ac2800 0x559e44ad55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:20.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.079+0000 7efff9767b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e44ad5ae0 0x559e44b44de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:20.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.079+0000 7efff9767b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7effc400e940 con 0x559e44a2f160 2024-05-09T00:58:20.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.079+0000 7efff9767b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7effc400e270 con 0x559e44ac2800 2024-05-09T00:58:20.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.079+0000 7efff9767b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7effc400b470 con 0x559e44ad5ae0 2024-05-09T00:58:20.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.079+0000 7efff9767b80 10 client.0 _ll_register_callbacks cb 0x559e448566c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:58:20.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.080+0000 7effe8aed700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e44a2f160 0x559e44ad5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:20.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.080+0000 7effe8aed700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e44a2f160 0x559e44ad5060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33008/0 (socket says 172.21.15.5:33008) 2024-05-09T00:58:20.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.080+0000 7effe8aed700 1 -- 172.21.15.5:0/2908145412 learned_addr learned my addr 172.21.15.5:0/2908145412 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:58:20.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.080+0000 7effe8aed700 1 -- 172.21.15.5:0/2908145412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e44ad5ae0 msgr2=0x559e44b44de0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:58:20.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.080+0000 7effe8aed700 1 --2- 172.21.15.5:0/2908145412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e44ad5ae0 0x559e44b44de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:20.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.080+0000 7effe8aed700 1 -- 172.21.15.5:0/2908145412 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559e44ac2800 msgr2=0x559e44ad55a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:58:20.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.080+0000 7effe8aed700 1 --2- 172.21.15.5:0/2908145412 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559e44ac2800 0x559e44ad55a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:20.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.080+0000 7effe8aed700 1 -- 172.21.15.5:0/2908145412 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e44b45350 con 0x559e44a2f160 2024-05-09T00:58:20.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.081+0000 7effe8aed700 1 --2- 172.21.15.5:0/2908145412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e44a2f160 0x559e44ad5060 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7effe406bbc0 tx=0x7effdc00a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=451e672b099620ed server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:58:20.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.083+0000 7effc3fff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:58:20.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.083+0000 7effc3fff700 1 -- 172.21.15.5:0/2908145412 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effdc01d150 con 0x559e44a2f160 2024-05-09T00:58:20.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.084+0000 7effc3fff700 1 -- 172.21.15.5:0/2908145412 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7effdc01d2f0 con 0x559e44a2f160 2024-05-09T00:58:20.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.087+0000 7efff9767b80 1 -- 172.21.15.5:0/2908145412 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559e449eca70 con 0x559e44a2f160 2024-05-09T00:58:20.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.088+0000 7effc3fff700 1 -- 172.21.15.5:0/2908145412 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7effdc01d4f0 con 0x559e44a2f160 2024-05-09T00:58:20.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.089+0000 7efff9767b80 10 client.16758 fetch_fsmap learned FSMap version 31 2024-05-09T00:58:20.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.089+0000 7efff9767b80 1 -- 172.21.15.5:0/2908145412 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559e44b46580 con 0x559e44a2f160 2024-05-09T00:58:20.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.093+0000 7effc3fff700 1 -- 172.21.15.5:0/2908145412 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x559e449eca70 con 0x559e44a2f160 2024-05-09T00:58:20.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.093+0000 7effc3fff700 1 -- 172.21.15.5:0/2908145412 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7effdc019b80 con 0x559e44a2f160 2024-05-09T00:58:20.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.093+0000 7efff9767b80 10 client.16758 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:58:20.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.093+0000 7efff9767b80 10 client.16758 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-05-09T00:58:20.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.093+0000 7efff9767b80 20 client.16758 populate_metadata read hostname 'smithi005' 2024-05-09T00:58:20.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.093+0000 7efff9767b80 1 --2- 172.21.15.5:0/2908145412 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x559e44b46cc0 0x559e44b490e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:20.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.093+0000 7efff9767b80 4 client.16758 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:58:20.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.093+0000 7efff9767b80 1 -- 172.21.15.5:0/2908145412 --> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559e44860970 con 0x559e44b46cc0 2024-05-09T00:58:20.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.099+0000 7effe3fff700 1 --2- 172.21.15.5:0/2908145412 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x559e44b46cc0 0x559e44b490e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:20.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:58:19 smithi131 ceph-mon[34547]: pgmap v730: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 856 MiB/s wr, 1.96k op/s 2024-05-09T00:58:20.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.105+0000 7effc3fff700 10 client.16758 ms_handle_connect on v2:172.21.15.157:6836/744863009 2024-05-09T00:58:20.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.105+0000 7effe3fff700 1 --2- 172.21.15.5:0/2908145412 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x559e44b46cc0 0x559e44b490e0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9dbe46a5f72d20d server_cookie=7a00429e662c22b4 in_seq=0 out_seq=0 2024-05-09T00:58:20.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.107+0000 7effc3fff700 1 -- 172.21.15.5:0/2908145412 <== mds.1 v2:172.21.15.157:6836/744863009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559e44860970 con 0x559e44b46cc0 2024-05-09T00:58:20.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.107+0000 7effc3fff700 10 client.16758 handle_command_reply: tid=0 2024-05-09T00:58:20.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.107+0000 7efff9767b80 1 client.16758 shutdown 2024-05-09T00:58:20.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.114+0000 7efff9767b80 1 -- 172.21.15.5:0/2908145412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e44a2f160 msgr2=0x559e44ad5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:20.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.114+0000 7efff9767b80 1 --2- 172.21.15.5:0/2908145412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e44a2f160 0x559e44ad5060 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7effe406bbc0 tx=0x7effdc00a8e0 comp rx=0 tx=0).stop 2024-05-09T00:58:20.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.117+0000 7efff9767b80 1 -- 172.21.15.5:0/2908145412 shutdown_connections 2024-05-09T00:58:20.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.117+0000 7efff9767b80 1 --2- 172.21.15.5:0/2908145412 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x559e44b46cc0 0x559e44b490e0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:20.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.117+0000 7efff9767b80 1 --2- 172.21.15.5:0/2908145412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e44ad5ae0 0x559e44b44de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:20.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.117+0000 7efff9767b80 1 --2- 172.21.15.5:0/2908145412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e44a2f160 0x559e44ad5060 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:20.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.117+0000 7efff9767b80 1 --2- 172.21.15.5:0/2908145412 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559e44ac2800 0x559e44ad55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:20.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.117+0000 7effc3fff700 0 client.16758 ms_handle_reset on v2:172.21.15.157:6836/744863009 2024-05-09T00:58:20.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.117+0000 7efff9767b80 1 -- 172.21.15.5:0/2908145412 >> 172.21.15.5:0/2908145412 conn(0x559e449a56a0 msgr2=0x559e44ac60d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:58:20.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.117+0000 7efff9767b80 1 -- 172.21.15.5:0/2908145412 shutdown_connections 2024-05-09T00:58:20.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.121+0000 7efff9767b80 1 -- 172.21.15.5:0/2908145412 wait complete. 2024-05-09T00:58:20.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.121+0000 7efff9767b80 20 client.16758 trim_cache size 0 max 16384 2024-05-09T00:58:20.124 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-09T00:58:20.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.124+0000 7effe9aef700 1 -- 172.21.15.5:0/1407341833 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7effd00b29b0 msgr2=0x7effd00b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:20.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.124+0000 7effe9aef700 1 --2- 172.21.15.5:0/1407341833 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7effd00b29b0 0x7effd00b4ec0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7effe40759c0 tx=0x7effd400a770 comp rx=0 tx=0).stop 2024-05-09T00:58:20.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.124+0000 7effe9aef700 1 -- 172.21.15.5:0/1407341833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7effe4110ab0 msgr2=0x7effe406deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:20.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.124+0000 7effe9aef700 1 --2- 172.21.15.5:0/1407341833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7effe4110ab0 0x7effe406deb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7effcc000c00 tx=0x7effcc0057c0 comp rx=0 tx=0).stop 2024-05-09T00:58:20.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.124+0000 7effe9aef700 1 -- 172.21.15.5:0/1407341833 shutdown_connections 2024-05-09T00:58:20.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.124+0000 7effe9aef700 1 --2- 172.21.15.5:0/1407341833 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7effd00b29b0 0x7effd00b4ec0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:20.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.124+0000 7effe9aef700 1 --2- 172.21.15.5:0/1407341833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7effe406e3f0 0x7effe41ad220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:20.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.124+0000 7effe9aef700 1 --2- 172.21.15.5:0/1407341833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effe4103850 0x7effe406d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:20.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.124+0000 7effe9aef700 1 --2- 172.21.15.5:0/1407341833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7effe4110ab0 0x7effe406deb0 secure :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7effcc000c00 tx=0x7effcc0057c0 comp rx=0 tx=0).stop 2024-05-09T00:58:20.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.124+0000 7effe9aef700 1 -- 172.21.15.5:0/1407341833 >> 172.21.15.5:0/1407341833 conn(0x7effe4078bf0 msgr2=0x7effe4111d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:58:20.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.128+0000 7effe9aef700 1 -- 172.21.15.5:0/1407341833 shutdown_connections 2024-05-09T00:58:20.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:20.128+0000 7effe9aef700 1 -- 172.21.15.5:0/1407341833 wait complete. 2024-05-09T00:58:21.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:58:21 smithi157 ceph-mon[34294]: pgmap v731: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 941 MiB/s wr, 1.99k op/s 2024-05-09T00:58:22.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:58:21 smithi005 ceph-mon[30365]: pgmap v731: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 941 MiB/s wr, 1.99k op/s 2024-05-09T00:58:22.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:58:21 smithi131 ceph-mon[34547]: pgmap v731: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 941 MiB/s wr, 1.99k op/s 2024-05-09T00:58:22.472 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T00:58:23.402 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:58:23.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:58:23 smithi157 ceph-mon[34294]: pgmap v732: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 851 MiB/s wr, 1.98k op/s 2024-05-09T00:58:24.046 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:58:23 smithi005 ceph-mon[30365]: pgmap v732: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 851 MiB/s wr, 1.98k op/s 2024-05-09T00:58:24.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:58:23 smithi131 ceph-mon[34547]: pgmap v732: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 851 MiB/s wr, 1.98k op/s 2024-05-09T00:58:25.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:58:25 smithi157 ceph-mon[34294]: pgmap v733: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 852 MiB/s wr, 2.10k op/s 2024-05-09T00:58:26.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:58:25 smithi005 ceph-mon[30365]: pgmap v733: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 852 MiB/s wr, 2.10k op/s 2024-05-09T00:58:26.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:58:25 smithi131 ceph-mon[34547]: pgmap v733: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 852 MiB/s wr, 2.10k op/s 2024-05-09T00:58:26.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:26.502+0000 7f8356ffd640 1 -- 172.21.15.5:0/2541958212 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7f836c0044b0 con 0x55dc757f51c0 2024-05-09T00:58:26.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:26.502+0000 7f8356ffd640 10 client.16743 handle_command_reply: tid=0 2024-05-09T00:58:26.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:26.502+0000 7f8385187740 1 client.16743 shutdown 2024-05-09T00:58:26.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:26.503+0000 7f8385187740 1 -- 172.21.15.5:0/2541958212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc758382f0 msgr2=0x55dc75c12e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:26.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:26.503+0000 7f8385187740 1 --2- 172.21.15.5:0/2541958212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc758382f0 0x55dc75c12e60 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f8370025a30 tx=0x7f8370007070 comp rx=0 tx=0).stop 2024-05-09T00:58:26.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:26.503+0000 7f8385187740 1 -- 172.21.15.5:0/2541958212 shutdown_connections 2024-05-09T00:58:26.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:26.503+0000 7f8385187740 1 --2- 172.21.15.5:0/2541958212 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55dc757f51c0 0x55dc75c86200 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:26.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:26.503+0000 7f8385187740 1 --2- 172.21.15.5:0/2541958212 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc758611b0 0x55dc75c162e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:26.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:26.503+0000 7f8385187740 1 --2- 172.21.15.5:0/2541958212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dc758404d0 0x55dc75c133a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:26.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:26.503+0000 7f8385187740 1 --2- 172.21.15.5:0/2541958212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc758382f0 0x55dc75c12e60 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:26.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:26.503+0000 7f8385187740 1 -- 172.21.15.5:0/2541958212 >> 172.21.15.5:0/2541958212 conn(0x55dc75855300 msgr2=0x55dc75c0d600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:58:26.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:26.503+0000 7f8356ffd640 0 client.16743 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:58:26.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:26.503+0000 7f8385187740 1 -- 172.21.15.5:0/2541958212 shutdown_connections 2024-05-09T00:58:26.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:26.503+0000 7f8385187740 1 -- 172.21.15.5:0/2541958212 wait complete. 2024-05-09T00:58:26.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:26.503+0000 7f8385187740 20 client.16743 trim_cache size 0 max 16384 2024-05-09T00:58:26.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-09T00:58:26.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-09T00:58:26.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-09T00:58:26.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:26.504+0000 7f837e1ff640 1 -- 172.21.15.5:0/3088868717 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f83600b2a00 msgr2=0x7f83600b4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:26.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:26.504+0000 7f837e1ff640 1 --2- 172.21.15.5:0/3088868717 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f83600b2a00 0x7f83600b4f10 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f8378109d90 tx=0x7f83700073c0 comp rx=0 tx=0).stop 2024-05-09T00:58:26.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:26.504+0000 7f837e1ff640 1 -- 172.21.15.5:0/3088868717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83781101a0 msgr2=0x7f83781b06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:26.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:26.504+0000 7f837e1ff640 1 --2- 172.21.15.5:0/3088868717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83781101a0 0x7f83781b06f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8368033040 tx=0x7f83680048f0 comp rx=0 tx=0).stop 2024-05-09T00:58:26.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:26.505+0000 7f837e1ff640 1 -- 172.21.15.5:0/3088868717 shutdown_connections 2024-05-09T00:58:26.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:26.505+0000 7f837e1ff640 1 --2- 172.21.15.5:0/3088868717 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f83600b2a00 0x7f83600b4f10 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:26.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:26.505+0000 7f837e1ff640 1 --2- 172.21.15.5:0/3088868717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83781101a0 0x7f83781b06f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:26.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:26.505+0000 7f837e1ff640 1 --2- 172.21.15.5:0/3088868717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f837810b480 0x7f83781a9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:26.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:26.505+0000 7f837e1ff640 1 --2- 172.21.15.5:0/3088868717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f837810a550 0x7f83781a9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:26.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:26.505+0000 7f837e1ff640 1 -- 172.21.15.5:0/3088868717 >> 172.21.15.5:0/3088868717 conn(0x7f8378070670 msgr2=0x7f837810c180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:58:26.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:26.505+0000 7f837e1ff640 1 -- 172.21.15.5:0/3088868717 shutdown_connections 2024-05-09T00:58:26.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:26.505+0000 7f837e1ff640 1 -- 172.21.15.5:0/3088868717 wait complete. 2024-05-09T00:58:27.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:58:27 smithi157 ceph-mon[34294]: pgmap v734: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 949 MiB/s wr, 2.44k op/s 2024-05-09T00:58:28.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:58:27 smithi005 ceph-mon[30365]: pgmap v734: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 949 MiB/s wr, 2.44k op/s 2024-05-09T00:58:28.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:58:27 smithi131 ceph-mon[34547]: pgmap v734: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 949 MiB/s wr, 2.44k op/s 2024-05-09T00:58:28.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.875+0000 7f0524d87700 1 -- 172.21.15.5:0/2735977879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0520106620 msgr2=0x7f0520106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:28.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.875+0000 7f0524d87700 1 --2- 172.21.15.5:0/2735977879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0520106620 0x7f0520106a30 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f0518009fc0 tx=0x7f051800b330 comp rx=0 tx=0).stop 2024-05-09T00:58:28.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.876+0000 7f0524d87700 1 -- 172.21.15.5:0/2735977879 shutdown_connections 2024-05-09T00:58:28.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.876+0000 7f0524d87700 1 --2- 172.21.15.5:0/2735977879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f052010b8a0 0x7f0520113200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:28.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.876+0000 7f0524d87700 1 --2- 172.21.15.5:0/2735977879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0520106620 0x7f0520106a30 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:28.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.876+0000 7f0524d87700 1 --2- 172.21.15.5:0/2735977879 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f052010af50 0x7f052010b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:28.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.876+0000 7f0524d87700 1 -- 172.21.15.5:0/2735977879 >> 172.21.15.5:0/2735977879 conn(0x7f0520102030 msgr2=0x7f0520104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:58:28.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.880+0000 7f0524d87700 1 -- 172.21.15.5:0/2735977879 shutdown_connections 2024-05-09T00:58:28.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.880+0000 7f0524d87700 1 -- 172.21.15.5:0/2735977879 wait complete. 2024-05-09T00:58:28.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.882+0000 7f0524d87700 1 Processor -- start 2024-05-09T00:58:28.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.887+0000 7f0524d87700 1 -- start start 2024-05-09T00:58:28.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.888+0000 7f0524d87700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0520106620 0x7f0520111f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:28.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.888+0000 7f0524d87700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f052010af50 0x7f0520112480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:28.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.888+0000 7f0524d87700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f052010b8a0 0x7f05201129c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:28.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.888+0000 7f0524d87700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0520078e00 con 0x7f052010b8a0 2024-05-09T00:58:28.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.888+0000 7f0524d87700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0520078c80 con 0x7f0520106620 2024-05-09T00:58:28.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.888+0000 7f0524d87700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0520078f80 con 0x7f052010af50 2024-05-09T00:58:28.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.892+0000 7f051effd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f052010af50 0x7f0520112480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:28.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.892+0000 7f051effd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f052010af50 0x7f0520112480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:38976/0 (socket says 172.21.15.5:38976) 2024-05-09T00:58:28.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.892+0000 7f051effd700 1 -- 172.21.15.5:0/3201986982 learned_addr learned my addr 172.21.15.5:0/3201986982 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:58:28.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.893+0000 7f051f7fe700 1 --2- 172.21.15.5:0/3201986982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0520106620 0x7f0520111f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:28.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.893+0000 7f051ffff700 1 --2- 172.21.15.5:0/3201986982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f052010b8a0 0x7f05201129c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:28.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.894+0000 7f051ffff700 1 -- 172.21.15.5:0/3201986982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f052010af50 msgr2=0x7f0520112480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:28.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.894+0000 7f051ffff700 1 --2- 172.21.15.5:0/3201986982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f052010af50 0x7f0520112480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:28.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.894+0000 7f051ffff700 1 -- 172.21.15.5:0/3201986982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0520106620 msgr2=0x7f0520111f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:28.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.894+0000 7f051ffff700 1 --2- 172.21.15.5:0/3201986982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0520106620 0x7f0520111f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:28.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.894+0000 7f051ffff700 1 -- 172.21.15.5:0/3201986982 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05201154a0 con 0x7f052010b8a0 2024-05-09T00:58:28.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.895+0000 7f051ffff700 1 --2- 172.21.15.5:0/3201986982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f052010b8a0 0x7f05201129c0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f051000d720 tx=0x7f051000da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:58:28.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.895+0000 7f051cff9700 1 -- 172.21.15.5:0/3201986982 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0510016070 con 0x7f052010b8a0 2024-05-09T00:58:28.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.895+0000 7f051cff9700 1 -- 172.21.15.5:0/3201986982 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0510005020 con 0x7f052010b8a0 2024-05-09T00:58:28.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.897+0000 7f051cff9700 1 -- 172.21.15.5:0/3201986982 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f051000fa50 con 0x7f052010b8a0 2024-05-09T00:58:28.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.900+0000 7f0524d87700 1 -- 172.21.15.5:0/3201986982 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0520115790 con 0x7f052010b8a0 2024-05-09T00:58:28.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.900+0000 7f0524d87700 1 -- 172.21.15.5:0/3201986982 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0520115c50 con 0x7f052010b8a0 2024-05-09T00:58:28.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.908+0000 7f05349ffb80 1 Processor -- start 2024-05-09T00:58:28.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.912+0000 7f05349ffb80 1 -- start start 2024-05-09T00:58:28.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.912+0000 7f05349ffb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55863d04c160 0x55863d0e5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:28.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.912+0000 7f05349ffb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55863cf624a0 0x55863d0e5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:28.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.912+0000 7f05349ffb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55863d0df800 0x55863d0dfc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:28.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.912+0000 7f05349ffb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55863d001ff0 con 0x55863d0df800 2024-05-09T00:58:28.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.912+0000 7f05349ffb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55863cfa07f0 con 0x55863d04c160 2024-05-09T00:58:28.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.912+0000 7f05349ffb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55863cffdfb0 con 0x55863cf624a0 2024-05-09T00:58:28.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.917+0000 7f051f7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55863cf624a0 0x55863d0e5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:28.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.917+0000 7f051effd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55863d04c160 0x55863d0e5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:28.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.917+0000 7f051effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55863d0df800 0x55863d0dfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:28.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.917+0000 7f051f7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55863cf624a0 0x55863d0e5c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:38992/0 (socket says 172.21.15.5:38992) 2024-05-09T00:58:28.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.917+0000 7f051f7fe700 1 -- 172.21.15.5:0/2653170170 learned_addr learned my addr 172.21.15.5:0/2653170170 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:58:28.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.922+0000 7f051f7fe700 1 -- 172.21.15.5:0/2653170170 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55863d04c160 msgr2=0x55863d0e5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:58:28.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.922+0000 7f051f7fe700 1 --2- 172.21.15.5:0/2653170170 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55863d04c160 0x55863d0e5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:28.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.922+0000 7f051f7fe700 1 -- 172.21.15.5:0/2653170170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55863d0df800 msgr2=0x55863d0dfc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:58:28.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.922+0000 7f051f7fe700 1 --2- 172.21.15.5:0/2653170170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55863d0df800 0x55863d0dfc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:28.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.922+0000 7f051f7fe700 1 -- 172.21.15.5:0/2653170170 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55863cf98370 con 0x55863cf624a0 2024-05-09T00:58:28.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.923+0000 7f051effd700 1 --2- 172.21.15.5:0/2653170170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55863d0df800 0x55863d0dfc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:58:28.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.923+0000 7f051cff9700 1 -- 172.21.15.5:0/3201986982 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f051000fbf0 con 0x7f052010b8a0 2024-05-09T00:58:28.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.925+0000 7f051cff9700 1 --2- 172.21.15.5:0/3201986982 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f05080b28e0 0x7f05080b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:28.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.925+0000 7f051cff9700 1 -- 172.21.15.5:0/3201986982 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f05100e1e50 con 0x7f052010b8a0 2024-05-09T00:58:28.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.927+0000 7f051effd700 1 --2- 172.21.15.5:0/3201986982 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f05080b28e0 0x7f05080b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:28.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.927+0000 7f051f7fe700 1 --2- 172.21.15.5:0/2653170170 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55863cf624a0 0x55863d0e5c80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0518015000 tx=0x7f0518009390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2ea45d8f979f69f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:58:28.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.930+0000 7f051effd700 1 --2- 172.21.15.5:0/3201986982 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f05080b28e0 0x7f05080b4df0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x55863d0e6310 tx=0x7f05140047c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:58:28.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.933+0000 7f0505ffb700 1 -- 172.21.15.5:0/2653170170 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0518003c00 con 0x55863cf624a0 2024-05-09T00:58:28.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.933+0000 7f0505ffb700 1 -- 172.21.15.5:0/2653170170 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0518003da0 con 0x55863cf624a0 2024-05-09T00:58:28.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.933+0000 7f0505ffb700 1 -- 172.21.15.5:0/2653170170 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0518007ce0 con 0x55863cf624a0 2024-05-09T00:58:28.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.935+0000 7f05349ffb80 1 -- 172.21.15.5:0/2653170170 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55863cf624a0 msgr2=0x55863d0e5c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:28.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.935+0000 7f05349ffb80 1 --2- 172.21.15.5:0/2653170170 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55863cf624a0 0x55863d0e5c80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0518015000 tx=0x7f0518009390 comp rx=0 tx=0).stop 2024-05-09T00:58:28.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.937+0000 7f05349ffb80 1 -- 172.21.15.5:0/2653170170 shutdown_connections 2024-05-09T00:58:28.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.937+0000 7f05349ffb80 1 --2- 172.21.15.5:0/2653170170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55863d0df800 0x55863d0dfc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:28.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.937+0000 7f05349ffb80 1 --2- 172.21.15.5:0/2653170170 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55863d04c160 0x55863d0e5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:28.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.937+0000 7f05349ffb80 1 --2- 172.21.15.5:0/2653170170 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55863cf624a0 0x55863d0e5c80 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:28.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.937+0000 7f05349ffb80 1 -- 172.21.15.5:0/2653170170 >> 172.21.15.5:0/2653170170 conn(0x55863cfc26a0 msgr2=0x55863d0da6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:58:28.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.938+0000 7f05349ffb80 1 -- 172.21.15.5:0/2653170170 shutdown_connections 2024-05-09T00:58:28.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.939+0000 7f05349ffb80 1 -- 172.21.15.5:0/2653170170 wait complete. 2024-05-09T00:58:28.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.940+0000 7f05349ffb80 1 -- start start 2024-05-09T00:58:28.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.940+0000 7f05349ffb80 1 Processor -- start 2024-05-09T00:58:28.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.941+0000 7f05349ffb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55863cf624a0 0x55863d0e9460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:28.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.942+0000 7f05349ffb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55863d04c160 0x55863d0e99a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:28.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.942+0000 7f05349ffb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55863d0df800 0x55863d0e9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:28.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.942+0000 7f05349ffb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55863ce7dba0 con 0x55863cf624a0 2024-05-09T00:58:28.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.942+0000 7f05349ffb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55863cfbc150 con 0x55863d0df800 2024-05-09T00:58:28.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.942+0000 7f05349ffb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55863d0e0660 con 0x55863d04c160 2024-05-09T00:58:28.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.942+0000 7f05349ffb80 10 client.0 _ll_register_callbacks cb 0x55863ce736c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:58:28.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.943+0000 7f051f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55863cf624a0 0x55863d0e9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:28.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.943+0000 7f051f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55863cf624a0 0x55863d0e9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35754/0 (socket says 172.21.15.5:35754) 2024-05-09T00:58:28.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.943+0000 7f051f7fe700 1 -- 172.21.15.5:0/3355715007 learned_addr learned my addr 172.21.15.5:0/3355715007 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:58:28.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.943+0000 7f051effd700 1 --2- 172.21.15.5:0/3355715007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55863d04c160 0x55863d0e99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:28.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.945+0000 7f051f7fe700 1 -- 172.21.15.5:0/3355715007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55863d04c160 msgr2=0x55863d0e99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:58:28.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.945+0000 7f051f7fe700 1 --2- 172.21.15.5:0/3355715007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55863d04c160 0x55863d0e99a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:28.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.945+0000 7f051f7fe700 1 -- 172.21.15.5:0/3355715007 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55863d0df800 msgr2=0x55863d0e9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:58:28.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.945+0000 7f051f7fe700 1 --2- 172.21.15.5:0/3355715007 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55863d0df800 0x55863d0e9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:28.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.945+0000 7f051f7fe700 1 -- 172.21.15.5:0/3355715007 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55863ce7d490 con 0x55863cf624a0 2024-05-09T00:58:28.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.947+0000 7f051f7fe700 1 --2- 172.21.15.5:0/3355715007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55863cf624a0 0x55863d0e9460 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f051800b6c0 tx=0x7f051800bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b16c74ec4931950c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:58:28.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.948+0000 7f04feffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:58:28.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.948+0000 7f04feffd700 1 -- 172.21.15.5:0/3355715007 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f051800bd90 con 0x55863cf624a0 2024-05-09T00:58:28.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.948+0000 7f05349ffb80 1 -- 172.21.15.5:0/3355715007 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0518003da0 con 0x55863cf624a0 2024-05-09T00:58:28.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.948+0000 7f04feffd700 1 -- 172.21.15.5:0/3355715007 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f0518007960 con 0x55863cf624a0 2024-05-09T00:58:28.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.949+0000 7f04feffd700 1 -- 172.21.15.5:0/3355715007 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0518007b60 con 0x55863cf624a0 2024-05-09T00:58:28.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.949+0000 7f05349ffb80 10 client.16770 fetch_fsmap learned FSMap version 31 2024-05-09T00:58:28.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.949+0000 7f05349ffb80 1 -- 172.21.15.5:0/3355715007 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55863d0f6dc0 con 0x55863cf624a0 2024-05-09T00:58:28.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.952+0000 7f04feffd700 1 -- 172.21.15.5:0/3355715007 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0518003da0 con 0x55863cf624a0 2024-05-09T00:58:28.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.952+0000 7f04feffd700 1 -- 172.21.15.5:0/3355715007 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f0518011070 con 0x55863cf624a0 2024-05-09T00:58:28.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.952+0000 7f05349ffb80 10 client.16770 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:58:28.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.952+0000 7f05349ffb80 10 client.16770 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:58:28.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.952+0000 7f05349ffb80 20 client.16770 populate_metadata read hostname 'smithi005' 2024-05-09T00:58:28.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.952+0000 7f05349ffb80 1 --2- 172.21.15.5:0/3355715007 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55863d0f9330 0x55863d0fb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:28.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.952+0000 7f05349ffb80 4 client.16770 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:58:28.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.952+0000 7f05349ffb80 1 -- 172.21.15.5:0/3355715007 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55863d00f9d0 con 0x55863d0f9330 2024-05-09T00:58:28.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.955+0000 7f051effd700 1 --2- 172.21.15.5:0/3355715007 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55863d0f9330 0x55863d0fb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:28.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.969+0000 7f051effd700 1 --2- 172.21.15.5:0/3355715007 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55863d0f9330 0x55863d0fb800 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ffdaebf6fc1da81 server_cookie=f62e44cc56985745 in_seq=0 out_seq=0 2024-05-09T00:58:28.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.969+0000 7f04feffd700 10 client.16770 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:58:28.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.975+0000 7f04feffd700 1 -- 172.21.15.5:0/3355715007 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55863d00f9d0 con 0x55863d0f9330 2024-05-09T00:58:28.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.975+0000 7f04feffd700 10 client.16770 handle_command_reply: tid=0 2024-05-09T00:58:28.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.977+0000 7f05349ffb80 1 client.16770 shutdown 2024-05-09T00:58:28.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.980+0000 7f05349ffb80 1 -- 172.21.15.5:0/3355715007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55863cf624a0 msgr2=0x55863d0e9460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:28.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.980+0000 7f05349ffb80 1 --2- 172.21.15.5:0/3355715007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55863cf624a0 0x55863d0e9460 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f051800b6c0 tx=0x7f051800bc20 comp rx=0 tx=0).stop 2024-05-09T00:58:28.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.981+0000 7f05349ffb80 1 -- 172.21.15.5:0/3355715007 shutdown_connections 2024-05-09T00:58:28.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.981+0000 7f05349ffb80 1 --2- 172.21.15.5:0/3355715007 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55863d0f9330 0x55863d0fb800 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:28.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.981+0000 7f05349ffb80 1 --2- 172.21.15.5:0/3355715007 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55863d0df800 0x55863d0e9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:28.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.981+0000 7f05349ffb80 1 --2- 172.21.15.5:0/3355715007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55863cf624a0 0x55863d0e9460 secure :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f051800b6c0 tx=0x7f051800bc20 comp rx=0 tx=0).stop 2024-05-09T00:58:28.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.981+0000 7f05349ffb80 1 --2- 172.21.15.5:0/3355715007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55863d04c160 0x55863d0e99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:28.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.981+0000 7f05349ffb80 1 -- 172.21.15.5:0/3355715007 >> 172.21.15.5:0/3355715007 conn(0x55863cfc26a0 msgr2=0x55863d0da3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:58:28.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.984+0000 7f04feffd700 0 client.16770 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:58:28.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.986+0000 7f05349ffb80 1 -- 172.21.15.5:0/3355715007 shutdown_connections 2024-05-09T00:58:28.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.986+0000 7f05349ffb80 1 -- 172.21.15.5:0/3355715007 wait complete. 2024-05-09T00:58:28.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:28.986+0000 7f05349ffb80 20 client.16770 trim_cache size 0 max 16384 2024-05-09T00:58:29.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.027+0000 7f05349ffb80 1 Processor -- start 2024-05-09T00:58:29.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.028+0000 7f05349ffb80 1 -- start start 2024-05-09T00:58:29.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.028+0000 7f05349ffb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55863d04c160 0x55863d0eb900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:29.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.028+0000 7f05349ffb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55863d0df800 0x55863d0e9d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:29.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.028+0000 7f05349ffb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55863d0f51b0 0x55863d0e4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:29.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.029+0000 7f05349ffb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f052011cf90 con 0x55863d04c160 2024-05-09T00:58:29.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.029+0000 7f05349ffb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f05201c79c0 con 0x55863d0f51b0 2024-05-09T00:58:29.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.029+0000 7f05349ffb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f052011d660 con 0x55863d0df800 2024-05-09T00:58:29.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.029+0000 7f051f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55863d04c160 0x55863d0eb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:29.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.029+0000 7f051f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55863d04c160 0x55863d0eb900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35762/0 (socket says 172.21.15.5:35762) 2024-05-09T00:58:29.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.029+0000 7f051f7fe700 1 -- 172.21.15.5:0/3323730324 learned_addr learned my addr 172.21.15.5:0/3323730324 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:58:29.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.030+0000 7f051f7fe700 1 -- 172.21.15.5:0/3323730324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55863d0df800 msgr2=0x55863d0e9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:58:29.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.030+0000 7f051f7fe700 1 --2- 172.21.15.5:0/3323730324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55863d0df800 0x55863d0e9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:29.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.030+0000 7f051ffff700 1 --2- 172.21.15.5:0/3323730324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55863d0f51b0 0x55863d0e4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:29.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.030+0000 7f051f7fe700 1 -- 172.21.15.5:0/3323730324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55863d0f51b0 msgr2=0x55863d0e4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:58:29.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.030+0000 7f051f7fe700 1 --2- 172.21.15.5:0/3323730324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55863d0f51b0 0x55863d0e4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:29.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.030+0000 7f051f7fe700 1 -- 172.21.15.5:0/3323730324 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55863d0e6110 con 0x55863d04c160 2024-05-09T00:58:29.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.030+0000 7f051f7fe700 1 --2- 172.21.15.5:0/3323730324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55863d04c160 0x55863d0eb900 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f051800d010 tx=0x7f05180021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=784d5390d81f1e8f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:58:29.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.031+0000 7f05057fa700 1 -- 172.21.15.5:0/3323730324 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0518007930 con 0x55863d04c160 2024-05-09T00:58:29.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.031+0000 7f05057fa700 1 -- 172.21.15.5:0/3323730324 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f051803b070 con 0x55863d04c160 2024-05-09T00:58:29.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.031+0000 7f05057fa700 1 -- 172.21.15.5:0/3323730324 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f051800bda0 con 0x55863d04c160 2024-05-09T00:58:29.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.033+0000 7f05349ffb80 1 -- 172.21.15.5:0/3323730324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55863d04c160 msgr2=0x55863d0eb900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:29.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.033+0000 7f05349ffb80 1 --2- 172.21.15.5:0/3323730324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55863d04c160 0x55863d0eb900 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f051800d010 tx=0x7f05180021f0 comp rx=0 tx=0).stop 2024-05-09T00:58:29.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.036+0000 7f05349ffb80 1 -- 172.21.15.5:0/3323730324 shutdown_connections 2024-05-09T00:58:29.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.036+0000 7f05349ffb80 1 --2- 172.21.15.5:0/3323730324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55863d0f51b0 0x55863d0e4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:29.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.037+0000 7f05349ffb80 1 --2- 172.21.15.5:0/3323730324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55863d04c160 0x55863d0eb900 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:29.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.037+0000 7f05349ffb80 1 --2- 172.21.15.5:0/3323730324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55863d0df800 0x55863d0e9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:29.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.037+0000 7f05349ffb80 1 -- 172.21.15.5:0/3323730324 >> 172.21.15.5:0/3323730324 conn(0x55863cfc26a0 msgr2=0x55863d0e7d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:58:29.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.039+0000 7f05349ffb80 1 -- 172.21.15.5:0/3323730324 shutdown_connections 2024-05-09T00:58:29.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.039+0000 7f05349ffb80 1 -- 172.21.15.5:0/3323730324 wait complete. 2024-05-09T00:58:29.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.040+0000 7f05349ffb80 1 -- start start 2024-05-09T00:58:29.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.040+0000 7f05349ffb80 1 Processor -- start 2024-05-09T00:58:29.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.041+0000 7f05349ffb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55863d04c160 0x55863d0f1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:29.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.042+0000 7f05349ffb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55863d0df800 0x55863d0fffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:29.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.042+0000 7f05349ffb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55863d0f1d30 0x55863d100570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:29.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.042+0000 7f05349ffb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f05201c72f0 con 0x55863d0df800 2024-05-09T00:58:29.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.043+0000 7f051f7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55863d04c160 0x55863d0f1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:29.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.043+0000 7f05349ffb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f05201c6c20 con 0x55863d0f1d30 2024-05-09T00:58:29.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.043+0000 7f05349ffb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f052011af30 con 0x55863d04c160 2024-05-09T00:58:29.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.043+0000 7f05349ffb80 10 client.0 _ll_register_callbacks cb 0x55863ce736c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:58:29.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.045+0000 7f051ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55863d0f1d30 0x55863d100570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:29.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.045+0000 7f051ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55863d0f1d30 0x55863d100570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:57724/0 (socket says 172.21.15.5:57724) 2024-05-09T00:58:29.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.045+0000 7f051ffff700 1 -- 172.21.15.5:0/93981696 learned_addr learned my addr 172.21.15.5:0/93981696 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:58:29.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.045+0000 7f051effd700 1 --2- 172.21.15.5:0/93981696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55863d0df800 0x55863d0fffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:29.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.047+0000 7f051effd700 1 -- 172.21.15.5:0/93981696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55863d04c160 msgr2=0x55863d0f1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:58:29.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.047+0000 7f051effd700 1 --2- 172.21.15.5:0/93981696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55863d04c160 0x55863d0f1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:29.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.048+0000 7f051effd700 1 -- 172.21.15.5:0/93981696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55863d0f1d30 msgr2=0x55863d100570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:58:29.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.048+0000 7f051effd700 1 --2- 172.21.15.5:0/93981696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55863d0f1d30 0x55863d100570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:29.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.048+0000 7f051effd700 1 -- 172.21.15.5:0/93981696 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55863d100d10 con 0x55863d0df800 2024-05-09T00:58:29.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.049+0000 7f051effd700 1 --2- 172.21.15.5:0/93981696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55863d0df800 0x55863d0fffb0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x55863d0fbf60 tx=0x7f0514009c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d33d44f9ea05476f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:58:29.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.050+0000 7f04feffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:58:29.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.050+0000 7f04feffd700 1 -- 172.21.15.5:0/93981696 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0514019040 con 0x55863d0df800 2024-05-09T00:58:29.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.050+0000 7f04feffd700 1 -- 172.21.15.5:0/93981696 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f0514004a30 con 0x55863d0df800 2024-05-09T00:58:29.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.050+0000 7f05349ffb80 1 -- 172.21.15.5:0/93981696 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f051803b070 con 0x55863d0df800 2024-05-09T00:58:29.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.053+0000 7f04feffd700 1 -- 172.21.15.5:0/93981696 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f051403d050 con 0x55863d0df800 2024-05-09T00:58:29.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.055+0000 7f05349ffb80 10 client.16776 fetch_fsmap learned FSMap version 31 2024-05-09T00:58:29.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.055+0000 7f05349ffb80 1 -- 172.21.15.5:0/93981696 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55863d162f50 con 0x55863d0df800 2024-05-09T00:58:29.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.057+0000 7f04feffd700 1 -- 172.21.15.5:0/93981696 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0514032070 con 0x55863d0df800 2024-05-09T00:58:29.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.057+0000 7f04feffd700 1 -- 172.21.15.5:0/93981696 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f051402e310 con 0x55863d0df800 2024-05-09T00:58:29.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.057+0000 7f05349ffb80 10 client.16776 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:58:29.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.057+0000 7f05349ffb80 10 client.16776 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:58:29.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.057+0000 7f05349ffb80 20 client.16776 populate_metadata read hostname 'smithi005' 2024-05-09T00:58:29.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.058+0000 7f05349ffb80 1 --2- 172.21.15.5:0/93981696 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55863d163610 0x55863d165a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:29.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.058+0000 7f05349ffb80 4 client.16776 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:58:29.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.058+0000 7f05349ffb80 1 -- 172.21.15.5:0/93981696 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f051803b070 con 0x55863d163610 2024-05-09T00:58:29.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.062+0000 7f051f7fe700 1 --2- 172.21.15.5:0/93981696 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55863d163610 0x55863d165a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:29.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.083+0000 7f04feffd700 10 client.16776 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:58:29.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.083+0000 7f051f7fe700 1 --2- 172.21.15.5:0/93981696 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55863d163610 0x55863d165a30 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d70a11b14ffa5ee0 server_cookie=2f976435ba0b0c14 in_seq=0 out_seq=0 2024-05-09T00:58:29.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.086+0000 7f04feffd700 1 -- 172.21.15.5:0/93981696 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f051803b070 con 0x55863d163610 2024-05-09T00:58:29.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.086+0000 7f04feffd700 10 client.16776 handle_command_reply: tid=0 2024-05-09T00:58:29.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.087+0000 7f05349ffb80 1 client.16776 shutdown 2024-05-09T00:58:29.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.088+0000 7f05349ffb80 1 -- 172.21.15.5:0/93981696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55863d0df800 msgr2=0x55863d0fffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:29.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.088+0000 7f05349ffb80 1 --2- 172.21.15.5:0/93981696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55863d0df800 0x55863d0fffb0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x55863d0fbf60 tx=0x7f0514009c90 comp rx=0 tx=0).stop 2024-05-09T00:58:29.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.089+0000 7f05349ffb80 1 -- 172.21.15.5:0/93981696 shutdown_connections 2024-05-09T00:58:29.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.089+0000 7f05349ffb80 1 --2- 172.21.15.5:0/93981696 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55863d163610 0x55863d165a30 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:29.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.089+0000 7f04feffd700 0 client.16776 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:58:29.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.089+0000 7f05349ffb80 1 --2- 172.21.15.5:0/93981696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55863d0f1d30 0x55863d100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:29.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.089+0000 7f05349ffb80 1 --2- 172.21.15.5:0/93981696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55863d04c160 0x55863d0f1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:29.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.089+0000 7f05349ffb80 1 --2- 172.21.15.5:0/93981696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55863d0df800 0x55863d0fffb0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:29.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.089+0000 7f05349ffb80 1 -- 172.21.15.5:0/93981696 >> 172.21.15.5:0/93981696 conn(0x55863cfc26a0 msgr2=0x55863d0e4570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:58:29.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.091+0000 7f05349ffb80 1 -- 172.21.15.5:0/93981696 shutdown_connections 2024-05-09T00:58:29.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.091+0000 7f05349ffb80 1 -- 172.21.15.5:0/93981696 wait complete. 2024-05-09T00:58:29.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.091+0000 7f05349ffb80 20 client.16776 trim_cache size 0 max 16384 2024-05-09T00:58:29.092 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-09T00:58:29.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.092+0000 7f0524d87700 1 -- 172.21.15.5:0/3201986982 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f05080b28e0 msgr2=0x7f05080b4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:29.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.092+0000 7f0524d87700 1 --2- 172.21.15.5:0/3201986982 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f05080b28e0 0x7f05080b4df0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x55863d0e6310 tx=0x7f05140047c0 comp rx=0 tx=0).stop 2024-05-09T00:58:29.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.092+0000 7f0524d87700 1 -- 172.21.15.5:0/3201986982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f052010b8a0 msgr2=0x7f05201129c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:29.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.092+0000 7f0524d87700 1 --2- 172.21.15.5:0/3201986982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f052010b8a0 0x7f05201129c0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f051000d720 tx=0x7f051000da30 comp rx=0 tx=0).stop 2024-05-09T00:58:29.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.093+0000 7f0524d87700 1 -- 172.21.15.5:0/3201986982 shutdown_connections 2024-05-09T00:58:29.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.093+0000 7f0524d87700 1 --2- 172.21.15.5:0/3201986982 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f05080b28e0 0x7f05080b4df0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:29.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.093+0000 7f0524d87700 1 --2- 172.21.15.5:0/3201986982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f052010b8a0 0x7f05201129c0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:29.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.093+0000 7f0524d87700 1 --2- 172.21.15.5:0/3201986982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0520106620 0x7f0520111f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:29.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.093+0000 7f0524d87700 1 --2- 172.21.15.5:0/3201986982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f052010af50 0x7f0520112480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:29.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.093+0000 7f0524d87700 1 -- 172.21.15.5:0/3201986982 >> 172.21.15.5:0/3201986982 conn(0x7f0520102030 msgr2=0x7f0520102440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:58:29.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.094+0000 7f0524d87700 1 -- 172.21.15.5:0/3201986982 shutdown_connections 2024-05-09T00:58:29.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:29.094+0000 7f0524d87700 1 -- 172.21.15.5:0/3201986982 wait complete. 2024-05-09T00:58:29.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:58:29 smithi157 ceph-mon[34294]: pgmap v735: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 868 MiB/s wr, 2.19k op/s 2024-05-09T00:58:30.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:58:29 smithi005 ceph-mon[30365]: pgmap v735: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 868 MiB/s wr, 2.19k op/s 2024-05-09T00:58:30.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:58:29 smithi131 ceph-mon[34547]: pgmap v735: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 868 MiB/s wr, 2.19k op/s 2024-05-09T00:58:31.662 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T00:58:31.662 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T00:58:32.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:58:31 smithi005 ceph-mon[30365]: pgmap v736: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 938 MiB/s wr, 2.35k op/s 2024-05-09T00:58:32.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:58:31 smithi131 ceph-mon[34547]: pgmap v736: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 938 MiB/s wr, 2.35k op/s 2024-05-09T00:58:32.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:58:31 smithi157 ceph-mon[34294]: pgmap v736: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 938 MiB/s wr, 2.35k op/s 2024-05-09T00:58:32.501 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:58:33.953 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:58:33 smithi005 ceph-mon[30365]: pgmap v737: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 792 MiB/s wr, 2.17k op/s 2024-05-09T00:58:34.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:58:33 smithi131 ceph-mon[34547]: pgmap v737: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 792 MiB/s wr, 2.17k op/s 2024-05-09T00:58:34.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:58:33 smithi157 ceph-mon[34294]: pgmap v737: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 792 MiB/s wr, 2.17k op/s 2024-05-09T00:58:35.988 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:58:35 smithi157 ceph-mon[34294]: pgmap v738: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 834 MiB/s wr, 2.31k op/s 2024-05-09T00:58:36.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:58:35 smithi005 ceph-mon[30365]: pgmap v738: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 834 MiB/s wr, 2.31k op/s 2024-05-09T00:58:36.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:58:35 smithi131 ceph-mon[34547]: pgmap v738: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 834 MiB/s wr, 2.31k op/s 2024-05-09T00:58:37.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:58:37 smithi005 ceph-mon[30365]: pgmap v739: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 936 MiB/s wr, 2.50k op/s 2024-05-09T00:58:38.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:58:37 smithi131 ceph-mon[34547]: pgmap v739: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 936 MiB/s wr, 2.50k op/s 2024-05-09T00:58:38.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:58:37 smithi157 ceph-mon[34294]: pgmap v739: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 936 MiB/s wr, 2.50k op/s 2024-05-09T00:58:38.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.841+0000 7feb9893a700 1 -- 172.21.15.5:0/3744965802 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb8c0b8320 msgr2=0x7feb8c0ba740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:38.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.841+0000 7feb9893a700 1 --2- 172.21.15.5:0/3744965802 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb8c0b8320 0x7feb8c0ba740 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7feb84009990 tx=0x7feb84009ca0 comp rx=0 tx=0).stop 2024-05-09T00:58:38.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.843+0000 7feb9893a700 1 -- 172.21.15.5:0/3744965802 shutdown_connections 2024-05-09T00:58:38.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.843+0000 7feb9893a700 1 --2- 172.21.15.5:0/3744965802 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb8c0b8320 0x7feb8c0ba740 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:38.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.843+0000 7feb9893a700 1 --2- 172.21.15.5:0/3744965802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb8c0a53c0 0x7feb8c0a57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:38.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.843+0000 7feb9893a700 1 --2- 172.21.15.5:0/3744965802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb8c0a5d10 0x7feb8c0b7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:38.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.843+0000 7feb9893a700 1 -- 172.21.15.5:0/3744965802 >> 172.21.15.5:0/3744965802 conn(0x7feb8c01af20 msgr2=0x7feb8c01b360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:58:38.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.849+0000 7feb9893a700 1 -- 172.21.15.5:0/3744965802 shutdown_connections 2024-05-09T00:58:38.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.852+0000 7feb9893a700 1 -- 172.21.15.5:0/3744965802 wait complete. 2024-05-09T00:58:38.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.853+0000 7feb9893a700 1 Processor -- start 2024-05-09T00:58:38.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.853+0000 7feb9893a700 1 -- start start 2024-05-09T00:58:38.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.853+0000 7feb9893a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb8c0a53c0 0x7feb8c0b6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:38.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.853+0000 7feb9893a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb8c0a5d10 0x7feb8c0af960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:38.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.853+0000 7feb9893a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb8c0b8320 0x7feb8c0b0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:38.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.853+0000 7feb9893a700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7feb8c0bde30 con 0x7feb8c0b8320 2024-05-09T00:58:38.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.853+0000 7feb9893a700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7feb8c0bdcb0 con 0x7feb8c0a53c0 2024-05-09T00:58:38.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.853+0000 7feb9893a700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7feb8c0bdfb0 con 0x7feb8c0a5d10 2024-05-09T00:58:38.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.854+0000 7feb92d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb8c0a53c0 0x7feb8c0b6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:38.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.854+0000 7feb9359e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb8c0b8320 0x7feb8c0b0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:38.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.854+0000 7feb9359e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb8c0b8320 0x7feb8c0b0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48106/0 (socket says 172.21.15.5:48106) 2024-05-09T00:58:38.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.854+0000 7feb9359e700 1 -- 172.21.15.5:0/2352592341 learned_addr learned my addr 172.21.15.5:0/2352592341 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:58:38.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.855+0000 7feb9359e700 1 -- 172.21.15.5:0/2352592341 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb8c0a5d10 msgr2=0x7feb8c0af960 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T00:58:38.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.855+0000 7feb9359e700 1 --2- 172.21.15.5:0/2352592341 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb8c0a5d10 0x7feb8c0af960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:38.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.855+0000 7feb9359e700 1 -- 172.21.15.5:0/2352592341 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb8c0a53c0 msgr2=0x7feb8c0b6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:38.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.855+0000 7feb9359e700 1 --2- 172.21.15.5:0/2352592341 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb8c0a53c0 0x7feb8c0b6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:38.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.855+0000 7feb9359e700 1 -- 172.21.15.5:0/2352592341 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb8c0b4360 con 0x7feb8c0b8320 2024-05-09T00:58:38.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.856+0000 7feb92d9d700 1 --2- 172.21.15.5:0/2352592341 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb8c0a53c0 0x7feb8c0b6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:58:38.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.856+0000 7feb9359e700 1 --2- 172.21.15.5:0/2352592341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb8c0b8320 0x7feb8c0b0020 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7feb84004790 tx=0x7feb840047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:58:38.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.857+0000 7feb7bfff700 1 -- 172.21.15.5:0/2352592341 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb84004830 con 0x7feb8c0b8320 2024-05-09T00:58:38.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.857+0000 7feb9893a700 1 -- 172.21.15.5:0/2352592341 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb8c0b45f0 con 0x7feb8c0b8320 2024-05-09T00:58:38.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.857+0000 7feb9893a700 1 -- 172.21.15.5:0/2352592341 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb8c0b4a40 con 0x7feb8c0b8320 2024-05-09T00:58:38.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.862+0000 7feb7bfff700 1 -- 172.21.15.5:0/2352592341 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb84005020 con 0x7feb8c0b8320 2024-05-09T00:58:38.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.862+0000 7feb7bfff700 1 -- 172.21.15.5:0/2352592341 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb84003f10 con 0x7feb8c0b8320 2024-05-09T00:58:38.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.863+0000 7feba7b51b80 1 Processor -- start 2024-05-09T00:58:38.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.867+0000 7feba7b51b80 1 -- start start 2024-05-09T00:58:38.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.868+0000 7feba7b51b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4d95471e0 0x55c4d95e06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:38.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.868+0000 7feba7b51b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4d945d520 0x55c4d95e0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:38.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.868+0000 7feba7b51b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4d95da880 0x55c4d95dac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:38.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.868+0000 7feba7b51b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c4d94fd070 con 0x55c4d95471e0 2024-05-09T00:58:38.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.868+0000 7feba7b51b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55c4d949b870 con 0x55c4d945d520 2024-05-09T00:58:38.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.868+0000 7feba7b51b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55c4d94f9030 con 0x55c4d95da880 2024-05-09T00:58:38.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.870+0000 7feb7bfff700 1 -- 172.21.15.5:0/2352592341 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7feb84028020 con 0x7feb8c0b8320 2024-05-09T00:58:38.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.870+0000 7feb9259c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4d95da880 0x55c4d95dac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:38.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.871+0000 7feb9259c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4d95da880 0x55c4d95dac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:51558/0 (socket says 172.21.15.5:51558) 2024-05-09T00:58:38.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.871+0000 7feb9259c700 1 -- 172.21.15.5:0/1269904838 learned_addr learned my addr 172.21.15.5:0/1269904838 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:58:38.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.872+0000 7feb9259c700 1 -- 172.21.15.5:0/1269904838 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4d945d520 msgr2=0x55c4d95e0c00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:58:38.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.872+0000 7feb9259c700 1 --2- 172.21.15.5:0/1269904838 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4d945d520 0x55c4d95e0c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:38.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.872+0000 7feb9259c700 1 -- 172.21.15.5:0/1269904838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4d95471e0 msgr2=0x55c4d95e06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:58:38.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.872+0000 7feb9259c700 1 --2- 172.21.15.5:0/1269904838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4d95471e0 0x55c4d95e06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:38.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.872+0000 7feb9259c700 1 -- 172.21.15.5:0/1269904838 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4d94933f0 con 0x55c4d95da880 2024-05-09T00:58:38.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.872+0000 7feb9259c700 1 --2- 172.21.15.5:0/1269904838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4d95da880 0x55c4d95dac90 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7feb8800bcf0 tx=0x7feb8800d440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=912246167dc8266b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:58:38.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.873+0000 7feb7bfff700 1 --2- 172.21.15.5:0/2352592341 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7feb7c0b2a90 0x7feb7c0b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:38.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.874+0000 7feb7bfff700 1 -- 172.21.15.5:0/2352592341 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7feb840e3eb0 con 0x7feb8c0b8320 2024-05-09T00:58:38.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.874+0000 7feb797fa700 1 -- 172.21.15.5:0/1269904838 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb88004030 con 0x55c4d95da880 2024-05-09T00:58:38.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.874+0000 7feb797fa700 1 -- 172.21.15.5:0/1269904838 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb880048b0 con 0x55c4d95da880 2024-05-09T00:58:38.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.874+0000 7feb797fa700 1 -- 172.21.15.5:0/1269904838 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb880065f0 con 0x55c4d95da880 2024-05-09T00:58:38.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.878+0000 7feb92d9d700 1 --2- 172.21.15.5:0/2352592341 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7feb7c0b2a90 0x7feb7c0b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:38.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.879+0000 7feb92d9d700 1 --2- 172.21.15.5:0/2352592341 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7feb7c0b2a90 0x7feb7c0b4fa0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7feb94067070 tx=0x7feb94064d40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:58:38.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.880+0000 7feba7b51b80 1 -- 172.21.15.5:0/1269904838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4d95da880 msgr2=0x55c4d95dac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:38.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.881+0000 7feba7b51b80 1 --2- 172.21.15.5:0/1269904838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4d95da880 0x55c4d95dac90 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7feb8800bcf0 tx=0x7feb8800d440 comp rx=0 tx=0).stop 2024-05-09T00:58:38.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.881+0000 7feba7b51b80 1 -- 172.21.15.5:0/1269904838 shutdown_connections 2024-05-09T00:58:38.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.881+0000 7feba7b51b80 1 --2- 172.21.15.5:0/1269904838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4d95da880 0x55c4d95dac90 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:38.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.881+0000 7feba7b51b80 1 --2- 172.21.15.5:0/1269904838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4d95471e0 0x55c4d95e06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:38.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.881+0000 7feba7b51b80 1 --2- 172.21.15.5:0/1269904838 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4d945d520 0x55c4d95e0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:38.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.881+0000 7feba7b51b80 1 -- 172.21.15.5:0/1269904838 >> 172.21.15.5:0/1269904838 conn(0x55c4d94bd720 msgr2=0x55c4d95d56a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:58:38.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.881+0000 7feba7b51b80 1 -- 172.21.15.5:0/1269904838 shutdown_connections 2024-05-09T00:58:38.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.883+0000 7feba7b51b80 1 -- 172.21.15.5:0/1269904838 wait complete. 2024-05-09T00:58:38.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.884+0000 7feba7b51b80 1 -- start start 2024-05-09T00:58:38.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.884+0000 7feba7b51b80 1 Processor -- start 2024-05-09T00:58:38.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.885+0000 7feba7b51b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4d945d520 0x55c4d95e4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:38.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.886+0000 7feba7b51b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4d95471e0 0x55c4d95e4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:38.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.886+0000 7feba7b51b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4d95da880 0x55c4d95e4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:38.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.886+0000 7feba7b51b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c4d9378c20 con 0x55c4d945d520 2024-05-09T00:58:38.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.886+0000 7feba7b51b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55c4d94b71d0 con 0x55c4d95471e0 2024-05-09T00:58:38.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.886+0000 7feba7b51b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55c4d95db670 con 0x55c4d95da880 2024-05-09T00:58:38.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.886+0000 7feb9259c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4d95471e0 0x55c4d95e4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:38.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.886+0000 7feb9259c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4d95471e0 0x55c4d95e4a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:48530/0 (socket says 172.21.15.5:48530) 2024-05-09T00:58:38.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.886+0000 7feb9259c700 1 -- 172.21.15.5:0/2066658663 learned_addr learned my addr 172.21.15.5:0/2066658663 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:58:38.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.887+0000 7feba7b51b80 10 client.0 _ll_register_callbacks cb 0x55c4d936e740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:58:38.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.887+0000 7feb9259c700 1 -- 172.21.15.5:0/2066658663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4d95da880 msgr2=0x55c4d95e4f80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:58:38.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.887+0000 7feb9359e700 1 --2- 172.21.15.5:0/2066658663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4d95da880 0x55c4d95e4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:38.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.887+0000 7feb9259c700 1 --2- 172.21.15.5:0/2066658663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4d95da880 0x55c4d95e4f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:38.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.887+0000 7feb9259c700 1 -- 172.21.15.5:0/2066658663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4d945d520 msgr2=0x55c4d95e4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:58:38.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.887+0000 7feb9259c700 1 --2- 172.21.15.5:0/2066658663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4d945d520 0x55c4d95e4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:38.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.887+0000 7feb9259c700 1 -- 172.21.15.5:0/2066658663 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4d9378510 con 0x55c4d95471e0 2024-05-09T00:58:38.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.888+0000 7feb9259c700 1 --2- 172.21.15.5:0/2066658663 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4d95471e0 0x55c4d95e4a40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7feb8800b440 tx=0x7feb880060a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6edc2d4f36d63fcb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:58:38.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.888+0000 7feb76ff5700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-09T00:58:38.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.889+0000 7feb76ff5700 1 -- 172.21.15.5:0/2066658663 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8801c050 con 0x55c4d95471e0 2024-05-09T00:58:38.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.889+0000 7feb76ff5700 1 -- 172.21.15.5:0/2066658663 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7feb88004400 con 0x55c4d95471e0 2024-05-09T00:58:38.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.889+0000 7feba7b51b80 1 -- 172.21.15.5:0/2066658663 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb880048b0 con 0x55c4d95471e0 2024-05-09T00:58:38.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.889+0000 7feb76ff5700 1 -- 172.21.15.5:0/2066658663 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7feb880061f0 con 0x55c4d95471e0 2024-05-09T00:58:38.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.891+0000 7feba7b51b80 10 client.26198 fetch_fsmap learned FSMap version 31 2024-05-09T00:58:38.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.891+0000 7feba7b51b80 1 -- 172.21.15.5:0/2066658663 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c4d95f1bb0 con 0x55c4d95471e0 2024-05-09T00:58:38.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.892+0000 7feb76ff5700 1 -- 172.21.15.5:0/2066658663 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb88003680 con 0x55c4d95471e0 2024-05-09T00:58:38.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.893+0000 7feb76ff5700 1 -- 172.21.15.5:0/2066658663 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7feb8801f070 con 0x55c4d95471e0 2024-05-09T00:58:38.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.894+0000 7feba7b51b80 10 client.26198 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:58:38.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.894+0000 7feba7b51b80 10 client.26198 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:58:38.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.894+0000 7feba7b51b80 20 client.26198 populate_metadata read hostname 'smithi005' 2024-05-09T00:58:38.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.894+0000 7feba7b51b80 1 --2- 172.21.15.5:0/2066658663 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55c4d95f43a0 0x55c4d95f6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:38.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.894+0000 7feba7b51b80 4 client.26198 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:58:38.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.894+0000 7feba7b51b80 1 -- 172.21.15.5:0/2066658663 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7feb880048b0 con 0x55c4d95f43a0 2024-05-09T00:58:38.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.897+0000 7feb92d9d700 1 --2- 172.21.15.5:0/2066658663 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55c4d95f43a0 0x55c4d95f6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:38.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.904+0000 7feb92d9d700 1 --2- 172.21.15.5:0/2066658663 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55c4d95f43a0 0x55c4d95f6810 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1cf6572e6e0a45b server_cookie=4593e60ecb7b39a8 in_seq=0 out_seq=0 2024-05-09T00:58:38.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.904+0000 7feb76ff5700 10 client.26198 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:58:38.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.912+0000 7feb76ff5700 1 -- 172.21.15.5:0/2066658663 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7feb880048b0 con 0x55c4d95f43a0 2024-05-09T00:58:38.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.912+0000 7feb76ff5700 10 client.26198 handle_command_reply: tid=0 2024-05-09T00:58:38.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.913+0000 7feba7b51b80 1 client.26198 shutdown 2024-05-09T00:58:38.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.913+0000 7feba7b51b80 1 -- 172.21.15.5:0/2066658663 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4d95471e0 msgr2=0x55c4d95e4a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:38.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.913+0000 7feba7b51b80 1 --2- 172.21.15.5:0/2066658663 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4d95471e0 0x55c4d95e4a40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7feb8800b440 tx=0x7feb880060a0 comp rx=0 tx=0).stop 2024-05-09T00:58:38.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.920+0000 7feba7b51b80 1 -- 172.21.15.5:0/2066658663 shutdown_connections 2024-05-09T00:58:38.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.920+0000 7feba7b51b80 1 --2- 172.21.15.5:0/2066658663 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55c4d95f43a0 0x55c4d95f6810 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:38.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.920+0000 7feba7b51b80 1 --2- 172.21.15.5:0/2066658663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4d95da880 0x55c4d95e4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:38.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.920+0000 7feba7b51b80 1 --2- 172.21.15.5:0/2066658663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4d945d520 0x55c4d95e4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:38.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.920+0000 7feba7b51b80 1 --2- 172.21.15.5:0/2066658663 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4d95471e0 0x55c4d95e4a40 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:38.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.920+0000 7feba7b51b80 1 -- 172.21.15.5:0/2066658663 >> 172.21.15.5:0/2066658663 conn(0x55c4d94bd720 msgr2=0x55c4d95d5400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:58:38.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.920+0000 7feb76ff5700 0 client.26198 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:58:38.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.922+0000 7feba7b51b80 1 -- 172.21.15.5:0/2066658663 shutdown_connections 2024-05-09T00:58:38.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.922+0000 7feba7b51b80 1 -- 172.21.15.5:0/2066658663 wait complete. 2024-05-09T00:58:38.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.922+0000 7feba7b51b80 20 client.26198 trim_cache size 0 max 16384 2024-05-09T00:58:38.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.974+0000 7feba7b51b80 1 Processor -- start 2024-05-09T00:58:38.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.974+0000 7feba7b51b80 1 -- start start 2024-05-09T00:58:38.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.975+0000 7feba7b51b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4d945d520 0x55c4d95f6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:38.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.975+0000 7feba7b51b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4d95471e0 0x55c4d95d9f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:38.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.975+0000 7feba7b51b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4d95da880 0x55c4d95f01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:38.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.975+0000 7feba7b51b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7feb8000d550 con 0x55c4d95471e0 2024-05-09T00:58:38.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.975+0000 7feb92d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4d945d520 0x55c4d95f6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:38.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.976+0000 7feb92d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4d945d520 0x55c4d95f6a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:48542/0 (socket says 172.21.15.5:48542) 2024-05-09T00:58:38.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.976+0000 7feb92d9d700 1 -- 172.21.15.5:0/1361578390 learned_addr learned my addr 172.21.15.5:0/1361578390 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:58:38.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.976+0000 7feba7b51b80 1 -- 172.21.15.5:0/1361578390 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7feb8000f090 con 0x55c4d945d520 2024-05-09T00:58:38.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.976+0000 7feba7b51b80 1 -- 172.21.15.5:0/1361578390 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7feb8000dc20 con 0x55c4d95da880 2024-05-09T00:58:38.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.978+0000 7feb9359e700 1 --2- 172.21.15.5:0/1361578390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4d95da880 0x55c4d95f01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:38.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.978+0000 7feb9259c700 1 --2- 172.21.15.5:0/1361578390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4d95471e0 0x55c4d95d9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:38.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.983+0000 7feb92d9d700 1 -- 172.21.15.5:0/1361578390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4d95da880 msgr2=0x55c4d95f01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:58:38.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.983+0000 7feb92d9d700 1 --2- 172.21.15.5:0/1361578390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4d95da880 0x55c4d95f01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:38.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.983+0000 7feb92d9d700 1 -- 172.21.15.5:0/1361578390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4d95471e0 msgr2=0x55c4d95d9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:58:38.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.983+0000 7feb92d9d700 1 --2- 172.21.15.5:0/1361578390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4d95471e0 0x55c4d95d9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:38.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.983+0000 7feb92d9d700 1 -- 172.21.15.5:0/1361578390 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4d95e1030 con 0x55c4d945d520 2024-05-09T00:58:38.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.984+0000 7feb92d9d700 1 --2- 172.21.15.5:0/1361578390 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4d945d520 0x55c4d95f6a20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7feb94066880 tx=0x7feb94069ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8b0eb99ebb7b9bf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:58:38.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.992+0000 7feb78ff9700 1 -- 172.21.15.5:0/1361578390 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb94081040 con 0x55c4d945d520 2024-05-09T00:58:38.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.992+0000 7feb78ff9700 1 -- 172.21.15.5:0/1361578390 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb9408f020 con 0x55c4d945d520 2024-05-09T00:58:38.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.992+0000 7feb78ff9700 1 -- 172.21.15.5:0/1361578390 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9408ba30 con 0x55c4d945d520 2024-05-09T00:58:38.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.993+0000 7feba7b51b80 1 -- 172.21.15.5:0/1361578390 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4d945d520 msgr2=0x55c4d95f6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:38.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.993+0000 7feba7b51b80 1 --2- 172.21.15.5:0/1361578390 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4d945d520 0x55c4d95f6a20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7feb94066880 tx=0x7feb94069ab0 comp rx=0 tx=0).stop 2024-05-09T00:58:38.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.994+0000 7feba7b51b80 1 -- 172.21.15.5:0/1361578390 shutdown_connections 2024-05-09T00:58:38.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.994+0000 7feba7b51b80 1 --2- 172.21.15.5:0/1361578390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4d95da880 0x55c4d95f01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:38.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.994+0000 7feba7b51b80 1 --2- 172.21.15.5:0/1361578390 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4d945d520 0x55c4d95f6a20 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:38.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.994+0000 7feba7b51b80 1 --2- 172.21.15.5:0/1361578390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4d95471e0 0x55c4d95d9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:38.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.994+0000 7feba7b51b80 1 -- 172.21.15.5:0/1361578390 >> 172.21.15.5:0/1361578390 conn(0x55c4d94bd720 msgr2=0x55c4d95d9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:58:38.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.995+0000 7feba7b51b80 1 -- 172.21.15.5:0/1361578390 shutdown_connections 2024-05-09T00:58:38.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.995+0000 7feba7b51b80 1 -- 172.21.15.5:0/1361578390 wait complete. 2024-05-09T00:58:38.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.996+0000 7feba7b51b80 1 -- start start 2024-05-09T00:58:38.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.996+0000 7feba7b51b80 1 Processor -- start 2024-05-09T00:58:38.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.997+0000 7feba7b51b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4d945d520 0x55c4d95ecba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:38.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.997+0000 7feba7b51b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4d95471e0 0x55c4d95fb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:38.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.997+0000 7feba7b51b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4d95da880 0x55c4d95fbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:38.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.997+0000 7feba7b51b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7feb8000e9c0 con 0x55c4d95da880 2024-05-09T00:58:38.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.997+0000 7feba7b51b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7feb8000e2f0 con 0x55c4d945d520 2024-05-09T00:58:38.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.997+0000 7feba7b51b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7feb8000b4f0 con 0x55c4d95471e0 2024-05-09T00:58:38.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.997+0000 7feba7b51b80 10 client.0 _ll_register_callbacks cb 0x55c4d936e740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:58:39.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.999+0000 7feb92d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4d945d520 0x55c4d95ecba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:39.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.999+0000 7feb9259c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4d95471e0 0x55c4d95fb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:39.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.999+0000 7feb9259c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4d95471e0 0x55c4d95fb620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:51580/0 (socket says 172.21.15.5:51580) 2024-05-09T00:58:39.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.999+0000 7feb9259c700 1 -- 172.21.15.5:0/3727311727 learned_addr learned my addr 172.21.15.5:0/3727311727 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:58:39.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:38.999+0000 7feb9359e700 1 --2- 172.21.15.5:0/3727311727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4d95da880 0x55c4d95fbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:39.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:39.001+0000 7feb9259c700 1 -- 172.21.15.5:0/3727311727 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4d945d520 msgr2=0x55c4d95ecba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:58:39.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:39.001+0000 7feb9259c700 1 --2- 172.21.15.5:0/3727311727 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4d945d520 0x55c4d95ecba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:39.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:39.001+0000 7feb9259c700 1 -- 172.21.15.5:0/3727311727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4d95da880 msgr2=0x55c4d95fbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:58:39.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:39.001+0000 7feb9259c700 1 --2- 172.21.15.5:0/3727311727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4d95da880 0x55c4d95fbc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:39.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:39.001+0000 7feb9259c700 1 -- 172.21.15.5:0/3727311727 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4d95fc390 con 0x55c4d95471e0 2024-05-09T00:58:39.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:39.002+0000 7feb9259c700 1 --2- 172.21.15.5:0/3727311727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4d95471e0 0x55c4d95fb620 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7feb8800d480 tx=0x7feb8800b700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad52f51e0d6bf3a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:58:39.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:39.002+0000 7feb76ff5700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-09T00:58:39.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:39.008+0000 7feb76ff5700 1 -- 172.21.15.5:0/3727311727 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8800d220 con 0x55c4d95471e0 2024-05-09T00:58:39.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:39.008+0000 7feba7b51b80 1 -- 172.21.15.5:0/3727311727 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb9408f020 con 0x55c4d95471e0 2024-05-09T00:58:39.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:39.008+0000 7feb76ff5700 1 -- 172.21.15.5:0/3727311727 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7feb88006390 con 0x55c4d95471e0 2024-05-09T00:58:39.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:39.009+0000 7feb76ff5700 1 -- 172.21.15.5:0/3727311727 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7feb8803a050 con 0x55c4d95471e0 2024-05-09T00:58:39.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:39.009+0000 7feba7b51b80 10 client.26179 fetch_fsmap learned FSMap version 31 2024-05-09T00:58:39.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:39.009+0000 7feba7b51b80 1 -- 172.21.15.5:0/3727311727 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c4d965eb80 con 0x55c4d95471e0 2024-05-09T00:58:39.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:39.012+0000 7feb76ff5700 1 -- 172.21.15.5:0/3727311727 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb9408f020 con 0x55c4d95471e0 2024-05-09T00:58:39.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:39.012+0000 7feb76ff5700 1 -- 172.21.15.5:0/3727311727 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7feb8801f070 con 0x55c4d95471e0 2024-05-09T00:58:39.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:39.013+0000 7feba7b51b80 10 client.26179 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:58:39.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:39.013+0000 7feba7b51b80 10 client.26179 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:58:39.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:39.013+0000 7feba7b51b80 20 client.26179 populate_metadata read hostname 'smithi005' 2024-05-09T00:58:39.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:39.013+0000 7feba7b51b80 1 --2- 172.21.15.5:0/3727311727 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55c4d965f200 0x55c4d9661620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:39.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:39.013+0000 7feba7b51b80 4 client.26179 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T00:58:39.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:39.013+0000 7feba7b51b80 1 -- 172.21.15.5:0/3727311727 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c4d950aa50 con 0x55c4d965f200 2024-05-09T00:58:39.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:39.017+0000 7feb92d9d700 1 --2- 172.21.15.5:0/3727311727 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55c4d965f200 0x55c4d9661620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:39.020 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:58:38 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:58:39.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:39.022+0000 7feb92d9d700 1 --2- 172.21.15.5:0/3727311727 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55c4d965f200 0x55c4d9661620 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2734f1af6cfb56bf server_cookie=b34f5deb72036db1 in_seq=0 out_seq=0 2024-05-09T00:58:39.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:39.022+0000 7feb76ff5700 10 client.26179 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:58:39.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:58:38 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:58:39.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:58:38 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:58:39.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:39.362+0000 7feb76ff5700 1 -- 172.21.15.5:0/3727311727 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c4d950aa50 con 0x55c4d965f200 2024-05-09T00:58:39.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:39.362+0000 7feb76ff5700 10 client.26179 handle_command_reply: tid=0 2024-05-09T00:58:39.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:39.362+0000 7feba7b51b80 1 client.26179 shutdown 2024-05-09T00:58:39.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:39.362+0000 7feba7b51b80 1 -- 172.21.15.5:0/3727311727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4d95471e0 msgr2=0x55c4d95fb620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:39.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:39.362+0000 7feba7b51b80 1 --2- 172.21.15.5:0/3727311727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4d95471e0 0x55c4d95fb620 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7feb8800d480 tx=0x7feb8800b700 comp rx=0 tx=0).stop 2024-05-09T00:58:39.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:39.363+0000 7feba7b51b80 1 -- 172.21.15.5:0/3727311727 shutdown_connections 2024-05-09T00:58:39.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:39.363+0000 7feba7b51b80 1 --2- 172.21.15.5:0/3727311727 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55c4d965f200 0x55c4d9661620 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:39.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:39.363+0000 7feba7b51b80 1 --2- 172.21.15.5:0/3727311727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4d95da880 0x55c4d95fbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:39.366 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-09T00:58:39.366 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-05-09T00:58:39.366 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "41f85d8b-889d-4179-bcda-0adf46184e67", 2024-05-09T00:58:39.366 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-05-09T00:58:39.366 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-09T00:58:39.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:39.363+0000 7feba7b51b80 1 --2- 172.21.15.5:0/3727311727 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4d945d520 0x55c4d95ecba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:39.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:39.363+0000 7feba7b51b80 1 --2- 172.21.15.5:0/3727311727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c4d95471e0 0x55c4d95fb620 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:39.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:39.363+0000 7feba7b51b80 1 -- 172.21.15.5:0/3727311727 >> 172.21.15.5:0/3727311727 conn(0x55c4d94bd720 msgr2=0x55c4d95de080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:58:39.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:39.363+0000 7feb76ff5700 0 client.26179 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:58:39.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:39.363+0000 7feba7b51b80 1 -- 172.21.15.5:0/3727311727 shutdown_connections 2024-05-09T00:58:39.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:39.363+0000 7feba7b51b80 1 -- 172.21.15.5:0/3727311727 wait complete. 2024-05-09T00:58:39.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:39.363+0000 7feba7b51b80 20 client.26179 trim_cache size 0 max 16384 2024-05-09T00:58:39.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:39.364+0000 7feb9893a700 1 -- 172.21.15.5:0/2352592341 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7feb7c0b2a90 msgr2=0x7feb7c0b4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:39.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:39.364+0000 7feb9893a700 1 --2- 172.21.15.5:0/2352592341 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7feb7c0b2a90 0x7feb7c0b4fa0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7feb94067070 tx=0x7feb94064d40 comp rx=0 tx=0).stop 2024-05-09T00:58:39.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:39.364+0000 7feb9893a700 1 -- 172.21.15.5:0/2352592341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb8c0b8320 msgr2=0x7feb8c0b0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:39.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:39.364+0000 7feb9893a700 1 --2- 172.21.15.5:0/2352592341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb8c0b8320 0x7feb8c0b0020 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7feb84004790 tx=0x7feb840047c0 comp rx=0 tx=0).stop 2024-05-09T00:58:39.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:39.364+0000 7feb9893a700 1 -- 172.21.15.5:0/2352592341 shutdown_connections 2024-05-09T00:58:39.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:39.364+0000 7feb9893a700 1 --2- 172.21.15.5:0/2352592341 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7feb7c0b2a90 0x7feb7c0b4fa0 secure :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7feb94067070 tx=0x7feb94064d40 comp rx=0 tx=0).stop 2024-05-09T00:58:39.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:39.364+0000 7feb9893a700 1 --2- 172.21.15.5:0/2352592341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb8c0b8320 0x7feb8c0b0020 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:39.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:39.364+0000 7feb9893a700 1 --2- 172.21.15.5:0/2352592341 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb8c0a53c0 0x7feb8c0b6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:39.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:39.364+0000 7feb9893a700 1 --2- 172.21.15.5:0/2352592341 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb8c0a5d10 0x7feb8c0af960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:39.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:39.364+0000 7feb9893a700 1 -- 172.21.15.5:0/2352592341 >> 172.21.15.5:0/2352592341 conn(0x7feb8c01af20 msgr2=0x7feb8c0a1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:58:39.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:39.367+0000 7feb9893a700 1 -- 172.21.15.5:0/2352592341 shutdown_connections 2024-05-09T00:58:39.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:39.367+0000 7feb9893a700 1 -- 172.21.15.5:0/2352592341 wait complete. 2024-05-09T00:58:39.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.919+0000 7f52bfc53640 1 -- 172.21.15.5:0/432721202 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52b810b300 msgr2=0x7f52b810b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:39.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.919+0000 7f52bfc53640 1 --2- 172.21.15.5:0/432721202 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52b810b300 0x7f52b810b710 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f52b4009860 tx=0x7f52b402a340 comp rx=0 tx=0).stop 2024-05-09T00:58:39.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.919+0000 7f52bfc53640 1 -- 172.21.15.5:0/432721202 shutdown_connections 2024-05-09T00:58:39.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.919+0000 7f52bfc53640 1 --2- 172.21.15.5:0/432721202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52b8110020 0x7f52b8110430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:39.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.919+0000 7f52bfc53640 1 --2- 172.21.15.5:0/432721202 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52b810b300 0x7f52b810b710 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:39.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.919+0000 7f52bfc53640 1 --2- 172.21.15.5:0/432721202 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52b810a480 0x7f52b810adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:39.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.919+0000 7f52bfc53640 1 -- 172.21.15.5:0/432721202 >> 172.21.15.5:0/432721202 conn(0x7f52b80704c0 msgr2=0x7f52b8072950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:58:39.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.920+0000 7f52bfc53640 1 -- 172.21.15.5:0/432721202 shutdown_connections 2024-05-09T00:58:39.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.920+0000 7f52bfc53640 1 -- 172.21.15.5:0/432721202 wait complete. 2024-05-09T00:58:39.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.921+0000 7f52bfc53640 1 Processor -- start 2024-05-09T00:58:39.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.921+0000 7f52bfc53640 1 -- start start 2024-05-09T00:58:39.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.922+0000 7f52bfc53640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52b810a480 0x7f52b8117240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:39.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.922+0000 7f52bfc53640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52b810b300 0x7f52b8117780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:39.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.922+0000 7f52bfc53640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52b8110020 0x7f52b8118670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:39.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.922+0000 7f52bfc53640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f52b811e430 con 0x7f52b8110020 2024-05-09T00:58:39.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.922+0000 7f52bfc53640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f52b811e2b0 con 0x7f52b810b300 2024-05-09T00:58:39.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.922+0000 7f52bfc53640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f52b811e5b0 con 0x7f52b810a480 2024-05-09T00:58:39.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.926+0000 7f52bd9ef640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52b810a480 0x7f52b8117240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:39.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.926+0000 7f52be1f0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52b8110020 0x7f52b8118670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:39.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.926+0000 7f52be1f0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52b8110020 0x7f52b8118670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48150/0 (socket says 172.21.15.5:48150) 2024-05-09T00:58:39.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.926+0000 7f52be1f0640 1 -- 172.21.15.5:0/349201918 learned_addr learned my addr 172.21.15.5:0/349201918 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:58:39.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.926+0000 7f52bd1ee640 1 --2- 172.21.15.5:0/349201918 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52b810b300 0x7f52b8117780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:39.928 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.928+0000 7f52bd9ef640 1 -- 172.21.15.5:0/349201918 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52b810b300 msgr2=0x7f52b8117780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:39.928 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.928+0000 7f52bd9ef640 1 --2- 172.21.15.5:0/349201918 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52b810b300 0x7f52b8117780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:39.928 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.928+0000 7f52bd9ef640 1 -- 172.21.15.5:0/349201918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52b8110020 msgr2=0x7f52b8118670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:39.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.928+0000 7f52bd9ef640 1 --2- 172.21.15.5:0/349201918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52b8110020 0x7f52b8118670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:39.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.928+0000 7f52bd9ef640 1 -- 172.21.15.5:0/349201918 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52b8118ce0 con 0x7f52b810a480 2024-05-09T00:58:39.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.928+0000 7f52bd9ef640 1 --2- 172.21.15.5:0/349201918 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52b810a480 0x7f52b8117240 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f52a800b3f0 tx=0x7f52a800b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:58:39.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.934+0000 7f52aeffd640 1 -- 172.21.15.5:0/349201918 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a8015020 con 0x7f52b810a480 2024-05-09T00:58:39.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.934+0000 7f52aeffd640 1 -- 172.21.15.5:0/349201918 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52a8011ce0 con 0x7f52b810a480 2024-05-09T00:58:39.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.934+0000 7f52aeffd640 1 -- 172.21.15.5:0/349201918 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a8010d80 con 0x7f52b810a480 2024-05-09T00:58:39.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.935+0000 7f52bfc53640 1 -- 172.21.15.5:0/349201918 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52b8118fd0 con 0x7f52b810a480 2024-05-09T00:58:39.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.935+0000 7f52bfc53640 1 -- 172.21.15.5:0/349201918 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52b81194d0 con 0x7f52b810a480 2024-05-09T00:58:39.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.942+0000 7f52c6be7740 1 Processor -- start 2024-05-09T00:58:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.943+0000 7f52c6be7740 1 -- start start 2024-05-09T00:58:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.943+0000 7f52c6be7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b80da661b0 0x55b80de0a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.943+0000 7f52c6be7740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b80da3d2f0 0x55b80de0ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.943+0000 7f52c6be7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b80da454d0 0x55b80de11b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.943+0000 7f52c6be7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b80dd6e7c0 con 0x55b80da661b0 2024-05-09T00:58:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.943+0000 7f52c6be7740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55b80dd0a0e0 con 0x55b80da3d2f0 2024-05-09T00:58:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.943+0000 7f52c6be7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b80dd6a780 con 0x55b80da454d0 2024-05-09T00:58:39.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.944+0000 7f52bd9ef640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b80da3d2f0 0x55b80de0ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.944+0000 7f52bd9ef640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b80da3d2f0 0x55b80de0ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:48582/0 (socket says 172.21.15.5:48582) 2024-05-09T00:58:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.944+0000 7f52bd9ef640 1 -- 172.21.15.5:0/1303013697 learned_addr learned my addr 172.21.15.5:0/1303013697 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:58:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.945+0000 7f52bd1ee640 1 --2- 172.21.15.5:0/1303013697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b80da454d0 0x55b80de11b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:39.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.945+0000 7f52bd1ee640 1 -- 172.21.15.5:0/1303013697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b80da3d2f0 msgr2=0x55b80de0ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:58:39.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.945+0000 7f52bd1ee640 1 --2- 172.21.15.5:0/1303013697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b80da3d2f0 0x55b80de0ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:39.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.945+0000 7f52bd1ee640 1 -- 172.21.15.5:0/1303013697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b80da661b0 msgr2=0x55b80de0a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:58:39.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.945+0000 7f52bd1ee640 1 --2- 172.21.15.5:0/1303013697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b80da661b0 0x55b80de0a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:39.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.945+0000 7f52bd1ee640 1 -- 172.21.15.5:0/1303013697 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b80dd337c0 con 0x55b80da454d0 2024-05-09T00:58:39.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.946+0000 7f52aeffd640 1 -- 172.21.15.5:0/349201918 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f52a8018020 con 0x7f52b810a480 2024-05-09T00:58:39.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.949+0000 7f52aeffd640 1 --2- 172.21.15.5:0/349201918 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f52a40b29f0 0x7f52a40b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:39.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.950+0000 7f52bd1ee640 1 --2- 172.21.15.5:0/1303013697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b80da454d0 0x55b80de11b40 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f52b81182e0 tx=0x7f52b4000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72da5b13efbb4dd6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:58:39.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.950+0000 7f52aeffd640 1 -- 172.21.15.5:0/349201918 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f52a80e1950 con 0x7f52b810a480 2024-05-09T00:58:39.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.950+0000 7f52be1f0640 1 --2- 172.21.15.5:0/349201918 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f52a40b29f0 0x7f52a40b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:39.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.950+0000 7f52a3fff640 1 -- 172.21.15.5:0/1303013697 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b4041030 con 0x55b80da454d0 2024-05-09T00:58:39.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.950+0000 7f52a3fff640 1 -- 172.21.15.5:0/1303013697 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52b4030e00 con 0x55b80da454d0 2024-05-09T00:58:39.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.950+0000 7f52a3fff640 1 -- 172.21.15.5:0/1303013697 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b4037040 con 0x55b80da454d0 2024-05-09T00:58:39.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.952+0000 7f52c6be7740 1 -- 172.21.15.5:0/1303013697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b80da454d0 msgr2=0x55b80de11b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:39.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.952+0000 7f52c6be7740 1 --2- 172.21.15.5:0/1303013697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b80da454d0 0x55b80de11b40 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f52b81182e0 tx=0x7f52b4000bc0 comp rx=0 tx=0).stop 2024-05-09T00:58:39.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.952+0000 7f52be1f0640 1 --2- 172.21.15.5:0/349201918 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f52a40b29f0 0x7f52a40b4f00 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f52b0009fd0 tx=0x7f52b0009a00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:58:39.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.952+0000 7f52c6be7740 1 -- 172.21.15.5:0/1303013697 shutdown_connections 2024-05-09T00:58:39.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.952+0000 7f52c6be7740 1 --2- 172.21.15.5:0/1303013697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b80da454d0 0x55b80de11b40 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:39.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.952+0000 7f52c6be7740 1 --2- 172.21.15.5:0/1303013697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b80da3d2f0 0x55b80de0ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:39.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.952+0000 7f52c6be7740 1 --2- 172.21.15.5:0/1303013697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b80da661b0 0x55b80de0a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:39.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.952+0000 7f52c6be7740 1 -- 172.21.15.5:0/1303013697 >> 172.21.15.5:0/1303013697 conn(0x55b80da5a300 msgr2=0x55b80de06050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:58:39.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.953+0000 7f52c6be7740 1 -- 172.21.15.5:0/1303013697 shutdown_connections 2024-05-09T00:58:39.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.953+0000 7f52c6be7740 1 -- 172.21.15.5:0/1303013697 wait complete. 2024-05-09T00:58:39.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.953+0000 7f52c6be7740 1 -- start start 2024-05-09T00:58:39.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.953+0000 7f52c6be7740 1 Processor -- start 2024-05-09T00:58:39.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.954+0000 7f52c6be7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b80da3d2f0 0x55b80de14390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:39.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.954+0000 7f52c6be7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b80da454d0 0x55b80de148d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:39.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.954+0000 7f52c6be7740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b80da661b0 0x55b80de1b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:39.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.954+0000 7f52c6be7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b80de12530 con 0x55b80da3d2f0 2024-05-09T00:58:39.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.954+0000 7f52c6be7740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55b80dcf59c0 con 0x55b80da661b0 2024-05-09T00:58:39.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.954+0000 7f52c6be7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b80de126b0 con 0x55b80da454d0 2024-05-09T00:58:39.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.954+0000 7f52c6be7740 10 client.0 _ll_register_callbacks cb 0x55b80dd22020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:58:39.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.955+0000 7f52bd1ee640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b80da454d0 0x55b80de148d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:39.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.955+0000 7f52bd1ee640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b80da454d0 0x55b80de148d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:51618/0 (socket says 172.21.15.5:51618) 2024-05-09T00:58:39.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.955+0000 7f52bd1ee640 1 -- 172.21.15.5:0/1696883989 learned_addr learned my addr 172.21.15.5:0/1696883989 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:58:39.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.957+0000 7f52bd9ef640 1 --2- 172.21.15.5:0/1696883989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b80da3d2f0 0x55b80de14390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:39.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.958+0000 7f52bd9ef640 1 -- 172.21.15.5:0/1696883989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b80da454d0 msgr2=0x55b80de148d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:58:39.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.958+0000 7f52bd9ef640 1 --2- 172.21.15.5:0/1696883989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b80da454d0 0x55b80de148d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:39.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.958+0000 7f52bd9ef640 1 -- 172.21.15.5:0/1696883989 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b80da661b0 msgr2=0x55b80de1b830 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:58:39.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.958+0000 7f52bd9ef640 1 --2- 172.21.15.5:0/1696883989 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b80da661b0 0x55b80de1b830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:39.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.958+0000 7f52bd9ef640 1 -- 172.21.15.5:0/1696883989 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b80dd1c1f0 con 0x55b80da3d2f0 2024-05-09T00:58:39.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.961+0000 7f52bd9ef640 1 --2- 172.21.15.5:0/1696883989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b80da3d2f0 0x55b80de14390 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f52a80a5f60 tx=0x7f52a8009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67e1de221bf74ea1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:58:39.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.968+0000 7f52a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:58:39.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.968+0000 7f52a17fa640 1 -- 172.21.15.5:0/1696883989 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a80a9650 con 0x55b80da3d2f0 2024-05-09T00:58:39.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.972+0000 7f52a17fa640 1 -- 172.21.15.5:0/1696883989 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f52a80a6170 con 0x55b80da3d2f0 2024-05-09T00:58:39.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.977+0000 7f52c6be7740 1 -- 172.21.15.5:0/1696883989 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52b4030e00 con 0x55b80da3d2f0 2024-05-09T00:58:39.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.987+0000 7f52a17fa640 1 -- 172.21.15.5:0/1696883989 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f52a8011e90 con 0x55b80da3d2f0 2024-05-09T00:58:39.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.987+0000 7f52c6be7740 10 client.16791 fetch_fsmap learned FSMap version 31 2024-05-09T00:58:39.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:39.987+0000 7f52c6be7740 1 -- 172.21.15.5:0/1696883989 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b80dad1790 con 0x55b80da3d2f0 2024-05-09T00:58:40.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.001+0000 7f52a17fa640 1 -- 172.21.15.5:0/1696883989 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52b4030e00 con 0x55b80da3d2f0 2024-05-09T00:58:40.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.001+0000 7f52a17fa640 1 -- 172.21.15.5:0/1696883989 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f52a80a9140 con 0x55b80da3d2f0 2024-05-09T00:58:40.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.001+0000 7f52c6be7740 10 client.16791 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:58:40.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.001+0000 7f52c6be7740 10 client.16791 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:58:40.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.001+0000 7f52c6be7740 20 client.16791 populate_metadata read hostname 'smithi005' 2024-05-09T00:58:40.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.001+0000 7f52c6be7740 1 --2- 172.21.15.5:0/1696883989 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55b80da24d80 0x55b80de21d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:40.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.001+0000 7f52c6be7740 4 client.16791 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:58:40.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.001+0000 7f52c6be7740 1 -- 172.21.15.5:0/1696883989 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b80dd7c3d0 con 0x55b80da24d80 2024-05-09T00:58:40.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.002+0000 7f52bd1ee640 1 --2- 172.21.15.5:0/1696883989 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55b80da24d80 0x55b80de21d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:40.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.004+0000 7f52bd1ee640 1 --2- 172.21.15.5:0/1696883989 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55b80da24d80 0x55b80de21d70 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9f2e39f8ab33688 server_cookie=8f14de0e89abfa00 in_seq=0 out_seq=0 2024-05-09T00:58:40.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.004+0000 7f52a17fa640 10 client.16791 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:58:40.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.008+0000 7f52a17fa640 1 -- 172.21.15.5:0/1696883989 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b80dd7c3d0 con 0x55b80da24d80 2024-05-09T00:58:40.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.008+0000 7f52a17fa640 10 client.16791 handle_command_reply: tid=0 2024-05-09T00:58:40.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.008+0000 7f52c6be7740 1 client.16791 shutdown 2024-05-09T00:58:40.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.010+0000 7f52c6be7740 1 -- 172.21.15.5:0/1696883989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b80da3d2f0 msgr2=0x55b80de14390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:40.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.010+0000 7f52c6be7740 1 --2- 172.21.15.5:0/1696883989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b80da3d2f0 0x55b80de14390 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f52a80a5f60 tx=0x7f52a8009310 comp rx=0 tx=0).stop 2024-05-09T00:58:40.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.017+0000 7f52c6be7740 1 -- 172.21.15.5:0/1696883989 shutdown_connections 2024-05-09T00:58:40.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.017+0000 7f52c6be7740 1 --2- 172.21.15.5:0/1696883989 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55b80da24d80 0x55b80de21d70 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:40.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.017+0000 7f52c6be7740 1 --2- 172.21.15.5:0/1696883989 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b80da661b0 0x55b80de1b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:40.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.017+0000 7f52c6be7740 1 --2- 172.21.15.5:0/1696883989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b80da454d0 0x55b80de148d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:40.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.017+0000 7f52c6be7740 1 --2- 172.21.15.5:0/1696883989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b80da3d2f0 0x55b80de14390 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:40.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.017+0000 7f52c6be7740 1 -- 172.21.15.5:0/1696883989 >> 172.21.15.5:0/1696883989 conn(0x55b80da5a300 msgr2=0x55b80de05290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:58:40.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.017+0000 7f52a17fa640 0 client.16791 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:58:40.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.017+0000 7f52c6be7740 1 -- 172.21.15.5:0/1696883989 shutdown_connections 2024-05-09T00:58:40.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.017+0000 7f52c6be7740 1 -- 172.21.15.5:0/1696883989 wait complete. 2024-05-09T00:58:40.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.018+0000 7f52c6be7740 20 client.16791 trim_cache size 0 max 16384 2024-05-09T00:58:40.047 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:58:39 smithi005 ceph-mon[30365]: pgmap v740: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 863 MiB/s wr, 2.04k op/s 2024-05-09T00:58:40.047 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:58:39 smithi005 ceph-mon[30365]: scrub queued for path: / 2024-05-09T00:58:40.047 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:58:39 smithi005 ceph-mon[30365]: scrub summary: idle+waiting paths [/] 2024-05-09T00:58:40.047 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:58:39 smithi005 ceph-mon[30365]: scrub summary: active paths [/] 2024-05-09T00:58:40.047 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:58:39 smithi005 ceph-mon[30365]: scrub summary: active paths [/] 2024-05-09T00:58:40.048 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:58:39 smithi005 ceph-mon[30365]: scrub summary: idle+waiting paths [/] 2024-05-09T00:58:40.048 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:58:39 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:58:40.048 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:58:39 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:58:40.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.081+0000 7f52c6be7740 1 Processor -- start 2024-05-09T00:58:40.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.082+0000 7f52c6be7740 1 -- start start 2024-05-09T00:58:40.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.082+0000 7f52c6be7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b80da3d2f0 0x55b80de0e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:40.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.082+0000 7f52c6be7740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b80da454d0 0x55b80de0ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:40.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.083+0000 7f52c6be7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b80da661b0 0x55b80de0d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:40.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.088+0000 7f52bd9ef640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b80da3d2f0 0x55b80de0e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:40.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.088+0000 7f52bd1ee640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b80da454d0 0x55b80de0ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:40.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.089+0000 7f52be1f0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b80da661b0 0x55b80de0d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:40.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.089+0000 7f52bd1ee640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b80da454d0 0x55b80de0ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:48594/0 (socket says 172.21.15.5:48594) 2024-05-09T00:58:40.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.089+0000 7f52bd1ee640 1 -- 172.21.15.5:0/3607525570 learned_addr learned my addr 172.21.15.5:0/3607525570 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:58:40.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.089+0000 7f52c6be7740 1 -- 172.21.15.5:0/3607525570 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f529c00e610 con 0x55b80da661b0 2024-05-09T00:58:40.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.089+0000 7f52c6be7740 1 -- 172.21.15.5:0/3607525570 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f529c010150 con 0x55b80da454d0 2024-05-09T00:58:40.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.089+0000 7f52c6be7740 1 -- 172.21.15.5:0/3607525570 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f529c00ece0 con 0x55b80da3d2f0 2024-05-09T00:58:40.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.090+0000 7f52bd1ee640 1 -- 172.21.15.5:0/3607525570 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b80da3d2f0 msgr2=0x55b80de0e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:58:40.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.090+0000 7f52bd1ee640 1 --2- 172.21.15.5:0/3607525570 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b80da3d2f0 0x55b80de0e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:40.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.090+0000 7f52bd1ee640 1 -- 172.21.15.5:0/3607525570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b80da661b0 msgr2=0x55b80de0d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:58:40.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.090+0000 7f52bd1ee640 1 --2- 172.21.15.5:0/3607525570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b80da661b0 0x55b80de0d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:40.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.090+0000 7f52bd1ee640 1 -- 172.21.15.5:0/3607525570 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b80db485f0 con 0x55b80da454d0 2024-05-09T00:58:40.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.091+0000 7f52bd1ee640 1 --2- 172.21.15.5:0/3607525570 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b80da454d0 0x55b80de0ed60 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f52b4035040 tx=0x7f52b403f040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc2729b76daeca53 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:58:40.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.093+0000 7f52a37fe640 1 -- 172.21.15.5:0/3607525570 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b4041070 con 0x55b80da454d0 2024-05-09T00:58:40.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.094+0000 7f52a37fe640 1 -- 172.21.15.5:0/3607525570 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52b4030e00 con 0x55b80da454d0 2024-05-09T00:58:40.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.094+0000 7f52a37fe640 1 -- 172.21.15.5:0/3607525570 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b4037040 con 0x55b80da454d0 2024-05-09T00:58:40.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.096+0000 7f52c6be7740 1 -- 172.21.15.5:0/3607525570 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b80da454d0 msgr2=0x55b80de0ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:40.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.096+0000 7f52c6be7740 1 --2- 172.21.15.5:0/3607525570 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b80da454d0 0x55b80de0ed60 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f52b4035040 tx=0x7f52b403f040 comp rx=0 tx=0).stop 2024-05-09T00:58:40.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.096+0000 7f52c6be7740 1 -- 172.21.15.5:0/3607525570 shutdown_connections 2024-05-09T00:58:40.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.096+0000 7f52c6be7740 1 --2- 172.21.15.5:0/3607525570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b80da661b0 0x55b80de0d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:40.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.096+0000 7f52c6be7740 1 --2- 172.21.15.5:0/3607525570 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b80da454d0 0x55b80de0ed60 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:40.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.096+0000 7f52c6be7740 1 --2- 172.21.15.5:0/3607525570 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b80da3d2f0 0x55b80de0e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:40.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.096+0000 7f52c6be7740 1 -- 172.21.15.5:0/3607525570 >> 172.21.15.5:0/3607525570 conn(0x55b80da5a300 msgr2=0x55b80de14330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:58:40.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.097+0000 7f52c6be7740 1 -- 172.21.15.5:0/3607525570 shutdown_connections 2024-05-09T00:58:40.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.097+0000 7f52c6be7740 1 -- 172.21.15.5:0/3607525570 wait complete. 2024-05-09T00:58:40.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.098+0000 7f52c6be7740 1 -- start start 2024-05-09T00:58:40.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.098+0000 7f52c6be7740 1 Processor -- start 2024-05-09T00:58:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.099+0000 7f52c6be7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b80da3d2f0 0x55b80de17bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.099+0000 7f52c6be7740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b80da454d0 0x55b80de18100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.100+0000 7f52c6be7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b80da661b0 0x55b80de88160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.100+0000 7f52c6be7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f529c00fa80 con 0x55b80da661b0 2024-05-09T00:58:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.100+0000 7f52c6be7740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f529c00f3b0 con 0x55b80da454d0 2024-05-09T00:58:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.100+0000 7f52c6be7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f529c00c5f0 con 0x55b80da3d2f0 2024-05-09T00:58:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.101+0000 7f52c6be7740 10 client.0 _ll_register_callbacks cb 0x7f52a8010f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:58:40.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.101+0000 7f52be1f0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b80da661b0 0x55b80de88160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:40.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.102+0000 7f52be1f0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b80da661b0 0x55b80de88160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48182/0 (socket says 172.21.15.5:48182) 2024-05-09T00:58:40.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.102+0000 7f52be1f0640 1 -- 172.21.15.5:0/719433556 learned_addr learned my addr 172.21.15.5:0/719433556 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:58:40.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.102+0000 7f52bd1ee640 1 --2- 172.21.15.5:0/719433556 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b80da454d0 0x55b80de18100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:40.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.102+0000 7f52bd9ef640 1 --2- 172.21.15.5:0/719433556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b80da3d2f0 0x55b80de17bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:40.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:58:39 smithi131 ceph-mon[34547]: pgmap v740: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 863 MiB/s wr, 2.04k op/s 2024-05-09T00:58:40.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:58:39 smithi131 ceph-mon[34547]: scrub queued for path: / 2024-05-09T00:58:40.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:58:39 smithi131 ceph-mon[34547]: scrub summary: idle+waiting paths [/] 2024-05-09T00:58:40.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:58:39 smithi131 ceph-mon[34547]: scrub summary: active paths [/] 2024-05-09T00:58:40.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:58:39 smithi131 ceph-mon[34547]: scrub summary: active paths [/] 2024-05-09T00:58:40.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:58:39 smithi131 ceph-mon[34547]: scrub summary: idle+waiting paths [/] 2024-05-09T00:58:40.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:58:39 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:58:40.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:58:39 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:58:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.104+0000 7f52bd1ee640 1 -- 172.21.15.5:0/719433556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b80da3d2f0 msgr2=0x55b80de17bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:58:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.104+0000 7f52bd1ee640 1 --2- 172.21.15.5:0/719433556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b80da3d2f0 0x55b80de17bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.104+0000 7f52bd1ee640 1 -- 172.21.15.5:0/719433556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b80da661b0 msgr2=0x55b80de88160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:58:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.104+0000 7f52bd1ee640 1 --2- 172.21.15.5:0/719433556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b80da661b0 0x55b80de88160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.104+0000 7f52bd1ee640 1 -- 172.21.15.5:0/719433556 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b80dd32ed0 con 0x55b80da454d0 2024-05-09T00:58:40.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.107+0000 7f52bd1ee640 1 --2- 172.21.15.5:0/719433556 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b80da454d0 0x55b80de18100 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x55b80daaf680 tx=0x7f52b402a9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35c79f9ab2612e45 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:58:40.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.107+0000 7f52a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-09T00:58:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.108+0000 7f52a17fa640 1 -- 172.21.15.5:0/719433556 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b4037070 con 0x55b80da454d0 2024-05-09T00:58:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.108+0000 7f52a17fa640 1 -- 172.21.15.5:0/719433556 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f52b4004240 con 0x55b80da454d0 2024-05-09T00:58:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.109+0000 7f52c6be7740 1 -- 172.21.15.5:0/719433556 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52b4030e00 con 0x55b80da454d0 2024-05-09T00:58:40.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.112+0000 7f52a17fa640 1 -- 172.21.15.5:0/719433556 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f52b4041070 con 0x55b80da454d0 2024-05-09T00:58:40.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.112+0000 7f52c6be7740 10 client.26219 fetch_fsmap learned FSMap version 31 2024-05-09T00:58:40.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.112+0000 7f52c6be7740 1 -- 172.21.15.5:0/719433556 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b80de893f0 con 0x55b80da454d0 2024-05-09T00:58:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.114+0000 7f52a17fa640 1 -- 172.21.15.5:0/719433556 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52b4004e20 con 0x55b80da454d0 2024-05-09T00:58:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.114+0000 7f52a17fa640 1 -- 172.21.15.5:0/719433556 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f52b40035f0 con 0x55b80da454d0 2024-05-09T00:58:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.114+0000 7f52c6be7740 10 client.26219 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:58:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.114+0000 7f52c6be7740 10 client.26219 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:58:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.114+0000 7f52c6be7740 20 client.26219 populate_metadata read hostname 'smithi005' 2024-05-09T00:58:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.115+0000 7f52c6be7740 1 --2- 172.21.15.5:0/719433556 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55b80d9fa1c0 0x55b80de8bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:40.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.115+0000 7f52c6be7740 4 client.26219 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:58:40.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.115+0000 7f52c6be7740 1 -- 172.21.15.5:0/719433556 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f52b4030e00 con 0x55b80d9fa1c0 2024-05-09T00:58:40.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.116+0000 7f52bd9ef640 1 --2- 172.21.15.5:0/719433556 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55b80d9fa1c0 0x55b80de8bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:40.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.119+0000 7f52a17fa640 10 client.26219 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:58:40.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:40.125+0000 7f52bd9ef640 1 --2- 172.21.15.5:0/719433556 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55b80d9fa1c0 0x55b80de8bcc0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=893a9a3d27e024f0 server_cookie=38df4cdab4e105f1 in_seq=0 out_seq=0 2024-05-09T00:58:40.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:58:39 smithi157 ceph-mon[34294]: pgmap v740: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 863 MiB/s wr, 2.04k op/s 2024-05-09T00:58:40.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:58:39 smithi157 ceph-mon[34294]: scrub queued for path: / 2024-05-09T00:58:40.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:58:39 smithi157 ceph-mon[34294]: scrub summary: idle+waiting paths [/] 2024-05-09T00:58:40.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:58:39 smithi157 ceph-mon[34294]: scrub summary: active paths [/] 2024-05-09T00:58:40.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:58:39 smithi157 ceph-mon[34294]: scrub summary: active paths [/] 2024-05-09T00:58:40.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:58:39 smithi157 ceph-mon[34294]: scrub summary: idle+waiting paths [/] 2024-05-09T00:58:40.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:58:39 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:58:40.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:58:39 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:58:41.555 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T00:58:42.049 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:58:41 smithi005 ceph-mon[30365]: pgmap v741: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 908 MiB/s wr, 1.94k op/s 2024-05-09T00:58:42.049 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:58:41 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:58:42.049 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:58:41 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:58:42.049 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:58:41 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:58:42.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:58:41 smithi131 ceph-mon[34547]: pgmap v741: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 908 MiB/s wr, 1.94k op/s 2024-05-09T00:58:42.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:58:41 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:58:42.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:58:41 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:58:42.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:58:41 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:58:42.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:58:41 smithi157 ceph-mon[34294]: pgmap v741: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 908 MiB/s wr, 1.94k op/s 2024-05-09T00:58:42.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:58:41 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:58:42.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:58:41 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:58:42.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:58:41 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:58:42.517 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:58:44.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:58:43 smithi131 ceph-mon[34547]: pgmap v742: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 817 MiB/s wr, 1.77k op/s 2024-05-09T00:58:44.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:58:43 smithi131 ceph-mon[34547]: scrub summary: idle+waiting paths [/] 2024-05-09T00:58:44.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:58:43 smithi157 ceph-mon[34294]: pgmap v742: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 817 MiB/s wr, 1.77k op/s 2024-05-09T00:58:44.262 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:58:43 smithi005 ceph-mon[30365]: pgmap v742: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 817 MiB/s wr, 1.77k op/s 2024-05-09T00:58:44.262 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:58:43 smithi005 ceph-mon[30365]: scrub summary: idle+waiting paths [/] 2024-05-09T00:58:44.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:58:44 smithi157 ceph-mon[34294]: scrub summary: idle+waiting paths [/] 2024-05-09T00:58:46.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:58:45 smithi005 ceph-mon[30365]: pgmap v743: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 892 MiB/s wr, 1.95k op/s 2024-05-09T00:58:46.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:58:45 smithi131 ceph-mon[34547]: pgmap v743: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 892 MiB/s wr, 1.95k op/s 2024-05-09T00:58:46.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:58:45 smithi157 ceph-mon[34294]: pgmap v743: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 892 MiB/s wr, 1.95k op/s 2024-05-09T00:58:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:46.290+0000 7f52a17fa640 1 -- 172.21.15.5:0/719433556 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7f52b4030e00 con 0x55b80d9fa1c0 2024-05-09T00:58:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:46.291+0000 7f52a17fa640 10 client.26219 handle_command_reply: tid=0 2024-05-09T00:58:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:46.291+0000 7f52c6be7740 1 client.26219 shutdown 2024-05-09T00:58:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:46.291+0000 7f52c6be7740 1 -- 172.21.15.5:0/719433556 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b80da454d0 msgr2=0x55b80de18100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:46.291+0000 7f52c6be7740 1 --2- 172.21.15.5:0/719433556 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b80da454d0 0x55b80de18100 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x55b80daaf680 tx=0x7f52b402a9c0 comp rx=0 tx=0).stop 2024-05-09T00:58:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:46.292+0000 7f52c6be7740 1 -- 172.21.15.5:0/719433556 shutdown_connections 2024-05-09T00:58:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:46.292+0000 7f52c6be7740 1 --2- 172.21.15.5:0/719433556 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55b80d9fa1c0 0x55b80de8bcc0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:46.292+0000 7f52c6be7740 1 --2- 172.21.15.5:0/719433556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b80da661b0 0x55b80de88160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:46.292+0000 7f52c6be7740 1 --2- 172.21.15.5:0/719433556 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b80da454d0 0x55b80de18100 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:46.292+0000 7f52c6be7740 1 --2- 172.21.15.5:0/719433556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b80da3d2f0 0x55b80de17bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:46.292+0000 7f52c6be7740 1 -- 172.21.15.5:0/719433556 >> 172.21.15.5:0/719433556 conn(0x55b80da5a300 msgr2=0x55b80de0d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:58:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:46.292+0000 7f52a17fa640 0 client.26219 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:58:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:46.292+0000 7f52c6be7740 1 -- 172.21.15.5:0/719433556 shutdown_connections 2024-05-09T00:58:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:46.292+0000 7f52c6be7740 1 -- 172.21.15.5:0/719433556 wait complete. 2024-05-09T00:58:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:46.293+0000 7f52c6be7740 20 client.26219 trim_cache size 0 max 16384 2024-05-09T00:58:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-09T00:58:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-09T00:58:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-09T00:58:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:46.297+0000 7f52bfc53640 1 -- 172.21.15.5:0/349201918 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f52a40b29f0 msgr2=0x7f52a40b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:46.297+0000 7f52bfc53640 1 --2- 172.21.15.5:0/349201918 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f52a40b29f0 0x7f52a40b4f00 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f52b0009fd0 tx=0x7f52b0009a00 comp rx=0 tx=0).stop 2024-05-09T00:58:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:46.298+0000 7f52bfc53640 1 -- 172.21.15.5:0/349201918 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52b810a480 msgr2=0x7f52b8117240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:46.298+0000 7f52bfc53640 1 --2- 172.21.15.5:0/349201918 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52b810a480 0x7f52b8117240 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f52a800b3f0 tx=0x7f52a800b8b0 comp rx=0 tx=0).stop 2024-05-09T00:58:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:46.298+0000 7f52bfc53640 1 -- 172.21.15.5:0/349201918 shutdown_connections 2024-05-09T00:58:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:46.298+0000 7f52bfc53640 1 --2- 172.21.15.5:0/349201918 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f52a40b29f0 0x7f52a40b4f00 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:46.298+0000 7f52bfc53640 1 --2- 172.21.15.5:0/349201918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52b8110020 0x7f52b8118670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:46.298+0000 7f52bfc53640 1 --2- 172.21.15.5:0/349201918 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52b810b300 0x7f52b8117780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:46.298+0000 7f52bfc53640 1 --2- 172.21.15.5:0/349201918 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52b810a480 0x7f52b8117240 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:46.298+0000 7f52bfc53640 1 -- 172.21.15.5:0/349201918 >> 172.21.15.5:0/349201918 conn(0x7f52b80704c0 msgr2=0x7f52b810c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:58:46.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:46.301+0000 7f52bfc53640 1 -- 172.21.15.5:0/349201918 shutdown_connections 2024-05-09T00:58:46.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:46.301+0000 7f52bfc53640 1 -- 172.21.15.5:0/349201918 wait complete. 2024-05-09T00:58:47.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:58:46 smithi131 ceph-mon[34547]: scrub summary: idle 2024-05-09T00:58:47.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:58:46 smithi131 ceph-mon[34547]: scrub summary: idle 2024-05-09T00:58:47.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:58:46 smithi157 ceph-mon[34294]: scrub summary: idle 2024-05-09T00:58:47.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:58:46 smithi157 ceph-mon[34294]: scrub summary: idle 2024-05-09T00:58:47.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:58:46 smithi005 ceph-mon[30365]: scrub summary: idle 2024-05-09T00:58:47.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:58:46 smithi005 ceph-mon[30365]: scrub summary: idle 2024-05-09T00:58:48.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:58:47 smithi131 ceph-mon[34547]: pgmap v744: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 911 MiB/s wr, 2.06k op/s 2024-05-09T00:58:48.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:58:47 smithi157 ceph-mon[34294]: pgmap v744: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 911 MiB/s wr, 2.06k op/s 2024-05-09T00:58:48.289 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:58:47 smithi005 ceph-mon[30365]: pgmap v744: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 911 MiB/s wr, 2.06k op/s 2024-05-09T00:58:48.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.617+0000 7f38d9bce700 1 -- 172.21.15.5:0/878858777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38d410b2b0 msgr2=0x7f38d4117830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:48.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.617+0000 7f38d9bce700 1 --2- 172.21.15.5:0/878858777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38d410b2b0 0x7f38d4117830 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f38cc00b0a0 tx=0x7f38cc00b3b0 comp rx=0 tx=0).stop 2024-05-09T00:58:48.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.618+0000 7f38d9bce700 1 -- 172.21.15.5:0/878858777 shutdown_connections 2024-05-09T00:58:48.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.618+0000 7f38d9bce700 1 --2- 172.21.15.5:0/878858777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38d410b2b0 0x7f38d4117830 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:48.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.618+0000 7f38d9bce700 1 --2- 172.21.15.5:0/878858777 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38d4110990 0x7f38d4110da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:48.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.618+0000 7f38d9bce700 1 --2- 172.21.15.5:0/878858777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38d410a960 0x7f38d410ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:48.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.618+0000 7f38d9bce700 1 -- 172.21.15.5:0/878858777 >> 172.21.15.5:0/878858777 conn(0x7f38d4076d30 msgr2=0x7f38d4077140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:58:48.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.622+0000 7f38d9bce700 1 -- 172.21.15.5:0/878858777 shutdown_connections 2024-05-09T00:58:48.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.622+0000 7f38d9bce700 1 -- 172.21.15.5:0/878858777 wait complete. 2024-05-09T00:58:48.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.624+0000 7f38d9bce700 1 Processor -- start 2024-05-09T00:58:48.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.626+0000 7f38d9bce700 1 -- start start 2024-05-09T00:58:48.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.627+0000 7f38d9bce700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38d410a960 0x7f38d41a1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:48.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.627+0000 7f38d9bce700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38d410b2b0 0x7f38d41a1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:48.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.627+0000 7f38d9bce700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38d4110990 0x7f38d41a5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:48.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.627+0000 7f38d2ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38d410b2b0 0x7f38d41a1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:48.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.628+0000 7f38d2ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38d410b2b0 0x7f38d41a1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51768/0 (socket says 172.21.15.5:51768) 2024-05-09T00:58:48.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.628+0000 7f38d2ffd700 1 -- 172.21.15.5:0/3395152862 learned_addr learned my addr 172.21.15.5:0/3395152862 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:58:48.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.628+0000 7f38d37fe700 1 --2- 172.21.15.5:0/3395152862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38d410a960 0x7f38d41a1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:48.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.628+0000 7f38d9bce700 1 -- 172.21.15.5:0/3395152862 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f38d411ab80 con 0x7f38d410b2b0 2024-05-09T00:58:48.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.628+0000 7f38d9bce700 1 -- 172.21.15.5:0/3395152862 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f38d411aa00 con 0x7f38d4110990 2024-05-09T00:58:48.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.628+0000 7f38d9bce700 1 -- 172.21.15.5:0/3395152862 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f38d411ad00 con 0x7f38d410a960 2024-05-09T00:58:48.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.628+0000 7f38d3fff700 1 --2- 172.21.15.5:0/3395152862 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38d4110990 0x7f38d41a5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:48.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.629+0000 7f38d2ffd700 1 -- 172.21.15.5:0/3395152862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38d410a960 msgr2=0x7f38d41a1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:48.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.629+0000 7f38d2ffd700 1 --2- 172.21.15.5:0/3395152862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38d410a960 0x7f38d41a1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:48.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.629+0000 7f38d2ffd700 1 -- 172.21.15.5:0/3395152862 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38d4110990 msgr2=0x7f38d41a5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:48.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.629+0000 7f38d2ffd700 1 --2- 172.21.15.5:0/3395152862 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38d4110990 0x7f38d41a5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:48.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.629+0000 7f38d2ffd700 1 -- 172.21.15.5:0/3395152862 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38d41a63f0 con 0x7f38d410b2b0 2024-05-09T00:58:48.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.630+0000 7f38d2ffd700 1 --2- 172.21.15.5:0/3395152862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38d410b2b0 0x7f38d41a1b10 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f38c800b500 tx=0x7f38c800b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:58:48.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.630+0000 7f38d0ff9700 1 -- 172.21.15.5:0/3395152862 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c8015020 con 0x7f38d410b2b0 2024-05-09T00:58:48.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.631+0000 7f38d0ff9700 1 -- 172.21.15.5:0/3395152862 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38c8011ce0 con 0x7f38d410b2b0 2024-05-09T00:58:48.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.631+0000 7f38d0ff9700 1 -- 172.21.15.5:0/3395152862 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c8010d40 con 0x7f38d410b2b0 2024-05-09T00:58:48.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.631+0000 7f38d9bce700 1 -- 172.21.15.5:0/3395152862 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38d41a66e0 con 0x7f38d410b2b0 2024-05-09T00:58:48.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.631+0000 7f38d9bce700 1 -- 172.21.15.5:0/3395152862 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38d41ae000 con 0x7f38d410b2b0 2024-05-09T00:58:48.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.639+0000 7f38e85e4b80 1 Processor -- start 2024-05-09T00:58:48.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.640+0000 7f38e85e4b80 1 -- start start 2024-05-09T00:58:48.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.641+0000 7f38e85e4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56531f8e6160 0x56531f97f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:48.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.641+0000 7f38e85e4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56531f7fc4a0 0x56531f97fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:48.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.641+0000 7f38e85e4b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56531f9796a0 0x56531f979ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:48.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.641+0000 7f38d37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56531f7fc4a0 0x56531f97fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:48.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.641+0000 7f38d37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56531f7fc4a0 0x56531f97fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51780/0 (socket says 172.21.15.5:51780) 2024-05-09T00:58:48.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.642+0000 7f38d37fe700 1 -- 172.21.15.5:0/1398560777 learned_addr learned my addr 172.21.15.5:0/1398560777 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:58:48.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.642+0000 7f38e85e4b80 1 -- 172.21.15.5:0/1398560777 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56531f89bff0 con 0x56531f7fc4a0 2024-05-09T00:58:48.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.642+0000 7f38e85e4b80 1 -- 172.21.15.5:0/1398560777 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x56531f83a7f0 con 0x56531f8e6160 2024-05-09T00:58:48.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.642+0000 7f38e85e4b80 1 -- 172.21.15.5:0/1398560777 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56531f897fb0 con 0x56531f9796a0 2024-05-09T00:58:48.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.643+0000 7f38d37fe700 1 -- 172.21.15.5:0/1398560777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56531f9796a0 msgr2=0x56531f979ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:58:48.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.643+0000 7f38d2ffd700 1 --2- 172.21.15.5:0/1398560777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56531f9796a0 0x56531f979ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:48.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.645+0000 7f38d37fe700 1 --2- 172.21.15.5:0/1398560777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56531f9796a0 0x56531f979ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:48.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.645+0000 7f38d37fe700 1 -- 172.21.15.5:0/1398560777 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56531f8e6160 msgr2=0x56531f97f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:58:48.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.645+0000 7f38d37fe700 1 --2- 172.21.15.5:0/1398560777 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56531f8e6160 0x56531f97f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:48.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.645+0000 7f38d37fe700 1 -- 172.21.15.5:0/1398560777 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56531f832370 con 0x56531f7fc4a0 2024-05-09T00:58:48.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.646+0000 7f38d0ff9700 1 -- 172.21.15.5:0/3395152862 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f38c8018020 con 0x7f38d410b2b0 2024-05-09T00:58:48.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.648+0000 7f38d0ff9700 1 --2- 172.21.15.5:0/3395152862 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f38bc0b2a90 0x7f38bc0b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:48.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.648+0000 7f38d0ff9700 1 -- 172.21.15.5:0/3395152862 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f38c80e5000 con 0x7f38d410b2b0 2024-05-09T00:58:48.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.649+0000 7f38d2ffd700 1 --2- 172.21.15.5:0/1398560777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56531f9796a0 0x56531f979ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:58:48.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.650+0000 7f38d37fe700 1 --2- 172.21.15.5:0/1398560777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56531f7fc4a0 0x56531f97fa30 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f38c400bd30 tx=0x7f38c40106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e7e5d76fa5b4a54 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:58:48.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.652+0000 7f38d3fff700 1 --2- 172.21.15.5:0/3395152862 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f38bc0b2a90 0x7f38bc0b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:48.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.653+0000 7f38d3fff700 1 --2- 172.21.15.5:0/3395152862 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f38bc0b2a90 0x7f38bc0b4fa0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f38cc009e60 tx=0x7f38cc006020 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:58:48.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.661+0000 7f38b9ffb700 1 -- 172.21.15.5:0/1398560777 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c4004030 con 0x56531f7fc4a0 2024-05-09T00:58:48.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.661+0000 7f38b9ffb700 1 -- 172.21.15.5:0/1398560777 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38c4003680 con 0x56531f7fc4a0 2024-05-09T00:58:48.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.662+0000 7f38e85e4b80 1 -- 172.21.15.5:0/1398560777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56531f7fc4a0 msgr2=0x56531f97fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:48.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.662+0000 7f38e85e4b80 1 --2- 172.21.15.5:0/1398560777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56531f7fc4a0 0x56531f97fa30 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f38c400bd30 tx=0x7f38c40106b0 comp rx=0 tx=0).stop 2024-05-09T00:58:48.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.663+0000 7f38e85e4b80 1 -- 172.21.15.5:0/1398560777 shutdown_connections 2024-05-09T00:58:48.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.663+0000 7f38e85e4b80 1 --2- 172.21.15.5:0/1398560777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56531f9796a0 0x56531f979ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:48.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.663+0000 7f38e85e4b80 1 --2- 172.21.15.5:0/1398560777 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56531f8e6160 0x56531f97f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:48.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.663+0000 7f38e85e4b80 1 --2- 172.21.15.5:0/1398560777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56531f7fc4a0 0x56531f97fa30 secure :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f38c400bd30 tx=0x7f38c40106b0 comp rx=0 tx=0).stop 2024-05-09T00:58:48.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.663+0000 7f38e85e4b80 1 -- 172.21.15.5:0/1398560777 >> 172.21.15.5:0/1398560777 conn(0x56531f85c6a0 msgr2=0x56531f974400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:58:48.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.666+0000 7f38e85e4b80 1 -- 172.21.15.5:0/1398560777 shutdown_connections 2024-05-09T00:58:48.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.668+0000 7f38e85e4b80 1 -- 172.21.15.5:0/1398560777 wait complete. 2024-05-09T00:58:48.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.669+0000 7f38e85e4b80 1 -- start start 2024-05-09T00:58:48.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.670+0000 7f38e85e4b80 1 Processor -- start 2024-05-09T00:58:48.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.672+0000 7f38e85e4b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56531f8e6160 0x56531f983260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:48.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.673+0000 7f38e85e4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56531f9796a0 0x56531f9837a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:48.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.673+0000 7f38e85e4b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56531f983ce0 0x56531f98f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:48.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.673+0000 7f38e85e4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56531f717ba0 con 0x56531f9796a0 2024-05-09T00:58:48.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.673+0000 7f38e85e4b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x56531f856150 con 0x56531f8e6160 2024-05-09T00:58:48.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.673+0000 7f38e85e4b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56531f9861d0 con 0x56531f983ce0 2024-05-09T00:58:48.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.673+0000 7f38e85e4b80 10 client.0 _ll_register_callbacks cb 0x56531f70d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:58:48.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.673+0000 7f38d2ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56531f9796a0 0x56531f9837a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:48.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.673+0000 7f38d2ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56531f9796a0 0x56531f9837a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51794/0 (socket says 172.21.15.5:51794) 2024-05-09T00:58:48.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.673+0000 7f38d2ffd700 1 -- 172.21.15.5:0/435429102 learned_addr learned my addr 172.21.15.5:0/435429102 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:58:48.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.675+0000 7f38d37fe700 1 --2- 172.21.15.5:0/435429102 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56531f8e6160 0x56531f983260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:48.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.675+0000 7f38d3fff700 1 --2- 172.21.15.5:0/435429102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56531f983ce0 0x56531f98f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:48.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.676+0000 7f38d2ffd700 1 -- 172.21.15.5:0/435429102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56531f983ce0 msgr2=0x56531f98f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:58:48.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.676+0000 7f38d2ffd700 1 --2- 172.21.15.5:0/435429102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56531f983ce0 0x56531f98f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:48.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.676+0000 7f38d2ffd700 1 -- 172.21.15.5:0/435429102 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56531f8e6160 msgr2=0x56531f983260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:58:48.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.676+0000 7f38d2ffd700 1 --2- 172.21.15.5:0/435429102 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56531f8e6160 0x56531f983260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:48.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.676+0000 7f38d2ffd700 1 -- 172.21.15.5:0/435429102 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56531f717490 con 0x56531f9796a0 2024-05-09T00:58:48.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.676+0000 7f38d37fe700 1 --2- 172.21.15.5:0/435429102 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56531f8e6160 0x56531f983260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:58:48.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.681+0000 7f38d2ffd700 1 --2- 172.21.15.5:0/435429102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56531f9796a0 0x56531f9837a0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f38c80a8610 tx=0x7f38c80abd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ad9d80c3c87364f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:58:48.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.682+0000 7f38b2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:58:48.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.683+0000 7f38b2ffd700 1 -- 172.21.15.5:0/435429102 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c80ac590 con 0x56531f9796a0 2024-05-09T00:58:48.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.684+0000 7f38b2ffd700 1 -- 172.21.15.5:0/435429102 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f38c80a8b80 con 0x56531f9796a0 2024-05-09T00:58:48.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.684+0000 7f38e85e4b80 1 -- 172.21.15.5:0/435429102 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56531f8a99d0 con 0x56531f9796a0 2024-05-09T00:58:48.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.685+0000 7f38b2ffd700 1 -- 172.21.15.5:0/435429102 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f38c80cd050 con 0x56531f9796a0 2024-05-09T00:58:48.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.685+0000 7f38e85e4b80 10 client.16806 fetch_fsmap learned FSMap version 31 2024-05-09T00:58:48.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.685+0000 7f38e85e4b80 1 -- 172.21.15.5:0/435429102 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56531f9a22d0 con 0x56531f9796a0 2024-05-09T00:58:48.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.688+0000 7f38b2ffd700 1 -- 172.21.15.5:0/435429102 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56531f8a99d0 con 0x56531f9796a0 2024-05-09T00:58:48.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.688+0000 7f38b2ffd700 1 -- 172.21.15.5:0/435429102 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f38c8092b80 con 0x56531f9796a0 2024-05-09T00:58:48.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.688+0000 7f38e85e4b80 10 client.16806 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:58:48.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.689+0000 7f38e85e4b80 10 client.16806 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:58:48.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.689+0000 7f38e85e4b80 20 client.16806 populate_metadata read hostname 'smithi005' 2024-05-09T00:58:48.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.689+0000 7f38e85e4b80 1 --2- 172.21.15.5:0/435429102 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56531f9a4cf0 0x56531f9a71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:48.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.689+0000 7f38e85e4b80 4 client.16806 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:58:48.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.689+0000 7f38e85e4b80 1 -- 172.21.15.5:0/435429102 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56531f8a3a70 con 0x56531f9a4cf0 2024-05-09T00:58:48.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.721+0000 7f38d37fe700 1 --2- 172.21.15.5:0/435429102 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56531f9a4cf0 0x56531f9a71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:48.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.727+0000 7f38b2ffd700 10 client.16806 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:58:48.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.727+0000 7f38d37fe700 1 --2- 172.21.15.5:0/435429102 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56531f9a4cf0 0x56531f9a71c0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=831a5ce9e32caa04 server_cookie=d9290c26c718e784 in_seq=0 out_seq=0 2024-05-09T00:58:48.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.733+0000 7f38b2ffd700 1 -- 172.21.15.5:0/435429102 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56531f8a3a70 con 0x56531f9a4cf0 2024-05-09T00:58:48.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.733+0000 7f38b2ffd700 10 client.16806 handle_command_reply: tid=0 2024-05-09T00:58:48.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.733+0000 7f38e85e4b80 1 client.16806 shutdown 2024-05-09T00:58:48.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.734+0000 7f38e85e4b80 1 -- 172.21.15.5:0/435429102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56531f9796a0 msgr2=0x56531f9837a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:48.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.734+0000 7f38e85e4b80 1 --2- 172.21.15.5:0/435429102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56531f9796a0 0x56531f9837a0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f38c80a8610 tx=0x7f38c80abd00 comp rx=0 tx=0).stop 2024-05-09T00:58:48.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.736+0000 7f38e85e4b80 1 -- 172.21.15.5:0/435429102 shutdown_connections 2024-05-09T00:58:48.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.736+0000 7f38e85e4b80 1 --2- 172.21.15.5:0/435429102 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56531f9a4cf0 0x56531f9a71c0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:48.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.736+0000 7f38e85e4b80 1 --2- 172.21.15.5:0/435429102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56531f983ce0 0x56531f98f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:48.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.736+0000 7f38e85e4b80 1 --2- 172.21.15.5:0/435429102 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56531f8e6160 0x56531f983260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:48.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.736+0000 7f38e85e4b80 1 --2- 172.21.15.5:0/435429102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56531f9796a0 0x56531f9837a0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:48.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.736+0000 7f38b2ffd700 0 client.16806 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:58:48.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.736+0000 7f38e85e4b80 1 -- 172.21.15.5:0/435429102 >> 172.21.15.5:0/435429102 conn(0x56531f85c6a0 msgr2=0x56531f9741c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:58:48.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.739+0000 7f38e85e4b80 1 -- 172.21.15.5:0/435429102 shutdown_connections 2024-05-09T00:58:48.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.739+0000 7f38e85e4b80 1 -- 172.21.15.5:0/435429102 wait complete. 2024-05-09T00:58:48.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.740+0000 7f38e85e4b80 20 client.16806 trim_cache size 0 max 16384 2024-05-09T00:58:48.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.792+0000 7f38e85e4b80 1 Processor -- start 2024-05-09T00:58:48.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.792+0000 7f38e85e4b80 1 -- start start 2024-05-09T00:58:48.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.794+0000 7f38e85e4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56531f8e6160 0x56531f98f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:48.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.794+0000 7f38e85e4b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56531f9796a0 0x56531f9ac9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:48.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.795+0000 7f38d37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56531f8e6160 0x56531f98f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:48.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.795+0000 7f38d37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56531f8e6160 0x56531f98f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51796/0 (socket says 172.21.15.5:51796) 2024-05-09T00:58:48.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.795+0000 7f38d37fe700 1 -- 172.21.15.5:0/3267959379 learned_addr learned my addr 172.21.15.5:0/3267959379 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:58:48.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.795+0000 7f38e85e4b80 1 --2- 172.21.15.5:0/3267959379 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56531f983ce0 0x56531f9acf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:48.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.796+0000 7f38d2ffd700 1 --2- 172.21.15.5:0/3267959379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56531f9796a0 0x56531f9ac9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:48.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.796+0000 7f38e85e4b80 1 -- 172.21.15.5:0/3267959379 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f38c000d4f0 con 0x56531f8e6160 2024-05-09T00:58:48.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.796+0000 7f38e85e4b80 1 -- 172.21.15.5:0/3267959379 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f38c000f030 con 0x56531f983ce0 2024-05-09T00:58:48.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.797+0000 7f38e85e4b80 1 -- 172.21.15.5:0/3267959379 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f38c000dbc0 con 0x56531f9796a0 2024-05-09T00:58:48.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.797+0000 7f38d37fe700 1 -- 172.21.15.5:0/3267959379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56531f9796a0 msgr2=0x56531f9ac9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:58:48.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.797+0000 7f38d37fe700 1 --2- 172.21.15.5:0/3267959379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56531f9796a0 0x56531f9ac9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:48.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.797+0000 7f38d37fe700 1 -- 172.21.15.5:0/3267959379 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56531f983ce0 msgr2=0x56531f9acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:58:48.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.797+0000 7f38d37fe700 1 --2- 172.21.15.5:0/3267959379 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56531f983ce0 0x56531f9acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:48.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.798+0000 7f38d37fe700 1 -- 172.21.15.5:0/3267959379 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56531f97f050 con 0x56531f8e6160 2024-05-09T00:58:48.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.798+0000 7f38d37fe700 1 --2- 172.21.15.5:0/3267959379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56531f8e6160 0x56531f98f130 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f38c400bc90 tx=0x7f38c4010e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=986dd5f7443fe7f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:58:48.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.798+0000 7f38b97fa700 1 -- 172.21.15.5:0/3267959379 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c402a020 con 0x56531f8e6160 2024-05-09T00:58:48.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.798+0000 7f38b97fa700 1 -- 172.21.15.5:0/3267959379 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38c40050b0 con 0x56531f8e6160 2024-05-09T00:58:48.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.798+0000 7f38b97fa700 1 -- 172.21.15.5:0/3267959379 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c4026660 con 0x56531f8e6160 2024-05-09T00:58:48.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.801+0000 7f38e85e4b80 1 -- 172.21.15.5:0/3267959379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56531f8e6160 msgr2=0x56531f98f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:48.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.801+0000 7f38e85e4b80 1 --2- 172.21.15.5:0/3267959379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56531f8e6160 0x56531f98f130 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f38c400bc90 tx=0x7f38c4010e50 comp rx=0 tx=0).stop 2024-05-09T00:58:48.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.802+0000 7f38e85e4b80 1 -- 172.21.15.5:0/3267959379 shutdown_connections 2024-05-09T00:58:48.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.802+0000 7f38e85e4b80 1 --2- 172.21.15.5:0/3267959379 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56531f983ce0 0x56531f9acf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:48.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.802+0000 7f38e85e4b80 1 --2- 172.21.15.5:0/3267959379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56531f8e6160 0x56531f98f130 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:48.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.802+0000 7f38e85e4b80 1 --2- 172.21.15.5:0/3267959379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56531f9796a0 0x56531f9ac9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:48.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.802+0000 7f38e85e4b80 1 -- 172.21.15.5:0/3267959379 >> 172.21.15.5:0/3267959379 conn(0x56531f85c6a0 msgr2=0x56531f981b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:58:48.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.803+0000 7f38e85e4b80 1 -- 172.21.15.5:0/3267959379 shutdown_connections 2024-05-09T00:58:48.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.803+0000 7f38e85e4b80 1 -- 172.21.15.5:0/3267959379 wait complete. 2024-05-09T00:58:48.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.804+0000 7f38e85e4b80 1 -- start start 2024-05-09T00:58:48.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.805+0000 7f38e85e4b80 1 Processor -- start 2024-05-09T00:58:48.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.806+0000 7f38e85e4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56531f8e6160 0x56531f9ac2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:48.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.806+0000 7f38d37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56531f8e6160 0x56531f9ac2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:48.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.806+0000 7f38d37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56531f8e6160 0x56531f9ac2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51808/0 (socket says 172.21.15.5:51808) 2024-05-09T00:58:48.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.807+0000 7f38e85e4b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56531f9796a0 0x56531f9ac810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:48.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.807+0000 7f38d37fe700 1 -- 172.21.15.5:0/2101022972 learned_addr learned my addr 172.21.15.5:0/2101022972 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:58:48.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.807+0000 7f38e85e4b80 1 --2- 172.21.15.5:0/2101022972 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56531f983ce0 0x56531f9acd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:48.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.807+0000 7f38e85e4b80 1 -- 172.21.15.5:0/2101022972 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f38c000e960 con 0x56531f8e6160 2024-05-09T00:58:48.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.807+0000 7f38e85e4b80 1 -- 172.21.15.5:0/2101022972 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f38c000e290 con 0x56531f983ce0 2024-05-09T00:58:48.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.808+0000 7f38e85e4b80 1 -- 172.21.15.5:0/2101022972 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f38c000b490 con 0x56531f9796a0 2024-05-09T00:58:48.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.808+0000 7f38d3fff700 1 --2- 172.21.15.5:0/2101022972 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56531f983ce0 0x56531f9acd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:48.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.808+0000 7f38d37fe700 1 -- 172.21.15.5:0/2101022972 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56531f9796a0 msgr2=0x56531f9ac810 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:58:48.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.808+0000 7f38d37fe700 1 --2- 172.21.15.5:0/2101022972 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56531f9796a0 0x56531f9ac810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:48.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.808+0000 7f38d37fe700 1 -- 172.21.15.5:0/2101022972 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56531f983ce0 msgr2=0x56531f9acd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:58:48.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.808+0000 7f38d37fe700 1 --2- 172.21.15.5:0/2101022972 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56531f983ce0 0x56531f9acd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:48.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.808+0000 7f38d37fe700 1 -- 172.21.15.5:0/2101022972 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56531fa0e5c0 con 0x56531f8e6160 2024-05-09T00:58:48.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.809+0000 7f38d37fe700 1 --2- 172.21.15.5:0/2101022972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56531f8e6160 0x56531f9ac2d0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f38c40057c0 tx=0x7f38c4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7491a90e3bcaa1c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:58:48.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.809+0000 7f38b2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:58:48.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.809+0000 7f38e85e4b80 10 client.0 _ll_register_callbacks cb 0x56531f70d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:58:48.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.809+0000 7f38e85e4b80 1 -- 172.21.15.5:0/2101022972 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38c40050b0 con 0x56531f8e6160 2024-05-09T00:58:48.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.809+0000 7f38b2ffd700 1 -- 172.21.15.5:0/2101022972 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c4011c50 con 0x56531f8e6160 2024-05-09T00:58:48.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.809+0000 7f38b2ffd700 1 -- 172.21.15.5:0/2101022972 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f38c4011df0 con 0x56531f8e6160 2024-05-09T00:58:48.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.810+0000 7f38b2ffd700 1 -- 172.21.15.5:0/2101022972 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f38c403e050 con 0x56531f8e6160 2024-05-09T00:58:48.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.810+0000 7f38e85e4b80 10 client.16812 fetch_fsmap learned FSMap version 31 2024-05-09T00:58:48.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.810+0000 7f38e85e4b80 1 -- 172.21.15.5:0/2101022972 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({fsmap=31}) -- 0x56531fa0f660 con 0x56531f8e6160 2024-05-09T00:58:48.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.810+0000 7f38b2ffd700 1 -- 172.21.15.5:0/2101022972 <== mon.0 v2:172.21.15.5:3300/0 4 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f38c4026420 con 0x56531f8e6160 2024-05-09T00:58:48.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.810+0000 7f38e85e4b80 10 client.16812 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:58:48.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.810+0000 7f38e85e4b80 10 client.16812 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:58:48.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.810+0000 7f38e85e4b80 20 client.16812 populate_metadata read hostname 'smithi005' 2024-05-09T00:58:48.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.810+0000 7f38e85e4b80 1 --2- 172.21.15.5:0/2101022972 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56531fa0f9e0 0x56531fa11e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:48.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.810+0000 7f38e85e4b80 4 client.16812 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:58:48.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.810+0000 7f38e85e4b80 1 -- 172.21.15.5:0/2101022972 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56531f717970 con 0x56531fa0f9e0 2024-05-09T00:58:48.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.813+0000 7f38d2ffd700 1 --2- 172.21.15.5:0/2101022972 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56531fa0f9e0 0x56531fa11e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:48.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.815+0000 7f38b2ffd700 10 client.16812 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:58:48.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.815+0000 7f38d2ffd700 1 --2- 172.21.15.5:0/2101022972 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56531fa0f9e0 0x56531fa11e00 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=983ae39223b286 server_cookie=3f83485b09fa8166 in_seq=0 out_seq=0 2024-05-09T00:58:48.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.823+0000 7f38b2ffd700 1 -- 172.21.15.5:0/2101022972 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56531f717970 con 0x56531fa0f9e0 2024-05-09T00:58:48.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.823+0000 7f38b2ffd700 10 client.16812 handle_command_reply: tid=0 2024-05-09T00:58:48.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.823+0000 7f38e85e4b80 1 client.16812 shutdown 2024-05-09T00:58:48.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.823+0000 7f38e85e4b80 1 -- 172.21.15.5:0/2101022972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56531f8e6160 msgr2=0x56531f9ac2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:48.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.823+0000 7f38e85e4b80 1 --2- 172.21.15.5:0/2101022972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56531f8e6160 0x56531f9ac2d0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f38c40057c0 tx=0x7f38c4000bc0 comp rx=0 tx=0).stop 2024-05-09T00:58:48.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.823+0000 7f38e85e4b80 1 -- 172.21.15.5:0/2101022972 shutdown_connections 2024-05-09T00:58:48.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.824+0000 7f38e85e4b80 1 --2- 172.21.15.5:0/2101022972 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56531fa0f9e0 0x56531fa11e00 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:48.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.824+0000 7f38b2ffd700 0 client.16812 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:58:48.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.824+0000 7f38e85e4b80 1 --2- 172.21.15.5:0/2101022972 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56531f983ce0 0x56531f9acd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:48.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.824+0000 7f38e85e4b80 1 --2- 172.21.15.5:0/2101022972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56531f8e6160 0x56531f9ac2d0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:48.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.824+0000 7f38e85e4b80 1 --2- 172.21.15.5:0/2101022972 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56531f9796a0 0x56531f9ac810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:48.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.824+0000 7f38e85e4b80 1 -- 172.21.15.5:0/2101022972 >> 172.21.15.5:0/2101022972 conn(0x56531f85c6a0 msgr2=0x56531f981740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:58:48.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.824+0000 7f38e85e4b80 1 -- 172.21.15.5:0/2101022972 shutdown_connections 2024-05-09T00:58:48.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.824+0000 7f38e85e4b80 1 -- 172.21.15.5:0/2101022972 wait complete. 2024-05-09T00:58:48.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.824+0000 7f38e85e4b80 20 client.16812 trim_cache size 0 max 16384 2024-05-09T00:58:48.826 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-09T00:58:48.826 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-05-09T00:58:48.826 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-05-09T00:58:48.827 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-09T00:58:48.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.825+0000 7f38d9bce700 1 -- 172.21.15.5:0/3395152862 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f38bc0b2a90 msgr2=0x7f38bc0b4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:48.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.825+0000 7f38d9bce700 1 --2- 172.21.15.5:0/3395152862 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f38bc0b2a90 0x7f38bc0b4fa0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f38cc009e60 tx=0x7f38cc006020 comp rx=0 tx=0).stop 2024-05-09T00:58:48.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.825+0000 7f38d9bce700 1 -- 172.21.15.5:0/3395152862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38d410b2b0 msgr2=0x7f38d41a1b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:48.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.825+0000 7f38d9bce700 1 --2- 172.21.15.5:0/3395152862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38d410b2b0 0x7f38d41a1b10 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f38c800b500 tx=0x7f38c800b810 comp rx=0 tx=0).stop 2024-05-09T00:58:48.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.825+0000 7f38d9bce700 1 -- 172.21.15.5:0/3395152862 shutdown_connections 2024-05-09T00:58:48.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.825+0000 7f38d9bce700 1 --2- 172.21.15.5:0/3395152862 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f38bc0b2a90 0x7f38bc0b4fa0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:48.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.825+0000 7f38d9bce700 1 --2- 172.21.15.5:0/3395152862 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38d4110990 0x7f38d41a5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:48.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.825+0000 7f38d9bce700 1 --2- 172.21.15.5:0/3395152862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38d410a960 0x7f38d41a1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:48.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.825+0000 7f38d9bce700 1 --2- 172.21.15.5:0/3395152862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38d410b2b0 0x7f38d41a1b10 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:48.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.825+0000 7f38d9bce700 1 -- 172.21.15.5:0/3395152862 >> 172.21.15.5:0/3395152862 conn(0x7f38d4076d30 msgr2=0x7f38d4108980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:58:48.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.825+0000 7f38d9bce700 1 -- 172.21.15.5:0/3395152862 shutdown_connections 2024-05-09T00:58:48.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:48.825+0000 7f38d9bce700 1 -- 172.21.15.5:0/3395152862 wait complete. 2024-05-09T00:58:50.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:58:49 smithi131 ceph-mon[34547]: pgmap v745: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 858 MiB/s wr, 2.08k op/s 2024-05-09T00:58:50.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:58:49 smithi157 ceph-mon[34294]: pgmap v745: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 858 MiB/s wr, 2.08k op/s 2024-05-09T00:58:50.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:58:49 smithi005 ceph-mon[30365]: pgmap v745: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 858 MiB/s wr, 2.08k op/s 2024-05-09T00:58:50.361 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T00:58:50.361 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T00:58:51.174 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:58:52.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:58:51 smithi005 ceph-mon[30365]: pgmap v746: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 884 MiB/s wr, 2.19k op/s 2024-05-09T00:58:52.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:58:51 smithi157 ceph-mon[34294]: pgmap v746: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 884 MiB/s wr, 2.19k op/s 2024-05-09T00:58:52.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:58:51 smithi131 ceph-mon[34547]: pgmap v746: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 884 MiB/s wr, 2.19k op/s 2024-05-09T00:58:52.487 INFO:tasks.workunit.client.0.smithi005.stdout:stopping iogen 2024-05-09T00:58:52.488 INFO:tasks.workunit.client.0.smithi005.stderr:+ echo 'stopping iogen' 2024-05-09T00:58:52.488 INFO:tasks.workunit.client.0.smithi005.stderr:+ ./iogen -k 2024-05-09T00:58:53.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:58:53 smithi005 ceph-mon[30365]: pgmap v747: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 819 MiB/s wr, 2.10k op/s 2024-05-09T00:58:54.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:58:53 smithi157 ceph-mon[34294]: pgmap v747: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 819 MiB/s wr, 2.10k op/s 2024-05-09T00:58:54.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:58:53 smithi131 ceph-mon[34547]: pgmap v747: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 819 MiB/s wr, 2.10k op/s 2024-05-09T00:58:56.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:58:55 smithi005 ceph-mon[30365]: pgmap v748: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 866 MiB/s wr, 2.17k op/s 2024-05-09T00:58:56.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:58:55 smithi157 ceph-mon[34294]: pgmap v748: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 866 MiB/s wr, 2.17k op/s 2024-05-09T00:58:56.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:56.330+0000 7f04b1797700 1 -- 172.21.15.5:0/3050416342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04ac082340 msgr2=0x7f04ac082750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:56.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:56.330+0000 7f04b1797700 1 --2- 172.21.15.5:0/3050416342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04ac082340 0x7f04ac082750 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f049c0098f0 tx=0x7f049c009c00 comp rx=0 tx=0).stop 2024-05-09T00:58:56.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:56.331+0000 7f04b1797700 1 -- 172.21.15.5:0/3050416342 shutdown_connections 2024-05-09T00:58:56.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:56.331+0000 7f04b1797700 1 --2- 172.21.15.5:0/3050416342 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04ac07c010 0x7f04ac07c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:56.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:56.331+0000 7f04b1797700 1 --2- 172.21.15.5:0/3050416342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04ac084080 0x7f04ac081e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:56.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:56.331+0000 7f04b1797700 1 --2- 172.21.15.5:0/3050416342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04ac082340 0x7f04ac082750 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:56.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:56.331+0000 7f04b1797700 1 -- 172.21.15.5:0/3050416342 >> 172.21.15.5:0/3050416342 conn(0x7f04ac076b00 msgr2=0x7f04ac076f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:58:56.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:56.331+0000 7f04b1797700 1 -- 172.21.15.5:0/3050416342 shutdown_connections 2024-05-09T00:58:56.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:56.331+0000 7f04b1797700 1 -- 172.21.15.5:0/3050416342 wait complete. 2024-05-09T00:58:56.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:56.333+0000 7f04b1797700 1 Processor -- start 2024-05-09T00:58:56.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:56.333+0000 7f04b1797700 1 -- start start 2024-05-09T00:58:56.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:56.334+0000 7f04b1797700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04ac07c010 0x7f04ac075360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:56.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:56.334+0000 7f04b1797700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04ac082340 0x7f04ac06e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:56.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:56.334+0000 7f04b1797700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04ac084080 0x7f04ac06eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:56.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:56.334+0000 7f04b1797700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f04ac1231a0 con 0x7f04ac07c010 2024-05-09T00:58:56.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:56.334+0000 7f04b1797700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f04ac123020 con 0x7f04ac082340 2024-05-09T00:58:56.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:56.334+0000 7f04b1797700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f04ac123320 con 0x7f04ac084080 2024-05-09T00:58:56.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:56.335+0000 7f04abfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04ac07c010 0x7f04ac075360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:56.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:56.335+0000 7f04abfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04ac07c010 0x7f04ac075360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32774/0 (socket says 172.21.15.5:32774) 2024-05-09T00:58:56.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:56.335+0000 7f04abfff700 1 -- 172.21.15.5:0/4087699439 learned_addr learned my addr 172.21.15.5:0/4087699439 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:58:56.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:56.336+0000 7f04abfff700 1 -- 172.21.15.5:0/4087699439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04ac084080 msgr2=0x7f04ac06eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:56.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:56.336+0000 7f04abfff700 1 --2- 172.21.15.5:0/4087699439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04ac084080 0x7f04ac06eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:56.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:56.336+0000 7f04abfff700 1 -- 172.21.15.5:0/4087699439 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04ac082340 msgr2=0x7f04ac06e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:56.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:56.336+0000 7f04abfff700 1 --2- 172.21.15.5:0/4087699439 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04ac082340 0x7f04ac06e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:56.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:56.336+0000 7f04abfff700 1 -- 172.21.15.5:0/4087699439 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04ac072e90 con 0x7f04ac07c010 2024-05-09T00:58:56.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:56.337+0000 7f04abfff700 1 --2- 172.21.15.5:0/4087699439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04ac07c010 0x7f04ac075360 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f04a400c0c0 tx=0x7f04a400c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:58:56.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:56.339+0000 7f04a97fa700 1 -- 172.21.15.5:0/4087699439 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a400cdc0 con 0x7f04ac07c010 2024-05-09T00:58:56.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:56.339+0000 7f04a97fa700 1 -- 172.21.15.5:0/4087699439 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04a40075c0 con 0x7f04ac07c010 2024-05-09T00:58:56.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:56.339+0000 7f04a97fa700 1 -- 172.21.15.5:0/4087699439 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a40146c0 con 0x7f04ac07c010 2024-05-09T00:58:56.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:56.342+0000 7f04b1797700 1 -- 172.21.15.5:0/4087699439 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04ac073120 con 0x7f04ac07c010 2024-05-09T00:58:56.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:56.342+0000 7f04b1797700 1 -- 172.21.15.5:0/4087699439 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04ac0735e0 con 0x7f04ac07c010 2024-05-09T00:58:56.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:56.347+0000 7f0492ffd700 1 -- 172.21.15.5:0/4087699439 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f048c005330 con 0x7f04ac07c010 2024-05-09T00:58:56.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:56.351+0000 7f04a97fa700 1 -- 172.21.15.5:0/4087699439 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f04a401b070 con 0x7f04ac07c010 2024-05-09T00:58:56.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:58:55 smithi131 ceph-mon[34547]: pgmap v748: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 866 MiB/s wr, 2.17k op/s 2024-05-09T00:58:56.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:56.353+0000 7f04a97fa700 1 --2- 172.21.15.5:0/4087699439 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f04940b2a80 0x7f04940b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:56.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:56.353+0000 7f04a97fa700 1 -- 172.21.15.5:0/4087699439 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f04a40e1ba0 con 0x7f04ac07c010 2024-05-09T00:58:56.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:56.355+0000 7f04ab7fe700 1 --2- 172.21.15.5:0/4087699439 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f04940b2a80 0x7f04940b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:56.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:56.355+0000 7f04ab7fe700 1 --2- 172.21.15.5:0/4087699439 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f04940b2a80 0x7f04940b4f90 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f049c009230 tx=0x7f049c009f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:58:56.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:56.365+0000 7f04a97fa700 1 -- 172.21.15.5:0/4087699439 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f04a4050df0 con 0x7f04ac07c010 2024-05-09T00:58:57.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:57.049+0000 7f0492ffd700 1 -- 172.21.15.5:0/4087699439 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f048c005620 con 0x7f04ac07c010 2024-05-09T00:58:57.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:57.053+0000 7f04a97fa700 1 -- 172.21.15.5:0/4087699439 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11926 (secure 0 0 0) 0x7f04a4010040 con 0x7f04ac07c010 2024-05-09T00:58:57.057 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T00:58:57.058 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2685771887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2685771887},{"type":"v1","addr":"172.21.15.5:6835","nonce":2685771887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/254017535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":254017535},{"type":"v1","addr":"172.21.15.5:6837","nonce":254017535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T00:43:45.856066+0000","modified":"2024-05-09T00:52:11.233619+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24508,"mds_1":24499,"mds_2":24494,"mds_3":24493,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14598":{"gid":14598,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2109255228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2109255228},{"type":"v1","addr":"172.21.15.5:6839","nonce":2109255228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/3809966329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3809966329},{"type":"v1","addr":"172.21.15.5:6841","nonce":3809966329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6833/3272897742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3272897742},{"type":"v1","addr":"172.21.15.131:6833","nonce":3272897742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1283093898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1283093898},{"type":"v1","addr":"172.21.15.131:6835","nonce":1283093898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/2809525455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2809525455},{"type":"v1","addr":"172.21.15.157:6833","nonce":2809525455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.131:6837/1633006186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":1633006186},{"type":"v1","addr":"172.21.15.131:6837","nonce":1633006186}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.157:6835/832898273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":832898273},{"type":"v1","addr":"172.21.15.157:6835","nonce":832898273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.131:6839/2748669631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2748669631},{"type":"v1","addr":"172.21.15.131:6839","nonce":2748669631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.157:6837/744863009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":744863009},{"type":"v1","addr":"172.21.15.157:6837","nonce":744863009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.157:6839/2779860399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":2779860399},{"type":"v1","addr":"172.21.15.157:6839","nonce":2779860399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24508,24494,24499,24493,24467]},"id":1}]} 2024-05-09T00:58:57.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:57.060+0000 7f0492ffd700 1 -- 172.21.15.5:0/4087699439 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f04940b2a80 msgr2=0x7f04940b4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:57.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:57.060+0000 7f0492ffd700 1 --2- 172.21.15.5:0/4087699439 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f04940b2a80 0x7f04940b4f90 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f049c009230 tx=0x7f049c009f50 comp rx=0 tx=0).stop 2024-05-09T00:58:57.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:57.060+0000 7f0492ffd700 1 -- 172.21.15.5:0/4087699439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04ac07c010 msgr2=0x7f04ac075360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:57.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:57.060+0000 7f0492ffd700 1 --2- 172.21.15.5:0/4087699439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04ac07c010 0x7f04ac075360 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f04a400c0c0 tx=0x7f04a400c480 comp rx=0 tx=0).stop 2024-05-09T00:58:57.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:57.064+0000 7f0492ffd700 1 -- 172.21.15.5:0/4087699439 shutdown_connections 2024-05-09T00:58:57.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:57.064+0000 7f0492ffd700 1 --2- 172.21.15.5:0/4087699439 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f04940b2a80 0x7f04940b4f90 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:57.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:57.064+0000 7f0492ffd700 1 --2- 172.21.15.5:0/4087699439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04ac084080 0x7f04ac06eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:57.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:57.064+0000 7f0492ffd700 1 --2- 172.21.15.5:0/4087699439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04ac07c010 0x7f04ac075360 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:57.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:57.064+0000 7f0492ffd700 1 --2- 172.21.15.5:0/4087699439 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04ac082340 0x7f04ac06e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:57.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:57.064+0000 7f0492ffd700 1 -- 172.21.15.5:0/4087699439 >> 172.21.15.5:0/4087699439 conn(0x7f04ac076b00 msgr2=0x7f04ac07fb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:58:57.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:57.068+0000 7f0492ffd700 1 -- 172.21.15.5:0/4087699439 shutdown_connections 2024-05-09T00:58:57.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:58:57.068+0000 7f0492ffd700 1 -- 172.21.15.5:0/4087699439 wait complete. 2024-05-09T00:58:57.072 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 31 2024-05-09T00:58:58.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:58:57 smithi157 ceph-mon[34294]: pgmap v749: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 913 MiB/s wr, 2.30k op/s 2024-05-09T00:58:58.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:58:57 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/4087699439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:58:58.283 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:58:57 smithi005 ceph-mon[30365]: pgmap v749: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 913 MiB/s wr, 2.30k op/s 2024-05-09T00:58:58.286 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:58:57 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/4087699439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:58:58.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:58:57 smithi131 ceph-mon[34547]: pgmap v749: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 913 MiB/s wr, 2.30k op/s 2024-05-09T00:58:58.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:58:57 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/4087699439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:58:59.671 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-09T00:58:59.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.670+0000 7f281d2a7640 1 -- 172.21.15.5:0/3889271097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f281810b480 msgr2=0x7f281810b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:59.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.670+0000 7f281d2a7640 1 --2- 172.21.15.5:0/3889271097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f281810b480 0x7f281810b890 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f280801a910 tx=0x7f280803b470 comp rx=0 tx=0).stop 2024-05-09T00:58:59.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.670+0000 7f281d2a7640 1 -- 172.21.15.5:0/3889271097 shutdown_connections 2024-05-09T00:58:59.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.670+0000 7f281d2a7640 1 --2- 172.21.15.5:0/3889271097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28181101a0 0x7f28181105b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:59.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.670+0000 7f281d2a7640 1 --2- 172.21.15.5:0/3889271097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f281810b480 0x7f281810b890 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:59.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.670+0000 7f281d2a7640 1 --2- 172.21.15.5:0/3889271097 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f281810a550 0x7f281810af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:59.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.670+0000 7f281d2a7640 1 -- 172.21.15.5:0/3889271097 >> 172.21.15.5:0/3889271097 conn(0x7f2818070670 msgr2=0x7f2818072b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:58:59.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.670+0000 7f281d2a7640 1 -- 172.21.15.5:0/3889271097 shutdown_connections 2024-05-09T00:58:59.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.670+0000 7f281d2a7640 1 -- 172.21.15.5:0/3889271097 wait complete. 2024-05-09T00:58:59.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.674+0000 7f281d2a7640 1 Processor -- start 2024-05-09T00:58:59.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.675+0000 7f281d2a7640 1 -- start start 2024-05-09T00:58:59.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.675+0000 7f281d2a7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f281810a550 0x7f28181a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:59.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.675+0000 7f281d2a7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f281810b480 0x7f28181a1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:59.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.675+0000 7f281d2a7640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28181101a0 0x7f28181a8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:59.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.675+0000 7f281d2a7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2818113f10 con 0x7f281810a550 2024-05-09T00:58:59.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.675+0000 7f281d2a7640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2818113d90 con 0x7f28181101a0 2024-05-09T00:58:59.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.675+0000 7f281d2a7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2818114090 con 0x7f281810b480 2024-05-09T00:58:59.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.676+0000 7f2816ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f281810a550 0x7f28181a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:59.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.676+0000 7f2816ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f281810a550 0x7f28181a0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32796/0 (socket says 172.21.15.5:32796) 2024-05-09T00:58:59.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.676+0000 7f2816ffd640 1 -- 172.21.15.5:0/3174901286 learned_addr learned my addr 172.21.15.5:0/3174901286 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:58:59.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.677+0000 7f28167fc640 1 --2- 172.21.15.5:0/3174901286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f281810b480 0x7f28181a1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:59.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.679+0000 7f28177fe640 1 --2- 172.21.15.5:0/3174901286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28181101a0 0x7f28181a8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:59.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.679+0000 7f28167fc640 1 -- 172.21.15.5:0/3174901286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28181101a0 msgr2=0x7f28181a8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:59.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.679+0000 7f28167fc640 1 --2- 172.21.15.5:0/3174901286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28181101a0 0x7f28181a8010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:59.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.679+0000 7f28167fc640 1 -- 172.21.15.5:0/3174901286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f281810a550 msgr2=0x7f28181a0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:59.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.679+0000 7f28167fc640 1 --2- 172.21.15.5:0/3174901286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f281810a550 0x7f28181a0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:59.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.679+0000 7f28167fc640 1 -- 172.21.15.5:0/3174901286 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28181a8680 con 0x7f281810b480 2024-05-09T00:58:59.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.680+0000 7f28167fc640 1 --2- 172.21.15.5:0/3174901286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f281810b480 0x7f28181a1080 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f280803ba60 tx=0x7f2808004980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:58:59.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.684+0000 7f27fffff640 1 -- 172.21.15.5:0/3174901286 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280804f020 con 0x7f281810b480 2024-05-09T00:58:59.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.684+0000 7f281d2a7640 1 -- 172.21.15.5:0/3174901286 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28181a8910 con 0x7f281810b480 2024-05-09T00:58:59.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.684+0000 7f281d2a7640 1 -- 172.21.15.5:0/3174901286 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28181a8e80 con 0x7f281810b480 2024-05-09T00:58:59.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.684+0000 7f27fffff640 1 -- 172.21.15.5:0/3174901286 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2808004240 con 0x7f281810b480 2024-05-09T00:58:59.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.686+0000 7f27fffff640 1 -- 172.21.15.5:0/3174901286 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2808046070 con 0x7f281810b480 2024-05-09T00:58:59.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.690+0000 7f2824230740 1 Processor -- start 2024-05-09T00:58:59.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.690+0000 7f2824230740 1 -- start start 2024-05-09T00:58:59.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.691+0000 7f2824230740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595262471b0 0x5595265eb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:59.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.693+0000 7f27fffff640 1 -- 172.21.15.5:0/3174901286 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f2808052020 con 0x7f281810b480 2024-05-09T00:58:59.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.695+0000 7f27fffff640 1 --2- 172.21.15.5:0/3174901286 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f28000b27d0 0x7f28000b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:59.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.696+0000 7f27fffff640 1 -- 172.21.15.5:0/3174901286 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f2808113830 con 0x7f281810b480 2024-05-09T00:58:59.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.696+0000 7f2824230740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55952621e2f0 0x5595265eba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:59.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.696+0000 7f2824230740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5595262264d0 0x5595265f2980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:59.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.697+0000 7f2824230740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55952654f7c0 con 0x5595262471b0 2024-05-09T00:58:59.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.697+0000 7f2824230740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5595264eb0e0 con 0x55952621e2f0 2024-05-09T00:58:59.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.697+0000 7f2824230740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55952654b780 con 0x5595262264d0 2024-05-09T00:58:59.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.697+0000 7f28177fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5595262264d0 0x5595265f2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:59.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.697+0000 7f28177fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5595262264d0 0x5595265f2980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:33438/0 (socket says 172.21.15.5:33438) 2024-05-09T00:58:59.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.697+0000 7f28177fe640 1 -- 172.21.15.5:0/2830119871 learned_addr learned my addr 172.21.15.5:0/2830119871 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:58:59.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.697+0000 7f28167fc640 1 --2- 172.21.15.5:0/2830119871 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55952621e2f0 0x5595265eba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:59.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.699+0000 7f2816ffd640 1 --2- 172.21.15.5:0/3174901286 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f28000b27d0 0x7f28000b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:59.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.699+0000 7f28167fc640 1 -- 172.21.15.5:0/2830119871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5595262264d0 msgr2=0x5595265f2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:58:59.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.699+0000 7f28167fc640 1 --2- 172.21.15.5:0/2830119871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5595262264d0 0x5595265f2980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:59.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.699+0000 7f28167fc640 1 -- 172.21.15.5:0/2830119871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595262471b0 msgr2=0x5595265eb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:58:59.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.699+0000 7f28167fc640 1 --2- 172.21.15.5:0/2830119871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595262471b0 0x5595265eb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:59.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.699+0000 7f28167fc640 1 -- 172.21.15.5:0/2830119871 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595265147c0 con 0x55952621e2f0 2024-05-09T00:58:59.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.700+0000 7f28167fc640 1 --2- 172.21.15.5:0/2830119871 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55952621e2f0 0x5595265eba80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f28081112a0 tx=0x7f280803b470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37ae70fcdc3edc8c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:58:59.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.701+0000 7f27fd7fa640 1 -- 172.21.15.5:0/2830119871 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28080df170 con 0x55952621e2f0 2024-05-09T00:58:59.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.701+0000 7f27fd7fa640 1 -- 172.21.15.5:0/2830119871 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28080e3c20 con 0x55952621e2f0 2024-05-09T00:58:59.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.703+0000 7f2824230740 1 -- 172.21.15.5:0/2830119871 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55952621e2f0 msgr2=0x5595265eba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:59.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.703+0000 7f2824230740 1 --2- 172.21.15.5:0/2830119871 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55952621e2f0 0x5595265eba80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f28081112a0 tx=0x7f280803b470 comp rx=0 tx=0).stop 2024-05-09T00:58:59.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.704+0000 7f2824230740 1 -- 172.21.15.5:0/2830119871 shutdown_connections 2024-05-09T00:58:59.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.704+0000 7f2824230740 1 --2- 172.21.15.5:0/2830119871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5595262264d0 0x5595265f2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:59.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.704+0000 7f2824230740 1 --2- 172.21.15.5:0/2830119871 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55952621e2f0 0x5595265eba80 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:59.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.704+0000 7f2824230740 1 --2- 172.21.15.5:0/2830119871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595262471b0 0x5595265eb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:59.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.704+0000 7f2824230740 1 -- 172.21.15.5:0/2830119871 >> 172.21.15.5:0/2830119871 conn(0x55952623b300 msgr2=0x5595265e6e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:58:59.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.704+0000 7f2824230740 1 -- 172.21.15.5:0/2830119871 shutdown_connections 2024-05-09T00:58:59.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.705+0000 7f2824230740 1 -- 172.21.15.5:0/2830119871 wait complete. 2024-05-09T00:58:59.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.705+0000 7f2824230740 1 -- start start 2024-05-09T00:58:59.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.706+0000 7f2824230740 1 Processor -- start 2024-05-09T00:58:59.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.706+0000 7f2824230740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5595262264d0 0x5595265f4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:59.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.706+0000 7f2824230740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595262471b0 0x5595265f53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:59.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.706+0000 7f2824230740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559526205d80 0x5595265fc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:59.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.707+0000 7f2824230740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5595265f3190 con 0x5595262471b0 2024-05-09T00:58:59.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.707+0000 7f2824230740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5595264d69c0 con 0x5595262264d0 2024-05-09T00:58:59.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.707+0000 7f2824230740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5595265f3310 con 0x559526205d80 2024-05-09T00:58:59.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.707+0000 7f2824230740 10 client.0 _ll_register_callbacks cb 0x559526503020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:58:59.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.707+0000 7f28177fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559526205d80 0x5595265fc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:59.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.707+0000 7f28167fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595262471b0 0x5595265f53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:59.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.708+0000 7f28177fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559526205d80 0x5595265fc310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:33454/0 (socket says 172.21.15.5:33454) 2024-05-09T00:58:59.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.708+0000 7f28177fe640 1 -- 172.21.15.5:0/3529262768 learned_addr learned my addr 172.21.15.5:0/3529262768 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:58:59.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.709+0000 7f2816ffd640 1 --2- 172.21.15.5:0/3529262768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5595262264d0 0x5595265f4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:59.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.709+0000 7f2816ffd640 1 --2- 172.21.15.5:0/3174901286 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f28000b27d0 0x7f28000b4ce0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f280c006fd0 tx=0x7f280c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:58:59.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.709+0000 7f28177fe640 1 -- 172.21.15.5:0/3529262768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5595262264d0 msgr2=0x5595265f4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:58:59.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.709+0000 7f28177fe640 1 --2- 172.21.15.5:0/3529262768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5595262264d0 0x5595265f4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:59.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.709+0000 7f28177fe640 1 -- 172.21.15.5:0/3529262768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595262471b0 msgr2=0x5595265f53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:58:59.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.709+0000 7f28177fe640 1 --2- 172.21.15.5:0/3529262768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595262471b0 0x5595265f53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:59.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.709+0000 7f28177fe640 1 -- 172.21.15.5:0/3529262768 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595264fd1f0 con 0x559526205d80 2024-05-09T00:58:59.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.711+0000 7f28177fe640 1 --2- 172.21.15.5:0/3529262768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559526205d80 0x5595265fc310 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2810006d20 tx=0x7f2810010ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d60adcf76e4d2dc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:58:59.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.711+0000 7f27faff5640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-09T00:58:59.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.712+0000 7f27faff5640 1 -- 172.21.15.5:0/3529262768 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2810018020 con 0x559526205d80 2024-05-09T00:58:59.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.712+0000 7f2824230740 1 -- 172.21.15.5:0/3529262768 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55952655d3d0 con 0x559526205d80 2024-05-09T00:58:59.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.712+0000 7f27faff5640 1 -- 172.21.15.5:0/3529262768 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f281000f9a0 con 0x559526205d80 2024-05-09T00:58:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.715+0000 7f27faff5640 1 -- 172.21.15.5:0/3529262768 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2810023050 con 0x559526205d80 2024-05-09T00:58:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.715+0000 7f2824230740 10 client.26218 fetch_fsmap learned FSMap version 31 2024-05-09T00:58:59.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.715+0000 7f2824230740 1 -- 172.21.15.5:0/3529262768 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5595262b2790 con 0x559526205d80 2024-05-09T00:58:59.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.716+0000 7f27faff5640 1 -- 172.21.15.5:0/3529262768 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55952655d3d0 con 0x559526205d80 2024-05-09T00:58:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.716+0000 7f27faff5640 1 -- 172.21.15.5:0/3529262768 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f28100133b0 con 0x559526205d80 2024-05-09T00:58:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.718+0000 7f2824230740 10 client.26218 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:58:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.719+0000 7f2824230740 10 client.26218 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:58:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.719+0000 7f2824230740 20 client.26218 populate_metadata read hostname 'smithi005' 2024-05-09T00:58:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.719+0000 7f2824230740 1 --2- 172.21.15.5:0/3529262768 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5595261db1c0 0x559526602930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.719+0000 7f2824230740 4 client.26218 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:58:59.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.719+0000 7f2824230740 1 -- 172.21.15.5:0/3529262768 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559526557240 con 0x5595261db1c0 2024-05-09T00:58:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.723+0000 7f2816ffd640 1 --2- 172.21.15.5:0/3529262768 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5595261db1c0 0x559526602930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:59.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.742+0000 7f2816ffd640 1 --2- 172.21.15.5:0/3529262768 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5595261db1c0 0x559526602930 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3c44167c4369a2f server_cookie=6503962088ddb6c1 in_seq=0 out_seq=0 2024-05-09T00:58:59.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.742+0000 7f27faff5640 10 client.26218 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:58:59.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.748+0000 7f27faff5640 1 -- 172.21.15.5:0/3529262768 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559526557240 con 0x5595261db1c0 2024-05-09T00:58:59.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.748+0000 7f27faff5640 10 client.26218 handle_command_reply: tid=0 2024-05-09T00:58:59.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.749+0000 7f2824230740 1 client.26218 shutdown 2024-05-09T00:58:59.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.749+0000 7f2824230740 1 -- 172.21.15.5:0/3529262768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559526205d80 msgr2=0x5595265fc310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:59.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.749+0000 7f2824230740 1 --2- 172.21.15.5:0/3529262768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559526205d80 0x5595265fc310 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2810006d20 tx=0x7f2810010ba0 comp rx=0 tx=0).stop 2024-05-09T00:58:59.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.752+0000 7f2824230740 1 -- 172.21.15.5:0/3529262768 shutdown_connections 2024-05-09T00:58:59.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.752+0000 7f2824230740 1 --2- 172.21.15.5:0/3529262768 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5595261db1c0 0x559526602930 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:59.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.752+0000 7f2824230740 1 --2- 172.21.15.5:0/3529262768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559526205d80 0x5595265fc310 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:59.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.752+0000 7f2824230740 1 --2- 172.21.15.5:0/3529262768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595262471b0 0x5595265f53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:59.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.752+0000 7f2824230740 1 --2- 172.21.15.5:0/3529262768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5595262264d0 0x5595265f4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:59.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.752+0000 7f2824230740 1 -- 172.21.15.5:0/3529262768 >> 172.21.15.5:0/3529262768 conn(0x55952623b300 msgr2=0x5595265e9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:58:59.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.754+0000 7f27faff5640 0 client.26218 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:58:59.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.754+0000 7f2824230740 1 -- 172.21.15.5:0/3529262768 shutdown_connections 2024-05-09T00:58:59.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.754+0000 7f2824230740 1 -- 172.21.15.5:0/3529262768 wait complete. 2024-05-09T00:58:59.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.755+0000 7f2824230740 20 client.26218 trim_cache size 0 max 16384 2024-05-09T00:58:59.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.803+0000 7f2824230740 1 Processor -- start 2024-05-09T00:58:59.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.803+0000 7f2824230740 1 -- start start 2024-05-09T00:58:59.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.804+0000 7f2824230740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559526205d80 0x5595265f5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:59.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.804+0000 7f2824230740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5595262264d0 0x5595265fc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:59.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.804+0000 7f2824230740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595262471b0 0x5595265fc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:59.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.804+0000 7f2824230740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f280400e690 con 0x5595262471b0 2024-05-09T00:58:59.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.804+0000 7f2824230740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f28040101d0 con 0x559526205d80 2024-05-09T00:58:59.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.804+0000 7f2824230740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f280400ed60 con 0x5595262264d0 2024-05-09T00:58:59.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.805+0000 7f2816ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559526205d80 0x5595265f5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:59.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.805+0000 7f2816ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559526205d80 0x5595265f5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:44178/0 (socket says 172.21.15.5:44178) 2024-05-09T00:58:59.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.805+0000 7f2816ffd640 1 -- 172.21.15.5:0/4038841709 learned_addr learned my addr 172.21.15.5:0/4038841709 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:58:59.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.806+0000 7f28167fc640 1 --2- 172.21.15.5:0/4038841709 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5595262264d0 0x5595265fc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:59.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.806+0000 7f28177fe640 1 --2- 172.21.15.5:0/4038841709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595262471b0 0x5595265fc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:59.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.807+0000 7f28167fc640 1 -- 172.21.15.5:0/4038841709 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559526205d80 msgr2=0x5595265f5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:58:59.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.807+0000 7f28167fc640 1 --2- 172.21.15.5:0/4038841709 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559526205d80 0x5595265f5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:59.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.807+0000 7f28167fc640 1 -- 172.21.15.5:0/4038841709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595262471b0 msgr2=0x5595265fc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:58:59.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.807+0000 7f28167fc640 1 --2- 172.21.15.5:0/4038841709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595262471b0 0x5595265fc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:59.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.807+0000 7f28167fc640 1 -- 172.21.15.5:0/4038841709 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595263295f0 con 0x5595262264d0 2024-05-09T00:58:59.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.810+0000 7f28167fc640 1 --2- 172.21.15.5:0/4038841709 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5595262264d0 0x5595265fc2d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x5595265ec8e0 tx=0x7f2808000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6a5b67ab84acb29 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:58:59.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.819+0000 7f27fcff9640 1 -- 172.21.15.5:0/4038841709 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28080d6c80 con 0x5595262264d0 2024-05-09T00:58:59.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.819+0000 7f27fcff9640 1 -- 172.21.15.5:0/4038841709 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28080df170 con 0x5595262264d0 2024-05-09T00:58:59.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.819+0000 7f27fcff9640 1 -- 172.21.15.5:0/4038841709 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28080e3790 con 0x5595262264d0 2024-05-09T00:58:59.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.822+0000 7f2824230740 1 -- 172.21.15.5:0/4038841709 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5595262264d0 msgr2=0x5595265fc2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:58:59.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.822+0000 7f2824230740 1 --2- 172.21.15.5:0/4038841709 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5595262264d0 0x5595265fc2d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x5595265ec8e0 tx=0x7f2808000bc0 comp rx=0 tx=0).stop 2024-05-09T00:58:59.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.828+0000 7f2824230740 1 -- 172.21.15.5:0/4038841709 shutdown_connections 2024-05-09T00:58:59.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.828+0000 7f2824230740 1 --2- 172.21.15.5:0/4038841709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595262471b0 0x5595265fc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:59.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.828+0000 7f2824230740 1 --2- 172.21.15.5:0/4038841709 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5595262264d0 0x5595265fc2d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:59.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.828+0000 7f2824230740 1 --2- 172.21.15.5:0/4038841709 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559526205d80 0x5595265f5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:59.831 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.828+0000 7f2824230740 1 -- 172.21.15.5:0/4038841709 >> 172.21.15.5:0/4038841709 conn(0x55952623b300 msgr2=0x5595265f4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:58:59.831 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.830+0000 7f2824230740 1 -- 172.21.15.5:0/4038841709 shutdown_connections 2024-05-09T00:58:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.835+0000 7f2824230740 1 -- 172.21.15.5:0/4038841709 wait complete. 2024-05-09T00:58:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.835+0000 7f2824230740 1 -- start start 2024-05-09T00:58:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.835+0000 7f2824230740 1 Processor -- start 2024-05-09T00:58:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.836+0000 7f2824230740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559526205d80 0x5595265fc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.836+0000 7f2824230740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5595262264d0 0x5595265f9370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.836+0000 7f2824230740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595262471b0 0x5595265f98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.836+0000 7f2824230740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f280400fb00 con 0x5595262471b0 2024-05-09T00:58:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.836+0000 7f2824230740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f280400f430 con 0x5595262264d0 2024-05-09T00:58:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.836+0000 7f2824230740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f280400c670 con 0x559526205d80 2024-05-09T00:58:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.836+0000 7f2824230740 10 client.0 _ll_register_callbacks cb 0x7f28100066d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:58:59.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.838+0000 7f28167fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5595262264d0 0x5595265f9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:59.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.838+0000 7f28177fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595262471b0 0x5595265f98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:59.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.838+0000 7f28177fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595262471b0 0x5595265f98f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32838/0 (socket says 172.21.15.5:32838) 2024-05-09T00:58:59.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.838+0000 7f28177fe640 1 -- 172.21.15.5:0/2722215580 learned_addr learned my addr 172.21.15.5:0/2722215580 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:58:59.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.838+0000 7f2816ffd640 1 --2- 172.21.15.5:0/2722215580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559526205d80 0x5595265fc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:59.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.840+0000 7f2816ffd640 1 -- 172.21.15.5:0/2722215580 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5595262264d0 msgr2=0x5595265f9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:58:59.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.840+0000 7f2816ffd640 1 --2- 172.21.15.5:0/2722215580 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5595262264d0 0x5595265f9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:59.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.840+0000 7f2816ffd640 1 -- 172.21.15.5:0/2722215580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595262471b0 msgr2=0x5595265f98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:58:59.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.840+0000 7f2816ffd640 1 --2- 172.21.15.5:0/2722215580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595262471b0 0x5595265f98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:58:59.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.840+0000 7f2816ffd640 1 -- 172.21.15.5:0/2722215580 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559526513ed0 con 0x559526205d80 2024-05-09T00:58:59.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.842+0000 7f2816ffd640 1 --2- 172.21.15.5:0/2722215580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559526205d80 0x5595265fc7f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x5595265e7810 tx=0x7f280c0046e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b78fb9259b4c7ed1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:58:59.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.849+0000 7f27faff5640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-09T00:58:59.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.849+0000 7f27faff5640 1 -- 172.21.15.5:0/2722215580 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280c034060 con 0x559526205d80 2024-05-09T00:58:59.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.849+0000 7f27faff5640 1 -- 172.21.15.5:0/2722215580 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f280c026130 con 0x559526205d80 2024-05-09T00:58:59.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.849+0000 7f2824230740 1 -- 172.21.15.5:0/2722215580 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28080df170 con 0x559526205d80 2024-05-09T00:58:59.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.853+0000 7f27faff5640 1 -- 172.21.15.5:0/2722215580 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f280c01b040 con 0x559526205d80 2024-05-09T00:58:59.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.853+0000 7f2824230740 10 client.26224 fetch_fsmap learned FSMap version 31 2024-05-09T00:58:59.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.853+0000 7f2824230740 1 -- 172.21.15.5:0/2722215580 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559526669dc0 con 0x559526205d80 2024-05-09T00:58:59.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.855+0000 7f27faff5640 1 -- 172.21.15.5:0/2722215580 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28080df170 con 0x559526205d80 2024-05-09T00:58:59.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.855+0000 7f27faff5640 1 -- 172.21.15.5:0/2722215580 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f280c02f7b0 con 0x559526205d80 2024-05-09T00:58:59.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.855+0000 7f2824230740 10 client.26224 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:58:59.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.855+0000 7f2824230740 10 client.26224 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:58:59.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.855+0000 7f2824230740 20 client.26224 populate_metadata read hostname 'smithi005' 2024-05-09T00:58:59.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.855+0000 7f2824230740 1 --2- 172.21.15.5:0/2722215580 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55952666a750 0x55952666cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:58:59.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.855+0000 7f2824230740 4 client.26224 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:58:59.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.855+0000 7f2824230740 1 -- 172.21.15.5:0/2722215580 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5595265f3f10 con 0x55952666a750 2024-05-09T00:58:59.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.858+0000 7f28167fc640 1 --2- 172.21.15.5:0/2722215580 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55952666a750 0x55952666cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:58:59.861 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.860+0000 7f27faff5640 10 client.26224 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:58:59.861 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:58:59.860+0000 7f28167fc640 1 --2- 172.21.15.5:0/2722215580 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55952666a750 0x55952666cb70 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4533217f2b22d38 server_cookie=c8426bc7e05dddb8 in_seq=0 out_seq=0 2024-05-09T00:59:00.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:58:59 smithi157 ceph-mon[34294]: pgmap v750: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 896 MiB/s wr, 2.27k op/s 2024-05-09T00:59:00.295 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:58:59 smithi005 ceph-mon[30365]: pgmap v750: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 896 MiB/s wr, 2.27k op/s 2024-05-09T00:59:00.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:58:59 smithi131 ceph-mon[34547]: pgmap v750: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 896 MiB/s wr, 2.27k op/s 2024-05-09T00:59:00.557 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:59:02.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:59:01 smithi157 ceph-mon[34294]: pgmap v751: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 884 MiB/s wr, 2.14k op/s 2024-05-09T00:59:02.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:59:01 smithi005 ceph-mon[30365]: pgmap v751: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 884 MiB/s wr, 2.14k op/s 2024-05-09T00:59:02.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:59:01 smithi131 ceph-mon[34547]: pgmap v751: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 884 MiB/s wr, 2.14k op/s 2024-05-09T00:59:04.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:59:03 smithi157 ceph-mon[34294]: pgmap v752: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 832 MiB/s wr, 2.09k op/s 2024-05-09T00:59:04.297 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:59:03 smithi005 ceph-mon[30365]: pgmap v752: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 832 MiB/s wr, 2.09k op/s 2024-05-09T00:59:04.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:59:03 smithi131 ceph-mon[34547]: pgmap v752: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 832 MiB/s wr, 2.09k op/s 2024-05-09T00:59:06.159 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:59:05 smithi005 ceph-mon[30365]: pgmap v753: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 887 MiB/s wr, 2.13k op/s 2024-05-09T00:59:06.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:59:05 smithi157 ceph-mon[34294]: pgmap v753: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 887 MiB/s wr, 2.13k op/s 2024-05-09T00:59:06.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.231+0000 7ff13f953700 1 -- 172.21.15.5:0/715212465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff13810a930 msgr2=0x7ff13810ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:06.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.231+0000 7ff13f953700 1 --2- 172.21.15.5:0/715212465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff13810a930 0x7ff13810ad40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff128009990 tx=0x7ff128009ca0 comp rx=0 tx=0).stop 2024-05-09T00:59:06.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.233+0000 7ff13f953700 1 -- 172.21.15.5:0/715212465 shutdown_connections 2024-05-09T00:59:06.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.233+0000 7ff13f953700 1 --2- 172.21.15.5:0/715212465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff13811d930 0x7ff13811fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:06.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.233+0000 7ff13f953700 1 --2- 172.21.15.5:0/715212465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff13810a930 0x7ff13810ad40 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:06.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.233+0000 7ff13f953700 1 --2- 172.21.15.5:0/715212465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff13810b280 0x7ff13811d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:06.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.233+0000 7ff13f953700 1 -- 172.21.15.5:0/715212465 >> 172.21.15.5:0/715212465 conn(0x7ff13806f4d0 msgr2=0x7ff13806f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:59:06.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.236+0000 7ff13f953700 1 -- 172.21.15.5:0/715212465 shutdown_connections 2024-05-09T00:59:06.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.240+0000 7ff13f953700 1 -- 172.21.15.5:0/715212465 wait complete. 2024-05-09T00:59:06.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.242+0000 7ff13f953700 1 Processor -- start 2024-05-09T00:59:06.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.243+0000 7ff13f953700 1 -- start start 2024-05-09T00:59:06.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.244+0000 7ff13f953700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff13810a930 0x7ff1381af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:06.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.244+0000 7ff13f953700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff13810b280 0x7ff1381afee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:06.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.244+0000 7ff13f953700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff13811d930 0x7ff1381a9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:06.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.244+0000 7ff13f953700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff138123330 con 0x7ff13810a930 2024-05-09T00:59:06.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.244+0000 7ff13f953700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff1381231b0 con 0x7ff13810b280 2024-05-09T00:59:06.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.244+0000 7ff13f953700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff1381234b0 con 0x7ff13811d930 2024-05-09T00:59:06.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.245+0000 7ff13def0700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff13811d930 0x7ff1381a9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:06.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.245+0000 7ff13def0700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff13811d930 0x7ff1381a9b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:60566/0 (socket says 172.21.15.5:60566) 2024-05-09T00:59:06.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.245+0000 7ff13def0700 1 -- 172.21.15.5:0/2644324910 learned_addr learned my addr 172.21.15.5:0/2644324910 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:59:06.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.247+0000 7ff13d6ef700 1 --2- 172.21.15.5:0/2644324910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff13810a930 0x7ff1381af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:06.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.247+0000 7ff13def0700 1 -- 172.21.15.5:0/2644324910 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff13810b280 msgr2=0x7ff1381afee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:06.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.247+0000 7ff13def0700 1 --2- 172.21.15.5:0/2644324910 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff13810b280 0x7ff1381afee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:06.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.247+0000 7ff13def0700 1 -- 172.21.15.5:0/2644324910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff13810a930 msgr2=0x7ff1381af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:06.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.247+0000 7ff13def0700 1 --2- 172.21.15.5:0/2644324910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff13810a930 0x7ff1381af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:06.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.247+0000 7ff13def0700 1 -- 172.21.15.5:0/2644324910 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1381aa260 con 0x7ff13811d930 2024-05-09T00:59:06.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.248+0000 7ff13def0700 1 --2- 172.21.15.5:0/2644324910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff13811d930 0x7ff1381a9b30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff13000ada0 tx=0x7ff13000c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:59:06.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.250+0000 7ff12e7fc700 1 -- 172.21.15.5:0/2644324910 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff130017020 con 0x7ff13811d930 2024-05-09T00:59:06.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.250+0000 7ff12e7fc700 1 -- 172.21.15.5:0/2644324910 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff130007e30 con 0x7ff13811d930 2024-05-09T00:59:06.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.250+0000 7ff12e7fc700 1 -- 172.21.15.5:0/2644324910 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1300053e0 con 0x7ff13811d930 2024-05-09T00:59:06.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.251+0000 7ff13f953700 1 -- 172.21.15.5:0/2644324910 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1381aa550 con 0x7ff13811d930 2024-05-09T00:59:06.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.251+0000 7ff13f953700 1 -- 172.21.15.5:0/2644324910 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff138112bd0 con 0x7ff13811d930 2024-05-09T00:59:06.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.260+0000 7ff14e369b80 1 Processor -- start 2024-05-09T00:59:06.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.261+0000 7ff12e7fc700 1 -- 172.21.15.5:0/2644324910 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7ff130005580 con 0x7ff13811d930 2024-05-09T00:59:06.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.264+0000 7ff12e7fc700 1 --2- 172.21.15.5:0/2644324910 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ff1240b2a90 0x7ff1240b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:06.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.265+0000 7ff14e369b80 1 -- start start 2024-05-09T00:59:06.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.265+0000 7ff14e369b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105fe45160 0x56105fede4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:06.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.265+0000 7ff14e369b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105fd5b4a0 0x56105fedea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:06.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.265+0000 7ff14e369b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56105fed86a0 0x56105fed8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:06.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.265+0000 7ff14e369b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56105fdfaff0 con 0x56105fd5b4a0 2024-05-09T00:59:06.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.265+0000 7ff14e369b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x56105fd997f0 con 0x56105fed86a0 2024-05-09T00:59:06.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.265+0000 7ff14e369b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56105fdf6fb0 con 0x56105fe45160 2024-05-09T00:59:06.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.266+0000 7ff12e7fc700 1 -- 172.21.15.5:0/2644324910 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7ff1300e1d90 con 0x7ff13811d930 2024-05-09T00:59:06.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.267+0000 7ff13d6ef700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105fe45160 0x56105fede4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:06.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.267+0000 7ff13d6ef700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105fe45160 0x56105fede4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:60582/0 (socket says 172.21.15.5:60582) 2024-05-09T00:59:06.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.267+0000 7ff13d6ef700 1 -- 172.21.15.5:0/1041600722 learned_addr learned my addr 172.21.15.5:0/1041600722 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:59:06.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.269+0000 7ff13def0700 1 --2- 172.21.15.5:0/1041600722 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56105fed86a0 0x56105fed8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:06.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.269+0000 7ff13d6ef700 1 -- 172.21.15.5:0/1041600722 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56105fed86a0 msgr2=0x56105fed8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:59:06.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.269+0000 7ff13d6ef700 1 --2- 172.21.15.5:0/1041600722 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56105fed86a0 0x56105fed8ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:06.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.269+0000 7ff13d6ef700 1 -- 172.21.15.5:0/1041600722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105fd5b4a0 msgr2=0x56105fedea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:59:06.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.269+0000 7ff13d6ef700 1 --2- 172.21.15.5:0/1041600722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105fd5b4a0 0x56105fedea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:06.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.269+0000 7ff13d6ef700 1 -- 172.21.15.5:0/1041600722 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56105fd91370 con 0x56105fe45160 2024-05-09T00:59:06.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.270+0000 7ff13d6ef700 1 --2- 172.21.15.5:0/1041600722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105fe45160 0x56105fede4f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff128009fa0 tx=0x7ff128009510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=767c5a7c62419f45 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:59:06.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.271+0000 7ff13ceee700 1 --2- 172.21.15.5:0/2644324910 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ff1240b2a90 0x7ff1240b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:06.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.273+0000 7ff1237fe700 1 -- 172.21.15.5:0/1041600722 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff128005310 con 0x56105fe45160 2024-05-09T00:59:06.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.273+0000 7ff1237fe700 1 -- 172.21.15.5:0/1041600722 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff128003d10 con 0x56105fe45160 2024-05-09T00:59:06.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.273+0000 7ff1237fe700 1 -- 172.21.15.5:0/1041600722 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12800f460 con 0x56105fe45160 2024-05-09T00:59:06.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.275+0000 7ff13ceee700 1 --2- 172.21.15.5:0/2644324910 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ff1240b2a90 0x7ff1240b4fa0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x56105feddfb0 tx=0x7ff13400a640 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:59:06.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.277+0000 7ff14e369b80 1 -- 172.21.15.5:0/1041600722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105fe45160 msgr2=0x56105fede4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:06.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.277+0000 7ff14e369b80 1 --2- 172.21.15.5:0/1041600722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105fe45160 0x56105fede4f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff128009fa0 tx=0x7ff128009510 comp rx=0 tx=0).stop 2024-05-09T00:59:06.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.281+0000 7ff14e369b80 1 -- 172.21.15.5:0/1041600722 shutdown_connections 2024-05-09T00:59:06.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.281+0000 7ff14e369b80 1 --2- 172.21.15.5:0/1041600722 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56105fed86a0 0x56105fed8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:06.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.281+0000 7ff14e369b80 1 --2- 172.21.15.5:0/1041600722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105fe45160 0x56105fede4f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:06.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.281+0000 7ff14e369b80 1 --2- 172.21.15.5:0/1041600722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105fd5b4a0 0x56105fedea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:06.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.281+0000 7ff14e369b80 1 -- 172.21.15.5:0/1041600722 >> 172.21.15.5:0/1041600722 conn(0x56105fdbb6a0 msgr2=0x56105fed3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:59:06.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.283+0000 7ff14e369b80 1 -- 172.21.15.5:0/1041600722 shutdown_connections 2024-05-09T00:59:06.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.283+0000 7ff14e369b80 1 -- 172.21.15.5:0/1041600722 wait complete. 2024-05-09T00:59:06.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.284+0000 7ff14e369b80 1 -- start start 2024-05-09T00:59:06.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.285+0000 7ff14e369b80 1 Processor -- start 2024-05-09T00:59:06.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.285+0000 7ff14e369b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56105fd5b4a0 0x56105fee2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:06.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.285+0000 7ff14e369b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105fe45160 0x56105fee26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:06.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.285+0000 7ff14e369b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105fed86a0 0x56105fee2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:06.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.285+0000 7ff14e369b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56105fc76ba0 con 0x56105fed86a0 2024-05-09T00:59:06.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.286+0000 7ff14e369b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x56105fdb5150 con 0x56105fd5b4a0 2024-05-09T00:59:06.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.286+0000 7ff14e369b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56105fed94e0 con 0x56105fe45160 2024-05-09T00:59:06.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.286+0000 7ff14e369b80 10 client.0 _ll_register_callbacks cb 0x56105fc6c6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:59:06.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.287+0000 7ff13d6ef700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56105fd5b4a0 0x56105fee2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:06.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.287+0000 7ff13d6ef700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56105fd5b4a0 0x56105fee2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:58312/0 (socket says 172.21.15.5:58312) 2024-05-09T00:59:06.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.287+0000 7ff13d6ef700 1 -- 172.21.15.5:0/3413073098 learned_addr learned my addr 172.21.15.5:0/3413073098 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:59:06.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.287+0000 7ff13ceee700 1 --2- 172.21.15.5:0/3413073098 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105fe45160 0x56105fee26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:06.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.290+0000 7ff13d6ef700 1 -- 172.21.15.5:0/3413073098 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105fe45160 msgr2=0x56105fee26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:59:06.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.290+0000 7ff13d6ef700 1 --2- 172.21.15.5:0/3413073098 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105fe45160 0x56105fee26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:06.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.290+0000 7ff13d6ef700 1 -- 172.21.15.5:0/3413073098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105fed86a0 msgr2=0x56105fee2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:59:06.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.290+0000 7ff13d6ef700 1 --2- 172.21.15.5:0/3413073098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105fed86a0 0x56105fee2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:06.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.290+0000 7ff13d6ef700 1 -- 172.21.15.5:0/3413073098 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56105fc76490 con 0x56105fd5b4a0 2024-05-09T00:59:06.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.292+0000 7ff13d6ef700 1 --2- 172.21.15.5:0/3413073098 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56105fd5b4a0 0x56105fee2160 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff128009ce0 tx=0x7ff12800bc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93226558a1cff78c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:59:06.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.292+0000 7ff120ff9700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-09T00:59:06.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.293+0000 7ff120ff9700 1 -- 172.21.15.5:0/3413073098 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff128005520 con 0x56105fd5b4a0 2024-05-09T00:59:06.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.294+0000 7ff14e369b80 1 -- 172.21.15.5:0/3413073098 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff128003d10 con 0x56105fd5b4a0 2024-05-09T00:59:06.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.294+0000 7ff120ff9700 1 -- 172.21.15.5:0/3413073098 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7ff12802a480 con 0x56105fd5b4a0 2024-05-09T00:59:06.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.295+0000 7ff120ff9700 1 -- 172.21.15.5:0/3413073098 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff12800be80 con 0x56105fd5b4a0 2024-05-09T00:59:06.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.295+0000 7ff14e369b80 10 client.26249 fetch_fsmap learned FSMap version 31 2024-05-09T00:59:06.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.295+0000 7ff14e369b80 1 -- 172.21.15.5:0/3413073098 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56105feefb00 con 0x56105fd5b4a0 2024-05-09T00:59:06.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.296+0000 7ff120ff9700 1 -- 172.21.15.5:0/3413073098 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff128003d10 con 0x56105fd5b4a0 2024-05-09T00:59:06.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.298+0000 7ff120ff9700 1 -- 172.21.15.5:0/3413073098 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7ff12800f740 con 0x56105fd5b4a0 2024-05-09T00:59:06.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.299+0000 7ff14e369b80 10 client.26249 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:59:06.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.299+0000 7ff14e369b80 10 client.26249 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T00:59:06.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.299+0000 7ff14e369b80 20 client.26249 populate_metadata read hostname 'smithi005' 2024-05-09T00:59:06.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.299+0000 7ff14e369b80 1 --2- 172.21.15.5:0/3413073098 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x56105fef2140 0x56105fef4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:06.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.299+0000 7ff14e369b80 4 client.26249 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:59:06.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.299+0000 7ff14e369b80 1 -- 172.21.15.5:0/3413073098 --> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56105fe089d0 con 0x56105fef2140 2024-05-09T00:59:06.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.300+0000 7ff13ceee700 1 --2- 172.21.15.5:0/3413073098 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x56105fef2140 0x56105fef4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:06.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.305+0000 7ff120ff9700 10 client.26249 ms_handle_connect on v2:172.21.15.131:6836/1633006186 2024-05-09T00:59:06.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.305+0000 7ff13ceee700 1 --2- 172.21.15.5:0/3413073098 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x56105fef2140 0x56105fef4610 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fd95f5f18a37c435 server_cookie=57825cfe01463bc8 in_seq=0 out_seq=0 2024-05-09T00:59:06.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.307+0000 7ff120ff9700 1 -- 172.21.15.5:0/3413073098 <== mds.4 v2:172.21.15.131:6836/1633006186 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56105fe089d0 con 0x56105fef2140 2024-05-09T00:59:06.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.307+0000 7ff120ff9700 10 client.26249 handle_command_reply: tid=0 2024-05-09T00:59:06.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.307+0000 7ff14e369b80 1 client.26249 shutdown 2024-05-09T00:59:06.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.308+0000 7ff14e369b80 1 -- 172.21.15.5:0/3413073098 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56105fd5b4a0 msgr2=0x56105fee2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:06.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.308+0000 7ff14e369b80 1 --2- 172.21.15.5:0/3413073098 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56105fd5b4a0 0x56105fee2160 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff128009ce0 tx=0x7ff12800bc20 comp rx=0 tx=0).stop 2024-05-09T00:59:06.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.311+0000 7ff14e369b80 1 -- 172.21.15.5:0/3413073098 shutdown_connections 2024-05-09T00:59:06.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.311+0000 7ff14e369b80 1 --2- 172.21.15.5:0/3413073098 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x56105fef2140 0x56105fef4610 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:06.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.311+0000 7ff14e369b80 1 --2- 172.21.15.5:0/3413073098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105fed86a0 0x56105fee2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:06.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.311+0000 7ff14e369b80 1 --2- 172.21.15.5:0/3413073098 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56105fd5b4a0 0x56105fee2160 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:06.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.311+0000 7ff14e369b80 1 --2- 172.21.15.5:0/3413073098 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105fe45160 0x56105fee26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:06.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.311+0000 7ff120ff9700 0 client.26249 ms_handle_reset on v2:172.21.15.131:6836/1633006186 2024-05-09T00:59:06.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.311+0000 7ff14e369b80 1 -- 172.21.15.5:0/3413073098 >> 172.21.15.5:0/3413073098 conn(0x56105fdbb6a0 msgr2=0x56105fed31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:59:06.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.314+0000 7ff14e369b80 1 -- 172.21.15.5:0/3413073098 shutdown_connections 2024-05-09T00:59:06.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.315+0000 7ff14e369b80 1 -- 172.21.15.5:0/3413073098 wait complete. 2024-05-09T00:59:06.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.315+0000 7ff14e369b80 20 client.26249 trim_cache size 0 max 16384 2024-05-09T00:59:06.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.350+0000 7ff14e369b80 1 Processor -- start 2024-05-09T00:59:06.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.350+0000 7ff14e369b80 1 -- start start 2024-05-09T00:59:06.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.351+0000 7ff14e369b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105fd5b4a0 0x56105fed2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:06.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.352+0000 7ff14e369b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105fe45160 0x56105fee2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:06.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.352+0000 7ff13d6ef700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105fd5b4a0 0x56105fed2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:06.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.352+0000 7ff13d6ef700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105fd5b4a0 0x56105fed2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41272/0 (socket says 172.21.15.5:41272) 2024-05-09T00:59:06.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.353+0000 7ff13d6ef700 1 -- 172.21.15.5:0/2959832858 learned_addr learned my addr 172.21.15.5:0/2959832858 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:59:06.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.353+0000 7ff14e369b80 1 --2- 172.21.15.5:0/2959832858 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56105fed86a0 0x56105fee1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:06.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:59:05 smithi131 ceph-mon[34547]: pgmap v753: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 887 MiB/s wr, 2.13k op/s 2024-05-09T00:59:06.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.354+0000 7ff14e369b80 1 -- 172.21.15.5:0/2959832858 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff11c00d3f0 con 0x56105fd5b4a0 2024-05-09T00:59:06.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.356+0000 7ff13ceee700 1 --2- 172.21.15.5:0/2959832858 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105fe45160 0x56105fee2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:06.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.356+0000 7ff14e369b80 1 -- 172.21.15.5:0/2959832858 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff11c00ef30 con 0x56105fed86a0 2024-05-09T00:59:06.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.357+0000 7ff14e369b80 1 -- 172.21.15.5:0/2959832858 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff11c00dac0 con 0x56105fe45160 2024-05-09T00:59:06.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.357+0000 7ff13def0700 1 --2- 172.21.15.5:0/2959832858 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56105fed86a0 0x56105fee1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:06.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.358+0000 7ff13ceee700 1 -- 172.21.15.5:0/2959832858 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56105fed86a0 msgr2=0x56105fee1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:59:06.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.358+0000 7ff13ceee700 1 --2- 172.21.15.5:0/2959832858 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56105fed86a0 0x56105fee1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:06.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.358+0000 7ff13ceee700 1 -- 172.21.15.5:0/2959832858 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105fd5b4a0 msgr2=0x56105fed2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:59:06.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.358+0000 7ff13ceee700 1 --2- 172.21.15.5:0/2959832858 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105fd5b4a0 0x56105fed2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:06.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.358+0000 7ff13ceee700 1 -- 172.21.15.5:0/2959832858 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56105fed4d00 con 0x56105fe45160 2024-05-09T00:59:06.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.360+0000 7ff13ceee700 1 --2- 172.21.15.5:0/2959832858 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105fe45160 0x56105fee2a20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff134005950 tx=0x7ff134012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5df04e9e9fb37deb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:59:06.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.361+0000 7ff122ffd700 1 -- 172.21.15.5:0/2959832858 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff134028070 con 0x56105fe45160 2024-05-09T00:59:06.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.361+0000 7ff122ffd700 1 -- 172.21.15.5:0/2959832858 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff134004830 con 0x56105fe45160 2024-05-09T00:59:06.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.361+0000 7ff122ffd700 1 -- 172.21.15.5:0/2959832858 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff134005350 con 0x56105fe45160 2024-05-09T00:59:06.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.362+0000 7ff14e369b80 1 -- 172.21.15.5:0/2959832858 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105fe45160 msgr2=0x56105fee2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:06.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.362+0000 7ff14e369b80 1 --2- 172.21.15.5:0/2959832858 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105fe45160 0x56105fee2a20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff134005950 tx=0x7ff134012000 comp rx=0 tx=0).stop 2024-05-09T00:59:06.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.365+0000 7ff14e369b80 1 -- 172.21.15.5:0/2959832858 shutdown_connections 2024-05-09T00:59:06.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.365+0000 7ff14e369b80 1 --2- 172.21.15.5:0/2959832858 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56105fed86a0 0x56105fee1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:06.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.365+0000 7ff14e369b80 1 --2- 172.21.15.5:0/2959832858 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105fd5b4a0 0x56105fed2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:06.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.365+0000 7ff14e369b80 1 --2- 172.21.15.5:0/2959832858 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105fe45160 0x56105fee2a20 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:06.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.365+0000 7ff14e369b80 1 -- 172.21.15.5:0/2959832858 >> 172.21.15.5:0/2959832858 conn(0x56105fdbb6a0 msgr2=0x56105fee0a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:59:06.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.368+0000 7ff14e369b80 1 -- 172.21.15.5:0/2959832858 shutdown_connections 2024-05-09T00:59:06.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.368+0000 7ff14e369b80 1 -- 172.21.15.5:0/2959832858 wait complete. 2024-05-09T00:59:06.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.369+0000 7ff14e369b80 1 -- start start 2024-05-09T00:59:06.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.369+0000 7ff14e369b80 1 Processor -- start 2024-05-09T00:59:06.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.370+0000 7ff14e369b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56105fd5b4a0 0x56105fee5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:06.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.370+0000 7ff14e369b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105fe45160 0x56105fee5e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:06.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.371+0000 7ff14e369b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105fed86a0 0x56105ff54980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:06.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.371+0000 7ff14e369b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff11c00e860 con 0x56105fed86a0 2024-05-09T00:59:06.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.371+0000 7ff13ceee700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105fe45160 0x56105fee5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:06.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.371+0000 7ff13ceee700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105fe45160 0x56105fee5e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:60612/0 (socket says 172.21.15.5:60612) 2024-05-09T00:59:06.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.371+0000 7ff13ceee700 1 -- 172.21.15.5:0/4248702334 learned_addr learned my addr 172.21.15.5:0/4248702334 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:59:06.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.371+0000 7ff13d6ef700 1 --2- 172.21.15.5:0/4248702334 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56105fd5b4a0 0x56105fee5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:06.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.371+0000 7ff14e369b80 1 -- 172.21.15.5:0/4248702334 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff11c00e190 con 0x56105fd5b4a0 2024-05-09T00:59:06.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.371+0000 7ff14e369b80 1 -- 172.21.15.5:0/4248702334 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff11c00b440 con 0x56105fe45160 2024-05-09T00:59:06.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.372+0000 7ff14e369b80 10 client.0 _ll_register_callbacks cb 0x56105fc6c6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:59:06.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.373+0000 7ff13ceee700 1 -- 172.21.15.5:0/4248702334 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56105fd5b4a0 msgr2=0x56105fee5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:59:06.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.373+0000 7ff13ceee700 1 --2- 172.21.15.5:0/4248702334 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56105fd5b4a0 0x56105fee5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:06.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.373+0000 7ff13ceee700 1 -- 172.21.15.5:0/4248702334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105fed86a0 msgr2=0x56105ff54980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:59:06.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.373+0000 7ff13ceee700 1 --2- 172.21.15.5:0/4248702334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105fed86a0 0x56105ff54980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:06.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.373+0000 7ff13ceee700 1 -- 172.21.15.5:0/4248702334 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56105ff54ec0 con 0x56105fe45160 2024-05-09T00:59:06.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.374+0000 7ff13ceee700 1 --2- 172.21.15.5:0/4248702334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105fe45160 0x56105fee5e80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff13400a840 tx=0x7ff13401fe00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92fc38d00dfd3e70 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:59:06.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.374+0000 7ff120ff9700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-09T00:59:06.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.375+0000 7ff120ff9700 1 -- 172.21.15.5:0/4248702334 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff134028040 con 0x56105fe45160 2024-05-09T00:59:06.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.375+0000 7ff14e369b80 1 -- 172.21.15.5:0/4248702334 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff134004830 con 0x56105fe45160 2024-05-09T00:59:06.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.375+0000 7ff120ff9700 1 -- 172.21.15.5:0/4248702334 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7ff13401dc80 con 0x56105fe45160 2024-05-09T00:59:06.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.376+0000 7ff120ff9700 1 -- 172.21.15.5:0/4248702334 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff13403c030 con 0x56105fe45160 2024-05-09T00:59:06.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.376+0000 7ff14e369b80 10 client.26239 fetch_fsmap learned FSMap version 31 2024-05-09T00:59:06.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.376+0000 7ff14e369b80 1 -- 172.21.15.5:0/4248702334 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56105ff5c1b0 con 0x56105fe45160 2024-05-09T00:59:06.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.377+0000 7ff120ff9700 1 -- 172.21.15.5:0/4248702334 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff134004830 con 0x56105fe45160 2024-05-09T00:59:06.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.379+0000 7ff120ff9700 1 -- 172.21.15.5:0/4248702334 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7ff134024ce0 con 0x56105fe45160 2024-05-09T00:59:06.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.379+0000 7ff14e369b80 10 client.26239 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:59:06.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.379+0000 7ff14e369b80 10 client.26239 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T00:59:06.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.379+0000 7ff14e369b80 20 client.26239 populate_metadata read hostname 'smithi005' 2024-05-09T00:59:06.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.379+0000 7ff14e369b80 1 --2- 172.21.15.5:0/4248702334 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x56105ff5c7a0 0x56105ff5ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:06.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.379+0000 7ff14e369b80 4 client.26239 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:59:06.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.379+0000 7ff14e369b80 1 -- 172.21.15.5:0/4248702334 --> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56105fe02a70 con 0x56105ff5c7a0 2024-05-09T00:59:06.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.380+0000 7ff13d6ef700 1 --2- 172.21.15.5:0/4248702334 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x56105ff5c7a0 0x56105ff5ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:06.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.386+0000 7ff13d6ef700 1 --2- 172.21.15.5:0/4248702334 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x56105ff5c7a0 0x56105ff5ebc0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d4e310c7bb3c9cac server_cookie=9c4d78c05331d504 in_seq=0 out_seq=0 2024-05-09T00:59:06.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.386+0000 7ff120ff9700 10 client.26239 ms_handle_connect on v2:172.21.15.131:6836/1633006186 2024-05-09T00:59:06.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.386+0000 7ff120ff9700 1 -- 172.21.15.5:0/4248702334 <== mds.4 v2:172.21.15.131:6836/1633006186 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56105fe02a70 con 0x56105ff5c7a0 2024-05-09T00:59:06.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.386+0000 7ff120ff9700 10 client.26239 handle_command_reply: tid=0 2024-05-09T00:59:06.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.387+0000 7ff14e369b80 1 client.26239 shutdown 2024-05-09T00:59:06.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.394+0000 7ff14e369b80 1 -- 172.21.15.5:0/4248702334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105fe45160 msgr2=0x56105fee5e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:06.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.394+0000 7ff14e369b80 1 --2- 172.21.15.5:0/4248702334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105fe45160 0x56105fee5e80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff13400a840 tx=0x7ff13401fe00 comp rx=0 tx=0).stop 2024-05-09T00:59:06.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.395+0000 7ff14e369b80 1 -- 172.21.15.5:0/4248702334 shutdown_connections 2024-05-09T00:59:06.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.395+0000 7ff14e369b80 1 --2- 172.21.15.5:0/4248702334 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x56105ff5c7a0 0x56105ff5ebc0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:06.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.395+0000 7ff14e369b80 1 --2- 172.21.15.5:0/4248702334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105fed86a0 0x56105ff54980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:06.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.395+0000 7ff14e369b80 1 --2- 172.21.15.5:0/4248702334 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56105fd5b4a0 0x56105fee5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:06.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.395+0000 7ff14e369b80 1 --2- 172.21.15.5:0/4248702334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105fe45160 0x56105fee5e80 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:06.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.395+0000 7ff14e369b80 1 -- 172.21.15.5:0/4248702334 >> 172.21.15.5:0/4248702334 conn(0x56105fdbb6a0 msgr2=0x56105fed7690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:59:06.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.395+0000 7ff120ff9700 0 client.26239 ms_handle_reset on v2:172.21.15.131:6836/1633006186 2024-05-09T00:59:06.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.402+0000 7ff14e369b80 1 -- 172.21.15.5:0/4248702334 shutdown_connections 2024-05-09T00:59:06.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.402+0000 7ff14e369b80 1 -- 172.21.15.5:0/4248702334 wait complete. 2024-05-09T00:59:06.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.402+0000 7ff14e369b80 20 client.26239 trim_cache size 0 max 16384 2024-05-09T00:59:06.404 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-09T00:59:06.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.404+0000 7ff13f953700 1 -- 172.21.15.5:0/2644324910 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ff1240b2a90 msgr2=0x7ff1240b4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:06.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.404+0000 7ff13f953700 1 --2- 172.21.15.5:0/2644324910 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ff1240b2a90 0x7ff1240b4fa0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x56105feddfb0 tx=0x7ff13400a640 comp rx=0 tx=0).stop 2024-05-09T00:59:06.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.404+0000 7ff13f953700 1 -- 172.21.15.5:0/2644324910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff13811d930 msgr2=0x7ff1381a9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:06.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.405+0000 7ff13f953700 1 --2- 172.21.15.5:0/2644324910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff13811d930 0x7ff1381a9b30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff13000ada0 tx=0x7ff13000c330 comp rx=0 tx=0).stop 2024-05-09T00:59:06.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.406+0000 7ff13f953700 1 -- 172.21.15.5:0/2644324910 shutdown_connections 2024-05-09T00:59:06.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.406+0000 7ff13f953700 1 --2- 172.21.15.5:0/2644324910 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7ff1240b2a90 0x7ff1240b4fa0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:06.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.406+0000 7ff13f953700 1 --2- 172.21.15.5:0/2644324910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff13811d930 0x7ff1381a9b30 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:06.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.406+0000 7ff13f953700 1 --2- 172.21.15.5:0/2644324910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff13810a930 0x7ff1381af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:06.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.406+0000 7ff13f953700 1 --2- 172.21.15.5:0/2644324910 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff13810b280 0x7ff1381afee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:06.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.406+0000 7ff13f953700 1 -- 172.21.15.5:0/2644324910 >> 172.21.15.5:0/2644324910 conn(0x7ff13806f4d0 msgr2=0x7ff13811b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:59:06.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.407+0000 7ff13f953700 1 -- 172.21.15.5:0/2644324910 shutdown_connections 2024-05-09T00:59:06.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:06.407+0000 7ff13f953700 1 -- 172.21.15.5:0/2644324910 wait complete. 2024-05-09T00:59:08.277 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:59:07 smithi005 ceph-mon[30365]: pgmap v754: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 909 MiB/s wr, 2.18k op/s 2024-05-09T00:59:08.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:59:07 smithi131 ceph-mon[34547]: pgmap v754: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 909 MiB/s wr, 2.18k op/s 2024-05-09T00:59:08.372 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-09T00:59:08.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:59:07 smithi157 ceph-mon[34294]: pgmap v754: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 909 MiB/s wr, 2.18k op/s 2024-05-09T00:59:09.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:09.092+0000 7f27faff5640 1 -- 172.21.15.5:0/2722215580 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5595265f3f10 con 0x55952666a750 2024-05-09T00:59:09.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:09.092+0000 7f27faff5640 10 client.26224 handle_command_reply: tid=0 2024-05-09T00:59:09.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:09.092+0000 7f2824230740 1 client.26224 shutdown 2024-05-09T00:59:09.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:09.092+0000 7f2824230740 1 -- 172.21.15.5:0/2722215580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559526205d80 msgr2=0x5595265fc7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:09.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:09.092+0000 7f2824230740 1 --2- 172.21.15.5:0/2722215580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559526205d80 0x5595265fc7f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x5595265e7810 tx=0x7f280c0046e0 comp rx=0 tx=0).stop 2024-05-09T00:59:09.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:09.092+0000 7f2824230740 1 -- 172.21.15.5:0/2722215580 shutdown_connections 2024-05-09T00:59:09.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:09.092+0000 7f2824230740 1 --2- 172.21.15.5:0/2722215580 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55952666a750 0x55952666cb70 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:09.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:09.092+0000 7f2824230740 1 --2- 172.21.15.5:0/2722215580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595262471b0 0x5595265f98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:09.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:09.092+0000 7f2824230740 1 --2- 172.21.15.5:0/2722215580 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5595262264d0 0x5595265f9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:09.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:09.092+0000 7f2824230740 1 --2- 172.21.15.5:0/2722215580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559526205d80 0x5595265fc7f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:09.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:09.092+0000 7f2824230740 1 -- 172.21.15.5:0/2722215580 >> 172.21.15.5:0/2722215580 conn(0x55952623b300 msgr2=0x5595265e6810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:59:09.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:09.093+0000 7f27faff5640 0 client.26224 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:59:09.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:09.093+0000 7f2824230740 1 -- 172.21.15.5:0/2722215580 shutdown_connections 2024-05-09T00:59:09.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:09.093+0000 7f2824230740 1 -- 172.21.15.5:0/2722215580 wait complete. 2024-05-09T00:59:09.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:09.093+0000 7f2824230740 20 client.26224 trim_cache size 0 max 16384 2024-05-09T00:59:09.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-09T00:59:09.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-09T00:59:09.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-09T00:59:09.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:09.096+0000 7f281d2a7640 1 -- 172.21.15.5:0/3174901286 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f28000b27d0 msgr2=0x7f28000b4ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:09.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:09.096+0000 7f281d2a7640 1 --2- 172.21.15.5:0/3174901286 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f28000b27d0 0x7f28000b4ce0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f280c006fd0 tx=0x7f280c009190 comp rx=0 tx=0).stop 2024-05-09T00:59:09.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:09.096+0000 7f281d2a7640 1 -- 172.21.15.5:0/3174901286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f281810b480 msgr2=0x7f28181a1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:09.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:09.096+0000 7f281d2a7640 1 --2- 172.21.15.5:0/3174901286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f281810b480 0x7f28181a1080 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f280803ba60 tx=0x7f2808004980 comp rx=0 tx=0).stop 2024-05-09T00:59:09.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:09.111+0000 7f281d2a7640 1 -- 172.21.15.5:0/3174901286 shutdown_connections 2024-05-09T00:59:09.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:09.111+0000 7f281d2a7640 1 --2- 172.21.15.5:0/3174901286 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f28000b27d0 0x7f28000b4ce0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:09.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:09.111+0000 7f281d2a7640 1 --2- 172.21.15.5:0/3174901286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28181101a0 0x7f28181a8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:09.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:09.111+0000 7f281d2a7640 1 --2- 172.21.15.5:0/3174901286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f281810b480 0x7f28181a1080 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:09.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:09.111+0000 7f281d2a7640 1 --2- 172.21.15.5:0/3174901286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f281810a550 0x7f28181a0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:09.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:09.111+0000 7f281d2a7640 1 -- 172.21.15.5:0/3174901286 >> 172.21.15.5:0/3174901286 conn(0x7f2818070670 msgr2=0x7f281810e190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:59:09.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:09.112+0000 7f281d2a7640 1 -- 172.21.15.5:0/3174901286 shutdown_connections 2024-05-09T00:59:09.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:09.112+0000 7f281d2a7640 1 -- 172.21.15.5:0/3174901286 wait complete. 2024-05-09T00:59:09.359 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:59:10.244 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:59:09 smithi005 ceph-mon[30365]: pgmap v755: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 871 MiB/s wr, 2.09k op/s 2024-05-09T00:59:10.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:59:09 smithi131 ceph-mon[34547]: pgmap v755: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 871 MiB/s wr, 2.09k op/s 2024-05-09T00:59:10.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:59:09 smithi157 ceph-mon[34294]: pgmap v755: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 871 MiB/s wr, 2.09k op/s 2024-05-09T00:59:12.295 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:59:11 smithi005 ceph-mon[30365]: pgmap v756: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 873 MiB/s wr, 1.93k op/s 2024-05-09T00:59:12.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:59:11 smithi131 ceph-mon[34547]: pgmap v756: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 873 MiB/s wr, 1.93k op/s 2024-05-09T00:59:12.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:59:11 smithi157 ceph-mon[34294]: pgmap v756: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 873 MiB/s wr, 1.93k op/s 2024-05-09T00:59:14.202 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:59:13 smithi005 ceph-mon[30365]: pgmap v757: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 842 MiB/s wr, 1.73k op/s 2024-05-09T00:59:14.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:59:13 smithi131 ceph-mon[34547]: pgmap v757: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 842 MiB/s wr, 1.73k op/s 2024-05-09T00:59:14.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:59:13 smithi157 ceph-mon[34294]: pgmap v757: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 842 MiB/s wr, 1.73k op/s 2024-05-09T00:59:15.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.176+0000 7fa601463700 1 -- 172.21.15.5:0/3332328701 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f8003c00 con 0x7fa5fc11d930 2024-05-09T00:59:15.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.177+0000 7fa6046c9700 1 -- 172.21.15.5:0/3332328701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5fc11d930 msgr2=0x7fa5fc11fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:15.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.177+0000 7fa6046c9700 1 --2- 172.21.15.5:0/3332328701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5fc11d930 0x7fa5fc11fd50 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8009990 tx=0x7fa5f8009ca0 comp rx=0 tx=0).stop 2024-05-09T00:59:15.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.177+0000 7fa6046c9700 1 -- 172.21.15.5:0/3332328701 shutdown_connections 2024-05-09T00:59:15.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.177+0000 7fa6046c9700 1 --2- 172.21.15.5:0/3332328701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5fc11d930 0x7fa5fc11fd50 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:15.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.177+0000 7fa6046c9700 1 --2- 172.21.15.5:0/3332328701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5fc10a7c0 0x7fa5fc10abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:15.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.177+0000 7fa6046c9700 1 --2- 172.21.15.5:0/3332328701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5fc10b110 0x7fa5fc11d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:15.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.177+0000 7fa6046c9700 1 -- 172.21.15.5:0/3332328701 >> 172.21.15.5:0/3332328701 conn(0x7fa5fc06f290 msgr2=0x7fa5fc06f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:59:15.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.178+0000 7fa6046c9700 1 -- 172.21.15.5:0/3332328701 shutdown_connections 2024-05-09T00:59:15.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.178+0000 7fa6046c9700 1 -- 172.21.15.5:0/3332328701 wait complete. 2024-05-09T00:59:15.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.179+0000 7fa6046c9700 1 Processor -- start 2024-05-09T00:59:15.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.180+0000 7fa6046c9700 1 -- start start 2024-05-09T00:59:15.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.180+0000 7fa6046c9700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5fc10a7c0 0x7fa5fc1af790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:15.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.180+0000 7fa6046c9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5fc10b110 0x7fa5fc1afcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:15.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.180+0000 7fa6046c9700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5fc11d930 0x7fa5fc1a9a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:15.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.180+0000 7fa6046c9700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa5fc1231f0 con 0x7fa5fc10b110 2024-05-09T00:59:15.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.181+0000 7fa6046c9700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa5fc123070 con 0x7fa5fc11d930 2024-05-09T00:59:15.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.181+0000 7fa6046c9700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa5fc123370 con 0x7fa5fc10a7c0 2024-05-09T00:59:15.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.182+0000 7fa601c64700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5fc10b110 0x7fa5fc1afcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:15.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.182+0000 7fa601c64700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5fc10b110 0x7fa5fc1afcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35908/0 (socket says 172.21.15.5:35908) 2024-05-09T00:59:15.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.182+0000 7fa601c64700 1 -- 172.21.15.5:0/526626587 learned_addr learned my addr 172.21.15.5:0/526626587 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:59:15.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.183+0000 7fa602465700 1 --2- 172.21.15.5:0/526626587 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5fc10a7c0 0x7fa5fc1af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:15.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.183+0000 7fa602c66700 1 --2- 172.21.15.5:0/526626587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5fc11d930 0x7fa5fc1a9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:15.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.183+0000 7fa601c64700 1 -- 172.21.15.5:0/526626587 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5fc10a7c0 msgr2=0x7fa5fc1af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:15.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.183+0000 7fa601c64700 1 --2- 172.21.15.5:0/526626587 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5fc10a7c0 0x7fa5fc1af790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:15.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.183+0000 7fa601c64700 1 -- 172.21.15.5:0/526626587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5fc11d930 msgr2=0x7fa5fc1a9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:15.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.183+0000 7fa601c64700 1 --2- 172.21.15.5:0/526626587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5fc11d930 0x7fa5fc1a9a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:15.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.183+0000 7fa601c64700 1 -- 172.21.15.5:0/526626587 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5fc1aa0d0 con 0x7fa5fc10b110 2024-05-09T00:59:15.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.183+0000 7fa601c64700 1 --2- 172.21.15.5:0/526626587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5fc10b110 0x7fa5fc1afcd0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec00b530 tx=0x7fa5ec00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:59:15.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.185+0000 7fa5f37fe700 1 -- 172.21.15.5:0/526626587 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5ec015020 con 0x7fa5fc10b110 2024-05-09T00:59:15.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.185+0000 7fa5f37fe700 1 -- 172.21.15.5:0/526626587 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5ec011ce0 con 0x7fa5fc10b110 2024-05-09T00:59:15.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.185+0000 7fa6046c9700 1 -- 172.21.15.5:0/526626587 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5fc1aa3c0 con 0x7fa5fc10b110 2024-05-09T00:59:15.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.185+0000 7fa6046c9700 1 -- 172.21.15.5:0/526626587 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5fc112bd0 con 0x7fa5fc10b110 2024-05-09T00:59:15.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.187+0000 7fa5f37fe700 1 -- 172.21.15.5:0/526626587 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5ec010d20 con 0x7fa5fc10b110 2024-05-09T00:59:15.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.193+0000 7fa6130dfb80 1 Processor -- start 2024-05-09T00:59:15.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.194+0000 7fa6130dfb80 1 -- start start 2024-05-09T00:59:15.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.194+0000 7fa6130dfb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cb3dfce160 0x55cb3e067640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:15.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.195+0000 7fa6130dfb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb3dee44a0 0x55cb3e067b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:15.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.195+0000 7fa6130dfb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb3e061800 0x55cb3e061c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:15.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.195+0000 7fa6130dfb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55cb3df83ff0 con 0x55cb3dee44a0 2024-05-09T00:59:15.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.195+0000 7fa6130dfb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55cb3df227f0 con 0x55cb3e061800 2024-05-09T00:59:15.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.195+0000 7fa6130dfb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55cb3df7ffb0 con 0x55cb3dfce160 2024-05-09T00:59:15.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.196+0000 7fa5f37fe700 1 -- 172.21.15.5:0/526626587 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fa5ec018020 con 0x7fa5fc10b110 2024-05-09T00:59:15.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.197+0000 7fa602c66700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cb3dfce160 0x55cb3e067640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:15.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.197+0000 7fa602c66700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cb3dfce160 0x55cb3e067640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:48180/0 (socket says 172.21.15.5:48180) 2024-05-09T00:59:15.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.197+0000 7fa602c66700 1 -- 172.21.15.5:0/1481988409 learned_addr learned my addr 172.21.15.5:0/1481988409 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:59:15.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.197+0000 7fa602465700 1 --2- 172.21.15.5:0/1481988409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb3dee44a0 0x55cb3e067b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:15.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.198+0000 7fa601c64700 1 --2- 172.21.15.5:0/1481988409 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb3e061800 0x55cb3e061c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:15.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.198+0000 7fa602465700 1 -- 172.21.15.5:0/1481988409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cb3dfce160 msgr2=0x55cb3e067640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:59:15.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.198+0000 7fa602465700 1 --2- 172.21.15.5:0/1481988409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cb3dfce160 0x55cb3e067640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:15.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.198+0000 7fa602465700 1 -- 172.21.15.5:0/1481988409 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb3e061800 msgr2=0x55cb3e061c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:59:15.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.198+0000 7fa602465700 1 --2- 172.21.15.5:0/1481988409 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb3e061800 0x55cb3e061c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:15.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.199+0000 7fa602465700 1 -- 172.21.15.5:0/1481988409 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb3df1a370 con 0x55cb3dee44a0 2024-05-09T00:59:15.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.199+0000 7fa602465700 1 --2- 172.21.15.5:0/1481988409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb3dee44a0 0x55cb3e067b80 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fa5f4007d00 tx=0x7fa5f400c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e43f6710ec14f887 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:59:15.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.200+0000 7fa5f37fe700 1 --2- 172.21.15.5:0/526626587 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fa5e80b2900 0x7fa5e80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:15.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.200+0000 7fa5f0ff9700 1 -- 172.21.15.5:0/1481988409 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f4015020 con 0x55cb3dee44a0 2024-05-09T00:59:15.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.200+0000 7fa602c66700 1 --2- 172.21.15.5:0/526626587 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fa5e80b2900 0x7fa5e80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:15.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.200+0000 7fa5f0ff9700 1 -- 172.21.15.5:0/1481988409 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5f4007e50 con 0x55cb3dee44a0 2024-05-09T00:59:15.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.200+0000 7fa5f37fe700 1 -- 172.21.15.5:0/526626587 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fa5ec0e18c0 con 0x7fa5fc10b110 2024-05-09T00:59:15.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.200+0000 7fa5f0ff9700 1 -- 172.21.15.5:0/1481988409 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f4010680 con 0x55cb3dee44a0 2024-05-09T00:59:15.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.202+0000 7fa6130dfb80 1 -- 172.21.15.5:0/1481988409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb3dee44a0 msgr2=0x55cb3e067b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:15.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.202+0000 7fa6130dfb80 1 --2- 172.21.15.5:0/1481988409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb3dee44a0 0x55cb3e067b80 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fa5f4007d00 tx=0x7fa5f400c470 comp rx=0 tx=0).stop 2024-05-09T00:59:15.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.202+0000 7fa602c66700 1 --2- 172.21.15.5:0/526626587 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fa5e80b2900 0x7fa5e80b4e10 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8005f50 tx=0x7fa5f8005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:59:15.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.202+0000 7fa6130dfb80 1 -- 172.21.15.5:0/1481988409 shutdown_connections 2024-05-09T00:59:15.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.202+0000 7fa6130dfb80 1 --2- 172.21.15.5:0/1481988409 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb3e061800 0x55cb3e061c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:15.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.202+0000 7fa6130dfb80 1 --2- 172.21.15.5:0/1481988409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cb3dfce160 0x55cb3e067640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:15.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.202+0000 7fa6130dfb80 1 --2- 172.21.15.5:0/1481988409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb3dee44a0 0x55cb3e067b80 secure :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fa5f4007d00 tx=0x7fa5f400c470 comp rx=0 tx=0).stop 2024-05-09T00:59:15.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.202+0000 7fa6130dfb80 1 -- 172.21.15.5:0/1481988409 >> 172.21.15.5:0/1481988409 conn(0x55cb3df446a0 msgr2=0x55cb3e05c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:59:15.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.203+0000 7fa6130dfb80 1 -- 172.21.15.5:0/1481988409 shutdown_connections 2024-05-09T00:59:15.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.204+0000 7fa6130dfb80 1 -- 172.21.15.5:0/1481988409 wait complete. 2024-05-09T00:59:15.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.204+0000 7fa6130dfb80 1 -- start start 2024-05-09T00:59:15.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.205+0000 7fa6130dfb80 1 Processor -- start 2024-05-09T00:59:15.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.206+0000 7fa6130dfb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb3dfce160 0x55cb3e06b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:15.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.206+0000 7fa6130dfb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb3e061800 0x55cb3e06b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:15.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.206+0000 7fa602465700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb3dfce160 0x55cb3e06b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:15.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.207+0000 7fa602465700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb3dfce160 0x55cb3e06b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35926/0 (socket says 172.21.15.5:35926) 2024-05-09T00:59:15.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.207+0000 7fa602465700 1 -- 172.21.15.5:0/4209263676 learned_addr learned my addr 172.21.15.5:0/4209263676 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:59:15.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.207+0000 7fa6130dfb80 1 --2- 172.21.15.5:0/4209263676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cb3e06bea0 0x55cb3e077600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:15.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.208+0000 7fa601c64700 1 --2- 172.21.15.5:0/4209263676 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb3e061800 0x55cb3e06b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:15.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.208+0000 7fa6130dfb80 1 -- 172.21.15.5:0/4209263676 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55cb3ddffba0 con 0x55cb3dfce160 2024-05-09T00:59:15.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.208+0000 7fa6130dfb80 1 -- 172.21.15.5:0/4209263676 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55cb3df3e150 con 0x55cb3e061800 2024-05-09T00:59:15.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.208+0000 7fa6130dfb80 1 -- 172.21.15.5:0/4209263676 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55cb3e062640 con 0x55cb3e06bea0 2024-05-09T00:59:15.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.209+0000 7fa6130dfb80 10 client.0 _ll_register_callbacks cb 0x55cb3ddf56c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:59:15.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.209+0000 7fa602c66700 1 --2- 172.21.15.5:0/4209263676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cb3e06bea0 0x55cb3e077600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:15.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.209+0000 7fa602465700 1 -- 172.21.15.5:0/4209263676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cb3e06bea0 msgr2=0x55cb3e077600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:59:15.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.209+0000 7fa602465700 1 --2- 172.21.15.5:0/4209263676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cb3e06bea0 0x55cb3e077600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:15.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.209+0000 7fa602465700 1 -- 172.21.15.5:0/4209263676 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb3e061800 msgr2=0x55cb3e06b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:59:15.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.209+0000 7fa602465700 1 --2- 172.21.15.5:0/4209263676 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb3e061800 0x55cb3e06b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:15.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.209+0000 7fa602465700 1 -- 172.21.15.5:0/4209263676 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb3ddff490 con 0x55cb3dfce160 2024-05-09T00:59:15.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.209+0000 7fa602c66700 1 --2- 172.21.15.5:0/4209263676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cb3e06bea0 0x55cb3e077600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:59:15.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.209+0000 7fa601c64700 1 --2- 172.21.15.5:0/4209263676 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb3e061800 0x55cb3e06b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:59:15.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.210+0000 7fa602465700 1 --2- 172.21.15.5:0/4209263676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb3dfce160 0x55cb3e06b420 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x55cb3e05e090 tx=0x7fa5f4011710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13cf5cd5d672add5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:59:15.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.210+0000 7fa5e5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:59:15.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.211+0000 7fa5e5ffb700 1 -- 172.21.15.5:0/4209263676 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f4025640 con 0x55cb3dfce160 2024-05-09T00:59:15.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.211+0000 7fa5e5ffb700 1 -- 172.21.15.5:0/4209263676 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fa5f401b430 con 0x55cb3dfce160 2024-05-09T00:59:15.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.211+0000 7fa6130dfb80 1 -- 172.21.15.5:0/4209263676 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5f4007e50 con 0x55cb3dfce160 2024-05-09T00:59:15.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.213+0000 7fa5e5ffb700 1 -- 172.21.15.5:0/4209263676 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa5f4015030 con 0x55cb3dfce160 2024-05-09T00:59:15.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.213+0000 7fa6130dfb80 10 client.16851 fetch_fsmap learned FSMap version 31 2024-05-09T00:59:15.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.213+0000 7fa6130dfb80 1 -- 172.21.15.5:0/4209263676 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55cb3e079260 con 0x55cb3dfce160 2024-05-09T00:59:15.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.214+0000 7fa5e5ffb700 1 -- 172.21.15.5:0/4209263676 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5f4007e50 con 0x55cb3dfce160 2024-05-09T00:59:15.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.215+0000 7fa5e5ffb700 1 -- 172.21.15.5:0/4209263676 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fa5f4023070 con 0x55cb3dfce160 2024-05-09T00:59:15.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.215+0000 7fa6130dfb80 10 client.16851 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:59:15.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.215+0000 7fa6130dfb80 10 client.16851 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T00:59:15.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.215+0000 7fa6130dfb80 20 client.16851 populate_metadata read hostname 'smithi005' 2024-05-09T00:59:15.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.215+0000 7fa6130dfb80 1 --2- 172.21.15.5:0/4209263676 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x55cb3e07b730 0x55cb3e07db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:15.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.215+0000 7fa6130dfb80 4 client.16851 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:59:15.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.215+0000 7fa6130dfb80 1 -- 172.21.15.5:0/4209263676 --> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb3df919d0 con 0x55cb3e07b730 2024-05-09T00:59:15.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.217+0000 7fa601c64700 1 --2- 172.21.15.5:0/4209263676 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x55cb3e07b730 0x55cb3e07db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:15.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.219+0000 7fa5e5ffb700 10 client.16851 ms_handle_connect on v2:172.21.15.157:6834/832898273 2024-05-09T00:59:15.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.219+0000 7fa601c64700 1 --2- 172.21.15.5:0/4209263676 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x55cb3e07b730 0x55cb3e07db50 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ec920e8c9faddee9 server_cookie=3bcf29884c239b9b in_seq=0 out_seq=0 2024-05-09T00:59:15.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.220+0000 7fa5e5ffb700 1 -- 172.21.15.5:0/4209263676 <== mds.3 v2:172.21.15.157:6834/832898273 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55cb3df919d0 con 0x55cb3e07b730 2024-05-09T00:59:15.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.220+0000 7fa5e5ffb700 10 client.16851 handle_command_reply: tid=0 2024-05-09T00:59:15.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.220+0000 7fa6130dfb80 1 client.16851 shutdown 2024-05-09T00:59:15.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.220+0000 7fa6130dfb80 1 -- 172.21.15.5:0/4209263676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb3dfce160 msgr2=0x55cb3e06b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:15.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.220+0000 7fa6130dfb80 1 --2- 172.21.15.5:0/4209263676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb3dfce160 0x55cb3e06b420 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x55cb3e05e090 tx=0x7fa5f4011710 comp rx=0 tx=0).stop 2024-05-09T00:59:15.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.220+0000 7fa6130dfb80 1 -- 172.21.15.5:0/4209263676 shutdown_connections 2024-05-09T00:59:15.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.220+0000 7fa6130dfb80 1 --2- 172.21.15.5:0/4209263676 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x55cb3e07b730 0x55cb3e07db50 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:15.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.220+0000 7fa6130dfb80 1 --2- 172.21.15.5:0/4209263676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cb3e06bea0 0x55cb3e077600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:15.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.220+0000 7fa6130dfb80 1 --2- 172.21.15.5:0/4209263676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb3dfce160 0x55cb3e06b420 secure :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0x55cb3e05e090 tx=0x7fa5f4011710 comp rx=0 tx=0).stop 2024-05-09T00:59:15.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.220+0000 7fa6130dfb80 1 --2- 172.21.15.5:0/4209263676 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb3e061800 0x55cb3e06b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:15.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.220+0000 7fa6130dfb80 1 -- 172.21.15.5:0/4209263676 >> 172.21.15.5:0/4209263676 conn(0x55cb3df446a0 msgr2=0x55cb3e05c380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:59:15.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.220+0000 7fa5e5ffb700 0 client.16851 ms_handle_reset on v2:172.21.15.157:6834/832898273 2024-05-09T00:59:15.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.222+0000 7fa6130dfb80 1 -- 172.21.15.5:0/4209263676 shutdown_connections 2024-05-09T00:59:15.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.222+0000 7fa6130dfb80 1 -- 172.21.15.5:0/4209263676 wait complete. 2024-05-09T00:59:15.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.222+0000 7fa6130dfb80 20 client.16851 trim_cache size 0 max 16384 2024-05-09T00:59:15.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.252+0000 7fa6130dfb80 1 Processor -- start 2024-05-09T00:59:15.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.252+0000 7fa6130dfb80 1 -- start start 2024-05-09T00:59:15.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.253+0000 7fa6130dfb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb3e061800 0x55cb3e066c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:15.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.253+0000 7fa6130dfb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb3e06bea0 0x55cb3e06d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:15.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.253+0000 7fa6130dfb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cb3e083150 0x55cb3e083560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:15.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.254+0000 7fa601c64700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb3e06bea0 0x55cb3e06d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:15.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.254+0000 7fa601c64700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb3e06bea0 0x55cb3e06d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35942/0 (socket says 172.21.15.5:35942) 2024-05-09T00:59:15.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.254+0000 7fa601c64700 1 -- 172.21.15.5:0/3809099383 learned_addr learned my addr 172.21.15.5:0/3809099383 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:59:15.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.254+0000 7fa602465700 1 --2- 172.21.15.5:0/3809099383 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb3e061800 0x55cb3e066c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:15.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.254+0000 7fa602c66700 1 --2- 172.21.15.5:0/3809099383 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cb3e083150 0x55cb3e083560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:15.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.253+0000 7fa6130dfb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa5fc11a0a0 con 0x55cb3e06bea0 2024-05-09T00:59:15.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.254+0000 7fa6130dfb80 1 -- 172.21.15.5:0/3809099383 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa5fc1c7e40 con 0x55cb3e061800 2024-05-09T00:59:15.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.254+0000 7fa6130dfb80 1 -- 172.21.15.5:0/3809099383 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa5fc11a770 con 0x55cb3e083150 2024-05-09T00:59:15.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.255+0000 7fa601c64700 1 -- 172.21.15.5:0/3809099383 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cb3e083150 msgr2=0x55cb3e083560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:59:15.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.255+0000 7fa601c64700 1 --2- 172.21.15.5:0/3809099383 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cb3e083150 0x55cb3e083560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:15.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.255+0000 7fa601c64700 1 -- 172.21.15.5:0/3809099383 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb3e061800 msgr2=0x55cb3e066c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:59:15.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.255+0000 7fa601c64700 1 --2- 172.21.15.5:0/3809099383 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb3e061800 0x55cb3e066c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:15.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.255+0000 7fa601c64700 1 -- 172.21.15.5:0/3809099383 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb3e05e2e0 con 0x55cb3e06bea0 2024-05-09T00:59:15.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.258+0000 7fa601c64700 1 --2- 172.21.15.5:0/3809099383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb3e06bea0 0x55cb3e06d8c0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec00b500 tx=0x7fa5ec0df200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c12d84f222339a98 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:59:15.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.263+0000 7fa5e7fff700 1 -- 172.21.15.5:0/3809099383 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5ec0a8960 con 0x55cb3e06bea0 2024-05-09T00:59:15.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.263+0000 7fa5e7fff700 1 -- 172.21.15.5:0/3809099383 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5ec0ab070 con 0x55cb3e06bea0 2024-05-09T00:59:15.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.263+0000 7fa5e7fff700 1 -- 172.21.15.5:0/3809099383 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5ec0a90b0 con 0x55cb3e06bea0 2024-05-09T00:59:15.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.265+0000 7fa6130dfb80 1 -- 172.21.15.5:0/3809099383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb3e06bea0 msgr2=0x55cb3e06d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:15.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.265+0000 7fa6130dfb80 1 --2- 172.21.15.5:0/3809099383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb3e06bea0 0x55cb3e06d8c0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec00b500 tx=0x7fa5ec0df200 comp rx=0 tx=0).stop 2024-05-09T00:59:15.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.266+0000 7fa6130dfb80 1 -- 172.21.15.5:0/3809099383 shutdown_connections 2024-05-09T00:59:15.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.266+0000 7fa6130dfb80 1 --2- 172.21.15.5:0/3809099383 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cb3e083150 0x55cb3e083560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:15.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.266+0000 7fa6130dfb80 1 --2- 172.21.15.5:0/3809099383 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb3e061800 0x55cb3e066c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:15.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.266+0000 7fa6130dfb80 1 --2- 172.21.15.5:0/3809099383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb3e06bea0 0x55cb3e06d8c0 secure :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec00b500 tx=0x7fa5ec0df200 comp rx=0 tx=0).stop 2024-05-09T00:59:15.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.266+0000 7fa6130dfb80 1 -- 172.21.15.5:0/3809099383 >> 172.21.15.5:0/3809099383 conn(0x55cb3df446a0 msgr2=0x55cb3e066580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:59:15.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.271+0000 7fa6130dfb80 1 -- 172.21.15.5:0/3809099383 shutdown_connections 2024-05-09T00:59:15.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.272+0000 7fa6130dfb80 1 -- 172.21.15.5:0/3809099383 wait complete. 2024-05-09T00:59:15.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.272+0000 7fa6130dfb80 1 -- start start 2024-05-09T00:59:15.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.273+0000 7fa6130dfb80 1 Processor -- start 2024-05-09T00:59:15.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.274+0000 7fa6130dfb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cb3e061800 0x55cb3e073920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:15.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.274+0000 7fa6130dfb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb3e083150 0x55cb3e06ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:15.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.274+0000 7fa6130dfb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb3e06f1c0 0x55cb3e06f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:15.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.274+0000 7fa6130dfb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa5fc1c7770 con 0x55cb3e083150 2024-05-09T00:59:15.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.274+0000 7fa6130dfb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa5fc11ae40 con 0x55cb3e06f1c0 2024-05-09T00:59:15.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.274+0000 7fa6130dfb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa5fc118040 con 0x55cb3e061800 2024-05-09T00:59:15.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.274+0000 7fa6130dfb80 10 client.0 _ll_register_callbacks cb 0x55cb3ddf56c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:59:15.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.275+0000 7fa602c66700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb3e06f1c0 0x55cb3e06f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:15.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.275+0000 7fa602c66700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb3e06f1c0 0x55cb3e06f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:57376/0 (socket says 172.21.15.5:57376) 2024-05-09T00:59:15.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.275+0000 7fa602c66700 1 -- 172.21.15.5:0/543224736 learned_addr learned my addr 172.21.15.5:0/543224736 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:59:15.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.275+0000 7fa602465700 1 --2- 172.21.15.5:0/543224736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cb3e061800 0x55cb3e073920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:15.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.278+0000 7fa602c66700 1 -- 172.21.15.5:0/543224736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cb3e061800 msgr2=0x55cb3e073920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:59:15.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.278+0000 7fa602c66700 1 --2- 172.21.15.5:0/543224736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cb3e061800 0x55cb3e073920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:15.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.278+0000 7fa602c66700 1 -- 172.21.15.5:0/543224736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb3e083150 msgr2=0x55cb3e06ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:59:15.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.278+0000 7fa602c66700 1 --2- 172.21.15.5:0/543224736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb3e083150 0x55cb3e06ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:15.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.278+0000 7fa602c66700 1 -- 172.21.15.5:0/543224736 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb3e0e4d00 con 0x55cb3e06f1c0 2024-05-09T00:59:15.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.280+0000 7fa602c66700 1 --2- 172.21.15.5:0/543224736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb3e06f1c0 0x55cb3e06f5d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa5f800b5c0 tx=0x7fa5f8020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd5e1a172492d07 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:59:15.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.280+0000 7fa5e5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-09T00:59:15.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.281+0000 7fa5e5ffb700 1 -- 172.21.15.5:0/543224736 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f8023660 con 0x55cb3e06f1c0 2024-05-09T00:59:15.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.281+0000 7fa6130dfb80 1 -- 172.21.15.5:0/543224736 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5ec0ab070 con 0x55cb3e06f1c0 2024-05-09T00:59:15.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.281+0000 7fa5e5ffb700 1 -- 172.21.15.5:0/543224736 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fa5f8023800 con 0x55cb3e06f1c0 2024-05-09T00:59:15.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.282+0000 7fa5e5ffb700 1 -- 172.21.15.5:0/543224736 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa5f8005690 con 0x55cb3e06f1c0 2024-05-09T00:59:15.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.291+0000 7fa6130dfb80 10 client.26267 fetch_fsmap learned FSMap version 31 2024-05-09T00:59:15.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.291+0000 7fa6130dfb80 1 -- 172.21.15.5:0/543224736 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55cb3e0e5e60 con 0x55cb3e06f1c0 2024-05-09T00:59:15.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.297+0000 7fa5e5ffb700 1 -- 172.21.15.5:0/543224736 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5ec0ab070 con 0x55cb3e06f1c0 2024-05-09T00:59:15.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.297+0000 7fa5e5ffb700 1 -- 172.21.15.5:0/543224736 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fa5f800fcd0 con 0x55cb3e06f1c0 2024-05-09T00:59:15.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.300+0000 7fa6130dfb80 10 client.26267 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:59:15.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.300+0000 7fa6130dfb80 10 client.26267 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T00:59:15.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.300+0000 7fa6130dfb80 20 client.26267 populate_metadata read hostname 'smithi005' 2024-05-09T00:59:15.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.300+0000 7fa6130dfb80 1 --2- 172.21.15.5:0/543224736 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x55cb3e0e6520 0x55cb3e0e8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:15.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.300+0000 7fa6130dfb80 4 client.26267 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:59:15.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.300+0000 7fa6130dfb80 1 -- 172.21.15.5:0/543224736 --> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cb3df8ba70 con 0x55cb3e0e6520 2024-05-09T00:59:15.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.302+0000 7fa602465700 1 --2- 172.21.15.5:0/543224736 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x55cb3e0e6520 0x55cb3e0e8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:15.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.304+0000 7fa602465700 1 --2- 172.21.15.5:0/543224736 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x55cb3e0e6520 0x55cb3e0e8960 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f97342a3166e7e20 server_cookie=50fe49a389314d70 in_seq=0 out_seq=0 2024-05-09T00:59:15.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.305+0000 7fa5e5ffb700 10 client.26267 ms_handle_connect on v2:172.21.15.157:6834/832898273 2024-05-09T00:59:15.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.305+0000 7fa5e5ffb700 1 -- 172.21.15.5:0/543224736 <== mds.3 v2:172.21.15.157:6834/832898273 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cb3df8ba70 con 0x55cb3e0e6520 2024-05-09T00:59:15.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.305+0000 7fa5e5ffb700 10 client.26267 handle_command_reply: tid=0 2024-05-09T00:59:15.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.308+0000 7fa6130dfb80 1 client.26267 shutdown 2024-05-09T00:59:15.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.309+0000 7fa6130dfb80 1 -- 172.21.15.5:0/543224736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb3e06f1c0 msgr2=0x55cb3e06f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:15.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.309+0000 7fa6130dfb80 1 --2- 172.21.15.5:0/543224736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb3e06f1c0 0x55cb3e06f5d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa5f800b5c0 tx=0x7fa5f8020000 comp rx=0 tx=0).stop 2024-05-09T00:59:15.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.311+0000 7fa6130dfb80 1 -- 172.21.15.5:0/543224736 shutdown_connections 2024-05-09T00:59:15.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.311+0000 7fa6130dfb80 1 --2- 172.21.15.5:0/543224736 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x55cb3e0e6520 0x55cb3e0e8960 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:15.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.311+0000 7fa6130dfb80 1 --2- 172.21.15.5:0/543224736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb3e06f1c0 0x55cb3e06f5d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:15.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.311+0000 7fa6130dfb80 1 --2- 172.21.15.5:0/543224736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cb3e061800 0x55cb3e073920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:15.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.311+0000 7fa6130dfb80 1 --2- 172.21.15.5:0/543224736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb3e083150 0x55cb3e06ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:15.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.311+0000 7fa6130dfb80 1 -- 172.21.15.5:0/543224736 >> 172.21.15.5:0/543224736 conn(0x55cb3df446a0 msgr2=0x55cb3e069cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:59:15.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.311+0000 7fa5e5ffb700 0 client.26267 ms_handle_reset on v2:172.21.15.157:6834/832898273 2024-05-09T00:59:15.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.313+0000 7fa6130dfb80 1 -- 172.21.15.5:0/543224736 shutdown_connections 2024-05-09T00:59:15.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.313+0000 7fa6130dfb80 1 -- 172.21.15.5:0/543224736 wait complete. 2024-05-09T00:59:15.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.314+0000 7fa6130dfb80 20 client.26267 trim_cache size 0 max 16384 2024-05-09T00:59:15.326 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-09T00:59:15.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.326+0000 7fa6046c9700 1 -- 172.21.15.5:0/526626587 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fa5e80b2900 msgr2=0x7fa5e80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:15.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.326+0000 7fa6046c9700 1 --2- 172.21.15.5:0/526626587 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fa5e80b2900 0x7fa5e80b4e10 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8005f50 tx=0x7fa5f8005e80 comp rx=0 tx=0).stop 2024-05-09T00:59:15.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.326+0000 7fa6046c9700 1 -- 172.21.15.5:0/526626587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5fc10b110 msgr2=0x7fa5fc1afcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:15.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.326+0000 7fa6046c9700 1 --2- 172.21.15.5:0/526626587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5fc10b110 0x7fa5fc1afcd0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec00b530 tx=0x7fa5ec00b8f0 comp rx=0 tx=0).stop 2024-05-09T00:59:15.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.327+0000 7fa6046c9700 1 -- 172.21.15.5:0/526626587 shutdown_connections 2024-05-09T00:59:15.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.327+0000 7fa6046c9700 1 --2- 172.21.15.5:0/526626587 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fa5e80b2900 0x7fa5e80b4e10 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:15.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.327+0000 7fa6046c9700 1 --2- 172.21.15.5:0/526626587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5fc11d930 0x7fa5fc1a9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:15.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.327+0000 7fa6046c9700 1 --2- 172.21.15.5:0/526626587 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5fc10a7c0 0x7fa5fc1af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:15.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.327+0000 7fa6046c9700 1 --2- 172.21.15.5:0/526626587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5fc10b110 0x7fa5fc1afcd0 secure :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec00b530 tx=0x7fa5ec00b8f0 comp rx=0 tx=0).stop 2024-05-09T00:59:15.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.327+0000 7fa6046c9700 1 -- 172.21.15.5:0/526626587 >> 172.21.15.5:0/526626587 conn(0x7fa5fc06f290 msgr2=0x7fa5fc11b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:59:15.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.328+0000 7fa6046c9700 1 -- 172.21.15.5:0/526626587 shutdown_connections 2024-05-09T00:59:15.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:15.329+0000 7fa6046c9700 1 -- 172.21.15.5:0/526626587 wait complete. 2024-05-09T00:59:16.286 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:59:15 smithi005 ceph-mon[30365]: pgmap v758: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 889 MiB/s wr, 1.86k op/s 2024-05-09T00:59:16.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:59:15 smithi131 ceph-mon[34547]: pgmap v758: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 889 MiB/s wr, 1.86k op/s 2024-05-09T00:59:16.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:59:15 smithi157 ceph-mon[34294]: pgmap v758: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 889 MiB/s wr, 1.86k op/s 2024-05-09T00:59:17.370 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T00:59:18.192 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:59:18.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:59:18 smithi005 ceph-mon[30365]: pgmap v759: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 895 MiB/s wr, 2.11k op/s 2024-05-09T00:59:18.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:59:18 smithi131 ceph-mon[34547]: pgmap v759: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 895 MiB/s wr, 2.11k op/s 2024-05-09T00:59:18.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:59:18 smithi157 ceph-mon[34294]: pgmap v759: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 895 MiB/s wr, 2.11k op/s 2024-05-09T00:59:20.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:59:20 smithi005 ceph-mon[30365]: pgmap v760: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 882 MiB/s wr, 2.06k op/s 2024-05-09T00:59:20.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:59:20 smithi131 ceph-mon[34547]: pgmap v760: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 882 MiB/s wr, 2.06k op/s 2024-05-09T00:59:20.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:59:20 smithi157 ceph-mon[34294]: pgmap v760: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 882 MiB/s wr, 2.06k op/s 2024-05-09T00:59:22.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:59:22 smithi005 ceph-mon[30365]: pgmap v761: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 888 MiB/s wr, 1.95k op/s 2024-05-09T00:59:22.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:59:22 smithi157 ceph-mon[34294]: pgmap v761: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 888 MiB/s wr, 1.95k op/s 2024-05-09T00:59:22.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:59:22 smithi131 ceph-mon[34547]: pgmap v761: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 888 MiB/s wr, 1.95k op/s 2024-05-09T00:59:22.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.437+0000 7f67b3fff640 1 -- 172.21.15.5:0/3777759442 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67b410b580 msgr2=0x7f67b4109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:22.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.437+0000 7f67b3fff640 1 --2- 172.21.15.5:0/3777759442 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67b410b580 0x7f67b4109cd0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f67a4009900 tx=0x7f67a402a440 comp rx=0 tx=0).stop 2024-05-09T00:59:22.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.445+0000 7f67b3fff640 1 -- 172.21.15.5:0/3777759442 shutdown_connections 2024-05-09T00:59:22.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.445+0000 7f67b3fff640 1 --2- 172.21.15.5:0/3777759442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67b4078f20 0x7f67b407b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:22.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.445+0000 7f67b3fff640 1 --2- 172.21.15.5:0/3777759442 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67b410a210 0x7f67b410a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:22.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.445+0000 7f67b3fff640 1 --2- 172.21.15.5:0/3777759442 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67b410b580 0x7f67b4109cd0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:22.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.445+0000 7f67b3fff640 1 -- 172.21.15.5:0/3777759442 >> 172.21.15.5:0/3777759442 conn(0x7f67b4070460 msgr2=0x7f67b4072910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:59:22.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.449+0000 7f67b3fff640 1 -- 172.21.15.5:0/3777759442 shutdown_connections 2024-05-09T00:59:22.454 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.453+0000 7f67b3fff640 1 -- 172.21.15.5:0/3777759442 wait complete. 2024-05-09T00:59:22.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.454+0000 7f67b3fff640 1 Processor -- start 2024-05-09T00:59:22.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.454+0000 7f67b3fff640 1 -- start start 2024-05-09T00:59:22.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.454+0000 7f67b3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67b4078f20 0x7f67b4084bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:22.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.454+0000 7f67b3fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67b410a210 0x7f67b40850f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:22.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.454+0000 7f67b3fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67b410b580 0x7f67b41c1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:22.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.455+0000 7f67b3fff640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f67b407ebb0 con 0x7f67b4078f20 2024-05-09T00:59:22.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.455+0000 7f67b3fff640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f67b407ea30 con 0x7f67b410b580 2024-05-09T00:59:22.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.455+0000 7f67b3fff640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f67b407ed30 con 0x7f67b410a210 2024-05-09T00:59:22.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.455+0000 7f67b2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67b4078f20 0x7f67b4084bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:22.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.455+0000 7f67b2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67b4078f20 0x7f67b4084bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35962/0 (socket says 172.21.15.5:35962) 2024-05-09T00:59:22.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.455+0000 7f67b2ffd640 1 -- 172.21.15.5:0/650464990 learned_addr learned my addr 172.21.15.5:0/650464990 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:59:22.459 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.458+0000 7f67b37fe640 1 --2- 172.21.15.5:0/650464990 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67b410b580 0x7f67b41c1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:22.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.458+0000 7f67b2ffd640 1 -- 172.21.15.5:0/650464990 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67b410a210 msgr2=0x7f67b40850f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T00:59:22.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.458+0000 7f67b2ffd640 1 --2- 172.21.15.5:0/650464990 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67b410a210 0x7f67b40850f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:22.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.458+0000 7f67b2ffd640 1 -- 172.21.15.5:0/650464990 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67b410b580 msgr2=0x7f67b41c1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:22.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.458+0000 7f67b2ffd640 1 --2- 172.21.15.5:0/650464990 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67b410b580 0x7f67b41c1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:22.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.458+0000 7f67b2ffd640 1 -- 172.21.15.5:0/650464990 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67b41c1d70 con 0x7f67b4078f20 2024-05-09T00:59:22.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.459+0000 7f67b2ffd640 1 --2- 172.21.15.5:0/650464990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67b4078f20 0x7f67b4084bb0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f67a4000c00 tx=0x7f67a4005560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:59:22.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.460+0000 7f67b0ff9640 1 -- 172.21.15.5:0/650464990 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a4041070 con 0x7f67b4078f20 2024-05-09T00:59:22.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.460+0000 7f67b3fff640 1 -- 172.21.15.5:0/650464990 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67b41c2000 con 0x7f67b4078f20 2024-05-09T00:59:22.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.460+0000 7f67b3fff640 1 -- 172.21.15.5:0/650464990 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67b41c2550 con 0x7f67b4078f20 2024-05-09T00:59:22.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.466+0000 7f67c05b0740 1 Processor -- start 2024-05-09T00:59:22.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.466+0000 7f67c05b0740 1 -- start start 2024-05-09T00:59:22.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.468+0000 7f67c05b0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5562a754f1b0 0x5562a78f36c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:22.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.468+0000 7f67c05b0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562a75262f0 0x5562a78f3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:22.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.469+0000 7f67c05b0740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562a752e4d0 0x5562a78fab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:22.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.469+0000 7f67c05b0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5562a78577c0 con 0x5562a75262f0 2024-05-09T00:59:22.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.469+0000 7f67c05b0740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5562a77f30e0 con 0x5562a752e4d0 2024-05-09T00:59:22.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.469+0000 7f67c05b0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5562a7853780 con 0x5562a754f1b0 2024-05-09T00:59:22.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.470+0000 7f67b0ff9640 1 -- 172.21.15.5:0/650464990 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67a403dd10 con 0x7f67b4078f20 2024-05-09T00:59:22.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.470+0000 7f67b37fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5562a754f1b0 0x5562a78f36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:22.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.470+0000 7f67b0ff9640 1 -- 172.21.15.5:0/650464990 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a4031e10 con 0x7f67b4078f20 2024-05-09T00:59:22.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.477+0000 7f67b37fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5562a754f1b0 0x5562a78f36c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:48248/0 (socket says 172.21.15.5:48248) 2024-05-09T00:59:22.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.477+0000 7f67b37fe640 1 -- 172.21.15.5:0/4284172289 learned_addr learned my addr 172.21.15.5:0/4284172289 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:59:22.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.477+0000 7f67b2ffd640 1 --2- 172.21.15.5:0/4284172289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562a75262f0 0x5562a78f3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:22.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.480+0000 7f67b0ff9640 1 -- 172.21.15.5:0/650464990 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f67a4044070 con 0x7f67b4078f20 2024-05-09T00:59:22.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.481+0000 7f67b37fe640 1 -- 172.21.15.5:0/4284172289 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562a752e4d0 msgr2=0x5562a78fab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:59:22.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.482+0000 7f67b0ff9640 1 --2- 172.21.15.5:0/650464990 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f67a00b29f0 0x7f67a00b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:22.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.483+0000 7f67b0ff9640 1 -- 172.21.15.5:0/650464990 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f67a4109550 con 0x7f67b4078f20 2024-05-09T00:59:22.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.483+0000 7f67abfff640 1 --2- 172.21.15.5:0/4284172289 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562a752e4d0 0x5562a78fab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:22.490 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.485+0000 7f67b37fe640 1 --2- 172.21.15.5:0/4284172289 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562a752e4d0 0x5562a78fab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:22.490 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.485+0000 7f67b37fe640 1 -- 172.21.15.5:0/4284172289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562a75262f0 msgr2=0x5562a78f3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:59:22.490 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.485+0000 7f67b37fe640 1 --2- 172.21.15.5:0/4284172289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562a75262f0 0x5562a78f3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:22.490 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.485+0000 7f67b37fe640 1 -- 172.21.15.5:0/4284172289 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562a781c7c0 con 0x5562a754f1b0 2024-05-09T00:59:22.490 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.489+0000 7f67b37fe640 1 --2- 172.21.15.5:0/650464990 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f67a00b29f0 0x7f67a00b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:22.490 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.489+0000 7f67abfff640 1 --2- 172.21.15.5:0/4284172289 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562a752e4d0 0x5562a78fab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:59:22.491 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.490+0000 7f67b37fe640 1 --2- 172.21.15.5:0/4284172289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5562a754f1b0 0x5562a78f36c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f67ac009fc0 tx=0x7f67ac010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=678eeff5141d6e4a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:59:22.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.495+0000 7f67b37fe640 1 --2- 172.21.15.5:0/650464990 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f67a00b29f0 0x7f67a00b4f00 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f67ac01f000 tx=0x7f67ac01b8a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:59:22.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.500+0000 7f67a97fa640 1 -- 172.21.15.5:0/4284172289 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ac022070 con 0x5562a754f1b0 2024-05-09T00:59:22.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.500+0000 7f67a97fa640 1 -- 172.21.15.5:0/4284172289 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67ac01c440 con 0x5562a754f1b0 2024-05-09T00:59:22.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.500+0000 7f67a97fa640 1 -- 172.21.15.5:0/4284172289 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ac01b5c0 con 0x5562a754f1b0 2024-05-09T00:59:22.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.502+0000 7f67c05b0740 1 -- 172.21.15.5:0/4284172289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5562a754f1b0 msgr2=0x5562a78f36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:22.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.503+0000 7f67c05b0740 1 --2- 172.21.15.5:0/4284172289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5562a754f1b0 0x5562a78f36c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f67ac009fc0 tx=0x7f67ac010860 comp rx=0 tx=0).stop 2024-05-09T00:59:22.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.503+0000 7f67c05b0740 1 -- 172.21.15.5:0/4284172289 shutdown_connections 2024-05-09T00:59:22.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.503+0000 7f67c05b0740 1 --2- 172.21.15.5:0/4284172289 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562a752e4d0 0x5562a78fab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:22.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.503+0000 7f67c05b0740 1 --2- 172.21.15.5:0/4284172289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562a75262f0 0x5562a78f3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:22.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.503+0000 7f67c05b0740 1 --2- 172.21.15.5:0/4284172289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5562a754f1b0 0x5562a78f36c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:22.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.504+0000 7f67c05b0740 1 -- 172.21.15.5:0/4284172289 >> 172.21.15.5:0/4284172289 conn(0x5562a7543300 msgr2=0x5562a78ef050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:59:22.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.504+0000 7f67c05b0740 1 -- 172.21.15.5:0/4284172289 shutdown_connections 2024-05-09T00:59:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.507+0000 7f67c05b0740 1 -- 172.21.15.5:0/4284172289 wait complete. 2024-05-09T00:59:22.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.508+0000 7f67c05b0740 1 -- start start 2024-05-09T00:59:22.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.509+0000 7f67c05b0740 1 Processor -- start 2024-05-09T00:59:22.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.518+0000 7f67c05b0740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562a75262f0 0x5562a78fd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:22.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.518+0000 7f67c05b0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562a752e4d0 0x5562a78fd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:22.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.518+0000 7f67c05b0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5562a754f1b0 0x5562a7904950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:22.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.519+0000 7f67c05b0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5562a78fb590 con 0x5562a752e4d0 2024-05-09T00:59:22.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.519+0000 7f67c05b0740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5562a77de9c0 con 0x5562a75262f0 2024-05-09T00:59:22.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.519+0000 7f67c05b0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5562a78fb710 con 0x5562a754f1b0 2024-05-09T00:59:22.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.519+0000 7f67c05b0740 10 client.0 _ll_register_callbacks cb 0x5562a780b020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:59:22.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.520+0000 7f67b2ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562a75262f0 0x5562a78fd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:22.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.520+0000 7f67b2ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562a75262f0 0x5562a78fd4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:57404/0 (socket says 172.21.15.5:57404) 2024-05-09T00:59:22.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.520+0000 7f67b2ffd640 1 -- 172.21.15.5:0/1078170020 learned_addr learned my addr 172.21.15.5:0/1078170020 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:59:22.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.522+0000 7f67b37fe640 1 --2- 172.21.15.5:0/1078170020 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5562a754f1b0 0x5562a7904950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:22.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.523+0000 7f67b2ffd640 1 -- 172.21.15.5:0/1078170020 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5562a754f1b0 msgr2=0x5562a7904950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:59:22.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.523+0000 7f67b2ffd640 1 --2- 172.21.15.5:0/1078170020 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5562a754f1b0 0x5562a7904950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:22.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.523+0000 7f67b2ffd640 1 -- 172.21.15.5:0/1078170020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562a752e4d0 msgr2=0x5562a78fd9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:59:22.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.523+0000 7f67b2ffd640 1 --2- 172.21.15.5:0/1078170020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562a752e4d0 0x5562a78fd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:22.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.523+0000 7f67b2ffd640 1 -- 172.21.15.5:0/1078170020 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562a78051f0 con 0x5562a75262f0 2024-05-09T00:59:22.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.530+0000 7f67b2ffd640 1 --2- 172.21.15.5:0/1078170020 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562a75262f0 0x5562a78fd4b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f67a40319c0 tx=0x7f67a40d0430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bf4f8268fcc1f04 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:59:22.543 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.541+0000 7f679a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-09T00:59:22.543 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.541+0000 7f679a7fc640 1 -- 172.21.15.5:0/1078170020 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a404b040 con 0x5562a75262f0 2024-05-09T00:59:22.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.541+0000 7f67c05b0740 1 -- 172.21.15.5:0/1078170020 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67ac01c440 con 0x5562a75262f0 2024-05-09T00:59:22.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.542+0000 7f679a7fc640 1 -- 172.21.15.5:0/1078170020 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f67a4109110 con 0x5562a75262f0 2024-05-09T00:59:22.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.547+0000 7f679a7fc640 1 -- 172.21.15.5:0/1078170020 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f67a40cc040 con 0x5562a75262f0 2024-05-09T00:59:22.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.549+0000 7f67c05b0740 10 client.26276 fetch_fsmap learned FSMap version 31 2024-05-09T00:59:22.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.549+0000 7f67c05b0740 1 -- 172.21.15.5:0/1078170020 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5562a75ba790 con 0x5562a75262f0 2024-05-09T00:59:22.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.551+0000 7f679a7fc640 1 -- 172.21.15.5:0/1078170020 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67ac01c440 con 0x5562a75262f0 2024-05-09T00:59:22.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.551+0000 7f679a7fc640 1 -- 172.21.15.5:0/1078170020 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f67a40d94b0 con 0x5562a75262f0 2024-05-09T00:59:22.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.551+0000 7f67c05b0740 10 client.26276 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:59:22.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.551+0000 7f67c05b0740 10 client.26276 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:59:22.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.551+0000 7f67c05b0740 20 client.26276 populate_metadata read hostname 'smithi005' 2024-05-09T00:59:22.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.551+0000 7f67c05b0740 1 --2- 172.21.15.5:0/1078170020 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5562a750dd80 0x5562a790ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:22.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.551+0000 7f67c05b0740 4 client.26276 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:59:22.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.551+0000 7f67c05b0740 1 -- 172.21.15.5:0/1078170020 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5562a78653d0 con 0x5562a750dd80 2024-05-09T00:59:22.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.559+0000 7f67abfff640 1 --2- 172.21.15.5:0/1078170020 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5562a750dd80 0x5562a790ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:22.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.577+0000 7f67abfff640 1 --2- 172.21.15.5:0/1078170020 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5562a750dd80 0x5562a790ae50 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82a3e3f7a10c6e2 server_cookie=a7974afddd1e1edf in_seq=0 out_seq=0 2024-05-09T00:59:22.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.577+0000 7f679a7fc640 10 client.26276 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:59:22.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.583+0000 7f679a7fc640 1 -- 172.21.15.5:0/1078170020 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5562a78653d0 con 0x5562a750dd80 2024-05-09T00:59:22.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.583+0000 7f679a7fc640 10 client.26276 handle_command_reply: tid=0 2024-05-09T00:59:22.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.584+0000 7f67c05b0740 1 client.26276 shutdown 2024-05-09T00:59:22.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.594+0000 7f67c05b0740 1 -- 172.21.15.5:0/1078170020 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562a75262f0 msgr2=0x5562a78fd4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:22.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.594+0000 7f67c05b0740 1 --2- 172.21.15.5:0/1078170020 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562a75262f0 0x5562a78fd4b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f67a40319c0 tx=0x7f67a40d0430 comp rx=0 tx=0).stop 2024-05-09T00:59:22.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.595+0000 7f67c05b0740 1 -- 172.21.15.5:0/1078170020 shutdown_connections 2024-05-09T00:59:22.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.595+0000 7f67c05b0740 1 --2- 172.21.15.5:0/1078170020 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5562a750dd80 0x5562a790ae50 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:22.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.595+0000 7f67c05b0740 1 --2- 172.21.15.5:0/1078170020 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5562a754f1b0 0x5562a7904950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:22.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.595+0000 7f67c05b0740 1 --2- 172.21.15.5:0/1078170020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562a752e4d0 0x5562a78fd9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:22.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.595+0000 7f67c05b0740 1 --2- 172.21.15.5:0/1078170020 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562a75262f0 0x5562a78fd4b0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:22.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.595+0000 7f67c05b0740 1 -- 172.21.15.5:0/1078170020 >> 172.21.15.5:0/1078170020 conn(0x5562a7543300 msgr2=0x5562a78ee290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:59:22.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.596+0000 7f679a7fc640 0 client.26276 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:59:22.597 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.596+0000 7f67c05b0740 1 -- 172.21.15.5:0/1078170020 shutdown_connections 2024-05-09T00:59:22.597 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.596+0000 7f67c05b0740 1 -- 172.21.15.5:0/1078170020 wait complete. 2024-05-09T00:59:22.597 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.596+0000 7f67c05b0740 20 client.26276 trim_cache size 0 max 16384 2024-05-09T00:59:22.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.642+0000 7f67c05b0740 1 Processor -- start 2024-05-09T00:59:22.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.642+0000 7f67c05b0740 1 -- start start 2024-05-09T00:59:22.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.643+0000 7f67c05b0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5562a75262f0 0x5562a78f7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:22.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.644+0000 7f67c05b0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562a752e4d0 0x5562a78f7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:22.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.644+0000 7f67c05b0740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562a754f1b0 0x5562a78fc740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:22.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.644+0000 7f67c05b0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f679400e570 con 0x5562a752e4d0 2024-05-09T00:59:22.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.644+0000 7f67c05b0740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f67940100b0 con 0x5562a754f1b0 2024-05-09T00:59:22.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.645+0000 7f67c05b0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f679400ec40 con 0x5562a75262f0 2024-05-09T00:59:22.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.645+0000 7f67b2ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5562a75262f0 0x5562a78f7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:22.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.645+0000 7f67b2ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5562a75262f0 0x5562a78f7870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:48268/0 (socket says 172.21.15.5:48268) 2024-05-09T00:59:22.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.645+0000 7f67b2ffd640 1 -- 172.21.15.5:0/2041159084 learned_addr learned my addr 172.21.15.5:0/2041159084 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:59:22.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.645+0000 7f67abfff640 1 --2- 172.21.15.5:0/2041159084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562a752e4d0 0x5562a78f7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:22.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.646+0000 7f67b2ffd640 1 -- 172.21.15.5:0/2041159084 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562a754f1b0 msgr2=0x5562a78fc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:59:22.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.646+0000 7f67b2ffd640 1 --2- 172.21.15.5:0/2041159084 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562a754f1b0 0x5562a78fc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:22.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.646+0000 7f67b2ffd640 1 -- 172.21.15.5:0/2041159084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562a752e4d0 msgr2=0x5562a78f7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:59:22.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.646+0000 7f67b2ffd640 1 --2- 172.21.15.5:0/2041159084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562a752e4d0 0x5562a78f7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:22.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.646+0000 7f67b2ffd640 1 -- 172.21.15.5:0/2041159084 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562a76315f0 con 0x5562a75262f0 2024-05-09T00:59:22.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.647+0000 7f67b2ffd640 1 --2- 172.21.15.5:0/2041159084 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5562a75262f0 0x5562a78f7870 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f67a4005ab0 tx=0x7f67a40d9870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f4072c1035a9225 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:59:22.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.648+0000 7f67a8ff9640 1 -- 172.21.15.5:0/2041159084 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a4094070 con 0x5562a75262f0 2024-05-09T00:59:22.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.649+0000 7f67a8ff9640 1 -- 172.21.15.5:0/2041159084 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67a40d05f0 con 0x5562a75262f0 2024-05-09T00:59:22.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.650+0000 7f67c05b0740 1 -- 172.21.15.5:0/2041159084 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5562a75262f0 msgr2=0x5562a78f7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:22.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.650+0000 7f67c05b0740 1 --2- 172.21.15.5:0/2041159084 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5562a75262f0 0x5562a78f7870 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f67a4005ab0 tx=0x7f67a40d9870 comp rx=0 tx=0).stop 2024-05-09T00:59:22.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.651+0000 7f67c05b0740 1 -- 172.21.15.5:0/2041159084 shutdown_connections 2024-05-09T00:59:22.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.651+0000 7f67c05b0740 1 --2- 172.21.15.5:0/2041159084 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562a754f1b0 0x5562a78fc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:22.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.651+0000 7f67c05b0740 1 --2- 172.21.15.5:0/2041159084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562a752e4d0 0x5562a78f7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:22.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.651+0000 7f67c05b0740 1 --2- 172.21.15.5:0/2041159084 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5562a75262f0 0x5562a78f7870 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:22.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.651+0000 7f67c05b0740 1 -- 172.21.15.5:0/2041159084 >> 172.21.15.5:0/2041159084 conn(0x5562a7543300 msgr2=0x5562a78fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:59:22.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.651+0000 7f67c05b0740 1 -- 172.21.15.5:0/2041159084 shutdown_connections 2024-05-09T00:59:22.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.652+0000 7f67c05b0740 1 -- 172.21.15.5:0/2041159084 wait complete. 2024-05-09T00:59:22.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.652+0000 7f67c05b0740 1 -- start start 2024-05-09T00:59:22.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.653+0000 7f67c05b0740 1 Processor -- start 2024-05-09T00:59:22.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.653+0000 7f67c05b0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562a752e4d0 0x5562a7903e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:22.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.654+0000 7f67c05b0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5562a754f1b0 0x5562a7900b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:22.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.654+0000 7f67c05b0740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562a74e31c0 0x5562a7901090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:22.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.654+0000 7f67c05b0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f679400f9e0 con 0x5562a752e4d0 2024-05-09T00:59:22.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.654+0000 7f67c05b0740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f679400f310 con 0x5562a74e31c0 2024-05-09T00:59:22.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.654+0000 7f67c05b0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f679400c550 con 0x5562a754f1b0 2024-05-09T00:59:22.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.654+0000 7f67abfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5562a754f1b0 0x5562a7900b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:22.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.655+0000 7f67abfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5562a754f1b0 0x5562a7900b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:48272/0 (socket says 172.21.15.5:48272) 2024-05-09T00:59:22.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.655+0000 7f67abfff640 1 -- 172.21.15.5:0/947027936 learned_addr learned my addr 172.21.15.5:0/947027936 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:59:22.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.655+0000 7f67b37fe640 1 --2- 172.21.15.5:0/947027936 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562a74e31c0 0x5562a7901090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:22.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.655+0000 7f67c05b0740 10 client.0 _ll_register_callbacks cb 0x7f67a4009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:59:22.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.655+0000 7f67abfff640 1 -- 172.21.15.5:0/947027936 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562a74e31c0 msgr2=0x5562a7901090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:59:22.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.655+0000 7f67abfff640 1 --2- 172.21.15.5:0/947027936 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562a74e31c0 0x5562a7901090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:22.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.655+0000 7f67abfff640 1 -- 172.21.15.5:0/947027936 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562a752e4d0 msgr2=0x5562a7903e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:59:22.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.655+0000 7f67abfff640 1 --2- 172.21.15.5:0/947027936 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562a752e4d0 0x5562a7903e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:22.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.656+0000 7f67abfff640 1 -- 172.21.15.5:0/947027936 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562a781bed0 con 0x5562a754f1b0 2024-05-09T00:59:22.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.656+0000 7f67abfff640 1 --2- 172.21.15.5:0/947027936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5562a754f1b0 0x5562a7900b50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x5562a78efde0 tx=0x7f679c00af30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7efc8897f2be9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:59:22.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.656+0000 7f67c05b0740 1 -- 172.21.15.5:0/947027936 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5562a785f240 con 0x5562a754f1b0 2024-05-09T00:59:22.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.657+0000 7f679a7fc640 10 client.26263 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-09T00:59:22.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.657+0000 7f679a7fc640 1 -- 172.21.15.5:0/947027936 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f679c024060 con 0x5562a754f1b0 2024-05-09T00:59:22.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.657+0000 7f679a7fc640 1 -- 172.21.15.5:0/947027936 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f679c01f430 con 0x5562a754f1b0 2024-05-09T00:59:22.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.657+0000 7f679a7fc640 1 -- 172.21.15.5:0/947027936 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f679c01f670 con 0x5562a754f1b0 2024-05-09T00:59:22.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.659+0000 7f67c05b0740 10 client.26263 fetch_fsmap learned FSMap version 31 2024-05-09T00:59:22.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.659+0000 7f67c05b0740 1 -- 172.21.15.5:0/947027936 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({fsmap=31}) -- 0x5562a7972250 con 0x5562a754f1b0 2024-05-09T00:59:22.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.660+0000 7f679a7fc640 1 -- 172.21.15.5:0/947027936 <== mon.1 v2:172.21.15.157:3300/0 4 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f679c01f810 con 0x5562a754f1b0 2024-05-09T00:59:22.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.660+0000 7f67c05b0740 10 client.26263 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:59:22.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.660+0000 7f67c05b0740 10 client.26263 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:59:22.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.660+0000 7f67c05b0740 20 client.26263 populate_metadata read hostname 'smithi005' 2024-05-09T00:59:22.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.660+0000 7f67c05b0740 1 --2- 172.21.15.5:0/947027936 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5562a7972a20 0x5562a7974e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:22.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.660+0000 7f67c05b0740 4 client.26263 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:59:22.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.660+0000 7f67c05b0740 1 -- 172.21.15.5:0/947027936 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5562a785f240 con 0x5562a7972a20 2024-05-09T00:59:22.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.665+0000 7f67b2ffd640 1 --2- 172.21.15.5:0/947027936 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5562a7972a20 0x5562a7974e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:22.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.668+0000 7f67b2ffd640 1 --2- 172.21.15.5:0/947027936 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5562a7972a20 0x5562a7974e90 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b06ccd7eacb7477 server_cookie=e2580b1cfb26c7b1 in_seq=0 out_seq=0 2024-05-09T00:59:22.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:22.669+0000 7f679a7fc640 10 client.26263 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:59:23.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.634+0000 7f71ccb90700 1 -- 172.21.15.5:0/793418699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71c8108570 msgr2=0x7f71c810fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:23.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.634+0000 7f71ccb90700 1 --2- 172.21.15.5:0/793418699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71c8108570 0x7f71c810fed0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f71c000f560 tx=0x7f71c000f870 comp rx=0 tx=0).stop 2024-05-09T00:59:23.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.635+0000 7f71ccb90700 1 -- 172.21.15.5:0/793418699 shutdown_connections 2024-05-09T00:59:23.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.635+0000 7f71ccb90700 1 --2- 172.21.15.5:0/793418699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71c8108570 0x7f71c810fed0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:23.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.635+0000 7f71ccb90700 1 --2- 172.21.15.5:0/793418699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71c81032f0 0x7f71c8103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:23.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.635+0000 7f71ccb90700 1 --2- 172.21.15.5:0/793418699 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71c8107c20 0x7f71c8108030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:23.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.635+0000 7f71ccb90700 1 -- 172.21.15.5:0/793418699 >> 172.21.15.5:0/793418699 conn(0x7f71c8078b00 msgr2=0x7f71c8078f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:59:23.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.638+0000 7f71ccb90700 1 -- 172.21.15.5:0/793418699 shutdown_connections 2024-05-09T00:59:23.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.638+0000 7f71ccb90700 1 -- 172.21.15.5:0/793418699 wait complete. 2024-05-09T00:59:23.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.640+0000 7f71ccb90700 1 Processor -- start 2024-05-09T00:59:23.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.640+0000 7f71ccb90700 1 -- start start 2024-05-09T00:59:23.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.641+0000 7f71ccb90700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71c81032f0 0x7f71c8118ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:23.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.642+0000 7f71ccb90700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71c8107c20 0x7f71c8112e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:23.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.642+0000 7f71ccb90700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71c8108570 0x7f71c81134a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:23.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.642+0000 7f71ccb90700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f71c8111370 con 0x7f71c81032f0 2024-05-09T00:59:23.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.642+0000 7f71ccb90700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f71c81111f0 con 0x7f71c8107c20 2024-05-09T00:59:23.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.642+0000 7f71c6ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71c8107c20 0x7f71c8112e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:23.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.642+0000 7f71c6ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71c8107c20 0x7f71c8112e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:35968/0 (socket says 172.21.15.5:35968) 2024-05-09T00:59:23.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.642+0000 7f71c6ffd700 1 -- 172.21.15.5:0/295076854 learned_addr learned my addr 172.21.15.5:0/295076854 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:59:23.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.643+0000 7f71c77fe700 1 --2- 172.21.15.5:0/295076854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71c81032f0 0x7f71c8118ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:23.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.643+0000 7f71c7fff700 1 --2- 172.21.15.5:0/295076854 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71c8108570 0x7f71c81134a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:23.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.643+0000 7f71ccb90700 1 -- 172.21.15.5:0/295076854 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f71c81114f0 con 0x7f71c8108570 2024-05-09T00:59:23.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.644+0000 7f71c7fff700 1 -- 172.21.15.5:0/295076854 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71c8107c20 msgr2=0x7f71c8112e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:23.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.644+0000 7f71c7fff700 1 --2- 172.21.15.5:0/295076854 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71c8107c20 0x7f71c8112e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:23.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.644+0000 7f71c7fff700 1 -- 172.21.15.5:0/295076854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71c81032f0 msgr2=0x7f71c8118ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:23.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.644+0000 7f71c7fff700 1 --2- 172.21.15.5:0/295076854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71c81032f0 0x7f71c8118ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:23.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.644+0000 7f71c7fff700 1 -- 172.21.15.5:0/295076854 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71c81b5db0 con 0x7f71c8108570 2024-05-09T00:59:23.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.645+0000 7f71c7fff700 1 --2- 172.21.15.5:0/295076854 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71c8108570 0x7f71c81134a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f71c000fc00 tx=0x7f71c0017900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:59:23.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.646+0000 7f71c4ff9700 1 -- 172.21.15.5:0/295076854 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c001c030 con 0x7f71c8108570 2024-05-09T00:59:23.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.646+0000 7f71c4ff9700 1 -- 172.21.15.5:0/295076854 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71c000d070 con 0x7f71c8108570 2024-05-09T00:59:23.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.648+0000 7f71c4ff9700 1 -- 172.21.15.5:0/295076854 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c00204b0 con 0x7f71c8108570 2024-05-09T00:59:23.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.649+0000 7f71ccb90700 1 -- 172.21.15.5:0/295076854 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71c81b5f80 con 0x7f71c8108570 2024-05-09T00:59:23.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.649+0000 7f71ccb90700 1 -- 172.21.15.5:0/295076854 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71c81b6320 con 0x7f71c8108570 2024-05-09T00:59:23.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.656+0000 7f71dc808b80 1 Processor -- start 2024-05-09T00:59:23.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.664+0000 7f71c4ff9700 1 -- 172.21.15.5:0/295076854 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f71c001f030 con 0x7f71c8108570 2024-05-09T00:59:23.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.683+0000 7f71c4ff9700 1 --2- 172.21.15.5:0/295076854 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f71b00b2860 0x7f71b00b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:23.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.683+0000 7f71c4ff9700 1 -- 172.21.15.5:0/295076854 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f71c00a4f20 con 0x7f71c8108570 2024-05-09T00:59:23.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.684+0000 7f71dc808b80 1 -- start start 2024-05-09T00:59:23.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.685+0000 7f71dc808b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564065bbfd00 0x564065cc8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:23.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.685+0000 7f71dc808b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564065b454a0 0x564065cc89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:23.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.685+0000 7f71dc808b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564065cc25e0 0x564065cc29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:23.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.685+0000 7f71dc808b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564065be50a0 con 0x564065b454a0 2024-05-09T00:59:23.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.685+0000 7f71dc808b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x564065b8a9a0 con 0x564065bbfd00 2024-05-09T00:59:23.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.685+0000 7f71dc808b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x564065be1060 con 0x564065cc25e0 2024-05-09T00:59:23.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.688+0000 7f71c7fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564065cc25e0 0x564065cc29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:23.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.689+0000 7f71c6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564065b454a0 0x564065cc89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:23.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.689+0000 7f71c6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564065b454a0 0x564065cc89d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34294/0 (socket says 172.21.15.5:34294) 2024-05-09T00:59:23.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.689+0000 7f71c6ffd700 1 -- 172.21.15.5:0/1626218619 learned_addr learned my addr 172.21.15.5:0/1626218619 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:59:23.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.689+0000 7f71c6ffd700 1 --2- 172.21.15.5:0/295076854 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f71b00b2860 0x7f71b00b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:23.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.696+0000 7f71c6ffd700 1 -- 172.21.15.5:0/1626218619 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564065cc25e0 msgr2=0x564065cc29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:59:23.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.696+0000 7f71c6ffd700 1 --2- 172.21.15.5:0/1626218619 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564065cc25e0 0x564065cc29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:23.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.696+0000 7f71c6ffd700 1 -- 172.21.15.5:0/1626218619 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564065bbfd00 msgr2=0x564065cc8490 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:59:23.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.696+0000 7f71c6ffd700 1 --2- 172.21.15.5:0/1626218619 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564065bbfd00 0x564065cc8490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:23.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.696+0000 7f71c6ffd700 1 -- 172.21.15.5:0/1626218619 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564065b7b500 con 0x564065b454a0 2024-05-09T00:59:23.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.699+0000 7f71c6ffd700 1 --2- 172.21.15.5:0/1626218619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564065b454a0 0x564065cc89d0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f71bc019840 tx=0x7f71bc019b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd4b64f86c79bc67 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:59:23.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.700+0000 7f71c6ffd700 1 --2- 172.21.15.5:0/295076854 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f71b00b2860 0x7f71b00b4d70 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f71bc01d000 tx=0x7f71bc01f000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:59:23.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.701+0000 7f71adffb700 1 -- 172.21.15.5:0/1626218619 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71bc027070 con 0x564065b454a0 2024-05-09T00:59:23.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.701+0000 7f71adffb700 1 -- 172.21.15.5:0/1626218619 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71bc022ad0 con 0x564065b454a0 2024-05-09T00:59:23.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.701+0000 7f71adffb700 1 -- 172.21.15.5:0/1626218619 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71bc0178f0 con 0x564065b454a0 2024-05-09T00:59:23.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.705+0000 7f71dc808b80 1 -- 172.21.15.5:0/1626218619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564065b454a0 msgr2=0x564065cc89d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:23.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.705+0000 7f71dc808b80 1 --2- 172.21.15.5:0/1626218619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564065b454a0 0x564065cc89d0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f71bc019840 tx=0x7f71bc019b50 comp rx=0 tx=0).stop 2024-05-09T00:59:23.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.714+0000 7f71dc808b80 1 -- 172.21.15.5:0/1626218619 shutdown_connections 2024-05-09T00:59:23.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.714+0000 7f71dc808b80 1 --2- 172.21.15.5:0/1626218619 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564065cc25e0 0x564065cc29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:23.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.714+0000 7f71dc808b80 1 --2- 172.21.15.5:0/1626218619 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564065bbfd00 0x564065cc8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:23.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.714+0000 7f71dc808b80 1 --2- 172.21.15.5:0/1626218619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564065b454a0 0x564065cc89d0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:23.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.714+0000 7f71dc808b80 1 -- 172.21.15.5:0/1626218619 >> 172.21.15.5:0/1626218619 conn(0x564065b48f70 msgr2=0x564065cbd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:59:23.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.714+0000 7f71dc808b80 1 -- 172.21.15.5:0/1626218619 shutdown_connections 2024-05-09T00:59:23.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.715+0000 7f71dc808b80 1 -- 172.21.15.5:0/1626218619 wait complete. 2024-05-09T00:59:23.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.715+0000 7f71dc808b80 1 -- start start 2024-05-09T00:59:23.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.715+0000 7f71dc808b80 1 Processor -- start 2024-05-09T00:59:23.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.716+0000 7f71dc808b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564065b454a0 0x564065ccc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:23.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.716+0000 7f71dc808b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564065bbfd00 0x564065ccc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:23.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.716+0000 7f71dc808b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564065cc25e0 0x564065ccccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:23.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.716+0000 7f71dc808b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564065cc3330 con 0x564065cc25e0 2024-05-09T00:59:23.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.716+0000 7f71dc808b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x564065b7fb20 con 0x564065bbfd00 2024-05-09T00:59:23.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.716+0000 7f71dc808b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x564065ce0210 con 0x564065b454a0 2024-05-09T00:59:23.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.716+0000 7f71dc808b80 10 client.0 _ll_register_callbacks cb 0x564065a566c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:59:23.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.718+0000 7f71c6ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564065bbfd00 0x564065ccc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:23.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.718+0000 7f71c77fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564065b454a0 0x564065ccc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:23.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.718+0000 7f71c6ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564065bbfd00 0x564065ccc770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:35986/0 (socket says 172.21.15.5:35986) 2024-05-09T00:59:23.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.719+0000 7f71c6ffd700 1 -- 172.21.15.5:0/124224072 learned_addr learned my addr 172.21.15.5:0/124224072 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:59:23.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.719+0000 7f71c7fff700 1 --2- 172.21.15.5:0/124224072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564065cc25e0 0x564065ccccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:23.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.720+0000 7f71c6ffd700 1 -- 172.21.15.5:0/124224072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564065b454a0 msgr2=0x564065ccc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:59:23.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.720+0000 7f71c6ffd700 1 --2- 172.21.15.5:0/124224072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564065b454a0 0x564065ccc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:23.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.720+0000 7f71c6ffd700 1 -- 172.21.15.5:0/124224072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564065cc25e0 msgr2=0x564065ccccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:59:23.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.721+0000 7f71c6ffd700 1 --2- 172.21.15.5:0/124224072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564065cc25e0 0x564065ccccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:23.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.721+0000 7f71c6ffd700 1 -- 172.21.15.5:0/124224072 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564065a60490 con 0x564065bbfd00 2024-05-09T00:59:23.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.729+0000 7f71c6ffd700 1 --2- 172.21.15.5:0/124224072 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564065bbfd00 0x564065ccc770 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f71bc000c00 tx=0x7f71bc005de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b60a7d9e7d53333c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:59:23.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.730+0000 7f71a6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-09T00:59:23.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.730+0000 7f71a6ffd700 1 -- 172.21.15.5:0/124224072 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71bc027050 con 0x564065bbfd00 2024-05-09T00:59:23.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.730+0000 7f71dc808b80 1 -- 172.21.15.5:0/124224072 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71bc022ad0 con 0x564065bbfd00 2024-05-09T00:59:23.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.730+0000 7f71a6ffd700 1 -- 172.21.15.5:0/124224072 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f71bc022420 con 0x564065bbfd00 2024-05-09T00:59:23.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.732+0000 7f71a6ffd700 1 -- 172.21.15.5:0/124224072 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f71bc03b050 con 0x564065bbfd00 2024-05-09T00:59:23.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.733+0000 7f71dc808b80 10 client.26288 fetch_fsmap learned FSMap version 31 2024-05-09T00:59:23.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.733+0000 7f71dc808b80 1 -- 172.21.15.5:0/124224072 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564065ceac70 con 0x564065bbfd00 2024-05-09T00:59:23.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.736+0000 7f71a6ffd700 1 -- 172.21.15.5:0/124224072 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71bc022ad0 con 0x564065bbfd00 2024-05-09T00:59:23.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.736+0000 7f71a6ffd700 1 -- 172.21.15.5:0/124224072 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f71bc02ba70 con 0x564065bbfd00 2024-05-09T00:59:23.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.736+0000 7f71dc808b80 10 client.26288 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:59:23.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.736+0000 7f71dc808b80 10 client.26288 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-05-09T00:59:23.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.736+0000 7f71dc808b80 20 client.26288 populate_metadata read hostname 'smithi005' 2024-05-09T00:59:23.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.736+0000 7f71dc808b80 1 --2- 172.21.15.5:0/124224072 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x564065ced5b0 0x564065cefa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:23.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.736+0000 7f71dc808b80 4 client.26288 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:59:23.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.736+0000 7f71dc808b80 1 -- 172.21.15.5:0/124224072 --> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564065bf2a80 con 0x564065ced5b0 2024-05-09T00:59:23.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.741+0000 7f71c77fe700 1 --2- 172.21.15.5:0/124224072 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x564065ced5b0 0x564065cefa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:23.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.753+0000 7f71c77fe700 1 --2- 172.21.15.5:0/124224072 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x564065ced5b0 0x564065cefa20 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=708bf851a88937bf server_cookie=cd78d642ba9559f9 in_seq=0 out_seq=0 2024-05-09T00:59:23.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.754+0000 7f71a6ffd700 10 client.26288 ms_handle_connect on v2:172.21.15.131:6838/2748669631 2024-05-09T00:59:23.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.757+0000 7f71a6ffd700 1 -- 172.21.15.5:0/124224072 <== mds.2 v2:172.21.15.131:6838/2748669631 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564065bf2a80 con 0x564065ced5b0 2024-05-09T00:59:23.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.757+0000 7f71a6ffd700 10 client.26288 handle_command_reply: tid=0 2024-05-09T00:59:23.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.757+0000 7f71dc808b80 1 client.26288 shutdown 2024-05-09T00:59:23.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.758+0000 7f71dc808b80 1 -- 172.21.15.5:0/124224072 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564065bbfd00 msgr2=0x564065ccc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:23.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.758+0000 7f71dc808b80 1 --2- 172.21.15.5:0/124224072 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564065bbfd00 0x564065ccc770 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f71bc000c00 tx=0x7f71bc005de0 comp rx=0 tx=0).stop 2024-05-09T00:59:23.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.758+0000 7f71dc808b80 1 -- 172.21.15.5:0/124224072 shutdown_connections 2024-05-09T00:59:23.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.758+0000 7f71dc808b80 1 --2- 172.21.15.5:0/124224072 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x564065ced5b0 0x564065cefa20 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:23.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.758+0000 7f71dc808b80 1 --2- 172.21.15.5:0/124224072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564065cc25e0 0x564065ccccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:23.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.758+0000 7f71dc808b80 1 --2- 172.21.15.5:0/124224072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564065b454a0 0x564065ccc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:23.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.758+0000 7f71dc808b80 1 --2- 172.21.15.5:0/124224072 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564065bbfd00 0x564065ccc770 secure :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f71bc000c00 tx=0x7f71bc005de0 comp rx=0 tx=0).stop 2024-05-09T00:59:23.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.758+0000 7f71dc808b80 1 -- 172.21.15.5:0/124224072 >> 172.21.15.5:0/124224072 conn(0x564065b48f70 msgr2=0x564065cbd2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:59:23.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.758+0000 7f71a6ffd700 0 client.26288 ms_handle_reset on v2:172.21.15.131:6838/2748669631 2024-05-09T00:59:23.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.759+0000 7f71dc808b80 1 -- 172.21.15.5:0/124224072 shutdown_connections 2024-05-09T00:59:23.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.759+0000 7f71dc808b80 1 -- 172.21.15.5:0/124224072 wait complete. 2024-05-09T00:59:23.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.760+0000 7f71dc808b80 20 client.26288 trim_cache size 0 max 16384 2024-05-09T00:59:23.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.790+0000 7f71dc808b80 1 Processor -- start 2024-05-09T00:59:23.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.790+0000 7f71dc808b80 1 -- start start 2024-05-09T00:59:23.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.791+0000 7f71dc808b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564065b454a0 0x564065cc1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:23.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.792+0000 7f71dc808b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564065cc25e0 0x564065cccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:23.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.792+0000 7f71dc808b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564065cc69f0 0x564065ce8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:23.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.792+0000 7f71c6ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564065cc25e0 0x564065cccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:23.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.792+0000 7f71c6ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564065cc25e0 0x564065cccaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:35998/0 (socket says 172.21.15.5:35998) 2024-05-09T00:59:23.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.792+0000 7f71c6ffd700 1 -- 172.21.15.5:0/2138237057 learned_addr learned my addr 172.21.15.5:0/2138237057 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:59:23.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.792+0000 7f71dc808b80 1 -- 172.21.15.5:0/2138237057 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f71c80741d0 con 0x564065cc69f0 2024-05-09T00:59:23.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.792+0000 7f71c7fff700 1 --2- 172.21.15.5:0/2138237057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564065cc69f0 0x564065ce8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:23.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.793+0000 7f71dc808b80 1 -- 172.21.15.5:0/2138237057 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f71c81b6810 con 0x564065cc25e0 2024-05-09T00:59:23.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.793+0000 7f71dc808b80 1 -- 172.21.15.5:0/2138237057 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f71c80748a0 con 0x564065b454a0 2024-05-09T00:59:23.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.794+0000 7f71c6ffd700 1 -- 172.21.15.5:0/2138237057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564065b454a0 msgr2=0x564065cc1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:59:23.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.794+0000 7f71c6ffd700 1 --2- 172.21.15.5:0/2138237057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564065b454a0 0x564065cc1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:23.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.794+0000 7f71c6ffd700 1 -- 172.21.15.5:0/2138237057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564065cc69f0 msgr2=0x564065ce8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:59:23.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.794+0000 7f71c6ffd700 1 --2- 172.21.15.5:0/2138237057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564065cc69f0 0x564065ce8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:23.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.794+0000 7f71c6ffd700 1 -- 172.21.15.5:0/2138237057 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564065ccf030 con 0x564065cc25e0 2024-05-09T00:59:23.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.795+0000 7f71c6ffd700 1 --2- 172.21.15.5:0/2138237057 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564065cc25e0 0x564065cccaf0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f71bc00b450 tx=0x7f71bc00bb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d99313d127c6d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:59:23.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.797+0000 7f71ad7fa700 1 -- 172.21.15.5:0/2138237057 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71bc027070 con 0x564065cc25e0 2024-05-09T00:59:23.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.797+0000 7f71ad7fa700 1 -- 172.21.15.5:0/2138237057 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71bc017960 con 0x564065cc25e0 2024-05-09T00:59:23.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.797+0000 7f71ad7fa700 1 -- 172.21.15.5:0/2138237057 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71bc0054a0 con 0x564065cc25e0 2024-05-09T00:59:23.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.799+0000 7f71dc808b80 1 -- 172.21.15.5:0/2138237057 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564065cc25e0 msgr2=0x564065cccaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:23.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.799+0000 7f71dc808b80 1 --2- 172.21.15.5:0/2138237057 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564065cc25e0 0x564065cccaf0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f71bc00b450 tx=0x7f71bc00bb70 comp rx=0 tx=0).stop 2024-05-09T00:59:23.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.799+0000 7f71dc808b80 1 -- 172.21.15.5:0/2138237057 shutdown_connections 2024-05-09T00:59:23.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.799+0000 7f71dc808b80 1 --2- 172.21.15.5:0/2138237057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564065cc69f0 0x564065ce8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:23.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.799+0000 7f71dc808b80 1 --2- 172.21.15.5:0/2138237057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564065b454a0 0x564065cc1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:23.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.799+0000 7f71dc808b80 1 --2- 172.21.15.5:0/2138237057 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564065cc25e0 0x564065cccaf0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:23.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.799+0000 7f71dc808b80 1 -- 172.21.15.5:0/2138237057 >> 172.21.15.5:0/2138237057 conn(0x564065b48f70 msgr2=0x564065cc7390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:59:23.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.800+0000 7f71dc808b80 1 -- 172.21.15.5:0/2138237057 shutdown_connections 2024-05-09T00:59:23.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.800+0000 7f71dc808b80 1 -- 172.21.15.5:0/2138237057 wait complete. 2024-05-09T00:59:23.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.801+0000 7f71dc808b80 1 -- start start 2024-05-09T00:59:23.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.801+0000 7f71dc808b80 1 Processor -- start 2024-05-09T00:59:23.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.802+0000 7f71dc808b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564065b454a0 0x564065cc8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:23.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.803+0000 7f71dc808b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564065cc25e0 0x564065cc91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:23.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.803+0000 7f71c77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564065b454a0 0x564065cc8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:23.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.803+0000 7f71c77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564065b454a0 0x564065cc8c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34330/0 (socket says 172.21.15.5:34330) 2024-05-09T00:59:23.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.803+0000 7f71c77fe700 1 -- 172.21.15.5:0/155018444 learned_addr learned my addr 172.21.15.5:0/155018444 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:59:23.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.803+0000 7f71dc808b80 1 --2- 172.21.15.5:0/155018444 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564065cc69f0 0x564065ce4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:23.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.803+0000 7f71dc808b80 1 -- 172.21.15.5:0/155018444 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f71c8075640 con 0x564065b454a0 2024-05-09T00:59:23.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.804+0000 7f71dc808b80 1 -- 172.21.15.5:0/155018444 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f71c8074f70 con 0x564065cc69f0 2024-05-09T00:59:23.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.804+0000 7f71c6ffd700 1 --2- 172.21.15.5:0/155018444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564065cc25e0 0x564065cc91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:23.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.804+0000 7f71dc808b80 1 -- 172.21.15.5:0/155018444 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f71c8072170 con 0x564065cc25e0 2024-05-09T00:59:23.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.804+0000 7f71c7fff700 1 --2- 172.21.15.5:0/155018444 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564065cc69f0 0x564065ce4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:23.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.804+0000 7f71dc808b80 10 client.0 _ll_register_callbacks cb 0x564065ccb6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:59:23.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.805+0000 7f71c77fe700 1 -- 172.21.15.5:0/155018444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564065cc25e0 msgr2=0x564065cc91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:59:23.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.805+0000 7f71c77fe700 1 --2- 172.21.15.5:0/155018444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564065cc25e0 0x564065cc91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:23.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.805+0000 7f71c77fe700 1 -- 172.21.15.5:0/155018444 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564065cc69f0 msgr2=0x564065ce4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:59:23.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.805+0000 7f71c77fe700 1 --2- 172.21.15.5:0/155018444 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564065cc69f0 0x564065ce4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:23.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.805+0000 7f71c77fe700 1 -- 172.21.15.5:0/155018444 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564065ce4790 con 0x564065b454a0 2024-05-09T00:59:23.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.806+0000 7f71c77fe700 1 --2- 172.21.15.5:0/155018444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564065b454a0 0x564065cc8c70 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x564065cf0150 tx=0x7f71b8015700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=396e2edefbd667ab server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:59:23.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.806+0000 7f71a6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:59:23.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.806+0000 7f71a6ffd700 1 -- 172.21.15.5:0/155018444 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b8007e40 con 0x564065b454a0 2024-05-09T00:59:23.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.806+0000 7f71a6ffd700 1 -- 172.21.15.5:0/155018444 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f71b8015ce0 con 0x564065b454a0 2024-05-09T00:59:23.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.806+0000 7f71dc808b80 1 -- 172.21.15.5:0/155018444 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71bc017960 con 0x564065b454a0 2024-05-09T00:59:23.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.807+0000 7f71a6ffd700 1 -- 172.21.15.5:0/155018444 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f71b8029030 con 0x564065b454a0 2024-05-09T00:59:23.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.807+0000 7f71dc808b80 10 client.16878 fetch_fsmap learned FSMap version 31 2024-05-09T00:59:23.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.807+0000 7f71dc808b80 1 -- 172.21.15.5:0/155018444 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564065ce59f0 con 0x564065b454a0 2024-05-09T00:59:23.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.808+0000 7f71a6ffd700 1 -- 172.21.15.5:0/155018444 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71b801a680 con 0x564065b454a0 2024-05-09T00:59:23.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.808+0000 7f71a6ffd700 1 -- 172.21.15.5:0/155018444 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f71b802e8e0 con 0x564065b454a0 2024-05-09T00:59:23.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.808+0000 7f71dc808b80 10 client.16878 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:59:23.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.808+0000 7f71dc808b80 10 client.16878 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-05-09T00:59:23.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.808+0000 7f71dc808b80 20 client.16878 populate_metadata read hostname 'smithi005' 2024-05-09T00:59:23.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.808+0000 7f71dc808b80 1 --2- 172.21.15.5:0/155018444 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x564065d57fd0 0x564065d5a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:23.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.808+0000 7f71dc808b80 4 client.16878 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:59:23.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.808+0000 7f71dc808b80 1 -- 172.21.15.5:0/155018444 --> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f71bc017960 con 0x564065d57fd0 2024-05-09T00:59:23.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.809+0000 7f71c6ffd700 1 --2- 172.21.15.5:0/155018444 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x564065d57fd0 0x564065d5a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:23.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.811+0000 7f71a6ffd700 10 client.16878 ms_handle_connect on v2:172.21.15.131:6838/2748669631 2024-05-09T00:59:23.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.811+0000 7f71c6ffd700 1 --2- 172.21.15.5:0/155018444 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x564065d57fd0 0x564065d5a3f0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7f96126facd26f7e server_cookie=33c3a58a14443049 in_seq=0 out_seq=0 2024-05-09T00:59:23.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.812+0000 7f71a6ffd700 1 -- 172.21.15.5:0/155018444 <== mds.2 v2:172.21.15.131:6838/2748669631 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f71bc017960 con 0x564065d57fd0 2024-05-09T00:59:23.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.812+0000 7f71a6ffd700 10 client.16878 handle_command_reply: tid=0 2024-05-09T00:59:23.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.813+0000 7f71dc808b80 1 client.16878 shutdown 2024-05-09T00:59:23.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.813+0000 7f71dc808b80 1 -- 172.21.15.5:0/155018444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564065b454a0 msgr2=0x564065cc8c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:23.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.813+0000 7f71dc808b80 1 --2- 172.21.15.5:0/155018444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564065b454a0 0x564065cc8c70 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x564065cf0150 tx=0x7f71b8015700 comp rx=0 tx=0).stop 2024-05-09T00:59:23.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.814+0000 7f71dc808b80 1 -- 172.21.15.5:0/155018444 shutdown_connections 2024-05-09T00:59:23.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.814+0000 7f71dc808b80 1 --2- 172.21.15.5:0/155018444 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x564065d57fd0 0x564065d5a3f0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:23.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.814+0000 7f71dc808b80 1 --2- 172.21.15.5:0/155018444 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564065cc69f0 0x564065ce4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:23.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.814+0000 7f71dc808b80 1 --2- 172.21.15.5:0/155018444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564065b454a0 0x564065cc8c70 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:23.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.814+0000 7f71dc808b80 1 --2- 172.21.15.5:0/155018444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564065cc25e0 0x564065cc91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:23.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.814+0000 7f71dc808b80 1 -- 172.21.15.5:0/155018444 >> 172.21.15.5:0/155018444 conn(0x564065b48f70 msgr2=0x564065cc7390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:59:23.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.814+0000 7f71a6ffd700 0 client.16878 ms_handle_reset on v2:172.21.15.131:6838/2748669631 2024-05-09T00:59:23.816 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-09T00:59:23.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.814+0000 7f71dc808b80 1 -- 172.21.15.5:0/155018444 shutdown_connections 2024-05-09T00:59:23.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.814+0000 7f71dc808b80 1 -- 172.21.15.5:0/155018444 wait complete. 2024-05-09T00:59:23.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.814+0000 7f71dc808b80 20 client.16878 trim_cache size 0 max 16384 2024-05-09T00:59:23.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.815+0000 7f71ccb90700 1 -- 172.21.15.5:0/295076854 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f71b00b2860 msgr2=0x7f71b00b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:23.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.815+0000 7f71ccb90700 1 --2- 172.21.15.5:0/295076854 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f71b00b2860 0x7f71b00b4d70 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f71bc01d000 tx=0x7f71bc01f000 comp rx=0 tx=0).stop 2024-05-09T00:59:23.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.815+0000 7f71ccb90700 1 -- 172.21.15.5:0/295076854 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71c8108570 msgr2=0x7f71c81134a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:23.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.815+0000 7f71ccb90700 1 --2- 172.21.15.5:0/295076854 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71c8108570 0x7f71c81134a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f71c000fc00 tx=0x7f71c0017900 comp rx=0 tx=0).stop 2024-05-09T00:59:23.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.816+0000 7f71ccb90700 1 -- 172.21.15.5:0/295076854 shutdown_connections 2024-05-09T00:59:23.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.816+0000 7f71ccb90700 1 --2- 172.21.15.5:0/295076854 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f71b00b2860 0x7f71b00b4d70 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:23.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.816+0000 7f71ccb90700 1 --2- 172.21.15.5:0/295076854 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71c8108570 0x7f71c81134a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:23.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.816+0000 7f71ccb90700 1 --2- 172.21.15.5:0/295076854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71c81032f0 0x7f71c8118ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:23.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.816+0000 7f71ccb90700 1 --2- 172.21.15.5:0/295076854 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71c8107c20 0x7f71c8112e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:23.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.816+0000 7f71ccb90700 1 -- 172.21.15.5:0/295076854 >> 172.21.15.5:0/295076854 conn(0x7f71c8078b00 msgr2=0x7f71c81006b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:59:23.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.817+0000 7f71ccb90700 1 -- 172.21.15.5:0/295076854 shutdown_connections 2024-05-09T00:59:23.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:23.817+0000 7f71ccb90700 1 -- 172.21.15.5:0/295076854 wait complete. 2024-05-09T00:59:24.293 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:59:24 smithi005 ceph-mon[30365]: pgmap v762: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 847 MiB/s wr, 1.86k op/s 2024-05-09T00:59:24.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:59:24 smithi131 ceph-mon[34547]: pgmap v762: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 847 MiB/s wr, 1.86k op/s 2024-05-09T00:59:24.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:59:24 smithi157 ceph-mon[34294]: pgmap v762: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 847 MiB/s wr, 1.86k op/s 2024-05-09T00:59:25.637 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T00:59:26.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:59:26 smithi005 ceph-mon[30365]: pgmap v763: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 885 MiB/s wr, 1.99k op/s 2024-05-09T00:59:26.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:59:26 smithi131 ceph-mon[34547]: pgmap v763: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 885 MiB/s wr, 1.99k op/s 2024-05-09T00:59:26.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:59:26 smithi157 ceph-mon[34294]: pgmap v763: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 885 MiB/s wr, 1.99k op/s 2024-05-09T00:59:26.622 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:59:27.485 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:59:27 smithi005 ceph-mon[30365]: pgmap v764: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 911 MiB/s wr, 1.90k op/s 2024-05-09T00:59:27.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:59:27 smithi131 ceph-mon[34547]: pgmap v764: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 911 MiB/s wr, 1.90k op/s 2024-05-09T00:59:27.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:59:27 smithi157 ceph-mon[34294]: pgmap v764: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 911 MiB/s wr, 1.90k op/s 2024-05-09T00:59:28.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:28.647+0000 7f679a7fc640 1 -- 172.21.15.5:0/947027936 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5562a785f240 con 0x5562a7972a20 2024-05-09T00:59:28.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:28.647+0000 7f679a7fc640 10 client.26263 handle_command_reply: tid=0 2024-05-09T00:59:28.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:28.648+0000 7f67c05b0740 1 client.26263 shutdown 2024-05-09T00:59:28.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:28.649+0000 7f67c05b0740 1 -- 172.21.15.5:0/947027936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5562a754f1b0 msgr2=0x5562a7900b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:28.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:28.649+0000 7f67c05b0740 1 --2- 172.21.15.5:0/947027936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5562a754f1b0 0x5562a7900b50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x5562a78efde0 tx=0x7f679c00af30 comp rx=0 tx=0).stop 2024-05-09T00:59:28.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:28.649+0000 7f67c05b0740 1 -- 172.21.15.5:0/947027936 shutdown_connections 2024-05-09T00:59:28.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:28.649+0000 7f67c05b0740 1 --2- 172.21.15.5:0/947027936 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5562a7972a20 0x5562a7974e90 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:28.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:28.649+0000 7f67c05b0740 1 --2- 172.21.15.5:0/947027936 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562a74e31c0 0x5562a7901090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:28.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:28.649+0000 7f67c05b0740 1 --2- 172.21.15.5:0/947027936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5562a754f1b0 0x5562a7900b50 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:28.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:28.649+0000 7f67c05b0740 1 --2- 172.21.15.5:0/947027936 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562a752e4d0 0x5562a7903e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:28.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:28.649+0000 7f67c05b0740 1 -- 172.21.15.5:0/947027936 >> 172.21.15.5:0/947027936 conn(0x5562a7543300 msgr2=0x5562a78fc740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:59:28.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:28.649+0000 7f679a7fc640 0 client.26263 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:59:28.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:28.649+0000 7f67c05b0740 1 -- 172.21.15.5:0/947027936 shutdown_connections 2024-05-09T00:59:28.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:28.649+0000 7f67c05b0740 1 -- 172.21.15.5:0/947027936 wait complete. 2024-05-09T00:59:28.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:28.650+0000 7f67c05b0740 20 client.26263 trim_cache size 0 max 16384 2024-05-09T00:59:28.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-09T00:59:28.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-09T00:59:28.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-09T00:59:28.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:28.659+0000 7f67b3fff640 1 -- 172.21.15.5:0/650464990 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f67a00b29f0 msgr2=0x7f67a00b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:28.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:28.659+0000 7f67b3fff640 1 --2- 172.21.15.5:0/650464990 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f67a00b29f0 0x7f67a00b4f00 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f67ac01f000 tx=0x7f67ac01b8a0 comp rx=0 tx=0).stop 2024-05-09T00:59:28.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:28.659+0000 7f67b3fff640 1 -- 172.21.15.5:0/650464990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67b4078f20 msgr2=0x7f67b4084bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:28.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:28.659+0000 7f67b3fff640 1 --2- 172.21.15.5:0/650464990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67b4078f20 0x7f67b4084bb0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f67a4000c00 tx=0x7f67a4005560 comp rx=0 tx=0).stop 2024-05-09T00:59:28.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:28.667+0000 7f67b3fff640 1 -- 172.21.15.5:0/650464990 shutdown_connections 2024-05-09T00:59:28.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:28.667+0000 7f67b3fff640 1 --2- 172.21.15.5:0/650464990 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f67a00b29f0 0x7f67a00b4f00 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:28.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:28.667+0000 7f67b3fff640 1 --2- 172.21.15.5:0/650464990 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67b410b580 0x7f67b41c1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:28.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:28.667+0000 7f67b3fff640 1 --2- 172.21.15.5:0/650464990 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67b410a210 0x7f67b40850f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:28.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:28.667+0000 7f67b3fff640 1 --2- 172.21.15.5:0/650464990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67b4078f20 0x7f67b4084bb0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:28.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:28.667+0000 7f67b3fff640 1 -- 172.21.15.5:0/650464990 >> 172.21.15.5:0/650464990 conn(0x7f67b4070460 msgr2=0x7f67b4074ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:59:28.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:28.669+0000 7f67b3fff640 1 -- 172.21.15.5:0/650464990 shutdown_connections 2024-05-09T00:59:28.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:28.674+0000 7f67b3fff640 1 -- 172.21.15.5:0/650464990 wait complete. 2024-05-09T00:59:29.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:59:29 smithi005 ceph-mon[30365]: pgmap v765: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 905 MiB/s wr, 1.73k op/s 2024-05-09T00:59:29.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:59:29 smithi131 ceph-mon[34547]: pgmap v765: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 905 MiB/s wr, 1.73k op/s 2024-05-09T00:59:29.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:59:29 smithi157 ceph-mon[34294]: pgmap v765: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 905 MiB/s wr, 1.73k op/s 2024-05-09T00:59:31.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.545+0000 7f5b80ab0700 1 -- 172.21.15.5:0/73107088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b7c108f10 msgr2=0x7f5b7c109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:31.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.545+0000 7f5b80ab0700 1 --2- 172.21.15.5:0/73107088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b7c108f10 0x7f5b7c109320 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f5b740099c0 tx=0x7f5b74009cd0 comp rx=0 tx=0).stop 2024-05-09T00:59:31.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.546+0000 7f5b80ab0700 1 -- 172.21.15.5:0/73107088 shutdown_connections 2024-05-09T00:59:31.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.546+0000 7f5b80ab0700 1 --2- 172.21.15.5:0/73107088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b7c109860 0x7f5b7c1111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:31.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.546+0000 7f5b80ab0700 1 --2- 172.21.15.5:0/73107088 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b7c1045e0 0x7f5b7c1049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:31.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.546+0000 7f5b80ab0700 1 --2- 172.21.15.5:0/73107088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b7c108f10 0x7f5b7c109320 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:31.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.546+0000 7f5b80ab0700 1 -- 172.21.15.5:0/73107088 >> 172.21.15.5:0/73107088 conn(0x7f5b7c0ffed0 msgr2=0x7f5b7c102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:59:31.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.548+0000 7f5b80ab0700 1 -- 172.21.15.5:0/73107088 shutdown_connections 2024-05-09T00:59:31.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.548+0000 7f5b80ab0700 1 -- 172.21.15.5:0/73107088 wait complete. 2024-05-09T00:59:31.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.550+0000 7f5b80ab0700 1 Processor -- start 2024-05-09T00:59:31.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.551+0000 7f5b80ab0700 1 -- start start 2024-05-09T00:59:31.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.551+0000 7f5b80ab0700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b7c1045e0 0x7f5b7c10d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:31.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.551+0000 7f5b80ab0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b7c108f10 0x7f5b7c10d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:31.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.552+0000 7f5b80ab0700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b7c109860 0x7f5b7c10dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:31.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.552+0000 7f5b80ab0700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5b7c077260 con 0x7f5b7c108f10 2024-05-09T00:59:31.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.552+0000 7f5b80ab0700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5b7c0770e0 con 0x7f5b7c109860 2024-05-09T00:59:31.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.552+0000 7f5b80ab0700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5b7c0773e0 con 0x7f5b7c1045e0 2024-05-09T00:59:31.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.554+0000 7f5b7ad9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b7c109860 0x7f5b7c10dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:31.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.554+0000 7f5b7ad9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b7c109860 0x7f5b7c10dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:36022/0 (socket says 172.21.15.5:36022) 2024-05-09T00:59:31.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.554+0000 7f5b7ad9d700 1 -- 172.21.15.5:0/2109973023 learned_addr learned my addr 172.21.15.5:0/2109973023 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:59:31.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.555+0000 7f5b7a59c700 1 --2- 172.21.15.5:0/2109973023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b7c1045e0 0x7f5b7c10d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:31.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.556+0000 7f5b79d9b700 1 --2- 172.21.15.5:0/2109973023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b7c108f10 0x7f5b7c10d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:31.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.560+0000 7f5b7ad9d700 1 -- 172.21.15.5:0/2109973023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b7c1045e0 msgr2=0x7f5b7c10d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:31.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.560+0000 7f5b7ad9d700 1 --2- 172.21.15.5:0/2109973023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b7c1045e0 0x7f5b7c10d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:31.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.560+0000 7f5b7ad9d700 1 -- 172.21.15.5:0/2109973023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b7c108f10 msgr2=0x7f5b7c10d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:31.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.560+0000 7f5b7ad9d700 1 --2- 172.21.15.5:0/2109973023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b7c108f10 0x7f5b7c10d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:31.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.560+0000 7f5b7ad9d700 1 -- 172.21.15.5:0/2109973023 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b7c1a4fe0 con 0x7f5b7c109860 2024-05-09T00:59:31.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.564+0000 7f5b7ad9d700 1 --2- 172.21.15.5:0/2109973023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b7c109860 0x7f5b7c10dde0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c00b4a0 tx=0x7f5b6c00b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:59:31.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.566+0000 7f5b737fe700 1 -- 172.21.15.5:0/2109973023 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c015020 con 0x7f5b7c109860 2024-05-09T00:59:31.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.566+0000 7f5b737fe700 1 -- 172.21.15.5:0/2109973023 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b6c011ce0 con 0x7f5b7c109860 2024-05-09T00:59:31.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.566+0000 7f5b737fe700 1 -- 172.21.15.5:0/2109973023 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c010d40 con 0x7f5b7c109860 2024-05-09T00:59:31.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.567+0000 7f5b80ab0700 1 -- 172.21.15.5:0/2109973023 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b7c1a52d0 con 0x7f5b7c109860 2024-05-09T00:59:31.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.567+0000 7f5b80ab0700 1 -- 172.21.15.5:0/2109973023 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b7c1b67b0 con 0x7f5b7c109860 2024-05-09T00:59:31.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.586+0000 7f5b737fe700 1 -- 172.21.15.5:0/2109973023 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f5b6c018020 con 0x7f5b7c109860 2024-05-09T00:59:31.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.590+0000 7f5b8f4c6b80 1 Processor -- start 2024-05-09T00:59:31.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.591+0000 7f5b8f4c6b80 1 -- start start 2024-05-09T00:59:31.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.591+0000 7f5b737fe700 1 --2- 172.21.15.5:0/2109973023 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f5b5c0b2a50 0x7f5b5c0b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:31.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.592+0000 7f5b737fe700 1 -- 172.21.15.5:0/2109973023 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f5b6c0e2960 con 0x7f5b7c109860 2024-05-09T00:59:31.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.593+0000 7f5b8f4c6b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5629262add00 0x5629263b6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:31.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.594+0000 7f5b8f4c6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629262334a0 0x5629263b6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:31.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.594+0000 7f5b8f4c6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5629263b0400 0x5629263b0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:31.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.594+0000 7f5b8f4c6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5629262d30a0 con 0x5629262334a0 2024-05-09T00:59:31.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.594+0000 7f5b8f4c6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5629262789a0 con 0x5629263b0400 2024-05-09T00:59:31.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.594+0000 7f5b8f4c6b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5629262cf060 con 0x5629262add00 2024-05-09T00:59:31.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.595+0000 7f5b79d9b700 1 --2- 172.21.15.5:0/2109973023 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f5b5c0b2a50 0x7f5b5c0b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:31.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.596+0000 7f5b7ad9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5629263b0400 0x5629263b0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:31.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.596+0000 7f5b7ad9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5629263b0400 0x5629263b0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:36034/0 (socket says 172.21.15.5:36034) 2024-05-09T00:59:31.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.596+0000 7f5b7ad9d700 1 -- 172.21.15.5:0/835816144 learned_addr learned my addr 172.21.15.5:0/835816144 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:59:31.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.596+0000 7f5b7ad9d700 1 -- 172.21.15.5:0/835816144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5629262add00 msgr2=0x5629263b6240 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:59:31.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.596+0000 7f5b7ad9d700 1 --2- 172.21.15.5:0/835816144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5629262add00 0x5629263b6240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:31.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.597+0000 7f5b7ad9d700 1 -- 172.21.15.5:0/835816144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629262334a0 msgr2=0x5629263b6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:59:31.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.597+0000 7f5b7ad9d700 1 --2- 172.21.15.5:0/835816144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629262334a0 0x5629263b6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:31.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.597+0000 7f5b7ad9d700 1 -- 172.21.15.5:0/835816144 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562926269500 con 0x5629263b0400 2024-05-09T00:59:31.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.598+0000 7f5b7ad9d700 1 --2- 172.21.15.5:0/835816144 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5629263b0400 0x5629263b0810 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c00b470 tx=0x7f5b6c0a9660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0cdff44852fa071 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:59:31.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.600+0000 7f5b70ff9700 1 -- 172.21.15.5:0/835816144 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c0a9ef0 con 0x5629263b0400 2024-05-09T00:59:31.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.600+0000 7f5b70ff9700 1 -- 172.21.15.5:0/835816144 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b6c0ae2a0 con 0x5629263b0400 2024-05-09T00:59:31.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.600+0000 7f5b70ff9700 1 -- 172.21.15.5:0/835816144 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c010380 con 0x5629263b0400 2024-05-09T00:59:31.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.602+0000 7f5b8f4c6b80 1 -- 172.21.15.5:0/835816144 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5629263b0400 msgr2=0x5629263b0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:31.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.602+0000 7f5b8f4c6b80 1 --2- 172.21.15.5:0/835816144 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5629263b0400 0x5629263b0810 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c00b470 tx=0x7f5b6c0a9660 comp rx=0 tx=0).stop 2024-05-09T00:59:31.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.606+0000 7f5b79d9b700 1 --2- 172.21.15.5:0/2109973023 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f5b5c0b2a50 0x7f5b5c0b4f60 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5b74009990 tx=0x7f5b74005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:59:31.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.606+0000 7f5b8f4c6b80 1 -- 172.21.15.5:0/835816144 shutdown_connections 2024-05-09T00:59:31.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.606+0000 7f5b8f4c6b80 1 --2- 172.21.15.5:0/835816144 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5629263b0400 0x5629263b0810 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:31.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.606+0000 7f5b8f4c6b80 1 --2- 172.21.15.5:0/835816144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5629262add00 0x5629263b6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:31.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.606+0000 7f5b8f4c6b80 1 --2- 172.21.15.5:0/835816144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629262334a0 0x5629263b6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:31.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.606+0000 7f5b8f4c6b80 1 -- 172.21.15.5:0/835816144 >> 172.21.15.5:0/835816144 conn(0x562926236f70 msgr2=0x5629263ab220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:59:31.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.610+0000 7f5b8f4c6b80 1 -- 172.21.15.5:0/835816144 shutdown_connections 2024-05-09T00:59:31.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.614+0000 7f5b8f4c6b80 1 -- 172.21.15.5:0/835816144 wait complete. 2024-05-09T00:59:31.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.615+0000 7f5b8f4c6b80 1 -- start start 2024-05-09T00:59:31.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.616+0000 7f5b8f4c6b80 1 Processor -- start 2024-05-09T00:59:31.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.617+0000 7f5b8f4c6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5629262334a0 0x5629263b9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:31.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.617+0000 7f5b8f4c6b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5629262add00 0x5629263ba4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:31.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.617+0000 7f5b8f4c6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629263b0400 0x5629263baa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:31.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.618+0000 7f5b8f4c6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5629263b1160 con 0x5629263b0400 2024-05-09T00:59:31.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.618+0000 7f5b8f4c6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x56292626db20 con 0x5629262334a0 2024-05-09T00:59:31.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.618+0000 7f5b8f4c6b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5629263bd020 con 0x5629262add00 2024-05-09T00:59:31.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.618+0000 7f5b7a59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5629262334a0 0x5629263b9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:31.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.618+0000 7f5b7a59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5629262334a0 0x5629263b9fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:36048/0 (socket says 172.21.15.5:36048) 2024-05-09T00:59:31.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.618+0000 7f5b7a59c700 1 -- 172.21.15.5:0/2586549096 learned_addr learned my addr 172.21.15.5:0/2586549096 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:59:31.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.618+0000 7f5b8f4c6b80 10 client.0 _ll_register_callbacks cb 0x5629261446c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:59:31.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.619+0000 7f5b7a59c700 1 -- 172.21.15.5:0/2586549096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5629262add00 msgr2=0x5629263ba4f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:59:31.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.619+0000 7f5b7a59c700 1 --2- 172.21.15.5:0/2586549096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5629262add00 0x5629263ba4f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:31.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.619+0000 7f5b7a59c700 1 -- 172.21.15.5:0/2586549096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629263b0400 msgr2=0x5629263baa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:59:31.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.619+0000 7f5b7a59c700 1 --2- 172.21.15.5:0/2586549096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629263b0400 0x5629263baa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:31.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.619+0000 7f5b7a59c700 1 -- 172.21.15.5:0/2586549096 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56292614e490 con 0x5629262334a0 2024-05-09T00:59:31.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.620+0000 7f5b7a59c700 1 --2- 172.21.15.5:0/2586549096 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5629262334a0 0x5629263b9fb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5b6400cab0 tx=0x7f5b6400cb90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4dd6ca1f69c09b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:59:31.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.621+0000 7f5b69ffb700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-09T00:59:31.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.621+0000 7f5b69ffb700 1 -- 172.21.15.5:0/2586549096 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6401a060 con 0x5629262334a0 2024-05-09T00:59:31.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.621+0000 7f5b8f4c6b80 1 -- 172.21.15.5:0/2586549096 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b6c0ae2a0 con 0x5629262334a0 2024-05-09T00:59:31.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.621+0000 7f5b69ffb700 1 -- 172.21.15.5:0/2586549096 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f5b640074f0 con 0x5629262334a0 2024-05-09T00:59:31.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.650+0000 7f5b69ffb700 1 -- 172.21.15.5:0/2586549096 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5b64007730 con 0x5629262334a0 2024-05-09T00:59:31.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.651+0000 7f5b8f4c6b80 10 client.26303 fetch_fsmap learned FSMap version 31 2024-05-09T00:59:31.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.651+0000 7f5b8f4c6b80 1 -- 172.21.15.5:0/2586549096 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5629263c7a50 con 0x5629262334a0 2024-05-09T00:59:31.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.653+0000 7f5b69ffb700 1 -- 172.21.15.5:0/2586549096 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b6c0ae2a0 con 0x5629262334a0 2024-05-09T00:59:31.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.654+0000 7f5b69ffb700 1 -- 172.21.15.5:0/2586549096 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f5b640209a0 con 0x5629262334a0 2024-05-09T00:59:31.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.654+0000 7f5b8f4c6b80 10 client.26303 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:59:31.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.654+0000 7f5b8f4c6b80 10 client.26303 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-05-09T00:59:31.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.654+0000 7f5b8f4c6b80 20 client.26303 populate_metadata read hostname 'smithi005' 2024-05-09T00:59:31.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.654+0000 7f5b8f4c6b80 1 --2- 172.21.15.5:0/2586549096 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x5629263ca0e0 0x5629263cc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:31.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.654+0000 7f5b8f4c6b80 4 client.26303 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:59:31.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.654+0000 7f5b8f4c6b80 1 -- 172.21.15.5:0/2586549096 --> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629262e0a80 con 0x5629263ca0e0 2024-05-09T00:59:31.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.660+0000 7f5b79d9b700 1 --2- 172.21.15.5:0/2586549096 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x5629263ca0e0 0x5629263cc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:31.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.670+0000 7f5b79d9b700 1 --2- 172.21.15.5:0/2586549096 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x5629263ca0e0 0x5629263cc5c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=98213aaffaa56e77 server_cookie=69937326788db12b in_seq=0 out_seq=0 2024-05-09T00:59:31.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.670+0000 7f5b69ffb700 10 client.26303 ms_handle_connect on v2:172.21.15.157:6836/744863009 2024-05-09T00:59:31.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.673+0000 7f5b69ffb700 1 -- 172.21.15.5:0/2586549096 <== mds.1 v2:172.21.15.157:6836/744863009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5629262e0a80 con 0x5629263ca0e0 2024-05-09T00:59:31.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.673+0000 7f5b69ffb700 10 client.26303 handle_command_reply: tid=0 2024-05-09T00:59:31.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.674+0000 7f5b8f4c6b80 1 client.26303 shutdown 2024-05-09T00:59:31.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.679+0000 7f5b8f4c6b80 1 -- 172.21.15.5:0/2586549096 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5629262334a0 msgr2=0x5629263b9fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:31.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.679+0000 7f5b8f4c6b80 1 --2- 172.21.15.5:0/2586549096 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5629262334a0 0x5629263b9fb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5b6400cab0 tx=0x7f5b6400cb90 comp rx=0 tx=0).stop 2024-05-09T00:59:31.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.679+0000 7f5b8f4c6b80 1 -- 172.21.15.5:0/2586549096 shutdown_connections 2024-05-09T00:59:31.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.679+0000 7f5b8f4c6b80 1 --2- 172.21.15.5:0/2586549096 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x5629263ca0e0 0x5629263cc5c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:31.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.679+0000 7f5b8f4c6b80 1 --2- 172.21.15.5:0/2586549096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629263b0400 0x5629263baa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:31.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.679+0000 7f5b8f4c6b80 1 --2- 172.21.15.5:0/2586549096 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5629262334a0 0x5629263b9fb0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:31.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.679+0000 7f5b8f4c6b80 1 --2- 172.21.15.5:0/2586549096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5629262add00 0x5629263ba4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:31.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.679+0000 7f5b8f4c6b80 1 -- 172.21.15.5:0/2586549096 >> 172.21.15.5:0/2586549096 conn(0x562926236f70 msgr2=0x5629263a4ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:59:31.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.680+0000 7f5b69ffb700 0 client.26303 ms_handle_reset on v2:172.21.15.157:6836/744863009 2024-05-09T00:59:31.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.682+0000 7f5b8f4c6b80 1 -- 172.21.15.5:0/2586549096 shutdown_connections 2024-05-09T00:59:31.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.683+0000 7f5b8f4c6b80 1 -- 172.21.15.5:0/2586549096 wait complete. 2024-05-09T00:59:31.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.684+0000 7f5b8f4c6b80 20 client.26303 trim_cache size 0 max 16384 2024-05-09T00:59:31.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.729+0000 7f5b8f4c6b80 1 Processor -- start 2024-05-09T00:59:31.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.730+0000 7f5b8f4c6b80 1 -- start start 2024-05-09T00:59:31.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.731+0000 7f5b8f4c6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629262334a0 0x5629263cc800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:31.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.732+0000 7f5b7a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629262334a0 0x5629263cc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:31.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.732+0000 7f5b7a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629262334a0 0x5629263cc800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34370/0 (socket says 172.21.15.5:34370) 2024-05-09T00:59:31.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.732+0000 7f5b7a59c700 1 -- 172.21.15.5:0/3746700518 learned_addr learned my addr 172.21.15.5:0/3746700518 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:59:31.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.734+0000 7f5b8f4c6b80 1 --2- 172.21.15.5:0/3746700518 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5629262add00 0x5629263bc360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:31.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.734+0000 7f5b8f4c6b80 1 --2- 172.21.15.5:0/3746700518 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5629263b0400 0x5629263c5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:31.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.734+0000 7f5b8f4c6b80 1 -- 172.21.15.5:0/3746700518 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6000d580 con 0x5629262334a0 2024-05-09T00:59:31.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.734+0000 7f5b8f4c6b80 1 -- 172.21.15.5:0/3746700518 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6000f0c0 con 0x5629262add00 2024-05-09T00:59:31.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.734+0000 7f5b8f4c6b80 1 -- 172.21.15.5:0/3746700518 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6000dc50 con 0x5629263b0400 2024-05-09T00:59:31.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.735+0000 7f5b7a59c700 1 -- 172.21.15.5:0/3746700518 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5629263b0400 msgr2=0x5629263c5e20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:59:31.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.736+0000 7f5b7a59c700 1 --2- 172.21.15.5:0/3746700518 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5629263b0400 0x5629263c5e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:31.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.736+0000 7f5b7a59c700 1 -- 172.21.15.5:0/3746700518 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5629262add00 msgr2=0x5629263bc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:59:31.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.736+0000 7f5b7a59c700 1 --2- 172.21.15.5:0/3746700518 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5629262add00 0x5629263bc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:31.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.736+0000 7f5b7a59c700 1 -- 172.21.15.5:0/3746700518 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629263b8520 con 0x5629262334a0 2024-05-09T00:59:31.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.737+0000 7f5b7a59c700 1 --2- 172.21.15.5:0/3746700518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629262334a0 0x5629263cc800 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f5b6400cfd0 tx=0x7f5b64009ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b8d372f5a0ff234 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:59:31.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.739+0000 7f5b6bfff700 1 -- 172.21.15.5:0/3746700518 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6400a400 con 0x5629262334a0 2024-05-09T00:59:31.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.739+0000 7f5b6bfff700 1 -- 172.21.15.5:0/3746700518 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b64007d80 con 0x5629262334a0 2024-05-09T00:59:31.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.739+0000 7f5b6bfff700 1 -- 172.21.15.5:0/3746700518 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b64020410 con 0x5629262334a0 2024-05-09T00:59:31.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.741+0000 7f5b8f4c6b80 1 -- 172.21.15.5:0/3746700518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629262334a0 msgr2=0x5629263cc800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:31.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.741+0000 7f5b8f4c6b80 1 --2- 172.21.15.5:0/3746700518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629262334a0 0x5629263cc800 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f5b6400cfd0 tx=0x7f5b64009ab0 comp rx=0 tx=0).stop 2024-05-09T00:59:31.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.743+0000 7f5b8f4c6b80 1 -- 172.21.15.5:0/3746700518 shutdown_connections 2024-05-09T00:59:31.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.743+0000 7f5b8f4c6b80 1 --2- 172.21.15.5:0/3746700518 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5629263b0400 0x5629263c5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:31.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.743+0000 7f5b8f4c6b80 1 --2- 172.21.15.5:0/3746700518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629262334a0 0x5629263cc800 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:31.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.743+0000 7f5b8f4c6b80 1 --2- 172.21.15.5:0/3746700518 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5629262add00 0x5629263bc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:31.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.743+0000 7f5b8f4c6b80 1 -- 172.21.15.5:0/3746700518 >> 172.21.15.5:0/3746700518 conn(0x562926236f70 msgr2=0x5629263cc0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:59:31.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.744+0000 7f5b8f4c6b80 1 -- 172.21.15.5:0/3746700518 shutdown_connections 2024-05-09T00:59:31.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.745+0000 7f5b8f4c6b80 1 -- 172.21.15.5:0/3746700518 wait complete. 2024-05-09T00:59:31.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.746+0000 7f5b8f4c6b80 1 -- start start 2024-05-09T00:59:31.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.746+0000 7f5b8f4c6b80 1 Processor -- start 2024-05-09T00:59:31.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.748+0000 7f5b8f4c6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5629262334a0 0x5629263c2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:31.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.749+0000 7f5b8f4c6b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5629262add00 0x5629263c2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:31.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.749+0000 7f5b8f4c6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629263b0400 0x5629263d0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:31.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.749+0000 7f5b8f4c6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6000e9f0 con 0x5629263b0400 2024-05-09T00:59:31.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.749+0000 7f5b8f4c6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6000e320 con 0x5629262334a0 2024-05-09T00:59:31.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.749+0000 7f5b8f4c6b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6000b520 con 0x5629262add00 2024-05-09T00:59:31.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.750+0000 7f5b8f4c6b80 10 client.0 _ll_register_callbacks cb 0x5629261446c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:59:31.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.751+0000 7f5b7ad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629263b0400 0x5629263d0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:31.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.751+0000 7f5b7ad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629263b0400 0x5629263d0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34386/0 (socket says 172.21.15.5:34386) 2024-05-09T00:59:31.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.751+0000 7f5b7ad9d700 1 -- 172.21.15.5:0/105799225 learned_addr learned my addr 172.21.15.5:0/105799225 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:59:31.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.751+0000 7f5b7a59c700 1 --2- 172.21.15.5:0/105799225 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5629262334a0 0x5629263c2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:31.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.758+0000 7f5b79d9b700 1 --2- 172.21.15.5:0/105799225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5629262add00 0x5629263c2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:31.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.759+0000 7f5b7a59c700 1 -- 172.21.15.5:0/105799225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5629262add00 msgr2=0x5629263c2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:59:31.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.759+0000 7f5b7a59c700 1 --2- 172.21.15.5:0/105799225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5629262add00 0x5629263c2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:31.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.759+0000 7f5b7a59c700 1 -- 172.21.15.5:0/105799225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629263b0400 msgr2=0x5629263d0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:59:31.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.759+0000 7f5b7a59c700 1 --2- 172.21.15.5:0/105799225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629263b0400 0x5629263d0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:31.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.759+0000 7f5b7a59c700 1 -- 172.21.15.5:0/105799225 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629263d2710 con 0x5629262334a0 2024-05-09T00:59:31.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.764+0000 7f5b7a59c700 1 --2- 172.21.15.5:0/105799225 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5629262334a0 0x5629263c2150 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5b64020ee0 tx=0x7f5b6400c9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9fea400718a6535 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:59:31.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.769+0000 7f5b69ffb700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-09T00:59:31.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.769+0000 7f5b69ffb700 1 -- 172.21.15.5:0/105799225 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b64036060 con 0x5629262334a0 2024-05-09T00:59:31.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.769+0000 7f5b69ffb700 1 -- 172.21.15.5:0/105799225 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f5b6400a810 con 0x5629262334a0 2024-05-09T00:59:31.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.769+0000 7f5b8f4c6b80 1 -- 172.21.15.5:0/105799225 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b64007d80 con 0x5629262334a0 2024-05-09T00:59:31.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.770+0000 7f5b69ffb700 1 -- 172.21.15.5:0/105799225 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5b64020410 con 0x5629262334a0 2024-05-09T00:59:31.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.770+0000 7f5b8f4c6b80 10 client.26306 fetch_fsmap learned FSMap version 31 2024-05-09T00:59:31.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.771+0000 7f5b8f4c6b80 1 -- 172.21.15.5:0/105799225 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562926433e10 con 0x5629262334a0 2024-05-09T00:59:31.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.778+0000 7f5b69ffb700 1 -- 172.21.15.5:0/105799225 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b64020a60 con 0x5629262334a0 2024-05-09T00:59:31.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.778+0000 7f5b69ffb700 1 -- 172.21.15.5:0/105799225 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f5b640318a0 con 0x5629262334a0 2024-05-09T00:59:31.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.780+0000 7f5b8f4c6b80 10 client.26306 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:59:31.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.780+0000 7f5b8f4c6b80 10 client.26306 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-05-09T00:59:31.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.781+0000 7f5b8f4c6b80 20 client.26306 populate_metadata read hostname 'smithi005' 2024-05-09T00:59:31.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.781+0000 7f5b8f4c6b80 1 --2- 172.21.15.5:0/105799225 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x5629264343b0 0x5629264367d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:31.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.781+0000 7f5b8f4c6b80 4 client.26306 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:59:31.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.781+0000 7f5b8f4c6b80 1 -- 172.21.15.5:0/105799225 --> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f5b64007d80 con 0x5629264343b0 2024-05-09T00:59:31.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.786+0000 7f5b79d9b700 1 --2- 172.21.15.5:0/105799225 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x5629264343b0 0x5629264367d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:31.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.802+0000 7f5b79d9b700 1 --2- 172.21.15.5:0/105799225 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x5629264343b0 0x5629264367d0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=80df151b684b3a89 server_cookie=d1713f797e3aecf2 in_seq=0 out_seq=0 2024-05-09T00:59:31.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.803+0000 7f5b69ffb700 10 client.26306 ms_handle_connect on v2:172.21.15.157:6836/744863009 2024-05-09T00:59:31.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.804+0000 7f5b69ffb700 1 -- 172.21.15.5:0/105799225 <== mds.1 v2:172.21.15.157:6836/744863009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f5b64007d80 con 0x5629264343b0 2024-05-09T00:59:31.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.804+0000 7f5b69ffb700 10 client.26306 handle_command_reply: tid=0 2024-05-09T00:59:31.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.804+0000 7f5b8f4c6b80 1 client.26306 shutdown 2024-05-09T00:59:31.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.805+0000 7f5b8f4c6b80 1 -- 172.21.15.5:0/105799225 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5629262334a0 msgr2=0x5629263c2150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:31.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.806+0000 7f5b8f4c6b80 1 --2- 172.21.15.5:0/105799225 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5629262334a0 0x5629263c2150 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5b64020ee0 tx=0x7f5b6400c9a0 comp rx=0 tx=0).stop 2024-05-09T00:59:31.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.812+0000 7f5b8f4c6b80 1 -- 172.21.15.5:0/105799225 shutdown_connections 2024-05-09T00:59:31.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.812+0000 7f5b8f4c6b80 1 --2- 172.21.15.5:0/105799225 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x5629264343b0 0x5629264367d0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:31.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.812+0000 7f5b69ffb700 0 client.26306 ms_handle_reset on v2:172.21.15.157:6836/744863009 2024-05-09T00:59:31.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.812+0000 7f5b8f4c6b80 1 --2- 172.21.15.5:0/105799225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629263b0400 0x5629263d0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:31.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.812+0000 7f5b8f4c6b80 1 --2- 172.21.15.5:0/105799225 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5629262334a0 0x5629263c2150 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:31.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.812+0000 7f5b8f4c6b80 1 --2- 172.21.15.5:0/105799225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5629262add00 0x5629263c2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:31.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.812+0000 7f5b8f4c6b80 1 -- 172.21.15.5:0/105799225 >> 172.21.15.5:0/105799225 conn(0x562926236f70 msgr2=0x5629263b5110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:59:31.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.813+0000 7f5b8f4c6b80 1 -- 172.21.15.5:0/105799225 shutdown_connections 2024-05-09T00:59:31.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.813+0000 7f5b8f4c6b80 1 -- 172.21.15.5:0/105799225 wait complete. 2024-05-09T00:59:31.815 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-09T00:59:31.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.813+0000 7f5b8f4c6b80 20 client.26306 trim_cache size 0 max 16384 2024-05-09T00:59:31.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.816+0000 7f5b80ab0700 1 -- 172.21.15.5:0/2109973023 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f5b5c0b2a50 msgr2=0x7f5b5c0b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:31.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.816+0000 7f5b80ab0700 1 --2- 172.21.15.5:0/2109973023 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f5b5c0b2a50 0x7f5b5c0b4f60 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5b74009990 tx=0x7f5b74005fb0 comp rx=0 tx=0).stop 2024-05-09T00:59:31.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.816+0000 7f5b80ab0700 1 -- 172.21.15.5:0/2109973023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b7c109860 msgr2=0x7f5b7c10dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:31.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.816+0000 7f5b80ab0700 1 --2- 172.21.15.5:0/2109973023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b7c109860 0x7f5b7c10dde0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c00b4a0 tx=0x7f5b6c00b860 comp rx=0 tx=0).stop 2024-05-09T00:59:31.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.821+0000 7f5b80ab0700 1 -- 172.21.15.5:0/2109973023 shutdown_connections 2024-05-09T00:59:31.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.821+0000 7f5b80ab0700 1 --2- 172.21.15.5:0/2109973023 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f5b5c0b2a50 0x7f5b5c0b4f60 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:31.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.821+0000 7f5b80ab0700 1 --2- 172.21.15.5:0/2109973023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b7c109860 0x7f5b7c10dde0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:31.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.821+0000 7f5b80ab0700 1 --2- 172.21.15.5:0/2109973023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b7c1045e0 0x7f5b7c10d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:31.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.821+0000 7f5b80ab0700 1 --2- 172.21.15.5:0/2109973023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b7c108f10 0x7f5b7c10d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:31.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.822+0000 7f5b80ab0700 1 -- 172.21.15.5:0/2109973023 >> 172.21.15.5:0/2109973023 conn(0x7f5b7c0ffed0 msgr2=0x7f5b7c1022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:59:31.823 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:59:31 smithi005 ceph-mon[30365]: pgmap v766: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 901 MiB/s wr, 1.79k op/s 2024-05-09T00:59:31.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.825+0000 7f5b80ab0700 1 -- 172.21.15.5:0/2109973023 shutdown_connections 2024-05-09T00:59:31.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:31.825+0000 7f5b80ab0700 1 -- 172.21.15.5:0/2109973023 wait complete. 2024-05-09T00:59:31.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:59:31 smithi157 ceph-mon[34294]: pgmap v766: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 901 MiB/s wr, 1.79k op/s 2024-05-09T00:59:32.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:59:31 smithi131 ceph-mon[34547]: pgmap v766: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 901 MiB/s wr, 1.79k op/s 2024-05-09T00:59:33.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:59:33 smithi157 ceph-mon[34294]: pgmap v767: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 862 MiB/s wr, 1.81k op/s 2024-05-09T00:59:34.040 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:59:33 smithi005 ceph-mon[30365]: pgmap v767: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 862 MiB/s wr, 1.81k op/s 2024-05-09T00:59:34.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:59:33 smithi131 ceph-mon[34547]: pgmap v767: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 862 MiB/s wr, 1.81k op/s 2024-05-09T00:59:34.991 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T00:59:35.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:59:35 smithi157 ceph-mon[34294]: pgmap v768: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 906 MiB/s wr, 1.94k op/s 2024-05-09T00:59:35.996 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:59:36.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:59:35 smithi005 ceph-mon[30365]: pgmap v768: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 906 MiB/s wr, 1.94k op/s 2024-05-09T00:59:36.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:59:35 smithi131 ceph-mon[34547]: pgmap v768: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 906 MiB/s wr, 1.94k op/s 2024-05-09T00:59:37.782 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:59:37 smithi005 ceph-mon[30365]: pgmap v769: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 932 MiB/s wr, 2.12k op/s 2024-05-09T00:59:37.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:59:37 smithi157 ceph-mon[34294]: pgmap v769: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 932 MiB/s wr, 2.12k op/s 2024-05-09T00:59:38.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:59:37 smithi131 ceph-mon[34547]: pgmap v769: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 932 MiB/s wr, 2.12k op/s 2024-05-09T00:59:39.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:59:39 smithi157 ceph-mon[34294]: pgmap v770: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 915 MiB/s wr, 2.20k op/s 2024-05-09T00:59:40.023 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:59:39 smithi005 ceph-mon[30365]: pgmap v770: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 915 MiB/s wr, 2.20k op/s 2024-05-09T00:59:40.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:59:39 smithi131 ceph-mon[34547]: pgmap v770: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 915 MiB/s wr, 2.20k op/s 2024-05-09T00:59:41.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.572+0000 7f56e4951700 1 -- 172.21.15.5:0/3624836737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56e0115030 msgr2=0x7f56e0117450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:41.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.572+0000 7f56e4951700 1 --2- 172.21.15.5:0/3624836737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56e0115030 0x7f56e0117450 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f56d8009f90 tx=0x7f56d800b330 comp rx=0 tx=0).stop 2024-05-09T00:59:41.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.573+0000 7f56e4951700 1 -- 172.21.15.5:0/3624836737 shutdown_connections 2024-05-09T00:59:41.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.573+0000 7f56e4951700 1 --2- 172.21.15.5:0/3624836737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56e0115030 0x7f56e0117450 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:41.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.573+0000 7f56e4951700 1 --2- 172.21.15.5:0/3624836737 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56e0102310 0x7f56e0102720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:41.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.573+0000 7f56e4951700 1 --2- 172.21.15.5:0/3624836737 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f56e0102cf0 0x7f56e0114af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:41.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.573+0000 7f56e4951700 1 -- 172.21.15.5:0/3624836737 >> 172.21.15.5:0/3624836737 conn(0x7f56e00fe0d0 msgr2=0x7f56e01004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:59:41.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.573+0000 7f56e4951700 1 -- 172.21.15.5:0/3624836737 shutdown_connections 2024-05-09T00:59:41.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.573+0000 7f56e4951700 1 -- 172.21.15.5:0/3624836737 wait complete. 2024-05-09T00:59:41.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.574+0000 7f56e4951700 1 Processor -- start 2024-05-09T00:59:41.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.574+0000 7f56e4951700 1 -- start start 2024-05-09T00:59:41.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.575+0000 7f56e4951700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56e0102310 0x7f56e0112480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:41.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.575+0000 7f56e4951700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f56e0102cf0 0x7f56e010b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:41.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.575+0000 7f56e4951700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56e0115030 0x7f56e010bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:41.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.576+0000 7f56e4951700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f56e011ab20 con 0x7f56e0115030 2024-05-09T00:59:41.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.576+0000 7f56e4951700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f56e011a9a0 con 0x7f56e0102cf0 2024-05-09T00:59:41.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.576+0000 7f56e4951700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f56e011aca0 con 0x7f56e0102310 2024-05-09T00:59:41.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.577+0000 7f56ded9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56e0115030 0x7f56e010bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:41.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.577+0000 7f56ddd9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f56e0102cf0 0x7f56e010b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:41.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.577+0000 7f56ddd9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f56e0102cf0 0x7f56e010b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:46878/0 (socket says 172.21.15.5:46878) 2024-05-09T00:59:41.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.577+0000 7f56ddd9b700 1 -- 172.21.15.5:0/3313259403 learned_addr learned my addr 172.21.15.5:0/3313259403 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:59:41.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.578+0000 7f56ded9d700 1 -- 172.21.15.5:0/3313259403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56e0102310 msgr2=0x7f56e0112480 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T00:59:41.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.578+0000 7f56ded9d700 1 --2- 172.21.15.5:0/3313259403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56e0102310 0x7f56e0112480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:41.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.578+0000 7f56ded9d700 1 -- 172.21.15.5:0/3313259403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f56e0102cf0 msgr2=0x7f56e010b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:41.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.578+0000 7f56ded9d700 1 --2- 172.21.15.5:0/3313259403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f56e0102cf0 0x7f56e010b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:41.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.578+0000 7f56ded9d700 1 -- 172.21.15.5:0/3313259403 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56e010ffb0 con 0x7f56e0115030 2024-05-09T00:59:41.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.579+0000 7f56ded9d700 1 --2- 172.21.15.5:0/3313259403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56e0115030 0x7f56e010bbb0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f56d8015000 tx=0x7f56d800bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:59:41.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.580+0000 7f56cf7fe700 1 -- 172.21.15.5:0/3313259403 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d8007ac0 con 0x7f56e0115030 2024-05-09T00:59:41.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.580+0000 7f56cf7fe700 1 -- 172.21.15.5:0/3313259403 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56d8020070 con 0x7f56e0115030 2024-05-09T00:59:41.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.580+0000 7f56cf7fe700 1 -- 172.21.15.5:0/3313259403 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d801b690 con 0x7f56e0115030 2024-05-09T00:59:41.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.580+0000 7f56e4951700 1 -- 172.21.15.5:0/3313259403 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56e0110240 con 0x7f56e0115030 2024-05-09T00:59:41.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.580+0000 7f56e4951700 1 -- 172.21.15.5:0/3313259403 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56e0110720 con 0x7f56e0115030 2024-05-09T00:59:41.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.590+0000 7f56f3367b80 1 Processor -- start 2024-05-09T00:59:41.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.591+0000 7f56cf7fe700 1 -- 172.21.15.5:0/3313259403 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f56d8005110 con 0x7f56e0115030 2024-05-09T00:59:41.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.592+0000 7f56f3367b80 1 -- start start 2024-05-09T00:59:41.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.592+0000 7f56f3367b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ff5656160 0x556ff56ef4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:41.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.593+0000 7f56f3367b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ff556c4a0 0x556ff56efa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:41.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.593+0000 7f56f3367b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556ff56e96a0 0x556ff56e9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:41.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.593+0000 7f56f3367b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556ff560bff0 con 0x556ff5656160 2024-05-09T00:59:41.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.593+0000 7f56f3367b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x556ff55aa7f0 con 0x556ff56e96a0 2024-05-09T00:59:41.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.593+0000 7f56f3367b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x556ff5607fb0 con 0x556ff556c4a0 2024-05-09T00:59:41.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.594+0000 7f56de59c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ff556c4a0 0x556ff56efa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:41.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.594+0000 7f56de59c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ff556c4a0 0x556ff56efa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:59108/0 (socket says 172.21.15.5:59108) 2024-05-09T00:59:41.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.594+0000 7f56de59c700 1 -- 172.21.15.5:0/462340265 learned_addr learned my addr 172.21.15.5:0/462340265 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:59:41.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.594+0000 7f56ddd9b700 1 --2- 172.21.15.5:0/462340265 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556ff56e96a0 0x556ff56e9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:41.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.594+0000 7f56ddd9b700 1 --2- 172.21.15.5:0/462340265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ff5656160 0x556ff56ef4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:41.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.594+0000 7f56cf7fe700 1 --2- 172.21.15.5:0/3313259403 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f56c80b2a10 0x7f56c80b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:41.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.595+0000 7f56ddd9b700 1 -- 172.21.15.5:0/462340265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ff556c4a0 msgr2=0x556ff56efa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:59:41.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.595+0000 7f56ddd9b700 1 --2- 172.21.15.5:0/462340265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ff556c4a0 0x556ff56efa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:41.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.595+0000 7f56ddd9b700 1 -- 172.21.15.5:0/462340265 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556ff56e96a0 msgr2=0x556ff56e9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:59:41.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.595+0000 7f56ddd9b700 1 --2- 172.21.15.5:0/462340265 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556ff56e96a0 0x556ff56e9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:41.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.595+0000 7f56cf7fe700 1 -- 172.21.15.5:0/3313259403 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f56d800f020 con 0x7f56e0115030 2024-05-09T00:59:41.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.595+0000 7f56ddd9b700 1 -- 172.21.15.5:0/462340265 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ff55a2370 con 0x556ff5656160 2024-05-09T00:59:41.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.596+0000 7f56ddd9b700 1 --2- 172.21.15.5:0/462340265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ff5656160 0x556ff56ef4f0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f56d4005d90 tx=0x7f56d400b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5347eab5e00b45 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:59:41.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.596+0000 7f56ded9d700 1 --2- 172.21.15.5:0/3313259403 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f56c80b2a10 0x7f56c80b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:41.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.597+0000 7f56ded9d700 1 --2- 172.21.15.5:0/3313259403 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f56c80b2a10 0x7f56c80b4f20 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f56d8013ff0 tx=0x7f56d80058f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:59:41.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.597+0000 7f56ccff9700 1 -- 172.21.15.5:0/462340265 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d401e3f0 con 0x556ff5656160 2024-05-09T00:59:41.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.597+0000 7f56ccff9700 1 -- 172.21.15.5:0/462340265 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56d400f8a0 con 0x556ff5656160 2024-05-09T00:59:41.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.597+0000 7f56ccff9700 1 -- 172.21.15.5:0/462340265 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d4010750 con 0x556ff5656160 2024-05-09T00:59:41.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.601+0000 7f56f3367b80 1 -- 172.21.15.5:0/462340265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ff5656160 msgr2=0x556ff56ef4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:41.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.601+0000 7f56f3367b80 1 --2- 172.21.15.5:0/462340265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ff5656160 0x556ff56ef4f0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f56d4005d90 tx=0x7f56d400b460 comp rx=0 tx=0).stop 2024-05-09T00:59:41.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.602+0000 7f56f3367b80 1 -- 172.21.15.5:0/462340265 shutdown_connections 2024-05-09T00:59:41.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.602+0000 7f56f3367b80 1 --2- 172.21.15.5:0/462340265 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556ff56e96a0 0x556ff56e9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:41.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.602+0000 7f56f3367b80 1 --2- 172.21.15.5:0/462340265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ff5656160 0x556ff56ef4f0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:41.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.602+0000 7f56f3367b80 1 --2- 172.21.15.5:0/462340265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ff556c4a0 0x556ff56efa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:41.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.602+0000 7f56f3367b80 1 -- 172.21.15.5:0/462340265 >> 172.21.15.5:0/462340265 conn(0x556ff55cc6a0 msgr2=0x556ff56e4400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:59:41.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.602+0000 7f56f3367b80 1 -- 172.21.15.5:0/462340265 shutdown_connections 2024-05-09T00:59:41.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.602+0000 7f56f3367b80 1 -- 172.21.15.5:0/462340265 wait complete. 2024-05-09T00:59:41.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.603+0000 7f56f3367b80 1 -- start start 2024-05-09T00:59:41.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.603+0000 7f56f3367b80 1 Processor -- start 2024-05-09T00:59:41.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.604+0000 7f56f3367b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ff556c4a0 0x556ff56f3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:41.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.604+0000 7f56f3367b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ff5656160 0x556ff56f3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:41.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.606+0000 7f56f3367b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556ff56e96a0 0x556ff56f3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:41.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.606+0000 7f56f3367b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556ff5487ba0 con 0x556ff556c4a0 2024-05-09T00:59:41.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.606+0000 7f56f3367b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x556ff55c6150 con 0x556ff56e96a0 2024-05-09T00:59:41.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.606+0000 7f56f3367b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x556ff56f61d0 con 0x556ff5656160 2024-05-09T00:59:41.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.606+0000 7f56f3367b80 10 client.0 _ll_register_callbacks cb 0x556ff547d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:59:41.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.606+0000 7f56ddd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ff556c4a0 0x556ff56f3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:41.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.606+0000 7f56ddd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ff556c4a0 0x556ff56f3230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60584/0 (socket says 172.21.15.5:60584) 2024-05-09T00:59:41.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.606+0000 7f56ddd9b700 1 -- 172.21.15.5:0/3457072422 learned_addr learned my addr 172.21.15.5:0/3457072422 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:59:41.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.607+0000 7f56de59c700 1 --2- 172.21.15.5:0/3457072422 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ff5656160 0x556ff56f3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:41.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.607+0000 7f56ddd9b700 1 --2- 172.21.15.5:0/3457072422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556ff56e96a0 0x556ff56f3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:41.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.608+0000 7f56ddd9b700 1 -- 172.21.15.5:0/3457072422 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ff5656160 msgr2=0x556ff56f3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:59:41.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.608+0000 7f56ddd9b700 1 --2- 172.21.15.5:0/3457072422 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ff5656160 0x556ff56f3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:41.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.608+0000 7f56ddd9b700 1 -- 172.21.15.5:0/3457072422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556ff56e96a0 msgr2=0x556ff56f3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:59:41.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.608+0000 7f56ddd9b700 1 --2- 172.21.15.5:0/3457072422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556ff56e96a0 0x556ff56f3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:41.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.608+0000 7f56ddd9b700 1 -- 172.21.15.5:0/3457072422 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ff5487490 con 0x556ff556c4a0 2024-05-09T00:59:41.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.609+0000 7f56ddd9b700 1 --2- 172.21.15.5:0/3457072422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ff556c4a0 0x556ff56f3230 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f56d400f800 tx=0x7f56d400ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=373e0e9f396c9492 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:59:41.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.609+0000 7f56c5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:59:41.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.610+0000 7f56c5ffb700 1 -- 172.21.15.5:0/3457072422 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d4004460 con 0x556ff556c4a0 2024-05-09T00:59:41.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.610+0000 7f56f3367b80 1 -- 172.21.15.5:0/3457072422 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56d400f8a0 con 0x556ff556c4a0 2024-05-09T00:59:41.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.610+0000 7f56c5ffb700 1 -- 172.21.15.5:0/3457072422 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f56d4005120 con 0x556ff556c4a0 2024-05-09T00:59:41.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.611+0000 7f56c5ffb700 1 -- 172.21.15.5:0/3457072422 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f56d4034050 con 0x556ff556c4a0 2024-05-09T00:59:41.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.611+0000 7f56f3367b80 10 client.16902 fetch_fsmap learned FSMap version 31 2024-05-09T00:59:41.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.611+0000 7f56f3367b80 1 -- 172.21.15.5:0/3457072422 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556ff5700a00 con 0x556ff556c4a0 2024-05-09T00:59:41.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.612+0000 7f56c5ffb700 1 -- 172.21.15.5:0/3457072422 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56d400f8a0 con 0x556ff556c4a0 2024-05-09T00:59:41.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.612+0000 7f56c5ffb700 1 -- 172.21.15.5:0/3457072422 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f56d4004c10 con 0x556ff556c4a0 2024-05-09T00:59:41.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.613+0000 7f56f3367b80 10 client.16902 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:59:41.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.613+0000 7f56f3367b80 10 client.16902 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:59:41.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.613+0000 7f56f3367b80 20 client.16902 populate_metadata read hostname 'smithi005' 2024-05-09T00:59:41.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.613+0000 7f56f3367b80 1 --2- 172.21.15.5:0/3457072422 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x556ff5702f80 0x556ff57053a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:41.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.613+0000 7f56f3367b80 4 client.16902 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:59:41.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.613+0000 7f56f3367b80 1 -- 172.21.15.5:0/3457072422 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ff56199d0 con 0x556ff5702f80 2024-05-09T00:59:41.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.614+0000 7f56de59c700 1 --2- 172.21.15.5:0/3457072422 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x556ff5702f80 0x556ff57053a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:41.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.620+0000 7f56de59c700 1 --2- 172.21.15.5:0/3457072422 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x556ff5702f80 0x556ff57053a0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c11e00182cbba584 server_cookie=94ccf21ae32430fb in_seq=0 out_seq=0 2024-05-09T00:59:41.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.620+0000 7f56c5ffb700 10 client.16902 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:59:41.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.624+0000 7f56c5ffb700 1 -- 172.21.15.5:0/3457072422 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556ff56199d0 con 0x556ff5702f80 2024-05-09T00:59:41.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.624+0000 7f56c5ffb700 10 client.16902 handle_command_reply: tid=0 2024-05-09T00:59:41.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.624+0000 7f56f3367b80 1 client.16902 shutdown 2024-05-09T00:59:41.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.624+0000 7f56f3367b80 1 -- 172.21.15.5:0/3457072422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ff556c4a0 msgr2=0x556ff56f3230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:41.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.624+0000 7f56f3367b80 1 --2- 172.21.15.5:0/3457072422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ff556c4a0 0x556ff56f3230 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f56d400f800 tx=0x7f56d400ba40 comp rx=0 tx=0).stop 2024-05-09T00:59:41.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.626+0000 7f56f3367b80 1 -- 172.21.15.5:0/3457072422 shutdown_connections 2024-05-09T00:59:41.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.626+0000 7f56f3367b80 1 --2- 172.21.15.5:0/3457072422 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x556ff5702f80 0x556ff57053a0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:41.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.626+0000 7f56f3367b80 1 --2- 172.21.15.5:0/3457072422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556ff56e96a0 0x556ff56f3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:41.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.626+0000 7f56f3367b80 1 --2- 172.21.15.5:0/3457072422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ff556c4a0 0x556ff56f3230 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:41.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.626+0000 7f56f3367b80 1 --2- 172.21.15.5:0/3457072422 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ff5656160 0x556ff56f3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:41.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.626+0000 7f56f3367b80 1 -- 172.21.15.5:0/3457072422 >> 172.21.15.5:0/3457072422 conn(0x556ff55cc6a0 msgr2=0x556ff56e41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:59:41.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.626+0000 7f56c5ffb700 0 client.16902 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:59:41.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.628+0000 7f56f3367b80 1 -- 172.21.15.5:0/3457072422 shutdown_connections 2024-05-09T00:59:41.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.629+0000 7f56f3367b80 1 -- 172.21.15.5:0/3457072422 wait complete. 2024-05-09T00:59:41.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.629+0000 7f56f3367b80 20 client.16902 trim_cache size 0 max 16384 2024-05-09T00:59:41.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.652+0000 7f56f3367b80 1 Processor -- start 2024-05-09T00:59:41.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.652+0000 7f56f3367b80 1 -- start start 2024-05-09T00:59:41.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.653+0000 7f56f3367b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556ff556c4a0 0x556ff56ee680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:41.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.653+0000 7f56f3367b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ff5656160 0x556ff56f2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:41.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.653+0000 7f56f3367b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ff56e96a0 0x556ff56f0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:41.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.653+0000 7f56f3367b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f56e01bda20 con 0x556ff5656160 2024-05-09T00:59:41.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.653+0000 7f56f3367b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f56e01bf560 con 0x556ff556c4a0 2024-05-09T00:59:41.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.653+0000 7f56f3367b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f56e01be0f0 con 0x556ff56e96a0 2024-05-09T00:59:41.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.655+0000 7f56ddd9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ff56e96a0 0x556ff56f0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:41.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.655+0000 7f56ddd9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ff56e96a0 0x556ff56f0eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:59126/0 (socket says 172.21.15.5:59126) 2024-05-09T00:59:41.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.655+0000 7f56ddd9b700 1 -- 172.21.15.5:0/1499801220 learned_addr learned my addr 172.21.15.5:0/1499801220 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:59:41.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.655+0000 7f56ddd9b700 1 --2- 172.21.15.5:0/1499801220 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556ff556c4a0 0x556ff56ee680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:41.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.657+0000 7f56de59c700 1 --2- 172.21.15.5:0/1499801220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ff5656160 0x556ff56f2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:41.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.658+0000 7f56de59c700 1 -- 172.21.15.5:0/1499801220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ff56e96a0 msgr2=0x556ff56f0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:59:41.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.658+0000 7f56de59c700 1 --2- 172.21.15.5:0/1499801220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ff56e96a0 0x556ff56f0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:41.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.658+0000 7f56de59c700 1 -- 172.21.15.5:0/1499801220 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556ff556c4a0 msgr2=0x556ff56ee680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:59:41.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.658+0000 7f56de59c700 1 --2- 172.21.15.5:0/1499801220 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556ff556c4a0 0x556ff56ee680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:41.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.658+0000 7f56de59c700 1 -- 172.21.15.5:0/1499801220 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ff56ddf60 con 0x556ff5656160 2024-05-09T00:59:41.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.659+0000 7f56de59c700 1 --2- 172.21.15.5:0/1499801220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ff5656160 0x556ff56f2550 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x556ff57058e0 tx=0x7f56d0010710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=827fb027100f31ad server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:59:41.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.670+0000 7f56c7fff700 1 -- 172.21.15.5:0/1499801220 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d0023070 con 0x556ff5656160 2024-05-09T00:59:41.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.670+0000 7f56c7fff700 1 -- 172.21.15.5:0/1499801220 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56d0010dc0 con 0x556ff5656160 2024-05-09T00:59:41.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.671+0000 7f56f3367b80 1 -- 172.21.15.5:0/1499801220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ff5656160 msgr2=0x556ff56f2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:41.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.671+0000 7f56f3367b80 1 --2- 172.21.15.5:0/1499801220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ff5656160 0x556ff56f2550 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x556ff57058e0 tx=0x7f56d0010710 comp rx=0 tx=0).stop 2024-05-09T00:59:41.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.672+0000 7f56f3367b80 1 -- 172.21.15.5:0/1499801220 shutdown_connections 2024-05-09T00:59:41.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.672+0000 7f56f3367b80 1 --2- 172.21.15.5:0/1499801220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ff56e96a0 0x556ff56f0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:41.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.672+0000 7f56f3367b80 1 --2- 172.21.15.5:0/1499801220 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556ff556c4a0 0x556ff56ee680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:41.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.672+0000 7f56f3367b80 1 --2- 172.21.15.5:0/1499801220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ff5656160 0x556ff56f2550 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:41.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.672+0000 7f56f3367b80 1 -- 172.21.15.5:0/1499801220 >> 172.21.15.5:0/1499801220 conn(0x556ff55cc6a0 msgr2=0x556ff56f31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:59:41.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.682+0000 7f56f3367b80 1 -- 172.21.15.5:0/1499801220 shutdown_connections 2024-05-09T00:59:41.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.684+0000 7f56f3367b80 1 -- 172.21.15.5:0/1499801220 wait complete. 2024-05-09T00:59:41.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.685+0000 7f56f3367b80 1 -- start start 2024-05-09T00:59:41.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.685+0000 7f56f3367b80 1 Processor -- start 2024-05-09T00:59:41.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.686+0000 7f56f3367b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ff556c4a0 0x556ff56fb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:41.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.686+0000 7f56f3367b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556ff5656160 0x556ff56fb820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:41.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.686+0000 7f56f3367b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ff56e96a0 0x556ff57686d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:41.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.686+0000 7f56f3367b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f56e01bee90 con 0x556ff556c4a0 2024-05-09T00:59:41.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.686+0000 7f56f3367b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f56e01be7c0 con 0x556ff5656160 2024-05-09T00:59:41.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.686+0000 7f56f3367b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f56e01bb9c0 con 0x556ff56e96a0 2024-05-09T00:59:41.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.686+0000 7f56f3367b80 10 client.0 _ll_register_callbacks cb 0x556ff56eeea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:59:41.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.686+0000 7f56ddd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ff556c4a0 0x556ff56fb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:41.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.686+0000 7f56ddd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ff556c4a0 0x556ff56fb2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60602/0 (socket says 172.21.15.5:60602) 2024-05-09T00:59:41.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.686+0000 7f56ddd9b700 1 -- 172.21.15.5:0/1144548611 learned_addr learned my addr 172.21.15.5:0/1144548611 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:59:41.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.690+0000 7f56ddd9b700 1 --2- 172.21.15.5:0/1144548611 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ff56e96a0 0x556ff57686d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:41.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.690+0000 7f56ddd9b700 1 -- 172.21.15.5:0/1144548611 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ff56e96a0 msgr2=0x556ff57686d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:59:41.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.690+0000 7f56ddd9b700 1 --2- 172.21.15.5:0/1144548611 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ff56e96a0 0x556ff57686d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:41.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.690+0000 7f56ddd9b700 1 -- 172.21.15.5:0/1144548611 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556ff5656160 msgr2=0x556ff56fb820 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:59:41.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.690+0000 7f56ddd9b700 1 --2- 172.21.15.5:0/1144548611 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556ff5656160 0x556ff56fb820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:41.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.690+0000 7f56ddd9b700 1 -- 172.21.15.5:0/1144548611 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ff5768c10 con 0x556ff556c4a0 2024-05-09T00:59:41.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.704+0000 7f56ddd9b700 1 --2- 172.21.15.5:0/1144548611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ff556c4a0 0x556ff56fb2e0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f56d4032040 tx=0x7f56d400ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeb22e77ad4da45 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:59:41.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.704+0000 7f56c5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:59:41.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.712+0000 7f56c5ffb700 1 -- 172.21.15.5:0/1144548611 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d4003e90 con 0x556ff556c4a0 2024-05-09T00:59:41.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.712+0000 7f56f3367b80 1 -- 172.21.15.5:0/1144548611 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56d0010dc0 con 0x556ff556c4a0 2024-05-09T00:59:41.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.712+0000 7f56c5ffb700 1 -- 172.21.15.5:0/1144548611 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f56d40050f0 con 0x556ff556c4a0 2024-05-09T00:59:41.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.713+0000 7f56c5ffb700 1 -- 172.21.15.5:0/1144548611 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f56d40052f0 con 0x556ff556c4a0 2024-05-09T00:59:41.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.714+0000 7f56f3367b80 10 client.16908 fetch_fsmap learned FSMap version 31 2024-05-09T00:59:41.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.715+0000 7f56f3367b80 1 -- 172.21.15.5:0/1144548611 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556ff570a370 con 0x556ff556c4a0 2024-05-09T00:59:41.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.716+0000 7f56c5ffb700 1 -- 172.21.15.5:0/1144548611 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56d0010dc0 con 0x556ff556c4a0 2024-05-09T00:59:41.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.716+0000 7f56c5ffb700 1 -- 172.21.15.5:0/1144548611 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f56d4047ac0 con 0x556ff556c4a0 2024-05-09T00:59:41.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.716+0000 7f56f3367b80 10 client.16908 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:59:41.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.716+0000 7f56f3367b80 10 client.16908 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:59:41.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.716+0000 7f56f3367b80 20 client.16908 populate_metadata read hostname 'smithi005' 2024-05-09T00:59:41.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.716+0000 7f56f3367b80 1 --2- 172.21.15.5:0/1144548611 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x556ff577e5a0 0x556ff57809c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:41.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.716+0000 7f56f3367b80 4 client.16908 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:59:41.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.716+0000 7f56f3367b80 1 -- 172.21.15.5:0/1144548611 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556ff5613a70 con 0x556ff577e5a0 2024-05-09T00:59:41.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.743+0000 7f56de59c700 1 --2- 172.21.15.5:0/1144548611 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x556ff577e5a0 0x556ff57809c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:41.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.755+0000 7f56de59c700 1 --2- 172.21.15.5:0/1144548611 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x556ff577e5a0 0x556ff57809c0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=299568ec79e207d1 server_cookie=88a2755fa10823cd in_seq=0 out_seq=0 2024-05-09T00:59:41.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.755+0000 7f56c5ffb700 10 client.16908 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:59:41.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.762+0000 7f56c5ffb700 1 -- 172.21.15.5:0/1144548611 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556ff5613a70 con 0x556ff577e5a0 2024-05-09T00:59:41.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.763+0000 7f56c5ffb700 10 client.16908 handle_command_reply: tid=0 2024-05-09T00:59:41.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.763+0000 7f56f3367b80 1 client.16908 shutdown 2024-05-09T00:59:41.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.763+0000 7f56f3367b80 1 -- 172.21.15.5:0/1144548611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ff556c4a0 msgr2=0x556ff56fb2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:41.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.763+0000 7f56f3367b80 1 --2- 172.21.15.5:0/1144548611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ff556c4a0 0x556ff56fb2e0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f56d4032040 tx=0x7f56d400ba40 comp rx=0 tx=0).stop 2024-05-09T00:59:41.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.763+0000 7f56f3367b80 1 -- 172.21.15.5:0/1144548611 shutdown_connections 2024-05-09T00:59:41.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.763+0000 7f56f3367b80 1 --2- 172.21.15.5:0/1144548611 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x556ff577e5a0 0x556ff57809c0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:41.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.763+0000 7f56f3367b80 1 --2- 172.21.15.5:0/1144548611 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ff56e96a0 0x556ff57686d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:41.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.763+0000 7f56f3367b80 1 --2- 172.21.15.5:0/1144548611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ff556c4a0 0x556ff56fb2e0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:41.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.763+0000 7f56f3367b80 1 --2- 172.21.15.5:0/1144548611 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556ff5656160 0x556ff56fb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:41.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.763+0000 7f56f3367b80 1 -- 172.21.15.5:0/1144548611 >> 172.21.15.5:0/1144548611 conn(0x556ff55cc6a0 msgr2=0x556ff56f22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:59:41.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.764+0000 7f56c5ffb700 0 client.16908 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:59:41.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.764+0000 7f56f3367b80 1 -- 172.21.15.5:0/1144548611 shutdown_connections 2024-05-09T00:59:41.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.764+0000 7f56f3367b80 1 -- 172.21.15.5:0/1144548611 wait complete. 2024-05-09T00:59:41.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.765+0000 7f56f3367b80 20 client.16908 trim_cache size 0 max 16384 2024-05-09T00:59:41.767 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-09T00:59:41.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.767+0000 7f56e4951700 1 -- 172.21.15.5:0/3313259403 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f56c80b2a10 msgr2=0x7f56c80b4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:41.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.767+0000 7f56e4951700 1 --2- 172.21.15.5:0/3313259403 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f56c80b2a10 0x7f56c80b4f20 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f56d8013ff0 tx=0x7f56d80058f0 comp rx=0 tx=0).stop 2024-05-09T00:59:41.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.767+0000 7f56e4951700 1 -- 172.21.15.5:0/3313259403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56e0115030 msgr2=0x7f56e010bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:41.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.767+0000 7f56e4951700 1 --2- 172.21.15.5:0/3313259403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56e0115030 0x7f56e010bbb0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f56d8015000 tx=0x7f56d800bf90 comp rx=0 tx=0).stop 2024-05-09T00:59:41.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.769+0000 7f56e4951700 1 -- 172.21.15.5:0/3313259403 shutdown_connections 2024-05-09T00:59:41.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.769+0000 7f56e4951700 1 --2- 172.21.15.5:0/3313259403 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f56c80b2a10 0x7f56c80b4f20 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:41.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.769+0000 7f56e4951700 1 --2- 172.21.15.5:0/3313259403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56e0115030 0x7f56e010bbb0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:41.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.769+0000 7f56e4951700 1 --2- 172.21.15.5:0/3313259403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56e0102310 0x7f56e0112480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:41.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.769+0000 7f56e4951700 1 --2- 172.21.15.5:0/3313259403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f56e0102cf0 0x7f56e010b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:41.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.769+0000 7f56e4951700 1 -- 172.21.15.5:0/3313259403 >> 172.21.15.5:0/3313259403 conn(0x7f56e00fe0d0 msgr2=0x7f56e01085e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:59:41.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.781+0000 7f56e4951700 1 -- 172.21.15.5:0/3313259403 shutdown_connections 2024-05-09T00:59:41.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:41.781+0000 7f56e4951700 1 -- 172.21.15.5:0/3313259403 wait complete. 2024-05-09T00:59:41.834 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:59:41 smithi005 ceph-mon[30365]: pgmap v771: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 909 MiB/s wr, 2.11k op/s 2024-05-09T00:59:41.834 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:59:41 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:59:41.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:59:41 smithi157 ceph-mon[34294]: pgmap v771: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 909 MiB/s wr, 2.11k op/s 2024-05-09T00:59:41.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:59:41 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:59:42.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.032+0000 7f6dba306640 1 -- 172.21.15.5:0/2105616724 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6db41101a0 msgr2=0x7f6db41105b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:42.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.032+0000 7f6dba306640 1 --2- 172.21.15.5:0/2105616724 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6db41101a0 0x7f6db41105b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6dac009ca0 tx=0x7f6dac02a8d0 comp rx=0 tx=0).stop 2024-05-09T00:59:42.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.041+0000 7f6dba306640 1 -- 172.21.15.5:0/2105616724 shutdown_connections 2024-05-09T00:59:42.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.041+0000 7f6dba306640 1 --2- 172.21.15.5:0/2105616724 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6db41101a0 0x7f6db41105b0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:42.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.041+0000 7f6dba306640 1 --2- 172.21.15.5:0/2105616724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6db410b480 0x7f6db410b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:42.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.041+0000 7f6dba306640 1 --2- 172.21.15.5:0/2105616724 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6db410a550 0x7f6db410af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:42.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.041+0000 7f6dba306640 1 -- 172.21.15.5:0/2105616724 >> 172.21.15.5:0/2105616724 conn(0x7f6db4070670 msgr2=0x7f6db4072b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:59:42.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.054+0000 7f6dba306640 1 -- 172.21.15.5:0/2105616724 shutdown_connections 2024-05-09T00:59:42.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.057+0000 7f6dba306640 1 -- 172.21.15.5:0/2105616724 wait complete. 2024-05-09T00:59:42.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.058+0000 7f6dba306640 1 Processor -- start 2024-05-09T00:59:42.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.058+0000 7f6dba306640 1 -- start start 2024-05-09T00:59:42.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.058+0000 7f6dba306640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6db410a550 0x7f6db41170a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:42.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.059+0000 7f6dba306640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6db410b480 0x7f6db4117620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:42.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.059+0000 7f6dba306640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6db41101a0 0x7f6db41185c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:42.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.059+0000 7f6dba306640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6db411e650 con 0x7f6db410a550 2024-05-09T00:59:42.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.059+0000 7f6dba306640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6db411e4d0 con 0x7f6db410b480 2024-05-09T00:59:42.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.059+0000 7f6dba306640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6db411e7d0 con 0x7f6db41101a0 2024-05-09T00:59:42.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.059+0000 7f6db88a3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6db41101a0 0x7f6db41185c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:42.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.059+0000 7f6db88a3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6db41101a0 0x7f6db41185c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:59160/0 (socket says 172.21.15.5:59160) 2024-05-09T00:59:42.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.059+0000 7f6db88a3640 1 -- 172.21.15.5:0/3534886321 learned_addr learned my addr 172.21.15.5:0/3534886321 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:59:42.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.060+0000 7f6db37fe640 1 --2- 172.21.15.5:0/3534886321 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6db410b480 0x7f6db4117620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:42.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.061+0000 7f6db37fe640 1 -- 172.21.15.5:0/3534886321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6db41101a0 msgr2=0x7f6db41185c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:42.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.061+0000 7f6db37fe640 1 --2- 172.21.15.5:0/3534886321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6db41101a0 0x7f6db41185c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:42.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.061+0000 7f6db37fe640 1 -- 172.21.15.5:0/3534886321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6db410a550 msgr2=0x7f6db41170a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T00:59:42.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.061+0000 7f6db37fe640 1 --2- 172.21.15.5:0/3534886321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6db410a550 0x7f6db41170a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:42.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.061+0000 7f6db37fe640 1 -- 172.21.15.5:0/3534886321 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6db4118c30 con 0x7f6db410b480 2024-05-09T00:59:42.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.062+0000 7f6db37fe640 1 --2- 172.21.15.5:0/3534886321 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6db410b480 0x7f6db4117620 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6da800c970 tx=0x7f6da800ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:59:42.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.066+0000 7f6db17fa640 1 -- 172.21.15.5:0/3534886321 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da8007bb0 con 0x7f6db410b480 2024-05-09T00:59:42.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.067+0000 7f6dba306640 1 -- 172.21.15.5:0/3534886321 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6db4118f20 con 0x7f6db410b480 2024-05-09T00:59:42.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.067+0000 7f6dba306640 1 -- 172.21.15.5:0/3534886321 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6db4119470 con 0x7f6db410b480 2024-05-09T00:59:42.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.067+0000 7f6db17fa640 1 -- 172.21.15.5:0/3534886321 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6da8007d50 con 0x7f6db410b480 2024-05-09T00:59:42.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.067+0000 7f6db17fa640 1 -- 172.21.15.5:0/3534886321 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da8004d80 con 0x7f6db410b480 2024-05-09T00:59:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.077+0000 7f6db17fa640 1 -- 172.21.15.5:0/3534886321 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f6da800f490 con 0x7f6db410b480 2024-05-09T00:59:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.080+0000 7f6db17fa640 1 --2- 172.21.15.5:0/3534886321 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f6d9c0b29b0 0x7f6d9c0b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.081+0000 7f6db17fa640 1 -- 172.21.15.5:0/3534886321 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f6da80e2c90 con 0x7f6db410b480 2024-05-09T00:59:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.082+0000 7f6dc11be740 1 Processor -- start 2024-05-09T00:59:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.082+0000 7f6dc11be740 1 -- start start 2024-05-09T00:59:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.082+0000 7f6dc11be740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e039b3d1b0 0x55e039ee1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.082+0000 7f6dc11be740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e039b142f0 0x55e039ee1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.082+0000 7f6dc11be740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e039b1c4d0 0x55e039ee8980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.083+0000 7f6dc11be740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e039e457c0 con 0x55e039b142f0 2024-05-09T00:59:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.083+0000 7f6dc11be740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55e039de10e0 con 0x55e039b1c4d0 2024-05-09T00:59:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.083+0000 7f6dc11be740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55e039e41780 con 0x55e039b3d1b0 2024-05-09T00:59:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.083+0000 7f6db3fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e039b3d1b0 0x55e039ee1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.083+0000 7f6db3fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e039b3d1b0 0x55e039ee1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:59170/0 (socket says 172.21.15.5:59170) 2024-05-09T00:59:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.083+0000 7f6db3fff640 1 -- 172.21.15.5:0/3468979018 learned_addr learned my addr 172.21.15.5:0/3468979018 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:59:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.083+0000 7f6db88a3640 1 --2- 172.21.15.5:0/3468979018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e039b1c4d0 0x55e039ee8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.084+0000 7f6db88a3640 1 -- 172.21.15.5:0/3468979018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e039b3d1b0 msgr2=0x55e039ee1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:59:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.084+0000 7f6db88a3640 1 --2- 172.21.15.5:0/3468979018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e039b3d1b0 0x55e039ee1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.084+0000 7f6db88a3640 1 -- 172.21.15.5:0/3468979018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e039b142f0 msgr2=0x55e039ee1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:59:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.084+0000 7f6db88a3640 1 --2- 172.21.15.5:0/3468979018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e039b142f0 0x55e039ee1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.084+0000 7f6db88a3640 1 -- 172.21.15.5:0/3468979018 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e039e0a7c0 con 0x55e039b1c4d0 2024-05-09T00:59:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.084+0000 7f6db3fff640 1 --2- 172.21.15.5:0/3468979018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e039b3d1b0 0x55e039ee1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:59:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.084+0000 7f6db3fff640 1 --2- 172.21.15.5:0/3534886321 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f6d9c0b29b0 0x7f6d9c0b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.085+0000 7f6db88a3640 1 --2- 172.21.15.5:0/3468979018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e039b1c4d0 0x55e039ee8980 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6dac0319e0 tx=0x7f6dac031b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=552c4ce6ca29cdbd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:59:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.088+0000 7f6d9a7fc640 1 -- 172.21.15.5:0/3468979018 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dac031460 con 0x55e039b1c4d0 2024-05-09T00:59:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.088+0000 7f6d9a7fc640 1 -- 172.21.15.5:0/3468979018 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6dac036030 con 0x55e039b1c4d0 2024-05-09T00:59:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.090+0000 7f6dc11be740 1 -- 172.21.15.5:0/3468979018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e039b1c4d0 msgr2=0x55e039ee8980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.090+0000 7f6dc11be740 1 --2- 172.21.15.5:0/3468979018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e039b1c4d0 0x55e039ee8980 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6dac0319e0 tx=0x7f6dac031b40 comp rx=0 tx=0).stop 2024-05-09T00:59:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.090+0000 7f6dc11be740 1 -- 172.21.15.5:0/3468979018 shutdown_connections 2024-05-09T00:59:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.090+0000 7f6dc11be740 1 --2- 172.21.15.5:0/3468979018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e039b1c4d0 0x55e039ee8980 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.090+0000 7f6dc11be740 1 --2- 172.21.15.5:0/3468979018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e039b142f0 0x55e039ee1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.090+0000 7f6dc11be740 1 --2- 172.21.15.5:0/3468979018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e039b3d1b0 0x55e039ee1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.090+0000 7f6dc11be740 1 -- 172.21.15.5:0/3468979018 >> 172.21.15.5:0/3468979018 conn(0x55e039b31300 msgr2=0x55e039edce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:59:42.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.091+0000 7f6dc11be740 1 -- 172.21.15.5:0/3468979018 shutdown_connections 2024-05-09T00:59:42.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.091+0000 7f6dc11be740 1 -- 172.21.15.5:0/3468979018 wait complete. 2024-05-09T00:59:42.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.092+0000 7f6dc11be740 1 -- start start 2024-05-09T00:59:42.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.092+0000 7f6db3fff640 1 --2- 172.21.15.5:0/3534886321 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f6d9c0b29b0 0x7f6d9c0b4ec0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x55e039ee26d0 tx=0x7f6da4005a50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:59:42.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.092+0000 7f6dc11be740 1 Processor -- start 2024-05-09T00:59:42.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.093+0000 7f6dc11be740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e039b142f0 0x55e039eeb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:42.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.093+0000 7f6dc11be740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e039b3d1b0 0x55e039eeb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:42.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.093+0000 7f6dc11be740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e039afbd80 0x55e039ef2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:42.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.093+0000 7f6dc11be740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e039ee93c0 con 0x55e039b142f0 2024-05-09T00:59:42.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.093+0000 7f6dc11be740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55e039dcc9c0 con 0x55e039afbd80 2024-05-09T00:59:42.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.093+0000 7f6dc11be740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55e039ee9540 con 0x55e039b3d1b0 2024-05-09T00:59:42.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.093+0000 7f6dc11be740 10 client.0 _ll_register_callbacks cb 0x55e039df9020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:59:42.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.094+0000 7f6db3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e039b142f0 0x55e039eeb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:42.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.094+0000 7f6db3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e039b142f0 0x55e039eeb0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60622/0 (socket says 172.21.15.5:60622) 2024-05-09T00:59:42.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.094+0000 7f6db3fff640 1 -- 172.21.15.5:0/2577409953 learned_addr learned my addr 172.21.15.5:0/2577409953 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:59:42.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.094+0000 7f6db3fff640 1 -- 172.21.15.5:0/2577409953 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e039b3d1b0 msgr2=0x55e039eeb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:59:42.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.094+0000 7f6db3fff640 1 --2- 172.21.15.5:0/2577409953 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e039b3d1b0 0x55e039eeb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:42.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.094+0000 7f6db3fff640 1 -- 172.21.15.5:0/2577409953 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e039afbd80 msgr2=0x55e039ef2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:59:42.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.094+0000 7f6db3fff640 1 --2- 172.21.15.5:0/2577409953 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e039afbd80 0x55e039ef2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:42.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.094+0000 7f6db3fff640 1 -- 172.21.15.5:0/2577409953 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e039df31f0 con 0x55e039b142f0 2024-05-09T00:59:42.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.095+0000 7f6db3fff640 1 --2- 172.21.15.5:0/2577409953 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e039b142f0 0x55e039eeb0c0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f6da400ca80 tx=0x7f6da40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16057cf62c889aff server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:59:42.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.095+0000 7f6d937fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:59:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.095+0000 7f6d937fe640 1 -- 172.21.15.5:0/2577409953 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da4022030 con 0x55e039b142f0 2024-05-09T00:59:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.095+0000 7f6dc11be740 1 -- 172.21.15.5:0/2577409953 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e039e533d0 con 0x55e039b142f0 2024-05-09T00:59:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.095+0000 7f6d937fe640 1 -- 172.21.15.5:0/2577409953 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f6da4009040 con 0x55e039b142f0 2024-05-09T00:59:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.096+0000 7f6d937fe640 1 -- 172.21.15.5:0/2577409953 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6da4004020 con 0x55e039b142f0 2024-05-09T00:59:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.096+0000 7f6dc11be740 10 client.16911 fetch_fsmap learned FSMap version 31 2024-05-09T00:59:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.096+0000 7f6dc11be740 1 -- 172.21.15.5:0/2577409953 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e039ba8790 con 0x55e039b142f0 2024-05-09T00:59:42.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.097+0000 7f6d937fe640 1 -- 172.21.15.5:0/2577409953 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55e039e533d0 con 0x55e039b142f0 2024-05-09T00:59:42.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.098+0000 7f6d937fe640 1 -- 172.21.15.5:0/2577409953 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f6da400f040 con 0x55e039b142f0 2024-05-09T00:59:42.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.098+0000 7f6dc11be740 10 client.16911 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:59:42.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.098+0000 7f6dc11be740 10 client.16911 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:59:42.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.098+0000 7f6dc11be740 20 client.16911 populate_metadata read hostname 'smithi005' 2024-05-09T00:59:42.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.098+0000 7f6dc11be740 1 --2- 172.21.15.5:0/2577409953 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55e039ad11c0 0x55e039ef8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:42.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.098+0000 7f6dc11be740 4 client.16911 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:59:42.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.098+0000 7f6dc11be740 1 -- 172.21.15.5:0/2577409953 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e039e4d240 con 0x55e039ad11c0 2024-05-09T00:59:42.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.099+0000 7f6db37fe640 1 --2- 172.21.15.5:0/2577409953 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55e039ad11c0 0x55e039ef8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.101+0000 7f6d937fe640 10 client.16911 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:59:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.101+0000 7f6db37fe640 1 --2- 172.21.15.5:0/2577409953 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55e039ad11c0 0x55e039ef8ce0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b78192d7d99bc3dc server_cookie=6996edb8c147d562 in_seq=0 out_seq=0 2024-05-09T00:59:42.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.104+0000 7f6d937fe640 1 -- 172.21.15.5:0/2577409953 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e039e4d240 con 0x55e039ad11c0 2024-05-09T00:59:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.104+0000 7f6d937fe640 10 client.16911 handle_command_reply: tid=0 2024-05-09T00:59:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.104+0000 7f6dc11be740 1 client.16911 shutdown 2024-05-09T00:59:42.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:59:41 smithi131 ceph-mon[34547]: pgmap v771: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 909 MiB/s wr, 2.11k op/s 2024-05-09T00:59:42.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:59:41 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:59:42.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.104+0000 7f6dc11be740 1 -- 172.21.15.5:0/2577409953 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e039b142f0 msgr2=0x55e039eeb0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:42.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.104+0000 7f6dc11be740 1 --2- 172.21.15.5:0/2577409953 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e039b142f0 0x55e039eeb0c0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f6da400ca80 tx=0x7f6da40046c0 comp rx=0 tx=0).stop 2024-05-09T00:59:42.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.104+0000 7f6dc11be740 1 -- 172.21.15.5:0/2577409953 shutdown_connections 2024-05-09T00:59:42.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.104+0000 7f6dc11be740 1 --2- 172.21.15.5:0/2577409953 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55e039ad11c0 0x55e039ef8ce0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.105+0000 7f6dc11be740 1 --2- 172.21.15.5:0/2577409953 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e039afbd80 0x55e039ef2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.105+0000 7f6dc11be740 1 --2- 172.21.15.5:0/2577409953 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e039b3d1b0 0x55e039eeb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.105+0000 7f6dc11be740 1 --2- 172.21.15.5:0/2577409953 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e039b142f0 0x55e039eeb0c0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.105+0000 7f6dc11be740 1 -- 172.21.15.5:0/2577409953 >> 172.21.15.5:0/2577409953 conn(0x55e039b31300 msgr2=0x55e039edfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:59:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.105+0000 7f6d937fe640 0 client.16911 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:59:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.105+0000 7f6dc11be740 1 -- 172.21.15.5:0/2577409953 shutdown_connections 2024-05-09T00:59:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.105+0000 7f6dc11be740 1 -- 172.21.15.5:0/2577409953 wait complete. 2024-05-09T00:59:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.105+0000 7f6dc11be740 20 client.16911 trim_cache size 0 max 16384 2024-05-09T00:59:42.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.151+0000 7f6dc11be740 1 Processor -- start 2024-05-09T00:59:42.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.151+0000 7f6dc11be740 1 -- start start 2024-05-09T00:59:42.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.152+0000 7f6dc11be740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e039afbd80 0x55e039ef8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:42.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.152+0000 7f6dc11be740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e039b142f0 0x55e039ee03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:42.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.152+0000 7f6db3fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e039afbd80 0x55e039ef8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:42.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.152+0000 7f6db3fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e039afbd80 0x55e039ef8c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:59180/0 (socket says 172.21.15.5:59180) 2024-05-09T00:59:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.159+0000 7f6db37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e039b142f0 0x55e039ee03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.159+0000 7f6dc11be740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e039b3d1b0 0x55e039edc710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.159+0000 7f6dc11be740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6da000e570 con 0x55e039b142f0 2024-05-09T00:59:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.159+0000 7f6dc11be740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6da00100b0 con 0x55e039b3d1b0 2024-05-09T00:59:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.159+0000 7f6dc11be740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6da000ec40 con 0x55e039afbd80 2024-05-09T00:59:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.159+0000 7f6db3fff640 1 -- 172.21.15.5:0/23120971 learned_addr learned my addr 172.21.15.5:0/23120971 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:59:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.159+0000 7f6db88a3640 1 --2- 172.21.15.5:0/23120971 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e039b3d1b0 0x55e039edc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.160+0000 7f6db3fff640 1 -- 172.21.15.5:0/23120971 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e039b3d1b0 msgr2=0x55e039edc710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:59:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.160+0000 7f6db3fff640 1 --2- 172.21.15.5:0/23120971 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e039b3d1b0 0x55e039edc710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.160+0000 7f6db3fff640 1 -- 172.21.15.5:0/23120971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e039b142f0 msgr2=0x55e039ee03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:59:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.160+0000 7f6db3fff640 1 --2- 172.21.15.5:0/23120971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e039b142f0 0x55e039ee03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.160+0000 7f6db3fff640 1 -- 172.21.15.5:0/23120971 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e039c1f5f0 con 0x55e039afbd80 2024-05-09T00:59:42.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.161+0000 7f6db3fff640 1 --2- 172.21.15.5:0/23120971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e039afbd80 0x55e039ef8c10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6da4004610 tx=0x7f6da4003a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d750582cda5bf15b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:59:42.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.161+0000 7f6d99ffb640 1 -- 172.21.15.5:0/23120971 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da4022070 con 0x55e039afbd80 2024-05-09T00:59:42.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.161+0000 7f6d99ffb640 1 -- 172.21.15.5:0/23120971 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6da4023040 con 0x55e039afbd80 2024-05-09T00:59:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.163+0000 7f6dc11be740 1 -- 172.21.15.5:0/23120971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e039afbd80 msgr2=0x55e039ef8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.163+0000 7f6dc11be740 1 --2- 172.21.15.5:0/23120971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e039afbd80 0x55e039ef8c10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6da4004610 tx=0x7f6da4003a40 comp rx=0 tx=0).stop 2024-05-09T00:59:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.165+0000 7f6dc11be740 1 -- 172.21.15.5:0/23120971 shutdown_connections 2024-05-09T00:59:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.165+0000 7f6dc11be740 1 --2- 172.21.15.5:0/23120971 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e039b3d1b0 0x55e039edc710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.165+0000 7f6dc11be740 1 --2- 172.21.15.5:0/23120971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e039b142f0 0x55e039ee03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.165+0000 7f6dc11be740 1 --2- 172.21.15.5:0/23120971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e039afbd80 0x55e039ef8c10 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.165+0000 7f6dc11be740 1 -- 172.21.15.5:0/23120971 >> 172.21.15.5:0/23120971 conn(0x55e039b31300 msgr2=0x55e039edfc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:59:42.169 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.169+0000 7f6dc11be740 1 -- 172.21.15.5:0/23120971 shutdown_connections 2024-05-09T00:59:42.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.171+0000 7f6dc11be740 1 -- 172.21.15.5:0/23120971 wait complete. 2024-05-09T00:59:42.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.171+0000 7f6dc11be740 1 -- start start 2024-05-09T00:59:42.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.172+0000 7f6dc11be740 1 Processor -- start 2024-05-09T00:59:42.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.172+0000 7f6dc11be740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e039afbd80 0x55e039eefb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.172+0000 7f6dc11be740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e039b142f0 0x55e039ef0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.172+0000 7f6dc11be740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e039b3d1b0 0x55e039f5f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.172+0000 7f6dc11be740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6da000f9e0 con 0x55e039b3d1b0 2024-05-09T00:59:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.172+0000 7f6dc11be740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6da000f310 con 0x55e039afbd80 2024-05-09T00:59:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.172+0000 7f6dc11be740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6da000c550 con 0x55e039b142f0 2024-05-09T00:59:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.172+0000 7f6dc11be740 10 client.0 _ll_register_callbacks cb 0x7f6da4016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:59:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.173+0000 7f6db3fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e039afbd80 0x55e039eefb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.173+0000 7f6db3fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e039afbd80 0x55e039eefb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:46964/0 (socket says 172.21.15.5:46964) 2024-05-09T00:59:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.173+0000 7f6db3fff640 1 -- 172.21.15.5:0/1394475929 learned_addr learned my addr 172.21.15.5:0/1394475929 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:59:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.173+0000 7f6db37fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e039b142f0 0x55e039ef0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:42.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.174+0000 7f6db37fe640 1 -- 172.21.15.5:0/1394475929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e039afbd80 msgr2=0x55e039eefb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:59:42.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.174+0000 7f6db37fe640 1 --2- 172.21.15.5:0/1394475929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e039afbd80 0x55e039eefb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:42.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.174+0000 7f6db37fe640 1 -- 172.21.15.5:0/1394475929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e039b3d1b0 msgr2=0x55e039f5f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:59:42.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.174+0000 7f6db37fe640 1 --2- 172.21.15.5:0/1394475929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e039b3d1b0 0x55e039f5f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:42.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.174+0000 7f6db37fe640 1 -- 172.21.15.5:0/1394475929 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e039e09ed0 con 0x55e039b142f0 2024-05-09T00:59:42.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.174+0000 7f6db37fe640 1 --2- 172.21.15.5:0/1394475929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e039b142f0 0x55e039ef0050 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x55e039ee28e0 tx=0x7f6da80047d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2418dd17a853979e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:59:42.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.176+0000 7f6d937fe640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-09T00:59:42.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.178+0000 7f6d937fe640 1 -- 172.21.15.5:0/1394475929 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da801d040 con 0x55e039b142f0 2024-05-09T00:59:42.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.178+0000 7f6dc11be740 1 -- 172.21.15.5:0/1394475929 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6da4023040 con 0x55e039b142f0 2024-05-09T00:59:42.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.178+0000 7f6d937fe640 1 -- 172.21.15.5:0/1394475929 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f6da80bf220 con 0x55e039b142f0 2024-05-09T00:59:42.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.182+0000 7f6d937fe640 1 -- 172.21.15.5:0/1394475929 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6da80aa280 con 0x55e039b142f0 2024-05-09T00:59:42.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.182+0000 7f6dc11be740 10 client.26311 fetch_fsmap learned FSMap version 31 2024-05-09T00:59:42.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.183+0000 7f6dc11be740 1 -- 172.21.15.5:0/1394475929 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e039f60430 con 0x55e039b142f0 2024-05-09T00:59:42.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.184+0000 7f6d937fe640 1 -- 172.21.15.5:0/1394475929 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6da4023040 con 0x55e039b142f0 2024-05-09T00:59:42.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.184+0000 7f6d937fe640 1 -- 172.21.15.5:0/1394475929 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f6da80a5bf0 con 0x55e039b142f0 2024-05-09T00:59:42.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.184+0000 7f6dc11be740 10 client.26311 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:59:42.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.184+0000 7f6dc11be740 10 client.26311 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T00:59:42.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.184+0000 7f6dc11be740 20 client.26311 populate_metadata read hostname 'smithi005' 2024-05-09T00:59:42.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.184+0000 7f6dc11be740 1 --2- 172.21.15.5:0/1394475929 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55e039f60db0 0x55e039f631d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:42.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.184+0000 7f6dc11be740 4 client.26311 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T00:59:42.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.184+0000 7f6dc11be740 1 -- 172.21.15.5:0/1394475929 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e039dc8350 con 0x55e039f60db0 2024-05-09T00:59:42.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.184+0000 7f6db3fff640 1 --2- 172.21.15.5:0/1394475929 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55e039f60db0 0x55e039f631d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:42.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.186+0000 7f6db3fff640 1 --2- 172.21.15.5:0/1394475929 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55e039f60db0 0x55e039f631d0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a30961feee02ef4 server_cookie=491ede2aee9d8b08 in_seq=0 out_seq=0 2024-05-09T00:59:42.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:42.186+0000 7f6d937fe640 10 client.26311 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:59:44.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:59:43 smithi005 ceph-mon[30365]: pgmap v772: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 866 MiB/s wr, 1.95k op/s 2024-05-09T00:59:44.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:59:43 smithi131 ceph-mon[34547]: pgmap v772: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 866 MiB/s wr, 1.95k op/s 2024-05-09T00:59:44.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:59:43 smithi157 ceph-mon[34294]: pgmap v772: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 866 MiB/s wr, 1.95k op/s 2024-05-09T00:59:45.040 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:59:44 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:59:45.040 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:59:44 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:59:45.040 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:59:44 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:59:45.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:59:44 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:59:45.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:59:44 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:59:45.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:59:44 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:59:45.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:59:44 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:59:45.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:59:44 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:59:45.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:59:44 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T00:59:46.041 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T00:59:46.041 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T00:59:46.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:59:45 smithi157 ceph-mon[34294]: pgmap v773: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 898 MiB/s wr, 1.98k op/s 2024-05-09T00:59:46.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:59:45 smithi005 ceph-mon[30365]: pgmap v773: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 898 MiB/s wr, 1.98k op/s 2024-05-09T00:59:46.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:59:45 smithi131 ceph-mon[34547]: pgmap v773: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 898 MiB/s wr, 1.98k op/s 2024-05-09T00:59:46.825 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:59:48.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:59:47 smithi157 ceph-mon[34294]: pgmap v774: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 878 MiB/s wr, 1.89k op/s 2024-05-09T00:59:48.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:59:47 smithi005 ceph-mon[30365]: pgmap v774: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 878 MiB/s wr, 1.89k op/s 2024-05-09T00:59:48.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:59:47 smithi131 ceph-mon[34547]: pgmap v774: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 878 MiB/s wr, 1.89k op/s 2024-05-09T00:59:50.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:59:49 smithi157 ceph-mon[34294]: pgmap v775: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 909 MiB/s wr, 1.79k op/s 2024-05-09T00:59:50.254 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:59:49 smithi005 ceph-mon[30365]: pgmap v775: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 909 MiB/s wr, 1.79k op/s 2024-05-09T00:59:50.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:59:49 smithi131 ceph-mon[34547]: pgmap v775: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 909 MiB/s wr, 1.79k op/s 2024-05-09T00:59:50.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:50.512+0000 7f6d937fe640 1 -- 172.21.15.5:0/1394475929 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e039dc8350 con 0x55e039f60db0 2024-05-09T00:59:50.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:50.512+0000 7f6d937fe640 10 client.26311 handle_command_reply: tid=0 2024-05-09T00:59:50.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:50.512+0000 7f6dc11be740 1 client.26311 shutdown 2024-05-09T00:59:50.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:50.512+0000 7f6dc11be740 1 -- 172.21.15.5:0/1394475929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e039b142f0 msgr2=0x55e039ef0050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:50.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:50.512+0000 7f6dc11be740 1 --2- 172.21.15.5:0/1394475929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e039b142f0 0x55e039ef0050 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x55e039ee28e0 tx=0x7f6da80047d0 comp rx=0 tx=0).stop 2024-05-09T00:59:50.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:50.512+0000 7f6dc11be740 1 -- 172.21.15.5:0/1394475929 shutdown_connections 2024-05-09T00:59:50.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:50.512+0000 7f6dc11be740 1 --2- 172.21.15.5:0/1394475929 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55e039f60db0 0x55e039f631d0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:50.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:50.512+0000 7f6dc11be740 1 --2- 172.21.15.5:0/1394475929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e039b3d1b0 0x55e039f5f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:50.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:50.512+0000 7f6dc11be740 1 --2- 172.21.15.5:0/1394475929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e039b142f0 0x55e039ef0050 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:50.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:50.512+0000 7f6dc11be740 1 --2- 172.21.15.5:0/1394475929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e039afbd80 0x55e039eefb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:50.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:50.512+0000 7f6d937fe640 0 client.26311 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:59:50.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:50.512+0000 7f6dc11be740 1 -- 172.21.15.5:0/1394475929 >> 172.21.15.5:0/1394475929 conn(0x55e039b31300 msgr2=0x55e039eeac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:59:50.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:50.512+0000 7f6dc11be740 1 -- 172.21.15.5:0/1394475929 shutdown_connections 2024-05-09T00:59:50.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:50.512+0000 7f6dc11be740 1 -- 172.21.15.5:0/1394475929 wait complete. 2024-05-09T00:59:50.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:50.513+0000 7f6dc11be740 20 client.26311 trim_cache size 0 max 16384 2024-05-09T00:59:50.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-09T00:59:50.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-09T00:59:50.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-09T00:59:50.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:50.513+0000 7f6dba306640 1 -- 172.21.15.5:0/3534886321 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f6d9c0b29b0 msgr2=0x7f6d9c0b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:50.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:50.513+0000 7f6dba306640 1 --2- 172.21.15.5:0/3534886321 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f6d9c0b29b0 0x7f6d9c0b4ec0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x55e039ee26d0 tx=0x7f6da4005a50 comp rx=0 tx=0).stop 2024-05-09T00:59:50.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:50.513+0000 7f6dba306640 1 -- 172.21.15.5:0/3534886321 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6db410b480 msgr2=0x7f6db4117620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:50.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:50.513+0000 7f6dba306640 1 --2- 172.21.15.5:0/3534886321 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6db410b480 0x7f6db4117620 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6da800c970 tx=0x7f6da800ce30 comp rx=0 tx=0).stop 2024-05-09T00:59:50.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:50.513+0000 7f6dba306640 1 -- 172.21.15.5:0/3534886321 shutdown_connections 2024-05-09T00:59:50.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:50.513+0000 7f6dba306640 1 --2- 172.21.15.5:0/3534886321 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f6d9c0b29b0 0x7f6d9c0b4ec0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:50.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:50.513+0000 7f6dba306640 1 --2- 172.21.15.5:0/3534886321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6db41101a0 0x7f6db41185c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:50.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:50.513+0000 7f6dba306640 1 --2- 172.21.15.5:0/3534886321 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6db410b480 0x7f6db4117620 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:50.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:50.513+0000 7f6dba306640 1 --2- 172.21.15.5:0/3534886321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6db410a550 0x7f6db41170a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:50.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:50.513+0000 7f6dba306640 1 -- 172.21.15.5:0/3534886321 >> 172.21.15.5:0/3534886321 conn(0x7f6db4070670 msgr2=0x7f6db410c180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:59:50.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:50.513+0000 7f6dba306640 1 -- 172.21.15.5:0/3534886321 shutdown_connections 2024-05-09T00:59:50.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T00:59:50.513+0000 7f6dba306640 1 -- 172.21.15.5:0/3534886321 wait complete. 2024-05-09T00:59:50.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.520+0000 7f37d14e6700 1 -- 172.21.15.5:0/388224315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37cc1034c0 msgr2=0x7f37cc1038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:50.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.521+0000 7f37d14e6700 1 --2- 172.21.15.5:0/388224315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37cc1034c0 0x7f37cc1038d0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f37bc0099c0 tx=0x7f37bc009cd0 comp rx=0 tx=0).stop 2024-05-09T00:59:50.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.521+0000 7f37d14e6700 1 -- 172.21.15.5:0/388224315 shutdown_connections 2024-05-09T00:59:50.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.521+0000 7f37d14e6700 1 --2- 172.21.15.5:0/388224315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37cc108740 0x7f37cc1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:50.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.521+0000 7f37d14e6700 1 --2- 172.21.15.5:0/388224315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37cc1034c0 0x7f37cc1038d0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:50.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.521+0000 7f37d14e6700 1 --2- 172.21.15.5:0/388224315 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37cc107df0 0x7f37cc108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:50.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.521+0000 7f37d14e6700 1 -- 172.21.15.5:0/388224315 >> 172.21.15.5:0/388224315 conn(0x7f37cc0fedc0 msgr2=0x7f37cc1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:59:50.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.521+0000 7f37d14e6700 1 -- 172.21.15.5:0/388224315 shutdown_connections 2024-05-09T00:59:50.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.521+0000 7f37d14e6700 1 -- 172.21.15.5:0/388224315 wait complete. 2024-05-09T00:59:50.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.522+0000 7f37d14e6700 1 Processor -- start 2024-05-09T00:59:50.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.522+0000 7f37d14e6700 1 -- start start 2024-05-09T00:59:50.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.522+0000 7f37d14e6700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37cc1034c0 0x7f37cc19f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:50.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.522+0000 7f37d14e6700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37cc107df0 0x7f37cc19ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:50.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.522+0000 7f37d14e6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37cc108740 0x7f37cc199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:50.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.522+0000 7f37d14e6700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc113370 con 0x7f37cc108740 2024-05-09T00:59:50.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.522+0000 7f37d14e6700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc1131f0 con 0x7f37cc1034c0 2024-05-09T00:59:50.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.522+0000 7f37d14e6700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc1134f0 con 0x7f37cc107df0 2024-05-09T00:59:50.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.523+0000 7f37cb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37cc108740 0x7f37cc199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:50.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.523+0000 7f37ca7fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37cc107df0 0x7f37cc19ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:50.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.523+0000 7f37cb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37cc108740 0x7f37cc199b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36758/0 (socket says 172.21.15.5:36758) 2024-05-09T00:59:50.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.523+0000 7f37cb7fe700 1 -- 172.21.15.5:0/136296169 learned_addr learned my addr 172.21.15.5:0/136296169 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:59:50.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.523+0000 7f37caffd700 1 --2- 172.21.15.5:0/136296169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37cc1034c0 0x7f37cc19f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:50.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.523+0000 7f37ca7fc700 1 -- 172.21.15.5:0/136296169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37cc1034c0 msgr2=0x7f37cc19f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:50.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.523+0000 7f37ca7fc700 1 --2- 172.21.15.5:0/136296169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37cc1034c0 0x7f37cc19f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:50.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.523+0000 7f37ca7fc700 1 -- 172.21.15.5:0/136296169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37cc108740 msgr2=0x7f37cc199b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:50.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.523+0000 7f37ca7fc700 1 --2- 172.21.15.5:0/136296169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37cc108740 0x7f37cc199b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:50.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.523+0000 7f37ca7fc700 1 -- 172.21.15.5:0/136296169 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37cc19a280 con 0x7f37cc107df0 2024-05-09T00:59:50.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.523+0000 7f37ca7fc700 1 --2- 172.21.15.5:0/136296169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37cc107df0 0x7f37cc19ff00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f37cc076ab0 tx=0x7f37c000dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:59:50.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.524+0000 7f37b3fff700 1 -- 172.21.15.5:0/136296169 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c0016070 con 0x7f37cc107df0 2024-05-09T00:59:50.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.524+0000 7f37d14e6700 1 -- 172.21.15.5:0/136296169 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37cc19a570 con 0x7f37cc107df0 2024-05-09T00:59:50.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.524+0000 7f37b3fff700 1 -- 172.21.15.5:0/136296169 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37c000fd40 con 0x7f37cc107df0 2024-05-09T00:59:50.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.524+0000 7f37b3fff700 1 -- 172.21.15.5:0/136296169 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c000e3b0 con 0x7f37cc107df0 2024-05-09T00:59:50.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.524+0000 7f37d14e6700 1 -- 172.21.15.5:0/136296169 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37cc06b610 con 0x7f37cc107df0 2024-05-09T00:59:50.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.527+0000 7f37b3fff700 1 -- 172.21.15.5:0/136296169 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f37c0019070 con 0x7f37cc107df0 2024-05-09T00:59:50.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.528+0000 7f37b3fff700 1 --2- 172.21.15.5:0/136296169 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f37b40b2880 0x7f37b40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:50.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.528+0000 7f37caffd700 1 --2- 172.21.15.5:0/136296169 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f37b40b2880 0x7f37b40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:50.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.528+0000 7f37b3fff700 1 -- 172.21.15.5:0/136296169 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f37c00e1c40 con 0x7f37cc107df0 2024-05-09T00:59:50.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.529+0000 7f37dfefcb80 1 Processor -- start 2024-05-09T00:59:50.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.529+0000 7f37dfefcb80 1 -- start start 2024-05-09T00:59:50.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.529+0000 7f37caffd700 1 --2- 172.21.15.5:0/136296169 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f37b40b2880 0x7f37b40b4d90 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f37bc000c00 tx=0x7f37bc01c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:59:50.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.529+0000 7f37dfefcb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56515aa9a1e0 0x56515ab33570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:50.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.529+0000 7f37dfefcb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56515a9b0520 0x56515ab33ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:50.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.529+0000 7f37dfefcb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56515ab2d720 0x56515ab2db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:50.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.529+0000 7f37dfefcb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56515aa50070 con 0x56515ab2d720 2024-05-09T00:59:50.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.529+0000 7f37dfefcb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x56515a9ee870 con 0x56515a9b0520 2024-05-09T00:59:50.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.529+0000 7f37dfefcb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56515aa4c030 con 0x56515aa9a1e0 2024-05-09T00:59:50.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.529+0000 7f37caffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56515aa9a1e0 0x56515ab33570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:50.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.529+0000 7f37cb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56515ab2d720 0x56515ab2db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:50.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.529+0000 7f37cb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56515ab2d720 0x56515ab2db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36768/0 (socket says 172.21.15.5:36768) 2024-05-09T00:59:50.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.529+0000 7f37cb7fe700 1 -- 172.21.15.5:0/1325972974 learned_addr learned my addr 172.21.15.5:0/1325972974 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:59:50.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.529+0000 7f37caffd700 1 --2- 172.21.15.5:0/1325972974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56515aa9a1e0 0x56515ab33570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:45784/0 (socket says 172.21.15.5:45784) 2024-05-09T00:59:50.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.529+0000 7f37ca7fc700 1 --2- 172.21.15.5:0/1325972974 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56515a9b0520 0x56515ab33ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:50.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.530+0000 7f37cb7fe700 1 -- 172.21.15.5:0/1325972974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56515aa9a1e0 msgr2=0x56515ab33570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:59:50.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.530+0000 7f37cb7fe700 1 --2- 172.21.15.5:0/1325972974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56515aa9a1e0 0x56515ab33570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:50.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.530+0000 7f37cb7fe700 1 -- 172.21.15.5:0/1325972974 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56515a9b0520 msgr2=0x56515ab33ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:59:50.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.530+0000 7f37cb7fe700 1 --2- 172.21.15.5:0/1325972974 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56515a9b0520 0x56515ab33ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:50.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.530+0000 7f37cb7fe700 1 -- 172.21.15.5:0/1325972974 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56515a9e63f0 con 0x56515ab2d720 2024-05-09T00:59:50.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.530+0000 7f37cb7fe700 1 --2- 172.21.15.5:0/1325972974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56515ab2d720 0x56515ab2db50 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f37c400e690 tx=0x7f37c4009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0b9eb2a191830ca server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:59:50.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.530+0000 7f37b17fa700 1 -- 172.21.15.5:0/1325972974 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c4019030 con 0x56515ab2d720 2024-05-09T00:59:50.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.530+0000 7f37b17fa700 1 -- 172.21.15.5:0/1325972974 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37c40092e0 con 0x56515ab2d720 2024-05-09T00:59:50.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.530+0000 7f37b17fa700 1 -- 172.21.15.5:0/1325972974 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c4007500 con 0x56515ab2d720 2024-05-09T00:59:50.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.531+0000 7f37dfefcb80 1 -- 172.21.15.5:0/1325972974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56515ab2d720 msgr2=0x56515ab2db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:50.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.531+0000 7f37dfefcb80 1 --2- 172.21.15.5:0/1325972974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56515ab2d720 0x56515ab2db50 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f37c400e690 tx=0x7f37c4009920 comp rx=0 tx=0).stop 2024-05-09T00:59:50.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.531+0000 7f37dfefcb80 1 -- 172.21.15.5:0/1325972974 shutdown_connections 2024-05-09T00:59:50.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.531+0000 7f37dfefcb80 1 --2- 172.21.15.5:0/1325972974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56515ab2d720 0x56515ab2db50 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:50.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.531+0000 7f37dfefcb80 1 --2- 172.21.15.5:0/1325972974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56515aa9a1e0 0x56515ab33570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:50.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.531+0000 7f37dfefcb80 1 --2- 172.21.15.5:0/1325972974 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56515a9b0520 0x56515ab33ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:50.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.531+0000 7f37dfefcb80 1 -- 172.21.15.5:0/1325972974 >> 172.21.15.5:0/1325972974 conn(0x56515aa10720 msgr2=0x56515ab28480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:59:50.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.531+0000 7f37dfefcb80 1 -- 172.21.15.5:0/1325972974 shutdown_connections 2024-05-09T00:59:50.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.531+0000 7f37dfefcb80 1 -- 172.21.15.5:0/1325972974 wait complete. 2024-05-09T00:59:50.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.532+0000 7f37dfefcb80 1 -- start start 2024-05-09T00:59:50.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.532+0000 7f37dfefcb80 1 Processor -- start 2024-05-09T00:59:50.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.532+0000 7f37dfefcb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56515a9b0520 0x56515ab37280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:50.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.532+0000 7f37dfefcb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56515aa9a1e0 0x56515ab377c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:50.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.532+0000 7f37dfefcb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56515ab2d720 0x56515ab37d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:50.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.532+0000 7f37dfefcb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56515a8cbc20 con 0x56515aa9a1e0 2024-05-09T00:59:50.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.532+0000 7f37dfefcb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x56515aa0a1d0 con 0x56515ab2d720 2024-05-09T00:59:50.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.532+0000 7f37dfefcb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56515ab3a250 con 0x56515a9b0520 2024-05-09T00:59:50.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.532+0000 7f37dfefcb80 10 client.0 _ll_register_callbacks cb 0x56515a8c1740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:59:50.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.532+0000 7f37ca7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56515aa9a1e0 0x56515ab377c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:50.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.532+0000 7f37ca7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56515aa9a1e0 0x56515ab377c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36784/0 (socket says 172.21.15.5:36784) 2024-05-09T00:59:50.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.532+0000 7f37ca7fc700 1 -- 172.21.15.5:0/4191757436 learned_addr learned my addr 172.21.15.5:0/4191757436 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:59:50.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.532+0000 7f37caffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56515a9b0520 0x56515ab37280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:50.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.533+0000 7f37cb7fe700 1 --2- 172.21.15.5:0/4191757436 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56515ab2d720 0x56515ab37d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:50.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.533+0000 7f37ca7fc700 1 -- 172.21.15.5:0/4191757436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56515a9b0520 msgr2=0x56515ab37280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:59:50.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.533+0000 7f37ca7fc700 1 --2- 172.21.15.5:0/4191757436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56515a9b0520 0x56515ab37280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:50.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.533+0000 7f37ca7fc700 1 -- 172.21.15.5:0/4191757436 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56515ab2d720 msgr2=0x56515ab37d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:59:50.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.533+0000 7f37ca7fc700 1 --2- 172.21.15.5:0/4191757436 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56515ab2d720 0x56515ab37d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:50.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.533+0000 7f37ca7fc700 1 -- 172.21.15.5:0/4191757436 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56515a8cb510 con 0x56515aa9a1e0 2024-05-09T00:59:50.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.533+0000 7f37cb7fe700 1 --2- 172.21.15.5:0/4191757436 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56515ab2d720 0x56515ab37d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:59:50.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.533+0000 7f37ca7fc700 1 --2- 172.21.15.5:0/4191757436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56515aa9a1e0 0x56515ab377c0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f37c00dfa80 tx=0x7f37c00a9770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2b2766f7a3515d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:59:50.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.533+0000 7f37aeff5700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:59:50.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.533+0000 7f37aeff5700 1 -- 172.21.15.5:0/4191757436 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c0020040 con 0x56515aa9a1e0 2024-05-09T00:59:50.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.533+0000 7f37aeff5700 1 -- 172.21.15.5:0/4191757436 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f37c00dfde0 con 0x56515aa9a1e0 2024-05-09T00:59:50.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.533+0000 7f37dfefcb80 1 -- 172.21.15.5:0/4191757436 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37c40092e0 con 0x56515aa9a1e0 2024-05-09T00:59:50.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.533+0000 7f37aeff5700 1 -- 172.21.15.5:0/4191757436 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f37c0010580 con 0x56515aa9a1e0 2024-05-09T00:59:50.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.533+0000 7f37dfefcb80 10 client.16923 fetch_fsmap learned FSMap version 31 2024-05-09T00:59:50.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.533+0000 7f37dfefcb80 1 -- 172.21.15.5:0/4191757436 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56515ab4d470 con 0x56515aa9a1e0 2024-05-09T00:59:50.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.533+0000 7f37aeff5700 1 -- 172.21.15.5:0/4191757436 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37c40092e0 con 0x56515aa9a1e0 2024-05-09T00:59:50.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.534+0000 7f37aeff5700 1 -- 172.21.15.5:0/4191757436 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f37c00ad170 con 0x56515aa9a1e0 2024-05-09T00:59:50.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.534+0000 7f37dfefcb80 10 client.16923 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:59:50.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.534+0000 7f37dfefcb80 10 client.16923 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:59:50.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.534+0000 7f37dfefcb80 20 client.16923 populate_metadata read hostname 'smithi005' 2024-05-09T00:59:50.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.534+0000 7f37dfefcb80 1 --2- 172.21.15.5:0/4191757436 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56515ab4f9f0 0x56515ab51e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:50.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.534+0000 7f37dfefcb80 4 client.16923 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:59:50.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.534+0000 7f37dfefcb80 1 -- 172.21.15.5:0/4191757436 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56515aa5da50 con 0x56515ab4f9f0 2024-05-09T00:59:50.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.534+0000 7f37caffd700 1 --2- 172.21.15.5:0/4191757436 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56515ab4f9f0 0x56515ab51e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:50.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.535+0000 7f37caffd700 1 --2- 172.21.15.5:0/4191757436 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56515ab4f9f0 0x56515ab51e30 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1242d3b979e3ec6 server_cookie=e1dba39f2e3b62c5 in_seq=0 out_seq=0 2024-05-09T00:59:50.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.535+0000 7f37aeff5700 10 client.16923 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:59:50.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.536+0000 7f37aeff5700 1 -- 172.21.15.5:0/4191757436 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56515aa5da50 con 0x56515ab4f9f0 2024-05-09T00:59:50.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.536+0000 7f37aeff5700 10 client.16923 handle_command_reply: tid=0 2024-05-09T00:59:50.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.536+0000 7f37dfefcb80 1 client.16923 shutdown 2024-05-09T00:59:50.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.536+0000 7f37dfefcb80 1 -- 172.21.15.5:0/4191757436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56515aa9a1e0 msgr2=0x56515ab377c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:50.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.536+0000 7f37dfefcb80 1 --2- 172.21.15.5:0/4191757436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56515aa9a1e0 0x56515ab377c0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f37c00dfa80 tx=0x7f37c00a9770 comp rx=0 tx=0).stop 2024-05-09T00:59:50.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.536+0000 7f37dfefcb80 1 -- 172.21.15.5:0/4191757436 shutdown_connections 2024-05-09T00:59:50.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.536+0000 7f37dfefcb80 1 --2- 172.21.15.5:0/4191757436 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56515ab4f9f0 0x56515ab51e30 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:50.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.537+0000 7f37dfefcb80 1 --2- 172.21.15.5:0/4191757436 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56515ab2d720 0x56515ab37d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:50.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.537+0000 7f37dfefcb80 1 --2- 172.21.15.5:0/4191757436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56515a9b0520 0x56515ab37280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:50.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.537+0000 7f37dfefcb80 1 --2- 172.21.15.5:0/4191757436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56515aa9a1e0 0x56515ab377c0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:50.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.537+0000 7f37aeff5700 0 client.16923 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:59:50.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.537+0000 7f37dfefcb80 1 -- 172.21.15.5:0/4191757436 >> 172.21.15.5:0/4191757436 conn(0x56515aa10720 msgr2=0x56515ab28240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:59:50.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.537+0000 7f37dfefcb80 1 -- 172.21.15.5:0/4191757436 shutdown_connections 2024-05-09T00:59:50.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.537+0000 7f37dfefcb80 1 -- 172.21.15.5:0/4191757436 wait complete. 2024-05-09T00:59:50.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.537+0000 7f37dfefcb80 20 client.16923 trim_cache size 0 max 16384 2024-05-09T00:59:50.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.558+0000 7f37dfefcb80 1 Processor -- start 2024-05-09T00:59:50.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.558+0000 7f37dfefcb80 1 -- start start 2024-05-09T00:59:50.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.558+0000 7f37dfefcb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56515a9b0520 0x56515ab31840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:50.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.558+0000 7f37dfefcb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56515aa9a1e0 0x56515ab36470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:50.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.558+0000 7f37dfefcb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56515ab2d720 0x56515ab37b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:50.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.558+0000 7f37dfefcb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc1acb00 con 0x56515ab2d720 2024-05-09T00:59:50.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.558+0000 7f37dfefcb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc1ae640 con 0x56515a9b0520 2024-05-09T00:59:50.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.558+0000 7f37dfefcb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc1ad1d0 con 0x56515aa9a1e0 2024-05-09T00:59:50.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.558+0000 7f37ca7fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56515aa9a1e0 0x56515ab36470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:50.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.558+0000 7f37ca7fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56515aa9a1e0 0x56515ab36470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:45794/0 (socket says 172.21.15.5:45794) 2024-05-09T00:59:50.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.558+0000 7f37ca7fc700 1 -- 172.21.15.5:0/2034448268 learned_addr learned my addr 172.21.15.5:0/2034448268 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:59:50.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.558+0000 7f37cb7fe700 1 --2- 172.21.15.5:0/2034448268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56515ab2d720 0x56515ab37b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:50.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.558+0000 7f37caffd700 1 --2- 172.21.15.5:0/2034448268 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56515a9b0520 0x56515ab31840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:50.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.559+0000 7f37ca7fc700 1 -- 172.21.15.5:0/2034448268 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56515a9b0520 msgr2=0x56515ab31840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:59:50.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.559+0000 7f37ca7fc700 1 --2- 172.21.15.5:0/2034448268 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56515a9b0520 0x56515ab31840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:50.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.559+0000 7f37ca7fc700 1 -- 172.21.15.5:0/2034448268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56515ab2d720 msgr2=0x56515ab37b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:59:50.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.559+0000 7f37ca7fc700 1 --2- 172.21.15.5:0/2034448268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56515ab2d720 0x56515ab37b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:50.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.559+0000 7f37ca7fc700 1 -- 172.21.15.5:0/2034448268 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56515ab3a080 con 0x56515aa9a1e0 2024-05-09T00:59:50.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.559+0000 7f37ca7fc700 1 --2- 172.21.15.5:0/2034448268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56515aa9a1e0 0x56515ab36470 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f37c0005950 tx=0x7f37c0010f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c2e367fcb40a627 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:59:50.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.559+0000 7f37b0ff9700 1 -- 172.21.15.5:0/2034448268 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c0020070 con 0x56515aa9a1e0 2024-05-09T00:59:50.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.559+0000 7f37b0ff9700 1 -- 172.21.15.5:0/2034448268 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37c00a8d40 con 0x56515aa9a1e0 2024-05-09T00:59:50.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.559+0000 7f37b0ff9700 1 -- 172.21.15.5:0/2034448268 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c00a8490 con 0x56515aa9a1e0 2024-05-09T00:59:50.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.560+0000 7f37dfefcb80 1 -- 172.21.15.5:0/2034448268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56515aa9a1e0 msgr2=0x56515ab36470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:50.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.560+0000 7f37dfefcb80 1 --2- 172.21.15.5:0/2034448268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56515aa9a1e0 0x56515ab36470 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f37c0005950 tx=0x7f37c0010f80 comp rx=0 tx=0).stop 2024-05-09T00:59:50.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.560+0000 7f37dfefcb80 1 -- 172.21.15.5:0/2034448268 shutdown_connections 2024-05-09T00:59:50.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.560+0000 7f37dfefcb80 1 --2- 172.21.15.5:0/2034448268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56515ab2d720 0x56515ab37b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:50.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.560+0000 7f37dfefcb80 1 --2- 172.21.15.5:0/2034448268 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56515a9b0520 0x56515ab31840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:50.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.560+0000 7f37dfefcb80 1 --2- 172.21.15.5:0/2034448268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56515aa9a1e0 0x56515ab36470 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:50.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.560+0000 7f37dfefcb80 1 -- 172.21.15.5:0/2034448268 >> 172.21.15.5:0/2034448268 conn(0x56515aa10720 msgr2=0x56515ab2cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:59:50.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.560+0000 7f37dfefcb80 1 -- 172.21.15.5:0/2034448268 shutdown_connections 2024-05-09T00:59:50.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.560+0000 7f37dfefcb80 1 -- 172.21.15.5:0/2034448268 wait complete. 2024-05-09T00:59:50.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.561+0000 7f37dfefcb80 1 -- start start 2024-05-09T00:59:50.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.561+0000 7f37dfefcb80 1 Processor -- start 2024-05-09T00:59:50.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.561+0000 7f37dfefcb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56515a9b0520 0x56515ab48380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:50.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.561+0000 7f37dfefcb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56515aa9a1e0 0x56515ab30870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:50.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.561+0000 7f37dfefcb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56515ab2d720 0x56515ab30f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:50.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.561+0000 7f37dfefcb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc1adf70 con 0x56515aa9a1e0 2024-05-09T00:59:50.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.561+0000 7f37dfefcb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc1ad8a0 con 0x56515ab2d720 2024-05-09T00:59:50.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.561+0000 7f37dfefcb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc1aaaa0 con 0x56515a9b0520 2024-05-09T00:59:50.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.561+0000 7f37dfefcb80 10 client.0 _ll_register_callbacks cb 0x56515a8c1740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:59:50.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.561+0000 7f37caffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56515a9b0520 0x56515ab48380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:50.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.561+0000 7f37caffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56515a9b0520 0x56515ab48380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:45798/0 (socket says 172.21.15.5:45798) 2024-05-09T00:59:50.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.561+0000 7f37caffd700 1 -- 172.21.15.5:0/1545348065 learned_addr learned my addr 172.21.15.5:0/1545348065 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:59:50.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.561+0000 7f37ca7fc700 1 --2- 172.21.15.5:0/1545348065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56515aa9a1e0 0x56515ab30870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:50.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.561+0000 7f37cb7fe700 1 --2- 172.21.15.5:0/1545348065 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56515ab2d720 0x56515ab30f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:50.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.562+0000 7f37caffd700 1 -- 172.21.15.5:0/1545348065 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56515ab2d720 msgr2=0x56515ab30f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:59:50.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.562+0000 7f37caffd700 1 --2- 172.21.15.5:0/1545348065 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56515ab2d720 0x56515ab30f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:50.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.562+0000 7f37caffd700 1 -- 172.21.15.5:0/1545348065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56515aa9a1e0 msgr2=0x56515ab30870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:59:50.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.562+0000 7f37caffd700 1 --2- 172.21.15.5:0/1545348065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56515aa9a1e0 0x56515ab30870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:50.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.562+0000 7f37caffd700 1 -- 172.21.15.5:0/1545348065 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56515ab21fe0 con 0x56515a9b0520 2024-05-09T00:59:50.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.562+0000 7f37ca7fc700 1 --2- 172.21.15.5:0/1545348065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56515aa9a1e0 0x56515ab30870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:59:50.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.562+0000 7f37caffd700 1 --2- 172.21.15.5:0/1545348065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56515a9b0520 0x56515ab48380 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f37bc00a010 tx=0x7f37bc005040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7469d3374dae3433 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:59:50.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.562+0000 7f37aeff5700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-09T00:59:50.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.562+0000 7f37aeff5700 1 -- 172.21.15.5:0/1545348065 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37bc037070 con 0x56515a9b0520 2024-05-09T00:59:50.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.562+0000 7f37dfefcb80 1 -- 172.21.15.5:0/1545348065 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37c00a8d40 con 0x56515a9b0520 2024-05-09T00:59:50.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.562+0000 7f37aeff5700 1 -- 172.21.15.5:0/1545348065 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f37bc003da0 con 0x56515a9b0520 2024-05-09T00:59:50.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.563+0000 7f37aeff5700 1 -- 172.21.15.5:0/1545348065 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f37bc00b790 con 0x56515a9b0520 2024-05-09T00:59:50.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.563+0000 7f37dfefcb80 10 client.26329 fetch_fsmap learned FSMap version 31 2024-05-09T00:59:50.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.563+0000 7f37dfefcb80 1 -- 172.21.15.5:0/1545348065 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56515abba1a0 con 0x56515a9b0520 2024-05-09T00:59:50.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.563+0000 7f37aeff5700 1 -- 172.21.15.5:0/1545348065 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37c00a8d40 con 0x56515a9b0520 2024-05-09T00:59:50.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.563+0000 7f37aeff5700 1 -- 172.21.15.5:0/1545348065 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f37bc023070 con 0x56515a9b0520 2024-05-09T00:59:50.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.563+0000 7f37dfefcb80 10 client.26329 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:59:50.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.563+0000 7f37dfefcb80 10 client.26329 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:59:50.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.563+0000 7f37dfefcb80 20 client.26329 populate_metadata read hostname 'smithi005' 2024-05-09T00:59:50.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.563+0000 7f37dfefcb80 1 --2- 172.21.15.5:0/1545348065 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56515abba830 0x56515abbcc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:50.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.563+0000 7f37dfefcb80 4 client.26329 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T00:59:50.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.563+0000 7f37dfefcb80 1 -- 172.21.15.5:0/1545348065 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56515aa57af0 con 0x56515abba830 2024-05-09T00:59:50.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.563+0000 7f37ca7fc700 1 --2- 172.21.15.5:0/1545348065 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56515abba830 0x56515abbcc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:50.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.564+0000 7f37aeff5700 10 client.26329 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:59:50.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.564+0000 7f37ca7fc700 1 --2- 172.21.15.5:0/1545348065 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56515abba830 0x56515abbcc50 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a2a57afdb20ae9 server_cookie=5a60902c89b1d5aa in_seq=0 out_seq=0 2024-05-09T00:59:50.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.566+0000 7f37aeff5700 1 -- 172.21.15.5:0/1545348065 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56515aa57af0 con 0x56515abba830 2024-05-09T00:59:50.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.566+0000 7f37aeff5700 10 client.26329 handle_command_reply: tid=0 2024-05-09T00:59:50.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.566+0000 7f37dfefcb80 1 client.26329 shutdown 2024-05-09T00:59:50.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.566+0000 7f37dfefcb80 1 -- 172.21.15.5:0/1545348065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56515a9b0520 msgr2=0x56515ab48380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:50.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.566+0000 7f37dfefcb80 1 --2- 172.21.15.5:0/1545348065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56515a9b0520 0x56515ab48380 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f37bc00a010 tx=0x7f37bc005040 comp rx=0 tx=0).stop 2024-05-09T00:59:50.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.566+0000 7f37dfefcb80 1 -- 172.21.15.5:0/1545348065 shutdown_connections 2024-05-09T00:59:50.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.566+0000 7f37dfefcb80 1 --2- 172.21.15.5:0/1545348065 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56515abba830 0x56515abbcc50 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:50.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.566+0000 7f37dfefcb80 1 --2- 172.21.15.5:0/1545348065 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56515ab2d720 0x56515ab30f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:50.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.566+0000 7f37dfefcb80 1 --2- 172.21.15.5:0/1545348065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56515a9b0520 0x56515ab48380 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:50.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.566+0000 7f37aeff5700 0 client.26329 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:59:50.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.566+0000 7f37dfefcb80 1 --2- 172.21.15.5:0/1545348065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56515aa9a1e0 0x56515ab30870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:50.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.566+0000 7f37dfefcb80 1 -- 172.21.15.5:0/1545348065 >> 172.21.15.5:0/1545348065 conn(0x56515aa10720 msgr2=0x56515ab39140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:59:50.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.566+0000 7f37dfefcb80 1 -- 172.21.15.5:0/1545348065 shutdown_connections 2024-05-09T00:59:50.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.566+0000 7f37dfefcb80 1 -- 172.21.15.5:0/1545348065 wait complete. 2024-05-09T00:59:50.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.566+0000 7f37dfefcb80 20 client.26329 trim_cache size 0 max 16384 2024-05-09T00:59:50.571 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-09T00:59:50.571 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-05-09T00:59:50.571 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "1b545c29-122a-48f2-bd62-6487559191a7", 2024-05-09T00:59:50.572 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-05-09T00:59:50.572 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-09T00:59:50.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.567+0000 7f37d14e6700 1 -- 172.21.15.5:0/136296169 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f37b40b2880 msgr2=0x7f37b40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:50.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.567+0000 7f37d14e6700 1 --2- 172.21.15.5:0/136296169 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f37b40b2880 0x7f37b40b4d90 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f37bc000c00 tx=0x7f37bc01c040 comp rx=0 tx=0).stop 2024-05-09T00:59:50.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.567+0000 7f37d14e6700 1 -- 172.21.15.5:0/136296169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37cc107df0 msgr2=0x7f37cc19ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:50.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.567+0000 7f37d14e6700 1 --2- 172.21.15.5:0/136296169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37cc107df0 0x7f37cc19ff00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f37cc076ab0 tx=0x7f37c000dc30 comp rx=0 tx=0).stop 2024-05-09T00:59:50.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.567+0000 7f37d14e6700 1 -- 172.21.15.5:0/136296169 shutdown_connections 2024-05-09T00:59:50.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.567+0000 7f37d14e6700 1 --2- 172.21.15.5:0/136296169 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f37b40b2880 0x7f37b40b4d90 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:50.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.567+0000 7f37d14e6700 1 --2- 172.21.15.5:0/136296169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37cc108740 0x7f37cc199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:50.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.567+0000 7f37d14e6700 1 --2- 172.21.15.5:0/136296169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37cc1034c0 0x7f37cc19f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:50.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.567+0000 7f37d14e6700 1 --2- 172.21.15.5:0/136296169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37cc107df0 0x7f37cc19ff00 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:50.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.567+0000 7f37d14e6700 1 -- 172.21.15.5:0/136296169 >> 172.21.15.5:0/136296169 conn(0x7f37cc0fedc0 msgr2=0x7f37cc1096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:59:50.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.567+0000 7f37d14e6700 1 -- 172.21.15.5:0/136296169 shutdown_connections 2024-05-09T00:59:50.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:50.567+0000 7f37d14e6700 1 -- 172.21.15.5:0/136296169 wait complete. 2024-05-09T00:59:51.474 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T00:59:51.745 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:59:52.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:59:51 smithi157 ceph-mon[34294]: pgmap v776: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 895 MiB/s wr, 1.69k op/s 2024-05-09T00:59:52.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:59:51 smithi157 ceph-mon[34294]: scrub queued for path: / 2024-05-09T00:59:52.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:59:51 smithi157 ceph-mon[34294]: scrub summary: idle+waiting paths [/] 2024-05-09T00:59:52.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:59:51 smithi157 ceph-mon[34294]: scrub summary: active paths [/] 2024-05-09T00:59:52.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:59:51 smithi157 ceph-mon[34294]: scrub summary: active paths [/] 2024-05-09T00:59:52.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:59:51 smithi157 ceph-mon[34294]: scrub summary: idle+waiting paths [/] 2024-05-09T00:59:52.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:59:51 smithi005 ceph-mon[30365]: pgmap v776: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 895 MiB/s wr, 1.69k op/s 2024-05-09T00:59:52.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:59:51 smithi005 ceph-mon[30365]: scrub queued for path: / 2024-05-09T00:59:52.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:59:51 smithi005 ceph-mon[30365]: scrub summary: idle+waiting paths [/] 2024-05-09T00:59:52.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:59:51 smithi005 ceph-mon[30365]: scrub summary: active paths [/] 2024-05-09T00:59:52.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:59:51 smithi005 ceph-mon[30365]: scrub summary: active paths [/] 2024-05-09T00:59:52.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:59:51 smithi005 ceph-mon[30365]: scrub summary: idle+waiting paths [/] 2024-05-09T00:59:52.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:59:51 smithi131 ceph-mon[34547]: pgmap v776: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 895 MiB/s wr, 1.69k op/s 2024-05-09T00:59:52.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:59:51 smithi131 ceph-mon[34547]: scrub queued for path: / 2024-05-09T00:59:52.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:59:51 smithi131 ceph-mon[34547]: scrub summary: idle+waiting paths [/] 2024-05-09T00:59:52.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:59:51 smithi131 ceph-mon[34547]: scrub summary: active paths [/] 2024-05-09T00:59:52.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:59:51 smithi131 ceph-mon[34547]: scrub summary: active paths [/] 2024-05-09T00:59:52.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:59:51 smithi131 ceph-mon[34547]: scrub summary: idle+waiting paths [/] 2024-05-09T00:59:53.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.919+0000 7f2d9dfb7700 1 -- 172.21.15.5:0/3889896607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d98108200 msgr2=0x7f2d98108610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:53.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.919+0000 7f2d9dfb7700 1 --2- 172.21.15.5:0/3889896607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d98108200 0x7f2d98108610 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f2d88009990 tx=0x7f2d88009ca0 comp rx=0 tx=0).stop 2024-05-09T00:59:53.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.919+0000 7f2d9dfb7700 1 -- 172.21.15.5:0/3889896607 shutdown_connections 2024-05-09T00:59:53.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.919+0000 7f2d9dfb7700 1 --2- 172.21.15.5:0/3889896607 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d981042b0 0x7f2d98110070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:53.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.919+0000 7f2d9dfb7700 1 --2- 172.21.15.5:0/3889896607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d98108200 0x7f2d98108610 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:53.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.919+0000 7f2d9dfb7700 1 --2- 172.21.15.5:0/3889896607 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d98103960 0x7f2d98103d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:53.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.919+0000 7f2d9dfb7700 1 -- 172.21.15.5:0/3889896607 >> 172.21.15.5:0/3889896607 conn(0x7f2d980fe4e0 msgr2=0x7f2d98100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:59:53.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.920+0000 7f2d9dfb7700 1 -- 172.21.15.5:0/3889896607 shutdown_connections 2024-05-09T00:59:53.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.920+0000 7f2d9dfb7700 1 -- 172.21.15.5:0/3889896607 wait complete. 2024-05-09T00:59:53.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.920+0000 7f2d9dfb7700 1 Processor -- start 2024-05-09T00:59:53.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.920+0000 7f2d9dfb7700 1 -- start start 2024-05-09T00:59:53.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.921+0000 7f2d9dfb7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d98103960 0x7f2d980fe9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:53.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.921+0000 7f2d9dfb7700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d981042b0 0x7f2d980fef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:53.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.921+0000 7f2d9dfb7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d98108200 0x7f2d980ff440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:53.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.921+0000 7f2d9dfb7700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2d98113370 con 0x7f2d98103960 2024-05-09T00:59:53.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.921+0000 7f2d9dfb7700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2d981131f0 con 0x7f2d98108200 2024-05-09T00:59:53.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.921+0000 7f2d9dfb7700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2d981134f0 con 0x7f2d981042b0 2024-05-09T00:59:53.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.921+0000 7f2d977fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d98103960 0x7f2d980fe9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:53.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.921+0000 7f2d977fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d98103960 0x7f2d980fe9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37970/0 (socket says 172.21.15.5:37970) 2024-05-09T00:59:53.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.921+0000 7f2d977fe700 1 -- 172.21.15.5:0/3706434369 learned_addr learned my addr 172.21.15.5:0/3706434369 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:59:53.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.921+0000 7f2d96ffd700 1 --2- 172.21.15.5:0/3706434369 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d981042b0 0x7f2d980fef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:53.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.921+0000 7f2d97fff700 1 --2- 172.21.15.5:0/3706434369 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d98108200 0x7f2d980ff440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:53.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.921+0000 7f2d977fe700 1 -- 172.21.15.5:0/3706434369 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d981042b0 msgr2=0x7f2d980fef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:53.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.921+0000 7f2d977fe700 1 --2- 172.21.15.5:0/3706434369 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d981042b0 0x7f2d980fef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:53.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.921+0000 7f2d977fe700 1 -- 172.21.15.5:0/3706434369 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d98108200 msgr2=0x7f2d980ff440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:53.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.921+0000 7f2d977fe700 1 --2- 172.21.15.5:0/3706434369 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d98108200 0x7f2d980ff440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:53.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.921+0000 7f2d977fe700 1 -- 172.21.15.5:0/3706434369 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d981a9150 con 0x7f2d98103960 2024-05-09T00:59:53.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.921+0000 7f2d96ffd700 1 --2- 172.21.15.5:0/3706434369 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d981042b0 0x7f2d980fef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:59:53.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.922+0000 7f2d977fe700 1 --2- 172.21.15.5:0/3706434369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d98103960 0x7f2d980fe9c0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f2d88006010 tx=0x7f2d880047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:59:53.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.922+0000 7f2d94ff9700 1 -- 172.21.15.5:0/3706434369 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d88027070 con 0x7f2d98103960 2024-05-09T00:59:53.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.922+0000 7f2d9dfb7700 1 -- 172.21.15.5:0/3706434369 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d981a93e0 con 0x7f2d98103960 2024-05-09T00:59:53.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.922+0000 7f2d94ff9700 1 -- 172.21.15.5:0/3706434369 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d88022470 con 0x7f2d98103960 2024-05-09T00:59:53.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.922+0000 7f2d94ff9700 1 -- 172.21.15.5:0/3706434369 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d88017650 con 0x7f2d98103960 2024-05-09T00:59:53.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.922+0000 7f2d9dfb7700 1 -- 172.21.15.5:0/3706434369 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d981a9890 con 0x7f2d98103960 2024-05-09T00:59:53.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.925+0000 7f2d94ff9700 1 -- 172.21.15.5:0/3706434369 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f2d8802a070 con 0x7f2d98103960 2024-05-09T00:59:53.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.926+0000 7f2d94ff9700 1 --2- 172.21.15.5:0/3706434369 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f2d800b2880 0x7f2d800b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:53.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.926+0000 7f2dac9cdb80 1 Processor -- start 2024-05-09T00:59:53.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.926+0000 7f2d96ffd700 1 --2- 172.21.15.5:0/3706434369 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f2d800b2880 0x7f2d800b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:53.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.926+0000 7f2dac9cdb80 1 -- start start 2024-05-09T00:59:53.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.926+0000 7f2d94ff9700 1 -- 172.21.15.5:0/3706434369 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f2d880e2fb0 con 0x7f2d98103960 2024-05-09T00:59:53.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.926+0000 7f2dac9cdb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b594fba160 0x55b5950534f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:53.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.927+0000 7f2dac9cdb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b594ed04a0 0x55b595053a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:53.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.927+0000 7f2dac9cdb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b59504d6a0 0x55b59504dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:53.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.927+0000 7f2dac9cdb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b594f6fff0 con 0x55b59504d6a0 2024-05-09T00:59:53.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.927+0000 7f2dac9cdb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55b594f0e7f0 con 0x55b594fba160 2024-05-09T00:59:53.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.927+0000 7f2dac9cdb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b594f6bfb0 con 0x55b594ed04a0 2024-05-09T00:59:53.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.927+0000 7f2d97fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b59504d6a0 0x55b59504dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:53.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.927+0000 7f2d977fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b594fba160 0x55b5950534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:53.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.927+0000 7f2d96ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b594ed04a0 0x55b595053a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:53.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.927+0000 7f2d97fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b59504d6a0 0x55b59504dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37976/0 (socket says 172.21.15.5:37976) 2024-05-09T00:59:53.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.927+0000 7f2d97fff700 1 -- 172.21.15.5:0/4231044639 learned_addr learned my addr 172.21.15.5:0/4231044639 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:59:53.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.927+0000 7f2d96ffd700 1 --2- 172.21.15.5:0/3706434369 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f2d800b2880 0x7f2d800b4d90 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f2d98101b40 tx=0x7f2d8c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:59:53.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.927+0000 7f2d97fff700 1 -- 172.21.15.5:0/4231044639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b594ed04a0 msgr2=0x55b595053a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:59:53.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.927+0000 7f2d97fff700 1 --2- 172.21.15.5:0/4231044639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b594ed04a0 0x55b595053a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:53.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.927+0000 7f2d97fff700 1 -- 172.21.15.5:0/4231044639 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b594fba160 msgr2=0x55b5950534f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:59:53.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.927+0000 7f2d97fff700 1 --2- 172.21.15.5:0/4231044639 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b594fba160 0x55b5950534f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:53.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.927+0000 7f2d97fff700 1 -- 172.21.15.5:0/4231044639 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b594f06370 con 0x55b59504d6a0 2024-05-09T00:59:53.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.927+0000 7f2d96ffd700 1 --2- 172.21.15.5:0/4231044639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b594ed04a0 0x55b595053a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:59:53.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.927+0000 7f2d97fff700 1 --2- 172.21.15.5:0/4231044639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b59504d6a0 0x55b59504dad0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f2d90003fa0 tx=0x7f2d90009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ca770db17286800 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:59:53.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.929+0000 7f2d7dffb700 1 -- 172.21.15.5:0/4231044639 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d90004bb0 con 0x55b59504d6a0 2024-05-09T00:59:53.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.929+0000 7f2d7dffb700 1 -- 172.21.15.5:0/4231044639 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d90004d50 con 0x55b59504d6a0 2024-05-09T00:59:53.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.929+0000 7f2d7dffb700 1 -- 172.21.15.5:0/4231044639 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d90007d60 con 0x55b59504d6a0 2024-05-09T00:59:53.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.930+0000 7f2dac9cdb80 1 -- 172.21.15.5:0/4231044639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b59504d6a0 msgr2=0x55b59504dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:53.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.930+0000 7f2dac9cdb80 1 --2- 172.21.15.5:0/4231044639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b59504d6a0 0x55b59504dad0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f2d90003fa0 tx=0x7f2d90009ba0 comp rx=0 tx=0).stop 2024-05-09T00:59:53.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.930+0000 7f2dac9cdb80 1 -- 172.21.15.5:0/4231044639 shutdown_connections 2024-05-09T00:59:53.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.930+0000 7f2dac9cdb80 1 --2- 172.21.15.5:0/4231044639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b59504d6a0 0x55b59504dad0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:53.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.930+0000 7f2dac9cdb80 1 --2- 172.21.15.5:0/4231044639 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b594fba160 0x55b5950534f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:53.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.930+0000 7f2dac9cdb80 1 --2- 172.21.15.5:0/4231044639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b594ed04a0 0x55b595053a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:53.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.930+0000 7f2dac9cdb80 1 -- 172.21.15.5:0/4231044639 >> 172.21.15.5:0/4231044639 conn(0x55b594f306a0 msgr2=0x55b595048400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:59:53.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.930+0000 7f2dac9cdb80 1 -- 172.21.15.5:0/4231044639 shutdown_connections 2024-05-09T00:59:53.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.930+0000 7f2dac9cdb80 1 -- 172.21.15.5:0/4231044639 wait complete. 2024-05-09T00:59:53.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.931+0000 7f2dac9cdb80 1 -- start start 2024-05-09T00:59:53.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.931+0000 7f2dac9cdb80 1 Processor -- start 2024-05-09T00:59:53.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.931+0000 7f2dac9cdb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b594ed04a0 0x55b5950571b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:53.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.931+0000 7f2dac9cdb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b594fba160 0x55b5950576f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:53.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.931+0000 7f2dac9cdb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b59504d6a0 0x55b595057c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:53.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.931+0000 7f2dac9cdb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b594debba0 con 0x55b59504d6a0 2024-05-09T00:59:53.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.931+0000 7f2dac9cdb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55b594f2a150 con 0x55b594ed04a0 2024-05-09T00:59:53.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.931+0000 7f2dac9cdb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b59505a1d0 con 0x55b594fba160 2024-05-09T00:59:53.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.931+0000 7f2dac9cdb80 10 client.0 _ll_register_callbacks cb 0x55b594de16c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:59:53.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.932+0000 7f2d97fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b59504d6a0 0x55b595057c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:53.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.932+0000 7f2d97fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b59504d6a0 0x55b595057c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37988/0 (socket says 172.21.15.5:37988) 2024-05-09T00:59:53.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.932+0000 7f2d977fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b594ed04a0 0x55b5950571b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:53.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.932+0000 7f2d96ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b594fba160 0x55b5950576f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:53.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.932+0000 7f2d97fff700 1 -- 172.21.15.5:0/2875659313 learned_addr learned my addr 172.21.15.5:0/2875659313 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:59:53.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.932+0000 7f2d97fff700 1 -- 172.21.15.5:0/2875659313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b594fba160 msgr2=0x55b5950576f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:59:53.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.932+0000 7f2d97fff700 1 --2- 172.21.15.5:0/2875659313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b594fba160 0x55b5950576f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:53.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.932+0000 7f2d97fff700 1 -- 172.21.15.5:0/2875659313 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b594ed04a0 msgr2=0x55b5950571b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:59:53.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.932+0000 7f2d97fff700 1 --2- 172.21.15.5:0/2875659313 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b594ed04a0 0x55b5950571b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:53.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.932+0000 7f2d97fff700 1 -- 172.21.15.5:0/2875659313 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b594deb490 con 0x55b59504d6a0 2024-05-09T00:59:53.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.932+0000 7f2d97fff700 1 --2- 172.21.15.5:0/2875659313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b59504d6a0 0x55b595057c30 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f2d90007ef0 tx=0x7f2d90007f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36154dbc08f0671e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:59:53.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.932+0000 7f2d76ffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T00:59:53.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.932+0000 7f2d76ffd700 1 -- 172.21.15.5:0/2875659313 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d90022070 con 0x55b59504d6a0 2024-05-09T00:59:53.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.932+0000 7f2d76ffd700 1 -- 172.21.15.5:0/2875659313 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f2d90004b50 con 0x55b59504d6a0 2024-05-09T00:59:53.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.932+0000 7f2dac9cdb80 1 -- 172.21.15.5:0/2875659313 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d90004d50 con 0x55b59504d6a0 2024-05-09T00:59:53.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.933+0000 7f2d76ffd700 1 -- 172.21.15.5:0/2875659313 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2d9002e050 con 0x55b59504d6a0 2024-05-09T00:59:53.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.933+0000 7f2dac9cdb80 10 client.16941 fetch_fsmap learned FSMap version 31 2024-05-09T00:59:53.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.933+0000 7f2dac9cdb80 1 -- 172.21.15.5:0/2875659313 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b595064b90 con 0x55b59504d6a0 2024-05-09T00:59:53.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.933+0000 7f2d76ffd700 1 -- 172.21.15.5:0/2875659313 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d90004d50 con 0x55b59504d6a0 2024-05-09T00:59:53.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.933+0000 7f2d76ffd700 1 -- 172.21.15.5:0/2875659313 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f2d9001e410 con 0x55b59504d6a0 2024-05-09T00:59:53.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.933+0000 7f2dac9cdb80 10 client.16941 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:59:53.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.933+0000 7f2dac9cdb80 10 client.16941 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:59:53.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.933+0000 7f2dac9cdb80 20 client.16941 populate_metadata read hostname 'smithi005' 2024-05-09T00:59:53.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.933+0000 7f2dac9cdb80 1 --2- 172.21.15.5:0/2875659313 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55b5950670c0 0x55b5950694e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:53.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.933+0000 7f2dac9cdb80 4 client.16941 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:59:53.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.933+0000 7f2dac9cdb80 1 -- 172.21.15.5:0/2875659313 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b594f7d9d0 con 0x55b5950670c0 2024-05-09T00:59:53.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.933+0000 7f2d977fe700 1 --2- 172.21.15.5:0/2875659313 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55b5950670c0 0x55b5950694e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:53.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.934+0000 7f2d76ffd700 10 client.16941 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:59:53.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.934+0000 7f2d977fe700 1 --2- 172.21.15.5:0/2875659313 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55b5950670c0 0x55b5950694e0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a56f200d2f88a1a4 server_cookie=c24111a0f2d791a3 in_seq=0 out_seq=0 2024-05-09T00:59:53.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.936+0000 7f2d76ffd700 1 -- 172.21.15.5:0/2875659313 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b594f7d9d0 con 0x55b5950670c0 2024-05-09T00:59:53.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.936+0000 7f2d76ffd700 10 client.16941 handle_command_reply: tid=0 2024-05-09T00:59:53.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.936+0000 7f2dac9cdb80 1 client.16941 shutdown 2024-05-09T00:59:53.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.936+0000 7f2dac9cdb80 1 -- 172.21.15.5:0/2875659313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b59504d6a0 msgr2=0x55b595057c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:53.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.936+0000 7f2dac9cdb80 1 --2- 172.21.15.5:0/2875659313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b59504d6a0 0x55b595057c30 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f2d90007ef0 tx=0x7f2d90007f20 comp rx=0 tx=0).stop 2024-05-09T00:59:53.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.936+0000 7f2dac9cdb80 1 -- 172.21.15.5:0/2875659313 shutdown_connections 2024-05-09T00:59:53.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.936+0000 7f2dac9cdb80 1 --2- 172.21.15.5:0/2875659313 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55b5950670c0 0x55b5950694e0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:53.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.936+0000 7f2dac9cdb80 1 --2- 172.21.15.5:0/2875659313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b59504d6a0 0x55b595057c30 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:53.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.936+0000 7f2dac9cdb80 1 --2- 172.21.15.5:0/2875659313 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b594ed04a0 0x55b5950571b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:53.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.936+0000 7f2dac9cdb80 1 --2- 172.21.15.5:0/2875659313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b594fba160 0x55b5950576f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:53.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.936+0000 7f2dac9cdb80 1 -- 172.21.15.5:0/2875659313 >> 172.21.15.5:0/2875659313 conn(0x55b594f306a0 msgr2=0x55b5950481c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:59:53.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.936+0000 7f2d76ffd700 0 client.16941 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:59:53.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.936+0000 7f2dac9cdb80 1 -- 172.21.15.5:0/2875659313 shutdown_connections 2024-05-09T00:59:53.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.936+0000 7f2dac9cdb80 1 -- 172.21.15.5:0/2875659313 wait complete. 2024-05-09T00:59:53.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.936+0000 7f2dac9cdb80 20 client.16941 trim_cache size 0 max 16384 2024-05-09T00:59:53.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.957+0000 7f2dac9cdb80 1 Processor -- start 2024-05-09T00:59:53.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.957+0000 7f2dac9cdb80 1 -- start start 2024-05-09T00:59:53.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.957+0000 7f2dac9cdb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b594ed04a0 0x55b5950517c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:53.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.957+0000 7f2dac9cdb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b594fba160 0x55b595056130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:53.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.957+0000 7f2dac9cdb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b59504d6a0 0x55b59504d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:53.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.957+0000 7f2dac9cdb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2d981b0e60 con 0x55b594ed04a0 2024-05-09T00:59:53.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.957+0000 7f2dac9cdb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2d981b29a0 con 0x55b59504d6a0 2024-05-09T00:59:53.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.957+0000 7f2dac9cdb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2d981b1530 con 0x55b594fba160 2024-05-09T00:59:53.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.957+0000 7f2d977fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b594ed04a0 0x55b5950517c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:53.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.957+0000 7f2d977fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b594ed04a0 0x55b5950517c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38000/0 (socket says 172.21.15.5:38000) 2024-05-09T00:59:53.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.957+0000 7f2d96ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b594fba160 0x55b595056130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:53.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.957+0000 7f2d977fe700 1 -- 172.21.15.5:0/1565479107 learned_addr learned my addr 172.21.15.5:0/1565479107 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:59:53.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.957+0000 7f2d97fff700 1 --2- 172.21.15.5:0/1565479107 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b59504d6a0 0x55b59504d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:53.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.957+0000 7f2d977fe700 1 -- 172.21.15.5:0/1565479107 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b594fba160 msgr2=0x55b595056130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:59:53.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.957+0000 7f2d977fe700 1 --2- 172.21.15.5:0/1565479107 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b594fba160 0x55b595056130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:53.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.957+0000 7f2d977fe700 1 -- 172.21.15.5:0/1565479107 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b59504d6a0 msgr2=0x55b59504d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:59:53.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.957+0000 7f2d977fe700 1 --2- 172.21.15.5:0/1565479107 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b59504d6a0 0x55b59504d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:53.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.957+0000 7f2d977fe700 1 -- 172.21.15.5:0/1565479107 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b595049d00 con 0x55b594ed04a0 2024-05-09T00:59:53.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.958+0000 7f2d977fe700 1 --2- 172.21.15.5:0/1565479107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b594ed04a0 0x55b5950517c0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f2d880aa330 tx=0x7f2d8801a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0beb23e0c58f897 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:59:53.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.958+0000 7f2d7d7fa700 1 -- 172.21.15.5:0/1565479107 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d880aad30 con 0x55b594ed04a0 2024-05-09T00:59:53.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.958+0000 7f2d7d7fa700 1 -- 172.21.15.5:0/1565479107 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d880aa630 con 0x55b594ed04a0 2024-05-09T00:59:53.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.958+0000 7f2d7d7fa700 1 -- 172.21.15.5:0/1565479107 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d880aa890 con 0x55b594ed04a0 2024-05-09T00:59:53.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.959+0000 7f2dac9cdb80 1 -- 172.21.15.5:0/1565479107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b594ed04a0 msgr2=0x55b5950517c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:53.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.959+0000 7f2dac9cdb80 1 --2- 172.21.15.5:0/1565479107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b594ed04a0 0x55b5950517c0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f2d880aa330 tx=0x7f2d8801a040 comp rx=0 tx=0).stop 2024-05-09T00:59:53.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.959+0000 7f2dac9cdb80 1 -- 172.21.15.5:0/1565479107 shutdown_connections 2024-05-09T00:59:53.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.959+0000 7f2dac9cdb80 1 --2- 172.21.15.5:0/1565479107 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b59504d6a0 0x55b59504d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:53.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.959+0000 7f2dac9cdb80 1 --2- 172.21.15.5:0/1565479107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b594ed04a0 0x55b5950517c0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:53.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.959+0000 7f2dac9cdb80 1 --2- 172.21.15.5:0/1565479107 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b594fba160 0x55b595056130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:53.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.959+0000 7f2dac9cdb80 1 -- 172.21.15.5:0/1565479107 >> 172.21.15.5:0/1565479107 conn(0x55b594f306a0 msgr2=0x55b595058860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:59:53.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.959+0000 7f2dac9cdb80 1 -- 172.21.15.5:0/1565479107 shutdown_connections 2024-05-09T00:59:53.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.959+0000 7f2dac9cdb80 1 -- 172.21.15.5:0/1565479107 wait complete. 2024-05-09T00:59:53.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.959+0000 7f2dac9cdb80 1 -- start start 2024-05-09T00:59:53.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.959+0000 7f2dac9cdb80 1 Processor -- start 2024-05-09T00:59:53.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.960+0000 7f2dac9cdb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b594ed04a0 0x55b5950543e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:53.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.960+0000 7f2dac9cdb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b594fba160 0x55b595054920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:53.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.960+0000 7f2dac9cdb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b59504d6a0 0x55b59505d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:53.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.960+0000 7f2dac9cdb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2d981b22d0 con 0x55b594fba160 2024-05-09T00:59:53.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.960+0000 7f2dac9cdb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2d981b1c00 con 0x55b59504d6a0 2024-05-09T00:59:53.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.960+0000 7f2dac9cdb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2d981aee00 con 0x55b594ed04a0 2024-05-09T00:59:53.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.960+0000 7f2dac9cdb80 10 client.0 _ll_register_callbacks cb 0x55b594de16c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:59:53.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.960+0000 7f2d96ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b594fba160 0x55b595054920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:53.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.960+0000 7f2d977fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b594ed04a0 0x55b5950543e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:53.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.960+0000 7f2d96ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b594fba160 0x55b595054920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38010/0 (socket says 172.21.15.5:38010) 2024-05-09T00:59:53.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.960+0000 7f2d96ffd700 1 -- 172.21.15.5:0/1171212629 learned_addr learned my addr 172.21.15.5:0/1171212629 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T00:59:53.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.960+0000 7f2d977fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b594ed04a0 0x55b5950543e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:53208/0 (socket says 172.21.15.5:53208) 2024-05-09T00:59:53.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.960+0000 7f2d97fff700 1 --2- 172.21.15.5:0/1171212629 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b59504d6a0 0x55b59505d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:53.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.960+0000 7f2d977fe700 1 -- 172.21.15.5:0/1171212629 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b59504d6a0 msgr2=0x55b59505d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:59:53.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.960+0000 7f2d977fe700 1 --2- 172.21.15.5:0/1171212629 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b59504d6a0 0x55b59505d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:53.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.960+0000 7f2d977fe700 1 -- 172.21.15.5:0/1171212629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b594fba160 msgr2=0x55b595054920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:59:53.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.960+0000 7f2d977fe700 1 --2- 172.21.15.5:0/1171212629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b594fba160 0x55b595054920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:53.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.960+0000 7f2d977fe700 1 -- 172.21.15.5:0/1171212629 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b595055120 con 0x55b594ed04a0 2024-05-09T00:59:53.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.960+0000 7f2d977fe700 1 --2- 172.21.15.5:0/1171212629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b594ed04a0 0x55b5950543e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2d880e0a80 tx=0x7f2d88092300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d6d04f46b0d3e12 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:59:53.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.960+0000 7f2d76ffd700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-09T00:59:53.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.961+0000 7f2d76ffd700 1 -- 172.21.15.5:0/1171212629 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d880b3400 con 0x55b594ed04a0 2024-05-09T00:59:53.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.961+0000 7f2d76ffd700 1 -- 172.21.15.5:0/1171212629 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f2d880e0b90 con 0x55b594ed04a0 2024-05-09T00:59:53.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.961+0000 7f2dac9cdb80 1 -- 172.21.15.5:0/1171212629 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d880aa630 con 0x55b594ed04a0 2024-05-09T00:59:53.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.961+0000 7f2d76ffd700 1 -- 172.21.15.5:0/1171212629 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2d880aad30 con 0x55b594ed04a0 2024-05-09T00:59:53.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.961+0000 7f2dac9cdb80 10 client.26347 fetch_fsmap learned FSMap version 31 2024-05-09T00:59:53.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.961+0000 7f2dac9cdb80 1 -- 172.21.15.5:0/1171212629 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b5950d1b00 con 0x55b594ed04a0 2024-05-09T00:59:53.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.961+0000 7f2d76ffd700 1 -- 172.21.15.5:0/1171212629 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d880aa630 con 0x55b594ed04a0 2024-05-09T00:59:53.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.961+0000 7f2d76ffd700 1 -- 172.21.15.5:0/1171212629 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f2d88090ba0 con 0x55b594ed04a0 2024-05-09T00:59:53.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.961+0000 7f2dac9cdb80 10 client.26347 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T00:59:53.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.961+0000 7f2dac9cdb80 10 client.26347 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:59:53.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.961+0000 7f2dac9cdb80 20 client.26347 populate_metadata read hostname 'smithi005' 2024-05-09T00:59:53.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.961+0000 7f2dac9cdb80 1 --2- 172.21.15.5:0/1171212629 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55b5950d2190 0x55b5950d45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:59:53.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.961+0000 7f2dac9cdb80 4 client.26347 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:59:53.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.961+0000 7f2dac9cdb80 1 -- 172.21.15.5:0/1171212629 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b594f77a70 con 0x55b5950d2190 2024-05-09T00:59:53.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.961+0000 7f2d96ffd700 1 --2- 172.21.15.5:0/1171212629 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55b5950d2190 0x55b5950d45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:59:53.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.962+0000 7f2d76ffd700 10 client.26347 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T00:59:53.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.962+0000 7f2d96ffd700 1 --2- 172.21.15.5:0/1171212629 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55b5950d2190 0x55b5950d45b0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8912931420a459a server_cookie=611933dbf6d3fbca in_seq=0 out_seq=0 2024-05-09T00:59:53.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.965+0000 7f2d76ffd700 1 -- 172.21.15.5:0/1171212629 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b594f77a70 con 0x55b5950d2190 2024-05-09T00:59:53.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.965+0000 7f2d76ffd700 10 client.26347 handle_command_reply: tid=0 2024-05-09T00:59:53.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.965+0000 7f2dac9cdb80 1 client.26347 shutdown 2024-05-09T00:59:53.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.965+0000 7f2dac9cdb80 1 -- 172.21.15.5:0/1171212629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b594ed04a0 msgr2=0x55b5950543e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:53.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.965+0000 7f2dac9cdb80 1 --2- 172.21.15.5:0/1171212629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b594ed04a0 0x55b5950543e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2d880e0a80 tx=0x7f2d88092300 comp rx=0 tx=0).stop 2024-05-09T00:59:53.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.965+0000 7f2dac9cdb80 1 -- 172.21.15.5:0/1171212629 shutdown_connections 2024-05-09T00:59:53.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.965+0000 7f2dac9cdb80 1 --2- 172.21.15.5:0/1171212629 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55b5950d2190 0x55b5950d45b0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:53.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.965+0000 7f2dac9cdb80 1 --2- 172.21.15.5:0/1171212629 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b59504d6a0 0x55b59505d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:53.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.965+0000 7f2dac9cdb80 1 --2- 172.21.15.5:0/1171212629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b594ed04a0 0x55b5950543e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:53.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.965+0000 7f2dac9cdb80 1 --2- 172.21.15.5:0/1171212629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b594fba160 0x55b595054920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:53.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.965+0000 7f2d76ffd700 0 client.26347 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T00:59:53.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.965+0000 7f2dac9cdb80 1 -- 172.21.15.5:0/1171212629 >> 172.21.15.5:0/1171212629 conn(0x55b594f306a0 msgr2=0x55b595059040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:59:53.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.965+0000 7f2dac9cdb80 1 -- 172.21.15.5:0/1171212629 shutdown_connections 2024-05-09T00:59:53.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.965+0000 7f2dac9cdb80 1 -- 172.21.15.5:0/1171212629 wait complete. 2024-05-09T00:59:53.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.965+0000 7f2dac9cdb80 20 client.26347 trim_cache size 0 max 16384 2024-05-09T00:59:53.971 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-09T00:59:53.971 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T00:59:53.971 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-05-09T00:59:53.971 INFO:teuthology.orchestra.run.smithi005.stdout: "1b545c29-122a-48f2-bd62-6487559191a7": { 2024-05-09T00:59:53.971 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-05-09T00:59:53.971 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "1b545c29-122a-48f2-bd62-6487559191a7", 2024-05-09T00:59:53.971 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-05-09T00:59:53.971 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T00:59:53.971 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T00:59:53.971 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-09T00:59:53.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.966+0000 7f2d9dfb7700 1 -- 172.21.15.5:0/3706434369 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f2d800b2880 msgr2=0x7f2d800b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:53.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.966+0000 7f2d9dfb7700 1 --2- 172.21.15.5:0/3706434369 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f2d800b2880 0x7f2d800b4d90 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f2d98101b40 tx=0x7f2d8c009260 comp rx=0 tx=0).stop 2024-05-09T00:59:53.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.966+0000 7f2d9dfb7700 1 -- 172.21.15.5:0/3706434369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d98103960 msgr2=0x7f2d980fe9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:59:53.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.966+0000 7f2d9dfb7700 1 --2- 172.21.15.5:0/3706434369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d98103960 0x7f2d980fe9c0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f2d88006010 tx=0x7f2d880047b0 comp rx=0 tx=0).stop 2024-05-09T00:59:53.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.966+0000 7f2d9dfb7700 1 -- 172.21.15.5:0/3706434369 shutdown_connections 2024-05-09T00:59:53.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.966+0000 7f2d9dfb7700 1 --2- 172.21.15.5:0/3706434369 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f2d800b2880 0x7f2d800b4d90 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:53.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.966+0000 7f2d9dfb7700 1 --2- 172.21.15.5:0/3706434369 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d98108200 0x7f2d980ff440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:53.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.966+0000 7f2d9dfb7700 1 --2- 172.21.15.5:0/3706434369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d98103960 0x7f2d980fe9c0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:53.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.966+0000 7f2d9dfb7700 1 --2- 172.21.15.5:0/3706434369 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d981042b0 0x7f2d980fef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:59:53.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.966+0000 7f2d9dfb7700 1 -- 172.21.15.5:0/3706434369 >> 172.21.15.5:0/3706434369 conn(0x7f2d980fe4e0 msgr2=0x7f2d98102b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:59:53.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.966+0000 7f2d9dfb7700 1 -- 172.21.15.5:0/3706434369 shutdown_connections 2024-05-09T00:59:53.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T00:59:53.966+0000 7f2d9dfb7700 1 -- 172.21.15.5:0/3706434369 wait complete. 2024-05-09T00:59:54.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:59:53 smithi157 ceph-mon[34294]: pgmap v777: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 808 MiB/s wr, 1.61k op/s 2024-05-09T00:59:54.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:59:53 smithi157 ceph-mon[34294]: scrub summary: idle+waiting paths [/] 2024-05-09T00:59:54.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:59:53 smithi005 ceph-mon[30365]: pgmap v777: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 808 MiB/s wr, 1.61k op/s 2024-05-09T00:59:54.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:59:53 smithi005 ceph-mon[30365]: scrub summary: idle+waiting paths [/] 2024-05-09T00:59:54.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:59:53 smithi131 ceph-mon[34547]: pgmap v777: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 808 MiB/s wr, 1.61k op/s 2024-05-09T00:59:54.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:59:53 smithi131 ceph-mon[34547]: scrub summary: idle+waiting paths [/] 2024-05-09T00:59:54.787 INFO:tasks.cephfs.filesystem:scrub status for tag:1b545c29-122a-48f2-bd62-6487559191a7 - {'path': '/', 'tag': '1b545c29-122a-48f2-bd62-6487559191a7', 'options': 'recursive,force'} 2024-05-09T00:59:56.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:59:55 smithi157 ceph-mon[34294]: pgmap v778: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 733 MiB/s wr, 1.50k op/s 2024-05-09T00:59:56.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:59:55 smithi005 ceph-mon[30365]: pgmap v778: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 733 MiB/s wr, 1.50k op/s 2024-05-09T00:59:56.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:59:55 smithi131 ceph-mon[34547]: pgmap v778: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 733 MiB/s wr, 1.50k op/s 2024-05-09T00:59:57.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:59:56 smithi157 ceph-mon[34294]: scrub summary: idle 2024-05-09T00:59:57.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:59:56 smithi157 ceph-mon[34294]: scrub summary: idle 2024-05-09T00:59:57.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:59:56 smithi005 ceph-mon[30365]: scrub summary: idle 2024-05-09T00:59:57.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:59:56 smithi005 ceph-mon[30365]: scrub summary: idle 2024-05-09T00:59:57.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:59:56 smithi131 ceph-mon[34547]: scrub summary: idle 2024-05-09T00:59:57.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:59:56 smithi131 ceph-mon[34547]: scrub summary: idle 2024-05-09T00:59:58.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:59:57 smithi005 ceph-mon[30365]: pgmap v779: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 603 MiB/s wr, 1.25k op/s 2024-05-09T00:59:58.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:59:57 smithi131 ceph-mon[34547]: pgmap v779: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 603 MiB/s wr, 1.25k op/s 2024-05-09T00:59:58.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:59:57 smithi157 ceph-mon[34294]: pgmap v779: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 603 MiB/s wr, 1.25k op/s 2024-05-09T01:00:00.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 00:59:59 smithi005 ceph-mon[30365]: pgmap v780: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 489 MiB/s wr, 1.03k op/s 2024-05-09T01:00:00.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 00:59:59 smithi131 ceph-mon[34547]: pgmap v780: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 489 MiB/s wr, 1.03k op/s 2024-05-09T01:00:00.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 00:59:59 smithi157 ceph-mon[34294]: pgmap v780: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 489 MiB/s wr, 1.03k op/s 2024-05-09T01:00:01.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:00:00 smithi005 ceph-mon[30365]: overall HEALTH_OK 2024-05-09T01:00:01.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:00:00 smithi131 ceph-mon[34547]: overall HEALTH_OK 2024-05-09T01:00:01.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:00:00 smithi157 ceph-mon[34294]: overall HEALTH_OK 2024-05-09T01:00:02.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:00:01 smithi005 ceph-mon[30365]: pgmap v781: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 295 MiB/s wr, 659 op/s 2024-05-09T01:00:02.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:00:01 smithi131 ceph-mon[34547]: pgmap v781: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 295 MiB/s wr, 659 op/s 2024-05-09T01:00:02.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:00:01 smithi157 ceph-mon[34294]: pgmap v781: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 295 MiB/s wr, 659 op/s 2024-05-09T01:00:03.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.633+0000 7fada67c7640 1 -- 172.21.15.5:0/3944962917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fada0105210 msgr2=0x7fada0105620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:03.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.633+0000 7fada67c7640 1 --2- 172.21.15.5:0/3944962917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fada0105210 0x7fada0105620 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fad90009900 tx=0x7fad9002a410 comp rx=0 tx=0).stop 2024-05-09T01:00:03.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.633+0000 7fada67c7640 1 -- 172.21.15.5:0/3944962917 shutdown_connections 2024-05-09T01:00:03.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.633+0000 7fada67c7640 1 --2- 172.21.15.5:0/3944962917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fada01023c0 0x7fada010be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:03.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.633+0000 7fada67c7640 1 --2- 172.21.15.5:0/3944962917 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fada01018c0 0x7fada0101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:03.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.633+0000 7fada67c7640 1 --2- 172.21.15.5:0/3944962917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fada0105210 0x7fada0105620 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:03.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.633+0000 7fada67c7640 1 -- 172.21.15.5:0/3944962917 >> 172.21.15.5:0/3944962917 conn(0x7fada00fd4c0 msgr2=0x7fada00ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:03.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.633+0000 7fada67c7640 1 -- 172.21.15.5:0/3944962917 shutdown_connections 2024-05-09T01:00:03.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.633+0000 7fada67c7640 1 -- 172.21.15.5:0/3944962917 wait complete. 2024-05-09T01:00:03.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.634+0000 7fada67c7640 1 Processor -- start 2024-05-09T01:00:03.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.634+0000 7fada67c7640 1 -- start start 2024-05-09T01:00:03.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.634+0000 7fada67c7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fada01018c0 0x7fada019a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:03.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.634+0000 7fada67c7640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fada01023c0 0x7fada019a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:03.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.634+0000 7fada67c7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fada0105210 0x7fada01a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:03.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.634+0000 7fada67c7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fada010f780 con 0x7fada0105210 2024-05-09T01:00:03.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.634+0000 7fada67c7640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fada010f600 con 0x7fada01023c0 2024-05-09T01:00:03.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.634+0000 7fada67c7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fada010f900 con 0x7fada01018c0 2024-05-09T01:00:03.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.634+0000 7fada4d64640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fada0105210 0x7fada01a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:03.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.634+0000 7fad9ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fada01018c0 0x7fada019a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:03.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.634+0000 7fada4d64640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fada0105210 0x7fada01a1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34002/0 (socket says 172.21.15.5:34002) 2024-05-09T01:00:03.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.634+0000 7fad9ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fada01018c0 0x7fada019a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:58514/0 (socket says 172.21.15.5:58514) 2024-05-09T01:00:03.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.634+0000 7fada4d64640 1 -- 172.21.15.5:0/1986039779 learned_addr learned my addr 172.21.15.5:0/1986039779 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:00:03.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.634+0000 7fad97fff640 1 --2- 172.21.15.5:0/1986039779 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fada01023c0 0x7fada019a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:03.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.634+0000 7fada4d64640 1 -- 172.21.15.5:0/1986039779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fada01018c0 msgr2=0x7fada019a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:03.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.634+0000 7fada4d64640 1 --2- 172.21.15.5:0/1986039779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fada01018c0 0x7fada019a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:03.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.634+0000 7fada4d64640 1 -- 172.21.15.5:0/1986039779 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fada01023c0 msgr2=0x7fada019a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:03.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.634+0000 7fada4d64640 1 --2- 172.21.15.5:0/1986039779 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fada01023c0 0x7fada019a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:03.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.634+0000 7fada4d64640 1 -- 172.21.15.5:0/1986039779 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fada01a1ec0 con 0x7fada0105210 2024-05-09T01:00:03.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.635+0000 7fada4d64640 1 --2- 172.21.15.5:0/1986039779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fada0105210 0x7fada01a1850 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fad9800eb10 tx=0x7fad9800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:03.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.635+0000 7fad9dffb640 1 -- 172.21.15.5:0/1986039779 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad98004b20 con 0x7fada0105210 2024-05-09T01:00:03.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.635+0000 7fad9dffb640 1 -- 172.21.15.5:0/1986039779 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad98004cc0 con 0x7fada0105210 2024-05-09T01:00:03.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.635+0000 7fad9dffb640 1 -- 172.21.15.5:0/1986039779 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad98010670 con 0x7fada0105210 2024-05-09T01:00:03.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.635+0000 7fada67c7640 1 -- 172.21.15.5:0/1986039779 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fada01a21b0 con 0x7fada0105210 2024-05-09T01:00:03.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.635+0000 7fada67c7640 1 -- 172.21.15.5:0/1986039779 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fada01a26b0 con 0x7fada0105210 2024-05-09T01:00:03.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.638+0000 7fad9dffb640 1 -- 172.21.15.5:0/1986039779 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fad98005020 con 0x7fada0105210 2024-05-09T01:00:03.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.638+0000 7fadad676740 1 Processor -- start 2024-05-09T01:00:03.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.638+0000 7fadad676740 1 -- start start 2024-05-09T01:00:03.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.638+0000 7fadad676740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b1f4a31b0 0x556b1f847500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:03.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.638+0000 7fadad676740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556b1f47a2f0 0x556b1f847a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:03.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.638+0000 7fadad676740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b1f4824d0 0x556b1f84e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:03.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.638+0000 7fadad676740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556b1f7ab7c0 con 0x556b1f4a31b0 2024-05-09T01:00:03.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.638+0000 7fadad676740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x556b1f7470e0 con 0x556b1f4824d0 2024-05-09T01:00:03.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.638+0000 7fadad676740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x556b1f7a7780 con 0x556b1f47a2f0 2024-05-09T01:00:03.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.638+0000 7fad97fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b1f4a31b0 0x556b1f847500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:03.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.638+0000 7fad9ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556b1f47a2f0 0x556b1f847a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:03.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.638+0000 7fad9ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556b1f47a2f0 0x556b1f847a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:58530/0 (socket says 172.21.15.5:58530) 2024-05-09T01:00:03.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.638+0000 7fad9ffff640 1 -- 172.21.15.5:0/3545184863 learned_addr learned my addr 172.21.15.5:0/3545184863 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:00:03.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.638+0000 7fad97fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b1f4a31b0 0x556b1f847500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34006/0 (socket says 172.21.15.5:34006) 2024-05-09T01:00:03.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.638+0000 7fad97fff640 1 --2- 172.21.15.5:0/3545184863 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b1f4824d0 0x556b1f84e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:03.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.638+0000 7fad9ffff640 1 -- 172.21.15.5:0/3545184863 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b1f4824d0 msgr2=0x556b1f84e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:03.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.639+0000 7fad9ffff640 1 --2- 172.21.15.5:0/3545184863 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b1f4824d0 0x556b1f84e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:03.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.639+0000 7fad9ffff640 1 -- 172.21.15.5:0/3545184863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b1f4a31b0 msgr2=0x556b1f847500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:03.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.639+0000 7fad9ffff640 1 --2- 172.21.15.5:0/3545184863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b1f4a31b0 0x556b1f847500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:03.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.639+0000 7fad9ffff640 1 -- 172.21.15.5:0/3545184863 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b1f7707c0 con 0x556b1f47a2f0 2024-05-09T01:00:03.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.639+0000 7fad97fff640 1 --2- 172.21.15.5:0/3545184863 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b1f4824d0 0x556b1f84e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:00:03.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.639+0000 7fad9dffb640 1 --2- 172.21.15.5:0/1986039779 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fad8c0b2860 0x7fad8c0b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:03.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.639+0000 7fad9ffff640 1 --2- 172.21.15.5:0/3545184863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556b1f47a2f0 0x556b1f847a80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fad90009230 tx=0x7fad90031b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c6f157cf01d88ba server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:03.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.639+0000 7fad97fff640 1 --2- 172.21.15.5:0/1986039779 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fad8c0b2860 0x7fad8c0b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:03.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.639+0000 7fad967fc640 1 -- 172.21.15.5:0/3545184863 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad90035030 con 0x556b1f47a2f0 2024-05-09T01:00:03.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.639+0000 7fad9dffb640 1 -- 172.21.15.5:0/1986039779 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fad980e2760 con 0x7fada0105210 2024-05-09T01:00:03.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.639+0000 7fad967fc640 1 -- 172.21.15.5:0/3545184863 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad90040070 con 0x556b1f47a2f0 2024-05-09T01:00:03.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.639+0000 7fad967fc640 1 -- 172.21.15.5:0/3545184863 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad90003950 con 0x556b1f47a2f0 2024-05-09T01:00:03.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.640+0000 7fadad676740 1 -- 172.21.15.5:0/3545184863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556b1f47a2f0 msgr2=0x556b1f847a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:03.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.640+0000 7fad97fff640 1 --2- 172.21.15.5:0/1986039779 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fad8c0b2860 0x7fad8c0b4d70 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fad88005d00 tx=0x7fad8800b670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:03.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.640+0000 7fadad676740 1 --2- 172.21.15.5:0/3545184863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556b1f47a2f0 0x556b1f847a80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fad90009230 tx=0x7fad90031b10 comp rx=0 tx=0).stop 2024-05-09T01:00:03.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.640+0000 7fadad676740 1 -- 172.21.15.5:0/3545184863 shutdown_connections 2024-05-09T01:00:03.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.640+0000 7fadad676740 1 --2- 172.21.15.5:0/3545184863 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b1f4824d0 0x556b1f84e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:03.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.640+0000 7fadad676740 1 --2- 172.21.15.5:0/3545184863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556b1f47a2f0 0x556b1f847a80 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:03.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.640+0000 7fadad676740 1 --2- 172.21.15.5:0/3545184863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b1f4a31b0 0x556b1f847500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:03.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.640+0000 7fadad676740 1 -- 172.21.15.5:0/3545184863 >> 172.21.15.5:0/3545184863 conn(0x556b1f497300 msgr2=0x556b1f842e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:03.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.640+0000 7fadad676740 1 -- 172.21.15.5:0/3545184863 shutdown_connections 2024-05-09T01:00:03.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.640+0000 7fadad676740 1 -- 172.21.15.5:0/3545184863 wait complete. 2024-05-09T01:00:03.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.640+0000 7fadad676740 1 -- start start 2024-05-09T01:00:03.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.641+0000 7fadad676740 1 Processor -- start 2024-05-09T01:00:03.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.641+0000 7fadad676740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b1f47a2f0 0x556b1f851180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:03.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.641+0000 7fadad676740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556b1f4824d0 0x556b1f8516c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:03.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.641+0000 7fadad676740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b1f4a31b0 0x556b1f858620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:03.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.641+0000 7fadad676740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556b1f84f370 con 0x556b1f4a31b0 2024-05-09T01:00:03.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.641+0000 7fadad676740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x556b1f7329c0 con 0x556b1f47a2f0 2024-05-09T01:00:03.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.641+0000 7fadad676740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x556b1f84f4f0 con 0x556b1f4824d0 2024-05-09T01:00:03.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.641+0000 7fadad676740 10 client.0 _ll_register_callbacks cb 0x556b1f75f020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:00:03.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.641+0000 7fada4d64640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b1f4a31b0 0x556b1f858620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:03.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.641+0000 7fad97fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556b1f4824d0 0x556b1f8516c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:03.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.641+0000 7fada4d64640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b1f4a31b0 0x556b1f858620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34008/0 (socket says 172.21.15.5:34008) 2024-05-09T01:00:03.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.641+0000 7fada4d64640 1 -- 172.21.15.5:0/3442904100 learned_addr learned my addr 172.21.15.5:0/3442904100 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:00:03.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.641+0000 7fad9ffff640 1 --2- 172.21.15.5:0/3442904100 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b1f47a2f0 0x556b1f851180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:03.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.641+0000 7fad97fff640 1 -- 172.21.15.5:0/3442904100 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b1f47a2f0 msgr2=0x556b1f851180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:03.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.641+0000 7fad97fff640 1 --2- 172.21.15.5:0/3442904100 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b1f47a2f0 0x556b1f851180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:03.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.642+0000 7fad97fff640 1 -- 172.21.15.5:0/3442904100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b1f4a31b0 msgr2=0x556b1f858620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:03.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.642+0000 7fad97fff640 1 --2- 172.21.15.5:0/3442904100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b1f4a31b0 0x556b1f858620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:03.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.642+0000 7fad97fff640 1 -- 172.21.15.5:0/3442904100 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b1f7591f0 con 0x556b1f4824d0 2024-05-09T01:00:03.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.642+0000 7fada4d64640 1 --2- 172.21.15.5:0/3442904100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b1f4a31b0 0x556b1f858620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:00:03.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.642+0000 7fad97fff640 1 --2- 172.21.15.5:0/3442904100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556b1f4824d0 0x556b1f8516c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x556b1f8486d0 tx=0x7fad88004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7134e2d1495c816d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:03.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.642+0000 7fad7f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-09T01:00:03.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.642+0000 7fad7f7fe640 1 -- 172.21.15.5:0/3442904100 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad880035f0 con 0x556b1f4824d0 2024-05-09T01:00:03.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.642+0000 7fadad676740 1 -- 172.21.15.5:0/3442904100 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad90040070 con 0x556b1f4824d0 2024-05-09T01:00:03.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.642+0000 7fad7f7fe640 1 -- 172.21.15.5:0/3442904100 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fad88003790 con 0x556b1f4824d0 2024-05-09T01:00:03.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.642+0000 7fad7f7fe640 1 -- 172.21.15.5:0/3442904100 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fad8802f050 con 0x556b1f4824d0 2024-05-09T01:00:03.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.642+0000 7fadad676740 10 client.26359 fetch_fsmap learned FSMap version 31 2024-05-09T01:00:03.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.642+0000 7fadad676740 1 -- 172.21.15.5:0/3442904100 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556b1f50e790 con 0x556b1f4824d0 2024-05-09T01:00:03.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.642+0000 7fad7f7fe640 1 -- 172.21.15.5:0/3442904100 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad90040070 con 0x556b1f4824d0 2024-05-09T01:00:03.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.643+0000 7fad7f7fe640 1 -- 172.21.15.5:0/3442904100 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fad88010b90 con 0x556b1f4824d0 2024-05-09T01:00:03.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.643+0000 7fadad676740 10 client.26359 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T01:00:03.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.643+0000 7fadad676740 10 client.26359 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T01:00:03.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.643+0000 7fadad676740 20 client.26359 populate_metadata read hostname 'smithi005' 2024-05-09T01:00:03.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.643+0000 7fadad676740 1 --2- 172.21.15.5:0/3442904100 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x556b1f461d80 0x556b1f85eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:03.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.643+0000 7fadad676740 4 client.26359 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:00:03.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.643+0000 7fadad676740 1 -- 172.21.15.5:0/3442904100 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b1f7b93d0 con 0x556b1f461d80 2024-05-09T01:00:03.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.643+0000 7fad9ffff640 1 --2- 172.21.15.5:0/3442904100 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x556b1f461d80 0x556b1f85eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:03.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.644+0000 7fad9ffff640 1 --2- 172.21.15.5:0/3442904100 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x556b1f461d80 0x556b1f85eba0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44414622a1c33a15 server_cookie=da465b349905ecb5 in_seq=0 out_seq=0 2024-05-09T01:00:03.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.644+0000 7fad7f7fe640 10 client.26359 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T01:00:03.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.645+0000 7fad7f7fe640 1 -- 172.21.15.5:0/3442904100 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556b1f7b93d0 con 0x556b1f461d80 2024-05-09T01:00:03.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.645+0000 7fad7f7fe640 10 client.26359 handle_command_reply: tid=0 2024-05-09T01:00:03.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.645+0000 7fadad676740 1 client.26359 shutdown 2024-05-09T01:00:03.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.646+0000 7fadad676740 1 -- 172.21.15.5:0/3442904100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556b1f4824d0 msgr2=0x556b1f8516c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:03.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.646+0000 7fadad676740 1 --2- 172.21.15.5:0/3442904100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556b1f4824d0 0x556b1f8516c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x556b1f8486d0 tx=0x7fad88004730 comp rx=0 tx=0).stop 2024-05-09T01:00:03.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.646+0000 7fadad676740 1 -- 172.21.15.5:0/3442904100 shutdown_connections 2024-05-09T01:00:03.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.646+0000 7fadad676740 1 --2- 172.21.15.5:0/3442904100 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x556b1f461d80 0x556b1f85eba0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:03.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.646+0000 7fadad676740 1 --2- 172.21.15.5:0/3442904100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b1f4a31b0 0x556b1f858620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:03.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.646+0000 7fadad676740 1 --2- 172.21.15.5:0/3442904100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556b1f4824d0 0x556b1f8516c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:03.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.646+0000 7fad7f7fe640 0 client.26359 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T01:00:03.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.646+0000 7fadad676740 1 --2- 172.21.15.5:0/3442904100 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b1f47a2f0 0x556b1f851180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:03.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.646+0000 7fadad676740 1 -- 172.21.15.5:0/3442904100 >> 172.21.15.5:0/3442904100 conn(0x556b1f497300 msgr2=0x556b1f841700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:03.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.646+0000 7fadad676740 1 -- 172.21.15.5:0/3442904100 shutdown_connections 2024-05-09T01:00:03.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.646+0000 7fadad676740 1 -- 172.21.15.5:0/3442904100 wait complete. 2024-05-09T01:00:03.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.646+0000 7fadad676740 20 client.26359 trim_cache size 0 max 16384 2024-05-09T01:00:03.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.661+0000 7fadad676740 1 Processor -- start 2024-05-09T01:00:03.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.661+0000 7fadad676740 1 -- start start 2024-05-09T01:00:03.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.662+0000 7fadad676740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556b1f47a2f0 0x556b1f863390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:03.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.662+0000 7fadad676740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b1f4824d0 0x556b1f858540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:03.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.662+0000 7fadad676740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b1f4a31b0 0x556b1f858a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:03.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.662+0000 7fadad676740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fada01ab530 con 0x556b1f4a31b0 2024-05-09T01:00:03.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.662+0000 7fadad676740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fada01ad070 con 0x556b1f4824d0 2024-05-09T01:00:03.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.662+0000 7fadad676740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fada01abc00 con 0x556b1f47a2f0 2024-05-09T01:00:03.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.662+0000 7fada4d64640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b1f4a31b0 0x556b1f858a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:03.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.662+0000 7fad9ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556b1f47a2f0 0x556b1f863390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:03.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.662+0000 7fada4d64640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b1f4a31b0 0x556b1f858a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34010/0 (socket says 172.21.15.5:34010) 2024-05-09T01:00:03.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.662+0000 7fad9ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556b1f47a2f0 0x556b1f863390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:58552/0 (socket says 172.21.15.5:58552) 2024-05-09T01:00:03.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.662+0000 7fada4d64640 1 -- 172.21.15.5:0/501370784 learned_addr learned my addr 172.21.15.5:0/501370784 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:00:03.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.662+0000 7fad97fff640 1 --2- 172.21.15.5:0/501370784 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b1f4824d0 0x556b1f858540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:03.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.662+0000 7fada4d64640 1 -- 172.21.15.5:0/501370784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556b1f47a2f0 msgr2=0x556b1f863390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:03.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.662+0000 7fada4d64640 1 --2- 172.21.15.5:0/501370784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556b1f47a2f0 0x556b1f863390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:03.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.662+0000 7fada4d64640 1 -- 172.21.15.5:0/501370784 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b1f4824d0 msgr2=0x556b1f858540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:03.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.662+0000 7fada4d64640 1 --2- 172.21.15.5:0/501370784 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b1f4824d0 0x556b1f858540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:03.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.662+0000 7fada4d64640 1 -- 172.21.15.5:0/501370784 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b1f5855f0 con 0x556b1f4a31b0 2024-05-09T01:00:03.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.662+0000 7fad97fff640 1 --2- 172.21.15.5:0/501370784 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b1f4824d0 0x556b1f858540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:00:03.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.662+0000 7fada4d64640 1 --2- 172.21.15.5:0/501370784 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b1f4a31b0 0x556b1f858a80 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fad980e05a0 tx=0x7fad980a9b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0b08babea842626 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:03.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.663+0000 7fad95ffb640 1 -- 172.21.15.5:0/501370784 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad980c4430 con 0x556b1f4a31b0 2024-05-09T01:00:03.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.663+0000 7fad95ffb640 1 -- 172.21.15.5:0/501370784 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad980aa450 con 0x556b1f4a31b0 2024-05-09T01:00:03.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.663+0000 7fad95ffb640 1 -- 172.21.15.5:0/501370784 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad980b2720 con 0x556b1f4a31b0 2024-05-09T01:00:03.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.663+0000 7fadad676740 1 -- 172.21.15.5:0/501370784 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b1f4a31b0 msgr2=0x556b1f858a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:03.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.663+0000 7fadad676740 1 --2- 172.21.15.5:0/501370784 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b1f4a31b0 0x556b1f858a80 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fad980e05a0 tx=0x7fad980a9b20 comp rx=0 tx=0).stop 2024-05-09T01:00:03.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.663+0000 7fadad676740 1 -- 172.21.15.5:0/501370784 shutdown_connections 2024-05-09T01:00:03.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.663+0000 7fadad676740 1 --2- 172.21.15.5:0/501370784 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b1f4a31b0 0x556b1f858a80 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:03.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.663+0000 7fadad676740 1 --2- 172.21.15.5:0/501370784 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b1f4824d0 0x556b1f858540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:03.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.663+0000 7fadad676740 1 --2- 172.21.15.5:0/501370784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556b1f47a2f0 0x556b1f863390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:03.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.663+0000 7fadad676740 1 -- 172.21.15.5:0/501370784 >> 172.21.15.5:0/501370784 conn(0x556b1f497300 msgr2=0x556b1f84b640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:03.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.663+0000 7fadad676740 1 -- 172.21.15.5:0/501370784 shutdown_connections 2024-05-09T01:00:03.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.663+0000 7fadad676740 1 -- 172.21.15.5:0/501370784 wait complete. 2024-05-09T01:00:03.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.664+0000 7fadad676740 1 -- start start 2024-05-09T01:00:03.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.664+0000 7fadad676740 1 Processor -- start 2024-05-09T01:00:03.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.664+0000 7fadad676740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556b1f47a2f0 0x556b1f855ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:03.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.664+0000 7fadad676740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b1f4824d0 0x556b1f856400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:03.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.664+0000 7fadad676740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b1f4a31b0 0x556b1f857330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:03.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.664+0000 7fadad676740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fada01ac9a0 con 0x556b1f4a31b0 2024-05-09T01:00:03.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.664+0000 7fadad676740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fada01ac2d0 con 0x556b1f4824d0 2024-05-09T01:00:03.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.664+0000 7fadad676740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fada01a9510 con 0x556b1f47a2f0 2024-05-09T01:00:03.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.664+0000 7fadad676740 10 client.0 _ll_register_callbacks cb 0x7fad8800b8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:00:03.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.664+0000 7fada4d64640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b1f4a31b0 0x556b1f857330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:03.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.664+0000 7fad9ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556b1f47a2f0 0x556b1f855ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:03.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.664+0000 7fada4d64640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b1f4a31b0 0x556b1f857330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34012/0 (socket says 172.21.15.5:34012) 2024-05-09T01:00:03.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.664+0000 7fada4d64640 1 -- 172.21.15.5:0/2207379316 learned_addr learned my addr 172.21.15.5:0/2207379316 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:00:03.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.665+0000 7fad97fff640 1 --2- 172.21.15.5:0/2207379316 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b1f4824d0 0x556b1f856400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:03.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.665+0000 7fada4d64640 1 -- 172.21.15.5:0/2207379316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556b1f47a2f0 msgr2=0x556b1f855ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:03.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.665+0000 7fada4d64640 1 --2- 172.21.15.5:0/2207379316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556b1f47a2f0 0x556b1f855ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:03.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.665+0000 7fada4d64640 1 -- 172.21.15.5:0/2207379316 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b1f4824d0 msgr2=0x556b1f856400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:03.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.665+0000 7fada4d64640 1 --2- 172.21.15.5:0/2207379316 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b1f4824d0 0x556b1f856400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:03.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.665+0000 7fada4d64640 1 -- 172.21.15.5:0/2207379316 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b1f76fed0 con 0x556b1f4a31b0 2024-05-09T01:00:03.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.665+0000 7fad9ffff640 1 --2- 172.21.15.5:0/2207379316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556b1f47a2f0 0x556b1f855ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:00:03.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.665+0000 7fada4d64640 1 --2- 172.21.15.5:0/2207379316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b1f4a31b0 0x556b1f857330 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fad980a5dd0 tx=0x7fad980aa250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db11655939e84e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:03.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.665+0000 7fad7f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T01:00:03.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.665+0000 7fad7f7fe640 1 -- 172.21.15.5:0/2207379316 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad98088070 con 0x556b1f4a31b0 2024-05-09T01:00:03.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.665+0000 7fad7f7fe640 1 -- 172.21.15.5:0/2207379316 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fad980b2930 con 0x556b1f4a31b0 2024-05-09T01:00:03.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.665+0000 7fadad676740 1 -- 172.21.15.5:0/2207379316 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad980aa450 con 0x556b1f4a31b0 2024-05-09T01:00:03.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.665+0000 7fad7f7fe640 1 -- 172.21.15.5:0/2207379316 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fad980c4430 con 0x556b1f4a31b0 2024-05-09T01:00:03.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.665+0000 7fadad676740 10 client.16965 fetch_fsmap learned FSMap version 31 2024-05-09T01:00:03.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.665+0000 7fadad676740 1 -- 172.21.15.5:0/2207379316 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556b1f858920 con 0x556b1f4a31b0 2024-05-09T01:00:03.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.666+0000 7fad7f7fe640 1 -- 172.21.15.5:0/2207379316 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad980aa450 con 0x556b1f4a31b0 2024-05-09T01:00:03.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.666+0000 7fad7f7fe640 1 -- 172.21.15.5:0/2207379316 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fad980a8a20 con 0x556b1f4a31b0 2024-05-09T01:00:03.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.666+0000 7fadad676740 10 client.16965 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T01:00:03.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.666+0000 7fadad676740 10 client.16965 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T01:00:03.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.666+0000 7fadad676740 20 client.16965 populate_metadata read hostname 'smithi005' 2024-05-09T01:00:03.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.666+0000 7fadad676740 1 --2- 172.21.15.5:0/2207379316 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x556b1f4371c0 0x556b1f8c8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:03.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.666+0000 7fadad676740 4 client.16965 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T01:00:03.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.666+0000 7fadad676740 1 -- 172.21.15.5:0/2207379316 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556b1f7b3240 con 0x556b1f4371c0 2024-05-09T01:00:03.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.666+0000 7fad9ffff640 1 --2- 172.21.15.5:0/2207379316 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x556b1f4371c0 0x556b1f8c8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:03.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.666+0000 7fad7f7fe640 10 client.16965 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T01:00:03.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:03.666+0000 7fad9ffff640 1 --2- 172.21.15.5:0/2207379316 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x556b1f4371c0 0x556b1f8c8f60 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f07ebdde5fd193e server_cookie=e65247e6ae977a84 in_seq=0 out_seq=0 2024-05-09T01:00:04.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:00:03 smithi005 ceph-mon[30365]: pgmap v782: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 160 MiB/s wr, 378 op/s 2024-05-09T01:00:04.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:00:03 smithi131 ceph-mon[34547]: pgmap v782: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 160 MiB/s wr, 378 op/s 2024-05-09T01:00:04.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:00:03 smithi157 ceph-mon[34294]: pgmap v782: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 160 MiB/s wr, 378 op/s 2024-05-09T01:00:06.237 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:00:05 smithi157 ceph-mon[34294]: pgmap v783: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 111 MiB/s wr, 292 op/s 2024-05-09T01:00:06.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:00:05 smithi005 ceph-mon[30365]: pgmap v783: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 111 MiB/s wr, 292 op/s 2024-05-09T01:00:06.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:00:05 smithi131 ceph-mon[34547]: pgmap v783: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 111 MiB/s wr, 292 op/s 2024-05-09T01:00:08.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:00:07 smithi005 ceph-mon[30365]: pgmap v784: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 86 MiB/s wr, 252 op/s 2024-05-09T01:00:08.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:00:07 smithi131 ceph-mon[34547]: pgmap v784: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 86 MiB/s wr, 252 op/s 2024-05-09T01:00:08.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:00:07 smithi157 ceph-mon[34294]: pgmap v784: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 86 MiB/s wr, 252 op/s 2024-05-09T01:00:10.293 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:00:09 smithi005 ceph-mon[30365]: pgmap v785: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 91 MiB/s wr, 294 op/s 2024-05-09T01:00:10.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:00:09 smithi131 ceph-mon[34547]: pgmap v785: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 91 MiB/s wr, 294 op/s 2024-05-09T01:00:10.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:10.396+0000 7fad7f7fe640 1 -- 172.21.15.5:0/2207379316 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556b1f7b3240 con 0x556b1f4371c0 2024-05-09T01:00:10.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:10.396+0000 7fad7f7fe640 10 client.16965 handle_command_reply: tid=0 2024-05-09T01:00:10.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:10.396+0000 7fadad676740 1 client.16965 shutdown 2024-05-09T01:00:10.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:10.396+0000 7fadad676740 1 -- 172.21.15.5:0/2207379316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b1f4a31b0 msgr2=0x556b1f857330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:10.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:10.396+0000 7fadad676740 1 --2- 172.21.15.5:0/2207379316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b1f4a31b0 0x556b1f857330 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fad980a5dd0 tx=0x7fad980aa250 comp rx=0 tx=0).stop 2024-05-09T01:00:10.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:10.397+0000 7fadad676740 1 -- 172.21.15.5:0/2207379316 shutdown_connections 2024-05-09T01:00:10.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:10.397+0000 7fadad676740 1 --2- 172.21.15.5:0/2207379316 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x556b1f4371c0 0x556b1f8c8f60 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:10.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:10.397+0000 7fadad676740 1 --2- 172.21.15.5:0/2207379316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b1f4a31b0 0x556b1f857330 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:10.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:10.397+0000 7fadad676740 1 --2- 172.21.15.5:0/2207379316 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b1f4824d0 0x556b1f856400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:10.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:10.397+0000 7fadad676740 1 --2- 172.21.15.5:0/2207379316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556b1f47a2f0 0x556b1f855ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:10.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:10.397+0000 7fad7f7fe640 0 client.16965 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T01:00:10.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:10.397+0000 7fadad676740 1 -- 172.21.15.5:0/2207379316 >> 172.21.15.5:0/2207379316 conn(0x556b1f497300 msgr2=0x556b1f845c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:10.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:10.397+0000 7fadad676740 1 -- 172.21.15.5:0/2207379316 shutdown_connections 2024-05-09T01:00:10.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:10.397+0000 7fadad676740 1 -- 172.21.15.5:0/2207379316 wait complete. 2024-05-09T01:00:10.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:10.397+0000 7fadad676740 20 client.16965 trim_cache size 0 max 16384 2024-05-09T01:00:10.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-09T01:00:10.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-09T01:00:10.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-09T01:00:10.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:10.398+0000 7fada67c7640 1 -- 172.21.15.5:0/1986039779 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fad8c0b2860 msgr2=0x7fad8c0b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:10.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:10.398+0000 7fada67c7640 1 --2- 172.21.15.5:0/1986039779 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fad8c0b2860 0x7fad8c0b4d70 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fad88005d00 tx=0x7fad8800b670 comp rx=0 tx=0).stop 2024-05-09T01:00:10.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:10.398+0000 7fada67c7640 1 -- 172.21.15.5:0/1986039779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fada0105210 msgr2=0x7fada01a1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:10.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:10.398+0000 7fada67c7640 1 --2- 172.21.15.5:0/1986039779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fada0105210 0x7fada01a1850 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fad9800eb10 tx=0x7fad9800c2d0 comp rx=0 tx=0).stop 2024-05-09T01:00:10.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:10.398+0000 7fada67c7640 1 -- 172.21.15.5:0/1986039779 shutdown_connections 2024-05-09T01:00:10.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:10.398+0000 7fada67c7640 1 --2- 172.21.15.5:0/1986039779 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fad8c0b2860 0x7fad8c0b4d70 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:10.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:10.398+0000 7fada67c7640 1 --2- 172.21.15.5:0/1986039779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fada0105210 0x7fada01a1850 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:10.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:10.398+0000 7fada67c7640 1 --2- 172.21.15.5:0/1986039779 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fada01023c0 0x7fada019a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:10.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:10.398+0000 7fada67c7640 1 --2- 172.21.15.5:0/1986039779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fada01018c0 0x7fada019a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:10.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:10.398+0000 7fada67c7640 1 -- 172.21.15.5:0/1986039779 >> 172.21.15.5:0/1986039779 conn(0x7fada00fd4c0 msgr2=0x7fada0105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:10.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:10.398+0000 7fada67c7640 1 -- 172.21.15.5:0/1986039779 shutdown_connections 2024-05-09T01:00:10.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:10.398+0000 7fada67c7640 1 -- 172.21.15.5:0/1986039779 wait complete. 2024-05-09T01:00:10.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:00:09 smithi157 ceph-mon[34294]: pgmap v785: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 91 MiB/s wr, 294 op/s 2024-05-09T01:00:11.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:00:10 smithi005 ceph-mon[30365]: client.15198 does not advance its oldest_flush_tid (533730), 100000 completed flushes recorded in session 2024-05-09T01:00:11.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:00:10 smithi131 ceph-mon[34547]: client.15198 does not advance its oldest_flush_tid (533730), 100000 completed flushes recorded in session 2024-05-09T01:00:11.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:00:10 smithi157 ceph-mon[34294]: client.15198 does not advance its oldest_flush_tid (533730), 100000 completed flushes recorded in session 2024-05-09T01:00:12.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:00:11 smithi005 ceph-mon[30365]: pgmap v786: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 88 MiB/s wr, 314 op/s 2024-05-09T01:00:12.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:00:11 smithi131 ceph-mon[34547]: pgmap v786: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 88 MiB/s wr, 314 op/s 2024-05-09T01:00:12.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:00:11 smithi157 ceph-mon[34294]: pgmap v786: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 88 MiB/s wr, 314 op/s 2024-05-09T01:00:14.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:00:13 smithi005 ceph-mon[30365]: pgmap v787: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 84 MiB/s wr, 316 op/s 2024-05-09T01:00:14.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:00:13 smithi131 ceph-mon[34547]: pgmap v787: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 84 MiB/s wr, 316 op/s 2024-05-09T01:00:14.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:00:13 smithi157 ceph-mon[34294]: pgmap v787: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 84 MiB/s wr, 316 op/s 2024-05-09T01:00:15.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:00:15 smithi005 ceph-mon[30365]: Health check failed: 1 clients failing to advance oldest client/flush tid (MDS_CLIENT_OLDEST_TID) 2024-05-09T01:00:15.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:00:15 smithi005 ceph-mon[30365]: mds.? [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] up:active 2024-05-09T01:00:15.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:00:15 smithi005 ceph-mon[30365]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T01:00:15.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:00:15 smithi131 ceph-mon[34547]: Health check failed: 1 clients failing to advance oldest client/flush tid (MDS_CLIENT_OLDEST_TID) 2024-05-09T01:00:15.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:00:15 smithi131 ceph-mon[34547]: mds.? [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] up:active 2024-05-09T01:00:15.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:00:15 smithi131 ceph-mon[34547]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T01:00:15.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:00:15 smithi157 ceph-mon[34294]: Health check failed: 1 clients failing to advance oldest client/flush tid (MDS_CLIENT_OLDEST_TID) 2024-05-09T01:00:15.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:00:15 smithi157 ceph-mon[34294]: mds.? [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] up:active 2024-05-09T01:00:15.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:00:15 smithi157 ceph-mon[34294]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T01:00:16.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:00:16 smithi005 ceph-mon[30365]: pgmap v788: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 89 MiB/s wr, 346 op/s 2024-05-09T01:00:16.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:00:16 smithi131 ceph-mon[34547]: pgmap v788: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 89 MiB/s wr, 346 op/s 2024-05-09T01:00:16.438 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:00:16 smithi157 ceph-mon[34294]: pgmap v788: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 89 MiB/s wr, 346 op/s 2024-05-09T01:00:18.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:00:18 smithi005 ceph-mon[30365]: pgmap v789: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 86 MiB/s wr, 330 op/s 2024-05-09T01:00:18.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:00:18 smithi131 ceph-mon[34547]: pgmap v789: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 86 MiB/s wr, 330 op/s 2024-05-09T01:00:18.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:00:18 smithi157 ceph-mon[34294]: pgmap v789: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 86 MiB/s wr, 330 op/s 2024-05-09T01:00:20.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:00:20 smithi005 ceph-mon[30365]: pgmap v790: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 93 MiB/s wr, 380 op/s 2024-05-09T01:00:20.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:00:20 smithi131 ceph-mon[34547]: pgmap v790: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 93 MiB/s wr, 380 op/s 2024-05-09T01:00:20.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:00:20 smithi157 ceph-mon[34294]: pgmap v790: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 93 MiB/s wr, 380 op/s 2024-05-09T01:00:22.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:00:22 smithi005 ceph-mon[30365]: pgmap v791: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 88 MiB/s wr, 407 op/s 2024-05-09T01:00:22.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:00:22 smithi131 ceph-mon[34547]: pgmap v791: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 88 MiB/s wr, 407 op/s 2024-05-09T01:00:22.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:00:22 smithi157 ceph-mon[34294]: pgmap v791: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 88 MiB/s wr, 407 op/s 2024-05-09T01:00:23.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.504+0000 7f74f7829640 1 -- 172.21.15.5:0/1275179697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74f0109a20 msgr2=0x7f74f010be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:23.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.504+0000 7f74f7829640 1 --2- 172.21.15.5:0/1275179697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74f0109a20 0x7f74f010be40 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f74e8009ca0 tx=0x7f74e802a8c0 comp rx=0 tx=0).stop 2024-05-09T01:00:23.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.504+0000 7f74f7829640 1 -- 172.21.15.5:0/1275179697 shutdown_connections 2024-05-09T01:00:23.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.504+0000 7f74f7829640 1 --2- 172.21.15.5:0/1275179697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74f0109a20 0x7f74f010be40 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:23.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.504+0000 7f74f7829640 1 --2- 172.21.15.5:0/1275179697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74f01070c0 0x7f74f01094e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:23.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.504+0000 7f74f7829640 1 --2- 172.21.15.5:0/1275179697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74f0104760 0x7f74f0106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:23.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.504+0000 7f74f7829640 1 -- 172.21.15.5:0/1275179697 >> 172.21.15.5:0/1275179697 conn(0x7f74f00fdd60 msgr2=0x7f74f0100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:23.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.505+0000 7f74f7829640 1 -- 172.21.15.5:0/1275179697 shutdown_connections 2024-05-09T01:00:23.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.505+0000 7f74f7829640 1 -- 172.21.15.5:0/1275179697 wait complete. 2024-05-09T01:00:23.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.505+0000 7f74f7829640 1 Processor -- start 2024-05-09T01:00:23.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.505+0000 7f74f7829640 1 -- start start 2024-05-09T01:00:23.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.505+0000 7f74f7829640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74f0104760 0x7f74f0103bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:23.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.505+0000 7f74f7829640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74f01070c0 0x7f74f0102250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:23.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.505+0000 7f74f7829640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74f0109a20 0x7f74f0102790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:23.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.505+0000 7f74f7829640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f74f010f7f0 con 0x7f74f0104760 2024-05-09T01:00:23.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.505+0000 7f74f7829640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f74f010f670 con 0x7f74f0109a20 2024-05-09T01:00:23.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.505+0000 7f74f7829640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f74f010f970 con 0x7f74f01070c0 2024-05-09T01:00:23.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.506+0000 7f74f55c5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74f0104760 0x7f74f0103bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:23.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.506+0000 7f74f55c5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74f0104760 0x7f74f0103bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46454/0 (socket says 172.21.15.5:46454) 2024-05-09T01:00:23.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.506+0000 7f74f55c5640 1 -- 172.21.15.5:0/359922581 learned_addr learned my addr 172.21.15.5:0/359922581 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:00:23.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.506+0000 7f74f4dc4640 1 --2- 172.21.15.5:0/359922581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74f01070c0 0x7f74f0102250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:23.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.506+0000 7f74f5dc6640 1 --2- 172.21.15.5:0/359922581 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74f0109a20 0x7f74f0102790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:23.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.506+0000 7f74f55c5640 1 -- 172.21.15.5:0/359922581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74f01070c0 msgr2=0x7f74f0102250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:23.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.506+0000 7f74f55c5640 1 --2- 172.21.15.5:0/359922581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74f01070c0 0x7f74f0102250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:23.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.506+0000 7f74f55c5640 1 -- 172.21.15.5:0/359922581 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74f0109a20 msgr2=0x7f74f0102790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:23.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.506+0000 7f74f55c5640 1 --2- 172.21.15.5:0/359922581 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74f0109a20 0x7f74f0102790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:23.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.506+0000 7f74f55c5640 1 -- 172.21.15.5:0/359922581 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74f0102ec0 con 0x7f74f0104760 2024-05-09T01:00:23.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.506+0000 7f74f55c5640 1 --2- 172.21.15.5:0/359922581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74f0104760 0x7f74f0103bb0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f74ec00e9f0 tx=0x7f74ec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:23.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.506+0000 7f74e67fc640 1 -- 172.21.15.5:0/359922581 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ec00ccc0 con 0x7f74f0104760 2024-05-09T01:00:23.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.506+0000 7f74e67fc640 1 -- 172.21.15.5:0/359922581 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74ec0054d0 con 0x7f74f0104760 2024-05-09T01:00:23.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.506+0000 7f74e67fc640 1 -- 172.21.15.5:0/359922581 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ec010430 con 0x7f74f0104760 2024-05-09T01:00:23.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.506+0000 7f74f7829640 1 -- 172.21.15.5:0/359922581 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74f01a43d0 con 0x7f74f0104760 2024-05-09T01:00:23.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.506+0000 7f74f7829640 1 -- 172.21.15.5:0/359922581 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74f01a4840 con 0x7f74f0104760 2024-05-09T01:00:23.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.509+0000 7f74e67fc640 1 -- 172.21.15.5:0/359922581 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f74ec0046c0 con 0x7f74f0104760 2024-05-09T01:00:23.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.509+0000 7f74fe7b3740 1 Processor -- start 2024-05-09T01:00:23.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.509+0000 7f74fe7b3740 1 -- start start 2024-05-09T01:00:23.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.509+0000 7f74fe7b3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56234d3521b0 0x56234d6f6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:23.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.509+0000 7f74fe7b3740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56234d3292f0 0x56234d6f6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:23.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.509+0000 7f74fe7b3740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56234d3314d0 0x56234d6fd980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:23.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.509+0000 7f74fe7b3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56234d65a7c0 con 0x56234d3521b0 2024-05-09T01:00:23.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.509+0000 7f74fe7b3740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x56234d5f60e0 con 0x56234d3314d0 2024-05-09T01:00:23.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.509+0000 7f74fe7b3740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56234d656780 con 0x56234d3292f0 2024-05-09T01:00:23.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.510+0000 7f74f5dc6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56234d3521b0 0x56234d6f6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:23.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.510+0000 7f74f55c5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56234d3292f0 0x56234d6f6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:23.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.510+0000 7f74f5dc6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56234d3521b0 0x56234d6f6500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46468/0 (socket says 172.21.15.5:46468) 2024-05-09T01:00:23.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.510+0000 7f74f5dc6640 1 -- 172.21.15.5:0/3545815410 learned_addr learned my addr 172.21.15.5:0/3545815410 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:00:23.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.510+0000 7f74f4dc4640 1 --2- 172.21.15.5:0/3545815410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56234d3314d0 0x56234d6fd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:23.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.510+0000 7f74f5dc6640 1 -- 172.21.15.5:0/3545815410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56234d3292f0 msgr2=0x56234d6f6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:23.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.510+0000 7f74f5dc6640 1 --2- 172.21.15.5:0/3545815410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56234d3292f0 0x56234d6f6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:23.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.510+0000 7f74f5dc6640 1 -- 172.21.15.5:0/3545815410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56234d3314d0 msgr2=0x56234d6fd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:23.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.510+0000 7f74f5dc6640 1 --2- 172.21.15.5:0/3545815410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56234d3314d0 0x56234d6fd980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:23.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.510+0000 7f74f5dc6640 1 -- 172.21.15.5:0/3545815410 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56234d61f7c0 con 0x56234d3521b0 2024-05-09T01:00:23.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.510+0000 7f74e67fc640 1 --2- 172.21.15.5:0/359922581 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f74dc0b2860 0x7f74dc0b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:23.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.510+0000 7f74f5dc6640 1 --2- 172.21.15.5:0/3545815410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56234d3521b0 0x56234d6f6500 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f74f0075fe0 tx=0x7f74e802f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1c0414842fec292 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:23.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.510+0000 7f74f55c5640 1 --2- 172.21.15.5:0/359922581 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f74dc0b2860 0x7f74dc0b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:23.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.510+0000 7f74e67fc640 1 -- 172.21.15.5:0/359922581 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f74ec0e25a0 con 0x7f74f0104760 2024-05-09T01:00:23.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.510+0000 7f74db7fe640 1 -- 172.21.15.5:0/3545815410 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e8031450 con 0x56234d3521b0 2024-05-09T01:00:23.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.510+0000 7f74db7fe640 1 -- 172.21.15.5:0/3545815410 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74e8036040 con 0x56234d3521b0 2024-05-09T01:00:23.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.510+0000 7f74db7fe640 1 -- 172.21.15.5:0/3545815410 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e80045e0 con 0x56234d3521b0 2024-05-09T01:00:23.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.511+0000 7f74fe7b3740 1 -- 172.21.15.5:0/3545815410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56234d3521b0 msgr2=0x56234d6f6500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:23.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.511+0000 7f74fe7b3740 1 --2- 172.21.15.5:0/3545815410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56234d3521b0 0x56234d6f6500 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f74f0075fe0 tx=0x7f74e802f3a0 comp rx=0 tx=0).stop 2024-05-09T01:00:23.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.511+0000 7f74f55c5640 1 --2- 172.21.15.5:0/359922581 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f74dc0b2860 0x7f74dc0b4d70 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x56234d6f78e0 tx=0x7f74ec0e80e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:23.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.511+0000 7f74fe7b3740 1 -- 172.21.15.5:0/3545815410 shutdown_connections 2024-05-09T01:00:23.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.511+0000 7f74fe7b3740 1 --2- 172.21.15.5:0/3545815410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56234d3314d0 0x56234d6fd980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:23.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.511+0000 7f74fe7b3740 1 --2- 172.21.15.5:0/3545815410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56234d3292f0 0x56234d6f6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:23.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.511+0000 7f74fe7b3740 1 --2- 172.21.15.5:0/3545815410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56234d3521b0 0x56234d6f6500 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:23.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.511+0000 7f74fe7b3740 1 -- 172.21.15.5:0/3545815410 >> 172.21.15.5:0/3545815410 conn(0x56234d346300 msgr2=0x56234d6f1e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:23.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.511+0000 7f74fe7b3740 1 -- 172.21.15.5:0/3545815410 shutdown_connections 2024-05-09T01:00:23.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.511+0000 7f74fe7b3740 1 -- 172.21.15.5:0/3545815410 wait complete. 2024-05-09T01:00:23.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.512+0000 7f74fe7b3740 1 -- start start 2024-05-09T01:00:23.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.512+0000 7f74fe7b3740 1 Processor -- start 2024-05-09T01:00:23.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.512+0000 7f74fe7b3740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56234d3292f0 0x56234d708a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:23.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.512+0000 7f74fe7b3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56234d3314d0 0x56234d708fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:23.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.512+0000 7f74fe7b3740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56234d3521b0 0x56234d70ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:23.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.512+0000 7f74fe7b3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56234d6fe370 con 0x56234d3314d0 2024-05-09T01:00:23.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.512+0000 7f74fe7b3740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x56234d5e19c0 con 0x56234d3521b0 2024-05-09T01:00:23.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.512+0000 7f74fe7b3740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56234d6fe4f0 con 0x56234d3292f0 2024-05-09T01:00:23.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.512+0000 7f74fe7b3740 10 client.0 _ll_register_callbacks cb 0x56234d60e020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:00:23.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.512+0000 7f74f4dc4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56234d3314d0 0x56234d708fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:23.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.512+0000 7f74f4dc4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56234d3314d0 0x56234d708fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46474/0 (socket says 172.21.15.5:46474) 2024-05-09T01:00:23.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.512+0000 7f74f4dc4640 1 -- 172.21.15.5:0/2159306191 learned_addr learned my addr 172.21.15.5:0/2159306191 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:00:23.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.512+0000 7f74f5dc6640 1 --2- 172.21.15.5:0/2159306191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56234d3292f0 0x56234d708a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:23.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.513+0000 7f74f5dc6640 1 --2- 172.21.15.5:0/2159306191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56234d3521b0 0x56234d70ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:23.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.513+0000 7f74f4dc4640 1 -- 172.21.15.5:0/2159306191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56234d3292f0 msgr2=0x56234d708a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:23.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.513+0000 7f74f4dc4640 1 --2- 172.21.15.5:0/2159306191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56234d3292f0 0x56234d708a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:23.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.513+0000 7f74f4dc4640 1 -- 172.21.15.5:0/2159306191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56234d3521b0 msgr2=0x56234d70ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:23.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.513+0000 7f74f4dc4640 1 --2- 172.21.15.5:0/2159306191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56234d3521b0 0x56234d70ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:23.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.513+0000 7f74f4dc4640 1 -- 172.21.15.5:0/2159306191 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56234d6081f0 con 0x56234d3314d0 2024-05-09T01:00:23.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.513+0000 7f74f5dc6640 1 --2- 172.21.15.5:0/2159306191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56234d3521b0 0x56234d70ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:00:23.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.513+0000 7f74f4dc4640 1 --2- 172.21.15.5:0/2159306191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56234d3314d0 0x56234d708fb0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f74e0012720 tx=0x7f74e0012be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52f0953295e553c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:23.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.513+0000 7f74d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T01:00:23.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.513+0000 7f74d8ff9640 1 -- 172.21.15.5:0/2159306191 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e0004030 con 0x56234d3314d0 2024-05-09T01:00:23.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.513+0000 7f74d8ff9640 1 -- 172.21.15.5:0/2159306191 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f74e00041d0 con 0x56234d3314d0 2024-05-09T01:00:23.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.513+0000 7f74fe7b3740 1 -- 172.21.15.5:0/2159306191 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74e8036040 con 0x56234d3314d0 2024-05-09T01:00:23.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.513+0000 7f74d8ff9640 1 -- 172.21.15.5:0/2159306191 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f74e0022030 con 0x56234d3314d0 2024-05-09T01:00:23.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.513+0000 7f74fe7b3740 10 client.16977 fetch_fsmap learned FSMap version 32 2024-05-09T01:00:23.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.513+0000 7f74fe7b3740 1 -- 172.21.15.5:0/2159306191 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56234d3bd790 con 0x56234d3314d0 2024-05-09T01:00:23.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.513+0000 7f74d8ff9640 1 -- 172.21.15.5:0/2159306191 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74e8036040 con 0x56234d3314d0 2024-05-09T01:00:23.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.514+0000 7f74d8ff9640 1 -- 172.21.15.5:0/2159306191 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f74e00038c0 con 0x56234d3314d0 2024-05-09T01:00:23.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.514+0000 7f74fe7b3740 10 client.16977 fetch_fsmap finished waiting for FSMap version 32 2024-05-09T01:00:23.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.514+0000 7f74fe7b3740 10 client.16977 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T01:00:23.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.514+0000 7f74fe7b3740 20 client.16977 populate_metadata read hostname 'smithi005' 2024-05-09T01:00:23.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.514+0000 7f74fe7b3740 1 --2- 172.21.15.5:0/2159306191 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56234d310d80 0x56234d716500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:23.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.514+0000 7f74fe7b3740 4 client.16977 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:00:23.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.514+0000 7f74fe7b3740 1 -- 172.21.15.5:0/2159306191 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56234d6683d0 con 0x56234d310d80 2024-05-09T01:00:23.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.514+0000 7f74f5dc6640 1 --2- 172.21.15.5:0/2159306191 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56234d310d80 0x56234d716500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:23.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.514+0000 7f74d8ff9640 10 client.16977 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T01:00:23.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.514+0000 7f74f5dc6640 1 --2- 172.21.15.5:0/2159306191 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56234d310d80 0x56234d716500 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1c75f99e9913aa8 server_cookie=b61365bc0ead644c in_seq=0 out_seq=0 2024-05-09T01:00:23.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.516+0000 7f74d8ff9640 1 -- 172.21.15.5:0/2159306191 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56234d6683d0 con 0x56234d310d80 2024-05-09T01:00:23.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.516+0000 7f74d8ff9640 10 client.16977 handle_command_reply: tid=0 2024-05-09T01:00:23.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.517+0000 7f74fe7b3740 1 client.16977 shutdown 2024-05-09T01:00:23.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.517+0000 7f74fe7b3740 1 -- 172.21.15.5:0/2159306191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56234d3314d0 msgr2=0x56234d708fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:23.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.517+0000 7f74fe7b3740 1 --2- 172.21.15.5:0/2159306191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56234d3314d0 0x56234d708fb0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f74e0012720 tx=0x7f74e0012be0 comp rx=0 tx=0).stop 2024-05-09T01:00:23.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.517+0000 7f74fe7b3740 1 -- 172.21.15.5:0/2159306191 shutdown_connections 2024-05-09T01:00:23.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.517+0000 7f74fe7b3740 1 --2- 172.21.15.5:0/2159306191 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56234d310d80 0x56234d716500 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:23.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.517+0000 7f74fe7b3740 1 --2- 172.21.15.5:0/2159306191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56234d3521b0 0x56234d70ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:23.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.517+0000 7f74fe7b3740 1 --2- 172.21.15.5:0/2159306191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56234d3314d0 0x56234d708fb0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:23.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.517+0000 7f74fe7b3740 1 --2- 172.21.15.5:0/2159306191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56234d3292f0 0x56234d708a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:23.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.517+0000 7f74d8ff9640 0 client.16977 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T01:00:23.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.517+0000 7f74fe7b3740 1 -- 172.21.15.5:0/2159306191 >> 172.21.15.5:0/2159306191 conn(0x56234d346300 msgr2=0x56234d6f4da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:23.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.517+0000 7f74fe7b3740 1 -- 172.21.15.5:0/2159306191 shutdown_connections 2024-05-09T01:00:23.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.517+0000 7f74fe7b3740 1 -- 172.21.15.5:0/2159306191 wait complete. 2024-05-09T01:00:23.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.517+0000 7f74fe7b3740 20 client.16977 trim_cache size 0 max 16384 2024-05-09T01:00:23.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.532+0000 7f74fe7b3740 1 Processor -- start 2024-05-09T01:00:23.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.532+0000 7f74fe7b3740 1 -- start start 2024-05-09T01:00:23.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.532+0000 7f74fe7b3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56234d3292f0 0x56234d71acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:23.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.532+0000 7f74fe7b3740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56234d3314d0 0x56234d70fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:23.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.532+0000 7f74fe7b3740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56234d3521b0 0x56234d710370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:23.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.532+0000 7f74fe7b3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f74f01a9220 con 0x56234d3292f0 2024-05-09T01:00:23.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.532+0000 7f74fe7b3740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f74f01aad60 con 0x56234d3521b0 2024-05-09T01:00:23.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.532+0000 7f74fe7b3740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f74f01a98f0 con 0x56234d3314d0 2024-05-09T01:00:23.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.532+0000 7f74f5dc6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56234d3292f0 0x56234d71acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:23.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.532+0000 7f74f5dc6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56234d3292f0 0x56234d71acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46488/0 (socket says 172.21.15.5:46488) 2024-05-09T01:00:23.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.532+0000 7f74f5dc6640 1 -- 172.21.15.5:0/1362906995 learned_addr learned my addr 172.21.15.5:0/1362906995 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:00:23.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.533+0000 7f74f4dc4640 1 --2- 172.21.15.5:0/1362906995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56234d3314d0 0x56234d70fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:23.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.533+0000 7f74f5dc6640 1 --2- 172.21.15.5:0/1362906995 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56234d3521b0 0x56234d710370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:23.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.533+0000 7f74f5dc6640 1 -- 172.21.15.5:0/1362906995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56234d3314d0 msgr2=0x56234d70fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:23.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.533+0000 7f74f5dc6640 1 --2- 172.21.15.5:0/1362906995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56234d3314d0 0x56234d70fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:23.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.533+0000 7f74f5dc6640 1 -- 172.21.15.5:0/1362906995 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56234d3521b0 msgr2=0x56234d710370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:23.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.533+0000 7f74f5dc6640 1 --2- 172.21.15.5:0/1362906995 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56234d3521b0 0x56234d710370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:23.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.533+0000 7f74f5dc6640 1 -- 172.21.15.5:0/1362906995 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56234d4345f0 con 0x56234d3292f0 2024-05-09T01:00:23.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.533+0000 7f74f5dc6640 1 --2- 172.21.15.5:0/1362906995 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56234d3292f0 0x56234d71acb0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f74e802ade0 tx=0x7f74e802f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f216b6e99a549dc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:23.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.533+0000 7f74daffd640 1 -- 172.21.15.5:0/1362906995 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e80316f0 con 0x56234d3292f0 2024-05-09T01:00:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.533+0000 7f74daffd640 1 -- 172.21.15.5:0/1362906995 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74e8036050 con 0x56234d3292f0 2024-05-09T01:00:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.533+0000 7f74daffd640 1 -- 172.21.15.5:0/1362906995 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e80047b0 con 0x56234d3292f0 2024-05-09T01:00:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.534+0000 7f74fe7b3740 1 -- 172.21.15.5:0/1362906995 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56234d3292f0 msgr2=0x56234d71acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.534+0000 7f74fe7b3740 1 --2- 172.21.15.5:0/1362906995 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56234d3292f0 0x56234d71acb0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f74e802ade0 tx=0x7f74e802f3a0 comp rx=0 tx=0).stop 2024-05-09T01:00:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.534+0000 7f74fe7b3740 1 -- 172.21.15.5:0/1362906995 shutdown_connections 2024-05-09T01:00:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.534+0000 7f74fe7b3740 1 --2- 172.21.15.5:0/1362906995 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56234d3521b0 0x56234d710370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.534+0000 7f74fe7b3740 1 --2- 172.21.15.5:0/1362906995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56234d3314d0 0x56234d70fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.534+0000 7f74fe7b3740 1 --2- 172.21.15.5:0/1362906995 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56234d3292f0 0x56234d71acb0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.534+0000 7f74fe7b3740 1 -- 172.21.15.5:0/1362906995 >> 172.21.15.5:0/1362906995 conn(0x56234d346300 msgr2=0x56234d6fa700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.534+0000 7f74fe7b3740 1 -- 172.21.15.5:0/1362906995 shutdown_connections 2024-05-09T01:00:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.534+0000 7f74fe7b3740 1 -- 172.21.15.5:0/1362906995 wait complete. 2024-05-09T01:00:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.534+0000 7f74fe7b3740 1 -- start start 2024-05-09T01:00:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.534+0000 7f74fe7b3740 1 Processor -- start 2024-05-09T01:00:23.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.535+0000 7f74fe7b3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56234d3292f0 0x56234d70d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:23.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.535+0000 7f74fe7b3740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56234d3314d0 0x56234d70da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:23.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.535+0000 7f74fe7b3740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56234d3521b0 0x56234d77c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:23.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.535+0000 7f74fe7b3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f74f01aa690 con 0x56234d3292f0 2024-05-09T01:00:23.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.535+0000 7f74fe7b3740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f74f01a9fc0 con 0x56234d3314d0 2024-05-09T01:00:23.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.535+0000 7f74fe7b3740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f74f01a7200 con 0x56234d3521b0 2024-05-09T01:00:23.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.535+0000 7f74fe7b3740 10 client.0 _ll_register_callbacks cb 0x7f74e0009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:00:23.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.535+0000 7f74f5dc6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56234d3292f0 0x56234d70d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:23.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.535+0000 7f74f5dc6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56234d3292f0 0x56234d70d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46492/0 (socket says 172.21.15.5:46492) 2024-05-09T01:00:23.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.535+0000 7f74f5dc6640 1 -- 172.21.15.5:0/2793474499 learned_addr learned my addr 172.21.15.5:0/2793474499 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:00:23.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.535+0000 7f74f4dc4640 1 --2- 172.21.15.5:0/2793474499 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56234d3314d0 0x56234d70da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:23.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.535+0000 7f74f5dc6640 1 --2- 172.21.15.5:0/2793474499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56234d3521b0 0x56234d77c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:23.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.535+0000 7f74f5dc6640 1 -- 172.21.15.5:0/2793474499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56234d3521b0 msgr2=0x56234d77c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:23.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.535+0000 7f74f5dc6640 1 --2- 172.21.15.5:0/2793474499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56234d3521b0 0x56234d77c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:23.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.535+0000 7f74f5dc6640 1 -- 172.21.15.5:0/2793474499 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56234d3314d0 msgr2=0x56234d70da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:23.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.535+0000 7f74f5dc6640 1 --2- 172.21.15.5:0/2793474499 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56234d3314d0 0x56234d70da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:23.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.535+0000 7f74f5dc6640 1 -- 172.21.15.5:0/2793474499 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56234d61eed0 con 0x56234d3292f0 2024-05-09T01:00:23.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.536+0000 7f74f5dc6640 1 --2- 172.21.15.5:0/2793474499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56234d3292f0 0x56234d70d4f0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f74e8009a60 tx=0x7f74e80094a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3522478266812d06 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:23.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.536+0000 7f74d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T01:00:23.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.536+0000 7f74d8ff9640 1 -- 172.21.15.5:0/2793474499 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e8004530 con 0x56234d3292f0 2024-05-09T01:00:23.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.536+0000 7f74d8ff9640 1 -- 172.21.15.5:0/2793474499 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f74e80046d0 con 0x56234d3292f0 2024-05-09T01:00:23.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.536+0000 7f74fe7b3740 1 -- 172.21.15.5:0/2793474499 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74e8036050 con 0x56234d3292f0 2024-05-09T01:00:23.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.536+0000 7f74d8ff9640 1 -- 172.21.15.5:0/2793474499 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f74e8003970 con 0x56234d3292f0 2024-05-09T01:00:23.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.536+0000 7f74fe7b3740 10 client.16983 fetch_fsmap learned FSMap version 32 2024-05-09T01:00:23.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.536+0000 7f74fe7b3740 1 -- 172.21.15.5:0/2793474499 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56234d77dc40 con 0x56234d3292f0 2024-05-09T01:00:23.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.536+0000 7f74d8ff9640 1 -- 172.21.15.5:0/2793474499 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74e8036050 con 0x56234d3292f0 2024-05-09T01:00:23.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.536+0000 7f74d8ff9640 1 -- 172.21.15.5:0/2793474499 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f74e8050040 con 0x56234d3292f0 2024-05-09T01:00:23.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.536+0000 7f74fe7b3740 10 client.16983 fetch_fsmap finished waiting for FSMap version 32 2024-05-09T01:00:23.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.536+0000 7f74fe7b3740 10 client.16983 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T01:00:23.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.536+0000 7f74fe7b3740 20 client.16983 populate_metadata read hostname 'smithi005' 2024-05-09T01:00:23.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.536+0000 7f74fe7b3740 1 --2- 172.21.15.5:0/2793474499 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56234d2e61c0 0x56234d780250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:23.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.536+0000 7f74fe7b3740 4 client.16983 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T01:00:23.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.536+0000 7f74fe7b3740 1 -- 172.21.15.5:0/2793474499 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56234d662240 con 0x56234d2e61c0 2024-05-09T01:00:23.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.537+0000 7f74f4dc4640 1 --2- 172.21.15.5:0/2793474499 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56234d2e61c0 0x56234d780250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:23.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.537+0000 7f74d8ff9640 10 client.16983 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T01:00:23.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:23.537+0000 7f74f4dc4640 1 --2- 172.21.15.5:0/2793474499 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56234d2e61c0 0x56234d780250 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c8867d58c91a9ba server_cookie=2e56f202e2fa30b3 in_seq=0 out_seq=0 2024-05-09T01:00:24.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:00:24 smithi005 ceph-mon[30365]: pgmap v792: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 81 MiB/s wr, 412 op/s 2024-05-09T01:00:24.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:00:24 smithi131 ceph-mon[34547]: pgmap v792: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 81 MiB/s wr, 412 op/s 2024-05-09T01:00:24.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:00:24 smithi157 ceph-mon[34294]: pgmap v792: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 81 MiB/s wr, 412 op/s 2024-05-09T01:00:24.788 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T01:00:25.065 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:00:26.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:00:26 smithi005 ceph-mon[30365]: pgmap v793: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 76 MiB/s wr, 433 op/s 2024-05-09T01:00:26.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:00:26 smithi131 ceph-mon[34547]: pgmap v793: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 76 MiB/s wr, 433 op/s 2024-05-09T01:00:26.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:00:26 smithi157 ceph-mon[34294]: pgmap v793: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 76 MiB/s wr, 433 op/s 2024-05-09T01:00:26.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.849+0000 7f6aecf0f700 1 -- 172.21.15.5:0/4128077564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ae8108780 msgr2=0x7f6ae81100e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:26.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.849+0000 7f6aecf0f700 1 --2- 172.21.15.5:0/4128077564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ae8108780 0x7f6ae81100e0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f6ae000b0d0 tx=0x7f6ae000b3e0 comp rx=0 tx=0).stop 2024-05-09T01:00:26.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.849+0000 7f6aecf0f700 1 -- 172.21.15.5:0/4128077564 shutdown_connections 2024-05-09T01:00:26.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.849+0000 7f6aecf0f700 1 --2- 172.21.15.5:0/4128077564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ae8108780 0x7f6ae81100e0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:26.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.849+0000 7f6aecf0f700 1 --2- 172.21.15.5:0/4128077564 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ae8103550 0x7f6ae8103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:26.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.849+0000 7f6aecf0f700 1 --2- 172.21.15.5:0/4128077564 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ae8107e30 0x7f6ae8108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:26.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.849+0000 7f6aecf0f700 1 -- 172.21.15.5:0/4128077564 >> 172.21.15.5:0/4128077564 conn(0x7f6ae8078e00 msgr2=0x7f6ae8079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:26.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.849+0000 7f6aecf0f700 1 -- 172.21.15.5:0/4128077564 shutdown_connections 2024-05-09T01:00:26.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.849+0000 7f6aecf0f700 1 -- 172.21.15.5:0/4128077564 wait complete. 2024-05-09T01:00:26.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.850+0000 7f6aecf0f700 1 Processor -- start 2024-05-09T01:00:26.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.851+0000 7f6aecf0f700 1 -- start start 2024-05-09T01:00:26.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.851+0000 7f6aecf0f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ae8103550 0x7f6ae819d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:26.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.851+0000 7f6aecf0f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ae8107e30 0x7f6ae819d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:26.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.851+0000 7f6aecf0f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ae8108780 0x7f6ae81a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:26.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.851+0000 7f6aecf0f700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae8112310 con 0x7f6ae8103550 2024-05-09T01:00:26.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.851+0000 7f6aecf0f700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae8112190 con 0x7f6ae8108780 2024-05-09T01:00:26.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.851+0000 7f6aecf0f700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae8112490 con 0x7f6ae8107e30 2024-05-09T01:00:26.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.851+0000 7f6ae659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ae8103550 0x7f6ae819d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:26.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.851+0000 7f6ae659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ae8103550 0x7f6ae819d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57144/0 (socket says 172.21.15.5:57144) 2024-05-09T01:00:26.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.851+0000 7f6ae659c700 1 -- 172.21.15.5:0/2543737353 learned_addr learned my addr 172.21.15.5:0/2543737353 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:00:26.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.851+0000 7f6ae5d9b700 1 --2- 172.21.15.5:0/2543737353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ae8107e30 0x7f6ae819d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:26.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.851+0000 7f6ae6d9d700 1 --2- 172.21.15.5:0/2543737353 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ae8108780 0x7f6ae81a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:26.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.851+0000 7f6ae659c700 1 -- 172.21.15.5:0/2543737353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ae8107e30 msgr2=0x7f6ae819d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:26.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.851+0000 7f6ae659c700 1 --2- 172.21.15.5:0/2543737353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ae8107e30 0x7f6ae819d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:26.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.851+0000 7f6ae659c700 1 -- 172.21.15.5:0/2543737353 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ae8108780 msgr2=0x7f6ae81a1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:26.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.851+0000 7f6ae659c700 1 --2- 172.21.15.5:0/2543737353 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ae8108780 0x7f6ae81a1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:26.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.851+0000 7f6ae659c700 1 -- 172.21.15.5:0/2543737353 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ae81a1f00 con 0x7f6ae8103550 2024-05-09T01:00:26.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.851+0000 7f6ae5d9b700 1 --2- 172.21.15.5:0/2543737353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ae8107e30 0x7f6ae819d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:00:26.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.851+0000 7f6ae659c700 1 --2- 172.21.15.5:0/2543737353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ae8103550 0x7f6ae819d030 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f6ad800e9a0 tx=0x7f6ad800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:26.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.852+0000 7f6ad77fe700 1 -- 172.21.15.5:0/2543737353 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad800ca10 con 0x7f6ae8103550 2024-05-09T01:00:26.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.852+0000 7f6ad77fe700 1 -- 172.21.15.5:0/2543737353 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ad800cbb0 con 0x7f6ae8103550 2024-05-09T01:00:26.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.852+0000 7f6aecf0f700 1 -- 172.21.15.5:0/2543737353 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ae81a21f0 con 0x7f6ae8103550 2024-05-09T01:00:26.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.852+0000 7f6ad77fe700 1 -- 172.21.15.5:0/2543737353 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad8018680 con 0x7f6ae8103550 2024-05-09T01:00:26.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.852+0000 7f6aecf0f700 1 -- 172.21.15.5:0/2543737353 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ae81a9b10 con 0x7f6ae8103550 2024-05-09T01:00:26.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.855+0000 7f6ad77fe700 1 -- 172.21.15.5:0/2543737353 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f6ad8018820 con 0x7f6ae8103550 2024-05-09T01:00:26.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.855+0000 7f6ad77fe700 1 --2- 172.21.15.5:0/2543737353 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f6ad00b2860 0x7f6ad00b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:26.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.856+0000 7f6ae5d9b700 1 --2- 172.21.15.5:0/2543737353 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f6ad00b2860 0x7f6ad00b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:26.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.856+0000 7f6ad77fe700 1 -- 172.21.15.5:0/2543737353 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f6ad80e2460 con 0x7f6ae8103550 2024-05-09T01:00:26.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.856+0000 7f6ae5d9b700 1 --2- 172.21.15.5:0/2543737353 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f6ad00b2860 0x7f6ad00b4d70 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8102b50 tx=0x7f6adc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:26.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.856+0000 7f6afb925b80 1 Processor -- start 2024-05-09T01:00:26.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.856+0000 7f6afb925b80 1 -- start start 2024-05-09T01:00:26.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.857+0000 7f6afb925b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5612fcab9160 0x5612fcb524f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:26.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.857+0000 7f6afb925b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612fc9cf4a0 0x5612fcb52a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:26.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.857+0000 7f6afb925b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612fcb4c6a0 0x5612fcb4cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:26.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.857+0000 7f6afb925b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5612fca6eff0 con 0x5612fc9cf4a0 2024-05-09T01:00:26.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.857+0000 7f6ae5d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612fc9cf4a0 0x5612fcb52a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:26.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.857+0000 7f6afb925b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5612fca0d7f0 con 0x5612fcab9160 2024-05-09T01:00:26.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.857+0000 7f6afb925b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5612fca6afb0 con 0x5612fcb4c6a0 2024-05-09T01:00:26.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.857+0000 7f6ae5d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612fc9cf4a0 0x5612fcb52a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57150/0 (socket says 172.21.15.5:57150) 2024-05-09T01:00:26.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.857+0000 7f6ae5d9b700 1 -- 172.21.15.5:0/587536399 learned_addr learned my addr 172.21.15.5:0/587536399 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:00:26.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.857+0000 7f6ae6d9d700 1 --2- 172.21.15.5:0/587536399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612fcb4c6a0 0x5612fcb4cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:26.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.857+0000 7f6ae659c700 1 --2- 172.21.15.5:0/587536399 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5612fcab9160 0x5612fcb524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:26.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.857+0000 7f6ae5d9b700 1 -- 172.21.15.5:0/587536399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612fcb4c6a0 msgr2=0x5612fcb4cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:26.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.857+0000 7f6ae5d9b700 1 --2- 172.21.15.5:0/587536399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612fcb4c6a0 0x5612fcb4cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:26.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.857+0000 7f6ae5d9b700 1 -- 172.21.15.5:0/587536399 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5612fcab9160 msgr2=0x5612fcb524f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:26.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.857+0000 7f6ae5d9b700 1 --2- 172.21.15.5:0/587536399 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5612fcab9160 0x5612fcb524f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:26.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.857+0000 7f6ae5d9b700 1 -- 172.21.15.5:0/587536399 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612fca05370 con 0x5612fc9cf4a0 2024-05-09T01:00:26.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.857+0000 7f6ae6d9d700 1 --2- 172.21.15.5:0/587536399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612fcb4c6a0 0x5612fcb4cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:00:26.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.857+0000 7f6ae5d9b700 1 --2- 172.21.15.5:0/587536399 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612fc9cf4a0 0x5612fcb52a30 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f6adc00e700 tx=0x7f6adc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b3962ae3c183cb5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:26.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.858+0000 7f6ad4ff9700 1 -- 172.21.15.5:0/587536399 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6adc004230 con 0x5612fc9cf4a0 2024-05-09T01:00:26.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.858+0000 7f6ad4ff9700 1 -- 172.21.15.5:0/587536399 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6adc0043d0 con 0x5612fc9cf4a0 2024-05-09T01:00:26.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.858+0000 7f6ad4ff9700 1 -- 172.21.15.5:0/587536399 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6adc01e760 con 0x5612fc9cf4a0 2024-05-09T01:00:26.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.859+0000 7f6afb925b80 1 -- 172.21.15.5:0/587536399 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612fc9cf4a0 msgr2=0x5612fcb52a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:26.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.859+0000 7f6afb925b80 1 --2- 172.21.15.5:0/587536399 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612fc9cf4a0 0x5612fcb52a30 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f6adc00e700 tx=0x7f6adc014000 comp rx=0 tx=0).stop 2024-05-09T01:00:26.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.859+0000 7f6afb925b80 1 -- 172.21.15.5:0/587536399 shutdown_connections 2024-05-09T01:00:26.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.859+0000 7f6afb925b80 1 --2- 172.21.15.5:0/587536399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612fcb4c6a0 0x5612fcb4cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:26.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.859+0000 7f6afb925b80 1 --2- 172.21.15.5:0/587536399 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5612fcab9160 0x5612fcb524f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:26.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.859+0000 7f6afb925b80 1 --2- 172.21.15.5:0/587536399 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612fc9cf4a0 0x5612fcb52a30 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:26.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.859+0000 7f6afb925b80 1 -- 172.21.15.5:0/587536399 >> 172.21.15.5:0/587536399 conn(0x5612fca2f6a0 msgr2=0x5612fcb47400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:26.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.859+0000 7f6afb925b80 1 -- 172.21.15.5:0/587536399 shutdown_connections 2024-05-09T01:00:26.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.859+0000 7f6afb925b80 1 -- 172.21.15.5:0/587536399 wait complete. 2024-05-09T01:00:26.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.859+0000 7f6afb925b80 1 -- start start 2024-05-09T01:00:26.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.859+0000 7f6afb925b80 1 Processor -- start 2024-05-09T01:00:26.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.860+0000 7f6afb925b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612fc9cf4a0 0x5612fcb56230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:26.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.860+0000 7f6afb925b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612fcab9160 0x5612fcb56770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:26.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.860+0000 7f6afb925b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5612fcb4c6a0 0x5612fcb56cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:26.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.860+0000 7f6afb925b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5612fc8eaba0 con 0x5612fc9cf4a0 2024-05-09T01:00:26.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.860+0000 7f6afb925b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5612fca29150 con 0x5612fcb4c6a0 2024-05-09T01:00:26.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.860+0000 7f6afb925b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5612fcb591d0 con 0x5612fcab9160 2024-05-09T01:00:26.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.860+0000 7f6afb925b80 10 client.0 _ll_register_callbacks cb 0x5612fc8e06c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:00:26.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.860+0000 7f6ae659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612fc9cf4a0 0x5612fcb56230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:26.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.860+0000 7f6ae5d9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612fcab9160 0x5612fcb56770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:26.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.860+0000 7f6ae659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612fc9cf4a0 0x5612fcb56230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57160/0 (socket says 172.21.15.5:57160) 2024-05-09T01:00:26.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.860+0000 7f6ae5d9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612fcab9160 0x5612fcb56770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:57982/0 (socket says 172.21.15.5:57982) 2024-05-09T01:00:26.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.860+0000 7f6ae659c700 1 -- 172.21.15.5:0/3243747851 learned_addr learned my addr 172.21.15.5:0/3243747851 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:00:26.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.860+0000 7f6ae6d9d700 1 --2- 172.21.15.5:0/3243747851 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5612fcb4c6a0 0x5612fcb56cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:26.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.860+0000 7f6ae659c700 1 -- 172.21.15.5:0/3243747851 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612fcab9160 msgr2=0x5612fcb56770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:26.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.860+0000 7f6ae659c700 1 --2- 172.21.15.5:0/3243747851 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612fcab9160 0x5612fcb56770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:26.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.860+0000 7f6ae659c700 1 -- 172.21.15.5:0/3243747851 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5612fcb4c6a0 msgr2=0x5612fcb56cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:26.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.860+0000 7f6ae659c700 1 --2- 172.21.15.5:0/3243747851 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5612fcb4c6a0 0x5612fcb56cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:26.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.860+0000 7f6ae659c700 1 -- 172.21.15.5:0/3243747851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612fc8ea490 con 0x5612fc9cf4a0 2024-05-09T01:00:26.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.860+0000 7f6ae5d9b700 1 --2- 172.21.15.5:0/3243747851 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612fcab9160 0x5612fcb56770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:00:26.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.861+0000 7f6ae659c700 1 --2- 172.21.15.5:0/3243747851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612fc9cf4a0 0x5612fcb56230 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f6ad80a6cc0 tx=0x7f6ad80a52b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57156a4721946d72 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:26.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.861+0000 7f6acdffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T01:00:26.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.861+0000 7f6acdffb700 1 -- 172.21.15.5:0/3243747851 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad80a5040 con 0x5612fc9cf4a0 2024-05-09T01:00:26.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.861+0000 7f6acdffb700 1 -- 172.21.15.5:0/3243747851 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f6ad80b2420 con 0x5612fc9cf4a0 2024-05-09T01:00:26.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.861+0000 7f6afb925b80 1 -- 172.21.15.5:0/3243747851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6adc0043d0 con 0x5612fc9cf4a0 2024-05-09T01:00:26.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.861+0000 7f6acdffb700 1 -- 172.21.15.5:0/3243747851 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6ad80aa150 con 0x5612fc9cf4a0 2024-05-09T01:00:26.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.861+0000 7f6afb925b80 10 client.16995 fetch_fsmap learned FSMap version 32 2024-05-09T01:00:26.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.861+0000 7f6afb925b80 1 -- 172.21.15.5:0/3243747851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5612fcb63c60 con 0x5612fc9cf4a0 2024-05-09T01:00:26.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.861+0000 7f6acdffb700 1 -- 172.21.15.5:0/3243747851 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6adc0043d0 con 0x5612fc9cf4a0 2024-05-09T01:00:26.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.861+0000 7f6acdffb700 1 -- 172.21.15.5:0/3243747851 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f6ad80702c0 con 0x5612fc9cf4a0 2024-05-09T01:00:26.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.861+0000 7f6afb925b80 10 client.16995 fetch_fsmap finished waiting for FSMap version 32 2024-05-09T01:00:26.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.861+0000 7f6afb925b80 10 client.16995 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T01:00:26.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.861+0000 7f6afb925b80 20 client.16995 populate_metadata read hostname 'smithi005' 2024-05-09T01:00:26.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.861+0000 7f6afb925b80 1 --2- 172.21.15.5:0/3243747851 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5612fcb661f0 0x5612fcb68610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:26.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.862+0000 7f6afb925b80 4 client.16995 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:00:26.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.862+0000 7f6afb925b80 1 -- 172.21.15.5:0/3243747851 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612fca7c9d0 con 0x5612fcb661f0 2024-05-09T01:00:26.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.862+0000 7f6ae5d9b700 1 --2- 172.21.15.5:0/3243747851 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5612fcb661f0 0x5612fcb68610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:26.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.862+0000 7f6ae5d9b700 1 --2- 172.21.15.5:0/3243747851 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5612fcb661f0 0x5612fcb68610 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d098ca5a7bcadf5 server_cookie=c25f657400810766 in_seq=0 out_seq=0 2024-05-09T01:00:26.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.862+0000 7f6acdffb700 10 client.16995 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T01:00:26.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.864+0000 7f6acdffb700 1 -- 172.21.15.5:0/3243747851 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5612fca7c9d0 con 0x5612fcb661f0 2024-05-09T01:00:26.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.864+0000 7f6acdffb700 10 client.16995 handle_command_reply: tid=0 2024-05-09T01:00:26.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.864+0000 7f6afb925b80 1 client.16995 shutdown 2024-05-09T01:00:26.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.865+0000 7f6afb925b80 1 -- 172.21.15.5:0/3243747851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612fc9cf4a0 msgr2=0x5612fcb56230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:26.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.865+0000 7f6afb925b80 1 --2- 172.21.15.5:0/3243747851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612fc9cf4a0 0x5612fcb56230 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f6ad80a6cc0 tx=0x7f6ad80a52b0 comp rx=0 tx=0).stop 2024-05-09T01:00:26.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.865+0000 7f6afb925b80 1 -- 172.21.15.5:0/3243747851 shutdown_connections 2024-05-09T01:00:26.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.865+0000 7f6afb925b80 1 --2- 172.21.15.5:0/3243747851 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5612fcb661f0 0x5612fcb68610 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:26.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.865+0000 7f6afb925b80 1 --2- 172.21.15.5:0/3243747851 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5612fcb4c6a0 0x5612fcb56cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:26.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.865+0000 7f6afb925b80 1 --2- 172.21.15.5:0/3243747851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612fc9cf4a0 0x5612fcb56230 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:26.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.865+0000 7f6acdffb700 0 client.16995 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T01:00:26.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.865+0000 7f6afb925b80 1 --2- 172.21.15.5:0/3243747851 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612fcab9160 0x5612fcb56770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:26.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.865+0000 7f6afb925b80 1 -- 172.21.15.5:0/3243747851 >> 172.21.15.5:0/3243747851 conn(0x5612fca2f6a0 msgr2=0x5612fcb471c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:26.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.865+0000 7f6afb925b80 1 -- 172.21.15.5:0/3243747851 shutdown_connections 2024-05-09T01:00:26.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.865+0000 7f6afb925b80 1 -- 172.21.15.5:0/3243747851 wait complete. 2024-05-09T01:00:26.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.865+0000 7f6afb925b80 20 client.16995 trim_cache size 0 max 16384 2024-05-09T01:00:26.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.886+0000 7f6afb925b80 1 Processor -- start 2024-05-09T01:00:26.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.886+0000 7f6afb925b80 1 -- start start 2024-05-09T01:00:26.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.886+0000 7f6afb925b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612fc9cf4a0 0x5612fcb56c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:26.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.886+0000 7f6afb925b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612fcab9160 0x5612fcb549f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:26.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.886+0000 7f6afb925b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5612fcb4c6a0 0x5612fcb54f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:26.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.886+0000 7f6afb925b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae81b0f00 con 0x5612fc9cf4a0 2024-05-09T01:00:26.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.886+0000 7f6afb925b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae81b2a40 con 0x5612fcb4c6a0 2024-05-09T01:00:26.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.886+0000 7f6afb925b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae81b15d0 con 0x5612fcab9160 2024-05-09T01:00:26.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.886+0000 7f6ae659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612fc9cf4a0 0x5612fcb56c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:26.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.886+0000 7f6ae5d9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612fcab9160 0x5612fcb549f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:26.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.886+0000 7f6ae659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612fc9cf4a0 0x5612fcb56c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57170/0 (socket says 172.21.15.5:57170) 2024-05-09T01:00:26.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.886+0000 7f6ae659c700 1 -- 172.21.15.5:0/3785247225 learned_addr learned my addr 172.21.15.5:0/3785247225 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:00:26.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.886+0000 7f6ae6d9d700 1 --2- 172.21.15.5:0/3785247225 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5612fcb4c6a0 0x5612fcb54f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:26.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.887+0000 7f6ae659c700 1 -- 172.21.15.5:0/3785247225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612fcab9160 msgr2=0x5612fcb549f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:26.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.887+0000 7f6ae659c700 1 --2- 172.21.15.5:0/3785247225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612fcab9160 0x5612fcb549f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:26.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.887+0000 7f6ae659c700 1 -- 172.21.15.5:0/3785247225 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5612fcb4c6a0 msgr2=0x5612fcb54f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:26.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.887+0000 7f6ae659c700 1 --2- 172.21.15.5:0/3785247225 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5612fcb4c6a0 0x5612fcb54f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:26.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.887+0000 7f6ae659c700 1 -- 172.21.15.5:0/3785247225 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612fcb59030 con 0x5612fc9cf4a0 2024-05-09T01:00:26.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.887+0000 7f6ae5d9b700 1 --2- 172.21.15.5:0/3785247225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612fcab9160 0x5612fcb549f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:00:26.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.887+0000 7f6ae659c700 1 --2- 172.21.15.5:0/3785247225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612fc9cf4a0 0x5612fcb56c20 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f6ad800e970 tx=0x7f6ad80a59c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2048c1b0905b8b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:26.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.887+0000 7f6acffff700 1 -- 172.21.15.5:0/3785247225 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad80a5040 con 0x5612fc9cf4a0 2024-05-09T01:00:26.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.887+0000 7f6acffff700 1 -- 172.21.15.5:0/3785247225 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ad80b2040 con 0x5612fc9cf4a0 2024-05-09T01:00:26.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.887+0000 7f6acffff700 1 -- 172.21.15.5:0/3785247225 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad80ade00 con 0x5612fc9cf4a0 2024-05-09T01:00:26.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.888+0000 7f6afb925b80 1 -- 172.21.15.5:0/3785247225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612fc9cf4a0 msgr2=0x5612fcb56c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:26.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.888+0000 7f6afb925b80 1 --2- 172.21.15.5:0/3785247225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612fc9cf4a0 0x5612fcb56c20 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f6ad800e970 tx=0x7f6ad80a59c0 comp rx=0 tx=0).stop 2024-05-09T01:00:26.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.888+0000 7f6afb925b80 1 -- 172.21.15.5:0/3785247225 shutdown_connections 2024-05-09T01:00:26.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.888+0000 7f6afb925b80 1 --2- 172.21.15.5:0/3785247225 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5612fcb4c6a0 0x5612fcb54f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:26.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.888+0000 7f6afb925b80 1 --2- 172.21.15.5:0/3785247225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612fc9cf4a0 0x5612fcb56c20 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:26.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.888+0000 7f6afb925b80 1 --2- 172.21.15.5:0/3785247225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612fcab9160 0x5612fcb549f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:26.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.888+0000 7f6afb925b80 1 -- 172.21.15.5:0/3785247225 >> 172.21.15.5:0/3785247225 conn(0x5612fca2f6a0 msgr2=0x5612fcb4bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:26.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.888+0000 7f6afb925b80 1 -- 172.21.15.5:0/3785247225 shutdown_connections 2024-05-09T01:00:26.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.888+0000 7f6afb925b80 1 -- 172.21.15.5:0/3785247225 wait complete. 2024-05-09T01:00:26.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.888+0000 7f6afb925b80 1 -- start start 2024-05-09T01:00:26.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.889+0000 7f6afb925b80 1 Processor -- start 2024-05-09T01:00:26.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.889+0000 7f6afb925b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612fc9cf4a0 0x5612fcb5e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:26.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.889+0000 7f6afb925b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612fcab9160 0x5612fcb5e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:26.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.889+0000 7f6afb925b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5612fcb4c6a0 0x5612fcb5b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:26.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.889+0000 7f6afb925b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae81b2370 con 0x5612fc9cf4a0 2024-05-09T01:00:26.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.889+0000 7f6afb925b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae81b1ca0 con 0x5612fcb4c6a0 2024-05-09T01:00:26.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.889+0000 7f6afb925b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae81aeea0 con 0x5612fcab9160 2024-05-09T01:00:26.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.889+0000 7f6afb925b80 10 client.0 _ll_register_callbacks cb 0x5612fc8e06c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:00:26.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.889+0000 7f6ae659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612fc9cf4a0 0x5612fcb5e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:26.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.889+0000 7f6ae5d9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612fcab9160 0x5612fcb5e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:26.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.889+0000 7f6ae659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612fc9cf4a0 0x5612fcb5e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57182/0 (socket says 172.21.15.5:57182) 2024-05-09T01:00:26.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.889+0000 7f6ae659c700 1 -- 172.21.15.5:0/4281698352 learned_addr learned my addr 172.21.15.5:0/4281698352 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:00:26.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.889+0000 7f6ae6d9d700 1 --2- 172.21.15.5:0/4281698352 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5612fcb4c6a0 0x5612fcb5b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:26.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.889+0000 7f6ae659c700 1 -- 172.21.15.5:0/4281698352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612fcab9160 msgr2=0x5612fcb5e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:26.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.889+0000 7f6ae659c700 1 --2- 172.21.15.5:0/4281698352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612fcab9160 0x5612fcb5e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:26.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.889+0000 7f6ae659c700 1 -- 172.21.15.5:0/4281698352 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5612fcb4c6a0 msgr2=0x5612fcb5b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:26.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.889+0000 7f6ae659c700 1 --2- 172.21.15.5:0/4281698352 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5612fcb4c6a0 0x5612fcb5b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:26.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.889+0000 7f6ae659c700 1 -- 172.21.15.5:0/4281698352 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612fcb5b870 con 0x5612fc9cf4a0 2024-05-09T01:00:26.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.890+0000 7f6ae659c700 1 --2- 172.21.15.5:0/4281698352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612fc9cf4a0 0x5612fcb5e2b0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f6ad80a99e0 tx=0x7f6ad80a8f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30a3026cf95d2fb2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:26.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.890+0000 7f6acdffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T01:00:26.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.890+0000 7f6acdffb700 1 -- 172.21.15.5:0/4281698352 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad80ca020 con 0x5612fc9cf4a0 2024-05-09T01:00:26.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.890+0000 7f6acdffb700 1 -- 172.21.15.5:0/4281698352 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f6ad80b2420 con 0x5612fc9cf4a0 2024-05-09T01:00:26.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.890+0000 7f6afb925b80 1 -- 172.21.15.5:0/4281698352 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ad80b2040 con 0x5612fc9cf4a0 2024-05-09T01:00:26.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.890+0000 7f6acdffb700 1 -- 172.21.15.5:0/4281698352 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6ad80a5040 con 0x5612fc9cf4a0 2024-05-09T01:00:26.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.890+0000 7f6afb925b80 10 client.17001 fetch_fsmap learned FSMap version 32 2024-05-09T01:00:26.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.890+0000 7f6afb925b80 1 -- 172.21.15.5:0/4281698352 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5612fcbd1140 con 0x5612fc9cf4a0 2024-05-09T01:00:26.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.890+0000 7f6acdffb700 1 -- 172.21.15.5:0/4281698352 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ad80b2040 con 0x5612fc9cf4a0 2024-05-09T01:00:26.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.890+0000 7f6acdffb700 1 -- 172.21.15.5:0/4281698352 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f6ad80aef90 con 0x5612fc9cf4a0 2024-05-09T01:00:26.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.890+0000 7f6afb925b80 10 client.17001 fetch_fsmap finished waiting for FSMap version 32 2024-05-09T01:00:26.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.890+0000 7f6afb925b80 10 client.17001 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T01:00:26.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.890+0000 7f6afb925b80 20 client.17001 populate_metadata read hostname 'smithi005' 2024-05-09T01:00:26.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.890+0000 7f6afb925b80 1 --2- 172.21.15.5:0/4281698352 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5612fcbd1500 0x5612fcbd3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:26.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.890+0000 7f6afb925b80 4 client.17001 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T01:00:26.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.890+0000 7f6afb925b80 1 -- 172.21.15.5:0/4281698352 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5612fca76a70 con 0x5612fcbd1500 2024-05-09T01:00:26.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.891+0000 7f6ae5d9b700 1 --2- 172.21.15.5:0/4281698352 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5612fcbd1500 0x5612fcbd3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:26.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.891+0000 7f6ae5d9b700 1 --2- 172.21.15.5:0/4281698352 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5612fcbd1500 0x5612fcbd3920 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66a5d767092baa4f server_cookie=5767c686de97d847 in_seq=0 out_seq=0 2024-05-09T01:00:26.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.891+0000 7f6acdffb700 10 client.17001 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T01:00:26.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.893+0000 7f6acdffb700 1 -- 172.21.15.5:0/4281698352 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5612fca76a70 con 0x5612fcbd1500 2024-05-09T01:00:26.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.893+0000 7f6acdffb700 10 client.17001 handle_command_reply: tid=0 2024-05-09T01:00:26.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.893+0000 7f6afb925b80 1 client.17001 shutdown 2024-05-09T01:00:26.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.893+0000 7f6afb925b80 1 -- 172.21.15.5:0/4281698352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612fc9cf4a0 msgr2=0x5612fcb5e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:26.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.893+0000 7f6afb925b80 1 --2- 172.21.15.5:0/4281698352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612fc9cf4a0 0x5612fcb5e2b0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f6ad80a99e0 tx=0x7f6ad80a8f60 comp rx=0 tx=0).stop 2024-05-09T01:00:26.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.893+0000 7f6afb925b80 1 -- 172.21.15.5:0/4281698352 shutdown_connections 2024-05-09T01:00:26.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.893+0000 7f6afb925b80 1 --2- 172.21.15.5:0/4281698352 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5612fcbd1500 0x5612fcbd3920 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:26.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.893+0000 7f6afb925b80 1 --2- 172.21.15.5:0/4281698352 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5612fcb4c6a0 0x5612fcb5b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:26.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.893+0000 7f6afb925b80 1 --2- 172.21.15.5:0/4281698352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612fc9cf4a0 0x5612fcb5e2b0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:26.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.893+0000 7f6afb925b80 1 --2- 172.21.15.5:0/4281698352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612fcab9160 0x5612fcb5e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:26.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.893+0000 7f6afb925b80 1 -- 172.21.15.5:0/4281698352 >> 172.21.15.5:0/4281698352 conn(0x5612fca2f6a0 msgr2=0x5612fcb549f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:26.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.893+0000 7f6acdffb700 0 client.17001 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T01:00:26.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.893+0000 7f6afb925b80 1 -- 172.21.15.5:0/4281698352 shutdown_connections 2024-05-09T01:00:26.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.893+0000 7f6afb925b80 1 -- 172.21.15.5:0/4281698352 wait complete. 2024-05-09T01:00:26.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.893+0000 7f6afb925b80 20 client.17001 trim_cache size 0 max 16384 2024-05-09T01:00:26.898 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-09T01:00:26.898 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-05-09T01:00:26.898 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-05-09T01:00:26.898 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-09T01:00:26.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.894+0000 7f6aecf0f700 1 -- 172.21.15.5:0/2543737353 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f6ad00b2860 msgr2=0x7f6ad00b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:26.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.894+0000 7f6aecf0f700 1 --2- 172.21.15.5:0/2543737353 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f6ad00b2860 0x7f6ad00b4d70 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8102b50 tx=0x7f6adc009260 comp rx=0 tx=0).stop 2024-05-09T01:00:26.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.894+0000 7f6aecf0f700 1 -- 172.21.15.5:0/2543737353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ae8103550 msgr2=0x7f6ae819d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:26.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.894+0000 7f6aecf0f700 1 --2- 172.21.15.5:0/2543737353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ae8103550 0x7f6ae819d030 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f6ad800e9a0 tx=0x7f6ad800ecb0 comp rx=0 tx=0).stop 2024-05-09T01:00:26.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.894+0000 7f6aecf0f700 1 -- 172.21.15.5:0/2543737353 shutdown_connections 2024-05-09T01:00:26.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.894+0000 7f6aecf0f700 1 --2- 172.21.15.5:0/2543737353 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f6ad00b2860 0x7f6ad00b4d70 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:26.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.894+0000 7f6aecf0f700 1 --2- 172.21.15.5:0/2543737353 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ae8108780 0x7f6ae81a1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:26.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.894+0000 7f6aecf0f700 1 --2- 172.21.15.5:0/2543737353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ae8103550 0x7f6ae819d030 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:26.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.894+0000 7f6aecf0f700 1 --2- 172.21.15.5:0/2543737353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ae8107e30 0x7f6ae819d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:26.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.894+0000 7f6aecf0f700 1 -- 172.21.15.5:0/2543737353 >> 172.21.15.5:0/2543737353 conn(0x7f6ae8078e00 msgr2=0x7f6ae8100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:26.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.894+0000 7f6aecf0f700 1 -- 172.21.15.5:0/2543737353 shutdown_connections 2024-05-09T01:00:26.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:26.894+0000 7f6aecf0f700 1 -- 172.21.15.5:0/2543737353 wait complete. 2024-05-09T01:00:27.707 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T01:00:27.707 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T01:00:27.974 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:00:28.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:00:28 smithi005 ceph-mon[30365]: pgmap v794: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 65 MiB/s wr, 419 op/s 2024-05-09T01:00:28.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:00:28 smithi131 ceph-mon[34547]: pgmap v794: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 65 MiB/s wr, 419 op/s 2024-05-09T01:00:28.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:00:28 smithi157 ceph-mon[34294]: pgmap v794: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 65 MiB/s wr, 419 op/s 2024-05-09T01:00:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:28.789+0000 7f74d8ff9640 1 -- 172.21.15.5:0/2793474499 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56234d662240 con 0x56234d2e61c0 2024-05-09T01:00:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:28.789+0000 7f74d8ff9640 10 client.16983 handle_command_reply: tid=0 2024-05-09T01:00:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:28.789+0000 7f74fe7b3740 1 client.16983 shutdown 2024-05-09T01:00:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:28.789+0000 7f74fe7b3740 1 -- 172.21.15.5:0/2793474499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56234d3292f0 msgr2=0x56234d70d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:28.789+0000 7f74fe7b3740 1 --2- 172.21.15.5:0/2793474499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56234d3292f0 0x56234d70d4f0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f74e8009a60 tx=0x7f74e80094a0 comp rx=0 tx=0).stop 2024-05-09T01:00:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:28.789+0000 7f74fe7b3740 1 -- 172.21.15.5:0/2793474499 shutdown_connections 2024-05-09T01:00:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:28.789+0000 7f74fe7b3740 1 --2- 172.21.15.5:0/2793474499 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56234d2e61c0 0x56234d780250 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:28.789+0000 7f74fe7b3740 1 --2- 172.21.15.5:0/2793474499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56234d3521b0 0x56234d77c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:28.789+0000 7f74fe7b3740 1 --2- 172.21.15.5:0/2793474499 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56234d3314d0 0x56234d70da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:28.789+0000 7f74d8ff9640 0 client.16983 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T01:00:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:28.789+0000 7f74fe7b3740 1 --2- 172.21.15.5:0/2793474499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56234d3292f0 0x56234d70d4f0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:28.789+0000 7f74fe7b3740 1 -- 172.21.15.5:0/2793474499 >> 172.21.15.5:0/2793474499 conn(0x56234d346300 msgr2=0x56234d6f4d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:28.789+0000 7f74fe7b3740 1 -- 172.21.15.5:0/2793474499 shutdown_connections 2024-05-09T01:00:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:28.789+0000 7f74fe7b3740 1 -- 172.21.15.5:0/2793474499 wait complete. 2024-05-09T01:00:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:28.789+0000 7f74fe7b3740 20 client.16983 trim_cache size 0 max 16384 2024-05-09T01:00:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-09T01:00:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-09T01:00:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-09T01:00:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:28.790+0000 7f74f7829640 1 -- 172.21.15.5:0/359922581 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f74dc0b2860 msgr2=0x7f74dc0b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:28.790+0000 7f74f7829640 1 --2- 172.21.15.5:0/359922581 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f74dc0b2860 0x7f74dc0b4d70 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x56234d6f78e0 tx=0x7f74ec0e80e0 comp rx=0 tx=0).stop 2024-05-09T01:00:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:28.790+0000 7f74f7829640 1 -- 172.21.15.5:0/359922581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74f0104760 msgr2=0x7f74f0103bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:28.790+0000 7f74f7829640 1 --2- 172.21.15.5:0/359922581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74f0104760 0x7f74f0103bb0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f74ec00e9f0 tx=0x7f74ec00eeb0 comp rx=0 tx=0).stop 2024-05-09T01:00:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:28.790+0000 7f74f7829640 1 -- 172.21.15.5:0/359922581 shutdown_connections 2024-05-09T01:00:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:28.790+0000 7f74f7829640 1 --2- 172.21.15.5:0/359922581 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f74dc0b2860 0x7f74dc0b4d70 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:28.790+0000 7f74f7829640 1 --2- 172.21.15.5:0/359922581 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74f0109a20 0x7f74f0102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:28.790+0000 7f74f7829640 1 --2- 172.21.15.5:0/359922581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74f01070c0 0x7f74f0102250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:28.790+0000 7f74f7829640 1 --2- 172.21.15.5:0/359922581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74f0104760 0x7f74f0103bb0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:28.790+0000 7f74f7829640 1 -- 172.21.15.5:0/359922581 >> 172.21.15.5:0/359922581 conn(0x7f74f00fdd60 msgr2=0x7f74f01055a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:28.790+0000 7f74f7829640 1 -- 172.21.15.5:0/359922581 shutdown_connections 2024-05-09T01:00:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:28.790+0000 7f74f7829640 1 -- 172.21.15.5:0/359922581 wait complete. 2024-05-09T01:00:29.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:29.694+0000 7f56f0cb4700 1 -- 172.21.15.5:0/371260634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56ec107df0 msgr2=0x7f56ec108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:29.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:29.694+0000 7f56f0cb4700 1 --2- 172.21.15.5:0/371260634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56ec107df0 0x7f56ec108200 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f56d40099c0 tx=0x7f56d4009cd0 comp rx=0 tx=0).stop 2024-05-09T01:00:29.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:29.694+0000 7f56f0cb4700 1 -- 172.21.15.5:0/371260634 shutdown_connections 2024-05-09T01:00:29.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:29.694+0000 7f56f0cb4700 1 --2- 172.21.15.5:0/371260634 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f56ec108740 0x7f56ec1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:29.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:29.694+0000 7f56f0cb4700 1 --2- 172.21.15.5:0/371260634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56ec1034c0 0x7f56ec1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:29.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:29.694+0000 7f56f0cb4700 1 --2- 172.21.15.5:0/371260634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56ec107df0 0x7f56ec108200 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:29.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:29.694+0000 7f56f0cb4700 1 -- 172.21.15.5:0/371260634 >> 172.21.15.5:0/371260634 conn(0x7f56ec0fedf0 msgr2=0x7f56ec101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:29.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:29.694+0000 7f56f0cb4700 1 -- 172.21.15.5:0/371260634 shutdown_connections 2024-05-09T01:00:29.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:29.695+0000 7f56f0cb4700 1 -- 172.21.15.5:0/371260634 wait complete. 2024-05-09T01:00:29.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:29.695+0000 7f56f0cb4700 1 Processor -- start 2024-05-09T01:00:29.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:29.696+0000 7f56f0cb4700 1 -- start start 2024-05-09T01:00:29.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:29.696+0000 7f56f0cb4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56ec1034c0 0x7f56ec19bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:29.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:29.696+0000 7f56f0cb4700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56ec107df0 0x7f56ec19c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:29.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:29.696+0000 7f56f0cb4700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f56ec108740 0x7f56ec19c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:29.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:29.696+0000 7f56f0cb4700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f56ec113350 con 0x7f56ec1034c0 2024-05-09T01:00:29.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:29.696+0000 7f56f0cb4700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f56ec1131d0 con 0x7f56ec108740 2024-05-09T01:00:29.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:29.696+0000 7f56f0cb4700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f56ec1134d0 con 0x7f56ec107df0 2024-05-09T01:00:29.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:29.696+0000 7f56ea59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56ec1034c0 0x7f56ec19bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:29.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:29.696+0000 7f56e1d9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56ec107df0 0x7f56ec19c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:29.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:29.696+0000 7f56ea59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56ec1034c0 0x7f56ec19bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57198/0 (socket says 172.21.15.5:57198) 2024-05-09T01:00:29.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:29.696+0000 7f56ea59c700 1 -- 172.21.15.5:0/2446986849 learned_addr learned my addr 172.21.15.5:0/2446986849 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:00:29.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:29.696+0000 7f56e1d9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56ec107df0 0x7f56ec19c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:58018/0 (socket says 172.21.15.5:58018) 2024-05-09T01:00:29.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:29.696+0000 7f56ead9d700 1 --2- 172.21.15.5:0/2446986849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f56ec108740 0x7f56ec19c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:29.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:29.696+0000 7f56ea59c700 1 -- 172.21.15.5:0/2446986849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56ec107df0 msgr2=0x7f56ec19c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:29.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:29.696+0000 7f56ea59c700 1 --2- 172.21.15.5:0/2446986849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56ec107df0 0x7f56ec19c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:29.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:29.696+0000 7f56ea59c700 1 -- 172.21.15.5:0/2446986849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f56ec108740 msgr2=0x7f56ec19c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:29.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:29.696+0000 7f56ea59c700 1 --2- 172.21.15.5:0/2446986849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f56ec108740 0x7f56ec19c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:29.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:29.696+0000 7f56ea59c700 1 -- 172.21.15.5:0/2446986849 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56ec1a7060 con 0x7f56ec1034c0 2024-05-09T01:00:29.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:29.696+0000 7f56ead9d700 1 --2- 172.21.15.5:0/2446986849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f56ec108740 0x7f56ec19c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:00:29.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:29.697+0000 7f56ea59c700 1 --2- 172.21.15.5:0/2446986849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56ec1034c0 0x7f56ec19bd10 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f56dc00d950 tx=0x7f56dc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:29.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:29.697+0000 7f56e3fff700 1 -- 172.21.15.5:0/2446986849 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56dc016070 con 0x7f56ec1034c0 2024-05-09T01:00:29.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:29.697+0000 7f56e3fff700 1 -- 172.21.15.5:0/2446986849 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56dc009df0 con 0x7f56ec1034c0 2024-05-09T01:00:29.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:29.697+0000 7f56f0cb4700 1 -- 172.21.15.5:0/2446986849 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56ec1a7350 con 0x7f56ec1034c0 2024-05-09T01:00:29.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:29.697+0000 7f56e3fff700 1 -- 172.21.15.5:0/2446986849 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56dc00f3c0 con 0x7f56ec1034c0 2024-05-09T01:00:29.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:29.697+0000 7f56f0cb4700 1 -- 172.21.15.5:0/2446986849 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56ec1a7860 con 0x7f56ec1034c0 2024-05-09T01:00:29.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:29.698+0000 7f56f0cb4700 1 -- 172.21.15.5:0/2446986849 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56ec0512c0 con 0x7f56ec1034c0 2024-05-09T01:00:29.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:29.701+0000 7f56e3fff700 1 -- 172.21.15.5:0/2446986849 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f56dc019070 con 0x7f56ec1034c0 2024-05-09T01:00:29.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:29.701+0000 7f56e3fff700 1 --2- 172.21.15.5:0/2446986849 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f56d80b2880 0x7f56d80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:29.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:29.702+0000 7f56e1d9b700 1 --2- 172.21.15.5:0/2446986849 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f56d80b2880 0x7f56d80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:29.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:29.702+0000 7f56e3fff700 1 -- 172.21.15.5:0/2446986849 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f56dc0e1e50 con 0x7f56ec1034c0 2024-05-09T01:00:29.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:29.702+0000 7f56e1d9b700 1 --2- 172.21.15.5:0/2446986849 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f56d80b2880 0x7f56d80b4d90 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f56d4006010 tx=0x7f56d4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:29.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:29.705+0000 7f56e3fff700 1 -- 172.21.15.5:0/2446986849 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f56dc0a9b40 con 0x7f56ec1034c0 2024-05-09T01:00:29.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:29.966+0000 7f56f0cb4700 1 -- 172.21.15.5:0/2446986849 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f56ec02e5f0 con 0x7f56ec1034c0 2024-05-09T01:00:29.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:29.967+0000 7f56e3fff700 1 -- 172.21.15.5:0/2446986849 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11928 (secure 0 0 0) 0x7f56dc0a9440 con 0x7f56ec1034c0 2024-05-09T01:00:29.968 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T01:00:29.968 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2685771887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2685771887},{"type":"v1","addr":"172.21.15.5:6835","nonce":2685771887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/254017535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":254017535},{"type":"v1","addr":"172.21.15.5:6837","nonce":254017535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T00:43:45.856066+0000","modified":"2024-05-09T01:00:13.979847+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24508,"mds_1":24499,"mds_2":24494,"mds_3":24493,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14598":{"gid":14598,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2109255228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2109255228},{"type":"v1","addr":"172.21.15.5:6839","nonce":2109255228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/3809966329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3809966329},{"type":"v1","addr":"172.21.15.5:6841","nonce":3809966329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6833/3272897742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3272897742},{"type":"v1","addr":"172.21.15.131:6833","nonce":3272897742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1283093898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1283093898},{"type":"v1","addr":"172.21.15.131:6835","nonce":1283093898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/2809525455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2809525455},{"type":"v1","addr":"172.21.15.157:6833","nonce":2809525455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.131:6837/1633006186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":1633006186},{"type":"v1","addr":"172.21.15.131:6837","nonce":1633006186}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.157:6835/832898273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":832898273},{"type":"v1","addr":"172.21.15.157:6835","nonce":832898273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.131:6839/2748669631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2748669631},{"type":"v1","addr":"172.21.15.131:6839","nonce":2748669631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.157:6837/744863009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":744863009},{"type":"v1","addr":"172.21.15.157:6837","nonce":744863009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":255,"addr":"172.21.15.157:6839/2779860399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":2779860399},{"type":"v1","addr":"172.21.15.157:6839","nonce":2779860399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24467,24493,24499,24508,24494]},"id":1}]} 2024-05-09T01:00:29.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:29.969+0000 7f56f0cb4700 1 -- 172.21.15.5:0/2446986849 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f56d80b2880 msgr2=0x7f56d80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:29.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:29.969+0000 7f56f0cb4700 1 --2- 172.21.15.5:0/2446986849 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f56d80b2880 0x7f56d80b4d90 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f56d4006010 tx=0x7f56d4000bc0 comp rx=0 tx=0).stop 2024-05-09T01:00:29.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:29.969+0000 7f56f0cb4700 1 -- 172.21.15.5:0/2446986849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56ec1034c0 msgr2=0x7f56ec19bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:29.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:29.969+0000 7f56f0cb4700 1 --2- 172.21.15.5:0/2446986849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56ec1034c0 0x7f56ec19bd10 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f56dc00d950 tx=0x7f56dc00dc60 comp rx=0 tx=0).stop 2024-05-09T01:00:29.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:29.969+0000 7f56f0cb4700 1 -- 172.21.15.5:0/2446986849 shutdown_connections 2024-05-09T01:00:29.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:29.969+0000 7f56f0cb4700 1 --2- 172.21.15.5:0/2446986849 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f56d80b2880 0x7f56d80b4d90 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:29.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:29.970+0000 7f56f0cb4700 1 --2- 172.21.15.5:0/2446986849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f56ec108740 0x7f56ec19c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:29.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:29.970+0000 7f56f0cb4700 1 --2- 172.21.15.5:0/2446986849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56ec1034c0 0x7f56ec19bd10 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:29.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:29.970+0000 7f56f0cb4700 1 --2- 172.21.15.5:0/2446986849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56ec107df0 0x7f56ec19c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:29.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:29.970+0000 7f56f0cb4700 1 -- 172.21.15.5:0/2446986849 >> 172.21.15.5:0/2446986849 conn(0x7f56ec0fedf0 msgr2=0x7f56ec1096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:29.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:29.970+0000 7f56f0cb4700 1 -- 172.21.15.5:0/2446986849 shutdown_connections 2024-05-09T01:00:29.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:29.970+0000 7f56f0cb4700 1 -- 172.21.15.5:0/2446986849 wait complete. 2024-05-09T01:00:29.972 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 32 2024-05-09T01:00:30.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:00:30 smithi005 ceph-mon[30365]: pgmap v795: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 55 MiB/s wr, 406 op/s 2024-05-09T01:00:30.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:00:30 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/2446986849' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:00:30.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:00:30 smithi131 ceph-mon[34547]: pgmap v795: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 55 MiB/s wr, 406 op/s 2024-05-09T01:00:30.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:00:30 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/2446986849' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:00:30.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:00:30 smithi157 ceph-mon[34294]: pgmap v795: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 55 MiB/s wr, 406 op/s 2024-05-09T01:00:30.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:00:30 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/2446986849' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:00:30.838 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-09T01:00:31.099 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:00:32.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:00:32 smithi131 ceph-mon[34547]: pgmap v796: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 37 MiB/s wr, 312 op/s 2024-05-09T01:00:32.393 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:00:32 smithi005 ceph-mon[30365]: pgmap v796: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 37 MiB/s wr, 312 op/s 2024-05-09T01:00:32.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:00:32 smithi157 ceph-mon[34294]: pgmap v796: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 37 MiB/s wr, 312 op/s 2024-05-09T01:00:32.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.983+0000 7fdfa28c8700 1 -- 172.21.15.5:0/2797700350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf9c1045f0 msgr2=0x7fdf9c104a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:32.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.983+0000 7fdfa28c8700 1 --2- 172.21.15.5:0/2797700350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf9c1045f0 0x7fdf9c104a00 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c0099c0 tx=0x7fdf8c009cd0 comp rx=0 tx=0).stop 2024-05-09T01:00:32.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.983+0000 7fdfa28c8700 1 -- 172.21.15.5:0/2797700350 shutdown_connections 2024-05-09T01:00:32.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.983+0000 7fdfa28c8700 1 --2- 172.21.15.5:0/2797700350 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf9c109870 0x7fdf9c1111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:32.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.983+0000 7fdfa28c8700 1 --2- 172.21.15.5:0/2797700350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf9c1045f0 0x7fdf9c104a00 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:32.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.983+0000 7fdfa28c8700 1 --2- 172.21.15.5:0/2797700350 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf9c108f20 0x7fdf9c109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:32.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.983+0000 7fdfa28c8700 1 -- 172.21.15.5:0/2797700350 >> 172.21.15.5:0/2797700350 conn(0x7fdf9c100040 msgr2=0x7fdf9c102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:32.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.983+0000 7fdfa28c8700 1 -- 172.21.15.5:0/2797700350 shutdown_connections 2024-05-09T01:00:32.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.983+0000 7fdfa28c8700 1 -- 172.21.15.5:0/2797700350 wait complete. 2024-05-09T01:00:32.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.983+0000 7fdfa28c8700 1 Processor -- start 2024-05-09T01:00:32.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.984+0000 7fdfa28c8700 1 -- start start 2024-05-09T01:00:32.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.984+0000 7fdfa28c8700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf9c1045f0 0x7fdf9c075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:32.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.984+0000 7fdfa28c8700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf9c108f20 0x7fdf9c076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:32.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.984+0000 7fdfa28c8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf9c109870 0x7fdf9c076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:32.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.984+0000 7fdfa28c8700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9c1144f0 con 0x7fdf9c109870 2024-05-09T01:00:32.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.984+0000 7fdfa28c8700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9c114370 con 0x7fdf9c1045f0 2024-05-09T01:00:32.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.984+0000 7fdfa28c8700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9c114670 con 0x7fdf9c108f20 2024-05-09T01:00:32.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.984+0000 7fdf9b7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf9c108f20 0x7fdf9c076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:32.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.984+0000 7fdfa0e65700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf9c109870 0x7fdf9c076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:32.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.984+0000 7fdf9b7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf9c108f20 0x7fdf9c076000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:58040/0 (socket says 172.21.15.5:58040) 2024-05-09T01:00:32.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.984+0000 7fdf9b7fe700 1 -- 172.21.15.5:0/3718137592 learned_addr learned my addr 172.21.15.5:0/3718137592 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:00:32.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.984+0000 7fdf9bfff700 1 --2- 172.21.15.5:0/3718137592 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf9c1045f0 0x7fdf9c075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:32.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.985+0000 7fdf9b7fe700 1 -- 172.21.15.5:0/3718137592 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf9c1045f0 msgr2=0x7fdf9c075ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:32.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.985+0000 7fdf9b7fe700 1 --2- 172.21.15.5:0/3718137592 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf9c1045f0 0x7fdf9c075ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:32.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.985+0000 7fdf9b7fe700 1 -- 172.21.15.5:0/3718137592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf9c109870 msgr2=0x7fdf9c076540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:32.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.985+0000 7fdf9b7fe700 1 --2- 172.21.15.5:0/3718137592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf9c109870 0x7fdf9c076540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:32.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.985+0000 7fdf9b7fe700 1 -- 172.21.15.5:0/3718137592 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf9c1a9260 con 0x7fdf9c108f20 2024-05-09T01:00:32.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.985+0000 7fdfa0e65700 1 --2- 172.21.15.5:0/3718137592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf9c109870 0x7fdf9c076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:00:32.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.985+0000 7fdf9b7fe700 1 --2- 172.21.15.5:0/3718137592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf9c108f20 0x7fdf9c076000 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdf9000d950 tx=0x7fdf9000dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:32.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.985+0000 7fdf997fa700 1 -- 172.21.15.5:0/3718137592 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf90016070 con 0x7fdf9c108f20 2024-05-09T01:00:32.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.985+0000 7fdf997fa700 1 -- 172.21.15.5:0/3718137592 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf90004830 con 0x7fdf9c108f20 2024-05-09T01:00:32.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.985+0000 7fdfa28c8700 1 -- 172.21.15.5:0/3718137592 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf9c1a9550 con 0x7fdf9c108f20 2024-05-09T01:00:32.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.985+0000 7fdf997fa700 1 -- 172.21.15.5:0/3718137592 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf90005520 con 0x7fdf9c108f20 2024-05-09T01:00:32.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.985+0000 7fdfa28c8700 1 -- 172.21.15.5:0/3718137592 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf9c1a9a90 con 0x7fdf9c108f20 2024-05-09T01:00:32.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.988+0000 7fdf997fa700 1 -- 172.21.15.5:0/3718137592 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fdf9000f690 con 0x7fdf9c108f20 2024-05-09T01:00:32.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.989+0000 7fdfb12deb80 1 Processor -- start 2024-05-09T01:00:32.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.989+0000 7fdfb12deb80 1 -- start start 2024-05-09T01:00:32.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.989+0000 7fdf997fa700 1 --2- 172.21.15.5:0/3718137592 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fdf840b2880 0x7fdf840b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:32.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.989+0000 7fdfb12deb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5654cfef4160 0x5654cff8d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:32.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.989+0000 7fdfb12deb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654cfe0a4a0 0x5654cff8da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:32.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.989+0000 7fdfb12deb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5654cff876a0 0x5654cff87ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:32.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.989+0000 7fdfb12deb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5654cfea9ff0 con 0x5654cfe0a4a0 2024-05-09T01:00:32.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.989+0000 7fdfb12deb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5654cfe487f0 con 0x5654cfef4160 2024-05-09T01:00:32.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.989+0000 7fdfb12deb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5654cfea5fb0 con 0x5654cff876a0 2024-05-09T01:00:32.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.989+0000 7fdfa0e65700 1 --2- 172.21.15.5:0/3718137592 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fdf840b2880 0x7fdf840b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:32.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.989+0000 7fdf997fa700 1 -- 172.21.15.5:0/3718137592 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fdf900e1f00 con 0x7fdf9c108f20 2024-05-09T01:00:32.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.989+0000 7fdf9b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654cfe0a4a0 0x5654cff8da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:32.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.990+0000 7fdfa0e65700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5654cff876a0 0x5654cff87ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:32.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.990+0000 7fdf9b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654cfe0a4a0 0x5654cff8da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57228/0 (socket says 172.21.15.5:57228) 2024-05-09T01:00:32.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.990+0000 7fdf9b7fe700 1 -- 172.21.15.5:0/3771886673 learned_addr learned my addr 172.21.15.5:0/3771886673 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:00:32.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.990+0000 7fdf9bfff700 1 --2- 172.21.15.5:0/3771886673 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5654cfef4160 0x5654cff8d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:32.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.990+0000 7fdf9b7fe700 1 -- 172.21.15.5:0/3771886673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5654cff876a0 msgr2=0x5654cff87ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:32.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.990+0000 7fdf9b7fe700 1 --2- 172.21.15.5:0/3771886673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5654cff876a0 0x5654cff87ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:32.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.990+0000 7fdf9b7fe700 1 -- 172.21.15.5:0/3771886673 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5654cfef4160 msgr2=0x5654cff8d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:32.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.990+0000 7fdf9b7fe700 1 --2- 172.21.15.5:0/3771886673 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5654cfef4160 0x5654cff8d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:32.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.990+0000 7fdf9b7fe700 1 -- 172.21.15.5:0/3771886673 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654cfe40370 con 0x5654cfe0a4a0 2024-05-09T01:00:32.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.990+0000 7fdf9b7fe700 1 --2- 172.21.15.5:0/3771886673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654cfe0a4a0 0x5654cff8da30 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fdf900df9d0 tx=0x7fdf9004ea60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5d3d73f56642c7f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:32.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.990+0000 7fdfa0e65700 1 --2- 172.21.15.5:0/3718137592 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fdf840b2880 0x7fdf840b4d90 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x5654cff8e0c0 tx=0x7fdf94009920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:32.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.990+0000 7fdf827fc700 1 -- 172.21.15.5:0/3771886673 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf900910d0 con 0x5654cfe0a4a0 2024-05-09T01:00:33.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.990+0000 7fdf827fc700 1 -- 172.21.15.5:0/3771886673 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf900a94f0 con 0x5654cfe0a4a0 2024-05-09T01:00:33.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.990+0000 7fdf827fc700 1 -- 172.21.15.5:0/3771886673 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf90004d70 con 0x5654cfe0a4a0 2024-05-09T01:00:33.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.991+0000 7fdfb12deb80 1 -- 172.21.15.5:0/3771886673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654cfe0a4a0 msgr2=0x5654cff8da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:33.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.991+0000 7fdfb12deb80 1 --2- 172.21.15.5:0/3771886673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654cfe0a4a0 0x5654cff8da30 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fdf900df9d0 tx=0x7fdf9004ea60 comp rx=0 tx=0).stop 2024-05-09T01:00:33.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.991+0000 7fdfb12deb80 1 -- 172.21.15.5:0/3771886673 shutdown_connections 2024-05-09T01:00:33.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.991+0000 7fdfb12deb80 1 --2- 172.21.15.5:0/3771886673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5654cff876a0 0x5654cff87ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:33.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.991+0000 7fdfb12deb80 1 --2- 172.21.15.5:0/3771886673 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5654cfef4160 0x5654cff8d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:33.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.991+0000 7fdfb12deb80 1 --2- 172.21.15.5:0/3771886673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654cfe0a4a0 0x5654cff8da30 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:33.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.991+0000 7fdfb12deb80 1 -- 172.21.15.5:0/3771886673 >> 172.21.15.5:0/3771886673 conn(0x5654cfe6a6a0 msgr2=0x5654cff82400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:33.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.991+0000 7fdfb12deb80 1 -- 172.21.15.5:0/3771886673 shutdown_connections 2024-05-09T01:00:33.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.991+0000 7fdfb12deb80 1 -- 172.21.15.5:0/3771886673 wait complete. 2024-05-09T01:00:33.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.992+0000 7fdfb12deb80 1 -- start start 2024-05-09T01:00:33.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.992+0000 7fdfb12deb80 1 Processor -- start 2024-05-09T01:00:33.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.992+0000 7fdfb12deb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5654cfe0a4a0 0x5654cff91260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:33.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.992+0000 7fdfb12deb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654cfef4160 0x5654cff917a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:33.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.992+0000 7fdfb12deb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5654cff876a0 0x5654cff91ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:33.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.992+0000 7fdfb12deb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5654cfd25ba0 con 0x5654cfef4160 2024-05-09T01:00:33.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.992+0000 7fdfb12deb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5654cfe64150 con 0x5654cff876a0 2024-05-09T01:00:33.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.992+0000 7fdfb12deb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5654cff941d0 con 0x5654cfe0a4a0 2024-05-09T01:00:33.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.992+0000 7fdfb12deb80 10 client.0 _ll_register_callbacks cb 0x5654cfd1b6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:00:33.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.993+0000 7fdf9bfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5654cfe0a4a0 0x5654cff91260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:33.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.993+0000 7fdf9b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654cfef4160 0x5654cff917a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:33.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.993+0000 7fdf9bfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5654cfe0a4a0 0x5654cff91260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:58056/0 (socket says 172.21.15.5:58056) 2024-05-09T01:00:33.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.993+0000 7fdf9bfff700 1 -- 172.21.15.5:0/696012976 learned_addr learned my addr 172.21.15.5:0/696012976 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:00:33.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.993+0000 7fdfa0e65700 1 --2- 172.21.15.5:0/696012976 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5654cff876a0 0x5654cff91ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:33.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.993+0000 7fdf9b7fe700 1 -- 172.21.15.5:0/696012976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5654cfe0a4a0 msgr2=0x5654cff91260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:33.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.993+0000 7fdf9b7fe700 1 --2- 172.21.15.5:0/696012976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5654cfe0a4a0 0x5654cff91260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:33.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.993+0000 7fdf9b7fe700 1 -- 172.21.15.5:0/696012976 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5654cff876a0 msgr2=0x5654cff91ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:33.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.993+0000 7fdf9b7fe700 1 --2- 172.21.15.5:0/696012976 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5654cff876a0 0x5654cff91ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:33.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.993+0000 7fdf9b7fe700 1 -- 172.21.15.5:0/696012976 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654cfd25490 con 0x5654cfef4160 2024-05-09T01:00:33.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.993+0000 7fdf9b7fe700 1 --2- 172.21.15.5:0/696012976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654cfef4160 0x5654cff917a0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fdf900ada60 tx=0x7fdf900a93d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57003a43c3e3037f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:33.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.993+0000 7fdf7b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T01:00:33.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.993+0000 7fdf7b7fe700 1 -- 172.21.15.5:0/696012976 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf900910d0 con 0x5654cfef4160 2024-05-09T01:00:33.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.993+0000 7fdf7b7fe700 1 -- 172.21.15.5:0/696012976 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fdf90004cc0 con 0x5654cfef4160 2024-05-09T01:00:33.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.993+0000 7fdfb12deb80 1 -- 172.21.15.5:0/696012976 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf900a94f0 con 0x5654cfef4160 2024-05-09T01:00:33.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.993+0000 7fdf7b7fe700 1 -- 172.21.15.5:0/696012976 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fdf900049e0 con 0x5654cfef4160 2024-05-09T01:00:33.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.993+0000 7fdfb12deb80 10 client.17019 fetch_fsmap learned FSMap version 32 2024-05-09T01:00:33.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.994+0000 7fdfb12deb80 1 -- 172.21.15.5:0/696012976 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5654cff9ec40 con 0x5654cfef4160 2024-05-09T01:00:33.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.994+0000 7fdf7b7fe700 1 -- 172.21.15.5:0/696012976 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf900a94f0 con 0x5654cfef4160 2024-05-09T01:00:33.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.994+0000 7fdf7b7fe700 1 -- 172.21.15.5:0/696012976 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fdf900b1ec0 con 0x5654cfef4160 2024-05-09T01:00:33.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.994+0000 7fdfb12deb80 10 client.17019 fetch_fsmap finished waiting for FSMap version 32 2024-05-09T01:00:33.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.994+0000 7fdfb12deb80 10 client.17019 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T01:00:33.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.994+0000 7fdfb12deb80 20 client.17019 populate_metadata read hostname 'smithi005' 2024-05-09T01:00:33.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.994+0000 7fdfb12deb80 1 --2- 172.21.15.5:0/696012976 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x5654cffa1110 0x5654cffa3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:33.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.994+0000 7fdfb12deb80 4 client.17019 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:00:33.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.994+0000 7fdfb12deb80 1 -- 172.21.15.5:0/696012976 --> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654cfeb79d0 con 0x5654cffa1110 2024-05-09T01:00:33.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.994+0000 7fdf9bfff700 1 --2- 172.21.15.5:0/696012976 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x5654cffa1110 0x5654cffa3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:33.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.995+0000 7fdf9bfff700 1 --2- 172.21.15.5:0/696012976 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x5654cffa1110 0x5654cffa3530 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b3b40baa140622a9 server_cookie=41754d6c7287eef5 in_seq=0 out_seq=0 2024-05-09T01:00:33.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.995+0000 7fdf7b7fe700 10 client.17019 ms_handle_connect on v2:172.21.15.131:6836/1633006186 2024-05-09T01:00:33.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.996+0000 7fdf7b7fe700 1 -- 172.21.15.5:0/696012976 <== mds.4 v2:172.21.15.131:6836/1633006186 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5654cfeb79d0 con 0x5654cffa1110 2024-05-09T01:00:33.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.996+0000 7fdf7b7fe700 10 client.17019 handle_command_reply: tid=0 2024-05-09T01:00:33.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.996+0000 7fdfb12deb80 1 client.17019 shutdown 2024-05-09T01:00:33.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.996+0000 7fdfb12deb80 1 -- 172.21.15.5:0/696012976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654cfef4160 msgr2=0x5654cff917a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:33.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.996+0000 7fdfb12deb80 1 --2- 172.21.15.5:0/696012976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654cfef4160 0x5654cff917a0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fdf900ada60 tx=0x7fdf900a93d0 comp rx=0 tx=0).stop 2024-05-09T01:00:33.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.996+0000 7fdfb12deb80 1 -- 172.21.15.5:0/696012976 shutdown_connections 2024-05-09T01:00:33.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.996+0000 7fdfb12deb80 1 --2- 172.21.15.5:0/696012976 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x5654cffa1110 0x5654cffa3530 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:33.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.996+0000 7fdfb12deb80 1 --2- 172.21.15.5:0/696012976 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5654cff876a0 0x5654cff91ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:33.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.996+0000 7fdfb12deb80 1 --2- 172.21.15.5:0/696012976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5654cfe0a4a0 0x5654cff91260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:33.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.996+0000 7fdfb12deb80 1 --2- 172.21.15.5:0/696012976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654cfef4160 0x5654cff917a0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:33.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.996+0000 7fdfb12deb80 1 -- 172.21.15.5:0/696012976 >> 172.21.15.5:0/696012976 conn(0x5654cfe6a6a0 msgr2=0x5654cff821c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:33.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.996+0000 7fdf7b7fe700 0 client.17019 ms_handle_reset on v2:172.21.15.131:6836/1633006186 2024-05-09T01:00:33.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.996+0000 7fdfb12deb80 1 -- 172.21.15.5:0/696012976 shutdown_connections 2024-05-09T01:00:33.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.996+0000 7fdfb12deb80 1 -- 172.21.15.5:0/696012976 wait complete. 2024-05-09T01:00:33.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:32.996+0000 7fdfb12deb80 20 client.17019 trim_cache size 0 max 16384 2024-05-09T01:00:33.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.012+0000 7fdfb12deb80 1 Processor -- start 2024-05-09T01:00:33.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.012+0000 7fdfb12deb80 1 -- start start 2024-05-09T01:00:33.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.012+0000 7fdfb12deb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5654cfe0a4a0 0x5654cff870c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:33.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.012+0000 7fdfb12deb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654cfef4160 0x5654cff8c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:33.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.012+0000 7fdfb12deb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5654cff876a0 0x5654cff93020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:33.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.012+0000 7fdfb12deb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9c1b0fc0 con 0x5654cfef4160 2024-05-09T01:00:33.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.012+0000 7fdfb12deb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9c1b2b00 con 0x5654cff876a0 2024-05-09T01:00:33.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.012+0000 7fdfb12deb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9c1b1690 con 0x5654cfe0a4a0 2024-05-09T01:00:33.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.012+0000 7fdf9b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654cfef4160 0x5654cff8c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:33.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.012+0000 7fdf9b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654cfef4160 0x5654cff8c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57250/0 (socket says 172.21.15.5:57250) 2024-05-09T01:00:33.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.012+0000 7fdf9b7fe700 1 -- 172.21.15.5:0/867431830 learned_addr learned my addr 172.21.15.5:0/867431830 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:00:33.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.012+0000 7fdf9bfff700 1 --2- 172.21.15.5:0/867431830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5654cfe0a4a0 0x5654cff870c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:33.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.012+0000 7fdfa0e65700 1 --2- 172.21.15.5:0/867431830 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5654cff876a0 0x5654cff93020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:33.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.012+0000 7fdf9b7fe700 1 -- 172.21.15.5:0/867431830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5654cfe0a4a0 msgr2=0x5654cff870c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:33.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.012+0000 7fdf9b7fe700 1 --2- 172.21.15.5:0/867431830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5654cfe0a4a0 0x5654cff870c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:33.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.012+0000 7fdf9b7fe700 1 -- 172.21.15.5:0/867431830 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5654cff876a0 msgr2=0x5654cff93020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:33.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.012+0000 7fdf9b7fe700 1 --2- 172.21.15.5:0/867431830 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5654cff876a0 0x5654cff93020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:33.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.012+0000 7fdf9b7fe700 1 -- 172.21.15.5:0/867431830 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654cff7bf60 con 0x5654cfef4160 2024-05-09T01:00:33.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.012+0000 7fdf9bfff700 1 --2- 172.21.15.5:0/867431830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5654cfe0a4a0 0x5654cff870c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:00:33.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.012+0000 7fdfa0e65700 1 --2- 172.21.15.5:0/867431830 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5654cff876a0 0x5654cff93020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:00:33.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.013+0000 7fdf9b7fe700 1 --2- 172.21.15.5:0/867431830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654cfef4160 0x5654cff8c9c0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fdf900a91a0 tx=0x7fdf90007620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27dd8261dfc3131 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:33.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.013+0000 7fdf81ffb700 1 -- 172.21.15.5:0/867431830 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf9004ee60 con 0x5654cfef4160 2024-05-09T01:00:33.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.013+0000 7fdf81ffb700 1 -- 172.21.15.5:0/867431830 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf900ad8a0 con 0x5654cfef4160 2024-05-09T01:00:33.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.013+0000 7fdf81ffb700 1 -- 172.21.15.5:0/867431830 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf900b1ec0 con 0x5654cfef4160 2024-05-09T01:00:33.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.014+0000 7fdfb12deb80 1 -- 172.21.15.5:0/867431830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654cfef4160 msgr2=0x5654cff8c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:33.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.014+0000 7fdfb12deb80 1 --2- 172.21.15.5:0/867431830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654cfef4160 0x5654cff8c9c0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fdf900a91a0 tx=0x7fdf90007620 comp rx=0 tx=0).stop 2024-05-09T01:00:33.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.014+0000 7fdfb12deb80 1 -- 172.21.15.5:0/867431830 shutdown_connections 2024-05-09T01:00:33.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.014+0000 7fdfb12deb80 1 --2- 172.21.15.5:0/867431830 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5654cff876a0 0x5654cff93020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:33.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.014+0000 7fdfb12deb80 1 --2- 172.21.15.5:0/867431830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5654cfe0a4a0 0x5654cff870c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:33.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.014+0000 7fdfb12deb80 1 --2- 172.21.15.5:0/867431830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654cfef4160 0x5654cff8c9c0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:33.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.014+0000 7fdfb12deb80 1 -- 172.21.15.5:0/867431830 >> 172.21.15.5:0/867431830 conn(0x5654cfe6a6a0 msgr2=0x5654cff92910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:33.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.014+0000 7fdfb12deb80 1 -- 172.21.15.5:0/867431830 shutdown_connections 2024-05-09T01:00:33.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.014+0000 7fdfb12deb80 1 -- 172.21.15.5:0/867431830 wait complete. 2024-05-09T01:00:33.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.014+0000 7fdfb12deb80 1 -- start start 2024-05-09T01:00:33.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.015+0000 7fdfb12deb80 1 Processor -- start 2024-05-09T01:00:33.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.015+0000 7fdfb12deb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5654cfe0a4a0 0x5654cffa6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:33.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.015+0000 7fdfb12deb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654cfef4160 0x5654cffa4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:33.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.015+0000 7fdfb12deb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5654cff876a0 0x5654cffa53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:33.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.015+0000 7fdfb12deb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9c1b2430 con 0x5654cfef4160 2024-05-09T01:00:33.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.015+0000 7fdfb12deb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9c1b1d60 con 0x5654cff876a0 2024-05-09T01:00:33.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.015+0000 7fdfb12deb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9c1aef60 con 0x5654cfe0a4a0 2024-05-09T01:00:33.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.015+0000 7fdfb12deb80 10 client.0 _ll_register_callbacks cb 0x5654cfd1b6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:00:33.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.015+0000 7fdf9b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654cfef4160 0x5654cffa4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:33.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.015+0000 7fdf9b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654cfef4160 0x5654cffa4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57266/0 (socket says 172.21.15.5:57266) 2024-05-09T01:00:33.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.015+0000 7fdf9bfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5654cfe0a4a0 0x5654cffa6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:33.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.015+0000 7fdf9b7fe700 1 -- 172.21.15.5:0/1789549818 learned_addr learned my addr 172.21.15.5:0/1789549818 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:00:33.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.015+0000 7fdfa0e65700 1 --2- 172.21.15.5:0/1789549818 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5654cff876a0 0x5654cffa53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:33.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.015+0000 7fdf9b7fe700 1 -- 172.21.15.5:0/1789549818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5654cfe0a4a0 msgr2=0x5654cffa6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:33.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.015+0000 7fdf9b7fe700 1 --2- 172.21.15.5:0/1789549818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5654cfe0a4a0 0x5654cffa6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:33.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.015+0000 7fdf9b7fe700 1 -- 172.21.15.5:0/1789549818 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5654cff876a0 msgr2=0x5654cffa53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:33.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.015+0000 7fdf9b7fe700 1 --2- 172.21.15.5:0/1789549818 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5654cff876a0 0x5654cffa53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:33.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.015+0000 7fdf9b7fe700 1 -- 172.21.15.5:0/1789549818 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654d0009660 con 0x5654cfef4160 2024-05-09T01:00:33.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.016+0000 7fdf9b7fe700 1 --2- 172.21.15.5:0/1789549818 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654cfef4160 0x5654cffa4d00 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fdf90007770 tx=0x7fdf90004ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30975341fb598c38 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:33.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.016+0000 7fdf7b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T01:00:33.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.016+0000 7fdf7b7fe700 1 -- 172.21.15.5:0/1789549818 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf900910d0 con 0x5654cfef4160 2024-05-09T01:00:33.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.016+0000 7fdf7b7fe700 1 -- 172.21.15.5:0/1789549818 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fdf9004ee00 con 0x5654cfef4160 2024-05-09T01:00:33.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.016+0000 7fdfb12deb80 1 -- 172.21.15.5:0/1789549818 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5654cfeb1a70 con 0x5654cfef4160 2024-05-09T01:00:33.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.016+0000 7fdf7b7fe700 1 -- 172.21.15.5:0/1789549818 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fdf900a9750 con 0x5654cfef4160 2024-05-09T01:00:33.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.016+0000 7fdfb12deb80 10 client.17025 fetch_fsmap learned FSMap version 32 2024-05-09T01:00:33.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.016+0000 7fdfb12deb80 1 -- 172.21.15.5:0/1789549818 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5654d0009660 con 0x5654cfef4160 2024-05-09T01:00:33.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.016+0000 7fdf7b7fe700 1 -- 172.21.15.5:0/1789549818 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5654cfeb1a70 con 0x5654cfef4160 2024-05-09T01:00:33.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.016+0000 7fdf7b7fe700 1 -- 172.21.15.5:0/1789549818 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fdf900a4f80 con 0x5654cfef4160 2024-05-09T01:00:33.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.016+0000 7fdfb12deb80 10 client.17025 fetch_fsmap finished waiting for FSMap version 32 2024-05-09T01:00:33.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.016+0000 7fdfb12deb80 10 client.17025 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T01:00:33.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.016+0000 7fdfb12deb80 20 client.17025 populate_metadata read hostname 'smithi005' 2024-05-09T01:00:33.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.016+0000 7fdfb12deb80 1 --2- 172.21.15.5:0/1789549818 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x5654d000aa50 0x5654d000ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:33.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.016+0000 7fdfb12deb80 4 client.17025 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:00:33.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.016+0000 7fdfb12deb80 1 -- 172.21.15.5:0/1789549818 --> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5654cfd25970 con 0x5654d000aa50 2024-05-09T01:00:33.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.017+0000 7fdf9bfff700 1 --2- 172.21.15.5:0/1789549818 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x5654d000aa50 0x5654d000ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:33.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.017+0000 7fdf7b7fe700 10 client.17025 ms_handle_connect on v2:172.21.15.131:6836/1633006186 2024-05-09T01:00:33.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.017+0000 7fdf9bfff700 1 --2- 172.21.15.5:0/1789549818 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x5654d000aa50 0x5654d000ce70 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1a80628b5b8a3ba9 server_cookie=578c72672d2568d2 in_seq=0 out_seq=0 2024-05-09T01:00:33.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.018+0000 7fdf7b7fe700 1 -- 172.21.15.5:0/1789549818 <== mds.4 v2:172.21.15.131:6836/1633006186 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5654cfd25970 con 0x5654d000aa50 2024-05-09T01:00:33.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.018+0000 7fdf7b7fe700 10 client.17025 handle_command_reply: tid=0 2024-05-09T01:00:33.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.018+0000 7fdfb12deb80 1 client.17025 shutdown 2024-05-09T01:00:33.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.018+0000 7fdfb12deb80 1 -- 172.21.15.5:0/1789549818 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654cfef4160 msgr2=0x5654cffa4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:33.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.018+0000 7fdfb12deb80 1 --2- 172.21.15.5:0/1789549818 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654cfef4160 0x5654cffa4d00 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fdf90007770 tx=0x7fdf90004ef0 comp rx=0 tx=0).stop 2024-05-09T01:00:33.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.018+0000 7fdfb12deb80 1 -- 172.21.15.5:0/1789549818 shutdown_connections 2024-05-09T01:00:33.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.018+0000 7fdfb12deb80 1 --2- 172.21.15.5:0/1789549818 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x5654d000aa50 0x5654d000ce70 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:33.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.018+0000 7fdfb12deb80 1 --2- 172.21.15.5:0/1789549818 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5654cff876a0 0x5654cffa53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:33.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.018+0000 7fdfb12deb80 1 --2- 172.21.15.5:0/1789549818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5654cfe0a4a0 0x5654cffa6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:33.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.018+0000 7fdfb12deb80 1 --2- 172.21.15.5:0/1789549818 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654cfef4160 0x5654cffa4d00 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:33.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.018+0000 7fdf7b7fe700 0 client.17025 ms_handle_reset on v2:172.21.15.131:6836/1633006186 2024-05-09T01:00:33.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.018+0000 7fdfb12deb80 1 -- 172.21.15.5:0/1789549818 >> 172.21.15.5:0/1789549818 conn(0x5654cfe6a6a0 msgr2=0x5654cff86e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:33.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.018+0000 7fdfb12deb80 1 -- 172.21.15.5:0/1789549818 shutdown_connections 2024-05-09T01:00:33.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.018+0000 7fdfb12deb80 1 -- 172.21.15.5:0/1789549818 wait complete. 2024-05-09T01:00:33.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.018+0000 7fdfb12deb80 20 client.17025 trim_cache size 0 max 16384 2024-05-09T01:00:33.024 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-09T01:00:33.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.019+0000 7fdfa28c8700 1 -- 172.21.15.5:0/3718137592 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fdf840b2880 msgr2=0x7fdf840b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:33.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.019+0000 7fdfa28c8700 1 --2- 172.21.15.5:0/3718137592 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fdf840b2880 0x7fdf840b4d90 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x5654cff8e0c0 tx=0x7fdf94009920 comp rx=0 tx=0).stop 2024-05-09T01:00:33.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.019+0000 7fdfa28c8700 1 -- 172.21.15.5:0/3718137592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf9c108f20 msgr2=0x7fdf9c076000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:33.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.019+0000 7fdfa28c8700 1 --2- 172.21.15.5:0/3718137592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf9c108f20 0x7fdf9c076000 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdf9000d950 tx=0x7fdf9000dc60 comp rx=0 tx=0).stop 2024-05-09T01:00:33.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.019+0000 7fdfa28c8700 1 -- 172.21.15.5:0/3718137592 shutdown_connections 2024-05-09T01:00:33.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.019+0000 7fdfa28c8700 1 --2- 172.21.15.5:0/3718137592 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fdf840b2880 0x7fdf840b4d90 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:33.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.019+0000 7fdfa28c8700 1 --2- 172.21.15.5:0/3718137592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf9c109870 0x7fdf9c076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:33.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.019+0000 7fdfa28c8700 1 --2- 172.21.15.5:0/3718137592 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf9c1045f0 0x7fdf9c075ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:33.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.019+0000 7fdfa28c8700 1 --2- 172.21.15.5:0/3718137592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf9c108f20 0x7fdf9c076000 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:33.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.019+0000 7fdfa28c8700 1 -- 172.21.15.5:0/3718137592 >> 172.21.15.5:0/3718137592 conn(0x7fdf9c100040 msgr2=0x7fdf9c108690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:33.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.019+0000 7fdfa28c8700 1 -- 172.21.15.5:0/3718137592 shutdown_connections 2024-05-09T01:00:33.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:33.019+0000 7fdfa28c8700 1 -- 172.21.15.5:0/3718137592 wait complete. 2024-05-09T01:00:33.817 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-09T01:00:34.072 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:00:34.317 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:00:34 smithi005 ceph-mon[30365]: pgmap v797: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 22 MiB/s wr, 206 op/s 2024-05-09T01:00:34.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:00:34 smithi131 ceph-mon[34547]: pgmap v797: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 22 MiB/s wr, 206 op/s 2024-05-09T01:00:34.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:00:34 smithi157 ceph-mon[34294]: pgmap v797: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 22 MiB/s wr, 206 op/s 2024-05-09T01:00:36.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.037+0000 7fca34e5c700 1 -- 172.21.15.5:0/1103917209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca30108770 msgr2=0x7fca301100d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:36.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.037+0000 7fca34e5c700 1 --2- 172.21.15.5:0/1103917209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca30108770 0x7fca301100d0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fca2800b0d0 tx=0x7fca2800b3e0 comp rx=0 tx=0).stop 2024-05-09T01:00:36.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.037+0000 7fca34e5c700 1 -- 172.21.15.5:0/1103917209 shutdown_connections 2024-05-09T01:00:36.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.037+0000 7fca34e5c700 1 --2- 172.21.15.5:0/1103917209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca30108770 0x7fca301100d0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:36.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.037+0000 7fca34e5c700 1 --2- 172.21.15.5:0/1103917209 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca30103540 0x7fca30103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:36.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.037+0000 7fca34e5c700 1 --2- 172.21.15.5:0/1103917209 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca30107e20 0x7fca30108230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:36.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.037+0000 7fca34e5c700 1 -- 172.21.15.5:0/1103917209 >> 172.21.15.5:0/1103917209 conn(0x7fca30078e50 msgr2=0x7fca30079290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:36.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.037+0000 7fca34e5c700 1 -- 172.21.15.5:0/1103917209 shutdown_connections 2024-05-09T01:00:36.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.037+0000 7fca34e5c700 1 -- 172.21.15.5:0/1103917209 wait complete. 2024-05-09T01:00:36.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.038+0000 7fca34e5c700 1 Processor -- start 2024-05-09T01:00:36.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.038+0000 7fca34e5c700 1 -- start start 2024-05-09T01:00:36.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.038+0000 7fca34e5c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca30103540 0x7fca301005c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:36.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.039+0000 7fca34e5c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca30107e20 0x7fca30100b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:36.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.039+0000 7fca34e5c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca30108770 0x7fca30101040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:36.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.039+0000 7fca34e5c700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fca301133a0 con 0x7fca30103540 2024-05-09T01:00:36.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.039+0000 7fca34e5c700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fca30113220 con 0x7fca30108770 2024-05-09T01:00:36.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.039+0000 7fca34e5c700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fca30113520 con 0x7fca30107e20 2024-05-09T01:00:36.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.039+0000 7fca2dd9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca30107e20 0x7fca30100b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:36.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.039+0000 7fca2e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca30103540 0x7fca301005c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:36.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.039+0000 7fca2dd9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca30107e20 0x7fca30100b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:41512/0 (socket says 172.21.15.5:41512) 2024-05-09T01:00:36.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.039+0000 7fca2dd9b700 1 -- 172.21.15.5:0/2349024096 learned_addr learned my addr 172.21.15.5:0/2349024096 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:00:36.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.039+0000 7fca2ed9d700 1 --2- 172.21.15.5:0/2349024096 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca30108770 0x7fca30101040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:36.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.039+0000 7fca2dd9b700 1 -- 172.21.15.5:0/2349024096 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca30108770 msgr2=0x7fca30101040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:36.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.039+0000 7fca2dd9b700 1 --2- 172.21.15.5:0/2349024096 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca30108770 0x7fca30101040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:36.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.039+0000 7fca2dd9b700 1 -- 172.21.15.5:0/2349024096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca30103540 msgr2=0x7fca301005c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:36.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.039+0000 7fca2dd9b700 1 --2- 172.21.15.5:0/2349024096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca30103540 0x7fca301005c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:36.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.039+0000 7fca2dd9b700 1 -- 172.21.15.5:0/2349024096 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca301a9270 con 0x7fca30107e20 2024-05-09T01:00:36.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.039+0000 7fca2dd9b700 1 --2- 172.21.15.5:0/2349024096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca30107e20 0x7fca30100b00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fca2400e970 tx=0x7fca2400ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:36.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.040+0000 7fca1f7fe700 1 -- 172.21.15.5:0/2349024096 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca24004a30 con 0x7fca30107e20 2024-05-09T01:00:36.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.040+0000 7fca1f7fe700 1 -- 172.21.15.5:0/2349024096 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca2400cc90 con 0x7fca30107e20 2024-05-09T01:00:36.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.040+0000 7fca34e5c700 1 -- 172.21.15.5:0/2349024096 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca301a9560 con 0x7fca30107e20 2024-05-09T01:00:36.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.040+0000 7fca1f7fe700 1 -- 172.21.15.5:0/2349024096 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca24005610 con 0x7fca30107e20 2024-05-09T01:00:36.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.040+0000 7fca34e5c700 1 -- 172.21.15.5:0/2349024096 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca301a9aa0 con 0x7fca30107e20 2024-05-09T01:00:36.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.043+0000 7fca1f7fe700 1 -- 172.21.15.5:0/2349024096 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fca2400c7e0 con 0x7fca30107e20 2024-05-09T01:00:36.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.043+0000 7fca1f7fe700 1 --2- 172.21.15.5:0/2349024096 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fca180b2880 0x7fca180b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:36.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.043+0000 7fca1f7fe700 1 -- 172.21.15.5:0/2349024096 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fca240e2330 con 0x7fca30107e20 2024-05-09T01:00:36.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.044+0000 7fca2e59c700 1 --2- 172.21.15.5:0/2349024096 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fca180b2880 0x7fca180b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:36.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.044+0000 7fca2e59c700 1 --2- 172.21.15.5:0/2349024096 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fca180b2880 0x7fca180b4d90 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fca20005950 tx=0x7fca2000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:36.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.044+0000 7fca43872b80 1 Processor -- start 2024-05-09T01:00:36.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.044+0000 7fca43872b80 1 -- start start 2024-05-09T01:00:36.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.045+0000 7fca43872b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105a387160 0x56105a4204f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:36.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.045+0000 7fca43872b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56105a29d4a0 0x56105a420a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:36.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.045+0000 7fca43872b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105a41a6a0 0x56105a41aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:36.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.045+0000 7fca43872b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56105a33cff0 con 0x56105a387160 2024-05-09T01:00:36.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.045+0000 7fca43872b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x56105a2db7f0 con 0x56105a29d4a0 2024-05-09T01:00:36.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.045+0000 7fca43872b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56105a338fb0 con 0x56105a41a6a0 2024-05-09T01:00:36.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.045+0000 7fca2e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105a387160 0x56105a4204f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:36.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.045+0000 7fca2e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105a387160 0x56105a4204f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50178/0 (socket says 172.21.15.5:50178) 2024-05-09T01:00:36.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.045+0000 7fca2e59c700 1 -- 172.21.15.5:0/1730074364 learned_addr learned my addr 172.21.15.5:0/1730074364 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:00:36.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.045+0000 7fca2ed9d700 1 --2- 172.21.15.5:0/1730074364 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105a41a6a0 0x56105a41aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:36.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.045+0000 7fca2dd9b700 1 --2- 172.21.15.5:0/1730074364 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56105a29d4a0 0x56105a420a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:36.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.045+0000 7fca2e59c700 1 -- 172.21.15.5:0/1730074364 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105a41a6a0 msgr2=0x56105a41aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:36.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.045+0000 7fca2e59c700 1 --2- 172.21.15.5:0/1730074364 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105a41a6a0 0x56105a41aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:36.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.045+0000 7fca2e59c700 1 -- 172.21.15.5:0/1730074364 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56105a29d4a0 msgr2=0x56105a420a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:36.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.045+0000 7fca2e59c700 1 --2- 172.21.15.5:0/1730074364 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56105a29d4a0 0x56105a420a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:36.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.045+0000 7fca2e59c700 1 -- 172.21.15.5:0/1730074364 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56105a2d3370 con 0x56105a387160 2024-05-09T01:00:36.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.045+0000 7fca2ed9d700 1 --2- 172.21.15.5:0/1730074364 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105a41a6a0 0x56105a41aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:00:36.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.045+0000 7fca2dd9b700 1 --2- 172.21.15.5:0/1730074364 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56105a29d4a0 0x56105a420a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:00:36.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.046+0000 7fca2e59c700 1 --2- 172.21.15.5:0/1730074364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105a387160 0x56105a4204f0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fca20010fa0 tx=0x7fca20013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=129996e3789c5cfe server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:36.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.046+0000 7fca1cff9700 1 -- 172.21.15.5:0/1730074364 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2000be60 con 0x56105a387160 2024-05-09T01:00:36.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.046+0000 7fca1cff9700 1 -- 172.21.15.5:0/1730074364 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca20011db0 con 0x56105a387160 2024-05-09T01:00:36.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.046+0000 7fca1cff9700 1 -- 172.21.15.5:0/1730074364 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2001c700 con 0x56105a387160 2024-05-09T01:00:36.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.047+0000 7fca43872b80 1 -- 172.21.15.5:0/1730074364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105a387160 msgr2=0x56105a4204f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:36.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.047+0000 7fca43872b80 1 --2- 172.21.15.5:0/1730074364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105a387160 0x56105a4204f0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fca20010fa0 tx=0x7fca20013000 comp rx=0 tx=0).stop 2024-05-09T01:00:36.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.047+0000 7fca43872b80 1 -- 172.21.15.5:0/1730074364 shutdown_connections 2024-05-09T01:00:36.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.047+0000 7fca43872b80 1 --2- 172.21.15.5:0/1730074364 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105a41a6a0 0x56105a41aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:36.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.047+0000 7fca43872b80 1 --2- 172.21.15.5:0/1730074364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105a387160 0x56105a4204f0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:36.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.047+0000 7fca43872b80 1 --2- 172.21.15.5:0/1730074364 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56105a29d4a0 0x56105a420a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:36.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.047+0000 7fca43872b80 1 -- 172.21.15.5:0/1730074364 >> 172.21.15.5:0/1730074364 conn(0x56105a2fd6a0 msgr2=0x56105a415400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:36.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.047+0000 7fca43872b80 1 -- 172.21.15.5:0/1730074364 shutdown_connections 2024-05-09T01:00:36.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.047+0000 7fca43872b80 1 -- 172.21.15.5:0/1730074364 wait complete. 2024-05-09T01:00:36.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.047+0000 7fca43872b80 1 -- start start 2024-05-09T01:00:36.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.048+0000 7fca43872b80 1 Processor -- start 2024-05-09T01:00:36.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.048+0000 7fca43872b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105a29d4a0 0x56105a424240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:36.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.048+0000 7fca43872b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105a387160 0x56105a424780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:36.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.048+0000 7fca43872b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56105a41a6a0 0x56105a424cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:36.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.048+0000 7fca43872b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56105a1b8ba0 con 0x56105a29d4a0 2024-05-09T01:00:36.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.048+0000 7fca43872b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x56105a2f7150 con 0x56105a41a6a0 2024-05-09T01:00:36.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.048+0000 7fca43872b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56105a4271d0 con 0x56105a387160 2024-05-09T01:00:36.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.048+0000 7fca43872b80 10 client.0 _ll_register_callbacks cb 0x56105a1ae6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:00:36.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.048+0000 7fca2dd9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105a387160 0x56105a424780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:36.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.048+0000 7fca2e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105a29d4a0 0x56105a424240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:36.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.048+0000 7fca2dd9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105a387160 0x56105a424780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:41542/0 (socket says 172.21.15.5:41542) 2024-05-09T01:00:36.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.048+0000 7fca2dd9b700 1 -- 172.21.15.5:0/3694480511 learned_addr learned my addr 172.21.15.5:0/3694480511 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:00:36.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.048+0000 7fca2ed9d700 1 --2- 172.21.15.5:0/3694480511 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56105a41a6a0 0x56105a424cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:36.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.048+0000 7fca2dd9b700 1 -- 172.21.15.5:0/3694480511 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56105a41a6a0 msgr2=0x56105a424cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:36.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.048+0000 7fca2dd9b700 1 --2- 172.21.15.5:0/3694480511 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56105a41a6a0 0x56105a424cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:36.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.049+0000 7fca2dd9b700 1 -- 172.21.15.5:0/3694480511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105a29d4a0 msgr2=0x56105a424240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:36.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.049+0000 7fca2dd9b700 1 --2- 172.21.15.5:0/3694480511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105a29d4a0 0x56105a424240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:36.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.049+0000 7fca2dd9b700 1 -- 172.21.15.5:0/3694480511 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56105a1b8490 con 0x56105a387160 2024-05-09T01:00:36.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.049+0000 7fca2dd9b700 1 --2- 172.21.15.5:0/3694480511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105a387160 0x56105a424780 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fca240dfe50 tx=0x7fca24004e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd8ca79d64716f5c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:36.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.049+0000 7fca15ffb700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-09T01:00:36.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.049+0000 7fca15ffb700 1 -- 172.21.15.5:0/3694480511 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca240c3060 con 0x56105a387160 2024-05-09T01:00:36.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.049+0000 7fca15ffb700 1 -- 172.21.15.5:0/3694480511 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fca240a9b80 con 0x56105a387160 2024-05-09T01:00:36.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.049+0000 7fca43872b80 1 -- 172.21.15.5:0/3694480511 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca20011db0 con 0x56105a387160 2024-05-09T01:00:36.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.049+0000 7fca15ffb700 1 -- 172.21.15.5:0/3694480511 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fca2400cad0 con 0x56105a387160 2024-05-09T01:00:36.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.049+0000 7fca43872b80 10 client.26437 fetch_fsmap learned FSMap version 32 2024-05-09T01:00:36.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.049+0000 7fca43872b80 1 -- 172.21.15.5:0/3694480511 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56105a431c70 con 0x56105a387160 2024-05-09T01:00:36.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.050+0000 7fca15ffb700 1 -- 172.21.15.5:0/3694480511 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca20011db0 con 0x56105a387160 2024-05-09T01:00:36.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.050+0000 7fca15ffb700 1 -- 172.21.15.5:0/3694480511 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fca2406f170 con 0x56105a387160 2024-05-09T01:00:36.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.050+0000 7fca43872b80 10 client.26437 fetch_fsmap finished waiting for FSMap version 32 2024-05-09T01:00:36.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.050+0000 7fca43872b80 10 client.26437 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T01:00:36.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.050+0000 7fca43872b80 20 client.26437 populate_metadata read hostname 'smithi005' 2024-05-09T01:00:36.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.050+0000 7fca43872b80 1 --2- 172.21.15.5:0/3694480511 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x56105a434630 0x56105a436a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:36.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.050+0000 7fca43872b80 4 client.26437 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:00:36.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.050+0000 7fca43872b80 1 -- 172.21.15.5:0/3694480511 --> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56105a34a9d0 con 0x56105a434630 2024-05-09T01:00:36.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.050+0000 7fca2e59c700 1 --2- 172.21.15.5:0/3694480511 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x56105a434630 0x56105a436a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:36.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.051+0000 7fca15ffb700 10 client.26437 ms_handle_connect on v2:172.21.15.157:6834/832898273 2024-05-09T01:00:36.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.051+0000 7fca2e59c700 1 --2- 172.21.15.5:0/3694480511 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x56105a434630 0x56105a436a80 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f94633cd7ad0f3bb server_cookie=73890f6f088601d7 in_seq=0 out_seq=0 2024-05-09T01:00:36.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.053+0000 7fca15ffb700 1 -- 172.21.15.5:0/3694480511 <== mds.3 v2:172.21.15.157:6834/832898273 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56105a34a9d0 con 0x56105a434630 2024-05-09T01:00:36.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.053+0000 7fca15ffb700 10 client.26437 handle_command_reply: tid=0 2024-05-09T01:00:36.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.053+0000 7fca43872b80 1 client.26437 shutdown 2024-05-09T01:00:36.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.053+0000 7fca43872b80 1 -- 172.21.15.5:0/3694480511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105a387160 msgr2=0x56105a424780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:36.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.053+0000 7fca43872b80 1 --2- 172.21.15.5:0/3694480511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105a387160 0x56105a424780 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fca240dfe50 tx=0x7fca24004e90 comp rx=0 tx=0).stop 2024-05-09T01:00:36.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.053+0000 7fca43872b80 1 -- 172.21.15.5:0/3694480511 shutdown_connections 2024-05-09T01:00:36.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.053+0000 7fca43872b80 1 --2- 172.21.15.5:0/3694480511 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x56105a434630 0x56105a436a80 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:36.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.053+0000 7fca43872b80 1 --2- 172.21.15.5:0/3694480511 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56105a41a6a0 0x56105a424cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:36.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.053+0000 7fca43872b80 1 --2- 172.21.15.5:0/3694480511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105a29d4a0 0x56105a424240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:36.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.053+0000 7fca43872b80 1 --2- 172.21.15.5:0/3694480511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105a387160 0x56105a424780 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:36.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.053+0000 7fca15ffb700 0 client.26437 ms_handle_reset on v2:172.21.15.157:6834/832898273 2024-05-09T01:00:36.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.053+0000 7fca43872b80 1 -- 172.21.15.5:0/3694480511 >> 172.21.15.5:0/3694480511 conn(0x56105a2fd6a0 msgr2=0x56105a4151c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:36.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.053+0000 7fca43872b80 1 -- 172.21.15.5:0/3694480511 shutdown_connections 2024-05-09T01:00:36.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.053+0000 7fca43872b80 1 -- 172.21.15.5:0/3694480511 wait complete. 2024-05-09T01:00:36.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.053+0000 7fca43872b80 20 client.26437 trim_cache size 0 max 16384 2024-05-09T01:00:36.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.069+0000 7fca43872b80 1 Processor -- start 2024-05-09T01:00:36.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.069+0000 7fca43872b80 1 -- start start 2024-05-09T01:00:36.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.069+0000 7fca43872b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56105a29d4a0 0x56105a422ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:36.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.069+0000 7fca43872b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105a387160 0x56105a42fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:36.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.069+0000 7fca43872b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105a41a6a0 0x56105a436cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:36.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.069+0000 7fca43872b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fca301b0ef0 con 0x56105a387160 2024-05-09T01:00:36.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.069+0000 7fca43872b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fca301b2a30 con 0x56105a29d4a0 2024-05-09T01:00:36.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.069+0000 7fca43872b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fca301b15c0 con 0x56105a41a6a0 2024-05-09T01:00:36.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.070+0000 7fca2ed9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105a41a6a0 0x56105a436cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:36.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.070+0000 7fca2ed9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105a41a6a0 0x56105a436cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:41558/0 (socket says 172.21.15.5:41558) 2024-05-09T01:00:36.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.070+0000 7fca2ed9d700 1 -- 172.21.15.5:0/4123743164 learned_addr learned my addr 172.21.15.5:0/4123743164 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:00:36.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.070+0000 7fca2dd9b700 1 --2- 172.21.15.5:0/4123743164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105a387160 0x56105a42fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:36.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.070+0000 7fca2e59c700 1 --2- 172.21.15.5:0/4123743164 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56105a29d4a0 0x56105a422ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:36.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.070+0000 7fca2ed9d700 1 -- 172.21.15.5:0/4123743164 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56105a29d4a0 msgr2=0x56105a422ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:36.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.070+0000 7fca2ed9d700 1 --2- 172.21.15.5:0/4123743164 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56105a29d4a0 0x56105a422ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:36.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.070+0000 7fca2ed9d700 1 -- 172.21.15.5:0/4123743164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105a387160 msgr2=0x56105a42fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:36.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.070+0000 7fca2ed9d700 1 --2- 172.21.15.5:0/4123743164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105a387160 0x56105a42fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:36.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.070+0000 7fca2ed9d700 1 -- 172.21.15.5:0/4123743164 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56105a40ef60 con 0x56105a41a6a0 2024-05-09T01:00:36.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.070+0000 7fca2ed9d700 1 --2- 172.21.15.5:0/4123743164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105a41a6a0 0x56105a436cb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fca28015000 tx=0x7fca2800b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95adb205a650ab2a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:36.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.070+0000 7fca17fff700 1 -- 172.21.15.5:0/4123743164 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2800bca0 con 0x56105a41a6a0 2024-05-09T01:00:36.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.070+0000 7fca17fff700 1 -- 172.21.15.5:0/4123743164 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca2800be40 con 0x56105a41a6a0 2024-05-09T01:00:36.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.071+0000 7fca17fff700 1 -- 172.21.15.5:0/4123743164 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca280075c0 con 0x56105a41a6a0 2024-05-09T01:00:36.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.071+0000 7fca43872b80 1 -- 172.21.15.5:0/4123743164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105a41a6a0 msgr2=0x56105a436cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:36.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.071+0000 7fca43872b80 1 --2- 172.21.15.5:0/4123743164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105a41a6a0 0x56105a436cb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fca28015000 tx=0x7fca2800b040 comp rx=0 tx=0).stop 2024-05-09T01:00:36.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.071+0000 7fca43872b80 1 -- 172.21.15.5:0/4123743164 shutdown_connections 2024-05-09T01:00:36.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.071+0000 7fca43872b80 1 --2- 172.21.15.5:0/4123743164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105a41a6a0 0x56105a436cb0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:36.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.071+0000 7fca43872b80 1 --2- 172.21.15.5:0/4123743164 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56105a29d4a0 0x56105a422ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:36.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.071+0000 7fca43872b80 1 --2- 172.21.15.5:0/4123743164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105a387160 0x56105a42fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:36.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.071+0000 7fca43872b80 1 -- 172.21.15.5:0/4123743164 >> 172.21.15.5:0/4123743164 conn(0x56105a2fd6a0 msgr2=0x56105a419690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:36.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.071+0000 7fca43872b80 1 -- 172.21.15.5:0/4123743164 shutdown_connections 2024-05-09T01:00:36.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.072+0000 7fca43872b80 1 -- 172.21.15.5:0/4123743164 wait complete. 2024-05-09T01:00:36.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.072+0000 7fca43872b80 1 -- start start 2024-05-09T01:00:36.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.072+0000 7fca43872b80 1 Processor -- start 2024-05-09T01:00:36.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.072+0000 7fca43872b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105a29d4a0 0x56105a43c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:36.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.072+0000 7fca43872b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105a387160 0x56105a42bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:36.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.072+0000 7fca43872b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56105a41a6a0 0x56105a42c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:36.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.072+0000 7fca43872b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fca301b2360 con 0x56105a29d4a0 2024-05-09T01:00:36.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.072+0000 7fca43872b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fca301b1c90 con 0x56105a41a6a0 2024-05-09T01:00:36.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.072+0000 7fca43872b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fca301aee90 con 0x56105a387160 2024-05-09T01:00:36.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.073+0000 7fca43872b80 10 client.0 _ll_register_callbacks cb 0x56105a1ae6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:00:36.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.073+0000 7fca2e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105a29d4a0 0x56105a43c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:36.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.073+0000 7fca2e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105a29d4a0 0x56105a43c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50220/0 (socket says 172.21.15.5:50220) 2024-05-09T01:00:36.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.073+0000 7fca2e59c700 1 -- 172.21.15.5:0/2231113465 learned_addr learned my addr 172.21.15.5:0/2231113465 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:00:36.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.073+0000 7fca2dd9b700 1 --2- 172.21.15.5:0/2231113465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105a387160 0x56105a42bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:36.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.073+0000 7fca2ed9d700 1 --2- 172.21.15.5:0/2231113465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56105a41a6a0 0x56105a42c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:36.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.073+0000 7fca2e59c700 1 -- 172.21.15.5:0/2231113465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105a387160 msgr2=0x56105a42bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:36.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.073+0000 7fca2e59c700 1 --2- 172.21.15.5:0/2231113465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105a387160 0x56105a42bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:36.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.073+0000 7fca2e59c700 1 -- 172.21.15.5:0/2231113465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56105a41a6a0 msgr2=0x56105a42c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:36.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.073+0000 7fca2e59c700 1 --2- 172.21.15.5:0/2231113465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56105a41a6a0 0x56105a42c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:36.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.073+0000 7fca2e59c700 1 -- 172.21.15.5:0/2231113465 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56105a42cac0 con 0x56105a29d4a0 2024-05-09T01:00:36.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.073+0000 7fca2dd9b700 1 --2- 172.21.15.5:0/2231113465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105a387160 0x56105a42bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:00:36.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.073+0000 7fca2ed9d700 1 --2- 172.21.15.5:0/2231113465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56105a41a6a0 0x56105a42c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:00:36.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.073+0000 7fca2e59c700 1 --2- 172.21.15.5:0/2231113465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105a29d4a0 0x56105a43c750 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fca20010f70 tx=0x7fca2001ee10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=272eee5ad789e126 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:36.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.073+0000 7fca15ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T01:00:36.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.073+0000 7fca15ffb700 1 -- 172.21.15.5:0/2231113465 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca20039070 con 0x56105a29d4a0 2024-05-09T01:00:36.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.073+0000 7fca15ffb700 1 -- 172.21.15.5:0/2231113465 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fca20005240 con 0x56105a29d4a0 2024-05-09T01:00:36.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.073+0000 7fca43872b80 1 -- 172.21.15.5:0/2231113465 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56105a344a70 con 0x56105a29d4a0 2024-05-09T01:00:36.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.074+0000 7fca15ffb700 1 -- 172.21.15.5:0/2231113465 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fca20045050 con 0x56105a29d4a0 2024-05-09T01:00:36.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.074+0000 7fca43872b80 10 client.17043 fetch_fsmap learned FSMap version 32 2024-05-09T01:00:36.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.074+0000 7fca43872b80 1 -- 172.21.15.5:0/2231113465 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56105a49de30 con 0x56105a29d4a0 2024-05-09T01:00:36.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.074+0000 7fca15ffb700 1 -- 172.21.15.5:0/2231113465 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56105a344a70 con 0x56105a29d4a0 2024-05-09T01:00:36.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.074+0000 7fca15ffb700 1 -- 172.21.15.5:0/2231113465 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fca2001e420 con 0x56105a29d4a0 2024-05-09T01:00:36.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.074+0000 7fca43872b80 10 client.17043 fetch_fsmap finished waiting for FSMap version 32 2024-05-09T01:00:36.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.074+0000 7fca43872b80 10 client.17043 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T01:00:36.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.074+0000 7fca43872b80 20 client.17043 populate_metadata read hostname 'smithi005' 2024-05-09T01:00:36.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.074+0000 7fca43872b80 1 --2- 172.21.15.5:0/2231113465 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x56105a49e0a0 0x56105a4a04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:36.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.074+0000 7fca43872b80 4 client.17043 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:00:36.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.074+0000 7fca43872b80 1 -- 172.21.15.5:0/2231113465 --> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56105a1b8970 con 0x56105a49e0a0 2024-05-09T01:00:36.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.074+0000 7fca2dd9b700 1 --2- 172.21.15.5:0/2231113465 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x56105a49e0a0 0x56105a4a04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:36.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.075+0000 7fca15ffb700 10 client.17043 ms_handle_connect on v2:172.21.15.157:6834/832898273 2024-05-09T01:00:36.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.075+0000 7fca2dd9b700 1 --2- 172.21.15.5:0/2231113465 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x56105a49e0a0 0x56105a4a04c0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=57a2784a78864586 server_cookie=b6fd6ebe14f005d0 in_seq=0 out_seq=0 2024-05-09T01:00:36.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.075+0000 7fca15ffb700 1 -- 172.21.15.5:0/2231113465 <== mds.3 v2:172.21.15.157:6834/832898273 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56105a1b8970 con 0x56105a49e0a0 2024-05-09T01:00:36.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.075+0000 7fca15ffb700 10 client.17043 handle_command_reply: tid=0 2024-05-09T01:00:36.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.075+0000 7fca43872b80 1 client.17043 shutdown 2024-05-09T01:00:36.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.075+0000 7fca43872b80 1 -- 172.21.15.5:0/2231113465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105a29d4a0 msgr2=0x56105a43c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:36.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.075+0000 7fca43872b80 1 --2- 172.21.15.5:0/2231113465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105a29d4a0 0x56105a43c750 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fca20010f70 tx=0x7fca2001ee10 comp rx=0 tx=0).stop 2024-05-09T01:00:36.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.075+0000 7fca43872b80 1 -- 172.21.15.5:0/2231113465 shutdown_connections 2024-05-09T01:00:36.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.075+0000 7fca43872b80 1 --2- 172.21.15.5:0/2231113465 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x56105a49e0a0 0x56105a4a04c0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:36.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.075+0000 7fca43872b80 1 --2- 172.21.15.5:0/2231113465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56105a41a6a0 0x56105a42c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:36.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.075+0000 7fca43872b80 1 --2- 172.21.15.5:0/2231113465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56105a29d4a0 0x56105a43c750 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:36.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.075+0000 7fca15ffb700 0 client.17043 ms_handle_reset on v2:172.21.15.157:6834/832898273 2024-05-09T01:00:36.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.075+0000 7fca43872b80 1 --2- 172.21.15.5:0/2231113465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56105a387160 0x56105a42bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:36.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.075+0000 7fca43872b80 1 -- 172.21.15.5:0/2231113465 >> 172.21.15.5:0/2231113465 conn(0x56105a2fd6a0 msgr2=0x56105a41fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:36.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.075+0000 7fca43872b80 1 -- 172.21.15.5:0/2231113465 shutdown_connections 2024-05-09T01:00:36.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.075+0000 7fca43872b80 1 -- 172.21.15.5:0/2231113465 wait complete. 2024-05-09T01:00:36.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.076+0000 7fca43872b80 20 client.17043 trim_cache size 0 max 16384 2024-05-09T01:00:36.083 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-09T01:00:36.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.076+0000 7fca34e5c700 1 -- 172.21.15.5:0/2349024096 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fca180b2880 msgr2=0x7fca180b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:36.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.076+0000 7fca34e5c700 1 --2- 172.21.15.5:0/2349024096 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fca180b2880 0x7fca180b4d90 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fca20005950 tx=0x7fca2000b410 comp rx=0 tx=0).stop 2024-05-09T01:00:36.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.076+0000 7fca34e5c700 1 -- 172.21.15.5:0/2349024096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca30107e20 msgr2=0x7fca30100b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:36.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.076+0000 7fca34e5c700 1 --2- 172.21.15.5:0/2349024096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca30107e20 0x7fca30100b00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fca2400e970 tx=0x7fca2400ec80 comp rx=0 tx=0).stop 2024-05-09T01:00:36.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.076+0000 7fca34e5c700 1 -- 172.21.15.5:0/2349024096 shutdown_connections 2024-05-09T01:00:36.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.076+0000 7fca34e5c700 1 --2- 172.21.15.5:0/2349024096 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fca180b2880 0x7fca180b4d90 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:36.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.076+0000 7fca34e5c700 1 --2- 172.21.15.5:0/2349024096 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca30108770 0x7fca30101040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:36.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.076+0000 7fca34e5c700 1 --2- 172.21.15.5:0/2349024096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca30103540 0x7fca301005c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:36.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.076+0000 7fca34e5c700 1 --2- 172.21.15.5:0/2349024096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca30107e20 0x7fca30100b00 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:36.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.076+0000 7fca34e5c700 1 -- 172.21.15.5:0/2349024096 >> 172.21.15.5:0/2349024096 conn(0x7fca30078e50 msgr2=0x7fca30075e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:36.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.076+0000 7fca34e5c700 1 -- 172.21.15.5:0/2349024096 shutdown_connections 2024-05-09T01:00:36.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:36.076+0000 7fca34e5c700 1 -- 172.21.15.5:0/2349024096 wait complete. 2024-05-09T01:00:36.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:00:36 smithi005 ceph-mon[30365]: pgmap v798: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 12 MiB/s wr, 138 op/s 2024-05-09T01:00:36.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:00:36 smithi131 ceph-mon[34547]: pgmap v798: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 12 MiB/s wr, 138 op/s 2024-05-09T01:00:36.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:00:36 smithi157 ceph-mon[34294]: pgmap v798: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 12 MiB/s wr, 138 op/s 2024-05-09T01:00:36.947 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T01:00:37.204 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:00:38.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:00:38 smithi131 ceph-mon[34547]: pgmap v799: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.9 MiB/s wr, 78 op/s 2024-05-09T01:00:38.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:00:38 smithi005 ceph-mon[30365]: pgmap v799: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.9 MiB/s wr, 78 op/s 2024-05-09T01:00:38.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:00:38 smithi157 ceph-mon[34294]: pgmap v799: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.9 MiB/s wr, 78 op/s 2024-05-09T01:00:39.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.126+0000 7f5aaa8d8700 1 -- 172.21.15.5:0/2721369900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5aa4107df0 msgr2=0x7f5aa4108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:39.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.126+0000 7f5aaa8d8700 1 --2- 172.21.15.5:0/2721369900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5aa4107df0 0x7f5aa4108200 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c0099c0 tx=0x7f5a8c009cd0 comp rx=0 tx=0).stop 2024-05-09T01:00:39.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.126+0000 7f5aaa8d8700 1 -- 172.21.15.5:0/2721369900 shutdown_connections 2024-05-09T01:00:39.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.126+0000 7f5aaa8d8700 1 --2- 172.21.15.5:0/2721369900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5aa4108740 0x7f5aa41100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:39.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.126+0000 7f5aaa8d8700 1 --2- 172.21.15.5:0/2721369900 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5aa41034c0 0x7f5aa41038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:39.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.126+0000 7f5aaa8d8700 1 --2- 172.21.15.5:0/2721369900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5aa4107df0 0x7f5aa4108200 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:39.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.126+0000 7f5aaa8d8700 1 -- 172.21.15.5:0/2721369900 >> 172.21.15.5:0/2721369900 conn(0x7f5aa40fedc0 msgr2=0x7f5aa41011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:39.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.126+0000 7f5aaa8d8700 1 -- 172.21.15.5:0/2721369900 shutdown_connections 2024-05-09T01:00:39.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.126+0000 7f5aaa8d8700 1 -- 172.21.15.5:0/2721369900 wait complete. 2024-05-09T01:00:39.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.127+0000 7f5aaa8d8700 1 Processor -- start 2024-05-09T01:00:39.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.127+0000 7f5aaa8d8700 1 -- start start 2024-05-09T01:00:39.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.127+0000 7f5aaa8d8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5aa41034c0 0x7f5aa419bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:39.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.127+0000 7f5aaa8d8700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5aa4107df0 0x7f5aa419c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:39.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.127+0000 7f5aaa8d8700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5aa4108740 0x7f5aa419c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:39.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.127+0000 7f5aaa8d8700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa4113350 con 0x7f5aa41034c0 2024-05-09T01:00:39.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.127+0000 7f5aaa8d8700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa41131d0 con 0x7f5aa4107df0 2024-05-09T01:00:39.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.127+0000 7f5aaa8d8700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa41134d0 con 0x7f5aa4108740 2024-05-09T01:00:39.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.127+0000 7f5aa3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5aa41034c0 0x7f5aa419bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:39.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.127+0000 7f5aa3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5aa41034c0 0x7f5aa419bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50244/0 (socket says 172.21.15.5:50244) 2024-05-09T01:00:39.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.127+0000 7f5aa3fff700 1 -- 172.21.15.5:0/3294447660 learned_addr learned my addr 172.21.15.5:0/3294447660 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:00:39.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.127+0000 7f5aa8e75700 1 --2- 172.21.15.5:0/3294447660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5aa4108740 0x7f5aa419c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:39.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.127+0000 7f5a9bfff700 1 --2- 172.21.15.5:0/3294447660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5aa4107df0 0x7f5aa419c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:39.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.128+0000 7f5aa8e75700 1 -- 172.21.15.5:0/3294447660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5aa4107df0 msgr2=0x7f5aa419c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:39.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.128+0000 7f5aa8e75700 1 --2- 172.21.15.5:0/3294447660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5aa4107df0 0x7f5aa419c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:39.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.128+0000 7f5aa8e75700 1 -- 172.21.15.5:0/3294447660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5aa41034c0 msgr2=0x7f5aa419bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:39.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.128+0000 7f5aa8e75700 1 --2- 172.21.15.5:0/3294447660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5aa41034c0 0x7f5aa419bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:39.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.128+0000 7f5aa8e75700 1 -- 172.21.15.5:0/3294447660 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5aa41a6fe0 con 0x7f5aa4108740 2024-05-09T01:00:39.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.128+0000 7f5aa8e75700 1 --2- 172.21.15.5:0/3294447660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5aa4108740 0x7f5aa419c730 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c00efc0 tx=0x7f5a9c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:39.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.128+0000 7f5aa1ffb700 1 -- 172.21.15.5:0/3294447660 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c0092e0 con 0x7f5aa4108740 2024-05-09T01:00:39.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.128+0000 7f5aa1ffb700 1 -- 172.21.15.5:0/3294447660 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a9c004bb0 con 0x7f5aa4108740 2024-05-09T01:00:39.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.128+0000 7f5aaa8d8700 1 -- 172.21.15.5:0/3294447660 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5aa41a7270 con 0x7f5aa4108740 2024-05-09T01:00:39.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.128+0000 7f5aa1ffb700 1 -- 172.21.15.5:0/3294447660 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c004020 con 0x7f5aa4108740 2024-05-09T01:00:39.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.128+0000 7f5aaa8d8700 1 -- 172.21.15.5:0/3294447660 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5aa41a7780 con 0x7f5aa4108740 2024-05-09T01:00:39.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.131+0000 7f5aa1ffb700 1 -- 172.21.15.5:0/3294447660 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f5a9c01b030 con 0x7f5aa4108740 2024-05-09T01:00:39.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.132+0000 7f5ab92eeb80 1 Processor -- start 2024-05-09T01:00:39.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.132+0000 7f5ab92eeb80 1 -- start start 2024-05-09T01:00:39.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.132+0000 7f5aa1ffb700 1 --2- 172.21.15.5:0/3294447660 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f5a900b2900 0x7f5a900b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:39.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.132+0000 7f5ab92eeb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c3d5b4d00 0x555c3d6bbad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:39.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.132+0000 7f5ab92eeb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c3d53a4a0 0x555c3d6bc030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:39.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.132+0000 7f5ab92eeb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555c3d6c01e0 0x555c3d6c05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:39.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.132+0000 7f5ab92eeb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555c3d5da0a0 con 0x555c3d53a4a0 2024-05-09T01:00:39.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.132+0000 7f5ab92eeb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x555c3d57f9a0 con 0x555c3d6c01e0 2024-05-09T01:00:39.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.132+0000 7f5ab92eeb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x555c3d5d6060 con 0x555c3d5b4d00 2024-05-09T01:00:39.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.132+0000 7f5a9bfff700 1 --2- 172.21.15.5:0/3294447660 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f5a900b2900 0x7f5a900b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:39.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.133+0000 7f5aa3fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c3d5b4d00 0x555c3d6bbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:39.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.133+0000 7f5aa3fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c3d5b4d00 0x555c3d6bbad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:41592/0 (socket says 172.21.15.5:41592) 2024-05-09T01:00:39.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.133+0000 7f5aa3fff700 1 -- 172.21.15.5:0/2066798673 learned_addr learned my addr 172.21.15.5:0/2066798673 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:00:39.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.133+0000 7f5a9bfff700 1 --2- 172.21.15.5:0/2066798673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c3d53a4a0 0x555c3d6bc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:39.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.133+0000 7f5aa1ffb700 1 -- 172.21.15.5:0/3294447660 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f5a9c0e1c70 con 0x7f5aa4108740 2024-05-09T01:00:39.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.133+0000 7f5aa8e75700 1 --2- 172.21.15.5:0/2066798673 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555c3d6c01e0 0x555c3d6c05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:39.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.133+0000 7f5aa3fff700 1 -- 172.21.15.5:0/2066798673 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555c3d6c01e0 msgr2=0x555c3d6c05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:39.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.133+0000 7f5aa3fff700 1 --2- 172.21.15.5:0/2066798673 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555c3d6c01e0 0x555c3d6c05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:39.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.133+0000 7f5aa3fff700 1 -- 172.21.15.5:0/2066798673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c3d53a4a0 msgr2=0x555c3d6bc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:39.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.133+0000 7f5aa3fff700 1 --2- 172.21.15.5:0/2066798673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c3d53a4a0 0x555c3d6bc030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:39.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.133+0000 7f5aa3fff700 1 -- 172.21.15.5:0/2066798673 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c3d570500 con 0x555c3d5b4d00 2024-05-09T01:00:39.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.133+0000 7f5a9bfff700 1 --2- 172.21.15.5:0/2066798673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c3d53a4a0 0x555c3d6bc030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:00:39.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.133+0000 7f5a9bfff700 1 --2- 172.21.15.5:0/3294447660 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f5a900b2900 0x7f5a900b4e10 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c006010 tx=0x7f5a8c009230 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:39.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.133+0000 7f5aa3fff700 1 --2- 172.21.15.5:0/2066798673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c3d5b4d00 0x555c3d6bbad0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5a9400bea0 tx=0x7f5a940106b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a96e771c80e47d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:39.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.133+0000 7f5a9a7fc700 1 -- 172.21.15.5:0/2066798673 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a94011440 con 0x555c3d5b4d00 2024-05-09T01:00:39.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.134+0000 7f5a9a7fc700 1 -- 172.21.15.5:0/2066798673 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a94011a90 con 0x555c3d5b4d00 2024-05-09T01:00:39.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.134+0000 7f5a9a7fc700 1 -- 172.21.15.5:0/2066798673 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a940199f0 con 0x555c3d5b4d00 2024-05-09T01:00:39.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.134+0000 7f5ab92eeb80 1 -- 172.21.15.5:0/2066798673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c3d5b4d00 msgr2=0x555c3d6bbad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:39.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.134+0000 7f5ab92eeb80 1 --2- 172.21.15.5:0/2066798673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c3d5b4d00 0x555c3d6bbad0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5a9400bea0 tx=0x7f5a940106b0 comp rx=0 tx=0).stop 2024-05-09T01:00:39.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.134+0000 7f5ab92eeb80 1 -- 172.21.15.5:0/2066798673 shutdown_connections 2024-05-09T01:00:39.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.134+0000 7f5ab92eeb80 1 --2- 172.21.15.5:0/2066798673 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555c3d6c01e0 0x555c3d6c05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:39.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.134+0000 7f5ab92eeb80 1 --2- 172.21.15.5:0/2066798673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c3d5b4d00 0x555c3d6bbad0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:39.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.134+0000 7f5ab92eeb80 1 --2- 172.21.15.5:0/2066798673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c3d53a4a0 0x555c3d6bc030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:39.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.134+0000 7f5ab92eeb80 1 -- 172.21.15.5:0/2066798673 >> 172.21.15.5:0/2066798673 conn(0x555c3d53df70 msgr2=0x555c3d6b2220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:39.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.134+0000 7f5ab92eeb80 1 -- 172.21.15.5:0/2066798673 shutdown_connections 2024-05-09T01:00:39.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.135+0000 7f5ab92eeb80 1 -- 172.21.15.5:0/2066798673 wait complete. 2024-05-09T01:00:39.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.135+0000 7f5ab92eeb80 1 -- start start 2024-05-09T01:00:39.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.135+0000 7f5ab92eeb80 1 Processor -- start 2024-05-09T01:00:39.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.136+0000 7f5ab92eeb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c3d53a4a0 0x555c3d6c53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:39.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.136+0000 7f5ab92eeb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c3d5b4d00 0x555c3d6c5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:39.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.136+0000 7f5ab92eeb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555c3d6c01e0 0x555c3d6c5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:39.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.136+0000 7f5ab92eeb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555c3d6c0f00 con 0x555c3d53a4a0 2024-05-09T01:00:39.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.136+0000 7f5ab92eeb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x555c3d574b20 con 0x555c3d6c01e0 2024-05-09T01:00:39.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.136+0000 7f5ab92eeb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x555c3d6c8470 con 0x555c3d5b4d00 2024-05-09T01:00:39.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.136+0000 7f5ab92eeb80 10 client.0 _ll_register_callbacks cb 0x555c3d44b6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:00:39.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.136+0000 7f5aa3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c3d53a4a0 0x555c3d6c53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:39.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.136+0000 7f5aa3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c3d53a4a0 0x555c3d6c53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50274/0 (socket says 172.21.15.5:50274) 2024-05-09T01:00:39.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.136+0000 7f5a9bfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c3d5b4d00 0x555c3d6c5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:39.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.136+0000 7f5aa3fff700 1 -- 172.21.15.5:0/1912016981 learned_addr learned my addr 172.21.15.5:0/1912016981 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:00:39.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.136+0000 7f5aa8e75700 1 --2- 172.21.15.5:0/1912016981 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555c3d6c01e0 0x555c3d6c5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:39.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.136+0000 7f5aa3fff700 1 -- 172.21.15.5:0/1912016981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c3d5b4d00 msgr2=0x555c3d6c5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:39.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.136+0000 7f5aa3fff700 1 --2- 172.21.15.5:0/1912016981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c3d5b4d00 0x555c3d6c5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:39.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.136+0000 7f5aa3fff700 1 -- 172.21.15.5:0/1912016981 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555c3d6c01e0 msgr2=0x555c3d6c5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:39.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.136+0000 7f5aa3fff700 1 --2- 172.21.15.5:0/1912016981 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555c3d6c01e0 0x555c3d6c5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:39.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.136+0000 7f5aa3fff700 1 -- 172.21.15.5:0/1912016981 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c3d455490 con 0x555c3d53a4a0 2024-05-09T01:00:39.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.136+0000 7f5aa3fff700 1 --2- 172.21.15.5:0/1912016981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c3d53a4a0 0x555c3d6c53e0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f5a94014000 tx=0x7f5a94019480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adf978d7f2f63615 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:39.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.136+0000 7f5a837fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T01:00:39.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.137+0000 7f5a837fe700 1 -- 172.21.15.5:0/1912016981 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a94026030 con 0x555c3d53a4a0 2024-05-09T01:00:39.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.137+0000 7f5a837fe700 1 -- 172.21.15.5:0/1912016981 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f5a9401f040 con 0x555c3d53a4a0 2024-05-09T01:00:39.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.137+0000 7f5ab92eeb80 1 -- 172.21.15.5:0/1912016981 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a94011a90 con 0x555c3d53a4a0 2024-05-09T01:00:39.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.137+0000 7f5a837fe700 1 -- 172.21.15.5:0/1912016981 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5a94011dc0 con 0x555c3d53a4a0 2024-05-09T01:00:39.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.137+0000 7f5ab92eeb80 10 client.17055 fetch_fsmap learned FSMap version 32 2024-05-09T01:00:39.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.137+0000 7f5ab92eeb80 1 -- 172.21.15.5:0/1912016981 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555c3d6d2d70 con 0x555c3d53a4a0 2024-05-09T01:00:39.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.137+0000 7f5a837fe700 1 -- 172.21.15.5:0/1912016981 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a94011a90 con 0x555c3d53a4a0 2024-05-09T01:00:39.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.137+0000 7f5a837fe700 1 -- 172.21.15.5:0/1912016981 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f5a94005040 con 0x555c3d53a4a0 2024-05-09T01:00:39.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.137+0000 7f5ab92eeb80 10 client.17055 fetch_fsmap finished waiting for FSMap version 32 2024-05-09T01:00:39.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.137+0000 7f5ab92eeb80 10 client.17055 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-05-09T01:00:39.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.137+0000 7f5ab92eeb80 20 client.17055 populate_metadata read hostname 'smithi005' 2024-05-09T01:00:39.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.137+0000 7f5ab92eeb80 1 --2- 172.21.15.5:0/1912016981 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x555c3d6d5240 0x555c3d6d7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:39.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.137+0000 7f5ab92eeb80 4 client.17055 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:00:39.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.137+0000 7f5ab92eeb80 1 -- 172.21.15.5:0/1912016981 --> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c3d5e7a80 con 0x555c3d6d5240 2024-05-09T01:00:39.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.138+0000 7f5a9bfff700 1 --2- 172.21.15.5:0/1912016981 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x555c3d6d5240 0x555c3d6d7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:39.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.138+0000 7f5a9bfff700 1 --2- 172.21.15.5:0/1912016981 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x555c3d6d5240 0x555c3d6d7660 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=80cbf65d9a095f2f server_cookie=f18aba914f962c75 in_seq=0 out_seq=0 2024-05-09T01:00:39.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.138+0000 7f5a837fe700 10 client.17055 ms_handle_connect on v2:172.21.15.131:6838/2748669631 2024-05-09T01:00:39.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.139+0000 7f5a837fe700 1 -- 172.21.15.5:0/1912016981 <== mds.2 v2:172.21.15.131:6838/2748669631 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555c3d5e7a80 con 0x555c3d6d5240 2024-05-09T01:00:39.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.139+0000 7f5a837fe700 10 client.17055 handle_command_reply: tid=0 2024-05-09T01:00:39.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.139+0000 7f5ab92eeb80 1 client.17055 shutdown 2024-05-09T01:00:39.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.139+0000 7f5ab92eeb80 1 -- 172.21.15.5:0/1912016981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c3d53a4a0 msgr2=0x555c3d6c53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:39.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.139+0000 7f5ab92eeb80 1 --2- 172.21.15.5:0/1912016981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c3d53a4a0 0x555c3d6c53e0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f5a94014000 tx=0x7f5a94019480 comp rx=0 tx=0).stop 2024-05-09T01:00:39.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.139+0000 7f5ab92eeb80 1 -- 172.21.15.5:0/1912016981 shutdown_connections 2024-05-09T01:00:39.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.139+0000 7f5ab92eeb80 1 --2- 172.21.15.5:0/1912016981 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x555c3d6d5240 0x555c3d6d7660 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:39.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.139+0000 7f5a837fe700 0 client.17055 ms_handle_reset on v2:172.21.15.131:6838/2748669631 2024-05-09T01:00:39.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.139+0000 7f5ab92eeb80 1 --2- 172.21.15.5:0/1912016981 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555c3d6c01e0 0x555c3d6c5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:39.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.139+0000 7f5ab92eeb80 1 --2- 172.21.15.5:0/1912016981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c3d53a4a0 0x555c3d6c53e0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:39.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.139+0000 7f5ab92eeb80 1 --2- 172.21.15.5:0/1912016981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c3d5b4d00 0x555c3d6c5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:39.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.139+0000 7f5ab92eeb80 1 -- 172.21.15.5:0/1912016981 >> 172.21.15.5:0/1912016981 conn(0x555c3d53df70 msgr2=0x555c3d6b63f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:39.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.139+0000 7f5ab92eeb80 1 -- 172.21.15.5:0/1912016981 shutdown_connections 2024-05-09T01:00:39.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.139+0000 7f5ab92eeb80 1 -- 172.21.15.5:0/1912016981 wait complete. 2024-05-09T01:00:39.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.140+0000 7f5ab92eeb80 20 client.17055 trim_cache size 0 max 16384 2024-05-09T01:00:39.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.155+0000 7f5ab92eeb80 1 Processor -- start 2024-05-09T01:00:39.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.155+0000 7f5ab92eeb80 1 -- start start 2024-05-09T01:00:39.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.155+0000 7f5ab92eeb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555c3d53a4a0 0x555c3d6d15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:39.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.155+0000 7f5ab92eeb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c3d5b4d00 0x555c3d6c5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:39.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.155+0000 7f5ab92eeb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c3d6c01e0 0x555c3d6bc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:39.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.155+0000 7f5ab92eeb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa41acaf0 con 0x555c3d6c01e0 2024-05-09T01:00:39.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.155+0000 7f5ab92eeb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa41ae630 con 0x555c3d53a4a0 2024-05-09T01:00:39.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.155+0000 7f5ab92eeb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa41ad1c0 con 0x555c3d5b4d00 2024-05-09T01:00:39.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.155+0000 7f5aa8e75700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c3d6c01e0 0x555c3d6bc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:39.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.155+0000 7f5aa8e75700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c3d6c01e0 0x555c3d6bc1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50284/0 (socket says 172.21.15.5:50284) 2024-05-09T01:00:39.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.155+0000 7f5aa8e75700 1 -- 172.21.15.5:0/987131839 learned_addr learned my addr 172.21.15.5:0/987131839 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:00:39.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.155+0000 7f5a9bfff700 1 --2- 172.21.15.5:0/987131839 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c3d5b4d00 0x555c3d6c5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:39.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.155+0000 7f5aa3fff700 1 --2- 172.21.15.5:0/987131839 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555c3d53a4a0 0x555c3d6d15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:39.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.155+0000 7f5aa8e75700 1 -- 172.21.15.5:0/987131839 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c3d5b4d00 msgr2=0x555c3d6c5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:39.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.155+0000 7f5aa8e75700 1 --2- 172.21.15.5:0/987131839 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c3d5b4d00 0x555c3d6c5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:39.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.155+0000 7f5aa8e75700 1 -- 172.21.15.5:0/987131839 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555c3d53a4a0 msgr2=0x555c3d6d15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:39.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.155+0000 7f5aa8e75700 1 --2- 172.21.15.5:0/987131839 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555c3d53a4a0 0x555c3d6d15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:39.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.155+0000 7f5aa8e75700 1 -- 172.21.15.5:0/987131839 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c3d6ae9d0 con 0x555c3d6c01e0 2024-05-09T01:00:39.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.156+0000 7f5aa8e75700 1 --2- 172.21.15.5:0/987131839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c3d6c01e0 0x555c3d6bc1c0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c004b80 tx=0x7f5a9c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f13fa2204c5f487e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:39.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.156+0000 7f5a99ffb700 1 -- 172.21.15.5:0/987131839 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c0ad640 con 0x555c3d6c01e0 2024-05-09T01:00:39.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.156+0000 7f5a99ffb700 1 -- 172.21.15.5:0/987131839 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a9c0b2060 con 0x555c3d6c01e0 2024-05-09T01:00:39.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.156+0000 7f5a99ffb700 1 -- 172.21.15.5:0/987131839 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c0b1bd0 con 0x555c3d6c01e0 2024-05-09T01:00:39.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.157+0000 7f5ab92eeb80 1 -- 172.21.15.5:0/987131839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c3d6c01e0 msgr2=0x555c3d6bc1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:39.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.157+0000 7f5ab92eeb80 1 --2- 172.21.15.5:0/987131839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c3d6c01e0 0x555c3d6bc1c0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c004b80 tx=0x7f5a9c00c9c0 comp rx=0 tx=0).stop 2024-05-09T01:00:39.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.157+0000 7f5ab92eeb80 1 -- 172.21.15.5:0/987131839 shutdown_connections 2024-05-09T01:00:39.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.157+0000 7f5ab92eeb80 1 --2- 172.21.15.5:0/987131839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c3d6c01e0 0x555c3d6bc1c0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:39.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.157+0000 7f5ab92eeb80 1 --2- 172.21.15.5:0/987131839 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555c3d53a4a0 0x555c3d6d15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:39.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.157+0000 7f5ab92eeb80 1 --2- 172.21.15.5:0/987131839 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c3d5b4d00 0x555c3d6c5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:39.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.157+0000 7f5ab92eeb80 1 -- 172.21.15.5:0/987131839 >> 172.21.15.5:0/987131839 conn(0x555c3d53df70 msgr2=0x555c3d6bbab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:39.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.157+0000 7f5ab92eeb80 1 -- 172.21.15.5:0/987131839 shutdown_connections 2024-05-09T01:00:39.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.157+0000 7f5ab92eeb80 1 -- 172.21.15.5:0/987131839 wait complete. 2024-05-09T01:00:39.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.157+0000 7f5ab92eeb80 1 -- start start 2024-05-09T01:00:39.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.157+0000 7f5ab92eeb80 1 Processor -- start 2024-05-09T01:00:39.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.158+0000 7f5ab92eeb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555c3d53a4a0 0x555c3d6d9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:39.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.158+0000 7f5ab92eeb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c3d5b4d00 0x555c3d6da370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:39.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.158+0000 7f5ab92eeb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c3d6c01e0 0x555c3d736830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:39.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.158+0000 7f5ab92eeb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa41adf60 con 0x555c3d6c01e0 2024-05-09T01:00:39.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.158+0000 7f5ab92eeb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa41ad890 con 0x555c3d53a4a0 2024-05-09T01:00:39.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.158+0000 7f5ab92eeb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa41aaa90 con 0x555c3d5b4d00 2024-05-09T01:00:39.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.158+0000 7f5ab92eeb80 10 client.0 _ll_register_callbacks cb 0x555c3d44b6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:00:39.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.158+0000 7f5aa8e75700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c3d6c01e0 0x555c3d736830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:39.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.158+0000 7f5a9bfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c3d5b4d00 0x555c3d6da370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:39.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.158+0000 7f5aa8e75700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c3d6c01e0 0x555c3d736830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50286/0 (socket says 172.21.15.5:50286) 2024-05-09T01:00:39.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.158+0000 7f5aa8e75700 1 -- 172.21.15.5:0/206702908 learned_addr learned my addr 172.21.15.5:0/206702908 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:00:39.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.158+0000 7f5aa3fff700 1 --2- 172.21.15.5:0/206702908 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555c3d53a4a0 0x555c3d6d9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:39.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.158+0000 7f5aa3fff700 1 -- 172.21.15.5:0/206702908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c3d5b4d00 msgr2=0x555c3d6da370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:39.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.158+0000 7f5aa3fff700 1 --2- 172.21.15.5:0/206702908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c3d5b4d00 0x555c3d6da370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:39.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.158+0000 7f5aa3fff700 1 -- 172.21.15.5:0/206702908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c3d6c01e0 msgr2=0x555c3d736830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:39.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.158+0000 7f5aa3fff700 1 --2- 172.21.15.5:0/206702908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c3d6c01e0 0x555c3d736830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:39.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.158+0000 7f5aa3fff700 1 -- 172.21.15.5:0/206702908 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c3d6b3d30 con 0x555c3d53a4a0 2024-05-09T01:00:39.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.158+0000 7f5aa8e75700 1 --2- 172.21.15.5:0/206702908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c3d6c01e0 0x555c3d736830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:00:39.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.158+0000 7f5a9bfff700 1 --2- 172.21.15.5:0/206702908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c3d5b4d00 0x555c3d6da370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:00:39.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.159+0000 7f5aa3fff700 1 --2- 172.21.15.5:0/206702908 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555c3d53a4a0 0x555c3d6d9e10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5a9400bdc0 tx=0x7f5a940048c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac7d2a9b68fea192 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:39.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.159+0000 7f5a837fe700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-09T01:00:39.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.159+0000 7f5a837fe700 1 -- 172.21.15.5:0/206702908 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9403a060 con 0x555c3d53a4a0 2024-05-09T01:00:39.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.159+0000 7f5ab92eeb80 1 -- 172.21.15.5:0/206702908 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a9c0b2060 con 0x555c3d53a4a0 2024-05-09T01:00:39.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.159+0000 7f5a837fe700 1 -- 172.21.15.5:0/206702908 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f5a94005680 con 0x555c3d53a4a0 2024-05-09T01:00:39.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.159+0000 7f5a837fe700 1 -- 172.21.15.5:0/206702908 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5a94026040 con 0x555c3d53a4a0 2024-05-09T01:00:39.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.159+0000 7f5ab92eeb80 10 client.26477 fetch_fsmap learned FSMap version 32 2024-05-09T01:00:39.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.159+0000 7f5ab92eeb80 1 -- 172.21.15.5:0/206702908 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555c3d73f110 con 0x555c3d53a4a0 2024-05-09T01:00:39.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.159+0000 7f5a837fe700 1 -- 172.21.15.5:0/206702908 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a9c0b2060 con 0x555c3d53a4a0 2024-05-09T01:00:39.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.159+0000 7f5a837fe700 1 -- 172.21.15.5:0/206702908 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f5a94004d20 con 0x555c3d53a4a0 2024-05-09T01:00:39.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.159+0000 7f5ab92eeb80 10 client.26477 fetch_fsmap finished waiting for FSMap version 32 2024-05-09T01:00:39.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.159+0000 7f5ab92eeb80 10 client.26477 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-05-09T01:00:39.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.159+0000 7f5ab92eeb80 20 client.26477 populate_metadata read hostname 'smithi005' 2024-05-09T01:00:39.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.159+0000 7f5ab92eeb80 1 --2- 172.21.15.5:0/206702908 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x555c3d6d5240 0x555c3d7417d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:39.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.159+0000 7f5ab92eeb80 4 client.26477 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:00:39.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.159+0000 7f5ab92eeb80 1 -- 172.21.15.5:0/206702908 --> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555c3d5e1b20 con 0x555c3d6d5240 2024-05-09T01:00:39.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.160+0000 7f5a9bfff700 1 --2- 172.21.15.5:0/206702908 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x555c3d6d5240 0x555c3d7417d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:39.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.160+0000 7f5a9bfff700 1 --2- 172.21.15.5:0/206702908 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x555c3d6d5240 0x555c3d7417d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=332e06af58ba7507 server_cookie=ac69446f34755320 in_seq=0 out_seq=0 2024-05-09T01:00:39.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.160+0000 7f5a837fe700 10 client.26477 ms_handle_connect on v2:172.21.15.131:6838/2748669631 2024-05-09T01:00:39.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.160+0000 7f5a837fe700 1 -- 172.21.15.5:0/206702908 <== mds.2 v2:172.21.15.131:6838/2748669631 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555c3d5e1b20 con 0x555c3d6d5240 2024-05-09T01:00:39.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.160+0000 7f5a837fe700 10 client.26477 handle_command_reply: tid=0 2024-05-09T01:00:39.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.160+0000 7f5ab92eeb80 1 client.26477 shutdown 2024-05-09T01:00:39.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.161+0000 7f5ab92eeb80 1 -- 172.21.15.5:0/206702908 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555c3d53a4a0 msgr2=0x555c3d6d9e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:39.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.161+0000 7f5ab92eeb80 1 --2- 172.21.15.5:0/206702908 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555c3d53a4a0 0x555c3d6d9e10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5a9400bdc0 tx=0x7f5a940048c0 comp rx=0 tx=0).stop 2024-05-09T01:00:39.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.161+0000 7f5ab92eeb80 1 -- 172.21.15.5:0/206702908 shutdown_connections 2024-05-09T01:00:39.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.161+0000 7f5ab92eeb80 1 --2- 172.21.15.5:0/206702908 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x555c3d6d5240 0x555c3d7417d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:39.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.161+0000 7f5ab92eeb80 1 --2- 172.21.15.5:0/206702908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c3d6c01e0 0x555c3d736830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:39.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.161+0000 7f5ab92eeb80 1 --2- 172.21.15.5:0/206702908 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555c3d53a4a0 0x555c3d6d9e10 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:39.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.161+0000 7f5a837fe700 0 client.26477 ms_handle_reset on v2:172.21.15.131:6838/2748669631 2024-05-09T01:00:39.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.161+0000 7f5ab92eeb80 1 --2- 172.21.15.5:0/206702908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c3d5b4d00 0x555c3d6da370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:39.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.161+0000 7f5ab92eeb80 1 -- 172.21.15.5:0/206702908 >> 172.21.15.5:0/206702908 conn(0x555c3d53df70 msgr2=0x555c3d6c3ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:39.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.161+0000 7f5ab92eeb80 1 -- 172.21.15.5:0/206702908 shutdown_connections 2024-05-09T01:00:39.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.161+0000 7f5ab92eeb80 1 -- 172.21.15.5:0/206702908 wait complete. 2024-05-09T01:00:39.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.161+0000 7f5ab92eeb80 20 client.26477 trim_cache size 0 max 16384 2024-05-09T01:00:39.168 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-09T01:00:39.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.162+0000 7f5aaa8d8700 1 -- 172.21.15.5:0/3294447660 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f5a900b2900 msgr2=0x7f5a900b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:39.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.162+0000 7f5aaa8d8700 1 --2- 172.21.15.5:0/3294447660 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f5a900b2900 0x7f5a900b4e10 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c006010 tx=0x7f5a8c009230 comp rx=0 tx=0).stop 2024-05-09T01:00:39.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.162+0000 7f5aaa8d8700 1 -- 172.21.15.5:0/3294447660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5aa4108740 msgr2=0x7f5aa419c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:39.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.162+0000 7f5aaa8d8700 1 --2- 172.21.15.5:0/3294447660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5aa4108740 0x7f5aa419c730 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c00efc0 tx=0x7f5a9c00c3a0 comp rx=0 tx=0).stop 2024-05-09T01:00:39.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.162+0000 7f5aaa8d8700 1 -- 172.21.15.5:0/3294447660 shutdown_connections 2024-05-09T01:00:39.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.162+0000 7f5aaa8d8700 1 --2- 172.21.15.5:0/3294447660 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f5a900b2900 0x7f5a900b4e10 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:39.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.162+0000 7f5aaa8d8700 1 --2- 172.21.15.5:0/3294447660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5aa4108740 0x7f5aa419c730 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:39.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.162+0000 7f5aaa8d8700 1 --2- 172.21.15.5:0/3294447660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5aa41034c0 0x7f5aa419bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:39.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.162+0000 7f5aaa8d8700 1 --2- 172.21.15.5:0/3294447660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5aa4107df0 0x7f5aa419c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:39.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.162+0000 7f5aaa8d8700 1 -- 172.21.15.5:0/3294447660 >> 172.21.15.5:0/3294447660 conn(0x7f5aa40fedc0 msgr2=0x7f5aa41096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:39.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.162+0000 7f5aaa8d8700 1 -- 172.21.15.5:0/3294447660 shutdown_connections 2024-05-09T01:00:39.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:39.162+0000 7f5aaa8d8700 1 -- 172.21.15.5:0/3294447660 wait complete. 2024-05-09T01:00:39.875 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T01:00:40.146 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:00:40.270 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:00:40 smithi005 ceph-mon[30365]: pgmap v800: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 7.4 MiB/s wr, 57 op/s 2024-05-09T01:00:40.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:00:40 smithi131 ceph-mon[34547]: pgmap v800: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 7.4 MiB/s wr, 57 op/s 2024-05-09T01:00:40.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:00:40 smithi157 ceph-mon[34294]: pgmap v800: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 7.4 MiB/s wr, 57 op/s 2024-05-09T01:00:41.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.819+0000 7fd5e70ca700 1 -- 172.21.15.5:0/952565065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5e0107df0 msgr2=0x7fd5e0108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:41.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.819+0000 7fd5e70ca700 1 --2- 172.21.15.5:0/952565065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5e0107df0 0x7fd5e0108200 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc0099c0 tx=0x7fd5cc009cd0 comp rx=0 tx=0).stop 2024-05-09T01:00:41.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.819+0000 7fd5e70ca700 1 -- 172.21.15.5:0/952565065 shutdown_connections 2024-05-09T01:00:41.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.819+0000 7fd5e70ca700 1 --2- 172.21.15.5:0/952565065 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5e0108740 0x7fd5e01100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:41.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.819+0000 7fd5e70ca700 1 --2- 172.21.15.5:0/952565065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5e01034c0 0x7fd5e01038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:41.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.819+0000 7fd5e70ca700 1 --2- 172.21.15.5:0/952565065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5e0107df0 0x7fd5e0108200 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:41.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.819+0000 7fd5e70ca700 1 -- 172.21.15.5:0/952565065 >> 172.21.15.5:0/952565065 conn(0x7fd5e00fedc0 msgr2=0x7fd5e01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:41.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.819+0000 7fd5e70ca700 1 -- 172.21.15.5:0/952565065 shutdown_connections 2024-05-09T01:00:41.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.819+0000 7fd5e70ca700 1 -- 172.21.15.5:0/952565065 wait complete. 2024-05-09T01:00:41.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.820+0000 7fd5e70ca700 1 Processor -- start 2024-05-09T01:00:41.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.820+0000 7fd5e70ca700 1 -- start start 2024-05-09T01:00:41.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.820+0000 7fd5e70ca700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5e01034c0 0x7fd5e019adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:41.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.820+0000 7fd5e70ca700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5e0107df0 0x7fd5e019b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:41.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.820+0000 7fd5e70ca700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5e0108740 0x7fd5e019f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:41.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.820+0000 7fd5e70ca700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e0113350 con 0x7fd5e0107df0 2024-05-09T01:00:41.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.820+0000 7fd5e70ca700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e01131d0 con 0x7fd5e0108740 2024-05-09T01:00:41.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.820+0000 7fd5e70ca700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e01134d0 con 0x7fd5e01034c0 2024-05-09T01:00:41.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.821+0000 7fd5dffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5e0107df0 0x7fd5e019b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:41.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.821+0000 7fd5e4e66700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5e01034c0 0x7fd5e019adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:41.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.821+0000 7fd5dffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5e0107df0 0x7fd5e019b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50314/0 (socket says 172.21.15.5:50314) 2024-05-09T01:00:41.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.821+0000 7fd5dffff700 1 -- 172.21.15.5:0/3305834669 learned_addr learned my addr 172.21.15.5:0/3305834669 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:00:41.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.821+0000 7fd5e4e66700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5e01034c0 0x7fd5e019adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:41632/0 (socket says 172.21.15.5:41632) 2024-05-09T01:00:41.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.821+0000 7fd5e5667700 1 --2- 172.21.15.5:0/3305834669 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5e0108740 0x7fd5e019f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:41.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.821+0000 7fd5dffff700 1 -- 172.21.15.5:0/3305834669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5e01034c0 msgr2=0x7fd5e019adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:41.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.821+0000 7fd5dffff700 1 --2- 172.21.15.5:0/3305834669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5e01034c0 0x7fd5e019adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:41.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.821+0000 7fd5dffff700 1 -- 172.21.15.5:0/3305834669 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5e0108740 msgr2=0x7fd5e019f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:41.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.821+0000 7fd5dffff700 1 --2- 172.21.15.5:0/3305834669 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5e0108740 0x7fd5e019f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:41.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.821+0000 7fd5dffff700 1 -- 172.21.15.5:0/3305834669 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5e019fc10 con 0x7fd5e0107df0 2024-05-09T01:00:41.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.821+0000 7fd5e4e66700 1 --2- 172.21.15.5:0/3305834669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5e01034c0 0x7fd5e019adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:00:41.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.821+0000 7fd5dffff700 1 --2- 172.21.15.5:0/3305834669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5e0107df0 0x7fd5e019b390 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc006010 tx=0x7fd5cc0055f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:41.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.821+0000 7fd5ddffb700 1 -- 172.21.15.5:0/3305834669 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5cc026070 con 0x7fd5e0107df0 2024-05-09T01:00:41.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.821+0000 7fd5ddffb700 1 -- 172.21.15.5:0/3305834669 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5cc00f460 con 0x7fd5e0107df0 2024-05-09T01:00:41.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.821+0000 7fd5e70ca700 1 -- 172.21.15.5:0/3305834669 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5e019fea0 con 0x7fd5e0107df0 2024-05-09T01:00:41.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.821+0000 7fd5ddffb700 1 -- 172.21.15.5:0/3305834669 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5cc009e60 con 0x7fd5e0107df0 2024-05-09T01:00:41.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.822+0000 7fd5e70ca700 1 -- 172.21.15.5:0/3305834669 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5e01a0380 con 0x7fd5e0107df0 2024-05-09T01:00:41.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.824+0000 7fd5ddffb700 1 -- 172.21.15.5:0/3305834669 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fd5cc029070 con 0x7fd5e0107df0 2024-05-09T01:00:41.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.825+0000 7fd5f5ae0b80 1 Processor -- start 2024-05-09T01:00:41.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.825+0000 7fd5f5ae0b80 1 -- start start 2024-05-09T01:00:41.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.826+0000 7fd5f5ae0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562df0cf5160 0x562df0d8cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:41.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.826+0000 7fd5ddffb700 1 --2- 172.21.15.5:0/3305834669 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fd5d00b2880 0x7fd5d00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:41.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.826+0000 7fd5f5ae0b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562df0c0b4a0 0x562df0d8d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:41.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.826+0000 7fd5f5ae0b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562df0d91400 0x562df0d91810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:41.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.826+0000 7fd5f5ae0b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562df0caaff0 con 0x562df0d91400 2024-05-09T01:00:41.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.826+0000 7fd5f5ae0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x562df0c497f0 con 0x562df0cf5160 2024-05-09T01:00:41.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.826+0000 7fd5f5ae0b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x562df0ca6fb0 con 0x562df0c0b4a0 2024-05-09T01:00:41.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.826+0000 7fd5e5667700 1 --2- 172.21.15.5:0/3305834669 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fd5d00b2880 0x7fd5d00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:41.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.826+0000 7fd5e5667700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562df0d91400 0x562df0d91810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:41.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.826+0000 7fd5ddffb700 1 -- 172.21.15.5:0/3305834669 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fd5cc0e2bd0 con 0x7fd5e0107df0 2024-05-09T01:00:41.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.826+0000 7fd5e4e66700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562df0cf5160 0x562df0d8cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:41.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.826+0000 7fd5e5667700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562df0d91400 0x562df0d91810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50326/0 (socket says 172.21.15.5:50326) 2024-05-09T01:00:41.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.826+0000 7fd5e5667700 1 -- 172.21.15.5:0/888146685 learned_addr learned my addr 172.21.15.5:0/888146685 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:00:41.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.826+0000 7fd5dffff700 1 --2- 172.21.15.5:0/888146685 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562df0c0b4a0 0x562df0d8d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:41.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.826+0000 7fd5e5667700 1 -- 172.21.15.5:0/888146685 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562df0c0b4a0 msgr2=0x562df0d8d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:41.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.826+0000 7fd5e5667700 1 --2- 172.21.15.5:0/888146685 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562df0c0b4a0 0x562df0d8d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:41.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.826+0000 7fd5e5667700 1 -- 172.21.15.5:0/888146685 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562df0cf5160 msgr2=0x562df0d8cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:41.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.826+0000 7fd5e5667700 1 --2- 172.21.15.5:0/888146685 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562df0cf5160 0x562df0d8cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:41.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.826+0000 7fd5e5667700 1 -- 172.21.15.5:0/888146685 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562df0c41370 con 0x562df0d91400 2024-05-09T01:00:41.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.827+0000 7fd5e5667700 1 --2- 172.21.15.5:0/3305834669 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fd5d00b2880 0x7fd5d00b4d90 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd5d8007fd0 tx=0x7fd5d8013380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:41.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.827+0000 7fd5e5667700 1 --2- 172.21.15.5:0/888146685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562df0d91400 0x562df0d91810 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fd5d8013780 tx=0x7fd5d801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=845560a2712369c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:41.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.827+0000 7fd5caffd700 1 -- 172.21.15.5:0/888146685 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d8017e90 con 0x562df0d91400 2024-05-09T01:00:41.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.827+0000 7fd5caffd700 1 -- 172.21.15.5:0/888146685 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5d801d070 con 0x562df0d91400 2024-05-09T01:00:41.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.827+0000 7fd5caffd700 1 -- 172.21.15.5:0/888146685 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d80199a0 con 0x562df0d91400 2024-05-09T01:00:41.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.828+0000 7fd5f5ae0b80 1 -- 172.21.15.5:0/888146685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562df0d91400 msgr2=0x562df0d91810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:41.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.828+0000 7fd5f5ae0b80 1 --2- 172.21.15.5:0/888146685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562df0d91400 0x562df0d91810 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fd5d8013780 tx=0x7fd5d801a000 comp rx=0 tx=0).stop 2024-05-09T01:00:41.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.828+0000 7fd5f5ae0b80 1 -- 172.21.15.5:0/888146685 shutdown_connections 2024-05-09T01:00:41.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.828+0000 7fd5f5ae0b80 1 --2- 172.21.15.5:0/888146685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562df0d91400 0x562df0d91810 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:41.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.828+0000 7fd5f5ae0b80 1 --2- 172.21.15.5:0/888146685 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562df0cf5160 0x562df0d8cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:41.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.828+0000 7fd5f5ae0b80 1 --2- 172.21.15.5:0/888146685 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562df0c0b4a0 0x562df0d8d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:41.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.828+0000 7fd5f5ae0b80 1 -- 172.21.15.5:0/888146685 >> 172.21.15.5:0/888146685 conn(0x562df0c6b6a0 msgr2=0x562df0d83400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:41.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.828+0000 7fd5f5ae0b80 1 -- 172.21.15.5:0/888146685 shutdown_connections 2024-05-09T01:00:41.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.828+0000 7fd5f5ae0b80 1 -- 172.21.15.5:0/888146685 wait complete. 2024-05-09T01:00:41.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.828+0000 7fd5f5ae0b80 1 -- start start 2024-05-09T01:00:41.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.829+0000 7fd5f5ae0b80 1 Processor -- start 2024-05-09T01:00:41.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.829+0000 7fd5f5ae0b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562df0c0b4a0 0x562df0d9c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:41.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.829+0000 7fd5f5ae0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562df0cf5160 0x562df0d9c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:41.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.829+0000 7fd5f5ae0b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562df0d91400 0x562df0d96660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:41.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.829+0000 7fd5f5ae0b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562df0b26ba0 con 0x562df0d91400 2024-05-09T01:00:41.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.829+0000 7fd5f5ae0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x562df0c65150 con 0x562df0cf5160 2024-05-09T01:00:41.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.829+0000 7fd5f5ae0b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x562df0d920f0 con 0x562df0c0b4a0 2024-05-09T01:00:41.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.829+0000 7fd5e5667700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562df0d91400 0x562df0d96660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:41.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.829+0000 7fd5f5ae0b80 10 client.0 _ll_register_callbacks cb 0x562df0b1c6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:00:41.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.829+0000 7fd5e5667700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562df0d91400 0x562df0d96660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50336/0 (socket says 172.21.15.5:50336) 2024-05-09T01:00:41.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.829+0000 7fd5e5667700 1 -- 172.21.15.5:0/3311943239 learned_addr learned my addr 172.21.15.5:0/3311943239 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:00:41.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.829+0000 7fd5e4e66700 1 --2- 172.21.15.5:0/3311943239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562df0c0b4a0 0x562df0d9c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:41.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.829+0000 7fd5dffff700 1 --2- 172.21.15.5:0/3311943239 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562df0cf5160 0x562df0d9c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:41.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.830+0000 7fd5e5667700 1 -- 172.21.15.5:0/3311943239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562df0c0b4a0 msgr2=0x562df0d9c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:41.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.830+0000 7fd5e5667700 1 --2- 172.21.15.5:0/3311943239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562df0c0b4a0 0x562df0d9c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:41.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.830+0000 7fd5e5667700 1 -- 172.21.15.5:0/3311943239 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562df0cf5160 msgr2=0x562df0d9c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:41.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.830+0000 7fd5e5667700 1 --2- 172.21.15.5:0/3311943239 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562df0cf5160 0x562df0d9c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:41.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.830+0000 7fd5e5667700 1 -- 172.21.15.5:0/3311943239 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562df0b26490 con 0x562df0d91400 2024-05-09T01:00:41.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.830+0000 7fd5dffff700 1 --2- 172.21.15.5:0/3311943239 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562df0cf5160 0x562df0d9c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:00:41.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.830+0000 7fd5e4e66700 1 --2- 172.21.15.5:0/3311943239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562df0c0b4a0 0x562df0d9c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:00:41.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.830+0000 7fd5e5667700 1 --2- 172.21.15.5:0/3311943239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562df0d91400 0x562df0d96660 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fd5d8017aa0 tx=0x7fd5d80199e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fdea10022c03b2d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:41.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.830+0000 7fd5bffff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T01:00:41.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.830+0000 7fd5bffff700 1 -- 172.21.15.5:0/3311943239 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d8030020 con 0x562df0d91400 2024-05-09T01:00:41.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.830+0000 7fd5f5ae0b80 1 -- 172.21.15.5:0/3311943239 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5d801d070 con 0x562df0d91400 2024-05-09T01:00:41.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.830+0000 7fd5bffff700 1 -- 172.21.15.5:0/3311943239 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fd5d8010040 con 0x562df0d91400 2024-05-09T01:00:41.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.830+0000 7fd5bffff700 1 -- 172.21.15.5:0/3311943239 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd5d8007dd0 con 0x562df0d91400 2024-05-09T01:00:41.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.830+0000 7fd5f5ae0b80 10 client.17073 fetch_fsmap learned FSMap version 32 2024-05-09T01:00:41.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.830+0000 7fd5f5ae0b80 1 -- 172.21.15.5:0/3311943239 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562df0da3e80 con 0x562df0d91400 2024-05-09T01:00:41.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.831+0000 7fd5bffff700 1 -- 172.21.15.5:0/3311943239 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5d801d070 con 0x562df0d91400 2024-05-09T01:00:41.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.831+0000 7fd5bffff700 1 -- 172.21.15.5:0/3311943239 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fd5d8016040 con 0x562df0d91400 2024-05-09T01:00:41.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.831+0000 7fd5f5ae0b80 10 client.17073 fetch_fsmap finished waiting for FSMap version 32 2024-05-09T01:00:41.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.831+0000 7fd5f5ae0b80 10 client.17073 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-05-09T01:00:41.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.831+0000 7fd5f5ae0b80 20 client.17073 populate_metadata read hostname 'smithi005' 2024-05-09T01:00:41.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.831+0000 7fd5f5ae0b80 1 --2- 172.21.15.5:0/3311943239 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x562df0da6350 0x562df0da8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:41.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.831+0000 7fd5f5ae0b80 4 client.17073 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:00:41.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.831+0000 7fd5f5ae0b80 1 -- 172.21.15.5:0/3311943239 --> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562df0cb89d0 con 0x562df0da6350 2024-05-09T01:00:41.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.831+0000 7fd5e4e66700 1 --2- 172.21.15.5:0/3311943239 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x562df0da6350 0x562df0da8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:41.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.832+0000 7fd5bffff700 10 client.17073 ms_handle_connect on v2:172.21.15.157:6836/744863009 2024-05-09T01:00:41.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.832+0000 7fd5e4e66700 1 --2- 172.21.15.5:0/3311943239 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x562df0da6350 0x562df0da8770 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=116b630b7cb9d758 server_cookie=d156ca8fecc1a4d8 in_seq=0 out_seq=0 2024-05-09T01:00:41.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.832+0000 7fd5bffff700 1 -- 172.21.15.5:0/3311943239 <== mds.1 v2:172.21.15.157:6836/744863009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562df0cb89d0 con 0x562df0da6350 2024-05-09T01:00:41.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.832+0000 7fd5bffff700 10 client.17073 handle_command_reply: tid=0 2024-05-09T01:00:41.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.832+0000 7fd5f5ae0b80 1 client.17073 shutdown 2024-05-09T01:00:41.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.833+0000 7fd5f5ae0b80 1 -- 172.21.15.5:0/3311943239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562df0d91400 msgr2=0x562df0d96660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:41.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.833+0000 7fd5f5ae0b80 1 --2- 172.21.15.5:0/3311943239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562df0d91400 0x562df0d96660 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fd5d8017aa0 tx=0x7fd5d80199e0 comp rx=0 tx=0).stop 2024-05-09T01:00:41.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.833+0000 7fd5f5ae0b80 1 -- 172.21.15.5:0/3311943239 shutdown_connections 2024-05-09T01:00:41.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.833+0000 7fd5f5ae0b80 1 --2- 172.21.15.5:0/3311943239 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x562df0da6350 0x562df0da8770 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:41.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.833+0000 7fd5f5ae0b80 1 --2- 172.21.15.5:0/3311943239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562df0d91400 0x562df0d96660 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:41.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.833+0000 7fd5f5ae0b80 1 --2- 172.21.15.5:0/3311943239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562df0c0b4a0 0x562df0d9c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:41.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.833+0000 7fd5f5ae0b80 1 --2- 172.21.15.5:0/3311943239 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562df0cf5160 0x562df0d9c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:41.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.833+0000 7fd5f5ae0b80 1 -- 172.21.15.5:0/3311943239 >> 172.21.15.5:0/3311943239 conn(0x562df0c6b6a0 msgr2=0x562df0d831c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:41.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.833+0000 7fd5bffff700 0 client.17073 ms_handle_reset on v2:172.21.15.157:6836/744863009 2024-05-09T01:00:41.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.833+0000 7fd5f5ae0b80 1 -- 172.21.15.5:0/3311943239 shutdown_connections 2024-05-09T01:00:41.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.833+0000 7fd5f5ae0b80 1 -- 172.21.15.5:0/3311943239 wait complete. 2024-05-09T01:00:41.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.833+0000 7fd5f5ae0b80 20 client.17073 trim_cache size 0 max 16384 2024-05-09T01:00:41.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.848+0000 7fd5f5ae0b80 1 Processor -- start 2024-05-09T01:00:41.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.848+0000 7fd5f5ae0b80 1 -- start start 2024-05-09T01:00:41.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.848+0000 7fd5f5ae0b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562df0c0b4a0 0x562df0d909c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:41.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.848+0000 7fd5f5ae0b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562df0cf5160 0x562df0d87da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:41.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.848+0000 7fd5f5ae0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562df0d91400 0x562df0d8f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:41.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.848+0000 7fd5f5ae0b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e01ac950 con 0x562df0cf5160 2024-05-09T01:00:41.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.849+0000 7fd5f5ae0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e01ae490 con 0x562df0d91400 2024-05-09T01:00:41.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.849+0000 7fd5f5ae0b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e01ad020 con 0x562df0c0b4a0 2024-05-09T01:00:41.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.849+0000 7fd5dffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562df0cf5160 0x562df0d87da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:41.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.849+0000 7fd5e4e66700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562df0c0b4a0 0x562df0d909c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:41.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.849+0000 7fd5dffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562df0cf5160 0x562df0d87da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50344/0 (socket says 172.21.15.5:50344) 2024-05-09T01:00:41.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.849+0000 7fd5dffff700 1 -- 172.21.15.5:0/373309215 learned_addr learned my addr 172.21.15.5:0/373309215 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:00:41.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.849+0000 7fd5e5667700 1 --2- 172.21.15.5:0/373309215 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562df0d91400 0x562df0d8f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:41.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.849+0000 7fd5e4e66700 1 -- 172.21.15.5:0/373309215 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562df0d91400 msgr2=0x562df0d8f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:41.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.849+0000 7fd5e4e66700 1 --2- 172.21.15.5:0/373309215 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562df0d91400 0x562df0d8f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:41.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.849+0000 7fd5e4e66700 1 -- 172.21.15.5:0/373309215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562df0cf5160 msgr2=0x562df0d87da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:41.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.849+0000 7fd5e4e66700 1 --2- 172.21.15.5:0/373309215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562df0cf5160 0x562df0d87da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:41.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.849+0000 7fd5e4e66700 1 -- 172.21.15.5:0/373309215 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562df0d7cf60 con 0x562df0c0b4a0 2024-05-09T01:00:41.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.849+0000 7fd5dffff700 1 --2- 172.21.15.5:0/373309215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562df0cf5160 0x562df0d87da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:00:41.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.849+0000 7fd5e4e66700 1 --2- 172.21.15.5:0/373309215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562df0c0b4a0 0x562df0d909c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x562df0b26c40 tx=0x7fd5d4009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=345605d825dbf16b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:41.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.849+0000 7fd5ca7fc700 1 -- 172.21.15.5:0/373309215 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d4004a30 con 0x562df0c0b4a0 2024-05-09T01:00:41.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.850+0000 7fd5ca7fc700 1 -- 172.21.15.5:0/373309215 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5d4024070 con 0x562df0c0b4a0 2024-05-09T01:00:41.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.850+0000 7fd5ca7fc700 1 -- 172.21.15.5:0/373309215 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d40038e0 con 0x562df0c0b4a0 2024-05-09T01:00:41.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.851+0000 7fd5f5ae0b80 1 -- 172.21.15.5:0/373309215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562df0c0b4a0 msgr2=0x562df0d909c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:41.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.851+0000 7fd5f5ae0b80 1 --2- 172.21.15.5:0/373309215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562df0c0b4a0 0x562df0d909c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x562df0b26c40 tx=0x7fd5d4009260 comp rx=0 tx=0).stop 2024-05-09T01:00:41.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.851+0000 7fd5f5ae0b80 1 -- 172.21.15.5:0/373309215 shutdown_connections 2024-05-09T01:00:41.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.851+0000 7fd5f5ae0b80 1 --2- 172.21.15.5:0/373309215 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562df0d91400 0x562df0d8f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:41.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.851+0000 7fd5f5ae0b80 1 --2- 172.21.15.5:0/373309215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562df0c0b4a0 0x562df0d909c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:41.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.851+0000 7fd5f5ae0b80 1 --2- 172.21.15.5:0/373309215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562df0cf5160 0x562df0d87da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:41.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.851+0000 7fd5f5ae0b80 1 -- 172.21.15.5:0/373309215 >> 172.21.15.5:0/373309215 conn(0x562df0c6b6a0 msgr2=0x562df0d87690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:41.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.851+0000 7fd5f5ae0b80 1 -- 172.21.15.5:0/373309215 shutdown_connections 2024-05-09T01:00:41.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.851+0000 7fd5f5ae0b80 1 -- 172.21.15.5:0/373309215 wait complete. 2024-05-09T01:00:41.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.851+0000 7fd5f5ae0b80 1 -- start start 2024-05-09T01:00:41.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.852+0000 7fd5f5ae0b80 1 Processor -- start 2024-05-09T01:00:41.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.852+0000 7fd5f5ae0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562df0c0b4a0 0x562df0da1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:41.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.852+0000 7fd5f5ae0b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562df0cf5160 0x562df0da2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:41.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.852+0000 7fd5f5ae0b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562df0d91400 0x562df0e089c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:41.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.852+0000 7fd5f5ae0b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e01addc0 con 0x562df0cf5160 2024-05-09T01:00:41.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.852+0000 7fd5f5ae0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e01ad6f0 con 0x562df0c0b4a0 2024-05-09T01:00:41.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.852+0000 7fd5f5ae0b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e01aa8f0 con 0x562df0d91400 2024-05-09T01:00:41.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.852+0000 7fd5f5ae0b80 10 client.0 _ll_register_callbacks cb 0x562df0b1c6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:00:41.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.852+0000 7fd5e5667700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562df0d91400 0x562df0e089c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:41.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.852+0000 7fd5dffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562df0cf5160 0x562df0da2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:41.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.852+0000 7fd5e5667700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562df0d91400 0x562df0e089c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:41672/0 (socket says 172.21.15.5:41672) 2024-05-09T01:00:41.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.852+0000 7fd5e5667700 1 -- 172.21.15.5:0/3247466303 learned_addr learned my addr 172.21.15.5:0/3247466303 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:00:41.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.852+0000 7fd5e4e66700 1 --2- 172.21.15.5:0/3247466303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562df0c0b4a0 0x562df0da1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:41.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.853+0000 7fd5dffff700 1 -- 172.21.15.5:0/3247466303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562df0d91400 msgr2=0x562df0e089c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:41.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.853+0000 7fd5dffff700 1 --2- 172.21.15.5:0/3247466303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562df0d91400 0x562df0e089c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:41.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.853+0000 7fd5dffff700 1 -- 172.21.15.5:0/3247466303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562df0c0b4a0 msgr2=0x562df0da1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:41.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.853+0000 7fd5dffff700 1 --2- 172.21.15.5:0/3247466303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562df0c0b4a0 0x562df0da1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:41.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.853+0000 7fd5dffff700 1 -- 172.21.15.5:0/3247466303 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562df0e08f30 con 0x562df0cf5160 2024-05-09T01:00:41.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.853+0000 7fd5dffff700 1 --2- 172.21.15.5:0/3247466303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562df0cf5160 0x562df0da2010 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc009d70 tx=0x7fd5cc0c4d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b50531947be2194c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:41.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.853+0000 7fd5bffff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T01:00:41.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.853+0000 7fd5bffff700 1 -- 172.21.15.5:0/3247466303 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5cc0aa8f0 con 0x562df0cf5160 2024-05-09T01:00:41.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.853+0000 7fd5bffff700 1 -- 172.21.15.5:0/3247466303 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fd5cc0aa450 con 0x562df0cf5160 2024-05-09T01:00:41.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.853+0000 7fd5f5ae0b80 1 -- 172.21.15.5:0/3247466303 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5d4024070 con 0x562df0cf5160 2024-05-09T01:00:41.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.853+0000 7fd5bffff700 1 -- 172.21.15.5:0/3247466303 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd5cc0aa1f0 con 0x562df0cf5160 2024-05-09T01:00:41.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.853+0000 7fd5f5ae0b80 10 client.17079 fetch_fsmap learned FSMap version 32 2024-05-09T01:00:41.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.853+0000 7fd5f5ae0b80 1 -- 172.21.15.5:0/3247466303 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562df0e10270 con 0x562df0cf5160 2024-05-09T01:00:41.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.854+0000 7fd5bffff700 1 -- 172.21.15.5:0/3247466303 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5d4024070 con 0x562df0cf5160 2024-05-09T01:00:41.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.854+0000 7fd5bffff700 1 -- 172.21.15.5:0/3247466303 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fd5cc0ae170 con 0x562df0cf5160 2024-05-09T01:00:41.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.854+0000 7fd5f5ae0b80 10 client.17079 fetch_fsmap finished waiting for FSMap version 32 2024-05-09T01:00:41.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.854+0000 7fd5f5ae0b80 10 client.17079 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-05-09T01:00:41.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.854+0000 7fd5f5ae0b80 20 client.17079 populate_metadata read hostname 'smithi005' 2024-05-09T01:00:41.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.854+0000 7fd5f5ae0b80 1 --2- 172.21.15.5:0/3247466303 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x562df0e10950 0x562df0e12d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:41.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.854+0000 7fd5f5ae0b80 4 client.17079 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:00:41.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.854+0000 7fd5f5ae0b80 1 -- 172.21.15.5:0/3247466303 --> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562df0cb2a70 con 0x562df0e10950 2024-05-09T01:00:41.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.854+0000 7fd5e4e66700 1 --2- 172.21.15.5:0/3247466303 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x562df0e10950 0x562df0e12d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:41.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.855+0000 7fd5bffff700 10 client.17079 ms_handle_connect on v2:172.21.15.157:6836/744863009 2024-05-09T01:00:41.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.855+0000 7fd5e4e66700 1 --2- 172.21.15.5:0/3247466303 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x562df0e10950 0x562df0e12d70 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7832e034c771b1dc server_cookie=115e510248b805b1 in_seq=0 out_seq=0 2024-05-09T01:00:41.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.855+0000 7fd5bffff700 1 -- 172.21.15.5:0/3247466303 <== mds.1 v2:172.21.15.157:6836/744863009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562df0cb2a70 con 0x562df0e10950 2024-05-09T01:00:41.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.855+0000 7fd5bffff700 10 client.17079 handle_command_reply: tid=0 2024-05-09T01:00:41.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.855+0000 7fd5f5ae0b80 1 client.17079 shutdown 2024-05-09T01:00:41.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.856+0000 7fd5f5ae0b80 1 -- 172.21.15.5:0/3247466303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562df0cf5160 msgr2=0x562df0da2010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:41.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.856+0000 7fd5f5ae0b80 1 --2- 172.21.15.5:0/3247466303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562df0cf5160 0x562df0da2010 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc009d70 tx=0x7fd5cc0c4d20 comp rx=0 tx=0).stop 2024-05-09T01:00:41.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.856+0000 7fd5f5ae0b80 1 -- 172.21.15.5:0/3247466303 shutdown_connections 2024-05-09T01:00:41.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.856+0000 7fd5f5ae0b80 1 --2- 172.21.15.5:0/3247466303 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x562df0e10950 0x562df0e12d70 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:41.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.856+0000 7fd5f5ae0b80 1 --2- 172.21.15.5:0/3247466303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562df0d91400 0x562df0e089c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:41.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.856+0000 7fd5f5ae0b80 1 --2- 172.21.15.5:0/3247466303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562df0c0b4a0 0x562df0da1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:41.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.856+0000 7fd5bffff700 0 client.17079 ms_handle_reset on v2:172.21.15.157:6836/744863009 2024-05-09T01:00:41.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.856+0000 7fd5f5ae0b80 1 --2- 172.21.15.5:0/3247466303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562df0cf5160 0x562df0da2010 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:41.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.856+0000 7fd5f5ae0b80 1 -- 172.21.15.5:0/3247466303 >> 172.21.15.5:0/3247466303 conn(0x562df0c6b6a0 msgr2=0x562df0d83310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:41.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.856+0000 7fd5f5ae0b80 1 -- 172.21.15.5:0/3247466303 shutdown_connections 2024-05-09T01:00:41.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.856+0000 7fd5f5ae0b80 1 -- 172.21.15.5:0/3247466303 wait complete. 2024-05-09T01:00:41.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.856+0000 7fd5f5ae0b80 20 client.17079 trim_cache size 0 max 16384 2024-05-09T01:00:41.864 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-09T01:00:41.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.857+0000 7fd5e70ca700 1 -- 172.21.15.5:0/3305834669 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fd5d00b2880 msgr2=0x7fd5d00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:41.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.857+0000 7fd5e70ca700 1 --2- 172.21.15.5:0/3305834669 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fd5d00b2880 0x7fd5d00b4d90 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd5d8007fd0 tx=0x7fd5d8013380 comp rx=0 tx=0).stop 2024-05-09T01:00:41.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.857+0000 7fd5e70ca700 1 -- 172.21.15.5:0/3305834669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5e0107df0 msgr2=0x7fd5e019b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:41.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.857+0000 7fd5e70ca700 1 --2- 172.21.15.5:0/3305834669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5e0107df0 0x7fd5e019b390 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc006010 tx=0x7fd5cc0055f0 comp rx=0 tx=0).stop 2024-05-09T01:00:41.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.857+0000 7fd5e70ca700 1 -- 172.21.15.5:0/3305834669 shutdown_connections 2024-05-09T01:00:41.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.857+0000 7fd5e70ca700 1 --2- 172.21.15.5:0/3305834669 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fd5d00b2880 0x7fd5d00b4d90 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:41.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.857+0000 7fd5e70ca700 1 --2- 172.21.15.5:0/3305834669 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5e0108740 0x7fd5e019f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:41.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.857+0000 7fd5e70ca700 1 --2- 172.21.15.5:0/3305834669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5e01034c0 0x7fd5e019adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:41.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.857+0000 7fd5e70ca700 1 --2- 172.21.15.5:0/3305834669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5e0107df0 0x7fd5e019b390 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:41.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.857+0000 7fd5e70ca700 1 -- 172.21.15.5:0/3305834669 >> 172.21.15.5:0/3305834669 conn(0x7fd5e00fedc0 msgr2=0x7fd5e006b910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:41.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.857+0000 7fd5e70ca700 1 -- 172.21.15.5:0/3305834669 shutdown_connections 2024-05-09T01:00:41.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:41.857+0000 7fd5e70ca700 1 -- 172.21.15.5:0/3305834669 wait complete. 2024-05-09T01:00:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.905+0000 7fcf2ca0c640 1 -- 172.21.15.5:0/3555593603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf281098e0 msgr2=0x7fcf2810bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.905+0000 7fcf2ca0c640 1 --2- 172.21.15.5:0/3555593603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf281098e0 0x7fcf2810bdf0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fcf20009ca0 tx=0x7fcf2002a8c0 comp rx=0 tx=0).stop 2024-05-09T01:00:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.906+0000 7fcf2ca0c640 1 -- 172.21.15.5:0/3555593603 shutdown_connections 2024-05-09T01:00:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.906+0000 7fcf2ca0c640 1 --2- 172.21.15.5:0/3555593603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf281098e0 0x7fcf2810bdf0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.906+0000 7fcf2ca0c640 1 --2- 172.21.15.5:0/3555593603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf28106db0 0x7fcf28109250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.906+0000 7fcf2ca0c640 1 --2- 172.21.15.5:0/3555593603 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf281043c0 0x7fcf281067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.906+0000 7fcf2ca0c640 1 -- 172.21.15.5:0/3555593603 >> 172.21.15.5:0/3555593603 conn(0x7fcf280fdd40 msgr2=0x7fcf281001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.906+0000 7fcf2ca0c640 1 -- 172.21.15.5:0/3555593603 shutdown_connections 2024-05-09T01:00:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.906+0000 7fcf2ca0c640 1 -- 172.21.15.5:0/3555593603 wait complete. 2024-05-09T01:00:41.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.906+0000 7fcf2ca0c640 1 Processor -- start 2024-05-09T01:00:41.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.907+0000 7fcf2ca0c640 1 -- start start 2024-05-09T01:00:41.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.907+0000 7fcf2ca0c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf281043c0 0x7fcf2819a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:41.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.907+0000 7fcf2ca0c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf28106db0 0x7fcf2819a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:41.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.907+0000 7fcf2ca0c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf281098e0 0x7fcf281a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.907+0000 7fcf2ca0c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2810f770 con 0x7fcf281098e0 2024-05-09T01:00:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.907+0000 7fcf2ca0c640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2810f5f0 con 0x7fcf28106db0 2024-05-09T01:00:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.907+0000 7fcf2ca0c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2810f8f0 con 0x7fcf281043c0 2024-05-09T01:00:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.907+0000 7fcf2659c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf281043c0 0x7fcf2819a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.907+0000 7fcf26d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf281098e0 0x7fcf281a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.907+0000 7fcf2659c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf281043c0 0x7fcf2819a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:41686/0 (socket says 172.21.15.5:41686) 2024-05-09T01:00:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.907+0000 7fcf26d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf281098e0 0x7fcf281a17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50376/0 (socket says 172.21.15.5:50376) 2024-05-09T01:00:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.907+0000 7fcf2659c640 1 -- 172.21.15.5:0/3602952550 learned_addr learned my addr 172.21.15.5:0/3602952550 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:00:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.907+0000 7fcf25d9b640 1 --2- 172.21.15.5:0/3602952550 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf28106db0 0x7fcf2819a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.907+0000 7fcf2659c640 1 -- 172.21.15.5:0/3602952550 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf28106db0 msgr2=0x7fcf2819a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.907+0000 7fcf2659c640 1 --2- 172.21.15.5:0/3602952550 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf28106db0 0x7fcf2819a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.907+0000 7fcf2659c640 1 -- 172.21.15.5:0/3602952550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf281098e0 msgr2=0x7fcf281a17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.907+0000 7fcf2659c640 1 --2- 172.21.15.5:0/3602952550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf281098e0 0x7fcf281a17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.907+0000 7fcf2659c640 1 -- 172.21.15.5:0/3602952550 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf281a1e20 con 0x7fcf281043c0 2024-05-09T01:00:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.908+0000 7fcf2659c640 1 --2- 172.21.15.5:0/3602952550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf281043c0 0x7fcf2819a2e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fcf1800e9a0 tx=0x7fcf1800ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.908+0000 7fcf177fe640 1 -- 172.21.15.5:0/3602952550 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf1800cd80 con 0x7fcf281043c0 2024-05-09T01:00:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.908+0000 7fcf2ca0c640 1 -- 172.21.15.5:0/3602952550 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf28102210 con 0x7fcf281043c0 2024-05-09T01:00:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.908+0000 7fcf177fe640 1 -- 172.21.15.5:0/3602952550 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf180035f0 con 0x7fcf281043c0 2024-05-09T01:00:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.908+0000 7fcf177fe640 1 -- 172.21.15.5:0/3602952550 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf18010690 con 0x7fcf281043c0 2024-05-09T01:00:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.908+0000 7fcf2ca0c640 1 -- 172.21.15.5:0/3602952550 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf281027b0 con 0x7fcf281043c0 2024-05-09T01:00:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.910+0000 7fcf33998740 1 Processor -- start 2024-05-09T01:00:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.911+0000 7fcf33998740 1 -- start start 2024-05-09T01:00:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.911+0000 7fcf33998740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1a03d41b0 0x55d1a0778500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.911+0000 7fcf33998740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1a03ab2f0 0x55d1a0778a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.911+0000 7fcf33998740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1a03b34d0 0x55d1a077f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.911+0000 7fcf33998740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d1a06dc7c0 con 0x55d1a03d41b0 2024-05-09T01:00:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.911+0000 7fcf33998740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55d1a06780e0 con 0x55d1a03b34d0 2024-05-09T01:00:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.911+0000 7fcf33998740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d1a06d8780 con 0x55d1a03ab2f0 2024-05-09T01:00:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.911+0000 7fcf26d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1a03d41b0 0x55d1a0778500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.911+0000 7fcf26d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1a03d41b0 0x55d1a0778500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50384/0 (socket says 172.21.15.5:50384) 2024-05-09T01:00:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.911+0000 7fcf26d9d640 1 -- 172.21.15.5:0/2567236359 learned_addr learned my addr 172.21.15.5:0/2567236359 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:00:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.911+0000 7fcf25d9b640 1 --2- 172.21.15.5:0/2567236359 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1a03b34d0 0x55d1a077f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.911+0000 7fcf177fe640 1 -- 172.21.15.5:0/3602952550 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fcf18004020 con 0x7fcf281043c0 2024-05-09T01:00:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.911+0000 7fcf26d9d640 1 -- 172.21.15.5:0/2567236359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1a03ab2f0 msgr2=0x55d1a0778a80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T01:00:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.911+0000 7fcf26d9d640 1 --2- 172.21.15.5:0/2567236359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1a03ab2f0 0x55d1a0778a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.911+0000 7fcf26d9d640 1 -- 172.21.15.5:0/2567236359 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1a03b34d0 msgr2=0x55d1a077f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.911+0000 7fcf26d9d640 1 --2- 172.21.15.5:0/2567236359 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1a03b34d0 0x55d1a077f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.911+0000 7fcf26d9d640 1 -- 172.21.15.5:0/2567236359 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1a06a17c0 con 0x55d1a03d41b0 2024-05-09T01:00:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.912+0000 7fcf26d9d640 1 --2- 172.21.15.5:0/2567236359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1a03d41b0 0x55d1a0778500 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fcf20035000 tx=0x7fcf2002f410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9438a339d4260da server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.912+0000 7fcf14ff9640 1 -- 172.21.15.5:0/2567236359 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2004c070 con 0x55d1a03d41b0 2024-05-09T01:00:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.912+0000 7fcf14ff9640 1 -- 172.21.15.5:0/2567236359 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf20036040 con 0x55d1a03d41b0 2024-05-09T01:00:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.912+0000 7fcf14ff9640 1 -- 172.21.15.5:0/2567236359 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2003ccc0 con 0x55d1a03d41b0 2024-05-09T01:00:41.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.913+0000 7fcf177fe640 1 --2- 172.21.15.5:0/3602952550 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fcf100b27e0 0x7fcf100b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:41.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.913+0000 7fcf2659c640 1 --2- 172.21.15.5:0/3602952550 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fcf100b27e0 0x7fcf100b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:41.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.913+0000 7fcf33998740 1 -- 172.21.15.5:0/2567236359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1a03d41b0 msgr2=0x55d1a0778500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:41.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.913+0000 7fcf33998740 1 --2- 172.21.15.5:0/2567236359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1a03d41b0 0x55d1a0778500 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fcf20035000 tx=0x7fcf2002f410 comp rx=0 tx=0).stop 2024-05-09T01:00:41.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.913+0000 7fcf177fe640 1 -- 172.21.15.5:0/3602952550 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fcf180e2510 con 0x7fcf281043c0 2024-05-09T01:00:41.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.913+0000 7fcf33998740 1 -- 172.21.15.5:0/2567236359 shutdown_connections 2024-05-09T01:00:41.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.913+0000 7fcf33998740 1 --2- 172.21.15.5:0/2567236359 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1a03b34d0 0x55d1a077f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:41.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.913+0000 7fcf33998740 1 --2- 172.21.15.5:0/2567236359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1a03ab2f0 0x55d1a0778a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:41.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.913+0000 7fcf33998740 1 --2- 172.21.15.5:0/2567236359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1a03d41b0 0x55d1a0778500 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:41.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.913+0000 7fcf33998740 1 -- 172.21.15.5:0/2567236359 >> 172.21.15.5:0/2567236359 conn(0x55d1a03c8300 msgr2=0x55d1a0773e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:41.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.913+0000 7fcf33998740 1 -- 172.21.15.5:0/2567236359 shutdown_connections 2024-05-09T01:00:41.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.913+0000 7fcf33998740 1 -- 172.21.15.5:0/2567236359 wait complete. 2024-05-09T01:00:41.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.913+0000 7fcf2659c640 1 --2- 172.21.15.5:0/3602952550 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fcf100b27e0 0x7fcf100b4cf0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fcf180a5b80 tx=0x7fcf180733f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:41.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.913+0000 7fcf33998740 1 -- start start 2024-05-09T01:00:41.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.914+0000 7fcf33998740 1 Processor -- start 2024-05-09T01:00:41.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.914+0000 7fcf33998740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1a03ab2f0 0x55d1a07822a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:41.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.914+0000 7fcf33998740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1a03b34d0 0x55d1a07827e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:41.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.914+0000 7fcf33998740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1a03d41b0 0x55d1a07896d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:41.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.914+0000 7fcf33998740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d1a0780370 con 0x55d1a03d41b0 2024-05-09T01:00:41.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.914+0000 7fcf33998740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55d1a06639c0 con 0x55d1a03ab2f0 2024-05-09T01:00:41.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.914+0000 7fcf33998740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d1a07804f0 con 0x55d1a03b34d0 2024-05-09T01:00:41.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.914+0000 7fcf33998740 10 client.0 _ll_register_callbacks cb 0x55d1a0690020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:00:41.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.914+0000 7fcf26d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1a03d41b0 0x55d1a07896d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:41.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.914+0000 7fcf26d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1a03d41b0 0x55d1a07896d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50386/0 (socket says 172.21.15.5:50386) 2024-05-09T01:00:41.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.914+0000 7fcf26d9d640 1 -- 172.21.15.5:0/3991998499 learned_addr learned my addr 172.21.15.5:0/3991998499 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:00:41.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.914+0000 7fcf25d9b640 1 --2- 172.21.15.5:0/3991998499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1a03b34d0 0x55d1a07827e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:41.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.914+0000 7fcf26d9d640 1 --2- 172.21.15.5:0/3991998499 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1a03ab2f0 0x55d1a07822a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:41.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.914+0000 7fcf26d9d640 1 -- 172.21.15.5:0/3991998499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1a03b34d0 msgr2=0x55d1a07827e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:41.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.914+0000 7fcf26d9d640 1 --2- 172.21.15.5:0/3991998499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1a03b34d0 0x55d1a07827e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:41.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.914+0000 7fcf26d9d640 1 -- 172.21.15.5:0/3991998499 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1a03ab2f0 msgr2=0x55d1a07822a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:41.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.914+0000 7fcf26d9d640 1 --2- 172.21.15.5:0/3991998499 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1a03ab2f0 0x55d1a07822a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:41.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.914+0000 7fcf26d9d640 1 -- 172.21.15.5:0/3991998499 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1a068a1f0 con 0x55d1a03d41b0 2024-05-09T01:00:41.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.915+0000 7fcf26d9d640 1 --2- 172.21.15.5:0/3991998499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1a03d41b0 0x55d1a07896d0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fcf20004af0 tx=0x7fcf2003c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=908611b2e1d21688 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:41.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.915+0000 7fcf0dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T01:00:41.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.915+0000 7fcf0dffb640 1 -- 172.21.15.5:0/3991998499 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2005b060 con 0x55d1a03d41b0 2024-05-09T01:00:41.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.915+0000 7fcf0dffb640 1 -- 172.21.15.5:0/3991998499 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fcf20003d80 con 0x55d1a03d41b0 2024-05-09T01:00:41.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.915+0000 7fcf33998740 1 -- 172.21.15.5:0/3991998499 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf20036040 con 0x55d1a03d41b0 2024-05-09T01:00:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.915+0000 7fcf0dffb640 1 -- 172.21.15.5:0/3991998499 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fcf2004c070 con 0x55d1a03d41b0 2024-05-09T01:00:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.915+0000 7fcf33998740 10 client.17091 fetch_fsmap learned FSMap version 32 2024-05-09T01:00:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.915+0000 7fcf33998740 1 -- 172.21.15.5:0/3991998499 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d1a043f790 con 0x55d1a03d41b0 2024-05-09T01:00:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.915+0000 7fcf0dffb640 1 -- 172.21.15.5:0/3991998499 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf20036040 con 0x55d1a03d41b0 2024-05-09T01:00:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.915+0000 7fcf0dffb640 1 -- 172.21.15.5:0/3991998499 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fcf2002f660 con 0x55d1a03d41b0 2024-05-09T01:00:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.916+0000 7fcf33998740 10 client.17091 fetch_fsmap finished waiting for FSMap version 32 2024-05-09T01:00:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.916+0000 7fcf33998740 10 client.17091 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T01:00:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.916+0000 7fcf33998740 20 client.17091 populate_metadata read hostname 'smithi005' 2024-05-09T01:00:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.916+0000 7fcf33998740 1 --2- 172.21.15.5:0/3991998499 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55d1a0392d80 0x55d1a078f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.916+0000 7fcf33998740 4 client.17091 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:00:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.916+0000 7fcf33998740 1 -- 172.21.15.5:0/3991998499 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1a06ea3d0 con 0x55d1a0392d80 2024-05-09T01:00:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.916+0000 7fcf25d9b640 1 --2- 172.21.15.5:0/3991998499 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55d1a0392d80 0x55d1a078f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.916+0000 7fcf25d9b640 1 --2- 172.21.15.5:0/3991998499 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55d1a0392d80 0x55d1a078f790 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c4b69f805074ad5 server_cookie=d62d89c1b6ab8443 in_seq=0 out_seq=0 2024-05-09T01:00:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.916+0000 7fcf0dffb640 10 client.17091 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T01:00:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.918+0000 7fcf0dffb640 1 -- 172.21.15.5:0/3991998499 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d1a06ea3d0 con 0x55d1a0392d80 2024-05-09T01:00:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.918+0000 7fcf0dffb640 10 client.17091 handle_command_reply: tid=0 2024-05-09T01:00:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.918+0000 7fcf33998740 1 client.17091 shutdown 2024-05-09T01:00:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.918+0000 7fcf33998740 1 -- 172.21.15.5:0/3991998499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1a03d41b0 msgr2=0x55d1a07896d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.918+0000 7fcf33998740 1 --2- 172.21.15.5:0/3991998499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1a03d41b0 0x55d1a07896d0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fcf20004af0 tx=0x7fcf2003c720 comp rx=0 tx=0).stop 2024-05-09T01:00:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.918+0000 7fcf33998740 1 -- 172.21.15.5:0/3991998499 shutdown_connections 2024-05-09T01:00:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.918+0000 7fcf33998740 1 --2- 172.21.15.5:0/3991998499 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55d1a0392d80 0x55d1a078f790 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.918+0000 7fcf33998740 1 --2- 172.21.15.5:0/3991998499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1a03d41b0 0x55d1a07896d0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.918+0000 7fcf33998740 1 --2- 172.21.15.5:0/3991998499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1a03b34d0 0x55d1a07827e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.918+0000 7fcf33998740 1 --2- 172.21.15.5:0/3991998499 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1a03ab2f0 0x55d1a07822a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.918+0000 7fcf33998740 1 -- 172.21.15.5:0/3991998499 >> 172.21.15.5:0/3991998499 conn(0x55d1a03c8300 msgr2=0x55d1a0776da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.918+0000 7fcf0dffb640 0 client.17091 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T01:00:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.918+0000 7fcf33998740 1 -- 172.21.15.5:0/3991998499 shutdown_connections 2024-05-09T01:00:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.918+0000 7fcf33998740 1 -- 172.21.15.5:0/3991998499 wait complete. 2024-05-09T01:00:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.918+0000 7fcf33998740 20 client.17091 trim_cache size 0 max 16384 2024-05-09T01:00:41.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.933+0000 7fcf33998740 1 Processor -- start 2024-05-09T01:00:41.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.934+0000 7fcf33998740 1 -- start start 2024-05-09T01:00:41.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.934+0000 7fcf33998740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1a03ab2f0 0x55d1a07808f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:41.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.934+0000 7fcf33998740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1a03b34d0 0x55d1a0780e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.934+0000 7fcf33998740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1a03d41b0 0x55d1a077c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.934+0000 7fcf33998740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcf281a9110 con 0x55d1a03ab2f0 2024-05-09T01:00:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.934+0000 7fcf33998740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcf281aac50 con 0x55d1a03d41b0 2024-05-09T01:00:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.934+0000 7fcf33998740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcf281a97e0 con 0x55d1a03b34d0 2024-05-09T01:00:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.934+0000 7fcf26d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1a03ab2f0 0x55d1a07808f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.934+0000 7fcf25d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1a03b34d0 0x55d1a0780e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.934+0000 7fcf25d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1a03b34d0 0x55d1a0780e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:41694/0 (socket says 172.21.15.5:41694) 2024-05-09T01:00:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.934+0000 7fcf26d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1a03ab2f0 0x55d1a07808f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50400/0 (socket says 172.21.15.5:50400) 2024-05-09T01:00:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.934+0000 7fcf25d9b640 1 -- 172.21.15.5:0/2134902519 learned_addr learned my addr 172.21.15.5:0/2134902519 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:00:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.934+0000 7fcf25d9b640 1 -- 172.21.15.5:0/2134902519 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1a03d41b0 msgr2=0x55d1a077c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.934+0000 7fcf25d9b640 1 --2- 172.21.15.5:0/2134902519 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1a03d41b0 0x55d1a077c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.934+0000 7fcf25d9b640 1 -- 172.21.15.5:0/2134902519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1a03ab2f0 msgr2=0x55d1a07808f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.934+0000 7fcf25d9b640 1 --2- 172.21.15.5:0/2134902519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1a03ab2f0 0x55d1a07808f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.934+0000 7fcf25d9b640 1 -- 172.21.15.5:0/2134902519 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1a04b65f0 con 0x55d1a03b34d0 2024-05-09T01:00:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.934+0000 7fcf26d9d640 1 --2- 172.21.15.5:0/2134902519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1a03ab2f0 0x55d1a07808f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:00:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.935+0000 7fcf25d9b640 1 --2- 172.21.15.5:0/2134902519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1a03b34d0 0x55d1a0780e30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x55d1a078fe30 tx=0x7fcf1c005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58328e4b5d6f2564 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.935+0000 7fcf0ffff640 1 -- 172.21.15.5:0/2134902519 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c025070 con 0x55d1a03b34d0 2024-05-09T01:00:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.935+0000 7fcf0ffff640 1 -- 172.21.15.5:0/2134902519 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf1c003ca0 con 0x55d1a03b34d0 2024-05-09T01:00:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.935+0000 7fcf0ffff640 1 -- 172.21.15.5:0/2134902519 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c0203d0 con 0x55d1a03b34d0 2024-05-09T01:00:41.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.936+0000 7fcf33998740 1 -- 172.21.15.5:0/2134902519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1a03b34d0 msgr2=0x55d1a0780e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:41.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.936+0000 7fcf33998740 1 --2- 172.21.15.5:0/2134902519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1a03b34d0 0x55d1a0780e30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x55d1a078fe30 tx=0x7fcf1c005f90 comp rx=0 tx=0).stop 2024-05-09T01:00:41.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.936+0000 7fcf33998740 1 -- 172.21.15.5:0/2134902519 shutdown_connections 2024-05-09T01:00:41.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.936+0000 7fcf33998740 1 --2- 172.21.15.5:0/2134902519 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1a03d41b0 0x55d1a077c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:41.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.936+0000 7fcf33998740 1 --2- 172.21.15.5:0/2134902519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1a03b34d0 0x55d1a0780e30 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:41.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.936+0000 7fcf33998740 1 --2- 172.21.15.5:0/2134902519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1a03ab2f0 0x55d1a07808f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:41.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.936+0000 7fcf33998740 1 -- 172.21.15.5:0/2134902519 >> 172.21.15.5:0/2134902519 conn(0x55d1a03c8300 msgr2=0x55d1a0776c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:41.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.936+0000 7fcf33998740 1 -- 172.21.15.5:0/2134902519 shutdown_connections 2024-05-09T01:00:41.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.936+0000 7fcf33998740 1 -- 172.21.15.5:0/2134902519 wait complete. 2024-05-09T01:00:41.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.936+0000 7fcf33998740 1 -- start start 2024-05-09T01:00:41.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.936+0000 7fcf33998740 1 Processor -- start 2024-05-09T01:00:41.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.936+0000 7fcf33998740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1a03ab2f0 0x55d1a0785e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:41.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.936+0000 7fcf33998740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1a03b34d0 0x55d1a07863b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:41.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.936+0000 7fcf33998740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1a03d41b0 0x55d1a07872e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:41.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.936+0000 7fcf33998740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcf281aa580 con 0x55d1a03d41b0 2024-05-09T01:00:41.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.936+0000 7fcf33998740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcf281a9eb0 con 0x55d1a03ab2f0 2024-05-09T01:00:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.936+0000 7fcf33998740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcf281a70f0 con 0x55d1a03b34d0 2024-05-09T01:00:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.936+0000 7fcf33998740 10 client.0 _ll_register_callbacks cb 0x7fcf2004a070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:00:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.937+0000 7fcf26d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1a03d41b0 0x55d1a07872e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.937+0000 7fcf25d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1a03b34d0 0x55d1a07863b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.937+0000 7fcf26d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1a03d41b0 0x55d1a07872e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50406/0 (socket says 172.21.15.5:50406) 2024-05-09T01:00:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.937+0000 7fcf26d9d640 1 -- 172.21.15.5:0/885460131 learned_addr learned my addr 172.21.15.5:0/885460131 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:00:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.937+0000 7fcf26d9d640 1 --2- 172.21.15.5:0/885460131 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1a03ab2f0 0x55d1a0785e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.937+0000 7fcf26d9d640 1 -- 172.21.15.5:0/885460131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1a03b34d0 msgr2=0x55d1a07863b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.937+0000 7fcf26d9d640 1 --2- 172.21.15.5:0/885460131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1a03b34d0 0x55d1a07863b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.937+0000 7fcf26d9d640 1 -- 172.21.15.5:0/885460131 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1a03ab2f0 msgr2=0x55d1a0785e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.937+0000 7fcf26d9d640 1 --2- 172.21.15.5:0/885460131 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1a03ab2f0 0x55d1a0785e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.937+0000 7fcf26d9d640 1 -- 172.21.15.5:0/885460131 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1a06a0ed0 con 0x55d1a03d41b0 2024-05-09T01:00:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.937+0000 7fcf26d9d640 1 --2- 172.21.15.5:0/885460131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1a03d41b0 0x55d1a07872e0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fcf2002fab0 tx=0x7fcf200021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ef521d0832b31d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.937+0000 7fcf0dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T01:00:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.937+0000 7fcf0dffb640 1 -- 172.21.15.5:0/885460131 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2005b040 con 0x55d1a03d41b0 2024-05-09T01:00:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.937+0000 7fcf0dffb640 1 -- 172.21.15.5:0/885460131 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fcf20004b40 con 0x55d1a03d41b0 2024-05-09T01:00:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.937+0000 7fcf33998740 1 -- 172.21.15.5:0/885460131 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf1c003ca0 con 0x55d1a03d41b0 2024-05-09T01:00:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.937+0000 7fcf0dffb640 1 -- 172.21.15.5:0/885460131 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fcf20004d40 con 0x55d1a03d41b0 2024-05-09T01:00:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.937+0000 7fcf33998740 10 client.17097 fetch_fsmap learned FSMap version 32 2024-05-09T01:00:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.937+0000 7fcf33998740 1 -- 172.21.15.5:0/885460131 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d1a0788960 con 0x55d1a03d41b0 2024-05-09T01:00:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.938+0000 7fcf0dffb640 1 -- 172.21.15.5:0/885460131 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf1c003ca0 con 0x55d1a03d41b0 2024-05-09T01:00:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.938+0000 7fcf0dffb640 1 -- 172.21.15.5:0/885460131 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fcf20073070 con 0x55d1a03d41b0 2024-05-09T01:00:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.938+0000 7fcf33998740 10 client.17097 fetch_fsmap finished waiting for FSMap version 32 2024-05-09T01:00:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.938+0000 7fcf33998740 10 client.17097 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T01:00:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.938+0000 7fcf33998740 20 client.17097 populate_metadata read hostname 'smithi005' 2024-05-09T01:00:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.938+0000 7fcf33998740 1 --2- 172.21.15.5:0/885460131 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55d1a03681c0 0x55d1a07f9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.938+0000 7fcf33998740 4 client.17097 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T01:00:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.938+0000 7fcf33998740 1 -- 172.21.15.5:0/885460131 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d1a06e4240 con 0x55d1a03681c0 2024-05-09T01:00:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.938+0000 7fcf25d9b640 1 --2- 172.21.15.5:0/885460131 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55d1a03681c0 0x55d1a07f9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.938+0000 7fcf0dffb640 10 client.17097 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T01:00:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:41.938+0000 7fcf25d9b640 1 --2- 172.21.15.5:0/885460131 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55d1a03681c0 0x55d1a07f9ad0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7cdf3c4c549b3cc server_cookie=e441fb51beb11654 in_seq=0 out_seq=0 2024-05-09T01:00:42.272 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:00:42 smithi005 ceph-mon[30365]: pgmap v801: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 15 MiB/s wr, 64 op/s 2024-05-09T01:00:42.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:00:42 smithi131 ceph-mon[34547]: pgmap v801: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 15 MiB/s wr, 64 op/s 2024-05-09T01:00:42.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:00:42 smithi157 ceph-mon[34294]: pgmap v801: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 15 MiB/s wr, 64 op/s 2024-05-09T01:00:42.725 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T01:00:42.982 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:00:44.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:00:44 smithi131 ceph-mon[34547]: pgmap v802: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 22 MiB/s wr, 90 op/s 2024-05-09T01:00:44.371 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:00:44 smithi005 ceph-mon[30365]: pgmap v802: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 22 MiB/s wr, 90 op/s 2024-05-09T01:00:44.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:00:44 smithi157 ceph-mon[34294]: pgmap v802: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 22 MiB/s wr, 90 op/s 2024-05-09T01:00:44.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.841+0000 7f9c7060a700 1 -- 172.21.15.5:0/3391231630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c681034c0 msgr2=0x7f9c681038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:44.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.841+0000 7f9c7060a700 1 --2- 172.21.15.5:0/3391231630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c681034c0 0x7f9c681038d0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f9c640099c0 tx=0x7f9c64009cd0 comp rx=0 tx=0).stop 2024-05-09T01:00:44.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.841+0000 7f9c7060a700 1 -- 172.21.15.5:0/3391231630 shutdown_connections 2024-05-09T01:00:44.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.841+0000 7f9c7060a700 1 --2- 172.21.15.5:0/3391231630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c68108740 0x7f9c681100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:44.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.841+0000 7f9c7060a700 1 --2- 172.21.15.5:0/3391231630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c681034c0 0x7f9c681038d0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:44.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.841+0000 7f9c7060a700 1 --2- 172.21.15.5:0/3391231630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c68107df0 0x7f9c68108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:44.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.841+0000 7f9c7060a700 1 -- 172.21.15.5:0/3391231630 >> 172.21.15.5:0/3391231630 conn(0x7f9c680fedc0 msgr2=0x7f9c681011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:44.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.841+0000 7f9c7060a700 1 -- 172.21.15.5:0/3391231630 shutdown_connections 2024-05-09T01:00:44.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.841+0000 7f9c7060a700 1 -- 172.21.15.5:0/3391231630 wait complete. 2024-05-09T01:00:44.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.842+0000 7f9c7060a700 1 Processor -- start 2024-05-09T01:00:44.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.842+0000 7f9c7060a700 1 -- start start 2024-05-09T01:00:44.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.843+0000 7f9c7060a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c681034c0 0x7f9c6819f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:44.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.843+0000 7f9c7060a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c68107df0 0x7f9c6819ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:44.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.843+0000 7f9c7060a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c68108740 0x7f9c68199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:44.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.843+0000 7f9c7060a700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9c68113370 con 0x7f9c68108740 2024-05-09T01:00:44.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.843+0000 7f9c7060a700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9c681131f0 con 0x7f9c68107df0 2024-05-09T01:00:44.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.843+0000 7f9c7060a700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9c681134f0 con 0x7f9c681034c0 2024-05-09T01:00:44.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.843+0000 7f9c6e3a6700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c681034c0 0x7f9c6819f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:44.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.843+0000 7f9c6eba7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c68108740 0x7f9c68199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:44.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.843+0000 7f9c6dba5700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c68107df0 0x7f9c6819ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:44.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.843+0000 7f9c6e3a6700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c681034c0 0x7f9c6819f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:54660/0 (socket says 172.21.15.5:54660) 2024-05-09T01:00:44.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.843+0000 7f9c6e3a6700 1 -- 172.21.15.5:0/1033210836 learned_addr learned my addr 172.21.15.5:0/1033210836 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:00:44.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.843+0000 7f9c6e3a6700 1 -- 172.21.15.5:0/1033210836 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c68107df0 msgr2=0x7f9c6819ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:44.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.843+0000 7f9c6e3a6700 1 --2- 172.21.15.5:0/1033210836 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c68107df0 0x7f9c6819ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:44.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.843+0000 7f9c6e3a6700 1 -- 172.21.15.5:0/1033210836 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c68108740 msgr2=0x7f9c68199b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:44.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.843+0000 7f9c6e3a6700 1 --2- 172.21.15.5:0/1033210836 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c68108740 0x7f9c68199b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:44.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.843+0000 7f9c6e3a6700 1 -- 172.21.15.5:0/1033210836 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c6819a280 con 0x7f9c681034c0 2024-05-09T01:00:44.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.843+0000 7f9c6dba5700 1 --2- 172.21.15.5:0/1033210836 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c68107df0 0x7f9c6819ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:00:44.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.843+0000 7f9c6eba7700 1 --2- 172.21.15.5:0/1033210836 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c68108740 0x7f9c68199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:00:44.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.844+0000 7f9c6e3a6700 1 --2- 172.21.15.5:0/1033210836 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c681034c0 0x7f9c6819f9c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9c64006010 tx=0x7f9c6400bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:44.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.844+0000 7f9c5b7fe700 1 -- 172.21.15.5:0/1033210836 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c64009e60 con 0x7f9c681034c0 2024-05-09T01:00:44.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.844+0000 7f9c7060a700 1 -- 172.21.15.5:0/1033210836 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c6819a510 con 0x7f9c681034c0 2024-05-09T01:00:44.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.844+0000 7f9c5b7fe700 1 -- 172.21.15.5:0/1033210836 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c64003c70 con 0x7f9c681034c0 2024-05-09T01:00:44.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.844+0000 7f9c5b7fe700 1 -- 172.21.15.5:0/1033210836 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c64019ae0 con 0x7f9c681034c0 2024-05-09T01:00:44.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.844+0000 7f9c7060a700 1 -- 172.21.15.5:0/1033210836 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c6806b610 con 0x7f9c681034c0 2024-05-09T01:00:44.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.847+0000 7f9c5b7fe700 1 -- 172.21.15.5:0/1033210836 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f9c64019c80 con 0x7f9c681034c0 2024-05-09T01:00:44.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.848+0000 7f9c7f020b80 1 Processor -- start 2024-05-09T01:00:44.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.848+0000 7f9c5b7fe700 1 --2- 172.21.15.5:0/1033210836 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f9c540b2880 0x7f9c540b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:44.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.848+0000 7f9c6dba5700 1 --2- 172.21.15.5:0/1033210836 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f9c540b2880 0x7f9c540b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:44.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.848+0000 7f9c7f020b80 1 -- start start 2024-05-09T01:00:44.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.848+0000 7f9c5b7fe700 1 -- 172.21.15.5:0/1033210836 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f9c640e31d0 con 0x7f9c681034c0 2024-05-09T01:00:44.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.848+0000 7f9c7f020b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55edd699c160 0x55edd6a3dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:44.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.848+0000 7f9c7f020b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edd68b24a0 0x55edd6a3e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:44.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.848+0000 7f9c7f020b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55edd6a37eb0 0x55edd6a382c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:44.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.848+0000 7f9c6dba5700 1 --2- 172.21.15.5:0/1033210836 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f9c540b2880 0x7f9c540b4d90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9c6819b280 tx=0x7f9c5c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:44.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.848+0000 7f9c7f020b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55edd6951ff0 con 0x55edd68b24a0 2024-05-09T01:00:44.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.848+0000 7f9c7f020b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55edd68f07f0 con 0x55edd6a37eb0 2024-05-09T01:00:44.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.848+0000 7f9c7f020b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55edd694dfb0 con 0x55edd699c160 2024-05-09T01:00:44.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.849+0000 7f9c6e3a6700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55edd699c160 0x55edd6a3dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:44.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.849+0000 7f9c6dba5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edd68b24a0 0x55edd6a3e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:44.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.849+0000 7f9c6e3a6700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55edd699c160 0x55edd6a3dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:54668/0 (socket says 172.21.15.5:54668) 2024-05-09T01:00:44.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.849+0000 7f9c6eba7700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55edd6a37eb0 0x55edd6a382c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:44.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.849+0000 7f9c6e3a6700 1 -- 172.21.15.5:0/4277414362 learned_addr learned my addr 172.21.15.5:0/4277414362 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:00:44.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.849+0000 7f9c6e3a6700 1 -- 172.21.15.5:0/4277414362 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55edd6a37eb0 msgr2=0x55edd6a382c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:44.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.849+0000 7f9c6e3a6700 1 --2- 172.21.15.5:0/4277414362 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55edd6a37eb0 0x55edd6a382c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:44.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.849+0000 7f9c6e3a6700 1 -- 172.21.15.5:0/4277414362 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edd68b24a0 msgr2=0x55edd6a3e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:44.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.849+0000 7f9c6e3a6700 1 --2- 172.21.15.5:0/4277414362 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edd68b24a0 0x55edd6a3e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:44.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.849+0000 7f9c6e3a6700 1 -- 172.21.15.5:0/4277414362 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edd68e8370 con 0x55edd699c160 2024-05-09T01:00:44.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.849+0000 7f9c6dba5700 1 --2- 172.21.15.5:0/4277414362 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edd68b24a0 0x55edd6a3e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:00:44.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.849+0000 7f9c6e3a6700 1 --2- 172.21.15.5:0/4277414362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55edd699c160 0x55edd6a3dcb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9c640a6870 tx=0x7f9c64003700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17400414fa984692 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:44.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.849+0000 7f9c58ff9700 1 -- 172.21.15.5:0/4277414362 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c640b10e0 con 0x55edd699c160 2024-05-09T01:00:44.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.849+0000 7f9c58ff9700 1 -- 172.21.15.5:0/4277414362 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c640aa300 con 0x55edd699c160 2024-05-09T01:00:44.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.849+0000 7f9c58ff9700 1 -- 172.21.15.5:0/4277414362 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c640165a0 con 0x55edd699c160 2024-05-09T01:00:44.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.850+0000 7f9c7f020b80 1 -- 172.21.15.5:0/4277414362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55edd699c160 msgr2=0x55edd6a3dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:44.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.850+0000 7f9c7f020b80 1 --2- 172.21.15.5:0/4277414362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55edd699c160 0x55edd6a3dcb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9c640a6870 tx=0x7f9c64003700 comp rx=0 tx=0).stop 2024-05-09T01:00:44.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.850+0000 7f9c7f020b80 1 -- 172.21.15.5:0/4277414362 shutdown_connections 2024-05-09T01:00:44.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.850+0000 7f9c7f020b80 1 --2- 172.21.15.5:0/4277414362 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55edd6a37eb0 0x55edd6a382c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:44.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.850+0000 7f9c7f020b80 1 --2- 172.21.15.5:0/4277414362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55edd699c160 0x55edd6a3dcb0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:44.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.850+0000 7f9c7f020b80 1 --2- 172.21.15.5:0/4277414362 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edd68b24a0 0x55edd6a3e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:44.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.850+0000 7f9c7f020b80 1 -- 172.21.15.5:0/4277414362 >> 172.21.15.5:0/4277414362 conn(0x55edd69126a0 msgr2=0x55edd6a2a400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:44.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.851+0000 7f9c7f020b80 1 -- 172.21.15.5:0/4277414362 shutdown_connections 2024-05-09T01:00:44.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.851+0000 7f9c7f020b80 1 -- 172.21.15.5:0/4277414362 wait complete. 2024-05-09T01:00:44.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.851+0000 7f9c7f020b80 1 -- start start 2024-05-09T01:00:44.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.851+0000 7f9c7f020b80 1 Processor -- start 2024-05-09T01:00:44.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.852+0000 7f9c7f020b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edd68b24a0 0x55edd6a47310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:44.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.852+0000 7f9c7f020b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55edd699c160 0x55edd6a47850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:44.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.852+0000 7f9c7f020b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55edd6a37eb0 0x55edd6a41710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:44.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.852+0000 7f9c7f020b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55edd67cdba0 con 0x55edd68b24a0 2024-05-09T01:00:44.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.852+0000 7f9c7f020b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55edd690c150 con 0x55edd6a37eb0 2024-05-09T01:00:44.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.852+0000 7f9c7f020b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55edd694dfb0 con 0x55edd699c160 2024-05-09T01:00:44.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.852+0000 7f9c7f020b80 10 client.0 _ll_register_callbacks cb 0x55edd67c36c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:00:44.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.852+0000 7f9c6e3a6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edd68b24a0 0x55edd6a47310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:44.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.852+0000 7f9c6e3a6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edd68b24a0 0x55edd6a47310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52986/0 (socket says 172.21.15.5:52986) 2024-05-09T01:00:44.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.852+0000 7f9c6e3a6700 1 -- 172.21.15.5:0/4018876569 learned_addr learned my addr 172.21.15.5:0/4018876569 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:00:44.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.852+0000 7f9c6dba5700 1 --2- 172.21.15.5:0/4018876569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55edd699c160 0x55edd6a47850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:44.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.852+0000 7f9c6eba7700 1 --2- 172.21.15.5:0/4018876569 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55edd6a37eb0 0x55edd6a41710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:44.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.852+0000 7f9c6e3a6700 1 -- 172.21.15.5:0/4018876569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55edd699c160 msgr2=0x55edd6a47850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:44.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.852+0000 7f9c6e3a6700 1 --2- 172.21.15.5:0/4018876569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55edd699c160 0x55edd6a47850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:44.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.852+0000 7f9c6e3a6700 1 -- 172.21.15.5:0/4018876569 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55edd6a37eb0 msgr2=0x55edd6a41710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:44.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.852+0000 7f9c6e3a6700 1 --2- 172.21.15.5:0/4018876569 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55edd6a37eb0 0x55edd6a41710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:44.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.852+0000 7f9c6e3a6700 1 -- 172.21.15.5:0/4018876569 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edd67cd490 con 0x55edd68b24a0 2024-05-09T01:00:44.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.852+0000 7f9c6dba5700 1 --2- 172.21.15.5:0/4018876569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55edd699c160 0x55edd6a47850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:00:44.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.853+0000 7f9c6e3a6700 1 --2- 172.21.15.5:0/4018876569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edd68b24a0 0x55edd6a47310 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f9c64011040 tx=0x7f9c640aad00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a515bf69593ec5e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:44.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.853+0000 7f9c51ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T01:00:44.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.853+0000 7f9c51ffb700 1 -- 172.21.15.5:0/4018876569 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c640c4060 con 0x55edd68b24a0 2024-05-09T01:00:44.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.853+0000 7f9c51ffb700 1 -- 172.21.15.5:0/4018876569 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f9c640b3340 con 0x55edd68b24a0 2024-05-09T01:00:44.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.853+0000 7f9c7f020b80 1 -- 172.21.15.5:0/4018876569 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c640aa300 con 0x55edd68b24a0 2024-05-09T01:00:44.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.853+0000 7f9c51ffb700 1 -- 172.21.15.5:0/4018876569 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9c640aa7f0 con 0x55edd68b24a0 2024-05-09T01:00:44.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.853+0000 7f9c7f020b80 10 client.17109 fetch_fsmap learned FSMap version 32 2024-05-09T01:00:44.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.853+0000 7f9c7f020b80 1 -- 172.21.15.5:0/4018876569 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55edd6a4ef30 con 0x55edd68b24a0 2024-05-09T01:00:44.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.854+0000 7f9c51ffb700 1 -- 172.21.15.5:0/4018876569 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c640aa300 con 0x55edd68b24a0 2024-05-09T01:00:44.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.854+0000 7f9c51ffb700 1 -- 172.21.15.5:0/4018876569 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f9c64021070 con 0x55edd68b24a0 2024-05-09T01:00:44.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.854+0000 7f9c7f020b80 10 client.17109 fetch_fsmap finished waiting for FSMap version 32 2024-05-09T01:00:44.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.854+0000 7f9c7f020b80 10 client.17109 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T01:00:44.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.854+0000 7f9c7f020b80 20 client.17109 populate_metadata read hostname 'smithi005' 2024-05-09T01:00:44.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.854+0000 7f9c7f020b80 1 --2- 172.21.15.5:0/4018876569 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55edd6a514f0 0x55edd6a53930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:44.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.854+0000 7f9c7f020b80 4 client.17109 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:00:44.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.854+0000 7f9c7f020b80 1 -- 172.21.15.5:0/4018876569 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55edd695f9d0 con 0x55edd6a514f0 2024-05-09T01:00:44.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.854+0000 7f9c6dba5700 1 --2- 172.21.15.5:0/4018876569 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55edd6a514f0 0x55edd6a53930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:44.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.855+0000 7f9c6dba5700 1 --2- 172.21.15.5:0/4018876569 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55edd6a514f0 0x55edd6a53930 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9776999078dfbc2d server_cookie=9e559768913aed4f in_seq=0 out_seq=0 2024-05-09T01:00:44.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.855+0000 7f9c51ffb700 10 client.17109 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T01:00:44.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.857+0000 7f9c51ffb700 1 -- 172.21.15.5:0/4018876569 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55edd695f9d0 con 0x55edd6a514f0 2024-05-09T01:00:44.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.857+0000 7f9c51ffb700 10 client.17109 handle_command_reply: tid=0 2024-05-09T01:00:44.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.858+0000 7f9c7f020b80 1 client.17109 shutdown 2024-05-09T01:00:44.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.858+0000 7f9c7f020b80 1 -- 172.21.15.5:0/4018876569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edd68b24a0 msgr2=0x55edd6a47310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:44.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.858+0000 7f9c7f020b80 1 --2- 172.21.15.5:0/4018876569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edd68b24a0 0x55edd6a47310 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f9c64011040 tx=0x7f9c640aad00 comp rx=0 tx=0).stop 2024-05-09T01:00:44.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.858+0000 7f9c7f020b80 1 -- 172.21.15.5:0/4018876569 shutdown_connections 2024-05-09T01:00:44.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.858+0000 7f9c7f020b80 1 --2- 172.21.15.5:0/4018876569 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55edd6a514f0 0x55edd6a53930 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:44.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.858+0000 7f9c51ffb700 0 client.17109 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T01:00:44.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.858+0000 7f9c7f020b80 1 --2- 172.21.15.5:0/4018876569 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55edd6a37eb0 0x55edd6a41710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:44.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.858+0000 7f9c7f020b80 1 --2- 172.21.15.5:0/4018876569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edd68b24a0 0x55edd6a47310 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:44.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.858+0000 7f9c7f020b80 1 --2- 172.21.15.5:0/4018876569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55edd699c160 0x55edd6a47850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:44.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.858+0000 7f9c7f020b80 1 -- 172.21.15.5:0/4018876569 >> 172.21.15.5:0/4018876569 conn(0x55edd69126a0 msgr2=0x55edd6a2a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:44.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.858+0000 7f9c7f020b80 1 -- 172.21.15.5:0/4018876569 shutdown_connections 2024-05-09T01:00:44.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.858+0000 7f9c7f020b80 1 -- 172.21.15.5:0/4018876569 wait complete. 2024-05-09T01:00:44.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.858+0000 7f9c7f020b80 20 client.17109 trim_cache size 0 max 16384 2024-05-09T01:00:44.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.876+0000 7f9c7f020b80 1 Processor -- start 2024-05-09T01:00:44.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.876+0000 7f9c7f020b80 1 -- start start 2024-05-09T01:00:44.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.876+0000 7f9c7f020b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55edd68b24a0 0x55edd6a3bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:44.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.876+0000 7f9c7f020b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55edd699c160 0x55edd6a590f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:44.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.876+0000 7f9c7f020b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edd6a37eb0 0x55edd6a59630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:44.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.876+0000 7f9c7f020b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9c681aca00 con 0x55edd6a37eb0 2024-05-09T01:00:44.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.876+0000 7f9c7f020b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9c681ae540 con 0x55edd699c160 2024-05-09T01:00:44.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.876+0000 7f9c7f020b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9c681ad0d0 con 0x55edd68b24a0 2024-05-09T01:00:44.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.876+0000 7f9c6eba7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edd6a37eb0 0x55edd6a59630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:44.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.876+0000 7f9c6e3a6700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55edd68b24a0 0x55edd6a3bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:44.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.876+0000 7f9c6dba5700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55edd699c160 0x55edd6a590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:44.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.876+0000 7f9c6eba7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edd6a37eb0 0x55edd6a59630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53000/0 (socket says 172.21.15.5:53000) 2024-05-09T01:00:44.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.876+0000 7f9c6eba7700 1 -- 172.21.15.5:0/3777194815 learned_addr learned my addr 172.21.15.5:0/3777194815 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:00:44.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.877+0000 7f9c6dba5700 1 -- 172.21.15.5:0/3777194815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55edd68b24a0 msgr2=0x55edd6a3bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:44.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.877+0000 7f9c6dba5700 1 --2- 172.21.15.5:0/3777194815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55edd68b24a0 0x55edd6a3bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:44.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.877+0000 7f9c6dba5700 1 -- 172.21.15.5:0/3777194815 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edd6a37eb0 msgr2=0x55edd6a59630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:44.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.877+0000 7f9c6dba5700 1 --2- 172.21.15.5:0/3777194815 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edd6a37eb0 0x55edd6a59630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:44.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.877+0000 7f9c6dba5700 1 -- 172.21.15.5:0/3777194815 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edd6a2b3d0 con 0x55edd699c160 2024-05-09T01:00:44.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.877+0000 7f9c6dba5700 1 --2- 172.21.15.5:0/3777194815 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55edd699c160 0x55edd6a590f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x55edd6a54060 tx=0x7f9c5c014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73494a06541277e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:44.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.877+0000 7f9c53fff700 1 -- 172.21.15.5:0/3777194815 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c01f790 con 0x55edd699c160 2024-05-09T01:00:44.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.877+0000 7f9c53fff700 1 -- 172.21.15.5:0/3777194815 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c5c030070 con 0x55edd699c160 2024-05-09T01:00:44.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.877+0000 7f9c53fff700 1 -- 172.21.15.5:0/3777194815 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c029420 con 0x55edd699c160 2024-05-09T01:00:44.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.877+0000 7f9c7f020b80 1 -- 172.21.15.5:0/3777194815 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55edd699c160 msgr2=0x55edd6a590f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:44.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.877+0000 7f9c7f020b80 1 --2- 172.21.15.5:0/3777194815 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55edd699c160 0x55edd6a590f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x55edd6a54060 tx=0x7f9c5c014000 comp rx=0 tx=0).stop 2024-05-09T01:00:44.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.877+0000 7f9c7f020b80 1 -- 172.21.15.5:0/3777194815 shutdown_connections 2024-05-09T01:00:44.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.877+0000 7f9c7f020b80 1 --2- 172.21.15.5:0/3777194815 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edd6a37eb0 0x55edd6a59630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:44.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.877+0000 7f9c7f020b80 1 --2- 172.21.15.5:0/3777194815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55edd68b24a0 0x55edd6a3bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:44.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.877+0000 7f9c7f020b80 1 --2- 172.21.15.5:0/3777194815 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55edd699c160 0x55edd6a590f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:44.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.877+0000 7f9c7f020b80 1 -- 172.21.15.5:0/3777194815 >> 172.21.15.5:0/3777194815 conn(0x55edd69126a0 msgr2=0x55edd6a3b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:44.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.878+0000 7f9c7f020b80 1 -- 172.21.15.5:0/3777194815 shutdown_connections 2024-05-09T01:00:44.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.878+0000 7f9c7f020b80 1 -- 172.21.15.5:0/3777194815 wait complete. 2024-05-09T01:00:44.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.878+0000 7f9c7f020b80 1 -- start start 2024-05-09T01:00:44.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.878+0000 7f9c7f020b80 1 Processor -- start 2024-05-09T01:00:44.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.880+0000 7f9c7f020b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edd68b24a0 0x55edd6a4daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:44.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.880+0000 7f9c7f020b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55edd699c160 0x55edd6a4ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:44.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.880+0000 7f9c7f020b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55edd6a37eb0 0x55edd6a4b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:44.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.880+0000 7f9c6e3a6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edd68b24a0 0x55edd6a4daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:44.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.880+0000 7f9c6e3a6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edd68b24a0 0x55edd6a4daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53004/0 (socket says 172.21.15.5:53004) 2024-05-09T01:00:44.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.880+0000 7f9c6e3a6700 1 -- 172.21.15.5:0/1343102220 learned_addr learned my addr 172.21.15.5:0/1343102220 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:00:44.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.880+0000 7f9c6dba5700 1 --2- 172.21.15.5:0/1343102220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55edd699c160 0x55edd6a4ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:44.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.880+0000 7f9c6eba7700 1 --2- 172.21.15.5:0/1343102220 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55edd6a37eb0 0x55edd6a4b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:44.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.880+0000 7f9c7f020b80 1 -- 172.21.15.5:0/1343102220 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9c681ade70 con 0x55edd68b24a0 2024-05-09T01:00:44.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.880+0000 7f9c7f020b80 1 -- 172.21.15.5:0/1343102220 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9c681ad7a0 con 0x55edd6a37eb0 2024-05-09T01:00:44.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.880+0000 7f9c7f020b80 1 -- 172.21.15.5:0/1343102220 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9c681aa9a0 con 0x55edd699c160 2024-05-09T01:00:44.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.880+0000 7f9c7f020b80 10 client.0 _ll_register_callbacks cb 0x55edd67c36c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:00:44.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.880+0000 7f9c6eba7700 1 -- 172.21.15.5:0/1343102220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55edd699c160 msgr2=0x55edd6a4ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:44.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.880+0000 7f9c6eba7700 1 --2- 172.21.15.5:0/1343102220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55edd699c160 0x55edd6a4ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:44.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.880+0000 7f9c6eba7700 1 -- 172.21.15.5:0/1343102220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edd68b24a0 msgr2=0x55edd6a4daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:44.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.880+0000 7f9c6eba7700 1 --2- 172.21.15.5:0/1343102220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edd68b24a0 0x55edd6a4daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:44.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.880+0000 7f9c6eba7700 1 -- 172.21.15.5:0/1343102220 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edd6a4b920 con 0x55edd6a37eb0 2024-05-09T01:00:44.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.880+0000 7f9c6e3a6700 1 --2- 172.21.15.5:0/1343102220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edd68b24a0 0x55edd6a4daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:00:44.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.880+0000 7f9c6dba5700 1 --2- 172.21.15.5:0/1343102220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55edd699c160 0x55edd6a4ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:00:44.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.880+0000 7f9c6eba7700 1 --2- 172.21.15.5:0/1343102220 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55edd6a37eb0 0x55edd6a4b3b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x55edd695a630 tx=0x7f9c6000b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0e134c9bf20594 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:44.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.880+0000 7f9c51ffb700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-09T01:00:44.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.880+0000 7f9c51ffb700 1 -- 172.21.15.5:0/1343102220 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c60020030 con 0x55edd6a37eb0 2024-05-09T01:00:44.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.880+0000 7f9c7f020b80 1 -- 172.21.15.5:0/1343102220 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c5c030070 con 0x55edd6a37eb0 2024-05-09T01:00:44.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.880+0000 7f9c51ffb700 1 -- 172.21.15.5:0/1343102220 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f9c60004bb0 con 0x55edd6a37eb0 2024-05-09T01:00:44.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.880+0000 7f9c51ffb700 1 -- 172.21.15.5:0/1343102220 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9c60004db0 con 0x55edd6a37eb0 2024-05-09T01:00:44.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.880+0000 7f9c7f020b80 10 client.26522 fetch_fsmap learned FSMap version 32 2024-05-09T01:00:44.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.880+0000 7f9c7f020b80 1 -- 172.21.15.5:0/1343102220 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55edd6abba10 con 0x55edd6a37eb0 2024-05-09T01:00:44.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.880+0000 7f9c51ffb700 1 -- 172.21.15.5:0/1343102220 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c5c030070 con 0x55edd6a37eb0 2024-05-09T01:00:44.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.880+0000 7f9c51ffb700 1 -- 172.21.15.5:0/1343102220 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f9c60004690 con 0x55edd6a37eb0 2024-05-09T01:00:44.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.881+0000 7f9c7f020b80 10 client.26522 fetch_fsmap finished waiting for FSMap version 32 2024-05-09T01:00:44.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.881+0000 7f9c7f020b80 10 client.26522 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T01:00:44.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.881+0000 7f9c7f020b80 20 client.26522 populate_metadata read hostname 'smithi005' 2024-05-09T01:00:44.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.881+0000 7f9c7f020b80 1 --2- 172.21.15.5:0/1343102220 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55edd6abc1d0 0x55edd6abe5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:44.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.881+0000 7f9c7f020b80 4 client.26522 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:00:44.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.881+0000 7f9c7f020b80 1 -- 172.21.15.5:0/1343102220 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55edd6959a70 con 0x55edd6abc1d0 2024-05-09T01:00:44.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.881+0000 7f9c6e3a6700 1 --2- 172.21.15.5:0/1343102220 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55edd6abc1d0 0x55edd6abe5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:44.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.881+0000 7f9c51ffb700 10 client.26522 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T01:00:44.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.881+0000 7f9c6e3a6700 1 --2- 172.21.15.5:0/1343102220 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55edd6abc1d0 0x55edd6abe5f0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5715d5d758e9cd5 server_cookie=9e7d26b3dafdba25 in_seq=0 out_seq=0 2024-05-09T01:00:44.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.885+0000 7f9c51ffb700 1 -- 172.21.15.5:0/1343102220 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55edd6959a70 con 0x55edd6abc1d0 2024-05-09T01:00:44.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.885+0000 7f9c51ffb700 10 client.26522 handle_command_reply: tid=0 2024-05-09T01:00:44.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.885+0000 7f9c7f020b80 1 client.26522 shutdown 2024-05-09T01:00:44.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.885+0000 7f9c7f020b80 1 -- 172.21.15.5:0/1343102220 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55edd6a37eb0 msgr2=0x55edd6a4b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:44.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.885+0000 7f9c7f020b80 1 --2- 172.21.15.5:0/1343102220 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55edd6a37eb0 0x55edd6a4b3b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x55edd695a630 tx=0x7f9c6000b040 comp rx=0 tx=0).stop 2024-05-09T01:00:44.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.885+0000 7f9c7f020b80 1 -- 172.21.15.5:0/1343102220 shutdown_connections 2024-05-09T01:00:44.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.885+0000 7f9c7f020b80 1 --2- 172.21.15.5:0/1343102220 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55edd6abc1d0 0x55edd6abe5f0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:44.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.885+0000 7f9c7f020b80 1 --2- 172.21.15.5:0/1343102220 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55edd6a37eb0 0x55edd6a4b3b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:44.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.885+0000 7f9c7f020b80 1 --2- 172.21.15.5:0/1343102220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edd68b24a0 0x55edd6a4daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:44.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.885+0000 7f9c51ffb700 0 client.26522 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T01:00:44.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.885+0000 7f9c7f020b80 1 --2- 172.21.15.5:0/1343102220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55edd699c160 0x55edd6a4ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:44.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.885+0000 7f9c7f020b80 1 -- 172.21.15.5:0/1343102220 >> 172.21.15.5:0/1343102220 conn(0x55edd69126a0 msgr2=0x55edd6a2e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:44.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.885+0000 7f9c7f020b80 1 -- 172.21.15.5:0/1343102220 shutdown_connections 2024-05-09T01:00:44.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.885+0000 7f9c7f020b80 1 -- 172.21.15.5:0/1343102220 wait complete. 2024-05-09T01:00:44.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.885+0000 7f9c7f020b80 20 client.26522 trim_cache size 0 max 16384 2024-05-09T01:00:44.888 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-09T01:00:44.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.886+0000 7f9c7060a700 1 -- 172.21.15.5:0/1033210836 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f9c540b2880 msgr2=0x7f9c540b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:44.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.886+0000 7f9c7060a700 1 --2- 172.21.15.5:0/1033210836 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f9c540b2880 0x7f9c540b4d90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9c6819b280 tx=0x7f9c5c009260 comp rx=0 tx=0).stop 2024-05-09T01:00:44.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.886+0000 7f9c7060a700 1 -- 172.21.15.5:0/1033210836 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c681034c0 msgr2=0x7f9c6819f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:44.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.886+0000 7f9c7060a700 1 --2- 172.21.15.5:0/1033210836 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c681034c0 0x7f9c6819f9c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9c64006010 tx=0x7f9c6400bf60 comp rx=0 tx=0).stop 2024-05-09T01:00:44.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.886+0000 7f9c7060a700 1 -- 172.21.15.5:0/1033210836 shutdown_connections 2024-05-09T01:00:44.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.886+0000 7f9c7060a700 1 --2- 172.21.15.5:0/1033210836 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f9c540b2880 0x7f9c540b4d90 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:44.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.886+0000 7f9c7060a700 1 --2- 172.21.15.5:0/1033210836 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c68108740 0x7f9c68199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:44.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.886+0000 7f9c7060a700 1 --2- 172.21.15.5:0/1033210836 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c681034c0 0x7f9c6819f9c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:44.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.886+0000 7f9c7060a700 1 --2- 172.21.15.5:0/1033210836 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c68107df0 0x7f9c6819ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:44.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.886+0000 7f9c7060a700 1 -- 172.21.15.5:0/1033210836 >> 172.21.15.5:0/1033210836 conn(0x7f9c680fedc0 msgr2=0x7f9c681096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:44.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.886+0000 7f9c7060a700 1 -- 172.21.15.5:0/1033210836 shutdown_connections 2024-05-09T01:00:44.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:44.886+0000 7f9c7060a700 1 -- 172.21.15.5:0/1033210836 wait complete. 2024-05-09T01:00:45.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:00:45 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:00:45.437 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:00:45 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:00:45.533 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:00:45 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:00:46.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:00:46 smithi131 ceph-mon[34547]: pgmap v803: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 27 MiB/s wr, 110 op/s 2024-05-09T01:00:46.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:00:46 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:00:46.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:00:46 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:00:46.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:00:46 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T01:00:46.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:00:46 smithi157 ceph-mon[34294]: pgmap v803: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 27 MiB/s wr, 110 op/s 2024-05-09T01:00:46.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:00:46 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:00:46.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:00:46 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:00:46.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:00:46 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T01:00:46.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:00:46 smithi005 ceph-mon[30365]: pgmap v803: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 27 MiB/s wr, 110 op/s 2024-05-09T01:00:46.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:00:46 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:00:46.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:00:46 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:00:46.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:00:46 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T01:00:46.562 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T01:00:46.562 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T01:00:46.823 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:00:48.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:00:48 smithi131 ceph-mon[34547]: pgmap v804: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 27 MiB/s wr, 110 op/s 2024-05-09T01:00:48.385 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:00:48 smithi005 ceph-mon[30365]: pgmap v804: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 27 MiB/s wr, 110 op/s 2024-05-09T01:00:48.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:00:48 smithi157 ceph-mon[34294]: pgmap v804: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 27 MiB/s wr, 110 op/s 2024-05-09T01:00:48.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.850+0000 7faf1ffcc700 1 -- 172.21.15.5:0/3053660309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf18107df0 msgr2=0x7faf18108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:48.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.850+0000 7faf1ffcc700 1 --2- 172.21.15.5:0/3053660309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf18107df0 0x7faf18108200 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7faf080099c0 tx=0x7faf08009cd0 comp rx=0 tx=0).stop 2024-05-09T01:00:48.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.850+0000 7faf1ffcc700 1 -- 172.21.15.5:0/3053660309 shutdown_connections 2024-05-09T01:00:48.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.850+0000 7faf1ffcc700 1 --2- 172.21.15.5:0/3053660309 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf18108740 0x7faf181100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:48.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.850+0000 7faf1ffcc700 1 --2- 172.21.15.5:0/3053660309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf181034c0 0x7faf181038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:48.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.850+0000 7faf1ffcc700 1 --2- 172.21.15.5:0/3053660309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf18107df0 0x7faf18108200 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:48.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.850+0000 7faf1ffcc700 1 -- 172.21.15.5:0/3053660309 >> 172.21.15.5:0/3053660309 conn(0x7faf180fedc0 msgr2=0x7faf181011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:48.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.850+0000 7faf1ffcc700 1 -- 172.21.15.5:0/3053660309 shutdown_connections 2024-05-09T01:00:48.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.850+0000 7faf1ffcc700 1 -- 172.21.15.5:0/3053660309 wait complete. 2024-05-09T01:00:48.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.851+0000 7faf1ffcc700 1 Processor -- start 2024-05-09T01:00:48.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.851+0000 7faf1ffcc700 1 -- start start 2024-05-09T01:00:48.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.851+0000 7faf1ffcc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf181034c0 0x7faf1819bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:48.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.852+0000 7faf1ffcc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf18107df0 0x7faf1819c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:48.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.852+0000 7faf1ffcc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf18108740 0x7faf1819c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:48.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.852+0000 7faf1ffcc700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7faf18113490 con 0x7faf18107df0 2024-05-09T01:00:48.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.852+0000 7faf1ffcc700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7faf18113310 con 0x7faf181034c0 2024-05-09T01:00:48.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.852+0000 7faf1ffcc700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7faf18113610 con 0x7faf18108740 2024-05-09T01:00:48.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.852+0000 7faf1d567700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf18107df0 0x7faf1819c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:48.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.852+0000 7faf1d567700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf18107df0 0x7faf1819c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53010/0 (socket says 172.21.15.5:53010) 2024-05-09T01:00:48.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.852+0000 7faf1d567700 1 -- 172.21.15.5:0/2839080523 learned_addr learned my addr 172.21.15.5:0/2839080523 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:00:48.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.852+0000 7faf1e569700 1 --2- 172.21.15.5:0/2839080523 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf18108740 0x7faf1819c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:48.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.852+0000 7faf1dd68700 1 --2- 172.21.15.5:0/2839080523 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf181034c0 0x7faf1819bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:48.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.852+0000 7faf1d567700 1 -- 172.21.15.5:0/2839080523 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf18108740 msgr2=0x7faf1819c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:48.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.852+0000 7faf1d567700 1 --2- 172.21.15.5:0/2839080523 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf18108740 0x7faf1819c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:48.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.852+0000 7faf1d567700 1 -- 172.21.15.5:0/2839080523 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf181034c0 msgr2=0x7faf1819bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:48.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.852+0000 7faf1d567700 1 --2- 172.21.15.5:0/2839080523 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf181034c0 0x7faf1819bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:48.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.852+0000 7faf1d567700 1 -- 172.21.15.5:0/2839080523 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf181a7120 con 0x7faf18107df0 2024-05-09T01:00:48.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.853+0000 7faf1d567700 1 --2- 172.21.15.5:0/2839080523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf18107df0 0x7faf1819c310 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7faf08009f50 tx=0x7faf080036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:48.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.853+0000 7faf0effd700 1 -- 172.21.15.5:0/2839080523 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf08026070 con 0x7faf18107df0 2024-05-09T01:00:48.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.853+0000 7faf0effd700 1 -- 172.21.15.5:0/2839080523 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf08004020 con 0x7faf18107df0 2024-05-09T01:00:48.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.853+0000 7faf0effd700 1 -- 172.21.15.5:0/2839080523 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0800fe30 con 0x7faf18107df0 2024-05-09T01:00:48.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.853+0000 7faf1ffcc700 1 -- 172.21.15.5:0/2839080523 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf181a73b0 con 0x7faf18107df0 2024-05-09T01:00:48.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.853+0000 7faf1ffcc700 1 -- 172.21.15.5:0/2839080523 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf181a77b0 con 0x7faf18107df0 2024-05-09T01:00:48.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.856+0000 7faf0effd700 1 -- 172.21.15.5:0/2839080523 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7faf08029070 con 0x7faf18107df0 2024-05-09T01:00:48.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.857+0000 7faf0effd700 1 --2- 172.21.15.5:0/2839080523 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7faf040b2900 0x7faf040b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:48.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.858+0000 7faf1dd68700 1 --2- 172.21.15.5:0/2839080523 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7faf040b2900 0x7faf040b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:48.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.858+0000 7faf0effd700 1 -- 172.21.15.5:0/2839080523 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7faf080e2e30 con 0x7faf18107df0 2024-05-09T01:00:48.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.858+0000 7faf1dd68700 1 --2- 172.21.15.5:0/2839080523 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7faf040b2900 0x7faf040b4e10 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7faf14006fd0 tx=0x7faf14008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:48.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.858+0000 7faf2e9e2b80 1 Processor -- start 2024-05-09T01:00:48.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.858+0000 7faf2e9e2b80 1 -- start start 2024-05-09T01:00:48.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.859+0000 7faf2e9e2b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56330c1eb1e0 0x56330c282dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:48.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.859+0000 7faf2e9e2b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56330c101520 0x56330c283370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:48.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.859+0000 7faf2e9e2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56330c287480 0x56330c287890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:48.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.859+0000 7faf1d567700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56330c101520 0x56330c283370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:48.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.859+0000 7faf1d567700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56330c101520 0x56330c283370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53022/0 (socket says 172.21.15.5:53022) 2024-05-09T01:00:48.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.859+0000 7faf1d567700 1 -- 172.21.15.5:0/1186194731 learned_addr learned my addr 172.21.15.5:0/1186194731 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:00:48.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.859+0000 7faf2e9e2b80 1 -- 172.21.15.5:0/1186194731 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56330c1a1070 con 0x56330c101520 2024-05-09T01:00:48.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.859+0000 7faf2e9e2b80 1 -- 172.21.15.5:0/1186194731 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x56330c13f870 con 0x56330c287480 2024-05-09T01:00:48.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.859+0000 7faf2e9e2b80 1 -- 172.21.15.5:0/1186194731 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56330c19d030 con 0x56330c1eb1e0 2024-05-09T01:00:48.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.859+0000 7faf1dd68700 1 --2- 172.21.15.5:0/1186194731 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56330c1eb1e0 0x56330c282dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:48.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.859+0000 7faf1e569700 1 --2- 172.21.15.5:0/1186194731 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56330c287480 0x56330c287890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:48.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.859+0000 7faf1d567700 1 -- 172.21.15.5:0/1186194731 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56330c1eb1e0 msgr2=0x56330c282dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:48.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.859+0000 7faf1d567700 1 --2- 172.21.15.5:0/1186194731 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56330c1eb1e0 0x56330c282dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:48.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.859+0000 7faf1d567700 1 -- 172.21.15.5:0/1186194731 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56330c287480 msgr2=0x56330c287890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:48.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.859+0000 7faf1d567700 1 --2- 172.21.15.5:0/1186194731 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56330c287480 0x56330c287890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:48.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.859+0000 7faf1d567700 1 -- 172.21.15.5:0/1186194731 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56330c1373f0 con 0x56330c101520 2024-05-09T01:00:48.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.860+0000 7faf1d567700 1 --2- 172.21.15.5:0/1186194731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56330c101520 0x56330c283370 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7faf080aa440 tx=0x7faf08009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70599cdcd52e2a5f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:48.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.860+0000 7faf03fff700 1 -- 172.21.15.5:0/1186194731 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf080920b0 con 0x56330c101520 2024-05-09T01:00:48.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.860+0000 7faf03fff700 1 -- 172.21.15.5:0/1186194731 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf080a9fc0 con 0x56330c101520 2024-05-09T01:00:48.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.860+0000 7faf03fff700 1 -- 172.21.15.5:0/1186194731 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf080aa9a0 con 0x56330c101520 2024-05-09T01:00:48.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.861+0000 7faf2e9e2b80 1 -- 172.21.15.5:0/1186194731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56330c101520 msgr2=0x56330c283370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:48.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.861+0000 7faf2e9e2b80 1 --2- 172.21.15.5:0/1186194731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56330c101520 0x56330c283370 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7faf080aa440 tx=0x7faf08009cd0 comp rx=0 tx=0).stop 2024-05-09T01:00:48.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.861+0000 7faf2e9e2b80 1 -- 172.21.15.5:0/1186194731 shutdown_connections 2024-05-09T01:00:48.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.861+0000 7faf2e9e2b80 1 --2- 172.21.15.5:0/1186194731 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56330c287480 0x56330c287890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:48.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.861+0000 7faf2e9e2b80 1 --2- 172.21.15.5:0/1186194731 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56330c1eb1e0 0x56330c282dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:48.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.861+0000 7faf2e9e2b80 1 --2- 172.21.15.5:0/1186194731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56330c101520 0x56330c283370 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:48.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.861+0000 7faf2e9e2b80 1 -- 172.21.15.5:0/1186194731 >> 172.21.15.5:0/1186194731 conn(0x56330c161720 msgr2=0x56330c279480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:48.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.861+0000 7faf2e9e2b80 1 -- 172.21.15.5:0/1186194731 shutdown_connections 2024-05-09T01:00:48.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.862+0000 7faf2e9e2b80 1 -- 172.21.15.5:0/1186194731 wait complete. 2024-05-09T01:00:48.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.862+0000 7faf2e9e2b80 1 -- start start 2024-05-09T01:00:48.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.862+0000 7faf2e9e2b80 1 Processor -- start 2024-05-09T01:00:48.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.863+0000 7faf2e9e2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56330c101520 0x56330c292140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:48.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.863+0000 7faf2e9e2b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56330c1eb1e0 0x56330c292680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:48.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.863+0000 7faf2e9e2b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56330c287480 0x56330c28c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:48.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.863+0000 7faf1d567700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56330c1eb1e0 0x56330c292680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:48.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.863+0000 7faf1d567700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56330c1eb1e0 0x56330c292680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53024/0 (socket says 172.21.15.5:53024) 2024-05-09T01:00:48.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.863+0000 7faf1d567700 1 -- 172.21.15.5:0/3421527534 learned_addr learned my addr 172.21.15.5:0/3421527534 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:00:48.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.863+0000 7faf2e9e2b80 1 -- 172.21.15.5:0/3421527534 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56330c01cc20 con 0x56330c1eb1e0 2024-05-09T01:00:48.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.863+0000 7faf2e9e2b80 1 -- 172.21.15.5:0/3421527534 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x56330c15b1d0 con 0x56330c101520 2024-05-09T01:00:48.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.863+0000 7faf2e9e2b80 1 -- 172.21.15.5:0/3421527534 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56330c287ff0 con 0x56330c287480 2024-05-09T01:00:48.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.863+0000 7faf1dd68700 1 --2- 172.21.15.5:0/3421527534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56330c101520 0x56330c292140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:48.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.863+0000 7faf1e569700 1 --2- 172.21.15.5:0/3421527534 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56330c287480 0x56330c28c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:48.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.863+0000 7faf2e9e2b80 10 client.0 _ll_register_callbacks cb 0x56330c012740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:00:48.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.864+0000 7faf1dd68700 1 -- 172.21.15.5:0/3421527534 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56330c287480 msgr2=0x56330c28c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:48.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.864+0000 7faf1dd68700 1 --2- 172.21.15.5:0/3421527534 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56330c287480 0x56330c28c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:48.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.864+0000 7faf1dd68700 1 -- 172.21.15.5:0/3421527534 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56330c1eb1e0 msgr2=0x56330c292680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:48.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.864+0000 7faf1dd68700 1 --2- 172.21.15.5:0/3421527534 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56330c1eb1e0 0x56330c292680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:48.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.864+0000 7faf1dd68700 1 -- 172.21.15.5:0/3421527534 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56330c01c510 con 0x56330c101520 2024-05-09T01:00:48.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.864+0000 7faf1dd68700 1 --2- 172.21.15.5:0/3421527534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56330c101520 0x56330c292140 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7faf1400dde0 tx=0x7faf14004860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59e18e20e6ef5b55 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:48.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.864+0000 7faf017fa700 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-09T01:00:48.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.864+0000 7faf017fa700 1 -- 172.21.15.5:0/3421527534 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf14024020 con 0x56330c101520 2024-05-09T01:00:48.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.864+0000 7faf2e9e2b80 1 -- 172.21.15.5:0/3421527534 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf080a9fc0 con 0x56330c101520 2024-05-09T01:00:48.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.864+0000 7faf017fa700 1 -- 172.21.15.5:0/3421527534 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7faf1400f040 con 0x56330c101520 2024-05-09T01:00:48.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.865+0000 7faf017fa700 1 -- 172.21.15.5:0/3421527534 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7faf1402f050 con 0x56330c101520 2024-05-09T01:00:48.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.865+0000 7faf2e9e2b80 10 client.26531 fetch_fsmap learned FSMap version 32 2024-05-09T01:00:48.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.865+0000 7faf2e9e2b80 1 -- 172.21.15.5:0/3421527534 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56330c299ba0 con 0x56330c101520 2024-05-09T01:00:48.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.865+0000 7faf017fa700 1 -- 172.21.15.5:0/3421527534 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf080a9fc0 con 0x56330c101520 2024-05-09T01:00:48.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.865+0000 7faf017fa700 1 -- 172.21.15.5:0/3421527534 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7faf14013070 con 0x56330c101520 2024-05-09T01:00:48.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.865+0000 7faf2e9e2b80 10 client.26531 fetch_fsmap finished waiting for FSMap version 32 2024-05-09T01:00:48.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.865+0000 7faf2e9e2b80 10 client.26531 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T01:00:48.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.865+0000 7faf2e9e2b80 20 client.26531 populate_metadata read hostname 'smithi005' 2024-05-09T01:00:48.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.865+0000 7faf2e9e2b80 1 --2- 172.21.15.5:0/3421527534 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56330c29c1d0 0x56330c29e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:48.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.865+0000 7faf2e9e2b80 4 client.26531 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:00:48.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.865+0000 7faf2e9e2b80 1 -- 172.21.15.5:0/3421527534 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56330c1aea50 con 0x56330c29c1d0 2024-05-09T01:00:48.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.865+0000 7faf1d567700 1 --2- 172.21.15.5:0/3421527534 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56330c29c1d0 0x56330c29e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:48.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.866+0000 7faf017fa700 10 client.26531 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T01:00:48.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.866+0000 7faf1d567700 1 --2- 172.21.15.5:0/3421527534 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56330c29c1d0 0x56330c29e630 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54a03fbcf6f18109 server_cookie=adca615c4c5539b2 in_seq=0 out_seq=0 2024-05-09T01:00:48.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.868+0000 7faf017fa700 1 -- 172.21.15.5:0/3421527534 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56330c1aea50 con 0x56330c29c1d0 2024-05-09T01:00:48.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.868+0000 7faf017fa700 10 client.26531 handle_command_reply: tid=0 2024-05-09T01:00:48.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.868+0000 7faf2e9e2b80 1 client.26531 shutdown 2024-05-09T01:00:48.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.868+0000 7faf2e9e2b80 1 -- 172.21.15.5:0/3421527534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56330c101520 msgr2=0x56330c292140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:48.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.868+0000 7faf2e9e2b80 1 --2- 172.21.15.5:0/3421527534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56330c101520 0x56330c292140 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7faf1400dde0 tx=0x7faf14004860 comp rx=0 tx=0).stop 2024-05-09T01:00:48.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.868+0000 7faf2e9e2b80 1 -- 172.21.15.5:0/3421527534 shutdown_connections 2024-05-09T01:00:48.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.868+0000 7faf2e9e2b80 1 --2- 172.21.15.5:0/3421527534 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56330c29c1d0 0x56330c29e630 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:48.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.868+0000 7faf2e9e2b80 1 --2- 172.21.15.5:0/3421527534 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56330c287480 0x56330c28c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:48.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.868+0000 7faf017fa700 0 client.26531 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T01:00:48.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.868+0000 7faf2e9e2b80 1 --2- 172.21.15.5:0/3421527534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56330c101520 0x56330c292140 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:48.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.868+0000 7faf2e9e2b80 1 --2- 172.21.15.5:0/3421527534 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56330c1eb1e0 0x56330c292680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:48.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.869+0000 7faf2e9e2b80 1 -- 172.21.15.5:0/3421527534 >> 172.21.15.5:0/3421527534 conn(0x56330c161720 msgr2=0x56330c279240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:48.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.869+0000 7faf2e9e2b80 1 -- 172.21.15.5:0/3421527534 shutdown_connections 2024-05-09T01:00:48.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.869+0000 7faf2e9e2b80 1 -- 172.21.15.5:0/3421527534 wait complete. 2024-05-09T01:00:48.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.869+0000 7faf2e9e2b80 20 client.26531 trim_cache size 0 max 16384 2024-05-09T01:00:48.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.890+0000 7faf2e9e2b80 1 Processor -- start 2024-05-09T01:00:48.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.890+0000 7faf2e9e2b80 1 -- start start 2024-05-09T01:00:48.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.890+0000 7faf2e9e2b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56330c101520 0x56330c28bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:48.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.890+0000 7faf2e9e2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56330c1eb1e0 0x56330c28c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:48.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.890+0000 7faf2e9e2b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56330c287480 0x56330c286380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:48.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.890+0000 7faf2e9e2b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7faf181acb70 con 0x56330c287480 2024-05-09T01:00:48.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.890+0000 7faf2e9e2b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7faf181ae6b0 con 0x56330c1eb1e0 2024-05-09T01:00:48.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.890+0000 7faf2e9e2b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7faf181ad240 con 0x56330c101520 2024-05-09T01:00:48.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.891+0000 7faf1dd68700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56330c101520 0x56330c28bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:48.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.891+0000 7faf1e569700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56330c287480 0x56330c286380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:48.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.891+0000 7faf1dd68700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56330c101520 0x56330c28bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:54720/0 (socket says 172.21.15.5:54720) 2024-05-09T01:00:48.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.891+0000 7faf1e569700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56330c287480 0x56330c286380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53026/0 (socket says 172.21.15.5:53026) 2024-05-09T01:00:48.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.891+0000 7faf1dd68700 1 -- 172.21.15.5:0/3564787729 learned_addr learned my addr 172.21.15.5:0/3564787729 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:00:48.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.891+0000 7faf1d567700 1 --2- 172.21.15.5:0/3564787729 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56330c1eb1e0 0x56330c28c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:48.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.891+0000 7faf1dd68700 1 -- 172.21.15.5:0/3564787729 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56330c1eb1e0 msgr2=0x56330c28c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:48.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.891+0000 7faf1dd68700 1 --2- 172.21.15.5:0/3564787729 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56330c1eb1e0 0x56330c28c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:48.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.891+0000 7faf1dd68700 1 -- 172.21.15.5:0/3564787729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56330c287480 msgr2=0x56330c286380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:48.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.891+0000 7faf1dd68700 1 --2- 172.21.15.5:0/3564787729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56330c287480 0x56330c286380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:48.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.891+0000 7faf1dd68700 1 -- 172.21.15.5:0/3564787729 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56330c28c870 con 0x56330c101520 2024-05-09T01:00:48.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.891+0000 7faf1dd68700 1 --2- 172.21.15.5:0/3564787729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56330c101520 0x56330c28bdf0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7faf14000c00 tx=0x7faf1400af60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47c100a0811da018 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:48.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.891+0000 7faf037fe700 1 -- 172.21.15.5:0/3564787729 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf14038070 con 0x56330c101520 2024-05-09T01:00:48.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.891+0000 7faf037fe700 1 -- 172.21.15.5:0/3564787729 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf140203f0 con 0x56330c101520 2024-05-09T01:00:48.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.891+0000 7faf037fe700 1 -- 172.21.15.5:0/3564787729 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf14024040 con 0x56330c101520 2024-05-09T01:00:48.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.892+0000 7faf2e9e2b80 1 -- 172.21.15.5:0/3564787729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56330c101520 msgr2=0x56330c28bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:48.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.892+0000 7faf2e9e2b80 1 --2- 172.21.15.5:0/3564787729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56330c101520 0x56330c28bdf0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7faf14000c00 tx=0x7faf1400af60 comp rx=0 tx=0).stop 2024-05-09T01:00:48.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.892+0000 7faf2e9e2b80 1 -- 172.21.15.5:0/3564787729 shutdown_connections 2024-05-09T01:00:48.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.892+0000 7faf2e9e2b80 1 --2- 172.21.15.5:0/3564787729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56330c287480 0x56330c286380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:48.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.892+0000 7faf2e9e2b80 1 --2- 172.21.15.5:0/3564787729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56330c101520 0x56330c28bdf0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:48.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.892+0000 7faf2e9e2b80 1 --2- 172.21.15.5:0/3564787729 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56330c1eb1e0 0x56330c28c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:48.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.892+0000 7faf2e9e2b80 1 -- 172.21.15.5:0/3564787729 >> 172.21.15.5:0/3564787729 conn(0x56330c161720 msgr2=0x56330c28ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:48.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.892+0000 7faf2e9e2b80 1 -- 172.21.15.5:0/3564787729 shutdown_connections 2024-05-09T01:00:48.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.892+0000 7faf2e9e2b80 1 -- 172.21.15.5:0/3564787729 wait complete. 2024-05-09T01:00:48.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.893+0000 7faf2e9e2b80 1 -- start start 2024-05-09T01:00:48.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.893+0000 7faf2e9e2b80 1 Processor -- start 2024-05-09T01:00:48.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.893+0000 7faf2e9e2b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56330c101520 0x56330c298430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:48.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.893+0000 7faf2e9e2b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56330c1eb1e0 0x56330c2989d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:48.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.893+0000 7faf2e9e2b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56330c287480 0x56330c296920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:48.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.893+0000 7faf2e9e2b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7faf181adfe0 con 0x56330c101520 2024-05-09T01:00:48.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.893+0000 7faf2e9e2b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7faf181ad910 con 0x56330c287480 2024-05-09T01:00:48.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.893+0000 7faf2e9e2b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7faf181aab10 con 0x56330c1eb1e0 2024-05-09T01:00:48.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.893+0000 7faf2e9e2b80 10 client.0 _ll_register_callbacks cb 0x56330c012740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:00:48.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.893+0000 7faf1d567700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56330c1eb1e0 0x56330c2989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:48.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.894+0000 7faf1dd68700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56330c101520 0x56330c298430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:48.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.894+0000 7faf1d567700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56330c1eb1e0 0x56330c2989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:54724/0 (socket says 172.21.15.5:54724) 2024-05-09T01:00:48.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.894+0000 7faf1d567700 1 -- 172.21.15.5:0/464473340 learned_addr learned my addr 172.21.15.5:0/464473340 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:00:48.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.894+0000 7faf1e569700 1 --2- 172.21.15.5:0/464473340 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56330c287480 0x56330c296920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:48.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.894+0000 7faf1d567700 1 -- 172.21.15.5:0/464473340 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56330c287480 msgr2=0x56330c296920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:48.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.894+0000 7faf1d567700 1 --2- 172.21.15.5:0/464473340 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56330c287480 0x56330c296920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:48.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.894+0000 7faf1d567700 1 -- 172.21.15.5:0/464473340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56330c101520 msgr2=0x56330c298430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:48.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.894+0000 7faf1d567700 1 --2- 172.21.15.5:0/464473340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56330c101520 0x56330c298430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:48.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.894+0000 7faf1d567700 1 -- 172.21.15.5:0/464473340 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56330c2970c0 con 0x56330c1eb1e0 2024-05-09T01:00:48.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.894+0000 7faf1dd68700 1 --2- 172.21.15.5:0/464473340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56330c101520 0x56330c298430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:00:48.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.894+0000 7faf1d567700 1 --2- 172.21.15.5:0/464473340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56330c1eb1e0 0x56330c2989d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7faf080aea20 tx=0x7faf080a4e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8edd8a38a9c73beb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:48.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.894+0000 7faf017fa700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-09T01:00:48.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.894+0000 7faf017fa700 1 -- 172.21.15.5:0/464473340 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf080a9fc0 con 0x56330c1eb1e0 2024-05-09T01:00:48.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.894+0000 7faf017fa700 1 -- 172.21.15.5:0/464473340 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7faf080aa920 con 0x56330c1eb1e0 2024-05-09T01:00:48.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.894+0000 7faf2e9e2b80 1 -- 172.21.15.5:0/464473340 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf140203f0 con 0x56330c1eb1e0 2024-05-09T01:00:48.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.894+0000 7faf017fa700 1 -- 172.21.15.5:0/464473340 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7faf080aabb0 con 0x56330c1eb1e0 2024-05-09T01:00:48.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.894+0000 7faf2e9e2b80 10 client.26527 fetch_fsmap learned FSMap version 32 2024-05-09T01:00:48.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.894+0000 7faf2e9e2b80 1 -- 172.21.15.5:0/464473340 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56330c3075a0 con 0x56330c1eb1e0 2024-05-09T01:00:48.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.895+0000 7faf017fa700 1 -- 172.21.15.5:0/464473340 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf140203f0 con 0x56330c1eb1e0 2024-05-09T01:00:48.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.895+0000 7faf017fa700 1 -- 172.21.15.5:0/464473340 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7faf080b2df0 con 0x56330c1eb1e0 2024-05-09T01:00:48.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.895+0000 7faf2e9e2b80 10 client.26527 fetch_fsmap finished waiting for FSMap version 32 2024-05-09T01:00:48.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.895+0000 7faf2e9e2b80 10 client.26527 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T01:00:48.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.895+0000 7faf2e9e2b80 20 client.26527 populate_metadata read hostname 'smithi005' 2024-05-09T01:00:48.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.895+0000 7faf2e9e2b80 1 --2- 172.21.15.5:0/464473340 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56330c307b70 0x56330c309f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:48.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.895+0000 7faf2e9e2b80 4 client.26527 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T01:00:48.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.895+0000 7faf2e9e2b80 1 -- 172.21.15.5:0/464473340 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56330c1a8af0 con 0x56330c307b70 2024-05-09T01:00:48.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.895+0000 7faf1dd68700 1 --2- 172.21.15.5:0/464473340 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56330c307b70 0x56330c309f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:48.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.895+0000 7faf017fa700 10 client.26527 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T01:00:48.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:48.896+0000 7faf1dd68700 1 --2- 172.21.15.5:0/464473340 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56330c307b70 0x56330c309f90 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42d7efc6521eec65 server_cookie=e70f2dd78eddcfe in_seq=0 out_seq=0 2024-05-09T01:00:49.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:49.194+0000 7faf017fa700 1 -- 172.21.15.5:0/464473340 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56330c1a8af0 con 0x56330c307b70 2024-05-09T01:00:49.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:49.194+0000 7faf017fa700 10 client.26527 handle_command_reply: tid=0 2024-05-09T01:00:49.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:49.194+0000 7faf2e9e2b80 1 client.26527 shutdown 2024-05-09T01:00:49.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:49.194+0000 7faf2e9e2b80 1 -- 172.21.15.5:0/464473340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56330c1eb1e0 msgr2=0x56330c2989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:49.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:49.194+0000 7faf2e9e2b80 1 --2- 172.21.15.5:0/464473340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56330c1eb1e0 0x56330c2989d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7faf080aea20 tx=0x7faf080a4e00 comp rx=0 tx=0).stop 2024-05-09T01:00:49.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:49.194+0000 7faf2e9e2b80 1 -- 172.21.15.5:0/464473340 shutdown_connections 2024-05-09T01:00:49.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:49.194+0000 7faf2e9e2b80 1 --2- 172.21.15.5:0/464473340 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x56330c307b70 0x56330c309f90 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:49.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:49.195+0000 7faf2e9e2b80 1 --2- 172.21.15.5:0/464473340 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56330c287480 0x56330c296920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:49.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:49.195+0000 7faf017fa700 0 client.26527 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T01:00:49.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:49.195+0000 7faf2e9e2b80 1 --2- 172.21.15.5:0/464473340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56330c101520 0x56330c298430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:49.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:49.195+0000 7faf2e9e2b80 1 --2- 172.21.15.5:0/464473340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56330c1eb1e0 0x56330c2989d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:49.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:49.195+0000 7faf2e9e2b80 1 -- 172.21.15.5:0/464473340 >> 172.21.15.5:0/464473340 conn(0x56330c161720 msgr2=0x56330c286380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:49.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:49.195+0000 7faf2e9e2b80 1 -- 172.21.15.5:0/464473340 shutdown_connections 2024-05-09T01:00:49.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:49.195+0000 7faf2e9e2b80 1 -- 172.21.15.5:0/464473340 wait complete. 2024-05-09T01:00:49.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:49.195+0000 7faf2e9e2b80 20 client.26527 trim_cache size 0 max 16384 2024-05-09T01:00:49.197 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-09T01:00:49.197 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-05-09T01:00:49.197 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "1b609027-503b-485e-bcb9-ba2628dcbe62", 2024-05-09T01:00:49.197 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-05-09T01:00:49.197 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-09T01:00:49.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:49.195+0000 7faf1ffcc700 1 -- 172.21.15.5:0/2839080523 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7faf040b2900 msgr2=0x7faf040b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:49.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:49.195+0000 7faf1ffcc700 1 --2- 172.21.15.5:0/2839080523 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7faf040b2900 0x7faf040b4e10 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7faf14006fd0 tx=0x7faf14008040 comp rx=0 tx=0).stop 2024-05-09T01:00:49.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:49.195+0000 7faf1ffcc700 1 -- 172.21.15.5:0/2839080523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf18107df0 msgr2=0x7faf1819c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:49.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:49.195+0000 7faf1ffcc700 1 --2- 172.21.15.5:0/2839080523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf18107df0 0x7faf1819c310 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7faf08009f50 tx=0x7faf080036f0 comp rx=0 tx=0).stop 2024-05-09T01:00:49.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:49.196+0000 7faf1ffcc700 1 -- 172.21.15.5:0/2839080523 shutdown_connections 2024-05-09T01:00:49.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:49.196+0000 7faf1ffcc700 1 --2- 172.21.15.5:0/2839080523 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7faf040b2900 0x7faf040b4e10 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:49.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:49.196+0000 7faf1ffcc700 1 --2- 172.21.15.5:0/2839080523 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf18108740 0x7faf1819c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:49.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:49.196+0000 7faf1ffcc700 1 --2- 172.21.15.5:0/2839080523 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf181034c0 0x7faf1819bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:49.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:49.196+0000 7faf1ffcc700 1 --2- 172.21.15.5:0/2839080523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf18107df0 0x7faf1819c310 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:49.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:49.196+0000 7faf1ffcc700 1 -- 172.21.15.5:0/2839080523 >> 172.21.15.5:0/2839080523 conn(0x7faf180fedc0 msgr2=0x7faf181096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:49.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:49.196+0000 7faf1ffcc700 1 -- 172.21.15.5:0/2839080523 shutdown_connections 2024-05-09T01:00:49.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:49.196+0000 7faf1ffcc700 1 -- 172.21.15.5:0/2839080523 wait complete. 2024-05-09T01:00:49.889 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T01:00:50.143 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:00:50.266 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:00:50 smithi005 ceph-mon[30365]: pgmap v805: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 27 MiB/s wr, 114 op/s 2024-05-09T01:00:50.266 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:00:50 smithi005 ceph-mon[30365]: scrub queued for path: / 2024-05-09T01:00:50.267 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:00:50 smithi005 ceph-mon[30365]: scrub summary: idle+waiting paths [/] 2024-05-09T01:00:50.267 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:00:50 smithi005 ceph-mon[30365]: scrub summary: active paths [/] 2024-05-09T01:00:50.357 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:00:50 smithi131 ceph-mon[34547]: pgmap v805: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 27 MiB/s wr, 114 op/s 2024-05-09T01:00:50.358 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:00:50 smithi131 ceph-mon[34547]: scrub queued for path: / 2024-05-09T01:00:50.358 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:00:50 smithi131 ceph-mon[34547]: scrub summary: idle+waiting paths [/] 2024-05-09T01:00:50.359 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:00:50 smithi131 ceph-mon[34547]: scrub summary: active paths [/] 2024-05-09T01:00:50.438 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:00:50 smithi157 ceph-mon[34294]: pgmap v805: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 27 MiB/s wr, 114 op/s 2024-05-09T01:00:50.438 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:00:50 smithi157 ceph-mon[34294]: scrub queued for path: / 2024-05-09T01:00:50.438 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:00:50 smithi157 ceph-mon[34294]: scrub summary: idle+waiting paths [/] 2024-05-09T01:00:50.438 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:00:50 smithi157 ceph-mon[34294]: scrub summary: active paths [/] 2024-05-09T01:00:50.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:50.753+0000 7fcf0dffb640 1 -- 172.21.15.5:0/885460131 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55d1a06e4240 con 0x55d1a03681c0 2024-05-09T01:00:50.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:50.753+0000 7fcf0dffb640 10 client.17097 handle_command_reply: tid=0 2024-05-09T01:00:50.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:50.753+0000 7fcf33998740 1 client.17097 shutdown 2024-05-09T01:00:50.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:50.753+0000 7fcf33998740 1 -- 172.21.15.5:0/885460131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1a03d41b0 msgr2=0x55d1a07872e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:50.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:50.753+0000 7fcf33998740 1 --2- 172.21.15.5:0/885460131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1a03d41b0 0x55d1a07872e0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fcf2002fab0 tx=0x7fcf200021d0 comp rx=0 tx=0).stop 2024-05-09T01:00:50.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:50.753+0000 7fcf33998740 1 -- 172.21.15.5:0/885460131 shutdown_connections 2024-05-09T01:00:50.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:50.753+0000 7fcf33998740 1 --2- 172.21.15.5:0/885460131 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55d1a03681c0 0x55d1a07f9ad0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:50.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:50.753+0000 7fcf33998740 1 --2- 172.21.15.5:0/885460131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1a03d41b0 0x55d1a07872e0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:50.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:50.753+0000 7fcf33998740 1 --2- 172.21.15.5:0/885460131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1a03b34d0 0x55d1a07863b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:50.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:50.753+0000 7fcf33998740 1 --2- 172.21.15.5:0/885460131 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d1a03ab2f0 0x55d1a0785e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:50.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:50.753+0000 7fcf0dffb640 0 client.17097 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T01:00:50.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:50.753+0000 7fcf33998740 1 -- 172.21.15.5:0/885460131 >> 172.21.15.5:0/885460131 conn(0x55d1a03c8300 msgr2=0x55d1a077c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:50.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:50.754+0000 7fcf33998740 1 -- 172.21.15.5:0/885460131 shutdown_connections 2024-05-09T01:00:50.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:50.754+0000 7fcf33998740 1 -- 172.21.15.5:0/885460131 wait complete. 2024-05-09T01:00:50.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:50.754+0000 7fcf33998740 20 client.17097 trim_cache size 0 max 16384 2024-05-09T01:00:50.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-09T01:00:50.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-09T01:00:50.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-09T01:00:50.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:50.754+0000 7fcf2ca0c640 1 -- 172.21.15.5:0/3602952550 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fcf100b27e0 msgr2=0x7fcf100b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:50.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:50.754+0000 7fcf2ca0c640 1 --2- 172.21.15.5:0/3602952550 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fcf100b27e0 0x7fcf100b4cf0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fcf180a5b80 tx=0x7fcf180733f0 comp rx=0 tx=0).stop 2024-05-09T01:00:50.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:50.754+0000 7fcf2ca0c640 1 -- 172.21.15.5:0/3602952550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf281043c0 msgr2=0x7fcf2819a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:50.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:50.754+0000 7fcf2ca0c640 1 --2- 172.21.15.5:0/3602952550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf281043c0 0x7fcf2819a2e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fcf1800e9a0 tx=0x7fcf1800ee60 comp rx=0 tx=0).stop 2024-05-09T01:00:50.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:50.754+0000 7fcf2ca0c640 1 -- 172.21.15.5:0/3602952550 shutdown_connections 2024-05-09T01:00:50.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:50.754+0000 7fcf2ca0c640 1 --2- 172.21.15.5:0/3602952550 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fcf100b27e0 0x7fcf100b4cf0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:50.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:50.754+0000 7fcf2ca0c640 1 --2- 172.21.15.5:0/3602952550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf281098e0 0x7fcf281a17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:50.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:50.754+0000 7fcf2ca0c640 1 --2- 172.21.15.5:0/3602952550 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf28106db0 0x7fcf2819a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:50.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:50.754+0000 7fcf2ca0c640 1 --2- 172.21.15.5:0/3602952550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf281043c0 0x7fcf2819a2e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:50.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:50.754+0000 7fcf2ca0c640 1 -- 172.21.15.5:0/3602952550 >> 172.21.15.5:0/3602952550 conn(0x7fcf280fdd40 msgr2=0x7fcf281052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:50.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:50.754+0000 7fcf2ca0c640 1 -- 172.21.15.5:0/3602952550 shutdown_connections 2024-05-09T01:00:50.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:00:50.754+0000 7fcf2ca0c640 1 -- 172.21.15.5:0/3602952550 wait complete. 2024-05-09T01:00:52.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.070+0000 7f9eb1e16700 1 -- 172.21.15.5:0/3267796567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9eac108f00 msgr2=0x7f9eac109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:52.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.070+0000 7f9eb1e16700 1 --2- 172.21.15.5:0/3267796567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9eac108f00 0x7f9eac109310 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f9ea00099c0 tx=0x7f9ea0009cd0 comp rx=0 tx=0).stop 2024-05-09T01:00:52.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.070+0000 7f9eb1e16700 1 -- 172.21.15.5:0/3267796567 shutdown_connections 2024-05-09T01:00:52.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.070+0000 7f9eb1e16700 1 --2- 172.21.15.5:0/3267796567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9eac109850 0x7f9eac1111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:52.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.070+0000 7f9eb1e16700 1 --2- 172.21.15.5:0/3267796567 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9eac1045d0 0x7f9eac1049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:52.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.070+0000 7f9eb1e16700 1 --2- 172.21.15.5:0/3267796567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9eac108f00 0x7f9eac109310 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:52.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.070+0000 7f9eb1e16700 1 -- 172.21.15.5:0/3267796567 >> 172.21.15.5:0/3267796567 conn(0x7f9eac0ffea0 msgr2=0x7f9eac1022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:52.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.070+0000 7f9eb1e16700 1 -- 172.21.15.5:0/3267796567 shutdown_connections 2024-05-09T01:00:52.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.070+0000 7f9eb1e16700 1 -- 172.21.15.5:0/3267796567 wait complete. 2024-05-09T01:00:52.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.071+0000 7f9eb1e16700 1 Processor -- start 2024-05-09T01:00:52.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.071+0000 7f9eb1e16700 1 -- start start 2024-05-09T01:00:52.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.071+0000 7f9eb1e16700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9eac1045d0 0x7f9eac19ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:52.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.072+0000 7f9eb1e16700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9eac108f00 0x7f9eac19b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:52.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.072+0000 7f9eb1e16700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9eac109850 0x7f9eac19f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:52.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.072+0000 7f9eb1e16700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac1144b0 con 0x7f9eac109850 2024-05-09T01:00:52.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.072+0000 7f9eb1e16700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac114330 con 0x7f9eac108f00 2024-05-09T01:00:52.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.072+0000 7f9eb1e16700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac114630 con 0x7f9eac1045d0 2024-05-09T01:00:52.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.072+0000 7f9eab7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9eac1045d0 0x7f9eac19ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:52.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.072+0000 7f9eab7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9eac1045d0 0x7f9eac19ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:54728/0 (socket says 172.21.15.5:54728) 2024-05-09T01:00:52.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.072+0000 7f9eabfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9eac109850 0x7f9eac19f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:52.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.072+0000 7f9eab7fe700 1 -- 172.21.15.5:0/1259759506 learned_addr learned my addr 172.21.15.5:0/1259759506 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:00:52.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.072+0000 7f9eaaffd700 1 --2- 172.21.15.5:0/1259759506 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9eac108f00 0x7f9eac19b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:52.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.072+0000 7f9eab7fe700 1 -- 172.21.15.5:0/1259759506 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9eac108f00 msgr2=0x7f9eac19b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:52.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.072+0000 7f9eab7fe700 1 --2- 172.21.15.5:0/1259759506 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9eac108f00 0x7f9eac19b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:52.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.072+0000 7f9eab7fe700 1 -- 172.21.15.5:0/1259759506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9eac109850 msgr2=0x7f9eac19f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:52.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.072+0000 7f9eab7fe700 1 --2- 172.21.15.5:0/1259759506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9eac109850 0x7f9eac19f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:52.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.072+0000 7f9eab7fe700 1 -- 172.21.15.5:0/1259759506 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eac19fcd0 con 0x7f9eac1045d0 2024-05-09T01:00:52.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.072+0000 7f9eab7fe700 1 --2- 172.21.15.5:0/1259759506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9eac1045d0 0x7f9eac19ae00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c00d950 tx=0x7f9e9c00dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:52.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.073+0000 7f9ea8ff9700 1 -- 172.21.15.5:0/1259759506 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c016070 con 0x7f9eac1045d0 2024-05-09T01:00:52.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.073+0000 7f9ea8ff9700 1 -- 172.21.15.5:0/1259759506 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e9c00fd40 con 0x7f9eac1045d0 2024-05-09T01:00:52.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.073+0000 7f9eb1e16700 1 -- 172.21.15.5:0/1259759506 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9eac19ffc0 con 0x7f9eac1045d0 2024-05-09T01:00:52.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.073+0000 7f9ea8ff9700 1 -- 172.21.15.5:0/1259759506 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c00e3b0 con 0x7f9eac1045d0 2024-05-09T01:00:52.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.073+0000 7f9eb1e16700 1 -- 172.21.15.5:0/1259759506 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9eac076b00 con 0x7f9eac1045d0 2024-05-09T01:00:52.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.076+0000 7f9ea8ff9700 1 -- 172.21.15.5:0/1259759506 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f9e9c019070 con 0x7f9eac1045d0 2024-05-09T01:00:52.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.076+0000 7f9ea8ff9700 1 --2- 172.21.15.5:0/1259759506 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f9e940b2900 0x7f9e940b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:52.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.076+0000 7f9eaaffd700 1 --2- 172.21.15.5:0/1259759506 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f9e940b2900 0x7f9e940b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:52.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.077+0000 7f9ea8ff9700 1 -- 172.21.15.5:0/1259759506 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f9e9c0e1cb0 con 0x7f9eac1045d0 2024-05-09T01:00:52.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.077+0000 7f9ec082cb80 1 Processor -- start 2024-05-09T01:00:52.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.077+0000 7f9ec082cb80 1 -- start start 2024-05-09T01:00:52.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.077+0000 7f9eaaffd700 1 --2- 172.21.15.5:0/1259759506 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f9e940b2900 0x7f9e940b4e10 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0000c00 tx=0x7f9ea001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:52.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.077+0000 7f9ec082cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2c7065160 0x55b2c70fe4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:52.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.077+0000 7f9ec082cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2c6f7b4a0 0x55b2c70fea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:52.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.077+0000 7f9ec082cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2c70f86a0 0x55b2c70f8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:52.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.077+0000 7f9ec082cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b2c701aff0 con 0x55b2c7065160 2024-05-09T01:00:52.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.077+0000 7f9ec082cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55b2c6fb97f0 con 0x55b2c70f86a0 2024-05-09T01:00:52.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.077+0000 7f9ec082cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b2c7016fb0 con 0x55b2c6f7b4a0 2024-05-09T01:00:52.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.077+0000 7f9eab7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2c7065160 0x55b2c70fe4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:52.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.078+0000 7f9eab7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2c7065160 0x55b2c70fe4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53058/0 (socket says 172.21.15.5:53058) 2024-05-09T01:00:52.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.078+0000 7f9eaaffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2c6f7b4a0 0x55b2c70fea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:52.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.078+0000 7f9eab7fe700 1 -- 172.21.15.5:0/458579161 learned_addr learned my addr 172.21.15.5:0/458579161 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:00:52.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.078+0000 7f9eabfff700 1 --2- 172.21.15.5:0/458579161 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2c70f86a0 0x55b2c70f8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:52.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.078+0000 7f9eaaffd700 1 -- 172.21.15.5:0/458579161 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2c70f86a0 msgr2=0x55b2c70f8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:52.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.078+0000 7f9eaaffd700 1 --2- 172.21.15.5:0/458579161 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2c70f86a0 0x55b2c70f8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:52.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.078+0000 7f9eaaffd700 1 -- 172.21.15.5:0/458579161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2c7065160 msgr2=0x55b2c70fe4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:52.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.078+0000 7f9eaaffd700 1 --2- 172.21.15.5:0/458579161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2c7065160 0x55b2c70fe4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:52.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.078+0000 7f9eaaffd700 1 -- 172.21.15.5:0/458579161 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2c6fb1370 con 0x55b2c6f7b4a0 2024-05-09T01:00:52.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.078+0000 7f9eab7fe700 1 --2- 172.21.15.5:0/458579161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2c7065160 0x55b2c70fe4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:00:52.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.078+0000 7f9eaaffd700 1 --2- 172.21.15.5:0/458579161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2c6f7b4a0 0x55b2c70fea30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0009990 tx=0x7f9ea0009cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc72014c0fa1f83a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:52.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.079+0000 7f9e91ffb700 1 -- 172.21.15.5:0/458579161 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea0034070 con 0x55b2c6f7b4a0 2024-05-09T01:00:52.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.079+0000 7f9e91ffb700 1 -- 172.21.15.5:0/458579161 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ea0003c70 con 0x55b2c6f7b4a0 2024-05-09T01:00:52.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.079+0000 7f9e91ffb700 1 -- 172.21.15.5:0/458579161 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea002f470 con 0x55b2c6f7b4a0 2024-05-09T01:00:52.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.080+0000 7f9ec082cb80 1 -- 172.21.15.5:0/458579161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2c6f7b4a0 msgr2=0x55b2c70fea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:52.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.080+0000 7f9ec082cb80 1 --2- 172.21.15.5:0/458579161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2c6f7b4a0 0x55b2c70fea30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0009990 tx=0x7f9ea0009cd0 comp rx=0 tx=0).stop 2024-05-09T01:00:52.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.080+0000 7f9ec082cb80 1 -- 172.21.15.5:0/458579161 shutdown_connections 2024-05-09T01:00:52.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.080+0000 7f9ec082cb80 1 --2- 172.21.15.5:0/458579161 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2c70f86a0 0x55b2c70f8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:52.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.080+0000 7f9ec082cb80 1 --2- 172.21.15.5:0/458579161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2c7065160 0x55b2c70fe4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:52.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.080+0000 7f9ec082cb80 1 --2- 172.21.15.5:0/458579161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2c6f7b4a0 0x55b2c70fea30 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:52.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.080+0000 7f9ec082cb80 1 -- 172.21.15.5:0/458579161 >> 172.21.15.5:0/458579161 conn(0x55b2c6fdb6a0 msgr2=0x55b2c70f3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:52.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.080+0000 7f9ec082cb80 1 -- 172.21.15.5:0/458579161 shutdown_connections 2024-05-09T01:00:52.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.080+0000 7f9ec082cb80 1 -- 172.21.15.5:0/458579161 wait complete. 2024-05-09T01:00:52.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.081+0000 7f9ec082cb80 1 -- start start 2024-05-09T01:00:52.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.081+0000 7f9ec082cb80 1 Processor -- start 2024-05-09T01:00:52.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.081+0000 7f9ec082cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2c6f7b4a0 0x55b2c7102260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:52.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.081+0000 7f9ec082cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2c7065160 0x55b2c71027a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:52.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.081+0000 7f9ec082cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2c70f86a0 0x55b2c7102ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:52.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.081+0000 7f9ec082cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b2c6e96ba0 con 0x55b2c7065160 2024-05-09T01:00:52.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.081+0000 7f9ec082cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55b2c6fd5150 con 0x55b2c6f7b4a0 2024-05-09T01:00:52.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.081+0000 7f9ec082cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55b2c71051d0 con 0x55b2c70f86a0 2024-05-09T01:00:52.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.082+0000 7f9ec082cb80 10 client.0 _ll_register_callbacks cb 0x55b2c6e8c6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:00:52.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.082+0000 7f9eaaffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2c7065160 0x55b2c71027a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:52.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.082+0000 7f9eabfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2c70f86a0 0x55b2c7102ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:52.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.082+0000 7f9eaaffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2c7065160 0x55b2c71027a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53060/0 (socket says 172.21.15.5:53060) 2024-05-09T01:00:52.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.082+0000 7f9eabfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2c70f86a0 0x55b2c7102ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:54748/0 (socket says 172.21.15.5:54748) 2024-05-09T01:00:52.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.082+0000 7f9eab7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2c6f7b4a0 0x55b2c7102260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:52.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.082+0000 7f9eaaffd700 1 -- 172.21.15.5:0/165193435 learned_addr learned my addr 172.21.15.5:0/165193435 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:00:52.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.082+0000 7f9eaaffd700 1 -- 172.21.15.5:0/165193435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2c70f86a0 msgr2=0x55b2c7102ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:52.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.082+0000 7f9eaaffd700 1 --2- 172.21.15.5:0/165193435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2c70f86a0 0x55b2c7102ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:52.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.082+0000 7f9eaaffd700 1 -- 172.21.15.5:0/165193435 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2c6f7b4a0 msgr2=0x55b2c7102260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:52.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.082+0000 7f9eaaffd700 1 --2- 172.21.15.5:0/165193435 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2c6f7b4a0 0x55b2c7102260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:52.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.082+0000 7f9eaaffd700 1 -- 172.21.15.5:0/165193435 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2c6e96490 con 0x55b2c7065160 2024-05-09T01:00:52.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.082+0000 7f9eabfff700 1 --2- 172.21.15.5:0/165193435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2c70f86a0 0x55b2c7102ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:00:52.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.082+0000 7f9eaaffd700 1 --2- 172.21.15.5:0/165193435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2c7065160 0x55b2c71027a0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f9ea00098e0 tx=0x7f9ea000b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=705e2d15ac4cc438 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:52.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.082+0000 7f9e8affd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T01:00:52.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.082+0000 7f9e8affd700 1 -- 172.21.15.5:0/165193435 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea003d060 con 0x55b2c7065160 2024-05-09T01:00:52.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.082+0000 7f9e8affd700 1 -- 172.21.15.5:0/165193435 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f9ea00037c0 con 0x55b2c7065160 2024-05-09T01:00:52.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.082+0000 7f9ec082cb80 1 -- 172.21.15.5:0/165193435 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ea0003c70 con 0x55b2c7065160 2024-05-09T01:00:52.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.083+0000 7f9e8affd700 1 -- 172.21.15.5:0/165193435 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9ea0034040 con 0x55b2c7065160 2024-05-09T01:00:52.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.083+0000 7f9ec082cb80 10 client.17145 fetch_fsmap learned FSMap version 32 2024-05-09T01:00:52.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.083+0000 7f9ec082cb80 1 -- 172.21.15.5:0/165193435 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b2c710fc00 con 0x55b2c7065160 2024-05-09T01:00:52.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.083+0000 7f9e8affd700 1 -- 172.21.15.5:0/165193435 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ea0003c70 con 0x55b2c7065160 2024-05-09T01:00:52.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.083+0000 7f9e8affd700 1 -- 172.21.15.5:0/165193435 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f9ea000f460 con 0x55b2c7065160 2024-05-09T01:00:52.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.083+0000 7f9ec082cb80 10 client.17145 fetch_fsmap finished waiting for FSMap version 32 2024-05-09T01:00:52.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.083+0000 7f9ec082cb80 10 client.17145 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T01:00:52.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.083+0000 7f9ec082cb80 20 client.17145 populate_metadata read hostname 'smithi005' 2024-05-09T01:00:52.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.083+0000 7f9ec082cb80 1 --2- 172.21.15.5:0/165193435 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55b2c71120d0 0x55b2c71144f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:52.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.083+0000 7f9ec082cb80 4 client.17145 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:00:52.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.083+0000 7f9ec082cb80 1 -- 172.21.15.5:0/165193435 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b2c70289d0 con 0x55b2c71120d0 2024-05-09T01:00:52.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.083+0000 7f9eab7fe700 1 --2- 172.21.15.5:0/165193435 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55b2c71120d0 0x55b2c71144f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:52.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.084+0000 7f9e8affd700 10 client.17145 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T01:00:52.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.084+0000 7f9eab7fe700 1 --2- 172.21.15.5:0/165193435 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55b2c71120d0 0x55b2c71144f0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3089b9b901da0f90 server_cookie=856f893125740f94 in_seq=0 out_seq=0 2024-05-09T01:00:52.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.086+0000 7f9e8affd700 1 -- 172.21.15.5:0/165193435 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b2c70289d0 con 0x55b2c71120d0 2024-05-09T01:00:52.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.086+0000 7f9e8affd700 10 client.17145 handle_command_reply: tid=0 2024-05-09T01:00:52.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.086+0000 7f9ec082cb80 1 client.17145 shutdown 2024-05-09T01:00:52.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.086+0000 7f9ec082cb80 1 -- 172.21.15.5:0/165193435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2c7065160 msgr2=0x55b2c71027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:52.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.086+0000 7f9ec082cb80 1 --2- 172.21.15.5:0/165193435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2c7065160 0x55b2c71027a0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f9ea00098e0 tx=0x7f9ea000b9c0 comp rx=0 tx=0).stop 2024-05-09T01:00:52.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.086+0000 7f9ec082cb80 1 -- 172.21.15.5:0/165193435 shutdown_connections 2024-05-09T01:00:52.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.086+0000 7f9ec082cb80 1 --2- 172.21.15.5:0/165193435 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55b2c71120d0 0x55b2c71144f0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:52.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.086+0000 7f9ec082cb80 1 --2- 172.21.15.5:0/165193435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2c70f86a0 0x55b2c7102ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:52.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.086+0000 7f9ec082cb80 1 --2- 172.21.15.5:0/165193435 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2c6f7b4a0 0x55b2c7102260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:52.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.086+0000 7f9ec082cb80 1 --2- 172.21.15.5:0/165193435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2c7065160 0x55b2c71027a0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:52.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.086+0000 7f9ec082cb80 1 -- 172.21.15.5:0/165193435 >> 172.21.15.5:0/165193435 conn(0x55b2c6fdb6a0 msgr2=0x55b2c70f31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:52.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.086+0000 7f9e8affd700 0 client.17145 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T01:00:52.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.086+0000 7f9ec082cb80 1 -- 172.21.15.5:0/165193435 shutdown_connections 2024-05-09T01:00:52.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.086+0000 7f9ec082cb80 1 -- 172.21.15.5:0/165193435 wait complete. 2024-05-09T01:00:52.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.086+0000 7f9ec082cb80 20 client.17145 trim_cache size 0 max 16384 2024-05-09T01:00:52.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.107+0000 7f9ec082cb80 1 Processor -- start 2024-05-09T01:00:52.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.107+0000 7f9ec082cb80 1 -- start start 2024-05-09T01:00:52.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.107+0000 7f9ec082cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2c6f7b4a0 0x55b2c7101580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:52.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.107+0000 7f9ec082cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2c7065160 0x55b2c7102b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:52.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.107+0000 7f9ec082cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2c70f86a0 0x55b2c710e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:52.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.107+0000 7f9ec082cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac1b0de0 con 0x55b2c7065160 2024-05-09T01:00:52.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.107+0000 7f9ec082cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac1b2920 con 0x55b2c70f86a0 2024-05-09T01:00:52.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.107+0000 7f9ec082cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac1b14b0 con 0x55b2c6f7b4a0 2024-05-09T01:00:52.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.108+0000 7f9eaaffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2c7065160 0x55b2c7102b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:52.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.108+0000 7f9eab7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2c6f7b4a0 0x55b2c7101580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:52.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.108+0000 7f9eaaffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2c7065160 0x55b2c7102b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53068/0 (socket says 172.21.15.5:53068) 2024-05-09T01:00:52.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.108+0000 7f9eab7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2c6f7b4a0 0x55b2c7101580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:54756/0 (socket says 172.21.15.5:54756) 2024-05-09T01:00:52.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.108+0000 7f9eaaffd700 1 -- 172.21.15.5:0/4079473047 learned_addr learned my addr 172.21.15.5:0/4079473047 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:00:52.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.108+0000 7f9eabfff700 1 --2- 172.21.15.5:0/4079473047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2c70f86a0 0x55b2c710e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:52.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.108+0000 7f9eaaffd700 1 -- 172.21.15.5:0/4079473047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2c6f7b4a0 msgr2=0x55b2c7101580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:52.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.108+0000 7f9eaaffd700 1 --2- 172.21.15.5:0/4079473047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2c6f7b4a0 0x55b2c7101580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:52.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.108+0000 7f9eaaffd700 1 -- 172.21.15.5:0/4079473047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2c70f86a0 msgr2=0x55b2c710e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:52.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.108+0000 7f9eaaffd700 1 --2- 172.21.15.5:0/4079473047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2c70f86a0 0x55b2c710e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:52.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.108+0000 7f9eaaffd700 1 -- 172.21.15.5:0/4079473047 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2c70f4cc0 con 0x55b2c7065160 2024-05-09T01:00:52.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.108+0000 7f9eab7fe700 1 --2- 172.21.15.5:0/4079473047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2c6f7b4a0 0x55b2c7101580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:00:52.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.108+0000 7f9eabfff700 1 --2- 172.21.15.5:0/4079473047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2c70f86a0 0x55b2c710e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:00:52.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.108+0000 7f9eaaffd700 1 --2- 172.21.15.5:0/4079473047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2c7065160 0x55b2c7102b20 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f9ea000bf70 tx=0x7f9ea000b600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83244912f9e42a8e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:52.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.108+0000 7f9e917fa700 1 -- 172.21.15.5:0/4079473047 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea0047070 con 0x55b2c7065160 2024-05-09T01:00:52.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.108+0000 7f9e917fa700 1 -- 172.21.15.5:0/4079473047 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ea000f460 con 0x55b2c7065160 2024-05-09T01:00:52.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.108+0000 7f9e917fa700 1 -- 172.21.15.5:0/4079473047 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea0028020 con 0x55b2c7065160 2024-05-09T01:00:52.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.109+0000 7f9ec082cb80 1 -- 172.21.15.5:0/4079473047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2c7065160 msgr2=0x55b2c7102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:52.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.109+0000 7f9ec082cb80 1 --2- 172.21.15.5:0/4079473047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2c7065160 0x55b2c7102b20 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f9ea000bf70 tx=0x7f9ea000b600 comp rx=0 tx=0).stop 2024-05-09T01:00:52.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.109+0000 7f9ec082cb80 1 -- 172.21.15.5:0/4079473047 shutdown_connections 2024-05-09T01:00:52.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.109+0000 7f9ec082cb80 1 --2- 172.21.15.5:0/4079473047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2c70f86a0 0x55b2c710e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:52.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.109+0000 7f9ec082cb80 1 --2- 172.21.15.5:0/4079473047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2c6f7b4a0 0x55b2c7101580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:52.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.109+0000 7f9ec082cb80 1 --2- 172.21.15.5:0/4079473047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2c7065160 0x55b2c7102b20 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:52.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.109+0000 7f9ec082cb80 1 -- 172.21.15.5:0/4079473047 >> 172.21.15.5:0/4079473047 conn(0x55b2c6fdb6a0 msgr2=0x55b2c70fbe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:52.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.109+0000 7f9ec082cb80 1 -- 172.21.15.5:0/4079473047 shutdown_connections 2024-05-09T01:00:52.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.109+0000 7f9ec082cb80 1 -- 172.21.15.5:0/4079473047 wait complete. 2024-05-09T01:00:52.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.109+0000 7f9ec082cb80 1 -- start start 2024-05-09T01:00:52.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.110+0000 7f9ec082cb80 1 Processor -- start 2024-05-09T01:00:52.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.110+0000 7f9ec082cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2c6f7b4a0 0x55b2c7114de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:52.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.110+0000 7f9ec082cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2c7065160 0x55b2c7115320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:52.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.110+0000 7f9ec082cb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2c70f86a0 0x55b2c7115860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:52.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.110+0000 7f9ec082cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac1b2250 con 0x55b2c7065160 2024-05-09T01:00:52.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.110+0000 7f9ec082cb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac1b1b80 con 0x55b2c70f86a0 2024-05-09T01:00:52.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.110+0000 7f9ec082cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac1aed80 con 0x55b2c6f7b4a0 2024-05-09T01:00:52.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.110+0000 7f9ec082cb80 10 client.0 _ll_register_callbacks cb 0x55b2c6e8c6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:00:52.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.110+0000 7f9eab7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2c6f7b4a0 0x55b2c7114de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:52.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.110+0000 7f9eaaffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2c7065160 0x55b2c7115320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:52.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.110+0000 7f9eaaffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2c7065160 0x55b2c7115320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53076/0 (socket says 172.21.15.5:53076) 2024-05-09T01:00:52.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.110+0000 7f9eab7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2c6f7b4a0 0x55b2c7114de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:54768/0 (socket says 172.21.15.5:54768) 2024-05-09T01:00:52.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.110+0000 7f9eaaffd700 1 -- 172.21.15.5:0/2189459121 learned_addr learned my addr 172.21.15.5:0/2189459121 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:00:52.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.110+0000 7f9eabfff700 1 --2- 172.21.15.5:0/2189459121 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2c70f86a0 0x55b2c7115860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:52.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.111+0000 7f9eaaffd700 1 -- 172.21.15.5:0/2189459121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2c6f7b4a0 msgr2=0x55b2c7114de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:52.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.111+0000 7f9eaaffd700 1 --2- 172.21.15.5:0/2189459121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2c6f7b4a0 0x55b2c7114de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:52.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.111+0000 7f9eaaffd700 1 -- 172.21.15.5:0/2189459121 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2c70f86a0 msgr2=0x55b2c7115860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:00:52.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.111+0000 7f9eaaffd700 1 --2- 172.21.15.5:0/2189459121 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2c70f86a0 0x55b2c7115860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:52.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.111+0000 7f9eaaffd700 1 -- 172.21.15.5:0/2189459121 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2c70ecf60 con 0x55b2c7065160 2024-05-09T01:00:52.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.111+0000 7f9eab7fe700 1 --2- 172.21.15.5:0/2189459121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2c6f7b4a0 0x55b2c7114de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:00:52.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.111+0000 7f9eaaffd700 1 --2- 172.21.15.5:0/2189459121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2c7065160 0x55b2c7115320 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0003a80 tx=0x7f9ea0005a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58c1c9dcd25255e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:00:52.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.111+0000 7f9e8affd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T01:00:52.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.111+0000 7f9e8affd700 1 -- 172.21.15.5:0/2189459121 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea0028030 con 0x55b2c7065160 2024-05-09T01:00:52.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.111+0000 7f9e8affd700 1 -- 172.21.15.5:0/2189459121 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f9ea000b9a0 con 0x55b2c7065160 2024-05-09T01:00:52.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.111+0000 7f9ec082cb80 1 -- 172.21.15.5:0/2189459121 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ea000f460 con 0x55b2c7065160 2024-05-09T01:00:52.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.111+0000 7f9e8affd700 1 -- 172.21.15.5:0/2189459121 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9ea0047040 con 0x55b2c7065160 2024-05-09T01:00:52.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.111+0000 7f9ec082cb80 10 client.17151 fetch_fsmap learned FSMap version 32 2024-05-09T01:00:52.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.111+0000 7f9ec082cb80 1 -- 172.21.15.5:0/2189459121 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b2c717c740 con 0x55b2c7065160 2024-05-09T01:00:52.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.111+0000 7f9e8affd700 1 -- 172.21.15.5:0/2189459121 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ea000f460 con 0x55b2c7065160 2024-05-09T01:00:52.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.112+0000 7f9e8affd700 1 -- 172.21.15.5:0/2189459121 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f9ea002f500 con 0x55b2c7065160 2024-05-09T01:00:52.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.112+0000 7f9ec082cb80 10 client.17151 fetch_fsmap finished waiting for FSMap version 32 2024-05-09T01:00:52.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.112+0000 7f9ec082cb80 10 client.17151 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T01:00:52.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.112+0000 7f9ec082cb80 20 client.17151 populate_metadata read hostname 'smithi005' 2024-05-09T01:00:52.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.112+0000 7f9ec082cb80 1 --2- 172.21.15.5:0/2189459121 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55b2c717cde0 0x55b2c717f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:00:52.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.112+0000 7f9ec082cb80 4 client.17151 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T01:00:52.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.112+0000 7f9ec082cb80 1 -- 172.21.15.5:0/2189459121 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b2c7022a70 con 0x55b2c717cde0 2024-05-09T01:00:52.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.112+0000 7f9eab7fe700 1 --2- 172.21.15.5:0/2189459121 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55b2c717cde0 0x55b2c717f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:00:52.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.112+0000 7f9e8affd700 10 client.17151 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T01:00:52.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.112+0000 7f9eab7fe700 1 --2- 172.21.15.5:0/2189459121 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55b2c717cde0 0x55b2c717f200 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25ab81d8a705fc05 server_cookie=f515340731c4ae33 in_seq=0 out_seq=0 2024-05-09T01:00:52.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.114+0000 7f9e8affd700 1 -- 172.21.15.5:0/2189459121 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+265 (crc 0 0 0) 0x55b2c7022a70 con 0x55b2c717cde0 2024-05-09T01:00:52.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.114+0000 7f9e8affd700 10 client.17151 handle_command_reply: tid=0 2024-05-09T01:00:52.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.114+0000 7f9ec082cb80 1 client.17151 shutdown 2024-05-09T01:00:52.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.114+0000 7f9ec082cb80 1 -- 172.21.15.5:0/2189459121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2c7065160 msgr2=0x55b2c7115320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:52.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.114+0000 7f9ec082cb80 1 --2- 172.21.15.5:0/2189459121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2c7065160 0x55b2c7115320 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0003a80 tx=0x7f9ea0005a70 comp rx=0 tx=0).stop 2024-05-09T01:00:52.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.114+0000 7f9ec082cb80 1 -- 172.21.15.5:0/2189459121 shutdown_connections 2024-05-09T01:00:52.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.114+0000 7f9ec082cb80 1 --2- 172.21.15.5:0/2189459121 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55b2c717cde0 0x55b2c717f200 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:52.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.114+0000 7f9ec082cb80 1 --2- 172.21.15.5:0/2189459121 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b2c70f86a0 0x55b2c7115860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:52.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.114+0000 7f9ec082cb80 1 --2- 172.21.15.5:0/2189459121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b2c6f7b4a0 0x55b2c7114de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:52.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.114+0000 7f9ec082cb80 1 --2- 172.21.15.5:0/2189459121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2c7065160 0x55b2c7115320 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:52.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.114+0000 7f9ec082cb80 1 -- 172.21.15.5:0/2189459121 >> 172.21.15.5:0/2189459121 conn(0x55b2c6fdb6a0 msgr2=0x55b2c71012e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:52.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.114+0000 7f9e8affd700 0 client.17151 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T01:00:52.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.114+0000 7f9ec082cb80 1 -- 172.21.15.5:0/2189459121 shutdown_connections 2024-05-09T01:00:52.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.114+0000 7f9ec082cb80 1 -- 172.21.15.5:0/2189459121 wait complete. 2024-05-09T01:00:52.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.114+0000 7f9ec082cb80 20 client.17151 trim_cache size 0 max 16384 2024-05-09T01:00:52.120 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-09T01:00:52.120 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "scrub active (1 inodes in the stack)", 2024-05-09T01:00:52.120 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-05-09T01:00:52.120 INFO:teuthology.orchestra.run.smithi005.stdout: "1b609027-503b-485e-bcb9-ba2628dcbe62": { 2024-05-09T01:00:52.120 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-05-09T01:00:52.120 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "1b609027-503b-485e-bcb9-ba2628dcbe62", 2024-05-09T01:00:52.121 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-05-09T01:00:52.121 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:00:52.121 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:00:52.121 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-09T01:00:52.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.115+0000 7f9eb1e16700 1 -- 172.21.15.5:0/1259759506 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f9e940b2900 msgr2=0x7f9e940b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:52.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.115+0000 7f9eb1e16700 1 --2- 172.21.15.5:0/1259759506 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f9e940b2900 0x7f9e940b4e10 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0000c00 tx=0x7f9ea001a040 comp rx=0 tx=0).stop 2024-05-09T01:00:52.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.115+0000 7f9eb1e16700 1 -- 172.21.15.5:0/1259759506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9eac1045d0 msgr2=0x7f9eac19ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:00:52.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.115+0000 7f9eb1e16700 1 --2- 172.21.15.5:0/1259759506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9eac1045d0 0x7f9eac19ae00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c00d950 tx=0x7f9e9c00dc60 comp rx=0 tx=0).stop 2024-05-09T01:00:52.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.115+0000 7f9eb1e16700 1 -- 172.21.15.5:0/1259759506 shutdown_connections 2024-05-09T01:00:52.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.115+0000 7f9eb1e16700 1 --2- 172.21.15.5:0/1259759506 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f9e940b2900 0x7f9e940b4e10 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:52.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.115+0000 7f9eb1e16700 1 --2- 172.21.15.5:0/1259759506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9eac109850 0x7f9eac19f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:52.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.115+0000 7f9eb1e16700 1 --2- 172.21.15.5:0/1259759506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9eac1045d0 0x7f9eac19ae00 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:52.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.115+0000 7f9eb1e16700 1 --2- 172.21.15.5:0/1259759506 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9eac108f00 0x7f9eac19b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:00:52.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.115+0000 7f9eb1e16700 1 -- 172.21.15.5:0/1259759506 >> 172.21.15.5:0/1259759506 conn(0x7f9eac0ffea0 msgr2=0x7f9eac10a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:00:52.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.115+0000 7f9eb1e16700 1 -- 172.21.15.5:0/1259759506 shutdown_connections 2024-05-09T01:00:52.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:00:52.115+0000 7f9eb1e16700 1 -- 172.21.15.5:0/1259759506 wait complete. 2024-05-09T01:00:52.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:00:52 smithi131 ceph-mon[34547]: pgmap v806: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 40 MiB/s wr, 207 op/s 2024-05-09T01:00:52.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:00:52 smithi157 ceph-mon[34294]: pgmap v806: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 40 MiB/s wr, 207 op/s 2024-05-09T01:00:52.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:00:52 smithi005 ceph-mon[30365]: pgmap v806: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 40 MiB/s wr, 207 op/s 2024-05-09T01:00:52.845 INFO:tasks.cephfs.filesystem:scrub status for tag:1b609027-503b-485e-bcb9-ba2628dcbe62 - {'path': '/', 'tag': '1b609027-503b-485e-bcb9-ba2628dcbe62', 'options': 'recursive,force'} 2024-05-09T01:00:54.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:00:54 smithi131 ceph-mon[34547]: pgmap v807: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 38 MiB/s wr, 232 op/s 2024-05-09T01:00:54.443 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:00:54 smithi157 ceph-mon[34294]: pgmap v807: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 38 MiB/s wr, 232 op/s 2024-05-09T01:00:54.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:00:54 smithi005 ceph-mon[30365]: pgmap v807: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 38 MiB/s wr, 232 op/s 2024-05-09T01:00:56.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:00:56 smithi131 ceph-mon[34547]: pgmap v808: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 44 MiB/s wr, 290 op/s 2024-05-09T01:00:56.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:00:56 smithi157 ceph-mon[34294]: pgmap v808: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 44 MiB/s wr, 290 op/s 2024-05-09T01:00:56.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:00:56 smithi005 ceph-mon[30365]: pgmap v808: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 44 MiB/s wr, 290 op/s 2024-05-09T01:00:58.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:00:58 smithi131 ceph-mon[34547]: pgmap v809: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 49 MiB/s wr, 352 op/s 2024-05-09T01:00:58.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:00:58 smithi157 ceph-mon[34294]: pgmap v809: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 49 MiB/s wr, 352 op/s 2024-05-09T01:00:58.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:00:58 smithi005 ceph-mon[30365]: pgmap v809: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 49 MiB/s wr, 352 op/s 2024-05-09T01:01:00.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:01:00 smithi131 ceph-mon[34547]: pgmap v810: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 66 MiB/s wr, 466 op/s 2024-05-09T01:01:00.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:01:00 smithi157 ceph-mon[34294]: pgmap v810: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 66 MiB/s wr, 466 op/s 2024-05-09T01:01:00.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:01:00 smithi005 ceph-mon[30365]: pgmap v810: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 66 MiB/s wr, 466 op/s 2024-05-09T01:01:02.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:01:02 smithi131 ceph-mon[34547]: pgmap v811: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 78 MiB/s wr, 604 op/s 2024-05-09T01:01:02.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:01:02 smithi157 ceph-mon[34294]: pgmap v811: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 78 MiB/s wr, 604 op/s 2024-05-09T01:01:02.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:01:02 smithi005 ceph-mon[30365]: pgmap v811: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 78 MiB/s wr, 604 op/s 2024-05-09T01:01:03.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.958+0000 7f4c60fe8640 1 -- 172.21.15.5:0/2670946341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c5c102b50 msgr2=0x7f4c5c103850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:03.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.958+0000 7f4c60fe8640 1 --2- 172.21.15.5:0/2670946341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c5c102b50 0x7f4c5c103850 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c009900 tx=0x7f4c4c02a3f0 comp rx=0 tx=0).stop 2024-05-09T01:01:03.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.958+0000 7f4c60fe8640 1 -- 172.21.15.5:0/2670946341 shutdown_connections 2024-05-09T01:01:03.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.958+0000 7f4c60fe8640 1 --2- 172.21.15.5:0/2670946341 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c5c06a4f0 0x7f4c5c06a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:03.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.958+0000 7f4c60fe8640 1 --2- 172.21.15.5:0/2670946341 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c5c103d90 0x7f4c5c1041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:03.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.958+0000 7f4c60fe8640 1 --2- 172.21.15.5:0/2670946341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c5c102b50 0x7f4c5c103850 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:03.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.958+0000 7f4c60fe8640 1 -- 172.21.15.5:0/2670946341 >> 172.21.15.5:0/2670946341 conn(0x7f4c5c0fdd40 msgr2=0x7f4c5c1001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:01:03.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.960+0000 7f4c60fe8640 1 -- 172.21.15.5:0/2670946341 shutdown_connections 2024-05-09T01:01:03.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.960+0000 7f4c60fe8640 1 -- 172.21.15.5:0/2670946341 wait complete. 2024-05-09T01:01:03.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.962+0000 7f4c60fe8640 1 Processor -- start 2024-05-09T01:01:03.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.962+0000 7f4c60fe8640 1 -- start start 2024-05-09T01:01:03.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.962+0000 7f4c60fe8640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c5c06a4f0 0x7f4c5c108500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:03.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.962+0000 7f4c60fe8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c5c102b50 0x7f4c5c106ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:03.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.962+0000 7f4c60fe8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c5c103d90 0x7f4c5c1070e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:03.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.962+0000 7f4c60fe8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5c06cf50 con 0x7f4c5c103d90 2024-05-09T01:01:03.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.962+0000 7f4c60fe8640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5c06cdd0 con 0x7f4c5c06a4f0 2024-05-09T01:01:03.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.962+0000 7f4c60fe8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5c06d0d0 con 0x7f4c5c102b50 2024-05-09T01:01:03.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.963+0000 7f4c5ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c5c103d90 0x7f4c5c1070e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:03.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.963+0000 7f4c59d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c5c102b50 0x7f4c5c106ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:03.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.963+0000 7f4c5ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c5c103d90 0x7f4c5c1070e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50196/0 (socket says 172.21.15.5:50196) 2024-05-09T01:01:03.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.963+0000 7f4c5ad9d640 1 -- 172.21.15.5:0/814332526 learned_addr learned my addr 172.21.15.5:0/814332526 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:01:03.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.963+0000 7f4c59d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c5c102b50 0x7f4c5c106ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:54232/0 (socket says 172.21.15.5:54232) 2024-05-09T01:01:03.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.963+0000 7f4c5a59c640 1 --2- 172.21.15.5:0/814332526 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c5c06a4f0 0x7f4c5c108500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:03.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.964+0000 7f4c59d9b640 1 -- 172.21.15.5:0/814332526 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c5c06a4f0 msgr2=0x7f4c5c108500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:03.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.964+0000 7f4c59d9b640 1 --2- 172.21.15.5:0/814332526 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c5c06a4f0 0x7f4c5c108500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:03.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.964+0000 7f4c59d9b640 1 -- 172.21.15.5:0/814332526 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c5c103d90 msgr2=0x7f4c5c1070e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:03.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.964+0000 7f4c59d9b640 1 --2- 172.21.15.5:0/814332526 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c5c103d90 0x7f4c5c1070e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:03.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.964+0000 7f4c59d9b640 1 -- 172.21.15.5:0/814332526 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c5c1a8eb0 con 0x7f4c5c102b50 2024-05-09T01:01:03.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.964+0000 7f4c5ad9d640 1 --2- 172.21.15.5:0/814332526 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c5c103d90 0x7f4c5c1070e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:01:03.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.965+0000 7f4c59d9b640 1 --2- 172.21.15.5:0/814332526 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c5c102b50 0x7f4c5c106ba0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4c4400e9f0 tx=0x7f4c4400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:01:03.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.966+0000 7f4c537fe640 1 -- 172.21.15.5:0/814332526 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4400ccc0 con 0x7f4c5c102b50 2024-05-09T01:01:03.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.966+0000 7f4c537fe640 1 -- 172.21.15.5:0/814332526 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c44005020 con 0x7f4c5c102b50 2024-05-09T01:01:03.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.966+0000 7f4c537fe640 1 -- 172.21.15.5:0/814332526 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c44004a00 con 0x7f4c5c102b50 2024-05-09T01:01:03.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.968+0000 7f4c60fe8640 1 -- 172.21.15.5:0/814332526 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c5c1a91a0 con 0x7f4c5c102b50 2024-05-09T01:01:03.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.968+0000 7f4c60fe8640 1 -- 172.21.15.5:0/814332526 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c5c1a96c0 con 0x7f4c5c102b50 2024-05-09T01:01:03.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.979+0000 7f4c67f70740 1 Processor -- start 2024-05-09T01:01:03.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.979+0000 7f4c537fe640 1 -- 172.21.15.5:0/814332526 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f4c44004ba0 con 0x7f4c5c102b50 2024-05-09T01:01:03.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.979+0000 7f4c67f70740 1 -- start start 2024-05-09T01:01:03.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.980+0000 7f4c67f70740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd806441b0 0x55fd809e8500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:03.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.980+0000 7f4c67f70740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd8061b2f0 0x55fd809e8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:03.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.980+0000 7f4c67f70740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd806234d0 0x55fd809ef980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:03.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.980+0000 7f4c67f70740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fd8094c7c0 con 0x55fd806234d0 2024-05-09T01:01:03.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.980+0000 7f4c67f70740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55fd808e80e0 con 0x55fd806441b0 2024-05-09T01:01:03.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.980+0000 7f4c67f70740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55fd80948780 con 0x55fd8061b2f0 2024-05-09T01:01:03.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.981+0000 7f4c5ad9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd806441b0 0x55fd809e8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:03.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.981+0000 7f4c5ad9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd806441b0 0x55fd809e8500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:39174/0 (socket says 172.21.15.5:39174) 2024-05-09T01:01:03.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.981+0000 7f4c5ad9d640 1 -- 172.21.15.5:0/4276675392 learned_addr learned my addr 172.21.15.5:0/4276675392 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:01:03.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.981+0000 7f4c5a59c640 1 --2- 172.21.15.5:0/4276675392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd8061b2f0 0x55fd809e8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.981+0000 7f4c59d9b640 1 --2- 172.21.15.5:0/4276675392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd806234d0 0x55fd809ef980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.982+0000 7f4c59d9b640 1 -- 172.21.15.5:0/4276675392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd8061b2f0 msgr2=0x55fd809e8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:01:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.982+0000 7f4c59d9b640 1 --2- 172.21.15.5:0/4276675392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd8061b2f0 0x55fd809e8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.982+0000 7f4c59d9b640 1 -- 172.21.15.5:0/4276675392 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd806441b0 msgr2=0x55fd809e8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:01:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.982+0000 7f4c59d9b640 1 --2- 172.21.15.5:0/4276675392 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd806441b0 0x55fd809e8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.982+0000 7f4c59d9b640 1 -- 172.21.15.5:0/4276675392 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd809117c0 con 0x55fd806234d0 2024-05-09T01:01:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.982+0000 7f4c5ad9d640 1 --2- 172.21.15.5:0/4276675392 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd806441b0 0x55fd809e8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:01:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.982+0000 7f4c59d9b640 1 --2- 172.21.15.5:0/4276675392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd806234d0 0x55fd809ef980 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f4c440dfbf0 tx=0x7f4c440ea2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c49604a097bed208 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:01:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.982+0000 7f4c537fe640 1 --2- 172.21.15.5:0/814332526 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f4c480b29b0 0x7f4c480b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.982+0000 7f4c5ad9d640 1 --2- 172.21.15.5:0/814332526 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f4c480b29b0 0x7f4c480b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.982+0000 7f4c50ff9640 1 -- 172.21.15.5:0/4276675392 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c440f5070 con 0x55fd806234d0 2024-05-09T01:01:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.983+0000 7f4c50ff9640 1 -- 172.21.15.5:0/4276675392 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c440daf20 con 0x55fd806234d0 2024-05-09T01:01:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.983+0000 7f4c50ff9640 1 -- 172.21.15.5:0/4276675392 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c440a9280 con 0x55fd806234d0 2024-05-09T01:01:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.983+0000 7f4c537fe640 1 -- 172.21.15.5:0/814332526 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f4c440e21e0 con 0x7f4c5c102b50 2024-05-09T01:01:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.984+0000 7f4c67f70740 1 -- 172.21.15.5:0/4276675392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd806234d0 msgr2=0x55fd809ef980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.984+0000 7f4c67f70740 1 --2- 172.21.15.5:0/4276675392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd806234d0 0x55fd809ef980 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f4c440dfbf0 tx=0x7f4c440ea2f0 comp rx=0 tx=0).stop 2024-05-09T01:01:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.984+0000 7f4c5ad9d640 1 --2- 172.21.15.5:0/814332526 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f4c480b29b0 0x7f4c480b4ec0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4c54009500 tx=0x7f4c5400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:01:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.984+0000 7f4c67f70740 1 -- 172.21.15.5:0/4276675392 shutdown_connections 2024-05-09T01:01:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.984+0000 7f4c67f70740 1 --2- 172.21.15.5:0/4276675392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd806234d0 0x55fd809ef980 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.985+0000 7f4c67f70740 1 --2- 172.21.15.5:0/4276675392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd8061b2f0 0x55fd809e8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.985+0000 7f4c67f70740 1 --2- 172.21.15.5:0/4276675392 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd806441b0 0x55fd809e8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.985+0000 7f4c67f70740 1 -- 172.21.15.5:0/4276675392 >> 172.21.15.5:0/4276675392 conn(0x55fd80638300 msgr2=0x55fd809e3e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:01:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.985+0000 7f4c67f70740 1 -- 172.21.15.5:0/4276675392 shutdown_connections 2024-05-09T01:01:03.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.985+0000 7f4c67f70740 1 -- 172.21.15.5:0/4276675392 wait complete. 2024-05-09T01:01:03.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.986+0000 7f4c67f70740 1 -- start start 2024-05-09T01:01:03.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.987+0000 7f4c67f70740 1 Processor -- start 2024-05-09T01:01:03.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.987+0000 7f4c67f70740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd8061b2f0 0x55fd809f2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:03.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.988+0000 7f4c67f70740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd806234d0 0x55fd809f27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:03.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.988+0000 7f4c67f70740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd806441b0 0x55fd809f9720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:03.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.988+0000 7f4c67f70740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fd809f03d0 con 0x55fd8061b2f0 2024-05-09T01:01:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.988+0000 7f4c67f70740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55fd808d39c0 con 0x55fd806234d0 2024-05-09T01:01:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.988+0000 7f4c67f70740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55fd809f0550 con 0x55fd806441b0 2024-05-09T01:01:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.988+0000 7f4c67f70740 10 client.0 _ll_register_callbacks cb 0x55fd80900020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:01:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.988+0000 7f4c5a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd8061b2f0 0x55fd809f2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.988+0000 7f4c5a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd8061b2f0 0x55fd809f2280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50206/0 (socket says 172.21.15.5:50206) 2024-05-09T01:01:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.988+0000 7f4c5a59c640 1 -- 172.21.15.5:0/2542774150 learned_addr learned my addr 172.21.15.5:0/2542774150 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:01:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.988+0000 7f4c5ad9d640 1 --2- 172.21.15.5:0/2542774150 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd806441b0 0x55fd809f9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.988+0000 7f4c59d9b640 1 --2- 172.21.15.5:0/2542774150 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd806234d0 0x55fd809f27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.989+0000 7f4c59d9b640 1 -- 172.21.15.5:0/2542774150 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd806441b0 msgr2=0x55fd809f9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:01:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.989+0000 7f4c59d9b640 1 --2- 172.21.15.5:0/2542774150 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd806441b0 0x55fd809f9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.989+0000 7f4c59d9b640 1 -- 172.21.15.5:0/2542774150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd8061b2f0 msgr2=0x55fd809f2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:01:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.989+0000 7f4c59d9b640 1 --2- 172.21.15.5:0/2542774150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd8061b2f0 0x55fd809f2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.989+0000 7f4c59d9b640 1 -- 172.21.15.5:0/2542774150 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd808fa1f0 con 0x55fd806234d0 2024-05-09T01:01:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.989+0000 7f4c59d9b640 1 --2- 172.21.15.5:0/2542774150 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd806234d0 0x55fd809f27c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4c440a5070 tx=0x7f4c440e56d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7614e857d4f1f2ce server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:01:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.989+0000 7f4c41ffb640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-09T01:01:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.989+0000 7f4c41ffb640 1 -- 172.21.15.5:0/2542774150 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4401c020 con 0x55fd806234d0 2024-05-09T01:01:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.990+0000 7f4c41ffb640 1 -- 172.21.15.5:0/2542774150 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f4c440e5d00 con 0x55fd806234d0 2024-05-09T01:01:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.990+0000 7f4c67f70740 1 -- 172.21.15.5:0/2542774150 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c440daf20 con 0x55fd806234d0 2024-05-09T01:01:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.990+0000 7f4c41ffb640 1 -- 172.21.15.5:0/2542774150 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4c440a5630 con 0x55fd806234d0 2024-05-09T01:01:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.990+0000 7f4c67f70740 10 client.26564 fetch_fsmap learned FSMap version 32 2024-05-09T01:01:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.990+0000 7f4c67f70740 1 -- 172.21.15.5:0/2542774150 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55fd806af790 con 0x55fd806234d0 2024-05-09T01:01:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.990+0000 7f4c41ffb640 1 -- 172.21.15.5:0/2542774150 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c440a9280 con 0x55fd806234d0 2024-05-09T01:01:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.990+0000 7f4c41ffb640 1 -- 172.21.15.5:0/2542774150 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f4c440e5350 con 0x55fd806234d0 2024-05-09T01:01:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.990+0000 7f4c67f70740 10 client.26564 fetch_fsmap finished waiting for FSMap version 32 2024-05-09T01:01:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.990+0000 7f4c67f70740 10 client.26564 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T01:01:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.990+0000 7f4c67f70740 20 client.26564 populate_metadata read hostname 'smithi005' 2024-05-09T01:01:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.991+0000 7f4c67f70740 1 --2- 172.21.15.5:0/2542774150 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55fd80602d80 0x55fd809ff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.991+0000 7f4c67f70740 4 client.26564 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:01:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.991+0000 7f4c67f70740 1 -- 172.21.15.5:0/2542774150 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4c440daf20 con 0x55fd80602d80 2024-05-09T01:01:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.991+0000 7f4c5a59c640 1 --2- 172.21.15.5:0/2542774150 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55fd80602d80 0x55fd809ff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.992+0000 7f4c5a59c640 1 --2- 172.21.15.5:0/2542774150 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55fd80602d80 0x55fd809ff7e0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1447219a42fc113 server_cookie=8a092eb8999243f7 in_seq=0 out_seq=0 2024-05-09T01:01:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.992+0000 7f4c41ffb640 10 client.26564 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T01:01:03.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.995+0000 7f4c41ffb640 1 -- 172.21.15.5:0/2542774150 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f4c440daf20 con 0x55fd80602d80 2024-05-09T01:01:03.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.995+0000 7f4c41ffb640 10 client.26564 handle_command_reply: tid=0 2024-05-09T01:01:03.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.995+0000 7f4c67f70740 1 client.26564 shutdown 2024-05-09T01:01:03.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.995+0000 7f4c67f70740 1 -- 172.21.15.5:0/2542774150 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd806234d0 msgr2=0x55fd809f27c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:03.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.995+0000 7f4c67f70740 1 --2- 172.21.15.5:0/2542774150 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd806234d0 0x55fd809f27c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4c440a5070 tx=0x7f4c440e56d0 comp rx=0 tx=0).stop 2024-05-09T01:01:03.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.995+0000 7f4c67f70740 1 -- 172.21.15.5:0/2542774150 shutdown_connections 2024-05-09T01:01:03.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.995+0000 7f4c67f70740 1 --2- 172.21.15.5:0/2542774150 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55fd80602d80 0x55fd809ff7e0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:03.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.995+0000 7f4c67f70740 1 --2- 172.21.15.5:0/2542774150 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd806441b0 0x55fd809f9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:03.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.995+0000 7f4c67f70740 1 --2- 172.21.15.5:0/2542774150 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd806234d0 0x55fd809f27c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:03.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.995+0000 7f4c67f70740 1 --2- 172.21.15.5:0/2542774150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd8061b2f0 0x55fd809f2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:03.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.995+0000 7f4c41ffb640 0 client.26564 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T01:01:03.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.995+0000 7f4c67f70740 1 -- 172.21.15.5:0/2542774150 >> 172.21.15.5:0/2542774150 conn(0x55fd80638300 msgr2=0x55fd809e6da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:01:03.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.995+0000 7f4c67f70740 1 -- 172.21.15.5:0/2542774150 shutdown_connections 2024-05-09T01:01:03.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.995+0000 7f4c67f70740 1 -- 172.21.15.5:0/2542774150 wait complete. 2024-05-09T01:01:03.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:03.995+0000 7f4c67f70740 20 client.26564 trim_cache size 0 max 16384 2024-05-09T01:01:04.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.010+0000 7f4c67f70740 1 Processor -- start 2024-05-09T01:01:04.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.011+0000 7f4c67f70740 1 -- start start 2024-05-09T01:01:04.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.011+0000 7f4c67f70740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd80602d80 0x55fd809ec650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:04.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.011+0000 7f4c67f70740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd8061b2f0 0x55fd809f9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:04.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.011+0000 7f4c67f70740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd806234d0 0x55fd809ecb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:04.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.011+0000 7f4c67f70740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5c1c2a90 con 0x55fd806234d0 2024-05-09T01:01:04.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.011+0000 7f4c67f70740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5c1c45d0 con 0x55fd80602d80 2024-05-09T01:01:04.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.011+0000 7f4c67f70740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5c1c3160 con 0x55fd8061b2f0 2024-05-09T01:01:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.011+0000 7f4c5ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd806234d0 0x55fd809ecb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.011+0000 7f4c5a59c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd80602d80 0x55fd809ec650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.011+0000 7f4c59d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd8061b2f0 0x55fd809f9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.011+0000 7f4c5a59c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd80602d80 0x55fd809ec650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:39200/0 (socket says 172.21.15.5:39200) 2024-05-09T01:01:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.011+0000 7f4c59d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd8061b2f0 0x55fd809f9640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:54254/0 (socket says 172.21.15.5:54254) 2024-05-09T01:01:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.011+0000 7f4c5a59c640 1 -- 172.21.15.5:0/3464182720 learned_addr learned my addr 172.21.15.5:0/3464182720 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:01:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.011+0000 7f4c5a59c640 1 -- 172.21.15.5:0/3464182720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd8061b2f0 msgr2=0x55fd809f9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:01:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.011+0000 7f4c5a59c640 1 --2- 172.21.15.5:0/3464182720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd8061b2f0 0x55fd809f9640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.011+0000 7f4c5a59c640 1 -- 172.21.15.5:0/3464182720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd806234d0 msgr2=0x55fd809ecb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:01:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.011+0000 7f4c5a59c640 1 --2- 172.21.15.5:0/3464182720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd806234d0 0x55fd809ecb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.011+0000 7f4c5a59c640 1 -- 172.21.15.5:0/3464182720 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd807265f0 con 0x55fd80602d80 2024-05-09T01:01:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.012+0000 7f4c5a59c640 1 --2- 172.21.15.5:0/3464182720 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd80602d80 0x55fd809ec650 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c009900 tx=0x7f4c4c000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88b497686db0957a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:01:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.012+0000 7f4c43fff640 1 -- 172.21.15.5:0/3464182720 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c035020 con 0x55fd80602d80 2024-05-09T01:01:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.012+0000 7f4c43fff640 1 -- 172.21.15.5:0/3464182720 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c4c04b070 con 0x55fd80602d80 2024-05-09T01:01:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.012+0000 7f4c43fff640 1 -- 172.21.15.5:0/3464182720 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c0046c0 con 0x55fd80602d80 2024-05-09T01:01:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.013+0000 7f4c67f70740 1 -- 172.21.15.5:0/3464182720 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd80602d80 msgr2=0x55fd809ec650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.013+0000 7f4c67f70740 1 --2- 172.21.15.5:0/3464182720 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd80602d80 0x55fd809ec650 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c009900 tx=0x7f4c4c000bc0 comp rx=0 tx=0).stop 2024-05-09T01:01:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.013+0000 7f4c67f70740 1 -- 172.21.15.5:0/3464182720 shutdown_connections 2024-05-09T01:01:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.013+0000 7f4c67f70740 1 --2- 172.21.15.5:0/3464182720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd806234d0 0x55fd809ecb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.013+0000 7f4c67f70740 1 --2- 172.21.15.5:0/3464182720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd8061b2f0 0x55fd809f9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.013+0000 7f4c67f70740 1 --2- 172.21.15.5:0/3464182720 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd80602d80 0x55fd809ec650 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.013+0000 7f4c67f70740 1 -- 172.21.15.5:0/3464182720 >> 172.21.15.5:0/3464182720 conn(0x55fd80638300 msgr2=0x55fd809e8480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:01:04.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.013+0000 7f4c67f70740 1 -- 172.21.15.5:0/3464182720 shutdown_connections 2024-05-09T01:01:04.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.013+0000 7f4c67f70740 1 -- 172.21.15.5:0/3464182720 wait complete. 2024-05-09T01:01:04.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.013+0000 7f4c67f70740 1 -- start start 2024-05-09T01:01:04.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.013+0000 7f4c67f70740 1 Processor -- start 2024-05-09T01:01:04.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.014+0000 7f4c67f70740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd80602d80 0x55fd809f6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:04.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.014+0000 7f4c67f70740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd8061b2f0 0x55fd809f6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:04.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.014+0000 7f4c67f70740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd806234d0 0x55fd809f7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:04.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.014+0000 7f4c67f70740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5c1c3f00 con 0x55fd806234d0 2024-05-09T01:01:04.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.014+0000 7f4c67f70740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5c1c3830 con 0x55fd80602d80 2024-05-09T01:01:04.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.014+0000 7f4c5a59c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd80602d80 0x55fd809f6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:04.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.014+0000 7f4c67f70740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5c1afa60 con 0x55fd8061b2f0 2024-05-09T01:01:04.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.014+0000 7f4c5a59c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd80602d80 0x55fd809f6950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.5:39202/0 (socket says 172.21.15.5:39202) 2024-05-09T01:01:04.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.014+0000 7f4c5a59c640 1 -- 172.21.15.5:0/522319561 learned_addr learned my addr 172.21.15.5:0/522319561 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:01:04.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.014+0000 7f4c67f70740 10 client.0 _ll_register_callbacks cb 0x7f4c440f3070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:01:04.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.014+0000 7f4c59d9b640 1 --2- 172.21.15.5:0/522319561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd8061b2f0 0x55fd809f6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:04.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.014+0000 7f4c5ad9d640 1 --2- 172.21.15.5:0/522319561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd806234d0 0x55fd809f7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:04.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.014+0000 7f4c5a59c640 1 -- 172.21.15.5:0/522319561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd8061b2f0 msgr2=0x55fd809f6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:01:04.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.014+0000 7f4c5a59c640 1 --2- 172.21.15.5:0/522319561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd8061b2f0 0x55fd809f6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:04.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.014+0000 7f4c5a59c640 1 -- 172.21.15.5:0/522319561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd806234d0 msgr2=0x55fd809f7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:01:04.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.014+0000 7f4c5a59c640 1 --2- 172.21.15.5:0/522319561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd806234d0 0x55fd809f7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:04.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.014+0000 7f4c5a59c640 1 -- 172.21.15.5:0/522319561 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd80910ed0 con 0x55fd80602d80 2024-05-09T01:01:04.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.014+0000 7f4c5a59c640 1 --2- 172.21.15.5:0/522319561 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd80602d80 0x55fd809f6950 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c009230 tx=0x7f4c4c009650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14eafca74b371607 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:01:04.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.014+0000 7f4c41ffb640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-09T01:01:04.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.015+0000 7f4c41ffb640 1 -- 172.21.15.5:0/522319561 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c053070 con 0x55fd80602d80 2024-05-09T01:01:04.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.015+0000 7f4c67f70740 1 -- 172.21.15.5:0/522319561 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c4c04b070 con 0x55fd80602d80 2024-05-09T01:01:04.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.015+0000 7f4c41ffb640 1 -- 172.21.15.5:0/522319561 <== mon.2 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f4c4c004230 con 0x55fd80602d80 2024-05-09T01:01:04.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.015+0000 7f4c41ffb640 1 -- 172.21.15.5:0/522319561 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4c4c05e050 con 0x55fd80602d80 2024-05-09T01:01:04.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.015+0000 7f4c67f70740 10 client.26570 fetch_fsmap learned FSMap version 32 2024-05-09T01:01:04.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.015+0000 7f4c67f70740 1 -- 172.21.15.5:0/522319561 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55fd809f9440 con 0x55fd80602d80 2024-05-09T01:01:04.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.015+0000 7f4c41ffb640 1 -- 172.21.15.5:0/522319561 <== mon.2 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c4c04b070 con 0x55fd80602d80 2024-05-09T01:01:04.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.015+0000 7f4c41ffb640 1 -- 172.21.15.5:0/522319561 <== mon.2 v2:172.21.15.131:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f4c4c03a040 con 0x55fd80602d80 2024-05-09T01:01:04.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.015+0000 7f4c67f70740 10 client.26570 fetch_fsmap finished waiting for FSMap version 32 2024-05-09T01:01:04.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.015+0000 7f4c67f70740 10 client.26570 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T01:01:04.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.015+0000 7f4c67f70740 20 client.26570 populate_metadata read hostname 'smithi005' 2024-05-09T01:01:04.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.015+0000 7f4c67f70740 1 --2- 172.21.15.5:0/522319561 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55fd806441b0 0x55fd80a69d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:04.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.015+0000 7f4c67f70740 4 client.26570 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T01:01:04.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.015+0000 7f4c67f70740 1 -- 172.21.15.5:0/522319561 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fd8095a3d0 con 0x55fd806441b0 2024-05-09T01:01:04.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.016+0000 7f4c59d9b640 1 --2- 172.21.15.5:0/522319561 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55fd806441b0 0x55fd80a69d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:04.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.016+0000 7f4c59d9b640 1 --2- 172.21.15.5:0/522319561 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55fd806441b0 0x55fd80a69d20 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdef8b047b655369 server_cookie=fddfe3e9745c348b in_seq=0 out_seq=0 2024-05-09T01:01:04.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:04.016+0000 7f4c41ffb640 10 client.26570 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T01:01:04.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:01:04 smithi131 ceph-mon[34547]: pgmap v812: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 71 MiB/s wr, 593 op/s 2024-05-09T01:01:04.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:01:04 smithi157 ceph-mon[34294]: pgmap v812: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 71 MiB/s wr, 593 op/s 2024-05-09T01:01:04.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:01:04 smithi005 ceph-mon[30365]: pgmap v812: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 71 MiB/s wr, 593 op/s 2024-05-09T01:01:06.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:01:06 smithi131 ceph-mon[34547]: pgmap v813: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 74 MiB/s wr, 668 op/s 2024-05-09T01:01:06.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:01:06 smithi157 ceph-mon[34294]: pgmap v813: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 74 MiB/s wr, 668 op/s 2024-05-09T01:01:06.454 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:06.453+0000 7f4c41ffb640 1 -- 172.21.15.5:0/522319561 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55fd8095a3d0 con 0x55fd806441b0 2024-05-09T01:01:06.454 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:06.453+0000 7f4c41ffb640 10 client.26570 handle_command_reply: tid=0 2024-05-09T01:01:06.454 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:06.453+0000 7f4c67f70740 1 client.26570 shutdown 2024-05-09T01:01:06.454 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:06.453+0000 7f4c67f70740 1 -- 172.21.15.5:0/522319561 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd80602d80 msgr2=0x55fd809f6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:06.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:06.453+0000 7f4c67f70740 1 --2- 172.21.15.5:0/522319561 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd80602d80 0x55fd809f6950 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c009230 tx=0x7f4c4c009650 comp rx=0 tx=0).stop 2024-05-09T01:01:06.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:06.453+0000 7f4c67f70740 1 -- 172.21.15.5:0/522319561 shutdown_connections 2024-05-09T01:01:06.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:06.453+0000 7f4c67f70740 1 --2- 172.21.15.5:0/522319561 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55fd806441b0 0x55fd80a69d20 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:06.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:06.453+0000 7f4c67f70740 1 --2- 172.21.15.5:0/522319561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd806234d0 0x55fd809f7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:06.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:06.453+0000 7f4c67f70740 1 --2- 172.21.15.5:0/522319561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd8061b2f0 0x55fd809f6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:06.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:06.453+0000 7f4c41ffb640 0 client.26570 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T01:01:06.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:06.453+0000 7f4c67f70740 1 --2- 172.21.15.5:0/522319561 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fd80602d80 0x55fd809f6950 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:06.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:06.453+0000 7f4c67f70740 1 -- 172.21.15.5:0/522319561 >> 172.21.15.5:0/522319561 conn(0x55fd80638300 msgr2=0x55fd809e8480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:01:06.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:06.453+0000 7f4c67f70740 1 -- 172.21.15.5:0/522319561 shutdown_connections 2024-05-09T01:01:06.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:06.453+0000 7f4c67f70740 1 -- 172.21.15.5:0/522319561 wait complete. 2024-05-09T01:01:06.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:06.454+0000 7f4c67f70740 20 client.26570 trim_cache size 0 max 16384 2024-05-09T01:01:06.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-09T01:01:06.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-09T01:01:06.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-09T01:01:06.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:06.454+0000 7f4c60fe8640 1 -- 172.21.15.5:0/814332526 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f4c480b29b0 msgr2=0x7f4c480b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:06.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:06.454+0000 7f4c60fe8640 1 --2- 172.21.15.5:0/814332526 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f4c480b29b0 0x7f4c480b4ec0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4c54009500 tx=0x7f4c5400b040 comp rx=0 tx=0).stop 2024-05-09T01:01:06.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:06.454+0000 7f4c60fe8640 1 -- 172.21.15.5:0/814332526 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c5c102b50 msgr2=0x7f4c5c106ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:06.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:06.454+0000 7f4c60fe8640 1 --2- 172.21.15.5:0/814332526 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c5c102b50 0x7f4c5c106ba0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4c4400e9f0 tx=0x7f4c4400eeb0 comp rx=0 tx=0).stop 2024-05-09T01:01:06.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:06.454+0000 7f4c60fe8640 1 -- 172.21.15.5:0/814332526 shutdown_connections 2024-05-09T01:01:06.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:06.454+0000 7f4c60fe8640 1 --2- 172.21.15.5:0/814332526 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f4c480b29b0 0x7f4c480b4ec0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:06.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:06.455+0000 7f4c60fe8640 1 --2- 172.21.15.5:0/814332526 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c5c103d90 0x7f4c5c1070e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:06.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:06.455+0000 7f4c60fe8640 1 --2- 172.21.15.5:0/814332526 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c5c102b50 0x7f4c5c106ba0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:06.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:06.455+0000 7f4c60fe8640 1 --2- 172.21.15.5:0/814332526 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c5c06a4f0 0x7f4c5c108500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:06.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:06.455+0000 7f4c60fe8640 1 -- 172.21.15.5:0/814332526 >> 172.21.15.5:0/814332526 conn(0x7f4c5c0fdd40 msgr2=0x7f4c5c108c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:01:06.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:06.455+0000 7f4c60fe8640 1 -- 172.21.15.5:0/814332526 shutdown_connections 2024-05-09T01:01:06.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-09T01:01:06.455+0000 7f4c60fe8640 1 -- 172.21.15.5:0/814332526 wait complete. 2024-05-09T01:01:06.458 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:01:06 smithi005 ceph-mon[30365]: pgmap v813: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 74 MiB/s wr, 668 op/s 2024-05-09T01:01:06.461 INFO:tasks.workunit.client.0.smithi005.stdout:OK 2024-05-09T01:01:06.461 INFO:tasks.workunit.client.0.smithi005.stderr:+ echo OK 2024-05-09T01:01:08.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:01:08 smithi131 ceph-mon[34547]: scrub summary: active paths [/] 2024-05-09T01:01:08.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:01:08 smithi131 ceph-mon[34547]: scrub summary: idle+waiting paths [/] 2024-05-09T01:01:08.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:01:08 smithi131 ceph-mon[34547]: scrub summary: idle+waiting paths [/] 2024-05-09T01:01:08.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:01:08 smithi131 ceph-mon[34547]: pgmap v814: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 68 MiB/s wr, 656 op/s 2024-05-09T01:01:08.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:01:08 smithi157 ceph-mon[34294]: scrub summary: active paths [/] 2024-05-09T01:01:08.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:01:08 smithi157 ceph-mon[34294]: scrub summary: idle+waiting paths [/] 2024-05-09T01:01:08.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:01:08 smithi157 ceph-mon[34294]: scrub summary: idle+waiting paths [/] 2024-05-09T01:01:08.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:01:08 smithi157 ceph-mon[34294]: pgmap v814: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 68 MiB/s wr, 656 op/s 2024-05-09T01:01:08.446 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:01:08 smithi005 ceph-mon[30365]: scrub summary: active paths [/] 2024-05-09T01:01:08.446 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:01:08 smithi005 ceph-mon[30365]: scrub summary: idle+waiting paths [/] 2024-05-09T01:01:08.447 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:01:08 smithi005 ceph-mon[30365]: scrub summary: idle+waiting paths [/] 2024-05-09T01:01:08.447 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:01:08 smithi005 ceph-mon[30365]: pgmap v814: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 68 MiB/s wr, 656 op/s 2024-05-09T01:01:08.498 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-09T01:01:08.498 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-09T01:01:08.632 INFO:tasks.workunit:Stopping ['suites/iogen.sh'] on client.0... 2024-05-09T01:01:08.632 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-05-09T01:01:08.929 DEBUG:teuthology.parallel:result is None 2024-05-09T01:01:08.930 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-05-09T01:01:08.964 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-05-09T01:01:08.964 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-05-09T01:01:08.975 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi005.front.sepia.ceph.com 2024-05-09T01:01:09.012 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-05-09T01:01:09.023 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-05-09T01:01:10.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:01:10 smithi131 ceph-mon[34547]: pgmap v815: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 40 MiB/s rd, 66 MiB/s wr, 902 op/s 2024-05-09T01:01:10.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:01:10 smithi131 ceph-mon[34547]: MDS health message cleared (mds.?): Client smithi005:0 failing to advance its oldest client/flush tid. 2024-05-09T01:01:10.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:01:10 smithi157 ceph-mon[34294]: pgmap v815: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 40 MiB/s rd, 66 MiB/s wr, 902 op/s 2024-05-09T01:01:10.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:01:10 smithi157 ceph-mon[34294]: MDS health message cleared (mds.?): Client smithi005:0 failing to advance its oldest client/flush tid. 2024-05-09T01:01:10.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:01:10 smithi005 ceph-mon[30365]: pgmap v815: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 40 MiB/s rd, 66 MiB/s wr, 902 op/s 2024-05-09T01:01:10.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:01:10 smithi005 ceph-mon[30365]: MDS health message cleared (mds.?): Client smithi005:0 failing to advance its oldest client/flush tid. 2024-05-09T01:01:11.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:01:11 smithi131 ceph-mon[34547]: Health check cleared: MDS_CLIENT_OLDEST_TID (was: 1 clients failing to advance oldest client/flush tid) 2024-05-09T01:01:11.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:01:11 smithi131 ceph-mon[34547]: Cluster is now healthy 2024-05-09T01:01:11.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:01:11 smithi131 ceph-mon[34547]: mds.? [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] up:active 2024-05-09T01:01:11.356 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:01:11 smithi131 ceph-mon[34547]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T01:01:11.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:01:11 smithi157 ceph-mon[34294]: Health check cleared: MDS_CLIENT_OLDEST_TID (was: 1 clients failing to advance oldest client/flush tid) 2024-05-09T01:01:11.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:01:11 smithi157 ceph-mon[34294]: Cluster is now healthy 2024-05-09T01:01:11.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:01:11 smithi157 ceph-mon[34294]: mds.? [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] up:active 2024-05-09T01:01:11.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:01:11 smithi157 ceph-mon[34294]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T01:01:11.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:01:11 smithi005 ceph-mon[30365]: Health check cleared: MDS_CLIENT_OLDEST_TID (was: 1 clients failing to advance oldest client/flush tid) 2024-05-09T01:01:11.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:01:11 smithi005 ceph-mon[30365]: Cluster is now healthy 2024-05-09T01:01:11.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:01:11 smithi005 ceph-mon[30365]: mds.? [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] up:active 2024-05-09T01:01:11.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:01:11 smithi005 ceph-mon[30365]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T01:01:12.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:01:12 smithi157 ceph-mon[34294]: pgmap v816: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 92 MiB/s rd, 49 MiB/s wr, 1.10k op/s 2024-05-09T01:01:12.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:01:12 smithi157 ceph-mon[34294]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T01:01:12.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:01:12 smithi005 ceph-mon[30365]: pgmap v816: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 92 MiB/s rd, 49 MiB/s wr, 1.10k op/s 2024-05-09T01:01:12.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:01:12 smithi005 ceph-mon[30365]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T01:01:12.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:01:12 smithi131 ceph-mon[34547]: pgmap v816: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 92 MiB/s rd, 49 MiB/s wr, 1.10k op/s 2024-05-09T01:01:12.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:01:12 smithi131 ceph-mon[34547]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T01:01:14.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:01:14 smithi157 ceph-mon[34294]: pgmap v817: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 120 MiB/s rd, 47 MiB/s wr, 1.19k op/s 2024-05-09T01:01:14.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:01:14 smithi005 ceph-mon[30365]: pgmap v817: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 120 MiB/s rd, 47 MiB/s wr, 1.19k op/s 2024-05-09T01:01:14.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:01:14 smithi131 ceph-mon[34547]: pgmap v817: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 120 MiB/s rd, 47 MiB/s wr, 1.19k op/s 2024-05-09T01:01:16.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:01:16 smithi157 ceph-mon[34294]: pgmap v818: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 127 MiB/s rd, 49 MiB/s wr, 1.19k op/s 2024-05-09T01:01:16.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:01:16 smithi005 ceph-mon[30365]: pgmap v818: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 127 MiB/s rd, 49 MiB/s wr, 1.19k op/s 2024-05-09T01:01:16.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:01:16 smithi131 ceph-mon[34547]: pgmap v818: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 127 MiB/s rd, 49 MiB/s wr, 1.19k op/s 2024-05-09T01:01:17.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:01:17 smithi157 ceph-mon[34294]: scrub summary: idle 2024-05-09T01:01:17.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:01:17 smithi157 ceph-mon[34294]: scrub summary: idle 2024-05-09T01:01:17.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:01:17 smithi005 ceph-mon[30365]: scrub summary: idle 2024-05-09T01:01:17.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:01:17 smithi005 ceph-mon[30365]: scrub summary: idle 2024-05-09T01:01:17.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:01:17 smithi131 ceph-mon[34547]: scrub summary: idle 2024-05-09T01:01:17.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:01:17 smithi131 ceph-mon[34547]: scrub summary: idle 2024-05-09T01:01:18.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:01:18 smithi157 ceph-mon[34294]: pgmap v819: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 127 MiB/s rd, 45 MiB/s wr, 1.08k op/s 2024-05-09T01:01:18.525 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:01:18 smithi005 ceph-mon[30365]: pgmap v819: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 127 MiB/s rd, 45 MiB/s wr, 1.08k op/s 2024-05-09T01:01:18.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:01:18 smithi131 ceph-mon[34547]: pgmap v819: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 127 MiB/s rd, 45 MiB/s wr, 1.08k op/s 2024-05-09T01:01:20.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:01:20 smithi157 ceph-mon[34294]: pgmap v820: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 126 MiB/s rd, 39 MiB/s wr, 1.01k op/s 2024-05-09T01:01:20.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:01:20 smithi005 ceph-mon[30365]: pgmap v820: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 126 MiB/s rd, 39 MiB/s wr, 1.01k op/s 2024-05-09T01:01:20.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:01:20 smithi131 ceph-mon[34547]: pgmap v820: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 126 MiB/s rd, 39 MiB/s wr, 1.01k op/s 2024-05-09T01:01:22.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:01:22 smithi157 ceph-mon[34294]: pgmap v821: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 89 MiB/s rd, 30 MiB/s wr, 682 op/s 2024-05-09T01:01:22.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:01:22 smithi005 ceph-mon[30365]: pgmap v821: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 89 MiB/s rd, 30 MiB/s wr, 682 op/s 2024-05-09T01:01:22.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:01:22 smithi131 ceph-mon[34547]: pgmap v821: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 89 MiB/s rd, 30 MiB/s wr, 682 op/s 2024-05-09T01:01:22.846 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T01:01:23.115 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:01:24.424 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:01:24 smithi005 ceph-mon[30365]: pgmap v822: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 36 MiB/s rd, 37 MiB/s wr, 398 op/s 2024-05-09T01:01:24.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:01:24 smithi157 ceph-mon[34294]: pgmap v822: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 36 MiB/s rd, 37 MiB/s wr, 398 op/s 2024-05-09T01:01:24.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:01:24 smithi131 ceph-mon[34547]: pgmap v822: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 36 MiB/s rd, 37 MiB/s wr, 398 op/s 2024-05-09T01:01:25.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.128+0000 7f207a983700 1 -- 172.21.15.5:0/1927341276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2074107e10 msgr2=0x7f2074108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:25.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.128+0000 7f207a983700 1 --2- 172.21.15.5:0/1927341276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2074107e10 0x7f2074108220 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f205c0099c0 tx=0x7f205c009cd0 comp rx=0 tx=0).stop 2024-05-09T01:01:25.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.129+0000 7f207a983700 1 -- 172.21.15.5:0/1927341276 shutdown_connections 2024-05-09T01:01:25.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.129+0000 7f207a983700 1 --2- 172.21.15.5:0/1927341276 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2074108760 0x7f20741100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:25.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.129+0000 7f207a983700 1 --2- 172.21.15.5:0/1927341276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20741034e0 0x7f20741038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:25.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.129+0000 7f207a983700 1 --2- 172.21.15.5:0/1927341276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2074107e10 0x7f2074108220 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:25.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.129+0000 7f207a983700 1 -- 172.21.15.5:0/1927341276 >> 172.21.15.5:0/1927341276 conn(0x7f20740fef30 msgr2=0x7f2074101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:01:25.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.129+0000 7f207a983700 1 -- 172.21.15.5:0/1927341276 shutdown_connections 2024-05-09T01:01:25.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.129+0000 7f207a983700 1 -- 172.21.15.5:0/1927341276 wait complete. 2024-05-09T01:01:25.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.129+0000 7f207a983700 1 Processor -- start 2024-05-09T01:01:25.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.130+0000 7f207a983700 1 -- start start 2024-05-09T01:01:25.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.130+0000 7f207a983700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20741034e0 0x7f207419af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:25.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.130+0000 7f207a983700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2074107e10 0x7f207419b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:25.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.130+0000 7f207a983700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2074108760 0x7f207419f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:25.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.130+0000 7f207a983700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2074113460 con 0x7f20741034e0 2024-05-09T01:01:25.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.130+0000 7f207a983700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f20741132e0 con 0x7f2074107e10 2024-05-09T01:01:25.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.130+0000 7f207a983700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f20741135e0 con 0x7f2074108760 2024-05-09T01:01:25.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.130+0000 7f2073fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20741034e0 0x7f207419af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:25.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.130+0000 7f2073fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20741034e0 0x7f207419af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41602/0 (socket says 172.21.15.5:41602) 2024-05-09T01:01:25.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.130+0000 7f2073fff700 1 -- 172.21.15.5:0/3360705598 learned_addr learned my addr 172.21.15.5:0/3360705598 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:01:25.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.130+0000 7f2078f20700 1 --2- 172.21.15.5:0/3360705598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2074108760 0x7f207419f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:25.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.130+0000 7f206bfff700 1 --2- 172.21.15.5:0/3360705598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2074107e10 0x7f207419b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:25.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.130+0000 7f2073fff700 1 -- 172.21.15.5:0/3360705598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2074108760 msgr2=0x7f207419f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:25.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.130+0000 7f2073fff700 1 --2- 172.21.15.5:0/3360705598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2074108760 0x7f207419f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:25.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.130+0000 7f2073fff700 1 -- 172.21.15.5:0/3360705598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2074107e10 msgr2=0x7f207419b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:25.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.130+0000 7f2073fff700 1 --2- 172.21.15.5:0/3360705598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2074107e10 0x7f207419b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:25.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.130+0000 7f2073fff700 1 -- 172.21.15.5:0/3360705598 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f207419fdd0 con 0x7f20741034e0 2024-05-09T01:01:25.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.131+0000 7f2073fff700 1 --2- 172.21.15.5:0/3360705598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20741034e0 0x7f207419af00 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f206400b530 tx=0x7f206400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:01:25.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.131+0000 7f2071ffb700 1 -- 172.21.15.5:0/3360705598 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2064015020 con 0x7f20741034e0 2024-05-09T01:01:25.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.131+0000 7f207a983700 1 -- 172.21.15.5:0/3360705598 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20741a00c0 con 0x7f20741034e0 2024-05-09T01:01:25.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.131+0000 7f2071ffb700 1 -- 172.21.15.5:0/3360705598 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2064011ce0 con 0x7f20741034e0 2024-05-09T01:01:25.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.131+0000 7f2071ffb700 1 -- 172.21.15.5:0/3360705598 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2064010dd0 con 0x7f20741034e0 2024-05-09T01:01:25.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.131+0000 7f207a983700 1 -- 172.21.15.5:0/3360705598 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20741a79e0 con 0x7f20741034e0 2024-05-09T01:01:25.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.134+0000 7f2071ffb700 1 -- 172.21.15.5:0/3360705598 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f2064018020 con 0x7f20741034e0 2024-05-09T01:01:25.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.135+0000 7f2071ffb700 1 --2- 172.21.15.5:0/3360705598 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f20600b2880 0x7f20600b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:25.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.135+0000 7f206bfff700 1 --2- 172.21.15.5:0/3360705598 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f20600b2880 0x7f20600b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:25.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.135+0000 7f2071ffb700 1 -- 172.21.15.5:0/3360705598 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f20640e1900 con 0x7f20741034e0 2024-05-09T01:01:25.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.135+0000 7f2089399b80 1 Processor -- start 2024-05-09T01:01:25.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.135+0000 7f2089399b80 1 -- start start 2024-05-09T01:01:25.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.136+0000 7f206bfff700 1 --2- 172.21.15.5:0/3360705598 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f20600b2880 0x7f20600b4d90 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f205c006010 tx=0x7f205c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:01:25.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.136+0000 7f2089399b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2b9fdad00 0x55d2ba0e7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:25.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.136+0000 7f2089399b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2b9f604a0 0x55d2ba0e7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:25.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.136+0000 7f2089399b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2ba0e1810 0x55d2ba0e1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:25.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.136+0000 7f2089399b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d2ba0000a0 con 0x55d2b9fdad00 2024-05-09T01:01:25.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.136+0000 7f2089399b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55d2b9fa59a0 con 0x55d2b9f604a0 2024-05-09T01:01:25.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.136+0000 7f2089399b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d2b9ffc060 con 0x55d2ba0e1810 2024-05-09T01:01:25.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.136+0000 7f2073fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2b9fdad00 0x55d2ba0e7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:25.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.136+0000 7f2073fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2b9fdad00 0x55d2ba0e7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41618/0 (socket says 172.21.15.5:41618) 2024-05-09T01:01:25.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.136+0000 7f2073fff700 1 -- 172.21.15.5:0/436107947 learned_addr learned my addr 172.21.15.5:0/436107947 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:01:25.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.136+0000 7f206bfff700 1 --2- 172.21.15.5:0/436107947 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2b9f604a0 0x55d2ba0e7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:25.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.136+0000 7f2078f20700 1 --2- 172.21.15.5:0/436107947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2ba0e1810 0x55d2ba0e1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:25.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.136+0000 7f2073fff700 1 -- 172.21.15.5:0/436107947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2ba0e1810 msgr2=0x55d2ba0e1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:01:25.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.136+0000 7f2073fff700 1 --2- 172.21.15.5:0/436107947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2ba0e1810 0x55d2ba0e1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:25.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.136+0000 7f2073fff700 1 -- 172.21.15.5:0/436107947 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2b9f604a0 msgr2=0x55d2ba0e7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:01:25.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.136+0000 7f2073fff700 1 --2- 172.21.15.5:0/436107947 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2b9f604a0 0x55d2ba0e7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:25.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.136+0000 7f2073fff700 1 -- 172.21.15.5:0/436107947 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2b9f96500 con 0x55d2b9fdad00 2024-05-09T01:01:25.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.136+0000 7f206bfff700 1 --2- 172.21.15.5:0/436107947 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2b9f604a0 0x55d2ba0e7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:01:25.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.136+0000 7f2073fff700 1 --2- 172.21.15.5:0/436107947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2b9fdad00 0x55d2ba0e7650 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f20640094b0 tx=0x7f20640a4ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bb4487ae33494af server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:01:25.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.137+0000 7f206a7fc700 1 -- 172.21.15.5:0/436107947 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20640a95f0 con 0x55d2b9fdad00 2024-05-09T01:01:25.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.137+0000 7f206a7fc700 1 -- 172.21.15.5:0/436107947 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20640a8ef0 con 0x55d2b9fdad00 2024-05-09T01:01:25.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.137+0000 7f206a7fc700 1 -- 172.21.15.5:0/436107947 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20640a8150 con 0x55d2b9fdad00 2024-05-09T01:01:25.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.138+0000 7f2089399b80 1 -- 172.21.15.5:0/436107947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2b9fdad00 msgr2=0x55d2ba0e7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:25.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.138+0000 7f2089399b80 1 --2- 172.21.15.5:0/436107947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2b9fdad00 0x55d2ba0e7650 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f20640094b0 tx=0x7f20640a4ec0 comp rx=0 tx=0).stop 2024-05-09T01:01:25.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.138+0000 7f2089399b80 1 -- 172.21.15.5:0/436107947 shutdown_connections 2024-05-09T01:01:25.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.138+0000 7f2089399b80 1 --2- 172.21.15.5:0/436107947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2ba0e1810 0x55d2ba0e1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:25.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.138+0000 7f2089399b80 1 --2- 172.21.15.5:0/436107947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2b9fdad00 0x55d2ba0e7650 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:25.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.138+0000 7f2089399b80 1 --2- 172.21.15.5:0/436107947 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2b9f604a0 0x55d2ba0e7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:25.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.138+0000 7f2089399b80 1 -- 172.21.15.5:0/436107947 >> 172.21.15.5:0/436107947 conn(0x55d2b9f63f70 msgr2=0x55d2ba0d8220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:01:25.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.138+0000 7f2089399b80 1 -- 172.21.15.5:0/436107947 shutdown_connections 2024-05-09T01:01:25.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.138+0000 7f2089399b80 1 -- 172.21.15.5:0/436107947 wait complete. 2024-05-09T01:01:25.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.138+0000 7f2089399b80 1 -- start start 2024-05-09T01:01:25.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.139+0000 7f2089399b80 1 Processor -- start 2024-05-09T01:01:25.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.139+0000 7f2089399b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2b9f604a0 0x55d2ba0eb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:25.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.139+0000 7f2089399b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2b9fdad00 0x55d2ba0eb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:25.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.139+0000 7f2089399b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2ba0e1810 0x55d2ba0ebed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:25.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.139+0000 7f2089399b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d2ba0e2570 con 0x55d2b9fdad00 2024-05-09T01:01:25.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.139+0000 7f2089399b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55d2b9f9ab20 con 0x55d2ba0e1810 2024-05-09T01:01:25.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.139+0000 7f2089399b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d2ba0ee430 con 0x55d2b9f604a0 2024-05-09T01:01:25.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.139+0000 7f2089399b80 10 client.0 _ll_register_callbacks cb 0x55d2b9e716c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:01:25.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.139+0000 7f206bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2b9fdad00 0x55d2ba0eb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:25.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.139+0000 7f206bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2b9fdad00 0x55d2ba0eb990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41626/0 (socket says 172.21.15.5:41626) 2024-05-09T01:01:25.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.139+0000 7f206bfff700 1 -- 172.21.15.5:0/2253019924 learned_addr learned my addr 172.21.15.5:0/2253019924 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:01:25.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.139+0000 7f2073fff700 1 --2- 172.21.15.5:0/2253019924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2b9f604a0 0x55d2ba0eb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:25.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.139+0000 7f2078f20700 1 --2- 172.21.15.5:0/2253019924 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2ba0e1810 0x55d2ba0ebed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:25.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.140+0000 7f2073fff700 1 -- 172.21.15.5:0/2253019924 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2ba0e1810 msgr2=0x55d2ba0ebed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:01:25.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.140+0000 7f2073fff700 1 --2- 172.21.15.5:0/2253019924 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2ba0e1810 0x55d2ba0ebed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:25.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.140+0000 7f2073fff700 1 -- 172.21.15.5:0/2253019924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2b9fdad00 msgr2=0x55d2ba0eb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:01:25.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.140+0000 7f2073fff700 1 --2- 172.21.15.5:0/2253019924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2b9fdad00 0x55d2ba0eb990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:25.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.140+0000 7f2073fff700 1 -- 172.21.15.5:0/2253019924 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2b9e7b490 con 0x55d2b9f604a0 2024-05-09T01:01:25.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.140+0000 7f2073fff700 1 --2- 172.21.15.5:0/2253019924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2b9f604a0 0x55d2ba0eb450 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x55d2ba0e21d0 tx=0x7f20640106d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2a5591ec0696466 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:01:25.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.140+0000 7f20537fe700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-09T01:01:25.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.140+0000 7f20537fe700 1 -- 172.21.15.5:0/2253019924 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f206406e070 con 0x55d2b9f604a0 2024-05-09T01:01:25.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.140+0000 7f2089399b80 1 -- 172.21.15.5:0/2253019924 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20640a8ef0 con 0x55d2b9f604a0 2024-05-09T01:01:25.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.140+0000 7f20537fe700 1 -- 172.21.15.5:0/2253019924 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f20640b1ad0 con 0x55d2b9f604a0 2024-05-09T01:01:25.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.140+0000 7f20537fe700 1 -- 172.21.15.5:0/2253019924 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f20640a95f0 con 0x55d2b9f604a0 2024-05-09T01:01:25.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.140+0000 7f2089399b80 10 client.26569 fetch_fsmap learned FSMap version 34 2024-05-09T01:01:25.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.140+0000 7f2089399b80 1 -- 172.21.15.5:0/2253019924 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d2ba0f8e20 con 0x55d2b9f604a0 2024-05-09T01:01:25.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.141+0000 7f20537fe700 1 -- 172.21.15.5:0/2253019924 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20640a8ef0 con 0x55d2b9f604a0 2024-05-09T01:01:25.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.141+0000 7f20537fe700 1 -- 172.21.15.5:0/2253019924 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7f20640a7bc0 con 0x55d2b9f604a0 2024-05-09T01:01:25.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.141+0000 7f2089399b80 10 client.26569 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T01:01:25.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.141+0000 7f2089399b80 10 client.26569 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T01:01:25.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.141+0000 7f2089399b80 20 client.26569 populate_metadata read hostname 'smithi005' 2024-05-09T01:01:25.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.141+0000 7f2089399b80 1 --2- 172.21.15.5:0/2253019924 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55d2ba0fb2f0 0x55d2ba0fd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:25.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.141+0000 7f2089399b80 4 client.26569 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:01:25.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.141+0000 7f2089399b80 1 -- 172.21.15.5:0/2253019924 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2ba00da80 con 0x55d2ba0fb2f0 2024-05-09T01:01:25.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.141+0000 7f206bfff700 1 --2- 172.21.15.5:0/2253019924 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55d2ba0fb2f0 0x55d2ba0fd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:25.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.142+0000 7f206bfff700 1 --2- 172.21.15.5:0/2253019924 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55d2ba0fb2f0 0x55d2ba0fd710 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7520c91a22642fe7 server_cookie=6f9a01739144409b in_seq=0 out_seq=0 2024-05-09T01:01:25.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.142+0000 7f20537fe700 10 client.26569 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T01:01:25.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.142+0000 7f20537fe700 1 -- 172.21.15.5:0/2253019924 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d2ba00da80 con 0x55d2ba0fb2f0 2024-05-09T01:01:25.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.142+0000 7f20537fe700 10 client.26569 handle_command_reply: tid=0 2024-05-09T01:01:25.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.142+0000 7f2089399b80 1 client.26569 shutdown 2024-05-09T01:01:25.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.143+0000 7f2089399b80 1 -- 172.21.15.5:0/2253019924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2b9f604a0 msgr2=0x55d2ba0eb450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:25.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.143+0000 7f2089399b80 1 --2- 172.21.15.5:0/2253019924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2b9f604a0 0x55d2ba0eb450 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x55d2ba0e21d0 tx=0x7f20640106d0 comp rx=0 tx=0).stop 2024-05-09T01:01:25.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.143+0000 7f2089399b80 1 -- 172.21.15.5:0/2253019924 shutdown_connections 2024-05-09T01:01:25.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.143+0000 7f2089399b80 1 --2- 172.21.15.5:0/2253019924 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55d2ba0fb2f0 0x55d2ba0fd710 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:25.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.143+0000 7f2089399b80 1 --2- 172.21.15.5:0/2253019924 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2ba0e1810 0x55d2ba0ebed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:25.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.143+0000 7f2089399b80 1 --2- 172.21.15.5:0/2253019924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2b9f604a0 0x55d2ba0eb450 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:25.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.143+0000 7f2089399b80 1 --2- 172.21.15.5:0/2253019924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2b9fdad00 0x55d2ba0eb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:25.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.143+0000 7f20537fe700 0 client.26569 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T01:01:25.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.143+0000 7f2089399b80 1 -- 172.21.15.5:0/2253019924 >> 172.21.15.5:0/2253019924 conn(0x55d2b9f63f70 msgr2=0x55d2ba0d1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:01:25.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.143+0000 7f2089399b80 1 -- 172.21.15.5:0/2253019924 shutdown_connections 2024-05-09T01:01:25.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.143+0000 7f2089399b80 1 -- 172.21.15.5:0/2253019924 wait complete. 2024-05-09T01:01:25.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.143+0000 7f2089399b80 20 client.26569 trim_cache size 0 max 16384 2024-05-09T01:01:25.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.163+0000 7f2089399b80 1 Processor -- start 2024-05-09T01:01:25.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.163+0000 7f2089399b80 1 -- start start 2024-05-09T01:01:25.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.163+0000 7f2089399b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2b9f604a0 0x55d2ba0ea410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:25.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.163+0000 7f2089399b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2b9fdad00 0x55d2ba0d2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:25.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.163+0000 7f2089399b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2ba0e1810 0x55d2ba0ed8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:25.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.163+0000 7f2089399b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f20741acc20 con 0x55d2ba0e1810 2024-05-09T01:01:25.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.163+0000 7f2089399b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f20741ae790 con 0x55d2b9f604a0 2024-05-09T01:01:25.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.163+0000 7f2089399b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f20741ad2f0 con 0x55d2b9fdad00 2024-05-09T01:01:25.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.163+0000 7f2078f20700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2ba0e1810 0x55d2ba0ed8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:25.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.163+0000 7f2078f20700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2ba0e1810 0x55d2ba0ed8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41640/0 (socket says 172.21.15.5:41640) 2024-05-09T01:01:25.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.163+0000 7f2078f20700 1 -- 172.21.15.5:0/15837195 learned_addr learned my addr 172.21.15.5:0/15837195 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:01:25.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.164+0000 7f2073fff700 1 --2- 172.21.15.5:0/15837195 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2b9f604a0 0x55d2ba0ea410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:25.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.164+0000 7f2078f20700 1 -- 172.21.15.5:0/15837195 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2b9fdad00 msgr2=0x55d2ba0d2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:01:25.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.164+0000 7f206bfff700 1 --2- 172.21.15.5:0/15837195 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2b9fdad00 0x55d2ba0d2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:25.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.164+0000 7f2078f20700 1 --2- 172.21.15.5:0/15837195 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2b9fdad00 0x55d2ba0d2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:25.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.164+0000 7f2078f20700 1 -- 172.21.15.5:0/15837195 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2b9f604a0 msgr2=0x55d2ba0ea410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:01:25.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.164+0000 7f2078f20700 1 --2- 172.21.15.5:0/15837195 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2b9f604a0 0x55d2ba0ea410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:25.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.164+0000 7f2078f20700 1 -- 172.21.15.5:0/15837195 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2ba0d4510 con 0x55d2ba0e1810 2024-05-09T01:01:25.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.164+0000 7f2073fff700 1 --2- 172.21.15.5:0/15837195 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2b9f604a0 0x55d2ba0ea410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:01:25.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.164+0000 7f206bfff700 1 --2- 172.21.15.5:0/15837195 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2b9fdad00 0x55d2ba0d2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:01:25.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.164+0000 7f2078f20700 1 --2- 172.21.15.5:0/15837195 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2ba0e1810 0x55d2ba0ed8f0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f206c00a5c0 tx=0x7f206c009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b964761bff8dad0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:01:25.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.164+0000 7f2069ffb700 1 -- 172.21.15.5:0/15837195 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f206c017830 con 0x55d2ba0e1810 2024-05-09T01:01:25.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.164+0000 7f2069ffb700 1 -- 172.21.15.5:0/15837195 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f206c01f400 con 0x55d2ba0e1810 2024-05-09T01:01:25.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.164+0000 7f2069ffb700 1 -- 172.21.15.5:0/15837195 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f206c0165a0 con 0x55d2ba0e1810 2024-05-09T01:01:25.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.165+0000 7f2089399b80 1 -- 172.21.15.5:0/15837195 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2ba0e1810 msgr2=0x55d2ba0ed8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:25.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.165+0000 7f2089399b80 1 --2- 172.21.15.5:0/15837195 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2ba0e1810 0x55d2ba0ed8f0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f206c00a5c0 tx=0x7f206c009650 comp rx=0 tx=0).stop 2024-05-09T01:01:25.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.165+0000 7f2089399b80 1 -- 172.21.15.5:0/15837195 shutdown_connections 2024-05-09T01:01:25.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.165+0000 7f2089399b80 1 --2- 172.21.15.5:0/15837195 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2ba0e1810 0x55d2ba0ed8f0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:25.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.165+0000 7f2089399b80 1 --2- 172.21.15.5:0/15837195 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2b9f604a0 0x55d2ba0ea410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:25.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.165+0000 7f2089399b80 1 --2- 172.21.15.5:0/15837195 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2b9fdad00 0x55d2ba0d2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:25.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.165+0000 7f2089399b80 1 -- 172.21.15.5:0/15837195 >> 172.21.15.5:0/15837195 conn(0x55d2b9f63f70 msgr2=0x55d2ba0e0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:01:25.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.165+0000 7f2089399b80 1 -- 172.21.15.5:0/15837195 shutdown_connections 2024-05-09T01:01:25.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.165+0000 7f2089399b80 1 -- 172.21.15.5:0/15837195 wait complete. 2024-05-09T01:01:25.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.166+0000 7f2089399b80 1 -- start start 2024-05-09T01:01:25.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.166+0000 7f2089399b80 1 Processor -- start 2024-05-09T01:01:25.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.166+0000 7f2089399b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2b9f604a0 0x55d2ba100c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:25.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.166+0000 7f2089399b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2b9fdad00 0x55d2ba0fe000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:25.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.166+0000 7f2089399b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2ba0e1810 0x55d2ba0fe540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:25.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.166+0000 7f2089399b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f20741ae0c0 con 0x55d2b9f604a0 2024-05-09T01:01:25.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.166+0000 7f2089399b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f20741ad9c0 con 0x55d2b9fdad00 2024-05-09T01:01:25.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.166+0000 7f2089399b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f20741aabc0 con 0x55d2ba0e1810 2024-05-09T01:01:25.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.166+0000 7f2089399b80 10 client.0 _ll_register_callbacks cb 0x55d2b9e716c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:01:25.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.166+0000 7f2073fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2b9f604a0 0x55d2ba100c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:25.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.166+0000 7f2073fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2b9f604a0 0x55d2ba100c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41656/0 (socket says 172.21.15.5:41656) 2024-05-09T01:01:25.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.166+0000 7f2073fff700 1 -- 172.21.15.5:0/2737155495 learned_addr learned my addr 172.21.15.5:0/2737155495 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:01:25.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.167+0000 7f206bfff700 1 --2- 172.21.15.5:0/2737155495 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2b9fdad00 0x55d2ba0fe000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:25.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.167+0000 7f2078f20700 1 --2- 172.21.15.5:0/2737155495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2ba0e1810 0x55d2ba0fe540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:25.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.167+0000 7f2073fff700 1 -- 172.21.15.5:0/2737155495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2ba0e1810 msgr2=0x55d2ba0fe540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:01:25.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.167+0000 7f2073fff700 1 --2- 172.21.15.5:0/2737155495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2ba0e1810 0x55d2ba0fe540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:25.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.167+0000 7f2073fff700 1 -- 172.21.15.5:0/2737155495 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2b9fdad00 msgr2=0x55d2ba0fe000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:01:25.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.167+0000 7f2073fff700 1 --2- 172.21.15.5:0/2737155495 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2b9fdad00 0x55d2ba0fe000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:25.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.167+0000 7f2073fff700 1 -- 172.21.15.5:0/2737155495 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2ba0fec80 con 0x55d2b9f604a0 2024-05-09T01:01:25.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.167+0000 7f2073fff700 1 --2- 172.21.15.5:0/2737155495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2b9f604a0 0x55d2ba100c80 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f20640a8e80 tx=0x7f20640a80a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1052deb52018b58 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:01:25.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.167+0000 7f20537fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T01:01:25.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.167+0000 7f20537fe700 1 -- 172.21.15.5:0/2737155495 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f206406e070 con 0x55d2b9f604a0 2024-05-09T01:01:25.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.167+0000 7f20537fe700 1 -- 172.21.15.5:0/2737155495 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f20640a54e0 con 0x55d2b9f604a0 2024-05-09T01:01:25.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.167+0000 7f2089399b80 1 -- 172.21.15.5:0/2737155495 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f206c01f400 con 0x55d2b9f604a0 2024-05-09T01:01:25.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.167+0000 7f20537fe700 1 -- 172.21.15.5:0/2737155495 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f20640a8ef0 con 0x55d2b9f604a0 2024-05-09T01:01:25.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.167+0000 7f2089399b80 10 client.17187 fetch_fsmap learned FSMap version 34 2024-05-09T01:01:25.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.167+0000 7f2089399b80 1 -- 172.21.15.5:0/2737155495 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d2ba0fec80 con 0x55d2b9f604a0 2024-05-09T01:01:25.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.168+0000 7f20537fe700 1 -- 172.21.15.5:0/2737155495 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f206c01f400 con 0x55d2b9f604a0 2024-05-09T01:01:25.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.168+0000 7f20537fe700 1 -- 172.21.15.5:0/2737155495 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7f20640a4860 con 0x55d2b9f604a0 2024-05-09T01:01:25.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.168+0000 7f2089399b80 10 client.17187 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T01:01:25.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.168+0000 7f2089399b80 10 client.17187 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T01:01:25.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.168+0000 7f2089399b80 20 client.17187 populate_metadata read hostname 'smithi005' 2024-05-09T01:01:25.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.168+0000 7f2089399b80 1 --2- 172.21.15.5:0/2737155495 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55d2ba165870 0x55d2ba167c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:25.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.168+0000 7f2089399b80 4 client.17187 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T01:01:25.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.168+0000 7f2089399b80 1 -- 172.21.15.5:0/2737155495 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d2ba007b20 con 0x55d2ba165870 2024-05-09T01:01:25.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.168+0000 7f206bfff700 1 --2- 172.21.15.5:0/2737155495 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55d2ba165870 0x55d2ba167c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:25.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.169+0000 7f20537fe700 10 client.17187 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T01:01:25.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.169+0000 7f206bfff700 1 --2- 172.21.15.5:0/2737155495 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55d2ba165870 0x55d2ba167c90 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a69ba7ae4fcf3a62 server_cookie=b2342afc351624d3 in_seq=0 out_seq=0 2024-05-09T01:01:25.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.169+0000 7f20537fe700 1 -- 172.21.15.5:0/2737155495 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d2ba007b20 con 0x55d2ba165870 2024-05-09T01:01:25.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.169+0000 7f20537fe700 10 client.17187 handle_command_reply: tid=0 2024-05-09T01:01:25.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.169+0000 7f2089399b80 1 client.17187 shutdown 2024-05-09T01:01:25.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.169+0000 7f2089399b80 1 -- 172.21.15.5:0/2737155495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2b9f604a0 msgr2=0x55d2ba100c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:25.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.169+0000 7f2089399b80 1 --2- 172.21.15.5:0/2737155495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2b9f604a0 0x55d2ba100c80 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f20640a8e80 tx=0x7f20640a80a0 comp rx=0 tx=0).stop 2024-05-09T01:01:25.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.169+0000 7f2089399b80 1 -- 172.21.15.5:0/2737155495 shutdown_connections 2024-05-09T01:01:25.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.169+0000 7f2089399b80 1 --2- 172.21.15.5:0/2737155495 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x55d2ba165870 0x55d2ba167c90 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:25.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.169+0000 7f2089399b80 1 --2- 172.21.15.5:0/2737155495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2ba0e1810 0x55d2ba0fe540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:25.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.169+0000 7f2089399b80 1 --2- 172.21.15.5:0/2737155495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2b9f604a0 0x55d2ba100c80 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:25.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.169+0000 7f2089399b80 1 --2- 172.21.15.5:0/2737155495 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d2b9fdad00 0x55d2ba0fe000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:25.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.169+0000 7f2089399b80 1 -- 172.21.15.5:0/2737155495 >> 172.21.15.5:0/2737155495 conn(0x55d2b9f63f70 msgr2=0x55d2ba0e9950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:01:25.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.169+0000 7f20537fe700 0 client.17187 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T01:01:25.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.170+0000 7f2089399b80 1 -- 172.21.15.5:0/2737155495 shutdown_connections 2024-05-09T01:01:25.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.170+0000 7f2089399b80 1 -- 172.21.15.5:0/2737155495 wait complete. 2024-05-09T01:01:25.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.170+0000 7f2089399b80 20 client.17187 trim_cache size 0 max 16384 2024-05-09T01:01:25.221 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-09T01:01:25.221 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-05-09T01:01:25.221 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-05-09T01:01:25.221 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-09T01:01:25.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.170+0000 7f207a983700 1 -- 172.21.15.5:0/3360705598 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f20600b2880 msgr2=0x7f20600b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:25.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.170+0000 7f207a983700 1 --2- 172.21.15.5:0/3360705598 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f20600b2880 0x7f20600b4d90 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f205c006010 tx=0x7f205c000bc0 comp rx=0 tx=0).stop 2024-05-09T01:01:25.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.170+0000 7f207a983700 1 -- 172.21.15.5:0/3360705598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20741034e0 msgr2=0x7f207419af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:25.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.170+0000 7f207a983700 1 --2- 172.21.15.5:0/3360705598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20741034e0 0x7f207419af00 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f206400b530 tx=0x7f206400b840 comp rx=0 tx=0).stop 2024-05-09T01:01:25.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.170+0000 7f207a983700 1 -- 172.21.15.5:0/3360705598 shutdown_connections 2024-05-09T01:01:25.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.170+0000 7f207a983700 1 --2- 172.21.15.5:0/3360705598 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f20600b2880 0x7f20600b4d90 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:25.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.170+0000 7f207a983700 1 --2- 172.21.15.5:0/3360705598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2074108760 0x7f207419f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:25.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.170+0000 7f207a983700 1 --2- 172.21.15.5:0/3360705598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20741034e0 0x7f207419af00 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:25.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.170+0000 7f207a983700 1 --2- 172.21.15.5:0/3360705598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2074107e10 0x7f207419b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:25.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.170+0000 7f207a983700 1 -- 172.21.15.5:0/3360705598 >> 172.21.15.5:0/3360705598 conn(0x7f20740fef30 msgr2=0x7f207406c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:01:25.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.171+0000 7f207a983700 1 -- 172.21.15.5:0/3360705598 shutdown_connections 2024-05-09T01:01:25.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:25.171+0000 7f207a983700 1 -- 172.21.15.5:0/3360705598 wait complete. 2024-05-09T01:01:26.073 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T01:01:26.073 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T01:01:26.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:01:26 smithi005 ceph-mon[30365]: pgmap v823: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 39 MiB/s wr, 233 op/s 2024-05-09T01:01:26.334 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:01:26.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:01:26 smithi157 ceph-mon[34294]: pgmap v823: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 39 MiB/s wr, 233 op/s 2024-05-09T01:01:26.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:01:26 smithi131 ceph-mon[34547]: pgmap v823: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 39 MiB/s wr, 233 op/s 2024-05-09T01:01:28.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:28.144+0000 7faf26f50700 1 -- 172.21.15.5:0/518255975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf20108740 msgr2=0x7faf201100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:28.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:28.144+0000 7faf26f50700 1 --2- 172.21.15.5:0/518255975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf20108740 0x7faf201100a0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7faf1800b0d0 tx=0x7faf1800b3e0 comp rx=0 tx=0).stop 2024-05-09T01:01:28.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:28.144+0000 7faf26f50700 1 -- 172.21.15.5:0/518255975 shutdown_connections 2024-05-09T01:01:28.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:28.144+0000 7faf26f50700 1 --2- 172.21.15.5:0/518255975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf20108740 0x7faf201100a0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:28.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:28.144+0000 7faf26f50700 1 --2- 172.21.15.5:0/518255975 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf201034c0 0x7faf201038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:28.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:28.144+0000 7faf26f50700 1 --2- 172.21.15.5:0/518255975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf20107df0 0x7faf20108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:28.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:28.144+0000 7faf26f50700 1 -- 172.21.15.5:0/518255975 >> 172.21.15.5:0/518255975 conn(0x7faf200fedf0 msgr2=0x7faf20101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:01:28.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:28.144+0000 7faf26f50700 1 -- 172.21.15.5:0/518255975 shutdown_connections 2024-05-09T01:01:28.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:28.144+0000 7faf26f50700 1 -- 172.21.15.5:0/518255975 wait complete. 2024-05-09T01:01:28.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:28.145+0000 7faf26f50700 1 Processor -- start 2024-05-09T01:01:28.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:28.145+0000 7faf26f50700 1 -- start start 2024-05-09T01:01:28.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:28.145+0000 7faf26f50700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf201034c0 0x7faf2019add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:28.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:28.145+0000 7faf26f50700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf20107df0 0x7faf2019b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:28.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:28.145+0000 7faf26f50700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf20108740 0x7faf2019f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:28.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:28.145+0000 7faf26f50700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7faf20113370 con 0x7faf20107df0 2024-05-09T01:01:28.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:28.145+0000 7faf26f50700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7faf201131f0 con 0x7faf20108740 2024-05-09T01:01:28.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:28.145+0000 7faf26f50700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7faf201134f0 con 0x7faf201034c0 2024-05-09T01:01:28.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:28.145+0000 7faf1ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf20107df0 0x7faf2019b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:28.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:28.146+0000 7faf1ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf20107df0 0x7faf2019b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41670/0 (socket says 172.21.15.5:41670) 2024-05-09T01:01:28.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:28.146+0000 7faf1ffff700 1 -- 172.21.15.5:0/3083237102 learned_addr learned my addr 172.21.15.5:0/3083237102 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:01:28.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:28.146+0000 7faf254ed700 1 --2- 172.21.15.5:0/3083237102 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf20108740 0x7faf2019f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:28.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:28.146+0000 7faf24cec700 1 --2- 172.21.15.5:0/3083237102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf201034c0 0x7faf2019add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:28.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:28.146+0000 7faf1ffff700 1 -- 172.21.15.5:0/3083237102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf201034c0 msgr2=0x7faf2019add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:28.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:28.146+0000 7faf1ffff700 1 --2- 172.21.15.5:0/3083237102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf201034c0 0x7faf2019add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:28.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:28.146+0000 7faf1ffff700 1 -- 172.21.15.5:0/3083237102 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf20108740 msgr2=0x7faf2019f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:28.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:28.146+0000 7faf1ffff700 1 --2- 172.21.15.5:0/3083237102 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf20108740 0x7faf2019f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:28.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:28.146+0000 7faf1ffff700 1 -- 172.21.15.5:0/3083237102 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf2019fca0 con 0x7faf20107df0 2024-05-09T01:01:28.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:28.146+0000 7faf254ed700 1 --2- 172.21.15.5:0/3083237102 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf20108740 0x7faf2019f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:01:28.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:28.146+0000 7faf1ffff700 1 --2- 172.21.15.5:0/3083237102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf20107df0 0x7faf2019b370 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7faf0c00e970 tx=0x7faf0c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:01:28.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:28.146+0000 7faf1dffb700 1 -- 172.21.15.5:0/3083237102 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c004a30 con 0x7faf20107df0 2024-05-09T01:01:28.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:28.146+0000 7faf1dffb700 1 -- 172.21.15.5:0/3083237102 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf0c00cc90 con 0x7faf20107df0 2024-05-09T01:01:28.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:28.146+0000 7faf26f50700 1 -- 172.21.15.5:0/3083237102 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf2019ff90 con 0x7faf20107df0 2024-05-09T01:01:28.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:28.146+0000 7faf1dffb700 1 -- 172.21.15.5:0/3083237102 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c005610 con 0x7faf20107df0 2024-05-09T01:01:28.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:28.147+0000 7faf26f50700 1 -- 172.21.15.5:0/3083237102 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf201a78b0 con 0x7faf20107df0 2024-05-09T01:01:28.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:28.147+0000 7faf26f50700 1 -- 172.21.15.5:0/3083237102 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf200512c0 con 0x7faf20107df0 2024-05-09T01:01:28.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:28.149+0000 7faf1dffb700 1 -- 172.21.15.5:0/3083237102 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7faf0c00c7e0 con 0x7faf20107df0 2024-05-09T01:01:28.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:28.150+0000 7faf1dffb700 1 --2- 172.21.15.5:0/3083237102 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7faf100b2880 0x7faf100b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:28.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:28.150+0000 7faf24cec700 1 --2- 172.21.15.5:0/3083237102 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7faf100b2880 0x7faf100b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:28.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:28.150+0000 7faf1dffb700 1 -- 172.21.15.5:0/3083237102 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7faf0c0e2520 con 0x7faf20107df0 2024-05-09T01:01:28.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:28.150+0000 7faf24cec700 1 --2- 172.21.15.5:0/3083237102 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7faf100b2880 0x7faf100b4d90 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7faf14009c60 tx=0x7faf14009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:01:28.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:28.153+0000 7faf1dffb700 1 -- 172.21.15.5:0/3083237102 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7faf0c054df0 con 0x7faf20107df0 2024-05-09T01:01:28.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:28.415+0000 7faf26f50700 1 -- 172.21.15.5:0/3083237102 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faf2019c130 con 0x7faf20107df0 2024-05-09T01:01:28.417 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:01:28 smithi005 ceph-mon[30365]: pgmap v824: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 37 MiB/s wr, 176 op/s 2024-05-09T01:01:28.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:28.416+0000 7faf1dffb700 1 -- 172.21.15.5:0/3083237102 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11929 (secure 0 0 0) 0x7faf0c0aa210 con 0x7faf20107df0 2024-05-09T01:01:28.418 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T01:01:28.418 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2685771887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2685771887},{"type":"v1","addr":"172.21.15.5:6835","nonce":2685771887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/254017535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":254017535},{"type":"v1","addr":"172.21.15.5:6837","nonce":254017535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T00:43:45.856066+0000","modified":"2024-05-09T01:01:10.302813+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24508,"mds_1":24499,"mds_2":24494,"mds_3":24493,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14598":{"gid":14598,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2109255228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2109255228},{"type":"v1","addr":"172.21.15.5:6839","nonce":2109255228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/3809966329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3809966329},{"type":"v1","addr":"172.21.15.5:6841","nonce":3809966329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6833/3272897742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3272897742},{"type":"v1","addr":"172.21.15.131:6833","nonce":3272897742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1283093898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1283093898},{"type":"v1","addr":"172.21.15.131:6835","nonce":1283093898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/2809525455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2809525455},{"type":"v1","addr":"172.21.15.157:6833","nonce":2809525455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.131:6837/1633006186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":1633006186},{"type":"v1","addr":"172.21.15.131:6837","nonce":1633006186}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.157:6835/832898273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":832898273},{"type":"v1","addr":"172.21.15.157:6835","nonce":832898273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.131:6839/2748669631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2748669631},{"type":"v1","addr":"172.21.15.131:6839","nonce":2748669631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.157:6837/744863009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":744863009},{"type":"v1","addr":"172.21.15.157:6837","nonce":744863009}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":269,"addr":"172.21.15.157:6839/2779860399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":2779860399},{"type":"v1","addr":"172.21.15.157:6839","nonce":2779860399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24493,24467,24499,24494,24508]},"id":1}]} 2024-05-09T01:01:28.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:28.418+0000 7faf26f50700 1 -- 172.21.15.5:0/3083237102 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7faf100b2880 msgr2=0x7faf100b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:28.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:28.418+0000 7faf26f50700 1 --2- 172.21.15.5:0/3083237102 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7faf100b2880 0x7faf100b4d90 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7faf14009c60 tx=0x7faf14009260 comp rx=0 tx=0).stop 2024-05-09T01:01:28.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:28.418+0000 7faf26f50700 1 -- 172.21.15.5:0/3083237102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf20107df0 msgr2=0x7faf2019b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:28.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:28.418+0000 7faf26f50700 1 --2- 172.21.15.5:0/3083237102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf20107df0 0x7faf2019b370 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7faf0c00e970 tx=0x7faf0c00ec80 comp rx=0 tx=0).stop 2024-05-09T01:01:28.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:28.418+0000 7faf26f50700 1 -- 172.21.15.5:0/3083237102 shutdown_connections 2024-05-09T01:01:28.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:28.418+0000 7faf26f50700 1 --2- 172.21.15.5:0/3083237102 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7faf100b2880 0x7faf100b4d90 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:28.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:28.418+0000 7faf26f50700 1 --2- 172.21.15.5:0/3083237102 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf20108740 0x7faf2019f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:28.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:28.418+0000 7faf26f50700 1 --2- 172.21.15.5:0/3083237102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf201034c0 0x7faf2019add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:28.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:28.418+0000 7faf26f50700 1 --2- 172.21.15.5:0/3083237102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf20107df0 0x7faf2019b370 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:28.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:28.418+0000 7faf26f50700 1 -- 172.21.15.5:0/3083237102 >> 172.21.15.5:0/3083237102 conn(0x7faf200fedf0 msgr2=0x7faf20075d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:01:28.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:28.418+0000 7faf26f50700 1 -- 172.21.15.5:0/3083237102 shutdown_connections 2024-05-09T01:01:28.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:28.418+0000 7faf26f50700 1 -- 172.21.15.5:0/3083237102 wait complete. 2024-05-09T01:01:28.421 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 34 2024-05-09T01:01:28.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:01:28 smithi157 ceph-mon[34294]: pgmap v824: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 37 MiB/s wr, 176 op/s 2024-05-09T01:01:28.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:01:28 smithi131 ceph-mon[34547]: pgmap v824: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 37 MiB/s wr, 176 op/s 2024-05-09T01:01:29.178 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-09T01:01:29.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:01:29 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/3083237102' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:01:29.439 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:01:29.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:01:29 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/3083237102' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:01:29.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:01:29 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/3083237102' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:01:30.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:01:30 smithi157 ceph-mon[34294]: pgmap v825: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 29 MiB/s wr, 145 op/s 2024-05-09T01:01:30.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:01:30 smithi005 ceph-mon[30365]: pgmap v825: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 29 MiB/s wr, 145 op/s 2024-05-09T01:01:30.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:01:30 smithi131 ceph-mon[34547]: pgmap v825: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 29 MiB/s wr, 145 op/s 2024-05-09T01:01:31.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.332+0000 7f2563cb8700 1 -- 172.21.15.5:0/1338013676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f255c108740 msgr2=0x7f255c1100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:31.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.332+0000 7f2563cb8700 1 --2- 172.21.15.5:0/1338013676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f255c108740 0x7f255c1100a0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f255400b0d0 tx=0x7f255400b3e0 comp rx=0 tx=0).stop 2024-05-09T01:01:31.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.332+0000 7f2563cb8700 1 -- 172.21.15.5:0/1338013676 shutdown_connections 2024-05-09T01:01:31.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.332+0000 7f2563cb8700 1 --2- 172.21.15.5:0/1338013676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f255c108740 0x7f255c1100a0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:31.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.332+0000 7f2563cb8700 1 --2- 172.21.15.5:0/1338013676 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f255c1034c0 0x7f255c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:31.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.332+0000 7f2563cb8700 1 --2- 172.21.15.5:0/1338013676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f255c107df0 0x7f255c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:31.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.332+0000 7f2563cb8700 1 -- 172.21.15.5:0/1338013676 >> 172.21.15.5:0/1338013676 conn(0x7f255c0fedf0 msgr2=0x7f255c101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:01:31.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.332+0000 7f2563cb8700 1 -- 172.21.15.5:0/1338013676 shutdown_connections 2024-05-09T01:01:31.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.332+0000 7f2563cb8700 1 -- 172.21.15.5:0/1338013676 wait complete. 2024-05-09T01:01:31.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.333+0000 7f2563cb8700 1 Processor -- start 2024-05-09T01:01:31.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.333+0000 7f2563cb8700 1 -- start start 2024-05-09T01:01:31.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.333+0000 7f2563cb8700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f255c1034c0 0x7f255c0ff2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:31.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.333+0000 7f2563cb8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f255c107df0 0x7f255c0ff810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:31.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.333+0000 7f2563cb8700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f255c108740 0x7f255c0ffd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:31.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.333+0000 7f2563cb8700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f255c113370 con 0x7f255c107df0 2024-05-09T01:01:31.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.333+0000 7f2563cb8700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f255c1131f0 con 0x7f255c1034c0 2024-05-09T01:01:31.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.333+0000 7f2563cb8700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f255c1134f0 con 0x7f255c108740 2024-05-09T01:01:31.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.333+0000 7f2561253700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f255c107df0 0x7f255c0ff810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:31.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.333+0000 7f2561253700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f255c107df0 0x7f255c0ff810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41688/0 (socket says 172.21.15.5:41688) 2024-05-09T01:01:31.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.333+0000 7f2561253700 1 -- 172.21.15.5:0/3845526258 learned_addr learned my addr 172.21.15.5:0/3845526258 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:01:31.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.334+0000 7f2562255700 1 --2- 172.21.15.5:0/3845526258 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f255c108740 0x7f255c0ffd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:31.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.334+0000 7f2561a54700 1 --2- 172.21.15.5:0/3845526258 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f255c1034c0 0x7f255c0ff2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:31.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.334+0000 7f2561a54700 1 -- 172.21.15.5:0/3845526258 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f255c108740 msgr2=0x7f255c0ffd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:31.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.334+0000 7f2561a54700 1 --2- 172.21.15.5:0/3845526258 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f255c108740 0x7f255c0ffd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:31.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.334+0000 7f2561a54700 1 -- 172.21.15.5:0/3845526258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f255c107df0 msgr2=0x7f255c0ff810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:31.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.334+0000 7f2561a54700 1 --2- 172.21.15.5:0/3845526258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f255c107df0 0x7f255c0ff810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:31.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.334+0000 7f2561a54700 1 -- 172.21.15.5:0/3845526258 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f255c1a91d0 con 0x7f255c1034c0 2024-05-09T01:01:31.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.334+0000 7f2561253700 1 --2- 172.21.15.5:0/3845526258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f255c107df0 0x7f255c0ff810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:01:31.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.334+0000 7f2562255700 1 --2- 172.21.15.5:0/3845526258 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f255c108740 0x7f255c0ffd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:01:31.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.334+0000 7f2561a54700 1 --2- 172.21.15.5:0/3845526258 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f255c1034c0 0x7f255c0ff2d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f255800e9f0 tx=0x7f255800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:01:31.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.334+0000 7f2552ffd700 1 -- 172.21.15.5:0/3845526258 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f255800cb70 con 0x7f255c1034c0 2024-05-09T01:01:31.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.334+0000 7f2563cb8700 1 -- 172.21.15.5:0/3845526258 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f255c1a94c0 con 0x7f255c1034c0 2024-05-09T01:01:31.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.334+0000 7f2552ffd700 1 -- 172.21.15.5:0/3845526258 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f255800ee90 con 0x7f255c1034c0 2024-05-09T01:01:31.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.335+0000 7f2552ffd700 1 -- 172.21.15.5:0/3845526258 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2558005020 con 0x7f255c1034c0 2024-05-09T01:01:31.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.335+0000 7f2563cb8700 1 -- 172.21.15.5:0/3845526258 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f255c1a9a50 con 0x7f255c1034c0 2024-05-09T01:01:31.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.337+0000 7f2552ffd700 1 -- 172.21.15.5:0/3845526258 <== mon.2 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f25580051c0 con 0x7f255c1034c0 2024-05-09T01:01:31.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.338+0000 7f2552ffd700 1 --2- 172.21.15.5:0/3845526258 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f25480b2880 0x7f25480b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:31.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.338+0000 7f2552ffd700 1 -- 172.21.15.5:0/3845526258 <== mon.2 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f25580e1f60 con 0x7f255c1034c0 2024-05-09T01:01:31.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.338+0000 7f2561253700 1 --2- 172.21.15.5:0/3845526258 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f25480b2880 0x7f25480b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:31.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.338+0000 7f25726ceb80 1 Processor -- start 2024-05-09T01:01:31.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.339+0000 7f2561253700 1 --2- 172.21.15.5:0/3845526258 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f25480b2880 0x7f25480b4d90 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f255c109a70 tx=0x7f254c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:01:31.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.339+0000 7f25726ceb80 1 -- start start 2024-05-09T01:01:31.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.339+0000 7f25726ceb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c599b8f160 0x55c599c284f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:31.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.339+0000 7f25726ceb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c599aa54a0 0x55c599c28a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:31.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.339+0000 7f25726ceb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c599c226a0 0x55c599c22ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:31.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.339+0000 7f25726ceb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c599b44ff0 con 0x55c599c226a0 2024-05-09T01:01:31.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.339+0000 7f25726ceb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55c599ae37f0 con 0x55c599b8f160 2024-05-09T01:01:31.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.339+0000 7f25726ceb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55c599b40fb0 con 0x55c599aa54a0 2024-05-09T01:01:31.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.339+0000 7f2562255700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c599c226a0 0x55c599c22ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:31.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.339+0000 7f2562255700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c599c226a0 0x55c599c22ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41690/0 (socket says 172.21.15.5:41690) 2024-05-09T01:01:31.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.339+0000 7f2562255700 1 -- 172.21.15.5:0/3155940287 learned_addr learned my addr 172.21.15.5:0/3155940287 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:01:31.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.339+0000 7f2561a54700 1 --2- 172.21.15.5:0/3155940287 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c599b8f160 0x55c599c284f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:31.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.339+0000 7f2561253700 1 --2- 172.21.15.5:0/3155940287 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c599aa54a0 0x55c599c28a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:31.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.340+0000 7f2561a54700 1 -- 172.21.15.5:0/3155940287 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c599aa54a0 msgr2=0x55c599c28a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:01:31.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.340+0000 7f2561a54700 1 --2- 172.21.15.5:0/3155940287 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c599aa54a0 0x55c599c28a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:31.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.340+0000 7f2561a54700 1 -- 172.21.15.5:0/3155940287 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c599c226a0 msgr2=0x55c599c22ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:01:31.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.340+0000 7f2561a54700 1 --2- 172.21.15.5:0/3155940287 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c599c226a0 0x55c599c22ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:31.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.340+0000 7f2561a54700 1 -- 172.21.15.5:0/3155940287 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c599adb370 con 0x55c599b8f160 2024-05-09T01:01:31.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.340+0000 7f2562255700 1 --2- 172.21.15.5:0/3155940287 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c599c226a0 0x55c599c22ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:01:31.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.340+0000 7f2561253700 1 --2- 172.21.15.5:0/3155940287 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c599aa54a0 0x55c599c28a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:01:31.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.340+0000 7f2561a54700 1 --2- 172.21.15.5:0/3155940287 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c599b8f160 0x55c599c284f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f255800e9c0 tx=0x7f25580a9080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=488a504e3f14718d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:01:31.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.340+0000 7f2547fff700 1 -- 172.21.15.5:0/3155940287 <== mon.2 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25580c3070 con 0x55c599b8f160 2024-05-09T01:01:31.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.340+0000 7f2547fff700 1 -- 172.21.15.5:0/3155940287 <== mon.2 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25580a94f0 con 0x55c599b8f160 2024-05-09T01:01:31.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.340+0000 7f2547fff700 1 -- 172.21.15.5:0/3155940287 <== mon.2 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f255801c070 con 0x55c599b8f160 2024-05-09T01:01:31.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.341+0000 7f25726ceb80 1 -- 172.21.15.5:0/3155940287 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c599b8f160 msgr2=0x55c599c284f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:31.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.341+0000 7f25726ceb80 1 --2- 172.21.15.5:0/3155940287 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c599b8f160 0x55c599c284f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f255800e9c0 tx=0x7f25580a9080 comp rx=0 tx=0).stop 2024-05-09T01:01:31.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.341+0000 7f25726ceb80 1 -- 172.21.15.5:0/3155940287 shutdown_connections 2024-05-09T01:01:31.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.341+0000 7f25726ceb80 1 --2- 172.21.15.5:0/3155940287 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c599c226a0 0x55c599c22ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:31.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.341+0000 7f25726ceb80 1 --2- 172.21.15.5:0/3155940287 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c599b8f160 0x55c599c284f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:31.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.341+0000 7f25726ceb80 1 --2- 172.21.15.5:0/3155940287 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c599aa54a0 0x55c599c28a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:31.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.341+0000 7f25726ceb80 1 -- 172.21.15.5:0/3155940287 >> 172.21.15.5:0/3155940287 conn(0x55c599b056a0 msgr2=0x55c599c1d400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:01:31.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.341+0000 7f25726ceb80 1 -- 172.21.15.5:0/3155940287 shutdown_connections 2024-05-09T01:01:31.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.341+0000 7f25726ceb80 1 -- 172.21.15.5:0/3155940287 wait complete. 2024-05-09T01:01:31.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.342+0000 7f25726ceb80 1 -- start start 2024-05-09T01:01:31.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.342+0000 7f25726ceb80 1 Processor -- start 2024-05-09T01:01:31.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.342+0000 7f25726ceb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c599aa54a0 0x55c599c2c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:31.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.342+0000 7f25726ceb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c599b8f160 0x55c599c2c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:31.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.342+0000 7f25726ceb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c599c226a0 0x55c599c2cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:31.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.342+0000 7f25726ceb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c5999c0ba0 con 0x55c599aa54a0 2024-05-09T01:01:31.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.342+0000 7f25726ceb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55c599aff150 con 0x55c599c226a0 2024-05-09T01:01:31.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.342+0000 7f25726ceb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55c599c2f1d0 con 0x55c599b8f160 2024-05-09T01:01:31.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.342+0000 7f25726ceb80 10 client.0 _ll_register_callbacks cb 0x55c5999b66c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:01:31.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.342+0000 7f2561a54700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c599aa54a0 0x55c599c2c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:31.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.342+0000 7f2561a54700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c599aa54a0 0x55c599c2c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41706/0 (socket says 172.21.15.5:41706) 2024-05-09T01:01:31.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.342+0000 7f2561a54700 1 -- 172.21.15.5:0/734607031 learned_addr learned my addr 172.21.15.5:0/734607031 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:01:31.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.343+0000 7f2561253700 1 --2- 172.21.15.5:0/734607031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c599b8f160 0x55c599c2c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:31.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.343+0000 7f2562255700 1 --2- 172.21.15.5:0/734607031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c599c226a0 0x55c599c2cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:31.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.343+0000 7f2561a54700 1 -- 172.21.15.5:0/734607031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c599b8f160 msgr2=0x55c599c2c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:01:31.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.343+0000 7f2561a54700 1 --2- 172.21.15.5:0/734607031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c599b8f160 0x55c599c2c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:31.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.343+0000 7f2561a54700 1 -- 172.21.15.5:0/734607031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c599c226a0 msgr2=0x55c599c2cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:01:31.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.343+0000 7f2561a54700 1 --2- 172.21.15.5:0/734607031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c599c226a0 0x55c599c2cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:31.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.343+0000 7f2561a54700 1 -- 172.21.15.5:0/734607031 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5999c0490 con 0x55c599aa54a0 2024-05-09T01:01:31.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.343+0000 7f2561a54700 1 --2- 172.21.15.5:0/734607031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c599aa54a0 0x55c599c2c2c0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f25580a5570 tx=0x7f25580ad950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bedd3a6de0dbfec4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:01:31.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.343+0000 7f25457fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T01:01:31.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.343+0000 7f25457fa700 1 -- 172.21.15.5:0/734607031 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f255806d020 con 0x55c599aa54a0 2024-05-09T01:01:31.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.343+0000 7f25457fa700 1 -- 172.21.15.5:0/734607031 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f25580b20c0 con 0x55c599aa54a0 2024-05-09T01:01:31.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.343+0000 7f25726ceb80 1 -- 172.21.15.5:0/734607031 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25580a94f0 con 0x55c599aa54a0 2024-05-09T01:01:31.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.343+0000 7f25457fa700 1 -- 172.21.15.5:0/734607031 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f25580c9030 con 0x55c599aa54a0 2024-05-09T01:01:31.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.343+0000 7f25726ceb80 10 client.17205 fetch_fsmap learned FSMap version 34 2024-05-09T01:01:31.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.343+0000 7f25726ceb80 1 -- 172.21.15.5:0/734607031 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c599c39c00 con 0x55c599aa54a0 2024-05-09T01:01:31.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.343+0000 7f25457fa700 1 -- 172.21.15.5:0/734607031 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25580a94f0 con 0x55c599aa54a0 2024-05-09T01:01:31.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.344+0000 7f25457fa700 1 -- 172.21.15.5:0/734607031 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7f255801f030 con 0x55c599aa54a0 2024-05-09T01:01:31.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.344+0000 7f25726ceb80 10 client.17205 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T01:01:31.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.344+0000 7f25726ceb80 10 client.17205 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T01:01:31.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.344+0000 7f25726ceb80 20 client.17205 populate_metadata read hostname 'smithi005' 2024-05-09T01:01:31.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.344+0000 7f25726ceb80 1 --2- 172.21.15.5:0/734607031 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x55c599c3c0d0 0x55c599c3e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:31.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.344+0000 7f25726ceb80 4 client.17205 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:01:31.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.344+0000 7f25726ceb80 1 -- 172.21.15.5:0/734607031 --> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c599b529d0 con 0x55c599c3c0d0 2024-05-09T01:01:31.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.344+0000 7f2561253700 1 --2- 172.21.15.5:0/734607031 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x55c599c3c0d0 0x55c599c3e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:31.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.344+0000 7f25457fa700 10 client.17205 ms_handle_connect on v2:172.21.15.131:6836/1633006186 2024-05-09T01:01:31.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.344+0000 7f2561253700 1 --2- 172.21.15.5:0/734607031 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x55c599c3c0d0 0x55c599c3e4f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=95ee972bee559c0e server_cookie=2e7d544092580f21 in_seq=0 out_seq=0 2024-05-09T01:01:31.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.345+0000 7f25457fa700 1 -- 172.21.15.5:0/734607031 <== mds.4 v2:172.21.15.131:6836/1633006186 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c599b529d0 con 0x55c599c3c0d0 2024-05-09T01:01:31.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.345+0000 7f25457fa700 10 client.17205 handle_command_reply: tid=0 2024-05-09T01:01:31.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.345+0000 7f25726ceb80 1 client.17205 shutdown 2024-05-09T01:01:31.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.345+0000 7f25726ceb80 1 -- 172.21.15.5:0/734607031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c599aa54a0 msgr2=0x55c599c2c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:31.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.345+0000 7f25726ceb80 1 --2- 172.21.15.5:0/734607031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c599aa54a0 0x55c599c2c2c0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f25580a5570 tx=0x7f25580ad950 comp rx=0 tx=0).stop 2024-05-09T01:01:31.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.345+0000 7f25726ceb80 1 -- 172.21.15.5:0/734607031 shutdown_connections 2024-05-09T01:01:31.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.345+0000 7f25726ceb80 1 --2- 172.21.15.5:0/734607031 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x55c599c3c0d0 0x55c599c3e4f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:31.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.345+0000 7f25726ceb80 1 --2- 172.21.15.5:0/734607031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c599c226a0 0x55c599c2cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:31.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.345+0000 7f25457fa700 0 client.17205 ms_handle_reset on v2:172.21.15.131:6836/1633006186 2024-05-09T01:01:31.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.345+0000 7f25726ceb80 1 --2- 172.21.15.5:0/734607031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c599aa54a0 0x55c599c2c2c0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:31.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.345+0000 7f25726ceb80 1 --2- 172.21.15.5:0/734607031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c599b8f160 0x55c599c2c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:31.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.345+0000 7f25726ceb80 1 -- 172.21.15.5:0/734607031 >> 172.21.15.5:0/734607031 conn(0x55c599b056a0 msgr2=0x55c599c1d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:01:31.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.345+0000 7f25726ceb80 1 -- 172.21.15.5:0/734607031 shutdown_connections 2024-05-09T01:01:31.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.346+0000 7f25726ceb80 1 -- 172.21.15.5:0/734607031 wait complete. 2024-05-09T01:01:31.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.346+0000 7f25726ceb80 20 client.17205 trim_cache size 0 max 16384 2024-05-09T01:01:31.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.361+0000 7f25726ceb80 1 Processor -- start 2024-05-09T01:01:31.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.361+0000 7f25726ceb80 1 -- start start 2024-05-09T01:01:31.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.361+0000 7f25726ceb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c599aa54a0 0x55c599c2abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:31.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.361+0000 7f25726ceb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c599b8f160 0x55c599c2b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:31.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.361+0000 7f25726ceb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c599c226a0 0x55c599c2c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:31.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.361+0000 7f25726ceb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f255c1b0f00 con 0x55c599aa54a0 2024-05-09T01:01:31.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.361+0000 7f25726ceb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f255c1b2a70 con 0x55c599b8f160 2024-05-09T01:01:31.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.361+0000 7f25726ceb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f255c1b15d0 con 0x55c599c226a0 2024-05-09T01:01:31.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.361+0000 7f2561a54700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c599aa54a0 0x55c599c2abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:31.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.361+0000 7f2561a54700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c599aa54a0 0x55c599c2abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41720/0 (socket says 172.21.15.5:41720) 2024-05-09T01:01:31.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.361+0000 7f2561a54700 1 -- 172.21.15.5:0/1266771493 learned_addr learned my addr 172.21.15.5:0/1266771493 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:01:31.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.362+0000 7f2561253700 1 --2- 172.21.15.5:0/1266771493 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c599b8f160 0x55c599c2b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:31.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.362+0000 7f2562255700 1 --2- 172.21.15.5:0/1266771493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c599c226a0 0x55c599c2c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:31.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.362+0000 7f2561a54700 1 -- 172.21.15.5:0/1266771493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c599c226a0 msgr2=0x55c599c2c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:01:31.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.362+0000 7f2561a54700 1 --2- 172.21.15.5:0/1266771493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c599c226a0 0x55c599c2c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:31.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.362+0000 7f2561a54700 1 -- 172.21.15.5:0/1266771493 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c599b8f160 msgr2=0x55c599c2b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:01:31.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.362+0000 7f2561a54700 1 --2- 172.21.15.5:0/1266771493 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c599b8f160 0x55c599c2b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:31.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.362+0000 7f2561a54700 1 -- 172.21.15.5:0/1266771493 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c599c1ed00 con 0x55c599aa54a0 2024-05-09T01:01:31.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.362+0000 7f2562255700 1 --2- 172.21.15.5:0/1266771493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c599c226a0 0x55c599c2c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:01:31.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.362+0000 7f2561a54700 1 --2- 172.21.15.5:0/1266771493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c599aa54a0 0x55c599c2abb0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f255800c5b0 tx=0x7f255800cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1115a6cb04cff64 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:01:31.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.362+0000 7f25477fe700 1 -- 172.21.15.5:0/1266771493 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f255806d070 con 0x55c599aa54a0 2024-05-09T01:01:31.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.362+0000 7f25477fe700 1 -- 172.21.15.5:0/1266771493 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2558069e50 con 0x55c599aa54a0 2024-05-09T01:01:31.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.362+0000 7f25477fe700 1 -- 172.21.15.5:0/1266771493 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25580c3030 con 0x55c599aa54a0 2024-05-09T01:01:31.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.363+0000 7f25726ceb80 1 -- 172.21.15.5:0/1266771493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c599aa54a0 msgr2=0x55c599c2abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:31.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.363+0000 7f25726ceb80 1 --2- 172.21.15.5:0/1266771493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c599aa54a0 0x55c599c2abb0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f255800c5b0 tx=0x7f255800cf40 comp rx=0 tx=0).stop 2024-05-09T01:01:31.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.363+0000 7f25726ceb80 1 -- 172.21.15.5:0/1266771493 shutdown_connections 2024-05-09T01:01:31.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.363+0000 7f25726ceb80 1 --2- 172.21.15.5:0/1266771493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c599c226a0 0x55c599c2c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:31.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.363+0000 7f25726ceb80 1 --2- 172.21.15.5:0/1266771493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c599aa54a0 0x55c599c2abb0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:31.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.363+0000 7f25726ceb80 1 --2- 172.21.15.5:0/1266771493 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c599b8f160 0x55c599c2b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:31.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.363+0000 7f25726ceb80 1 -- 172.21.15.5:0/1266771493 >> 172.21.15.5:0/1266771493 conn(0x55c599b056a0 msgr2=0x55c599c27380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:01:31.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.363+0000 7f25726ceb80 1 -- 172.21.15.5:0/1266771493 shutdown_connections 2024-05-09T01:01:31.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.363+0000 7f25726ceb80 1 -- 172.21.15.5:0/1266771493 wait complete. 2024-05-09T01:01:31.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.363+0000 7f25726ceb80 1 -- start start 2024-05-09T01:01:31.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.363+0000 7f25726ceb80 1 Processor -- start 2024-05-09T01:01:31.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.364+0000 7f25726ceb80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c599aa54a0 0x55c599c41740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:31.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.364+0000 7f25726ceb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c599b8f160 0x55c599c294b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:31.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.364+0000 7f25726ceb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c599c226a0 0x55c599c299f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:31.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.364+0000 7f25726ceb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f255c1b23a0 con 0x55c599c226a0 2024-05-09T01:01:31.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.364+0000 7f25726ceb80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f255c1b1ca0 con 0x55c599aa54a0 2024-05-09T01:01:31.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.364+0000 7f25726ceb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f255c1aeea0 con 0x55c599b8f160 2024-05-09T01:01:31.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.364+0000 7f25726ceb80 10 client.0 _ll_register_callbacks cb 0x55c5999b66c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:01:31.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.364+0000 7f2562255700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c599c226a0 0x55c599c299f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:31.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.364+0000 7f2562255700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c599c226a0 0x55c599c299f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41730/0 (socket says 172.21.15.5:41730) 2024-05-09T01:01:31.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.364+0000 7f2562255700 1 -- 172.21.15.5:0/3349419630 learned_addr learned my addr 172.21.15.5:0/3349419630 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:01:31.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.364+0000 7f2561a54700 1 --2- 172.21.15.5:0/3349419630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c599aa54a0 0x55c599c41740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:31.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.364+0000 7f2561253700 1 --2- 172.21.15.5:0/3349419630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c599b8f160 0x55c599c294b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:31.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.364+0000 7f2562255700 1 -- 172.21.15.5:0/3349419630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c599b8f160 msgr2=0x55c599c294b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:01:31.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.364+0000 7f2562255700 1 --2- 172.21.15.5:0/3349419630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c599b8f160 0x55c599c294b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:31.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.364+0000 7f2562255700 1 -- 172.21.15.5:0/3349419630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c599aa54a0 msgr2=0x55c599c41740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:01:31.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.364+0000 7f2562255700 1 --2- 172.21.15.5:0/3349419630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c599aa54a0 0x55c599c41740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:31.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.364+0000 7f2562255700 1 -- 172.21.15.5:0/3349419630 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c599c29f30 con 0x55c599c226a0 2024-05-09T01:01:31.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.364+0000 7f2561a54700 1 --2- 172.21.15.5:0/3349419630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c599aa54a0 0x55c599c41740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:01:31.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.364+0000 7f2561253700 1 --2- 172.21.15.5:0/3349419630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c599b8f160 0x55c599c294b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:01:31.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.365+0000 7f2562255700 1 --2- 172.21.15.5:0/3349419630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c599c226a0 0x55c599c299f0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f2554003d30 tx=0x7f2554003e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b9010ceb0402d5b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:01:31.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.365+0000 7f25457fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T01:01:31.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.365+0000 7f25457fa700 1 -- 172.21.15.5:0/3349419630 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2554007a50 con 0x55c599c226a0 2024-05-09T01:01:31.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.365+0000 7f25457fa700 1 -- 172.21.15.5:0/3349419630 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f2554007bf0 con 0x55c599c226a0 2024-05-09T01:01:31.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.365+0000 7f25726ceb80 1 -- 172.21.15.5:0/3349419630 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2558069e50 con 0x55c599c226a0 2024-05-09T01:01:31.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.365+0000 7f25457fa700 1 -- 172.21.15.5:0/3349419630 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f25540050b0 con 0x55c599c226a0 2024-05-09T01:01:31.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.365+0000 7f25726ceb80 10 client.17211 fetch_fsmap learned FSMap version 34 2024-05-09T01:01:31.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.365+0000 7f25726ceb80 1 -- 172.21.15.5:0/3349419630 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c599ca6140 con 0x55c599c226a0 2024-05-09T01:01:31.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.365+0000 7f25457fa700 1 -- 172.21.15.5:0/3349419630 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2558069e50 con 0x55c599c226a0 2024-05-09T01:01:31.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.365+0000 7f25457fa700 1 -- 172.21.15.5:0/3349419630 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7f255402e030 con 0x55c599c226a0 2024-05-09T01:01:31.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.366+0000 7f25726ceb80 10 client.17211 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T01:01:31.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.366+0000 7f25726ceb80 10 client.17211 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T01:01:31.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.366+0000 7f25726ceb80 20 client.17211 populate_metadata read hostname 'smithi005' 2024-05-09T01:01:31.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.366+0000 7f25726ceb80 1 --2- 172.21.15.5:0/3349419630 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x55c599ca6900 0x55c599ca8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:31.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.366+0000 7f25726ceb80 4 client.17211 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:01:31.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.366+0000 7f25726ceb80 1 -- 172.21.15.5:0/3349419630 --> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c599b4ca70 con 0x55c599ca6900 2024-05-09T01:01:31.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.366+0000 7f2561a54700 1 --2- 172.21.15.5:0/3349419630 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x55c599ca6900 0x55c599ca8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:31.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.366+0000 7f25457fa700 10 client.17211 ms_handle_connect on v2:172.21.15.131:6836/1633006186 2024-05-09T01:01:31.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.366+0000 7f2561a54700 1 --2- 172.21.15.5:0/3349419630 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x55c599ca6900 0x55c599ca8d20 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7d1bf720798cd978 server_cookie=8061b5871c27af7 in_seq=0 out_seq=0 2024-05-09T01:01:31.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.367+0000 7f25457fa700 1 -- 172.21.15.5:0/3349419630 <== mds.4 v2:172.21.15.131:6836/1633006186 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c599b4ca70 con 0x55c599ca6900 2024-05-09T01:01:31.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.367+0000 7f25457fa700 10 client.17211 handle_command_reply: tid=0 2024-05-09T01:01:31.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.367+0000 7f25726ceb80 1 client.17211 shutdown 2024-05-09T01:01:31.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.367+0000 7f25726ceb80 1 -- 172.21.15.5:0/3349419630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c599c226a0 msgr2=0x55c599c299f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:31.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.367+0000 7f25726ceb80 1 --2- 172.21.15.5:0/3349419630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c599c226a0 0x55c599c299f0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f2554003d30 tx=0x7f2554003e10 comp rx=0 tx=0).stop 2024-05-09T01:01:31.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.367+0000 7f25726ceb80 1 -- 172.21.15.5:0/3349419630 shutdown_connections 2024-05-09T01:01:31.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.367+0000 7f25726ceb80 1 --2- 172.21.15.5:0/3349419630 >> [v2:172.21.15.131:6836/1633006186,v1:172.21.15.131:6837/1633006186] conn(0x55c599ca6900 0x55c599ca8d20 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:31.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.367+0000 7f25726ceb80 1 --2- 172.21.15.5:0/3349419630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c599c226a0 0x55c599c299f0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:31.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.367+0000 7f25726ceb80 1 --2- 172.21.15.5:0/3349419630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c599aa54a0 0x55c599c41740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:31.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.367+0000 7f25457fa700 0 client.17211 ms_handle_reset on v2:172.21.15.131:6836/1633006186 2024-05-09T01:01:31.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.367+0000 7f25726ceb80 1 --2- 172.21.15.5:0/3349419630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c599b8f160 0x55c599c294b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:31.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.367+0000 7f25726ceb80 1 -- 172.21.15.5:0/3349419630 >> 172.21.15.5:0/3349419630 conn(0x55c599b056a0 msgr2=0x55c599c2d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:01:31.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.367+0000 7f25726ceb80 1 -- 172.21.15.5:0/3349419630 shutdown_connections 2024-05-09T01:01:31.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.367+0000 7f25726ceb80 1 -- 172.21.15.5:0/3349419630 wait complete. 2024-05-09T01:01:31.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.367+0000 7f25726ceb80 20 client.17211 trim_cache size 0 max 16384 2024-05-09T01:01:31.374 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-09T01:01:31.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.368+0000 7f2563cb8700 1 -- 172.21.15.5:0/3845526258 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f25480b2880 msgr2=0x7f25480b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:31.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.368+0000 7f2563cb8700 1 --2- 172.21.15.5:0/3845526258 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f25480b2880 0x7f25480b4d90 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f255c109a70 tx=0x7f254c009260 comp rx=0 tx=0).stop 2024-05-09T01:01:31.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.368+0000 7f2563cb8700 1 -- 172.21.15.5:0/3845526258 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f255c1034c0 msgr2=0x7f255c0ff2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:31.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.368+0000 7f2563cb8700 1 --2- 172.21.15.5:0/3845526258 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f255c1034c0 0x7f255c0ff2d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f255800e9f0 tx=0x7f255800ed00 comp rx=0 tx=0).stop 2024-05-09T01:01:31.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.368+0000 7f2563cb8700 1 -- 172.21.15.5:0/3845526258 shutdown_connections 2024-05-09T01:01:31.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.368+0000 7f2563cb8700 1 --2- 172.21.15.5:0/3845526258 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f25480b2880 0x7f25480b4d90 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:31.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.368+0000 7f2563cb8700 1 --2- 172.21.15.5:0/3845526258 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f255c108740 0x7f255c0ffd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:31.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.368+0000 7f2563cb8700 1 --2- 172.21.15.5:0/3845526258 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f255c1034c0 0x7f255c0ff2d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:31.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.368+0000 7f2563cb8700 1 --2- 172.21.15.5:0/3845526258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f255c107df0 0x7f255c0ff810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:31.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.368+0000 7f2563cb8700 1 -- 172.21.15.5:0/3845526258 >> 172.21.15.5:0/3845526258 conn(0x7f255c0fedf0 msgr2=0x7f255c075d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:01:31.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.368+0000 7f2563cb8700 1 -- 172.21.15.5:0/3845526258 shutdown_connections 2024-05-09T01:01:31.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:31.368+0000 7f2563cb8700 1 -- 172.21.15.5:0/3845526258 wait complete. 2024-05-09T01:01:32.153 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-09T01:01:32.409 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:01:32.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:01:32 smithi157 ceph-mon[34294]: pgmap v826: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 29 MiB/s wr, 141 op/s 2024-05-09T01:01:32.532 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:01:32 smithi005 ceph-mon[30365]: pgmap v826: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 29 MiB/s wr, 141 op/s 2024-05-09T01:01:32.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:01:32 smithi131 ceph-mon[34547]: pgmap v826: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 29 MiB/s wr, 141 op/s 2024-05-09T01:01:34.182 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:01:34 smithi005 ceph-mon[30365]: pgmap v827: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 36 MiB/s wr, 176 op/s 2024-05-09T01:01:34.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.171+0000 7fb7c0fa1700 1 -- 172.21.15.5:0/4195359446 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7bc108760 msgr2=0x7fb7bc1100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:34.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.171+0000 7fb7c0fa1700 1 --2- 172.21.15.5:0/4195359446 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7bc108760 0x7fb7bc1100c0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fb7b40099c0 tx=0x7fb7b4009cd0 comp rx=0 tx=0).stop 2024-05-09T01:01:34.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.171+0000 7fb7c0fa1700 1 -- 172.21.15.5:0/4195359446 shutdown_connections 2024-05-09T01:01:34.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.171+0000 7fb7c0fa1700 1 --2- 172.21.15.5:0/4195359446 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7bc108760 0x7fb7bc1100c0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:34.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.171+0000 7fb7c0fa1700 1 --2- 172.21.15.5:0/4195359446 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7bc1034e0 0x7fb7bc1038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:34.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.171+0000 7fb7c0fa1700 1 --2- 172.21.15.5:0/4195359446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7bc107e10 0x7fb7bc108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:34.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.171+0000 7fb7c0fa1700 1 -- 172.21.15.5:0/4195359446 >> 172.21.15.5:0/4195359446 conn(0x7fb7bc0fef30 msgr2=0x7fb7bc101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:01:34.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.171+0000 7fb7c0fa1700 1 -- 172.21.15.5:0/4195359446 shutdown_connections 2024-05-09T01:01:34.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.171+0000 7fb7c0fa1700 1 -- 172.21.15.5:0/4195359446 wait complete. 2024-05-09T01:01:34.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.172+0000 7fb7c0fa1700 1 Processor -- start 2024-05-09T01:01:34.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.172+0000 7fb7c0fa1700 1 -- start start 2024-05-09T01:01:34.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.172+0000 7fb7c0fa1700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7bc1034e0 0x7fb7bc19aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:34.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.172+0000 7fb7c0fa1700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7bc107e10 0x7fb7bc19b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:34.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.173+0000 7fb7c0fa1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7bc108760 0x7fb7bc19f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:34.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.173+0000 7fb7c0fa1700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb7bc113450 con 0x7fb7bc108760 2024-05-09T01:01:34.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.173+0000 7fb7c0fa1700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb7bc1132d0 con 0x7fb7bc1034e0 2024-05-09T01:01:34.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.173+0000 7fb7c0fa1700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb7bc1135d0 con 0x7fb7bc107e10 2024-05-09T01:01:34.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.173+0000 7fb7bad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7bc108760 0x7fb7bc19f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:34.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.173+0000 7fb7bad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7bc108760 0x7fb7bc19f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57964/0 (socket says 172.21.15.5:57964) 2024-05-09T01:01:34.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.173+0000 7fb7bad9d700 1 -- 172.21.15.5:0/3976156688 learned_addr learned my addr 172.21.15.5:0/3976156688 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:01:34.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.173+0000 7fb7ba59c700 1 --2- 172.21.15.5:0/3976156688 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7bc1034e0 0x7fb7bc19aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:34.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.173+0000 7fb7b9d9b700 1 --2- 172.21.15.5:0/3976156688 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7bc107e10 0x7fb7bc19b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:34.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.173+0000 7fb7bad9d700 1 -- 172.21.15.5:0/3976156688 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7bc107e10 msgr2=0x7fb7bc19b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:34.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.173+0000 7fb7bad9d700 1 --2- 172.21.15.5:0/3976156688 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7bc107e10 0x7fb7bc19b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:34.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.173+0000 7fb7bad9d700 1 -- 172.21.15.5:0/3976156688 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7bc1034e0 msgr2=0x7fb7bc19aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:34.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.173+0000 7fb7bad9d700 1 --2- 172.21.15.5:0/3976156688 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7bc1034e0 0x7fb7bc19aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:34.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.173+0000 7fb7bad9d700 1 -- 172.21.15.5:0/3976156688 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7bc19fda0 con 0x7fb7bc108760 2024-05-09T01:01:34.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.173+0000 7fb7bad9d700 1 --2- 172.21.15.5:0/3976156688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7bc108760 0x7fb7bc19f700 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fb7b40047b0 tx=0x7fb7b4009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:01:34.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.173+0000 7fb7ab7fe700 1 -- 172.21.15.5:0/3976156688 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b4004810 con 0x7fb7bc108760 2024-05-09T01:01:34.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.173+0000 7fb7c0fa1700 1 -- 172.21.15.5:0/3976156688 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7bc1a0030 con 0x7fb7bc108760 2024-05-09T01:01:34.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.173+0000 7fb7ab7fe700 1 -- 172.21.15.5:0/3976156688 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7b4005030 con 0x7fb7bc108760 2024-05-09T01:01:34.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.173+0000 7fb7ab7fe700 1 -- 172.21.15.5:0/3976156688 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b4003d30 con 0x7fb7bc108760 2024-05-09T01:01:34.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.174+0000 7fb7c0fa1700 1 -- 172.21.15.5:0/3976156688 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7bc1a0500 con 0x7fb7bc108760 2024-05-09T01:01:34.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.177+0000 7fb7ab7fe700 1 -- 172.21.15.5:0/3976156688 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fb7b4003f50 con 0x7fb7bc108760 2024-05-09T01:01:34.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.177+0000 7fb7cf9b7b80 1 Processor -- start 2024-05-09T01:01:34.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.178+0000 7fb7cf9b7b80 1 -- start start 2024-05-09T01:01:34.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.178+0000 7fb7ab7fe700 1 --2- 172.21.15.5:0/3976156688 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fb7a40b2880 0x7fb7a40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:34.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.178+0000 7fb7cf9b7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ed1e36d00 0x555ed1f43650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:34.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.178+0000 7fb7cf9b7b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ed1dbc4a0 0x555ed1f43b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:34.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.178+0000 7fb7cf9b7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ed1f3d810 0x555ed1f3dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:34.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.178+0000 7fb7b9d9b700 1 --2- 172.21.15.5:0/3976156688 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fb7a40b2880 0x7fb7a40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:34.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.178+0000 7fb7cf9b7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555ed1e5c0a0 con 0x555ed1e36d00 2024-05-09T01:01:34.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.178+0000 7fb7cf9b7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x555ed1e019a0 con 0x555ed1f3d810 2024-05-09T01:01:34.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.178+0000 7fb7cf9b7b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x555ed1e58060 con 0x555ed1dbc4a0 2024-05-09T01:01:34.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.178+0000 7fb7ab7fe700 1 -- 172.21.15.5:0/3976156688 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fb7b40e2dc0 con 0x7fb7bc108760 2024-05-09T01:01:34.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.178+0000 7fb7ba59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ed1e36d00 0x555ed1f43650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:34.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.178+0000 7fb7ba59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ed1e36d00 0x555ed1f43650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57980/0 (socket says 172.21.15.5:57980) 2024-05-09T01:01:34.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.178+0000 7fb7ba59c700 1 -- 172.21.15.5:0/2093934698 learned_addr learned my addr 172.21.15.5:0/2093934698 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:01:34.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.178+0000 7fb7bad9d700 1 --2- 172.21.15.5:0/2093934698 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ed1f3d810 0x555ed1f3dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:34.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.178+0000 7fb7b9d9b700 1 --2- 172.21.15.5:0/2093934698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ed1dbc4a0 0x555ed1f43b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:34.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.178+0000 7fb7b9d9b700 1 --2- 172.21.15.5:0/3976156688 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fb7a40b2880 0x7fb7a40b4d90 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb7b0006fd0 tx=0x7fb7b0009d10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:01:34.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.179+0000 7fb7ba59c700 1 -- 172.21.15.5:0/2093934698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ed1dbc4a0 msgr2=0x555ed1f43b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:01:34.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.179+0000 7fb7ba59c700 1 --2- 172.21.15.5:0/2093934698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ed1dbc4a0 0x555ed1f43b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:34.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.179+0000 7fb7ba59c700 1 -- 172.21.15.5:0/2093934698 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ed1f3d810 msgr2=0x555ed1f3dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:01:34.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.179+0000 7fb7ba59c700 1 --2- 172.21.15.5:0/2093934698 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ed1f3d810 0x555ed1f3dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:34.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.179+0000 7fb7ba59c700 1 -- 172.21.15.5:0/2093934698 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ed1df2500 con 0x555ed1e36d00 2024-05-09T01:01:34.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.179+0000 7fb7ba59c700 1 --2- 172.21.15.5:0/2093934698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ed1e36d00 0x555ed1f43650 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac009990 tx=0x7fb7ac009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee86889c4acf598b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:01:34.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.179+0000 7fb7a8ff9700 1 -- 172.21.15.5:0/2093934698 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac004780 con 0x555ed1e36d00 2024-05-09T01:01:34.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.179+0000 7fb7a8ff9700 1 -- 172.21.15.5:0/2093934698 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7ac01a070 con 0x555ed1e36d00 2024-05-09T01:01:34.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.179+0000 7fb7a8ff9700 1 -- 172.21.15.5:0/2093934698 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac015770 con 0x555ed1e36d00 2024-05-09T01:01:34.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.180+0000 7fb7cf9b7b80 1 -- 172.21.15.5:0/2093934698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ed1e36d00 msgr2=0x555ed1f43650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:34.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.180+0000 7fb7cf9b7b80 1 --2- 172.21.15.5:0/2093934698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ed1e36d00 0x555ed1f43650 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac009990 tx=0x7fb7ac009ca0 comp rx=0 tx=0).stop 2024-05-09T01:01:34.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.180+0000 7fb7cf9b7b80 1 -- 172.21.15.5:0/2093934698 shutdown_connections 2024-05-09T01:01:34.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.180+0000 7fb7cf9b7b80 1 --2- 172.21.15.5:0/2093934698 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ed1f3d810 0x555ed1f3dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:34.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.180+0000 7fb7cf9b7b80 1 --2- 172.21.15.5:0/2093934698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ed1e36d00 0x555ed1f43650 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:34.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.180+0000 7fb7cf9b7b80 1 --2- 172.21.15.5:0/2093934698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ed1dbc4a0 0x555ed1f43b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:34.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.180+0000 7fb7cf9b7b80 1 -- 172.21.15.5:0/2093934698 >> 172.21.15.5:0/2093934698 conn(0x555ed1dbff70 msgr2=0x555ed1f34220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:01:34.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.180+0000 7fb7cf9b7b80 1 -- 172.21.15.5:0/2093934698 shutdown_connections 2024-05-09T01:01:34.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.180+0000 7fb7cf9b7b80 1 -- 172.21.15.5:0/2093934698 wait complete. 2024-05-09T01:01:34.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.180+0000 7fb7cf9b7b80 1 -- start start 2024-05-09T01:01:34.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.181+0000 7fb7cf9b7b80 1 Processor -- start 2024-05-09T01:01:34.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.181+0000 7fb7cf9b7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ed1dbc4a0 0x555ed1f47400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:34.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.181+0000 7fb7cf9b7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ed1e36d00 0x555ed1f47940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:34.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.181+0000 7fb7cf9b7b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ed1f3d810 0x555ed1f47e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:34.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.181+0000 7fb7cf9b7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555ed1f3e570 con 0x555ed1dbc4a0 2024-05-09T01:01:34.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.181+0000 7fb7cf9b7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x555ed1df6b20 con 0x555ed1e36d00 2024-05-09T01:01:34.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.181+0000 7fb7cf9b7b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x555ed1f4a430 con 0x555ed1f3d810 2024-05-09T01:01:34.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.181+0000 7fb7cf9b7b80 10 client.0 _ll_register_callbacks cb 0x555ed1ccd6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:01:34.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.181+0000 7fb7ba59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ed1dbc4a0 0x555ed1f47400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:34.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.181+0000 7fb7ba59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ed1dbc4a0 0x555ed1f47400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57990/0 (socket says 172.21.15.5:57990) 2024-05-09T01:01:34.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.181+0000 7fb7ba59c700 1 -- 172.21.15.5:0/3444601706 learned_addr learned my addr 172.21.15.5:0/3444601706 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:01:34.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.181+0000 7fb7b9d9b700 1 --2- 172.21.15.5:0/3444601706 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ed1e36d00 0x555ed1f47940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:34.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.181+0000 7fb7bad9d700 1 --2- 172.21.15.5:0/3444601706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ed1f3d810 0x555ed1f47e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:34.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.181+0000 7fb7ba59c700 1 -- 172.21.15.5:0/3444601706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ed1f3d810 msgr2=0x555ed1f47e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:01:34.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.181+0000 7fb7ba59c700 1 --2- 172.21.15.5:0/3444601706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ed1f3d810 0x555ed1f47e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:34.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.181+0000 7fb7ba59c700 1 -- 172.21.15.5:0/3444601706 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ed1e36d00 msgr2=0x555ed1f47940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:01:34.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.182+0000 7fb7ba59c700 1 --2- 172.21.15.5:0/3444601706 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ed1e36d00 0x555ed1f47940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:34.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.182+0000 7fb7ba59c700 1 -- 172.21.15.5:0/3444601706 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ed1cd7490 con 0x555ed1dbc4a0 2024-05-09T01:01:34.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.182+0000 7fb7ba59c700 1 --2- 172.21.15.5:0/3444601706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ed1dbc4a0 0x555ed1f47400 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac007e30 tx=0x7fb7ac0077b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab2075d413b04801 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:01:34.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.182+0000 7fb7a1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T01:01:34.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.182+0000 7fb7a1ffb700 1 -- 172.21.15.5:0/3444601706 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac023070 con 0x555ed1dbc4a0 2024-05-09T01:01:34.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.182+0000 7fb7a1ffb700 1 -- 172.21.15.5:0/3444601706 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fb7ac015480 con 0x555ed1dbc4a0 2024-05-09T01:01:34.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.182+0000 7fb7cf9b7b80 1 -- 172.21.15.5:0/3444601706 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7ac01a070 con 0x555ed1dbc4a0 2024-05-09T01:01:34.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.182+0000 7fb7a1ffb700 1 -- 172.21.15.5:0/3444601706 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb7ac02f050 con 0x555ed1dbc4a0 2024-05-09T01:01:34.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.182+0000 7fb7cf9b7b80 10 client.17223 fetch_fsmap learned FSMap version 34 2024-05-09T01:01:34.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.182+0000 7fb7cf9b7b80 1 -- 172.21.15.5:0/3444601706 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x555ed1f54dd0 con 0x555ed1dbc4a0 2024-05-09T01:01:34.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.182+0000 7fb7a1ffb700 1 -- 172.21.15.5:0/3444601706 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7ac01a070 con 0x555ed1dbc4a0 2024-05-09T01:01:34.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.182+0000 7fb7a1ffb700 1 -- 172.21.15.5:0/3444601706 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7fb7ac005150 con 0x555ed1dbc4a0 2024-05-09T01:01:34.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.183+0000 7fb7cf9b7b80 10 client.17223 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T01:01:34.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.183+0000 7fb7cf9b7b80 10 client.17223 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T01:01:34.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.183+0000 7fb7cf9b7b80 20 client.17223 populate_metadata read hostname 'smithi005' 2024-05-09T01:01:34.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.183+0000 7fb7cf9b7b80 1 --2- 172.21.15.5:0/3444601706 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x555ed1f57300 0x555ed1f59720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:34.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.183+0000 7fb7cf9b7b80 4 client.17223 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:01:34.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.183+0000 7fb7cf9b7b80 1 -- 172.21.15.5:0/3444601706 --> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ed1e69a80 con 0x555ed1f57300 2024-05-09T01:01:34.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.183+0000 7fb7b9d9b700 1 --2- 172.21.15.5:0/3444601706 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x555ed1f57300 0x555ed1f59720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:34.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.183+0000 7fb7a1ffb700 10 client.17223 ms_handle_connect on v2:172.21.15.157:6834/832898273 2024-05-09T01:01:34.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.183+0000 7fb7b9d9b700 1 --2- 172.21.15.5:0/3444601706 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x555ed1f57300 0x555ed1f59720 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9fa0f0517f819b57 server_cookie=4d164549ade5f96a in_seq=0 out_seq=0 2024-05-09T01:01:34.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.184+0000 7fb7a1ffb700 1 -- 172.21.15.5:0/3444601706 <== mds.3 v2:172.21.15.157:6834/832898273 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555ed1e69a80 con 0x555ed1f57300 2024-05-09T01:01:34.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.184+0000 7fb7a1ffb700 10 client.17223 handle_command_reply: tid=0 2024-05-09T01:01:34.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.184+0000 7fb7cf9b7b80 1 client.17223 shutdown 2024-05-09T01:01:34.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.184+0000 7fb7cf9b7b80 1 -- 172.21.15.5:0/3444601706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ed1dbc4a0 msgr2=0x555ed1f47400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:34.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.184+0000 7fb7cf9b7b80 1 --2- 172.21.15.5:0/3444601706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ed1dbc4a0 0x555ed1f47400 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac007e30 tx=0x7fb7ac0077b0 comp rx=0 tx=0).stop 2024-05-09T01:01:34.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.184+0000 7fb7cf9b7b80 1 -- 172.21.15.5:0/3444601706 shutdown_connections 2024-05-09T01:01:34.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.184+0000 7fb7cf9b7b80 1 --2- 172.21.15.5:0/3444601706 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x555ed1f57300 0x555ed1f59720 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:34.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.184+0000 7fb7cf9b7b80 1 --2- 172.21.15.5:0/3444601706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ed1f3d810 0x555ed1f47e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:34.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.184+0000 7fb7cf9b7b80 1 --2- 172.21.15.5:0/3444601706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ed1dbc4a0 0x555ed1f47400 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:34.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.184+0000 7fb7cf9b7b80 1 --2- 172.21.15.5:0/3444601706 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ed1e36d00 0x555ed1f47940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:34.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.184+0000 7fb7a1ffb700 0 client.17223 ms_handle_reset on v2:172.21.15.157:6834/832898273 2024-05-09T01:01:34.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.184+0000 7fb7cf9b7b80 1 -- 172.21.15.5:0/3444601706 >> 172.21.15.5:0/3444601706 conn(0x555ed1dbff70 msgr2=0x555ed1f2dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:01:34.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.184+0000 7fb7cf9b7b80 1 -- 172.21.15.5:0/3444601706 shutdown_connections 2024-05-09T01:01:34.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.184+0000 7fb7cf9b7b80 1 -- 172.21.15.5:0/3444601706 wait complete. 2024-05-09T01:01:34.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.185+0000 7fb7cf9b7b80 20 client.17223 trim_cache size 0 max 16384 2024-05-09T01:01:34.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.205+0000 7fb7cf9b7b80 1 Processor -- start 2024-05-09T01:01:34.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.205+0000 7fb7cf9b7b80 1 -- start start 2024-05-09T01:01:34.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.205+0000 7fb7cf9b7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ed1dbc4a0 0x555ed1f2ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:34.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.205+0000 7fb7cf9b7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ed1e36d00 0x555ed1f46360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:34.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.205+0000 7fb7cf9b7b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ed1f3d810 0x555ed1f53600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:34.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.205+0000 7fb7cf9b7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb7bc1aca50 con 0x555ed1dbc4a0 2024-05-09T01:01:34.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.205+0000 7fb7cf9b7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb7bc1ae5c0 con 0x555ed1e36d00 2024-05-09T01:01:34.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.205+0000 7fb7cf9b7b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb7bc1ad120 con 0x555ed1f3d810 2024-05-09T01:01:34.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.205+0000 7fb7ba59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ed1dbc4a0 0x555ed1f2ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:34.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.205+0000 7fb7ba59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ed1dbc4a0 0x555ed1f2ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58002/0 (socket says 172.21.15.5:58002) 2024-05-09T01:01:34.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.205+0000 7fb7ba59c700 1 -- 172.21.15.5:0/825665546 learned_addr learned my addr 172.21.15.5:0/825665546 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:01:34.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.206+0000 7fb7bad9d700 1 --2- 172.21.15.5:0/825665546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ed1f3d810 0x555ed1f53600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:34.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.206+0000 7fb7b9d9b700 1 --2- 172.21.15.5:0/825665546 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ed1e36d00 0x555ed1f46360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:34.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.206+0000 7fb7ba59c700 1 -- 172.21.15.5:0/825665546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ed1f3d810 msgr2=0x555ed1f53600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:01:34.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.206+0000 7fb7ba59c700 1 --2- 172.21.15.5:0/825665546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ed1f3d810 0x555ed1f53600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:34.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.206+0000 7fb7ba59c700 1 -- 172.21.15.5:0/825665546 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ed1e36d00 msgr2=0x555ed1f46360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:01:34.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.206+0000 7fb7ba59c700 1 --2- 172.21.15.5:0/825665546 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ed1e36d00 0x555ed1f46360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:34.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.206+0000 7fb7ba59c700 1 -- 172.21.15.5:0/825665546 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ed1f35aa0 con 0x555ed1dbc4a0 2024-05-09T01:01:34.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.206+0000 7fb7b9d9b700 1 --2- 172.21.15.5:0/825665546 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ed1e36d00 0x555ed1f46360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:01:34.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.206+0000 7fb7ba59c700 1 --2- 172.21.15.5:0/825665546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ed1dbc4a0 0x555ed1f2ea20 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac006130 tx=0x7fb7ac0049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e3e44869b1351e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:01:34.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.206+0000 7fb7a3fff700 1 -- 172.21.15.5:0/825665546 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac023040 con 0x555ed1dbc4a0 2024-05-09T01:01:34.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.206+0000 7fb7a3fff700 1 -- 172.21.15.5:0/825665546 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7ac01a050 con 0x555ed1dbc4a0 2024-05-09T01:01:34.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.206+0000 7fb7a3fff700 1 -- 172.21.15.5:0/825665546 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac005e80 con 0x555ed1dbc4a0 2024-05-09T01:01:34.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.207+0000 7fb7cf9b7b80 1 -- 172.21.15.5:0/825665546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ed1dbc4a0 msgr2=0x555ed1f2ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:34.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.207+0000 7fb7cf9b7b80 1 --2- 172.21.15.5:0/825665546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ed1dbc4a0 0x555ed1f2ea20 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac006130 tx=0x7fb7ac0049d0 comp rx=0 tx=0).stop 2024-05-09T01:01:34.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.207+0000 7fb7cf9b7b80 1 -- 172.21.15.5:0/825665546 shutdown_connections 2024-05-09T01:01:34.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.207+0000 7fb7cf9b7b80 1 --2- 172.21.15.5:0/825665546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ed1f3d810 0x555ed1f53600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:34.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.207+0000 7fb7cf9b7b80 1 --2- 172.21.15.5:0/825665546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ed1dbc4a0 0x555ed1f2ea20 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:34.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.207+0000 7fb7cf9b7b80 1 --2- 172.21.15.5:0/825665546 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ed1e36d00 0x555ed1f46360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:34.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.207+0000 7fb7cf9b7b80 1 -- 172.21.15.5:0/825665546 >> 172.21.15.5:0/825665546 conn(0x555ed1dbff70 msgr2=0x555ed1f410e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:01:34.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.207+0000 7fb7cf9b7b80 1 -- 172.21.15.5:0/825665546 shutdown_connections 2024-05-09T01:01:34.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.207+0000 7fb7cf9b7b80 1 -- 172.21.15.5:0/825665546 wait complete. 2024-05-09T01:01:34.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.207+0000 7fb7cf9b7b80 1 -- start start 2024-05-09T01:01:34.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.208+0000 7fb7cf9b7b80 1 Processor -- start 2024-05-09T01:01:34.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.208+0000 7fb7cf9b7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ed1dbc4a0 0x555ed1f5c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:34.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.208+0000 7fb7cf9b7b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ed1e36d00 0x555ed1f59d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:34.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.208+0000 7fb7cf9b7b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ed1f3d810 0x555ed1f5a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:34.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.208+0000 7fb7cf9b7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb7bc1adef0 con 0x555ed1dbc4a0 2024-05-09T01:01:34.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.208+0000 7fb7cf9b7b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb7bc1ad7f0 con 0x555ed1f3d810 2024-05-09T01:01:34.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.208+0000 7fb7cf9b7b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb7bc1aa9f0 con 0x555ed1e36d00 2024-05-09T01:01:34.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.208+0000 7fb7cf9b7b80 10 client.0 _ll_register_callbacks cb 0x555ed1ccd6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:01:34.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.208+0000 7fb7ba59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ed1dbc4a0 0x555ed1f5c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:34.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.208+0000 7fb7ba59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ed1dbc4a0 0x555ed1f5c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58006/0 (socket says 172.21.15.5:58006) 2024-05-09T01:01:34.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.208+0000 7fb7ba59c700 1 -- 172.21.15.5:0/2860863808 learned_addr learned my addr 172.21.15.5:0/2860863808 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:01:34.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.208+0000 7fb7b9d9b700 1 --2- 172.21.15.5:0/2860863808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ed1e36d00 0x555ed1f59d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:34.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.208+0000 7fb7bad9d700 1 --2- 172.21.15.5:0/2860863808 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ed1f3d810 0x555ed1f5a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:34.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.208+0000 7fb7ba59c700 1 -- 172.21.15.5:0/2860863808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ed1e36d00 msgr2=0x555ed1f59d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:01:34.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.208+0000 7fb7ba59c700 1 --2- 172.21.15.5:0/2860863808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ed1e36d00 0x555ed1f59d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:34.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.208+0000 7fb7ba59c700 1 -- 172.21.15.5:0/2860863808 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ed1f3d810 msgr2=0x555ed1f5a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:01:34.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.208+0000 7fb7ba59c700 1 --2- 172.21.15.5:0/2860863808 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ed1f3d810 0x555ed1f5a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:34.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.208+0000 7fb7ba59c700 1 -- 172.21.15.5:0/2860863808 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ed1f5a7e0 con 0x555ed1dbc4a0 2024-05-09T01:01:34.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.208+0000 7fb7bad9d700 1 --2- 172.21.15.5:0/2860863808 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ed1f3d810 0x555ed1f5a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:01:34.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.209+0000 7fb7ba59c700 1 --2- 172.21.15.5:0/2860863808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ed1dbc4a0 0x555ed1f5c9e0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac009ce0 tx=0x7fb7ac005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ae89592abc78de0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:01:34.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.209+0000 7fb7a1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T01:01:34.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.209+0000 7fb7a1ffb700 1 -- 172.21.15.5:0/2860863808 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac005970 con 0x555ed1dbc4a0 2024-05-09T01:01:34.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.209+0000 7fb7a1ffb700 1 -- 172.21.15.5:0/2860863808 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fb7ac0050f0 con 0x555ed1dbc4a0 2024-05-09T01:01:34.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.209+0000 7fb7cf9b7b80 1 -- 172.21.15.5:0/2860863808 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7ac01a050 con 0x555ed1dbc4a0 2024-05-09T01:01:34.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.209+0000 7fb7a1ffb700 1 -- 172.21.15.5:0/2860863808 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb7ac0052f0 con 0x555ed1dbc4a0 2024-05-09T01:01:34.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.209+0000 7fb7cf9b7b80 10 client.17229 fetch_fsmap learned FSMap version 34 2024-05-09T01:01:34.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.209+0000 7fb7cf9b7b80 1 -- 172.21.15.5:0/2860863808 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x555ed1fc13e0 con 0x555ed1dbc4a0 2024-05-09T01:01:34.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.209+0000 7fb7a1ffb700 1 -- 172.21.15.5:0/2860863808 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7ac01a050 con 0x555ed1dbc4a0 2024-05-09T01:01:34.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.209+0000 7fb7a1ffb700 1 -- 172.21.15.5:0/2860863808 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7fb7ac045440 con 0x555ed1dbc4a0 2024-05-09T01:01:34.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.210+0000 7fb7cf9b7b80 10 client.17229 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T01:01:34.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.210+0000 7fb7cf9b7b80 10 client.17229 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-05-09T01:01:34.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.210+0000 7fb7cf9b7b80 20 client.17229 populate_metadata read hostname 'smithi005' 2024-05-09T01:01:34.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.210+0000 7fb7cf9b7b80 1 --2- 172.21.15.5:0/2860863808 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x555ed1fc1920 0x555ed1fc3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:34.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.210+0000 7fb7cf9b7b80 4 client.17229 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:01:34.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.210+0000 7fb7cf9b7b80 1 -- 172.21.15.5:0/2860863808 --> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555ed1e63b20 con 0x555ed1fc1920 2024-05-09T01:01:34.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.210+0000 7fb7b9d9b700 1 --2- 172.21.15.5:0/2860863808 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x555ed1fc1920 0x555ed1fc3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:34.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.210+0000 7fb7a1ffb700 10 client.17229 ms_handle_connect on v2:172.21.15.157:6834/832898273 2024-05-09T01:01:34.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.210+0000 7fb7b9d9b700 1 --2- 172.21.15.5:0/2860863808 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x555ed1fc1920 0x555ed1fc3d40 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1355e55d0b33395e server_cookie=61a748e554a9ed9b in_seq=0 out_seq=0 2024-05-09T01:01:34.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.210+0000 7fb7a1ffb700 1 -- 172.21.15.5:0/2860863808 <== mds.3 v2:172.21.15.157:6834/832898273 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555ed1e63b20 con 0x555ed1fc1920 2024-05-09T01:01:34.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.210+0000 7fb7a1ffb700 10 client.17229 handle_command_reply: tid=0 2024-05-09T01:01:34.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.210+0000 7fb7cf9b7b80 1 client.17229 shutdown 2024-05-09T01:01:34.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.210+0000 7fb7cf9b7b80 1 -- 172.21.15.5:0/2860863808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ed1dbc4a0 msgr2=0x555ed1f5c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:34.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.210+0000 7fb7cf9b7b80 1 --2- 172.21.15.5:0/2860863808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ed1dbc4a0 0x555ed1f5c9e0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac009ce0 tx=0x7fb7ac005ef0 comp rx=0 tx=0).stop 2024-05-09T01:01:34.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.211+0000 7fb7cf9b7b80 1 -- 172.21.15.5:0/2860863808 shutdown_connections 2024-05-09T01:01:34.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.211+0000 7fb7cf9b7b80 1 --2- 172.21.15.5:0/2860863808 >> [v2:172.21.15.157:6834/832898273,v1:172.21.15.157:6835/832898273] conn(0x555ed1fc1920 0x555ed1fc3d40 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:34.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.211+0000 7fb7cf9b7b80 1 --2- 172.21.15.5:0/2860863808 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555ed1f3d810 0x555ed1f5a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:34.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.211+0000 7fb7cf9b7b80 1 --2- 172.21.15.5:0/2860863808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ed1dbc4a0 0x555ed1f5c9e0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:34.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.211+0000 7fb7cf9b7b80 1 --2- 172.21.15.5:0/2860863808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ed1e36d00 0x555ed1f59d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:34.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.211+0000 7fb7cf9b7b80 1 -- 172.21.15.5:0/2860863808 >> 172.21.15.5:0/2860863808 conn(0x555ed1dbff70 msgr2=0x555ed1f2e780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:01:34.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.211+0000 7fb7a1ffb700 0 client.17229 ms_handle_reset on v2:172.21.15.157:6834/832898273 2024-05-09T01:01:34.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.211+0000 7fb7cf9b7b80 1 -- 172.21.15.5:0/2860863808 shutdown_connections 2024-05-09T01:01:34.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.211+0000 7fb7cf9b7b80 1 -- 172.21.15.5:0/2860863808 wait complete. 2024-05-09T01:01:34.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.211+0000 7fb7cf9b7b80 20 client.17229 trim_cache size 0 max 16384 2024-05-09T01:01:34.217 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-09T01:01:34.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.211+0000 7fb7c0fa1700 1 -- 172.21.15.5:0/3976156688 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fb7a40b2880 msgr2=0x7fb7a40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:34.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.211+0000 7fb7c0fa1700 1 --2- 172.21.15.5:0/3976156688 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fb7a40b2880 0x7fb7a40b4d90 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb7b0006fd0 tx=0x7fb7b0009d10 comp rx=0 tx=0).stop 2024-05-09T01:01:34.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.211+0000 7fb7c0fa1700 1 -- 172.21.15.5:0/3976156688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7bc108760 msgr2=0x7fb7bc19f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:34.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.211+0000 7fb7c0fa1700 1 --2- 172.21.15.5:0/3976156688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7bc108760 0x7fb7bc19f700 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fb7b40047b0 tx=0x7fb7b4009580 comp rx=0 tx=0).stop 2024-05-09T01:01:34.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.211+0000 7fb7c0fa1700 1 -- 172.21.15.5:0/3976156688 shutdown_connections 2024-05-09T01:01:34.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.211+0000 7fb7c0fa1700 1 --2- 172.21.15.5:0/3976156688 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fb7a40b2880 0x7fb7a40b4d90 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:34.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.211+0000 7fb7c0fa1700 1 --2- 172.21.15.5:0/3976156688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7bc108760 0x7fb7bc19f700 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:34.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.211+0000 7fb7c0fa1700 1 --2- 172.21.15.5:0/3976156688 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7bc1034e0 0x7fb7bc19aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:34.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.211+0000 7fb7c0fa1700 1 --2- 172.21.15.5:0/3976156688 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7bc107e10 0x7fb7bc19b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:34.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.211+0000 7fb7c0fa1700 1 -- 172.21.15.5:0/3976156688 >> 172.21.15.5:0/3976156688 conn(0x7fb7bc0fef30 msgr2=0x7fb7bc06c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:01:34.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.212+0000 7fb7c0fa1700 1 -- 172.21.15.5:0/3976156688 shutdown_connections 2024-05-09T01:01:34.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:34.212+0000 7fb7c0fa1700 1 -- 172.21.15.5:0/3976156688 wait complete. 2024-05-09T01:01:34.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:01:34 smithi157 ceph-mon[34294]: pgmap v827: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 36 MiB/s wr, 176 op/s 2024-05-09T01:01:34.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:01:34 smithi131 ceph-mon[34547]: pgmap v827: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 36 MiB/s wr, 176 op/s 2024-05-09T01:01:34.904 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T01:01:35.159 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:01:36.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:01:36 smithi005 ceph-mon[30365]: pgmap v828: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 39 MiB/s wr, 192 op/s 2024-05-09T01:01:36.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:01:36 smithi157 ceph-mon[34294]: pgmap v828: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 39 MiB/s wr, 192 op/s 2024-05-09T01:01:36.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:01:36 smithi131 ceph-mon[34547]: pgmap v828: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 39 MiB/s wr, 192 op/s 2024-05-09T01:01:36.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.941+0000 7f612ae1a700 1 -- 172.21.15.5:0/3396893796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6124108760 msgr2=0x7f61241100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:36.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.941+0000 7f612ae1a700 1 --2- 172.21.15.5:0/3396893796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6124108760 0x7f61241100c0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f611c00b0d0 tx=0x7f611c00b3e0 comp rx=0 tx=0).stop 2024-05-09T01:01:36.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.942+0000 7f612ae1a700 1 -- 172.21.15.5:0/3396893796 shutdown_connections 2024-05-09T01:01:36.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.942+0000 7f612ae1a700 1 --2- 172.21.15.5:0/3396893796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6124108760 0x7f61241100c0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:36.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.942+0000 7f612ae1a700 1 --2- 172.21.15.5:0/3396893796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61241034e0 0x7f61241038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:36.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.942+0000 7f612ae1a700 1 --2- 172.21.15.5:0/3396893796 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6124107e10 0x7f6124108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:36.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.942+0000 7f612ae1a700 1 -- 172.21.15.5:0/3396893796 >> 172.21.15.5:0/3396893796 conn(0x7f61240fef30 msgr2=0x7f6124101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:01:36.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.942+0000 7f612ae1a700 1 -- 172.21.15.5:0/3396893796 shutdown_connections 2024-05-09T01:01:36.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.942+0000 7f612ae1a700 1 -- 172.21.15.5:0/3396893796 wait complete. 2024-05-09T01:01:36.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.942+0000 7f612ae1a700 1 Processor -- start 2024-05-09T01:01:36.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.942+0000 7f612ae1a700 1 -- start start 2024-05-09T01:01:36.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.943+0000 7f612ae1a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61241034e0 0x7f612419adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:36.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.943+0000 7f612ae1a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6124107e10 0x7f612419b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:36.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.943+0000 7f612ae1a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6124108760 0x7f612419f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:36.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.943+0000 7f612ae1a700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f61241133b0 con 0x7f6124107e10 2024-05-09T01:01:36.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.943+0000 7f612ae1a700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6124113230 con 0x7f6124108760 2024-05-09T01:01:36.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.943+0000 7f612ae1a700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6124113530 con 0x7f61241034e0 2024-05-09T01:01:36.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.943+0000 7f6123fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6124107e10 0x7f612419b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:36.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.943+0000 7f6123fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6124107e10 0x7f612419b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58016/0 (socket says 172.21.15.5:58016) 2024-05-09T01:01:36.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.943+0000 7f6123fff700 1 -- 172.21.15.5:0/4127621423 learned_addr learned my addr 172.21.15.5:0/4127621423 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:01:36.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.943+0000 7f61293b7700 1 --2- 172.21.15.5:0/4127621423 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6124108760 0x7f612419f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:36.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.943+0000 7f6128bb6700 1 --2- 172.21.15.5:0/4127621423 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61241034e0 0x7f612419adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:36.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.943+0000 7f6123fff700 1 -- 172.21.15.5:0/4127621423 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61241034e0 msgr2=0x7f612419adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:36.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.943+0000 7f6123fff700 1 --2- 172.21.15.5:0/4127621423 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61241034e0 0x7f612419adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:36.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.943+0000 7f6123fff700 1 -- 172.21.15.5:0/4127621423 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6124108760 msgr2=0x7f612419f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:36.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.943+0000 7f6123fff700 1 --2- 172.21.15.5:0/4127621423 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6124108760 0x7f612419f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:36.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.943+0000 7f6123fff700 1 -- 172.21.15.5:0/4127621423 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f612419fc80 con 0x7f6124107e10 2024-05-09T01:01:36.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.943+0000 7f6123fff700 1 --2- 172.21.15.5:0/4127621423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6124107e10 0x7f612419b350 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f610c00d980 tx=0x7f610c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:01:36.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.943+0000 7f6121ffb700 1 -- 172.21.15.5:0/4127621423 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f610c017070 con 0x7f6124107e10 2024-05-09T01:01:36.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.944+0000 7f6121ffb700 1 -- 172.21.15.5:0/4127621423 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f610c00fd40 con 0x7f6124107e10 2024-05-09T01:01:36.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.944+0000 7f612ae1a700 1 -- 172.21.15.5:0/4127621423 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f612419ff70 con 0x7f6124107e10 2024-05-09T01:01:36.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.944+0000 7f6121ffb700 1 -- 172.21.15.5:0/4127621423 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f610c00e3b0 con 0x7f6124107e10 2024-05-09T01:01:36.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.944+0000 7f612ae1a700 1 -- 172.21.15.5:0/4127621423 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61241a7890 con 0x7f6124107e10 2024-05-09T01:01:36.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.947+0000 7f6121ffb700 1 -- 172.21.15.5:0/4127621423 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f610c00e550 con 0x7f6124107e10 2024-05-09T01:01:36.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.948+0000 7f6121ffb700 1 --2- 172.21.15.5:0/4127621423 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f61100b2880 0x7f61100b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:36.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.948+0000 7f6128bb6700 1 --2- 172.21.15.5:0/4127621423 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f61100b2880 0x7f61100b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:36.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.948+0000 7f6121ffb700 1 -- 172.21.15.5:0/4127621423 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f610c0e1d40 con 0x7f6124107e10 2024-05-09T01:01:36.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.948+0000 7f6139830b80 1 Processor -- start 2024-05-09T01:01:36.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.948+0000 7f6139830b80 1 -- start start 2024-05-09T01:01:36.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.948+0000 7f6128bb6700 1 --2- 172.21.15.5:0/4127621423 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f61100b2880 0x7f61100b4d90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f6114009c60 tx=0x7f6114009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:01:36.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.949+0000 7f6139830b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d4cda4160 0x564d4ce3d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:36.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.949+0000 7f6139830b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d4ccba4a0 0x564d4ce3da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:36.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.949+0000 7f6139830b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564d4ce376a0 0x564d4ce37ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:36.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.949+0000 7f6139830b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564d4cd59ff0 con 0x564d4ccba4a0 2024-05-09T01:01:36.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.949+0000 7f6139830b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x564d4ccf87f0 con 0x564d4ce376a0 2024-05-09T01:01:36.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.949+0000 7f6139830b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x564d4cd55fb0 con 0x564d4cda4160 2024-05-09T01:01:36.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.949+0000 7f6123fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d4ccba4a0 0x564d4ce3da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:36.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.949+0000 7f6128bb6700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d4cda4160 0x564d4ce3d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:36.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.949+0000 7f6123fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d4ccba4a0 0x564d4ce3da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58018/0 (socket says 172.21.15.5:58018) 2024-05-09T01:01:36.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.949+0000 7f6128bb6700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d4cda4160 0x564d4ce3d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:38886/0 (socket says 172.21.15.5:38886) 2024-05-09T01:01:36.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.949+0000 7f6123fff700 1 -- 172.21.15.5:0/1708744062 learned_addr learned my addr 172.21.15.5:0/1708744062 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:01:36.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.949+0000 7f61293b7700 1 --2- 172.21.15.5:0/1708744062 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564d4ce376a0 0x564d4ce37ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:36.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.949+0000 7f6123fff700 1 -- 172.21.15.5:0/1708744062 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d4cda4160 msgr2=0x564d4ce3d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:01:36.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.949+0000 7f6123fff700 1 --2- 172.21.15.5:0/1708744062 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d4cda4160 0x564d4ce3d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:36.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.949+0000 7f6123fff700 1 -- 172.21.15.5:0/1708744062 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564d4ce376a0 msgr2=0x564d4ce37ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:01:36.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.949+0000 7f6123fff700 1 --2- 172.21.15.5:0/1708744062 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564d4ce376a0 0x564d4ce37ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:36.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.949+0000 7f6123fff700 1 -- 172.21.15.5:0/1708744062 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d4ccf0370 con 0x564d4ccba4a0 2024-05-09T01:01:36.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.949+0000 7f6128bb6700 1 --2- 172.21.15.5:0/1708744062 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d4cda4160 0x564d4ce3d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:01:36.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.949+0000 7f6123fff700 1 --2- 172.21.15.5:0/1708744062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d4ccba4a0 0x564d4ce3da30 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f610c0a5350 tx=0x7f610c0a8b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=375cf7db93b5b409 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:01:36.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.950+0000 7f611affd700 1 -- 172.21.15.5:0/1708744062 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f610c020070 con 0x564d4ccba4a0 2024-05-09T01:01:36.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.950+0000 7f611affd700 1 -- 172.21.15.5:0/1708744062 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f610c0a92d0 con 0x564d4ccba4a0 2024-05-09T01:01:36.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.950+0000 7f611affd700 1 -- 172.21.15.5:0/1708744062 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f610c0b1f00 con 0x564d4ccba4a0 2024-05-09T01:01:36.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.950+0000 7f6139830b80 1 -- 172.21.15.5:0/1708744062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d4ccba4a0 msgr2=0x564d4ce3da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:36.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.950+0000 7f6139830b80 1 --2- 172.21.15.5:0/1708744062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d4ccba4a0 0x564d4ce3da30 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f610c0a5350 tx=0x7f610c0a8b40 comp rx=0 tx=0).stop 2024-05-09T01:01:36.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.951+0000 7f6139830b80 1 -- 172.21.15.5:0/1708744062 shutdown_connections 2024-05-09T01:01:36.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.951+0000 7f6139830b80 1 --2- 172.21.15.5:0/1708744062 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564d4ce376a0 0x564d4ce37ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:36.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.951+0000 7f6139830b80 1 --2- 172.21.15.5:0/1708744062 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d4cda4160 0x564d4ce3d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:36.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.951+0000 7f6139830b80 1 --2- 172.21.15.5:0/1708744062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d4ccba4a0 0x564d4ce3da30 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:36.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.951+0000 7f6139830b80 1 -- 172.21.15.5:0/1708744062 >> 172.21.15.5:0/1708744062 conn(0x564d4cd1a6a0 msgr2=0x564d4ce32400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:01:36.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.951+0000 7f6139830b80 1 -- 172.21.15.5:0/1708744062 shutdown_connections 2024-05-09T01:01:36.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.951+0000 7f6139830b80 1 -- 172.21.15.5:0/1708744062 wait complete. 2024-05-09T01:01:36.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.951+0000 7f6139830b80 1 -- start start 2024-05-09T01:01:36.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.951+0000 7f6139830b80 1 Processor -- start 2024-05-09T01:01:36.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.952+0000 7f6139830b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d4ccba4a0 0x564d4ce41230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:36.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.952+0000 7f6139830b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d4cda4160 0x564d4ce41770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:36.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.952+0000 7f6139830b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564d4ce376a0 0x564d4ce41cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:36.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.952+0000 7f6139830b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564d4cbd5ba0 con 0x564d4cda4160 2024-05-09T01:01:36.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.952+0000 7f6139830b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x564d4cd14150 con 0x564d4ce376a0 2024-05-09T01:01:36.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.952+0000 7f6139830b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x564d4ce441d0 con 0x564d4ccba4a0 2024-05-09T01:01:36.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.952+0000 7f6139830b80 10 client.0 _ll_register_callbacks cb 0x564d4cbcb6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:01:36.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.952+0000 7f6123fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d4cda4160 0x564d4ce41770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:36.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.952+0000 7f6123fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d4cda4160 0x564d4ce41770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58020/0 (socket says 172.21.15.5:58020) 2024-05-09T01:01:36.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.952+0000 7f6123fff700 1 -- 172.21.15.5:0/2587076319 learned_addr learned my addr 172.21.15.5:0/2587076319 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:01:36.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.952+0000 7f6128bb6700 1 --2- 172.21.15.5:0/2587076319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d4ccba4a0 0x564d4ce41230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:36.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.952+0000 7f61293b7700 1 --2- 172.21.15.5:0/2587076319 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564d4ce376a0 0x564d4ce41cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:36.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.952+0000 7f6123fff700 1 -- 172.21.15.5:0/2587076319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d4ccba4a0 msgr2=0x564d4ce41230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:01:36.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.952+0000 7f6123fff700 1 --2- 172.21.15.5:0/2587076319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d4ccba4a0 0x564d4ce41230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:36.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.952+0000 7f6123fff700 1 -- 172.21.15.5:0/2587076319 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564d4ce376a0 msgr2=0x564d4ce41cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:01:36.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.952+0000 7f6123fff700 1 --2- 172.21.15.5:0/2587076319 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564d4ce376a0 0x564d4ce41cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:36.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.952+0000 7f6123fff700 1 -- 172.21.15.5:0/2587076319 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d4cbd5490 con 0x564d4cda4160 2024-05-09T01:01:36.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.952+0000 7f6128bb6700 1 --2- 172.21.15.5:0/2587076319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d4ccba4a0 0x564d4ce41230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:01:36.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.952+0000 7f6123fff700 1 --2- 172.21.15.5:0/2587076319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d4cda4160 0x564d4ce41770 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f610c0a8fd0 tx=0x7f610c0a97e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4fdb67185b68e55 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:01:36.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.952+0000 7f6103fff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T01:01:36.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.952+0000 7f6103fff700 1 -- 172.21.15.5:0/2587076319 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f610c020030 con 0x564d4cda4160 2024-05-09T01:01:36.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.953+0000 7f6103fff700 1 -- 172.21.15.5:0/2587076319 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f610c0a58c0 con 0x564d4cda4160 2024-05-09T01:01:36.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.953+0000 7f6139830b80 1 -- 172.21.15.5:0/2587076319 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f610c0a92d0 con 0x564d4cda4160 2024-05-09T01:01:36.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.953+0000 7f6103fff700 1 -- 172.21.15.5:0/2587076319 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f610c0b2130 con 0x564d4cda4160 2024-05-09T01:01:36.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.953+0000 7f6139830b80 10 client.17241 fetch_fsmap learned FSMap version 34 2024-05-09T01:01:36.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.953+0000 7f6139830b80 1 -- 172.21.15.5:0/2587076319 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x564d4ce4eb80 con 0x564d4cda4160 2024-05-09T01:01:36.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.953+0000 7f6103fff700 1 -- 172.21.15.5:0/2587076319 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f610c0a92d0 con 0x564d4cda4160 2024-05-09T01:01:36.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.953+0000 7f6103fff700 1 -- 172.21.15.5:0/2587076319 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7f610c0a86c0 con 0x564d4cda4160 2024-05-09T01:01:36.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.953+0000 7f6139830b80 10 client.17241 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T01:01:36.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.953+0000 7f6139830b80 10 client.17241 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-05-09T01:01:36.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.953+0000 7f6139830b80 20 client.17241 populate_metadata read hostname 'smithi005' 2024-05-09T01:01:36.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.953+0000 7f6139830b80 1 --2- 172.21.15.5:0/2587076319 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x564d4ce51050 0x564d4ce53470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:36.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.953+0000 7f6139830b80 4 client.17241 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:01:36.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.953+0000 7f6139830b80 1 -- 172.21.15.5:0/2587076319 --> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d4cd679d0 con 0x564d4ce51050 2024-05-09T01:01:36.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.954+0000 7f6128bb6700 1 --2- 172.21.15.5:0/2587076319 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x564d4ce51050 0x564d4ce53470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:36.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.954+0000 7f6103fff700 10 client.17241 ms_handle_connect on v2:172.21.15.131:6838/2748669631 2024-05-09T01:01:36.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.954+0000 7f6128bb6700 1 --2- 172.21.15.5:0/2587076319 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x564d4ce51050 0x564d4ce53470 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=49a95e83f6543050 server_cookie=efb3a41cb6ebcdbf in_seq=0 out_seq=0 2024-05-09T01:01:36.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.955+0000 7f6103fff700 1 -- 172.21.15.5:0/2587076319 <== mds.2 v2:172.21.15.131:6838/2748669631 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564d4cd679d0 con 0x564d4ce51050 2024-05-09T01:01:36.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.955+0000 7f6103fff700 10 client.17241 handle_command_reply: tid=0 2024-05-09T01:01:36.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.955+0000 7f6139830b80 1 client.17241 shutdown 2024-05-09T01:01:36.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.955+0000 7f6139830b80 1 -- 172.21.15.5:0/2587076319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d4cda4160 msgr2=0x564d4ce41770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:36.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.955+0000 7f6139830b80 1 --2- 172.21.15.5:0/2587076319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d4cda4160 0x564d4ce41770 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f610c0a8fd0 tx=0x7f610c0a97e0 comp rx=0 tx=0).stop 2024-05-09T01:01:36.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.955+0000 7f6139830b80 1 -- 172.21.15.5:0/2587076319 shutdown_connections 2024-05-09T01:01:36.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.955+0000 7f6139830b80 1 --2- 172.21.15.5:0/2587076319 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x564d4ce51050 0x564d4ce53470 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:36.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.955+0000 7f6139830b80 1 --2- 172.21.15.5:0/2587076319 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564d4ce376a0 0x564d4ce41cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:36.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.955+0000 7f6139830b80 1 --2- 172.21.15.5:0/2587076319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d4ccba4a0 0x564d4ce41230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:36.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.955+0000 7f6103fff700 0 client.17241 ms_handle_reset on v2:172.21.15.131:6838/2748669631 2024-05-09T01:01:36.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.955+0000 7f6139830b80 1 --2- 172.21.15.5:0/2587076319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d4cda4160 0x564d4ce41770 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:36.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.955+0000 7f6139830b80 1 -- 172.21.15.5:0/2587076319 >> 172.21.15.5:0/2587076319 conn(0x564d4cd1a6a0 msgr2=0x564d4ce321c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:01:36.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.955+0000 7f6139830b80 1 -- 172.21.15.5:0/2587076319 shutdown_connections 2024-05-09T01:01:36.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.955+0000 7f6139830b80 1 -- 172.21.15.5:0/2587076319 wait complete. 2024-05-09T01:01:36.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.955+0000 7f6139830b80 20 client.17241 trim_cache size 0 max 16384 2024-05-09T01:01:36.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.970+0000 7f6139830b80 1 Processor -- start 2024-05-09T01:01:36.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.970+0000 7f6139830b80 1 -- start start 2024-05-09T01:01:36.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.971+0000 7f6139830b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d4ccba4a0 0x564d4ce40190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:36.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.971+0000 7f6139830b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564d4cda4160 0x564d4ce3b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:36.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.971+0000 7f6139830b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d4ce376a0 0x564d4ce4d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:36.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.971+0000 7f6139830b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f61241acb00 con 0x564d4ce376a0 2024-05-09T01:01:36.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.971+0000 7f6139830b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f61241ae670 con 0x564d4cda4160 2024-05-09T01:01:36.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.971+0000 7f6139830b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f61241ad1d0 con 0x564d4ccba4a0 2024-05-09T01:01:36.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.971+0000 7f61293b7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d4ce376a0 0x564d4ce4d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:36.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.971+0000 7f61293b7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d4ce376a0 0x564d4ce4d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58034/0 (socket says 172.21.15.5:58034) 2024-05-09T01:01:36.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.971+0000 7f61293b7700 1 -- 172.21.15.5:0/821497025 learned_addr learned my addr 172.21.15.5:0/821497025 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:01:36.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.971+0000 7f6123fff700 1 --2- 172.21.15.5:0/821497025 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564d4cda4160 0x564d4ce3b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:36.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.971+0000 7f6128bb6700 1 --2- 172.21.15.5:0/821497025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d4ccba4a0 0x564d4ce40190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:36.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.971+0000 7f61293b7700 1 -- 172.21.15.5:0/821497025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d4ccba4a0 msgr2=0x564d4ce40190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:01:36.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.971+0000 7f61293b7700 1 --2- 172.21.15.5:0/821497025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d4ccba4a0 0x564d4ce40190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:36.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.971+0000 7f61293b7700 1 -- 172.21.15.5:0/821497025 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564d4cda4160 msgr2=0x564d4ce3b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:01:36.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.971+0000 7f61293b7700 1 --2- 172.21.15.5:0/821497025 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564d4cda4160 0x564d4ce3b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:36.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.971+0000 7f61293b7700 1 -- 172.21.15.5:0/821497025 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d4ce33cc0 con 0x564d4ce376a0 2024-05-09T01:01:36.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.971+0000 7f61293b7700 1 --2- 172.21.15.5:0/821497025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d4ce376a0 0x564d4ce4d030 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f611c0093c0 tx=0x7f611c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acac593a1b7d3713 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:01:36.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.972+0000 7f611a7fc700 1 -- 172.21.15.5:0/821497025 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611c00bca0 con 0x564d4ce376a0 2024-05-09T01:01:36.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.972+0000 7f611a7fc700 1 -- 172.21.15.5:0/821497025 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f611c00be40 con 0x564d4ce376a0 2024-05-09T01:01:36.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.972+0000 7f611a7fc700 1 -- 172.21.15.5:0/821497025 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611c0075c0 con 0x564d4ce376a0 2024-05-09T01:01:36.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.972+0000 7f6139830b80 1 -- 172.21.15.5:0/821497025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d4ce376a0 msgr2=0x564d4ce4d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:36.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.972+0000 7f6139830b80 1 --2- 172.21.15.5:0/821497025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d4ce376a0 0x564d4ce4d030 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f611c0093c0 tx=0x7f611c00b040 comp rx=0 tx=0).stop 2024-05-09T01:01:36.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.972+0000 7f6139830b80 1 -- 172.21.15.5:0/821497025 shutdown_connections 2024-05-09T01:01:36.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.972+0000 7f6139830b80 1 --2- 172.21.15.5:0/821497025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d4ce376a0 0x564d4ce4d030 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:36.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.973+0000 7f6139830b80 1 --2- 172.21.15.5:0/821497025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d4ccba4a0 0x564d4ce40190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:36.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.973+0000 7f6139830b80 1 --2- 172.21.15.5:0/821497025 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564d4cda4160 0x564d4ce3b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:36.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.973+0000 7f6139830b80 1 -- 172.21.15.5:0/821497025 >> 172.21.15.5:0/821497025 conn(0x564d4cd1a6a0 msgr2=0x564d4ce3ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:01:36.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.973+0000 7f6139830b80 1 -- 172.21.15.5:0/821497025 shutdown_connections 2024-05-09T01:01:36.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.973+0000 7f6139830b80 1 -- 172.21.15.5:0/821497025 wait complete. 2024-05-09T01:01:36.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.973+0000 7f6139830b80 1 -- start start 2024-05-09T01:01:36.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.973+0000 7f6139830b80 1 Processor -- start 2024-05-09T01:01:36.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.973+0000 7f6139830b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d4ccba4a0 0x564d4ce53810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:36.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.973+0000 7f6139830b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d4cda4160 0x564d4ce53d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:36.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.973+0000 7f6139830b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564d4ce376a0 0x564d4ceb9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:36.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.974+0000 7f6139830b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f61241adfa0 con 0x564d4cda4160 2024-05-09T01:01:36.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.974+0000 7f6139830b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f61241ad8a0 con 0x564d4ce376a0 2024-05-09T01:01:36.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.974+0000 7f6139830b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f61241aaaa0 con 0x564d4ccba4a0 2024-05-09T01:01:36.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.974+0000 7f6139830b80 10 client.0 _ll_register_callbacks cb 0x564d4cbcb6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:01:36.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.974+0000 7f6123fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d4cda4160 0x564d4ce53d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:36.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.974+0000 7f6128bb6700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d4ccba4a0 0x564d4ce53810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:36.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.974+0000 7f6128bb6700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d4ccba4a0 0x564d4ce53810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.5:38906/0 (socket says 172.21.15.5:38906) 2024-05-09T01:01:36.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.974+0000 7f6128bb6700 1 -- 172.21.15.5:0/3128751327 learned_addr learned my addr 172.21.15.5:0/3128751327 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:01:36.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.974+0000 7f6123fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d4cda4160 0x564d4ce53d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58050/0 (socket says 172.21.15.5:58050) 2024-05-09T01:01:36.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.974+0000 7f61293b7700 1 --2- 172.21.15.5:0/3128751327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564d4ce376a0 0x564d4ceb9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:36.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.974+0000 7f6128bb6700 1 -- 172.21.15.5:0/3128751327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564d4ce376a0 msgr2=0x564d4ceb9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:01:36.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.974+0000 7f6128bb6700 1 --2- 172.21.15.5:0/3128751327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564d4ce376a0 0x564d4ceb9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:36.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.974+0000 7f6128bb6700 1 -- 172.21.15.5:0/3128751327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d4cda4160 msgr2=0x564d4ce53d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:01:36.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.974+0000 7f6128bb6700 1 --2- 172.21.15.5:0/3128751327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d4cda4160 0x564d4ce53d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:36.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.974+0000 7f6128bb6700 1 -- 172.21.15.5:0/3128751327 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d4ceb9ad0 con 0x564d4ccba4a0 2024-05-09T01:01:36.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.974+0000 7f6123fff700 1 --2- 172.21.15.5:0/3128751327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d4cda4160 0x564d4ce53d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:01:36.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.974+0000 7f6128bb6700 1 --2- 172.21.15.5:0/3128751327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d4ccba4a0 0x564d4ce53810 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6114009eb0 tx=0x7f6114013040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7614f331da82a3d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:01:36.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.974+0000 7f6103fff700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-05-09T01:01:36.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.974+0000 7f6103fff700 1 -- 172.21.15.5:0/3128751327 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611402e070 con 0x564d4ccba4a0 2024-05-09T01:01:36.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.974+0000 7f6139830b80 1 -- 172.21.15.5:0/3128751327 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564d4cd61a70 con 0x564d4ccba4a0 2024-05-09T01:01:36.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.974+0000 7f6103fff700 1 -- 172.21.15.5:0/3128751327 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f611402a880 con 0x564d4ccba4a0 2024-05-09T01:01:36.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.975+0000 7f6103fff700 1 -- 172.21.15.5:0/3128751327 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f6114039050 con 0x564d4ccba4a0 2024-05-09T01:01:36.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.975+0000 7f6139830b80 10 client.26626 fetch_fsmap learned FSMap version 34 2024-05-09T01:01:36.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.975+0000 7f6139830b80 1 -- 172.21.15.5:0/3128751327 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x564d4cebad60 con 0x564d4ccba4a0 2024-05-09T01:01:36.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.975+0000 7f6103fff700 1 -- 172.21.15.5:0/3128751327 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x564d4cd61a70 con 0x564d4ccba4a0 2024-05-09T01:01:36.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.975+0000 7f6103fff700 1 -- 172.21.15.5:0/3128751327 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7f6114005400 con 0x564d4ccba4a0 2024-05-09T01:01:36.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.975+0000 7f6139830b80 10 client.26626 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T01:01:36.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.975+0000 7f6139830b80 10 client.26626 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-05-09T01:01:36.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.975+0000 7f6139830b80 20 client.26626 populate_metadata read hostname 'smithi005' 2024-05-09T01:01:36.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.975+0000 7f6139830b80 1 --2- 172.21.15.5:0/3128751327 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x564d4cebb420 0x564d4cebd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:36.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.975+0000 7f6139830b80 4 client.26626 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:01:36.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.975+0000 7f6139830b80 1 -- 172.21.15.5:0/3128751327 --> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564d4cbd5970 con 0x564d4cebb420 2024-05-09T01:01:36.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.975+0000 7f6123fff700 1 --2- 172.21.15.5:0/3128751327 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x564d4cebb420 0x564d4cebd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:36.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.976+0000 7f6103fff700 10 client.26626 ms_handle_connect on v2:172.21.15.131:6838/2748669631 2024-05-09T01:01:36.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.976+0000 7f6123fff700 1 --2- 172.21.15.5:0/3128751327 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x564d4cebb420 0x564d4cebd840 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cb691eed173998a5 server_cookie=6af9aa6e2e6343c5 in_seq=0 out_seq=0 2024-05-09T01:01:36.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.976+0000 7f6103fff700 1 -- 172.21.15.5:0/3128751327 <== mds.2 v2:172.21.15.131:6838/2748669631 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564d4cbd5970 con 0x564d4cebb420 2024-05-09T01:01:36.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.976+0000 7f6103fff700 10 client.26626 handle_command_reply: tid=0 2024-05-09T01:01:36.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.976+0000 7f6139830b80 1 client.26626 shutdown 2024-05-09T01:01:36.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.976+0000 7f6139830b80 1 -- 172.21.15.5:0/3128751327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d4ccba4a0 msgr2=0x564d4ce53810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:36.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.976+0000 7f6139830b80 1 --2- 172.21.15.5:0/3128751327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d4ccba4a0 0x564d4ce53810 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6114009eb0 tx=0x7f6114013040 comp rx=0 tx=0).stop 2024-05-09T01:01:36.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.976+0000 7f6139830b80 1 -- 172.21.15.5:0/3128751327 shutdown_connections 2024-05-09T01:01:36.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.976+0000 7f6139830b80 1 --2- 172.21.15.5:0/3128751327 >> [v2:172.21.15.131:6838/2748669631,v1:172.21.15.131:6839/2748669631] conn(0x564d4cebb420 0x564d4cebd840 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:36.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.976+0000 7f6139830b80 1 --2- 172.21.15.5:0/3128751327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564d4ce376a0 0x564d4ceb9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:36.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.976+0000 7f6139830b80 1 --2- 172.21.15.5:0/3128751327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d4ccba4a0 0x564d4ce53810 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:36.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.976+0000 7f6139830b80 1 --2- 172.21.15.5:0/3128751327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d4cda4160 0x564d4ce53d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:36.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.976+0000 7f6103fff700 0 client.26626 ms_handle_reset on v2:172.21.15.131:6838/2748669631 2024-05-09T01:01:36.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.976+0000 7f6139830b80 1 -- 172.21.15.5:0/3128751327 >> 172.21.15.5:0/3128751327 conn(0x564d4cd1a6a0 msgr2=0x564d4ce36690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:01:36.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.976+0000 7f6139830b80 1 -- 172.21.15.5:0/3128751327 shutdown_connections 2024-05-09T01:01:36.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.976+0000 7f6139830b80 1 -- 172.21.15.5:0/3128751327 wait complete. 2024-05-09T01:01:36.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.976+0000 7f6139830b80 20 client.26626 trim_cache size 0 max 16384 2024-05-09T01:01:36.983 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-09T01:01:36.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.977+0000 7f612ae1a700 1 -- 172.21.15.5:0/4127621423 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f61100b2880 msgr2=0x7f61100b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:36.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.977+0000 7f612ae1a700 1 --2- 172.21.15.5:0/4127621423 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f61100b2880 0x7f61100b4d90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f6114009c60 tx=0x7f6114009260 comp rx=0 tx=0).stop 2024-05-09T01:01:36.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.977+0000 7f612ae1a700 1 -- 172.21.15.5:0/4127621423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6124107e10 msgr2=0x7f612419b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:36.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.977+0000 7f612ae1a700 1 --2- 172.21.15.5:0/4127621423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6124107e10 0x7f612419b350 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f610c00d980 tx=0x7f610c00dc90 comp rx=0 tx=0).stop 2024-05-09T01:01:36.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.977+0000 7f612ae1a700 1 -- 172.21.15.5:0/4127621423 shutdown_connections 2024-05-09T01:01:36.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.977+0000 7f612ae1a700 1 --2- 172.21.15.5:0/4127621423 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f61100b2880 0x7f61100b4d90 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:36.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.977+0000 7f612ae1a700 1 --2- 172.21.15.5:0/4127621423 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6124108760 0x7f612419f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:36.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.977+0000 7f612ae1a700 1 --2- 172.21.15.5:0/4127621423 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61241034e0 0x7f612419adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:36.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.977+0000 7f612ae1a700 1 --2- 172.21.15.5:0/4127621423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6124107e10 0x7f612419b350 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:36.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.977+0000 7f612ae1a700 1 -- 172.21.15.5:0/4127621423 >> 172.21.15.5:0/4127621423 conn(0x7f61240fef30 msgr2=0x7f612406c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:01:36.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.977+0000 7f612ae1a700 1 -- 172.21.15.5:0/4127621423 shutdown_connections 2024-05-09T01:01:36.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:36.977+0000 7f612ae1a700 1 -- 172.21.15.5:0/4127621423 wait complete. 2024-05-09T01:01:37.897 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T01:01:38.167 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:01:38.411 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:01:38 smithi005 ceph-mon[30365]: pgmap v829: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 37 MiB/s wr, 179 op/s 2024-05-09T01:01:38.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:01:38 smithi157 ceph-mon[34294]: pgmap v829: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 37 MiB/s wr, 179 op/s 2024-05-09T01:01:38.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:01:38 smithi131 ceph-mon[34547]: pgmap v829: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 37 MiB/s wr, 179 op/s 2024-05-09T01:01:39.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.919+0000 7f77474ba700 1 -- 172.21.15.5:0/2218677595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7740109880 msgr2=0x7f77401111e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:39.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.919+0000 7f77474ba700 1 --2- 172.21.15.5:0/2218677595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7740109880 0x7f77401111e0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f773800b0d0 tx=0x7f773800b3e0 comp rx=0 tx=0).stop 2024-05-09T01:01:39.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.919+0000 7f77474ba700 1 -- 172.21.15.5:0/2218677595 shutdown_connections 2024-05-09T01:01:39.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.919+0000 7f77474ba700 1 --2- 172.21.15.5:0/2218677595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7740109880 0x7f77401111e0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:39.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.919+0000 7f77474ba700 1 --2- 172.21.15.5:0/2218677595 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7740104650 0x7f7740104a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:39.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.919+0000 7f77474ba700 1 --2- 172.21.15.5:0/2218677595 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7740108f30 0x7f7740109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:39.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.919+0000 7f77474ba700 1 -- 172.21.15.5:0/2218677595 >> 172.21.15.5:0/2218677595 conn(0x7f7740079f00 msgr2=0x7f774007a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:01:39.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.919+0000 7f77474ba700 1 -- 172.21.15.5:0/2218677595 shutdown_connections 2024-05-09T01:01:39.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.919+0000 7f77474ba700 1 -- 172.21.15.5:0/2218677595 wait complete. 2024-05-09T01:01:39.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.920+0000 7f77474ba700 1 Processor -- start 2024-05-09T01:01:39.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.920+0000 7f77474ba700 1 -- start start 2024-05-09T01:01:39.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.920+0000 7f77474ba700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7740104650 0x7f774019adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:39.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.920+0000 7f77474ba700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7740108f30 0x7f774019b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:39.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.920+0000 7f77474ba700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7740109880 0x7f774019f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:39.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.920+0000 7f77474ba700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f77401144b0 con 0x7f7740108f30 2024-05-09T01:01:39.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.921+0000 7f77474ba700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7740114330 con 0x7f7740104650 2024-05-09T01:01:39.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.921+0000 7f77474ba700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7740114630 con 0x7f7740109880 2024-05-09T01:01:39.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.921+0000 7f7744a55700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7740108f30 0x7f774019b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:39.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.921+0000 7f7744a55700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7740108f30 0x7f774019b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58074/0 (socket says 172.21.15.5:58074) 2024-05-09T01:01:39.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.921+0000 7f7744a55700 1 -- 172.21.15.5:0/3583620331 learned_addr learned my addr 172.21.15.5:0/3583620331 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:01:39.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.921+0000 7f7745256700 1 --2- 172.21.15.5:0/3583620331 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7740104650 0x7f774019adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:39.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.921+0000 7f7745a57700 1 --2- 172.21.15.5:0/3583620331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7740109880 0x7f774019f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:39.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.921+0000 7f7744a55700 1 -- 172.21.15.5:0/3583620331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7740109880 msgr2=0x7f774019f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:39.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.921+0000 7f7744a55700 1 --2- 172.21.15.5:0/3583620331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7740109880 0x7f774019f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:39.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.921+0000 7f7744a55700 1 -- 172.21.15.5:0/3583620331 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7740104650 msgr2=0x7f774019adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:39.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.921+0000 7f7744a55700 1 --2- 172.21.15.5:0/3583620331 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7740104650 0x7f774019adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:39.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.921+0000 7f7744a55700 1 -- 172.21.15.5:0/3583620331 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f774019fbe0 con 0x7f7740108f30 2024-05-09T01:01:39.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.921+0000 7f7745a57700 1 --2- 172.21.15.5:0/3583620331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7740109880 0x7f774019f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:01:39.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.921+0000 7f7744a55700 1 --2- 172.21.15.5:0/3583620331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7740108f30 0x7f774019b360 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f773000c9e0 tx=0x7f773000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:01:39.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.921+0000 7f77367fc700 1 -- 172.21.15.5:0/3583620331 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7730007aa0 con 0x7f7740108f30 2024-05-09T01:01:39.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.921+0000 7f77474ba700 1 -- 172.21.15.5:0/3583620331 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f774019fed0 con 0x7f7740108f30 2024-05-09T01:01:39.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.921+0000 7f77367fc700 1 -- 172.21.15.5:0/3583620331 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7730007c40 con 0x7f7740108f30 2024-05-09T01:01:39.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.921+0000 7f77367fc700 1 -- 172.21.15.5:0/3583620331 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7730018660 con 0x7f7740108f30 2024-05-09T01:01:39.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.921+0000 7f77474ba700 1 -- 172.21.15.5:0/3583620331 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77401a0410 con 0x7f7740108f30 2024-05-09T01:01:39.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.924+0000 7f77367fc700 1 -- 172.21.15.5:0/3583620331 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f7730004830 con 0x7f7740108f30 2024-05-09T01:01:39.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.925+0000 7f77367fc700 1 --2- 172.21.15.5:0/3583620331 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f772c0b2880 0x7f772c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:39.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.925+0000 7f7745256700 1 --2- 172.21.15.5:0/3583620331 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f772c0b2880 0x7f772c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:39.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.925+0000 7f77367fc700 1 -- 172.21.15.5:0/3583620331 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f77300e2200 con 0x7f7740108f30 2024-05-09T01:01:39.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.925+0000 7f7755ed0b80 1 Processor -- start 2024-05-09T01:01:39.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.926+0000 7f7755ed0b80 1 -- start start 2024-05-09T01:01:39.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.926+0000 7f7745256700 1 --2- 172.21.15.5:0/3583620331 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f772c0b2880 0x7f772c0b4d90 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f773c007be0 tx=0x7f773c0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:01:39.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.926+0000 7f7755ed0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556a07a2160 0x5556a083b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:39.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.926+0000 7f7755ed0b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556a06b84a0 0x5556a083ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:39.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.926+0000 7f7755ed0b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556a08356a0 0x5556a0835ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:39.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.926+0000 7f7755ed0b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5556a0757ff0 con 0x5556a06b84a0 2024-05-09T01:01:39.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.926+0000 7f7755ed0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5556a06f67f0 con 0x5556a07a2160 2024-05-09T01:01:39.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.926+0000 7f7755ed0b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5556a0753fb0 con 0x5556a08356a0 2024-05-09T01:01:39.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.926+0000 7f7744a55700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556a06b84a0 0x5556a083ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:39.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.926+0000 7f7744a55700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556a06b84a0 0x5556a083ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58082/0 (socket says 172.21.15.5:58082) 2024-05-09T01:01:39.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.926+0000 7f7744a55700 1 -- 172.21.15.5:0/2121155031 learned_addr learned my addr 172.21.15.5:0/2121155031 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:01:39.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.926+0000 7f7745256700 1 --2- 172.21.15.5:0/2121155031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556a07a2160 0x5556a083b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:39.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.926+0000 7f7745a57700 1 --2- 172.21.15.5:0/2121155031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556a08356a0 0x5556a0835ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:39.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.926+0000 7f7744a55700 1 -- 172.21.15.5:0/2121155031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556a08356a0 msgr2=0x5556a0835ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:01:39.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.926+0000 7f7744a55700 1 --2- 172.21.15.5:0/2121155031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556a08356a0 0x5556a0835ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:39.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.926+0000 7f7744a55700 1 -- 172.21.15.5:0/2121155031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556a07a2160 msgr2=0x5556a083b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:01:39.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.926+0000 7f7744a55700 1 --2- 172.21.15.5:0/2121155031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556a07a2160 0x5556a083b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:39.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.926+0000 7f7744a55700 1 -- 172.21.15.5:0/2121155031 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556a06ee370 con 0x5556a06b84a0 2024-05-09T01:01:39.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.927+0000 7f7744a55700 1 --2- 172.21.15.5:0/2121155031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556a06b84a0 0x5556a083ba30 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f77300a6a00 tx=0x7f77300a9670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a378c98810210b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:01:39.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.927+0000 7f772b7fe700 1 -- 172.21.15.5:0/2121155031 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77300c2020 con 0x5556a06b84a0 2024-05-09T01:01:39.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.927+0000 7f772b7fe700 1 -- 172.21.15.5:0/2121155031 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77300cc0b0 con 0x5556a06b84a0 2024-05-09T01:01:39.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.927+0000 7f772b7fe700 1 -- 172.21.15.5:0/2121155031 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77300b2160 con 0x5556a06b84a0 2024-05-09T01:01:39.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.928+0000 7f7755ed0b80 1 -- 172.21.15.5:0/2121155031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556a06b84a0 msgr2=0x5556a083ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:39.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.928+0000 7f7755ed0b80 1 --2- 172.21.15.5:0/2121155031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556a06b84a0 0x5556a083ba30 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f77300a6a00 tx=0x7f77300a9670 comp rx=0 tx=0).stop 2024-05-09T01:01:39.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.928+0000 7f7755ed0b80 1 -- 172.21.15.5:0/2121155031 shutdown_connections 2024-05-09T01:01:39.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.928+0000 7f7755ed0b80 1 --2- 172.21.15.5:0/2121155031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556a08356a0 0x5556a0835ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:39.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.928+0000 7f7755ed0b80 1 --2- 172.21.15.5:0/2121155031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556a07a2160 0x5556a083b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:39.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.928+0000 7f7755ed0b80 1 --2- 172.21.15.5:0/2121155031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556a06b84a0 0x5556a083ba30 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:39.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.928+0000 7f7755ed0b80 1 -- 172.21.15.5:0/2121155031 >> 172.21.15.5:0/2121155031 conn(0x5556a07186a0 msgr2=0x5556a0830400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:01:39.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.928+0000 7f7755ed0b80 1 -- 172.21.15.5:0/2121155031 shutdown_connections 2024-05-09T01:01:39.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.928+0000 7f7755ed0b80 1 -- 172.21.15.5:0/2121155031 wait complete. 2024-05-09T01:01:39.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.928+0000 7f7755ed0b80 1 -- start start 2024-05-09T01:01:39.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.929+0000 7f7755ed0b80 1 Processor -- start 2024-05-09T01:01:39.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.929+0000 7f7755ed0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556a06b84a0 0x5556a083f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:39.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.929+0000 7f7755ed0b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556a07a2160 0x5556a083f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:39.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.929+0000 7f7755ed0b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556a08356a0 0x5556a083fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:39.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.929+0000 7f7755ed0b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5556a05d3ba0 con 0x5556a08356a0 2024-05-09T01:01:39.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.929+0000 7f7755ed0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5556a0712150 con 0x5556a06b84a0 2024-05-09T01:01:39.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.929+0000 7f7755ed0b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5556a08421d0 con 0x5556a07a2160 2024-05-09T01:01:39.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.929+0000 7f7755ed0b80 10 client.0 _ll_register_callbacks cb 0x5556a05c96c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:01:39.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.929+0000 7f7745a57700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556a08356a0 0x5556a083fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:39.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.929+0000 7f7745a57700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556a08356a0 0x5556a083fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58090/0 (socket says 172.21.15.5:58090) 2024-05-09T01:01:39.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.929+0000 7f7745a57700 1 -- 172.21.15.5:0/623394886 learned_addr learned my addr 172.21.15.5:0/623394886 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:01:39.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.929+0000 7f7744a55700 1 --2- 172.21.15.5:0/623394886 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556a07a2160 0x5556a083f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:39.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.929+0000 7f7745256700 1 --2- 172.21.15.5:0/623394886 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556a06b84a0 0x5556a083f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:39.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.930+0000 7f7745a57700 1 -- 172.21.15.5:0/623394886 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556a07a2160 msgr2=0x5556a083f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:01:39.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.930+0000 7f7745a57700 1 --2- 172.21.15.5:0/623394886 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556a07a2160 0x5556a083f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:39.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.930+0000 7f7745a57700 1 -- 172.21.15.5:0/623394886 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556a06b84a0 msgr2=0x5556a083f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:01:39.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.930+0000 7f7745a57700 1 --2- 172.21.15.5:0/623394886 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556a06b84a0 0x5556a083f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:39.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.930+0000 7f7745a57700 1 -- 172.21.15.5:0/623394886 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556a05d3490 con 0x5556a08356a0 2024-05-09T01:01:39.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.930+0000 7f7745256700 1 --2- 172.21.15.5:0/623394886 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556a06b84a0 0x5556a083f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:01:39.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.930+0000 7f7744a55700 1 --2- 172.21.15.5:0/623394886 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556a07a2160 0x5556a083f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:01:39.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.930+0000 7f7745a57700 1 --2- 172.21.15.5:0/623394886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556a08356a0 0x5556a083fce0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f7738004d00 tx=0x7f7738004de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=570c21625e986ddc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:01:39.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.930+0000 7f7728ff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T01:01:39.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.930+0000 7f7728ff9700 1 -- 172.21.15.5:0/623394886 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7738021070 con 0x5556a08356a0 2024-05-09T01:01:39.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.930+0000 7f7728ff9700 1 -- 172.21.15.5:0/623394886 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f77380079d0 con 0x5556a08356a0 2024-05-09T01:01:39.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.930+0000 7f7755ed0b80 1 -- 172.21.15.5:0/623394886 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77300cc0b0 con 0x5556a08356a0 2024-05-09T01:01:39.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.930+0000 7f7728ff9700 1 -- 172.21.15.5:0/623394886 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f773802b030 con 0x5556a08356a0 2024-05-09T01:01:39.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.930+0000 7f7755ed0b80 10 client.17259 fetch_fsmap learned FSMap version 34 2024-05-09T01:01:39.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.930+0000 7f7755ed0b80 1 -- 172.21.15.5:0/623394886 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5556a084cc40 con 0x5556a08356a0 2024-05-09T01:01:39.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.930+0000 7f7728ff9700 1 -- 172.21.15.5:0/623394886 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77300cc0b0 con 0x5556a08356a0 2024-05-09T01:01:39.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.931+0000 7f7728ff9700 1 -- 172.21.15.5:0/623394886 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7f7738011070 con 0x5556a08356a0 2024-05-09T01:01:39.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.931+0000 7f7755ed0b80 10 client.17259 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T01:01:39.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.931+0000 7f7755ed0b80 10 client.17259 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-05-09T01:01:39.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.931+0000 7f7755ed0b80 20 client.17259 populate_metadata read hostname 'smithi005' 2024-05-09T01:01:39.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.931+0000 7f7755ed0b80 1 --2- 172.21.15.5:0/623394886 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x5556a084f320 0x5556a0851760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:39.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.931+0000 7f7755ed0b80 4 client.17259 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:01:39.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.931+0000 7f7755ed0b80 1 -- 172.21.15.5:0/623394886 --> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5556a07659d0 con 0x5556a084f320 2024-05-09T01:01:39.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.931+0000 7f7745256700 1 --2- 172.21.15.5:0/623394886 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x5556a084f320 0x5556a0851760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:39.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.931+0000 7f7728ff9700 10 client.17259 ms_handle_connect on v2:172.21.15.157:6836/744863009 2024-05-09T01:01:39.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.931+0000 7f7745256700 1 --2- 172.21.15.5:0/623394886 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x5556a084f320 0x5556a0851760 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=46f91fd8ff508f20 server_cookie=a58bd959eaa89904 in_seq=0 out_seq=0 2024-05-09T01:01:39.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.932+0000 7f7728ff9700 1 -- 172.21.15.5:0/623394886 <== mds.1 v2:172.21.15.157:6836/744863009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5556a07659d0 con 0x5556a084f320 2024-05-09T01:01:39.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.932+0000 7f7728ff9700 10 client.17259 handle_command_reply: tid=0 2024-05-09T01:01:39.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.932+0000 7f7755ed0b80 1 client.17259 shutdown 2024-05-09T01:01:39.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.932+0000 7f7755ed0b80 1 -- 172.21.15.5:0/623394886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556a08356a0 msgr2=0x5556a083fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:39.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.932+0000 7f7755ed0b80 1 --2- 172.21.15.5:0/623394886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556a08356a0 0x5556a083fce0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f7738004d00 tx=0x7f7738004de0 comp rx=0 tx=0).stop 2024-05-09T01:01:39.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.932+0000 7f7755ed0b80 1 -- 172.21.15.5:0/623394886 shutdown_connections 2024-05-09T01:01:39.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.932+0000 7f7755ed0b80 1 --2- 172.21.15.5:0/623394886 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x5556a084f320 0x5556a0851760 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:39.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.932+0000 7f7755ed0b80 1 --2- 172.21.15.5:0/623394886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556a08356a0 0x5556a083fce0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:39.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.932+0000 7f7755ed0b80 1 --2- 172.21.15.5:0/623394886 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556a06b84a0 0x5556a083f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:39.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.932+0000 7f7728ff9700 0 client.17259 ms_handle_reset on v2:172.21.15.157:6836/744863009 2024-05-09T01:01:39.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.932+0000 7f7755ed0b80 1 --2- 172.21.15.5:0/623394886 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556a07a2160 0x5556a083f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:39.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.932+0000 7f7755ed0b80 1 -- 172.21.15.5:0/623394886 >> 172.21.15.5:0/623394886 conn(0x5556a07186a0 msgr2=0x5556a08301c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:01:39.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.933+0000 7f7755ed0b80 1 -- 172.21.15.5:0/623394886 shutdown_connections 2024-05-09T01:01:39.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.933+0000 7f7755ed0b80 1 -- 172.21.15.5:0/623394886 wait complete. 2024-05-09T01:01:39.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.933+0000 7f7755ed0b80 20 client.17259 trim_cache size 0 max 16384 2024-05-09T01:01:39.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.948+0000 7f7755ed0b80 1 Processor -- start 2024-05-09T01:01:39.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.948+0000 7f7755ed0b80 1 -- start start 2024-05-09T01:01:39.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.948+0000 7f7755ed0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556a06b84a0 0x5556a083fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:39.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.948+0000 7f7755ed0b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556a07a2160 0x5556a083a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:39.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.948+0000 7f7755ed0b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556a08356a0 0x5556a083a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:39.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.948+0000 7f7755ed0b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f77401aeb30 con 0x5556a07a2160 2024-05-09T01:01:39.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.948+0000 7f7755ed0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f77401b06a0 con 0x5556a06b84a0 2024-05-09T01:01:39.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.948+0000 7f7755ed0b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f77401af200 con 0x5556a08356a0 2024-05-09T01:01:39.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.948+0000 7f7744a55700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556a07a2160 0x5556a083a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:39.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.948+0000 7f7744a55700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556a07a2160 0x5556a083a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58094/0 (socket says 172.21.15.5:58094) 2024-05-09T01:01:39.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.948+0000 7f7744a55700 1 -- 172.21.15.5:0/814127082 learned_addr learned my addr 172.21.15.5:0/814127082 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:01:39.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.948+0000 7f7745256700 1 --2- 172.21.15.5:0/814127082 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556a06b84a0 0x5556a083fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:39.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.948+0000 7f7745a57700 1 --2- 172.21.15.5:0/814127082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556a08356a0 0x5556a083a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:39.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.949+0000 7f7744a55700 1 -- 172.21.15.5:0/814127082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556a08356a0 msgr2=0x5556a083a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:01:39.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.949+0000 7f7744a55700 1 --2- 172.21.15.5:0/814127082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556a08356a0 0x5556a083a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:39.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.949+0000 7f7744a55700 1 -- 172.21.15.5:0/814127082 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556a06b84a0 msgr2=0x5556a083fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:01:39.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.949+0000 7f7744a55700 1 --2- 172.21.15.5:0/814127082 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556a06b84a0 0x5556a083fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:39.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.949+0000 7f7744a55700 1 -- 172.21.15.5:0/814127082 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556a0841b00 con 0x5556a07a2160 2024-05-09T01:01:39.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.949+0000 7f7745256700 1 --2- 172.21.15.5:0/814127082 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556a06b84a0 0x5556a083fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:01:39.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.949+0000 7f7745a57700 1 --2- 172.21.15.5:0/814127082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556a08356a0 0x5556a083a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:01:39.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.949+0000 7f7744a55700 1 --2- 172.21.15.5:0/814127082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556a07a2160 0x5556a083a350 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f77300074a0 tx=0x7f773000c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce81653d68392097 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:01:39.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.949+0000 7f772affd700 1 -- 172.21.15.5:0/814127082 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77300c2070 con 0x5556a07a2160 2024-05-09T01:01:39.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.949+0000 7f772affd700 1 -- 172.21.15.5:0/814127082 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77300c7020 con 0x5556a07a2160 2024-05-09T01:01:39.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.949+0000 7f772affd700 1 -- 172.21.15.5:0/814127082 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77300bee40 con 0x5556a07a2160 2024-05-09T01:01:39.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.950+0000 7f7755ed0b80 1 -- 172.21.15.5:0/814127082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556a07a2160 msgr2=0x5556a083a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:39.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.950+0000 7f7755ed0b80 1 --2- 172.21.15.5:0/814127082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556a07a2160 0x5556a083a350 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f77300074a0 tx=0x7f773000c930 comp rx=0 tx=0).stop 2024-05-09T01:01:39.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.950+0000 7f7755ed0b80 1 -- 172.21.15.5:0/814127082 shutdown_connections 2024-05-09T01:01:39.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.950+0000 7f7755ed0b80 1 --2- 172.21.15.5:0/814127082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556a08356a0 0x5556a083a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:39.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.950+0000 7f7755ed0b80 1 --2- 172.21.15.5:0/814127082 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556a06b84a0 0x5556a083fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:39.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.950+0000 7f7755ed0b80 1 --2- 172.21.15.5:0/814127082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556a07a2160 0x5556a083a350 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:39.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.950+0000 7f7755ed0b80 1 -- 172.21.15.5:0/814127082 >> 172.21.15.5:0/814127082 conn(0x5556a07186a0 msgr2=0x5556a083db50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:01:39.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.950+0000 7f7755ed0b80 1 -- 172.21.15.5:0/814127082 shutdown_connections 2024-05-09T01:01:39.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.950+0000 7f7755ed0b80 1 -- 172.21.15.5:0/814127082 wait complete. 2024-05-09T01:01:39.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.950+0000 7f7755ed0b80 1 -- start start 2024-05-09T01:01:39.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.950+0000 7f7755ed0b80 1 Processor -- start 2024-05-09T01:01:39.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.951+0000 7f7755ed0b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556a06b84a0 0x5556a0847500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:39.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.951+0000 7f7755ed0b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556a07a2160 0x5556a0847a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:39.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.951+0000 7f7755ed0b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556a08356a0 0x5556a0856270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:39.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.951+0000 7f7755ed0b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f77401affd0 con 0x5556a07a2160 2024-05-09T01:01:39.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.951+0000 7f7755ed0b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f77401af8d0 con 0x5556a06b84a0 2024-05-09T01:01:39.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.951+0000 7f7755ed0b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f77401acad0 con 0x5556a08356a0 2024-05-09T01:01:39.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.951+0000 7f7755ed0b80 10 client.0 _ll_register_callbacks cb 0x5556a05c96c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:01:39.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.951+0000 7f7744a55700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556a07a2160 0x5556a0847a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:39.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.951+0000 7f7744a55700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556a07a2160 0x5556a0847a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58096/0 (socket says 172.21.15.5:58096) 2024-05-09T01:01:39.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.951+0000 7f7744a55700 1 -- 172.21.15.5:0/3416864665 learned_addr learned my addr 172.21.15.5:0/3416864665 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:01:39.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.951+0000 7f7745256700 1 --2- 172.21.15.5:0/3416864665 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556a06b84a0 0x5556a0847500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:39.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.951+0000 7f7745a57700 1 --2- 172.21.15.5:0/3416864665 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556a08356a0 0x5556a0856270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:39.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.951+0000 7f7744a55700 1 -- 172.21.15.5:0/3416864665 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556a08356a0 msgr2=0x5556a0856270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:01:39.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.951+0000 7f7744a55700 1 --2- 172.21.15.5:0/3416864665 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556a08356a0 0x5556a0856270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:39.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.951+0000 7f7744a55700 1 -- 172.21.15.5:0/3416864665 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556a06b84a0 msgr2=0x5556a0847500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:01:39.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.951+0000 7f7744a55700 1 --2- 172.21.15.5:0/3416864665 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556a06b84a0 0x5556a0847500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:39.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.951+0000 7f7744a55700 1 -- 172.21.15.5:0/3416864665 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556a0856a10 con 0x5556a07a2160 2024-05-09T01:01:39.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.951+0000 7f7745a57700 1 --2- 172.21.15.5:0/3416864665 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556a08356a0 0x5556a0856270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:01:39.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.952+0000 7f7744a55700 1 --2- 172.21.15.5:0/3416864665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556a07a2160 0x5556a0847a60 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f773000ffd0 tx=0x7f773000a6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58181a1fcc574e36 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:01:39.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.952+0000 7f7728ff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T01:01:39.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.952+0000 7f7728ff9700 1 -- 172.21.15.5:0/3416864665 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77300c2030 con 0x5556a07a2160 2024-05-09T01:01:39.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.952+0000 7f7728ff9700 1 -- 172.21.15.5:0/3416864665 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f77300c8070 con 0x5556a07a2160 2024-05-09T01:01:39.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.952+0000 7f7755ed0b80 1 -- 172.21.15.5:0/3416864665 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77300c7020 con 0x5556a07a2160 2024-05-09T01:01:39.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.952+0000 7f7728ff9700 1 -- 172.21.15.5:0/3416864665 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f77300b25f0 con 0x5556a07a2160 2024-05-09T01:01:39.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.952+0000 7f7755ed0b80 10 client.17265 fetch_fsmap learned FSMap version 34 2024-05-09T01:01:39.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.952+0000 7f7755ed0b80 1 -- 172.21.15.5:0/3416864665 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5556a08b9550 con 0x5556a07a2160 2024-05-09T01:01:39.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.952+0000 7f7728ff9700 1 -- 172.21.15.5:0/3416864665 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77300c7020 con 0x5556a07a2160 2024-05-09T01:01:39.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.952+0000 7f7728ff9700 1 -- 172.21.15.5:0/3416864665 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7f77300a5100 con 0x5556a07a2160 2024-05-09T01:01:39.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.952+0000 7f7755ed0b80 10 client.17265 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T01:01:39.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.952+0000 7f7755ed0b80 10 client.17265 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-05-09T01:01:39.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.952+0000 7f7755ed0b80 20 client.17265 populate_metadata read hostname 'smithi005' 2024-05-09T01:01:39.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.952+0000 7f7755ed0b80 1 --2- 172.21.15.5:0/3416864665 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x5556a08b97d0 0x5556a08bbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:39.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.952+0000 7f7755ed0b80 4 client.17265 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:01:39.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.952+0000 7f7755ed0b80 1 -- 172.21.15.5:0/3416864665 --> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5556a075fa70 con 0x5556a08b97d0 2024-05-09T01:01:39.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.953+0000 7f7745256700 1 --2- 172.21.15.5:0/3416864665 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x5556a08b97d0 0x5556a08bbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:39.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.953+0000 7f7728ff9700 10 client.17265 ms_handle_connect on v2:172.21.15.157:6836/744863009 2024-05-09T01:01:39.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.953+0000 7f7745256700 1 --2- 172.21.15.5:0/3416864665 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x5556a08b97d0 0x5556a08bbbf0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1a6dad0bc341863b server_cookie=e8ded92bd98fca13 in_seq=0 out_seq=0 2024-05-09T01:01:39.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.953+0000 7f7728ff9700 1 -- 172.21.15.5:0/3416864665 <== mds.1 v2:172.21.15.157:6836/744863009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5556a075fa70 con 0x5556a08b97d0 2024-05-09T01:01:39.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.953+0000 7f7728ff9700 10 client.17265 handle_command_reply: tid=0 2024-05-09T01:01:39.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.953+0000 7f7755ed0b80 1 client.17265 shutdown 2024-05-09T01:01:39.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.953+0000 7f7755ed0b80 1 -- 172.21.15.5:0/3416864665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556a07a2160 msgr2=0x5556a0847a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:39.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.953+0000 7f7755ed0b80 1 --2- 172.21.15.5:0/3416864665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556a07a2160 0x5556a0847a60 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f773000ffd0 tx=0x7f773000a6b0 comp rx=0 tx=0).stop 2024-05-09T01:01:39.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.953+0000 7f7755ed0b80 1 -- 172.21.15.5:0/3416864665 shutdown_connections 2024-05-09T01:01:39.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.953+0000 7f7755ed0b80 1 --2- 172.21.15.5:0/3416864665 >> [v2:172.21.15.157:6836/744863009,v1:172.21.15.157:6837/744863009] conn(0x5556a08b97d0 0x5556a08bbbf0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:39.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.954+0000 7f7755ed0b80 1 --2- 172.21.15.5:0/3416864665 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556a08356a0 0x5556a0856270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:39.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.954+0000 7f7755ed0b80 1 --2- 172.21.15.5:0/3416864665 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556a06b84a0 0x5556a0847500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:39.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.954+0000 7f7755ed0b80 1 --2- 172.21.15.5:0/3416864665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556a07a2160 0x5556a0847a60 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:39.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.954+0000 7f7755ed0b80 1 -- 172.21.15.5:0/3416864665 >> 172.21.15.5:0/3416864665 conn(0x5556a07186a0 msgr2=0x5556a0838e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:01:39.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.954+0000 7f7728ff9700 0 client.17265 ms_handle_reset on v2:172.21.15.157:6836/744863009 2024-05-09T01:01:39.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.954+0000 7f7755ed0b80 1 -- 172.21.15.5:0/3416864665 shutdown_connections 2024-05-09T01:01:39.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.954+0000 7f7755ed0b80 1 -- 172.21.15.5:0/3416864665 wait complete. 2024-05-09T01:01:39.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.954+0000 7f7755ed0b80 20 client.17265 trim_cache size 0 max 16384 2024-05-09T01:01:39.961 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-09T01:01:39.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.954+0000 7f77474ba700 1 -- 172.21.15.5:0/3583620331 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f772c0b2880 msgr2=0x7f772c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:39.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.954+0000 7f77474ba700 1 --2- 172.21.15.5:0/3583620331 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f772c0b2880 0x7f772c0b4d90 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f773c007be0 tx=0x7f773c0073d0 comp rx=0 tx=0).stop 2024-05-09T01:01:39.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.954+0000 7f77474ba700 1 -- 172.21.15.5:0/3583620331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7740108f30 msgr2=0x7f774019b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:39.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.954+0000 7f77474ba700 1 --2- 172.21.15.5:0/3583620331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7740108f30 0x7f774019b360 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f773000c9e0 tx=0x7f773000ccf0 comp rx=0 tx=0).stop 2024-05-09T01:01:39.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.954+0000 7f77474ba700 1 -- 172.21.15.5:0/3583620331 shutdown_connections 2024-05-09T01:01:39.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.954+0000 7f77474ba700 1 --2- 172.21.15.5:0/3583620331 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f772c0b2880 0x7f772c0b4d90 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:39.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.954+0000 7f77474ba700 1 --2- 172.21.15.5:0/3583620331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7740109880 0x7f774019f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:39.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.954+0000 7f77474ba700 1 --2- 172.21.15.5:0/3583620331 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7740104650 0x7f774019adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:39.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.954+0000 7f77474ba700 1 --2- 172.21.15.5:0/3583620331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7740108f30 0x7f774019b360 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:39.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.954+0000 7f77474ba700 1 -- 172.21.15.5:0/3583620331 >> 172.21.15.5:0/3583620331 conn(0x7f7740079f00 msgr2=0x7f7740101f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:01:39.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.955+0000 7f77474ba700 1 -- 172.21.15.5:0/3583620331 shutdown_connections 2024-05-09T01:01:39.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:39.955+0000 7f77474ba700 1 -- 172.21.15.5:0/3583620331 wait complete. 2024-05-09T01:01:40.414 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:01:40 smithi005 ceph-mon[30365]: pgmap v830: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 45 MiB/s wr, 221 op/s 2024-05-09T01:01:40.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:01:40 smithi157 ceph-mon[34294]: pgmap v830: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 45 MiB/s wr, 221 op/s 2024-05-09T01:01:40.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:01:40 smithi131 ceph-mon[34547]: pgmap v830: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 45 MiB/s wr, 221 op/s 2024-05-09T01:01:40.705 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T01:01:40.964 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:01:42.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:01:42 smithi157 ceph-mon[34294]: pgmap v831: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 55 MiB/s wr, 262 op/s 2024-05-09T01:01:42.451 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:01:42 smithi005 ceph-mon[30365]: pgmap v831: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 55 MiB/s wr, 262 op/s 2024-05-09T01:01:42.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:01:42 smithi131 ceph-mon[34547]: pgmap v831: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 55 MiB/s wr, 262 op/s 2024-05-09T01:01:42.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.832+0000 7fe64afc0700 1 -- 172.21.15.5:0/4284659466 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe644108f00 msgr2=0x7fe644109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:42.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.832+0000 7fe64afc0700 1 --2- 172.21.15.5:0/4284659466 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe644108f00 0x7fe644109310 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fe6300099c0 tx=0x7fe630009cd0 comp rx=0 tx=0).stop 2024-05-09T01:01:42.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.832+0000 7fe64afc0700 1 -- 172.21.15.5:0/4284659466 shutdown_connections 2024-05-09T01:01:42.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.832+0000 7fe64afc0700 1 --2- 172.21.15.5:0/4284659466 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe644109850 0x7fe6441111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:42.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.832+0000 7fe64afc0700 1 --2- 172.21.15.5:0/4284659466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6441045d0 0x7fe6441049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:42.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.832+0000 7fe64afc0700 1 --2- 172.21.15.5:0/4284659466 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe644108f00 0x7fe644109310 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:42.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.832+0000 7fe64afc0700 1 -- 172.21.15.5:0/4284659466 >> 172.21.15.5:0/4284659466 conn(0x7fe6440ffea0 msgr2=0x7fe6441022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:01:42.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.833+0000 7fe64afc0700 1 -- 172.21.15.5:0/4284659466 shutdown_connections 2024-05-09T01:01:42.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.833+0000 7fe64afc0700 1 -- 172.21.15.5:0/4284659466 wait complete. 2024-05-09T01:01:42.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.833+0000 7fe64afc0700 1 Processor -- start 2024-05-09T01:01:42.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.833+0000 7fe64afc0700 1 -- start start 2024-05-09T01:01:42.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.834+0000 7fe64afc0700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6441045d0 0x7fe644076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:42.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.834+0000 7fe64afc0700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe644108f00 0x7fe644077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:42.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.834+0000 7fe64afc0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe644109850 0x7fe644077650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:42.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.834+0000 7fe64afc0700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe6441144b0 con 0x7fe644109850 2024-05-09T01:01:42.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.834+0000 7fe64afc0700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe644114330 con 0x7fe644108f00 2024-05-09T01:01:42.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.834+0000 7fe64afc0700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe644114630 con 0x7fe6441045d0 2024-05-09T01:01:42.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.834+0000 7fe64955d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe644109850 0x7fe644077650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:42.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.834+0000 7fe64955d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe644109850 0x7fe644077650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58108/0 (socket says 172.21.15.5:58108) 2024-05-09T01:01:42.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.834+0000 7fe64955d700 1 -- 172.21.15.5:0/2080587777 learned_addr learned my addr 172.21.15.5:0/2080587777 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:01:42.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.834+0000 7fe643fff700 1 --2- 172.21.15.5:0/2080587777 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe644108f00 0x7fe644077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:42.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.834+0000 7fe648d5c700 1 --2- 172.21.15.5:0/2080587777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6441045d0 0x7fe644076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:42.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.834+0000 7fe64955d700 1 -- 172.21.15.5:0/2080587777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6441045d0 msgr2=0x7fe644076bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:42.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.834+0000 7fe64955d700 1 --2- 172.21.15.5:0/2080587777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6441045d0 0x7fe644076bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:42.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.834+0000 7fe64955d700 1 -- 172.21.15.5:0/2080587777 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe644108f00 msgr2=0x7fe644077110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:42.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.834+0000 7fe64955d700 1 --2- 172.21.15.5:0/2080587777 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe644108f00 0x7fe644077110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:42.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.834+0000 7fe64955d700 1 -- 172.21.15.5:0/2080587777 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6441a91a0 con 0x7fe644109850 2024-05-09T01:01:42.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.834+0000 7fe643fff700 1 --2- 172.21.15.5:0/2080587777 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe644108f00 0x7fe644077110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:01:42.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.834+0000 7fe64955d700 1 --2- 172.21.15.5:0/2080587777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe644109850 0x7fe644077650 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fe63c007ee0 tx=0x7fe63c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:01:42.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.835+0000 7fe641ffb700 1 -- 172.21.15.5:0/2080587777 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63c004bb0 con 0x7fe644109850 2024-05-09T01:01:42.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.835+0000 7fe641ffb700 1 -- 172.21.15.5:0/2080587777 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe63c004d50 con 0x7fe644109850 2024-05-09T01:01:42.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.835+0000 7fe64afc0700 1 -- 172.21.15.5:0/2080587777 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6441a9490 con 0x7fe644109850 2024-05-09T01:01:42.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.835+0000 7fe641ffb700 1 -- 172.21.15.5:0/2080587777 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63c005920 con 0x7fe644109850 2024-05-09T01:01:42.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.835+0000 7fe64afc0700 1 -- 172.21.15.5:0/2080587777 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6441a9950 con 0x7fe644109850 2024-05-09T01:01:42.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.837+0000 7fe641ffb700 1 -- 172.21.15.5:0/2080587777 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fe63c01a020 con 0x7fe644109850 2024-05-09T01:01:42.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.838+0000 7fe641ffb700 1 --2- 172.21.15.5:0/2080587777 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe6340b2880 0x7fe6340b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:42.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.838+0000 7fe648d5c700 1 --2- 172.21.15.5:0/2080587777 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe6340b2880 0x7fe6340b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:42.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.838+0000 7fe641ffb700 1 -- 172.21.15.5:0/2080587777 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fe63c0e1860 con 0x7fe644109850 2024-05-09T01:01:42.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.839+0000 7fe6599d6b80 1 Processor -- start 2024-05-09T01:01:42.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.839+0000 7fe6599d6b80 1 -- start start 2024-05-09T01:01:42.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.839+0000 7fe648d5c700 1 --2- 172.21.15.5:0/2080587777 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe6340b2880 0x7fe6340b4d90 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe638005950 tx=0x7fe63800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:01:42.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.839+0000 7fe6599d6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5576b98edd00 0x5576b99f6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:42.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.839+0000 7fe6599d6b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576b98734a0 0x5576b99f6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:42.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.839+0000 7fe6599d6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576b99f0400 0x5576b99f0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:42.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.839+0000 7fe6599d6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5576b99130a0 con 0x5576b99f0400 2024-05-09T01:01:42.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.839+0000 7fe6599d6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5576b98b89a0 con 0x5576b98edd00 2024-05-09T01:01:42.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.839+0000 7fe6599d6b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5576b990f060 con 0x5576b98734a0 2024-05-09T01:01:42.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.839+0000 7fe64955d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576b99f0400 0x5576b99f0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:42.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.839+0000 7fe64955d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576b99f0400 0x5576b99f0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58124/0 (socket says 172.21.15.5:58124) 2024-05-09T01:01:42.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.839+0000 7fe64955d700 1 -- 172.21.15.5:0/2016584177 learned_addr learned my addr 172.21.15.5:0/2016584177 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:01:42.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.839+0000 7fe643fff700 1 --2- 172.21.15.5:0/2016584177 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576b98734a0 0x5576b99f6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:42.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.839+0000 7fe648d5c700 1 --2- 172.21.15.5:0/2016584177 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5576b98edd00 0x5576b99f6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:42.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.840+0000 7fe64955d700 1 -- 172.21.15.5:0/2016584177 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576b98734a0 msgr2=0x5576b99f6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:01:42.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.840+0000 7fe64955d700 1 --2- 172.21.15.5:0/2016584177 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576b98734a0 0x5576b99f6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:42.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.840+0000 7fe64955d700 1 -- 172.21.15.5:0/2016584177 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5576b98edd00 msgr2=0x5576b99f6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:01:42.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.840+0000 7fe64955d700 1 --2- 172.21.15.5:0/2016584177 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5576b98edd00 0x5576b99f6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:42.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.840+0000 7fe64955d700 1 -- 172.21.15.5:0/2016584177 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576b98a9500 con 0x5576b99f0400 2024-05-09T01:01:42.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.840+0000 7fe64955d700 1 --2- 172.21.15.5:0/2016584177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576b99f0400 0x5576b99f0810 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fe63c0a60c0 tx=0x7fe63c0a8870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5518317f7a53eb3d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:01:42.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.840+0000 7fe62effd700 1 -- 172.21.15.5:0/2016584177 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63c0a9550 con 0x5576b99f0400 2024-05-09T01:01:42.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.840+0000 7fe62effd700 1 -- 172.21.15.5:0/2016584177 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe63c0ab070 con 0x5576b99f0400 2024-05-09T01:01:42.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.840+0000 7fe62effd700 1 -- 172.21.15.5:0/2016584177 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63c0a90b0 con 0x5576b99f0400 2024-05-09T01:01:42.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.841+0000 7fe6599d6b80 1 -- 172.21.15.5:0/2016584177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576b99f0400 msgr2=0x5576b99f0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:42.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.841+0000 7fe6599d6b80 1 --2- 172.21.15.5:0/2016584177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576b99f0400 0x5576b99f0810 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fe63c0a60c0 tx=0x7fe63c0a8870 comp rx=0 tx=0).stop 2024-05-09T01:01:42.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.841+0000 7fe6599d6b80 1 -- 172.21.15.5:0/2016584177 shutdown_connections 2024-05-09T01:01:42.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.841+0000 7fe6599d6b80 1 --2- 172.21.15.5:0/2016584177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576b99f0400 0x5576b99f0810 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:42.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.841+0000 7fe6599d6b80 1 --2- 172.21.15.5:0/2016584177 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5576b98edd00 0x5576b99f6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:42.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.841+0000 7fe6599d6b80 1 --2- 172.21.15.5:0/2016584177 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576b98734a0 0x5576b99f6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:42.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.841+0000 7fe6599d6b80 1 -- 172.21.15.5:0/2016584177 >> 172.21.15.5:0/2016584177 conn(0x5576b9876f70 msgr2=0x5576b99eb220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:01:42.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.841+0000 7fe6599d6b80 1 -- 172.21.15.5:0/2016584177 shutdown_connections 2024-05-09T01:01:42.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.841+0000 7fe6599d6b80 1 -- 172.21.15.5:0/2016584177 wait complete. 2024-05-09T01:01:42.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.841+0000 7fe6599d6b80 1 -- start start 2024-05-09T01:01:42.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.842+0000 7fe6599d6b80 1 Processor -- start 2024-05-09T01:01:42.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.842+0000 7fe6599d6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576b98734a0 0x5576b99fa040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:42.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.842+0000 7fe6599d6b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576b98edd00 0x5576b99fa580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:42.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.842+0000 7fe6599d6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5576b99f0400 0x5576b99faac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:42.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.842+0000 7fe6599d6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5576b99f1160 con 0x5576b98734a0 2024-05-09T01:01:42.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.842+0000 7fe6599d6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5576b98adb20 con 0x5576b99f0400 2024-05-09T01:01:42.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.842+0000 7fe6599d6b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5576b99fd020 con 0x5576b98edd00 2024-05-09T01:01:42.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.842+0000 7fe6599d6b80 10 client.0 _ll_register_callbacks cb 0x5576b97846c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:01:42.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.842+0000 7fe648d5c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576b98734a0 0x5576b99fa040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:42.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.842+0000 7fe648d5c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576b98734a0 0x5576b99fa040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58132/0 (socket says 172.21.15.5:58132) 2024-05-09T01:01:42.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.842+0000 7fe648d5c700 1 -- 172.21.15.5:0/3234655103 learned_addr learned my addr 172.21.15.5:0/3234655103 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:01:42.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.843+0000 7fe643fff700 1 --2- 172.21.15.5:0/3234655103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576b98edd00 0x5576b99fa580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:42.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.843+0000 7fe648d5c700 1 -- 172.21.15.5:0/3234655103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576b98edd00 msgr2=0x5576b99fa580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:01:42.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.843+0000 7fe648d5c700 1 --2- 172.21.15.5:0/3234655103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576b98edd00 0x5576b99fa580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:42.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.843+0000 7fe64955d700 1 --2- 172.21.15.5:0/3234655103 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5576b99f0400 0x5576b99faac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:42.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.843+0000 7fe648d5c700 1 -- 172.21.15.5:0/3234655103 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5576b99f0400 msgr2=0x5576b99faac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:01:42.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.843+0000 7fe648d5c700 1 --2- 172.21.15.5:0/3234655103 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5576b99f0400 0x5576b99faac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:42.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.843+0000 7fe648d5c700 1 -- 172.21.15.5:0/3234655103 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576b978e490 con 0x5576b98734a0 2024-05-09T01:01:42.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.843+0000 7fe648d5c700 1 --2- 172.21.15.5:0/3234655103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576b98734a0 0x5576b99fa040 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fe6380116c0 tx=0x7fe6380047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8b8af04e6f5f7b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:01:42.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.843+0000 7fe623fff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T01:01:42.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.843+0000 7fe623fff700 1 -- 172.21.15.5:0/3234655103 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe638004a30 con 0x5576b98734a0 2024-05-09T01:01:42.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.843+0000 7fe623fff700 1 -- 172.21.15.5:0/3234655103 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fe638004bd0 con 0x5576b98734a0 2024-05-09T01:01:42.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.843+0000 7fe6599d6b80 1 -- 172.21.15.5:0/3234655103 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe63c0ab070 con 0x5576b98734a0 2024-05-09T01:01:42.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.843+0000 7fe623fff700 1 -- 172.21.15.5:0/3234655103 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe63802d030 con 0x5576b98734a0 2024-05-09T01:01:42.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.843+0000 7fe6599d6b80 10 client.17277 fetch_fsmap learned FSMap version 34 2024-05-09T01:01:42.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.843+0000 7fe6599d6b80 1 -- 172.21.15.5:0/3234655103 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5576b9a07ac0 con 0x5576b98734a0 2024-05-09T01:01:42.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.844+0000 7fe623fff700 1 -- 172.21.15.5:0/3234655103 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe63c0ab070 con 0x5576b98734a0 2024-05-09T01:01:42.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.844+0000 7fe623fff700 1 -- 172.21.15.5:0/3234655103 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7fe63801f840 con 0x5576b98734a0 2024-05-09T01:01:42.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.844+0000 7fe6599d6b80 10 client.17277 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T01:01:42.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.844+0000 7fe6599d6b80 10 client.17277 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T01:01:42.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.844+0000 7fe6599d6b80 20 client.17277 populate_metadata read hostname 'smithi005' 2024-05-09T01:01:42.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.844+0000 7fe6599d6b80 1 --2- 172.21.15.5:0/3234655103 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5576b9a0a100 0x5576b9a0c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:42.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.844+0000 7fe6599d6b80 4 client.17277 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T01:01:42.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.844+0000 7fe6599d6b80 1 -- 172.21.15.5:0/3234655103 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5576b9920a80 con 0x5576b9a0a100 2024-05-09T01:01:42.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.844+0000 7fe643fff700 1 --2- 172.21.15.5:0/3234655103 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5576b9a0a100 0x5576b9a0c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:42.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.845+0000 7fe623fff700 10 client.17277 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T01:01:42.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.845+0000 7fe643fff700 1 --2- 172.21.15.5:0/3234655103 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5576b9a0a100 0x5576b9a0c560 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd771381b369f91d server_cookie=9ec34a75d3905681 in_seq=0 out_seq=0 2024-05-09T01:01:42.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.846+0000 7fe623fff700 1 -- 172.21.15.5:0/3234655103 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5576b9920a80 con 0x5576b9a0a100 2024-05-09T01:01:42.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.846+0000 7fe623fff700 10 client.17277 handle_command_reply: tid=0 2024-05-09T01:01:42.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.846+0000 7fe6599d6b80 1 client.17277 shutdown 2024-05-09T01:01:42.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.846+0000 7fe6599d6b80 1 -- 172.21.15.5:0/3234655103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576b98734a0 msgr2=0x5576b99fa040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:42.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.846+0000 7fe6599d6b80 1 --2- 172.21.15.5:0/3234655103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576b98734a0 0x5576b99fa040 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fe6380116c0 tx=0x7fe6380047c0 comp rx=0 tx=0).stop 2024-05-09T01:01:42.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.846+0000 7fe6599d6b80 1 -- 172.21.15.5:0/3234655103 shutdown_connections 2024-05-09T01:01:42.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.846+0000 7fe6599d6b80 1 --2- 172.21.15.5:0/3234655103 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5576b9a0a100 0x5576b9a0c560 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:42.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.846+0000 7fe6599d6b80 1 --2- 172.21.15.5:0/3234655103 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5576b99f0400 0x5576b99faac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:42.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.846+0000 7fe6599d6b80 1 --2- 172.21.15.5:0/3234655103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576b98734a0 0x5576b99fa040 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:42.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.846+0000 7fe6599d6b80 1 --2- 172.21.15.5:0/3234655103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576b98edd00 0x5576b99fa580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:42.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.846+0000 7fe6599d6b80 1 -- 172.21.15.5:0/3234655103 >> 172.21.15.5:0/3234655103 conn(0x5576b9876f70 msgr2=0x5576b99e4ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:01:42.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.846+0000 7fe623fff700 0 client.17277 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T01:01:42.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.846+0000 7fe6599d6b80 1 -- 172.21.15.5:0/3234655103 shutdown_connections 2024-05-09T01:01:42.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.846+0000 7fe6599d6b80 1 -- 172.21.15.5:0/3234655103 wait complete. 2024-05-09T01:01:42.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.846+0000 7fe6599d6b80 20 client.17277 trim_cache size 0 max 16384 2024-05-09T01:01:42.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.862+0000 7fe6599d6b80 1 Processor -- start 2024-05-09T01:01:42.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.862+0000 7fe6599d6b80 1 -- start start 2024-05-09T01:01:42.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.862+0000 7fe6599d6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576b98734a0 0x5576b9a0c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:42.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.862+0000 7fe6599d6b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576b98edd00 0x5576b99eb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:42.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.862+0000 7fe6599d6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5576b99f0400 0x5576b99f9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:42.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.862+0000 7fe6599d6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe6441b0ec0 con 0x5576b98734a0 2024-05-09T01:01:42.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.862+0000 7fe6599d6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe6441b2a30 con 0x5576b99f0400 2024-05-09T01:01:42.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.862+0000 7fe6599d6b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe6441b1590 con 0x5576b98edd00 2024-05-09T01:01:42.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.862+0000 7fe648d5c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576b98734a0 0x5576b9a0c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:42.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.862+0000 7fe648d5c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576b98734a0 0x5576b9a0c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58148/0 (socket says 172.21.15.5:58148) 2024-05-09T01:01:42.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.862+0000 7fe648d5c700 1 -- 172.21.15.5:0/2201830931 learned_addr learned my addr 172.21.15.5:0/2201830931 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:01:42.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.862+0000 7fe64955d700 1 --2- 172.21.15.5:0/2201830931 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5576b99f0400 0x5576b99f9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:42.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.862+0000 7fe643fff700 1 --2- 172.21.15.5:0/2201830931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576b98edd00 0x5576b99eb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:42.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.862+0000 7fe648d5c700 1 -- 172.21.15.5:0/2201830931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576b98edd00 msgr2=0x5576b99eb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:01:42.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.862+0000 7fe648d5c700 1 --2- 172.21.15.5:0/2201830931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576b98edd00 0x5576b99eb0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:42.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.862+0000 7fe648d5c700 1 -- 172.21.15.5:0/2201830931 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5576b99f0400 msgr2=0x5576b99f9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:01:42.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.862+0000 7fe648d5c700 1 --2- 172.21.15.5:0/2201830931 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5576b99f0400 0x5576b99f9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:42.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.862+0000 7fe648d5c700 1 -- 172.21.15.5:0/2201830931 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576b99ebf70 con 0x5576b98734a0 2024-05-09T01:01:42.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.863+0000 7fe648d5c700 1 --2- 172.21.15.5:0/2201830931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576b98734a0 0x5576b9a0c780 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fe638000c00 tx=0x7fe638010fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c0c694155754ca2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:01:42.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.863+0000 7fe62e7fc700 1 -- 172.21.15.5:0/2201830931 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe638040070 con 0x5576b98734a0 2024-05-09T01:01:42.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.863+0000 7fe62e7fc700 1 -- 172.21.15.5:0/2201830931 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe638023070 con 0x5576b98734a0 2024-05-09T01:01:42.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.863+0000 7fe62e7fc700 1 -- 172.21.15.5:0/2201830931 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe638037040 con 0x5576b98734a0 2024-05-09T01:01:42.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.864+0000 7fe6599d6b80 1 -- 172.21.15.5:0/2201830931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576b98734a0 msgr2=0x5576b9a0c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:42.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.864+0000 7fe6599d6b80 1 --2- 172.21.15.5:0/2201830931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576b98734a0 0x5576b9a0c780 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fe638000c00 tx=0x7fe638010fa0 comp rx=0 tx=0).stop 2024-05-09T01:01:42.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.864+0000 7fe6599d6b80 1 -- 172.21.15.5:0/2201830931 shutdown_connections 2024-05-09T01:01:42.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.864+0000 7fe6599d6b80 1 --2- 172.21.15.5:0/2201830931 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5576b99f0400 0x5576b99f9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:42.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.864+0000 7fe6599d6b80 1 --2- 172.21.15.5:0/2201830931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576b98734a0 0x5576b9a0c780 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:42.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.864+0000 7fe6599d6b80 1 --2- 172.21.15.5:0/2201830931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576b98edd00 0x5576b99eb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:42.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.864+0000 7fe6599d6b80 1 -- 172.21.15.5:0/2201830931 >> 172.21.15.5:0/2201830931 conn(0x5576b9876f70 msgr2=0x5576b99f8930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:01:42.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.864+0000 7fe6599d6b80 1 -- 172.21.15.5:0/2201830931 shutdown_connections 2024-05-09T01:01:42.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.864+0000 7fe6599d6b80 1 -- 172.21.15.5:0/2201830931 wait complete. 2024-05-09T01:01:42.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.864+0000 7fe6599d6b80 1 -- start start 2024-05-09T01:01:42.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.864+0000 7fe6599d6b80 1 Processor -- start 2024-05-09T01:01:42.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.865+0000 7fe6599d6b80 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5576b98734a0 0x5576b9a120e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:42.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.865+0000 7fe6599d6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576b98edd00 0x5576b9a02150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:42.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.865+0000 7fe6599d6b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576b99f0400 0x5576b9a02690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:42.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.865+0000 7fe6599d6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe6441b2360 con 0x5576b98edd00 2024-05-09T01:01:42.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.865+0000 7fe6599d6b80 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe6441b1c60 con 0x5576b98734a0 2024-05-09T01:01:42.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.865+0000 7fe6599d6b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe6441aee60 con 0x5576b99f0400 2024-05-09T01:01:42.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.865+0000 7fe6599d6b80 10 client.0 _ll_register_callbacks cb 0x5576b97846c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T01:01:42.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.865+0000 7fe643fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576b98edd00 0x5576b9a02150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:42.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.865+0000 7fe643fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576b98edd00 0x5576b9a02150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58150/0 (socket says 172.21.15.5:58150) 2024-05-09T01:01:42.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.865+0000 7fe643fff700 1 -- 172.21.15.5:0/1606334357 learned_addr learned my addr 172.21.15.5:0/1606334357 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:01:42.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.865+0000 7fe648d5c700 1 --2- 172.21.15.5:0/1606334357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5576b98734a0 0x5576b9a120e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:42.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.865+0000 7fe64955d700 1 --2- 172.21.15.5:0/1606334357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576b99f0400 0x5576b9a02690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:42.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.865+0000 7fe643fff700 1 -- 172.21.15.5:0/1606334357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576b99f0400 msgr2=0x5576b9a02690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:01:42.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.865+0000 7fe643fff700 1 --2- 172.21.15.5:0/1606334357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576b99f0400 0x5576b9a02690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:42.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.865+0000 7fe643fff700 1 -- 172.21.15.5:0/1606334357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5576b98734a0 msgr2=0x5576b9a120e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T01:01:42.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.865+0000 7fe643fff700 1 --2- 172.21.15.5:0/1606334357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5576b98734a0 0x5576b9a120e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:42.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.865+0000 7fe643fff700 1 -- 172.21.15.5:0/1606334357 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576b9a10d10 con 0x5576b98edd00 2024-05-09T01:01:42.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.865+0000 7fe648d5c700 1 --2- 172.21.15.5:0/1606334357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5576b98734a0 0x5576b9a120e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:01:42.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.865+0000 7fe643fff700 1 --2- 172.21.15.5:0/1606334357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576b98edd00 0x5576b9a02150 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fe630000c00 tx=0x7fe63000bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72478e5e268afea3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:01:42.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.865+0000 7fe623fff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-09T01:01:42.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.866+0000 7fe623fff700 1 -- 172.21.15.5:0/1606334357 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63002b070 con 0x5576b98edd00 2024-05-09T01:01:42.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.866+0000 7fe623fff700 1 -- 172.21.15.5:0/1606334357 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fe630003a60 con 0x5576b98edd00 2024-05-09T01:01:42.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.866+0000 7fe6599d6b80 1 -- 172.21.15.5:0/1606334357 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe638023070 con 0x5576b98edd00 2024-05-09T01:01:42.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.866+0000 7fe623fff700 1 -- 172.21.15.5:0/1606334357 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe630037050 con 0x5576b98edd00 2024-05-09T01:01:42.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.866+0000 7fe6599d6b80 10 client.17283 fetch_fsmap learned FSMap version 34 2024-05-09T01:01:42.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.866+0000 7fe6599d6b80 1 -- 172.21.15.5:0/1606334357 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5576b9a739c0 con 0x5576b98edd00 2024-05-09T01:01:42.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.866+0000 7fe623fff700 1 -- 172.21.15.5:0/1606334357 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe638023070 con 0x5576b98edd00 2024-05-09T01:01:42.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.866+0000 7fe623fff700 1 -- 172.21.15.5:0/1606334357 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7fe630027170 con 0x5576b98edd00 2024-05-09T01:01:42.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.866+0000 7fe6599d6b80 10 client.17283 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T01:01:42.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.866+0000 7fe6599d6b80 10 client.17283 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T01:01:42.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.866+0000 7fe6599d6b80 20 client.17283 populate_metadata read hostname 'smithi005' 2024-05-09T01:01:42.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.866+0000 7fe6599d6b80 1 --2- 172.21.15.5:0/1606334357 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5576b9a740a0 0x5576b9a764c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:42.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.866+0000 7fe6599d6b80 4 client.17283 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T01:01:42.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.866+0000 7fe6599d6b80 1 -- 172.21.15.5:0/1606334357 --> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5576b991ab20 con 0x5576b9a740a0 2024-05-09T01:01:42.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.867+0000 7fe648d5c700 1 --2- 172.21.15.5:0/1606334357 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5576b9a740a0 0x5576b9a764c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:42.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.867+0000 7fe648d5c700 1 --2- 172.21.15.5:0/1606334357 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5576b9a740a0 0x5576b9a764c0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d9f36e3891ca84c server_cookie=8e648c6d1ceebc1c in_seq=0 out_seq=0 2024-05-09T01:01:42.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.867+0000 7fe623fff700 10 client.17283 ms_handle_connect on v2:172.21.15.157:6838/2779860399 2024-05-09T01:01:42.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.867+0000 7fe623fff700 1 -- 172.21.15.5:0/1606334357 <== mds.0 v2:172.21.15.157:6838/2779860399 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5576b991ab20 con 0x5576b9a740a0 2024-05-09T01:01:42.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.867+0000 7fe623fff700 10 client.17283 handle_command_reply: tid=0 2024-05-09T01:01:42.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.867+0000 7fe6599d6b80 1 client.17283 shutdown 2024-05-09T01:01:42.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.867+0000 7fe6599d6b80 1 -- 172.21.15.5:0/1606334357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576b98edd00 msgr2=0x5576b9a02150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:42.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.867+0000 7fe6599d6b80 1 --2- 172.21.15.5:0/1606334357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576b98edd00 0x5576b9a02150 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fe630000c00 tx=0x7fe63000bb40 comp rx=0 tx=0).stop 2024-05-09T01:01:42.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.867+0000 7fe6599d6b80 1 -- 172.21.15.5:0/1606334357 shutdown_connections 2024-05-09T01:01:42.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.867+0000 7fe6599d6b80 1 --2- 172.21.15.5:0/1606334357 >> [v2:172.21.15.157:6838/2779860399,v1:172.21.15.157:6839/2779860399] conn(0x5576b9a740a0 0x5576b9a764c0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:42.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.868+0000 7fe6599d6b80 1 --2- 172.21.15.5:0/1606334357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5576b99f0400 0x5576b9a02690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:42.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.868+0000 7fe6599d6b80 1 --2- 172.21.15.5:0/1606334357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5576b98734a0 0x5576b9a120e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:42.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.868+0000 7fe6599d6b80 1 --2- 172.21.15.5:0/1606334357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576b98edd00 0x5576b9a02150 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:42.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.868+0000 7fe6599d6b80 1 -- 172.21.15.5:0/1606334357 >> 172.21.15.5:0/1606334357 conn(0x5576b9876f70 msgr2=0x5576b99f8930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:01:42.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.868+0000 7fe623fff700 0 client.17283 ms_handle_reset on v2:172.21.15.157:6838/2779860399 2024-05-09T01:01:42.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.868+0000 7fe6599d6b80 1 -- 172.21.15.5:0/1606334357 shutdown_connections 2024-05-09T01:01:42.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.868+0000 7fe6599d6b80 1 -- 172.21.15.5:0/1606334357 wait complete. 2024-05-09T01:01:42.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.868+0000 7fe6599d6b80 20 client.17283 trim_cache size 0 max 16384 2024-05-09T01:01:42.874 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-09T01:01:42.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.868+0000 7fe64afc0700 1 -- 172.21.15.5:0/2080587777 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe6340b2880 msgr2=0x7fe6340b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:42.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.868+0000 7fe64afc0700 1 --2- 172.21.15.5:0/2080587777 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe6340b2880 0x7fe6340b4d90 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe638005950 tx=0x7fe63800b410 comp rx=0 tx=0).stop 2024-05-09T01:01:42.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.868+0000 7fe64afc0700 1 -- 172.21.15.5:0/2080587777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe644109850 msgr2=0x7fe644077650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:42.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.868+0000 7fe64afc0700 1 --2- 172.21.15.5:0/2080587777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe644109850 0x7fe644077650 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fe63c007ee0 tx=0x7fe63c00d3b0 comp rx=0 tx=0).stop 2024-05-09T01:01:42.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.869+0000 7fe64afc0700 1 -- 172.21.15.5:0/2080587777 shutdown_connections 2024-05-09T01:01:42.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.869+0000 7fe64afc0700 1 --2- 172.21.15.5:0/2080587777 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe6340b2880 0x7fe6340b4d90 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:42.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.869+0000 7fe64afc0700 1 --2- 172.21.15.5:0/2080587777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe644109850 0x7fe644077650 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:42.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.869+0000 7fe64afc0700 1 --2- 172.21.15.5:0/2080587777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6441045d0 0x7fe644076bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:42.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.869+0000 7fe64afc0700 1 --2- 172.21.15.5:0/2080587777 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe644108f00 0x7fe644077110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:42.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.869+0000 7fe64afc0700 1 -- 172.21.15.5:0/2080587777 >> 172.21.15.5:0/2080587777 conn(0x7fe6440ffea0 msgr2=0x7fe64410a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:01:42.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.869+0000 7fe64afc0700 1 -- 172.21.15.5:0/2080587777 shutdown_connections 2024-05-09T01:01:42.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:42.869+0000 7fe64afc0700 1 -- 172.21.15.5:0/2080587777 wait complete. 2024-05-09T01:01:43.523 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_c9da4049f638c0f84bdd9b4e33c42e86046ac554/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_c9da4049f638c0f84bdd9b4e33c42e86046ac554/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph-c_c9da4049f638c0f84bdd9b4e33c42e86046ac554/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-05-09T01:01:43.524 INFO:tasks.fwd_scrub:done joining 2024-05-09T01:01:43.524 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-05-09T01:01:43.535 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T01:01:43.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:43.655+0000 7fdf72766640 1 -- 172.21.15.5:0/2948702834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf6c102b30 msgr2=0x7fdf6c103830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:43.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:43.655+0000 7fdf72766640 1 --2- 172.21.15.5:0/2948702834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf6c102b30 0x7fdf6c103830 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c009900 tx=0x7fdf5c02a410 comp rx=0 tx=0).stop 2024-05-09T01:01:43.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:43.656+0000 7fdf72766640 1 -- 172.21.15.5:0/2948702834 shutdown_connections 2024-05-09T01:01:43.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:43.656+0000 7fdf72766640 1 --2- 172.21.15.5:0/2948702834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf6c108a70 0x7fdf6c10ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:43.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:43.656+0000 7fdf72766640 1 --2- 172.21.15.5:0/2948702834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf6c103d70 0x7fdf6c104180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:43.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:43.656+0000 7fdf72766640 1 --2- 172.21.15.5:0/2948702834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf6c102b30 0x7fdf6c103830 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:43.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:43.656+0000 7fdf72766640 1 -- 172.21.15.5:0/2948702834 >> 172.21.15.5:0/2948702834 conn(0x7fdf6c0fdd20 msgr2=0x7fdf6c1001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:01:43.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:43.656+0000 7fdf72766640 1 -- 172.21.15.5:0/2948702834 shutdown_connections 2024-05-09T01:01:43.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:43.656+0000 7fdf72766640 1 -- 172.21.15.5:0/2948702834 wait complete. 2024-05-09T01:01:43.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:43.656+0000 7fdf72766640 1 Processor -- start 2024-05-09T01:01:43.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:43.656+0000 7fdf72766640 1 -- start start 2024-05-09T01:01:43.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:43.657+0000 7fdf72766640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf6c102b30 0x7fdf6c0ffd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:43.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:43.657+0000 7fdf72766640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf6c103d70 0x7fdf6c0fe410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:43.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:43.657+0000 7fdf72766640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf6c108a70 0x7fdf6c0fe950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:43.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:43.657+0000 7fdf72766640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6c10e660 con 0x7fdf6c102b30 2024-05-09T01:01:43.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:43.657+0000 7fdf72766640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6c10e4e0 con 0x7fdf6c103d70 2024-05-09T01:01:43.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:43.657+0000 7fdf72766640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6c10e7e0 con 0x7fdf6c108a70 2024-05-09T01:01:43.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:43.657+0000 7fdf6bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf6c102b30 0x7fdf6c0ffd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:43.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:43.657+0000 7fdf6bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf6c102b30 0x7fdf6c0ffd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46220/0 (socket says 172.21.15.5:46220) 2024-05-09T01:01:43.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:43.657+0000 7fdf6bfff640 1 -- 172.21.15.5:0/4122215755 learned_addr learned my addr 172.21.15.5:0/4122215755 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:01:43.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:43.657+0000 7fdf6b7fe640 1 --2- 172.21.15.5:0/4122215755 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf6c103d70 0x7fdf6c0fe410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:43.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:43.657+0000 7fdf70d03640 1 --2- 172.21.15.5:0/4122215755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf6c108a70 0x7fdf6c0fe950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:43.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:43.657+0000 7fdf6bfff640 1 -- 172.21.15.5:0/4122215755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf6c108a70 msgr2=0x7fdf6c0fe950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:43.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:43.657+0000 7fdf6bfff640 1 --2- 172.21.15.5:0/4122215755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf6c108a70 0x7fdf6c0fe950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:43.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:43.657+0000 7fdf6bfff640 1 -- 172.21.15.5:0/4122215755 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf6c103d70 msgr2=0x7fdf6c0fe410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:43.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:43.657+0000 7fdf6bfff640 1 --2- 172.21.15.5:0/4122215755 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf6c103d70 0x7fdf6c0fe410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:43.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:43.657+0000 7fdf6bfff640 1 -- 172.21.15.5:0/4122215755 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf6c0feff0 con 0x7fdf6c102b30 2024-05-09T01:01:43.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:43.657+0000 7fdf6bfff640 1 --2- 172.21.15.5:0/4122215755 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf6c102b30 0x7fdf6c0ffd70 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c031440 tx=0x7fdf5c003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:01:43.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:43.658+0000 7fdf697fa640 1 -- 172.21.15.5:0/4122215755 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c03f070 con 0x7fdf6c102b30 2024-05-09T01:01:43.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:43.658+0000 7fdf697fa640 1 -- 172.21.15.5:0/4122215755 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf5c03b460 con 0x7fdf6c102b30 2024-05-09T01:01:43.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:43.658+0000 7fdf72766640 1 -- 172.21.15.5:0/4122215755 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf6c1a43f0 con 0x7fdf6c102b30 2024-05-09T01:01:43.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:43.658+0000 7fdf697fa640 1 -- 172.21.15.5:0/4122215755 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c035030 con 0x7fdf6c102b30 2024-05-09T01:01:43.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:43.658+0000 7fdf72766640 1 -- 172.21.15.5:0/4122215755 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf6c1a47a0 con 0x7fdf6c102b30 2024-05-09T01:01:43.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:43.659+0000 7fdf72766640 1 -- 172.21.15.5:0/4122215755 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf6c1a4a90 con 0x7fdf6c102b30 2024-05-09T01:01:43.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:43.660+0000 7fdf697fa640 1 -- 172.21.15.5:0/4122215755 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fdf5c042070 con 0x7fdf6c102b30 2024-05-09T01:01:43.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:43.661+0000 7fdf697fa640 1 --2- 172.21.15.5:0/4122215755 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fdf540b27f0 0x7fdf540b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:43.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:43.661+0000 7fdf6b7fe640 1 --2- 172.21.15.5:0/4122215755 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fdf540b27f0 0x7fdf540b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:43.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:43.661+0000 7fdf697fa640 1 -- 172.21.15.5:0/4122215755 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fdf5c102f30 con 0x7fdf6c102b30 2024-05-09T01:01:43.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:43.662+0000 7fdf6b7fe640 1 --2- 172.21.15.5:0/4122215755 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fdf540b27f0 0x7fdf540b4d00 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c0ff710 tx=0x7fdf6000a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:01:43.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:43.665+0000 7fdf697fa640 1 -- 172.21.15.5:0/4122215755 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdf5c0cac20 con 0x7fdf6c102b30 2024-05-09T01:01:43.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:43.863+0000 7fdf72766640 1 -- 172.21.15.5:0/4122215755 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdf6c1a4c60 con 0x7fdf6c102b30 2024-05-09T01:01:43.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:43.865+0000 7fdf697fa640 1 -- 172.21.15.5:0/4122215755 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22745 (secure 0 0 0) 0x7fdf5c0ca520 con 0x7fdf6c102b30 2024-05-09T01:01:43.866 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T01:01:43.866 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":82,"fsid":"d85814c6-0d9b-11ef-bc97-c7b262605968","created":"2024-05-09T00:35:25.623377+0000","modified":"2024-05-09T00:45:23.926493+0000","last_up_change":"2024-05-09T00:42:38.525335+0000","last_in_change":"2024-05-09T00:42:17.990219+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T00:38:47.844734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T00:43:39.452106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T00:43:42.708106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f8d484bb-2477-4377-ae68-5bce571333bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2963385132},{"type":"v1","addr":"172.21.15.5:6803","nonce":2963385132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2963385132},{"type":"v1","addr":"172.21.15.5:6805","nonce":2963385132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2963385132},{"type":"v1","addr":"172.21.15.5:6809","nonce":2963385132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2963385132},{"type":"v1","addr":"172.21.15.5:6807","nonce":2963385132}]},"public_addr":"172.21.15.5:6803/2963385132","cluster_addr":"172.21.15.5:6805/2963385132","heartbeat_back_addr":"172.21.15.5:6809/2963385132","heartbeat_front_addr":"172.21.15.5:6807/2963385132","state":["exists","up"]},{"osd":1,"uuid":"1c123676-7144-4a67-93a6-fd49e36a5c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3837683163},{"type":"v1","addr":"172.21.15.131:6801","nonce":3837683163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3837683163},{"type":"v1","addr":"172.21.15.131:6803","nonce":3837683163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3837683163},{"type":"v1","addr":"172.21.15.131:6807","nonce":3837683163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3837683163},{"type":"v1","addr":"172.21.15.131:6805","nonce":3837683163}]},"public_addr":"172.21.15.131:6801/3837683163","cluster_addr":"172.21.15.131:6803/3837683163","heartbeat_back_addr":"172.21.15.131:6807/3837683163","heartbeat_front_addr":"172.21.15.131:6805/3837683163","state":["exists","up"]},{"osd":2,"uuid":"95ca51f0-b48b-485c-a5f7-b15c01e754b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":484818139},{"type":"v1","addr":"172.21.15.157:6801","nonce":484818139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":484818139},{"type":"v1","addr":"172.21.15.157:6803","nonce":484818139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":484818139},{"type":"v1","addr":"172.21.15.157:6807","nonce":484818139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":484818139},{"type":"v1","addr":"172.21.15.157:6805","nonce":484818139}]},"public_addr":"172.21.15.157:6801/484818139","cluster_addr":"172.21.15.157:6803/484818139","heartbeat_back_addr":"172.21.15.157:6807/484818139","heartbeat_front_addr":"172.21.15.157:6805/484818139","state":["exists","up"]},{"osd":3,"uuid":"ed7577fb-3676-4a08-b40d-5474eb88974f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":2357377277},{"type":"v1","addr":"172.21.15.5:6811","nonce":2357377277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":2357377277},{"type":"v1","addr":"172.21.15.5:6813","nonce":2357377277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":2357377277},{"type":"v1","addr":"172.21.15.5:6817","nonce":2357377277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":2357377277},{"type":"v1","addr":"172.21.15.5:6815","nonce":2357377277}]},"public_addr":"172.21.15.5:6811/2357377277","cluster_addr":"172.21.15.5:6813/2357377277","heartbeat_back_addr":"172.21.15.5:6817/2357377277","heartbeat_front_addr":"172.21.15.5:6815/2357377277","state":["exists","up"]},{"osd":4,"uuid":"fb5dcf79-a6f1-4b1a-9e3a-087fb5132472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":2915418378},{"type":"v1","addr":"172.21.15.131:6809","nonce":2915418378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":2915418378},{"type":"v1","addr":"172.21.15.131:6811","nonce":2915418378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":2915418378},{"type":"v1","addr":"172.21.15.131:6815","nonce":2915418378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":2915418378},{"type":"v1","addr":"172.21.15.131:6813","nonce":2915418378}]},"public_addr":"172.21.15.131:6809/2915418378","cluster_addr":"172.21.15.131:6811/2915418378","heartbeat_back_addr":"172.21.15.131:6815/2915418378","heartbeat_front_addr":"172.21.15.131:6813/2915418378","state":["exists","up"]},{"osd":5,"uuid":"f01ed5d3-8066-4d25-a113-280c83e75ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4188818663},{"type":"v1","addr":"172.21.15.157:6809","nonce":4188818663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":4188818663},{"type":"v1","addr":"172.21.15.157:6811","nonce":4188818663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":4188818663},{"type":"v1","addr":"172.21.15.157:6815","nonce":4188818663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":4188818663},{"type":"v1","addr":"172.21.15.157:6813","nonce":4188818663}]},"public_addr":"172.21.15.157:6809/4188818663","cluster_addr":"172.21.15.157:6811/4188818663","heartbeat_back_addr":"172.21.15.157:6815/4188818663","heartbeat_front_addr":"172.21.15.157:6813/4188818663","state":["exists","up"]},{"osd":6,"uuid":"932379a8-c97b-4acc-8665-81f7d9cbce65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2930397813},{"type":"v1","addr":"172.21.15.5:6819","nonce":2930397813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2930397813},{"type":"v1","addr":"172.21.15.5:6821","nonce":2930397813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2930397813},{"type":"v1","addr":"172.21.15.5:6825","nonce":2930397813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2930397813},{"type":"v1","addr":"172.21.15.5:6823","nonce":2930397813}]},"public_addr":"172.21.15.5:6819/2930397813","cluster_addr":"172.21.15.5:6821/2930397813","heartbeat_back_addr":"172.21.15.5:6825/2930397813","heartbeat_front_addr":"172.21.15.5:6823/2930397813","state":["exists","up"]},{"osd":7,"uuid":"3c0a0573-6465-4bf6-b70f-310d49857a6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":4125045219},{"type":"v1","addr":"172.21.15.131:6817","nonce":4125045219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":4125045219},{"type":"v1","addr":"172.21.15.131:6819","nonce":4125045219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":4125045219},{"type":"v1","addr":"172.21.15.131:6823","nonce":4125045219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":4125045219},{"type":"v1","addr":"172.21.15.131:6821","nonce":4125045219}]},"public_addr":"172.21.15.131:6817/4125045219","cluster_addr":"172.21.15.131:6819/4125045219","heartbeat_back_addr":"172.21.15.131:6823/4125045219","heartbeat_front_addr":"172.21.15.131:6821/4125045219","state":["exists","up"]},{"osd":8,"uuid":"c49f0a00-f5ae-40ed-8f38-7b953ba1c17c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2001626768},{"type":"v1","addr":"172.21.15.157:6817","nonce":2001626768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2001626768},{"type":"v1","addr":"172.21.15.157:6819","nonce":2001626768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2001626768},{"type":"v1","addr":"172.21.15.157:6823","nonce":2001626768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2001626768},{"type":"v1","addr":"172.21.15.157:6821","nonce":2001626768}]},"public_addr":"172.21.15.157:6817/2001626768","cluster_addr":"172.21.15.157:6819/2001626768","heartbeat_back_addr":"172.21.15.157:6823/2001626768","heartbeat_front_addr":"172.21.15.157:6821/2001626768","state":["exists","up"]},{"osd":9,"uuid":"1927dfe7-c1af-4a15-bb71-d58d97c8734a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":4287220015},{"type":"v1","addr":"172.21.15.5:6827","nonce":4287220015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":4287220015},{"type":"v1","addr":"172.21.15.5:6829","nonce":4287220015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":4287220015},{"type":"v1","addr":"172.21.15.5:6833","nonce":4287220015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":4287220015},{"type":"v1","addr":"172.21.15.5:6831","nonce":4287220015}]},"public_addr":"172.21.15.5:6827/4287220015","cluster_addr":"172.21.15.5:6829/4287220015","heartbeat_back_addr":"172.21.15.5:6833/4287220015","heartbeat_front_addr":"172.21.15.5:6831/4287220015","state":["exists","up"]},{"osd":10,"uuid":"fbb4f3e3-eb7d-4ecb-a30b-55b8910375ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1571783985},{"type":"v1","addr":"172.21.15.131:6825","nonce":1571783985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1571783985},{"type":"v1","addr":"172.21.15.131:6827","nonce":1571783985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1571783985},{"type":"v1","addr":"172.21.15.131:6831","nonce":1571783985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1571783985},{"type":"v1","addr":"172.21.15.131:6829","nonce":1571783985}]},"public_addr":"172.21.15.131:6825/1571783985","cluster_addr":"172.21.15.131:6827/1571783985","heartbeat_back_addr":"172.21.15.131:6831/1571783985","heartbeat_front_addr":"172.21.15.131:6829/1571783985","state":["exists","up"]},{"osd":11,"uuid":"b858453f-eda1-4e25-89c8-94b54c830938","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3261151188},{"type":"v1","addr":"172.21.15.157:6825","nonce":3261151188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":3261151188},{"type":"v1","addr":"172.21.15.157:6827","nonce":3261151188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":3261151188},{"type":"v1","addr":"172.21.15.157:6831","nonce":3261151188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":3261151188},{"type":"v1","addr":"172.21.15.157:6829","nonce":3261151188}]},"public_addr":"172.21.15.157:6825/3261151188","cluster_addr":"172.21.15.157:6827/3261151188","heartbeat_back_addr":"172.21.15.157:6831/3261151188","heartbeat_front_addr":"172.21.15.157:6829/3261151188","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:37:57.528483+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:38:21.477482+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:38:45.780266+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:39:10.938791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:39:35.734857+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:40:00.169040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:40:26.494635+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:40:51.881718+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:41:17.697112+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:41:44.826479+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:42:10.504768+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:42:36.605447+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6},{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":6}]},{"pgid":"3.19","mappings":[{"from":3,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":7}]},{"pgid":"3.35","mappings":[{"from":1,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/3405015786":"2024-05-10T00:36:21.742363+0000","172.21.15.5:0/2669115557":"2024-05-10T00:36:21.742363+0000","172.21.15.5:6801/287842266":"2024-05-10T00:36:21.742363+0000","172.21.15.5:6800/752513054":"2024-05-10T00:35:53.128955+0000","172.21.15.5:6800/287842266":"2024-05-10T00:36:21.742363+0000","172.21.15.5:6801/752513054":"2024-05-10T00:35:53.128955+0000","172.21.15.5:0/1954876662":"2024-05-10T00:35:53.128955+0000","172.21.15.5:0/2545175121":"2024-05-10T00:36:21.742363+0000","172.21.15.5:0/3754653663":"2024-05-10T00:35:53.128955+0000","172.21.15.5:0/651514359":"2024-05-10T00:35:53.128955+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T01:01:43.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:43.867+0000 7fdf72766640 1 -- 172.21.15.5:0/4122215755 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fdf540b27f0 msgr2=0x7fdf540b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:43.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:43.867+0000 7fdf72766640 1 --2- 172.21.15.5:0/4122215755 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fdf540b27f0 0x7fdf540b4d00 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c0ff710 tx=0x7fdf6000a2d0 comp rx=0 tx=0).stop 2024-05-09T01:01:43.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:43.867+0000 7fdf72766640 1 -- 172.21.15.5:0/4122215755 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf6c102b30 msgr2=0x7fdf6c0ffd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:43.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:43.867+0000 7fdf72766640 1 --2- 172.21.15.5:0/4122215755 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf6c102b30 0x7fdf6c0ffd70 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c031440 tx=0x7fdf5c003d50 comp rx=0 tx=0).stop 2024-05-09T01:01:43.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:43.867+0000 7fdf72766640 1 -- 172.21.15.5:0/4122215755 shutdown_connections 2024-05-09T01:01:43.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:43.867+0000 7fdf72766640 1 --2- 172.21.15.5:0/4122215755 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fdf540b27f0 0x7fdf540b4d00 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:43.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:43.867+0000 7fdf72766640 1 --2- 172.21.15.5:0/4122215755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf6c108a70 0x7fdf6c0fe950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:43.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:43.867+0000 7fdf72766640 1 --2- 172.21.15.5:0/4122215755 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf6c103d70 0x7fdf6c0fe410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:43.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:43.867+0000 7fdf72766640 1 --2- 172.21.15.5:0/4122215755 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf6c102b30 0x7fdf6c0ffd70 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:43.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:43.867+0000 7fdf72766640 1 -- 172.21.15.5:0/4122215755 >> 172.21.15.5:0/4122215755 conn(0x7fdf6c0fdd20 msgr2=0x7fdf6c074f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:01:43.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:43.867+0000 7fdf72766640 1 -- 172.21.15.5:0/4122215755 shutdown_connections 2024-05-09T01:01:43.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:43.867+0000 7fdf72766640 1 -- 172.21.15.5:0/4122215755 wait complete. 2024-05-09T01:01:43.880 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T00:38:47.844734+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T00:43:39.452106+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T00:43:42.708106+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T01:01:43.880 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T01:01:44.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:43.998+0000 7fa7553be640 1 -- 172.21.15.5:0/1026841503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7501022a0 msgr2=0x7fa750102fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:44.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:43.998+0000 7fa7553be640 1 --2- 172.21.15.5:0/1026841503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7501022a0 0x7fa750102fa0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fa74000aa10 tx=0x7fa74002b720 comp rx=0 tx=0).stop 2024-05-09T01:01:44.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:43.998+0000 7fa7553be640 1 -- 172.21.15.5:0/1026841503 shutdown_connections 2024-05-09T01:01:44.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:43.998+0000 7fa7553be640 1 --2- 172.21.15.5:0/1026841503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7501081e0 0x7fa75010a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:44.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:43.998+0000 7fa7553be640 1 --2- 172.21.15.5:0/1026841503 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7501034e0 0x7fa7501038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:44.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:43.998+0000 7fa7553be640 1 --2- 172.21.15.5:0/1026841503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7501022a0 0x7fa750102fa0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:44.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:43.998+0000 7fa7553be640 1 -- 172.21.15.5:0/1026841503 >> 172.21.15.5:0/1026841503 conn(0x7fa7500fd4b0 msgr2=0x7fa7500ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:01:44.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:43.999+0000 7fa7553be640 1 -- 172.21.15.5:0/1026841503 shutdown_connections 2024-05-09T01:01:44.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:43.999+0000 7fa7553be640 1 -- 172.21.15.5:0/1026841503 wait complete. 2024-05-09T01:01:44.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:43.999+0000 7fa7553be640 1 Processor -- start 2024-05-09T01:01:44.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:43.999+0000 7fa7553be640 1 -- start start 2024-05-09T01:01:44.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.000+0000 7fa7553be640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7501022a0 0x7fa750198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:44.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.000+0000 7fa7553be640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7501034e0 0x7fa750198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:44.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.000+0000 7fa7553be640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7501081e0 0x7fa75019f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:44.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.000+0000 7fa7553be640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa75010d530 con 0x7fa7501034e0 2024-05-09T01:01:44.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.000+0000 7fa7553be640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa75010d3b0 con 0x7fa7501081e0 2024-05-09T01:01:44.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.000+0000 7fa7553be640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa75010d6b0 con 0x7fa7501022a0 2024-05-09T01:01:44.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.000+0000 7fa7467fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7501034e0 0x7fa750198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:44.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.000+0000 7fa7467fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7501034e0 0x7fa750198690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46242/0 (socket says 172.21.15.5:46242) 2024-05-09T01:01:44.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.000+0000 7fa7467fc640 1 -- 172.21.15.5:0/967937912 learned_addr learned my addr 172.21.15.5:0/967937912 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:01:44.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.000+0000 7fa74effd640 1 --2- 172.21.15.5:0/967937912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7501022a0 0x7fa750198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:44.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.000+0000 7fa74f7fe640 1 --2- 172.21.15.5:0/967937912 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7501081e0 0x7fa75019f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:44.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.000+0000 7fa7467fc640 1 -- 172.21.15.5:0/967937912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7501022a0 msgr2=0x7fa750198150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:44.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.000+0000 7fa7467fc640 1 --2- 172.21.15.5:0/967937912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7501022a0 0x7fa750198150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:44.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.000+0000 7fa7467fc640 1 -- 172.21.15.5:0/967937912 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7501081e0 msgr2=0x7fa75019f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:44.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.000+0000 7fa7467fc640 1 --2- 172.21.15.5:0/967937912 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7501081e0 0x7fa75019f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:44.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.000+0000 7fa7467fc640 1 -- 172.21.15.5:0/967937912 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa75019fc90 con 0x7fa7501034e0 2024-05-09T01:01:44.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.000+0000 7fa74f7fe640 1 --2- 172.21.15.5:0/967937912 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7501081e0 0x7fa75019f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:01:44.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.000+0000 7fa7467fc640 1 --2- 172.21.15.5:0/967937912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7501034e0 0x7fa750198690 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fa73800e9f0 tx=0x7fa73800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:01:44.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.000+0000 7fa74cff9640 1 -- 172.21.15.5:0/967937912 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa73800cd80 con 0x7fa7501034e0 2024-05-09T01:01:44.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.000+0000 7fa74cff9640 1 -- 172.21.15.5:0/967937912 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7380108e0 con 0x7fa7501034e0 2024-05-09T01:01:44.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.000+0000 7fa7553be640 1 -- 172.21.15.5:0/967937912 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa75019ff80 con 0x7fa7501034e0 2024-05-09T01:01:44.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.000+0000 7fa74cff9640 1 -- 172.21.15.5:0/967937912 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa738018980 con 0x7fa7501034e0 2024-05-09T01:01:44.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.001+0000 7fa7553be640 1 -- 172.21.15.5:0/967937912 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7501a03e0 con 0x7fa7501034e0 2024-05-09T01:01:44.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.001+0000 7fa7553be640 1 -- 172.21.15.5:0/967937912 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa750106ff0 con 0x7fa7501034e0 2024-05-09T01:01:44.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.003+0000 7fa74cff9640 1 -- 172.21.15.5:0/967937912 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fa738010430 con 0x7fa7501034e0 2024-05-09T01:01:44.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.004+0000 7fa74cff9640 1 --2- 172.21.15.5:0/967937912 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fa73c0b27f0 0x7fa73c0b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:44.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.004+0000 7fa74effd640 1 --2- 172.21.15.5:0/967937912 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fa73c0b27f0 0x7fa73c0b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:44.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.004+0000 7fa74cff9640 1 -- 172.21.15.5:0/967937912 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7fa7380e2a90 con 0x7fa7501034e0 2024-05-09T01:01:44.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.005+0000 7fa74effd640 1 --2- 172.21.15.5:0/967937912 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fa73c0b27f0 0x7fa73c0b4d00 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa740005ab0 tx=0x7fa740030510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:01:44.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.008+0000 7fa74cff9640 1 -- 172.21.15.5:0/967937912 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa7380aa780 con 0x7fa7501034e0 2024-05-09T01:01:44.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.202+0000 7fa7553be640 1 -- 172.21.15.5:0/967937912 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa750063900 con 0x7fa7501034e0 2024-05-09T01:01:44.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.203+0000 7fa74cff9640 1 -- 172.21.15.5:0/967937912 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7fa7380aa080 con 0x7fa7501034e0 2024-05-09T01:01:44.204 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-09T01:01:44.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.205+0000 7fa7553be640 1 -- 172.21.15.5:0/967937912 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fa73c0b27f0 msgr2=0x7fa73c0b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:44.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.205+0000 7fa7553be640 1 --2- 172.21.15.5:0/967937912 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fa73c0b27f0 0x7fa73c0b4d00 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa740005ab0 tx=0x7fa740030510 comp rx=0 tx=0).stop 2024-05-09T01:01:44.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.205+0000 7fa7553be640 1 -- 172.21.15.5:0/967937912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7501034e0 msgr2=0x7fa750198690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:44.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.205+0000 7fa7553be640 1 --2- 172.21.15.5:0/967937912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7501034e0 0x7fa750198690 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fa73800e9f0 tx=0x7fa73800eeb0 comp rx=0 tx=0).stop 2024-05-09T01:01:44.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.205+0000 7fa7553be640 1 -- 172.21.15.5:0/967937912 shutdown_connections 2024-05-09T01:01:44.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.205+0000 7fa7553be640 1 --2- 172.21.15.5:0/967937912 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fa73c0b27f0 0x7fa73c0b4d00 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:44.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.205+0000 7fa7553be640 1 --2- 172.21.15.5:0/967937912 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7501081e0 0x7fa75019f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:44.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.205+0000 7fa7553be640 1 --2- 172.21.15.5:0/967937912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7501034e0 0x7fa750198690 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:44.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.205+0000 7fa7553be640 1 --2- 172.21.15.5:0/967937912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7501022a0 0x7fa750198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:44.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.205+0000 7fa7553be640 1 -- 172.21.15.5:0/967937912 >> 172.21.15.5:0/967937912 conn(0x7fa7500fd4b0 msgr2=0x7fa7501041c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:01:44.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.205+0000 7fa7553be640 1 -- 172.21.15.5:0/967937912 shutdown_connections 2024-05-09T01:01:44.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.205+0000 7fa7553be640 1 -- 172.21.15.5:0/967937912 wait complete. 2024-05-09T01:01:44.218 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T01:01:44.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.337+0000 7f0107b6c640 1 -- 172.21.15.5:0/493224907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0100109b60 msgr2=0x7f010010bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:44.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.337+0000 7f0107b6c640 1 --2- 172.21.15.5:0/493224907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0100109b60 0x7f010010bf80 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f00f8009900 tx=0x7f00f802a410 comp rx=0 tx=0).stop 2024-05-09T01:01:44.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.337+0000 7f0107b6c640 1 -- 172.21.15.5:0/493224907 shutdown_connections 2024-05-09T01:01:44.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.337+0000 7f0107b6c640 1 --2- 172.21.15.5:0/493224907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0100109b60 0x7f010010bf80 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:44.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.337+0000 7f0107b6c640 1 --2- 172.21.15.5:0/493224907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0100104e60 0x7f0100105270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:44.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.337+0000 7f0107b6c640 1 --2- 172.21.15.5:0/493224907 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0100103c20 0x7f0100104920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:44.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.337+0000 7f0107b6c640 1 -- 172.21.15.5:0/493224907 >> 172.21.15.5:0/493224907 conn(0x7f01000fee10 msgr2=0x7f01001012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:01:44.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.337+0000 7f0107b6c640 1 -- 172.21.15.5:0/493224907 shutdown_connections 2024-05-09T01:01:44.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.337+0000 7f0107b6c640 1 -- 172.21.15.5:0/493224907 wait complete. 2024-05-09T01:01:44.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.338+0000 7f0107b6c640 1 Processor -- start 2024-05-09T01:01:44.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.338+0000 7f0107b6c640 1 -- start start 2024-05-09T01:01:44.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.338+0000 7f0107b6c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0100103c20 0x7f010019a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:44.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.338+0000 7f0107b6c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0100104e60 0x7f010019a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:44.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.338+0000 7f0107b6c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0100109b60 0x7f01001a1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:44.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.338+0000 7f0107b6c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f010010f7a0 con 0x7f0100109b60 2024-05-09T01:01:44.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.338+0000 7f0107b6c640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f010010f620 con 0x7f0100103c20 2024-05-09T01:01:44.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.338+0000 7f0107b6c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f010010f920 con 0x7f0100104e60 2024-05-09T01:01:44.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.338+0000 7f0106109640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0100109b60 0x7f01001a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:44.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.338+0000 7f0106109640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0100109b60 0x7f01001a1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46246/0 (socket says 172.21.15.5:46246) 2024-05-09T01:01:44.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.338+0000 7f0106109640 1 -- 172.21.15.5:0/2651537516 learned_addr learned my addr 172.21.15.5:0/2651537516 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:01:44.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.338+0000 7f0105107640 1 --2- 172.21.15.5:0/2651537516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0100104e60 0x7f010019a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:44.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.338+0000 7f0105908640 1 --2- 172.21.15.5:0/2651537516 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0100103c20 0x7f010019a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:44.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.338+0000 7f0106109640 1 -- 172.21.15.5:0/2651537516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0100104e60 msgr2=0x7f010019a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:44.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.338+0000 7f0106109640 1 --2- 172.21.15.5:0/2651537516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0100104e60 0x7f010019a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:44.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.338+0000 7f0106109640 1 -- 172.21.15.5:0/2651537516 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0100103c20 msgr2=0x7f010019a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:44.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.338+0000 7f0106109640 1 --2- 172.21.15.5:0/2651537516 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0100103c20 0x7f010019a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:44.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.338+0000 7f0106109640 1 -- 172.21.15.5:0/2651537516 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01001a1dd0 con 0x7f0100109b60 2024-05-09T01:01:44.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.338+0000 7f0105107640 1 --2- 172.21.15.5:0/2651537516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0100104e60 0x7f010019a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:01:44.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.338+0000 7f0105908640 1 --2- 172.21.15.5:0/2651537516 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0100103c20 0x7f010019a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:01:44.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.339+0000 7f0106109640 1 --2- 172.21.15.5:0/2651537516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0100109b60 0x7f01001a1810 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f00f802a930 tx=0x7f00f8003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:01:44.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.339+0000 7f00f6ffd640 1 -- 172.21.15.5:0/2651537516 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f803f070 con 0x7f0100109b60 2024-05-09T01:01:44.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.339+0000 7f0107b6c640 1 -- 172.21.15.5:0/2651537516 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01001a2060 con 0x7f0100109b60 2024-05-09T01:01:44.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.339+0000 7f00f6ffd640 1 -- 172.21.15.5:0/2651537516 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00f8031c50 con 0x7f0100109b60 2024-05-09T01:01:44.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.339+0000 7f00f6ffd640 1 -- 172.21.15.5:0/2651537516 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f8035030 con 0x7f0100109b60 2024-05-09T01:01:44.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.339+0000 7f0107b6c640 1 -- 172.21.15.5:0/2651537516 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01001a2550 con 0x7f0100109b60 2024-05-09T01:01:44.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.340+0000 7f0107b6c640 1 -- 172.21.15.5:0/2651537516 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0100108970 con 0x7f0100109b60 2024-05-09T01:01:44.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.341+0000 7f00f6ffd640 1 -- 172.21.15.5:0/2651537516 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f00f8042070 con 0x7f0100109b60 2024-05-09T01:01:44.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.343+0000 7f00f6ffd640 1 --2- 172.21.15.5:0/2651537516 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f00ec0b27f0 0x7f00ec0b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:44.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.343+0000 7f0105908640 1 --2- 172.21.15.5:0/2651537516 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f00ec0b27f0 0x7f00ec0b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:44.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.343+0000 7f00f6ffd640 1 -- 172.21.15.5:0/2651537516 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f00f8103300 con 0x7f0100109b60 2024-05-09T01:01:44.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.343+0000 7f0105908640 1 --2- 172.21.15.5:0/2651537516 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f00ec0b27f0 0x7f00ec0b4d00 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f0100104820 tx=0x7f00fc007870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:01:44.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.346+0000 7f00f6ffd640 1 -- 172.21.15.5:0/2651537516 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f00f80caff0 con 0x7f0100109b60 2024-05-09T01:01:44.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:01:44 smithi157 ceph-mon[34294]: pgmap v832: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 63 MiB/s wr, 374 op/s 2024-05-09T01:01:44.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:01:44 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/4122215755' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T01:01:44.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.539+0000 7f0107b6c640 1 -- 172.21.15.5:0/2651537516 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f01000638d0 con 0x7f0100109b60 2024-05-09T01:01:44.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.540+0000 7f00f6ffd640 1 -- 172.21.15.5:0/2651537516 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) ==== 104+0+11 (secure 0 0 0) 0x7f00f80d3070 con 0x7f0100109b60 2024-05-09T01:01:44.541 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-09T01:01:44.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:01:44 smithi005 ceph-mon[30365]: pgmap v832: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 63 MiB/s wr, 374 op/s 2024-05-09T01:01:44.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:01:44 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/4122215755' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T01:01:44.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.542+0000 7f0107b6c640 1 -- 172.21.15.5:0/2651537516 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f00ec0b27f0 msgr2=0x7f00ec0b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:44.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.542+0000 7f0107b6c640 1 --2- 172.21.15.5:0/2651537516 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f00ec0b27f0 0x7f00ec0b4d00 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f0100104820 tx=0x7f00fc007870 comp rx=0 tx=0).stop 2024-05-09T01:01:44.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.542+0000 7f0107b6c640 1 -- 172.21.15.5:0/2651537516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0100109b60 msgr2=0x7f01001a1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:44.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.542+0000 7f0107b6c640 1 --2- 172.21.15.5:0/2651537516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0100109b60 0x7f01001a1810 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f00f802a930 tx=0x7f00f8003b90 comp rx=0 tx=0).stop 2024-05-09T01:01:44.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.542+0000 7f0107b6c640 1 -- 172.21.15.5:0/2651537516 shutdown_connections 2024-05-09T01:01:44.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.542+0000 7f0107b6c640 1 --2- 172.21.15.5:0/2651537516 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f00ec0b27f0 0x7f00ec0b4d00 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:44.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.542+0000 7f0107b6c640 1 --2- 172.21.15.5:0/2651537516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0100109b60 0x7f01001a1810 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:44.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.542+0000 7f0107b6c640 1 --2- 172.21.15.5:0/2651537516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0100104e60 0x7f010019a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:44.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.542+0000 7f0107b6c640 1 --2- 172.21.15.5:0/2651537516 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0100103c20 0x7f010019a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:44.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.542+0000 7f0107b6c640 1 -- 172.21.15.5:0/2651537516 >> 172.21.15.5:0/2651537516 conn(0x7f01000fee10 msgr2=0x7f0100105b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:01:44.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.542+0000 7f0107b6c640 1 -- 172.21.15.5:0/2651537516 shutdown_connections 2024-05-09T01:01:44.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.542+0000 7f0107b6c640 1 -- 172.21.15.5:0/2651537516 wait complete. 2024-05-09T01:01:44.555 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T01:01:44.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:01:44 smithi131 ceph-mon[34547]: pgmap v832: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 63 MiB/s wr, 374 op/s 2024-05-09T01:01:44.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:01:44 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/4122215755' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T01:01:44.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.673+0000 7f4a27972640 1 -- 172.21.15.5:0/1415272066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a20101f70 msgr2=0x7f4a20102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:44.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.673+0000 7f4a27972640 1 --2- 172.21.15.5:0/1415272066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a20101f70 0x7f4a20102380 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c009900 tx=0x7f4a1c02a410 comp rx=0 tx=0).stop 2024-05-09T01:01:44.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.673+0000 7f4a27972640 1 -- 172.21.15.5:0/1415272066 shutdown_connections 2024-05-09T01:01:44.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.673+0000 7f4a27972640 1 --2- 172.21.15.5:0/1415272066 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a201098c0 0x7f4a2010bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:44.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.673+0000 7f4a27972640 1 --2- 172.21.15.5:0/1415272066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a20106d50 0x7f4a20109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:44.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.673+0000 7f4a27972640 1 --2- 172.21.15.5:0/1415272066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a20101f70 0x7f4a20102380 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:44.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.673+0000 7f4a27972640 1 -- 172.21.15.5:0/1415272066 >> 172.21.15.5:0/1415272066 conn(0x7f4a200fd480 msgr2=0x7f4a200ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:01:44.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.673+0000 7f4a27972640 1 -- 172.21.15.5:0/1415272066 shutdown_connections 2024-05-09T01:01:44.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.673+0000 7f4a27972640 1 -- 172.21.15.5:0/1415272066 wait complete. 2024-05-09T01:01:44.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.673+0000 7f4a27972640 1 Processor -- start 2024-05-09T01:01:44.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.673+0000 7f4a27972640 1 -- start start 2024-05-09T01:01:44.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.674+0000 7f4a27972640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a20101f70 0x7f4a2019a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:44.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.674+0000 7f4a27972640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a20106d50 0x7f4a2019a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:44.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.674+0000 7f4a27972640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a201098c0 0x7f4a201a17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:44.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.674+0000 7f4a27972640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4a2010f720 con 0x7f4a20101f70 2024-05-09T01:01:44.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.674+0000 7f4a27972640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4a2010f5a0 con 0x7f4a20106d50 2024-05-09T01:01:44.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.674+0000 7f4a27972640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4a2010f8a0 con 0x7f4a201098c0 2024-05-09T01:01:44.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.674+0000 7f4a2570e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a20101f70 0x7f4a2019a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:44.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.674+0000 7f4a2570e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a20101f70 0x7f4a2019a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46272/0 (socket says 172.21.15.5:46272) 2024-05-09T01:01:44.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.674+0000 7f4a2570e640 1 -- 172.21.15.5:0/1959256042 learned_addr learned my addr 172.21.15.5:0/1959256042 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:01:44.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.674+0000 7f4a25f0f640 1 --2- 172.21.15.5:0/1959256042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a201098c0 0x7f4a201a17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:44.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.674+0000 7f4a24f0d640 1 --2- 172.21.15.5:0/1959256042 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a20106d50 0x7f4a2019a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:44.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.674+0000 7f4a2570e640 1 -- 172.21.15.5:0/1959256042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a201098c0 msgr2=0x7f4a201a17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:44.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.674+0000 7f4a2570e640 1 --2- 172.21.15.5:0/1959256042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a201098c0 0x7f4a201a17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:44.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.674+0000 7f4a2570e640 1 -- 172.21.15.5:0/1959256042 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a20106d50 msgr2=0x7f4a2019a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:44.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.674+0000 7f4a2570e640 1 --2- 172.21.15.5:0/1959256042 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a20106d50 0x7f4a2019a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:44.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.674+0000 7f4a2570e640 1 -- 172.21.15.5:0/1959256042 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a201a1e30 con 0x7f4a20101f70 2024-05-09T01:01:44.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.674+0000 7f4a24f0d640 1 --2- 172.21.15.5:0/1959256042 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a20106d50 0x7f4a2019a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:01:44.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.674+0000 7f4a2570e640 1 --2- 172.21.15.5:0/1959256042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a20101f70 0x7f4a2019a2f0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c006010 tx=0x7f4a1c031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:01:44.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.674+0000 7f4a167fc640 1 -- 172.21.15.5:0/1959256042 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c03f030 con 0x7f4a20101f70 2024-05-09T01:01:44.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.675+0000 7f4a167fc640 1 -- 172.21.15.5:0/1959256042 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a1c003aa0 con 0x7f4a20101f70 2024-05-09T01:01:44.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.675+0000 7f4a167fc640 1 -- 172.21.15.5:0/1959256042 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c035070 con 0x7f4a20101f70 2024-05-09T01:01:44.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.675+0000 7f4a27972640 1 -- 172.21.15.5:0/1959256042 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a201a20c0 con 0x7f4a20101f70 2024-05-09T01:01:44.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.675+0000 7f4a27972640 1 -- 172.21.15.5:0/1959256042 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a201a25b0 con 0x7f4a20101f70 2024-05-09T01:01:44.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.676+0000 7f4a27972640 1 -- 172.21.15.5:0/1959256042 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a200fefa0 con 0x7f4a20101f70 2024-05-09T01:01:44.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.677+0000 7f4a167fc640 1 -- 172.21.15.5:0/1959256042 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f4a1c042030 con 0x7f4a20101f70 2024-05-09T01:01:44.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.678+0000 7f4a167fc640 1 --2- 172.21.15.5:0/1959256042 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f4a0c0b27f0 0x7f4a0c0b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:44.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.678+0000 7f4a24f0d640 1 --2- 172.21.15.5:0/1959256042 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f4a0c0b27f0 0x7f4a0c0b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:44.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.678+0000 7f4a167fc640 1 -- 172.21.15.5:0/1959256042 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f4a1c102970 con 0x7f4a20101f70 2024-05-09T01:01:44.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.679+0000 7f4a24f0d640 1 --2- 172.21.15.5:0/1959256042 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f4a0c0b27f0 0x7f4a0c0b4d00 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4a100058c0 tx=0x7f4a10005d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:01:44.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.683+0000 7f4a167fc640 1 -- 172.21.15.5:0/1959256042 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4a1c0ca600 con 0x7f4a20101f70 2024-05-09T01:01:44.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.877+0000 7f4a27972640 1 -- 172.21.15.5:0/1959256042 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4a200638d0 con 0x7f4a20101f70 2024-05-09T01:01:44.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.878+0000 7f4a167fc640 1 -- 172.21.15.5:0/1959256042 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) ==== 100+0+11 (secure 0 0 0) 0x7f4a1c0ec020 con 0x7f4a20101f70 2024-05-09T01:01:44.878 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-05-09T01:01:44.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.880+0000 7f4a27972640 1 -- 172.21.15.5:0/1959256042 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f4a0c0b27f0 msgr2=0x7f4a0c0b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:44.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.880+0000 7f4a27972640 1 --2- 172.21.15.5:0/1959256042 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f4a0c0b27f0 0x7f4a0c0b4d00 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4a100058c0 tx=0x7f4a10005d50 comp rx=0 tx=0).stop 2024-05-09T01:01:44.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.880+0000 7f4a27972640 1 -- 172.21.15.5:0/1959256042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a20101f70 msgr2=0x7f4a2019a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:44.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.880+0000 7f4a27972640 1 --2- 172.21.15.5:0/1959256042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a20101f70 0x7f4a2019a2f0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c006010 tx=0x7f4a1c031a50 comp rx=0 tx=0).stop 2024-05-09T01:01:44.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.880+0000 7f4a27972640 1 -- 172.21.15.5:0/1959256042 shutdown_connections 2024-05-09T01:01:44.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.880+0000 7f4a27972640 1 --2- 172.21.15.5:0/1959256042 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f4a0c0b27f0 0x7f4a0c0b4d00 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:44.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.880+0000 7f4a27972640 1 --2- 172.21.15.5:0/1959256042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a201098c0 0x7f4a201a17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:44.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.880+0000 7f4a27972640 1 --2- 172.21.15.5:0/1959256042 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a20106d50 0x7f4a2019a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:44.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.880+0000 7f4a27972640 1 --2- 172.21.15.5:0/1959256042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a20101f70 0x7f4a2019a2f0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:44.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.880+0000 7f4a27972640 1 -- 172.21.15.5:0/1959256042 >> 172.21.15.5:0/1959256042 conn(0x7f4a200fd480 msgr2=0x7f4a20107de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:01:44.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.880+0000 7f4a27972640 1 -- 172.21.15.5:0/1959256042 shutdown_connections 2024-05-09T01:01:44.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:44.880+0000 7f4a27972640 1 -- 172.21.15.5:0/1959256042 wait complete. 2024-05-09T01:01:44.892 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-05-09T01:01:45.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:45.007+0000 7f5ba1bce640 1 -- 172.21.15.5:0/3458836670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b9c103c40 msgr2=0x7f5b9c104940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:45.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:45.007+0000 7f5ba1bce640 1 --2- 172.21.15.5:0/3458836670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b9c103c40 0x7f5b9c104940 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c009900 tx=0x7f5b8c02a410 comp rx=0 tx=0).stop 2024-05-09T01:01:45.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:45.007+0000 7f5ba1bce640 1 -- 172.21.15.5:0/3458836670 shutdown_connections 2024-05-09T01:01:45.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:45.007+0000 7f5ba1bce640 1 --2- 172.21.15.5:0/3458836670 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b9c109b80 0x7f5b9c10bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:45.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:45.007+0000 7f5ba1bce640 1 --2- 172.21.15.5:0/3458836670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b9c104e80 0x7f5b9c105290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:45.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:45.007+0000 7f5ba1bce640 1 --2- 172.21.15.5:0/3458836670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b9c103c40 0x7f5b9c104940 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:45.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:45.007+0000 7f5ba1bce640 1 -- 172.21.15.5:0/3458836670 >> 172.21.15.5:0/3458836670 conn(0x7f5b9c0fee70 msgr2=0x7f5b9c1012e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:01:45.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:45.007+0000 7f5ba1bce640 1 -- 172.21.15.5:0/3458836670 shutdown_connections 2024-05-09T01:01:45.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:45.007+0000 7f5ba1bce640 1 -- 172.21.15.5:0/3458836670 wait complete. 2024-05-09T01:01:45.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:45.008+0000 7f5ba1bce640 1 Processor -- start 2024-05-09T01:01:45.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:45.008+0000 7f5ba1bce640 1 -- start start 2024-05-09T01:01:45.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:45.009+0000 7f5ba1bce640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b9c103c40 0x7f5b9c19a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:45.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:45.009+0000 7f5ba1bce640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b9c104e80 0x7f5b9c19a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:45.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:45.009+0000 7f5ba1bce640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b9c109b80 0x7f5b9c1a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:45.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:45.009+0000 7f5ba1bce640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c10f7a0 con 0x7f5b9c103c40 2024-05-09T01:01:45.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:45.009+0000 7f5ba1bce640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c10f620 con 0x7f5b9c104e80 2024-05-09T01:01:45.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:45.009+0000 7f5ba1bce640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c10f920 con 0x7f5b9c109b80 2024-05-09T01:01:45.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:45.009+0000 7f5b9b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b9c103c40 0x7f5b9c19a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:45.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:45.009+0000 7f5b9b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b9c103c40 0x7f5b9c19a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46300/0 (socket says 172.21.15.5:46300) 2024-05-09T01:01:45.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:45.009+0000 7f5b9b7fe640 1 -- 172.21.15.5:0/308566076 learned_addr learned my addr 172.21.15.5:0/308566076 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:01:45.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:45.009+0000 7f5b9affd640 1 --2- 172.21.15.5:0/308566076 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b9c104e80 0x7f5b9c19a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:45.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:45.009+0000 7f5b9bfff640 1 --2- 172.21.15.5:0/308566076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b9c109b80 0x7f5b9c1a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:45.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:45.009+0000 7f5b9b7fe640 1 -- 172.21.15.5:0/308566076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b9c109b80 msgr2=0x7f5b9c1a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:45.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:45.009+0000 7f5b9b7fe640 1 --2- 172.21.15.5:0/308566076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b9c109b80 0x7f5b9c1a18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:45.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:45.009+0000 7f5b9b7fe640 1 -- 172.21.15.5:0/308566076 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b9c104e80 msgr2=0x7f5b9c19a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:45.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:45.009+0000 7f5b9b7fe640 1 --2- 172.21.15.5:0/308566076 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b9c104e80 0x7f5b9c19a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:45.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:45.009+0000 7f5b9b7fe640 1 -- 172.21.15.5:0/308566076 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b9c1a1f40 con 0x7f5b9c103c40 2024-05-09T01:01:45.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:45.009+0000 7f5b9bfff640 1 --2- 172.21.15.5:0/308566076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b9c109b80 0x7f5b9c1a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:01:45.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:45.009+0000 7f5b9affd640 1 --2- 172.21.15.5:0/308566076 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b9c104e80 0x7f5b9c19a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:01:45.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:45.009+0000 7f5b9b7fe640 1 --2- 172.21.15.5:0/308566076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b9c103c40 0x7f5b9c19a400 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c032fd0 tx=0x7f5b8c0042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:01:45.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:45.009+0000 7f5b98ff9640 1 -- 172.21.15.5:0/308566076 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c03f070 con 0x7f5b9c103c40 2024-05-09T01:01:45.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:45.009+0000 7f5ba1bce640 1 -- 172.21.15.5:0/308566076 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b9c1a21d0 con 0x7f5b9c103c40 2024-05-09T01:01:45.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:45.009+0000 7f5b98ff9640 1 -- 172.21.15.5:0/308566076 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b8c0044d0 con 0x7f5b9c103c40 2024-05-09T01:01:45.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:45.009+0000 7f5b98ff9640 1 -- 172.21.15.5:0/308566076 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c035070 con 0x7f5b9c103c40 2024-05-09T01:01:45.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:45.009+0000 7f5ba1bce640 1 -- 172.21.15.5:0/308566076 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b9c1a2790 con 0x7f5b9c103c40 2024-05-09T01:01:45.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:45.010+0000 7f5ba1bce640 1 -- 172.21.15.5:0/308566076 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b9c108990 con 0x7f5b9c103c40 2024-05-09T01:01:45.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:45.013+0000 7f5b98ff9640 1 -- 172.21.15.5:0/308566076 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f5b8c03b5f0 con 0x7f5b9c103c40 2024-05-09T01:01:45.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:45.014+0000 7f5b98ff9640 1 --2- 172.21.15.5:0/308566076 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f5b840b27f0 0x7f5b840b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:01:45.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:45.014+0000 7f5b98ff9640 1 -- 172.21.15.5:0/308566076 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f5b8c102fe0 con 0x7f5b9c103c40 2024-05-09T01:01:45.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:45.014+0000 7f5b9affd640 1 --2- 172.21.15.5:0/308566076 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f5b840b27f0 0x7f5b840b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:01:45.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:45.014+0000 7f5b9affd640 1 --2- 172.21.15.5:0/308566076 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f5b840b27f0 0x7f5b840b4d00 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c19b4a0 tx=0x7f5b90008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:01:45.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:45.017+0000 7f5b98ff9640 1 -- 172.21.15.5:0/308566076 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5b8c0cac70 con 0x7f5b9c103c40 2024-05-09T01:01:45.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:45.272+0000 7f5ba1bce640 1 -- 172.21.15.5:0/308566076 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f5b9c0638d0 con 0x7f5b9c103c40 2024-05-09T01:01:45.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:45.282+0000 7f5b98ff9640 1 -- 172.21.15.5:0/308566076 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) ==== 81+0+815628 (secure 0 0 0) 0x7f5b8c0ca570 con 0x7f5b9c103c40 2024-05-09T01:01:45.294 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T01:01:45.294 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-09T01:01:45.295 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 18, 2024-05-09T01:01:45.295 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.295 INFO:teuthology.orchestra.run.smithi005.stdout: "active_gid": 14152, 2024-05-09T01:01:45.295 INFO:teuthology.orchestra.run.smithi005.stdout: "active_name": "x", 2024-05-09T01:01:45.295 INFO:teuthology.orchestra.run.smithi005.stdout: "active_addrs": { 2024-05-09T01:01:45.295 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-09T01:01:45.295 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:45.295 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-09T01:01:45.295 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6800", 2024-05-09T01:01:45.295 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 91856593 2024-05-09T01:01:45.295 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.295 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:45.295 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-09T01:01:45.295 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6801", 2024-05-09T01:01:45.296 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 91856593 2024-05-09T01:01:45.296 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.296 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-09T01:01:45.296 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.296 INFO:teuthology.orchestra.run.smithi005.stdout: "active_addr": "172.21.15.5:6801/91856593", 2024-05-09T01:01:45.296 INFO:teuthology.orchestra.run.smithi005.stdout: "active_change": "2024-05-09T00:36:21.742491+0000", 2024-05-09T01:01:45.296 INFO:teuthology.orchestra.run.smithi005.stdout: "active_mgr_features": 4540701547738038271, 2024-05-09T01:01:45.296 INFO:teuthology.orchestra.run.smithi005.stdout: "available": true, 2024-05-09T01:01:45.296 INFO:teuthology.orchestra.run.smithi005.stdout: "standbys": [ 2024-05-09T01:01:45.296 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:45.296 INFO:teuthology.orchestra.run.smithi005.stdout: "gid": 24106, 2024-05-09T01:01:45.296 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "z", 2024-05-09T01:01:45.296 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_features": 4540701547738038271, 2024-05-09T01:01:45.296 INFO:teuthology.orchestra.run.smithi005.stdout: "available_modules": [ 2024-05-09T01:01:45.297 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:45.297 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "alerts", 2024-05-09T01:01:45.297 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:45.297 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:45.297 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:45.297 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-05-09T01:01:45.297 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-05-09T01:01:45.297 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-09T01:01:45.297 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.297 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.297 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-05-09T01:01:45.297 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.297 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.297 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.298 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How frequently to reexamine health status", 2024-05-09T01:01:45.298 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.298 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.298 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.298 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.298 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:45.298 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:45.298 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.298 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.298 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.298 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.298 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.298 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.299 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.299 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.299 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.299 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.299 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.299 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.299 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.299 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.299 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.299 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.299 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.299 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.299 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.299 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:45.300 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:45.300 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.300 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.300 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.300 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.300 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.300 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.300 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.300 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.300 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.300 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.300 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.300 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.300 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:45.301 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:45.301 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.301 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.301 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.301 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:45.301 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.301 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.301 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.301 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.301 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.302 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.302 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.302 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.302 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.302 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.302 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.302 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.302 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.302 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.302 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.302 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:45.302 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:45.303 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.303 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.303 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.303 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.303 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.303 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.303 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.303 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.303 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.303 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.303 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.303 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.303 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_destination": { 2024-05-09T01:01:45.303 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_destination", 2024-05-09T01:01:45.304 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.304 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.304 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.304 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.304 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.304 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.304 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.304 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Email address to send alerts to", 2024-05-09T01:01:45.304 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.304 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.304 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.304 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.304 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_from_name": { 2024-05-09T01:01:45.304 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_from_name", 2024-05-09T01:01:45.305 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.305 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.305 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.305 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "Ceph", 2024-05-09T01:01:45.305 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.305 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.305 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.305 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Email From: name", 2024-05-09T01:01:45.305 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.305 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.305 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.305 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.305 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_host": { 2024-05-09T01:01:45.305 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_host", 2024-05-09T01:01:45.305 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.306 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.306 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.306 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.306 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.306 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.306 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.306 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP server", 2024-05-09T01:01:45.306 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.306 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.306 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.306 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.306 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_password": { 2024-05-09T01:01:45.307 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_password", 2024-05-09T01:01:45.307 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.307 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.307 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.307 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.307 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.307 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.307 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.307 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Password to authenticate with", 2024-05-09T01:01:45.307 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.307 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.307 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.307 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.307 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_port": { 2024-05-09T01:01:45.308 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_port", 2024-05-09T01:01:45.308 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.308 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.308 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.308 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "465", 2024-05-09T01:01:45.308 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.308 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.308 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.308 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP port", 2024-05-09T01:01:45.308 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.308 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.308 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.308 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.308 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_sender": { 2024-05-09T01:01:45.308 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_sender", 2024-05-09T01:01:45.309 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.309 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.309 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.309 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.309 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.309 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.309 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.309 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP envelope sender", 2024-05-09T01:01:45.309 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.309 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.309 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.309 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.309 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_ssl": { 2024-05-09T01:01:45.310 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_ssl", 2024-05-09T01:01:45.310 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.310 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.310 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.310 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:45.310 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.310 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.310 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.310 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-09T01:01:45.310 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.310 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.310 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.310 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.311 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_user": { 2024-05-09T01:01:45.311 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_user", 2024-05-09T01:01:45.311 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.311 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.311 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.311 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.311 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.311 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.311 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.311 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "User to authenticate as", 2024-05-09T01:01:45.311 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.311 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.311 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.311 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.312 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:45.312 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:45.312 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.312 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:45.312 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.312 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:45.312 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.312 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.312 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.312 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.313 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.313 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.313 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.313 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.313 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.313 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.313 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:45.313 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "balancer", 2024-05-09T01:01:45.313 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:45.313 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:45.313 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:45.313 INFO:teuthology.orchestra.run.smithi005.stdout: "active": { 2024-05-09T01:01:45.313 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "active", 2024-05-09T01:01:45.313 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.314 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.314 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.314 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:45.314 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.314 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.314 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.314 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically balance PGs across cluster", 2024-05-09T01:01:45.314 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.314 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.315 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.315 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.315 INFO:teuthology.orchestra.run.smithi005.stdout: "begin_time": { 2024-05-09T01:01:45.315 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "begin_time", 2024-05-09T01:01:45.315 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.315 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.315 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.315 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0000", 2024-05-09T01:01:45.315 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.315 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.315 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.315 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "beginning time of day to automatically balance", 2024-05-09T01:01:45.315 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T01:01:45.315 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.316 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.316 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.316 INFO:teuthology.orchestra.run.smithi005.stdout: "begin_weekday": { 2024-05-09T01:01:45.316 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "begin_weekday", 2024-05-09T01:01:45.316 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-05-09T01:01:45.316 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.316 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.316 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:45.316 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0", 2024-05-09T01:01:45.316 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "6", 2024-05-09T01:01:45.316 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.316 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-09T01:01:45.316 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T01:01:45.316 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.317 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.317 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.317 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_max_iterations": { 2024-05-09T01:01:45.317 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_max_iterations", 2024-05-09T01:01:45.317 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-05-09T01:01:45.317 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.317 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.317 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "25", 2024-05-09T01:01:45.317 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-05-09T01:01:45.317 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "250", 2024-05-09T01:01:45.317 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.317 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-09T01:01:45.317 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.317 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.318 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.318 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.318 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_metrics": { 2024-05-09T01:01:45.318 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_metrics", 2024-05-09T01:01:45.318 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.318 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.318 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.318 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "pgs,objects,bytes", 2024-05-09T01:01:45.318 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.318 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.318 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.319 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-09T01:01:45.319 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-09T01:01:45.319 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.319 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.319 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.319 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_step": { 2024-05-09T01:01:45.319 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_step", 2024-05-09T01:01:45.319 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-05-09T01:01:45.319 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.319 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.319 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0.5", 2024-05-09T01:01:45.319 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0.001", 2024-05-09T01:01:45.319 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "0.999", 2024-05-09T01:01:45.319 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.320 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "aggressiveness of optimization", 2024-05-09T01:01:45.320 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-09T01:01:45.320 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.320 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.320 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.320 INFO:teuthology.orchestra.run.smithi005.stdout: "end_time": { 2024-05-09T01:01:45.320 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "end_time", 2024-05-09T01:01:45.320 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.320 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.320 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.320 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "2359", 2024-05-09T01:01:45.320 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.320 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.320 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.321 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "ending time of day to automatically balance", 2024-05-09T01:01:45.321 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T01:01:45.321 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.321 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.321 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.321 INFO:teuthology.orchestra.run.smithi005.stdout: "end_weekday": { 2024-05-09T01:01:45.321 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "end_weekday", 2024-05-09T01:01:45.321 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-05-09T01:01:45.321 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.321 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.321 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:45.321 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0", 2024-05-09T01:01:45.321 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "6", 2024-05-09T01:01:45.321 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.322 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-09T01:01:45.322 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T01:01:45.322 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.322 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.322 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.322 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:45.322 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:45.322 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.322 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.322 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.322 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.323 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.323 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.323 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.323 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.323 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.323 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.323 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.323 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.323 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.323 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.323 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.324 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.324 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.324 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.324 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.324 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:45.324 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:45.324 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.324 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.324 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.324 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.324 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.325 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.325 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.325 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.325 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.325 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.325 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.325 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.325 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:45.325 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:45.325 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.325 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.325 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.325 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:45.325 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.326 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.326 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.326 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.326 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.326 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.326 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.326 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.326 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.326 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.326 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.326 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.326 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.326 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.327 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.327 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:45.327 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:45.327 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.327 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.327 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.327 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.327 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.327 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.327 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.327 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.327 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.327 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.328 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.328 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.328 INFO:teuthology.orchestra.run.smithi005.stdout: "min_score": { 2024-05-09T01:01:45.328 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "min_score", 2024-05-09T01:01:45.328 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-05-09T01:01:45.328 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.328 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.328 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:45.328 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.328 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.328 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.328 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-09T01:01:45.328 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.329 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.329 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.329 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.329 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": { 2024-05-09T01:01:45.329 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mode", 2024-05-09T01:01:45.329 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.329 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.329 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.329 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "upmap", 2024-05-09T01:01:45.329 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.330 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.330 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.330 INFO:teuthology.orchestra.run.smithi005.stdout: "crush-compat", 2024-05-09T01:01:45.330 INFO:teuthology.orchestra.run.smithi005.stdout: "none", 2024-05-09T01:01:45.330 INFO:teuthology.orchestra.run.smithi005.stdout: "read", 2024-05-09T01:01:45.330 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap", 2024-05-09T01:01:45.330 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap-read" 2024-05-09T01:01:45.330 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.330 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Balancer mode", 2024-05-09T01:01:45.330 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.330 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.330 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.331 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.331 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_ids": { 2024-05-09T01:01:45.331 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pool_ids", 2024-05-09T01:01:45.331 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.331 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.331 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.331 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.331 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.332 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.332 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.332 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-09T01:01:45.332 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.332 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.332 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.332 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.332 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-05-09T01:01:45.332 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-05-09T01:01:45.332 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-09T01:01:45.332 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.332 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.332 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-05-09T01:01:45.332 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.333 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.333 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.333 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-09T01:01:45.333 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.333 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.333 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.333 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.334 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:45.334 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:45.334 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.334 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:45.334 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.334 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:45.334 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.334 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.334 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.334 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.335 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.335 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.335 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.335 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.335 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap_max_deviation": { 2024-05-09T01:01:45.335 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "upmap_max_deviation", 2024-05-09T01:01:45.335 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.335 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.335 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.335 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-05-09T01:01:45.335 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-05-09T01:01:45.335 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.335 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.336 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-09T01:01:45.336 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-09T01:01:45.336 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.336 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.336 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.336 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap_max_optimizations": { 2024-05-09T01:01:45.336 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "upmap_max_optimizations", 2024-05-09T01:01:45.336 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-05-09T01:01:45.336 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.336 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.336 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-05-09T01:01:45.336 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.336 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.336 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.337 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-09T01:01:45.337 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.337 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.337 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.337 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.337 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.337 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.337 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:45.337 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cephadm", 2024-05-09T01:01:45.337 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:45.337 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:45.337 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:45.337 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_down_multiplier": { 2024-05-09T01:01:45.338 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_down_multiplier", 2024-05-09T01:01:45.338 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-05-09T01:01:45.338 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.338 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.338 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3.0", 2024-05-09T01:01:45.338 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.338 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.338 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.338 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-09T01:01:45.338 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.338 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.338 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.339 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.339 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_refresh_rate": { 2024-05-09T01:01:45.339 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_refresh_rate", 2024-05-09T01:01:45.339 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-09T01:01:45.339 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.339 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.339 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "20", 2024-05-09T01:01:45.339 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.339 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.339 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.339 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-09T01:01:45.339 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.339 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.340 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.340 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.340 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_starting_port": { 2024-05-09T01:01:45.340 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_starting_port", 2024-05-09T01:01:45.340 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.340 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.340 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.340 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "4721", 2024-05-09T01:01:45.340 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.340 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.340 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.340 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-09T01:01:45.340 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.340 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.341 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.341 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.341 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_ptrace": { 2024-05-09T01:01:45.341 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_ptrace", 2024-05-09T01:01:45.341 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.341 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.341 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.341 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.341 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.341 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.341 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.341 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-09T01:01:45.341 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-09T01:01:45.341 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.342 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.342 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.342 INFO:teuthology.orchestra.run.smithi005.stdout: "autotune_interval": { 2024-05-09T01:01:45.342 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "autotune_interval", 2024-05-09T01:01:45.342 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-09T01:01:45.342 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.342 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.342 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-05-09T01:01:45.342 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.342 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.342 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.342 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-09T01:01:45.343 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.343 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.343 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.343 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.343 INFO:teuthology.orchestra.run.smithi005.stdout: "autotune_memory_target_ratio": { 2024-05-09T01:01:45.343 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "autotune_memory_target_ratio", 2024-05-09T01:01:45.343 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-05-09T01:01:45.343 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.343 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.343 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0.7", 2024-05-09T01:01:45.343 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.343 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.343 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.344 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-09T01:01:45.344 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.344 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.344 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.344 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.344 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm_log_destination": { 2024-05-09T01:01:45.344 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cephadm_log_destination", 2024-05-09T01:01:45.344 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.344 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.345 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.345 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.345 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.345 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.345 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.345 INFO:teuthology.orchestra.run.smithi005.stdout: "file", 2024-05-09T01:01:45.345 INFO:teuthology.orchestra.run.smithi005.stdout: "file,syslog", 2024-05-09T01:01:45.345 INFO:teuthology.orchestra.run.smithi005.stdout: "syslog" 2024-05-09T01:01:45.345 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.345 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-09T01:01:45.345 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.345 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.345 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.345 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.346 INFO:teuthology.orchestra.run.smithi005.stdout: "cgroups_split": { 2024-05-09T01:01:45.346 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cgroups_split", 2024-05-09T01:01:45.346 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.346 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.346 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.346 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:45.346 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.346 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.346 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.346 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-09T01:01:45.346 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.346 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.346 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.347 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.347 INFO:teuthology.orchestra.run.smithi005.stdout: "config_checks_enabled": { 2024-05-09T01:01:45.347 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_checks_enabled", 2024-05-09T01:01:45.347 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.347 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.347 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.347 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.347 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.347 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.347 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.347 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-09T01:01:45.348 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.348 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.348 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.348 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.348 INFO:teuthology.orchestra.run.smithi005.stdout: "config_dashboard": { 2024-05-09T01:01:45.348 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_dashboard", 2024-05-09T01:01:45.348 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.348 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.348 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.348 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:45.348 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.348 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.348 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.349 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-09T01:01:45.349 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.349 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.349 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.349 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.349 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_alertmanager": { 2024-05-09T01:01:45.349 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_alertmanager", 2024-05-09T01:01:45.349 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.349 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.349 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.349 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-09T01:01:45.349 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.349 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.350 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.350 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-05-09T01:01:45.350 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.350 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.350 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.350 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.350 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_base": { 2024-05-09T01:01:45.350 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_base", 2024-05-09T01:01:45.350 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.350 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.350 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.350 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-09T01:01:45.350 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.351 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.351 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.351 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Container image name, without the tag", 2024-05-09T01:01:45.351 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.351 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.351 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.351 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.351 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_elasticsearch": { 2024-05-09T01:01:45.351 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_elasticsearch", 2024-05-09T01:01:45.351 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.351 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.351 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.351 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-09T01:01:45.351 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.352 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.352 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.352 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "elasticsearch container image", 2024-05-09T01:01:45.352 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.352 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.352 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.352 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.352 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_grafana": { 2024-05-09T01:01:45.352 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_grafana", 2024-05-09T01:01:45.352 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.352 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.352 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.353 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-09T01:01:45.353 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.353 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.353 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.353 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-05-09T01:01:45.353 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.353 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.353 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.353 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.353 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_haproxy": { 2024-05-09T01:01:45.353 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_haproxy", 2024-05-09T01:01:45.353 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.353 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.354 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.354 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-09T01:01:45.354 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.354 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.354 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.354 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "HAproxy container image", 2024-05-09T01:01:45.354 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.354 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.354 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.354 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.354 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_agent": { 2024-05-09T01:01:45.354 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_agent", 2024-05-09T01:01:45.354 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.355 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.355 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.355 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-09T01:01:45.355 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.355 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.355 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.355 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger agent container image", 2024-05-09T01:01:45.355 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.355 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.356 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.356 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.356 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_collector": { 2024-05-09T01:01:45.356 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_collector", 2024-05-09T01:01:45.356 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.356 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.356 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.356 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-09T01:01:45.356 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.356 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.356 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.356 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger collector container image", 2024-05-09T01:01:45.356 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.357 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.357 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.357 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.357 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_query": { 2024-05-09T01:01:45.357 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_query", 2024-05-09T01:01:45.357 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.357 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.357 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.357 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-09T01:01:45.357 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.357 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.358 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.358 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger query container image", 2024-05-09T01:01:45.358 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.358 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.358 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.358 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.358 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_keepalived": { 2024-05-09T01:01:45.358 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_keepalived", 2024-05-09T01:01:45.358 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.358 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.358 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.358 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-09T01:01:45.358 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.358 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.359 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.359 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Keepalived container image", 2024-05-09T01:01:45.359 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.359 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.359 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.359 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.359 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_loki": { 2024-05-09T01:01:45.359 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_loki", 2024-05-09T01:01:45.359 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.359 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.359 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.360 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-09T01:01:45.360 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.360 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.360 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.360 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Loki container image", 2024-05-09T01:01:45.360 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.360 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.360 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.360 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.360 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_node_exporter": { 2024-05-09T01:01:45.360 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_node_exporter", 2024-05-09T01:01:45.360 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.360 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.360 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.361 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-09T01:01:45.361 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.361 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.361 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.361 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-05-09T01:01:45.361 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.361 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.361 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.361 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.361 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_nvmeof": { 2024-05-09T01:01:45.361 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_nvmeof", 2024-05-09T01:01:45.361 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.361 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.361 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.362 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-09T01:01:45.362 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.362 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.362 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.362 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Nvme-of container image", 2024-05-09T01:01:45.362 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.362 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.362 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.362 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.362 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_prometheus": { 2024-05-09T01:01:45.362 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_prometheus", 2024-05-09T01:01:45.363 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.363 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.363 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.363 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-09T01:01:45.363 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.363 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.363 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.363 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-05-09T01:01:45.363 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.363 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.363 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.363 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.363 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_promtail": { 2024-05-09T01:01:45.363 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_promtail", 2024-05-09T01:01:45.363 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.364 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.364 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.364 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-09T01:01:45.364 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.364 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.364 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.364 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Promtail container image", 2024-05-09T01:01:45.364 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.364 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.364 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.364 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.364 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_samba": { 2024-05-09T01:01:45.364 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_samba", 2024-05-09T01:01:45.365 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.365 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.365 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.365 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-09T01:01:45.365 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.365 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.365 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.365 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Samba/SMB container image", 2024-05-09T01:01:45.365 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.366 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.366 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.366 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.366 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_snmp_gateway": { 2024-05-09T01:01:45.366 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_snmp_gateway", 2024-05-09T01:01:45.366 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.366 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.366 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.366 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-09T01:01:45.366 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.366 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.366 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.367 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SNMP Gateway container image", 2024-05-09T01:01:45.367 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.367 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.367 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.367 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.367 INFO:teuthology.orchestra.run.smithi005.stdout: "container_init": { 2024-05-09T01:01:45.367 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_init", 2024-05-09T01:01:45.367 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.367 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.367 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.367 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:45.367 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.367 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.368 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.368 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Run podman/docker with `--init`", 2024-05-09T01:01:45.368 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.368 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.368 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.368 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.368 INFO:teuthology.orchestra.run.smithi005.stdout: "daemon_cache_timeout": { 2024-05-09T01:01:45.368 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "daemon_cache_timeout", 2024-05-09T01:01:45.368 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-09T01:01:45.368 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.368 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.369 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-05-09T01:01:45.369 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.369 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.369 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.369 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-09T01:01:45.369 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.369 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.369 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.369 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.370 INFO:teuthology.orchestra.run.smithi005.stdout: "default_cephadm_command_timeout": { 2024-05-09T01:01:45.370 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "default_cephadm_command_timeout", 2024-05-09T01:01:45.370 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.370 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.370 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.370 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "900", 2024-05-09T01:01:45.370 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.370 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.370 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.370 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-09T01:01:45.370 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.370 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.370 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.371 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.371 INFO:teuthology.orchestra.run.smithi005.stdout: "default_registry": { 2024-05-09T01:01:45.371 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "default_registry", 2024-05-09T01:01:45.371 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.371 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.371 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.371 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io", 2024-05-09T01:01:45.371 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.371 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.371 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.371 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-09T01:01:45.371 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.371 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.372 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.372 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.372 INFO:teuthology.orchestra.run.smithi005.stdout: "device_cache_timeout": { 2024-05-09T01:01:45.372 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_cache_timeout", 2024-05-09T01:01:45.372 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-09T01:01:45.372 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.372 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.372 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1800", 2024-05-09T01:01:45.372 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.372 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.372 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.372 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache device inventory", 2024-05-09T01:01:45.372 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.372 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.373 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.373 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.373 INFO:teuthology.orchestra.run.smithi005.stdout: "device_enhanced_scan": { 2024-05-09T01:01:45.373 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_enhanced_scan", 2024-05-09T01:01:45.373 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.373 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.373 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.373 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.373 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.373 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.373 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.373 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-09T01:01:45.373 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.374 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.374 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.374 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.374 INFO:teuthology.orchestra.run.smithi005.stdout: "facts_cache_timeout": { 2024-05-09T01:01:45.374 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "facts_cache_timeout", 2024-05-09T01:01:45.374 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-09T01:01:45.374 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.374 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.374 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-05-09T01:01:45.374 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.374 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.374 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.374 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache host facts data", 2024-05-09T01:01:45.374 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.375 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.375 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.375 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.375 INFO:teuthology.orchestra.run.smithi005.stdout: "grafana_dashboards_path": { 2024-05-09T01:01:45.375 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "grafana_dashboards_path", 2024-05-09T01:01:45.375 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.375 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.375 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.375 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-09T01:01:45.375 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.375 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.375 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.376 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-09T01:01:45.376 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.376 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.376 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.376 INFO:teuthology.orchestra.run.smithi005.stdout: "host_check_interval": { 2024-05-09T01:01:45.376 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "host_check_interval", 2024-05-09T01:01:45.376 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-09T01:01:45.376 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.377 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.377 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-05-09T01:01:45.377 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.377 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.377 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.377 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to perform a host check", 2024-05-09T01:01:45.377 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.377 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.377 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.377 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.377 INFO:teuthology.orchestra.run.smithi005.stdout: "hw_monitoring": { 2024-05-09T01:01:45.377 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "hw_monitoring", 2024-05-09T01:01:45.377 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.377 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.377 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.378 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.378 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.378 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.378 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.378 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-09T01:01:45.378 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.378 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.378 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.378 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.378 INFO:teuthology.orchestra.run.smithi005.stdout: "inventory_list_all": { 2024-05-09T01:01:45.378 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "inventory_list_all", 2024-05-09T01:01:45.378 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.378 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.378 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.379 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.379 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.379 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.379 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.379 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-09T01:01:45.379 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.379 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.379 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.379 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.379 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:45.379 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:45.379 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.379 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.380 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.380 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.380 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.380 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.380 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.380 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.380 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.380 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.380 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.380 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.380 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.380 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.380 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.381 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.381 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.381 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.381 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.381 INFO:teuthology.orchestra.run.smithi005.stdout: "log_refresh_metadata": { 2024-05-09T01:01:45.381 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_refresh_metadata", 2024-05-09T01:01:45.381 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.381 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.381 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.381 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.381 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.381 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.381 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.381 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-09T01:01:45.382 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.382 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.382 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.382 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.382 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:45.382 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:45.382 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.382 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.382 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.382 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:45.382 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.382 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.382 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.383 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-09T01:01:45.383 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.383 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.383 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.383 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.383 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:45.383 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:45.383 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.383 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.383 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.383 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:45.383 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.383 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.383 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.383 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.384 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.384 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.384 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.384 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.384 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.384 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.384 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.384 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.384 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.384 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.384 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.385 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:45.385 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:45.385 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.385 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.385 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.385 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.385 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.385 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.385 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.385 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.385 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.385 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.385 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.386 INFO:teuthology.orchestra.run.smithi005.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-09T01:01:45.386 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-09T01:01:45.386 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.386 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.386 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.386 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.386 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.386 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.386 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.387 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-09T01:01:45.387 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.387 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.387 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.387 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.387 INFO:teuthology.orchestra.run.smithi005.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-09T01:01:45.387 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-09T01:01:45.387 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.387 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.387 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.387 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "*", 2024-05-09T01:01:45.387 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.387 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.388 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.388 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-09T01:01:45.388 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.388 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.388 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.388 INFO:teuthology.orchestra.run.smithi005.stdout: "max_count_per_host": { 2024-05-09T01:01:45.388 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_count_per_host", 2024-05-09T01:01:45.388 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.388 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.388 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.388 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-05-09T01:01:45.388 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.389 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.389 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.389 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "max number of daemons per service per host", 2024-05-09T01:01:45.389 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.389 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.389 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.389 INFO:teuthology.orchestra.run.smithi005.stdout: "max_osd_draining_count": { 2024-05-09T01:01:45.389 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_osd_draining_count", 2024-05-09T01:01:45.389 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.389 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.389 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.389 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-05-09T01:01:45.389 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.390 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.390 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.390 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-09T01:01:45.390 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.390 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.390 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.390 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.390 INFO:teuthology.orchestra.run.smithi005.stdout: "migration_current": { 2024-05-09T01:01:45.390 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "migration_current", 2024-05-09T01:01:45.390 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.390 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.390 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.390 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.391 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.391 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.391 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.391 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "internal - do not modify", 2024-05-09T01:01:45.391 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.391 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.391 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.391 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.391 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": { 2024-05-09T01:01:45.391 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mode", 2024-05-09T01:01:45.391 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.392 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.392 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.392 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "root", 2024-05-09T01:01:45.392 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.392 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.392 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.392 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm-package", 2024-05-09T01:01:45.392 INFO:teuthology.orchestra.run.smithi005.stdout: "root" 2024-05-09T01:01:45.392 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.392 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "mode for remote execution of cephadm", 2024-05-09T01:01:45.392 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.392 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.392 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.393 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.393 INFO:teuthology.orchestra.run.smithi005.stdout: "oob_default_addr": { 2024-05-09T01:01:45.393 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "oob_default_addr", 2024-05-09T01:01:45.393 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.393 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.393 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.393 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "169.254.1.1", 2024-05-09T01:01:45.393 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.393 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.393 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.393 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-09T01:01:45.393 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.394 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.394 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.394 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.394 INFO:teuthology.orchestra.run.smithi005.stdout: "prometheus_alerts_path": { 2024-05-09T01:01:45.394 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus_alerts_path", 2024-05-09T01:01:45.394 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.394 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.394 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.394 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-09T01:01:45.394 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.394 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.394 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.394 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-09T01:01:45.395 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.395 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.395 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.395 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.395 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_insecure": { 2024-05-09T01:01:45.395 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_insecure", 2024-05-09T01:01:45.395 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.395 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.395 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.395 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.395 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.395 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.395 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.396 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-09T01:01:45.396 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.396 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.396 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.396 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.396 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_password": { 2024-05-09T01:01:45.396 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_password", 2024-05-09T01:01:45.396 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.396 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.396 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.396 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.396 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.396 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.396 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.396 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-09T01:01:45.397 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.397 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.397 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.397 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.397 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_url": { 2024-05-09T01:01:45.397 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_url", 2024-05-09T01:01:45.397 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.397 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.397 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.397 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.398 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.398 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.398 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.398 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-09T01:01:45.398 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.398 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.398 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.398 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.398 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_username": { 2024-05-09T01:01:45.398 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_username", 2024-05-09T01:01:45.398 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.398 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.399 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.399 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.399 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.399 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.399 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.399 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-09T01:01:45.399 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.399 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.399 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.399 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.399 INFO:teuthology.orchestra.run.smithi005.stdout: "secure_monitoring_stack": { 2024-05-09T01:01:45.399 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "secure_monitoring_stack", 2024-05-09T01:01:45.399 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.399 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.399 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.400 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.400 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.400 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.400 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.400 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-09T01:01:45.400 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.400 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.400 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.400 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.400 INFO:teuthology.orchestra.run.smithi005.stdout: "service_discovery_port": { 2024-05-09T01:01:45.400 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "service_discovery_port", 2024-05-09T01:01:45.400 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.400 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.400 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.401 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8765", 2024-05-09T01:01:45.401 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.401 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.401 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.401 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "cephadm service discovery port", 2024-05-09T01:01:45.401 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.401 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.401 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.401 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.401 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:45.401 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:45.401 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.401 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:45.401 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.402 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:45.402 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.402 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.402 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.402 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.402 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.402 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.402 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.402 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.402 INFO:teuthology.orchestra.run.smithi005.stdout: "ssh_config_file": { 2024-05-09T01:01:45.402 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssh_config_file", 2024-05-09T01:01:45.402 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.403 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.403 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.403 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.403 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.403 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.403 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.403 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-09T01:01:45.403 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.403 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.403 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.403 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.403 INFO:teuthology.orchestra.run.smithi005.stdout: "use_agent": { 2024-05-09T01:01:45.403 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "use_agent", 2024-05-09T01:01:45.403 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.404 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.404 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.404 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.404 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.404 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.404 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.404 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-09T01:01:45.404 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.404 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.404 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.404 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.404 INFO:teuthology.orchestra.run.smithi005.stdout: "use_repo_digest": { 2024-05-09T01:01:45.404 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "use_repo_digest", 2024-05-09T01:01:45.404 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.405 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.405 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.405 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:45.405 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.405 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.405 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.405 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-09T01:01:45.405 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.405 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.405 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.405 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.405 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_failed_host_check": { 2024-05-09T01:01:45.405 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_failed_host_check", 2024-05-09T01:01:45.406 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.406 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.406 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.406 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:45.406 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.406 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.406 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.406 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if the host check fails", 2024-05-09T01:01:45.406 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.406 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.406 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.407 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.407 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_stray_daemons": { 2024-05-09T01:01:45.407 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_stray_daemons", 2024-05-09T01:01:45.407 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.407 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.407 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.407 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:45.407 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.407 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.408 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.408 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-09T01:01:45.408 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.408 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.408 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.408 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.408 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_stray_hosts": { 2024-05-09T01:01:45.408 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_stray_hosts", 2024-05-09T01:01:45.408 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.408 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.408 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.408 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:45.409 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.409 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.409 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.409 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-09T01:01:45.409 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.409 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.409 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.409 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.409 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.409 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.409 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:45.409 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crash", 2024-05-09T01:01:45.409 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:45.410 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:45.410 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:45.410 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:45.410 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:45.410 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.410 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.410 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.410 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.410 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.410 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.410 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.410 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.410 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.411 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.411 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.411 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.411 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.411 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.411 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.411 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.411 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.411 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.411 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.411 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:45.411 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:45.411 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.412 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.412 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.412 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.412 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.412 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.412 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.412 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.412 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.412 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.412 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.412 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.412 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:45.412 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:45.412 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.413 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.413 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.413 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:45.413 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.413 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.413 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.413 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.413 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.413 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.413 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.413 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.413 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.413 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.413 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.414 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.414 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.414 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.414 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.414 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:45.414 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:45.414 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.414 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.414 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.414 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.414 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.414 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.414 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.415 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.415 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.415 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.415 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.415 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.415 INFO:teuthology.orchestra.run.smithi005.stdout: "retain_interval": { 2024-05-09T01:01:45.415 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "retain_interval", 2024-05-09T01:01:45.415 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-09T01:01:45.415 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.415 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.415 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "31536000", 2024-05-09T01:01:45.415 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.416 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.416 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.416 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-09T01:01:45.416 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.416 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.416 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.416 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.416 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:45.416 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:45.416 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.416 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:45.416 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.416 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:45.417 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.417 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.417 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.417 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.417 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.417 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.417 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.417 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.417 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_recent_interval": { 2024-05-09T01:01:45.417 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_recent_interval", 2024-05-09T01:01:45.417 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-09T01:01:45.417 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.417 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.418 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1209600", 2024-05-09T01:01:45.418 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.418 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.418 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.418 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-09T01:01:45.418 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.418 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.419 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.419 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.419 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.419 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.419 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:45.419 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "dashboard", 2024-05-09T01:01:45.419 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:45.419 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:45.419 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:45.419 INFO:teuthology.orchestra.run.smithi005.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-09T01:01:45.419 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-09T01:01:45.419 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.419 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.420 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.420 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-05-09T01:01:45.420 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.420 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.420 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.420 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.420 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.420 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.420 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.420 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.420 INFO:teuthology.orchestra.run.smithi005.stdout: "ALERTMANAGER_API_HOST": { 2024-05-09T01:01:45.420 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-09T01:01:45.420 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.420 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.421 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.421 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.421 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.421 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.421 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.421 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.421 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.421 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.421 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.421 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.421 INFO:teuthology.orchestra.run.smithi005.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-09T01:01:45.421 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-09T01:01:45.421 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.421 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.422 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.422 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:45.422 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.422 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.422 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.422 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.422 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.422 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.422 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.422 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.422 INFO:teuthology.orchestra.run.smithi005.stdout: "AUDIT_API_ENABLED": { 2024-05-09T01:01:45.422 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "AUDIT_API_ENABLED", 2024-05-09T01:01:45.422 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.422 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.423 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.423 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.423 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.423 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.423 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.423 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.423 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.423 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.423 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.423 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.423 INFO:teuthology.orchestra.run.smithi005.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-09T01:01:45.423 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-09T01:01:45.423 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.423 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.424 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.424 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:45.424 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.424 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.424 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.424 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.424 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.424 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.424 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.424 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.424 INFO:teuthology.orchestra.run.smithi005.stdout: "ENABLE_BROWSABLE_API": { 2024-05-09T01:01:45.424 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-09T01:01:45.424 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.424 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.425 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.425 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:45.425 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.425 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.425 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.425 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.425 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.425 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.425 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.425 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.425 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-09T01:01:45.426 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-09T01:01:45.426 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.426 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.426 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.426 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:45.426 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.426 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.426 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.426 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.426 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.426 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.426 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.427 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.427 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-09T01:01:45.427 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-09T01:01:45.427 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.427 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.427 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.427 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:45.427 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.427 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.427 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.427 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.427 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.427 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.427 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.428 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.428 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-09T01:01:45.428 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-09T01:01:45.428 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.428 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.428 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.428 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:45.428 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.428 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.428 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.428 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.428 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.429 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.429 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.429 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.429 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-09T01:01:45.429 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-09T01:01:45.429 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.429 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.429 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.429 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:45.429 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.429 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.429 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.430 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.430 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.430 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.430 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.430 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.430 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-09T01:01:45.430 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-09T01:01:45.430 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.430 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.430 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.430 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:45.430 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.430 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.431 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.431 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.431 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.431 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.431 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.431 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.431 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-09T01:01:45.431 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-09T01:01:45.431 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.431 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.431 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.431 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:45.431 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.431 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.432 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.432 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.432 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.432 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.432 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.432 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.432 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-09T01:01:45.433 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-09T01:01:45.433 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.433 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.433 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.433 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:45.433 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.433 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.433 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.433 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.433 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.433 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.433 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.433 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.433 INFO:teuthology.orchestra.run.smithi005.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-09T01:01:45.434 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-09T01:01:45.434 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.434 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.434 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.434 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.434 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.434 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.434 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.434 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.434 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.434 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.434 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.434 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.434 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_PASSWORD": { 2024-05-09T01:01:45.435 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-09T01:01:45.435 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.435 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.435 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.435 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-05-09T01:01:45.435 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.435 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.435 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.435 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.435 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.435 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.435 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.435 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.436 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-09T01:01:45.436 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-09T01:01:45.436 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.436 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.436 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.436 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:45.436 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.436 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.436 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.436 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.436 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.436 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.436 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.436 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.437 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_URL": { 2024-05-09T01:01:45.437 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_URL", 2024-05-09T01:01:45.437 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.437 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.437 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.437 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.437 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.437 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.437 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.437 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.437 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.437 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.437 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.438 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.438 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_USERNAME": { 2024-05-09T01:01:45.438 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-09T01:01:45.438 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.438 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.438 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.438 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-05-09T01:01:45.438 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.438 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.438 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.438 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.438 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.438 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.439 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.439 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.439 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-09T01:01:45.439 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-09T01:01:45.439 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.439 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.439 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.439 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.439 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.440 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.440 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.440 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.440 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.440 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.440 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.440 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.440 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-09T01:01:45.440 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-09T01:01:45.440 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.441 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.441 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.441 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.441 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.441 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.441 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.441 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.441 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.441 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.441 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.441 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.441 INFO:teuthology.orchestra.run.smithi005.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-09T01:01:45.441 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-09T01:01:45.441 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.442 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.442 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.442 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:45.442 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.442 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.442 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.442 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.442 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.442 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.442 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.442 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.442 INFO:teuthology.orchestra.run.smithi005.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-09T01:01:45.442 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-09T01:01:45.443 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.443 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.443 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.443 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.443 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.443 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.443 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.443 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.443 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.443 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.443 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.443 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.443 INFO:teuthology.orchestra.run.smithi005.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-09T01:01:45.443 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-09T01:01:45.444 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.444 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.444 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.444 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "[]", 2024-05-09T01:01:45.444 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.444 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.444 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.444 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.444 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.444 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.444 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.444 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.444 INFO:teuthology.orchestra.run.smithi005.stdout: "MULTICLUSTER_CONFIG": { 2024-05-09T01:01:45.444 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-09T01:01:45.445 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.445 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.445 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.445 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "{}", 2024-05-09T01:01:45.445 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.445 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.445 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.445 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.445 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.445 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.445 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.445 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.446 INFO:teuthology.orchestra.run.smithi005.stdout: "PROMETHEUS_API_HOST": { 2024-05-09T01:01:45.446 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-09T01:01:45.446 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.446 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.446 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.446 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.446 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.446 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.446 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.446 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.446 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.446 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.446 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.446 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.446 INFO:teuthology.orchestra.run.smithi005.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-09T01:01:45.447 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-09T01:01:45.447 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.447 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.447 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.447 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:45.447 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.447 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.447 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.447 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.447 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.447 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.447 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.448 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.448 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-09T01:01:45.448 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-09T01:01:45.448 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.448 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.448 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.448 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.448 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.448 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.448 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.448 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.449 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.449 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.449 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.449 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.449 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-09T01:01:45.449 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-09T01:01:45.449 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.449 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.449 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.449 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.449 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.449 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.450 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.450 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.450 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.450 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.450 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.450 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.450 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-09T01:01:45.450 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-09T01:01:45.450 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.450 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.450 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.451 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:45.451 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.451 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.451 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.451 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.451 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.451 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.451 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.451 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.451 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-09T01:01:45.451 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-09T01:01:45.451 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.452 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.452 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.452 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:45.452 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.452 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.452 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.452 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.452 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.452 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.452 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.452 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.452 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-09T01:01:45.452 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-09T01:01:45.452 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.453 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.453 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.453 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.453 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.453 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.453 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.453 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.453 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.453 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.453 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.454 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.454 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-09T01:01:45.454 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-09T01:01:45.454 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.454 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.454 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.455 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.455 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.455 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.455 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.455 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.455 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.456 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.456 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.456 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.456 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-09T01:01:45.456 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-09T01:01:45.456 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.457 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.457 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.457 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.457 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.457 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.457 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.458 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.458 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.458 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.458 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.458 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.458 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_ENABLED": { 2024-05-09T01:01:45.459 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-09T01:01:45.459 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.459 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.459 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.459 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:45.459 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.460 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.460 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.460 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.460 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.460 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.460 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.461 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.461 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-09T01:01:45.461 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-09T01:01:45.461 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.461 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.461 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.462 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-09T01:01:45.462 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.462 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.462 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.462 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.463 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.463 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.463 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.463 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.463 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-09T01:01:45.463 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-09T01:01:45.463 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.463 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.463 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.464 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-05-09T01:01:45.464 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.464 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.464 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.464 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.464 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.465 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.465 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.465 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.465 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-09T01:01:45.465 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-09T01:01:45.465 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.466 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.466 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.466 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8", 2024-05-09T01:01:45.466 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.466 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.466 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.467 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.467 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.467 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.467 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.467 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.467 INFO:teuthology.orchestra.run.smithi005.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-09T01:01:45.467 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-09T01:01:45.468 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.468 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.468 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.468 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "45", 2024-05-09T01:01:45.468 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.468 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.469 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.469 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.469 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.469 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.469 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.469 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.470 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_ACCESS_KEY": { 2024-05-09T01:01:45.470 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-09T01:01:45.470 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.470 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.470 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.470 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.471 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.471 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.471 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.471 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.471 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.471 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.472 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.472 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.472 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-09T01:01:45.472 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-09T01:01:45.472 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.473 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.473 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.473 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-05-09T01:01:45.473 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.473 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.473 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.473 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.474 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.474 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.474 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.474 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.474 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_SECRET_KEY": { 2024-05-09T01:01:45.474 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-09T01:01:45.475 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.475 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.475 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.475 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.475 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.475 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.476 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.476 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.476 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.476 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.476 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.476 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.477 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_SSL_VERIFY": { 2024-05-09T01:01:45.477 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-09T01:01:45.477 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.477 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.477 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.478 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:45.478 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.478 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.478 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.478 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.478 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.479 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.479 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.479 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.479 INFO:teuthology.orchestra.run.smithi005.stdout: "UNSAFE_TLS_v1_2": { 2024-05-09T01:01:45.479 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-09T01:01:45.479 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.480 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.480 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.480 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.480 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.480 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.480 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.481 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.481 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.481 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.481 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.481 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.481 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-09T01:01:45.482 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-09T01:01:45.482 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.482 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.482 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.482 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:45.482 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.483 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.483 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.483 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.483 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.483 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.483 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.484 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.484 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-09T01:01:45.484 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-09T01:01:45.484 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.484 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.484 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.485 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-05-09T01:01:45.485 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.485 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.485 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.485 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.485 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.486 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.486 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.486 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.486 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-09T01:01:45.486 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-09T01:01:45.486 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.486 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.486 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.487 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-05-09T01:01:45.487 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.487 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.487 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.487 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.487 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.487 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.487 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.488 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.488 INFO:teuthology.orchestra.run.smithi005.stdout: "cross_origin_url": { 2024-05-09T01:01:45.488 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cross_origin_url", 2024-05-09T01:01:45.488 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.488 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.488 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.488 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.488 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.489 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.489 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.489 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.489 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.489 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.489 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.489 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.490 INFO:teuthology.orchestra.run.smithi005.stdout: "crt_file": { 2024-05-09T01:01:45.490 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crt_file", 2024-05-09T01:01:45.490 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.490 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.490 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.490 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.490 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.490 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.490 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.490 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.490 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.490 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.490 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.491 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.491 INFO:teuthology.orchestra.run.smithi005.stdout: "debug": { 2024-05-09T01:01:45.491 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "debug", 2024-05-09T01:01:45.491 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.491 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.491 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.491 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.491 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.491 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.491 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.491 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable/disable debug options", 2024-05-09T01:01:45.492 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.492 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.492 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.492 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.492 INFO:teuthology.orchestra.run.smithi005.stdout: "jwt_token_ttl": { 2024-05-09T01:01:45.492 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "jwt_token_ttl", 2024-05-09T01:01:45.492 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.492 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.492 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.492 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "28800", 2024-05-09T01:01:45.492 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.492 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.493 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.493 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.493 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.493 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.493 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.493 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.493 INFO:teuthology.orchestra.run.smithi005.stdout: "key_file": { 2024-05-09T01:01:45.493 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "key_file", 2024-05-09T01:01:45.493 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.493 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.493 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.493 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.493 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.494 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.494 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.494 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.494 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.494 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.494 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.494 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.494 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:45.494 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:45.494 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.494 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.494 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.494 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.495 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.495 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.495 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.495 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.495 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.495 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.495 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.495 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.495 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.495 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.495 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.495 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.495 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.495 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.496 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.496 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:45.496 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:45.496 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.496 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.496 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.496 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.496 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.496 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.496 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.496 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.496 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.496 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.496 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.497 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.497 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:45.497 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:45.497 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.497 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.497 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.497 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:45.497 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.497 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.497 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.497 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.497 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.497 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.497 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.498 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.498 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.498 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.498 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.498 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.498 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.498 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.498 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.498 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:45.498 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:45.498 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.499 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.499 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.499 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.499 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.499 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.499 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.499 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.499 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.499 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.499 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.499 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.499 INFO:teuthology.orchestra.run.smithi005.stdout: "motd": { 2024-05-09T01:01:45.499 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "motd", 2024-05-09T01:01:45.500 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.500 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.500 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.500 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.500 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.500 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.500 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.500 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "The message of the day", 2024-05-09T01:01:45.500 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.500 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.500 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.500 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.500 INFO:teuthology.orchestra.run.smithi005.stdout: "redirect_resolve_ip_addr": { 2024-05-09T01:01:45.501 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "redirect_resolve_ip_addr", 2024-05-09T01:01:45.501 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.501 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.501 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.501 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.501 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.501 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.501 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.501 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.501 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.501 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.501 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.501 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.501 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-05-09T01:01:45.501 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-05-09T01:01:45.502 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.502 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.502 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.502 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "::", 2024-05-09T01:01:45.502 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.502 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.502 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.502 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.502 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.502 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.502 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.502 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.503 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-05-09T01:01:45.503 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-05-09T01:01:45.503 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.503 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.503 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.503 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8080", 2024-05-09T01:01:45.503 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.503 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.503 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.503 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.503 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.503 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.503 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.504 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.504 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:45.504 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:45.504 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.504 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:45.504 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.504 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:45.504 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.504 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.504 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.504 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.504 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.504 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.505 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.505 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.505 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl": { 2024-05-09T01:01:45.505 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl", 2024-05-09T01:01:45.505 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.505 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.505 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.505 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:45.505 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.505 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.505 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.505 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.505 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.505 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.506 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.506 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.506 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl_server_port": { 2024-05-09T01:01:45.506 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl_server_port", 2024-05-09T01:01:45.506 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.506 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.506 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.506 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8443", 2024-05-09T01:01:45.506 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.506 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.506 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.506 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.506 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.506 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.507 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.507 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.507 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_behaviour": { 2024-05-09T01:01:45.507 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_behaviour", 2024-05-09T01:01:45.507 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.507 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.507 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.507 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "redirect", 2024-05-09T01:01:45.507 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.507 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.507 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.507 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.507 INFO:teuthology.orchestra.run.smithi005.stdout: "redirect" 2024-05-09T01:01:45.508 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.508 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.508 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.508 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.508 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.508 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.508 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_error_status_code": { 2024-05-09T01:01:45.508 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_error_status_code", 2024-05-09T01:01:45.508 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.508 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.508 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.508 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "500", 2024-05-09T01:01:45.508 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "400", 2024-05-09T01:01:45.508 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "599", 2024-05-09T01:01:45.509 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.509 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.509 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.509 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.509 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.509 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.509 INFO:teuthology.orchestra.run.smithi005.stdout: "url_prefix": { 2024-05-09T01:01:45.509 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "url_prefix", 2024-05-09T01:01:45.509 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.509 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.509 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.509 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.509 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.509 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.510 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.510 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.510 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.510 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.510 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.510 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.510 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.510 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.510 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:45.510 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "devicehealth", 2024-05-09T01:01:45.510 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:45.510 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:45.510 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:45.511 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_monitoring": { 2024-05-09T01:01:45.511 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enable_monitoring", 2024-05-09T01:01:45.511 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.511 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.511 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.511 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:45.511 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.511 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.511 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.511 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "monitor device health metrics", 2024-05-09T01:01:45.511 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.511 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.511 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.512 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.512 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:45.512 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:45.512 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.512 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.512 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.512 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.512 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.512 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.512 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.513 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.513 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.513 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.513 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.513 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.513 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.513 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.513 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.513 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.513 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.513 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.514 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.514 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:45.514 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:45.514 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.514 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.514 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.514 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.514 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.514 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.514 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.514 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.514 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.515 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.515 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.515 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.515 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:45.515 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:45.515 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.515 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.515 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.515 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:45.515 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.515 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.515 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.515 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.515 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.516 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.516 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.516 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.516 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.516 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.516 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.516 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.516 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.516 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.516 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.516 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:45.517 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:45.517 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.517 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.517 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.517 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.517 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.517 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.517 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.517 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.517 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.517 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.517 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.517 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.518 INFO:teuthology.orchestra.run.smithi005.stdout: "mark_out_threshold": { 2024-05-09T01:01:45.518 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mark_out_threshold", 2024-05-09T01:01:45.518 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-09T01:01:45.518 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.518 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.518 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "2419200", 2024-05-09T01:01:45.518 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.518 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.518 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.518 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-09T01:01:45.518 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.518 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.518 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.518 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.519 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": { 2024-05-09T01:01:45.519 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pool_name", 2024-05-09T01:01:45.519 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.519 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.519 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.519 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "device_health_metrics", 2024-05-09T01:01:45.519 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.519 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.519 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.519 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-09T01:01:45.519 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.519 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.519 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.520 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.520 INFO:teuthology.orchestra.run.smithi005.stdout: "retention_period": { 2024-05-09T01:01:45.520 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "retention_period", 2024-05-09T01:01:45.520 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-09T01:01:45.520 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.520 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.520 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15552000", 2024-05-09T01:01:45.520 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.520 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.520 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.520 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long to retain device health metrics", 2024-05-09T01:01:45.520 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.520 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.521 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.521 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.521 INFO:teuthology.orchestra.run.smithi005.stdout: "scrape_frequency": { 2024-05-09T01:01:45.521 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "scrape_frequency", 2024-05-09T01:01:45.521 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-09T01:01:45.521 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.521 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.521 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "86400", 2024-05-09T01:01:45.521 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.521 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.521 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.521 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-09T01:01:45.521 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.521 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.521 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.522 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.522 INFO:teuthology.orchestra.run.smithi005.stdout: "self_heal": { 2024-05-09T01:01:45.522 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "self_heal", 2024-05-09T01:01:45.522 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.522 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.522 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.522 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:45.522 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.522 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.522 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.522 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-09T01:01:45.522 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.522 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.523 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.523 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.523 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-05-09T01:01:45.523 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-05-09T01:01:45.523 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-09T01:01:45.523 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.523 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.523 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-05-09T01:01:45.523 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.524 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.524 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.524 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to wake up and check device health", 2024-05-09T01:01:45.524 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.524 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.524 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.524 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.524 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:45.524 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:45.524 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.524 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:45.524 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.524 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:45.524 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.525 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.525 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.525 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.525 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.525 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.525 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.525 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.525 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_threshold": { 2024-05-09T01:01:45.525 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_threshold", 2024-05-09T01:01:45.525 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-09T01:01:45.525 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.525 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.525 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "7257600", 2024-05-09T01:01:45.525 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.526 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.526 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.526 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-09T01:01:45.526 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.526 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.526 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.526 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.526 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.526 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.526 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:45.526 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "diskprediction_local", 2024-05-09T01:01:45.527 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:45.527 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:45.527 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:45.527 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:45.527 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:45.527 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.527 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.527 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.527 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.527 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.527 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.527 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.527 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.528 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.528 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.528 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.528 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.528 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.528 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.528 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.528 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.528 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.528 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.528 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.528 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:45.528 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:45.529 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.529 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.529 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.529 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.529 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.529 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.529 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.529 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.529 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.529 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.529 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.529 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.529 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:45.529 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:45.530 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.530 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.530 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.530 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:45.530 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.530 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.530 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.530 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.530 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.530 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.530 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.530 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.531 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.531 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.531 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.531 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.531 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.531 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.532 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.532 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:45.532 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:45.532 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.532 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.532 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.533 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.533 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.533 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.533 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.533 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.533 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.534 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.534 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.534 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.534 INFO:teuthology.orchestra.run.smithi005.stdout: "predict_interval": { 2024-05-09T01:01:45.534 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "predict_interval", 2024-05-09T01:01:45.534 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.534 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.535 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.535 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "86400", 2024-05-09T01:01:45.535 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.535 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.535 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.535 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.536 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.536 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.536 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.536 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.536 INFO:teuthology.orchestra.run.smithi005.stdout: "predictor_model": { 2024-05-09T01:01:45.536 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "predictor_model", 2024-05-09T01:01:45.536 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.537 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.537 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.537 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "prophetstor", 2024-05-09T01:01:45.537 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.537 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.537 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.537 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.537 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.538 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.538 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.538 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.538 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-05-09T01:01:45.538 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-05-09T01:01:45.538 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.539 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.539 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.539 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-05-09T01:01:45.539 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.539 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.539 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.540 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.540 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.540 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.540 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.540 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.540 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:45.540 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:45.541 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.541 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:45.541 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.541 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:45.541 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.541 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.541 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.541 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.541 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.541 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.541 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.541 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.541 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.542 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.542 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:45.542 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "influx", 2024-05-09T01:01:45.542 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": false, 2024-05-09T01:01:45.542 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "influxdb python module not found", 2024-05-09T01:01:45.542 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:45.542 INFO:teuthology.orchestra.run.smithi005.stdout: "batch_size": { 2024-05-09T01:01:45.542 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "batch_size", 2024-05-09T01:01:45.542 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.542 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.542 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.542 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5000", 2024-05-09T01:01:45.542 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.543 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.543 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.543 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-09T01:01:45.543 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.543 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.543 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.543 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.543 INFO:teuthology.orchestra.run.smithi005.stdout: "database": { 2024-05-09T01:01:45.543 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "database", 2024-05-09T01:01:45.543 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.543 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.543 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.543 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "ceph", 2024-05-09T01:01:45.543 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.544 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.544 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.544 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-09T01:01:45.544 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.544 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.544 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.544 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.544 INFO:teuthology.orchestra.run.smithi005.stdout: "hostname": { 2024-05-09T01:01:45.544 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "hostname", 2024-05-09T01:01:45.544 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.544 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.544 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.544 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.544 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.545 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.545 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.545 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB server hostname", 2024-05-09T01:01:45.545 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.545 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.545 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.545 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-05-09T01:01:45.545 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-05-09T01:01:45.545 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-09T01:01:45.545 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.545 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.545 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "30", 2024-05-09T01:01:45.546 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "5", 2024-05-09T01:01:45.546 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.546 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.546 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-09T01:01:45.546 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.546 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.546 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.546 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.546 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:45.546 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:45.546 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.547 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.547 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.547 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.547 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.547 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.547 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.547 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.547 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.547 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.547 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.547 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.547 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.547 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.547 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.548 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.548 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.548 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.548 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.548 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:45.548 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:45.548 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.548 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.548 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.548 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.549 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.549 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.549 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.549 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.549 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.549 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.549 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.549 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.549 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:45.549 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:45.549 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.549 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.549 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.549 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:45.550 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.550 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.550 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.550 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.550 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.550 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.550 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.550 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.550 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.550 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.550 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.550 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.551 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.551 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.551 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.551 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:45.551 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:45.551 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.551 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.551 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.551 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.551 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.551 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.551 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.551 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.551 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.552 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.552 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.552 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.552 INFO:teuthology.orchestra.run.smithi005.stdout: "password": { 2024-05-09T01:01:45.552 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "password", 2024-05-09T01:01:45.552 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.552 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.552 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.552 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.552 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.552 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.552 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.552 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "password of InfluxDB server user", 2024-05-09T01:01:45.553 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.553 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.553 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.553 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.553 INFO:teuthology.orchestra.run.smithi005.stdout: "port": { 2024-05-09T01:01:45.553 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "port", 2024-05-09T01:01:45.553 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.553 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.553 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.553 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8086", 2024-05-09T01:01:45.553 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.553 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.553 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.553 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB server port", 2024-05-09T01:01:45.554 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.554 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.554 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.554 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.554 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:45.554 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:45.554 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.554 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:45.554 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.554 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:45.554 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.554 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.554 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.555 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.555 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.555 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.555 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.555 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.555 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl": { 2024-05-09T01:01:45.555 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl", 2024-05-09T01:01:45.555 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.555 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.555 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.555 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "false", 2024-05-09T01:01:45.555 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.555 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.556 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.556 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T01:01:45.556 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.556 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.556 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.556 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.556 INFO:teuthology.orchestra.run.smithi005.stdout: "threads": { 2024-05-09T01:01:45.556 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "threads", 2024-05-09T01:01:45.556 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.556 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.556 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.556 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-05-09T01:01:45.556 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-05-09T01:01:45.556 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "32", 2024-05-09T01:01:45.556 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.557 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-09T01:01:45.557 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.557 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.557 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.557 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.557 INFO:teuthology.orchestra.run.smithi005.stdout: "username": { 2024-05-09T01:01:45.557 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "username", 2024-05-09T01:01:45.557 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.557 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.557 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.557 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.557 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.557 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.558 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.558 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "username of InfluxDB server user", 2024-05-09T01:01:45.558 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.558 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.558 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.558 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.558 INFO:teuthology.orchestra.run.smithi005.stdout: "verify_ssl": { 2024-05-09T01:01:45.558 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "verify_ssl", 2024-05-09T01:01:45.558 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.558 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.558 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.558 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "true", 2024-05-09T01:01:45.559 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.559 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.559 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.559 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T01:01:45.559 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.559 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.559 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.559 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.559 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.559 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.559 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:45.559 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "insights", 2024-05-09T01:01:45.559 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:45.559 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:45.560 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:45.560 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:45.560 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:45.560 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.560 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.560 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.560 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.560 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.560 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.560 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.560 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.560 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.560 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.560 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.561 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.561 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.561 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.561 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.561 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.561 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.561 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.561 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:45.561 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:45.562 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.562 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.562 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.562 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.562 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.562 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.562 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.562 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.562 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.562 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.562 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.563 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.563 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:45.563 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:45.563 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.563 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.563 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.563 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:45.563 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.563 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.563 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.563 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.563 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.564 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.564 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.564 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.564 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.564 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.564 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.564 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.564 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.564 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.564 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.564 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:45.564 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:45.564 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.564 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.565 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.565 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.565 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.565 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.565 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.565 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.565 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.565 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.565 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.565 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:45.565 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:45.565 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.565 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:45.566 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.566 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:45.566 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.566 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.566 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.566 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.566 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.566 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.567 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.567 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.567 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.567 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.567 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:45.567 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "iostat", 2024-05-09T01:01:45.567 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:45.567 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:45.567 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:45.567 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:45.567 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:45.567 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.568 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.568 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.568 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.568 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.568 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.568 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.568 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.568 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.568 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.568 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.568 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.568 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.569 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.569 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.569 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.569 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.569 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.569 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.569 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:45.569 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:45.569 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.569 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.569 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.570 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.570 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.570 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.570 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.570 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.570 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.570 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.570 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.570 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.570 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:45.570 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:45.571 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.571 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.571 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.571 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:45.571 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.571 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.571 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.571 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.571 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.571 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.571 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.571 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.572 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.572 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.572 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.572 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.572 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.572 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.572 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.572 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:45.572 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:45.572 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.572 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.572 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.573 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.573 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.573 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.573 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.573 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.573 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.573 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.573 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.573 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.573 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:45.573 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:45.573 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.573 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:45.574 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.574 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:45.574 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.574 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.574 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.574 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.574 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.574 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.574 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.574 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.574 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.574 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.575 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:45.575 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "k8sevents", 2024-05-09T01:01:45.575 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:45.575 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:45.575 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:45.575 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_event_retention_days": { 2024-05-09T01:01:45.575 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ceph_event_retention_days", 2024-05-09T01:01:45.575 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.575 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.575 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.575 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "7", 2024-05-09T01:01:45.575 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.575 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.576 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.576 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-09T01:01:45.576 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.576 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.576 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.576 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.576 INFO:teuthology.orchestra.run.smithi005.stdout: "config_check_secs": { 2024-05-09T01:01:45.576 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_check_secs", 2024-05-09T01:01:45.576 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.576 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.576 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.576 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-05-09T01:01:45.576 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "10", 2024-05-09T01:01:45.577 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.577 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.577 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-09T01:01:45.577 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.577 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.577 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.577 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:45.577 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:45.577 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.577 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.577 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.578 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.578 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.578 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.578 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.578 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.578 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.578 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.578 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.578 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.578 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.578 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.578 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.578 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.579 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.579 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.579 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:45.579 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:45.579 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.579 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.579 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.579 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.579 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.579 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.579 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.579 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.579 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.580 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.580 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.580 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:45.580 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:45.580 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.580 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.580 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.580 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:45.580 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.580 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.580 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.580 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.580 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.580 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.581 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.581 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.581 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.581 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.581 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.581 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.581 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.581 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.581 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:45.581 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:45.581 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.581 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.582 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.582 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.582 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.582 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.582 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.582 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.582 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.582 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.582 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.582 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:45.582 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:45.582 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.583 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:45.583 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.583 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:45.583 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.583 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.583 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.583 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.583 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.583 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.583 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.583 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.583 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.584 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:45.584 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "localpool", 2024-05-09T01:01:45.584 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:45.584 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:45.584 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:45.584 INFO:teuthology.orchestra.run.smithi005.stdout: "failure_domain": { 2024-05-09T01:01:45.584 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "failure_domain", 2024-05-09T01:01:45.584 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.584 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.584 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.584 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "host", 2024-05-09T01:01:45.584 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.584 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.584 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.585 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "failure domain for any created local pool", 2024-05-09T01:01:45.585 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-09T01:01:45.585 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.585 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.585 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:45.585 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:45.585 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.585 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.585 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.585 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.585 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.585 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.585 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.586 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.586 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.586 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.586 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.586 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.586 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.586 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.586 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.586 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.586 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.586 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.587 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:45.587 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:45.587 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.587 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.587 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.587 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.587 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.587 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.587 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.587 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.587 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.587 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.587 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.587 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:45.588 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:45.588 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.588 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.588 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.588 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:45.588 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.588 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.588 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.588 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.588 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.588 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.588 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.588 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.588 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.589 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.589 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.589 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.589 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.589 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.589 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:45.589 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:45.589 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.589 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.589 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.589 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.589 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.589 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.590 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.590 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.590 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.590 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.590 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.590 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": { 2024-05-09T01:01:45.590 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "min_size", 2024-05-09T01:01:45.590 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.590 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.590 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.590 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.590 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.591 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.591 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.591 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default min_size for any created local pool", 2024-05-09T01:01:45.591 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-09T01:01:45.591 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.591 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.591 INFO:teuthology.orchestra.run.smithi005.stdout: "num_rep": { 2024-05-09T01:01:45.591 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "num_rep", 2024-05-09T01:01:45.591 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.591 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.591 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.592 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3", 2024-05-09T01:01:45.592 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.592 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.592 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.592 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default replica count for any created local pool", 2024-05-09T01:01:45.592 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.592 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.592 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.592 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": { 2024-05-09T01:01:45.593 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pg_num", 2024-05-09T01:01:45.593 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.593 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.593 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.593 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "128", 2024-05-09T01:01:45.593 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.593 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.593 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.593 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default pg_num for any created local pool", 2024-05-09T01:01:45.593 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.593 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.593 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.594 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.594 INFO:teuthology.orchestra.run.smithi005.stdout: "prefix": { 2024-05-09T01:01:45.594 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prefix", 2024-05-09T01:01:45.594 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.594 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.594 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.594 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.594 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.594 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.594 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.594 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "name prefix for any created local pool", 2024-05-09T01:01:45.594 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.595 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.595 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.595 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:45.595 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:45.595 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.595 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:45.595 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.595 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:45.595 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.595 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.595 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.595 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.595 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.596 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.596 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.596 INFO:teuthology.orchestra.run.smithi005.stdout: "subtree": { 2024-05-09T01:01:45.596 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "subtree", 2024-05-09T01:01:45.596 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.596 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.596 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.596 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "rack", 2024-05-09T01:01:45.596 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.596 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.596 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.596 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-09T01:01:45.597 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-09T01:01:45.597 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.597 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.597 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.597 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.597 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:45.597 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mds_autoscaler", 2024-05-09T01:01:45.597 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:45.597 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:45.597 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:45.597 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:45.597 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:45.597 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.598 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.598 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.598 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.598 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.598 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.598 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.598 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.598 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.598 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.598 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.598 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.598 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.598 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.599 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.599 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.599 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.599 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.599 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.599 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:45.599 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:45.599 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.599 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.599 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.599 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.599 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.599 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.599 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.600 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.600 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.600 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.600 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.600 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:45.600 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:45.600 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.600 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.600 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.600 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:45.600 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.600 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.600 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.601 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.601 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.601 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.601 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.601 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.601 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.601 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.601 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.601 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.601 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.601 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.601 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:45.602 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:45.602 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.602 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.602 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.602 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.602 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.602 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.602 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.602 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.602 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.602 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.602 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.603 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.603 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:45.603 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:45.603 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.603 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:45.603 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.603 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:45.603 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.603 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.603 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.603 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.603 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.603 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.603 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.603 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.604 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.604 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:45.604 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mirroring", 2024-05-09T01:01:45.604 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:45.604 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:45.604 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:45.604 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:45.604 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:45.604 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.604 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.604 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.604 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.604 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.604 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.605 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.605 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.605 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.605 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.605 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.605 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.605 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.605 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.605 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.605 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.605 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.605 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.605 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.605 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:45.605 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:45.606 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.606 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.606 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.606 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.606 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.606 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.606 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.606 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.606 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.606 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.606 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.606 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.607 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:45.607 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:45.607 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.607 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.607 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.607 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:45.607 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.607 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.607 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.607 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.607 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.607 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.607 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.608 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.608 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.608 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.608 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.608 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.608 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.608 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.608 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:45.608 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:45.608 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.608 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.608 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.608 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.608 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.609 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.609 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.609 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.609 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.609 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.609 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.609 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:45.609 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:45.609 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.609 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:45.609 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.609 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:45.609 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.609 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.610 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.610 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.610 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.610 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.610 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.610 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.610 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.610 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:45.610 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "nfs", 2024-05-09T01:01:45.610 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:45.610 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:45.610 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:45.611 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:45.611 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:45.611 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.611 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.611 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.611 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.611 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.611 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.611 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.611 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.611 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.611 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.611 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.611 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.612 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.612 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.612 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.612 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.612 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.612 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.612 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:45.612 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:45.612 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.612 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.612 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.612 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.613 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.613 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.613 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.613 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.613 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.613 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.613 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.613 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:45.613 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:45.613 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.613 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.613 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.613 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:45.614 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.614 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.614 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.614 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.614 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.614 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.614 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.614 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.614 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.614 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.614 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.614 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.615 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.615 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.615 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.615 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:45.615 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:45.615 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.615 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.615 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.615 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.615 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.615 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.615 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.615 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.615 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.615 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.616 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.616 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.616 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:45.616 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:45.616 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.616 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:45.616 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.616 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:45.616 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.616 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.616 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.616 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.616 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.616 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.617 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.617 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.617 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.617 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:45.617 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "orchestrator", 2024-05-09T01:01:45.617 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:45.617 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:45.617 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:45.617 INFO:teuthology.orchestra.run.smithi005.stdout: "fail_fs": { 2024-05-09T01:01:45.617 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "fail_fs", 2024-05-09T01:01:45.618 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.618 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.618 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.618 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.618 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.618 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.618 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.618 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-09T01:01:45.618 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.618 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.618 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.619 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:45.619 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:45.619 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.619 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.619 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.619 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.619 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.619 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.619 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.619 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.619 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.619 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.619 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.619 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.620 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.620 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.620 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.620 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.620 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.620 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.620 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:45.620 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:45.620 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.620 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.621 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.621 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.621 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.621 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.621 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.621 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.621 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.621 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.621 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.621 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.621 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:45.621 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:45.621 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.622 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.622 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.622 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:45.622 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.622 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.622 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.622 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.622 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.622 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.622 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.622 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.622 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.622 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.623 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.623 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.623 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.623 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.623 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.623 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:45.623 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:45.623 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.623 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.623 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.623 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.623 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.623 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.623 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.624 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.624 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.624 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.624 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.624 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.624 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator": { 2024-05-09T01:01:45.624 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "orchestrator", 2024-05-09T01:01:45.624 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.624 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.624 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.624 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.624 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.624 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.624 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.625 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm", 2024-05-09T01:01:45.625 INFO:teuthology.orchestra.run.smithi005.stdout: "rook", 2024-05-09T01:01:45.625 INFO:teuthology.orchestra.run.smithi005.stdout: "test_orchestrator" 2024-05-09T01:01:45.625 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.625 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Orchestrator backend", 2024-05-09T01:01:45.625 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.625 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.625 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.625 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.625 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:45.625 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:45.625 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.625 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:45.625 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.625 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:45.626 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.626 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.626 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.626 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.626 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.626 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.626 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.626 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.626 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.626 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.626 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:45.626 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "osd_perf_query", 2024-05-09T01:01:45.626 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:45.627 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:45.627 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:45.627 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:45.627 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:45.627 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.627 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.627 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.627 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.627 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.627 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.627 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.628 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.628 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.628 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.628 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.628 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.628 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.628 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.628 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.628 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.628 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.628 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.628 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:45.628 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:45.629 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.629 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.629 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.629 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.629 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.629 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.629 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.629 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.629 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.629 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.629 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.629 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.629 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:45.629 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:45.630 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.630 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.630 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.630 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:45.630 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.630 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.630 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.630 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.630 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.630 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.630 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.630 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.630 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.630 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.631 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.631 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.631 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.631 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.631 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.631 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:45.631 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:45.631 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.631 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.631 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.631 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.631 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.632 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.632 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.632 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.632 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.632 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.632 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.632 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.632 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:45.632 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:45.632 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.632 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:45.632 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.632 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:45.633 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.633 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.633 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.633 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.633 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.633 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.633 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.633 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.633 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.633 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.633 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:45.633 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "osd_support", 2024-05-09T01:01:45.633 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:45.633 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:45.634 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:45.634 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:45.634 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:45.634 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.634 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.634 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.634 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.634 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.634 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.634 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.634 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.634 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.634 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.634 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.635 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.635 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.635 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.635 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.635 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.635 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.635 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.635 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.635 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:45.635 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:45.635 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.635 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.635 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.636 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.636 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.636 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.636 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.636 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.636 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.636 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.636 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.636 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.636 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:45.637 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:45.637 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.637 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.637 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.637 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:45.637 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.637 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.637 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.637 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.637 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.637 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.637 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.637 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.637 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.638 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.638 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.638 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.638 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.638 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.638 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.638 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:45.638 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:45.638 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.638 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.638 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.638 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.638 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.639 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.639 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.639 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.639 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.639 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.639 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.639 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.639 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:45.639 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:45.639 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.639 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:45.639 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.640 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:45.640 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.640 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.640 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.640 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.640 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.640 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.640 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.640 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.640 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.640 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.640 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:45.640 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pg_autoscaler", 2024-05-09T01:01:45.640 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:45.641 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:45.641 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:45.641 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:45.641 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:45.641 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.641 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.641 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.641 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.641 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.641 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.641 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.641 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.641 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.641 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.642 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.642 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.642 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.642 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.642 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.642 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.642 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.642 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.642 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.642 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:45.642 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:45.642 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.642 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.642 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.643 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.643 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.643 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.643 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.643 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.643 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.643 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.643 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.643 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:45.643 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:45.643 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.643 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.643 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.644 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:45.644 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.644 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.644 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.644 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.644 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.644 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.644 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.644 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.644 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.644 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.644 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.644 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.644 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.645 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.645 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.645 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:45.645 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:45.645 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.645 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.645 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.645 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.645 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.645 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.645 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.645 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.646 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.646 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.646 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.646 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.646 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-05-09T01:01:45.646 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-05-09T01:01:45.646 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-09T01:01:45.646 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.646 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.646 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-05-09T01:01:45.646 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.646 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.647 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.647 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.647 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.647 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.647 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.647 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.647 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:45.647 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:45.647 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.647 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:45.647 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.647 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:45.647 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.647 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.648 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.648 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.648 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.648 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.648 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.648 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.648 INFO:teuthology.orchestra.run.smithi005.stdout: "threshold": { 2024-05-09T01:01:45.648 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "threshold", 2024-05-09T01:01:45.648 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-05-09T01:01:45.648 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.648 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.648 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3.0", 2024-05-09T01:01:45.648 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1.0", 2024-05-09T01:01:45.649 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.649 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.649 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "scaling threshold", 2024-05-09T01:01:45.649 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-09T01:01:45.649 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.649 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.649 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.649 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.649 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.650 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:45.650 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "progress", 2024-05-09T01:01:45.650 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:45.650 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:45.650 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:45.650 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_pg_recovery_event": { 2024-05-09T01:01:45.650 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_pg_recovery_event", 2024-05-09T01:01:45.650 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.650 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.650 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.650 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.651 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.651 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.651 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.651 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-09T01:01:45.651 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.651 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.651 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.651 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.651 INFO:teuthology.orchestra.run.smithi005.stdout: "enabled": { 2024-05-09T01:01:45.651 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enabled", 2024-05-09T01:01:45.651 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.651 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.651 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.652 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:45.652 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.652 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.652 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.652 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.652 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.652 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.652 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.652 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.652 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:45.652 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:45.652 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.653 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.653 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.653 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.653 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.653 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.653 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.653 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.653 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.653 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.653 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.653 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.653 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.653 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.654 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.654 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.654 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.654 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.654 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.654 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:45.654 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:45.654 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.654 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.654 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.654 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.654 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.655 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.655 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.655 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.655 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.655 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.655 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.655 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.655 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:45.655 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:45.655 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.655 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.655 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.655 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:45.655 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.656 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.656 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.656 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.656 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.656 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.656 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.656 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.656 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.656 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.656 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.656 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.657 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.657 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.657 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.657 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:45.657 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:45.657 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.657 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.657 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.657 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.657 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.657 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.657 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.657 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.658 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.658 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.658 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.658 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.658 INFO:teuthology.orchestra.run.smithi005.stdout: "max_completed_events": { 2024-05-09T01:01:45.658 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_completed_events", 2024-05-09T01:01:45.658 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.658 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.658 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.658 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "50", 2024-05-09T01:01:45.658 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.658 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.658 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.659 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "number of past completed events to remember", 2024-05-09T01:01:45.659 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.659 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.659 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.659 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.659 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-05-09T01:01:45.659 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-05-09T01:01:45.659 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-09T01:01:45.659 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.659 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.660 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-05-09T01:01:45.660 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.660 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.660 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.660 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long the module is going to sleep", 2024-05-09T01:01:45.660 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.660 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.660 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.660 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.660 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:45.660 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:45.660 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.661 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:45.661 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.661 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:45.661 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.661 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.661 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.661 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.661 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.661 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.661 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.661 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.661 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.661 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.662 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:45.662 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus", 2024-05-09T01:01:45.662 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:45.662 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:45.662 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:45.662 INFO:teuthology.orchestra.run.smithi005.stdout: "cache": { 2024-05-09T01:01:45.662 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cache", 2024-05-09T01:01:45.662 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.662 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.662 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.663 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:45.663 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.663 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.663 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.663 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.663 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.663 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.663 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.663 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.663 INFO:teuthology.orchestra.run.smithi005.stdout: "exclude_perf_counters": { 2024-05-09T01:01:45.663 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "exclude_perf_counters", 2024-05-09T01:01:45.664 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.664 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.664 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.664 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:45.664 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.664 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.664 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.664 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-09T01:01:45.664 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-09T01:01:45.664 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.664 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.664 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.665 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:45.665 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:45.665 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.665 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.665 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.665 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.665 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.665 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.665 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.665 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.665 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.665 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.665 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.666 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.666 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.666 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.666 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.666 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.666 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.666 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.666 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.666 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:45.667 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:45.667 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.667 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.667 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.667 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.667 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.667 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.667 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.667 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.667 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.667 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.667 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.667 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.668 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:45.668 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:45.668 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.668 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.668 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.668 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:45.668 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.668 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.668 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.668 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.668 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.668 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.669 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.669 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.669 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.669 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.669 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.669 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.669 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.669 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.669 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.669 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:45.669 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:45.669 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.670 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.670 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.670 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.670 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.670 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.670 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.670 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.670 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.671 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.671 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.671 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.671 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_stats_pools": { 2024-05-09T01:01:45.671 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_stats_pools", 2024-05-09T01:01:45.671 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.672 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.672 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.672 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.672 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.672 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.672 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.673 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.673 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.673 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.673 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.673 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.673 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-09T01:01:45.674 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-09T01:01:45.674 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.674 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.674 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.675 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "300", 2024-05-09T01:01:45.675 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.675 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.675 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.675 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.675 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.676 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.676 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.676 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.676 INFO:teuthology.orchestra.run.smithi005.stdout: "scrape_interval": { 2024-05-09T01:01:45.676 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "scrape_interval", 2024-05-09T01:01:45.676 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-05-09T01:01:45.676 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.677 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.677 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15.0", 2024-05-09T01:01:45.677 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.677 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.677 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.677 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.677 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.678 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.678 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.678 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.678 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-05-09T01:01:45.678 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-05-09T01:01:45.678 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.678 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.679 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.679 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "::", 2024-05-09T01:01:45.679 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.679 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.679 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.679 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-09T01:01:45.679 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.680 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.680 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.680 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.680 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-05-09T01:01:45.680 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-05-09T01:01:45.680 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.680 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.680 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.681 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "9283", 2024-05-09T01:01:45.681 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.681 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.681 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.681 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-09T01:01:45.681 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.681 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.682 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.682 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.682 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:45.682 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:45.682 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.683 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:45.683 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.683 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:45.683 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.683 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.683 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.684 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.684 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.684 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.684 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.684 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.684 INFO:teuthology.orchestra.run.smithi005.stdout: "stale_cache_strategy": { 2024-05-09T01:01:45.684 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "stale_cache_strategy", 2024-05-09T01:01:45.685 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.685 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.685 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.685 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "log", 2024-05-09T01:01:45.685 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.686 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.686 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.686 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.686 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.686 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.687 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.687 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.687 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_behaviour": { 2024-05-09T01:01:45.687 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_behaviour", 2024-05-09T01:01:45.687 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.687 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.687 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.687 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "default", 2024-05-09T01:01:45.687 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.687 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.688 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.688 INFO:teuthology.orchestra.run.smithi005.stdout: "default", 2024-05-09T01:01:45.688 INFO:teuthology.orchestra.run.smithi005.stdout: "error" 2024-05-09T01:01:45.688 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.688 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.688 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.688 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.688 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.688 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.688 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_error_status_code": { 2024-05-09T01:01:45.688 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_error_status_code", 2024-05-09T01:01:45.689 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.689 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.689 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.689 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "500", 2024-05-09T01:01:45.689 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "400", 2024-05-09T01:01:45.689 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "599", 2024-05-09T01:01:45.689 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.689 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.689 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.689 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.690 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.690 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.690 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.690 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.690 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:45.690 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_support", 2024-05-09T01:01:45.690 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:45.690 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:45.690 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:45.691 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:45.691 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:45.691 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.691 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.691 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.691 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.691 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.691 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.691 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.691 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.691 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.691 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.692 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.692 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.692 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.692 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.692 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.692 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.692 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.692 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.692 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.692 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:45.692 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:45.692 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.692 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.693 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.693 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.693 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.693 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.693 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.693 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.693 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.693 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.693 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.693 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.693 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:45.694 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:45.694 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.694 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.694 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.694 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:45.694 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.694 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.694 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.695 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.695 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.695 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.695 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.695 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.695 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.695 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.695 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.695 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.695 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.695 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.695 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.696 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:45.696 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:45.696 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.696 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.696 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.696 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.696 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.696 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.696 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.696 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.696 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.696 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.696 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.697 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.697 INFO:teuthology.orchestra.run.smithi005.stdout: "max_concurrent_snap_create": { 2024-05-09T01:01:45.697 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_concurrent_snap_create", 2024-05-09T01:01:45.697 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.697 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.697 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.697 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-05-09T01:01:45.697 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.697 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.698 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.698 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.698 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.698 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.698 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.698 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.698 INFO:teuthology.orchestra.run.smithi005.stdout: "mirror_snapshot_schedule": { 2024-05-09T01:01:45.698 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mirror_snapshot_schedule", 2024-05-09T01:01:45.698 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.698 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.699 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.699 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.699 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.699 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.699 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.699 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.699 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.699 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.699 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.699 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.699 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:45.700 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:45.700 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.700 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:45.700 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.700 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:45.700 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.700 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.700 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.700 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.700 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.701 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.701 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.701 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.701 INFO:teuthology.orchestra.run.smithi005.stdout: "trash_purge_schedule": { 2024-05-09T01:01:45.701 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "trash_purge_schedule", 2024-05-09T01:01:45.701 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.701 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.701 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.701 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.701 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.701 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.701 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.701 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.701 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.702 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.702 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.702 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.702 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.702 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.702 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:45.702 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "restful", 2024-05-09T01:01:45.702 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:45.702 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:45.702 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:45.702 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_auth": { 2024-05-09T01:01:45.702 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enable_auth", 2024-05-09T01:01:45.703 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.703 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.703 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.703 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:45.703 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.703 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.703 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.703 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.703 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.703 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.703 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.704 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.704 INFO:teuthology.orchestra.run.smithi005.stdout: "key_file": { 2024-05-09T01:01:45.704 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "key_file", 2024-05-09T01:01:45.704 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.704 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.704 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.704 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.704 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.704 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.704 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.704 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.704 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.704 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.705 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.705 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.705 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:45.705 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:45.705 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.705 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.705 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.705 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.705 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.705 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.705 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.705 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.706 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.706 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.706 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.706 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.706 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.706 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.706 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.706 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.706 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.706 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.706 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.706 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:45.707 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:45.707 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.707 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.707 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.707 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.707 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.707 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.707 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.707 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.707 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.708 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.708 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.708 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.708 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:45.708 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:45.708 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.708 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.708 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.708 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:45.708 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.708 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.708 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.709 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.709 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.709 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.709 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.709 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.709 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.710 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.710 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.710 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.710 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.710 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.710 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.711 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:45.711 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:45.711 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.711 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.711 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.711 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.711 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.711 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.711 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.711 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.712 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.712 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.712 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.712 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.712 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-05-09T01:01:45.712 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-05-09T01:01:45.712 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.712 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.712 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.712 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.712 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.712 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.712 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.713 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.713 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.713 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.713 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.713 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.713 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-05-09T01:01:45.713 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-05-09T01:01:45.713 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.713 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.713 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.714 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.714 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.714 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.714 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.714 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.714 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.714 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.714 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.714 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.714 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:45.715 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:45.715 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.715 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:45.715 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.715 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:45.715 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.715 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.715 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.715 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.715 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.716 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.716 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.716 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.716 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.716 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.716 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:45.716 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rgw", 2024-05-09T01:01:45.717 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:45.717 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:45.717 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:45.717 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:45.717 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:45.717 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.717 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.717 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.717 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.717 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.717 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.718 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.718 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.718 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.718 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.718 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.718 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.718 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.718 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.719 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.719 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.719 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.719 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.719 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.719 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:45.719 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:45.719 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.719 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.719 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.720 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.720 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.720 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.720 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.720 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.720 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.720 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.720 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.720 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.721 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:45.721 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:45.721 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.721 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.721 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.721 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:45.721 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.721 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.722 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.722 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.722 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.722 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.722 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.722 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.722 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.722 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.722 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.723 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.723 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.723 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.723 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.723 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:45.723 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:45.723 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.723 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.723 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.723 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.723 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.724 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.724 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.724 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.724 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.724 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.724 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.724 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.724 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:45.724 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:45.725 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.725 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:45.725 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.725 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:45.725 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.725 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.725 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.725 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.725 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.725 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.725 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.726 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.726 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.726 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.726 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:45.726 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rook", 2024-05-09T01:01:45.726 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:45.726 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:45.726 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:45.726 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:45.726 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:45.726 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.726 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.727 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.727 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.727 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.727 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.727 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.727 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.727 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.727 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.727 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.727 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.727 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.727 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.728 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.728 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.728 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.728 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.728 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.728 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:45.728 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:45.728 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.728 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.729 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.729 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.729 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.729 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.729 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.729 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.729 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.729 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.729 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.730 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.730 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:45.730 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:45.730 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.730 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.730 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.730 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:45.730 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.730 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.731 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.731 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.731 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.731 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.731 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.731 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.731 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.731 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.731 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.731 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.732 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.732 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.732 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.732 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:45.732 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:45.732 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.732 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.732 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.732 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.732 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.733 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.733 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.733 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.733 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.733 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.733 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.733 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.733 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:45.733 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:45.733 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.733 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:45.733 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.733 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:45.733 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.734 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.734 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.734 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.734 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.734 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.734 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.734 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.734 INFO:teuthology.orchestra.run.smithi005.stdout: "storage_class": { 2024-05-09T01:01:45.734 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "storage_class", 2024-05-09T01:01:45.734 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.734 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.735 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.735 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "local", 2024-05-09T01:01:45.735 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.735 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.735 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.735 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-09T01:01:45.735 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.735 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.735 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.735 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.735 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.735 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.736 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:45.736 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "selftest", 2024-05-09T01:01:45.736 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:45.736 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:45.736 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:45.736 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:45.736 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:45.736 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.736 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.736 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.737 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.737 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.737 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.737 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.737 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.737 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.737 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.737 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.737 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.737 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.738 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.738 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.738 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.738 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.738 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.738 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.738 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:45.738 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:45.738 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.738 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.738 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.738 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.739 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.739 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.739 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.739 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.739 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.739 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.739 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.739 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.739 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:45.739 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:45.740 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.740 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.740 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.740 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:45.740 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.740 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.740 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.740 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.740 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.740 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.740 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.740 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.741 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.741 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.741 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.741 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.741 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.741 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.741 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.741 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:45.741 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:45.741 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.741 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.741 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.742 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.742 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.742 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.742 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.742 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.742 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.742 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.742 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.742 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.742 INFO:teuthology.orchestra.run.smithi005.stdout: "roption1": { 2024-05-09T01:01:45.742 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "roption1", 2024-05-09T01:01:45.743 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.743 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.743 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.743 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.743 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.743 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.743 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.743 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.743 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.743 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.743 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.743 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.743 INFO:teuthology.orchestra.run.smithi005.stdout: "roption2": { 2024-05-09T01:01:45.743 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "roption2", 2024-05-09T01:01:45.744 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.744 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.744 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.744 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "xyz", 2024-05-09T01:01:45.744 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.744 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.744 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.744 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.744 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.744 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.745 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.745 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.745 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption1": { 2024-05-09T01:01:45.745 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption1", 2024-05-09T01:01:45.745 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.745 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.745 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.745 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.745 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.745 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.746 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.746 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.746 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.746 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.746 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.746 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.746 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption2": { 2024-05-09T01:01:45.746 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption2", 2024-05-09T01:01:45.746 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.747 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.747 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.747 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.747 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.747 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.747 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.748 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.748 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.748 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.748 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.748 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.748 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption3": { 2024-05-09T01:01:45.748 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption3", 2024-05-09T01:01:45.748 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-05-09T01:01:45.748 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.749 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.749 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.749 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.749 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.749 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.749 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.749 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.749 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.749 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.749 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.750 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption4": { 2024-05-09T01:01:45.750 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption4", 2024-05-09T01:01:45.750 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.750 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.750 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.750 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.750 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.751 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.751 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.751 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.751 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.751 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.751 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.751 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.751 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption5": { 2024-05-09T01:01:45.751 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption5", 2024-05-09T01:01:45.752 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.752 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.752 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.752 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.752 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.752 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.752 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.752 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.752 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.752 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.752 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.753 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.753 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption6": { 2024-05-09T01:01:45.753 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption6", 2024-05-09T01:01:45.753 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.753 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.753 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.753 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:45.753 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.753 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.753 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.753 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.753 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.754 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.754 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.754 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.754 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption7": { 2024-05-09T01:01:45.754 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption7", 2024-05-09T01:01:45.754 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.754 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.754 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.754 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.754 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-05-09T01:01:45.755 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "42", 2024-05-09T01:01:45.755 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.755 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.755 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.755 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.755 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.755 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.755 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:45.755 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:45.755 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.756 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:45.756 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.756 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:45.756 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.756 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.756 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.757 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.757 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.757 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.757 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.757 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.757 INFO:teuthology.orchestra.run.smithi005.stdout: "testkey": { 2024-05-09T01:01:45.757 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testkey", 2024-05-09T01:01:45.757 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.758 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.758 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.758 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.758 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.758 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.758 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.759 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.759 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.759 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.759 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.759 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.760 INFO:teuthology.orchestra.run.smithi005.stdout: "testlkey": { 2024-05-09T01:01:45.760 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testlkey", 2024-05-09T01:01:45.760 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.760 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.760 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.760 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.760 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.761 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.761 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.761 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.761 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.761 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.761 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.761 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.761 INFO:teuthology.orchestra.run.smithi005.stdout: "testnewline": { 2024-05-09T01:01:45.761 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testnewline", 2024-05-09T01:01:45.761 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.762 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.762 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.762 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.762 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.762 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.762 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.762 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.762 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.762 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.762 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.762 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.763 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.763 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.763 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:45.763 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smb", 2024-05-09T01:01:45.763 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:45.763 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:45.763 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:45.764 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:45.764 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:45.764 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.764 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.764 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.764 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.764 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.764 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.764 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.764 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.765 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.765 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.765 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.765 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.765 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.765 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.765 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.765 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.765 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.765 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.765 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.766 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:45.766 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:45.766 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.766 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.766 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.766 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.766 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.766 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.766 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.767 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.767 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.767 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.767 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.767 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.767 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:45.767 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:45.767 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.767 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.767 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.767 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:45.767 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.768 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.768 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.768 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.768 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.768 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.768 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.768 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.768 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.768 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.769 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.769 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.769 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.769 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.769 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.769 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:45.769 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:45.769 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.769 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.769 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.770 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.770 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.770 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.770 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.770 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.770 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.770 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.770 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.770 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.770 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:45.770 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:45.771 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.771 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:45.771 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.771 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:45.771 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.771 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.771 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.771 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.771 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.771 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.771 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.771 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.772 INFO:teuthology.orchestra.run.smithi005.stdout: "update_orchestration": { 2024-05-09T01:01:45.772 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "update_orchestration", 2024-05-09T01:01:45.772 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.772 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.772 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.772 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:45.772 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.772 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.772 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.772 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-09T01:01:45.773 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.773 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.773 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.773 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.773 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.773 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.773 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:45.773 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snap_schedule", 2024-05-09T01:01:45.773 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:45.773 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:45.774 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:45.774 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_m_granularity": { 2024-05-09T01:01:45.774 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_m_granularity", 2024-05-09T01:01:45.774 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.774 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.774 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.774 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.774 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.774 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.774 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.774 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow minute scheduled snapshots", 2024-05-09T01:01:45.774 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.775 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.775 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.775 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.775 INFO:teuthology.orchestra.run.smithi005.stdout: "dump_on_update": { 2024-05-09T01:01:45.775 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "dump_on_update", 2024-05-09T01:01:45.775 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.775 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.775 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.775 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.776 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.776 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.776 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.777 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "dump database to debug log on update", 2024-05-09T01:01:45.777 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.777 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.777 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.777 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.778 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:45.778 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:45.778 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.778 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.778 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.778 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.778 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.778 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.778 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.779 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.779 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.779 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.779 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.779 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.779 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.779 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.779 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.779 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.779 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.779 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.779 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.780 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:45.780 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:45.780 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.780 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.780 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.780 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.780 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.780 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.781 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.781 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.781 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.781 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.781 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.781 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.781 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:45.781 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:45.781 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.781 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.781 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.782 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:45.782 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.782 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.782 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.782 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.782 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.782 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.782 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.782 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.782 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.782 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.783 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.783 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.783 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.783 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.783 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.783 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:45.783 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:45.783 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.783 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.783 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.784 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.784 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.784 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.784 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.784 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.784 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.784 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.784 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.784 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.784 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:45.784 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:45.785 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.785 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:45.785 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.785 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:45.785 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.785 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.785 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.785 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.785 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.785 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.786 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.786 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.786 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.786 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.786 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:45.786 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "stats", 2024-05-09T01:01:45.786 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:45.786 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:45.786 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:45.786 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:45.787 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:45.787 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.787 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.787 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.787 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.787 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.787 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.787 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.787 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.787 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.788 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.788 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.788 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.788 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.788 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.788 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.788 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.788 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.788 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.789 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.789 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:45.789 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:45.789 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.789 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.789 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.789 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.789 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.789 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.789 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.790 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.790 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.790 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.790 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.790 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.790 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:45.790 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:45.790 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.790 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.790 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.791 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:45.791 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.791 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.791 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.791 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.791 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.791 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.791 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.791 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.791 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.792 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.792 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.792 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.792 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.792 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.792 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.792 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:45.792 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:45.792 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.792 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.792 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.792 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.793 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.793 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.793 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.793 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.793 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.793 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.793 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.793 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.793 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:45.793 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:45.794 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.794 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:45.794 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.794 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:45.794 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.794 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.794 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.794 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.794 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.794 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.795 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.795 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.795 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.795 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.795 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:45.795 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "status", 2024-05-09T01:01:45.795 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:45.795 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:45.795 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:45.795 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:45.795 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:45.796 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.796 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.796 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.796 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.796 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.796 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.796 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.796 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.796 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.796 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.796 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.796 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.796 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.796 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.797 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.797 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.797 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.797 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.797 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.797 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:45.797 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:45.797 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.797 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.797 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.797 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.797 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.797 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.797 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.798 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.798 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.798 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.798 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.798 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.798 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:45.798 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:45.798 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.798 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.798 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.798 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:45.798 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.798 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.799 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.799 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.799 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.799 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.799 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.799 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.799 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.799 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.799 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.799 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.799 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.799 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.799 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.799 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:45.800 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:45.800 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.800 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.800 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.800 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.800 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.800 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.800 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.800 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.800 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.800 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.800 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.801 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.801 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:45.801 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:45.801 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.801 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:45.801 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.801 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:45.801 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.801 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.801 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.802 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.802 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.802 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.802 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.802 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.802 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.802 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.802 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:45.802 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "telegraf", 2024-05-09T01:01:45.802 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:45.802 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:45.803 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:45.803 INFO:teuthology.orchestra.run.smithi005.stdout: "address": { 2024-05-09T01:01:45.803 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "address", 2024-05-09T01:01:45.803 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.803 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.803 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.803 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-09T01:01:45.803 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.803 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.803 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.803 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.804 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.804 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.804 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.804 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.804 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-05-09T01:01:45.804 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-05-09T01:01:45.804 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-09T01:01:45.804 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.804 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.804 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15", 2024-05-09T01:01:45.804 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.804 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.804 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.805 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.805 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.805 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.805 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.805 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.805 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:45.805 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:45.805 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.805 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.806 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.806 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.806 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.806 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.806 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.806 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.806 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.806 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.806 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.806 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.807 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.807 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.807 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.807 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.807 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.807 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.807 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.807 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:45.807 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:45.807 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.807 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.808 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.808 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.808 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.808 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.808 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.808 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.808 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.808 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.808 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.808 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.809 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:45.809 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:45.809 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.809 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.809 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.809 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:45.809 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.809 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.809 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.809 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.810 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.810 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.810 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.810 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.810 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.810 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.810 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.810 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.810 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.810 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.810 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.811 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:45.811 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:45.811 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.811 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.811 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.811 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.811 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.811 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.811 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.811 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.813 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.813 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.813 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.813 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.813 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:45.814 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:45.814 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.814 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:45.814 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.814 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:45.814 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.814 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.814 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.814 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.815 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.815 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.815 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.815 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.815 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.815 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.815 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:45.815 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "telemetry", 2024-05-09T01:01:45.816 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:45.816 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:45.816 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:45.816 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_basic": { 2024-05-09T01:01:45.816 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_basic", 2024-05-09T01:01:45.816 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.816 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.816 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.816 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:45.816 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.816 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.817 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.817 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-09T01:01:45.817 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.817 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.817 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.817 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.817 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_crash": { 2024-05-09T01:01:45.817 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_crash", 2024-05-09T01:01:45.817 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.817 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.817 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.817 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:45.817 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.818 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.818 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.818 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-09T01:01:45.818 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.818 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.818 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.818 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.818 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_device": { 2024-05-09T01:01:45.818 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_device", 2024-05-09T01:01:45.818 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.818 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.819 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.819 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:45.819 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.819 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.819 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.819 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-09T01:01:45.819 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.819 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.819 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.819 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.819 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_ident": { 2024-05-09T01:01:45.819 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_ident", 2024-05-09T01:01:45.819 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.820 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.820 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.820 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.820 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.820 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.820 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.820 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-09T01:01:45.820 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.820 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.820 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.820 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.820 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_perf": { 2024-05-09T01:01:45.821 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_perf", 2024-05-09T01:01:45.821 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.821 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.821 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.821 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.821 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.821 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.821 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.822 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-09T01:01:45.822 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.822 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.822 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.822 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.822 INFO:teuthology.orchestra.run.smithi005.stdout: "contact": { 2024-05-09T01:01:45.822 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "contact", 2024-05-09T01:01:45.822 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.822 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.822 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.822 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.822 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.823 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.823 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.823 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.823 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.823 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.823 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.823 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.823 INFO:teuthology.orchestra.run.smithi005.stdout: "description": { 2024-05-09T01:01:45.823 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "description", 2024-05-09T01:01:45.823 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.823 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.823 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.824 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.824 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.824 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.824 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.824 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.824 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.824 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.824 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.824 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.825 INFO:teuthology.orchestra.run.smithi005.stdout: "device_url": { 2024-05-09T01:01:45.825 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_url", 2024-05-09T01:01:45.825 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.825 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.826 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.826 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-09T01:01:45.826 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.826 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.826 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.826 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.826 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.826 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.826 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.826 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.827 INFO:teuthology.orchestra.run.smithi005.stdout: "enabled": { 2024-05-09T01:01:45.827 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enabled", 2024-05-09T01:01:45.827 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.827 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.827 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.827 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.827 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.827 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.827 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.827 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.827 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.827 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.827 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.828 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.828 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-05-09T01:01:45.828 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-05-09T01:01:45.828 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.828 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.828 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.828 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "24", 2024-05-09T01:01:45.828 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "8", 2024-05-09T01:01:45.828 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.829 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.829 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.829 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.829 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.829 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.829 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.829 INFO:teuthology.orchestra.run.smithi005.stdout: "last_opt_revision": { 2024-05-09T01:01:45.829 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "last_opt_revision", 2024-05-09T01:01:45.829 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.829 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.830 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.830 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1", 2024-05-09T01:01:45.830 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.830 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.830 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.830 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.830 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.830 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.830 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.830 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.830 INFO:teuthology.orchestra.run.smithi005.stdout: "leaderboard": { 2024-05-09T01:01:45.830 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "leaderboard", 2024-05-09T01:01:45.831 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.831 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.831 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.831 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.831 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.831 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.831 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.831 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.831 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.831 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.831 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.831 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.832 INFO:teuthology.orchestra.run.smithi005.stdout: "leaderboard_description": { 2024-05-09T01:01:45.832 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "leaderboard_description", 2024-05-09T01:01:45.832 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.832 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.832 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.832 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.832 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.832 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.832 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.832 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.832 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.832 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.832 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.833 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.833 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:45.833 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:45.833 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.833 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.833 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.833 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.833 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.833 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.834 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.834 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.834 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.834 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.834 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.834 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.834 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.834 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.834 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.835 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.835 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.835 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.835 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.835 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:45.835 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:45.835 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.835 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.835 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.835 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.836 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.836 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.836 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.836 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.836 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.836 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.836 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.836 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.836 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:45.836 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:45.836 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.836 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.836 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.837 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:45.837 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.837 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.837 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.837 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.837 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.837 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.837 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.837 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.838 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.838 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.838 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.838 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.838 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.838 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.838 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.839 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:45.839 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:45.839 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.839 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.839 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.839 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.839 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.839 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.839 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.839 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.840 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.840 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.840 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.840 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.840 INFO:teuthology.orchestra.run.smithi005.stdout: "organization": { 2024-05-09T01:01:45.840 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "organization", 2024-05-09T01:01:45.840 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.840 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.840 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.840 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.840 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.841 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.841 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.841 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.841 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.841 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.841 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.841 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.841 INFO:teuthology.orchestra.run.smithi005.stdout: "proxy": { 2024-05-09T01:01:45.841 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "proxy", 2024-05-09T01:01:45.841 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.841 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.842 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.842 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.842 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.842 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.842 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.842 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.842 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.842 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.842 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.842 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.842 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:45.842 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:45.843 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.843 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:45.843 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.843 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:45.843 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.843 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.843 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.843 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.843 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.844 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.844 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.844 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.844 INFO:teuthology.orchestra.run.smithi005.stdout: "url": { 2024-05-09T01:01:45.844 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "url", 2024-05-09T01:01:45.844 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.844 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.844 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.844 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-09T01:01:45.845 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.845 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.845 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.845 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.845 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.845 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.845 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.845 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.845 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.845 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.845 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:45.845 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "test_orchestrator", 2024-05-09T01:01:45.846 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:45.846 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:45.846 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:45.846 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:45.846 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:45.846 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.846 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.846 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.846 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.846 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.847 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.847 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.847 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.847 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.847 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.847 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.847 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.847 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.847 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.847 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.848 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.848 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.848 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.848 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.848 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:45.848 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:45.848 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.848 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.848 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.848 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.848 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.849 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.849 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.849 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.849 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.849 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.849 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.849 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.849 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:45.849 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:45.849 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.849 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.850 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.850 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:45.850 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.850 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.850 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.850 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.850 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.850 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.850 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.850 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.850 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.851 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.851 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.851 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.851 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.851 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.851 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.852 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:45.852 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:45.852 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.852 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.853 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.853 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.853 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.853 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.853 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.854 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.854 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.854 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.854 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.854 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.854 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:45.854 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:45.854 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.854 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:45.854 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.854 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:45.855 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.855 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.855 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.855 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.855 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.855 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.855 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.855 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.855 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.855 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.855 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:45.855 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "volumes", 2024-05-09T01:01:45.856 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:45.856 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:45.856 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:45.856 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:45.856 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:45.856 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.856 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.856 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.856 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.856 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.856 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.856 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.856 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.857 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.857 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.857 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.857 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.857 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.857 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.857 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.857 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.857 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.857 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.857 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.857 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:45.858 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:45.858 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.858 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.858 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.858 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.858 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.858 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.858 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.858 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.858 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.859 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.859 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.859 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.859 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:45.859 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:45.859 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.859 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.859 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.859 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:45.859 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.859 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.859 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.860 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.860 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.860 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.860 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.860 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.860 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.860 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.860 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.860 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.860 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.860 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.860 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.860 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:45.860 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:45.861 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.861 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.861 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.861 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.861 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.861 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.861 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.861 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.861 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.861 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.861 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.862 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.862 INFO:teuthology.orchestra.run.smithi005.stdout: "max_concurrent_clones": { 2024-05-09T01:01:45.862 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_concurrent_clones", 2024-05-09T01:01:45.862 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.862 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.862 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.862 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "4", 2024-05-09T01:01:45.862 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.862 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.862 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.863 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-09T01:01:45.863 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.863 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.863 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.863 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.863 INFO:teuthology.orchestra.run.smithi005.stdout: "periodic_async_work": { 2024-05-09T01:01:45.863 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "periodic_async_work", 2024-05-09T01:01:45.863 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.863 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.863 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.863 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.864 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.864 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.864 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.864 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Periodically check for async work", 2024-05-09T01:01:45.864 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.864 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.864 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.864 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.864 INFO:teuthology.orchestra.run.smithi005.stdout: "snapshot_clone_delay": { 2024-05-09T01:01:45.864 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snapshot_clone_delay", 2024-05-09T01:01:45.865 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.865 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.865 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.865 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:45.865 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.865 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.865 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.865 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-09T01:01:45.865 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.865 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.865 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.865 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.866 INFO:teuthology.orchestra.run.smithi005.stdout: "snapshot_clone_no_wait": { 2024-05-09T01:01:45.866 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snapshot_clone_no_wait", 2024-05-09T01:01:45.866 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.866 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.866 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.866 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:45.866 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.866 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.867 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.867 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-09T01:01:45.867 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.867 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.867 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.867 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.867 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:45.867 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:45.867 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.867 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:45.867 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.867 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:45.868 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.868 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.868 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.868 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.868 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.868 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.868 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.868 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.868 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.868 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.868 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:45.869 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix", 2024-05-09T01:01:45.869 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:45.869 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:45.869 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:45.869 INFO:teuthology.orchestra.run.smithi005.stdout: "discovery_interval": { 2024-05-09T01:01:45.869 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "discovery_interval", 2024-05-09T01:01:45.869 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-05-09T01:01:45.869 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.869 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.869 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "100", 2024-05-09T01:01:45.870 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.870 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.870 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.870 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.870 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.870 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.870 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.870 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.871 INFO:teuthology.orchestra.run.smithi005.stdout: "identifier": { 2024-05-09T01:01:45.871 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "identifier", 2024-05-09T01:01:45.871 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.871 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.871 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.871 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.871 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.871 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.871 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.872 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.872 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.872 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.872 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.872 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.872 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-05-09T01:01:45.872 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-05-09T01:01:45.872 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-09T01:01:45.873 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.873 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.873 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-05-09T01:01:45.873 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.874 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.874 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.874 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.874 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.874 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.874 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.874 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.875 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:45.875 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:45.875 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.875 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.875 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.875 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.875 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.875 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.875 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.875 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.875 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.876 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.876 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.876 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.876 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.876 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.876 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.876 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.876 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.876 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.876 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.876 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:45.877 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:45.877 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.877 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.877 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.877 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.877 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.877 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.877 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.877 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.877 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.878 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.878 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.878 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.878 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:45.878 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:45.878 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.878 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.878 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.878 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:45.878 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.878 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.879 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.879 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.879 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.879 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.879 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.879 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.879 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.879 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.879 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.879 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.879 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.880 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.880 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.880 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:45.880 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:45.880 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.880 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.880 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.880 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.880 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.880 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.880 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.880 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.881 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.881 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.881 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.881 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.881 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:45.881 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:45.881 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.881 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:45.881 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.881 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:45.882 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.882 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.882 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.882 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.882 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.882 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.882 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.882 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.882 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_host": { 2024-05-09T01:01:45.882 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_host", 2024-05-09T01:01:45.882 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.882 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.883 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.883 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.883 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.883 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.883 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.883 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.883 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.883 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.883 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.884 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.884 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_port": { 2024-05-09T01:01:45.884 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_port", 2024-05-09T01:01:45.884 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.884 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.884 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.884 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10051", 2024-05-09T01:01:45.884 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.884 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.885 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.885 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.885 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.885 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.885 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.885 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.885 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_sender": { 2024-05-09T01:01:45.885 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_sender", 2024-05-09T01:01:45.885 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.885 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.885 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.885 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-09T01:01:45.886 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.886 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.886 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.886 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.886 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.886 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.886 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.886 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.886 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.886 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.887 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-09T01:01:45.887 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.887 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:45.887 INFO:teuthology.orchestra.run.smithi005.stdout: "gid": 24113, 2024-05-09T01:01:45.887 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "y", 2024-05-09T01:01:45.887 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_features": 4540701547738038271, 2024-05-09T01:01:45.887 INFO:teuthology.orchestra.run.smithi005.stdout: "available_modules": [ 2024-05-09T01:01:45.887 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:45.887 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "alerts", 2024-05-09T01:01:45.887 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:45.887 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:45.887 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:45.888 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-05-09T01:01:45.888 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-05-09T01:01:45.888 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-09T01:01:45.888 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.888 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.888 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-05-09T01:01:45.888 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.888 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.888 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.888 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How frequently to reexamine health status", 2024-05-09T01:01:45.888 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.888 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.889 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.889 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.889 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:45.889 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:45.889 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.889 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.889 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.889 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.889 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.889 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.889 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.889 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.890 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.890 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.890 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.890 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.890 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.890 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.890 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.890 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.890 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.890 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.890 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.891 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:45.891 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:45.891 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.891 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.891 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.891 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.891 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.891 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.891 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.891 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.891 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.892 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.892 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.892 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.892 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:45.892 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:45.892 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.892 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.892 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.892 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:45.892 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.892 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.892 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.893 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.893 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.893 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.893 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.893 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.893 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.893 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.893 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.893 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.893 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.893 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.894 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.894 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:45.894 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:45.894 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.894 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.894 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.894 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.894 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.895 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.895 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.895 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.895 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.895 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.895 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.895 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.896 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_destination": { 2024-05-09T01:01:45.896 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_destination", 2024-05-09T01:01:45.896 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.896 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.896 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.896 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.896 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.896 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.896 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.897 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Email address to send alerts to", 2024-05-09T01:01:45.897 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.897 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.897 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.897 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.898 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_from_name": { 2024-05-09T01:01:45.898 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_from_name", 2024-05-09T01:01:45.898 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.898 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.898 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.898 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "Ceph", 2024-05-09T01:01:45.898 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.898 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.898 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.898 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Email From: name", 2024-05-09T01:01:45.899 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.899 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.899 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.899 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.899 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_host": { 2024-05-09T01:01:45.899 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_host", 2024-05-09T01:01:45.899 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.899 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.899 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.899 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.900 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.900 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.900 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.900 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP server", 2024-05-09T01:01:45.900 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.900 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.900 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.900 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.900 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_password": { 2024-05-09T01:01:45.900 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_password", 2024-05-09T01:01:45.900 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.901 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.901 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.901 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.901 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.901 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.901 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.901 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Password to authenticate with", 2024-05-09T01:01:45.901 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.902 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.902 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.902 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.902 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_port": { 2024-05-09T01:01:45.902 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_port", 2024-05-09T01:01:45.902 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.902 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.902 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.902 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "465", 2024-05-09T01:01:45.902 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.903 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.903 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.903 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP port", 2024-05-09T01:01:45.903 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.903 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.903 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.903 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.903 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_sender": { 2024-05-09T01:01:45.903 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_sender", 2024-05-09T01:01:45.903 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.903 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.904 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.904 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.904 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.904 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.904 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.904 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP envelope sender", 2024-05-09T01:01:45.904 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.904 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.904 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.904 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.904 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_ssl": { 2024-05-09T01:01:45.905 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_ssl", 2024-05-09T01:01:45.905 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.905 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.905 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.905 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:45.905 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.905 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.905 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.905 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-09T01:01:45.905 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.905 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.906 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.906 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.906 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_user": { 2024-05-09T01:01:45.906 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_user", 2024-05-09T01:01:45.906 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.906 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.906 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.906 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.906 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.906 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.906 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.907 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "User to authenticate as", 2024-05-09T01:01:45.907 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.907 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.907 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.907 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.907 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:45.907 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:45.907 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.907 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:45.907 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.908 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:45.908 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.908 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.908 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.908 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.908 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.908 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.908 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.908 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.908 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.909 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.909 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:45.909 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "balancer", 2024-05-09T01:01:45.909 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:45.909 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:45.909 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:45.909 INFO:teuthology.orchestra.run.smithi005.stdout: "active": { 2024-05-09T01:01:45.909 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "active", 2024-05-09T01:01:45.909 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.909 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.909 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.910 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:45.910 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.910 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.910 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.910 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically balance PGs across cluster", 2024-05-09T01:01:45.910 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.910 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.910 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.910 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.910 INFO:teuthology.orchestra.run.smithi005.stdout: "begin_time": { 2024-05-09T01:01:45.910 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "begin_time", 2024-05-09T01:01:45.910 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.910 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.911 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.911 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0000", 2024-05-09T01:01:45.911 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.911 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.911 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.911 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "beginning time of day to automatically balance", 2024-05-09T01:01:45.911 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T01:01:45.911 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.911 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.911 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.911 INFO:teuthology.orchestra.run.smithi005.stdout: "begin_weekday": { 2024-05-09T01:01:45.912 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "begin_weekday", 2024-05-09T01:01:45.912 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-05-09T01:01:45.912 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.912 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.912 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:45.912 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0", 2024-05-09T01:01:45.912 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "6", 2024-05-09T01:01:45.913 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.913 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-09T01:01:45.913 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T01:01:45.913 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.913 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.913 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.914 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_max_iterations": { 2024-05-09T01:01:45.914 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_max_iterations", 2024-05-09T01:01:45.914 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-05-09T01:01:45.914 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.914 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.915 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "25", 2024-05-09T01:01:45.915 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-05-09T01:01:45.915 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "250", 2024-05-09T01:01:45.915 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.916 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-09T01:01:45.916 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.916 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.916 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.916 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.916 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_metrics": { 2024-05-09T01:01:45.916 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_metrics", 2024-05-09T01:01:45.916 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.916 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.917 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.917 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "pgs,objects,bytes", 2024-05-09T01:01:45.917 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.917 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.917 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.917 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-09T01:01:45.917 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-09T01:01:45.917 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.917 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.917 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.917 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_step": { 2024-05-09T01:01:45.918 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_step", 2024-05-09T01:01:45.918 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-05-09T01:01:45.918 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.918 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.918 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0.5", 2024-05-09T01:01:45.918 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0.001", 2024-05-09T01:01:45.918 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "0.999", 2024-05-09T01:01:45.918 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.918 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "aggressiveness of optimization", 2024-05-09T01:01:45.918 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-09T01:01:45.918 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.919 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.919 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.919 INFO:teuthology.orchestra.run.smithi005.stdout: "end_time": { 2024-05-09T01:01:45.919 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "end_time", 2024-05-09T01:01:45.919 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.919 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.919 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.920 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "2359", 2024-05-09T01:01:45.920 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.920 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.920 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.920 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "ending time of day to automatically balance", 2024-05-09T01:01:45.920 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T01:01:45.920 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.920 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.920 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.920 INFO:teuthology.orchestra.run.smithi005.stdout: "end_weekday": { 2024-05-09T01:01:45.920 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "end_weekday", 2024-05-09T01:01:45.921 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-05-09T01:01:45.921 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.921 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.921 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:45.921 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0", 2024-05-09T01:01:45.921 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "6", 2024-05-09T01:01:45.921 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.921 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-09T01:01:45.921 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T01:01:45.921 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.921 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.922 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.922 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:45.922 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:45.922 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.922 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.922 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.922 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.922 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.922 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.923 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.923 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.923 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.923 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.923 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.923 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.923 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.923 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.923 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.923 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.923 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.923 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.924 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.924 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:45.924 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:45.924 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.924 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.924 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.924 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.924 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.924 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.924 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.924 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.924 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.925 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.925 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.925 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.925 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:45.925 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:45.925 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.925 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.925 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.925 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:45.925 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.926 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.926 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.926 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.926 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.926 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.926 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.926 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.926 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.926 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.926 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.927 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.927 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.927 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.927 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.927 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:45.927 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:45.927 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.927 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.927 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.927 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.927 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.927 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.928 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.928 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.928 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.928 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.928 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.928 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.928 INFO:teuthology.orchestra.run.smithi005.stdout: "min_score": { 2024-05-09T01:01:45.928 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "min_score", 2024-05-09T01:01:45.928 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-05-09T01:01:45.928 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.928 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.928 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:45.929 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.929 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.929 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.929 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-09T01:01:45.929 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.929 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.929 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.929 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.929 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": { 2024-05-09T01:01:45.929 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mode", 2024-05-09T01:01:45.929 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.930 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.930 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.930 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "upmap", 2024-05-09T01:01:45.930 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.930 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.930 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.930 INFO:teuthology.orchestra.run.smithi005.stdout: "crush-compat", 2024-05-09T01:01:45.930 INFO:teuthology.orchestra.run.smithi005.stdout: "none", 2024-05-09T01:01:45.930 INFO:teuthology.orchestra.run.smithi005.stdout: "read", 2024-05-09T01:01:45.930 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap", 2024-05-09T01:01:45.930 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap-read" 2024-05-09T01:01:45.931 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.931 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Balancer mode", 2024-05-09T01:01:45.931 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.931 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.931 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.931 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.931 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_ids": { 2024-05-09T01:01:45.931 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pool_ids", 2024-05-09T01:01:45.931 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.931 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.931 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.931 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.931 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.932 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.932 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.932 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-09T01:01:45.932 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.932 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.932 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.932 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.932 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-05-09T01:01:45.932 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-05-09T01:01:45.932 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-09T01:01:45.932 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.932 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.933 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-05-09T01:01:45.933 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.933 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.933 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.933 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-09T01:01:45.933 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.933 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.933 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.933 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.933 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:45.933 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:45.933 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.933 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:45.934 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.934 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:45.934 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.934 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.934 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.934 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.934 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.934 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.934 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.934 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.935 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap_max_deviation": { 2024-05-09T01:01:45.935 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "upmap_max_deviation", 2024-05-09T01:01:45.935 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.935 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.935 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.935 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-05-09T01:01:45.935 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-05-09T01:01:45.935 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.935 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.935 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-09T01:01:45.935 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-09T01:01:45.936 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.936 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.936 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.936 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap_max_optimizations": { 2024-05-09T01:01:45.936 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "upmap_max_optimizations", 2024-05-09T01:01:45.936 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-05-09T01:01:45.936 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.936 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.936 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-05-09T01:01:45.936 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.937 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.937 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.937 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-09T01:01:45.937 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.937 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.937 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.937 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.937 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:45.937 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.937 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:45.937 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cephadm", 2024-05-09T01:01:45.937 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:45.937 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:45.938 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:45.938 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_down_multiplier": { 2024-05-09T01:01:45.938 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_down_multiplier", 2024-05-09T01:01:45.938 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-05-09T01:01:45.938 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.938 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.938 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3.0", 2024-05-09T01:01:45.938 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.938 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.939 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.939 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-09T01:01:45.939 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.939 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.939 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.939 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.939 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_refresh_rate": { 2024-05-09T01:01:45.939 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_refresh_rate", 2024-05-09T01:01:45.939 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-09T01:01:45.939 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.940 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.940 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "20", 2024-05-09T01:01:45.940 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.940 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.940 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.940 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-09T01:01:45.940 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.940 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.940 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.940 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.940 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_starting_port": { 2024-05-09T01:01:45.941 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_starting_port", 2024-05-09T01:01:45.941 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.941 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.941 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.941 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "4721", 2024-05-09T01:01:45.941 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.941 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.941 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.941 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-09T01:01:45.941 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.941 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.941 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.942 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.942 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_ptrace": { 2024-05-09T01:01:45.942 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_ptrace", 2024-05-09T01:01:45.942 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.942 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.942 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.942 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.942 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.942 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.942 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.942 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-09T01:01:45.942 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-09T01:01:45.943 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.943 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.943 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.943 INFO:teuthology.orchestra.run.smithi005.stdout: "autotune_interval": { 2024-05-09T01:01:45.943 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "autotune_interval", 2024-05-09T01:01:45.943 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-09T01:01:45.943 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.943 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.943 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-05-09T01:01:45.943 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.943 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.944 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.944 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-09T01:01:45.944 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.944 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.944 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.944 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.944 INFO:teuthology.orchestra.run.smithi005.stdout: "autotune_memory_target_ratio": { 2024-05-09T01:01:45.944 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "autotune_memory_target_ratio", 2024-05-09T01:01:45.944 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-05-09T01:01:45.944 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.944 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.944 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0.7", 2024-05-09T01:01:45.945 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.945 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.945 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.945 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-09T01:01:45.945 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.945 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.945 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.945 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.945 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm_log_destination": { 2024-05-09T01:01:45.945 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cephadm_log_destination", 2024-05-09T01:01:45.945 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.945 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.946 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.946 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.946 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.946 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.946 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.946 INFO:teuthology.orchestra.run.smithi005.stdout: "file", 2024-05-09T01:01:45.946 INFO:teuthology.orchestra.run.smithi005.stdout: "file,syslog", 2024-05-09T01:01:45.946 INFO:teuthology.orchestra.run.smithi005.stdout: "syslog" 2024-05-09T01:01:45.946 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.946 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-09T01:01:45.947 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.947 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.947 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.947 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.947 INFO:teuthology.orchestra.run.smithi005.stdout: "cgroups_split": { 2024-05-09T01:01:45.947 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cgroups_split", 2024-05-09T01:01:45.947 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.947 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.947 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.947 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:45.947 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.947 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.948 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.948 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-09T01:01:45.948 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.948 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.948 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.948 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.948 INFO:teuthology.orchestra.run.smithi005.stdout: "config_checks_enabled": { 2024-05-09T01:01:45.948 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_checks_enabled", 2024-05-09T01:01:45.948 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.948 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.948 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.948 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.948 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.949 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.949 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.949 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-09T01:01:45.949 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.949 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.949 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.949 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.949 INFO:teuthology.orchestra.run.smithi005.stdout: "config_dashboard": { 2024-05-09T01:01:45.949 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_dashboard", 2024-05-09T01:01:45.950 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.950 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.950 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.950 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:45.950 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.950 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.950 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.950 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-09T01:01:45.950 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.950 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.951 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.951 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.951 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_alertmanager": { 2024-05-09T01:01:45.951 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_alertmanager", 2024-05-09T01:01:45.951 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.951 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.951 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.951 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-09T01:01:45.951 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.951 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.952 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.952 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-05-09T01:01:45.952 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.952 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.952 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.952 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.952 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_base": { 2024-05-09T01:01:45.952 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_base", 2024-05-09T01:01:45.953 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.953 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.953 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.953 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-09T01:01:45.953 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.953 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.953 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.953 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Container image name, without the tag", 2024-05-09T01:01:45.953 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.953 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.953 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.954 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.954 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_elasticsearch": { 2024-05-09T01:01:45.954 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_elasticsearch", 2024-05-09T01:01:45.954 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.954 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.954 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.954 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-09T01:01:45.954 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.954 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.954 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.955 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "elasticsearch container image", 2024-05-09T01:01:45.955 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.955 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.955 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.955 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.955 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_grafana": { 2024-05-09T01:01:45.955 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_grafana", 2024-05-09T01:01:45.955 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.955 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.955 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.955 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-09T01:01:45.955 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.956 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.956 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.956 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-05-09T01:01:45.956 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.956 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.956 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.956 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.956 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_haproxy": { 2024-05-09T01:01:45.956 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_haproxy", 2024-05-09T01:01:45.957 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.957 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.957 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.957 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-09T01:01:45.957 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.957 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.957 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.957 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "HAproxy container image", 2024-05-09T01:01:45.957 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.957 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.957 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.958 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.958 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_agent": { 2024-05-09T01:01:45.958 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_agent", 2024-05-09T01:01:45.958 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.958 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.958 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.958 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-09T01:01:45.958 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.958 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.959 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.959 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger agent container image", 2024-05-09T01:01:45.959 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.959 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.959 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.959 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.959 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_collector": { 2024-05-09T01:01:45.960 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_collector", 2024-05-09T01:01:45.960 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.960 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.960 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.960 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-09T01:01:45.960 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.960 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.960 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.960 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger collector container image", 2024-05-09T01:01:45.960 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.960 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.960 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.961 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.961 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_query": { 2024-05-09T01:01:45.961 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_query", 2024-05-09T01:01:45.961 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.961 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.961 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.961 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-09T01:01:45.961 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.961 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.961 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.962 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger query container image", 2024-05-09T01:01:45.962 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.962 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.962 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.963 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.963 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_keepalived": { 2024-05-09T01:01:45.963 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_keepalived", 2024-05-09T01:01:45.963 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.963 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.963 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.963 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-09T01:01:45.963 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.964 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.964 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.964 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Keepalived container image", 2024-05-09T01:01:45.964 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.964 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.964 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.964 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.964 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_loki": { 2024-05-09T01:01:45.964 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_loki", 2024-05-09T01:01:45.964 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.964 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.964 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.965 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-09T01:01:45.965 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.965 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.965 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.965 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Loki container image", 2024-05-09T01:01:45.965 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.965 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.965 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.965 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.965 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_node_exporter": { 2024-05-09T01:01:45.965 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_node_exporter", 2024-05-09T01:01:45.965 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.966 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.966 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.966 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-09T01:01:45.966 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.966 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.966 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.966 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-05-09T01:01:45.966 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.966 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.966 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.966 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.966 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_nvmeof": { 2024-05-09T01:01:45.967 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_nvmeof", 2024-05-09T01:01:45.967 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.967 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.967 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.967 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-09T01:01:45.967 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.967 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.967 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.967 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Nvme-of container image", 2024-05-09T01:01:45.968 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.968 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.968 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.968 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.968 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_prometheus": { 2024-05-09T01:01:45.968 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_prometheus", 2024-05-09T01:01:45.968 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.968 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.968 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.968 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-09T01:01:45.968 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.968 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.969 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.969 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-05-09T01:01:45.969 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.969 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.969 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.969 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.969 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_promtail": { 2024-05-09T01:01:45.969 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_promtail", 2024-05-09T01:01:45.969 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.969 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.970 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.970 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-09T01:01:45.970 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.970 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.970 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.970 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Promtail container image", 2024-05-09T01:01:45.970 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.970 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.971 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.971 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.971 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_samba": { 2024-05-09T01:01:45.971 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_samba", 2024-05-09T01:01:45.971 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.971 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.971 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.971 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-09T01:01:45.972 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.972 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.972 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.972 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Samba/SMB container image", 2024-05-09T01:01:45.972 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.972 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.972 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.972 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.972 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_snmp_gateway": { 2024-05-09T01:01:45.972 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_snmp_gateway", 2024-05-09T01:01:45.972 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.972 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.973 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.973 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-09T01:01:45.973 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.973 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.973 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.973 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SNMP Gateway container image", 2024-05-09T01:01:45.973 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.973 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.973 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.973 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.973 INFO:teuthology.orchestra.run.smithi005.stdout: "container_init": { 2024-05-09T01:01:45.973 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_init", 2024-05-09T01:01:45.974 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.974 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.974 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.974 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:45.974 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.974 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.974 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.974 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Run podman/docker with `--init`", 2024-05-09T01:01:45.974 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.975 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.975 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.975 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.975 INFO:teuthology.orchestra.run.smithi005.stdout: "daemon_cache_timeout": { 2024-05-09T01:01:45.975 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "daemon_cache_timeout", 2024-05-09T01:01:45.975 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-09T01:01:45.975 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.975 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.975 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-05-09T01:01:45.975 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.975 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.975 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.976 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-09T01:01:45.976 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.976 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.976 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.976 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.976 INFO:teuthology.orchestra.run.smithi005.stdout: "default_cephadm_command_timeout": { 2024-05-09T01:01:45.976 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "default_cephadm_command_timeout", 2024-05-09T01:01:45.976 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.976 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.976 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.976 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "900", 2024-05-09T01:01:45.977 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.977 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.977 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.977 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-09T01:01:45.977 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.977 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.977 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.977 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.977 INFO:teuthology.orchestra.run.smithi005.stdout: "default_registry": { 2024-05-09T01:01:45.977 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "default_registry", 2024-05-09T01:01:45.977 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.978 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.978 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.978 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io", 2024-05-09T01:01:45.978 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.978 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.978 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.978 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-09T01:01:45.978 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.978 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.978 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.978 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.978 INFO:teuthology.orchestra.run.smithi005.stdout: "device_cache_timeout": { 2024-05-09T01:01:45.979 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_cache_timeout", 2024-05-09T01:01:45.979 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-09T01:01:45.979 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.979 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.979 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1800", 2024-05-09T01:01:45.979 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.979 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.979 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.979 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache device inventory", 2024-05-09T01:01:45.979 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.979 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.979 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.980 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.980 INFO:teuthology.orchestra.run.smithi005.stdout: "device_enhanced_scan": { 2024-05-09T01:01:45.980 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_enhanced_scan", 2024-05-09T01:01:45.980 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.980 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.980 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.980 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.980 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.980 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.980 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.980 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-09T01:01:45.980 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.981 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.981 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.981 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.981 INFO:teuthology.orchestra.run.smithi005.stdout: "facts_cache_timeout": { 2024-05-09T01:01:45.981 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "facts_cache_timeout", 2024-05-09T01:01:45.981 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-09T01:01:45.981 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.981 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.981 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-05-09T01:01:45.981 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.981 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.981 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.982 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache host facts data", 2024-05-09T01:01:45.982 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.982 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.982 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.982 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.982 INFO:teuthology.orchestra.run.smithi005.stdout: "grafana_dashboards_path": { 2024-05-09T01:01:45.982 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "grafana_dashboards_path", 2024-05-09T01:01:45.983 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.983 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.983 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.983 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-09T01:01:45.983 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.984 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.984 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.984 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-09T01:01:45.984 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.984 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.985 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.985 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.985 INFO:teuthology.orchestra.run.smithi005.stdout: "host_check_interval": { 2024-05-09T01:01:45.985 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "host_check_interval", 2024-05-09T01:01:45.985 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-09T01:01:45.985 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.985 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.985 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-05-09T01:01:45.985 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.985 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.986 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.986 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to perform a host check", 2024-05-09T01:01:45.986 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.986 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.986 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.986 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.986 INFO:teuthology.orchestra.run.smithi005.stdout: "hw_monitoring": { 2024-05-09T01:01:45.986 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "hw_monitoring", 2024-05-09T01:01:45.986 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.986 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.986 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.986 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.987 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.987 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.987 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.987 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-09T01:01:45.987 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.987 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.987 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.987 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.987 INFO:teuthology.orchestra.run.smithi005.stdout: "inventory_list_all": { 2024-05-09T01:01:45.987 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "inventory_list_all", 2024-05-09T01:01:45.987 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.987 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.987 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.988 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.988 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.988 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.988 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.988 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-09T01:01:45.988 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.988 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.988 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.988 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.988 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:45.988 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:45.989 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.989 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.989 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.989 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:45.989 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.989 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.989 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.989 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.989 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.989 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.989 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.989 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.990 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.990 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.990 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.990 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.990 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.990 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.990 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.990 INFO:teuthology.orchestra.run.smithi005.stdout: "log_refresh_metadata": { 2024-05-09T01:01:45.990 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_refresh_metadata", 2024-05-09T01:01:45.991 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.991 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.991 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.991 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.991 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.991 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.991 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.991 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-09T01:01:45.991 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.991 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.991 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.991 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.992 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:45.992 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:45.992 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.992 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.992 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.992 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:45.992 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.992 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.992 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.992 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-09T01:01:45.992 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.992 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.993 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.993 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.993 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:45.993 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:45.993 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.993 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.993 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.993 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:45.993 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.993 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.993 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:45.993 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:45.993 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:45.994 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:45.994 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:45.994 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:45.994 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:45.994 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:45.994 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.994 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.994 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.994 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.994 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.995 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:45.995 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:45.995 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.995 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.995 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:45.995 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.995 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.995 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.995 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.995 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:45.995 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.995 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.996 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.996 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.996 INFO:teuthology.orchestra.run.smithi005.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-09T01:01:45.996 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-09T01:01:45.996 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:45.996 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.996 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.996 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:45.996 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.997 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.997 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.997 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-09T01:01:45.997 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.997 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.997 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.997 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.997 INFO:teuthology.orchestra.run.smithi005.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-09T01:01:45.997 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-09T01:01:45.997 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:45.997 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.997 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.998 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "*", 2024-05-09T01:01:45.998 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.998 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.998 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.998 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-09T01:01:45.998 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.998 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:45.998 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:45.998 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:45.999 INFO:teuthology.orchestra.run.smithi005.stdout: "max_count_per_host": { 2024-05-09T01:01:45.999 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_count_per_host", 2024-05-09T01:01:45.999 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:45.999 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:45.999 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:45.999 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-05-09T01:01:45.999 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:45.999 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:45.999 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:45.999 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "max number of daemons per service per host", 2024-05-09T01:01:45.999 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:45.999 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.000 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.000 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.000 INFO:teuthology.orchestra.run.smithi005.stdout: "max_osd_draining_count": { 2024-05-09T01:01:46.000 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_osd_draining_count", 2024-05-09T01:01:46.000 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.000 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.000 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.001 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-05-09T01:01:46.001 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.001 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.001 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.001 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-09T01:01:46.001 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.001 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.002 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.002 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.002 INFO:teuthology.orchestra.run.smithi005.stdout: "migration_current": { 2024-05-09T01:01:46.002 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "migration_current", 2024-05-09T01:01:46.002 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.002 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.002 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.002 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.003 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.003 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.003 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.003 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "internal - do not modify", 2024-05-09T01:01:46.003 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.003 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.003 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.003 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.003 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": { 2024-05-09T01:01:46.003 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mode", 2024-05-09T01:01:46.003 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.003 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.004 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.004 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "root", 2024-05-09T01:01:46.004 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.004 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.004 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.004 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm-package", 2024-05-09T01:01:46.004 INFO:teuthology.orchestra.run.smithi005.stdout: "root" 2024-05-09T01:01:46.004 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.004 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "mode for remote execution of cephadm", 2024-05-09T01:01:46.004 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.004 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.005 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.005 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.005 INFO:teuthology.orchestra.run.smithi005.stdout: "oob_default_addr": { 2024-05-09T01:01:46.005 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "oob_default_addr", 2024-05-09T01:01:46.005 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.005 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.005 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.005 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "169.254.1.1", 2024-05-09T01:01:46.005 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.005 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.005 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.005 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-09T01:01:46.006 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.006 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.006 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.006 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.006 INFO:teuthology.orchestra.run.smithi005.stdout: "prometheus_alerts_path": { 2024-05-09T01:01:46.006 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus_alerts_path", 2024-05-09T01:01:46.006 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.006 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.006 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.007 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-09T01:01:46.007 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.007 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.007 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.007 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-09T01:01:46.007 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.007 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.007 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.007 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.007 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_insecure": { 2024-05-09T01:01:46.007 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_insecure", 2024-05-09T01:01:46.008 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.008 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.008 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.008 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.008 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.008 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.008 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.008 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-09T01:01:46.008 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.008 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.008 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.008 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.009 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_password": { 2024-05-09T01:01:46.009 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_password", 2024-05-09T01:01:46.009 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.009 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.009 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.009 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.009 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.009 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.009 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.009 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-09T01:01:46.009 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.010 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.010 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.010 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.010 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_url": { 2024-05-09T01:01:46.010 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_url", 2024-05-09T01:01:46.010 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.010 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.010 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.010 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.010 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.010 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.011 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.011 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-09T01:01:46.011 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.011 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.011 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.011 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.011 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_username": { 2024-05-09T01:01:46.011 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_username", 2024-05-09T01:01:46.011 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.011 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.011 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.011 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.012 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.012 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.012 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.012 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-09T01:01:46.012 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.012 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.012 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.012 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.012 INFO:teuthology.orchestra.run.smithi005.stdout: "secure_monitoring_stack": { 2024-05-09T01:01:46.012 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "secure_monitoring_stack", 2024-05-09T01:01:46.012 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.012 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.013 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.013 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.013 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.013 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.013 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.013 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-09T01:01:46.013 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.013 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.013 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.013 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.013 INFO:teuthology.orchestra.run.smithi005.stdout: "service_discovery_port": { 2024-05-09T01:01:46.013 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "service_discovery_port", 2024-05-09T01:01:46.013 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.014 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.014 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.014 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8765", 2024-05-09T01:01:46.014 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.014 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.014 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.014 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "cephadm service discovery port", 2024-05-09T01:01:46.014 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.014 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.014 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.014 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.014 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:46.015 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:46.015 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.015 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:46.015 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.015 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.015 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.015 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.015 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.015 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.015 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.015 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.015 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.016 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.016 INFO:teuthology.orchestra.run.smithi005.stdout: "ssh_config_file": { 2024-05-09T01:01:46.016 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssh_config_file", 2024-05-09T01:01:46.017 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.017 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.017 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.018 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.018 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.018 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.018 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.018 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-09T01:01:46.018 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.018 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.018 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.018 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.018 INFO:teuthology.orchestra.run.smithi005.stdout: "use_agent": { 2024-05-09T01:01:46.018 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "use_agent", 2024-05-09T01:01:46.019 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.019 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.019 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.019 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.019 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.019 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.019 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.019 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-09T01:01:46.019 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.019 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.019 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.019 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.020 INFO:teuthology.orchestra.run.smithi005.stdout: "use_repo_digest": { 2024-05-09T01:01:46.020 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "use_repo_digest", 2024-05-09T01:01:46.020 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.020 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.020 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.020 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.020 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.020 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.020 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.020 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-09T01:01:46.020 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.020 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.021 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.021 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.021 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_failed_host_check": { 2024-05-09T01:01:46.021 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_failed_host_check", 2024-05-09T01:01:46.021 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.021 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.021 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.021 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.021 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.021 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.021 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.021 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if the host check fails", 2024-05-09T01:01:46.022 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.022 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.022 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.022 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.022 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_stray_daemons": { 2024-05-09T01:01:46.022 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_stray_daemons", 2024-05-09T01:01:46.022 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.022 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.022 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.022 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.022 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.023 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.023 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.023 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-09T01:01:46.023 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.023 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.023 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.023 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.023 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_stray_hosts": { 2024-05-09T01:01:46.023 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_stray_hosts", 2024-05-09T01:01:46.023 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.024 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.024 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.024 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.024 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.024 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.024 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.024 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-09T01:01:46.024 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.024 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.024 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.024 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.025 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.025 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.025 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:46.025 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crash", 2024-05-09T01:01:46.025 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:46.025 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:46.025 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:46.026 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:46.026 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:46.026 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.026 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.026 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.026 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.026 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.026 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.026 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.026 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.027 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.027 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.027 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.027 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.027 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.027 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.028 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.028 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.028 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.028 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.028 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.028 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:46.028 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:46.028 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.028 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.028 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.029 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.029 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.029 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.029 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.029 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.029 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.029 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.029 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.029 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.029 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:46.029 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:46.029 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.029 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.030 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.030 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:46.030 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.030 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.030 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.030 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.030 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.030 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.030 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.030 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.031 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.031 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.031 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.031 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.031 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.031 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.031 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.031 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:46.031 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:46.031 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.031 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.032 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.032 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.032 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.032 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.032 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.032 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.032 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.032 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.032 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.032 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.032 INFO:teuthology.orchestra.run.smithi005.stdout: "retain_interval": { 2024-05-09T01:01:46.032 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "retain_interval", 2024-05-09T01:01:46.032 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-09T01:01:46.033 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.033 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.033 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "31536000", 2024-05-09T01:01:46.033 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.033 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.033 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.033 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-09T01:01:46.033 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.033 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.033 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.033 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.033 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:46.034 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:46.034 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.034 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:46.034 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.034 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.034 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.034 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.034 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.034 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.034 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.034 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.035 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.035 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.035 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_recent_interval": { 2024-05-09T01:01:46.035 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_recent_interval", 2024-05-09T01:01:46.035 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-09T01:01:46.035 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.035 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.035 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1209600", 2024-05-09T01:01:46.035 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.035 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.036 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.036 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-09T01:01:46.036 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.036 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.036 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.036 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.036 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.036 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.036 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:46.036 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "dashboard", 2024-05-09T01:01:46.036 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:46.036 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:46.037 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:46.037 INFO:teuthology.orchestra.run.smithi005.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-09T01:01:46.037 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-09T01:01:46.037 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.037 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.037 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.037 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-05-09T01:01:46.037 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.037 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.037 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.037 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.037 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.037 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.038 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.038 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.038 INFO:teuthology.orchestra.run.smithi005.stdout: "ALERTMANAGER_API_HOST": { 2024-05-09T01:01:46.038 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-09T01:01:46.038 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.038 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.038 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.038 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.038 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.038 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.038 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.039 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.039 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.039 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.039 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.039 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.039 INFO:teuthology.orchestra.run.smithi005.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-09T01:01:46.039 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-09T01:01:46.039 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.039 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.040 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.040 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.040 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.040 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.040 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.040 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.040 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.040 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.040 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.040 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.040 INFO:teuthology.orchestra.run.smithi005.stdout: "AUDIT_API_ENABLED": { 2024-05-09T01:01:46.040 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "AUDIT_API_ENABLED", 2024-05-09T01:01:46.041 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.041 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.041 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.041 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.041 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.041 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.041 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.041 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.041 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.041 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.041 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.041 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.041 INFO:teuthology.orchestra.run.smithi005.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-09T01:01:46.042 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-09T01:01:46.042 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.042 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.042 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.042 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.042 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.042 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.042 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.042 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.042 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.042 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.042 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.043 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.043 INFO:teuthology.orchestra.run.smithi005.stdout: "ENABLE_BROWSABLE_API": { 2024-05-09T01:01:46.043 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-09T01:01:46.043 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.043 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.043 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.043 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.043 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.043 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.043 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.043 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.043 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.043 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.044 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.044 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.044 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-09T01:01:46.044 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-09T01:01:46.044 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.044 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.044 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.045 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.045 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.045 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.045 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.045 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.046 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.046 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.046 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.046 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.046 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-09T01:01:46.046 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-09T01:01:46.046 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.046 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.047 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.047 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.047 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.047 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.047 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.047 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.047 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.047 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.047 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.047 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.047 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-09T01:01:46.047 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-09T01:01:46.047 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.048 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.048 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.048 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.048 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.048 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.048 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.048 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.048 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.048 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.048 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.048 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.048 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-09T01:01:46.048 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-09T01:01:46.048 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.049 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.049 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.049 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.049 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.049 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.049 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.049 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.049 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.049 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.049 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.049 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.049 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-09T01:01:46.050 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-09T01:01:46.050 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.050 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.050 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.050 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.050 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.050 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.050 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.050 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.050 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.051 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.051 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.051 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.051 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-09T01:01:46.051 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-09T01:01:46.051 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.051 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.051 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.051 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.051 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.051 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.051 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.051 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.051 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.052 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.052 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.052 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.052 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-09T01:01:46.052 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-09T01:01:46.052 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.052 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.052 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.052 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.052 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.052 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.052 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.053 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.053 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.053 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.053 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.053 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.053 INFO:teuthology.orchestra.run.smithi005.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-09T01:01:46.053 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-09T01:01:46.053 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.053 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.053 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.053 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.054 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.054 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.054 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.054 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.054 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.054 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.054 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.054 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.054 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_PASSWORD": { 2024-05-09T01:01:46.054 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-09T01:01:46.054 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.054 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.055 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.055 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-05-09T01:01:46.055 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.055 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.055 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.055 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.055 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.055 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.055 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.055 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.055 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-09T01:01:46.056 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-09T01:01:46.056 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.056 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.056 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.056 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.056 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.056 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.057 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.057 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.057 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.057 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.057 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.057 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.057 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_URL": { 2024-05-09T01:01:46.057 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_URL", 2024-05-09T01:01:46.057 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.057 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.057 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.058 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.058 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.058 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.058 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.058 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.058 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.058 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.058 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.058 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.058 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_USERNAME": { 2024-05-09T01:01:46.058 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-09T01:01:46.058 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.059 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.059 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.059 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-05-09T01:01:46.059 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.059 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.059 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.059 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.059 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.059 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.059 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.059 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.059 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-09T01:01:46.059 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-09T01:01:46.060 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.060 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.060 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.060 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.060 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.060 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.060 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.060 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.060 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.060 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.060 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.060 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.060 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-09T01:01:46.060 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-09T01:01:46.061 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.061 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.061 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.061 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.061 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.061 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.061 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.061 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.061 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.061 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.061 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.061 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.061 INFO:teuthology.orchestra.run.smithi005.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-09T01:01:46.062 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-09T01:01:46.062 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.062 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.062 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.062 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.062 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.062 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.062 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.062 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.062 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.062 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.062 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.062 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.063 INFO:teuthology.orchestra.run.smithi005.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-09T01:01:46.063 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-09T01:01:46.063 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.063 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.063 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.063 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.063 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.063 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.063 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.063 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.063 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.063 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.063 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.063 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.064 INFO:teuthology.orchestra.run.smithi005.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-09T01:01:46.064 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-09T01:01:46.064 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.064 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.064 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.064 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "[]", 2024-05-09T01:01:46.064 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.064 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.064 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.064 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.064 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.064 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.064 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.065 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.065 INFO:teuthology.orchestra.run.smithi005.stdout: "MULTICLUSTER_CONFIG": { 2024-05-09T01:01:46.065 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-09T01:01:46.065 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.065 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.065 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.065 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "{}", 2024-05-09T01:01:46.065 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.065 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.065 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.065 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.065 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.065 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.066 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.066 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.066 INFO:teuthology.orchestra.run.smithi005.stdout: "PROMETHEUS_API_HOST": { 2024-05-09T01:01:46.066 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-09T01:01:46.066 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.066 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.066 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.066 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.066 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.066 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.066 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.066 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.066 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.067 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.067 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.067 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.067 INFO:teuthology.orchestra.run.smithi005.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-09T01:01:46.067 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-09T01:01:46.067 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.067 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.067 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.067 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.067 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.067 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.067 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.067 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.068 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.068 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.068 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.068 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.068 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-09T01:01:46.068 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-09T01:01:46.068 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.068 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.068 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.068 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.068 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.068 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.069 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.069 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.069 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.069 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.069 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.069 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.069 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-09T01:01:46.069 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-09T01:01:46.069 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.069 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.069 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.070 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.070 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.070 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.070 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.070 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.070 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.070 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.070 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.070 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.070 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-09T01:01:46.070 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-09T01:01:46.070 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.070 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.071 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.071 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.071 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.071 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.071 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.071 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.071 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.071 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.071 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.071 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.071 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-09T01:01:46.071 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-09T01:01:46.072 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.072 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.072 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.072 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.072 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.072 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.072 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.072 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.072 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.072 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.072 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.073 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.073 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-09T01:01:46.073 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-09T01:01:46.073 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.073 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.073 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.073 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.073 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.073 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.073 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.073 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.073 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.073 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.073 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.074 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.074 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-09T01:01:46.074 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-09T01:01:46.074 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.074 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.074 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.074 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.074 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.074 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.074 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.074 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.074 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.074 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.074 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.075 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.075 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-09T01:01:46.075 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-09T01:01:46.075 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.075 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.075 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.075 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.075 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.075 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.075 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.076 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.076 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.076 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.076 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.076 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.076 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_ENABLED": { 2024-05-09T01:01:46.076 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-09T01:01:46.076 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.076 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.076 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.076 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.076 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.077 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.077 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.077 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.077 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.077 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.077 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.077 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.077 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-09T01:01:46.077 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-09T01:01:46.078 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.078 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.078 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.078 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-09T01:01:46.078 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.078 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.078 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.078 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.078 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.078 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.078 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.078 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.079 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-09T01:01:46.079 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-09T01:01:46.079 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.079 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.079 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.079 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-05-09T01:01:46.079 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.079 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.079 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.079 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.079 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.080 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.080 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.080 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.080 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-09T01:01:46.080 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-09T01:01:46.080 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.080 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.080 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.080 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8", 2024-05-09T01:01:46.080 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.080 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.080 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.080 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.080 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.081 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.081 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.081 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.081 INFO:teuthology.orchestra.run.smithi005.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-09T01:01:46.081 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-09T01:01:46.081 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.081 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.081 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.081 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "45", 2024-05-09T01:01:46.081 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.081 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.081 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.081 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.081 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.082 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.082 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.082 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.082 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_ACCESS_KEY": { 2024-05-09T01:01:46.082 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-09T01:01:46.082 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.082 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.082 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.082 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.082 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.082 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.082 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.082 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.083 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.083 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.083 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.083 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.083 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-09T01:01:46.083 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-09T01:01:46.083 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.083 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.083 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.083 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-05-09T01:01:46.083 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.083 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.083 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.084 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.084 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.084 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.084 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.084 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.084 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_SECRET_KEY": { 2024-05-09T01:01:46.084 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-09T01:01:46.084 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.084 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.084 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.084 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.085 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.085 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.085 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.085 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.085 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.085 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.085 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.085 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.085 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_SSL_VERIFY": { 2024-05-09T01:01:46.085 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-09T01:01:46.085 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.085 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.085 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.086 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.086 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.086 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.086 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.086 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.086 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.086 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.087 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.087 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.087 INFO:teuthology.orchestra.run.smithi005.stdout: "UNSAFE_TLS_v1_2": { 2024-05-09T01:01:46.087 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-09T01:01:46.087 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.087 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.087 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.087 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.087 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.087 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.087 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.087 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.087 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.088 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.088 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.088 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.088 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-09T01:01:46.088 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-09T01:01:46.088 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.089 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.089 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.089 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.089 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.089 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.089 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.089 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.089 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.089 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.089 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.090 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.090 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-09T01:01:46.090 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-09T01:01:46.090 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.090 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.090 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.090 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-05-09T01:01:46.090 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.090 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.090 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.090 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.091 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.091 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.091 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.091 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.091 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-09T01:01:46.091 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-09T01:01:46.091 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.091 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.091 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.091 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-05-09T01:01:46.091 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.091 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.092 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.092 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.092 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.092 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.092 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.092 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.092 INFO:teuthology.orchestra.run.smithi005.stdout: "cross_origin_url": { 2024-05-09T01:01:46.092 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cross_origin_url", 2024-05-09T01:01:46.092 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.092 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.092 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.092 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.092 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.092 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.093 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.093 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.093 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.093 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.093 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.093 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.093 INFO:teuthology.orchestra.run.smithi005.stdout: "crt_file": { 2024-05-09T01:01:46.093 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crt_file", 2024-05-09T01:01:46.093 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.093 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.093 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.093 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.093 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.094 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.094 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.094 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.094 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.094 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.094 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.094 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.094 INFO:teuthology.orchestra.run.smithi005.stdout: "debug": { 2024-05-09T01:01:46.094 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "debug", 2024-05-09T01:01:46.094 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.094 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.094 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.094 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.095 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.095 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.095 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.095 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable/disable debug options", 2024-05-09T01:01:46.095 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.095 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.095 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.095 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.095 INFO:teuthology.orchestra.run.smithi005.stdout: "jwt_token_ttl": { 2024-05-09T01:01:46.095 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "jwt_token_ttl", 2024-05-09T01:01:46.095 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.095 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.095 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.095 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "28800", 2024-05-09T01:01:46.096 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.096 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.096 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.096 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.096 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.096 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.096 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.096 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.096 INFO:teuthology.orchestra.run.smithi005.stdout: "key_file": { 2024-05-09T01:01:46.096 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "key_file", 2024-05-09T01:01:46.096 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.096 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.096 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.097 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.097 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.097 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.097 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.097 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.097 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.097 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.097 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.097 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.097 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:46.097 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:46.097 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.098 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.098 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.098 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.098 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.098 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.098 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.098 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.098 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.098 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.098 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.098 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.098 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.099 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.099 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.099 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.099 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.099 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.099 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.100 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:46.100 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:46.100 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.100 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.100 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.100 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.100 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.100 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.100 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.100 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.100 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.100 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.101 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.101 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.101 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:46.101 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:46.101 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.101 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.101 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.101 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:46.101 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.101 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.101 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.101 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.101 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.101 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.102 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.102 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.102 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.102 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.102 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.102 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.102 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.102 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.102 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.103 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:46.103 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:46.103 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.103 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.103 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.103 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.104 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.104 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.104 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.104 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.104 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.104 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.104 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.104 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.104 INFO:teuthology.orchestra.run.smithi005.stdout: "motd": { 2024-05-09T01:01:46.104 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "motd", 2024-05-09T01:01:46.104 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.104 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.104 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.105 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.105 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.105 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.105 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.105 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "The message of the day", 2024-05-09T01:01:46.105 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.105 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.105 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.105 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.105 INFO:teuthology.orchestra.run.smithi005.stdout: "redirect_resolve_ip_addr": { 2024-05-09T01:01:46.105 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "redirect_resolve_ip_addr", 2024-05-09T01:01:46.105 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.105 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.106 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.106 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.106 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.106 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.106 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.106 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.106 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.106 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.106 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.106 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.106 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-05-09T01:01:46.106 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-05-09T01:01:46.106 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.107 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.107 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.107 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "::", 2024-05-09T01:01:46.107 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.107 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.107 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.107 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.107 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.107 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.107 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.107 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.107 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-05-09T01:01:46.107 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-05-09T01:01:46.107 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.108 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.108 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.108 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8080", 2024-05-09T01:01:46.108 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.108 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.108 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.108 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.108 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.108 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.108 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.108 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.108 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:46.108 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:46.109 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.109 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:46.109 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.109 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.109 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.109 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.109 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.109 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.109 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.109 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.109 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.109 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.109 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl": { 2024-05-09T01:01:46.109 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl", 2024-05-09T01:01:46.110 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.110 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.110 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.110 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.110 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.110 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.110 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.110 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.110 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.110 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.110 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.110 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.110 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl_server_port": { 2024-05-09T01:01:46.111 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl_server_port", 2024-05-09T01:01:46.111 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.111 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.111 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.111 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8443", 2024-05-09T01:01:46.111 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.111 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.111 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.111 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.111 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.111 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.111 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.111 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.111 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_behaviour": { 2024-05-09T01:01:46.112 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_behaviour", 2024-05-09T01:01:46.112 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.112 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.112 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.112 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "redirect", 2024-05-09T01:01:46.112 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.112 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.112 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.112 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.112 INFO:teuthology.orchestra.run.smithi005.stdout: "redirect" 2024-05-09T01:01:46.112 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.112 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.112 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.112 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.113 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.113 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.113 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_error_status_code": { 2024-05-09T01:01:46.113 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_error_status_code", 2024-05-09T01:01:46.113 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.113 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.113 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.113 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "500", 2024-05-09T01:01:46.113 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "400", 2024-05-09T01:01:46.113 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "599", 2024-05-09T01:01:46.113 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.113 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.113 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.113 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.113 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.114 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.114 INFO:teuthology.orchestra.run.smithi005.stdout: "url_prefix": { 2024-05-09T01:01:46.114 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "url_prefix", 2024-05-09T01:01:46.114 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.114 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.114 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.114 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.114 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.114 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.114 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.114 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.115 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.115 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.115 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.115 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.115 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.115 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.115 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:46.115 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "devicehealth", 2024-05-09T01:01:46.115 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:46.115 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:46.115 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:46.115 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_monitoring": { 2024-05-09T01:01:46.115 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enable_monitoring", 2024-05-09T01:01:46.115 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.116 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.116 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.116 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.116 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.116 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.116 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.116 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "monitor device health metrics", 2024-05-09T01:01:46.116 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.116 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.116 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.116 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.116 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:46.117 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:46.117 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.117 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.117 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.117 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.117 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.117 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.117 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.117 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.117 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.117 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.117 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.117 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.118 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.118 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.118 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.118 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.118 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.118 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.118 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.118 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:46.118 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:46.118 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.118 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.119 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.119 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.119 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.119 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.119 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.119 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.119 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.119 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.119 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.119 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.119 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:46.119 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:46.119 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.120 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.120 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.120 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:46.120 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.120 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.120 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.120 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.120 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.120 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.120 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.120 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.120 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.120 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.120 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.121 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.121 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.121 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.121 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.121 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:46.121 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:46.121 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.121 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.121 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.121 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.121 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.121 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.121 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.121 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.122 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.122 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.122 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.122 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.122 INFO:teuthology.orchestra.run.smithi005.stdout: "mark_out_threshold": { 2024-05-09T01:01:46.122 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mark_out_threshold", 2024-05-09T01:01:46.122 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-09T01:01:46.122 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.122 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.122 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "2419200", 2024-05-09T01:01:46.122 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.122 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.122 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.122 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-09T01:01:46.123 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.123 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.123 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.123 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.123 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": { 2024-05-09T01:01:46.123 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pool_name", 2024-05-09T01:01:46.123 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.123 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.123 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.123 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "device_health_metrics", 2024-05-09T01:01:46.123 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.123 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.123 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.124 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-09T01:01:46.124 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.124 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.124 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.124 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.124 INFO:teuthology.orchestra.run.smithi005.stdout: "retention_period": { 2024-05-09T01:01:46.124 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "retention_period", 2024-05-09T01:01:46.124 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-09T01:01:46.124 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.124 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.124 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15552000", 2024-05-09T01:01:46.124 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.125 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.125 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.125 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long to retain device health metrics", 2024-05-09T01:01:46.125 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.125 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.125 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.125 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.125 INFO:teuthology.orchestra.run.smithi005.stdout: "scrape_frequency": { 2024-05-09T01:01:46.125 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "scrape_frequency", 2024-05-09T01:01:46.125 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-09T01:01:46.125 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.125 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.125 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "86400", 2024-05-09T01:01:46.125 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.126 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.126 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.126 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-09T01:01:46.126 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.126 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.126 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.126 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.126 INFO:teuthology.orchestra.run.smithi005.stdout: "self_heal": { 2024-05-09T01:01:46.126 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "self_heal", 2024-05-09T01:01:46.126 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.126 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.126 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.126 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.126 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.127 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.127 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.127 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-09T01:01:46.127 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.127 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.127 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.127 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.127 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-05-09T01:01:46.127 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-05-09T01:01:46.127 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-09T01:01:46.127 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.127 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.127 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-05-09T01:01:46.127 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.128 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.128 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.128 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to wake up and check device health", 2024-05-09T01:01:46.128 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.128 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.128 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.128 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.128 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:46.128 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:46.128 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.128 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:46.129 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.129 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.129 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.129 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.129 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.129 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.129 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.129 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.129 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.129 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.129 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_threshold": { 2024-05-09T01:01:46.129 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_threshold", 2024-05-09T01:01:46.129 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-09T01:01:46.130 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.130 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.130 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "7257600", 2024-05-09T01:01:46.130 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.130 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.130 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.130 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-09T01:01:46.130 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.130 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.130 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.130 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.130 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.130 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.130 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:46.131 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "diskprediction_local", 2024-05-09T01:01:46.131 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:46.131 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:46.131 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:46.131 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:46.131 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:46.131 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.131 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.131 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.131 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.131 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.131 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.132 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.132 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.132 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.132 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.132 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.132 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.132 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.132 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.132 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.132 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.132 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.132 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.132 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.133 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:46.133 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:46.133 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.133 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.133 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.133 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.133 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.133 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.133 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.133 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.133 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.133 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.133 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.134 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:46.134 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:46.134 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.134 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.134 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.134 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:46.134 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.134 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.134 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.134 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.134 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.134 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.135 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.135 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.135 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.135 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.135 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.135 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.135 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.135 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.135 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.135 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:46.135 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:46.135 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.135 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.136 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.136 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.136 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.136 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.136 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.136 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.136 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.136 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.136 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.137 INFO:teuthology.orchestra.run.smithi005.stdout: "predict_interval": { 2024-05-09T01:01:46.137 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "predict_interval", 2024-05-09T01:01:46.137 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.137 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.137 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.137 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "86400", 2024-05-09T01:01:46.137 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.137 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.137 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.137 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.137 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.138 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.138 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.138 INFO:teuthology.orchestra.run.smithi005.stdout: "predictor_model": { 2024-05-09T01:01:46.138 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "predictor_model", 2024-05-09T01:01:46.138 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.139 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.139 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.139 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "prophetstor", 2024-05-09T01:01:46.139 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.139 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.139 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.140 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.140 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.140 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.140 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.140 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.140 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-05-09T01:01:46.140 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-05-09T01:01:46.141 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.141 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.141 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.141 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-05-09T01:01:46.141 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.141 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.141 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.141 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.141 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.141 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.141 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.142 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:46.142 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:46.142 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.142 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:46.142 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.142 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.142 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.142 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.142 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.143 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.143 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.143 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.143 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.143 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.143 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.143 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:46.143 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "influx", 2024-05-09T01:01:46.144 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": false, 2024-05-09T01:01:46.144 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "influxdb python module not found", 2024-05-09T01:01:46.144 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:46.144 INFO:teuthology.orchestra.run.smithi005.stdout: "batch_size": { 2024-05-09T01:01:46.144 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "batch_size", 2024-05-09T01:01:46.144 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.144 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.144 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.144 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5000", 2024-05-09T01:01:46.144 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.144 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.144 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.144 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-09T01:01:46.145 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.145 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.145 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.145 INFO:teuthology.orchestra.run.smithi005.stdout: "database": { 2024-05-09T01:01:46.145 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "database", 2024-05-09T01:01:46.145 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.145 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.145 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.145 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "ceph", 2024-05-09T01:01:46.145 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.145 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.145 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.145 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-09T01:01:46.146 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.146 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.146 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.146 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.146 INFO:teuthology.orchestra.run.smithi005.stdout: "hostname": { 2024-05-09T01:01:46.146 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "hostname", 2024-05-09T01:01:46.146 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.146 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.146 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.146 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.146 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.146 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.146 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.147 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB server hostname", 2024-05-09T01:01:46.147 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.147 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.147 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.147 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-05-09T01:01:46.147 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-05-09T01:01:46.147 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-09T01:01:46.147 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.147 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.147 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "30", 2024-05-09T01:01:46.147 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "5", 2024-05-09T01:01:46.147 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.148 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.148 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-09T01:01:46.148 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.148 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.148 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.148 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:46.148 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:46.148 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.148 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.148 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.148 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.148 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.148 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.149 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.149 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.149 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.149 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.149 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.149 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.149 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.149 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.149 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.149 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.149 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.149 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.149 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:46.150 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:46.150 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.150 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.150 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.150 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.150 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.150 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.150 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.150 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.150 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.150 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.150 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.151 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:46.151 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:46.151 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.151 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.151 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.151 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:46.151 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.151 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.151 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.151 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.151 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.151 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.152 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.152 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.152 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.152 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.152 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.152 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.152 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.152 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.152 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:46.152 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:46.152 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.152 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.153 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.153 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.153 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.153 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.153 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.153 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.153 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.153 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.153 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.153 INFO:teuthology.orchestra.run.smithi005.stdout: "password": { 2024-05-09T01:01:46.154 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "password", 2024-05-09T01:01:46.154 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.154 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.154 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.154 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.154 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.154 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.154 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.154 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "password of InfluxDB server user", 2024-05-09T01:01:46.154 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.154 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.154 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.155 INFO:teuthology.orchestra.run.smithi005.stdout: "port": { 2024-05-09T01:01:46.155 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "port", 2024-05-09T01:01:46.155 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.155 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.155 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.155 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8086", 2024-05-09T01:01:46.155 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.155 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.155 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.155 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB server port", 2024-05-09T01:01:46.155 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.155 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.155 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.156 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:46.156 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:46.156 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.156 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:46.156 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.156 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.156 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.156 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.156 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.156 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.156 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.156 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.156 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.157 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.157 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl": { 2024-05-09T01:01:46.157 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl", 2024-05-09T01:01:46.157 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.157 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.157 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.157 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "false", 2024-05-09T01:01:46.157 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.157 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.157 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.157 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T01:01:46.157 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.157 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.157 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.158 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.158 INFO:teuthology.orchestra.run.smithi005.stdout: "threads": { 2024-05-09T01:01:46.158 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "threads", 2024-05-09T01:01:46.158 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.158 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.158 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.158 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-05-09T01:01:46.158 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-05-09T01:01:46.158 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "32", 2024-05-09T01:01:46.158 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.158 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-09T01:01:46.159 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.159 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.159 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.159 INFO:teuthology.orchestra.run.smithi005.stdout: "username": { 2024-05-09T01:01:46.159 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "username", 2024-05-09T01:01:46.159 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.159 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.159 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.159 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.159 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.159 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.159 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.159 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "username of InfluxDB server user", 2024-05-09T01:01:46.160 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.160 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.160 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.160 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.160 INFO:teuthology.orchestra.run.smithi005.stdout: "verify_ssl": { 2024-05-09T01:01:46.160 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "verify_ssl", 2024-05-09T01:01:46.160 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.160 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.160 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.160 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "true", 2024-05-09T01:01:46.160 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.161 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.161 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.161 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T01:01:46.161 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.161 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.161 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.162 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.162 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.162 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.162 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:46.162 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "insights", 2024-05-09T01:01:46.162 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:46.162 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:46.163 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:46.163 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:46.163 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:46.163 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.163 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.163 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.163 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.163 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.164 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.164 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.164 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.164 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.164 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.164 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.165 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.165 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.165 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.165 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.165 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.165 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.165 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.166 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.166 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:46.166 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:46.166 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.166 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.166 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.167 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.167 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.167 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.167 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.167 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.167 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.167 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.167 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.168 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:46.168 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:46.168 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.168 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.168 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.168 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:46.168 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.168 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.169 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.169 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.169 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.169 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.169 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.169 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.169 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.169 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.169 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.170 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.170 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.170 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.170 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.170 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:46.170 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:46.170 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.170 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.170 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.170 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.170 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.170 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.171 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.171 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.171 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.171 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.171 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.171 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:46.171 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:46.171 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.171 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:46.171 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.171 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.171 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.172 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.172 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.172 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.172 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.172 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.172 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.172 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.172 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:46.172 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "iostat", 2024-05-09T01:01:46.172 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:46.172 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:46.172 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:46.173 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:46.173 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:46.173 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.173 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.173 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.173 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.173 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.173 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.173 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.173 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.173 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.173 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.173 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.173 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.174 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.174 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.174 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.174 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.174 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.174 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.174 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:46.174 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:46.174 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.174 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.174 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.175 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.175 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.175 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.175 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.175 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.175 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.175 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.175 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.175 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:46.175 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:46.176 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.176 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.176 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.176 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:46.176 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.176 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.176 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.176 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.176 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.176 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.177 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.177 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.177 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.177 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.177 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.177 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.177 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.177 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.177 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.177 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:46.177 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:46.177 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.178 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.178 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.178 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.178 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.178 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.178 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.178 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.178 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.178 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.178 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.178 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:46.178 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:46.179 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.179 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:46.179 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.179 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.179 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.179 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.179 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.179 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.179 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.179 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.179 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.179 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.180 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.180 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:46.180 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "k8sevents", 2024-05-09T01:01:46.180 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:46.180 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:46.180 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:46.180 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_event_retention_days": { 2024-05-09T01:01:46.180 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ceph_event_retention_days", 2024-05-09T01:01:46.180 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.181 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.181 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.181 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "7", 2024-05-09T01:01:46.181 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.181 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.181 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.182 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-09T01:01:46.182 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.182 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.182 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.182 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.182 INFO:teuthology.orchestra.run.smithi005.stdout: "config_check_secs": { 2024-05-09T01:01:46.182 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_check_secs", 2024-05-09T01:01:46.183 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.183 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.183 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.183 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-05-09T01:01:46.183 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "10", 2024-05-09T01:01:46.183 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.183 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.183 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-09T01:01:46.184 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.184 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.184 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.184 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:46.184 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:46.184 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.184 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.184 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.185 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.185 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.185 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.185 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.185 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.185 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.185 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.185 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.186 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.186 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.186 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.186 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.186 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.186 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.186 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.186 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.186 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:46.186 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:46.186 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.186 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.186 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.187 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.187 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.187 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.187 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.187 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.187 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.187 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.187 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.187 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.187 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:46.187 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:46.187 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.187 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.187 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.188 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:46.188 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.188 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.188 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.188 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.188 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.188 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.188 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.188 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.188 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.188 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.188 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.188 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.189 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.189 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.189 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:46.189 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:46.189 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.189 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.189 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.189 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.189 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.189 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.189 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.189 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.189 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.190 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.190 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.190 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.190 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:46.190 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:46.190 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.190 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:46.190 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.190 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.190 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.190 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.190 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.190 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.191 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.191 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.191 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.191 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.191 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.191 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:46.191 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "localpool", 2024-05-09T01:01:46.191 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:46.191 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:46.191 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:46.191 INFO:teuthology.orchestra.run.smithi005.stdout: "failure_domain": { 2024-05-09T01:01:46.191 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "failure_domain", 2024-05-09T01:01:46.191 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.192 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.192 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.192 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "host", 2024-05-09T01:01:46.192 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.192 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.192 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.192 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "failure domain for any created local pool", 2024-05-09T01:01:46.192 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-09T01:01:46.192 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.192 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.192 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:46.193 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:46.193 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.193 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.193 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.193 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.193 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.193 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.193 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.193 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.193 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.194 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.194 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.194 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.194 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.194 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.194 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.194 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.194 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.194 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.194 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:46.194 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:46.195 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.195 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.195 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.195 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.195 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.195 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.195 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.195 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.195 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.195 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.195 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.195 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:46.196 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:46.196 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.196 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.196 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.196 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:46.196 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.196 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.196 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.196 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.196 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.197 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.197 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.197 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.197 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.197 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.197 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.197 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.197 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.197 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.197 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:46.198 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:46.198 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.198 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.198 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.198 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.198 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.198 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.198 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.198 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.198 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.199 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.199 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.199 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": { 2024-05-09T01:01:46.199 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "min_size", 2024-05-09T01:01:46.199 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.199 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.199 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.199 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.199 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.199 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.199 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.200 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default min_size for any created local pool", 2024-05-09T01:01:46.200 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-09T01:01:46.200 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.200 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.200 INFO:teuthology.orchestra.run.smithi005.stdout: "num_rep": { 2024-05-09T01:01:46.200 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "num_rep", 2024-05-09T01:01:46.200 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.200 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.200 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.200 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3", 2024-05-09T01:01:46.200 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.201 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.201 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.201 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default replica count for any created local pool", 2024-05-09T01:01:46.201 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.201 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.201 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.201 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.201 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": { 2024-05-09T01:01:46.201 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pg_num", 2024-05-09T01:01:46.201 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.201 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.201 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.201 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "128", 2024-05-09T01:01:46.201 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.202 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.202 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.202 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default pg_num for any created local pool", 2024-05-09T01:01:46.202 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.202 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.202 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.202 INFO:teuthology.orchestra.run.smithi005.stdout: "prefix": { 2024-05-09T01:01:46.202 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prefix", 2024-05-09T01:01:46.202 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.202 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.202 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.202 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.203 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.203 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.203 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.203 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "name prefix for any created local pool", 2024-05-09T01:01:46.203 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.203 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.203 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.203 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:46.203 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:46.203 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.204 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:46.204 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.204 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.204 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.204 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.204 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.204 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.204 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.204 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.204 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.205 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.205 INFO:teuthology.orchestra.run.smithi005.stdout: "subtree": { 2024-05-09T01:01:46.205 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "subtree", 2024-05-09T01:01:46.205 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.205 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.205 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.205 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "rack", 2024-05-09T01:01:46.205 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.205 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.205 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.205 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-09T01:01:46.205 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-09T01:01:46.205 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.205 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.206 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.206 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.206 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:46.206 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mds_autoscaler", 2024-05-09T01:01:46.206 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:46.206 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:46.206 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:46.206 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:46.206 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:46.206 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.206 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.206 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.207 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.207 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.207 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.207 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.207 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.207 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.207 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.207 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.207 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.207 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.207 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.207 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.207 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.207 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.208 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.208 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:46.208 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:46.208 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.208 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.208 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.208 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.208 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.208 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.208 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.208 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.208 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.208 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.209 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.209 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.209 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:46.209 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:46.209 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.209 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.209 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.209 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:46.209 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.209 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.209 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.209 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.209 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.210 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.210 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.210 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.210 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.210 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.210 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.210 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.210 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.210 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.210 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:46.210 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:46.210 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.210 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.211 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.211 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.211 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.211 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.211 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.211 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.211 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.211 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.211 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.211 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:46.211 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:46.211 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.211 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:46.212 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.212 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.212 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.212 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.212 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.212 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.212 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.212 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.212 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.212 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.212 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.212 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:46.213 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mirroring", 2024-05-09T01:01:46.213 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:46.213 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:46.213 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:46.213 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:46.213 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:46.213 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.213 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.213 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.213 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.213 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.213 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.213 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.214 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.214 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.214 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.214 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.214 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.214 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.214 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.214 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.214 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.214 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.214 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.214 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:46.215 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:46.215 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.215 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.215 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.215 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.215 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.215 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.215 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.215 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.215 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.215 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.215 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.215 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:46.216 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:46.216 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.216 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.216 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.216 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:46.216 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.216 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.216 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.216 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.216 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.216 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.216 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.216 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.216 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.217 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.217 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.217 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.217 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.217 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.217 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.217 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:46.217 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:46.217 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.217 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.217 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.217 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.218 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.218 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.218 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.218 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.218 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.218 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.218 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.218 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:46.218 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:46.218 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.218 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:46.218 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.219 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.219 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.219 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.219 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.219 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.219 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.219 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.219 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.219 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.219 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.219 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.219 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:46.219 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "nfs", 2024-05-09T01:01:46.220 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:46.220 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:46.220 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:46.220 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:46.220 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:46.220 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.220 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.220 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.220 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.220 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.220 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.220 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.220 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.220 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.221 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.221 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.221 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.221 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.221 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.221 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.221 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.221 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.221 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.221 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.221 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:46.221 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:46.222 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.222 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.222 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.222 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.222 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.222 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.222 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.222 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.222 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.222 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.222 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.222 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.222 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:46.223 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:46.223 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.223 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.223 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.223 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:46.223 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.223 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.223 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.223 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.223 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.223 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.223 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.223 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.223 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.224 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.224 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.224 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.224 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.224 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.224 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.224 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:46.224 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:46.224 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.224 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.224 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.225 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.225 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.225 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.225 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.225 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.225 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.225 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.225 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.225 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:46.225 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:46.225 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.225 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:46.225 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.226 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.226 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.226 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.226 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.226 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.226 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.226 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.226 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.226 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.226 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.227 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:46.227 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "orchestrator", 2024-05-09T01:01:46.227 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:46.227 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:46.227 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:46.227 INFO:teuthology.orchestra.run.smithi005.stdout: "fail_fs": { 2024-05-09T01:01:46.227 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "fail_fs", 2024-05-09T01:01:46.227 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.227 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.227 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.227 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.228 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.228 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.228 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.228 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-09T01:01:46.228 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.228 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.228 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.228 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.228 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:46.228 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:46.228 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.228 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.228 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.229 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.229 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.229 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.229 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.229 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.229 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.229 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.229 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.229 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.229 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.229 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.230 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.230 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.230 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.230 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.230 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.231 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:46.231 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:46.231 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.231 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.231 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.231 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.232 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.232 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.232 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.232 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.232 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.232 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.232 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.233 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.233 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:46.233 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:46.233 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.233 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.233 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.233 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:46.233 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.233 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.233 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.233 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.233 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.234 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.234 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.234 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.234 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.234 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.234 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.234 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.234 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.235 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.235 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.235 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:46.235 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:46.235 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.235 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.235 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.236 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.236 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.236 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.236 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.236 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.236 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.236 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.236 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.236 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.236 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator": { 2024-05-09T01:01:46.236 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "orchestrator", 2024-05-09T01:01:46.236 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.236 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.237 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.237 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.237 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.237 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.237 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.237 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm", 2024-05-09T01:01:46.237 INFO:teuthology.orchestra.run.smithi005.stdout: "rook", 2024-05-09T01:01:46.237 INFO:teuthology.orchestra.run.smithi005.stdout: "test_orchestrator" 2024-05-09T01:01:46.237 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.237 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Orchestrator backend", 2024-05-09T01:01:46.237 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.237 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.238 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.238 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.238 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:46.238 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:46.238 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.238 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:46.238 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.238 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.238 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.238 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.238 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.238 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.238 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.239 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.239 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.239 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.239 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.239 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.239 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:46.239 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "osd_perf_query", 2024-05-09T01:01:46.239 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:46.239 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:46.239 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:46.239 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:46.239 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:46.239 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.239 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.240 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.240 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.240 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.240 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.240 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.240 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.240 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.240 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.240 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.240 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.240 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.240 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.240 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.240 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.241 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.241 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.241 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.241 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:46.241 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:46.241 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.241 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.241 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.241 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.242 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.242 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.242 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.242 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.242 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.242 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.242 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.242 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.242 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:46.242 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:46.242 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.242 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.242 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.243 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:46.243 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.243 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.243 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.243 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.243 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.243 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.243 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.243 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.243 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.243 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.243 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.243 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.243 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.244 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.244 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.244 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:46.244 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:46.244 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.244 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.244 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.244 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.244 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.244 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.244 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.244 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.244 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.244 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.244 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.245 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.245 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:46.245 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:46.245 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.245 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:46.245 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.245 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.245 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.245 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.245 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.245 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.245 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.246 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.246 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.246 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.246 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.246 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.246 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:46.246 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "osd_support", 2024-05-09T01:01:46.246 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:46.246 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:46.246 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:46.246 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:46.246 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:46.246 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.247 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.247 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.247 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.247 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.247 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.247 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.247 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.247 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.247 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.247 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.247 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.247 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.247 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.247 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.248 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.248 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.248 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.248 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.248 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:46.248 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:46.248 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.248 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.248 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.248 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.248 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.248 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.249 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.249 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.249 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.249 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.249 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.249 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.250 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:46.250 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:46.250 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.250 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.250 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.250 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:46.251 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.251 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.251 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.251 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.251 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.252 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.252 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.252 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.252 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.252 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.252 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.252 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.253 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.253 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.253 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.253 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:46.253 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:46.253 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.253 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.253 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.253 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.253 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.253 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.254 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.254 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.254 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.254 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.254 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.254 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.254 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:46.254 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:46.254 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.254 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:46.254 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.255 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.255 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.255 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.255 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.255 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.255 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.255 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.255 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.255 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.255 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.255 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.255 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:46.255 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pg_autoscaler", 2024-05-09T01:01:46.255 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:46.256 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:46.256 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:46.256 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:46.256 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:46.256 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.256 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.256 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.256 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.256 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.256 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.256 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.256 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.256 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.256 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.257 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.257 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.257 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.257 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.257 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.257 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.257 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.257 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.257 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.257 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:46.257 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:46.257 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.257 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.257 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.258 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.258 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.258 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.258 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.258 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.258 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.258 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.258 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.258 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.258 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:46.258 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:46.258 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.258 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.258 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.259 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:46.259 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.259 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.259 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.259 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.259 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.259 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.259 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.259 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.260 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.260 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.260 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.260 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.260 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.260 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.260 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.260 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:46.260 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:46.260 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.260 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.260 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.260 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.260 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.261 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.261 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.261 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.261 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.261 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.261 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.261 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.261 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-05-09T01:01:46.261 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-05-09T01:01:46.261 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-09T01:01:46.261 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.261 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.261 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-05-09T01:01:46.261 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.262 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.262 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.262 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.262 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.262 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.262 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.262 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.262 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:46.262 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:46.262 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.262 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:46.262 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.262 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.263 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.263 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.263 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.263 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.263 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.263 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.263 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.263 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.263 INFO:teuthology.orchestra.run.smithi005.stdout: "threshold": { 2024-05-09T01:01:46.263 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "threshold", 2024-05-09T01:01:46.263 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-05-09T01:01:46.263 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.263 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.263 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3.0", 2024-05-09T01:01:46.264 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1.0", 2024-05-09T01:01:46.264 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.264 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.264 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "scaling threshold", 2024-05-09T01:01:46.264 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-09T01:01:46.264 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.264 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.264 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.264 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.264 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.264 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:46.265 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "progress", 2024-05-09T01:01:46.265 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:46.265 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:46.265 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:46.265 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_pg_recovery_event": { 2024-05-09T01:01:46.265 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_pg_recovery_event", 2024-05-09T01:01:46.265 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.265 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.265 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.265 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.265 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.265 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.265 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.265 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-09T01:01:46.266 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.266 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.266 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.266 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.266 INFO:teuthology.orchestra.run.smithi005.stdout: "enabled": { 2024-05-09T01:01:46.266 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enabled", 2024-05-09T01:01:46.266 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.266 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.266 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.266 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.266 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.266 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.266 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.267 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.267 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.267 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.267 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.267 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.267 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:46.267 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:46.267 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.267 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.267 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.267 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.267 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.267 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.267 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.268 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.268 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.268 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.268 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.268 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.268 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.268 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.268 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.268 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.268 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.268 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.268 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.268 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:46.269 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:46.269 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.269 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.269 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.269 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.269 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.269 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.269 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.269 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.269 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.270 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.270 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.270 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.270 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:46.270 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:46.270 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.270 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.270 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.270 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:46.270 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.271 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.271 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.271 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.271 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.271 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.271 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.271 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.271 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.271 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.271 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.271 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.271 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.271 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.272 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.272 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:46.272 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:46.272 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.272 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.272 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.272 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.272 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.272 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.272 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.272 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.272 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.272 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.273 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.273 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.273 INFO:teuthology.orchestra.run.smithi005.stdout: "max_completed_events": { 2024-05-09T01:01:46.273 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_completed_events", 2024-05-09T01:01:46.273 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.273 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.273 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.273 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "50", 2024-05-09T01:01:46.273 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.273 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.273 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.273 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "number of past completed events to remember", 2024-05-09T01:01:46.273 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.274 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.274 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.274 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.274 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-05-09T01:01:46.274 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-05-09T01:01:46.274 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-09T01:01:46.274 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.274 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.274 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-05-09T01:01:46.274 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.274 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.274 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.274 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long the module is going to sleep", 2024-05-09T01:01:46.274 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.275 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.275 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.275 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.275 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:46.275 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:46.275 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.275 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:46.275 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.275 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.275 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.275 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.276 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.276 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.276 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.276 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.276 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.276 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.276 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.276 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.276 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:46.276 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus", 2024-05-09T01:01:46.276 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:46.276 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:46.276 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:46.276 INFO:teuthology.orchestra.run.smithi005.stdout: "cache": { 2024-05-09T01:01:46.276 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cache", 2024-05-09T01:01:46.277 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.277 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.277 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.277 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.277 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.277 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.277 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.277 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.277 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.277 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.277 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.278 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.278 INFO:teuthology.orchestra.run.smithi005.stdout: "exclude_perf_counters": { 2024-05-09T01:01:46.278 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "exclude_perf_counters", 2024-05-09T01:01:46.278 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.278 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.278 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.278 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.278 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.278 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.278 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.278 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-09T01:01:46.278 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-09T01:01:46.278 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.278 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.279 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.279 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:46.279 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:46.279 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.279 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.279 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.279 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.279 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.279 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.279 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.279 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.279 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.279 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.280 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.280 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.280 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.280 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.280 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.280 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.280 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.280 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.280 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.280 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:46.280 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:46.280 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.280 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.280 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.281 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.281 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.281 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.281 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.281 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.281 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.281 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.281 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.281 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.281 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:46.281 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:46.281 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.281 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.281 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.282 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:46.282 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.282 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.282 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.282 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.282 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.282 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.282 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.282 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.282 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.282 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.282 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.282 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.283 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.283 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.283 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.283 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:46.283 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:46.283 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.283 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.283 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.283 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.283 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.283 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.283 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.283 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.283 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.284 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.284 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.284 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.284 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_stats_pools": { 2024-05-09T01:01:46.284 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_stats_pools", 2024-05-09T01:01:46.284 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.284 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.284 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.284 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.284 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.284 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.284 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.284 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.284 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.284 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.285 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.285 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.285 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-09T01:01:46.285 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-09T01:01:46.285 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.285 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.285 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.285 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "300", 2024-05-09T01:01:46.285 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.285 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.285 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.285 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.285 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.286 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.286 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.286 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.286 INFO:teuthology.orchestra.run.smithi005.stdout: "scrape_interval": { 2024-05-09T01:01:46.286 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "scrape_interval", 2024-05-09T01:01:46.286 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-05-09T01:01:46.286 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.286 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.286 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15.0", 2024-05-09T01:01:46.286 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.286 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.286 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.286 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.287 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.287 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.287 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.287 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.287 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-05-09T01:01:46.287 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-05-09T01:01:46.287 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.287 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.287 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.287 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "::", 2024-05-09T01:01:46.287 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.287 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.287 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.287 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-09T01:01:46.288 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.288 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.288 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.288 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.288 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-05-09T01:01:46.288 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-05-09T01:01:46.288 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.288 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.288 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.288 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "9283", 2024-05-09T01:01:46.288 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.288 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.288 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.288 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-09T01:01:46.289 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.289 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.289 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.289 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.289 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:46.289 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:46.289 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.289 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:46.289 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.289 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.289 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.289 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.289 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.289 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.290 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.290 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.290 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.290 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.290 INFO:teuthology.orchestra.run.smithi005.stdout: "stale_cache_strategy": { 2024-05-09T01:01:46.290 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "stale_cache_strategy", 2024-05-09T01:01:46.290 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.290 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.290 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.290 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "log", 2024-05-09T01:01:46.290 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.290 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.291 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.291 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.291 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.291 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.291 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.291 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.291 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_behaviour": { 2024-05-09T01:01:46.291 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_behaviour", 2024-05-09T01:01:46.291 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.291 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.291 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.291 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "default", 2024-05-09T01:01:46.292 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.292 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.292 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.292 INFO:teuthology.orchestra.run.smithi005.stdout: "default", 2024-05-09T01:01:46.292 INFO:teuthology.orchestra.run.smithi005.stdout: "error" 2024-05-09T01:01:46.292 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.292 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.292 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.292 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.292 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.292 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.292 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_error_status_code": { 2024-05-09T01:01:46.292 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_error_status_code", 2024-05-09T01:01:46.293 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.293 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.293 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.293 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "500", 2024-05-09T01:01:46.293 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "400", 2024-05-09T01:01:46.293 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "599", 2024-05-09T01:01:46.293 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.293 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.294 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.294 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.294 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.294 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.294 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.294 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.294 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:46.294 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_support", 2024-05-09T01:01:46.294 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:46.294 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:46.294 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:46.294 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:46.295 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:46.295 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.295 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.295 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.295 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.296 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.296 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.296 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.296 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.296 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.296 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.296 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.296 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.296 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.296 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.296 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.297 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.297 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.297 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.297 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.297 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:46.297 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:46.297 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.297 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.297 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.297 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.297 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.297 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.297 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.298 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.298 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.298 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.298 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.298 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.298 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:46.298 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:46.298 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.298 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.298 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.298 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:46.298 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.298 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.298 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.299 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.299 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.299 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.299 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.299 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.299 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.299 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.299 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.299 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.299 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.299 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.299 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.299 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:46.299 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:46.300 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.300 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.300 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.300 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.300 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.300 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.300 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.300 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.300 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.300 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.300 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.300 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.300 INFO:teuthology.orchestra.run.smithi005.stdout: "max_concurrent_snap_create": { 2024-05-09T01:01:46.300 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_concurrent_snap_create", 2024-05-09T01:01:46.301 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.301 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.301 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.301 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-05-09T01:01:46.301 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.301 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.301 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.301 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.301 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.301 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.301 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.301 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.301 INFO:teuthology.orchestra.run.smithi005.stdout: "mirror_snapshot_schedule": { 2024-05-09T01:01:46.301 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mirror_snapshot_schedule", 2024-05-09T01:01:46.301 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.302 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.302 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.302 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.302 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.302 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.302 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.302 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.302 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.302 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.302 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.302 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.303 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:46.303 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:46.303 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.303 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:46.303 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.303 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.303 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.304 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.304 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.304 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.304 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.304 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.304 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.304 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.304 INFO:teuthology.orchestra.run.smithi005.stdout: "trash_purge_schedule": { 2024-05-09T01:01:46.304 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "trash_purge_schedule", 2024-05-09T01:01:46.304 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.304 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.305 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.305 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.305 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.305 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.305 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.305 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.305 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.305 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.306 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.306 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.306 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.306 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.306 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:46.306 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "restful", 2024-05-09T01:01:46.306 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:46.306 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:46.306 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:46.306 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_auth": { 2024-05-09T01:01:46.306 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enable_auth", 2024-05-09T01:01:46.306 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.306 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.307 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.307 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.307 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.307 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.307 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.307 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.307 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.307 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.307 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.307 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.307 INFO:teuthology.orchestra.run.smithi005.stdout: "key_file": { 2024-05-09T01:01:46.307 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "key_file", 2024-05-09T01:01:46.307 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.308 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.308 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.308 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.308 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.308 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.308 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.308 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.308 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.308 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.308 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.308 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.308 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:46.309 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:46.309 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.309 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.309 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.309 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.309 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.310 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.310 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.310 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.310 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.310 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.310 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.311 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.311 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.311 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.311 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.311 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.311 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.312 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.312 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.312 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:46.312 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:46.312 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.313 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.313 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.313 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.313 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.313 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.313 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.313 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.313 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.314 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.314 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.314 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.314 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:46.314 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:46.314 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.314 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.314 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.314 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:46.314 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.314 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.314 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.314 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.315 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.315 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.315 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.315 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.315 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.315 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.315 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.315 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.315 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.315 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.315 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.315 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:46.315 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:46.316 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.316 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.316 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.316 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.316 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.316 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.316 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.316 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.316 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.316 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.316 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.316 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.316 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-05-09T01:01:46.317 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-05-09T01:01:46.317 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.317 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.317 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.317 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.317 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.317 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.317 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.317 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.317 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.317 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.317 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.317 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.317 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-05-09T01:01:46.318 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-05-09T01:01:46.318 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.318 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.318 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.318 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.318 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.318 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.318 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.318 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.318 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.318 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.318 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.318 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.319 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:46.319 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:46.319 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.319 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:46.319 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.319 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.319 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.319 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.319 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.319 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.319 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.319 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.319 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.319 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.320 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.320 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.320 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:46.320 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rgw", 2024-05-09T01:01:46.320 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:46.320 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:46.320 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:46.320 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:46.320 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:46.320 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.320 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.320 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.320 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.320 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.321 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.321 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.321 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.321 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.321 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.321 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.321 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.321 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.321 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.321 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.321 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.321 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.321 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.321 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.322 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:46.322 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:46.322 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.322 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.322 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.322 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.322 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.322 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.322 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.322 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.323 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.323 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.323 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.323 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.323 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:46.323 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:46.323 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.323 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.323 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.323 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:46.324 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.324 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.324 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.324 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.324 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.324 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.324 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.324 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.324 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.324 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.324 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.324 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.324 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.324 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.325 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.325 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:46.325 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:46.325 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.325 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.325 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.325 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.325 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.325 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.325 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.326 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.326 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.326 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.326 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.326 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.326 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:46.326 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:46.327 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.327 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:46.327 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.327 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.327 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.327 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.327 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.327 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.327 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.328 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.328 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.328 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.328 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.328 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.328 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:46.328 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rook", 2024-05-09T01:01:46.328 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:46.328 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:46.328 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:46.328 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:46.328 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:46.328 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.329 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.329 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.329 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.329 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.329 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.329 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.329 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.329 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.329 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.329 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.330 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.330 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.330 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.330 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.330 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.330 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.330 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.330 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.330 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:46.330 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:46.330 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.330 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.331 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.331 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.331 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.331 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.331 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.331 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.331 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.331 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.331 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.331 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.331 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:46.331 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:46.332 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.332 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.332 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.332 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:46.332 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.332 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.332 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.332 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.332 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.332 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.333 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.333 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.333 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.333 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.333 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.333 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.333 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.333 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.333 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.333 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:46.333 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:46.333 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.333 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.334 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.334 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.334 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.334 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.334 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.334 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.334 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.334 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.334 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.334 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.334 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:46.334 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:46.334 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.335 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:46.335 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.335 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.335 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.335 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.335 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.335 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.335 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.335 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.335 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.335 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.335 INFO:teuthology.orchestra.run.smithi005.stdout: "storage_class": { 2024-05-09T01:01:46.335 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "storage_class", 2024-05-09T01:01:46.336 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.336 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.336 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.336 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "local", 2024-05-09T01:01:46.336 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.336 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.336 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.336 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-09T01:01:46.336 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.336 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.337 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.337 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.337 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.337 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.337 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:46.337 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "selftest", 2024-05-09T01:01:46.337 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:46.337 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:46.337 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:46.337 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:46.337 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:46.337 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.338 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.338 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.338 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.338 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.338 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.338 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.338 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.338 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.338 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.338 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.338 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.338 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.339 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.339 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.339 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.339 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.339 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.339 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.339 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:46.340 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:46.340 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.340 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.340 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.340 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.340 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.340 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.340 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.340 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.340 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.340 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.340 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.340 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.340 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:46.341 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:46.341 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.341 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.341 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.341 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:46.341 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.341 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.341 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.341 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.341 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.341 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.341 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.341 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.342 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.342 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.342 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.342 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.342 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.342 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.342 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.342 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:46.342 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:46.342 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.342 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.343 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.343 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.343 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.343 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.343 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.343 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.343 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.343 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.343 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.343 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.343 INFO:teuthology.orchestra.run.smithi005.stdout: "roption1": { 2024-05-09T01:01:46.343 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "roption1", 2024-05-09T01:01:46.343 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.344 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.344 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.344 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.344 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.344 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.344 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.344 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.344 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.344 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.344 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.344 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.344 INFO:teuthology.orchestra.run.smithi005.stdout: "roption2": { 2024-05-09T01:01:46.344 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "roption2", 2024-05-09T01:01:46.344 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.345 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.345 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.345 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "xyz", 2024-05-09T01:01:46.345 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.345 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.345 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.345 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.345 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.345 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.345 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.345 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.345 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption1": { 2024-05-09T01:01:46.345 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption1", 2024-05-09T01:01:46.345 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.346 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.346 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.346 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.346 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.346 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.346 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.346 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.346 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.346 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.346 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.346 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.347 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption2": { 2024-05-09T01:01:46.347 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption2", 2024-05-09T01:01:46.347 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.347 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.347 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.347 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.347 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.347 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.347 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.347 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.347 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.348 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.348 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.348 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.348 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption3": { 2024-05-09T01:01:46.348 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption3", 2024-05-09T01:01:46.348 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-05-09T01:01:46.348 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.348 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.348 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.348 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.348 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.348 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.348 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.349 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.349 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.349 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.349 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.349 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption4": { 2024-05-09T01:01:46.349 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption4", 2024-05-09T01:01:46.349 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.349 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.349 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.349 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.349 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.349 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.349 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.349 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.350 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.350 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.350 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.350 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.350 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption5": { 2024-05-09T01:01:46.350 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption5", 2024-05-09T01:01:46.350 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.350 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.350 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.350 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.350 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.350 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.350 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.351 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.351 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.351 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.351 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.351 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.351 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption6": { 2024-05-09T01:01:46.351 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption6", 2024-05-09T01:01:46.351 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.351 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.351 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.351 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.351 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.351 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.351 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.352 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.352 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.352 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.352 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.352 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.352 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption7": { 2024-05-09T01:01:46.352 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption7", 2024-05-09T01:01:46.352 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.352 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.352 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.352 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.352 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-05-09T01:01:46.352 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "42", 2024-05-09T01:01:46.353 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.353 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.353 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.353 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.353 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.353 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.353 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:46.354 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:46.354 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.354 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:46.354 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.354 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.354 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.354 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.354 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.354 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.354 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.355 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.355 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.355 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.355 INFO:teuthology.orchestra.run.smithi005.stdout: "testkey": { 2024-05-09T01:01:46.355 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testkey", 2024-05-09T01:01:46.355 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.355 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.355 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.355 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.355 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.355 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.355 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.355 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.356 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.356 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.356 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.356 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.356 INFO:teuthology.orchestra.run.smithi005.stdout: "testlkey": { 2024-05-09T01:01:46.356 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testlkey", 2024-05-09T01:01:46.356 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.356 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.356 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.356 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.356 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.356 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.356 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.356 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.356 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.357 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.357 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.357 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.357 INFO:teuthology.orchestra.run.smithi005.stdout: "testnewline": { 2024-05-09T01:01:46.357 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testnewline", 2024-05-09T01:01:46.357 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.357 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.357 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.357 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.357 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.357 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.357 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.357 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.358 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.358 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.358 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.358 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.358 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.358 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.358 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:46.358 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smb", 2024-05-09T01:01:46.358 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:46.358 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:46.358 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:46.358 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:46.358 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:46.358 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.359 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.359 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.359 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.359 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.359 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.359 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.359 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.359 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.359 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.359 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.360 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.360 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.360 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.360 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.360 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.360 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.360 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.360 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.360 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:46.361 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:46.361 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.361 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.361 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.361 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.361 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.361 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.361 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.361 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.361 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.361 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.361 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.362 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.362 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:46.362 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:46.362 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.362 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.362 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.363 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:46.363 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.363 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.363 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.363 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.363 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.363 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.364 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.364 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.364 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.364 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.364 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.364 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.364 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.364 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.364 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.364 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:46.364 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:46.364 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.365 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.365 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.365 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.365 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.365 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.365 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.365 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.365 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.365 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.365 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.365 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.366 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:46.366 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:46.366 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.366 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:46.366 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.366 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.366 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.366 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.366 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.366 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.366 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.367 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.367 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.367 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.367 INFO:teuthology.orchestra.run.smithi005.stdout: "update_orchestration": { 2024-05-09T01:01:46.367 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "update_orchestration", 2024-05-09T01:01:46.367 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.368 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.368 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.368 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.368 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.368 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.368 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.369 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-09T01:01:46.369 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.369 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.369 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.369 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.369 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.370 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.370 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:46.370 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snap_schedule", 2024-05-09T01:01:46.370 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:46.370 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:46.370 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:46.371 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_m_granularity": { 2024-05-09T01:01:46.371 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_m_granularity", 2024-05-09T01:01:46.371 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.371 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.371 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.371 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.371 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.372 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.372 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.372 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow minute scheduled snapshots", 2024-05-09T01:01:46.372 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.372 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.373 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.373 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.373 INFO:teuthology.orchestra.run.smithi005.stdout: "dump_on_update": { 2024-05-09T01:01:46.373 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "dump_on_update", 2024-05-09T01:01:46.373 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.373 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.373 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.374 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.374 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.374 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.374 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.374 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "dump database to debug log on update", 2024-05-09T01:01:46.375 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.375 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.375 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.375 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.375 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:46.375 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:46.375 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.376 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.376 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.376 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.376 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.376 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.376 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.377 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.377 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.377 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.377 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.377 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.377 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.378 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.378 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.378 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.378 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.378 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.378 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.378 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:46.378 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:46.379 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.379 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.379 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.379 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.379 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.379 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.379 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.379 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.379 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.379 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.379 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.379 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.379 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:46.379 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:46.380 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.380 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.380 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.380 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:46.380 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.380 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.380 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.380 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.380 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.380 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.380 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.380 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.380 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.381 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.381 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.381 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.381 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.381 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.381 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.381 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:46.381 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:46.381 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.381 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.381 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.381 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.381 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.381 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.382 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.382 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.382 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.382 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.382 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.382 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.382 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:46.382 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:46.382 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.382 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:46.382 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.382 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.382 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.383 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.383 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.383 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.383 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.383 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.383 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.383 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.383 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.383 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.383 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:46.383 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "stats", 2024-05-09T01:01:46.383 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:46.383 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:46.383 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:46.384 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:46.384 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:46.384 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.384 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.384 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.384 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.384 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.384 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.384 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.384 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.384 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.384 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.384 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.384 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.385 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.385 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.385 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.385 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.385 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.385 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.385 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:46.385 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:46.385 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.385 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.385 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.385 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.385 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.386 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.386 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.386 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.386 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.386 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.386 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.386 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.386 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:46.386 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:46.386 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.386 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.386 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.386 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:46.387 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.387 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.387 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.387 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.387 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.387 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.387 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.387 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.387 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.387 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.387 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.387 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.387 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.387 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.388 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:46.388 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:46.388 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.388 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.388 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.388 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.388 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.388 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.388 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.388 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.388 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.388 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.388 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.389 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:46.389 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:46.389 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.389 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:46.389 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.389 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.389 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.389 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.389 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.389 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.389 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.389 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.389 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.389 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.390 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.390 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.390 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:46.390 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "status", 2024-05-09T01:01:46.390 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:46.390 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:46.390 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:46.390 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:46.390 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:46.390 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.390 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.390 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.390 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.391 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.391 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.391 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.391 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.391 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.391 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.391 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.391 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.391 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.391 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.391 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.391 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.391 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.391 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.391 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.392 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:46.392 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:46.392 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.392 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.392 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.392 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.392 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.392 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.392 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.392 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.392 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.392 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.392 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.392 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.393 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:46.393 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:46.393 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.393 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.393 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.393 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:46.393 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.393 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.393 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.393 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.393 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.393 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.393 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.393 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.394 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.394 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.394 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.394 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.394 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.394 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.394 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.394 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:46.394 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:46.394 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.394 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.394 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.394 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.395 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.395 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.395 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.395 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.395 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.395 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.395 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.395 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.395 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:46.395 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:46.395 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.395 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:46.395 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.396 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.396 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.396 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.396 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.396 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.396 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.396 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.396 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.396 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.396 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.396 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.396 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:46.396 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "telegraf", 2024-05-09T01:01:46.397 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:46.397 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:46.397 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:46.397 INFO:teuthology.orchestra.run.smithi005.stdout: "address": { 2024-05-09T01:01:46.397 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "address", 2024-05-09T01:01:46.397 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.397 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.397 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.397 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-09T01:01:46.397 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.397 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.397 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.397 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.398 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.398 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.398 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.398 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.398 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-05-09T01:01:46.398 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-05-09T01:01:46.398 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-09T01:01:46.398 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.398 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.398 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15", 2024-05-09T01:01:46.398 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.398 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.398 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.398 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.399 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.399 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.399 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.399 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.399 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:46.399 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:46.399 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.399 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.399 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.399 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.399 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.399 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.399 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.399 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.400 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.400 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.400 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.400 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.400 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.400 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.400 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.400 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.400 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.401 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.401 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.401 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:46.401 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:46.401 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.402 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.402 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.402 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.402 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.402 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.402 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.402 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.402 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.402 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.402 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.402 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.402 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:46.402 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:46.403 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.403 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.403 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.403 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:46.403 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.403 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.403 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.403 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.403 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.403 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.403 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.403 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.403 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.403 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.404 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.404 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.404 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.404 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.404 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.404 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:46.404 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:46.404 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.404 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.404 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.404 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.404 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.404 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.404 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.405 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.405 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.405 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.405 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.405 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.405 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:46.405 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:46.405 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.405 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:46.405 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.405 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.405 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.405 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.405 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.406 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.406 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.406 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.406 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.406 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.406 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.406 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.406 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:46.406 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "telemetry", 2024-05-09T01:01:46.406 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:46.406 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:46.406 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:46.406 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_basic": { 2024-05-09T01:01:46.406 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_basic", 2024-05-09T01:01:46.407 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.407 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.407 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.407 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.407 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.407 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.407 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.407 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-09T01:01:46.407 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.407 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.407 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.407 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.407 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_crash": { 2024-05-09T01:01:46.408 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_crash", 2024-05-09T01:01:46.408 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.408 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.408 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.408 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.408 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.408 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.408 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.408 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-09T01:01:46.408 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.408 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.408 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.408 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.408 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_device": { 2024-05-09T01:01:46.409 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_device", 2024-05-09T01:01:46.409 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.409 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.409 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.409 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.409 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.409 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.409 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.409 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-09T01:01:46.409 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.409 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.409 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.409 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.409 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_ident": { 2024-05-09T01:01:46.410 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_ident", 2024-05-09T01:01:46.410 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.410 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.410 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.410 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.410 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.410 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.410 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.410 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-09T01:01:46.410 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.410 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.410 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.410 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.411 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_perf": { 2024-05-09T01:01:46.411 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_perf", 2024-05-09T01:01:46.411 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.411 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.411 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.411 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.411 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.411 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.411 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.411 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-09T01:01:46.411 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.411 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.411 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.411 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.412 INFO:teuthology.orchestra.run.smithi005.stdout: "contact": { 2024-05-09T01:01:46.412 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "contact", 2024-05-09T01:01:46.412 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.412 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.412 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.412 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.412 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.412 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.412 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.412 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.412 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.412 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.412 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.412 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.413 INFO:teuthology.orchestra.run.smithi005.stdout: "description": { 2024-05-09T01:01:46.413 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "description", 2024-05-09T01:01:46.413 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.413 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.413 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.413 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.413 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.413 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.413 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.413 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.413 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.413 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.414 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.414 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.414 INFO:teuthology.orchestra.run.smithi005.stdout: "device_url": { 2024-05-09T01:01:46.414 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_url", 2024-05-09T01:01:46.414 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.414 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.414 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.414 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-09T01:01:46.414 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.414 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.414 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.414 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.414 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.415 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.415 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.415 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.415 INFO:teuthology.orchestra.run.smithi005.stdout: "enabled": { 2024-05-09T01:01:46.415 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enabled", 2024-05-09T01:01:46.415 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.415 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.415 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.415 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.415 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.415 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.415 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.415 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.415 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.416 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.416 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.416 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.416 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-05-09T01:01:46.416 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-05-09T01:01:46.416 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.416 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.416 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.416 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "24", 2024-05-09T01:01:46.416 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "8", 2024-05-09T01:01:46.416 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.416 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.416 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.416 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.417 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.417 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.417 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.417 INFO:teuthology.orchestra.run.smithi005.stdout: "last_opt_revision": { 2024-05-09T01:01:46.417 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "last_opt_revision", 2024-05-09T01:01:46.417 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.417 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.417 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.417 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1", 2024-05-09T01:01:46.417 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.417 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.417 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.417 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.417 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.418 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.418 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.418 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.418 INFO:teuthology.orchestra.run.smithi005.stdout: "leaderboard": { 2024-05-09T01:01:46.418 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "leaderboard", 2024-05-09T01:01:46.418 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.418 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.418 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.418 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.418 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.418 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.418 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.418 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.419 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.419 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.419 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.419 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.419 INFO:teuthology.orchestra.run.smithi005.stdout: "leaderboard_description": { 2024-05-09T01:01:46.419 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "leaderboard_description", 2024-05-09T01:01:46.419 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.419 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.419 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.419 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.419 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.419 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.419 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.419 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.420 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.420 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.420 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.420 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.420 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:46.420 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:46.420 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.420 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.420 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.420 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.420 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.420 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.420 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.420 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.421 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.421 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.421 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.421 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.421 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.421 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.421 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.421 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.421 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.421 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.421 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.421 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:46.421 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:46.422 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.422 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.422 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.422 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.422 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.422 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.422 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.422 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.422 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.422 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.422 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.422 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.422 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:46.422 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:46.423 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.423 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.423 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.423 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:46.423 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.423 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.423 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.423 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.423 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.423 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.423 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.423 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.423 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.423 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.424 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.424 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.424 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.424 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.424 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.424 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:46.424 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:46.424 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.424 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.424 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.424 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.424 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.424 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.425 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.425 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.425 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.425 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.425 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.425 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.425 INFO:teuthology.orchestra.run.smithi005.stdout: "organization": { 2024-05-09T01:01:46.425 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "organization", 2024-05-09T01:01:46.425 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.425 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.425 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.425 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.425 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.425 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.426 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.426 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.426 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.426 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.426 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.426 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.426 INFO:teuthology.orchestra.run.smithi005.stdout: "proxy": { 2024-05-09T01:01:46.426 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "proxy", 2024-05-09T01:01:46.426 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.426 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.426 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.426 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.426 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.427 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.427 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.427 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.427 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.427 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.427 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.427 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.427 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:46.427 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:46.427 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.427 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:46.427 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.427 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.427 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.428 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.428 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.428 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.428 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.428 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.428 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.428 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.428 INFO:teuthology.orchestra.run.smithi005.stdout: "url": { 2024-05-09T01:01:46.428 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "url", 2024-05-09T01:01:46.428 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.428 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.428 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.428 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-09T01:01:46.429 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.429 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.429 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.429 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.429 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.429 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.429 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.429 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.429 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.429 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.429 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:46.429 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "test_orchestrator", 2024-05-09T01:01:46.429 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:46.429 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:46.430 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:46.430 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:46.430 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:46.430 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.430 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.430 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.430 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.430 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.430 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.430 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.430 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.430 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.430 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.430 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.431 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.431 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.431 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.431 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.431 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.431 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.431 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.431 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.431 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:46.431 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:46.431 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.431 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.431 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.431 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.432 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.432 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.432 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.432 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.432 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.432 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.432 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.432 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.432 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:46.432 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:46.432 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.432 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.432 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.432 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:46.433 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.433 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.433 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.433 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.433 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.433 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.433 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.433 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.433 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.433 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.433 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.433 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.433 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.433 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.434 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.434 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:46.434 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:46.434 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.434 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.434 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.434 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.434 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.434 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.434 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.434 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.434 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.434 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.434 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.435 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.435 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:46.435 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:46.435 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.435 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:46.435 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.435 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.435 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.435 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.435 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.435 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.435 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.435 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.435 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.436 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.436 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.436 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.436 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:46.436 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "volumes", 2024-05-09T01:01:46.436 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:46.436 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:46.436 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:46.436 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:46.436 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:46.436 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.436 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.436 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.437 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.437 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.437 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.437 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.437 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.437 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.437 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.437 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.437 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.437 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.437 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.437 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.437 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.437 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.438 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.438 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.438 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:46.438 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:46.438 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.438 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.438 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.438 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.438 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.438 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.438 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.438 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.438 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.438 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.439 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.439 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.439 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:46.439 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:46.439 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.439 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.439 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.439 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:46.439 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.439 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.439 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.439 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.439 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.440 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.440 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.440 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.440 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.440 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.440 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.440 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.440 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.440 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.440 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.440 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:46.440 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:46.440 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.441 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.441 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.441 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.441 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.441 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.441 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.441 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.441 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.441 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.441 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.441 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.441 INFO:teuthology.orchestra.run.smithi005.stdout: "max_concurrent_clones": { 2024-05-09T01:01:46.441 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_concurrent_clones", 2024-05-09T01:01:46.442 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.442 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.442 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.442 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "4", 2024-05-09T01:01:46.442 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.442 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.442 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.442 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-09T01:01:46.442 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.442 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.442 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.442 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.442 INFO:teuthology.orchestra.run.smithi005.stdout: "periodic_async_work": { 2024-05-09T01:01:46.443 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "periodic_async_work", 2024-05-09T01:01:46.443 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.443 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.443 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.443 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.443 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.443 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.443 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.443 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Periodically check for async work", 2024-05-09T01:01:46.443 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.443 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.443 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.443 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.444 INFO:teuthology.orchestra.run.smithi005.stdout: "snapshot_clone_delay": { 2024-05-09T01:01:46.444 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snapshot_clone_delay", 2024-05-09T01:01:46.444 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.444 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.444 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.444 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.444 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.444 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.444 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.444 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-09T01:01:46.444 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.444 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.444 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.444 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.445 INFO:teuthology.orchestra.run.smithi005.stdout: "snapshot_clone_no_wait": { 2024-05-09T01:01:46.445 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snapshot_clone_no_wait", 2024-05-09T01:01:46.445 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.445 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.445 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.445 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.445 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.445 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.445 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.445 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-09T01:01:46.445 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.445 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.445 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.446 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.446 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:46.446 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:46.446 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.446 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:46.446 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.446 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.446 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.446 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.446 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.446 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.446 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.446 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.446 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.447 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.447 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.447 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.447 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:46.447 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix", 2024-05-09T01:01:46.447 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:46.447 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:46.447 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:46.447 INFO:teuthology.orchestra.run.smithi005.stdout: "discovery_interval": { 2024-05-09T01:01:46.447 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "discovery_interval", 2024-05-09T01:01:46.447 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-05-09T01:01:46.447 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.447 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.448 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "100", 2024-05-09T01:01:46.448 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.448 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.448 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.448 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.448 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.448 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.448 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.448 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.448 INFO:teuthology.orchestra.run.smithi005.stdout: "identifier": { 2024-05-09T01:01:46.448 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "identifier", 2024-05-09T01:01:46.448 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.448 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.448 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.449 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.449 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.449 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.449 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.449 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.449 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.449 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.449 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.449 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.449 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-05-09T01:01:46.449 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-05-09T01:01:46.449 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-09T01:01:46.449 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.449 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.450 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-05-09T01:01:46.450 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.450 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.450 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.450 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.450 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.450 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.450 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.450 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.450 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:46.450 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:46.450 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.450 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.451 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.451 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.451 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.451 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.451 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.451 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.451 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.451 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.451 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.451 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.451 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.451 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.451 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.451 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.452 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.452 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.452 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.452 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:46.452 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:46.452 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.452 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.452 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.452 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.452 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.452 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.452 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.452 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.452 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.453 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.453 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.453 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.453 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:46.453 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:46.453 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.453 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.453 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.453 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:46.453 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.453 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.453 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.453 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.454 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.454 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.454 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.454 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.454 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.454 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.454 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.454 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.454 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.454 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.454 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.454 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:46.454 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:46.455 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.455 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.455 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.455 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.455 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.455 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.455 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.455 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.455 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.455 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.455 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.455 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.455 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:46.455 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:46.456 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.456 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:46.456 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.456 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.456 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.456 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.456 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.456 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.456 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.456 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.456 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.456 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.456 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_host": { 2024-05-09T01:01:46.456 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_host", 2024-05-09T01:01:46.457 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.457 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.457 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.457 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.457 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.457 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.457 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.457 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.457 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.457 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.457 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.457 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.457 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_port": { 2024-05-09T01:01:46.457 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_port", 2024-05-09T01:01:46.458 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.458 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.458 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.458 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10051", 2024-05-09T01:01:46.458 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.458 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.458 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.458 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.458 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.458 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.458 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.458 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.458 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_sender": { 2024-05-09T01:01:46.459 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_sender", 2024-05-09T01:01:46.459 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.459 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.459 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.459 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-09T01:01:46.459 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.459 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.459 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.459 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.459 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.459 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.459 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.459 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.459 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.460 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.460 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-09T01:01:46.460 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.460 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.460 INFO:teuthology.orchestra.run.smithi005.stdout: "modules": [ 2024-05-09T01:01:46.460 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm", 2024-05-09T01:01:46.460 INFO:teuthology.orchestra.run.smithi005.stdout: "dashboard", 2024-05-09T01:01:46.460 INFO:teuthology.orchestra.run.smithi005.stdout: "iostat", 2024-05-09T01:01:46.460 INFO:teuthology.orchestra.run.smithi005.stdout: "nfs", 2024-05-09T01:01:46.460 INFO:teuthology.orchestra.run.smithi005.stdout: "restful" 2024-05-09T01:01:46.460 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.460 INFO:teuthology.orchestra.run.smithi005.stdout: "available_modules": [ 2024-05-09T01:01:46.460 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:46.460 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "alerts", 2024-05-09T01:01:46.461 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:46.461 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:46.461 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:46.461 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-05-09T01:01:46.461 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-05-09T01:01:46.461 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-09T01:01:46.461 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.461 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.461 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-05-09T01:01:46.461 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.461 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.461 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.461 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How frequently to reexamine health status", 2024-05-09T01:01:46.462 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.462 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.462 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.462 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.462 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:46.462 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:46.462 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.462 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.462 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.462 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.462 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.462 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.462 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.462 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.463 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.463 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.463 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.463 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.463 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.463 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.463 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.463 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.463 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.463 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.463 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.463 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:46.463 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:46.463 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.464 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.464 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.464 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.464 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.464 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.464 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.464 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.464 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.464 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.464 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.464 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.464 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:46.464 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:46.465 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.465 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.465 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.465 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:46.465 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.465 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.465 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.465 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.465 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.465 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.465 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.465 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.465 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.465 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.466 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.466 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.466 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.466 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.466 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.466 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:46.466 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:46.466 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.466 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.466 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.466 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.466 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.466 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.467 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.467 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.467 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.467 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.467 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.467 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.467 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_destination": { 2024-05-09T01:01:46.467 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_destination", 2024-05-09T01:01:46.467 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.467 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.467 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.467 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.467 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.467 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.468 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.468 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Email address to send alerts to", 2024-05-09T01:01:46.468 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.468 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.468 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.468 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.468 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_from_name": { 2024-05-09T01:01:46.468 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_from_name", 2024-05-09T01:01:46.468 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.468 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.468 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.468 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "Ceph", 2024-05-09T01:01:46.468 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.468 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.468 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.469 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Email From: name", 2024-05-09T01:01:46.469 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.469 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.469 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.469 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.469 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_host": { 2024-05-09T01:01:46.469 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_host", 2024-05-09T01:01:46.469 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.469 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.469 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.469 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.469 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.469 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.470 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.470 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP server", 2024-05-09T01:01:46.470 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.470 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.470 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.470 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.470 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_password": { 2024-05-09T01:01:46.470 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_password", 2024-05-09T01:01:46.470 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.470 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.470 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.470 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.470 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.470 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.471 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.471 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Password to authenticate with", 2024-05-09T01:01:46.471 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.471 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.471 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.471 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.471 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_port": { 2024-05-09T01:01:46.471 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_port", 2024-05-09T01:01:46.471 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.471 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.471 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.471 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "465", 2024-05-09T01:01:46.471 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.471 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.472 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.472 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP port", 2024-05-09T01:01:46.472 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.472 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.472 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.472 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.472 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_sender": { 2024-05-09T01:01:46.472 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_sender", 2024-05-09T01:01:46.472 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.472 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.472 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.472 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.472 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.472 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.473 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.473 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP envelope sender", 2024-05-09T01:01:46.473 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.473 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.473 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.473 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.473 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_ssl": { 2024-05-09T01:01:46.473 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_ssl", 2024-05-09T01:01:46.473 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.473 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.473 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.473 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.473 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.474 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.474 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.474 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-09T01:01:46.474 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.474 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.474 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.474 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.474 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_user": { 2024-05-09T01:01:46.474 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_user", 2024-05-09T01:01:46.474 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.474 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.474 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.474 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.474 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.475 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.475 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.475 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "User to authenticate as", 2024-05-09T01:01:46.475 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.475 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.475 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.475 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.475 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:46.475 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:46.475 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.475 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:46.475 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.475 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.476 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.476 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.476 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.476 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.476 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.476 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.476 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.476 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.476 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.476 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.476 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:46.476 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "balancer", 2024-05-09T01:01:46.476 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:46.476 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:46.477 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:46.477 INFO:teuthology.orchestra.run.smithi005.stdout: "active": { 2024-05-09T01:01:46.477 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "active", 2024-05-09T01:01:46.477 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.477 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.477 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.477 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.477 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.477 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.477 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.477 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically balance PGs across cluster", 2024-05-09T01:01:46.477 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.477 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.477 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.478 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.478 INFO:teuthology.orchestra.run.smithi005.stdout: "begin_time": { 2024-05-09T01:01:46.478 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "begin_time", 2024-05-09T01:01:46.478 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.478 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.478 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.478 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0000", 2024-05-09T01:01:46.478 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.478 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.478 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.478 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "beginning time of day to automatically balance", 2024-05-09T01:01:46.478 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T01:01:46.478 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.478 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.479 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.479 INFO:teuthology.orchestra.run.smithi005.stdout: "begin_weekday": { 2024-05-09T01:01:46.479 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "begin_weekday", 2024-05-09T01:01:46.479 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-05-09T01:01:46.479 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.479 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.479 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.479 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0", 2024-05-09T01:01:46.479 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "6", 2024-05-09T01:01:46.479 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.479 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-09T01:01:46.479 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T01:01:46.479 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.480 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.480 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.480 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_max_iterations": { 2024-05-09T01:01:46.480 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_max_iterations", 2024-05-09T01:01:46.480 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-05-09T01:01:46.480 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.480 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.480 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "25", 2024-05-09T01:01:46.480 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-05-09T01:01:46.480 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "250", 2024-05-09T01:01:46.480 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.480 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-09T01:01:46.480 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.480 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.481 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.481 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.481 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_metrics": { 2024-05-09T01:01:46.481 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_metrics", 2024-05-09T01:01:46.481 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.481 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.481 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.481 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "pgs,objects,bytes", 2024-05-09T01:01:46.481 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.481 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.481 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.481 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-09T01:01:46.481 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-09T01:01:46.481 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.482 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.482 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.482 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_step": { 2024-05-09T01:01:46.482 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_step", 2024-05-09T01:01:46.482 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-05-09T01:01:46.482 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.482 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.482 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0.5", 2024-05-09T01:01:46.482 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0.001", 2024-05-09T01:01:46.482 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "0.999", 2024-05-09T01:01:46.482 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.482 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "aggressiveness of optimization", 2024-05-09T01:01:46.482 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-09T01:01:46.482 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.483 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.483 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.483 INFO:teuthology.orchestra.run.smithi005.stdout: "end_time": { 2024-05-09T01:01:46.483 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "end_time", 2024-05-09T01:01:46.483 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.483 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.483 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.483 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "2359", 2024-05-09T01:01:46.483 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.483 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.483 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.483 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "ending time of day to automatically balance", 2024-05-09T01:01:46.483 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T01:01:46.483 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.484 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.484 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.484 INFO:teuthology.orchestra.run.smithi005.stdout: "end_weekday": { 2024-05-09T01:01:46.484 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "end_weekday", 2024-05-09T01:01:46.484 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-05-09T01:01:46.484 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.484 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.484 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.484 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0", 2024-05-09T01:01:46.484 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "6", 2024-05-09T01:01:46.484 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.484 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-09T01:01:46.484 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T01:01:46.485 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.485 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.485 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.485 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:46.485 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:46.485 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.485 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.485 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.485 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.485 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.485 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.485 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.485 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.485 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.486 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.486 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.486 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.486 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.486 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.486 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.486 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.486 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.486 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.486 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.486 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:46.486 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:46.486 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.487 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.487 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.487 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.487 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.487 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.487 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.487 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.487 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.487 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.487 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.487 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.487 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:46.487 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:46.487 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.488 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.488 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.488 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:46.488 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.488 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.488 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.488 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.488 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.488 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.488 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.488 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.488 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.488 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.488 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.489 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.489 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.489 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.489 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.489 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:46.489 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:46.489 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.489 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.489 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.489 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.489 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.489 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.489 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.489 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.490 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.490 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.490 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.490 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.490 INFO:teuthology.orchestra.run.smithi005.stdout: "min_score": { 2024-05-09T01:01:46.490 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "min_score", 2024-05-09T01:01:46.490 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-05-09T01:01:46.490 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.490 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.490 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.490 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.490 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.490 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.490 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-09T01:01:46.491 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.491 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.491 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.491 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.491 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": { 2024-05-09T01:01:46.491 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mode", 2024-05-09T01:01:46.491 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.491 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.491 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.491 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "upmap", 2024-05-09T01:01:46.491 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.491 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.491 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.492 INFO:teuthology.orchestra.run.smithi005.stdout: "crush-compat", 2024-05-09T01:01:46.492 INFO:teuthology.orchestra.run.smithi005.stdout: "none", 2024-05-09T01:01:46.492 INFO:teuthology.orchestra.run.smithi005.stdout: "read", 2024-05-09T01:01:46.492 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap", 2024-05-09T01:01:46.492 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap-read" 2024-05-09T01:01:46.492 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.492 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Balancer mode", 2024-05-09T01:01:46.492 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.492 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.492 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.492 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.492 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_ids": { 2024-05-09T01:01:46.492 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pool_ids", 2024-05-09T01:01:46.492 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.493 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.493 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.493 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.493 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.493 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.493 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.493 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-09T01:01:46.493 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.493 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.493 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.493 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.493 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-05-09T01:01:46.493 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-05-09T01:01:46.493 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-09T01:01:46.494 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.494 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.494 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-05-09T01:01:46.494 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.494 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.494 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.494 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-09T01:01:46.494 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.494 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.494 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.494 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.494 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:46.494 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:46.494 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.495 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:46.495 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.495 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.495 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.495 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.495 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.495 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.495 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.495 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.495 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.495 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.495 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap_max_deviation": { 2024-05-09T01:01:46.495 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "upmap_max_deviation", 2024-05-09T01:01:46.495 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.496 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.496 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.496 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-05-09T01:01:46.496 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-05-09T01:01:46.496 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.496 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.496 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-09T01:01:46.496 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-09T01:01:46.496 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.496 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.496 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.496 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap_max_optimizations": { 2024-05-09T01:01:46.496 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "upmap_max_optimizations", 2024-05-09T01:01:46.496 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-05-09T01:01:46.497 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.497 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.497 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-05-09T01:01:46.497 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.497 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.497 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.497 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-09T01:01:46.497 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.497 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.497 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.497 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.497 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.497 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.497 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:46.497 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cephadm", 2024-05-09T01:01:46.498 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:46.498 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:46.498 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:46.498 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_down_multiplier": { 2024-05-09T01:01:46.498 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_down_multiplier", 2024-05-09T01:01:46.498 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-05-09T01:01:46.498 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.498 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.498 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3.0", 2024-05-09T01:01:46.498 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.499 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.499 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.499 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-09T01:01:46.499 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.499 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.499 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.499 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.499 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_refresh_rate": { 2024-05-09T01:01:46.499 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_refresh_rate", 2024-05-09T01:01:46.499 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-09T01:01:46.499 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.500 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.500 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "20", 2024-05-09T01:01:46.500 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.500 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.500 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.500 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-09T01:01:46.500 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.500 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.500 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.500 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.500 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_starting_port": { 2024-05-09T01:01:46.501 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_starting_port", 2024-05-09T01:01:46.501 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.501 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.501 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.501 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "4721", 2024-05-09T01:01:46.501 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.501 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.501 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.501 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-09T01:01:46.501 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.501 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.502 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.502 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.502 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_ptrace": { 2024-05-09T01:01:46.502 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_ptrace", 2024-05-09T01:01:46.502 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.502 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.502 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.502 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.502 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.502 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.502 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.503 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-09T01:01:46.503 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-09T01:01:46.503 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.503 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.503 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.503 INFO:teuthology.orchestra.run.smithi005.stdout: "autotune_interval": { 2024-05-09T01:01:46.503 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "autotune_interval", 2024-05-09T01:01:46.503 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-09T01:01:46.503 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.503 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.503 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-05-09T01:01:46.504 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.504 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.504 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.504 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-09T01:01:46.504 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.504 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.504 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.504 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.504 INFO:teuthology.orchestra.run.smithi005.stdout: "autotune_memory_target_ratio": { 2024-05-09T01:01:46.504 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "autotune_memory_target_ratio", 2024-05-09T01:01:46.504 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-05-09T01:01:46.504 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.505 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.505 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0.7", 2024-05-09T01:01:46.505 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.505 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.505 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.505 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-09T01:01:46.505 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.505 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.505 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.505 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.505 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm_log_destination": { 2024-05-09T01:01:46.505 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cephadm_log_destination", 2024-05-09T01:01:46.506 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.506 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.506 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.506 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.506 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.506 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.506 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.506 INFO:teuthology.orchestra.run.smithi005.stdout: "file", 2024-05-09T01:01:46.506 INFO:teuthology.orchestra.run.smithi005.stdout: "file,syslog", 2024-05-09T01:01:46.506 INFO:teuthology.orchestra.run.smithi005.stdout: "syslog" 2024-05-09T01:01:46.506 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.507 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-09T01:01:46.507 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.507 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.507 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.507 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.507 INFO:teuthology.orchestra.run.smithi005.stdout: "cgroups_split": { 2024-05-09T01:01:46.507 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cgroups_split", 2024-05-09T01:01:46.507 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.507 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.507 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.507 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.507 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.508 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.508 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.508 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-09T01:01:46.508 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.508 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.508 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.508 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.508 INFO:teuthology.orchestra.run.smithi005.stdout: "config_checks_enabled": { 2024-05-09T01:01:46.508 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_checks_enabled", 2024-05-09T01:01:46.508 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.508 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.509 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.509 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.509 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.509 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.509 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.509 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-09T01:01:46.509 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.509 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.509 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.509 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.509 INFO:teuthology.orchestra.run.smithi005.stdout: "config_dashboard": { 2024-05-09T01:01:46.509 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_dashboard", 2024-05-09T01:01:46.510 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.510 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.510 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.510 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.510 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.510 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.510 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.510 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-09T01:01:46.510 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.510 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.510 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.511 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.511 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_alertmanager": { 2024-05-09T01:01:46.511 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_alertmanager", 2024-05-09T01:01:46.511 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.511 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.511 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.511 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-09T01:01:46.511 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.511 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.511 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.511 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-05-09T01:01:46.511 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.512 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.512 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.512 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.512 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_base": { 2024-05-09T01:01:46.512 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_base", 2024-05-09T01:01:46.512 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.512 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.512 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.512 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-09T01:01:46.512 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.512 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.513 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.513 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Container image name, without the tag", 2024-05-09T01:01:46.513 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.513 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.513 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.513 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.513 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_elasticsearch": { 2024-05-09T01:01:46.513 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_elasticsearch", 2024-05-09T01:01:46.513 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.513 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.513 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.513 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-09T01:01:46.514 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.514 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.514 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.514 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "elasticsearch container image", 2024-05-09T01:01:46.514 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.514 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.514 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.514 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.514 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_grafana": { 2024-05-09T01:01:46.514 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_grafana", 2024-05-09T01:01:46.514 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.515 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.515 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.515 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-09T01:01:46.515 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.515 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.515 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.515 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-05-09T01:01:46.515 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.515 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.515 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.515 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.515 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_haproxy": { 2024-05-09T01:01:46.516 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_haproxy", 2024-05-09T01:01:46.516 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.516 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.516 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.516 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-09T01:01:46.516 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.516 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.516 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.516 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "HAproxy container image", 2024-05-09T01:01:46.516 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.516 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.517 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.517 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.517 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_agent": { 2024-05-09T01:01:46.517 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_agent", 2024-05-09T01:01:46.517 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.517 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.517 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.517 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-09T01:01:46.517 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.517 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.517 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.517 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger agent container image", 2024-05-09T01:01:46.517 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.518 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.518 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.518 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.518 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_collector": { 2024-05-09T01:01:46.518 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_collector", 2024-05-09T01:01:46.518 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.518 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.518 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.518 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-09T01:01:46.518 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.518 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.518 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.519 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger collector container image", 2024-05-09T01:01:46.519 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.519 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.519 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.519 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.519 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_query": { 2024-05-09T01:01:46.519 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_query", 2024-05-09T01:01:46.519 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.519 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.519 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.519 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-09T01:01:46.519 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.520 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.520 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.520 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger query container image", 2024-05-09T01:01:46.520 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.520 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.520 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.520 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.520 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_keepalived": { 2024-05-09T01:01:46.520 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_keepalived", 2024-05-09T01:01:46.520 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.520 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.521 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.521 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-09T01:01:46.521 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.521 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.521 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.521 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Keepalived container image", 2024-05-09T01:01:46.521 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.521 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.521 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.521 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.521 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_loki": { 2024-05-09T01:01:46.522 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_loki", 2024-05-09T01:01:46.522 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.522 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.522 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.522 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-09T01:01:46.522 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.522 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.522 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.522 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Loki container image", 2024-05-09T01:01:46.522 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.522 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.522 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.523 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.523 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_node_exporter": { 2024-05-09T01:01:46.523 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_node_exporter", 2024-05-09T01:01:46.523 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.523 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.523 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.523 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-09T01:01:46.523 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.523 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.523 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.523 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-05-09T01:01:46.523 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.524 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.524 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.524 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.524 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_nvmeof": { 2024-05-09T01:01:46.524 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_nvmeof", 2024-05-09T01:01:46.524 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.524 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.524 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.524 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-09T01:01:46.524 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.524 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.524 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.525 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Nvme-of container image", 2024-05-09T01:01:46.525 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.525 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.525 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.525 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.525 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_prometheus": { 2024-05-09T01:01:46.525 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_prometheus", 2024-05-09T01:01:46.525 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.525 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.525 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.525 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-09T01:01:46.526 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.526 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.526 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.526 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-05-09T01:01:46.526 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.526 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.526 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.526 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.526 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_promtail": { 2024-05-09T01:01:46.526 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_promtail", 2024-05-09T01:01:46.526 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.527 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.527 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.527 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-09T01:01:46.527 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.527 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.527 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.527 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Promtail container image", 2024-05-09T01:01:46.527 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.527 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.527 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.527 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.527 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_samba": { 2024-05-09T01:01:46.527 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_samba", 2024-05-09T01:01:46.528 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.528 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.528 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.528 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-09T01:01:46.528 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.528 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.528 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.528 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Samba/SMB container image", 2024-05-09T01:01:46.528 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.528 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.528 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.528 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.529 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_snmp_gateway": { 2024-05-09T01:01:46.529 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_snmp_gateway", 2024-05-09T01:01:46.529 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.529 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.529 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.529 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-09T01:01:46.529 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.529 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.529 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.529 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SNMP Gateway container image", 2024-05-09T01:01:46.529 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.529 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.530 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.530 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.530 INFO:teuthology.orchestra.run.smithi005.stdout: "container_init": { 2024-05-09T01:01:46.530 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_init", 2024-05-09T01:01:46.530 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.530 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.530 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.530 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.530 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.530 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.530 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.530 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Run podman/docker with `--init`", 2024-05-09T01:01:46.531 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.531 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.531 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.531 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.531 INFO:teuthology.orchestra.run.smithi005.stdout: "daemon_cache_timeout": { 2024-05-09T01:01:46.531 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "daemon_cache_timeout", 2024-05-09T01:01:46.531 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-09T01:01:46.531 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.531 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.531 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-05-09T01:01:46.531 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.531 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.532 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.532 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-09T01:01:46.532 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.532 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.532 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.532 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.532 INFO:teuthology.orchestra.run.smithi005.stdout: "default_cephadm_command_timeout": { 2024-05-09T01:01:46.532 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "default_cephadm_command_timeout", 2024-05-09T01:01:46.532 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.532 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.532 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.532 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "900", 2024-05-09T01:01:46.533 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.533 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.533 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.533 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-09T01:01:46.533 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.533 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.533 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.533 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.533 INFO:teuthology.orchestra.run.smithi005.stdout: "default_registry": { 2024-05-09T01:01:46.533 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "default_registry", 2024-05-09T01:01:46.533 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.534 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.534 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.534 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io", 2024-05-09T01:01:46.534 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.534 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.534 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.534 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-09T01:01:46.534 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.534 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.534 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.534 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.534 INFO:teuthology.orchestra.run.smithi005.stdout: "device_cache_timeout": { 2024-05-09T01:01:46.534 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_cache_timeout", 2024-05-09T01:01:46.535 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-09T01:01:46.535 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.535 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.535 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1800", 2024-05-09T01:01:46.535 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.535 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.535 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.535 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache device inventory", 2024-05-09T01:01:46.535 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.535 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.535 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.535 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.535 INFO:teuthology.orchestra.run.smithi005.stdout: "device_enhanced_scan": { 2024-05-09T01:01:46.536 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_enhanced_scan", 2024-05-09T01:01:46.536 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.536 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.536 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.536 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.536 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.536 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.536 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.536 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-09T01:01:46.536 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.536 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.536 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.536 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.537 INFO:teuthology.orchestra.run.smithi005.stdout: "facts_cache_timeout": { 2024-05-09T01:01:46.537 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "facts_cache_timeout", 2024-05-09T01:01:46.537 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-09T01:01:46.537 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.537 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.537 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-05-09T01:01:46.537 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.537 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.537 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.537 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache host facts data", 2024-05-09T01:01:46.537 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.537 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.538 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.538 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.538 INFO:teuthology.orchestra.run.smithi005.stdout: "grafana_dashboards_path": { 2024-05-09T01:01:46.538 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "grafana_dashboards_path", 2024-05-09T01:01:46.538 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.538 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.538 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.538 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-09T01:01:46.538 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.538 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.538 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.539 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-09T01:01:46.539 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.539 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.539 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.539 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.539 INFO:teuthology.orchestra.run.smithi005.stdout: "host_check_interval": { 2024-05-09T01:01:46.539 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "host_check_interval", 2024-05-09T01:01:46.539 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-09T01:01:46.539 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.539 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.539 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-05-09T01:01:46.539 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.540 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.540 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.540 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to perform a host check", 2024-05-09T01:01:46.540 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.540 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.540 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.540 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.540 INFO:teuthology.orchestra.run.smithi005.stdout: "hw_monitoring": { 2024-05-09T01:01:46.540 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "hw_monitoring", 2024-05-09T01:01:46.540 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.540 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.541 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.541 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.541 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.541 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.541 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.541 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-09T01:01:46.541 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.541 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.541 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.541 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.541 INFO:teuthology.orchestra.run.smithi005.stdout: "inventory_list_all": { 2024-05-09T01:01:46.541 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "inventory_list_all", 2024-05-09T01:01:46.542 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.542 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.542 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.542 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.542 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.542 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.542 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.542 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-09T01:01:46.542 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.542 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.542 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.543 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.543 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:46.543 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:46.543 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.543 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.543 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.543 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.543 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.543 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.543 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.543 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.544 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.544 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.544 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.544 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.544 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.544 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.544 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.544 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.544 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.544 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.544 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.544 INFO:teuthology.orchestra.run.smithi005.stdout: "log_refresh_metadata": { 2024-05-09T01:01:46.545 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_refresh_metadata", 2024-05-09T01:01:46.545 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.545 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.545 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.545 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.545 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.545 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.545 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.545 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-09T01:01:46.545 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.545 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.545 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.546 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:46.546 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:46.546 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.546 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.546 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.546 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.546 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.546 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.546 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.546 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-09T01:01:46.546 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.547 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.547 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.547 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.547 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:46.547 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:46.547 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.547 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.547 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.547 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:46.547 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.547 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.547 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.547 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.548 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.548 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.548 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.548 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.548 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.548 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.548 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.548 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.548 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.548 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.548 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.548 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:46.548 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:46.549 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.549 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.549 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.549 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.549 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.549 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.549 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.549 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.549 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.549 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.549 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.549 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.549 INFO:teuthology.orchestra.run.smithi005.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-09T01:01:46.550 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-09T01:01:46.550 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.550 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.550 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.550 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.550 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.550 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.550 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.550 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-09T01:01:46.550 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.551 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.551 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.551 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.551 INFO:teuthology.orchestra.run.smithi005.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-09T01:01:46.551 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-09T01:01:46.551 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.551 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.551 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.551 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "*", 2024-05-09T01:01:46.551 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.551 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.551 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.551 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-09T01:01:46.552 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.552 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.552 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.552 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.552 INFO:teuthology.orchestra.run.smithi005.stdout: "max_count_per_host": { 2024-05-09T01:01:46.552 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_count_per_host", 2024-05-09T01:01:46.552 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.552 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.552 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.552 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-05-09T01:01:46.552 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.552 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.552 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.553 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "max number of daemons per service per host", 2024-05-09T01:01:46.553 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.553 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.553 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.553 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.553 INFO:teuthology.orchestra.run.smithi005.stdout: "max_osd_draining_count": { 2024-05-09T01:01:46.553 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_osd_draining_count", 2024-05-09T01:01:46.553 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.553 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.553 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.554 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-05-09T01:01:46.554 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.554 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.554 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.554 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-09T01:01:46.554 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.554 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.554 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.554 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.554 INFO:teuthology.orchestra.run.smithi005.stdout: "migration_current": { 2024-05-09T01:01:46.554 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "migration_current", 2024-05-09T01:01:46.555 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.555 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.555 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.555 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.555 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.555 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.555 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.555 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "internal - do not modify", 2024-05-09T01:01:46.555 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.555 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.555 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.556 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.556 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": { 2024-05-09T01:01:46.556 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mode", 2024-05-09T01:01:46.556 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.556 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.556 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.556 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "root", 2024-05-09T01:01:46.556 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.556 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.556 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.556 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm-package", 2024-05-09T01:01:46.556 INFO:teuthology.orchestra.run.smithi005.stdout: "root" 2024-05-09T01:01:46.557 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.557 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "mode for remote execution of cephadm", 2024-05-09T01:01:46.557 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.557 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.557 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.557 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.557 INFO:teuthology.orchestra.run.smithi005.stdout: "oob_default_addr": { 2024-05-09T01:01:46.557 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "oob_default_addr", 2024-05-09T01:01:46.557 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.557 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.557 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.557 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "169.254.1.1", 2024-05-09T01:01:46.558 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.558 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.558 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.558 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-09T01:01:46.558 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.558 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.558 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.558 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.558 INFO:teuthology.orchestra.run.smithi005.stdout: "prometheus_alerts_path": { 2024-05-09T01:01:46.558 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus_alerts_path", 2024-05-09T01:01:46.558 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.558 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.559 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.559 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-09T01:01:46.559 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.559 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.559 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.559 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-09T01:01:46.559 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.559 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.559 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.559 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.559 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_insecure": { 2024-05-09T01:01:46.560 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_insecure", 2024-05-09T01:01:46.560 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.560 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.560 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.560 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.560 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.560 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.560 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.560 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-09T01:01:46.560 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.560 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.561 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.561 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_password": { 2024-05-09T01:01:46.561 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_password", 2024-05-09T01:01:46.561 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.561 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.561 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.561 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.561 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.561 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.561 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.561 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-09T01:01:46.561 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.562 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.562 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.562 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.562 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_url": { 2024-05-09T01:01:46.562 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_url", 2024-05-09T01:01:46.562 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.562 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.562 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.562 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.562 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.562 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.562 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.563 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-09T01:01:46.563 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.563 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.563 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.563 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.563 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_username": { 2024-05-09T01:01:46.563 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_username", 2024-05-09T01:01:46.563 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.563 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.563 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.563 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.563 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.564 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.564 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.564 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-09T01:01:46.564 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.564 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.564 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.564 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.564 INFO:teuthology.orchestra.run.smithi005.stdout: "secure_monitoring_stack": { 2024-05-09T01:01:46.564 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "secure_monitoring_stack", 2024-05-09T01:01:46.564 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.564 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.565 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.565 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.565 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.565 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.565 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.565 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-09T01:01:46.565 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.565 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.565 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.565 INFO:teuthology.orchestra.run.smithi005.stdout: "service_discovery_port": { 2024-05-09T01:01:46.565 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "service_discovery_port", 2024-05-09T01:01:46.566 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.566 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.566 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.566 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8765", 2024-05-09T01:01:46.566 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.566 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.566 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.566 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "cephadm service discovery port", 2024-05-09T01:01:46.566 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.566 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.566 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.567 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.567 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:46.567 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:46.567 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.567 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:46.567 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.567 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.567 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.567 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.567 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.567 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.567 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.568 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.568 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.568 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.568 INFO:teuthology.orchestra.run.smithi005.stdout: "ssh_config_file": { 2024-05-09T01:01:46.568 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssh_config_file", 2024-05-09T01:01:46.568 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.568 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.568 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.568 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.568 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.568 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.568 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.569 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-09T01:01:46.569 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.569 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.569 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.569 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.569 INFO:teuthology.orchestra.run.smithi005.stdout: "use_agent": { 2024-05-09T01:01:46.569 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "use_agent", 2024-05-09T01:01:46.569 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.569 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.569 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.569 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.569 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.570 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.570 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.570 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-09T01:01:46.570 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.570 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.570 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.570 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.570 INFO:teuthology.orchestra.run.smithi005.stdout: "use_repo_digest": { 2024-05-09T01:01:46.570 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "use_repo_digest", 2024-05-09T01:01:46.570 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.571 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.571 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.571 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.571 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.571 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.571 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.571 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-09T01:01:46.571 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.571 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.571 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.571 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.572 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_failed_host_check": { 2024-05-09T01:01:46.572 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_failed_host_check", 2024-05-09T01:01:46.572 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.572 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.572 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.572 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.572 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.572 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.572 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.572 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if the host check fails", 2024-05-09T01:01:46.572 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.572 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.573 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.573 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.573 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_stray_daemons": { 2024-05-09T01:01:46.573 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_stray_daemons", 2024-05-09T01:01:46.573 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.573 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.573 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.573 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.573 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.573 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.573 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.574 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-09T01:01:46.574 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.574 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.574 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.574 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.574 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_stray_hosts": { 2024-05-09T01:01:46.574 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_stray_hosts", 2024-05-09T01:01:46.574 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.574 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.574 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.574 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.575 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.575 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.575 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.575 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-09T01:01:46.575 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.575 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.575 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.575 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.575 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.575 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.575 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:46.575 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crash", 2024-05-09T01:01:46.576 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:46.576 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:46.576 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:46.576 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:46.576 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:46.576 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.576 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.576 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.576 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.576 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.576 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.576 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.577 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.577 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.577 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.577 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.577 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.577 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.577 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.577 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.577 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.577 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.577 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.578 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:46.578 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:46.578 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.578 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.578 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.578 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.578 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.578 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.578 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.578 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.578 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.579 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.579 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.579 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:46.579 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:46.579 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.579 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.579 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.579 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:46.579 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.579 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.579 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.580 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.580 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.580 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.580 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.580 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.580 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.580 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.580 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.580 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.580 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.580 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.580 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:46.581 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:46.581 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.581 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.581 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.581 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.581 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.581 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.581 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.581 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.581 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.581 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.582 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.582 INFO:teuthology.orchestra.run.smithi005.stdout: "retain_interval": { 2024-05-09T01:01:46.582 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "retain_interval", 2024-05-09T01:01:46.582 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-09T01:01:46.582 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.582 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.582 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "31536000", 2024-05-09T01:01:46.582 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.582 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.582 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.583 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-09T01:01:46.583 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.583 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.583 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.583 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:46.583 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:46.583 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.583 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:46.583 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.583 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.584 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.584 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.584 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.584 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.584 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.584 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.584 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.584 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_recent_interval": { 2024-05-09T01:01:46.584 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_recent_interval", 2024-05-09T01:01:46.584 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-09T01:01:46.584 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.585 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.585 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1209600", 2024-05-09T01:01:46.585 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.585 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.585 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.585 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-09T01:01:46.585 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.585 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.585 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.585 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.585 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:46.586 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "dashboard", 2024-05-09T01:01:46.586 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:46.586 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:46.586 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:46.586 INFO:teuthology.orchestra.run.smithi005.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-09T01:01:46.586 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-09T01:01:46.586 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.586 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.586 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.586 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-05-09T01:01:46.587 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.587 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.587 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.587 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.587 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.587 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.587 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.587 INFO:teuthology.orchestra.run.smithi005.stdout: "ALERTMANAGER_API_HOST": { 2024-05-09T01:01:46.587 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-09T01:01:46.587 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.587 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.588 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.588 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.588 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.588 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.588 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.588 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.588 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.588 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.588 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.588 INFO:teuthology.orchestra.run.smithi005.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-09T01:01:46.588 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-09T01:01:46.589 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.589 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.589 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.589 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.589 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.589 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.589 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.589 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.589 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.589 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.589 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.590 INFO:teuthology.orchestra.run.smithi005.stdout: "AUDIT_API_ENABLED": { 2024-05-09T01:01:46.590 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "AUDIT_API_ENABLED", 2024-05-09T01:01:46.590 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.590 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.590 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.590 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.590 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.590 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.590 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.590 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.591 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.591 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.591 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.591 INFO:teuthology.orchestra.run.smithi005.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-09T01:01:46.591 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-09T01:01:46.591 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.591 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.591 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.591 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.592 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.592 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.592 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.592 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.592 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.592 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.592 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.592 INFO:teuthology.orchestra.run.smithi005.stdout: "ENABLE_BROWSABLE_API": { 2024-05-09T01:01:46.592 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-09T01:01:46.592 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.592 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.592 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.593 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.593 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.593 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.593 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.593 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.593 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.593 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.593 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.593 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-09T01:01:46.593 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-09T01:01:46.594 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.594 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.594 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.594 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.594 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.594 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.594 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.594 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.594 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.594 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.594 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.594 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.595 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-09T01:01:46.595 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-09T01:01:46.595 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.595 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.595 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.595 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.595 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.595 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.595 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.595 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.595 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.596 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.596 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.596 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-09T01:01:46.596 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-09T01:01:46.596 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.596 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.596 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.596 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.596 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.596 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.596 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.596 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.597 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.597 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.597 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.597 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-09T01:01:46.597 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-09T01:01:46.597 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.597 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.597 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.597 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.597 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.598 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.598 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.598 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.598 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.598 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.598 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.598 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-09T01:01:46.598 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-09T01:01:46.598 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.598 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.599 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.599 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.599 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.599 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.599 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.599 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.599 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.599 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.599 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.599 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.599 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-09T01:01:46.599 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-09T01:01:46.600 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.600 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.600 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.600 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.600 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.600 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.600 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.600 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.600 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.600 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.600 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.601 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-09T01:01:46.601 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-09T01:01:46.601 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.601 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.601 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.601 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.601 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.601 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.601 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.601 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.601 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.601 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.602 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.602 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.602 INFO:teuthology.orchestra.run.smithi005.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-09T01:01:46.602 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-09T01:01:46.602 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.602 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.602 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.602 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.602 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.602 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.602 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.603 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.603 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.603 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.603 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.603 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.603 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_PASSWORD": { 2024-05-09T01:01:46.603 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-09T01:01:46.603 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.603 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.603 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.603 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-05-09T01:01:46.603 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.603 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.604 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.604 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.604 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.604 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.604 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.604 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-09T01:01:46.604 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-09T01:01:46.604 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.604 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.604 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.604 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.605 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.605 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.605 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.605 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.605 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.605 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.605 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.605 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.605 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_URL": { 2024-05-09T01:01:46.605 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_URL", 2024-05-09T01:01:46.605 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.606 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.606 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.606 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.606 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.606 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.606 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.606 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.606 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.606 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.606 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.606 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.606 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_USERNAME": { 2024-05-09T01:01:46.607 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-09T01:01:46.607 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.607 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.607 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.607 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-05-09T01:01:46.607 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.607 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.607 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.607 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.607 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.607 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.607 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.608 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-09T01:01:46.608 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-09T01:01:46.608 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.608 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.608 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.608 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.608 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.608 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.608 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.608 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.609 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.609 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.609 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.609 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-09T01:01:46.609 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-09T01:01:46.609 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.609 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.609 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.609 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.610 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.610 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.610 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.610 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.610 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.610 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.610 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.610 INFO:teuthology.orchestra.run.smithi005.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-09T01:01:46.610 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-09T01:01:46.610 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.611 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.611 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.611 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.611 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.611 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.611 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.611 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.611 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.611 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.611 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.611 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.611 INFO:teuthology.orchestra.run.smithi005.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-09T01:01:46.612 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-09T01:01:46.612 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.612 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.612 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.612 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.612 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.612 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.612 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.612 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.612 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.612 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.612 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.613 INFO:teuthology.orchestra.run.smithi005.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-09T01:01:46.613 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-09T01:01:46.613 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.613 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.613 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.613 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "[]", 2024-05-09T01:01:46.613 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.613 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.613 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.613 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.614 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.614 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.614 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.614 INFO:teuthology.orchestra.run.smithi005.stdout: "MULTICLUSTER_CONFIG": { 2024-05-09T01:01:46.614 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-09T01:01:46.614 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.614 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.614 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.614 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "{}", 2024-05-09T01:01:46.615 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.615 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.615 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.615 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.615 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.615 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.615 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.615 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.615 INFO:teuthology.orchestra.run.smithi005.stdout: "PROMETHEUS_API_HOST": { 2024-05-09T01:01:46.615 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-09T01:01:46.615 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.616 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.616 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.616 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.616 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.616 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.616 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.616 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.616 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.616 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.616 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.616 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.616 INFO:teuthology.orchestra.run.smithi005.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-09T01:01:46.617 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-09T01:01:46.617 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.617 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.617 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.617 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.617 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.617 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.617 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.617 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.617 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.617 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.617 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.618 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-09T01:01:46.618 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-09T01:01:46.618 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.618 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.618 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.618 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.618 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.618 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.618 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.618 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.619 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.619 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.619 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.619 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.619 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-09T01:01:46.619 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-09T01:01:46.619 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.619 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.619 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.619 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.619 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.620 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.620 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.620 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.620 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.620 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.620 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.620 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-09T01:01:46.620 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-09T01:01:46.620 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.620 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.620 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.621 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.621 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.621 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.621 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.621 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.621 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.621 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.621 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.621 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.621 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-09T01:01:46.621 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-09T01:01:46.621 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.622 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.622 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.622 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.622 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.622 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.622 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.622 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.622 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.622 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.622 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.622 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.623 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-09T01:01:46.623 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-09T01:01:46.623 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.623 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.623 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.623 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.623 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.623 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.623 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.623 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.623 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.623 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.623 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.624 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.624 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-09T01:01:46.624 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-09T01:01:46.624 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.624 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.624 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.624 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.624 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.624 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.624 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.624 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.624 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.624 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.625 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.625 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.625 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-09T01:01:46.625 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-09T01:01:46.625 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.625 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.625 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.625 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.625 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.625 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.625 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.626 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.626 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.626 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.626 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.626 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.626 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_ENABLED": { 2024-05-09T01:01:46.626 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-09T01:01:46.626 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.626 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.626 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.626 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.627 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.627 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.627 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.627 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.627 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.627 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.627 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.627 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.627 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-09T01:01:46.627 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-09T01:01:46.627 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.627 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.628 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.628 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-09T01:01:46.628 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.628 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.628 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.628 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.628 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.628 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.628 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.628 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-09T01:01:46.628 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-09T01:01:46.629 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.629 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.629 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.629 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-05-09T01:01:46.629 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.629 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.629 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.629 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.629 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.630 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.630 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.630 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.630 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-09T01:01:46.630 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-09T01:01:46.630 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.631 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.631 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.631 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8", 2024-05-09T01:01:46.631 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.631 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.631 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.631 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.632 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.632 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.632 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.632 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.632 INFO:teuthology.orchestra.run.smithi005.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-09T01:01:46.632 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-09T01:01:46.632 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.633 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.633 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.633 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "45", 2024-05-09T01:01:46.633 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.633 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.633 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.634 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.634 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.634 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.634 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.634 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.634 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_ACCESS_KEY": { 2024-05-09T01:01:46.635 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-09T01:01:46.635 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.635 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.635 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.635 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.635 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.635 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.636 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.636 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.636 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.636 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.636 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.636 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.636 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-09T01:01:46.637 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-09T01:01:46.637 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.637 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.637 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.637 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-05-09T01:01:46.637 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.638 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.638 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.638 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.638 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.638 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.638 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.639 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.639 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_SECRET_KEY": { 2024-05-09T01:01:46.639 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-09T01:01:46.639 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.639 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.639 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.639 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.639 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.639 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.639 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.640 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.640 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.640 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.640 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.640 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.640 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_SSL_VERIFY": { 2024-05-09T01:01:46.640 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-09T01:01:46.640 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.640 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.640 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.640 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.641 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.641 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.641 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.641 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.641 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.641 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.641 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.641 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.641 INFO:teuthology.orchestra.run.smithi005.stdout: "UNSAFE_TLS_v1_2": { 2024-05-09T01:01:46.641 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-09T01:01:46.641 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.642 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.642 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.642 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.642 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.642 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.642 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.642 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.642 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.642 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.642 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.642 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.642 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-09T01:01:46.643 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-09T01:01:46.643 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.643 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.643 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.643 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.643 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.643 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.643 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.643 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.643 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.644 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.644 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.644 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.644 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-09T01:01:46.644 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-09T01:01:46.644 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.644 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.644 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.644 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-05-09T01:01:46.644 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.644 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.644 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.645 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.645 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.645 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.645 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.645 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.645 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-09T01:01:46.645 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-09T01:01:46.645 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.645 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.645 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.645 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-05-09T01:01:46.645 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.646 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.646 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.646 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.646 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.646 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.646 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.646 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.646 INFO:teuthology.orchestra.run.smithi005.stdout: "cross_origin_url": { 2024-05-09T01:01:46.646 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cross_origin_url", 2024-05-09T01:01:46.646 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.646 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.646 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.647 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.647 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.647 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.647 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.647 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.647 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.647 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.647 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.647 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.647 INFO:teuthology.orchestra.run.smithi005.stdout: "crt_file": { 2024-05-09T01:01:46.647 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crt_file", 2024-05-09T01:01:46.647 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.648 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.648 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.648 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.648 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.648 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.648 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.648 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.648 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.648 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.648 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.648 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.648 INFO:teuthology.orchestra.run.smithi005.stdout: "debug": { 2024-05-09T01:01:46.649 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "debug", 2024-05-09T01:01:46.649 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.649 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.649 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.649 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.649 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.649 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.649 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.649 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable/disable debug options", 2024-05-09T01:01:46.649 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.649 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.650 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.650 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.650 INFO:teuthology.orchestra.run.smithi005.stdout: "jwt_token_ttl": { 2024-05-09T01:01:46.650 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "jwt_token_ttl", 2024-05-09T01:01:46.650 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.650 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.650 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.650 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "28800", 2024-05-09T01:01:46.650 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.650 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.650 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.650 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.651 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.651 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.651 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.651 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.651 INFO:teuthology.orchestra.run.smithi005.stdout: "key_file": { 2024-05-09T01:01:46.651 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "key_file", 2024-05-09T01:01:46.651 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.651 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.651 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.651 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.651 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.651 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.652 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.652 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.652 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.652 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.652 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.652 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.652 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:46.652 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:46.652 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.652 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.652 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.652 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.652 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.653 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.653 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.653 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.653 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.653 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.653 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.653 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.653 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.653 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.653 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.653 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.653 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.654 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.654 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.654 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:46.654 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:46.654 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.654 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.654 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.654 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.654 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.654 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.654 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.655 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.655 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.655 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.655 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.655 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.655 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:46.655 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:46.655 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.655 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.655 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.655 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:46.656 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.656 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.656 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.656 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.656 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.656 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.656 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.656 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.656 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.656 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.656 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.656 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.657 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.657 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.657 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.657 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:46.657 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:46.657 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.657 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.657 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.657 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.657 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.657 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.658 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.658 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.658 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.658 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.658 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.658 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.658 INFO:teuthology.orchestra.run.smithi005.stdout: "motd": { 2024-05-09T01:01:46.658 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "motd", 2024-05-09T01:01:46.658 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.658 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.658 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.659 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.659 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.659 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.659 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.659 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "The message of the day", 2024-05-09T01:01:46.659 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.659 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.659 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.659 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.659 INFO:teuthology.orchestra.run.smithi005.stdout: "redirect_resolve_ip_addr": { 2024-05-09T01:01:46.659 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "redirect_resolve_ip_addr", 2024-05-09T01:01:46.660 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.660 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.660 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.660 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.660 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.660 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.660 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.660 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.660 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.660 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.660 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.661 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.661 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-05-09T01:01:46.661 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-05-09T01:01:46.661 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.661 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.661 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.661 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "::", 2024-05-09T01:01:46.661 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.661 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.661 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.661 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.662 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.662 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.662 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.662 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.662 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-05-09T01:01:46.662 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-05-09T01:01:46.662 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.662 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.662 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.662 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8080", 2024-05-09T01:01:46.662 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.662 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.663 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.663 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.663 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.663 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.663 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.663 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.663 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:46.663 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:46.663 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.663 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:46.663 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.664 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.664 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.664 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.664 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.664 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.664 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.664 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.664 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.664 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.664 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl": { 2024-05-09T01:01:46.664 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl", 2024-05-09T01:01:46.664 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.664 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.665 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.665 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.665 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.665 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.665 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.665 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.665 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.665 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.665 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.665 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.665 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl_server_port": { 2024-05-09T01:01:46.665 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl_server_port", 2024-05-09T01:01:46.666 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.666 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.666 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.666 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8443", 2024-05-09T01:01:46.666 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.666 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.666 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.666 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.666 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.666 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.666 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.666 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.667 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_behaviour": { 2024-05-09T01:01:46.667 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_behaviour", 2024-05-09T01:01:46.667 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.667 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.667 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.667 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "redirect", 2024-05-09T01:01:46.667 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.667 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.667 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.667 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.667 INFO:teuthology.orchestra.run.smithi005.stdout: "redirect" 2024-05-09T01:01:46.667 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.668 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.668 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.668 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.668 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.668 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.668 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_error_status_code": { 2024-05-09T01:01:46.668 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_error_status_code", 2024-05-09T01:01:46.668 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.668 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.668 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.668 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "500", 2024-05-09T01:01:46.668 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "400", 2024-05-09T01:01:46.668 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "599", 2024-05-09T01:01:46.669 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.669 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.669 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.669 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.669 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.669 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.669 INFO:teuthology.orchestra.run.smithi005.stdout: "url_prefix": { 2024-05-09T01:01:46.669 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "url_prefix", 2024-05-09T01:01:46.669 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.669 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.669 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.670 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.670 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.670 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.670 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.670 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.670 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.670 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.670 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.670 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.670 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.670 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.670 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:46.671 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "devicehealth", 2024-05-09T01:01:46.671 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:46.671 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:46.671 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:46.671 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_monitoring": { 2024-05-09T01:01:46.671 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enable_monitoring", 2024-05-09T01:01:46.671 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.671 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.671 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.671 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.671 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.671 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.672 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.672 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "monitor device health metrics", 2024-05-09T01:01:46.672 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.672 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.672 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.672 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.672 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:46.672 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:46.672 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.672 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.672 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.672 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.672 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.673 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.673 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.673 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.673 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.673 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.673 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.673 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.673 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.673 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.673 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.673 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.673 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.673 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.674 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.674 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:46.674 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:46.674 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.674 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.674 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.674 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.674 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.674 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.674 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.674 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.675 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.675 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.675 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.675 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.675 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:46.675 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:46.675 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.675 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.675 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.675 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:46.675 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.675 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.676 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.676 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.676 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.676 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.676 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.676 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.676 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.676 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.676 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.676 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.676 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.676 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.677 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.677 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:46.677 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:46.677 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.677 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.677 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.677 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.677 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.677 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.677 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.677 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.677 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.678 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.678 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.678 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.678 INFO:teuthology.orchestra.run.smithi005.stdout: "mark_out_threshold": { 2024-05-09T01:01:46.678 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mark_out_threshold", 2024-05-09T01:01:46.678 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-09T01:01:46.678 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.678 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.678 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "2419200", 2024-05-09T01:01:46.678 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.678 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.678 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.679 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-09T01:01:46.679 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.679 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.679 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.679 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.679 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": { 2024-05-09T01:01:46.679 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pool_name", 2024-05-09T01:01:46.679 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.679 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.679 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.679 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "device_health_metrics", 2024-05-09T01:01:46.679 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.680 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.680 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.680 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-09T01:01:46.680 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.680 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.680 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.680 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.680 INFO:teuthology.orchestra.run.smithi005.stdout: "retention_period": { 2024-05-09T01:01:46.680 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "retention_period", 2024-05-09T01:01:46.680 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-09T01:01:46.680 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.680 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.681 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15552000", 2024-05-09T01:01:46.681 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.681 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.681 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.681 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long to retain device health metrics", 2024-05-09T01:01:46.681 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.681 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.681 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.681 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.681 INFO:teuthology.orchestra.run.smithi005.stdout: "scrape_frequency": { 2024-05-09T01:01:46.681 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "scrape_frequency", 2024-05-09T01:01:46.682 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-09T01:01:46.682 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.682 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.682 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "86400", 2024-05-09T01:01:46.682 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.682 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.682 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.682 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-09T01:01:46.682 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.682 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.683 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.683 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.683 INFO:teuthology.orchestra.run.smithi005.stdout: "self_heal": { 2024-05-09T01:01:46.683 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "self_heal", 2024-05-09T01:01:46.683 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.683 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.683 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.683 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.683 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.683 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.683 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.683 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-09T01:01:46.684 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.684 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.684 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.684 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.684 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-05-09T01:01:46.684 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-05-09T01:01:46.684 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-09T01:01:46.684 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.684 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.684 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-05-09T01:01:46.684 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.684 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.685 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.685 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to wake up and check device health", 2024-05-09T01:01:46.685 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.685 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.685 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.685 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.685 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:46.685 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:46.685 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.685 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:46.685 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.685 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.686 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.686 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.686 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.686 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.686 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.686 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.686 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.686 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.686 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_threshold": { 2024-05-09T01:01:46.686 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_threshold", 2024-05-09T01:01:46.686 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-09T01:01:46.686 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.687 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.687 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "7257600", 2024-05-09T01:01:46.687 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.687 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.687 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.687 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-09T01:01:46.687 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.687 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.687 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.687 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.687 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.687 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.688 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:46.688 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "diskprediction_local", 2024-05-09T01:01:46.688 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:46.688 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:46.688 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:46.688 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:46.688 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:46.688 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.688 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.688 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.688 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.688 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.689 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.689 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.689 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.689 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.689 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.689 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.689 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.689 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.689 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.689 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.689 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.689 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.690 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.690 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.690 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:46.690 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:46.690 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.690 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.690 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.690 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.690 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.690 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.690 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.690 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.691 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.691 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.691 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.691 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.691 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:46.691 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:46.691 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.691 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.691 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.691 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:46.691 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.691 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.692 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.692 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.692 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.692 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.692 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.692 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.692 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.692 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.692 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.692 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.692 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.692 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.692 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.693 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:46.693 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:46.693 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.693 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.693 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.693 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.693 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.693 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.693 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.693 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.693 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.694 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.694 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.694 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.694 INFO:teuthology.orchestra.run.smithi005.stdout: "predict_interval": { 2024-05-09T01:01:46.694 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "predict_interval", 2024-05-09T01:01:46.694 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.694 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.694 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.694 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "86400", 2024-05-09T01:01:46.694 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.694 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.694 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.695 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.695 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.695 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.695 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.695 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.695 INFO:teuthology.orchestra.run.smithi005.stdout: "predictor_model": { 2024-05-09T01:01:46.695 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "predictor_model", 2024-05-09T01:01:46.695 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.695 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.695 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.695 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "prophetstor", 2024-05-09T01:01:46.696 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.696 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.696 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.696 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.696 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.696 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.696 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.696 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.696 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-05-09T01:01:46.696 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-05-09T01:01:46.696 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.696 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.697 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.697 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-05-09T01:01:46.697 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.697 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.697 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.697 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.697 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.697 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.697 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.697 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.697 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:46.697 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:46.698 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.698 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:46.698 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.698 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.698 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.698 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.698 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.698 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.698 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.698 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.698 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.699 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.699 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.699 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.699 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:46.699 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "influx", 2024-05-09T01:01:46.699 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": false, 2024-05-09T01:01:46.699 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "influxdb python module not found", 2024-05-09T01:01:46.699 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:46.699 INFO:teuthology.orchestra.run.smithi005.stdout: "batch_size": { 2024-05-09T01:01:46.699 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "batch_size", 2024-05-09T01:01:46.699 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.699 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.700 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.700 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5000", 2024-05-09T01:01:46.700 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.700 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.700 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.700 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-09T01:01:46.700 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.700 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.700 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.700 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.700 INFO:teuthology.orchestra.run.smithi005.stdout: "database": { 2024-05-09T01:01:46.701 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "database", 2024-05-09T01:01:46.701 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.701 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.701 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.701 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "ceph", 2024-05-09T01:01:46.701 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.701 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.701 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.701 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-09T01:01:46.701 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.701 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.701 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.702 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.702 INFO:teuthology.orchestra.run.smithi005.stdout: "hostname": { 2024-05-09T01:01:46.702 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "hostname", 2024-05-09T01:01:46.702 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.702 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.702 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.702 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.702 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.702 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.702 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.703 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB server hostname", 2024-05-09T01:01:46.703 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.703 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.703 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.703 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.703 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-05-09T01:01:46.703 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-05-09T01:01:46.703 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-09T01:01:46.703 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.703 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.703 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "30", 2024-05-09T01:01:46.703 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "5", 2024-05-09T01:01:46.703 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.704 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.704 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-09T01:01:46.704 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.704 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.704 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.704 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.704 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:46.704 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:46.704 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.704 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.704 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.705 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.705 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.705 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.705 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.705 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.705 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.705 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.705 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.705 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.705 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.705 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.705 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.706 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.706 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.706 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.706 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.706 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:46.706 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:46.706 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.706 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.706 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.706 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.706 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.706 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.707 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.707 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.707 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.707 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.707 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.707 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.707 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:46.707 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:46.707 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.707 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.707 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.707 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:46.708 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.708 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.708 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.708 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.708 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.708 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.708 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.708 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.708 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.708 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.708 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.709 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.709 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.709 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.709 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.709 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:46.709 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:46.709 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.709 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.709 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.709 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.709 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.709 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.710 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.710 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.710 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.710 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.710 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.710 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.710 INFO:teuthology.orchestra.run.smithi005.stdout: "password": { 2024-05-09T01:01:46.710 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "password", 2024-05-09T01:01:46.710 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.710 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.710 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.711 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.711 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.711 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.711 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.711 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "password of InfluxDB server user", 2024-05-09T01:01:46.711 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.711 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.711 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.711 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.711 INFO:teuthology.orchestra.run.smithi005.stdout: "port": { 2024-05-09T01:01:46.711 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "port", 2024-05-09T01:01:46.711 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.711 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.712 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.712 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8086", 2024-05-09T01:01:46.712 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.712 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.712 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.712 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB server port", 2024-05-09T01:01:46.712 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.712 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.712 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.712 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.712 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:46.712 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:46.713 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.713 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:46.713 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.713 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.713 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.713 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.713 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.713 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.713 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.713 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.713 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.713 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.713 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl": { 2024-05-09T01:01:46.714 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl", 2024-05-09T01:01:46.714 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.714 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.714 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.714 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "false", 2024-05-09T01:01:46.714 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.714 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.714 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.714 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T01:01:46.714 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.714 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.714 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.715 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.715 INFO:teuthology.orchestra.run.smithi005.stdout: "threads": { 2024-05-09T01:01:46.715 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "threads", 2024-05-09T01:01:46.715 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.715 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.715 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.715 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-05-09T01:01:46.715 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-05-09T01:01:46.715 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "32", 2024-05-09T01:01:46.715 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.715 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-09T01:01:46.715 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.715 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.716 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.716 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.716 INFO:teuthology.orchestra.run.smithi005.stdout: "username": { 2024-05-09T01:01:46.716 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "username", 2024-05-09T01:01:46.716 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.716 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.716 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.716 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.716 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.716 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.716 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.716 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "username of InfluxDB server user", 2024-05-09T01:01:46.717 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.717 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.717 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.717 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.717 INFO:teuthology.orchestra.run.smithi005.stdout: "verify_ssl": { 2024-05-09T01:01:46.717 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "verify_ssl", 2024-05-09T01:01:46.717 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.717 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.717 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.717 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "true", 2024-05-09T01:01:46.717 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.717 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.717 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.718 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T01:01:46.718 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.718 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.718 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.718 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.718 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.718 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.718 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:46.718 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "insights", 2024-05-09T01:01:46.718 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:46.718 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:46.718 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:46.718 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:46.719 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:46.719 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.719 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.719 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.719 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.719 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.719 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.719 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.719 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.719 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.719 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.719 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.720 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.720 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.720 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.720 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.720 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.720 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.720 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.720 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.720 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:46.720 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:46.720 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.720 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.721 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.721 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.721 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.721 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.721 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.721 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.721 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.721 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.721 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.721 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.721 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:46.721 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:46.721 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.722 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.722 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.722 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:46.722 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.722 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.722 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.722 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.722 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.722 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.722 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.722 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.723 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.723 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.723 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.723 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.723 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.723 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.723 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.723 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:46.723 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:46.723 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.723 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.723 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.723 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.723 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.724 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.724 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.724 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.724 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.724 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.724 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.724 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.724 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:46.724 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:46.724 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.724 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:46.724 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.724 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.725 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.725 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.725 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.725 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.725 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.725 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.725 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.725 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.725 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.725 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.725 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:46.725 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "iostat", 2024-05-09T01:01:46.726 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:46.726 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:46.726 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:46.726 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:46.726 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:46.726 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.726 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.726 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.726 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.726 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.726 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.726 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.727 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.727 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.727 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.727 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.727 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.727 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.727 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.727 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.727 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.727 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.727 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.727 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.728 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:46.728 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:46.728 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.728 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.728 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.728 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.728 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.728 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.728 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.728 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.728 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.728 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.729 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.729 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.729 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:46.729 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:46.729 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.729 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.729 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.729 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:46.729 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.729 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.729 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.730 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.730 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.730 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.730 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.730 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.730 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.730 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.730 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.730 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.730 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.730 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.730 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.731 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:46.731 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:46.731 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.731 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.731 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.731 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.731 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.731 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.731 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.731 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.731 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.731 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.732 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.732 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.732 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:46.732 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:46.732 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.732 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:46.732 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.732 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.732 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.732 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.732 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.733 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.733 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.733 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.733 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.733 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.733 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.733 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.733 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:46.733 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "k8sevents", 2024-05-09T01:01:46.733 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:46.733 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:46.733 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:46.734 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_event_retention_days": { 2024-05-09T01:01:46.734 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ceph_event_retention_days", 2024-05-09T01:01:46.734 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.734 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.734 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.734 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "7", 2024-05-09T01:01:46.734 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.734 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.734 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.734 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-09T01:01:46.734 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.735 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.735 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.735 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.735 INFO:teuthology.orchestra.run.smithi005.stdout: "config_check_secs": { 2024-05-09T01:01:46.735 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_check_secs", 2024-05-09T01:01:46.735 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.735 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.735 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.735 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-05-09T01:01:46.735 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "10", 2024-05-09T01:01:46.735 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.736 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.736 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-09T01:01:46.736 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.736 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.736 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.736 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.736 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:46.736 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:46.736 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.736 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.736 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.736 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.737 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.737 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.737 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.737 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.737 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.737 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.737 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.737 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.737 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.737 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.737 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.737 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.738 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.738 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.738 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.738 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:46.738 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:46.738 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.738 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.738 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.738 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.738 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.738 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.738 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.739 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.739 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.739 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.739 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.739 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.739 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:46.739 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:46.739 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.739 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.739 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.739 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:46.740 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.740 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.740 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.740 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.740 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.740 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.740 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.740 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.740 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.740 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.740 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.740 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.741 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.741 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.741 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.741 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:46.741 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:46.741 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.741 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.741 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.741 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.741 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.741 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.742 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.742 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.742 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.742 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.742 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.742 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.742 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:46.742 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:46.742 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.742 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:46.742 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.742 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.743 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.743 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.743 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.743 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.743 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.743 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.743 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.743 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.743 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.743 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.743 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:46.743 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "localpool", 2024-05-09T01:01:46.743 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:46.744 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:46.744 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:46.744 INFO:teuthology.orchestra.run.smithi005.stdout: "failure_domain": { 2024-05-09T01:01:46.744 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "failure_domain", 2024-05-09T01:01:46.744 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.744 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.744 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.744 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "host", 2024-05-09T01:01:46.744 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.744 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.744 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.744 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "failure domain for any created local pool", 2024-05-09T01:01:46.745 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-09T01:01:46.745 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.745 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.745 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.745 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:46.745 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:46.745 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.745 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.745 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.745 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.745 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.745 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.745 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.746 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.746 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.746 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.746 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.746 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.746 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.746 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.746 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.746 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.746 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.746 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.746 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.747 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:46.747 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:46.747 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.747 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.747 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.747 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.747 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.747 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.747 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.747 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.747 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.747 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.748 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.748 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.748 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:46.748 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:46.748 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.748 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.748 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.748 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:46.748 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.748 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.748 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.748 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.748 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.749 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.749 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.749 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.749 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.749 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.749 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.749 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.749 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.749 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.749 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.749 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:46.749 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:46.750 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.750 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.750 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.750 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.750 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.750 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.750 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.750 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.750 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.750 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.750 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.750 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.750 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": { 2024-05-09T01:01:46.751 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "min_size", 2024-05-09T01:01:46.751 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.751 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.751 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.751 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.751 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.751 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.751 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.751 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default min_size for any created local pool", 2024-05-09T01:01:46.751 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-09T01:01:46.751 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.751 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.752 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.752 INFO:teuthology.orchestra.run.smithi005.stdout: "num_rep": { 2024-05-09T01:01:46.752 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "num_rep", 2024-05-09T01:01:46.752 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.752 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.752 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.752 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3", 2024-05-09T01:01:46.752 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.752 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.752 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.752 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default replica count for any created local pool", 2024-05-09T01:01:46.752 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.753 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.753 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.753 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.753 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": { 2024-05-09T01:01:46.753 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pg_num", 2024-05-09T01:01:46.753 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.753 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.753 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.753 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "128", 2024-05-09T01:01:46.753 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.753 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.753 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.754 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default pg_num for any created local pool", 2024-05-09T01:01:46.754 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.754 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.754 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.754 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.754 INFO:teuthology.orchestra.run.smithi005.stdout: "prefix": { 2024-05-09T01:01:46.754 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prefix", 2024-05-09T01:01:46.754 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.754 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.754 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.754 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.754 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.755 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.755 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.755 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "name prefix for any created local pool", 2024-05-09T01:01:46.755 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.755 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.755 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.755 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.755 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:46.755 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:46.755 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.755 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:46.756 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.756 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.756 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.756 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.756 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.756 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.756 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.756 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.756 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.756 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.756 INFO:teuthology.orchestra.run.smithi005.stdout: "subtree": { 2024-05-09T01:01:46.756 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "subtree", 2024-05-09T01:01:46.756 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.757 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.757 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.757 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "rack", 2024-05-09T01:01:46.757 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.757 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.757 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.757 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-09T01:01:46.757 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-09T01:01:46.757 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.757 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.757 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.758 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.758 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.758 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:46.758 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mds_autoscaler", 2024-05-09T01:01:46.758 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:46.758 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:46.758 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:46.758 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:46.758 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:46.758 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.758 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.759 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.759 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.759 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.759 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.759 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.759 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.759 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.759 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.759 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.759 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.759 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.759 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.759 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.760 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.760 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.760 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.760 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.760 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:46.760 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:46.760 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.760 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.760 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.760 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.760 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.760 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.761 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.761 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.761 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.761 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.761 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.761 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.761 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:46.761 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:46.761 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.761 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.761 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.761 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:46.762 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.762 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.762 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.762 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.762 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.762 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.762 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.762 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.762 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.762 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.762 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.763 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.763 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.763 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.763 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.763 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:46.763 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:46.763 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.763 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.763 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.763 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.763 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.763 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.764 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.764 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.764 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.764 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.764 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.764 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.764 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:46.764 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:46.764 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.764 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:46.764 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.764 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.764 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.765 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.765 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.765 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.765 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.765 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.765 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.765 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.765 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.765 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.765 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:46.765 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mirroring", 2024-05-09T01:01:46.766 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:46.766 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:46.766 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:46.766 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:46.766 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:46.766 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.766 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.766 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.766 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.766 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.766 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.766 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.767 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.767 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.767 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.767 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.767 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.767 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.767 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.767 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.767 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.767 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.767 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.767 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.768 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:46.768 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:46.768 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.768 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.768 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.768 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.768 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.768 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.768 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.768 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.768 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.769 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.769 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.769 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.769 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:46.769 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:46.769 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.769 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.769 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.769 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:46.769 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.769 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.769 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.770 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.770 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.770 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.770 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.770 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.770 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.770 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.770 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.770 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.770 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.770 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.770 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.771 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:46.771 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:46.771 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.771 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.771 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.771 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.771 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.771 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.771 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.771 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.771 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.772 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.772 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.772 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.772 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:46.772 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:46.772 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.772 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:46.772 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.772 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.772 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.772 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.772 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.773 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.773 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.773 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.773 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.773 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.773 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.773 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.773 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:46.773 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "nfs", 2024-05-09T01:01:46.773 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:46.773 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:46.773 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:46.774 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:46.774 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:46.774 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.774 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.774 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.774 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.774 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.774 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.774 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.774 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.774 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.775 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.775 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.775 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.775 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.775 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.775 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.775 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.775 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.775 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.775 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.775 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:46.775 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:46.775 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.776 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.776 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.776 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.776 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.776 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.776 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.776 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.776 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.776 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.776 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.776 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.776 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:46.777 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:46.777 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.777 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.777 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.777 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:46.777 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.777 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.777 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.777 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.777 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.777 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.777 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.778 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.778 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.778 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.778 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.778 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.778 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.778 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.778 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.778 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:46.778 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:46.778 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.778 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.779 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.779 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.779 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.779 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.779 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.779 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.779 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.779 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.779 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.779 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.779 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:46.780 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:46.780 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.780 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:46.780 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.780 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.780 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.780 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.780 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.780 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.780 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.780 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.780 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.781 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.781 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.781 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.781 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:46.781 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "orchestrator", 2024-05-09T01:01:46.781 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:46.781 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:46.781 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:46.781 INFO:teuthology.orchestra.run.smithi005.stdout: "fail_fs": { 2024-05-09T01:01:46.781 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "fail_fs", 2024-05-09T01:01:46.781 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.781 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.781 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.782 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.782 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.782 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.782 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.782 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-09T01:01:46.782 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.782 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.782 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.782 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.782 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:46.782 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:46.783 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.783 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.783 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.783 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.783 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.783 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.783 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.783 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.783 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.783 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.783 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.783 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.783 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.784 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.784 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.784 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.784 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.784 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.784 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.784 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:46.784 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:46.784 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.784 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.784 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.785 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.785 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.785 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.785 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.785 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.785 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.785 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.785 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.785 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.785 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:46.785 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:46.786 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.786 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.786 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.786 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:46.786 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.786 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.786 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.786 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.786 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.786 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.786 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.786 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.787 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.787 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.787 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.787 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.787 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.787 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.787 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.787 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:46.787 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:46.787 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.787 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.787 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.788 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.788 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.788 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.788 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.788 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.788 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.788 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.788 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.788 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.788 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator": { 2024-05-09T01:01:46.788 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "orchestrator", 2024-05-09T01:01:46.788 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.788 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.788 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.789 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.789 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.789 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.789 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.789 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm", 2024-05-09T01:01:46.789 INFO:teuthology.orchestra.run.smithi005.stdout: "rook", 2024-05-09T01:01:46.789 INFO:teuthology.orchestra.run.smithi005.stdout: "test_orchestrator" 2024-05-09T01:01:46.789 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.789 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Orchestrator backend", 2024-05-09T01:01:46.789 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.789 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.789 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.789 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.790 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:46.790 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:46.790 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.790 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:46.790 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.790 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.790 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.790 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.790 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.790 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.790 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.790 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.791 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.791 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.791 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.791 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.791 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:46.791 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "osd_perf_query", 2024-05-09T01:01:46.791 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:46.791 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:46.791 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:46.791 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:46.792 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:46.792 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.792 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.792 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.792 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.792 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.792 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.792 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.792 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.792 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.792 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.792 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.793 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.793 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.793 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.793 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.793 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.793 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.793 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.793 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.793 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:46.793 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:46.793 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.794 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.794 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.794 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.794 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.794 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.794 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.794 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.794 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.794 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.794 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.794 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.795 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:46.795 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:46.795 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.795 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.795 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.795 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:46.795 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.795 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.795 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.795 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.795 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.796 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.796 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.796 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.796 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.796 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.796 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.796 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.796 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.796 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.796 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.796 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:46.796 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:46.797 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.797 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.797 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.797 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.797 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.797 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.797 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.797 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.797 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.797 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.797 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.797 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.798 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:46.798 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:46.798 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.798 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:46.798 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.798 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.798 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.798 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.798 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.798 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.798 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.798 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.799 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.799 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.799 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.799 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.799 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:46.799 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "osd_support", 2024-05-09T01:01:46.799 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:46.799 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:46.799 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:46.799 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:46.799 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:46.799 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.800 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.800 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.800 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.800 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.800 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.800 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.800 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.800 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.800 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.800 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.801 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.801 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.801 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.801 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.801 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.801 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.801 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.801 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.801 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:46.801 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:46.801 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.801 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.802 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.802 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.802 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.802 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.802 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.802 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.802 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.802 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.802 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.802 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.802 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:46.802 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:46.803 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.803 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.803 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.803 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:46.803 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.803 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.803 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.803 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.803 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.803 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.803 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.804 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.804 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.804 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.804 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.804 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.804 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.804 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.804 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.804 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:46.804 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:46.804 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.804 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.805 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.805 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.805 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.805 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.805 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.805 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.805 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.805 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.805 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.805 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.805 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:46.805 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:46.806 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.806 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:46.806 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.806 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.806 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.806 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.806 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.806 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.806 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.806 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.806 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.807 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.807 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.807 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.807 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:46.807 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pg_autoscaler", 2024-05-09T01:01:46.807 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:46.807 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:46.807 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:46.807 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:46.807 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:46.807 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.807 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.808 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.808 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.808 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.808 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.808 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.808 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.808 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.808 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.808 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.808 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.808 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.808 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.809 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.809 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.809 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.809 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.809 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.809 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:46.809 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:46.809 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.809 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.809 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.809 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.810 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.810 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.810 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.810 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.810 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.810 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.810 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.810 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.810 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:46.810 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:46.810 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.811 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.811 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.811 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:46.811 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.811 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.811 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.811 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.811 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.811 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.811 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.811 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.811 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.811 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.812 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.812 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.812 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.812 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.812 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.812 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:46.812 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:46.812 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.812 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.812 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.812 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.812 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.813 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.813 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.813 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.813 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.813 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.813 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.813 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.813 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-05-09T01:01:46.813 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-05-09T01:01:46.813 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-09T01:01:46.813 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.813 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.814 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-05-09T01:01:46.814 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.814 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.814 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.814 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.814 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.814 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.814 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.814 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.814 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:46.814 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:46.815 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.815 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:46.815 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.815 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.815 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.815 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.815 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.815 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.815 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.815 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.815 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.815 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.816 INFO:teuthology.orchestra.run.smithi005.stdout: "threshold": { 2024-05-09T01:01:46.816 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "threshold", 2024-05-09T01:01:46.816 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-05-09T01:01:46.816 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.816 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.816 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3.0", 2024-05-09T01:01:46.816 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1.0", 2024-05-09T01:01:46.816 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.816 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.816 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "scaling threshold", 2024-05-09T01:01:46.816 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-09T01:01:46.816 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.817 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.817 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.817 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.817 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.817 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:46.817 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "progress", 2024-05-09T01:01:46.817 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:46.817 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:46.817 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:46.817 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_pg_recovery_event": { 2024-05-09T01:01:46.817 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_pg_recovery_event", 2024-05-09T01:01:46.817 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.818 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.818 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.818 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.818 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.818 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.818 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.818 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-09T01:01:46.818 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.818 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.818 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.818 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.818 INFO:teuthology.orchestra.run.smithi005.stdout: "enabled": { 2024-05-09T01:01:46.819 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enabled", 2024-05-09T01:01:46.819 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.819 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.819 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.819 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.819 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.819 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.819 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.819 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.819 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.819 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.819 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.820 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.820 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:46.820 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:46.820 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.820 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.820 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.820 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.820 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.820 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.820 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.820 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.820 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.821 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.821 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.821 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.821 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.821 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.821 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.821 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.821 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.821 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.821 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.821 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:46.821 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:46.822 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.822 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.822 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.822 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.822 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.822 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.822 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.822 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.822 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.822 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.822 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.823 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.823 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:46.823 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:46.823 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.823 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.823 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.823 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:46.823 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.823 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.823 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.823 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.823 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.823 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.824 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.824 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.824 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.824 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.824 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.824 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.824 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.824 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.824 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.824 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:46.824 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:46.824 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.825 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.825 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.825 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.825 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.825 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.825 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.825 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.825 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.825 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.825 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.825 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.825 INFO:teuthology.orchestra.run.smithi005.stdout: "max_completed_events": { 2024-05-09T01:01:46.825 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_completed_events", 2024-05-09T01:01:46.826 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.826 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.826 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.826 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "50", 2024-05-09T01:01:46.826 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.826 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.826 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.826 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "number of past completed events to remember", 2024-05-09T01:01:46.826 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.826 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.826 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.826 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.827 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-05-09T01:01:46.827 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-05-09T01:01:46.827 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-09T01:01:46.827 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.827 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.827 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-05-09T01:01:46.827 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.827 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.827 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.827 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long the module is going to sleep", 2024-05-09T01:01:46.827 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.827 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.828 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.828 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.828 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:46.828 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:46.828 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.828 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:46.828 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.828 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.828 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.828 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.828 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.828 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.829 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.829 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.829 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.829 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.829 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.829 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.829 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:46.829 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus", 2024-05-09T01:01:46.829 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:46.829 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:46.829 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:46.829 INFO:teuthology.orchestra.run.smithi005.stdout: "cache": { 2024-05-09T01:01:46.830 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cache", 2024-05-09T01:01:46.830 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.830 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.830 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.830 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.830 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.830 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.830 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.830 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.830 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.830 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.830 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.831 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.831 INFO:teuthology.orchestra.run.smithi005.stdout: "exclude_perf_counters": { 2024-05-09T01:01:46.831 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "exclude_perf_counters", 2024-05-09T01:01:46.831 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.831 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.831 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.831 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.831 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.831 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.831 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.831 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-09T01:01:46.831 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-09T01:01:46.832 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.832 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.832 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.832 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:46.832 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:46.832 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.832 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.832 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.832 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.832 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.832 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.833 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.833 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.833 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.833 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.833 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.833 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.833 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.833 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.833 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.833 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.833 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.833 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.834 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.834 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:46.834 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:46.834 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.834 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.834 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.834 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.834 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.834 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.834 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.834 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.834 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.835 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.835 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.835 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.835 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:46.835 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:46.835 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.835 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.835 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.835 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:46.835 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.835 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.835 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.836 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.836 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.836 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.836 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.836 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.836 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.836 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.836 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.836 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.836 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.836 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.836 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.836 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:46.837 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:46.837 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.837 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.837 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.837 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.837 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.837 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.837 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.837 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.837 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.837 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.837 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.838 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.838 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_stats_pools": { 2024-05-09T01:01:46.838 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_stats_pools", 2024-05-09T01:01:46.838 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.838 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.838 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.838 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.838 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.838 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.838 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.838 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.838 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.839 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.839 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.839 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.839 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-09T01:01:46.839 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-09T01:01:46.839 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.839 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.839 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.839 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "300", 2024-05-09T01:01:46.839 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.839 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.839 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.840 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.840 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.840 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.840 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.841 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.841 INFO:teuthology.orchestra.run.smithi005.stdout: "scrape_interval": { 2024-05-09T01:01:46.841 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "scrape_interval", 2024-05-09T01:01:46.841 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-05-09T01:01:46.841 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.842 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.842 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15.0", 2024-05-09T01:01:46.842 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.842 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.842 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.842 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.842 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.842 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.842 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.842 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.842 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-05-09T01:01:46.842 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-05-09T01:01:46.843 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.843 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.843 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.843 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "::", 2024-05-09T01:01:46.843 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.843 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.843 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.843 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-09T01:01:46.843 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.843 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.843 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.844 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.844 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-05-09T01:01:46.844 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-05-09T01:01:46.844 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.844 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.844 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.844 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "9283", 2024-05-09T01:01:46.844 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.844 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.844 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.844 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-09T01:01:46.845 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.845 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.845 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.845 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.845 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:46.845 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:46.845 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.845 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:46.845 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.845 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.845 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.845 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.845 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.846 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.846 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.846 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.846 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.846 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.846 INFO:teuthology.orchestra.run.smithi005.stdout: "stale_cache_strategy": { 2024-05-09T01:01:46.846 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "stale_cache_strategy", 2024-05-09T01:01:46.846 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.846 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.846 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.847 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "log", 2024-05-09T01:01:46.847 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.847 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.847 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.847 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.847 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.847 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.847 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.847 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.847 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_behaviour": { 2024-05-09T01:01:46.847 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_behaviour", 2024-05-09T01:01:46.848 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.848 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.848 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.848 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "default", 2024-05-09T01:01:46.848 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.848 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.848 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.848 INFO:teuthology.orchestra.run.smithi005.stdout: "default", 2024-05-09T01:01:46.848 INFO:teuthology.orchestra.run.smithi005.stdout: "error" 2024-05-09T01:01:46.848 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.848 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.848 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.848 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.849 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.849 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.849 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_error_status_code": { 2024-05-09T01:01:46.849 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_error_status_code", 2024-05-09T01:01:46.849 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.849 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.849 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.849 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "500", 2024-05-09T01:01:46.849 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "400", 2024-05-09T01:01:46.849 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "599", 2024-05-09T01:01:46.849 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.850 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.850 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.850 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.850 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.850 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.850 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.850 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.850 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:46.850 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_support", 2024-05-09T01:01:46.850 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:46.850 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:46.850 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:46.851 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:46.851 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:46.851 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.851 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.851 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.851 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.851 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.851 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.851 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.851 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.851 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.851 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.851 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.852 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.852 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.852 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.852 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.852 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.852 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.852 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.852 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.852 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:46.852 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:46.853 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.853 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.853 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.853 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.853 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.853 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.853 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.853 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.853 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.853 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.853 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.854 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.854 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:46.854 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:46.854 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.854 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.854 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.854 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:46.854 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.854 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.854 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.854 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.854 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.855 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.855 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.855 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.855 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.855 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.855 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.855 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.855 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.855 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.855 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.855 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:46.855 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:46.856 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.856 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.856 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.856 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.856 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.856 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.856 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.856 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.856 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.856 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.857 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.857 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.857 INFO:teuthology.orchestra.run.smithi005.stdout: "max_concurrent_snap_create": { 2024-05-09T01:01:46.857 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_concurrent_snap_create", 2024-05-09T01:01:46.857 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.857 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.857 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.857 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-05-09T01:01:46.857 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.857 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.858 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.858 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.858 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.858 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.858 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.858 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.858 INFO:teuthology.orchestra.run.smithi005.stdout: "mirror_snapshot_schedule": { 2024-05-09T01:01:46.858 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mirror_snapshot_schedule", 2024-05-09T01:01:46.858 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.858 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.858 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.858 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.859 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.859 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.859 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.859 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.859 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.859 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.859 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.859 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.859 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:46.859 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:46.859 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.859 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:46.860 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.860 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.860 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.860 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.860 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.860 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.860 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.860 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.860 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.860 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.860 INFO:teuthology.orchestra.run.smithi005.stdout: "trash_purge_schedule": { 2024-05-09T01:01:46.861 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "trash_purge_schedule", 2024-05-09T01:01:46.861 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.861 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.861 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.861 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.861 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.861 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.861 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.861 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.861 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.861 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.861 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.862 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.862 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.862 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.862 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:46.862 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "restful", 2024-05-09T01:01:46.862 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:46.862 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:46.862 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:46.862 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_auth": { 2024-05-09T01:01:46.862 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enable_auth", 2024-05-09T01:01:46.862 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.863 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.863 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.863 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.863 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.863 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.863 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.863 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.863 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.863 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.863 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.863 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.864 INFO:teuthology.orchestra.run.smithi005.stdout: "key_file": { 2024-05-09T01:01:46.864 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "key_file", 2024-05-09T01:01:46.864 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.864 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.864 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.864 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.864 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.864 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.864 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.864 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.864 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.864 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.865 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.865 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.865 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:46.865 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:46.865 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.865 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.865 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.865 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.865 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.865 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.865 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.865 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.865 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.866 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.866 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.866 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.866 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.866 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.866 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.866 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.866 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.866 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.866 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.866 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:46.867 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:46.867 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.867 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.867 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.867 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.867 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.867 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.867 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.867 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.867 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.867 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.867 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.868 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.868 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:46.868 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:46.868 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.868 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.868 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.868 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:46.868 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.868 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.868 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.868 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.868 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.868 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.869 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.869 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.869 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.869 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.869 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.869 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.869 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.869 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.869 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.869 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:46.869 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:46.870 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.870 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.870 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.870 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.870 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.870 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.870 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.870 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.870 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.870 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.870 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.870 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.871 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-05-09T01:01:46.871 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-05-09T01:01:46.871 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.871 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.871 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.871 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.871 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.871 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.871 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.871 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.871 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.871 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.872 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.872 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.872 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-05-09T01:01:46.872 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-05-09T01:01:46.872 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.872 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.872 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.872 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.872 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.872 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.872 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.873 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.873 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.873 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.873 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.873 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.873 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:46.873 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:46.873 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.873 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:46.873 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.874 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.874 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.874 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.874 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.874 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.874 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.874 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.874 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.874 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.874 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.875 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.875 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:46.875 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rgw", 2024-05-09T01:01:46.875 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:46.875 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:46.875 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:46.875 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:46.875 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:46.875 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.875 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.876 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.876 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.876 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.876 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.876 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.876 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.876 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.876 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.876 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.876 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.876 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.876 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.877 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.877 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.877 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.877 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.877 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.877 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:46.877 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:46.877 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.877 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.877 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.878 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.878 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.878 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.878 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.878 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.878 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.878 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.878 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.878 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.878 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:46.878 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:46.878 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.879 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.879 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.879 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:46.879 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.879 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.879 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.879 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.879 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.879 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.879 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.879 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.879 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.880 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.880 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.880 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.880 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.880 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.880 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.880 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:46.880 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:46.880 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.880 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.880 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.880 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.881 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.881 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.881 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.881 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.881 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.881 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.881 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.881 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.881 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:46.881 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:46.881 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.881 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:46.882 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.882 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.882 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.882 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.882 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.882 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.882 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.882 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.882 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.882 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.882 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.883 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.883 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:46.883 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rook", 2024-05-09T01:01:46.883 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:46.883 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:46.883 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:46.883 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:46.883 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:46.883 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.883 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.883 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.883 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.884 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.884 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.884 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.884 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.884 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.884 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.884 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.884 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.884 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.884 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.884 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.884 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.885 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.885 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.885 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.885 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:46.885 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:46.885 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.885 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.885 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.885 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.885 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.885 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.885 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.886 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.886 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.886 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.886 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.886 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.886 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:46.886 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:46.886 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.886 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.886 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.886 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:46.887 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.887 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.887 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.887 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.887 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.887 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.887 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.887 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.887 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.887 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.887 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.887 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.888 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.888 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.888 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.888 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:46.888 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:46.888 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.888 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.888 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.888 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.888 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.888 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.888 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.889 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.889 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.889 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.889 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.889 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.889 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:46.889 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:46.889 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.889 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:46.889 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.889 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.889 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.890 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.890 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.890 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.890 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.890 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.890 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.890 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.890 INFO:teuthology.orchestra.run.smithi005.stdout: "storage_class": { 2024-05-09T01:01:46.890 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "storage_class", 2024-05-09T01:01:46.890 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.890 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.891 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.891 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "local", 2024-05-09T01:01:46.891 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.891 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.891 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.891 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-09T01:01:46.891 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.891 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.891 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.891 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.891 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.892 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.892 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:46.892 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "selftest", 2024-05-09T01:01:46.892 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:46.892 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:46.892 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:46.892 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:46.892 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:46.892 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.892 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.892 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.892 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.893 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.893 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.893 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.893 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.893 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.893 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.893 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.893 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.893 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.893 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.893 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.894 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.894 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.894 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.894 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.894 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:46.894 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:46.894 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.894 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.894 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.894 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.894 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.895 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.895 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.895 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.895 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.895 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.895 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.895 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.895 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:46.895 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:46.895 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.895 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.896 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.896 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:46.896 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.896 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.896 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.896 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.896 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.896 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.896 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.896 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.896 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.896 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.897 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.897 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.897 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.897 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.897 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.897 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:46.897 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:46.897 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.897 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.897 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.897 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.898 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.898 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.898 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.898 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.898 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.898 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.898 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.898 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.898 INFO:teuthology.orchestra.run.smithi005.stdout: "roption1": { 2024-05-09T01:01:46.898 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "roption1", 2024-05-09T01:01:46.898 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.899 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.899 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.899 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.899 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.899 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.899 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.899 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.899 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.899 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.899 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.899 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.899 INFO:teuthology.orchestra.run.smithi005.stdout: "roption2": { 2024-05-09T01:01:46.900 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "roption2", 2024-05-09T01:01:46.900 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.900 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.900 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.900 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "xyz", 2024-05-09T01:01:46.900 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.900 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.900 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.900 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.900 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.900 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.900 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.901 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.901 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption1": { 2024-05-09T01:01:46.901 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption1", 2024-05-09T01:01:46.901 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.901 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.901 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.901 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.901 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.901 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.901 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.901 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.901 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.902 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.902 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.902 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.902 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption2": { 2024-05-09T01:01:46.902 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption2", 2024-05-09T01:01:46.902 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.902 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.902 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.902 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.902 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.902 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.902 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.903 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.903 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.903 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.903 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.903 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.903 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption3": { 2024-05-09T01:01:46.903 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption3", 2024-05-09T01:01:46.903 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-05-09T01:01:46.903 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.903 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.903 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.903 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.904 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.904 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.904 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.904 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.904 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.904 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.904 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.904 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption4": { 2024-05-09T01:01:46.904 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption4", 2024-05-09T01:01:46.904 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.904 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.905 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.905 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.905 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.905 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.905 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.905 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.905 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.905 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.905 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.905 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.905 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption5": { 2024-05-09T01:01:46.905 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption5", 2024-05-09T01:01:46.906 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.906 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.906 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.906 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.906 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.906 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.906 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.906 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.906 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.906 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.906 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.906 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.907 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption6": { 2024-05-09T01:01:46.907 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption6", 2024-05-09T01:01:46.907 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.907 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.907 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.907 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.907 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.907 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.907 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.907 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.907 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.908 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.908 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.908 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.908 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption7": { 2024-05-09T01:01:46.908 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption7", 2024-05-09T01:01:46.908 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.908 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.908 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.908 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.908 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-05-09T01:01:46.908 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "42", 2024-05-09T01:01:46.909 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.909 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.909 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.909 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.909 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.909 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.909 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:46.909 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:46.909 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.909 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:46.909 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.909 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.910 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.910 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.910 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.910 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.910 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.910 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.910 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.910 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.910 INFO:teuthology.orchestra.run.smithi005.stdout: "testkey": { 2024-05-09T01:01:46.910 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testkey", 2024-05-09T01:01:46.910 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.911 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.911 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.911 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.911 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.911 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.911 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.911 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.911 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.911 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.911 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.911 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.911 INFO:teuthology.orchestra.run.smithi005.stdout: "testlkey": { 2024-05-09T01:01:46.912 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testlkey", 2024-05-09T01:01:46.912 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.912 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.912 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.912 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.912 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.912 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.912 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.912 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.912 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.912 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.913 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.913 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.913 INFO:teuthology.orchestra.run.smithi005.stdout: "testnewline": { 2024-05-09T01:01:46.913 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testnewline", 2024-05-09T01:01:46.913 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.913 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.913 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.913 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.913 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.913 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.913 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.913 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.913 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.914 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.914 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.914 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.914 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.914 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.914 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:46.914 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smb", 2024-05-09T01:01:46.914 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:46.914 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:46.914 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:46.914 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:46.915 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:46.915 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.915 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.915 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.915 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.915 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.915 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.915 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.915 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.915 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.915 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.915 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.916 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.916 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.916 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.916 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.916 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.916 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.916 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.916 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.916 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:46.916 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:46.916 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.917 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.917 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.917 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.917 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.917 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.917 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.917 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.917 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.917 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.917 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.917 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.918 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:46.918 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:46.918 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.918 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.918 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.918 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:46.918 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.918 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.918 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.918 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.919 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.919 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.919 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.919 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.919 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.919 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.919 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.919 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.919 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.919 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.919 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.919 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:46.920 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:46.920 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.920 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.920 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.920 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.920 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.920 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.920 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.920 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.920 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.920 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.920 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.921 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.921 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:46.921 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:46.921 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.921 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:46.921 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.921 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.921 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.921 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.921 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.921 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.921 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.922 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.922 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.922 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.922 INFO:teuthology.orchestra.run.smithi005.stdout: "update_orchestration": { 2024-05-09T01:01:46.922 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "update_orchestration", 2024-05-09T01:01:46.922 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.922 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.922 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.922 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.922 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.922 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.922 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.923 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-09T01:01:46.923 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.923 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.923 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.923 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.923 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.923 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.923 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:46.923 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snap_schedule", 2024-05-09T01:01:46.923 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:46.923 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:46.923 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:46.924 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_m_granularity": { 2024-05-09T01:01:46.924 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_m_granularity", 2024-05-09T01:01:46.924 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.924 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.924 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.924 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.924 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.924 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.924 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.924 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow minute scheduled snapshots", 2024-05-09T01:01:46.924 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.924 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.925 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.925 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.925 INFO:teuthology.orchestra.run.smithi005.stdout: "dump_on_update": { 2024-05-09T01:01:46.925 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "dump_on_update", 2024-05-09T01:01:46.925 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.925 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.925 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.925 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.925 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.925 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.925 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.925 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "dump database to debug log on update", 2024-05-09T01:01:46.925 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.926 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.926 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.926 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.926 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:46.926 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:46.926 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.926 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.926 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.926 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.926 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.926 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.926 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.927 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.927 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.927 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.927 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.927 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.927 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.927 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.927 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.927 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.927 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.927 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.928 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.928 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:46.928 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:46.928 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.928 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.928 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.928 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.928 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.928 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.928 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.928 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.928 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.929 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.929 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.929 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.929 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:46.929 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:46.929 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.929 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.929 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.929 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:46.929 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.929 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.930 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.930 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.930 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.930 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.930 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.930 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.930 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.930 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.930 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.930 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.930 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.931 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.931 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.931 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:46.931 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:46.931 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.931 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.931 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.931 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.931 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.931 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.931 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.931 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.932 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.932 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.932 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.932 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.932 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:46.932 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:46.932 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.932 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:46.932 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.932 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.932 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.932 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.932 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.933 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.933 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.933 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.933 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.933 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.933 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.933 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.933 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:46.933 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "stats", 2024-05-09T01:01:46.933 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:46.933 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:46.933 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:46.934 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:46.934 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:46.934 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.934 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.934 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.934 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.934 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.934 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.934 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.934 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.934 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.934 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.935 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.935 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.935 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.935 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.935 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.935 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.935 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.935 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.935 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.935 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:46.935 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:46.936 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.936 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.936 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.936 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.936 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.936 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.936 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.936 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.936 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.936 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.936 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.936 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.937 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:46.937 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:46.937 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.937 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.937 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.937 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:46.937 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.937 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.937 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.937 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.937 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.937 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.938 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.938 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.938 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.938 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.938 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.938 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.938 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.938 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.938 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.938 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:46.938 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:46.939 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.939 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.939 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.939 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.939 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.939 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.939 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.939 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.939 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.939 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.939 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.939 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.940 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:46.940 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:46.940 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.940 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:46.940 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.940 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.940 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.940 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.940 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.940 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.940 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.940 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.940 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.941 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.941 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.941 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.941 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:46.941 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "status", 2024-05-09T01:01:46.941 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:46.941 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:46.941 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:46.941 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:46.941 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:46.941 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.941 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.942 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.942 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.942 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.942 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.942 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.942 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.942 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.942 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.942 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.942 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.942 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.942 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.943 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.943 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.943 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.943 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.943 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.943 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:46.943 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:46.943 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.943 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.943 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.943 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.943 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.944 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.944 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.944 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.944 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.944 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.944 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.944 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.944 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:46.944 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:46.944 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.944 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.945 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.945 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:46.945 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.945 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.945 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.945 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.945 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.945 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.945 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.945 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.945 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.945 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.946 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.946 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.946 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.946 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.946 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.946 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:46.946 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:46.946 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.946 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.946 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.946 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.946 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.946 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.947 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.947 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.947 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.947 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.947 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.947 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.947 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:46.947 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:46.947 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.947 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:46.947 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.947 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.948 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.948 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.948 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.948 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.948 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.948 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.948 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.948 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.948 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.948 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.948 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:46.948 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "telegraf", 2024-05-09T01:01:46.949 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:46.949 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:46.949 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:46.949 INFO:teuthology.orchestra.run.smithi005.stdout: "address": { 2024-05-09T01:01:46.949 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "address", 2024-05-09T01:01:46.949 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.949 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.949 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.949 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-09T01:01:46.949 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.949 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.949 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.949 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.950 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.950 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.950 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.950 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.950 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-05-09T01:01:46.950 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-05-09T01:01:46.950 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-09T01:01:46.950 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.950 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.950 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15", 2024-05-09T01:01:46.950 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.951 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.951 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.951 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.951 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.951 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.951 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.951 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.951 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:46.951 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:46.951 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.951 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.951 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.952 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.952 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.952 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.952 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.952 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.952 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.952 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.952 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.952 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.952 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.952 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.952 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.953 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.953 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.953 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.953 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.953 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:46.953 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:46.953 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.953 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.953 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.953 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.953 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.953 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.953 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.954 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.954 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.954 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.954 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.954 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.954 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:46.954 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:46.954 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.954 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.954 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.954 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:46.955 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.955 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.955 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.955 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.955 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.955 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.955 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.955 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.955 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.955 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.955 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.955 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.955 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.956 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.956 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.956 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:46.956 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:46.956 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.956 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.956 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.956 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.956 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.956 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.956 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.956 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.957 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.957 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.957 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.957 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.957 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:46.957 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:46.957 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.957 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:46.957 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.957 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.957 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.957 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.957 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.958 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.958 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.958 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.958 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.958 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.958 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.958 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.958 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:46.958 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "telemetry", 2024-05-09T01:01:46.958 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:46.958 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:46.958 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:46.959 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_basic": { 2024-05-09T01:01:46.959 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_basic", 2024-05-09T01:01:46.959 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.959 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.959 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.959 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.959 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.959 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.959 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.959 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-09T01:01:46.959 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.959 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.960 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.960 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.960 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_crash": { 2024-05-09T01:01:46.960 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_crash", 2024-05-09T01:01:46.960 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.960 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.960 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.960 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.960 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.960 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.960 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.960 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-09T01:01:46.961 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.961 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.961 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.961 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.961 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_device": { 2024-05-09T01:01:46.961 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_device", 2024-05-09T01:01:46.961 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.961 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.961 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.961 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:46.961 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.961 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.962 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.962 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-09T01:01:46.962 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.962 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.962 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.962 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.962 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_ident": { 2024-05-09T01:01:46.962 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_ident", 2024-05-09T01:01:46.962 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.962 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.962 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.962 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.963 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.963 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.963 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.963 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-09T01:01:46.963 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.963 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.963 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.963 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.963 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_perf": { 2024-05-09T01:01:46.963 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_perf", 2024-05-09T01:01:46.963 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.964 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.964 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.964 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.964 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.964 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.964 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.964 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-09T01:01:46.964 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.964 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.964 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.964 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.964 INFO:teuthology.orchestra.run.smithi005.stdout: "contact": { 2024-05-09T01:01:46.965 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "contact", 2024-05-09T01:01:46.965 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.965 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.965 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.965 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.965 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.965 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.965 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.965 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.965 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.965 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.965 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.965 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.966 INFO:teuthology.orchestra.run.smithi005.stdout: "description": { 2024-05-09T01:01:46.966 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "description", 2024-05-09T01:01:46.966 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.966 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.966 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.966 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.966 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.966 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.966 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.966 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.966 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.966 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.967 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.967 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.967 INFO:teuthology.orchestra.run.smithi005.stdout: "device_url": { 2024-05-09T01:01:46.967 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_url", 2024-05-09T01:01:46.967 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.967 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.967 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.967 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-09T01:01:46.967 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.967 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.967 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.967 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.968 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.968 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.968 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.968 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.968 INFO:teuthology.orchestra.run.smithi005.stdout: "enabled": { 2024-05-09T01:01:46.968 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enabled", 2024-05-09T01:01:46.968 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.968 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.968 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.968 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.968 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.969 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.969 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.969 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.969 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.969 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.969 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.969 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.969 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-05-09T01:01:46.969 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-05-09T01:01:46.969 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.969 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.969 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.970 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "24", 2024-05-09T01:01:46.970 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "8", 2024-05-09T01:01:46.970 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.970 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.970 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.970 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.970 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.970 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.970 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.970 INFO:teuthology.orchestra.run.smithi005.stdout: "last_opt_revision": { 2024-05-09T01:01:46.970 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "last_opt_revision", 2024-05-09T01:01:46.970 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.971 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.971 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.971 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1", 2024-05-09T01:01:46.971 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.971 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.971 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.971 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.971 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.971 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.971 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.971 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.971 INFO:teuthology.orchestra.run.smithi005.stdout: "leaderboard": { 2024-05-09T01:01:46.972 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "leaderboard", 2024-05-09T01:01:46.972 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.972 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.972 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.972 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.972 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.972 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.972 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.972 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.972 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.972 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.973 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.973 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.973 INFO:teuthology.orchestra.run.smithi005.stdout: "leaderboard_description": { 2024-05-09T01:01:46.973 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "leaderboard_description", 2024-05-09T01:01:46.973 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.973 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.973 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.973 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.973 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.973 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.973 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.974 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.974 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.974 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.974 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.974 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.974 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:46.974 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:46.974 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.974 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.974 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.974 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.975 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.975 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.975 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.975 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.975 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.975 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.975 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.975 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.975 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.975 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.975 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.975 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.976 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.976 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.976 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.976 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:46.976 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:46.976 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.976 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.976 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.976 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.976 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.976 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.976 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.977 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.977 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.977 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.977 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.977 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.977 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:46.977 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:46.977 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.977 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.977 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.977 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:46.978 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.978 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.978 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.978 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.978 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.978 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.978 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.978 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.978 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.978 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.978 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.979 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.979 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.979 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.979 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.979 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:46.979 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:46.979 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.979 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.979 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.979 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.979 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.979 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.980 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.980 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.980 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.980 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.980 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.980 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.980 INFO:teuthology.orchestra.run.smithi005.stdout: "organization": { 2024-05-09T01:01:46.980 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "organization", 2024-05-09T01:01:46.980 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.980 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.980 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.981 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.981 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.981 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.981 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.981 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.981 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.981 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.981 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.981 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.981 INFO:teuthology.orchestra.run.smithi005.stdout: "proxy": { 2024-05-09T01:01:46.981 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "proxy", 2024-05-09T01:01:46.981 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.982 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.982 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.982 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.982 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.982 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.982 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.982 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.982 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.982 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.982 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.982 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.983 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:46.983 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:46.983 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.983 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:46.983 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.983 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.983 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.983 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.983 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.983 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.983 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.984 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.984 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.984 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.984 INFO:teuthology.orchestra.run.smithi005.stdout: "url": { 2024-05-09T01:01:46.984 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "url", 2024-05-09T01:01:46.984 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.984 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.984 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.984 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-09T01:01:46.984 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.984 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.984 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.985 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.985 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.985 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.985 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.985 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.985 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.985 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.985 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:46.985 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "test_orchestrator", 2024-05-09T01:01:46.985 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:46.985 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:46.985 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:46.986 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:46.986 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:46.986 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.986 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.986 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.986 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.986 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.986 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.986 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.986 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.986 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.986 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.987 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.987 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.987 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.987 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.987 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.987 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.987 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.987 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.987 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.987 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:46.987 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:46.987 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.988 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.988 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.988 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.988 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.988 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.988 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.988 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.988 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.988 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.988 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.988 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.988 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:46.989 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:46.989 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.989 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.989 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.989 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:46.989 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.989 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.989 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.989 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.989 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.989 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.989 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.990 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.990 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.990 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.990 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.990 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.990 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.990 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.990 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.990 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:46.990 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:46.990 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.990 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.991 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.991 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.991 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.991 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.991 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.991 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.991 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.991 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.991 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.991 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.991 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:46.991 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:46.991 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.992 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:46.992 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.992 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:46.992 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.992 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.992 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.992 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.992 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.992 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.992 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.992 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.992 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:46.993 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.993 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:46.993 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "volumes", 2024-05-09T01:01:46.993 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:46.993 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:46.993 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:46.993 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:46.993 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:46.993 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.993 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.993 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.993 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:46.993 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.994 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.994 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.994 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.994 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.994 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.994 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.994 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.994 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.994 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.994 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.994 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.994 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.995 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.995 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.995 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:46.995 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:46.995 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.995 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.995 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.995 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.995 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.995 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.995 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.995 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.996 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.996 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.996 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.996 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.996 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:46.996 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:46.996 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:46.996 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.996 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.996 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:46.996 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.996 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.997 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:46.997 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:46.997 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:46.997 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:46.997 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:46.997 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:46.997 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:46.997 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:46.997 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.997 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.997 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.997 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.998 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.998 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:46.998 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:46.998 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:46.998 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.998 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:46.998 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:46.998 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.998 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.998 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.998 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:46.998 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:46.999 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:46.999 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:46.999 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:46.999 INFO:teuthology.orchestra.run.smithi005.stdout: "max_concurrent_clones": { 2024-05-09T01:01:46.999 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_concurrent_clones", 2024-05-09T01:01:46.999 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:46.999 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:46.999 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:46.999 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "4", 2024-05-09T01:01:46.999 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:46.999 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:46.999 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:46.999 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-09T01:01:47.000 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:47.000 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:47.000 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:47.000 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:47.000 INFO:teuthology.orchestra.run.smithi005.stdout: "periodic_async_work": { 2024-05-09T01:01:47.000 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "periodic_async_work", 2024-05-09T01:01:47.000 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:47.000 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:47.000 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:47.000 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:47.000 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:47.001 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:47.001 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:47.001 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Periodically check for async work", 2024-05-09T01:01:47.001 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:47.001 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:47.001 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:47.001 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:47.001 INFO:teuthology.orchestra.run.smithi005.stdout: "snapshot_clone_delay": { 2024-05-09T01:01:47.001 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snapshot_clone_delay", 2024-05-09T01:01:47.001 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:47.001 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:47.001 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:47.001 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:47.002 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:47.002 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:47.002 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:47.002 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-09T01:01:47.002 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:47.002 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:47.002 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:47.002 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:47.002 INFO:teuthology.orchestra.run.smithi005.stdout: "snapshot_clone_no_wait": { 2024-05-09T01:01:47.002 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snapshot_clone_no_wait", 2024-05-09T01:01:47.002 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:47.003 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:47.003 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:47.003 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-09T01:01:47.003 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:47.003 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:47.003 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:47.003 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-09T01:01:47.003 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:47.003 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:47.003 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:47.003 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:47.003 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:47.004 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:47.004 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:47.004 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:47.004 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:47.004 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:47.004 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:47.004 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:47.004 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:47.004 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:47.004 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:47.004 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:47.004 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:47.005 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:47.005 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:47.005 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:47.005 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:47.005 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix", 2024-05-09T01:01:47.005 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-09T01:01:47.005 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-09T01:01:47.005 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-09T01:01:47.005 INFO:teuthology.orchestra.run.smithi005.stdout: "discovery_interval": { 2024-05-09T01:01:47.005 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "discovery_interval", 2024-05-09T01:01:47.005 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-05-09T01:01:47.005 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:47.006 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:47.006 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "100", 2024-05-09T01:01:47.006 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:47.006 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:47.006 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:47.006 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:47.006 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:47.006 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:47.006 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:47.006 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:47.006 INFO:teuthology.orchestra.run.smithi005.stdout: "identifier": { 2024-05-09T01:01:47.007 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "identifier", 2024-05-09T01:01:47.007 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:47.007 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:47.007 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:47.007 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:47.007 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:47.007 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:47.007 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:47.007 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:47.007 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:47.007 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:47.007 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:47.008 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:47.008 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-05-09T01:01:47.008 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-05-09T01:01:47.008 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-09T01:01:47.008 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:47.008 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:47.008 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-05-09T01:01:47.008 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:47.008 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:47.008 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:47.008 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:47.008 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:47.009 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:47.009 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:47.009 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:47.009 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-09T01:01:47.009 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-09T01:01:47.009 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:47.009 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:47.009 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:47.009 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:47.009 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:47.009 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:47.009 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:47.010 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:47.010 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:47.010 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:47.010 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:47.010 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:47.010 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:47.010 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:47.010 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:47.010 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:47.010 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:47.010 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:47.010 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:47.011 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-09T01:01:47.011 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-09T01:01:47.011 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:47.011 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:47.011 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:47.011 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:47.011 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:47.011 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:47.011 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:47.011 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:47.011 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:47.011 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:47.012 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:47.012 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:47.012 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-09T01:01:47.012 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-09T01:01:47.012 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:47.012 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:47.012 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:47.012 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-09T01:01:47.012 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:47.012 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:47.012 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-09T01:01:47.013 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-09T01:01:47.013 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-09T01:01:47.013 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-09T01:01:47.013 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-09T01:01:47.013 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-09T01:01:47.013 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-09T01:01:47.013 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:47.013 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:47.013 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:47.013 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:47.013 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:47.013 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:47.014 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-09T01:01:47.014 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-09T01:01:47.014 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-09T01:01:47.014 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:47.014 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:47.014 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-09T01:01:47.014 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:47.014 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:47.014 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:47.014 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:47.014 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:47.014 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:47.015 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:47.015 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:47.015 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-09T01:01:47.015 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-09T01:01:47.015 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:47.015 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-09T01:01:47.015 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-09T01:01:47.015 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-09T01:01:47.015 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:47.015 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:47.015 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:47.016 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:47.016 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:47.016 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:47.016 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:47.016 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:47.016 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_host": { 2024-05-09T01:01:47.016 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_host", 2024-05-09T01:01:47.016 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:47.016 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:47.016 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:47.016 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-09T01:01:47.016 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:47.016 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:47.017 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:47.017 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:47.017 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:47.017 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:47.017 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:47.017 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:47.017 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_port": { 2024-05-09T01:01:47.017 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_port", 2024-05-09T01:01:47.017 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-09T01:01:47.017 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:47.017 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:47.017 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10051", 2024-05-09T01:01:47.018 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:47.018 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:47.018 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:47.018 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:47.018 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:47.018 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:47.018 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:47.018 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:47.018 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_sender": { 2024-05-09T01:01:47.018 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_sender", 2024-05-09T01:01:47.018 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-09T01:01:47.019 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-09T01:01:47.019 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-09T01:01:47.019 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-09T01:01:47.019 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-09T01:01:47.019 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-09T01:01:47.019 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-09T01:01:47.019 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-09T01:01:47.019 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-09T01:01:47.019 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-09T01:01:47.019 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-09T01:01:47.019 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:47.019 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:47.020 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:47.020 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:47.020 INFO:teuthology.orchestra.run.smithi005.stdout: "services": { 2024-05-09T01:01:47.020 INFO:teuthology.orchestra.run.smithi005.stdout: "dashboard": "https://172.21.15.5:8443/" 2024-05-09T01:01:47.020 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:47.020 INFO:teuthology.orchestra.run.smithi005.stdout: "always_on_modules": { 2024-05-09T01:01:47.020 INFO:teuthology.orchestra.run.smithi005.stdout: "octopus": [ 2024-05-09T01:01:47.020 INFO:teuthology.orchestra.run.smithi005.stdout: "balancer", 2024-05-09T01:01:47.020 INFO:teuthology.orchestra.run.smithi005.stdout: "crash", 2024-05-09T01:01:47.020 INFO:teuthology.orchestra.run.smithi005.stdout: "devicehealth", 2024-05-09T01:01:47.020 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator", 2024-05-09T01:01:47.020 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscaler", 2024-05-09T01:01:47.020 INFO:teuthology.orchestra.run.smithi005.stdout: "progress", 2024-05-09T01:01:47.021 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_support", 2024-05-09T01:01:47.021 INFO:teuthology.orchestra.run.smithi005.stdout: "status", 2024-05-09T01:01:47.021 INFO:teuthology.orchestra.run.smithi005.stdout: "telemetry", 2024-05-09T01:01:47.021 INFO:teuthology.orchestra.run.smithi005.stdout: "volumes" 2024-05-09T01:01:47.021 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:47.021 INFO:teuthology.orchestra.run.smithi005.stdout: "pacific": [ 2024-05-09T01:01:47.021 INFO:teuthology.orchestra.run.smithi005.stdout: "balancer", 2024-05-09T01:01:47.021 INFO:teuthology.orchestra.run.smithi005.stdout: "crash", 2024-05-09T01:01:47.021 INFO:teuthology.orchestra.run.smithi005.stdout: "devicehealth", 2024-05-09T01:01:47.021 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator", 2024-05-09T01:01:47.021 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscaler", 2024-05-09T01:01:47.021 INFO:teuthology.orchestra.run.smithi005.stdout: "progress", 2024-05-09T01:01:47.022 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_support", 2024-05-09T01:01:47.022 INFO:teuthology.orchestra.run.smithi005.stdout: "status", 2024-05-09T01:01:47.022 INFO:teuthology.orchestra.run.smithi005.stdout: "telemetry", 2024-05-09T01:01:47.022 INFO:teuthology.orchestra.run.smithi005.stdout: "volumes" 2024-05-09T01:01:47.022 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:47.022 INFO:teuthology.orchestra.run.smithi005.stdout: "quincy": [ 2024-05-09T01:01:47.022 INFO:teuthology.orchestra.run.smithi005.stdout: "balancer", 2024-05-09T01:01:47.022 INFO:teuthology.orchestra.run.smithi005.stdout: "crash", 2024-05-09T01:01:47.022 INFO:teuthology.orchestra.run.smithi005.stdout: "devicehealth", 2024-05-09T01:01:47.022 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator", 2024-05-09T01:01:47.022 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscaler", 2024-05-09T01:01:47.023 INFO:teuthology.orchestra.run.smithi005.stdout: "progress", 2024-05-09T01:01:47.023 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_support", 2024-05-09T01:01:47.023 INFO:teuthology.orchestra.run.smithi005.stdout: "status", 2024-05-09T01:01:47.023 INFO:teuthology.orchestra.run.smithi005.stdout: "telemetry", 2024-05-09T01:01:47.023 INFO:teuthology.orchestra.run.smithi005.stdout: "volumes" 2024-05-09T01:01:47.023 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:47.023 INFO:teuthology.orchestra.run.smithi005.stdout: "reef": [ 2024-05-09T01:01:47.023 INFO:teuthology.orchestra.run.smithi005.stdout: "balancer", 2024-05-09T01:01:47.023 INFO:teuthology.orchestra.run.smithi005.stdout: "crash", 2024-05-09T01:01:47.023 INFO:teuthology.orchestra.run.smithi005.stdout: "devicehealth", 2024-05-09T01:01:47.023 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator", 2024-05-09T01:01:47.023 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscaler", 2024-05-09T01:01:47.024 INFO:teuthology.orchestra.run.smithi005.stdout: "progress", 2024-05-09T01:01:47.024 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_support", 2024-05-09T01:01:47.024 INFO:teuthology.orchestra.run.smithi005.stdout: "status", 2024-05-09T01:01:47.024 INFO:teuthology.orchestra.run.smithi005.stdout: "telemetry", 2024-05-09T01:01:47.024 INFO:teuthology.orchestra.run.smithi005.stdout: "volumes" 2024-05-09T01:01:47.024 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-09T01:01:47.024 INFO:teuthology.orchestra.run.smithi005.stdout: "squid": [ 2024-05-09T01:01:47.024 INFO:teuthology.orchestra.run.smithi005.stdout: "balancer", 2024-05-09T01:01:47.024 INFO:teuthology.orchestra.run.smithi005.stdout: "crash", 2024-05-09T01:01:47.024 INFO:teuthology.orchestra.run.smithi005.stdout: "devicehealth", 2024-05-09T01:01:47.024 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator", 2024-05-09T01:01:47.024 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscaler", 2024-05-09T01:01:47.025 INFO:teuthology.orchestra.run.smithi005.stdout: "progress", 2024-05-09T01:01:47.025 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_support", 2024-05-09T01:01:47.025 INFO:teuthology.orchestra.run.smithi005.stdout: "status", 2024-05-09T01:01:47.025 INFO:teuthology.orchestra.run.smithi005.stdout: "telemetry", 2024-05-09T01:01:47.025 INFO:teuthology.orchestra.run.smithi005.stdout: "volumes" 2024-05-09T01:01:47.025 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-09T01:01:47.025 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:47.025 INFO:teuthology.orchestra.run.smithi005.stdout: "last_failure_osd_epoch": 3, 2024-05-09T01:01:47.025 INFO:teuthology.orchestra.run.smithi005.stdout: "active_clients": [ 2024-05-09T01:01:47.025 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:47.025 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "devicehealth", 2024-05-09T01:01:47.025 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-09T01:01:47.026 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:47.026 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-09T01:01:47.026 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:0", 2024-05-09T01:01:47.026 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1108120909 2024-05-09T01:01:47.026 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:47.026 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-09T01:01:47.026 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:47.026 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:47.026 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "libcephsqlite", 2024-05-09T01:01:47.026 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-09T01:01:47.026 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:47.026 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-09T01:01:47.027 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:0", 2024-05-09T01:01:47.027 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 911031686 2024-05-09T01:01:47.027 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:47.027 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-09T01:01:47.027 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:47.027 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:47.027 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_support", 2024-05-09T01:01:47.027 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-09T01:01:47.027 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:47.027 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-09T01:01:47.027 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:0", 2024-05-09T01:01:47.027 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2452940029 2024-05-09T01:01:47.027 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:47.028 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-09T01:01:47.028 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:47.028 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:47.028 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "volumes", 2024-05-09T01:01:47.028 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-09T01:01:47.028 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-09T01:01:47.028 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-09T01:01:47.028 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:0", 2024-05-09T01:01:47.028 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 3323633009 2024-05-09T01:01:47.028 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:47.028 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-09T01:01:47.028 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:47.028 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-09T01:01:47.029 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-09T01:01:47.030 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:01:45 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/967937912' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T01:01:47.030 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:01:45 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/2651537516' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T01:01:47.030 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:01:45 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/1959256042' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T01:01:47.030 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:01:46 smithi131 ceph-mon[34547]: pgmap v833: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 69 MiB/s wr, 534 op/s 2024-05-09T01:01:47.030 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:01:46 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/308566076' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-09T01:01:47.030 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:01:46 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:01:47.031 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:01:46 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:01:47.031 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:01:46 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:01:47.031 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:01:46 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T01:01:47.031 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:01:45 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/967937912' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T01:01:47.031 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:01:45 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/2651537516' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T01:01:47.031 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:01:45 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/1959256042' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T01:01:47.031 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:01:46 smithi157 ceph-mon[34294]: pgmap v833: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 69 MiB/s wr, 534 op/s 2024-05-09T01:01:47.031 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:01:46 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/308566076' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-09T01:01:47.031 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:01:46 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:01:47.031 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:01:46 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:01:47.031 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:01:46 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:01:47.031 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:01:46 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T01:01:47.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:45.288+0000 7f5ba1bce640 1 -- 172.21.15.5:0/308566076 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f5b840b27f0 msgr2=0x7f5b840b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:47.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:45.288+0000 7f5ba1bce640 1 --2- 172.21.15.5:0/308566076 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f5b840b27f0 0x7f5b840b4d00 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c19b4a0 tx=0x7f5b90008040 comp rx=0 tx=0).stop 2024-05-09T01:01:47.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:45.288+0000 7f5ba1bce640 1 -- 172.21.15.5:0/308566076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b9c103c40 msgr2=0x7f5b9c19a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:01:47.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:45.288+0000 7f5ba1bce640 1 --2- 172.21.15.5:0/308566076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b9c103c40 0x7f5b9c19a400 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c032fd0 tx=0x7f5b8c0042e0 comp rx=0 tx=0).stop 2024-05-09T01:01:47.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:45.288+0000 7f5ba1bce640 1 -- 172.21.15.5:0/308566076 shutdown_connections 2024-05-09T01:01:47.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:45.288+0000 7f5ba1bce640 1 --2- 172.21.15.5:0/308566076 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f5b840b27f0 0x7f5b840b4d00 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:47.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:45.288+0000 7f5ba1bce640 1 --2- 172.21.15.5:0/308566076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b9c109b80 0x7f5b9c1a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:47.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:45.288+0000 7f5ba1bce640 1 --2- 172.21.15.5:0/308566076 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b9c104e80 0x7f5b9c19a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:47.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:45.288+0000 7f5ba1bce640 1 --2- 172.21.15.5:0/308566076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b9c103c40 0x7f5b9c19a400 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:01:47.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:45.288+0000 7f5ba1bce640 1 -- 172.21.15.5:0/308566076 >> 172.21.15.5:0/308566076 conn(0x7f5b9c0fee70 msgr2=0x7f5b9c105b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:01:47.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:45.288+0000 7f5ba1bce640 1 -- 172.21.15.5:0/308566076 shutdown_connections 2024-05-09T01:01:47.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:01:45.288+0000 7f5ba1bce640 1 -- 172.21.15.5:0/308566076 wait complete. 2024-05-09T01:01:47.040 DEBUG:tasks.check_counter:Getting stats from a 2024-05-09T01:01:47.040 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph daemon mds.a perf dump 2024-05-09T01:01:47.040 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:01:45 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/967937912' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T01:01:47.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:01:45 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/2651537516' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T01:01:47.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:01:45 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/1959256042' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T01:01:47.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:01:46 smithi005 ceph-mon[30365]: pgmap v833: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 69 MiB/s wr, 534 op/s 2024-05-09T01:01:47.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:01:46 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/308566076' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-09T01:01:47.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:01:46 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:01:47.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:01:46 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:01:47.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:01:46 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:01:47.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:01:46 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T01:01:47.339 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:01:47.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:01:47 smithi157 ceph-mon[34294]: pgmap v834: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 70 MiB/s wr, 656 op/s 2024-05-09T01:01:47.476 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:01:47 smithi005 ceph-mon[30365]: pgmap v834: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 70 MiB/s wr, 656 op/s 2024-05-09T01:01:47.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:01:47 smithi131 ceph-mon[34547]: pgmap v834: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 70 MiB/s wr, 656 op/s 2024-05-09T01:01:49.300 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-09T01:01:49.300 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T01:01:49.300 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 2129, 2024-05-09T01:01:49.300 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 2129, 2024-05-09T01:01:49.300 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 639517, 2024-05-09T01:01:49.300 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 676199, 2024-05-09T01:01:49.300 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 6, 2024-05-09T01:01:49.300 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 2, 2024-05-09T01:01:49.300 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.324414096, 2024-05-09T01:01:49.300 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.076090941, 2024-05-09T01:01:49.301 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.101097163, 2024-05-09T01:01:49.301 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.123269119, 2024-05-09T01:01:49.301 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:01:49.301 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 2128, 2024-05-09T01:01:49.301 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.288747380, 2024-05-09T01:01:49.301 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000135689 2024-05-09T01:01:49.301 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:49.301 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:01:49.301 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:49.301 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:49.301 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:49.301 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:49.301 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-05-09T01:01:49.302 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 448 2024-05-09T01:01:49.302 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:49.302 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T01:01:49.302 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 1267, 2024-05-09T01:01:49.302 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 1257, 2024-05-09T01:01:49.302 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 836088, 2024-05-09T01:01:49.302 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 486496, 2024-05-09T01:01:49.302 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 5, 2024-05-09T01:01:49.302 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 3, 2024-05-09T01:01:49.302 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.230013209, 2024-05-09T01:01:49.302 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.055887406, 2024-05-09T01:01:49.302 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.067528471, 2024-05-09T01:01:49.302 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.092342271, 2024-05-09T01:01:49.303 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:01:49.303 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 1256, 2024-05-09T01:01:49.303 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.166879517, 2024-05-09T01:01:49.303 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000132865 2024-05-09T01:01:49.303 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:49.303 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:01:49.303 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:49.303 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:49.303 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:49.303 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:49.303 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 600752, 2024-05-09T01:01:49.303 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 183056 2024-05-09T01:01:49.303 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:49.304 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T01:01:49.304 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 3, 2024-05-09T01:01:49.304 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 226, 2024-05-09T01:01:49.304 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 588, 2024-05-09T01:01:49.304 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 561710, 2024-05-09T01:01:49.304 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 6, 2024-05-09T01:01:49.304 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 3, 2024-05-09T01:01:49.304 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.022795850, 2024-05-09T01:01:49.304 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.017468276, 2024-05-09T01:01:49.304 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.002856364, 2024-05-09T01:01:49.304 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000022167, 2024-05-09T01:01:49.304 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:01:49.304 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 224, 2024-05-09T01:01:49.304 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.019943339, 2024-05-09T01:01:49.304 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000089032 2024-05-09T01:01:49.305 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:49.305 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:01:49.305 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:49.305 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:49.305 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:49.305 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:49.305 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-09T01:01:49.305 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 561392 2024-05-09T01:01:49.305 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:49.305 INFO:teuthology.orchestra.run.smithi005.stdout: "cct": { 2024-05-09T01:01:49.305 INFO:teuthology.orchestra.run.smithi005.stdout: "total_workers": 1, 2024-05-09T01:01:49.305 INFO:teuthology.orchestra.run.smithi005.stdout: "unhealthy_workers": 0 2024-05-09T01:01:49.306 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:49.306 INFO:teuthology.orchestra.run.smithi005.stdout: "finisher-MDSRank": { 2024-05-09T01:01:49.306 INFO:teuthology.orchestra.run.smithi005.stdout: "queue_len": 0, 2024-05-09T01:01:49.306 INFO:teuthology.orchestra.run.smithi005.stdout: "complete_latency": { 2024-05-09T01:01:49.306 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 5088, 2024-05-09T01:01:49.306 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.236367247, 2024-05-09T01:01:49.306 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000046455 2024-05-09T01:01:49.306 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:49.306 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:49.306 INFO:teuthology.orchestra.run.smithi005.stdout: "finisher-PurgeQueue": { 2024-05-09T01:01:49.306 INFO:teuthology.orchestra.run.smithi005.stdout: "queue_len": 0, 2024-05-09T01:01:49.306 INFO:teuthology.orchestra.run.smithi005.stdout: "complete_latency": { 2024-05-09T01:01:49.306 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:49.307 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:49.307 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:49.307 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:49.307 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:49.307 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": { 2024-05-09T01:01:49.307 INFO:teuthology.orchestra.run.smithi005.stdout: "request": 0, 2024-05-09T01:01:49.307 INFO:teuthology.orchestra.run.smithi005.stdout: "reply": 0, 2024-05-09T01:01:49.307 INFO:teuthology.orchestra.run.smithi005.stdout: "reply_latency": { 2024-05-09T01:01:49.307 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:49.307 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:49.307 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:49.307 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:49.307 INFO:teuthology.orchestra.run.smithi005.stdout: "slow_reply": 0, 2024-05-09T01:01:49.308 INFO:teuthology.orchestra.run.smithi005.stdout: "forward": 0, 2024-05-09T01:01:49.308 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_fetch_complete": 0, 2024-05-09T01:01:49.308 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_fetch_keys": 0, 2024-05-09T01:01:49.308 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_commit": 0, 2024-05-09T01:01:49.308 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_split": 0, 2024-05-09T01:01:49.308 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_merge": 0, 2024-05-09T01:01:49.308 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes": 0, 2024-05-09T01:01:49.308 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_top": 0, 2024-05-09T01:01:49.308 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_bottom": 0, 2024-05-09T01:01:49.308 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_pin_tail": 0, 2024-05-09T01:01:49.308 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_pinned": 0, 2024-05-09T01:01:49.308 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_expired": 0, 2024-05-09T01:01:49.308 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_with_caps": 0, 2024-05-09T01:01:49.308 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": 0, 2024-05-09T01:01:49.309 INFO:teuthology.orchestra.run.smithi005.stdout: "subtrees": 1, 2024-05-09T01:01:49.309 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse": 0, 2024-05-09T01:01:49.309 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_hit": 0, 2024-05-09T01:01:49.309 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_forward": 0, 2024-05-09T01:01:49.309 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_discover": 0, 2024-05-09T01:01:49.309 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_dir_fetch": 0, 2024-05-09T01:01:49.309 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_remote_ino": 0, 2024-05-09T01:01:49.309 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_lock": 0, 2024-05-09T01:01:49.309 INFO:teuthology.orchestra.run.smithi005.stdout: "load_cent": 0, 2024-05-09T01:01:49.309 INFO:teuthology.orchestra.run.smithi005.stdout: "q": 0, 2024-05-09T01:01:49.309 INFO:teuthology.orchestra.run.smithi005.stdout: "exported": 0, 2024-05-09T01:01:49.309 INFO:teuthology.orchestra.run.smithi005.stdout: "exported_inodes": 0, 2024-05-09T01:01:49.309 INFO:teuthology.orchestra.run.smithi005.stdout: "imported": 0, 2024-05-09T01:01:49.310 INFO:teuthology.orchestra.run.smithi005.stdout: "imported_inodes": 0, 2024-05-09T01:01:49.310 INFO:teuthology.orchestra.run.smithi005.stdout: "openino_dir_fetch": 0, 2024-05-09T01:01:49.310 INFO:teuthology.orchestra.run.smithi005.stdout: "openino_backtrace_fetch": 0, 2024-05-09T01:01:49.310 INFO:teuthology.orchestra.run.smithi005.stdout: "openino_peer_discover": 0, 2024-05-09T01:01:49.310 INFO:teuthology.orchestra.run.smithi005.stdout: "root_rfiles": 0, 2024-05-09T01:01:49.310 INFO:teuthology.orchestra.run.smithi005.stdout: "root_rbytes": 0, 2024-05-09T01:01:49.310 INFO:teuthology.orchestra.run.smithi005.stdout: "root_rsnaps": 0, 2024-05-09T01:01:49.310 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_backtrace_fetch": 0, 2024-05-09T01:01:49.310 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_set_tag": 0, 2024-05-09T01:01:49.310 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T01:01:49.310 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_inotable_repaired": 0, 2024-05-09T01:01:49.310 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_dir_inodes": 0, 2024-05-09T01:01:49.310 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T01:01:49.311 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T01:01:49.311 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_file_inodes": 0, 2024-05-09T01:01:49.311 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_inode_file_caps": 0, 2024-05-09T01:01:49.311 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_revoke": 0, 2024-05-09T01:01:49.311 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_grant": 0, 2024-05-09T01:01:49.311 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T01:01:49.311 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T01:01:49.311 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T01:01:49.311 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_caps": 0, 2024-05-09T01:01:49.311 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_caps_dirty": 0, 2024-05-09T01:01:49.311 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_cap_release": 0, 2024-05-09T01:01:49.311 INFO:teuthology.orchestra.run.smithi005.stdout: "process_request_cap_release": 0 2024-05-09T01:01:49.311 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:49.311 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_cache": { 2024-05-09T01:01:49.312 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_update": 0, 2024-05-09T01:01:49.312 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_update_receipt": 0, 2024-05-09T01:01:49.312 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_try_discover": 0, 2024-05-09T01:01:49.312 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_send_discover": 0, 2024-05-09T01:01:49.312 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_handle_discover": 0, 2024-05-09T01:01:49.312 INFO:teuthology.orchestra.run.smithi005.stdout: "num_strays": 0, 2024-05-09T01:01:49.312 INFO:teuthology.orchestra.run.smithi005.stdout: "num_strays_delayed": 0, 2024-05-09T01:01:49.312 INFO:teuthology.orchestra.run.smithi005.stdout: "num_strays_enqueuing": 0, 2024-05-09T01:01:49.312 INFO:teuthology.orchestra.run.smithi005.stdout: "strays_created": 0, 2024-05-09T01:01:49.312 INFO:teuthology.orchestra.run.smithi005.stdout: "strays_enqueued": 0, 2024-05-09T01:01:49.312 INFO:teuthology.orchestra.run.smithi005.stdout: "strays_reintegrated": 0, 2024-05-09T01:01:49.312 INFO:teuthology.orchestra.run.smithi005.stdout: "strays_migrated": 0, 2024-05-09T01:01:49.312 INFO:teuthology.orchestra.run.smithi005.stdout: "num_recovering_processing": 0, 2024-05-09T01:01:49.312 INFO:teuthology.orchestra.run.smithi005.stdout: "num_recovering_enqueued": 0, 2024-05-09T01:01:49.313 INFO:teuthology.orchestra.run.smithi005.stdout: "num_recovering_prioritized": 0, 2024-05-09T01:01:49.313 INFO:teuthology.orchestra.run.smithi005.stdout: "recovery_started": 0, 2024-05-09T01:01:49.313 INFO:teuthology.orchestra.run.smithi005.stdout: "recovery_completed": 0, 2024-05-09T01:01:49.313 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_quiesce_path": 0, 2024-05-09T01:01:49.313 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_quiesce_inode": 0, 2024-05-09T01:01:49.313 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T01:01:49.313 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_exportdir": 0, 2024-05-09T01:01:49.313 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_flush": 0, 2024-05-09T01:01:49.313 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_fragmentdir": 0, 2024-05-09T01:01:49.313 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_fragstats": 0, 2024-05-09T01:01:49.313 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_inodestats": 0 2024-05-09T01:01:49.313 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:49.313 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_log": { 2024-05-09T01:01:49.314 INFO:teuthology.orchestra.run.smithi005.stdout: "evlrg": 0, 2024-05-09T01:01:49.314 INFO:teuthology.orchestra.run.smithi005.stdout: "evadd": 0, 2024-05-09T01:01:49.314 INFO:teuthology.orchestra.run.smithi005.stdout: "evex": 0, 2024-05-09T01:01:49.314 INFO:teuthology.orchestra.run.smithi005.stdout: "evtrm": 0, 2024-05-09T01:01:49.314 INFO:teuthology.orchestra.run.smithi005.stdout: "ev": 5, 2024-05-09T01:01:49.314 INFO:teuthology.orchestra.run.smithi005.stdout: "evexg": 0, 2024-05-09T01:01:49.314 INFO:teuthology.orchestra.run.smithi005.stdout: "evexd": 0, 2024-05-09T01:01:49.314 INFO:teuthology.orchestra.run.smithi005.stdout: "segadd": 0, 2024-05-09T01:01:49.314 INFO:teuthology.orchestra.run.smithi005.stdout: "segex": 0, 2024-05-09T01:01:49.314 INFO:teuthology.orchestra.run.smithi005.stdout: "segtrm": 0, 2024-05-09T01:01:49.314 INFO:teuthology.orchestra.run.smithi005.stdout: "seg": 2, 2024-05-09T01:01:49.314 INFO:teuthology.orchestra.run.smithi005.stdout: "segmjr": 2, 2024-05-09T01:01:49.314 INFO:teuthology.orchestra.run.smithi005.stdout: "segexg": 0, 2024-05-09T01:01:49.315 INFO:teuthology.orchestra.run.smithi005.stdout: "segexd": 0, 2024-05-09T01:01:49.315 INFO:teuthology.orchestra.run.smithi005.stdout: "expos": 4194304, 2024-05-09T01:01:49.315 INFO:teuthology.orchestra.run.smithi005.stdout: "wrpos": 4195732, 2024-05-09T01:01:49.315 INFO:teuthology.orchestra.run.smithi005.stdout: "rdpos": 4195424, 2024-05-09T01:01:49.315 INFO:teuthology.orchestra.run.smithi005.stdout: "jlat": { 2024-05-09T01:01:49.315 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:49.315 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:49.315 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:49.315 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:49.315 INFO:teuthology.orchestra.run.smithi005.stdout: "replayed": 5 2024-05-09T01:01:49.315 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:49.315 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_mem": { 2024-05-09T01:01:49.315 INFO:teuthology.orchestra.run.smithi005.stdout: "ino": 3, 2024-05-09T01:01:49.315 INFO:teuthology.orchestra.run.smithi005.stdout: "ino+": 3, 2024-05-09T01:01:49.316 INFO:teuthology.orchestra.run.smithi005.stdout: "ino-": 0, 2024-05-09T01:01:49.316 INFO:teuthology.orchestra.run.smithi005.stdout: "dir": 1, 2024-05-09T01:01:49.316 INFO:teuthology.orchestra.run.smithi005.stdout: "dir+": 1, 2024-05-09T01:01:49.316 INFO:teuthology.orchestra.run.smithi005.stdout: "dir-": 0, 2024-05-09T01:01:49.316 INFO:teuthology.orchestra.run.smithi005.stdout: "dn": 0, 2024-05-09T01:01:49.316 INFO:teuthology.orchestra.run.smithi005.stdout: "dn+": 0, 2024-05-09T01:01:49.316 INFO:teuthology.orchestra.run.smithi005.stdout: "dn-": 0, 2024-05-09T01:01:49.316 INFO:teuthology.orchestra.run.smithi005.stdout: "cap": 0, 2024-05-09T01:01:49.316 INFO:teuthology.orchestra.run.smithi005.stdout: "cap+": 0, 2024-05-09T01:01:49.316 INFO:teuthology.orchestra.run.smithi005.stdout: "cap-": 0, 2024-05-09T01:01:49.316 INFO:teuthology.orchestra.run.smithi005.stdout: "rss": 122004, 2024-05-09T01:01:49.316 INFO:teuthology.orchestra.run.smithi005.stdout: "heap": 215440 2024-05-09T01:01:49.316 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:49.316 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_server": { 2024-05-09T01:01:49.317 INFO:teuthology.orchestra.run.smithi005.stdout: "dispatch_client_request": 0, 2024-05-09T01:01:49.317 INFO:teuthology.orchestra.run.smithi005.stdout: "dispatch_server_request": 0, 2024-05-09T01:01:49.317 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_request": 0, 2024-05-09T01:01:49.317 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_session": 0, 2024-05-09T01:01:49.317 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_peer_request": 0, 2024-05-09T01:01:49.317 INFO:teuthology.orchestra.run.smithi005.stdout: "req_create_latency": { 2024-05-09T01:01:49.317 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:49.317 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:49.317 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:49.317 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:49.317 INFO:teuthology.orchestra.run.smithi005.stdout: "req_getattr_latency": { 2024-05-09T01:01:49.317 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:49.317 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:49.317 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:49.318 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:49.318 INFO:teuthology.orchestra.run.smithi005.stdout: "req_getfilelock_latency": { 2024-05-09T01:01:49.318 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:49.318 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:49.318 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:49.318 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:49.318 INFO:teuthology.orchestra.run.smithi005.stdout: "req_link_latency": { 2024-05-09T01:01:49.318 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:49.318 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:49.318 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:49.318 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:49.318 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookup_latency": { 2024-05-09T01:01:49.318 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:49.319 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:49.319 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:49.319 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:49.319 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookuphash_latency": { 2024-05-09T01:01:49.319 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:49.319 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:49.319 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:49.319 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:49.319 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookupino_latency": { 2024-05-09T01:01:49.319 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:49.319 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:49.319 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:49.319 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:49.319 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookupname_latency": { 2024-05-09T01:01:49.320 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:49.320 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:49.320 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:49.320 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:49.320 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookupparent_latency": { 2024-05-09T01:01:49.320 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:49.320 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:49.320 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:49.320 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:49.320 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookupsnap_latency": { 2024-05-09T01:01:49.320 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:49.320 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:49.320 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:49.320 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:49.320 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lssnap_latency": { 2024-05-09T01:01:49.321 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:49.321 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:49.321 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:49.321 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:49.321 INFO:teuthology.orchestra.run.smithi005.stdout: "req_mkdir_latency": { 2024-05-09T01:01:49.321 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:49.322 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:49.322 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:49.322 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:49.322 INFO:teuthology.orchestra.run.smithi005.stdout: "req_mknod_latency": { 2024-05-09T01:01:49.322 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:49.322 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:49.322 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:49.323 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:49.323 INFO:teuthology.orchestra.run.smithi005.stdout: "req_mksnap_latency": { 2024-05-09T01:01:49.323 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:49.323 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:49.323 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:49.323 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:49.323 INFO:teuthology.orchestra.run.smithi005.stdout: "req_open_latency": { 2024-05-09T01:01:49.323 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:49.323 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:49.323 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:49.323 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:49.323 INFO:teuthology.orchestra.run.smithi005.stdout: "req_readdir_latency": { 2024-05-09T01:01:49.323 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:49.324 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:49.324 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:49.324 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:49.324 INFO:teuthology.orchestra.run.smithi005.stdout: "req_rename_latency": { 2024-05-09T01:01:49.324 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:49.324 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:49.324 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:49.324 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:49.324 INFO:teuthology.orchestra.run.smithi005.stdout: "req_renamesnap_latency": { 2024-05-09T01:01:49.324 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:49.324 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:49.324 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:49.324 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:49.325 INFO:teuthology.orchestra.run.smithi005.stdout: "req_snapdiff_latency": { 2024-05-09T01:01:49.325 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:49.325 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:49.325 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:49.325 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:49.325 INFO:teuthology.orchestra.run.smithi005.stdout: "req_rmdir_latency": { 2024-05-09T01:01:49.325 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:49.325 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:49.325 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:49.325 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:49.325 INFO:teuthology.orchestra.run.smithi005.stdout: "req_rmsnap_latency": { 2024-05-09T01:01:49.325 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:49.325 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:49.326 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:49.326 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:49.326 INFO:teuthology.orchestra.run.smithi005.stdout: "req_rmxattr_latency": { 2024-05-09T01:01:49.326 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:49.326 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:49.326 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:49.326 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:49.326 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setattr_latency": { 2024-05-09T01:01:49.326 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:49.326 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:49.326 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:49.326 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:49.326 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setdirlayout_latency": { 2024-05-09T01:01:49.327 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:49.327 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:49.327 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:49.327 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:49.327 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setfilelock_latency": { 2024-05-09T01:01:49.327 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:49.327 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:49.327 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:49.327 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:49.327 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setlayout_latency": { 2024-05-09T01:01:49.327 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:49.327 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:49.327 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:49.327 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:49.328 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setxattr_latency": { 2024-05-09T01:01:49.328 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:49.328 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:49.328 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:49.328 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:49.328 INFO:teuthology.orchestra.run.smithi005.stdout: "req_symlink_latency": { 2024-05-09T01:01:49.328 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:49.328 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:49.328 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:49.328 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:49.328 INFO:teuthology.orchestra.run.smithi005.stdout: "req_unlink_latency": { 2024-05-09T01:01:49.328 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:49.328 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:49.328 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:49.329 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:49.329 INFO:teuthology.orchestra.run.smithi005.stdout: "cap_revoke_eviction": 0, 2024-05-09T01:01:49.329 INFO:teuthology.orchestra.run.smithi005.stdout: "cap_acquisition_throttle": 0, 2024-05-09T01:01:49.329 INFO:teuthology.orchestra.run.smithi005.stdout: "req_getvxattr_latency": { 2024-05-09T01:01:49.329 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:49.329 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:49.329 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:49.329 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:49.329 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:49.329 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_sessions": { 2024-05-09T01:01:49.329 INFO:teuthology.orchestra.run.smithi005.stdout: "session_count": 1, 2024-05-09T01:01:49.329 INFO:teuthology.orchestra.run.smithi005.stdout: "session_add": 2, 2024-05-09T01:01:49.329 INFO:teuthology.orchestra.run.smithi005.stdout: "session_remove": 1, 2024-05-09T01:01:49.330 INFO:teuthology.orchestra.run.smithi005.stdout: "sessions_open": 1, 2024-05-09T01:01:49.330 INFO:teuthology.orchestra.run.smithi005.stdout: "sessions_stale": 0, 2024-05-09T01:01:49.330 INFO:teuthology.orchestra.run.smithi005.stdout: "total_load": 0, 2024-05-09T01:01:49.330 INFO:teuthology.orchestra.run.smithi005.stdout: "average_load": 0, 2024-05-09T01:01:49.330 INFO:teuthology.orchestra.run.smithi005.stdout: "avg_session_uptime": 2087, 2024-05-09T01:01:49.330 INFO:teuthology.orchestra.run.smithi005.stdout: "mdthresh_evicted": 0 2024-05-09T01:01:49.330 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:49.330 INFO:teuthology.orchestra.run.smithi005.stdout: "mempool": { 2024-05-09T01:01:49.330 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_bytes": 0, 2024-05-09T01:01:49.330 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_items": 0, 2024-05-09T01:01:49.330 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T01:01:49.330 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_items": 0, 2024-05-09T01:01:49.330 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T01:01:49.331 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_items": 0, 2024-05-09T01:01:49.331 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T01:01:49.331 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T01:01:49.331 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T01:01:49.331 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T01:01:49.331 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T01:01:49.331 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_items": 0, 2024-05-09T01:01:49.331 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T01:01:49.331 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T01:01:49.331 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_bytes": 0, 2024-05-09T01:01:49.331 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_items": 0, 2024-05-09T01:01:49.331 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_bytes": 0, 2024-05-09T01:01:49.331 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_items": 0, 2024-05-09T01:01:49.331 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T01:01:49.332 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T01:01:49.332 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T01:01:49.332 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T01:01:49.332 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T01:01:49.332 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_items": 0, 2024-05-09T01:01:49.332 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_bytes": 0, 2024-05-09T01:01:49.332 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_items": 0, 2024-05-09T01:01:49.332 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T01:01:49.332 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T01:01:49.332 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_bytes": 0, 2024-05-09T01:01:49.332 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_items": 0, 2024-05-09T01:01:49.332 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_bytes": 0, 2024-05-09T01:01:49.332 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_items": 0, 2024-05-09T01:01:49.332 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T01:01:49.333 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_items": 0, 2024-05-09T01:01:49.333 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T01:01:49.333 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_items": 0, 2024-05-09T01:01:49.333 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_bytes": 190145, 2024-05-09T01:01:49.333 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_items": 52, 2024-05-09T01:01:49.333 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_bytes": 0, 2024-05-09T01:01:49.333 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_items": 0, 2024-05-09T01:01:49.333 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_bytes": 0, 2024-05-09T01:01:49.333 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_items": 0, 2024-05-09T01:01:49.333 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_bytes": 0, 2024-05-09T01:01:49.333 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_items": 0, 2024-05-09T01:01:49.333 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_bytes": 0, 2024-05-09T01:01:49.333 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_items": 0, 2024-05-09T01:01:49.333 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_bytes": 7736, 2024-05-09T01:01:49.334 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_items": 164, 2024-05-09T01:01:49.334 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T01:01:49.334 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_items": 0, 2024-05-09T01:01:49.334 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_bytes": 0, 2024-05-09T01:01:49.334 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_items": 0, 2024-05-09T01:01:49.334 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_bytes": 8512, 2024-05-09T01:01:49.334 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_items": 13, 2024-05-09T01:01:49.334 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_bytes": 0, 2024-05-09T01:01:49.334 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_items": 0, 2024-05-09T01:01:49.334 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_bytes": 0, 2024-05-09T01:01:49.334 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_items": 0 2024-05-09T01:01:49.334 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:49.334 INFO:teuthology.orchestra.run.smithi005.stdout: "objecter": { 2024-05-09T01:01:49.335 INFO:teuthology.orchestra.run.smithi005.stdout: "op_active": 0, 2024-05-09T01:01:49.335 INFO:teuthology.orchestra.run.smithi005.stdout: "op_laggy": 0, 2024-05-09T01:01:49.335 INFO:teuthology.orchestra.run.smithi005.stdout: "op_send": 3080, 2024-05-09T01:01:49.335 INFO:teuthology.orchestra.run.smithi005.stdout: "op_send_bytes": 21, 2024-05-09T01:01:49.335 INFO:teuthology.orchestra.run.smithi005.stdout: "op_resend": 3, 2024-05-09T01:01:49.335 INFO:teuthology.orchestra.run.smithi005.stdout: "op_reply": 3080, 2024-05-09T01:01:49.335 INFO:teuthology.orchestra.run.smithi005.stdout: "op_latency": { 2024-05-09T01:01:49.335 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 3080, 2024-05-09T01:01:49.335 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 7.537845724, 2024-05-09T01:01:49.335 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.002447352 2024-05-09T01:01:49.335 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:49.335 INFO:teuthology.orchestra.run.smithi005.stdout: "op_inflight": 0, 2024-05-09T01:01:49.335 INFO:teuthology.orchestra.run.smithi005.stdout: "oplen_avg": { 2024-05-09T01:01:49.335 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 3080, 2024-05-09T01:01:49.336 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 3081 2024-05-09T01:01:49.336 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:49.336 INFO:teuthology.orchestra.run.smithi005.stdout: "op": 3080, 2024-05-09T01:01:49.336 INFO:teuthology.orchestra.run.smithi005.stdout: "op_r": 3080, 2024-05-09T01:01:49.336 INFO:teuthology.orchestra.run.smithi005.stdout: "op_w": 0, 2024-05-09T01:01:49.336 INFO:teuthology.orchestra.run.smithi005.stdout: "op_rmw": 0, 2024-05-09T01:01:49.336 INFO:teuthology.orchestra.run.smithi005.stdout: "op_pg": 0, 2024-05-09T01:01:49.336 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_stat": 2011, 2024-05-09T01:01:49.336 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_create": 0, 2024-05-09T01:01:49.336 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_read": 1067, 2024-05-09T01:01:49.336 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_write": 0, 2024-05-09T01:01:49.336 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_writefull": 0, 2024-05-09T01:01:49.336 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_writesame": 0, 2024-05-09T01:01:49.336 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_append": 0, 2024-05-09T01:01:49.337 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_zero": 0, 2024-05-09T01:01:49.337 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_truncate": 0, 2024-05-09T01:01:49.337 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_delete": 0, 2024-05-09T01:01:49.337 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_mapext": 0, 2024-05-09T01:01:49.337 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_sparse_read": 0, 2024-05-09T01:01:49.337 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_clonerange": 0, 2024-05-09T01:01:49.337 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_getxattr": 1, 2024-05-09T01:01:49.337 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_setxattr": 0, 2024-05-09T01:01:49.337 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_cmpxattr": 0, 2024-05-09T01:01:49.337 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_rmxattr": 0, 2024-05-09T01:01:49.337 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_resetxattrs": 0, 2024-05-09T01:01:49.337 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_call": 0, 2024-05-09T01:01:49.337 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_watch": 0, 2024-05-09T01:01:49.338 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_notify": 0, 2024-05-09T01:01:49.338 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T01:01:49.338 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_pgls": 0, 2024-05-09T01:01:49.338 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_pgls_filter": 0, 2024-05-09T01:01:49.338 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_other": 0, 2024-05-09T01:01:49.338 INFO:teuthology.orchestra.run.smithi005.stdout: "linger_active": 0, 2024-05-09T01:01:49.338 INFO:teuthology.orchestra.run.smithi005.stdout: "linger_send": 0, 2024-05-09T01:01:49.338 INFO:teuthology.orchestra.run.smithi005.stdout: "linger_resend": 0, 2024-05-09T01:01:49.338 INFO:teuthology.orchestra.run.smithi005.stdout: "linger_ping": 0, 2024-05-09T01:01:49.338 INFO:teuthology.orchestra.run.smithi005.stdout: "poolop_active": 0, 2024-05-09T01:01:49.338 INFO:teuthology.orchestra.run.smithi005.stdout: "poolop_send": 0, 2024-05-09T01:01:49.338 INFO:teuthology.orchestra.run.smithi005.stdout: "poolop_resend": 0, 2024-05-09T01:01:49.338 INFO:teuthology.orchestra.run.smithi005.stdout: "poolstat_active": 0, 2024-05-09T01:01:49.339 INFO:teuthology.orchestra.run.smithi005.stdout: "poolstat_send": 0, 2024-05-09T01:01:49.339 INFO:teuthology.orchestra.run.smithi005.stdout: "poolstat_resend": 0, 2024-05-09T01:01:49.339 INFO:teuthology.orchestra.run.smithi005.stdout: "statfs_active": 0, 2024-05-09T01:01:49.339 INFO:teuthology.orchestra.run.smithi005.stdout: "statfs_send": 0, 2024-05-09T01:01:49.339 INFO:teuthology.orchestra.run.smithi005.stdout: "statfs_resend": 0, 2024-05-09T01:01:49.339 INFO:teuthology.orchestra.run.smithi005.stdout: "command_active": 0, 2024-05-09T01:01:49.339 INFO:teuthology.orchestra.run.smithi005.stdout: "command_send": 0, 2024-05-09T01:01:49.339 INFO:teuthology.orchestra.run.smithi005.stdout: "command_resend": 0, 2024-05-09T01:01:49.339 INFO:teuthology.orchestra.run.smithi005.stdout: "map_epoch": 82, 2024-05-09T01:01:49.339 INFO:teuthology.orchestra.run.smithi005.stdout: "map_full": 0, 2024-05-09T01:01:49.339 INFO:teuthology.orchestra.run.smithi005.stdout: "map_inc": 4, 2024-05-09T01:01:49.339 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_sessions": 5, 2024-05-09T01:01:49.339 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_session_open": 7, 2024-05-09T01:01:49.339 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_session_close": 2, 2024-05-09T01:01:49.340 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_laggy": 0, 2024-05-09T01:01:49.340 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_wr": 0, 2024-05-09T01:01:49.340 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_rd": 2, 2024-05-09T01:01:49.340 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_del": 0 2024-05-09T01:01:49.340 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:49.340 INFO:teuthology.orchestra.run.smithi005.stdout: "oft": { 2024-05-09T01:01:49.340 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_total_objs": 0, 2024-05-09T01:01:49.340 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_total_kv_pairs": 0, 2024-05-09T01:01:49.340 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_total_updates": 0, 2024-05-09T01:01:49.340 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_total_removes": 0 2024-05-09T01:01:49.340 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:49.340 INFO:teuthology.orchestra.run.smithi005.stdout: "osd-slow-ops": { 2024-05-09T01:01:49.340 INFO:teuthology.orchestra.run.smithi005.stdout: "slow_ops_count": 0 2024-05-09T01:01:49.340 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:49.341 INFO:teuthology.orchestra.run.smithi005.stdout: "purge_queue": { 2024-05-09T01:01:49.341 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executing_ops": 0, 2024-05-09T01:01:49.341 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executing_ops_high_water": 0, 2024-05-09T01:01:49.341 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executing": 0, 2024-05-09T01:01:49.341 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executing_high_water": 0, 2024-05-09T01:01:49.341 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executed_ops": 0, 2024-05-09T01:01:49.341 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executed": 0, 2024-05-09T01:01:49.341 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_item_in_journal": 0 2024-05-09T01:01:49.341 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:49.341 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T01:01:49.341 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-05-09T01:01:49.341 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 104857600, 2024-05-09T01:01:49.341 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-05-09T01:01:49.342 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 3395, 2024-05-09T01:01:49.342 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 1151581, 2024-05-09T01:01:49.342 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-05-09T01:01:49.342 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 3395, 2024-05-09T01:01:49.342 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2024-05-09T01:01:49.342 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2024-05-09T01:01:49.342 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 3395, 2024-05-09T01:01:49.342 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 1151581, 2024-05-09T01:01:49.342 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-05-09T01:01:49.342 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:49.342 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:49.342 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:49.342 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:49.342 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:49.343 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-objecter_bytes": { 2024-05-09T01:01:49.343 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-05-09T01:01:49.343 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 104857600, 2024-05-09T01:01:49.343 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-05-09T01:01:49.343 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 0, 2024-05-09T01:01:49.343 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 0, 2024-05-09T01:01:49.343 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-05-09T01:01:49.343 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 0, 2024-05-09T01:01:49.343 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 3080, 2024-05-09T01:01:49.343 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 1433, 2024-05-09T01:01:49.343 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 3080, 2024-05-09T01:01:49.343 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 1433, 2024-05-09T01:01:49.343 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-05-09T01:01:49.343 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:49.344 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:49.344 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:49.344 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:49.344 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:49.344 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-objecter_ops": { 2024-05-09T01:01:49.344 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-05-09T01:01:49.344 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 1024, 2024-05-09T01:01:49.344 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-05-09T01:01:49.344 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 0, 2024-05-09T01:01:49.344 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 0, 2024-05-09T01:01:49.344 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-05-09T01:01:49.344 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 0, 2024-05-09T01:01:49.344 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 3080, 2024-05-09T01:01:49.344 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 3080, 2024-05-09T01:01:49.345 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 3080, 2024-05-09T01:01:49.345 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 3080, 2024-05-09T01:01:49.345 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-05-09T01:01:49.345 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:49.345 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:49.345 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:49.345 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:49.345 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:49.345 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-write_buf_throttle": { 2024-05-09T01:01:49.345 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-05-09T01:01:49.345 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 3758096384, 2024-05-09T01:01:49.345 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-05-09T01:01:49.345 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 0, 2024-05-09T01:01:49.346 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 0, 2024-05-09T01:01:49.346 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-05-09T01:01:49.346 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 0, 2024-05-09T01:01:49.346 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2024-05-09T01:01:49.346 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2024-05-09T01:01:49.346 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 0, 2024-05-09T01:01:49.346 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 0, 2024-05-09T01:01:49.346 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-05-09T01:01:49.346 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:49.346 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:49.346 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:49.346 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:49.346 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:49.347 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-write_buf_throttle-0x55c49a9ae8f0": { 2024-05-09T01:01:49.347 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-05-09T01:01:49.347 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 3758096384, 2024-05-09T01:01:49.347 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-05-09T01:01:49.347 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 0, 2024-05-09T01:01:49.347 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 0, 2024-05-09T01:01:49.347 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-05-09T01:01:49.347 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 0, 2024-05-09T01:01:49.347 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2024-05-09T01:01:49.347 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2024-05-09T01:01:49.347 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 0, 2024-05-09T01:01:49.347 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 0, 2024-05-09T01:01:49.347 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-05-09T01:01:49.347 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:49.348 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:49.348 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:49.348 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:49.348 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:49.348 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-09T01:01:49.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:01:49 smithi157 ceph-mon[34294]: pgmap v835: 97 pgs: 97 active+clean; 8.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 61 MiB/s wr, 639 op/s 2024-05-09T01:01:50.040 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:01:49 smithi005 ceph-mon[30365]: pgmap v835: 97 pgs: 97 active+clean; 8.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 61 MiB/s wr, 639 op/s 2024-05-09T01:01:50.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:01:49 smithi131 ceph-mon[34547]: pgmap v835: 97 pgs: 97 active+clean; 8.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 61 MiB/s wr, 639 op/s 2024-05-09T01:01:50.239 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-05-09T01:01:50.239 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-05-09T01:01:50.240 DEBUG:tasks.check_counter:Getting stats from d 2024-05-09T01:01:50.240 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph daemon mds.d perf dump 2024-05-09T01:01:50.497 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:01:51.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:01:51 smithi157 ceph-mon[34294]: pgmap v836: 97 pgs: 97 active+clean; 7.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 44 MiB/s wr, 593 op/s 2024-05-09T01:01:51.952 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:01:51 smithi005 ceph-mon[30365]: pgmap v836: 97 pgs: 97 active+clean; 7.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 44 MiB/s wr, 593 op/s 2024-05-09T01:01:52.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:01:51 smithi131 ceph-mon[34547]: pgmap v836: 97 pgs: 97 active+clean; 7.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 44 MiB/s wr, 593 op/s 2024-05-09T01:01:52.543 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-09T01:01:52.544 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T01:01:52.544 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 0, 2024-05-09T01:01:52.544 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 2, 2024-05-09T01:01:52.544 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 0, 2024-05-09T01:01:52.544 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 0, 2024-05-09T01:01:52.544 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 4, 2024-05-09T01:01:52.544 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 0, 2024-05-09T01:01:52.544 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.001642535, 2024-05-09T01:01:52.544 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.000000000, 2024-05-09T01:01:52.544 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.000378246, 2024-05-09T01:01:52.544 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T01:01:52.544 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:01:52.545 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:52.545 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:52.545 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:52.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:52.545 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:01:52.545 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:52.545 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:52.545 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:52.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:52.545 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T01:01:52.545 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T01:01:52.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:52.546 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T01:01:52.546 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 2, 2024-05-09T01:01:52.546 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 230, 2024-05-09T01:01:52.546 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 320, 2024-05-09T01:01:52.546 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 197888, 2024-05-09T01:01:52.546 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 4, 2024-05-09T01:01:52.546 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 2, 2024-05-09T01:01:52.546 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.025254432, 2024-05-09T01:01:52.546 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.017567091, 2024-05-09T01:01:52.546 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.002473251, 2024-05-09T01:01:52.546 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T01:01:52.546 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:01:52.546 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 228, 2024-05-09T01:01:52.546 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.021568675, 2024-05-09T01:01:52.547 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000094599 2024-05-09T01:01:52.547 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:52.547 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:01:52.547 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:52.547 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:52.547 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:52.547 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:52.547 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-09T01:01:52.547 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 197888 2024-05-09T01:01:52.547 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:52.547 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T01:01:52.547 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 308, 2024-05-09T01:01:52.547 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 303, 2024-05-09T01:01:52.548 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 541136, 2024-05-09T01:01:52.548 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 185952, 2024-05-09T01:01:52.548 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 2, 2024-05-09T01:01:52.548 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 1, 2024-05-09T01:01:52.548 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.059658336, 2024-05-09T01:01:52.548 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.023936927, 2024-05-09T01:01:52.548 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.024239611, 2024-05-09T01:01:52.548 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.009962131, 2024-05-09T01:01:52.548 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:01:52.548 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 303, 2024-05-09T01:01:52.548 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.045775565, 2024-05-09T01:01:52.548 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000151074 2024-05-09T01:01:52.548 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:52.549 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:01:52.549 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:52.549 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:52.549 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:52.549 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:52.549 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 541136, 2024-05-09T01:01:52.549 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 185952 2024-05-09T01:01:52.549 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:52.549 INFO:teuthology.orchestra.run.smithi005.stdout: "cct": { 2024-05-09T01:01:52.549 INFO:teuthology.orchestra.run.smithi005.stdout: "total_workers": 0, 2024-05-09T01:01:52.549 INFO:teuthology.orchestra.run.smithi005.stdout: "unhealthy_workers": 0 2024-05-09T01:01:52.549 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:52.549 INFO:teuthology.orchestra.run.smithi005.stdout: "mempool": { 2024-05-09T01:01:52.549 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_bytes": 0, 2024-05-09T01:01:52.549 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_items": 0, 2024-05-09T01:01:52.550 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T01:01:52.550 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_items": 0, 2024-05-09T01:01:52.550 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T01:01:52.550 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_items": 0, 2024-05-09T01:01:52.550 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T01:01:52.550 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T01:01:52.550 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T01:01:52.550 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T01:01:52.550 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T01:01:52.550 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_items": 0, 2024-05-09T01:01:52.550 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T01:01:52.550 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T01:01:52.550 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_bytes": 0, 2024-05-09T01:01:52.551 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_items": 0, 2024-05-09T01:01:52.551 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_bytes": 0, 2024-05-09T01:01:52.551 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_items": 0, 2024-05-09T01:01:52.551 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T01:01:52.551 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T01:01:52.551 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T01:01:52.551 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T01:01:52.551 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T01:01:52.551 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_items": 0, 2024-05-09T01:01:52.551 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_bytes": 0, 2024-05-09T01:01:52.551 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_items": 0, 2024-05-09T01:01:52.551 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T01:01:52.551 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T01:01:52.551 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_bytes": 0, 2024-05-09T01:01:52.551 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_items": 0, 2024-05-09T01:01:52.552 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_bytes": 0, 2024-05-09T01:01:52.552 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_items": 0, 2024-05-09T01:01:52.552 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T01:01:52.552 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_items": 0, 2024-05-09T01:01:52.552 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T01:01:52.552 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_items": 0, 2024-05-09T01:01:52.552 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_bytes": 148744, 2024-05-09T01:01:52.552 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_items": 30, 2024-05-09T01:01:52.552 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_bytes": 0, 2024-05-09T01:01:52.552 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_items": 0, 2024-05-09T01:01:52.552 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_bytes": 0, 2024-05-09T01:01:52.552 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_items": 0, 2024-05-09T01:01:52.552 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_bytes": 0, 2024-05-09T01:01:52.552 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_items": 0, 2024-05-09T01:01:52.553 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_bytes": 0, 2024-05-09T01:01:52.553 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_items": 0, 2024-05-09T01:01:52.553 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_bytes": 0, 2024-05-09T01:01:52.553 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_items": 0, 2024-05-09T01:01:52.553 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T01:01:52.553 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_items": 0, 2024-05-09T01:01:52.553 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_bytes": 0, 2024-05-09T01:01:52.553 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_items": 0, 2024-05-09T01:01:52.553 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_bytes": 928, 2024-05-09T01:01:52.553 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_items": 2, 2024-05-09T01:01:52.553 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_bytes": 0, 2024-05-09T01:01:52.553 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_items": 0, 2024-05-09T01:01:52.553 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_bytes": 0, 2024-05-09T01:01:52.553 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_items": 0 2024-05-09T01:01:52.553 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:52.554 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T01:01:52.554 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-05-09T01:01:52.554 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 104857600, 2024-05-09T01:01:52.554 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-05-09T01:01:52.554 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 306, 2024-05-09T01:01:52.554 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 495221, 2024-05-09T01:01:52.554 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-05-09T01:01:52.554 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 306, 2024-05-09T01:01:52.554 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2024-05-09T01:01:52.554 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2024-05-09T01:01:52.554 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 306, 2024-05-09T01:01:52.554 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 495221, 2024-05-09T01:01:52.555 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-05-09T01:01:52.555 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:52.555 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:52.555 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:52.555 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:52.555 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:52.555 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-09T01:01:53.426 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-05-09T01:01:53.426 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-05-09T01:01:53.426 DEBUG:tasks.check_counter:Getting stats from g 2024-05-09T01:01:53.427 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph daemon mds.g perf dump 2024-05-09T01:01:53.682 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:01:53.790 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:01:53 smithi005 ceph-mon[30365]: pgmap v837: 97 pgs: 97 active+clean; 6.1 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 34 MiB/s wr, 573 op/s 2024-05-09T01:01:53.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:01:53 smithi157 ceph-mon[34294]: pgmap v837: 97 pgs: 97 active+clean; 6.1 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 34 MiB/s wr, 573 op/s 2024-05-09T01:01:54.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:01:53 smithi131 ceph-mon[34547]: pgmap v837: 97 pgs: 97 active+clean; 6.1 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 34 MiB/s wr, 573 op/s 2024-05-09T01:01:55.674 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-09T01:01:55.675 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T01:01:55.675 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 1384, 2024-05-09T01:01:55.675 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 1373, 2024-05-09T01:01:55.675 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 962080, 2024-05-09T01:01:55.676 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 523614, 2024-05-09T01:01:55.676 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 5, 2024-05-09T01:01:55.676 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 1, 2024-05-09T01:01:55.676 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.275150090, 2024-05-09T01:01:55.676 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.089021507, 2024-05-09T01:01:55.676 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.073057302, 2024-05-09T01:01:55.677 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.097894228, 2024-05-09T01:01:55.677 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:01:55.677 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 1373, 2024-05-09T01:01:55.677 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.270281904, 2024-05-09T01:01:55.677 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000196854 2024-05-09T01:01:55.677 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:55.678 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:01:55.678 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:55.678 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:55.678 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:55.678 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:55.678 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 604576, 2024-05-09T01:01:55.679 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 184944 2024-05-09T01:01:55.679 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:55.679 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T01:01:55.679 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 1067, 2024-05-09T01:01:55.679 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 1292, 2024-05-09T01:01:55.679 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 280072, 2024-05-09T01:01:55.679 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 814069, 2024-05-09T01:01:55.680 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 5, 2024-05-09T01:01:55.680 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 4, 2024-05-09T01:01:55.680 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.179302775, 2024-05-09T01:01:55.680 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.058094688, 2024-05-09T01:01:55.680 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.049073016, 2024-05-09T01:01:55.680 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.059425974, 2024-05-09T01:01:55.680 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:01:55.681 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 1290, 2024-05-09T01:01:55.681 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.156927375, 2024-05-09T01:01:55.681 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000121649 2024-05-09T01:01:55.681 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:55.681 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:01:55.681 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:55.682 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:55.682 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:55.682 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:55.682 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-09T01:01:55.682 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 475024 2024-05-09T01:01:55.682 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:55.682 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T01:01:55.682 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 4, 2024-05-09T01:01:55.682 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 6, 2024-05-09T01:01:55.682 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 1629, 2024-05-09T01:01:55.682 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 1284, 2024-05-09T01:01:55.682 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 8, 2024-05-09T01:01:55.683 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 4, 2024-05-09T01:01:55.683 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.003942557, 2024-05-09T01:01:55.683 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.000086801, 2024-05-09T01:01:55.683 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.003325867, 2024-05-09T01:01:55.683 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000124117, 2024-05-09T01:01:55.683 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:01:55.683 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 4, 2024-05-09T01:01:55.683 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.002737286, 2024-05-09T01:01:55.683 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000684321 2024-05-09T01:01:55.683 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:55.683 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:01:55.683 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:55.683 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:55.684 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:55.684 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:55.684 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T01:01:55.684 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T01:01:55.684 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:55.684 INFO:teuthology.orchestra.run.smithi005.stdout: "cct": { 2024-05-09T01:01:55.684 INFO:teuthology.orchestra.run.smithi005.stdout: "total_workers": 1, 2024-05-09T01:01:55.684 INFO:teuthology.orchestra.run.smithi005.stdout: "unhealthy_workers": 0 2024-05-09T01:01:55.684 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:55.684 INFO:teuthology.orchestra.run.smithi005.stdout: "finisher-MDSRank": { 2024-05-09T01:01:55.684 INFO:teuthology.orchestra.run.smithi005.stdout: "queue_len": 0, 2024-05-09T01:01:55.684 INFO:teuthology.orchestra.run.smithi005.stdout: "complete_latency": { 2024-05-09T01:01:55.684 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 4261, 2024-05-09T01:01:55.684 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.194223360, 2024-05-09T01:01:55.685 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000045581 2024-05-09T01:01:55.685 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:55.685 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:55.685 INFO:teuthology.orchestra.run.smithi005.stdout: "finisher-PurgeQueue": { 2024-05-09T01:01:55.685 INFO:teuthology.orchestra.run.smithi005.stdout: "queue_len": 0, 2024-05-09T01:01:55.685 INFO:teuthology.orchestra.run.smithi005.stdout: "complete_latency": { 2024-05-09T01:01:55.685 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:55.685 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:55.685 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:55.685 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:55.685 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:55.686 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": { 2024-05-09T01:01:55.686 INFO:teuthology.orchestra.run.smithi005.stdout: "request": 0, 2024-05-09T01:01:55.686 INFO:teuthology.orchestra.run.smithi005.stdout: "reply": 0, 2024-05-09T01:01:55.686 INFO:teuthology.orchestra.run.smithi005.stdout: "reply_latency": { 2024-05-09T01:01:55.686 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:55.686 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:55.686 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:55.686 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:55.686 INFO:teuthology.orchestra.run.smithi005.stdout: "slow_reply": 0, 2024-05-09T01:01:55.686 INFO:teuthology.orchestra.run.smithi005.stdout: "forward": 0, 2024-05-09T01:01:55.686 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_fetch_complete": 0, 2024-05-09T01:01:55.686 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_fetch_keys": 0, 2024-05-09T01:01:55.687 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_commit": 0, 2024-05-09T01:01:55.687 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_split": 0, 2024-05-09T01:01:55.687 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_merge": 0, 2024-05-09T01:01:55.687 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes": 0, 2024-05-09T01:01:55.687 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_top": 0, 2024-05-09T01:01:55.687 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_bottom": 0, 2024-05-09T01:01:55.687 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_pin_tail": 0, 2024-05-09T01:01:55.687 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_pinned": 0, 2024-05-09T01:01:55.687 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_expired": 0, 2024-05-09T01:01:55.687 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_with_caps": 0, 2024-05-09T01:01:55.687 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": 0, 2024-05-09T01:01:55.687 INFO:teuthology.orchestra.run.smithi005.stdout: "subtrees": 1, 2024-05-09T01:01:55.687 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse": 0, 2024-05-09T01:01:55.688 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_hit": 0, 2024-05-09T01:01:55.688 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_forward": 0, 2024-05-09T01:01:55.688 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_discover": 0, 2024-05-09T01:01:55.688 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_dir_fetch": 0, 2024-05-09T01:01:55.688 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_remote_ino": 0, 2024-05-09T01:01:55.688 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_lock": 0, 2024-05-09T01:01:55.688 INFO:teuthology.orchestra.run.smithi005.stdout: "load_cent": 0, 2024-05-09T01:01:55.688 INFO:teuthology.orchestra.run.smithi005.stdout: "q": 0, 2024-05-09T01:01:55.688 INFO:teuthology.orchestra.run.smithi005.stdout: "exported": 0, 2024-05-09T01:01:55.688 INFO:teuthology.orchestra.run.smithi005.stdout: "exported_inodes": 0, 2024-05-09T01:01:55.688 INFO:teuthology.orchestra.run.smithi005.stdout: "imported": 0, 2024-05-09T01:01:55.688 INFO:teuthology.orchestra.run.smithi005.stdout: "imported_inodes": 0, 2024-05-09T01:01:55.688 INFO:teuthology.orchestra.run.smithi005.stdout: "openino_dir_fetch": 0, 2024-05-09T01:01:55.688 INFO:teuthology.orchestra.run.smithi005.stdout: "openino_backtrace_fetch": 0, 2024-05-09T01:01:55.689 INFO:teuthology.orchestra.run.smithi005.stdout: "openino_peer_discover": 0, 2024-05-09T01:01:55.689 INFO:teuthology.orchestra.run.smithi005.stdout: "root_rfiles": 0, 2024-05-09T01:01:55.689 INFO:teuthology.orchestra.run.smithi005.stdout: "root_rbytes": 0, 2024-05-09T01:01:55.689 INFO:teuthology.orchestra.run.smithi005.stdout: "root_rsnaps": 0, 2024-05-09T01:01:55.689 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_backtrace_fetch": 0, 2024-05-09T01:01:55.689 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_set_tag": 0, 2024-05-09T01:01:55.689 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T01:01:55.689 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_inotable_repaired": 0, 2024-05-09T01:01:55.689 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_dir_inodes": 0, 2024-05-09T01:01:55.689 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T01:01:55.689 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T01:01:55.689 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_file_inodes": 0, 2024-05-09T01:01:55.689 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_inode_file_caps": 0, 2024-05-09T01:01:55.690 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_revoke": 0, 2024-05-09T01:01:55.690 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_grant": 0, 2024-05-09T01:01:55.690 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T01:01:55.690 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T01:01:55.690 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T01:01:55.690 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_caps": 0, 2024-05-09T01:01:55.690 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_caps_dirty": 0, 2024-05-09T01:01:55.690 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_cap_release": 0, 2024-05-09T01:01:55.690 INFO:teuthology.orchestra.run.smithi005.stdout: "process_request_cap_release": 0 2024-05-09T01:01:55.690 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:55.690 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_cache": { 2024-05-09T01:01:55.691 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_update": 0, 2024-05-09T01:01:55.691 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_update_receipt": 0, 2024-05-09T01:01:55.691 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_try_discover": 0, 2024-05-09T01:01:55.691 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_send_discover": 0, 2024-05-09T01:01:55.691 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_handle_discover": 0, 2024-05-09T01:01:55.691 INFO:teuthology.orchestra.run.smithi005.stdout: "num_strays": 0, 2024-05-09T01:01:55.691 INFO:teuthology.orchestra.run.smithi005.stdout: "num_strays_delayed": 0, 2024-05-09T01:01:55.691 INFO:teuthology.orchestra.run.smithi005.stdout: "num_strays_enqueuing": 0, 2024-05-09T01:01:55.691 INFO:teuthology.orchestra.run.smithi005.stdout: "strays_created": 0, 2024-05-09T01:01:55.691 INFO:teuthology.orchestra.run.smithi005.stdout: "strays_enqueued": 0, 2024-05-09T01:01:55.691 INFO:teuthology.orchestra.run.smithi005.stdout: "strays_reintegrated": 0, 2024-05-09T01:01:55.691 INFO:teuthology.orchestra.run.smithi005.stdout: "strays_migrated": 0, 2024-05-09T01:01:55.691 INFO:teuthology.orchestra.run.smithi005.stdout: "num_recovering_processing": 0, 2024-05-09T01:01:55.691 INFO:teuthology.orchestra.run.smithi005.stdout: "num_recovering_enqueued": 0, 2024-05-09T01:01:55.692 INFO:teuthology.orchestra.run.smithi005.stdout: "num_recovering_prioritized": 0, 2024-05-09T01:01:55.692 INFO:teuthology.orchestra.run.smithi005.stdout: "recovery_started": 0, 2024-05-09T01:01:55.692 INFO:teuthology.orchestra.run.smithi005.stdout: "recovery_completed": 0, 2024-05-09T01:01:55.692 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_quiesce_path": 0, 2024-05-09T01:01:55.692 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_quiesce_inode": 0, 2024-05-09T01:01:55.692 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T01:01:55.692 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_exportdir": 0, 2024-05-09T01:01:55.692 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_flush": 0, 2024-05-09T01:01:55.692 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_fragmentdir": 0, 2024-05-09T01:01:55.692 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_fragstats": 0, 2024-05-09T01:01:55.692 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_inodestats": 0 2024-05-09T01:01:55.692 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:55.692 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_log": { 2024-05-09T01:01:55.693 INFO:teuthology.orchestra.run.smithi005.stdout: "evlrg": 0, 2024-05-09T01:01:55.693 INFO:teuthology.orchestra.run.smithi005.stdout: "evadd": 0, 2024-05-09T01:01:55.693 INFO:teuthology.orchestra.run.smithi005.stdout: "evex": 0, 2024-05-09T01:01:55.693 INFO:teuthology.orchestra.run.smithi005.stdout: "evtrm": 0, 2024-05-09T01:01:55.693 INFO:teuthology.orchestra.run.smithi005.stdout: "ev": 2, 2024-05-09T01:01:55.693 INFO:teuthology.orchestra.run.smithi005.stdout: "evexg": 0, 2024-05-09T01:01:55.693 INFO:teuthology.orchestra.run.smithi005.stdout: "evexd": 0, 2024-05-09T01:01:55.693 INFO:teuthology.orchestra.run.smithi005.stdout: "segadd": 0, 2024-05-09T01:01:55.693 INFO:teuthology.orchestra.run.smithi005.stdout: "segex": 0, 2024-05-09T01:01:55.693 INFO:teuthology.orchestra.run.smithi005.stdout: "segtrm": 0, 2024-05-09T01:01:55.693 INFO:teuthology.orchestra.run.smithi005.stdout: "seg": 2, 2024-05-09T01:01:55.693 INFO:teuthology.orchestra.run.smithi005.stdout: "segmjr": 2, 2024-05-09T01:01:55.693 INFO:teuthology.orchestra.run.smithi005.stdout: "segexg": 0, 2024-05-09T01:01:55.694 INFO:teuthology.orchestra.run.smithi005.stdout: "segexd": 0, 2024-05-09T01:01:55.694 INFO:teuthology.orchestra.run.smithi005.stdout: "expos": 4194304, 2024-05-09T01:01:55.694 INFO:teuthology.orchestra.run.smithi005.stdout: "wrpos": 4194892, 2024-05-09T01:01:55.694 INFO:teuthology.orchestra.run.smithi005.stdout: "rdpos": 4194352, 2024-05-09T01:01:55.694 INFO:teuthology.orchestra.run.smithi005.stdout: "jlat": { 2024-05-09T01:01:55.694 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:55.694 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:55.694 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:55.694 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:55.694 INFO:teuthology.orchestra.run.smithi005.stdout: "replayed": 2 2024-05-09T01:01:55.694 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:55.694 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_mem": { 2024-05-09T01:01:55.694 INFO:teuthology.orchestra.run.smithi005.stdout: "ino": 3, 2024-05-09T01:01:55.695 INFO:teuthology.orchestra.run.smithi005.stdout: "ino+": 3, 2024-05-09T01:01:55.695 INFO:teuthology.orchestra.run.smithi005.stdout: "ino-": 0, 2024-05-09T01:01:55.695 INFO:teuthology.orchestra.run.smithi005.stdout: "dir": 1, 2024-05-09T01:01:55.695 INFO:teuthology.orchestra.run.smithi005.stdout: "dir+": 1, 2024-05-09T01:01:55.695 INFO:teuthology.orchestra.run.smithi005.stdout: "dir-": 0, 2024-05-09T01:01:55.695 INFO:teuthology.orchestra.run.smithi005.stdout: "dn": 0, 2024-05-09T01:01:55.695 INFO:teuthology.orchestra.run.smithi005.stdout: "dn+": 0, 2024-05-09T01:01:55.695 INFO:teuthology.orchestra.run.smithi005.stdout: "dn-": 0, 2024-05-09T01:01:55.695 INFO:teuthology.orchestra.run.smithi005.stdout: "cap": 0, 2024-05-09T01:01:55.695 INFO:teuthology.orchestra.run.smithi005.stdout: "cap+": 0, 2024-05-09T01:01:55.695 INFO:teuthology.orchestra.run.smithi005.stdout: "cap-": 0, 2024-05-09T01:01:55.695 INFO:teuthology.orchestra.run.smithi005.stdout: "rss": 119032, 2024-05-09T01:01:55.695 INFO:teuthology.orchestra.run.smithi005.stdout: "heap": 215440 2024-05-09T01:01:55.695 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:55.695 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_server": { 2024-05-09T01:01:55.696 INFO:teuthology.orchestra.run.smithi005.stdout: "dispatch_client_request": 0, 2024-05-09T01:01:55.696 INFO:teuthology.orchestra.run.smithi005.stdout: "dispatch_server_request": 0, 2024-05-09T01:01:55.696 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_request": 0, 2024-05-09T01:01:55.696 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_session": 0, 2024-05-09T01:01:55.696 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_peer_request": 0, 2024-05-09T01:01:55.696 INFO:teuthology.orchestra.run.smithi005.stdout: "req_create_latency": { 2024-05-09T01:01:55.696 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:55.696 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:55.696 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:55.696 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:55.696 INFO:teuthology.orchestra.run.smithi005.stdout: "req_getattr_latency": { 2024-05-09T01:01:55.696 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:55.696 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:55.696 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:55.697 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:55.697 INFO:teuthology.orchestra.run.smithi005.stdout: "req_getfilelock_latency": { 2024-05-09T01:01:55.697 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:55.697 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:55.697 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:55.697 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:55.697 INFO:teuthology.orchestra.run.smithi005.stdout: "req_link_latency": { 2024-05-09T01:01:55.697 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:55.697 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:55.697 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:55.697 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:55.697 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookup_latency": { 2024-05-09T01:01:55.697 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:55.697 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:55.697 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:55.698 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:55.698 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookuphash_latency": { 2024-05-09T01:01:55.698 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:55.698 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:55.698 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:55.698 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:55.698 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookupino_latency": { 2024-05-09T01:01:55.698 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:55.698 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:55.698 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:55.698 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:55.698 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookupname_latency": { 2024-05-09T01:01:55.698 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:55.699 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:55.699 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:55.699 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:55.699 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookupparent_latency": { 2024-05-09T01:01:55.699 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:55.699 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:55.699 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:55.699 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:55.699 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookupsnap_latency": { 2024-05-09T01:01:55.699 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:55.699 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:55.699 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:55.699 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:55.699 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lssnap_latency": { 2024-05-09T01:01:55.699 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:55.700 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:55.700 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:55.700 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:55.700 INFO:teuthology.orchestra.run.smithi005.stdout: "req_mkdir_latency": { 2024-05-09T01:01:55.700 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:55.700 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:55.701 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:55.701 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:55.701 INFO:teuthology.orchestra.run.smithi005.stdout: "req_mknod_latency": { 2024-05-09T01:01:55.701 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:55.701 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:55.701 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:55.701 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:55.701 INFO:teuthology.orchestra.run.smithi005.stdout: "req_mksnap_latency": { 2024-05-09T01:01:55.701 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:55.701 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:55.701 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:55.701 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:55.701 INFO:teuthology.orchestra.run.smithi005.stdout: "req_open_latency": { 2024-05-09T01:01:55.702 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:55.702 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:55.702 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:55.702 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:55.702 INFO:teuthology.orchestra.run.smithi005.stdout: "req_readdir_latency": { 2024-05-09T01:01:55.702 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:55.702 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:55.702 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:55.702 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:55.702 INFO:teuthology.orchestra.run.smithi005.stdout: "req_rename_latency": { 2024-05-09T01:01:55.702 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:55.702 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:55.703 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:55.703 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:55.703 INFO:teuthology.orchestra.run.smithi005.stdout: "req_renamesnap_latency": { 2024-05-09T01:01:55.703 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:55.703 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:55.703 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:55.703 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:55.703 INFO:teuthology.orchestra.run.smithi005.stdout: "req_snapdiff_latency": { 2024-05-09T01:01:55.703 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:55.703 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:55.703 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:55.703 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:55.703 INFO:teuthology.orchestra.run.smithi005.stdout: "req_rmdir_latency": { 2024-05-09T01:01:55.703 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:55.704 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:55.704 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:55.704 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:55.704 INFO:teuthology.orchestra.run.smithi005.stdout: "req_rmsnap_latency": { 2024-05-09T01:01:55.704 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:55.704 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:55.704 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:55.704 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:55.704 INFO:teuthology.orchestra.run.smithi005.stdout: "req_rmxattr_latency": { 2024-05-09T01:01:55.704 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:55.704 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:55.704 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:55.704 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:55.705 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setattr_latency": { 2024-05-09T01:01:55.705 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:55.705 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:55.705 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:55.705 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:55.705 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setdirlayout_latency": { 2024-05-09T01:01:55.705 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:55.705 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:55.705 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:55.705 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:55.705 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setfilelock_latency": { 2024-05-09T01:01:55.705 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:55.705 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:55.705 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:55.706 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:55.706 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setlayout_latency": { 2024-05-09T01:01:55.706 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:55.706 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:55.706 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:55.706 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:55.706 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setxattr_latency": { 2024-05-09T01:01:55.706 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:55.706 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:55.706 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:55.706 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:55.706 INFO:teuthology.orchestra.run.smithi005.stdout: "req_symlink_latency": { 2024-05-09T01:01:55.706 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:55.707 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:55.707 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:55.707 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:55.707 INFO:teuthology.orchestra.run.smithi005.stdout: "req_unlink_latency": { 2024-05-09T01:01:55.707 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:55.707 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:55.707 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:55.707 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:55.707 INFO:teuthology.orchestra.run.smithi005.stdout: "cap_revoke_eviction": 0, 2024-05-09T01:01:55.707 INFO:teuthology.orchestra.run.smithi005.stdout: "cap_acquisition_throttle": 0, 2024-05-09T01:01:55.708 INFO:teuthology.orchestra.run.smithi005.stdout: "req_getvxattr_latency": { 2024-05-09T01:01:55.708 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:55.708 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:55.708 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:55.708 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:55.708 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:55.708 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_sessions": { 2024-05-09T01:01:55.708 INFO:teuthology.orchestra.run.smithi005.stdout: "session_count": 0, 2024-05-09T01:01:55.708 INFO:teuthology.orchestra.run.smithi005.stdout: "session_add": 0, 2024-05-09T01:01:55.708 INFO:teuthology.orchestra.run.smithi005.stdout: "session_remove": 0, 2024-05-09T01:01:55.708 INFO:teuthology.orchestra.run.smithi005.stdout: "sessions_open": 0, 2024-05-09T01:01:55.708 INFO:teuthology.orchestra.run.smithi005.stdout: "sessions_stale": 0, 2024-05-09T01:01:55.708 INFO:teuthology.orchestra.run.smithi005.stdout: "total_load": 0, 2024-05-09T01:01:55.708 INFO:teuthology.orchestra.run.smithi005.stdout: "average_load": 0, 2024-05-09T01:01:55.709 INFO:teuthology.orchestra.run.smithi005.stdout: "avg_session_uptime": 0, 2024-05-09T01:01:55.709 INFO:teuthology.orchestra.run.smithi005.stdout: "mdthresh_evicted": 0 2024-05-09T01:01:55.709 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:55.709 INFO:teuthology.orchestra.run.smithi005.stdout: "mempool": { 2024-05-09T01:01:55.709 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_bytes": 0, 2024-05-09T01:01:55.709 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_items": 0, 2024-05-09T01:01:55.709 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T01:01:55.709 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_items": 0, 2024-05-09T01:01:55.709 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T01:01:55.709 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_items": 0, 2024-05-09T01:01:55.709 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T01:01:55.710 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T01:01:55.710 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T01:01:55.710 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T01:01:55.710 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T01:01:55.710 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_items": 0, 2024-05-09T01:01:55.710 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T01:01:55.710 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T01:01:55.710 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_bytes": 0, 2024-05-09T01:01:55.710 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_items": 0, 2024-05-09T01:01:55.710 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_bytes": 0, 2024-05-09T01:01:55.710 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_items": 0, 2024-05-09T01:01:55.710 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T01:01:55.710 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T01:01:55.711 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T01:01:55.711 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T01:01:55.711 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T01:01:55.711 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_items": 0, 2024-05-09T01:01:55.711 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_bytes": 0, 2024-05-09T01:01:55.711 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_items": 0, 2024-05-09T01:01:55.711 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T01:01:55.711 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T01:01:55.711 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_bytes": 0, 2024-05-09T01:01:55.711 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_items": 0, 2024-05-09T01:01:55.711 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_bytes": 0, 2024-05-09T01:01:55.711 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_items": 0, 2024-05-09T01:01:55.711 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T01:01:55.712 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_items": 0, 2024-05-09T01:01:55.712 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T01:01:55.712 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_items": 0, 2024-05-09T01:01:55.712 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_bytes": 198264, 2024-05-09T01:01:55.712 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_items": 57, 2024-05-09T01:01:55.712 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_bytes": 0, 2024-05-09T01:01:55.712 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_items": 0, 2024-05-09T01:01:55.712 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_bytes": 0, 2024-05-09T01:01:55.712 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_items": 0, 2024-05-09T01:01:55.712 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_bytes": 0, 2024-05-09T01:01:55.712 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_items": 0, 2024-05-09T01:01:55.712 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_bytes": 0, 2024-05-09T01:01:55.712 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_items": 0, 2024-05-09T01:01:55.712 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_bytes": 7736, 2024-05-09T01:01:55.712 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_items": 164, 2024-05-09T01:01:55.713 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T01:01:55.713 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_items": 0, 2024-05-09T01:01:55.713 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_bytes": 0, 2024-05-09T01:01:55.713 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_items": 0, 2024-05-09T01:01:55.713 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_bytes": 8512, 2024-05-09T01:01:55.713 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_items": 13, 2024-05-09T01:01:55.713 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_bytes": 0, 2024-05-09T01:01:55.713 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_items": 0, 2024-05-09T01:01:55.714 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_bytes": 0, 2024-05-09T01:01:55.714 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_items": 0 2024-05-09T01:01:55.714 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:55.714 INFO:teuthology.orchestra.run.smithi005.stdout: "objecter": { 2024-05-09T01:01:55.714 INFO:teuthology.orchestra.run.smithi005.stdout: "op_active": 1, 2024-05-09T01:01:55.714 INFO:teuthology.orchestra.run.smithi005.stdout: "op_laggy": 0, 2024-05-09T01:01:55.714 INFO:teuthology.orchestra.run.smithi005.stdout: "op_send": 2135, 2024-05-09T01:01:55.714 INFO:teuthology.orchestra.run.smithi005.stdout: "op_send_bytes": 21, 2024-05-09T01:01:55.715 INFO:teuthology.orchestra.run.smithi005.stdout: "op_resend": 3, 2024-05-09T01:01:55.715 INFO:teuthology.orchestra.run.smithi005.stdout: "op_reply": 2134, 2024-05-09T01:01:55.715 INFO:teuthology.orchestra.run.smithi005.stdout: "op_latency": { 2024-05-09T01:01:55.715 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 2134, 2024-05-09T01:01:55.715 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 7.529845550, 2024-05-09T01:01:55.715 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.003528512 2024-05-09T01:01:55.715 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:55.715 INFO:teuthology.orchestra.run.smithi005.stdout: "op_inflight": 0, 2024-05-09T01:01:55.716 INFO:teuthology.orchestra.run.smithi005.stdout: "oplen_avg": { 2024-05-09T01:01:55.716 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 2135, 2024-05-09T01:01:55.716 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 2136 2024-05-09T01:01:55.716 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:55.716 INFO:teuthology.orchestra.run.smithi005.stdout: "op": 2135, 2024-05-09T01:01:55.716 INFO:teuthology.orchestra.run.smithi005.stdout: "op_r": 2135, 2024-05-09T01:01:55.716 INFO:teuthology.orchestra.run.smithi005.stdout: "op_w": 0, 2024-05-09T01:01:55.716 INFO:teuthology.orchestra.run.smithi005.stdout: "op_rmw": 0, 2024-05-09T01:01:55.717 INFO:teuthology.orchestra.run.smithi005.stdout: "op_pg": 0, 2024-05-09T01:01:55.717 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_stat": 1064, 2024-05-09T01:01:55.717 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_create": 0, 2024-05-09T01:01:55.717 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_read": 1069, 2024-05-09T01:01:55.717 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_write": 0, 2024-05-09T01:01:55.717 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_writefull": 0, 2024-05-09T01:01:55.718 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_writesame": 0, 2024-05-09T01:01:55.718 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_append": 0, 2024-05-09T01:01:55.718 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_zero": 0, 2024-05-09T01:01:55.718 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_truncate": 0, 2024-05-09T01:01:55.718 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_delete": 0, 2024-05-09T01:01:55.718 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_mapext": 0, 2024-05-09T01:01:55.719 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_sparse_read": 0, 2024-05-09T01:01:55.719 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_clonerange": 0, 2024-05-09T01:01:55.719 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_getxattr": 1, 2024-05-09T01:01:55.719 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_setxattr": 0, 2024-05-09T01:01:55.719 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_cmpxattr": 0, 2024-05-09T01:01:55.719 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_rmxattr": 0, 2024-05-09T01:01:55.719 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_resetxattrs": 0, 2024-05-09T01:01:55.720 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_call": 0, 2024-05-09T01:01:55.720 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_watch": 0, 2024-05-09T01:01:55.720 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_notify": 0, 2024-05-09T01:01:55.720 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T01:01:55.720 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_pgls": 0, 2024-05-09T01:01:55.720 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_pgls_filter": 0, 2024-05-09T01:01:55.720 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_other": 0, 2024-05-09T01:01:55.720 INFO:teuthology.orchestra.run.smithi005.stdout: "linger_active": 0, 2024-05-09T01:01:55.721 INFO:teuthology.orchestra.run.smithi005.stdout: "linger_send": 0, 2024-05-09T01:01:55.721 INFO:teuthology.orchestra.run.smithi005.stdout: "linger_resend": 0, 2024-05-09T01:01:55.721 INFO:teuthology.orchestra.run.smithi005.stdout: "linger_ping": 0, 2024-05-09T01:01:55.721 INFO:teuthology.orchestra.run.smithi005.stdout: "poolop_active": 0, 2024-05-09T01:01:55.721 INFO:teuthology.orchestra.run.smithi005.stdout: "poolop_send": 0, 2024-05-09T01:01:55.721 INFO:teuthology.orchestra.run.smithi005.stdout: "poolop_resend": 0, 2024-05-09T01:01:55.721 INFO:teuthology.orchestra.run.smithi005.stdout: "poolstat_active": 0, 2024-05-09T01:01:55.721 INFO:teuthology.orchestra.run.smithi005.stdout: "poolstat_send": 0, 2024-05-09T01:01:55.722 INFO:teuthology.orchestra.run.smithi005.stdout: "poolstat_resend": 0, 2024-05-09T01:01:55.722 INFO:teuthology.orchestra.run.smithi005.stdout: "statfs_active": 0, 2024-05-09T01:01:55.722 INFO:teuthology.orchestra.run.smithi005.stdout: "statfs_send": 0, 2024-05-09T01:01:55.722 INFO:teuthology.orchestra.run.smithi005.stdout: "statfs_resend": 0, 2024-05-09T01:01:55.722 INFO:teuthology.orchestra.run.smithi005.stdout: "command_active": 0, 2024-05-09T01:01:55.722 INFO:teuthology.orchestra.run.smithi005.stdout: "command_send": 0, 2024-05-09T01:01:55.723 INFO:teuthology.orchestra.run.smithi005.stdout: "command_resend": 0, 2024-05-09T01:01:55.723 INFO:teuthology.orchestra.run.smithi005.stdout: "map_epoch": 82, 2024-05-09T01:01:55.723 INFO:teuthology.orchestra.run.smithi005.stdout: "map_full": 0, 2024-05-09T01:01:55.723 INFO:teuthology.orchestra.run.smithi005.stdout: "map_inc": 4, 2024-05-09T01:01:55.723 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_sessions": 6, 2024-05-09T01:01:55.723 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_session_open": 9, 2024-05-09T01:01:55.724 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_session_close": 3, 2024-05-09T01:01:55.724 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_laggy": 0, 2024-05-09T01:01:55.724 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_wr": 0, 2024-05-09T01:01:55.724 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_rd": 2, 2024-05-09T01:01:55.724 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_del": 0 2024-05-09T01:01:55.724 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:55.724 INFO:teuthology.orchestra.run.smithi005.stdout: "oft": { 2024-05-09T01:01:55.725 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_total_objs": 0, 2024-05-09T01:01:55.725 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_total_kv_pairs": 0, 2024-05-09T01:01:55.725 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_total_updates": 0, 2024-05-09T01:01:55.725 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_total_removes": 0 2024-05-09T01:01:55.725 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:55.725 INFO:teuthology.orchestra.run.smithi005.stdout: "osd-slow-ops": { 2024-05-09T01:01:55.725 INFO:teuthology.orchestra.run.smithi005.stdout: "slow_ops_count": 0 2024-05-09T01:01:55.725 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:55.725 INFO:teuthology.orchestra.run.smithi005.stdout: "purge_queue": { 2024-05-09T01:01:55.725 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executing_ops": 0, 2024-05-09T01:01:55.725 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executing_ops_high_water": 0, 2024-05-09T01:01:55.725 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executing": 0, 2024-05-09T01:01:55.726 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executing_high_water": 0, 2024-05-09T01:01:55.726 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executed_ops": 0, 2024-05-09T01:01:55.726 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executed": 0, 2024-05-09T01:01:55.726 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_item_in_journal": 0 2024-05-09T01:01:55.726 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:55.726 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T01:01:55.726 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-05-09T01:01:55.726 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 104857600, 2024-05-09T01:01:55.726 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-05-09T01:01:55.726 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 2452, 2024-05-09T01:01:55.726 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 1004262, 2024-05-09T01:01:55.726 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-05-09T01:01:55.726 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 2452, 2024-05-09T01:01:55.727 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2024-05-09T01:01:55.727 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2024-05-09T01:01:55.727 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 2452, 2024-05-09T01:01:55.727 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 1004262, 2024-05-09T01:01:55.727 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-05-09T01:01:55.727 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:55.727 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:55.727 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:55.727 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:55.727 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:55.727 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-objecter_bytes": { 2024-05-09T01:01:55.727 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-05-09T01:01:55.727 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 104857600, 2024-05-09T01:01:55.728 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-05-09T01:01:55.728 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 0, 2024-05-09T01:01:55.728 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 0, 2024-05-09T01:01:55.728 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-05-09T01:01:55.728 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 0, 2024-05-09T01:01:55.728 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 2136, 2024-05-09T01:01:55.728 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 593, 2024-05-09T01:01:55.728 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 2135, 2024-05-09T01:01:55.728 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 593, 2024-05-09T01:01:55.728 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-05-09T01:01:55.728 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:55.728 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:55.728 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:55.728 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:55.728 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:55.729 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-objecter_ops": { 2024-05-09T01:01:55.729 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 1, 2024-05-09T01:01:55.729 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 1024, 2024-05-09T01:01:55.729 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-05-09T01:01:55.729 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 0, 2024-05-09T01:01:55.729 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 0, 2024-05-09T01:01:55.729 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-05-09T01:01:55.729 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 0, 2024-05-09T01:01:55.729 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 2136, 2024-05-09T01:01:55.729 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 2136, 2024-05-09T01:01:55.729 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 2135, 2024-05-09T01:01:55.729 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 2135, 2024-05-09T01:01:55.729 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-05-09T01:01:55.729 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:55.729 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:55.730 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:55.730 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:55.730 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:55.730 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-write_buf_throttle": { 2024-05-09T01:01:55.730 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-05-09T01:01:55.730 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 3758096384, 2024-05-09T01:01:55.730 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-05-09T01:01:55.730 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 0, 2024-05-09T01:01:55.730 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 0, 2024-05-09T01:01:55.730 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-05-09T01:01:55.730 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 0, 2024-05-09T01:01:55.730 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2024-05-09T01:01:55.730 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2024-05-09T01:01:55.731 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 0, 2024-05-09T01:01:55.731 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 0, 2024-05-09T01:01:55.731 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-05-09T01:01:55.731 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:55.731 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:55.731 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:55.731 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:55.731 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:55.731 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-write_buf_throttle-0x5572f71f6410": { 2024-05-09T01:01:55.731 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-05-09T01:01:55.731 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 3758096384, 2024-05-09T01:01:55.731 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-05-09T01:01:55.731 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 0, 2024-05-09T01:01:55.731 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 0, 2024-05-09T01:01:55.731 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-05-09T01:01:55.731 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 0, 2024-05-09T01:01:55.732 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2024-05-09T01:01:55.732 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2024-05-09T01:01:55.732 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 0, 2024-05-09T01:01:55.732 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 0, 2024-05-09T01:01:55.732 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-05-09T01:01:55.732 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:55.732 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:55.732 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:55.732 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:55.732 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:55.732 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-09T01:01:55.790 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:01:55 smithi005 ceph-mon[30365]: pgmap v838: 97 pgs: 97 active+clean; 6.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 26 MiB/s wr, 465 op/s 2024-05-09T01:01:55.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:01:55 smithi157 ceph-mon[34294]: pgmap v838: 97 pgs: 97 active+clean; 6.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 26 MiB/s wr, 465 op/s 2024-05-09T01:01:56.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:01:55 smithi131 ceph-mon[34547]: pgmap v838: 97 pgs: 97 active+clean; 6.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 26 MiB/s wr, 465 op/s 2024-05-09T01:01:56.480 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-05-09T01:01:56.480 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-05-09T01:01:56.480 DEBUG:tasks.check_counter:Getting stats from j 2024-05-09T01:01:56.480 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph daemon mds.j perf dump 2024-05-09T01:01:56.735 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:01:57.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:01:57 smithi005 ceph-mon[30365]: pgmap v839: 97 pgs: 97 active+clean; 6.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 12 MiB/s wr, 270 op/s 2024-05-09T01:01:57.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:01:57 smithi131 ceph-mon[34547]: pgmap v839: 97 pgs: 97 active+clean; 6.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 12 MiB/s wr, 270 op/s 2024-05-09T01:01:57.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:01:57 smithi157 ceph-mon[34294]: pgmap v839: 97 pgs: 97 active+clean; 6.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 12 MiB/s wr, 270 op/s 2024-05-09T01:01:58.588 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-09T01:01:58.588 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T01:01:58.588 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 4, 2024-05-09T01:01:58.588 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 4, 2024-05-09T01:01:58.588 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 2944, 2024-05-09T01:01:58.588 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 448, 2024-05-09T01:01:58.588 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 4, 2024-05-09T01:01:58.588 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 0, 2024-05-09T01:01:58.589 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.001774507, 2024-05-09T01:01:58.589 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.000042104, 2024-05-09T01:01:58.589 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.000781847, 2024-05-09T01:01:58.589 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T01:01:58.589 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:01:58.589 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 3, 2024-05-09T01:01:58.589 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000837425, 2024-05-09T01:01:58.589 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000279141 2024-05-09T01:01:58.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:58.589 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:01:58.589 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:58.589 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:58.589 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:58.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:58.590 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-05-09T01:01:58.590 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 448 2024-05-09T01:01:58.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:58.590 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T01:01:58.590 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 2, 2024-05-09T01:01:58.590 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 232, 2024-05-09T01:01:58.590 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 320, 2024-05-09T01:01:58.590 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 198240, 2024-05-09T01:01:58.590 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 4, 2024-05-09T01:01:58.590 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 2, 2024-05-09T01:01:58.590 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.025216538, 2024-05-09T01:01:58.590 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.017313336, 2024-05-09T01:01:58.590 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.002267344, 2024-05-09T01:01:58.591 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T01:01:58.591 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:01:58.591 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 230, 2024-05-09T01:01:58.591 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.038675073, 2024-05-09T01:01:58.591 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000168152 2024-05-09T01:01:58.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:58.591 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:01:58.591 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:58.591 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:58.591 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:58.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:58.591 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-09T01:01:58.591 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 198240 2024-05-09T01:01:58.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:58.591 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T01:01:58.592 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 307, 2024-05-09T01:01:58.592 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 304, 2024-05-09T01:01:58.592 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 539008, 2024-05-09T01:01:58.592 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 187440, 2024-05-09T01:01:58.592 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 2, 2024-05-09T01:01:58.592 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 1, 2024-05-09T01:01:58.592 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.068802035, 2024-05-09T01:01:58.592 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.023760350, 2024-05-09T01:01:58.592 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.027999020, 2024-05-09T01:01:58.592 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.015642366, 2024-05-09T01:01:58.592 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:01:58.592 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 303, 2024-05-09T01:01:58.592 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.050410367, 2024-05-09T01:01:58.592 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000166370 2024-05-09T01:01:58.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:58.593 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:01:58.593 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:58.593 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:58.593 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:58.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:58.593 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 539008, 2024-05-09T01:01:58.593 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 187440 2024-05-09T01:01:58.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:58.593 INFO:teuthology.orchestra.run.smithi005.stdout: "cct": { 2024-05-09T01:01:58.593 INFO:teuthology.orchestra.run.smithi005.stdout: "total_workers": 0, 2024-05-09T01:01:58.593 INFO:teuthology.orchestra.run.smithi005.stdout: "unhealthy_workers": 0 2024-05-09T01:01:58.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:58.593 INFO:teuthology.orchestra.run.smithi005.stdout: "mempool": { 2024-05-09T01:01:58.594 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_bytes": 0, 2024-05-09T01:01:58.594 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_items": 0, 2024-05-09T01:01:58.594 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T01:01:58.594 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_items": 0, 2024-05-09T01:01:58.594 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T01:01:58.594 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_items": 0, 2024-05-09T01:01:58.594 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T01:01:58.594 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T01:01:58.594 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T01:01:58.594 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T01:01:58.594 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T01:01:58.594 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_items": 0, 2024-05-09T01:01:58.594 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T01:01:58.594 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T01:01:58.595 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_bytes": 0, 2024-05-09T01:01:58.595 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_items": 0, 2024-05-09T01:01:58.595 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_bytes": 0, 2024-05-09T01:01:58.595 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_items": 0, 2024-05-09T01:01:58.595 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T01:01:58.595 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T01:01:58.595 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T01:01:58.595 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T01:01:58.595 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T01:01:58.595 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_items": 0, 2024-05-09T01:01:58.595 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_bytes": 0, 2024-05-09T01:01:58.595 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_items": 0, 2024-05-09T01:01:58.595 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T01:01:58.595 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T01:01:58.596 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_bytes": 0, 2024-05-09T01:01:58.596 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_items": 0, 2024-05-09T01:01:58.596 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_bytes": 0, 2024-05-09T01:01:58.596 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_items": 0, 2024-05-09T01:01:58.596 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T01:01:58.596 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_items": 0, 2024-05-09T01:01:58.596 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T01:01:58.596 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_items": 0, 2024-05-09T01:01:58.596 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_bytes": 148744, 2024-05-09T01:01:58.596 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_items": 30, 2024-05-09T01:01:58.596 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_bytes": 0, 2024-05-09T01:01:58.596 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_items": 0, 2024-05-09T01:01:58.596 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_bytes": 0, 2024-05-09T01:01:58.596 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_items": 0, 2024-05-09T01:01:58.597 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_bytes": 0, 2024-05-09T01:01:58.597 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_items": 0, 2024-05-09T01:01:58.597 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_bytes": 0, 2024-05-09T01:01:58.597 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_items": 0, 2024-05-09T01:01:58.597 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_bytes": 0, 2024-05-09T01:01:58.597 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_items": 0, 2024-05-09T01:01:58.597 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T01:01:58.597 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_items": 0, 2024-05-09T01:01:58.597 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_bytes": 0, 2024-05-09T01:01:58.597 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_items": 0, 2024-05-09T01:01:58.597 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_bytes": 928, 2024-05-09T01:01:58.597 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_items": 2, 2024-05-09T01:01:58.597 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_bytes": 0, 2024-05-09T01:01:58.597 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_items": 0, 2024-05-09T01:01:58.597 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_bytes": 0, 2024-05-09T01:01:58.598 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_items": 0 2024-05-09T01:01:58.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-09T01:01:58.598 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T01:01:58.598 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-05-09T01:01:58.598 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 104857600, 2024-05-09T01:01:58.598 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-05-09T01:01:58.598 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 309, 2024-05-09T01:01:58.598 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 495611, 2024-05-09T01:01:58.598 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-05-09T01:01:58.598 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 309, 2024-05-09T01:01:58.598 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2024-05-09T01:01:58.598 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2024-05-09T01:01:58.598 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 309, 2024-05-09T01:01:58.599 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 495611, 2024-05-09T01:01:58.599 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-05-09T01:01:58.599 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-09T01:01:58.599 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-09T01:01:58.599 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-09T01:01:58.599 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:58.599 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-09T01:01:58.599 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-09T01:01:59.306 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-05-09T01:01:59.306 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-05-09T01:01:59.306 DEBUG:tasks.check_counter:Getting stats from b 2024-05-09T01:01:59.306 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph daemon mds.b perf dump 2024-05-09T01:01:59.622 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T01:01:59.745 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:01:59 smithi131 ceph-mon[34547]: pgmap v840: 97 pgs: 97 active+clean; 6.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.2 KiB/s wr, 99 op/s 2024-05-09T01:01:59.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:01:59 smithi157 ceph-mon[34294]: pgmap v840: 97 pgs: 97 active+clean; 6.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.2 KiB/s wr, 99 op/s 2024-05-09T01:02:00.040 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:01:59 smithi005 ceph-mon[30365]: pgmap v840: 97 pgs: 97 active+clean; 6.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.2 KiB/s wr, 99 op/s 2024-05-09T01:02:01.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:01 smithi131 ceph-mon[34547]: pgmap v841: 97 pgs: 97 active+clean; 6.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 511 B/s wr, 71 op/s 2024-05-09T01:02:01.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:01 smithi157 ceph-mon[34294]: pgmap v841: 97 pgs: 97 active+clean; 6.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 511 B/s wr, 71 op/s 2024-05-09T01:02:02.027 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-09T01:02:02.027 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T01:02:02.027 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 212, 2024-05-09T01:02:02.027 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 214, 2024-05-09T01:02:02.027 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 124193, 2024-05-09T01:02:02.027 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 174948, 2024-05-09T01:02:02.027 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 16, 2024-05-09T01:02:02.027 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 3, 2024-05-09T01:02:02.027 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.034654966, 2024-05-09T01:02:02.027 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.009245096, 2024-05-09T01:02:02.028 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.021927370, 2024-05-09T01:02:02.028 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.000570505, 2024-05-09T01:02:02.028 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:02:02.028 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 214, 2024-05-09T01:02:02.028 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.027684510, 2024-05-09T01:02:02.028 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000129366 2024-05-09T01:02:02.028 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:02.028 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:02:02.028 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 350, 2024-05-09T01:02:02.028 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000099128, 2024-05-09T01:02:02.028 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000283 2024-05-09T01:02:02.028 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:02.028 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 4032, 2024-05-09T01:02:02.029 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 1344 2024-05-09T01:02:02.029 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:02.029 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T01:02:02.029 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 40, 2024-05-09T01:02:02.029 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 269, 2024-05-09T01:02:02.029 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 7873, 2024-05-09T01:02:02.029 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 842138, 2024-05-09T01:02:02.029 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 34, 2024-05-09T01:02:02.029 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 10, 2024-05-09T01:02:02.029 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.029876596, 2024-05-09T01:02:02.029 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.011033899, 2024-05-09T01:02:02.029 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.015477044, 2024-05-09T01:02:02.029 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.000443351, 2024-05-09T01:02:02.030 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:02:02.030 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 263, 2024-05-09T01:02:02.030 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.042641654, 2024-05-09T01:02:02.030 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000162135 2024-05-09T01:02:02.030 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:02.030 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:02:02.030 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 22, 2024-05-09T01:02:02.030 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000004885, 2024-05-09T01:02:02.030 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000222 2024-05-09T01:02:02.030 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:02.030 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-09T01:02:02.030 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 566592 2024-05-09T01:02:02.030 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:02.031 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T01:02:02.031 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 1856, 2024-05-09T01:02:02.031 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 1475, 2024-05-09T01:02:02.031 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 1038150, 2024-05-09T01:02:02.031 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 637134, 2024-05-09T01:02:02.031 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 9, 2024-05-09T01:02:02.031 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 3, 2024-05-09T01:02:02.031 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.188675297, 2024-05-09T01:02:02.031 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.065385674, 2024-05-09T01:02:02.031 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.130697802, 2024-05-09T01:02:02.031 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.017097990, 2024-05-09T01:02:02.031 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:02:02.031 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 1477, 2024-05-09T01:02:02.031 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.087944438, 2024-05-09T01:02:02.032 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000059542 2024-05-09T01:02:02.032 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:02.032 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:02:02.032 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 2498, 2024-05-09T01:02:02.032 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000835156, 2024-05-09T01:02:02.032 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000334 2024-05-09T01:02:02.032 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:02.032 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 625312, 2024-05-09T01:02:02.032 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 189456 2024-05-09T01:02:02.032 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:02.032 INFO:teuthology.orchestra.run.smithi131.stdout: "cct": { 2024-05-09T01:02:02.032 INFO:teuthology.orchestra.run.smithi131.stdout: "total_workers": 1, 2024-05-09T01:02:02.032 INFO:teuthology.orchestra.run.smithi131.stdout: "unhealthy_workers": 0 2024-05-09T01:02:02.033 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:02.033 INFO:teuthology.orchestra.run.smithi131.stdout: "finisher-MDSRank": { 2024-05-09T01:02:02.033 INFO:teuthology.orchestra.run.smithi131.stdout: "queue_len": 0, 2024-05-09T01:02:02.033 INFO:teuthology.orchestra.run.smithi131.stdout: "complete_latency": { 2024-05-09T01:02:02.033 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 47, 2024-05-09T01:02:02.033 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.002246045, 2024-05-09T01:02:02.033 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000047788 2024-05-09T01:02:02.033 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-09T01:02:02.033 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:02.033 INFO:teuthology.orchestra.run.smithi131.stdout: "finisher-PurgeQueue": { 2024-05-09T01:02:02.033 INFO:teuthology.orchestra.run.smithi131.stdout: "queue_len": 0, 2024-05-09T01:02:02.033 INFO:teuthology.orchestra.run.smithi131.stdout: "complete_latency": { 2024-05-09T01:02:02.033 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 2, 2024-05-09T01:02:02.034 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000007972, 2024-05-09T01:02:02.034 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000003986 2024-05-09T01:02:02.034 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-09T01:02:02.034 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:02.034 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": { 2024-05-09T01:02:02.034 INFO:teuthology.orchestra.run.smithi131.stdout: "request": 2, 2024-05-09T01:02:02.034 INFO:teuthology.orchestra.run.smithi131.stdout: "reply": 1, 2024-05-09T01:02:02.034 INFO:teuthology.orchestra.run.smithi131.stdout: "reply_latency": { 2024-05-09T01:02:02.034 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 1, 2024-05-09T01:02:02.034 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.001271712, 2024-05-09T01:02:02.034 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.001271712 2024-05-09T01:02:02.034 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:02.035 INFO:teuthology.orchestra.run.smithi131.stdout: "slow_reply": 0, 2024-05-09T01:02:02.035 INFO:teuthology.orchestra.run.smithi131.stdout: "forward": 1, 2024-05-09T01:02:02.035 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_fetch_complete": 0, 2024-05-09T01:02:02.035 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_fetch_keys": 0, 2024-05-09T01:02:02.035 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_commit": 11, 2024-05-09T01:02:02.035 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_split": 0, 2024-05-09T01:02:02.035 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_merge": 0, 2024-05-09T01:02:02.035 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes": 14, 2024-05-09T01:02:02.035 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_top": 0, 2024-05-09T01:02:02.035 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_bottom": 14, 2024-05-09T01:02:02.035 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_pin_tail": 0, 2024-05-09T01:02:02.035 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_pinned": 14, 2024-05-09T01:02:02.035 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_expired": 0, 2024-05-09T01:02:02.035 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_with_caps": 1, 2024-05-09T01:02:02.035 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": 1, 2024-05-09T01:02:02.036 INFO:teuthology.orchestra.run.smithi131.stdout: "subtrees": 2, 2024-05-09T01:02:02.036 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse": 3, 2024-05-09T01:02:02.036 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_hit": 1, 2024-05-09T01:02:02.036 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_forward": 0, 2024-05-09T01:02:02.036 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_discover": 1, 2024-05-09T01:02:02.036 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_dir_fetch": 0, 2024-05-09T01:02:02.036 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_remote_ino": 0, 2024-05-09T01:02:02.036 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_lock": 0, 2024-05-09T01:02:02.036 INFO:teuthology.orchestra.run.smithi131.stdout: "load_cent": 2000, 2024-05-09T01:02:02.036 INFO:teuthology.orchestra.run.smithi131.stdout: "q": 2, 2024-05-09T01:02:02.036 INFO:teuthology.orchestra.run.smithi131.stdout: "exported": 0, 2024-05-09T01:02:02.036 INFO:teuthology.orchestra.run.smithi131.stdout: "exported_inodes": 0, 2024-05-09T01:02:02.036 INFO:teuthology.orchestra.run.smithi131.stdout: "imported": 0, 2024-05-09T01:02:02.036 INFO:teuthology.orchestra.run.smithi131.stdout: "imported_inodes": 0, 2024-05-09T01:02:02.037 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_dir_fetch": 0, 2024-05-09T01:02:02.037 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_backtrace_fetch": 0, 2024-05-09T01:02:02.037 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_peer_discover": 0, 2024-05-09T01:02:02.037 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rfiles": 21, 2024-05-09T01:02:02.037 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rbytes": 10738136400, 2024-05-09T01:02:02.037 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rsnaps": 0, 2024-05-09T01:02:02.037 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_backtrace_fetch": 0, 2024-05-09T01:02:02.037 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_set_tag": 0, 2024-05-09T01:02:02.037 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T01:02:02.037 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_inotable_repaired": 0, 2024-05-09T01:02:02.037 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dir_inodes": 0, 2024-05-09T01:02:02.037 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T01:02:02.037 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T01:02:02.038 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_file_inodes": 0, 2024-05-09T01:02:02.038 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_inode_file_caps": 0, 2024-05-09T01:02:02.038 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_revoke": 2, 2024-05-09T01:02:02.038 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_grant": 3, 2024-05-09T01:02:02.038 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T01:02:02.038 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T01:02:02.038 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T01:02:02.038 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_caps": 2, 2024-05-09T01:02:02.038 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_caps_dirty": 0, 2024-05-09T01:02:02.038 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_cap_release": 0, 2024-05-09T01:02:02.038 INFO:teuthology.orchestra.run.smithi131.stdout: "process_request_cap_release": 1 2024-05-09T01:02:02.038 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:02.038 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_cache": { 2024-05-09T01:02:02.038 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_update": 0, 2024-05-09T01:02:02.039 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_update_receipt": 0, 2024-05-09T01:02:02.039 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_try_discover": 0, 2024-05-09T01:02:02.039 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_send_discover": 2, 2024-05-09T01:02:02.039 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_handle_discover": 0, 2024-05-09T01:02:02.039 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays": 0, 2024-05-09T01:02:02.039 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays_delayed": 0, 2024-05-09T01:02:02.039 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays_enqueuing": 0, 2024-05-09T01:02:02.039 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_created": 0, 2024-05-09T01:02:02.039 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_enqueued": 0, 2024-05-09T01:02:02.039 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_reintegrated": 0, 2024-05-09T01:02:02.039 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_migrated": 0, 2024-05-09T01:02:02.039 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_processing": 0, 2024-05-09T01:02:02.039 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_enqueued": 0, 2024-05-09T01:02:02.039 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_prioritized": 0, 2024-05-09T01:02:02.040 INFO:teuthology.orchestra.run.smithi131.stdout: "recovery_started": 0, 2024-05-09T01:02:02.040 INFO:teuthology.orchestra.run.smithi131.stdout: "recovery_completed": 0, 2024-05-09T01:02:02.040 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_quiesce_path": 0, 2024-05-09T01:02:02.040 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_quiesce_inode": 0, 2024-05-09T01:02:02.040 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T01:02:02.040 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_exportdir": 0, 2024-05-09T01:02:02.040 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_flush": 0, 2024-05-09T01:02:02.040 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_fragmentdir": 0, 2024-05-09T01:02:02.040 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_fragstats": 0, 2024-05-09T01:02:02.040 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_inodestats": 0 2024-05-09T01:02:02.040 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:02.040 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_log": { 2024-05-09T01:02:02.040 INFO:teuthology.orchestra.run.smithi131.stdout: "evlrg": 0, 2024-05-09T01:02:02.040 INFO:teuthology.orchestra.run.smithi131.stdout: "evadd": 5, 2024-05-09T01:02:02.041 INFO:teuthology.orchestra.run.smithi131.stdout: "evex": 0, 2024-05-09T01:02:02.041 INFO:teuthology.orchestra.run.smithi131.stdout: "evtrm": 0, 2024-05-09T01:02:02.041 INFO:teuthology.orchestra.run.smithi131.stdout: "ev": 5, 2024-05-09T01:02:02.041 INFO:teuthology.orchestra.run.smithi131.stdout: "evexg": 0, 2024-05-09T01:02:02.041 INFO:teuthology.orchestra.run.smithi131.stdout: "evexd": 0, 2024-05-09T01:02:02.041 INFO:teuthology.orchestra.run.smithi131.stdout: "segadd": 2, 2024-05-09T01:02:02.041 INFO:teuthology.orchestra.run.smithi131.stdout: "segex": 0, 2024-05-09T01:02:02.041 INFO:teuthology.orchestra.run.smithi131.stdout: "segtrm": 0, 2024-05-09T01:02:02.041 INFO:teuthology.orchestra.run.smithi131.stdout: "seg": 2, 2024-05-09T01:02:02.041 INFO:teuthology.orchestra.run.smithi131.stdout: "segmjr": 2, 2024-05-09T01:02:02.041 INFO:teuthology.orchestra.run.smithi131.stdout: "segexg": 0, 2024-05-09T01:02:02.041 INFO:teuthology.orchestra.run.smithi131.stdout: "segexd": 0, 2024-05-09T01:02:02.041 INFO:teuthology.orchestra.run.smithi131.stdout: "expos": 4194304, 2024-05-09T01:02:02.041 INFO:teuthology.orchestra.run.smithi131.stdout: "wrpos": 4195732, 2024-05-09T01:02:02.042 INFO:teuthology.orchestra.run.smithi131.stdout: "rdpos": 0, 2024-05-09T01:02:02.042 INFO:teuthology.orchestra.run.smithi131.stdout: "jlat": { 2024-05-09T01:02:02.042 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 4, 2024-05-09T01:02:02.042 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.008372936, 2024-05-09T01:02:02.042 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.002093234 2024-05-09T01:02:02.042 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:02.042 INFO:teuthology.orchestra.run.smithi131.stdout: "replayed": 0 2024-05-09T01:02:02.042 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:02.042 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_mem": { 2024-05-09T01:02:02.042 INFO:teuthology.orchestra.run.smithi131.stdout: "ino": 17, 2024-05-09T01:02:02.042 INFO:teuthology.orchestra.run.smithi131.stdout: "ino+": 17, 2024-05-09T01:02:02.042 INFO:teuthology.orchestra.run.smithi131.stdout: "ino-": 0, 2024-05-09T01:02:02.043 INFO:teuthology.orchestra.run.smithi131.stdout: "dir": 15, 2024-05-09T01:02:02.043 INFO:teuthology.orchestra.run.smithi131.stdout: "dir+": 16, 2024-05-09T01:02:02.043 INFO:teuthology.orchestra.run.smithi131.stdout: "dir-": 1, 2024-05-09T01:02:02.043 INFO:teuthology.orchestra.run.smithi131.stdout: "dn": 14, 2024-05-09T01:02:02.043 INFO:teuthology.orchestra.run.smithi131.stdout: "dn+": 14, 2024-05-09T01:02:02.043 INFO:teuthology.orchestra.run.smithi131.stdout: "dn-": 0, 2024-05-09T01:02:02.043 INFO:teuthology.orchestra.run.smithi131.stdout: "cap": 1, 2024-05-09T01:02:02.043 INFO:teuthology.orchestra.run.smithi131.stdout: "cap+": 1, 2024-05-09T01:02:02.043 INFO:teuthology.orchestra.run.smithi131.stdout: "cap-": 0, 2024-05-09T01:02:02.043 INFO:teuthology.orchestra.run.smithi131.stdout: "rss": 124936, 2024-05-09T01:02:02.043 INFO:teuthology.orchestra.run.smithi131.stdout: "heap": 223632 2024-05-09T01:02:02.043 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:02.043 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_server": { 2024-05-09T01:02:02.043 INFO:teuthology.orchestra.run.smithi131.stdout: "dispatch_client_request": 3, 2024-05-09T01:02:02.044 INFO:teuthology.orchestra.run.smithi131.stdout: "dispatch_server_request": 0, 2024-05-09T01:02:02.044 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_request": 2, 2024-05-09T01:02:02.044 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_session": 17, 2024-05-09T01:02:02.044 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_peer_request": 0, 2024-05-09T01:02:02.044 INFO:teuthology.orchestra.run.smithi131.stdout: "req_create_latency": { 2024-05-09T01:02:02.044 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:02.044 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:02.044 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:02.044 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:02.044 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getattr_latency": { 2024-05-09T01:02:02.044 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 1, 2024-05-09T01:02:02.044 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.001271712, 2024-05-09T01:02:02.044 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.001271712 2024-05-09T01:02:02.044 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:02.045 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getfilelock_latency": { 2024-05-09T01:02:02.045 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:02.045 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:02.045 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:02.045 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:02.045 INFO:teuthology.orchestra.run.smithi131.stdout: "req_link_latency": { 2024-05-09T01:02:02.045 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:02.045 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:02.045 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:02.045 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:02.045 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookup_latency": { 2024-05-09T01:02:02.045 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:02.045 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:02.046 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:02.046 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:02.046 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookuphash_latency": { 2024-05-09T01:02:02.046 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:02.046 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:02.046 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:02.046 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:02.046 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupino_latency": { 2024-05-09T01:02:02.046 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:02.046 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:02.046 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:02.046 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:02.046 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupname_latency": { 2024-05-09T01:02:02.046 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:02.047 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:02.047 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:02.047 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:02.047 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupparent_latency": { 2024-05-09T01:02:02.047 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:02.047 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:02.047 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:02.047 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:02.047 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupsnap_latency": { 2024-05-09T01:02:02.047 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:02.047 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:02.047 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:02.047 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:02.048 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lssnap_latency": { 2024-05-09T01:02:02.048 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:02.048 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:02.048 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:02.048 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:02.048 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mkdir_latency": { 2024-05-09T01:02:02.049 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:01 smithi005 ceph-mon[30365]: pgmap v841: 97 pgs: 97 active+clean; 6.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 511 B/s wr, 71 op/s 2024-05-09T01:02:02.049 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:02.049 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:02.049 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:02.049 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:02.049 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mknod_latency": { 2024-05-09T01:02:02.049 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:02.049 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:02.050 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:02.050 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:02.050 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mksnap_latency": { 2024-05-09T01:02:02.050 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:02.050 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:02.050 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:02.050 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:02.050 INFO:teuthology.orchestra.run.smithi131.stdout: "req_open_latency": { 2024-05-09T01:02:02.050 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:02.050 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:02.050 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:02.050 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:02.051 INFO:teuthology.orchestra.run.smithi131.stdout: "req_readdir_latency": { 2024-05-09T01:02:02.051 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:02.051 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:02.051 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:02.051 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:02.051 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rename_latency": { 2024-05-09T01:02:02.051 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:02.051 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:02.051 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:02.051 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:02.051 INFO:teuthology.orchestra.run.smithi131.stdout: "req_renamesnap_latency": { 2024-05-09T01:02:02.051 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:02.051 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:02.051 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:02.052 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:02.052 INFO:teuthology.orchestra.run.smithi131.stdout: "req_snapdiff_latency": { 2024-05-09T01:02:02.052 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:02.052 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:02.052 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:02.052 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:02.052 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmdir_latency": { 2024-05-09T01:02:02.052 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:02.052 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:02.052 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:02.052 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:02.052 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmsnap_latency": { 2024-05-09T01:02:02.053 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:02.053 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:02.053 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:02.053 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:02.053 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmxattr_latency": { 2024-05-09T01:02:02.053 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:02.053 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:02.053 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:02.053 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:02.053 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setattr_latency": { 2024-05-09T01:02:02.053 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:02.053 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:02.053 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:02.053 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:02.054 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setdirlayout_latency": { 2024-05-09T01:02:02.054 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:02.054 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:02.054 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:02.054 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:02.054 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setfilelock_latency": { 2024-05-09T01:02:02.054 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:02.054 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:02.054 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:02.054 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:02.054 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setlayout_latency": { 2024-05-09T01:02:02.054 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:02.054 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:02.055 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:02.055 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:02.055 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setxattr_latency": { 2024-05-09T01:02:02.055 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:02.055 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:02.055 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:02.055 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:02.055 INFO:teuthology.orchestra.run.smithi131.stdout: "req_symlink_latency": { 2024-05-09T01:02:02.055 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:02.055 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:02.055 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:02.055 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:02.055 INFO:teuthology.orchestra.run.smithi131.stdout: "req_unlink_latency": { 2024-05-09T01:02:02.056 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:02.056 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:02.056 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:02.056 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:02.056 INFO:teuthology.orchestra.run.smithi131.stdout: "cap_revoke_eviction": 0, 2024-05-09T01:02:02.056 INFO:teuthology.orchestra.run.smithi131.stdout: "cap_acquisition_throttle": 0, 2024-05-09T01:02:02.056 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getvxattr_latency": { 2024-05-09T01:02:02.056 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:02.056 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:02.056 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:02.056 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-09T01:02:02.056 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:02.056 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_sessions": { 2024-05-09T01:02:02.057 INFO:teuthology.orchestra.run.smithi131.stdout: "session_count": 1, 2024-05-09T01:02:02.057 INFO:teuthology.orchestra.run.smithi131.stdout: "session_add": 2, 2024-05-09T01:02:02.057 INFO:teuthology.orchestra.run.smithi131.stdout: "session_remove": 1, 2024-05-09T01:02:02.057 INFO:teuthology.orchestra.run.smithi131.stdout: "sessions_open": 1, 2024-05-09T01:02:02.057 INFO:teuthology.orchestra.run.smithi131.stdout: "sessions_stale": 0, 2024-05-09T01:02:02.057 INFO:teuthology.orchestra.run.smithi131.stdout: "total_load": 1, 2024-05-09T01:02:02.057 INFO:teuthology.orchestra.run.smithi131.stdout: "average_load": 1, 2024-05-09T01:02:02.057 INFO:teuthology.orchestra.run.smithi131.stdout: "avg_session_uptime": 894, 2024-05-09T01:02:02.057 INFO:teuthology.orchestra.run.smithi131.stdout: "mdthresh_evicted": 0 2024-05-09T01:02:02.057 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:02.057 INFO:teuthology.orchestra.run.smithi131.stdout: "mempool": { 2024-05-09T01:02:02.057 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_bytes": 0, 2024-05-09T01:02:02.057 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_items": 0, 2024-05-09T01:02:02.058 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T01:02:02.058 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_items": 0, 2024-05-09T01:02:02.058 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T01:02:02.058 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_items": 0, 2024-05-09T01:02:02.058 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T01:02:02.058 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T01:02:02.058 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T01:02:02.058 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T01:02:02.058 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T01:02:02.058 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_items": 0, 2024-05-09T01:02:02.058 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T01:02:02.058 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T01:02:02.058 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_bytes": 0, 2024-05-09T01:02:02.059 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_items": 0, 2024-05-09T01:02:02.059 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_bytes": 0, 2024-05-09T01:02:02.059 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_items": 0, 2024-05-09T01:02:02.059 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T01:02:02.059 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T01:02:02.059 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T01:02:02.059 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T01:02:02.059 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T01:02:02.059 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_items": 0, 2024-05-09T01:02:02.059 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_bytes": 0, 2024-05-09T01:02:02.059 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_items": 0, 2024-05-09T01:02:02.059 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T01:02:02.059 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T01:02:02.059 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_bytes": 0, 2024-05-09T01:02:02.060 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_items": 0, 2024-05-09T01:02:02.060 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_bytes": 0, 2024-05-09T01:02:02.060 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_items": 0, 2024-05-09T01:02:02.060 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T01:02:02.060 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_items": 0, 2024-05-09T01:02:02.060 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T01:02:02.060 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_items": 0, 2024-05-09T01:02:02.060 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_bytes": 249119, 2024-05-09T01:02:02.060 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_items": 89, 2024-05-09T01:02:02.060 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_bytes": 0, 2024-05-09T01:02:02.060 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_items": 0, 2024-05-09T01:02:02.060 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_bytes": 0, 2024-05-09T01:02:02.060 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_items": 0, 2024-05-09T01:02:02.060 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_bytes": 0, 2024-05-09T01:02:02.060 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_items": 0, 2024-05-09T01:02:02.061 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_bytes": 0, 2024-05-09T01:02:02.061 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_items": 0, 2024-05-09T01:02:02.061 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_bytes": 7736, 2024-05-09T01:02:02.061 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_items": 164, 2024-05-09T01:02:02.061 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T01:02:02.061 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_items": 0, 2024-05-09T01:02:02.061 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_bytes": 0, 2024-05-09T01:02:02.061 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_items": 0, 2024-05-09T01:02:02.061 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_bytes": 68037, 2024-05-09T01:02:02.061 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_items": 288, 2024-05-09T01:02:02.061 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_bytes": 0, 2024-05-09T01:02:02.061 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_items": 0, 2024-05-09T01:02:02.061 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_bytes": 0, 2024-05-09T01:02:02.061 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_items": 0 2024-05-09T01:02:02.062 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:02.062 INFO:teuthology.orchestra.run.smithi131.stdout: "objecter": { 2024-05-09T01:02:02.062 INFO:teuthology.orchestra.run.smithi131.stdout: "op_active": 0, 2024-05-09T01:02:02.062 INFO:teuthology.orchestra.run.smithi131.stdout: "op_laggy": 0, 2024-05-09T01:02:02.062 INFO:teuthology.orchestra.run.smithi131.stdout: "op_send": 42, 2024-05-09T01:02:02.062 INFO:teuthology.orchestra.run.smithi131.stdout: "op_send_bytes": 12135, 2024-05-09T01:02:02.062 INFO:teuthology.orchestra.run.smithi131.stdout: "op_resend": 31, 2024-05-09T01:02:02.062 INFO:teuthology.orchestra.run.smithi131.stdout: "op_reply": 42, 2024-05-09T01:02:02.062 INFO:teuthology.orchestra.run.smithi131.stdout: "op_latency": { 2024-05-09T01:02:02.062 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 42, 2024-05-09T01:02:02.062 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.234995050, 2024-05-09T01:02:02.062 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.005595120 2024-05-09T01:02:02.062 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:02.063 INFO:teuthology.orchestra.run.smithi131.stdout: "op_inflight": 0, 2024-05-09T01:02:02.063 INFO:teuthology.orchestra.run.smithi131.stdout: "oplen_avg": { 2024-05-09T01:02:02.063 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 42, 2024-05-09T01:02:02.063 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 66 2024-05-09T01:02:02.063 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:02.063 INFO:teuthology.orchestra.run.smithi131.stdout: "op": 42, 2024-05-09T01:02:02.063 INFO:teuthology.orchestra.run.smithi131.stdout: "op_r": 0, 2024-05-09T01:02:02.063 INFO:teuthology.orchestra.run.smithi131.stdout: "op_w": 42, 2024-05-09T01:02:02.063 INFO:teuthology.orchestra.run.smithi131.stdout: "op_rmw": 0, 2024-05-09T01:02:02.063 INFO:teuthology.orchestra.run.smithi131.stdout: "op_pg": 0, 2024-05-09T01:02:02.063 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_stat": 0, 2024-05-09T01:02:02.063 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_create": 10, 2024-05-09T01:02:02.063 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_read": 0, 2024-05-09T01:02:02.063 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_write": 4, 2024-05-09T01:02:02.063 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_writefull": 7, 2024-05-09T01:02:02.064 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_writesame": 0, 2024-05-09T01:02:02.064 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_append": 0, 2024-05-09T01:02:02.064 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_zero": 0, 2024-05-09T01:02:02.064 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_truncate": 0, 2024-05-09T01:02:02.064 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_delete": 6, 2024-05-09T01:02:02.064 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_mapext": 0, 2024-05-09T01:02:02.064 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_sparse_read": 0, 2024-05-09T01:02:02.064 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_clonerange": 0, 2024-05-09T01:02:02.064 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_getxattr": 0, 2024-05-09T01:02:02.064 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_setxattr": 20, 2024-05-09T01:02:02.064 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_cmpxattr": 0, 2024-05-09T01:02:02.064 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_rmxattr": 0, 2024-05-09T01:02:02.064 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_resetxattrs": 0, 2024-05-09T01:02:02.065 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_call": 0, 2024-05-09T01:02:02.065 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_watch": 0, 2024-05-09T01:02:02.065 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_notify": 0, 2024-05-09T01:02:02.065 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T01:02:02.065 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_pgls": 0, 2024-05-09T01:02:02.065 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_pgls_filter": 0, 2024-05-09T01:02:02.065 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_other": 0, 2024-05-09T01:02:02.065 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_active": 0, 2024-05-09T01:02:02.065 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_send": 0, 2024-05-09T01:02:02.065 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_resend": 0, 2024-05-09T01:02:02.065 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_ping": 0, 2024-05-09T01:02:02.065 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_active": 0, 2024-05-09T01:02:02.065 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_send": 0, 2024-05-09T01:02:02.065 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_resend": 0, 2024-05-09T01:02:02.065 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_active": 0, 2024-05-09T01:02:02.066 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_send": 0, 2024-05-09T01:02:02.066 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_resend": 0, 2024-05-09T01:02:02.066 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_active": 0, 2024-05-09T01:02:02.066 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_send": 0, 2024-05-09T01:02:02.066 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_resend": 0, 2024-05-09T01:02:02.066 INFO:teuthology.orchestra.run.smithi131.stdout: "command_active": 0, 2024-05-09T01:02:02.066 INFO:teuthology.orchestra.run.smithi131.stdout: "command_send": 0, 2024-05-09T01:02:02.066 INFO:teuthology.orchestra.run.smithi131.stdout: "command_resend": 0, 2024-05-09T01:02:02.066 INFO:teuthology.orchestra.run.smithi131.stdout: "map_epoch": 82, 2024-05-09T01:02:02.066 INFO:teuthology.orchestra.run.smithi131.stdout: "map_full": 0, 2024-05-09T01:02:02.066 INFO:teuthology.orchestra.run.smithi131.stdout: "map_inc": 4, 2024-05-09T01:02:02.066 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_sessions": 8, 2024-05-09T01:02:02.066 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_session_open": 13, 2024-05-09T01:02:02.067 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_session_close": 5, 2024-05-09T01:02:02.067 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_laggy": 0, 2024-05-09T01:02:02.067 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_wr": 18, 2024-05-09T01:02:02.067 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_rd": 0, 2024-05-09T01:02:02.067 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_del": 1 2024-05-09T01:02:02.067 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:02.067 INFO:teuthology.orchestra.run.smithi131.stdout: "oft": { 2024-05-09T01:02:02.067 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_objs": 1, 2024-05-09T01:02:02.067 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_kv_pairs": 0, 2024-05-09T01:02:02.067 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_updates": 0, 2024-05-09T01:02:02.067 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_removes": 0 2024-05-09T01:02:02.067 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:02.067 INFO:teuthology.orchestra.run.smithi131.stdout: "osd-slow-ops": { 2024-05-09T01:02:02.067 INFO:teuthology.orchestra.run.smithi131.stdout: "slow_ops_count": 0 2024-05-09T01:02:02.068 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:02.068 INFO:teuthology.orchestra.run.smithi131.stdout: "purge_queue": { 2024-05-09T01:02:02.068 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_ops": 0, 2024-05-09T01:02:02.068 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_ops_high_water": 0, 2024-05-09T01:02:02.068 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing": 0, 2024-05-09T01:02:02.068 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_high_water": 0, 2024-05-09T01:02:02.068 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executed_ops": 0, 2024-05-09T01:02:02.068 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executed": 0, 2024-05-09T01:02:02.068 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_item_in_journal": 0 2024-05-09T01:02:02.069 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:02.069 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T01:02:02.069 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-09T01:02:02.069 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 104857600, 2024-05-09T01:02:02.069 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-09T01:02:02.069 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 2104, 2024-05-09T01:02:02.069 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 959862, 2024-05-09T01:02:02.069 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-09T01:02:02.069 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 2104, 2024-05-09T01:02:02.069 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-05-09T01:02:02.069 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-05-09T01:02:02.069 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 2104, 2024-05-09T01:02:02.069 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 959862, 2024-05-09T01:02:02.069 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-09T01:02:02.070 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:02.070 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:02.070 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:02.070 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-09T01:02:02.070 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:02.070 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-objecter_bytes": { 2024-05-09T01:02:02.070 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-09T01:02:02.070 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 104857600, 2024-05-09T01:02:02.070 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-09T01:02:02.070 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-05-09T01:02:02.070 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-05-09T01:02:02.070 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-09T01:02:02.070 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-05-09T01:02:02.070 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 42, 2024-05-09T01:02:02.071 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 12135, 2024-05-09T01:02:02.071 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 42, 2024-05-09T01:02:02.071 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 12135, 2024-05-09T01:02:02.071 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-09T01:02:02.071 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:02.071 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:02.071 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:02.071 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-09T01:02:02.071 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:02.071 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-objecter_ops": { 2024-05-09T01:02:02.071 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-09T01:02:02.071 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 1024, 2024-05-09T01:02:02.071 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-09T01:02:02.072 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-05-09T01:02:02.072 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-05-09T01:02:02.072 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-09T01:02:02.072 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-05-09T01:02:02.072 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 42, 2024-05-09T01:02:02.072 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 42, 2024-05-09T01:02:02.072 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 42, 2024-05-09T01:02:02.072 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 42, 2024-05-09T01:02:02.072 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-09T01:02:02.072 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:02.072 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:02.072 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:02.072 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-09T01:02:02.072 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:02.073 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-write_buf_throttle": { 2024-05-09T01:02:02.073 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-09T01:02:02.073 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 3758096384, 2024-05-09T01:02:02.073 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-09T01:02:02.073 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-05-09T01:02:02.073 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-05-09T01:02:02.073 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-09T01:02:02.073 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-05-09T01:02:02.073 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-05-09T01:02:02.073 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-05-09T01:02:02.073 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 0, 2024-05-09T01:02:02.073 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 0, 2024-05-09T01:02:02.073 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-09T01:02:02.073 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:02.073 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:02.074 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:02.074 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-09T01:02:02.074 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:02.074 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-write_buf_throttle-0x55dce63824e0": { 2024-05-09T01:02:02.074 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-09T01:02:02.074 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 3758096384, 2024-05-09T01:02:02.074 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-09T01:02:02.074 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 5, 2024-05-09T01:02:02.074 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 1428, 2024-05-09T01:02:02.074 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-09T01:02:02.074 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 5, 2024-05-09T01:02:02.074 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-05-09T01:02:02.074 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-05-09T01:02:02.075 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 4, 2024-05-09T01:02:02.075 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 1428, 2024-05-09T01:02:02.075 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-09T01:02:02.075 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:02.075 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:02.075 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:02.075 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-09T01:02:02.075 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-09T01:02:02.075 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-09T01:02:02.887 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-05-09T01:02:02.887 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-05-09T01:02:02.887 DEBUG:tasks.check_counter:Getting stats from e 2024-05-09T01:02:02.887 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph daemon mds.e perf dump 2024-05-09T01:02:03.133 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T01:02:03.857 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:03 smithi131 ceph-mon[34547]: pgmap v842: 97 pgs: 97 active+clean; 6.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 511 B/s wr, 35 op/s 2024-05-09T01:02:03.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:03 smithi157 ceph-mon[34294]: pgmap v842: 97 pgs: 97 active+clean; 6.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 511 B/s wr, 35 op/s 2024-05-09T01:02:04.040 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:03 smithi005 ceph-mon[30365]: pgmap v842: 97 pgs: 97 active+clean; 6.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 511 B/s wr, 35 op/s 2024-05-09T01:02:05.135 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-09T01:02:05.136 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T01:02:05.136 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 3142, 2024-05-09T01:02:05.136 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 3142, 2024-05-09T01:02:05.136 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 913030, 2024-05-09T01:02:05.136 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 998339, 2024-05-09T01:02:05.136 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 6, 2024-05-09T01:02:05.136 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 2, 2024-05-09T01:02:05.136 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.279074790, 2024-05-09T01:02:05.136 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.077050452, 2024-05-09T01:02:05.136 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.074940915, 2024-05-09T01:02:05.136 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.107957160, 2024-05-09T01:02:05.137 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:02:05.137 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 3141, 2024-05-09T01:02:05.137 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.101921209, 2024-05-09T01:02:05.137 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000032448 2024-05-09T01:02:05.137 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:05.137 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:02:05.137 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:05.137 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:05.137 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:05.137 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:05.137 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-05-09T01:02:05.137 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 448 2024-05-09T01:02:05.137 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:05.137 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T01:02:05.138 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 324, 2024-05-09T01:02:05.138 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 314, 2024-05-09T01:02:05.138 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 603820, 2024-05-09T01:02:05.138 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 189034, 2024-05-09T01:02:05.138 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 3, 2024-05-09T01:02:05.138 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 2, 2024-05-09T01:02:05.138 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.053840010, 2024-05-09T01:02:05.138 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.019267126, 2024-05-09T01:02:05.138 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.023583947, 2024-05-09T01:02:05.138 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.009899811, 2024-05-09T01:02:05.138 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:02:05.138 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 313, 2024-05-09T01:02:05.138 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.013864527, 2024-05-09T01:02:05.139 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000044295 2024-05-09T01:02:05.139 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:05.139 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:02:05.139 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:05.139 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:05.139 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:05.139 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:05.139 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 603264, 2024-05-09T01:02:05.139 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 188336 2024-05-09T01:02:05.139 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:05.139 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T01:02:05.139 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 3, 2024-05-09T01:02:05.139 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 232, 2024-05-09T01:02:05.140 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 588, 2024-05-09T01:02:05.140 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 565998, 2024-05-09T01:02:05.140 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 6, 2024-05-09T01:02:05.140 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 3, 2024-05-09T01:02:05.140 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.015569131, 2024-05-09T01:02:05.140 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.010388213, 2024-05-09T01:02:05.140 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.002763233, 2024-05-09T01:02:05.140 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.000028573, 2024-05-09T01:02:05.140 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:02:05.140 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 230, 2024-05-09T01:02:05.140 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.039448019, 2024-05-09T01:02:05.140 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000171513 2024-05-09T01:02:05.140 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:05.140 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:02:05.141 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:05.141 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:05.141 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:05.141 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:05.141 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-09T01:02:05.141 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 565680 2024-05-09T01:02:05.141 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:05.141 INFO:teuthology.orchestra.run.smithi131.stdout: "cct": { 2024-05-09T01:02:05.141 INFO:teuthology.orchestra.run.smithi131.stdout: "total_workers": 1, 2024-05-09T01:02:05.141 INFO:teuthology.orchestra.run.smithi131.stdout: "unhealthy_workers": 0 2024-05-09T01:02:05.141 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:05.142 INFO:teuthology.orchestra.run.smithi131.stdout: "finisher-MDSRank": { 2024-05-09T01:02:05.142 INFO:teuthology.orchestra.run.smithi131.stdout: "queue_len": 0, 2024-05-09T01:02:05.142 INFO:teuthology.orchestra.run.smithi131.stdout: "complete_latency": { 2024-05-09T01:02:05.142 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 5236, 2024-05-09T01:02:05.142 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.157386660, 2024-05-09T01:02:05.142 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000030058 2024-05-09T01:02:05.142 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-09T01:02:05.142 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:05.142 INFO:teuthology.orchestra.run.smithi131.stdout: "finisher-PurgeQueue": { 2024-05-09T01:02:05.142 INFO:teuthology.orchestra.run.smithi131.stdout: "queue_len": 0, 2024-05-09T01:02:05.142 INFO:teuthology.orchestra.run.smithi131.stdout: "complete_latency": { 2024-05-09T01:02:05.142 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:05.142 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:05.143 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:05.143 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-09T01:02:05.143 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:05.143 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": { 2024-05-09T01:02:05.143 INFO:teuthology.orchestra.run.smithi131.stdout: "request": 0, 2024-05-09T01:02:05.143 INFO:teuthology.orchestra.run.smithi131.stdout: "reply": 0, 2024-05-09T01:02:05.143 INFO:teuthology.orchestra.run.smithi131.stdout: "reply_latency": { 2024-05-09T01:02:05.143 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:05.143 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:05.143 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:05.143 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:05.143 INFO:teuthology.orchestra.run.smithi131.stdout: "slow_reply": 0, 2024-05-09T01:02:05.143 INFO:teuthology.orchestra.run.smithi131.stdout: "forward": 0, 2024-05-09T01:02:05.143 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_fetch_complete": 0, 2024-05-09T01:02:05.144 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_fetch_keys": 0, 2024-05-09T01:02:05.144 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_commit": 0, 2024-05-09T01:02:05.144 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_split": 0, 2024-05-09T01:02:05.144 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_merge": 0, 2024-05-09T01:02:05.144 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes": 3, 2024-05-09T01:02:05.144 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_top": 0, 2024-05-09T01:02:05.144 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_bottom": 3, 2024-05-09T01:02:05.144 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_pin_tail": 0, 2024-05-09T01:02:05.144 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_pinned": 3, 2024-05-09T01:02:05.144 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_expired": 0, 2024-05-09T01:02:05.144 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_with_caps": 0, 2024-05-09T01:02:05.144 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": 0, 2024-05-09T01:02:05.144 INFO:teuthology.orchestra.run.smithi131.stdout: "subtrees": 1, 2024-05-09T01:02:05.144 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse": 0, 2024-05-09T01:02:05.145 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_hit": 0, 2024-05-09T01:02:05.145 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_forward": 0, 2024-05-09T01:02:05.145 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_discover": 0, 2024-05-09T01:02:05.145 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_dir_fetch": 0, 2024-05-09T01:02:05.145 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_remote_ino": 0, 2024-05-09T01:02:05.145 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_lock": 0, 2024-05-09T01:02:05.145 INFO:teuthology.orchestra.run.smithi131.stdout: "load_cent": 0, 2024-05-09T01:02:05.145 INFO:teuthology.orchestra.run.smithi131.stdout: "q": 0, 2024-05-09T01:02:05.145 INFO:teuthology.orchestra.run.smithi131.stdout: "exported": 0, 2024-05-09T01:02:05.145 INFO:teuthology.orchestra.run.smithi131.stdout: "exported_inodes": 0, 2024-05-09T01:02:05.145 INFO:teuthology.orchestra.run.smithi131.stdout: "imported": 0, 2024-05-09T01:02:05.145 INFO:teuthology.orchestra.run.smithi131.stdout: "imported_inodes": 0, 2024-05-09T01:02:05.145 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_dir_fetch": 0, 2024-05-09T01:02:05.146 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_backtrace_fetch": 0, 2024-05-09T01:02:05.146 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_peer_discover": 0, 2024-05-09T01:02:05.146 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rfiles": 0, 2024-05-09T01:02:05.146 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rbytes": 0, 2024-05-09T01:02:05.146 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rsnaps": 0, 2024-05-09T01:02:05.146 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_backtrace_fetch": 0, 2024-05-09T01:02:05.146 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_set_tag": 0, 2024-05-09T01:02:05.146 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T01:02:05.146 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_inotable_repaired": 0, 2024-05-09T01:02:05.146 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dir_inodes": 0, 2024-05-09T01:02:05.146 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T01:02:05.146 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T01:02:05.146 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_file_inodes": 0, 2024-05-09T01:02:05.147 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_inode_file_caps": 0, 2024-05-09T01:02:05.147 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_revoke": 0, 2024-05-09T01:02:05.147 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_grant": 0, 2024-05-09T01:02:05.147 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T01:02:05.147 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T01:02:05.147 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T01:02:05.147 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_caps": 0, 2024-05-09T01:02:05.147 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_caps_dirty": 0, 2024-05-09T01:02:05.147 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_cap_release": 0, 2024-05-09T01:02:05.147 INFO:teuthology.orchestra.run.smithi131.stdout: "process_request_cap_release": 0 2024-05-09T01:02:05.147 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:05.147 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_cache": { 2024-05-09T01:02:05.147 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_update": 0, 2024-05-09T01:02:05.147 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_update_receipt": 0, 2024-05-09T01:02:05.148 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_try_discover": 0, 2024-05-09T01:02:05.148 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_send_discover": 0, 2024-05-09T01:02:05.148 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_handle_discover": 0, 2024-05-09T01:02:05.148 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays": 0, 2024-05-09T01:02:05.148 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays_delayed": 0, 2024-05-09T01:02:05.148 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays_enqueuing": 0, 2024-05-09T01:02:05.148 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_created": 0, 2024-05-09T01:02:05.148 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_enqueued": 0, 2024-05-09T01:02:05.148 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_reintegrated": 0, 2024-05-09T01:02:05.148 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_migrated": 0, 2024-05-09T01:02:05.148 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_processing": 0, 2024-05-09T01:02:05.148 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_enqueued": 0, 2024-05-09T01:02:05.148 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_prioritized": 0, 2024-05-09T01:02:05.148 INFO:teuthology.orchestra.run.smithi131.stdout: "recovery_started": 0, 2024-05-09T01:02:05.148 INFO:teuthology.orchestra.run.smithi131.stdout: "recovery_completed": 0, 2024-05-09T01:02:05.149 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_quiesce_path": 0, 2024-05-09T01:02:05.149 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_quiesce_inode": 0, 2024-05-09T01:02:05.149 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T01:02:05.149 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_exportdir": 0, 2024-05-09T01:02:05.149 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_flush": 0, 2024-05-09T01:02:05.149 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_fragmentdir": 0, 2024-05-09T01:02:05.149 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_fragstats": 0, 2024-05-09T01:02:05.149 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_inodestats": 0 2024-05-09T01:02:05.149 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:05.149 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_log": { 2024-05-09T01:02:05.149 INFO:teuthology.orchestra.run.smithi131.stdout: "evlrg": 0, 2024-05-09T01:02:05.149 INFO:teuthology.orchestra.run.smithi131.stdout: "evadd": 0, 2024-05-09T01:02:05.149 INFO:teuthology.orchestra.run.smithi131.stdout: "evex": 0, 2024-05-09T01:02:05.149 INFO:teuthology.orchestra.run.smithi131.stdout: "evtrm": 0, 2024-05-09T01:02:05.150 INFO:teuthology.orchestra.run.smithi131.stdout: "ev": 15, 2024-05-09T01:02:05.150 INFO:teuthology.orchestra.run.smithi131.stdout: "evexg": 0, 2024-05-09T01:02:05.150 INFO:teuthology.orchestra.run.smithi131.stdout: "evexd": 0, 2024-05-09T01:02:05.150 INFO:teuthology.orchestra.run.smithi131.stdout: "segadd": 0, 2024-05-09T01:02:05.150 INFO:teuthology.orchestra.run.smithi131.stdout: "segex": 0, 2024-05-09T01:02:05.150 INFO:teuthology.orchestra.run.smithi131.stdout: "segtrm": 0, 2024-05-09T01:02:05.150 INFO:teuthology.orchestra.run.smithi131.stdout: "seg": 2, 2024-05-09T01:02:05.150 INFO:teuthology.orchestra.run.smithi131.stdout: "segmjr": 2, 2024-05-09T01:02:05.150 INFO:teuthology.orchestra.run.smithi131.stdout: "segexg": 0, 2024-05-09T01:02:05.150 INFO:teuthology.orchestra.run.smithi131.stdout: "segexd": 0, 2024-05-09T01:02:05.150 INFO:teuthology.orchestra.run.smithi131.stdout: "expos": 4194304, 2024-05-09T01:02:05.150 INFO:teuthology.orchestra.run.smithi131.stdout: "wrpos": 4217445, 2024-05-09T01:02:05.150 INFO:teuthology.orchestra.run.smithi131.stdout: "rdpos": 4215794, 2024-05-09T01:02:05.151 INFO:teuthology.orchestra.run.smithi131.stdout: "jlat": { 2024-05-09T01:02:05.151 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:05.151 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:05.151 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:05.151 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:05.151 INFO:teuthology.orchestra.run.smithi131.stdout: "replayed": 15 2024-05-09T01:02:05.151 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:05.151 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_mem": { 2024-05-09T01:02:05.151 INFO:teuthology.orchestra.run.smithi131.stdout: "ino": 4, 2024-05-09T01:02:05.151 INFO:teuthology.orchestra.run.smithi131.stdout: "ino+": 16, 2024-05-09T01:02:05.151 INFO:teuthology.orchestra.run.smithi131.stdout: "ino-": 12, 2024-05-09T01:02:05.151 INFO:teuthology.orchestra.run.smithi131.stdout: "dir": 2, 2024-05-09T01:02:05.151 INFO:teuthology.orchestra.run.smithi131.stdout: "dir+": 13, 2024-05-09T01:02:05.151 INFO:teuthology.orchestra.run.smithi131.stdout: "dir-": 11, 2024-05-09T01:02:05.152 INFO:teuthology.orchestra.run.smithi131.stdout: "dn": 3, 2024-05-09T01:02:05.152 INFO:teuthology.orchestra.run.smithi131.stdout: "dn+": 15, 2024-05-09T01:02:05.152 INFO:teuthology.orchestra.run.smithi131.stdout: "dn-": 12, 2024-05-09T01:02:05.152 INFO:teuthology.orchestra.run.smithi131.stdout: "cap": 0, 2024-05-09T01:02:05.152 INFO:teuthology.orchestra.run.smithi131.stdout: "cap+": 0, 2024-05-09T01:02:05.152 INFO:teuthology.orchestra.run.smithi131.stdout: "cap-": 0, 2024-05-09T01:02:05.152 INFO:teuthology.orchestra.run.smithi131.stdout: "rss": 123984, 2024-05-09T01:02:05.152 INFO:teuthology.orchestra.run.smithi131.stdout: "heap": 215440 2024-05-09T01:02:05.152 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:05.152 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_server": { 2024-05-09T01:02:05.152 INFO:teuthology.orchestra.run.smithi131.stdout: "dispatch_client_request": 0, 2024-05-09T01:02:05.152 INFO:teuthology.orchestra.run.smithi131.stdout: "dispatch_server_request": 0, 2024-05-09T01:02:05.152 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_request": 0, 2024-05-09T01:02:05.152 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_session": 0, 2024-05-09T01:02:05.152 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_peer_request": 0, 2024-05-09T01:02:05.153 INFO:teuthology.orchestra.run.smithi131.stdout: "req_create_latency": { 2024-05-09T01:02:05.153 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:05.153 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:05.153 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:05.153 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:05.153 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getattr_latency": { 2024-05-09T01:02:05.153 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:05.153 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:05.153 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:05.153 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:05.153 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getfilelock_latency": { 2024-05-09T01:02:05.153 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:05.153 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:05.153 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:05.154 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:05.154 INFO:teuthology.orchestra.run.smithi131.stdout: "req_link_latency": { 2024-05-09T01:02:05.154 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:05.154 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:05.154 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:05.154 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:05.154 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookup_latency": { 2024-05-09T01:02:05.154 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:05.154 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:05.154 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:05.154 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:05.154 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookuphash_latency": { 2024-05-09T01:02:05.154 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:05.155 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:05.155 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:05.155 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:05.155 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupino_latency": { 2024-05-09T01:02:05.155 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:05.155 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:05.155 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:05.155 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:05.155 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupname_latency": { 2024-05-09T01:02:05.155 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:05.155 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:05.155 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:05.155 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:05.155 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupparent_latency": { 2024-05-09T01:02:05.155 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:05.156 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:05.156 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:05.156 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:05.156 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupsnap_latency": { 2024-05-09T01:02:05.156 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:05.156 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:05.156 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:05.156 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:05.156 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lssnap_latency": { 2024-05-09T01:02:05.156 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:05.156 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:05.156 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:05.156 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:05.156 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mkdir_latency": { 2024-05-09T01:02:05.157 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:05.157 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:05.157 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:05.158 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:05.158 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mknod_latency": { 2024-05-09T01:02:05.158 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:05.158 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:05.158 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:05.158 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:05.158 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mksnap_latency": { 2024-05-09T01:02:05.158 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:05.158 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:05.158 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:05.158 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:05.158 INFO:teuthology.orchestra.run.smithi131.stdout: "req_open_latency": { 2024-05-09T01:02:05.158 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:05.159 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:05.159 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:05.159 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:05.159 INFO:teuthology.orchestra.run.smithi131.stdout: "req_readdir_latency": { 2024-05-09T01:02:05.159 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:05.159 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:05.159 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:05.159 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:05.159 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rename_latency": { 2024-05-09T01:02:05.159 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:05.159 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:05.159 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:05.159 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:05.160 INFO:teuthology.orchestra.run.smithi131.stdout: "req_renamesnap_latency": { 2024-05-09T01:02:05.160 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:05.160 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:05.160 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:05.160 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:05.160 INFO:teuthology.orchestra.run.smithi131.stdout: "req_snapdiff_latency": { 2024-05-09T01:02:05.160 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:05.160 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:05.160 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:05.160 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:05.160 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmdir_latency": { 2024-05-09T01:02:05.160 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:05.160 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:05.161 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:05.161 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:05.161 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmsnap_latency": { 2024-05-09T01:02:05.161 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:05.161 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:05.161 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:05.161 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:05.161 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmxattr_latency": { 2024-05-09T01:02:05.161 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:05.161 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:05.161 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:05.162 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:05.162 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setattr_latency": { 2024-05-09T01:02:05.162 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:05.162 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:05.162 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:05.162 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:05.162 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setdirlayout_latency": { 2024-05-09T01:02:05.162 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:05.162 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:05.162 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:05.162 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:05.163 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setfilelock_latency": { 2024-05-09T01:02:05.163 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:05.163 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:05.163 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:05.163 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:05.163 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setlayout_latency": { 2024-05-09T01:02:05.163 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:05.163 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:05.163 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:05.163 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:05.163 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setxattr_latency": { 2024-05-09T01:02:05.163 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:05.163 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:05.164 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:05.164 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:05.164 INFO:teuthology.orchestra.run.smithi131.stdout: "req_symlink_latency": { 2024-05-09T01:02:05.164 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:05.164 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:05.164 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:05.164 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:05.164 INFO:teuthology.orchestra.run.smithi131.stdout: "req_unlink_latency": { 2024-05-09T01:02:05.164 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:05.164 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:05.164 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:05.164 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:05.165 INFO:teuthology.orchestra.run.smithi131.stdout: "cap_revoke_eviction": 0, 2024-05-09T01:02:05.165 INFO:teuthology.orchestra.run.smithi131.stdout: "cap_acquisition_throttle": 0, 2024-05-09T01:02:05.165 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getvxattr_latency": { 2024-05-09T01:02:05.165 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:05.165 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:05.165 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:05.165 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-09T01:02:05.165 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:05.165 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_sessions": { 2024-05-09T01:02:05.165 INFO:teuthology.orchestra.run.smithi131.stdout: "session_count": 1, 2024-05-09T01:02:05.165 INFO:teuthology.orchestra.run.smithi131.stdout: "session_add": 2, 2024-05-09T01:02:05.165 INFO:teuthology.orchestra.run.smithi131.stdout: "session_remove": 1, 2024-05-09T01:02:05.165 INFO:teuthology.orchestra.run.smithi131.stdout: "sessions_open": 1, 2024-05-09T01:02:05.165 INFO:teuthology.orchestra.run.smithi131.stdout: "sessions_stale": 0, 2024-05-09T01:02:05.166 INFO:teuthology.orchestra.run.smithi131.stdout: "total_load": 0, 2024-05-09T01:02:05.166 INFO:teuthology.orchestra.run.smithi131.stdout: "average_load": 0, 2024-05-09T01:02:05.166 INFO:teuthology.orchestra.run.smithi131.stdout: "avg_session_uptime": 2376, 2024-05-09T01:02:05.166 INFO:teuthology.orchestra.run.smithi131.stdout: "mdthresh_evicted": 0 2024-05-09T01:02:05.166 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:05.166 INFO:teuthology.orchestra.run.smithi131.stdout: "mempool": { 2024-05-09T01:02:05.166 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_bytes": 0, 2024-05-09T01:02:05.166 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_items": 0, 2024-05-09T01:02:05.166 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T01:02:05.166 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_items": 0, 2024-05-09T01:02:05.166 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T01:02:05.166 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_items": 0, 2024-05-09T01:02:05.166 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T01:02:05.167 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T01:02:05.167 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T01:02:05.167 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T01:02:05.167 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T01:02:05.167 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_items": 0, 2024-05-09T01:02:05.167 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T01:02:05.167 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T01:02:05.167 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_bytes": 0, 2024-05-09T01:02:05.167 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_items": 0, 2024-05-09T01:02:05.167 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_bytes": 0, 2024-05-09T01:02:05.167 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_items": 0, 2024-05-09T01:02:05.167 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T01:02:05.167 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T01:02:05.167 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T01:02:05.168 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T01:02:05.168 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T01:02:05.168 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_items": 0, 2024-05-09T01:02:05.168 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_bytes": 0, 2024-05-09T01:02:05.168 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_items": 0, 2024-05-09T01:02:05.168 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T01:02:05.168 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T01:02:05.168 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_bytes": 0, 2024-05-09T01:02:05.168 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_items": 0, 2024-05-09T01:02:05.168 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_bytes": 0, 2024-05-09T01:02:05.168 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_items": 0, 2024-05-09T01:02:05.168 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T01:02:05.168 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_items": 0, 2024-05-09T01:02:05.168 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T01:02:05.169 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_items": 0, 2024-05-09T01:02:05.169 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_bytes": 182104, 2024-05-09T01:02:05.169 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_items": 49, 2024-05-09T01:02:05.169 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_bytes": 0, 2024-05-09T01:02:05.169 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_items": 0, 2024-05-09T01:02:05.169 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_bytes": 0, 2024-05-09T01:02:05.169 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_items": 0, 2024-05-09T01:02:05.169 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_bytes": 0, 2024-05-09T01:02:05.169 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_items": 0, 2024-05-09T01:02:05.169 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_bytes": 0, 2024-05-09T01:02:05.169 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_items": 0, 2024-05-09T01:02:05.169 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_bytes": 7736, 2024-05-09T01:02:05.169 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_items": 164, 2024-05-09T01:02:05.169 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T01:02:05.170 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_items": 0, 2024-05-09T01:02:05.170 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_bytes": 0, 2024-05-09T01:02:05.170 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_items": 0, 2024-05-09T01:02:05.170 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_bytes": 14160, 2024-05-09T01:02:05.170 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_items": 40, 2024-05-09T01:02:05.170 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_bytes": 0, 2024-05-09T01:02:05.170 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_items": 0, 2024-05-09T01:02:05.170 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_bytes": 0, 2024-05-09T01:02:05.170 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_items": 0 2024-05-09T01:02:05.170 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:05.170 INFO:teuthology.orchestra.run.smithi131.stdout: "objecter": { 2024-05-09T01:02:05.170 INFO:teuthology.orchestra.run.smithi131.stdout: "op_active": 0, 2024-05-09T01:02:05.170 INFO:teuthology.orchestra.run.smithi131.stdout: "op_laggy": 0, 2024-05-09T01:02:05.170 INFO:teuthology.orchestra.run.smithi131.stdout: "op_send": 3141, 2024-05-09T01:02:05.171 INFO:teuthology.orchestra.run.smithi131.stdout: "op_send_bytes": 21, 2024-05-09T01:02:05.171 INFO:teuthology.orchestra.run.smithi131.stdout: "op_resend": 3, 2024-05-09T01:02:05.171 INFO:teuthology.orchestra.run.smithi131.stdout: "op_reply": 3141, 2024-05-09T01:02:05.171 INFO:teuthology.orchestra.run.smithi131.stdout: "op_latency": { 2024-05-09T01:02:05.171 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 3141, 2024-05-09T01:02:05.171 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 5.015895299, 2024-05-09T01:02:05.171 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.001596910 2024-05-09T01:02:05.171 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:05.171 INFO:teuthology.orchestra.run.smithi131.stdout: "op_inflight": 0, 2024-05-09T01:02:05.171 INFO:teuthology.orchestra.run.smithi131.stdout: "oplen_avg": { 2024-05-09T01:02:05.171 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 3141, 2024-05-09T01:02:05.171 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 3142 2024-05-09T01:02:05.171 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:05.171 INFO:teuthology.orchestra.run.smithi131.stdout: "op": 3141, 2024-05-09T01:02:05.172 INFO:teuthology.orchestra.run.smithi131.stdout: "op_r": 3141, 2024-05-09T01:02:05.172 INFO:teuthology.orchestra.run.smithi131.stdout: "op_w": 0, 2024-05-09T01:02:05.172 INFO:teuthology.orchestra.run.smithi131.stdout: "op_rmw": 0, 2024-05-09T01:02:05.172 INFO:teuthology.orchestra.run.smithi131.stdout: "op_pg": 0, 2024-05-09T01:02:05.172 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_stat": 2049, 2024-05-09T01:02:05.172 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_create": 0, 2024-05-09T01:02:05.172 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_read": 1090, 2024-05-09T01:02:05.172 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_write": 0, 2024-05-09T01:02:05.172 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_writefull": 0, 2024-05-09T01:02:05.172 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_writesame": 0, 2024-05-09T01:02:05.172 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_append": 0, 2024-05-09T01:02:05.172 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_zero": 0, 2024-05-09T01:02:05.172 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_truncate": 0, 2024-05-09T01:02:05.172 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_delete": 0, 2024-05-09T01:02:05.173 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_mapext": 0, 2024-05-09T01:02:05.173 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_sparse_read": 0, 2024-05-09T01:02:05.173 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_clonerange": 0, 2024-05-09T01:02:05.173 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_getxattr": 1, 2024-05-09T01:02:05.173 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_setxattr": 0, 2024-05-09T01:02:05.173 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_cmpxattr": 0, 2024-05-09T01:02:05.173 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_rmxattr": 0, 2024-05-09T01:02:05.173 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_resetxattrs": 0, 2024-05-09T01:02:05.173 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_call": 0, 2024-05-09T01:02:05.173 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_watch": 0, 2024-05-09T01:02:05.173 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_notify": 0, 2024-05-09T01:02:05.173 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T01:02:05.173 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_pgls": 0, 2024-05-09T01:02:05.173 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_pgls_filter": 0, 2024-05-09T01:02:05.173 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_other": 0, 2024-05-09T01:02:05.174 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_active": 0, 2024-05-09T01:02:05.174 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_send": 0, 2024-05-09T01:02:05.174 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_resend": 0, 2024-05-09T01:02:05.174 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_ping": 0, 2024-05-09T01:02:05.174 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_active": 0, 2024-05-09T01:02:05.174 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_send": 0, 2024-05-09T01:02:05.174 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_resend": 0, 2024-05-09T01:02:05.174 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_active": 0, 2024-05-09T01:02:05.174 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_send": 0, 2024-05-09T01:02:05.174 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_resend": 0, 2024-05-09T01:02:05.174 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_active": 0, 2024-05-09T01:02:05.174 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_send": 0, 2024-05-09T01:02:05.174 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_resend": 0, 2024-05-09T01:02:05.174 INFO:teuthology.orchestra.run.smithi131.stdout: "command_active": 0, 2024-05-09T01:02:05.174 INFO:teuthology.orchestra.run.smithi131.stdout: "command_send": 0, 2024-05-09T01:02:05.175 INFO:teuthology.orchestra.run.smithi131.stdout: "command_resend": 0, 2024-05-09T01:02:05.175 INFO:teuthology.orchestra.run.smithi131.stdout: "map_epoch": 82, 2024-05-09T01:02:05.175 INFO:teuthology.orchestra.run.smithi131.stdout: "map_full": 0, 2024-05-09T01:02:05.175 INFO:teuthology.orchestra.run.smithi131.stdout: "map_inc": 4, 2024-05-09T01:02:05.175 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_sessions": 4, 2024-05-09T01:02:05.175 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_session_open": 5, 2024-05-09T01:02:05.175 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_session_close": 1, 2024-05-09T01:02:05.175 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_laggy": 0, 2024-05-09T01:02:05.175 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_wr": 0, 2024-05-09T01:02:05.175 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_rd": 2, 2024-05-09T01:02:05.175 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_del": 0 2024-05-09T01:02:05.175 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:05.175 INFO:teuthology.orchestra.run.smithi131.stdout: "oft": { 2024-05-09T01:02:05.175 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_objs": 0, 2024-05-09T01:02:05.176 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_kv_pairs": 0, 2024-05-09T01:02:05.176 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_updates": 0, 2024-05-09T01:02:05.176 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_removes": 0 2024-05-09T01:02:05.176 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:05.176 INFO:teuthology.orchestra.run.smithi131.stdout: "osd-slow-ops": { 2024-05-09T01:02:05.176 INFO:teuthology.orchestra.run.smithi131.stdout: "slow_ops_count": 0 2024-05-09T01:02:05.176 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:05.176 INFO:teuthology.orchestra.run.smithi131.stdout: "purge_queue": { 2024-05-09T01:02:05.176 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_ops": 0, 2024-05-09T01:02:05.176 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_ops_high_water": 0, 2024-05-09T01:02:05.176 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing": 0, 2024-05-09T01:02:05.176 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_high_water": 0, 2024-05-09T01:02:05.176 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executed_ops": 0, 2024-05-09T01:02:05.176 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executed": 0, 2024-05-09T01:02:05.176 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_item_in_journal": 0 2024-05-09T01:02:05.177 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:05.177 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T01:02:05.177 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-09T01:02:05.177 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 104857600, 2024-05-09T01:02:05.177 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-09T01:02:05.177 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 3465, 2024-05-09T01:02:05.177 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 1186058, 2024-05-09T01:02:05.177 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-09T01:02:05.177 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 3465, 2024-05-09T01:02:05.177 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-05-09T01:02:05.177 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-05-09T01:02:05.177 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 3465, 2024-05-09T01:02:05.177 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 1186058, 2024-05-09T01:02:05.177 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-09T01:02:05.178 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:05.178 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:05.178 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:05.178 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-09T01:02:05.178 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:05.178 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-objecter_bytes": { 2024-05-09T01:02:05.178 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-09T01:02:05.178 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 104857600, 2024-05-09T01:02:05.178 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-09T01:02:05.178 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-05-09T01:02:05.178 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-05-09T01:02:05.178 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-09T01:02:05.178 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-05-09T01:02:05.178 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 3141, 2024-05-09T01:02:05.178 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 23146, 2024-05-09T01:02:05.179 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 3141, 2024-05-09T01:02:05.179 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 23146, 2024-05-09T01:02:05.179 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-09T01:02:05.179 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:05.179 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:05.179 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:05.179 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-09T01:02:05.179 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:05.179 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-objecter_ops": { 2024-05-09T01:02:05.179 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-09T01:02:05.179 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 1024, 2024-05-09T01:02:05.179 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-09T01:02:05.179 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-05-09T01:02:05.179 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-05-09T01:02:05.180 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-09T01:02:05.180 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-05-09T01:02:05.180 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 3141, 2024-05-09T01:02:05.180 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 3141, 2024-05-09T01:02:05.180 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 3141, 2024-05-09T01:02:05.180 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 3141, 2024-05-09T01:02:05.180 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-09T01:02:05.180 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:05.180 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:05.180 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:05.180 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-09T01:02:05.180 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:05.180 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-write_buf_throttle": { 2024-05-09T01:02:05.180 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-09T01:02:05.180 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 3758096384, 2024-05-09T01:02:05.181 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-09T01:02:05.181 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-05-09T01:02:05.181 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-05-09T01:02:05.181 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-09T01:02:05.181 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-05-09T01:02:05.181 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-05-09T01:02:05.181 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-05-09T01:02:05.181 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 0, 2024-05-09T01:02:05.181 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 0, 2024-05-09T01:02:05.181 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-09T01:02:05.181 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:05.181 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:05.181 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:05.181 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-09T01:02:05.181 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:05.182 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-write_buf_throttle-0x563f009368f0": { 2024-05-09T01:02:05.182 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-09T01:02:05.182 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 3758096384, 2024-05-09T01:02:05.182 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-09T01:02:05.182 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-05-09T01:02:05.182 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-05-09T01:02:05.182 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-09T01:02:05.182 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-05-09T01:02:05.182 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-05-09T01:02:05.182 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-05-09T01:02:05.182 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 0, 2024-05-09T01:02:05.182 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 0, 2024-05-09T01:02:05.182 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-09T01:02:05.182 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:05.183 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:05.183 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:05.183 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-09T01:02:05.183 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-09T01:02:05.183 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-09T01:02:05.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:05 smithi131 ceph-mon[34547]: pgmap v843: 97 pgs: 97 active+clean; 6.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 13 op/s 2024-05-09T01:02:05.918 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-05-09T01:02:05.919 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-05-09T01:02:05.919 DEBUG:tasks.check_counter:Getting stats from h 2024-05-09T01:02:05.919 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph daemon mds.h perf dump 2024-05-09T01:02:05.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:05 smithi157 ceph-mon[34294]: pgmap v843: 97 pgs: 97 active+clean; 6.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 13 op/s 2024-05-09T01:02:06.040 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:05 smithi005 ceph-mon[30365]: pgmap v843: 97 pgs: 97 active+clean; 6.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 13 op/s 2024-05-09T01:02:06.164 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T01:02:07.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:07 smithi131 ceph-mon[34547]: pgmap v844: 97 pgs: 97 active+clean; 6.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T01:02:07.927 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-09T01:02:07.927 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T01:02:07.927 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 464, 2024-05-09T01:02:07.927 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 500, 2024-05-09T01:02:07.927 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 676281, 2024-05-09T01:02:07.927 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 662525, 2024-05-09T01:02:07.927 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 32, 2024-05-09T01:02:07.927 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 3, 2024-05-09T01:02:07.927 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.080307359, 2024-05-09T01:02:07.927 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.024928298, 2024-05-09T01:02:07.927 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.041964234, 2024-05-09T01:02:07.928 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.010225597, 2024-05-09T01:02:07.928 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:02:07.928 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 497, 2024-05-09T01:02:07.928 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.041616097, 2024-05-09T01:02:07.928 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000083734 2024-05-09T01:02:07.928 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:07.928 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:02:07.928 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 176, 2024-05-09T01:02:07.928 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000044669, 2024-05-09T01:02:07.928 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000253 2024-05-09T01:02:07.928 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:07.928 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 615600, 2024-05-09T01:02:07.928 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 363264 2024-05-09T01:02:07.928 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:07.929 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T01:02:07.929 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 667, 2024-05-09T01:02:07.929 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 1044, 2024-05-09T01:02:07.929 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 208780, 2024-05-09T01:02:07.929 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 692539, 2024-05-09T01:02:07.929 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 15, 2024-05-09T01:02:07.929 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 6, 2024-05-09T01:02:07.929 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.103019684, 2024-05-09T01:02:07.929 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.039063300, 2024-05-09T01:02:07.929 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.057549541, 2024-05-09T01:02:07.929 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.006755891, 2024-05-09T01:02:07.929 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:02:07.929 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 1044, 2024-05-09T01:02:07.930 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.061895933, 2024-05-09T01:02:07.930 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000059287 2024-05-09T01:02:07.930 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:07.930 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:02:07.930 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 1474, 2024-05-09T01:02:07.930 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000510171, 2024-05-09T01:02:07.930 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000346 2024-05-09T01:02:07.930 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:07.930 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-09T01:02:07.930 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 396352 2024-05-09T01:02:07.930 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:07.930 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T01:02:07.931 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 102, 2024-05-09T01:02:07.931 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 102, 2024-05-09T01:02:07.931 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 62848, 2024-05-09T01:02:07.931 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 63870, 2024-05-09T01:02:07.931 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 9, 2024-05-09T01:02:07.931 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 4, 2024-05-09T01:02:07.931 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.012296233, 2024-05-09T01:02:07.931 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.004149401, 2024-05-09T01:02:07.931 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.008831412, 2024-05-09T01:02:07.931 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.000331988, 2024-05-09T01:02:07.931 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:02:07.931 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 101, 2024-05-09T01:02:07.931 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.023282628, 2024-05-09T01:02:07.931 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000230521 2024-05-09T01:02:07.932 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:07.932 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:02:07.932 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 151, 2024-05-09T01:02:07.932 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000035324, 2024-05-09T01:02:07.932 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000233 2024-05-09T01:02:07.932 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:07.932 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-05-09T01:02:07.932 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 448 2024-05-09T01:02:07.932 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:07.932 INFO:teuthology.orchestra.run.smithi131.stdout: "cct": { 2024-05-09T01:02:07.932 INFO:teuthology.orchestra.run.smithi131.stdout: "total_workers": 1, 2024-05-09T01:02:07.932 INFO:teuthology.orchestra.run.smithi131.stdout: "unhealthy_workers": 0 2024-05-09T01:02:07.932 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:07.932 INFO:teuthology.orchestra.run.smithi131.stdout: "finisher-MDSRank": { 2024-05-09T01:02:07.933 INFO:teuthology.orchestra.run.smithi131.stdout: "queue_len": 0, 2024-05-09T01:02:07.933 INFO:teuthology.orchestra.run.smithi131.stdout: "complete_latency": { 2024-05-09T01:02:07.933 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 19, 2024-05-09T01:02:07.933 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.001420892, 2024-05-09T01:02:07.933 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000074783 2024-05-09T01:02:07.933 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-09T01:02:07.933 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:07.933 INFO:teuthology.orchestra.run.smithi131.stdout: "finisher-PurgeQueue": { 2024-05-09T01:02:07.933 INFO:teuthology.orchestra.run.smithi131.stdout: "queue_len": 0, 2024-05-09T01:02:07.933 INFO:teuthology.orchestra.run.smithi131.stdout: "complete_latency": { 2024-05-09T01:02:07.933 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 2, 2024-05-09T01:02:07.933 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000015694, 2024-05-09T01:02:07.933 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000007847 2024-05-09T01:02:07.933 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-09T01:02:07.933 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:07.934 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": { 2024-05-09T01:02:07.934 INFO:teuthology.orchestra.run.smithi131.stdout: "request": 0, 2024-05-09T01:02:07.934 INFO:teuthology.orchestra.run.smithi131.stdout: "reply": 0, 2024-05-09T01:02:07.934 INFO:teuthology.orchestra.run.smithi131.stdout: "reply_latency": { 2024-05-09T01:02:07.934 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:07.934 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:07.934 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:07.934 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:07.934 INFO:teuthology.orchestra.run.smithi131.stdout: "slow_reply": 0, 2024-05-09T01:02:07.934 INFO:teuthology.orchestra.run.smithi131.stdout: "forward": 0, 2024-05-09T01:02:07.934 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_fetch_complete": 0, 2024-05-09T01:02:07.934 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_fetch_keys": 0, 2024-05-09T01:02:07.934 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_commit": 11, 2024-05-09T01:02:07.934 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_split": 0, 2024-05-09T01:02:07.935 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_merge": 0, 2024-05-09T01:02:07.935 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes": 10, 2024-05-09T01:02:07.935 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_top": 0, 2024-05-09T01:02:07.935 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_bottom": 10, 2024-05-09T01:02:07.935 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_pin_tail": 0, 2024-05-09T01:02:07.935 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_pinned": 10, 2024-05-09T01:02:07.935 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_expired": 0, 2024-05-09T01:02:07.935 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_with_caps": 0, 2024-05-09T01:02:07.935 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": 0, 2024-05-09T01:02:07.935 INFO:teuthology.orchestra.run.smithi131.stdout: "subtrees": 1, 2024-05-09T01:02:07.935 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse": 0, 2024-05-09T01:02:07.935 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_hit": 0, 2024-05-09T01:02:07.935 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_forward": 0, 2024-05-09T01:02:07.935 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_discover": 0, 2024-05-09T01:02:07.936 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_dir_fetch": 0, 2024-05-09T01:02:07.936 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_remote_ino": 0, 2024-05-09T01:02:07.936 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_lock": 0, 2024-05-09T01:02:07.936 INFO:teuthology.orchestra.run.smithi131.stdout: "load_cent": 0, 2024-05-09T01:02:07.936 INFO:teuthology.orchestra.run.smithi131.stdout: "q": 0, 2024-05-09T01:02:07.936 INFO:teuthology.orchestra.run.smithi131.stdout: "exported": 0, 2024-05-09T01:02:07.936 INFO:teuthology.orchestra.run.smithi131.stdout: "exported_inodes": 0, 2024-05-09T01:02:07.936 INFO:teuthology.orchestra.run.smithi131.stdout: "imported": 0, 2024-05-09T01:02:07.936 INFO:teuthology.orchestra.run.smithi131.stdout: "imported_inodes": 0, 2024-05-09T01:02:07.936 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_dir_fetch": 0, 2024-05-09T01:02:07.937 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_backtrace_fetch": 0, 2024-05-09T01:02:07.937 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_peer_discover": 0, 2024-05-09T01:02:07.937 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rfiles": 21, 2024-05-09T01:02:07.937 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rbytes": 10738136400, 2024-05-09T01:02:07.937 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rsnaps": 0, 2024-05-09T01:02:07.937 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_backtrace_fetch": 0, 2024-05-09T01:02:07.937 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_set_tag": 0, 2024-05-09T01:02:07.937 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T01:02:07.937 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_inotable_repaired": 0, 2024-05-09T01:02:07.937 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dir_inodes": 0, 2024-05-09T01:02:07.937 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T01:02:07.937 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T01:02:07.938 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_file_inodes": 0, 2024-05-09T01:02:07.938 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_inode_file_caps": 0, 2024-05-09T01:02:07.938 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_revoke": 0, 2024-05-09T01:02:07.938 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_grant": 0, 2024-05-09T01:02:07.938 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T01:02:07.938 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T01:02:07.938 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T01:02:07.938 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_caps": 0, 2024-05-09T01:02:07.938 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_caps_dirty": 0, 2024-05-09T01:02:07.938 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_cap_release": 0, 2024-05-09T01:02:07.938 INFO:teuthology.orchestra.run.smithi131.stdout: "process_request_cap_release": 0 2024-05-09T01:02:07.939 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:07.939 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_cache": { 2024-05-09T01:02:07.939 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_update": 0, 2024-05-09T01:02:07.939 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_update_receipt": 0, 2024-05-09T01:02:07.939 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_try_discover": 0, 2024-05-09T01:02:07.939 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_send_discover": 1, 2024-05-09T01:02:07.939 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_handle_discover": 0, 2024-05-09T01:02:07.939 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays": 0, 2024-05-09T01:02:07.939 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays_delayed": 0, 2024-05-09T01:02:07.939 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays_enqueuing": 0, 2024-05-09T01:02:07.939 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_created": 0, 2024-05-09T01:02:07.940 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_enqueued": 0, 2024-05-09T01:02:07.940 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_reintegrated": 0, 2024-05-09T01:02:07.940 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_migrated": 0, 2024-05-09T01:02:07.940 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_processing": 0, 2024-05-09T01:02:07.940 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_enqueued": 0, 2024-05-09T01:02:07.940 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_prioritized": 0, 2024-05-09T01:02:07.940 INFO:teuthology.orchestra.run.smithi131.stdout: "recovery_started": 0, 2024-05-09T01:02:07.940 INFO:teuthology.orchestra.run.smithi131.stdout: "recovery_completed": 0, 2024-05-09T01:02:07.940 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_quiesce_path": 0, 2024-05-09T01:02:07.940 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_quiesce_inode": 0, 2024-05-09T01:02:07.940 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T01:02:07.940 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_exportdir": 0, 2024-05-09T01:02:07.941 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_flush": 0, 2024-05-09T01:02:07.941 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_fragmentdir": 0, 2024-05-09T01:02:07.941 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_fragstats": 0, 2024-05-09T01:02:07.941 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_inodestats": 0 2024-05-09T01:02:07.941 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:07.941 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_log": { 2024-05-09T01:02:07.941 INFO:teuthology.orchestra.run.smithi131.stdout: "evlrg": 0, 2024-05-09T01:02:07.941 INFO:teuthology.orchestra.run.smithi131.stdout: "evadd": 2, 2024-05-09T01:02:07.941 INFO:teuthology.orchestra.run.smithi131.stdout: "evex": 0, 2024-05-09T01:02:07.941 INFO:teuthology.orchestra.run.smithi131.stdout: "evtrm": 0, 2024-05-09T01:02:07.942 INFO:teuthology.orchestra.run.smithi131.stdout: "ev": 2, 2024-05-09T01:02:07.942 INFO:teuthology.orchestra.run.smithi131.stdout: "evexg": 0, 2024-05-09T01:02:07.942 INFO:teuthology.orchestra.run.smithi131.stdout: "evexd": 0, 2024-05-09T01:02:07.942 INFO:teuthology.orchestra.run.smithi131.stdout: "segadd": 2, 2024-05-09T01:02:07.942 INFO:teuthology.orchestra.run.smithi131.stdout: "segex": 0, 2024-05-09T01:02:07.942 INFO:teuthology.orchestra.run.smithi131.stdout: "segtrm": 0, 2024-05-09T01:02:07.942 INFO:teuthology.orchestra.run.smithi131.stdout: "seg": 2, 2024-05-09T01:02:07.942 INFO:teuthology.orchestra.run.smithi131.stdout: "segmjr": 2, 2024-05-09T01:02:07.942 INFO:teuthology.orchestra.run.smithi131.stdout: "segexg": 0, 2024-05-09T01:02:07.942 INFO:teuthology.orchestra.run.smithi131.stdout: "segexd": 0, 2024-05-09T01:02:07.942 INFO:teuthology.orchestra.run.smithi131.stdout: "expos": 4194304, 2024-05-09T01:02:07.942 INFO:teuthology.orchestra.run.smithi131.stdout: "wrpos": 4194892, 2024-05-09T01:02:07.942 INFO:teuthology.orchestra.run.smithi131.stdout: "rdpos": 0, 2024-05-09T01:02:07.942 INFO:teuthology.orchestra.run.smithi131.stdout: "jlat": { 2024-05-09T01:02:07.943 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 1, 2024-05-09T01:02:07.943 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.001606848, 2024-05-09T01:02:07.943 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.001606848 2024-05-09T01:02:07.943 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:07.943 INFO:teuthology.orchestra.run.smithi131.stdout: "replayed": 0 2024-05-09T01:02:07.943 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:07.943 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_mem": { 2024-05-09T01:02:07.943 INFO:teuthology.orchestra.run.smithi131.stdout: "ino": 13, 2024-05-09T01:02:07.943 INFO:teuthology.orchestra.run.smithi131.stdout: "ino+": 13, 2024-05-09T01:02:07.943 INFO:teuthology.orchestra.run.smithi131.stdout: "ino-": 0, 2024-05-09T01:02:07.943 INFO:teuthology.orchestra.run.smithi131.stdout: "dir": 11, 2024-05-09T01:02:07.943 INFO:teuthology.orchestra.run.smithi131.stdout: "dir+": 12, 2024-05-09T01:02:07.943 INFO:teuthology.orchestra.run.smithi131.stdout: "dir-": 1, 2024-05-09T01:02:07.943 INFO:teuthology.orchestra.run.smithi131.stdout: "dn": 10, 2024-05-09T01:02:07.944 INFO:teuthology.orchestra.run.smithi131.stdout: "dn+": 10, 2024-05-09T01:02:07.944 INFO:teuthology.orchestra.run.smithi131.stdout: "dn-": 0, 2024-05-09T01:02:07.944 INFO:teuthology.orchestra.run.smithi131.stdout: "cap": 0, 2024-05-09T01:02:07.944 INFO:teuthology.orchestra.run.smithi131.stdout: "cap+": 0, 2024-05-09T01:02:07.944 INFO:teuthology.orchestra.run.smithi131.stdout: "cap-": 0, 2024-05-09T01:02:07.944 INFO:teuthology.orchestra.run.smithi131.stdout: "rss": 120700, 2024-05-09T01:02:07.944 INFO:teuthology.orchestra.run.smithi131.stdout: "heap": 223632 2024-05-09T01:02:07.944 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:07.944 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_server": { 2024-05-09T01:02:07.944 INFO:teuthology.orchestra.run.smithi131.stdout: "dispatch_client_request": 0, 2024-05-09T01:02:07.944 INFO:teuthology.orchestra.run.smithi131.stdout: "dispatch_server_request": 0, 2024-05-09T01:02:07.944 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_request": 0, 2024-05-09T01:02:07.944 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_session": 0, 2024-05-09T01:02:07.944 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_peer_request": 0, 2024-05-09T01:02:07.944 INFO:teuthology.orchestra.run.smithi131.stdout: "req_create_latency": { 2024-05-09T01:02:07.945 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:07.945 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:07.945 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:07.945 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:07.945 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getattr_latency": { 2024-05-09T01:02:07.946 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:07.946 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:07.946 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:07.946 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:07.946 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getfilelock_latency": { 2024-05-09T01:02:07.946 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:07.946 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:07.946 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:07.947 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:07.947 INFO:teuthology.orchestra.run.smithi131.stdout: "req_link_latency": { 2024-05-09T01:02:07.947 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:07.947 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:07.947 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:07.947 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:07.947 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookup_latency": { 2024-05-09T01:02:07.947 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:07.947 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:07.947 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:07.947 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:07.947 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookuphash_latency": { 2024-05-09T01:02:07.947 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:07.947 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:07.948 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:07.948 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:07.948 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupino_latency": { 2024-05-09T01:02:07.948 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:07.948 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:07.948 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:07.948 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:07.948 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupname_latency": { 2024-05-09T01:02:07.948 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:07.948 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:07.948 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:07.948 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:07.948 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupparent_latency": { 2024-05-09T01:02:07.948 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:07.948 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:07.949 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:07.949 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:07.949 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupsnap_latency": { 2024-05-09T01:02:07.949 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:07.949 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:07.949 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:07.949 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:07.949 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lssnap_latency": { 2024-05-09T01:02:07.949 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:07.950 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:07.950 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:07.950 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:07.950 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mkdir_latency": { 2024-05-09T01:02:07.950 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:07 smithi157 ceph-mon[34294]: pgmap v844: 97 pgs: 97 active+clean; 6.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T01:02:07.951 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:07.951 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:07.951 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:07.951 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:07.951 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mknod_latency": { 2024-05-09T01:02:07.951 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:07.951 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:07.951 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:07.951 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:07.951 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mksnap_latency": { 2024-05-09T01:02:07.951 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:07.953 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:07.953 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:07.953 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:07.953 INFO:teuthology.orchestra.run.smithi131.stdout: "req_open_latency": { 2024-05-09T01:02:07.953 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:07.953 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:07.953 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:07.953 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:07.954 INFO:teuthology.orchestra.run.smithi131.stdout: "req_readdir_latency": { 2024-05-09T01:02:07.954 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:07.954 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:07.954 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:07.954 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:07.954 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rename_latency": { 2024-05-09T01:02:07.954 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:07.954 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:07.954 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:07.954 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:07.954 INFO:teuthology.orchestra.run.smithi131.stdout: "req_renamesnap_latency": { 2024-05-09T01:02:07.954 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:07.954 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:07.955 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:07.955 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:07.955 INFO:teuthology.orchestra.run.smithi131.stdout: "req_snapdiff_latency": { 2024-05-09T01:02:07.955 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:07.955 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:07.955 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:07.955 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:07.955 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmdir_latency": { 2024-05-09T01:02:07.955 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:07.955 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:07.955 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:07.955 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:07.955 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmsnap_latency": { 2024-05-09T01:02:07.955 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:07.956 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:07.956 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:07.956 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:07.956 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmxattr_latency": { 2024-05-09T01:02:07.956 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:07.956 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:07.956 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:07.957 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:07.957 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setattr_latency": { 2024-05-09T01:02:07.957 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:07.957 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:07.957 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:07.957 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:07.957 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setdirlayout_latency": { 2024-05-09T01:02:07.957 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:07.957 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:07.957 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:07.957 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:07.957 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setfilelock_latency": { 2024-05-09T01:02:07.958 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:07.958 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:07.958 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:07.958 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:07.958 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setlayout_latency": { 2024-05-09T01:02:07.958 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:07.958 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:07.958 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:07.958 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:07.958 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setxattr_latency": { 2024-05-09T01:02:07.958 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:07.958 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:07.958 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:07.958 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:07.959 INFO:teuthology.orchestra.run.smithi131.stdout: "req_symlink_latency": { 2024-05-09T01:02:07.959 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:07.959 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:07.959 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:07.959 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:07.959 INFO:teuthology.orchestra.run.smithi131.stdout: "req_unlink_latency": { 2024-05-09T01:02:07.959 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:07.959 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:07.959 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:07.959 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:07.959 INFO:teuthology.orchestra.run.smithi131.stdout: "cap_revoke_eviction": 0, 2024-05-09T01:02:07.959 INFO:teuthology.orchestra.run.smithi131.stdout: "cap_acquisition_throttle": 0, 2024-05-09T01:02:07.959 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getvxattr_latency": { 2024-05-09T01:02:07.959 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:07.960 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:07.960 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:07.960 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-09T01:02:07.960 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:07.960 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_sessions": { 2024-05-09T01:02:07.960 INFO:teuthology.orchestra.run.smithi131.stdout: "session_count": 0, 2024-05-09T01:02:07.960 INFO:teuthology.orchestra.run.smithi131.stdout: "session_add": 0, 2024-05-09T01:02:07.960 INFO:teuthology.orchestra.run.smithi131.stdout: "session_remove": 0, 2024-05-09T01:02:07.960 INFO:teuthology.orchestra.run.smithi131.stdout: "sessions_open": 0, 2024-05-09T01:02:07.960 INFO:teuthology.orchestra.run.smithi131.stdout: "sessions_stale": 0, 2024-05-09T01:02:07.960 INFO:teuthology.orchestra.run.smithi131.stdout: "total_load": 0, 2024-05-09T01:02:07.960 INFO:teuthology.orchestra.run.smithi131.stdout: "average_load": 0, 2024-05-09T01:02:07.960 INFO:teuthology.orchestra.run.smithi131.stdout: "avg_session_uptime": 0, 2024-05-09T01:02:07.960 INFO:teuthology.orchestra.run.smithi131.stdout: "mdthresh_evicted": 0 2024-05-09T01:02:07.961 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:07.961 INFO:teuthology.orchestra.run.smithi131.stdout: "mempool": { 2024-05-09T01:02:07.961 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_bytes": 0, 2024-05-09T01:02:07.961 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_items": 0, 2024-05-09T01:02:07.961 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T01:02:07.961 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_items": 0, 2024-05-09T01:02:07.961 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T01:02:07.961 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_items": 0, 2024-05-09T01:02:07.961 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T01:02:07.961 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T01:02:07.961 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T01:02:07.961 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T01:02:07.961 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T01:02:07.961 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_items": 0, 2024-05-09T01:02:07.962 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T01:02:07.962 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T01:02:07.962 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_bytes": 0, 2024-05-09T01:02:07.962 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_items": 0, 2024-05-09T01:02:07.962 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_bytes": 0, 2024-05-09T01:02:07.962 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_items": 0, 2024-05-09T01:02:07.962 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T01:02:07.962 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T01:02:07.962 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T01:02:07.962 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T01:02:07.962 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T01:02:07.962 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_items": 0, 2024-05-09T01:02:07.962 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_bytes": 0, 2024-05-09T01:02:07.962 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_items": 0, 2024-05-09T01:02:07.963 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T01:02:07.963 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T01:02:07.963 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_bytes": 0, 2024-05-09T01:02:07.963 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_items": 0, 2024-05-09T01:02:07.963 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_bytes": 0, 2024-05-09T01:02:07.963 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_items": 0, 2024-05-09T01:02:07.963 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T01:02:07.963 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_items": 0, 2024-05-09T01:02:07.963 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T01:02:07.963 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_items": 0, 2024-05-09T01:02:07.963 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_bytes": 255799, 2024-05-09T01:02:07.963 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_items": 88, 2024-05-09T01:02:07.963 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_bytes": 0, 2024-05-09T01:02:07.964 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_items": 0, 2024-05-09T01:02:07.964 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_bytes": 0, 2024-05-09T01:02:07.964 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_items": 0, 2024-05-09T01:02:07.964 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_bytes": 0, 2024-05-09T01:02:07.964 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_items": 0, 2024-05-09T01:02:07.964 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_bytes": 0, 2024-05-09T01:02:07.964 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_items": 0, 2024-05-09T01:02:07.964 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_bytes": 7736, 2024-05-09T01:02:07.964 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_items": 164, 2024-05-09T01:02:07.965 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T01:02:07.965 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_items": 0, 2024-05-09T01:02:07.965 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_bytes": 0, 2024-05-09T01:02:07.965 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_items": 0, 2024-05-09T01:02:07.965 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_bytes": 50872, 2024-05-09T01:02:07.965 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_items": 198, 2024-05-09T01:02:07.965 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_bytes": 0, 2024-05-09T01:02:07.965 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_items": 0, 2024-05-09T01:02:07.965 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_bytes": 0, 2024-05-09T01:02:07.965 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_items": 0 2024-05-09T01:02:07.965 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:07.966 INFO:teuthology.orchestra.run.smithi131.stdout: "objecter": { 2024-05-09T01:02:07.966 INFO:teuthology.orchestra.run.smithi131.stdout: "op_active": 0, 2024-05-09T01:02:07.966 INFO:teuthology.orchestra.run.smithi131.stdout: "op_laggy": 0, 2024-05-09T01:02:07.966 INFO:teuthology.orchestra.run.smithi131.stdout: "op_send": 35, 2024-05-09T01:02:07.966 INFO:teuthology.orchestra.run.smithi131.stdout: "op_send_bytes": 10710, 2024-05-09T01:02:07.966 INFO:teuthology.orchestra.run.smithi131.stdout: "op_resend": 33, 2024-05-09T01:02:07.966 INFO:teuthology.orchestra.run.smithi131.stdout: "op_reply": 35, 2024-05-09T01:02:07.966 INFO:teuthology.orchestra.run.smithi131.stdout: "op_latency": { 2024-05-09T01:02:07.966 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 35, 2024-05-09T01:02:07.966 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.227995178, 2024-05-09T01:02:07.966 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.006514147 2024-05-09T01:02:07.966 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:07.966 INFO:teuthology.orchestra.run.smithi131.stdout: "op_inflight": 0, 2024-05-09T01:02:07.966 INFO:teuthology.orchestra.run.smithi131.stdout: "oplen_avg": { 2024-05-09T01:02:07.967 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 35, 2024-05-09T01:02:07.967 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 57 2024-05-09T01:02:07.967 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:07.967 INFO:teuthology.orchestra.run.smithi131.stdout: "op": 35, 2024-05-09T01:02:07.967 INFO:teuthology.orchestra.run.smithi131.stdout: "op_r": 0, 2024-05-09T01:02:07.967 INFO:teuthology.orchestra.run.smithi131.stdout: "op_w": 35, 2024-05-09T01:02:07.967 INFO:teuthology.orchestra.run.smithi131.stdout: "op_rmw": 0, 2024-05-09T01:02:07.967 INFO:teuthology.orchestra.run.smithi131.stdout: "op_pg": 0, 2024-05-09T01:02:07.967 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_stat": 0, 2024-05-09T01:02:07.967 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_create": 10, 2024-05-09T01:02:07.967 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_read": 0, 2024-05-09T01:02:07.967 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_write": 1, 2024-05-09T01:02:07.967 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_writefull": 5, 2024-05-09T01:02:07.968 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_writesame": 0, 2024-05-09T01:02:07.968 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_append": 0, 2024-05-09T01:02:07.968 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_zero": 0, 2024-05-09T01:02:07.968 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_truncate": 0, 2024-05-09T01:02:07.968 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_delete": 6, 2024-05-09T01:02:07.968 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_mapext": 0, 2024-05-09T01:02:07.968 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_sparse_read": 0, 2024-05-09T01:02:07.968 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_clonerange": 0, 2024-05-09T01:02:07.968 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_getxattr": 0, 2024-05-09T01:02:07.968 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_setxattr": 20, 2024-05-09T01:02:07.968 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_cmpxattr": 0, 2024-05-09T01:02:07.968 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_rmxattr": 0, 2024-05-09T01:02:07.968 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_resetxattrs": 0, 2024-05-09T01:02:07.968 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_call": 0, 2024-05-09T01:02:07.969 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_watch": 0, 2024-05-09T01:02:07.969 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_notify": 0, 2024-05-09T01:02:07.969 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T01:02:07.969 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_pgls": 0, 2024-05-09T01:02:07.969 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_pgls_filter": 0, 2024-05-09T01:02:07.969 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_other": 0, 2024-05-09T01:02:07.969 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_active": 0, 2024-05-09T01:02:07.969 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_send": 0, 2024-05-09T01:02:07.969 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_resend": 0, 2024-05-09T01:02:07.969 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_ping": 0, 2024-05-09T01:02:07.969 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_active": 0, 2024-05-09T01:02:07.969 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_send": 0, 2024-05-09T01:02:07.969 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_resend": 0, 2024-05-09T01:02:07.969 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_active": 0, 2024-05-09T01:02:07.969 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_send": 0, 2024-05-09T01:02:07.970 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_resend": 0, 2024-05-09T01:02:07.970 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_active": 0, 2024-05-09T01:02:07.970 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_send": 0, 2024-05-09T01:02:07.970 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_resend": 0, 2024-05-09T01:02:07.970 INFO:teuthology.orchestra.run.smithi131.stdout: "command_active": 0, 2024-05-09T01:02:07.970 INFO:teuthology.orchestra.run.smithi131.stdout: "command_send": 0, 2024-05-09T01:02:07.970 INFO:teuthology.orchestra.run.smithi131.stdout: "command_resend": 0, 2024-05-09T01:02:07.970 INFO:teuthology.orchestra.run.smithi131.stdout: "map_epoch": 82, 2024-05-09T01:02:07.970 INFO:teuthology.orchestra.run.smithi131.stdout: "map_full": 0, 2024-05-09T01:02:07.970 INFO:teuthology.orchestra.run.smithi131.stdout: "map_inc": 4, 2024-05-09T01:02:07.970 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_sessions": 9, 2024-05-09T01:02:07.970 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_session_open": 12, 2024-05-09T01:02:07.970 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_session_close": 3, 2024-05-09T01:02:07.971 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_laggy": 0, 2024-05-09T01:02:07.971 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_wr": 14, 2024-05-09T01:02:07.971 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_rd": 0, 2024-05-09T01:02:07.971 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_del": 1 2024-05-09T01:02:07.971 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:07.971 INFO:teuthology.orchestra.run.smithi131.stdout: "oft": { 2024-05-09T01:02:07.971 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_objs": 1, 2024-05-09T01:02:07.971 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_kv_pairs": 0, 2024-05-09T01:02:07.971 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_updates": 0, 2024-05-09T01:02:07.971 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_removes": 0 2024-05-09T01:02:07.971 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:07.971 INFO:teuthology.orchestra.run.smithi131.stdout: "osd-slow-ops": { 2024-05-09T01:02:07.971 INFO:teuthology.orchestra.run.smithi131.stdout: "slow_ops_count": 0 2024-05-09T01:02:07.971 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:07.971 INFO:teuthology.orchestra.run.smithi131.stdout: "purge_queue": { 2024-05-09T01:02:07.972 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_ops": 0, 2024-05-09T01:02:07.972 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_ops_high_water": 0, 2024-05-09T01:02:07.972 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing": 0, 2024-05-09T01:02:07.972 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_high_water": 0, 2024-05-09T01:02:07.972 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executed_ops": 0, 2024-05-09T01:02:07.972 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executed": 0, 2024-05-09T01:02:07.972 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_item_in_journal": 0 2024-05-09T01:02:07.972 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:07.972 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T01:02:07.972 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-09T01:02:07.972 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 104857600, 2024-05-09T01:02:07.972 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-09T01:02:07.972 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 1229, 2024-05-09T01:02:07.972 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 816349, 2024-05-09T01:02:07.972 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-09T01:02:07.973 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 1229, 2024-05-09T01:02:07.973 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-05-09T01:02:07.973 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-05-09T01:02:07.973 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 1229, 2024-05-09T01:02:07.973 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 816349, 2024-05-09T01:02:07.973 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-09T01:02:07.973 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:07.973 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:07.973 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:07.973 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-09T01:02:07.973 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:07.973 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-objecter_bytes": { 2024-05-09T01:02:07.973 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-09T01:02:07.973 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 104857600, 2024-05-09T01:02:07.973 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-09T01:02:07.974 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-05-09T01:02:07.974 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-05-09T01:02:07.974 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-09T01:02:07.974 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-05-09T01:02:07.974 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 35, 2024-05-09T01:02:07.974 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 10710, 2024-05-09T01:02:07.974 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 35, 2024-05-09T01:02:07.974 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 10710, 2024-05-09T01:02:07.974 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-09T01:02:07.974 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:07.974 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:07.974 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:07.974 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-09T01:02:07.975 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:07.975 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-objecter_ops": { 2024-05-09T01:02:07.975 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-09T01:02:07.975 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 1024, 2024-05-09T01:02:07.975 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-09T01:02:07.975 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-05-09T01:02:07.975 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-05-09T01:02:07.975 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-09T01:02:07.975 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-05-09T01:02:07.975 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 35, 2024-05-09T01:02:07.975 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 35, 2024-05-09T01:02:07.975 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 35, 2024-05-09T01:02:07.975 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 35, 2024-05-09T01:02:07.975 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-09T01:02:07.975 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:07.976 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:07.976 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:07.976 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-09T01:02:07.976 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:07.976 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-write_buf_throttle": { 2024-05-09T01:02:07.976 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-09T01:02:07.976 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 3758096384, 2024-05-09T01:02:07.976 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-09T01:02:07.976 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-05-09T01:02:07.976 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-05-09T01:02:07.976 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-09T01:02:07.976 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-05-09T01:02:07.976 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-05-09T01:02:07.976 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-05-09T01:02:07.977 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 0, 2024-05-09T01:02:07.977 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 0, 2024-05-09T01:02:07.977 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-09T01:02:07.977 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:07.977 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:07.977 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:07.977 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-09T01:02:07.977 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:07.977 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-write_buf_throttle-0x55af2361e4e0": { 2024-05-09T01:02:07.977 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-09T01:02:07.977 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 3758096384, 2024-05-09T01:02:07.977 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-09T01:02:07.977 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 2, 2024-05-09T01:02:07.977 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 588, 2024-05-09T01:02:07.977 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-09T01:02:07.978 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 2, 2024-05-09T01:02:07.978 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-05-09T01:02:07.978 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-05-09T01:02:07.978 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 1, 2024-05-09T01:02:07.978 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 588, 2024-05-09T01:02:07.978 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-09T01:02:07.978 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:07.978 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:07.978 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:07.978 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-09T01:02:07.978 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-09T01:02:07.978 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-09T01:02:08.040 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:07 smithi005 ceph-mon[30365]: pgmap v844: 97 pgs: 97 active+clean; 6.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T01:02:08.721 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-05-09T01:02:08.721 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-05-09T01:02:08.721 DEBUG:tasks.check_counter:Getting stats from k 2024-05-09T01:02:08.721 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph daemon mds.k perf dump 2024-05-09T01:02:08.959 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T01:02:09.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:09 smithi157 ceph-mon[34294]: pgmap v845: 97 pgs: 97 active+clean; 6.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 15 MiB/s wr, 165 op/s 2024-05-09T01:02:10.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:09 smithi005 ceph-mon[30365]: pgmap v845: 97 pgs: 97 active+clean; 6.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 15 MiB/s wr, 165 op/s 2024-05-09T01:02:10.069 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:09 smithi131 ceph-mon[34547]: pgmap v845: 97 pgs: 97 active+clean; 6.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 15 MiB/s wr, 165 op/s 2024-05-09T01:02:10.681 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-09T01:02:10.681 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T01:02:10.681 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 1069, 2024-05-09T01:02:10.681 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 1071, 2024-05-09T01:02:10.681 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 282081, 2024-05-09T01:02:10.681 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 340006, 2024-05-09T01:02:10.681 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 7, 2024-05-09T01:02:10.681 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 2, 2024-05-09T01:02:10.682 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.108026459, 2024-05-09T01:02:10.682 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.039168773, 2024-05-09T01:02:10.682 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.026903495, 2024-05-09T01:02:10.682 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.031688121, 2024-05-09T01:02:10.682 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:02:10.682 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 1069, 2024-05-09T01:02:10.682 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.044818270, 2024-05-09T01:02:10.682 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000041925 2024-05-09T01:02:10.682 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:10.682 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:02:10.682 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:10.682 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:10.683 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:10.683 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:10.683 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T01:02:10.683 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T01:02:10.683 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:10.683 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T01:02:10.683 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 1286, 2024-05-09T01:02:10.683 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 1274, 2024-05-09T01:02:10.683 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 842520, 2024-05-09T01:02:10.683 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 495679, 2024-05-09T01:02:10.683 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 4, 2024-05-09T01:02:10.683 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 3, 2024-05-09T01:02:10.684 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.167253510, 2024-05-09T01:02:10.684 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.042946734, 2024-05-09T01:02:10.684 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.053917863, 2024-05-09T01:02:10.684 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.062065867, 2024-05-09T01:02:10.684 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:02:10.684 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 1274, 2024-05-09T01:02:10.684 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.049791070, 2024-05-09T01:02:10.684 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000039082 2024-05-09T01:02:10.684 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:10.684 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:02:10.684 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:10.684 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:10.684 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:10.685 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:10.685 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 607568, 2024-05-09T01:02:10.685 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 191984 2024-05-09T01:02:10.685 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:10.685 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T01:02:10.685 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 1063, 2024-05-09T01:02:10.685 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 1296, 2024-05-09T01:02:10.685 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 356992, 2024-05-09T01:02:10.685 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 817380, 2024-05-09T01:02:10.685 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 4, 2024-05-09T01:02:10.685 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 3, 2024-05-09T01:02:10.685 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.156008610, 2024-05-09T01:02:10.685 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.049148927, 2024-05-09T01:02:10.686 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.041417887, 2024-05-09T01:02:10.686 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.051566285, 2024-05-09T01:02:10.686 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:02:10.686 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 1294, 2024-05-09T01:02:10.686 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.072212143, 2024-05-09T01:02:10.686 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000055805 2024-05-09T01:02:10.686 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:10.686 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:02:10.686 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:10.686 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:10.686 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:10.686 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:10.686 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-09T01:02:10.687 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 479664 2024-05-09T01:02:10.687 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:10.687 INFO:teuthology.orchestra.run.smithi131.stdout: "cct": { 2024-05-09T01:02:10.687 INFO:teuthology.orchestra.run.smithi131.stdout: "total_workers": 1, 2024-05-09T01:02:10.687 INFO:teuthology.orchestra.run.smithi131.stdout: "unhealthy_workers": 0 2024-05-09T01:02:10.687 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:10.687 INFO:teuthology.orchestra.run.smithi131.stdout: "finisher-MDSRank": { 2024-05-09T01:02:10.687 INFO:teuthology.orchestra.run.smithi131.stdout: "queue_len": 0, 2024-05-09T01:02:10.687 INFO:teuthology.orchestra.run.smithi131.stdout: "complete_latency": { 2024-05-09T01:02:10.687 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 5172, 2024-05-09T01:02:10.687 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.184843207, 2024-05-09T01:02:10.687 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000035739 2024-05-09T01:02:10.687 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-09T01:02:10.688 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:10.688 INFO:teuthology.orchestra.run.smithi131.stdout: "finisher-PurgeQueue": { 2024-05-09T01:02:10.688 INFO:teuthology.orchestra.run.smithi131.stdout: "queue_len": 0, 2024-05-09T01:02:10.688 INFO:teuthology.orchestra.run.smithi131.stdout: "complete_latency": { 2024-05-09T01:02:10.688 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:10.688 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:10.688 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:10.688 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-09T01:02:10.688 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:10.688 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": { 2024-05-09T01:02:10.689 INFO:teuthology.orchestra.run.smithi131.stdout: "request": 0, 2024-05-09T01:02:10.689 INFO:teuthology.orchestra.run.smithi131.stdout: "reply": 0, 2024-05-09T01:02:10.689 INFO:teuthology.orchestra.run.smithi131.stdout: "reply_latency": { 2024-05-09T01:02:10.689 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:10.689 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:10.689 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:10.689 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:10.689 INFO:teuthology.orchestra.run.smithi131.stdout: "slow_reply": 0, 2024-05-09T01:02:10.689 INFO:teuthology.orchestra.run.smithi131.stdout: "forward": 0, 2024-05-09T01:02:10.689 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_fetch_complete": 0, 2024-05-09T01:02:10.689 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_fetch_keys": 0, 2024-05-09T01:02:10.689 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_commit": 0, 2024-05-09T01:02:10.689 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_split": 0, 2024-05-09T01:02:10.690 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_merge": 0, 2024-05-09T01:02:10.690 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes": 0, 2024-05-09T01:02:10.690 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_top": 0, 2024-05-09T01:02:10.690 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_bottom": 0, 2024-05-09T01:02:10.690 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_pin_tail": 0, 2024-05-09T01:02:10.690 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_pinned": 0, 2024-05-09T01:02:10.690 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_expired": 0, 2024-05-09T01:02:10.690 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_with_caps": 0, 2024-05-09T01:02:10.690 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": 0, 2024-05-09T01:02:10.690 INFO:teuthology.orchestra.run.smithi131.stdout: "subtrees": 1, 2024-05-09T01:02:10.690 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse": 0, 2024-05-09T01:02:10.690 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_hit": 0, 2024-05-09T01:02:10.690 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_forward": 0, 2024-05-09T01:02:10.691 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_discover": 0, 2024-05-09T01:02:10.691 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_dir_fetch": 0, 2024-05-09T01:02:10.691 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_remote_ino": 0, 2024-05-09T01:02:10.691 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_lock": 0, 2024-05-09T01:02:10.691 INFO:teuthology.orchestra.run.smithi131.stdout: "load_cent": 0, 2024-05-09T01:02:10.691 INFO:teuthology.orchestra.run.smithi131.stdout: "q": 0, 2024-05-09T01:02:10.691 INFO:teuthology.orchestra.run.smithi131.stdout: "exported": 0, 2024-05-09T01:02:10.691 INFO:teuthology.orchestra.run.smithi131.stdout: "exported_inodes": 0, 2024-05-09T01:02:10.691 INFO:teuthology.orchestra.run.smithi131.stdout: "imported": 0, 2024-05-09T01:02:10.691 INFO:teuthology.orchestra.run.smithi131.stdout: "imported_inodes": 0, 2024-05-09T01:02:10.691 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_dir_fetch": 0, 2024-05-09T01:02:10.691 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_backtrace_fetch": 0, 2024-05-09T01:02:10.691 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_peer_discover": 0, 2024-05-09T01:02:10.691 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rfiles": 0, 2024-05-09T01:02:10.692 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rbytes": 0, 2024-05-09T01:02:10.692 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rsnaps": 0, 2024-05-09T01:02:10.692 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_backtrace_fetch": 0, 2024-05-09T01:02:10.692 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_set_tag": 0, 2024-05-09T01:02:10.692 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T01:02:10.692 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_inotable_repaired": 0, 2024-05-09T01:02:10.692 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dir_inodes": 0, 2024-05-09T01:02:10.692 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T01:02:10.692 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T01:02:10.692 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_file_inodes": 0, 2024-05-09T01:02:10.692 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_inode_file_caps": 0, 2024-05-09T01:02:10.692 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_revoke": 0, 2024-05-09T01:02:10.692 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_grant": 0, 2024-05-09T01:02:10.692 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T01:02:10.693 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T01:02:10.693 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T01:02:10.693 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_caps": 0, 2024-05-09T01:02:10.693 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_caps_dirty": 0, 2024-05-09T01:02:10.693 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_cap_release": 0, 2024-05-09T01:02:10.693 INFO:teuthology.orchestra.run.smithi131.stdout: "process_request_cap_release": 0 2024-05-09T01:02:10.693 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:10.693 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_cache": { 2024-05-09T01:02:10.693 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_update": 0, 2024-05-09T01:02:10.693 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_update_receipt": 0, 2024-05-09T01:02:10.693 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_try_discover": 0, 2024-05-09T01:02:10.693 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_send_discover": 0, 2024-05-09T01:02:10.693 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_handle_discover": 0, 2024-05-09T01:02:10.694 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays": 0, 2024-05-09T01:02:10.694 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays_delayed": 0, 2024-05-09T01:02:10.694 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays_enqueuing": 0, 2024-05-09T01:02:10.694 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_created": 0, 2024-05-09T01:02:10.694 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_enqueued": 0, 2024-05-09T01:02:10.694 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_reintegrated": 0, 2024-05-09T01:02:10.694 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_migrated": 0, 2024-05-09T01:02:10.694 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_processing": 0, 2024-05-09T01:02:10.694 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_enqueued": 0, 2024-05-09T01:02:10.694 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_prioritized": 0, 2024-05-09T01:02:10.694 INFO:teuthology.orchestra.run.smithi131.stdout: "recovery_started": 0, 2024-05-09T01:02:10.694 INFO:teuthology.orchestra.run.smithi131.stdout: "recovery_completed": 0, 2024-05-09T01:02:10.694 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_quiesce_path": 0, 2024-05-09T01:02:10.695 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_quiesce_inode": 0, 2024-05-09T01:02:10.695 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T01:02:10.695 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_exportdir": 0, 2024-05-09T01:02:10.695 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_flush": 0, 2024-05-09T01:02:10.695 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_fragmentdir": 0, 2024-05-09T01:02:10.695 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_fragstats": 0, 2024-05-09T01:02:10.695 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_inodestats": 0 2024-05-09T01:02:10.695 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:10.695 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_log": { 2024-05-09T01:02:10.695 INFO:teuthology.orchestra.run.smithi131.stdout: "evlrg": 0, 2024-05-09T01:02:10.695 INFO:teuthology.orchestra.run.smithi131.stdout: "evadd": 0, 2024-05-09T01:02:10.695 INFO:teuthology.orchestra.run.smithi131.stdout: "evex": 0, 2024-05-09T01:02:10.695 INFO:teuthology.orchestra.run.smithi131.stdout: "evtrm": 0, 2024-05-09T01:02:10.695 INFO:teuthology.orchestra.run.smithi131.stdout: "ev": 5, 2024-05-09T01:02:10.696 INFO:teuthology.orchestra.run.smithi131.stdout: "evexg": 0, 2024-05-09T01:02:10.696 INFO:teuthology.orchestra.run.smithi131.stdout: "evexd": 0, 2024-05-09T01:02:10.696 INFO:teuthology.orchestra.run.smithi131.stdout: "segadd": 0, 2024-05-09T01:02:10.696 INFO:teuthology.orchestra.run.smithi131.stdout: "segex": 0, 2024-05-09T01:02:10.696 INFO:teuthology.orchestra.run.smithi131.stdout: "segtrm": 0, 2024-05-09T01:02:10.696 INFO:teuthology.orchestra.run.smithi131.stdout: "seg": 2, 2024-05-09T01:02:10.696 INFO:teuthology.orchestra.run.smithi131.stdout: "segmjr": 2, 2024-05-09T01:02:10.696 INFO:teuthology.orchestra.run.smithi131.stdout: "segexg": 0, 2024-05-09T01:02:10.696 INFO:teuthology.orchestra.run.smithi131.stdout: "segexd": 0, 2024-05-09T01:02:10.696 INFO:teuthology.orchestra.run.smithi131.stdout: "expos": 4194304, 2024-05-09T01:02:10.696 INFO:teuthology.orchestra.run.smithi131.stdout: "wrpos": 4195732, 2024-05-09T01:02:10.696 INFO:teuthology.orchestra.run.smithi131.stdout: "rdpos": 4195424, 2024-05-09T01:02:10.696 INFO:teuthology.orchestra.run.smithi131.stdout: "jlat": { 2024-05-09T01:02:10.697 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:10.697 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:10.697 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:10.697 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:10.697 INFO:teuthology.orchestra.run.smithi131.stdout: "replayed": 5 2024-05-09T01:02:10.697 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:10.697 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_mem": { 2024-05-09T01:02:10.697 INFO:teuthology.orchestra.run.smithi131.stdout: "ino": 3, 2024-05-09T01:02:10.697 INFO:teuthology.orchestra.run.smithi131.stdout: "ino+": 3, 2024-05-09T01:02:10.697 INFO:teuthology.orchestra.run.smithi131.stdout: "ino-": 0, 2024-05-09T01:02:10.697 INFO:teuthology.orchestra.run.smithi131.stdout: "dir": 1, 2024-05-09T01:02:10.697 INFO:teuthology.orchestra.run.smithi131.stdout: "dir+": 1, 2024-05-09T01:02:10.697 INFO:teuthology.orchestra.run.smithi131.stdout: "dir-": 0, 2024-05-09T01:02:10.697 INFO:teuthology.orchestra.run.smithi131.stdout: "dn": 0, 2024-05-09T01:02:10.697 INFO:teuthology.orchestra.run.smithi131.stdout: "dn+": 0, 2024-05-09T01:02:10.698 INFO:teuthology.orchestra.run.smithi131.stdout: "dn-": 0, 2024-05-09T01:02:10.698 INFO:teuthology.orchestra.run.smithi131.stdout: "cap": 0, 2024-05-09T01:02:10.698 INFO:teuthology.orchestra.run.smithi131.stdout: "cap+": 0, 2024-05-09T01:02:10.698 INFO:teuthology.orchestra.run.smithi131.stdout: "cap-": 0, 2024-05-09T01:02:10.698 INFO:teuthology.orchestra.run.smithi131.stdout: "rss": 118908, 2024-05-09T01:02:10.698 INFO:teuthology.orchestra.run.smithi131.stdout: "heap": 215440 2024-05-09T01:02:10.698 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:10.698 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_server": { 2024-05-09T01:02:10.698 INFO:teuthology.orchestra.run.smithi131.stdout: "dispatch_client_request": 0, 2024-05-09T01:02:10.698 INFO:teuthology.orchestra.run.smithi131.stdout: "dispatch_server_request": 0, 2024-05-09T01:02:10.698 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_request": 0, 2024-05-09T01:02:10.698 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_session": 0, 2024-05-09T01:02:10.698 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_peer_request": 0, 2024-05-09T01:02:10.698 INFO:teuthology.orchestra.run.smithi131.stdout: "req_create_latency": { 2024-05-09T01:02:10.699 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:10.699 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:10.699 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:10.699 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:10.699 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getattr_latency": { 2024-05-09T01:02:10.699 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:10.699 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:10.699 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:10.699 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:10.699 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getfilelock_latency": { 2024-05-09T01:02:10.699 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:10.699 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:10.699 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:10.699 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:10.700 INFO:teuthology.orchestra.run.smithi131.stdout: "req_link_latency": { 2024-05-09T01:02:10.700 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:10.700 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:10.700 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:10.700 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:10.700 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookup_latency": { 2024-05-09T01:02:10.700 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:10.700 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:10.700 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:10.700 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:10.700 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookuphash_latency": { 2024-05-09T01:02:10.700 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:10.700 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:10.700 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:10.701 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:10.701 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupino_latency": { 2024-05-09T01:02:10.701 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:10.701 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:10.701 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:10.701 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:10.701 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupname_latency": { 2024-05-09T01:02:10.701 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:10.701 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:10.701 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:10.701 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:10.701 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupparent_latency": { 2024-05-09T01:02:10.701 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:10.702 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:10.702 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:10.702 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:10.702 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupsnap_latency": { 2024-05-09T01:02:10.702 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:10.702 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:10.702 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:10.702 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:10.702 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lssnap_latency": { 2024-05-09T01:02:10.702 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:10.702 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:10.702 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:10.702 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:10.703 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mkdir_latency": { 2024-05-09T01:02:10.703 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:10.703 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:10.703 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:10.704 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:10.704 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mknod_latency": { 2024-05-09T01:02:10.704 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:10.704 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:10.704 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:10.704 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:10.704 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mksnap_latency": { 2024-05-09T01:02:10.704 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:10.704 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:10.704 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:10.704 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:10.704 INFO:teuthology.orchestra.run.smithi131.stdout: "req_open_latency": { 2024-05-09T01:02:10.704 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:10.705 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:10.705 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:10.705 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:10.705 INFO:teuthology.orchestra.run.smithi131.stdout: "req_readdir_latency": { 2024-05-09T01:02:10.705 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:10.705 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:10.705 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:10.705 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:10.705 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rename_latency": { 2024-05-09T01:02:10.705 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:10.705 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:10.705 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:10.705 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:10.705 INFO:teuthology.orchestra.run.smithi131.stdout: "req_renamesnap_latency": { 2024-05-09T01:02:10.706 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:10.706 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:10.706 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:10.706 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:10.706 INFO:teuthology.orchestra.run.smithi131.stdout: "req_snapdiff_latency": { 2024-05-09T01:02:10.706 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:10.706 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:10.706 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:10.706 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:10.706 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmdir_latency": { 2024-05-09T01:02:10.706 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:10.706 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:10.706 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:10.707 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:10.707 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmsnap_latency": { 2024-05-09T01:02:10.707 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:10.707 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:10.707 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:10.707 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:10.707 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmxattr_latency": { 2024-05-09T01:02:10.707 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:10.707 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:10.707 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:10.707 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:10.707 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setattr_latency": { 2024-05-09T01:02:10.707 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:10.708 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:10.708 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:10.708 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:10.708 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setdirlayout_latency": { 2024-05-09T01:02:10.708 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:10.708 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:10.708 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:10.708 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:10.708 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setfilelock_latency": { 2024-05-09T01:02:10.708 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:10.708 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:10.708 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:10.708 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:10.709 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setlayout_latency": { 2024-05-09T01:02:10.709 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:10.709 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:10.709 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:10.709 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:10.709 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setxattr_latency": { 2024-05-09T01:02:10.709 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:10.709 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:10.709 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:10.709 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:10.709 INFO:teuthology.orchestra.run.smithi131.stdout: "req_symlink_latency": { 2024-05-09T01:02:10.709 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:10.709 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:10.710 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:10.710 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:10.710 INFO:teuthology.orchestra.run.smithi131.stdout: "req_unlink_latency": { 2024-05-09T01:02:10.710 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:10.710 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:10.710 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:10.710 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:10.710 INFO:teuthology.orchestra.run.smithi131.stdout: "cap_revoke_eviction": 0, 2024-05-09T01:02:10.710 INFO:teuthology.orchestra.run.smithi131.stdout: "cap_acquisition_throttle": 0, 2024-05-09T01:02:10.710 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getvxattr_latency": { 2024-05-09T01:02:10.710 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:10.710 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:10.710 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:10.711 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-09T01:02:10.711 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:10.711 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_sessions": { 2024-05-09T01:02:10.711 INFO:teuthology.orchestra.run.smithi131.stdout: "session_count": 1, 2024-05-09T01:02:10.711 INFO:teuthology.orchestra.run.smithi131.stdout: "session_add": 2, 2024-05-09T01:02:10.711 INFO:teuthology.orchestra.run.smithi131.stdout: "session_remove": 1, 2024-05-09T01:02:10.711 INFO:teuthology.orchestra.run.smithi131.stdout: "sessions_open": 1, 2024-05-09T01:02:10.711 INFO:teuthology.orchestra.run.smithi131.stdout: "sessions_stale": 0, 2024-05-09T01:02:10.711 INFO:teuthology.orchestra.run.smithi131.stdout: "total_load": 0, 2024-05-09T01:02:10.711 INFO:teuthology.orchestra.run.smithi131.stdout: "average_load": 0, 2024-05-09T01:02:10.711 INFO:teuthology.orchestra.run.smithi131.stdout: "avg_session_uptime": 2379, 2024-05-09T01:02:10.711 INFO:teuthology.orchestra.run.smithi131.stdout: "mdthresh_evicted": 0 2024-05-09T01:02:10.711 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:10.712 INFO:teuthology.orchestra.run.smithi131.stdout: "mempool": { 2024-05-09T01:02:10.712 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_bytes": 0, 2024-05-09T01:02:10.712 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_items": 0, 2024-05-09T01:02:10.712 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T01:02:10.712 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_items": 0, 2024-05-09T01:02:10.712 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T01:02:10.712 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_items": 0, 2024-05-09T01:02:10.712 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T01:02:10.712 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T01:02:10.712 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T01:02:10.712 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T01:02:10.712 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T01:02:10.712 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_items": 0, 2024-05-09T01:02:10.712 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T01:02:10.713 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T01:02:10.713 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_bytes": 0, 2024-05-09T01:02:10.713 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_items": 0, 2024-05-09T01:02:10.713 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_bytes": 0, 2024-05-09T01:02:10.713 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_items": 0, 2024-05-09T01:02:10.713 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T01:02:10.713 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T01:02:10.713 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T01:02:10.713 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T01:02:10.713 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T01:02:10.713 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_items": 0, 2024-05-09T01:02:10.713 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_bytes": 0, 2024-05-09T01:02:10.713 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_items": 0, 2024-05-09T01:02:10.713 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T01:02:10.714 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T01:02:10.714 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_bytes": 0, 2024-05-09T01:02:10.714 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_items": 0, 2024-05-09T01:02:10.714 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_bytes": 0, 2024-05-09T01:02:10.714 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_items": 0, 2024-05-09T01:02:10.714 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T01:02:10.714 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_items": 0, 2024-05-09T01:02:10.714 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T01:02:10.714 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_items": 0, 2024-05-09T01:02:10.714 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_bytes": 190167, 2024-05-09T01:02:10.714 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_items": 52, 2024-05-09T01:02:10.714 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_bytes": 0, 2024-05-09T01:02:10.714 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_items": 0, 2024-05-09T01:02:10.715 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_bytes": 0, 2024-05-09T01:02:10.715 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_items": 0, 2024-05-09T01:02:10.715 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_bytes": 0, 2024-05-09T01:02:10.715 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_items": 0, 2024-05-09T01:02:10.715 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_bytes": 0, 2024-05-09T01:02:10.715 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_items": 0, 2024-05-09T01:02:10.715 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_bytes": 7736, 2024-05-09T01:02:10.715 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_items": 164, 2024-05-09T01:02:10.715 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T01:02:10.715 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_items": 0, 2024-05-09T01:02:10.715 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_bytes": 0, 2024-05-09T01:02:10.715 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_items": 0, 2024-05-09T01:02:10.715 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_bytes": 8512, 2024-05-09T01:02:10.716 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_items": 13, 2024-05-09T01:02:10.716 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_bytes": 0, 2024-05-09T01:02:10.716 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_items": 0, 2024-05-09T01:02:10.716 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_bytes": 0, 2024-05-09T01:02:10.716 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_items": 0 2024-05-09T01:02:10.716 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:10.716 INFO:teuthology.orchestra.run.smithi131.stdout: "objecter": { 2024-05-09T01:02:10.716 INFO:teuthology.orchestra.run.smithi131.stdout: "op_active": 0, 2024-05-09T01:02:10.716 INFO:teuthology.orchestra.run.smithi131.stdout: "op_laggy": 0, 2024-05-09T01:02:10.716 INFO:teuthology.orchestra.run.smithi131.stdout: "op_send": 3086, 2024-05-09T01:02:10.716 INFO:teuthology.orchestra.run.smithi131.stdout: "op_send_bytes": 21, 2024-05-09T01:02:10.716 INFO:teuthology.orchestra.run.smithi131.stdout: "op_resend": 3, 2024-05-09T01:02:10.716 INFO:teuthology.orchestra.run.smithi131.stdout: "op_reply": 3086, 2024-05-09T01:02:10.716 INFO:teuthology.orchestra.run.smithi131.stdout: "op_latency": { 2024-05-09T01:02:10.716 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 3086, 2024-05-09T01:02:10.717 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 26.287453036, 2024-05-09T01:02:10.717 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.008518293 2024-05-09T01:02:10.717 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:10.717 INFO:teuthology.orchestra.run.smithi131.stdout: "op_inflight": 0, 2024-05-09T01:02:10.717 INFO:teuthology.orchestra.run.smithi131.stdout: "oplen_avg": { 2024-05-09T01:02:10.717 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 3086, 2024-05-09T01:02:10.717 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 3087 2024-05-09T01:02:10.717 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:10.717 INFO:teuthology.orchestra.run.smithi131.stdout: "op": 3086, 2024-05-09T01:02:10.717 INFO:teuthology.orchestra.run.smithi131.stdout: "op_r": 3086, 2024-05-09T01:02:10.717 INFO:teuthology.orchestra.run.smithi131.stdout: "op_w": 0, 2024-05-09T01:02:10.717 INFO:teuthology.orchestra.run.smithi131.stdout: "op_rmw": 0, 2024-05-09T01:02:10.717 INFO:teuthology.orchestra.run.smithi131.stdout: "op_pg": 0, 2024-05-09T01:02:10.718 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_stat": 2015, 2024-05-09T01:02:10.718 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_create": 0, 2024-05-09T01:02:10.718 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_read": 1069, 2024-05-09T01:02:10.718 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_write": 0, 2024-05-09T01:02:10.718 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_writefull": 0, 2024-05-09T01:02:10.718 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_writesame": 0, 2024-05-09T01:02:10.718 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_append": 0, 2024-05-09T01:02:10.718 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_zero": 0, 2024-05-09T01:02:10.718 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_truncate": 0, 2024-05-09T01:02:10.718 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_delete": 0, 2024-05-09T01:02:10.718 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_mapext": 0, 2024-05-09T01:02:10.718 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_sparse_read": 0, 2024-05-09T01:02:10.718 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_clonerange": 0, 2024-05-09T01:02:10.718 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_getxattr": 1, 2024-05-09T01:02:10.719 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_setxattr": 0, 2024-05-09T01:02:10.719 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_cmpxattr": 0, 2024-05-09T01:02:10.719 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_rmxattr": 0, 2024-05-09T01:02:10.719 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_resetxattrs": 0, 2024-05-09T01:02:10.719 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_call": 0, 2024-05-09T01:02:10.719 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_watch": 0, 2024-05-09T01:02:10.719 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_notify": 0, 2024-05-09T01:02:10.719 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T01:02:10.719 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_pgls": 0, 2024-05-09T01:02:10.719 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_pgls_filter": 0, 2024-05-09T01:02:10.719 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_other": 0, 2024-05-09T01:02:10.719 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_active": 0, 2024-05-09T01:02:10.719 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_send": 0, 2024-05-09T01:02:10.720 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_resend": 0, 2024-05-09T01:02:10.720 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_ping": 0, 2024-05-09T01:02:10.720 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_active": 0, 2024-05-09T01:02:10.720 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_send": 0, 2024-05-09T01:02:10.720 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_resend": 0, 2024-05-09T01:02:10.720 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_active": 0, 2024-05-09T01:02:10.720 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_send": 0, 2024-05-09T01:02:10.720 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_resend": 0, 2024-05-09T01:02:10.720 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_active": 0, 2024-05-09T01:02:10.720 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_send": 0, 2024-05-09T01:02:10.720 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_resend": 0, 2024-05-09T01:02:10.720 INFO:teuthology.orchestra.run.smithi131.stdout: "command_active": 0, 2024-05-09T01:02:10.720 INFO:teuthology.orchestra.run.smithi131.stdout: "command_send": 0, 2024-05-09T01:02:10.720 INFO:teuthology.orchestra.run.smithi131.stdout: "command_resend": 0, 2024-05-09T01:02:10.720 INFO:teuthology.orchestra.run.smithi131.stdout: "map_epoch": 82, 2024-05-09T01:02:10.721 INFO:teuthology.orchestra.run.smithi131.stdout: "map_full": 0, 2024-05-09T01:02:10.721 INFO:teuthology.orchestra.run.smithi131.stdout: "map_inc": 4, 2024-05-09T01:02:10.721 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_sessions": 5, 2024-05-09T01:02:10.721 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_session_open": 6, 2024-05-09T01:02:10.721 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_session_close": 1, 2024-05-09T01:02:10.721 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_laggy": 0, 2024-05-09T01:02:10.721 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_wr": 0, 2024-05-09T01:02:10.721 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_rd": 2, 2024-05-09T01:02:10.721 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_del": 0 2024-05-09T01:02:10.721 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:10.721 INFO:teuthology.orchestra.run.smithi131.stdout: "oft": { 2024-05-09T01:02:10.721 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_objs": 0, 2024-05-09T01:02:10.721 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_kv_pairs": 0, 2024-05-09T01:02:10.722 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_updates": 0, 2024-05-09T01:02:10.722 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_removes": 0 2024-05-09T01:02:10.722 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:10.722 INFO:teuthology.orchestra.run.smithi131.stdout: "osd-slow-ops": { 2024-05-09T01:02:10.722 INFO:teuthology.orchestra.run.smithi131.stdout: "slow_ops_count": 0 2024-05-09T01:02:10.722 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:10.722 INFO:teuthology.orchestra.run.smithi131.stdout: "purge_queue": { 2024-05-09T01:02:10.722 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_ops": 0, 2024-05-09T01:02:10.722 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_ops_high_water": 0, 2024-05-09T01:02:10.722 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing": 0, 2024-05-09T01:02:10.722 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_high_water": 0, 2024-05-09T01:02:10.722 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executed_ops": 0, 2024-05-09T01:02:10.723 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executed": 0, 2024-05-09T01:02:10.723 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_item_in_journal": 0 2024-05-09T01:02:10.723 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:10.723 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T01:02:10.723 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-09T01:02:10.723 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 104857600, 2024-05-09T01:02:10.723 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-09T01:02:10.723 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 3414, 2024-05-09T01:02:10.723 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 1154586, 2024-05-09T01:02:10.723 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-09T01:02:10.723 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 3414, 2024-05-09T01:02:10.723 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-05-09T01:02:10.723 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-05-09T01:02:10.723 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 3414, 2024-05-09T01:02:10.723 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 1154586, 2024-05-09T01:02:10.724 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-09T01:02:10.724 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:10.724 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:10.724 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:10.724 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-09T01:02:10.724 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:10.724 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-objecter_bytes": { 2024-05-09T01:02:10.724 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-09T01:02:10.724 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 104857600, 2024-05-09T01:02:10.724 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-09T01:02:10.724 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-05-09T01:02:10.724 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-05-09T01:02:10.724 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-09T01:02:10.725 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-05-09T01:02:10.725 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 3086, 2024-05-09T01:02:10.725 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 1433, 2024-05-09T01:02:10.725 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 3086, 2024-05-09T01:02:10.725 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 1433, 2024-05-09T01:02:10.725 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-09T01:02:10.725 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:10.725 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:10.725 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:10.725 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-09T01:02:10.725 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:10.725 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-objecter_ops": { 2024-05-09T01:02:10.725 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-09T01:02:10.726 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 1024, 2024-05-09T01:02:10.726 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-09T01:02:10.726 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-05-09T01:02:10.726 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-05-09T01:02:10.726 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-09T01:02:10.726 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-05-09T01:02:10.726 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 3086, 2024-05-09T01:02:10.726 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 3086, 2024-05-09T01:02:10.726 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 3086, 2024-05-09T01:02:10.726 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 3086, 2024-05-09T01:02:10.726 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-09T01:02:10.726 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:10.726 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:10.727 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:10.727 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-09T01:02:10.727 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:10.727 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-write_buf_throttle": { 2024-05-09T01:02:10.727 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-09T01:02:10.727 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 3758096384, 2024-05-09T01:02:10.727 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-09T01:02:10.727 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-05-09T01:02:10.727 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-05-09T01:02:10.727 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-09T01:02:10.727 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-05-09T01:02:10.727 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-05-09T01:02:10.727 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-05-09T01:02:10.728 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 0, 2024-05-09T01:02:10.728 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 0, 2024-05-09T01:02:10.728 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-09T01:02:10.728 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:10.728 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:10.728 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:10.728 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-09T01:02:10.728 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-09T01:02:10.728 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-write_buf_throttle-0x56461c086680": { 2024-05-09T01:02:10.728 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-09T01:02:10.728 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 3758096384, 2024-05-09T01:02:10.728 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-09T01:02:10.728 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-05-09T01:02:10.728 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-05-09T01:02:10.729 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-09T01:02:10.729 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-05-09T01:02:10.729 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-05-09T01:02:10.729 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-05-09T01:02:10.729 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 0, 2024-05-09T01:02:10.729 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 0, 2024-05-09T01:02:10.729 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-09T01:02:10.729 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-09T01:02:10.729 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-09T01:02:10.729 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-09T01:02:10.729 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-09T01:02:10.729 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-09T01:02:10.729 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-09T01:02:11.429 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-05-09T01:02:11.429 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-05-09T01:02:11.429 DEBUG:tasks.check_counter:Getting stats from c 2024-05-09T01:02:11.429 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph daemon mds.c perf dump 2024-05-09T01:02:11.765 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T01:02:11.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:11 smithi131 ceph-mon[34547]: pgmap v846: 97 pgs: 97 active+clean; 6.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 30 MiB/s wr, 304 op/s 2024-05-09T01:02:11.888 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:11 smithi157 ceph-mon[34294]: pgmap v846: 97 pgs: 97 active+clean; 6.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 30 MiB/s wr, 304 op/s 2024-05-09T01:02:12.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:11 smithi005 ceph-mon[30365]: pgmap v846: 97 pgs: 97 active+clean; 6.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 30 MiB/s wr, 304 op/s 2024-05-09T01:02:13.797 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:13 smithi157 ceph-mon[34294]: pgmap v847: 97 pgs: 97 active+clean; 6.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 39 MiB/s wr, 394 op/s 2024-05-09T01:02:14.040 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:13 smithi005 ceph-mon[30365]: pgmap v847: 97 pgs: 97 active+clean; 6.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 39 MiB/s wr, 394 op/s 2024-05-09T01:02:14.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:13 smithi131 ceph-mon[34547]: pgmap v847: 97 pgs: 97 active+clean; 6.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 39 MiB/s wr, 394 op/s 2024-05-09T01:02:14.333 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-05-09T01:02:14.333 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T01:02:14.333 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 2676, 2024-05-09T01:02:14.333 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 1841, 2024-05-09T01:02:14.333 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 1272882, 2024-05-09T01:02:14.333 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 748942, 2024-05-09T01:02:14.333 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 12, 2024-05-09T01:02:14.333 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 5, 2024-05-09T01:02:14.334 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.237563767, 2024-05-09T01:02:14.334 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.092497140, 2024-05-09T01:02:14.334 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.163944032, 2024-05-09T01:02:14.334 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.018387516, 2024-05-09T01:02:14.334 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:02:14.334 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1841, 2024-05-09T01:02:14.334 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.141832410, 2024-05-09T01:02:14.334 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000077040 2024-05-09T01:02:14.334 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:14.334 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:02:14.335 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 3416, 2024-05-09T01:02:14.335 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000952085, 2024-05-09T01:02:14.335 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000278 2024-05-09T01:02:14.335 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:14.335 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 636752, 2024-05-09T01:02:14.335 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 200224 2024-05-09T01:02:14.335 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:14.335 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T01:02:14.335 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 128, 2024-05-09T01:02:14.335 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 364, 2024-05-09T01:02:14.336 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 64978, 2024-05-09T01:02:14.336 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 761125, 2024-05-09T01:02:14.336 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 24, 2024-05-09T01:02:14.336 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 4, 2024-05-09T01:02:14.336 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.038297680, 2024-05-09T01:02:14.336 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.016084335, 2024-05-09T01:02:14.336 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.019063975, 2024-05-09T01:02:14.336 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.000361504, 2024-05-09T01:02:14.336 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:02:14.336 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 362, 2024-05-09T01:02:14.336 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.044359040, 2024-05-09T01:02:14.336 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000122538 2024-05-09T01:02:14.336 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:14.337 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:02:14.337 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 207, 2024-05-09T01:02:14.337 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000071881, 2024-05-09T01:02:14.337 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000347 2024-05-09T01:02:14.337 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:14.337 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 1408, 2024-05-09T01:02:14.337 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 571920 2024-05-09T01:02:14.337 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:14.337 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T01:02:14.337 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 138, 2024-05-09T01:02:14.337 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 140, 2024-05-09T01:02:14.337 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 73745, 2024-05-09T01:02:14.337 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 245892, 2024-05-09T01:02:14.338 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 24, 2024-05-09T01:02:14.338 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 6, 2024-05-09T01:02:14.338 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.025301858, 2024-05-09T01:02:14.338 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.005755417, 2024-05-09T01:02:14.338 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.017701807, 2024-05-09T01:02:14.338 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.001610071, 2024-05-09T01:02:14.338 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:02:14.338 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 138, 2024-05-09T01:02:14.338 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.023330899, 2024-05-09T01:02:14.338 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000169064 2024-05-09T01:02:14.338 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:14.338 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:02:14.338 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 166, 2024-05-09T01:02:14.338 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000072047, 2024-05-09T01:02:14.339 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000434 2024-05-09T01:02:14.339 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:14.339 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T01:02:14.339 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T01:02:14.339 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:14.339 INFO:teuthology.orchestra.run.smithi157.stdout: "cct": { 2024-05-09T01:02:14.339 INFO:teuthology.orchestra.run.smithi157.stdout: "total_workers": 1, 2024-05-09T01:02:14.339 INFO:teuthology.orchestra.run.smithi157.stdout: "unhealthy_workers": 0 2024-05-09T01:02:14.339 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:14.339 INFO:teuthology.orchestra.run.smithi157.stdout: "finisher-MDSRank": { 2024-05-09T01:02:14.339 INFO:teuthology.orchestra.run.smithi157.stdout: "queue_len": 0, 2024-05-09T01:02:14.339 INFO:teuthology.orchestra.run.smithi157.stdout: "complete_latency": { 2024-05-09T01:02:14.339 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 85, 2024-05-09T01:02:14.340 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.006205899, 2024-05-09T01:02:14.340 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000073010 2024-05-09T01:02:14.340 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-09T01:02:14.340 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:14.340 INFO:teuthology.orchestra.run.smithi157.stdout: "finisher-PurgeQueue": { 2024-05-09T01:02:14.340 INFO:teuthology.orchestra.run.smithi157.stdout: "queue_len": 0, 2024-05-09T01:02:14.340 INFO:teuthology.orchestra.run.smithi157.stdout: "complete_latency": { 2024-05-09T01:02:14.340 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 17, 2024-05-09T01:02:14.340 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000630334, 2024-05-09T01:02:14.340 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000037078 2024-05-09T01:02:14.340 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-09T01:02:14.341 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:14.341 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": { 2024-05-09T01:02:14.341 INFO:teuthology.orchestra.run.smithi157.stdout: "request": 5, 2024-05-09T01:02:14.341 INFO:teuthology.orchestra.run.smithi157.stdout: "reply": 4, 2024-05-09T01:02:14.341 INFO:teuthology.orchestra.run.smithi157.stdout: "reply_latency": { 2024-05-09T01:02:14.341 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 4, 2024-05-09T01:02:14.341 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.005666836, 2024-05-09T01:02:14.341 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.001416709 2024-05-09T01:02:14.341 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:14.341 INFO:teuthology.orchestra.run.smithi157.stdout: "slow_reply": 0, 2024-05-09T01:02:14.341 INFO:teuthology.orchestra.run.smithi157.stdout: "forward": 1, 2024-05-09T01:02:14.341 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_fetch_complete": 0, 2024-05-09T01:02:14.341 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_fetch_keys": 0, 2024-05-09T01:02:14.341 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_commit": 11, 2024-05-09T01:02:14.342 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_split": 0, 2024-05-09T01:02:14.342 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_merge": 0, 2024-05-09T01:02:14.342 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes": 15, 2024-05-09T01:02:14.342 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_top": 0, 2024-05-09T01:02:14.342 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_bottom": 15, 2024-05-09T01:02:14.342 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_pin_tail": 0, 2024-05-09T01:02:14.342 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_pinned": 13, 2024-05-09T01:02:14.342 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_expired": 8, 2024-05-09T01:02:14.342 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_with_caps": 0, 2024-05-09T01:02:14.342 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": 0, 2024-05-09T01:02:14.342 INFO:teuthology.orchestra.run.smithi157.stdout: "subtrees": 3, 2024-05-09T01:02:14.342 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse": 12, 2024-05-09T01:02:14.342 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_hit": 6, 2024-05-09T01:02:14.343 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_forward": 1, 2024-05-09T01:02:14.343 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_discover": 1, 2024-05-09T01:02:14.343 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_dir_fetch": 0, 2024-05-09T01:02:14.343 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_remote_ino": 0, 2024-05-09T01:02:14.343 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_lock": 0, 2024-05-09T01:02:14.343 INFO:teuthology.orchestra.run.smithi157.stdout: "load_cent": 0, 2024-05-09T01:02:14.343 INFO:teuthology.orchestra.run.smithi157.stdout: "q": 0, 2024-05-09T01:02:14.343 INFO:teuthology.orchestra.run.smithi157.stdout: "exported": 1, 2024-05-09T01:02:14.343 INFO:teuthology.orchestra.run.smithi157.stdout: "exported_inodes": 0, 2024-05-09T01:02:14.343 INFO:teuthology.orchestra.run.smithi157.stdout: "imported": 1, 2024-05-09T01:02:14.343 INFO:teuthology.orchestra.run.smithi157.stdout: "imported_inodes": 2, 2024-05-09T01:02:14.343 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_dir_fetch": 0, 2024-05-09T01:02:14.343 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_backtrace_fetch": 14, 2024-05-09T01:02:14.344 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_peer_discover": 0, 2024-05-09T01:02:14.344 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rfiles": 21, 2024-05-09T01:02:14.344 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rbytes": 10738136400, 2024-05-09T01:02:14.344 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rsnaps": 0, 2024-05-09T01:02:14.344 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_backtrace_fetch": 14, 2024-05-09T01:02:14.344 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_set_tag": 0, 2024-05-09T01:02:14.344 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T01:02:14.344 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_inotable_repaired": 0, 2024-05-09T01:02:14.344 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dir_inodes": 0, 2024-05-09T01:02:14.344 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T01:02:14.344 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T01:02:14.344 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_file_inodes": 14, 2024-05-09T01:02:14.345 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_inode_file_caps": 0, 2024-05-09T01:02:14.345 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_revoke": 1, 2024-05-09T01:02:14.345 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_grant": 3, 2024-05-09T01:02:14.345 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T01:02:14.345 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T01:02:14.345 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T01:02:14.345 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_caps": 1, 2024-05-09T01:02:14.345 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_caps_dirty": 0, 2024-05-09T01:02:14.345 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_cap_release": 2, 2024-05-09T01:02:14.345 INFO:teuthology.orchestra.run.smithi157.stdout: "process_request_cap_release": 4 2024-05-09T01:02:14.345 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:14.345 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_cache": { 2024-05-09T01:02:14.345 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_update": 0, 2024-05-09T01:02:14.346 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_update_receipt": 0, 2024-05-09T01:02:14.346 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_try_discover": 0, 2024-05-09T01:02:14.346 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_send_discover": 2, 2024-05-09T01:02:14.346 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_handle_discover": 0, 2024-05-09T01:02:14.346 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays": 0, 2024-05-09T01:02:14.346 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays_delayed": 0, 2024-05-09T01:02:14.346 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays_enqueuing": 0, 2024-05-09T01:02:14.346 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_created": 2, 2024-05-09T01:02:14.346 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_enqueued": 2, 2024-05-09T01:02:14.346 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_reintegrated": 0, 2024-05-09T01:02:14.346 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_migrated": 0, 2024-05-09T01:02:14.346 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_processing": 0, 2024-05-09T01:02:14.346 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_enqueued": 0, 2024-05-09T01:02:14.346 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_prioritized": 0, 2024-05-09T01:02:14.347 INFO:teuthology.orchestra.run.smithi157.stdout: "recovery_started": 0, 2024-05-09T01:02:14.347 INFO:teuthology.orchestra.run.smithi157.stdout: "recovery_completed": 0, 2024-05-09T01:02:14.347 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_quiesce_path": 0, 2024-05-09T01:02:14.347 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_quiesce_inode": 0, 2024-05-09T01:02:14.347 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T01:02:14.347 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_exportdir": 1, 2024-05-09T01:02:14.347 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_flush": 0, 2024-05-09T01:02:14.347 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_fragmentdir": 0, 2024-05-09T01:02:14.347 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_fragstats": 0, 2024-05-09T01:02:14.347 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_inodestats": 0 2024-05-09T01:02:14.347 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:14.347 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_log": { 2024-05-09T01:02:14.347 INFO:teuthology.orchestra.run.smithi157.stdout: "evlrg": 0, 2024-05-09T01:02:14.347 INFO:teuthology.orchestra.run.smithi157.stdout: "evadd": 15, 2024-05-09T01:02:14.347 INFO:teuthology.orchestra.run.smithi157.stdout: "evex": 0, 2024-05-09T01:02:14.348 INFO:teuthology.orchestra.run.smithi157.stdout: "evtrm": 0, 2024-05-09T01:02:14.348 INFO:teuthology.orchestra.run.smithi157.stdout: "ev": 15, 2024-05-09T01:02:14.348 INFO:teuthology.orchestra.run.smithi157.stdout: "evexg": 0, 2024-05-09T01:02:14.348 INFO:teuthology.orchestra.run.smithi157.stdout: "evexd": 0, 2024-05-09T01:02:14.348 INFO:teuthology.orchestra.run.smithi157.stdout: "segadd": 2, 2024-05-09T01:02:14.348 INFO:teuthology.orchestra.run.smithi157.stdout: "segex": 0, 2024-05-09T01:02:14.348 INFO:teuthology.orchestra.run.smithi157.stdout: "segtrm": 0, 2024-05-09T01:02:14.348 INFO:teuthology.orchestra.run.smithi157.stdout: "seg": 2, 2024-05-09T01:02:14.348 INFO:teuthology.orchestra.run.smithi157.stdout: "segmjr": 2, 2024-05-09T01:02:14.348 INFO:teuthology.orchestra.run.smithi157.stdout: "segexg": 0, 2024-05-09T01:02:14.348 INFO:teuthology.orchestra.run.smithi157.stdout: "segexd": 0, 2024-05-09T01:02:14.348 INFO:teuthology.orchestra.run.smithi157.stdout: "expos": 4194304, 2024-05-09T01:02:14.348 INFO:teuthology.orchestra.run.smithi157.stdout: "wrpos": 4217445, 2024-05-09T01:02:14.348 INFO:teuthology.orchestra.run.smithi157.stdout: "rdpos": 0, 2024-05-09T01:02:14.348 INFO:teuthology.orchestra.run.smithi157.stdout: "jlat": { 2024-05-09T01:02:14.349 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 11, 2024-05-09T01:02:14.349 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.027549255, 2024-05-09T01:02:14.349 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.002504477 2024-05-09T01:02:14.349 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:14.349 INFO:teuthology.orchestra.run.smithi157.stdout: "replayed": 0 2024-05-09T01:02:14.349 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:14.349 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_mem": { 2024-05-09T01:02:14.349 INFO:teuthology.orchestra.run.smithi157.stdout: "ino": 19, 2024-05-09T01:02:14.349 INFO:teuthology.orchestra.run.smithi157.stdout: "ino+": 25, 2024-05-09T01:02:14.349 INFO:teuthology.orchestra.run.smithi157.stdout: "ino-": 6, 2024-05-09T01:02:14.349 INFO:teuthology.orchestra.run.smithi157.stdout: "dir": 18, 2024-05-09T01:02:14.349 INFO:teuthology.orchestra.run.smithi157.stdout: "dir+": 31, 2024-05-09T01:02:14.349 INFO:teuthology.orchestra.run.smithi157.stdout: "dir-": 13, 2024-05-09T01:02:14.350 INFO:teuthology.orchestra.run.smithi157.stdout: "dn": 15, 2024-05-09T01:02:14.350 INFO:teuthology.orchestra.run.smithi157.stdout: "dn+": 27, 2024-05-09T01:02:14.350 INFO:teuthology.orchestra.run.smithi157.stdout: "dn-": 12, 2024-05-09T01:02:14.350 INFO:teuthology.orchestra.run.smithi157.stdout: "cap": 0, 2024-05-09T01:02:14.350 INFO:teuthology.orchestra.run.smithi157.stdout: "cap+": 5, 2024-05-09T01:02:14.350 INFO:teuthology.orchestra.run.smithi157.stdout: "cap-": 5, 2024-05-09T01:02:14.350 INFO:teuthology.orchestra.run.smithi157.stdout: "rss": 122180, 2024-05-09T01:02:14.350 INFO:teuthology.orchestra.run.smithi157.stdout: "heap": 223632 2024-05-09T01:02:14.350 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:14.350 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_server": { 2024-05-09T01:02:14.350 INFO:teuthology.orchestra.run.smithi157.stdout: "dispatch_client_request": 9, 2024-05-09T01:02:14.350 INFO:teuthology.orchestra.run.smithi157.stdout: "dispatch_server_request": 1, 2024-05-09T01:02:14.350 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_request": 5, 2024-05-09T01:02:14.350 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_session": 15, 2024-05-09T01:02:14.351 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_peer_request": 5, 2024-05-09T01:02:14.351 INFO:teuthology.orchestra.run.smithi157.stdout: "req_create_latency": { 2024-05-09T01:02:14.351 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:14.351 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:14.351 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:14.351 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:14.351 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getattr_latency": { 2024-05-09T01:02:14.351 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:14.351 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:14.351 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:14.351 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:14.351 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getfilelock_latency": { 2024-05-09T01:02:14.352 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:14.352 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:14.352 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:14.352 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:14.352 INFO:teuthology.orchestra.run.smithi157.stdout: "req_link_latency": { 2024-05-09T01:02:14.352 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:14.352 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:14.352 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:14.352 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:14.352 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookup_latency": { 2024-05-09T01:02:14.352 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1, 2024-05-09T01:02:14.352 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000416954, 2024-05-09T01:02:14.352 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000416954 2024-05-09T01:02:14.352 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:14.352 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookuphash_latency": { 2024-05-09T01:02:14.353 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:14.353 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:14.353 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:14.353 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:14.353 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupino_latency": { 2024-05-09T01:02:14.353 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:14.353 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:14.353 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:14.353 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:14.353 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupname_latency": { 2024-05-09T01:02:14.353 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:14.353 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:14.353 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:14.353 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:14.353 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupparent_latency": { 2024-05-09T01:02:14.354 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:14.354 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:14.354 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:14.354 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:14.354 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupsnap_latency": { 2024-05-09T01:02:14.354 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:14.354 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:14.354 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:14.354 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:14.354 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lssnap_latency": { 2024-05-09T01:02:14.354 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:14.354 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:14.354 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:14.355 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:14.355 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mkdir_latency": { 2024-05-09T01:02:14.355 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:14.355 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:14.355 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:14.356 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:14.356 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mknod_latency": { 2024-05-09T01:02:14.356 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:14.356 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:14.356 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:14.356 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:14.356 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mksnap_latency": { 2024-05-09T01:02:14.356 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:14.356 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:14.356 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:14.356 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:14.356 INFO:teuthology.orchestra.run.smithi157.stdout: "req_open_latency": { 2024-05-09T01:02:14.356 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:14.356 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:14.357 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:14.357 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:14.357 INFO:teuthology.orchestra.run.smithi157.stdout: "req_readdir_latency": { 2024-05-09T01:02:14.357 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1, 2024-05-09T01:02:14.357 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000484275, 2024-05-09T01:02:14.357 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000484275 2024-05-09T01:02:14.357 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:14.357 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rename_latency": { 2024-05-09T01:02:14.357 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:14.357 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:14.357 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:14.357 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:14.357 INFO:teuthology.orchestra.run.smithi157.stdout: "req_renamesnap_latency": { 2024-05-09T01:02:14.358 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:14.358 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:14.358 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:14.358 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:14.358 INFO:teuthology.orchestra.run.smithi157.stdout: "req_snapdiff_latency": { 2024-05-09T01:02:14.358 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:14.358 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:14.358 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:14.358 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:14.358 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmdir_latency": { 2024-05-09T01:02:14.358 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:14.358 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:14.358 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:14.359 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:14.359 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmsnap_latency": { 2024-05-09T01:02:14.359 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:14.359 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:14.359 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:14.359 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:14.359 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmxattr_latency": { 2024-05-09T01:02:14.359 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:14.359 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:14.359 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:14.359 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:14.359 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setattr_latency": { 2024-05-09T01:02:14.359 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:14.359 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:14.360 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:14.360 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:14.360 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setdirlayout_latency": { 2024-05-09T01:02:14.360 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:14.360 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:14.360 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:14.360 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:14.360 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setfilelock_latency": { 2024-05-09T01:02:14.360 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:14.360 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:14.360 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:14.360 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:14.360 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setlayout_latency": { 2024-05-09T01:02:14.361 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:14.361 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:14.361 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:14.361 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:14.361 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setxattr_latency": { 2024-05-09T01:02:14.361 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:14.361 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:14.361 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:14.361 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:14.361 INFO:teuthology.orchestra.run.smithi157.stdout: "req_symlink_latency": { 2024-05-09T01:02:14.361 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:14.361 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:14.361 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:14.361 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:14.362 INFO:teuthology.orchestra.run.smithi157.stdout: "req_unlink_latency": { 2024-05-09T01:02:14.362 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 2, 2024-05-09T01:02:14.362 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.004765607, 2024-05-09T01:02:14.362 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.002382803 2024-05-09T01:02:14.362 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:14.362 INFO:teuthology.orchestra.run.smithi157.stdout: "cap_revoke_eviction": 0, 2024-05-09T01:02:14.362 INFO:teuthology.orchestra.run.smithi157.stdout: "cap_acquisition_throttle": 0, 2024-05-09T01:02:14.362 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getvxattr_latency": { 2024-05-09T01:02:14.362 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:14.362 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:14.362 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:14.362 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-09T01:02:14.362 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:14.362 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_sessions": { 2024-05-09T01:02:14.363 INFO:teuthology.orchestra.run.smithi157.stdout: "session_count": 1, 2024-05-09T01:02:14.363 INFO:teuthology.orchestra.run.smithi157.stdout: "session_add": 2, 2024-05-09T01:02:14.363 INFO:teuthology.orchestra.run.smithi157.stdout: "session_remove": 1, 2024-05-09T01:02:14.363 INFO:teuthology.orchestra.run.smithi157.stdout: "sessions_open": 1, 2024-05-09T01:02:14.363 INFO:teuthology.orchestra.run.smithi157.stdout: "sessions_stale": 0, 2024-05-09T01:02:14.363 INFO:teuthology.orchestra.run.smithi157.stdout: "total_load": 3, 2024-05-09T01:02:14.363 INFO:teuthology.orchestra.run.smithi157.stdout: "average_load": 3, 2024-05-09T01:02:14.363 INFO:teuthology.orchestra.run.smithi157.stdout: "avg_session_uptime": 2332, 2024-05-09T01:02:14.363 INFO:teuthology.orchestra.run.smithi157.stdout: "mdthresh_evicted": 0 2024-05-09T01:02:14.363 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:14.363 INFO:teuthology.orchestra.run.smithi157.stdout: "mempool": { 2024-05-09T01:02:14.363 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_bytes": 0, 2024-05-09T01:02:14.363 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_items": 0, 2024-05-09T01:02:14.363 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T01:02:14.364 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_items": 0, 2024-05-09T01:02:14.364 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T01:02:14.364 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_items": 0, 2024-05-09T01:02:14.364 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T01:02:14.364 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T01:02:14.364 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T01:02:14.364 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T01:02:14.364 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T01:02:14.364 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_items": 0, 2024-05-09T01:02:14.364 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T01:02:14.364 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T01:02:14.364 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_bytes": 0, 2024-05-09T01:02:14.364 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_items": 0, 2024-05-09T01:02:14.364 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_bytes": 0, 2024-05-09T01:02:14.365 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_items": 0, 2024-05-09T01:02:14.365 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T01:02:14.365 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T01:02:14.365 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T01:02:14.365 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T01:02:14.365 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T01:02:14.365 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_items": 0, 2024-05-09T01:02:14.365 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_bytes": 0, 2024-05-09T01:02:14.365 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_items": 0, 2024-05-09T01:02:14.365 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T01:02:14.365 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T01:02:14.365 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_bytes": 0, 2024-05-09T01:02:14.366 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_items": 0, 2024-05-09T01:02:14.366 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_bytes": 0, 2024-05-09T01:02:14.366 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_items": 0, 2024-05-09T01:02:14.366 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T01:02:14.366 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_items": 0, 2024-05-09T01:02:14.366 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T01:02:14.366 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_items": 0, 2024-05-09T01:02:14.366 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_bytes": 301276, 2024-05-09T01:02:14.366 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_items": 112, 2024-05-09T01:02:14.366 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_bytes": 0, 2024-05-09T01:02:14.366 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_items": 0, 2024-05-09T01:02:14.366 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_bytes": 0, 2024-05-09T01:02:14.366 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_items": 0, 2024-05-09T01:02:14.367 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_bytes": 0, 2024-05-09T01:02:14.367 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_items": 0, 2024-05-09T01:02:14.367 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_bytes": 0, 2024-05-09T01:02:14.367 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_items": 0, 2024-05-09T01:02:14.367 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_bytes": 7736, 2024-05-09T01:02:14.367 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_items": 164, 2024-05-09T01:02:14.367 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T01:02:14.367 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_items": 0, 2024-05-09T01:02:14.367 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_bytes": 0, 2024-05-09T01:02:14.367 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_items": 0, 2024-05-09T01:02:14.367 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_bytes": 76781, 2024-05-09T01:02:14.367 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_items": 315, 2024-05-09T01:02:14.368 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_bytes": 0, 2024-05-09T01:02:14.368 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_items": 0, 2024-05-09T01:02:14.368 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_bytes": 0, 2024-05-09T01:02:14.368 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_items": 0 2024-05-09T01:02:14.368 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:14.368 INFO:teuthology.orchestra.run.smithi157.stdout: "objecter": { 2024-05-09T01:02:14.368 INFO:teuthology.orchestra.run.smithi157.stdout: "op_active": 0, 2024-05-09T01:02:14.368 INFO:teuthology.orchestra.run.smithi157.stdout: "op_laggy": 0, 2024-05-09T01:02:14.368 INFO:teuthology.orchestra.run.smithi157.stdout: "op_send": 78, 2024-05-09T01:02:14.368 INFO:teuthology.orchestra.run.smithi157.stdout: "op_send_bytes": 35174, 2024-05-09T01:02:14.368 INFO:teuthology.orchestra.run.smithi157.stdout: "op_resend": 33, 2024-05-09T01:02:14.368 INFO:teuthology.orchestra.run.smithi157.stdout: "op_reply": 78, 2024-05-09T01:02:14.368 INFO:teuthology.orchestra.run.smithi157.stdout: "op_latency": { 2024-05-09T01:02:14.368 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 78, 2024-05-09T01:02:14.368 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 1.249978596, 2024-05-09T01:02:14.369 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.016025366 2024-05-09T01:02:14.369 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:14.369 INFO:teuthology.orchestra.run.smithi157.stdout: "op_inflight": 0, 2024-05-09T01:02:14.369 INFO:teuthology.orchestra.run.smithi157.stdout: "oplen_avg": { 2024-05-09T01:02:14.369 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 78, 2024-05-09T01:02:14.369 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 104 2024-05-09T01:02:14.369 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:14.369 INFO:teuthology.orchestra.run.smithi157.stdout: "op": 78, 2024-05-09T01:02:14.369 INFO:teuthology.orchestra.run.smithi157.stdout: "op_r": 15, 2024-05-09T01:02:14.369 INFO:teuthology.orchestra.run.smithi157.stdout: "op_w": 63, 2024-05-09T01:02:14.369 INFO:teuthology.orchestra.run.smithi157.stdout: "op_rmw": 0, 2024-05-09T01:02:14.369 INFO:teuthology.orchestra.run.smithi157.stdout: "op_pg": 0, 2024-05-09T01:02:14.369 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_stat": 0, 2024-05-09T01:02:14.369 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_create": 10, 2024-05-09T01:02:14.370 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_read": 1, 2024-05-09T01:02:14.370 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_write": 12, 2024-05-09T01:02:14.370 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_writefull": 10, 2024-05-09T01:02:14.370 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_writesame": 0, 2024-05-09T01:02:14.370 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_append": 0, 2024-05-09T01:02:14.370 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_zero": 0, 2024-05-09T01:02:14.370 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_truncate": 0, 2024-05-09T01:02:14.370 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_delete": 14, 2024-05-09T01:02:14.370 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_mapext": 0, 2024-05-09T01:02:14.370 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_sparse_read": 0, 2024-05-09T01:02:14.370 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_clonerange": 0, 2024-05-09T01:02:14.370 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_getxattr": 14, 2024-05-09T01:02:14.371 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_setxattr": 20, 2024-05-09T01:02:14.371 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_cmpxattr": 0, 2024-05-09T01:02:14.371 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_rmxattr": 0, 2024-05-09T01:02:14.371 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_resetxattrs": 0, 2024-05-09T01:02:14.371 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_call": 0, 2024-05-09T01:02:14.371 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_watch": 0, 2024-05-09T01:02:14.371 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_notify": 0, 2024-05-09T01:02:14.371 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T01:02:14.371 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_pgls": 0, 2024-05-09T01:02:14.371 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_pgls_filter": 0, 2024-05-09T01:02:14.371 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_other": 0, 2024-05-09T01:02:14.371 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_active": 0, 2024-05-09T01:02:14.371 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_send": 0, 2024-05-09T01:02:14.371 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_resend": 0, 2024-05-09T01:02:14.372 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_ping": 0, 2024-05-09T01:02:14.372 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_active": 0, 2024-05-09T01:02:14.372 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_send": 0, 2024-05-09T01:02:14.372 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_resend": 0, 2024-05-09T01:02:14.372 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_active": 0, 2024-05-09T01:02:14.372 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_send": 0, 2024-05-09T01:02:14.372 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_resend": 0, 2024-05-09T01:02:14.372 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_active": 0, 2024-05-09T01:02:14.372 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_send": 0, 2024-05-09T01:02:14.372 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_resend": 0, 2024-05-09T01:02:14.372 INFO:teuthology.orchestra.run.smithi157.stdout: "command_active": 0, 2024-05-09T01:02:14.372 INFO:teuthology.orchestra.run.smithi157.stdout: "command_send": 0, 2024-05-09T01:02:14.372 INFO:teuthology.orchestra.run.smithi157.stdout: "command_resend": 0, 2024-05-09T01:02:14.372 INFO:teuthology.orchestra.run.smithi157.stdout: "map_epoch": 82, 2024-05-09T01:02:14.372 INFO:teuthology.orchestra.run.smithi157.stdout: "map_full": 0, 2024-05-09T01:02:14.373 INFO:teuthology.orchestra.run.smithi157.stdout: "map_inc": 4, 2024-05-09T01:02:14.373 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_sessions": 10, 2024-05-09T01:02:14.373 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_session_open": 14, 2024-05-09T01:02:14.373 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_session_close": 4, 2024-05-09T01:02:14.373 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_laggy": 0, 2024-05-09T01:02:14.373 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_wr": 20, 2024-05-09T01:02:14.373 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_rd": 0, 2024-05-09T01:02:14.373 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_del": 3 2024-05-09T01:02:14.373 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:14.373 INFO:teuthology.orchestra.run.smithi157.stdout: "oft": { 2024-05-09T01:02:14.373 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_objs": 1, 2024-05-09T01:02:14.373 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_kv_pairs": 0, 2024-05-09T01:02:14.373 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_updates": 0, 2024-05-09T01:02:14.374 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_removes": 0 2024-05-09T01:02:14.374 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:14.374 INFO:teuthology.orchestra.run.smithi157.stdout: "osd-slow-ops": { 2024-05-09T01:02:14.374 INFO:teuthology.orchestra.run.smithi157.stdout: "slow_ops_count": 0 2024-05-09T01:02:14.374 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:14.374 INFO:teuthology.orchestra.run.smithi157.stdout: "purge_queue": { 2024-05-09T01:02:14.374 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_ops": 0, 2024-05-09T01:02:14.374 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_ops_high_water": 2, 2024-05-09T01:02:14.374 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing": 0, 2024-05-09T01:02:14.374 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_high_water": 2, 2024-05-09T01:02:14.374 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executed_ops": 2, 2024-05-09T01:02:14.374 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executed": 2, 2024-05-09T01:02:14.374 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_item_in_journal": 0 2024-05-09T01:02:14.374 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:14.375 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T01:02:14.375 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-05-09T01:02:14.375 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2024-05-09T01:02:14.375 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-05-09T01:02:14.375 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 2938, 2024-05-09T01:02:14.375 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 1125057, 2024-05-09T01:02:14.375 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-05-09T01:02:14.375 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 2938, 2024-05-09T01:02:14.375 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-05-09T01:02:14.375 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-05-09T01:02:14.375 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 2938, 2024-05-09T01:02:14.375 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 1125057, 2024-05-09T01:02:14.375 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-05-09T01:02:14.375 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:14.376 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:14.376 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:14.376 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-09T01:02:14.376 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:14.376 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-objecter_bytes": { 2024-05-09T01:02:14.376 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-05-09T01:02:14.376 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2024-05-09T01:02:14.376 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-05-09T01:02:14.376 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-05-09T01:02:14.376 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-05-09T01:02:14.376 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-05-09T01:02:14.376 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-05-09T01:02:14.376 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 78, 2024-05-09T01:02:14.376 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 35376, 2024-05-09T01:02:14.377 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 78, 2024-05-09T01:02:14.377 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 35376, 2024-05-09T01:02:14.377 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-05-09T01:02:14.377 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:14.377 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:14.377 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:14.377 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-09T01:02:14.377 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:14.377 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-objecter_ops": { 2024-05-09T01:02:14.377 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-05-09T01:02:14.377 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 1024, 2024-05-09T01:02:14.377 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-05-09T01:02:14.377 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-05-09T01:02:14.377 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-05-09T01:02:14.378 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-05-09T01:02:14.378 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-05-09T01:02:14.378 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 78, 2024-05-09T01:02:14.378 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 78, 2024-05-09T01:02:14.378 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 78, 2024-05-09T01:02:14.378 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 78, 2024-05-09T01:02:14.378 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-05-09T01:02:14.378 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:14.378 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:14.378 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:14.378 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-09T01:02:14.378 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:14.378 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-write_buf_throttle": { 2024-05-09T01:02:14.378 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-05-09T01:02:14.378 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 3758096384, 2024-05-09T01:02:14.379 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-05-09T01:02:14.379 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 2, 2024-05-09T01:02:14.379 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 202, 2024-05-09T01:02:14.379 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-05-09T01:02:14.379 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 2, 2024-05-09T01:02:14.379 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-05-09T01:02:14.379 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-05-09T01:02:14.379 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 1, 2024-05-09T01:02:14.379 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 202, 2024-05-09T01:02:14.379 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-05-09T01:02:14.379 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:14.379 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:14.379 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:14.379 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-09T01:02:14.380 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:14.380 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-write_buf_throttle-0x55aa923404e0": { 2024-05-09T01:02:14.380 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-05-09T01:02:14.380 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 3758096384, 2024-05-09T01:02:14.380 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-05-09T01:02:14.380 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 15, 2024-05-09T01:02:14.380 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 23141, 2024-05-09T01:02:14.380 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-05-09T01:02:14.380 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 15, 2024-05-09T01:02:14.380 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-05-09T01:02:14.380 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-05-09T01:02:14.380 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 11, 2024-05-09T01:02:14.380 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 23141, 2024-05-09T01:02:14.380 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-05-09T01:02:14.380 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:14.381 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:14.381 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:14.381 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-09T01:02:14.381 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-09T01:02:14.381 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-05-09T01:02:15.395 INFO:tasks.check_counter:Daemon mds.c mds.exported=1 2024-05-09T01:02:15.395 INFO:tasks.check_counter:Daemon mds.c mds.imported=1 2024-05-09T01:02:15.395 DEBUG:tasks.check_counter:Getting stats from f 2024-05-09T01:02:15.395 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph daemon mds.f perf dump 2024-05-09T01:02:15.644 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T01:02:15.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:15 smithi157 ceph-mon[34294]: pgmap v848: 97 pgs: 97 active+clean; 5.2 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 47 MiB/s wr, 505 op/s 2024-05-09T01:02:16.040 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:15 smithi005 ceph-mon[30365]: pgmap v848: 97 pgs: 97 active+clean; 5.2 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 47 MiB/s wr, 505 op/s 2024-05-09T01:02:16.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:15 smithi131 ceph-mon[34547]: pgmap v848: 97 pgs: 97 active+clean; 5.2 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 47 MiB/s wr, 505 op/s 2024-05-09T01:02:17.833 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:17 smithi157 ceph-mon[34294]: pgmap v849: 97 pgs: 97 active+clean; 4.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 54 MiB/s wr, 604 op/s 2024-05-09T01:02:18.040 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:17 smithi005 ceph-mon[30365]: pgmap v849: 97 pgs: 97 active+clean; 4.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 54 MiB/s wr, 604 op/s 2024-05-09T01:02:18.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:17 smithi131 ceph-mon[34547]: pgmap v849: 97 pgs: 97 active+clean; 4.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 54 MiB/s wr, 604 op/s 2024-05-09T01:02:18.418 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-05-09T01:02:18.419 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T01:02:18.419 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 677, 2024-05-09T01:02:18.419 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 880, 2024-05-09T01:02:18.419 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 211282, 2024-05-09T01:02:18.419 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 390303, 2024-05-09T01:02:18.419 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 10, 2024-05-09T01:02:18.419 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 2, 2024-05-09T01:02:18.419 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.098917705, 2024-05-09T01:02:18.419 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.041623102, 2024-05-09T01:02:18.419 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.049306005, 2024-05-09T01:02:18.419 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.006499765, 2024-05-09T01:02:18.419 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:02:18.420 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 884, 2024-05-09T01:02:18.420 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.083194008, 2024-05-09T01:02:18.420 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000094110 2024-05-09T01:02:18.420 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:18.420 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:02:18.420 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1471, 2024-05-09T01:02:18.420 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000447818, 2024-05-09T01:02:18.420 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000304 2024-05-09T01:02:18.420 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:18.420 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T01:02:18.420 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T01:02:18.420 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:18.420 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T01:02:18.421 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 122, 2024-05-09T01:02:18.421 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 352, 2024-05-09T01:02:18.421 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 65353, 2024-05-09T01:02:18.421 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 896727, 2024-05-09T01:02:18.421 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 33, 2024-05-09T01:02:18.421 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 6, 2024-05-09T01:02:18.421 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.037338909, 2024-05-09T01:02:18.421 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.013619697, 2024-05-09T01:02:18.421 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.021935942, 2024-05-09T01:02:18.421 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.000314762, 2024-05-09T01:02:18.421 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:02:18.421 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 351, 2024-05-09T01:02:18.421 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.035458428, 2024-05-09T01:02:18.422 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000101021 2024-05-09T01:02:18.422 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:18.422 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:02:18.422 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 175, 2024-05-09T01:02:18.422 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000046915, 2024-05-09T01:02:18.422 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000268 2024-05-09T01:02:18.422 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:18.422 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-05-09T01:02:18.422 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 573120 2024-05-09T01:02:18.422 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:18.422 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T01:02:18.422 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 470, 2024-05-09T01:02:18.422 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 462, 2024-05-09T01:02:18.423 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 683950, 2024-05-09T01:02:18.423 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 311103, 2024-05-09T01:02:18.423 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 16, 2024-05-09T01:02:18.423 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 7, 2024-05-09T01:02:18.423 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.075275943, 2024-05-09T01:02:18.423 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.024176084, 2024-05-09T01:02:18.423 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.040210789, 2024-05-09T01:02:18.423 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.010954176, 2024-05-09T01:02:18.423 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:02:18.423 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 455, 2024-05-09T01:02:18.423 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.041367517, 2024-05-09T01:02:18.423 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000090917 2024-05-09T01:02:18.423 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:18.423 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:02:18.424 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 187, 2024-05-09T01:02:18.424 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000054668, 2024-05-09T01:02:18.424 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000292 2024-05-09T01:02:18.424 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:18.424 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 624304, 2024-05-09T01:02:18.424 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 196592 2024-05-09T01:02:18.424 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:18.424 INFO:teuthology.orchestra.run.smithi157.stdout: "cct": { 2024-05-09T01:02:18.424 INFO:teuthology.orchestra.run.smithi157.stdout: "total_workers": 1, 2024-05-09T01:02:18.424 INFO:teuthology.orchestra.run.smithi157.stdout: "unhealthy_workers": 0 2024-05-09T01:02:18.424 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:18.424 INFO:teuthology.orchestra.run.smithi157.stdout: "finisher-MDSRank": { 2024-05-09T01:02:18.424 INFO:teuthology.orchestra.run.smithi157.stdout: "queue_len": 0, 2024-05-09T01:02:18.425 INFO:teuthology.orchestra.run.smithi157.stdout: "complete_latency": { 2024-05-09T01:02:18.425 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 47, 2024-05-09T01:02:18.425 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.001421687, 2024-05-09T01:02:18.425 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000030248 2024-05-09T01:02:18.425 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-09T01:02:18.425 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:18.425 INFO:teuthology.orchestra.run.smithi157.stdout: "finisher-PurgeQueue": { 2024-05-09T01:02:18.425 INFO:teuthology.orchestra.run.smithi157.stdout: "queue_len": 0, 2024-05-09T01:02:18.425 INFO:teuthology.orchestra.run.smithi157.stdout: "complete_latency": { 2024-05-09T01:02:18.425 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 2, 2024-05-09T01:02:18.426 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000006757, 2024-05-09T01:02:18.426 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000003378 2024-05-09T01:02:18.426 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-09T01:02:18.426 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:18.426 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": { 2024-05-09T01:02:18.426 INFO:teuthology.orchestra.run.smithi157.stdout: "request": 1, 2024-05-09T01:02:18.426 INFO:teuthology.orchestra.run.smithi157.stdout: "reply": 1, 2024-05-09T01:02:18.426 INFO:teuthology.orchestra.run.smithi157.stdout: "reply_latency": { 2024-05-09T01:02:18.426 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1, 2024-05-09T01:02:18.426 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000133812, 2024-05-09T01:02:18.426 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000133812 2024-05-09T01:02:18.426 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:18.426 INFO:teuthology.orchestra.run.smithi157.stdout: "slow_reply": 0, 2024-05-09T01:02:18.427 INFO:teuthology.orchestra.run.smithi157.stdout: "forward": 0, 2024-05-09T01:02:18.427 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_fetch_complete": 0, 2024-05-09T01:02:18.427 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_fetch_keys": 0, 2024-05-09T01:02:18.427 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_commit": 11, 2024-05-09T01:02:18.427 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_split": 0, 2024-05-09T01:02:18.427 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_merge": 0, 2024-05-09T01:02:18.427 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes": 10, 2024-05-09T01:02:18.427 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_top": 0, 2024-05-09T01:02:18.427 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_bottom": 10, 2024-05-09T01:02:18.427 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_pin_tail": 0, 2024-05-09T01:02:18.427 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_pinned": 10, 2024-05-09T01:02:18.427 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_expired": 0, 2024-05-09T01:02:18.427 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_with_caps": 1, 2024-05-09T01:02:18.427 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": 1, 2024-05-09T01:02:18.428 INFO:teuthology.orchestra.run.smithi157.stdout: "subtrees": 1, 2024-05-09T01:02:18.428 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse": 0, 2024-05-09T01:02:18.428 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_hit": 0, 2024-05-09T01:02:18.428 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_forward": 0, 2024-05-09T01:02:18.428 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_discover": 0, 2024-05-09T01:02:18.428 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_dir_fetch": 0, 2024-05-09T01:02:18.428 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_remote_ino": 0, 2024-05-09T01:02:18.428 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_lock": 0, 2024-05-09T01:02:18.428 INFO:teuthology.orchestra.run.smithi157.stdout: "load_cent": 0, 2024-05-09T01:02:18.428 INFO:teuthology.orchestra.run.smithi157.stdout: "q": 0, 2024-05-09T01:02:18.428 INFO:teuthology.orchestra.run.smithi157.stdout: "exported": 0, 2024-05-09T01:02:18.428 INFO:teuthology.orchestra.run.smithi157.stdout: "exported_inodes": 0, 2024-05-09T01:02:18.428 INFO:teuthology.orchestra.run.smithi157.stdout: "imported": 0, 2024-05-09T01:02:18.428 INFO:teuthology.orchestra.run.smithi157.stdout: "imported_inodes": 0, 2024-05-09T01:02:18.429 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_dir_fetch": 0, 2024-05-09T01:02:18.429 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_backtrace_fetch": 0, 2024-05-09T01:02:18.429 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_peer_discover": 0, 2024-05-09T01:02:18.429 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rfiles": 21, 2024-05-09T01:02:18.429 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rbytes": 10738136400, 2024-05-09T01:02:18.429 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rsnaps": 0, 2024-05-09T01:02:18.429 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_backtrace_fetch": 0, 2024-05-09T01:02:18.429 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_set_tag": 0, 2024-05-09T01:02:18.429 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T01:02:18.429 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_inotable_repaired": 0, 2024-05-09T01:02:18.429 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dir_inodes": 0, 2024-05-09T01:02:18.429 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T01:02:18.429 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T01:02:18.429 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_file_inodes": 0, 2024-05-09T01:02:18.430 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_inode_file_caps": 0, 2024-05-09T01:02:18.430 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_revoke": 0, 2024-05-09T01:02:18.430 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_grant": 0, 2024-05-09T01:02:18.430 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T01:02:18.430 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T01:02:18.430 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T01:02:18.430 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_caps": 0, 2024-05-09T01:02:18.430 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_caps_dirty": 0, 2024-05-09T01:02:18.430 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_cap_release": 0, 2024-05-09T01:02:18.430 INFO:teuthology.orchestra.run.smithi157.stdout: "process_request_cap_release": 0 2024-05-09T01:02:18.430 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:18.430 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_cache": { 2024-05-09T01:02:18.430 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_update": 0, 2024-05-09T01:02:18.431 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_update_receipt": 0, 2024-05-09T01:02:18.431 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_try_discover": 0, 2024-05-09T01:02:18.431 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_send_discover": 1, 2024-05-09T01:02:18.431 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_handle_discover": 0, 2024-05-09T01:02:18.431 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays": 0, 2024-05-09T01:02:18.431 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays_delayed": 0, 2024-05-09T01:02:18.431 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays_enqueuing": 0, 2024-05-09T01:02:18.431 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_created": 0, 2024-05-09T01:02:18.431 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_enqueued": 0, 2024-05-09T01:02:18.431 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_reintegrated": 0, 2024-05-09T01:02:18.431 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_migrated": 0, 2024-05-09T01:02:18.431 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_processing": 0, 2024-05-09T01:02:18.431 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_enqueued": 0, 2024-05-09T01:02:18.431 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_prioritized": 0, 2024-05-09T01:02:18.432 INFO:teuthology.orchestra.run.smithi157.stdout: "recovery_started": 0, 2024-05-09T01:02:18.432 INFO:teuthology.orchestra.run.smithi157.stdout: "recovery_completed": 0, 2024-05-09T01:02:18.432 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_quiesce_path": 0, 2024-05-09T01:02:18.432 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_quiesce_inode": 0, 2024-05-09T01:02:18.432 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T01:02:18.432 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_exportdir": 0, 2024-05-09T01:02:18.432 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_flush": 0, 2024-05-09T01:02:18.432 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_fragmentdir": 0, 2024-05-09T01:02:18.432 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_fragstats": 0, 2024-05-09T01:02:18.432 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_inodestats": 0 2024-05-09T01:02:18.432 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:18.432 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_log": { 2024-05-09T01:02:18.432 INFO:teuthology.orchestra.run.smithi157.stdout: "evlrg": 0, 2024-05-09T01:02:18.432 INFO:teuthology.orchestra.run.smithi157.stdout: "evadd": 5, 2024-05-09T01:02:18.433 INFO:teuthology.orchestra.run.smithi157.stdout: "evex": 0, 2024-05-09T01:02:18.433 INFO:teuthology.orchestra.run.smithi157.stdout: "evtrm": 0, 2024-05-09T01:02:18.433 INFO:teuthology.orchestra.run.smithi157.stdout: "ev": 5, 2024-05-09T01:02:18.433 INFO:teuthology.orchestra.run.smithi157.stdout: "evexg": 0, 2024-05-09T01:02:18.433 INFO:teuthology.orchestra.run.smithi157.stdout: "evexd": 0, 2024-05-09T01:02:18.433 INFO:teuthology.orchestra.run.smithi157.stdout: "segadd": 2, 2024-05-09T01:02:18.433 INFO:teuthology.orchestra.run.smithi157.stdout: "segex": 0, 2024-05-09T01:02:18.433 INFO:teuthology.orchestra.run.smithi157.stdout: "segtrm": 0, 2024-05-09T01:02:18.433 INFO:teuthology.orchestra.run.smithi157.stdout: "seg": 2, 2024-05-09T01:02:18.433 INFO:teuthology.orchestra.run.smithi157.stdout: "segmjr": 2, 2024-05-09T01:02:18.433 INFO:teuthology.orchestra.run.smithi157.stdout: "segexg": 0, 2024-05-09T01:02:18.433 INFO:teuthology.orchestra.run.smithi157.stdout: "segexd": 0, 2024-05-09T01:02:18.433 INFO:teuthology.orchestra.run.smithi157.stdout: "expos": 4194304, 2024-05-09T01:02:18.434 INFO:teuthology.orchestra.run.smithi157.stdout: "wrpos": 4195732, 2024-05-09T01:02:18.434 INFO:teuthology.orchestra.run.smithi157.stdout: "rdpos": 0, 2024-05-09T01:02:18.434 INFO:teuthology.orchestra.run.smithi157.stdout: "jlat": { 2024-05-09T01:02:18.434 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 4, 2024-05-09T01:02:18.434 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.006719479, 2024-05-09T01:02:18.434 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.001679869 2024-05-09T01:02:18.434 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:18.434 INFO:teuthology.orchestra.run.smithi157.stdout: "replayed": 0 2024-05-09T01:02:18.434 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:18.434 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_mem": { 2024-05-09T01:02:18.434 INFO:teuthology.orchestra.run.smithi157.stdout: "ino": 13, 2024-05-09T01:02:18.434 INFO:teuthology.orchestra.run.smithi157.stdout: "ino+": 13, 2024-05-09T01:02:18.434 INFO:teuthology.orchestra.run.smithi157.stdout: "ino-": 0, 2024-05-09T01:02:18.434 INFO:teuthology.orchestra.run.smithi157.stdout: "dir": 11, 2024-05-09T01:02:18.435 INFO:teuthology.orchestra.run.smithi157.stdout: "dir+": 12, 2024-05-09T01:02:18.435 INFO:teuthology.orchestra.run.smithi157.stdout: "dir-": 1, 2024-05-09T01:02:18.435 INFO:teuthology.orchestra.run.smithi157.stdout: "dn": 10, 2024-05-09T01:02:18.435 INFO:teuthology.orchestra.run.smithi157.stdout: "dn+": 10, 2024-05-09T01:02:18.435 INFO:teuthology.orchestra.run.smithi157.stdout: "dn-": 0, 2024-05-09T01:02:18.435 INFO:teuthology.orchestra.run.smithi157.stdout: "cap": 1, 2024-05-09T01:02:18.435 INFO:teuthology.orchestra.run.smithi157.stdout: "cap+": 1, 2024-05-09T01:02:18.435 INFO:teuthology.orchestra.run.smithi157.stdout: "cap-": 0, 2024-05-09T01:02:18.435 INFO:teuthology.orchestra.run.smithi157.stdout: "rss": 123472, 2024-05-09T01:02:18.435 INFO:teuthology.orchestra.run.smithi157.stdout: "heap": 223632 2024-05-09T01:02:18.435 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:18.435 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_server": { 2024-05-09T01:02:18.435 INFO:teuthology.orchestra.run.smithi157.stdout: "dispatch_client_request": 1, 2024-05-09T01:02:18.436 INFO:teuthology.orchestra.run.smithi157.stdout: "dispatch_server_request": 0, 2024-05-09T01:02:18.436 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_request": 1, 2024-05-09T01:02:18.436 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_session": 17, 2024-05-09T01:02:18.436 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_peer_request": 0, 2024-05-09T01:02:18.436 INFO:teuthology.orchestra.run.smithi157.stdout: "req_create_latency": { 2024-05-09T01:02:18.436 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:18.436 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:18.436 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:18.436 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:18.436 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getattr_latency": { 2024-05-09T01:02:18.436 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:18.436 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:18.436 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:18.436 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:18.437 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getfilelock_latency": { 2024-05-09T01:02:18.437 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:18.437 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:18.437 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:18.437 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:18.437 INFO:teuthology.orchestra.run.smithi157.stdout: "req_link_latency": { 2024-05-09T01:02:18.437 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:18.437 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:18.437 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:18.437 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:18.437 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookup_latency": { 2024-05-09T01:02:18.437 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:18.437 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:18.438 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:18.438 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:18.438 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookuphash_latency": { 2024-05-09T01:02:18.438 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:18.438 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:18.438 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:18.438 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:18.438 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupino_latency": { 2024-05-09T01:02:18.438 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1, 2024-05-09T01:02:18.438 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000133812, 2024-05-09T01:02:18.438 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000133812 2024-05-09T01:02:18.438 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:18.438 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupname_latency": { 2024-05-09T01:02:18.439 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:18.439 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:18.439 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:18.439 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:18.439 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupparent_latency": { 2024-05-09T01:02:18.439 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:18.439 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:18.439 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:18.439 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:18.439 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupsnap_latency": { 2024-05-09T01:02:18.439 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:18.439 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:18.439 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:18.439 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:18.439 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lssnap_latency": { 2024-05-09T01:02:18.440 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:18.440 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:18.440 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:18.440 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:18.440 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mkdir_latency": { 2024-05-09T01:02:18.440 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:18.440 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:18.441 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:18.441 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:18.441 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mknod_latency": { 2024-05-09T01:02:18.441 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:18.441 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:18.441 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:18.441 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:18.441 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mksnap_latency": { 2024-05-09T01:02:18.441 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:18.441 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:18.441 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:18.441 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:18.441 INFO:teuthology.orchestra.run.smithi157.stdout: "req_open_latency": { 2024-05-09T01:02:18.442 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:18.442 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:18.442 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:18.442 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:18.442 INFO:teuthology.orchestra.run.smithi157.stdout: "req_readdir_latency": { 2024-05-09T01:02:18.442 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:18.442 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:18.442 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:18.442 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:18.442 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rename_latency": { 2024-05-09T01:02:18.442 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:18.442 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:18.443 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:18.443 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:18.443 INFO:teuthology.orchestra.run.smithi157.stdout: "req_renamesnap_latency": { 2024-05-09T01:02:18.443 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:18.443 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:18.443 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:18.443 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:18.443 INFO:teuthology.orchestra.run.smithi157.stdout: "req_snapdiff_latency": { 2024-05-09T01:02:18.443 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:18.443 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:18.443 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:18.443 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:18.443 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmdir_latency": { 2024-05-09T01:02:18.444 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:18.444 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:18.444 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:18.444 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:18.444 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmsnap_latency": { 2024-05-09T01:02:18.444 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:18.444 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:18.444 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:18.444 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:18.444 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmxattr_latency": { 2024-05-09T01:02:18.444 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:18.444 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:18.444 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:18.444 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:18.445 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setattr_latency": { 2024-05-09T01:02:18.445 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:18.445 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:18.445 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:18.445 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:18.445 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setdirlayout_latency": { 2024-05-09T01:02:18.445 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:18.445 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:18.445 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:18.445 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:18.446 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setfilelock_latency": { 2024-05-09T01:02:18.446 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:18.446 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:18.446 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:18.446 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:18.446 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setlayout_latency": { 2024-05-09T01:02:18.446 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:18.446 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:18.446 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:18.446 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:18.446 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setxattr_latency": { 2024-05-09T01:02:18.446 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:18.446 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:18.446 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:18.447 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:18.447 INFO:teuthology.orchestra.run.smithi157.stdout: "req_symlink_latency": { 2024-05-09T01:02:18.447 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:18.447 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:18.447 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:18.447 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:18.447 INFO:teuthology.orchestra.run.smithi157.stdout: "req_unlink_latency": { 2024-05-09T01:02:18.447 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:18.447 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:18.447 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:18.447 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:18.447 INFO:teuthology.orchestra.run.smithi157.stdout: "cap_revoke_eviction": 0, 2024-05-09T01:02:18.447 INFO:teuthology.orchestra.run.smithi157.stdout: "cap_acquisition_throttle": 0, 2024-05-09T01:02:18.448 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getvxattr_latency": { 2024-05-09T01:02:18.448 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:18.448 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:18.448 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:18.448 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-09T01:02:18.448 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:18.448 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_sessions": { 2024-05-09T01:02:18.448 INFO:teuthology.orchestra.run.smithi157.stdout: "session_count": 1, 2024-05-09T01:02:18.448 INFO:teuthology.orchestra.run.smithi157.stdout: "session_add": 2, 2024-05-09T01:02:18.448 INFO:teuthology.orchestra.run.smithi157.stdout: "session_remove": 1, 2024-05-09T01:02:18.448 INFO:teuthology.orchestra.run.smithi157.stdout: "sessions_open": 1, 2024-05-09T01:02:18.448 INFO:teuthology.orchestra.run.smithi157.stdout: "sessions_stale": 0, 2024-05-09T01:02:18.448 INFO:teuthology.orchestra.run.smithi157.stdout: "total_load": 1, 2024-05-09T01:02:18.449 INFO:teuthology.orchestra.run.smithi157.stdout: "average_load": 1, 2024-05-09T01:02:18.449 INFO:teuthology.orchestra.run.smithi157.stdout: "avg_session_uptime": 898, 2024-05-09T01:02:18.449 INFO:teuthology.orchestra.run.smithi157.stdout: "mdthresh_evicted": 0 2024-05-09T01:02:18.449 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:18.449 INFO:teuthology.orchestra.run.smithi157.stdout: "mempool": { 2024-05-09T01:02:18.449 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_bytes": 0, 2024-05-09T01:02:18.449 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_items": 0, 2024-05-09T01:02:18.449 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T01:02:18.449 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_items": 0, 2024-05-09T01:02:18.449 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T01:02:18.449 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_items": 0, 2024-05-09T01:02:18.449 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T01:02:18.449 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T01:02:18.450 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T01:02:18.450 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T01:02:18.450 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T01:02:18.450 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_items": 0, 2024-05-09T01:02:18.450 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T01:02:18.450 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T01:02:18.450 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_bytes": 0, 2024-05-09T01:02:18.450 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_items": 0, 2024-05-09T01:02:18.450 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_bytes": 0, 2024-05-09T01:02:18.450 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_items": 0, 2024-05-09T01:02:18.450 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T01:02:18.450 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T01:02:18.450 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T01:02:18.451 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T01:02:18.451 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T01:02:18.451 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_items": 0, 2024-05-09T01:02:18.451 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_bytes": 0, 2024-05-09T01:02:18.451 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_items": 0, 2024-05-09T01:02:18.451 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T01:02:18.451 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T01:02:18.451 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_bytes": 0, 2024-05-09T01:02:18.451 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_items": 0, 2024-05-09T01:02:18.451 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_bytes": 0, 2024-05-09T01:02:18.451 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_items": 0, 2024-05-09T01:02:18.451 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T01:02:18.451 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_items": 0, 2024-05-09T01:02:18.452 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T01:02:18.452 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_items": 0, 2024-05-09T01:02:18.452 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_bytes": 230995, 2024-05-09T01:02:18.452 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_items": 78, 2024-05-09T01:02:18.452 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_bytes": 0, 2024-05-09T01:02:18.452 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_items": 0, 2024-05-09T01:02:18.452 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_bytes": 0, 2024-05-09T01:02:18.452 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_items": 0, 2024-05-09T01:02:18.452 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_bytes": 0, 2024-05-09T01:02:18.452 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_items": 0, 2024-05-09T01:02:18.452 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_bytes": 0, 2024-05-09T01:02:18.452 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_items": 0, 2024-05-09T01:02:18.452 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_bytes": 7736, 2024-05-09T01:02:18.452 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_items": 164, 2024-05-09T01:02:18.453 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T01:02:18.453 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_items": 0, 2024-05-09T01:02:18.453 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_bytes": 0, 2024-05-09T01:02:18.453 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_items": 0, 2024-05-09T01:02:18.453 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_bytes": 51240, 2024-05-09T01:02:18.453 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_items": 200, 2024-05-09T01:02:18.453 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_bytes": 0, 2024-05-09T01:02:18.453 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_items": 0, 2024-05-09T01:02:18.453 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_bytes": 0, 2024-05-09T01:02:18.453 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_items": 0 2024-05-09T01:02:18.453 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:18.453 INFO:teuthology.orchestra.run.smithi157.stdout: "objecter": { 2024-05-09T01:02:18.453 INFO:teuthology.orchestra.run.smithi157.stdout: "op_active": 0, 2024-05-09T01:02:18.453 INFO:teuthology.orchestra.run.smithi157.stdout: "op_laggy": 0, 2024-05-09T01:02:18.454 INFO:teuthology.orchestra.run.smithi157.stdout: "op_send": 40, 2024-05-09T01:02:18.454 INFO:teuthology.orchestra.run.smithi157.stdout: "op_send_bytes": 11730, 2024-05-09T01:02:18.454 INFO:teuthology.orchestra.run.smithi157.stdout: "op_resend": 33, 2024-05-09T01:02:18.454 INFO:teuthology.orchestra.run.smithi157.stdout: "op_reply": 40, 2024-05-09T01:02:18.454 INFO:teuthology.orchestra.run.smithi157.stdout: "op_latency": { 2024-05-09T01:02:18.454 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 40, 2024-05-09T01:02:18.454 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.222995986, 2024-05-09T01:02:18.454 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.005574899 2024-05-09T01:02:18.454 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:18.454 INFO:teuthology.orchestra.run.smithi157.stdout: "op_inflight": 0, 2024-05-09T01:02:18.454 INFO:teuthology.orchestra.run.smithi157.stdout: "oplen_avg": { 2024-05-09T01:02:18.454 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 40, 2024-05-09T01:02:18.454 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 62 2024-05-09T01:02:18.455 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:18.455 INFO:teuthology.orchestra.run.smithi157.stdout: "op": 40, 2024-05-09T01:02:18.455 INFO:teuthology.orchestra.run.smithi157.stdout: "op_r": 0, 2024-05-09T01:02:18.455 INFO:teuthology.orchestra.run.smithi157.stdout: "op_w": 40, 2024-05-09T01:02:18.455 INFO:teuthology.orchestra.run.smithi157.stdout: "op_rmw": 0, 2024-05-09T01:02:18.455 INFO:teuthology.orchestra.run.smithi157.stdout: "op_pg": 0, 2024-05-09T01:02:18.455 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_stat": 0, 2024-05-09T01:02:18.455 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_create": 10, 2024-05-09T01:02:18.455 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_read": 0, 2024-05-09T01:02:18.455 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_write": 4, 2024-05-09T01:02:18.455 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_writefull": 7, 2024-05-09T01:02:18.455 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_writesame": 0, 2024-05-09T01:02:18.455 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_append": 0, 2024-05-09T01:02:18.455 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_zero": 0, 2024-05-09T01:02:18.456 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_truncate": 0, 2024-05-09T01:02:18.456 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_delete": 6, 2024-05-09T01:02:18.456 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_mapext": 0, 2024-05-09T01:02:18.456 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_sparse_read": 0, 2024-05-09T01:02:18.456 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_clonerange": 0, 2024-05-09T01:02:18.456 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_getxattr": 0, 2024-05-09T01:02:18.456 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_setxattr": 20, 2024-05-09T01:02:18.456 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_cmpxattr": 0, 2024-05-09T01:02:18.456 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_rmxattr": 0, 2024-05-09T01:02:18.456 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_resetxattrs": 0, 2024-05-09T01:02:18.456 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_call": 0, 2024-05-09T01:02:18.456 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_watch": 0, 2024-05-09T01:02:18.456 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_notify": 0, 2024-05-09T01:02:18.456 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T01:02:18.456 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_pgls": 0, 2024-05-09T01:02:18.457 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_pgls_filter": 0, 2024-05-09T01:02:18.457 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_other": 0, 2024-05-09T01:02:18.457 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_active": 0, 2024-05-09T01:02:18.457 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_send": 0, 2024-05-09T01:02:18.457 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_resend": 0, 2024-05-09T01:02:18.457 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_ping": 0, 2024-05-09T01:02:18.457 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_active": 0, 2024-05-09T01:02:18.457 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_send": 0, 2024-05-09T01:02:18.457 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_resend": 0, 2024-05-09T01:02:18.457 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_active": 0, 2024-05-09T01:02:18.457 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_send": 0, 2024-05-09T01:02:18.457 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_resend": 0, 2024-05-09T01:02:18.457 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_active": 0, 2024-05-09T01:02:18.458 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_send": 0, 2024-05-09T01:02:18.458 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_resend": 0, 2024-05-09T01:02:18.458 INFO:teuthology.orchestra.run.smithi157.stdout: "command_active": 0, 2024-05-09T01:02:18.458 INFO:teuthology.orchestra.run.smithi157.stdout: "command_send": 0, 2024-05-09T01:02:18.458 INFO:teuthology.orchestra.run.smithi157.stdout: "command_resend": 0, 2024-05-09T01:02:18.458 INFO:teuthology.orchestra.run.smithi157.stdout: "map_epoch": 82, 2024-05-09T01:02:18.458 INFO:teuthology.orchestra.run.smithi157.stdout: "map_full": 0, 2024-05-09T01:02:18.458 INFO:teuthology.orchestra.run.smithi157.stdout: "map_inc": 4, 2024-05-09T01:02:18.458 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_sessions": 8, 2024-05-09T01:02:18.458 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_session_open": 13, 2024-05-09T01:02:18.458 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_session_close": 5, 2024-05-09T01:02:18.458 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_laggy": 0, 2024-05-09T01:02:18.458 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_wr": 14, 2024-05-09T01:02:18.458 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_rd": 0, 2024-05-09T01:02:18.459 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_del": 1 2024-05-09T01:02:18.459 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:18.459 INFO:teuthology.orchestra.run.smithi157.stdout: "oft": { 2024-05-09T01:02:18.459 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_objs": 1, 2024-05-09T01:02:18.459 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_kv_pairs": 0, 2024-05-09T01:02:18.459 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_updates": 0, 2024-05-09T01:02:18.459 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_removes": 0 2024-05-09T01:02:18.459 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:18.459 INFO:teuthology.orchestra.run.smithi157.stdout: "osd-slow-ops": { 2024-05-09T01:02:18.459 INFO:teuthology.orchestra.run.smithi157.stdout: "slow_ops_count": 0 2024-05-09T01:02:18.459 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:18.459 INFO:teuthology.orchestra.run.smithi157.stdout: "purge_queue": { 2024-05-09T01:02:18.459 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_ops": 0, 2024-05-09T01:02:18.459 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_ops_high_water": 0, 2024-05-09T01:02:18.460 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing": 0, 2024-05-09T01:02:18.460 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_high_water": 0, 2024-05-09T01:02:18.460 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executed_ops": 0, 2024-05-09T01:02:18.460 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executed": 0, 2024-05-09T01:02:18.460 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_item_in_journal": 0 2024-05-09T01:02:18.460 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:18.460 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T01:02:18.460 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-05-09T01:02:18.460 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2024-05-09T01:02:18.460 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-05-09T01:02:18.460 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 1265, 2024-05-09T01:02:18.460 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 825225, 2024-05-09T01:02:18.460 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-05-09T01:02:18.460 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 1265, 2024-05-09T01:02:18.461 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-05-09T01:02:18.461 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-05-09T01:02:18.461 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 1265, 2024-05-09T01:02:18.461 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 825225, 2024-05-09T01:02:18.461 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-05-09T01:02:18.461 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:18.461 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:18.461 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:18.461 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-09T01:02:18.461 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:18.461 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-objecter_bytes": { 2024-05-09T01:02:18.461 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-05-09T01:02:18.461 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2024-05-09T01:02:18.461 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-05-09T01:02:18.461 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-05-09T01:02:18.462 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-05-09T01:02:18.462 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-05-09T01:02:18.462 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-05-09T01:02:18.462 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 40, 2024-05-09T01:02:18.462 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 11730, 2024-05-09T01:02:18.462 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 40, 2024-05-09T01:02:18.462 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 11730, 2024-05-09T01:02:18.462 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-05-09T01:02:18.462 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:18.462 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:18.462 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:18.462 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-09T01:02:18.462 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:18.463 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-objecter_ops": { 2024-05-09T01:02:18.463 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-05-09T01:02:18.463 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 1024, 2024-05-09T01:02:18.463 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-05-09T01:02:18.463 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-05-09T01:02:18.463 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-05-09T01:02:18.463 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-05-09T01:02:18.463 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-05-09T01:02:18.463 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 40, 2024-05-09T01:02:18.463 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 40, 2024-05-09T01:02:18.463 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 40, 2024-05-09T01:02:18.463 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 40, 2024-05-09T01:02:18.463 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-05-09T01:02:18.463 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:18.464 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:18.464 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:18.464 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-09T01:02:18.464 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:18.464 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-write_buf_throttle": { 2024-05-09T01:02:18.464 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-05-09T01:02:18.464 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 3758096384, 2024-05-09T01:02:18.464 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-05-09T01:02:18.464 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-05-09T01:02:18.464 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-05-09T01:02:18.464 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-05-09T01:02:18.464 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-05-09T01:02:18.464 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-05-09T01:02:18.464 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-05-09T01:02:18.464 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 0, 2024-05-09T01:02:18.465 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 0, 2024-05-09T01:02:18.465 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-05-09T01:02:18.465 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:18.465 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:18.465 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:18.465 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-09T01:02:18.465 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:18.465 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-write_buf_throttle-0x555aadbe04e0": { 2024-05-09T01:02:18.465 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-05-09T01:02:18.465 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 3758096384, 2024-05-09T01:02:18.465 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-05-09T01:02:18.465 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 5, 2024-05-09T01:02:18.465 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 1428, 2024-05-09T01:02:18.465 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-05-09T01:02:18.466 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 5, 2024-05-09T01:02:18.466 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-05-09T01:02:18.466 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-05-09T01:02:18.466 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 4, 2024-05-09T01:02:18.466 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 1428, 2024-05-09T01:02:18.466 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-05-09T01:02:18.466 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:18.466 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:18.466 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:18.466 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-09T01:02:18.466 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-09T01:02:18.466 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-05-09T01:02:19.253 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-05-09T01:02:19.253 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-05-09T01:02:19.253 DEBUG:tasks.check_counter:Getting stats from i 2024-05-09T01:02:19.254 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph daemon mds.i perf dump 2024-05-09T01:02:19.505 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T01:02:19.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:19 smithi157 ceph-mon[34294]: pgmap v850: 97 pgs: 97 active+clean; 3.3 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 61 MiB/s wr, 723 op/s 2024-05-09T01:02:20.040 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:19 smithi005 ceph-mon[30365]: pgmap v850: 97 pgs: 97 active+clean; 3.3 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 61 MiB/s wr, 723 op/s 2024-05-09T01:02:20.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:19 smithi131 ceph-mon[34547]: pgmap v850: 97 pgs: 97 active+clean; 3.3 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 61 MiB/s wr, 723 op/s 2024-05-09T01:02:21.404 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-05-09T01:02:21.404 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T01:02:21.405 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 5749, 2024-05-09T01:02:21.405 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 6335, 2024-05-09T01:02:21.405 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 2046087, 2024-05-09T01:02:21.405 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 366173462, 2024-05-09T01:02:21.405 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 29, 2024-05-09T01:02:21.405 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 3, 2024-05-09T01:02:21.405 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 2.809655721, 2024-05-09T01:02:21.405 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 1.913619058, 2024-05-09T01:02:21.405 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.296909446, 2024-05-09T01:02:21.405 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.533990717, 2024-05-09T01:02:21.405 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:02:21.405 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 6340, 2024-05-09T01:02:21.405 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 3.309004067, 2024-05-09T01:02:21.406 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000521924 2024-05-09T01:02:21.406 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:21.406 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:02:21.406 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 4524, 2024-05-09T01:02:21.406 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.003736361, 2024-05-09T01:02:21.406 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000825 2024-05-09T01:02:21.406 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:21.406 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 642128, 2024-05-09T01:02:21.406 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 210528 2024-05-09T01:02:21.406 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:21.406 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T01:02:21.406 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 4952, 2024-05-09T01:02:21.407 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 4972, 2024-05-09T01:02:21.407 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 1409605, 2024-05-09T01:02:21.407 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 464997739, 2024-05-09T01:02:21.407 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 11, 2024-05-09T01:02:21.407 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 6, 2024-05-09T01:02:21.407 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 2.920996116, 2024-05-09T01:02:21.407 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 2.125795450, 2024-05-09T01:02:21.407 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.163444639, 2024-05-09T01:02:21.407 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.613849374, 2024-05-09T01:02:21.407 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:02:21.407 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 4971, 2024-05-09T01:02:21.407 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 4.555369467, 2024-05-09T01:02:21.407 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000916388 2024-05-09T01:02:21.408 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:21.408 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:02:21.408 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1311, 2024-05-09T01:02:21.408 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000328950, 2024-05-09T01:02:21.408 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000250 2024-05-09T01:02:21.408 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:21.408 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-05-09T01:02:21.408 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 582080 2024-05-09T01:02:21.408 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:21.408 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T01:02:21.408 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 725590, 2024-05-09T01:02:21.408 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 728127, 2024-05-09T01:02:21.408 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 253619114, 2024-05-09T01:02:21.409 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 682637222, 2024-05-09T01:02:21.409 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 160, 2024-05-09T01:02:21.409 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 8, 2024-05-09T01:02:21.409 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 44.780109675, 2024-05-09T01:02:21.409 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 17.762798651, 2024-05-09T01:02:21.409 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 64.024077201, 2024-05-09T01:02:21.409 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 1.196877741, 2024-05-09T01:02:21.409 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:02:21.409 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 728119, 2024-05-09T01:02:21.409 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 145.319197543, 2024-05-09T01:02:21.409 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000199581 2024-05-09T01:02:21.409 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:21.409 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:02:21.409 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1234219, 2024-05-09T01:02:21.410 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.408312965, 2024-05-09T01:02:21.410 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000330 2024-05-09T01:02:21.410 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:21.410 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 63776, 2024-05-09T01:02:21.410 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 30448 2024-05-09T01:02:21.410 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:21.410 INFO:teuthology.orchestra.run.smithi157.stdout: "cct": { 2024-05-09T01:02:21.410 INFO:teuthology.orchestra.run.smithi157.stdout: "total_workers": 1, 2024-05-09T01:02:21.410 INFO:teuthology.orchestra.run.smithi157.stdout: "unhealthy_workers": 0 2024-05-09T01:02:21.410 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:21.410 INFO:teuthology.orchestra.run.smithi157.stdout: "finisher-MDSRank": { 2024-05-09T01:02:21.410 INFO:teuthology.orchestra.run.smithi157.stdout: "queue_len": 0, 2024-05-09T01:02:21.411 INFO:teuthology.orchestra.run.smithi157.stdout: "complete_latency": { 2024-05-09T01:02:21.411 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 3274, 2024-05-09T01:02:21.411 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 695.751313758, 2024-05-09T01:02:21.411 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.212508037 2024-05-09T01:02:21.411 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-09T01:02:21.411 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:21.411 INFO:teuthology.orchestra.run.smithi157.stdout: "finisher-PurgeQueue": { 2024-05-09T01:02:21.411 INFO:teuthology.orchestra.run.smithi157.stdout: "queue_len": 0, 2024-05-09T01:02:21.411 INFO:teuthology.orchestra.run.smithi157.stdout: "complete_latency": { 2024-05-09T01:02:21.411 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1963, 2024-05-09T01:02:21.411 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.061943971, 2024-05-09T01:02:21.411 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000031555 2024-05-09T01:02:21.411 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-09T01:02:21.412 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:21.412 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": { 2024-05-09T01:02:21.412 INFO:teuthology.orchestra.run.smithi157.stdout: "request": 135, 2024-05-09T01:02:21.412 INFO:teuthology.orchestra.run.smithi157.stdout: "reply": 134, 2024-05-09T01:02:21.412 INFO:teuthology.orchestra.run.smithi157.stdout: "reply_latency": { 2024-05-09T01:02:21.412 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 134, 2024-05-09T01:02:21.412 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 387.268078127, 2024-05-09T01:02:21.412 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 2.890060284 2024-05-09T01:02:21.412 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:21.412 INFO:teuthology.orchestra.run.smithi157.stdout: "slow_reply": 0, 2024-05-09T01:02:21.412 INFO:teuthology.orchestra.run.smithi157.stdout: "forward": 1, 2024-05-09T01:02:21.412 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_fetch_complete": 0, 2024-05-09T01:02:21.412 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_fetch_keys": 0, 2024-05-09T01:02:21.413 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_commit": 149, 2024-05-09T01:02:21.413 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_split": 1, 2024-05-09T01:02:21.413 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_merge": 0, 2024-05-09T01:02:21.413 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes": 35, 2024-05-09T01:02:21.413 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_top": 7, 2024-05-09T01:02:21.413 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_bottom": 28, 2024-05-09T01:02:21.413 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_pin_tail": 0, 2024-05-09T01:02:21.413 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_pinned": 25, 2024-05-09T01:02:21.413 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_expired": 2, 2024-05-09T01:02:21.413 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_with_caps": 7, 2024-05-09T01:02:21.413 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": 7, 2024-05-09T01:02:21.413 INFO:teuthology.orchestra.run.smithi157.stdout: "subtrees": 2, 2024-05-09T01:02:21.413 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse": 214, 2024-05-09T01:02:21.413 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_hit": 192, 2024-05-09T01:02:21.414 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_forward": 0, 2024-05-09T01:02:21.414 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_discover": 0, 2024-05-09T01:02:21.414 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_dir_fetch": 0, 2024-05-09T01:02:21.414 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_remote_ino": 0, 2024-05-09T01:02:21.414 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_lock": 0, 2024-05-09T01:02:21.414 INFO:teuthology.orchestra.run.smithi157.stdout: "load_cent": 0, 2024-05-09T01:02:21.414 INFO:teuthology.orchestra.run.smithi157.stdout: "q": 0, 2024-05-09T01:02:21.414 INFO:teuthology.orchestra.run.smithi157.stdout: "exported": 1, 2024-05-09T01:02:21.414 INFO:teuthology.orchestra.run.smithi157.stdout: "exported_inodes": 2, 2024-05-09T01:02:21.414 INFO:teuthology.orchestra.run.smithi157.stdout: "imported": 1, 2024-05-09T01:02:21.414 INFO:teuthology.orchestra.run.smithi157.stdout: "imported_inodes": 0, 2024-05-09T01:02:21.414 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_dir_fetch": 0, 2024-05-09T01:02:21.414 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_backtrace_fetch": 286, 2024-05-09T01:02:21.415 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_peer_discover": 0, 2024-05-09T01:02:21.415 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rfiles": 4, 2024-05-09T01:02:21.415 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rbytes": 234, 2024-05-09T01:02:21.415 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rsnaps": 0, 2024-05-09T01:02:21.415 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_backtrace_fetch": 286, 2024-05-09T01:02:21.415 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_set_tag": 0, 2024-05-09T01:02:21.415 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T01:02:21.415 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_inotable_repaired": 0, 2024-05-09T01:02:21.415 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dir_inodes": 112, 2024-05-09T01:02:21.415 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dir_base_inodes": 11, 2024-05-09T01:02:21.415 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dirfrag_rstats": 101, 2024-05-09T01:02:21.415 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_file_inodes": 174, 2024-05-09T01:02:21.415 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_inode_file_caps": 3, 2024-05-09T01:02:21.415 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_revoke": 18, 2024-05-09T01:02:21.416 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_grant": 3206, 2024-05-09T01:02:21.416 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T01:02:21.416 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T01:02:21.416 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T01:02:21.416 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_caps": 719117, 2024-05-09T01:02:21.416 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_caps_dirty": 719029, 2024-05-09T01:02:21.416 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_cap_release": 13, 2024-05-09T01:02:21.416 INFO:teuthology.orchestra.run.smithi157.stdout: "process_request_cap_release": 63 2024-05-09T01:02:21.416 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:21.416 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_cache": { 2024-05-09T01:02:21.416 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_update": 0, 2024-05-09T01:02:21.416 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_update_receipt": 0, 2024-05-09T01:02:21.416 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_try_discover": 0, 2024-05-09T01:02:21.417 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_send_discover": 0, 2024-05-09T01:02:21.417 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_handle_discover": 6, 2024-05-09T01:02:21.417 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays": 7, 2024-05-09T01:02:21.417 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays_delayed": 0, 2024-05-09T01:02:21.417 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays_enqueuing": 0, 2024-05-09T01:02:21.417 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_created": 20, 2024-05-09T01:02:21.417 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_enqueued": 13, 2024-05-09T01:02:21.417 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_reintegrated": 0, 2024-05-09T01:02:21.417 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_migrated": 0, 2024-05-09T01:02:21.417 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_processing": 0, 2024-05-09T01:02:21.417 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_enqueued": 0, 2024-05-09T01:02:21.417 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_prioritized": 0, 2024-05-09T01:02:21.417 INFO:teuthology.orchestra.run.smithi157.stdout: "recovery_started": 0, 2024-05-09T01:02:21.418 INFO:teuthology.orchestra.run.smithi157.stdout: "recovery_completed": 0, 2024-05-09T01:02:21.418 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_quiesce_path": 0, 2024-05-09T01:02:21.418 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_quiesce_inode": 0, 2024-05-09T01:02:21.418 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_enqueue_scrub": 11, 2024-05-09T01:02:21.418 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_exportdir": 1, 2024-05-09T01:02:21.418 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_flush": 0, 2024-05-09T01:02:21.418 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_fragmentdir": 1, 2024-05-09T01:02:21.418 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_fragstats": 0, 2024-05-09T01:02:21.418 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_inodestats": 0 2024-05-09T01:02:21.418 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:21.418 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_log": { 2024-05-09T01:02:21.418 INFO:teuthology.orchestra.run.smithi157.stdout: "evlrg": 0, 2024-05-09T01:02:21.418 INFO:teuthology.orchestra.run.smithi157.stdout: "evadd": 720612, 2024-05-09T01:02:21.419 INFO:teuthology.orchestra.run.smithi157.stdout: "evex": 720109, 2024-05-09T01:02:21.419 INFO:teuthology.orchestra.run.smithi157.stdout: "evtrm": 719085, 2024-05-09T01:02:21.419 INFO:teuthology.orchestra.run.smithi157.stdout: "ev": 1527, 2024-05-09T01:02:21.419 INFO:teuthology.orchestra.run.smithi157.stdout: "evexg": 0, 2024-05-09T01:02:21.419 INFO:teuthology.orchestra.run.smithi157.stdout: "evexd": 34912, 2024-05-09T01:02:21.419 INFO:teuthology.orchestra.run.smithi157.stdout: "segadd": 847, 2024-05-09T01:02:21.419 INFO:teuthology.orchestra.run.smithi157.stdout: "segex": 845, 2024-05-09T01:02:21.419 INFO:teuthology.orchestra.run.smithi157.stdout: "segtrm": 844, 2024-05-09T01:02:21.419 INFO:teuthology.orchestra.run.smithi157.stdout: "seg": 3, 2024-05-09T01:02:21.419 INFO:teuthology.orchestra.run.smithi157.stdout: "segmjr": 1, 2024-05-09T01:02:21.419 INFO:teuthology.orchestra.run.smithi157.stdout: "segexg": 0, 2024-05-09T01:02:21.419 INFO:teuthology.orchestra.run.smithi157.stdout: "segexd": 39, 2024-05-09T01:02:21.419 INFO:teuthology.orchestra.run.smithi157.stdout: "expos": 1347372392, 2024-05-09T01:02:21.420 INFO:teuthology.orchestra.run.smithi157.stdout: "wrpos": 1350591604, 2024-05-09T01:02:21.420 INFO:teuthology.orchestra.run.smithi157.stdout: "rdpos": 0, 2024-05-09T01:02:21.420 INFO:teuthology.orchestra.run.smithi157.stdout: "jlat": { 2024-05-09T01:02:21.420 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 6976, 2024-05-09T01:02:21.420 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 3244.004177936, 2024-05-09T01:02:21.420 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.465023534 2024-05-09T01:02:21.420 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:21.420 INFO:teuthology.orchestra.run.smithi157.stdout: "replayed": 0 2024-05-09T01:02:21.420 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:21.420 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_mem": { 2024-05-09T01:02:21.420 INFO:teuthology.orchestra.run.smithi157.stdout: "ino": 31, 2024-05-09T01:02:21.420 INFO:teuthology.orchestra.run.smithi157.stdout: "ino+": 56, 2024-05-09T01:02:21.420 INFO:teuthology.orchestra.run.smithi157.stdout: "ino-": 25, 2024-05-09T01:02:21.421 INFO:teuthology.orchestra.run.smithi157.stdout: "dir": 27, 2024-05-09T01:02:21.421 INFO:teuthology.orchestra.run.smithi157.stdout: "dir+": 30, 2024-05-09T01:02:21.421 INFO:teuthology.orchestra.run.smithi157.stdout: "dir-": 3, 2024-05-09T01:02:21.421 INFO:teuthology.orchestra.run.smithi157.stdout: "dn": 34, 2024-05-09T01:02:21.421 INFO:teuthology.orchestra.run.smithi157.stdout: "dn+": 65, 2024-05-09T01:02:21.421 INFO:teuthology.orchestra.run.smithi157.stdout: "dn-": 31, 2024-05-09T01:02:21.421 INFO:teuthology.orchestra.run.smithi157.stdout: "cap": 2, 2024-05-09T01:02:21.421 INFO:teuthology.orchestra.run.smithi157.stdout: "cap+": 40, 2024-05-09T01:02:21.421 INFO:teuthology.orchestra.run.smithi157.stdout: "cap-": 38, 2024-05-09T01:02:21.421 INFO:teuthology.orchestra.run.smithi157.stdout: "rss": 162816, 2024-05-09T01:02:21.421 INFO:teuthology.orchestra.run.smithi157.stdout: "heap": 231824 2024-05-09T01:02:21.421 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:21.421 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_server": { 2024-05-09T01:02:21.421 INFO:teuthology.orchestra.run.smithi157.stdout: "dispatch_client_request": 206, 2024-05-09T01:02:21.422 INFO:teuthology.orchestra.run.smithi157.stdout: "dispatch_server_request": 3, 2024-05-09T01:02:21.422 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_request": 135, 2024-05-09T01:02:21.422 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_session": 20, 2024-05-09T01:02:21.422 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_peer_request": 8, 2024-05-09T01:02:21.422 INFO:teuthology.orchestra.run.smithi157.stdout: "req_create_latency": { 2024-05-09T01:02:21.422 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 22, 2024-05-09T01:02:21.422 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.028828485, 2024-05-09T01:02:21.422 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.001310385 2024-05-09T01:02:21.422 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:21.422 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getattr_latency": { 2024-05-09T01:02:21.422 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 22, 2024-05-09T01:02:21.422 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 381.430062113, 2024-05-09T01:02:21.422 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 17.337730096 2024-05-09T01:02:21.423 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:21.423 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getfilelock_latency": { 2024-05-09T01:02:21.423 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:21.423 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:21.423 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:21.423 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:21.423 INFO:teuthology.orchestra.run.smithi157.stdout: "req_link_latency": { 2024-05-09T01:02:21.423 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:21.423 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:21.423 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:21.423 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:21.423 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookup_latency": { 2024-05-09T01:02:21.423 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 29, 2024-05-09T01:02:21.423 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 5.668092524, 2024-05-09T01:02:21.424 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.195451466 2024-05-09T01:02:21.424 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:21.424 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookuphash_latency": { 2024-05-09T01:02:21.424 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:21.424 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:21.424 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:21.424 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:21.424 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupino_latency": { 2024-05-09T01:02:21.424 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1, 2024-05-09T01:02:21.424 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000378720, 2024-05-09T01:02:21.424 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000378720 2024-05-09T01:02:21.424 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:21.424 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupname_latency": { 2024-05-09T01:02:21.424 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:21.425 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:21.425 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:21.425 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:21.425 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupparent_latency": { 2024-05-09T01:02:21.425 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:21.425 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:21.425 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:21.425 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:21.425 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupsnap_latency": { 2024-05-09T01:02:21.425 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:21.425 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:21.425 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:21.425 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:21.425 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lssnap_latency": { 2024-05-09T01:02:21.426 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:21.426 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:21.426 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:21.426 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:21.427 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mkdir_latency": { 2024-05-09T01:02:21.427 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 10, 2024-05-09T01:02:21.427 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.018270689, 2024-05-09T01:02:21.427 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.001827068 2024-05-09T01:02:21.427 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:21.427 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mknod_latency": { 2024-05-09T01:02:21.427 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:21.427 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:21.427 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:21.427 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:21.427 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mksnap_latency": { 2024-05-09T01:02:21.427 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:21.427 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:21.427 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:21.428 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:21.428 INFO:teuthology.orchestra.run.smithi157.stdout: "req_open_latency": { 2024-05-09T01:02:21.428 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1, 2024-05-09T01:02:21.428 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000416233, 2024-05-09T01:02:21.428 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000416233 2024-05-09T01:02:21.428 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:21.428 INFO:teuthology.orchestra.run.smithi157.stdout: "req_readdir_latency": { 2024-05-09T01:02:21.428 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 19, 2024-05-09T01:02:21.428 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.022786161, 2024-05-09T01:02:21.428 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.001199271 2024-05-09T01:02:21.428 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:21.428 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rename_latency": { 2024-05-09T01:02:21.428 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 2, 2024-05-09T01:02:21.428 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.003130779, 2024-05-09T01:02:21.429 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.001565389 2024-05-09T01:02:21.429 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:21.429 INFO:teuthology.orchestra.run.smithi157.stdout: "req_renamesnap_latency": { 2024-05-09T01:02:21.429 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:21.429 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:21.429 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:21.429 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:21.429 INFO:teuthology.orchestra.run.smithi157.stdout: "req_snapdiff_latency": { 2024-05-09T01:02:21.429 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:21.429 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:21.429 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:21.429 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:21.429 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmdir_latency": { 2024-05-09T01:02:21.429 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 4, 2024-05-09T01:02:21.430 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.042143977, 2024-05-09T01:02:21.430 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.010535994 2024-05-09T01:02:21.430 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:21.430 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmsnap_latency": { 2024-05-09T01:02:21.430 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:21.430 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:21.430 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:21.430 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:21.430 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmxattr_latency": { 2024-05-09T01:02:21.430 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:21.430 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:21.430 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:21.430 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:21.431 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setattr_latency": { 2024-05-09T01:02:21.431 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1, 2024-05-09T01:02:21.431 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.001346781, 2024-05-09T01:02:21.431 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.001346781 2024-05-09T01:02:21.431 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:21.431 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setdirlayout_latency": { 2024-05-09T01:02:21.431 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:21.431 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:21.431 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:21.431 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:21.431 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setfilelock_latency": { 2024-05-09T01:02:21.431 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:21.431 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:21.431 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:21.432 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:21.432 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setlayout_latency": { 2024-05-09T01:02:21.432 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:21.432 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:21.432 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:21.432 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:21.432 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setxattr_latency": { 2024-05-09T01:02:21.432 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 7, 2024-05-09T01:02:21.432 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.022084247, 2024-05-09T01:02:21.432 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.003154892 2024-05-09T01:02:21.432 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:21.432 INFO:teuthology.orchestra.run.smithi157.stdout: "req_symlink_latency": { 2024-05-09T01:02:21.432 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:21.432 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:21.433 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:21.433 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:21.433 INFO:teuthology.orchestra.run.smithi157.stdout: "req_unlink_latency": { 2024-05-09T01:02:21.433 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 16, 2024-05-09T01:02:21.433 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.030537418, 2024-05-09T01:02:21.433 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.001908588 2024-05-09T01:02:21.433 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:21.433 INFO:teuthology.orchestra.run.smithi157.stdout: "cap_revoke_eviction": 0, 2024-05-09T01:02:21.433 INFO:teuthology.orchestra.run.smithi157.stdout: "cap_acquisition_throttle": 0, 2024-05-09T01:02:21.433 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getvxattr_latency": { 2024-05-09T01:02:21.433 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:21.433 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:21.433 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:21.433 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-09T01:02:21.434 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:21.434 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_sessions": { 2024-05-09T01:02:21.434 INFO:teuthology.orchestra.run.smithi157.stdout: "session_count": 1, 2024-05-09T01:02:21.434 INFO:teuthology.orchestra.run.smithi157.stdout: "session_add": 2, 2024-05-09T01:02:21.434 INFO:teuthology.orchestra.run.smithi157.stdout: "session_remove": 1, 2024-05-09T01:02:21.434 INFO:teuthology.orchestra.run.smithi157.stdout: "sessions_open": 1, 2024-05-09T01:02:21.434 INFO:teuthology.orchestra.run.smithi157.stdout: "sessions_stale": 0, 2024-05-09T01:02:21.434 INFO:teuthology.orchestra.run.smithi157.stdout: "total_load": 72, 2024-05-09T01:02:21.434 INFO:teuthology.orchestra.run.smithi157.stdout: "average_load": 72, 2024-05-09T01:02:21.434 INFO:teuthology.orchestra.run.smithi157.stdout: "avg_session_uptime": 910, 2024-05-09T01:02:21.434 INFO:teuthology.orchestra.run.smithi157.stdout: "mdthresh_evicted": 0 2024-05-09T01:02:21.434 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:21.434 INFO:teuthology.orchestra.run.smithi157.stdout: "mempool": { 2024-05-09T01:02:21.434 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_bytes": 0, 2024-05-09T01:02:21.435 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_items": 0, 2024-05-09T01:02:21.435 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T01:02:21.435 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_items": 0, 2024-05-09T01:02:21.435 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T01:02:21.435 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_items": 0, 2024-05-09T01:02:21.435 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T01:02:21.435 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T01:02:21.435 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T01:02:21.435 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T01:02:21.435 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T01:02:21.435 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_items": 0, 2024-05-09T01:02:21.435 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T01:02:21.435 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T01:02:21.435 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_bytes": 0, 2024-05-09T01:02:21.436 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_items": 0, 2024-05-09T01:02:21.436 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_bytes": 0, 2024-05-09T01:02:21.436 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_items": 0, 2024-05-09T01:02:21.436 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T01:02:21.436 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T01:02:21.436 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T01:02:21.436 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T01:02:21.436 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T01:02:21.436 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_items": 0, 2024-05-09T01:02:21.436 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_bytes": 0, 2024-05-09T01:02:21.436 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_items": 0, 2024-05-09T01:02:21.436 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T01:02:21.436 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T01:02:21.436 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_bytes": 0, 2024-05-09T01:02:21.437 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_items": 0, 2024-05-09T01:02:21.437 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_bytes": 0, 2024-05-09T01:02:21.437 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_items": 0, 2024-05-09T01:02:21.437 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T01:02:21.437 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_items": 0, 2024-05-09T01:02:21.437 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T01:02:21.437 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_items": 0, 2024-05-09T01:02:21.437 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_bytes": 382109, 2024-05-09T01:02:21.437 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_items": 142, 2024-05-09T01:02:21.437 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_bytes": 0, 2024-05-09T01:02:21.437 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_items": 0, 2024-05-09T01:02:21.437 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_bytes": 0, 2024-05-09T01:02:21.437 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_items": 0, 2024-05-09T01:02:21.437 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_bytes": 0, 2024-05-09T01:02:21.438 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_items": 0, 2024-05-09T01:02:21.438 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_bytes": 0, 2024-05-09T01:02:21.438 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_items": 0, 2024-05-09T01:02:21.438 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_bytes": 7736, 2024-05-09T01:02:21.438 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_items": 164, 2024-05-09T01:02:21.438 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T01:02:21.438 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_items": 0, 2024-05-09T01:02:21.438 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_bytes": 0, 2024-05-09T01:02:21.438 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_items": 0, 2024-05-09T01:02:21.438 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_bytes": 147454, 2024-05-09T01:02:21.438 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_items": 2316, 2024-05-09T01:02:21.438 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_bytes": 0, 2024-05-09T01:02:21.438 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_items": 0, 2024-05-09T01:02:21.438 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_bytes": 0, 2024-05-09T01:02:21.439 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_items": 0 2024-05-09T01:02:21.439 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:21.439 INFO:teuthology.orchestra.run.smithi157.stdout: "objecter": { 2024-05-09T01:02:21.439 INFO:teuthology.orchestra.run.smithi157.stdout: "op_active": 0, 2024-05-09T01:02:21.439 INFO:teuthology.orchestra.run.smithi157.stdout: "op_laggy": 0, 2024-05-09T01:02:21.439 INFO:teuthology.orchestra.run.smithi157.stdout: "op_send": 11443, 2024-05-09T01:02:21.439 INFO:teuthology.orchestra.run.smithi157.stdout: "op_send_bytes": 1360969167, 2024-05-09T01:02:21.439 INFO:teuthology.orchestra.run.smithi157.stdout: "op_resend": 32, 2024-05-09T01:02:21.439 INFO:teuthology.orchestra.run.smithi157.stdout: "op_reply": 11443, 2024-05-09T01:02:21.439 INFO:teuthology.orchestra.run.smithi157.stdout: "op_latency": { 2024-05-09T01:02:21.439 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 11443, 2024-05-09T01:02:21.439 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 445.327006062, 2024-05-09T01:02:21.439 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.038916980 2024-05-09T01:02:21.439 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:21.440 INFO:teuthology.orchestra.run.smithi157.stdout: "op_inflight": 0, 2024-05-09T01:02:21.440 INFO:teuthology.orchestra.run.smithi157.stdout: "oplen_avg": { 2024-05-09T01:02:21.440 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 11443, 2024-05-09T01:02:21.440 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 11819 2024-05-09T01:02:21.440 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:21.440 INFO:teuthology.orchestra.run.smithi157.stdout: "op": 11443, 2024-05-09T01:02:21.440 INFO:teuthology.orchestra.run.smithi157.stdout: "op_r": 315, 2024-05-09T01:02:21.440 INFO:teuthology.orchestra.run.smithi157.stdout: "op_w": 11128, 2024-05-09T01:02:21.440 INFO:teuthology.orchestra.run.smithi157.stdout: "op_rmw": 0, 2024-05-09T01:02:21.440 INFO:teuthology.orchestra.run.smithi157.stdout: "op_pg": 0, 2024-05-09T01:02:21.440 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_stat": 119, 2024-05-09T01:02:21.440 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_create": 44, 2024-05-09T01:02:21.440 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_read": 18, 2024-05-09T01:02:21.440 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_write": 6983, 2024-05-09T01:02:21.441 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_writefull": 191, 2024-05-09T01:02:21.441 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_writesame": 0, 2024-05-09T01:02:21.441 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_append": 0, 2024-05-09T01:02:21.441 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_zero": 0, 2024-05-09T01:02:21.441 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_truncate": 1, 2024-05-09T01:02:21.441 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_delete": 3214, 2024-05-09T01:02:21.441 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_mapext": 0, 2024-05-09T01:02:21.441 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_sparse_read": 0, 2024-05-09T01:02:21.441 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_clonerange": 0, 2024-05-09T01:02:21.441 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_getxattr": 297, 2024-05-09T01:02:21.441 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_setxattr": 88, 2024-05-09T01:02:21.441 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_cmpxattr": 0, 2024-05-09T01:02:21.441 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_rmxattr": 0, 2024-05-09T01:02:21.441 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_resetxattrs": 0, 2024-05-09T01:02:21.442 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_call": 0, 2024-05-09T01:02:21.442 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_watch": 0, 2024-05-09T01:02:21.442 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_notify": 0, 2024-05-09T01:02:21.442 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T01:02:21.442 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_pgls": 0, 2024-05-09T01:02:21.442 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_pgls_filter": 0, 2024-05-09T01:02:21.442 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_other": 0, 2024-05-09T01:02:21.442 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_active": 0, 2024-05-09T01:02:21.442 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_send": 0, 2024-05-09T01:02:21.442 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_resend": 0, 2024-05-09T01:02:21.442 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_ping": 0, 2024-05-09T01:02:21.442 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_active": 0, 2024-05-09T01:02:21.442 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_send": 0, 2024-05-09T01:02:21.442 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_resend": 0, 2024-05-09T01:02:21.443 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_active": 0, 2024-05-09T01:02:21.443 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_send": 0, 2024-05-09T01:02:21.443 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_resend": 0, 2024-05-09T01:02:21.443 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_active": 0, 2024-05-09T01:02:21.443 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_send": 0, 2024-05-09T01:02:21.443 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_resend": 0, 2024-05-09T01:02:21.443 INFO:teuthology.orchestra.run.smithi157.stdout: "command_active": 0, 2024-05-09T01:02:21.443 INFO:teuthology.orchestra.run.smithi157.stdout: "command_send": 0, 2024-05-09T01:02:21.443 INFO:teuthology.orchestra.run.smithi157.stdout: "command_resend": 0, 2024-05-09T01:02:21.443 INFO:teuthology.orchestra.run.smithi157.stdout: "map_epoch": 82, 2024-05-09T01:02:21.443 INFO:teuthology.orchestra.run.smithi157.stdout: "map_full": 0, 2024-05-09T01:02:21.443 INFO:teuthology.orchestra.run.smithi157.stdout: "map_inc": 4, 2024-05-09T01:02:21.443 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_sessions": 12, 2024-05-09T01:02:21.443 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_session_open": 12, 2024-05-09T01:02:21.444 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_session_close": 0, 2024-05-09T01:02:21.444 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_laggy": 0, 2024-05-09T01:02:21.444 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_wr": 844, 2024-05-09T01:02:21.444 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_rd": 0, 2024-05-09T01:02:21.444 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_del": 20 2024-05-09T01:02:21.444 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:21.444 INFO:teuthology.orchestra.run.smithi157.stdout: "oft": { 2024-05-09T01:02:21.444 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_objs": 1, 2024-05-09T01:02:21.444 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_kv_pairs": 0, 2024-05-09T01:02:21.444 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_updates": 0, 2024-05-09T01:02:21.444 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_removes": 0 2024-05-09T01:02:21.444 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:21.444 INFO:teuthology.orchestra.run.smithi157.stdout: "osd-slow-ops": { 2024-05-09T01:02:21.444 INFO:teuthology.orchestra.run.smithi157.stdout: "slow_ops_count": 6 2024-05-09T01:02:21.445 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:21.445 INFO:teuthology.orchestra.run.smithi157.stdout: "purge_queue": { 2024-05-09T01:02:21.445 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_ops": 0, 2024-05-09T01:02:21.445 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_ops_high_water": 514, 2024-05-09T01:02:21.445 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing": 0, 2024-05-09T01:02:21.445 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_high_water": 6, 2024-05-09T01:02:21.445 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executed_ops": 2061, 2024-05-09T01:02:21.445 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executed": 13, 2024-05-09T01:02:21.445 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_item_in_journal": 0 2024-05-09T01:02:21.445 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:21.445 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T01:02:21.445 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-05-09T01:02:21.445 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2024-05-09T01:02:21.446 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-05-09T01:02:21.446 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 736287, 2024-05-09T01:02:21.446 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 190779134, 2024-05-09T01:02:21.446 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-05-09T01:02:21.446 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 736287, 2024-05-09T01:02:21.446 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-05-09T01:02:21.446 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-05-09T01:02:21.446 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 736287, 2024-05-09T01:02:21.446 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 190779134, 2024-05-09T01:02:21.446 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-05-09T01:02:21.446 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:21.446 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:21.446 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:21.446 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-09T01:02:21.447 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:21.447 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-objecter_bytes": { 2024-05-09T01:02:21.447 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-05-09T01:02:21.447 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2024-05-09T01:02:21.447 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-05-09T01:02:21.447 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-05-09T01:02:21.447 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-05-09T01:02:21.447 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-05-09T01:02:21.447 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-05-09T01:02:21.447 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 11443, 2024-05-09T01:02:21.447 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 1360970480, 2024-05-09T01:02:21.447 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 11443, 2024-05-09T01:02:21.447 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 1360970480, 2024-05-09T01:02:21.447 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-05-09T01:02:21.448 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:21.448 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:21.448 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:21.448 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-09T01:02:21.448 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:21.448 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-objecter_ops": { 2024-05-09T01:02:21.448 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-05-09T01:02:21.448 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 1024, 2024-05-09T01:02:21.448 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-05-09T01:02:21.448 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-05-09T01:02:21.448 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-05-09T01:02:21.448 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-05-09T01:02:21.448 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-05-09T01:02:21.449 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 11443, 2024-05-09T01:02:21.449 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 11443, 2024-05-09T01:02:21.449 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 11443, 2024-05-09T01:02:21.449 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 11443, 2024-05-09T01:02:21.449 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-05-09T01:02:21.449 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:21.449 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:21.449 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:21.449 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-09T01:02:21.449 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:21.449 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-write_buf_throttle": { 2024-05-09T01:02:21.449 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-05-09T01:02:21.449 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 3758096384, 2024-05-09T01:02:21.449 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-05-09T01:02:21.450 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 13, 2024-05-09T01:02:21.450 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 1313, 2024-05-09T01:02:21.450 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-05-09T01:02:21.450 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 13, 2024-05-09T01:02:21.450 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-05-09T01:02:21.450 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-05-09T01:02:21.450 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 7, 2024-05-09T01:02:21.450 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 1313, 2024-05-09T01:02:21.450 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-05-09T01:02:21.450 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:21.450 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:21.450 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:21.450 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-09T01:02:21.450 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:21.451 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-write_buf_throttle-0x55717bb6c680": { 2024-05-09T01:02:21.451 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 11493, 2024-05-09T01:02:21.451 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 3758096384, 2024-05-09T01:02:21.451 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-05-09T01:02:21.451 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 720612, 2024-05-09T01:02:21.451 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 1346397300, 2024-05-09T01:02:21.451 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-05-09T01:02:21.451 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 720612, 2024-05-09T01:02:21.451 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-05-09T01:02:21.451 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-05-09T01:02:21.451 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 6976, 2024-05-09T01:02:21.451 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 1346385807, 2024-05-09T01:02:21.451 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-05-09T01:02:21.451 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:21.452 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:21.452 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:21.452 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-09T01:02:21.452 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-09T01:02:21.452 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-05-09T01:02:21.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:21 smithi157 ceph-mon[34294]: pgmap v851: 97 pgs: 97 active+clean; 2.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 51 MiB/s wr, 665 op/s 2024-05-09T01:02:22.040 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:21 smithi005 ceph-mon[30365]: pgmap v851: 97 pgs: 97 active+clean; 2.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 51 MiB/s wr, 665 op/s 2024-05-09T01:02:22.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:21 smithi131 ceph-mon[34547]: pgmap v851: 97 pgs: 97 active+clean; 2.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 51 MiB/s wr, 665 op/s 2024-05-09T01:02:22.222 INFO:tasks.check_counter:Daemon mds.i mds.exported=1 2024-05-09T01:02:22.222 INFO:tasks.check_counter:Daemon mds.i mds.imported=1 2024-05-09T01:02:22.222 DEBUG:tasks.check_counter:Getting stats from l 2024-05-09T01:02:22.222 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph daemon mds.l perf dump 2024-05-09T01:02:22.589 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T01:02:23.811 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:23 smithi157 ceph-mon[34294]: pgmap v852: 97 pgs: 97 active+clean; 2.0 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 40 MiB/s wr, 585 op/s 2024-05-09T01:02:24.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:23 smithi005 ceph-mon[30365]: pgmap v852: 97 pgs: 97 active+clean; 2.0 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 40 MiB/s wr, 585 op/s 2024-05-09T01:02:24.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:23 smithi131 ceph-mon[34547]: pgmap v852: 97 pgs: 97 active+clean; 2.0 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 40 MiB/s wr, 585 op/s 2024-05-09T01:02:24.401 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-05-09T01:02:24.401 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T01:02:24.401 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 2751, 2024-05-09T01:02:24.402 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 2750, 2024-05-09T01:02:24.402 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 487427935, 2024-05-09T01:02:24.402 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 873686, 2024-05-09T01:02:24.402 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 8, 2024-05-09T01:02:24.402 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 4, 2024-05-09T01:02:24.402 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.626512312, 2024-05-09T01:02:24.402 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.074686052, 2024-05-09T01:02:24.402 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.418833746, 2024-05-09T01:02:24.402 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.107217616, 2024-05-09T01:02:24.402 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:02:24.402 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 2749, 2024-05-09T01:02:24.402 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.130388906, 2024-05-09T01:02:24.403 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000047431 2024-05-09T01:02:24.403 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:24.403 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:02:24.403 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:24.403 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:24.403 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:24.403 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:24.403 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-05-09T01:02:24.403 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 448 2024-05-09T01:02:24.403 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:24.403 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T01:02:24.403 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 1201, 2024-05-09T01:02:24.404 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 1437, 2024-05-09T01:02:24.404 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 424015207, 2024-05-09T01:02:24.404 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 866706, 2024-05-09T01:02:24.404 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 7, 2024-05-09T01:02:24.404 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 6, 2024-05-09T01:02:24.404 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.426294339, 2024-05-09T01:02:24.404 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.040330068, 2024-05-09T01:02:24.404 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.326830221, 2024-05-09T01:02:24.404 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.044557828, 2024-05-09T01:02:24.404 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:02:24.404 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1435, 2024-05-09T01:02:24.404 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.082672662, 2024-05-09T01:02:24.404 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000057611 2024-05-09T01:02:24.404 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:24.405 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:02:24.405 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:24.405 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:24.405 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:24.405 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:24.405 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-09T01:02:24.405 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 485040 2024-05-09T01:02:24.405 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:24.405 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T01:02:24.405 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 1930, 2024-05-09T01:02:24.405 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 1920, 2024-05-09T01:02:24.405 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 437322680, 2024-05-09T01:02:24.405 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 702421, 2024-05-09T01:02:24.406 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 5, 2024-05-09T01:02:24.406 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 4, 2024-05-09T01:02:24.406 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.570281570, 2024-05-09T01:02:24.406 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.063706364, 2024-05-09T01:02:24.406 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.416183683, 2024-05-09T01:02:24.406 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.071376588, 2024-05-09T01:02:24.406 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T01:02:24.406 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1919, 2024-05-09T01:02:24.406 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.136145458, 2024-05-09T01:02:24.406 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000070946 2024-05-09T01:02:24.406 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:24.406 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-05-09T01:02:24.407 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:24.407 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:24.407 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:24.407 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:24.407 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 605872, 2024-05-09T01:02:24.407 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 194256 2024-05-09T01:02:24.407 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:24.407 INFO:teuthology.orchestra.run.smithi157.stdout: "cct": { 2024-05-09T01:02:24.407 INFO:teuthology.orchestra.run.smithi157.stdout: "total_workers": 1, 2024-05-09T01:02:24.407 INFO:teuthology.orchestra.run.smithi157.stdout: "unhealthy_workers": 0 2024-05-09T01:02:24.407 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:24.407 INFO:teuthology.orchestra.run.smithi157.stdout: "finisher-MDSRank": { 2024-05-09T01:02:24.407 INFO:teuthology.orchestra.run.smithi157.stdout: "queue_len": 0, 2024-05-09T01:02:24.408 INFO:teuthology.orchestra.run.smithi157.stdout: "complete_latency": { 2024-05-09T01:02:24.408 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 8064, 2024-05-09T01:02:24.408 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.267355946, 2024-05-09T01:02:24.408 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000033154 2024-05-09T01:02:24.408 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-09T01:02:24.408 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:24.408 INFO:teuthology.orchestra.run.smithi157.stdout: "finisher-PurgeQueue": { 2024-05-09T01:02:24.408 INFO:teuthology.orchestra.run.smithi157.stdout: "queue_len": 0, 2024-05-09T01:02:24.408 INFO:teuthology.orchestra.run.smithi157.stdout: "complete_latency": { 2024-05-09T01:02:24.408 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:24.408 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:24.408 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:24.408 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-09T01:02:24.409 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:24.409 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": { 2024-05-09T01:02:24.409 INFO:teuthology.orchestra.run.smithi157.stdout: "request": 0, 2024-05-09T01:02:24.409 INFO:teuthology.orchestra.run.smithi157.stdout: "reply": 0, 2024-05-09T01:02:24.409 INFO:teuthology.orchestra.run.smithi157.stdout: "reply_latency": { 2024-05-09T01:02:24.409 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:24.409 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:24.410 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:24.410 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:24.410 INFO:teuthology.orchestra.run.smithi157.stdout: "slow_reply": 0, 2024-05-09T01:02:24.410 INFO:teuthology.orchestra.run.smithi157.stdout: "forward": 0, 2024-05-09T01:02:24.410 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_fetch_complete": 0, 2024-05-09T01:02:24.410 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_fetch_keys": 0, 2024-05-09T01:02:24.411 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_commit": 0, 2024-05-09T01:02:24.411 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_split": 0, 2024-05-09T01:02:24.411 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_merge": 0, 2024-05-09T01:02:24.411 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes": 52, 2024-05-09T01:02:24.411 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_top": 4, 2024-05-09T01:02:24.411 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_bottom": 48, 2024-05-09T01:02:24.412 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_pin_tail": 0, 2024-05-09T01:02:24.412 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_pinned": 45, 2024-05-09T01:02:24.412 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_expired": 0, 2024-05-09T01:02:24.412 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_with_caps": 0, 2024-05-09T01:02:24.412 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": 0, 2024-05-09T01:02:24.413 INFO:teuthology.orchestra.run.smithi157.stdout: "subtrees": 2, 2024-05-09T01:02:24.413 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse": 0, 2024-05-09T01:02:24.413 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_hit": 0, 2024-05-09T01:02:24.413 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_forward": 0, 2024-05-09T01:02:24.413 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_discover": 0, 2024-05-09T01:02:24.413 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_dir_fetch": 0, 2024-05-09T01:02:24.414 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_remote_ino": 0, 2024-05-09T01:02:24.414 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_lock": 0, 2024-05-09T01:02:24.414 INFO:teuthology.orchestra.run.smithi157.stdout: "load_cent": 0, 2024-05-09T01:02:24.414 INFO:teuthology.orchestra.run.smithi157.stdout: "q": 0, 2024-05-09T01:02:24.414 INFO:teuthology.orchestra.run.smithi157.stdout: "exported": 0, 2024-05-09T01:02:24.415 INFO:teuthology.orchestra.run.smithi157.stdout: "exported_inodes": 0, 2024-05-09T01:02:24.415 INFO:teuthology.orchestra.run.smithi157.stdout: "imported": 0, 2024-05-09T01:02:24.415 INFO:teuthology.orchestra.run.smithi157.stdout: "imported_inodes": 0, 2024-05-09T01:02:24.415 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_dir_fetch": 0, 2024-05-09T01:02:24.415 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_backtrace_fetch": 0, 2024-05-09T01:02:24.415 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_peer_discover": 0, 2024-05-09T01:02:24.416 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rfiles": 4, 2024-05-09T01:02:24.416 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rbytes": 234, 2024-05-09T01:02:24.416 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rsnaps": 0, 2024-05-09T01:02:24.416 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_backtrace_fetch": 0, 2024-05-09T01:02:24.416 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_set_tag": 0, 2024-05-09T01:02:24.417 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T01:02:24.417 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_inotable_repaired": 0, 2024-05-09T01:02:24.417 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dir_inodes": 0, 2024-05-09T01:02:24.417 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T01:02:24.417 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T01:02:24.417 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_file_inodes": 0, 2024-05-09T01:02:24.418 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_inode_file_caps": 0, 2024-05-09T01:02:24.418 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_revoke": 0, 2024-05-09T01:02:24.418 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_grant": 0, 2024-05-09T01:02:24.418 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T01:02:24.418 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T01:02:24.419 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T01:02:24.419 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_caps": 0, 2024-05-09T01:02:24.419 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_caps_dirty": 0, 2024-05-09T01:02:24.419 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_cap_release": 0, 2024-05-09T01:02:24.419 INFO:teuthology.orchestra.run.smithi157.stdout: "process_request_cap_release": 0 2024-05-09T01:02:24.420 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:24.420 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_cache": { 2024-05-09T01:02:24.420 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_update": 0, 2024-05-09T01:02:24.420 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_update_receipt": 0, 2024-05-09T01:02:24.420 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_try_discover": 0, 2024-05-09T01:02:24.420 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_send_discover": 0, 2024-05-09T01:02:24.420 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_handle_discover": 0, 2024-05-09T01:02:24.421 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays": 0, 2024-05-09T01:02:24.421 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays_delayed": 0, 2024-05-09T01:02:24.421 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays_enqueuing": 0, 2024-05-09T01:02:24.421 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_created": 0, 2024-05-09T01:02:24.421 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_enqueued": 0, 2024-05-09T01:02:24.421 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_reintegrated": 0, 2024-05-09T01:02:24.422 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_migrated": 0, 2024-05-09T01:02:24.422 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_processing": 0, 2024-05-09T01:02:24.422 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_enqueued": 0, 2024-05-09T01:02:24.422 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_prioritized": 0, 2024-05-09T01:02:24.422 INFO:teuthology.orchestra.run.smithi157.stdout: "recovery_started": 0, 2024-05-09T01:02:24.422 INFO:teuthology.orchestra.run.smithi157.stdout: "recovery_completed": 0, 2024-05-09T01:02:24.423 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_quiesce_path": 0, 2024-05-09T01:02:24.423 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_quiesce_inode": 0, 2024-05-09T01:02:24.423 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T01:02:24.423 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_exportdir": 0, 2024-05-09T01:02:24.423 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_flush": 0, 2024-05-09T01:02:24.423 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_fragmentdir": 0, 2024-05-09T01:02:24.424 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_fragstats": 0, 2024-05-09T01:02:24.424 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_inodestats": 0 2024-05-09T01:02:24.424 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:24.424 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_log": { 2024-05-09T01:02:24.424 INFO:teuthology.orchestra.run.smithi157.stdout: "evlrg": 0, 2024-05-09T01:02:24.424 INFO:teuthology.orchestra.run.smithi157.stdout: "evadd": 0, 2024-05-09T01:02:24.424 INFO:teuthology.orchestra.run.smithi157.stdout: "evex": 0, 2024-05-09T01:02:24.425 INFO:teuthology.orchestra.run.smithi157.stdout: "evtrm": 0, 2024-05-09T01:02:24.425 INFO:teuthology.orchestra.run.smithi157.stdout: "ev": 720612, 2024-05-09T01:02:24.425 INFO:teuthology.orchestra.run.smithi157.stdout: "evexg": 0, 2024-05-09T01:02:24.425 INFO:teuthology.orchestra.run.smithi157.stdout: "evexd": 0, 2024-05-09T01:02:24.425 INFO:teuthology.orchestra.run.smithi157.stdout: "segadd": 0, 2024-05-09T01:02:24.425 INFO:teuthology.orchestra.run.smithi157.stdout: "segex": 0, 2024-05-09T01:02:24.426 INFO:teuthology.orchestra.run.smithi157.stdout: "segtrm": 0, 2024-05-09T01:02:24.426 INFO:teuthology.orchestra.run.smithi157.stdout: "seg": 3, 2024-05-09T01:02:24.426 INFO:teuthology.orchestra.run.smithi157.stdout: "segmjr": 82, 2024-05-09T01:02:24.426 INFO:teuthology.orchestra.run.smithi157.stdout: "segexg": 0, 2024-05-09T01:02:24.426 INFO:teuthology.orchestra.run.smithi157.stdout: "segexd": 0, 2024-05-09T01:02:24.426 INFO:teuthology.orchestra.run.smithi157.stdout: "expos": 1347372392, 2024-05-09T01:02:24.426 INFO:teuthology.orchestra.run.smithi157.stdout: "wrpos": 1350591604, 2024-05-09T01:02:24.426 INFO:teuthology.orchestra.run.smithi157.stdout: "rdpos": 1350589543, 2024-05-09T01:02:24.427 INFO:teuthology.orchestra.run.smithi157.stdout: "jlat": { 2024-05-09T01:02:24.427 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:24.427 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:24.427 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:24.427 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:24.427 INFO:teuthology.orchestra.run.smithi157.stdout: "replayed": 720612 2024-05-09T01:02:24.427 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:24.428 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_mem": { 2024-05-09T01:02:24.428 INFO:teuthology.orchestra.run.smithi157.stdout: "ino": 21, 2024-05-09T01:02:24.428 INFO:teuthology.orchestra.run.smithi157.stdout: "ino+": 36, 2024-05-09T01:02:24.428 INFO:teuthology.orchestra.run.smithi157.stdout: "ino-": 15, 2024-05-09T01:02:24.428 INFO:teuthology.orchestra.run.smithi157.stdout: "dir": 18, 2024-05-09T01:02:24.428 INFO:teuthology.orchestra.run.smithi157.stdout: "dir+": 21, 2024-05-09T01:02:24.428 INFO:teuthology.orchestra.run.smithi157.stdout: "dir-": 3, 2024-05-09T01:02:24.428 INFO:teuthology.orchestra.run.smithi157.stdout: "dn": 52, 2024-05-09T01:02:24.429 INFO:teuthology.orchestra.run.smithi157.stdout: "dn+": 55, 2024-05-09T01:02:24.429 INFO:teuthology.orchestra.run.smithi157.stdout: "dn-": 3, 2024-05-09T01:02:24.429 INFO:teuthology.orchestra.run.smithi157.stdout: "cap": 0, 2024-05-09T01:02:24.429 INFO:teuthology.orchestra.run.smithi157.stdout: "cap+": 0, 2024-05-09T01:02:24.429 INFO:teuthology.orchestra.run.smithi157.stdout: "cap-": 0, 2024-05-09T01:02:24.429 INFO:teuthology.orchestra.run.smithi157.stdout: "rss": 124780, 2024-05-09T01:02:24.429 INFO:teuthology.orchestra.run.smithi157.stdout: "heap": 215440 2024-05-09T01:02:24.430 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:24.430 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_server": { 2024-05-09T01:02:24.430 INFO:teuthology.orchestra.run.smithi157.stdout: "dispatch_client_request": 0, 2024-05-09T01:02:24.430 INFO:teuthology.orchestra.run.smithi157.stdout: "dispatch_server_request": 0, 2024-05-09T01:02:24.430 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_request": 0, 2024-05-09T01:02:24.430 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_session": 0, 2024-05-09T01:02:24.430 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_peer_request": 0, 2024-05-09T01:02:24.431 INFO:teuthology.orchestra.run.smithi157.stdout: "req_create_latency": { 2024-05-09T01:02:24.431 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:24.431 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:24.431 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:24.431 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:24.432 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getattr_latency": { 2024-05-09T01:02:24.432 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:24.432 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:24.432 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:24.432 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:24.432 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getfilelock_latency": { 2024-05-09T01:02:24.433 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:24.433 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:24.433 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:24.433 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:24.433 INFO:teuthology.orchestra.run.smithi157.stdout: "req_link_latency": { 2024-05-09T01:02:24.433 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:24.433 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:24.433 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:24.433 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:24.433 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookup_latency": { 2024-05-09T01:02:24.434 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:24.434 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:24.434 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:24.434 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:24.434 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookuphash_latency": { 2024-05-09T01:02:24.434 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:24.434 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:24.434 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:24.434 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:24.434 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupino_latency": { 2024-05-09T01:02:24.434 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:24.434 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:24.435 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:24.435 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:24.435 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupname_latency": { 2024-05-09T01:02:24.435 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:24.435 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:24.435 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:24.435 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:24.435 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupparent_latency": { 2024-05-09T01:02:24.435 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:24.435 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:24.435 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:24.435 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:24.435 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupsnap_latency": { 2024-05-09T01:02:24.435 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:24.436 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:24.436 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:24.436 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:24.436 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lssnap_latency": { 2024-05-09T01:02:24.436 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:24.436 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:24.436 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:24.436 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:24.437 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mkdir_latency": { 2024-05-09T01:02:24.437 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:24.437 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:24.437 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:24.437 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:24.437 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mknod_latency": { 2024-05-09T01:02:24.437 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:24.437 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:24.437 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:24.438 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:24.438 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mksnap_latency": { 2024-05-09T01:02:24.438 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:24.438 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:24.438 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:24.438 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:24.438 INFO:teuthology.orchestra.run.smithi157.stdout: "req_open_latency": { 2024-05-09T01:02:24.438 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:24.438 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:24.438 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:24.438 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:24.438 INFO:teuthology.orchestra.run.smithi157.stdout: "req_readdir_latency": { 2024-05-09T01:02:24.438 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:24.439 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:24.439 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:24.439 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:24.439 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rename_latency": { 2024-05-09T01:02:24.439 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:24.439 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:24.439 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:24.439 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:24.439 INFO:teuthology.orchestra.run.smithi157.stdout: "req_renamesnap_latency": { 2024-05-09T01:02:24.439 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:24.439 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:24.439 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:24.439 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:24.440 INFO:teuthology.orchestra.run.smithi157.stdout: "req_snapdiff_latency": { 2024-05-09T01:02:24.440 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:24.440 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:24.440 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:24.440 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:24.440 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmdir_latency": { 2024-05-09T01:02:24.440 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:24.440 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:24.440 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:24.440 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:24.440 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmsnap_latency": { 2024-05-09T01:02:24.441 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:24.441 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:24.441 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:24.441 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:24.441 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmxattr_latency": { 2024-05-09T01:02:24.441 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:24.441 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:24.441 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:24.441 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:24.441 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setattr_latency": { 2024-05-09T01:02:24.441 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:24.442 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:24.442 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:24.442 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:24.442 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setdirlayout_latency": { 2024-05-09T01:02:24.442 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:24.442 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:24.442 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:24.442 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:24.442 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setfilelock_latency": { 2024-05-09T01:02:24.442 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:24.442 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:24.443 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:24.443 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:24.443 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setlayout_latency": { 2024-05-09T01:02:24.443 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:24.443 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:24.443 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:24.443 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:24.443 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setxattr_latency": { 2024-05-09T01:02:24.443 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:24.444 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:24.444 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:24.444 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:24.444 INFO:teuthology.orchestra.run.smithi157.stdout: "req_symlink_latency": { 2024-05-09T01:02:24.444 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:24.444 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:24.445 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:24.445 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:24.445 INFO:teuthology.orchestra.run.smithi157.stdout: "req_unlink_latency": { 2024-05-09T01:02:24.445 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:24.445 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:24.445 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:24.446 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:24.446 INFO:teuthology.orchestra.run.smithi157.stdout: "cap_revoke_eviction": 0, 2024-05-09T01:02:24.446 INFO:teuthology.orchestra.run.smithi157.stdout: "cap_acquisition_throttle": 0, 2024-05-09T01:02:24.446 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getvxattr_latency": { 2024-05-09T01:02:24.446 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:24.447 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:24.447 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:24.447 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-09T01:02:24.447 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:24.447 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_sessions": { 2024-05-09T01:02:24.448 INFO:teuthology.orchestra.run.smithi157.stdout: "session_count": 1, 2024-05-09T01:02:24.448 INFO:teuthology.orchestra.run.smithi157.stdout: "session_add": 2, 2024-05-09T01:02:24.448 INFO:teuthology.orchestra.run.smithi157.stdout: "session_remove": 1, 2024-05-09T01:02:24.448 INFO:teuthology.orchestra.run.smithi157.stdout: "sessions_open": 1, 2024-05-09T01:02:24.448 INFO:teuthology.orchestra.run.smithi157.stdout: "sessions_stale": 0, 2024-05-09T01:02:24.448 INFO:teuthology.orchestra.run.smithi157.stdout: "total_load": 0, 2024-05-09T01:02:24.449 INFO:teuthology.orchestra.run.smithi157.stdout: "average_load": 0, 2024-05-09T01:02:24.449 INFO:teuthology.orchestra.run.smithi157.stdout: "avg_session_uptime": 2342, 2024-05-09T01:02:24.449 INFO:teuthology.orchestra.run.smithi157.stdout: "mdthresh_evicted": 0 2024-05-09T01:02:24.449 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:24.449 INFO:teuthology.orchestra.run.smithi157.stdout: "mempool": { 2024-05-09T01:02:24.450 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_bytes": 0, 2024-05-09T01:02:24.450 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_items": 0, 2024-05-09T01:02:24.450 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T01:02:24.450 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_items": 0, 2024-05-09T01:02:24.450 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T01:02:24.450 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_items": 0, 2024-05-09T01:02:24.451 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T01:02:24.451 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T01:02:24.451 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T01:02:24.451 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T01:02:24.451 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T01:02:24.452 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_items": 0, 2024-05-09T01:02:24.452 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T01:02:24.452 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T01:02:24.452 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_bytes": 0, 2024-05-09T01:02:24.452 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_items": 0, 2024-05-09T01:02:24.452 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_bytes": 0, 2024-05-09T01:02:24.453 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_items": 0, 2024-05-09T01:02:24.453 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T01:02:24.453 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T01:02:24.453 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T01:02:24.453 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T01:02:24.453 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T01:02:24.453 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_items": 0, 2024-05-09T01:02:24.454 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_bytes": 0, 2024-05-09T01:02:24.454 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_items": 0, 2024-05-09T01:02:24.454 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T01:02:24.454 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T01:02:24.454 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_bytes": 0, 2024-05-09T01:02:24.454 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_items": 0, 2024-05-09T01:02:24.454 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_bytes": 0, 2024-05-09T01:02:24.455 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_items": 0, 2024-05-09T01:02:24.455 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T01:02:24.455 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_items": 0, 2024-05-09T01:02:24.455 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T01:02:24.455 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_items": 0, 2024-05-09T01:02:24.455 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_bytes": 238960, 2024-05-09T01:02:24.455 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_items": 85, 2024-05-09T01:02:24.455 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_bytes": 0, 2024-05-09T01:02:24.456 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_items": 0, 2024-05-09T01:02:24.456 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_bytes": 0, 2024-05-09T01:02:24.456 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_items": 0, 2024-05-09T01:02:24.456 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_bytes": 0, 2024-05-09T01:02:24.456 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_items": 0, 2024-05-09T01:02:24.456 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_bytes": 0, 2024-05-09T01:02:24.456 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_items": 0, 2024-05-09T01:02:24.457 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_bytes": 7736, 2024-05-09T01:02:24.457 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_items": 164, 2024-05-09T01:02:24.457 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T01:02:24.457 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_items": 0, 2024-05-09T01:02:24.457 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_bytes": 0, 2024-05-09T01:02:24.457 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_items": 0, 2024-05-09T01:02:24.458 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_bytes": 111436, 2024-05-09T01:02:24.458 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_items": 1339, 2024-05-09T01:02:24.458 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_bytes": 0, 2024-05-09T01:02:24.458 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_items": 0, 2024-05-09T01:02:24.458 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_bytes": 0, 2024-05-09T01:02:24.459 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_items": 0 2024-05-09T01:02:24.459 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:24.459 INFO:teuthology.orchestra.run.smithi157.stdout: "objecter": { 2024-05-09T01:02:24.459 INFO:teuthology.orchestra.run.smithi157.stdout: "op_active": 0, 2024-05-09T01:02:24.459 INFO:teuthology.orchestra.run.smithi157.stdout: "op_laggy": 0, 2024-05-09T01:02:24.459 INFO:teuthology.orchestra.run.smithi157.stdout: "op_send": 5544, 2024-05-09T01:02:24.460 INFO:teuthology.orchestra.run.smithi157.stdout: "op_send_bytes": 26, 2024-05-09T01:02:24.460 INFO:teuthology.orchestra.run.smithi157.stdout: "op_resend": 4, 2024-05-09T01:02:24.460 INFO:teuthology.orchestra.run.smithi157.stdout: "op_reply": 5544, 2024-05-09T01:02:24.460 INFO:teuthology.orchestra.run.smithi157.stdout: "op_latency": { 2024-05-09T01:02:24.460 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 5544, 2024-05-09T01:02:24.461 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 26.755519066, 2024-05-09T01:02:24.461 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.004826031 2024-05-09T01:02:24.461 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:24.461 INFO:teuthology.orchestra.run.smithi157.stdout: "op_inflight": 0, 2024-05-09T01:02:24.461 INFO:teuthology.orchestra.run.smithi157.stdout: "oplen_avg": { 2024-05-09T01:02:24.461 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 5544, 2024-05-09T01:02:24.462 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 5545 2024-05-09T01:02:24.462 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:24.462 INFO:teuthology.orchestra.run.smithi157.stdout: "op": 5544, 2024-05-09T01:02:24.462 INFO:teuthology.orchestra.run.smithi157.stdout: "op_r": 5544, 2024-05-09T01:02:24.462 INFO:teuthology.orchestra.run.smithi157.stdout: "op_w": 0, 2024-05-09T01:02:24.463 INFO:teuthology.orchestra.run.smithi157.stdout: "op_rmw": 0, 2024-05-09T01:02:24.463 INFO:teuthology.orchestra.run.smithi157.stdout: "op_pg": 0, 2024-05-09T01:02:24.463 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_stat": 3593, 2024-05-09T01:02:24.463 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_create": 0, 2024-05-09T01:02:24.463 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_read": 1948, 2024-05-09T01:02:24.464 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_write": 0, 2024-05-09T01:02:24.464 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_writefull": 0, 2024-05-09T01:02:24.464 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_writesame": 0, 2024-05-09T01:02:24.464 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_append": 0, 2024-05-09T01:02:24.464 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_zero": 0, 2024-05-09T01:02:24.464 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_truncate": 0, 2024-05-09T01:02:24.465 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_delete": 0, 2024-05-09T01:02:24.465 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_mapext": 0, 2024-05-09T01:02:24.465 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_sparse_read": 0, 2024-05-09T01:02:24.465 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_clonerange": 0, 2024-05-09T01:02:24.465 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_getxattr": 2, 2024-05-09T01:02:24.465 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_setxattr": 0, 2024-05-09T01:02:24.465 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_cmpxattr": 0, 2024-05-09T01:02:24.466 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_rmxattr": 0, 2024-05-09T01:02:24.466 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_resetxattrs": 0, 2024-05-09T01:02:24.466 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_call": 0, 2024-05-09T01:02:24.466 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_watch": 0, 2024-05-09T01:02:24.466 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_notify": 0, 2024-05-09T01:02:24.466 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T01:02:24.467 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_pgls": 0, 2024-05-09T01:02:24.467 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_pgls_filter": 0, 2024-05-09T01:02:24.467 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_other": 0, 2024-05-09T01:02:24.467 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_active": 0, 2024-05-09T01:02:24.467 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_send": 0, 2024-05-09T01:02:24.467 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_resend": 0, 2024-05-09T01:02:24.467 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_ping": 0, 2024-05-09T01:02:24.468 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_active": 0, 2024-05-09T01:02:24.468 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_send": 0, 2024-05-09T01:02:24.468 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_resend": 0, 2024-05-09T01:02:24.468 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_active": 0, 2024-05-09T01:02:24.468 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_send": 0, 2024-05-09T01:02:24.468 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_resend": 0, 2024-05-09T01:02:24.468 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_active": 0, 2024-05-09T01:02:24.469 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_send": 0, 2024-05-09T01:02:24.469 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_resend": 0, 2024-05-09T01:02:24.469 INFO:teuthology.orchestra.run.smithi157.stdout: "command_active": 0, 2024-05-09T01:02:24.469 INFO:teuthology.orchestra.run.smithi157.stdout: "command_send": 0, 2024-05-09T01:02:24.469 INFO:teuthology.orchestra.run.smithi157.stdout: "command_resend": 0, 2024-05-09T01:02:24.469 INFO:teuthology.orchestra.run.smithi157.stdout: "map_epoch": 82, 2024-05-09T01:02:24.469 INFO:teuthology.orchestra.run.smithi157.stdout: "map_full": 0, 2024-05-09T01:02:24.470 INFO:teuthology.orchestra.run.smithi157.stdout: "map_inc": 4, 2024-05-09T01:02:24.470 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_sessions": 11, 2024-05-09T01:02:24.470 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_session_open": 11, 2024-05-09T01:02:24.470 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_session_close": 0, 2024-05-09T01:02:24.470 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_laggy": 0, 2024-05-09T01:02:24.470 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_wr": 0, 2024-05-09T01:02:24.470 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_rd": 2, 2024-05-09T01:02:24.471 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_del": 0 2024-05-09T01:02:24.471 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:24.471 INFO:teuthology.orchestra.run.smithi157.stdout: "oft": { 2024-05-09T01:02:24.471 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_objs": 0, 2024-05-09T01:02:24.471 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_kv_pairs": 0, 2024-05-09T01:02:24.471 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_updates": 0, 2024-05-09T01:02:24.471 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_removes": 0 2024-05-09T01:02:24.471 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:24.471 INFO:teuthology.orchestra.run.smithi157.stdout: "osd-slow-ops": { 2024-05-09T01:02:24.472 INFO:teuthology.orchestra.run.smithi157.stdout: "slow_ops_count": 0 2024-05-09T01:02:24.472 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:24.472 INFO:teuthology.orchestra.run.smithi157.stdout: "purge_queue": { 2024-05-09T01:02:24.472 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_ops": 0, 2024-05-09T01:02:24.472 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_ops_high_water": 0, 2024-05-09T01:02:24.472 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing": 0, 2024-05-09T01:02:24.472 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_high_water": 0, 2024-05-09T01:02:24.472 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executed_ops": 0, 2024-05-09T01:02:24.472 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executed": 0, 2024-05-09T01:02:24.472 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_item_in_journal": 0 2024-05-09T01:02:24.472 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:24.472 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T01:02:24.472 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-05-09T01:02:24.472 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2024-05-09T01:02:24.473 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-05-09T01:02:24.473 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 5878, 2024-05-09T01:02:24.473 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 1348216801, 2024-05-09T01:02:24.473 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-05-09T01:02:24.473 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 5878, 2024-05-09T01:02:24.473 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-05-09T01:02:24.473 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-05-09T01:02:24.473 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 5878, 2024-05-09T01:02:24.473 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 1348216801, 2024-05-09T01:02:24.473 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-05-09T01:02:24.473 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:24.473 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:24.473 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:24.474 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-09T01:02:24.474 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:24.474 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-objecter_bytes": { 2024-05-09T01:02:24.474 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-05-09T01:02:24.474 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2024-05-09T01:02:24.474 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-05-09T01:02:24.474 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-05-09T01:02:24.474 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-05-09T01:02:24.474 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-05-09T01:02:24.474 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-05-09T01:02:24.474 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 5544, 2024-05-09T01:02:24.474 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 1346654182, 2024-05-09T01:02:24.474 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 5544, 2024-05-09T01:02:24.475 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 1346654182, 2024-05-09T01:02:24.475 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-05-09T01:02:24.475 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:24.475 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:24.475 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:24.475 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-09T01:02:24.475 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:24.475 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-objecter_ops": { 2024-05-09T01:02:24.475 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-05-09T01:02:24.475 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 1024, 2024-05-09T01:02:24.475 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-05-09T01:02:24.475 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-05-09T01:02:24.475 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-05-09T01:02:24.475 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-05-09T01:02:24.476 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-05-09T01:02:24.476 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 5544, 2024-05-09T01:02:24.476 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 5544, 2024-05-09T01:02:24.476 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 5544, 2024-05-09T01:02:24.476 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 5544, 2024-05-09T01:02:24.476 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-05-09T01:02:24.476 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:24.476 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:24.476 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:24.476 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-09T01:02:24.476 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:24.476 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-write_buf_throttle": { 2024-05-09T01:02:24.476 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-05-09T01:02:24.476 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 3758096384, 2024-05-09T01:02:24.476 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-05-09T01:02:24.477 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-05-09T01:02:24.477 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-05-09T01:02:24.477 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-05-09T01:02:24.477 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-05-09T01:02:24.477 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-05-09T01:02:24.477 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-05-09T01:02:24.477 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 0, 2024-05-09T01:02:24.477 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 0, 2024-05-09T01:02:24.477 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-05-09T01:02:24.477 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:24.477 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:24.477 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:24.477 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-09T01:02:24.477 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-05-09T01:02:24.478 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-write_buf_throttle-0x55652924c8f0": { 2024-05-09T01:02:24.478 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-05-09T01:02:24.478 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 3758096384, 2024-05-09T01:02:24.478 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-05-09T01:02:24.478 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-05-09T01:02:24.478 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-05-09T01:02:24.478 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-05-09T01:02:24.478 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-05-09T01:02:24.478 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-05-09T01:02:24.478 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-05-09T01:02:24.478 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 0, 2024-05-09T01:02:24.478 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 0, 2024-05-09T01:02:24.478 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-05-09T01:02:24.478 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-05-09T01:02:24.479 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-05-09T01:02:24.479 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-05-09T01:02:24.479 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-09T01:02:24.479 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-05-09T01:02:24.479 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-05-09T01:02:25.342 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-05-09T01:02:25.342 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-05-09T01:02:25.342 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-05-09T01:02:25.353 INFO:tasks.kclient:Unmounting kernel clients... 2024-05-09T01:02:25.353 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-09T01:02:25.353 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T01:02:25.382 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-09T01:02:25.383 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T01:02:25.445 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph osd blocklist ls 2024-05-09T01:02:25.750 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:02:25.790 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:25 smithi005 ceph-mon[30365]: pgmap v853: 97 pgs: 97 active+clean; 1.2 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 31 MiB/s wr, 512 op/s 2024-05-09T01:02:25.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:25 smithi157 ceph-mon[34294]: pgmap v853: 97 pgs: 97 active+clean; 1.2 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 31 MiB/s wr, 512 op/s 2024-05-09T01:02:26.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:25 smithi131 ceph-mon[34547]: pgmap v853: 97 pgs: 97 active+clean; 1.2 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 31 MiB/s wr, 512 op/s 2024-05-09T01:02:27.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:27 smithi005 ceph-mon[30365]: pgmap v854: 97 pgs: 97 active+clean; 513 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 23 MiB/s wr, 416 op/s 2024-05-09T01:02:27.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:27 smithi157 ceph-mon[34294]: pgmap v854: 97 pgs: 97 active+clean; 513 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 23 MiB/s wr, 416 op/s 2024-05-09T01:02:27.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:27.951+0000 7f0df034b700 1 -- 172.21.15.5:0/1128403194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0de8108f30 msgr2=0x7f0de8109340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:02:27.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:27.951+0000 7f0df034b700 1 --2- 172.21.15.5:0/1128403194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0de8108f30 0x7f0de8109340 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f0dd80099c0 tx=0x7f0dd8009cd0 comp rx=0 tx=0).stop 2024-05-09T01:02:27.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:27.951+0000 7f0df034b700 1 -- 172.21.15.5:0/1128403194 shutdown_connections 2024-05-09T01:02:27.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:27.951+0000 7f0df034b700 1 --2- 172.21.15.5:0/1128403194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0de8109880 0x7f0de81111e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:27.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:27.951+0000 7f0df034b700 1 --2- 172.21.15.5:0/1128403194 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0de8104650 0x7f0de8104a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:27.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:27.951+0000 7f0df034b700 1 --2- 172.21.15.5:0/1128403194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0de8108f30 0x7f0de8109340 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:27.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:27.951+0000 7f0df034b700 1 -- 172.21.15.5:0/1128403194 >> 172.21.15.5:0/1128403194 conn(0x7f0de8079f00 msgr2=0x7f0de807a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:02:27.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:27.951+0000 7f0df034b700 1 -- 172.21.15.5:0/1128403194 shutdown_connections 2024-05-09T01:02:27.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:27.951+0000 7f0df034b700 1 -- 172.21.15.5:0/1128403194 wait complete. 2024-05-09T01:02:27.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:27.952+0000 7f0df034b700 1 Processor -- start 2024-05-09T01:02:27.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:27.952+0000 7f0df034b700 1 -- start start 2024-05-09T01:02:27.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:27.952+0000 7f0df034b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0de8104650 0x7f0de819af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:02:27.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:27.952+0000 7f0df034b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0de8108f30 0x7f0de819b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:02:27.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:27.952+0000 7f0df034b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0de8109880 0x7f0de819f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:02:27.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:27.952+0000 7f0df034b700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0de8114490 con 0x7f0de8109880 2024-05-09T01:02:27.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:27.952+0000 7f0df034b700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0de8114310 con 0x7f0de8104650 2024-05-09T01:02:27.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:27.952+0000 7f0df034b700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0de8114610 con 0x7f0de8108f30 2024-05-09T01:02:27.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:27.952+0000 7f0dee8e8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0de8109880 0x7f0de819f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:02:27.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:27.952+0000 7f0dee8e8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0de8109880 0x7f0de819f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48492/0 (socket says 172.21.15.5:48492) 2024-05-09T01:02:27.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:27.952+0000 7f0dee8e8700 1 -- 172.21.15.5:0/933387999 learned_addr learned my addr 172.21.15.5:0/933387999 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:02:27.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:27.953+0000 7f0dee0e7700 1 --2- 172.21.15.5:0/933387999 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0de8104650 0x7f0de819af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:02:27.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:27.953+0000 7f0ded8e6700 1 --2- 172.21.15.5:0/933387999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0de8108f30 0x7f0de819b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:02:27.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:27.953+0000 7f0dee8e8700 1 -- 172.21.15.5:0/933387999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0de8108f30 msgr2=0x7f0de819b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:02:27.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:27.953+0000 7f0dee8e8700 1 --2- 172.21.15.5:0/933387999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0de8108f30 0x7f0de819b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:27.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:27.953+0000 7f0dee8e8700 1 -- 172.21.15.5:0/933387999 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0de8104650 msgr2=0x7f0de819af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:02:27.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:27.953+0000 7f0dee8e8700 1 --2- 172.21.15.5:0/933387999 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0de8104650 0x7f0de819af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:27.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:27.953+0000 7f0dee8e8700 1 -- 172.21.15.5:0/933387999 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0de819fe10 con 0x7f0de8109880 2024-05-09T01:02:27.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:27.953+0000 7f0dee8e8700 1 --2- 172.21.15.5:0/933387999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0de8109880 0x7f0de819f770 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f0de0007ee0 tx=0x7f0de000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:02:27.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:27.953+0000 7f0ddf7fe700 1 -- 172.21.15.5:0/933387999 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de000dbd0 con 0x7f0de8109880 2024-05-09T01:02:27.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:27.953+0000 7f0df034b700 1 -- 172.21.15.5:0/933387999 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0de81a0100 con 0x7f0de8109880 2024-05-09T01:02:27.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:27.953+0000 7f0ddf7fe700 1 -- 172.21.15.5:0/933387999 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0de000dd70 con 0x7f0de8109880 2024-05-09T01:02:27.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:27.953+0000 7f0ddf7fe700 1 -- 172.21.15.5:0/933387999 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de0005990 con 0x7f0de8109880 2024-05-09T01:02:27.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:27.953+0000 7f0df034b700 1 -- 172.21.15.5:0/933387999 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0de81a7a20 con 0x7f0de8109880 2024-05-09T01:02:27.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:27.954+0000 7f0df034b700 1 -- 172.21.15.5:0/933387999 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0de8105ab0 con 0x7f0de8109880 2024-05-09T01:02:27.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:27.956+0000 7f0ddf7fe700 1 -- 172.21.15.5:0/933387999 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f0de001a020 con 0x7f0de8109880 2024-05-09T01:02:27.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:27.956+0000 7f0ddf7fe700 1 --2- 172.21.15.5:0/933387999 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f0dd40b2880 0x7f0dd40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:02:27.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:27.957+0000 7f0dee0e7700 1 --2- 172.21.15.5:0/933387999 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f0dd40b2880 0x7f0dd40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:02:27.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:27.957+0000 7f0ddf7fe700 1 -- 172.21.15.5:0/933387999 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f0de00e2000 con 0x7f0de8109880 2024-05-09T01:02:27.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:27.957+0000 7f0dee0e7700 1 --2- 172.21.15.5:0/933387999 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f0dd40b2880 0x7f0dd40b4d90 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0de4005950 tx=0x7f0de400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:02:27.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:27.960+0000 7f0ddf7fe700 1 -- 172.21.15.5:0/933387999 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0de0010070 con 0x7f0de8109880 2024-05-09T01:02:28.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:27 smithi131 ceph-mon[34547]: pgmap v854: 97 pgs: 97 active+clean; 513 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 23 MiB/s wr, 416 op/s 2024-05-09T01:02:28.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:28.191+0000 7f0df034b700 1 -- 172.21.15.5:0/933387999 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0de819c2a0 con 0x7f0de8109880 2024-05-09T01:02:28.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:28.191+0000 7f0ddf7fe700 1 -- 172.21.15.5:0/933387999 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v82) ==== 81+0+577 (secure 0 0 0) 0x7f0de00a9cf0 con 0x7f0de8109880 2024-05-09T01:02:28.192 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/3405015786 2024-05-10T00:36:21.742363+0000 2024-05-09T01:02:28.192 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/2669115557 2024-05-10T00:36:21.742363+0000 2024-05-09T01:02:28.192 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6801/287842266 2024-05-10T00:36:21.742363+0000 2024-05-09T01:02:28.192 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6800/752513054 2024-05-10T00:35:53.128955+0000 2024-05-09T01:02:28.193 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6800/287842266 2024-05-10T00:36:21.742363+0000 2024-05-09T01:02:28.193 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6801/752513054 2024-05-10T00:35:53.128955+0000 2024-05-09T01:02:28.193 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/1954876662 2024-05-10T00:35:53.128955+0000 2024-05-09T01:02:28.193 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/2545175121 2024-05-10T00:36:21.742363+0000 2024-05-09T01:02:28.193 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/3754653663 2024-05-10T00:35:53.128955+0000 2024-05-09T01:02:28.193 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/651514359 2024-05-10T00:35:53.128955+0000 2024-05-09T01:02:28.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:28.193+0000 7f0df034b700 1 -- 172.21.15.5:0/933387999 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f0dd40b2880 msgr2=0x7f0dd40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:02:28.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:28.193+0000 7f0df034b700 1 --2- 172.21.15.5:0/933387999 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f0dd40b2880 0x7f0dd40b4d90 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0de4005950 tx=0x7f0de400b410 comp rx=0 tx=0).stop 2024-05-09T01:02:28.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:28.193+0000 7f0df034b700 1 -- 172.21.15.5:0/933387999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0de8109880 msgr2=0x7f0de819f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:02:28.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:28.193+0000 7f0df034b700 1 --2- 172.21.15.5:0/933387999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0de8109880 0x7f0de819f770 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f0de0007ee0 tx=0x7f0de000d3b0 comp rx=0 tx=0).stop 2024-05-09T01:02:28.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:28.194+0000 7f0df034b700 1 -- 172.21.15.5:0/933387999 shutdown_connections 2024-05-09T01:02:28.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:28.194+0000 7f0df034b700 1 --2- 172.21.15.5:0/933387999 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f0dd40b2880 0x7f0dd40b4d90 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:28.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:28.194+0000 7f0df034b700 1 --2- 172.21.15.5:0/933387999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0de8109880 0x7f0de819f770 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:28.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:28.194+0000 7f0df034b700 1 --2- 172.21.15.5:0/933387999 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0de8104650 0x7f0de819af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:28.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:28.194+0000 7f0df034b700 1 --2- 172.21.15.5:0/933387999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0de8108f30 0x7f0de819b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:28.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:28.194+0000 7f0df034b700 1 -- 172.21.15.5:0/933387999 >> 172.21.15.5:0/933387999 conn(0x7f0de8079f00 msgr2=0x7f0de81021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:02:28.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:28.194+0000 7f0df034b700 1 -- 172.21.15.5:0/933387999 shutdown_connections 2024-05-09T01:02:28.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:28.194+0000 7f0df034b700 1 -- 172.21.15.5:0/933387999 wait complete. 2024-05-09T01:02:28.196 INFO:teuthology.orchestra.run.smithi005.stderr:listed 10 entries 2024-05-09T01:02:28.790 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:28 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/933387999' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T01:02:28.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:28 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/933387999' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T01:02:29.028 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-09T01:02:29.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T01:02:29.029 DEBUG:teuthology.orchestra.run.smithi005:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-09T01:02:29.086 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2024-05-09T01:02:29.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T01:02:29.086 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-09T01:02:29.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:28 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/933387999' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T01:02:29.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T01:02:29.148 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-05-09T01:02:29.205 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-09T01:02:29.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T01:02:29.205 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T01:02:29.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T01:02:29.250 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link delete ceph-brx 2024-05-09T01:02:29.349 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-09T01:02:29.360 INFO:tasks.cephadm:Teardown begin 2024-05-09T01:02:29.361 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-09T01:02:29.361 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T01:02:29.620 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:02:29.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:29 smithi005 ceph-mon[30365]: pgmap v855: 97 pgs: 97 active+clean; 4.6 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 15 MiB/s wr, 329 op/s 2024-05-09T01:02:29.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:29 smithi157 ceph-mon[34294]: pgmap v855: 97 pgs: 97 active+clean; 4.6 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 15 MiB/s wr, 329 op/s 2024-05-09T01:02:30.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:29 smithi131 ceph-mon[34547]: pgmap v855: 97 pgs: 97 active+clean; 4.6 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 15 MiB/s wr, 329 op/s 2024-05-09T01:02:31.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:31.532+0000 7f0a4f789700 1 -- 172.21.15.5:0/4119168389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a48104650 msgr2=0x7f0a48104a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:02:31.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:31.532+0000 7f0a4f789700 1 --2- 172.21.15.5:0/4119168389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a48104650 0x7f0a48104a60 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f0a440099c0 tx=0x7f0a44009cd0 comp rx=0 tx=0).stop 2024-05-09T01:02:31.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:31.532+0000 7f0a4f789700 1 -- 172.21.15.5:0/4119168389 shutdown_connections 2024-05-09T01:02:31.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:31.532+0000 7f0a4f789700 1 --2- 172.21.15.5:0/4119168389 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a48109880 0x7f0a481111e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:31.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:31.532+0000 7f0a4f789700 1 --2- 172.21.15.5:0/4119168389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a48104650 0x7f0a48104a60 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:31.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:31.532+0000 7f0a4f789700 1 --2- 172.21.15.5:0/4119168389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a48108f30 0x7f0a48109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:31.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:31.532+0000 7f0a4f789700 1 -- 172.21.15.5:0/4119168389 >> 172.21.15.5:0/4119168389 conn(0x7f0a48079f00 msgr2=0x7f0a4807a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:02:31.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:31.532+0000 7f0a4f789700 1 -- 172.21.15.5:0/4119168389 shutdown_connections 2024-05-09T01:02:31.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:31.532+0000 7f0a4f789700 1 -- 172.21.15.5:0/4119168389 wait complete. 2024-05-09T01:02:31.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:31.533+0000 7f0a4f789700 1 Processor -- start 2024-05-09T01:02:31.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:31.533+0000 7f0a4f789700 1 -- start start 2024-05-09T01:02:31.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:31.533+0000 7f0a4f789700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a48104650 0x7f0a48075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:02:31.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:31.533+0000 7f0a4f789700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a48108f30 0x7f0a48076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:02:31.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:31.533+0000 7f0a4f789700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a48109880 0x7f0a48076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:02:31.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:31.533+0000 7f0a4f789700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0a481144b0 con 0x7f0a48108f30 2024-05-09T01:02:31.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:31.533+0000 7f0a4f789700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0a48114330 con 0x7f0a48109880 2024-05-09T01:02:31.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:31.533+0000 7f0a4f789700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0a48114630 con 0x7f0a48104650 2024-05-09T01:02:31.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:31.533+0000 7f0a4cd24700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a48108f30 0x7f0a48076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:02:31.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:31.533+0000 7f0a4cd24700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a48108f30 0x7f0a48076000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48524/0 (socket says 172.21.15.5:48524) 2024-05-09T01:02:31.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:31.533+0000 7f0a4cd24700 1 -- 172.21.15.5:0/980763748 learned_addr learned my addr 172.21.15.5:0/980763748 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:02:31.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:31.533+0000 7f0a4dd26700 1 --2- 172.21.15.5:0/980763748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a48109880 0x7f0a48076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:02:31.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:31.533+0000 7f0a4d525700 1 --2- 172.21.15.5:0/980763748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a48104650 0x7f0a48075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:02:31.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:31.533+0000 7f0a4cd24700 1 -- 172.21.15.5:0/980763748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a48104650 msgr2=0x7f0a48075ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:02:31.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:31.533+0000 7f0a4cd24700 1 --2- 172.21.15.5:0/980763748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a48104650 0x7f0a48075ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:31.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:31.533+0000 7f0a4cd24700 1 -- 172.21.15.5:0/980763748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a48109880 msgr2=0x7f0a48076540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:02:31.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:31.533+0000 7f0a4cd24700 1 --2- 172.21.15.5:0/980763748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a48109880 0x7f0a48076540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:31.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:31.533+0000 7f0a4cd24700 1 -- 172.21.15.5:0/980763748 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a481a91d0 con 0x7f0a48108f30 2024-05-09T01:02:31.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:31.533+0000 7f0a4dd26700 1 --2- 172.21.15.5:0/980763748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a48109880 0x7f0a48076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:02:31.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:31.533+0000 7f0a4d525700 1 --2- 172.21.15.5:0/980763748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a48104650 0x7f0a48075ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:02:31.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:31.534+0000 7f0a4cd24700 1 --2- 172.21.15.5:0/980763748 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a48108f30 0x7f0a48076000 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f0a3800d980 tx=0x7f0a3800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:02:31.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:31.534+0000 7f0a3e7fc700 1 -- 172.21.15.5:0/980763748 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a38017070 con 0x7f0a48108f30 2024-05-09T01:02:31.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:31.534+0000 7f0a3e7fc700 1 -- 172.21.15.5:0/980763748 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a38010460 con 0x7f0a48108f30 2024-05-09T01:02:31.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:31.534+0000 7f0a4f789700 1 -- 172.21.15.5:0/980763748 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a481a94c0 con 0x7f0a48108f30 2024-05-09T01:02:31.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:31.534+0000 7f0a3e7fc700 1 -- 172.21.15.5:0/980763748 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a3800de60 con 0x7f0a48108f30 2024-05-09T01:02:31.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:31.534+0000 7f0a4f789700 1 -- 172.21.15.5:0/980763748 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a481a9a50 con 0x7f0a48108f30 2024-05-09T01:02:31.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:31.535+0000 7f0a4f789700 1 -- 172.21.15.5:0/980763748 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a480512c0 con 0x7f0a48108f30 2024-05-09T01:02:31.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:31.536+0000 7f0a3e7fc700 1 -- 172.21.15.5:0/980763748 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f0a38010610 con 0x7f0a48108f30 2024-05-09T01:02:31.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:31.537+0000 7f0a3e7fc700 1 --2- 172.21.15.5:0/980763748 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f0a340b2900 0x7f0a340b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:02:31.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:31.538+0000 7f0a4d525700 1 --2- 172.21.15.5:0/980763748 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f0a340b2900 0x7f0a340b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:02:31.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:31.538+0000 7f0a3e7fc700 1 -- 172.21.15.5:0/980763748 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f0a380e1de0 con 0x7f0a48108f30 2024-05-09T01:02:31.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:31.538+0000 7f0a4d525700 1 --2- 172.21.15.5:0/980763748 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f0a340b2900 0x7f0a340b4e10 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f0a44003800 tx=0x7f0a44005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:02:31.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:31.543+0000 7f0a3e7fc700 1 -- 172.21.15.5:0/980763748 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0a380a9a70 con 0x7f0a48108f30 2024-05-09T01:02:31.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:31.806+0000 7f0a4f789700 1 -- 172.21.15.5:0/980763748 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0a4802e5f0 con 0x7f0a48108f30 2024-05-09T01:02:31.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:31.807+0000 7f0a3e7fc700 1 -- 172.21.15.5:0/980763748 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11929 (secure 0 0 0) 0x7f0a380a9370 con 0x7f0a48108f30 2024-05-09T01:02:31.808 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T01:02:31.808 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2685771887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2685771887},{"type":"v1","addr":"172.21.15.5:6835","nonce":2685771887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/254017535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":254017535},{"type":"v1","addr":"172.21.15.5:6837","nonce":254017535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T00:43:45.856066+0000","modified":"2024-05-09T01:01:10.302813+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24508,"mds_1":24499,"mds_2":24494,"mds_3":24493,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14598":{"gid":14598,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2109255228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2109255228},{"type":"v1","addr":"172.21.15.5:6839","nonce":2109255228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/3809966329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3809966329},{"type":"v1","addr":"172.21.15.5:6841","nonce":3809966329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6833/3272897742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3272897742},{"type":"v1","addr":"172.21.15.131:6833","nonce":3272897742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1283093898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1283093898},{"type":"v1","addr":"172.21.15.131:6835","nonce":1283093898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/2809525455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2809525455},{"type":"v1","addr":"172.21.15.157:6833","nonce":2809525455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.131:6837/1633006186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":1633006186},{"type":"v1","addr":"172.21.15.131:6837","nonce":1633006186}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.157:6835/832898273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":832898273},{"type":"v1","addr":"172.21.15.157:6835","nonce":832898273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.131:6839/2748669631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2748669631},{"type":"v1","addr":"172.21.15.131:6839","nonce":2748669631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.157:6837/744863009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":744863009},{"type":"v1","addr":"172.21.15.157:6837","nonce":744863009}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":269,"addr":"172.21.15.157:6839/2779860399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":2779860399},{"type":"v1","addr":"172.21.15.157:6839","nonce":2779860399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24493,24467,24499,24494,24508]},"id":1}]} 2024-05-09T01:02:31.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:31.809+0000 7f0a4f789700 1 -- 172.21.15.5:0/980763748 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f0a340b2900 msgr2=0x7f0a340b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:02:31.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:31.809+0000 7f0a4f789700 1 --2- 172.21.15.5:0/980763748 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f0a340b2900 0x7f0a340b4e10 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f0a44003800 tx=0x7f0a44005e80 comp rx=0 tx=0).stop 2024-05-09T01:02:31.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:31.809+0000 7f0a4f789700 1 -- 172.21.15.5:0/980763748 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a48108f30 msgr2=0x7f0a48076000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:02:31.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:31.809+0000 7f0a4f789700 1 --2- 172.21.15.5:0/980763748 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a48108f30 0x7f0a48076000 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f0a3800d980 tx=0x7f0a3800dd40 comp rx=0 tx=0).stop 2024-05-09T01:02:31.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:31.809+0000 7f0a4f789700 1 -- 172.21.15.5:0/980763748 shutdown_connections 2024-05-09T01:02:31.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:31.809+0000 7f0a4f789700 1 --2- 172.21.15.5:0/980763748 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f0a340b2900 0x7f0a340b4e10 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:31.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:31.809+0000 7f0a4f789700 1 --2- 172.21.15.5:0/980763748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a48109880 0x7f0a48076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:31.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:31.809+0000 7f0a4f789700 1 --2- 172.21.15.5:0/980763748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a48104650 0x7f0a48075ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:31.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:31.809+0000 7f0a4f789700 1 --2- 172.21.15.5:0/980763748 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a48108f30 0x7f0a48076000 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:31.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:31.809+0000 7f0a4f789700 1 -- 172.21.15.5:0/980763748 >> 172.21.15.5:0/980763748 conn(0x7f0a48079f00 msgr2=0x7f0a481020d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:02:31.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:31.809+0000 7f0a4f789700 1 -- 172.21.15.5:0/980763748 shutdown_connections 2024-05-09T01:02:31.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:31.809+0000 7f0a4f789700 1 -- 172.21.15.5:0/980763748 wait complete. 2024-05-09T01:02:31.811 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 34 2024-05-09T01:02:31.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:31 smithi157 ceph-mon[34294]: pgmap v856: 97 pgs: 97 active+clean; 4.6 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 8.9 MiB/s wr, 210 op/s 2024-05-09T01:02:32.040 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:31 smithi005 ceph-mon[30365]: pgmap v856: 97 pgs: 97 active+clean; 4.6 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 8.9 MiB/s wr, 210 op/s 2024-05-09T01:02:32.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:31 smithi131 ceph-mon[34547]: pgmap v856: 97 pgs: 97 active+clean; 4.6 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 8.9 MiB/s wr, 210 op/s 2024-05-09T01:02:32.591 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T01:02:32.851 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:02:32.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:32 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/980763748' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:02:32.974 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:32 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/980763748' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:02:33.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:32 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/980763748' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:02:33.938 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:33 smithi157 ceph-mon[34294]: pgmap v857: 97 pgs: 97 active+clean; 4.6 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.4 MiB/s wr, 114 op/s 2024-05-09T01:02:34.040 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:33 smithi005 ceph-mon[30365]: pgmap v857: 97 pgs: 97 active+clean; 4.6 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.4 MiB/s wr, 114 op/s 2024-05-09T01:02:34.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:33 smithi131 ceph-mon[34547]: pgmap v857: 97 pgs: 97 active+clean; 4.6 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.4 MiB/s wr, 114 op/s 2024-05-09T01:02:34.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:34.832+0000 7f657e25d700 1 -- 172.21.15.5:0/1071202355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6578106570 msgr2=0x7f6578106980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:02:34.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:34.832+0000 7f657e25d700 1 --2- 172.21.15.5:0/1071202355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6578106570 0x7f6578106980 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f6560009990 tx=0x7f6560009ca0 comp rx=0 tx=0).stop 2024-05-09T01:02:34.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:34.832+0000 7f657e25d700 1 -- 172.21.15.5:0/1071202355 shutdown_connections 2024-05-09T01:02:34.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:34.832+0000 7f657e25d700 1 --2- 172.21.15.5:0/1071202355 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6578106ec0 0x7f6578113350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:34.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:34.832+0000 7f657e25d700 1 --2- 172.21.15.5:0/1071202355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f657810c5b0 0x7f657810c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:34.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:34.832+0000 7f657e25d700 1 --2- 172.21.15.5:0/1071202355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6578106570 0x7f6578106980 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:34.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:34.832+0000 7f657e25d700 1 -- 172.21.15.5:0/1071202355 >> 172.21.15.5:0/1071202355 conn(0x7f6578102050 msgr2=0x7f65781044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:02:34.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:34.832+0000 7f657e25d700 1 -- 172.21.15.5:0/1071202355 shutdown_connections 2024-05-09T01:02:34.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:34.832+0000 7f657e25d700 1 -- 172.21.15.5:0/1071202355 wait complete. 2024-05-09T01:02:34.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:34.833+0000 7f657e25d700 1 Processor -- start 2024-05-09T01:02:34.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:34.833+0000 7f657e25d700 1 -- start start 2024-05-09T01:02:34.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:34.833+0000 7f657e25d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6578106570 0x7f657819d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:02:34.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:34.833+0000 7f657e25d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6578106ec0 0x7f657819d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:02:34.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:34.833+0000 7f657e25d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f657810c5b0 0x7f65781a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:02:34.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:34.833+0000 7f657e25d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6578116690 con 0x7f6578106570 2024-05-09T01:02:34.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:34.833+0000 7f657e25d700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6578116510 con 0x7f657810c5b0 2024-05-09T01:02:34.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:34.833+0000 7f657e25d700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6578116810 con 0x7f6578106ec0 2024-05-09T01:02:34.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:34.833+0000 7f65777fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6578106570 0x7f657819d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:02:34.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:34.833+0000 7f65777fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6578106570 0x7f657819d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57556/0 (socket says 172.21.15.5:57556) 2024-05-09T01:02:34.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:34.833+0000 7f65777fe700 1 -- 172.21.15.5:0/15845973 learned_addr learned my addr 172.21.15.5:0/15845973 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:02:34.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:34.834+0000 7f656effd700 1 --2- 172.21.15.5:0/15845973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6578106ec0 0x7f657819d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:02:34.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:34.834+0000 7f6577fff700 1 --2- 172.21.15.5:0/15845973 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f657810c5b0 0x7f65781a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:02:34.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:34.834+0000 7f65777fe700 1 -- 172.21.15.5:0/15845973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6578106ec0 msgr2=0x7f657819d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:02:34.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:34.834+0000 7f65777fe700 1 --2- 172.21.15.5:0/15845973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6578106ec0 0x7f657819d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:34.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:34.834+0000 7f65777fe700 1 -- 172.21.15.5:0/15845973 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f657810c5b0 msgr2=0x7f65781a1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:02:34.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:34.834+0000 7f65777fe700 1 --2- 172.21.15.5:0/15845973 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f657810c5b0 0x7f65781a1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:34.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:34.834+0000 7f65777fe700 1 -- 172.21.15.5:0/15845973 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65781a1f00 con 0x7f6578106570 2024-05-09T01:02:34.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:34.834+0000 7f6577fff700 1 --2- 172.21.15.5:0/15845973 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f657810c5b0 0x7f65781a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T01:02:34.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:34.834+0000 7f65777fe700 1 --2- 172.21.15.5:0/15845973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6578106570 0x7f657819d0e0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f656800c8f0 tx=0x7f656800cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:02:34.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:34.834+0000 7f65757fa700 1 -- 172.21.15.5:0/15845973 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6568007940 con 0x7f6578106570 2024-05-09T01:02:34.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:34.834+0000 7f657e25d700 1 -- 172.21.15.5:0/15845973 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65781a21f0 con 0x7f6578106570 2024-05-09T01:02:34.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:34.834+0000 7f65757fa700 1 -- 172.21.15.5:0/15845973 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f656800f450 con 0x7f6578106570 2024-05-09T01:02:34.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:34.834+0000 7f65757fa700 1 -- 172.21.15.5:0/15845973 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6568018660 con 0x7f6578106570 2024-05-09T01:02:34.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:34.834+0000 7f657e25d700 1 -- 172.21.15.5:0/15845973 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65781a2700 con 0x7f6578106570 2024-05-09T01:02:34.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:34.835+0000 7f657e25d700 1 -- 172.21.15.5:0/15845973 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65780512c0 con 0x7f6578106570 2024-05-09T01:02:34.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:34.837+0000 7f65757fa700 1 -- 172.21.15.5:0/15845973 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f656800f890 con 0x7f6578106570 2024-05-09T01:02:34.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:34.838+0000 7f65757fa700 1 --2- 172.21.15.5:0/15845973 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f65640b2880 0x7f65640b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:02:34.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:34.838+0000 7f656effd700 1 --2- 172.21.15.5:0/15845973 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f65640b2880 0x7f65640b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:02:34.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:34.838+0000 7f65757fa700 1 -- 172.21.15.5:0/15845973 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f65680e21c0 con 0x7f6578106570 2024-05-09T01:02:34.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:34.839+0000 7f656effd700 1 --2- 172.21.15.5:0/15845973 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f65640b2880 0x7f65640b4d90 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f65600059d0 tx=0x7f6560005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:02:34.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:34.842+0000 7f65757fa700 1 -- 172.21.15.5:0/15845973 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6568014030 con 0x7f6578106570 2024-05-09T01:02:35.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:35.104+0000 7f657e25d700 1 -- 172.21.15.5:0/15845973 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f657819e470 con 0x7f6578106570 2024-05-09T01:02:35.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:35.105+0000 7f65757fa700 1 -- 172.21.15.5:0/15845973 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11929 (secure 0 0 0) 0x7f657819e470 con 0x7f6578106570 2024-05-09T01:02:35.106 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T01:02:35.106 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2685771887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2685771887},{"type":"v1","addr":"172.21.15.5:6835","nonce":2685771887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/254017535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":254017535},{"type":"v1","addr":"172.21.15.5:6837","nonce":254017535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T00:43:45.856066+0000","modified":"2024-05-09T01:01:10.302813+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24508,"mds_1":24499,"mds_2":24494,"mds_3":24493,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14598":{"gid":14598,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2109255228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2109255228},{"type":"v1","addr":"172.21.15.5:6839","nonce":2109255228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/3809966329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3809966329},{"type":"v1","addr":"172.21.15.5:6841","nonce":3809966329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6833/3272897742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3272897742},{"type":"v1","addr":"172.21.15.131:6833","nonce":3272897742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1283093898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1283093898},{"type":"v1","addr":"172.21.15.131:6835","nonce":1283093898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/2809525455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2809525455},{"type":"v1","addr":"172.21.15.157:6833","nonce":2809525455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.131:6837/1633006186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":1633006186},{"type":"v1","addr":"172.21.15.131:6837","nonce":1633006186}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.157:6835/832898273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":832898273},{"type":"v1","addr":"172.21.15.157:6835","nonce":832898273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.131:6839/2748669631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":2748669631},{"type":"v1","addr":"172.21.15.131:6839","nonce":2748669631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.157:6837/744863009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":744863009},{"type":"v1","addr":"172.21.15.157:6837","nonce":744863009}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":269,"addr":"172.21.15.157:6839/2779860399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":2779860399},{"type":"v1","addr":"172.21.15.157:6839","nonce":2779860399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24493,24467,24499,24494,24508]},"id":1}]} 2024-05-09T01:02:35.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:35.108+0000 7f657e25d700 1 -- 172.21.15.5:0/15845973 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f65640b2880 msgr2=0x7f65640b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:02:35.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:35.108+0000 7f657e25d700 1 --2- 172.21.15.5:0/15845973 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f65640b2880 0x7f65640b4d90 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f65600059d0 tx=0x7f6560005960 comp rx=0 tx=0).stop 2024-05-09T01:02:35.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:35.108+0000 7f657e25d700 1 -- 172.21.15.5:0/15845973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6578106570 msgr2=0x7f657819d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:02:35.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:35.108+0000 7f657e25d700 1 --2- 172.21.15.5:0/15845973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6578106570 0x7f657819d0e0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f656800c8f0 tx=0x7f656800cc00 comp rx=0 tx=0).stop 2024-05-09T01:02:35.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:35.108+0000 7f657e25d700 1 -- 172.21.15.5:0/15845973 shutdown_connections 2024-05-09T01:02:35.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:35.108+0000 7f657e25d700 1 --2- 172.21.15.5:0/15845973 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f65640b2880 0x7f65640b4d90 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:35.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:35.108+0000 7f657e25d700 1 --2- 172.21.15.5:0/15845973 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f657810c5b0 0x7f65781a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:35.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:35.108+0000 7f657e25d700 1 --2- 172.21.15.5:0/15845973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6578106570 0x7f657819d0e0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:35.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:35.108+0000 7f657e25d700 1 --2- 172.21.15.5:0/15845973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6578106ec0 0x7f657819d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:35.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:35.108+0000 7f657e25d700 1 -- 172.21.15.5:0/15845973 >> 172.21.15.5:0/15845973 conn(0x7f6578102050 msgr2=0x7f6578103860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:02:35.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:35.108+0000 7f657e25d700 1 -- 172.21.15.5:0/15845973 shutdown_connections 2024-05-09T01:02:35.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:35.108+0000 7f657e25d700 1 -- 172.21.15.5:0/15845973 wait complete. 2024-05-09T01:02:35.110 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 34 2024-05-09T01:02:35.790 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:35 smithi005 ceph-mon[30365]: pgmap v858: 97 pgs: 97 active+clean; 4.6 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.4 KiB/s wr, 55 op/s 2024-05-09T01:02:35.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:35 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/15845973' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:02:35.895 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T01:02:35.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:35 smithi157 ceph-mon[34294]: pgmap v858: 97 pgs: 97 active+clean; 4.6 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.4 KiB/s wr, 55 op/s 2024-05-09T01:02:35.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:35 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/15845973' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:02:36.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:35 smithi131 ceph-mon[34547]: pgmap v858: 97 pgs: 97 active+clean; 4.6 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.4 KiB/s wr, 55 op/s 2024-05-09T01:02:36.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:35 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/15845973' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T01:02:36.149 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:02:37.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:37 smithi157 ceph-mon[34294]: pgmap v859: 97 pgs: 97 active+clean; 4.6 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.3 KiB/s wr, 37 op/s 2024-05-09T01:02:37.946 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:37 smithi005 ceph-mon[30365]: pgmap v859: 97 pgs: 97 active+clean; 4.6 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.3 KiB/s wr, 37 op/s 2024-05-09T01:02:37.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:37.979+0000 7f7291df2700 1 -- 172.21.15.5:0/3676300818 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f728c10c5e0 msgr2=0x7f728c10c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:02:37.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:37.979+0000 7f7291df2700 1 --2- 172.21.15.5:0/3676300818 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f728c10c5e0 0x7f728c10c9f0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f727c009990 tx=0x7f727c009ca0 comp rx=0 tx=0).stop 2024-05-09T01:02:37.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:37.979+0000 7f7291df2700 1 -- 172.21.15.5:0/3676300818 shutdown_connections 2024-05-09T01:02:37.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:37.979+0000 7f7291df2700 1 --2- 172.21.15.5:0/3676300818 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f728c106ef0 0x7f728c113380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:37.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:37.979+0000 7f7291df2700 1 --2- 172.21.15.5:0/3676300818 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f728c10c5e0 0x7f728c10c9f0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:37.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:37.979+0000 7f7291df2700 1 --2- 172.21.15.5:0/3676300818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f728c1065a0 0x7f728c1069b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:37.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:37.979+0000 7f7291df2700 1 -- 172.21.15.5:0/3676300818 >> 172.21.15.5:0/3676300818 conn(0x7f728c102080 msgr2=0x7f728c1044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:02:37.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:37.979+0000 7f7291df2700 1 -- 172.21.15.5:0/3676300818 shutdown_connections 2024-05-09T01:02:37.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:37.979+0000 7f7291df2700 1 -- 172.21.15.5:0/3676300818 wait complete. 2024-05-09T01:02:37.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:37.980+0000 7f7291df2700 1 Processor -- start 2024-05-09T01:02:37.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:37.980+0000 7f7291df2700 1 -- start start 2024-05-09T01:02:37.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:37.981+0000 7f7291df2700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f728c1065a0 0x7f728c19d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:02:37.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:37.981+0000 7f7291df2700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f728c106ef0 0x7f728c19d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:02:37.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:37.981+0000 7f7291df2700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f728c10c5e0 0x7f728c1a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:02:37.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:37.981+0000 7f7291df2700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f728c1166f0 con 0x7f728c10c5e0 2024-05-09T01:02:37.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:37.981+0000 7f7291df2700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f728c116570 con 0x7f728c106ef0 2024-05-09T01:02:37.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:37.981+0000 7f7291df2700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f728c116870 con 0x7f728c1065a0 2024-05-09T01:02:37.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:37.981+0000 7f728bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f728c10c5e0 0x7f728c1a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:02:37.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:37.981+0000 7f728bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f728c10c5e0 0x7f728c1a1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57584/0 (socket says 172.21.15.5:57584) 2024-05-09T01:02:37.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:37.981+0000 7f728bfff700 1 -- 172.21.15.5:0/2951255723 learned_addr learned my addr 172.21.15.5:0/2951255723 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:02:37.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:37.981+0000 7f728b7fe700 1 --2- 172.21.15.5:0/2951255723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f728c1065a0 0x7f728c19d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:02:37.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:37.981+0000 7f728affd700 1 --2- 172.21.15.5:0/2951255723 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f728c106ef0 0x7f728c19d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:02:37.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:37.981+0000 7f728bfff700 1 -- 172.21.15.5:0/2951255723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f728c1065a0 msgr2=0x7f728c19d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:02:37.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:37.981+0000 7f728bfff700 1 --2- 172.21.15.5:0/2951255723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f728c1065a0 0x7f728c19d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:37.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:37.981+0000 7f728bfff700 1 -- 172.21.15.5:0/2951255723 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f728c106ef0 msgr2=0x7f728c19d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:02:37.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:37.981+0000 7f728bfff700 1 --2- 172.21.15.5:0/2951255723 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f728c106ef0 0x7f728c19d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:37.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:37.981+0000 7f728bfff700 1 -- 172.21.15.5:0/2951255723 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f728c1a1fb0 con 0x7f728c10c5e0 2024-05-09T01:02:37.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:37.981+0000 7f728affd700 1 --2- 172.21.15.5:0/2951255723 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f728c106ef0 0x7f728c19d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T01:02:37.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:37.982+0000 7f728bfff700 1 --2- 172.21.15.5:0/2951255723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f728c10c5e0 0x7f728c1a1910 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f7284007ee0 tx=0x7f728400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:02:37.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:37.982+0000 7f7288ff9700 1 -- 172.21.15.5:0/2951255723 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f728400ddd0 con 0x7f728c10c5e0 2024-05-09T01:02:37.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:37.982+0000 7f7291df2700 1 -- 172.21.15.5:0/2951255723 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f728c1a22a0 con 0x7f728c10c5e0 2024-05-09T01:02:37.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:37.982+0000 7f7288ff9700 1 -- 172.21.15.5:0/2951255723 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7284004020 con 0x7f728c10c5e0 2024-05-09T01:02:37.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:37.982+0000 7f7288ff9700 1 -- 172.21.15.5:0/2951255723 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7284015660 con 0x7f728c10c5e0 2024-05-09T01:02:37.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:37.982+0000 7f7291df2700 1 -- 172.21.15.5:0/2951255723 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f728c1a9bc0 con 0x7f728c10c5e0 2024-05-09T01:02:37.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:37.983+0000 7f7291df2700 1 -- 172.21.15.5:0/2951255723 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f728c10e9a0 con 0x7f728c10c5e0 2024-05-09T01:02:37.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:37.984+0000 7f7288ff9700 1 -- 172.21.15.5:0/2951255723 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f72840041d0 con 0x7f728c10c5e0 2024-05-09T01:02:37.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:37.985+0000 7f7288ff9700 1 --2- 172.21.15.5:0/2951255723 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f72780b2880 0x7f72780b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:02:37.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:37.985+0000 7f728b7fe700 1 --2- 172.21.15.5:0/2951255723 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f72780b2880 0x7f72780b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:02:37.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:37.985+0000 7f7288ff9700 1 -- 172.21.15.5:0/2951255723 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f72840e24b0 con 0x7f728c10c5e0 2024-05-09T01:02:37.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:37.986+0000 7f728b7fe700 1 --2- 172.21.15.5:0/2951255723 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f72780b2880 0x7f72780b4d90 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f727c006010 tx=0x7f727c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:02:37.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:37.989+0000 7f7288ff9700 1 -- 172.21.15.5:0/2951255723 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7284010070 con 0x7f728c10c5e0 2024-05-09T01:02:38.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:37 smithi131 ceph-mon[34547]: pgmap v859: 97 pgs: 97 active+clean; 4.6 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.3 KiB/s wr, 37 op/s 2024-05-09T01:02:38.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:38.206+0000 7f7291df2700 1 -- 172.21.15.5:0/2951255723 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f728c19e440 con 0x7f728c10c5e0 2024-05-09T01:02:38.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:38.208+0000 7f7288ff9700 1 -- 172.21.15.5:0/2951255723 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22745 (secure 0 0 0) 0x7f72840aa140 con 0x7f728c10c5e0 2024-05-09T01:02:38.209 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-09T01:02:38.209 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":82,"fsid":"d85814c6-0d9b-11ef-bc97-c7b262605968","created":"2024-05-09T00:35:25.623377+0000","modified":"2024-05-09T00:45:23.926493+0000","last_up_change":"2024-05-09T00:42:38.525335+0000","last_in_change":"2024-05-09T00:42:17.990219+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T00:38:47.844734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T00:43:39.452106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T00:43:42.708106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f8d484bb-2477-4377-ae68-5bce571333bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2963385132},{"type":"v1","addr":"172.21.15.5:6803","nonce":2963385132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2963385132},{"type":"v1","addr":"172.21.15.5:6805","nonce":2963385132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2963385132},{"type":"v1","addr":"172.21.15.5:6809","nonce":2963385132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2963385132},{"type":"v1","addr":"172.21.15.5:6807","nonce":2963385132}]},"public_addr":"172.21.15.5:6803/2963385132","cluster_addr":"172.21.15.5:6805/2963385132","heartbeat_back_addr":"172.21.15.5:6809/2963385132","heartbeat_front_addr":"172.21.15.5:6807/2963385132","state":["exists","up"]},{"osd":1,"uuid":"1c123676-7144-4a67-93a6-fd49e36a5c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3837683163},{"type":"v1","addr":"172.21.15.131:6801","nonce":3837683163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3837683163},{"type":"v1","addr":"172.21.15.131:6803","nonce":3837683163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3837683163},{"type":"v1","addr":"172.21.15.131:6807","nonce":3837683163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3837683163},{"type":"v1","addr":"172.21.15.131:6805","nonce":3837683163}]},"public_addr":"172.21.15.131:6801/3837683163","cluster_addr":"172.21.15.131:6803/3837683163","heartbeat_back_addr":"172.21.15.131:6807/3837683163","heartbeat_front_addr":"172.21.15.131:6805/3837683163","state":["exists","up"]},{"osd":2,"uuid":"95ca51f0-b48b-485c-a5f7-b15c01e754b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":484818139},{"type":"v1","addr":"172.21.15.157:6801","nonce":484818139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":484818139},{"type":"v1","addr":"172.21.15.157:6803","nonce":484818139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":484818139},{"type":"v1","addr":"172.21.15.157:6807","nonce":484818139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":484818139},{"type":"v1","addr":"172.21.15.157:6805","nonce":484818139}]},"public_addr":"172.21.15.157:6801/484818139","cluster_addr":"172.21.15.157:6803/484818139","heartbeat_back_addr":"172.21.15.157:6807/484818139","heartbeat_front_addr":"172.21.15.157:6805/484818139","state":["exists","up"]},{"osd":3,"uuid":"ed7577fb-3676-4a08-b40d-5474eb88974f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":2357377277},{"type":"v1","addr":"172.21.15.5:6811","nonce":2357377277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":2357377277},{"type":"v1","addr":"172.21.15.5:6813","nonce":2357377277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":2357377277},{"type":"v1","addr":"172.21.15.5:6817","nonce":2357377277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":2357377277},{"type":"v1","addr":"172.21.15.5:6815","nonce":2357377277}]},"public_addr":"172.21.15.5:6811/2357377277","cluster_addr":"172.21.15.5:6813/2357377277","heartbeat_back_addr":"172.21.15.5:6817/2357377277","heartbeat_front_addr":"172.21.15.5:6815/2357377277","state":["exists","up"]},{"osd":4,"uuid":"fb5dcf79-a6f1-4b1a-9e3a-087fb5132472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":2915418378},{"type":"v1","addr":"172.21.15.131:6809","nonce":2915418378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":2915418378},{"type":"v1","addr":"172.21.15.131:6811","nonce":2915418378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":2915418378},{"type":"v1","addr":"172.21.15.131:6815","nonce":2915418378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":2915418378},{"type":"v1","addr":"172.21.15.131:6813","nonce":2915418378}]},"public_addr":"172.21.15.131:6809/2915418378","cluster_addr":"172.21.15.131:6811/2915418378","heartbeat_back_addr":"172.21.15.131:6815/2915418378","heartbeat_front_addr":"172.21.15.131:6813/2915418378","state":["exists","up"]},{"osd":5,"uuid":"f01ed5d3-8066-4d25-a113-280c83e75ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4188818663},{"type":"v1","addr":"172.21.15.157:6809","nonce":4188818663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":4188818663},{"type":"v1","addr":"172.21.15.157:6811","nonce":4188818663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":4188818663},{"type":"v1","addr":"172.21.15.157:6815","nonce":4188818663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":4188818663},{"type":"v1","addr":"172.21.15.157:6813","nonce":4188818663}]},"public_addr":"172.21.15.157:6809/4188818663","cluster_addr":"172.21.15.157:6811/4188818663","heartbeat_back_addr":"172.21.15.157:6815/4188818663","heartbeat_front_addr":"172.21.15.157:6813/4188818663","state":["exists","up"]},{"osd":6,"uuid":"932379a8-c97b-4acc-8665-81f7d9cbce65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2930397813},{"type":"v1","addr":"172.21.15.5:6819","nonce":2930397813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2930397813},{"type":"v1","addr":"172.21.15.5:6821","nonce":2930397813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2930397813},{"type":"v1","addr":"172.21.15.5:6825","nonce":2930397813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2930397813},{"type":"v1","addr":"172.21.15.5:6823","nonce":2930397813}]},"public_addr":"172.21.15.5:6819/2930397813","cluster_addr":"172.21.15.5:6821/2930397813","heartbeat_back_addr":"172.21.15.5:6825/2930397813","heartbeat_front_addr":"172.21.15.5:6823/2930397813","state":["exists","up"]},{"osd":7,"uuid":"3c0a0573-6465-4bf6-b70f-310d49857a6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":4125045219},{"type":"v1","addr":"172.21.15.131:6817","nonce":4125045219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":4125045219},{"type":"v1","addr":"172.21.15.131:6819","nonce":4125045219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":4125045219},{"type":"v1","addr":"172.21.15.131:6823","nonce":4125045219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":4125045219},{"type":"v1","addr":"172.21.15.131:6821","nonce":4125045219}]},"public_addr":"172.21.15.131:6817/4125045219","cluster_addr":"172.21.15.131:6819/4125045219","heartbeat_back_addr":"172.21.15.131:6823/4125045219","heartbeat_front_addr":"172.21.15.131:6821/4125045219","state":["exists","up"]},{"osd":8,"uuid":"c49f0a00-f5ae-40ed-8f38-7b953ba1c17c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2001626768},{"type":"v1","addr":"172.21.15.157:6817","nonce":2001626768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2001626768},{"type":"v1","addr":"172.21.15.157:6819","nonce":2001626768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2001626768},{"type":"v1","addr":"172.21.15.157:6823","nonce":2001626768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2001626768},{"type":"v1","addr":"172.21.15.157:6821","nonce":2001626768}]},"public_addr":"172.21.15.157:6817/2001626768","cluster_addr":"172.21.15.157:6819/2001626768","heartbeat_back_addr":"172.21.15.157:6823/2001626768","heartbeat_front_addr":"172.21.15.157:6821/2001626768","state":["exists","up"]},{"osd":9,"uuid":"1927dfe7-c1af-4a15-bb71-d58d97c8734a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":4287220015},{"type":"v1","addr":"172.21.15.5:6827","nonce":4287220015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":4287220015},{"type":"v1","addr":"172.21.15.5:6829","nonce":4287220015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":4287220015},{"type":"v1","addr":"172.21.15.5:6833","nonce":4287220015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":4287220015},{"type":"v1","addr":"172.21.15.5:6831","nonce":4287220015}]},"public_addr":"172.21.15.5:6827/4287220015","cluster_addr":"172.21.15.5:6829/4287220015","heartbeat_back_addr":"172.21.15.5:6833/4287220015","heartbeat_front_addr":"172.21.15.5:6831/4287220015","state":["exists","up"]},{"osd":10,"uuid":"fbb4f3e3-eb7d-4ecb-a30b-55b8910375ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1571783985},{"type":"v1","addr":"172.21.15.131:6825","nonce":1571783985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1571783985},{"type":"v1","addr":"172.21.15.131:6827","nonce":1571783985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1571783985},{"type":"v1","addr":"172.21.15.131:6831","nonce":1571783985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1571783985},{"type":"v1","addr":"172.21.15.131:6829","nonce":1571783985}]},"public_addr":"172.21.15.131:6825/1571783985","cluster_addr":"172.21.15.131:6827/1571783985","heartbeat_back_addr":"172.21.15.131:6831/1571783985","heartbeat_front_addr":"172.21.15.131:6829/1571783985","state":["exists","up"]},{"osd":11,"uuid":"b858453f-eda1-4e25-89c8-94b54c830938","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3261151188},{"type":"v1","addr":"172.21.15.157:6825","nonce":3261151188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":3261151188},{"type":"v1","addr":"172.21.15.157:6827","nonce":3261151188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":3261151188},{"type":"v1","addr":"172.21.15.157:6831","nonce":3261151188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":3261151188},{"type":"v1","addr":"172.21.15.157:6829","nonce":3261151188}]},"public_addr":"172.21.15.157:6825/3261151188","cluster_addr":"172.21.15.157:6827/3261151188","heartbeat_back_addr":"172.21.15.157:6831/3261151188","heartbeat_front_addr":"172.21.15.157:6829/3261151188","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:37:57.528483+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:38:21.477482+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:38:45.780266+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:39:10.938791+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:39:35.734857+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:40:00.169040+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:40:26.494635+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:40:51.881718+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:41:17.697112+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:41:44.826479+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:42:10.504768+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:42:36.605447+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6},{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":6}]},{"pgid":"3.19","mappings":[{"from":3,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":7}]},{"pgid":"3.35","mappings":[{"from":1,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/3405015786":"2024-05-10T00:36:21.742363+0000","172.21.15.5:0/2669115557":"2024-05-10T00:36:21.742363+0000","172.21.15.5:6801/287842266":"2024-05-10T00:36:21.742363+0000","172.21.15.5:6800/752513054":"2024-05-10T00:35:53.128955+0000","172.21.15.5:6800/287842266":"2024-05-10T00:36:21.742363+0000","172.21.15.5:6801/752513054":"2024-05-10T00:35:53.128955+0000","172.21.15.5:0/1954876662":"2024-05-10T00:35:53.128955+0000","172.21.15.5:0/2545175121":"2024-05-10T00:36:21.742363+0000","172.21.15.5:0/3754653663":"2024-05-10T00:35:53.128955+0000","172.21.15.5:0/651514359":"2024-05-10T00:35:53.128955+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T01:02:38.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:38.210+0000 7f7291df2700 1 -- 172.21.15.5:0/2951255723 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f72780b2880 msgr2=0x7f72780b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:02:38.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:38.210+0000 7f7291df2700 1 --2- 172.21.15.5:0/2951255723 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f72780b2880 0x7f72780b4d90 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f727c006010 tx=0x7f727c005960 comp rx=0 tx=0).stop 2024-05-09T01:02:38.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:38.210+0000 7f7291df2700 1 -- 172.21.15.5:0/2951255723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f728c10c5e0 msgr2=0x7f728c1a1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:02:38.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:38.210+0000 7f7291df2700 1 --2- 172.21.15.5:0/2951255723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f728c10c5e0 0x7f728c1a1910 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f7284007ee0 tx=0x7f728400d3b0 comp rx=0 tx=0).stop 2024-05-09T01:02:38.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:38.210+0000 7f7291df2700 1 -- 172.21.15.5:0/2951255723 shutdown_connections 2024-05-09T01:02:38.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:38.210+0000 7f7291df2700 1 --2- 172.21.15.5:0/2951255723 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f72780b2880 0x7f72780b4d90 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:38.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:38.210+0000 7f7291df2700 1 --2- 172.21.15.5:0/2951255723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f728c10c5e0 0x7f728c1a1910 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:38.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:38.210+0000 7f7291df2700 1 --2- 172.21.15.5:0/2951255723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f728c1065a0 0x7f728c19d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:38.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:38.210+0000 7f7291df2700 1 --2- 172.21.15.5:0/2951255723 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f728c106ef0 0x7f728c19d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:38.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:38.210+0000 7f7291df2700 1 -- 172.21.15.5:0/2951255723 >> 172.21.15.5:0/2951255723 conn(0x7f728c102080 msgr2=0x7f728c1037c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:02:38.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:38.210+0000 7f7291df2700 1 -- 172.21.15.5:0/2951255723 shutdown_connections 2024-05-09T01:02:38.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:38.210+0000 7f7291df2700 1 -- 172.21.15.5:0/2951255723 wait complete. 2024-05-09T01:02:38.859 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:38 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/2951255723' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T01:02:38.892 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph fs fail cephfs 2024-05-09T01:02:38.974 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:38 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/2951255723' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T01:02:39.006 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:38 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/2951255723' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T01:02:39.149 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:02:39.959 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:39 smithi005 ceph-mon[30365]: pgmap v860: 97 pgs: 97 active+clean; 4.6 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.3 KiB/s wr, 24 op/s 2024-05-09T01:02:40.105 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:39 smithi131 ceph-mon[34547]: pgmap v860: 97 pgs: 97 active+clean; 4.6 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.3 KiB/s wr, 24 op/s 2024-05-09T01:02:40.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:39 smithi157 ceph-mon[34294]: pgmap v860: 97 pgs: 97 active+clean; 4.6 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.3 KiB/s wr, 24 op/s 2024-05-09T01:02:40.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:40.935+0000 7f180adcf700 1 -- 172.21.15.5:0/4173159744 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1804107e30 msgr2=0x7f1804108240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:02:40.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:40.935+0000 7f180adcf700 1 --2- 172.21.15.5:0/4173159744 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1804107e30 0x7f1804108240 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f17f00099c0 tx=0x7f17f0009cd0 comp rx=0 tx=0).stop 2024-05-09T01:02:40.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:40.935+0000 7f180adcf700 1 -- 172.21.15.5:0/4173159744 shutdown_connections 2024-05-09T01:02:40.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:40.935+0000 7f180adcf700 1 --2- 172.21.15.5:0/4173159744 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1804108780 0x7f18041100e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:40.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:40.935+0000 7f180adcf700 1 --2- 172.21.15.5:0/4173159744 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1804103550 0x7f1804103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:40.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:40.935+0000 7f180adcf700 1 --2- 172.21.15.5:0/4173159744 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1804107e30 0x7f1804108240 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:40.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:40.935+0000 7f180adcf700 1 -- 172.21.15.5:0/4173159744 >> 172.21.15.5:0/4173159744 conn(0x7f1804078e00 msgr2=0x7f1804079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:02:40.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:40.935+0000 7f180adcf700 1 -- 172.21.15.5:0/4173159744 shutdown_connections 2024-05-09T01:02:40.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:40.935+0000 7f180adcf700 1 -- 172.21.15.5:0/4173159744 wait complete. 2024-05-09T01:02:40.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:40.936+0000 7f180adcf700 1 Processor -- start 2024-05-09T01:02:40.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:40.936+0000 7f180adcf700 1 -- start start 2024-05-09T01:02:40.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:40.936+0000 7f180adcf700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1804103550 0x7f180419bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:02:40.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:40.936+0000 7f180adcf700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1804107e30 0x7f180419c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:02:40.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:40.936+0000 7f180adcf700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1804108780 0x7f180419c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:02:40.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:40.937+0000 7f180adcf700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1804113390 con 0x7f1804107e30 2024-05-09T01:02:40.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:40.937+0000 7f180adcf700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1804113210 con 0x7f1804103550 2024-05-09T01:02:40.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:40.937+0000 7f180adcf700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1804113510 con 0x7f1804108780 2024-05-09T01:02:40.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:40.937+0000 7f1803fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1804107e30 0x7f180419c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:02:40.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:40.937+0000 7f1803fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1804107e30 0x7f180419c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57596/0 (socket says 172.21.15.5:57596) 2024-05-09T01:02:40.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:40.937+0000 7f1803fff700 1 -- 172.21.15.5:0/3489239680 learned_addr learned my addr 172.21.15.5:0/3489239680 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:02:40.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:40.937+0000 7f180936c700 1 --2- 172.21.15.5:0/3489239680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1804108780 0x7f180419c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:02:40.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:40.937+0000 7f1808b6b700 1 --2- 172.21.15.5:0/3489239680 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1804103550 0x7f180419bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:02:40.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:40.937+0000 7f1803fff700 1 -- 172.21.15.5:0/3489239680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1804108780 msgr2=0x7f180419c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:02:40.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:40.937+0000 7f1803fff700 1 --2- 172.21.15.5:0/3489239680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1804108780 0x7f180419c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:40.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:40.937+0000 7f1803fff700 1 -- 172.21.15.5:0/3489239680 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1804103550 msgr2=0x7f180419bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:02:40.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:40.937+0000 7f1803fff700 1 --2- 172.21.15.5:0/3489239680 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1804103550 0x7f180419bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:40.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:40.937+0000 7f1803fff700 1 -- 172.21.15.5:0/3489239680 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18041a6fe0 con 0x7f1804107e30 2024-05-09T01:02:40.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:40.937+0000 7f180936c700 1 --2- 172.21.15.5:0/3489239680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1804108780 0x7f180419c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T01:02:40.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:40.937+0000 7f1803fff700 1 --2- 172.21.15.5:0/3489239680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1804107e30 0x7f180419c1d0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f17f00045e0 tx=0x7f17f0004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:02:40.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:40.938+0000 7f1801ffb700 1 -- 172.21.15.5:0/3489239680 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f0026070 con 0x7f1804107e30 2024-05-09T01:02:40.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:40.938+0000 7f180adcf700 1 -- 172.21.15.5:0/3489239680 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1804199a80 con 0x7f1804107e30 2024-05-09T01:02:40.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:40.938+0000 7f1801ffb700 1 -- 172.21.15.5:0/3489239680 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17f000f460 con 0x7f1804107e30 2024-05-09T01:02:40.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:40.938+0000 7f1801ffb700 1 -- 172.21.15.5:0/3489239680 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f0009e60 con 0x7f1804107e30 2024-05-09T01:02:40.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:40.938+0000 7f180adcf700 1 -- 172.21.15.5:0/3489239680 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1804199fb0 con 0x7f1804107e30 2024-05-09T01:02:40.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:40.939+0000 7f180adcf700 1 -- 172.21.15.5:0/3489239680 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18040512c0 con 0x7f1804107e30 2024-05-09T01:02:40.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:40.941+0000 7f1801ffb700 1 -- 172.21.15.5:0/3489239680 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f17f0029070 con 0x7f1804107e30 2024-05-09T01:02:40.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:40.941+0000 7f1801ffb700 1 --2- 172.21.15.5:0/3489239680 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f17f40b2900 0x7f17f40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:02:40.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:40.942+0000 7f1808b6b700 1 --2- 172.21.15.5:0/3489239680 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f17f40b2900 0x7f17f40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:02:40.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:40.942+0000 7f1801ffb700 1 -- 172.21.15.5:0/3489239680 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8285+0+0 (secure 0 0 0) 0x7f17f00e2ef0 con 0x7f1804107e30 2024-05-09T01:02:40.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:40.942+0000 7f1808b6b700 1 --2- 172.21.15.5:0/3489239680 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f17f40b2900 0x7f17f40b4e10 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f17f8005950 tx=0x7f17f800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:02:40.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:40.945+0000 7f1801ffb700 1 -- 172.21.15.5:0/3489239680 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f17f0013070 con 0x7f1804107e30 2024-05-09T01:02:41.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:41.208+0000 7f180adcf700 1 -- 172.21.15.5:0/3489239680 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f180402e5f0 con 0x7f1804107e30 2024-05-09T01:02:41.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:41.210+0000 7f1801ffb700 1 -- 172.21.15.5:0/3489239680 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v35) ==== 161+0+0 (secure 0 0 0) 0x7f17f00aac90 con 0x7f1804107e30 2024-05-09T01:02:41.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:41.214+0000 7f180adcf700 1 -- 172.21.15.5:0/3489239680 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f17f40b2900 msgr2=0x7f17f40b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:02:41.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:41.214+0000 7f180adcf700 1 --2- 172.21.15.5:0/3489239680 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f17f40b2900 0x7f17f40b4e10 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f17f8005950 tx=0x7f17f800b410 comp rx=0 tx=0).stop 2024-05-09T01:02:41.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:41.214+0000 7f180adcf700 1 -- 172.21.15.5:0/3489239680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1804107e30 msgr2=0x7f180419c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:02:41.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:41.214+0000 7f180adcf700 1 --2- 172.21.15.5:0/3489239680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1804107e30 0x7f180419c1d0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f17f00045e0 tx=0x7f17f0004610 comp rx=0 tx=0).stop 2024-05-09T01:02:41.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:41.214+0000 7f180adcf700 1 -- 172.21.15.5:0/3489239680 shutdown_connections 2024-05-09T01:02:41.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:41.214+0000 7f180adcf700 1 --2- 172.21.15.5:0/3489239680 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f17f40b2900 0x7f17f40b4e10 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:41.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:41.214+0000 7f180adcf700 1 --2- 172.21.15.5:0/3489239680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1804108780 0x7f180419c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:41.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:41.214+0000 7f180adcf700 1 --2- 172.21.15.5:0/3489239680 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1804103550 0x7f180419bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:41.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:41.214+0000 7f180adcf700 1 --2- 172.21.15.5:0/3489239680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1804107e30 0x7f180419c1d0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:41.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:41.214+0000 7f180adcf700 1 -- 172.21.15.5:0/3489239680 >> 172.21.15.5:0/3489239680 conn(0x7f1804078e00 msgr2=0x7f1804100f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:02:41.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:41.214+0000 7f180adcf700 1 -- 172.21.15.5:0/3489239680 shutdown_connections 2024-05-09T01:02:41.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:41.214+0000 7f180adcf700 1 -- 172.21.15.5:0/3489239680 wait complete. 2024-05-09T01:02:41.216 INFO:teuthology.orchestra.run.smithi005.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-09T01:02:41.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:41 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mon-a[30361]: 2024-05-09T01:02:41.209+0000 7fa63db8e700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T01:02:41.541 INFO:journalctl@ceph.mds.a.smithi005.stdout:May 09 01:02:41 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-a[72705]: ignoring --setuser ceph since I am not root 2024-05-09T01:02:41.541 INFO:journalctl@ceph.mds.a.smithi005.stdout:May 09 01:02:41 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-a[72705]: ignoring --setgroup ceph since I am not root 2024-05-09T01:02:41.542 INFO:journalctl@ceph.mds.a.smithi005.stdout:May 09 01:02:41 smithi005 ceph-mds[72709]: -- 172.21.15.5:0/1548477053 <== mon.0 v2:172.21.15.5:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562d56e1ba40 con 0x562d56e63180 2024-05-09T01:02:41.542 INFO:journalctl@ceph.mds.a.smithi005.stdout:May 09 01:02:41 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-a[72705]: starting mds.a at 2024-05-09T01:02:41.542 INFO:journalctl@ceph.mds.g.smithi005.stdout:May 09 01:02:41 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-g[72281]: ignoring --setuser ceph since I am not root 2024-05-09T01:02:41.542 INFO:journalctl@ceph.mds.g.smithi005.stdout:May 09 01:02:41 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-g[72281]: ignoring --setgroup ceph since I am not root 2024-05-09T01:02:41.542 INFO:journalctl@ceph.mds.g.smithi005.stdout:May 09 01:02:41 smithi005 ceph-mds[72292]: -- 172.21.15.5:0/1176511418 <== mon.0 v2:172.21.15.5:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55861c917a40 con 0x55861c95f600 2024-05-09T01:02:41.542 INFO:journalctl@ceph.mds.g.smithi005.stdout:May 09 01:02:41 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-g[72281]: starting mds.g at 2024-05-09T01:02:41.689 INFO:journalctl@ceph.mds.c.smithi157.stdout:May 09 01:02:41 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-c[64565]: ignoring --setuser ceph since I am not root 2024-05-09T01:02:41.689 INFO:journalctl@ceph.mds.c.smithi157.stdout:May 09 01:02:41 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-c[64565]: ignoring --setgroup ceph since I am not root 2024-05-09T01:02:41.689 INFO:journalctl@ceph.mds.c.smithi157.stdout:May 09 01:02:41 smithi157 ceph-mds[64569]: -- 172.21.15.157:0/3115809979 <== mon.2 v2:172.21.15.131:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5654e983fa40 con 0x5654e9887180 2024-05-09T01:02:41.689 INFO:journalctl@ceph.mds.c.smithi157.stdout:May 09 01:02:41 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-c[64565]: starting mds.c at 2024-05-09T01:02:41.690 INFO:journalctl@ceph.mds.i.smithi157.stdout:May 09 01:02:41 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-i[64868]: ignoring --setuser ceph since I am not root 2024-05-09T01:02:41.690 INFO:journalctl@ceph.mds.i.smithi157.stdout:May 09 01:02:41 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-i[64868]: ignoring --setgroup ceph since I am not root 2024-05-09T01:02:41.690 INFO:journalctl@ceph.mds.i.smithi157.stdout:May 09 01:02:41 smithi157 ceph-mds[64872]: -- 172.21.15.157:0/3177484365 <== mon.2 v2:172.21.15.131:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ec07d07a40 con 0x55ec07d4f600 2024-05-09T01:02:41.690 INFO:journalctl@ceph.mds.i.smithi157.stdout:May 09 01:02:41 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-i[64868]: starting mds.i at 2024-05-09T01:02:41.691 INFO:journalctl@ceph.mds.f.smithi157.stdout:May 09 01:02:41 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-f[64254]: ignoring --setuser ceph since I am not root 2024-05-09T01:02:41.691 INFO:journalctl@ceph.mds.f.smithi157.stdout:May 09 01:02:41 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-f[64254]: ignoring --setgroup ceph since I am not root 2024-05-09T01:02:41.691 INFO:journalctl@ceph.mds.f.smithi157.stdout:May 09 01:02:41 smithi157 ceph-mds[64258]: -- 172.21.15.157:0/2541942515 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fba7913a40 con 0x55fba795b600 2024-05-09T01:02:41.691 INFO:journalctl@ceph.mds.f.smithi157.stdout:May 09 01:02:41 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-f[64254]: starting mds.f at 2024-05-09T01:02:41.691 INFO:journalctl@ceph.mds.l.smithi157.stdout:May 09 01:02:41 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-l[63935]: ignoring --setuser ceph since I am not root 2024-05-09T01:02:41.691 INFO:journalctl@ceph.mds.l.smithi157.stdout:May 09 01:02:41 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-l[63935]: ignoring --setgroup ceph since I am not root 2024-05-09T01:02:41.691 INFO:journalctl@ceph.mds.l.smithi157.stdout:May 09 01:02:41 smithi157 ceph-mds[63939]: -- 172.21.15.157:0/1316417023 <== mon.0 v2:172.21.15.5:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562d3e2d3a40 con 0x562d3e31b180 2024-05-09T01:02:41.691 INFO:journalctl@ceph.mds.l.smithi157.stdout:May 09 01:02:41 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-l[63935]: starting mds.l at 2024-05-09T01:02:41.855 INFO:journalctl@ceph.mds.h.smithi131.stdout:May 09 01:02:41 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-h[64299]: ignoring --setuser ceph since I am not root 2024-05-09T01:02:41.856 INFO:journalctl@ceph.mds.h.smithi131.stdout:May 09 01:02:41 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-h[64299]: ignoring --setgroup ceph since I am not root 2024-05-09T01:02:41.856 INFO:journalctl@ceph.mds.h.smithi131.stdout:May 09 01:02:41 smithi131 ceph-mds[64303]: -- 172.21.15.131:0/1503889811 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563d25ec7a40 con 0x563d25f0fa80 2024-05-09T01:02:41.856 INFO:journalctl@ceph.mds.h.smithi131.stdout:May 09 01:02:41 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-h[64299]: starting mds.h at 2024-05-09T01:02:41.856 INFO:journalctl@ceph.mds.k.smithi131.stdout:May 09 01:02:41 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-k[63667]: ignoring --setuser ceph since I am not root 2024-05-09T01:02:41.856 INFO:journalctl@ceph.mds.k.smithi131.stdout:May 09 01:02:41 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-k[63667]: ignoring --setgroup ceph since I am not root 2024-05-09T01:02:41.856 INFO:journalctl@ceph.mds.k.smithi131.stdout:May 09 01:02:41 smithi131 ceph-mds[63671]: -- 172.21.15.131:0/1674739000 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5609036084e0 con 0x560903653600 2024-05-09T01:02:41.856 INFO:journalctl@ceph.mds.k.smithi131.stdout:May 09 01:02:41 smithi131 ceph-mds[63671]: -- 172.21.15.131:0/1674739000 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56090360ba40 con 0x560903653600 2024-05-09T01:02:41.856 INFO:journalctl@ceph.mds.k.smithi131.stdout:May 09 01:02:41 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-k[63667]: starting mds.k at 2024-05-09T01:02:41.857 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:41 smithi131 ceph-mon[34547]: pgmap v861: 97 pgs: 97 active+clean; 4.6 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-09T01:02:41.857 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:41 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/3489239680' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-09T01:02:41.857 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:41 smithi131 ceph-mon[34547]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-09T01:02:41.857 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:41 smithi131 ceph-mon[34547]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T01:02:41.857 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:41 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/3489239680' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-09T01:02:41.857 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:41 smithi131 ceph-mon[34547]: osdmap e83: 12 total, 12 up, 12 in 2024-05-09T01:02:41.857 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:41 smithi131 ceph-mon[34547]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-09T01:02:41.857 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:41 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T01:02:41.857 INFO:journalctl@ceph.mds.b.smithi131.stdout:May 09 01:02:41 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-b[64607]: ignoring --setuser ceph since I am not root 2024-05-09T01:02:41.857 INFO:journalctl@ceph.mds.b.smithi131.stdout:May 09 01:02:41 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-b[64607]: ignoring --setgroup ceph since I am not root 2024-05-09T01:02:41.858 INFO:journalctl@ceph.mds.b.smithi131.stdout:May 09 01:02:41 smithi131 ceph-mds[64611]: -- 172.21.15.131:0/1351019356 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55845653c4e0 con 0x558456587600 2024-05-09T01:02:41.858 INFO:journalctl@ceph.mds.b.smithi131.stdout:May 09 01:02:41 smithi131 ceph-mds[64611]: -- 172.21.15.131:0/1351019356 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55845653fa40 con 0x558456587600 2024-05-09T01:02:41.858 INFO:journalctl@ceph.mds.b.smithi131.stdout:May 09 01:02:41 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-b[64607]: starting mds.b at 2024-05-09T01:02:41.858 INFO:journalctl@ceph.mds.e.smithi131.stdout:May 09 01:02:41 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-e[63988]: ignoring --setuser ceph since I am not root 2024-05-09T01:02:41.858 INFO:journalctl@ceph.mds.e.smithi131.stdout:May 09 01:02:41 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-e[63988]: ignoring --setgroup ceph since I am not root 2024-05-09T01:02:41.858 INFO:journalctl@ceph.mds.e.smithi131.stdout:May 09 01:02:41 smithi131 ceph-mds[63992]: -- 172.21.15.131:0/3970035595 <== mon.2 v2:172.21.15.131:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563a68c21a40 con 0x563a68c69a80 2024-05-09T01:02:41.858 INFO:journalctl@ceph.mds.e.smithi131.stdout:May 09 01:02:41 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-e[63988]: starting mds.e at 2024-05-09T01:02:41.979 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:41 smithi005 ceph-mon[30365]: pgmap v861: 97 pgs: 97 active+clean; 4.6 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-09T01:02:41.979 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:41 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/3489239680' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-09T01:02:41.979 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:41 smithi005 ceph-mon[30365]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-09T01:02:41.979 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:41 smithi005 ceph-mon[30365]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T01:02:41.979 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:41 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/3489239680' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-09T01:02:41.979 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:41 smithi005 ceph-mon[30365]: osdmap e83: 12 total, 12 up, 12 in 2024-05-09T01:02:41.979 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:41 smithi005 ceph-mon[30365]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-09T01:02:41.979 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:41 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T01:02:42.011 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-05-09T01:02:42.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:41 smithi157 ceph-mon[34294]: pgmap v861: 97 pgs: 97 active+clean; 4.6 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-09T01:02:42.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:41 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/3489239680' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-09T01:02:42.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:41 smithi157 ceph-mon[34294]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-09T01:02:42.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:41 smithi157 ceph-mon[34294]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T01:02:42.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:41 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/3489239680' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-09T01:02:42.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:41 smithi157 ceph-mon[34294]: osdmap e83: 12 total, 12 up, 12 in 2024-05-09T01:02:42.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:41 smithi157 ceph-mon[34294]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-09T01:02:42.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:41 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T01:02:42.274 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:02:43.538 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:43 smithi005 ceph-mon[30365]: mds.? [v2:172.21.15.157:6833/158585272,v1:172.21.15.157:6835/158585272] up:boot 2024-05-09T01:02:43.538 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:43 smithi005 ceph-mon[30365]: mds.? [v2:172.21.15.157:6832/2057596278,v1:172.21.15.157:6834/2057596278] up:boot 2024-05-09T01:02:43.538 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:43 smithi005 ceph-mon[30365]: mds.? [v2:172.21.15.157:6836/1223172480,v1:172.21.15.157:6838/1223172480] up:boot 2024-05-09T01:02:43.538 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:43 smithi005 ceph-mon[30365]: mds.? [v2:172.21.15.157:6837/2782560766,v1:172.21.15.157:6839/2782560766] up:boot 2024-05-09T01:02:43.538 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:43 smithi005 ceph-mon[30365]: mds.? [v2:172.21.15.5:6838/2169062886,v1:172.21.15.5:6840/2169062886] up:boot 2024-05-09T01:02:43.538 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:43 smithi005 ceph-mon[30365]: mds.? [v2:172.21.15.5:6839/3286494978,v1:172.21.15.5:6841/3286494978] up:boot 2024-05-09T01:02:43.539 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:43 smithi005 ceph-mon[30365]: mds.? [v2:172.21.15.131:6832/3501320491,v1:172.21.15.131:6836/3501320491] up:boot 2024-05-09T01:02:43.539 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:43 smithi005 ceph-mon[30365]: mds.? [v2:172.21.15.131:6833/2906636136,v1:172.21.15.131:6837/2906636136] up:boot 2024-05-09T01:02:43.539 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:43 smithi005 ceph-mon[30365]: mds.? [v2:172.21.15.131:6835/2584779580,v1:172.21.15.131:6839/2584779580] up:boot 2024-05-09T01:02:43.539 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:43 smithi005 ceph-mon[30365]: mds.? [v2:172.21.15.131:6834/3476581633,v1:172.21.15.131:6838/3476581633] up:boot 2024-05-09T01:02:43.539 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:43 smithi005 ceph-mon[30365]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-05-09T01:02:43.539 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:43 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-09T01:02:43.539 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:43 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-09T01:02:43.539 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:43 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-09T01:02:43.539 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:43 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T01:02:43.539 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:43 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T01:02:43.539 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:43 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T01:02:43.539 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:43 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-09T01:02:43.539 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:43 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T01:02:43.539 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:43 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T01:02:43.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:43 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-09T01:02:43.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:43 smithi005 ceph-mon[30365]: pgmap v863: 97 pgs: 97 active+clean; 4.6 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 409 B/s wr, 10 op/s 2024-05-09T01:02:43.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:43 smithi131 ceph-mon[34547]: mds.? [v2:172.21.15.157:6833/158585272,v1:172.21.15.157:6835/158585272] up:boot 2024-05-09T01:02:43.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:43 smithi131 ceph-mon[34547]: mds.? [v2:172.21.15.157:6832/2057596278,v1:172.21.15.157:6834/2057596278] up:boot 2024-05-09T01:02:43.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:43 smithi131 ceph-mon[34547]: mds.? [v2:172.21.15.157:6836/1223172480,v1:172.21.15.157:6838/1223172480] up:boot 2024-05-09T01:02:43.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:43 smithi131 ceph-mon[34547]: mds.? [v2:172.21.15.157:6837/2782560766,v1:172.21.15.157:6839/2782560766] up:boot 2024-05-09T01:02:43.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:43 smithi131 ceph-mon[34547]: mds.? [v2:172.21.15.5:6838/2169062886,v1:172.21.15.5:6840/2169062886] up:boot 2024-05-09T01:02:43.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:43 smithi131 ceph-mon[34547]: mds.? [v2:172.21.15.5:6839/3286494978,v1:172.21.15.5:6841/3286494978] up:boot 2024-05-09T01:02:43.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:43 smithi131 ceph-mon[34547]: mds.? [v2:172.21.15.131:6832/3501320491,v1:172.21.15.131:6836/3501320491] up:boot 2024-05-09T01:02:43.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:43 smithi131 ceph-mon[34547]: mds.? [v2:172.21.15.131:6833/2906636136,v1:172.21.15.131:6837/2906636136] up:boot 2024-05-09T01:02:43.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:43 smithi131 ceph-mon[34547]: mds.? [v2:172.21.15.131:6835/2584779580,v1:172.21.15.131:6839/2584779580] up:boot 2024-05-09T01:02:43.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:43 smithi131 ceph-mon[34547]: mds.? [v2:172.21.15.131:6834/3476581633,v1:172.21.15.131:6838/3476581633] up:boot 2024-05-09T01:02:43.607 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:43 smithi131 ceph-mon[34547]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-05-09T01:02:43.607 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:43 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-09T01:02:43.607 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:43 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-09T01:02:43.607 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:43 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-09T01:02:43.607 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:43 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T01:02:43.607 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:43 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T01:02:43.607 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:43 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T01:02:43.607 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:43 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-09T01:02:43.607 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:43 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T01:02:43.607 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:43 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T01:02:43.607 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:43 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-09T01:02:43.607 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:43 smithi131 ceph-mon[34547]: pgmap v863: 97 pgs: 97 active+clean; 4.6 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 409 B/s wr, 10 op/s 2024-05-09T01:02:43.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:43 smithi157 ceph-mon[34294]: mds.? [v2:172.21.15.157:6833/158585272,v1:172.21.15.157:6835/158585272] up:boot 2024-05-09T01:02:43.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:43 smithi157 ceph-mon[34294]: mds.? [v2:172.21.15.157:6832/2057596278,v1:172.21.15.157:6834/2057596278] up:boot 2024-05-09T01:02:43.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:43 smithi157 ceph-mon[34294]: mds.? [v2:172.21.15.157:6836/1223172480,v1:172.21.15.157:6838/1223172480] up:boot 2024-05-09T01:02:43.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:43 smithi157 ceph-mon[34294]: mds.? [v2:172.21.15.157:6837/2782560766,v1:172.21.15.157:6839/2782560766] up:boot 2024-05-09T01:02:43.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:43 smithi157 ceph-mon[34294]: mds.? [v2:172.21.15.5:6838/2169062886,v1:172.21.15.5:6840/2169062886] up:boot 2024-05-09T01:02:43.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:43 smithi157 ceph-mon[34294]: mds.? [v2:172.21.15.5:6839/3286494978,v1:172.21.15.5:6841/3286494978] up:boot 2024-05-09T01:02:43.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:43 smithi157 ceph-mon[34294]: mds.? [v2:172.21.15.131:6832/3501320491,v1:172.21.15.131:6836/3501320491] up:boot 2024-05-09T01:02:43.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:43 smithi157 ceph-mon[34294]: mds.? [v2:172.21.15.131:6833/2906636136,v1:172.21.15.131:6837/2906636136] up:boot 2024-05-09T01:02:43.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:43 smithi157 ceph-mon[34294]: mds.? [v2:172.21.15.131:6835/2584779580,v1:172.21.15.131:6839/2584779580] up:boot 2024-05-09T01:02:43.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:43 smithi157 ceph-mon[34294]: mds.? [v2:172.21.15.131:6834/3476581633,v1:172.21.15.131:6838/3476581633] up:boot 2024-05-09T01:02:43.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:43 smithi157 ceph-mon[34294]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-05-09T01:02:43.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:43 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-09T01:02:43.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:43 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-09T01:02:43.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:43 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-09T01:02:43.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:43 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T01:02:43.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:43 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T01:02:43.688 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:43 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T01:02:43.688 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:43 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-09T01:02:43.688 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:43 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T01:02:43.688 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:43 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T01:02:43.688 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:43 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-09T01:02:43.688 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:43 smithi157 ceph-mon[34294]: pgmap v863: 97 pgs: 97 active+clean; 4.6 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 409 B/s wr, 10 op/s 2024-05-09T01:02:44.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:44.004+0000 7f1712ecd700 1 -- 172.21.15.5:0/1828089247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f170c105500 msgr2=0x7f170c105910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:02:44.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:44.004+0000 7f1712ecd700 1 --2- 172.21.15.5:0/1828089247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f170c105500 0x7f170c105910 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f1700009990 tx=0x7f1700009ca0 comp rx=0 tx=0).stop 2024-05-09T01:02:44.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:44.004+0000 7f1712ecd700 1 -- 172.21.15.5:0/1828089247 shutdown_connections 2024-05-09T01:02:44.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:44.004+0000 7f1712ecd700 1 --2- 172.21.15.5:0/1828089247 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f170c10e930 0x7f170c110e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:44.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:44.004+0000 7f1712ecd700 1 --2- 172.21.15.5:0/1828089247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f170c105500 0x7f170c105910 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:44.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:44.004+0000 7f1712ecd700 1 --2- 172.21.15.5:0/1828089247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f170c105e50 0x7f170c10e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:44.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:44.004+0000 7f1712ecd700 1 -- 172.21.15.5:0/1828089247 >> 172.21.15.5:0/1828089247 conn(0x7f170c0fef30 msgr2=0x7f170c101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:02:44.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:44.004+0000 7f1712ecd700 1 -- 172.21.15.5:0/1828089247 shutdown_connections 2024-05-09T01:02:44.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:44.004+0000 7f1712ecd700 1 -- 172.21.15.5:0/1828089247 wait complete. 2024-05-09T01:02:44.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:44.005+0000 7f1712ecd700 1 Processor -- start 2024-05-09T01:02:44.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:44.005+0000 7f1712ecd700 1 -- start start 2024-05-09T01:02:44.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:44.005+0000 7f1712ecd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f170c105500 0x7f170c1031f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:02:44.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:44.005+0000 7f1712ecd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f170c105e50 0x7f170c103730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:02:44.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:44.005+0000 7f1712ecd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f170c10e930 0x7f170c103c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:02:44.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:44.005+0000 7f1712ecd700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f170c114450 con 0x7f170c10e930 2024-05-09T01:02:44.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:44.005+0000 7f1712ecd700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f170c1142d0 con 0x7f170c105e50 2024-05-09T01:02:44.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:44.005+0000 7f1712ecd700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f170c1145d0 con 0x7f170c105500 2024-05-09T01:02:44.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:44.005+0000 7f171146a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f170c10e930 0x7f170c103c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:02:44.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:44.005+0000 7f171146a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f170c10e930 0x7f170c103c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44156/0 (socket says 172.21.15.5:44156) 2024-05-09T01:02:44.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:44.005+0000 7f171146a700 1 -- 172.21.15.5:0/1985686775 learned_addr learned my addr 172.21.15.5:0/1985686775 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:02:44.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:44.005+0000 7f1710c69700 1 --2- 172.21.15.5:0/1985686775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f170c105500 0x7f170c1031f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:02:44.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:44.006+0000 7f170bfff700 1 --2- 172.21.15.5:0/1985686775 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f170c105e50 0x7f170c103730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:02:44.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:44.006+0000 7f171146a700 1 -- 172.21.15.5:0/1985686775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f170c105500 msgr2=0x7f170c1031f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:02:44.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:44.006+0000 7f171146a700 1 --2- 172.21.15.5:0/1985686775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f170c105500 0x7f170c1031f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:44.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:44.006+0000 7f171146a700 1 -- 172.21.15.5:0/1985686775 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f170c105e50 msgr2=0x7f170c103730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:02:44.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:44.006+0000 7f171146a700 1 --2- 172.21.15.5:0/1985686775 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f170c105e50 0x7f170c103730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:44.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:44.006+0000 7f171146a700 1 -- 172.21.15.5:0/1985686775 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f170c1a9150 con 0x7f170c10e930 2024-05-09T01:02:44.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:44.006+0000 7f171146a700 1 --2- 172.21.15.5:0/1985686775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f170c10e930 0x7f170c103c70 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f170400ada0 tx=0x7f170400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:02:44.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:44.006+0000 7f1709ffb700 1 -- 172.21.15.5:0/1985686775 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1704017020 con 0x7f170c10e930 2024-05-09T01:02:44.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:44.006+0000 7f1712ecd700 1 -- 172.21.15.5:0/1985686775 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f170c1a9440 con 0x7f170c10e930 2024-05-09T01:02:44.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:44.006+0000 7f1709ffb700 1 -- 172.21.15.5:0/1985686775 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f170400ce30 con 0x7f170c10e930 2024-05-09T01:02:44.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:44.006+0000 7f1709ffb700 1 -- 172.21.15.5:0/1985686775 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1704005350 con 0x7f170c10e930 2024-05-09T01:02:44.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:44.006+0000 7f1712ecd700 1 -- 172.21.15.5:0/1985686775 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f170c1a9930 con 0x7f170c10e930 2024-05-09T01:02:44.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:44.007+0000 7f1712ecd700 1 -- 172.21.15.5:0/1985686775 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f170c0512c0 con 0x7f170c10e930 2024-05-09T01:02:44.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:44.009+0000 7f1709ffb700 1 -- 172.21.15.5:0/1985686775 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f170400c980 con 0x7f170c10e930 2024-05-09T01:02:44.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:44.010+0000 7f1709ffb700 1 --2- 172.21.15.5:0/1985686775 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f16fc0b2880 0x7f16fc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:02:44.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:44.010+0000 7f1710c69700 1 --2- 172.21.15.5:0/1985686775 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f16fc0b2880 0x7f16fc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:02:44.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:44.010+0000 7f1709ffb700 1 -- 172.21.15.5:0/1985686775 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8715+0+0 (secure 0 0 0) 0x7f17040e20c0 con 0x7f170c10e930 2024-05-09T01:02:44.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:44.010+0000 7f1710c69700 1 --2- 172.21.15.5:0/1985686775 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f16fc0b2880 0x7f16fc0b4d90 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f170000b5c0 tx=0x7f1700005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:02:44.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:44.013+0000 7f1709ffb700 1 -- 172.21.15.5:0/1985686775 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1704021020 con 0x7f170c10e930 2024-05-09T01:02:44.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:44.276+0000 7f1712ecd700 1 -- 172.21.15.5:0/1985686775 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f170c02e7b0 con 0x7f170c10e930 2024-05-09T01:02:44.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:44.328+0000 7f1709ffb700 1 -- 172.21.15.5:0/1985686775 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v37) ==== 104+0+0 (secure 0 0 0) 0x7f17040a9c30 con 0x7f170c10e930 2024-05-09T01:02:44.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:44.332+0000 7f1712ecd700 1 -- 172.21.15.5:0/1985686775 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f16fc0b2880 msgr2=0x7f16fc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:02:44.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:44.332+0000 7f1712ecd700 1 --2- 172.21.15.5:0/1985686775 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f16fc0b2880 0x7f16fc0b4d90 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f170000b5c0 tx=0x7f1700005960 comp rx=0 tx=0).stop 2024-05-09T01:02:44.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:44.332+0000 7f1712ecd700 1 -- 172.21.15.5:0/1985686775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f170c10e930 msgr2=0x7f170c103c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:02:44.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:44.332+0000 7f1712ecd700 1 --2- 172.21.15.5:0/1985686775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f170c10e930 0x7f170c103c70 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f170400ada0 tx=0x7f170400c330 comp rx=0 tx=0).stop 2024-05-09T01:02:44.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:44.332+0000 7f1712ecd700 1 -- 172.21.15.5:0/1985686775 shutdown_connections 2024-05-09T01:02:44.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:44.332+0000 7f1712ecd700 1 --2- 172.21.15.5:0/1985686775 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f16fc0b2880 0x7f16fc0b4d90 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:44.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:44.332+0000 7f1712ecd700 1 --2- 172.21.15.5:0/1985686775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f170c10e930 0x7f170c103c70 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:44.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:44.332+0000 7f1712ecd700 1 --2- 172.21.15.5:0/1985686775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f170c105500 0x7f170c1031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:44.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:44.332+0000 7f1712ecd700 1 --2- 172.21.15.5:0/1985686775 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f170c105e50 0x7f170c103730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:44.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:44.332+0000 7f1712ecd700 1 -- 172.21.15.5:0/1985686775 >> 172.21.15.5:0/1985686775 conn(0x7f170c0fef30 msgr2=0x7f170c0ff5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:02:44.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:44.332+0000 7f1712ecd700 1 -- 172.21.15.5:0/1985686775 shutdown_connections 2024-05-09T01:02:44.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:44.332+0000 7f1712ecd700 1 -- 172.21.15.5:0/1985686775 wait complete. 2024-05-09T01:02:44.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:44 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/1985686775' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T01:02:44.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:44 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/1985686775' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T01:02:44.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:44 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/1985686775' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T01:02:45.117 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-09T01:02:45.380 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:02:45.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:45 smithi131 ceph-mon[34547]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-09T01:02:45.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:45 smithi131 ceph-mon[34547]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-09T01:02:45.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:45 smithi131 ceph-mon[34547]: Cluster is now healthy 2024-05-09T01:02:45.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:45 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/1985686775' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-09T01:02:45.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:45 smithi131 ceph-mon[34547]: fsmap 12 up:standby 2024-05-09T01:02:45.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:45 smithi131 ceph-mon[34547]: pgmap v864: 97 pgs: 97 active+clean; 4.6 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-09T01:02:45.678 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:45 smithi005 ceph-mon[30365]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-09T01:02:45.678 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:45 smithi005 ceph-mon[30365]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-09T01:02:45.678 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:45 smithi005 ceph-mon[30365]: Cluster is now healthy 2024-05-09T01:02:45.678 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:45 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/1985686775' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-09T01:02:45.678 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:45 smithi005 ceph-mon[30365]: fsmap 12 up:standby 2024-05-09T01:02:45.678 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:45 smithi005 ceph-mon[30365]: pgmap v864: 97 pgs: 97 active+clean; 4.6 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-09T01:02:45.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:45 smithi157 ceph-mon[34294]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-09T01:02:45.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:45 smithi157 ceph-mon[34294]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-09T01:02:45.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:45 smithi157 ceph-mon[34294]: Cluster is now healthy 2024-05-09T01:02:45.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:45 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/1985686775' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-09T01:02:45.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:45 smithi157 ceph-mon[34294]: fsmap 12 up:standby 2024-05-09T01:02:45.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:45 smithi157 ceph-mon[34294]: pgmap v864: 97 pgs: 97 active+clean; 4.6 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-09T01:02:46.539 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:46 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:02:46.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:46 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:02:46.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:46 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T01:02:47.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:47.131+0000 7f4d10a99700 1 -- 172.21.15.5:0/843629917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d0c107e10 msgr2=0x7f4d0c108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:02:47.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:47.131+0000 7f4d10a99700 1 --2- 172.21.15.5:0/843629917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d0c107e10 0x7f4d0c108220 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f4d000099c0 tx=0x7f4d00009cd0 comp rx=0 tx=0).stop 2024-05-09T01:02:47.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:47.131+0000 7f4d10a99700 1 -- 172.21.15.5:0/843629917 shutdown_connections 2024-05-09T01:02:47.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:47.131+0000 7f4d10a99700 1 --2- 172.21.15.5:0/843629917 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d0c108760 0x7f4d0c1100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:47.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:47.131+0000 7f4d10a99700 1 --2- 172.21.15.5:0/843629917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d0c1034e0 0x7f4d0c1038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:47.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:47.131+0000 7f4d10a99700 1 --2- 172.21.15.5:0/843629917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d0c107e10 0x7f4d0c108220 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:47.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:47.131+0000 7f4d10a99700 1 -- 172.21.15.5:0/843629917 >> 172.21.15.5:0/843629917 conn(0x7f4d0c0fef50 msgr2=0x7f4d0c101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:02:47.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:47.131+0000 7f4d10a99700 1 -- 172.21.15.5:0/843629917 shutdown_connections 2024-05-09T01:02:47.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:47.131+0000 7f4d10a99700 1 -- 172.21.15.5:0/843629917 wait complete. 2024-05-09T01:02:47.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:47.132+0000 7f4d10a99700 1 Processor -- start 2024-05-09T01:02:47.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:47.132+0000 7f4d10a99700 1 -- start start 2024-05-09T01:02:47.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:47.133+0000 7f4d10a99700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d0c1034e0 0x7f4d0c19aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:02:47.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:47.133+0000 7f4d10a99700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d0c107e10 0x7f4d0c19b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:02:47.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:47.133+0000 7f4d10a99700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d0c108760 0x7f4d0c19f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:02:47.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:47.133+0000 7f4d10a99700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4d0c113370 con 0x7f4d0c1034e0 2024-05-09T01:02:47.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:47.133+0000 7f4d10a99700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4d0c1131f0 con 0x7f4d0c108760 2024-05-09T01:02:47.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:47.133+0000 7f4d10a99700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4d0c1134f0 con 0x7f4d0c107e10 2024-05-09T01:02:47.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:47.133+0000 7f4d0a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d0c1034e0 0x7f4d0c19aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:02:47.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:47.133+0000 7f4d0a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d0c1034e0 0x7f4d0c19aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44166/0 (socket says 172.21.15.5:44166) 2024-05-09T01:02:47.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:47.133+0000 7f4d0a59c700 1 -- 172.21.15.5:0/2059258342 learned_addr learned my addr 172.21.15.5:0/2059258342 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:02:47.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:47.133+0000 7f4d0ad9d700 1 --2- 172.21.15.5:0/2059258342 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d0c108760 0x7f4d0c19f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:02:47.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:47.133+0000 7f4d09d9b700 1 --2- 172.21.15.5:0/2059258342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d0c107e10 0x7f4d0c19b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:02:47.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:47.133+0000 7f4d0a59c700 1 -- 172.21.15.5:0/2059258342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d0c107e10 msgr2=0x7f4d0c19b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:02:47.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:47.133+0000 7f4d0a59c700 1 --2- 172.21.15.5:0/2059258342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d0c107e10 0x7f4d0c19b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:47.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:47.133+0000 7f4d0a59c700 1 -- 172.21.15.5:0/2059258342 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d0c108760 msgr2=0x7f4d0c19f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:02:47.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:47.133+0000 7f4d0a59c700 1 --2- 172.21.15.5:0/2059258342 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d0c108760 0x7f4d0c19f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:47.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:47.133+0000 7f4d0a59c700 1 -- 172.21.15.5:0/2059258342 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d0c19fdc0 con 0x7f4d0c1034e0 2024-05-09T01:02:47.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:47.133+0000 7f4d0a59c700 1 --2- 172.21.15.5:0/2059258342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d0c1034e0 0x7f4d0c19aef0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc00e9a0 tx=0x7f4cfc00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:02:47.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:47.134+0000 7f4cfb7fe700 1 -- 172.21.15.5:0/2059258342 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc00cbd0 con 0x7f4d0c1034e0 2024-05-09T01:02:47.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:47.134+0000 7f4d10a99700 1 -- 172.21.15.5:0/2059258342 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d0c1a00b0 con 0x7f4d0c1034e0 2024-05-09T01:02:47.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:47.134+0000 7f4cfb7fe700 1 -- 172.21.15.5:0/2059258342 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cfc00cd70 con 0x7f4d0c1034e0 2024-05-09T01:02:47.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:47.134+0000 7f4cfb7fe700 1 -- 172.21.15.5:0/2059258342 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc018680 con 0x7f4d0c1034e0 2024-05-09T01:02:47.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:47.134+0000 7f4d10a99700 1 -- 172.21.15.5:0/2059258342 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d0c1a79d0 con 0x7f4d0c1034e0 2024-05-09T01:02:47.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:47.135+0000 7f4d10a99700 1 -- 172.21.15.5:0/2059258342 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d0c0512c0 con 0x7f4d0c1034e0 2024-05-09T01:02:47.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:47.136+0000 7f4cfb7fe700 1 -- 172.21.15.5:0/2059258342 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f4cfc010430 con 0x7f4d0c1034e0 2024-05-09T01:02:47.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:47.137+0000 7f4cfb7fe700 1 --2- 172.21.15.5:0/2059258342 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f4cf40b2880 0x7f4cf40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:02:47.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:47.138+0000 7f4d09d9b700 1 --2- 172.21.15.5:0/2059258342 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f4cf40b2880 0x7f4cf40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:02:47.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:47.138+0000 7f4cfb7fe700 1 -- 172.21.15.5:0/2059258342 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8715+0+0 (secure 0 0 0) 0x7f4cfc0e27a0 con 0x7f4d0c1034e0 2024-05-09T01:02:47.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:47.138+0000 7f4d09d9b700 1 --2- 172.21.15.5:0/2059258342 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f4cf40b2880 0x7f4cf40b4d90 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f4d00000c00 tx=0x7f4d0001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:02:47.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:47.143+0000 7f4cfb7fe700 1 -- 172.21.15.5:0/2059258342 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4cfc0aa2e0 con 0x7f4d0c1034e0 2024-05-09T01:02:47.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:47.359+0000 7f4d10a99700 1 -- 172.21.15.5:0/2059258342 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4d0c1a7ba0 con 0x7f4d0c1034e0 2024-05-09T01:02:47.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:47 smithi005 ceph-mon[30365]: pgmap v865: 97 pgs: 97 active+clean; 4.6 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-05-09T01:02:47.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:47 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:02:47.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:47 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:02:47.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:47 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:02:47.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:47 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:02:47.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:47 smithi005 ceph-mon[30365]: Adjusting osd_memory_target on smithi005 to 177.2M 2024-05-09T01:02:47.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:47 smithi005 ceph-mon[30365]: Unable to set osd_memory_target on smithi005 to 185878425: error parsing value: Value '185878425' is below minimum 939524096 2024-05-09T01:02:47.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:47 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:02:47.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:47 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:02:47.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:47 smithi005 ceph-mon[30365]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T01:02:47.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:47 smithi131 ceph-mon[34547]: pgmap v865: 97 pgs: 97 active+clean; 4.6 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-05-09T01:02:47.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:47 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:02:47.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:47 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:02:47.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:47 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:02:47.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:47 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:02:47.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:47 smithi131 ceph-mon[34547]: Adjusting osd_memory_target on smithi005 to 177.2M 2024-05-09T01:02:47.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:47 smithi131 ceph-mon[34547]: Unable to set osd_memory_target on smithi005 to 185878425: error parsing value: Value '185878425' is below minimum 939524096 2024-05-09T01:02:47.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:47 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:02:47.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:47 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:02:47.606 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:47 smithi131 ceph-mon[34547]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T01:02:47.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:47 smithi157 ceph-mon[34294]: pgmap v865: 97 pgs: 97 active+clean; 4.6 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-05-09T01:02:47.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:47 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:02:47.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:47 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:02:47.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:47 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:02:47.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:47 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T01:02:47.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:47 smithi157 ceph-mon[34294]: Adjusting osd_memory_target on smithi005 to 177.2M 2024-05-09T01:02:47.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:47 smithi157 ceph-mon[34294]: Unable to set osd_memory_target on smithi005 to 185878425: error parsing value: Value '185878425' is below minimum 939524096 2024-05-09T01:02:47.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:47 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T01:02:47.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:47 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T01:02:47.687 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:47 smithi157 ceph-mon[34294]: from='mgr.14152 172.21.15.5:0/3103624851' entity='mgr.x' 2024-05-09T01:02:48.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:48.354+0000 7f4cfb7fe700 1 -- 172.21.15.5:0/2059258342 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v84) ==== 181+0+0 (secure 0 0 0) 0x7f4cfc0a9be0 con 0x7f4d0c1034e0 2024-05-09T01:02:48.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:48.357+0000 7f4d10a99700 1 -- 172.21.15.5:0/2059258342 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f4cf40b2880 msgr2=0x7f4cf40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:02:48.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:48.357+0000 7f4d10a99700 1 --2- 172.21.15.5:0/2059258342 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f4cf40b2880 0x7f4cf40b4d90 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f4d00000c00 tx=0x7f4d0001a040 comp rx=0 tx=0).stop 2024-05-09T01:02:48.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:48.357+0000 7f4d10a99700 1 -- 172.21.15.5:0/2059258342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d0c1034e0 msgr2=0x7f4d0c19aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:02:48.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:48.357+0000 7f4d10a99700 1 --2- 172.21.15.5:0/2059258342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d0c1034e0 0x7f4d0c19aef0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc00e9a0 tx=0x7f4cfc00ed60 comp rx=0 tx=0).stop 2024-05-09T01:02:48.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:48.358+0000 7f4d10a99700 1 -- 172.21.15.5:0/2059258342 shutdown_connections 2024-05-09T01:02:48.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:48.358+0000 7f4d10a99700 1 --2- 172.21.15.5:0/2059258342 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7f4cf40b2880 0x7f4cf40b4d90 secure :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f4d00000c00 tx=0x7f4d0001a040 comp rx=0 tx=0).stop 2024-05-09T01:02:48.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:48.358+0000 7f4d10a99700 1 --2- 172.21.15.5:0/2059258342 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d0c108760 0x7f4d0c19f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:48.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:48.358+0000 7f4d10a99700 1 --2- 172.21.15.5:0/2059258342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d0c1034e0 0x7f4d0c19aef0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:48.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:48.358+0000 7f4d10a99700 1 --2- 172.21.15.5:0/2059258342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d0c107e10 0x7f4d0c19b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:48.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:48.358+0000 7f4d10a99700 1 -- 172.21.15.5:0/2059258342 >> 172.21.15.5:0/2059258342 conn(0x7f4d0c0fef50 msgr2=0x7f4d0c101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:02:48.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:48.359+0000 7f4d10a99700 1 -- 172.21.15.5:0/2059258342 shutdown_connections 2024-05-09T01:02:48.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:48.359+0000 7f4d10a99700 1 -- 172.21.15.5:0/2059258342 wait complete. 2024-05-09T01:02:48.360 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' removed 2024-05-09T01:02:48.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:48 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/2059258342' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T01:02:48.605 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:48 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/2059258342' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T01:02:48.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:48 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/2059258342' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T01:02:49.065 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-09T01:02:49.327 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T01:02:49.612 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:49 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/2059258342' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-09T01:02:49.612 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:49 smithi005 ceph-mon[30365]: osdmap e84: 12 total, 12 up, 12 in 2024-05-09T01:02:49.612 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:49 smithi005 ceph-mon[30365]: pgmap v867: 65 pgs: 65 active+clean; 577 KiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:02:49.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:49 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/2059258342' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-09T01:02:49.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:49 smithi157 ceph-mon[34294]: osdmap e84: 12 total, 12 up, 12 in 2024-05-09T01:02:49.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:49 smithi157 ceph-mon[34294]: pgmap v867: 65 pgs: 65 active+clean; 577 KiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:02:49.789 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:49 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/2059258342' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-09T01:02:49.789 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:49 smithi131 ceph-mon[34547]: osdmap e84: 12 total, 12 up, 12 in 2024-05-09T01:02:49.789 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:49 smithi131 ceph-mon[34547]: pgmap v867: 65 pgs: 65 active+clean; 577 KiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:02:51.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:51.363+0000 7fe7f3499700 1 -- 172.21.15.5:0/3908138038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7ec110c50 msgr2=0x7fe7ec113070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:02:51.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:51.363+0000 7fe7f3499700 1 --2- 172.21.15.5:0/3908138038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7ec110c50 0x7fe7ec113070 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fe7e400b0a0 tx=0x7fe7e400b3b0 comp rx=0 tx=0).stop 2024-05-09T01:02:51.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:51.363+0000 7fe7f3499700 1 -- 172.21.15.5:0/3908138038 shutdown_connections 2024-05-09T01:02:51.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:51.363+0000 7fe7f3499700 1 --2- 172.21.15.5:0/3908138038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7ec110c50 0x7fe7ec113070 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:51.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:51.363+0000 7fe7f3499700 1 --2- 172.21.15.5:0/3908138038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7ec1030a0 0x7fe7ec1034b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:51.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:51.363+0000 7fe7f3499700 1 --2- 172.21.15.5:0/3908138038 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7ec103a80 0x7fe7ec110710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:51.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:51.363+0000 7fe7f3499700 1 -- 172.21.15.5:0/3908138038 >> 172.21.15.5:0/3908138038 conn(0x7fe7ec0fef30 msgr2=0x7fe7ec101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:02:51.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:51.363+0000 7fe7f3499700 1 -- 172.21.15.5:0/3908138038 shutdown_connections 2024-05-09T01:02:51.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:51.363+0000 7fe7f3499700 1 -- 172.21.15.5:0/3908138038 wait complete. 2024-05-09T01:02:51.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:51.364+0000 7fe7f3499700 1 Processor -- start 2024-05-09T01:02:51.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:51.364+0000 7fe7f3499700 1 -- start start 2024-05-09T01:02:51.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:51.364+0000 7fe7f3499700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7ec1030a0 0x7fe7ec19cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:02:51.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:51.365+0000 7fe7f3499700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7ec103a80 0x7fe7ec19d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:02:51.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:51.365+0000 7fe7f3499700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7ec110c50 0x7fe7ec1a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:02:51.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:51.365+0000 7fe7f3499700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec1166b0 con 0x7fe7ec1030a0 2024-05-09T01:02:51.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:51.365+0000 7fe7f3499700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec116530 con 0x7fe7ec110c50 2024-05-09T01:02:51.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:51.365+0000 7fe7f3499700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec116830 con 0x7fe7ec103a80 2024-05-09T01:02:51.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:51.365+0000 7fe7f1235700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7ec1030a0 0x7fe7ec19cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:02:51.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:51.365+0000 7fe7f1235700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7ec1030a0 0x7fe7ec19cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44184/0 (socket says 172.21.15.5:44184) 2024-05-09T01:02:51.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:51.365+0000 7fe7f1235700 1 -- 172.21.15.5:0/635590424 learned_addr learned my addr 172.21.15.5:0/635590424 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-09T01:02:51.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:51.365+0000 7fe7f0a34700 1 --2- 172.21.15.5:0/635590424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7ec103a80 0x7fe7ec19d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:02:51.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:51.365+0000 7fe7f1a36700 1 --2- 172.21.15.5:0/635590424 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7ec110c50 0x7fe7ec1a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:02:51.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:51.365+0000 7fe7f1235700 1 -- 172.21.15.5:0/635590424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7ec103a80 msgr2=0x7fe7ec19d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:02:51.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:51.365+0000 7fe7f1235700 1 --2- 172.21.15.5:0/635590424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7ec103a80 0x7fe7ec19d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:51.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:51.365+0000 7fe7f1235700 1 -- 172.21.15.5:0/635590424 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7ec110c50 msgr2=0x7fe7ec1a17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:02:51.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:51.365+0000 7fe7f1235700 1 --2- 172.21.15.5:0/635590424 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7ec110c50 0x7fe7ec1a17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:51.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:51.365+0000 7fe7f1235700 1 -- 172.21.15.5:0/635590424 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7ec1a1da0 con 0x7fe7ec1030a0 2024-05-09T01:02:51.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:51.365+0000 7fe7f1235700 1 --2- 172.21.15.5:0/635590424 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7ec1030a0 0x7fe7ec19cf80 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fe7e800c9b0 tx=0x7fe7e800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:02:51.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:51.366+0000 7fe7e27fc700 1 -- 172.21.15.5:0/635590424 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e8007b00 con 0x7fe7ec1030a0 2024-05-09T01:02:51.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:51.366+0000 7fe7e27fc700 1 -- 172.21.15.5:0/635590424 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7e800ce50 con 0x7fe7ec1030a0 2024-05-09T01:02:51.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:51.366+0000 7fe7f3499700 1 -- 172.21.15.5:0/635590424 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7ec1a2090 con 0x7fe7ec1030a0 2024-05-09T01:02:51.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:51.366+0000 7fe7e27fc700 1 -- 172.21.15.5:0/635590424 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e8018660 con 0x7fe7ec1030a0 2024-05-09T01:02:51.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:51.366+0000 7fe7f3499700 1 -- 172.21.15.5:0/635590424 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7ec1a25d0 con 0x7fe7ec1030a0 2024-05-09T01:02:51.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:51.367+0000 7fe7f3499700 1 -- 172.21.15.5:0/635590424 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7ec0512c0 con 0x7fe7ec1030a0 2024-05-09T01:02:51.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:51.368+0000 7fe7e27fc700 1 -- 172.21.15.5:0/635590424 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fe7e800f450 con 0x7fe7ec1030a0 2024-05-09T01:02:51.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:51.369+0000 7fe7e27fc700 1 --2- 172.21.15.5:0/635590424 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe7d80b2880 0x7fe7d80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T01:02:51.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:51.370+0000 7fe7f0a34700 1 --2- 172.21.15.5:0/635590424 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe7d80b2880 0x7fe7d80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T01:02:51.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:51.370+0000 7fe7e27fc700 1 -- 172.21.15.5:0/635590424 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8190+0+0 (secure 0 0 0) 0x7fe7e80e2100 con 0x7fe7ec1030a0 2024-05-09T01:02:51.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:51.370+0000 7fe7f0a34700 1 --2- 172.21.15.5:0/635590424 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe7d80b2880 0x7fe7d80b4d90 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec104980 tx=0x7fe7dc007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T01:02:51.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:51.375+0000 7fe7e27fc700 1 -- 172.21.15.5:0/635590424 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe7e8014030 con 0x7fe7ec1030a0 2024-05-09T01:02:51.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:51.588+0000 7fe7f3499700 1 -- 172.21.15.5:0/635590424 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe7ec19e310 con 0x7fe7ec1030a0 2024-05-09T01:02:51.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:51.638+0000 7fe7e27fc700 1 -- 172.21.15.5:0/635590424 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v85) ==== 169+0+0 (secure 0 0 0) 0x7fe7ec19e310 con 0x7fe7ec1030a0 2024-05-09T01:02:51.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:51.642+0000 7fe7f3499700 1 -- 172.21.15.5:0/635590424 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe7d80b2880 msgr2=0x7fe7d80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:02:51.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:51.642+0000 7fe7f3499700 1 --2- 172.21.15.5:0/635590424 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe7d80b2880 0x7fe7d80b4d90 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec104980 tx=0x7fe7dc007400 comp rx=0 tx=0).stop 2024-05-09T01:02:51.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:51.642+0000 7fe7f3499700 1 -- 172.21.15.5:0/635590424 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7ec1030a0 msgr2=0x7fe7ec19cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T01:02:51.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:51.642+0000 7fe7f3499700 1 --2- 172.21.15.5:0/635590424 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7ec1030a0 0x7fe7ec19cf80 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fe7e800c9b0 tx=0x7fe7e800ccc0 comp rx=0 tx=0).stop 2024-05-09T01:02:51.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:51.644+0000 7fe7f3499700 1 -- 172.21.15.5:0/635590424 shutdown_connections 2024-05-09T01:02:51.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:51.644+0000 7fe7f3499700 1 --2- 172.21.15.5:0/635590424 >> [v2:172.21.15.5:6800/91856593,v1:172.21.15.5:6801/91856593] conn(0x7fe7d80b2880 0x7fe7d80b4d90 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:51.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:51.644+0000 7fe7f3499700 1 --2- 172.21.15.5:0/635590424 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7ec110c50 0x7fe7ec1a17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:51.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:51.644+0000 7fe7f3499700 1 --2- 172.21.15.5:0/635590424 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7ec1030a0 0x7fe7ec19cf80 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:51.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:51.644+0000 7fe7f3499700 1 --2- 172.21.15.5:0/635590424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7ec103a80 0x7fe7ec19d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T01:02:51.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:51.644+0000 7fe7f3499700 1 -- 172.21.15.5:0/635590424 >> 172.21.15.5:0/635590424 conn(0x7fe7ec0fef30 msgr2=0x7fe7ec1126c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T01:02:51.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:51.645+0000 7fe7f3499700 1 -- 172.21.15.5:0/635590424 shutdown_connections 2024-05-09T01:02:51.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-09T01:02:51.645+0000 7fe7f3499700 1 -- 172.21.15.5:0/635590424 wait complete. 2024-05-09T01:02:51.647 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' removed 2024-05-09T01:02:51.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:51 smithi157 ceph-mon[34294]: pgmap v868: 65 pgs: 65 active+clean; 577 KiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:02:51.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:51 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/635590424' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T01:02:51.997 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:51 smithi005 ceph-mon[30365]: pgmap v868: 65 pgs: 65 active+clean; 577 KiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:02:51.997 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:51 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/635590424' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T01:02:52.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:51 smithi131 ceph-mon[34547]: pgmap v868: 65 pgs: 65 active+clean; 577 KiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T01:02:52.106 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:51 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/635590424' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T01:02:52.423 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T01:02:52.463 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T01:02:52.491 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T01:02:52.537 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-09T01:02:52.538 DEBUG:teuthology.orchestra.run.smithi005:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-09T01:02:52.554 DEBUG:teuthology.orchestra.run.smithi131:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-09T01:02:52.569 DEBUG:teuthology.orchestra.run.smithi157:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-09T01:02:52.585 INFO:tasks.cephadm:Stopping all daemons... 2024-05-09T01:02:52.585 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-05-09T01:02:52.585 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@mon.a 2024-05-09T01:02:52.653 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:52 smithi005 systemd[1]: Stopping Ceph mon.a for d85814c6-0d9b-11ef-bc97-c7b262605968... 2024-05-09T01:02:52.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:52 smithi131 ceph-mon[34547]: from='client.? 172.21.15.5:0/635590424' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-09T01:02:52.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:52 smithi131 ceph-mon[34547]: osdmap e85: 12 total, 12 up, 12 in 2024-05-09T01:02:52.915 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:52 smithi005 ceph-mon[30365]: from='client.? 172.21.15.5:0/635590424' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-09T01:02:52.915 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:52 smithi005 ceph-mon[30365]: osdmap e85: 12 total, 12 up, 12 in 2024-05-09T01:02:52.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:52 smithi157 ceph-mon[34294]: from='client.? 172.21.15.5:0/635590424' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-09T01:02:52.936 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:52 smithi157 ceph-mon[34294]: osdmap e85: 12 total, 12 up, 12 in 2024-05-09T01:02:53.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:52 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mon-a[30361]: 2024-05-09T01:02:52.914+0000 7fa643399700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T01:02:53.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:52 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mon-a[30361]: 2024-05-09T01:02:52.914+0000 7fa643399700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-05-09T01:02:53.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:53 smithi005 podman[111504]: 2024-05-09 01:02:53.109428863 +0000 UTC m=+0.315044536 container died f5caa1eb3625918c35c8f3c9e52216eda888cedb2bb0bb7210833dc5e83eae49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mon-a, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T01:02:53.790 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:53 smithi005 podman[111504]: 2024-05-09 01:02:53.565126819 +0000 UTC m=+0.770742492 container cleanup f5caa1eb3625918c35c8f3c9e52216eda888cedb2bb0bb7210833dc5e83eae49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T01:02:53.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 09 01:02:53 smithi005 bash[111504]: ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mon-a 2024-05-09T01:02:54.028 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@mon.a.service' 2024-05-09T01:02:54.067 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T01:02:54.068 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-05-09T01:02:54.068 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-05-09T01:02:54.068 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@mon.b 2024-05-09T01:02:54.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:54 smithi131 systemd[1]: Stopping Ceph mon.b for d85814c6-0d9b-11ef-bc97-c7b262605968... 2024-05-09T01:02:54.855 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:54 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mon-b[34543]: 2024-05-09T01:02:54.567+0000 7f7dac79f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T01:02:54.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:54 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mon-b[34543]: 2024-05-09T01:02:54.567+0000 7f7dac79f700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-05-09T01:02:54.856 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:54 smithi131 podman[69754]: 2024-05-09 01:02:54.761663377 +0000 UTC m=+0.299626549 container died 88c17aba445508300f76ffc89657bbc1fb77190e6fc153f51692d293f8dc6395 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mon-b, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True) 2024-05-09T01:02:55.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:55 smithi131 podman[69754]: 2024-05-09 01:02:55.201378247 +0000 UTC m=+0.739341420 container cleanup 88c17aba445508300f76ffc89657bbc1fb77190e6fc153f51692d293f8dc6395 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mon-b, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-05-09T01:02:55.355 INFO:journalctl@ceph.mon.b.smithi131.stdout:May 09 01:02:55 smithi131 bash[69754]: ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mon-b 2024-05-09T01:02:55.620 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@mon.b.service' 2024-05-09T01:02:55.674 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T01:02:55.674 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-05-09T01:02:55.675 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-05-09T01:02:55.675 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@mon.c 2024-05-09T01:02:56.075 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:55 smithi157 systemd[1]: Stopping Ceph mon.c for d85814c6-0d9b-11ef-bc97-c7b262605968... 2024-05-09T01:02:56.422 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:56 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mon-c[34276]: 2024-05-09T01:02:56.074+0000 7f47be04e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T01:02:56.422 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:56 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mon-c[34276]: 2024-05-09T01:02:56.074+0000 7f47be04e700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-05-09T01:02:56.422 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:56 smithi157 podman[70636]: 2024-05-09 01:02:56.136095233 +0000 UTC m=+0.201441364 container died e534101f6e7ce052a3a73b84eb1564095aa21e0804c70da28b798dcecafb1683 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mon-c, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T01:02:56.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:56 smithi157 podman[70636]: 2024-05-09 01:02:56.533609217 +0000 UTC m=+0.598955344 container cleanup e534101f6e7ce052a3a73b84eb1564095aa21e0804c70da28b798dcecafb1683 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mon-c, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux ) 2024-05-09T01:02:56.686 INFO:journalctl@ceph.mon.c.smithi157.stdout:May 09 01:02:56 smithi157 bash[70636]: ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mon-c 2024-05-09T01:02:56.953 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@mon.c.service' 2024-05-09T01:02:57.018 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T01:02:57.018 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-05-09T01:02:57.018 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-05-09T01:02:57.019 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@mgr.x 2024-05-09T01:02:57.338 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 01:02:57 smithi005 systemd[1]: Stopping Ceph mgr.x for d85814c6-0d9b-11ef-bc97-c7b262605968... 2024-05-09T01:02:57.689 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 01:02:57 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x[30599]: 2024-05-09T01:02:57.336+0000 7f76a64bf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T01:02:57.689 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 01:02:57 smithi005 podman[111626]: 2024-05-09 01:02:57.368283814 +0000 UTC m=+0.143904763 container died f58a8233012eb42c7810a06f2f9b2c763fd814149165533dee971ce9734c587e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T01:02:58.040 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 01:02:57 smithi005 podman[111626]: 2024-05-09 01:02:57.862971947 +0000 UTC m=+0.638592933 container cleanup f58a8233012eb42c7810a06f2f9b2c763fd814149165533dee971ce9734c587e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, org.label-schema.build-date=20240507) 2024-05-09T01:02:58.040 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 09 01:02:57 smithi005 bash[111626]: ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-x 2024-05-09T01:02:58.293 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@mgr.x.service' 2024-05-09T01:02:58.331 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T01:02:58.331 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-05-09T01:02:58.331 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-05-09T01:02:58.331 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@mgr.y 2024-05-09T01:02:58.644 INFO:journalctl@ceph.mgr.y.smithi131.stdout:May 09 01:02:58 smithi131 systemd[1]: Stopping Ceph mgr.y for d85814c6-0d9b-11ef-bc97-c7b262605968... 2024-05-09T01:02:58.925 INFO:journalctl@ceph.mgr.y.smithi131.stdout:May 09 01:02:58 smithi131 podman[69876]: 2024-05-09 01:02:58.644348746 +0000 UTC m=+0.125500780 container died 2b75e33e95712081ad67b8d6c76cb71de98da9f4888b782d69b0cca141487036 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-y, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T01:02:59.293 INFO:journalctl@ceph.mgr.y.smithi131.stdout:May 09 01:02:59 smithi131 podman[69876]: 2024-05-09 01:02:59.024865077 +0000 UTC m=+0.506017109 container cleanup 2b75e33e95712081ad67b8d6c76cb71de98da9f4888b782d69b0cca141487036 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-05-09T01:02:59.293 INFO:journalctl@ceph.mgr.y.smithi131.stdout:May 09 01:02:59 smithi131 bash[69876]: ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-y 2024-05-09T01:02:59.441 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@mgr.y.service' 2024-05-09T01:02:59.474 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T01:02:59.475 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-05-09T01:02:59.475 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-05-09T01:02:59.475 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@mgr.z 2024-05-09T01:02:59.810 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 09 01:02:59 smithi157 systemd[1]: Stopping Ceph mgr.z for d85814c6-0d9b-11ef-bc97-c7b262605968... 2024-05-09T01:03:00.115 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 09 01:02:59 smithi157 podman[70760]: 2024-05-09 01:02:59.809522145 +0000 UTC m=+0.143602771 container died f48ec4b59ddabf4fae5feda5fd5dd9c51a43a3ade703d3cf1a476dc4f8aa867a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-z, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-09T01:03:00.436 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 09 01:03:00 smithi157 podman[70760]: 2024-05-09 01:03:00.222871991 +0000 UTC m=+0.556952629 container cleanup f48ec4b59ddabf4fae5feda5fd5dd9c51a43a3ade703d3cf1a476dc4f8aa867a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-z, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-09T01:03:00.436 INFO:journalctl@ceph.mgr.z.smithi157.stdout:May 09 01:03:00 smithi157 bash[70760]: ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mgr-z 2024-05-09T01:03:00.584 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@mgr.z.service' 2024-05-09T01:03:00.618 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T01:03:00.618 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-05-09T01:03:00.618 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-09T01:03:00.618 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@osd.0 2024-05-09T01:03:00.952 INFO:journalctl@ceph.osd.0.smithi005.stdout:May 09 01:03:00 smithi005 systemd[1]: Stopping Ceph osd.0 for d85814c6-0d9b-11ef-bc97-c7b262605968... 2024-05-09T01:03:01.290 INFO:journalctl@ceph.osd.0.smithi005.stdout:May 09 01:03:00 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-0[45505]: 2024-05-09T01:03:00.950+0000 7f67fc173700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T01:03:01.291 INFO:journalctl@ceph.osd.0.smithi005.stdout:May 09 01:03:00 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-0[45505]: 2024-05-09T01:03:00.950+0000 7f67fc173700 -1 osd.0 85 *** Got signal Terminated *** 2024-05-09T01:03:01.291 INFO:journalctl@ceph.osd.0.smithi005.stdout:May 09 01:03:00 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-0[45505]: 2024-05-09T01:03:00.950+0000 7f67fc173700 -1 osd.0 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T01:03:06.769 INFO:journalctl@ceph.osd.0.smithi005.stdout:May 09 01:03:06 smithi005 podman[111748]: 2024-05-09 01:03:06.480968314 +0000 UTC m=+5.660554385 container died 1eccb3bc3b5623102c157c3f4f3a3ca780b77d3e32602de5cc8e51030e2c1971 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-05-09T01:03:07.041 INFO:journalctl@ceph.osd.0.smithi005.stdout:May 09 01:03:06 smithi005 podman[111748]: 2024-05-09 01:03:06.943932363 +0000 UTC m=+6.123518431 container cleanup 1eccb3bc3b5623102c157c3f4f3a3ca780b77d3e32602de5cc8e51030e2c1971 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5) 2024-05-09T01:03:07.041 INFO:journalctl@ceph.osd.0.smithi005.stdout:May 09 01:03:06 smithi005 bash[111748]: ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-0 2024-05-09T01:03:07.727 INFO:journalctl@ceph.osd.0.smithi005.stdout:May 09 01:03:07 smithi005 podman[111767]: 2024-05-09 01:03:07.354729999 +0000 UTC m=+0.867820406 container remove 1eccb3bc3b5623102c157c3f4f3a3ca780b77d3e32602de5cc8e51030e2c1971 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-0, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-05-09T01:03:08.041 INFO:journalctl@ceph.osd.0.smithi005.stdout:May 09 01:03:07 smithi005 podman[111855]: 2024-05-09 01:03:07.630234848 +0000 UTC m=+0.018563714 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T01:03:08.041 INFO:journalctl@ceph.osd.0.smithi005.stdout:May 09 01:03:07 smithi005 podman[111855]: 2024-05-09 01:03:07.747344978 +0000 UTC m=+0.135673837 container create f908ce4474190f6a2297ea6ddd7a7261f91272e79f25491c18e48358737caa29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-09T01:03:08.041 INFO:journalctl@ceph.osd.0.smithi005.stdout:May 09 01:03:07 smithi005 podman[111855]: 2024-05-09 01:03:07.959311074 +0000 UTC m=+0.347639934 container init f908ce4474190f6a2297ea6ddd7a7261f91272e79f25491c18e48358737caa29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True) 2024-05-09T01:03:08.041 INFO:journalctl@ceph.osd.0.smithi005.stdout:May 09 01:03:07 smithi005 podman[111855]: 2024-05-09 01:03:07.962739603 +0000 UTC m=+0.351068461 container start f908ce4474190f6a2297ea6ddd7a7261f91272e79f25491c18e48358737caa29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-05-09T01:03:08.319 INFO:journalctl@ceph.osd.0.smithi005.stdout:May 09 01:03:08 smithi005 podman[111855]: 2024-05-09 01:03:08.077281396 +0000 UTC m=+0.465610288 container attach f908ce4474190f6a2297ea6ddd7a7261f91272e79f25491c18e48358737caa29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-05-09T01:03:08.790 INFO:journalctl@ceph.osd.0.smithi005.stdout:May 09 01:03:08 smithi005 podman[111855]: 2024-05-09 01:03:08.552279397 +0000 UTC m=+0.940608247 container died f908ce4474190f6a2297ea6ddd7a7261f91272e79f25491c18e48358737caa29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS) 2024-05-09T01:03:09.260 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@osd.0.service' 2024-05-09T01:03:09.302 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T01:03:09.302 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-09T01:03:09.302 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-09T01:03:09.303 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@osd.1 2024-05-09T01:03:09.605 INFO:journalctl@ceph.osd.1.smithi131.stdout:May 09 01:03:09 smithi131 systemd[1]: Stopping Ceph osd.1 for d85814c6-0d9b-11ef-bc97-c7b262605968... 2024-05-09T01:03:09.871 INFO:journalctl@ceph.osd.1.smithi131.stdout:May 09 01:03:09 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-1[39147]: 2024-05-09T01:03:09.638+0000 7f2fef780700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T01:03:09.871 INFO:journalctl@ceph.osd.1.smithi131.stdout:May 09 01:03:09 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-1[39147]: 2024-05-09T01:03:09.638+0000 7f2fef780700 -1 osd.1 85 *** Got signal Terminated *** 2024-05-09T01:03:09.871 INFO:journalctl@ceph.osd.1.smithi131.stdout:May 09 01:03:09 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-1[39147]: 2024-05-09T01:03:09.638+0000 7f2fef780700 -1 osd.1 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T01:03:15.356 INFO:journalctl@ceph.osd.1.smithi131.stdout:May 09 01:03:15 smithi131 podman[70002]: 2024-05-09 01:03:15.059238015 +0000 UTC m=+5.553327304 container died b700fc5c1809249194b9209b485e30f9e71aa491598ec87bd8baad8ca7b22531 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-1, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-09T01:03:15.710 INFO:journalctl@ceph.osd.1.smithi131.stdout:May 09 01:03:15 smithi131 podman[70002]: 2024-05-09 01:03:15.474966426 +0000 UTC m=+5.969055717 container cleanup b700fc5c1809249194b9209b485e30f9e71aa491598ec87bd8baad8ca7b22531 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-1, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T01:03:15.710 INFO:journalctl@ceph.osd.1.smithi131.stdout:May 09 01:03:15 smithi131 bash[70002]: ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-1 2024-05-09T01:03:16.105 INFO:journalctl@ceph.osd.1.smithi131.stdout:May 09 01:03:15 smithi131 podman[70031]: 2024-05-09 01:03:15.710252928 +0000 UTC m=+0.650978976 container remove b700fc5c1809249194b9209b485e30f9e71aa491598ec87bd8baad8ca7b22531 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507) 2024-05-09T01:03:16.434 INFO:journalctl@ceph.osd.1.smithi131.stdout:May 09 01:03:16 smithi131 podman[70120]: 2024-05-09 01:03:16.043585256 +0000 UTC m=+0.017959498 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T01:03:16.434 INFO:journalctl@ceph.osd.1.smithi131.stdout:May 09 01:03:16 smithi131 podman[70120]: 2024-05-09 01:03:16.186267207 +0000 UTC m=+0.160641426 container create 1a95ad9ad5ac3ebbd1eafa73151984d61d4591c816d9f4d5f96656d64d41bea2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-05-09T01:03:16.855 INFO:journalctl@ceph.osd.1.smithi131.stdout:May 09 01:03:16 smithi131 podman[70120]: 2024-05-09 01:03:16.434589254 +0000 UTC m=+0.408963481 container init 1a95ad9ad5ac3ebbd1eafa73151984d61d4591c816d9f4d5f96656d64d41bea2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T01:03:16.855 INFO:journalctl@ceph.osd.1.smithi131.stdout:May 09 01:03:16 smithi131 podman[70120]: 2024-05-09 01:03:16.43804236 +0000 UTC m=+0.412416587 container start 1a95ad9ad5ac3ebbd1eafa73151984d61d4591c816d9f4d5f96656d64d41bea2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-05-09T01:03:16.856 INFO:journalctl@ceph.osd.1.smithi131.stdout:May 09 01:03:16 smithi131 podman[70120]: 2024-05-09 01:03:16.556575823 +0000 UTC m=+0.530950062 container attach 1a95ad9ad5ac3ebbd1eafa73151984d61d4591c816d9f4d5f96656d64d41bea2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-1-deactivate, ceph=True, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T01:03:17.235 INFO:journalctl@ceph.osd.1.smithi131.stdout:May 09 01:03:16 smithi131 podman[70120]: 2024-05-09 01:03:16.955122094 +0000 UTC m=+0.929496323 container died 1a95ad9ad5ac3ebbd1eafa73151984d61d4591c816d9f4d5f96656d64d41bea2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-05-09T01:03:17.564 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@osd.1.service' 2024-05-09T01:03:17.603 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T01:03:17.604 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-09T01:03:17.604 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-09T01:03:17.604 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@osd.2 2024-05-09T01:03:17.919 INFO:journalctl@ceph.osd.2.smithi157.stdout:May 09 01:03:17 smithi157 systemd[1]: Stopping Ceph osd.2 for d85814c6-0d9b-11ef-bc97-c7b262605968... 2024-05-09T01:03:18.186 INFO:journalctl@ceph.osd.2.smithi157.stdout:May 09 01:03:17 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-2[39453]: 2024-05-09T01:03:17.918+0000 7f7361fc4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T01:03:18.186 INFO:journalctl@ceph.osd.2.smithi157.stdout:May 09 01:03:17 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-2[39453]: 2024-05-09T01:03:17.918+0000 7f7361fc4700 -1 osd.2 85 *** Got signal Terminated *** 2024-05-09T01:03:18.186 INFO:journalctl@ceph.osd.2.smithi157.stdout:May 09 01:03:17 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-2[39453]: 2024-05-09T01:03:17.918+0000 7f7361fc4700 -1 osd.2 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T01:03:23.686 INFO:journalctl@ceph.osd.2.smithi157.stdout:May 09 01:03:23 smithi157 podman[70897]: 2024-05-09 01:03:23.431730713 +0000 UTC m=+5.629167901 container died 7198533d124e30277c00c4870b64d16bd4ddca3738cf95a6d4ae7dba64fe2a1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-2, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-05-09T01:03:24.154 INFO:journalctl@ceph.osd.2.smithi157.stdout:May 09 01:03:23 smithi157 podman[70897]: 2024-05-09 01:03:23.835359253 +0000 UTC m=+6.032796446 container cleanup 7198533d124e30277c00c4870b64d16bd4ddca3738cf95a6d4ae7dba64fe2a1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-05-09T01:03:24.154 INFO:journalctl@ceph.osd.2.smithi157.stdout:May 09 01:03:23 smithi157 bash[70897]: ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-2 2024-05-09T01:03:24.436 INFO:journalctl@ceph.osd.2.smithi157.stdout:May 09 01:03:24 smithi157 podman[70915]: 2024-05-09 01:03:24.153631974 +0000 UTC m=+0.717472286 container remove 7198533d124e30277c00c4870b64d16bd4ddca3738cf95a6d4ae7dba64fe2a1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T01:03:24.749 INFO:journalctl@ceph.osd.2.smithi157.stdout:May 09 01:03:24 smithi157 podman[71010]: 2024-05-09 01:03:24.363658977 +0000 UTC m=+0.018239847 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T01:03:24.749 INFO:journalctl@ceph.osd.2.smithi157.stdout:May 09 01:03:24 smithi157 podman[71010]: 2024-05-09 01:03:24.487898781 +0000 UTC m=+0.142479634 container create d742d09d70ff5e60ce76e876efbc1b9821664901b2d1e8f688146349c71182ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-09T01:03:24.749 INFO:journalctl@ceph.osd.2.smithi157.stdout:May 09 01:03:24 smithi157 podman[71010]: 2024-05-09 01:03:24.748915844 +0000 UTC m=+0.403496691 container init d742d09d70ff5e60ce76e876efbc1b9821664901b2d1e8f688146349c71182ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-05-09T01:03:25.186 INFO:journalctl@ceph.osd.2.smithi157.stdout:May 09 01:03:24 smithi157 podman[71010]: 2024-05-09 01:03:24.752560184 +0000 UTC m=+0.407141042 container start d742d09d70ff5e60ce76e876efbc1b9821664901b2d1e8f688146349c71182ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-09T01:03:25.186 INFO:journalctl@ceph.osd.2.smithi157.stdout:May 09 01:03:24 smithi157 podman[71010]: 2024-05-09 01:03:24.882713706 +0000 UTC m=+0.537294566 container attach d742d09d70ff5e60ce76e876efbc1b9821664901b2d1e8f688146349c71182ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True) 2024-05-09T01:03:25.546 INFO:journalctl@ceph.osd.2.smithi157.stdout:May 09 01:03:25 smithi157 podman[71010]: 2024-05-09 01:03:25.270460329 +0000 UTC m=+0.925041189 container died d742d09d70ff5e60ce76e876efbc1b9821664901b2d1e8f688146349c71182ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-2-deactivate, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-05-09T01:03:25.896 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@osd.2.service' 2024-05-09T01:03:25.935 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T01:03:25.935 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-09T01:03:25.936 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-09T01:03:25.936 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@osd.3 2024-05-09T01:03:26.267 INFO:journalctl@ceph.osd.3.smithi005.stdout:May 09 01:03:25 smithi005 systemd[1]: Stopping Ceph osd.3 for d85814c6-0d9b-11ef-bc97-c7b262605968... 2024-05-09T01:03:26.540 INFO:journalctl@ceph.osd.3.smithi005.stdout:May 09 01:03:26 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-3[52012]: 2024-05-09T01:03:26.266+0000 7f428a636700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T01:03:26.541 INFO:journalctl@ceph.osd.3.smithi005.stdout:May 09 01:03:26 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-3[52012]: 2024-05-09T01:03:26.266+0000 7f428a636700 -1 osd.3 85 *** Got signal Terminated *** 2024-05-09T01:03:26.541 INFO:journalctl@ceph.osd.3.smithi005.stdout:May 09 01:03:26 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-3[52012]: 2024-05-09T01:03:26.266+0000 7f428a636700 -1 osd.3 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T01:03:27.540 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:27 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:27.142+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.737970+0000 front 2024-05-09T01:03:04.738119+0000 (oldest deadline 2024-05-09T01:03:27.037934+0000) 2024-05-09T01:03:28.540 INFO:journalctl@ceph.osd.3.smithi005.stdout:May 09 01:03:28 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-3[52012]: 2024-05-09T01:03:28.203+0000 7f4282be4700 -1 osd.3 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:01.909218+0000 front 2024-05-09T01:03:01.909250+0000 (oldest deadline 2024-05-09T01:03:27.209389+0000) 2024-05-09T01:03:28.541 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:28 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:28.100+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.737970+0000 front 2024-05-09T01:03:04.738119+0000 (oldest deadline 2024-05-09T01:03:27.037934+0000) 2024-05-09T01:03:28.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:28 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:28.376+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:03:29.040 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:28 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6[58994]: 2024-05-09T01:03:28.711+0000 7f5abd6ec700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.279932+0000 front 2024-05-09T01:03:02.279987+0000 (oldest deadline 2024-05-09T01:03:28.680789+0000) 2024-05-09T01:03:29.186 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:03:28 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:03:28.871+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.533336+0000 front 2024-05-09T01:03:04.533539+0000 (oldest deadline 2024-05-09T01:03:28.633473+0000) 2024-05-09T01:03:29.541 INFO:journalctl@ceph.osd.3.smithi005.stdout:May 09 01:03:29 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-3[52012]: 2024-05-09T01:03:29.207+0000 7f4282be4700 -1 osd.3 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:01.909218+0000 front 2024-05-09T01:03:01.909250+0000 (oldest deadline 2024-05-09T01:03:27.209389+0000) 2024-05-09T01:03:29.541 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:29 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:29.056+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.737970+0000 front 2024-05-09T01:03:04.738119+0000 (oldest deadline 2024-05-09T01:03:27.037934+0000) 2024-05-09T01:03:29.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:29 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:29.411+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:03:30.040 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:29 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6[58994]: 2024-05-09T01:03:29.674+0000 7f5abd6ec700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.279932+0000 front 2024-05-09T01:03:02.279987+0000 (oldest deadline 2024-05-09T01:03:28.680789+0000) 2024-05-09T01:03:30.186 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:03:29 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:03:29.870+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.533336+0000 front 2024-05-09T01:03:04.533539+0000 (oldest deadline 2024-05-09T01:03:28.633473+0000) 2024-05-09T01:03:30.540 INFO:journalctl@ceph.osd.3.smithi005.stdout:May 09 01:03:30 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-3[52012]: 2024-05-09T01:03:30.208+0000 7f4282be4700 -1 osd.3 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:01.909218+0000 front 2024-05-09T01:03:01.909250+0000 (oldest deadline 2024-05-09T01:03:27.209389+0000) 2024-05-09T01:03:30.541 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:30 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:30.080+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.737970+0000 front 2024-05-09T01:03:04.738119+0000 (oldest deadline 2024-05-09T01:03:27.037934+0000) 2024-05-09T01:03:30.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:30 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:30.371+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:03:31.040 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:30 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6[58994]: 2024-05-09T01:03:30.682+0000 7f5abd6ec700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.279932+0000 front 2024-05-09T01:03:02.279987+0000 (oldest deadline 2024-05-09T01:03:28.680789+0000) 2024-05-09T01:03:31.186 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:03:30 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:03:30.822+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.533336+0000 front 2024-05-09T01:03:04.533539+0000 (oldest deadline 2024-05-09T01:03:28.633473+0000) 2024-05-09T01:03:31.186 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 09 01:03:31 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-5[45949]: 2024-05-09T01:03:31.063+0000 7f3e5cc54700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.261838+0000 front 2024-05-09T01:03:04.261951+0000 (oldest deadline 2024-05-09T01:03:30.161642+0000) 2024-05-09T01:03:31.540 INFO:journalctl@ceph.osd.3.smithi005.stdout:May 09 01:03:31 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-3[52012]: 2024-05-09T01:03:31.162+0000 7f4282be4700 -1 osd.3 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:01.909218+0000 front 2024-05-09T01:03:01.909250+0000 (oldest deadline 2024-05-09T01:03:27.209389+0000) 2024-05-09T01:03:31.541 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:31 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:31.046+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.737970+0000 front 2024-05-09T01:03:04.738119+0000 (oldest deadline 2024-05-09T01:03:27.037934+0000) 2024-05-09T01:03:31.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:31 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:31.367+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:03:32.040 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:31 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6[58994]: 2024-05-09T01:03:31.708+0000 7f5abd6ec700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.279932+0000 front 2024-05-09T01:03:02.279987+0000 (oldest deadline 2024-05-09T01:03:28.680789+0000) 2024-05-09T01:03:32.041 INFO:journalctl@ceph.osd.3.smithi005.stdout:May 09 01:03:31 smithi005 podman[111970]: 2024-05-09 01:03:31.813131848 +0000 UTC m=+5.664642855 container died 7f23345f1379fe10f859b8f61f9e0907adf2247d9f8459f1644c3c53551328e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-3, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T01:03:32.186 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:03:31 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:03:31.785+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.533336+0000 front 2024-05-09T01:03:04.533539+0000 (oldest deadline 2024-05-09T01:03:28.633473+0000) 2024-05-09T01:03:32.186 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 09 01:03:32 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-5[45949]: 2024-05-09T01:03:32.021+0000 7f3e5cc54700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.261838+0000 front 2024-05-09T01:03:04.261951+0000 (oldest deadline 2024-05-09T01:03:30.161642+0000) 2024-05-09T01:03:32.309 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:32 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:32.043+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.737970+0000 front 2024-05-09T01:03:04.738119+0000 (oldest deadline 2024-05-09T01:03:27.037934+0000) 2024-05-09T01:03:32.576 INFO:journalctl@ceph.osd.3.smithi005.stdout:May 09 01:03:32 smithi005 podman[111970]: 2024-05-09 01:03:32.308344871 +0000 UTC m=+6.159855891 container cleanup 7f23345f1379fe10f859b8f61f9e0907adf2247d9f8459f1644c3c53551328e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-3, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T01:03:32.577 INFO:journalctl@ceph.osd.3.smithi005.stdout:May 09 01:03:32 smithi005 bash[111970]: ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-3 2024-05-09T01:03:32.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:32 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:32.397+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:03:32.946 INFO:journalctl@ceph.osd.3.smithi005.stdout:May 09 01:03:32 smithi005 podman[111990]: 2024-05-09 01:03:32.576593222 +0000 UTC m=+0.756397804 container remove 7f23345f1379fe10f859b8f61f9e0907adf2247d9f8459f1644c3c53551328e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-05-09T01:03:32.946 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:32 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6[58994]: 2024-05-09T01:03:32.683+0000 7f5abd6ec700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.279932+0000 front 2024-05-09T01:03:02.279987+0000 (oldest deadline 2024-05-09T01:03:28.680789+0000) 2024-05-09T01:03:33.186 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:03:32 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:03:32.833+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.533336+0000 front 2024-05-09T01:03:04.533539+0000 (oldest deadline 2024-05-09T01:03:28.633473+0000) 2024-05-09T01:03:33.186 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 09 01:03:32 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-5[45949]: 2024-05-09T01:03:32.997+0000 7f3e5cc54700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.261838+0000 front 2024-05-09T01:03:04.261951+0000 (oldest deadline 2024-05-09T01:03:30.161642+0000) 2024-05-09T01:03:33.264 INFO:journalctl@ceph.osd.3.smithi005.stdout:May 09 01:03:32 smithi005 podman[112083]: 2024-05-09 01:03:32.849505469 +0000 UTC m=+0.018576796 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T01:03:33.264 INFO:journalctl@ceph.osd.3.smithi005.stdout:May 09 01:03:32 smithi005 podman[112083]: 2024-05-09 01:03:32.985849737 +0000 UTC m=+0.154921086 container create df2e1dc7d7a9d0f318a1801209c698b1cad14553d5587cb57e6fb5caa863de22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-3-deactivate, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507) 2024-05-09T01:03:33.264 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:33 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:33.036+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.737970+0000 front 2024-05-09T01:03:04.738119+0000 (oldest deadline 2024-05-09T01:03:27.037934+0000) 2024-05-09T01:03:33.540 INFO:journalctl@ceph.osd.3.smithi005.stdout:May 09 01:03:33 smithi005 podman[112083]: 2024-05-09 01:03:33.263588162 +0000 UTC m=+0.432659481 container init df2e1dc7d7a9d0f318a1801209c698b1cad14553d5587cb57e6fb5caa863de22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-05-09T01:03:33.541 INFO:journalctl@ceph.osd.3.smithi005.stdout:May 09 01:03:33 smithi005 podman[112083]: 2024-05-09 01:03:33.26792634 +0000 UTC m=+0.436997650 container start df2e1dc7d7a9d0f318a1801209c698b1cad14553d5587cb57e6fb5caa863de22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T01:03:33.541 INFO:journalctl@ceph.osd.3.smithi005.stdout:May 09 01:03:33 smithi005 podman[112083]: 2024-05-09 01:03:33.363637254 +0000 UTC m=+0.532708561 container attach df2e1dc7d7a9d0f318a1801209c698b1cad14553d5587cb57e6fb5caa863de22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T01:03:33.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:33 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:33.426+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:03:33.925 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:33 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6[58994]: 2024-05-09T01:03:33.705+0000 7f5abd6ec700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.279932+0000 front 2024-05-09T01:03:02.279987+0000 (oldest deadline 2024-05-09T01:03:28.680789+0000) 2024-05-09T01:03:33.925 INFO:journalctl@ceph.osd.3.smithi005.stdout:May 09 01:03:33 smithi005 podman[112083]: 2024-05-09 01:03:33.57813369 +0000 UTC m=+0.747204997 container died df2e1dc7d7a9d0f318a1801209c698b1cad14553d5587cb57e6fb5caa863de22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-05-09T01:03:34.186 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:03:33 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:03:33.851+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.533336+0000 front 2024-05-09T01:03:04.533539+0000 (oldest deadline 2024-05-09T01:03:28.633473+0000) 2024-05-09T01:03:34.186 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 09 01:03:34 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-5[45949]: 2024-05-09T01:03:34.007+0000 7f3e5cc54700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.261838+0000 front 2024-05-09T01:03:04.261951+0000 (oldest deadline 2024-05-09T01:03:30.161642+0000) 2024-05-09T01:03:34.290 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:34 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:34.075+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.737970+0000 front 2024-05-09T01:03:04.738119+0000 (oldest deadline 2024-05-09T01:03:27.037934+0000) 2024-05-09T01:03:34.347 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@osd.3.service' 2024-05-09T01:03:34.392 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T01:03:34.392 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-09T01:03:34.393 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-09T01:03:34.393 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@osd.4 2024-05-09T01:03:34.681 INFO:journalctl@ceph.osd.4.smithi131.stdout:May 09 01:03:34 smithi131 systemd[1]: Stopping Ceph osd.4 for d85814c6-0d9b-11ef-bc97-c7b262605968... 2024-05-09T01:03:34.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:34 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:34.434+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:03:35.040 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:34 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6[58994]: 2024-05-09T01:03:34.706+0000 7f5abd6ec700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.279932+0000 front 2024-05-09T01:03:02.279987+0000 (oldest deadline 2024-05-09T01:03:28.680789+0000) 2024-05-09T01:03:35.105 INFO:journalctl@ceph.osd.4.smithi131.stdout:May 09 01:03:34 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-4[45803]: 2024-05-09T01:03:34.679+0000 7f71b82f0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T01:03:35.105 INFO:journalctl@ceph.osd.4.smithi131.stdout:May 09 01:03:34 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-4[45803]: 2024-05-09T01:03:34.679+0000 7f71b82f0700 -1 osd.4 85 *** Got signal Terminated *** 2024-05-09T01:03:35.105 INFO:journalctl@ceph.osd.4.smithi131.stdout:May 09 01:03:34 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-4[45803]: 2024-05-09T01:03:34.679+0000 7f71b82f0700 -1 osd.4 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T01:03:35.186 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:03:34 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:03:34.804+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.533336+0000 front 2024-05-09T01:03:04.533539+0000 (oldest deadline 2024-05-09T01:03:28.633473+0000) 2024-05-09T01:03:35.186 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 09 01:03:34 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-5[45949]: 2024-05-09T01:03:34.977+0000 7f3e5cc54700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.261838+0000 front 2024-05-09T01:03:04.261951+0000 (oldest deadline 2024-05-09T01:03:30.161642+0000) 2024-05-09T01:03:35.540 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:35 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:35.084+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.737970+0000 front 2024-05-09T01:03:04.738119+0000 (oldest deadline 2024-05-09T01:03:27.037934+0000) 2024-05-09T01:03:35.762 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:35 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:35.464+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:03:36.014 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:03:35 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:03:35.761+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.533336+0000 front 2024-05-09T01:03:04.533539+0000 (oldest deadline 2024-05-09T01:03:28.633473+0000) 2024-05-09T01:03:36.040 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:35 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6[58994]: 2024-05-09T01:03:35.695+0000 7f5abd6ec700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.279932+0000 front 2024-05-09T01:03:02.279987+0000 (oldest deadline 2024-05-09T01:03:28.680789+0000) 2024-05-09T01:03:36.184 INFO:journalctl@ceph.osd.4.smithi131.stdout:May 09 01:03:35 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-4[45803]: 2024-05-09T01:03:35.895+0000 7f71b48a6700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:14.230122+0000 front 2024-05-09T01:03:14.230181+0000 (oldest deadline 2024-05-09T01:03:35.831037+0000) 2024-05-09T01:03:36.418 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 09 01:03:36 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-5[45949]: 2024-05-09T01:03:36.013+0000 7f3e5cc54700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.261838+0000 front 2024-05-09T01:03:04.261951+0000 (oldest deadline 2024-05-09T01:03:30.161642+0000) 2024-05-09T01:03:36.419 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 09 01:03:36 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-5[45949]: 2024-05-09T01:03:36.013+0000 7f3e5cc54700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:11.262975+0000 front 2024-05-09T01:03:11.263071+0000 (oldest deadline 2024-05-09T01:03:35.362814+0000) 2024-05-09T01:03:36.503 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:03:36 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:03:36.182+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.667724+0000 front 2024-05-09T01:03:13.667762+0000 (oldest deadline 2024-05-09T01:03:35.967853+0000) 2024-05-09T01:03:36.540 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:36 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:36.098+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.737970+0000 front 2024-05-09T01:03:04.738119+0000 (oldest deadline 2024-05-09T01:03:27.037934+0000) 2024-05-09T01:03:36.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:36 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:36.418+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:03:36.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:36 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:36.418+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:12.758527+0000 front 2024-05-09T01:03:12.758364+0000 (oldest deadline 2024-05-09T01:03:35.658450+0000) 2024-05-09T01:03:36.855 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:03:36 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7[52359]: 2024-05-09T01:03:36.501+0000 7f4f992ef700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:14.583613+0000 front 2024-05-09T01:03:14.583770+0000 (oldest deadline 2024-05-09T01:03:35.683825+0000) 2024-05-09T01:03:37.005 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:03:36 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:03:36.726+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.533336+0000 front 2024-05-09T01:03:04.533539+0000 (oldest deadline 2024-05-09T01:03:28.633473+0000) 2024-05-09T01:03:37.040 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:36 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6[58994]: 2024-05-09T01:03:36.665+0000 7f5abd6ec700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.279932+0000 front 2024-05-09T01:03:02.279987+0000 (oldest deadline 2024-05-09T01:03:28.680789+0000) 2024-05-09T01:03:37.175 INFO:journalctl@ceph.osd.4.smithi131.stdout:May 09 01:03:36 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-4[45803]: 2024-05-09T01:03:36.905+0000 7f71b48a6700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:14.230122+0000 front 2024-05-09T01:03:14.230181+0000 (oldest deadline 2024-05-09T01:03:35.831037+0000) 2024-05-09T01:03:37.282 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 09 01:03:37 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-5[45949]: 2024-05-09T01:03:37.004+0000 7f3e5cc54700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.261838+0000 front 2024-05-09T01:03:04.261951+0000 (oldest deadline 2024-05-09T01:03:30.161642+0000) 2024-05-09T01:03:37.282 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 09 01:03:37 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-5[45949]: 2024-05-09T01:03:37.004+0000 7f3e5cc54700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:11.262975+0000 front 2024-05-09T01:03:11.263071+0000 (oldest deadline 2024-05-09T01:03:35.362814+0000) 2024-05-09T01:03:37.332 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:37 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:37.136+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.737970+0000 front 2024-05-09T01:03:04.738119+0000 (oldest deadline 2024-05-09T01:03:27.037934+0000) 2024-05-09T01:03:37.547 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:03:37 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:03:37.174+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.667724+0000 front 2024-05-09T01:03:13.667762+0000 (oldest deadline 2024-05-09T01:03:35.967853+0000) 2024-05-09T01:03:37.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:37 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:37.429+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:03:37.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:37 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:37.429+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:12.758527+0000 front 2024-05-09T01:03:12.758364+0000 (oldest deadline 2024-05-09T01:03:35.658450+0000) 2024-05-09T01:03:37.855 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:03:37 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7[52359]: 2024-05-09T01:03:37.546+0000 7f4f992ef700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:14.583613+0000 front 2024-05-09T01:03:14.583770+0000 (oldest deadline 2024-05-09T01:03:35.683825+0000) 2024-05-09T01:03:37.988 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:03:37 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:03:37.689+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.533336+0000 front 2024-05-09T01:03:04.533539+0000 (oldest deadline 2024-05-09T01:03:28.633473+0000) 2024-05-09T01:03:38.040 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:37 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6[58994]: 2024-05-09T01:03:37.629+0000 7f5abd6ec700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.279932+0000 front 2024-05-09T01:03:02.279987+0000 (oldest deadline 2024-05-09T01:03:28.680789+0000) 2024-05-09T01:03:38.152 INFO:journalctl@ceph.osd.4.smithi131.stdout:May 09 01:03:37 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-4[45803]: 2024-05-09T01:03:37.897+0000 7f71b48a6700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:14.230122+0000 front 2024-05-09T01:03:14.230181+0000 (oldest deadline 2024-05-09T01:03:35.831037+0000) 2024-05-09T01:03:38.436 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 09 01:03:37 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-5[45949]: 2024-05-09T01:03:37.987+0000 7f3e5cc54700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.261838+0000 front 2024-05-09T01:03:04.261951+0000 (oldest deadline 2024-05-09T01:03:30.161642+0000) 2024-05-09T01:03:38.436 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 09 01:03:37 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-5[45949]: 2024-05-09T01:03:37.987+0000 7f3e5cc54700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:11.262975+0000 front 2024-05-09T01:03:11.263071+0000 (oldest deadline 2024-05-09T01:03:35.362814+0000) 2024-05-09T01:03:38.540 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:38 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:38.149+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.737970+0000 front 2024-05-09T01:03:04.738119+0000 (oldest deadline 2024-05-09T01:03:27.037934+0000) 2024-05-09T01:03:38.567 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:03:38 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:03:38.151+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.667724+0000 front 2024-05-09T01:03:13.667762+0000 (oldest deadline 2024-05-09T01:03:35.967853+0000) 2024-05-09T01:03:38.855 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:03:38 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7[52359]: 2024-05-09T01:03:38.567+0000 7f4f992ef700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:14.583613+0000 front 2024-05-09T01:03:14.583770+0000 (oldest deadline 2024-05-09T01:03:35.683825+0000) 2024-05-09T01:03:38.936 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:03:38 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:03:38.664+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.533336+0000 front 2024-05-09T01:03:04.533539+0000 (oldest deadline 2024-05-09T01:03:28.633473+0000) 2024-05-09T01:03:38.936 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:38 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:38.442+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:03:38.936 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:38 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:38.442+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:12.758527+0000 front 2024-05-09T01:03:12.758364+0000 (oldest deadline 2024-05-09T01:03:35.658450+0000) 2024-05-09T01:03:39.040 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:38 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6[58994]: 2024-05-09T01:03:38.609+0000 7f5abd6ec700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.279932+0000 front 2024-05-09T01:03:02.279987+0000 (oldest deadline 2024-05-09T01:03:28.680789+0000) 2024-05-09T01:03:39.041 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:38 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6[58994]: 2024-05-09T01:03:38.609+0000 7f5abd6ec700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:14.482253+0000 front 2024-05-09T01:03:14.482294+0000 (oldest deadline 2024-05-09T01:03:38.582313+0000) 2024-05-09T01:03:39.190 INFO:journalctl@ceph.osd.4.smithi131.stdout:May 09 01:03:38 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-4[45803]: 2024-05-09T01:03:38.923+0000 7f71b48a6700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:14.230122+0000 front 2024-05-09T01:03:14.230181+0000 (oldest deadline 2024-05-09T01:03:35.831037+0000) 2024-05-09T01:03:39.415 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 09 01:03:39 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-5[45949]: 2024-05-09T01:03:39.028+0000 7f3e5cc54700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.261838+0000 front 2024-05-09T01:03:04.261951+0000 (oldest deadline 2024-05-09T01:03:30.161642+0000) 2024-05-09T01:03:39.415 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 09 01:03:39 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-5[45949]: 2024-05-09T01:03:39.028+0000 7f3e5cc54700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:11.262975+0000 front 2024-05-09T01:03:11.263071+0000 (oldest deadline 2024-05-09T01:03:35.362814+0000) 2024-05-09T01:03:39.415 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:39 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:39.414+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:03:39.415 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:39 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:39.414+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:12.758527+0000 front 2024-05-09T01:03:12.758364+0000 (oldest deadline 2024-05-09T01:03:35.658450+0000) 2024-05-09T01:03:39.540 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:39 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:39.188+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.737970+0000 front 2024-05-09T01:03:04.738119+0000 (oldest deadline 2024-05-09T01:03:27.037934+0000) 2024-05-09T01:03:39.541 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:39 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:39.188+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.439829+0000 front 2024-05-09T01:03:13.439669+0000 (oldest deadline 2024-05-09T01:03:38.739771+0000) 2024-05-09T01:03:39.563 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:03:39 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:03:39.190+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.667724+0000 front 2024-05-09T01:03:13.667762+0000 (oldest deadline 2024-05-09T01:03:35.967853+0000) 2024-05-09T01:03:39.686 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:03:39 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:03:39.653+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.533336+0000 front 2024-05-09T01:03:04.533539+0000 (oldest deadline 2024-05-09T01:03:28.633473+0000) 2024-05-09T01:03:39.855 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:03:39 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7[52359]: 2024-05-09T01:03:39.562+0000 7f4f992ef700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:14.583613+0000 front 2024-05-09T01:03:14.583770+0000 (oldest deadline 2024-05-09T01:03:35.683825+0000) 2024-05-09T01:03:40.040 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:39 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6[58994]: 2024-05-09T01:03:39.598+0000 7f5abd6ec700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.279932+0000 front 2024-05-09T01:03:02.279987+0000 (oldest deadline 2024-05-09T01:03:28.680789+0000) 2024-05-09T01:03:40.040 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:39 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6[58994]: 2024-05-09T01:03:39.598+0000 7f5abd6ec700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:14.482253+0000 front 2024-05-09T01:03:14.482294+0000 (oldest deadline 2024-05-09T01:03:38.582313+0000) 2024-05-09T01:03:40.355 INFO:journalctl@ceph.osd.4.smithi131.stdout:May 09 01:03:40 smithi131 podman[70226]: 2024-05-09 01:03:40.094062049 +0000 UTC m=+5.505605801 container died 4d4f2d76c4f39dafc3ad1b036cba358e23df330cdd61a934c51975ba6b103308 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-4, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5) 2024-05-09T01:03:40.355 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:03:40 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:03:40.145+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.667724+0000 front 2024-05-09T01:03:13.667762+0000 (oldest deadline 2024-05-09T01:03:35.967853+0000) 2024-05-09T01:03:40.378 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 09 01:03:40 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-5[45949]: 2024-05-09T01:03:40.025+0000 7f3e5cc54700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.261838+0000 front 2024-05-09T01:03:04.261951+0000 (oldest deadline 2024-05-09T01:03:30.161642+0000) 2024-05-09T01:03:40.378 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 09 01:03:40 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-5[45949]: 2024-05-09T01:03:40.025+0000 7f3e5cc54700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:11.262975+0000 front 2024-05-09T01:03:11.263071+0000 (oldest deadline 2024-05-09T01:03:35.362814+0000) 2024-05-09T01:03:40.540 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:40 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:40.232+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.737970+0000 front 2024-05-09T01:03:04.738119+0000 (oldest deadline 2024-05-09T01:03:27.037934+0000) 2024-05-09T01:03:40.541 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:40 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:40.232+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.439829+0000 front 2024-05-09T01:03:13.439669+0000 (oldest deadline 2024-05-09T01:03:38.739771+0000) 2024-05-09T01:03:40.685 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:40 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:40.377+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:03:40.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:40 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:40.377+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:12.758527+0000 front 2024-05-09T01:03:12.758364+0000 (oldest deadline 2024-05-09T01:03:35.658450+0000) 2024-05-09T01:03:40.732 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:03:40 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7[52359]: 2024-05-09T01:03:40.512+0000 7f4f992ef700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:14.583613+0000 front 2024-05-09T01:03:14.583770+0000 (oldest deadline 2024-05-09T01:03:35.683825+0000) 2024-05-09T01:03:40.732 INFO:journalctl@ceph.osd.4.smithi131.stdout:May 09 01:03:40 smithi131 podman[70226]: 2024-05-09 01:03:40.497305197 +0000 UTC m=+5.908848954 container cleanup 4d4f2d76c4f39dafc3ad1b036cba358e23df330cdd61a934c51975ba6b103308 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-4, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T01:03:40.732 INFO:journalctl@ceph.osd.4.smithi131.stdout:May 09 01:03:40 smithi131 bash[70226]: ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-4 2024-05-09T01:03:40.936 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:03:40 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:03:40.684+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.533336+0000 front 2024-05-09T01:03:04.533539+0000 (oldest deadline 2024-05-09T01:03:28.633473+0000) 2024-05-09T01:03:41.040 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:40 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6[58994]: 2024-05-09T01:03:40.607+0000 7f5abd6ec700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.279932+0000 front 2024-05-09T01:03:02.279987+0000 (oldest deadline 2024-05-09T01:03:28.680789+0000) 2024-05-09T01:03:41.041 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:40 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6[58994]: 2024-05-09T01:03:40.607+0000 7f5abd6ec700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:14.482253+0000 front 2024-05-09T01:03:14.482294+0000 (oldest deadline 2024-05-09T01:03:38.582313+0000) 2024-05-09T01:03:41.061 INFO:journalctl@ceph.osd.4.smithi131.stdout:May 09 01:03:40 smithi131 podman[70244]: 2024-05-09 01:03:40.732038314 +0000 UTC m=+0.631800475 container remove 4d4f2d76c4f39dafc3ad1b036cba358e23df330cdd61a934c51975ba6b103308 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-4, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-05-09T01:03:41.344 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:03:41 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:03:41.145+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.667724+0000 front 2024-05-09T01:03:13.667762+0000 (oldest deadline 2024-05-09T01:03:35.967853+0000) 2024-05-09T01:03:41.345 INFO:journalctl@ceph.osd.4.smithi131.stdout:May 09 01:03:41 smithi131 podman[70344]: 2024-05-09 01:03:40.96409299 +0000 UTC m=+0.024625888 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T01:03:41.345 INFO:journalctl@ceph.osd.4.smithi131.stdout:May 09 01:03:41 smithi131 podman[70344]: 2024-05-09 01:03:41.082987109 +0000 UTC m=+0.143520003 container create 216c93486c7529a005ba9d735e899f84f9064a0f08b6883853d52232055db44d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240507, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-09T01:03:41.345 INFO:journalctl@ceph.osd.4.smithi131.stdout:May 09 01:03:41 smithi131 podman[70344]: 2024-05-09 01:03:41.344539205 +0000 UTC m=+0.405072109 container init 216c93486c7529a005ba9d735e899f84f9064a0f08b6883853d52232055db44d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD) 2024-05-09T01:03:41.371 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 09 01:03:41 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-5[45949]: 2024-05-09T01:03:41.028+0000 7f3e5cc54700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.261838+0000 front 2024-05-09T01:03:04.261951+0000 (oldest deadline 2024-05-09T01:03:30.161642+0000) 2024-05-09T01:03:41.372 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 09 01:03:41 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-5[45949]: 2024-05-09T01:03:41.028+0000 7f3e5cc54700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:11.262975+0000 front 2024-05-09T01:03:11.263071+0000 (oldest deadline 2024-05-09T01:03:35.362814+0000) 2024-05-09T01:03:41.540 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:41 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:41.231+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.737970+0000 front 2024-05-09T01:03:04.738119+0000 (oldest deadline 2024-05-09T01:03:27.037934+0000) 2024-05-09T01:03:41.540 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:41 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:41.231+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.439829+0000 front 2024-05-09T01:03:13.439669+0000 (oldest deadline 2024-05-09T01:03:38.739771+0000) 2024-05-09T01:03:41.605 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:03:41 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7[52359]: 2024-05-09T01:03:41.550+0000 7f4f992ef700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:14.583613+0000 front 2024-05-09T01:03:14.583770+0000 (oldest deadline 2024-05-09T01:03:35.683825+0000) 2024-05-09T01:03:41.606 INFO:journalctl@ceph.osd.4.smithi131.stdout:May 09 01:03:41 smithi131 podman[70344]: 2024-05-09 01:03:41.348130393 +0000 UTC m=+0.408663290 container start 216c93486c7529a005ba9d735e899f84f9064a0f08b6883853d52232055db44d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True) 2024-05-09T01:03:41.606 INFO:journalctl@ceph.osd.4.smithi131.stdout:May 09 01:03:41 smithi131 podman[70344]: 2024-05-09 01:03:41.419086162 +0000 UTC m=+0.479619058 container attach 216c93486c7529a005ba9d735e899f84f9064a0f08b6883853d52232055db44d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-4-deactivate, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-05-09T01:03:41.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:41 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:41.370+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:03:41.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:41 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:41.370+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:12.758527+0000 front 2024-05-09T01:03:12.758364+0000 (oldest deadline 2024-05-09T01:03:35.658450+0000) 2024-05-09T01:03:41.964 INFO:journalctl@ceph.osd.4.smithi131.stdout:May 09 01:03:41 smithi131 podman[70344]: 2024-05-09 01:03:41.641197118 +0000 UTC m=+0.701730016 container died 216c93486c7529a005ba9d735e899f84f9064a0f08b6883853d52232055db44d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-4-deactivate, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-05-09T01:03:42.040 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:41 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6[58994]: 2024-05-09T01:03:41.649+0000 7f5abd6ec700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.279932+0000 front 2024-05-09T01:03:02.279987+0000 (oldest deadline 2024-05-09T01:03:28.680789+0000) 2024-05-09T01:03:42.041 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:41 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6[58994]: 2024-05-09T01:03:41.649+0000 7f5abd6ec700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:14.482253+0000 front 2024-05-09T01:03:14.482294+0000 (oldest deadline 2024-05-09T01:03:38.582313+0000) 2024-05-09T01:03:42.044 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:03:41 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:03:41.713+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.533336+0000 front 2024-05-09T01:03:04.533539+0000 (oldest deadline 2024-05-09T01:03:28.633473+0000) 2024-05-09T01:03:42.281 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:03:42 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:03:42.170+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.667724+0000 front 2024-05-09T01:03:13.667762+0000 (oldest deadline 2024-05-09T01:03:35.967853+0000) 2024-05-09T01:03:42.297 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@osd.4.service' 2024-05-09T01:03:42.331 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 09 01:03:42 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-5[45949]: 2024-05-09T01:03:42.043+0000 7f3e5cc54700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.261838+0000 front 2024-05-09T01:03:04.261951+0000 (oldest deadline 2024-05-09T01:03:30.161642+0000) 2024-05-09T01:03:42.331 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 09 01:03:42 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-5[45949]: 2024-05-09T01:03:42.043+0000 7f3e5cc54700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:11.262975+0000 front 2024-05-09T01:03:11.263071+0000 (oldest deadline 2024-05-09T01:03:35.362814+0000) 2024-05-09T01:03:42.337 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T01:03:42.337 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-09T01:03:42.337 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-09T01:03:42.337 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@osd.5 2024-05-09T01:03:42.540 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:42 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:42.271+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.737970+0000 front 2024-05-09T01:03:04.738119+0000 (oldest deadline 2024-05-09T01:03:27.037934+0000) 2024-05-09T01:03:42.541 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:42 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:42.271+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.439829+0000 front 2024-05-09T01:03:13.439669+0000 (oldest deadline 2024-05-09T01:03:38.739771+0000) 2024-05-09T01:03:42.605 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:03:42 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7[52359]: 2024-05-09T01:03:42.527+0000 7f4f992ef700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:14.583613+0000 front 2024-05-09T01:03:14.583770+0000 (oldest deadline 2024-05-09T01:03:35.683825+0000) 2024-05-09T01:03:42.664 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 09 01:03:42 smithi157 systemd[1]: Stopping Ceph osd.5 for d85814c6-0d9b-11ef-bc97-c7b262605968... 2024-05-09T01:03:42.665 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:42 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:42.330+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:03:42.665 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:42 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:42.330+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:12.758527+0000 front 2024-05-09T01:03:12.758364+0000 (oldest deadline 2024-05-09T01:03:35.658450+0000) 2024-05-09T01:03:42.936 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:03:42 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:03:42.744+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.533336+0000 front 2024-05-09T01:03:04.533539+0000 (oldest deadline 2024-05-09T01:03:28.633473+0000) 2024-05-09T01:03:42.936 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 09 01:03:42 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-5[45949]: 2024-05-09T01:03:42.664+0000 7f3e6069e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T01:03:42.936 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 09 01:03:42 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-5[45949]: 2024-05-09T01:03:42.664+0000 7f3e6069e700 -1 osd.5 85 *** Got signal Terminated *** 2024-05-09T01:03:42.937 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 09 01:03:42 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-5[45949]: 2024-05-09T01:03:42.664+0000 7f3e6069e700 -1 osd.5 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T01:03:43.041 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:42 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6[58994]: 2024-05-09T01:03:42.629+0000 7f5abd6ec700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.279932+0000 front 2024-05-09T01:03:02.279987+0000 (oldest deadline 2024-05-09T01:03:28.680789+0000) 2024-05-09T01:03:43.041 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:42 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6[58994]: 2024-05-09T01:03:42.629+0000 7f5abd6ec700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:14.482253+0000 front 2024-05-09T01:03:14.482294+0000 (oldest deadline 2024-05-09T01:03:38.582313+0000) 2024-05-09T01:03:43.305 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 09 01:03:43 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-5[45949]: 2024-05-09T01:03:43.009+0000 7f3e5cc54700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.261838+0000 front 2024-05-09T01:03:04.261951+0000 (oldest deadline 2024-05-09T01:03:30.161642+0000) 2024-05-09T01:03:43.305 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 09 01:03:43 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-5[45949]: 2024-05-09T01:03:43.009+0000 7f3e5cc54700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:11.262975+0000 front 2024-05-09T01:03:11.263071+0000 (oldest deadline 2024-05-09T01:03:35.362814+0000) 2024-05-09T01:03:43.533 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:03:43 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:03:43.178+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.667724+0000 front 2024-05-09T01:03:13.667762+0000 (oldest deadline 2024-05-09T01:03:35.967853+0000) 2024-05-09T01:03:43.617 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:43 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:43.315+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.737970+0000 front 2024-05-09T01:03:04.738119+0000 (oldest deadline 2024-05-09T01:03:27.037934+0000) 2024-05-09T01:03:43.617 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:43 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:43.315+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.439829+0000 front 2024-05-09T01:03:13.439669+0000 (oldest deadline 2024-05-09T01:03:38.739771+0000) 2024-05-09T01:03:43.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:43 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:43.304+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:03:43.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:43 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:43.304+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:12.758527+0000 front 2024-05-09T01:03:12.758364+0000 (oldest deadline 2024-05-09T01:03:35.658450+0000) 2024-05-09T01:03:43.855 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:03:43 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7[52359]: 2024-05-09T01:03:43.532+0000 7f4f992ef700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:14.583613+0000 front 2024-05-09T01:03:14.583770+0000 (oldest deadline 2024-05-09T01:03:35.683825+0000) 2024-05-09T01:03:44.040 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:43 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6[58994]: 2024-05-09T01:03:43.616+0000 7f5abd6ec700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.279932+0000 front 2024-05-09T01:03:02.279987+0000 (oldest deadline 2024-05-09T01:03:28.680789+0000) 2024-05-09T01:03:44.041 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:43 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6[58994]: 2024-05-09T01:03:43.616+0000 7f5abd6ec700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:14.482253+0000 front 2024-05-09T01:03:14.482294+0000 (oldest deadline 2024-05-09T01:03:38.582313+0000) 2024-05-09T01:03:44.186 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:03:43 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:03:43.748+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.533336+0000 front 2024-05-09T01:03:04.533539+0000 (oldest deadline 2024-05-09T01:03:28.633473+0000) 2024-05-09T01:03:44.186 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 09 01:03:43 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-5[45949]: 2024-05-09T01:03:43.969+0000 7f3e5cc54700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.261838+0000 front 2024-05-09T01:03:04.261951+0000 (oldest deadline 2024-05-09T01:03:30.161642+0000) 2024-05-09T01:03:44.186 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 09 01:03:43 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-5[45949]: 2024-05-09T01:03:43.969+0000 7f3e5cc54700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:11.262975+0000 front 2024-05-09T01:03:11.263071+0000 (oldest deadline 2024-05-09T01:03:35.362814+0000) 2024-05-09T01:03:44.557 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:03:44 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:03:44.161+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.667724+0000 front 2024-05-09T01:03:13.667762+0000 (oldest deadline 2024-05-09T01:03:35.967853+0000) 2024-05-09T01:03:44.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:44 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:44.307+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:03:44.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:44 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:44.307+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:12.758527+0000 front 2024-05-09T01:03:12.758364+0000 (oldest deadline 2024-05-09T01:03:35.658450+0000) 2024-05-09T01:03:44.790 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:44 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6[58994]: 2024-05-09T01:03:44.581+0000 7f5abd6ec700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.279932+0000 front 2024-05-09T01:03:02.279987+0000 (oldest deadline 2024-05-09T01:03:28.680789+0000) 2024-05-09T01:03:44.791 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:44 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6[58994]: 2024-05-09T01:03:44.581+0000 7f5abd6ec700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:14.482253+0000 front 2024-05-09T01:03:14.482294+0000 (oldest deadline 2024-05-09T01:03:38.582313+0000) 2024-05-09T01:03:44.791 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:44 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:44.341+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.737970+0000 front 2024-05-09T01:03:04.738119+0000 (oldest deadline 2024-05-09T01:03:27.037934+0000) 2024-05-09T01:03:44.791 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:44 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:44.341+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.439829+0000 front 2024-05-09T01:03:13.439669+0000 (oldest deadline 2024-05-09T01:03:38.739771+0000) 2024-05-09T01:03:44.791 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:44 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:44.341+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:18.740444+0000 front 2024-05-09T01:03:18.740379+0000 (oldest deadline 2024-05-09T01:03:44.040389+0000) 2024-05-09T01:03:44.855 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:03:44 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7[52359]: 2024-05-09T01:03:44.556+0000 7f4f992ef700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:14.583613+0000 front 2024-05-09T01:03:14.583770+0000 (oldest deadline 2024-05-09T01:03:35.683825+0000) 2024-05-09T01:03:44.959 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:03:44 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:03:44.701+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.533336+0000 front 2024-05-09T01:03:04.533539+0000 (oldest deadline 2024-05-09T01:03:28.633473+0000) 2024-05-09T01:03:44.959 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:03:44 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:03:44.701+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:18.535819+0000 front 2024-05-09T01:03:18.536013+0000 (oldest deadline 2024-05-09T01:03:44.435939+0000) 2024-05-09T01:03:45.319 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 09 01:03:44 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-5[45949]: 2024-05-09T01:03:44.958+0000 7f3e5cc54700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.261838+0000 front 2024-05-09T01:03:04.261951+0000 (oldest deadline 2024-05-09T01:03:30.161642+0000) 2024-05-09T01:03:45.319 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 09 01:03:44 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-5[45949]: 2024-05-09T01:03:44.958+0000 7f3e5cc54700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:11.262975+0000 front 2024-05-09T01:03:11.263071+0000 (oldest deadline 2024-05-09T01:03:35.362814+0000) 2024-05-09T01:03:45.320 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 09 01:03:44 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-5[45949]: 2024-05-09T01:03:44.958+0000 7f3e5cc54700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:22.364324+0000 front 2024-05-09T01:03:22.364350+0000 (oldest deadline 2024-05-09T01:03:44.664434+0000) 2024-05-09T01:03:45.539 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:03:45 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:03:45.167+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.667724+0000 front 2024-05-09T01:03:13.667762+0000 (oldest deadline 2024-05-09T01:03:35.967853+0000) 2024-05-09T01:03:45.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:45 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:45.318+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:03:45.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:45 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:45.318+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:12.758527+0000 front 2024-05-09T01:03:12.758364+0000 (oldest deadline 2024-05-09T01:03:35.658450+0000) 2024-05-09T01:03:45.790 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:45 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6[58994]: 2024-05-09T01:03:45.558+0000 7f5abd6ec700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.279932+0000 front 2024-05-09T01:03:02.279987+0000 (oldest deadline 2024-05-09T01:03:28.680789+0000) 2024-05-09T01:03:45.791 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:45 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6[58994]: 2024-05-09T01:03:45.558+0000 7f5abd6ec700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:14.482253+0000 front 2024-05-09T01:03:14.482294+0000 (oldest deadline 2024-05-09T01:03:38.582313+0000) 2024-05-09T01:03:45.791 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:45 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6[58994]: 2024-05-09T01:03:45.558+0000 7f5abd6ec700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:21.384074+0000 front 2024-05-09T01:03:21.384257+0000 (oldest deadline 2024-05-09T01:03:44.883973+0000) 2024-05-09T01:03:45.791 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:45 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:45.382+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.737970+0000 front 2024-05-09T01:03:04.738119+0000 (oldest deadline 2024-05-09T01:03:27.037934+0000) 2024-05-09T01:03:45.791 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:45 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:45.382+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.439829+0000 front 2024-05-09T01:03:13.439669+0000 (oldest deadline 2024-05-09T01:03:38.739771+0000) 2024-05-09T01:03:45.791 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:45 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:45.382+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:18.740444+0000 front 2024-05-09T01:03:18.740379+0000 (oldest deadline 2024-05-09T01:03:44.040389+0000) 2024-05-09T01:03:45.855 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:03:45 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7[52359]: 2024-05-09T01:03:45.538+0000 7f4f992ef700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:14.583613+0000 front 2024-05-09T01:03:14.583770+0000 (oldest deadline 2024-05-09T01:03:35.683825+0000) 2024-05-09T01:03:46.186 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:03:45 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:03:45.744+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.533336+0000 front 2024-05-09T01:03:04.533539+0000 (oldest deadline 2024-05-09T01:03:28.633473+0000) 2024-05-09T01:03:46.186 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:03:45 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:03:45.744+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:18.535819+0000 front 2024-05-09T01:03:18.536013+0000 (oldest deadline 2024-05-09T01:03:44.435939+0000) 2024-05-09T01:03:46.186 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 09 01:03:45 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-5[45949]: 2024-05-09T01:03:45.941+0000 7f3e5cc54700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.261838+0000 front 2024-05-09T01:03:04.261951+0000 (oldest deadline 2024-05-09T01:03:30.161642+0000) 2024-05-09T01:03:46.187 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 09 01:03:45 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-5[45949]: 2024-05-09T01:03:45.941+0000 7f3e5cc54700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:11.262975+0000 front 2024-05-09T01:03:11.263071+0000 (oldest deadline 2024-05-09T01:03:35.362814+0000) 2024-05-09T01:03:46.187 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 09 01:03:45 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-5[45949]: 2024-05-09T01:03:45.941+0000 7f3e5cc54700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:22.364324+0000 front 2024-05-09T01:03:22.364350+0000 (oldest deadline 2024-05-09T01:03:44.664434+0000) 2024-05-09T01:03:46.572 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:03:46 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:03:46.155+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.667724+0000 front 2024-05-09T01:03:13.667762+0000 (oldest deadline 2024-05-09T01:03:35.967853+0000) 2024-05-09T01:03:46.572 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:03:46 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:03:46.155+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.668856+0000 front 2024-05-09T01:03:20.668887+0000 (oldest deadline 2024-05-09T01:03:45.968898+0000) 2024-05-09T01:03:46.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:46 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:46.284+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:03:46.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:46 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:46.284+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:12.758527+0000 front 2024-05-09T01:03:12.758364+0000 (oldest deadline 2024-05-09T01:03:35.658450+0000) 2024-05-09T01:03:46.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:46 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:46.284+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.259808+0000 front 2024-05-09T01:03:20.259685+0000 (oldest deadline 2024-05-09T01:03:46.159864+0000) 2024-05-09T01:03:46.790 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:46 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6[58994]: 2024-05-09T01:03:46.570+0000 7f5abd6ec700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.279932+0000 front 2024-05-09T01:03:02.279987+0000 (oldest deadline 2024-05-09T01:03:28.680789+0000) 2024-05-09T01:03:46.791 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:46 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6[58994]: 2024-05-09T01:03:46.570+0000 7f5abd6ec700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:14.482253+0000 front 2024-05-09T01:03:14.482294+0000 (oldest deadline 2024-05-09T01:03:38.582313+0000) 2024-05-09T01:03:46.791 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:46 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6[58994]: 2024-05-09T01:03:46.570+0000 7f5abd6ec700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:21.384074+0000 front 2024-05-09T01:03:21.384257+0000 (oldest deadline 2024-05-09T01:03:44.883973+0000) 2024-05-09T01:03:46.791 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:46 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:46.395+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.737970+0000 front 2024-05-09T01:03:04.738119+0000 (oldest deadline 2024-05-09T01:03:27.037934+0000) 2024-05-09T01:03:46.791 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:46 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:46.395+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.439829+0000 front 2024-05-09T01:03:13.439669+0000 (oldest deadline 2024-05-09T01:03:38.739771+0000) 2024-05-09T01:03:46.791 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:46 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:46.395+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:18.740444+0000 front 2024-05-09T01:03:18.740379+0000 (oldest deadline 2024-05-09T01:03:44.040389+0000) 2024-05-09T01:03:46.855 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:03:46 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7[52359]: 2024-05-09T01:03:46.571+0000 7f4f992ef700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:14.583613+0000 front 2024-05-09T01:03:14.583770+0000 (oldest deadline 2024-05-09T01:03:35.683825+0000) 2024-05-09T01:03:46.855 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:03:46 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7[52359]: 2024-05-09T01:03:46.571+0000 7f4f992ef700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:21.485772+0000 front 2024-05-09T01:03:21.485552+0000 (oldest deadline 2024-05-09T01:03:46.185602+0000) 2024-05-09T01:03:46.986 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:03:46 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:03:46.720+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.533336+0000 front 2024-05-09T01:03:04.533539+0000 (oldest deadline 2024-05-09T01:03:28.633473+0000) 2024-05-09T01:03:46.986 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:03:46 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:03:46.720+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:18.535819+0000 front 2024-05-09T01:03:18.536013+0000 (oldest deadline 2024-05-09T01:03:44.435939+0000) 2024-05-09T01:03:47.300 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 09 01:03:46 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-5[45949]: 2024-05-09T01:03:46.986+0000 7f3e5cc54700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.261838+0000 front 2024-05-09T01:03:04.261951+0000 (oldest deadline 2024-05-09T01:03:30.161642+0000) 2024-05-09T01:03:47.301 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 09 01:03:46 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-5[45949]: 2024-05-09T01:03:46.986+0000 7f3e5cc54700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:11.262975+0000 front 2024-05-09T01:03:11.263071+0000 (oldest deadline 2024-05-09T01:03:35.362814+0000) 2024-05-09T01:03:47.301 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 09 01:03:46 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-5[45949]: 2024-05-09T01:03:46.986+0000 7f3e5cc54700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:22.364324+0000 front 2024-05-09T01:03:22.364350+0000 (oldest deadline 2024-05-09T01:03:44.664434+0000) 2024-05-09T01:03:47.596 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:47 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:47.345+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.737970+0000 front 2024-05-09T01:03:04.738119+0000 (oldest deadline 2024-05-09T01:03:27.037934+0000) 2024-05-09T01:03:47.596 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:47 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:47.345+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.439829+0000 front 2024-05-09T01:03:13.439669+0000 (oldest deadline 2024-05-09T01:03:38.739771+0000) 2024-05-09T01:03:47.596 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:47 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:47.345+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:18.740444+0000 front 2024-05-09T01:03:18.740379+0000 (oldest deadline 2024-05-09T01:03:44.040389+0000) 2024-05-09T01:03:47.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:03:47 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:03:47.106+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.667724+0000 front 2024-05-09T01:03:13.667762+0000 (oldest deadline 2024-05-09T01:03:35.967853+0000) 2024-05-09T01:03:47.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:03:47 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:03:47.106+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.668856+0000 front 2024-05-09T01:03:20.668887+0000 (oldest deadline 2024-05-09T01:03:45.968898+0000) 2024-05-09T01:03:47.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:47 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:47.300+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:03:47.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:47 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:47.300+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:12.758527+0000 front 2024-05-09T01:03:12.758364+0000 (oldest deadline 2024-05-09T01:03:35.658450+0000) 2024-05-09T01:03:47.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:47 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:47.300+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.259808+0000 front 2024-05-09T01:03:20.259685+0000 (oldest deadline 2024-05-09T01:03:46.159864+0000) 2024-05-09T01:03:48.040 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:47 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6[58994]: 2024-05-09T01:03:47.595+0000 7f5abd6ec700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.279932+0000 front 2024-05-09T01:03:02.279987+0000 (oldest deadline 2024-05-09T01:03:28.680789+0000) 2024-05-09T01:03:48.041 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:47 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6[58994]: 2024-05-09T01:03:47.595+0000 7f5abd6ec700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:14.482253+0000 front 2024-05-09T01:03:14.482294+0000 (oldest deadline 2024-05-09T01:03:38.582313+0000) 2024-05-09T01:03:48.041 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:47 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6[58994]: 2024-05-09T01:03:47.595+0000 7f5abd6ec700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:21.384074+0000 front 2024-05-09T01:03:21.384257+0000 (oldest deadline 2024-05-09T01:03:44.883973+0000) 2024-05-09T01:03:48.105 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:03:47 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7[52359]: 2024-05-09T01:03:47.618+0000 7f4f992ef700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:14.583613+0000 front 2024-05-09T01:03:14.583770+0000 (oldest deadline 2024-05-09T01:03:35.683825+0000) 2024-05-09T01:03:48.105 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:03:47 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7[52359]: 2024-05-09T01:03:47.618+0000 7f4f992ef700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:21.485772+0000 front 2024-05-09T01:03:21.485552+0000 (oldest deadline 2024-05-09T01:03:46.185602+0000) 2024-05-09T01:03:48.161 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:03:47 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:03:47.734+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.533336+0000 front 2024-05-09T01:03:04.533539+0000 (oldest deadline 2024-05-09T01:03:28.633473+0000) 2024-05-09T01:03:48.161 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:03:47 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:03:47.734+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:18.535819+0000 front 2024-05-09T01:03:18.536013+0000 (oldest deadline 2024-05-09T01:03:44.435939+0000) 2024-05-09T01:03:48.436 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 09 01:03:48 smithi157 podman[71141]: 2024-05-09 01:03:48.161342966 +0000 UTC m=+5.629426859 container died c9277eab325af661c06ff626524b7122abbf8fbfd41687cedf79ff8add51456a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-5, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-09T01:03:48.436 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:48 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:48.260+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:03:48.436 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:48 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:48.260+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:12.758527+0000 front 2024-05-09T01:03:12.758364+0000 (oldest deadline 2024-05-09T01:03:35.658450+0000) 2024-05-09T01:03:48.436 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:48 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:48.260+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.259808+0000 front 2024-05-09T01:03:20.259685+0000 (oldest deadline 2024-05-09T01:03:46.159864+0000) 2024-05-09T01:03:48.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:03:48 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:03:48.141+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.667724+0000 front 2024-05-09T01:03:13.667762+0000 (oldest deadline 2024-05-09T01:03:35.967853+0000) 2024-05-09T01:03:48.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:03:48 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:03:48.141+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.668856+0000 front 2024-05-09T01:03:20.668887+0000 (oldest deadline 2024-05-09T01:03:45.968898+0000) 2024-05-09T01:03:48.790 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:48 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:48.375+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.737970+0000 front 2024-05-09T01:03:04.738119+0000 (oldest deadline 2024-05-09T01:03:27.037934+0000) 2024-05-09T01:03:48.791 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:48 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:48.375+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.439829+0000 front 2024-05-09T01:03:13.439669+0000 (oldest deadline 2024-05-09T01:03:38.739771+0000) 2024-05-09T01:03:48.791 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:48 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:48.375+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:18.740444+0000 front 2024-05-09T01:03:18.740379+0000 (oldest deadline 2024-05-09T01:03:44.040389+0000) 2024-05-09T01:03:48.791 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:48 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6[58994]: 2024-05-09T01:03:48.613+0000 7f5abd6ec700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.279932+0000 front 2024-05-09T01:03:02.279987+0000 (oldest deadline 2024-05-09T01:03:28.680789+0000) 2024-05-09T01:03:48.791 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:48 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6[58994]: 2024-05-09T01:03:48.613+0000 7f5abd6ec700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:14.482253+0000 front 2024-05-09T01:03:14.482294+0000 (oldest deadline 2024-05-09T01:03:38.582313+0000) 2024-05-09T01:03:48.791 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:48 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6[58994]: 2024-05-09T01:03:48.613+0000 7f5abd6ec700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:21.384074+0000 front 2024-05-09T01:03:21.384257+0000 (oldest deadline 2024-05-09T01:03:44.883973+0000) 2024-05-09T01:03:48.824 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 09 01:03:48 smithi157 podman[71141]: 2024-05-09 01:03:48.572987574 +0000 UTC m=+6.041071470 container cleanup c9277eab325af661c06ff626524b7122abbf8fbfd41687cedf79ff8add51456a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-05-09T01:03:48.824 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 09 01:03:48 smithi157 bash[71141]: ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-5 2024-05-09T01:03:48.824 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:03:48 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:03:48.723+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.533336+0000 front 2024-05-09T01:03:04.533539+0000 (oldest deadline 2024-05-09T01:03:28.633473+0000) 2024-05-09T01:03:48.824 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:03:48 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:03:48.723+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:18.535819+0000 front 2024-05-09T01:03:18.536013+0000 (oldest deadline 2024-05-09T01:03:44.435939+0000) 2024-05-09T01:03:49.105 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:03:48 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7[52359]: 2024-05-09T01:03:48.643+0000 7f4f992ef700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:14.583613+0000 front 2024-05-09T01:03:14.583770+0000 (oldest deadline 2024-05-09T01:03:35.683825+0000) 2024-05-09T01:03:49.105 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:03:48 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7[52359]: 2024-05-09T01:03:48.643+0000 7f4f992ef700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:21.485772+0000 front 2024-05-09T01:03:21.485552+0000 (oldest deadline 2024-05-09T01:03:46.185602+0000) 2024-05-09T01:03:49.116 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 09 01:03:48 smithi157 podman[71158]: 2024-05-09 01:03:48.824371614 +0000 UTC m=+0.650074404 container remove c9277eab325af661c06ff626524b7122abbf8fbfd41687cedf79ff8add51456a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507) 2024-05-09T01:03:49.436 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:49 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:49.271+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:03:49.436 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:49 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:49.271+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:12.758527+0000 front 2024-05-09T01:03:12.758364+0000 (oldest deadline 2024-05-09T01:03:35.658450+0000) 2024-05-09T01:03:49.436 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:49 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:49.271+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.259808+0000 front 2024-05-09T01:03:20.259685+0000 (oldest deadline 2024-05-09T01:03:46.159864+0000) 2024-05-09T01:03:49.436 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 09 01:03:49 smithi157 podman[71249]: 2024-05-09 01:03:49.116842255 +0000 UTC m=+0.110489435 container create 5e7c6a6dc8426a780a875bef4662e9e85a1c8fc7162853e8c16b29d318f2e4ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5) 2024-05-09T01:03:49.436 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 09 01:03:49 smithi157 podman[71249]: 2024-05-09 01:03:49.02467433 +0000 UTC m=+0.018321504 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T01:03:49.436 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 09 01:03:49 smithi157 podman[71249]: 2024-05-09 01:03:49.361923263 +0000 UTC m=+0.355570437 container init 5e7c6a6dc8426a780a875bef4662e9e85a1c8fc7162853e8c16b29d318f2e4ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-05-09T01:03:49.437 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 09 01:03:49 smithi157 podman[71249]: 2024-05-09 01:03:49.365240112 +0000 UTC m=+0.358887288 container start 5e7c6a6dc8426a780a875bef4662e9e85a1c8fc7162853e8c16b29d318f2e4ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-5-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T01:03:49.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:03:49 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:03:49.184+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.667724+0000 front 2024-05-09T01:03:13.667762+0000 (oldest deadline 2024-05-09T01:03:35.967853+0000) 2024-05-09T01:03:49.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:03:49 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:03:49.184+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.668856+0000 front 2024-05-09T01:03:20.668887+0000 (oldest deadline 2024-05-09T01:03:45.968898+0000) 2024-05-09T01:03:49.617 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:49 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:49.333+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.737970+0000 front 2024-05-09T01:03:04.738119+0000 (oldest deadline 2024-05-09T01:03:27.037934+0000) 2024-05-09T01:03:49.617 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:49 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:49.333+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.439829+0000 front 2024-05-09T01:03:13.439669+0000 (oldest deadline 2024-05-09T01:03:38.739771+0000) 2024-05-09T01:03:49.617 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:49 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:49.333+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:18.740444+0000 front 2024-05-09T01:03:18.740379+0000 (oldest deadline 2024-05-09T01:03:44.040389+0000) 2024-05-09T01:03:49.936 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 09 01:03:49 smithi157 podman[71249]: 2024-05-09 01:03:49.454671943 +0000 UTC m=+0.448319134 container attach 5e7c6a6dc8426a780a875bef4662e9e85a1c8fc7162853e8c16b29d318f2e4ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-05-09T01:03:49.937 INFO:journalctl@ceph.osd.5.smithi157.stdout:May 09 01:03:49 smithi157 podman[71249]: 2024-05-09 01:03:49.66332637 +0000 UTC m=+0.656973535 container died 5e7c6a6dc8426a780a875bef4662e9e85a1c8fc7162853e8c16b29d318f2e4ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux ) 2024-05-09T01:03:49.937 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:03:49 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:03:49.680+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.533336+0000 front 2024-05-09T01:03:04.533539+0000 (oldest deadline 2024-05-09T01:03:28.633473+0000) 2024-05-09T01:03:49.937 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:03:49 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:03:49.680+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:18.535819+0000 front 2024-05-09T01:03:18.536013+0000 (oldest deadline 2024-05-09T01:03:44.435939+0000) 2024-05-09T01:03:50.041 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:49 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6[58994]: 2024-05-09T01:03:49.617+0000 7f5abd6ec700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.279932+0000 front 2024-05-09T01:03:02.279987+0000 (oldest deadline 2024-05-09T01:03:28.680789+0000) 2024-05-09T01:03:50.041 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:49 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6[58994]: 2024-05-09T01:03:49.617+0000 7f5abd6ec700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:14.482253+0000 front 2024-05-09T01:03:14.482294+0000 (oldest deadline 2024-05-09T01:03:38.582313+0000) 2024-05-09T01:03:50.041 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:49 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6[58994]: 2024-05-09T01:03:49.617+0000 7f5abd6ec700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:21.384074+0000 front 2024-05-09T01:03:21.384257+0000 (oldest deadline 2024-05-09T01:03:44.883973+0000) 2024-05-09T01:03:50.105 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:03:49 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7[52359]: 2024-05-09T01:03:49.646+0000 7f4f992ef700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:14.583613+0000 front 2024-05-09T01:03:14.583770+0000 (oldest deadline 2024-05-09T01:03:35.683825+0000) 2024-05-09T01:03:50.105 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:03:49 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7[52359]: 2024-05-09T01:03:49.646+0000 7f4f992ef700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:21.485772+0000 front 2024-05-09T01:03:21.485552+0000 (oldest deadline 2024-05-09T01:03:46.185602+0000) 2024-05-09T01:03:50.369 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@osd.5.service' 2024-05-09T01:03:50.414 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T01:03:50.415 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-09T01:03:50.415 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-05-09T01:03:50.415 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@osd.6 2024-05-09T01:03:50.599 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:50 smithi005 systemd[1]: Stopping Ceph osd.6 for d85814c6-0d9b-11ef-bc97-c7b262605968... 2024-05-09T01:03:50.599 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:50 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:50.344+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.737970+0000 front 2024-05-09T01:03:04.738119+0000 (oldest deadline 2024-05-09T01:03:27.037934+0000) 2024-05-09T01:03:50.599 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:50 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:50.344+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.439829+0000 front 2024-05-09T01:03:13.439669+0000 (oldest deadline 2024-05-09T01:03:38.739771+0000) 2024-05-09T01:03:50.599 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:50 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:50.344+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:18.740444+0000 front 2024-05-09T01:03:18.740379+0000 (oldest deadline 2024-05-09T01:03:44.040389+0000) 2024-05-09T01:03:50.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:03:50 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:03:50.216+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.667724+0000 front 2024-05-09T01:03:13.667762+0000 (oldest deadline 2024-05-09T01:03:35.967853+0000) 2024-05-09T01:03:50.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:03:50 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:03:50.216+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.668856+0000 front 2024-05-09T01:03:20.668887+0000 (oldest deadline 2024-05-09T01:03:45.968898+0000) 2024-05-09T01:03:50.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:50 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:50.282+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:03:50.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:50 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:50.282+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:12.758527+0000 front 2024-05-09T01:03:12.758364+0000 (oldest deadline 2024-05-09T01:03:35.658450+0000) 2024-05-09T01:03:50.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:50 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:50.282+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.259808+0000 front 2024-05-09T01:03:20.259685+0000 (oldest deadline 2024-05-09T01:03:46.159864+0000) 2024-05-09T01:03:50.922 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:03:50 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7[52359]: 2024-05-09T01:03:50.621+0000 7f4f992ef700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:14.583613+0000 front 2024-05-09T01:03:14.583770+0000 (oldest deadline 2024-05-09T01:03:35.683825+0000) 2024-05-09T01:03:50.922 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:03:50 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7[52359]: 2024-05-09T01:03:50.621+0000 7f4f992ef700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:21.485772+0000 front 2024-05-09T01:03:21.485552+0000 (oldest deadline 2024-05-09T01:03:46.185602+0000) 2024-05-09T01:03:51.040 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:50 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6[58994]: 2024-05-09T01:03:50.597+0000 7f5abd6ec700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.279932+0000 front 2024-05-09T01:03:02.279987+0000 (oldest deadline 2024-05-09T01:03:28.680789+0000) 2024-05-09T01:03:51.041 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:50 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6[58994]: 2024-05-09T01:03:50.597+0000 7f5abd6ec700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:14.482253+0000 front 2024-05-09T01:03:14.482294+0000 (oldest deadline 2024-05-09T01:03:38.582313+0000) 2024-05-09T01:03:51.041 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:50 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6[58994]: 2024-05-09T01:03:50.597+0000 7f5abd6ec700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:21.384074+0000 front 2024-05-09T01:03:21.384257+0000 (oldest deadline 2024-05-09T01:03:44.883973+0000) 2024-05-09T01:03:51.041 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:50 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6[58994]: 2024-05-09T01:03:50.729+0000 7f5ac1136700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T01:03:51.041 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:50 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6[58994]: 2024-05-09T01:03:50.729+0000 7f5ac1136700 -1 osd.6 85 *** Got signal Terminated *** 2024-05-09T01:03:51.041 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:50 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6[58994]: 2024-05-09T01:03:50.729+0000 7f5ac1136700 -1 osd.6 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T01:03:51.186 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:03:50 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:03:50.715+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.533336+0000 front 2024-05-09T01:03:04.533539+0000 (oldest deadline 2024-05-09T01:03:28.633473+0000) 2024-05-09T01:03:51.186 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:03:50 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:03:50.716+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:18.535819+0000 front 2024-05-09T01:03:18.536013+0000 (oldest deadline 2024-05-09T01:03:44.435939+0000) 2024-05-09T01:03:51.529 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:03:51 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:03:51.253+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.667724+0000 front 2024-05-09T01:03:13.667762+0000 (oldest deadline 2024-05-09T01:03:35.967853+0000) 2024-05-09T01:03:51.529 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:03:51 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:03:51.253+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.668856+0000 front 2024-05-09T01:03:20.668887+0000 (oldest deadline 2024-05-09T01:03:45.968898+0000) 2024-05-09T01:03:51.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:51 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:51.271+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:03:51.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:51 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:51.271+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:12.758527+0000 front 2024-05-09T01:03:12.758364+0000 (oldest deadline 2024-05-09T01:03:35.658450+0000) 2024-05-09T01:03:51.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:51 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:51.271+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.259808+0000 front 2024-05-09T01:03:20.259685+0000 (oldest deadline 2024-05-09T01:03:46.159864+0000) 2024-05-09T01:03:51.790 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:51 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6[58994]: 2024-05-09T01:03:51.615+0000 7f5abd6ec700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.279932+0000 front 2024-05-09T01:03:02.279987+0000 (oldest deadline 2024-05-09T01:03:28.680789+0000) 2024-05-09T01:03:51.791 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:51 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6[58994]: 2024-05-09T01:03:51.615+0000 7f5abd6ec700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:14.482253+0000 front 2024-05-09T01:03:14.482294+0000 (oldest deadline 2024-05-09T01:03:38.582313+0000) 2024-05-09T01:03:51.791 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:51 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6[58994]: 2024-05-09T01:03:51.615+0000 7f5abd6ec700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:21.384074+0000 front 2024-05-09T01:03:21.384257+0000 (oldest deadline 2024-05-09T01:03:44.883973+0000) 2024-05-09T01:03:51.791 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:51 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:51.380+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.737970+0000 front 2024-05-09T01:03:04.738119+0000 (oldest deadline 2024-05-09T01:03:27.037934+0000) 2024-05-09T01:03:51.791 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:51 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:51.380+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.439829+0000 front 2024-05-09T01:03:13.439669+0000 (oldest deadline 2024-05-09T01:03:38.739771+0000) 2024-05-09T01:03:51.791 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:51 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:51.380+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:18.740444+0000 front 2024-05-09T01:03:18.740379+0000 (oldest deadline 2024-05-09T01:03:44.040389+0000) 2024-05-09T01:03:51.855 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:03:51 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7[52359]: 2024-05-09T01:03:51.651+0000 7f4f992ef700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:14.583613+0000 front 2024-05-09T01:03:14.583770+0000 (oldest deadline 2024-05-09T01:03:35.683825+0000) 2024-05-09T01:03:51.855 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:03:51 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7[52359]: 2024-05-09T01:03:51.651+0000 7f4f992ef700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:21.485772+0000 front 2024-05-09T01:03:21.485552+0000 (oldest deadline 2024-05-09T01:03:46.185602+0000) 2024-05-09T01:03:52.186 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:03:51 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:03:51.731+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.533336+0000 front 2024-05-09T01:03:04.533539+0000 (oldest deadline 2024-05-09T01:03:28.633473+0000) 2024-05-09T01:03:52.186 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:03:51 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:03:51.731+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:18.535819+0000 front 2024-05-09T01:03:18.536013+0000 (oldest deadline 2024-05-09T01:03:44.435939+0000) 2024-05-09T01:03:52.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:03:52 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:03:52.301+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.667724+0000 front 2024-05-09T01:03:13.667762+0000 (oldest deadline 2024-05-09T01:03:35.967853+0000) 2024-05-09T01:03:52.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:03:52 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:03:52.301+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.668856+0000 front 2024-05-09T01:03:20.668887+0000 (oldest deadline 2024-05-09T01:03:45.968898+0000) 2024-05-09T01:03:52.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:52 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:52.315+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:03:52.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:52 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:52.315+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:12.758527+0000 front 2024-05-09T01:03:12.758364+0000 (oldest deadline 2024-05-09T01:03:35.658450+0000) 2024-05-09T01:03:52.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:52 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:52.315+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.259808+0000 front 2024-05-09T01:03:20.259685+0000 (oldest deadline 2024-05-09T01:03:46.159864+0000) 2024-05-09T01:03:52.790 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:52 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6[58994]: 2024-05-09T01:03:52.663+0000 7f5abd6ec700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.279932+0000 front 2024-05-09T01:03:02.279987+0000 (oldest deadline 2024-05-09T01:03:28.680789+0000) 2024-05-09T01:03:52.791 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:52 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6[58994]: 2024-05-09T01:03:52.663+0000 7f5abd6ec700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:14.482253+0000 front 2024-05-09T01:03:14.482294+0000 (oldest deadline 2024-05-09T01:03:38.582313+0000) 2024-05-09T01:03:52.791 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:52 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6[58994]: 2024-05-09T01:03:52.663+0000 7f5abd6ec700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:21.384074+0000 front 2024-05-09T01:03:21.384257+0000 (oldest deadline 2024-05-09T01:03:44.883973+0000) 2024-05-09T01:03:52.791 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:52 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:52.426+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.737970+0000 front 2024-05-09T01:03:04.738119+0000 (oldest deadline 2024-05-09T01:03:27.037934+0000) 2024-05-09T01:03:52.791 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:52 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:52.426+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.439829+0000 front 2024-05-09T01:03:13.439669+0000 (oldest deadline 2024-05-09T01:03:38.739771+0000) 2024-05-09T01:03:52.791 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:52 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:52.426+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:18.740444+0000 front 2024-05-09T01:03:18.740379+0000 (oldest deadline 2024-05-09T01:03:44.040389+0000) 2024-05-09T01:03:52.791 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:52 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:52.426+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:30.342688+0000 front 2024-05-09T01:03:30.342762+0000 (oldest deadline 2024-05-09T01:03:52.042878+0000) 2024-05-09T01:03:53.105 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:03:52 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7[52359]: 2024-05-09T01:03:52.665+0000 7f4f992ef700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:14.583613+0000 front 2024-05-09T01:03:14.583770+0000 (oldest deadline 2024-05-09T01:03:35.683825+0000) 2024-05-09T01:03:53.105 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:03:52 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7[52359]: 2024-05-09T01:03:52.665+0000 7f4f992ef700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:21.485772+0000 front 2024-05-09T01:03:21.485552+0000 (oldest deadline 2024-05-09T01:03:46.185602+0000) 2024-05-09T01:03:53.186 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:03:52 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:03:52.770+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.533336+0000 front 2024-05-09T01:03:04.533539+0000 (oldest deadline 2024-05-09T01:03:28.633473+0000) 2024-05-09T01:03:53.186 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:03:52 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:03:52.770+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:18.535819+0000 front 2024-05-09T01:03:18.536013+0000 (oldest deadline 2024-05-09T01:03:44.435939+0000) 2024-05-09T01:03:53.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:03:53 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:03:53.316+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.667724+0000 front 2024-05-09T01:03:13.667762+0000 (oldest deadline 2024-05-09T01:03:35.967853+0000) 2024-05-09T01:03:53.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:03:53 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:03:53.316+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.668856+0000 front 2024-05-09T01:03:20.668887+0000 (oldest deadline 2024-05-09T01:03:45.968898+0000) 2024-05-09T01:03:53.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:53 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:53.312+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:03:53.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:53 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:53.312+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:12.758527+0000 front 2024-05-09T01:03:12.758364+0000 (oldest deadline 2024-05-09T01:03:35.658450+0000) 2024-05-09T01:03:53.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:53 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:53.312+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.259808+0000 front 2024-05-09T01:03:20.259685+0000 (oldest deadline 2024-05-09T01:03:46.159864+0000) 2024-05-09T01:03:53.790 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:53 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6[58994]: 2024-05-09T01:03:53.675+0000 7f5abd6ec700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.279932+0000 front 2024-05-09T01:03:02.279987+0000 (oldest deadline 2024-05-09T01:03:28.680789+0000) 2024-05-09T01:03:53.791 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:53 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6[58994]: 2024-05-09T01:03:53.675+0000 7f5abd6ec700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:14.482253+0000 front 2024-05-09T01:03:14.482294+0000 (oldest deadline 2024-05-09T01:03:38.582313+0000) 2024-05-09T01:03:53.791 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:53 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6[58994]: 2024-05-09T01:03:53.675+0000 7f5abd6ec700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:21.384074+0000 front 2024-05-09T01:03:21.384257+0000 (oldest deadline 2024-05-09T01:03:44.883973+0000) 2024-05-09T01:03:53.791 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:53 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:53.433+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.737970+0000 front 2024-05-09T01:03:04.738119+0000 (oldest deadline 2024-05-09T01:03:27.037934+0000) 2024-05-09T01:03:53.791 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:53 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:53.433+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.439829+0000 front 2024-05-09T01:03:13.439669+0000 (oldest deadline 2024-05-09T01:03:38.739771+0000) 2024-05-09T01:03:53.791 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:53 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:53.433+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:18.740444+0000 front 2024-05-09T01:03:18.740379+0000 (oldest deadline 2024-05-09T01:03:44.040389+0000) 2024-05-09T01:03:53.791 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:53 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:53.433+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:30.342688+0000 front 2024-05-09T01:03:30.342762+0000 (oldest deadline 2024-05-09T01:03:52.042878+0000) 2024-05-09T01:03:54.105 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:03:53 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7[52359]: 2024-05-09T01:03:53.695+0000 7f4f992ef700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:14.583613+0000 front 2024-05-09T01:03:14.583770+0000 (oldest deadline 2024-05-09T01:03:35.683825+0000) 2024-05-09T01:03:54.106 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:03:53 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7[52359]: 2024-05-09T01:03:53.695+0000 7f4f992ef700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:21.485772+0000 front 2024-05-09T01:03:21.485552+0000 (oldest deadline 2024-05-09T01:03:46.185602+0000) 2024-05-09T01:03:54.186 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:03:53 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:03:53.811+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.533336+0000 front 2024-05-09T01:03:04.533539+0000 (oldest deadline 2024-05-09T01:03:28.633473+0000) 2024-05-09T01:03:54.186 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:03:53 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:03:53.811+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:18.535819+0000 front 2024-05-09T01:03:18.536013+0000 (oldest deadline 2024-05-09T01:03:44.435939+0000) 2024-05-09T01:03:54.186 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:03:53 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:03:53.811+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:30.638891+0000 front 2024-05-09T01:03:30.639015+0000 (oldest deadline 2024-05-09T01:03:53.538952+0000) 2024-05-09T01:03:54.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:03:54 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:03:54.292+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.667724+0000 front 2024-05-09T01:03:13.667762+0000 (oldest deadline 2024-05-09T01:03:35.967853+0000) 2024-05-09T01:03:54.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:03:54 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:03:54.292+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.668856+0000 front 2024-05-09T01:03:20.668887+0000 (oldest deadline 2024-05-09T01:03:45.968898+0000) 2024-05-09T01:03:54.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:03:54 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:03:54.292+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:29.371010+0000 front 2024-05-09T01:03:29.370862+0000 (oldest deadline 2024-05-09T01:03:54.070791+0000) 2024-05-09T01:03:54.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:54 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:54.351+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:03:54.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:54 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:54.351+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:12.758527+0000 front 2024-05-09T01:03:12.758364+0000 (oldest deadline 2024-05-09T01:03:35.658450+0000) 2024-05-09T01:03:54.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:54 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:54.351+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.259808+0000 front 2024-05-09T01:03:20.259685+0000 (oldest deadline 2024-05-09T01:03:46.159864+0000) 2024-05-09T01:03:54.790 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:54 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6[58994]: 2024-05-09T01:03:54.645+0000 7f5abd6ec700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.279932+0000 front 2024-05-09T01:03:02.279987+0000 (oldest deadline 2024-05-09T01:03:28.680789+0000) 2024-05-09T01:03:54.791 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:54 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6[58994]: 2024-05-09T01:03:54.645+0000 7f5abd6ec700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:14.482253+0000 front 2024-05-09T01:03:14.482294+0000 (oldest deadline 2024-05-09T01:03:38.582313+0000) 2024-05-09T01:03:54.791 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:54 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6[58994]: 2024-05-09T01:03:54.645+0000 7f5abd6ec700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:21.384074+0000 front 2024-05-09T01:03:21.384257+0000 (oldest deadline 2024-05-09T01:03:44.883973+0000) 2024-05-09T01:03:54.791 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:54 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:54.483+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.737970+0000 front 2024-05-09T01:03:04.738119+0000 (oldest deadline 2024-05-09T01:03:27.037934+0000) 2024-05-09T01:03:54.791 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:54 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:54.483+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.439829+0000 front 2024-05-09T01:03:13.439669+0000 (oldest deadline 2024-05-09T01:03:38.739771+0000) 2024-05-09T01:03:54.791 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:54 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:54.483+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:18.740444+0000 front 2024-05-09T01:03:18.740379+0000 (oldest deadline 2024-05-09T01:03:44.040389+0000) 2024-05-09T01:03:54.791 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:54 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:54.483+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:30.342688+0000 front 2024-05-09T01:03:30.342762+0000 (oldest deadline 2024-05-09T01:03:52.042878+0000) 2024-05-09T01:03:55.105 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:03:54 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7[52359]: 2024-05-09T01:03:54.649+0000 7f4f992ef700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:14.583613+0000 front 2024-05-09T01:03:14.583770+0000 (oldest deadline 2024-05-09T01:03:35.683825+0000) 2024-05-09T01:03:55.105 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:03:54 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7[52359]: 2024-05-09T01:03:54.649+0000 7f4f992ef700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:21.485772+0000 front 2024-05-09T01:03:21.485552+0000 (oldest deadline 2024-05-09T01:03:46.185602+0000) 2024-05-09T01:03:55.186 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:03:54 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:03:54.771+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.533336+0000 front 2024-05-09T01:03:04.533539+0000 (oldest deadline 2024-05-09T01:03:28.633473+0000) 2024-05-09T01:03:55.186 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:03:54 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:03:54.771+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:18.535819+0000 front 2024-05-09T01:03:18.536013+0000 (oldest deadline 2024-05-09T01:03:44.435939+0000) 2024-05-09T01:03:55.186 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:03:54 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:03:54.771+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:30.638891+0000 front 2024-05-09T01:03:30.639015+0000 (oldest deadline 2024-05-09T01:03:53.538952+0000) 2024-05-09T01:03:55.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:03:55 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:03:55.279+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.667724+0000 front 2024-05-09T01:03:13.667762+0000 (oldest deadline 2024-05-09T01:03:35.967853+0000) 2024-05-09T01:03:55.606 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:03:55 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:03:55.279+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.668856+0000 front 2024-05-09T01:03:20.668887+0000 (oldest deadline 2024-05-09T01:03:45.968898+0000) 2024-05-09T01:03:55.606 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:03:55 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:03:55.279+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:29.371010+0000 front 2024-05-09T01:03:29.370862+0000 (oldest deadline 2024-05-09T01:03:54.070791+0000) 2024-05-09T01:03:55.666 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:55 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:55.458+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.737970+0000 front 2024-05-09T01:03:04.738119+0000 (oldest deadline 2024-05-09T01:03:27.037934+0000) 2024-05-09T01:03:55.666 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:55 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:55.458+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.439829+0000 front 2024-05-09T01:03:13.439669+0000 (oldest deadline 2024-05-09T01:03:38.739771+0000) 2024-05-09T01:03:55.666 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:55 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:55.458+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:18.740444+0000 front 2024-05-09T01:03:18.740379+0000 (oldest deadline 2024-05-09T01:03:44.040389+0000) 2024-05-09T01:03:55.666 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:55 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:55.458+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:30.342688+0000 front 2024-05-09T01:03:30.342762+0000 (oldest deadline 2024-05-09T01:03:52.042878+0000) 2024-05-09T01:03:55.666 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:55 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6[58994]: 2024-05-09T01:03:55.663+0000 7f5abd6ec700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.279932+0000 front 2024-05-09T01:03:02.279987+0000 (oldest deadline 2024-05-09T01:03:28.680789+0000) 2024-05-09T01:03:55.667 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:55 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6[58994]: 2024-05-09T01:03:55.663+0000 7f5abd6ec700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:14.482253+0000 front 2024-05-09T01:03:14.482294+0000 (oldest deadline 2024-05-09T01:03:38.582313+0000) 2024-05-09T01:03:55.667 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:55 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6[58994]: 2024-05-09T01:03:55.663+0000 7f5abd6ec700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:21.384074+0000 front 2024-05-09T01:03:21.384257+0000 (oldest deadline 2024-05-09T01:03:44.883973+0000) 2024-05-09T01:03:55.667 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:55 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6[58994]: 2024-05-09T01:03:55.663+0000 7f5abd6ec700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:30.085425+0000 front 2024-05-09T01:03:30.085516+0000 (oldest deadline 2024-05-09T01:03:54.785670+0000) 2024-05-09T01:03:55.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:55 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:55.317+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:03:55.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:55 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:55.317+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:12.758527+0000 front 2024-05-09T01:03:12.758364+0000 (oldest deadline 2024-05-09T01:03:35.658450+0000) 2024-05-09T01:03:55.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:55 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:55.317+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.259808+0000 front 2024-05-09T01:03:20.259685+0000 (oldest deadline 2024-05-09T01:03:46.159864+0000) 2024-05-09T01:03:56.105 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:03:55 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7[52359]: 2024-05-09T01:03:55.688+0000 7f4f992ef700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:14.583613+0000 front 2024-05-09T01:03:14.583770+0000 (oldest deadline 2024-05-09T01:03:35.683825+0000) 2024-05-09T01:03:56.106 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:03:55 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7[52359]: 2024-05-09T01:03:55.688+0000 7f4f992ef700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:21.485772+0000 front 2024-05-09T01:03:21.485552+0000 (oldest deadline 2024-05-09T01:03:46.185602+0000) 2024-05-09T01:03:56.106 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:03:55 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7[52359]: 2024-05-09T01:03:55.688+0000 7f4f992ef700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:26.686854+0000 front 2024-05-09T01:03:26.686891+0000 (oldest deadline 2024-05-09T01:03:55.487416+0000) 2024-05-09T01:03:56.186 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:03:55 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:03:55.776+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.533336+0000 front 2024-05-09T01:03:04.533539+0000 (oldest deadline 2024-05-09T01:03:28.633473+0000) 2024-05-09T01:03:56.186 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:03:55 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:03:55.776+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:18.535819+0000 front 2024-05-09T01:03:18.536013+0000 (oldest deadline 2024-05-09T01:03:44.435939+0000) 2024-05-09T01:03:56.186 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:03:55 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:03:55.776+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:30.638891+0000 front 2024-05-09T01:03:30.639015+0000 (oldest deadline 2024-05-09T01:03:53.538952+0000) 2024-05-09T01:03:56.363 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:56 smithi005 podman[112199]: 2024-05-09 01:03:56.081492095 +0000 UTC m=+5.465667345 container died 5347863aa7fcaf89d58960167d309a086d0b029315a52edcf11617d7dcca2c72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-05-09T01:03:56.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:03:56 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:03:56.235+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.667724+0000 front 2024-05-09T01:03:13.667762+0000 (oldest deadline 2024-05-09T01:03:35.967853+0000) 2024-05-09T01:03:56.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:03:56 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:03:56.235+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.668856+0000 front 2024-05-09T01:03:20.668887+0000 (oldest deadline 2024-05-09T01:03:45.968898+0000) 2024-05-09T01:03:56.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:03:56 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:03:56.235+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:29.371010+0000 front 2024-05-09T01:03:29.370862+0000 (oldest deadline 2024-05-09T01:03:54.070791+0000) 2024-05-09T01:03:56.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:56 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:56.350+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:03:56.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:56 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:56.350+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:12.758527+0000 front 2024-05-09T01:03:12.758364+0000 (oldest deadline 2024-05-09T01:03:35.658450+0000) 2024-05-09T01:03:56.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:56 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:56.350+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.259808+0000 front 2024-05-09T01:03:20.259685+0000 (oldest deadline 2024-05-09T01:03:46.159864+0000) 2024-05-09T01:03:56.790 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:56 smithi005 podman[112199]: 2024-05-09 01:03:56.537321738 +0000 UTC m=+5.921497027 container cleanup 5347863aa7fcaf89d58960167d309a086d0b029315a52edcf11617d7dcca2c72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6, org.label-schema.build-date=20240507, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-05-09T01:03:56.790 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:56 smithi005 bash[112199]: ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6 2024-05-09T01:03:56.791 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:56 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:56.497+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.737970+0000 front 2024-05-09T01:03:04.738119+0000 (oldest deadline 2024-05-09T01:03:27.037934+0000) 2024-05-09T01:03:56.791 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:56 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:56.497+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.439829+0000 front 2024-05-09T01:03:13.439669+0000 (oldest deadline 2024-05-09T01:03:38.739771+0000) 2024-05-09T01:03:56.791 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:56 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:56.497+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:18.740444+0000 front 2024-05-09T01:03:18.740379+0000 (oldest deadline 2024-05-09T01:03:44.040389+0000) 2024-05-09T01:03:56.791 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:56 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:56.497+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:30.342688+0000 front 2024-05-09T01:03:30.342762+0000 (oldest deadline 2024-05-09T01:03:52.042878+0000) 2024-05-09T01:03:57.105 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:03:56 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7[52359]: 2024-05-09T01:03:56.639+0000 7f4f992ef700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:14.583613+0000 front 2024-05-09T01:03:14.583770+0000 (oldest deadline 2024-05-09T01:03:35.683825+0000) 2024-05-09T01:03:57.105 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:03:56 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7[52359]: 2024-05-09T01:03:56.639+0000 7f4f992ef700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:21.485772+0000 front 2024-05-09T01:03:21.485552+0000 (oldest deadline 2024-05-09T01:03:46.185602+0000) 2024-05-09T01:03:57.105 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:03:56 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7[52359]: 2024-05-09T01:03:56.639+0000 7f4f992ef700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:26.686854+0000 front 2024-05-09T01:03:26.686891+0000 (oldest deadline 2024-05-09T01:03:55.487416+0000) 2024-05-09T01:03:57.186 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:03:56 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:03:56.797+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.533336+0000 front 2024-05-09T01:03:04.533539+0000 (oldest deadline 2024-05-09T01:03:28.633473+0000) 2024-05-09T01:03:57.187 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:03:56 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:03:56.797+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:18.535819+0000 front 2024-05-09T01:03:18.536013+0000 (oldest deadline 2024-05-09T01:03:44.435939+0000) 2024-05-09T01:03:57.187 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:03:56 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:03:56.797+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:30.638891+0000 front 2024-05-09T01:03:30.639015+0000 (oldest deadline 2024-05-09T01:03:53.538952+0000) 2024-05-09T01:03:57.188 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:56 smithi005 podman[112217]: 2024-05-09 01:03:56.86396895 +0000 UTC m=+0.783507536 container remove 5347863aa7fcaf89d58960167d309a086d0b029315a52edcf11617d7dcca2c72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-05-09T01:03:57.476 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:57 smithi005 podman[112306]: 2024-05-09 01:03:57.091064473 +0000 UTC m=+0.018856645 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T01:03:57.477 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:57 smithi005 podman[112306]: 2024-05-09 01:03:57.223177771 +0000 UTC m=+0.150969981 container create 8193d74e5ae6db1a167df081c936f41a28eb11f3699b8c8a22408b29fbdc5dbf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-09T01:03:57.477 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:57 smithi005 podman[112306]: 2024-05-09 01:03:57.476348278 +0000 UTC m=+0.404140434 container init 8193d74e5ae6db1a167df081c936f41a28eb11f3699b8c8a22408b29fbdc5dbf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6-deactivate, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5) 2024-05-09T01:03:57.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:03:57 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:03:57.232+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.667724+0000 front 2024-05-09T01:03:13.667762+0000 (oldest deadline 2024-05-09T01:03:35.967853+0000) 2024-05-09T01:03:57.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:03:57 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:03:57.232+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.668856+0000 front 2024-05-09T01:03:20.668887+0000 (oldest deadline 2024-05-09T01:03:45.968898+0000) 2024-05-09T01:03:57.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:03:57 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:03:57.232+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:29.371010+0000 front 2024-05-09T01:03:29.370862+0000 (oldest deadline 2024-05-09T01:03:54.070791+0000) 2024-05-09T01:03:57.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:57 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:57.374+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:03:57.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:57 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:57.374+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:12.758527+0000 front 2024-05-09T01:03:12.758364+0000 (oldest deadline 2024-05-09T01:03:35.658450+0000) 2024-05-09T01:03:57.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:57 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:57.374+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.259808+0000 front 2024-05-09T01:03:20.259685+0000 (oldest deadline 2024-05-09T01:03:46.159864+0000) 2024-05-09T01:03:57.786 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:57 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:57.522+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.737970+0000 front 2024-05-09T01:03:04.738119+0000 (oldest deadline 2024-05-09T01:03:27.037934+0000) 2024-05-09T01:03:57.786 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:57 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:57.522+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.439829+0000 front 2024-05-09T01:03:13.439669+0000 (oldest deadline 2024-05-09T01:03:38.739771+0000) 2024-05-09T01:03:57.786 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:57 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:57.522+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:18.740444+0000 front 2024-05-09T01:03:18.740379+0000 (oldest deadline 2024-05-09T01:03:44.040389+0000) 2024-05-09T01:03:57.786 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:57 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:57.522+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:30.342688+0000 front 2024-05-09T01:03:30.342762+0000 (oldest deadline 2024-05-09T01:03:52.042878+0000) 2024-05-09T01:03:57.786 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:57 smithi005 podman[112306]: 2024-05-09 01:03:57.480356832 +0000 UTC m=+0.408148977 container start 8193d74e5ae6db1a167df081c936f41a28eb11f3699b8c8a22408b29fbdc5dbf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6-deactivate, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_CLEAN=True) 2024-05-09T01:03:57.786 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:57 smithi005 podman[112306]: 2024-05-09 01:03:57.55939165 +0000 UTC m=+0.487183822 container attach 8193d74e5ae6db1a167df081c936f41a28eb11f3699b8c8a22408b29fbdc5dbf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-05-09T01:03:58.040 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 09 01:03:57 smithi005 podman[112306]: 2024-05-09 01:03:57.786070925 +0000 UTC m=+0.713863082 container died 8193d74e5ae6db1a167df081c936f41a28eb11f3699b8c8a22408b29fbdc5dbf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-6-deactivate, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-09T01:03:58.105 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:03:57 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7[52359]: 2024-05-09T01:03:57.679+0000 7f4f992ef700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:14.583613+0000 front 2024-05-09T01:03:14.583770+0000 (oldest deadline 2024-05-09T01:03:35.683825+0000) 2024-05-09T01:03:58.105 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:03:57 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7[52359]: 2024-05-09T01:03:57.679+0000 7f4f992ef700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:21.485772+0000 front 2024-05-09T01:03:21.485552+0000 (oldest deadline 2024-05-09T01:03:46.185602+0000) 2024-05-09T01:03:58.105 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:03:57 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7[52359]: 2024-05-09T01:03:57.679+0000 7f4f992ef700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:26.686854+0000 front 2024-05-09T01:03:26.686891+0000 (oldest deadline 2024-05-09T01:03:55.487416+0000) 2024-05-09T01:03:58.186 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:03:57 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:03:57.816+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.533336+0000 front 2024-05-09T01:03:04.533539+0000 (oldest deadline 2024-05-09T01:03:28.633473+0000) 2024-05-09T01:03:58.186 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:03:57 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:03:57.816+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:18.535819+0000 front 2024-05-09T01:03:18.536013+0000 (oldest deadline 2024-05-09T01:03:44.435939+0000) 2024-05-09T01:03:58.186 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:03:57 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:03:57.816+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:30.638891+0000 front 2024-05-09T01:03:30.639015+0000 (oldest deadline 2024-05-09T01:03:53.538952+0000) 2024-05-09T01:03:58.597 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@osd.6.service' 2024-05-09T01:03:58.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:03:58 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:03:58.267+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.667724+0000 front 2024-05-09T01:03:13.667762+0000 (oldest deadline 2024-05-09T01:03:35.967853+0000) 2024-05-09T01:03:58.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:03:58 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:03:58.267+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.668856+0000 front 2024-05-09T01:03:20.668887+0000 (oldest deadline 2024-05-09T01:03:45.968898+0000) 2024-05-09T01:03:58.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:03:58 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:03:58.267+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:29.371010+0000 front 2024-05-09T01:03:29.370862+0000 (oldest deadline 2024-05-09T01:03:54.070791+0000) 2024-05-09T01:03:58.640 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T01:03:58.641 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-05-09T01:03:58.641 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-05-09T01:03:58.641 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@osd.7 2024-05-09T01:03:58.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:58 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:58.368+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:03:58.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:58 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:58.368+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:12.758527+0000 front 2024-05-09T01:03:12.758364+0000 (oldest deadline 2024-05-09T01:03:35.658450+0000) 2024-05-09T01:03:58.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:58 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:58.368+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.259808+0000 front 2024-05-09T01:03:20.259685+0000 (oldest deadline 2024-05-09T01:03:46.159864+0000) 2024-05-09T01:03:58.790 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:58 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:58.501+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.737970+0000 front 2024-05-09T01:03:04.738119+0000 (oldest deadline 2024-05-09T01:03:27.037934+0000) 2024-05-09T01:03:58.790 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:58 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:58.501+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.439829+0000 front 2024-05-09T01:03:13.439669+0000 (oldest deadline 2024-05-09T01:03:38.739771+0000) 2024-05-09T01:03:58.790 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:58 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:58.501+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:18.740444+0000 front 2024-05-09T01:03:18.740379+0000 (oldest deadline 2024-05-09T01:03:44.040389+0000) 2024-05-09T01:03:58.791 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:58 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:58.501+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:30.342688+0000 front 2024-05-09T01:03:30.342762+0000 (oldest deadline 2024-05-09T01:03:52.042878+0000) 2024-05-09T01:03:58.951 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:03:58 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7[52359]: 2024-05-09T01:03:58.657+0000 7f4f992ef700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:14.583613+0000 front 2024-05-09T01:03:14.583770+0000 (oldest deadline 2024-05-09T01:03:35.683825+0000) 2024-05-09T01:03:58.951 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:03:58 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7[52359]: 2024-05-09T01:03:58.657+0000 7f4f992ef700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:21.485772+0000 front 2024-05-09T01:03:21.485552+0000 (oldest deadline 2024-05-09T01:03:46.185602+0000) 2024-05-09T01:03:58.951 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:03:58 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7[52359]: 2024-05-09T01:03:58.657+0000 7f4f992ef700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:26.686854+0000 front 2024-05-09T01:03:26.686891+0000 (oldest deadline 2024-05-09T01:03:55.487416+0000) 2024-05-09T01:03:58.951 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:03:58 smithi131 systemd[1]: Stopping Ceph osd.7 for d85814c6-0d9b-11ef-bc97-c7b262605968... 2024-05-09T01:03:59.186 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:03:58 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:03:58.840+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.533336+0000 front 2024-05-09T01:03:04.533539+0000 (oldest deadline 2024-05-09T01:03:28.633473+0000) 2024-05-09T01:03:59.186 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:03:58 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:03:58.840+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:18.535819+0000 front 2024-05-09T01:03:18.536013+0000 (oldest deadline 2024-05-09T01:03:44.435939+0000) 2024-05-09T01:03:59.186 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:03:58 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:03:58.840+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:30.638891+0000 front 2024-05-09T01:03:30.639015+0000 (oldest deadline 2024-05-09T01:03:53.538952+0000) 2024-05-09T01:03:59.246 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:03:58 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7[52359]: 2024-05-09T01:03:58.950+0000 7f4f9cd39700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T01:03:59.246 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:03:58 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7[52359]: 2024-05-09T01:03:58.950+0000 7f4f9cd39700 -1 osd.7 85 *** Got signal Terminated *** 2024-05-09T01:03:59.246 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:03:58 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7[52359]: 2024-05-09T01:03:58.950+0000 7f4f9cd39700 -1 osd.7 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T01:03:59.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:03:59 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:03:59.245+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.667724+0000 front 2024-05-09T01:03:13.667762+0000 (oldest deadline 2024-05-09T01:03:35.967853+0000) 2024-05-09T01:03:59.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:03:59 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:03:59.245+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.668856+0000 front 2024-05-09T01:03:20.668887+0000 (oldest deadline 2024-05-09T01:03:45.968898+0000) 2024-05-09T01:03:59.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:03:59 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:03:59.245+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:29.371010+0000 front 2024-05-09T01:03:29.370862+0000 (oldest deadline 2024-05-09T01:03:54.070791+0000) 2024-05-09T01:03:59.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:59 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:59.326+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:03:59.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:59 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:59.326+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:12.758527+0000 front 2024-05-09T01:03:12.758364+0000 (oldest deadline 2024-05-09T01:03:35.658450+0000) 2024-05-09T01:03:59.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:03:59 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:03:59.326+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.259808+0000 front 2024-05-09T01:03:20.259685+0000 (oldest deadline 2024-05-09T01:03:46.159864+0000) 2024-05-09T01:03:59.790 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:59 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:59.505+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.737970+0000 front 2024-05-09T01:03:04.738119+0000 (oldest deadline 2024-05-09T01:03:27.037934+0000) 2024-05-09T01:03:59.790 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:59 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:59.506+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.439829+0000 front 2024-05-09T01:03:13.439669+0000 (oldest deadline 2024-05-09T01:03:38.739771+0000) 2024-05-09T01:03:59.791 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:59 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:59.506+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:18.740444+0000 front 2024-05-09T01:03:18.740379+0000 (oldest deadline 2024-05-09T01:03:44.040389+0000) 2024-05-09T01:03:59.791 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:03:59 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:03:59.506+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:30.342688+0000 front 2024-05-09T01:03:30.342762+0000 (oldest deadline 2024-05-09T01:03:52.042878+0000) 2024-05-09T01:04:00.105 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:03:59 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7[52359]: 2024-05-09T01:03:59.685+0000 7f4f992ef700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:14.583613+0000 front 2024-05-09T01:03:14.583770+0000 (oldest deadline 2024-05-09T01:03:35.683825+0000) 2024-05-09T01:04:00.105 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:03:59 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7[52359]: 2024-05-09T01:03:59.686+0000 7f4f992ef700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:21.485772+0000 front 2024-05-09T01:03:21.485552+0000 (oldest deadline 2024-05-09T01:03:46.185602+0000) 2024-05-09T01:04:00.105 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:03:59 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7[52359]: 2024-05-09T01:03:59.686+0000 7f4f992ef700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:26.686854+0000 front 2024-05-09T01:03:26.686891+0000 (oldest deadline 2024-05-09T01:03:55.487416+0000) 2024-05-09T01:04:00.186 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:03:59 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:03:59.793+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.533336+0000 front 2024-05-09T01:03:04.533539+0000 (oldest deadline 2024-05-09T01:03:28.633473+0000) 2024-05-09T01:04:00.186 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:03:59 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:03:59.793+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:18.535819+0000 front 2024-05-09T01:03:18.536013+0000 (oldest deadline 2024-05-09T01:03:44.435939+0000) 2024-05-09T01:04:00.186 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:03:59 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:03:59.793+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:30.638891+0000 front 2024-05-09T01:03:30.639015+0000 (oldest deadline 2024-05-09T01:03:53.538952+0000) 2024-05-09T01:04:00.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:00 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:00.269+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.667724+0000 front 2024-05-09T01:03:13.667762+0000 (oldest deadline 2024-05-09T01:03:35.967853+0000) 2024-05-09T01:04:00.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:00 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:00.269+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.668856+0000 front 2024-05-09T01:03:20.668887+0000 (oldest deadline 2024-05-09T01:03:45.968898+0000) 2024-05-09T01:04:00.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:00 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:00.269+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:29.371010+0000 front 2024-05-09T01:03:29.370862+0000 (oldest deadline 2024-05-09T01:03:54.070791+0000) 2024-05-09T01:04:00.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:00 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:00.355+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:04:00.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:00 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:00.355+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:12.758527+0000 front 2024-05-09T01:03:12.758364+0000 (oldest deadline 2024-05-09T01:03:35.658450+0000) 2024-05-09T01:04:00.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:00 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:00.355+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.259808+0000 front 2024-05-09T01:03:20.259685+0000 (oldest deadline 2024-05-09T01:03:46.159864+0000) 2024-05-09T01:04:00.790 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:00 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:00.488+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.737970+0000 front 2024-05-09T01:03:04.738119+0000 (oldest deadline 2024-05-09T01:03:27.037934+0000) 2024-05-09T01:04:00.790 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:00 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:00.488+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.439829+0000 front 2024-05-09T01:03:13.439669+0000 (oldest deadline 2024-05-09T01:03:38.739771+0000) 2024-05-09T01:04:00.790 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:00 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:00.488+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:18.740444+0000 front 2024-05-09T01:03:18.740379+0000 (oldest deadline 2024-05-09T01:03:44.040389+0000) 2024-05-09T01:04:00.791 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:00 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:00.488+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:30.342688+0000 front 2024-05-09T01:03:30.342762+0000 (oldest deadline 2024-05-09T01:03:52.042878+0000) 2024-05-09T01:04:01.105 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:04:00 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7[52359]: 2024-05-09T01:04:00.659+0000 7f4f992ef700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:14.583613+0000 front 2024-05-09T01:03:14.583770+0000 (oldest deadline 2024-05-09T01:03:35.683825+0000) 2024-05-09T01:04:01.105 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:04:00 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7[52359]: 2024-05-09T01:04:00.659+0000 7f4f992ef700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:21.485772+0000 front 2024-05-09T01:03:21.485552+0000 (oldest deadline 2024-05-09T01:03:46.185602+0000) 2024-05-09T01:04:01.105 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:04:00 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7[52359]: 2024-05-09T01:04:00.659+0000 7f4f992ef700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:26.686854+0000 front 2024-05-09T01:03:26.686891+0000 (oldest deadline 2024-05-09T01:03:55.487416+0000) 2024-05-09T01:04:01.186 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:04:00 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:04:00.771+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.533336+0000 front 2024-05-09T01:03:04.533539+0000 (oldest deadline 2024-05-09T01:03:28.633473+0000) 2024-05-09T01:04:01.186 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:04:00 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:04:00.771+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:18.535819+0000 front 2024-05-09T01:03:18.536013+0000 (oldest deadline 2024-05-09T01:03:44.435939+0000) 2024-05-09T01:04:01.186 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:04:00 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:04:00.771+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:30.638891+0000 front 2024-05-09T01:03:30.639015+0000 (oldest deadline 2024-05-09T01:03:53.538952+0000) 2024-05-09T01:04:01.186 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:04:00 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:04:00.771+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:37.040226+0000 front 2024-05-09T01:03:37.040409+0000 (oldest deadline 2024-05-09T01:04:00.540204+0000) 2024-05-09T01:04:01.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:01 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:01.286+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.667724+0000 front 2024-05-09T01:03:13.667762+0000 (oldest deadline 2024-05-09T01:03:35.967853+0000) 2024-05-09T01:04:01.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:01 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:01.286+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.668856+0000 front 2024-05-09T01:03:20.668887+0000 (oldest deadline 2024-05-09T01:03:45.968898+0000) 2024-05-09T01:04:01.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:01 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:01.286+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:29.371010+0000 front 2024-05-09T01:03:29.370862+0000 (oldest deadline 2024-05-09T01:03:54.070791+0000) 2024-05-09T01:04:01.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:01 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:01.286+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:36.371823+0000 front 2024-05-09T01:03:36.371873+0000 (oldest deadline 2024-05-09T01:04:01.071919+0000) 2024-05-09T01:04:01.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:01 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:01.400+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:04:01.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:01 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:01.400+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:12.758527+0000 front 2024-05-09T01:03:12.758364+0000 (oldest deadline 2024-05-09T01:03:35.658450+0000) 2024-05-09T01:04:01.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:01 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:01.400+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.259808+0000 front 2024-05-09T01:03:20.259685+0000 (oldest deadline 2024-05-09T01:03:46.159864+0000) 2024-05-09T01:04:01.790 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:01 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:01.499+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.737970+0000 front 2024-05-09T01:03:04.738119+0000 (oldest deadline 2024-05-09T01:03:27.037934+0000) 2024-05-09T01:04:01.790 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:01 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:01.499+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.439829+0000 front 2024-05-09T01:03:13.439669+0000 (oldest deadline 2024-05-09T01:03:38.739771+0000) 2024-05-09T01:04:01.790 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:01 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:01.499+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:18.740444+0000 front 2024-05-09T01:03:18.740379+0000 (oldest deadline 2024-05-09T01:03:44.040389+0000) 2024-05-09T01:04:01.791 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:01 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:01.499+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:30.342688+0000 front 2024-05-09T01:03:30.342762+0000 (oldest deadline 2024-05-09T01:03:52.042878+0000) 2024-05-09T01:04:02.105 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:04:01 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7[52359]: 2024-05-09T01:04:01.667+0000 7f4f992ef700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:14.583613+0000 front 2024-05-09T01:03:14.583770+0000 (oldest deadline 2024-05-09T01:03:35.683825+0000) 2024-05-09T01:04:02.105 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:04:01 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7[52359]: 2024-05-09T01:04:01.667+0000 7f4f992ef700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:21.485772+0000 front 2024-05-09T01:03:21.485552+0000 (oldest deadline 2024-05-09T01:03:46.185602+0000) 2024-05-09T01:04:02.105 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:04:01 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7[52359]: 2024-05-09T01:04:01.667+0000 7f4f992ef700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:26.686854+0000 front 2024-05-09T01:03:26.686891+0000 (oldest deadline 2024-05-09T01:03:55.487416+0000) 2024-05-09T01:04:02.105 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:04:01 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7[52359]: 2024-05-09T01:04:01.667+0000 7f4f992ef700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:38.888761+0000 front 2024-05-09T01:03:38.888818+0000 (oldest deadline 2024-05-09T01:04:01.188910+0000) 2024-05-09T01:04:02.186 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:04:01 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:04:01.751+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.533336+0000 front 2024-05-09T01:03:04.533539+0000 (oldest deadline 2024-05-09T01:03:28.633473+0000) 2024-05-09T01:04:02.186 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:04:01 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:04:01.751+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:18.535819+0000 front 2024-05-09T01:03:18.536013+0000 (oldest deadline 2024-05-09T01:03:44.435939+0000) 2024-05-09T01:04:02.186 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:04:01 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:04:01.751+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:30.638891+0000 front 2024-05-09T01:03:30.639015+0000 (oldest deadline 2024-05-09T01:03:53.538952+0000) 2024-05-09T01:04:02.186 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:04:01 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:04:01.751+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:37.040226+0000 front 2024-05-09T01:03:37.040409+0000 (oldest deadline 2024-05-09T01:04:00.540204+0000) 2024-05-09T01:04:02.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:02 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:02.239+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.667724+0000 front 2024-05-09T01:03:13.667762+0000 (oldest deadline 2024-05-09T01:03:35.967853+0000) 2024-05-09T01:04:02.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:02 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:02.239+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.668856+0000 front 2024-05-09T01:03:20.668887+0000 (oldest deadline 2024-05-09T01:03:45.968898+0000) 2024-05-09T01:04:02.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:02 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:02.239+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:29.371010+0000 front 2024-05-09T01:03:29.370862+0000 (oldest deadline 2024-05-09T01:03:54.070791+0000) 2024-05-09T01:04:02.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:02 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:02.239+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:36.371823+0000 front 2024-05-09T01:03:36.371873+0000 (oldest deadline 2024-05-09T01:04:01.071919+0000) 2024-05-09T01:04:02.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:02 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:02.411+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:04:02.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:02 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:02.411+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:12.758527+0000 front 2024-05-09T01:03:12.758364+0000 (oldest deadline 2024-05-09T01:03:35.658450+0000) 2024-05-09T01:04:02.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:02 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:02.411+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.259808+0000 front 2024-05-09T01:03:20.259685+0000 (oldest deadline 2024-05-09T01:03:46.159864+0000) 2024-05-09T01:04:02.790 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:02 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:02.515+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.737970+0000 front 2024-05-09T01:03:04.738119+0000 (oldest deadline 2024-05-09T01:03:27.037934+0000) 2024-05-09T01:04:02.790 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:02 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:02.515+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.439829+0000 front 2024-05-09T01:03:13.439669+0000 (oldest deadline 2024-05-09T01:03:38.739771+0000) 2024-05-09T01:04:02.791 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:02 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:02.515+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:18.740444+0000 front 2024-05-09T01:03:18.740379+0000 (oldest deadline 2024-05-09T01:03:44.040389+0000) 2024-05-09T01:04:02.791 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:02 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:02.515+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:30.342688+0000 front 2024-05-09T01:03:30.342762+0000 (oldest deadline 2024-05-09T01:03:52.042878+0000) 2024-05-09T01:04:03.119 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:04:02 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7[52359]: 2024-05-09T01:04:02.695+0000 7f4f992ef700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:14.583613+0000 front 2024-05-09T01:03:14.583770+0000 (oldest deadline 2024-05-09T01:03:35.683825+0000) 2024-05-09T01:04:03.119 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:04:02 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7[52359]: 2024-05-09T01:04:02.695+0000 7f4f992ef700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:21.485772+0000 front 2024-05-09T01:03:21.485552+0000 (oldest deadline 2024-05-09T01:03:46.185602+0000) 2024-05-09T01:04:03.119 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:04:02 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7[52359]: 2024-05-09T01:04:02.695+0000 7f4f992ef700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:26.686854+0000 front 2024-05-09T01:03:26.686891+0000 (oldest deadline 2024-05-09T01:03:55.487416+0000) 2024-05-09T01:04:03.119 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:04:02 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7[52359]: 2024-05-09T01:04:02.695+0000 7f4f992ef700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:38.888761+0000 front 2024-05-09T01:03:38.888818+0000 (oldest deadline 2024-05-09T01:04:01.188910+0000) 2024-05-09T01:04:03.186 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:04:02 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:04:02.750+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.533336+0000 front 2024-05-09T01:03:04.533539+0000 (oldest deadline 2024-05-09T01:03:28.633473+0000) 2024-05-09T01:04:03.186 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:04:02 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:04:02.750+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:18.535819+0000 front 2024-05-09T01:03:18.536013+0000 (oldest deadline 2024-05-09T01:03:44.435939+0000) 2024-05-09T01:04:03.186 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:04:02 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:04:02.750+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:30.638891+0000 front 2024-05-09T01:03:30.639015+0000 (oldest deadline 2024-05-09T01:03:53.538952+0000) 2024-05-09T01:04:03.186 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:04:02 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:04:02.750+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:37.040226+0000 front 2024-05-09T01:03:37.040409+0000 (oldest deadline 2024-05-09T01:04:00.540204+0000) 2024-05-09T01:04:03.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:03 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:03.225+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.667724+0000 front 2024-05-09T01:03:13.667762+0000 (oldest deadline 2024-05-09T01:03:35.967853+0000) 2024-05-09T01:04:03.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:03 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:03.225+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.668856+0000 front 2024-05-09T01:03:20.668887+0000 (oldest deadline 2024-05-09T01:03:45.968898+0000) 2024-05-09T01:04:03.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:03 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:03.225+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:29.371010+0000 front 2024-05-09T01:03:29.370862+0000 (oldest deadline 2024-05-09T01:03:54.070791+0000) 2024-05-09T01:04:03.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:03 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:03.225+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:36.371823+0000 front 2024-05-09T01:03:36.371873+0000 (oldest deadline 2024-05-09T01:04:01.071919+0000) 2024-05-09T01:04:03.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:03 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:03.387+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:04:03.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:03 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:03.387+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:12.758527+0000 front 2024-05-09T01:03:12.758364+0000 (oldest deadline 2024-05-09T01:03:35.658450+0000) 2024-05-09T01:04:03.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:03 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:03.387+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.259808+0000 front 2024-05-09T01:03:20.259685+0000 (oldest deadline 2024-05-09T01:03:46.159864+0000) 2024-05-09T01:04:03.790 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:03 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:03.466+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.737970+0000 front 2024-05-09T01:03:04.738119+0000 (oldest deadline 2024-05-09T01:03:27.037934+0000) 2024-05-09T01:04:03.790 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:03 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:03.466+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.439829+0000 front 2024-05-09T01:03:13.439669+0000 (oldest deadline 2024-05-09T01:03:38.739771+0000) 2024-05-09T01:04:03.790 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:03 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:03.466+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:18.740444+0000 front 2024-05-09T01:03:18.740379+0000 (oldest deadline 2024-05-09T01:03:44.040389+0000) 2024-05-09T01:04:03.791 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:03 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:03.466+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:30.342688+0000 front 2024-05-09T01:03:30.342762+0000 (oldest deadline 2024-05-09T01:03:52.042878+0000) 2024-05-09T01:04:04.105 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:04:03 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7[52359]: 2024-05-09T01:04:03.703+0000 7f4f992ef700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:14.583613+0000 front 2024-05-09T01:03:14.583770+0000 (oldest deadline 2024-05-09T01:03:35.683825+0000) 2024-05-09T01:04:04.105 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:04:03 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7[52359]: 2024-05-09T01:04:03.703+0000 7f4f992ef700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:21.485772+0000 front 2024-05-09T01:03:21.485552+0000 (oldest deadline 2024-05-09T01:03:46.185602+0000) 2024-05-09T01:04:04.105 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:04:03 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7[52359]: 2024-05-09T01:04:03.703+0000 7f4f992ef700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:26.686854+0000 front 2024-05-09T01:03:26.686891+0000 (oldest deadline 2024-05-09T01:03:55.487416+0000) 2024-05-09T01:04:04.105 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:04:03 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7[52359]: 2024-05-09T01:04:03.703+0000 7f4f992ef700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:38.888761+0000 front 2024-05-09T01:03:38.888818+0000 (oldest deadline 2024-05-09T01:04:01.188910+0000) 2024-05-09T01:04:04.186 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:04:03 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:04:03.748+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.533336+0000 front 2024-05-09T01:03:04.533539+0000 (oldest deadline 2024-05-09T01:03:28.633473+0000) 2024-05-09T01:04:04.186 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:04:03 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:04:03.748+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:18.535819+0000 front 2024-05-09T01:03:18.536013+0000 (oldest deadline 2024-05-09T01:03:44.435939+0000) 2024-05-09T01:04:04.186 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:04:03 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:04:03.748+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:30.638891+0000 front 2024-05-09T01:03:30.639015+0000 (oldest deadline 2024-05-09T01:03:53.538952+0000) 2024-05-09T01:04:04.186 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:04:03 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:04:03.748+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:37.040226+0000 front 2024-05-09T01:03:37.040409+0000 (oldest deadline 2024-05-09T01:04:00.540204+0000) 2024-05-09T01:04:04.594 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:04:04 smithi131 podman[70452]: 2024-05-09 01:04:04.333529205 +0000 UTC m=+5.497942973 container died 3c7858244e712a381dcbb0536f7c499fc59d05e4c2cf0559fa5a64d64e14bb64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-09T01:04:04.594 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:04 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:04.199+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.667724+0000 front 2024-05-09T01:03:13.667762+0000 (oldest deadline 2024-05-09T01:03:35.967853+0000) 2024-05-09T01:04:04.594 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:04 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:04.199+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.668856+0000 front 2024-05-09T01:03:20.668887+0000 (oldest deadline 2024-05-09T01:03:45.968898+0000) 2024-05-09T01:04:04.594 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:04 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:04.199+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:29.371010+0000 front 2024-05-09T01:03:29.370862+0000 (oldest deadline 2024-05-09T01:03:54.070791+0000) 2024-05-09T01:04:04.594 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:04 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:04.199+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:36.371823+0000 front 2024-05-09T01:03:36.371873+0000 (oldest deadline 2024-05-09T01:04:01.071919+0000) 2024-05-09T01:04:04.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:04 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:04.355+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:04:04.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:04 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:04.355+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:12.758527+0000 front 2024-05-09T01:03:12.758364+0000 (oldest deadline 2024-05-09T01:03:35.658450+0000) 2024-05-09T01:04:04.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:04 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:04.355+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.259808+0000 front 2024-05-09T01:03:20.259685+0000 (oldest deadline 2024-05-09T01:03:46.159864+0000) 2024-05-09T01:04:04.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:04 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:04.355+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:38.362243+0000 front 2024-05-09T01:03:38.362207+0000 (oldest deadline 2024-05-09T01:04:03.662309+0000) 2024-05-09T01:04:04.790 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:04 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:04.515+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.737970+0000 front 2024-05-09T01:03:04.738119+0000 (oldest deadline 2024-05-09T01:03:27.037934+0000) 2024-05-09T01:04:04.790 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:04 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:04.515+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.439829+0000 front 2024-05-09T01:03:13.439669+0000 (oldest deadline 2024-05-09T01:03:38.739771+0000) 2024-05-09T01:04:04.791 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:04 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:04.515+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:18.740444+0000 front 2024-05-09T01:03:18.740379+0000 (oldest deadline 2024-05-09T01:03:44.040389+0000) 2024-05-09T01:04:04.791 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:04 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:04.515+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:30.342688+0000 front 2024-05-09T01:03:30.342762+0000 (oldest deadline 2024-05-09T01:03:52.042878+0000) 2024-05-09T01:04:04.855 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:04:04 smithi131 podman[70452]: 2024-05-09 01:04:04.684138357 +0000 UTC m=+5.848552130 container cleanup 3c7858244e712a381dcbb0536f7c499fc59d05e4c2cf0559fa5a64d64e14bb64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-09T01:04:04.855 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:04:04 smithi131 bash[70452]: ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7 2024-05-09T01:04:05.186 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:04:04 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:04:04.703+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.533336+0000 front 2024-05-09T01:03:04.533539+0000 (oldest deadline 2024-05-09T01:03:28.633473+0000) 2024-05-09T01:04:05.186 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:04:04 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:04:04.703+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:18.535819+0000 front 2024-05-09T01:03:18.536013+0000 (oldest deadline 2024-05-09T01:03:44.435939+0000) 2024-05-09T01:04:05.186 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:04:04 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:04:04.703+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:30.638891+0000 front 2024-05-09T01:03:30.639015+0000 (oldest deadline 2024-05-09T01:03:53.538952+0000) 2024-05-09T01:04:05.186 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:04:04 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:04:04.703+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:37.040226+0000 front 2024-05-09T01:03:37.040409+0000 (oldest deadline 2024-05-09T01:04:00.540204+0000) 2024-05-09T01:04:05.294 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:04:04 smithi131 podman[70469]: 2024-05-09 01:04:04.943866259 +0000 UTC m=+0.607403798 container remove 3c7858244e712a381dcbb0536f7c499fc59d05e4c2cf0559fa5a64d64e14bb64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True) 2024-05-09T01:04:05.295 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:05 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:05.169+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.667724+0000 front 2024-05-09T01:03:13.667762+0000 (oldest deadline 2024-05-09T01:03:35.967853+0000) 2024-05-09T01:04:05.295 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:05 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:05.169+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.668856+0000 front 2024-05-09T01:03:20.668887+0000 (oldest deadline 2024-05-09T01:03:45.968898+0000) 2024-05-09T01:04:05.295 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:05 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:05.169+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:29.371010+0000 front 2024-05-09T01:03:29.370862+0000 (oldest deadline 2024-05-09T01:03:54.070791+0000) 2024-05-09T01:04:05.295 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:05 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:05.169+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:36.371823+0000 front 2024-05-09T01:03:36.371873+0000 (oldest deadline 2024-05-09T01:04:01.071919+0000) 2024-05-09T01:04:05.555 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:04:05 smithi131 podman[70559]: 2024-05-09 01:04:05.294704159 +0000 UTC m=+0.112922842 container create b06bcd65749285b915903154ccae8eebc08cc9131f46bd5acccb44dddfc44638 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-05-09T01:04:05.555 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:04:05 smithi131 podman[70559]: 2024-05-09 01:04:05.19989258 +0000 UTC m=+0.018111284 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T01:04:05.556 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:04:05 smithi131 podman[70559]: 2024-05-09 01:04:05.555527633 +0000 UTC m=+0.373746317 container init b06bcd65749285b915903154ccae8eebc08cc9131f46bd5acccb44dddfc44638 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7-deactivate, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-05-09T01:04:05.675 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:05 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:05.404+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:04:05.675 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:05 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:05.404+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:12.758527+0000 front 2024-05-09T01:03:12.758364+0000 (oldest deadline 2024-05-09T01:03:35.658450+0000) 2024-05-09T01:04:05.675 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:05 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:05.404+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.259808+0000 front 2024-05-09T01:03:20.259685+0000 (oldest deadline 2024-05-09T01:03:46.159864+0000) 2024-05-09T01:04:05.675 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:05 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:05.404+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:38.362243+0000 front 2024-05-09T01:03:38.362207+0000 (oldest deadline 2024-05-09T01:04:03.662309+0000) 2024-05-09T01:04:05.790 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:05 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:05.528+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.737970+0000 front 2024-05-09T01:03:04.738119+0000 (oldest deadline 2024-05-09T01:03:27.037934+0000) 2024-05-09T01:04:05.791 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:05 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:05.529+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.439829+0000 front 2024-05-09T01:03:13.439669+0000 (oldest deadline 2024-05-09T01:03:38.739771+0000) 2024-05-09T01:04:05.791 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:05 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:05.529+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:18.740444+0000 front 2024-05-09T01:03:18.740379+0000 (oldest deadline 2024-05-09T01:03:44.040389+0000) 2024-05-09T01:04:05.791 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:05 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:05.529+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:30.342688+0000 front 2024-05-09T01:03:30.342762+0000 (oldest deadline 2024-05-09T01:03:52.042878+0000) 2024-05-09T01:04:05.853 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:04:05 smithi131 podman[70559]: 2024-05-09 01:04:05.55878949 +0000 UTC m=+0.377008175 container start b06bcd65749285b915903154ccae8eebc08cc9131f46bd5acccb44dddfc44638 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True) 2024-05-09T01:04:05.853 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:04:05 smithi131 podman[70559]: 2024-05-09 01:04:05.655801556 +0000 UTC m=+0.474020252 container attach b06bcd65749285b915903154ccae8eebc08cc9131f46bd5acccb44dddfc44638 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240507) 2024-05-09T01:04:05.936 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:04:05 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:04:05.674+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.533336+0000 front 2024-05-09T01:03:04.533539+0000 (oldest deadline 2024-05-09T01:03:28.633473+0000) 2024-05-09T01:04:05.936 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:04:05 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:04:05.674+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:18.535819+0000 front 2024-05-09T01:03:18.536013+0000 (oldest deadline 2024-05-09T01:03:44.435939+0000) 2024-05-09T01:04:05.936 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:04:05 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:04:05.674+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:30.638891+0000 front 2024-05-09T01:03:30.639015+0000 (oldest deadline 2024-05-09T01:03:53.538952+0000) 2024-05-09T01:04:05.936 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:04:05 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:04:05.674+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:37.040226+0000 front 2024-05-09T01:03:37.040409+0000 (oldest deadline 2024-05-09T01:04:00.540204+0000) 2024-05-09T01:04:06.105 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:04:05 smithi131 podman[70559]: 2024-05-09 01:04:05.853163181 +0000 UTC m=+0.671381866 container died b06bcd65749285b915903154ccae8eebc08cc9131f46bd5acccb44dddfc44638 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507) 2024-05-09T01:04:06.374 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@osd.7.service' 2024-05-09T01:04:06.396 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:06 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:06.185+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.667724+0000 front 2024-05-09T01:03:13.667762+0000 (oldest deadline 2024-05-09T01:03:35.967853+0000) 2024-05-09T01:04:06.396 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:06 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:06.185+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.668856+0000 front 2024-05-09T01:03:20.668887+0000 (oldest deadline 2024-05-09T01:03:45.968898+0000) 2024-05-09T01:04:06.396 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:06 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:06.185+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:29.371010+0000 front 2024-05-09T01:03:29.370862+0000 (oldest deadline 2024-05-09T01:03:54.070791+0000) 2024-05-09T01:04:06.397 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:06 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:06.185+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:36.371823+0000 front 2024-05-09T01:03:36.371873+0000 (oldest deadline 2024-05-09T01:04:01.071919+0000) 2024-05-09T01:04:06.397 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:04:06 smithi131 podman[70585]: 2024-05-09 01:04:06.359503611 +0000 UTC m=+0.501019342 container remove b06bcd65749285b915903154ccae8eebc08cc9131f46bd5acccb44dddfc44638 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-05-09T01:04:06.397 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:04:06 smithi131 systemd[1]: ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@osd.7.service: Deactivated successfully. 2024-05-09T01:04:06.397 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:04:06 smithi131 systemd[1]: Stopped Ceph osd.7 for d85814c6-0d9b-11ef-bc97-c7b262605968. 2024-05-09T01:04:06.397 INFO:journalctl@ceph.osd.7.smithi131.stdout:May 09 01:04:06 smithi131 systemd[1]: ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@osd.7.service: Consumed 10min 13.913s CPU time. 2024-05-09T01:04:06.411 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T01:04:06.412 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-05-09T01:04:06.412 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-05-09T01:04:06.412 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@osd.8 2024-05-09T01:04:06.645 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:04:06 smithi157 systemd[1]: Stopping Ceph osd.8 for d85814c6-0d9b-11ef-bc97-c7b262605968... 2024-05-09T01:04:06.645 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:06 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:06.387+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:04:06.646 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:06 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:06.387+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:12.758527+0000 front 2024-05-09T01:03:12.758364+0000 (oldest deadline 2024-05-09T01:03:35.658450+0000) 2024-05-09T01:04:06.646 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:06 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:06.387+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.259808+0000 front 2024-05-09T01:03:20.259685+0000 (oldest deadline 2024-05-09T01:03:46.159864+0000) 2024-05-09T01:04:06.646 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:06 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:06.387+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:38.362243+0000 front 2024-05-09T01:03:38.362207+0000 (oldest deadline 2024-05-09T01:04:03.662309+0000) 2024-05-09T01:04:06.936 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:04:06 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:04:06.644+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.533336+0000 front 2024-05-09T01:03:04.533539+0000 (oldest deadline 2024-05-09T01:03:28.633473+0000) 2024-05-09T01:04:06.936 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:04:06 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:04:06.644+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:18.535819+0000 front 2024-05-09T01:03:18.536013+0000 (oldest deadline 2024-05-09T01:03:44.435939+0000) 2024-05-09T01:04:06.936 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:04:06 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:04:06.644+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:30.638891+0000 front 2024-05-09T01:03:30.639015+0000 (oldest deadline 2024-05-09T01:03:53.538952+0000) 2024-05-09T01:04:06.936 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:04:06 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:04:06.644+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:37.040226+0000 front 2024-05-09T01:03:37.040409+0000 (oldest deadline 2024-05-09T01:04:00.540204+0000) 2024-05-09T01:04:06.936 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:04:06 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:04:06.733+0000 7f2aefcfb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T01:04:06.937 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:04:06 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:04:06.733+0000 7f2aefcfb700 -1 osd.8 85 *** Got signal Terminated *** 2024-05-09T01:04:06.937 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:04:06 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:04:06.733+0000 7f2aefcfb700 -1 osd.8 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T01:04:07.040 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:06 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:06.564+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.737970+0000 front 2024-05-09T01:03:04.738119+0000 (oldest deadline 2024-05-09T01:03:27.037934+0000) 2024-05-09T01:04:07.040 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:06 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:06.564+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.439829+0000 front 2024-05-09T01:03:13.439669+0000 (oldest deadline 2024-05-09T01:03:38.739771+0000) 2024-05-09T01:04:07.040 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:06 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:06.564+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:18.740444+0000 front 2024-05-09T01:03:18.740379+0000 (oldest deadline 2024-05-09T01:03:44.040389+0000) 2024-05-09T01:04:07.041 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:06 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:06.564+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:30.342688+0000 front 2024-05-09T01:03:30.342762+0000 (oldest deadline 2024-05-09T01:03:52.042878+0000) 2024-05-09T01:04:07.557 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:07 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:07.167+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.667724+0000 front 2024-05-09T01:03:13.667762+0000 (oldest deadline 2024-05-09T01:03:35.967853+0000) 2024-05-09T01:04:07.557 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:07 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:07.167+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.668856+0000 front 2024-05-09T01:03:20.668887+0000 (oldest deadline 2024-05-09T01:03:45.968898+0000) 2024-05-09T01:04:07.557 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:07 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:07.167+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:29.371010+0000 front 2024-05-09T01:03:29.370862+0000 (oldest deadline 2024-05-09T01:03:54.070791+0000) 2024-05-09T01:04:07.557 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:07 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:07.167+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:36.371823+0000 front 2024-05-09T01:03:36.371873+0000 (oldest deadline 2024-05-09T01:04:01.071919+0000) 2024-05-09T01:04:07.631 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:07 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:07.361+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:04:07.631 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:07 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:07.361+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:12.758527+0000 front 2024-05-09T01:03:12.758364+0000 (oldest deadline 2024-05-09T01:03:35.658450+0000) 2024-05-09T01:04:07.631 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:07 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:07.361+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.259808+0000 front 2024-05-09T01:03:20.259685+0000 (oldest deadline 2024-05-09T01:03:46.159864+0000) 2024-05-09T01:04:07.631 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:07 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:07.361+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:38.362243+0000 front 2024-05-09T01:03:38.362207+0000 (oldest deadline 2024-05-09T01:04:03.662309+0000) 2024-05-09T01:04:07.908 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:04:07 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:04:07.630+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.533336+0000 front 2024-05-09T01:03:04.533539+0000 (oldest deadline 2024-05-09T01:03:28.633473+0000) 2024-05-09T01:04:07.908 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:04:07 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:04:07.630+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:18.535819+0000 front 2024-05-09T01:03:18.536013+0000 (oldest deadline 2024-05-09T01:03:44.435939+0000) 2024-05-09T01:04:07.908 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:04:07 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:04:07.630+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:30.638891+0000 front 2024-05-09T01:03:30.639015+0000 (oldest deadline 2024-05-09T01:03:53.538952+0000) 2024-05-09T01:04:07.908 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:04:07 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:04:07.630+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:37.040226+0000 front 2024-05-09T01:03:37.040409+0000 (oldest deadline 2024-05-09T01:04:00.540204+0000) 2024-05-09T01:04:07.942 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:07 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:07.548+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.737970+0000 front 2024-05-09T01:03:04.738119+0000 (oldest deadline 2024-05-09T01:03:27.037934+0000) 2024-05-09T01:04:07.942 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:07 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:07.548+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.439829+0000 front 2024-05-09T01:03:13.439669+0000 (oldest deadline 2024-05-09T01:03:38.739771+0000) 2024-05-09T01:04:07.942 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:07 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:07.548+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:18.740444+0000 front 2024-05-09T01:03:18.740379+0000 (oldest deadline 2024-05-09T01:03:44.040389+0000) 2024-05-09T01:04:07.942 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:07 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:07.548+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:30.342688+0000 front 2024-05-09T01:03:30.342762+0000 (oldest deadline 2024-05-09T01:03:52.042878+0000) 2024-05-09T01:04:08.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:08 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:08.134+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.667724+0000 front 2024-05-09T01:03:13.667762+0000 (oldest deadline 2024-05-09T01:03:35.967853+0000) 2024-05-09T01:04:08.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:08 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:08.134+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.668856+0000 front 2024-05-09T01:03:20.668887+0000 (oldest deadline 2024-05-09T01:03:45.968898+0000) 2024-05-09T01:04:08.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:08 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:08.134+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:29.371010+0000 front 2024-05-09T01:03:29.370862+0000 (oldest deadline 2024-05-09T01:03:54.070791+0000) 2024-05-09T01:04:08.606 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:08 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:08.134+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:36.371823+0000 front 2024-05-09T01:03:36.371873+0000 (oldest deadline 2024-05-09T01:04:01.071919+0000) 2024-05-09T01:04:08.686 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:04:08 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:04:08.593+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.533336+0000 front 2024-05-09T01:03:04.533539+0000 (oldest deadline 2024-05-09T01:03:28.633473+0000) 2024-05-09T01:04:08.686 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:04:08 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:04:08.593+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:18.535819+0000 front 2024-05-09T01:03:18.536013+0000 (oldest deadline 2024-05-09T01:03:44.435939+0000) 2024-05-09T01:04:08.686 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:04:08 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:04:08.593+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:30.638891+0000 front 2024-05-09T01:03:30.639015+0000 (oldest deadline 2024-05-09T01:03:53.538952+0000) 2024-05-09T01:04:08.686 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:04:08 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:04:08.593+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:37.040226+0000 front 2024-05-09T01:03:37.040409+0000 (oldest deadline 2024-05-09T01:04:00.540204+0000) 2024-05-09T01:04:08.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:08 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:08.373+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:04:08.687 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:08 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:08.373+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:12.758527+0000 front 2024-05-09T01:03:12.758364+0000 (oldest deadline 2024-05-09T01:03:35.658450+0000) 2024-05-09T01:04:08.687 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:08 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:08.373+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.259808+0000 front 2024-05-09T01:03:20.259685+0000 (oldest deadline 2024-05-09T01:03:46.159864+0000) 2024-05-09T01:04:08.687 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:08 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:08.373+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:38.362243+0000 front 2024-05-09T01:03:38.362207+0000 (oldest deadline 2024-05-09T01:04:03.662309+0000) 2024-05-09T01:04:09.040 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:08 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:08.565+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.737970+0000 front 2024-05-09T01:03:04.738119+0000 (oldest deadline 2024-05-09T01:03:27.037934+0000) 2024-05-09T01:04:09.041 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:08 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:08.565+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.439829+0000 front 2024-05-09T01:03:13.439669+0000 (oldest deadline 2024-05-09T01:03:38.739771+0000) 2024-05-09T01:04:09.041 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:08 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:08.565+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:18.740444+0000 front 2024-05-09T01:03:18.740379+0000 (oldest deadline 2024-05-09T01:03:44.040389+0000) 2024-05-09T01:04:09.041 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:08 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:08.565+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:30.342688+0000 front 2024-05-09T01:03:30.342762+0000 (oldest deadline 2024-05-09T01:03:52.042878+0000) 2024-05-09T01:04:09.507 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:09 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:09.106+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.667724+0000 front 2024-05-09T01:03:13.667762+0000 (oldest deadline 2024-05-09T01:03:35.967853+0000) 2024-05-09T01:04:09.507 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:09 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:09.106+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.668856+0000 front 2024-05-09T01:03:20.668887+0000 (oldest deadline 2024-05-09T01:03:45.968898+0000) 2024-05-09T01:04:09.507 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:09 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:09.106+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:29.371010+0000 front 2024-05-09T01:03:29.370862+0000 (oldest deadline 2024-05-09T01:03:54.070791+0000) 2024-05-09T01:04:09.507 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:09 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:09.106+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:36.371823+0000 front 2024-05-09T01:03:36.371873+0000 (oldest deadline 2024-05-09T01:04:01.071919+0000) 2024-05-09T01:04:09.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:09 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:09.370+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:04:09.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:09 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:09.370+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:12.758527+0000 front 2024-05-09T01:03:12.758364+0000 (oldest deadline 2024-05-09T01:03:35.658450+0000) 2024-05-09T01:04:09.687 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:09 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:09.370+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.259808+0000 front 2024-05-09T01:03:20.259685+0000 (oldest deadline 2024-05-09T01:03:46.159864+0000) 2024-05-09T01:04:09.687 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:09 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:09.370+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:38.362243+0000 front 2024-05-09T01:03:38.362207+0000 (oldest deadline 2024-05-09T01:04:03.662309+0000) 2024-05-09T01:04:09.687 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:04:09 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:04:09.613+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.533336+0000 front 2024-05-09T01:03:04.533539+0000 (oldest deadline 2024-05-09T01:03:28.633473+0000) 2024-05-09T01:04:09.687 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:04:09 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:04:09.613+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:18.535819+0000 front 2024-05-09T01:03:18.536013+0000 (oldest deadline 2024-05-09T01:03:44.435939+0000) 2024-05-09T01:04:09.688 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:04:09 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:04:09.613+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:30.638891+0000 front 2024-05-09T01:03:30.639015+0000 (oldest deadline 2024-05-09T01:03:53.538952+0000) 2024-05-09T01:04:09.688 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:04:09 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:04:09.613+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:37.040226+0000 front 2024-05-09T01:03:37.040409+0000 (oldest deadline 2024-05-09T01:04:00.540204+0000) 2024-05-09T01:04:10.041 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:09 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:09.576+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.737970+0000 front 2024-05-09T01:03:04.738119+0000 (oldest deadline 2024-05-09T01:03:27.037934+0000) 2024-05-09T01:04:10.041 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:09 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:09.576+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.439829+0000 front 2024-05-09T01:03:13.439669+0000 (oldest deadline 2024-05-09T01:03:38.739771+0000) 2024-05-09T01:04:10.041 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:09 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:09.576+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:18.740444+0000 front 2024-05-09T01:03:18.740379+0000 (oldest deadline 2024-05-09T01:03:44.040389+0000) 2024-05-09T01:04:10.041 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:09 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:09.576+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:30.342688+0000 front 2024-05-09T01:03:30.342762+0000 (oldest deadline 2024-05-09T01:03:52.042878+0000) 2024-05-09T01:04:10.355 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:10 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:10.063+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.667724+0000 front 2024-05-09T01:03:13.667762+0000 (oldest deadline 2024-05-09T01:03:35.967853+0000) 2024-05-09T01:04:10.355 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:10 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:10.063+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.668856+0000 front 2024-05-09T01:03:20.668887+0000 (oldest deadline 2024-05-09T01:03:45.968898+0000) 2024-05-09T01:04:10.355 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:10 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:10.063+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:29.371010+0000 front 2024-05-09T01:03:29.370862+0000 (oldest deadline 2024-05-09T01:03:54.070791+0000) 2024-05-09T01:04:10.355 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:10 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:10.063+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:36.371823+0000 front 2024-05-09T01:03:36.371873+0000 (oldest deadline 2024-05-09T01:04:01.071919+0000) 2024-05-09T01:04:10.355 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:10 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:10.063+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-09T01:03:46.873806+0000 front 2024-05-09T01:03:46.873608+0000 (oldest deadline 2024-05-09T01:04:09.773530+0000) 2024-05-09T01:04:10.609 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:10 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:10.345+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:04:10.609 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:10 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:10.345+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:12.758527+0000 front 2024-05-09T01:03:12.758364+0000 (oldest deadline 2024-05-09T01:03:35.658450+0000) 2024-05-09T01:04:10.609 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:10 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:10.345+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.259808+0000 front 2024-05-09T01:03:20.259685+0000 (oldest deadline 2024-05-09T01:03:46.159864+0000) 2024-05-09T01:04:10.609 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:10 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:10.345+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:38.362243+0000 front 2024-05-09T01:03:38.362207+0000 (oldest deadline 2024-05-09T01:04:03.662309+0000) 2024-05-09T01:04:10.609 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:10 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:10.345+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-09T01:03:45.863988+0000 front 2024-05-09T01:03:45.864013+0000 (oldest deadline 2024-05-09T01:04:09.964129+0000) 2024-05-09T01:04:10.936 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:04:10 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:04:10.608+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.533336+0000 front 2024-05-09T01:03:04.533539+0000 (oldest deadline 2024-05-09T01:03:28.633473+0000) 2024-05-09T01:04:10.936 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:04:10 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:04:10.608+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:18.535819+0000 front 2024-05-09T01:03:18.536013+0000 (oldest deadline 2024-05-09T01:03:44.435939+0000) 2024-05-09T01:04:10.936 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:04:10 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:04:10.608+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:30.638891+0000 front 2024-05-09T01:03:30.639015+0000 (oldest deadline 2024-05-09T01:03:53.538952+0000) 2024-05-09T01:04:10.936 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:04:10 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:04:10.608+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:37.040226+0000 front 2024-05-09T01:03:37.040409+0000 (oldest deadline 2024-05-09T01:04:00.540204+0000) 2024-05-09T01:04:10.936 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:04:10 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:04:10.608+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-09T01:03:46.941846+0000 front 2024-05-09T01:03:46.941917+0000 (oldest deadline 2024-05-09T01:04:09.841994+0000) 2024-05-09T01:04:11.041 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:10 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:10.608+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.737970+0000 front 2024-05-09T01:03:04.738119+0000 (oldest deadline 2024-05-09T01:03:27.037934+0000) 2024-05-09T01:04:11.041 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:10 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:10.608+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.439829+0000 front 2024-05-09T01:03:13.439669+0000 (oldest deadline 2024-05-09T01:03:38.739771+0000) 2024-05-09T01:04:11.041 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:10 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:10.608+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:18.740444+0000 front 2024-05-09T01:03:18.740379+0000 (oldest deadline 2024-05-09T01:03:44.040389+0000) 2024-05-09T01:04:11.041 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:10 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:10.608+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:30.342688+0000 front 2024-05-09T01:03:30.342762+0000 (oldest deadline 2024-05-09T01:03:52.042878+0000) 2024-05-09T01:04:11.041 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:10 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:10.608+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-09T01:03:47.646441+0000 front 2024-05-09T01:03:47.646483+0000 (oldest deadline 2024-05-09T01:04:10.546424+0000) 2024-05-09T01:04:11.356 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:11 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:11.063+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.667724+0000 front 2024-05-09T01:03:13.667762+0000 (oldest deadline 2024-05-09T01:03:35.967853+0000) 2024-05-09T01:04:11.356 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:11 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:11.063+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.668856+0000 front 2024-05-09T01:03:20.668887+0000 (oldest deadline 2024-05-09T01:03:45.968898+0000) 2024-05-09T01:04:11.356 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:11 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:11.063+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:29.371010+0000 front 2024-05-09T01:03:29.370862+0000 (oldest deadline 2024-05-09T01:03:54.070791+0000) 2024-05-09T01:04:11.356 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:11 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:11.063+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:36.371823+0000 front 2024-05-09T01:03:36.371873+0000 (oldest deadline 2024-05-09T01:04:01.071919+0000) 2024-05-09T01:04:11.356 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:11 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:11.063+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-09T01:03:46.873806+0000 front 2024-05-09T01:03:46.873608+0000 (oldest deadline 2024-05-09T01:04:09.773530+0000) 2024-05-09T01:04:11.627 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:11 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:11.329+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:04:11.628 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:11 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:11.329+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:12.758527+0000 front 2024-05-09T01:03:12.758364+0000 (oldest deadline 2024-05-09T01:03:35.658450+0000) 2024-05-09T01:04:11.628 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:11 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:11.329+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.259808+0000 front 2024-05-09T01:03:20.259685+0000 (oldest deadline 2024-05-09T01:03:46.159864+0000) 2024-05-09T01:04:11.628 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:11 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:11.329+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:38.362243+0000 front 2024-05-09T01:03:38.362207+0000 (oldest deadline 2024-05-09T01:04:03.662309+0000) 2024-05-09T01:04:11.628 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:11 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:11.329+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-09T01:03:45.863988+0000 front 2024-05-09T01:03:45.864013+0000 (oldest deadline 2024-05-09T01:04:09.964129+0000) 2024-05-09T01:04:11.936 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:04:11 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:04:11.627+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.533336+0000 front 2024-05-09T01:03:04.533539+0000 (oldest deadline 2024-05-09T01:03:28.633473+0000) 2024-05-09T01:04:11.936 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:04:11 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:04:11.627+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:18.535819+0000 front 2024-05-09T01:03:18.536013+0000 (oldest deadline 2024-05-09T01:03:44.435939+0000) 2024-05-09T01:04:11.936 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:04:11 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:04:11.627+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:30.638891+0000 front 2024-05-09T01:03:30.639015+0000 (oldest deadline 2024-05-09T01:03:53.538952+0000) 2024-05-09T01:04:11.936 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:04:11 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:04:11.627+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:37.040226+0000 front 2024-05-09T01:03:37.040409+0000 (oldest deadline 2024-05-09T01:04:00.540204+0000) 2024-05-09T01:04:11.936 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:04:11 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8[52652]: 2024-05-09T01:04:11.627+0000 7f2aec2b1700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-09T01:03:46.941846+0000 front 2024-05-09T01:03:46.941917+0000 (oldest deadline 2024-05-09T01:04:09.841994+0000) 2024-05-09T01:04:12.040 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:11 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:11.570+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.737970+0000 front 2024-05-09T01:03:04.738119+0000 (oldest deadline 2024-05-09T01:03:27.037934+0000) 2024-05-09T01:04:12.040 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:11 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:11.570+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.439829+0000 front 2024-05-09T01:03:13.439669+0000 (oldest deadline 2024-05-09T01:03:38.739771+0000) 2024-05-09T01:04:12.041 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:11 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:11.570+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:18.740444+0000 front 2024-05-09T01:03:18.740379+0000 (oldest deadline 2024-05-09T01:03:44.040389+0000) 2024-05-09T01:04:12.041 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:11 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:11.570+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:30.342688+0000 front 2024-05-09T01:03:30.342762+0000 (oldest deadline 2024-05-09T01:03:52.042878+0000) 2024-05-09T01:04:12.041 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:11 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:11.570+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-09T01:03:47.646441+0000 front 2024-05-09T01:03:47.646483+0000 (oldest deadline 2024-05-09T01:04:10.546424+0000) 2024-05-09T01:04:12.355 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:12 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:12.099+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.667724+0000 front 2024-05-09T01:03:13.667762+0000 (oldest deadline 2024-05-09T01:03:35.967853+0000) 2024-05-09T01:04:12.355 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:12 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:12.099+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.668856+0000 front 2024-05-09T01:03:20.668887+0000 (oldest deadline 2024-05-09T01:03:45.968898+0000) 2024-05-09T01:04:12.355 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:12 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:12.099+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:29.371010+0000 front 2024-05-09T01:03:29.370862+0000 (oldest deadline 2024-05-09T01:03:54.070791+0000) 2024-05-09T01:04:12.356 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:12 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:12.099+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:36.371823+0000 front 2024-05-09T01:03:36.371873+0000 (oldest deadline 2024-05-09T01:04:01.071919+0000) 2024-05-09T01:04:12.356 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:12 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:12.099+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-09T01:03:46.873806+0000 front 2024-05-09T01:03:46.873608+0000 (oldest deadline 2024-05-09T01:04:09.773530+0000) 2024-05-09T01:04:12.392 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:04:12 smithi157 podman[71353]: 2024-05-09 01:04:12.103576601 +0000 UTC m=+5.502384417 container died 049b0aa67a4606a019c31b3053bfd8d5fb0430f12674674d81fb357513ff257a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux ) 2024-05-09T01:04:12.392 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:12 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:12.281+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:04:12.392 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:12 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:12.281+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:12.758527+0000 front 2024-05-09T01:03:12.758364+0000 (oldest deadline 2024-05-09T01:03:35.658450+0000) 2024-05-09T01:04:12.392 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:12 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:12.281+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.259808+0000 front 2024-05-09T01:03:20.259685+0000 (oldest deadline 2024-05-09T01:03:46.159864+0000) 2024-05-09T01:04:12.392 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:12 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:12.281+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:38.362243+0000 front 2024-05-09T01:03:38.362207+0000 (oldest deadline 2024-05-09T01:04:03.662309+0000) 2024-05-09T01:04:12.393 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:12 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:12.281+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-09T01:03:45.863988+0000 front 2024-05-09T01:03:45.864013+0000 (oldest deadline 2024-05-09T01:04:09.964129+0000) 2024-05-09T01:04:12.686 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:04:12 smithi157 podman[71353]: 2024-05-09 01:04:12.391712383 +0000 UTC m=+5.790520200 container cleanup 049b0aa67a4606a019c31b3053bfd8d5fb0430f12674674d81fb357513ff257a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS) 2024-05-09T01:04:12.686 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:04:12 smithi157 bash[71353]: ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8 2024-05-09T01:04:12.686 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:04:12 smithi157 podman[71382]: 2024-05-09 01:04:12.601837298 +0000 UTC m=+0.496692545 container remove 049b0aa67a4606a019c31b3053bfd8d5fb0430f12674674d81fb357513ff257a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-09T01:04:13.041 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:12 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:12.617+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.737970+0000 front 2024-05-09T01:03:04.738119+0000 (oldest deadline 2024-05-09T01:03:27.037934+0000) 2024-05-09T01:04:13.041 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:12 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:12.617+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.439829+0000 front 2024-05-09T01:03:13.439669+0000 (oldest deadline 2024-05-09T01:03:38.739771+0000) 2024-05-09T01:04:13.041 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:12 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:12.617+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:18.740444+0000 front 2024-05-09T01:03:18.740379+0000 (oldest deadline 2024-05-09T01:03:44.040389+0000) 2024-05-09T01:04:13.041 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:12 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:12.617+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:30.342688+0000 front 2024-05-09T01:03:30.342762+0000 (oldest deadline 2024-05-09T01:03:52.042878+0000) 2024-05-09T01:04:13.041 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:12 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:12.617+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-09T01:03:47.646441+0000 front 2024-05-09T01:03:47.646483+0000 (oldest deadline 2024-05-09T01:04:10.546424+0000) 2024-05-09T01:04:13.263 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:04:13 smithi157 podman[71474]: 2024-05-09 01:04:12.914326263 +0000 UTC m=+0.018060327 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T01:04:13.263 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:04:13 smithi157 podman[71474]: 2024-05-09 01:04:13.027431707 +0000 UTC m=+0.131165766 container create 1117188b5087fbc900a9fc007a56b64589d9b9a685714d73e87ad75db0a9117b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240507, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T01:04:13.264 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:04:13 smithi157 podman[71474]: 2024-05-09 01:04:13.263283346 +0000 UTC m=+0.367017414 container init 1117188b5087fbc900a9fc007a56b64589d9b9a685714d73e87ad75db0a9117b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-05-09T01:04:13.551 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:04:13 smithi157 podman[71474]: 2024-05-09 01:04:13.266670376 +0000 UTC m=+0.370404444 container start 1117188b5087fbc900a9fc007a56b64589d9b9a685714d73e87ad75db0a9117b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS) 2024-05-09T01:04:13.551 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:04:13 smithi157 podman[71474]: 2024-05-09 01:04:13.338115376 +0000 UTC m=+0.441849444 container attach 1117188b5087fbc900a9fc007a56b64589d9b9a685714d73e87ad75db0a9117b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-05-09T01:04:13.551 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:13 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:13.291+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:04:13.551 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:13 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:13.291+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:12.758527+0000 front 2024-05-09T01:03:12.758364+0000 (oldest deadline 2024-05-09T01:03:35.658450+0000) 2024-05-09T01:04:13.551 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:13 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:13.291+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.259808+0000 front 2024-05-09T01:03:20.259685+0000 (oldest deadline 2024-05-09T01:03:46.159864+0000) 2024-05-09T01:04:13.552 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:13 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:13.291+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:38.362243+0000 front 2024-05-09T01:03:38.362207+0000 (oldest deadline 2024-05-09T01:04:03.662309+0000) 2024-05-09T01:04:13.552 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:13 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:13.291+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-09T01:03:45.863988+0000 front 2024-05-09T01:03:45.864013+0000 (oldest deadline 2024-05-09T01:04:09.964129+0000) 2024-05-09T01:04:13.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:13 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:13.133+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.667724+0000 front 2024-05-09T01:03:13.667762+0000 (oldest deadline 2024-05-09T01:03:35.967853+0000) 2024-05-09T01:04:13.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:13 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:13.133+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.668856+0000 front 2024-05-09T01:03:20.668887+0000 (oldest deadline 2024-05-09T01:03:45.968898+0000) 2024-05-09T01:04:13.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:13 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:13.133+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:29.371010+0000 front 2024-05-09T01:03:29.370862+0000 (oldest deadline 2024-05-09T01:03:54.070791+0000) 2024-05-09T01:04:13.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:13 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:13.133+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:36.371823+0000 front 2024-05-09T01:03:36.371873+0000 (oldest deadline 2024-05-09T01:04:01.071919+0000) 2024-05-09T01:04:13.606 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:13 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:13.133+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-09T01:03:46.873806+0000 front 2024-05-09T01:03:46.873608+0000 (oldest deadline 2024-05-09T01:04:09.773530+0000) 2024-05-09T01:04:13.849 INFO:journalctl@ceph.osd.8.smithi157.stdout:May 09 01:04:13 smithi157 podman[71474]: 2024-05-09 01:04:13.550963081 +0000 UTC m=+0.654697143 container died 1117188b5087fbc900a9fc007a56b64589d9b9a685714d73e87ad75db0a9117b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-09T01:04:14.040 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:13 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:13.573+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.737970+0000 front 2024-05-09T01:03:04.738119+0000 (oldest deadline 2024-05-09T01:03:27.037934+0000) 2024-05-09T01:04:14.041 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:13 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:13.573+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.439829+0000 front 2024-05-09T01:03:13.439669+0000 (oldest deadline 2024-05-09T01:03:38.739771+0000) 2024-05-09T01:04:14.041 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:13 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:13.573+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:18.740444+0000 front 2024-05-09T01:03:18.740379+0000 (oldest deadline 2024-05-09T01:03:44.040389+0000) 2024-05-09T01:04:14.041 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:13 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:13.573+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:30.342688+0000 front 2024-05-09T01:03:30.342762+0000 (oldest deadline 2024-05-09T01:03:52.042878+0000) 2024-05-09T01:04:14.041 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:13 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:13.573+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-09T01:03:47.646441+0000 front 2024-05-09T01:03:47.646483+0000 (oldest deadline 2024-05-09T01:04:10.546424+0000) 2024-05-09T01:04:14.190 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@osd.8.service' 2024-05-09T01:04:14.229 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T01:04:14.230 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-05-09T01:04:14.230 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-05-09T01:04:14.230 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@osd.9 2024-05-09T01:04:14.436 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:14 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:14.314+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:04:14.436 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:14 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:14.314+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:12.758527+0000 front 2024-05-09T01:03:12.758364+0000 (oldest deadline 2024-05-09T01:03:35.658450+0000) 2024-05-09T01:04:14.436 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:14 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:14.314+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.259808+0000 front 2024-05-09T01:03:20.259685+0000 (oldest deadline 2024-05-09T01:03:46.159864+0000) 2024-05-09T01:04:14.436 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:14 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:14.314+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:38.362243+0000 front 2024-05-09T01:03:38.362207+0000 (oldest deadline 2024-05-09T01:04:03.662309+0000) 2024-05-09T01:04:14.436 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:14 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:14.314+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-09T01:03:45.863988+0000 front 2024-05-09T01:03:45.864013+0000 (oldest deadline 2024-05-09T01:04:09.964129+0000) 2024-05-09T01:04:14.540 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:14 smithi005 systemd[1]: Stopping Ceph osd.9 for d85814c6-0d9b-11ef-bc97-c7b262605968... 2024-05-09T01:04:14.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:14 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:14.167+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.667724+0000 front 2024-05-09T01:03:13.667762+0000 (oldest deadline 2024-05-09T01:03:35.967853+0000) 2024-05-09T01:04:14.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:14 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:14.167+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.668856+0000 front 2024-05-09T01:03:20.668887+0000 (oldest deadline 2024-05-09T01:03:45.968898+0000) 2024-05-09T01:04:14.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:14 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:14.167+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:29.371010+0000 front 2024-05-09T01:03:29.370862+0000 (oldest deadline 2024-05-09T01:03:54.070791+0000) 2024-05-09T01:04:14.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:14 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:14.167+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:36.371823+0000 front 2024-05-09T01:03:36.371873+0000 (oldest deadline 2024-05-09T01:04:01.071919+0000) 2024-05-09T01:04:14.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:14 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:14.167+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-09T01:03:46.873806+0000 front 2024-05-09T01:03:46.873608+0000 (oldest deadline 2024-05-09T01:04:09.773530+0000) 2024-05-09T01:04:15.040 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:14 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:14.557+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.737970+0000 front 2024-05-09T01:03:04.738119+0000 (oldest deadline 2024-05-09T01:03:27.037934+0000) 2024-05-09T01:04:15.041 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:14 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:14.557+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.439829+0000 front 2024-05-09T01:03:13.439669+0000 (oldest deadline 2024-05-09T01:03:38.739771+0000) 2024-05-09T01:04:15.041 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:14 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:14.557+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:18.740444+0000 front 2024-05-09T01:03:18.740379+0000 (oldest deadline 2024-05-09T01:03:44.040389+0000) 2024-05-09T01:04:15.041 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:14 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:14.557+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:30.342688+0000 front 2024-05-09T01:03:30.342762+0000 (oldest deadline 2024-05-09T01:03:52.042878+0000) 2024-05-09T01:04:15.041 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:14 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:14.557+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-09T01:03:47.646441+0000 front 2024-05-09T01:03:47.646483+0000 (oldest deadline 2024-05-09T01:04:10.546424+0000) 2024-05-09T01:04:15.041 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:14 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:14.576+0000 7f73538e7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T01:04:15.041 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:14 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:14.576+0000 7f73538e7700 -1 osd.9 85 *** Got signal Terminated *** 2024-05-09T01:04:15.041 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:14 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:14.576+0000 7f73538e7700 -1 osd.9 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T01:04:15.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:15 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:15.187+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.667724+0000 front 2024-05-09T01:03:13.667762+0000 (oldest deadline 2024-05-09T01:03:35.967853+0000) 2024-05-09T01:04:15.606 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:15 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:15.187+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.668856+0000 front 2024-05-09T01:03:20.668887+0000 (oldest deadline 2024-05-09T01:03:45.968898+0000) 2024-05-09T01:04:15.606 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:15 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:15.187+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:29.371010+0000 front 2024-05-09T01:03:29.370862+0000 (oldest deadline 2024-05-09T01:03:54.070791+0000) 2024-05-09T01:04:15.606 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:15 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:15.187+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:36.371823+0000 front 2024-05-09T01:03:36.371873+0000 (oldest deadline 2024-05-09T01:04:01.071919+0000) 2024-05-09T01:04:15.606 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:15 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:15.187+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-09T01:03:46.873806+0000 front 2024-05-09T01:03:46.873608+0000 (oldest deadline 2024-05-09T01:04:09.773530+0000) 2024-05-09T01:04:15.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:15 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:15.339+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:04:15.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:15 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:15.339+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:12.758527+0000 front 2024-05-09T01:03:12.758364+0000 (oldest deadline 2024-05-09T01:03:35.658450+0000) 2024-05-09T01:04:15.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:15 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:15.339+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.259808+0000 front 2024-05-09T01:03:20.259685+0000 (oldest deadline 2024-05-09T01:03:46.159864+0000) 2024-05-09T01:04:15.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:15 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:15.339+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:38.362243+0000 front 2024-05-09T01:03:38.362207+0000 (oldest deadline 2024-05-09T01:04:03.662309+0000) 2024-05-09T01:04:15.687 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:15 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:15.339+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-09T01:03:45.863988+0000 front 2024-05-09T01:03:45.864013+0000 (oldest deadline 2024-05-09T01:04:09.964129+0000) 2024-05-09T01:04:16.041 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:15 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:15.579+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.737970+0000 front 2024-05-09T01:03:04.738119+0000 (oldest deadline 2024-05-09T01:03:27.037934+0000) 2024-05-09T01:04:16.041 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:15 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:15.579+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.439829+0000 front 2024-05-09T01:03:13.439669+0000 (oldest deadline 2024-05-09T01:03:38.739771+0000) 2024-05-09T01:04:16.041 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:15 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:15.579+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:18.740444+0000 front 2024-05-09T01:03:18.740379+0000 (oldest deadline 2024-05-09T01:03:44.040389+0000) 2024-05-09T01:04:16.041 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:15 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:15.579+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:30.342688+0000 front 2024-05-09T01:03:30.342762+0000 (oldest deadline 2024-05-09T01:03:52.042878+0000) 2024-05-09T01:04:16.041 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:15 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:15.579+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-09T01:03:47.646441+0000 front 2024-05-09T01:03:47.646483+0000 (oldest deadline 2024-05-09T01:04:10.546424+0000) 2024-05-09T01:04:16.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:16 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:16.188+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.667724+0000 front 2024-05-09T01:03:13.667762+0000 (oldest deadline 2024-05-09T01:03:35.967853+0000) 2024-05-09T01:04:16.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:16 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:16.188+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.668856+0000 front 2024-05-09T01:03:20.668887+0000 (oldest deadline 2024-05-09T01:03:45.968898+0000) 2024-05-09T01:04:16.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:16 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:16.188+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:29.371010+0000 front 2024-05-09T01:03:29.370862+0000 (oldest deadline 2024-05-09T01:03:54.070791+0000) 2024-05-09T01:04:16.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:16 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:16.188+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:36.371823+0000 front 2024-05-09T01:03:36.371873+0000 (oldest deadline 2024-05-09T01:04:01.071919+0000) 2024-05-09T01:04:16.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:16 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:16.188+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-09T01:03:46.873806+0000 front 2024-05-09T01:03:46.873608+0000 (oldest deadline 2024-05-09T01:04:09.773530+0000) 2024-05-09T01:04:16.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:16 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:16.326+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:04:16.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:16 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:16.326+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:12.758527+0000 front 2024-05-09T01:03:12.758364+0000 (oldest deadline 2024-05-09T01:03:35.658450+0000) 2024-05-09T01:04:16.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:16 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:16.326+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.259808+0000 front 2024-05-09T01:03:20.259685+0000 (oldest deadline 2024-05-09T01:03:46.159864+0000) 2024-05-09T01:04:16.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:16 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:16.326+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:38.362243+0000 front 2024-05-09T01:03:38.362207+0000 (oldest deadline 2024-05-09T01:04:03.662309+0000) 2024-05-09T01:04:16.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:16 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:16.326+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-09T01:03:45.863988+0000 front 2024-05-09T01:03:45.864013+0000 (oldest deadline 2024-05-09T01:04:09.964129+0000) 2024-05-09T01:04:17.040 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:16 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:16.628+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.737970+0000 front 2024-05-09T01:03:04.738119+0000 (oldest deadline 2024-05-09T01:03:27.037934+0000) 2024-05-09T01:04:17.041 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:16 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:16.628+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.439829+0000 front 2024-05-09T01:03:13.439669+0000 (oldest deadline 2024-05-09T01:03:38.739771+0000) 2024-05-09T01:04:17.041 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:16 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:16.628+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:18.740444+0000 front 2024-05-09T01:03:18.740379+0000 (oldest deadline 2024-05-09T01:03:44.040389+0000) 2024-05-09T01:04:17.041 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:16 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:16.628+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:30.342688+0000 front 2024-05-09T01:03:30.342762+0000 (oldest deadline 2024-05-09T01:03:52.042878+0000) 2024-05-09T01:04:17.041 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:16 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:16.628+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-09T01:03:47.646441+0000 front 2024-05-09T01:03:47.646483+0000 (oldest deadline 2024-05-09T01:04:10.546424+0000) 2024-05-09T01:04:17.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:17 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:17.222+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.667724+0000 front 2024-05-09T01:03:13.667762+0000 (oldest deadline 2024-05-09T01:03:35.967853+0000) 2024-05-09T01:04:17.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:17 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:17.222+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.668856+0000 front 2024-05-09T01:03:20.668887+0000 (oldest deadline 2024-05-09T01:03:45.968898+0000) 2024-05-09T01:04:17.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:17 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:17.222+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:29.371010+0000 front 2024-05-09T01:03:29.370862+0000 (oldest deadline 2024-05-09T01:03:54.070791+0000) 2024-05-09T01:04:17.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:17 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:17.222+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:36.371823+0000 front 2024-05-09T01:03:36.371873+0000 (oldest deadline 2024-05-09T01:04:01.071919+0000) 2024-05-09T01:04:17.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:17 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:17.222+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-09T01:03:46.873806+0000 front 2024-05-09T01:03:46.873608+0000 (oldest deadline 2024-05-09T01:04:09.773530+0000) 2024-05-09T01:04:17.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:17 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:17.341+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:04:17.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:17 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:17.341+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:12.758527+0000 front 2024-05-09T01:03:12.758364+0000 (oldest deadline 2024-05-09T01:03:35.658450+0000) 2024-05-09T01:04:17.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:17 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:17.341+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.259808+0000 front 2024-05-09T01:03:20.259685+0000 (oldest deadline 2024-05-09T01:03:46.159864+0000) 2024-05-09T01:04:17.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:17 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:17.341+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:38.362243+0000 front 2024-05-09T01:03:38.362207+0000 (oldest deadline 2024-05-09T01:04:03.662309+0000) 2024-05-09T01:04:17.687 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:17 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:17.341+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-09T01:03:45.863988+0000 front 2024-05-09T01:03:45.864013+0000 (oldest deadline 2024-05-09T01:04:09.964129+0000) 2024-05-09T01:04:17.998 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:17 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:17.589+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.737970+0000 front 2024-05-09T01:03:04.738119+0000 (oldest deadline 2024-05-09T01:03:27.037934+0000) 2024-05-09T01:04:17.998 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:17 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:17.589+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.439829+0000 front 2024-05-09T01:03:13.439669+0000 (oldest deadline 2024-05-09T01:03:38.739771+0000) 2024-05-09T01:04:17.998 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:17 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:17.589+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:18.740444+0000 front 2024-05-09T01:03:18.740379+0000 (oldest deadline 2024-05-09T01:03:44.040389+0000) 2024-05-09T01:04:17.998 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:17 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:17.589+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:30.342688+0000 front 2024-05-09T01:03:30.342762+0000 (oldest deadline 2024-05-09T01:03:52.042878+0000) 2024-05-09T01:04:17.998 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:17 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:17.589+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-09T01:03:47.646441+0000 front 2024-05-09T01:03:47.646483+0000 (oldest deadline 2024-05-09T01:04:10.546424+0000) 2024-05-09T01:04:17.998 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:17 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:17.589+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-05-09T01:03:51.647300+0000 front 2024-05-09T01:03:51.647357+0000 (oldest deadline 2024-05-09T01:04:17.547295+0000) 2024-05-09T01:04:18.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:18 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:18.200+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.667724+0000 front 2024-05-09T01:03:13.667762+0000 (oldest deadline 2024-05-09T01:03:35.967853+0000) 2024-05-09T01:04:18.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:18 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:18.200+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.668856+0000 front 2024-05-09T01:03:20.668887+0000 (oldest deadline 2024-05-09T01:03:45.968898+0000) 2024-05-09T01:04:18.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:18 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:18.200+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:29.371010+0000 front 2024-05-09T01:03:29.370862+0000 (oldest deadline 2024-05-09T01:03:54.070791+0000) 2024-05-09T01:04:18.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:18 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:18.200+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:36.371823+0000 front 2024-05-09T01:03:36.371873+0000 (oldest deadline 2024-05-09T01:04:01.071919+0000) 2024-05-09T01:04:18.606 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:18 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:18.200+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-09T01:03:46.873806+0000 front 2024-05-09T01:03:46.873608+0000 (oldest deadline 2024-05-09T01:04:09.773530+0000) 2024-05-09T01:04:18.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:18 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:18.364+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:04:18.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:18 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:18.364+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:12.758527+0000 front 2024-05-09T01:03:12.758364+0000 (oldest deadline 2024-05-09T01:03:35.658450+0000) 2024-05-09T01:04:18.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:18 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:18.364+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.259808+0000 front 2024-05-09T01:03:20.259685+0000 (oldest deadline 2024-05-09T01:03:46.159864+0000) 2024-05-09T01:04:18.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:18 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:18.364+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:38.362243+0000 front 2024-05-09T01:03:38.362207+0000 (oldest deadline 2024-05-09T01:04:03.662309+0000) 2024-05-09T01:04:18.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:18 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:18.364+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-09T01:03:45.863988+0000 front 2024-05-09T01:03:45.864013+0000 (oldest deadline 2024-05-09T01:04:09.964129+0000) 2024-05-09T01:04:18.687 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:18 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:18.364+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-05-09T01:03:54.665305+0000 front 2024-05-09T01:03:54.665343+0000 (oldest deadline 2024-05-09T01:04:17.565166+0000) 2024-05-09T01:04:19.040 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:18 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:18.555+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.737970+0000 front 2024-05-09T01:03:04.738119+0000 (oldest deadline 2024-05-09T01:03:27.037934+0000) 2024-05-09T01:04:19.041 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:18 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:18.555+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.439829+0000 front 2024-05-09T01:03:13.439669+0000 (oldest deadline 2024-05-09T01:03:38.739771+0000) 2024-05-09T01:04:19.041 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:18 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:18.555+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:18.740444+0000 front 2024-05-09T01:03:18.740379+0000 (oldest deadline 2024-05-09T01:03:44.040389+0000) 2024-05-09T01:04:19.041 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:18 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:18.555+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:30.342688+0000 front 2024-05-09T01:03:30.342762+0000 (oldest deadline 2024-05-09T01:03:52.042878+0000) 2024-05-09T01:04:19.041 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:18 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:18.555+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-09T01:03:47.646441+0000 front 2024-05-09T01:03:47.646483+0000 (oldest deadline 2024-05-09T01:04:10.546424+0000) 2024-05-09T01:04:19.041 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:18 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:18.555+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-05-09T01:03:51.647300+0000 front 2024-05-09T01:03:51.647357+0000 (oldest deadline 2024-05-09T01:04:17.547295+0000) 2024-05-09T01:04:19.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:19 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:19.219+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.667724+0000 front 2024-05-09T01:03:13.667762+0000 (oldest deadline 2024-05-09T01:03:35.967853+0000) 2024-05-09T01:04:19.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:19 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:19.219+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.668856+0000 front 2024-05-09T01:03:20.668887+0000 (oldest deadline 2024-05-09T01:03:45.968898+0000) 2024-05-09T01:04:19.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:19 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:19.219+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:29.371010+0000 front 2024-05-09T01:03:29.370862+0000 (oldest deadline 2024-05-09T01:03:54.070791+0000) 2024-05-09T01:04:19.606 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:19 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:19.219+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:36.371823+0000 front 2024-05-09T01:03:36.371873+0000 (oldest deadline 2024-05-09T01:04:01.071919+0000) 2024-05-09T01:04:19.606 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:19 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:19.219+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-09T01:03:46.873806+0000 front 2024-05-09T01:03:46.873608+0000 (oldest deadline 2024-05-09T01:04:09.773530+0000) 2024-05-09T01:04:19.606 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:19 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:19.219+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-05-09T01:03:53.774919+0000 front 2024-05-09T01:03:53.774963+0000 (oldest deadline 2024-05-09T01:04:19.074815+0000) 2024-05-09T01:04:19.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:19 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:19.334+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:04:19.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:19 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:19.334+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:12.758527+0000 front 2024-05-09T01:03:12.758364+0000 (oldest deadline 2024-05-09T01:03:35.658450+0000) 2024-05-09T01:04:19.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:19 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:19.334+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.259808+0000 front 2024-05-09T01:03:20.259685+0000 (oldest deadline 2024-05-09T01:03:46.159864+0000) 2024-05-09T01:04:19.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:19 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:19.334+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:38.362243+0000 front 2024-05-09T01:03:38.362207+0000 (oldest deadline 2024-05-09T01:04:03.662309+0000) 2024-05-09T01:04:19.687 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:19 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:19.334+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-09T01:03:45.863988+0000 front 2024-05-09T01:03:45.864013+0000 (oldest deadline 2024-05-09T01:04:09.964129+0000) 2024-05-09T01:04:19.687 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:19 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:19.334+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-05-09T01:03:54.665305+0000 front 2024-05-09T01:03:54.665343+0000 (oldest deadline 2024-05-09T01:04:17.565166+0000) 2024-05-09T01:04:19.790 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:19 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:19.530+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:04.737970+0000 front 2024-05-09T01:03:04.738119+0000 (oldest deadline 2024-05-09T01:03:27.037934+0000) 2024-05-09T01:04:19.791 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:19 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:19.530+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.439829+0000 front 2024-05-09T01:03:13.439669+0000 (oldest deadline 2024-05-09T01:03:38.739771+0000) 2024-05-09T01:04:19.791 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:19 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:19.530+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:18.740444+0000 front 2024-05-09T01:03:18.740379+0000 (oldest deadline 2024-05-09T01:03:44.040389+0000) 2024-05-09T01:04:19.791 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:19 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:19.530+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:30.342688+0000 front 2024-05-09T01:03:30.342762+0000 (oldest deadline 2024-05-09T01:03:52.042878+0000) 2024-05-09T01:04:19.791 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:19 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:19.530+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-09T01:03:47.646441+0000 front 2024-05-09T01:03:47.646483+0000 (oldest deadline 2024-05-09T01:04:10.546424+0000) 2024-05-09T01:04:19.791 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:19 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9[65728]: 2024-05-09T01:04:19.530+0000 7f734fe9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-05-09T01:03:51.647300+0000 front 2024-05-09T01:03:51.647357+0000 (oldest deadline 2024-05-09T01:04:17.547295+0000) 2024-05-09T01:04:20.319 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:20 smithi005 podman[112424]: 2024-05-09 01:04:20.054917686 +0000 UTC m=+5.619469584 container died ec5d178f6a124d2ea955d18e62bc59ea4c46e7ad236f4c4afe0efe5ac7381590 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-09T01:04:20.560 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:20 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:20.214+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.667724+0000 front 2024-05-09T01:03:13.667762+0000 (oldest deadline 2024-05-09T01:03:35.967853+0000) 2024-05-09T01:04:20.560 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:20 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:20.214+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.668856+0000 front 2024-05-09T01:03:20.668887+0000 (oldest deadline 2024-05-09T01:03:45.968898+0000) 2024-05-09T01:04:20.560 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:20 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:20.214+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:29.371010+0000 front 2024-05-09T01:03:29.370862+0000 (oldest deadline 2024-05-09T01:03:54.070791+0000) 2024-05-09T01:04:20.560 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:20 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:20.214+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:36.371823+0000 front 2024-05-09T01:03:36.371873+0000 (oldest deadline 2024-05-09T01:04:01.071919+0000) 2024-05-09T01:04:20.560 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:20 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:20.214+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-09T01:03:46.873806+0000 front 2024-05-09T01:03:46.873608+0000 (oldest deadline 2024-05-09T01:04:09.773530+0000) 2024-05-09T01:04:20.560 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:20 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:20.214+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-05-09T01:03:53.774919+0000 front 2024-05-09T01:03:53.774963+0000 (oldest deadline 2024-05-09T01:04:19.074815+0000) 2024-05-09T01:04:20.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:20 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:20.331+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:04:20.687 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:20 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:20.331+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:12.758527+0000 front 2024-05-09T01:03:12.758364+0000 (oldest deadline 2024-05-09T01:03:35.658450+0000) 2024-05-09T01:04:20.687 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:20 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:20.331+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.259808+0000 front 2024-05-09T01:03:20.259685+0000 (oldest deadline 2024-05-09T01:03:46.159864+0000) 2024-05-09T01:04:20.687 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:20 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:20.331+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:38.362243+0000 front 2024-05-09T01:03:38.362207+0000 (oldest deadline 2024-05-09T01:04:03.662309+0000) 2024-05-09T01:04:20.687 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:20 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:20.331+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-09T01:03:45.863988+0000 front 2024-05-09T01:03:45.864013+0000 (oldest deadline 2024-05-09T01:04:09.964129+0000) 2024-05-09T01:04:20.687 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:20 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:20.331+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-05-09T01:03:54.665305+0000 front 2024-05-09T01:03:54.665343+0000 (oldest deadline 2024-05-09T01:04:17.565166+0000) 2024-05-09T01:04:20.791 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:20 smithi005 podman[112424]: 2024-05-09 01:04:20.507550026 +0000 UTC m=+6.072101918 container cleanup ec5d178f6a124d2ea955d18e62bc59ea4c46e7ad236f4c4afe0efe5ac7381590 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T01:04:20.791 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:20 smithi005 bash[112424]: ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9 2024-05-09T01:04:21.197 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:20 smithi005 podman[112441]: 2024-05-09 01:04:20.825428631 +0000 UTC m=+0.767236997 container remove ec5d178f6a124d2ea955d18e62bc59ea4c46e7ad236f4c4afe0efe5ac7381590 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0) 2024-05-09T01:04:21.487 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:21 smithi005 podman[112527]: 2024-05-09 01:04:21.099622577 +0000 UTC m=+0.018429880 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T01:04:21.487 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:21 smithi005 podman[112527]: 2024-05-09 01:04:21.201076935 +0000 UTC m=+0.119884237 container create 1e7811885cf51794f3215146f0d80935a46dafb0888ece24e0e469a7a0fafbd6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-05-09T01:04:21.487 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:21 smithi005 podman[112527]: 2024-05-09 01:04:21.386897281 +0000 UTC m=+0.305704591 container init 1e7811885cf51794f3215146f0d80935a46dafb0888ece24e0e469a7a0fafbd6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T01:04:21.487 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:21 smithi005 podman[112527]: 2024-05-09 01:04:21.39020472 +0000 UTC m=+0.309012025 container start 1e7811885cf51794f3215146f0d80935a46dafb0888ece24e0e469a7a0fafbd6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2) 2024-05-09T01:04:21.487 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:21 smithi005 podman[112527]: 2024-05-09 01:04:21.486859712 +0000 UTC m=+0.405667023 container attach 1e7811885cf51794f3215146f0d80935a46dafb0888ece24e0e469a7a0fafbd6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T01:04:21.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:21 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:21.197+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.667724+0000 front 2024-05-09T01:03:13.667762+0000 (oldest deadline 2024-05-09T01:03:35.967853+0000) 2024-05-09T01:04:21.606 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:21 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:21.197+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.668856+0000 front 2024-05-09T01:03:20.668887+0000 (oldest deadline 2024-05-09T01:03:45.968898+0000) 2024-05-09T01:04:21.606 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:21 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:21.197+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:29.371010+0000 front 2024-05-09T01:03:29.370862+0000 (oldest deadline 2024-05-09T01:03:54.070791+0000) 2024-05-09T01:04:21.606 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:21 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:21.197+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:36.371823+0000 front 2024-05-09T01:03:36.371873+0000 (oldest deadline 2024-05-09T01:04:01.071919+0000) 2024-05-09T01:04:21.606 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:21 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:21.197+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-09T01:03:46.873806+0000 front 2024-05-09T01:03:46.873608+0000 (oldest deadline 2024-05-09T01:04:09.773530+0000) 2024-05-09T01:04:21.606 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:21 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:21.197+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-05-09T01:03:53.774919+0000 front 2024-05-09T01:03:53.774963+0000 (oldest deadline 2024-05-09T01:04:19.074815+0000) 2024-05-09T01:04:21.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:21 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:21.345+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:04:21.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:21 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:21.345+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:12.758527+0000 front 2024-05-09T01:03:12.758364+0000 (oldest deadline 2024-05-09T01:03:35.658450+0000) 2024-05-09T01:04:21.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:21 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:21.345+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.259808+0000 front 2024-05-09T01:03:20.259685+0000 (oldest deadline 2024-05-09T01:03:46.159864+0000) 2024-05-09T01:04:21.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:21 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:21.346+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:38.362243+0000 front 2024-05-09T01:03:38.362207+0000 (oldest deadline 2024-05-09T01:04:03.662309+0000) 2024-05-09T01:04:21.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:21 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:21.346+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-09T01:03:45.863988+0000 front 2024-05-09T01:03:45.864013+0000 (oldest deadline 2024-05-09T01:04:09.964129+0000) 2024-05-09T01:04:21.687 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:21 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:21.346+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-05-09T01:03:54.665305+0000 front 2024-05-09T01:03:54.665343+0000 (oldest deadline 2024-05-09T01:04:17.565166+0000) 2024-05-09T01:04:21.791 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:21 smithi005 podman[112527]: 2024-05-09 01:04:21.699736518 +0000 UTC m=+0.618543813 container died 1e7811885cf51794f3215146f0d80935a46dafb0888ece24e0e469a7a0fafbd6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-05-09T01:04:22.233 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 09 01:04:22 smithi005 podman[112553]: 2024-05-09 01:04:22.215403596 +0000 UTC m=+0.509878149 container remove 1e7811885cf51794f3215146f0d80935a46dafb0888ece24e0e469a7a0fafbd6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240507, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-05-09T01:04:22.238 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@osd.9.service' 2024-05-09T01:04:22.280 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T01:04:22.281 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-05-09T01:04:22.281 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-05-09T01:04:22.281 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@osd.10 2024-05-09T01:04:22.588 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:22 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:22.175+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.667724+0000 front 2024-05-09T01:03:13.667762+0000 (oldest deadline 2024-05-09T01:03:35.967853+0000) 2024-05-09T01:04:22.588 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:22 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:22.175+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.668856+0000 front 2024-05-09T01:03:20.668887+0000 (oldest deadline 2024-05-09T01:03:45.968898+0000) 2024-05-09T01:04:22.588 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:22 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:22.175+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:29.371010+0000 front 2024-05-09T01:03:29.370862+0000 (oldest deadline 2024-05-09T01:03:54.070791+0000) 2024-05-09T01:04:22.588 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:22 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:22.175+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:36.371823+0000 front 2024-05-09T01:03:36.371873+0000 (oldest deadline 2024-05-09T01:04:01.071919+0000) 2024-05-09T01:04:22.588 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:22 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:22.175+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-09T01:03:46.873806+0000 front 2024-05-09T01:03:46.873608+0000 (oldest deadline 2024-05-09T01:04:09.773530+0000) 2024-05-09T01:04:22.588 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:22 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:22.175+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-05-09T01:03:53.774919+0000 front 2024-05-09T01:03:53.774963+0000 (oldest deadline 2024-05-09T01:04:19.074815+0000) 2024-05-09T01:04:22.588 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:22 smithi131 systemd[1]: Stopping Ceph osd.10 for d85814c6-0d9b-11ef-bc97-c7b262605968... 2024-05-09T01:04:22.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:22 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:22.357+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:04:22.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:22 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:22.357+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:12.758527+0000 front 2024-05-09T01:03:12.758364+0000 (oldest deadline 2024-05-09T01:03:35.658450+0000) 2024-05-09T01:04:22.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:22 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:22.357+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.259808+0000 front 2024-05-09T01:03:20.259685+0000 (oldest deadline 2024-05-09T01:03:46.159864+0000) 2024-05-09T01:04:22.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:22 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:22.357+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:38.362243+0000 front 2024-05-09T01:03:38.362207+0000 (oldest deadline 2024-05-09T01:04:03.662309+0000) 2024-05-09T01:04:22.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:22 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:22.357+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-09T01:03:45.863988+0000 front 2024-05-09T01:03:45.864013+0000 (oldest deadline 2024-05-09T01:04:09.964129+0000) 2024-05-09T01:04:22.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:22 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:22.357+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-05-09T01:03:54.665305+0000 front 2024-05-09T01:03:54.665343+0000 (oldest deadline 2024-05-09T01:04:17.565166+0000) 2024-05-09T01:04:22.855 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:22 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:22.586+0000 7f102af30700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T01:04:22.855 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:22 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:22.586+0000 7f102af30700 -1 osd.10 85 *** Got signal Terminated *** 2024-05-09T01:04:22.855 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:22 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:22.586+0000 7f102af30700 -1 osd.10 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T01:04:23.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:23 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:23.209+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.667724+0000 front 2024-05-09T01:03:13.667762+0000 (oldest deadline 2024-05-09T01:03:35.967853+0000) 2024-05-09T01:04:23.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:23 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:23.209+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.668856+0000 front 2024-05-09T01:03:20.668887+0000 (oldest deadline 2024-05-09T01:03:45.968898+0000) 2024-05-09T01:04:23.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:23 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:23.209+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:29.371010+0000 front 2024-05-09T01:03:29.370862+0000 (oldest deadline 2024-05-09T01:03:54.070791+0000) 2024-05-09T01:04:23.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:23 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:23.209+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:36.371823+0000 front 2024-05-09T01:03:36.371873+0000 (oldest deadline 2024-05-09T01:04:01.071919+0000) 2024-05-09T01:04:23.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:23 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:23.209+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-09T01:03:46.873806+0000 front 2024-05-09T01:03:46.873608+0000 (oldest deadline 2024-05-09T01:04:09.773530+0000) 2024-05-09T01:04:23.606 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:23 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:23.209+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-05-09T01:03:53.774919+0000 front 2024-05-09T01:03:53.774963+0000 (oldest deadline 2024-05-09T01:04:19.074815+0000) 2024-05-09T01:04:23.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:23 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:23.329+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:04:23.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:23 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:23.329+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:12.758527+0000 front 2024-05-09T01:03:12.758364+0000 (oldest deadline 2024-05-09T01:03:35.658450+0000) 2024-05-09T01:04:23.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:23 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:23.329+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.259808+0000 front 2024-05-09T01:03:20.259685+0000 (oldest deadline 2024-05-09T01:03:46.159864+0000) 2024-05-09T01:04:23.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:23 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:23.329+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:38.362243+0000 front 2024-05-09T01:03:38.362207+0000 (oldest deadline 2024-05-09T01:04:03.662309+0000) 2024-05-09T01:04:23.687 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:23 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:23.329+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-09T01:03:45.863988+0000 front 2024-05-09T01:03:45.864013+0000 (oldest deadline 2024-05-09T01:04:09.964129+0000) 2024-05-09T01:04:23.687 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:23 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:23.329+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-05-09T01:03:54.665305+0000 front 2024-05-09T01:03:54.665343+0000 (oldest deadline 2024-05-09T01:04:17.565166+0000) 2024-05-09T01:04:24.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:24 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:24.201+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.667724+0000 front 2024-05-09T01:03:13.667762+0000 (oldest deadline 2024-05-09T01:03:35.967853+0000) 2024-05-09T01:04:24.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:24 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:24.201+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.668856+0000 front 2024-05-09T01:03:20.668887+0000 (oldest deadline 2024-05-09T01:03:45.968898+0000) 2024-05-09T01:04:24.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:24 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:24.201+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:29.371010+0000 front 2024-05-09T01:03:29.370862+0000 (oldest deadline 2024-05-09T01:03:54.070791+0000) 2024-05-09T01:04:24.606 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:24 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:24.201+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:36.371823+0000 front 2024-05-09T01:03:36.371873+0000 (oldest deadline 2024-05-09T01:04:01.071919+0000) 2024-05-09T01:04:24.606 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:24 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:24.201+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-09T01:03:46.873806+0000 front 2024-05-09T01:03:46.873608+0000 (oldest deadline 2024-05-09T01:04:09.773530+0000) 2024-05-09T01:04:24.606 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:24 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:24.201+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-05-09T01:03:53.774919+0000 front 2024-05-09T01:03:53.774963+0000 (oldest deadline 2024-05-09T01:04:19.074815+0000) 2024-05-09T01:04:24.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:24 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:24.338+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:04:24.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:24 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:24.338+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:12.758527+0000 front 2024-05-09T01:03:12.758364+0000 (oldest deadline 2024-05-09T01:03:35.658450+0000) 2024-05-09T01:04:24.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:24 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:24.338+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.259808+0000 front 2024-05-09T01:03:20.259685+0000 (oldest deadline 2024-05-09T01:03:46.159864+0000) 2024-05-09T01:04:24.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:24 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:24.338+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:38.362243+0000 front 2024-05-09T01:03:38.362207+0000 (oldest deadline 2024-05-09T01:04:03.662309+0000) 2024-05-09T01:04:24.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:24 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:24.338+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-09T01:03:45.863988+0000 front 2024-05-09T01:03:45.864013+0000 (oldest deadline 2024-05-09T01:04:09.964129+0000) 2024-05-09T01:04:24.687 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:24 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:24.338+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-05-09T01:03:54.665305+0000 front 2024-05-09T01:03:54.665343+0000 (oldest deadline 2024-05-09T01:04:17.565166+0000) 2024-05-09T01:04:25.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:25 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:25.182+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.667724+0000 front 2024-05-09T01:03:13.667762+0000 (oldest deadline 2024-05-09T01:03:35.967853+0000) 2024-05-09T01:04:25.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:25 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:25.182+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.668856+0000 front 2024-05-09T01:03:20.668887+0000 (oldest deadline 2024-05-09T01:03:45.968898+0000) 2024-05-09T01:04:25.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:25 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:25.182+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:29.371010+0000 front 2024-05-09T01:03:29.370862+0000 (oldest deadline 2024-05-09T01:03:54.070791+0000) 2024-05-09T01:04:25.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:25 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:25.182+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:36.371823+0000 front 2024-05-09T01:03:36.371873+0000 (oldest deadline 2024-05-09T01:04:01.071919+0000) 2024-05-09T01:04:25.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:25 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:25.182+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-09T01:03:46.873806+0000 front 2024-05-09T01:03:46.873608+0000 (oldest deadline 2024-05-09T01:04:09.773530+0000) 2024-05-09T01:04:25.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:25 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:25.182+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-05-09T01:03:53.774919+0000 front 2024-05-09T01:03:53.774963+0000 (oldest deadline 2024-05-09T01:04:19.074815+0000) 2024-05-09T01:04:25.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:25 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:25.293+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:04:25.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:25 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:25.293+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:12.758527+0000 front 2024-05-09T01:03:12.758364+0000 (oldest deadline 2024-05-09T01:03:35.658450+0000) 2024-05-09T01:04:25.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:25 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:25.293+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.259808+0000 front 2024-05-09T01:03:20.259685+0000 (oldest deadline 2024-05-09T01:03:46.159864+0000) 2024-05-09T01:04:25.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:25 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:25.293+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:38.362243+0000 front 2024-05-09T01:03:38.362207+0000 (oldest deadline 2024-05-09T01:04:03.662309+0000) 2024-05-09T01:04:25.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:25 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:25.293+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-09T01:03:45.863988+0000 front 2024-05-09T01:03:45.864013+0000 (oldest deadline 2024-05-09T01:04:09.964129+0000) 2024-05-09T01:04:25.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:25 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:25.293+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-05-09T01:03:54.665305+0000 front 2024-05-09T01:03:54.665343+0000 (oldest deadline 2024-05-09T01:04:17.565166+0000) 2024-05-09T01:04:26.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:26 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:26.195+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.667724+0000 front 2024-05-09T01:03:13.667762+0000 (oldest deadline 2024-05-09T01:03:35.967853+0000) 2024-05-09T01:04:26.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:26 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:26.195+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.668856+0000 front 2024-05-09T01:03:20.668887+0000 (oldest deadline 2024-05-09T01:03:45.968898+0000) 2024-05-09T01:04:26.606 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:26 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:26.195+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:29.371010+0000 front 2024-05-09T01:03:29.370862+0000 (oldest deadline 2024-05-09T01:03:54.070791+0000) 2024-05-09T01:04:26.606 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:26 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:26.195+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:36.371823+0000 front 2024-05-09T01:03:36.371873+0000 (oldest deadline 2024-05-09T01:04:01.071919+0000) 2024-05-09T01:04:26.606 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:26 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:26.195+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-09T01:03:46.873806+0000 front 2024-05-09T01:03:46.873608+0000 (oldest deadline 2024-05-09T01:04:09.773530+0000) 2024-05-09T01:04:26.606 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:26 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:26.195+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-05-09T01:03:53.774919+0000 front 2024-05-09T01:03:53.774963+0000 (oldest deadline 2024-05-09T01:04:19.074815+0000) 2024-05-09T01:04:26.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:26 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:26.328+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:04:26.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:26 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:26.328+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:12.758527+0000 front 2024-05-09T01:03:12.758364+0000 (oldest deadline 2024-05-09T01:03:35.658450+0000) 2024-05-09T01:04:26.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:26 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:26.328+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.259808+0000 front 2024-05-09T01:03:20.259685+0000 (oldest deadline 2024-05-09T01:03:46.159864+0000) 2024-05-09T01:04:26.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:26 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:26.328+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:38.362243+0000 front 2024-05-09T01:03:38.362207+0000 (oldest deadline 2024-05-09T01:04:03.662309+0000) 2024-05-09T01:04:26.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:26 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:26.328+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-09T01:03:45.863988+0000 front 2024-05-09T01:03:45.864013+0000 (oldest deadline 2024-05-09T01:04:09.964129+0000) 2024-05-09T01:04:26.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:26 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:26.328+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-05-09T01:03:54.665305+0000 front 2024-05-09T01:03:54.665343+0000 (oldest deadline 2024-05-09T01:04:17.565166+0000) 2024-05-09T01:04:27.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:27 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:27.151+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:13.667724+0000 front 2024-05-09T01:03:13.667762+0000 (oldest deadline 2024-05-09T01:03:35.967853+0000) 2024-05-09T01:04:27.606 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:27 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:27.151+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.668856+0000 front 2024-05-09T01:03:20.668887+0000 (oldest deadline 2024-05-09T01:03:45.968898+0000) 2024-05-09T01:04:27.606 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:27 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:27.151+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-09T01:03:29.371010+0000 front 2024-05-09T01:03:29.370862+0000 (oldest deadline 2024-05-09T01:03:54.070791+0000) 2024-05-09T01:04:27.606 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:27 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:27.151+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:36.371823+0000 front 2024-05-09T01:03:36.371873+0000 (oldest deadline 2024-05-09T01:04:01.071919+0000) 2024-05-09T01:04:27.606 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:27 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:27.151+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-09T01:03:46.873806+0000 front 2024-05-09T01:03:46.873608+0000 (oldest deadline 2024-05-09T01:04:09.773530+0000) 2024-05-09T01:04:27.606 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:27 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10[59393]: 2024-05-09T01:04:27.151+0000 7f10274e6700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-05-09T01:03:53.774919+0000 front 2024-05-09T01:03:53.774963+0000 (oldest deadline 2024-05-09T01:04:19.074815+0000) 2024-05-09T01:04:27.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:27 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:27.367+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:04:27.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:27 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:27.367+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:12.758527+0000 front 2024-05-09T01:03:12.758364+0000 (oldest deadline 2024-05-09T01:03:35.658450+0000) 2024-05-09T01:04:27.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:27 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:27.367+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.259808+0000 front 2024-05-09T01:03:20.259685+0000 (oldest deadline 2024-05-09T01:03:46.159864+0000) 2024-05-09T01:04:27.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:27 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:27.367+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:38.362243+0000 front 2024-05-09T01:03:38.362207+0000 (oldest deadline 2024-05-09T01:04:03.662309+0000) 2024-05-09T01:04:27.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:27 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:27.367+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-09T01:03:45.863988+0000 front 2024-05-09T01:03:45.864013+0000 (oldest deadline 2024-05-09T01:04:09.964129+0000) 2024-05-09T01:04:27.687 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:27 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:27.367+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-05-09T01:03:54.665305+0000 front 2024-05-09T01:03:54.665343+0000 (oldest deadline 2024-05-09T01:04:17.565166+0000) 2024-05-09T01:04:27.687 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:27 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:27.367+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6820 osd.7 since back 2024-05-09T01:04:02.866337+0000 front 2024-05-09T01:04:02.866360+0000 (oldest deadline 2024-05-09T01:04:26.366215+0000) 2024-05-09T01:04:28.355 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:28 smithi131 podman[70677]: 2024-05-09 01:04:28.085927944 +0000 UTC m=+5.614104336 container died aded6b04f52b4c88326539c7754d283e0c1806c433c11a5b8b4a185ef025b349 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS) 2024-05-09T01:04:28.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:28 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:28.383+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:04:28.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:28 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:28.383+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:12.758527+0000 front 2024-05-09T01:03:12.758364+0000 (oldest deadline 2024-05-09T01:03:35.658450+0000) 2024-05-09T01:04:28.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:28 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:28.383+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.259808+0000 front 2024-05-09T01:03:20.259685+0000 (oldest deadline 2024-05-09T01:03:46.159864+0000) 2024-05-09T01:04:28.687 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:28 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:28.383+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:38.362243+0000 front 2024-05-09T01:03:38.362207+0000 (oldest deadline 2024-05-09T01:04:03.662309+0000) 2024-05-09T01:04:28.687 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:28 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:28.383+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-09T01:03:45.863988+0000 front 2024-05-09T01:03:45.864013+0000 (oldest deadline 2024-05-09T01:04:09.964129+0000) 2024-05-09T01:04:28.687 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:28 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:28.383+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-05-09T01:03:54.665305+0000 front 2024-05-09T01:03:54.665343+0000 (oldest deadline 2024-05-09T01:04:17.565166+0000) 2024-05-09T01:04:28.687 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:28 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:28.383+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6820 osd.7 since back 2024-05-09T01:04:02.866337+0000 front 2024-05-09T01:04:02.866360+0000 (oldest deadline 2024-05-09T01:04:26.366215+0000) 2024-05-09T01:04:28.855 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:28 smithi131 podman[70677]: 2024-05-09 01:04:28.378823248 +0000 UTC m=+5.906999644 container cleanup aded6b04f52b4c88326539c7754d283e0c1806c433c11a5b8b4a185ef025b349 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T01:04:28.855 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:28 smithi131 bash[70677]: ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10 2024-05-09T01:04:28.855 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:28 smithi131 podman[70694]: 2024-05-09 01:04:28.605056656 +0000 UTC m=+0.518344510 container remove aded6b04f52b4c88326539c7754d283e0c1806c433c11a5b8b4a185ef025b349 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-05-09T01:04:29.200 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:28 smithi131 podman[70784]: 2024-05-09 01:04:28.81107603 +0000 UTC m=+0.018054213 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T01:04:29.200 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:28 smithi131 podman[70784]: 2024-05-09 01:04:28.922663357 +0000 UTC m=+0.129641527 container create 025c7dfa59417a555acb95a6ef1026eaeb0fea4baa2ae432167491389fabb07c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=) 2024-05-09T01:04:29.201 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:29 smithi131 podman[70784]: 2024-05-09 01:04:29.133685036 +0000 UTC m=+0.340663211 container init 025c7dfa59417a555acb95a6ef1026eaeb0fea4baa2ae432167491389fabb07c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True) 2024-05-09T01:04:29.201 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:29 smithi131 podman[70784]: 2024-05-09 01:04:29.137365123 +0000 UTC m=+0.344343295 container start 025c7dfa59417a555acb95a6ef1026eaeb0fea4baa2ae432167491389fabb07c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-05-09T01:04:29.201 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:29 smithi131 podman[70784]: 2024-05-09 01:04:29.200087798 +0000 UTC m=+0.407065981 container attach 025c7dfa59417a555acb95a6ef1026eaeb0fea4baa2ae432167491389fabb07c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-09T01:04:29.605 INFO:journalctl@ceph.osd.10.smithi131.stdout:May 09 01:04:29 smithi131 podman[70784]: 2024-05-09 01:04:29.425759903 +0000 UTC m=+0.632738079 container died 025c7dfa59417a555acb95a6ef1026eaeb0fea4baa2ae432167491389fabb07c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-10-deactivate, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-05-09T01:04:29.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:29 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:29.356+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:04:29.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:29 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:29.356+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:12.758527+0000 front 2024-05-09T01:03:12.758364+0000 (oldest deadline 2024-05-09T01:03:35.658450+0000) 2024-05-09T01:04:29.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:29 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:29.356+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.259808+0000 front 2024-05-09T01:03:20.259685+0000 (oldest deadline 2024-05-09T01:03:46.159864+0000) 2024-05-09T01:04:29.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:29 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:29.356+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:38.362243+0000 front 2024-05-09T01:03:38.362207+0000 (oldest deadline 2024-05-09T01:04:03.662309+0000) 2024-05-09T01:04:29.687 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:29 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:29.356+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-09T01:03:45.863988+0000 front 2024-05-09T01:03:45.864013+0000 (oldest deadline 2024-05-09T01:04:09.964129+0000) 2024-05-09T01:04:29.687 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:29 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:29.356+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-05-09T01:03:54.665305+0000 front 2024-05-09T01:03:54.665343+0000 (oldest deadline 2024-05-09T01:04:17.565166+0000) 2024-05-09T01:04:29.687 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:29 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:29.356+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6820 osd.7 since back 2024-05-09T01:04:02.866337+0000 front 2024-05-09T01:04:02.866360+0000 (oldest deadline 2024-05-09T01:04:26.366215+0000) 2024-05-09T01:04:30.034 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@osd.10.service' 2024-05-09T01:04:30.075 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T01:04:30.075 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-05-09T01:04:30.075 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-05-09T01:04:30.075 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@osd.11 2024-05-09T01:04:30.374 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:30 smithi157 systemd[1]: Stopping Ceph osd.11 for d85814c6-0d9b-11ef-bc97-c7b262605968... 2024-05-09T01:04:30.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:30 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:30.373+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:04:30.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:30 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:30.373+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:12.758527+0000 front 2024-05-09T01:03:12.758364+0000 (oldest deadline 2024-05-09T01:03:35.658450+0000) 2024-05-09T01:04:30.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:30 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:30.373+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.259808+0000 front 2024-05-09T01:03:20.259685+0000 (oldest deadline 2024-05-09T01:03:46.159864+0000) 2024-05-09T01:04:30.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:30 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:30.373+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:38.362243+0000 front 2024-05-09T01:03:38.362207+0000 (oldest deadline 2024-05-09T01:04:03.662309+0000) 2024-05-09T01:04:30.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:30 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:30.373+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-09T01:03:45.863988+0000 front 2024-05-09T01:03:45.864013+0000 (oldest deadline 2024-05-09T01:04:09.964129+0000) 2024-05-09T01:04:30.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:30 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:30.373+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-05-09T01:03:54.665305+0000 front 2024-05-09T01:03:54.665343+0000 (oldest deadline 2024-05-09T01:04:17.565166+0000) 2024-05-09T01:04:30.687 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:30 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:30.373+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6820 osd.7 since back 2024-05-09T01:04:02.866337+0000 front 2024-05-09T01:04:02.866360+0000 (oldest deadline 2024-05-09T01:04:26.366215+0000) 2024-05-09T01:04:30.687 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:30 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:30.392+0000 7f3ba566b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T01:04:30.687 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:30 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:30.392+0000 7f3ba566b700 -1 osd.11 85 *** Got signal Terminated *** 2024-05-09T01:04:30.687 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:30 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:30.392+0000 7f3ba566b700 -1 osd.11 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T01:04:31.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:31 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:31.359+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:04:31.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:31 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:31.359+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:12.758527+0000 front 2024-05-09T01:03:12.758364+0000 (oldest deadline 2024-05-09T01:03:35.658450+0000) 2024-05-09T01:04:31.687 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:31 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:31.359+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.259808+0000 front 2024-05-09T01:03:20.259685+0000 (oldest deadline 2024-05-09T01:03:46.159864+0000) 2024-05-09T01:04:31.687 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:31 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:31.359+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:38.362243+0000 front 2024-05-09T01:03:38.362207+0000 (oldest deadline 2024-05-09T01:04:03.662309+0000) 2024-05-09T01:04:31.687 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:31 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:31.359+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-09T01:03:45.863988+0000 front 2024-05-09T01:03:45.864013+0000 (oldest deadline 2024-05-09T01:04:09.964129+0000) 2024-05-09T01:04:31.687 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:31 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:31.359+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-05-09T01:03:54.665305+0000 front 2024-05-09T01:03:54.665343+0000 (oldest deadline 2024-05-09T01:04:17.565166+0000) 2024-05-09T01:04:31.687 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:31 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:31.359+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6820 osd.7 since back 2024-05-09T01:04:02.866337+0000 front 2024-05-09T01:04:02.866360+0000 (oldest deadline 2024-05-09T01:04:26.366215+0000) 2024-05-09T01:04:32.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:32 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:32.314+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:04:32.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:32 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:32.314+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:12.758527+0000 front 2024-05-09T01:03:12.758364+0000 (oldest deadline 2024-05-09T01:03:35.658450+0000) 2024-05-09T01:04:32.687 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:32 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:32.314+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.259808+0000 front 2024-05-09T01:03:20.259685+0000 (oldest deadline 2024-05-09T01:03:46.159864+0000) 2024-05-09T01:04:32.687 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:32 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:32.314+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:38.362243+0000 front 2024-05-09T01:03:38.362207+0000 (oldest deadline 2024-05-09T01:04:03.662309+0000) 2024-05-09T01:04:32.687 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:32 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:32.314+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-09T01:03:45.863988+0000 front 2024-05-09T01:03:45.864013+0000 (oldest deadline 2024-05-09T01:04:09.964129+0000) 2024-05-09T01:04:32.687 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:32 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:32.314+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-05-09T01:03:54.665305+0000 front 2024-05-09T01:03:54.665343+0000 (oldest deadline 2024-05-09T01:04:17.565166+0000) 2024-05-09T01:04:32.687 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:32 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:32.314+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6820 osd.7 since back 2024-05-09T01:04:02.866337+0000 front 2024-05-09T01:04:02.866360+0000 (oldest deadline 2024-05-09T01:04:26.366215+0000) 2024-05-09T01:04:33.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:33 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:33.299+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:04:33.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:33 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:33.299+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:12.758527+0000 front 2024-05-09T01:03:12.758364+0000 (oldest deadline 2024-05-09T01:03:35.658450+0000) 2024-05-09T01:04:33.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:33 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:33.299+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.259808+0000 front 2024-05-09T01:03:20.259685+0000 (oldest deadline 2024-05-09T01:03:46.159864+0000) 2024-05-09T01:04:33.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:33 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:33.299+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:38.362243+0000 front 2024-05-09T01:03:38.362207+0000 (oldest deadline 2024-05-09T01:04:03.662309+0000) 2024-05-09T01:04:33.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:33 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:33.299+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-09T01:03:45.863988+0000 front 2024-05-09T01:03:45.864013+0000 (oldest deadline 2024-05-09T01:04:09.964129+0000) 2024-05-09T01:04:33.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:33 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:33.299+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-05-09T01:03:54.665305+0000 front 2024-05-09T01:03:54.665343+0000 (oldest deadline 2024-05-09T01:04:17.565166+0000) 2024-05-09T01:04:33.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:33 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:33.299+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6820 osd.7 since back 2024-05-09T01:04:02.866337+0000 front 2024-05-09T01:04:02.866360+0000 (oldest deadline 2024-05-09T01:04:26.366215+0000) 2024-05-09T01:04:34.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:34 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:34.296+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:04:34.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:34 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:34.296+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:12.758527+0000 front 2024-05-09T01:03:12.758364+0000 (oldest deadline 2024-05-09T01:03:35.658450+0000) 2024-05-09T01:04:34.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:34 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:34.296+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.259808+0000 front 2024-05-09T01:03:20.259685+0000 (oldest deadline 2024-05-09T01:03:46.159864+0000) 2024-05-09T01:04:34.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:34 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:34.296+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:38.362243+0000 front 2024-05-09T01:03:38.362207+0000 (oldest deadline 2024-05-09T01:04:03.662309+0000) 2024-05-09T01:04:34.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:34 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:34.296+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-09T01:03:45.863988+0000 front 2024-05-09T01:03:45.864013+0000 (oldest deadline 2024-05-09T01:04:09.964129+0000) 2024-05-09T01:04:34.687 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:34 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:34.296+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-05-09T01:03:54.665305+0000 front 2024-05-09T01:03:54.665343+0000 (oldest deadline 2024-05-09T01:04:17.565166+0000) 2024-05-09T01:04:34.687 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:34 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:34.296+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6820 osd.7 since back 2024-05-09T01:04:02.866337+0000 front 2024-05-09T01:04:02.866360+0000 (oldest deadline 2024-05-09T01:04:26.366215+0000) 2024-05-09T01:04:35.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:35 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:35.321+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-09T01:03:02.157105+0000 front 2024-05-09T01:03:02.157272+0000 (oldest deadline 2024-05-09T01:03:27.457235+0000) 2024-05-09T01:04:35.687 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:35 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:35.321+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6804 osd.1 since back 2024-05-09T01:03:12.758527+0000 front 2024-05-09T01:03:12.758364+0000 (oldest deadline 2024-05-09T01:03:35.658450+0000) 2024-05-09T01:04:35.687 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:35 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:35.321+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-05-09T01:03:20.259808+0000 front 2024-05-09T01:03:20.259685+0000 (oldest deadline 2024-05-09T01:03:46.159864+0000) 2024-05-09T01:04:35.687 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:35 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:35.321+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6812 osd.4 since back 2024-05-09T01:03:38.362243+0000 front 2024-05-09T01:03:38.362207+0000 (oldest deadline 2024-05-09T01:04:03.662309+0000) 2024-05-09T01:04:35.687 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:35 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:35.321+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-05-09T01:03:45.863988+0000 front 2024-05-09T01:03:45.864013+0000 (oldest deadline 2024-05-09T01:04:09.964129+0000) 2024-05-09T01:04:35.687 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:35 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:35.321+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-05-09T01:03:54.665305+0000 front 2024-05-09T01:03:54.665343+0000 (oldest deadline 2024-05-09T01:04:17.565166+0000) 2024-05-09T01:04:35.687 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:35 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:35.321+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.131:6820 osd.7 since back 2024-05-09T01:04:02.866337+0000 front 2024-05-09T01:04:02.866360+0000 (oldest deadline 2024-05-09T01:04:26.366215+0000) 2024-05-09T01:04:35.687 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:35 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11[59417]: 2024-05-09T01:04:35.321+0000 7f3ba1c21700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.157:6820 osd.8 since back 2024-05-09T01:04:09.767574+0000 front 2024-05-09T01:04:09.767608+0000 (oldest deadline 2024-05-09T01:04:34.467611+0000) 2024-05-09T01:04:36.243 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:35 smithi157 podman[71579]: 2024-05-09 01:04:35.951917674 +0000 UTC m=+5.674940796 container died cd0bccf8b439fa67c7155fa4830b3c611ed7a41efce26bb4eadba41d20c44051 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-09T01:04:36.243 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:36 smithi157 podman[71579]: 2024-05-09 01:04:36.243041095 +0000 UTC m=+5.966064204 container cleanup cd0bccf8b439fa67c7155fa4830b3c611ed7a41efce26bb4eadba41d20c44051 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11, org.label-schema.build-date=20240507, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T01:04:36.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:36 smithi157 bash[71579]: ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11 2024-05-09T01:04:36.686 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:36 smithi157 podman[71596]: 2024-05-09 01:04:36.48552226 +0000 UTC m=+0.530503120 container remove cd0bccf8b439fa67c7155fa4830b3c611ed7a41efce26bb4eadba41d20c44051 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T01:04:37.122 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:36 smithi157 podman[71685]: 2024-05-09 01:04:36.708357687 +0000 UTC m=+0.017976860 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T01:04:37.122 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:36 smithi157 podman[71685]: 2024-05-09 01:04:36.811249288 +0000 UTC m=+0.120868489 container create da34ed0cd82942aa036bcec06f71fda1431e373d4e6f5c409e31303edc4811d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T01:04:37.122 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:37 smithi157 podman[71685]: 2024-05-09 01:04:37.038360368 +0000 UTC m=+0.347979524 container init da34ed0cd82942aa036bcec06f71fda1431e373d4e6f5c409e31303edc4811d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, ceph=True) 2024-05-09T01:04:37.122 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:37 smithi157 podman[71685]: 2024-05-09 01:04:37.041502289 +0000 UTC m=+0.351121455 container start da34ed0cd82942aa036bcec06f71fda1431e373d4e6f5c409e31303edc4811d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-09T01:04:37.122 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:37 smithi157 podman[71685]: 2024-05-09 01:04:37.121926173 +0000 UTC m=+0.431545345 container attach da34ed0cd82942aa036bcec06f71fda1431e373d4e6f5c409e31303edc4811d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=) 2024-05-09T01:04:37.436 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:37 smithi157 podman[71685]: 2024-05-09 01:04:37.332311104 +0000 UTC m=+0.641930270 container died da34ed0cd82942aa036bcec06f71fda1431e373d4e6f5c409e31303edc4811d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11-deactivate, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T01:04:37.845 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@osd.11.service' 2024-05-09T01:04:37.867 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:37 smithi157 podman[71711]: 2024-05-09 01:04:37.825457665 +0000 UTC m=+0.488477831 container remove da34ed0cd82942aa036bcec06f71fda1431e373d4e6f5c409e31303edc4811d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-osd-11-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T01:04:37.867 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:37 smithi157 systemd[1]: ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@osd.11.service: Deactivated successfully. 2024-05-09T01:04:37.867 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:37 smithi157 systemd[1]: Stopped Ceph osd.11 for d85814c6-0d9b-11ef-bc97-c7b262605968. 2024-05-09T01:04:37.868 INFO:journalctl@ceph.osd.11.smithi157.stdout:May 09 01:04:37 smithi157 systemd[1]: ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@osd.11.service: Consumed 10min 33.115s CPU time. 2024-05-09T01:04:37.883 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T01:04:37.883 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-05-09T01:04:37.883 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-05-09T01:04:37.883 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@mds.a 2024-05-09T01:04:38.193 INFO:journalctl@ceph.mds.a.smithi005.stdout:May 09 01:04:37 smithi005 systemd[1]: Stopping Ceph mds.a for d85814c6-0d9b-11ef-bc97-c7b262605968... 2024-05-09T01:04:38.476 INFO:journalctl@ceph.mds.a.smithi005.stdout:May 09 01:04:38 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-a[72705]: 2024-05-09T01:04:38.192+0000 7f6212f3b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T01:04:38.476 INFO:journalctl@ceph.mds.a.smithi005.stdout:May 09 01:04:38 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-a[72705]: 2024-05-09T01:04:38.192+0000 7f6212f3b700 -1 mds.a *** got signal Terminated *** 2024-05-09T01:04:41.693 INFO:journalctl@ceph.mds.a.smithi005.stdout:May 09 01:04:41 smithi005 podman[112632]: 2024-05-09 01:04:41.415033954 +0000 UTC m=+3.332031191 container died bc99ce756e918571edfb27ce835edd0c5f6db4366bca728d4a3be60faa2cac4a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-a, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0) 2024-05-09T01:04:42.040 INFO:journalctl@ceph.mds.a.smithi005.stdout:May 09 01:04:41 smithi005 podman[112632]: 2024-05-09 01:04:41.858319652 +0000 UTC m=+3.775316896 container cleanup bc99ce756e918571edfb27ce835edd0c5f6db4366bca728d4a3be60faa2cac4a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-a, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-05-09T01:04:42.041 INFO:journalctl@ceph.mds.a.smithi005.stdout:May 09 01:04:41 smithi005 bash[112632]: ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-a 2024-05-09T01:04:42.307 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@mds.a.service' 2024-05-09T01:04:42.331 INFO:journalctl@ceph.mds.a.smithi005.stdout:May 09 01:04:42 smithi005 podman[112658]: 2024-05-09 01:04:42.071977491 +0000 UTC m=+0.652331439 container remove bc99ce756e918571edfb27ce835edd0c5f6db4366bca728d4a3be60faa2cac4a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-a, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS) 2024-05-09T01:04:42.331 INFO:journalctl@ceph.mds.a.smithi005.stdout:May 09 01:04:42 smithi005 systemd[1]: ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@mds.a.service: Deactivated successfully. 2024-05-09T01:04:42.331 INFO:journalctl@ceph.mds.a.smithi005.stdout:May 09 01:04:42 smithi005 systemd[1]: Stopped Ceph mds.a for d85814c6-0d9b-11ef-bc97-c7b262605968. 2024-05-09T01:04:42.331 INFO:journalctl@ceph.mds.a.smithi005.stdout:May 09 01:04:42 smithi005 systemd[1]: ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@mds.a.service: Consumed 3.109s CPU time. 2024-05-09T01:04:42.345 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T01:04:42.345 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-05-09T01:04:42.345 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-05-09T01:04:42.345 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@mds.d 2024-05-09T01:04:42.684 INFO:journalctl@ceph.mds.d.smithi005.stdout:May 09 01:04:42 smithi005 systemd[1]: Stopping Ceph mds.d for d85814c6-0d9b-11ef-bc97-c7b262605968... 2024-05-09T01:04:43.041 INFO:journalctl@ceph.mds.d.smithi005.stdout:May 09 01:04:42 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-d[71431]: 2024-05-09T01:04:42.683+0000 7f499b8cf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T01:04:43.041 INFO:journalctl@ceph.mds.d.smithi005.stdout:May 09 01:04:42 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-d[71431]: 2024-05-09T01:04:42.683+0000 7f499b8cf700 -1 mds.d *** got signal Terminated *** 2024-05-09T01:04:46.974 INFO:journalctl@ceph.mds.d.smithi005.stdout:May 09 01:04:46 smithi005 podman[112764]: 2024-05-09 01:04:46.698376314 +0000 UTC m=+4.128401594 container died c546b98bbf6a116bf988abcec95d62c860ff0440294a2e8e558ca53991a12f84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-d, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-05-09T01:04:47.290 INFO:journalctl@ceph.mds.d.smithi005.stdout:May 09 01:04:47 smithi005 podman[112764]: 2024-05-09 01:04:47.074086774 +0000 UTC m=+4.504112053 container cleanup c546b98bbf6a116bf988abcec95d62c860ff0440294a2e8e558ca53991a12f84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-d, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-05-09T01:04:47.291 INFO:journalctl@ceph.mds.d.smithi005.stdout:May 09 01:04:47 smithi005 bash[112764]: ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-d 2024-05-09T01:04:47.448 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@mds.d.service' 2024-05-09T01:04:47.483 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T01:04:47.483 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-05-09T01:04:47.483 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-05-09T01:04:47.483 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@mds.g 2024-05-09T01:04:47.790 INFO:journalctl@ceph.mds.g.smithi005.stdout:May 09 01:04:47 smithi005 systemd[1]: Stopping Ceph mds.g for d85814c6-0d9b-11ef-bc97-c7b262605968... 2024-05-09T01:04:48.291 INFO:journalctl@ceph.mds.g.smithi005.stdout:May 09 01:04:47 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-g[72281]: 2024-05-09T01:04:47.811+0000 7f2d0dcf0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T01:04:48.291 INFO:journalctl@ceph.mds.g.smithi005.stdout:May 09 01:04:47 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-g[72281]: 2024-05-09T01:04:47.811+0000 7f2d0dcf0700 -1 mds.g *** got signal Terminated *** 2024-05-09T01:04:49.713 INFO:journalctl@ceph.mds.g.smithi005.stdout:May 09 01:04:49 smithi005 podman[112887]: 2024-05-09 01:04:49.424408578 +0000 UTC m=+1.713069606 container died b9bda7917e09f666928b818ab5530b34a9dad48fcac19c782d8dce11a159f278 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-g, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-05-09T01:04:50.040 INFO:journalctl@ceph.mds.g.smithi005.stdout:May 09 01:04:49 smithi005 podman[112887]: 2024-05-09 01:04:49.878369085 +0000 UTC m=+2.167030144 container cleanup b9bda7917e09f666928b818ab5530b34a9dad48fcac19c782d8dce11a159f278 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-g, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-09T01:04:50.041 INFO:journalctl@ceph.mds.g.smithi005.stdout:May 09 01:04:49 smithi005 bash[112887]: ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-g 2024-05-09T01:04:50.314 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@mds.g.service' 2024-05-09T01:04:50.336 INFO:journalctl@ceph.mds.g.smithi005.stdout:May 09 01:04:50 smithi005 podman[112900]: 2024-05-09 01:04:50.063135083 +0000 UTC m=+0.630294112 container remove b9bda7917e09f666928b818ab5530b34a9dad48fcac19c782d8dce11a159f278 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-g, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T01:04:50.336 INFO:journalctl@ceph.mds.g.smithi005.stdout:May 09 01:04:50 smithi005 systemd[1]: ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@mds.g.service: Deactivated successfully. 2024-05-09T01:04:50.336 INFO:journalctl@ceph.mds.g.smithi005.stdout:May 09 01:04:50 smithi005 systemd[1]: Stopped Ceph mds.g for d85814c6-0d9b-11ef-bc97-c7b262605968. 2024-05-09T01:04:50.336 INFO:journalctl@ceph.mds.g.smithi005.stdout:May 09 01:04:50 smithi005 systemd[1]: ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@mds.g.service: Consumed 2.895s CPU time. 2024-05-09T01:04:50.348 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T01:04:50.348 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-05-09T01:04:50.348 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-05-09T01:04:50.348 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@mds.j 2024-05-09T01:04:50.666 INFO:journalctl@ceph.mds.j.smithi005.stdout:May 09 01:04:50 smithi005 systemd[1]: Stopping Ceph mds.j for d85814c6-0d9b-11ef-bc97-c7b262605968... 2024-05-09T01:04:51.040 INFO:journalctl@ceph.mds.j.smithi005.stdout:May 09 01:04:50 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-j[70985]: 2024-05-09T01:04:50.664+0000 7f880db22700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T01:04:51.040 INFO:journalctl@ceph.mds.j.smithi005.stdout:May 09 01:04:50 smithi005 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-j[70985]: 2024-05-09T01:04:50.664+0000 7f880db22700 -1 mds.j *** got signal Terminated *** 2024-05-09T01:04:54.540 INFO:journalctl@ceph.mds.j.smithi005.stdout:May 09 01:04:54 smithi005 podman[113007]: 2024-05-09 01:04:54.2168802 +0000 UTC m=+3.645983531 container died 8569b4a1ca697e56da3ba3fbc190dfe2933f197a3eadc9725de9be5fc58a4568 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-j, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-05-09T01:04:55.040 INFO:journalctl@ceph.mds.j.smithi005.stdout:May 09 01:04:54 smithi005 podman[113007]: 2024-05-09 01:04:54.632445379 +0000 UTC m=+4.061548738 container cleanup 8569b4a1ca697e56da3ba3fbc190dfe2933f197a3eadc9725de9be5fc58a4568 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-j, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T01:04:55.041 INFO:journalctl@ceph.mds.j.smithi005.stdout:May 09 01:04:54 smithi005 bash[113007]: ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-j 2024-05-09T01:04:55.041 INFO:journalctl@ceph.mds.j.smithi005.stdout:May 09 01:04:54 smithi005 podman[113021]: 2024-05-09 01:04:54.833436708 +0000 UTC m=+0.619610734 container remove 8569b4a1ca697e56da3ba3fbc190dfe2933f197a3eadc9725de9be5fc58a4568 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-j, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-09T01:04:55.071 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@mds.j.service' 2024-05-09T01:04:55.104 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T01:04:55.104 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-05-09T01:04:55.104 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-05-09T01:04:55.104 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@mds.b 2024-05-09T01:04:55.387 INFO:journalctl@ceph.mds.b.smithi131.stdout:May 09 01:04:55 smithi131 systemd[1]: Stopping Ceph mds.b for d85814c6-0d9b-11ef-bc97-c7b262605968... 2024-05-09T01:04:55.855 INFO:journalctl@ceph.mds.b.smithi131.stdout:May 09 01:04:55 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-b[64607]: 2024-05-09T01:04:55.386+0000 7f6784e45700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T01:04:55.855 INFO:journalctl@ceph.mds.b.smithi131.stdout:May 09 01:04:55 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-b[64607]: 2024-05-09T01:04:55.386+0000 7f6784e45700 -1 mds.b *** got signal Terminated *** 2024-05-09T01:04:57.855 INFO:journalctl@ceph.mds.b.smithi131.stdout:May 09 01:04:57 smithi131 podman[70890]: 2024-05-09 01:04:57.619564899 +0000 UTC m=+2.328229965 container died 2c466d9bfca33b33f227b54388e65cbe72a2701aed8ffe485bb70cd2622b6de7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD) 2024-05-09T01:04:58.163 INFO:journalctl@ceph.mds.b.smithi131.stdout:May 09 01:04:57 smithi131 podman[70890]: 2024-05-09 01:04:57.953920639 +0000 UTC m=+2.662585698 container cleanup 2c466d9bfca33b33f227b54388e65cbe72a2701aed8ffe485bb70cd2622b6de7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-b, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T01:04:58.163 INFO:journalctl@ceph.mds.b.smithi131.stdout:May 09 01:04:57 smithi131 bash[70890]: ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-b 2024-05-09T01:04:58.373 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@mds.b.service' 2024-05-09T01:04:58.407 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T01:04:58.407 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-05-09T01:04:58.408 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-05-09T01:04:58.408 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@mds.e 2024-05-09T01:04:58.855 INFO:journalctl@ceph.mds.e.smithi131.stdout:May 09 01:04:58 smithi131 systemd[1]: Stopping Ceph mds.e for d85814c6-0d9b-11ef-bc97-c7b262605968... 2024-05-09T01:04:58.855 INFO:journalctl@ceph.mds.e.smithi131.stdout:May 09 01:04:58 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-e[63988]: 2024-05-09T01:04:58.715+0000 7f1689315700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T01:04:58.855 INFO:journalctl@ceph.mds.e.smithi131.stdout:May 09 01:04:58 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-e[63988]: 2024-05-09T01:04:58.715+0000 7f1689315700 -1 mds.e *** got signal Terminated *** 2024-05-09T01:05:02.105 INFO:journalctl@ceph.mds.e.smithi131.stdout:May 09 01:05:01 smithi131 podman[71025]: 2024-05-09 01:05:01.61249957 +0000 UTC m=+2.993145652 container died 30bda55db0ed02e5af8b8fb07128aebf64e4430f76311cb1e675f1855fe47da1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-e, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T01:05:02.566 INFO:journalctl@ceph.mds.e.smithi131.stdout:May 09 01:05:02 smithi131 podman[71025]: 2024-05-09 01:05:02.274131989 +0000 UTC m=+3.654778069 container cleanup 30bda55db0ed02e5af8b8fb07128aebf64e4430f76311cb1e675f1855fe47da1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-e, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True) 2024-05-09T01:05:02.566 INFO:journalctl@ceph.mds.e.smithi131.stdout:May 09 01:05:02 smithi131 bash[71025]: ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-e 2024-05-09T01:05:02.691 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@mds.e.service' 2024-05-09T01:05:02.723 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T01:05:02.723 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-05-09T01:05:02.723 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-05-09T01:05:02.723 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@mds.h 2024-05-09T01:05:02.855 INFO:journalctl@ceph.mds.h.smithi131.stdout:May 09 01:05:02 smithi131 systemd[1]: Stopping Ceph mds.h for d85814c6-0d9b-11ef-bc97-c7b262605968... 2024-05-09T01:05:03.355 INFO:journalctl@ceph.mds.h.smithi131.stdout:May 09 01:05:03 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-h[64299]: 2024-05-09T01:05:03.069+0000 7fc612b62700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T01:05:03.355 INFO:journalctl@ceph.mds.h.smithi131.stdout:May 09 01:05:03 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-h[64299]: 2024-05-09T01:05:03.069+0000 7fc612b62700 -1 mds.h *** got signal Terminated *** 2024-05-09T01:05:05.920 INFO:journalctl@ceph.mds.h.smithi131.stdout:May 09 01:05:05 smithi131 podman[71147]: 2024-05-09 01:05:05.620566552 +0000 UTC m=+2.686787466 container died ab258a8336b81c38d4b9066332ec596848cd2f1f72b11ed3249a9603aea10f25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-h, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T01:05:06.262 INFO:journalctl@ceph.mds.h.smithi131.stdout:May 09 01:05:05 smithi131 podman[71147]: 2024-05-09 01:05:05.994713719 +0000 UTC m=+3.060934639 container cleanup ab258a8336b81c38d4b9066332ec596848cd2f1f72b11ed3249a9603aea10f25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-h, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T01:05:06.262 INFO:journalctl@ceph.mds.h.smithi131.stdout:May 09 01:05:05 smithi131 bash[71147]: ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-h 2024-05-09T01:05:06.410 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@mds.h.service' 2024-05-09T01:05:06.441 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T01:05:06.441 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-05-09T01:05:06.441 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-05-09T01:05:06.442 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@mds.k 2024-05-09T01:05:06.855 INFO:journalctl@ceph.mds.k.smithi131.stdout:May 09 01:05:06 smithi131 systemd[1]: Stopping Ceph mds.k for d85814c6-0d9b-11ef-bc97-c7b262605968... 2024-05-09T01:05:06.855 INFO:journalctl@ceph.mds.k.smithi131.stdout:May 09 01:05:06 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-k[63667]: 2024-05-09T01:05:06.764+0000 7fb0afc36700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T01:05:06.855 INFO:journalctl@ceph.mds.k.smithi131.stdout:May 09 01:05:06 smithi131 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-k[63667]: 2024-05-09T01:05:06.764+0000 7fb0afc36700 -1 mds.k *** got signal Terminated *** 2024-05-09T01:05:09.940 INFO:journalctl@ceph.mds.k.smithi131.stdout:May 09 01:05:09 smithi131 podman[71268]: 2024-05-09 01:05:09.6171876 +0000 UTC m=+2.960664377 container died 6b161fb19c155a80d32256cb62308c9ba27d8c96ca48ac764b7ad7809ac103e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-k, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-05-09T01:05:10.354 INFO:journalctl@ceph.mds.k.smithi131.stdout:May 09 01:05:09 smithi131 podman[71268]: 2024-05-09 01:05:09.940111911 +0000 UTC m=+3.283588747 container cleanup 6b161fb19c155a80d32256cb62308c9ba27d8c96ca48ac764b7ad7809ac103e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-k, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS) 2024-05-09T01:05:10.354 INFO:journalctl@ceph.mds.k.smithi131.stdout:May 09 01:05:09 smithi131 bash[71268]: ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-k 2024-05-09T01:05:10.354 INFO:journalctl@ceph.mds.k.smithi131.stdout:May 09 01:05:10 smithi131 podman[71282]: 2024-05-09 01:05:10.149360465 +0000 UTC m=+0.531147946 container remove 6b161fb19c155a80d32256cb62308c9ba27d8c96ca48ac764b7ad7809ac103e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-k, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux ) 2024-05-09T01:05:10.368 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@mds.k.service' 2024-05-09T01:05:10.400 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T01:05:10.400 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-05-09T01:05:10.400 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-05-09T01:05:10.400 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@mds.c 2024-05-09T01:05:10.686 INFO:journalctl@ceph.mds.c.smithi157.stdout:May 09 01:05:10 smithi157 systemd[1]: Stopping Ceph mds.c for d85814c6-0d9b-11ef-bc97-c7b262605968... 2024-05-09T01:05:11.186 INFO:journalctl@ceph.mds.c.smithi157.stdout:May 09 01:05:10 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-c[64565]: 2024-05-09T01:05:10.717+0000 7f1b733e6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T01:05:11.186 INFO:journalctl@ceph.mds.c.smithi157.stdout:May 09 01:05:10 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-c[64565]: 2024-05-09T01:05:10.717+0000 7f1b733e6700 -1 mds.c *** got signal Terminated *** 2024-05-09T01:05:13.621 INFO:journalctl@ceph.mds.c.smithi157.stdout:May 09 01:05:13 smithi157 podman[71804]: 2024-05-09 01:05:13.35433167 +0000 UTC m=+2.761403657 container died 7b1acef3ad12f84662f6151558b6034649e614abb548d90ccbfc920a810a59c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-c, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-05-09T01:05:13.936 INFO:journalctl@ceph.mds.c.smithi157.stdout:May 09 01:05:13 smithi157 podman[71804]: 2024-05-09 01:05:13.73283899 +0000 UTC m=+3.139910973 container cleanup 7b1acef3ad12f84662f6151558b6034649e614abb548d90ccbfc920a810a59c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-c, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-05-09T01:05:13.936 INFO:journalctl@ceph.mds.c.smithi157.stdout:May 09 01:05:13 smithi157 bash[71804]: ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-c 2024-05-09T01:05:14.151 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@mds.c.service' 2024-05-09T01:05:14.187 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T01:05:14.187 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-05-09T01:05:14.187 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-05-09T01:05:14.187 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@mds.f 2024-05-09T01:05:14.686 INFO:journalctl@ceph.mds.f.smithi157.stdout:May 09 01:05:14 smithi157 systemd[1]: Stopping Ceph mds.f for d85814c6-0d9b-11ef-bc97-c7b262605968... 2024-05-09T01:05:14.686 INFO:journalctl@ceph.mds.f.smithi157.stdout:May 09 01:05:14 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-f[64254]: 2024-05-09T01:05:14.511+0000 7f64b8474700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T01:05:14.686 INFO:journalctl@ceph.mds.f.smithi157.stdout:May 09 01:05:14 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-f[64254]: 2024-05-09T01:05:14.511+0000 7f64b8474700 -1 mds.f *** got signal Terminated *** 2024-05-09T01:05:17.605 INFO:journalctl@ceph.mds.f.smithi157.stdout:May 09 01:05:17 smithi157 podman[71937]: 2024-05-09 01:05:17.340362531 +0000 UTC m=+2.929615466 container died e5281c48d3084a1e59d21fe4ad892c6cd3313fa9208fad7d0c615f1a6d3d12af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-f, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T01:05:17.936 INFO:journalctl@ceph.mds.f.smithi157.stdout:May 09 01:05:17 smithi157 podman[71937]: 2024-05-09 01:05:17.661330884 +0000 UTC m=+3.250583824 container cleanup e5281c48d3084a1e59d21fe4ad892c6cd3313fa9208fad7d0c615f1a6d3d12af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-f, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T01:05:17.936 INFO:journalctl@ceph.mds.f.smithi157.stdout:May 09 01:05:17 smithi157 bash[71937]: ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-f 2024-05-09T01:05:17.936 INFO:journalctl@ceph.mds.f.smithi157.stdout:May 09 01:05:17 smithi157 podman[71952]: 2024-05-09 01:05:17.837380919 +0000 UTC m=+0.493268715 container remove e5281c48d3084a1e59d21fe4ad892c6cd3313fa9208fad7d0c615f1a6d3d12af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-f, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T01:05:18.061 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@mds.f.service' 2024-05-09T01:05:18.093 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T01:05:18.093 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-05-09T01:05:18.093 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-05-09T01:05:18.093 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@mds.i 2024-05-09T01:05:18.390 INFO:journalctl@ceph.mds.i.smithi157.stdout:May 09 01:05:18 smithi157 systemd[1]: Stopping Ceph mds.i for d85814c6-0d9b-11ef-bc97-c7b262605968... 2024-05-09T01:05:18.686 INFO:journalctl@ceph.mds.i.smithi157.stdout:May 09 01:05:18 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-i[64868]: 2024-05-09T01:05:18.389+0000 7f3da26cb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T01:05:18.686 INFO:journalctl@ceph.mds.i.smithi157.stdout:May 09 01:05:18 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-i[64868]: 2024-05-09T01:05:18.389+0000 7f3da26cb700 -1 mds.i *** got signal Terminated *** 2024-05-09T01:05:21.607 INFO:journalctl@ceph.mds.i.smithi157.stdout:May 09 01:05:21 smithi157 podman[72059]: 2024-05-09 01:05:21.340638688 +0000 UTC m=+3.030659326 container died a20c29b1f166f0fd8013c74f229f7f2ac920d0db7f033341c306d98d66aaed96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-i, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T01:05:21.936 INFO:journalctl@ceph.mds.i.smithi157.stdout:May 09 01:05:21 smithi157 podman[72059]: 2024-05-09 01:05:21.689288022 +0000 UTC m=+3.379308669 container cleanup a20c29b1f166f0fd8013c74f229f7f2ac920d0db7f033341c306d98d66aaed96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T01:05:21.936 INFO:journalctl@ceph.mds.i.smithi157.stdout:May 09 01:05:21 smithi157 bash[72059]: ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-i 2024-05-09T01:05:22.172 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@mds.i.service' 2024-05-09T01:05:22.204 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T01:05:22.204 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-05-09T01:05:22.204 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-05-09T01:05:22.204 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@mds.l 2024-05-09T01:05:22.686 INFO:journalctl@ceph.mds.l.smithi157.stdout:May 09 01:05:22 smithi157 systemd[1]: Stopping Ceph mds.l for d85814c6-0d9b-11ef-bc97-c7b262605968... 2024-05-09T01:05:22.686 INFO:journalctl@ceph.mds.l.smithi157.stdout:May 09 01:05:22 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-l[63935]: 2024-05-09T01:05:22.502+0000 7f247190c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T01:05:22.686 INFO:journalctl@ceph.mds.l.smithi157.stdout:May 09 01:05:22 smithi157 ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-l[63935]: 2024-05-09T01:05:22.502+0000 7f247190c700 -1 mds.l *** got signal Terminated *** 2024-05-09T01:05:25.654 INFO:journalctl@ceph.mds.l.smithi157.stdout:May 09 01:05:25 smithi157 podman[72178]: 2024-05-09 01:05:25.35404284 +0000 UTC m=+2.930517656 container died de9011eb50a6b782f16ba9be1cc57d4e2845967f4cdb9c138874b76483a27097 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T01:05:25.936 INFO:journalctl@ceph.mds.l.smithi157.stdout:May 09 01:05:25 smithi157 podman[72178]: 2024-05-09 01:05:25.737145679 +0000 UTC m=+3.313620490 container cleanup de9011eb50a6b782f16ba9be1cc57d4e2845967f4cdb9c138874b76483a27097 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-09T01:05:25.936 INFO:journalctl@ceph.mds.l.smithi157.stdout:May 09 01:05:25 smithi157 bash[72178]: ceph-d85814c6-0d9b-11ef-bc97-c7b262605968-mds-l 2024-05-09T01:05:26.040 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d85814c6-0d9b-11ef-bc97-c7b262605968@mds.l.service' 2024-05-09T01:05:26.071 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T01:05:26.071 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-05-09T01:05:26.071 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-09T01:05:26.275 INFO:teuthology.orchestra.run.smithi005.stdout:Deleting cluster with fsid: d85814c6-0d9b-11ef-bc97-c7b262605968 2024-05-09T01:05:28.866 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-09T01:05:29.065 INFO:teuthology.orchestra.run.smithi131.stdout:Deleting cluster with fsid: d85814c6-0d9b-11ef-bc97-c7b262605968 2024-05-09T01:05:31.537 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-09T01:05:31.732 INFO:teuthology.orchestra.run.smithi157.stdout:Deleting cluster with fsid: d85814c6-0d9b-11ef-bc97-c7b262605968 2024-05-09T01:05:34.137 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T01:05:34.171 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T01:05:34.202 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T01:05:34.228 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-09T01:05:34.231 DEBUG:teuthology.misc:Transferring archived files from smithi005:/var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7698888/remote/smithi005/crash 2024-05-09T01:05:34.232 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar c -f - -C /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/crash -- . 2024-05-09T01:05:34.273 INFO:teuthology.orchestra.run.smithi005.stderr:tar: /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-09T01:05:34.273 INFO:teuthology.orchestra.run.smithi005.stderr:tar: Error is not recoverable: exiting now 2024-05-09T01:05:34.275 DEBUG:teuthology.misc:Transferring archived files from smithi131:/var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7698888/remote/smithi131/crash 2024-05-09T01:05:34.276 DEBUG:teuthology.orchestra.run.smithi131:> sudo tar c -f - -C /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/crash -- . 2024-05-09T01:05:34.323 INFO:teuthology.orchestra.run.smithi131.stderr:tar: /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-09T01:05:34.323 INFO:teuthology.orchestra.run.smithi131.stderr:tar: Error is not recoverable: exiting now 2024-05-09T01:05:34.325 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7698888/remote/smithi157/crash 2024-05-09T01:05:34.326 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/crash -- . 2024-05-09T01:05:34.365 INFO:teuthology.orchestra.run.smithi157.stderr:tar: /var/lib/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-09T01:05:34.365 INFO:teuthology.orchestra.run.smithi157.stderr:tar: Error is not recoverable: exiting now 2024-05-09T01:05:34.366 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-09T01:05:34.367 DEBUG:teuthology.orchestra.run.smithi005:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-05-09T01:05:34.471 INFO:teuthology.orchestra.run.smithi005.stdout:2024-05-09T01:00:09.975539+0000 mds.i (mds.0) 51 : cluster [WRN] client.15198 does not advance its oldest_flush_tid (533730), 100000 completed flushes recorded in session 2024-05-09T01:05:34.472 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-05-09T01:05:34.472 DEBUG:teuthology.orchestra.run.smithi005:> sudo egrep '\[SEC\]' /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-05-09T01:05:34.510 DEBUG:teuthology.orchestra.run.smithi005:> sudo egrep '\[ERR\]' /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-05-09T01:05:34.588 DEBUG:teuthology.orchestra.run.smithi005:> sudo egrep '\[WRN\]' /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-05-09T01:05:34.666 INFO:teuthology.orchestra.run.smithi005.stdout:2024-05-09T01:00:09.975539+0000 mds.i (mds.0) 51 : cluster [WRN] client.15198 does not advance its oldest_flush_tid (533730), 100000 completed flushes recorded in session 2024-05-09T01:05:34.666 INFO:tasks.cephadm:Compressing logs... 2024-05-09T01:05:34.666 DEBUG:teuthology.orchestra.run.smithi005:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-09T01:05:34.709 DEBUG:teuthology.orchestra.run.smithi131:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-09T01:05:34.711 DEBUG:teuthology.orchestra.run.smithi157:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-09T01:05:34.745 INFO:teuthology.orchestra.run.smithi005.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-09T01:05:34.746 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-09T01:05:34.746 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph.log 2024-05-09T01:05:34.746 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph.audit.log 2024-05-09T01:05:34.749 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-mds.g.log 2024-05-09T01:05:34.750 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-mds.j.log 2024-05-09T01:05:34.750 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-osd.3.log 2024-05-09T01:05:34.750 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-osd.0.log 2024-05-09T01:05:34.750 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-mgr.x.log 2024-05-09T01:05:34.750 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-mon.a.log 2024-05-09T01:05:34.750 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-osd.6.log 2024-05-09T01:05:34.750 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph.cephadm.log 2024-05-09T01:05:34.750 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-mds.a.log 2024-05-09T01:05:34.750 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-volume.log 2024-05-09T01:05:34.750 INFO:teuthology.orchestra.run.smithi005.stderr: 87.0% -- replaced with /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph.log.gz 2024-05-09T01:05:34.750 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-mds.d.log 2024-05-09T01:05:34.750 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-osd.9.log 2024-05-09T01:05:34.751 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-mds.d.log: /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-osd.9.log: 94.5% -- replaced with /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-mds.j.log.gz 2024-05-09T01:05:34.752 INFO:teuthology.orchestra.run.smithi157.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-09T01:05:34.752 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-09T01:05:34.752 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph.log 2024-05-09T01:05:34.752 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-osd.2.log 2024-05-09T01:05:34.752 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-mds.f.log 2024-05-09T01:05:34.752 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph.audit.log 2024-05-09T01:05:34.758 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-osd.5.log 2024-05-09T01:05:34.758 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-mds.c.log 2024-05-09T01:05:34.758 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-mon.c.log 2024-05-09T01:05:34.758 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-mgr.z.log 2024-05-09T01:05:34.758 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-mds.i.log 2024-05-09T01:05:34.758 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph.cephadm.log 2024-05-09T01:05:34.758 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-osd.11.log 2024-05-09T01:05:34.758 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-osd.8.log 2024-05-09T01:05:34.758 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-volume.log 2024-05-09T01:05:34.759 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-mds.l.log 2024-05-09T01:05:34.760 INFO:teuthology.orchestra.run.smithi131.stderr:find: ‘/var/log/rbd-target-api’: No such file or directorygzip 2024-05-09T01:05:34.760 INFO:teuthology.orchestra.run.smithi131.stderr: -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-09T01:05:34.760 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph.log 2024-05-09T01:05:34.760 INFO:teuthology.orchestra.run.smithi131.stderr:gzip/var/log/ceph/cephadm.log: -5 --verbose -- /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph.audit.log 2024-05-09T01:05:34.760 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-osd.7.log 2024-05-09T01:05:34.760 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph.log: /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-mds.b.log 2024-05-09T01:05:34.760 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-osd.4.log 2024-05-09T01:05:34.760 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-osd.1.log 2024-05-09T01:05:34.760 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-mgr.y.log 2024-05-09T01:05:34.760 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-mds.h.log 2024-05-09T01:05:34.760 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-osd.10.log 2024-05-09T01:05:34.760 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph.cephadm.log 2024-05-09T01:05:34.760 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-mds.e.log 2024-05-09T01:05:34.761 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-volume.log 2024-05-09T01:05:34.761 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-mon.b.log 2024-05-09T01:05:34.761 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-mds.k.log 2024-05-09T01:05:34.835 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-mon.b.log: /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-mds.k.log: 93.1% -- replaced with /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-mgr.y.log.gz 2024-05-09T01:05:34.845 INFO:teuthology.orchestra.run.smithi131.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-09T01:05:34.860 INFO:teuthology.orchestra.run.smithi005.stderr: 94.4% -- replaced with /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-mds.d.log.gz 2024-05-09T01:05:34.884 INFO:teuthology.orchestra.run.smithi131.stderr: 82.6% -- replaced with /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph.cephadm.log.gz 2024-05-09T01:05:34.895 INFO:teuthology.orchestra.run.smithi131.stderr: 90.5% -- replaced with /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph.audit.log.gz 2024-05-09T01:05:34.938 INFO:teuthology.orchestra.run.smithi005.stderr: 94.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-09T01:05:34.955 INFO:teuthology.orchestra.run.smithi005.stderr: 91.1% -- replaced with /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-volume.log.gz 2024-05-09T01:05:34.992 INFO:teuthology.orchestra.run.smithi131.stderr: 91.0% -- replaced with /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-volume.log.gz 2024-05-09T01:05:35.008 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-volume.log: /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-mds.l.log: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-09T01:05:35.019 INFO:teuthology.orchestra.run.smithi157.stderr: 93.2% -- replaced with /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-mgr.z.log.gz 2024-05-09T01:05:35.034 INFO:teuthology.orchestra.run.smithi005.stderr: 82.9% -- replaced with /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph.cephadm.log.gz 2024-05-09T01:05:35.086 INFO:teuthology.orchestra.run.smithi157.stderr: 82.5% -- replaced with /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph.cephadm.log.gz 2024-05-09T01:05:35.097 INFO:teuthology.orchestra.run.smithi157.stderr: 90.5% -- replaced with /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph.audit.log.gz 2024-05-09T01:05:35.141 INFO:teuthology.orchestra.run.smithi005.stderr: 90.3% -- replaced with /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph.audit.log.gz 2024-05-09T01:05:35.342 INFO:teuthology.orchestra.run.smithi157.stderr: 91.0% -- replaced with /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-volume.log.gz 2024-05-09T01:05:35.767 INFO:teuthology.orchestra.run.smithi131.stderr: 93.2% -- replaced with /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-mds.h.log.gz 2024-05-09T01:05:35.979 INFO:teuthology.orchestra.run.smithi157.stderr: 93.0% -- replaced with /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-mds.f.log.gz 2024-05-09T01:05:36.000 INFO:teuthology.orchestra.run.smithi131.stderr: 94.5% -- replaced with /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-mds.k.log.gz 2024-05-09T01:05:36.025 INFO:teuthology.orchestra.run.smithi131.stderr: 94.5% -- replaced with /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-mds.e.log.gz 2024-05-09T01:05:36.027 INFO:teuthology.orchestra.run.smithi005.stderr: 94.5% -- replaced with /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-mds.a.log.gz 2024-05-09T01:05:36.118 INFO:teuthology.orchestra.run.smithi131.stderr: 92.7% -- replaced with /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-mds.b.log.gz 2024-05-09T01:05:36.127 INFO:teuthology.orchestra.run.smithi005.stderr: 94.5% -- replaced with /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-mds.g.log.gz 2024-05-09T01:05:36.151 INFO:teuthology.orchestra.run.smithi131.stderr: 87.0% -- replaced with /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph.log.gz 2024-05-09T01:05:36.302 INFO:teuthology.orchestra.run.smithi157.stderr: 92.8% -- replaced with /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-mds.c.log.gz 2024-05-09T01:05:36.606 INFO:teuthology.orchestra.run.smithi157.stderr: 87.1% -- replaced with /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph.log.gz 2024-05-09T01:05:37.544 INFO:teuthology.orchestra.run.smithi005.stderr: 90.3% -- replaced with /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-mgr.x.log.gz 2024-05-09T01:05:38.268 INFO:teuthology.orchestra.run.smithi131.stderr: 91.8% -- replaced with /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-mon.b.log.gz 2024-05-09T01:05:44.572 INFO:teuthology.orchestra.run.smithi005.stderr: 90.1% -- replaced with /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-mon.a.log.gz 2024-05-09T01:05:46.806 INFO:teuthology.orchestra.run.smithi157.stderr: 91.8% -- replaced with /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-mon.c.log.gz 2024-05-09T01:06:57.243 INFO:teuthology.orchestra.run.smithi131.stderr: 93.3% -- replaced with /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-osd.7.log.gz 2024-05-09T01:07:11.018 INFO:teuthology.orchestra.run.smithi005.stderr: 93.3% -- replaced with /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-osd.9.log.gz 2024-05-09T01:07:32.125 INFO:teuthology.orchestra.run.smithi131.stderr: 93.3% -- replaced with /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-osd.1.log.gz 2024-05-09T01:10:06.189 INFO:teuthology.orchestra.run.smithi131.stderr: 93.4% -- replaced with /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-osd.4.log.gz 2024-05-09T01:10:08.929 INFO:teuthology.orchestra.run.smithi131.stderr: 93.3% -- replaced with /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-osd.10.log.gz 2024-05-09T01:10:08.931 INFO:teuthology.orchestra.run.smithi131.stderr: 2024-05-09T01:10:08.931 INFO:teuthology.orchestra.run.smithi131.stderr:real 4m34.205s 2024-05-09T01:10:08.931 INFO:teuthology.orchestra.run.smithi131.stderr:user 5m2.802s 2024-05-09T01:10:08.931 INFO:teuthology.orchestra.run.smithi131.stderr:sys 0m12.944s 2024-05-09T01:12:01.133 INFO:teuthology.orchestra.run.smithi005.stderr: 93.2% -- replaced with /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-osd.0.log.gz 2024-05-09T01:12:46.216 INFO:teuthology.orchestra.run.smithi005.stderr: 93.3% -- replaced with /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-osd.6.log.gz 2024-05-09T01:13:04.463 INFO:teuthology.orchestra.run.smithi005.stderr: 93.2% -- replaced with /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-osd.3.log.gz 2024-05-09T01:13:04.465 INFO:teuthology.orchestra.run.smithi005.stderr: 2024-05-09T01:13:04.465 INFO:teuthology.orchestra.run.smithi005.stderr:real 7m29.742s 2024-05-09T01:13:04.465 INFO:teuthology.orchestra.run.smithi005.stderr:user 5m1.419s 2024-05-09T01:13:04.465 INFO:teuthology.orchestra.run.smithi005.stderr:sys 0m12.631s 2024-05-09T01:17:16.496 INFO:teuthology.orchestra.run.smithi157.stderr: 97.4% -- replaced with /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-mds.l.log.gz 2024-05-09T01:22:09.108 INFO:teuthology.orchestra.run.smithi157.stderr: 93.3% -- replaced with /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-osd.8.log.gz 2024-05-09T01:22:28.090 INFO:teuthology.orchestra.run.smithi157.stderr: 93.3% -- replaced with /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-osd.11.log.gz 2024-05-09T01:24:11.499 INFO:teuthology.orchestra.run.smithi157.stderr: 93.4% -- replaced with /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-osd.2.log.gz 2024-05-09T01:25:02.319 INFO:teuthology.orchestra.run.smithi157.stderr: 93.3% -- replaced with /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-osd.5.log.gz 2024-05-09T01:28:32.013 INFO:teuthology.orchestra.run.smithi157.stderr: 93.9% -- replaced with /var/log/ceph/d85814c6-0d9b-11ef-bc97-c7b262605968/ceph-mds.i.log.gz 2024-05-09T01:28:32.015 INFO:teuthology.orchestra.run.smithi157.stderr: 2024-05-09T01:28:32.015 INFO:teuthology.orchestra.run.smithi157.stderr:real 22m57.292s 2024-05-09T01:28:32.015 INFO:teuthology.orchestra.run.smithi157.stderr:user 9m15.786s 2024-05-09T01:28:32.015 INFO:teuthology.orchestra.run.smithi157.stderr:sys 0m28.307s 2024-05-09T01:28:32.015 INFO:tasks.cephadm:Archiving logs... 2024-05-09T01:28:32.016 DEBUG:teuthology.misc:Transferring archived files from smithi005:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7698888/remote/smithi005/log 2024-05-09T01:28:32.018 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-09T01:29:17.995 DEBUG:teuthology.misc:Transferring archived files from smithi131:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7698888/remote/smithi131/log 2024-05-09T01:29:17.996 DEBUG:teuthology.orchestra.run.smithi131:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-09T01:30:07.579 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7698888/remote/smithi157/log 2024-05-09T01:30:07.580 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-09T01:31:32.860 INFO:tasks.cephadm:Removing cluster... 2024-05-09T01:31:32.860 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 --force 2024-05-09T01:31:33.064 INFO:teuthology.orchestra.run.smithi005.stdout:Deleting cluster with fsid: d85814c6-0d9b-11ef-bc97-c7b262605968 2024-05-09T01:31:33.636 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 --force 2024-05-09T01:31:33.843 INFO:teuthology.orchestra.run.smithi131.stdout:Deleting cluster with fsid: d85814c6-0d9b-11ef-bc97-c7b262605968 2024-05-09T01:31:34.471 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d85814c6-0d9b-11ef-bc97-c7b262605968 --force 2024-05-09T01:31:34.966 INFO:teuthology.orchestra.run.smithi157.stdout:Deleting cluster with fsid: d85814c6-0d9b-11ef-bc97-c7b262605968 2024-05-09T01:31:35.752 INFO:tasks.cephadm:Removing cephadm ... 2024-05-09T01:31:35.753 DEBUG:teuthology.orchestra.run.smithi005:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-09T01:31:35.774 DEBUG:teuthology.orchestra.run.smithi131:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-09T01:31:35.805 DEBUG:teuthology.orchestra.run.smithi157:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-09T01:31:35.821 INFO:tasks.cephadm:Teardown complete 2024-05-09T01:31:35.821 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-09T01:31:35.832 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-09T01:31:35.833 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-09T01:31:35.836 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-09T01:31:35.848 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-09T01:31:35.999 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-09T01:31:36.060 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-09T01:31:36.061 DEBUG:teuthology.orchestra.run.smithi005:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T01:31:36.065 DEBUG:teuthology.orchestra.run.smithi131:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T01:31:36.072 DEBUG:teuthology.orchestra.run.smithi157:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T01:31:36.081 INFO:teuthology.orchestra.run.smithi005.stderr:bash: line 1: ntpq: command not found 2024-05-09T01:31:36.081 INFO:teuthology.orchestra.run.smithi131.stderr:bash: line 1: ntpq: command not found 2024-05-09T01:31:36.086 INFO:teuthology.orchestra.run.smithi005.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T01:31:36.086 INFO:teuthology.orchestra.run.smithi005.stdout:=============================================================================== 2024-05-09T01:31:36.086 INFO:teuthology.orchestra.run.smithi005.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 33 +1046us[+1170us] +/- 61ms 2024-05-09T01:31:36.086 INFO:teuthology.orchestra.run.smithi005.stdout:^* hv02.front.sepia.ceph.com 3 6 377 31 -964us[ -840us] +/- 64ms 2024-05-09T01:31:36.087 INFO:teuthology.orchestra.run.smithi005.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 35 +29us[ +153us] +/- 65ms 2024-05-09T01:31:36.087 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T01:31:36.087 INFO:teuthology.orchestra.run.smithi157.stderr:bash: line 1: ntpq: command not found 2024-05-09T01:31:36.101 INFO:teuthology.orchestra.run.smithi131.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T01:31:36.101 INFO:teuthology.orchestra.run.smithi131.stdout:=============================================================================== 2024-05-09T01:31:36.102 INFO:teuthology.orchestra.run.smithi131.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 32 +1194us[+1194us] +/- 61ms 2024-05-09T01:31:36.102 INFO:teuthology.orchestra.run.smithi131.stdout:^* hv02.front.sepia.ceph.com 3 6 377 34 -835us[ -769us] +/- 64ms 2024-05-09T01:31:36.102 INFO:teuthology.orchestra.run.smithi131.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 32 +133us[ +133us] +/- 65ms 2024-05-09T01:31:36.102 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T01:31:36.117 INFO:teuthology.orchestra.run.smithi157.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T01:31:36.117 INFO:teuthology.orchestra.run.smithi157.stdout:=============================================================================== 2024-05-09T01:31:36.117 INFO:teuthology.orchestra.run.smithi157.stdout:^* hv01.front.sepia.ceph.com 3 6 377 30 +1096us[+1174us] +/- 61ms 2024-05-09T01:31:36.118 INFO:teuthology.orchestra.run.smithi157.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 28 -947us[ -947us] +/- 64ms 2024-05-09T01:31:36.118 INFO:teuthology.orchestra.run.smithi157.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 31 +18us[ +96us] +/- 65ms 2024-05-09T01:31:36.118 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T01:31:36.119 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-09T01:31:36.133 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-09T01:31:36.134 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-09T01:31:36.158 DEBUG:teuthology.orchestra.run.smithi005:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-09T01:31:36.344 DEBUG:teuthology.orchestra.run.smithi131:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-09T01:31:36.444 DEBUG:teuthology.orchestra.run.smithi157:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-09T01:31:36.522 DEBUG:teuthology.orchestra.run.smithi005:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T01:31:36.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T01:31:36.557 DEBUG:teuthology.orchestra.run.smithi131:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T01:31:36.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T01:31:36.586 DEBUG:teuthology.orchestra.run.smithi157:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T01:31:36.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T01:31:36.615 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-09T01:31:36.626 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-09T01:31:36.657 INFO:teuthology.task.internal:Duration was 3852.571702 seconds 2024-05-09T01:31:36.658 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-09T01:31:36.708 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-09T01:31:36.708 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-09T01:31:36.711 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-09T01:31:36.715 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-09T01:31:36.771 INFO:teuthology.orchestra.run.smithi131.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T01:31:36.778 INFO:teuthology.orchestra.run.smithi005.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T01:31:36.805 INFO:teuthology.orchestra.run.smithi157.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T01:31:37.232 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-09T01:31:37.233 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi005.front.sepia.ceph.com 2024-05-09T01:31:37.233 DEBUG:teuthology.orchestra.run.smithi005:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-09T01:31:37.260 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi131.front.sepia.ceph.com 2024-05-09T01:31:37.261 DEBUG:teuthology.orchestra.run.smithi131:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-09T01:31:37.290 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi157.front.sepia.ceph.com 2024-05-09T01:31:37.291 DEBUG:teuthology.orchestra.run.smithi157:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-09T01:31:37.319 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-09T01:31:37.319 DEBUG:teuthology.orchestra.run.smithi005:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-09T01:31:37.322 DEBUG:teuthology.orchestra.run.smithi131:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-09T01:31:37.333 DEBUG:teuthology.orchestra.run.smithi157:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-09T01:31:37.384 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-09T01:31:37.385 DEBUG:teuthology.orchestra.run.smithi005:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-09T01:31:37.394 DEBUG:teuthology.orchestra.run.smithi131:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-09T01:31:37.400 DEBUG:teuthology.orchestra.run.smithi157:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-09T01:31:38.438 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-09T01:31:38.450 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-09T01:31:38.450 DEBUG:teuthology.orchestra.run.smithi005:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-09T01:31:38.482 DEBUG:teuthology.orchestra.run.smithi131:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-09T01:31:38.526 DEBUG:teuthology.orchestra.run.smithi157:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-09T01:31:38.555 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-09T01:31:38.567 DEBUG:teuthology.orchestra.run.smithi005:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-09T01:31:38.569 DEBUG:teuthology.orchestra.run.smithi131:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-09T01:31:38.571 DEBUG:teuthology.orchestra.run.smithi157:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-09T01:31:38.598 INFO:teuthology.orchestra.run.smithi005.stdout:kernel.core_pattern = core 2024-05-09T01:31:38.598 INFO:teuthology.orchestra.run.smithi131.stdout:kernel.core_pattern = core 2024-05-09T01:31:38.645 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern = core 2024-05-09T01:31:38.662 DEBUG:teuthology.orchestra.run.smithi005:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-09T01:31:38.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T01:31:38.677 DEBUG:teuthology.orchestra.run.smithi131:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-09T01:31:38.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T01:31:38.692 DEBUG:teuthology.orchestra.run.smithi157:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-09T01:31:38.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T01:31:38.707 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-09T01:31:38.719 INFO:teuthology.task.internal:Transferring archived files... 2024-05-09T01:31:38.720 DEBUG:teuthology.misc:Transferring archived files from smithi005:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7698888/remote/smithi005 2024-05-09T01:31:38.721 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-09T01:31:38.784 DEBUG:teuthology.misc:Transferring archived files from smithi131:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7698888/remote/smithi131 2024-05-09T01:31:38.784 DEBUG:teuthology.orchestra.run.smithi131:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-09T01:31:38.822 DEBUG:teuthology.misc:Transferring archived files from smithi157:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7698888/remote/smithi157 2024-05-09T01:31:38.823 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-09T01:31:38.859 INFO:teuthology.task.internal:Removing archive directory... 2024-05-09T01:31:38.860 DEBUG:teuthology.orchestra.run.smithi005:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-09T01:31:38.863 DEBUG:teuthology.orchestra.run.smithi131:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-09T01:31:38.865 DEBUG:teuthology.orchestra.run.smithi157:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-09T01:31:38.909 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-09T01:31:38.923 INFO:teuthology.task.internal:Not uploading archives. 2024-05-09T01:31:38.923 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-09T01:31:38.971 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-09T01:31:38.971 DEBUG:teuthology.orchestra.run.smithi005:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-09T01:31:38.974 DEBUG:teuthology.orchestra.run.smithi131:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-09T01:31:38.977 DEBUG:teuthology.orchestra.run.smithi157:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-09T01:31:38.987 INFO:teuthology.orchestra.run.smithi005.stdout: 265710 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 9 01:31 /home/ubuntu/cephtest 2024-05-09T01:31:38.990 INFO:teuthology.orchestra.run.smithi131.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 9 01:31 /home/ubuntu/cephtest 2024-05-09T01:31:38.991 INFO:teuthology.orchestra.run.smithi157.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 9 01:31 /home/ubuntu/cephtest 2024-05-09T01:31:38.993 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-09T01:31:39.018 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-09T01:31:39.117 INFO:teuthology.nuke:Checking targets against current locks 2024-05-09T01:31:39.144 DEBUG:teuthology.nuke:shortname: smithi005 2024-05-09T01:31:39.144 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-09T01:31:39.167 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi005.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7698888', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-09 00:18:33.124130', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBA4Hv9slO+MjNDhUYt2EwiomGMBiJHmDpPun2+FpDW89jU+j08x3tiRf6Z0AEh98LPxX9n24S3XvqJC/zlX33MM='} 2024-05-09T01:31:39.171 DEBUG:teuthology.nuke:shortname: smithi131 2024-05-09T01:31:39.172 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-09T01:31:39.193 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi131.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7698888', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-09 00:18:33.121001', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOsjKm3PZ9Klq0XIIPYVde+hyKwTiazJE2JR/2Bmm8ryk58ZbvZ84cp3CXHlU8uctTCGyuGGVny5zR80Ba+8G+s='} 2024-05-09T01:31:39.198 DEBUG:teuthology.nuke:shortname: smithi157 2024-05-09T01:31:39.198 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-09T01:31:39.219 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi157.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7698888', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-09 00:18:33.122676', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBemtDUBbgz4nYjiOeKNVoETJxKL8NwQeov5VR38uFbJ0YLisq23SxsTP6AvE4jQHTQ19FL6UmoAUn2dDPhi1rA='} 2024-05-09T01:31:39.230 INFO:teuthology.orchestra.console.smithi005:Power off 2024-05-09T01:31:39.230 DEBUG:teuthology.orchestra.console.smithi005:pexpect command: ipmitool -H smithi005.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-09T01:31:39.254 DEBUG:teuthology.orchestra.console.smithi005:power off output: Chassis Power Control: Down/Off 2024-05-09T01:31:39.254 DEBUG:teuthology.orchestra.console.smithi005:pexpect command: ipmitool -H smithi005.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-09T01:31:39.273 INFO:teuthology.orchestra.console.smithi131:Power off 2024-05-09T01:31:39.273 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-09T01:31:39.283 DEBUG:teuthology.orchestra.console.smithi005:check power output: Chassis Power is on 2024-05-09T01:31:39.296 DEBUG:teuthology.orchestra.console.smithi131:power off output: Chassis Power Control: Down/Off 2024-05-09T01:31:39.296 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-09T01:31:39.311 INFO:teuthology.orchestra.console.smithi157:Power off 2024-05-09T01:31:39.311 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-09T01:31:39.321 DEBUG:teuthology.orchestra.console.smithi131:check power output: Chassis Power is on 2024-05-09T01:31:39.333 DEBUG:teuthology.orchestra.console.smithi157:power off output: Chassis Power Control: Down/Off 2024-05-09T01:31:39.333 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-09T01:31:39.357 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is on 2024-05-09T01:31:43.285 DEBUG:teuthology.orchestra.console.smithi005:pexpect command: ipmitool -H smithi005.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-09T01:31:43.322 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-09T01:31:43.358 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-09T01:31:43.396 DEBUG:teuthology.orchestra.console.smithi005:check power output: Chassis Power is on 2024-05-09T01:31:43.432 DEBUG:teuthology.orchestra.console.smithi131:check power output: Chassis Power is on 2024-05-09T01:31:43.468 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is on 2024-05-09T01:31:47.398 DEBUG:teuthology.orchestra.console.smithi005:pexpect command: ipmitool -H smithi005.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-09T01:31:47.432 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-09T01:31:47.468 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-09T01:31:47.508 DEBUG:teuthology.orchestra.console.smithi005:check power output: Chassis Power is off 2024-05-09T01:31:47.542 DEBUG:teuthology.orchestra.console.smithi131:check power output: Chassis Power is off 2024-05-09T01:31:47.579 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is off 2024-05-09T01:31:47.609 INFO:teuthology.orchestra.console.smithi005:Power off completed 2024-05-09T01:31:47.643 INFO:teuthology.orchestra.console.smithi131:Power off completed 2024-05-09T01:31:47.680 INFO:teuthology.orchestra.console.smithi157:Power off completed 2024-05-09T01:31:47.785 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/suites/iogen}} duration: 3852.5717022418976 failure_reason: '"2024-05-09T01:00:09.975539+0000 mds.i (mds.0) 51 : cluster [WRN] client.15198 does not advance its oldest_flush_tid (533730), 100000 completed flushes recorded in session" in cluster log' flavor: default owner: scheduled_pdonnell@teuthology success: false 2024-05-09T01:31:47.786 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-09T01:31:47.874 INFO:teuthology.run:FAIL